2020-11-28T21:34:40.084 INFO:root:teuthology version: 1.0.0-e0aaa9bf 2020-11-28T21:34:40.085 DEBUG:teuthology.run:Teuthology command: teuthology --block --archive /home/teuthworker/archive/teuthology-2020-11-25_04:10:04-fs-octopus-distro-basic-smithi/5655905 --owner scheduled_teuthology@teuthology --name teuthology-2020-11-25_04:10:04-fs-octopus-distro-basic-smithi --description fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests} --verbose --lock -- /tmp/teuthology-worker.bj1_r77y.tmp 2020-11-28T21:34:40.126 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-11-28T21:34:40.191 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2020-11-25_04:10:04-fs-octopus-distro-basic-smithi/5655905 branch: octopus description: fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests} email: ceph-qa@ceph.io first_in_suite: false job_id: '5655905' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: teuthology-2020-11-25_04:10:04-fs-octopus-distro-basic-smithi nuke-on-error: true openstack: - volumes: count: 4 size: 30 - machine: disk: 200 os_type: ubuntu os_version: '18.04' overrides: admin_socket: branch: octopus ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 15m rados osd op timeout: 15m mds: debug mds: 20 debug ms: 1 mds bal frag: true mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 15m rados osd op timeout: 15m mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug filestore: 20 debug journal: 20 debug ms: 20 debug osd: 25 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - evicting unresponsive client - POOL_APP_NOT_ENABLED - has not responded to cap revoke by MDS for over - MDS_CLIENT_LATE_RELEASE - responding to mclientcaps sha1: a83a3572a957356977342f461361c1cd3ec06dbd ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: sha1: a83a3572a957356977342f461361c1cd3ec06dbd thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: octopus sha1: a83a3572a957356977342f461361c1cd3ec06dbd owner: scheduled_teuthology@teuthology priority: 1000 repo: git://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - - mon.b - mon.c - mgr.x - mds.b - osd.4 - osd.5 - osd.6 - osd.7 - - client.0 - - client.1 sha1: a83a3572a957356977342f461361c1cd3ec06dbd sleep_before_teardown: 0 suite: fs suite_branch: octopus suite_path: /home/teuthworker/src/git.ceph.com_ceph_octopus/qa suite_relpath: qa suite_repo: git://git.ceph.com/ceph.git suite_sha1: a83a3572a957356977342f461361c1cd3ec06dbd tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell rpm: - python3-cephfs extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_misc teuthology_branch: master timestamp: 2020-11-25_04:10:04 tube: smithi user: teuthology verbose: true worker_log: /home/teuthworker/archive/worker_logs/worker.smithi.25733 2020-11-28T21:34:40.191 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_octopus/qa; will attempt to use it 2020-11-28T21:34:40.192 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks 2020-11-28T21:34:40.192 INFO:teuthology.run_tasks:Running task internal.check_packages... 2020-11-28T21:34:40.194 INFO:teuthology.task.internal:Checking packages... 2020-11-28T21:34:40.212 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'basic' and ceph hash 'a83a3572a957356977342f461361c1cd3ec06dbd' 2020-11-28T21:34:40.213 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2020-11-28T21:34:40.213 INFO:teuthology.packaging:ref: None 2020-11-28T21:34:40.213 INFO:teuthology.packaging:tag: None 2020-11-28T21:34:40.214 INFO:teuthology.packaging:branch: octopus 2020-11-28T21:34:40.214 INFO:teuthology.packaging:sha1: a83a3572a957356977342f461361c1cd3ec06dbd 2020-11-28T21:34:40.214 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref=octopus 2020-11-28T21:34:40.489 INFO:teuthology.task.internal:Found packages for ceph version 15.2.6-607-g3b1861ac-1bionic 2020-11-28T21:34:40.489 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2020-11-28T21:34:40.500 INFO:teuthology.task.internal:no buildpackages task found 2020-11-28T21:34:40.501 INFO:teuthology.run_tasks:Running task internal.lock_machines... 2020-11-28T21:34:40.548 INFO:teuthology.task.internal.lock_machines:Locking machines... 2020-11-28T21:34:40.549 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-11-28T21:34:40.602 DEBUG:teuthology.lock.ops:lock_many request: {'locked_by': 'scheduled_teuthology@teuthology', 'count': 4, 'machine_type': 'smithi', 'description': '/home/teuthworker/archive/teuthology-2020-11-25_04:10:04-fs-octopus-distro-basic-smithi/5655905'} 2020-11-28T21:34:40.625 DEBUG:teuthology.lock.ops:locked ubuntu@smithi162.front.sepia.ceph.com, ubuntu@smithi118.front.sepia.ceph.com, ubuntu@smithi192.front.sepia.ceph.com, ubuntu@smithi096.front.sepia.ceph.com 2020-11-28T21:34:40.738 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f83b93dd950>, signals=[15]) 2020-11-28T21:34:40.970 INFO:teuthology.provision.fog.smithi096:Scheduling deploy of ubuntu 18.04 2020-11-28T21:34:40.973 INFO:teuthology.provision.fog.smithi162:Scheduling deploy of ubuntu 18.04 2020-11-28T21:34:40.975 INFO:teuthology.provision.fog.smithi118:Scheduling deploy of ubuntu 18.04 2020-11-28T21:34:40.997 INFO:teuthology.provision.fog.smithi192:Scheduling deploy of ubuntu 18.04 2020-11-28T21:34:41.462 INFO:teuthology.orchestra.console:Power off smithi118 2020-11-28T21:34:41.463 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-11-28T21:34:41.485 INFO:teuthology.orchestra.console:Power off smithi096 2020-11-28T21:34:41.486 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-11-28T21:34:41.504 INFO:teuthology.orchestra.console:Power off smithi162 2020-11-28T21:34:41.504 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-11-28T21:34:41.515 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-28T21:34:41.526 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-28T21:34:41.538 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-28T21:34:41.556 INFO:teuthology.orchestra.console:Power off smithi192 2020-11-28T21:34:41.556 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-11-28T21:34:41.579 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-28T21:34:45.566 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-28T21:34:45.575 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-28T21:34:45.581 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-28T21:34:45.608 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-28T21:34:49.679 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-28T21:34:49.689 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-28T21:34:49.698 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-28T21:34:49.719 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-28T21:34:49.900 INFO:teuthology.orchestra.console:Power off for smithi118 completed 2020-11-28T21:34:49.909 INFO:teuthology.orchestra.console:Power off for smithi096 completed 2020-11-28T21:34:50.002 INFO:teuthology.orchestra.console:Power on smithi118 2020-11-28T21:34:50.002 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-11-28T21:34:50.013 INFO:teuthology.orchestra.console:Power on smithi096 2020-11-28T21:34:50.013 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-11-28T21:34:50.036 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-28T21:34:50.050 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-28T21:34:53.792 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-28T21:34:53.829 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-28T21:34:54.003 INFO:teuthology.orchestra.console:Power off for smithi162 completed 2020-11-28T21:34:54.037 INFO:teuthology.orchestra.console:Power off for smithi192 completed 2020-11-28T21:34:54.076 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-28T21:34:54.084 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-28T21:34:54.105 INFO:teuthology.orchestra.console:Power on smithi162 2020-11-28T21:34:54.106 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-11-28T21:34:54.125 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-28T21:34:54.139 INFO:teuthology.orchestra.console:Power on smithi192 2020-11-28T21:34:54.139 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-11-28T21:34:54.160 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-28T21:34:58.150 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-28T21:34:58.182 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-28T21:34:58.191 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-28T21:34:58.200 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-28T21:34:58.360 INFO:teuthology.orchestra.console:Power on for smithi162 completed 2020-11-28T21:34:58.409 INFO:teuthology.orchestra.console:Power on for smithi118 completed 2020-11-28T21:34:58.457 INFO:teuthology.orchestra.console:Power on for smithi096 completed 2020-11-28T21:34:58.461 INFO:teuthology.provision.fog.smithi162:Waiting for deploy to finish 2020-11-28T21:34:58.510 INFO:teuthology.provision.fog.smithi118:Waiting for deploy to finish 2020-11-28T21:34:58.558 INFO:teuthology.provision.fog.smithi096:Waiting for deploy to finish 2020-11-28T21:35:02.294 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-28T21:35:02.504 INFO:teuthology.orchestra.console:Power on for smithi192 completed 2020-11-28T21:35:02.605 INFO:teuthology.provision.fog.smithi192:Waiting for deploy to finish 2020-11-28T21:37:45.202 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:37:45.302 ERROR:teuthology.orchestra.connection:Error connecting to smithi096.front.sepia.ceph.com Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/connection.py", line 105, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/client.py", line 446, in connect passphrase, File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/client.py", line 764, in _auth raise saved_exception File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/client.py", line 740, in _auth self._transport.auth_publickey(username, key) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 1580, in auth_publickey return self.auth_handler.wait_for_response(my_event) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/auth_handler.py", line 250, in wait_for_response raise e paramiko.ssh_exception.AuthenticationException: Authentication failed. 2020-11-28T21:38:00.178 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:38:04.132 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:38:09.231 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:38:13.199 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:38:15.381 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:38:18.291 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:38:22.255 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:38:24.463 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:38:27.343 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:38:31.311 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:38:33.519 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:38:36.403 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:38:40.367 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:38:42.574 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:38:45.456 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:38:49.423 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:38:51.631 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:38:52.307 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:38:54.511 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:38:58.479 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:39:00.687 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:39:03.567 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:39:09.747 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:39:12.623 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:39:18.799 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:39:23.845 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-11-28T21:39:29.849 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:39:29.860 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-11-28T21:39:35.862 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:39:35.987 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:39:36.006 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-11-28T21:39:36.049 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-11-28T21:39:42.008 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:39:42.018 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-11-28T21:39:42.051 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:39:42.062 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-11-28T21:39:44.186 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-11-28T21:39:48.021 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:39:48.031 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-11-28T21:39:48.064 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:39:48.075 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-11-28T21:39:50.187 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:39:50.199 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-11-28T21:39:54.033 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:39:54.044 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-11-28T21:39:54.079 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:39:54.160 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-11-28T21:39:54.162 INFO:teuthology.orchestra.run.smithi096:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-11-28T21:39:54.545 INFO:teuthology.provision.fog.smithi096:Node is ready 2020-11-28T21:39:54.546 INFO:teuthology.orchestra.run.smithi096:> hostname 2020-11-28T21:39:54.591 INFO:teuthology.orchestra.run.smithi096.stdout:smithi096.front.sepia.ceph.com 2020-11-28T21:39:54.592 INFO:teuthology.orchestra.run.smithi096:> grep smithi096.front.sepia.ceph.com /etc/hosts 2020-11-28T21:39:54.641 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96 smithi096.front.sepia.ceph.com smithi096 2020-11-28T21:39:54.642 INFO:teuthology.orchestra.run.smithi096:> sudo hostname smithi096 2020-11-28T21:39:54.792 INFO:teuthology.orchestra.run.smithi096:> sudo sed -i -e 's/smithi096.front.sepia.ceph.com/smithi096/g' /etc/hosts 2020-11-28T21:39:54.800 INFO:teuthology.orchestra.run.smithi096:> sudo sed -i -e 's/172.21.15.96/172.21.15.96/g' /etc/hosts 2020-11-28T21:39:54.858 INFO:teuthology.orchestra.run.smithi096:> sudo sed -i -e 's/smithi096.front.sepia.ceph.com/smithi096/g' /etc/hostname 2020-11-28T21:39:54.912 INFO:teuthology.orchestra.run.smithi096:> sudo hostname smithi096 2020-11-28T21:39:54.969 INFO:teuthology.provision.fog.smithi096:Deploy complete! 2020-11-28T21:39:54.970 DEBUG:teuthology.parallel:result is None 2020-11-28T21:39:56.201 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:39:56.212 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-11-28T21:40:00.053 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:40:00.064 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-11-28T21:40:02.215 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:40:06.066 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:40:06.146 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-11-28T21:40:06.147 INFO:teuthology.orchestra.run.smithi192:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-11-28T21:40:06.604 INFO:teuthology.provision.fog.smithi192:Node is ready 2020-11-28T21:40:06.605 INFO:teuthology.orchestra.run.smithi192:> hostname 2020-11-28T21:40:06.650 INFO:teuthology.orchestra.run.smithi192.stdout:smithi192.front.sepia.ceph.com 2020-11-28T21:40:06.651 INFO:teuthology.orchestra.run.smithi192:> grep smithi192.front.sepia.ceph.com /etc/hosts 2020-11-28T21:40:06.699 INFO:teuthology.orchestra.run.smithi192.stdout:172.21.15.192 smithi192.front.sepia.ceph.com smithi192 2020-11-28T21:40:06.699 INFO:teuthology.orchestra.run.smithi192:> sudo hostname smithi192 2020-11-28T21:40:06.784 INFO:teuthology.orchestra.run.smithi192:> sudo sed -i -e 's/smithi192.front.sepia.ceph.com/smithi192/g' /etc/hosts 2020-11-28T21:40:06.840 INFO:teuthology.orchestra.run.smithi192:> sudo sed -i -e 's/172.21.15.192/172.21.15.192/g' /etc/hosts 2020-11-28T21:40:06.896 INFO:teuthology.orchestra.run.smithi192:> sudo sed -i -e 's/smithi192.front.sepia.ceph.com/smithi192/g' /etc/hostname 2020-11-28T21:40:06.951 INFO:teuthology.orchestra.run.smithi192:> sudo hostname smithi192 2020-11-28T21:40:07.006 INFO:teuthology.provision.fog.smithi192:Deploy complete! 2020-11-28T21:40:07.007 DEBUG:teuthology.parallel:result is None 2020-11-28T21:40:08.228 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:40:08.239 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-11-28T21:40:14.241 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:40:14.379 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-11-28T21:40:14.380 INFO:teuthology.orchestra.run.smithi162:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-11-28T21:40:14.779 INFO:teuthology.provision.fog.smithi162:Node is ready 2020-11-28T21:40:14.779 INFO:teuthology.orchestra.run.smithi162:> hostname 2020-11-28T21:40:14.828 INFO:teuthology.orchestra.run.smithi162.stdout:smithi162.front.sepia.ceph.com 2020-11-28T21:40:14.829 INFO:teuthology.orchestra.run.smithi162:> grep smithi162.front.sepia.ceph.com /etc/hosts 2020-11-28T21:40:14.877 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162 smithi162.front.sepia.ceph.com smithi162 2020-11-28T21:40:14.877 INFO:teuthology.orchestra.run.smithi162:> sudo hostname smithi162 2020-11-28T21:40:15.039 INFO:teuthology.orchestra.run.smithi162:> sudo sed -i -e 's/smithi162.front.sepia.ceph.com/smithi162/g' /etc/hosts 2020-11-28T21:40:15.053 INFO:teuthology.orchestra.run.smithi162:> sudo sed -i -e 's/172.21.15.162/172.21.15.162/g' /etc/hosts 2020-11-28T21:40:15.110 INFO:teuthology.orchestra.run.smithi162:> sudo sed -i -e 's/smithi162.front.sepia.ceph.com/smithi162/g' /etc/hostname 2020-11-28T21:40:15.165 INFO:teuthology.orchestra.run.smithi162:> sudo hostname smithi162 2020-11-28T21:40:15.219 INFO:teuthology.provision.fog.smithi162:Deploy complete! 2020-11-28T21:40:15.219 DEBUG:teuthology.parallel:result is None 2020-11-28T21:40:24.802 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:40:24.897 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-11-28T21:40:24.897 INFO:teuthology.orchestra.run.smithi118:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-11-28T21:40:25.281 INFO:teuthology.provision.fog.smithi118:Node is ready 2020-11-28T21:40:25.281 INFO:teuthology.orchestra.run.smithi118:> hostname 2020-11-28T21:40:25.328 INFO:teuthology.orchestra.run.smithi118.stdout:smithi118.front.sepia.ceph.com 2020-11-28T21:40:25.328 INFO:teuthology.orchestra.run.smithi118:> grep smithi118.front.sepia.ceph.com /etc/hosts 2020-11-28T21:40:25.377 INFO:teuthology.orchestra.run.smithi118.stdout:172.21.15.118 smithi118.front.sepia.ceph.com smithi118 2020-11-28T21:40:25.377 INFO:teuthology.orchestra.run.smithi118:> sudo hostname smithi118 2020-11-28T21:40:25.455 INFO:teuthology.orchestra.run.smithi118:> sudo sed -i -e 's/smithi118.front.sepia.ceph.com/smithi118/g' /etc/hosts 2020-11-28T21:40:25.514 INFO:teuthology.orchestra.run.smithi118:> sudo sed -i -e 's/172.21.15.118/172.21.15.118/g' /etc/hosts 2020-11-28T21:40:25.570 INFO:teuthology.orchestra.run.smithi118:> sudo sed -i -e 's/smithi118.front.sepia.ceph.com/smithi118/g' /etc/hostname 2020-11-28T21:40:25.627 INFO:teuthology.orchestra.run.smithi118:> sudo hostname smithi118 2020-11-28T21:40:25.685 INFO:teuthology.provision.fog.smithi118:Deploy complete! 2020-11-28T21:40:25.685 DEBUG:teuthology.parallel:result is None 2020-11-28T21:40:25.870 INFO:teuthology.lock.ops:Checking smithi162.front.sepia.ceph.com 2020-11-28T21:40:25.871 INFO:teuthology.lock.ops:New key found. Updating... 2020-11-28T21:40:25.887 INFO:teuthology.lock.ops:Checking smithi118.front.sepia.ceph.com 2020-11-28T21:40:25.887 INFO:teuthology.lock.ops:New key found. Updating... 2020-11-28T21:40:25.900 INFO:teuthology.lock.ops:Checking smithi192.front.sepia.ceph.com 2020-11-28T21:40:25.901 INFO:teuthology.lock.ops:New key found. Updating... 2020-11-28T21:40:25.914 INFO:teuthology.lock.ops:Checking smithi096.front.sepia.ceph.com 2020-11-28T21:40:25.915 INFO:teuthology.lock.ops:New key found. Updating... 2020-11-28T21:40:25.930 INFO:teuthology.lock.ops:Updating [smithi162.front.sepia.ceph.com]: set os type and version on server 2020-11-28T21:40:25.930 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-28T21:40:25.932 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:40:25.999 INFO:teuthology.orchestra.run.smithi162:> true 2020-11-28T21:40:26.218 INFO:teuthology.orchestra.run.smithi162:> uname -m 2020-11-28T21:40:26.266 INFO:teuthology.orchestra.run.smithi162.stdout:x86_64 2020-11-28T21:40:26.266 INFO:teuthology.orchestra.run.smithi162:> cat /etc/os-release 2020-11-28T21:40:26.313 INFO:teuthology.orchestra.run.smithi162.stdout:NAME="Ubuntu" 2020-11-28T21:40:26.314 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-11-28T21:40:26.314 INFO:teuthology.orchestra.run.smithi162.stdout:ID=ubuntu 2020-11-28T21:40:26.314 INFO:teuthology.orchestra.run.smithi162.stdout:ID_LIKE=debian 2020-11-28T21:40:26.315 INFO:teuthology.orchestra.run.smithi162.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-11-28T21:40:26.315 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION_ID="18.04" 2020-11-28T21:40:26.315 INFO:teuthology.orchestra.run.smithi162.stdout:HOME_URL="https://www.ubuntu.com/" 2020-11-28T21:40:26.315 INFO:teuthology.orchestra.run.smithi162.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-11-28T21:40:26.315 INFO:teuthology.orchestra.run.smithi162.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-11-28T21:40:26.316 INFO:teuthology.orchestra.run.smithi162.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-11-28T21:40:26.316 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION_CODENAME=bionic 2020-11-28T21:40:26.316 INFO:teuthology.orchestra.run.smithi162.stdout:UBUNTU_CODENAME=bionic 2020-11-28T21:40:26.317 INFO:teuthology.lock.ops:Updating smithi162.front.sepia.ceph.com on lock server 2020-11-28T21:40:26.334 INFO:teuthology.lock.ops:Updating [smithi118.front.sepia.ceph.com]: set os type and version on server 2020-11-28T21:40:26.334 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-28T21:40:26.335 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:40:26.412 INFO:teuthology.orchestra.run.smithi118:> true 2020-11-28T21:40:26.604 INFO:teuthology.orchestra.run.smithi118:> uname -m 2020-11-28T21:40:26.654 INFO:teuthology.orchestra.run.smithi118.stdout:x86_64 2020-11-28T21:40:26.655 INFO:teuthology.orchestra.run.smithi118:> cat /etc/os-release 2020-11-28T21:40:26.701 INFO:teuthology.orchestra.run.smithi118.stdout:NAME="Ubuntu" 2020-11-28T21:40:26.702 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-11-28T21:40:26.702 INFO:teuthology.orchestra.run.smithi118.stdout:ID=ubuntu 2020-11-28T21:40:26.702 INFO:teuthology.orchestra.run.smithi118.stdout:ID_LIKE=debian 2020-11-28T21:40:26.703 INFO:teuthology.orchestra.run.smithi118.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-11-28T21:40:26.703 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION_ID="18.04" 2020-11-28T21:40:26.704 INFO:teuthology.orchestra.run.smithi118.stdout:HOME_URL="https://www.ubuntu.com/" 2020-11-28T21:40:26.704 INFO:teuthology.orchestra.run.smithi118.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-11-28T21:40:26.705 INFO:teuthology.orchestra.run.smithi118.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-11-28T21:40:26.705 INFO:teuthology.orchestra.run.smithi118.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-11-28T21:40:26.706 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION_CODENAME=bionic 2020-11-28T21:40:26.706 INFO:teuthology.orchestra.run.smithi118.stdout:UBUNTU_CODENAME=bionic 2020-11-28T21:40:26.706 INFO:teuthology.lock.ops:Updating smithi118.front.sepia.ceph.com on lock server 2020-11-28T21:40:26.723 INFO:teuthology.lock.ops:Updating [smithi192.front.sepia.ceph.com]: set os type and version on server 2020-11-28T21:40:26.724 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-28T21:40:26.726 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:40:26.797 INFO:teuthology.orchestra.run.smithi192:> true 2020-11-28T21:40:26.978 INFO:teuthology.orchestra.run.smithi192:> uname -m 2020-11-28T21:40:27.024 INFO:teuthology.orchestra.run.smithi192.stdout:x86_64 2020-11-28T21:40:27.024 INFO:teuthology.orchestra.run.smithi192:> cat /etc/os-release 2020-11-28T21:40:27.072 INFO:teuthology.orchestra.run.smithi192.stdout:NAME="Ubuntu" 2020-11-28T21:40:27.072 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-11-28T21:40:27.073 INFO:teuthology.orchestra.run.smithi192.stdout:ID=ubuntu 2020-11-28T21:40:27.073 INFO:teuthology.orchestra.run.smithi192.stdout:ID_LIKE=debian 2020-11-28T21:40:27.074 INFO:teuthology.orchestra.run.smithi192.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-11-28T21:40:27.074 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION_ID="18.04" 2020-11-28T21:40:27.074 INFO:teuthology.orchestra.run.smithi192.stdout:HOME_URL="https://www.ubuntu.com/" 2020-11-28T21:40:27.075 INFO:teuthology.orchestra.run.smithi192.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-11-28T21:40:27.075 INFO:teuthology.orchestra.run.smithi192.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-11-28T21:40:27.075 INFO:teuthology.orchestra.run.smithi192.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-11-28T21:40:27.076 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION_CODENAME=bionic 2020-11-28T21:40:27.076 INFO:teuthology.orchestra.run.smithi192.stdout:UBUNTU_CODENAME=bionic 2020-11-28T21:40:27.076 INFO:teuthology.lock.ops:Updating smithi192.front.sepia.ceph.com on lock server 2020-11-28T21:40:27.094 INFO:teuthology.lock.ops:Updating [smithi096.front.sepia.ceph.com]: set os type and version on server 2020-11-28T21:40:27.094 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-28T21:40:27.095 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:40:27.165 INFO:teuthology.orchestra.run.smithi096:> true 2020-11-28T21:40:27.391 INFO:teuthology.orchestra.run.smithi096:> uname -m 2020-11-28T21:40:27.437 INFO:teuthology.orchestra.run.smithi096.stdout:x86_64 2020-11-28T21:40:27.437 INFO:teuthology.orchestra.run.smithi096:> cat /etc/os-release 2020-11-28T21:40:27.485 INFO:teuthology.orchestra.run.smithi096.stdout:NAME="Ubuntu" 2020-11-28T21:40:27.485 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-11-28T21:40:27.485 INFO:teuthology.orchestra.run.smithi096.stdout:ID=ubuntu 2020-11-28T21:40:27.486 INFO:teuthology.orchestra.run.smithi096.stdout:ID_LIKE=debian 2020-11-28T21:40:27.486 INFO:teuthology.orchestra.run.smithi096.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-11-28T21:40:27.486 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_ID="18.04" 2020-11-28T21:40:27.486 INFO:teuthology.orchestra.run.smithi096.stdout:HOME_URL="https://www.ubuntu.com/" 2020-11-28T21:40:27.486 INFO:teuthology.orchestra.run.smithi096.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-11-28T21:40:27.487 INFO:teuthology.orchestra.run.smithi096.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-11-28T21:40:27.487 INFO:teuthology.orchestra.run.smithi096.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-11-28T21:40:27.487 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_CODENAME=bionic 2020-11-28T21:40:27.487 INFO:teuthology.orchestra.run.smithi096.stdout:UBUNTU_CODENAME=bionic 2020-11-28T21:40:27.491 INFO:teuthology.lock.ops:Updating smithi096.front.sepia.ceph.com on lock server 2020-11-28T21:40:27.509 INFO:teuthology.task.internal.lock_machines:4 smithi machines locked this try, 4/4 locked so far 2020-11-28T21:40:27.581 INFO:teuthology.task.internal.lock_machines:Locked targets: smithi096.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDvbnZHmcs9Xr2dBzBkSrjn+U8zaCQ70Sp063V8Nq8wcY427+jzxcoOsW0cdDx+EvMYLu1rcO1zzQeNeX2zVCOI+8pCelRU0qINQigRedJsOc+4QxKDEMqyaa+mJJfO7L3sXCgsGxp9Vel3mpgq0UwEkyEOj0NENyVh5TbjVIBUJ5GNo94VW4OvwdaOtyZ7LoDUY5SIAhGp+IWzFUwb3ltgGtJnyJZ9sWtW6i8CWanTKaCWsYlVoUMtPTIiqXVI+90NkGBPRXtWmiMOyE4VDQqSQoop7UgcsUG7IFUzpTyOKZaFqFcgu0H2AMvrxcHnN8i13nN+nQ/UU90zSUDTEd1 smithi118.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDqeTK6CjSUKc3bD1q56WU85Yjz05zVszqohM/uxV1dA2+yWJwMqoSDcCtSr98rR1Q5a9LbPykWBlZ08UDRlPKjtYtM1wP+axpmXPRL/Mp2DaL7d98ZxPrVuCO2TwqwjLogQ1vr72jL8ze7mTdjei+I/QgFl9thSTa68NoCYflPkIAbhUhP+z7LNUSB32PHJibPtRmYSNQXkAC1aDFWfnb9Zzz/xHft2R9D7P9xXD0Oum+L2H+cFmCBA0e4qhP+HtIRFBOZ+fCaE5M/RUpRVnODQKCuyXxwkRYOIeQTUn4a4VGbcCq+rWnJelzTVG87HruHauGJRMxzJ6PuO6zz2osb smithi162.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDR4skALubozLqiyWNdPiV3nXDJ59kK6uDCVCkvXxL23MBmGaGOBFcR13j69Vitu69OMVzysGee1ys5ovVzIHqiCzw/mKgxlYTlZ1r7wUXS8zxT6sn4towy+HnVWwUDi2VRguyHMTA2Tt0cIUa5xpiB1u5QRAIDxKF+sC+l/Rranvk2rMgIbnBUSldMtcfxzc/hGV7BRieV+Av2i+mf9hwVd8L6XMkwW+cwftwQaxGsk+0aspR/ANoNGW9IIP48uykU+mpv8seqzzloRNhSw0Wdh19BzFMkOPt+nfkOT7OxEV8G3ONu/bE0MxkBzhjwK9HBv2dhLXvuYdHl5dnivWsJ smithi192.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDG63qqli+Xx28jOQ/UofpMSrbdhqWqhJXbtgTZPAHScGFWeLmG2DjPrJlXvaJvE7cu1AXUj5VXo+zCpvxD1TgvykDEg/9CU+nXDLej/KX0ccpu0RJUHCBu9/c6X7VuCLoHMScuZ9d6v/sWLtRFHkCvKESo+dWbgQYh+Wa7ESay7X19VdoyJWJCiHraxZt4+uro5Ivq4pON1syctfHztqYEw/5d/TkgIqUkrmtauZHpChUJ0pcit03GAWZv3zKCKJDR4aw99hJ7nCiAegaxVJztjOxXPwvXXOMKGJbNWx0l/StPOQs83JkUXuOscggDlXWlau2XlrYcicpl2X1LrXod 2020-11-28T21:40:27.582 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-11-28T21:40:27.646 INFO:teuthology.run_tasks:Running task internal.save_config... 2020-11-28T21:40:27.654 INFO:teuthology.task.internal:Saving configuration 2020-11-28T21:40:27.687 INFO:teuthology.run_tasks:Running task internal.check_lock... 2020-11-28T21:40:27.695 INFO:teuthology.task.internal.check_lock:Checking locks... 2020-11-28T21:40:27.713 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-11-28 21:34:40.609577', 'description': '/home/teuthworker/archive/teuthology-2020-11-25_04:10:04-fs-octopus-distro-basic-smithi/5655905', 'locked_by': 'scheduled_teuthology@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMrOcuGF5MQrnf7QxMyTckBFd4GLCYlE4krW9pxLMKeg', 'os_version': '18.04', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi162.front.sepia.ceph.com'} 2020-11-28T21:40:27.730 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-11-28 21:34:40.609598', 'description': '/home/teuthworker/archive/teuthology-2020-11-25_04:10:04-fs-octopus-distro-basic-smithi/5655905', 'locked_by': 'scheduled_teuthology@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAVtloKr1B2vjoGK0nKvwVIFPwyarPflLOCEYKnCErJo', 'os_version': '18.04', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi118.front.sepia.ceph.com'} 2020-11-28T21:40:27.747 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-11-28 21:34:40.609614', 'description': '/home/teuthworker/archive/teuthology-2020-11-25_04:10:04-fs-octopus-distro-basic-smithi/5655905', 'locked_by': 'scheduled_teuthology@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMYpja7wWigIwZz8dDdR70giPGClIH78j/yiRDS66kve', 'os_version': '18.04', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi192.front.sepia.ceph.com'} 2020-11-28T21:40:27.766 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-11-28 21:34:40.609630', 'description': '/home/teuthworker/archive/teuthology-2020-11-25_04:10:04-fs-octopus-distro-basic-smithi/5655905', 'locked_by': 'scheduled_teuthology@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIND2IQLPmGTg/GzJrvlLvGFK7VXr4h/9htamUzhXomkR', 'os_version': '18.04', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi096.front.sepia.ceph.com'} 2020-11-28T21:40:27.767 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2020-11-28T21:40:27.774 INFO:teuthology.task.internal:roles: ubuntu@smithi162.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3'] 2020-11-28T21:40:27.774 INFO:teuthology.task.internal:roles: ubuntu@smithi118.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2020-11-28T21:40:27.774 INFO:teuthology.task.internal:roles: ubuntu@smithi192.front.sepia.ceph.com - ['client.0'] 2020-11-28T21:40:27.775 INFO:teuthology.task.internal:roles: ubuntu@smithi096.front.sepia.ceph.com - ['client.1'] 2020-11-28T21:40:27.775 INFO:teuthology.run_tasks:Running task console_log... 2020-11-28T21:40:27.906 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f83b930d048>, signals=[15]) 2020-11-28T21:40:27.907 INFO:teuthology.run_tasks:Running task internal.connect... 2020-11-28T21:40:27.917 INFO:teuthology.task.internal:Opening connections... 2020-11-28T21:40:27.917 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi162.front.sepia.ceph.com 2020-11-28T21:40:27.920 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:40:28.001 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi118.front.sepia.ceph.com 2020-11-28T21:40:28.003 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:40:28.127 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi192.front.sepia.ceph.com 2020-11-28T21:40:28.128 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:40:28.206 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi096.front.sepia.ceph.com 2020-11-28T21:40:28.208 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:40:28.284 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2020-11-28T21:40:28.292 INFO:teuthology.orchestra.run.smithi162:> uname -m 2020-11-28T21:40:28.297 INFO:teuthology.orchestra.run.smithi162.stdout:x86_64 2020-11-28T21:40:28.297 INFO:teuthology.orchestra.run.smithi162:> cat /etc/os-release 2020-11-28T21:40:28.341 INFO:teuthology.orchestra.run.smithi162.stdout:NAME="Ubuntu" 2020-11-28T21:40:28.341 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-11-28T21:40:28.342 INFO:teuthology.orchestra.run.smithi162.stdout:ID=ubuntu 2020-11-28T21:40:28.342 INFO:teuthology.orchestra.run.smithi162.stdout:ID_LIKE=debian 2020-11-28T21:40:28.342 INFO:teuthology.orchestra.run.smithi162.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-11-28T21:40:28.342 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION_ID="18.04" 2020-11-28T21:40:28.342 INFO:teuthology.orchestra.run.smithi162.stdout:HOME_URL="https://www.ubuntu.com/" 2020-11-28T21:40:28.343 INFO:teuthology.orchestra.run.smithi162.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-11-28T21:40:28.343 INFO:teuthology.orchestra.run.smithi162.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-11-28T21:40:28.343 INFO:teuthology.orchestra.run.smithi162.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-11-28T21:40:28.343 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION_CODENAME=bionic 2020-11-28T21:40:28.343 INFO:teuthology.orchestra.run.smithi162.stdout:UBUNTU_CODENAME=bionic 2020-11-28T21:40:28.344 INFO:teuthology.lock.ops:Updating smithi162.front.sepia.ceph.com on lock server 2020-11-28T21:40:28.360 INFO:teuthology.orchestra.run.smithi118:> uname -m 2020-11-28T21:40:28.365 INFO:teuthology.orchestra.run.smithi118.stdout:x86_64 2020-11-28T21:40:28.365 INFO:teuthology.orchestra.run.smithi118:> cat /etc/os-release 2020-11-28T21:40:28.414 INFO:teuthology.orchestra.run.smithi118.stdout:NAME="Ubuntu" 2020-11-28T21:40:28.414 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-11-28T21:40:28.415 INFO:teuthology.orchestra.run.smithi118.stdout:ID=ubuntu 2020-11-28T21:40:28.415 INFO:teuthology.orchestra.run.smithi118.stdout:ID_LIKE=debian 2020-11-28T21:40:28.415 INFO:teuthology.orchestra.run.smithi118.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-11-28T21:40:28.415 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION_ID="18.04" 2020-11-28T21:40:28.416 INFO:teuthology.orchestra.run.smithi118.stdout:HOME_URL="https://www.ubuntu.com/" 2020-11-28T21:40:28.416 INFO:teuthology.orchestra.run.smithi118.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-11-28T21:40:28.416 INFO:teuthology.orchestra.run.smithi118.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-11-28T21:40:28.416 INFO:teuthology.orchestra.run.smithi118.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-11-28T21:40:28.416 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION_CODENAME=bionic 2020-11-28T21:40:28.417 INFO:teuthology.orchestra.run.smithi118.stdout:UBUNTU_CODENAME=bionic 2020-11-28T21:40:28.417 INFO:teuthology.lock.ops:Updating smithi118.front.sepia.ceph.com on lock server 2020-11-28T21:40:28.466 INFO:teuthology.orchestra.run.smithi192:> uname -m 2020-11-28T21:40:28.471 INFO:teuthology.orchestra.run.smithi192.stdout:x86_64 2020-11-28T21:40:28.471 INFO:teuthology.orchestra.run.smithi192:> cat /etc/os-release 2020-11-28T21:40:28.515 INFO:teuthology.orchestra.run.smithi192.stdout:NAME="Ubuntu" 2020-11-28T21:40:28.515 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-11-28T21:40:28.515 INFO:teuthology.orchestra.run.smithi192.stdout:ID=ubuntu 2020-11-28T21:40:28.516 INFO:teuthology.orchestra.run.smithi192.stdout:ID_LIKE=debian 2020-11-28T21:40:28.516 INFO:teuthology.orchestra.run.smithi192.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-11-28T21:40:28.516 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION_ID="18.04" 2020-11-28T21:40:28.516 INFO:teuthology.orchestra.run.smithi192.stdout:HOME_URL="https://www.ubuntu.com/" 2020-11-28T21:40:28.516 INFO:teuthology.orchestra.run.smithi192.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-11-28T21:40:28.517 INFO:teuthology.orchestra.run.smithi192.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-11-28T21:40:28.517 INFO:teuthology.orchestra.run.smithi192.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-11-28T21:40:28.517 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION_CODENAME=bionic 2020-11-28T21:40:28.517 INFO:teuthology.orchestra.run.smithi192.stdout:UBUNTU_CODENAME=bionic 2020-11-28T21:40:28.518 INFO:teuthology.lock.ops:Updating smithi192.front.sepia.ceph.com on lock server 2020-11-28T21:40:28.532 INFO:teuthology.orchestra.run.smithi096:> uname -m 2020-11-28T21:40:28.537 INFO:teuthology.orchestra.run.smithi096.stdout:x86_64 2020-11-28T21:40:28.537 INFO:teuthology.orchestra.run.smithi096:> cat /etc/os-release 2020-11-28T21:40:28.584 INFO:teuthology.orchestra.run.smithi096.stdout:NAME="Ubuntu" 2020-11-28T21:40:28.585 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-11-28T21:40:28.585 INFO:teuthology.orchestra.run.smithi096.stdout:ID=ubuntu 2020-11-28T21:40:28.585 INFO:teuthology.orchestra.run.smithi096.stdout:ID_LIKE=debian 2020-11-28T21:40:28.585 INFO:teuthology.orchestra.run.smithi096.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-11-28T21:40:28.586 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_ID="18.04" 2020-11-28T21:40:28.586 INFO:teuthology.orchestra.run.smithi096.stdout:HOME_URL="https://www.ubuntu.com/" 2020-11-28T21:40:28.586 INFO:teuthology.orchestra.run.smithi096.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-11-28T21:40:28.586 INFO:teuthology.orchestra.run.smithi096.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-11-28T21:40:28.586 INFO:teuthology.orchestra.run.smithi096.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-11-28T21:40:28.587 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_CODENAME=bionic 2020-11-28T21:40:28.587 INFO:teuthology.orchestra.run.smithi096.stdout:UBUNTU_CODENAME=bionic 2020-11-28T21:40:28.587 INFO:teuthology.lock.ops:Updating smithi096.front.sepia.ceph.com on lock server 2020-11-28T21:40:28.603 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2020-11-28T21:40:28.641 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2020-11-28T21:40:28.663 INFO:teuthology.task.internal:Checking for old test directory... 2020-11-28T21:40:28.664 INFO:teuthology.orchestra.run.smithi096:> test '!' -e /home/ubuntu/cephtest 2020-11-28T21:40:28.666 INFO:teuthology.orchestra.run.smithi118:> test '!' -e /home/ubuntu/cephtest 2020-11-28T21:40:28.669 INFO:teuthology.orchestra.run.smithi162:> test '!' -e /home/ubuntu/cephtest 2020-11-28T21:40:28.672 INFO:teuthology.orchestra.run.smithi192:> test '!' -e /home/ubuntu/cephtest 2020-11-28T21:40:28.677 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2020-11-28T21:40:28.683 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2020-11-28T21:40:28.684 INFO:teuthology.orchestra.run.smithi096:> test -z $(ls -A /var/lib/ceph) 2020-11-28T21:40:28.717 INFO:teuthology.orchestra.run.smithi118:> test -z $(ls -A /var/lib/ceph) 2020-11-28T21:40:28.721 INFO:teuthology.orchestra.run.smithi162:> test -z $(ls -A /var/lib/ceph) 2020-11-28T21:40:28.723 INFO:teuthology.orchestra.run.smithi192:> test -z $(ls -A /var/lib/ceph) 2020-11-28T21:40:28.731 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2020-11-28T21:40:28.814 INFO:teuthology.run_tasks:Running task kernel... 2020-11-28T21:40:28.830 DEBUG:teuthology.task.kernel:normalized config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mgr.y': {'kdb': True, 'sha1': 'distro'}, 'mds.a': {'kdb': True, 'sha1': 'distro'}, 'mds.c': {'kdb': True, 'sha1': 'distro'}, 'osd.0': {'kdb': True, 'sha1': 'distro'}, 'osd.1': {'kdb': True, 'sha1': 'distro'}, 'osd.2': {'kdb': True, 'sha1': 'distro'}, 'osd.3': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'mon.c': {'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'kdb': True, 'sha1': 'distro'}, 'mds.b': {'kdb': True, 'sha1': 'distro'}, 'osd.4': {'kdb': True, 'sha1': 'distro'}, 'osd.5': {'kdb': True, 'sha1': 'distro'}, 'osd.6': {'kdb': True, 'sha1': 'distro'}, 'osd.7': {'kdb': True, 'sha1': 'distro'}, 'client.0': {'kdb': True, 'sha1': 'distro'}, 'client.1': {'kdb': True, 'sha1': 'distro'}} 2020-11-28T21:40:28.831 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'client.0': {'kdb': True, 'sha1': 'distro'}, 'client.1': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2020-11-28T21:40:28.831 INFO:teuthology.orchestra.run.smithi162:> uname -r 2020-11-28T21:40:28.836 INFO:teuthology.orchestra.run.smithi162.stdout:4.15.0-124-generic 2020-11-28T21:40:28.837 INFO:teuthology.task.kernel:Running kernel on smithi162: 4.15.0-124-generic 2020-11-28T21:40:28.837 INFO:teuthology.orchestra.run.smithi162:> sudo apt-get clean 2020-11-28T21:40:28.985 INFO:teuthology.orchestra.run.smithi162:> sudo apt-get update 2020-11-28T21:40:29.110 INFO:teuthology.orchestra.run.smithi162.stdout:Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2020-11-28T21:40:29.229 INFO:teuthology.orchestra.run.smithi162.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-11-28T21:40:29.310 INFO:teuthology.orchestra.run.smithi162.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2020-11-28T21:40:29.403 INFO:teuthology.orchestra.run.smithi162.stdout:Get:4 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [1,425 kB] 2020-11-28T21:40:29.542 INFO:teuthology.orchestra.run.smithi162.stdout:Get:5 http://security.ubuntu.com/ubuntu bionic-security/main i386 Packages [861 kB] 2020-11-28T21:40:29.559 INFO:teuthology.orchestra.run.smithi162.stdout:Get:6 http://security.ubuntu.com/ubuntu bionic-security/universe i386 Packages [967 kB] 2020-11-28T21:40:29.571 INFO:teuthology.orchestra.run.smithi162.stdout:Get:7 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1,089 kB] 2020-11-28T21:40:29.577 INFO:teuthology.orchestra.run.smithi162.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2020-11-28T21:40:29.583 INFO:teuthology.orchestra.run.smithi162.stdout:Get:9 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [243 kB] 2020-11-28T21:40:29.708 INFO:teuthology.orchestra.run.smithi162.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages [1,162 kB] 2020-11-28T21:40:30.053 INFO:teuthology.orchestra.run.smithi162.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [1,761 kB] 2020-11-28T21:40:30.156 INFO:teuthology.orchestra.run.smithi162.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,693 kB] 2020-11-28T21:40:30.215 INFO:teuthology.orchestra.run.smithi162.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages [1,547 kB] 2020-11-28T21:40:30.251 INFO:teuthology.orchestra.run.smithi162.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [356 kB] 2020-11-28T21:40:31.137 INFO:teuthology.orchestra.run.smithi162.stdout:Fetched 11.4 MB in 2s (5,907 kB/s) 2020-11-28T21:40:32.855 INFO:teuthology.orchestra.run.smithi162.stdout:Reading package lists... 2020-11-28T21:40:32.902 INFO:teuthology.orchestra.run.smithi162:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2020-11-28T21:40:33.000 INFO:teuthology.orchestra.run.smithi162.stdout:Reading package lists... 2020-11-28T21:40:33.256 INFO:teuthology.orchestra.run.smithi162.stdout:Building dependency tree... 2020-11-28T21:40:33.256 INFO:teuthology.orchestra.run.smithi162.stdout:Reading state information... 2020-11-28T21:40:33.336 INFO:teuthology.orchestra.run.smithi162.stderr:E: Unable to locate package linux-image-current-generic 2020-11-28T21:40:33.337 DEBUG:teuthology.orchestra.run:got remote process result: 100 2020-11-28T21:40:33.338 INFO:teuthology.orchestra.run.smithi162:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2020-11-28T21:40:33.417 INFO:teuthology.orchestra.run.smithi162.stdout:Reading package lists... 2020-11-28T21:40:33.648 INFO:teuthology.orchestra.run.smithi162.stdout:Building dependency tree... 2020-11-28T21:40:33.649 INFO:teuthology.orchestra.run.smithi162.stdout:Reading state information... 2020-11-28T21:40:33.853 INFO:teuthology.orchestra.run.smithi162.stdout:linux-image-generic is already the newest version (4.15.0.124.111). 2020-11-28T21:40:33.854 INFO:teuthology.orchestra.run.smithi162.stdout:linux-image-generic set to manually installed. 2020-11-28T21:40:33.854 INFO:teuthology.orchestra.run.smithi162.stdout:0 upgraded, 0 newly installed, 0 to remove and 5 not upgraded. 2020-11-28T21:40:33.855 INFO:teuthology.orchestra.run.smithi162:> dpkg -s linux-image-generic 2020-11-28T21:40:33.875 INFO:teuthology.orchestra.run.smithi162.stdout:Package: linux-image-generic 2020-11-28T21:40:33.876 INFO:teuthology.orchestra.run.smithi162.stdout:Status: install ok installed 2020-11-28T21:40:33.877 INFO:teuthology.orchestra.run.smithi162.stdout:Priority: optional 2020-11-28T21:40:33.877 INFO:teuthology.orchestra.run.smithi162.stdout:Section: kernel 2020-11-28T21:40:33.877 INFO:teuthology.orchestra.run.smithi162.stdout:Installed-Size: 16 2020-11-28T21:40:33.877 INFO:teuthology.orchestra.run.smithi162.stdout:Maintainer: Ubuntu Kernel Team 2020-11-28T21:40:33.878 INFO:teuthology.orchestra.run.smithi162.stdout:Architecture: amd64 2020-11-28T21:40:33.878 INFO:teuthology.orchestra.run.smithi162.stdout:Source: linux-meta 2020-11-28T21:40:33.878 INFO:teuthology.orchestra.run.smithi162.stdout:Version: 4.15.0.124.111 2020-11-28T21:40:33.878 INFO:teuthology.orchestra.run.smithi162.stdout:Provides: wireguard-modules (= 1.0.20200611-1ubuntu1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2020-11-28T21:40:33.878 INFO:teuthology.orchestra.run.smithi162.stdout:Depends: linux-image-4.15.0-124-generic, linux-modules-extra-4.15.0-124-generic, linux-firmware, intel-microcode, amd64-microcode 2020-11-28T21:40:33.879 INFO:teuthology.orchestra.run.smithi162.stdout:Recommends: thermald 2020-11-28T21:40:33.879 INFO:teuthology.orchestra.run.smithi162.stdout:Description: Generic Linux kernel image 2020-11-28T21:40:33.879 INFO:teuthology.orchestra.run.smithi162.stdout: This package will always depend on the latest generic kernel image 2020-11-28T21:40:33.879 INFO:teuthology.orchestra.run.smithi162.stdout: available. 2020-11-28T21:40:33.880 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2020-11-28T21:40:33.880 INFO:teuthology.orchestra.run.smithi118:> uname -r 2020-11-28T21:40:33.886 INFO:teuthology.orchestra.run.smithi118.stdout:4.15.0-124-generic 2020-11-28T21:40:33.887 INFO:teuthology.task.kernel:Running kernel on smithi118: 4.15.0-124-generic 2020-11-28T21:40:33.887 INFO:teuthology.orchestra.run.smithi118:> sudo apt-get clean 2020-11-28T21:40:34.039 INFO:teuthology.orchestra.run.smithi118:> sudo apt-get update 2020-11-28T21:40:34.169 INFO:teuthology.orchestra.run.smithi118.stdout:Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2020-11-28T21:40:34.285 INFO:teuthology.orchestra.run.smithi118.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-11-28T21:40:34.364 INFO:teuthology.orchestra.run.smithi118.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2020-11-28T21:40:34.437 INFO:teuthology.orchestra.run.smithi118.stdout:Get:4 http://security.ubuntu.com/ubuntu bionic-security/main i386 Packages [861 kB] 2020-11-28T21:40:34.550 INFO:teuthology.orchestra.run.smithi118.stdout:Get:5 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [1,425 kB] 2020-11-28T21:40:34.579 INFO:teuthology.orchestra.run.smithi118.stdout:Get:6 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1,089 kB] 2020-11-28T21:40:34.599 INFO:teuthology.orchestra.run.smithi118.stdout:Get:7 http://security.ubuntu.com/ubuntu bionic-security/universe i386 Packages [967 kB] 2020-11-28T21:40:34.611 INFO:teuthology.orchestra.run.smithi118.stdout:Get:8 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [243 kB] 2020-11-28T21:40:34.628 INFO:teuthology.orchestra.run.smithi118.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2020-11-28T21:40:34.738 INFO:teuthology.orchestra.run.smithi118.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages [1,162 kB] 2020-11-28T21:40:35.074 INFO:teuthology.orchestra.run.smithi118.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [1,761 kB] 2020-11-28T21:40:35.173 INFO:teuthology.orchestra.run.smithi118.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,693 kB] 2020-11-28T21:40:35.226 INFO:teuthology.orchestra.run.smithi118.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages [1,547 kB] 2020-11-28T21:40:35.265 INFO:teuthology.orchestra.run.smithi118.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [356 kB] 2020-11-28T21:40:36.236 INFO:teuthology.orchestra.run.smithi118.stdout:Fetched 11.4 MB in 2s (5,906 kB/s) 2020-11-28T21:40:37.743 INFO:teuthology.orchestra.run.smithi118.stdout:Reading package lists... 2020-11-28T21:40:37.788 INFO:teuthology.orchestra.run.smithi118:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2020-11-28T21:40:37.876 INFO:teuthology.orchestra.run.smithi118.stdout:Reading package lists... 2020-11-28T21:40:38.106 INFO:teuthology.orchestra.run.smithi118.stdout:Building dependency tree... 2020-11-28T21:40:38.107 INFO:teuthology.orchestra.run.smithi118.stdout:Reading state information... 2020-11-28T21:40:38.184 INFO:teuthology.orchestra.run.smithi118.stderr:E: Unable to locate package linux-image-current-generic 2020-11-28T21:40:38.185 DEBUG:teuthology.orchestra.run:got remote process result: 100 2020-11-28T21:40:38.186 INFO:teuthology.orchestra.run.smithi118:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2020-11-28T21:40:38.269 INFO:teuthology.orchestra.run.smithi118.stdout:Reading package lists... 2020-11-28T21:40:38.509 INFO:teuthology.orchestra.run.smithi118.stdout:Building dependency tree... 2020-11-28T21:40:38.509 INFO:teuthology.orchestra.run.smithi118.stdout:Reading state information... 2020-11-28T21:40:38.712 INFO:teuthology.orchestra.run.smithi118.stdout:linux-image-generic is already the newest version (4.15.0.124.111). 2020-11-28T21:40:38.713 INFO:teuthology.orchestra.run.smithi118.stdout:linux-image-generic set to manually installed. 2020-11-28T21:40:38.713 INFO:teuthology.orchestra.run.smithi118.stdout:0 upgraded, 0 newly installed, 0 to remove and 5 not upgraded. 2020-11-28T21:40:38.715 INFO:teuthology.orchestra.run.smithi118:> dpkg -s linux-image-generic 2020-11-28T21:40:38.739 INFO:teuthology.orchestra.run.smithi118.stdout:Package: linux-image-generic 2020-11-28T21:40:38.740 INFO:teuthology.orchestra.run.smithi118.stdout:Status: install ok installed 2020-11-28T21:40:38.740 INFO:teuthology.orchestra.run.smithi118.stdout:Priority: optional 2020-11-28T21:40:38.741 INFO:teuthology.orchestra.run.smithi118.stdout:Section: kernel 2020-11-28T21:40:38.741 INFO:teuthology.orchestra.run.smithi118.stdout:Installed-Size: 16 2020-11-28T21:40:38.741 INFO:teuthology.orchestra.run.smithi118.stdout:Maintainer: Ubuntu Kernel Team 2020-11-28T21:40:38.742 INFO:teuthology.orchestra.run.smithi118.stdout:Architecture: amd64 2020-11-28T21:40:38.742 INFO:teuthology.orchestra.run.smithi118.stdout:Source: linux-meta 2020-11-28T21:40:38.743 INFO:teuthology.orchestra.run.smithi118.stdout:Version: 4.15.0.124.111 2020-11-28T21:40:38.743 INFO:teuthology.orchestra.run.smithi118.stdout:Provides: wireguard-modules (= 1.0.20200611-1ubuntu1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2020-11-28T21:40:38.743 INFO:teuthology.orchestra.run.smithi118.stdout:Depends: linux-image-4.15.0-124-generic, linux-modules-extra-4.15.0-124-generic, linux-firmware, intel-microcode, amd64-microcode 2020-11-28T21:40:38.744 INFO:teuthology.orchestra.run.smithi118.stdout:Recommends: thermald 2020-11-28T21:40:38.744 INFO:teuthology.orchestra.run.smithi118.stdout:Description: Generic Linux kernel image 2020-11-28T21:40:38.745 INFO:teuthology.orchestra.run.smithi118.stdout: This package will always depend on the latest generic kernel image 2020-11-28T21:40:38.745 INFO:teuthology.orchestra.run.smithi118.stdout: available. 2020-11-28T21:40:38.746 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2020-11-28T21:40:38.746 INFO:teuthology.orchestra.run.smithi192:> uname -r 2020-11-28T21:40:38.753 INFO:teuthology.orchestra.run.smithi192.stdout:4.15.0-124-generic 2020-11-28T21:40:38.754 INFO:teuthology.task.kernel:Running kernel on smithi192: 4.15.0-124-generic 2020-11-28T21:40:38.754 INFO:teuthology.orchestra.run.smithi192:> sudo apt-get clean 2020-11-28T21:40:38.893 INFO:teuthology.orchestra.run.smithi192:> sudo apt-get update 2020-11-28T21:40:39.151 INFO:teuthology.orchestra.run.smithi192.stdout:Hit:1 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-11-28T21:40:39.151 INFO:teuthology.orchestra.run.smithi192.stdout:Get:2 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2020-11-28T21:40:39.216 INFO:teuthology.orchestra.run.smithi192.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2020-11-28T21:40:39.507 INFO:teuthology.orchestra.run.smithi192.stdout:Get:4 http://security.ubuntu.com/ubuntu bionic-security/main i386 Packages [861 kB] 2020-11-28T21:40:39.514 INFO:teuthology.orchestra.run.smithi192.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2020-11-28T21:40:39.639 INFO:teuthology.orchestra.run.smithi192.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [1,761 kB] 2020-11-28T21:40:39.875 INFO:teuthology.orchestra.run.smithi192.stdout:Get:7 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [1,425 kB] 2020-11-28T21:40:39.975 INFO:teuthology.orchestra.run.smithi192.stdout:Get:8 http://security.ubuntu.com/ubuntu bionic-security/universe i386 Packages [967 kB] 2020-11-28T21:40:40.020 INFO:teuthology.orchestra.run.smithi192.stdout:Get:9 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1,089 kB] 2020-11-28T21:40:40.029 INFO:teuthology.orchestra.run.smithi192.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages [1,162 kB] 2020-11-28T21:40:40.061 INFO:teuthology.orchestra.run.smithi192.stdout:Get:11 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [243 kB] 2020-11-28T21:40:40.062 INFO:teuthology.orchestra.run.smithi192.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,693 kB] 2020-11-28T21:40:40.135 INFO:teuthology.orchestra.run.smithi192.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages [1,547 kB] 2020-11-28T21:40:40.190 INFO:teuthology.orchestra.run.smithi192.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [356 kB] 2020-11-28T21:40:41.444 INFO:teuthology.orchestra.run.smithi192.stdout:Fetched 11.4 MB in 2s (4,866 kB/s) 2020-11-28T21:40:42.938 INFO:teuthology.orchestra.run.smithi192.stdout:Reading package lists... 2020-11-28T21:40:42.987 INFO:teuthology.orchestra.run.smithi192:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2020-11-28T21:40:43.082 INFO:teuthology.orchestra.run.smithi192.stdout:Reading package lists... 2020-11-28T21:40:43.404 INFO:teuthology.orchestra.run.smithi192.stdout:Building dependency tree... 2020-11-28T21:40:43.405 INFO:teuthology.orchestra.run.smithi192.stdout:Reading state information... 2020-11-28T21:40:43.487 INFO:teuthology.orchestra.run.smithi192.stderr:E: Unable to locate package linux-image-current-generic 2020-11-28T21:40:43.488 DEBUG:teuthology.orchestra.run:got remote process result: 100 2020-11-28T21:40:43.489 INFO:teuthology.orchestra.run.smithi192:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2020-11-28T21:40:43.571 INFO:teuthology.orchestra.run.smithi192.stdout:Reading package lists... 2020-11-28T21:40:43.822 INFO:teuthology.orchestra.run.smithi192.stdout:Building dependency tree... 2020-11-28T21:40:43.823 INFO:teuthology.orchestra.run.smithi192.stdout:Reading state information... 2020-11-28T21:40:44.029 INFO:teuthology.orchestra.run.smithi192.stdout:linux-image-generic is already the newest version (4.15.0.124.111). 2020-11-28T21:40:44.029 INFO:teuthology.orchestra.run.smithi192.stdout:linux-image-generic set to manually installed. 2020-11-28T21:40:44.029 INFO:teuthology.orchestra.run.smithi192.stdout:0 upgraded, 0 newly installed, 0 to remove and 5 not upgraded. 2020-11-28T21:40:44.030 INFO:teuthology.orchestra.run.smithi192:> dpkg -s linux-image-generic 2020-11-28T21:40:44.052 INFO:teuthology.orchestra.run.smithi192.stdout:Package: linux-image-generic 2020-11-28T21:40:44.052 INFO:teuthology.orchestra.run.smithi192.stdout:Status: install ok installed 2020-11-28T21:40:44.053 INFO:teuthology.orchestra.run.smithi192.stdout:Priority: optional 2020-11-28T21:40:44.053 INFO:teuthology.orchestra.run.smithi192.stdout:Section: kernel 2020-11-28T21:40:44.053 INFO:teuthology.orchestra.run.smithi192.stdout:Installed-Size: 16 2020-11-28T21:40:44.053 INFO:teuthology.orchestra.run.smithi192.stdout:Maintainer: Ubuntu Kernel Team 2020-11-28T21:40:44.053 INFO:teuthology.orchestra.run.smithi192.stdout:Architecture: amd64 2020-11-28T21:40:44.054 INFO:teuthology.orchestra.run.smithi192.stdout:Source: linux-meta 2020-11-28T21:40:44.054 INFO:teuthology.orchestra.run.smithi192.stdout:Version: 4.15.0.124.111 2020-11-28T21:40:44.054 INFO:teuthology.orchestra.run.smithi192.stdout:Provides: wireguard-modules (= 1.0.20200611-1ubuntu1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2020-11-28T21:40:44.054 INFO:teuthology.orchestra.run.smithi192.stdout:Depends: linux-image-4.15.0-124-generic, linux-modules-extra-4.15.0-124-generic, linux-firmware, intel-microcode, amd64-microcode 2020-11-28T21:40:44.055 INFO:teuthology.orchestra.run.smithi192.stdout:Recommends: thermald 2020-11-28T21:40:44.055 INFO:teuthology.orchestra.run.smithi192.stdout:Description: Generic Linux kernel image 2020-11-28T21:40:44.055 INFO:teuthology.orchestra.run.smithi192.stdout: This package will always depend on the latest generic kernel image 2020-11-28T21:40:44.055 INFO:teuthology.orchestra.run.smithi192.stdout: available. 2020-11-28T21:40:44.055 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2020-11-28T21:40:44.056 INFO:teuthology.orchestra.run.smithi096:> uname -r 2020-11-28T21:40:44.063 INFO:teuthology.orchestra.run.smithi096.stdout:4.15.0-124-generic 2020-11-28T21:40:44.064 INFO:teuthology.task.kernel:Running kernel on smithi096: 4.15.0-124-generic 2020-11-28T21:40:44.064 INFO:teuthology.orchestra.run.smithi096:> sudo apt-get clean 2020-11-28T21:40:44.316 INFO:teuthology.orchestra.run.smithi096:> sudo apt-get update 2020-11-28T21:40:44.453 INFO:teuthology.orchestra.run.smithi096.stdout:Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2020-11-28T21:40:44.567 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-11-28T21:40:44.646 INFO:teuthology.orchestra.run.smithi096.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2020-11-28T21:40:44.699 INFO:teuthology.orchestra.run.smithi096.stdout:Get:4 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [1,425 kB] 2020-11-28T21:40:44.841 INFO:teuthology.orchestra.run.smithi096.stdout:Get:5 http://security.ubuntu.com/ubuntu bionic-security/main i386 Packages [861 kB] 2020-11-28T21:40:44.856 INFO:teuthology.orchestra.run.smithi096.stdout:Get:6 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1,089 kB] 2020-11-28T21:40:44.869 INFO:teuthology.orchestra.run.smithi096.stdout:Get:7 http://security.ubuntu.com/ubuntu bionic-security/universe i386 Packages [967 kB] 2020-11-28T21:40:44.883 INFO:teuthology.orchestra.run.smithi096.stdout:Get:8 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [243 kB] 2020-11-28T21:40:44.904 INFO:teuthology.orchestra.run.smithi096.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2020-11-28T21:40:45.026 INFO:teuthology.orchestra.run.smithi096.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [1,761 kB] 2020-11-28T21:40:45.345 INFO:teuthology.orchestra.run.smithi096.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages [1,162 kB] 2020-11-28T21:40:45.397 INFO:teuthology.orchestra.run.smithi096.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages [1,547 kB] 2020-11-28T21:40:45.446 INFO:teuthology.orchestra.run.smithi096.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,693 kB] 2020-11-28T21:40:45.480 INFO:teuthology.orchestra.run.smithi096.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [356 kB] 2020-11-28T21:40:46.359 INFO:teuthology.orchestra.run.smithi096.stdout:Fetched 11.4 MB in 2s (6,345 kB/s) 2020-11-28T21:40:47.947 INFO:teuthology.orchestra.run.smithi096.stdout:Reading package lists... 2020-11-28T21:40:47.992 INFO:teuthology.orchestra.run.smithi096:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2020-11-28T21:40:48.067 INFO:teuthology.orchestra.run.smithi096.stdout:Reading package lists... 2020-11-28T21:40:48.304 INFO:teuthology.orchestra.run.smithi096.stdout:Building dependency tree... 2020-11-28T21:40:48.304 INFO:teuthology.orchestra.run.smithi096.stdout:Reading state information... 2020-11-28T21:40:48.383 INFO:teuthology.orchestra.run.smithi096.stderr:E: Unable to locate package linux-image-current-generic 2020-11-28T21:40:48.384 DEBUG:teuthology.orchestra.run:got remote process result: 100 2020-11-28T21:40:48.385 INFO:teuthology.orchestra.run.smithi096:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2020-11-28T21:40:48.481 INFO:teuthology.orchestra.run.smithi096.stdout:Reading package lists... 2020-11-28T21:40:48.743 INFO:teuthology.orchestra.run.smithi096.stdout:Building dependency tree... 2020-11-28T21:40:48.744 INFO:teuthology.orchestra.run.smithi096.stdout:Reading state information... 2020-11-28T21:40:48.949 INFO:teuthology.orchestra.run.smithi096.stdout:linux-image-generic is already the newest version (4.15.0.124.111). 2020-11-28T21:40:48.949 INFO:teuthology.orchestra.run.smithi096.stdout:linux-image-generic set to manually installed. 2020-11-28T21:40:48.950 INFO:teuthology.orchestra.run.smithi096.stdout:0 upgraded, 0 newly installed, 0 to remove and 5 not upgraded. 2020-11-28T21:40:48.951 INFO:teuthology.orchestra.run.smithi096:> dpkg -s linux-image-generic 2020-11-28T21:40:48.976 INFO:teuthology.orchestra.run.smithi096.stdout:Package: linux-image-generic 2020-11-28T21:40:48.976 INFO:teuthology.orchestra.run.smithi096.stdout:Status: install ok installed 2020-11-28T21:40:48.976 INFO:teuthology.orchestra.run.smithi096.stdout:Priority: optional 2020-11-28T21:40:48.976 INFO:teuthology.orchestra.run.smithi096.stdout:Section: kernel 2020-11-28T21:40:48.977 INFO:teuthology.orchestra.run.smithi096.stdout:Installed-Size: 16 2020-11-28T21:40:48.977 INFO:teuthology.orchestra.run.smithi096.stdout:Maintainer: Ubuntu Kernel Team 2020-11-28T21:40:48.977 INFO:teuthology.orchestra.run.smithi096.stdout:Architecture: amd64 2020-11-28T21:40:48.977 INFO:teuthology.orchestra.run.smithi096.stdout:Source: linux-meta 2020-11-28T21:40:48.977 INFO:teuthology.orchestra.run.smithi096.stdout:Version: 4.15.0.124.111 2020-11-28T21:40:48.978 INFO:teuthology.orchestra.run.smithi096.stdout:Provides: wireguard-modules (= 1.0.20200611-1ubuntu1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2020-11-28T21:40:48.978 INFO:teuthology.orchestra.run.smithi096.stdout:Depends: linux-image-4.15.0-124-generic, linux-modules-extra-4.15.0-124-generic, linux-firmware, intel-microcode, amd64-microcode 2020-11-28T21:40:48.978 INFO:teuthology.orchestra.run.smithi096.stdout:Recommends: thermald 2020-11-28T21:40:48.978 INFO:teuthology.orchestra.run.smithi096.stdout:Description: Generic Linux kernel image 2020-11-28T21:40:48.979 INFO:teuthology.orchestra.run.smithi096.stdout: This package will always depend on the latest generic kernel image 2020-11-28T21:40:48.979 INFO:teuthology.orchestra.run.smithi096.stdout: available. 2020-11-28T21:40:48.979 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2020-11-28T21:40:48.979 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2020-11-28T21:40:48.980 INFO:teuthology.orchestra.run.smithi162:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-11-28T21:40:49.001 INFO:teuthology.orchestra.run.smithi162.stdout:ttyS1 2020-11-28T21:40:49.021 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2020-11-28T21:40:49.022 INFO:teuthology.orchestra.run.smithi118:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-11-28T21:40:49.040 INFO:teuthology.orchestra.run.smithi118.stdout:ttyS1 2020-11-28T21:40:49.064 INFO:teuthology.task.kernel:Enabling kdb on client.0... 2020-11-28T21:40:49.065 INFO:teuthology.orchestra.run.smithi192:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-11-28T21:40:49.083 INFO:teuthology.orchestra.run.smithi192.stdout:ttyS1 2020-11-28T21:40:49.103 INFO:teuthology.task.kernel:Enabling kdb on client.1... 2020-11-28T21:40:49.104 INFO:teuthology.orchestra.run.smithi096:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-11-28T21:40:49.124 INFO:teuthology.orchestra.run.smithi096.stdout:ttyS1 2020-11-28T21:40:49.143 INFO:teuthology.run_tasks:Running task internal.base... 2020-11-28T21:40:49.160 INFO:teuthology.task.internal:Creating test directory... 2020-11-28T21:40:49.160 INFO:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-11-28T21:40:49.188 INFO:teuthology.orchestra.run.smithi118:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-11-28T21:40:49.192 INFO:teuthology.orchestra.run.smithi162:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-11-28T21:40:49.194 INFO:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-11-28T21:40:49.200 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2020-11-28T21:40:49.208 INFO:teuthology.run_tasks:Running task internal.archive... 2020-11-28T21:40:49.226 INFO:teuthology.task.internal:Creating archive directory... 2020-11-28T21:40:49.227 INFO:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-11-28T21:40:49.235 INFO:teuthology.orchestra.run.smithi118:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-11-28T21:40:49.239 INFO:teuthology.orchestra.run.smithi162:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-11-28T21:40:49.241 INFO:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-11-28T21:40:49.257 INFO:teuthology.run_tasks:Running task internal.coredump... 2020-11-28T21:40:49.270 INFO:teuthology.task.internal:Enabling coredump saving... 2020-11-28T21:40:49.271 INFO:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-11-28T21:40:49.292 INFO:teuthology.orchestra.run.smithi118:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-11-28T21:40:49.295 INFO:teuthology.orchestra.run.smithi162:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-11-28T21:40:49.297 INFO:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-11-28T21:40:49.308 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-28T21:40:49.311 INFO:teuthology.orchestra.run.smithi118.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-28T21:40:49.313 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-28T21:40:49.316 INFO:teuthology.orchestra.run.smithi162.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-28T21:40:49.320 INFO:teuthology.orchestra.run.smithi118.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-28T21:40:49.323 INFO:teuthology.orchestra.run.smithi162.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-28T21:40:49.324 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-28T21:40:49.331 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-28T21:40:49.332 INFO:teuthology.run_tasks:Running task internal.sudo... 2020-11-28T21:40:49.344 INFO:teuthology.task.internal:Configuring sudo... 2020-11-28T21:40:49.345 INFO:teuthology.orchestra.run.smithi096:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-11-28T21:40:49.361 INFO:teuthology.orchestra.run.smithi118:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-11-28T21:40:49.364 INFO:teuthology.orchestra.run.smithi162:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-11-28T21:40:49.369 INFO:teuthology.orchestra.run.smithi192:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-11-28T21:40:49.389 INFO:teuthology.run_tasks:Running task internal.syslog... 2020-11-28T21:40:49.406 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2020-11-28T21:40:49.407 INFO:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-11-28T21:40:49.416 INFO:teuthology.orchestra.run.smithi118:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-11-28T21:40:49.421 INFO:teuthology.orchestra.run.smithi162:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-11-28T21:40:49.426 INFO:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-11-28T21:40:49.436 INFO:teuthology.orchestra.run.smithi162:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-11-28T21:40:49.478 INFO:teuthology.orchestra.run.smithi162:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-11-28T21:40:49.530 INFO:teuthology.orchestra.run.smithi162:> set -ex 2020-11-28T21:40:49.531 INFO:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-28T21:40:49.576 INFO:teuthology.orchestra.run.smithi162.stderr:+ sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-28T21:40:49.595 INFO:teuthology.orchestra.run.smithi162.stderr:0+1 records in 2020-11-28T21:40:49.595 INFO:teuthology.orchestra.run.smithi162.stderr:0+1 records out 2020-11-28T21:40:49.595 INFO:teuthology.orchestra.run.smithi162.stderr:152 bytes copied, 3.4053e-05 s, 4.5 MB/s 2020-11-28T21:40:49.596 INFO:teuthology.orchestra.run.smithi118:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-11-28T21:40:49.603 INFO:teuthology.orchestra.run.smithi118:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-11-28T21:40:49.652 INFO:teuthology.orchestra.run.smithi118:> set -ex 2020-11-28T21:40:49.653 INFO:teuthology.orchestra.run.smithi118:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-28T21:40:49.698 INFO:teuthology.orchestra.run.smithi118.stderr:+ sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-28T21:40:49.728 INFO:teuthology.orchestra.run.smithi118.stderr:0+1 records in 2020-11-28T21:40:49.729 INFO:teuthology.orchestra.run.smithi118.stderr:0+1 records out 2020-11-28T21:40:49.729 INFO:teuthology.orchestra.run.smithi118.stderr:152 bytes copied, 3.244e-05 s, 4.7 MB/s 2020-11-28T21:40:49.730 INFO:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-11-28T21:40:49.737 INFO:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-11-28T21:40:49.784 INFO:teuthology.orchestra.run.smithi192:> set -ex 2020-11-28T21:40:49.784 INFO:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-28T21:40:49.829 INFO:teuthology.orchestra.run.smithi192.stderr:+ sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-28T21:40:49.854 INFO:teuthology.orchestra.run.smithi192.stderr:0+1 records in 2020-11-28T21:40:49.855 INFO:teuthology.orchestra.run.smithi192.stderr:0+1 records out 2020-11-28T21:40:49.855 INFO:teuthology.orchestra.run.smithi192.stderr:152 bytes copied, 3.549e-05 s, 4.3 MB/s 2020-11-28T21:40:49.856 INFO:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-11-28T21:40:49.863 INFO:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-11-28T21:40:49.913 INFO:teuthology.orchestra.run.smithi096:> set -ex 2020-11-28T21:40:49.913 INFO:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-28T21:40:49.958 INFO:teuthology.orchestra.run.smithi096.stderr:+ sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-28T21:40:49.977 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records in 2020-11-28T21:40:49.977 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records out 2020-11-28T21:40:49.977 INFO:teuthology.orchestra.run.smithi096.stderr:152 bytes copied, 3.1476e-05 s, 4.8 MB/s 2020-11-28T21:40:49.978 INFO:teuthology.orchestra.run.smithi096:> sudo service rsyslog restart 2020-11-28T21:40:50.024 INFO:teuthology.orchestra.run.smithi118:> sudo service rsyslog restart 2020-11-28T21:40:50.026 INFO:teuthology.orchestra.run.smithi162:> sudo service rsyslog restart 2020-11-28T21:40:50.028 INFO:teuthology.orchestra.run.smithi192:> sudo service rsyslog restart 2020-11-28T21:40:50.070 INFO:teuthology.run_tasks:Running task internal.timer... 2020-11-28T21:40:50.077 INFO:teuthology.task.internal:Starting timer... 2020-11-28T21:40:50.078 INFO:teuthology.run_tasks:Running task pcp... 2020-11-28T21:40:50.088 INFO:teuthology.run_tasks:Running task selinux... 2020-11-28T21:40:50.098 DEBUG:teuthology.task.selinux:Excluding smithi162: OS 'ubuntu' does not support SELinux 2020-11-28T21:40:50.099 DEBUG:teuthology.task.selinux:Excluding smithi118: OS 'ubuntu' does not support SELinux 2020-11-28T21:40:50.099 DEBUG:teuthology.task.selinux:Excluding smithi192: OS 'ubuntu' does not support SELinux 2020-11-28T21:40:50.100 DEBUG:teuthology.task.selinux:Excluding smithi096: OS 'ubuntu' does not support SELinux 2020-11-28T21:40:50.100 DEBUG:teuthology.task.selinux:Getting current SELinux state 2020-11-28T21:40:50.100 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2020-11-28T21:40:50.100 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2020-11-28T21:40:50.101 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2020-11-28T21:40:50.111 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master was just updated; assuming it is current 2020-11-28T21:40:50.112 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master to branch origin/master 2020-11-28T21:40:50.124 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}] 2020-11-28T21:40:50.126 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi162.front.sepia.ceph.com,smithi118.front.sepia.ceph.com,smithi192.front.sepia.ceph.com,smithi096.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml 2020-11-28T21:40:50.466 INFO:teuthology.task.ansible.out:Using /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/ansible.cfg as config file 2020-11-28T21:40:51.336 INFO:teuthology.task.ansible.out:[WARNING]: file /home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/roles/testnode/tasks/setup-opensuse.yml is empty and had no tasks to include 2020-11-28T21:40:53.905 INFO:teuthology.task.ansible.out: PLAY [all] ********************************************************************* 2020-11-28T21:40:53.929 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:53 +0000 (0:00:00.073) 0:00:00.073 ***** 2020-11-28T21:40:53.943 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:53 +0000 (0:00:00.014) 0:00:00.088 ***** 2020-11-28T21:40:53.960 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:53 +0000 (0:00:00.016) 0:00:00.104 ***** 2020-11-28T21:40:53.976 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:53 +0000 (0:00:00.016) 0:00:00.121 ***** 2020-11-28T21:40:55.480 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi096.front.sepia.ceph.com] 2020-11-28T21:40:55.534 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] 2020-11-28T21:40:55.577 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:55 +0000 (0:00:01.601) 0:00:01.722 ***** 2020-11-28T21:40:55.617 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] 2020-11-28T21:40:55.682 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:55 +0000 (0:00:00.104) 0:00:01.827 ***** 2020-11-28T21:40:55.692 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] 2020-11-28T21:40:55.760 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:55 +0000 (0:00:00.078) 0:00:01.905 ***** 2020-11-28T21:40:55.801 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:55 +0000 (0:00:00.041) 0:00:01.946 ***** 2020-11-28T21:40:55.818 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2020-11-28T21:40:55.820 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2020-11-28T21:40:55.855 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:55 +0000 (0:00:00.053) 0:00:02.000 ***** 2020-11-28T21:40:55.863 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2020-11-28T21:40:55.900 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:55 +0000 (0:00:00.045) 0:00:02.045 ***** 2020-11-28T21:40:55.907 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2020-11-28T21:40:55.944 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:55 +0000 (0:00:00.043) 0:00:02.089 ***** 2020-11-28T21:40:55.951 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2020-11-28T21:40:55.988 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:55 +0000 (0:00:00.043) 0:00:02.132 ***** 2020-11-28T21:40:56.295 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** changed: [smithi162.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} changed: [smithi192.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} changed: [smithi096.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2020-11-28T21:40:56.332 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:56 +0000 (0:00:00.344) 0:00:02.476 ***** 2020-11-28T21:40:56.372 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:56 +0000 (0:00:00.038) 0:00:02.515 ***** 2020-11-28T21:40:56.405 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:56 +0000 (0:00:00.034) 0:00:02.550 ***** 2020-11-28T21:40:56.417 INFO:teuthology.task.ansible.out:changed: [smithi118.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2020-11-28T21:40:56.459 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:56 +0000 (0:00:00.051) 0:00:02.602 ***** 2020-11-28T21:40:56.648 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** 2020-11-28T21:40:56.650 INFO:teuthology.task.ansible.out:changed: [smithi192.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.062119", "end": "2020-11-28 21:40:56.972651", "rc": 0, "start": "2020-11-28 21:40:56.910532", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} 2020-11-28T21:40:56.691 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:56 +0000 (0:00:00.234) 0:00:02.836 ***** 2020-11-28T21:40:56.698 INFO:teuthology.task.ansible.out:changed: [smithi162.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.074350", "end": "2020-11-28 21:40:57.069471", "rc": 0, "start": "2020-11-28 21:40:56.995121", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} 2020-11-28T21:40:56.701 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.064353", "end": "2020-11-28 21:40:57.058988", "rc": 0, "start": "2020-11-28 21:40:56.994635", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} changed: [smithi118.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.073541", "end": "2020-11-28 21:40:56.753443", "rc": 0, "start": "2020-11-28 21:40:56.679902", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} 2020-11-28T21:40:56.741 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:56 +0000 (0:00:00.049) 0:00:02.886 ***** 2020-11-28T21:40:56.777 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:56 +0000 (0:00:00.035) 0:00:02.921 ***** 2020-11-28T21:40:56.816 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:56 +0000 (0:00:00.038) 0:00:02.960 ***** 2020-11-28T21:40:56.932 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** 2020-11-28T21:40:56.934 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-28T21:40:56.975 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:56 +0000 (0:00:00.159) 0:00:03.120 ***** 2020-11-28T21:40:56.981 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-28T21:40:56.983 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-28T21:40:56.985 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-28T21:40:57.026 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:57 +0000 (0:00:00.050) 0:00:03.170 ***** 2020-11-28T21:40:57.060 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:57 +0000 (0:00:00.034) 0:00:03.205 ***** 2020-11-28T21:40:57.104 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:57 +0000 (0:00:00.043) 0:00:03.249 ***** 2020-11-28T21:40:57.628 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** 2020-11-28T21:40:57.630 INFO:teuthology.task.ansible.out:changed: [smithi192.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1606599656.9876602-221304888209987/source", "state": "file", "uid": 0} 2020-11-28T21:40:57.673 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:57 +0000 (0:00:00.568) 0:00:03.817 ***** 2020-11-28T21:40:57.679 INFO:teuthology.task.ansible.out:changed: [smithi162.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1606599657.1232462-241152730437341/source", "state": "file", "uid": 0} 2020-11-28T21:40:57.680 INFO:teuthology.task.ansible.out:changed: [smithi118.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1606599657.0759223-22946535036928/source", "state": "file", "uid": 0} 2020-11-28T21:40:57.682 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1606599657.039624-204053157406921/source", "state": "file", "uid": 0} 2020-11-28T21:40:57.719 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:57 +0000 (0:00:00.046) 0:00:03.864 ***** 2020-11-28T21:40:57.753 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:57 +0000 (0:00:00.033) 0:00:03.898 ***** 2020-11-28T21:40:57.787 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:57 +0000 (0:00:00.034) 0:00:03.932 ***** 2020-11-28T21:40:57.985 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Add authorized keys for the ansible user.] ************* 2020-11-28T21:40:57.987 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2020-11-28T21:40:57.989 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} ok: [smithi192.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail 2020-11-28T21:40:57.990 INFO:teuthology.task.ansible.out:.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@ 2020-11-28T21:40:57.992 INFO:teuthology.task.ansible.out:aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} ok: [smithi162.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEz 2020-11-28T21:40:57.993 INFO:teuthology.task.ansible.out:IKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2020-11-28T21:40:58.077 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2020-11-28T21:40:58.079 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2020-11-28T21:40:58.247 INFO:teuthology.task.ansible.out: PLAY [teuthology] ************************************************************** skipping: no hosts matched 2020-11-28T21:40:58.269 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2020-11-28T21:40:58.425 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:58 +0000 (0:00:00.638) 0:00:04.570 ***** 2020-11-28T21:40:58.461 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:58 +0000 (0:00:00.035) 0:00:04.606 ***** 2020-11-28T21:40:58.496 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:58 +0000 (0:00:00.034) 0:00:04.641 ***** 2020-11-28T21:40:58.530 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:58 +0000 (0:00:00.034) 0:00:04.675 ***** 2020-11-28T21:40:59.105 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* 2020-11-28T21:40:59.106 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] 2020-11-28T21:40:59.157 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] 2020-11-28T21:40:59.209 INFO:teuthology.task.ansible.out:[WARNING]: Using run_once with the free strategy is not currently supported. This task will still be executed for every host in the inventory list. 2020-11-28T21:40:59.211 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:59 +0000 (0:00:00.678) 0:00:05.354 ***** 2020-11-28T21:40:59.250 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] 2020-11-28T21:40:59.265 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] 2020-11-28T21:40:59.314 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:59 +0000 (0:00:00.104) 0:00:05.458 ***** 2020-11-28T21:40:59.434 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:59 +0000 (0:00:00.116) 0:00:05.574 ***** 2020-11-28T21:40:59.473 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:59 +0000 (0:00:00.043) 0:00:05.618 ***** 2020-11-28T21:40:59.575 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** changed: [smithi096.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.182060", "end": "2020-11-28 21:40:59.547234", "rc": 0, "start": "2020-11-28 21:40:59.365174", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-11-28T21:40:59.625 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:59 +0000 (0:00:00.151) 0:00:05.770 ***** 2020-11-28T21:40:59.684 INFO:teuthology.task.ansible.out:changed: [smithi118.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.180580", "end": "2020-11-28 21:40:59.655111", "rc": 0, "start": "2020-11-28 21:40:59.474531", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-11-28T21:40:59.723 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:59 +0000 (0:00:00.097) 0:00:05.867 ***** 2020-11-28T21:40:59.782 INFO:teuthology.task.ansible.out:changed: [smithi162.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.163807", "end": "2020-11-28 21:40:59.757854", "rc": 0, "start": "2020-11-28 21:40:59.594047", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-11-28T21:40:59.784 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.022679", "end": "2020-11-28 21:41:00.176467", "failed_when_result": false, "rc": 0, "start": "2020-11-28 21:41:00.153788", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-11-28T21:40:59.824 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:59 +0000 (0:00:00.101) 0:00:05.969 ***** 2020-11-28T21:40:59.862 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:59 +0000 (0:00:00.037) 0:00:06.007 ***** 2020-11-28T21:40:59.869 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2020-11-28T21:40:59.871 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"ansible_facts": {"perform_users_role": false}, "changed": false} TASK [users : Check keys_repo HEAD sha1] *************************************** changed: [smithi192.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.164265", "end": "2020-11-28 21:40:59.834527", "rc": 0, "start": "2020-11-28 21:40:59.670262", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-11-28T21:40:59.918 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:59 +0000 (0:00:00.053) 0:00:06.060 ***** 2020-11-28T21:40:59.951 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:59 +0000 (0:00:00.035) 0:00:06.096 ***** 2020-11-28T21:40:59.957 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2020-11-28T21:40:59.958 INFO:teuthology.task.ansible.out:changed: [smithi118.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.017534", "end": "2020-11-28 21:40:59.963328", "failed_when_result": false, "rc": 0, "start": "2020-11-28 21:40:59.945794", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-11-28T21:40:59.959 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:00.001 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:40:59 +0000 (0:00:00.049) 0:00:06.145 ***** 2020-11-28T21:41:00.035 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:00 +0000 (0:00:00.035) 0:00:06.180 ***** 2020-11-28T21:41:00.042 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2020-11-28T21:41:00.044 INFO:teuthology.task.ansible.out:changed: [smithi162.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.016656", "end": "2020-11-28 21:41:00.420879", "failed_when_result": false, "rc": 0, "start": "2020-11-28 21:41:00.404223", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} TASK [users : Remove managed_admin_users from managed_users] ******************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:00.085 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:00 +0000 (0:00:00.049) 0:00:06.230 ***** 2020-11-28T21:41:00.120 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:00 +0000 (0:00:00.035) 0:00:06.265 ***** 2020-11-28T21:41:00.130 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** ok: [smithi118.front.sepia.ceph.com] => {"ansible_facts": {"perform_users_role": false}, "changed": false} TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** changed: [smithi192.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.011519", "end": "2020-11-28 21:41:00.428245", "failed_when_result": false, "rc": 0, "start": "2020-11-28 21:41:00.416726", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} TASK [users : Filter the managed_users list] *********************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:00.178 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:00 +0000 (0:00:00.057) 0:00:06.323 ***** 2020-11-28T21:41:00.215 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:00 +0000 (0:00:00.036) 0:00:06.359 ***** 2020-11-28T21:41:00.247 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:00 +0000 (0:00:00.032) 0:00:06.392 ***** 2020-11-28T21:41:00.253 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2020-11-28T21:41:00.254 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => {"ansible_facts": {"perform_users_role": false}, "changed": false} TASK [users : Filter the managed_admin_users list] ***************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Merge extra_admin_users into managed_admin_users] **************** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:00.292 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:00 +0000 (0:00:00.044) 0:00:06.437 ***** 2020-11-28T21:41:00.333 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:00 +0000 (0:00:00.040) 0:00:06.478 ***** 2020-11-28T21:41:00.367 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:00 +0000 (0:00:00.033) 0:00:06.511 ***** 2020-11-28T21:41:00.373 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2020-11-28T21:41:00.375 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"ansible_facts": {"perform_users_role": false}, "changed": false} TASK [users : Set UID_MIN to 1001] ********************************************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Remove managed_admin_users from managed_users] ******************* skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:00.412 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:00 +0000 (0:00:00.044) 0:00:06.556 ***** 2020-11-28T21:41:00.446 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:00 +0000 (0:00:00.034) 0:00:06.590 ***** 2020-11-28T21:41:00.485 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:00 +0000 (0:00:00.039) 0:00:06.630 ***** 2020-11-28T21:41:00.491 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2020-11-28T21:41:00.492 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:00.494 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:00.494 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:00.495 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-11-28T21:41:00.495 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:00.495 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c7 2020-11-28T21:41:00.497 INFO:teuthology.task.ansible.out:90bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6 2020-11-28T21:41:00.498 INFO:teuthology.task.ansible.out:\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:00.533 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:00 +0000 (0:00:00.047) 0:00:06.677 ***** 2020-11-28T21:41:00.578 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:00 +0000 (0:00:00.044) 0:00:06.722 ***** 2020-11-28T21:41:00.588 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:00.589 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2020-11-28T21:41:00.590 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:00.590 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-11-28T21:41:00.590 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:00.591 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:00.592 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f30 2020-11-28T21:41:00.594 INFO:teuthology.task.ansible.out:09adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6I 2020-11-28T21:41:00.595 INFO:teuthology.task.ansible.out:nynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8 2020-11-28T21:41:00.597 INFO:teuthology.task.ansible.out:595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} TASK [users : Filter the managed_admin_users list] ***************************** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317a 2020-11-28T21:41:00.598 INFO:teuthology.task.ansible.out:e89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:00.635 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:00 +0000 (0:00:00.056) 0:00:06.779 ***** 2020-11-28T21:41:00.670 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:00 +0000 (0:00:00.035) 0:00:06.815 ***** 2020-11-28T21:41:00.676 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:00.677 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:00.679 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:00.679 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:00.680 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users wi 2020-11-28T21:41:00.681 INFO:teuthology.task.ansible.out:th sudo access.] ************************ skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G sh 2020-11-28T21:41:00.683 INFO:teuthology.task.ansible.out:isK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:00.683 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.co 2020-11-28T21:41:00.685 INFO:teuthology.task.ansible.out:m] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reaso 2020-11-28T21:41:00.686 INFO:teuthology.task.ansible.out:n": "Conditional result was False"} TASK [users : Set UID_MIN to 1001] ********************************************* skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e36 2020-11-28T21:41:00.688 INFO:teuthology.task.ansible.out:08"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:00.725 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:00 +0000 (0:00:00.054) 0:00:06.870 ***** 2020-11-28T21:41:00.761 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:00 +0000 (0:00:00.034) 0:00:06.904 ***** 2020-11-28T21:41:00.767 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:00.769 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} TASK [users : Remove managed_admin_users from managed_users] ******************* skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:00.769 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:00.770 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb 2020-11-28T21:41:00.772 INFO:teuthology.task.ansible.out:O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'mwatts', 2020-11-28T21:41:00.773 INFO:teuthology.task.ansible.out: 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:00.774 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat. 2020-11-28T21:41:00.776 INFO:teuthology.task.ansible.out:com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c 2020-11-28T21:41:00.778 INFO:teuthology.task.ansible.out:7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:00.778 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip 2020-11-28T21:41:00.780 INFO:teuthology.task.ansible.out:_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:00.782 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ 2020-11-28T21:41:00.783 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:00.826 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:00 +0000 (0:00:00.066) 0:00:06.971 ***** 2020-11-28T21:41:00.865 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:00 +0000 (0:00:00.038) 0:00:07.010 ***** 2020-11-28T21:41:00.871 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:00.872 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:00.874 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:00.875 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:00.878 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:00.879 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT 2020-11-28T21:41:00.881 INFO:teuthology.task.ansible.out:7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": " 2020-11-28T21:41:00.883 INFO:teuthology.task.ansible.out:mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was Fal 2020-11-28T21:41:00.884 INFO:teuthology.task.ansible.out:se"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e 2020-11-28T21:41:00.886 INFO:teuthology.task.ansible.out:4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} TASK [users : Filter the managed_admin_users list] ***************************** skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin 2020-11-28T21:41:00.887 INFO:teuthology.task.ansible.out: users with sudo access.] ************************ skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "ya 2020-11-28T21:41:00.889 INFO:teuthology.task.ansible.out:arit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0 2020-11-28T21:41:00.890 INFO:teuthology.task.ansible.out:g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:00.928 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:00 +0000 (0:00:00.063) 0:00:07.073 ***** 2020-11-28T21:41:00.934 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:00.936 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:00.937 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:00.938 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'pnawracay', 2020-11-28T21:41:00.942 INFO:teuthology.task.ansible.out:'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} TASK [users : Filter the managed_users list] *********************************** skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307 2020-11-28T21:41:00.944 INFO:teuthology.task.ansible.out:bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108 2020-11-28T21:41:00.945 INFO:teuthology.task.ansible.out:629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovp 2020-11-28T21:41:00.947 INFO:teuthology.task.ansible.out:n": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': ' 2020-11-28T21:41:00.952 INFO:teuthology.task.ansible.out:mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJVi 2020-11-28T21:41:00.954 INFO:teuthology.task.ansible.out:O6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\ 2020-11-28T21:41:00.955 INFO:teuthology.task.ansible.out:n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:00.989 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:00 +0000 (0:00:00.061) 0:00:07.134 ***** 2020-11-28T21:41:01.001 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.003 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.004 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.006 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.007 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} TASK [users : Set UID_MIN to 1001] ********************************************* skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.009 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.010 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.012 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.013 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.015 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.016 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.017 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.018 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.019 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.021 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.022 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.024 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.026 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.028 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.028 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.029 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.031 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.032 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.034 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.034 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.035 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.037 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.039 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.039 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.097 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:01 +0000 (0:00:00.107) 0:00:07.241 ***** 2020-11-28T21:41:01.139 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:01 +0000 (0:00:00.042) 0:00:07.284 ***** 2020-11-28T21:41:01.155 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.156 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.163 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 2020-11-28T21:41:01.164 INFO:teuthology.task.ansible.out:'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.165 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.166 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.167 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.168 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.169 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.170 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.172 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.173 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.174 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.175 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.176 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.176 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.178 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.179 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.180 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.180 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.182 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.183 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.185 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.185 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.185 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.186 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939b 2020-11-28T21:41:01.187 INFO:teuthology.task.ansible.out:fdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.188 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w a 2020-11-28T21:41:01.189 INFO:teuthology.task.ansible.out:b1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.191 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4 2020-11-28T21:41:01.193 INFO:teuthology.task.ansible.out:XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} TASK [users : Set UID_MIN to 1001] ********************************************* skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 's 2020-11-28T21:41:01.194 INFO:teuthology.task.ansible.out:krah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.196 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "i 2020-11-28T21:41:01.198 INFO:teuthology.task.ansible.out:tem": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c 2020-11-28T21:41:01.199 INFO:teuthology.task.ansible.out:2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.236 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:01 +0000 (0:00:00.096) 0:00:07.381 ***** 2020-11-28T21:41:01.242 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.245 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.246 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe6 2020-11-28T21:41:01.249 INFO:teuthology.task.ansible.out:95fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf 2020-11-28T21:41:01.251 INFO:teuthology.task.ansible.out:0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed57 2020-11-28T21:41:01.253 INFO:teuthology.task.ansible.out:0983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d37 2020-11-28T21:41:01.256 INFO:teuthology.task.ansible.out:7e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.261 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.263 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TX 2020-11-28T21:41:01.265 INFO:teuthology.task.ansible.out:iHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.282 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.285 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.324 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:01 +0000 (0:00:00.088) 0:00:07.469 ***** 2020-11-28T21:41:01.332 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.334 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8 2020-11-28T21:41:01.336 INFO:teuthology.task.ansible.out:I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.337 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f91609 2020-11-28T21:41:01.339 INFO:teuthology.task.ansible.out:0f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.340 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.fr 2020-11-28T21:41:01.342 INFO:teuthology.task.ansible.out:ont.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.342 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094 2020-11-28T21:41:01.345 INFO:teuthology.task.ansible.out:d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c0 2020-11-28T21:41:01.347 INFO:teuthology.task.ansible.out:7e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason 2020-11-28T21:41:01.351 INFO:teuthology.task.ansible.out:": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b 2020-11-28T21:41:01.352 INFO:teuthology.task.ansible.out:45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba533 2020-11-28T21:41:01.354 INFO:teuthology.task.ansible.out:97c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.361 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.362 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.363 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c 2020-11-28T21:41:01.365 INFO:teuthology.task.ansible.out:62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.365 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b3 2020-11-28T21:41:01.366 INFO:teuthology.task.ansible.out:4f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.368 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.368 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'akr 2020-11-28T21:41:01.370 INFO:teuthology.task.ansible.out:aitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} TASK [users : Create all users without sudo access.] *************************** 2020-11-28T21:41:01.371 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.373 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.374 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.376 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.377 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.378 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.379 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.379 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnja 2020-11-28T21:41:01.381 INFO:teuthology.task.ansible.out:ceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.425 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:01 +0000 (0:00:00.100) 0:00:07.570 ***** 2020-11-28T21:41:01.432 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.434 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.434 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'aem 2020-11-28T21:41:01.439 INFO:teuthology.task.ansible.out:erson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQAB 2020-11-28T21:41:01.441 INFO:teuthology.task.ansible.out:AAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} skipping: [smithi162 2020-11-28T21:41:01.442 INFO:teuthology.task.ansible.out:.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZp 2020-11-28T21:41:01.444 INFO:teuthology.task.ansible.out:JZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip 2020-11-28T21:41:01.446 INFO:teuthology.task.ansible.out:_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf 2020-11-28T21:41:01.447 INFO:teuthology.task.ansible.out:@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8d 2020-11-28T21:41:01.449 INFO:teuthology.task.ansible.out:de844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) 2020-11-28T21:41:01.450 INFO:teuthology.task.ansible.out: => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.461 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.463 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.465 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.466 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.468 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.524 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:01 +0000 (0:00:00.099) 0:00:07.669 ***** 2020-11-28T21:41:01.533 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.534 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.536 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.537 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.538 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.538 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.539 INFO:teuthology.task.ansible.out:skipping: [smithi16 2020-11-28T21:41:01.540 INFO:teuthology.task.ansible.out:2.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.541 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph 2020-11-28T21:41:01.542 INFO:teuthology.task.ansible.out:.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": 2020-11-28T21:41:01.546 INFO:teuthology.task.ansible.out: "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat. 2020-11-28T21:41:01.548 INFO:teuthology.task.ansible.out:com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehud 2020-11-28T21:41:01.549 INFO:teuthology.task.ansible.out:asa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.551 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.551 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.552 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.553 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.554 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.557 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.559 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.560 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDC 2020-11-28T21:41:01.561 INFO:teuthology.task.ansible.out:w ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "ite 2020-11-28T21:41:01.563 INFO:teuthology.task.ansible.out:m", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.581 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.582 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.583 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be 2020-11-28T21:41:01.584 INFO:teuthology.task.ansible.out:94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.626 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:01 +0000 (0:00:00.101) 0:00:07.771 ***** 2020-11-28T21:41:01.667 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:01 +0000 (0:00:00.041) 0:00:07.812 ***** 2020-11-28T21:41:01.674 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.677 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.677 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw 2020-11-28T21:41:01.679 INFO:teuthology.task.ansible.out: a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.681 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.681 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.681 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7 2020-11-28T21:41:01.684 INFO:teuthology.task.ansible.out:wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.684 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuB 2020-11-28T21:41:01.686 INFO:teuthology.task.ansible.out:YwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.687 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b 2020-11-28T21:41:01.688 INFO:teuthology.task.ansible.out:44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.689 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.690 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d 2020-11-28T21:41:01.692 INFO:teuthology.task.ansible.out:57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys using the keys repo] ********************** 2020-11-28T21:41:01.692 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ 2020-11-28T21:41:01.693 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vsha 2020-11-28T21:41:01.695 INFO:teuthology.task.ansible.out:nkar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11 2020-11-28T21:41:01.697 INFO:teuthology.task.ansible.out:aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip 2020-11-28T21:41:01.699 INFO:teuthology.task.ansible.out:_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} TASK [users : Create all users without sudo access.] *************************** TASK [users : Create all admin users with sudo access.] ************************ 2020-11-28T21:41:01.699 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => 2020-11-28T21:41:01.701 INFO:teuthology.task.ansible.out: (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.738 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:01 +0000 (0:00:00.070) 0:00:07.883 ***** 2020-11-28T21:41:01.778 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:01 +0000 (0:00:00.039) 0:00:07.922 ***** 2020-11-28T21:41:01.784 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.785 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.786 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.787 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.787 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.789 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.789 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.791 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.793 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.793 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.795 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.795 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.796 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": 2020-11-28T21:41:01.797 INFO:teuthology.task.ansible.out: {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.798 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.799 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.800 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.800 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-11-28T21:41:01.801 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ 2020-11-28T21:41:01.801 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was Fals 2020-11-28T21:41:01.803 INFO:teuthology.task.ansible.out:e"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.804 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.807 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.807 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.810 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.812 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.857 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:01 +0000 (0:00:00.079) 0:00:08.002 ***** 2020-11-28T21:41:01.865 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.866 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.867 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.868 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.869 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.870 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.871 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.872 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", 2020-11-28T21:41:01.874 INFO:teuthology.task.ansible.out:"changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.916 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:01 +0000 (0:00:00.058) 0:00:08.061 ***** 2020-11-28T21:41:01.923 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.926 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.928 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.930 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.932 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.934 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.935 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.935 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.938 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.939 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.941 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.942 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.942 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.943 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8 2020-11-28T21:41:01.945 INFO:teuthology.task.ansible.out:yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} TASK [users : Filter the revoked_users list] *********************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.946 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.949 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.952 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.952 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.954 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@s 2020-11-28T21:41:01.955 INFO:teuthology.task.ansible.out:use Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.957 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (it 2020-11-28T21:41:01.961 INFO:teuthology.task.ansible.out:em={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5k 2020-11-28T21:41:01.963 INFO:teuthology.task.ansible.out:JJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe 2020-11-28T21:41:01.965 INFO:teuthology.task.ansible.out:16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.986 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.988 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ 2020-11-28T21:41:01.990 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.990 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.991 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:01.991 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "z 2020-11-28T21:41:01.993 INFO:teuthology.task.ansible.out:dover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.055 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:02 +0000 (0:00:00.138) 0:00:08.200 ***** 2020-11-28T21:41:02.112 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:02 +0000 (0:00:00.056) 0:00:08.256 ***** 2020-11-28T21:41:02.181 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:02 +0000 (0:00:00.069) 0:00:08.325 ***** 2020-11-28T21:41:02.183 INFO:teuthology.task.ansible.out: 2020-11-28T21:41:02.194 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.197 INFO:teuthology.task.ansible.out: skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.199 INFO:teuthology.task.ansible.out: skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.201 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.203 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.204 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.206 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.208 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.210 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.212 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.214 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi096.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.216 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.218 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.220 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi096.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.220 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.221 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-11-28T21:41:02.222 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.222 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.223 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping 2020-11-28T21:41:02.228 INFO:teuthology.task.ansible.out:: [smithi096.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.229 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi096.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys using the keys repo] ********************** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi096.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/ 2020-11-28T21:41:02.230 INFO:teuthology.task.ansible.out:AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi096.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked user 2020-11-28T21:41:02.233 INFO:teuthology.task.ansible.out:s] ******************************************** skipping: [smithi096.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi096.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.234 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi096.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] *** 2020-11-28T21:41:02.235 INFO:teuthology.task.ansible.out:********************* skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} TASK [users : Create all users without sudo access.] *************************** 2020-11-28T21:41:02.326 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:02 +0000 (0:00:00.145) 0:00:08.471 ***** 2020-11-28T21:41:02.382 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:02 +0000 (0:00:00.055) 0:00:08.526 ***** 2020-11-28T21:41:02.392 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-11-28T21:41:02.394 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.396 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.397 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.398 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-11-28T21:41:02.398 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.398 INFO:teuthology.task.ansible.out: 2020-11-28T21:41:02.400 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.400 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi096.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.402 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ 2020-11-28T21:41:02.402 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.404 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi096.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.405 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ 2020-11-28T21:41:02.406 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.407 INFO:teuthology.task.ansible.out: 2020-11-28T21:41:02.409 INFO:teuthology.task.ansible.out:TASK [users : Remove revoked users] ******************************************** skipping: [smithi096.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ 2020-11-28T21:41:02.409 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.411 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-11-28T21:41:02.413 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.413 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.415 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** 2020-11-28T21:41:02.415 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.417 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.419 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-11-28T21:41:02.423 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.455 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-11-28T21:41:02.457 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.457 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi096.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.457 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.459 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.459 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.460 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.460 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.461 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.462 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.462 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.467 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "skip_reason": "Conditional result was False"} TASK [users : Merge managed_users and managed_admin users] ********************* skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked us 2020-11-28T21:41:02.469 INFO:teuthology.task.ansible.out:ers] ******************************************** skipping: [smithi096.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.525 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:02 +0000 (0:00:00.143) 0:00:08.670 ***** 2020-11-28T21:41:02.560 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:02 +0000 (0:00:00.034) 0:00:08.704 ***** 2020-11-28T21:41:02.599 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:02 +0000 (0:00:00.039) 0:00:08.744 ***** 2020-11-28T21:41:02.635 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:02 +0000 (0:00:00.035) 0:00:08.780 ***** 2020-11-28T21:41:02.641 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** 2020-11-28T21:41:02.643 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Clone the keys repo] ********************************************* skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.697 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:02 +0000 (0:00:00.061) 0:00:08.841 ***** 2020-11-28T21:41:02.742 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:02 +0000 (0:00:00.045) 0:00:08.886 ***** 2020-11-28T21:41:02.749 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2020-11-28T21:41:02.751 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi118.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.751 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.752 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.793 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:02 +0000 (0:00:00.051) 0:00:08.938 ***** 2020-11-28T21:41:02.800 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.802 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.803 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.804 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi118.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.805 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.805 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.805 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.806 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.844 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:02 +0000 (0:00:00.051) 0:00:08.989 ***** 2020-11-28T21:41:02.850 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.851 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "skip_reason": "Conditional result was False"} TASK [users : Merge managed_users and managed_admin users] ********************* 2020-11-28T21:41:02.852 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.852 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi118.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.853 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.915 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:02 +0000 (0:00:00.070) 0:00:09.059 ***** 2020-11-28T21:41:02.922 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.924 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.924 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.925 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.926 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.928 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi118.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.928 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.929 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.929 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.930 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.930 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.931 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.931 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", 2020-11-28T21:41:02.933 INFO:teuthology.task.ansible.out: "changed": false, "item": "taco", "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.976 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:02 +0000 (0:00:00.061) 0:00:09.121 ***** 2020-11-28T21:41:02.985 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.986 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.988 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.989 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.990 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi118.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:02.991 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.035 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:03 +0000 (0:00:00.058) 0:00:09.179 ***** 2020-11-28T21:41:03.067 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:03 +0000 (0:00:00.032) 0:00:09.212 ***** 2020-11-28T21:41:03.080 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** 2020-11-28T21:41:03.082 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.118 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:03 +0000 (0:00:00.051) 0:00:09.263 ***** 2020-11-28T21:41:03.124 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2020-11-28T21:41:03.126 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.163 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:03 +0000 (0:00:00.044) 0:00:09.307 ***** 2020-11-28T21:41:03.168 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-11-28T21:41:03.170 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "skip_reason": "Conditional result was False"} TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2020-11-28T21:41:03.171 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "eefb312a9cf82af5b9a46fb88fa640babe5fe20f", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/keys-repo-sha1", "size": 40, "state": "file", "uid": 0} 2020-11-28T21:41:03.171 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi162.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.213 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:03 +0000 (0:00:00.049) 0:00:09.357 ***** 2020-11-28T21:41:03.221 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.223 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-11-28T21:41:03.223 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.224 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-11-28T21:41:03.224 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.226 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.226 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.263 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:03 +0000 (0:00:00.050) 0:00:09.408 ***** 2020-11-28T21:41:03.270 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.272 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.272 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.273 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.273 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** ok: [smithi096.front.sepia.ceph.com] => { "msg": "Host smithi096.front.sepia.ceph.com is running Ubuntu 18.04 (bionic)" } 2020-11-28T21:41:03.274 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi162.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.274 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.275 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.276 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.312 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:03 +0000 (0:00:00.048) 0:00:09.457 ***** 2020-11-28T21:41:03.320 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.321 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "skip_reason": "Conditional result was False"} TASK [users : Filter the revoked_users list] *********************************** skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.359 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:03 +0000 (0:00:00.046) 0:00:09.503 ***** 2020-11-28T21:41:03.365 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2020-11-28T21:41:03.367 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"changed": false, "checksum": "eefb312a9cf82af5b9a46fb88fa640babe5fe20f", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/keys-repo-sha1", "size": 40, "state": "file", "uid": 0} TASK [common : Including major version specific variables.] ******************** ok: [smithi096.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2020-11-28T21:41:03.409 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:03 +0000 (0:00:00.048) 0:00:09.552 ***** 2020-11-28T21:41:03.443 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:03 +0000 (0:00:00.035) 0:00:09.587 ***** 2020-11-28T21:41:03.449 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-11-28T21:41:03.451 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.451 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.451 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.452 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.452 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.452 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.453 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.454 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.454 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.454 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.455 INFO:teuthology.task.ansible.out:skipping: [smithi19 2020-11-28T21:41:03.456 INFO:teuthology.task.ansible.out:2.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.457 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.457 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.458 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi192.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.458 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "s 2020-11-28T21:41:03.460 INFO:teuthology.task.ansible.out:kip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.497 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:03 +0000 (0:00:00.053) 0:00:09.641 ***** 2020-11-28T21:41:03.503 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.503 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.504 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.504 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.505 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.505 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.506 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.506 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.506 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2020-11-28T21:41:03.507 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => { "msg": "Host smithi118.front.sepia.ceph.com is running Ubuntu 18.04 (bionic)" } TASK [users : Remove revoked users] ******************************************** 2020-11-28T21:41:03.507 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.508 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.508 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "it 2020-11-28T21:41:03.510 INFO:teuthology.task.ansible.out:em": "fche", "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.549 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:03 +0000 (0:00:00.052) 0:00:09.694 ***** 2020-11-28T21:41:03.587 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:03 +0000 (0:00:00.037) 0:00:09.732 ***** 2020-11-28T21:41:03.593 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2020-11-28T21:41:03.595 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Including major version specific variables.] ******************** ok: [smithi118.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2020-11-28T21:41:03.637 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:03 +0000 (0:00:00.050) 0:00:09.782 ***** 2020-11-28T21:41:03.646 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-11-28T21:41:03.648 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.648 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.648 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.649 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.649 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.650 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.650 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.651 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.651 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=a 2020-11-28T21:41:03.653 INFO:teuthology.task.ansible.out:damyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.800 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:03 +0000 (0:00:00.162) 0:00:09.944 ***** 2020-11-28T21:41:03.807 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.808 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2020-11-28T21:41:03.808 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi192.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.808 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.809 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.810 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.811 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.811 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.813 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.814 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.815 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.815 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.816 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:03.858 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:03 +0000 (0:00:00.058) 0:00:10.003 ***** 2020-11-28T21:41:03.892 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:03 +0000 (0:00:00.033) 0:00:10.037 ***** 2020-11-28T21:41:03.898 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2020-11-28T21:41:03.900 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Wait for reboot in case of systemd workaround] ****************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** ok: [smithi162.front.sepia.ceph.com] => {"changed": false, "checksum": "eefb312a9cf82af5b9a46fb88fa640babe5fe20f", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/keys-repo-sha1", "size": 40, "state": "file", "uid": 0} 2020-11-28T21:41:03.940 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:03 +0000 (0:00:00.047) 0:00:10.084 ***** 2020-11-28T21:41:03.979 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:03 +0000 (0:00:00.039) 0:00:10.124 ***** 2020-11-28T21:41:04.017 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:04 +0000 (0:00:00.038) 0:00:10.162 ***** 2020-11-28T21:41:04.024 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2020-11-28T21:41:04.026 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:04.064 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:04 +0000 (0:00:00.046) 0:00:10.209 ***** 2020-11-28T21:41:04.097 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:04 +0000 (0:00:00.033) 0:00:10.242 ***** 2020-11-28T21:41:04.104 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2020-11-28T21:41:04.105 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => { "msg": "Host smithi162.front.sepia.ceph.com is running Ubuntu 18.04 (bionic)" } TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:04.141 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:04 +0000 (0:00:00.043) 0:00:10.286 ***** 2020-11-28T21:41:04.179 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:04 +0000 (0:00:00.037) 0:00:10.323 ***** 2020-11-28T21:41:04.185 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2020-11-28T21:41:04.188 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Install yum on Fedora 22 and later] ***************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:04.227 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:04 +0000 (0:00:00.047) 0:00:10.371 ***** 2020-11-28T21:41:04.261 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:04 +0000 (0:00:00.034) 0:00:10.406 ***** 2020-11-28T21:41:04.267 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2020-11-28T21:41:04.269 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"changed": false, "checksum": "eefb312a9cf82af5b9a46fb88fa640babe5fe20f", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/keys-repo-sha1", "size": 40, "state": "file", "uid": 0} TASK [common : Including major version specific variables.] ******************** ok: [smithi162.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2020-11-28T21:41:04.270 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:04.305 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:04 +0000 (0:00:00.044) 0:00:10.450 ***** 2020-11-28T21:41:04.352 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:04 +0000 (0:00:00.047) 0:00:10.497 ***** 2020-11-28T21:41:04.396 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:04 +0000 (0:00:00.043) 0:00:10.540 ***** 2020-11-28T21:41:04.402 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2020-11-28T21:41:04.403 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Include Red Hat encrypted variables.] *************************** skipping: [smithi096.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi096.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-28T21:41:04.404 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:04.443 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:04 +0000 (0:00:00.047) 0:00:10.588 ***** 2020-11-28T21:41:04.482 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:04 +0000 (0:00:00.038) 0:00:10.626 ***** 2020-11-28T21:41:04.519 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:04 +0000 (0:00:00.037) 0:00:10.664 ***** 2020-11-28T21:41:04.526 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2020-11-28T21:41:04.527 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => { "msg": "Host smithi192.front.sepia.ceph.com is running Ubuntu 18.04 (bionic)" } TASK [common : Set have_entitlements] ****************************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:04.528 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:04.565 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:04 +0000 (0:00:00.045) 0:00:10.710 ***** 2020-11-28T21:41:04.599 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:04 +0000 (0:00:00.033) 0:00:10.744 ***** 2020-11-28T21:41:04.632 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:04 +0000 (0:00:00.032) 0:00:10.776 ***** 2020-11-28T21:41:04.637 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2020-11-28T21:41:04.639 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Install CA Cert from Satellite Server] ************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:04.676 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:04 +0000 (0:00:00.044) 0:00:10.820 ***** 2020-11-28T21:41:04.709 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:04 +0000 (0:00:00.033) 0:00:10.854 ***** 2020-11-28T21:41:04.743 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:04 +0000 (0:00:00.033) 0:00:10.887 ***** 2020-11-28T21:41:04.749 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** ok: [smithi192.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2020-11-28T21:41:04.751 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** skipping: [smithi096.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Install yum on Fedora 22 and later] ***************************** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:04.788 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:04 +0000 (0:00:00.045) 0:00:10.932 ***** 2020-11-28T21:41:04.821 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:04 +0000 (0:00:00.033) 0:00:10.966 ***** 2020-11-28T21:41:04.853 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:04 +0000 (0:00:00.031) 0:00:10.998 ***** 2020-11-28T21:41:04.859 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2020-11-28T21:41:04.861 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set rhsm_registered if we're already registered] **************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set entitlements_path] ****************************************** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:04.904 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:04 +0000 (0:00:00.050) 0:00:11.048 ***** 2020-11-28T21:41:04.954 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:04 +0000 (0:00:00.051) 0:00:11.099 ***** 2020-11-28T21:41:04.990 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:04 +0000 (0:00:00.034) 0:00:11.134 ***** 2020-11-28T21:41:04.995 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:04.997 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** skipping: [smithi096.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Include Red Hat encrypted variables.] *************************** skipping: [smithi118.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi118.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-28T21:41:05.039 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:05 +0000 (0:00:00.050) 0:00:11.184 ***** 2020-11-28T21:41:05.076 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:05 +0000 (0:00:00.036) 0:00:11.221 ***** 2020-11-28T21:41:05.119 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:05 +0000 (0:00:00.042) 0:00:11.264 ***** 2020-11-28T21:41:05.127 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2020-11-28T21:41:05.129 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set rhsm_registered if we just registered] ********************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:05.129 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:05.167 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:05 +0000 (0:00:00.047) 0:00:11.311 ***** 2020-11-28T21:41:05.200 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:05 +0000 (0:00:00.033) 0:00:11.345 ***** 2020-11-28T21:41:05.234 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:05 +0000 (0:00:00.033) 0:00:11.378 ***** 2020-11-28T21:41:05.239 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2020-11-28T21:41:05.241 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : List CDN releases available to system] ************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Install CA Cert from Satellite Server] ************************** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:05.283 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:05 +0000 (0:00:00.049) 0:00:11.428 ***** 2020-11-28T21:41:05.318 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:05 +0000 (0:00:00.035) 0:00:11.463 ***** 2020-11-28T21:41:05.357 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:05 +0000 (0:00:00.039) 0:00:11.502 ***** 2020-11-28T21:41:05.363 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2020-11-28T21:41:05.365 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Determine if node is registered with subscription-manager.] ***** skipping: [smithi118.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-28T21:41:05.407 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:05 +0000 (0:00:00.049) 0:00:11.551 ***** 2020-11-28T21:41:05.439 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:05 +0000 (0:00:00.032) 0:00:11.584 ***** 2020-11-28T21:41:05.473 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:05 +0000 (0:00:00.033) 0:00:11.617 ***** 2020-11-28T21:41:05.481 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set rhsm_registered if we're already registered] **************** 2020-11-28T21:41:05.483 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:05.518 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:05 +0000 (0:00:00.044) 0:00:11.662 ***** 2020-11-28T21:41:05.559 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:05 +0000 (0:00:00.040) 0:00:11.703 ***** 2020-11-28T21:41:05.597 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:05 +0000 (0:00:00.038) 0:00:11.741 ***** 2020-11-28T21:41:05.603 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2020-11-28T21:41:05.605 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* skipping: [smithi096.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Register with subscription-manager.] **************************** skipping: [smithi118.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-28T21:41:05.645 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:05 +0000 (0:00:00.048) 0:00:11.790 ***** 2020-11-28T21:41:05.691 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:05 +0000 (0:00:00.045) 0:00:11.836 ***** 2020-11-28T21:41:05.724 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:05 +0000 (0:00:00.033) 0:00:11.869 ***** 2020-11-28T21:41:05.730 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2020-11-28T21:41:05.731 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Clean yum cache] ************************************************ skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set rhsm_registered if we just registered] ********************** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:05.784 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:05 +0000 (0:00:00.059) 0:00:11.929 ***** 2020-11-28T21:41:05.835 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:05 +0000 (0:00:00.050) 0:00:11.979 ***** 2020-11-28T21:41:05.874 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:05 +0000 (0:00:00.038) 0:00:12.018 ***** 2020-11-28T21:41:05.879 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2020-11-28T21:41:05.881 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Get list of enabled RHSM repos] ********************************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:05.926 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:05 +0000 (0:00:00.053) 0:00:12.071 ***** 2020-11-28T21:41:05.961 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:05 +0000 (0:00:00.034) 0:00:12.105 ***** 2020-11-28T21:41:05.967 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2020-11-28T21:41:05.969 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Store list of enabled repos] ************************************ skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:06.006 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:06 +0000 (0:00:00.045) 0:00:12.151 ***** 2020-11-28T21:41:06.041 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:06 +0000 (0:00:00.034) 0:00:12.186 ***** 2020-11-28T21:41:06.073 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:06 +0000 (0:00:00.032) 0:00:12.218 ***** 2020-11-28T21:41:06.079 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2020-11-28T21:41:06.081 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:06.117 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:06 +0000 (0:00:00.043) 0:00:12.261 ***** 2020-11-28T21:41:06.153 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:06 +0000 (0:00:00.036) 0:00:12.298 ***** 2020-11-28T21:41:06.186 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:06 +0000 (0:00:00.032) 0:00:12.331 ***** 2020-11-28T21:41:06.194 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2020-11-28T21:41:06.196 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:06.236 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:06 +0000 (0:00:00.049) 0:00:12.380 ***** 2020-11-28T21:41:06.272 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:06 +0000 (0:00:00.036) 0:00:12.417 ***** 2020-11-28T21:41:06.314 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:06 +0000 (0:00:00.041) 0:00:12.459 ***** 2020-11-28T21:41:06.323 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2020-11-28T21:41:06.326 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi162.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Set replace_repos true if newly-subscribed] ********************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:06.327 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* skipping: [smithi118.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-28T21:41:06.389 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:06 +0000 (0:00:00.074) 0:00:12.533 ***** 2020-11-28T21:41:06.439 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:06 +0000 (0:00:00.050) 0:00:12.583 ***** 2020-11-28T21:41:06.479 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:06 +0000 (0:00:00.040) 0:00:12.623 ***** 2020-11-28T21:41:06.486 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2020-11-28T21:41:06.488 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Disable all rhsm repos] ***************************************** skipping: [smithi096.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Clean yum cache] ************************************************ skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:06.527 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:06 +0000 (0:00:00.048) 0:00:12.672 ***** 2020-11-28T21:41:06.562 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:06 +0000 (0:00:00.034) 0:00:12.706 ***** 2020-11-28T21:41:06.596 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:06 +0000 (0:00:00.033) 0:00:12.740 ***** 2020-11-28T21:41:06.601 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2020-11-28T21:41:06.603 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Enable necessary rhsm repos] ************************************ skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Get list of enabled RHSM repos] ********************************* skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:06.640 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:06 +0000 (0:00:00.044) 0:00:12.784 ***** 2020-11-28T21:41:06.674 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:06 +0000 (0:00:00.034) 0:00:12.819 ***** 2020-11-28T21:41:06.709 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:06 +0000 (0:00:00.034) 0:00:12.854 ***** 2020-11-28T21:41:06.715 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Remove old apt-mirror repository definition.] ******************* 2020-11-28T21:41:06.717 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Store list of enabled repos] ************************************ skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:06.758 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:06 +0000 (0:00:00.049) 0:00:12.903 ***** 2020-11-28T21:41:06.794 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:06 +0000 (0:00:00.035) 0:00:12.939 ***** 2020-11-28T21:41:06.833 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:06 +0000 (0:00:00.039) 0:00:12.978 ***** 2020-11-28T21:41:06.840 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:06.841 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:06.877 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:06 +0000 (0:00:00.043) 0:00:13.021 ***** 2020-11-28T21:41:06.910 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:06 +0000 (0:00:00.032) 0:00:13.054 ***** 2020-11-28T21:41:06.945 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:06 +0000 (0:00:00.035) 0:00:13.089 ***** 2020-11-28T21:41:06.949 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2020-11-28T21:41:06.951 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi192.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:06.953 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:06.954 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:06.988 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:06 +0000 (0:00:00.043) 0:00:13.133 ***** 2020-11-28T21:41:07.022 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:07 +0000 (0:00:00.033) 0:00:13.166 ***** 2020-11-28T21:41:07.057 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:07 +0000 (0:00:00.035) 0:00:13.202 ***** 2020-11-28T21:41:07.063 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** skipping: [smithi162.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Configure local epel mirrorlists] ******************************* skipping: [smithi096.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:07.065 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} TASK [common : Set replace_repos true if newly-subscribed] ********************* skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:07.101 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:07 +0000 (0:00:00.043) 0:00:13.246 ***** 2020-11-28T21:41:07.135 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:07 +0000 (0:00:00.033) 0:00:13.279 ***** 2020-11-28T21:41:07.167 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:07 +0000 (0:00:00.032) 0:00:13.312 ***** 2020-11-28T21:41:07.173 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2020-11-28T21:41:07.174 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Clean yum cache] ************************************************ skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Disable all rhsm repos] ***************************************** skipping: [smithi118.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-28T21:41:07.217 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:07 +0000 (0:00:00.049) 0:00:13.361 ***** 2020-11-28T21:41:07.250 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:07 +0000 (0:00:00.033) 0:00:13.395 ***** 2020-11-28T21:41:07.286 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:07 +0000 (0:00:00.035) 0:00:13.430 ***** 2020-11-28T21:41:07.291 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2020-11-28T21:41:07.293 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Enable necessary rhsm repos] ************************************ skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:07.333 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:07 +0000 (0:00:00.047) 0:00:13.478 ***** 2020-11-28T21:41:07.376 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:07 +0000 (0:00:00.043) 0:00:13.521 ***** 2020-11-28T21:41:07.383 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2020-11-28T21:41:07.384 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:07.385 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:07.386 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.019966", "end": "2020-11-28 21:41:07.752858", "rc": 0, "start": "2020-11-28 21:41:07.732892", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2020-11-28T21:41:07.424 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:07 +0000 (0:00:00.047) 0:00:13.569 ***** 2020-11-28T21:41:07.463 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:07 +0000 (0:00:00.038) 0:00:13.608 ***** 2020-11-28T21:41:07.499 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:07 +0000 (0:00:00.035) 0:00:13.644 ***** 2020-11-28T21:41:07.508 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** skipping: [smithi162.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Set the timezone in /etc/timezone] ****************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:07.510 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:07.547 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:07 +0000 (0:00:00.047) 0:00:13.691 ***** 2020-11-28T21:41:07.582 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:07 +0000 (0:00:00.035) 0:00:13.727 ***** 2020-11-28T21:41:07.617 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:07 +0000 (0:00:00.035) 0:00:13.762 ***** 2020-11-28T21:41:07.624 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2020-11-28T21:41:07.626 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Inform the OS of the new timezone] ****************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** skipping: [smithi118.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:07.627 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:07.663 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:07 +0000 (0:00:00.045) 0:00:13.808 ***** 2020-11-28T21:41:07.697 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:07 +0000 (0:00:00.033) 0:00:13.842 ***** 2020-11-28T21:41:07.732 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:07 +0000 (0:00:00.035) 0:00:13.877 ***** 2020-11-28T21:41:07.738 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2020-11-28T21:41:07.740 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Get the current timezone] *************************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Configure local epel mirrorlists] ******************************* skipping: [smithi118.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:07.741 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:07.777 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:07 +0000 (0:00:00.045) 0:00:13.922 ***** 2020-11-28T21:41:07.812 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:07 +0000 (0:00:00.034) 0:00:13.957 ***** 2020-11-28T21:41:07.845 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:07 +0000 (0:00:00.033) 0:00:13.990 ***** 2020-11-28T21:41:07.851 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2020-11-28T21:41:07.853 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set the timezone] *********************************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Clean yum cache] ************************************************ skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:07.889 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:07 +0000 (0:00:00.043) 0:00:14.033 ***** 2020-11-28T21:41:07.921 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:07 +0000 (0:00:00.032) 0:00:14.066 ***** 2020-11-28T21:41:07.953 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:07 +0000 (0:00:00.032) 0:00:14.098 ***** 2020-11-28T21:41:07.968 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2020-11-28T21:41:07.970 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Add base OpenSUSE Leap repo] ************************************ skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:08.013 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:08 +0000 (0:00:00.059) 0:00:14.158 ***** 2020-11-28T21:41:08.060 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:08 +0000 (0:00:00.046) 0:00:14.205 ***** 2020-11-28T21:41:08.067 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2020-11-28T21:41:08.069 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Add updates OpenSUSE Leap repo] ********************************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:08.070 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** ok: [smithi118.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.007532", "end": "2020-11-28 21:41:08.125456", "rc": 0, "start": "2020-11-28 21:41:08.117924", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2020-11-28T21:41:08.108 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:08 +0000 (0:00:00.048) 0:00:14.253 ***** 2020-11-28T21:41:08.143 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:08 +0000 (0:00:00.034) 0:00:14.288 ***** 2020-11-28T21:41:08.177 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:08 +0000 (0:00:00.033) 0:00:14.321 ***** 2020-11-28T21:41:08.182 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2020-11-28T21:41:08.184 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Refresh repos] ************************************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set the timezone in /etc/timezone] ****************************** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:08.219 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:08 +0000 (0:00:00.043) 0:00:14.364 ***** 2020-11-28T21:41:08.254 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:08 +0000 (0:00:00.035) 0:00:14.399 ***** 2020-11-28T21:41:08.288 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:08 +0000 (0:00:00.033) 0:00:14.433 ***** 2020-11-28T21:41:08.295 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:08.297 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:08.337 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:08 +0000 (0:00:00.048) 0:00:14.481 ***** 2020-11-28T21:41:08.375 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:08 +0000 (0:00:00.038) 0:00:14.520 ***** 2020-11-28T21:41:08.381 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2020-11-28T21:41:08.382 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Get the current timezone] *************************************** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:08.422 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:08 +0000 (0:00:00.047) 0:00:14.567 ***** 2020-11-28T21:41:08.460 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:08 +0000 (0:00:00.037) 0:00:14.605 ***** 2020-11-28T21:41:08.467 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2020-11-28T21:41:08.468 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set the timezone] *********************************************** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:08.516 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:08 +0000 (0:00:00.055) 0:00:14.661 ***** 2020-11-28T21:41:08.558 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:08 +0000 (0:00:00.041) 0:00:14.702 ***** 2020-11-28T21:41:08.564 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2020-11-28T21:41:08.565 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Add base OpenSUSE Leap repo] ************************************ skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:08.600 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:08 +0000 (0:00:00.042) 0:00:14.745 ***** 2020-11-28T21:41:08.632 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:08 +0000 (0:00:00.032) 0:00:14.777 ***** 2020-11-28T21:41:08.638 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2020-11-28T21:41:08.640 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Add updates OpenSUSE Leap repo] ********************************* skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:08.675 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:08 +0000 (0:00:00.042) 0:00:14.820 ***** 2020-11-28T21:41:08.707 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:08 +0000 (0:00:00.032) 0:00:14.852 ***** 2020-11-28T21:41:08.714 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-11-28T21:41:08.716 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Refresh repos] ************************************************** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:08.756 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:08 +0000 (0:00:00.048) 0:00:14.900 ***** 2020-11-28T21:41:08.790 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:08 +0000 (0:00:00.034) 0:00:14.935 ***** 2020-11-28T21:41:08.802 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:08.835 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:08 +0000 (0:00:00.045) 0:00:14.980 ***** 2020-11-28T21:41:08.842 INFO:teuthology.task.ansible.out: 2020-11-28T21:41:08.844 INFO:teuthology.task.ansible.out:TASK [common : Get list of enabled RHSM repos] ********************************* skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:08.881 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:08 +0000 (0:00:00.045) 0:00:15.026 ***** 2020-11-28T21:41:08.888 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2020-11-28T21:41:08.890 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-28T21:41:08.929 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:08 +0000 (0:00:00.046) 0:00:15.073 ***** 2020-11-28T21:41:08.934 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2020-11-28T21:41:08.936 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:08.990 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:08 +0000 (0:00:00.061) 0:00:15.134 ***** 2020-11-28T21:41:08.998 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-11-28T21:41:08.999 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:09.036 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:09 +0000 (0:00:00.046) 0:00:15.181 ***** 2020-11-28T21:41:09.042 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2020-11-28T21:41:09.044 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:09.081 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:09 +0000 (0:00:00.044) 0:00:15.225 ***** 2020-11-28T21:41:09.087 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2020-11-28T21:41:09.088 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:09.124 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:09 +0000 (0:00:00.043) 0:00:15.269 ***** 2020-11-28T21:41:09.129 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2020-11-28T21:41:09.130 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set the hardware clock] ***************************************** 2020-11-28T21:41:09.131 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.759026", "end": "2020-11-28 21:41:09.500412", "rc": 0, "start": "2020-11-28 21:41:08.741386", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-28T21:41:09.168 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:09 +0000 (0:00:00.043) 0:00:15.313 ***** 2020-11-28T21:41:09.202 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:09 +0000 (0:00:00.033) 0:00:15.346 ***** 2020-11-28T21:41:09.215 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2020-11-28T21:41:09.217 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Install Kerberos Packages (RedHat)] ***************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:09.258 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:09 +0000 (0:00:00.056) 0:00:15.402 ***** 2020-11-28T21:41:09.306 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:09 +0000 (0:00:00.048) 0:00:15.451 ***** 2020-11-28T21:41:09.316 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2020-11-28T21:41:09.318 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:09.368 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:09 +0000 (0:00:00.061) 0:00:15.513 ***** 2020-11-28T21:41:09.376 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2020-11-28T21:41:09.377 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:09.413 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:09 +0000 (0:00:00.045) 0:00:15.558 ***** 2020-11-28T21:41:09.419 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2020-11-28T21:41:09.421 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-28T21:41:09.458 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:09 +0000 (0:00:00.044) 0:00:15.603 ***** 2020-11-28T21:41:09.465 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2020-11-28T21:41:09.467 INFO:teuthology.task.ansible.out:[WARNING]: Consider using the apt module rather than running 'apt-get'. If you need to use command because apt is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.023854", "end": "2020-11-28 21:41:09.804995", "rc": 0, "start": "2020-11-28 21:41:09.781141", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-28T21:41:09.467 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** changed: [smithi118.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.537947", "end": "2020-11-28 21:41:09.500520", "rc": 0, "start": "2020-11-28 21:41:08.962573", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-28T21:41:09.468 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:09.505 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:09 +0000 (0:00:00.046) 0:00:15.649 ***** 2020-11-28T21:41:09.540 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:09 +0000 (0:00:00.034) 0:00:15.684 ***** 2020-11-28T21:41:09.576 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:09 +0000 (0:00:00.034) 0:00:15.719 ***** 2020-11-28T21:41:09.580 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:09.583 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:09.622 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:09 +0000 (0:00:00.047) 0:00:15.766 ***** 2020-11-28T21:41:09.661 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:09 +0000 (0:00:00.038) 0:00:15.805 ***** 2020-11-28T21:41:09.669 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:09.712 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:09 +0000 (0:00:00.051) 0:00:15.857 ***** 2020-11-28T21:41:09.720 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2020-11-28T21:41:09.721 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:09.758 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:09 +0000 (0:00:00.046) 0:00:15.903 ***** 2020-11-28T21:41:09.765 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2020-11-28T21:41:09.767 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-28T21:41:09.805 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:09 +0000 (0:00:00.046) 0:00:15.950 ***** 2020-11-28T21:41:09.810 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2020-11-28T21:41:09.811 INFO:teuthology.task.ansible.out:changed: [smithi118.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.022780", "end": "2020-11-28 21:41:09.841493", "rc": 0, "start": "2020-11-28 21:41:09.818713", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-28T21:41:09.813 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:09.855 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:09 +0000 (0:00:00.049) 0:00:16.000 ***** 2020-11-28T21:41:09.888 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:09 +0000 (0:00:00.033) 0:00:16.033 ***** 2020-11-28T21:41:09.895 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2020-11-28T21:41:09.896 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:09.945 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:09 +0000 (0:00:00.057) 0:00:16.090 ***** 2020-11-28T21:41:09.952 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2020-11-28T21:41:09.953 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:09.991 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:09 +0000 (0:00:00.045) 0:00:16.136 ***** 2020-11-28T21:41:09.998 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2020-11-28T21:41:10.000 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:10.039 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:10 +0000 (0:00:00.045) 0:00:16.182 ***** 2020-11-28T21:41:10.043 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2020-11-28T21:41:10.045 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:10.083 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:10 +0000 (0:00:00.045) 0:00:16.228 ***** 2020-11-28T21:41:10.090 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2020-11-28T21:41:10.092 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:10.128 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:10 +0000 (0:00:00.044) 0:00:16.272 ***** 2020-11-28T21:41:10.134 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-11-28T21:41:10.136 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:10.173 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:10 +0000 (0:00:00.045) 0:00:16.318 ***** 2020-11-28T21:41:10.179 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2020-11-28T21:41:10.181 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:10.219 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:10 +0000 (0:00:00.045) 0:00:16.364 ***** 2020-11-28T21:41:10.246 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2020-11-28T21:41:10.248 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:10.257 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:10.294 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:10 +0000 (0:00:00.075) 0:00:16.439 ***** 2020-11-28T21:41:10.321 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-11-28T21:41:10.323 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Get the current timezone] *************************************** ok: [smithi162.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.014885", "end": "2020-11-28 21:41:10.726262", "rc": 0, "start": "2020-11-28 21:41:10.711377", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2020-11-28T21:41:10.367 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:10 +0000 (0:00:00.072) 0:00:16.512 ***** 2020-11-28T21:41:10.404 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:10 +0000 (0:00:00.036) 0:00:16.548 ***** 2020-11-28T21:41:10.411 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2020-11-28T21:41:10.413 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:10.448 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:10 +0000 (0:00:00.044) 0:00:16.593 ***** 2020-11-28T21:41:10.488 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:10.537 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:10 +0000 (0:00:00.089) 0:00:16.682 ***** 2020-11-28T21:41:10.544 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-11-28T21:41:10.545 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.008400", "end": "2020-11-28 21:41:10.855654", "rc": 0, "start": "2020-11-28 21:41:10.847254", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2020-11-28T21:41:10.582 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:10 +0000 (0:00:00.044) 0:00:16.726 ***** 2020-11-28T21:41:10.588 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-11-28T21:41:10.590 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:10.626 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:10 +0000 (0:00:00.043) 0:00:16.770 ***** 2020-11-28T21:41:10.640 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:10.698 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:10 +0000 (0:00:00.071) 0:00:16.842 ***** 2020-11-28T21:41:10.707 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2020-11-28T21:41:10.709 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:10.765 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:10 +0000 (0:00:00.067) 0:00:16.910 ***** 2020-11-28T21:41:10.774 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2020-11-28T21:41:10.776 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:10.836 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:10 +0000 (0:00:00.071) 0:00:16.981 ***** 2020-11-28T21:41:10.842 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2020-11-28T21:41:10.844 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:10.902 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:10 +0000 (0:00:00.064) 0:00:17.046 ***** 2020-11-28T21:41:10.909 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-11-28T21:41:10.911 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:10.946 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:10 +0000 (0:00:00.045) 0:00:17.091 ***** 2020-11-28T21:41:10.953 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2020-11-28T21:41:10.954 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:10.991 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:10 +0000 (0:00:00.044) 0:00:17.136 ***** 2020-11-28T21:41:10.998 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2020-11-28T21:41:11.000 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:11.034 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:11 +0000 (0:00:00.042) 0:00:17.179 ***** 2020-11-28T21:41:11.040 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2020-11-28T21:41:11.042 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:11.083 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:11 +0000 (0:00:00.049) 0:00:17.228 ***** 2020-11-28T21:41:11.090 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2020-11-28T21:41:11.091 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:11.130 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:11 +0000 (0:00:00.047) 0:00:17.275 ***** 2020-11-28T21:41:11.157 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2020-11-28T21:41:11.159 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:11.208 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:11 +0000 (0:00:00.077) 0:00:17.353 ***** 2020-11-28T21:41:11.237 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:11.278 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:11 +0000 (0:00:00.069) 0:00:17.422 ***** 2020-11-28T21:41:12.096 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2020-11-28T21:41:12.098 INFO:teuthology.task.ansible.out:changed: [smithi162.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.884005", "end": "2020-11-28 21:41:12.500339", "rc": 0, "start": "2020-11-28 21:41:11.616334", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-28T21:41:12.144 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:12 +0000 (0:00:00.866) 0:00:18.289 ***** 2020-11-28T21:41:12.183 INFO:teuthology.task.ansible.out:changed: [smithi192.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.783040", "end": "2020-11-28 21:41:12.500367", "rc": 0, "start": "2020-11-28 21:41:11.717327", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-28T21:41:12.232 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:12 +0000 (0:00:00.087) 0:00:18.376 ***** 2020-11-28T21:41:12.242 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:12.289 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:12 +0000 (0:00:00.056) 0:00:18.433 ***** 2020-11-28T21:41:12.296 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:12.352 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:12 +0000 (0:00:00.063) 0:00:18.496 ***** 2020-11-28T21:41:12.353 INFO:teuthology.task.ansible.out: 2020-11-28T21:41:12.476 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2020-11-28T21:41:12.477 INFO:teuthology.task.ansible.out:changed: [smithi162.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.024744", "end": "2020-11-28 21:41:12.880072", "rc": 0, "start": "2020-11-28 21:41:12.855328", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-28T21:41:12.534 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:12 +0000 (0:00:00.182) 0:00:18.679 ***** 2020-11-28T21:41:12.544 INFO:teuthology.task.ansible.out:changed: [smithi192.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.026550", "end": "2020-11-28 21:41:12.838236", "rc": 0, "start": "2020-11-28 21:41:12.811686", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-28T21:41:12.586 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:12 +0000 (0:00:00.051) 0:00:18.730 ***** 2020-11-28T21:41:12.813 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** ok: [smithi118.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1606599636, "cache_updated": false, "changed": false} 2020-11-28T21:41:12.850 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:12 +0000 (0:00:00.264) 0:00:18.995 ***** 2020-11-28T21:41:12.852 INFO:teuthology.task.ansible.out: 2020-11-28T21:41:12.942 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1606599646, "cache_updated": false, "changed": false} 2020-11-28T21:41:12.980 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:12 +0000 (0:00:00.129) 0:00:19.125 ***** 2020-11-28T21:41:13.790 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2020-11-28T21:41:13.792 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"cache_update_time": 1606599636, "cache_updated": false, "changed": false} 2020-11-28T21:41:13.831 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:13 +0000 (0:00:00.850) 0:00:19.975 ***** 2020-11-28T21:41:13.859 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2020-11-28T21:41:13.860 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:13.896 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:13 +0000 (0:00:00.065) 0:00:20.041 ***** 2020-11-28T21:41:13.903 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2020-11-28T21:41:13.904 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"cache_update_time": 1606599646, "cache_updated": false, "changed": false} 2020-11-28T21:41:13.950 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:13 +0000 (0:00:00.053) 0:00:20.095 ***** 2020-11-28T21:41:13.977 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2020-11-28T21:41:13.980 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:14.014 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:14 +0000 (0:00:00.064) 0:00:20.159 ***** 2020-11-28T21:41:14.206 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* ok: [smithi118.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2020-11-28T21:41:14.248 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:14 +0000 (0:00:00.234) 0:00:20.393 ***** 2020-11-28T21:41:14.339 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2020-11-28T21:41:14.376 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:14 +0000 (0:00:00.128) 0:00:20.521 ***** 2020-11-28T21:41:15.072 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2020-11-28T21:41:15.074 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"cache_update_time": 1606599636, "cache_updated": false, "changed": false} 2020-11-28T21:41:15.111 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:15 +0000 (0:00:00.734) 0:00:21.256 ***** 2020-11-28T21:41:15.244 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"cache_update_time": 1606599646, "cache_updated": false, "changed": false} 2020-11-28T21:41:15.283 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:15 +0000 (0:00:00.172) 0:00:21.428 ***** 2020-11-28T21:41:15.365 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** ok: [smithi162.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1606599631, "cache_updated": false, "changed": false} 2020-11-28T21:41:15.405 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:15 +0000 (0:00:00.122) 0:00:21.550 ***** 2020-11-28T21:41:15.424 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-11-28T21:41:15.425 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2020-11-28T21:41:15.619 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2020-11-28T21:41:15.662 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** ok: [smithi192.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1606599641, "cache_updated": false, "changed": false} 2020-11-28T21:41:15.704 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:15 +0000 (0:00:00.298) 0:00:21.849 ***** 2020-11-28T21:41:15.705 INFO:teuthology.task.ansible.out: 2020-11-28T21:41:15.714 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** ok: [smithi118.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2020-11-28T21:41:15.955 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2020-11-28T21:41:16.003 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2020-11-28T21:41:16.052 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:16 +0000 (0:00:00.348) 0:00:22.197 ***** 2020-11-28T21:41:16.054 INFO:teuthology.task.ansible.out: 2020-11-28T21:41:16.198 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2020-11-28T21:41:16.199 INFO:teuthology.task.ansible.out: ok: [smithi118.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2020-11-28T21:41:16.246 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:16 +0000 (0:00:00.190) 0:00:22.387 ***** 2020-11-28T21:41:16.272 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-11-28T21:41:16.275 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2020-11-28T21:41:16.311 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:16 +0000 (0:00:00.068) 0:00:22.456 ***** 2020-11-28T21:41:16.344 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2020-11-28T21:41:16.346 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => {"cache_update_time": 1606599631, "cache_updated": false, "changed": false} 2020-11-28T21:41:16.393 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:16 +0000 (0:00:00.081) 0:00:22.538 ***** 2020-11-28T21:41:16.428 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:16.480 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:16 +0000 (0:00:00.086) 0:00:22.625 ***** 2020-11-28T21:41:16.487 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2020-11-28T21:41:16.489 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2020-11-28T21:41:16.531 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:16 +0000 (0:00:00.049) 0:00:22.674 ***** 2020-11-28T21:41:16.611 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2020-11-28T21:41:16.612 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"cache_update_time": 1606599641, "cache_updated": false, "changed": false} 2020-11-28T21:41:16.651 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:16 +0000 (0:00:00.120) 0:00:22.795 ***** 2020-11-28T21:41:16.658 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-11-28T21:41:16.660 INFO:teuthology.task.ansible.out:changed: [smithi118.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": true, "checksum": "00d0404b545463be4a65f475e0870e07a48f35b6", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "md5sum": "14041bf2666d8f8d277567958c7e9f69", "mode": "0755", "owner": "root", "size": 13355, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1606599676.2656476-153104015367243/source", "state": "file", "uid": 0} 2020-11-28T21:41:16.681 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2020-11-28T21:41:16.683 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:16.719 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:16 +0000 (0:00:00.068) 0:00:22.864 ***** 2020-11-28T21:41:16.825 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* ok: [smithi162.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2020-11-28T21:41:16.863 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:16 +0000 (0:00:00.143) 0:00:23.007 ***** 2020-11-28T21:41:16.921 INFO:teuthology.task.ansible.out: 2020-11-28T21:41:16.922 INFO:teuthology.task.ansible.out:TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** ok: [smithi118.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2020-11-28T21:41:16.955 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": true, "checksum": "00d0404b545463be4a65f475e0870e07a48f35b6", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "md5sum": "14041bf2666d8f8d277567958c7e9f69", "mode": "0755", "owner": "root", "size": 13355, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1606599676.556583-65110444092296/source", "state": "file", "uid": 0} 2020-11-28T21:41:17.034 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2020-11-28T21:41:17.035 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2020-11-28T21:41:17.080 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:17 +0000 (0:00:00.216) 0:00:23.224 ***** 2020-11-28T21:41:17.198 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-11-28T21:41:17.200 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2020-11-28T21:41:17.249 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:17 +0000 (0:00:00.169) 0:00:23.394 ***** 2020-11-28T21:41:17.257 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2020-11-28T21:41:17.295 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2020-11-28T21:41:17.297 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2020-11-28T21:41:17.343 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:17 +0000 (0:00:00.093) 0:00:23.487 ***** 2020-11-28T21:41:17.370 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2020-11-28T21:41:17.372 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:17.406 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:17 +0000 (0:00:00.063) 0:00:23.551 ***** 2020-11-28T21:41:17.434 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* skipping: [smithi118.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-28T21:41:17.477 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:17 +0000 (0:00:00.071) 0:00:23.622 ***** 2020-11-28T21:41:17.515 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2020-11-28T21:41:17.517 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:17.527 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:17.540 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:17.553 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** ok: [smithi096.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2020-11-28T21:41:17.591 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:17 +0000 (0:00:00.114) 0:00:23.736 ***** 2020-11-28T21:41:17.625 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:17 +0000 (0:00:00.033) 0:00:23.770 ***** 2020-11-28T21:41:17.641 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2020-11-28T21:41:17.643 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2020-11-28T21:41:17.678 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:17 +0000 (0:00:00.053) 0:00:23.823 ***** 2020-11-28T21:41:17.685 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2020-11-28T21:41:17.686 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:17.727 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:17 +0000 (0:00:00.046) 0:00:23.869 ***** 2020-11-28T21:41:17.731 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2020-11-28T21:41:17.734 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Make sure smartmontools is installed] *************************** ok: [smithi162.front.sepia.ceph.com] => {"cache_update_time": 1606599631, "cache_updated": false, "changed": false} 2020-11-28T21:41:17.787 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:17 +0000 (0:00:00.061) 0:00:23.931 ***** 2020-11-28T21:41:17.842 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:17 +0000 (0:00:00.055) 0:00:23.987 ***** 2020-11-28T21:41:17.849 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2020-11-28T21:41:17.850 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-28T21:41:17.891 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:17 +0000 (0:00:00.048) 0:00:24.035 ***** 2020-11-28T21:41:17.936 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** skipping: [smithi096.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:17.947 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:17.948 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:17.961 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:17.997 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:17 +0000 (0:00:00.106) 0:00:24.141 ***** 2020-11-28T21:41:18.026 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:18.062 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:18 +0000 (0:00:00.065) 0:00:24.206 ***** 2020-11-28T21:41:18.063 INFO:teuthology.task.ansible.out: 2020-11-28T21:41:18.080 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2020-11-28T21:41:18.082 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"cache_update_time": 1606599641, "cache_updated": false, "changed": false} 2020-11-28T21:41:18.117 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:18 +0000 (0:00:00.055) 0:00:24.262 ***** 2020-11-28T21:41:18.215 INFO:teuthology.task.ansible.out: 2020-11-28T21:41:18.217 INFO:teuthology.task.ansible.out:TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-11-28T21:41:18.217 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2020-11-28T21:41:18.476 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2020-11-28T21:41:18.519 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2020-11-28T21:41:18.585 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** ok: [smithi118.front.sepia.ceph.com] => {"cache_update_time": 1606599636, "cache_updated": false, "changed": false} 2020-11-28T21:41:18.624 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:18 +0000 (0:00:00.506) 0:00:24.769 ***** 2020-11-28T21:41:18.759 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-11-28T21:41:18.760 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2020-11-28T21:41:18.820 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:18 +0000 (0:00:00.196) 0:00:24.965 ***** 2020-11-28T21:41:18.830 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2020-11-28T21:41:18.942 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2020-11-28T21:41:18.943 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2020-11-28T21:41:18.948 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* ok: [smithi162.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2020-11-28T21:41:18.979 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:18 +0000 (0:00:00.158) 0:00:25.124 ***** 2020-11-28T21:41:19.015 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:19 +0000 (0:00:00.035) 0:00:25.160 ***** 2020-11-28T21:41:19.021 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2020-11-28T21:41:19.023 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"cache_update_time": 1606599646, "cache_updated": false, "changed": false} 2020-11-28T21:41:19.059 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:19 +0000 (0:00:00.043) 0:00:25.203 ***** 2020-11-28T21:41:19.065 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-11-28T21:41:19.067 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2020-11-28T21:41:19.105 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:19 +0000 (0:00:00.046) 0:00:25.250 ***** 2020-11-28T21:41:19.237 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2020-11-28T21:41:19.238 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2020-11-28T21:41:19.280 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:19 +0000 (0:00:00.175) 0:00:25.425 ***** 2020-11-28T21:41:19.296 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2020-11-28T21:41:19.298 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2020-11-28T21:41:19.339 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:19 +0000 (0:00:00.058) 0:00:25.484 ***** 2020-11-28T21:41:19.377 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-11-28T21:41:19.379 INFO:teuthology.task.ansible.out:changed: [smithi162.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": true, "checksum": "00d0404b545463be4a65f475e0870e07a48f35b6", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "md5sum": "14041bf2666d8f8d277567958c7e9f69", "mode": "0755", "owner": "root", "size": 13355, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1606599679.0378828-27834146572695/source", "state": "file", "uid": 0} 2020-11-28T21:41:19.389 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2020-11-28T21:41:19.391 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2020-11-28T21:41:19.426 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:19 +0000 (0:00:00.087) 0:00:25.571 ***** 2020-11-28T21:41:19.499 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-11-28T21:41:19.501 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2020-11-28T21:41:19.611 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2020-11-28T21:41:19.649 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:19 +0000 (0:00:00.222) 0:00:25.793 ***** 2020-11-28T21:41:19.668 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** changed: [smithi192.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": true, "checksum": "00d0404b545463be4a65f475e0870e07a48f35b6", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "md5sum": "14041bf2666d8f8d277567958c7e9f69", "mode": "0755", "owner": "root", "size": 13355, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1606599679.305574-169610550156791/source", "state": "file", "uid": 0} 2020-11-28T21:41:19.679 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2020-11-28T21:41:19.681 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:19.723 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:19 +0000 (0:00:00.074) 0:00:25.868 ***** 2020-11-28T21:41:19.749 INFO:teuthology.task.ansible.out: 2020-11-28T21:41:19.753 INFO:teuthology.task.ansible.out:TASK [common : Upload nagios check_mem script] ********************************* 2020-11-28T21:41:19.755 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2020-11-28T21:41:19.794 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:19 +0000 (0:00:00.071) 0:00:25.939 ***** 2020-11-28T21:41:19.802 INFO:teuthology.task.ansible.out: 2020-11-28T21:41:19.802 INFO:teuthology.task.ansible.out:TASK [common : Check firewalld status] ***************************************** 2020-11-28T21:41:19.803 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-28T21:41:19.846 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:19 +0000 (0:00:00.052) 0:00:25.991 ***** 2020-11-28T21:41:19.898 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2020-11-28T21:41:19.899 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:19.941 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:19 +0000 (0:00:00.094) 0:00:26.085 ***** 2020-11-28T21:41:19.948 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-11-28T21:41:19.950 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2020-11-28T21:41:19.960 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-11-28T21:41:19.962 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2020-11-28T21:41:19.972 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-11-28T21:41:19.974 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2020-11-28T21:41:20.017 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:20 +0000 (0:00:00.076) 0:00:26.162 ***** 2020-11-28T21:41:20.058 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2020-11-28T21:41:20.061 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2020-11-28T21:41:20.100 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:20 +0000 (0:00:00.083) 0:00:26.245 ***** 2020-11-28T21:41:20.113 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ ok: [smithi096.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2020-11-28T21:41:20.153 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:20 +0000 (0:00:00.052) 0:00:26.298 ***** 2020-11-28T21:41:20.155 INFO:teuthology.task.ansible.out: 2020-11-28T21:41:20.161 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:20.201 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:20 +0000 (0:00:00.047) 0:00:26.345 ***** 2020-11-28T21:41:20.210 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:20.248 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:20 +0000 (0:00:00.047) 0:00:26.392 ***** 2020-11-28T21:41:20.255 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-11-28T21:41:20.257 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2020-11-28T21:41:20.259 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2020-11-28T21:41:20.259 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-28T21:41:20.296 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:20 +0000 (0:00:00.047) 0:00:26.440 ***** 2020-11-28T21:41:20.330 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:20 +0000 (0:00:00.033) 0:00:26.474 ***** 2020-11-28T21:41:20.338 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2020-11-28T21:41:20.339 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Upload nagios nrpe config.] ************************************* ok: [smithi118.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2020-11-28T21:41:20.340 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** skipping: [smithi162.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:20.341 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:20.342 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:20.380 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:20 +0000 (0:00:00.050) 0:00:26.525 ***** 2020-11-28T21:41:20.415 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:20 +0000 (0:00:00.034) 0:00:26.560 ***** 2020-11-28T21:41:20.422 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:20.423 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2020-11-28T21:41:20.423 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2020-11-28T21:41:20.424 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:20.463 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:20 +0000 (0:00:00.047) 0:00:26.607 ***** 2020-11-28T21:41:20.501 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:20 +0000 (0:00:00.038) 0:00:26.646 ***** 2020-11-28T21:41:20.541 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:20 +0000 (0:00:00.038) 0:00:26.684 ***** 2020-11-28T21:41:20.549 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2020-11-28T21:41:20.551 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:20.590 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:20 +0000 (0:00:00.050) 0:00:26.735 ***** 2020-11-28T21:41:20.598 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2020-11-28T21:41:20.600 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:20.636 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:20 +0000 (0:00:00.045) 0:00:26.781 ***** 2020-11-28T21:41:20.668 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* skipping: [smithi192.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-28T21:41:20.710 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:20 +0000 (0:00:00.074) 0:00:26.855 ***** 2020-11-28T21:41:20.747 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2020-11-28T21:41:20.749 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:20.759 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:20.761 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:20.772 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:20.811 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:20 +0000 (0:00:00.099) 0:00:26.954 ***** 2020-11-28T21:41:20.816 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2020-11-28T21:41:20.818 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2020-11-28T21:41:20.854 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:20 +0000 (0:00:00.044) 0:00:26.999 ***** 2020-11-28T21:41:20.863 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2020-11-28T21:41:20.866 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:20.902 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:20 +0000 (0:00:00.047) 0:00:27.047 ***** 2020-11-28T21:41:21.037 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2020-11-28T21:41:21.038 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2020-11-28 21:39:55 UTC", "ActiveEnterTimestampMonotonic": "17823759", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "basic.target time-sync.target systemd-journald.socket var-run.mount system.slice network.target -.mount remote-fs.target nss-lookup.target local-fs.target sysinit.target systemd-tmpfiles-setup.service", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2020-11-28 21:39:55 UTC", "AssertTimestampMonotonic": "17822882", "Before": "xdm.service getty@tty1.service shutdown.target plymouth-quit.service multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2020-11-28 21:39:55 UTC", "ConditionTimestampMonotonic": "17822882", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target nrpe.socket", "ControlGroup": "/system.slice/nagios-nrpe-ser 2020-11-28T21:41:21.040 INFO:teuthology.task.ansible.out:ver.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EnvironmentFile": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1312", "ExecMainStartTimestamp": "Sat 2020-11-28 21:39:55 UTC", "ExecMainStartTimestampMonotonic": "17823737", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2020-11-28 21:39:55 UTC", "InactiveExitTimestampMonotonic": "17823759", "InvocationID": "90acc0d34d9d4307b8c8fe083732bd56", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "i 2020-11-28T21:41:21.041 INFO:teuthology.task.ansible.out:nfinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1312", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nagios/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target system.slice -.mount", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no" 2020-11-28T21:41:21.043 INFO:teuthology.task.ansible.out:, "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2020-11-28 21:39:55 UTC", "StateChangeTimestampMonotonic": "17823759", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "109", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2020-11-28 21:39:55 UTC", "WatchdogTimestampMonotonic": "17823758", "WatchdogUSec": "0"}} 2020-11-28T21:41:21.076 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:21 +0000 (0:00:00.173) 0:00:27.221 ***** 2020-11-28T21:41:21.097 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2020-11-28 21:39:21 UTC", "ActiveEnterTimestampMonotonic": "18343494", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "network.target system.slice local-fs.target remote-fs.target var-run.mount systemd-tmpfiles-setup.service basic.target sysinit.target systemd-journald.socket -.mount time-sync.target nss-lookup.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2020-11-28 21:39:21 UTC", "AssertTimestampMonotonic": "18342802", "Before": "xdm.service multi-user.target getty@tty1.service plymouth-quit.service shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2020-11-28 21:39:21 UTC", "ConditionTimestampMonotonic": "18342802", "ConfigurationDirectoryMode": "0755", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nagios-nrpe-ser 2020-11-28T21:41:21.098 INFO:teuthology.task.ansible.out:ver.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EnvironmentFile": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1297", "ExecMainStartTimestamp": "Sat 2020-11-28 21:39:21 UTC", "ExecMainStartTimestampMonotonic": "18343472", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2020-11-28 21:39:21 UTC", "InactiveExitTimestampMonotonic": "18343494", "InvocationID": "1b08eb1e2bd742209fed631bd4553220", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "i 2020-11-28T21:41:21.101 INFO:teuthology.task.ansible.out:nfinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1297", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nagios/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount system.slice sysinit.target", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no" 2020-11-28T21:41:21.103 INFO:teuthology.task.ansible.out:, "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2020-11-28 21:39:21 UTC", "StateChangeTimestampMonotonic": "18343494", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "109", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2020-11-28 21:39:21 UTC", "WatchdogTimestampMonotonic": "18343494", "WatchdogUSec": "0"}} TASK [common : Get SELinux status] ********************************************* skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:21.141 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:21 +0000 (0:00:00.064) 0:00:27.285 ***** 2020-11-28T21:41:21.175 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:21 +0000 (0:00:00.034) 0:00:27.320 ***** 2020-11-28T21:41:21.183 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:21.223 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:21 +0000 (0:00:00.047) 0:00:27.368 ***** 2020-11-28T21:41:21.230 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2020-11-28T21:41:21.232 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:21.269 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:21 +0000 (0:00:00.046) 0:00:27.414 ***** 2020-11-28T21:41:21.275 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:21.312 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:21 +0000 (0:00:00.043) 0:00:27.457 ***** 2020-11-28T21:41:21.318 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2020-11-28T21:41:21.320 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:21.366 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:21 +0000 (0:00:00.053) 0:00:27.511 ***** 2020-11-28T21:41:21.374 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:21.411 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:21 +0000 (0:00:00.044) 0:00:27.556 ***** 2020-11-28T21:41:21.418 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:21.457 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:21 +0000 (0:00:00.046) 0:00:27.602 ***** 2020-11-28T21:41:21.464 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:21.502 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:21 +0000 (0:00:00.043) 0:00:27.646 ***** 2020-11-28T21:41:21.515 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:21.551 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:21 +0000 (0:00:00.048) 0:00:27.695 ***** 2020-11-28T21:41:21.556 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:21.594 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:21 +0000 (0:00:00.043) 0:00:27.739 ***** 2020-11-28T21:41:21.600 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2020-11-28T21:41:21.602 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:21.644 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:21 +0000 (0:00:00.049) 0:00:27.788 ***** 2020-11-28T21:41:21.652 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2020-11-28T21:41:21.653 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => {"cache_update_time": 1606599631, "cache_updated": false, "changed": false} 2020-11-28T21:41:21.653 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:21.689 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:21 +0000 (0:00:00.045) 0:00:27.834 ***** 2020-11-28T21:41:21.725 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:21 +0000 (0:00:00.035) 0:00:27.869 ***** 2020-11-28T21:41:21.731 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2020-11-28T21:41:21.733 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:21.771 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:21 +0000 (0:00:00.046) 0:00:27.916 ***** 2020-11-28T21:41:21.807 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:21 +0000 (0:00:00.036) 0:00:27.952 ***** 2020-11-28T21:41:21.814 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2020-11-28T21:41:21.815 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"cache_update_time": 1606599641, "cache_updated": false, "changed": false} TASK [common : nrpe - Build SELinux policy package] **************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:21.853 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:21 +0000 (0:00:00.045) 0:00:27.998 ***** 2020-11-28T21:41:21.896 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:21 +0000 (0:00:00.043) 0:00:28.041 ***** 2020-11-28T21:41:21.904 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:21.906 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:21.961 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:21 +0000 (0:00:00.064) 0:00:28.105 ***** 2020-11-28T21:41:21.998 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:21 +0000 (0:00:00.037) 0:00:28.143 ***** 2020-11-28T21:41:22.006 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2020-11-28T21:41:22.006 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:22.048 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:22 +0000 (0:00:00.050) 0:00:28.193 ***** 2020-11-28T21:41:22.055 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2020-11-28T21:41:22.057 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:22.095 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:22 +0000 (0:00:00.046) 0:00:28.240 ***** 2020-11-28T21:41:22.103 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2020-11-28T21:41:22.105 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2020-11-28T21:41:22.145 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:22 +0000 (0:00:00.049) 0:00:28.290 ***** 2020-11-28T21:41:22.152 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2020-11-28T21:41:22.153 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:22.196 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:22 +0000 (0:00:00.051) 0:00:28.341 ***** 2020-11-28T21:41:22.231 INFO:teuthology.task.ansible.out: 2020-11-28T21:41:22.233 INFO:teuthology.task.ansible.out:TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** ok: [smithi192.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} TASK [common : Check for podman] *********************************************** fatal: [smithi096.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": "podman --version", "msg": "[Errno 2] No such file or directory", "rc": 2} ...ignoring 2020-11-28T21:41:22.273 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:22 +0000 (0:00:00.076) 0:00:28.418 ***** 2020-11-28T21:41:22.308 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:22 +0000 (0:00:00.034) 0:00:28.452 ***** 2020-11-28T21:41:22.314 INFO:teuthology.task.ansible.out: TASK [common : set_fact] ******************************************************* 2020-11-28T21:41:22.316 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:22.355 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:22 +0000 (0:00:00.047) 0:00:28.500 ***** 2020-11-28T21:41:22.372 INFO:teuthology.task.ansible.out: TASK [common : Check for podman] *********************************************** 2020-11-28T21:41:22.374 INFO:teuthology.task.ansible.out:fatal: [smithi118.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": "podman --version", "msg": "[Errno 2] No such file or directory", "rc": 2} ...ignoring 2020-11-28T21:41:22.424 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:22 +0000 (0:00:00.068) 0:00:28.569 ***** 2020-11-28T21:41:22.455 INFO:teuthology.task.ansible.out: TASK [common : set_fact] ******************************************************* 2020-11-28T21:41:22.455 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:22.495 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:22 +0000 (0:00:00.071) 0:00:28.640 ***** 2020-11-28T21:41:22.502 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2020-11-28T21:41:22.504 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2020-11-28T21:41:22.544 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:22 +0000 (0:00:00.046) 0:00:28.687 ***** 2020-11-28T21:41:22.644 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2020-11-28T21:41:22.697 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:22 +0000 (0:00:00.149) 0:00:28.837 ***** 2020-11-28T21:41:22.711 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ ok: [smithi162.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2020-11-28T21:41:22.826 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2020-11-28T21:41:22.870 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:22 +0000 (0:00:00.177) 0:00:29.015 ***** 2020-11-28T21:41:22.876 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2020-11-28T21:41:22.899 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2020-11-28T21:41:22.901 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:22.935 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:22 +0000 (0:00:00.065) 0:00:29.080 ***** 2020-11-28T21:41:22.963 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2020-11-28T21:41:22.965 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ ok: [smithi192.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2020-11-28T21:41:23.002 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:22 +0000 (0:00:00.066) 0:00:29.147 ***** 2020-11-28T21:41:23.036 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:23 +0000 (0:00:00.033) 0:00:29.181 ***** 2020-11-28T21:41:23.053 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:23.092 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:23 +0000 (0:00:00.055) 0:00:29.237 ***** 2020-11-28T21:41:23.099 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2020-11-28T21:41:23.102 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:23.143 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:23 +0000 (0:00:00.051) 0:00:29.288 ***** 2020-11-28T21:41:23.182 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** skipping: [smithi192.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-28T21:41:23.331 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:23 +0000 (0:00:00.187) 0:00:29.476 ***** 2020-11-28T21:41:23.358 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2020-11-28T21:41:23.360 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:23.395 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:23 +0000 (0:00:00.063) 0:00:29.540 ***** 2020-11-28T21:41:23.422 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2020-11-28T21:41:23.424 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2020-11-28T21:41:23.463 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:23 +0000 (0:00:00.067) 0:00:29.607 ***** 2020-11-28T21:41:23.685 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2020-11-28T21:41:23.686 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2020-11-28 21:39:43 UTC", "ActiveEnterTimestampMonotonic": "18194622", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "network.target -.mount remote-fs.target local-fs.target nss-lookup.target basic.target time-sync.target systemd-tmpfiles-setup.service sysinit.target var-run.mount systemd-journald.socket system.slice", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2020-11-28 21:39:43 UTC", "AssertTimestampMonotonic": "18193925", "Before": "xdm.service multi-user.target shutdown.target plymouth-quit.service getty@tty1.service", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2020-11-28 21:39:43 UTC", "ConditionTimestampMonotonic": "18193925", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target nrpe.socket", "ControlGroup": "/system.slice/nagios-nrpe-ser 2020-11-28T21:41:23.688 INFO:teuthology.task.ansible.out:ver.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EnvironmentFile": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1328", "ExecMainStartTimestamp": "Sat 2020-11-28 21:39:43 UTC", "ExecMainStartTimestampMonotonic": "18194594", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2020-11-28 21:39:43 UTC", "InactiveExitTimestampMonotonic": "18194622", "InvocationID": "f2664f3f346d49a281f0b94b5c8a8c9f", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "i 2020-11-28T21:41:23.689 INFO:teuthology.task.ansible.out:nfinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1328", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nagios/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target system.slice -.mount", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no" 2020-11-28T21:41:23.691 INFO:teuthology.task.ansible.out:, "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2020-11-28 21:39:43 UTC", "StateChangeTimestampMonotonic": "18194622", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "109", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2020-11-28 21:39:43 UTC", "WatchdogTimestampMonotonic": "18194622", "WatchdogUSec": "0"}} 2020-11-28T21:41:23.729 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:23 +0000 (0:00:00.266) 0:00:29.874 ***** 2020-11-28T21:41:23.736 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2020-11-28T21:41:23.738 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2020-11-28T21:41:23.778 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:23 +0000 (0:00:00.046) 0:00:29.920 ***** 2020-11-28T21:41:23.781 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2020-11-28T21:41:23.783 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:23.824 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:23 +0000 (0:00:00.048) 0:00:29.969 ***** 2020-11-28T21:41:23.851 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:23.887 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:23 +0000 (0:00:00.063) 0:00:30.032 ***** 2020-11-28T21:41:23.918 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:23.955 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:23 +0000 (0:00:00.067) 0:00:30.100 ***** 2020-11-28T21:41:23.983 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2020-11-28T21:41:23.990 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:24.026 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:24 +0000 (0:00:00.070) 0:00:30.170 ***** 2020-11-28T21:41:24.034 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2020-11-28T21:41:24.036 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2020-11-28 21:39:33 UTC", "ActiveEnterTimestampMonotonic": "18347542", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "-.mount basic.target network.target time-sync.target systemd-tmpfiles-setup.service var-run.mount systemd-journald.socket remote-fs.target system.slice local-fs.target nss-lookup.target sysinit.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2020-11-28 21:39:33 UTC", "AssertTimestampMonotonic": "18346278", "Before": "shutdown.target plymouth-quit.service getty@tty1.service xdm.service multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2020-11-28 21:39:33 UTC", "ConditionTimestampMonotonic": "18346278", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target nrpe.socket", "ControlGroup": "/system.slice/nagios-nrpe-ser 2020-11-28T21:41:24.038 INFO:teuthology.task.ansible.out:ver.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EnvironmentFile": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1300", "ExecMainStartTimestamp": "Sat 2020-11-28 21:39:33 UTC", "ExecMainStartTimestampMonotonic": "18347509", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2020-11-28 21:39:33 UTC", "InactiveExitTimestampMonotonic": "18347542", "InvocationID": "b963991d34724d139a75116c2b158353", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "i 2020-11-28T21:41:24.040 INFO:teuthology.task.ansible.out:nfinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1300", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nagios/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target -.mount", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no" 2020-11-28T21:41:24.041 INFO:teuthology.task.ansible.out:, "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2020-11-28 21:39:33 UTC", "StateChangeTimestampMonotonic": "18347542", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "109", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2020-11-28 21:39:33 UTC", "WatchdogTimestampMonotonic": "18347541", "WatchdogUSec": "0"}} 2020-11-28T21:41:24.083 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:24 +0000 (0:00:00.057) 0:00:30.227 ***** 2020-11-28T21:41:24.090 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2020-11-28T21:41:24.092 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:24.135 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:24 +0000 (0:00:00.051) 0:00:30.279 ***** 2020-11-28T21:41:24.141 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2020-11-28T21:41:24.143 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:24.181 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:24 +0000 (0:00:00.045) 0:00:30.325 ***** 2020-11-28T21:41:24.188 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2020-11-28T21:41:24.190 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:24.230 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:24 +0000 (0:00:00.049) 0:00:30.375 ***** 2020-11-28T21:41:24.238 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2020-11-28T21:41:24.240 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:24.280 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:24 +0000 (0:00:00.049) 0:00:30.425 ***** 2020-11-28T21:41:24.290 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2020-11-28T21:41:24.292 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:24.329 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:24 +0000 (0:00:00.048) 0:00:30.474 ***** 2020-11-28T21:41:24.338 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2020-11-28T21:41:24.340 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:24.377 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:24 +0000 (0:00:00.048) 0:00:30.522 ***** 2020-11-28T21:41:24.383 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2020-11-28T21:41:24.385 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:24.429 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:24 +0000 (0:00:00.052) 0:00:30.574 ***** 2020-11-28T21:41:24.438 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:24.476 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:24 +0000 (0:00:00.046) 0:00:30.620 ***** 2020-11-28T21:41:24.482 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:24.525 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:24 +0000 (0:00:00.047) 0:00:30.668 ***** 2020-11-28T21:41:24.530 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:24.570 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:24 +0000 (0:00:00.043) 0:00:30.711 ***** 2020-11-28T21:41:24.572 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2020-11-28T21:41:24.574 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:24.610 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:24 +0000 (0:00:00.043) 0:00:30.755 ***** 2020-11-28T21:41:24.617 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2020-11-28T21:41:24.618 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:24.658 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:24 +0000 (0:00:00.046) 0:00:30.801 ***** 2020-11-28T21:41:24.683 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2020-11-28T21:41:24.685 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:24.720 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:24 +0000 (0:00:00.063) 0:00:30.864 ***** 2020-11-28T21:41:24.750 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:24.791 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:24 +0000 (0:00:00.071) 0:00:30.936 ***** 2020-11-28T21:41:24.798 INFO:teuthology.task.ansible.out: TASK [common : Check for podman] *********************************************** 2020-11-28T21:41:24.800 INFO:teuthology.task.ansible.out:fatal: [smithi162.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": "podman --version", "msg": "[Errno 2] No such file or directory", "rc": 2} ...ignoring 2020-11-28T21:41:24.844 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:24 +0000 (0:00:00.051) 0:00:30.988 ***** 2020-11-28T21:41:24.852 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:24.918 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:24 +0000 (0:00:00.074) 0:00:31.062 ***** 2020-11-28T21:41:24.928 INFO:teuthology.task.ansible.out: TASK [common : set_fact] ******************************************************* 2020-11-28T21:41:24.930 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:24.971 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:24 +0000 (0:00:00.051) 0:00:31.114 ***** 2020-11-28T21:41:24.976 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2020-11-28T21:41:24.977 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:25.023 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:25 +0000 (0:00:00.053) 0:00:31.167 ***** 2020-11-28T21:41:25.174 INFO:teuthology.task.ansible.out: TASK [common : Check for podman] *********************************************** 2020-11-28T21:41:25.175 INFO:teuthology.task.ansible.out:fatal: [smithi192.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": "podman --version", "msg": "[Errno 2] No such file or directory", "rc": 2} ...ignoring 2020-11-28T21:41:25.210 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:25 +0000 (0:00:00.187) 0:00:31.354 ***** 2020-11-28T21:41:25.237 INFO:teuthology.task.ansible.out: TASK [common : set_fact] ******************************************************* skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:25.277 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:25 +0000 (0:00:00.067) 0:00:31.422 ***** 2020-11-28T21:41:26.991 INFO:teuthology.task.ansible.out: TASK [common : Check for docker] *********************************************** 2020-11-28T21:41:26.994 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "cmd": ["docker", "--version"], "delta": "0:00:04.477310", "end": "2020-11-28 21:41:27.368258", "rc": 0, "start": "2020-11-28 21:41:22.890948", "stderr": "", "stderr_lines": [], "stdout": "Docker version 19.03.6, build 369ce74a3c", "stdout_lines": ["Docker version 19.03.6, build 369ce74a3c"]} changed: [smithi118.front.sepia.ceph.com] => {"changed": true, "cmd": ["docker", "--version"], "delta": "0:00:04.348844", "end": "2020-11-28 21:41:27.055601", "rc": 0, "start": "2020-11-28 21:41:22.706757", "stderr": "", "stderr_lines": [], "stdout": "Docker version 19.03.6, build 369ce74a3c", "stdout_lines": ["Docker version 19.03.6, build 369ce74a3c"]} 2020-11-28T21:41:27.029 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:27 +0000 (0:00:01.751) 0:00:33.174 ***** 2020-11-28T21:41:27.065 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:27 +0000 (0:00:00.035) 0:00:33.210 ***** 2020-11-28T21:41:27.072 INFO:teuthology.task.ansible.out: TASK [common : set_fact] ******************************************************* 2020-11-28T21:41:27.074 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"ansible_facts": {"container_mirror_cert_path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000"}, "changed": false} 2020-11-28T21:41:27.110 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:27 +0000 (0:00:00.045) 0:00:33.255 ***** 2020-11-28T21:41:27.117 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"ansible_facts": {"container_mirror_cert_path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000"}, "changed": false} 2020-11-28T21:41:27.154 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:27 +0000 (0:00:00.044) 0:00:33.299 ***** 2020-11-28T21:41:27.175 INFO:teuthology.task.ansible.out: TASK [common : Include encrypted variables] ************************************ ok: [smithi096.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} ok: [smithi096.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-28T21:41:27.220 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:27 +0000 (0:00:00.065) 0:00:33.365 ***** 2020-11-28T21:41:27.228 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-28T21:41:27.230 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-28T21:41:27.273 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:27 +0000 (0:00:00.052) 0:00:33.418 ***** 2020-11-28T21:41:27.354 INFO:teuthology.task.ansible.out: TASK [common : Create /etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000] *** 2020-11-28T21:41:27.356 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2020-11-28T21:41:27.393 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:27 +0000 (0:00:00.119) 0:00:33.537 ***** 2020-11-28T21:41:27.421 INFO:teuthology.task.ansible.out:changed: [smithi118.front.sepia.ceph.com] => {"changed": true, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2020-11-28T21:41:27.461 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:27 +0000 (0:00:00.068) 0:00:33.606 ***** 2020-11-28T21:41:27.750 INFO:teuthology.task.ansible.out: TASK [common : Copy docker-mirror.front.sepia.ceph.com:5000 self-signed cert] *** 2020-11-28T21:41:27.753 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "checksum": "d386a5d594d8bb6c408cf08541a4eb609577a03d", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "md5sum": "e2cf8b6a8ad65ba91ea9fc53d49f1bf4", "mode": "0644", "owner": "root", "size": 2236, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1606599687.4202-105996694423364/source", "state": "file", "uid": 0} 2020-11-28T21:41:27.788 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:27 +0000 (0:00:00.326) 0:00:33.932 ***** 2020-11-28T21:41:27.821 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2020-11-28T21:41:27.826 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2020-11-28T21:41:27.889 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:27 +0000 (0:00:00.101) 0:00:34.034 ***** 2020-11-28T21:41:27.897 INFO:teuthology.task.ansible.out: TASK [common : Copy docker-mirror.front.sepia.ceph.com:5000 self-signed cert] *** 2020-11-28T21:41:27.899 INFO:teuthology.task.ansible.out:changed: [smithi118.front.sepia.ceph.com] => {"changed": true, "checksum": "d386a5d594d8bb6c408cf08541a4eb609577a03d", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "md5sum": "e2cf8b6a8ad65ba91ea9fc53d49f1bf4", "mode": "0644", "owner": "root", "size": 2236, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1606599687.4945197-102472726161371/source", "state": "file", "uid": 0} 2020-11-28T21:41:27.953 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:27 +0000 (0:00:00.063) 0:00:34.097 ***** 2020-11-28T21:41:27.960 INFO:teuthology.task.ansible.out: 2020-11-28T21:41:27.980 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2020-11-28T21:41:27.982 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "collectl", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2020-11-28T21:41:27.990 INFO:teuthology.task.ansible.out: 2020-11-28T21:41:28.041 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:28 +0000 (0:00:00.088) 0:00:34.186 ***** 2020-11-28T21:41:28.048 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2020-11-28T21:41:28.050 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2020-11-28T21:41:28.096 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:28 +0000 (0:00:00.054) 0:00:34.241 ***** 2020-11-28T21:41:28.103 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2020-11-28T21:41:28.105 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3", "libssl1.0.0"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "python-virtualenv", "python-configobj", "python-gevent", "python-numpy", "python-matplotlib", "python-nose", "btrfs-tools", "lttng-tools", "libtool-bin", "docker.io", "python3-nose"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"} 2020-11-28T21:41:28.158 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:28 +0000 (0:00:00.061) 0:00:34.302 ***** 2020-11-28T21:41:28.165 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2020-11-28T21:41:28.167 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "collectl", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2020-11-28T21:41:28.215 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:28 +0000 (0:00:00.057) 0:00:34.360 ***** 2020-11-28T21:41:28.226 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2020-11-28T21:41:28.228 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-11-28T21:41:28.272 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:28 +0000 (0:00:00.056) 0:00:34.416 ***** 2020-11-28T21:41:28.278 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2020-11-28T21:41:28.280 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3", "libssl1.0.0"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "python-virtualenv", "python-configobj", "python-gevent", "python-numpy", "python-matplotlib", "python-nose", "btrfs-tools", "lttng-tools", "libtool-bin", "docker.io", "python3-nose"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"} 2020-11-28T21:41:28.322 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:28 +0000 (0:00:00.050) 0:00:34.467 ***** 2020-11-28T21:41:28.377 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2020-11-28T21:41:28.379 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-11-28T21:41:28.431 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:28 +0000 (0:00:00.108) 0:00:34.576 ***** 2020-11-28T21:41:28.438 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2020-11-28T21:41:28.439 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2020-11-28T21:41:28.499 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:28 +0000 (0:00:00.067) 0:00:34.644 ***** 2020-11-28T21:41:28.589 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2020-11-28T21:41:28.642 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:28 +0000 (0:00:00.142) 0:00:34.786 ***** 2020-11-28T21:41:28.659 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "gid": 1235, "name": "ubuntu", "state": "present", "system": false} 2020-11-28T21:41:28.705 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:28 +0000 (0:00:00.061) 0:00:34.847 ***** 2020-11-28T21:41:28.741 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:28.778 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:28 +0000 (0:00:00.075) 0:00:34.922 ***** 2020-11-28T21:41:28.778 INFO:teuthology.task.ansible.out: 2020-11-28T21:41:28.802 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* ok: [smithi118.front.sepia.ceph.com] => {"changed": false, "gid": 1235, "name": "ubuntu", "state": "present", "system": false} 2020-11-28T21:41:28.837 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:28 +0000 (0:00:00.059) 0:00:34.982 ***** 2020-11-28T21:41:28.844 INFO:teuthology.task.ansible.out: 2020-11-28T21:41:28.846 INFO:teuthology.task.ansible.out:TASK [testnode : Delete the teuthology users password.] ************************ skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:28.883 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:28 +0000 (0:00:00.046) 0:00:35.028 ***** 2020-11-28T21:41:28.891 INFO:teuthology.task.ansible.out: 2020-11-28T21:41:28.892 INFO:teuthology.task.ansible.out:TASK [testnode : Create the teuthology user.] ********************************** 2020-11-28T21:41:28.894 INFO:teuthology.task.ansible.out: skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:28.932 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:28 +0000 (0:00:00.048) 0:00:35.077 ***** 2020-11-28T21:41:28.971 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:29.008 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:29 +0000 (0:00:00.076) 0:00:35.153 ***** 2020-11-28T21:41:29.052 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2020-11-28T21:41:29.053 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2020-11-28T21:41:29.110 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:29 +0000 (0:00:00.101) 0:00:35.254 ***** 2020-11-28T21:41:29.182 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2020-11-28T21:41:29.223 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:29 +0000 (0:00:00.113) 0:00:35.368 ***** 2020-11-28T21:41:29.493 INFO:teuthology.task.ansible.out: TASK [common : Check for docker] *********************************************** changed: [smithi162.front.sepia.ceph.com] => {"changed": true, "cmd": ["docker", "--version"], "delta": "0:00:04.375111", "end": "2020-11-28 21:41:29.878690", "rc": 0, "start": "2020-11-28 21:41:25.503579", "stderr": "", "stderr_lines": [], "stdout": "Docker version 19.03.6, build 369ce74a3c", "stdout_lines": ["Docker version 19.03.6, build 369ce74a3c"]} 2020-11-28T21:41:29.533 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:29 +0000 (0:00:00.309) 0:00:35.677 ***** 2020-11-28T21:41:29.561 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2020-11-28T21:41:29.567 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2020-11-28T21:41:29.605 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:29 +0000 (0:00:00.072) 0:00:35.750 ***** 2020-11-28T21:41:29.614 INFO:teuthology.task.ansible.out: TASK [common : set_fact] ******************************************************* ok: [smithi162.front.sepia.ceph.com] => {"ansible_facts": {"container_mirror_cert_path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000"}, "changed": false} 2020-11-28T21:41:29.688 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:29 +0000 (0:00:00.083) 0:00:35.833 ***** 2020-11-28T21:41:29.703 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2020-11-28T21:41:29.705 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi096"}, "changed": false} 2020-11-28T21:41:29.705 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** ok: [smithi118.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2020-11-28T21:41:29.750 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:29 +0000 (0:00:00.061) 0:00:35.895 ***** 2020-11-28T21:41:29.788 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:29 +0000 (0:00:00.037) 0:00:35.932 ***** 2020-11-28T21:41:29.794 INFO:teuthology.task.ansible.out: TASK [common : Include encrypted variables] ************************************ 2020-11-28T21:41:29.796 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-28T21:41:29.808 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-28T21:41:29.850 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:29 +0000 (0:00:00.060) 0:00:35.993 ***** 2020-11-28T21:41:29.855 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2020-11-28T21:41:29.857 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi118"}, "changed": false} 2020-11-28T21:41:29.897 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:29 +0000 (0:00:00.049) 0:00:36.042 ***** 2020-11-28T21:41:29.960 INFO:teuthology.task.ansible.out: TASK [common : Check for docker] *********************************************** 2020-11-28T21:41:29.962 INFO:teuthology.task.ansible.out:changed: [smithi192.front.sepia.ceph.com] => {"changed": true, "cmd": ["docker", "--version"], "delta": "0:00:04.529998", "end": "2020-11-28 21:41:30.257992", "rc": 0, "start": "2020-11-28 21:41:25.727994", "stderr": "", "stderr_lines": [], "stdout": "Docker version 19.03.6, build 369ce74a3c", "stdout_lines": ["Docker version 19.03.6, build 369ce74a3c"]} 2020-11-28T21:41:30.015 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:30 +0000 (0:00:00.116) 0:00:36.159 ***** 2020-11-28T21:41:30.021 INFO:teuthology.task.ansible.out: TASK [common : Create /etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000] *** 2020-11-28T21:41:30.023 INFO:teuthology.task.ansible.out:changed: [smithi162.front.sepia.ceph.com] => {"changed": true, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2020-11-28T21:41:30.066 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:30 +0000 (0:00:00.051) 0:00:36.211 ***** 2020-11-28T21:41:30.074 INFO:teuthology.task.ansible.out: TASK [common : set_fact] ******************************************************* 2020-11-28T21:41:30.076 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"ansible_facts": {"container_mirror_cert_path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000"}, "changed": false} 2020-11-28T21:41:30.119 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:30 +0000 (0:00:00.052) 0:00:36.263 ***** 2020-11-28T21:41:30.266 INFO:teuthology.task.ansible.out: TASK [common : Include encrypted variables] ************************************ 2020-11-28T21:41:30.267 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} ok: [smithi192.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-28T21:41:30.335 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:30 +0000 (0:00:00.214) 0:00:36.478 ***** 2020-11-28T21:41:30.490 INFO:teuthology.task.ansible.out: TASK [common : Create /etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000] *** changed: [smithi192.front.sepia.ceph.com] => {"changed": true, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2020-11-28T21:41:30.526 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:30 +0000 (0:00:00.193) 0:00:36.671 ***** 2020-11-28T21:41:30.532 INFO:teuthology.task.ansible.out: TASK [common : Copy docker-mirror.front.sepia.ceph.com:5000 self-signed cert] *** 2020-11-28T21:41:30.534 INFO:teuthology.task.ansible.out:changed: [smithi162.front.sepia.ceph.com] => {"changed": true, "checksum": "d386a5d594d8bb6c408cf08541a4eb609577a03d", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "md5sum": "e2cf8b6a8ad65ba91ea9fc53d49f1bf4", "mode": "0644", "owner": "root", "size": 2236, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1606599690.0959399-134415255493964/source", "state": "file", "uid": 0} 2020-11-28T21:41:30.575 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:30 +0000 (0:00:00.048) 0:00:36.720 ***** 2020-11-28T21:41:30.603 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2020-11-28T21:41:30.605 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2020-11-28T21:41:30.640 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:30 +0000 (0:00:00.064) 0:00:36.784 ***** 2020-11-28T21:41:30.685 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2020-11-28T21:41:30.687 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "collectl", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2020-11-28T21:41:30.725 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:30 +0000 (0:00:00.084) 0:00:36.869 ***** 2020-11-28T21:41:30.752 INFO:teuthology.task.ansible.out: 2020-11-28T21:41:30.754 INFO:teuthology.task.ansible.out:TASK [testnode : Set the system's hostname] ************************************ ok: [smithi118.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "", "ansible_fqdn": "smithi118", "ansible_hostname": "smithi118", "ansible_nodename": "smithi118"}, "changed": false, "name": "smithi118"} 2020-11-28T21:41:30.792 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:30 +0000 (0:00:00.066) 0:00:36.936 ***** 2020-11-28T21:41:30.799 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2020-11-28T21:41:30.801 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3", "libssl1.0.0"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "python-virtualenv", "python-configobj", "python-gevent", "python-numpy", "python-matplotlib", "python-nose", "btrfs-tools", "lttng-tools", "libtool-bin", "docker.io", "python3-nose"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"} 2020-11-28T21:41:30.801 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ ok: [smithi096.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "", "ansible_fqdn": "smithi096", "ansible_hostname": "smithi096", "ansible_nodename": "smithi096"}, "changed": false, "name": "smithi096"} 2020-11-28T21:41:30.861 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:30 +0000 (0:00:00.066) 0:00:37.003 ***** 2020-11-28T21:41:30.896 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:30 +0000 (0:00:00.037) 0:00:37.041 ***** 2020-11-28T21:41:30.906 INFO:teuthology.task.ansible.out: TASK [common : Copy docker-mirror.front.sepia.ceph.com:5000 self-signed cert] *** changed: [smithi192.front.sepia.ceph.com] => {"changed": true, "checksum": "d386a5d594d8bb6c408cf08541a4eb609577a03d", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "md5sum": "e2cf8b6a8ad65ba91ea9fc53d49f1bf4", "mode": "0644", "owner": "root", "size": 2236, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1606599690.5590968-262933322982818/source", "state": "file", "uid": 0} 2020-11-28T21:41:30.945 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:30 +0000 (0:00:00.048) 0:00:37.090 ***** 2020-11-28T21:41:30.951 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2020-11-28T21:41:30.953 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-11-28T21:41:31.002 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:30 +0000 (0:00:00.057) 0:00:37.147 ***** 2020-11-28T21:41:31.009 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** ok: [smithi192.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2020-11-28T21:41:31.055 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:31 +0000 (0:00:00.053) 0:00:37.200 ***** 2020-11-28T21:41:31.095 INFO:teuthology.task.ansible.out: 2020-11-28T21:41:31.097 INFO:teuthology.task.ansible.out:TASK [testnode : Including distro specific variables.] ************************* ok: [smithi192.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "collectl", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2020-11-28T21:41:31.141 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:31 +0000 (0:00:00.085) 0:00:37.285 ***** 2020-11-28T21:41:31.147 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2020-11-28T21:41:31.149 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} TASK [testnode : Ensure the sudo group exists.] ******************************** ok: [smithi162.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2020-11-28T21:41:31.190 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:31 +0000 (0:00:00.049) 0:00:37.334 ***** 2020-11-28T21:41:31.223 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:31 +0000 (0:00:00.033) 0:00:37.368 ***** 2020-11-28T21:41:31.229 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2020-11-28T21:41:31.231 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3", "libssl1.0.0"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "python-virtualenv", "python-configobj", "python-gevent", "python-numpy", "python-matplotlib", "python-nose", "btrfs-tools", "lttng-tools", "libtool-bin", "docker.io", "python3-nose"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"} 2020-11-28T21:41:31.231 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2020-11-28T21:41:31.232 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2020-11-28T21:41:31.270 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:31 +0000 (0:00:00.046) 0:00:37.415 ***** 2020-11-28T21:41:31.309 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:31 +0000 (0:00:00.038) 0:00:37.453 ***** 2020-11-28T21:41:31.359 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2020-11-28T21:41:31.361 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => {"changed": false, "gid": 1235, "name": "ubuntu", "state": "present", "system": false} TASK [testnode : Including version specific variables.] ************************ ok: [smithi192.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-11-28T21:41:31.399 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:31 +0000 (0:00:00.089) 0:00:37.543 ***** 2020-11-28T21:41:31.445 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:31 +0000 (0:00:00.043) 0:00:37.587 ***** 2020-11-28T21:41:31.449 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2020-11-28T21:41:31.451 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:31.500 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:31 +0000 (0:00:00.057) 0:00:37.645 ***** 2020-11-28T21:41:31.543 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2020-11-28T21:41:31.545 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload ssh_config] ******************************************** ok: [smithi118.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2020-11-28T21:41:31.590 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:31 +0000 (0:00:00.090) 0:00:37.735 ***** 2020-11-28T21:41:31.631 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:31 +0000 (0:00:00.040) 0:00:37.776 ***** 2020-11-28T21:41:31.640 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2020-11-28T21:41:31.641 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} TASK [testnode : Upload ssh_config] ******************************************** ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2020-11-28T21:41:31.710 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:31 +0000 (0:00:00.078) 0:00:37.854 ***** 2020-11-28T21:41:31.758 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:31 +0000 (0:00:00.048) 0:00:37.903 ***** 2020-11-28T21:41:31.799 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2020-11-28T21:41:31.800 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2020-11-28T21:41:31.842 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:31 +0000 (0:00:00.084) 0:00:37.987 ***** 2020-11-28T21:41:31.920 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2020-11-28T21:41:31.921 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"changed": false, "gid": 1235, "name": "ubuntu", "state": "present", "system": false} 2020-11-28T21:41:31.964 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:31 +0000 (0:00:00.121) 0:00:38.109 ***** 2020-11-28T21:41:31.993 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:32.034 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:32 +0000 (0:00:00.070) 0:00:38.179 ***** 2020-11-28T21:41:32.061 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2020-11-28T21:41:32.063 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:32.101 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:32 +0000 (0:00:00.066) 0:00:38.245 ***** 2020-11-28T21:41:32.119 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** ok: [smithi118.front.sepia.ceph.com] => {"attempts": 1, "changed": false, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2020-11-28T21:41:32.190 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:32 +0000 (0:00:00.087) 0:00:38.333 ***** 2020-11-28T21:41:32.217 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:32.252 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:32 +0000 (0:00:00.063) 0:00:38.396 ***** 2020-11-28T21:41:32.258 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** ok: [smithi096.front.sepia.ceph.com] => {"attempts": 1, "changed": false, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2020-11-28T21:41:32.260 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** ok: [smithi162.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2020-11-28T21:41:32.296 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:32 +0000 (0:00:00.044) 0:00:38.440 ***** 2020-11-28T21:41:32.334 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:32 +0000 (0:00:00.038) 0:00:38.479 ***** 2020-11-28T21:41:32.341 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2020-11-28T21:41:32.343 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} TASK [testnode : Set mode on /etc/fuse.conf] *********************************** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create remote.conf] ******************************************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:32.379 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:32 +0000 (0:00:00.045) 0:00:38.524 ***** 2020-11-28T21:41:32.413 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:32 +0000 (0:00:00.034) 0:00:38.558 ***** 2020-11-28T21:41:32.446 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:32 +0000 (0:00:00.033) 0:00:38.591 ***** 2020-11-28T21:41:32.453 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2020-11-28T21:41:32.455 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi162"}, "changed": false} TASK [testnode : Set mode on /etc/fuse.conf] *********************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure the group kvm exists.] ********************************* skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:32.490 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:32 +0000 (0:00:00.044) 0:00:38.635 ***** 2020-11-28T21:41:32.523 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:32 +0000 (0:00:00.033) 0:00:38.668 ***** 2020-11-28T21:41:32.557 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:32 +0000 (0:00:00.033) 0:00:38.701 ***** 2020-11-28T21:41:32.563 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:32.568 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:32.602 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:32 +0000 (0:00:00.044) 0:00:38.746 ***** 2020-11-28T21:41:32.669 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:32 +0000 (0:00:00.067) 0:00:38.814 ***** 2020-11-28T21:41:32.676 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:32.732 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:32 +0000 (0:00:00.062) 0:00:38.877 ***** 2020-11-28T21:41:32.739 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-11-28T21:41:32.741 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:32.783 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:32 +0000 (0:00:00.048) 0:00:38.925 ***** 2020-11-28T21:41:32.789 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:32.826 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:32 +0000 (0:00:00.045) 0:00:38.971 ***** 2020-11-28T21:41:32.832 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-11-28T21:41:32.834 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:32.872 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:32 +0000 (0:00:00.044) 0:00:39.015 ***** 2020-11-28T21:41:32.878 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:32.918 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:32 +0000 (0:00:00.046) 0:00:39.062 ***** 2020-11-28T21:41:32.924 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2020-11-28T21:41:32.926 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} TASK [testnode : Rebuild rpmdb] ************************************************ skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:32.974 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:32 +0000 (0:00:00.056) 0:00:39.118 ***** 2020-11-28T21:41:33.009 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:33 +0000 (0:00:00.035) 0:00:39.154 ***** 2020-11-28T21:41:33.017 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Configure local mirrorlists] ********************************** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set the system's hostname] ************************************ ok: [smithi162.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "", "ansible_fqdn": "smithi162", "ansible_hostname": "smithi162", "ansible_nodename": "smithi162"}, "changed": false, "name": "smithi162"} 2020-11-28T21:41:33.058 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:33 +0000 (0:00:00.047) 0:00:39.201 ***** 2020-11-28T21:41:33.093 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:33 +0000 (0:00:00.036) 0:00:39.238 ***** 2020-11-28T21:41:33.130 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:33 +0000 (0:00:00.036) 0:00:39.274 ***** 2020-11-28T21:41:33.137 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2020-11-28T21:41:33.138 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi192"}, "changed": false} 2020-11-28T21:41:33.139 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2020-11-28T21:41:33.187 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:33 +0000 (0:00:00.055) 0:00:39.330 ***** 2020-11-28T21:41:33.228 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:33 +0000 (0:00:00.042) 0:00:39.372 ***** 2020-11-28T21:41:33.264 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:33 +0000 (0:00:00.036) 0:00:39.409 ***** 2020-11-28T21:41:33.271 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-11-28T21:41:33.309 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:33 +0000 (0:00:00.044) 0:00:39.454 ***** 2020-11-28T21:41:33.345 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:33 +0000 (0:00:00.036) 0:00:39.490 ***** 2020-11-28T21:41:33.392 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:33 +0000 (0:00:00.046) 0:00:39.537 ***** 2020-11-28T21:41:33.399 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2020-11-28T21:41:33.437 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:33 +0000 (0:00:00.044) 0:00:39.581 ***** 2020-11-28T21:41:33.483 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:33 +0000 (0:00:00.046) 0:00:39.628 ***** 2020-11-28T21:41:33.490 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2020-11-28T21:41:33.492 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} TASK [testnode : Install GPG] ************************************************** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:33.528 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:33 +0000 (0:00:00.044) 0:00:39.672 ***** 2020-11-28T21:41:33.567 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:33 +0000 (0:00:00.040) 0:00:39.712 ***** 2020-11-28T21:41:33.574 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:33.613 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:33 +0000 (0:00:00.045) 0:00:39.758 ***** 2020-11-28T21:41:33.619 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2020-11-28T21:41:33.621 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:33.660 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:33 +0000 (0:00:00.046) 0:00:39.805 ***** 2020-11-28T21:41:33.667 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:33.669 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:33.704 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:33 +0000 (0:00:00.043) 0:00:39.848 ***** 2020-11-28T21:41:33.710 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2020-11-28T21:41:33.711 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:33.747 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:33 +0000 (0:00:00.043) 0:00:39.892 ***** 2020-11-28T21:41:33.754 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:33.755 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2020-11-28T21:41:33.756 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "", "ansible_fqdn": "smithi192", "ansible_hostname": "smithi192", "ansible_nodename": "smithi192"}, "changed": false, "name": "smithi192"} 2020-11-28T21:41:33.794 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:33 +0000 (0:00:00.046) 0:00:39.939 ***** 2020-11-28T21:41:33.828 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:33 +0000 (0:00:00.034) 0:00:39.973 ***** 2020-11-28T21:41:33.835 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-28T21:41:33.837 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:33.875 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:33 +0000 (0:00:00.047) 0:00:40.020 ***** 2020-11-28T21:41:33.911 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:33 +0000 (0:00:00.035) 0:00:40.056 ***** 2020-11-28T21:41:33.923 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** ok: [smithi162.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} TASK [testnode : Ensure yum_utils is present.] ********************************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:33.956 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:33 +0000 (0:00:00.045) 0:00:40.101 ***** 2020-11-28T21:41:33.991 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:33 +0000 (0:00:00.034) 0:00:40.136 ***** 2020-11-28T21:41:33.998 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:33.999 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:34.044 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:34 +0000 (0:00:00.053) 0:00:40.189 ***** 2020-11-28T21:41:34.079 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:34 +0000 (0:00:00.035) 0:00:40.224 ***** 2020-11-28T21:41:34.087 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2020-11-28T21:41:34.089 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:34.132 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:34 +0000 (0:00:00.052) 0:00:40.277 ***** 2020-11-28T21:41:34.134 INFO:teuthology.task.ansible.out: 2020-11-28T21:41:34.143 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2020-11-28T21:41:34.145 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:34.193 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:34 +0000 (0:00:00.060) 0:00:40.338 ***** 2020-11-28T21:41:34.204 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2020-11-28T21:41:34.206 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:34.244 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:34 +0000 (0:00:00.051) 0:00:40.389 ***** 2020-11-28T21:41:34.280 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:34 +0000 (0:00:00.035) 0:00:40.424 ***** 2020-11-28T21:41:34.286 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2020-11-28T21:41:34.288 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure ceph packages are not present.] ************************ skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:34.326 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:34 +0000 (0:00:00.046) 0:00:40.471 ***** 2020-11-28T21:41:34.362 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:34 +0000 (0:00:00.034) 0:00:40.506 ***** 2020-11-28T21:41:34.369 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-11-28T21:41:34.370 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:34.405 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:34 +0000 (0:00:00.043) 0:00:40.549 ***** 2020-11-28T21:41:34.411 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2020-11-28T21:41:34.413 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:34.450 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:34 +0000 (0:00:00.045) 0:00:40.594 ***** 2020-11-28T21:41:34.456 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-28T21:41:34.458 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:34.495 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:34 +0000 (0:00:00.045) 0:00:40.640 ***** 2020-11-28T21:41:34.503 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2020-11-28T21:41:34.505 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => {"attempts": 1, "changed": false, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} TASK [testnode : Ensure ceph packages are not present.] ************************ skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:34.544 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:34 +0000 (0:00:00.047) 0:00:40.688 ***** 2020-11-28T21:41:34.579 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:34 +0000 (0:00:00.035) 0:00:40.723 ***** 2020-11-28T21:41:34.586 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2020-11-28T21:41:34.587 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure ceph dependency packages are not present.] ************* skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:34.624 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:34 +0000 (0:00:00.045) 0:00:40.769 ***** 2020-11-28T21:41:34.663 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:34 +0000 (0:00:00.038) 0:00:40.808 ***** 2020-11-28T21:41:34.670 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** ok: [smithi192.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2020-11-28T21:41:34.672 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Remove packages] ********************************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:34.712 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:34 +0000 (0:00:00.048) 0:00:40.856 ***** 2020-11-28T21:41:34.747 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:34 +0000 (0:00:00.035) 0:00:40.892 ***** 2020-11-28T21:41:34.782 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:34 +0000 (0:00:00.035) 0:00:40.927 ***** 2020-11-28T21:41:34.788 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:34.790 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set mode on /etc/fuse.conf] *********************************** skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:34.829 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:34 +0000 (0:00:00.046) 0:00:40.973 ***** 2020-11-28T21:41:34.869 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:34 +0000 (0:00:00.040) 0:00:41.014 ***** 2020-11-28T21:41:34.903 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:34 +0000 (0:00:00.033) 0:00:41.047 ***** 2020-11-28T21:41:34.909 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2020-11-28T21:41:34.910 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install epel packages] **************************************** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:34.946 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:34 +0000 (0:00:00.043) 0:00:41.091 ***** 2020-11-28T21:41:34.980 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:34 +0000 (0:00:00.033) 0:00:41.124 ***** 2020-11-28T21:41:34.986 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-11-28T21:41:34.988 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Stop and disable iptables] ************************************ skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:35.026 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:35 +0000 (0:00:00.046) 0:00:41.171 ***** 2020-11-28T21:41:35.060 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:35 +0000 (0:00:00.033) 0:00:41.205 ***** 2020-11-28T21:41:35.066 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-11-28T21:41:35.067 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Enable SELinux] *********************************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:35.103 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:35 +0000 (0:00:00.043) 0:00:41.248 ***** 2020-11-28T21:41:35.138 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:35 +0000 (0:00:00.034) 0:00:41.282 ***** 2020-11-28T21:41:35.144 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-11-28T21:41:35.146 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install abrt] ************************************************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:35.183 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:35 +0000 (0:00:00.044) 0:00:41.327 ***** 2020-11-28T21:41:35.217 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:35 +0000 (0:00:00.034) 0:00:41.362 ***** 2020-11-28T21:41:35.225 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Enable abrt-auto-reporting] *********************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:35.261 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:35 +0000 (0:00:00.043) 0:00:41.406 ***** 2020-11-28T21:41:35.295 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:35 +0000 (0:00:00.033) 0:00:41.440 ***** 2020-11-28T21:41:35.300 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-11-28T21:41:35.302 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add ssh pubkeys] ********************************************** 2020-11-28T21:41:35.302 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"attempts": 1, "changed": false, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2020-11-28T21:41:35.340 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:35 +0000 (0:00:00.045) 0:00:41.485 ***** 2020-11-28T21:41:35.374 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:35 +0000 (0:00:00.033) 0:00:41.518 ***** 2020-11-28T21:41:35.411 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:35 +0000 (0:00:00.037) 0:00:41.556 ***** 2020-11-28T21:41:35.417 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2020-11-28T21:41:35.419 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Configure /etc/security/limits.conf] ************************** skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:35.455 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:35 +0000 (0:00:00.044) 0:00:41.600 ***** 2020-11-28T21:41:35.492 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:35 +0000 (0:00:00.036) 0:00:41.637 ***** 2020-11-28T21:41:35.527 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:35 +0000 (0:00:00.034) 0:00:41.671 ***** 2020-11-28T21:41:35.534 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2020-11-28T21:41:35.535 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Restart abrtd] ************************************************ skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Stop and disable iptables] ************************************ skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:35.572 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:35 +0000 (0:00:00.045) 0:00:41.716 ***** 2020-11-28T21:41:35.606 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:35 +0000 (0:00:00.034) 0:00:41.751 ***** 2020-11-28T21:41:35.641 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:35 +0000 (0:00:00.035) 0:00:41.786 ***** 2020-11-28T21:41:35.648 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-11-28T21:41:35.650 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Enable SELinux] *********************************************** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:35.694 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:35 +0000 (0:00:00.053) 0:00:41.839 ***** 2020-11-28T21:41:35.744 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:35 +0000 (0:00:00.049) 0:00:41.889 ***** 2020-11-28T21:41:35.756 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for custom repos] *************************************** 2020-11-28T21:41:35.758 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.005154", "end": "2020-11-28 21:41:36.085779", "rc": 0, "start": "2020-11-28 21:41:36.080625", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} TASK [testnode : Install abrt] ************************************************* skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:35.793 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:35 +0000 (0:00:00.049) 0:00:41.938 ***** 2020-11-28T21:41:35.828 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:35 +0000 (0:00:00.034) 0:00:41.972 ***** 2020-11-28T21:41:35.868 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:35 +0000 (0:00:00.040) 0:00:42.013 ***** 2020-11-28T21:41:35.874 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2020-11-28T21:41:35.875 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Remove custom repos] ****************************************** 2020-11-28T21:41:35.875 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:35.917 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:35 +0000 (0:00:00.048) 0:00:42.061 ***** 2020-11-28T21:41:35.952 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:35 +0000 (0:00:00.034) 0:00:42.096 ***** 2020-11-28T21:41:35.994 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:35 +0000 (0:00:00.041) 0:00:42.138 ***** 2020-11-28T21:41:36.002 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-11-28T21:41:36.004 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:36.047 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:36 +0000 (0:00:00.053) 0:00:42.192 ***** 2020-11-28T21:41:36.099 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:36 +0000 (0:00:00.051) 0:00:42.243 ***** 2020-11-28T21:41:36.108 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2020-11-28T21:41:36.110 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2020-11-28T21:41:36.110 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:36.156 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:36 +0000 (0:00:00.055) 0:00:42.299 ***** 2020-11-28T21:41:36.194 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:36 +0000 (0:00:00.039) 0:00:42.338 ***** 2020-11-28T21:41:36.202 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-11-28T21:41:36.204 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Restart abrtd] ************************************************ skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:36.250 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:36 +0000 (0:00:00.056) 0:00:42.395 ***** 2020-11-28T21:41:36.292 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:36 +0000 (0:00:00.038) 0:00:42.433 ***** 2020-11-28T21:41:36.297 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-11-28T21:41:36.298 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2020-11-28T21:41:36.298 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2020-11-28T21:41:36.337 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:36 +0000 (0:00:00.048) 0:00:42.482 ***** 2020-11-28T21:41:36.373 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:36 +0000 (0:00:00.036) 0:00:42.518 ***** 2020-11-28T21:41:36.379 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-11-28T21:41:36.381 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add sources list] ********************************************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:36.418 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:36 +0000 (0:00:00.044) 0:00:42.562 ***** 2020-11-28T21:41:36.451 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:36 +0000 (0:00:00.033) 0:00:42.596 ***** 2020-11-28T21:41:36.458 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2020-11-28T21:41:36.460 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.005115", "end": "2020-11-28 21:41:36.454673", "rc": 0, "start": "2020-11-28 21:41:36.449558", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} TASK [testnode : Enable copr repos] ******************************************** 2020-11-28T21:41:36.500 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:36 +0000 (0:00:00.048) 0:00:42.645 ***** 2020-11-28T21:41:36.542 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:36 +0000 (0:00:00.040) 0:00:42.686 ***** 2020-11-28T21:41:36.558 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-11-28T21:41:36.560 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Remove custom repos] ****************************************** 2020-11-28T21:41:36.647 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:36 +0000 (0:00:00.105) 0:00:42.792 ***** 2020-11-28T21:41:36.697 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:36 +0000 (0:00:00.050) 0:00:42.842 ***** 2020-11-28T21:41:36.704 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:36.746 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:36 +0000 (0:00:00.048) 0:00:42.891 ***** 2020-11-28T21:41:36.753 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-11-28T21:41:36.755 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:36.811 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:36 +0000 (0:00:00.064) 0:00:42.956 ***** 2020-11-28T21:41:36.817 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2020-11-28T21:41:36.819 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:36.829 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:36.867 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:36 +0000 (0:00:00.055) 0:00:43.012 ***** 2020-11-28T21:41:36.872 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2020-11-28T21:41:36.874 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:36.916 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:36 +0000 (0:00:00.049) 0:00:43.061 ***** 2020-11-28T21:41:36.924 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2020-11-28T21:41:36.925 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:36.968 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:36 +0000 (0:00:00.052) 0:00:43.113 ***** 2020-11-28T21:41:36.976 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2020-11-28T21:41:37.029 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:37 +0000 (0:00:00.060) 0:00:43.173 ***** 2020-11-28T21:41:37.036 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-28T21:41:37.038 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:37.039 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** ok: [smithi118.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2020-11-28T21:41:37.077 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:37 +0000 (0:00:00.048) 0:00:43.222 ***** 2020-11-28T21:41:37.111 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:37 +0000 (0:00:00.033) 0:00:43.255 ***** 2020-11-28T21:41:37.118 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-11-28T21:41:37.119 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:37.156 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:37 +0000 (0:00:00.044) 0:00:43.300 ***** 2020-11-28T21:41:37.192 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:37 +0000 (0:00:00.036) 0:00:43.337 ***** 2020-11-28T21:41:37.202 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:37.246 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:37 +0000 (0:00:00.053) 0:00:43.390 ***** 2020-11-28T21:41:37.254 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2020-11-28T21:41:37.300 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:37 +0000 (0:00:00.054) 0:00:43.445 ***** 2020-11-28T21:41:37.308 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2020-11-28T21:41:37.310 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:37.347 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:37 +0000 (0:00:00.045) 0:00:43.491 ***** 2020-11-28T21:41:37.352 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2020-11-28T21:41:37.358 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:37.394 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:37 +0000 (0:00:00.043) 0:00:43.535 ***** 2020-11-28T21:41:37.397 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2020-11-28T21:41:37.399 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:37.435 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:37 +0000 (0:00:00.045) 0:00:43.580 ***** 2020-11-28T21:41:37.442 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2020-11-28T21:41:37.443 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-11-28T21:41:37.479 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:37 +0000 (0:00:00.043) 0:00:43.623 ***** 2020-11-28T21:41:37.484 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2020-11-28T21:41:37.486 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:37.531 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:37 +0000 (0:00:00.051) 0:00:43.675 ***** 2020-11-28T21:41:37.537 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2020-11-28T21:41:37.539 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:37.576 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:37 +0000 (0:00:00.045) 0:00:43.721 ***** 2020-11-28T21:41:37.583 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:37.619 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:37 +0000 (0:00:00.042) 0:00:43.764 ***** 2020-11-28T21:41:37.625 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-28T21:41:37.627 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:37.664 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:37 +0000 (0:00:00.044) 0:00:43.809 ***** 2020-11-28T21:41:37.672 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-11-28T21:41:37.673 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:37.713 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:37 +0000 (0:00:00.049) 0:00:43.858 ***** 2020-11-28T21:41:37.720 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2020-11-28T21:41:37.722 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:37.758 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:37 +0000 (0:00:00.044) 0:00:43.903 ***** 2020-11-28T21:41:37.768 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-28T21:41:37.769 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:37.807 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:37 +0000 (0:00:00.048) 0:00:43.952 ***** 2020-11-28T21:41:37.815 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2020-11-28T21:41:37.816 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:37.861 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:37 +0000 (0:00:00.053) 0:00:44.005 ***** 2020-11-28T21:41:37.868 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:37.918 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:37 +0000 (0:00:00.054) 0:00:44.060 ***** 2020-11-28T21:41:37.932 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2020-11-28T21:41:37.934 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:37.938 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* ok: [smithi096.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2020-11-28T21:41:37.976 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:37 +0000 (0:00:00.059) 0:00:44.120 ***** 2020-11-28T21:41:37.981 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-11-28T21:41:37.986 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:38.023 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:38 +0000 (0:00:00.047) 0:00:44.168 ***** 2020-11-28T21:41:38.030 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:38.073 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:38 +0000 (0:00:00.050) 0:00:44.218 ***** 2020-11-28T21:41:38.080 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-28T21:41:38.082 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:38.135 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:38 +0000 (0:00:00.062) 0:00:44.280 ***** 2020-11-28T21:41:38.143 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:38.185 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:38 +0000 (0:00:00.049) 0:00:44.330 ***** 2020-11-28T21:41:38.199 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install apt keys] ********************************************* ok: [smithi118.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2020-11-28T21:41:38.236 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:38 +0000 (0:00:00.051) 0:00:44.381 ***** 2020-11-28T21:41:38.253 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-11-28T21:41:38.255 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:38.289 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:38 +0000 (0:00:00.052) 0:00:44.434 ***** 2020-11-28T21:41:38.295 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2020-11-28T21:41:38.296 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:38.335 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:38 +0000 (0:00:00.046) 0:00:44.480 ***** 2020-11-28T21:41:38.341 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-28T21:41:38.343 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:38.385 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:38 +0000 (0:00:00.050) 0:00:44.530 ***** 2020-11-28T21:41:38.391 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2020-11-28T21:41:38.392 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:38.433 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:38 +0000 (0:00:00.046) 0:00:44.576 ***** 2020-11-28T21:41:38.440 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:38.482 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:38 +0000 (0:00:00.045) 0:00:44.622 ***** 2020-11-28T21:41:38.484 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* 2020-11-28T21:41:38.486 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:38.527 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:38 +0000 (0:00:00.049) 0:00:44.672 ***** 2020-11-28T21:41:38.534 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-11-28T21:41:38.539 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:38.575 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:38 +0000 (0:00:00.048) 0:00:44.720 ***** 2020-11-28T21:41:38.581 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2020-11-28T21:41:38.583 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:38.622 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:38 +0000 (0:00:00.047) 0:00:44.767 ***** 2020-11-28T21:41:38.629 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-28T21:41:38.631 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:38.672 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:38 +0000 (0:00:00.049) 0:00:44.817 ***** 2020-11-28T21:41:38.679 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:38.720 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:38 +0000 (0:00:00.048) 0:00:44.865 ***** 2020-11-28T21:41:38.727 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2020-11-28T21:41:38.728 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:38.764 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:38 +0000 (0:00:00.043) 0:00:44.909 ***** 2020-11-28T21:41:38.770 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2020-11-28T21:41:38.773 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:38.812 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:38 +0000 (0:00:00.047) 0:00:44.956 ***** 2020-11-28T21:41:38.820 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2020-11-28T21:41:38.821 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install apt keys] ********************************************* ok: [smithi096.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2020-11-28T21:41:38.866 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:38 +0000 (0:00:00.048) 0:00:45.005 ***** 2020-11-28T21:41:38.900 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:38 +0000 (0:00:00.039) 0:00:45.044 ***** 2020-11-28T21:41:38.906 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2020-11-28T21:41:38.908 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:38.953 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:38 +0000 (0:00:00.053) 0:00:45.098 ***** 2020-11-28T21:41:38.964 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:39.005 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:38 +0000 (0:00:00.051) 0:00:45.149 ***** 2020-11-28T21:41:39.034 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* 2020-11-28T21:41:39.036 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:39.071 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:39 +0000 (0:00:00.066) 0:00:45.216 ***** 2020-11-28T21:41:39.077 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2020-11-28T21:41:39.079 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2020-11-28T21:41:39.117 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:39 +0000 (0:00:00.046) 0:00:45.262 ***** 2020-11-28T21:41:39.125 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2020-11-28T21:41:39.127 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:39.128 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** ok: [smithi162.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.005140", "end": "2020-11-28 21:41:39.494932", "rc": 0, "start": "2020-11-28 21:41:39.489792", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-28T21:41:39.164 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:39 +0000 (0:00:00.046) 0:00:45.308 ***** 2020-11-28T21:41:39.198 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:39 +0000 (0:00:00.034) 0:00:45.343 ***** 2020-11-28T21:41:39.206 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2020-11-28T21:41:39.252 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:39 +0000 (0:00:00.051) 0:00:45.395 ***** 2020-11-28T21:41:39.256 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2020-11-28T21:41:39.258 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:39.294 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:39 +0000 (0:00:00.044) 0:00:45.439 ***** 2020-11-28T21:41:39.322 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:39.361 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:39 +0000 (0:00:00.065) 0:00:45.505 ***** 2020-11-28T21:41:39.388 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:39.426 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:39 +0000 (0:00:00.065) 0:00:45.571 ***** 2020-11-28T21:41:39.428 INFO:teuthology.task.ansible.out: 2020-11-28T21:41:39.573 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2020-11-28T21:41:39.575 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} TASK [testnode : Check for custom repos] *************************************** ok: [smithi192.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.005226", "end": "2020-11-28 21:41:39.876582", "rc": 0, "start": "2020-11-28 21:41:39.871356", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-28T21:41:39.612 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:39 +0000 (0:00:00.186) 0:00:45.757 ***** 2020-11-28T21:41:39.650 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:39 +0000 (0:00:00.034) 0:00:45.791 ***** 2020-11-28T21:41:39.652 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2020-11-28T21:41:39.654 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:39.690 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:39 +0000 (0:00:00.043) 0:00:45.835 ***** 2020-11-28T21:41:39.702 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2020-11-28T21:41:39.739 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:39 +0000 (0:00:00.048) 0:00:45.884 ***** 2020-11-28T21:41:39.794 INFO:teuthology.task.ansible.out: 2020-11-28T21:41:39.796 INFO:teuthology.task.ansible.out:TASK [testnode : Install python-apt] ******************************************* ok: [smithi096.front.sepia.ceph.com] => {"cache_update_time": 1606599646, "cache_updated": false, "changed": false} 2020-11-28T21:41:39.836 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:39 +0000 (0:00:00.097) 0:00:45.981 ***** 2020-11-28T21:41:39.864 INFO:teuthology.task.ansible.out: 2020-11-28T21:41:39.866 INFO:teuthology.task.ansible.out:TASK [testnode : Add local apt repos.] ***************************************** 2020-11-28T21:41:39.902 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:39 +0000 (0:00:00.066) 0:00:46.047 ***** 2020-11-28T21:41:40.015 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2020-11-28T21:41:40.017 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"cache_update_time": 1606599636, "cache_updated": false, "changed": false} 2020-11-28T21:41:40.056 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:40 +0000 (0:00:00.154) 0:00:46.201 ***** 2020-11-28T21:41:40.067 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2020-11-28T21:41:40.069 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2020-11-28T21:41:40.106 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:40 +0000 (0:00:00.049) 0:00:46.251 ***** 2020-11-28T21:41:40.112 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2020-11-28T21:41:40.114 INFO:teuthology.task.ansible.out:[WARNING]: Consider using 'become', 'become_method', and 'become_user' rather than running sudo changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.033264", "end": "2020-11-28 21:41:40.444406", "rc": 0, "start": "2020-11-28 21:41:40.411142", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} TASK [testnode : Add local apt repos.] ***************************************** 2020-11-28T21:41:40.154 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:40 +0000 (0:00:00.048) 0:00:46.299 ***** 2020-11-28T21:41:40.189 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:40 +0000 (0:00:00.034) 0:00:46.334 ***** 2020-11-28T21:41:40.196 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2020-11-28T21:41:40.198 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:40.238 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:40 +0000 (0:00:00.047) 0:00:46.381 ***** 2020-11-28T21:41:40.349 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2020-11-28T21:41:40.351 INFO:teuthology.task.ansible.out:changed: [smithi118.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.032484", "end": "2020-11-28 21:41:40.405425", "rc": 0, "start": "2020-11-28 21:41:40.372941", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-28T21:41:40.387 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:40 +0000 (0:00:00.150) 0:00:46.532 ***** 2020-11-28T21:41:40.706 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2020-11-28T21:41:40.707 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2020-11-28T21:41:41.226 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2020-11-28T21:41:41.612 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2020-11-28T21:41:41.651 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:41 +0000 (0:00:01.263) 0:00:47.796 ***** 2020-11-28T21:41:42.171 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2020-11-28T21:41:42.212 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:42 +0000 (0:00:00.561) 0:00:48.357 ***** 2020-11-28T21:41:42.562 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* ok: [smithi162.front.sepia.ceph.com] => {"cache_update_time": 1606599631, "cache_updated": false, "changed": false} 2020-11-28T21:41:42.599 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:42 +0000 (0:00:00.386) 0:00:48.744 ***** 2020-11-28T21:41:42.629 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2020-11-28T21:41:42.666 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:42 +0000 (0:00:00.065) 0:00:48.809 ***** 2020-11-28T21:41:42.824 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** changed: [smithi162.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.022824", "end": "2020-11-28 21:41:43.211867", "rc": 0, "start": "2020-11-28 21:41:43.189043", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-28T21:41:42.861 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:42 +0000 (0:00:00.196) 0:00:49.006 ***** 2020-11-28T21:41:43.020 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2020-11-28T21:41:43.022 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"cache_update_time": 1606599646, "cache_updated": false, "changed": false} 2020-11-28T21:41:43.060 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:43 +0000 (0:00:00.198) 0:00:49.204 ***** 2020-11-28T21:41:43.131 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2020-11-28T21:41:43.133 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"cache_update_time": 1606599641, "cache_updated": false, "changed": false} 2020-11-28T21:41:43.176 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:43 +0000 (0:00:00.116) 0:00:49.321 ***** 2020-11-28T21:41:43.203 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2020-11-28T21:41:43.243 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:43 +0000 (0:00:00.066) 0:00:49.387 ***** 2020-11-28T21:41:43.254 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** ok: [smithi118.front.sepia.ceph.com] => {"cache_update_time": 1606599636, "cache_updated": false, "changed": false} 2020-11-28T21:41:43.289 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:43 +0000 (0:00:00.046) 0:00:49.434 ***** 2020-11-28T21:41:43.406 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** changed: [smithi192.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.032820", "end": "2020-11-28 21:41:43.701084", "rc": 0, "start": "2020-11-28 21:41:43.668264", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-28T21:41:43.448 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:43 +0000 (0:00:00.158) 0:00:49.593 ***** 2020-11-28T21:41:45.623 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2020-11-28T21:41:45.626 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => {"cache_update_time": 1606599631, "cache_updated": false, "changed": false} 2020-11-28T21:41:45.673 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:45 +0000 (0:00:02.220) 0:00:51.814 ***** 2020-11-28T21:41:46.219 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"cache_update_time": 1606599641, "cache_updated": false, "changed": false} 2020-11-28T21:41:46.259 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:46 +0000 (0:00:00.586) 0:00:52.401 ***** 2020-11-28T21:41:51.548 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2020-11-28T21:41:51.550 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 5 not upgraded.\nAfter this operation, 47.6 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 102076 files and directories 2020-11-28T21:41:51.551 INFO:teuthology.task.ansible.out: currently installed.)\r\nRemoving qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving librbd1 (12.2.13-0ubuntu0.18.04.5) ...\r\nRemoving librados2 (12.2.13-0ubuntu0.18.04.5) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 5 not upgraded.", "After this operation, 47.6 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Readin 2020-11-28T21:41:51.553 INFO:teuthology.task.ansible.out:g database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 102076 files and directories currently installed.)", "Removing qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing librbd1 (12.2.13-0ubuntu0.18.04.5) ...", "Removing librados2 (12.2.13-0ubuntu0.18.04.5) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2020-11-28T21:41:51.602 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:51 +0000 (0:00:05.346) 0:00:57.747 ***** 2020-11-28T21:41:51.768 INFO:teuthology.task.ansible.out:changed: [smithi118.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 5 not upgraded.\nAfter this operation, 47.6 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 102076 files and directories 2020-11-28T21:41:51.770 INFO:teuthology.task.ansible.out: currently installed.)\r\nRemoving qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving librbd1 (12.2.13-0ubuntu0.18.04.5) ...\r\nRemoving librados2 (12.2.13-0ubuntu0.18.04.5) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 5 not upgraded.", "After this operation, 47.6 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Readin 2020-11-28T21:41:51.771 INFO:teuthology.task.ansible.out:g database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 102076 files and directories currently installed.)", "Removing qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing librbd1 (12.2.13-0ubuntu0.18.04.5) ...", "Removing librados2 (12.2.13-0ubuntu0.18.04.5) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2020-11-28T21:41:51.806 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:51 +0000 (0:00:00.203) 0:00:57.951 ***** 2020-11-28T21:41:52.517 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-28T21:41:52.518 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"cache_update_time": 1606599646, "cache_updated": false, "changed": false} 2020-11-28T21:41:52.555 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:52 +0000 (0:00:00.749) 0:00:58.700 ***** 2020-11-28T21:41:52.589 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:52.627 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:52 +0000 (0:00:00.071) 0:00:58.772 ***** 2020-11-28T21:41:52.709 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-28T21:41:52.711 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"cache_update_time": 1606599636, "cache_updated": false, "changed": false} 2020-11-28T21:41:52.746 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:52 +0000 (0:00:00.118) 0:00:58.891 ***** 2020-11-28T21:41:52.773 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2020-11-28T21:41:52.775 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:52.810 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:52 +0000 (0:00:00.063) 0:00:58.954 ***** 2020-11-28T21:41:54.034 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2020-11-28T21:41:54.035 INFO:teuthology.task.ansible.out:changed: [smithi162.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 5 not upgraded.\nAfter this operation, 47.6 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 102076 files and directories 2020-11-28T21:41:54.037 INFO:teuthology.task.ansible.out: currently installed.)\r\nRemoving qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving librbd1 (12.2.13-0ubuntu0.18.04.5) ...\r\nRemoving librados2 (12.2.13-0ubuntu0.18.04.5) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 5 not upgraded.", "After this operation, 47.6 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Readin 2020-11-28T21:41:54.038 INFO:teuthology.task.ansible.out:g database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 102076 files and directories currently installed.)", "Removing qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing librbd1 (12.2.13-0ubuntu0.18.04.5) ...", "Removing librados2 (12.2.13-0ubuntu0.18.04.5) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2020-11-28T21:41:54.077 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:54 +0000 (0:00:01.267) 0:01:00.222 ***** 2020-11-28T21:41:54.931 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-28T21:41:54.933 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => {"cache_update_time": 1606599631, "cache_updated": false, "changed": false} TASK [testnode : Ensure packages are not present.] ***************************** changed: [smithi192.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 5 not upgraded.\nAfter this operation, 47.6 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading dat 2020-11-28T21:41:54.934 INFO:teuthology.task.ansible.out:abase ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 102076 files and directories currently installed.)\r\nRemoving qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving librbd1 (12.2.13-0ubuntu0.18.04.5) ...\r\nRemoving librados2 (12.2.13-0ubuntu0.18.04.5) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 5 not upgraded.", "After this operation, 47.6 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading 2020-11-28T21:41:54.938 INFO:teuthology.task.ansible.out:database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 102076 files and directories currently installed.)", "Removing qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing librbd1 (12.2.13-0ubuntu0.18.04.5) ...", "Removing librados2 (12.2.13-0ubuntu0.18.04.5) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2020-11-28T21:41:54.973 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:54 +0000 (0:00:00.895) 0:01:01.118 ***** 2020-11-28T21:41:55.008 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:55 +0000 (0:00:00.034) 0:01:01.152 ***** 2020-11-28T21:41:55.013 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2020-11-28T21:41:55.015 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:55.052 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:55 +0000 (0:00:00.043) 0:01:01.196 ***** 2020-11-28T21:41:55.853 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-28T21:41:55.855 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"cache_update_time": 1606599641, "cache_updated": false, "changed": false} 2020-11-28T21:41:55.890 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:55 +0000 (0:00:00.838) 0:01:02.035 ***** 2020-11-28T21:41:55.918 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2020-11-28T21:41:55.920 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:41:55.955 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:41:55 +0000 (0:00:00.064) 0:01:02.100 ***** 2020-11-28T21:42:01.256 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-28T21:42:01.257 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"cache_update_time": 1606599646, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\ngit is already the newest version (1:2.17.1-1ubuntu0.7).\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n samba vde2 sgabios ovmf debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 6 newly installed, 0 to remove and 5 not upgraded.\nNeed to get 10.4 MB of archives.\nAfter this operation, 47.6 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.5 [2723 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.5 [923 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.33 [40.1 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.33 [672 kB]\nGet:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.33 [5193 kB]\nGet:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.33 [873 kB]\nFetched 10.4 MB in 1s (9390 kB/s)\nSelecting previously unselected package librados2.\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 2020-11-28T21:42:01.259 INFO:teuthology.task.ansible.out: 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 101934 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.5_amd64.deb ...\r\nUnpacking librados2 (12.2.13-0ubuntu0.18.04.5) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.5_amd64.deb ...\r\nUnpacking librbd1 (12.2.13-0ubuntu0.18.04.5) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up librados2 (12.2.13-0ubuntu0.18.04.5) ...\r\nSetting up librbd1 (12.2.13-0ubuntu0.18.04.5) ...\r\nSetting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu 2020-11-28T21:42:01.260 INFO:teuthology.task.ansible.out:1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "git is already the newest version (1:2.17.1-1ubuntu0.7).", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " samba vde2 sgabios ovmf debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 6 newly installed, 0 to remove and 5 not upgraded.", "Need to get 10.4 MB of archives.", "After this operation, 47.6 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.5 [2723 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.5 [923 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.33 [40.1 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.33 [672 kB]", "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.33 [5193 kB]", "Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.33 [873 kB]", "Fetched 10.4 MB in 1s (9390 kB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Rea 2020-11-28T21:42:01.262 INFO:teuthology.task.ansible.out:ding database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 101934 files and directories currently installed.)", "Preparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.5_amd64.deb ...", "Unpacking librados2 (12.2.13-0ubuntu0.18.04.5) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.5_amd64.deb ...", "Unpacking librbd1 (12.2.13-0ubuntu0.18.04.5) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up librados2 (12.2.13-0ubuntu0.18.04.5) ...", "Setting up librbd1 (12.2.13-0ubuntu0.18.04.5) ...", "Setting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2020-11-28T21:42:01.296 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:01 +0000 (0:00:05.340) 0:01:07.441 ***** 2020-11-28T21:42:01.357 INFO:teuthology.task.ansible.out:changed: [smithi118.front.sepia.ceph.com] => {"cache_update_time": 1606599636, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\ngit is already the newest version (1:2.17.1-1ubuntu0.7).\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n samba vde2 sgabios ovmf debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 6 newly installed, 0 to remove and 5 not upgraded.\nNeed to get 10.4 MB of archives.\nAfter this operation, 47.6 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.5 [2723 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.5 [923 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.33 [40.1 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.33 [672 kB]\nGet:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.33 [5193 kB]\nGet:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.33 [873 kB]\nFetched 10.4 MB in 1s (9890 kB/s)\nSelecting previously unselected package librados2.\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 2020-11-28T21:42:01.359 INFO:teuthology.task.ansible.out: 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 101934 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.5_amd64.deb ...\r\nUnpacking librados2 (12.2.13-0ubuntu0.18.04.5) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.5_amd64.deb ...\r\nUnpacking librbd1 (12.2.13-0ubuntu0.18.04.5) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up librados2 (12.2.13-0ubuntu0.18.04.5) ...\r\nSetting up librbd1 (12.2.13-0ubuntu0.18.04.5) ...\r\nSetting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu 2020-11-28T21:42:01.361 INFO:teuthology.task.ansible.out:1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "git is already the newest version (1:2.17.1-1ubuntu0.7).", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " samba vde2 sgabios ovmf debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 6 newly installed, 0 to remove and 5 not upgraded.", "Need to get 10.4 MB of archives.", "After this operation, 47.6 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.5 [2723 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.5 [923 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.33 [40.1 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.33 [672 kB]", "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.33 [5193 kB]", "Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.33 [873 kB]", "Fetched 10.4 MB in 1s (9890 kB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Rea 2020-11-28T21:42:01.363 INFO:teuthology.task.ansible.out:ding database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 101934 files and directories currently installed.)", "Preparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.5_amd64.deb ...", "Unpacking librados2 (12.2.13-0ubuntu0.18.04.5) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.5_amd64.deb ...", "Unpacking librbd1 (12.2.13-0ubuntu0.18.04.5) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up librados2 (12.2.13-0ubuntu0.18.04.5) ...", "Setting up librbd1 (12.2.13-0ubuntu0.18.04.5) ...", "Setting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2020-11-28T21:42:01.401 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:01 +0000 (0:00:00.105) 0:01:07.546 ***** 2020-11-28T21:42:02.244 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2020-11-28T21:42:02.246 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"cache_update_time": 1606599646, "cache_updated": false, "changed": false} 2020-11-28T21:42:02.297 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:02 +0000 (0:00:00.895) 0:01:08.442 ***** 2020-11-28T21:42:02.328 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"cache_update_time": 1606599636, "cache_updated": false, "changed": false} 2020-11-28T21:42:02.396 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:02 +0000 (0:00:00.098) 0:01:08.540 ***** 2020-11-28T21:42:02.449 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2020-11-28T21:42:02.451 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2020-11-28T21:42:02.496 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:02 +0000 (0:00:00.099) 0:01:08.640 ***** 2020-11-28T21:42:02.523 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2020-11-28T21:42:02.525 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:02.563 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:02 +0000 (0:00:00.067) 0:01:08.708 ***** 2020-11-28T21:42:02.571 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2020-11-28T21:42:02.573 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2020-11-28T21:42:02.613 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:02 +0000 (0:00:00.049) 0:01:08.758 ***** 2020-11-28T21:42:02.641 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2020-11-28T21:42:02.643 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:02.677 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:02 +0000 (0:00:00.063) 0:01:08.821 ***** 2020-11-28T21:42:02.693 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2020-11-28T21:42:02.695 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2020-11-28T21:42:02.730 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:02 +0000 (0:00:00.052) 0:01:08.874 ***** 2020-11-28T21:42:02.820 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2020-11-28T21:42:02.857 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:02 +0000 (0:00:00.127) 0:01:09.001 ***** 2020-11-28T21:42:03.065 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2020-11-28T21:42:03.065 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2020-11-28T21:42:03.116 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:03 +0000 (0:00:00.258) 0:01:09.260 ***** 2020-11-28T21:42:03.196 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2020-11-28T21:42:03.237 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:03 +0000 (0:00:00.121) 0:01:09.382 ***** 2020-11-28T21:42:03.271 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2020-11-28T21:42:03.273 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1235, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-11-28T21:42:03.307 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:03 +0000 (0:00:00.069) 0:01:09.451 ***** 2020-11-28T21:42:03.409 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1235, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-11-28T21:42:03.449 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:03 +0000 (0:00:00.142) 0:01:09.593 ***** 2020-11-28T21:42:03.456 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2020-11-28T21:42:03.458 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-28T21:42:03.499 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:03 +0000 (0:00:00.049) 0:01:09.643 ***** 2020-11-28T21:42:03.590 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-28T21:42:03.630 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:03 +0000 (0:00:00.131) 0:01:09.775 ***** 2020-11-28T21:42:03.637 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-28T21:42:03.639 INFO:teuthology.task.ansible.out:changed: [smithi162.front.sepia.ceph.com] => {"cache_update_time": 1606599631, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\ngit is already the newest version (1:2.17.1-1ubuntu0.7).\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n samba vde2 sgabios ovmf debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 6 newly installed, 0 to remove and 5 not upgraded.\nNeed to get 10.4 MB of archives.\nAfter this operation, 47.6 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.5 [2723 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.5 [923 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.33 [40.1 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.33 [672 kB]\nGet:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.33 [5193 kB]\nGet:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.33 [873 kB]\nFetched 10.4 MB in 1s (9933 kB/s)\nSelecting previously unselected package librados2.\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 2020-11-28T21:42:03.640 INFO:teuthology.task.ansible.out: 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 101934 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.5_amd64.deb ...\r\nUnpacking librados2 (12.2.13-0ubuntu0.18.04.5) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.5_amd64.deb ...\r\nUnpacking librbd1 (12.2.13-0ubuntu0.18.04.5) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up librados2 (12.2.13-0ubuntu0.18.04.5) ...\r\nSetting up librbd1 (12.2.13-0ubuntu0.18.04.5) ...\r\nSetting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu 2020-11-28T21:42:03.642 INFO:teuthology.task.ansible.out:1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "git is already the newest version (1:2.17.1-1ubuntu0.7).", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " samba vde2 sgabios ovmf debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 6 newly installed, 0 to remove and 5 not upgraded.", "Need to get 10.4 MB of archives.", "After this operation, 47.6 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.5 [2723 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.5 [923 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.33 [40.1 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.33 [672 kB]", "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.33 [5193 kB]", "Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.33 [873 kB]", "Fetched 10.4 MB in 1s (9933 kB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Rea 2020-11-28T21:42:03.644 INFO:teuthology.task.ansible.out:ding database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 101934 files and directories currently installed.)", "Preparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.5_amd64.deb ...", "Unpacking librados2 (12.2.13-0ubuntu0.18.04.5) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.5_amd64.deb ...", "Unpacking librbd1 (12.2.13-0ubuntu0.18.04.5) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up librados2 (12.2.13-0ubuntu0.18.04.5) ...", "Setting up librbd1 (12.2.13-0ubuntu0.18.04.5) ...", "Setting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2020-11-28T21:42:03.679 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:03 +0000 (0:00:00.049) 0:01:09.824 ***** 2020-11-28T21:42:03.795 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2020-11-28T21:42:03.797 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Sat 2020-11-28 21:39:21 UTC", "ActiveEnterTimestampMonotonic": "18972225", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "remote-fs.target sysinit.target -.mount basic.target nss-lookup.target systemd-journald.socket system.slice network.target systemd-tmpfiles-setup.service", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2020-11-28 21:39:21 UTC", "AssertTimestampMonotonic": "18343613", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2020-11-28 21:39:21 UTC", "ConditionTimestampMonotonic": "18343613", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", " 2020-11-28T21:42:03.799 INFO:teuthology.task.ansible.out:DropInPaths": "/lib/systemd/system/apache2.service.d/apache2-systemd.conf", "DynamicUser": "no", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1378", "ExecMainStartTimestamp": "Sat 2020-11-28 21:39:21 UTC", "ExecMainStartTimestampMonotonic": "18972214", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2020-11-28 21:39:21 UTC", "InactiveExitTimestampMonotonic": "18344324", "InvocationID": "c2dd1e5f499f41c594051bba27650fca", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESof 2020-11-28T21:42:03.801 INFO:teuthology.task.ansible.out:t": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1378", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount sysinit.target system.slice", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIG 2020-11-28T21:42:03.803 INFO:teuthology.task.ansible.out:HUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2020-11-28 21:39:21 UTC", "StateChangeTimestampMonotonic": "18972225", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2020-11-28 21:39:21 UTC", "WatchdogTimestampMonotonic": "18972224", "WatchdogUSec": "0"}} 2020-11-28T21:42:03.835 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:03 +0000 (0:00:00.155) 0:01:09.980 ***** 2020-11-28T21:42:03.862 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-11-28T21:42:03.864 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:03.899 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:03 +0000 (0:00:00.064) 0:01:10.044 ***** 2020-11-28T21:42:03.918 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2020-11-28T21:42:03.921 INFO:teuthology.task.ansible.out:changed: [smithi118.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Sat 2020-11-28 21:39:56 UTC", "ActiveEnterTimestampMonotonic": "18505993", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "nss-lookup.target network.target systemd-journald.socket sysinit.target remote-fs.target -.mount systemd-tmpfiles-setup.service basic.target system.slice", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2020-11-28 21:39:55 UTC", "AssertTimestampMonotonic": "17823896", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2020-11-28 21:39:55 UTC", "ConditionTimestampMonotonic": "17823896", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", " 2020-11-28T21:42:03.922 INFO:teuthology.task.ansible.out:DropInPaths": "/lib/systemd/system/apache2.service.d/apache2-systemd.conf", "DynamicUser": "no", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1403", "ExecMainStartTimestamp": "Sat 2020-11-28 21:39:56 UTC", "ExecMainStartTimestampMonotonic": "18505958", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2020-11-28 21:39:55 UTC", "InactiveExitTimestampMonotonic": "17825140", "InvocationID": "099b6e78f86c41fa97dd7a1ae600e978", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESof 2020-11-28T21:42:03.925 INFO:teuthology.task.ansible.out:t": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1403", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target -.mount", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIG 2020-11-28T21:42:03.926 INFO:teuthology.task.ansible.out:HUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2020-11-28 21:39:56 UTC", "StateChangeTimestampMonotonic": "18505993", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2020-11-28 21:39:56 UTC", "WatchdogTimestampMonotonic": "18505991", "WatchdogUSec": "0"}} 2020-11-28T21:42:03.958 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:03 +0000 (0:00:00.058) 0:01:10.103 ***** 2020-11-28T21:42:03.965 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-11-28T21:42:03.967 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:04.001 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:03 +0000 (0:00:00.043) 0:01:10.146 ***** 2020-11-28T21:42:04.009 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-11-28T21:42:04.013 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:04.048 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:04 +0000 (0:00:00.046) 0:01:10.192 ***** 2020-11-28T21:42:04.054 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-11-28T21:42:04.056 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:04.091 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:04 +0000 (0:00:00.043) 0:01:10.236 ***** 2020-11-28T21:42:04.098 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-11-28T21:42:04.099 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:04.136 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:04 +0000 (0:00:00.044) 0:01:10.280 ***** 2020-11-28T21:42:04.142 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-11-28T21:42:04.144 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:04.188 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:04 +0000 (0:00:00.052) 0:01:10.333 ***** 2020-11-28T21:42:04.197 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-11-28T21:42:04.200 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:04.239 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:04 +0000 (0:00:00.050) 0:01:10.384 ***** 2020-11-28T21:42:04.248 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:04.291 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:04 +0000 (0:00:00.051) 0:01:10.436 ***** 2020-11-28T21:42:04.300 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:04.338 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:04 +0000 (0:00:00.047) 0:01:10.483 ***** 2020-11-28T21:42:04.345 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-11-28T21:42:04.346 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:04.381 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:04 +0000 (0:00:00.043) 0:01:10.526 ***** 2020-11-28T21:42:04.388 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-11-28T21:42:04.389 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:04.424 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:04 +0000 (0:00:00.042) 0:01:10.569 ***** 2020-11-28T21:42:04.432 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:04.468 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:04 +0000 (0:00:00.043) 0:01:10.613 ***** 2020-11-28T21:42:04.475 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-11-28T21:42:04.478 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:04.512 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:04 +0000 (0:00:00.043) 0:01:10.657 ***** 2020-11-28T21:42:04.519 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:04.520 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-28T21:42:04.521 INFO:teuthology.task.ansible.out:changed: [smithi192.front.sepia.ceph.com] => {"cache_update_time": 1606599641, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\ngit is already the newest version (1:2.17.1-1ubuntu0.7).\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n samba vde2 sgabios ovmf debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 6 newly installed, 0 to remove and 5 not upgraded.\nNeed to get 10.4 MB of archives.\nAfter this operation, 47.6 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.5 [2723 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.5 [923 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.33 [40.1 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.33 [672 kB]\nGet:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.33 [5193 kB]\nGet:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.33 [873 kB]\nFetched 10.4 MB in 1s (9647 kB/s)\nSelecting previously unselected package librados2.\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database 2020-11-28T21:42:04.523 INFO:teuthology.task.ansible.out:... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 101934 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.5_amd64.deb ...\r\nUnpacking librados2 (12.2.13-0ubuntu0.18.04.5) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.5_amd64.deb ...\r\nUnpacking librbd1 (12.2.13-0ubuntu0.18.04.5) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up librados2 (12.2.13-0ubuntu0.18.04.5) ...\r\nSetting up librbd1 (12.2.13-0ubuntu0.18.04.5) ...\r\nSetting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nProcessing triggers 2020-11-28T21:42:04.524 INFO:teuthology.task.ansible.out: for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "git is already the newest version (1:2.17.1-1ubuntu0.7).", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " samba vde2 sgabios ovmf debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 6 newly installed, 0 to remove and 5 not upgraded.", "Need to get 10.4 MB of archives.", "After this operation, 47.6 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.5 [2723 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.5 [923 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.33 [40.1 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.33 [672 kB]", "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.33 [5193 kB]", "Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.33 [873 kB]", "Fetched 10.4 MB in 1s (9647 kB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Readin 2020-11-28T21:42:04.526 INFO:teuthology.task.ansible.out:g database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 101934 files and directories currently installed.)", "Preparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.5_amd64.deb ...", "Unpacking librados2 (12.2.13-0ubuntu0.18.04.5) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.5_amd64.deb ...", "Unpacking librbd1 (12.2.13-0ubuntu0.18.04.5) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up librados2 (12.2.13-0ubuntu0.18.04.5) ...", "Setting up librbd1 (12.2.13-0ubuntu0.18.04.5) ...", "Setting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2020-11-28T21:42:04.562 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:04 +0000 (0:00:00.049) 0:01:10.707 ***** 2020-11-28T21:42:04.600 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:04 +0000 (0:00:00.037) 0:01:10.745 ***** 2020-11-28T21:42:04.607 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-11-28T21:42:04.608 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : set_fact] ***************************************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install non aarch64 packages] ********************************* ok: [smithi162.front.sepia.ceph.com] => {"cache_update_time": 1606599631, "cache_updated": false, "changed": false} 2020-11-28T21:42:04.648 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:04 +0000 (0:00:00.048) 0:01:10.793 ***** 2020-11-28T21:42:04.687 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:04 +0000 (0:00:00.038) 0:01:10.832 ***** 2020-11-28T21:42:04.720 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:04 +0000 (0:00:00.032) 0:01:10.865 ***** 2020-11-28T21:42:04.726 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-11-28T21:42:04.728 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure ceph dependency packages are not present.] ************* skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:04.764 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:04 +0000 (0:00:00.044) 0:01:10.909 ***** 2020-11-28T21:42:04.804 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:04 +0000 (0:00:00.039) 0:01:10.948 ***** 2020-11-28T21:42:04.811 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-28T21:42:04.812 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:04.848 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:04 +0000 (0:00:00.044) 0:01:10.993 ***** 2020-11-28T21:42:04.857 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-28T21:42:04.859 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** ok: [smithi162.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2020-11-28T21:42:04.896 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:04 +0000 (0:00:00.047) 0:01:11.040 ***** 2020-11-28T21:42:04.931 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:04 +0000 (0:00:00.035) 0:01:11.076 ***** 2020-11-28T21:42:04.937 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-28T21:42:04.939 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Remove packages] ********************************************** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:04.976 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:04 +0000 (0:00:00.044) 0:01:11.121 ***** 2020-11-28T21:42:05.009 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:05 +0000 (0:00:00.033) 0:01:11.154 ***** 2020-11-28T21:42:05.015 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2020-11-28T21:42:05.017 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:05.053 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:05 +0000 (0:00:00.043) 0:01:11.197 ***** 2020-11-28T21:42:05.086 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:05 +0000 (0:00:00.033) 0:01:11.230 ***** 2020-11-28T21:42:05.093 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-28T21:42:05.095 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:05.129 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:05 +0000 (0:00:00.043) 0:01:11.274 ***** 2020-11-28T21:42:05.163 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:05 +0000 (0:00:00.033) 0:01:11.308 ***** 2020-11-28T21:42:05.170 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-28T21:42:05.172 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:05.207 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:05 +0000 (0:00:00.044) 0:01:11.352 ***** 2020-11-28T21:42:05.214 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-28T21:42:05.215 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:05.252 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:05 +0000 (0:00:00.043) 0:01:11.395 ***** 2020-11-28T21:42:05.258 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2020-11-28T21:42:05.258 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2020-11-28T21:42:05.259 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:05.295 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:05 +0000 (0:00:00.044) 0:01:11.440 ***** 2020-11-28T21:42:05.328 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:05 +0000 (0:00:00.032) 0:01:11.473 ***** 2020-11-28T21:42:05.334 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-11-28T21:42:05.336 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:05.371 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:05 +0000 (0:00:00.043) 0:01:11.516 ***** 2020-11-28T21:42:05.412 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:05 +0000 (0:00:00.040) 0:01:11.556 ***** 2020-11-28T21:42:05.418 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-11-28T21:42:05.419 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:05.455 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:05 +0000 (0:00:00.043) 0:01:11.600 ***** 2020-11-28T21:42:05.461 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-28T21:42:05.463 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:05.499 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:05 +0000 (0:00:00.044) 0:01:11.644 ***** 2020-11-28T21:42:05.505 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-11-28T21:42:05.506 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install non aarch64 packages] ********************************* ok: [smithi192.front.sepia.ceph.com] => {"cache_update_time": 1606599641, "cache_updated": false, "changed": false} 2020-11-28T21:42:05.548 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:05 +0000 (0:00:00.048) 0:01:11.693 ***** 2020-11-28T21:42:05.583 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:05 +0000 (0:00:00.034) 0:01:11.728 ***** 2020-11-28T21:42:05.589 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-28T21:42:05.591 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:05.627 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:05 +0000 (0:00:00.044) 0:01:11.772 ***** 2020-11-28T21:42:05.662 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:05 +0000 (0:00:00.035) 0:01:11.807 ***** 2020-11-28T21:42:05.668 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-28T21:42:05.670 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload /etc/fuse.conf.] *************************************** ok: [smithi162.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2020-11-28T21:42:05.710 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:05 +0000 (0:00:00.047) 0:01:11.854 ***** 2020-11-28T21:42:05.743 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:05 +0000 (0:00:00.032) 0:01:11.887 ***** 2020-11-28T21:42:05.750 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-28T21:42:05.751 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** ok: [smithi192.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:05.788 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:05 +0000 (0:00:00.045) 0:01:11.933 ***** 2020-11-28T21:42:05.822 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:05 +0000 (0:00:00.034) 0:01:11.967 ***** 2020-11-28T21:42:05.856 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:05 +0000 (0:00:00.033) 0:01:12.000 ***** 2020-11-28T21:42:05.862 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-28T21:42:05.864 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:05.900 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:05 +0000 (0:00:00.044) 0:01:12.045 ***** 2020-11-28T21:42:05.933 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:05 +0000 (0:00:00.032) 0:01:12.078 ***** 2020-11-28T21:42:05.939 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2020-11-28T21:42:05.940 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2020-11-28T21:42:05.941 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1235, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-11-28T21:42:05.941 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:05.980 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:05 +0000 (0:00:00.047) 0:01:12.125 ***** 2020-11-28T21:42:06.014 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:06 +0000 (0:00:00.033) 0:01:12.159 ***** 2020-11-28T21:42:06.047 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:06 +0000 (0:00:00.033) 0:01:12.192 ***** 2020-11-28T21:42:06.054 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-11-28T21:42:06.056 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:06.097 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:06 +0000 (0:00:00.049) 0:01:12.242 ***** 2020-11-28T21:42:06.135 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:06 +0000 (0:00:00.037) 0:01:12.280 ***** 2020-11-28T21:42:06.142 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2020-11-28T21:42:06.144 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:06.180 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:06 +0000 (0:00:00.044) 0:01:12.324 ***** 2020-11-28T21:42:06.185 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2020-11-28T21:42:06.187 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:06.225 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:06 +0000 (0:00:00.045) 0:01:12.369 ***** 2020-11-28T21:42:06.259 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:06 +0000 (0:00:00.034) 0:01:12.404 ***** 2020-11-28T21:42:06.265 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2020-11-28T21:42:06.267 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:06.305 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:06 +0000 (0:00:00.045) 0:01:12.449 ***** 2020-11-28T21:42:06.338 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:06 +0000 (0:00:00.033) 0:01:12.483 ***** 2020-11-28T21:42:06.344 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2020-11-28T21:42:06.345 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:06.383 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:06 +0000 (0:00:00.044) 0:01:12.527 ***** 2020-11-28T21:42:06.420 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:06 +0000 (0:00:00.036) 0:01:12.564 ***** 2020-11-28T21:42:06.447 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-28T21:42:06.449 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:06.484 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:06 +0000 (0:00:00.064) 0:01:12.629 ***** 2020-11-28T21:42:06.512 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-28T21:42:06.514 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:06.553 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:06 +0000 (0:00:00.068) 0:01:12.698 ***** 2020-11-28T21:42:06.561 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2020-11-28T21:42:06.562 INFO:teuthology.task.ansible.out:changed: [smithi162.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Sat 2020-11-28 21:39:44 UTC", "ActiveEnterTimestampMonotonic": "18786638", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "system.slice -.mount nss-lookup.target basic.target sysinit.target remote-fs.target network.target systemd-journald.socket systemd-tmpfiles-setup.service", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2020-11-28 21:39:43 UTC", "AssertTimestampMonotonic": "18194738", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2020-11-28 21:39:43 UTC", "ConditionTimestampMonotonic": "18194738", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", " 2020-11-28T21:42:06.564 INFO:teuthology.task.ansible.out:DropInPaths": "/lib/systemd/system/apache2.service.d/apache2-systemd.conf", "DynamicUser": "no", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1430", "ExecMainStartTimestamp": "Sat 2020-11-28 21:39:44 UTC", "ExecMainStartTimestampMonotonic": "18786620", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2020-11-28 21:39:43 UTC", "InactiveExitTimestampMonotonic": "18195314", "InvocationID": "eef692e24ac243dcac0519d332a78eb4", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESof 2020-11-28T21:42:06.566 INFO:teuthology.task.ansible.out:t": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1430", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount sysinit.target system.slice", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIG 2020-11-28T21:42:06.567 INFO:teuthology.task.ansible.out:HUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2020-11-28 21:39:44 UTC", "StateChangeTimestampMonotonic": "18786638", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2020-11-28 21:39:44 UTC", "WatchdogTimestampMonotonic": "18786637", "WatchdogUSec": "0"}} 2020-11-28T21:42:06.600 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:06 +0000 (0:00:00.047) 0:01:12.745 ***** 2020-11-28T21:42:06.616 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:06.651 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:06 +0000 (0:00:00.050) 0:01:12.796 ***** 2020-11-28T21:42:06.657 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-11-28T21:42:06.659 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload /etc/fuse.conf.] *************************************** ok: [smithi192.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2020-11-28T21:42:06.695 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:06 +0000 (0:00:00.044) 0:01:12.840 ***** 2020-11-28T21:42:06.732 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:06 +0000 (0:00:00.036) 0:01:12.877 ***** 2020-11-28T21:42:06.738 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-28T21:42:06.740 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2020-11-28T21:42:06.742 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:06.796 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:06 +0000 (0:00:00.063) 0:01:12.941 ***** 2020-11-28T21:42:06.834 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:06 +0000 (0:00:00.036) 0:01:12.977 ***** 2020-11-28T21:42:06.872 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:06 +0000 (0:00:00.039) 0:01:13.017 ***** 2020-11-28T21:42:06.881 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-11-28T21:42:06.883 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set grub config.] ********************************************* skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:06.883 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2020-11-28T21:42:06.884 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1235, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-11-28T21:42:06.934 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:06 +0000 (0:00:00.061) 0:01:13.078 ***** 2020-11-28T21:42:06.969 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:06 +0000 (0:00:00.034) 0:01:13.113 ***** 2020-11-28T21:42:07.002 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:06 +0000 (0:00:00.033) 0:01:13.146 ***** 2020-11-28T21:42:07.008 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-11-28T21:42:07.009 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:07.049 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:07 +0000 (0:00:00.046) 0:01:13.193 ***** 2020-11-28T21:42:07.075 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:07.112 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:07 +0000 (0:00:00.063) 0:01:13.257 ***** 2020-11-28T21:42:07.118 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2020-11-28T21:42:07.119 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2020-11-28T21:42:07.158 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:07 +0000 (0:00:00.046) 0:01:13.303 ***** 2020-11-28T21:42:07.164 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2020-11-28T21:42:07.165 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} TASK [testnode : Configure /etc/security/limits.conf] ************************** skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:07.209 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:07 +0000 (0:00:00.050) 0:01:13.353 ***** 2020-11-28T21:42:07.243 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:07 +0000 (0:00:00.034) 0:01:13.387 ***** 2020-11-28T21:42:07.251 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:07.297 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:07 +0000 (0:00:00.054) 0:01:13.442 ***** 2020-11-28T21:42:07.305 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2020-11-28T21:42:07.307 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2020-11-28T21:42:07.344 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:07 +0000 (0:00:00.046) 0:01:13.489 ***** 2020-11-28T21:42:07.351 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-11-28T21:42:07.353 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:07.394 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:07 +0000 (0:00:00.049) 0:01:13.538 ***** 2020-11-28T21:42:07.420 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-11-28T21:42:07.422 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:07.465 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:07 +0000 (0:00:00.071) 0:01:13.609 ***** 2020-11-28T21:42:07.484 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-11-28T21:42:07.487 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2020-11-28T21:42:07.528 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:07 +0000 (0:00:00.062) 0:01:13.672 ***** 2020-11-28T21:42:07.541 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-28T21:42:07.543 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:07.547 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2020-11-28T21:42:07.550 INFO:teuthology.task.ansible.out:changed: [smithi192.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Sat 2020-11-28 21:39:33 UTC", "ActiveEnterTimestampMonotonic": "18966960", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "basic.target system.slice network.target sysinit.target systemd-journald.socket remote-fs.target nss-lookup.target systemd-tmpfiles-setup.service -.mount", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2020-11-28 21:39:33 UTC", "AssertTimestampMonotonic": "18345319", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2020-11-28 21:39:33 UTC", "ConditionTimestampMonotonic": "18345318", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", " 2020-11-28T21:42:07.554 INFO:teuthology.task.ansible.out:DropInPaths": "/lib/systemd/system/apache2.service.d/apache2-systemd.conf", "DynamicUser": "no", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1535", "ExecMainStartTimestamp": "Sat 2020-11-28 21:39:33 UTC", "ExecMainStartTimestampMonotonic": "18966933", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2020-11-28 21:39:33 UTC", "InactiveExitTimestampMonotonic": "18346243", "InvocationID": "5d37825b6ee542479f54d3e232425901", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESof 2020-11-28T21:42:07.557 INFO:teuthology.task.ansible.out:t": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1535", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice -.mount sysinit.target", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIG 2020-11-28T21:42:07.559 INFO:teuthology.task.ansible.out:HUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2020-11-28 21:39:33 UTC", "StateChangeTimestampMonotonic": "18966960", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2020-11-28 21:39:33 UTC", "WatchdogTimestampMonotonic": "18966958", "WatchdogUSec": "0"}} 2020-11-28T21:42:07.590 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:07 +0000 (0:00:00.062) 0:01:13.735 ***** 2020-11-28T21:42:07.635 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:07 +0000 (0:00:00.041) 0:01:13.776 ***** 2020-11-28T21:42:07.639 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-11-28T21:42:07.643 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Remove packages] ********************************************** skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:07.681 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:07 +0000 (0:00:00.049) 0:01:13.826 ***** 2020-11-28T21:42:07.717 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:07 +0000 (0:00:00.035) 0:01:13.861 ***** 2020-11-28T21:42:07.723 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-11-28T21:42:07.725 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2020-11-28T21:42:07.763 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:07 +0000 (0:00:00.046) 0:01:13.908 ***** 2020-11-28T21:42:07.804 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:07 +0000 (0:00:00.040) 0:01:13.948 ***** 2020-11-28T21:42:07.814 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:07.851 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:07 +0000 (0:00:00.046) 0:01:13.995 ***** 2020-11-28T21:42:07.856 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-11-28T21:42:07.858 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:07.894 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:07 +0000 (0:00:00.043) 0:01:14.039 ***** 2020-11-28T21:42:07.900 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-28T21:42:07.902 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:07.942 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:07 +0000 (0:00:00.048) 0:01:14.087 ***** 2020-11-28T21:42:07.948 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-11-28T21:42:07.951 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:07.994 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:07 +0000 (0:00:00.051) 0:01:14.139 ***** 2020-11-28T21:42:08.001 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:08.039 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:08 +0000 (0:00:00.044) 0:01:14.183 ***** 2020-11-28T21:42:08.045 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-11-28T21:42:08.046 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload weekly kernel-clean crontab.] ************************** ok: [smithi118.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2020-11-28T21:42:08.085 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:08 +0000 (0:00:00.046) 0:01:14.230 ***** 2020-11-28T21:42:08.119 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:08 +0000 (0:00:00.033) 0:01:14.264 ***** 2020-11-28T21:42:08.125 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-28T21:42:08.126 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:08.127 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2020-11-28T21:42:08.127 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2020-11-28T21:42:08.164 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:08 +0000 (0:00:00.045) 0:01:14.309 ***** 2020-11-28T21:42:08.198 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:08 +0000 (0:00:00.033) 0:01:14.343 ***** 2020-11-28T21:42:08.204 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-11-28T21:42:08.206 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:08.244 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:08 +0000 (0:00:00.046) 0:01:14.389 ***** 2020-11-28T21:42:08.250 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-28T21:42:08.252 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:08.291 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:08 +0000 (0:00:00.046) 0:01:14.435 ***** 2020-11-28T21:42:08.298 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-11-28T21:42:08.300 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** ok: [smithi096.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-28T21:42:08.337 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:08 +0000 (0:00:00.046) 0:01:14.482 ***** 2020-11-28T21:42:08.372 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:08 +0000 (0:00:00.035) 0:01:14.517 ***** 2020-11-28T21:42:08.378 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-28T21:42:08.380 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Work around broken wget on wheezy.] *************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:08.416 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:08 +0000 (0:00:00.043) 0:01:14.561 ***** 2020-11-28T21:42:08.449 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:08 +0000 (0:00:00.032) 0:01:14.594 ***** 2020-11-28T21:42:08.455 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-11-28T21:42:08.456 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* ok: [smithi118.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2020-11-28T21:42:08.457 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:08.502 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:08 +0000 (0:00:00.052) 0:01:14.646 ***** 2020-11-28T21:42:08.540 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:08 +0000 (0:00:00.038) 0:01:14.685 ***** 2020-11-28T21:42:08.578 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:08 +0000 (0:00:00.037) 0:01:14.722 ***** 2020-11-28T21:42:08.584 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-11-28T21:42:08.586 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:08.623 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:08 +0000 (0:00:00.045) 0:01:14.768 ***** 2020-11-28T21:42:08.658 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:08 +0000 (0:00:00.034) 0:01:14.803 ***** 2020-11-28T21:42:08.664 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-11-28T21:42:08.666 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check to see if we need to edit /etc/profile.] **************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:08.703 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:08 +0000 (0:00:00.044) 0:01:14.848 ***** 2020-11-28T21:42:08.744 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:08 +0000 (0:00:00.041) 0:01:14.889 ***** 2020-11-28T21:42:08.751 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-11-28T21:42:08.753 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Update /etc/profile if needed.] ******************************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:08.792 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:08 +0000 (0:00:00.047) 0:01:14.936 ***** 2020-11-28T21:42:08.826 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:08 +0000 (0:00:00.034) 0:01:14.971 ***** 2020-11-28T21:42:08.833 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-28T21:42:08.835 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : set_fact] ***************************************************** ok: [smithi096.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2020-11-28T21:42:08.872 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:08 +0000 (0:00:00.045) 0:01:15.017 ***** 2020-11-28T21:42:08.906 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:08 +0000 (0:00:00.033) 0:01:15.051 ***** 2020-11-28T21:42:08.912 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-11-28T21:42:08.914 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Enable kernel modules to load at boot time.] ****************** ok: [smithi118.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2020-11-28T21:42:08.968 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:08 +0000 (0:00:00.061) 0:01:15.113 ***** 2020-11-28T21:42:09.005 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:08 +0000 (0:00:00.034) 0:01:15.147 ***** 2020-11-28T21:42:09.009 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2020-11-28T21:42:09.011 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} TASK [testnode : Remove packages] ********************************************** 2020-11-28T21:42:09.011 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:09.011 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi096.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:09.012 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:09.013 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} ok: [smithi096.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2020-11-28T21:42:09.014 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:09.014 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:09.016 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:09.016 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:09.055 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:09 +0000 (0:00:00.051) 0:01:15.199 ***** 2020-11-28T21:42:09.061 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:09.063 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-28T21:42:09.064 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:09.104 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:09 +0000 (0:00:00.049) 0:01:15.249 ***** 2020-11-28T21:42:09.159 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:09 +0000 (0:00:00.054) 0:01:15.304 ***** 2020-11-28T21:42:09.169 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-28T21:42:09.172 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** ok: [smithi118.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-28T21:42:09.211 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:09 +0000 (0:00:00.051) 0:01:15.355 ***** 2020-11-28T21:42:09.246 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:09 +0000 (0:00:00.034) 0:01:15.390 ***** 2020-11-28T21:42:09.252 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-28T21:42:09.254 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Work around broken wget on wheezy.] *************************** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:09.291 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:09 +0000 (0:00:00.045) 0:01:15.436 ***** 2020-11-28T21:42:09.326 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:09 +0000 (0:00:00.034) 0:01:15.471 ***** 2020-11-28T21:42:09.332 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-28T21:42:09.334 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for teuthology-lock command] **************************** changed: [smithi096.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003500", "end": "2020-11-28 21:42:09.276024", "rc": 0, "start": "2020-11-28 21:42:09.272524", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2020-11-28T21:42:09.334 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:09.373 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:09 +0000 (0:00:00.047) 0:01:15.518 ***** 2020-11-28T21:42:09.408 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:09 +0000 (0:00:00.035) 0:01:15.553 ***** 2020-11-28T21:42:09.442 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:09 +0000 (0:00:00.033) 0:01:15.587 ***** 2020-11-28T21:42:09.448 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-28T21:42:09.450 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Mark system down if NVMe card missing] ************************ skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:09.490 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:09 +0000 (0:00:00.047) 0:01:15.635 ***** 2020-11-28T21:42:09.525 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:09 +0000 (0:00:00.034) 0:01:15.669 ***** 2020-11-28T21:42:09.558 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:09 +0000 (0:00:00.033) 0:01:15.702 ***** 2020-11-28T21:42:09.565 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-11-28T21:42:09.567 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check to see if we need to edit /etc/profile.] **************** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:09.602 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:09 +0000 (0:00:00.044) 0:01:15.747 ***** 2020-11-28T21:42:09.639 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:09 +0000 (0:00:00.036) 0:01:15.784 ***** 2020-11-28T21:42:09.676 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:09 +0000 (0:00:00.036) 0:01:15.820 ***** 2020-11-28T21:42:09.682 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-28T21:42:09.684 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Update /etc/profile if needed.] ******************************* skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:09.721 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:09 +0000 (0:00:00.044) 0:01:15.865 ***** 2020-11-28T21:42:09.761 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:09 +0000 (0:00:00.040) 0:01:15.905 ***** 2020-11-28T21:42:09.798 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:09 +0000 (0:00:00.037) 0:01:15.943 ***** 2020-11-28T21:42:09.807 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:09.809 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** ok: [smithi118.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2020-11-28T21:42:09.850 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:09 +0000 (0:00:00.052) 0:01:15.995 ***** 2020-11-28T21:42:09.889 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:09 +0000 (0:00:00.038) 0:01:16.033 ***** 2020-11-28T21:42:09.897 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-28T21:42:09.898 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi118.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:09.899 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:09.947 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:09 +0000 (0:00:00.058) 0:01:16.092 ***** 2020-11-28T21:42:09.958 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi118.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} ok: [smithi118.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} skipping: [smithi118.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:09.960 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:09.995 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:09 +0000 (0:00:00.048) 0:01:16.140 ***** 2020-11-28T21:42:10.002 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:10.003 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi118.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:10.004 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:10.004 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:10.047 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:10 +0000 (0:00:00.051) 0:01:16.191 ***** 2020-11-28T21:42:10.081 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:10 +0000 (0:00:00.034) 0:01:16.226 ***** 2020-11-28T21:42:10.087 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-28T21:42:10.089 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:10.130 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:10 +0000 (0:00:00.048) 0:01:16.275 ***** 2020-11-28T21:42:10.136 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2020-11-28T21:42:10.138 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:10.180 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:10 +0000 (0:00:00.048) 0:01:16.323 ***** 2020-11-28T21:42:10.194 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-11-28T21:42:10.195 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:10.236 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:10 +0000 (0:00:00.056) 0:01:16.380 ***** 2020-11-28T21:42:10.275 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2020-11-28T21:42:10.276 INFO:teuthology.task.ansible.out:changed: [smithi118.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.018999", "end": "2020-11-28 21:42:10.234321", "rc": 0, "start": "2020-11-28 21:42:10.215322", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:10.319 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:10 +0000 (0:00:00.083) 0:01:16.464 ***** 2020-11-28T21:42:10.353 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:10 +0000 (0:00:00.033) 0:01:16.497 ***** 2020-11-28T21:42:10.358 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2020-11-28T21:42:10.359 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} TASK [testnode : Mark system down if NVMe card missing] ************************ skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:10.399 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:10 +0000 (0:00:00.046) 0:01:16.544 ***** 2020-11-28T21:42:10.436 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:10 +0000 (0:00:00.037) 0:01:16.581 ***** 2020-11-28T21:42:10.442 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-11-28T21:42:10.444 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:10.481 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:10 +0000 (0:00:00.044) 0:01:16.626 ***** 2020-11-28T21:42:10.517 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:10 +0000 (0:00:00.035) 0:01:16.661 ***** 2020-11-28T21:42:10.523 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2020-11-28T21:42:10.524 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:10.564 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:10 +0000 (0:00:00.047) 0:01:16.709 ***** 2020-11-28T21:42:10.571 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-28T21:42:10.573 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:10.611 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:10 +0000 (0:00:00.046) 0:01:16.756 ***** 2020-11-28T21:42:10.617 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2020-11-28T21:42:10.619 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"cache_update_time": 1606599646, "cache_updated": false, "changed": false} 2020-11-28T21:42:10.658 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:10 +0000 (0:00:00.046) 0:01:16.803 ***** 2020-11-28T21:42:10.665 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-28T21:42:10.667 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:10.702 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:10 +0000 (0:00:00.043) 0:01:16.846 ***** 2020-11-28T21:42:10.709 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2020-11-28T21:42:10.710 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:10.748 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:10 +0000 (0:00:00.045) 0:01:16.892 ***** 2020-11-28T21:42:10.755 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-28T21:42:10.758 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** ok: [smithi162.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2020-11-28T21:42:10.798 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:10 +0000 (0:00:00.050) 0:01:16.943 ***** 2020-11-28T21:42:10.836 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:10 +0000 (0:00:00.037) 0:01:16.980 ***** 2020-11-28T21:42:10.843 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-11-28T21:42:10.845 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'block_used': 13643536, 'uuid': '099240f7-127e-4e3e-8e24-c4358fbf763e', 'size_total': 983407812608, 'block_total': 240089798, 'mount': '/', 'block_available': 226446262, 'size_available': 927523889152, 'fstype': 'ext4', 'inode_total': 61046784, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 113957, 'block_size': 4096, 'inode_available': 60932827}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226446262, "block_size": 4096, "block_total": 240089798, "block_used": 13643536, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60932827, "inode_total": 61046784, "inode_used": 113957, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927523889152, "size_total": 983407812608, "uuid": "099240f7-127e-4e3e-8e24-c4358fbf763e"}} 2020-11-28T21:42:10.845 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:10.884 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:10 +0000 (0:00:00.048) 0:01:17.029 ***** 2020-11-28T21:42:10.918 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:10 +0000 (0:00:00.034) 0:01:17.063 ***** 2020-11-28T21:42:10.924 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-28T21:42:10.926 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:10.964 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:10 +0000 (0:00:00.045) 0:01:17.108 ***** 2020-11-28T21:42:10.990 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-28T21:42:10.992 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:11.028 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:11 +0000 (0:00:00.064) 0:01:17.172 ***** 2020-11-28T21:42:11.044 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2020-11-28T21:42:11.046 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.010933", "end": "2020-11-28 21:42:11.394810", "rc": 0, "start": "2020-11-28 21:42:11.383877", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:11.086 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:11 +0000 (0:00:00.057) 0:01:17.230 ***** 2020-11-28T21:42:11.093 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-28T21:42:11.094 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:11.131 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:11 +0000 (0:00:00.045) 0:01:17.275 ***** 2020-11-28T21:42:11.136 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2020-11-28T21:42:11.138 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'block_used': 13643536, 'uuid': '099240f7-127e-4e3e-8e24-c4358fbf763e', 'size_total': 983407812608, 'block_total': 240089798, 'mount': '/', 'block_available': 226446262, 'size_available': 927523889152, 'fstype': 'ext4', 'inode_total': 61046784, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 113957, 'block_size': 4096, 'inode_available': 60932827}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226446262, "block_size": 4096, "block_total": 240089798, "block_used": 13643536, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60932827, "inode_total": 61046784, "inode_used": 113957, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927523889152, "size_total": 983407812608, "uuid": "099240f7-127e-4e3e-8e24-c4358fbf763e"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:11.176 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:11 +0000 (0:00:00.044) 0:01:17.320 ***** 2020-11-28T21:42:11.183 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2020-11-28T21:42:11.184 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:11.220 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:11 +0000 (0:00:00.045) 0:01:17.365 ***** 2020-11-28T21:42:11.226 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-11-28T21:42:11.275 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:11 +0000 (0:00:00.055) 0:01:17.420 ***** 2020-11-28T21:42:11.284 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2020-11-28T21:42:11.285 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2020-11-28T21:42:11.326 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:11 +0000 (0:00:00.050) 0:01:17.471 ***** 2020-11-28T21:42:11.328 INFO:teuthology.task.ansible.out: 2020-11-28T21:42:11.346 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-28T21:42:11.347 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:11.360 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2020-11-28T21:42:11.361 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2020-11-28T21:42:11.403 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:11 +0000 (0:00:00.076) 0:01:17.548 ***** 2020-11-28T21:42:11.424 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2020-11-28T21:42:11.425 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"cache_update_time": 1606599636, "cache_updated": false, "changed": false} 2020-11-28T21:42:11.461 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:11 +0000 (0:00:00.057) 0:01:17.605 ***** 2020-11-28T21:42:11.488 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2020-11-28T21:42:11.490 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:11.526 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:11 +0000 (0:00:00.065) 0:01:17.671 ***** 2020-11-28T21:42:11.563 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-11-28T21:42:11.565 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => (item={'block_used': 13643527, 'uuid': '099240f7-127e-4e3e-8e24-c4358fbf763e', 'size_total': 983407812608, 'block_total': 240089798, 'mount': '/', 'block_available': 226446271, 'size_available': 927523926016, 'fstype': 'ext4', 'inode_total': 61046784, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 113957, 'block_size': 4096, 'inode_available': 60932827}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226446271, "block_size": 4096, "block_total": 240089798, "block_used": 13643527, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60932827, "inode_total": 61046784, "inode_used": 113957, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927523926016, "size_total": 983407812608, "uuid": "099240f7-127e-4e3e-8e24-c4358fbf763e"}} 2020-11-28T21:42:11.602 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:11 +0000 (0:00:00.075) 0:01:17.746 ***** 2020-11-28T21:42:11.671 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2020-11-28T21:42:11.673 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2020-11-28T21:42:11.710 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:11 +0000 (0:00:00.107) 0:01:17.854 ***** 2020-11-28T21:42:11.727 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-11-28T21:42:11.728 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} TASK [testnode : Compile list of non-root partitions] ************************** changed: [smithi118.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.004337", "end": "2020-11-28 21:42:11.774944", "rc": 0, "start": "2020-11-28 21:42:11.770607", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:11.766 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:11 +0000 (0:00:00.057) 0:01:17.911 ***** 2020-11-28T21:42:11.805 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:11 +0000 (0:00:00.032) 0:01:17.944 ***** 2020-11-28T21:42:11.816 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2020-11-28T21:42:11.818 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'block_used': 13643527, 'uuid': '099240f7-127e-4e3e-8e24-c4358fbf763e', 'size_total': 983407812608, 'block_total': 240089798, 'mount': '/', 'block_available': 226446271, 'size_available': 927523926016, 'fstype': 'ext4', 'inode_total': 61046784, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 113957, 'block_size': 4096, 'inode_available': 60932827}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226446271, "block_size": 4096, "block_total": 240089798, "block_used": 13643527, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60932827, "inode_total": 61046784, "inode_used": 113957, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927523926016, "size_total": 983407812608, "uuid": "099240f7-127e-4e3e-8e24-c4358fbf763e"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:11.853 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:11 +0000 (0:00:00.053) 0:01:17.997 ***** 2020-11-28T21:42:11.859 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-11-28T21:42:11.861 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:11.899 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:11 +0000 (0:00:00.045) 0:01:18.043 ***** 2020-11-28T21:42:11.905 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-11-28T21:42:11.951 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:11 +0000 (0:00:00.052) 0:01:18.095 ***** 2020-11-28T21:42:12.001 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-28T21:42:12.003 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:12.025 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** ok: [smithi162.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2020-11-28T21:42:12.077 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:12 +0000 (0:00:00.125) 0:01:18.222 ***** 2020-11-28T21:42:12.210 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2020-11-28T21:42:12.212 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-28T21:42:12.259 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:12 +0000 (0:00:00.182) 0:01:18.404 ***** 2020-11-28T21:42:12.266 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2020-11-28T21:42:12.268 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2020-11-28T21:42:12.309 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:12 +0000 (0:00:00.050) 0:01:18.454 ***** 2020-11-28T21:42:12.315 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2020-11-28T21:42:12.317 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:12.369 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:12 +0000 (0:00:00.059) 0:01:18.514 ***** 2020-11-28T21:42:12.399 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2020-11-28T21:42:12.401 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:12.447 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:12 +0000 (0:00:00.077) 0:01:18.592 ***** 2020-11-28T21:42:12.475 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2020-11-28T21:42:12.477 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:12.512 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:12 +0000 (0:00:00.065) 0:01:18.657 ***** 2020-11-28T21:42:12.539 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:12.577 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:12 +0000 (0:00:00.064) 0:01:18.722 ***** 2020-11-28T21:42:12.604 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2020-11-28T21:42:12.606 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:12.641 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:12 +0000 (0:00:00.064) 0:01:18.786 ***** 2020-11-28T21:42:12.657 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2020-11-28T21:42:12.659 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2020-11-28T21:42:12.701 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:12 +0000 (0:00:00.059) 0:01:18.846 ***** 2020-11-28T21:42:12.709 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** ok: [smithi162.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2020-11-28T21:42:12.761 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:12 +0000 (0:00:00.059) 0:01:18.905 ***** 2020-11-28T21:42:12.826 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi162.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:12.837 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:12.838 INFO:teuthology.task.ansible.out: 2020-11-28T21:42:12.849 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:12.860 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:12.883 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2020-11-28T21:42:12.883 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:12.885 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:12.895 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:12.906 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:12.922 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:12.930 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:12.942 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:12.986 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:12 +0000 (0:00:00.225) 0:01:19.131 ***** 2020-11-28T21:42:13.013 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2020-11-28T21:42:13.016 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2020-11-28T21:42:13.059 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:13 +0000 (0:00:00.072) 0:01:19.204 ***** 2020-11-28T21:42:13.186 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2020-11-28T21:42:13.188 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-28T21:42:13.231 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:13 +0000 (0:00:00.172) 0:01:19.376 ***** 2020-11-28T21:42:13.238 INFO:teuthology.task.ansible.out: 2020-11-28T21:42:13.239 INFO:teuthology.task.ansible.out:TASK [testnode : Check for teuthology-lock command] **************************** changed: [smithi162.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.004096", "end": "2020-11-28 21:42:13.163747", "rc": 0, "start": "2020-11-28 21:42:13.159651", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2020-11-28T21:42:13.293 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:13 +0000 (0:00:00.061) 0:01:19.438 ***** 2020-11-28T21:42:13.300 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2020-11-28T21:42:13.302 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:13.340 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:13 +0000 (0:00:00.046) 0:01:19.484 ***** 2020-11-28T21:42:13.345 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2020-11-28T21:42:13.347 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:13.388 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:13 +0000 (0:00:00.047) 0:01:19.532 ***** 2020-11-28T21:42:13.396 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2020-11-28T21:42:13.397 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:13.399 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-28T21:42:13.400 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-sWS1ITiPTJGlYd83LhoeEyBvcvYeHke0h1j2fFtfLAmrItJaS2xKhxeQaejsBvcz'], 'uuids': ['ed258c3f-7e48-4c6a-9d48-42d8ef91d944']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.918260", "end": "2020-11-28 21:42:13.734444", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-sWS1ITiPTJGlYd83LhoeEyBvcvYeHke0h1j2fFtfLAmrItJaS2xKhxeQaejsBvcz"], "labels": [], "masters": [], "uuids": ["ed258c3f-7e48-4c6a-9d48-42d8ef91d944"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:11.816184", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk u 2020-11-28T21:42:13.402 INFO:teuthology.task.ansible.out:sing fdisk or", "other utilities."]} 2020-11-28T21:42:13.451 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:13 +0000 (0:00:00.063) 0:01:19.595 ***** 2020-11-28T21:42:13.457 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:13.459 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:13.460 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:13.506 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:13 +0000 (0:00:00.055) 0:01:19.651 ***** 2020-11-28T21:42:13.515 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2020-11-28T21:42:13.517 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:13.554 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:13 +0000 (0:00:00.048) 0:01:19.699 ***** 2020-11-28T21:42:13.561 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2020-11-28T21:42:13.562 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:13.602 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:13 +0000 (0:00:00.047) 0:01:19.746 ***** 2020-11-28T21:42:13.609 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2020-11-28T21:42:13.610 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:13.671 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:13 +0000 (0:00:00.069) 0:01:19.816 ***** 2020-11-28T21:42:13.699 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2020-11-28T21:42:13.701 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:13.735 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:13 +0000 (0:00:00.063) 0:01:19.879 ***** 2020-11-28T21:42:13.761 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-28T21:42:13.763 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2020-11-28T21:42:13.797 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:13 +0000 (0:00:00.062) 0:01:19.942 ***** 2020-11-28T21:42:13.844 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi192.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:13.857 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:13.858 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:13.868 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:13.878 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:13.890 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:13.900 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2020-11-28T21:42:13.912 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:13.922 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:13.933 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:13.944 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:13.946 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:13.955 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:13.967 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:14.012 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:14 +0000 (0:00:00.214) 0:01:20.156 ***** 2020-11-28T21:42:14.187 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** changed: [smithi118.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-cxIO3Vu9nCI0CWBRg4zW02tKixX2sfUVRDX397v178R7Yrond2gALCMRI3qUUJWH'], 'uuids': ['ff8fb738-87f4-49a3-97fa-95044323f23d']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:02.058557", "end": "2020-11-28 21:42:14.219465", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-cxIO3Vu9nCI0CWBRg4zW02tKixX2sfUVRDX397v178R7Yrond2gALCMRI3qUUJWH"], "labels": [], "masters": [], "uuids": ["ff8fb738-87f4-49a3-97fa-95044323f23d"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:12.160908", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk u 2020-11-28T21:42:14.189 INFO:teuthology.task.ansible.out:sing fdisk or", "other utilities."]} 2020-11-28T21:42:14.203 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} TASK [testnode : Check for teuthology-lock command] **************************** changed: [smithi192.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003151", "end": "2020-11-28 21:42:14.172475", "rc": 0, "start": "2020-11-28 21:42:14.169324", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} TASK [testnode : Zap all non-root disks] *************************************** 2020-11-28T21:42:14.204 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:14.246 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:14 +0000 (0:00:00.234) 0:01:20.391 ***** 2020-11-28T21:42:14.283 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2020-11-28T21:42:14.285 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:14.323 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:14 +0000 (0:00:00.077) 0:01:20.468 ***** 2020-11-28T21:42:14.362 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2020-11-28T21:42:14.364 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:14.398 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:14 +0000 (0:00:00.074) 0:01:20.543 ***** 2020-11-28T21:42:14.425 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2020-11-28T21:42:14.427 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:14.461 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:14 +0000 (0:00:00.062) 0:01:20.605 ***** 2020-11-28T21:42:14.572 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2020-11-28T21:42:14.573 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => {"cache_update_time": 1606599631, "cache_updated": false, "changed": false} 2020-11-28T21:42:14.611 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:14 +0000 (0:00:00.150) 0:01:20.756 ***** 2020-11-28T21:42:14.639 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2020-11-28T21:42:14.641 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Zap all non-root disks] *************************************** changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-sWS1ITiPTJGlYd83LhoeEyBvcvYeHke0zwmllWT0MEeGksivSwGYCDaVkKkswPe2'], 'uuids': ['6ad7ec53-063a-4c4b-8e29-4580e40cee2e']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.115246", "end": "2020-11-28 21:42:14.990367", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-sWS1ITiPTJGlYd83LhoeEyBvcvYeHke0zwmllWT0MEeGksivSwGYCDaVkKkswPe2"], "labels": [], "masters": [], "uuids": ["6ad7ec53-063a-4c4b-8e29-4580e40cee2e"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:13.875121", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot 2020-11-28T21:42:14.642 INFO:teuthology.task.ansible.out: or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QGMP', 'wwn-0x5000c50091dfbebc'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '099240f7-127e-4e3e-8e24-c4358fbf763e', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QGMP-part1', 'wwn-0x5000c50091dfbebc-part1'], 'uuids': ['099240f7-127e-4e3e-8e24-c4358fbf763e']}, 'sectors': '1953476096', 'start': '2048', 'holders': [], 'size': '931.49 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c50091dfbebc', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5QGMP', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QGMP", "wwn-0x5000c50091dfbebc"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QGMP-part1", "wwn-0x5000c50091dfbebc-part1"], "labels": [], "masters": [], "uuids": ["099240f7-127e-4e3e-8e24-c4358fbf763e"]}, "sectors": "1953476096", "sectorsize": 512, "size": "931.49 GB", "start": "2048", "uuid": "099240f7-127e-4e3e-8e24-c4358fbf763e"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": 2020-11-28T21:42:14.644 INFO:teuthology.task.ansible.out:"512", "serial": "Z1W5QGMP", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50091dfbebc"}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:14.678 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:14 +0000 (0:00:00.067) 0:01:20.823 ***** 2020-11-28T21:42:14.716 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-11-28T21:42:14.717 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => (item={'block_used': 13643536, 'uuid': '099240f7-127e-4e3e-8e24-c4358fbf763e', 'size_total': 983407812608, 'block_total': 240089798, 'mount': '/', 'block_available': 226446262, 'size_available': 927523889152, 'fstype': 'ext4', 'inode_total': 61046784, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 113957, 'block_size': 4096, 'inode_available': 60932827}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226446262, "block_size": 4096, "block_total": 240089798, "block_used": 13643536, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60932827, "inode_total": 61046784, "inode_used": 113957, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927523889152, "size_total": 983407812608, "uuid": "099240f7-127e-4e3e-8e24-c4358fbf763e"}} 2020-11-28T21:42:14.753 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:14 +0000 (0:00:00.074) 0:01:20.898 ***** 2020-11-28T21:42:14.885 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2020-11-28T21:42:14.887 INFO:teuthology.task.ansible.out:changed: [smithi162.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.008425", "end": "2020-11-28 21:42:15.254555", "rc": 0, "start": "2020-11-28 21:42:15.246130", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:14.923 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:14 +0000 (0:00:00.169) 0:01:21.067 ***** 2020-11-28T21:42:14.961 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2020-11-28T21:42:14.963 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'block_used': 13643536, 'uuid': '099240f7-127e-4e3e-8e24-c4358fbf763e', 'size_total': 983407812608, 'block_total': 240089798, 'mount': '/', 'block_available': 226446262, 'size_available': 927523889152, 'fstype': 'ext4', 'inode_total': 61046784, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 113957, 'block_size': 4096, 'inode_available': 60932827}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226446262, "block_size": 4096, "block_total": 240089798, "block_used": 13643536, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60932827, "inode_total": 61046784, "inode_used": 113957, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927523889152, "size_total": 983407812608, "uuid": "099240f7-127e-4e3e-8e24-c4358fbf763e"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:14.999 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:14 +0000 (0:00:00.076) 0:01:21.143 ***** 2020-11-28T21:42:15.026 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-11-28T21:42:15.066 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:15 +0000 (0:00:00.067) 0:01:21.211 ***** 2020-11-28T21:42:15.114 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-28T21:42:15.115 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:15.335 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2020-11-28T21:42:15.337 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"cache_update_time": 1606599641, "cache_updated": false, "changed": false} 2020-11-28T21:42:15.373 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:15 +0000 (0:00:00.306) 0:01:21.517 ***** 2020-11-28T21:42:15.400 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2020-11-28T21:42:15.402 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:15.444 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:15 +0000 (0:00:00.071) 0:01:21.589 ***** 2020-11-28T21:42:15.451 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-28T21:42:15.453 INFO:teuthology.task.ansible.out:changed: [smithi118.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-cxIO3Vu9nCI0CWBRg4zW02tKixX2sfUV7EdA1ZiAWUZhxvw4rINHSzsD2udbxn2A'], 'uuids': ['f824dad7-cd7d-4a8d-8897-bce32af615db']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.105385", "end": "2020-11-28 21:42:15.452401", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-cxIO3Vu9nCI0CWBRg4zW02tKixX2sfUV7EdA1ZiAWUZhxvw4rINHSzsD2udbxn2A"], "labels": [], "masters": [], "uuids": ["f824dad7-cd7d-4a8d-8897-bce32af615db"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:14.347016", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-28T21:42:15.454 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => 2020-11-28T21:42:15.456 INFO:teuthology.task.ansible.out:(item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QARN', 'wwn-0x5000c50091e0b046'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '099240f7-127e-4e3e-8e24-c4358fbf763e', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QARN-part1', 'wwn-0x5000c50091e0b046-part1'], 'uuids': ['099240f7-127e-4e3e-8e24-c4358fbf763e']}, 'sectors': '1953476096', 'start': '2048', 'holders': [], 'size': '931.49 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c50091e0b046', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5QARN', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QARN", "wwn-0x5000c50091e0b046"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QARN-part1", "wwn-0x5000c50091e0b046-part1"], "labels": [], "masters": [], "uuids": ["099240f7-127e-4e3e-8e24-c4358fbf763e"]}, "sectors": "1953476096", "sectorsize": 512, "size": "931.49 GB", "start": "2048", "uuid": "099240f7-127e-4e3e-8e24-c4358fbf763e"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5QARN", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50091e0b046"}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:15.489 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ ok: [smithi192.front.sepia.ceph.com] => (item={'block_used': 13643537, 'uuid': '099240f7-127e-4e3e-8e24-c4358fbf763e', 'size_total': 983407812608, 'block_total': 240089798, 'mount': '/', 'block_available': 226446261, 'size_available': 927523885056, 'fstype': 'ext4', 'inode_total': 61046784, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 113957, 'block_size': 4096, 'inode_available': 60932827}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226446261, "block_size": 4096, "block_total": 240089798, "block_used": 13643537, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60932827, "inode_total": 61046784, "inode_used": 113957, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927523885056, "size_total": 983407812608, "uuid": "099240f7-127e-4e3e-8e24-c4358fbf763e"}} 2020-11-28T21:42:15.525 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:15 +0000 (0:00:00.081) 0:01:21.670 ***** 2020-11-28T21:42:15.660 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2020-11-28T21:42:15.661 INFO:teuthology.task.ansible.out:changed: [smithi192.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.007886", "end": "2020-11-28 21:42:15.944882", "rc": 0, "start": "2020-11-28 21:42:15.936996", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:15.697 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:15 +0000 (0:00:00.171) 0:01:21.841 ***** 2020-11-28T21:42:15.746 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** skipping: [smithi192.front.sepia.ceph.com] => (item={'block_used': 13643537, 'uuid': '099240f7-127e-4e3e-8e24-c4358fbf763e', 'size_total': 983407812608, 'block_total': 240089798, 'mount': '/', 'block_available': 226446261, 'size_available': 927523885056, 'fstype': 'ext4', 'inode_total': 61046784, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 113957, 'block_size': 4096, 'inode_available': 60932827}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226446261, "block_size": 4096, "block_total": 240089798, "block_used": 13643537, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60932827, "inode_total": 61046784, "inode_used": 113957, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927523885056, "size_total": 983407812608, "uuid": "099240f7-127e-4e3e-8e24-c4358fbf763e"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:15.784 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:15 +0000 (0:00:00.087) 0:01:21.929 ***** 2020-11-28T21:42:15.812 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-11-28T21:42:15.850 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:15 +0000 (0:00:00.066) 0:01:21.995 ***** 2020-11-28T21:42:15.901 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-28T21:42:15.903 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:15.924 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-uHGxkk-T6rC-fH4y-NQbs-GDq7-AL7K-wisbeo', 'nvme-INTEL_SSDPEDMD400G4_PHFT620400NH400BGN', 'nvme-nvme.8086-504846543632303430304e4834303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.154260", "end": "2020-11-28 21:42:16.276764", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-uHGxkk-T6rC-fH4y-NQbs-GDq7-AL7K-wisbeo", "nvme-INTEL_SSDPEDMD400G4_PHFT620400NH400BGN", "nvme-nvme.8086-504846543632303430304e4834303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:15.122504", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! 2020-11-28T21:42:15.926 INFO:teuthology.task.ansible.out:You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-28T21:42:15.940 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:15.952 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:16.705 INFO:teuthology.task.ansible.out:changed: [smithi118.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '937703088', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-l0TlgJ-rjGn-hbuZ-ttiB-p8Vl-J0D1-eYgYsh', 'nvme-INTEL_SSDPED1D480GA_PHMB747500JS480DGN', 'nvme-nvme.8086-50484d423734373530304a5334383044474e-494e54454c2053534450454431443438304741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPED1D480GA', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '447.13 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.135605", "end": "2020-11-28 21:42:16.732421", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series", "links": {"ids": ["lvm-pv-uuid-l0TlgJ-rjGn-hbuZ-ttiB-p8Vl-J0D1-eYgYsh", "nvme-INTEL_SSDPED1D480GA_PHMB747500JS480DGN", "nvme-nvme.8086-50484d423734373530304a5334383044474e-494e54454c2053534450454431443438304741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPED1D480GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "937703088", "sectorsize": "512", "size": "447.13 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:15.596816", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! You may now pa 2020-11-28T21:42:16.706 INFO:teuthology.task.ansible.out:rtition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} skipping: [smithi118.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:16.717 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:17.036 INFO:teuthology.task.ansible.out:changed: [smithi162.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-rjW7d4X5PJFxmKb6ntPdx0xRaZEdvy6bh6b3VPYP9IoRhiKa0dEEnkAjUDINN65g'], 'uuids': ['873abb87-1cbf-490b-a4ac-a921021ba5e8']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.791109", "end": "2020-11-28 21:42:17.393847", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-rjW7d4X5PJFxmKb6ntPdx0xRaZEdvy6bh6b3VPYP9IoRhiKa0dEEnkAjUDINN65g"], "labels": [], "masters": [], "uuids": ["873abb87-1cbf-490b-a4ac-a921021ba5e8"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:15.602738", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-28T21:42:17.038 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:17.049 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:17.207 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-sWS1ITiPTJGlYd83LhoeEyBvcvYeHke0rttP2IL32JUjte60imeRmPNgGG59qLd3'], 'uuids': ['ce00b196-2f46-44ec-8a61-7fe0f180c0a9']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.129129", "end": "2020-11-28 21:42:17.547938", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-sWS1ITiPTJGlYd83LhoeEyBvcvYeHke0rttP2IL32JUjte60imeRmPNgGG59qLd3"], "labels": [], "masters": [], "uuids": ["ce00b196-2f46-44ec-8a61-7fe0f180c0a9"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:16.418809", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-28T21:42:17.219 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:17.230 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:17.652 INFO:teuthology.task.ansible.out:changed: [smithi192.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-XqS7OPtzpkHiN1h6pKSHcGqXVlUIYflV2Y8SuIPl6f7Mo2vHDpgOWM7j6iAe0K7u'], 'uuids': ['c6b7ffc0-933f-426a-92e1-eafee4aa79ca']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.615544", "end": "2020-11-28 21:42:17.919952", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-XqS7OPtzpkHiN1h6pKSHcGqXVlUIYflV2Y8SuIPl6f7Mo2vHDpgOWM7j6iAe0K7u"], "labels": [], "masters": [], "uuids": ["c6b7ffc0-933f-426a-92e1-eafee4aa79ca"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:16.304408", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-28T21:42:17.653 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:17.665 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:17.982 INFO:teuthology.task.ansible.out:changed: [smithi118.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-cxIO3Vu9nCI0CWBRg4zW02tKixX2sfUVhfcCxwkImsj2bBHKad3VAhztY8MUbGRh'], 'uuids': ['ded34bd8-0920-4a79-8aa9-18772f2f9594']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.147633", "end": "2020-11-28 21:42:18.009556", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-cxIO3Vu9nCI0CWBRg4zW02tKixX2sfUVhfcCxwkImsj2bBHKad3VAhztY8MUbGRh"], "labels": [], "masters": [], "uuids": ["ded34bd8-0920-4a79-8aa9-18772f2f9594"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:16.861923", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-28T21:42:17.994 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:17.996 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:18.284 INFO:teuthology.task.ansible.out:changed: [smithi162.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-rjW7d4X5PJFxmKb6ntPdx0xRaZEdvy6bGn2iWj7HSLxjIughXIXL8H7qQuvGY8nK'], 'uuids': ['08f042f6-6507-4c3f-a299-a6dc512047ac']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.123185", "end": "2020-11-28 21:42:18.645997", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-rjW7d4X5PJFxmKb6ntPdx0xRaZEdvy6bGn2iWj7HSLxjIughXIXL8H7qQuvGY8nK"], "labels": [], "masters": [], "uuids": ["08f042f6-6507-4c3f-a299-a6dc512047ac"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:17.522812", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-28T21:42:18.295 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XTKR', 'wwn-0x5000c5009293e333'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '099240f7-127e-4e3e-8e24-c4358fbf763e', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XTKR-part1', 'wwn-0x5000c5009293e333-part1'], 'uuids': ['099240f7-127e-4e3e-8e24-c4358fbf763e']}, 'sectors': '1953476096', 'start': '2048', 'holders': [], 'size': '931.49 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c5009293e333', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5XTKR', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XTKR", "wwn-0x5000c5009293e333"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XTKR-part1", "wwn-0x5000c5009293e333-part1"], "labels": [], "masters": [], "uuids": ["099240f7-127e-4e3e-8e24-c4358fbf763e"]}, "sectors": "1953476096", "sectorsize": 512, "size": "931.49 GB", "start": "2048", "uuid": "099240f7-127e-4e3e-8e24-c4358fbf763e"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XTKR", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c5009293e333"}}, "skip_reason": 2020-11-28T21:42:18.297 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2020-11-28T21:42:18.475 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-sWS1ITiPTJGlYd83LhoeEyBvcvYeHke0AvqgS9l1Uox1Lw51RSynYe1DmrUilcjU'], 'uuids': ['a0b3d8c0-5fb6-4786-a041-8be59608ccfd']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.117322", "end": "2020-11-28 21:42:18.821811", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-sWS1ITiPTJGlYd83LhoeEyBvcvYeHke0AvqgS9l1Uox1Lw51RSynYe1DmrUilcjU"], "labels": [], "masters": [], "uuids": ["a0b3d8c0-5fb6-4786-a041-8be59608ccfd"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:17.704489", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-28T21:42:18.487 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:18.920 INFO:teuthology.task.ansible.out:changed: [smithi192.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-XqS7OPtzpkHiN1h6pKSHcGqXVlUIYflV1hzTictJljwPxSw5Nnn96p1MxHej4a3m'], 'uuids': ['3b8ceb95-9471-40a5-8e1f-30291f991790']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.134840", "end": "2020-11-28 21:42:19.196862", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-XqS7OPtzpkHiN1h6pKSHcGqXVlUIYflV1hzTictJljwPxSw5Nnn96p1MxHej4a3m"], "labels": [], "masters": [], "uuids": ["3b8ceb95-9471-40a5-8e1f-30291f991790"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:18.062022", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-28T21:42:18.932 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XLLK', 'wwn-0x5000c50092939e0b'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '099240f7-127e-4e3e-8e24-c4358fbf763e', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XLLK-part1', 'wwn-0x5000c50092939e0b-part1'], 'uuids': ['099240f7-127e-4e3e-8e24-c4358fbf763e']}, 'sectors': '1953476096', 'start': '2048', 'holders': [], 'size': '931.49 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c50092939e0b', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5XLLK', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XLLK", "wwn-0x5000c50092939e0b"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XLLK-part1", "wwn-0x5000c50092939e0b-part1"], "labels": [], "masters": [], "uuids": ["099240f7-127e-4e3e-8e24-c4358fbf763e"]}, "sectors": "1953476096", "sectorsize": 512, "size": "931.49 GB", "start": "2048", "uuid": "099240f7-127e-4e3e-8e24-c4358fbf763e"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XLLK", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50092939e0b"}}, "skip_reason": 2020-11-28T21:42:18.934 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2020-11-28T21:42:19.218 INFO:teuthology.task.ansible.out:changed: [smithi118.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-cxIO3Vu9nCI0CWBRg4zW02tKixX2sfUVwf0esMQepFlsZYIQC6dUyBrKS3rPi1OU'], 'uuids': ['c295c62f-8f6f-4c36-87ba-c9518b3cc67c']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.099945", "end": "2020-11-28 21:42:19.259129", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-cxIO3Vu9nCI0CWBRg4zW02tKixX2sfUVwf0esMQepFlsZYIQC6dUyBrKS3rPi1OU"], "labels": [], "masters": [], "uuids": ["c295c62f-8f6f-4c36-87ba-c9518b3cc67c"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:18.159184", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-28T21:42:19.230 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:19.577 INFO:teuthology.task.ansible.out:changed: [smithi162.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-1Dbw0i-EBH1-OnjY-lDS7-NpBI-uU9r-G7BEfq', 'nvme-INTEL_SSDPEDMD400G4_CVFT623300JU400BGN', 'nvme-nvme.8086-435646543632333330304a5534303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.143299", "end": "2020-11-28 21:42:19.941217", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-1Dbw0i-EBH1-OnjY-lDS7-NpBI-uU9r-G7BEfq", "nvme-INTEL_SSDPEDMD400G4_CVFT623300JU400BGN", "nvme-nvme.8086-435646543632333330304a5534303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:18.797918", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! 2020-11-28T21:42:19.580 INFO:teuthology.task.ansible.out:You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-28T21:42:19.590 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:19.603 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:19.723 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-sWS1ITiPTJGlYd83LhoeEyBvcvYeHke0c3jb2z4se6vOYYLCKKVIkncZXyYYhcMk'], 'uuids': ['90cd01c2-f932-4969-b9e5-6006ecb0047e']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.123778", "end": "2020-11-28 21:42:20.071919", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-sWS1ITiPTJGlYd83LhoeEyBvcvYeHke0c3jb2z4se6vOYYLCKKVIkncZXyYYhcMk"], "labels": [], "masters": [], "uuids": ["90cd01c2-f932-4969-b9e5-6006ecb0047e"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:18.948141", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-28T21:42:19.794 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:19 +0000 (0:00:03.943) 0:01:25.939 ***** 2020-11-28T21:42:19.843 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-28T21:42:19.845 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:20.106 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-sWS1ITiPTJGlYd83LhoeEyBvcvYeHke0h1j2fFtfLAmrItJaS2xKhxeQaejsBvcz'], 'uuids': ['ed258c3f-7e48-4c6a-9d48-42d8ef91d944']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.146297", "end": "2020-11-28 21:42:20.458971", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-sWS1ITiPTJGlYd83LhoeEyBvcvYeHke0h1j2fFtfLAmrItJaS2xKhxeQaejsBvcz"], "labels": [], "masters": [], "uuids": ["ed258c3f-7e48-4c6a-9d48-42d8ef91d944"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:20.312674", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.141978 s, 812 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.141978 s, 812 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:20.117 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:20.129 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:20.214 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-28T21:42:20.216 INFO:teuthology.task.ansible.out:changed: [smithi192.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-30lnTh-xnbu-AoH9-tHHr-1VSp-TtqW-ai2SU9', 'nvme-INTEL_SSDPEDMD400G4_PHFT620500NN400BGN', 'nvme-nvme.8086-504846543632303530304e4e34303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.169491", "end": "2020-11-28 21:42:20.492525", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-30lnTh-xnbu-AoH9-tHHr-1VSp-TtqW-ai2SU9", "nvme-INTEL_SSDPEDMD400G4_PHFT620500NN400BGN", "nvme-nvme.8086-504846543632303530304e4e34303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:19.323034", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! 2020-11-28T21:42:20.217 INFO:teuthology.task.ansible.out:You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-28T21:42:20.226 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:20.248 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:20.409 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-28T21:42:20.410 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-sWS1ITiPTJGlYd83LhoeEyBvcvYeHke0zwmllWT0MEeGksivSwGYCDaVkKkswPe2'], 'uuids': ['6ad7ec53-063a-4c4b-8e29-4580e40cee2e']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.155033", "end": "2020-11-28 21:42:20.757360", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-sWS1ITiPTJGlYd83LhoeEyBvcvYeHke0zwmllWT0MEeGksivSwGYCDaVkKkswPe2"], "labels": [], "masters": [], "uuids": ["6ad7ec53-063a-4c4b-8e29-4580e40cee2e"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:20.602327", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.15036 s, 767 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.15036 s, 767 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:20.422 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QGMP', 'wwn-0x5000c50091dfbebc'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '099240f7-127e-4e3e-8e24-c4358fbf763e', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QGMP-part1', 'wwn-0x5000c50091dfbebc-part1'], 'uuids': ['099240f7-127e-4e3e-8e24-c4358fbf763e']}, 'sectors': '1953476096', 'start': '2048', 'holders': [], 'size': '931.49 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c50091dfbebc', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5QGMP', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QGMP", "wwn-0x5000c50091dfbebc"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QGMP-part1", "wwn-0x5000c50091dfbebc-part1"], "labels": [], "masters": [], "uuids": ["099240f7-127e-4e3e-8e24-c4358fbf763e"]}, "sectors": "1953476096", "sectorsize": 512, "size": "931.49 GB", "start": "2048", "uuid": "099240f7-127e-4e3e-8e24-c4358fbf763e"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5QGMP", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50091dfbebc"}}, "skip_reason": 2020-11-28T21:42:20.424 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2020-11-28T21:42:20.465 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-28T21:42:20.468 INFO:teuthology.task.ansible.out:changed: [smithi118.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-cxIO3Vu9nCI0CWBRg4zW02tKixX2sfUVCwsMSlCuEzevcyDl8RPPARsAkdiJP5Tn'], 'uuids': ['fbc4f316-4d86-40e9-8dff-094b7d3b68a8']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.106697", "end": "2020-11-28 21:42:20.501684", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-cxIO3Vu9nCI0CWBRg4zW02tKixX2sfUVCwsMSlCuEzevcyDl8RPPARsAkdiJP5Tn"], "labels": [], "masters": [], "uuids": ["fbc4f316-4d86-40e9-8dff-094b7d3b68a8"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:19.394987", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-28T21:42:20.513 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:20 +0000 (0:00:00.719) 0:01:26.658 ***** 2020-11-28T21:42:20.565 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-28T21:42:20.567 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:20.589 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-uHGxkk-T6rC-fH4y-NQbs-GDq7-AL7K-wisbeo', 'nvme-INTEL_SSDPEDMD400G4_PHFT620400NH400BGN', 'nvme-nvme.8086-504846543632303430304e4834303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.048301", "end": "2020-11-28 21:42:20.934842", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-uHGxkk-T6rC-fH4y-NQbs-GDq7-AL7K-wisbeo", "nvme-INTEL_SSDPEDMD400G4_PHFT620400NH400BGN", "nvme-nvme.8086-504846543632303430304e4834303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:20.886541", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0440415 s, 2.6 GB/s", "std 2020-11-28T21:42:20.590 INFO:teuthology.task.ansible.out:err_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0440415 s, 2.6 GB/s"], "stdout": "", "stdout_lines": []} skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:20.601 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:20.783 INFO:teuthology.task.ansible.out:changed: [smithi118.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-cxIO3Vu9nCI0CWBRg4zW02tKixX2sfUVRDX397v178R7Yrond2gALCMRI3qUUJWH'], 'uuids': ['ff8fb738-87f4-49a3-97fa-95044323f23d']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.107770", "end": "2020-11-28 21:42:20.819206", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-cxIO3Vu9nCI0CWBRg4zW02tKixX2sfUVRDX397v178R7Yrond2gALCMRI3qUUJWH"], "labels": [], "masters": [], "uuids": ["ff8fb738-87f4-49a3-97fa-95044323f23d"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:20.711436", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.104674 s, 1.1 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.104674 s, 1.1 GB/s"], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:20.794 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:20.806 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:20.860 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-28T21:42:20.862 INFO:teuthology.task.ansible.out:changed: [smithi162.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-rjW7d4X5PJFxmKb6ntPdx0xRaZEdvy6bLshnlWWjy8kG6R9dxsKm07Ygu2dabmZy'], 'uuids': ['f70bdb3e-c65f-4d14-902f-3ceac1a56e18']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.135991", "end": "2020-11-28 21:42:21.228377", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-rjW7d4X5PJFxmKb6ntPdx0xRaZEdvy6bLshnlWWjy8kG6R9dxsKm07Ygu2dabmZy"], "labels": [], "masters": [], "uuids": ["f70bdb3e-c65f-4d14-902f-3ceac1a56e18"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:20.092386", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-28T21:42:20.873 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-28T21:42:20.875 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-sWS1ITiPTJGlYd83LhoeEyBvcvYeHke0rttP2IL32JUjte60imeRmPNgGG59qLd3'], 'uuids': ['ce00b196-2f46-44ec-8a61-7fe0f180c0a9']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.157114", "end": "2020-11-28 21:42:21.225451", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-sWS1ITiPTJGlYd83LhoeEyBvcvYeHke0rttP2IL32JUjte60imeRmPNgGG59qLd3"], "labels": [], "masters": [], "uuids": ["ce00b196-2f46-44ec-8a61-7fe0f180c0a9"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:21.068337", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.154302 s, 748 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.154302 s, 748 MB/s"], "stdout": "", "stdout_lines": []} TASK [testnode : Zap all non-root disks] *************************************** 2020-11-28T21:42:20.875 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0 2020-11-28T21:42:20.877 INFO:teuthology.task.ansible.out:', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-28T21:42:20.877 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:20.879 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** skipping: [smithi162.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:20.891 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:21.030 INFO:teuthology.task.ansible.out:changed: [smithi118.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-cxIO3Vu9nCI0CWBRg4zW02tKixX2sfUV7EdA1ZiAWUZhxvw4rINHSzsD2udbxn2A'], 'uuids': ['f824dad7-cd7d-4a8d-8897-bce32af615db']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.104249", "end": "2020-11-28 21:42:21.052951", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-cxIO3Vu9nCI0CWBRg4zW02tKixX2sfUV7EdA1ZiAWUZhxvw4rINHSzsD2udbxn2A"], "labels": [], "masters": [], "uuids": ["f824dad7-cd7d-4a8d-8897-bce32af615db"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:20.948702", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.100247 s, 1.2 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.100247 s, 1.2 GB/s"], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:21.033 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QARN', 'wwn-0x5000c50091e0b046'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '099240f7-127e-4e3e-8e24-c4358fbf763e', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QARN-part1', 'wwn-0x5000c50091e0b046-part1'], 'uuids': ['099240f7-127e-4e3e-8e24-c4358fbf763e']}, 'sectors': '1953476096', 'start': '2048', 'holders': [], 'size': '931.49 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c50091e0b046', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5QARN', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QARN", "wwn-0x5000c50091e0b046"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QARN-part1", "wwn-0x5000c50091e0b046-part1"], "labels": [], "masters": [], "uuids": ["099240f7-127e-4e3e-8e24-c4358fbf763e"]}, "sectors": "1953476096", "sectorsize": 512, "size": "931.49 GB", "start": "2048", "uuid": "099240f7-127e-4e3e-8e24-c4358fbf763e"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5QARN", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50091e0b046"}}, "skip_reason": 2020-11-28T21:42:21.035 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2020-11-28T21:42:21.160 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-sWS1ITiPTJGlYd83LhoeEyBvcvYeHke0AvqgS9l1Uox1Lw51RSynYe1DmrUilcjU'], 'uuids': ['a0b3d8c0-5fb6-4786-a041-8be59608ccfd']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.153852", "end": "2020-11-28 21:42:21.505727", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-sWS1ITiPTJGlYd83LhoeEyBvcvYeHke0AvqgS9l1Uox1Lw51RSynYe1DmrUilcjU"], "labels": [], "masters": [], "uuids": ["a0b3d8c0-5fb6-4786-a041-8be59608ccfd"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:21.351875", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.149233 s, 773 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.149233 s, 773 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:21.172 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:21.194 INFO:teuthology.task.ansible.out:changed: [smithi118.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '937703088', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-l0TlgJ-rjGn-hbuZ-ttiB-p8Vl-J0D1-eYgYsh', 'nvme-INTEL_SSDPED1D480GA_PHMB747500JS480DGN', 'nvme-nvme.8086-50484d423734373530304a5334383044474e-494e54454c2053534450454431443438304741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPED1D480GA', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '447.13 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.044582", "end": "2020-11-28 21:42:21.232103", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series", "links": {"ids": ["lvm-pv-uuid-l0TlgJ-rjGn-hbuZ-ttiB-p8Vl-J0D1-eYgYsh", "nvme-INTEL_SSDPED1D480GA_PHMB747500JS480DGN", "nvme-nvme.8086-50484d423734373530304a5334383044474e-494e54454c2053534450454431443438304741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPED1D480GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "937703088", "sectorsize": "512", "size": "447.13 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:21.187521", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0415165 s, 2.8 GB/s", "stderr_lines": [" 2020-11-28T21:42:21.196 INFO:teuthology.task.ansible.out:110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0415165 s, 2.8 GB/s"], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:21.207 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:21.219 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:21.418 INFO:teuthology.task.ansible.out:changed: [smithi118.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-cxIO3Vu9nCI0CWBRg4zW02tKixX2sfUVhfcCxwkImsj2bBHKad3VAhztY8MUbGRh'], 'uuids': ['ded34bd8-0920-4a79-8aa9-18772f2f9594']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.097529", "end": "2020-11-28 21:42:21.454521", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-cxIO3Vu9nCI0CWBRg4zW02tKixX2sfUVhfcCxwkImsj2bBHKad3VAhztY8MUbGRh"], "labels": [], "masters": [], "uuids": ["ded34bd8-0920-4a79-8aa9-18772f2f9594"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:21.356992", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.093556 s, 1.2 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.093556 s, 1.2 GB/s"], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:21.430 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:21.442 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:21.443 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-sWS1ITiPTJGlYd83LhoeEyBvcvYeHke0c3jb2z4se6vOYYLCKKVIkncZXyYYhcMk'], 'uuids': ['90cd01c2-f932-4969-b9e5-6006ecb0047e']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.160085", "end": "2020-11-28 21:42:21.787257", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-sWS1ITiPTJGlYd83LhoeEyBvcvYeHke0c3jb2z4se6vOYYLCKKVIkncZXyYYhcMk"], "labels": [], "masters": [], "uuids": ["90cd01c2-f932-4969-b9e5-6006ecb0047e"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:21.627172", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.157261 s, 733 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.157261 s, 733 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:21.494 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:21 +0000 (0:00:00.981) 0:01:27.639 ***** 2020-11-28T21:42:21.502 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-28T21:42:21.502 INFO:teuthology.task.ansible.out:changed: [smithi192.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-XqS7OPtzpkHiN1h6pKSHcGqXVlUIYflVGOCsmOGYw4wyRJHqTEt0DQloUvMUP6de'], 'uuids': ['f2763df2-c9d7-4dc5-ab4a-4b1f0b9a8567']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.111296", "end": "2020-11-28 21:42:21.739673", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-XqS7OPtzpkHiN1h6pKSHcGqXVlUIYflVGOCsmOGYw4wyRJHqTEt0DQloUvMUP6de"], "labels": [], "masters": [], "uuids": ["f2763df2-c9d7-4dc5-ab4a-4b1f0b9a8567"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:20.628377", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-28T21:42:21.504 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:21.506 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:21.655 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-28T21:42:21.656 INFO:teuthology.task.ansible.out:changed: [smithi118.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-cxIO3Vu9nCI0CWBRg4zW02tKixX2sfUVwf0esMQepFlsZYIQC6dUyBrKS3rPi1OU'], 'uuids': ['c295c62f-8f6f-4c36-87ba-c9518b3cc67c']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.108425", "end": "2020-11-28 21:42:21.692688", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-cxIO3Vu9nCI0CWBRg4zW02tKixX2sfUVwf0esMQepFlsZYIQC6dUyBrKS3rPi1OU"], "labels": [], "masters": [], "uuids": ["c295c62f-8f6f-4c36-87ba-c9518b3cc67c"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:21.584263", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.103908 s, 1.1 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.103908 s, 1.1 GB/s"], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:21.668 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:21.732 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** changed: [smithi096.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.113754", "end": "2020-11-28 21:42:22.076470", "rc": 0, "start": "2020-11-28 21:42:21.962716", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:21.770 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:21 +0000 (0:00:00.275) 0:01:27.915 ***** 2020-11-28T21:42:21.871 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-28T21:42:21.873 INFO:teuthology.task.ansible.out:changed: [smithi118.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-cxIO3Vu9nCI0CWBRg4zW02tKixX2sfUVCwsMSlCuEzevcyDl8RPPARsAkdiJP5Tn'], 'uuids': ['fbc4f316-4d86-40e9-8dff-094b7d3b68a8']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.100204", "end": "2020-11-28 21:42:21.905053", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-cxIO3Vu9nCI0CWBRg4zW02tKixX2sfUVCwsMSlCuEzevcyDl8RPPARsAkdiJP5Tn"], "labels": [], "masters": [], "uuids": ["fbc4f316-4d86-40e9-8dff-094b7d3b68a8"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:21.804849", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0978598 s, 1.2 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0978598 s, 1.2 GB/s"], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:21.918 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:21 +0000 (0:00:00.148) 0:01:28.063 ***** 2020-11-28T21:42:21.956 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2020-11-28T21:42:21.958 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.055309", "end": "2020-11-28 21:42:22.304718", "rc": 0, "start": "2020-11-28 21:42:22.249409", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:21.993 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:21 +0000 (0:00:00.074) 0:01:28.138 ***** 2020-11-28T21:42:22.020 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-11-28T21:42:22.059 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:22 +0000 (0:00:00.065) 0:01:28.203 ***** 2020-11-28T21:42:22.085 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:22.124 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:22 +0000 (0:00:00.065) 0:01:28.269 ***** 2020-11-28T21:42:22.131 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-28T21:42:22.133 INFO:teuthology.task.ansible.out:changed: [smithi162.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-rjW7d4X5PJFxmKb6ntPdx0xRaZEdvy6by93u1bhZOf3G73tYhnaLIEBEeCegDT68'], 'uuids': ['d4c04505-2f09-43b3-a97e-084e4bd1211d']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.103321", "end": "2020-11-28 21:42:22.476697", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-rjW7d4X5PJFxmKb6ntPdx0xRaZEdvy6by93u1bhZOf3G73tYhnaLIEBEeCegDT68"], "labels": [], "masters": [], "uuids": ["d4c04505-2f09-43b3-a97e-084e4bd1211d"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:21.373376", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-28T21:42:22.134 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => 2020-11-28T21:42:22.136 INFO:teuthology.task.ansible.out:(item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:22.155 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2020-11-28T21:42:22.157 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:22.193 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:22 +0000 (0:00:00.068) 0:01:28.338 ***** 2020-11-28T21:42:22.200 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2020-11-28T21:42:22.201 INFO:teuthology.task.ansible.out:changed: [smithi118.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.125383", "end": "2020-11-28 21:42:22.204384", "rc": 0, "start": "2020-11-28 21:42:22.079001", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:22.237 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:22 +0000 (0:00:00.044) 0:01:28.382 ***** 2020-11-28T21:42:22.243 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-11-28T21:42:22.245 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:22.284 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:22 +0000 (0:00:00.046) 0:01:28.429 ***** 2020-11-28T21:42:22.312 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2020-11-28T21:42:22.314 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:22.348 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:22 +0000 (0:00:00.063) 0:01:28.492 ***** 2020-11-28T21:42:22.416 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** changed: [smithi118.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.049443", "end": "2020-11-28 21:42:22.457475", "rc": 0, "start": "2020-11-28 21:42:22.408032", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:22.455 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:22 +0000 (0:00:00.107) 0:01:28.600 ***** 2020-11-28T21:42:22.472 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2020-11-28T21:42:22.474 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-28T21:42:22.509 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:22 +0000 (0:00:00.053) 0:01:28.653 ***** 2020-11-28T21:42:22.515 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-11-28T21:42:22.556 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:22 +0000 (0:00:00.047) 0:01:28.700 ***** 2020-11-28T21:42:22.562 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-11-28T21:42:22.564 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'block_used': 13643536, 'uuid': '099240f7-127e-4e3e-8e24-c4358fbf763e', 'size_total': 983407812608, 'block_total': 240089798, 'mount': '/', 'block_available': 226446262, 'size_available': 927523889152, 'fstype': 'ext4', 'inode_total': 61046784, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 113957, 'block_size': 4096, 'inode_available': 60932827}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226446262, "block_size": 4096, "block_total": 240089798, "block_used": 13643536, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60932827, "inode_total": 61046784, "inode_used": 113957, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927523889152, "size_total": 983407812608, "uuid": "099240f7-127e-4e3e-8e24-c4358fbf763e"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:22.601 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:22 +0000 (0:00:00.045) 0:01:28.746 ***** 2020-11-28T21:42:22.606 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2020-11-28T21:42:22.608 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:22.646 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:22 +0000 (0:00:00.044) 0:01:28.790 ***** 2020-11-28T21:42:22.651 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2020-11-28T21:42:22.653 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:22.689 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:22 +0000 (0:00:00.043) 0:01:28.834 ***** 2020-11-28T21:42:22.695 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2020-11-28T21:42:22.696 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:22.735 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:22 +0000 (0:00:00.045) 0:01:28.879 ***** 2020-11-28T21:42:22.740 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-28T21:42:22.742 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:22.776 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:22 +0000 (0:00:00.041) 0:01:28.921 ***** 2020-11-28T21:42:22.783 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-28T21:42:22.784 INFO:teuthology.task.ansible.out:changed: [smithi192.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-XqS7OPtzpkHiN1h6pKSHcGqXVlUIYflV2nbdDSGx61uJafMjM4eftPL7dxAD2CDF'], 'uuids': ['c0cbf01a-4ba0-40ca-b4e1-aa422e6b8e36']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.147449", "end": "2020-11-28 21:42:23.019204", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-XqS7OPtzpkHiN1h6pKSHcGqXVlUIYflV2nbdDSGx61uJafMjM4eftPL7dxAD2CDF"], "labels": [], "masters": [], "uuids": ["c0cbf01a-4ba0-40ca-b4e1-aa422e6b8e36"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:21.871755", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} TASK [testnode : Erase /scratch_devs so we k 2020-11-28T21:42:22.786 INFO:teuthology.task.ansible.out:now it's accurate] ***************** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Zap all non-root disks] *************************************** skipping: [smithi192.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:22.829 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:22 +0000 (0:00:00.052) 0:01:28.974 ***** 2020-11-28T21:42:22.835 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-28T21:42:22.837 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:22.872 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:22 +0000 (0:00:00.043) 0:01:29.017 ***** 2020-11-28T21:42:22.881 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2020-11-28T21:42:22.882 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:22.917 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:22 +0000 (0:00:00.044) 0:01:29.062 ***** 2020-11-28T21:42:22.923 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2020-11-28T21:42:22.924 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:22.963 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:22 +0000 (0:00:00.045) 0:01:29.108 ***** 2020-11-28T21:42:22.991 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2020-11-28T21:42:22.994 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:23.027 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:23 +0000 (0:00:00.064) 0:01:29.172 ***** 2020-11-28T21:42:23.045 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ ok: [smithi118.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-28T21:42:23.093 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:23 +0000 (0:00:00.065) 0:01:29.237 ***** 2020-11-28T21:42:23.100 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2020-11-28T21:42:23.102 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:23.150 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:23 +0000 (0:00:00.057) 0:01:29.295 ***** 2020-11-28T21:42:23.157 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-11-28T21:42:23.158 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'block_used': 13643527, 'uuid': '099240f7-127e-4e3e-8e24-c4358fbf763e', 'size_total': 983407812608, 'block_total': 240089798, 'mount': '/', 'block_available': 226446271, 'size_available': 927523926016, 'fstype': 'ext4', 'inode_total': 61046784, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 113957, 'block_size': 4096, 'inode_available': 60932827}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226446271, "block_size": 4096, "block_total": 240089798, "block_used": 13643527, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60932827, "inode_total": 61046784, "inode_used": 113957, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927523926016, "size_total": 983407812608, "uuid": "099240f7-127e-4e3e-8e24-c4358fbf763e"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:23.196 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:23 +0000 (0:00:00.046) 0:01:29.341 ***** 2020-11-28T21:42:23.235 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:23.275 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:23 +0000 (0:00:00.078) 0:01:29.419 ***** 2020-11-28T21:42:23.303 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:23.340 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:23 +0000 (0:00:00.065) 0:01:29.485 ***** 2020-11-28T21:42:23.358 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-28T21:42:23.360 INFO:teuthology.task.ansible.out:changed: [smithi162.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-rjW7d4X5PJFxmKb6ntPdx0xRaZEdvy6b3jejylTYClpmLu5dlZBRXkLW887dnjFJ'], 'uuids': ['f9aba611-d92d-427f-97ce-2ff0a13898d1']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.108278", "end": "2020-11-28 21:42:23.717687", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-rjW7d4X5PJFxmKb6ntPdx0xRaZEdvy6b3jejylTYClpmLu5dlZBRXkLW887dnjFJ"], "labels": [], "masters": [], "uuids": ["f9aba611-d92d-427f-97ce-2ff0a13898d1"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:22.609409", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-28T21:42:23.370 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-28T21:42:23.371 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:23.404 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:23 +0000 (0:00:00.064) 0:01:29.549 ***** 2020-11-28T21:42:23.437 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:23 +0000 (0:00:00.032) 0:01:29.582 ***** 2020-11-28T21:42:23.444 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2020-11-28T21:42:23.446 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:23.482 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:23 +0000 (0:00:00.044) 0:01:29.626 ***** 2020-11-28T21:42:23.520 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2020-11-28T21:42:23.522 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:23.559 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:23 +0000 (0:00:00.077) 0:01:29.704 ***** 2020-11-28T21:42:23.599 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-28T21:42:23.600 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:23.696 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2020-11-28T21:42:23.698 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-11-28T21:42:23.838 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:23 +0000 (0:00:00.278) 0:01:29.982 ***** 2020-11-28T21:42:23.844 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2020-11-28T21:42:23.846 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:23.883 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:23 +0000 (0:00:00.045) 0:01:30.027 ***** 2020-11-28T21:42:23.889 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-28T21:42:23.890 INFO:teuthology.task.ansible.out:changed: [smithi162.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-rjW7d4X5PJFxmKb6ntPdx0xRaZEdvy6bh6b3VPYP9IoRhiKa0dEEnkAjUDINN65g'], 'uuids': ['873abb87-1cbf-490b-a4ac-a921021ba5e8']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.153318", "end": "2020-11-28 21:42:24.237495", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-rjW7d4X5PJFxmKb6ntPdx0xRaZEdvy6bh6b3VPYP9IoRhiKa0dEEnkAjUDINN65g"], "labels": [], "masters": [], "uuids": ["873abb87-1cbf-490b-a4ac-a921021ba5e8"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:24.084177", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.14844 s, 777 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.14844 s, 777 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:23.902 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:23.903 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:23.979 INFO:teuthology.task.ansible.out: 2020-11-28T21:42:23.980 INFO:teuthology.task.ansible.out:TASK [testnode : Zap all non-root disks] *************************************** changed: [smithi192.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-XqS7OPtzpkHiN1h6pKSHcGqXVlUIYflV9UtMz1rVxbvF8aZcDu0n3RMtLrg0DTDj'], 'uuids': ['31457c42-bb67-4376-8149-b4bd20e09288']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.097659", "end": "2020-11-28 21:42:24.247564", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-XqS7OPtzpkHiN1h6pKSHcGqXVlUIYflV9UtMz1rVxbvF8aZcDu0n3RMtLrg0DTDj"], "labels": [], "masters": [], "uuids": ["31457c42-bb67-4376-8149-b4bd20e09288"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:23.149905", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk usi 2020-11-28T21:42:23.982 INFO:teuthology.task.ansible.out:ng fdisk or", "other utilities."]} 2020-11-28T21:42:24.030 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:24 +0000 (0:00:00.146) 0:01:30.174 ***** 2020-11-28T21:42:24.068 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-28T21:42:24.070 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:24.166 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2020-11-28T21:42:24.167 INFO:teuthology.task.ansible.out:changed: [smithi118.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-11-28T21:42:24.204 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:24 +0000 (0:00:00.174) 0:01:30.349 ***** 2020-11-28T21:42:24.211 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-28T21:42:24.212 INFO:teuthology.task.ansible.out:changed: [smithi162.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-rjW7d4X5PJFxmKb6ntPdx0xRaZEdvy6bGn2iWj7HSLxjIughXIXL8H7qQuvGY8nK'], 'uuids': ['08f042f6-6507-4c3f-a299-a6dc512047ac']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.156223", "end": "2020-11-28 21:42:24.540549", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-rjW7d4X5PJFxmKb6ntPdx0xRaZEdvy6bGn2iWj7HSLxjIughXIXL8H7qQuvGY8nK"], "labels": [], "masters": [], "uuids": ["08f042f6-6507-4c3f-a299-a6dc512047ac"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:24.384326", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.153346 s, 752 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.153346 s, 752 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:24.214 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XTKR', 'wwn-0x5000c5009293e333'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '099240f7-127e-4e3e-8e24-c4358fbf763e', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XTKR-part1', 'wwn-0x5000c5009293e333-part1'], 'uuids': ['099240f7-127e-4e3e-8e24-c4358fbf763e']}, 'sectors': '1953476096', 'start': '2048', 'holders': [], 'size': '931.49 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c5009293e333', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5XTKR', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XTKR", "wwn-0x5000c5009293e333"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XTKR-part1", "wwn-0x5000c5009293e333-part1"], "labels": [], "masters": [], "uuids": ["099240f7-127e-4e3e-8e24-c4358fbf763e"]}, "sectors": "1953476096", "sectorsize": 512, "size": "931.49 GB", "start": "2048", "uuid": "099240f7-127e-4e3e-8e24-c4358fbf763e"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XTKR", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c5009293e333"}}, "skip_reason": 2020-11-28T21:42:24.215 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2020-11-28T21:42:24.246 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-11-28T21:42:24.249 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-28T21:42:24.321 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-28T21:42:24.323 INFO:teuthology.task.ansible.out:changed: [smithi192.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-XqS7OPtzpkHiN1h6pKSHcGqXVlUIYflV2Y8SuIPl6f7Mo2vHDpgOWM7j6iAe0K7u'], 'uuids': ['c6b7ffc0-933f-426a-92e1-eafee4aa79ca']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.139200", "end": "2020-11-28 21:42:24.596046", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-XqS7OPtzpkHiN1h6pKSHcGqXVlUIYflV2Y8SuIPl6f7Mo2vHDpgOWM7j6iAe0K7u"], "labels": [], "masters": [], "uuids": ["c6b7ffc0-933f-426a-92e1-eafee4aa79ca"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:24.456846", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.135702 s, 850 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.135702 s, 850 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:24.343 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:24.345 INFO:teuthology.task.ansible.out:changed: [smithi162.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-1Dbw0i-EBH1-OnjY-lDS7-NpBI-uU9r-G7BEfq', 'nvme-INTEL_SSDPEDMD400G4_CVFT623300JU400BGN', 'nvme-nvme.8086-435646543632333330304a5534303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.046093", "end": "2020-11-28 21:42:24.701713", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-1Dbw0i-EBH1-OnjY-lDS7-NpBI-uU9r-G7BEfq", "nvme-INTEL_SSDPEDMD400G4_CVFT623300JU400BGN", "nvme-nvme.8086-435646543632333330304a5534303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:24.655620", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0430872 s, 2.7 GB/s", "std 2020-11-28T21:42:24.346 INFO:teuthology.task.ansible.out:err_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0430872 s, 2.7 GB/s"], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:24.356 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:24.357 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:24.370 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:24.467 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-28T21:42:24.510 INFO:teuthology.task.ansible.out:changed: [smithi118.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-28T21:42:24.616 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-28T21:42:24.618 INFO:teuthology.task.ansible.out:changed: [smithi192.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-XqS7OPtzpkHiN1h6pKSHcGqXVlUIYflV1hzTictJljwPxSw5Nnn96p1MxHej4a3m'], 'uuids': ['3b8ceb95-9471-40a5-8e1f-30291f991790']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.144137", "end": "2020-11-28 21:42:24.884652", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-XqS7OPtzpkHiN1h6pKSHcGqXVlUIYflV1hzTictJljwPxSw5Nnn96p1MxHej4a3m"], "labels": [], "masters": [], "uuids": ["3b8ceb95-9471-40a5-8e1f-30291f991790"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:24.740515", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.141754 s, 814 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.141754 s, 814 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:24.628 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XLLK', 'wwn-0x5000c50092939e0b'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '099240f7-127e-4e3e-8e24-c4358fbf763e', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XLLK-part1', 'wwn-0x5000c50092939e0b-part1'], 'uuids': ['099240f7-127e-4e3e-8e24-c4358fbf763e']}, 'sectors': '1953476096', 'start': '2048', 'holders': [], 'size': '931.49 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c50092939e0b', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5XLLK', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XLLK", "wwn-0x5000c50092939e0b"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XLLK-part1", "wwn-0x5000c50092939e0b-part1"], "labels": [], "masters": [], "uuids": ["099240f7-127e-4e3e-8e24-c4358fbf763e"]}, "sectors": "1953476096", "sectorsize": 512, "size": "931.49 GB", "start": "2048", "uuid": "099240f7-127e-4e3e-8e24-c4358fbf763e"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XLLK", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50092939e0b"}}, "skip_reason": 2020-11-28T21:42:24.631 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2020-11-28T21:42:24.684 INFO:teuthology.task.ansible.out:changed: [smithi162.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-rjW7d4X5PJFxmKb6ntPdx0xRaZEdvy6bLshnlWWjy8kG6R9dxsKm07Ygu2dabmZy'], 'uuids': ['f70bdb3e-c65f-4d14-902f-3ceac1a56e18']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.165520", "end": "2020-11-28 21:42:25.035783", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-rjW7d4X5PJFxmKb6ntPdx0xRaZEdvy6bLshnlWWjy8kG6R9dxsKm07Ygu2dabmZy"], "labels": [], "masters": [], "uuids": ["f70bdb3e-c65f-4d14-902f-3ceac1a56e18"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:24.870263", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.161141 s, 716 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.161141 s, 716 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:24.696 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-11-28T21:42:24.698 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** skipping: [smithi162.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:24.709 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:24.732 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-11-28T21:42:24.733 INFO:teuthology.task.ansible.out:changed: [smithi118.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-28T21:42:24.807 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-28T21:42:24.809 INFO:teuthology.task.ansible.out:changed: [smithi192.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-30lnTh-xnbu-AoH9-tHHr-1VSp-TtqW-ai2SU9', 'nvme-INTEL_SSDPEDMD400G4_PHFT620500NN400BGN', 'nvme-nvme.8086-504846543632303530304e4e34303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.048465", "end": "2020-11-28 21:42:25.079768", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-30lnTh-xnbu-AoH9-tHHr-1VSp-TtqW-ai2SU9", "nvme-INTEL_SSDPEDMD400G4_PHFT620500NN400BGN", "nvme-nvme.8086-504846543632303530304e4e34303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:25.031303", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0438145 s, 2.6 GB/s", "std 2020-11-28T21:42:24.811 INFO:teuthology.task.ansible.out:err_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0438145 s, 2.6 GB/s"], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:24.819 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:24.830 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:24.905 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-11-28T21:42:24.906 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-28T21:42:24.950 INFO:teuthology.task.ansible.out:changed: [smithi118.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-28T21:42:24.995 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-28T21:42:24.996 INFO:teuthology.task.ansible.out:changed: [smithi162.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-rjW7d4X5PJFxmKb6ntPdx0xRaZEdvy6by93u1bhZOf3G73tYhnaLIEBEeCegDT68'], 'uuids': ['d4c04505-2f09-43b3-a97e-084e4bd1211d']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.152838", "end": "2020-11-28 21:42:25.348021", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-rjW7d4X5PJFxmKb6ntPdx0xRaZEdvy6by93u1bhZOf3G73tYhnaLIEBEeCegDT68"], "labels": [], "masters": [], "uuids": ["d4c04505-2f09-43b3-a97e-084e4bd1211d"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:25.195183", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.148598 s, 776 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.148598 s, 776 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:25.010 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:25.097 INFO:teuthology.task.ansible.out:changed: [smithi192.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-XqS7OPtzpkHiN1h6pKSHcGqXVlUIYflVGOCsmOGYw4wyRJHqTEt0DQloUvMUP6de'], 'uuids': ['f2763df2-c9d7-4dc5-ab4a-4b1f0b9a8567']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.153496", "end": "2020-11-28 21:42:25.369855", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-XqS7OPtzpkHiN1h6pKSHcGqXVlUIYflVGOCsmOGYw4wyRJHqTEt0DQloUvMUP6de"], "labels": [], "masters": [], "uuids": ["f2763df2-c9d7-4dc5-ab4a-4b1f0b9a8567"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:25.216359", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.151144 s, 763 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.151144 s, 763 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:25.109 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:25.120 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:25.121 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-11-28T21:42:25.121 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-28T21:42:25.166 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:25 +0000 (0:00:00.961) 0:01:31.311 ***** 2020-11-28T21:42:25.179 INFO:teuthology.task.ansible.out:changed: [smithi118.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-28T21:42:25.283 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-28T21:42:25.285 INFO:teuthology.task.ansible.out:changed: [smithi162.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-rjW7d4X5PJFxmKb6ntPdx0xRaZEdvy6b3jejylTYClpmLu5dlZBRXkLW887dnjFJ'], 'uuids': ['f9aba611-d92d-427f-97ce-2ff0a13898d1']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.153323", "end": "2020-11-28 21:42:25.641169", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-rjW7d4X5PJFxmKb6ntPdx0xRaZEdvy6b3jejylTYClpmLu5dlZBRXkLW887dnjFJ"], "labels": [], "masters": [], "uuids": ["f9aba611-d92d-427f-97ce-2ff0a13898d1"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:25.487846", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.150493 s, 766 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.150493 s, 766 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:25.306 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-11-28T21:42:25.308 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-11-28T21:42:25.371 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:25 +0000 (0:00:00.204) 0:01:31.515 ***** 2020-11-28T21:42:25.409 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:25 +0000 (0:00:00.038) 0:01:31.553 ***** 2020-11-28T21:42:25.414 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-11-28T21:42:25.416 INFO:teuthology.task.ansible.out:changed: [smithi118.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-28T21:42:25.419 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** changed: [smithi192.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-XqS7OPtzpkHiN1h6pKSHcGqXVlUIYflV2nbdDSGx61uJafMjM4eftPL7dxAD2CDF'], 'uuids': ['c0cbf01a-4ba0-40ca-b4e1-aa422e6b8e36']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.146923", "end": "2020-11-28 21:42:25.659872", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-XqS7OPtzpkHiN1h6pKSHcGqXVlUIYflV2nbdDSGx61uJafMjM4eftPL7dxAD2CDF"], "labels": [], "masters": [], "uuids": ["c0cbf01a-4ba0-40ca-b4e1-aa422e6b8e36"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:25.512949", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.143423 s, 804 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.143423 s, 804 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:25.420 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': 2020-11-28T21:42:25.422 INFO:teuthology.task.ansible.out:'0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:25.458 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:25 +0000 (0:00:00.049) 0:01:31.603 ***** 2020-11-28T21:42:25.528 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-11-28T21:42:25.530 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-28T21:42:25.605 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-11-28T21:42:25.606 INFO:teuthology.task.ansible.out:changed: [smithi118.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-11-28T21:42:25.651 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:25 +0000 (0:00:00.193) 0:01:31.796 ***** 2020-11-28T21:42:25.681 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-11-28T21:42:25.681 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-28T21:42:25.696 INFO:teuthology.task.ansible.out: 2020-11-28T21:42:25.698 INFO:teuthology.task.ansible.out:TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-28T21:42:25.699 INFO:teuthology.task.ansible.out:changed: [smithi192.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-XqS7OPtzpkHiN1h6pKSHcGqXVlUIYflV9UtMz1rVxbvF8aZcDu0n3RMtLrg0DTDj'], 'uuids': ['31457c42-bb67-4376-8149-b4bd20e09288']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.153507", "end": "2020-11-28 21:42:25.957028", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-XqS7OPtzpkHiN1h6pKSHcGqXVlUIYflV9UtMz1rVxbvF8aZcDu0n3RMtLrg0DTDj"], "labels": [], "masters": [], "uuids": ["31457c42-bb67-4376-8149-b4bd20e09288"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-28 21:42:25.803521", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.148864 s, 775 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.148864 s, 775 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:25.727 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2020-11-28T21:42:25.728 INFO:teuthology.task.ansible.out:changed: [smithi162.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.149857", "end": "2020-11-28 21:42:26.057185", "rc": 0, "start": "2020-11-28 21:42:25.907328", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:25.772 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:25 +0000 (0:00:00.120) 0:01:31.916 ***** 2020-11-28T21:42:25.813 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:25 +0000 (0:00:00.041) 0:01:31.958 ***** 2020-11-28T21:42:25.822 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-11-28T21:42:25.824 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-28T21:42:25.837 INFO:teuthology.task.ansible.out:changed: [smithi118.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-28T21:42:25.938 INFO:teuthology.task.ansible.out:changed: [smithi118.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-28T21:42:25.964 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-28T21:42:25.966 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2020-11-28T21:42:25.966 INFO:teuthology.task.ansible.out:changed: [smithi162.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.052722", "end": "2020-11-28 21:42:26.321112", "rc": 0, "start": "2020-11-28 21:42:26.268390", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:26.022 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:26 +0000 (0:00:00.209) 0:01:32.167 ***** 2020-11-28T21:42:26.030 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-11-28T21:42:26.032 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:26.098 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:26 +0000 (0:00:00.075) 0:01:32.243 ***** 2020-11-28T21:42:26.109 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-11-28T21:42:26.111 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** changed: [smithi118.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-28T21:42:26.179 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:26 +0000 (0:00:00.080) 0:01:32.323 ***** 2020-11-28T21:42:26.186 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2020-11-28T21:42:26.188 INFO:teuthology.task.ansible.out:changed: [smithi192.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.149667", "end": "2020-11-28 21:42:26.376357", "rc": 0, "start": "2020-11-28 21:42:26.226690", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:26.241 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:26 +0000 (0:00:00.059) 0:01:32.383 ***** 2020-11-28T21:42:26.243 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2020-11-28T21:42:26.246 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Write /scratch_devs] ****************************************** changed: [smithi118.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-28T21:42:26.246 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:26.307 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:26 +0000 (0:00:00.069) 0:01:32.452 ***** 2020-11-28T21:42:26.325 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2020-11-28T21:42:26.326 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2020-11-28T21:42:26.367 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:26 +0000 (0:00:00.060) 0:01:32.512 ***** 2020-11-28T21:42:26.419 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:26 +0000 (0:00:00.051) 0:01:32.564 ***** 2020-11-28T21:42:26.428 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:26.434 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** ok: [smithi096.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2020-11-28T21:42:26.473 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:26 +0000 (0:00:00.054) 0:01:32.618 ***** 2020-11-28T21:42:26.509 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:26 +0000 (0:00:00.035) 0:01:32.654 ***** 2020-11-28T21:42:26.517 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2020-11-28T21:42:26.519 INFO:teuthology.task.ansible.out:changed: [smithi192.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.049800", "end": "2020-11-28 21:42:26.711590", "rc": 0, "start": "2020-11-28 21:42:26.661790", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:26.554 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:26 +0000 (0:00:00.045) 0:01:32.699 ***** 2020-11-28T21:42:26.560 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-11-28T21:42:26.562 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:26.598 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:26 +0000 (0:00:00.043) 0:01:32.743 ***** 2020-11-28T21:42:26.604 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-11-28T21:42:26.604 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2020-11-28T21:42:26.605 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2020-11-28T21:42:26.640 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:26 +0000 (0:00:00.042) 0:01:32.785 ***** 2020-11-28T21:42:26.673 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:26 +0000 (0:00:00.032) 0:01:32.818 ***** 2020-11-28T21:42:26.679 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2020-11-28T21:42:26.680 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set xfs_opts on newer OSes] *********************************** ok: [smithi118.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2020-11-28T21:42:26.743 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:26 +0000 (0:00:00.069) 0:01:32.887 ***** 2020-11-28T21:42:26.782 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:26 +0000 (0:00:00.038) 0:01:32.926 ***** 2020-11-28T21:42:26.790 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2020-11-28T21:42:26.791 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:26.839 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:26 +0000 (0:00:00.057) 0:01:32.984 ***** 2020-11-28T21:42:26.866 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2020-11-28T21:42:26.868 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"changed": true} TASK [testnode : Write partition entries to {{ item.0.device }}] *************** skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:26.906 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:26 +0000 (0:00:00.066) 0:01:33.051 ***** 2020-11-28T21:42:26.942 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:26 +0000 (0:00:00.036) 0:01:33.087 ***** 2020-11-28T21:42:26.949 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2020-11-28T21:42:26.951 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-28T21:42:26.999 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:26 +0000 (0:00:00.056) 0:01:33.144 ***** 2020-11-28T21:42:27.009 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-11-28T21:42:27.011 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:27.088 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:27 +0000 (0:00:00.088) 0:01:33.232 ***** 2020-11-28T21:42:27.096 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-11-28T21:42:27.098 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'block_used': 13643536, 'uuid': '099240f7-127e-4e3e-8e24-c4358fbf763e', 'size_total': 983407812608, 'block_total': 240089798, 'mount': '/', 'block_available': 226446262, 'size_available': 927523889152, 'fstype': 'ext4', 'inode_total': 61046784, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 113957, 'block_size': 4096, 'inode_available': 60932827}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226446262, "block_size": 4096, "block_total": 240089798, "block_used": 13643536, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60932827, "inode_total": 61046784, "inode_used": 113957, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927523889152, "size_total": 983407812608, "uuid": "099240f7-127e-4e3e-8e24-c4358fbf763e"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:27.137 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:27 +0000 (0:00:00.049) 0:01:33.282 ***** 2020-11-28T21:42:27.144 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2020-11-28T21:42:27.146 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:27.186 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:27 +0000 (0:00:00.049) 0:01:33.331 ***** 2020-11-28T21:42:27.196 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2020-11-28T21:42:27.198 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:27.263 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:27 +0000 (0:00:00.076) 0:01:33.407 ***** 2020-11-28T21:42:27.273 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2020-11-28T21:42:27.274 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-11-28T21:42:27.312 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:27 +0000 (0:00:00.049) 0:01:33.457 ***** 2020-11-28T21:42:27.318 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-28T21:42:27.320 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ ok: [smithi192.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-28T21:42:27.381 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:27 +0000 (0:00:00.068) 0:01:33.526 ***** 2020-11-28T21:42:27.418 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:27 +0000 (0:00:00.036) 0:01:33.562 ***** 2020-11-28T21:42:27.423 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-28T21:42:27.425 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:27.465 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:27 +0000 (0:00:00.047) 0:01:33.610 ***** 2020-11-28T21:42:27.482 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2020-11-28T21:42:27.484 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "gid": 1235, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2020-11-28T21:42:27.484 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ skipping: [smithi192.front.sepia.ceph.com] => (item={'block_used': 13643537, 'uuid': '099240f7-127e-4e3e-8e24-c4358fbf763e', 'size_total': 983407812608, 'block_total': 240089798, 'mount': '/', 'block_available': 226446261, 'size_available': 927523885056, 'fstype': 'ext4', 'inode_total': 61046784, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 113957, 'block_size': 4096, 'inode_available': 60932827}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226446261, "block_size": 4096, "block_total": 240089798, "block_used": 13643537, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60932827, "inode_total": 61046784, "inode_used": 113957, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927523885056, "size_total": 983407812608, "uuid": "099240f7-127e-4e3e-8e24-c4358fbf763e"}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:27.522 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:27 +0000 (0:00:00.057) 0:01:33.667 ***** 2020-11-28T21:42:27.565 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:27 +0000 (0:00:00.042) 0:01:33.709 ***** 2020-11-28T21:42:27.572 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2020-11-28T21:42:27.574 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:27.617 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:27 +0000 (0:00:00.052) 0:01:33.762 ***** 2020-11-28T21:42:27.623 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2020-11-28T21:42:27.625 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:27.661 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:27 +0000 (0:00:00.043) 0:01:33.805 ***** 2020-11-28T21:42:27.667 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2020-11-28T21:42:27.668 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add CPAN config directory for the root user.] ***************** 2020-11-28T21:42:27.669 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2020-11-28T21:42:27.705 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:27 +0000 (0:00:00.044) 0:01:33.850 ***** 2020-11-28T21:42:27.739 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:27 +0000 (0:00:00.033) 0:01:33.884 ***** 2020-11-28T21:42:27.747 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-28T21:42:27.749 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:27.789 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:27 +0000 (0:00:00.049) 0:01:33.934 ***** 2020-11-28T21:42:27.794 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2020-11-28T21:42:27.802 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:27.839 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:27 +0000 (0:00:00.050) 0:01:33.984 ***** 2020-11-28T21:42:27.846 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-28T21:42:27.848 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:27.885 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:27 +0000 (0:00:00.045) 0:01:34.030 ***** 2020-11-28T21:42:27.913 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* changed: [smithi118.front.sepia.ceph.com] => {"changed": true} 2020-11-28T21:42:27.952 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:27 +0000 (0:00:00.067) 0:01:34.097 ***** 2020-11-28T21:42:27.959 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2020-11-28T21:42:27.961 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:28.000 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:27 +0000 (0:00:00.047) 0:01:34.145 ***** 2020-11-28T21:42:28.028 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2020-11-28T21:42:28.029 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1235, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} TASK [testnode : Determine desired logical volume percentage size] ************* skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:28.065 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:28 +0000 (0:00:00.064) 0:01:34.210 ***** 2020-11-28T21:42:28.100 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:28 +0000 (0:00:00.034) 0:01:34.244 ***** 2020-11-28T21:42:28.127 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2020-11-28T21:42:28.129 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:28.166 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:28 +0000 (0:00:00.066) 0:01:34.311 ***** 2020-11-28T21:42:28.172 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2020-11-28T21:42:28.174 INFO:teuthology.task.ansible.out:changed: [smithi118.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-11-28T21:42:28.233 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:28 +0000 (0:00:00.067) 0:01:34.378 ***** 2020-11-28T21:42:28.242 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2020-11-28T21:42:28.244 INFO:teuthology.task.ansible.out:changed: [smithi162.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-11-28T21:42:28.284 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:28 +0000 (0:00:00.049) 0:01:34.427 ***** 2020-11-28T21:42:28.401 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2020-11-28T21:42:28.403 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2020-11-28T21:42:28.445 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:28 +0000 (0:00:00.162) 0:01:34.590 ***** 2020-11-28T21:42:28.457 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2020-11-28T21:42:28.460 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"changed": false, "gid": 1235, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2020-11-28T21:42:28.501 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:28 +0000 (0:00:00.056) 0:01:34.646 ***** 2020-11-28T21:42:28.522 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2020-11-28T21:42:28.524 INFO:teuthology.task.ansible.out:changed: [smithi192.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-11-28T21:42:28.574 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:28 +0000 (0:00:00.072) 0:01:34.719 ***** 2020-11-28T21:42:28.581 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-11-28T21:42:28.583 INFO:teuthology.task.ansible.out:changed: [smithi162.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-28T21:42:28.659 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2020-11-28T21:42:28.661 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2020-11-28T21:42:28.698 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:28 +0000 (0:00:00.123) 0:01:34.843 ***** 2020-11-28T21:42:28.810 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-11-28T21:42:28.811 INFO:teuthology.task.ansible.out:changed: [smithi162.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-28T21:42:28.853 INFO:teuthology.task.ansible.out:changed: [smithi192.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-28T21:42:29.006 INFO:teuthology.task.ansible.out:changed: [smithi162.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-28T21:42:29.028 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2020-11-28T21:42:29.030 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1235, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2020-11-28T21:42:29.073 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:29 +0000 (0:00:00.375) 0:01:35.218 ***** 2020-11-28T21:42:29.102 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-11-28T21:42:29.104 INFO:teuthology.task.ansible.out:changed: [smithi192.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-28T21:42:29.198 INFO:teuthology.task.ansible.out:changed: [smithi162.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-28T21:42:29.325 INFO:teuthology.task.ansible.out:changed: [smithi192.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-28T21:42:29.347 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2020-11-28T21:42:29.349 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"cache_update_time": 1606599646, "cache_updated": false, "changed": false} 2020-11-28T21:42:29.384 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:29 +0000 (0:00:00.311) 0:01:35.529 ***** 2020-11-28T21:42:29.412 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2020-11-28T21:42:29.414 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2020-11-28T21:42:29.466 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:29 +0000 (0:00:00.081) 0:01:35.611 ***** 2020-11-28T21:42:29.476 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-11-28T21:42:29.478 INFO:teuthology.task.ansible.out:changed: [smithi162.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-28T21:42:29.517 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:29 +0000 (0:00:00.051) 0:01:35.662 ***** 2020-11-28T21:42:29.557 INFO:teuthology.task.ansible.out:changed: [smithi192.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-28T21:42:29.569 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2020-11-28T21:42:29.570 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.045162", "end": "2020-11-28 21:42:29.919415", "rc": 0, "start": "2020-11-28 21:42:29.874253", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.26.1/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.26.1/Amazon/S3.pm"]} 2020-11-28T21:42:29.606 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:29 +0000 (0:00:00.088) 0:01:35.751 ***** 2020-11-28T21:42:29.633 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:29.668 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:29 +0000 (0:00:00.061) 0:01:35.812 ***** 2020-11-28T21:42:29.674 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-11-28T21:42:29.676 INFO:teuthology.task.ansible.out:changed: [smithi162.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-11-28T21:42:29.712 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:29 +0000 (0:00:00.044) 0:01:35.857 ***** 2020-11-28T21:42:29.718 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2020-11-28T21:42:29.720 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:29.755 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:29 +0000 (0:00:00.042) 0:01:35.899 ***** 2020-11-28T21:42:29.803 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-11-28T21:42:29.805 INFO:teuthology.task.ansible.out:changed: [smithi192.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-28T21:42:29.844 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:29 +0000 (0:00:00.089) 0:01:35.989 ***** 2020-11-28T21:42:29.875 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-11-28T21:42:29.876 INFO:teuthology.task.ansible.out:changed: [smithi162.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-28T21:42:29.975 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** changed: [smithi192.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-11-28T21:42:30.016 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:30 +0000 (0:00:00.171) 0:01:36.160 ***** 2020-11-28T21:42:30.022 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-11-28T21:42:30.024 INFO:teuthology.task.ansible.out:changed: [smithi162.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-28T21:42:30.131 INFO:teuthology.task.ansible.out:changed: [smithi162.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-28T21:42:30.155 INFO:teuthology.task.ansible.out:changed: [smithi192.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-28T21:42:30.243 INFO:teuthology.task.ansible.out:changed: [smithi162.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-28T21:42:30.265 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:30.310 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:30 +0000 (0:00:00.294) 0:01:36.455 ***** 2020-11-28T21:42:30.318 INFO:teuthology.task.ansible.out:changed: [smithi192.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-28T21:42:30.343 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** ok: [smithi118.front.sepia.ceph.com] => {"cache_update_time": 1606599636, "cache_updated": false, "changed": false} 2020-11-28T21:42:30.380 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:30 +0000 (0:00:00.069) 0:01:36.524 ***** 2020-11-28T21:42:30.409 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-11-28T21:42:30.411 INFO:teuthology.task.ansible.out:changed: [smithi192.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-28T21:42:30.455 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** ok: [smithi162.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2020-11-28T21:42:30.495 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:30 +0000 (0:00:00.115) 0:01:36.640 ***** 2020-11-28T21:42:30.537 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2020-11-28T21:42:30.538 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} TASK [testnode : Write /scratch_devs] ****************************************** 2020-11-28T21:42:30.539 INFO:teuthology.task.ansible.out:changed: [smithi192.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-28T21:42:30.541 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2020-11-28T21:42:30.542 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.045697", "end": "2020-11-28 21:42:30.577968", "rc": 0, "start": "2020-11-28 21:42:30.532271", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.26.1/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.26.1/Amazon/S3.pm"]} 2020-11-28T21:42:30.582 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:30 +0000 (0:00:00.087) 0:01:36.727 ***** 2020-11-28T21:42:30.617 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:30 +0000 (0:00:00.034) 0:01:36.762 ***** 2020-11-28T21:42:30.624 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-11-28T21:42:30.625 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:30.627 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2020-11-28T21:42:30.629 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:30.666 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:30 +0000 (0:00:00.048) 0:01:36.810 ***** 2020-11-28T21:42:30.699 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:30 +0000 (0:00:00.033) 0:01:36.844 ***** 2020-11-28T21:42:30.705 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2020-11-28T21:42:30.707 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"cache_update_time": 1606599646, "cache_updated": false, "changed": false} TASK [testnode : Install ntp package on rpm based systems.] ******************** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:30.743 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:30 +0000 (0:00:00.043) 0:01:36.888 ***** 2020-11-28T21:42:30.777 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:30 +0000 (0:00:00.033) 0:01:36.921 ***** 2020-11-28T21:42:30.782 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2020-11-28T21:42:30.784 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:30.818 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:30 +0000 (0:00:00.041) 0:01:36.963 ***** 2020-11-28T21:42:30.835 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2020-11-28T21:42:30.837 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2020-11-28T21:42:30.873 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:30 +0000 (0:00:00.054) 0:01:37.018 ***** 2020-11-28T21:42:30.910 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2020-11-28T21:42:30.912 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2020-11-28T21:42:30.944 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:30 +0000 (0:00:00.070) 0:01:37.089 ***** 2020-11-28T21:42:31.142 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2020-11-28T21:42:31.144 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2020-11-28T21:42:31.187 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:31 +0000 (0:00:00.242) 0:01:37.331 ***** 2020-11-28T21:42:31.194 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2020-11-28T21:42:31.196 INFO:teuthology.task.ansible.out:changed: [smithi192.front.sepia.ceph.com] => {"changed": true} 2020-11-28T21:42:31.251 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:31 +0000 (0:00:00.064) 0:01:37.395 ***** 2020-11-28T21:42:31.257 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2020-11-28T21:42:31.259 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:31.298 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:31 +0000 (0:00:00.047) 0:01:37.443 ***** 2020-11-28T21:42:31.442 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2020-11-28T21:42:31.444 INFO:teuthology.task.ansible.out:changed: [smithi192.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-11-28T21:42:31.483 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:31 +0000 (0:00:00.184) 0:01:37.628 ***** 2020-11-28T21:42:31.573 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2020-11-28T21:42:31.575 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"cache_update_time": 1606599636, "cache_updated": false, "changed": false} 2020-11-28T21:42:31.610 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:31 +0000 (0:00:00.127) 0:01:37.755 ***** 2020-11-28T21:42:31.628 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2020-11-28T21:42:31.629 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"changed": false, "gid": 1235, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2020-11-28T21:42:31.664 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:31 +0000 (0:00:00.054) 0:01:37.809 ***** 2020-11-28T21:42:31.670 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2020-11-28T21:42:31.671 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:31.708 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:31 +0000 (0:00:00.043) 0:01:37.853 ***** 2020-11-28T21:42:31.808 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2020-11-28T21:42:31.810 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2020-11-28T21:42:31.847 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:31 +0000 (0:00:00.139) 0:01:37.992 ***** 2020-11-28T21:42:31.864 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2020-11-28T21:42:31.866 INFO:teuthology.task.ansible.out:changed: [smithi162.front.sepia.ceph.com] => {"changed": true} 2020-11-28T21:42:31.919 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:31 +0000 (0:00:00.072) 0:01:38.064 ***** 2020-11-28T21:42:32.001 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2020-11-28T21:42:32.003 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2020-11-28T21:42:32.042 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:32 +0000 (0:00:00.123) 0:01:38.187 ***** 2020-11-28T21:42:32.069 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2020-11-28T21:42:32.071 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:32.107 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:32 +0000 (0:00:00.064) 0:01:38.252 ***** 2020-11-28T21:42:32.124 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* changed: [smithi162.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-11-28T21:42:32.164 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:32 +0000 (0:00:00.057) 0:01:38.309 ***** 2020-11-28T21:42:32.170 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2020-11-28T21:42:32.172 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1235, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2020-11-28T21:42:32.210 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:32 +0000 (0:00:00.045) 0:01:38.355 ***** 2020-11-28T21:42:32.312 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2020-11-28T21:42:32.313 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => {"changed": false, "gid": 1235, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2020-11-28T21:42:32.360 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:32 +0000 (0:00:00.149) 0:01:38.504 ***** 2020-11-28T21:42:32.366 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2020-11-28T21:42:32.368 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2020-11-28 21:40:01 UTC", "ActiveEnterTimestampMonotonic": "24008794", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "basic.target network.target sysinit.target systemd-tmpfiles-setup.service -.mount system.slice systemd-journald.socket", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2020-11-28 21:39:55 UTC", "AssertTimestampMonotonic": "17528037", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2020-11-28 21:39:55 UTC", "ConditionTimestampMonotonic": "17528036", "ConfigurationDirectoryMode": "0755", "Conflicts": "systemd-timesyncd.service shutdown.target", "ControlGroup": "/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documen 2020-11-28T21:42:32.369 INFO:teuthology.task.ansible.out:tation": "man:ntpd(8)", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1991", "ExecMainStartTimestamp": "Sat 2020-11-28 21:40:01 UTC", "ExecMainStartTimestampMonotonic": "24008782", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2020-11-28 21:39:55 UTC", "InactiveExitTimestampMonotonic": "17528902", "InvocationID": "d9b6550740954c8b91fe5606509063d3", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "Limi 2020-11-28T21:42:32.371 INFO:teuthology.task.ansible.out:tRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1991", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target system.slice -.mount", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2020-11-28 21:40:01 UTC", "S 2020-11-28T21:42:32.373 INFO:teuthology.task.ansible.out:tateChangeTimestampMonotonic": "24008794", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2020-11-28 21:40:01 UTC", "WatchdogTimestampMonotonic": "24008792", "WatchdogUSec": "0"}} 2020-11-28T21:42:32.407 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:32 +0000 (0:00:00.047) 0:01:38.552 ***** 2020-11-28T21:42:32.434 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-28T21:42:32.435 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2020-11-28T21:42:32.475 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:32 +0000 (0:00:00.068) 0:01:38.620 ***** 2020-11-28T21:42:32.493 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2020-11-28T21:42:32.495 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2020-11-28T21:42:32.551 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:32 +0000 (0:00:00.074) 0:01:38.695 ***** 2020-11-28T21:42:32.560 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-28T21:42:32.562 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Make sure ntpd is running.] *********************************** 2020-11-28T21:42:32.563 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2020-11-28 21:39:27 UTC", "ActiveEnterTimestampMonotonic": "24373551", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "basic.target system.slice systemd-journald.socket sysinit.target network.target -.mount systemd-tmpfiles-setup.service", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2020-11-28 21:39:20 UTC", "AssertTimestampMonotonic": "17906190", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2020-11-28 21:39:20 UTC", "ConditionTimestampMonotonic": "17906190", "ConfigurationDirectoryMode": "0755", "Conflicts": "systemd-timesyncd.service shutdown.target", "ControlGroup": "/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documen 2020-11-28T21:42:32.565 INFO:teuthology.task.ansible.out:tation": "man:ntpd(8)", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1958", "ExecMainStartTimestamp": "Sat 2020-11-28 21:39:27 UTC", "ExecMainStartTimestampMonotonic": "24373522", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2020-11-28 21:39:20 UTC", "InactiveExitTimestampMonotonic": "17907264", "InvocationID": "0f0d85bfe60e4d8091996e48fbc11836", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "Limi 2020-11-28T21:42:32.566 INFO:teuthology.task.ansible.out:tRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1958", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target system.slice -.mount", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2020-11-28 21:39:27 UTC", "S 2020-11-28T21:42:32.568 INFO:teuthology.task.ansible.out:tateChangeTimestampMonotonic": "24373551", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2020-11-28 21:39:27 UTC", "WatchdogTimestampMonotonic": "24373548", "WatchdogUSec": "0"}} TASK [testnode : Upload CPAN config for root.] ********************************* ok: [smithi192.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2020-11-28T21:42:32.603 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:32 +0000 (0:00:00.052) 0:01:38.747 ***** 2020-11-28T21:42:32.637 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:32 +0000 (0:00:00.034) 0:01:38.782 ***** 2020-11-28T21:42:32.670 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:32 +0000 (0:00:00.032) 0:01:38.815 ***** 2020-11-28T21:42:32.677 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-28T21:42:32.679 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} TASK [testnode : Install python-pip on rpm based systems.] ********************* skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:32.715 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:32 +0000 (0:00:00.044) 0:01:38.860 ***** 2020-11-28T21:42:32.751 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:32 +0000 (0:00:00.035) 0:01:38.895 ***** 2020-11-28T21:42:32.764 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:32.824 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:32 +0000 (0:00:00.072) 0:01:38.968 ***** 2020-11-28T21:42:32.842 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2020-11-28T21:42:32.843 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1235, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2020-11-28T21:42:32.878 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:32 +0000 (0:00:00.054) 0:01:39.023 ***** 2020-11-28T21:42:32.885 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2020-11-28T21:42:32.887 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:32.923 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:32 +0000 (0:00:00.044) 0:01:39.068 ***** 2020-11-28T21:42:33.191 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2020-11-28T21:42:33.192 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2020-11-28T21:42:33.233 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:33 +0000 (0:00:00.309) 0:01:39.377 ***** 2020-11-28T21:42:33.569 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2020-11-28T21:42:33.571 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"cache_update_time": 1606599636, "cache_updated": false, "changed": false} 2020-11-28T21:42:33.608 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:33 +0000 (0:00:00.375) 0:01:39.753 ***** 2020-11-28T21:42:33.615 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2020-11-28T21:42:33.617 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"cache_update_time": 1606599641, "cache_updated": false, "changed": false} 2020-11-28T21:42:33.653 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:33 +0000 (0:00:00.044) 0:01:39.798 ***** 2020-11-28T21:42:33.660 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2020-11-28T21:42:33.662 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:33.703 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:33 +0000 (0:00:00.049) 0:01:39.848 ***** 2020-11-28T21:42:33.815 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2020-11-28T21:42:33.816 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"cache_update_time": 1606599646, "cache_updated": false, "changed": false} 2020-11-28T21:42:33.853 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:33 +0000 (0:00:00.149) 0:01:39.998 ***** 2020-11-28T21:42:33.860 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2020-11-28T21:42:33.861 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.045856", "end": "2020-11-28 21:42:34.104812", "rc": 0, "start": "2020-11-28 21:42:34.058956", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.26.1/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.26.1/Amazon/S3.pm"]} TASK [testnode : Create the .pip directory for the teuthology user.] *********** ok: [smithi118.front.sepia.ceph.com] => {"changed": false, "gid": 1235, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2020-11-28T21:42:33.901 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:33 +0000 (0:00:00.048) 0:01:40.046 ***** 2020-11-28T21:42:33.944 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:33 +0000 (0:00:00.042) 0:01:40.089 ***** 2020-11-28T21:42:33.951 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2020-11-28T21:42:33.953 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:34.002 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:33 +0000 (0:00:00.058) 0:01:40.147 ***** 2020-11-28T21:42:34.010 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2020-11-28T21:42:34.011 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:34.051 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:34 +0000 (0:00:00.048) 0:01:40.195 ***** 2020-11-28T21:42:34.079 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2020-11-28T21:42:34.081 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:34.125 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:34 +0000 (0:00:00.074) 0:01:40.270 ***** 2020-11-28T21:42:34.132 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2020-11-28T21:42:34.134 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => {"cache_update_time": 1606599631, "cache_updated": false, "changed": false} TASK [testnode : Create the .pip directory for the teuthology user.] *********** ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "gid": 1235, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2020-11-28T21:42:34.178 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:34 +0000 (0:00:00.052) 0:01:40.323 ***** 2020-11-28T21:42:34.225 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:34 +0000 (0:00:00.047) 0:01:40.370 ***** 2020-11-28T21:42:34.234 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2020-11-28T21:42:34.236 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1235, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2020-11-28T21:42:34.275 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:34 +0000 (0:00:00.049) 0:01:40.419 ***** 2020-11-28T21:42:34.397 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2020-11-28T21:42:34.399 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.044718", "end": "2020-11-28 21:42:34.751929", "rc": 0, "start": "2020-11-28 21:42:34.707211", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.26.1/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.26.1/Amazon/S3.pm"]} 2020-11-28T21:42:34.435 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:34 +0000 (0:00:00.160) 0:01:40.580 ***** 2020-11-28T21:42:34.462 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2020-11-28T21:42:34.464 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:34.506 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:34 +0000 (0:00:00.070) 0:01:40.650 ***** 2020-11-28T21:42:34.513 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2020-11-28T21:42:34.515 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1235, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2020-11-28T21:42:34.552 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:34 +0000 (0:00:00.046) 0:01:40.697 ***** 2020-11-28T21:42:34.558 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2020-11-28T21:42:34.560 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:34.603 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:34 +0000 (0:00:00.050) 0:01:40.747 ***** 2020-11-28T21:42:34.946 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2020-11-28T21:42:34.947 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"cache_update_time": 1606599641, "cache_updated": false, "changed": false} 2020-11-28T21:42:34.982 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:34 +0000 (0:00:00.379) 0:01:41.126 ***** 2020-11-28T21:42:35.009 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2020-11-28T21:42:35.011 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:35.049 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:35 +0000 (0:00:00.066) 0:01:41.193 ***** 2020-11-28T21:42:35.308 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2020-11-28T21:42:35.310 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages\nRequirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages", "Requirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2020-11-28T21:42:35.371 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:35 +0000 (0:00:00.323) 0:01:41.516 ***** 2020-11-28T21:42:35.383 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2020-11-28T21:42:35.385 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2020-11-28T21:42:35.429 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:35 +0000 (0:00:00.057) 0:01:41.574 ***** 2020-11-28T21:42:35.436 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2020-11-28T21:42:35.437 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages\nRequirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages", "Requirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2020-11-28T21:42:35.479 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:35 +0000 (0:00:00.049) 0:01:41.624 ***** 2020-11-28T21:42:35.486 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2020-11-28T21:42:35.488 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:35.534 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:35 +0000 (0:00:00.054) 0:01:41.678 ***** 2020-11-28T21:42:35.542 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2020-11-28T21:42:35.544 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => {"cache_update_time": 1606599631, "cache_updated": false, "changed": false} 2020-11-28T21:42:35.580 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:35 +0000 (0:00:00.046) 0:01:41.725 ***** 2020-11-28T21:42:35.608 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2020-11-28T21:42:35.610 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:35.647 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:35 +0000 (0:00:00.066) 0:01:41.791 ***** 2020-11-28T21:42:35.741 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2020-11-28T21:42:35.743 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2020-11-28 21:39:39 UTC", "ActiveEnterTimestampMonotonic": "24544633", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "systemd-journald.socket network.target basic.target -.mount system.slice systemd-tmpfiles-setup.service sysinit.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2020-11-28 21:39:32 UTC", "AssertTimestampMonotonic": "17972100", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2020-11-28 21:39:32 UTC", "ConditionTimestampMonotonic": "17972100", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target systemd-timesyncd.service", "ControlGroup": "/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documen 2020-11-28T21:42:35.744 INFO:teuthology.task.ansible.out:tation": "man:ntpd(8)", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1963", "ExecMainStartTimestamp": "Sat 2020-11-28 21:39:39 UTC", "ExecMainStartTimestampMonotonic": "24544571", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2020-11-28 21:39:32 UTC", "InactiveExitTimestampMonotonic": "17972815", "InvocationID": "1edfe820dd0d47cba45aff5a57619881", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "Limi 2020-11-28T21:42:35.746 INFO:teuthology.task.ansible.out:tRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1963", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target -.mount system.slice", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2020-11-28 21:39:39 UTC", "S 2020-11-28T21:42:35.748 INFO:teuthology.task.ansible.out:tateChangeTimestampMonotonic": "24544633", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2020-11-28 21:39:39 UTC", "WatchdogTimestampMonotonic": "24544629", "WatchdogUSec": "0"}} 2020-11-28T21:42:35.782 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:35 +0000 (0:00:00.134) 0:01:41.926 ***** 2020-11-28T21:42:35.784 INFO:teuthology.task.ansible.out: 2020-11-28T21:42:35.810 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-28T21:42:35.812 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2020-11-28T21:42:35.847 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:35 +0000 (0:00:00.066) 0:01:41.992 ***** 2020-11-28T21:42:35.875 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:35.912 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:35 +0000 (0:00:00.064) 0:01:42.057 ***** 2020-11-28T21:42:35.919 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* ok: [smithi096.front.sepia.ceph.com] => {"changed": false} 2020-11-28T21:42:35.967 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:35 +0000 (0:00:00.054) 0:01:42.112 ***** 2020-11-28T21:42:35.973 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2020-11-28T21:42:35.975 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create the ntp.conf file.] ************************************ ok: [smithi162.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2020-11-28T21:42:36.012 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:36 +0000 (0:00:00.044) 0:01:42.156 ***** 2020-11-28T21:42:36.045 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:36 +0000 (0:00:00.033) 0:01:42.190 ***** 2020-11-28T21:42:36.051 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ ok: [smithi096.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2020-11-28T21:42:36.053 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:36.089 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:36 +0000 (0:00:00.043) 0:01:42.234 ***** 2020-11-28T21:42:36.123 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:36 +0000 (0:00:00.033) 0:01:42.267 ***** 2020-11-28T21:42:36.129 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2020-11-28T21:42:36.130 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"changed": false} 2020-11-28T21:42:36.168 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:36 +0000 (0:00:00.045) 0:01:42.313 ***** 2020-11-28T21:42:36.198 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2020-11-28T21:42:36.198 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2020-11-28T21:42:36.241 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:36 +0000 (0:00:00.072) 0:01:42.386 ***** 2020-11-28T21:42:36.254 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2020-11-28T21:42:36.256 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1606599581.5051837, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1606599581.5011857, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 6119, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1606599581.5011857, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "1817098686", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2020-11-28T21:42:36.301 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:36 +0000 (0:00:00.059) 0:01:42.445 ***** 2020-11-28T21:42:36.334 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2020-11-28T21:42:36.338 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2020-11-28 21:39:50 UTC", "ActiveEnterTimestampMonotonic": "24372773", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "-.mount systemd-tmpfiles-setup.service basic.target network.target sysinit.target systemd-journald.socket system.slice", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2020-11-28 21:39:43 UTC", "AssertTimestampMonotonic": "17784041", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2020-11-28 21:39:43 UTC", "ConditionTimestampMonotonic": "17784041", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target systemd-timesyncd.service", "ControlGroup": "/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documen 2020-11-28T21:42:36.339 INFO:teuthology.task.ansible.out:tation": "man:ntpd(8)", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "2000", "ExecMainStartTimestamp": "Sat 2020-11-28 21:39:50 UTC", "ExecMainStartTimestampMonotonic": "24372760", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2020-11-28 21:39:43 UTC", "InactiveExitTimestampMonotonic": "17784851", "InvocationID": "a54ca073ab2d4ea3aefdab74edea8442", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "Limi 2020-11-28T21:42:36.341 INFO:teuthology.task.ansible.out:tRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "2000", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice -.mount sysinit.target", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2020-11-28 21:39:50 UTC", "S 2020-11-28T21:42:36.343 INFO:teuthology.task.ansible.out:tateChangeTimestampMonotonic": "24372773", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2020-11-28 21:39:50 UTC", "WatchdogTimestampMonotonic": "24372772", "WatchdogUSec": "0"}} 2020-11-28T21:42:36.378 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:36 +0000 (0:00:00.077) 0:01:42.522 ***** 2020-11-28T21:42:36.395 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2020-11-28T21:42:36.396 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1606599616.3206215, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1606599616.3166234, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 6119, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1606599616.3166234, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "1817098686", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} TASK [testnode : set_fact] ***************************************************** ok: [smithi162.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2020-11-28T21:42:36.438 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:36 +0000 (0:00:00.060) 0:01:42.583 ***** 2020-11-28T21:42:36.479 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:36 +0000 (0:00:00.040) 0:01:42.624 ***** 2020-11-28T21:42:36.510 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-28T21:42:36.514 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:36.547 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:36 +0000 (0:00:00.068) 0:01:42.692 ***** 2020-11-28T21:42:36.574 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2020-11-28T21:42:36.576 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:36.614 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:36 +0000 (0:00:00.066) 0:01:42.759 ***** 2020-11-28T21:42:36.674 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2020-11-28T21:42:36.724 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:36 +0000 (0:00:00.109) 0:01:42.868 ***** 2020-11-28T21:42:36.733 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2020-11-28T21:42:36.785 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:36 +0000 (0:00:00.061) 0:01:42.930 ***** 2020-11-28T21:42:36.795 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2020-11-28T21:42:36.797 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-11-28T21:42:36.851 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:36 +0000 (0:00:00.065) 0:01:42.995 ***** 2020-11-28T21:42:36.858 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-11-28T21:42:36.903 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:36 +0000 (0:00:00.052) 0:01:43.048 ***** 2020-11-28T21:42:36.913 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2020-11-28T21:42:36.915 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install python-pip on apt based systems.] ********************* ok: [smithi192.front.sepia.ceph.com] => {"cache_update_time": 1606599641, "cache_updated": false, "changed": false} 2020-11-28T21:42:36.954 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:36 +0000 (0:00:00.049) 0:01:43.098 ***** 2020-11-28T21:42:37.011 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:37 +0000 (0:00:00.057) 0:01:43.156 ***** 2020-11-28T21:42:37.019 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2020-11-28T21:42:37.020 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:37.060 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:37 +0000 (0:00:00.049) 0:01:43.205 ***** 2020-11-28T21:42:37.069 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2020-11-28T21:42:37.071 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:37.115 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:37 +0000 (0:00:00.054) 0:01:43.260 ***** 2020-11-28T21:42:37.125 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** changed: [smithi096.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-11-28T21:42:37.164 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:37 +0000 (0:00:00.048) 0:01:43.308 ***** 2020-11-28T21:42:37.193 INFO:teuthology.task.ansible.out:changed: [smithi118.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-11-28T21:42:37.235 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:37 +0000 (0:00:00.070) 0:01:43.379 ***** 2020-11-28T21:42:37.252 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2020-11-28T21:42:37.254 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"changed": false, "gid": 1235, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2020-11-28T21:42:37.306 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:37 +0000 (0:00:00.067) 0:01:43.446 ***** 2020-11-28T21:42:37.310 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2020-11-28T21:42:37.312 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-11-28T21:42:37.352 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:37 +0000 (0:00:00.049) 0:01:43.496 ***** 2020-11-28T21:42:37.369 INFO:teuthology.task.ansible.out:changed: [smithi118.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-11-28T21:42:37.416 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:37 +0000 (0:00:00.064) 0:01:43.561 ***** 2020-11-28T21:42:37.423 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2020-11-28T21:42:37.424 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:37.461 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:37 +0000 (0:00:00.045) 0:01:43.606 ***** 2020-11-28T21:42:37.468 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:37.509 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:37 +0000 (0:00:00.047) 0:01:43.653 ***** 2020-11-28T21:42:37.516 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* skipping: [smithi096.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:37.517 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2020-11-28T21:42:37.517 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => {"cache_update_time": 1606599631, "cache_updated": false, "changed": false} 2020-11-28T21:42:37.518 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* skipping: [smithi096.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:37.565 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:37 +0000 (0:00:00.056) 0:01:43.709 ***** 2020-11-28T21:42:37.571 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:37.573 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:37.575 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:37.610 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:37 +0000 (0:00:00.045) 0:01:43.755 ***** 2020-11-28T21:42:37.616 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:37.618 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:37.620 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:37.669 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:37 +0000 (0:00:00.059) 0:01:43.814 ***** 2020-11-28T21:42:37.705 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:37 +0000 (0:00:00.035) 0:01:43.849 ***** 2020-11-28T21:42:37.712 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2020-11-28T21:42:37.713 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1235, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} TASK [testnode : Define net_to_configure var] ********************************** skipping: [smithi096.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:37.714 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:37.714 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:37.715 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skip 2020-11-28T21:42:37.717 INFO:teuthology.task.ansible.out:ped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:37.717 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:37.760 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:37 +0000 (0:00:00.054) 0:01:43.904 ***** 2020-11-28T21:42:37.796 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:37 +0000 (0:00:00.036) 0:01:43.941 ***** 2020-11-28T21:42:37.805 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi118.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:37.807 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:37.850 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:37 +0000 (0:00:00.053) 0:01:43.994 ***** 2020-11-28T21:42:37.898 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:37 +0000 (0:00:00.048) 0:01:44.043 ***** 2020-11-28T21:42:37.908 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2020-11-28T21:42:37.910 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => {"changed": false, "gid": 1235, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2020-11-28T21:42:37.910 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:37.969 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:37 +0000 (0:00:00.070) 0:01:44.114 ***** 2020-11-28T21:42:38.006 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:37 +0000 (0:00:00.036) 0:01:44.151 ***** 2020-11-28T21:42:38.013 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-28T21:42:38.015 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:38.054 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:38 +0000 (0:00:00.047) 0:01:44.198 ***** 2020-11-28T21:42:38.092 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:38 +0000 (0:00:00.037) 0:01:44.236 ***** 2020-11-28T21:42:38.100 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:38.137 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:38 +0000 (0:00:00.045) 0:01:44.281 ***** 2020-11-28T21:42:38.151 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2020-11-28T21:42:38.153 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:38.211 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:38 +0000 (0:00:00.071) 0:01:44.353 ***** 2020-11-28T21:42:38.219 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:38.260 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:38 +0000 (0:00:00.051) 0:01:44.404 ***** 2020-11-28T21:42:38.268 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:38.306 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:38 +0000 (0:00:00.046) 0:01:44.451 ***** 2020-11-28T21:42:38.313 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-11-28T21:42:38.314 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:38.360 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:38 +0000 (0:00:00.053) 0:01:44.505 ***** 2020-11-28T21:42:38.366 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2020-11-28T21:42:38.368 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create pip.conf and configure it to use our mirror] *********** ok: [smithi162.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1235, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2020-11-28T21:42:38.423 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:38 +0000 (0:00:00.062) 0:01:44.568 ***** 2020-11-28T21:42:38.478 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:38 +0000 (0:00:00.054) 0:01:44.622 ***** 2020-11-28T21:42:38.486 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:38.489 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:38.524 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:38 +0000 (0:00:00.046) 0:01:44.669 ***** 2020-11-28T21:42:38.562 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:38 +0000 (0:00:00.037) 0:01:44.706 ***** 2020-11-28T21:42:38.571 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* ok: [smithi192.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages\nRequirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages", "Requirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2020-11-28T21:42:38.618 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:38 +0000 (0:00:00.054) 0:01:44.761 ***** 2020-11-28T21:42:38.624 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-11-28T21:42:38.626 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:38.666 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:38 +0000 (0:00:00.047) 0:01:44.809 ***** 2020-11-28T21:42:38.706 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:38 +0000 (0:00:00.040) 0:01:44.850 ***** 2020-11-28T21:42:38.740 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:38.794 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:38 +0000 (0:00:00.088) 0:01:44.938 ***** 2020-11-28T21:42:38.832 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-11-28T21:42:38.834 INFO:teuthology.task.ansible.out:skipping: [smithi118.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-11-28T21:42:38.834 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.010103", "end": "2020-11-28 21:42:39.162941", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-11-28 21:42:39.152838", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:38.892 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:38 +0000 (0:00:00.098) 0:01:45.036 ***** 2020-11-28T21:42:38.981 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.005800", "end": "2020-11-28 21:42:39.308821", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-11-28 21:42:39.303021", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:39.054 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.008947", "end": "2020-11-28 21:42:39.072934", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-11-28 21:42:39.063987", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:39.098 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.005202", "end": "2020-11-28 21:42:39.443084", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-11-28 21:42:39.437882", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:39.152 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* ok: [smithi192.front.sepia.ceph.com] => {"changed": false} 2020-11-28T21:42:39.193 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:39 +0000 (0:00:00.301) 0:01:45.338 ***** 2020-11-28T21:42:39.200 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** ok: [smithi118.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.006062", "end": "2020-11-28 21:42:39.193202", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-11-28 21:42:39.187140", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:39.222 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2020-11-28T21:42:39.224 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2020-11-28T21:42:39.259 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:39 +0000 (0:00:00.066) 0:01:45.404 ***** 2020-11-28T21:42:39.317 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-11-28T21:42:39.319 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.005597", "end": "2020-11-28 21:42:39.326815", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-11-28 21:42:39.321218", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} ok: [smithi096.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.102600", "end": "2020-11-28 21:42:39.645236", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-11-28 21:42:39.542636", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:39.382 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:39 +0000 (0:00:00.123) 0:01:45.527 ***** 2020-11-28T21:42:39.405 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* ok: [smithi162.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages\nRequirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages", "Requirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2020-11-28T21:42:39.439 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:39 +0000 (0:00:00.056) 0:01:45.583 ***** 2020-11-28T21:42:39.445 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-11-28T21:42:39.447 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** ok: [smithi192.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1606599593.7330384, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1606599593.7290406, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 6119, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1606599593.7290406, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "1817098686", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2020-11-28T21:42:39.497 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:39 +0000 (0:00:00.058) 0:01:45.642 ***** 2020-11-28T21:42:39.534 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:39 +0000 (0:00:00.036) 0:01:45.679 ***** 2020-11-28T21:42:39.540 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-11-28T21:42:39.541 INFO:teuthology.task.ansible.out:ok: [smithi118.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.028136", "end": "2020-11-28 21:42:39.470665", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-11-28 21:42:39.442529", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:39.581 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:39 +0000 (0:00:00.046) 0:01:45.725 ***** 2020-11-28T21:42:39.619 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-11-28T21:42:39.655 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:39 +0000 (0:00:00.074) 0:01:45.800 ***** 2020-11-28T21:42:39.662 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2020-11-28T21:42:39.663 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2020-11-28T21:42:39.715 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2020-11-28T21:42:39.718 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2020-11-28T21:42:39.786 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:39 +0000 (0:00:00.130) 0:01:45.930 ***** 2020-11-28T21:42:39.816 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** changed: [smithi118.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** ok: [smithi192.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-11-28T21:42:39.877 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:39 +0000 (0:00:00.091) 0:01:46.021 ***** 2020-11-28T21:42:39.926 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2020-11-28T21:42:39.926 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:39.961 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:39 +0000 (0:00:00.084) 0:01:46.106 ***** 2020-11-28T21:42:40.078 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* ok: [smithi162.front.sepia.ceph.com] => {"changed": false} 2020-11-28T21:42:40.116 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:40 +0000 (0:00:00.154) 0:01:46.261 ***** 2020-11-28T21:42:40.123 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2020-11-28T21:42:40.125 INFO:teuthology.task.ansible.out:changed: [smithi192.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-11-28T21:42:40.164 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:40 +0000 (0:00:00.047) 0:01:46.309 ***** 2020-11-28T21:42:40.172 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2020-11-28T21:42:40.173 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2020-11-28T21:42:40.213 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:40 +0000 (0:00:00.047) 0:01:46.357 ***** 2020-11-28T21:42:40.284 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2020-11-28T21:42:40.286 INFO:teuthology.task.ansible.out:changed: [smithi192.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-11-28T21:42:40.325 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:40 +0000 (0:00:00.110) 0:01:46.468 ***** 2020-11-28T21:42:40.341 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** ok: [smithi162.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1606599604.545598, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1606599604.5416, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 6119, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1606599604.5416, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "1817098686", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2020-11-28T21:42:40.382 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:40 +0000 (0:00:00.058) 0:01:46.526 ***** 2020-11-28T21:42:40.389 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2020-11-28T21:42:40.390 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:40.433 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:40 +0000 (0:00:00.051) 0:01:46.578 ***** 2020-11-28T21:42:40.480 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2020-11-28T21:42:40.482 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:40.507 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:40.509 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:40.556 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:40 +0000 (0:00:00.123) 0:01:46.701 ***** 2020-11-28T21:42:40.562 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2020-11-28T21:42:40.564 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2020-11-28T21:42:40.603 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:40 +0000 (0:00:00.046) 0:01:46.747 ***** 2020-11-28T21:42:40.615 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2020-11-28T21:42:40.617 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:40.639 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:40.641 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** ok: [smithi162.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-11-28T21:42:40.681 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:40 +0000 (0:00:00.078) 0:01:46.826 ***** 2020-11-28T21:42:40.689 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2020-11-28T21:42:40.691 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi192.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:40.743 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:40 +0000 (0:00:00.061) 0:01:46.887 ***** 2020-11-28T21:42:40.750 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2020-11-28T21:42:40.752 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:40.810 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:40 +0000 (0:00:00.066) 0:01:46.954 ***** 2020-11-28T21:42:40.817 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-28T21:42:40.821 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:40.858 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:40 +0000 (0:00:00.048) 0:01:47.003 ***** 2020-11-28T21:42:40.898 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2020-11-28T21:42:40.899 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:40.941 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:40 +0000 (0:00:00.082) 0:01:47.085 ***** 2020-11-28T21:42:40.948 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2020-11-28T21:42:40.950 INFO:teuthology.task.ansible.out:changed: [smithi162.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-11-28T21:42:40.991 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:40 +0000 (0:00:00.049) 0:01:47.135 ***** 2020-11-28T21:42:40.997 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2020-11-28T21:42:40.998 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:41.042 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:41 +0000 (0:00:00.051) 0:01:47.187 ***** 2020-11-28T21:42:41.081 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2020-11-28T21:42:41.083 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:41.144 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:41 +0000 (0:00:00.102) 0:01:47.289 ***** 2020-11-28T21:42:41.153 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2020-11-28T21:42:41.155 INFO:teuthology.task.ansible.out:changed: [smithi162.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-11-28T21:42:41.215 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:41 +0000 (0:00:00.070) 0:01:47.360 ***** 2020-11-28T21:42:41.225 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2020-11-28T21:42:41.226 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:41.275 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:41 +0000 (0:00:00.059) 0:01:47.420 ***** 2020-11-28T21:42:41.281 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2020-11-28T21:42:41.283 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:41.330 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:41 +0000 (0:00:00.050) 0:01:47.471 ***** 2020-11-28T21:42:41.338 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-11-28T21:42:41.339 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:41.386 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:41 +0000 (0:00:00.059) 0:01:47.530 ***** 2020-11-28T21:42:41.392 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2020-11-28T21:42:41.393 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:41.394 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:41.394 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2020-11-28T21:42:41.433 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:41 +0000 (0:00:00.047) 0:01:47.578 ***** 2020-11-28T21:42:41.440 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2020-11-28T21:42:41.441 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:41.490 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:41 +0000 (0:00:00.056) 0:01:47.635 ***** 2020-11-28T21:42:41.492 INFO:teuthology.task.ansible.out: 2020-11-28T21:42:41.497 INFO:teuthology.task.ansible.out: 2020-11-28T21:42:41.500 INFO:teuthology.task.ansible.out:TASK [testnode : Define net_to_configure var] ********************************** skipping: [smithi162.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:41.500 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi162.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:41.502 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:41.540 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:41 +0000 (0:00:00.049) 0:01:47.685 ***** 2020-11-28T21:42:41.546 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-11-28T21:42:41.548 INFO:teuthology.task.ansible.out:skipping: [smithi192.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:41.582 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:41 +0000 (0:00:00.042) 0:01:47.727 ***** 2020-11-28T21:42:41.589 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-28T21:42:41.591 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:41.641 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:41 +0000 (0:00:00.052) 0:01:47.779 ***** 2020-11-28T21:42:41.663 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:41.718 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:41 +0000 (0:00:00.080) 0:01:47.860 ***** 2020-11-28T21:42:41.748 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-11-28T21:42:41.752 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.011286", "end": "2020-11-28 21:42:41.995450", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-11-28 21:42:41.984164", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:41.799 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:41 +0000 (0:00:00.083) 0:01:47.944 ***** 2020-11-28T21:42:41.830 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2020-11-28T21:42:41.831 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:41.868 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:41 +0000 (0:00:00.069) 0:01:48.013 ***** 2020-11-28T21:42:41.870 INFO:teuthology.task.ansible.out: 2020-11-28T21:42:41.888 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** ok: [smithi192.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.006009", "end": "2020-11-28 21:42:42.139604", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-11-28 21:42:42.133595", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:41.898 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:41.936 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:41 +0000 (0:00:00.067) 0:01:48.081 ***** 2020-11-28T21:42:41.964 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:42.009 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:42 +0000 (0:00:00.071) 0:01:48.152 ***** 2020-11-28T21:42:42.024 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-11-28T21:42:42.025 INFO:teuthology.task.ansible.out:ok: [smithi192.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.005572", "end": "2020-11-28 21:42:42.275369", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-11-28 21:42:42.269797", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:42.036 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2020-11-28T21:42:42.037 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:42.086 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:42 +0000 (0:00:00.079) 0:01:48.231 ***** 2020-11-28T21:42:42.114 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-11-28T21:42:42.116 INFO:teuthology.task.ansible.out:skipping: [smithi162.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-28T21:42:42.150 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:42 +0000 (0:00:00.063) 0:01:48.295 ***** 2020-11-28T21:42:42.158 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** ok: [smithi192.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.018687", "end": "2020-11-28 21:42:42.412797", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-11-28 21:42:42.394110", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:42.208 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:42 +0000 (0:00:00.058) 0:01:48.353 ***** 2020-11-28T21:42:42.236 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-11-28T21:42:42.276 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:42 +0000 (0:00:00.067) 0:01:48.421 ***** 2020-11-28T21:42:42.314 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-11-28T21:42:42.316 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.009433", "end": "2020-11-28 21:42:42.649106", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-11-28 21:42:42.639673", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:42.390 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2020-11-28T21:42:42.392 INFO:teuthology.task.ansible.out:changed: [smithi192.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2020-11-28T21:42:42.474 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-11-28T21:42:42.475 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.004294", "end": "2020-11-28 21:42:42.782693", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-11-28 21:42:42.778399", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:42.550 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.005488", "end": "2020-11-28 21:42:42.904482", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-11-28 21:42:42.898994", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:42.702 INFO:teuthology.task.ansible.out:ok: [smithi162.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.020371", "end": "2020-11-28 21:42:43.042852", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-11-28 21:42:43.022481", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-28T21:42:42.739 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:42 +0000 (0:00:00.462) 0:01:48.883 ***** 2020-11-28T21:42:42.766 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-11-28T21:42:42.806 INFO:teuthology.task.ansible.out:Saturday 28 November 2020 21:42:42 +0000 (0:00:00.068) 0:01:48.951 ***** 2020-11-28T21:42:42.939 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** changed: [smithi162.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2020-11-28T21:42:43.050 INFO:teuthology.task.ansible.out: PLAY [cobbler] ***************************************************************** skipping: no hosts matched PLAY [paddles] ***************************************************************** skipping: no hosts matched 2020-11-28T21:42:43.051 INFO:teuthology.task.ansible.out: PLAY [pulpito] ***************************************************************** skipping: no hosts matched PLAY RECAP ********************************************************************* 2020-11-28T21:42:43.052 INFO:teuthology.task.ansible.out:smithi096.front.sepia.ceph.com : ok=114 changed=30 unreachable=0 failed=0 skipped=182 rescued=0 ignored=1 smithi118.front.sepia.ceph.com : ok=114 changed=30 unreachable=0 failed=0 skipped=182 rescued=0 ignored=1 smithi162.front.sepia.ceph.com : ok=114 changed=30 unreachable=0 failed=0 skipped=182 rescued=0 ignored=1 smithi192.front.sepia.ceph.com : ok=114 changed=30 unreachable=0 failed=0 skipped=182 rescued=0 ignored=1 Saturday 28 November 2020 21:42:43 +0000 (0:00:00.245) 0:01:49.196 ***** =============================================================================== 2020-11-28T21:42:43.054 INFO:teuthology.task.ansible.out:testnode : Ensure packages are not present. ----------------------------- 5.35s testnode : Install packages --------------------------------------------- 5.34s testnode : Zap all non-root disks --------------------------------------- 3.94s testnode : Update apt cache. -------------------------------------------- 2.22s common : Check for docker ----------------------------------------------- 1.75s Gathering Facts --------------------------------------------------------- 1.60s common : Set the hardware clock ----------------------------------------- 0.87s Gathering Facts --------------------------------------------------------- 0.68s ansible-managed : Add authorized keys for the ansible user. ------------- 0.64s ansible-managed : Create the cephlab_sudo sudoers.d file. --------------- 0.57s common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/. --- 0.51s testnode : Install python-apt ------------------------------------------- 0.39s testnode : Install ntp package on apt based systems. -------------------- 0.38s testnode : Ensure perl-doc is installed on apt systems. ----------------- 0.38s 2020-11-28T21:42:43.059 INFO:teuthology.task.ansible.out:common : Install Kerberos Packages (Debian) ----------------------------- 0.35s ansible-managed : Create the ansible user. ------------------------------ 0.34s testnode : Write /scratch_devs ------------------------------------------ 0.29s common : Update apt cache ----------------------------------------------- 0.27s testnode : Touch /ceph-qa-ready ----------------------------------------- 0.25s testnode : Create xfs filesystem on /dev/vg_nvme/lv_5 ------------------- 0.24s 2020-11-28T21:42:43.378 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi162.front.sepia.ceph.com'), Remote(name='ubuntu@smithi118.front.sepia.ceph.com'), Remote(name='ubuntu@smithi192.front.sepia.ceph.com'), Remote(name='ubuntu@smithi096.front.sepia.ceph.com')] 2020-11-28T21:42:43.379 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-28T21:42:43.380 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:42:43.455 INFO:teuthology.orchestra.run.smithi162:> true 2020-11-28T21:42:43.664 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-28T21:42:43.717 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:42:43.802 INFO:teuthology.orchestra.run.smithi118:> true 2020-11-28T21:42:43.987 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-28T21:42:43.988 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:42:44.062 INFO:teuthology.orchestra.run.smithi192:> true 2020-11-28T21:42:44.255 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-28T21:42:44.255 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-28T21:42:44.333 INFO:teuthology.orchestra.run.smithi096:> true 2020-11-28T21:42:44.523 INFO:teuthology.run_tasks:Running task clock... 2020-11-28T21:42:44.534 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2020-11-28T21:42:44.534 INFO:teuthology.orchestra.run:Running command with timeout 360 2020-11-28T21:42:44.535 INFO:teuthology.orchestra.run.smithi162:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-11-28T21:42:44.591 INFO:teuthology.orchestra.run.smithi162.stdout:28 Nov 21:42:44 ntpd[7143]: ntpd 4.2.8p10@1.3728-o (1): Starting 2020-11-28T21:42:44.591 INFO:teuthology.orchestra.run.smithi162.stdout:28 Nov 21:42:44 ntpd[7143]: Command line: ntpd -gq 2020-11-28T21:42:44.593 INFO:teuthology.orchestra.run.smithi162.stdout:28 Nov 21:42:44 ntpd[7143]: proto: precision = 0.093 usec (-23) 2020-11-28T21:42:44.593 INFO:teuthology.orchestra.run.smithi162.stdout:28 Nov 21:42:44 ntpd[7143]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-11-28T21:42:44.593 INFO:teuthology.orchestra.run.smithi162.stdout:28 Nov 21:42:44 ntpd[7143]: restrict ::: KOD does nothing without LIMITED. 2020-11-28T21:42:44.593 INFO:teuthology.orchestra.run.smithi162.stdout:28 Nov 21:42:44 ntpd[7143]: Listen and drop on 0 v6wildcard [::]:123 2020-11-28T21:42:44.594 INFO:teuthology.orchestra.run.smithi162.stdout:28 Nov 21:42:44 ntpd[7143]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2020-11-28T21:42:44.594 INFO:teuthology.orchestra.run.smithi162.stdout:28 Nov 21:42:44 ntpd[7143]: Listen normally on 2 lo 127.0.0.1:123 2020-11-28T21:42:44.595 INFO:teuthology.orchestra.run.smithi162.stdout:28 Nov 21:42:44 ntpd[7143]: Listen normally on 3 enp3s0f1 172.21.15.162:123 2020-11-28T21:42:44.596 INFO:teuthology.orchestra.run.smithi162.stdout:28 Nov 21:42:44 ntpd[7143]: Listen normally on 4 lo [::1]:123 2020-11-28T21:42:44.596 INFO:teuthology.orchestra.run.smithi162.stdout:28 Nov 21:42:44 ntpd[7143]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7f41%5]:123 2020-11-28T21:42:44.596 INFO:teuthology.orchestra.run.smithi162.stdout:28 Nov 21:42:44 ntpd[7143]: Listening on routing socket on fd #22 for interface updates 2020-11-28T21:42:44.596 INFO:teuthology.orchestra.run.smithi162.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-11-28T21:42:44.597 INFO:teuthology.orchestra.run.smithi162.stderr:restrict ::: KOD does nothing without LIMITED. 2020-11-28T21:42:45.592 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:45 ntpd[7143]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:42:45.592 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:45 ntpd[7143]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:42:45.593 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:45 ntpd[7143]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:42:45.593 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:45 ntpd[7143]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:42:46.594 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:46 ntpd[7143]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:42:46.594 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:46 ntpd[7143]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:42:46.594 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:46 ntpd[7143]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:42:46.595 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:46 ntpd[7143]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:42:46.595 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:46 ntpd[7143]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:42:46.595 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:46 ntpd[7143]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:42:46.595 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:46 ntpd[7143]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:42:46.595 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:46 ntpd[7143]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:42:47.593 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:47 ntpd[7143]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:42:47.594 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:47 ntpd[7143]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:42:47.594 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:47 ntpd[7143]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:42:47.594 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:47 ntpd[7143]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:42:47.594 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:47 ntpd[7143]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:42:47.595 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:47 ntpd[7143]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:42:47.595 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:47 ntpd[7143]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:42:47.595 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:47 ntpd[7143]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:42:48.594 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:48 ntpd[7143]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:42:48.594 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:48 ntpd[7143]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:42:48.594 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:48 ntpd[7143]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:42:48.594 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:48 ntpd[7143]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:42:48.595 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:48 ntpd[7143]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:42:48.595 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:48 ntpd[7143]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:42:48.595 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:48 ntpd[7143]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:42:48.595 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:48 ntpd[7143]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:42:49.594 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:49 ntpd[7143]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:42:49.595 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:49 ntpd[7143]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:42:49.595 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:49 ntpd[7143]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:42:49.595 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:49 ntpd[7143]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:42:49.595 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:49 ntpd[7143]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:42:49.595 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:49 ntpd[7143]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:42:49.596 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:49 ntpd[7143]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:42:49.596 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:49 ntpd[7143]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:42:50.595 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:50 ntpd[7143]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:42:50.595 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:50 ntpd[7143]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:42:50.595 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:50 ntpd[7143]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:42:50.595 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:50 ntpd[7143]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:42:50.596 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:50 ntpd[7143]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:42:50.596 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:50 ntpd[7143]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:42:50.596 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:50 ntpd[7143]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:42:50.596 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:50 ntpd[7143]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:42:51.595 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:51 ntpd[7143]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:42:51.595 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:51 ntpd[7143]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:42:51.596 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:51 ntpd[7143]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:42:51.596 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:51 ntpd[7143]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:42:51.596 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:51 ntpd[7143]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2020-11-28T21:42:51.596 INFO:teuthology.orchestra.run.smithi162.stderr:28 Nov 21:42:51 ntpd[7143]: can't open /var/log/ntpstats/loopstats.20201128: Permission denied 2020-11-28T21:42:51.597 INFO:teuthology.orchestra.run.smithi162.stdout:28 Nov 21:42:51 ntpd[7143]: ntpd: time set -0.374754 s 2020-11-28T21:42:51.597 INFO:teuthology.orchestra.run.smithi162.stdout:ntpd: time set -0.374754s 2020-11-28T21:42:51.656 INFO:teuthology.orchestra.run.smithi162.stdout: remote refid st t when poll reach delay offset jitter 2020-11-28T21:42:51.657 INFO:teuthology.orchestra.run.smithi162.stdout:============================================================================== 2020-11-28T21:42:51.657 INFO:teuthology.orchestra.run.smithi162.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-28T21:42:51.657 INFO:teuthology.orchestra.run.smithi162.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-28T21:42:51.657 INFO:teuthology.orchestra.run.smithi162.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-28T21:42:51.657 INFO:teuthology.orchestra.run.smithi162.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-28T21:42:51.659 INFO:teuthology.orchestra.run:Running command with timeout 360 2020-11-28T21:42:51.660 INFO:teuthology.orchestra.run.smithi118:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-11-28T21:42:51.715 INFO:teuthology.orchestra.run.smithi118.stdout:28 Nov 21:42:51 ntpd[7131]: ntpd 4.2.8p10@1.3728-o (1): Starting 2020-11-28T21:42:51.716 INFO:teuthology.orchestra.run.smithi118.stdout:28 Nov 21:42:51 ntpd[7131]: Command line: ntpd -gq 2020-11-28T21:42:51.719 INFO:teuthology.orchestra.run.smithi118.stdout:28 Nov 21:42:51 ntpd[7131]: proto: precision = 0.123 usec (-23) 2020-11-28T21:42:51.719 INFO:teuthology.orchestra.run.smithi118.stdout:28 Nov 21:42:51 ntpd[7131]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-11-28T21:42:51.719 INFO:teuthology.orchestra.run.smithi118.stdout:28 Nov 21:42:51 ntpd[7131]: restrict ::: KOD does nothing without LIMITED. 2020-11-28T21:42:51.720 INFO:teuthology.orchestra.run.smithi118.stdout:28 Nov 21:42:51 ntpd[7131]: Listen and drop on 0 v6wildcard [::]:123 2020-11-28T21:42:51.720 INFO:teuthology.orchestra.run.smithi118.stdout:28 Nov 21:42:51 ntpd[7131]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2020-11-28T21:42:51.720 INFO:teuthology.orchestra.run.smithi118.stdout:28 Nov 21:42:51 ntpd[7131]: Listen normally on 2 lo 127.0.0.1:123 2020-11-28T21:42:51.720 INFO:teuthology.orchestra.run.smithi118.stdout:28 Nov 21:42:51 ntpd[7131]: Listen normally on 3 enp3s0f1 172.21.15.118:123 2020-11-28T21:42:51.721 INFO:teuthology.orchestra.run.smithi118.stdout:28 Nov 21:42:51 ntpd[7131]: Listen normally on 4 lo [::1]:123 2020-11-28T21:42:51.721 INFO:teuthology.orchestra.run.smithi118.stdout:28 Nov 21:42:51 ntpd[7131]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d725%5]:123 2020-11-28T21:42:51.721 INFO:teuthology.orchestra.run.smithi118.stdout:28 Nov 21:42:51 ntpd[7131]: Listening on routing socket on fd #22 for interface updates 2020-11-28T21:42:51.722 INFO:teuthology.orchestra.run.smithi118.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-11-28T21:42:51.722 INFO:teuthology.orchestra.run.smithi118.stderr:restrict ::: KOD does nothing without LIMITED. 2020-11-28T21:42:54.718 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:54 ntpd[7131]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:42:54.718 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:54 ntpd[7131]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:42:54.718 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:54 ntpd[7131]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:42:54.719 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:54 ntpd[7131]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:42:55.719 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:55 ntpd[7131]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:42:55.719 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:55 ntpd[7131]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:42:55.720 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:55 ntpd[7131]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:42:55.720 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:55 ntpd[7131]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:42:55.721 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:55 ntpd[7131]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:42:55.721 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:55 ntpd[7131]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:42:55.721 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:55 ntpd[7131]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:42:55.722 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:55 ntpd[7131]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:42:56.719 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:56 ntpd[7131]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:42:56.719 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:56 ntpd[7131]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:42:56.719 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:56 ntpd[7131]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:42:56.719 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:56 ntpd[7131]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:42:56.720 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:56 ntpd[7131]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:42:56.720 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:56 ntpd[7131]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:42:56.720 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:56 ntpd[7131]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:42:56.720 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:56 ntpd[7131]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:42:57.719 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:57 ntpd[7131]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:42:57.720 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:57 ntpd[7131]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:42:57.720 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:57 ntpd[7131]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:42:57.720 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:57 ntpd[7131]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:42:57.720 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:57 ntpd[7131]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:42:57.721 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:57 ntpd[7131]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:42:57.721 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:57 ntpd[7131]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:42:57.721 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:57 ntpd[7131]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:42:58.719 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:58 ntpd[7131]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:42:58.720 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:58 ntpd[7131]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:42:58.720 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:58 ntpd[7131]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:42:58.720 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:58 ntpd[7131]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:42:58.721 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:58 ntpd[7131]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:42:58.721 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:58 ntpd[7131]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:42:58.721 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:58 ntpd[7131]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:42:58.721 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:58 ntpd[7131]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:42:59.720 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:59 ntpd[7131]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:42:59.720 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:59 ntpd[7131]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:42:59.721 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:59 ntpd[7131]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:42:59.721 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:59 ntpd[7131]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:42:59.721 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:59 ntpd[7131]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:42:59.721 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:59 ntpd[7131]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:42:59.721 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:59 ntpd[7131]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:42:59.722 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:42:59 ntpd[7131]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:43:00.721 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:43:00 ntpd[7131]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:43:00.721 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:43:00 ntpd[7131]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:43:00.722 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:43:00 ntpd[7131]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:43:00.722 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:43:00 ntpd[7131]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:43:00.722 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:43:00 ntpd[7131]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2020-11-28T21:43:00.722 INFO:teuthology.orchestra.run.smithi118.stderr:28 Nov 21:43:00 ntpd[7131]: can't open /var/log/ntpstats/loopstats.20201128: Permission denied 2020-11-28T21:43:00.723 INFO:teuthology.orchestra.run.smithi118.stdout:28 Nov 21:43:00 ntpd[7131]: ntpd: time set -0.039635 s 2020-11-28T21:43:00.723 INFO:teuthology.orchestra.run.smithi118.stdout:ntpd: time set -0.039635s 2020-11-28T21:43:00.786 INFO:teuthology.orchestra.run.smithi118.stdout: remote refid st t when poll reach delay offset jitter 2020-11-28T21:43:00.786 INFO:teuthology.orchestra.run.smithi118.stdout:============================================================================== 2020-11-28T21:43:00.786 INFO:teuthology.orchestra.run.smithi118.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-28T21:43:00.787 INFO:teuthology.orchestra.run.smithi118.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-28T21:43:00.787 INFO:teuthology.orchestra.run.smithi118.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-28T21:43:00.787 INFO:teuthology.orchestra.run.smithi118.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-28T21:43:00.788 INFO:teuthology.orchestra.run:Running command with timeout 360 2020-11-28T21:43:00.788 INFO:teuthology.orchestra.run.smithi192:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-11-28T21:43:00.849 INFO:teuthology.orchestra.run.smithi192.stdout:28 Nov 21:43:00 ntpd[7110]: ntpd 4.2.8p10@1.3728-o (1): Starting 2020-11-28T21:43:00.850 INFO:teuthology.orchestra.run.smithi192.stdout:28 Nov 21:43:00 ntpd[7110]: Command line: ntpd -gq 2020-11-28T21:43:00.851 INFO:teuthology.orchestra.run.smithi192.stdout:28 Nov 21:43:00 ntpd[7110]: proto: precision = 0.114 usec (-23) 2020-11-28T21:43:00.851 INFO:teuthology.orchestra.run.smithi192.stdout:28 Nov 21:43:00 ntpd[7110]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-11-28T21:43:00.851 INFO:teuthology.orchestra.run.smithi192.stdout:28 Nov 21:43:00 ntpd[7110]: restrict ::: KOD does nothing without LIMITED. 2020-11-28T21:43:00.851 INFO:teuthology.orchestra.run.smithi192.stdout:28 Nov 21:43:00 ntpd[7110]: Listen and drop on 0 v6wildcard [::]:123 2020-11-28T21:43:00.852 INFO:teuthology.orchestra.run.smithi192.stdout:28 Nov 21:43:00 ntpd[7110]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2020-11-28T21:43:00.852 INFO:teuthology.orchestra.run.smithi192.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-11-28T21:43:00.852 INFO:teuthology.orchestra.run.smithi192.stderr:restrict ::: KOD does nothing without LIMITED. 2020-11-28T21:43:00.853 INFO:teuthology.orchestra.run.smithi192.stdout:28 Nov 21:43:00 ntpd[7110]: Listen normally on 2 lo 127.0.0.1:123 2020-11-28T21:43:00.853 INFO:teuthology.orchestra.run.smithi192.stdout:28 Nov 21:43:00 ntpd[7110]: Listen normally on 3 enp3s0f1 172.21.15.192:123 2020-11-28T21:43:00.853 INFO:teuthology.orchestra.run.smithi192.stdout:28 Nov 21:43:00 ntpd[7110]: Listen normally on 4 lo [::1]:123 2020-11-28T21:43:00.854 INFO:teuthology.orchestra.run.smithi192.stdout:28 Nov 21:43:00 ntpd[7110]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:8049%5]:123 2020-11-28T21:43:00.854 INFO:teuthology.orchestra.run.smithi192.stdout:28 Nov 21:43:00 ntpd[7110]: Listening on routing socket on fd #22 for interface updates 2020-11-28T21:43:01.850 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:01 ntpd[7110]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:43:01.851 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:01 ntpd[7110]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:43:01.851 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:01 ntpd[7110]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:43:01.851 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:01 ntpd[7110]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:43:02.850 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:02 ntpd[7110]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:43:02.850 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:02 ntpd[7110]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:43:02.850 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:02 ntpd[7110]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:43:02.851 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:02 ntpd[7110]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:43:03.850 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:03 ntpd[7110]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:43:03.851 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:03 ntpd[7110]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:43:03.851 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:03 ntpd[7110]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:43:03.851 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:03 ntpd[7110]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:43:04.852 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:04 ntpd[7110]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:43:04.853 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:04 ntpd[7110]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:43:04.853 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:04 ntpd[7110]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:43:04.854 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:04 ntpd[7110]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:43:04.854 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:04 ntpd[7110]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:43:04.855 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:04 ntpd[7110]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:43:04.855 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:04 ntpd[7110]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:43:04.855 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:04 ntpd[7110]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:43:05.850 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:05 ntpd[7110]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:43:05.851 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:05 ntpd[7110]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:43:05.851 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:05 ntpd[7110]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:43:05.851 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:05 ntpd[7110]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:43:05.851 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:05 ntpd[7110]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:43:05.852 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:05 ntpd[7110]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:43:05.852 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:05 ntpd[7110]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:43:05.852 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:05 ntpd[7110]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:43:06.851 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:06 ntpd[7110]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:43:06.852 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:06 ntpd[7110]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:43:06.852 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:06 ntpd[7110]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:43:06.853 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:06 ntpd[7110]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:43:06.853 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:06 ntpd[7110]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:43:06.853 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:06 ntpd[7110]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:43:06.854 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:06 ntpd[7110]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:43:06.854 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:06 ntpd[7110]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:43:07.851 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:07 ntpd[7110]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:43:07.851 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:07 ntpd[7110]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:43:07.851 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:07 ntpd[7110]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:43:07.851 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:07 ntpd[7110]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:43:07.852 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:07 ntpd[7110]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:43:07.852 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:07 ntpd[7110]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:43:07.852 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:07 ntpd[7110]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:43:07.852 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:07 ntpd[7110]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:43:07.852 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:07 ntpd[7110]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2020-11-28T21:43:07.853 INFO:teuthology.orchestra.run.smithi192.stderr:28 Nov 21:43:07 ntpd[7110]: can't open /var/log/ntpstats/loopstats.20201128: Permission denied 2020-11-28T21:43:07.853 INFO:teuthology.orchestra.run.smithi192.stdout:28 Nov 21:43:07 ntpd[7110]: ntpd: time slew +0.001306 s 2020-11-28T21:43:07.853 INFO:teuthology.orchestra.run.smithi192.stdout:ntpd: time slew +0.001306s 2020-11-28T21:43:07.902 INFO:teuthology.orchestra.run.smithi192.stdout: remote refid st t when poll reach delay offset jitter 2020-11-28T21:43:07.903 INFO:teuthology.orchestra.run.smithi192.stdout:============================================================================== 2020-11-28T21:43:07.903 INFO:teuthology.orchestra.run.smithi192.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-28T21:43:07.903 INFO:teuthology.orchestra.run.smithi192.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-28T21:43:07.903 INFO:teuthology.orchestra.run.smithi192.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-28T21:43:07.903 INFO:teuthology.orchestra.run.smithi192.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-28T21:43:07.904 INFO:teuthology.orchestra.run:Running command with timeout 360 2020-11-28T21:43:07.905 INFO:teuthology.orchestra.run.smithi096:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-11-28T21:43:07.968 INFO:teuthology.orchestra.run.smithi096.stdout:28 Nov 21:43:07 ntpd[7144]: ntpd 4.2.8p10@1.3728-o (1): Starting 2020-11-28T21:43:07.968 INFO:teuthology.orchestra.run.smithi096.stdout:28 Nov 21:43:07 ntpd[7144]: Command line: ntpd -gq 2020-11-28T21:43:07.969 INFO:teuthology.orchestra.run.smithi096.stdout:28 Nov 21:43:07 ntpd[7144]: proto: precision = 0.124 usec (-23) 2020-11-28T21:43:07.970 INFO:teuthology.orchestra.run.smithi096.stdout:28 Nov 21:43:07 ntpd[7144]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-11-28T21:43:07.970 INFO:teuthology.orchestra.run.smithi096.stdout:28 Nov 21:43:07 ntpd[7144]: restrict ::: KOD does nothing without LIMITED. 2020-11-28T21:43:07.970 INFO:teuthology.orchestra.run.smithi096.stdout:28 Nov 21:43:07 ntpd[7144]: Listen and drop on 0 v6wildcard [::]:123 2020-11-28T21:43:07.970 INFO:teuthology.orchestra.run.smithi096.stdout:28 Nov 21:43:07 ntpd[7144]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2020-11-28T21:43:07.971 INFO:teuthology.orchestra.run.smithi096.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-11-28T21:43:07.971 INFO:teuthology.orchestra.run.smithi096.stderr:restrict ::: KOD does nothing without LIMITED. 2020-11-28T21:43:07.972 INFO:teuthology.orchestra.run.smithi096.stdout:28 Nov 21:43:07 ntpd[7144]: Listen normally on 2 lo 127.0.0.1:123 2020-11-28T21:43:07.972 INFO:teuthology.orchestra.run.smithi096.stdout:28 Nov 21:43:07 ntpd[7144]: Listen normally on 3 enp3s0f1 172.21.15.96:123 2020-11-28T21:43:07.972 INFO:teuthology.orchestra.run.smithi096.stdout:28 Nov 21:43:07 ntpd[7144]: Listen normally on 4 lo [::1]:123 2020-11-28T21:43:07.972 INFO:teuthology.orchestra.run.smithi096.stdout:28 Nov 21:43:07 ntpd[7144]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:cc0d%5]:123 2020-11-28T21:43:07.973 INFO:teuthology.orchestra.run.smithi096.stdout:28 Nov 21:43:07 ntpd[7144]: Listening on routing socket on fd #22 for interface updates 2020-11-28T21:43:08.968 INFO:teuthology.orchestra.run.smithi096.stderr:28 Nov 21:43:08 ntpd[7144]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:43:08.968 INFO:teuthology.orchestra.run.smithi096.stderr:28 Nov 21:43:08 ntpd[7144]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:43:08.969 INFO:teuthology.orchestra.run.smithi096.stderr:28 Nov 21:43:08 ntpd[7144]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:43:08.969 INFO:teuthology.orchestra.run.smithi096.stderr:28 Nov 21:43:08 ntpd[7144]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:43:09.968 INFO:teuthology.orchestra.run.smithi096.stderr:28 Nov 21:43:09 ntpd[7144]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:43:09.968 INFO:teuthology.orchestra.run.smithi096.stderr:28 Nov 21:43:09 ntpd[7144]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:43:09.968 INFO:teuthology.orchestra.run.smithi096.stderr:28 Nov 21:43:09 ntpd[7144]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:43:09.969 INFO:teuthology.orchestra.run.smithi096.stderr:28 Nov 21:43:09 ntpd[7144]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:43:10.969 INFO:teuthology.orchestra.run.smithi096.stderr:28 Nov 21:43:10 ntpd[7144]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:43:10.969 INFO:teuthology.orchestra.run.smithi096.stderr:28 Nov 21:43:10 ntpd[7144]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:43:10.969 INFO:teuthology.orchestra.run.smithi096.stderr:28 Nov 21:43:10 ntpd[7144]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:43:10.970 INFO:teuthology.orchestra.run.smithi096.stderr:28 Nov 21:43:10 ntpd[7144]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:43:11.968 INFO:teuthology.orchestra.run.smithi096.stderr:28 Nov 21:43:11 ntpd[7144]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:43:11.968 INFO:teuthology.orchestra.run.smithi096.stderr:28 Nov 21:43:11 ntpd[7144]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:43:11.969 INFO:teuthology.orchestra.run.smithi096.stderr:28 Nov 21:43:11 ntpd[7144]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:43:11.969 INFO:teuthology.orchestra.run.smithi096.stderr:28 Nov 21:43:11 ntpd[7144]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:43:12.969 INFO:teuthology.orchestra.run.smithi096.stderr:28 Nov 21:43:12 ntpd[7144]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:43:12.970 INFO:teuthology.orchestra.run.smithi096.stderr:28 Nov 21:43:12 ntpd[7144]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:43:12.970 INFO:teuthology.orchestra.run.smithi096.stderr:28 Nov 21:43:12 ntpd[7144]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:43:12.971 INFO:teuthology.orchestra.run.smithi096.stderr:28 Nov 21:43:12 ntpd[7144]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:43:12.971 INFO:teuthology.orchestra.run.smithi096.stderr:28 Nov 21:43:12 ntpd[7144]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:43:12.971 INFO:teuthology.orchestra.run.smithi096.stderr:28 Nov 21:43:12 ntpd[7144]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:43:12.971 INFO:teuthology.orchestra.run.smithi096.stderr:28 Nov 21:43:12 ntpd[7144]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:43:12.972 INFO:teuthology.orchestra.run.smithi096.stderr:28 Nov 21:43:12 ntpd[7144]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:43:12.972 INFO:teuthology.orchestra.run.smithi096.stderr:28 Nov 21:43:12 ntpd[7144]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:43:12.972 INFO:teuthology.orchestra.run.smithi096.stderr:28 Nov 21:43:12 ntpd[7144]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:43:12.972 INFO:teuthology.orchestra.run.smithi096.stderr:28 Nov 21:43:12 ntpd[7144]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:43:12.973 INFO:teuthology.orchestra.run.smithi096.stderr:28 Nov 21:43:12 ntpd[7144]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:43:13.968 INFO:teuthology.orchestra.run.smithi096.stderr:28 Nov 21:43:13 ntpd[7144]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:43:13.969 INFO:teuthology.orchestra.run.smithi096.stderr:28 Nov 21:43:13 ntpd[7144]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:43:13.969 INFO:teuthology.orchestra.run.smithi096.stderr:28 Nov 21:43:13 ntpd[7144]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:43:13.969 INFO:teuthology.orchestra.run.smithi096.stderr:28 Nov 21:43:13 ntpd[7144]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:43:14.968 INFO:teuthology.orchestra.run.smithi096.stderr:28 Nov 21:43:14 ntpd[7144]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:43:14.969 INFO:teuthology.orchestra.run.smithi096.stderr:28 Nov 21:43:14 ntpd[7144]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:43:14.969 INFO:teuthology.orchestra.run.smithi096.stderr:28 Nov 21:43:14 ntpd[7144]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:43:14.969 INFO:teuthology.orchestra.run.smithi096.stderr:28 Nov 21:43:14 ntpd[7144]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:43:14.969 INFO:teuthology.orchestra.run.smithi096.stderr:28 Nov 21:43:14 ntpd[7144]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-28T21:43:14.970 INFO:teuthology.orchestra.run.smithi096.stderr:28 Nov 21:43:14 ntpd[7144]: can't open /var/log/ntpstats/rawstats.20201128: Permission denied 2020-11-28T21:43:14.970 INFO:teuthology.orchestra.run.smithi096.stderr:28 Nov 21:43:14 ntpd[7144]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-28T21:43:14.970 INFO:teuthology.orchestra.run.smithi096.stderr:28 Nov 21:43:14 ntpd[7144]: can't open /var/log/ntpstats/peerstats.20201128: Permission denied 2020-11-28T21:43:14.970 INFO:teuthology.orchestra.run.smithi096.stderr:28 Nov 21:43:14 ntpd[7144]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2020-11-28T21:43:14.971 INFO:teuthology.orchestra.run.smithi096.stderr:28 Nov 21:43:14 ntpd[7144]: can't open /var/log/ntpstats/loopstats.20201128: Permission denied 2020-11-28T21:43:14.971 INFO:teuthology.orchestra.run.smithi096.stdout:28 Nov 21:43:14 ntpd[7144]: ntpd: time slew -0.000727 s 2020-11-28T21:43:14.971 INFO:teuthology.orchestra.run.smithi096.stdout:ntpd: time slew -0.000727s 2020-11-28T21:43:15.021 INFO:teuthology.orchestra.run.smithi096.stdout: remote refid st t when poll reach delay offset jitter 2020-11-28T21:43:15.021 INFO:teuthology.orchestra.run.smithi096.stdout:============================================================================== 2020-11-28T21:43:15.021 INFO:teuthology.orchestra.run.smithi096.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-28T21:43:15.022 INFO:teuthology.orchestra.run.smithi096.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-28T21:43:15.022 INFO:teuthology.orchestra.run.smithi096.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-28T21:43:15.022 INFO:teuthology.orchestra.run.smithi096.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-28T21:43:15.023 INFO:teuthology.run_tasks:Running task install... 2020-11-28T21:43:15.356 DEBUG:teuthology.task.install:project ceph 2020-11-28T21:43:15.356 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'sha1': 'a83a3572a957356977342f461361c1cd3ec06dbd'}} 2020-11-28T21:43:15.357 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'sha1': 'a83a3572a957356977342f461361c1cd3ec06dbd'} 2020-11-28T21:43:15.357 INFO:teuthology.task.install:Using flavor: basic 2020-11-28T21:43:15.363 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2020-11-28T21:43:15.364 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']} 2020-11-28T21:43:15.364 INFO:teuthology.orchestra.run.smithi162:> sudo lsb_release -is 2020-11-28T21:43:15.421 INFO:teuthology.orchestra.run.smithi162.stdout:Ubuntu 2020-11-28T21:43:15.427 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-28T21:43:15.428 INFO:teuthology.orchestra.run.smithi118:> sudo lsb_release -is 2020-11-28T21:43:15.429 INFO:teuthology.orchestra.run.smithi162:> sudo apt-key list | grep Ceph 2020-11-28T21:43:15.486 INFO:teuthology.orchestra.run.smithi118.stdout:Ubuntu 2020-11-28T21:43:15.493 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-28T21:43:15.494 INFO:teuthology.orchestra.run.smithi192:> sudo lsb_release -is 2020-11-28T21:43:15.495 INFO:teuthology.orchestra.run.smithi118:> sudo apt-key list | grep Ceph 2020-11-28T21:43:15.516 INFO:teuthology.orchestra.run.smithi162.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2020-11-28T21:43:15.549 INFO:teuthology.orchestra.run.smithi192.stdout:Ubuntu 2020-11-28T21:43:15.557 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-28T21:43:15.557 INFO:teuthology.orchestra.run.smithi096:> sudo lsb_release -is 2020-11-28T21:43:15.558 INFO:teuthology.orchestra.run.smithi192:> sudo apt-key list | grep Ceph 2020-11-28T21:43:15.570 INFO:teuthology.orchestra.run.smithi162.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2020-11-28T21:43:15.570 INFO:teuthology.orchestra.run.smithi162.stdout:uid [ unknown] Ceph.com (release key) 2020-11-28T21:43:15.583 INFO:teuthology.orchestra.run.smithi162:> sudo lsb_release -is 2020-11-28T21:43:15.589 INFO:teuthology.orchestra.run.smithi118.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2020-11-28T21:43:15.608 INFO:teuthology.orchestra.run.smithi096.stdout:Ubuntu 2020-11-28T21:43:15.617 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-28T21:43:15.618 INFO:teuthology.orchestra.run.smithi096:> sudo apt-key list | grep Ceph 2020-11-28T21:43:15.641 INFO:teuthology.orchestra.run.smithi162.stdout:Ubuntu 2020-11-28T21:43:15.645 INFO:teuthology.orchestra.run.smithi118.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2020-11-28T21:43:15.645 INFO:teuthology.orchestra.run.smithi118.stdout:uid [ unknown] Ceph.com (release key) 2020-11-28T21:43:15.647 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-28T21:43:15.647 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2020-11-28T21:43:15.648 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2020-11-28T21:43:15.649 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=a83a3572a957356977342f461361c1cd3ec06dbd 2020-11-28T21:43:15.653 INFO:teuthology.orchestra.run.smithi118:> sudo lsb_release -is 2020-11-28T21:43:15.658 INFO:teuthology.orchestra.run.smithi192.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2020-11-28T21:43:15.703 INFO:teuthology.orchestra.run.smithi118.stdout:Ubuntu 2020-11-28T21:43:15.710 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-28T21:43:15.712 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2020-11-28T21:43:15.713 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2020-11-28T21:43:15.715 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=a83a3572a957356977342f461361c1cd3ec06dbd 2020-11-28T21:43:15.720 INFO:teuthology.orchestra.run.smithi192.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2020-11-28T21:43:15.720 INFO:teuthology.orchestra.run.smithi192.stdout:uid [ unknown] Ceph.com (release key) 2020-11-28T21:43:15.721 INFO:teuthology.orchestra.run.smithi192:> sudo lsb_release -is 2020-11-28T21:43:15.762 INFO:teuthology.orchestra.run.smithi096.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2020-11-28T21:43:15.777 INFO:teuthology.orchestra.run.smithi192.stdout:Ubuntu 2020-11-28T21:43:15.783 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-28T21:43:15.783 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2020-11-28T21:43:15.783 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2020-11-28T21:43:15.784 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=a83a3572a957356977342f461361c1cd3ec06dbd 2020-11-28T21:43:15.828 INFO:teuthology.orchestra.run.smithi096.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2020-11-28T21:43:15.829 INFO:teuthology.orchestra.run.smithi096.stdout:uid [ unknown] Ceph.com (release key) 2020-11-28T21:43:15.829 INFO:teuthology.orchestra.run.smithi096:> sudo lsb_release -is 2020-11-28T21:43:15.850 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default/ 2020-11-28T21:43:15.851 INFO:teuthology.task.install.deb:Package version is 15.2.6-595-ga83a3572-1bionic 2020-11-28T21:43:15.886 INFO:teuthology.orchestra.run.smithi096.stdout:Ubuntu 2020-11-28T21:43:15.887 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-28T21:43:15.887 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2020-11-28T21:43:15.887 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2020-11-28T21:43:15.888 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=a83a3572a957356977342f461361c1cd3ec06dbd 2020-11-28T21:43:15.893 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default/ 2020-11-28T21:43:15.894 INFO:teuthology.task.install.deb:Package version is 15.2.6-595-ga83a3572-1bionic 2020-11-28T21:43:15.976 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default/ 2020-11-28T21:43:15.977 INFO:teuthology.task.install.deb:Package version is 15.2.6-595-ga83a3572-1bionic 2020-11-28T21:43:15.983 INFO:teuthology.orchestra.run.smithi162:> set -ex 2020-11-28T21:43:15.983 INFO:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-11-28T21:43:15.990 INFO:teuthology.orchestra.run.smithi162.stderr:+ sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-11-28T21:43:16.000 INFO:teuthology.orchestra.run.smithi162.stderr:0+1 records in 2020-11-28T21:43:16.000 INFO:teuthology.orchestra.run.smithi162.stderr:0+1 records out 2020-11-28T21:43:16.000 INFO:teuthology.orchestra.run.smithi162.stderr:143 bytes copied, 7.527e-05 s, 1.9 MB/s 2020-11-28T21:43:16.001 INFO:teuthology.orchestra.run.smithi162:> sudo apt-get update 2020-11-28T21:43:16.034 INFO:teuthology.orchestra.run.smithi118:> set -ex 2020-11-28T21:43:16.036 INFO:teuthology.orchestra.run.smithi118:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-11-28T21:43:16.044 INFO:teuthology.orchestra.run.smithi118.stderr:+ sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-11-28T21:43:16.052 INFO:teuthology.orchestra.run.smithi118.stderr:0+1 records in 2020-11-28T21:43:16.052 INFO:teuthology.orchestra.run.smithi118.stderr:0+1 records out 2020-11-28T21:43:16.052 INFO:teuthology.orchestra.run.smithi118.stderr:143 bytes copied, 4.264e-05 s, 3.4 MB/s 2020-11-28T21:43:16.053 INFO:teuthology.orchestra.run.smithi118:> sudo apt-get update 2020-11-28T21:43:16.059 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default/ 2020-11-28T21:43:16.060 INFO:teuthology.task.install.deb:Package version is 15.2.6-595-ga83a3572-1bionic 2020-11-28T21:43:16.106 INFO:teuthology.orchestra.run.smithi192:> set -ex 2020-11-28T21:43:16.107 INFO:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-11-28T21:43:16.112 INFO:teuthology.orchestra.run.smithi192.stderr:+ sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-11-28T21:43:16.120 INFO:teuthology.orchestra.run.smithi192.stderr:0+1 records in 2020-11-28T21:43:16.120 INFO:teuthology.orchestra.run.smithi192.stderr:0+1 records out 2020-11-28T21:43:16.121 INFO:teuthology.orchestra.run.smithi192.stderr:143 bytes copied, 6.9157e-05 s, 2.1 MB/s 2020-11-28T21:43:16.122 INFO:teuthology.orchestra.run.smithi192:> sudo apt-get update 2020-11-28T21:43:16.181 INFO:teuthology.orchestra.run.smithi096:> set -ex 2020-11-28T21:43:16.182 INFO:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-11-28T21:43:16.188 INFO:teuthology.orchestra.run.smithi096.stderr:+ sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-11-28T21:43:16.197 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records in 2020-11-28T21:43:16.197 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records out 2020-11-28T21:43:16.198 INFO:teuthology.orchestra.run.smithi096.stderr:143 bytes copied, 5.2038e-05 s, 2.7 MB/s 2020-11-28T21:43:16.199 INFO:teuthology.orchestra.run.smithi096:> sudo apt-get update 2020-11-28T21:43:16.268 INFO:teuthology.orchestra.run.smithi162.stdout:Ign:1 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic InRelease 2020-11-28T21:43:16.291 INFO:teuthology.orchestra.run.smithi162.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic Release [7,681 B] 2020-11-28T21:43:16.325 INFO:teuthology.orchestra.run.smithi162.stdout:Hit:3 http://security.ubuntu.com/ubuntu bionic-security InRelease 2020-11-28T21:43:16.325 INFO:teuthology.orchestra.run.smithi162.stdout:Hit:4 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-11-28T21:43:16.326 INFO:teuthology.orchestra.run.smithi162.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic Release.gpg 2020-11-28T21:43:16.330 INFO:teuthology.orchestra.run.smithi118.stdout:Ign:1 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic InRelease 2020-11-28T21:43:16.349 INFO:teuthology.orchestra.run.smithi162.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main i386 Packages [3,098 B] 2020-11-28T21:43:16.361 INFO:teuthology.orchestra.run.smithi118.stdout:Hit:2 http://security.ubuntu.com/ubuntu bionic-security InRelease 2020-11-28T21:43:16.362 INFO:teuthology.orchestra.run.smithi118.stdout:Hit:3 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-11-28T21:43:16.363 INFO:teuthology.orchestra.run.smithi118.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic Release [7,681 B] 2020-11-28T21:43:16.383 INFO:teuthology.orchestra.run.smithi162.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 Packages [15.8 kB] 2020-11-28T21:43:16.385 INFO:teuthology.orchestra.run.smithi118.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic Release.gpg 2020-11-28T21:43:16.396 INFO:teuthology.orchestra.run.smithi162.stdout:Hit:8 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2020-11-28T21:43:16.398 INFO:teuthology.orchestra.run.smithi192.stdout:Ign:1 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic InRelease 2020-11-28T21:43:16.409 INFO:teuthology.orchestra.run.smithi118.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 Packages [15.8 kB] 2020-11-28T21:43:16.421 INFO:teuthology.orchestra.run.smithi192.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic Release [7,681 B] 2020-11-28T21:43:16.429 INFO:teuthology.orchestra.run.smithi118.stdout:Hit:7 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2020-11-28T21:43:16.435 INFO:teuthology.orchestra.run.smithi192.stdout:Hit:3 http://security.ubuntu.com/ubuntu bionic-security InRelease 2020-11-28T21:43:16.435 INFO:teuthology.orchestra.run.smithi192.stdout:Hit:4 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-11-28T21:43:16.442 INFO:teuthology.orchestra.run.smithi118.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main i386 Packages [3,098 B] 2020-11-28T21:43:16.446 INFO:teuthology.orchestra.run.smithi192.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic Release.gpg 2020-11-28T21:43:16.469 INFO:teuthology.orchestra.run.smithi192.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main i386 Packages [3,098 B] 2020-11-28T21:43:16.474 INFO:teuthology.orchestra.run.smithi096.stdout:Ign:1 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic InRelease 2020-11-28T21:43:16.477 INFO:teuthology.orchestra.run.smithi162.stdout:Hit:9 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2020-11-28T21:43:16.501 INFO:teuthology.orchestra.run.smithi192.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 Packages [15.8 kB] 2020-11-28T21:43:16.502 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-11-28T21:43:16.502 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:3 http://security.ubuntu.com/ubuntu bionic-security InRelease 2020-11-28T21:43:16.502 INFO:teuthology.orchestra.run.smithi096.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic Release [7,681 B] 2020-11-28T21:43:16.503 INFO:teuthology.orchestra.run.smithi192.stdout:Hit:8 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2020-11-28T21:43:16.508 INFO:teuthology.orchestra.run.smithi118.stdout:Hit:9 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2020-11-28T21:43:16.526 INFO:teuthology.orchestra.run.smithi096.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic Release.gpg 2020-11-28T21:43:16.549 INFO:teuthology.orchestra.run.smithi096.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 Packages [15.8 kB] 2020-11-28T21:43:16.568 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:7 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2020-11-28T21:43:16.582 INFO:teuthology.orchestra.run.smithi096.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main i386 Packages [3,098 B] 2020-11-28T21:43:16.583 INFO:teuthology.orchestra.run.smithi192.stdout:Hit:9 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2020-11-28T21:43:16.646 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:9 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2020-11-28T21:43:16.941 INFO:teuthology.orchestra.run.smithi162.stdout:Fetched 26.6 kB in 1s (36.0 kB/s) 2020-11-28T21:43:16.964 INFO:teuthology.orchestra.run.smithi118.stdout:Fetched 26.6 kB in 1s (37.6 kB/s) 2020-11-28T21:43:17.031 INFO:teuthology.orchestra.run.smithi192.stdout:Fetched 26.6 kB in 1s (37.7 kB/s) 2020-11-28T21:43:17.093 INFO:teuthology.orchestra.run.smithi096.stdout:Fetched 26.6 kB in 1s (38.2 kB/s) 2020-11-28T21:43:18.176 INFO:teuthology.orchestra.run.smithi162.stdout:Reading package lists... 2020-11-28T21:43:18.186 INFO:teuthology.orchestra.run.smithi118.stdout:Reading package lists... 2020-11-28T21:43:18.223 INFO:teuthology.orchestra.run.smithi162:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.6-595-ga83a3572-1bionic cephadm=15.2.6-595-ga83a3572-1bionic ceph-mds=15.2.6-595-ga83a3572-1bionic ceph-mgr=15.2.6-595-ga83a3572-1bionic ceph-common=15.2.6-595-ga83a3572-1bionic ceph-fuse=15.2.6-595-ga83a3572-1bionic ceph-test=15.2.6-595-ga83a3572-1bionic radosgw=15.2.6-595-ga83a3572-1bionic python3-rados=15.2.6-595-ga83a3572-1bionic python3-rgw=15.2.6-595-ga83a3572-1bionic python3-cephfs=15.2.6-595-ga83a3572-1bionic python3-rbd=15.2.6-595-ga83a3572-1bionic libcephfs2=15.2.6-595-ga83a3572-1bionic libcephfs-dev=15.2.6-595-ga83a3572-1bionic librados2=15.2.6-595-ga83a3572-1bionic librbd1=15.2.6-595-ga83a3572-1bionic rbd-fuse=15.2.6-595-ga83a3572-1bionic python3-cephfs=15.2.6-595-ga83a3572-1bionic cephfs-shell=15.2.6-595-ga83a3572-1bionic 2020-11-28T21:43:18.230 INFO:teuthology.orchestra.run.smithi118:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.6-595-ga83a3572-1bionic cephadm=15.2.6-595-ga83a3572-1bionic ceph-mds=15.2.6-595-ga83a3572-1bionic ceph-mgr=15.2.6-595-ga83a3572-1bionic ceph-common=15.2.6-595-ga83a3572-1bionic ceph-fuse=15.2.6-595-ga83a3572-1bionic ceph-test=15.2.6-595-ga83a3572-1bionic radosgw=15.2.6-595-ga83a3572-1bionic python3-rados=15.2.6-595-ga83a3572-1bionic python3-rgw=15.2.6-595-ga83a3572-1bionic python3-cephfs=15.2.6-595-ga83a3572-1bionic python3-rbd=15.2.6-595-ga83a3572-1bionic libcephfs2=15.2.6-595-ga83a3572-1bionic libcephfs-dev=15.2.6-595-ga83a3572-1bionic librados2=15.2.6-595-ga83a3572-1bionic librbd1=15.2.6-595-ga83a3572-1bionic rbd-fuse=15.2.6-595-ga83a3572-1bionic python3-cephfs=15.2.6-595-ga83a3572-1bionic cephfs-shell=15.2.6-595-ga83a3572-1bionic 2020-11-28T21:43:18.259 INFO:teuthology.orchestra.run.smithi192.stdout:Reading package lists... 2020-11-28T21:43:18.305 INFO:teuthology.orchestra.run.smithi162.stdout:Reading package lists... 2020-11-28T21:43:18.307 INFO:teuthology.orchestra.run.smithi192:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.6-595-ga83a3572-1bionic cephadm=15.2.6-595-ga83a3572-1bionic ceph-mds=15.2.6-595-ga83a3572-1bionic ceph-mgr=15.2.6-595-ga83a3572-1bionic ceph-common=15.2.6-595-ga83a3572-1bionic ceph-fuse=15.2.6-595-ga83a3572-1bionic ceph-test=15.2.6-595-ga83a3572-1bionic radosgw=15.2.6-595-ga83a3572-1bionic python3-rados=15.2.6-595-ga83a3572-1bionic python3-rgw=15.2.6-595-ga83a3572-1bionic python3-cephfs=15.2.6-595-ga83a3572-1bionic python3-rbd=15.2.6-595-ga83a3572-1bionic libcephfs2=15.2.6-595-ga83a3572-1bionic libcephfs-dev=15.2.6-595-ga83a3572-1bionic librados2=15.2.6-595-ga83a3572-1bionic librbd1=15.2.6-595-ga83a3572-1bionic rbd-fuse=15.2.6-595-ga83a3572-1bionic python3-cephfs=15.2.6-595-ga83a3572-1bionic cephfs-shell=15.2.6-595-ga83a3572-1bionic 2020-11-28T21:43:18.309 INFO:teuthology.orchestra.run.smithi118.stdout:Reading package lists... 2020-11-28T21:43:18.322 INFO:teuthology.orchestra.run.smithi096.stdout:Reading package lists... 2020-11-28T21:43:18.369 INFO:teuthology.orchestra.run.smithi096:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.6-595-ga83a3572-1bionic cephadm=15.2.6-595-ga83a3572-1bionic ceph-mds=15.2.6-595-ga83a3572-1bionic ceph-mgr=15.2.6-595-ga83a3572-1bionic ceph-common=15.2.6-595-ga83a3572-1bionic ceph-fuse=15.2.6-595-ga83a3572-1bionic ceph-test=15.2.6-595-ga83a3572-1bionic radosgw=15.2.6-595-ga83a3572-1bionic python3-rados=15.2.6-595-ga83a3572-1bionic python3-rgw=15.2.6-595-ga83a3572-1bionic python3-cephfs=15.2.6-595-ga83a3572-1bionic python3-rbd=15.2.6-595-ga83a3572-1bionic libcephfs2=15.2.6-595-ga83a3572-1bionic libcephfs-dev=15.2.6-595-ga83a3572-1bionic librados2=15.2.6-595-ga83a3572-1bionic librbd1=15.2.6-595-ga83a3572-1bionic rbd-fuse=15.2.6-595-ga83a3572-1bionic python3-cephfs=15.2.6-595-ga83a3572-1bionic cephfs-shell=15.2.6-595-ga83a3572-1bionic 2020-11-28T21:43:18.383 INFO:teuthology.orchestra.run.smithi192.stdout:Reading package lists... 2020-11-28T21:43:18.455 INFO:teuthology.orchestra.run.smithi096.stdout:Reading package lists... 2020-11-28T21:43:18.549 INFO:teuthology.orchestra.run.smithi118.stdout:Building dependency tree... 2020-11-28T21:43:18.550 INFO:teuthology.orchestra.run.smithi118.stdout:Reading state information... 2020-11-28T21:43:18.554 INFO:teuthology.orchestra.run.smithi162.stdout:Building dependency tree... 2020-11-28T21:43:18.555 INFO:teuthology.orchestra.run.smithi162.stdout:Reading state information... 2020-11-28T21:43:18.640 INFO:teuthology.orchestra.run.smithi192.stdout:Building dependency tree... 2020-11-28T21:43:18.641 INFO:teuthology.orchestra.run.smithi192.stdout:Reading state information... 2020-11-28T21:43:18.711 INFO:teuthology.orchestra.run.smithi096.stdout:Building dependency tree... 2020-11-28T21:43:18.712 INFO:teuthology.orchestra.run.smithi118.stdout:The following additional packages will be installed: 2020-11-28T21:43:18.712 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2020-11-28T21:43:18.712 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2020-11-28T21:43:18.713 INFO:teuthology.orchestra.run.smithi096.stdout:Reading state information... 2020-11-28T21:43:18.714 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-rook ceph-mon ceph-osd jq libjq1 libleveldb1v5 liblttng-ust0 2020-11-28T21:43:18.715 INFO:teuthology.orchestra.run.smithi118.stdout: liboath0 libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2020-11-28T21:43:18.715 INFO:teuthology.orchestra.run.smithi118.stdout: libsnappy1v5 libxslt1.1 nvme-cli python-attr python-funcsigs 2020-11-28T21:43:18.715 INFO:teuthology.orchestra.run.smithi118.stdout: python-pastedeploy-tpl python-pluggy python-py python-pytest 2020-11-28T21:43:18.716 INFO:teuthology.orchestra.run.smithi118.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2020-11-28T21:43:18.716 INFO:teuthology.orchestra.run.smithi118.stdout: python3-ceph-argparse python3-ceph-common python3-cffi-backend 2020-11-28T21:43:18.716 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2020-11-28T21:43:18.716 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cycler python3-dateutil python3-decorator python3-fasteners 2020-11-28T21:43:18.717 INFO:teuthology.orchestra.run.smithi118.stdout: python3-html5lib python3-httplib2 python3-jinja2 python3-joblib 2020-11-28T21:43:18.717 INFO:teuthology.orchestra.run.smithi118.stdout: python3-json-pointer python3-jsonpatch python3-jwt python3-kubernetes 2020-11-28T21:43:18.717 INFO:teuthology.orchestra.run.smithi118.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2020-11-28T21:43:18.717 INFO:teuthology.orchestra.run.smithi118.stdout: python3-matplotlib python3-monotonic python3-numpy python3-oauth2client 2020-11-28T21:43:18.718 INFO:teuthology.orchestra.run.smithi118.stdout: python3-olefile python3-openssl python3-paste python3-pastedeploy 2020-11-28T21:43:18.718 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pastescript python3-pecan python3-pil python3-pluggy 2020-11-28T21:43:18.718 INFO:teuthology.orchestra.run.smithi118.stdout: python3-prettytable python3-py python3-pyasn1 python3-pyasn1-modules 2020-11-28T21:43:18.718 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pyinotify python3-pyparsing python3-pyperclip python3-pytest 2020-11-28T21:43:18.718 INFO:teuthology.orchestra.run.smithi118.stdout: python3-repoze.lru python3-routes python3-rsa python3-scipy 2020-11-28T21:43:18.719 INFO:teuthology.orchestra.run.smithi118.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2020-11-28T21:43:18.719 INFO:teuthology.orchestra.run.smithi118.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2020-11-28T21:43:18.719 INFO:teuthology.orchestra.run.smithi118.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2020-11-28T21:43:18.719 INFO:teuthology.orchestra.run.smithi118.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2020-11-28T21:43:18.719 INFO:teuthology.orchestra.run.smithi118.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2020-11-28T21:43:18.721 INFO:teuthology.orchestra.run.smithi162.stdout:The following additional packages will be installed: 2020-11-28T21:43:18.721 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2020-11-28T21:43:18.721 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2020-11-28T21:43:18.722 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-rook ceph-mon ceph-osd jq libjq1 libleveldb1v5 liblttng-ust0 2020-11-28T21:43:18.722 INFO:teuthology.orchestra.run.smithi162.stdout: liboath0 libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2020-11-28T21:43:18.722 INFO:teuthology.orchestra.run.smithi162.stdout: libsnappy1v5 libxslt1.1 nvme-cli python-attr python-funcsigs 2020-11-28T21:43:18.722 INFO:teuthology.orchestra.run.smithi162.stdout: python-pastedeploy-tpl python-pluggy python-py python-pytest 2020-11-28T21:43:18.722 INFO:teuthology.orchestra.run.smithi162.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2020-11-28T21:43:18.723 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-argparse python3-ceph-common python3-cffi-backend 2020-11-28T21:43:18.723 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2020-11-28T21:43:18.723 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cycler python3-dateutil python3-decorator python3-fasteners 2020-11-28T21:43:18.723 INFO:teuthology.orchestra.run.smithi162.stdout: python3-html5lib python3-httplib2 python3-jinja2 python3-joblib 2020-11-28T21:43:18.723 INFO:teuthology.orchestra.run.smithi162.stdout: python3-json-pointer python3-jsonpatch python3-jwt python3-kubernetes 2020-11-28T21:43:18.724 INFO:teuthology.orchestra.run.smithi162.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2020-11-28T21:43:18.724 INFO:teuthology.orchestra.run.smithi162.stdout: python3-matplotlib python3-monotonic python3-numpy python3-oauth2client 2020-11-28T21:43:18.724 INFO:teuthology.orchestra.run.smithi162.stdout: python3-olefile python3-openssl python3-paste python3-pastedeploy 2020-11-28T21:43:18.724 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pastescript python3-pecan python3-pil python3-pluggy 2020-11-28T21:43:18.724 INFO:teuthology.orchestra.run.smithi162.stdout: python3-prettytable python3-py python3-pyasn1 python3-pyasn1-modules 2020-11-28T21:43:18.725 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyinotify python3-pyparsing python3-pyperclip python3-pytest 2020-11-28T21:43:18.725 INFO:teuthology.orchestra.run.smithi162.stdout: python3-repoze.lru python3-routes python3-rsa python3-scipy 2020-11-28T21:43:18.726 INFO:teuthology.orchestra.run.smithi162.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2020-11-28T21:43:18.726 INFO:teuthology.orchestra.run.smithi162.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2020-11-28T21:43:18.726 INFO:teuthology.orchestra.run.smithi162.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2020-11-28T21:43:18.726 INFO:teuthology.orchestra.run.smithi162.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2020-11-28T21:43:18.726 INFO:teuthology.orchestra.run.smithi162.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2020-11-28T21:43:18.727 INFO:teuthology.orchestra.run.smithi162.stdout:Suggested packages: 2020-11-28T21:43:18.727 INFO:teuthology.orchestra.run.smithi162.stdout: python3-influxdb python-attr-doc python-funcsigs-doc subversion 2020-11-28T21:43:18.727 INFO:teuthology.orchestra.run.smithi162.stdout: python-pytest-xdist python-mock python-cryptography-doc 2020-11-28T21:43:18.727 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cryptography-vectors python-cycler-doc python3-genshi 2020-11-28T21:43:18.727 INFO:teuthology.orchestra.run.smithi162.stdout: python-jinja2-doc python3-crypto python3-lxml-dbg python-lxml-doc 2020-11-28T21:43:18.728 INFO:teuthology.orchestra.run.smithi162.stdout: python3-beaker python-mako-doc dvipng ffmpeg gir1.2-gtk-3.0 ghostscript 2020-11-28T21:43:18.728 INFO:teuthology.orchestra.run.smithi162.stdout: inkscape ipython3 python-matplotlib-doc python3-cairocffi python3-gi-cairo 2020-11-28T21:43:18.728 INFO:teuthology.orchestra.run.smithi162.stdout: python3-gobject python3-pyqt4 python3-sip python3-tornado 2020-11-28T21:43:18.728 INFO:teuthology.orchestra.run.smithi162.stdout: texlive-extra-utils texlive-latex-extra ttf-staypuft python-numpy-doc 2020-11-28T21:43:18.729 INFO:teuthology.orchestra.run.smithi162.stdout: python3-dev python3-numpy-dbg python-openssl-doc python3-openssl-dbg 2020-11-28T21:43:18.729 INFO:teuthology.orchestra.run.smithi162.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2020-11-28T21:43:18.729 INFO:teuthology.orchestra.run.smithi162.stdout: python-pecan-doc python-pil-doc python3-pil-dbg python-pyinotify-doc 2020-11-28T21:43:18.729 INFO:teuthology.orchestra.run.smithi162.stdout: python-pyparsing-doc python-scipy-doc python-setuptools-doc python3-dap 2020-11-28T21:43:18.729 INFO:teuthology.orchestra.run.smithi162.stdout: python-sklearn-doc python-sqlalchemy-doc python3-psycopg2 python3-pymysql 2020-11-28T21:43:18.730 INFO:teuthology.orchestra.run.smithi162.stdout: python3-fdb tix python3-tk-dbg python-waitress-doc python-webob-doc 2020-11-28T21:43:18.730 INFO:teuthology.orchestra.run.smithi162.stdout: python-webtest-doc python3-termcolor python3-watchdog python-werkzeug-doc 2020-11-28T21:43:18.730 INFO:teuthology.orchestra.run.smithi118.stdout:Suggested packages: 2020-11-28T21:43:18.730 INFO:teuthology.orchestra.run.smithi118.stdout: python3-influxdb python-attr-doc python-funcsigs-doc subversion 2020-11-28T21:43:18.731 INFO:teuthology.orchestra.run.smithi118.stdout: python-pytest-xdist python-mock python-cryptography-doc 2020-11-28T21:43:18.731 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cryptography-vectors python-cycler-doc python3-genshi 2020-11-28T21:43:18.731 INFO:teuthology.orchestra.run.smithi118.stdout: python-jinja2-doc python3-crypto python3-lxml-dbg python-lxml-doc 2020-11-28T21:43:18.731 INFO:teuthology.orchestra.run.smithi118.stdout: python3-beaker python-mako-doc dvipng ffmpeg gir1.2-gtk-3.0 ghostscript 2020-11-28T21:43:18.732 INFO:teuthology.orchestra.run.smithi118.stdout: inkscape ipython3 python-matplotlib-doc python3-cairocffi python3-gi-cairo 2020-11-28T21:43:18.732 INFO:teuthology.orchestra.run.smithi118.stdout: python3-gobject python3-pyqt4 python3-sip python3-tornado 2020-11-28T21:43:18.732 INFO:teuthology.orchestra.run.smithi118.stdout: texlive-extra-utils texlive-latex-extra ttf-staypuft python-numpy-doc 2020-11-28T21:43:18.732 INFO:teuthology.orchestra.run.smithi118.stdout: python3-dev python3-numpy-dbg python-openssl-doc python3-openssl-dbg 2020-11-28T21:43:18.732 INFO:teuthology.orchestra.run.smithi118.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2020-11-28T21:43:18.732 INFO:teuthology.orchestra.run.smithi118.stdout: python-pecan-doc python-pil-doc python3-pil-dbg python-pyinotify-doc 2020-11-28T21:43:18.733 INFO:teuthology.orchestra.run.smithi118.stdout: python-pyparsing-doc python-scipy-doc python-setuptools-doc python3-dap 2020-11-28T21:43:18.733 INFO:teuthology.orchestra.run.smithi118.stdout: python-sklearn-doc python-sqlalchemy-doc python3-psycopg2 python3-pymysql 2020-11-28T21:43:18.733 INFO:teuthology.orchestra.run.smithi118.stdout: python3-fdb tix python3-tk-dbg python-waitress-doc python-webob-doc 2020-11-28T21:43:18.733 INFO:teuthology.orchestra.run.smithi118.stdout: python-webtest-doc python3-termcolor python3-watchdog python-werkzeug-doc 2020-11-28T21:43:18.807 INFO:teuthology.orchestra.run.smithi192.stdout:The following additional packages will be installed: 2020-11-28T21:43:18.807 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2020-11-28T21:43:18.808 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2020-11-28T21:43:18.809 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook ceph-mon ceph-osd jq libjq1 libleveldb1v5 liblttng-ust0 2020-11-28T21:43:18.809 INFO:teuthology.orchestra.run.smithi192.stdout: liboath0 libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2020-11-28T21:43:18.810 INFO:teuthology.orchestra.run.smithi192.stdout: libsnappy1v5 libxslt1.1 nvme-cli python-attr python-funcsigs 2020-11-28T21:43:18.810 INFO:teuthology.orchestra.run.smithi192.stdout: python-pastedeploy-tpl python-pluggy python-py python-pytest 2020-11-28T21:43:18.810 INFO:teuthology.orchestra.run.smithi192.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2020-11-28T21:43:18.810 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse python3-ceph-common python3-cffi-backend 2020-11-28T21:43:18.810 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2020-11-28T21:43:18.811 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cycler python3-dateutil python3-decorator python3-fasteners 2020-11-28T21:43:18.811 INFO:teuthology.orchestra.run.smithi192.stdout: python3-html5lib python3-httplib2 python3-jinja2 python3-joblib 2020-11-28T21:43:18.811 INFO:teuthology.orchestra.run.smithi192.stdout: python3-json-pointer python3-jsonpatch python3-jwt python3-kubernetes 2020-11-28T21:43:18.811 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2020-11-28T21:43:18.811 INFO:teuthology.orchestra.run.smithi192.stdout: python3-matplotlib python3-monotonic python3-numpy python3-oauth2client 2020-11-28T21:43:18.812 INFO:teuthology.orchestra.run.smithi192.stdout: python3-olefile python3-openssl python3-paste python3-pastedeploy 2020-11-28T21:43:18.812 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pastescript python3-pecan python3-pil python3-pluggy 2020-11-28T21:43:18.812 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable python3-py python3-pyasn1 python3-pyasn1-modules 2020-11-28T21:43:18.812 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyinotify python3-pyparsing python3-pyperclip python3-pytest 2020-11-28T21:43:18.812 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze.lru python3-routes python3-rsa python3-scipy 2020-11-28T21:43:18.813 INFO:teuthology.orchestra.run.smithi192.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2020-11-28T21:43:18.813 INFO:teuthology.orchestra.run.smithi192.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2020-11-28T21:43:18.813 INFO:teuthology.orchestra.run.smithi192.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2020-11-28T21:43:18.813 INFO:teuthology.orchestra.run.smithi192.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2020-11-28T21:43:18.813 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2020-11-28T21:43:18.814 INFO:teuthology.orchestra.run.smithi192.stdout:Suggested packages: 2020-11-28T21:43:18.814 INFO:teuthology.orchestra.run.smithi192.stdout: python3-influxdb python-attr-doc python-funcsigs-doc subversion 2020-11-28T21:43:18.815 INFO:teuthology.orchestra.run.smithi192.stdout: python-pytest-xdist python-mock python-cryptography-doc 2020-11-28T21:43:18.815 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography-vectors python-cycler-doc python3-genshi 2020-11-28T21:43:18.815 INFO:teuthology.orchestra.run.smithi192.stdout: python-jinja2-doc python3-crypto python3-lxml-dbg python-lxml-doc 2020-11-28T21:43:18.815 INFO:teuthology.orchestra.run.smithi192.stdout: python3-beaker python-mako-doc dvipng ffmpeg gir1.2-gtk-3.0 ghostscript 2020-11-28T21:43:18.815 INFO:teuthology.orchestra.run.smithi192.stdout: inkscape ipython3 python-matplotlib-doc python3-cairocffi python3-gi-cairo 2020-11-28T21:43:18.816 INFO:teuthology.orchestra.run.smithi192.stdout: python3-gobject python3-pyqt4 python3-sip python3-tornado 2020-11-28T21:43:18.816 INFO:teuthology.orchestra.run.smithi192.stdout: texlive-extra-utils texlive-latex-extra ttf-staypuft python-numpy-doc 2020-11-28T21:43:18.816 INFO:teuthology.orchestra.run.smithi192.stdout: python3-dev python3-numpy-dbg python-openssl-doc python3-openssl-dbg 2020-11-28T21:43:18.816 INFO:teuthology.orchestra.run.smithi192.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2020-11-28T21:43:18.816 INFO:teuthology.orchestra.run.smithi192.stdout: python-pecan-doc python-pil-doc python3-pil-dbg python-pyinotify-doc 2020-11-28T21:43:18.817 INFO:teuthology.orchestra.run.smithi192.stdout: python-pyparsing-doc python-scipy-doc python-setuptools-doc python3-dap 2020-11-28T21:43:18.817 INFO:teuthology.orchestra.run.smithi192.stdout: python-sklearn-doc python-sqlalchemy-doc python3-psycopg2 python3-pymysql 2020-11-28T21:43:18.817 INFO:teuthology.orchestra.run.smithi192.stdout: python3-fdb tix python3-tk-dbg python-waitress-doc python-webob-doc 2020-11-28T21:43:18.817 INFO:teuthology.orchestra.run.smithi192.stdout: python-webtest-doc python3-termcolor python3-watchdog python-werkzeug-doc 2020-11-28T21:43:18.819 INFO:teuthology.orchestra.run.smithi118.stdout:The following NEW packages will be installed: 2020-11-28T21:43:18.820 INFO:teuthology.orchestra.run.smithi118.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2020-11-28T21:43:18.820 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2020-11-28T21:43:18.820 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2020-11-28T21:43:18.822 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-rook ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq 2020-11-28T21:43:18.822 INFO:teuthology.orchestra.run.smithi118.stdout: libcephfs-dev libcephfs2 libjq1 libleveldb1v5 liblttng-ust0 liboath0 2020-11-28T21:43:18.822 INFO:teuthology.orchestra.run.smithi118.stdout: libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 libsnappy1v5 2020-11-28T21:43:18.822 INFO:teuthology.orchestra.run.smithi118.stdout: libxslt1.1 nvme-cli python-attr python-funcsigs python-pastedeploy-tpl 2020-11-28T21:43:18.823 INFO:teuthology.orchestra.run.smithi118.stdout: python-pluggy python-py python-pytest python3-asn1crypto python3-attr 2020-11-28T21:43:18.823 INFO:teuthology.orchestra.run.smithi118.stdout: python3-bcrypt python3-bs4 python3-ceph-argparse python3-ceph-common 2020-11-28T21:43:18.823 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cephfs python3-cffi-backend python3-cherrypy3 python3-cmd2 2020-11-28T21:43:18.823 INFO:teuthology.orchestra.run.smithi118.stdout: python3-colorama python3-cryptography python3-cycler python3-dateutil 2020-11-28T21:43:18.823 INFO:teuthology.orchestra.run.smithi118.stdout: python3-decorator python3-fasteners python3-html5lib python3-httplib2 2020-11-28T21:43:18.824 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jinja2 python3-joblib python3-json-pointer python3-jsonpatch 2020-11-28T21:43:18.824 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2020-11-28T21:43:18.824 INFO:teuthology.orchestra.run.smithi118.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2020-11-28T21:43:18.824 INFO:teuthology.orchestra.run.smithi118.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2020-11-28T21:43:18.824 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2020-11-28T21:43:18.825 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2020-11-28T21:43:18.825 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pyperclip 2020-11-28T21:43:18.825 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-rgw 2020-11-28T21:43:18.825 INFO:teuthology.orchestra.run.smithi118.stdout: python3-routes python3-rsa python3-scipy python3-setuptools 2020-11-28T21:43:18.825 INFO:teuthology.orchestra.run.smithi118.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2020-11-28T21:43:18.826 INFO:teuthology.orchestra.run.smithi118.stdout: python3-sklearn python3-sklearn-lib python3-sqlalchemy python3-tempita 2020-11-28T21:43:18.826 INFO:teuthology.orchestra.run.smithi118.stdout: python3-tk python3-tz python3-uritemplate python3-waitress 2020-11-28T21:43:18.826 INFO:teuthology.orchestra.run.smithi118.stdout: python3-webencodings python3-webob python3-websocket python3-webtest 2020-11-28T21:43:18.826 INFO:teuthology.orchestra.run.smithi118.stdout: python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2020-11-28T21:43:18.828 INFO:teuthology.orchestra.run.smithi162.stdout:The following NEW packages will be installed: 2020-11-28T21:43:18.828 INFO:teuthology.orchestra.run.smithi162.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2020-11-28T21:43:18.828 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2020-11-28T21:43:18.829 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2020-11-28T21:43:18.829 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-rook ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq 2020-11-28T21:43:18.829 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs-dev libcephfs2 libjq1 libleveldb1v5 liblttng-ust0 liboath0 2020-11-28T21:43:18.829 INFO:teuthology.orchestra.run.smithi162.stdout: libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 libsnappy1v5 2020-11-28T21:43:18.829 INFO:teuthology.orchestra.run.smithi162.stdout: libxslt1.1 nvme-cli python-attr python-funcsigs python-pastedeploy-tpl 2020-11-28T21:43:18.830 INFO:teuthology.orchestra.run.smithi162.stdout: python-pluggy python-py python-pytest python3-asn1crypto python3-attr 2020-11-28T21:43:18.830 INFO:teuthology.orchestra.run.smithi162.stdout: python3-bcrypt python3-bs4 python3-ceph-argparse python3-ceph-common 2020-11-28T21:43:18.830 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cephfs python3-cffi-backend python3-cherrypy3 python3-cmd2 2020-11-28T21:43:18.830 INFO:teuthology.orchestra.run.smithi162.stdout: python3-colorama python3-cryptography python3-cycler python3-dateutil 2020-11-28T21:43:18.830 INFO:teuthology.orchestra.run.smithi162.stdout: python3-decorator python3-fasteners python3-html5lib python3-httplib2 2020-11-28T21:43:18.831 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jinja2 python3-joblib python3-json-pointer python3-jsonpatch 2020-11-28T21:43:18.831 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2020-11-28T21:43:18.831 INFO:teuthology.orchestra.run.smithi162.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2020-11-28T21:43:18.831 INFO:teuthology.orchestra.run.smithi162.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2020-11-28T21:43:18.831 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2020-11-28T21:43:18.832 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2020-11-28T21:43:18.832 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pyperclip 2020-11-28T21:43:18.832 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-rgw 2020-11-28T21:43:18.832 INFO:teuthology.orchestra.run.smithi162.stdout: python3-routes python3-rsa python3-scipy python3-setuptools 2020-11-28T21:43:18.832 INFO:teuthology.orchestra.run.smithi162.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2020-11-28T21:43:18.833 INFO:teuthology.orchestra.run.smithi162.stdout: python3-sklearn python3-sklearn-lib python3-sqlalchemy python3-tempita 2020-11-28T21:43:18.833 INFO:teuthology.orchestra.run.smithi162.stdout: python3-tk python3-tz python3-uritemplate python3-waitress 2020-11-28T21:43:18.833 INFO:teuthology.orchestra.run.smithi162.stdout: python3-webencodings python3-webob python3-websocket python3-webtest 2020-11-28T21:43:18.833 INFO:teuthology.orchestra.run.smithi162.stdout: python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2020-11-28T21:43:18.833 INFO:teuthology.orchestra.run.smithi162.stdout:The following packages will be upgraded: 2020-11-28T21:43:18.834 INFO:teuthology.orchestra.run.smithi118.stdout:The following packages will be upgraded: 2020-11-28T21:43:18.834 INFO:teuthology.orchestra.run.smithi118.stdout: librados2 librbd1 2020-11-28T21:43:18.834 INFO:teuthology.orchestra.run.smithi162.stdout: librados2 librbd1 2020-11-28T21:43:18.876 INFO:teuthology.orchestra.run.smithi096.stdout:The following additional packages will be installed: 2020-11-28T21:43:18.876 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2020-11-28T21:43:18.876 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2020-11-28T21:43:18.878 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-rook ceph-mon ceph-osd jq libjq1 libleveldb1v5 liblttng-ust0 2020-11-28T21:43:18.878 INFO:teuthology.orchestra.run.smithi096.stdout: liboath0 libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2020-11-28T21:43:18.878 INFO:teuthology.orchestra.run.smithi096.stdout: libsnappy1v5 libxslt1.1 nvme-cli python-attr python-funcsigs 2020-11-28T21:43:18.878 INFO:teuthology.orchestra.run.smithi096.stdout: python-pastedeploy-tpl python-pluggy python-py python-pytest 2020-11-28T21:43:18.879 INFO:teuthology.orchestra.run.smithi096.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2020-11-28T21:43:18.879 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-argparse python3-ceph-common python3-cffi-backend 2020-11-28T21:43:18.879 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2020-11-28T21:43:18.879 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cycler python3-dateutil python3-decorator python3-fasteners 2020-11-28T21:43:18.879 INFO:teuthology.orchestra.run.smithi096.stdout: python3-html5lib python3-httplib2 python3-jinja2 python3-joblib 2020-11-28T21:43:18.880 INFO:teuthology.orchestra.run.smithi096.stdout: python3-json-pointer python3-jsonpatch python3-jwt python3-kubernetes 2020-11-28T21:43:18.880 INFO:teuthology.orchestra.run.smithi096.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2020-11-28T21:43:18.880 INFO:teuthology.orchestra.run.smithi096.stdout: python3-matplotlib python3-monotonic python3-numpy python3-oauth2client 2020-11-28T21:43:18.880 INFO:teuthology.orchestra.run.smithi096.stdout: python3-olefile python3-openssl python3-paste python3-pastedeploy 2020-11-28T21:43:18.880 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pastescript python3-pecan python3-pil python3-pluggy 2020-11-28T21:43:18.881 INFO:teuthology.orchestra.run.smithi096.stdout: python3-prettytable python3-py python3-pyasn1 python3-pyasn1-modules 2020-11-28T21:43:18.881 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyinotify python3-pyparsing python3-pyperclip python3-pytest 2020-11-28T21:43:18.881 INFO:teuthology.orchestra.run.smithi096.stdout: python3-repoze.lru python3-routes python3-rsa python3-scipy 2020-11-28T21:43:18.881 INFO:teuthology.orchestra.run.smithi096.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2020-11-28T21:43:18.881 INFO:teuthology.orchestra.run.smithi096.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2020-11-28T21:43:18.882 INFO:teuthology.orchestra.run.smithi096.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2020-11-28T21:43:18.882 INFO:teuthology.orchestra.run.smithi096.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2020-11-28T21:43:18.882 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2020-11-28T21:43:18.883 INFO:teuthology.orchestra.run.smithi096.stdout:Suggested packages: 2020-11-28T21:43:18.883 INFO:teuthology.orchestra.run.smithi096.stdout: python3-influxdb python-attr-doc python-funcsigs-doc subversion 2020-11-28T21:43:18.883 INFO:teuthology.orchestra.run.smithi096.stdout: python-pytest-xdist python-mock python-cryptography-doc 2020-11-28T21:43:18.883 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cryptography-vectors python-cycler-doc python3-genshi 2020-11-28T21:43:18.883 INFO:teuthology.orchestra.run.smithi096.stdout: python-jinja2-doc python3-crypto python3-lxml-dbg python-lxml-doc 2020-11-28T21:43:18.884 INFO:teuthology.orchestra.run.smithi096.stdout: python3-beaker python-mako-doc dvipng ffmpeg gir1.2-gtk-3.0 ghostscript 2020-11-28T21:43:18.884 INFO:teuthology.orchestra.run.smithi096.stdout: inkscape ipython3 python-matplotlib-doc python3-cairocffi python3-gi-cairo 2020-11-28T21:43:18.884 INFO:teuthology.orchestra.run.smithi096.stdout: python3-gobject python3-pyqt4 python3-sip python3-tornado 2020-11-28T21:43:18.884 INFO:teuthology.orchestra.run.smithi096.stdout: texlive-extra-utils texlive-latex-extra ttf-staypuft python-numpy-doc 2020-11-28T21:43:18.884 INFO:teuthology.orchestra.run.smithi096.stdout: python3-dev python3-numpy-dbg python-openssl-doc python3-openssl-dbg 2020-11-28T21:43:18.885 INFO:teuthology.orchestra.run.smithi096.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2020-11-28T21:43:18.885 INFO:teuthology.orchestra.run.smithi096.stdout: python-pecan-doc python-pil-doc python3-pil-dbg python-pyinotify-doc 2020-11-28T21:43:18.885 INFO:teuthology.orchestra.run.smithi096.stdout: python-pyparsing-doc python-scipy-doc python-setuptools-doc python3-dap 2020-11-28T21:43:18.885 INFO:teuthology.orchestra.run.smithi096.stdout: python-sklearn-doc python-sqlalchemy-doc python3-psycopg2 python3-pymysql 2020-11-28T21:43:18.885 INFO:teuthology.orchestra.run.smithi096.stdout: python3-fdb tix python3-tk-dbg python-waitress-doc python-webob-doc 2020-11-28T21:43:18.886 INFO:teuthology.orchestra.run.smithi096.stdout: python-webtest-doc python3-termcolor python3-watchdog python-werkzeug-doc 2020-11-28T21:43:18.915 INFO:teuthology.orchestra.run.smithi192.stdout:The following NEW packages will be installed: 2020-11-28T21:43:18.916 INFO:teuthology.orchestra.run.smithi192.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2020-11-28T21:43:18.916 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2020-11-28T21:43:18.917 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2020-11-28T21:43:18.917 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq 2020-11-28T21:43:18.921 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-dev libcephfs2 libjq1 libleveldb1v5 liblttng-ust0 liboath0 2020-11-28T21:43:18.921 INFO:teuthology.orchestra.run.smithi192.stdout: libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 libsnappy1v5 2020-11-28T21:43:18.921 INFO:teuthology.orchestra.run.smithi192.stdout: libxslt1.1 nvme-cli python-attr python-funcsigs python-pastedeploy-tpl 2020-11-28T21:43:18.922 INFO:teuthology.orchestra.run.smithi192.stdout: python-pluggy python-py python-pytest python3-asn1crypto python3-attr 2020-11-28T21:43:18.922 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt python3-bs4 python3-ceph-argparse python3-ceph-common 2020-11-28T21:43:18.923 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs python3-cffi-backend python3-cherrypy3 python3-cmd2 2020-11-28T21:43:18.923 INFO:teuthology.orchestra.run.smithi192.stdout: python3-colorama python3-cryptography python3-cycler python3-dateutil 2020-11-28T21:43:18.923 INFO:teuthology.orchestra.run.smithi192.stdout: python3-decorator python3-fasteners python3-html5lib python3-httplib2 2020-11-28T21:43:18.924 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jinja2 python3-joblib python3-json-pointer python3-jsonpatch 2020-11-28T21:43:18.924 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2020-11-28T21:43:18.925 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2020-11-28T21:43:18.925 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2020-11-28T21:43:18.925 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2020-11-28T21:43:18.926 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2020-11-28T21:43:18.926 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pyperclip 2020-11-28T21:43:18.926 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-rgw 2020-11-28T21:43:18.927 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes python3-rsa python3-scipy python3-setuptools 2020-11-28T21:43:18.927 INFO:teuthology.orchestra.run.smithi192.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2020-11-28T21:43:18.928 INFO:teuthology.orchestra.run.smithi192.stdout: python3-sklearn python3-sklearn-lib python3-sqlalchemy python3-tempita 2020-11-28T21:43:18.928 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tk python3-tz python3-uritemplate python3-waitress 2020-11-28T21:43:18.928 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webencodings python3-webob python3-websocket python3-webtest 2020-11-28T21:43:18.929 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2020-11-28T21:43:18.929 INFO:teuthology.orchestra.run.smithi192.stdout:The following packages will be upgraded: 2020-11-28T21:43:18.930 INFO:teuthology.orchestra.run.smithi192.stdout: librados2 librbd1 2020-11-28T21:43:18.984 INFO:teuthology.orchestra.run.smithi096.stdout:The following NEW packages will be installed: 2020-11-28T21:43:18.985 INFO:teuthology.orchestra.run.smithi096.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2020-11-28T21:43:18.985 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2020-11-28T21:43:18.985 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2020-11-28T21:43:18.990 INFO:teuthology.orchestra.run.smithi162.stdout:2 upgraded, 115 newly installed, 0 to remove and 5 not upgraded. 2020-11-28T21:43:18.990 INFO:teuthology.orchestra.run.smithi162.stdout:Need to get 156 MB of archives. 2020-11-28T21:43:18.991 INFO:teuthology.orchestra.run.smithi162.stdout:After this operation, 724 MB of additional disk space will be used. 2020-11-28T21:43:18.991 INFO:teuthology.orchestra.run.smithi162.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 15.2.6-595-ga83a3572-1bionic [1,634 kB] 2020-11-28T21:43:18.992 INFO:teuthology.orchestra.run.smithi118.stdout:2 upgraded, 115 newly installed, 0 to remove and 5 not upgraded. 2020-11-28T21:43:18.992 INFO:teuthology.orchestra.run.smithi118.stdout:Need to get 156 MB of archives. 2020-11-28T21:43:18.992 INFO:teuthology.orchestra.run.smithi118.stdout:After this operation, 724 MB of additional disk space will be used. 2020-11-28T21:43:18.993 INFO:teuthology.orchestra.run.smithi118.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 15.2.6-595-ga83a3572-1bionic [1,634 kB] 2020-11-28T21:43:18.993 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-rook ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq 2020-11-28T21:43:18.994 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs-dev libcephfs2 libjq1 libleveldb1v5 liblttng-ust0 liboath0 2020-11-28T21:43:18.994 INFO:teuthology.orchestra.run.smithi096.stdout: libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 libsnappy1v5 2020-11-28T21:43:18.994 INFO:teuthology.orchestra.run.smithi096.stdout: libxslt1.1 nvme-cli python-attr python-funcsigs python-pastedeploy-tpl 2020-11-28T21:43:18.995 INFO:teuthology.orchestra.run.smithi096.stdout: python-pluggy python-py python-pytest python3-asn1crypto python3-attr 2020-11-28T21:43:18.995 INFO:teuthology.orchestra.run.smithi096.stdout: python3-bcrypt python3-bs4 python3-ceph-argparse python3-ceph-common 2020-11-28T21:43:18.996 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cephfs python3-cffi-backend python3-cherrypy3 python3-cmd2 2020-11-28T21:43:18.996 INFO:teuthology.orchestra.run.smithi096.stdout: python3-colorama python3-cryptography python3-cycler python3-dateutil 2020-11-28T21:43:18.996 INFO:teuthology.orchestra.run.smithi096.stdout: python3-decorator python3-fasteners python3-html5lib python3-httplib2 2020-11-28T21:43:18.997 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jinja2 python3-joblib python3-json-pointer python3-jsonpatch 2020-11-28T21:43:18.997 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2020-11-28T21:43:18.997 INFO:teuthology.orchestra.run.smithi096.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2020-11-28T21:43:18.998 INFO:teuthology.orchestra.run.smithi096.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2020-11-28T21:43:18.998 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2020-11-28T21:43:18.999 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2020-11-28T21:43:18.999 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pyperclip 2020-11-28T21:43:18.999 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-rgw 2020-11-28T21:43:19.000 INFO:teuthology.orchestra.run.smithi096.stdout: python3-routes python3-rsa python3-scipy python3-setuptools 2020-11-28T21:43:19.000 INFO:teuthology.orchestra.run.smithi096.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2020-11-28T21:43:19.001 INFO:teuthology.orchestra.run.smithi096.stdout: python3-sklearn python3-sklearn-lib python3-sqlalchemy python3-tempita 2020-11-28T21:43:19.001 INFO:teuthology.orchestra.run.smithi096.stdout: python3-tk python3-tz python3-uritemplate python3-waitress 2020-11-28T21:43:19.001 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webencodings python3-webob python3-websocket python3-webtest 2020-11-28T21:43:19.002 INFO:teuthology.orchestra.run.smithi096.stdout: python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2020-11-28T21:43:19.002 INFO:teuthology.orchestra.run.smithi096.stdout:The following packages will be upgraded: 2020-11-28T21:43:19.002 INFO:teuthology.orchestra.run.smithi096.stdout: librados2 librbd1 2020-11-28T21:43:19.010 INFO:teuthology.orchestra.run.smithi118.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [52.2 kB] 2020-11-28T21:43:19.015 INFO:teuthology.orchestra.run.smithi162.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [52.2 kB] 2020-11-28T21:43:19.083 INFO:teuthology.orchestra.run.smithi192.stdout:2 upgraded, 115 newly installed, 0 to remove and 5 not upgraded. 2020-11-28T21:43:19.083 INFO:teuthology.orchestra.run.smithi192.stdout:Need to get 156 MB of archives. 2020-11-28T21:43:19.083 INFO:teuthology.orchestra.run.smithi192.stdout:After this operation, 724 MB of additional disk space will be used. 2020-11-28T21:43:19.084 INFO:teuthology.orchestra.run.smithi192.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 15.2.6-595-ga83a3572-1bionic [1,634 kB] 2020-11-28T21:43:19.106 INFO:teuthology.orchestra.run.smithi192.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [52.2 kB] 2020-11-28T21:43:19.162 INFO:teuthology.orchestra.run.smithi096.stdout:2 upgraded, 115 newly installed, 0 to remove and 5 not upgraded. 2020-11-28T21:43:19.163 INFO:teuthology.orchestra.run.smithi096.stdout:Need to get 156 MB of archives. 2020-11-28T21:43:19.163 INFO:teuthology.orchestra.run.smithi096.stdout:After this operation, 724 MB of additional disk space will be used. 2020-11-28T21:43:19.163 INFO:teuthology.orchestra.run.smithi096.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 15.2.6-595-ga83a3572-1bionic [1,634 kB] 2020-11-28T21:43:19.175 INFO:teuthology.orchestra.run.smithi096.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [52.2 kB] 2020-11-28T21:43:19.177 INFO:teuthology.orchestra.run.smithi118.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 15.2.6-595-ga83a3572-1bionic [3,056 kB] 2020-11-28T21:43:19.183 INFO:teuthology.orchestra.run.smithi162.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 15.2.6-595-ga83a3572-1bionic [3,056 kB] 2020-11-28T21:43:19.256 INFO:teuthology.orchestra.run.smithi118.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyperclip all 1.6.0-1 [9,376 B] 2020-11-28T21:43:19.257 INFO:teuthology.orchestra.run.smithi118.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cmd2 all 0.7.9-0ubuntu1 [34.2 kB] 2020-11-28T21:43:19.268 INFO:teuthology.orchestra.run.smithi162.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyperclip all 1.6.0-1 [9,376 B] 2020-11-28T21:43:19.277 INFO:teuthology.orchestra.run.smithi162.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cmd2 all 0.7.9-0ubuntu1 [34.2 kB] 2020-11-28T21:43:19.279 INFO:teuthology.orchestra.run.smithi118.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-uritemplate all 0.6-1ubuntu1 [5,126 B] 2020-11-28T21:43:19.280 INFO:teuthology.orchestra.run.smithi118.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2020-11-28T21:43:19.288 INFO:teuthology.orchestra.run.smithi162.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-uritemplate all 0.6-1ubuntu1 [5,126 B] 2020-11-28T21:43:19.288 INFO:teuthology.orchestra.run.smithi162.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2020-11-28T21:43:19.356 INFO:teuthology.orchestra.run.smithi192.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyperclip all 1.6.0-1 [9,376 B] 2020-11-28T21:43:19.364 INFO:teuthology.orchestra.run.smithi192.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cmd2 all 0.7.9-0ubuntu1 [34.2 kB] 2020-11-28T21:43:19.374 INFO:teuthology.orchestra.run.smithi192.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-uritemplate all 0.6-1ubuntu1 [5,126 B] 2020-11-28T21:43:19.375 INFO:teuthology.orchestra.run.smithi192.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2020-11-28T21:43:19.396 INFO:teuthology.orchestra.run.smithi118.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 15.2.6-595-ga83a3572-1bionic [435 kB] 2020-11-28T21:43:19.419 INFO:teuthology.orchestra.run.smithi118.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-prettytable all 0.7.2-3 [19.7 kB] 2020-11-28T21:43:19.420 INFO:teuthology.orchestra.run.smithi162.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 15.2.6-595-ga83a3572-1bionic [435 kB] 2020-11-28T21:43:19.429 INFO:teuthology.orchestra.run.smithi118.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 librabbitmq4 amd64 0.8.0-1ubuntu0.18.04.2 [33.9 kB] 2020-11-28T21:43:19.434 INFO:teuthology.orchestra.run.smithi162.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-prettytable all 0.7.2-3 [19.7 kB] 2020-11-28T21:43:19.435 INFO:teuthology.orchestra.run.smithi118.stdout:Get:11 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 python3-rados amd64 15.2.6-595-ga83a3572-1bionic [332 kB] 2020-11-28T21:43:19.437 INFO:teuthology.orchestra.run.smithi096.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyperclip all 1.6.0-1 [9,376 B] 2020-11-28T21:43:19.440 INFO:teuthology.orchestra.run.smithi118.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic/universe amd64 librdkafka1 amd64 0.11.3-1build1 [293 kB] 2020-11-28T21:43:19.444 INFO:teuthology.orchestra.run.smithi162.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 librabbitmq4 amd64 0.8.0-1ubuntu0.18.04.2 [33.9 kB] 2020-11-28T21:43:19.445 INFO:teuthology.orchestra.run.smithi096.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cmd2 all 0.7.9-0ubuntu1 [34.2 kB] 2020-11-28T21:43:19.446 INFO:teuthology.orchestra.run.smithi118.stdout:Get:13 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-argparse all 15.2.6-595-ga83a3572-1bionic [20.7 kB] 2020-11-28T21:43:19.447 INFO:teuthology.orchestra.run.smithi118.stdout:Get:14 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 python3-cephfs amd64 15.2.6-595-ga83a3572-1bionic [133 kB] 2020-11-28T21:43:19.454 INFO:teuthology.orchestra.run.smithi096.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-uritemplate all 0.6-1ubuntu1 [5,126 B] 2020-11-28T21:43:19.455 INFO:teuthology.orchestra.run.smithi096.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2020-11-28T21:43:19.456 INFO:teuthology.orchestra.run.smithi162.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/universe amd64 librdkafka1 amd64 0.11.3-1build1 [293 kB] 2020-11-28T21:43:19.460 INFO:teuthology.orchestra.run.smithi162.stdout:Get:12 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 python3-rados amd64 15.2.6-595-ga83a3572-1bionic [332 kB] 2020-11-28T21:43:19.478 INFO:teuthology.orchestra.run.smithi118.stdout:Get:15 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-common all 15.2.6-595-ga83a3572-1bionic [27.8 kB] 2020-11-28T21:43:19.479 INFO:teuthology.orchestra.run.smithi118.stdout:Get:16 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 python3-rbd amd64 15.2.6-595-ga83a3572-1bionic [312 kB] 2020-11-28T21:43:19.488 INFO:teuthology.orchestra.run.smithi118.stdout:Get:17 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 15.2.6-595-ga83a3572-1bionic [2,890 kB] 2020-11-28T21:43:19.499 INFO:teuthology.orchestra.run.smithi162.stdout:Get:13 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-argparse all 15.2.6-595-ga83a3572-1bionic [20.7 kB] 2020-11-28T21:43:19.500 INFO:teuthology.orchestra.run.smithi162.stdout:Get:14 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 python3-cephfs amd64 15.2.6-595-ga83a3572-1bionic [133 kB] 2020-11-28T21:43:19.504 INFO:teuthology.orchestra.run.smithi162.stdout:Get:15 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-common all 15.2.6-595-ga83a3572-1bionic [27.8 kB] 2020-11-28T21:43:19.506 INFO:teuthology.orchestra.run.smithi162.stdout:Get:16 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 python3-rbd amd64 15.2.6-595-ga83a3572-1bionic [312 kB] 2020-11-28T21:43:19.514 INFO:teuthology.orchestra.run.smithi192.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-prettytable all 0.7.2-3 [19.7 kB] 2020-11-28T21:43:19.515 INFO:teuthology.orchestra.run.smithi162.stdout:Get:17 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 15.2.6-595-ga83a3572-1bionic [2,890 kB] 2020-11-28T21:43:19.520 INFO:teuthology.orchestra.run.smithi118.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2020-11-28T21:43:19.523 INFO:teuthology.orchestra.run.smithi118.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2020-11-28T21:43:19.527 INFO:teuthology.orchestra.run.smithi192.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 librabbitmq4 amd64 0.8.0-1ubuntu0.18.04.2 [33.9 kB] 2020-11-28T21:43:19.535 INFO:teuthology.orchestra.run.smithi162.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2020-11-28T21:43:19.538 INFO:teuthology.orchestra.run.smithi162.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2020-11-28T21:43:19.540 INFO:teuthology.orchestra.run.smithi192.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic/universe amd64 librdkafka1 amd64 0.11.3-1build1 [293 kB] 2020-11-28T21:43:19.549 INFO:teuthology.orchestra.run.smithi118.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2020-11-28T21:43:19.555 INFO:teuthology.orchestra.run.smithi118.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-dateutil all 2.6.1-1 [52.3 kB] 2020-11-28T21:43:19.562 INFO:teuthology.orchestra.run.smithi162.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2020-11-28T21:43:19.564 INFO:teuthology.orchestra.run.smithi118.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-asn1crypto all 0.24.0-1 [72.8 kB] 2020-11-28T21:43:19.568 INFO:teuthology.orchestra.run.smithi162.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-dateutil all 2.6.1-1 [52.3 kB] 2020-11-28T21:43:19.575 INFO:teuthology.orchestra.run.smithi118.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cffi-backend amd64 1.11.5-1 [64.6 kB] 2020-11-28T21:43:19.576 INFO:teuthology.orchestra.run.smithi162.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-asn1crypto all 0.24.0-1 [72.8 kB] 2020-11-28T21:43:19.582 INFO:teuthology.orchestra.run.smithi118.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-cryptography amd64 2.1.4-1ubuntu1.4 [220 kB] 2020-11-28T21:43:19.586 INFO:teuthology.orchestra.run.smithi162.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cffi-backend amd64 1.11.5-1 [64.6 kB] 2020-11-28T21:43:19.594 INFO:teuthology.orchestra.run.smithi162.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-cryptography amd64 2.1.4-1ubuntu1.4 [220 kB] 2020-11-28T21:43:19.597 INFO:teuthology.orchestra.run.smithi096.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-prettytable all 0.7.2-3 [19.7 kB] 2020-11-28T21:43:19.605 INFO:teuthology.orchestra.run.smithi118.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-openssl all 17.5.0-1ubuntu1 [41.5 kB] 2020-11-28T21:43:19.606 INFO:teuthology.orchestra.run.smithi096.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 librabbitmq4 amd64 0.8.0-1ubuntu0.18.04.2 [33.9 kB] 2020-11-28T21:43:19.607 INFO:teuthology.orchestra.run.smithi118.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-bcrypt amd64 3.1.4-2 [29.9 kB] 2020-11-28T21:43:19.616 INFO:teuthology.orchestra.run.smithi162.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-openssl all 17.5.0-1ubuntu1 [41.5 kB] 2020-11-28T21:43:19.618 INFO:teuthology.orchestra.run.smithi162.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-bcrypt amd64 3.1.4-2 [29.9 kB] 2020-11-28T21:43:19.620 INFO:teuthology.orchestra.run.smithi096.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic/universe amd64 librdkafka1 amd64 0.11.3-1build1 [293 kB] 2020-11-28T21:43:19.621 INFO:teuthology.orchestra.run.smithi192.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2020-11-28T21:43:19.625 INFO:teuthology.orchestra.run.smithi192.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2020-11-28T21:43:19.650 INFO:teuthology.orchestra.run.smithi192.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2020-11-28T21:43:19.658 INFO:teuthology.orchestra.run.smithi192.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-dateutil all 2.6.1-1 [52.3 kB] 2020-11-28T21:43:19.659 INFO:teuthology.orchestra.run.smithi118.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cherrypy3 all 8.9.1-2 [160 kB] 2020-11-28T21:43:19.667 INFO:teuthology.orchestra.run.smithi118.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jwt all 1.5.3+ds1-1 [15.9 kB] 2020-11-28T21:43:19.667 INFO:teuthology.orchestra.run.smithi192.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-asn1crypto all 0.24.0-1 [72.8 kB] 2020-11-28T21:43:19.668 INFO:teuthology.orchestra.run.smithi118.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-logutils all 0.3.3-5 [16.7 kB] 2020-11-28T21:43:19.669 INFO:teuthology.orchestra.run.smithi118.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-markupsafe amd64 1.0-1build1 [13.5 kB] 2020-11-28T21:43:19.669 INFO:teuthology.orchestra.run.smithi118.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-mako all 1.0.7+ds1-1 [59.3 kB] 2020-11-28T21:43:19.672 INFO:teuthology.orchestra.run.smithi118.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-simplegeneric all 0.8.1-1 [11.5 kB] 2020-11-28T21:43:19.677 INFO:teuthology.orchestra.run.smithi192.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cffi-backend amd64 1.11.5-1 [64.6 kB] 2020-11-28T21:43:19.678 INFO:teuthology.orchestra.run.smithi162.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cherrypy3 all 8.9.1-2 [160 kB] 2020-11-28T21:43:19.684 INFO:teuthology.orchestra.run.smithi192.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-cryptography amd64 2.1.4-1ubuntu1.4 [220 kB] 2020-11-28T21:43:19.686 INFO:teuthology.orchestra.run.smithi162.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jwt all 1.5.3+ds1-1 [15.9 kB] 2020-11-28T21:43:19.687 INFO:teuthology.orchestra.run.smithi162.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-logutils all 0.3.3-5 [16.7 kB] 2020-11-28T21:43:19.688 INFO:teuthology.orchestra.run.smithi162.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-markupsafe amd64 1.0-1build1 [13.5 kB] 2020-11-28T21:43:19.688 INFO:teuthology.orchestra.run.smithi162.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-mako all 1.0.7+ds1-1 [59.3 kB] 2020-11-28T21:43:19.690 INFO:teuthology.orchestra.run.smithi162.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-simplegeneric all 0.8.1-1 [11.5 kB] 2020-11-28T21:43:19.690 INFO:teuthology.orchestra.run.smithi162.stdout:Get:33 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-singledispatch all 3.4.0.3-2 [7,022 B] 2020-11-28T21:43:19.693 INFO:teuthology.orchestra.run.smithi096.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2020-11-28T21:43:19.696 INFO:teuthology.orchestra.run.smithi192.stdout:Get:17 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 15.2.6-595-ga83a3572-1bionic [3,056 kB] 2020-11-28T21:43:19.697 INFO:teuthology.orchestra.run.smithi096.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2020-11-28T21:43:19.704 INFO:teuthology.orchestra.run.smithi118.stdout:Get:33 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 python3-rgw amd64 15.2.6-595-ga83a3572-1bionic [99.4 kB] 2020-11-28T21:43:19.707 INFO:teuthology.orchestra.run.smithi192.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-openssl all 17.5.0-1ubuntu1 [41.5 kB] 2020-11-28T21:43:19.711 INFO:teuthology.orchestra.run.smithi192.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-bcrypt amd64 3.1.4-2 [29.9 kB] 2020-11-28T21:43:19.720 INFO:teuthology.orchestra.run.smithi096.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2020-11-28T21:43:19.728 INFO:teuthology.orchestra.run.smithi096.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-dateutil all 2.6.1-1 [52.3 kB] 2020-11-28T21:43:19.737 INFO:teuthology.orchestra.run.smithi118.stdout:Get:34 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-singledispatch all 3.4.0.3-2 [7,022 B] 2020-11-28T21:43:19.737 INFO:teuthology.orchestra.run.smithi118.stdout:Get:35 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 15.2.6-595-ga83a3572-1bionic [336 kB] 2020-11-28T21:43:19.738 INFO:teuthology.orchestra.run.smithi096.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-asn1crypto all 0.24.0-1 [72.8 kB] 2020-11-28T21:43:19.738 INFO:teuthology.orchestra.run.smithi118.stdout:Get:36 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2020-11-28T21:43:19.739 INFO:teuthology.orchestra.run.smithi118.stdout:Get:37 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-bs4 all 4.6.0-1 [67.8 kB] 2020-11-28T21:43:19.743 INFO:teuthology.orchestra.run.smithi118.stdout:Get:38 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-waitress all 1.0.1-1 [53.4 kB] 2020-11-28T21:43:19.747 INFO:teuthology.orchestra.run.smithi118.stdout:Get:39 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 15.2.6-595-ga83a3572-1bionic [18.8 MB] 2020-11-28T21:43:19.750 INFO:teuthology.orchestra.run.smithi096.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cffi-backend amd64 1.11.5-1 [64.6 kB] 2020-11-28T21:43:19.753 INFO:teuthology.orchestra.run.smithi192.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cherrypy3 all 8.9.1-2 [160 kB] 2020-11-28T21:43:19.755 INFO:teuthology.orchestra.run.smithi096.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-cryptography amd64 2.1.4-1ubuntu1.4 [220 kB] 2020-11-28T21:43:19.757 INFO:teuthology.orchestra.run.smithi162.stdout:Get:34 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2020-11-28T21:43:19.760 INFO:teuthology.orchestra.run.smithi162.stdout:Get:35 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-bs4 all 4.6.0-1 [67.8 kB] 2020-11-28T21:43:19.762 INFO:teuthology.orchestra.run.smithi192.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jwt all 1.5.3+ds1-1 [15.9 kB] 2020-11-28T21:43:19.763 INFO:teuthology.orchestra.run.smithi162.stdout:Get:36 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-waitress all 1.0.1-1 [53.4 kB] 2020-11-28T21:43:19.763 INFO:teuthology.orchestra.run.smithi192.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-logutils all 0.3.3-5 [16.7 kB] 2020-11-28T21:43:19.764 INFO:teuthology.orchestra.run.smithi192.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-markupsafe amd64 1.0-1build1 [13.5 kB] 2020-11-28T21:43:19.764 INFO:teuthology.orchestra.run.smithi192.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-mako all 1.0.7+ds1-1 [59.3 kB] 2020-11-28T21:43:19.769 INFO:teuthology.orchestra.run.smithi162.stdout:Get:37 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 python3-rgw amd64 15.2.6-595-ga83a3572-1bionic [99.4 kB] 2020-11-28T21:43:19.772 INFO:teuthology.orchestra.run.smithi162.stdout:Get:38 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 15.2.6-595-ga83a3572-1bionic [336 kB] 2020-11-28T21:43:19.773 INFO:teuthology.orchestra.run.smithi096.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-openssl all 17.5.0-1ubuntu1 [41.5 kB] 2020-11-28T21:43:19.777 INFO:teuthology.orchestra.run.smithi096.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-bcrypt amd64 3.1.4-2 [29.9 kB] 2020-11-28T21:43:19.783 INFO:teuthology.orchestra.run.smithi162.stdout:Get:39 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 15.2.6-595-ga83a3572-1bionic [18.8 MB] 2020-11-28T21:43:19.815 INFO:teuthology.orchestra.run.smithi118.stdout:Get:40 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tempita all 0.5.2-2 [13.9 kB] 2020-11-28T21:43:19.815 INFO:teuthology.orchestra.run.smithi118.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2020-11-28T21:43:19.832 INFO:teuthology.orchestra.run.smithi192.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-simplegeneric all 0.8.1-1 [11.5 kB] 2020-11-28T21:43:19.832 INFO:teuthology.orchestra.run.smithi192.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-singledispatch all 3.4.0.3-2 [7,022 B] 2020-11-28T21:43:19.833 INFO:teuthology.orchestra.run.smithi192.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2020-11-28T21:43:19.835 INFO:teuthology.orchestra.run.smithi192.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-bs4 all 4.6.0-1 [67.8 kB] 2020-11-28T21:43:19.836 INFO:teuthology.orchestra.run.smithi096.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cherrypy3 all 8.9.1-2 [160 kB] 2020-11-28T21:43:19.837 INFO:teuthology.orchestra.run.smithi118.stdout:Get:42 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2020-11-28T21:43:19.837 INFO:teuthology.orchestra.run.smithi118.stdout:Get:43 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastedeploy all 1.5.2-4 [13.4 kB] 2020-11-28T21:43:19.838 INFO:teuthology.orchestra.run.smithi162.stdout:Get:40 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tempita all 0.5.2-2 [13.9 kB] 2020-11-28T21:43:19.838 INFO:teuthology.orchestra.run.smithi162.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2020-11-28T21:43:19.838 INFO:teuthology.orchestra.run.smithi118.stdout:Get:44 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webtest all 2.0.28-1ubuntu1 [27.9 kB] 2020-11-28T21:43:19.838 INFO:teuthology.orchestra.run.smithi118.stdout:Get:45 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pecan all 1.2.1-2 [86.1 kB] 2020-11-28T21:43:19.839 INFO:teuthology.orchestra.run.smithi192.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-waitress all 1.0.1-1 [53.4 kB] 2020-11-28T21:43:19.845 INFO:teuthology.orchestra.run.smithi096.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jwt all 1.5.3+ds1-1 [15.9 kB] 2020-11-28T21:43:19.846 INFO:teuthology.orchestra.run.smithi096.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-logutils all 0.3.3-5 [16.7 kB] 2020-11-28T21:43:19.846 INFO:teuthology.orchestra.run.smithi096.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-markupsafe amd64 1.0-1build1 [13.5 kB] 2020-11-28T21:43:19.847 INFO:teuthology.orchestra.run.smithi096.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-mako all 1.0.7+ds1-1 [59.3 kB] 2020-11-28T21:43:19.850 INFO:teuthology.orchestra.run.smithi096.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-simplegeneric all 0.8.1-1 [11.5 kB] 2020-11-28T21:43:19.850 INFO:teuthology.orchestra.run.smithi096.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-singledispatch all 3.4.0.3-2 [7,022 B] 2020-11-28T21:43:19.857 INFO:teuthology.orchestra.run.smithi162.stdout:Get:42 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2020-11-28T21:43:19.858 INFO:teuthology.orchestra.run.smithi162.stdout:Get:43 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastedeploy all 1.5.2-4 [13.4 kB] 2020-11-28T21:43:19.858 INFO:teuthology.orchestra.run.smithi162.stdout:Get:44 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webtest all 2.0.28-1ubuntu1 [27.9 kB] 2020-11-28T21:43:19.859 INFO:teuthology.orchestra.run.smithi162.stdout:Get:45 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pecan all 1.2.1-2 [86.1 kB] 2020-11-28T21:43:19.863 INFO:teuthology.orchestra.run.smithi162.stdout:Get:46 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-werkzeug all 0.14.1+dfsg1-1 [174 kB] 2020-11-28T21:43:19.893 INFO:teuthology.orchestra.run.smithi118.stdout:Get:46 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-werkzeug all 0.14.1+dfsg1-1 [174 kB] 2020-11-28T21:43:19.900 INFO:teuthology.orchestra.run.smithi118.stdout:Get:47 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jinja2 all 2.10-1ubuntu0.18.04.1 [95.4 kB] 2020-11-28T21:43:19.905 INFO:teuthology.orchestra.run.smithi118.stdout:Get:48 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-repoze.lru all 0.7-1 [11.7 kB] 2020-11-28T21:43:19.905 INFO:teuthology.orchestra.run.smithi118.stdout:Get:49 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-routes all 2.4.1-1 [28.4 kB] 2020-11-28T21:43:19.910 INFO:teuthology.orchestra.run.smithi192.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tempita all 0.5.2-2 [13.9 kB] 2020-11-28T21:43:19.911 INFO:teuthology.orchestra.run.smithi192.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2020-11-28T21:43:19.914 INFO:teuthology.orchestra.run.smithi096.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2020-11-28T21:43:19.916 INFO:teuthology.orchestra.run.smithi162.stdout:Get:47 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jinja2 all 2.10-1ubuntu0.18.04.1 [95.4 kB] 2020-11-28T21:43:19.917 INFO:teuthology.orchestra.run.smithi096.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-bs4 all 4.6.0-1 [67.8 kB] 2020-11-28T21:43:19.918 INFO:teuthology.orchestra.run.smithi096.stdout:Get:28 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 15.2.6-595-ga83a3572-1bionic [3,056 kB] 2020-11-28T21:43:19.923 INFO:teuthology.orchestra.run.smithi096.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-waitress all 1.0.1-1 [53.4 kB] 2020-11-28T21:43:19.923 INFO:teuthology.orchestra.run.smithi162.stdout:Get:48 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-repoze.lru all 0.7-1 [11.7 kB] 2020-11-28T21:43:19.924 INFO:teuthology.orchestra.run.smithi162.stdout:Get:49 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-routes all 2.4.1-1 [28.4 kB] 2020-11-28T21:43:19.931 INFO:teuthology.orchestra.run.smithi192.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2020-11-28T21:43:19.931 INFO:teuthology.orchestra.run.smithi192.stdout:Get:33 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastedeploy all 1.5.2-4 [13.4 kB] 2020-11-28T21:43:19.932 INFO:teuthology.orchestra.run.smithi192.stdout:Get:34 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webtest all 2.0.28-1ubuntu1 [27.9 kB] 2020-11-28T21:43:19.971 INFO:teuthology.orchestra.run.smithi118.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] 2020-11-28T21:43:19.988 INFO:teuthology.orchestra.run.smithi192.stdout:Get:35 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pecan all 1.2.1-2 [86.1 kB] 2020-11-28T21:43:19.991 INFO:teuthology.orchestra.run.smithi192.stdout:Get:36 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-werkzeug all 0.14.1+dfsg1-1 [174 kB] 2020-11-28T21:43:19.993 INFO:teuthology.orchestra.run.smithi096.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tempita all 0.5.2-2 [13.9 kB] 2020-11-28T21:43:19.993 INFO:teuthology.orchestra.run.smithi096.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2020-11-28T21:43:19.995 INFO:teuthology.orchestra.run.smithi162.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] 2020-11-28T21:43:20.000 INFO:teuthology.orchestra.run.smithi192.stdout:Get:37 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jinja2 all 2.10-1ubuntu0.18.04.1 [95.4 kB] 2020-11-28T21:43:20.004 INFO:teuthology.orchestra.run.smithi192.stdout:Get:38 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-repoze.lru all 0.7-1 [11.7 kB] 2020-11-28T21:43:20.004 INFO:teuthology.orchestra.run.smithi192.stdout:Get:39 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-routes all 2.4.1-1 [28.4 kB] 2020-11-28T21:43:20.012 INFO:teuthology.orchestra.run.smithi096.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2020-11-28T21:43:20.012 INFO:teuthology.orchestra.run.smithi096.stdout:Get:33 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastedeploy all 1.5.2-4 [13.4 kB] 2020-11-28T21:43:20.013 INFO:teuthology.orchestra.run.smithi096.stdout:Get:34 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webtest all 2.0.28-1ubuntu1 [27.9 kB] 2020-11-28T21:43:20.014 INFO:teuthology.orchestra.run.smithi096.stdout:Get:35 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pecan all 1.2.1-2 [86.1 kB] 2020-11-28T21:43:20.017 INFO:teuthology.orchestra.run.smithi096.stdout:Get:36 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-werkzeug all 0.14.1+dfsg1-1 [174 kB] 2020-11-28T21:43:20.031 INFO:teuthology.orchestra.run.smithi162.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-decorator all 4.1.2-1 [9,364 B] 2020-11-28T21:43:20.031 INFO:teuthology.orchestra.run.smithi162.stdout:Get:52 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-scipy amd64 0.19.1-2ubuntu1 [9,619 kB] 2020-11-28T21:43:20.051 INFO:teuthology.orchestra.run.smithi118.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-decorator all 4.1.2-1 [9,364 B] 2020-11-28T21:43:20.052 INFO:teuthology.orchestra.run.smithi118.stdout:Get:52 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-scipy amd64 0.19.1-2ubuntu1 [9,619 kB] 2020-11-28T21:43:20.066 INFO:teuthology.orchestra.run.smithi192.stdout:Get:40 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] 2020-11-28T21:43:20.071 INFO:teuthology.orchestra.run.smithi096.stdout:Get:37 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jinja2 all 2.10-1ubuntu0.18.04.1 [95.4 kB] 2020-11-28T21:43:20.075 INFO:teuthology.orchestra.run.smithi096.stdout:Get:38 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-repoze.lru all 0.7-1 [11.7 kB] 2020-11-28T21:43:20.076 INFO:teuthology.orchestra.run.smithi096.stdout:Get:39 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-routes all 2.4.1-1 [28.4 kB] 2020-11-28T21:43:20.150 INFO:teuthology.orchestra.run.smithi096.stdout:Get:40 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] 2020-11-28T21:43:20.154 INFO:teuthology.orchestra.run.smithi192.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-decorator all 4.1.2-1 [9,364 B] 2020-11-28T21:43:20.154 INFO:teuthology.orchestra.run.smithi192.stdout:Get:42 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-scipy amd64 0.19.1-2ubuntu1 [9,619 kB] 2020-11-28T21:43:20.185 INFO:teuthology.orchestra.run.smithi096.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-decorator all 4.1.2-1 [9,364 B] 2020-11-28T21:43:20.185 INFO:teuthology.orchestra.run.smithi096.stdout:Get:42 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-scipy amd64 0.19.1-2ubuntu1 [9,619 kB] 2020-11-28T21:43:20.323 INFO:teuthology.orchestra.run.smithi118.stdout:Get:53 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn-lib amd64 0.19.1-3 [1,429 kB] 2020-11-28T21:43:20.334 INFO:teuthology.orchestra.run.smithi162.stdout:Get:53 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn-lib amd64 0.19.1-3 [1,429 kB] 2020-11-28T21:43:20.349 INFO:teuthology.orchestra.run.smithi162.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-joblib all 0.11-1 [109 kB] 2020-11-28T21:43:20.351 INFO:teuthology.orchestra.run.smithi162.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn all 0.19.1-3 [1,373 kB] 2020-11-28T21:43:20.367 INFO:teuthology.orchestra.run.smithi118.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-joblib all 0.11-1 [109 kB] 2020-11-28T21:43:20.369 INFO:teuthology.orchestra.run.smithi118.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn all 0.19.1-3 [1,373 kB] 2020-11-28T21:43:20.401 INFO:teuthology.orchestra.run.smithi118.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-monotonic all 1.1-2 [5,332 B] 2020-11-28T21:43:20.402 INFO:teuthology.orchestra.run.smithi118.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-fasteners all 0.12.0-3 [13.5 kB] 2020-11-28T21:43:20.403 INFO:teuthology.orchestra.run.smithi118.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-httplib2 all 0.9.2+dfsg-1ubuntu0.2 [28.6 kB] 2020-11-28T21:43:20.403 INFO:teuthology.orchestra.run.smithi118.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1 all 0.4.2-3 [46.8 kB] 2020-11-28T21:43:20.404 INFO:teuthology.orchestra.run.smithi118.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1-modules all 0.2.1-0.2 [32.9 kB] 2020-11-28T21:43:20.405 INFO:teuthology.orchestra.run.smithi118.stdout:Get:61 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-rsa all 3.4.2-1 [29.0 kB] 2020-11-28T21:43:20.406 INFO:teuthology.orchestra.run.smithi118.stdout:Get:62 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-sqlalchemy all 1.1.11+ds1-1ubuntu1 [680 kB] 2020-11-28T21:43:20.416 INFO:teuthology.orchestra.run.smithi162.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-monotonic all 1.1-2 [5,332 B] 2020-11-28T21:43:20.416 INFO:teuthology.orchestra.run.smithi162.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-fasteners all 0.12.0-3 [13.5 kB] 2020-11-28T21:43:20.417 INFO:teuthology.orchestra.run.smithi162.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-httplib2 all 0.9.2+dfsg-1ubuntu0.2 [28.6 kB] 2020-11-28T21:43:20.417 INFO:teuthology.orchestra.run.smithi162.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1 all 0.4.2-3 [46.8 kB] 2020-11-28T21:43:20.418 INFO:teuthology.orchestra.run.smithi162.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1-modules all 0.2.1-0.2 [32.9 kB] 2020-11-28T21:43:20.418 INFO:teuthology.orchestra.run.smithi162.stdout:Get:61 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-rsa all 3.4.2-1 [29.0 kB] 2020-11-28T21:43:20.419 INFO:teuthology.orchestra.run.smithi162.stdout:Get:62 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-sqlalchemy all 1.1.11+ds1-1ubuntu1 [680 kB] 2020-11-28T21:43:20.426 INFO:teuthology.orchestra.run.smithi192.stdout:Get:43 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn-lib amd64 0.19.1-3 [1,429 kB] 2020-11-28T21:43:20.473 INFO:teuthology.orchestra.run.smithi192.stdout:Get:44 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-joblib all 0.11-1 [109 kB] 2020-11-28T21:43:20.474 INFO:teuthology.orchestra.run.smithi192.stdout:Get:45 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn all 0.19.1-3 [1,373 kB] 2020-11-28T21:43:20.480 INFO:teuthology.orchestra.run.smithi118.stdout:Get:63 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-oauth2client all 4.1.2-3 [61.5 kB] 2020-11-28T21:43:20.482 INFO:teuthology.orchestra.run.smithi118.stdout:Get:64 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-setuptools all 39.0.1-2 [248 kB] 2020-11-28T21:43:20.490 INFO:teuthology.orchestra.run.smithi096.stdout:Get:43 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn-lib amd64 0.19.1-3 [1,429 kB] 2020-11-28T21:43:20.493 INFO:teuthology.orchestra.run.smithi162.stdout:Get:63 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-oauth2client all 4.1.2-3 [61.5 kB] 2020-11-28T21:43:20.496 INFO:teuthology.orchestra.run.smithi162.stdout:Get:64 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-setuptools all 39.0.1-2 [248 kB] 2020-11-28T21:43:20.505 INFO:teuthology.orchestra.run.smithi096.stdout:Get:44 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-joblib all 0.11-1 [109 kB] 2020-11-28T21:43:20.506 INFO:teuthology.orchestra.run.smithi192.stdout:Get:46 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-monotonic all 1.1-2 [5,332 B] 2020-11-28T21:43:20.506 INFO:teuthology.orchestra.run.smithi192.stdout:Get:47 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-fasteners all 0.12.0-3 [13.5 kB] 2020-11-28T21:43:20.506 INFO:teuthology.orchestra.run.smithi192.stdout:Get:48 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-httplib2 all 0.9.2+dfsg-1ubuntu0.2 [28.6 kB] 2020-11-28T21:43:20.507 INFO:teuthology.orchestra.run.smithi096.stdout:Get:45 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn all 0.19.1-3 [1,373 kB] 2020-11-28T21:43:20.507 INFO:teuthology.orchestra.run.smithi192.stdout:Get:49 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1 all 0.4.2-3 [46.8 kB] 2020-11-28T21:43:20.508 INFO:teuthology.orchestra.run.smithi192.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1-modules all 0.2.1-0.2 [32.9 kB] 2020-11-28T21:43:20.508 INFO:teuthology.orchestra.run.smithi192.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-rsa all 3.4.2-1 [29.0 kB] 2020-11-28T21:43:20.509 INFO:teuthology.orchestra.run.smithi192.stdout:Get:52 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-sqlalchemy all 1.1.11+ds1-1ubuntu1 [680 kB] 2020-11-28T21:43:20.560 INFO:teuthology.orchestra.run.smithi118.stdout:Get:65 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-websocket all 0.44.0-0ubuntu2 [30.7 kB] 2020-11-28T21:43:20.561 INFO:teuthology.orchestra.run.smithi118.stdout:Get:66 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-kubernetes all 2.0.0-2ubuntu1 [181 kB] 2020-11-28T21:43:20.565 INFO:teuthology.orchestra.run.smithi096.stdout:Get:46 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-monotonic all 1.1-2 [5,332 B] 2020-11-28T21:43:20.565 INFO:teuthology.orchestra.run.smithi096.stdout:Get:47 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-fasteners all 0.12.0-3 [13.5 kB] 2020-11-28T21:43:20.565 INFO:teuthology.orchestra.run.smithi096.stdout:Get:48 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-httplib2 all 0.9.2+dfsg-1ubuntu0.2 [28.6 kB] 2020-11-28T21:43:20.566 INFO:teuthology.orchestra.run.smithi118.stdout:Get:67 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-json-pointer all 1.10-1 [7,478 B] 2020-11-28T21:43:20.567 INFO:teuthology.orchestra.run.smithi118.stdout:Get:68 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jsonpatch all 1.19+really1.16-1fakesync1 [13.4 kB] 2020-11-28T21:43:20.567 INFO:teuthology.orchestra.run.smithi096.stdout:Get:49 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1 all 0.4.2-3 [46.8 kB] 2020-11-28T21:43:20.568 INFO:teuthology.orchestra.run.smithi118.stdout:Get:69 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2020-11-28T21:43:20.568 INFO:teuthology.orchestra.run.smithi096.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1-modules all 0.2.1-0.2 [32.9 kB] 2020-11-28T21:43:20.568 INFO:teuthology.orchestra.run.smithi096.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-rsa all 3.4.2-1 [29.0 kB] 2020-11-28T21:43:20.569 INFO:teuthology.orchestra.run.smithi118.stdout:Get:70 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2020-11-28T21:43:20.570 INFO:teuthology.orchestra.run.smithi096.stdout:Get:52 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-sqlalchemy all 1.1.11+ds1-1ubuntu1 [680 kB] 2020-11-28T21:43:20.571 INFO:teuthology.orchestra.run.smithi118.stdout:Get:71 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2020-11-28T21:43:20.571 INFO:teuthology.orchestra.run.smithi118.stdout:Get:72 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2020-11-28T21:43:20.573 INFO:teuthology.orchestra.run.smithi162.stdout:Get:65 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-websocket all 0.44.0-0ubuntu2 [30.7 kB] 2020-11-28T21:43:20.574 INFO:teuthology.orchestra.run.smithi162.stdout:Get:66 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-kubernetes all 2.0.0-2ubuntu1 [181 kB] 2020-11-28T21:43:20.579 INFO:teuthology.orchestra.run.smithi162.stdout:Get:67 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-json-pointer all 1.10-1 [7,478 B] 2020-11-28T21:43:20.579 INFO:teuthology.orchestra.run.smithi162.stdout:Get:68 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jsonpatch all 1.19+really1.16-1fakesync1 [13.4 kB] 2020-11-28T21:43:20.582 INFO:teuthology.orchestra.run.smithi162.stdout:Get:69 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2020-11-28T21:43:20.582 INFO:teuthology.orchestra.run.smithi162.stdout:Get:70 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2020-11-28T21:43:20.583 INFO:teuthology.orchestra.run.smithi192.stdout:Get:53 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-oauth2client all 4.1.2-3 [61.5 kB] 2020-11-28T21:43:20.584 INFO:teuthology.orchestra.run.smithi162.stdout:Get:71 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2020-11-28T21:43:20.584 INFO:teuthology.orchestra.run.smithi162.stdout:Get:72 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2020-11-28T21:43:20.586 INFO:teuthology.orchestra.run.smithi192.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-setuptools all 39.0.1-2 [248 kB] 2020-11-28T21:43:20.637 INFO:teuthology.orchestra.run.smithi118.stdout:Get:73 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.2 [150 kB] 2020-11-28T21:43:20.643 INFO:teuthology.orchestra.run.smithi118.stdout:Get:74 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2020-11-28T21:43:20.648 INFO:teuthology.orchestra.run.smithi096.stdout:Get:53 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-oauth2client all 4.1.2-3 [61.5 kB] 2020-11-28T21:43:20.650 INFO:teuthology.orchestra.run.smithi096.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-setuptools all 39.0.1-2 [248 kB] 2020-11-28T21:43:20.652 INFO:teuthology.orchestra.run.smithi162.stdout:Get:73 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.2 [150 kB] 2020-11-28T21:43:20.657 INFO:teuthology.orchestra.run.smithi162.stdout:Get:74 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2020-11-28T21:43:20.657 INFO:teuthology.orchestra.run.smithi096.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-websocket all 0.44.0-0ubuntu2 [30.7 kB] 2020-11-28T21:43:20.658 INFO:teuthology.orchestra.run.smithi096.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-kubernetes all 2.0.0-2ubuntu1 [181 kB] 2020-11-28T21:43:20.662 INFO:teuthology.orchestra.run.smithi192.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-websocket all 0.44.0-0ubuntu2 [30.7 kB] 2020-11-28T21:43:20.662 INFO:teuthology.orchestra.run.smithi096.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-json-pointer all 1.10-1 [7,478 B] 2020-11-28T21:43:20.663 INFO:teuthology.orchestra.run.smithi096.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jsonpatch all 1.19+really1.16-1fakesync1 [13.4 kB] 2020-11-28T21:43:20.663 INFO:teuthology.orchestra.run.smithi096.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2020-11-28T21:43:20.663 INFO:teuthology.orchestra.run.smithi192.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-kubernetes all 2.0.0-2ubuntu1 [181 kB] 2020-11-28T21:43:20.664 INFO:teuthology.orchestra.run.smithi096.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2020-11-28T21:43:20.665 INFO:teuthology.orchestra.run.smithi096.stdout:Get:61 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2020-11-28T21:43:20.668 INFO:teuthology.orchestra.run.smithi192.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-json-pointer all 1.10-1 [7,478 B] 2020-11-28T21:43:20.669 INFO:teuthology.orchestra.run.smithi192.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jsonpatch all 1.19+really1.16-1fakesync1 [13.4 kB] 2020-11-28T21:43:20.669 INFO:teuthology.orchestra.run.smithi192.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2020-11-28T21:43:20.673 INFO:teuthology.orchestra.run.smithi192.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2020-11-28T21:43:20.676 INFO:teuthology.orchestra.run.smithi192.stdout:Get:61 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2020-11-28T21:43:20.677 INFO:teuthology.orchestra.run.smithi192.stdout:Get:62 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2020-11-28T21:43:20.715 INFO:teuthology.orchestra.run.smithi118.stdout:Get:75 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-colorama all 0.3.7-1 [14.9 kB] 2020-11-28T21:43:20.716 INFO:teuthology.orchestra.run.smithi118.stdout:Get:76 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 nvme-cli amd64 1.5-1ubuntu1.1 [184 kB] 2020-11-28T21:43:20.723 INFO:teuthology.orchestra.run.smithi118.stdout:Get:77 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-attr all 17.4.0-2 [23.7 kB] 2020-11-28T21:43:20.724 INFO:teuthology.orchestra.run.smithi118.stdout:Get:78 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-funcsigs all 1.0.2-4 [13.5 kB] 2020-11-28T21:43:20.725 INFO:teuthology.orchestra.run.smithi118.stdout:Get:79 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pluggy all 0.6.0-1 [12.2 kB] 2020-11-28T21:43:20.725 INFO:teuthology.orchestra.run.smithi118.stdout:Get:80 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-py all 1.5.2-1 [65.1 kB] 2020-11-28T21:43:20.728 INFO:teuthology.orchestra.run.smithi118.stdout:Get:81 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pytest all 3.3.2-2 [142 kB] 2020-11-28T21:43:20.728 INFO:teuthology.orchestra.run.smithi096.stdout:Get:62 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2020-11-28T21:43:20.730 INFO:teuthology.orchestra.run.smithi118.stdout:Get:82 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-attr all 17.4.0-2 [23.8 kB] 2020-11-28T21:43:20.732 INFO:teuthology.orchestra.run.smithi162.stdout:Get:75 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-colorama all 0.3.7-1 [14.9 kB] 2020-11-28T21:43:20.732 INFO:teuthology.orchestra.run.smithi162.stdout:Get:76 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 nvme-cli amd64 1.5-1ubuntu1.1 [184 kB] 2020-11-28T21:43:20.738 INFO:teuthology.orchestra.run.smithi162.stdout:Get:77 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-attr all 17.4.0-2 [23.7 kB] 2020-11-28T21:43:20.739 INFO:teuthology.orchestra.run.smithi162.stdout:Get:78 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-funcsigs all 1.0.2-4 [13.5 kB] 2020-11-28T21:43:20.740 INFO:teuthology.orchestra.run.smithi162.stdout:Get:79 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pluggy all 0.6.0-1 [12.2 kB] 2020-11-28T21:43:20.740 INFO:teuthology.orchestra.run.smithi162.stdout:Get:80 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-py all 1.5.2-1 [65.1 kB] 2020-11-28T21:43:20.741 INFO:teuthology.orchestra.run.smithi192.stdout:Get:63 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.2 [150 kB] 2020-11-28T21:43:20.741 INFO:teuthology.orchestra.run.smithi162.stdout:Get:81 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pytest all 3.3.2-2 [142 kB] 2020-11-28T21:43:20.744 INFO:teuthology.orchestra.run.smithi162.stdout:Get:82 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-attr all 17.4.0-2 [23.8 kB] 2020-11-28T21:43:20.744 INFO:teuthology.orchestra.run.smithi096.stdout:Get:63 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.2 [150 kB] 2020-11-28T21:43:20.745 INFO:teuthology.orchestra.run.smithi192.stdout:Get:64 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2020-11-28T21:43:20.750 INFO:teuthology.orchestra.run.smithi096.stdout:Get:64 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2020-11-28T21:43:20.758 INFO:teuthology.orchestra.run.smithi096.stdout:Get:65 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-colorama all 0.3.7-1 [14.9 kB] 2020-11-28T21:43:20.759 INFO:teuthology.orchestra.run.smithi096.stdout:Get:66 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 nvme-cli amd64 1.5-1ubuntu1.1 [184 kB] 2020-11-28T21:43:20.761 INFO:teuthology.orchestra.run.smithi192.stdout:Get:65 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 15.2.6-595-ga83a3572-1bionic [435 kB] 2020-11-28T21:43:20.762 INFO:teuthology.orchestra.run.smithi096.stdout:Get:67 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-attr all 17.4.0-2 [23.7 kB] 2020-11-28T21:43:20.762 INFO:teuthology.orchestra.run.smithi096.stdout:Get:68 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-funcsigs all 1.0.2-4 [13.5 kB] 2020-11-28T21:43:20.763 INFO:teuthology.orchestra.run.smithi096.stdout:Get:69 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pluggy all 0.6.0-1 [12.2 kB] 2020-11-28T21:43:20.763 INFO:teuthology.orchestra.run.smithi096.stdout:Get:70 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-py all 1.5.2-1 [65.1 kB] 2020-11-28T21:43:20.794 INFO:teuthology.orchestra.run.smithi118.stdout:Get:83 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cycler all 0.10.0-1 [7,622 B] 2020-11-28T21:43:20.794 INFO:teuthology.orchestra.run.smithi118.stdout:Get:84 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webencodings all 0.5-2 [10.4 kB] 2020-11-28T21:43:20.807 INFO:teuthology.orchestra.run.smithi096.stdout:Get:71 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pytest all 3.3.2-2 [142 kB] 2020-11-28T21:43:20.811 INFO:teuthology.orchestra.run.smithi162.stdout:Get:83 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cycler all 0.10.0-1 [7,622 B] 2020-11-28T21:43:20.812 INFO:teuthology.orchestra.run.smithi162.stdout:Get:84 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webencodings all 0.5-2 [10.4 kB] 2020-11-28T21:43:20.818 INFO:teuthology.orchestra.run.smithi192.stdout:Get:66 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-colorama all 0.3.7-1 [14.9 kB] 2020-11-28T21:43:20.819 INFO:teuthology.orchestra.run.smithi192.stdout:Get:67 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 nvme-cli amd64 1.5-1ubuntu1.1 [184 kB] 2020-11-28T21:43:20.826 INFO:teuthology.orchestra.run.smithi192.stdout:Get:68 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-attr all 17.4.0-2 [23.7 kB] 2020-11-28T21:43:20.827 INFO:teuthology.orchestra.run.smithi192.stdout:Get:69 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-funcsigs all 1.0.2-4 [13.5 kB] 2020-11-28T21:43:20.828 INFO:teuthology.orchestra.run.smithi192.stdout:Get:70 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pluggy all 0.6.0-1 [12.2 kB] 2020-11-28T21:43:20.828 INFO:teuthology.orchestra.run.smithi192.stdout:Get:71 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-py all 1.5.2-1 [65.1 kB] 2020-11-28T21:43:20.830 INFO:teuthology.orchestra.run.smithi192.stdout:Get:72 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pytest all 3.3.2-2 [142 kB] 2020-11-28T21:43:20.831 INFO:teuthology.orchestra.run.smithi192.stdout:Get:73 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-attr all 17.4.0-2 [23.8 kB] 2020-11-28T21:43:20.842 INFO:teuthology.orchestra.run.smithi096.stdout:Get:72 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-attr all 17.4.0-2 [23.8 kB] 2020-11-28T21:43:20.843 INFO:teuthology.orchestra.run.smithi096.stdout:Get:73 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cycler all 0.10.0-1 [7,622 B] 2020-11-28T21:43:20.843 INFO:teuthology.orchestra.run.smithi096.stdout:Get:74 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webencodings all 0.5-2 [10.4 kB] 2020-11-28T21:43:20.844 INFO:teuthology.orchestra.run.smithi096.stdout:Get:75 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-html5lib all 0.999999999-1 [81.9 kB] 2020-11-28T21:43:20.847 INFO:teuthology.orchestra.run.smithi096.stdout:Get:76 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-lxml amd64 4.2.1-1ubuntu0.1 [1,091 kB] 2020-11-28T21:43:20.867 INFO:teuthology.orchestra.run.smithi096.stdout:Get:77 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tz all 2018.3-2 [25.1 kB] 2020-11-28T21:43:20.867 INFO:teuthology.orchestra.run.smithi096.stdout:Get:78 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-matplotlib amd64 2.1.1-2ubuntu3 [3,907 kB] 2020-11-28T21:43:20.872 INFO:teuthology.orchestra.run.smithi118.stdout:Get:85 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-html5lib all 0.999999999-1 [81.9 kB] 2020-11-28T21:43:20.874 INFO:teuthology.orchestra.run.smithi118.stdout:Get:86 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-lxml amd64 4.2.1-1ubuntu0.1 [1,091 kB] 2020-11-28T21:43:20.891 INFO:teuthology.orchestra.run.smithi162.stdout:Get:85 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-html5lib all 0.999999999-1 [81.9 kB] 2020-11-28T21:43:20.892 INFO:teuthology.orchestra.run.smithi162.stdout:Get:86 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-lxml amd64 4.2.1-1ubuntu0.1 [1,091 kB] 2020-11-28T21:43:20.896 INFO:teuthology.orchestra.run.smithi118.stdout:Get:87 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tz all 2018.3-2 [25.1 kB] 2020-11-28T21:43:20.896 INFO:teuthology.orchestra.run.smithi192.stdout:Get:74 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cycler all 0.10.0-1 [7,622 B] 2020-11-28T21:43:20.897 INFO:teuthology.orchestra.run.smithi118.stdout:Get:88 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-matplotlib amd64 2.1.1-2ubuntu3 [3,907 kB] 2020-11-28T21:43:20.897 INFO:teuthology.orchestra.run.smithi192.stdout:Get:75 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webencodings all 0.5-2 [10.4 kB] 2020-11-28T21:43:20.912 INFO:teuthology.orchestra.run.smithi162.stdout:Get:87 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tz all 2018.3-2 [25.1 kB] 2020-11-28T21:43:20.913 INFO:teuthology.orchestra.run.smithi162.stdout:Get:88 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-matplotlib amd64 2.1.1-2ubuntu3 [3,907 kB] 2020-11-28T21:43:20.941 INFO:teuthology.orchestra.run.smithi192.stdout:Get:76 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 python3-rados amd64 15.2.6-595-ga83a3572-1bionic [332 kB] 2020-11-28T21:43:20.969 INFO:teuthology.orchestra.run.smithi096.stdout:Get:79 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] 2020-11-28T21:43:20.969 INFO:teuthology.orchestra.run.smithi096.stdout:Get:80 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastescript all 2.0.2-2 [54.6 kB] 2020-11-28T21:43:20.970 INFO:teuthology.orchestra.run.smithi096.stdout:Get:81 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.3 [330 kB] 2020-11-28T21:43:20.974 INFO:teuthology.orchestra.run.smithi192.stdout:Get:77 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-html5lib all 0.999999999-1 [81.9 kB] 2020-11-28T21:43:20.976 INFO:teuthology.orchestra.run.smithi096.stdout:Get:82 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pluggy all 0.6.0-1 [12.3 kB] 2020-11-28T21:43:20.976 INFO:teuthology.orchestra.run.smithi096.stdout:Get:83 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-py all 1.5.2-1 [65.2 kB] 2020-11-28T21:43:20.977 INFO:teuthology.orchestra.run.smithi096.stdout:Get:84 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyinotify all 0.9.6-1 [24.7 kB] 2020-11-28T21:43:20.977 INFO:teuthology.orchestra.run.smithi096.stdout:Get:85 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pytest all 3.3.2-2 [142 kB] 2020-11-28T21:43:20.978 INFO:teuthology.orchestra.run.smithi192.stdout:Get:78 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-lxml amd64 4.2.1-1ubuntu0.1 [1,091 kB] 2020-11-28T21:43:20.978 INFO:teuthology.orchestra.run.smithi096.stdout:Get:86 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-simplejson amd64 3.13.2-1 [49.6 kB] 2020-11-28T21:43:20.980 INFO:teuthology.orchestra.run.smithi096.stdout:Get:87 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-tk amd64 3.6.9-1~18.04 [119 kB] 2020-11-28T21:43:20.997 INFO:teuthology.orchestra.run.smithi118.stdout:Get:89 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] 2020-11-28T21:43:20.998 INFO:teuthology.orchestra.run.smithi192.stdout:Get:79 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tz all 2018.3-2 [25.1 kB] 2020-11-28T21:43:20.998 INFO:teuthology.orchestra.run.smithi192.stdout:Get:80 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-matplotlib amd64 2.1.1-2ubuntu3 [3,907 kB] 2020-11-28T21:43:20.999 INFO:teuthology.orchestra.run.smithi118.stdout:Get:90 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastescript all 2.0.2-2 [54.6 kB] 2020-11-28T21:43:20.999 INFO:teuthology.orchestra.run.smithi118.stdout:Get:91 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.3 [330 kB] 2020-11-28T21:43:21.000 INFO:teuthology.orchestra.run.smithi162.stdout:Get:89 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] 2020-11-28T21:43:21.005 INFO:teuthology.orchestra.run.smithi118.stdout:Get:92 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pluggy all 0.6.0-1 [12.3 kB] 2020-11-28T21:43:21.006 INFO:teuthology.orchestra.run.smithi118.stdout:Get:93 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-py all 1.5.2-1 [65.2 kB] 2020-11-28T21:43:21.006 INFO:teuthology.orchestra.run.smithi118.stdout:Get:94 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyinotify all 0.9.6-1 [24.7 kB] 2020-11-28T21:43:21.028 INFO:teuthology.orchestra.run.smithi118.stdout:Get:95 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pytest all 3.3.2-2 [142 kB] 2020-11-28T21:43:21.032 INFO:teuthology.orchestra.run.smithi118.stdout:Get:96 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-simplejson amd64 3.13.2-1 [49.6 kB] 2020-11-28T21:43:21.034 INFO:teuthology.orchestra.run.smithi118.stdout:Get:97 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-tk amd64 3.6.9-1~18.04 [119 kB] 2020-11-28T21:43:21.049 INFO:teuthology.orchestra.run.smithi162.stdout:Get:90 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastescript all 2.0.2-2 [54.6 kB] 2020-11-28T21:43:21.050 INFO:teuthology.orchestra.run.smithi162.stdout:Get:91 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.3 [330 kB] 2020-11-28T21:43:21.055 INFO:teuthology.orchestra.run.smithi162.stdout:Get:92 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pluggy all 0.6.0-1 [12.3 kB] 2020-11-28T21:43:21.055 INFO:teuthology.orchestra.run.smithi162.stdout:Get:93 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-py all 1.5.2-1 [65.2 kB] 2020-11-28T21:43:21.056 INFO:teuthology.orchestra.run.smithi162.stdout:Get:94 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyinotify all 0.9.6-1 [24.7 kB] 2020-11-28T21:43:21.057 INFO:teuthology.orchestra.run.smithi162.stdout:Get:95 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pytest all 3.3.2-2 [142 kB] 2020-11-28T21:43:21.059 INFO:teuthology.orchestra.run.smithi162.stdout:Get:96 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-simplejson amd64 3.13.2-1 [49.6 kB] 2020-11-28T21:43:21.059 INFO:teuthology.orchestra.run.smithi162.stdout:Get:97 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-tk amd64 3.6.9-1~18.04 [119 kB] 2020-11-28T21:43:21.072 INFO:teuthology.orchestra.run.smithi192.stdout:Get:81 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-argparse all 15.2.6-595-ga83a3572-1bionic [20.7 kB] 2020-11-28T21:43:21.073 INFO:teuthology.orchestra.run.smithi192.stdout:Get:82 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 python3-cephfs amd64 15.2.6-595-ga83a3572-1bionic [133 kB] 2020-11-28T21:43:21.082 INFO:teuthology.orchestra.run.smithi192.stdout:Get:83 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-common all 15.2.6-595-ga83a3572-1bionic [27.8 kB] 2020-11-28T21:43:21.132 INFO:teuthology.orchestra.run.smithi192.stdout:Get:84 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] 2020-11-28T21:43:21.133 INFO:teuthology.orchestra.run.smithi192.stdout:Get:85 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastescript all 2.0.2-2 [54.6 kB] 2020-11-28T21:43:21.135 INFO:teuthology.orchestra.run.smithi192.stdout:Get:86 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.3 [330 kB] 2020-11-28T21:43:21.139 INFO:teuthology.orchestra.run.smithi192.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 python3-rbd amd64 15.2.6-595-ga83a3572-1bionic [312 kB] 2020-11-28T21:43:21.144 INFO:teuthology.orchestra.run.smithi192.stdout:Get:88 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pluggy all 0.6.0-1 [12.3 kB] 2020-11-28T21:43:21.144 INFO:teuthology.orchestra.run.smithi192.stdout:Get:89 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-py all 1.5.2-1 [65.2 kB] 2020-11-28T21:43:21.145 INFO:teuthology.orchestra.run.smithi192.stdout:Get:90 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyinotify all 0.9.6-1 [24.7 kB] 2020-11-28T21:43:21.146 INFO:teuthology.orchestra.run.smithi192.stdout:Get:91 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pytest all 3.3.2-2 [142 kB] 2020-11-28T21:43:21.147 INFO:teuthology.orchestra.run.smithi192.stdout:Get:92 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-simplejson amd64 3.13.2-1 [49.6 kB] 2020-11-28T21:43:21.147 INFO:teuthology.orchestra.run.smithi192.stdout:Get:93 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-tk amd64 3.6.9-1~18.04 [119 kB] 2020-11-28T21:43:21.193 INFO:teuthology.orchestra.run.smithi118.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 15.2.6-595-ga83a3572-1bionic [5,170 kB] 2020-11-28T21:43:21.212 INFO:teuthology.orchestra.run.smithi192.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 15.2.6-595-ga83a3572-1bionic [2,890 kB] 2020-11-28T21:43:21.298 INFO:teuthology.orchestra.run.smithi162.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 15.2.6-595-ga83a3572-1bionic [5,170 kB] 2020-11-28T21:43:21.369 INFO:teuthology.orchestra.run.smithi096.stdout:Get:88 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 15.2.6-595-ga83a3572-1bionic [435 kB] 2020-11-28T21:43:21.450 INFO:teuthology.orchestra.run.smithi096.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 python3-rados amd64 15.2.6-595-ga83a3572-1bionic [332 kB] 2020-11-28T21:43:21.585 INFO:teuthology.orchestra.run.smithi096.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-argparse all 15.2.6-595-ga83a3572-1bionic [20.7 kB] 2020-11-28T21:43:21.587 INFO:teuthology.orchestra.run.smithi096.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 python3-cephfs amd64 15.2.6-595-ga83a3572-1bionic [133 kB] 2020-11-28T21:43:21.591 INFO:teuthology.orchestra.run.smithi096.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-common all 15.2.6-595-ga83a3572-1bionic [27.8 kB] 2020-11-28T21:43:21.594 INFO:teuthology.orchestra.run.smithi096.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 python3-rbd amd64 15.2.6-595-ga83a3572-1bionic [312 kB] 2020-11-28T21:43:21.615 INFO:teuthology.orchestra.run.smithi118.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-modules-core all 15.2.6-595-ga83a3572-1bionic [148 kB] 2020-11-28T21:43:21.619 INFO:teuthology.orchestra.run.smithi118.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 15.2.6-595-ga83a3572-1bionic [1,282 kB] 2020-11-28T21:43:21.670 INFO:teuthology.orchestra.run.smithi096.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 15.2.6-595-ga83a3572-1bionic [2,890 kB] 2020-11-28T21:43:21.706 INFO:teuthology.orchestra.run.smithi118.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 15.2.6-595-ga83a3572-1bionic [5,944 kB] 2020-11-28T21:43:21.731 INFO:teuthology.orchestra.run.smithi162.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-modules-core all 15.2.6-595-ga83a3572-1bionic [148 kB] 2020-11-28T21:43:21.742 INFO:teuthology.orchestra.run.smithi162.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 15.2.6-595-ga83a3572-1bionic [1,282 kB] 2020-11-28T21:43:21.834 INFO:teuthology.orchestra.run.smithi162.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 15.2.6-595-ga83a3572-1bionic [5,944 kB] 2020-11-28T21:43:22.237 INFO:teuthology.orchestra.run.smithi118.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 15.2.6-595-ga83a3572-1bionic [22.7 MB] 2020-11-28T21:43:22.267 INFO:teuthology.orchestra.run.smithi192.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 python3-rgw amd64 15.2.6-595-ga83a3572-1bionic [99.4 kB] 2020-11-28T21:43:22.343 INFO:teuthology.orchestra.run.smithi192.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 15.2.6-595-ga83a3572-1bionic [336 kB] 2020-11-28T21:43:22.381 INFO:teuthology.orchestra.run.smithi096.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 python3-rgw amd64 15.2.6-595-ga83a3572-1bionic [99.4 kB] 2020-11-28T21:43:22.384 INFO:teuthology.orchestra.run.smithi096.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 15.2.6-595-ga83a3572-1bionic [336 kB] 2020-11-28T21:43:22.430 INFO:teuthology.orchestra.run.smithi192.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 15.2.6-595-ga83a3572-1bionic [18.8 MB] 2020-11-28T21:43:22.449 INFO:teuthology.orchestra.run.smithi096.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 15.2.6-595-ga83a3572-1bionic [18.8 MB] 2020-11-28T21:43:22.468 INFO:teuthology.orchestra.run.smithi162.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 15.2.6-595-ga83a3572-1bionic [22.7 MB] 2020-11-28T21:43:23.918 INFO:teuthology.orchestra.run.smithi118.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 15.2.6-595-ga83a3572-1bionic [3,892 B] 2020-11-28T21:43:23.918 INFO:teuthology.orchestra.run.smithi118.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 15.2.6-595-ga83a3572-1bionic [515 kB] 2020-11-28T21:43:23.975 INFO:teuthology.orchestra.run.smithi118.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 15.2.6-595-ga83a3572-1bionic [1,850 kB] 2020-11-28T21:43:24.088 INFO:teuthology.orchestra.run.smithi118.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 cephadm amd64 15.2.6-595-ga83a3572-1bionic [51.3 kB] 2020-11-28T21:43:24.090 INFO:teuthology.orchestra.run.smithi118.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-cephadm all 15.2.6-595-ga83a3572-1bionic [55.3 kB] 2020-11-28T21:43:24.093 INFO:teuthology.orchestra.run.smithi118.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-dashboard all 15.2.6-595-ga83a3572-1bionic [3,177 kB] 2020-11-28T21:43:24.332 INFO:teuthology.orchestra.run.smithi118.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-cloud all 15.2.6-595-ga83a3572-1bionic [34.2 kB] 2020-11-28T21:43:24.333 INFO:teuthology.orchestra.run.smithi118.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-local all 15.2.6-595-ga83a3572-1bionic [7,886 kB] 2020-11-28T21:43:24.846 INFO:teuthology.orchestra.run.smithi162.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 15.2.6-595-ga83a3572-1bionic [3,892 B] 2020-11-28T21:43:24.846 INFO:teuthology.orchestra.run.smithi162.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 15.2.6-595-ga83a3572-1bionic [515 kB] 2020-11-28T21:43:24.906 INFO:teuthology.orchestra.run.smithi162.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 15.2.6-595-ga83a3572-1bionic [1,850 kB] 2020-11-28T21:43:24.949 INFO:teuthology.orchestra.run.smithi118.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-k8sevents all 15.2.6-595-ga83a3572-1bionic [4,008 B] 2020-11-28T21:43:24.949 INFO:teuthology.orchestra.run.smithi118.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-rook all 15.2.6-595-ga83a3572-1bionic [22.4 kB] 2020-11-28T21:43:24.949 INFO:teuthology.orchestra.run.smithi118.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 15.2.6-595-ga83a3572-1bionic [43.9 MB] 2020-11-28T21:43:25.185 INFO:teuthology.orchestra.run.smithi162.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 cephadm amd64 15.2.6-595-ga83a3572-1bionic [51.3 kB] 2020-11-28T21:43:25.186 INFO:teuthology.orchestra.run.smithi162.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-cephadm all 15.2.6-595-ga83a3572-1bionic [55.3 kB] 2020-11-28T21:43:25.187 INFO:teuthology.orchestra.run.smithi162.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-dashboard all 15.2.6-595-ga83a3572-1bionic [3,177 kB] 2020-11-28T21:43:25.533 INFO:teuthology.orchestra.run.smithi162.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-cloud all 15.2.6-595-ga83a3572-1bionic [34.2 kB] 2020-11-28T21:43:25.536 INFO:teuthology.orchestra.run.smithi162.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-local all 15.2.6-595-ga83a3572-1bionic [7,886 kB] 2020-11-28T21:43:26.133 INFO:teuthology.orchestra.run.smithi096.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 15.2.6-595-ga83a3572-1bionic [5,170 kB] 2020-11-28T21:43:26.582 INFO:teuthology.orchestra.run.smithi162.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-k8sevents all 15.2.6-595-ga83a3572-1bionic [4,008 B] 2020-11-28T21:43:26.586 INFO:teuthology.orchestra.run.smithi162.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-rook all 15.2.6-595-ga83a3572-1bionic [22.4 kB] 2020-11-28T21:43:26.589 INFO:teuthology.orchestra.run.smithi162.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 15.2.6-595-ga83a3572-1bionic [43.9 MB] 2020-11-28T21:43:27.054 INFO:teuthology.orchestra.run.smithi096.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-modules-core all 15.2.6-595-ga83a3572-1bionic [148 kB] 2020-11-28T21:43:27.060 INFO:teuthology.orchestra.run.smithi096.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 15.2.6-595-ga83a3572-1bionic [1,282 kB] 2020-11-28T21:43:27.294 INFO:teuthology.orchestra.run.smithi096.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 15.2.6-595-ga83a3572-1bionic [5,944 kB] 2020-11-28T21:43:27.984 INFO:teuthology.orchestra.run.smithi118.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 cephfs-shell all 15.2.6-595-ga83a3572-1bionic [15.7 kB] 2020-11-28T21:43:27.985 INFO:teuthology.orchestra.run.smithi118.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 15.2.6-595-ga83a3572-1bionic [17.9 kB] 2020-11-28T21:43:27.985 INFO:teuthology.orchestra.run.smithi118.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 15.2.6-595-ga83a3572-1bionic [8,846 kB] 2020-11-28T21:43:28.395 INFO:teuthology.orchestra.run.smithi096.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 15.2.6-595-ga83a3572-1bionic [22.7 MB] 2020-11-28T21:43:28.614 INFO:teuthology.orchestra.run.smithi118.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 15.2.6-595-ga83a3572-1bionic [62.5 kB] 2020-11-28T21:43:28.790 INFO:teuthology.orchestra.run.smithi192.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 15.2.6-595-ga83a3572-1bionic [5,170 kB] 2020-11-28T21:43:29.322 INFO:teuthology.orchestra.run.smithi118.stdout:Fetched 156 MB in 10s (16.0 MB/s) 2020-11-28T21:43:29.479 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-pyparsing. 2020-11-28T21:43:29.518 INFO:teuthology.orchestra.run.smithi118.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102077 files and directories currently installed.) 2020-11-28T21:43:29.521 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../000-python3-pyparsing_2.2.0+dfsg1-2_all.deb ... 2020-11-28T21:43:29.561 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-pyparsing (2.2.0+dfsg1-2) ... 2020-11-28T21:43:29.733 INFO:teuthology.orchestra.run.smithi192.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-modules-core all 15.2.6-595-ga83a3572-1bionic [148 kB] 2020-11-28T21:43:29.739 INFO:teuthology.orchestra.run.smithi192.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 15.2.6-595-ga83a3572-1bionic [1,282 kB] 2020-11-28T21:43:29.839 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-pyperclip. 2020-11-28T21:43:29.851 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../001-python3-pyperclip_1.6.0-1_all.deb ... 2020-11-28T21:43:29.880 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-pyperclip (1.6.0-1) ... 2020-11-28T21:43:29.994 INFO:teuthology.orchestra.run.smithi192.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 15.2.6-595-ga83a3572-1bionic [5,944 kB] 2020-11-28T21:43:30.125 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-cmd2. 2020-11-28T21:43:30.135 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../002-python3-cmd2_0.7.9-0ubuntu1_all.deb ... 2020-11-28T21:43:30.166 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-cmd2 (0.7.9-0ubuntu1) ... 2020-11-28T21:43:30.485 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-uritemplate. 2020-11-28T21:43:30.497 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../003-python3-uritemplate_0.6-1ubuntu1_all.deb ... 2020-11-28T21:43:30.527 INFO:teuthology.orchestra.run.smithi162.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 cephfs-shell all 15.2.6-595-ga83a3572-1bionic [15.7 kB] 2020-11-28T21:43:30.528 INFO:teuthology.orchestra.run.smithi162.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 15.2.6-595-ga83a3572-1bionic [17.9 kB] 2020-11-28T21:43:30.528 INFO:teuthology.orchestra.run.smithi162.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 15.2.6-595-ga83a3572-1bionic [8,846 kB] 2020-11-28T21:43:30.535 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-uritemplate (0.6-1ubuntu1) ... 2020-11-28T21:43:30.744 INFO:teuthology.orchestra.run.smithi096.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 15.2.6-595-ga83a3572-1bionic [3,892 B] 2020-11-28T21:43:30.745 INFO:teuthology.orchestra.run.smithi096.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 15.2.6-595-ga83a3572-1bionic [515 kB] 2020-11-28T21:43:30.788 INFO:teuthology.orchestra.run.smithi096.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 15.2.6-595-ga83a3572-1bionic [1,850 kB] 2020-11-28T21:43:30.846 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2020-11-28T21:43:30.858 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../004-liblttng-ust0_2.10.1-1_amd64.deb ... 2020-11-28T21:43:30.938 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2020-11-28T21:43:30.967 INFO:teuthology.orchestra.run.smithi096.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 cephadm amd64 15.2.6-595-ga83a3572-1bionic [51.3 kB] 2020-11-28T21:43:30.969 INFO:teuthology.orchestra.run.smithi096.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-cephadm all 15.2.6-595-ga83a3572-1bionic [55.3 kB] 2020-11-28T21:43:30.971 INFO:teuthology.orchestra.run.smithi096.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-dashboard all 15.2.6-595-ga83a3572-1bionic [3,177 kB] 2020-11-28T21:43:31.069 INFO:teuthology.orchestra.run.smithi192.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 15.2.6-595-ga83a3572-1bionic [22.7 MB] 2020-11-28T21:43:31.129 INFO:teuthology.orchestra.run.smithi162.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 15.2.6-595-ga83a3572-1bionic [62.5 kB] 2020-11-28T21:43:31.209 INFO:teuthology.orchestra.run.smithi096.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-cloud all 15.2.6-595-ga83a3572-1bionic [34.2 kB] 2020-11-28T21:43:31.210 INFO:teuthology.orchestra.run.smithi096.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-local all 15.2.6-595-ga83a3572-1bionic [7,886 kB] 2020-11-28T21:43:31.412 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../005-librbd1_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:31.534 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking librbd1 (15.2.6-595-ga83a3572-1bionic) over (12.2.13-0ubuntu0.18.04.5) ... 2020-11-28T21:43:31.617 INFO:teuthology.orchestra.run.smithi096.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-k8sevents all 15.2.6-595-ga83a3572-1bionic [4,008 B] 2020-11-28T21:43:31.617 INFO:teuthology.orchestra.run.smithi096.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-rook all 15.2.6-595-ga83a3572-1bionic [22.4 kB] 2020-11-28T21:43:31.618 INFO:teuthology.orchestra.run.smithi096.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 15.2.6-595-ga83a3572-1bionic [43.9 MB] 2020-11-28T21:43:31.834 INFO:teuthology.orchestra.run.smithi162.stdout:Fetched 156 MB in 12s (12.7 MB/s) 2020-11-28T21:43:32.005 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-pyparsing. 2020-11-28T21:43:32.040 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../006-librados2_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:32.044 INFO:teuthology.orchestra.run.smithi162.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102077 files and directories currently installed.) 2020-11-28T21:43:32.047 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../000-python3-pyparsing_2.2.0+dfsg1-2_all.deb ... 2020-11-28T21:43:32.087 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-pyparsing (2.2.0+dfsg1-2) ... 2020-11-28T21:43:32.154 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking librados2 (15.2.6-595-ga83a3572-1bionic) over (12.2.13-0ubuntu0.18.04.5) ... 2020-11-28T21:43:32.398 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-pyperclip. 2020-11-28T21:43:32.410 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../001-python3-pyperclip_1.6.0-1_all.deb ... 2020-11-28T21:43:32.448 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-pyperclip (1.6.0-1) ... 2020-11-28T21:43:32.744 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-cmd2. 2020-11-28T21:43:32.756 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../002-python3-cmd2_0.7.9-0ubuntu1_all.deb ... 2020-11-28T21:43:32.783 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libcephfs2. 2020-11-28T21:43:32.794 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-cmd2 (0.7.9-0ubuntu1) ... 2020-11-28T21:43:32.795 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../007-libcephfs2_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:32.833 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libcephfs2 (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:33.097 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-uritemplate. 2020-11-28T21:43:33.108 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../003-python3-uritemplate_0.6-1ubuntu1_all.deb ... 2020-11-28T21:43:33.128 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-rados. 2020-11-28T21:43:33.139 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../008-python3-rados_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:33.147 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-uritemplate (0.6-1ubuntu1) ... 2020-11-28T21:43:33.177 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-rados (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:33.491 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2020-11-28T21:43:33.503 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../004-liblttng-ust0_2.10.1-1_amd64.deb ... 2020-11-28T21:43:33.529 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-ceph-argparse. 2020-11-28T21:43:33.541 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../009-python3-ceph-argparse_15.2.6-595-ga83a3572-1bionic_all.deb ... 2020-11-28T21:43:33.575 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2020-11-28T21:43:33.579 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-ceph-argparse (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:33.857 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-cephfs. 2020-11-28T21:43:33.869 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../010-python3-cephfs_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:33.882 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../005-librbd1_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:33.907 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-cephfs (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:33.978 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking librbd1 (15.2.6-595-ga83a3572-1bionic) over (12.2.13-0ubuntu0.18.04.5) ... 2020-11-28T21:43:34.226 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-ceph-common. 2020-11-28T21:43:34.239 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../011-python3-ceph-common_15.2.6-595-ga83a3572-1bionic_all.deb ... 2020-11-28T21:43:34.276 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-ceph-common (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:34.519 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../006-librados2_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:34.595 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-prettytable. 2020-11-28T21:43:34.606 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../012-python3-prettytable_0.7.2-3_all.deb ... 2020-11-28T21:43:34.640 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking librados2 (15.2.6-595-ga83a3572-1bionic) over (12.2.13-0ubuntu0.18.04.5) ... 2020-11-28T21:43:34.645 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-prettytable (0.7.2-3) ... 2020-11-28T21:43:34.881 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-rbd. 2020-11-28T21:43:34.893 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../013-python3-rbd_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:34.931 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-rbd (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:35.242 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package librabbitmq4:amd64. 2020-11-28T21:43:35.252 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libcephfs2. 2020-11-28T21:43:35.254 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../014-librabbitmq4_0.8.0-1ubuntu0.18.04.2_amd64.deb ... 2020-11-28T21:43:35.263 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../007-libcephfs2_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:35.294 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libcephfs2 (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:35.312 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2020-11-28T21:43:35.580 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-rados. 2020-11-28T21:43:35.592 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../008-python3-rados_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:35.630 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-rados (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:35.653 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package librdkafka1:amd64. 2020-11-28T21:43:35.666 INFO:teuthology.orchestra.run.smithi192.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 15.2.6-595-ga83a3572-1bionic [3,892 B] 2020-11-28T21:43:35.666 INFO:teuthology.orchestra.run.smithi192.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 15.2.6-595-ga83a3572-1bionic [515 kB] 2020-11-28T21:43:35.667 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../015-librdkafka1_0.11.3-1build1_amd64.deb ... 2020-11-28T21:43:35.704 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking librdkafka1:amd64 (0.11.3-1build1) ... 2020-11-28T21:43:35.812 INFO:teuthology.orchestra.run.smithi192.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 15.2.6-595-ga83a3572-1bionic [1,850 kB] 2020-11-28T21:43:35.974 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-ceph-argparse. 2020-11-28T21:43:35.986 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../009-python3-ceph-argparse_15.2.6-595-ga83a3572-1bionic_all.deb ... 2020-11-28T21:43:36.024 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-ceph-argparse (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:36.031 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package librgw2. 2020-11-28T21:43:36.044 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../016-librgw2_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:36.082 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking librgw2 (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:36.310 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-cephfs. 2020-11-28T21:43:36.321 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../010-python3-cephfs_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:36.330 INFO:teuthology.orchestra.run.smithi192.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 cephadm amd64 15.2.6-595-ga83a3572-1bionic [51.3 kB] 2020-11-28T21:43:36.353 INFO:teuthology.orchestra.run.smithi192.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-cephadm all 15.2.6-595-ga83a3572-1bionic [55.3 kB] 2020-11-28T21:43:36.357 INFO:teuthology.orchestra.run.smithi192.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-dashboard all 15.2.6-595-ga83a3572-1bionic [3,177 kB] 2020-11-28T21:43:36.360 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-cephfs (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:36.583 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-rgw. 2020-11-28T21:43:36.596 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../017-python3-rgw_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:36.625 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-rgw (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:36.654 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-ceph-common. 2020-11-28T21:43:36.667 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../011-python3-ceph-common_15.2.6-595-ga83a3572-1bionic_all.deb ... 2020-11-28T21:43:36.696 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-ceph-common (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:36.929 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2020-11-28T21:43:36.940 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../018-libsnappy1v5_1.1.7-1_amd64.deb ... 2020-11-28T21:43:36.980 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2020-11-28T21:43:37.007 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-prettytable. 2020-11-28T21:43:37.019 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../012-python3-prettytable_0.7.2-3_all.deb ... 2020-11-28T21:43:37.057 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-prettytable (0.7.2-3) ... 2020-11-28T21:43:37.240 INFO:teuthology.orchestra.run.smithi192.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-cloud all 15.2.6-595-ga83a3572-1bionic [34.2 kB] 2020-11-28T21:43:37.248 INFO:teuthology.orchestra.run.smithi192.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-local all 15.2.6-595-ga83a3572-1bionic [7,886 kB] 2020-11-28T21:43:37.308 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2020-11-28T21:43:37.317 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-rbd. 2020-11-28T21:43:37.320 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../019-libleveldb1v5_1.20-2_amd64.deb ... 2020-11-28T21:43:37.329 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../013-python3-rbd_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:37.357 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2020-11-28T21:43:37.368 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-rbd (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:37.685 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package liboath0. 2020-11-28T21:43:37.698 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../020-liboath0_2.6.1-1_amd64.deb ... 2020-11-28T21:43:37.728 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package librabbitmq4:amd64. 2020-11-28T21:43:37.735 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking liboath0 (2.6.1-1) ... 2020-11-28T21:43:37.739 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../014-librabbitmq4_0.8.0-1ubuntu0.18.04.2_amd64.deb ... 2020-11-28T21:43:37.778 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2020-11-28T21:43:38.047 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libradosstriper1. 2020-11-28T21:43:38.060 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../021-libradosstriper1_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:38.097 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libradosstriper1 (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:38.098 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package librdkafka1:amd64. 2020-11-28T21:43:38.109 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../015-librdkafka1_0.11.3-1build1_amd64.deb ... 2020-11-28T21:43:38.140 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking librdkafka1:amd64 (0.11.3-1build1) ... 2020-11-28T21:43:38.476 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package librgw2. 2020-11-28T21:43:38.487 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../016-librgw2_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:38.526 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking librgw2 (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:38.533 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-common. 2020-11-28T21:43:38.546 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../022-ceph-common_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:38.584 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-common (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:39.053 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-rgw. 2020-11-28T21:43:39.122 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../017-python3-rgw_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:39.122 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-rgw (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:39.380 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2020-11-28T21:43:39.393 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../018-libsnappy1v5_1.1.7-1_amd64.deb ... 2020-11-28T21:43:39.408 INFO:teuthology.orchestra.run.smithi192.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-k8sevents all 15.2.6-595-ga83a3572-1bionic [4,008 B] 2020-11-28T21:43:39.409 INFO:teuthology.orchestra.run.smithi192.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-rook all 15.2.6-595-ga83a3572-1bionic [22.4 kB] 2020-11-28T21:43:39.409 INFO:teuthology.orchestra.run.smithi192.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 15.2.6-595-ga83a3572-1bionic [43.9 MB] 2020-11-28T21:43:39.422 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2020-11-28T21:43:39.733 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2020-11-28T21:43:39.745 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../019-libleveldb1v5_1.20-2_amd64.deb ... 2020-11-28T21:43:39.783 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2020-11-28T21:43:40.122 INFO:teuthology.orchestra.run.smithi096.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 cephfs-shell all 15.2.6-595-ga83a3572-1bionic [15.7 kB] 2020-11-28T21:43:40.123 INFO:teuthology.orchestra.run.smithi096.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 15.2.6-595-ga83a3572-1bionic [17.9 kB] 2020-11-28T21:43:40.125 INFO:teuthology.orchestra.run.smithi096.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 15.2.6-595-ga83a3572-1bionic [8,846 kB] 2020-11-28T21:43:40.161 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package liboath0. 2020-11-28T21:43:40.173 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../020-liboath0_2.6.1-1_amd64.deb ... 2020-11-28T21:43:40.211 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking liboath0 (2.6.1-1) ... 2020-11-28T21:43:40.365 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-base. 2020-11-28T21:43:40.376 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../023-ceph-base_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:40.449 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-base (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:40.506 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libradosstriper1. 2020-11-28T21:43:40.517 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../021-libradosstriper1_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:40.555 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libradosstriper1 (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:40.925 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-common. 2020-11-28T21:43:40.937 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../022-ceph-common_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:41.000 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-common (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:41.020 INFO:teuthology.orchestra.run.smithi096.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 15.2.6-595-ga83a3572-1bionic [62.5 kB] 2020-11-28T21:43:41.302 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-dateutil. 2020-11-28T21:43:41.315 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../024-python3-dateutil_2.6.1-1_all.deb ... 2020-11-28T21:43:41.352 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-dateutil (2.6.1-1) ... 2020-11-28T21:43:41.681 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-asn1crypto. 2020-11-28T21:43:41.693 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../025-python3-asn1crypto_0.24.0-1_all.deb ... 2020-11-28T21:43:41.731 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-asn1crypto (0.24.0-1) ... 2020-11-28T21:43:41.768 INFO:teuthology.orchestra.run.smithi096.stdout:Fetched 156 MB in 22s (7,071 kB/s) 2020-11-28T21:43:41.912 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pyparsing. 2020-11-28T21:43:41.947 INFO:teuthology.orchestra.run.smithi096.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102077 files and directories currently installed.) 2020-11-28T21:43:41.949 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../000-python3-pyparsing_2.2.0+dfsg1-2_all.deb ... 2020-11-28T21:43:41.993 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pyparsing (2.2.0+dfsg1-2) ... 2020-11-28T21:43:42.000 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-cffi-backend. 2020-11-28T21:43:42.013 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../026-python3-cffi-backend_1.11.5-1_amd64.deb ... 2020-11-28T21:43:42.050 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-cffi-backend (1.11.5-1) ... 2020-11-28T21:43:42.255 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pyperclip. 2020-11-28T21:43:42.267 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../001-python3-pyperclip_1.6.0-1_all.deb ... 2020-11-28T21:43:42.296 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pyperclip (1.6.0-1) ... 2020-11-28T21:43:42.381 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-cryptography. 2020-11-28T21:43:42.393 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../027-python3-cryptography_2.1.4-1ubuntu1.4_amd64.deb ... 2020-11-28T21:43:42.436 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-cryptography (2.1.4-1ubuntu1.4) ... 2020-11-28T21:43:42.616 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-cmd2. 2020-11-28T21:43:42.627 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../002-python3-cmd2_0.7.9-0ubuntu1_all.deb ... 2020-11-28T21:43:42.666 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-cmd2 (0.7.9-0ubuntu1) ... 2020-11-28T21:43:42.797 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-base. 2020-11-28T21:43:42.807 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-openssl. 2020-11-28T21:43:42.808 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../023-ceph-base_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:42.819 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../028-python3-openssl_17.5.0-1ubuntu1_all.deb ... 2020-11-28T21:43:42.855 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-base (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:42.857 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-openssl (17.5.0-1ubuntu1) ... 2020-11-28T21:43:42.985 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-uritemplate. 2020-11-28T21:43:42.998 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../003-python3-uritemplate_0.6-1ubuntu1_all.deb ... 2020-11-28T21:43:43.035 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-uritemplate (0.6-1ubuntu1) ... 2020-11-28T21:43:43.076 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2020-11-28T21:43:43.088 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../029-ceph-mgr-modules-core_15.2.6-595-ga83a3572-1bionic_all.deb ... 2020-11-28T21:43:43.118 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-mgr-modules-core (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:43.207 INFO:teuthology.orchestra.run.smithi192.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 cephfs-shell all 15.2.6-595-ga83a3572-1bionic [15.7 kB] 2020-11-28T21:43:43.208 INFO:teuthology.orchestra.run.smithi192.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 15.2.6-595-ga83a3572-1bionic [17.9 kB] 2020-11-28T21:43:43.208 INFO:teuthology.orchestra.run.smithi192.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 15.2.6-595-ga83a3572-1bionic [8,846 kB] 2020-11-28T21:43:43.297 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2020-11-28T21:43:43.307 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../004-liblttng-ust0_2.10.1-1_amd64.deb ... 2020-11-28T21:43:43.372 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2020-11-28T21:43:43.456 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-bcrypt. 2020-11-28T21:43:43.467 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../030-python3-bcrypt_3.1.4-2_amd64.deb ... 2020-11-28T21:43:43.506 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-bcrypt (3.1.4-2) ... 2020-11-28T21:43:43.676 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-dateutil. 2020-11-28T21:43:43.689 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../024-python3-dateutil_2.6.1-1_all.deb ... 2020-11-28T21:43:43.695 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../005-librbd1_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:43.717 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-dateutil (2.6.1-1) ... 2020-11-28T21:43:43.784 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-cherrypy3. 2020-11-28T21:43:43.796 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../031-python3-cherrypy3_8.9.1-2_all.deb ... 2020-11-28T21:43:43.801 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking librbd1 (15.2.6-595-ga83a3572-1bionic) over (12.2.13-0ubuntu0.18.04.5) ... 2020-11-28T21:43:43.834 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-cherrypy3 (8.9.1-2) ... 2020-11-28T21:43:43.844 INFO:teuthology.orchestra.run.smithi192.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/octopus/a83a3572a957356977342f461361c1cd3ec06dbd/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 15.2.6-595-ga83a3572-1bionic [62.5 kB] 2020-11-28T21:43:44.046 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-asn1crypto. 2020-11-28T21:43:44.058 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../025-python3-asn1crypto_0.24.0-1_all.deb ... 2020-11-28T21:43:44.095 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-asn1crypto (0.24.0-1) ... 2020-11-28T21:43:44.171 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-jwt. 2020-11-28T21:43:44.183 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../032-python3-jwt_1.5.3+ds1-1_all.deb ... 2020-11-28T21:43:44.229 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-jwt (1.5.3+ds1-1) ... 2020-11-28T21:43:44.324 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../006-librados2_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:44.373 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-cffi-backend. 2020-11-28T21:43:44.383 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../026-python3-cffi-backend_1.11.5-1_amd64.deb ... 2020-11-28T21:43:44.423 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-cffi-backend (1.11.5-1) ... 2020-11-28T21:43:44.447 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking librados2 (15.2.6-595-ga83a3572-1bionic) over (12.2.13-0ubuntu0.18.04.5) ... 2020-11-28T21:43:44.524 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-logutils. 2020-11-28T21:43:44.537 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../033-python3-logutils_0.3.3-5_all.deb ... 2020-11-28T21:43:44.571 INFO:teuthology.orchestra.run.smithi192.stdout:Fetched 156 MB in 25s (6,263 kB/s) 2020-11-28T21:43:44.575 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-logutils (0.3.3-5) ... 2020-11-28T21:43:44.734 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-cryptography. 2020-11-28T21:43:44.742 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-pyparsing. 2020-11-28T21:43:44.746 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../027-python3-cryptography_2.1.4-1ubuntu1.4_amd64.deb ... 2020-11-28T21:43:44.779 INFO:teuthology.orchestra.run.smithi192.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102077 files and directories currently installed.) 2020-11-28T21:43:44.782 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../000-python3-pyparsing_2.2.0+dfsg1-2_all.deb ... 2020-11-28T21:43:44.783 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-cryptography (2.1.4-1ubuntu1.4) ... 2020-11-28T21:43:44.815 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-pyparsing (2.2.0+dfsg1-2) ... 2020-11-28T21:43:44.877 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-markupsafe. 2020-11-28T21:43:44.888 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../034-python3-markupsafe_1.0-1build1_amd64.deb ... 2020-11-28T21:43:44.927 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-markupsafe (1.0-1build1) ... 2020-11-28T21:43:45.060 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libcephfs2. 2020-11-28T21:43:45.070 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../007-libcephfs2_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:45.101 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libcephfs2 (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:45.118 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-pyperclip. 2020-11-28T21:43:45.128 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-openssl. 2020-11-28T21:43:45.130 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../001-python3-pyperclip_1.6.0-1_all.deb ... 2020-11-28T21:43:45.139 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../028-python3-openssl_17.5.0-1ubuntu1_all.deb ... 2020-11-28T21:43:45.168 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-pyperclip (1.6.0-1) ... 2020-11-28T21:43:45.178 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-openssl (17.5.0-1ubuntu1) ... 2020-11-28T21:43:45.222 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-mako. 2020-11-28T21:43:45.234 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../035-python3-mako_1.0.7+ds1-1_all.deb ... 2020-11-28T21:43:45.272 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-mako (1.0.7+ds1-1) ... 2020-11-28T21:43:45.395 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-rados. 2020-11-28T21:43:45.406 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../008-python3-rados_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:45.414 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2020-11-28T21:43:45.426 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../029-ceph-mgr-modules-core_15.2.6-595-ga83a3572-1bionic_all.deb ... 2020-11-28T21:43:45.445 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-rados (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:45.462 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-cmd2. 2020-11-28T21:43:45.474 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../002-python3-cmd2_0.7.9-0ubuntu1_all.deb ... 2020-11-28T21:43:45.486 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-mgr-modules-core (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:45.512 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-cmd2 (0.7.9-0ubuntu1) ... 2020-11-28T21:43:45.592 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-simplegeneric. 2020-11-28T21:43:45.604 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../036-python3-simplegeneric_0.8.1-1_all.deb ... 2020-11-28T21:43:45.642 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-simplegeneric (0.8.1-1) ... 2020-11-28T21:43:45.790 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-ceph-argparse. 2020-11-28T21:43:45.802 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../009-python3-ceph-argparse_15.2.6-595-ga83a3572-1bionic_all.deb ... 2020-11-28T21:43:45.826 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-bcrypt. 2020-11-28T21:43:45.837 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../030-python3-bcrypt_3.1.4-2_amd64.deb ... 2020-11-28T21:43:45.841 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-ceph-argparse (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:45.848 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-uritemplate. 2020-11-28T21:43:45.859 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../003-python3-uritemplate_0.6-1ubuntu1_all.deb ... 2020-11-28T21:43:45.869 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-singledispatch. 2020-11-28T21:43:45.876 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-bcrypt (3.1.4-2) ... 2020-11-28T21:43:45.882 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../037-python3-singledispatch_3.4.0.3-2_all.deb ... 2020-11-28T21:43:45.898 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-uritemplate (0.6-1ubuntu1) ... 2020-11-28T21:43:45.911 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-singledispatch (3.4.0.3-2) ... 2020-11-28T21:43:46.134 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-cephfs. 2020-11-28T21:43:46.137 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-cherrypy3. 2020-11-28T21:43:46.146 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../010-python3-cephfs_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:46.148 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../031-python3-cherrypy3_8.9.1-2_all.deb ... 2020-11-28T21:43:46.152 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2020-11-28T21:43:46.161 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../004-liblttng-ust0_2.10.1-1_amd64.deb ... 2020-11-28T21:43:46.178 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-cherrypy3 (8.9.1-2) ... 2020-11-28T21:43:46.184 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-cephfs (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:46.214 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-webob. 2020-11-28T21:43:46.226 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../038-python3-webob_1%3a1.7.3-2fakesync1_all.deb ... 2020-11-28T21:43:46.242 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2020-11-28T21:43:46.264 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-webob (1:1.7.3-2fakesync1) ... 2020-11-28T21:43:46.478 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-ceph-common. 2020-11-28T21:43:46.491 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../011-python3-ceph-common_15.2.6-595-ga83a3572-1bionic_all.deb ... 2020-11-28T21:43:46.528 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-ceph-common (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:46.574 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-jwt. 2020-11-28T21:43:46.586 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../032-python3-jwt_1.5.3+ds1-1_all.deb ... 2020-11-28T21:43:46.591 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../005-librbd1_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:46.623 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-jwt (1.5.3+ds1-1) ... 2020-11-28T21:43:46.634 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-bs4. 2020-11-28T21:43:46.646 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../039-python3-bs4_4.6.0-1_all.deb ... 2020-11-28T21:43:46.684 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-bs4 (4.6.0-1) ... 2020-11-28T21:43:46.713 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking librbd1 (15.2.6-595-ga83a3572-1bionic) over (12.2.13-0ubuntu0.18.04.5) ... 2020-11-28T21:43:46.823 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-prettytable. 2020-11-28T21:43:46.836 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../012-python3-prettytable_0.7.2-3_all.deb ... 2020-11-28T21:43:46.873 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-prettytable (0.7.2-3) ... 2020-11-28T21:43:46.902 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-logutils. 2020-11-28T21:43:46.913 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../033-python3-logutils_0.3.3-5_all.deb ... 2020-11-28T21:43:46.952 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-logutils (0.3.3-5) ... 2020-11-28T21:43:46.995 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-waitress. 2020-11-28T21:43:47.007 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../040-python3-waitress_1.0.1-1_all.deb ... 2020-11-28T21:43:47.045 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-waitress (1.0.1-1) ... 2020-11-28T21:43:47.126 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-rbd. 2020-11-28T21:43:47.138 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../013-python3-rbd_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:47.176 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-rbd (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:47.262 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../006-librados2_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:47.271 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-markupsafe. 2020-11-28T21:43:47.283 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../034-python3-markupsafe_1.0-1build1_amd64.deb ... 2020-11-28T21:43:47.322 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-markupsafe (1.0-1build1) ... 2020-11-28T21:43:47.374 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-tempita. 2020-11-28T21:43:47.385 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../041-python3-tempita_0.5.2-2_all.deb ... 2020-11-28T21:43:47.400 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking librados2 (15.2.6-595-ga83a3572-1bionic) over (12.2.13-0ubuntu0.18.04.5) ... 2020-11-28T21:43:47.424 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-tempita (0.5.2-2) ... 2020-11-28T21:43:47.528 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package librabbitmq4:amd64. 2020-11-28T21:43:47.541 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../014-librabbitmq4_0.8.0-1ubuntu0.18.04.2_amd64.deb ... 2020-11-28T21:43:47.570 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2020-11-28T21:43:47.623 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-mako. 2020-11-28T21:43:47.636 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../035-python3-mako_1.0.7+ds1-1_all.deb ... 2020-11-28T21:43:47.665 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-mako (1.0.7+ds1-1) ... 2020-11-28T21:43:47.677 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-paste. 2020-11-28T21:43:47.688 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../042-python3-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2020-11-28T21:43:47.727 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-paste (2.0.3+dfsg-4ubuntu1) ... 2020-11-28T21:43:47.898 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package librdkafka1:amd64. 2020-11-28T21:43:47.912 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../015-librdkafka1_0.11.3-1build1_amd64.deb ... 2020-11-28T21:43:47.948 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking librdkafka1:amd64 (0.11.3-1build1) ... 2020-11-28T21:43:47.977 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-simplegeneric. 2020-11-28T21:43:47.989 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../036-python3-simplegeneric_0.8.1-1_all.deb ... 2020-11-28T21:43:48.004 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package libcephfs2. 2020-11-28T21:43:48.016 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../007-libcephfs2_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:48.022 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2020-11-28T21:43:48.026 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-simplegeneric (0.8.1-1) ... 2020-11-28T21:43:48.034 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../043-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2020-11-28T21:43:48.053 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking libcephfs2 (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:48.071 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2020-11-28T21:43:48.287 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-singledispatch. 2020-11-28T21:43:48.299 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../037-python3-singledispatch_3.4.0.3-2_all.deb ... 2020-11-28T21:43:48.310 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package librgw2. 2020-11-28T21:43:48.320 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../016-librgw2_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:48.329 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-singledispatch (3.4.0.3-2) ... 2020-11-28T21:43:48.349 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-pastedeploy. 2020-11-28T21:43:48.359 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking librgw2 (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:48.362 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../044-python3-pastedeploy_1.5.2-4_all.deb ... 2020-11-28T21:43:48.381 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-rados. 2020-11-28T21:43:48.394 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../008-python3-rados_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:48.399 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-pastedeploy (1.5.2-4) ... 2020-11-28T21:43:48.431 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-rados (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:48.648 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-webob. 2020-11-28T21:43:48.660 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../038-python3-webob_1%3a1.7.3-2fakesync1_all.deb ... 2020-11-28T21:43:48.698 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-webob (1:1.7.3-2fakesync1) ... 2020-11-28T21:43:48.702 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-webtest. 2020-11-28T21:43:48.714 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../045-python3-webtest_2.0.28-1ubuntu1_all.deb ... 2020-11-28T21:43:48.752 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-webtest (2.0.28-1ubuntu1) ... 2020-11-28T21:43:48.758 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-ceph-argparse. 2020-11-28T21:43:48.771 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../009-python3-ceph-argparse_15.2.6-595-ga83a3572-1bionic_all.deb ... 2020-11-28T21:43:48.809 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-ceph-argparse (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:48.854 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-rgw. 2020-11-28T21:43:48.871 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../017-python3-rgw_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:48.903 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-rgw (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:49.009 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-bs4. 2020-11-28T21:43:49.021 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../039-python3-bs4_4.6.0-1_all.deb ... 2020-11-28T21:43:49.058 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-pecan. 2020-11-28T21:43:49.059 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-bs4 (4.6.0-1) ... 2020-11-28T21:43:49.070 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../046-python3-pecan_1.2.1-2_all.deb ... 2020-11-28T21:43:49.078 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-cephfs. 2020-11-28T21:43:49.091 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../010-python3-cephfs_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:49.099 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-pecan (1.2.1-2) ... 2020-11-28T21:43:49.128 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-cephfs (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:49.223 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2020-11-28T21:43:49.235 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../018-libsnappy1v5_1.1.7-1_amd64.deb ... 2020-11-28T21:43:49.273 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2020-11-28T21:43:49.378 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-waitress. 2020-11-28T21:43:49.391 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../040-python3-waitress_1.0.1-1_all.deb ... 2020-11-28T21:43:49.413 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-werkzeug. 2020-11-28T21:43:49.425 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../047-python3-werkzeug_0.14.1+dfsg1-1_all.deb ... 2020-11-28T21:43:49.428 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-waitress (1.0.1-1) ... 2020-11-28T21:43:49.447 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-ceph-common. 2020-11-28T21:43:49.454 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-werkzeug (0.14.1+dfsg1-1) ... 2020-11-28T21:43:49.458 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../011-python3-ceph-common_15.2.6-595-ga83a3572-1bionic_all.deb ... 2020-11-28T21:43:49.497 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-ceph-common (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:49.576 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2020-11-28T21:43:49.588 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../019-libleveldb1v5_1.20-2_amd64.deb ... 2020-11-28T21:43:49.626 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2020-11-28T21:43:49.748 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-tempita. 2020-11-28T21:43:49.761 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../041-python3-tempita_0.5.2-2_all.deb ... 2020-11-28T21:43:49.774 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-mgr. 2020-11-28T21:43:49.785 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../048-ceph-mgr_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:49.791 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-prettytable. 2020-11-28T21:43:49.798 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-tempita (0.5.2-2) ... 2020-11-28T21:43:49.804 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../012-python3-prettytable_0.7.2-3_all.deb ... 2020-11-28T21:43:49.816 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-mgr (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:49.842 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-prettytable (0.7.2-3) ... 2020-11-28T21:43:49.987 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package liboath0. 2020-11-28T21:43:50.000 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../020-liboath0_2.6.1-1_amd64.deb ... 2020-11-28T21:43:50.037 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking liboath0 (2.6.1-1) ... 2020-11-28T21:43:50.093 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-paste. 2020-11-28T21:43:50.105 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../042-python3-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2020-11-28T21:43:50.110 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-rbd. 2020-11-28T21:43:50.122 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../013-python3-rbd_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:50.134 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-paste (2.0.3+dfsg-4ubuntu1) ... 2020-11-28T21:43:50.160 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-rbd (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:50.245 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-mon. 2020-11-28T21:43:50.256 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../049-ceph-mon_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:50.294 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-mon (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:50.331 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libradosstriper1. 2020-11-28T21:43:50.344 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../021-libradosstriper1_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:50.381 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libradosstriper1 (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:50.455 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package librabbitmq4:amd64. 2020-11-28T21:43:50.462 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2020-11-28T21:43:50.468 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../014-librabbitmq4_0.8.0-1ubuntu0.18.04.2_amd64.deb ... 2020-11-28T21:43:50.474 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../043-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2020-11-28T21:43:50.496 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2020-11-28T21:43:50.512 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2020-11-28T21:43:50.814 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-pastedeploy. 2020-11-28T21:43:50.826 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../044-python3-pastedeploy_1.5.2-4_all.deb ... 2020-11-28T21:43:50.849 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package librdkafka1:amd64. 2020-11-28T21:43:50.862 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../015-librdkafka1_0.11.3-1build1_amd64.deb ... 2020-11-28T21:43:50.864 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-pastedeploy (1.5.2-4) ... 2020-11-28T21:43:50.899 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking librdkafka1:amd64 (0.11.3-1build1) ... 2020-11-28T21:43:51.174 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-webtest. 2020-11-28T21:43:51.185 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../045-python3-webtest_2.0.28-1ubuntu1_all.deb ... 2020-11-28T21:43:51.198 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-osd. 2020-11-28T21:43:51.210 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../050-ceph-osd_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:51.224 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-webtest (2.0.28-1ubuntu1) ... 2020-11-28T21:43:51.244 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package librgw2. 2020-11-28T21:43:51.248 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-osd (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:51.256 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../016-librgw2_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:51.293 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking librgw2 (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:51.660 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-common. 2020-11-28T21:43:51.673 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../022-ceph-common_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:51.711 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-common (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:51.821 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-rgw. 2020-11-28T21:43:51.832 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../017-python3-rgw_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:51.871 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-rgw (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:52.173 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2020-11-28T21:43:52.184 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../018-libsnappy1v5_1.1.7-1_amd64.deb ... 2020-11-28T21:43:52.232 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2020-11-28T21:43:52.285 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-pecan. 2020-11-28T21:43:52.296 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../046-python3-pecan_1.2.1-2_all.deb ... 2020-11-28T21:43:52.335 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-pecan (1.2.1-2) ... 2020-11-28T21:43:52.551 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2020-11-28T21:43:52.563 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../019-libleveldb1v5_1.20-2_amd64.deb ... 2020-11-28T21:43:52.602 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2020-11-28T21:43:52.631 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-werkzeug. 2020-11-28T21:43:52.642 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../047-python3-werkzeug_0.14.1+dfsg1-1_all.deb ... 2020-11-28T21:43:52.672 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-werkzeug (0.14.1+dfsg1-1) ... 2020-11-28T21:43:53.018 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-mgr. 2020-11-28T21:43:53.029 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../048-ceph-mgr_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:53.059 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-mgr (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:53.104 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph. 2020-11-28T21:43:53.117 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../051-ceph_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:53.155 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:53.424 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-base. 2020-11-28T21:43:53.435 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../023-ceph-base_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:53.483 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-base (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:53.504 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-mon. 2020-11-28T21:43:53.507 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-fuse. 2020-11-28T21:43:53.516 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../049-ceph-mon_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:53.519 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../052-ceph-fuse_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:53.554 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-mon (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:53.557 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-fuse (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:53.654 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package liboath0. 2020-11-28T21:43:53.666 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../020-liboath0_2.6.1-1_amd64.deb ... 2020-11-28T21:43:53.704 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking liboath0 (2.6.1-1) ... 2020-11-28T21:43:53.886 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-mds. 2020-11-28T21:43:53.897 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../053-ceph-mds_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:53.927 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-mds (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:53.956 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package libradosstriper1. 2020-11-28T21:43:53.968 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../021-libradosstriper1_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:54.006 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking libradosstriper1 (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:54.370 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-dateutil. 2020-11-28T21:43:54.383 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../024-python3-dateutil_2.6.1-1_all.deb ... 2020-11-28T21:43:54.420 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-dateutil (2.6.1-1) ... 2020-11-28T21:43:54.439 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package cephadm. 2020-11-28T21:43:54.452 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../054-cephadm_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:54.459 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package ceph-common. 2020-11-28T21:43:54.465 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-osd. 2020-11-28T21:43:54.471 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../022-ceph-common_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:54.477 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../050-ceph-osd_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:54.489 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking cephadm (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:54.509 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking ceph-common (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:54.532 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-osd (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:54.748 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-asn1crypto. 2020-11-28T21:43:54.750 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-jinja2. 2020-11-28T21:43:54.759 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../025-python3-asn1crypto_0.24.0-1_all.deb ... 2020-11-28T21:43:54.761 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../055-python3-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2020-11-28T21:43:54.792 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-11-28T21:43:54.798 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-asn1crypto (0.24.0-1) ... 2020-11-28T21:43:55.051 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-cffi-backend. 2020-11-28T21:43:55.054 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2020-11-28T21:43:55.063 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../026-python3-cffi-backend_1.11.5-1_amd64.deb ... 2020-11-28T21:43:55.064 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../056-ceph-mgr-cephadm_15.2.6-595-ga83a3572-1bionic_all.deb ... 2020-11-28T21:43:55.092 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-cffi-backend (1.11.5-1) ... 2020-11-28T21:43:55.095 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-mgr-cephadm (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:55.395 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-cryptography. 2020-11-28T21:43:55.407 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../027-python3-cryptography_2.1.4-1ubuntu1.4_amd64.deb ... 2020-11-28T21:43:55.432 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-repoze.lru. 2020-11-28T21:43:55.445 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../057-python3-repoze.lru_0.7-1_all.deb ... 2020-11-28T21:43:55.445 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-cryptography (2.1.4-1ubuntu1.4) ... 2020-11-28T21:43:55.482 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-repoze.lru (0.7-1) ... 2020-11-28T21:43:55.802 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-routes. 2020-11-28T21:43:55.807 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-openssl. 2020-11-28T21:43:55.814 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../058-python3-routes_2.4.1-1_all.deb ... 2020-11-28T21:43:55.818 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../028-python3-openssl_17.5.0-1ubuntu1_all.deb ... 2020-11-28T21:43:55.852 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-routes (2.4.1-1) ... 2020-11-28T21:43:55.857 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-openssl (17.5.0-1ubuntu1) ... 2020-11-28T21:43:56.154 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2020-11-28T21:43:56.165 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../059-ceph-mgr-dashboard_15.2.6-595-ga83a3572-1bionic_all.deb ... 2020-11-28T21:43:56.204 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-mgr-dashboard (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:56.256 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package ceph-base. 2020-11-28T21:43:56.267 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../023-ceph-base_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:56.314 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking ceph-base (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:56.318 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2020-11-28T21:43:56.330 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../029-ceph-mgr-modules-core_15.2.6-595-ga83a3572-1bionic_all.deb ... 2020-11-28T21:43:56.368 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mgr-modules-core (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:56.421 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph. 2020-11-28T21:43:56.432 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../051-ceph_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:56.470 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:56.747 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-bcrypt. 2020-11-28T21:43:56.759 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../030-python3-bcrypt_3.1.4-2_amd64.deb ... 2020-11-28T21:43:56.790 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-fuse. 2020-11-28T21:43:56.797 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-bcrypt (3.1.4-2) ... 2020-11-28T21:43:56.801 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../052-ceph-fuse_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:56.839 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-fuse (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:57.059 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-cherrypy3. 2020-11-28T21:43:57.069 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../031-python3-cherrypy3_8.9.1-2_all.deb ... 2020-11-28T21:43:57.109 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-cherrypy3 (8.9.1-2) ... 2020-11-28T21:43:57.168 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-mds. 2020-11-28T21:43:57.178 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../053-ceph-mds_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:57.202 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-dateutil. 2020-11-28T21:43:57.209 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-mds (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:57.214 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../024-python3-dateutil_2.6.1-1_all.deb ... 2020-11-28T21:43:57.252 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-dateutil (2.6.1-1) ... 2020-11-28T21:43:57.363 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2020-11-28T21:43:57.376 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../060-ceph-mgr-diskprediction-cloud_15.2.6-595-ga83a3572-1bionic_all.deb ... 2020-11-28T21:43:57.413 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:57.437 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-jwt. 2020-11-28T21:43:57.448 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../032-python3-jwt_1.5.3+ds1-1_all.deb ... 2020-11-28T21:43:57.487 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-jwt (1.5.3+ds1-1) ... 2020-11-28T21:43:57.572 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-asn1crypto. 2020-11-28T21:43:57.583 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../025-python3-asn1crypto_0.24.0-1_all.deb ... 2020-11-28T21:43:57.621 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-asn1crypto (0.24.0-1) ... 2020-11-28T21:43:57.704 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package cephadm. 2020-11-28T21:43:57.716 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../054-cephadm_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:43:57.717 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-numpy. 2020-11-28T21:43:57.729 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../061-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... 2020-11-28T21:43:57.748 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-logutils. 2020-11-28T21:43:57.754 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking cephadm (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:57.760 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../033-python3-logutils_0.3.3-5_all.deb ... 2020-11-28T21:43:57.767 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... 2020-11-28T21:43:57.790 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-logutils (0.3.3-5) ... 2020-11-28T21:43:57.916 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-cffi-backend. 2020-11-28T21:43:57.927 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../026-python3-cffi-backend_1.11.5-1_amd64.deb ... 2020-11-28T21:43:57.966 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-cffi-backend (1.11.5-1) ... 2020-11-28T21:43:58.073 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-jinja2. 2020-11-28T21:43:58.086 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../055-python3-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2020-11-28T21:43:58.093 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-markupsafe. 2020-11-28T21:43:58.106 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../034-python3-markupsafe_1.0-1build1_amd64.deb ... 2020-11-28T21:43:58.123 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-11-28T21:43:58.143 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-markupsafe (1.0-1build1) ... 2020-11-28T21:43:58.276 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-cryptography. 2020-11-28T21:43:58.288 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../027-python3-cryptography_2.1.4-1ubuntu1.4_amd64.deb ... 2020-11-28T21:43:58.326 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-cryptography (2.1.4-1ubuntu1.4) ... 2020-11-28T21:43:58.381 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-decorator. 2020-11-28T21:43:58.392 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../062-python3-decorator_4.1.2-1_all.deb ... 2020-11-28T21:43:58.431 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-decorator (4.1.2-1) ... 2020-11-28T21:43:58.446 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-mako. 2020-11-28T21:43:58.451 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2020-11-28T21:43:58.456 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../035-python3-mako_1.0.7+ds1-1_all.deb ... 2020-11-28T21:43:58.463 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../056-ceph-mgr-cephadm_15.2.6-595-ga83a3572-1bionic_all.deb ... 2020-11-28T21:43:58.496 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-mako (1.0.7+ds1-1) ... 2020-11-28T21:43:58.501 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-mgr-cephadm (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:58.705 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-openssl. 2020-11-28T21:43:58.716 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../028-python3-openssl_17.5.0-1ubuntu1_all.deb ... 2020-11-28T21:43:58.717 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-scipy. 2020-11-28T21:43:58.729 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../063-python3-scipy_0.19.1-2ubuntu1_amd64.deb ... 2020-11-28T21:43:58.755 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-openssl (17.5.0-1ubuntu1) ... 2020-11-28T21:43:58.767 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-scipy (0.19.1-2ubuntu1) ... 2020-11-28T21:43:58.796 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-repoze.lru. 2020-11-28T21:43:58.809 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../057-python3-repoze.lru_0.7-1_all.deb ... 2020-11-28T21:43:58.815 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-simplegeneric. 2020-11-28T21:43:58.827 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../036-python3-simplegeneric_0.8.1-1_all.deb ... 2020-11-28T21:43:58.846 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-repoze.lru (0.7-1) ... 2020-11-28T21:43:58.865 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-simplegeneric (0.8.1-1) ... 2020-11-28T21:43:59.016 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2020-11-28T21:43:59.027 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../029-ceph-mgr-modules-core_15.2.6-595-ga83a3572-1bionic_all.deb ... 2020-11-28T21:43:59.057 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking ceph-mgr-modules-core (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:59.106 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-routes. 2020-11-28T21:43:59.110 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-singledispatch. 2020-11-28T21:43:59.120 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../058-python3-routes_2.4.1-1_all.deb ... 2020-11-28T21:43:59.121 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../037-python3-singledispatch_3.4.0.3-2_all.deb ... 2020-11-28T21:43:59.148 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-routes (2.4.1-1) ... 2020-11-28T21:43:59.152 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-singledispatch (3.4.0.3-2) ... 2020-11-28T21:43:59.395 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-bcrypt. 2020-11-28T21:43:59.406 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../030-python3-bcrypt_3.1.4-2_amd64.deb ... 2020-11-28T21:43:59.443 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2020-11-28T21:43:59.455 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../059-ceph-mgr-dashboard_15.2.6-595-ga83a3572-1bionic_all.deb ... 2020-11-28T21:43:59.461 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-bcrypt (3.1.4-2) ... 2020-11-28T21:43:59.471 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-webob. 2020-11-28T21:43:59.483 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../038-python3-webob_1%3a1.7.3-2fakesync1_all.deb ... 2020-11-28T21:43:59.493 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-mgr-dashboard (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:43:59.521 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-webob (1:1.7.3-2fakesync1) ... 2020-11-28T21:43:59.748 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-cherrypy3. 2020-11-28T21:43:59.760 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../031-python3-cherrypy3_8.9.1-2_all.deb ... 2020-11-28T21:43:59.797 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-cherrypy3 (8.9.1-2) ... 2020-11-28T21:43:59.801 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-sklearn-lib. 2020-11-28T21:43:59.814 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../064-python3-sklearn-lib_0.19.1-3_amd64.deb ... 2020-11-28T21:43:59.832 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-bs4. 2020-11-28T21:43:59.845 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../039-python3-bs4_4.6.0-1_all.deb ... 2020-11-28T21:43:59.851 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-sklearn-lib (0.19.1-3) ... 2020-11-28T21:43:59.882 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-bs4 (4.6.0-1) ... 2020-11-28T21:44:00.142 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-jwt. 2020-11-28T21:44:00.228 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-waitress. 2020-11-28T21:44:00.228 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../040-python3-waitress_1.0.1-1_all.deb ... 2020-11-28T21:44:00.228 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../032-python3-jwt_1.5.3+ds1-1_all.deb ... 2020-11-28T21:44:00.229 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-jwt (1.5.3+ds1-1) ... 2020-11-28T21:44:00.252 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-waitress (1.0.1-1) ... 2020-11-28T21:44:00.310 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-joblib. 2020-11-28T21:44:00.323 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../065-python3-joblib_0.11-1_all.deb ... 2020-11-28T21:44:00.363 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-joblib (0.11-1) ... 2020-11-28T21:44:00.374 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2020-11-28T21:44:00.386 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../060-ceph-mgr-diskprediction-cloud_15.2.6-595-ga83a3572-1bionic_all.deb ... 2020-11-28T21:44:00.425 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:00.504 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-logutils. 2020-11-28T21:44:00.514 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../033-python3-logutils_0.3.3-5_all.deb ... 2020-11-28T21:44:00.553 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-logutils (0.3.3-5) ... 2020-11-28T21:44:00.563 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-tempita. 2020-11-28T21:44:00.576 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../041-python3-tempita_0.5.2-2_all.deb ... 2020-11-28T21:44:00.614 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-tempita (0.5.2-2) ... 2020-11-28T21:44:00.700 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-sklearn. 2020-11-28T21:44:00.712 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../066-python3-sklearn_0.19.1-3_all.deb ... 2020-11-28T21:44:00.750 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-sklearn (0.19.1-3) ... 2020-11-28T21:44:00.769 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-numpy. 2020-11-28T21:44:00.781 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../061-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... 2020-11-28T21:44:00.806 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-markupsafe. 2020-11-28T21:44:00.819 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../034-python3-markupsafe_1.0-1build1_amd64.deb ... 2020-11-28T21:44:00.835 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... 2020-11-28T21:44:00.864 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-markupsafe (1.0-1build1) ... 2020-11-28T21:44:00.866 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-paste. 2020-11-28T21:44:00.877 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../042-python3-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2020-11-28T21:44:00.916 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-paste (2.0.3+dfsg-4ubuntu1) ... 2020-11-28T21:44:01.175 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-mako. 2020-11-28T21:44:01.190 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../035-python3-mako_1.0.7+ds1-1_all.deb ... 2020-11-28T21:44:01.225 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-mako (1.0.7+ds1-1) ... 2020-11-28T21:44:01.228 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2020-11-28T21:44:01.240 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../043-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2020-11-28T21:44:01.266 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2020-11-28T21:44:01.278 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2020-11-28T21:44:01.279 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../067-ceph-mgr-diskprediction-local_15.2.6-595-ga83a3572-1bionic_all.deb ... 2020-11-28T21:44:01.332 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:01.449 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-decorator. 2020-11-28T21:44:01.461 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../062-python3-decorator_4.1.2-1_all.deb ... 2020-11-28T21:44:01.499 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-decorator (4.1.2-1) ... 2020-11-28T21:44:01.514 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pastedeploy. 2020-11-28T21:44:01.526 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../044-python3-pastedeploy_1.5.2-4_all.deb ... 2020-11-28T21:44:01.536 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-simplegeneric. 2020-11-28T21:44:01.549 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../036-python3-simplegeneric_0.8.1-1_all.deb ... 2020-11-28T21:44:01.564 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pastedeploy (1.5.2-4) ... 2020-11-28T21:44:01.586 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-simplegeneric (0.8.1-1) ... 2020-11-28T21:44:01.819 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-scipy. 2020-11-28T21:44:01.832 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../063-python3-scipy_0.19.1-2ubuntu1_amd64.deb ... 2020-11-28T21:44:01.868 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-webtest. 2020-11-28T21:44:01.868 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-scipy (0.19.1-2ubuntu1) ... 2020-11-28T21:44:01.879 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../045-python3-webtest_2.0.28-1ubuntu1_all.deb ... 2020-11-28T21:44:01.889 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-singledispatch. 2020-11-28T21:44:01.900 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../037-python3-singledispatch_3.4.0.3-2_all.deb ... 2020-11-28T21:44:01.918 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-webtest (2.0.28-1ubuntu1) ... 2020-11-28T21:44:01.939 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-singledispatch (3.4.0.3-2) ... 2020-11-28T21:44:02.250 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-webob. 2020-11-28T21:44:02.261 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../038-python3-webob_1%3a1.7.3-2fakesync1_all.deb ... 2020-11-28T21:44:02.262 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pecan. 2020-11-28T21:44:02.274 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../046-python3-pecan_1.2.1-2_all.deb ... 2020-11-28T21:44:02.300 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-webob (1:1.7.3-2fakesync1) ... 2020-11-28T21:44:02.312 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pecan (1.2.1-2) ... 2020-11-28T21:44:02.460 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-monotonic. 2020-11-28T21:44:02.474 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../068-python3-monotonic_1.1-2_all.deb ... 2020-11-28T21:44:02.519 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-monotonic (1.1-2) ... 2020-11-28T21:44:02.561 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-bs4. 2020-11-28T21:44:02.574 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../039-python3-bs4_4.6.0-1_all.deb ... 2020-11-28T21:44:02.602 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-bs4 (4.6.0-1) ... 2020-11-28T21:44:02.634 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-werkzeug. 2020-11-28T21:44:02.647 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../047-python3-werkzeug_0.14.1+dfsg1-1_all.deb ... 2020-11-28T21:44:02.676 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-werkzeug (0.14.1+dfsg1-1) ... 2020-11-28T21:44:02.822 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-fasteners. 2020-11-28T21:44:02.834 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../069-python3-fasteners_0.12.0-3_all.deb ... 2020-11-28T21:44:02.855 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-waitress. 2020-11-28T21:44:02.867 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../040-python3-waitress_1.0.1-1_all.deb ... 2020-11-28T21:44:02.871 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-fasteners (0.12.0-3) ... 2020-11-28T21:44:02.913 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-waitress (1.0.1-1) ... 2020-11-28T21:44:03.004 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mgr. 2020-11-28T21:44:03.017 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../048-ceph-mgr_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:44:03.026 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-sklearn-lib. 2020-11-28T21:44:03.039 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../064-python3-sklearn-lib_0.19.1-3_amd64.deb ... 2020-11-28T21:44:03.054 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mgr (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:03.068 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-sklearn-lib (0.19.1-3) ... 2020-11-28T21:44:03.133 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-httplib2. 2020-11-28T21:44:03.146 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../070-python3-httplib2_0.9.2+dfsg-1ubuntu0.2_all.deb ... 2020-11-28T21:44:03.183 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-tempita. 2020-11-28T21:44:03.183 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2020-11-28T21:44:03.194 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../041-python3-tempita_0.5.2-2_all.deb ... 2020-11-28T21:44:03.233 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-tempita (0.5.2-2) ... 2020-11-28T21:44:03.453 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-pyasn1. 2020-11-28T21:44:03.457 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mon. 2020-11-28T21:44:03.466 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../071-python3-pyasn1_0.4.2-3_all.deb ... 2020-11-28T21:44:03.469 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../049-ceph-mon_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:44:03.503 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-pyasn1 (0.4.2-3) ... 2020-11-28T21:44:03.507 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mon (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:03.536 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-paste. 2020-11-28T21:44:03.538 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-joblib. 2020-11-28T21:44:03.547 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../042-python3-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2020-11-28T21:44:03.549 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../065-python3-joblib_0.11-1_all.deb ... 2020-11-28T21:44:03.585 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-paste (2.0.3+dfsg-4ubuntu1) ... 2020-11-28T21:44:03.587 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-joblib (0.11-1) ... 2020-11-28T21:44:03.815 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-pyasn1-modules. 2020-11-28T21:44:03.827 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../072-python3-pyasn1-modules_0.2.1-0.2_all.deb ... 2020-11-28T21:44:03.864 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2) ... 2020-11-28T21:44:03.922 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2020-11-28T21:44:03.934 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../043-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2020-11-28T21:44:03.941 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-sklearn. 2020-11-28T21:44:03.953 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../066-python3-sklearn_0.19.1-3_all.deb ... 2020-11-28T21:44:03.972 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2020-11-28T21:44:03.990 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-sklearn (0.19.1-3) ... 2020-11-28T21:44:04.193 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-rsa. 2020-11-28T21:44:04.206 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../073-python3-rsa_3.4.2-1_all.deb ... 2020-11-28T21:44:04.243 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-rsa (3.4.2-1) ... 2020-11-28T21:44:04.283 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-pastedeploy. 2020-11-28T21:44:04.295 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../044-python3-pastedeploy_1.5.2-4_all.deb ... 2020-11-28T21:44:04.333 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-pastedeploy (1.5.2-4) ... 2020-11-28T21:44:04.394 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-osd. 2020-11-28T21:44:04.405 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../050-ceph-osd_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:44:04.445 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-osd (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:04.505 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2020-11-28T21:44:04.517 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../067-ceph-mgr-diskprediction-local_15.2.6-595-ga83a3572-1bionic_all.deb ... 2020-11-28T21:44:04.555 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:04.571 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-sqlalchemy. 2020-11-28T21:44:04.585 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../074-python3-sqlalchemy_1.1.11+ds1-1ubuntu1_all.deb ... 2020-11-28T21:44:04.629 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2020-11-28T21:44:04.676 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-webtest. 2020-11-28T21:44:04.687 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../045-python3-webtest_2.0.28-1ubuntu1_all.deb ... 2020-11-28T21:44:04.726 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-webtest (2.0.28-1ubuntu1) ... 2020-11-28T21:44:05.062 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-pecan. 2020-11-28T21:44:05.067 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-oauth2client. 2020-11-28T21:44:05.074 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../046-python3-pecan_1.2.1-2_all.deb ... 2020-11-28T21:44:05.080 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../075-python3-oauth2client_4.1.2-3_all.deb ... 2020-11-28T21:44:05.112 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-pecan (1.2.1-2) ... 2020-11-28T21:44:05.117 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-oauth2client (4.1.2-3) ... 2020-11-28T21:44:05.441 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-werkzeug. 2020-11-28T21:44:05.454 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../047-python3-werkzeug_0.14.1+dfsg1-1_all.deb ... 2020-11-28T21:44:05.462 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-setuptools. 2020-11-28T21:44:05.475 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../076-python3-setuptools_39.0.1-2_all.deb ... 2020-11-28T21:44:05.508 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-werkzeug (0.14.1+dfsg1-1) ... 2020-11-28T21:44:05.512 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-setuptools (39.0.1-2) ... 2020-11-28T21:44:05.599 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-monotonic. 2020-11-28T21:44:05.611 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../068-python3-monotonic_1.1-2_all.deb ... 2020-11-28T21:44:05.649 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-monotonic (1.1-2) ... 2020-11-28T21:44:05.886 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package ceph-mgr. 2020-11-28T21:44:05.898 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../048-ceph-mgr_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:44:05.899 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-websocket. 2020-11-28T21:44:05.911 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../077-python3-websocket_0.44.0-0ubuntu2_all.deb ... 2020-11-28T21:44:05.936 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking ceph-mgr (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:05.949 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-websocket (0.44.0-0ubuntu2) ... 2020-11-28T21:44:05.952 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-fasteners. 2020-11-28T21:44:05.964 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../069-python3-fasteners_0.12.0-3_all.deb ... 2020-11-28T21:44:06.002 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-fasteners (0.12.0-3) ... 2020-11-28T21:44:06.280 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-httplib2. 2020-11-28T21:44:06.286 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-kubernetes. 2020-11-28T21:44:06.294 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../070-python3-httplib2_0.9.2+dfsg-1ubuntu0.2_all.deb ... 2020-11-28T21:44:06.297 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../078-python3-kubernetes_2.0.0-2ubuntu1_all.deb ... 2020-11-28T21:44:06.316 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph. 2020-11-28T21:44:06.329 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../051-ceph_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:44:06.336 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-kubernetes (2.0.0-2ubuntu1) ... 2020-11-28T21:44:06.338 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2020-11-28T21:44:06.355 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package ceph-mon. 2020-11-28T21:44:06.366 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:06.367 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../049-ceph-mon_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:44:06.397 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking ceph-mon (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:06.666 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-pyasn1. 2020-11-28T21:44:06.679 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../071-python3-pyasn1_0.4.2-3_all.deb ... 2020-11-28T21:44:06.703 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-fuse. 2020-11-28T21:44:06.708 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-pyasn1 (0.4.2-3) ... 2020-11-28T21:44:06.714 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../052-ceph-fuse_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:44:06.732 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2020-11-28T21:44:06.744 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../079-ceph-mgr-k8sevents_15.2.6-595-ga83a3572-1bionic_all.deb ... 2020-11-28T21:44:06.752 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-fuse (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:06.790 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-mgr-k8sevents (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:07.060 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-json-pointer. 2020-11-28T21:44:07.072 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../080-python3-json-pointer_1.10-1_all.deb ... 2020-11-28T21:44:07.086 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-pyasn1-modules. 2020-11-28T21:44:07.099 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../072-python3-pyasn1-modules_0.2.1-0.2_all.deb ... 2020-11-28T21:44:07.136 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2) ... 2020-11-28T21:44:07.139 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mds. 2020-11-28T21:44:07.151 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../053-ceph-mds_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:44:07.179 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-json-pointer (1.10-1) ... 2020-11-28T21:44:07.189 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mds (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:07.326 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package ceph-osd. 2020-11-28T21:44:07.338 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../050-ceph-osd_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:44:07.375 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking ceph-osd (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:07.438 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-rsa. 2020-11-28T21:44:07.451 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../073-python3-rsa_3.4.2-1_all.deb ... 2020-11-28T21:44:07.488 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-rsa (3.4.2-1) ... 2020-11-28T21:44:07.540 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-jsonpatch. 2020-11-28T21:44:07.554 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../081-python3-jsonpatch_1.19+really1.16-1fakesync1_all.deb ... 2020-11-28T21:44:07.610 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2020-11-28T21:44:07.644 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package cephadm. 2020-11-28T21:44:07.657 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../054-cephadm_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:44:07.694 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking cephadm (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:07.776 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-sqlalchemy. 2020-11-28T21:44:07.786 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../074-python3-sqlalchemy_1.1.11+ds1-1ubuntu1_all.deb ... 2020-11-28T21:44:07.824 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2020-11-28T21:44:07.877 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-mgr-rook. 2020-11-28T21:44:07.888 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../082-ceph-mgr-rook_15.2.6-595-ga83a3572-1bionic_all.deb ... 2020-11-28T21:44:07.919 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-mgr-rook (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:08.030 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-jinja2. 2020-11-28T21:44:08.042 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../055-python3-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2020-11-28T21:44:08.080 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-11-28T21:44:08.231 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-oauth2client. 2020-11-28T21:44:08.242 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../075-python3-oauth2client_4.1.2-3_all.deb ... 2020-11-28T21:44:08.263 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libonig4:amd64. 2020-11-28T21:44:08.275 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../083-libonig4_6.7.0-1_amd64.deb ... 2020-11-28T21:44:08.281 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-oauth2client (4.1.2-3) ... 2020-11-28T21:44:08.314 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2020-11-28T21:44:08.375 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2020-11-28T21:44:08.388 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../056-ceph-mgr-cephadm_15.2.6-595-ga83a3572-1bionic_all.deb ... 2020-11-28T21:44:08.425 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mgr-cephadm (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:08.626 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-setuptools. 2020-11-28T21:44:08.638 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../076-python3-setuptools_39.0.1-2_all.deb ... 2020-11-28T21:44:08.667 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libjq1:amd64. 2020-11-28T21:44:08.675 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-setuptools (39.0.1-2) ... 2020-11-28T21:44:08.679 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../084-libjq1_1.5+dfsg-2_amd64.deb ... 2020-11-28T21:44:08.717 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2020-11-28T21:44:08.728 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-repoze.lru. 2020-11-28T21:44:08.739 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../057-python3-repoze.lru_0.7-1_all.deb ... 2020-11-28T21:44:08.778 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-repoze.lru (0.7-1) ... 2020-11-28T21:44:08.979 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package jq. 2020-11-28T21:44:08.991 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../085-jq_1.5+dfsg-2_amd64.deb ... 2020-11-28T21:44:09.020 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking jq (1.5+dfsg-2) ... 2020-11-28T21:44:09.047 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-routes. 2020-11-28T21:44:09.054 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-websocket. 2020-11-28T21:44:09.059 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../058-python3-routes_2.4.1-1_all.deb ... 2020-11-28T21:44:09.065 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../077-python3-websocket_0.44.0-0ubuntu2_all.deb ... 2020-11-28T21:44:09.097 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-routes (2.4.1-1) ... 2020-11-28T21:44:09.104 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-websocket (0.44.0-0ubuntu2) ... 2020-11-28T21:44:09.231 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package ceph. 2020-11-28T21:44:09.244 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../051-ceph_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:44:09.272 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking ceph (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:09.273 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package socat. 2020-11-28T21:44:09.285 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../086-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2020-11-28T21:44:09.322 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2020-11-28T21:44:09.425 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2020-11-28T21:44:09.432 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-kubernetes. 2020-11-28T21:44:09.436 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../059-ceph-mgr-dashboard_15.2.6-595-ga83a3572-1bionic_all.deb ... 2020-11-28T21:44:09.445 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../078-python3-kubernetes_2.0.0-2ubuntu1_all.deb ... 2020-11-28T21:44:09.475 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mgr-dashboard (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:09.490 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-kubernetes (2.0.0-2ubuntu1) ... 2020-11-28T21:44:09.583 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package ceph-fuse. 2020-11-28T21:44:09.595 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../052-ceph-fuse_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:44:09.633 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking ceph-fuse (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:09.703 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libxslt1.1:amd64. 2020-11-28T21:44:09.717 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../087-libxslt1.1_1.1.29-5ubuntu0.2_amd64.deb ... 2020-11-28T21:44:09.761 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-11-28T21:44:09.886 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2020-11-28T21:44:09.899 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../079-ceph-mgr-k8sevents_15.2.6-595-ga83a3572-1bionic_all.deb ... 2020-11-28T21:44:09.928 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-mgr-k8sevents (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:10.015 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package xmlstarlet. 2020-11-28T21:44:10.029 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../088-xmlstarlet_1.6.1-2_amd64.deb ... 2020-11-28T21:44:10.037 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package ceph-mds. 2020-11-28T21:44:10.049 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../053-ceph-mds_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:44:10.064 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2020-11-28T21:44:10.087 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking ceph-mds (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:10.247 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-json-pointer. 2020-11-28T21:44:10.260 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../080-python3-json-pointer_1.10-1_all.deb ... 2020-11-28T21:44:10.323 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2020-11-28T21:44:10.337 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../060-ceph-mgr-diskprediction-cloud_15.2.6-595-ga83a3572-1bionic_all.deb ... 2020-11-28T21:44:10.382 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:10.382 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-json-pointer (1.10-1) ... 2020-11-28T21:44:10.427 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-test. 2020-11-28T21:44:10.441 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../089-ceph-test_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:44:10.485 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-test (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:10.531 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package cephadm. 2020-11-28T21:44:10.542 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../054-cephadm_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:44:10.573 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking cephadm (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:10.694 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-jsonpatch. 2020-11-28T21:44:10.706 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../081-python3-jsonpatch_1.19+really1.16-1fakesync1_all.deb ... 2020-11-28T21:44:10.709 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-numpy. 2020-11-28T21:44:10.722 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../061-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... 2020-11-28T21:44:10.746 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2020-11-28T21:44:10.759 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... 2020-11-28T21:44:10.909 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-jinja2. 2020-11-28T21:44:10.920 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../055-python3-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2020-11-28T21:44:10.959 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-11-28T21:44:11.072 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-mgr-rook. 2020-11-28T21:44:11.084 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../082-ceph-mgr-rook_15.2.6-595-ga83a3572-1bionic_all.deb ... 2020-11-28T21:44:11.122 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-mgr-rook (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:11.270 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2020-11-28T21:44:11.282 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../056-ceph-mgr-cephadm_15.2.6-595-ga83a3572-1bionic_all.deb ... 2020-11-28T21:44:11.320 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking ceph-mgr-cephadm (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:11.365 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-decorator. 2020-11-28T21:44:11.376 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../062-python3-decorator_4.1.2-1_all.deb ... 2020-11-28T21:44:11.407 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-decorator (4.1.2-1) ... 2020-11-28T21:44:11.433 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libonig4:amd64. 2020-11-28T21:44:11.446 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../083-libonig4_6.7.0-1_amd64.deb ... 2020-11-28T21:44:11.491 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2020-11-28T21:44:11.673 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-repoze.lru. 2020-11-28T21:44:11.684 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../057-python3-repoze.lru_0.7-1_all.deb ... 2020-11-28T21:44:11.701 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-scipy. 2020-11-28T21:44:11.713 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../063-python3-scipy_0.19.1-2ubuntu1_amd64.deb ... 2020-11-28T21:44:11.723 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-repoze.lru (0.7-1) ... 2020-11-28T21:44:11.751 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-scipy (0.19.1-2ubuntu1) ... 2020-11-28T21:44:11.811 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libjq1:amd64. 2020-11-28T21:44:11.825 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../084-libjq1_1.5+dfsg-2_amd64.deb ... 2020-11-28T21:44:11.869 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2020-11-28T21:44:12.026 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-routes. 2020-11-28T21:44:12.037 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../058-python3-routes_2.4.1-1_all.deb ... 2020-11-28T21:44:12.076 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-routes (2.4.1-1) ... 2020-11-28T21:44:12.105 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package jq. 2020-11-28T21:44:12.119 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../085-jq_1.5+dfsg-2_amd64.deb ... 2020-11-28T21:44:12.164 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking jq (1.5+dfsg-2) ... 2020-11-28T21:44:12.345 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2020-11-28T21:44:12.358 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../059-ceph-mgr-dashboard_15.2.6-595-ga83a3572-1bionic_all.deb ... 2020-11-28T21:44:12.387 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking ceph-mgr-dashboard (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:12.408 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package socat. 2020-11-28T21:44:12.419 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../086-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2020-11-28T21:44:12.458 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2020-11-28T21:44:12.834 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-sklearn-lib. 2020-11-28T21:44:12.847 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../064-python3-sklearn-lib_0.19.1-3_amd64.deb ... 2020-11-28T21:44:12.853 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libxslt1.1:amd64. 2020-11-28T21:44:12.867 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../087-libxslt1.1_1.1.29-5ubuntu0.2_amd64.deb ... 2020-11-28T21:44:12.875 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-sklearn-lib (0.19.1-3) ... 2020-11-28T21:44:12.911 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-11-28T21:44:13.189 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package xmlstarlet. 2020-11-28T21:44:13.201 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../088-xmlstarlet_1.6.1-2_amd64.deb ... 2020-11-28T21:44:13.231 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2020-11-28T21:44:13.277 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2020-11-28T21:44:13.289 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../060-ceph-mgr-diskprediction-cloud_15.2.6-595-ga83a3572-1bionic_all.deb ... 2020-11-28T21:44:13.321 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-joblib. 2020-11-28T21:44:13.332 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../065-python3-joblib_0.11-1_all.deb ... 2020-11-28T21:44:13.348 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:13.387 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-joblib (0.11-1) ... 2020-11-28T21:44:13.561 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-test. 2020-11-28T21:44:13.573 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../089-ceph-test_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:44:13.610 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-test (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:13.674 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-sklearn. 2020-11-28T21:44:13.685 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../066-python3-sklearn_0.19.1-3_all.deb ... 2020-11-28T21:44:13.715 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-sklearn (0.19.1-3) ... 2020-11-28T21:44:13.839 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-colorama. 2020-11-28T21:44:13.851 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../090-python3-colorama_0.3.7-1_all.deb ... 2020-11-28T21:44:13.889 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-colorama (0.3.7-1) ... 2020-11-28T21:44:14.038 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-numpy. 2020-11-28T21:44:14.049 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../061-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... 2020-11-28T21:44:14.088 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... 2020-11-28T21:44:14.134 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package cephfs-shell. 2020-11-28T21:44:14.146 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../091-cephfs-shell_15.2.6-595-ga83a3572-1bionic_all.deb ... 2020-11-28T21:44:14.213 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2020-11-28T21:44:14.218 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking cephfs-shell (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:14.226 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../067-ceph-mgr-diskprediction-local_15.2.6-595-ga83a3572-1bionic_all.deb ... 2020-11-28T21:44:14.265 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:14.470 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libcephfs-dev. 2020-11-28T21:44:14.483 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../092-libcephfs-dev_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:44:14.528 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libcephfs-dev (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:14.694 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-decorator. 2020-11-28T21:44:14.708 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../062-python3-decorator_4.1.2-1_all.deb ... 2020-11-28T21:44:14.752 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-decorator (4.1.2-1) ... 2020-11-28T21:44:14.764 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package nvme-cli. 2020-11-28T21:44:14.775 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../093-nvme-cli_1.5-1ubuntu1.1_amd64.deb ... 2020-11-28T21:44:14.805 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking nvme-cli (1.5-1ubuntu1.1) ... 2020-11-28T21:44:15.088 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-scipy. 2020-11-28T21:44:15.101 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../063-python3-scipy_0.19.1-2ubuntu1_amd64.deb ... 2020-11-28T21:44:15.138 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-scipy (0.19.1-2ubuntu1) ... 2020-11-28T21:44:15.152 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python-attr. 2020-11-28T21:44:15.163 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../094-python-attr_17.4.0-2_all.deb ... 2020-11-28T21:44:15.227 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python-attr (17.4.0-2) ... 2020-11-28T21:44:15.366 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-monotonic. 2020-11-28T21:44:15.378 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../068-python3-monotonic_1.1-2_all.deb ... 2020-11-28T21:44:15.416 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-monotonic (1.1-2) ... 2020-11-28T21:44:15.530 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python-funcsigs. 2020-11-28T21:44:15.541 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../095-python-funcsigs_1.0.2-4_all.deb ... 2020-11-28T21:44:15.580 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python-funcsigs (1.0.2-4) ... 2020-11-28T21:44:15.702 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-fasteners. 2020-11-28T21:44:15.716 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../069-python3-fasteners_0.12.0-3_all.deb ... 2020-11-28T21:44:15.761 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-fasteners (0.12.0-3) ... 2020-11-28T21:44:15.882 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python-pluggy. 2020-11-28T21:44:15.895 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../096-python-pluggy_0.6.0-1_all.deb ... 2020-11-28T21:44:15.932 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python-pluggy (0.6.0-1) ... 2020-11-28T21:44:16.046 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-httplib2. 2020-11-28T21:44:16.060 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../070-python3-httplib2_0.9.2+dfsg-1ubuntu0.2_all.deb ... 2020-11-28T21:44:16.105 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2020-11-28T21:44:16.235 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python-py. 2020-11-28T21:44:16.247 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../097-python-py_1.5.2-1_all.deb ... 2020-11-28T21:44:16.279 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-sklearn-lib. 2020-11-28T21:44:16.285 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python-py (1.5.2-1) ... 2020-11-28T21:44:16.292 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../064-python3-sklearn-lib_0.19.1-3_amd64.deb ... 2020-11-28T21:44:16.329 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-sklearn-lib (0.19.1-3) ... 2020-11-28T21:44:16.399 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pyasn1. 2020-11-28T21:44:16.411 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../071-python3-pyasn1_0.4.2-3_all.deb ... 2020-11-28T21:44:16.483 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pyasn1 (0.4.2-3) ... 2020-11-28T21:44:16.605 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python-pytest. 2020-11-28T21:44:16.618 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../098-python-pytest_3.3.2-2_all.deb ... 2020-11-28T21:44:16.655 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python-pytest (3.3.2-2) ... 2020-11-28T21:44:16.740 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-joblib. 2020-11-28T21:44:16.753 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../065-python3-joblib_0.11-1_all.deb ... 2020-11-28T21:44:16.790 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-joblib (0.11-1) ... 2020-11-28T21:44:16.802 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pyasn1-modules. 2020-11-28T21:44:16.815 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../072-python3-pyasn1-modules_0.2.1-0.2_all.deb ... 2020-11-28T21:44:16.852 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2) ... 2020-11-28T21:44:16.913 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-colorama. 2020-11-28T21:44:16.927 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../090-python3-colorama_0.3.7-1_all.deb ... 2020-11-28T21:44:16.964 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-colorama (0.3.7-1) ... 2020-11-28T21:44:16.976 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-attr. 2020-11-28T21:44:16.987 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../099-python3-attr_17.4.0-2_all.deb ... 2020-11-28T21:44:17.025 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-attr (17.4.0-2) ... 2020-11-28T21:44:17.121 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-sklearn. 2020-11-28T21:44:17.135 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../066-python3-sklearn_0.19.1-3_all.deb ... 2020-11-28T21:44:17.162 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-sklearn (0.19.1-3) ... 2020-11-28T21:44:17.172 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-rsa. 2020-11-28T21:44:17.183 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../073-python3-rsa_3.4.2-1_all.deb ... 2020-11-28T21:44:17.222 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-rsa (3.4.2-1) ... 2020-11-28T21:44:17.353 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-cycler. 2020-11-28T21:44:17.367 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../100-python3-cycler_0.10.0-1_all.deb ... 2020-11-28T21:44:17.411 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-cycler (0.10.0-1) ... 2020-11-28T21:44:17.516 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-sqlalchemy. 2020-11-28T21:44:17.530 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../074-python3-sqlalchemy_1.1.11+ds1-1ubuntu1_all.deb ... 2020-11-28T21:44:17.541 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package cephfs-shell. 2020-11-28T21:44:17.553 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../091-cephfs-shell_15.2.6-595-ga83a3572-1bionic_all.deb ... 2020-11-28T21:44:17.575 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2020-11-28T21:44:17.591 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking cephfs-shell (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:17.636 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2020-11-28T21:44:17.649 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../067-ceph-mgr-diskprediction-local_15.2.6-595-ga83a3572-1bionic_all.deb ... 2020-11-28T21:44:17.698 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-webencodings. 2020-11-28T21:44:17.712 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../101-python3-webencodings_0.5-2_all.deb ... 2020-11-28T21:44:17.719 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:17.756 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-webencodings (0.5-2) ... 2020-11-28T21:44:17.852 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libcephfs-dev. 2020-11-28T21:44:17.864 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../092-libcephfs-dev_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:44:17.902 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libcephfs-dev (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:18.034 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-html5lib. 2020-11-28T21:44:18.037 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-oauth2client. 2020-11-28T21:44:18.047 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../102-python3-html5lib_0.999999999-1_all.deb ... 2020-11-28T21:44:18.050 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../075-python3-oauth2client_4.1.2-3_all.deb ... 2020-11-28T21:44:18.083 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-html5lib (0.999999999-1) ... 2020-11-28T21:44:18.087 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-oauth2client (4.1.2-3) ... 2020-11-28T21:44:18.121 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package nvme-cli. 2020-11-28T21:44:18.134 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../093-nvme-cli_1.5-1ubuntu1.1_amd64.deb ... 2020-11-28T21:44:18.172 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking nvme-cli (1.5-1ubuntu1.1) ... 2020-11-28T21:44:18.365 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-setuptools. 2020-11-28T21:44:18.376 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../076-python3-setuptools_39.0.1-2_all.deb ... 2020-11-28T21:44:18.415 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-setuptools (39.0.1-2) ... 2020-11-28T21:44:18.429 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-lxml:amd64. 2020-11-28T21:44:18.442 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../103-python3-lxml_4.2.1-1ubuntu0.1_amd64.deb ... 2020-11-28T21:44:18.479 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-11-28T21:44:18.518 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python-attr. 2020-11-28T21:44:18.529 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../094-python-attr_17.4.0-2_all.deb ... 2020-11-28T21:44:18.601 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python-attr (17.4.0-2) ... 2020-11-28T21:44:18.801 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-websocket. 2020-11-28T21:44:18.814 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../077-python3-websocket_0.44.0-0ubuntu2_all.deb ... 2020-11-28T21:44:18.851 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-websocket (0.44.0-0ubuntu2) ... 2020-11-28T21:44:18.881 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-monotonic. 2020-11-28T21:44:18.890 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-tz. 2020-11-28T21:44:18.893 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../068-python3-monotonic_1.1-2_all.deb ... 2020-11-28T21:44:18.902 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../104-python3-tz_2018.3-2_all.deb ... 2020-11-28T21:44:18.903 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python-funcsigs. 2020-11-28T21:44:18.916 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../095-python-funcsigs_1.0.2-4_all.deb ... 2020-11-28T21:44:18.930 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-monotonic (1.1-2) ... 2020-11-28T21:44:18.940 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-tz (2018.3-2) ... 2020-11-28T21:44:18.953 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python-funcsigs (1.0.2-4) ... 2020-11-28T21:44:19.163 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-kubernetes. 2020-11-28T21:44:19.174 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../078-python3-kubernetes_2.0.0-2ubuntu1_all.deb ... 2020-11-28T21:44:19.213 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-kubernetes (2.0.0-2ubuntu1) ... 2020-11-28T21:44:19.233 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-fasteners. 2020-11-28T21:44:19.235 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-matplotlib. 2020-11-28T21:44:19.247 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../069-python3-fasteners_0.12.0-3_all.deb ... 2020-11-28T21:44:19.248 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../105-python3-matplotlib_2.1.1-2ubuntu3_amd64.deb ... 2020-11-28T21:44:19.248 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python-pluggy. 2020-11-28T21:44:19.259 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../096-python-pluggy_0.6.0-1_all.deb ... 2020-11-28T21:44:19.276 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-matplotlib (2.1.1-2ubuntu3) ... 2020-11-28T21:44:19.289 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python-pluggy (0.6.0-1) ... 2020-11-28T21:44:19.292 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-fasteners (0.12.0-3) ... 2020-11-28T21:44:19.576 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python-py. 2020-11-28T21:44:19.588 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../097-python-py_1.5.2-1_all.deb ... 2020-11-28T21:44:19.595 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-httplib2. 2020-11-28T21:44:19.606 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../070-python3-httplib2_0.9.2+dfsg-1ubuntu0.2_all.deb ... 2020-11-28T21:44:19.608 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2020-11-28T21:44:19.620 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../079-ceph-mgr-k8sevents_15.2.6-595-ga83a3572-1bionic_all.deb ... 2020-11-28T21:44:19.625 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python-py (1.5.2-1) ... 2020-11-28T21:44:19.637 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2020-11-28T21:44:19.658 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mgr-k8sevents (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:19.924 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-olefile. 2020-11-28T21:44:19.927 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-json-pointer. 2020-11-28T21:44:19.938 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../106-python3-olefile_0.45.1-1_all.deb ... 2020-11-28T21:44:19.939 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../080-python3-json-pointer_1.10-1_all.deb ... 2020-11-28T21:44:19.939 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-pyasn1. 2020-11-28T21:44:19.945 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python-pytest. 2020-11-28T21:44:19.951 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../071-python3-pyasn1_0.4.2-3_all.deb ... 2020-11-28T21:44:19.958 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../098-python-pytest_3.3.2-2_all.deb ... 2020-11-28T21:44:19.982 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-olefile (0.45.1-1) ... 2020-11-28T21:44:19.989 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-pyasn1 (0.4.2-3) ... 2020-11-28T21:44:19.995 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python-pytest (3.3.2-2) ... 2020-11-28T21:44:20.054 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-json-pointer (1.10-1) ... 2020-11-28T21:44:20.311 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-pastescript. 2020-11-28T21:44:20.323 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-2_all.deb ... 2020-11-28T21:44:20.334 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-pyasn1-modules. 2020-11-28T21:44:20.347 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../072-python3-pyasn1-modules_0.2.1-0.2_all.deb ... 2020-11-28T21:44:20.357 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-attr. 2020-11-28T21:44:20.360 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-pastescript (2.0.2-2) ... 2020-11-28T21:44:20.369 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../099-python3-attr_17.4.0-2_all.deb ... 2020-11-28T21:44:20.384 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2) ... 2020-11-28T21:44:20.399 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-jsonpatch. 2020-11-28T21:44:20.407 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-attr (17.4.0-2) ... 2020-11-28T21:44:20.411 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../081-python3-jsonpatch_1.19+really1.16-1fakesync1_all.deb ... 2020-11-28T21:44:20.452 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2020-11-28T21:44:20.696 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-rsa. 2020-11-28T21:44:20.708 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../073-python3-rsa_3.4.2-1_all.deb ... 2020-11-28T21:44:20.709 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-cycler. 2020-11-28T21:44:20.714 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-pil:amd64. 2020-11-28T21:44:20.721 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../100-python3-cycler_0.10.0-1_all.deb ... 2020-11-28T21:44:20.727 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mgr-rook. 2020-11-28T21:44:20.728 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../108-python3-pil_5.1.0-1ubuntu0.3_amd64.deb ... 2020-11-28T21:44:20.739 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../082-ceph-mgr-rook_15.2.6-595-ga83a3572-1bionic_all.deb ... 2020-11-28T21:44:20.746 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-rsa (3.4.2-1) ... 2020-11-28T21:44:20.760 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-cycler (0.10.0-1) ... 2020-11-28T21:44:20.772 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.3) ... 2020-11-28T21:44:20.777 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mgr-rook (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:21.046 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-webencodings. 2020-11-28T21:44:21.057 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-sqlalchemy. 2020-11-28T21:44:21.058 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../101-python3-webencodings_0.5-2_all.deb ... 2020-11-28T21:44:21.069 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../074-python3-sqlalchemy_1.1.11+ds1-1ubuntu1_all.deb ... 2020-11-28T21:44:21.080 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libonig4:amd64. 2020-11-28T21:44:21.093 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../083-libonig4_6.7.0-1_amd64.deb ... 2020-11-28T21:44:21.096 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-webencodings (0.5-2) ... 2020-11-28T21:44:21.107 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2020-11-28T21:44:21.130 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2020-11-28T21:44:21.134 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-pluggy. 2020-11-28T21:44:21.146 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../109-python3-pluggy_0.6.0-1_all.deb ... 2020-11-28T21:44:21.208 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-pluggy (0.6.0-1) ... 2020-11-28T21:44:21.365 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-html5lib. 2020-11-28T21:44:21.379 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../102-python3-html5lib_0.999999999-1_all.deb ... 2020-11-28T21:44:21.415 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-html5lib (0.999999999-1) ... 2020-11-28T21:44:21.483 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libjq1:amd64. 2020-11-28T21:44:21.495 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../084-libjq1_1.5+dfsg-2_amd64.deb ... 2020-11-28T21:44:21.512 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-py. 2020-11-28T21:44:21.524 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../110-python3-py_1.5.2-1_all.deb ... 2020-11-28T21:44:21.533 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2020-11-28T21:44:21.545 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-oauth2client. 2020-11-28T21:44:21.557 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../075-python3-oauth2client_4.1.2-3_all.deb ... 2020-11-28T21:44:21.561 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-py (1.5.2-1) ... 2020-11-28T21:44:21.595 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-oauth2client (4.1.2-3) ... 2020-11-28T21:44:21.769 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-lxml:amd64. 2020-11-28T21:44:21.780 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../103-python3-lxml_4.2.1-1ubuntu0.1_amd64.deb ... 2020-11-28T21:44:21.819 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package jq. 2020-11-28T21:44:21.827 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-11-28T21:44:21.831 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../085-jq_1.5+dfsg-2_amd64.deb ... 2020-11-28T21:44:21.869 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking jq (1.5+dfsg-2) ... 2020-11-28T21:44:21.898 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-pyinotify. 2020-11-28T21:44:21.911 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1_all.deb ... 2020-11-28T21:44:21.923 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-setuptools. 2020-11-28T21:44:21.935 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../076-python3-setuptools_39.0.1-2_all.deb ... 2020-11-28T21:44:21.965 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-pyinotify (0.9.6-1) ... 2020-11-28T21:44:21.973 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-setuptools (39.0.1-2) ... 2020-11-28T21:44:22.105 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package socat. 2020-11-28T21:44:22.116 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../086-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2020-11-28T21:44:22.146 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2020-11-28T21:44:22.264 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-tz. 2020-11-28T21:44:22.276 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-pytest. 2020-11-28T21:44:22.278 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../104-python3-tz_2018.3-2_all.deb ... 2020-11-28T21:44:22.290 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../112-python3-pytest_3.3.2-2_all.deb ... 2020-11-28T21:44:22.322 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-tz (2018.3-2) ... 2020-11-28T21:44:22.334 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-pytest (3.3.2-2) ... 2020-11-28T21:44:22.377 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-websocket. 2020-11-28T21:44:22.389 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../077-python3-websocket_0.44.0-0ubuntu2_all.deb ... 2020-11-28T21:44:22.427 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-websocket (0.44.0-0ubuntu2) ... 2020-11-28T21:44:22.516 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libxslt1.1:amd64. 2020-11-28T21:44:22.528 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../087-libxslt1.1_1.1.29-5ubuntu0.2_amd64.deb ... 2020-11-28T21:44:22.566 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-11-28T21:44:22.616 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-matplotlib. 2020-11-28T21:44:22.628 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../105-python3-matplotlib_2.1.1-2ubuntu3_amd64.deb ... 2020-11-28T21:44:22.638 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-simplejson. 2020-11-28T21:44:22.652 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../113-python3-simplejson_3.13.2-1_amd64.deb ... 2020-11-28T21:44:22.666 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-matplotlib (2.1.1-2ubuntu3) ... 2020-11-28T21:44:22.688 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-simplejson (3.13.2-1) ... 2020-11-28T21:44:22.722 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-kubernetes. 2020-11-28T21:44:22.734 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../078-python3-kubernetes_2.0.0-2ubuntu1_all.deb ... 2020-11-28T21:44:22.772 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-kubernetes (2.0.0-2ubuntu1) ... 2020-11-28T21:44:22.844 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package xmlstarlet. 2020-11-28T21:44:22.858 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../088-xmlstarlet_1.6.1-2_amd64.deb ... 2020-11-28T21:44:22.903 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2020-11-28T21:44:22.966 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-tk:amd64. 2020-11-28T21:44:22.980 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../114-python3-tk_3.6.9-1~18.04_amd64.deb ... 2020-11-28T21:44:23.024 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-tk:amd64 (3.6.9-1~18.04) ... 2020-11-28T21:44:23.168 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2020-11-28T21:44:23.180 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../079-ceph-mgr-k8sevents_15.2.6-595-ga83a3572-1bionic_all.deb ... 2020-11-28T21:44:23.218 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking ceph-mgr-k8sevents (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:23.240 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-test. 2020-11-28T21:44:23.252 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../089-ceph-test_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:44:23.290 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-test (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:23.347 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-olefile. 2020-11-28T21:44:23.361 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../106-python3-olefile_0.45.1-1_all.deb ... 2020-11-28T21:44:23.405 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-olefile (0.45.1-1) ... 2020-11-28T21:44:23.470 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package radosgw. 2020-11-28T21:44:23.484 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../115-radosgw_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:44:23.512 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-json-pointer. 2020-11-28T21:44:23.526 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../080-python3-json-pointer_1.10-1_all.deb ... 2020-11-28T21:44:23.528 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking radosgw (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:23.656 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-json-pointer (1.10-1) ... 2020-11-28T21:44:23.674 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-pastescript. 2020-11-28T21:44:23.686 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-2_all.deb ... 2020-11-28T21:44:23.724 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-pastescript (2.0.2-2) ... 2020-11-28T21:44:24.026 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-jsonpatch. 2020-11-28T21:44:24.036 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-pil:amd64. 2020-11-28T21:44:24.038 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../081-python3-jsonpatch_1.19+really1.16-1fakesync1_all.deb ... 2020-11-28T21:44:24.049 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../108-python3-pil_5.1.0-1ubuntu0.3_amd64.deb ... 2020-11-28T21:44:24.078 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2020-11-28T21:44:24.086 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.3) ... 2020-11-28T21:44:24.337 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package ceph-mgr-rook. 2020-11-28T21:44:24.350 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../082-ceph-mgr-rook_15.2.6-595-ga83a3572-1bionic_all.deb ... 2020-11-28T21:44:24.379 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking ceph-mgr-rook (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:24.423 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-pluggy. 2020-11-28T21:44:24.437 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../109-python3-pluggy_0.6.0-1_all.deb ... 2020-11-28T21:44:24.457 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package rbd-fuse. 2020-11-28T21:44:24.470 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../116-rbd-fuse_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:44:24.481 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-pluggy (0.6.0-1) ... 2020-11-28T21:44:24.506 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking rbd-fuse (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:24.724 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package libonig4:amd64. 2020-11-28T21:44:24.736 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../083-libonig4_6.7.0-1_amd64.deb ... 2020-11-28T21:44:24.760 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-logutils (0.3.3-5) ... 2020-11-28T21:44:24.774 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2020-11-28T21:44:24.809 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-py. 2020-11-28T21:44:24.820 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../110-python3-py_1.5.2-1_all.deb ... 2020-11-28T21:44:24.859 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-py (1.5.2-1) ... 2020-11-28T21:44:25.070 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-werkzeug (0.14.1+dfsg1-1) ... 2020-11-28T21:44:25.119 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package libjq1:amd64. 2020-11-28T21:44:25.131 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../084-libjq1_1.5+dfsg-2_amd64.deb ... 2020-11-28T21:44:25.168 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2020-11-28T21:44:25.170 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-pyinotify. 2020-11-28T21:44:25.183 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1_all.deb ... 2020-11-28T21:44:25.212 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-pyinotify (0.9.6-1) ... 2020-11-28T21:44:25.455 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package jq. 2020-11-28T21:44:25.463 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up librdkafka1:amd64 (0.11.3-1build1) ... 2020-11-28T21:44:25.468 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../085-jq_1.5+dfsg-2_amd64.deb ... 2020-11-28T21:44:25.515 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-pytest. 2020-11-28T21:44:25.528 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../112-python3-pytest_3.3.2-2_all.deb ... 2020-11-28T21:44:25.530 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking jq (1.5+dfsg-2) ... 2020-11-28T21:44:25.564 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-pytest (3.3.2-2) ... 2020-11-28T21:44:25.589 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2020-11-28T21:44:25.749 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package socat. 2020-11-28T21:44:25.762 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../086-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2020-11-28T21:44:25.800 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2020-11-28T21:44:25.893 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-simplejson. 2020-11-28T21:44:25.905 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../113-python3-simplejson_3.13.2-1_amd64.deb ... 2020-11-28T21:44:25.943 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-simplejson (3.13.2-1) ... 2020-11-28T21:44:26.178 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package libxslt1.1:amd64. 2020-11-28T21:44:26.183 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-pil:amd64 (5.1.0-1ubuntu0.3) ... 2020-11-28T21:44:26.190 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../087-libxslt1.1_1.1.29-5ubuntu0.2_amd64.deb ... 2020-11-28T21:44:26.227 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-11-28T21:44:26.280 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-tk:amd64. 2020-11-28T21:44:26.294 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../114-python3-tk_3.6.9-1~18.04_amd64.deb ... 2020-11-28T21:44:26.338 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-tk:amd64 (3.6.9-1~18.04) ... 2020-11-28T21:44:26.531 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package xmlstarlet. 2020-11-28T21:44:26.545 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../088-xmlstarlet_1.6.1-2_amd64.deb ... 2020-11-28T21:44:26.569 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-simplegeneric (0.8.1-1) ... 2020-11-28T21:44:26.589 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2020-11-28T21:44:26.593 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-colorama. 2020-11-28T21:44:26.606 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../090-python3-colorama_0.3.7-1_all.deb ... 2020-11-28T21:44:26.711 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-colorama (0.3.7-1) ... 2020-11-28T21:44:26.775 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package radosgw. 2020-11-28T21:44:26.786 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../115-radosgw_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:44:26.824 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking radosgw (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:26.853 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-webencodings (0.5-2) ... 2020-11-28T21:44:26.927 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package ceph-test. 2020-11-28T21:44:26.940 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../089-ceph-test_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:44:26.976 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking ceph-test (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:27.088 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package cephfs-shell. 2020-11-28T21:44:27.099 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../091-cephfs-shell_15.2.6-595-ga83a3572-1bionic_all.deb ... 2020-11-28T21:44:27.129 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking cephfs-shell (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:27.155 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python-attr (17.4.0-2) ... 2020-11-28T21:44:27.356 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-cffi-backend (1.11.5-1) ... 2020-11-28T21:44:27.390 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libcephfs-dev. 2020-11-28T21:44:27.403 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../092-libcephfs-dev_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:44:27.473 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libcephfs-dev (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:27.740 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-py (1.5.2-1) ... 2020-11-28T21:44:27.750 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package nvme-cli. 2020-11-28T21:44:27.763 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../093-nvme-cli_1.5-1ubuntu1.1_amd64.deb ... 2020-11-28T21:44:27.778 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package rbd-fuse. 2020-11-28T21:44:27.791 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../116-rbd-fuse_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:44:27.800 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking nvme-cli (1.5-1ubuntu1.1) ... 2020-11-28T21:44:27.827 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking rbd-fuse (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:28.050 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-waitress (1.0.1-1) ... 2020-11-28T21:44:28.095 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-logutils (0.3.3-5) ... 2020-11-28T21:44:28.130 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python-attr. 2020-11-28T21:44:28.144 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../094-python-attr_17.4.0-2_all.deb ... 2020-11-28T21:44:28.188 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python-attr (17.4.0-2) ... 2020-11-28T21:44:28.399 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-werkzeug (0.14.1+dfsg1-1) ... 2020-11-28T21:44:28.407 INFO:teuthology.orchestra.run.smithi118.stdout:update-alternatives: using /usr/bin/waitress-serve-python3 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2020-11-28T21:44:28.432 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python-funcsigs. 2020-11-28T21:44:28.444 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../095-python-funcsigs_1.0.2-4_all.deb ... 2020-11-28T21:44:28.474 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python-funcsigs (1.0.2-4) ... 2020-11-28T21:44:28.695 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-olefile (0.45.1-1) ... 2020-11-28T21:44:28.767 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up librdkafka1:amd64 (0.11.3-1build1) ... 2020-11-28T21:44:28.777 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python-pluggy. 2020-11-28T21:44:28.789 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../096-python-pluggy_0.6.0-1_all.deb ... 2020-11-28T21:44:28.826 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python-pluggy (0.6.0-1) ... 2020-11-28T21:44:28.893 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2020-11-28T21:44:28.996 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-ceph-argparse (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:29.129 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python-py. 2020-11-28T21:44:29.142 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../097-python-py_1.5.2-1_all.deb ... 2020-11-28T21:44:29.179 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python-py (1.5.2-1) ... 2020-11-28T21:44:29.322 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-tempita (0.5.2-2) ... 2020-11-28T21:44:29.487 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-pil:amd64 (5.1.0-1ubuntu0.3) ... 2020-11-28T21:44:29.518 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python-pytest. 2020-11-28T21:44:29.528 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../098-python-pytest_3.3.2-2_all.deb ... 2020-11-28T21:44:29.565 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python-pytest (3.3.2-2) ... 2020-11-28T21:44:29.615 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2020-11-28T21:44:29.716 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-numpy (1:1.13.3-2ubuntu1) ... 2020-11-28T21:44:29.855 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-simplegeneric (0.8.1-1) ... 2020-11-28T21:44:29.894 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-attr. 2020-11-28T21:44:29.908 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../099-python3-attr_17.4.0-2_all.deb ... 2020-11-28T21:44:29.952 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-attr (17.4.0-2) ... 2020-11-28T21:44:30.140 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-webencodings (0.5-2) ... 2020-11-28T21:44:30.222 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-cycler. 2020-11-28T21:44:30.236 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../100-python3-cycler_0.10.0-1_all.deb ... 2020-11-28T21:44:30.288 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-cycler (0.10.0-1) ... 2020-11-28T21:44:30.297 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-colorama. 2020-11-28T21:44:30.309 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../090-python3-colorama_0.3.7-1_all.deb ... 2020-11-28T21:44:30.347 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-colorama (0.3.7-1) ... 2020-11-28T21:44:30.433 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python-attr (17.4.0-2) ... 2020-11-28T21:44:30.549 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-webencodings. 2020-11-28T21:44:30.562 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../101-python3-webencodings_0.5-2_all.deb ... 2020-11-28T21:44:30.575 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package cephfs-shell. 2020-11-28T21:44:30.589 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../091-cephfs-shell_15.2.6-595-ga83a3572-1bionic_all.deb ... 2020-11-28T21:44:30.599 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-webencodings (0.5-2) ... 2020-11-28T21:44:30.633 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking cephfs-shell (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:30.642 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-cffi-backend (1.11.5-1) ... 2020-11-28T21:44:30.644 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-webob (1:1.7.3-2fakesync1) ... 2020-11-28T21:44:30.777 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-py (1.5.2-1) ... 2020-11-28T21:44:30.902 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-html5lib. 2020-11-28T21:44:30.911 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package libcephfs-dev. 2020-11-28T21:44:30.915 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../102-python3-html5lib_0.999999999-1_all.deb ... 2020-11-28T21:44:30.922 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../092-libcephfs-dev_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:44:30.952 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-html5lib (0.999999999-1) ... 2020-11-28T21:44:30.961 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking libcephfs-dev (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:30.963 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-bcrypt (3.1.4-2) ... 2020-11-28T21:44:31.095 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-waitress (1.0.1-1) ... 2020-11-28T21:44:31.255 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package nvme-cli. 2020-11-28T21:44:31.256 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python-py (1.5.2-1) ... 2020-11-28T21:44:31.267 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../093-nvme-cli_1.5-1ubuntu1.1_amd64.deb ... 2020-11-28T21:44:31.288 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-lxml:amd64. 2020-11-28T21:44:31.300 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../103-python3-lxml_4.2.1-1ubuntu0.1_amd64.deb ... 2020-11-28T21:44:31.423 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking nvme-cli (1.5-1ubuntu1.1) ... 2020-11-28T21:44:31.423 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-11-28T21:44:31.440 INFO:teuthology.orchestra.run.smithi162.stdout:update-alternatives: using /usr/bin/waitress-serve-python3 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2020-11-28T21:44:31.475 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-simplejson (3.13.2-1) ... 2020-11-28T21:44:31.635 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python-attr. 2020-11-28T21:44:31.650 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../094-python-attr_17.4.0-2_all.deb ... 2020-11-28T21:44:31.693 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python-attr (17.4.0-2) ... 2020-11-28T21:44:31.714 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-olefile (0.45.1-1) ... 2020-11-28T21:44:31.767 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-tz. 2020-11-28T21:44:31.775 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-json-pointer (1.10-1) ... 2020-11-28T21:44:31.778 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../104-python3-tz_2018.3-2_all.deb ... 2020-11-28T21:44:31.816 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-tz (2018.3-2) ... 2020-11-28T21:44:31.866 INFO:teuthology.orchestra.run.smithi118.stdout:update-alternatives: using /usr/bin/python3-jsonpointer to provide /usr/bin/jsonpointer (jsonpointer) in auto mode 2020-11-28T21:44:31.996 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python-funcsigs. 2020-11-28T21:44:32.010 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../095-python-funcsigs_1.0.2-4_all.deb ... 2020-11-28T21:44:32.015 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-ceph-argparse (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:32.054 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python-funcsigs (1.0.2-4) ... 2020-11-28T21:44:32.102 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2020-11-28T21:44:32.119 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-matplotlib. 2020-11-28T21:44:32.131 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../105-python3-matplotlib_2.1.1-2ubuntu3_amd64.deb ... 2020-11-28T21:44:32.177 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-matplotlib (2.1.1-2ubuntu3) ... 2020-11-28T21:44:32.228 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python-pluggy (0.6.0-1) ... 2020-11-28T21:44:32.341 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-tempita (0.5.2-2) ... 2020-11-28T21:44:32.357 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python-pluggy. 2020-11-28T21:44:32.370 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../096-python-pluggy_0.6.0-1_all.deb ... 2020-11-28T21:44:32.407 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python-pluggy (0.6.0-1) ... 2020-11-28T21:44:32.429 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-colorama (0.3.7-1) ... 2020-11-28T21:44:32.643 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2020-11-28T21:44:32.710 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python-py. 2020-11-28T21:44:32.724 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../097-python-py_1.5.2-1_all.deb ... 2020-11-28T21:44:32.739 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-singledispatch (3.4.0.3-2) ... 2020-11-28T21:44:32.760 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python-py (1.5.2-1) ... 2020-11-28T21:44:32.777 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-numpy (1:1.13.3-2ubuntu1) ... 2020-11-28T21:44:32.833 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-olefile. 2020-11-28T21:44:32.846 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../106-python3-olefile_0.45.1-1_all.deb ... 2020-11-28T21:44:32.884 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-olefile (0.45.1-1) ... 2020-11-28T21:44:33.031 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-pyparsing (2.2.0+dfsg1-2) ... 2020-11-28T21:44:33.038 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python-pytest. 2020-11-28T21:44:33.052 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../098-python-pytest_3.3.2-2_all.deb ... 2020-11-28T21:44:33.097 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python-pytest (3.3.2-2) ... 2020-11-28T21:44:33.194 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pastescript. 2020-11-28T21:44:33.208 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-2_all.deb ... 2020-11-28T21:44:33.260 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pastescript (2.0.2-2) ... 2020-11-28T21:44:33.358 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-cycler (0.10.0-1) ... 2020-11-28T21:44:33.409 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-attr. 2020-11-28T21:44:33.422 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../099-python3-attr_17.4.0-2_all.deb ... 2020-11-28T21:44:33.459 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-attr (17.4.0-2) ... 2020-11-28T21:44:33.597 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pil:amd64. 2020-11-28T21:44:33.609 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../108-python3-pil_5.1.0-1ubuntu0.3_amd64.deb ... 2020-11-28T21:44:33.625 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-cherrypy3 (8.9.1-2) ... 2020-11-28T21:44:33.647 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.3) ... 2020-11-28T21:44:33.697 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-webob (1:1.7.3-2fakesync1) ... 2020-11-28T21:44:33.761 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-cycler. 2020-11-28T21:44:33.774 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../100-python3-cycler_0.10.0-1_all.deb ... 2020-11-28T21:44:33.811 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-cycler (0.10.0-1) ... 2020-11-28T21:44:34.000 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pluggy. 2020-11-28T21:44:34.002 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-bs4 (4.6.0-1) ... 2020-11-28T21:44:34.012 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../109-python3-pluggy_0.6.0-1_all.deb ... 2020-11-28T21:44:34.023 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-bcrypt (3.1.4-2) ... 2020-11-28T21:44:34.050 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pluggy (0.6.0-1) ... 2020-11-28T21:44:34.115 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-webencodings. 2020-11-28T21:44:34.127 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../101-python3-webencodings_0.5-2_all.deb ... 2020-11-28T21:44:34.181 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-webencodings (0.5-2) ... 2020-11-28T21:44:34.299 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python-py (1.5.2-1) ... 2020-11-28T21:44:34.345 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-pluggy (0.6.0-1) ... 2020-11-28T21:44:34.361 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-py. 2020-11-28T21:44:34.374 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../110-python3-py_1.5.2-1_all.deb ... 2020-11-28T21:44:34.411 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-py (1.5.2-1) ... 2020-11-28T21:44:34.451 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-html5lib. 2020-11-28T21:44:34.463 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../102-python3-html5lib_0.999999999-1_all.deb ... 2020-11-28T21:44:34.501 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-html5lib (0.999999999-1) ... 2020-11-28T21:44:34.517 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-simplejson (3.13.2-1) ... 2020-11-28T21:44:34.638 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up cephadm (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:34.706 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pyinotify. 2020-11-28T21:44:34.718 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1_all.deb ... 2020-11-28T21:44:34.756 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pyinotify (0.9.6-1) ... 2020-11-28T21:44:34.795 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-lxml:amd64. 2020-11-28T21:44:34.808 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../103-python3-lxml_4.2.1-1ubuntu0.1_amd64.deb ... 2020-11-28T21:44:34.846 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-11-28T21:44:34.860 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-json-pointer (1.10-1) ... 2020-11-28T21:44:34.933 INFO:teuthology.orchestra.run.smithi162.stdout:update-alternatives: using /usr/bin/python3-jsonpointer to provide /usr/bin/jsonpointer (jsonpointer) in auto mode 2020-11-28T21:44:35.067 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pytest. 2020-11-28T21:44:35.079 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../112-python3-pytest_3.3.2-2_all.deb ... 2020-11-28T21:44:35.117 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pytest (3.3.2-2) ... 2020-11-28T21:44:35.178 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2020-11-28T21:44:35.287 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python-pluggy (0.6.0-1) ... 2020-11-28T21:44:35.299 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-tz. 2020-11-28T21:44:35.313 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../104-python3-tz_2018.3-2_all.deb ... 2020-11-28T21:44:35.332 INFO:teuthology.orchestra.run.smithi118.stdout:Adding system user cephadm....done 2020-11-28T21:44:35.349 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-tz (2018.3-2) ... 2020-11-28T21:44:35.470 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-simplejson. 2020-11-28T21:44:35.483 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../113-python3-simplejson_3.13.2-1_amd64.deb ... 2020-11-28T21:44:35.487 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-markupsafe (1.0-1build1) ... 2020-11-28T21:44:35.488 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-colorama (0.3.7-1) ... 2020-11-28T21:44:35.520 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-simplejson (3.13.2-1) ... 2020-11-28T21:44:35.652 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-matplotlib. 2020-11-28T21:44:35.664 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../105-python3-matplotlib_2.1.1-2ubuntu3_amd64.deb ... 2020-11-28T21:44:35.702 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-matplotlib (2.1.1-2ubuntu3) ... 2020-11-28T21:44:35.781 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-singledispatch (3.4.0.3-2) ... 2020-11-28T21:44:35.788 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-11-28T21:44:35.840 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-tk:amd64. 2020-11-28T21:44:35.852 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../114-python3-tk_3.6.9-1~18.04_amd64.deb ... 2020-11-28T21:44:35.914 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up nvme-cli (1.5-1ubuntu1.1) ... 2020-11-28T21:44:35.931 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-tk:amd64 (3.6.9-1~18.04) ... 2020-11-28T21:44:36.074 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-pyparsing (2.2.0+dfsg1-2) ... 2020-11-28T21:44:36.082 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-asn1crypto (0.24.0-1) ... 2020-11-28T21:44:36.376 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package radosgw. 2020-11-28T21:44:36.388 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../115-radosgw_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:44:36.391 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-olefile. 2020-11-28T21:44:36.400 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-cycler (0.10.0-1) ... 2020-11-28T21:44:36.403 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../106-python3-olefile_0.45.1-1_all.deb ... 2020-11-28T21:44:36.425 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-prettytable (0.7.2-3) ... 2020-11-28T21:44:36.426 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking radosgw (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:36.457 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-olefile (0.45.1-1) ... 2020-11-28T21:44:36.693 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-cherrypy3 (8.9.1-2) ... 2020-11-28T21:44:36.701 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2020-11-28T21:44:36.719 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-pastescript. 2020-11-28T21:44:36.731 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-2_all.deb ... 2020-11-28T21:44:36.794 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-pastescript (2.0.2-2) ... 2020-11-28T21:44:36.835 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-pyasn1 (0.4.2-3) ... 2020-11-28T21:44:37.061 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-bs4 (4.6.0-1) ... 2020-11-28T21:44:37.131 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-pil:amd64. 2020-11-28T21:44:37.143 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../108-python3-pil_5.1.0-1ubuntu0.3_amd64.deb ... 2020-11-28T21:44:37.162 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-ceph-common (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:37.181 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.3) ... 2020-11-28T21:44:37.379 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-pluggy (0.6.0-1) ... 2020-11-28T21:44:37.380 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package rbd-fuse. 2020-11-28T21:44:37.391 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../116-rbd-fuse_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:44:37.429 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking rbd-fuse (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:37.474 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-monotonic (1.1-2) ... 2020-11-28T21:44:37.551 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-pluggy. 2020-11-28T21:44:37.563 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../109-python3-pluggy_0.6.0-1_all.deb ... 2020-11-28T21:44:37.601 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-pluggy (0.6.0-1) ... 2020-11-28T21:44:37.672 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up cephadm (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:37.716 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-logutils (0.3.3-5) ... 2020-11-28T21:44:37.772 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2020-11-28T21:44:37.890 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-11-28T21:44:37.920 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-py. 2020-11-28T21:44:37.933 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../110-python3-py_1.5.2-1_all.deb ... 2020-11-28T21:44:37.970 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-py (1.5.2-1) ... 2020-11-28T21:44:38.009 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-werkzeug (0.14.1+dfsg1-1) ... 2020-11-28T21:44:38.225 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-html5lib (0.999999999-1) ... 2020-11-28T21:44:38.282 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-pyinotify. 2020-11-28T21:44:38.296 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1_all.deb ... 2020-11-28T21:44:38.340 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-pyinotify (0.9.6-1) ... 2020-11-28T21:44:38.364 INFO:teuthology.orchestra.run.smithi162.stdout:Adding system user cephadm....done 2020-11-28T21:44:38.377 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up librdkafka1:amd64 (0.11.3-1build1) ... 2020-11-28T21:44:38.503 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2020-11-28T21:44:38.512 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-markupsafe (1.0-1build1) ... 2020-11-28T21:44:38.561 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2020-11-28T21:44:38.643 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-pytest. 2020-11-28T21:44:38.650 INFO:teuthology.orchestra.run.smithi118.stdout:update-alternatives: using /usr/bin/python3-jsondiff to provide /usr/bin/jsondiff (jsondiff) in auto mode 2020-11-28T21:44:38.657 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../112-python3-pytest_3.3.2-2_all.deb ... 2020-11-28T21:44:38.682 INFO:teuthology.orchestra.run.smithi118.stdout:update-alternatives: using /usr/bin/python3-jsonpatch to provide /usr/bin/jsonpatch (jsonpatch) in auto mode 2020-11-28T21:44:38.693 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-pytest (3.3.2-2) ... 2020-11-28T21:44:38.804 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-11-28T21:44:38.920 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-repoze.lru (0.7-1) ... 2020-11-28T21:44:38.930 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up nvme-cli (1.5-1ubuntu1.1) ... 2020-11-28T21:44:38.980 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-simplejson. 2020-11-28T21:44:38.994 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../113-python3-simplejson_3.13.2-1_amd64.deb ... 2020-11-28T21:44:39.030 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-simplejson (3.13.2-1) ... 2020-11-28T21:44:39.073 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-asn1crypto (0.24.0-1) ... 2020-11-28T21:44:39.097 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pil:amd64 (5.1.0-1ubuntu0.3) ... 2020-11-28T21:44:39.238 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2020-11-28T21:44:39.333 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-tk:amd64. 2020-11-28T21:44:39.347 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../114-python3-tk_3.6.9-1~18.04_amd64.deb ... 2020-11-28T21:44:39.400 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-tk:amd64 (3.6.9-1~18.04) ... 2020-11-28T21:44:39.425 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-prettytable (0.7.2-3) ... 2020-11-28T21:44:39.457 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-simplegeneric (0.8.1-1) ... 2020-11-28T21:44:39.539 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2) ... 2020-11-28T21:44:39.726 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2020-11-28T21:44:39.733 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-webencodings (0.5-2) ... 2020-11-28T21:44:39.845 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package radosgw. 2020-11-28T21:44:39.852 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-pyasn1 (0.4.2-3) ... 2020-11-28T21:44:39.858 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../115-radosgw_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:44:39.882 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-attr (17.4.0-2) ... 2020-11-28T21:44:39.895 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking radosgw (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:40.018 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python-attr (17.4.0-2) ... 2020-11-28T21:44:40.166 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python-funcsigs (1.0.2-4) ... 2020-11-28T21:44:40.178 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-ceph-common (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:40.219 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-cffi-backend (1.11.5-1) ... 2020-11-28T21:44:40.345 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-py (1.5.2-1) ... 2020-11-28T21:44:40.367 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-paste (2.0.3+dfsg-4ubuntu1) ... 2020-11-28T21:44:40.471 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-monotonic (1.1-2) ... 2020-11-28T21:44:40.671 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-waitress (1.0.1-1) ... 2020-11-28T21:44:40.756 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2020-11-28T21:44:40.778 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-websocket (0.44.0-0ubuntu2) ... 2020-11-28T21:44:40.832 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package rbd-fuse. 2020-11-28T21:44:40.844 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../116-rbd-fuse_15.2.6-595-ga83a3572-1bionic_amd64.deb ... 2020-11-28T21:44:40.867 INFO:teuthology.orchestra.run.smithi118.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2020-11-28T21:44:40.881 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-11-28T21:44:40.882 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking rbd-fuse (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:41.023 INFO:teuthology.orchestra.run.smithi096.stdout:update-alternatives: using /usr/bin/waitress-serve-python3 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2020-11-28T21:44:41.113 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up liboath0 (2.6.1-1) ... 2020-11-28T21:44:41.176 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-logutils (0.3.3-5) ... 2020-11-28T21:44:41.241 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-html5lib (0.999999999-1) ... 2020-11-28T21:44:41.282 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-olefile (0.45.1-1) ... 2020-11-28T21:44:41.298 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2020-11-28T21:44:41.415 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-tk:amd64 (3.6.9-1~18.04) ... 2020-11-28T21:44:41.487 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-werkzeug (0.14.1+dfsg1-1) ... 2020-11-28T21:44:41.567 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-ceph-argparse (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:41.601 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2020-11-28T21:44:41.650 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2020-11-28T21:44:41.691 INFO:teuthology.orchestra.run.smithi162.stdout:update-alternatives: using /usr/bin/python3-jsondiff to provide /usr/bin/jsondiff (jsondiff) in auto mode 2020-11-28T21:44:41.723 INFO:teuthology.orchestra.run.smithi162.stdout:update-alternatives: using /usr/bin/python3-jsonpatch to provide /usr/bin/jsonpatch (jsonpatch) in auto mode 2020-11-28T21:44:41.759 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-setuptools (39.0.1-2) ... 2020-11-28T21:44:41.847 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up librdkafka1:amd64 (0.11.3-1build1) ... 2020-11-28T21:44:41.868 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-tempita (0.5.2-2) ... 2020-11-28T21:44:41.944 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-repoze.lru (0.7-1) ... 2020-11-28T21:44:41.998 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2020-11-28T21:44:42.170 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2020-11-28T21:44:42.178 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2020-11-28T21:44:42.246 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2020-11-28T21:44:42.295 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-numpy (1:1.13.3-2ubuntu1) ... 2020-11-28T21:44:42.304 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-decorator (4.1.2-1) ... 2020-11-28T21:44:42.547 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2) ... 2020-11-28T21:44:42.593 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-pil:amd64 (5.1.0-1ubuntu0.3) ... 2020-11-28T21:44:42.605 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-tz (2018.3-2) ... 2020-11-28T21:44:42.898 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-uritemplate (0.6-1ubuntu1) ... 2020-11-28T21:44:42.914 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-attr (17.4.0-2) ... 2020-11-28T21:44:42.969 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-simplegeneric (0.8.1-1) ... 2020-11-28T21:44:43.190 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-webob (1:1.7.3-2fakesync1) ... 2020-11-28T21:44:43.224 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-jwt (1.5.3+ds1-1) ... 2020-11-28T21:44:43.225 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python-funcsigs (1.0.2-4) ... 2020-11-28T21:44:43.245 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-webencodings (0.5-2) ... 2020-11-28T21:44:43.425 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-paste (2.0.3+dfsg-4ubuntu1) ... 2020-11-28T21:44:43.508 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-bcrypt (3.1.4-2) ... 2020-11-28T21:44:43.525 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-joblib (0.11-1) ... 2020-11-28T21:44:43.547 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python-attr (17.4.0-2) ... 2020-11-28T21:44:43.749 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-cffi-backend (1.11.5-1) ... 2020-11-28T21:44:43.792 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python-py (1.5.2-1) ... 2020-11-28T21:44:43.827 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-websocket (0.44.0-0ubuntu2) ... 2020-11-28T21:44:43.866 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-py (1.5.2-1) ... 2020-11-28T21:44:43.893 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-routes (2.4.1-1) ... 2020-11-28T21:44:43.908 INFO:teuthology.orchestra.run.smithi162.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2020-11-28T21:44:44.019 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-simplejson (3.13.2-1) ... 2020-11-28T21:44:44.162 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up liboath0 (2.6.1-1) ... 2020-11-28T21:44:44.192 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-waitress (1.0.1-1) ... 2020-11-28T21:44:44.228 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-pyinotify (0.9.6-1) ... 2020-11-28T21:44:44.321 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2020-11-28T21:44:44.362 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-json-pointer (1.10-1) ... 2020-11-28T21:44:44.443 INFO:teuthology.orchestra.run.smithi096.stdout:update-alternatives: using /usr/bin/python3-jsonpointer to provide /usr/bin/jsonpointer (jsonpointer) in auto mode 2020-11-28T21:44:44.447 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-tk:amd64 (3.6.9-1~18.04) ... 2020-11-28T21:44:44.545 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-dateutil (2.6.1-1) ... 2020-11-28T21:44:44.661 INFO:teuthology.orchestra.run.smithi192.stdout:update-alternatives: using /usr/bin/waitress-serve-python3 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2020-11-28T21:44:44.671 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2020-11-28T21:44:44.690 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2020-11-28T21:44:44.772 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python-pluggy (0.6.0-1) ... 2020-11-28T21:44:44.816 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-setuptools (39.0.1-2) ... 2020-11-28T21:44:44.872 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-pyperclip (1.6.0-1) ... 2020-11-28T21:44:44.948 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-colorama (0.3.7-1) ... 2020-11-28T21:44:45.181 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-cryptography (2.1.4-1ubuntu1.4) ... 2020-11-28T21:44:45.187 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-olefile (0.45.1-1) ... 2020-11-28T21:44:45.209 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2020-11-28T21:44:45.258 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-singledispatch (3.4.0.3-2) ... 2020-11-28T21:44:45.327 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-decorator (4.1.2-1) ... 2020-11-28T21:44:45.480 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-ceph-argparse (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:45.541 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-fasteners (0.12.0-3) ... 2020-11-28T21:44:45.559 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pyparsing (2.2.0+dfsg1-2) ... 2020-11-28T21:44:45.612 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-tz (2018.3-2) ... 2020-11-28T21:44:45.790 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-tempita (0.5.2-2) ... 2020-11-28T21:44:45.818 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up jq (1.5+dfsg-2) ... 2020-11-28T21:44:45.887 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-cycler (0.10.0-1) ... 2020-11-28T21:44:45.912 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-uritemplate (0.6-1ubuntu1) ... 2020-11-28T21:44:45.977 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-scipy (0.19.1-2ubuntu1) ... 2020-11-28T21:44:46.091 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2020-11-28T21:44:46.163 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-cherrypy3 (8.9.1-2) ... 2020-11-28T21:44:46.192 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-numpy (1:1.13.3-2ubuntu1) ... 2020-11-28T21:44:46.214 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-jwt (1.5.3+ds1-1) ... 2020-11-28T21:44:46.506 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-bs4 (4.6.0-1) ... 2020-11-28T21:44:46.523 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-joblib (0.11-1) ... 2020-11-28T21:44:46.834 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pluggy (0.6.0-1) ... 2020-11-28T21:44:46.883 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-routes (2.4.1-1) ... 2020-11-28T21:44:47.150 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-11-28T21:44:47.153 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-webob (1:1.7.3-2fakesync1) ... 2020-11-28T21:44:47.158 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up cephadm (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:47.185 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-pyinotify (0.9.6-1) ... 2020-11-28T21:44:47.463 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-bcrypt (3.1.4-2) ... 2020-11-28T21:44:47.485 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-mako (1.0.7+ds1-1) ... 2020-11-28T21:44:47.494 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-dateutil (2.6.1-1) ... 2020-11-28T21:44:47.748 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python-py (1.5.2-1) ... 2020-11-28T21:44:47.761 INFO:teuthology.orchestra.run.smithi096.stdout:Adding system user cephadm....done 2020-11-28T21:44:47.803 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-pyperclip (1.6.0-1) ... 2020-11-28T21:44:47.820 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-sklearn-lib (0.19.1-3) ... 2020-11-28T21:44:47.890 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-markupsafe (1.0-1build1) ... 2020-11-28T21:44:47.963 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-pytest (3.3.2-2) ... 2020-11-28T21:44:47.974 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-simplejson (3.13.2-1) ... 2020-11-28T21:44:48.183 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-11-28T21:44:48.221 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-cryptography (2.1.4-1ubuntu1.4) ... 2020-11-28T21:44:48.309 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up nvme-cli (1.5-1ubuntu1.1) ... 2020-11-28T21:44:48.317 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-json-pointer (1.10-1) ... 2020-11-28T21:44:48.339 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-cmd2 (0.7.9-0ubuntu1) ... 2020-11-28T21:44:48.398 INFO:teuthology.orchestra.run.smithi192.stdout:update-alternatives: using /usr/bin/python3-jsonpointer to provide /usr/bin/jsonpointer (jsonpointer) in auto mode 2020-11-28T21:44:48.452 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-asn1crypto (0.24.0-1) ... 2020-11-28T21:44:48.589 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-fasteners (0.12.0-3) ... 2020-11-28T21:44:48.635 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2020-11-28T21:44:48.648 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python-pytest (3.3.2-2) ... 2020-11-28T21:44:48.786 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python-pluggy (0.6.0-1) ... 2020-11-28T21:44:48.795 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-prettytable (0.7.2-3) ... 2020-11-28T21:44:48.883 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2020-11-28T21:44:48.924 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up jq (1.5+dfsg-2) ... 2020-11-28T21:44:49.013 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-colorama (0.3.7-1) ... 2020-11-28T21:44:49.017 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up xmlstarlet (1.6.1-2) ... 2020-11-28T21:44:49.050 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-scipy (0.19.1-2ubuntu1) ... 2020-11-28T21:44:49.087 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2020-11-28T21:44:49.144 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up librados2 (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:49.206 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pyasn1 (0.4.2-3) ... 2020-11-28T21:44:49.270 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-rsa (3.4.2-1) ... 2020-11-28T21:44:49.314 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-singledispatch (3.4.0.3-2) ... 2020-11-28T21:44:49.507 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-ceph-common (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:49.588 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libcephfs2 (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:49.607 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-pyparsing (2.2.0+dfsg1-2) ... 2020-11-28T21:44:49.714 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libcephfs-dev (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:49.816 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-monotonic (1.1-2) ... 2020-11-28T21:44:49.840 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-rados (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:49.883 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-cycler (0.10.0-1) ... 2020-11-28T21:44:49.991 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-openssl (17.5.0-1ubuntu1) ... 2020-11-28T21:44:50.101 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2020-11-28T21:44:50.151 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-cherrypy3 (8.9.1-2) ... 2020-11-28T21:44:50.224 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-11-28T21:44:50.226 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-11-28T21:44:50.394 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-pastedeploy (1.5.2-4) ... 2020-11-28T21:44:50.519 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-bs4 (4.6.0-1) ... 2020-11-28T21:44:50.539 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-mako (1.0.7+ds1-1) ... 2020-11-28T21:44:50.594 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-html5lib (0.999999999-1) ... 2020-11-28T21:44:50.695 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-oauth2client (4.1.2-3) ... 2020-11-28T21:44:50.857 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-sklearn-lib (0.19.1-3) ... 2020-11-28T21:44:50.887 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-pluggy (0.6.0-1) ... 2020-11-28T21:44:50.954 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2020-11-28T21:44:50.983 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-pytest (3.3.2-2) ... 2020-11-28T21:44:51.022 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-matplotlib (2.1.1-2ubuntu3) ... 2020-11-28T21:44:51.027 INFO:teuthology.orchestra.run.smithi096.stdout:update-alternatives: using /usr/bin/python3-jsondiff to provide /usr/bin/jsondiff (jsondiff) in auto mode 2020-11-28T21:44:51.051 INFO:teuthology.orchestra.run.smithi096.stdout:update-alternatives: using /usr/bin/python3-jsonpatch to provide /usr/bin/jsonpatch (jsonpatch) in auto mode 2020-11-28T21:44:51.180 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up cephadm (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:51.264 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-repoze.lru (0.7-1) ... 2020-11-28T21:44:51.360 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-cmd2 (0.7.9-0ubuntu1) ... 2020-11-28T21:44:51.557 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2020-11-28T21:44:51.661 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python-pytest (3.3.2-2) ... 2020-11-28T21:44:51.848 INFO:teuthology.orchestra.run.smithi192.stdout:Adding system user cephadm....done 2020-11-28T21:44:51.850 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2) ... 2020-11-28T21:44:51.883 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-sklearn (0.19.1-3) ... 2020-11-28T21:44:51.904 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2020-11-28T21:44:52.012 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-markupsafe (1.0-1build1) ... 2020-11-28T21:44:52.030 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up xmlstarlet (1.6.1-2) ... 2020-11-28T21:44:52.156 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up librados2 (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:52.201 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-attr (17.4.0-2) ... 2020-11-28T21:44:52.282 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-rsa (3.4.2-1) ... 2020-11-28T21:44:52.339 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-11-28T21:44:52.456 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up nvme-cli (1.5-1ubuntu1.1) ... 2020-11-28T21:44:52.527 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python-funcsigs (1.0.2-4) ... 2020-11-28T21:44:52.591 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libcephfs2 (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:52.599 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-asn1crypto (0.24.0-1) ... 2020-11-28T21:44:52.704 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-webtest (2.0.28-1ubuntu1) ... 2020-11-28T21:44:52.717 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libcephfs-dev (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:52.720 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-paste (2.0.3+dfsg-4ubuntu1) ... 2020-11-28T21:44:52.843 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-rados (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:52.900 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-prettytable (0.7.2-3) ... 2020-11-28T21:44:52.952 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-openssl (17.5.0-1ubuntu1) ... 2020-11-28T21:44:52.989 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libradosstriper1 (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:53.123 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-cephfs (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:53.138 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-websocket (0.44.0-0ubuntu2) ... 2020-11-28T21:44:53.202 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2020-11-28T21:44:53.211 INFO:teuthology.orchestra.run.smithi096.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2020-11-28T21:44:53.328 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-pyasn1 (0.4.2-3) ... 2020-11-28T21:44:53.330 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-pastedeploy (1.5.2-4) ... 2020-11-28T21:44:53.424 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up librgw2 (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:53.465 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up liboath0 (2.6.1-1) ... 2020-11-28T21:44:53.533 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-fuse (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:53.641 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2020-11-28T21:44:53.642 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-oauth2client (4.1.2-3) ... 2020-11-28T21:44:53.663 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-ceph-common (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:53.727 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-11-28T21:44:53.727 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-11-28T21:44:53.768 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-tk:amd64 (3.6.9-1~18.04) ... 2020-11-28T21:44:53.966 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-matplotlib (2.1.1-2ubuntu3) ... 2020-11-28T21:44:53.989 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-monotonic (1.1-2) ... 2020-11-28T21:44:53.994 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up librbd1 (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:54.027 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2020-11-28T21:44:54.120 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up cephfs-shell (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:54.161 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-setuptools (39.0.1-2) ... 2020-11-28T21:44:54.246 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-pecan (1.2.1-2) ... 2020-11-28T21:44:54.299 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2020-11-28T21:44:54.336 INFO:teuthology.orchestra.run.smithi118.stdout:update-alternatives: using /usr/bin/python3-pecan to provide /usr/bin/pecan (pecan) in auto mode 2020-11-28T21:44:54.376 INFO:teuthology.orchestra.run.smithi118.stdout:update-alternatives: using /usr/bin/python3-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2020-11-28T21:44:54.416 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-11-28T21:44:54.563 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2020-11-28T21:44:54.664 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-decorator (4.1.2-1) ... 2020-11-28T21:44:54.691 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-kubernetes (2.0.0-2ubuntu1) ... 2020-11-28T21:44:54.751 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-html5lib (0.999999999-1) ... 2020-11-28T21:44:54.815 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-sklearn (0.19.1-3) ... 2020-11-28T21:44:54.940 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-tz (2018.3-2) ... 2020-11-28T21:44:55.086 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2020-11-28T21:44:55.176 INFO:teuthology.orchestra.run.smithi192.stdout:update-alternatives: using /usr/bin/python3-jsondiff to provide /usr/bin/jsondiff (jsondiff) in auto mode 2020-11-28T21:44:55.208 INFO:teuthology.orchestra.run.smithi192.stdout:update-alternatives: using /usr/bin/python3-jsonpatch to provide /usr/bin/jsonpatch (jsonpatch) in auto mode 2020-11-28T21:44:55.242 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-uritemplate (0.6-1ubuntu1) ... 2020-11-28T21:44:55.352 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-mgr-modules-core (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:55.455 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-repoze.lru (0.7-1) ... 2020-11-28T21:44:55.480 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-pastescript (2.0.2-2) ... 2020-11-28T21:44:55.534 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-jwt (1.5.3+ds1-1) ... 2020-11-28T21:44:55.632 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-webtest (2.0.28-1ubuntu1) ... 2020-11-28T21:44:55.779 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2020-11-28T21:44:55.796 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-rgw (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:55.877 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-joblib (0.11-1) ... 2020-11-28T21:44:55.917 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libradosstriper1 (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:55.939 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up rbd-fuse (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:56.042 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-cephfs (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:56.065 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-rbd (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:56.090 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2) ... 2020-11-28T21:44:56.191 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-common (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:56.237 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-routes (2.4.1-1) ... 2020-11-28T21:44:56.335 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up librgw2 (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:56.433 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-attr (17.4.0-2) ... 2020-11-28T21:44:56.453 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-fuse (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:56.529 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pyinotify (0.9.6-1) ... 2020-11-28T21:44:56.652 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-11-28T21:44:56.652 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-11-28T21:44:56.692 INFO:teuthology.orchestra.run.smithi118.stdout:Adding group ceph....done 2020-11-28T21:44:56.726 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python-funcsigs (1.0.2-4) ... 2020-11-28T21:44:56.822 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-dateutil (2.6.1-1) ... 2020-11-28T21:44:56.911 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-paste (2.0.3+dfsg-4ubuntu1) ... 2020-11-28T21:44:56.929 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up librbd1 (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:57.055 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up cephfs-shell (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:57.095 INFO:teuthology.orchestra.run.smithi118.stdout:Adding system user ceph....done 2020-11-28T21:44:57.115 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pyperclip (1.6.0-1) ... 2020-11-28T21:44:57.181 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-pecan (1.2.1-2) ... 2020-11-28T21:44:57.270 INFO:teuthology.orchestra.run.smithi118.stdout:Setting system user ceph properties....done 2020-11-28T21:44:57.271 INFO:teuthology.orchestra.run.smithi162.stdout:update-alternatives: using /usr/bin/python3-pecan to provide /usr/bin/pecan (pecan) in auto mode 2020-11-28T21:44:57.303 INFO:teuthology.orchestra.run.smithi162.stdout:update-alternatives: using /usr/bin/python3-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2020-11-28T21:44:57.313 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-websocket (0.44.0-0ubuntu2) ... 2020-11-28T21:44:57.401 INFO:teuthology.orchestra.run.smithi192.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2020-11-28T21:44:57.425 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2020-11-28T21:44:57.595 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-kubernetes (2.0.0-2ubuntu1) ... 2020-11-28T21:44:57.642 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2020-11-28T21:44:57.648 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up liboath0 (2.6.1-1) ... 2020-11-28T21:44:57.710 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-cryptography (2.1.4-1ubuntu1.4) ... 2020-11-28T21:44:57.824 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2020-11-28T21:44:57.934 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-tk:amd64 (3.6.9-1~18.04) ... 2020-11-28T21:44:57.935 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up radosgw (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:58.101 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-fasteners (0.12.0-3) ... 2020-11-28T21:44:58.160 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2020-11-28T21:44:58.192 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-11-28T21:44:58.192 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-11-28T21:44:58.264 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-mgr-modules-core (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:58.278 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-setuptools (39.0.1-2) ... 2020-11-28T21:44:58.407 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-pastescript (2.0.2-2) ... 2020-11-28T21:44:58.435 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up jq (1.5+dfsg-2) ... 2020-11-28T21:44:58.570 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-scipy (0.19.1-2ubuntu1) ... 2020-11-28T21:44:58.679 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-test (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:58.696 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2020-11-28T21:44:58.716 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-rgw (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:58.805 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-base (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:58.822 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-decorator (4.1.2-1) ... 2020-11-28T21:44:58.834 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up rbd-fuse (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:58.952 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-rbd (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:59.061 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-common (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:59.132 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-tz (2018.3-2) ... 2020-11-28T21:44:59.162 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2020-11-28T21:44:59.423 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-mgr (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:59.442 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-uritemplate (0.6-1ubuntu1) ... 2020-11-28T21:44:59.613 INFO:teuthology.orchestra.run.smithi162.stdout:Adding group ceph....done 2020-11-28T21:44:59.639 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-11-28T21:44:59.640 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-11-28T21:44:59.709 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-jwt (1.5.3+ds1-1) ... 2020-11-28T21:44:59.750 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-11-28T21:44:59.908 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-mgr-diskprediction-local (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:44:59.999 INFO:teuthology.orchestra.run.smithi162.stdout:Adding system user ceph....done 2020-11-28T21:45:00.002 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-joblib (0.11-1) ... 2020-11-28T21:45:00.026 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-osd (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:00.059 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-mako (1.0.7+ds1-1) ... 2020-11-28T21:45:00.166 INFO:teuthology.orchestra.run.smithi162.stdout:Setting system user ceph properties....done 2020-11-28T21:45:00.318 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2020-11-28T21:45:00.395 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-routes (2.4.1-1) ... 2020-11-28T21:45:00.402 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-sklearn-lib (0.19.1-3) ... 2020-11-28T21:45:00.495 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-11-28T21:45:00.495 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-11-28T21:45:00.530 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pytest (3.3.2-2) ... 2020-11-28T21:45:00.534 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2020-11-28T21:45:00.713 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-pyinotify (0.9.6-1) ... 2020-11-28T21:45:00.755 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-mgr-cephadm (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:00.814 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up radosgw (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:00.888 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-cmd2 (0.7.9-0ubuntu1) ... 2020-11-28T21:45:00.923 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:01.015 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-dateutil (2.6.1-1) ... 2020-11-28T21:45:01.066 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-mgr-rook (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:01.072 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-11-28T21:45:01.073 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-11-28T21:45:01.164 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python-pytest (3.3.2-2) ... 2020-11-28T21:45:01.200 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-mds (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:01.333 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-pyperclip (1.6.0-1) ... 2020-11-28T21:45:01.400 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-11-28T21:45:01.400 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-11-28T21:45:01.432 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2020-11-28T21:45:01.449 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-test (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:01.558 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up xmlstarlet (1.6.1-2) ... 2020-11-28T21:45:01.576 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-base (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:01.626 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-cryptography (2.1.4-1ubuntu1.4) ... 2020-11-28T21:45:01.677 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-mon (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:01.702 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up librados2 (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:01.827 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-rsa (3.4.2-1) ... 2020-11-28T21:45:01.880 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-11-28T21:45:01.880 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-11-28T21:45:01.907 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2020-11-28T21:45:01.994 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-fasteners (0.12.0-3) ... 2020-11-28T21:45:02.137 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libcephfs2 (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:02.162 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-mgr-k8sevents (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:02.182 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-mgr (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:02.246 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libcephfs-dev (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:02.289 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-mgr-dashboard (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:02.312 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up jq (1.5+dfsg-2) ... 2020-11-28T21:45:02.355 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-rados (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:02.411 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-11-28T21:45:02.412 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-11-28T21:45:02.430 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:02.440 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-scipy (0.19.1-2ubuntu1) ... 2020-11-28T21:45:02.456 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-openssl (17.5.0-1ubuntu1) ... 2020-11-28T21:45:02.557 INFO:teuthology.orchestra.run.smithi118.stdout:Processing triggers for systemd (237-3ubuntu10.43) ... 2020-11-28T21:45:02.695 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-mgr-diskprediction-local (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:02.766 INFO:teuthology.orchestra.run.smithi118.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2020-11-28T21:45:02.838 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-osd (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:02.867 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pastedeploy (1.5.2-4) ... 2020-11-28T21:45:03.168 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-oauth2client (4.1.2-3) ... 2020-11-28T21:45:03.276 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-11-28T21:45:03.277 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-11-28T21:45:03.487 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-matplotlib (2.1.1-2ubuntu3) ... 2020-11-28T21:45:03.550 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-mgr-cephadm (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:03.620 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-11-28T21:45:03.709 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:03.852 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-mgr-rook (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:03.938 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-mako (1.0.7+ds1-1) ... 2020-11-28T21:45:03.986 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-mds (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:04.246 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-11-28T21:45:04.246 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-11-28T21:45:04.273 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-sklearn-lib (0.19.1-3) ... 2020-11-28T21:45:04.323 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-sklearn (0.19.1-3) ... 2020-11-28T21:45:04.422 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-pytest (3.3.2-2) ... 2020-11-28T21:45:04.521 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-mon (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:04.738 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-11-28T21:45:04.738 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-11-28T21:45:04.792 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-cmd2 (0.7.9-0ubuntu1) ... 2020-11-28T21:45:04.815 INFO:teuthology.orchestra.run.smithi118.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2020-11-28T21:45:04.855 INFO:teuthology.orchestra.run.smithi118.stdout:ureadahead will be reprofiled on next reboot 2020-11-28T21:45:04.912 INFO:teuthology.orchestra.run.smithi118.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2020-11-28T21:45:04.997 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-mgr-k8sevents (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:05.085 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python-pytest (3.3.2-2) ... 2020-11-28T21:45:05.123 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-mgr-dashboard (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:05.129 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-webtest (2.0.28-1ubuntu1) ... 2020-11-28T21:45:05.291 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:05.319 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2020-11-28T21:45:05.358 INFO:teuthology.orchestra.run.smithi118.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-11-28T21:45:05.360 INFO:teuthology.orchestra.run.smithi118:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2020-11-28T21:45:05.418 INFO:teuthology.orchestra.run.smithi162.stdout:Processing triggers for systemd (237-3ubuntu10.43) ... 2020-11-28T21:45:05.420 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libradosstriper1 (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:05.443 INFO:teuthology.orchestra.run.smithi118.stdout:Reading package lists... 2020-11-28T21:45:05.449 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up xmlstarlet (1.6.1-2) ... 2020-11-28T21:45:05.538 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-cephfs (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:05.571 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up librados2 (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:05.669 INFO:teuthology.orchestra.run.smithi162.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2020-11-28T21:45:05.698 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-rsa (3.4.2-1) ... 2020-11-28T21:45:05.703 INFO:teuthology.orchestra.run.smithi118.stdout:Building dependency tree... 2020-11-28T21:45:05.704 INFO:teuthology.orchestra.run.smithi118.stdout:Reading state information... 2020-11-28T21:45:05.805 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up librgw2 (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:05.879 INFO:teuthology.orchestra.run.smithi118.stdout:libtool-bin is already the newest version (2.4.6-2). 2020-11-28T21:45:05.879 INFO:teuthology.orchestra.run.smithi118.stdout:xfslibs-dev is already the newest version (4.9.0+nmu1ubuntu2). 2020-11-28T21:45:05.879 INFO:teuthology.orchestra.run.smithi118.stdout:uuid-dev is already the newest version (2.31.1-0.4ubuntu3.7). 2020-11-28T21:45:05.880 INFO:teuthology.orchestra.run.smithi118.stdout:The following additional packages will be installed: 2020-11-28T21:45:05.881 INFO:teuthology.orchestra.run.smithi118.stdout: libbison-dev libfl-dev libfl2 zlib1g-dev 2020-11-28T21:45:05.883 INFO:teuthology.orchestra.run.smithi118.stdout:Suggested packages: 2020-11-28T21:45:05.884 INFO:teuthology.orchestra.run.smithi118.stdout: bison-doc flex-doc indent-doc libssl-doc 2020-11-28T21:45:05.906 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-fuse (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:05.909 INFO:teuthology.orchestra.run.smithi118.stdout:The following NEW packages will be installed: 2020-11-28T21:45:05.910 INFO:teuthology.orchestra.run.smithi118.stdout: bison dump flex indent libaio-dev libbison-dev libelf-dev libfl-dev libfl2 2020-11-28T21:45:05.911 INFO:teuthology.orchestra.run.smithi118.stdout: libssl-dev zlib1g-dev 2020-11-28T21:45:06.024 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up libcephfs2 (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:06.089 INFO:teuthology.orchestra.run.smithi118.stdout:0 upgraded, 11 newly installed, 0 to remove and 5 not upgraded. 2020-11-28T21:45:06.090 INFO:teuthology.orchestra.run.smithi118.stdout:Need to get 2,960 kB of archives. 2020-11-28T21:45:06.090 INFO:teuthology.orchestra.run.smithi118.stdout:After this operation, 12.2 MB of additional disk space will be used. 2020-11-28T21:45:06.091 INFO:teuthology.orchestra.run.smithi118.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic/main amd64 flex amd64 2.6.4-6 [316 kB] 2020-11-28T21:45:06.127 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-11-28T21:45:06.127 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-11-28T21:45:06.175 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up libcephfs-dev (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:06.293 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-rados (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:06.394 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-openssl (17.5.0-1ubuntu1) ... 2020-11-28T21:45:06.416 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up librbd1 (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:06.540 INFO:teuthology.orchestra.run.smithi118.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/main amd64 libbison-dev amd64 2:3.0.4.dfsg-1build1 [339 kB] 2020-11-28T21:45:06.542 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up cephfs-shell (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:06.620 INFO:teuthology.orchestra.run.smithi118.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 bison amd64 2:3.0.4.dfsg-1build1 [266 kB] 2020-11-28T21:45:06.644 INFO:teuthology.orchestra.run.smithi118.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/universe amd64 dump amd64 0.4b46-3 [129 kB] 2020-11-28T21:45:06.654 INFO:teuthology.orchestra.run.smithi118.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 indent amd64 2.2.11-5 [80.7 kB] 2020-11-28T21:45:06.660 INFO:teuthology.orchestra.run.smithi118.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libaio-dev amd64 0.3.110-5ubuntu0.1 [12.8 kB] 2020-11-28T21:45:06.662 INFO:teuthology.orchestra.run.smithi118.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic/main amd64 zlib1g-dev amd64 1:1.2.11.dfsg-0ubuntu2 [176 kB] 2020-11-28T21:45:06.668 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pecan (1.2.1-2) ... 2020-11-28T21:45:06.710 INFO:teuthology.orchestra.run.smithi118.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libelf-dev amd64 0.170-0.4ubuntu0.1 [57.3 kB] 2020-11-28T21:45:06.714 INFO:teuthology.orchestra.run.smithi118.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl2 amd64 2.6.4-6 [11.4 kB] 2020-11-28T21:45:06.715 INFO:teuthology.orchestra.run.smithi118.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl-dev amd64 2.6.4-6 [6,320 B] 2020-11-28T21:45:06.716 INFO:teuthology.orchestra.run.smithi118.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libssl-dev amd64 1.1.1-1ubuntu2.1~18.04.6 [1,566 kB] 2020-11-28T21:45:06.758 INFO:teuthology.orchestra.run.smithi096.stdout:update-alternatives: using /usr/bin/python3-pecan to provide /usr/bin/pecan (pecan) in auto mode 2020-11-28T21:45:06.788 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-pastedeploy (1.5.2-4) ... 2020-11-28T21:45:06.790 INFO:teuthology.orchestra.run.smithi096.stdout:update-alternatives: using /usr/bin/python3-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2020-11-28T21:45:07.089 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-oauth2client (4.1.2-3) ... 2020-11-28T21:45:07.105 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-kubernetes (2.0.0-2ubuntu1) ... 2020-11-28T21:45:07.191 INFO:teuthology.orchestra.run.smithi118.stdout:Fetched 2,960 kB in 1s (3,370 kB/s) 2020-11-28T21:45:07.310 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package flex. 2020-11-28T21:45:07.351 INFO:teuthology.orchestra.run.smithi118.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 109921 files and directories currently installed.) 2020-11-28T21:45:07.354 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../00-flex_2.6.4-6_amd64.deb ... 2020-11-28T21:45:07.391 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking flex (2.6.4-6) ... 2020-11-28T21:45:07.416 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-matplotlib (2.1.1-2ubuntu3) ... 2020-11-28T21:45:07.680 INFO:teuthology.orchestra.run.smithi162.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2020-11-28T21:45:07.722 INFO:teuthology.orchestra.run.smithi162.stdout:ureadahead will be reprofiled on next reboot 2020-11-28T21:45:07.766 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mgr-modules-core (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:07.780 INFO:teuthology.orchestra.run.smithi162.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2020-11-28T21:45:07.821 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libbison-dev:amd64. 2020-11-28T21:45:07.832 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../01-libbison-dev_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2020-11-28T21:45:07.870 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2020-11-28T21:45:07.891 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pastescript (2.0.2-2) ... 2020-11-28T21:45:08.198 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package bison. 2020-11-28T21:45:08.209 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-rgw (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:08.212 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../02-bison_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2020-11-28T21:45:08.231 INFO:teuthology.orchestra.run.smithi162.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-11-28T21:45:08.233 INFO:teuthology.orchestra.run.smithi162:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2020-11-28T21:45:08.261 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking bison (2:3.0.4.dfsg-1build1) ... 2020-11-28T21:45:08.285 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-sklearn (0.19.1-3) ... 2020-11-28T21:45:08.312 INFO:teuthology.orchestra.run.smithi162.stdout:Reading package lists... 2020-11-28T21:45:08.335 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up rbd-fuse (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:08.462 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-rbd (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:08.573 INFO:teuthology.orchestra.run.smithi162.stdout:Building dependency tree... 2020-11-28T21:45:08.574 INFO:teuthology.orchestra.run.smithi162.stdout:Reading state information... 2020-11-28T21:45:08.579 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-common (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:08.748 INFO:teuthology.orchestra.run.smithi162.stdout:libtool-bin is already the newest version (2.4.6-2). 2020-11-28T21:45:08.749 INFO:teuthology.orchestra.run.smithi162.stdout:xfslibs-dev is already the newest version (4.9.0+nmu1ubuntu2). 2020-11-28T21:45:08.749 INFO:teuthology.orchestra.run.smithi162.stdout:uuid-dev is already the newest version (2.31.1-0.4ubuntu3.7). 2020-11-28T21:45:08.749 INFO:teuthology.orchestra.run.smithi162.stdout:The following additional packages will be installed: 2020-11-28T21:45:08.750 INFO:teuthology.orchestra.run.smithi162.stdout: libbison-dev libfl-dev libfl2 zlib1g-dev 2020-11-28T21:45:08.751 INFO:teuthology.orchestra.run.smithi162.stdout:Suggested packages: 2020-11-28T21:45:08.751 INFO:teuthology.orchestra.run.smithi162.stdout: bison-doc flex-doc indent-doc libssl-doc 2020-11-28T21:45:08.753 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package dump. 2020-11-28T21:45:08.767 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../03-dump_0.4b46-3_amd64.deb ... 2020-11-28T21:45:08.777 INFO:teuthology.orchestra.run.smithi162.stdout:The following NEW packages will be installed: 2020-11-28T21:45:08.778 INFO:teuthology.orchestra.run.smithi162.stdout: bison dump flex indent libaio-dev libbison-dev libelf-dev libfl-dev libfl2 2020-11-28T21:45:08.779 INFO:teuthology.orchestra.run.smithi162.stdout: libssl-dev zlib1g-dev 2020-11-28T21:45:08.828 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking dump (0.4b46-3) ... 2020-11-28T21:45:08.959 INFO:teuthology.orchestra.run.smithi162.stdout:0 upgraded, 11 newly installed, 0 to remove and 5 not upgraded. 2020-11-28T21:45:08.959 INFO:teuthology.orchestra.run.smithi162.stdout:Need to get 2,960 kB of archives. 2020-11-28T21:45:08.959 INFO:teuthology.orchestra.run.smithi162.stdout:After this operation, 12.2 MB of additional disk space will be used. 2020-11-28T21:45:08.959 INFO:teuthology.orchestra.run.smithi162.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic/main amd64 flex amd64 2.6.4-6 [316 kB] 2020-11-28T21:45:09.056 INFO:teuthology.orchestra.run.smithi096.stdout:Adding group ceph....done 2020-11-28T21:45:09.098 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-webtest (2.0.28-1ubuntu1) ... 2020-11-28T21:45:09.190 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package indent. 2020-11-28T21:45:09.202 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../04-indent_2.2.11-5_amd64.deb ... 2020-11-28T21:45:09.240 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking indent (2.2.11-5) ... 2020-11-28T21:45:09.396 INFO:teuthology.orchestra.run.smithi162.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/main amd64 libbison-dev amd64 2:3.0.4.dfsg-1build1 [339 kB] 2020-11-28T21:45:09.399 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up libradosstriper1 (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:09.467 INFO:teuthology.orchestra.run.smithi096.stdout:Adding system user ceph....done 2020-11-28T21:45:09.475 INFO:teuthology.orchestra.run.smithi162.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 bison amd64 2:3.0.4.dfsg-1build1 [266 kB] 2020-11-28T21:45:09.501 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libaio-dev:amd64. 2020-11-28T21:45:09.513 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../05-libaio-dev_0.3.110-5ubuntu0.1_amd64.deb ... 2020-11-28T21:45:09.526 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-cephfs (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:09.526 INFO:teuthology.orchestra.run.smithi162.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/universe amd64 dump amd64 0.4b46-3 [129 kB] 2020-11-28T21:45:09.544 INFO:teuthology.orchestra.run.smithi162.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 indent amd64 2.2.11-5 [80.7 kB] 2020-11-28T21:45:09.551 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2020-11-28T21:45:09.554 INFO:teuthology.orchestra.run.smithi162.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libaio-dev amd64 0.3.110-5ubuntu0.1 [12.8 kB] 2020-11-28T21:45:09.556 INFO:teuthology.orchestra.run.smithi162.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic/main amd64 zlib1g-dev amd64 1:1.2.11.dfsg-0ubuntu2 [176 kB] 2020-11-28T21:45:09.567 INFO:teuthology.orchestra.run.smithi162.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libelf-dev amd64 0.170-0.4ubuntu0.1 [57.3 kB] 2020-11-28T21:45:09.572 INFO:teuthology.orchestra.run.smithi162.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl2 amd64 2.6.4-6 [11.4 kB] 2020-11-28T21:45:09.574 INFO:teuthology.orchestra.run.smithi162.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl-dev amd64 2.6.4-6 [6,320 B] 2020-11-28T21:45:09.574 INFO:teuthology.orchestra.run.smithi162.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libssl-dev amd64 1.1.1-1ubuntu2.1~18.04.6 [1,566 kB] 2020-11-28T21:45:09.684 INFO:teuthology.orchestra.run.smithi096.stdout:Setting system user ceph properties....done 2020-11-28T21:45:09.813 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package zlib1g-dev:amd64. 2020-11-28T21:45:09.818 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up librgw2 (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:09.826 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../06-zlib1g-dev_1%3a1.2.11.dfsg-0ubuntu2_amd64.deb ... 2020-11-28T21:45:09.836 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2020-11-28T21:45:09.871 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2020-11-28T21:45:09.919 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up ceph-fuse (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:10.043 INFO:teuthology.orchestra.run.smithi162.stdout:Fetched 2,960 kB in 1s (3,396 kB/s) 2020-11-28T21:45:10.072 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2020-11-28T21:45:10.143 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-11-28T21:45:10.144 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-11-28T21:45:10.162 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package flex. 2020-11-28T21:45:10.166 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libelf-dev:amd64. 2020-11-28T21:45:10.178 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../07-libelf-dev_0.170-0.4ubuntu0.1_amd64.deb ... 2020-11-28T21:45:10.204 INFO:teuthology.orchestra.run.smithi162.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 109921 files and directories currently installed.) 2020-11-28T21:45:10.207 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../00-flex_2.6.4-6_amd64.deb ... 2020-11-28T21:45:10.216 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2020-11-28T21:45:10.235 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking flex (2.6.4-6) ... 2020-11-28T21:45:10.373 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up radosgw (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:10.404 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up librbd1 (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:10.536 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libfl2:amd64. 2020-11-28T21:45:10.547 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up cephfs-shell (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:10.548 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../08-libfl2_2.6.4-6_amd64.deb ... 2020-11-28T21:45:10.611 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libfl2:amd64 (2.6.4-6) ... 2020-11-28T21:45:10.615 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-11-28T21:45:10.616 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-11-28T21:45:10.631 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libbison-dev:amd64. 2020-11-28T21:45:10.642 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../01-libbison-dev_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2020-11-28T21:45:10.673 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-pecan (1.2.1-2) ... 2020-11-28T21:45:10.683 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2020-11-28T21:45:10.763 INFO:teuthology.orchestra.run.smithi192.stdout:update-alternatives: using /usr/bin/python3-pecan to provide /usr/bin/pecan (pecan) in auto mode 2020-11-28T21:45:10.795 INFO:teuthology.orchestra.run.smithi192.stdout:update-alternatives: using /usr/bin/python3-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2020-11-28T21:45:10.897 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libfl-dev:amd64. 2020-11-28T21:45:10.909 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../09-libfl-dev_2.6.4-6_amd64.deb ... 2020-11-28T21:45:10.947 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libfl-dev:amd64 (2.6.4-6) ... 2020-11-28T21:45:11.001 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package bison. 2020-11-28T21:45:11.008 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-test (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:11.014 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../02-bison_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2020-11-28T21:45:11.047 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking bison (2:3.0.4.dfsg-1build1) ... 2020-11-28T21:45:11.101 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-kubernetes (2.0.0-2ubuntu1) ... 2020-11-28T21:45:11.134 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-base (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:11.183 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libssl-dev:amd64. 2020-11-28T21:45:11.196 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../10-libssl-dev_1.1.1-1ubuntu2.1~18.04.6_amd64.deb ... 2020-11-28T21:45:11.233 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.6) ... 2020-11-28T21:45:11.455 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package dump. 2020-11-28T21:45:11.467 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../03-dump_0.4b46-3_amd64.deb ... 2020-11-28T21:45:11.475 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2020-11-28T21:45:11.505 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking dump (0.4b46-3) ... 2020-11-28T21:45:11.683 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up flex (2.6.4-6) ... 2020-11-28T21:45:11.738 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up ceph-mgr-modules-core (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:11.769 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mgr (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:11.801 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package indent. 2020-11-28T21:45:11.814 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../04-indent_2.2.11-5_amd64.deb ... 2020-11-28T21:45:11.822 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up dump (0.4b46-3) ... 2020-11-28T21:45:11.863 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-pastescript (2.0.2-2) ... 2020-11-28T21:45:11.867 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking indent (2.2.11-5) ... 2020-11-28T21:45:11.922 INFO:teuthology.orchestra.run.smithi118.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2020-11-28T21:45:11.985 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-11-28T21:45:11.986 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-11-28T21:45:11.990 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.6) ... 2020-11-28T21:45:12.100 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up indent (2.2.11-5) ... 2020-11-28T21:45:12.120 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libaio-dev:amd64. 2020-11-28T21:45:12.131 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../05-libaio-dev_0.3.110-5ubuntu0.1_amd64.deb ... 2020-11-28T21:45:12.161 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2020-11-28T21:45:12.195 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-rgw (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:12.200 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2020-11-28T21:45:12.287 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mgr-diskprediction-local (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:12.327 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libfl2:amd64 (2.6.4-6) ... 2020-11-28T21:45:12.332 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up rbd-fuse (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:12.415 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package zlib1g-dev:amd64. 2020-11-28T21:45:12.421 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-osd (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:12.428 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../06-zlib1g-dev_1%3a1.2.11.dfsg-0ubuntu2_amd64.deb ... 2020-11-28T21:45:12.453 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up bison (2:3.0.4.dfsg-1build1) ... 2020-11-28T21:45:12.458 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-rbd (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:12.473 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2020-11-28T21:45:12.551 INFO:teuthology.orchestra.run.smithi118.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2020-11-28T21:45:12.576 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up ceph-common (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:12.621 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2020-11-28T21:45:12.772 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2020-11-28T21:45:12.801 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libelf-dev:amd64. 2020-11-28T21:45:12.814 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../07-libelf-dev_0.170-0.4ubuntu0.1_amd64.deb ... 2020-11-28T21:45:12.851 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2020-11-28T21:45:12.876 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-11-28T21:45:12.876 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-11-28T21:45:12.898 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2020-11-28T21:45:13.024 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libfl-dev:amd64 (2.6.4-6) ... 2020-11-28T21:45:13.102 INFO:teuthology.orchestra.run.smithi192.stdout:Adding group ceph....done 2020-11-28T21:45:13.134 INFO:teuthology.orchestra.run.smithi118.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2020-11-28T21:45:13.171 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libfl2:amd64. 2020-11-28T21:45:13.174 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mgr-cephadm (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:13.184 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../08-libfl2_2.6.4-6_amd64.deb ... 2020-11-28T21:45:13.255 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libfl2:amd64 (2.6.4-6) ... 2020-11-28T21:45:13.317 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:13.468 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mgr-rook (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:13.480 INFO:teuthology.orchestra.run.smithi192.stdout:Adding system user ceph....done 2020-11-28T21:45:13.549 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libfl-dev:amd64. 2020-11-28T21:45:13.562 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../09-libfl-dev_2.6.4-6_amd64.deb ... 2020-11-28T21:45:13.600 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libfl-dev:amd64 (2.6.4-6) ... 2020-11-28T21:45:13.611 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mds (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:13.664 INFO:teuthology.orchestra.run.smithi192.stdout:Setting system user ceph properties....done 2020-11-28T21:45:13.811 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2020-11-28T21:45:13.823 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-11-28T21:45:13.823 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-11-28T21:45:13.852 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libssl-dev:amd64. 2020-11-28T21:45:13.865 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../10-libssl-dev_1.1.1-1ubuntu2.1~18.04.6_amd64.deb ... 2020-11-28T21:45:13.902 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.6) ... 2020-11-28T21:45:14.031 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2020-11-28T21:45:14.104 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mon (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:14.303 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up radosgw (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:14.305 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-11-28T21:45:14.305 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-11-28T21:45:14.378 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up flex (2.6.4-6) ... 2020-11-28T21:45:14.499 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up dump (0.4b46-3) ... 2020-11-28T21:45:14.561 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-11-28T21:45:14.562 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-11-28T21:45:14.588 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mgr-k8sevents (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:14.602 INFO:teuthology.orchestra.run.smithi162.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2020-11-28T21:45:14.668 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.6) ... 2020-11-28T21:45:14.698 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mgr-dashboard (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:14.769 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up indent (2.2.11-5) ... 2020-11-28T21:45:14.833 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:14.886 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2020-11-28T21:45:14.929 INFO:teuthology.orchestra.run.smithi118.stdout:Processing triggers for install-info (6.5.0.dfsg.1-2) ... 2020-11-28T21:45:14.930 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up ceph-test (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:14.959 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for systemd (237-3ubuntu10.43) ... 2020-11-28T21:45:15.012 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libfl2:amd64 (2.6.4-6) ... 2020-11-28T21:45:15.056 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up ceph-base (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:15.138 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up bison (2:3.0.4.dfsg-1build1) ... 2020-11-28T21:45:15.160 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2020-11-28T21:45:15.206 INFO:teuthology.orchestra.run.smithi118.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2020-11-28T21:45:15.228 INFO:teuthology.orchestra.run.smithi162.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2020-11-28T21:45:15.298 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2020-11-28T21:45:15.424 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2020-11-28T21:45:15.551 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2020-11-28T21:45:15.654 INFO:teuthology.orchestra.run.smithi118.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-11-28T21:45:15.657 DEBUG:teuthology.parallel:result is None 2020-11-28T21:45:15.661 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libfl-dev:amd64 (2.6.4-6) ... 2020-11-28T21:45:15.778 INFO:teuthology.orchestra.run.smithi162.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2020-11-28T21:45:15.929 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2020-11-28T21:45:16.200 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up ceph-mgr (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:16.423 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-11-28T21:45:16.424 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-11-28T21:45:16.702 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up ceph-mgr-diskprediction-local (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:16.820 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up ceph-osd (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:17.179 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2020-11-28T21:45:17.222 INFO:teuthology.orchestra.run.smithi096.stdout:ureadahead will be reprofiled on next reboot 2020-11-28T21:45:17.271 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2020-11-28T21:45:17.275 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-11-28T21:45:17.275 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-11-28T21:45:17.497 INFO:teuthology.orchestra.run.smithi162.stdout:Processing triggers for install-info (6.5.0.dfsg.1-2) ... 2020-11-28T21:45:17.547 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up ceph-mgr-cephadm (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:17.699 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:17.700 INFO:teuthology.orchestra.run.smithi096.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-11-28T21:45:17.702 INFO:teuthology.orchestra.run.smithi096:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2020-11-28T21:45:17.757 INFO:teuthology.orchestra.run.smithi162.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2020-11-28T21:45:17.811 INFO:teuthology.orchestra.run.smithi096.stdout:Reading package lists... 2020-11-28T21:45:17.841 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up ceph-mgr-rook (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:17.984 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up ceph-mds (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:18.069 INFO:teuthology.orchestra.run.smithi096.stdout:Building dependency tree... 2020-11-28T21:45:18.070 INFO:teuthology.orchestra.run.smithi096.stdout:Reading state information... 2020-11-28T21:45:18.187 INFO:teuthology.orchestra.run.smithi162.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-11-28T21:45:18.189 DEBUG:teuthology.parallel:result is None 2020-11-28T21:45:18.204 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-11-28T21:45:18.204 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-11-28T21:45:18.248 INFO:teuthology.orchestra.run.smithi096.stdout:libtool-bin is already the newest version (2.4.6-2). 2020-11-28T21:45:18.249 INFO:teuthology.orchestra.run.smithi096.stdout:xfslibs-dev is already the newest version (4.9.0+nmu1ubuntu2). 2020-11-28T21:45:18.249 INFO:teuthology.orchestra.run.smithi096.stdout:uuid-dev is already the newest version (2.31.1-0.4ubuntu3.7). 2020-11-28T21:45:18.249 INFO:teuthology.orchestra.run.smithi096.stdout:The following additional packages will be installed: 2020-11-28T21:45:18.250 INFO:teuthology.orchestra.run.smithi096.stdout: libbison-dev libfl-dev libfl2 zlib1g-dev 2020-11-28T21:45:18.251 INFO:teuthology.orchestra.run.smithi096.stdout:Suggested packages: 2020-11-28T21:45:18.251 INFO:teuthology.orchestra.run.smithi096.stdout: bison-doc flex-doc indent-doc libssl-doc 2020-11-28T21:45:18.278 INFO:teuthology.orchestra.run.smithi096.stdout:The following NEW packages will be installed: 2020-11-28T21:45:18.278 INFO:teuthology.orchestra.run.smithi096.stdout: bison dump flex indent libaio-dev libbison-dev libelf-dev libfl-dev libfl2 2020-11-28T21:45:18.279 INFO:teuthology.orchestra.run.smithi096.stdout: libssl-dev zlib1g-dev 2020-11-28T21:45:18.464 INFO:teuthology.orchestra.run.smithi096.stdout:0 upgraded, 11 newly installed, 0 to remove and 5 not upgraded. 2020-11-28T21:45:18.465 INFO:teuthology.orchestra.run.smithi096.stdout:Need to get 2,960 kB of archives. 2020-11-28T21:45:18.465 INFO:teuthology.orchestra.run.smithi096.stdout:After this operation, 12.2 MB of additional disk space will be used. 2020-11-28T21:45:18.465 INFO:teuthology.orchestra.run.smithi096.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic/main amd64 flex amd64 2.6.4-6 [316 kB] 2020-11-28T21:45:18.485 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up ceph-mon (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:18.693 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-11-28T21:45:18.693 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-11-28T21:45:18.913 INFO:teuthology.orchestra.run.smithi096.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/main amd64 libbison-dev amd64 2:3.0.4.dfsg-1build1 [339 kB] 2020-11-28T21:45:18.970 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up ceph-mgr-k8sevents (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:18.996 INFO:teuthology.orchestra.run.smithi096.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 bison amd64 2:3.0.4.dfsg-1build1 [266 kB] 2020-11-28T21:45:19.049 INFO:teuthology.orchestra.run.smithi096.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/universe amd64 dump amd64 0.4b46-3 [129 kB] 2020-11-28T21:45:19.068 INFO:teuthology.orchestra.run.smithi096.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 indent amd64 2.2.11-5 [80.7 kB] 2020-11-28T21:45:19.078 INFO:teuthology.orchestra.run.smithi096.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libaio-dev amd64 0.3.110-5ubuntu0.1 [12.8 kB] 2020-11-28T21:45:19.079 INFO:teuthology.orchestra.run.smithi096.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic/main amd64 zlib1g-dev amd64 1:1.2.11.dfsg-0ubuntu2 [176 kB] 2020-11-28T21:45:19.093 INFO:teuthology.orchestra.run.smithi096.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libelf-dev amd64 0.170-0.4ubuntu0.1 [57.3 kB] 2020-11-28T21:45:19.098 INFO:teuthology.orchestra.run.smithi096.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl2 amd64 2.6.4-6 [11.4 kB] 2020-11-28T21:45:19.100 INFO:teuthology.orchestra.run.smithi096.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl-dev amd64 2.6.4-6 [6,320 B] 2020-11-28T21:45:19.101 INFO:teuthology.orchestra.run.smithi096.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libssl-dev amd64 1.1.1-1ubuntu2.1~18.04.6 [1,566 kB] 2020-11-28T21:45:19.104 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up ceph-mgr-dashboard (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:19.263 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up ceph (15.2.6-595-ga83a3572-1bionic) ... 2020-11-28T21:45:19.390 INFO:teuthology.orchestra.run.smithi192.stdout:Processing triggers for systemd (237-3ubuntu10.43) ... 2020-11-28T21:45:19.571 INFO:teuthology.orchestra.run.smithi096.stdout:Fetched 2,960 kB in 1s (3,285 kB/s) 2020-11-28T21:45:19.591 INFO:teuthology.orchestra.run.smithi192.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2020-11-28T21:45:19.710 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package flex. 2020-11-28T21:45:19.750 INFO:teuthology.orchestra.run.smithi096.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 109921 files and directories currently installed.) 2020-11-28T21:45:19.753 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../00-flex_2.6.4-6_amd64.deb ... 2020-11-28T21:45:19.792 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking flex (2.6.4-6) ... 2020-11-28T21:45:20.171 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libbison-dev:amd64. 2020-11-28T21:45:20.184 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../01-libbison-dev_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2020-11-28T21:45:20.221 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2020-11-28T21:45:20.574 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package bison. 2020-11-28T21:45:20.586 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../02-bison_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2020-11-28T21:45:20.628 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking bison (2:3.0.4.dfsg-1build1) ... 2020-11-28T21:45:21.027 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package dump. 2020-11-28T21:45:21.041 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../03-dump_0.4b46-3_amd64.deb ... 2020-11-28T21:45:21.086 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking dump (0.4b46-3) ... 2020-11-28T21:45:21.372 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package indent. 2020-11-28T21:45:21.385 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../04-indent_2.2.11-5_amd64.deb ... 2020-11-28T21:45:21.422 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking indent (2.2.11-5) ... 2020-11-28T21:45:21.683 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libaio-dev:amd64. 2020-11-28T21:45:21.694 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../05-libaio-dev_0.3.110-5ubuntu0.1_amd64.deb ... 2020-11-28T21:45:21.733 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2020-11-28T21:45:21.844 INFO:teuthology.orchestra.run.smithi192.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2020-11-28T21:45:21.879 INFO:teuthology.orchestra.run.smithi192.stdout:ureadahead will be reprofiled on next reboot 2020-11-28T21:45:21.928 INFO:teuthology.orchestra.run.smithi192.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2020-11-28T21:45:21.970 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package zlib1g-dev:amd64. 2020-11-28T21:45:21.983 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../06-zlib1g-dev_1%3a1.2.11.dfsg-0ubuntu2_amd64.deb ... 2020-11-28T21:45:22.011 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2020-11-28T21:45:22.306 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libelf-dev:amd64. 2020-11-28T21:45:22.319 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../07-libelf-dev_0.170-0.4ubuntu0.1_amd64.deb ... 2020-11-28T21:45:22.356 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2020-11-28T21:45:22.365 INFO:teuthology.orchestra.run.smithi192.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-11-28T21:45:22.367 INFO:teuthology.orchestra.run.smithi192:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2020-11-28T21:45:22.447 INFO:teuthology.orchestra.run.smithi192.stdout:Reading package lists... 2020-11-28T21:45:22.700 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libfl2:amd64. 2020-11-28T21:45:22.706 INFO:teuthology.orchestra.run.smithi192.stdout:Building dependency tree... 2020-11-28T21:45:22.706 INFO:teuthology.orchestra.run.smithi192.stdout:Reading state information... 2020-11-28T21:45:22.712 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../08-libfl2_2.6.4-6_amd64.deb ... 2020-11-28T21:45:22.801 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libfl2:amd64 (2.6.4-6) ... 2020-11-28T21:45:22.879 INFO:teuthology.orchestra.run.smithi192.stdout:libtool-bin is already the newest version (2.4.6-2). 2020-11-28T21:45:22.879 INFO:teuthology.orchestra.run.smithi192.stdout:xfslibs-dev is already the newest version (4.9.0+nmu1ubuntu2). 2020-11-28T21:45:22.879 INFO:teuthology.orchestra.run.smithi192.stdout:uuid-dev is already the newest version (2.31.1-0.4ubuntu3.7). 2020-11-28T21:45:22.879 INFO:teuthology.orchestra.run.smithi192.stdout:The following additional packages will be installed: 2020-11-28T21:45:22.880 INFO:teuthology.orchestra.run.smithi192.stdout: libbison-dev libfl-dev libfl2 zlib1g-dev 2020-11-28T21:45:22.881 INFO:teuthology.orchestra.run.smithi192.stdout:Suggested packages: 2020-11-28T21:45:22.881 INFO:teuthology.orchestra.run.smithi192.stdout: bison-doc flex-doc indent-doc libssl-doc 2020-11-28T21:45:22.907 INFO:teuthology.orchestra.run.smithi192.stdout:The following NEW packages will be installed: 2020-11-28T21:45:22.907 INFO:teuthology.orchestra.run.smithi192.stdout: bison dump flex indent libaio-dev libbison-dev libelf-dev libfl-dev libfl2 2020-11-28T21:45:22.908 INFO:teuthology.orchestra.run.smithi192.stdout: libssl-dev zlib1g-dev 2020-11-28T21:45:23.091 INFO:teuthology.orchestra.run.smithi192.stdout:0 upgraded, 11 newly installed, 0 to remove and 5 not upgraded. 2020-11-28T21:45:23.091 INFO:teuthology.orchestra.run.smithi192.stdout:Need to get 2,960 kB of archives. 2020-11-28T21:45:23.092 INFO:teuthology.orchestra.run.smithi192.stdout:After this operation, 12.2 MB of additional disk space will be used. 2020-11-28T21:45:23.092 INFO:teuthology.orchestra.run.smithi192.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic/main amd64 flex amd64 2.6.4-6 [316 kB] 2020-11-28T21:45:23.112 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libfl-dev:amd64. 2020-11-28T21:45:23.125 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../09-libfl-dev_2.6.4-6_amd64.deb ... 2020-11-28T21:45:23.154 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libfl-dev:amd64 (2.6.4-6) ... 2020-11-28T21:45:23.406 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libssl-dev:amd64. 2020-11-28T21:45:23.419 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../10-libssl-dev_1.1.1-1ubuntu2.1~18.04.6_amd64.deb ... 2020-11-28T21:45:23.457 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.6) ... 2020-11-28T21:45:23.535 INFO:teuthology.orchestra.run.smithi192.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/main amd64 libbison-dev amd64 2:3.0.4.dfsg-1build1 [339 kB] 2020-11-28T21:45:23.615 INFO:teuthology.orchestra.run.smithi192.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 bison amd64 2:3.0.4.dfsg-1build1 [266 kB] 2020-11-28T21:45:23.665 INFO:teuthology.orchestra.run.smithi192.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/universe amd64 dump amd64 0.4b46-3 [129 kB] 2020-11-28T21:45:23.684 INFO:teuthology.orchestra.run.smithi192.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 indent amd64 2.2.11-5 [80.7 kB] 2020-11-28T21:45:23.695 INFO:teuthology.orchestra.run.smithi192.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libaio-dev amd64 0.3.110-5ubuntu0.1 [12.8 kB] 2020-11-28T21:45:23.696 INFO:teuthology.orchestra.run.smithi192.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic/main amd64 zlib1g-dev amd64 1:1.2.11.dfsg-0ubuntu2 [176 kB] 2020-11-28T21:45:23.708 INFO:teuthology.orchestra.run.smithi192.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libelf-dev amd64 0.170-0.4ubuntu0.1 [57.3 kB] 2020-11-28T21:45:23.713 INFO:teuthology.orchestra.run.smithi192.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl2 amd64 2.6.4-6 [11.4 kB] 2020-11-28T21:45:23.714 INFO:teuthology.orchestra.run.smithi192.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl-dev amd64 2.6.4-6 [6,320 B] 2020-11-28T21:45:23.715 INFO:teuthology.orchestra.run.smithi192.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libssl-dev amd64 1.1.1-1ubuntu2.1~18.04.6 [1,566 kB] 2020-11-28T21:45:23.947 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up flex (2.6.4-6) ... 2020-11-28T21:45:24.070 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up dump (0.4b46-3) ... 2020-11-28T21:45:24.171 INFO:teuthology.orchestra.run.smithi096.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2020-11-28T21:45:24.195 INFO:teuthology.orchestra.run.smithi192.stdout:Fetched 2,960 kB in 1s (3,354 kB/s) 2020-11-28T21:45:24.229 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.6) ... 2020-11-28T21:45:24.355 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up indent (2.2.11-5) ... 2020-11-28T21:45:24.367 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package flex. 2020-11-28T21:45:24.409 INFO:teuthology.orchestra.run.smithi192.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 109921 files and directories currently installed.) 2020-11-28T21:45:24.412 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../00-flex_2.6.4-6_amd64.deb ... 2020-11-28T21:45:24.449 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking flex (2.6.4-6) ... 2020-11-28T21:45:24.456 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2020-11-28T21:45:24.574 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libfl2:amd64 (2.6.4-6) ... 2020-11-28T21:45:24.700 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up bison (2:3.0.4.dfsg-1build1) ... 2020-11-28T21:45:24.798 INFO:teuthology.orchestra.run.smithi096.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2020-11-28T21:45:24.836 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package libbison-dev:amd64. 2020-11-28T21:45:24.847 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../01-libbison-dev_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2020-11-28T21:45:24.868 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2020-11-28T21:45:24.886 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2020-11-28T21:45:24.995 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2020-11-28T21:45:25.121 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2020-11-28T21:45:25.214 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package bison. 2020-11-28T21:45:25.226 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../02-bison_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2020-11-28T21:45:25.246 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libfl-dev:amd64 (2.6.4-6) ... 2020-11-28T21:45:25.269 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking bison (2:3.0.4.dfsg-1build1) ... 2020-11-28T21:45:25.373 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2020-11-28T21:45:25.710 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package dump. 2020-11-28T21:45:25.722 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../03-dump_0.4b46-3_amd64.deb ... 2020-11-28T21:45:25.760 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking dump (0.4b46-3) ... 2020-11-28T21:45:26.072 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package indent. 2020-11-28T21:45:26.083 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../04-indent_2.2.11-5_amd64.deb ... 2020-11-28T21:45:26.122 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking indent (2.2.11-5) ... 2020-11-28T21:45:26.374 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package libaio-dev:amd64. 2020-11-28T21:45:26.388 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../05-libaio-dev_0.3.110-5ubuntu0.1_amd64.deb ... 2020-11-28T21:45:26.433 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2020-11-28T21:45:26.686 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package zlib1g-dev:amd64. 2020-11-28T21:45:26.698 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../06-zlib1g-dev_1%3a1.2.11.dfsg-0ubuntu2_amd64.deb ... 2020-11-28T21:45:26.736 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2020-11-28T21:45:27.047 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package libelf-dev:amd64. 2020-11-28T21:45:27.061 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../07-libelf-dev_0.170-0.4ubuntu0.1_amd64.deb ... 2020-11-28T21:45:27.105 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2020-11-28T21:45:27.183 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for install-info (6.5.0.dfsg.1-2) ... 2020-11-28T21:45:27.433 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package libfl2:amd64. 2020-11-28T21:45:27.446 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../08-libfl2_2.6.4-6_amd64.deb ... 2020-11-28T21:45:27.468 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2020-11-28T21:45:27.526 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking libfl2:amd64 (2.6.4-6) ... 2020-11-28T21:45:27.803 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package libfl-dev:amd64. 2020-11-28T21:45:27.816 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../09-libfl-dev_2.6.4-6_amd64.deb ... 2020-11-28T21:45:27.853 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking libfl-dev:amd64 (2.6.4-6) ... 2020-11-28T21:45:27.913 INFO:teuthology.orchestra.run.smithi096.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-11-28T21:45:27.915 DEBUG:teuthology.parallel:result is None 2020-11-28T21:45:28.105 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package libssl-dev:amd64. 2020-11-28T21:45:28.118 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../10-libssl-dev_1.1.1-1ubuntu2.1~18.04.6_amd64.deb ... 2020-11-28T21:45:28.155 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.6) ... 2020-11-28T21:45:28.600 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up flex (2.6.4-6) ... 2020-11-28T21:45:28.744 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up dump (0.4b46-3) ... 2020-11-28T21:45:28.845 INFO:teuthology.orchestra.run.smithi192.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2020-11-28T21:45:28.912 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.6) ... 2020-11-28T21:45:29.038 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up indent (2.2.11-5) ... 2020-11-28T21:45:29.164 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2020-11-28T21:45:29.290 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up libfl2:amd64 (2.6.4-6) ... 2020-11-28T21:45:29.408 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up bison (2:3.0.4.dfsg-1build1) ... 2020-11-28T21:45:29.481 INFO:teuthology.orchestra.run.smithi192.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2020-11-28T21:45:29.560 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2020-11-28T21:45:29.686 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2020-11-28T21:45:29.812 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2020-11-28T21:45:29.938 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up libfl-dev:amd64 (2.6.4-6) ... 2020-11-28T21:45:30.064 INFO:teuthology.orchestra.run.smithi192.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2020-11-28T21:45:31.784 INFO:teuthology.orchestra.run.smithi192.stdout:Processing triggers for install-info (6.5.0.dfsg.1-2) ... 2020-11-28T21:45:32.043 INFO:teuthology.orchestra.run.smithi192.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2020-11-28T21:45:32.526 INFO:teuthology.orchestra.run.smithi192.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-11-28T21:45:32.528 DEBUG:teuthology.parallel:result is None 2020-11-28T21:45:32.529 INFO:teuthology.orchestra.run.smithi162:> sudo lsb_release -is 2020-11-28T21:45:32.585 INFO:teuthology.orchestra.run.smithi162.stdout:Ubuntu 2020-11-28T21:45:32.590 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-28T21:45:32.591 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=a83a3572a957356977342f461361c1cd3ec06dbd 2020-11-28T21:45:32.767 INFO:teuthology.orchestra.run.smithi162:> dpkg-query -W -f '${Version}' ceph 2020-11-28T21:45:32.793 INFO:teuthology.orchestra.run.smithi162.stdout:15.2.6-595-ga83a3572-1bionic 2020-11-28T21:45:32.794 INFO:teuthology.packaging:The installed version of ceph is 15.2.6-595-ga83a3572-1bionic 2020-11-28T21:45:32.795 INFO:teuthology.task.install:The correct ceph version 15.2.6-595-ga83a3572-1bionic is installed. 2020-11-28T21:45:32.796 INFO:teuthology.orchestra.run.smithi118:> sudo lsb_release -is 2020-11-28T21:45:32.855 INFO:teuthology.orchestra.run.smithi118.stdout:Ubuntu 2020-11-28T21:45:32.860 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-28T21:45:32.861 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=a83a3572a957356977342f461361c1cd3ec06dbd 2020-11-28T21:45:33.038 INFO:teuthology.orchestra.run.smithi118:> dpkg-query -W -f '${Version}' ceph 2020-11-28T21:45:33.064 INFO:teuthology.orchestra.run.smithi118.stdout:15.2.6-595-ga83a3572-1bionic 2020-11-28T21:45:33.064 INFO:teuthology.packaging:The installed version of ceph is 15.2.6-595-ga83a3572-1bionic 2020-11-28T21:45:33.065 INFO:teuthology.task.install:The correct ceph version 15.2.6-595-ga83a3572-1bionic is installed. 2020-11-28T21:45:33.066 INFO:teuthology.orchestra.run.smithi192:> sudo lsb_release -is 2020-11-28T21:45:33.124 INFO:teuthology.orchestra.run.smithi192.stdout:Ubuntu 2020-11-28T21:45:33.130 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-28T21:45:33.130 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=a83a3572a957356977342f461361c1cd3ec06dbd 2020-11-28T21:45:33.289 INFO:teuthology.orchestra.run.smithi192:> dpkg-query -W -f '${Version}' ceph 2020-11-28T21:45:33.312 INFO:teuthology.orchestra.run.smithi192.stdout:15.2.6-595-ga83a3572-1bionic 2020-11-28T21:45:33.313 INFO:teuthology.packaging:The installed version of ceph is 15.2.6-595-ga83a3572-1bionic 2020-11-28T21:45:33.313 INFO:teuthology.task.install:The correct ceph version 15.2.6-595-ga83a3572-1bionic is installed. 2020-11-28T21:45:33.313 INFO:teuthology.orchestra.run.smithi096:> sudo lsb_release -is 2020-11-28T21:45:33.371 INFO:teuthology.orchestra.run.smithi096.stdout:Ubuntu 2020-11-28T21:45:33.376 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-28T21:45:33.377 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=a83a3572a957356977342f461361c1cd3ec06dbd 2020-11-28T21:45:33.537 INFO:teuthology.orchestra.run.smithi096:> dpkg-query -W -f '${Version}' ceph 2020-11-28T21:45:33.736 INFO:teuthology.orchestra.run.smithi096.stdout:15.2.6-595-ga83a3572-1bionic 2020-11-28T21:45:33.736 INFO:teuthology.packaging:The installed version of ceph is 15.2.6-595-ga83a3572-1bionic 2020-11-28T21:45:33.737 INFO:teuthology.task.install:The correct ceph version 15.2.6-595-ga83a3572-1bionic is installed. 2020-11-28T21:45:33.737 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2020-11-28T21:45:33.738 INFO:teuthology.orchestra.run.smithi162:> set -ex 2020-11-28T21:45:33.738 INFO:teuthology.orchestra.run.smithi162:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-28T21:45:33.743 INFO:teuthology.orchestra.run.smithi162.stderr:+ sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-28T21:45:33.753 INFO:teuthology.orchestra.run.smithi162.stderr:24+1 records in 2020-11-28T21:45:33.753 INFO:teuthology.orchestra.run.smithi162.stderr:24+1 records out 2020-11-28T21:45:33.753 INFO:teuthology.orchestra.run.smithi162.stderr:12667 bytes (13 kB, 12 KiB) copied, 0.000213041 s, 59.5 MB/s 2020-11-28T21:45:33.754 INFO:teuthology.orchestra.run.smithi118:> set -ex 2020-11-28T21:45:33.754 INFO:teuthology.orchestra.run.smithi118:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-28T21:45:33.764 INFO:teuthology.orchestra.run.smithi118.stderr:+ sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-28T21:45:33.771 INFO:teuthology.orchestra.run.smithi118.stderr:24+1 records in 2020-11-28T21:45:33.771 INFO:teuthology.orchestra.run.smithi118.stderr:24+1 records out 2020-11-28T21:45:33.771 INFO:teuthology.orchestra.run.smithi118.stderr:12667 bytes (13 kB, 12 KiB) copied, 0.0002164 s, 58.5 MB/s 2020-11-28T21:45:33.772 INFO:teuthology.orchestra.run.smithi192:> set -ex 2020-11-28T21:45:33.772 INFO:teuthology.orchestra.run.smithi192:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-28T21:45:33.777 INFO:teuthology.orchestra.run.smithi192.stderr:+ sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-28T21:45:33.787 INFO:teuthology.orchestra.run.smithi192.stderr:24+1 records in 2020-11-28T21:45:33.787 INFO:teuthology.orchestra.run.smithi192.stderr:24+1 records out 2020-11-28T21:45:33.787 INFO:teuthology.orchestra.run.smithi192.stderr:12667 bytes (13 kB, 12 KiB) copied, 0.000217393 s, 58.3 MB/s 2020-11-28T21:45:33.788 INFO:teuthology.orchestra.run.smithi096:> set -ex 2020-11-28T21:45:33.788 INFO:teuthology.orchestra.run.smithi096:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-28T21:45:33.793 INFO:teuthology.orchestra.run.smithi096.stderr:+ sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-28T21:45:33.803 INFO:teuthology.orchestra.run.smithi096.stderr:24+1 records in 2020-11-28T21:45:33.803 INFO:teuthology.orchestra.run.smithi096.stderr:24+1 records out 2020-11-28T21:45:33.803 INFO:teuthology.orchestra.run.smithi096.stderr:12667 bytes (13 kB, 12 KiB) copied, 0.000218054 s, 58.1 MB/s 2020-11-28T21:45:33.804 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2020-11-28T21:45:33.804 INFO:teuthology.orchestra.run.smithi162:> set -ex 2020-11-28T21:45:33.805 INFO:teuthology.orchestra.run.smithi162:> sudo dd of=/usr/bin/daemon-helper 2020-11-28T21:45:33.809 INFO:teuthology.orchestra.run.smithi162.stderr:+ sudo dd of=/usr/bin/daemon-helper 2020-11-28T21:45:33.817 INFO:teuthology.orchestra.run.smithi162.stderr:5+1 records in 2020-11-28T21:45:33.817 INFO:teuthology.orchestra.run.smithi162.stderr:5+1 records out 2020-11-28T21:45:33.817 INFO:teuthology.orchestra.run.smithi162.stderr:3058 bytes (3.1 kB, 3.0 KiB) copied, 9.966e-05 s, 30.7 MB/s 2020-11-28T21:45:33.818 INFO:teuthology.orchestra.run.smithi162:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-11-28T21:45:33.873 INFO:teuthology.orchestra.run.smithi118:> set -ex 2020-11-28T21:45:33.874 INFO:teuthology.orchestra.run.smithi118:> sudo dd of=/usr/bin/daemon-helper 2020-11-28T21:45:33.879 INFO:teuthology.orchestra.run.smithi118.stderr:+ sudo dd of=/usr/bin/daemon-helper 2020-11-28T21:45:33.887 INFO:teuthology.orchestra.run.smithi118.stderr:5+1 records in 2020-11-28T21:45:33.888 INFO:teuthology.orchestra.run.smithi118.stderr:5+1 records out 2020-11-28T21:45:33.888 INFO:teuthology.orchestra.run.smithi118.stderr:3058 bytes (3.1 kB, 3.0 KiB) copied, 9.7974e-05 s, 31.2 MB/s 2020-11-28T21:45:33.889 INFO:teuthology.orchestra.run.smithi118:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-11-28T21:45:33.947 INFO:teuthology.orchestra.run.smithi192:> set -ex 2020-11-28T21:45:33.948 INFO:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/daemon-helper 2020-11-28T21:45:33.953 INFO:teuthology.orchestra.run.smithi192.stderr:+ sudo dd of=/usr/bin/daemon-helper 2020-11-28T21:45:33.962 INFO:teuthology.orchestra.run.smithi192.stderr:5+1 records in 2020-11-28T21:45:33.963 INFO:teuthology.orchestra.run.smithi192.stderr:5+1 records out 2020-11-28T21:45:33.963 INFO:teuthology.orchestra.run.smithi192.stderr:3058 bytes (3.1 kB, 3.0 KiB) copied, 9.4264e-05 s, 32.4 MB/s 2020-11-28T21:45:33.964 INFO:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-11-28T21:45:34.021 INFO:teuthology.orchestra.run.smithi096:> set -ex 2020-11-28T21:45:34.021 INFO:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/daemon-helper 2020-11-28T21:45:34.027 INFO:teuthology.orchestra.run.smithi096.stderr:+ sudo dd of=/usr/bin/daemon-helper 2020-11-28T21:45:34.035 INFO:teuthology.orchestra.run.smithi096.stderr:5+1 records in 2020-11-28T21:45:34.035 INFO:teuthology.orchestra.run.smithi096.stderr:5+1 records out 2020-11-28T21:45:34.036 INFO:teuthology.orchestra.run.smithi096.stderr:3058 bytes (3.1 kB, 3.0 KiB) copied, 9.913e-05 s, 30.8 MB/s 2020-11-28T21:45:34.037 INFO:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-11-28T21:45:34.096 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2020-11-28T21:45:34.096 INFO:teuthology.orchestra.run.smithi162:> set -ex 2020-11-28T21:45:34.097 INFO:teuthology.orchestra.run.smithi162:> sudo dd of=/usr/bin/adjust-ulimits 2020-11-28T21:45:34.101 INFO:teuthology.orchestra.run.smithi162.stderr:+ sudo dd of=/usr/bin/adjust-ulimits 2020-11-28T21:45:34.110 INFO:teuthology.orchestra.run.smithi162.stderr:0+1 records in 2020-11-28T21:45:34.111 INFO:teuthology.orchestra.run.smithi162.stderr:0+1 records out 2020-11-28T21:45:34.111 INFO:teuthology.orchestra.run.smithi162.stderr:299 bytes copied, 7.4577e-05 s, 4.0 MB/s 2020-11-28T21:45:34.112 INFO:teuthology.orchestra.run.smithi162:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-11-28T21:45:34.169 INFO:teuthology.orchestra.run.smithi118:> set -ex 2020-11-28T21:45:34.169 INFO:teuthology.orchestra.run.smithi118:> sudo dd of=/usr/bin/adjust-ulimits 2020-11-28T21:45:34.174 INFO:teuthology.orchestra.run.smithi118.stderr:+ sudo dd of=/usr/bin/adjust-ulimits 2020-11-28T21:45:34.183 INFO:teuthology.orchestra.run.smithi118.stderr:0+1 records in 2020-11-28T21:45:34.183 INFO:teuthology.orchestra.run.smithi118.stderr:0+1 records out 2020-11-28T21:45:34.183 INFO:teuthology.orchestra.run.smithi118.stderr:299 bytes copied, 7.2734e-05 s, 4.1 MB/s 2020-11-28T21:45:34.184 INFO:teuthology.orchestra.run.smithi118:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-11-28T21:45:34.240 INFO:teuthology.orchestra.run.smithi192:> set -ex 2020-11-28T21:45:34.240 INFO:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/adjust-ulimits 2020-11-28T21:45:34.245 INFO:teuthology.orchestra.run.smithi192.stderr:+ sudo dd of=/usr/bin/adjust-ulimits 2020-11-28T21:45:34.253 INFO:teuthology.orchestra.run.smithi192.stderr:0+1 records in 2020-11-28T21:45:34.254 INFO:teuthology.orchestra.run.smithi192.stderr:0+1 records out 2020-11-28T21:45:34.254 INFO:teuthology.orchestra.run.smithi192.stderr:299 bytes copied, 6.84e-05 s, 4.4 MB/s 2020-11-28T21:45:34.255 INFO:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-11-28T21:45:34.312 INFO:teuthology.orchestra.run.smithi096:> set -ex 2020-11-28T21:45:34.313 INFO:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/adjust-ulimits 2020-11-28T21:45:34.318 INFO:teuthology.orchestra.run.smithi096.stderr:+ sudo dd of=/usr/bin/adjust-ulimits 2020-11-28T21:45:34.327 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records in 2020-11-28T21:45:34.327 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records out 2020-11-28T21:45:34.327 INFO:teuthology.orchestra.run.smithi096.stderr:299 bytes copied, 7.0513e-05 s, 4.2 MB/s 2020-11-28T21:45:34.328 INFO:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-11-28T21:45:34.387 INFO:teuthology.run_tasks:Running task ceph... 2020-11-28T21:45:34.440 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2020-11-28T21:45:34.441 INFO:teuthology.orchestra.run.smithi096:> sudo chmod 777 /var/log/ceph 2020-11-28T21:45:34.446 INFO:teuthology.orchestra.run.smithi118:> sudo chmod 777 /var/log/ceph 2020-11-28T21:45:34.448 INFO:teuthology.orchestra.run.smithi162:> sudo chmod 777 /var/log/ceph 2020-11-28T21:45:34.451 INFO:teuthology.orchestra.run.smithi192:> sudo chmod 777 /var/log/ceph 2020-11-28T21:45:34.465 INFO:tasks.ceph:Disabling ceph logrotate... 2020-11-28T21:45:34.466 INFO:teuthology.orchestra.run.smithi096:> sudo rm -f -- /etc/logrotate.d/ceph 2020-11-28T21:45:34.499 INFO:teuthology.orchestra.run.smithi118:> sudo rm -f -- /etc/logrotate.d/ceph 2020-11-28T21:45:34.503 INFO:teuthology.orchestra.run.smithi162:> sudo rm -f -- /etc/logrotate.d/ceph 2020-11-28T21:45:34.505 INFO:teuthology.orchestra.run.smithi192:> sudo rm -f -- /etc/logrotate.d/ceph 2020-11-28T21:45:34.520 INFO:tasks.ceph:Creating extra log directories... 2020-11-28T21:45:34.520 INFO:teuthology.orchestra.run.smithi096:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-11-28T21:45:34.560 INFO:teuthology.orchestra.run.smithi118:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-11-28T21:45:34.562 INFO:teuthology.orchestra.run.smithi162:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-11-28T21:45:34.565 INFO:teuthology.orchestra.run.smithi192:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-11-28T21:45:34.581 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2020-11-28T21:45:34.581 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2020-11-28T21:45:34.582 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2020-11-28T21:45:34.582 INFO:teuthology.orchestra.run.smithi162:> set -ex 2020-11-28T21:45:34.583 INFO:teuthology.orchestra.run.smithi162:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-11-28T21:45:34.623 INFO:teuthology.orchestra.run.smithi162.stderr:+ dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-11-28T21:45:34.625 INFO:teuthology.orchestra.run.smithi162.stderr:0+1 records in 2020-11-28T21:45:34.625 INFO:teuthology.orchestra.run.smithi162.stderr:0+1 records out 2020-11-28T21:45:34.625 INFO:teuthology.orchestra.run.smithi162.stderr:408 bytes copied, 7.5204e-05 s, 5.4 MB/s 2020-11-28T21:45:34.625 INFO:teuthology.orchestra.run.smithi162:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2020-11-28T21:45:34.702 INFO:teuthology.orchestra.run.smithi118:> set -ex 2020-11-28T21:45:34.703 INFO:teuthology.orchestra.run.smithi118:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-11-28T21:45:34.708 INFO:teuthology.orchestra.run.smithi118.stderr:+ dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-11-28T21:45:34.709 INFO:teuthology.orchestra.run.smithi118.stderr:0+1 records in 2020-11-28T21:45:34.710 INFO:teuthology.orchestra.run.smithi118.stderr:0+1 records out 2020-11-28T21:45:34.710 INFO:teuthology.orchestra.run.smithi118.stderr:408 bytes copied, 7.256e-05 s, 5.6 MB/s 2020-11-28T21:45:34.710 INFO:teuthology.orchestra.run.smithi118:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2020-11-28T21:45:34.788 INFO:teuthology.orchestra.run.smithi192:> set -ex 2020-11-28T21:45:34.788 INFO:teuthology.orchestra.run.smithi192:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-11-28T21:45:34.793 INFO:teuthology.orchestra.run.smithi192.stderr:+ dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-11-28T21:45:34.795 INFO:teuthology.orchestra.run.smithi192.stderr:0+1 records in 2020-11-28T21:45:34.795 INFO:teuthology.orchestra.run.smithi192.stderr:0+1 records out 2020-11-28T21:45:34.795 INFO:teuthology.orchestra.run.smithi192.stderr:408 bytes copied, 7.691e-05 s, 5.3 MB/s 2020-11-28T21:45:34.795 INFO:teuthology.orchestra.run.smithi192:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2020-11-28T21:45:34.874 INFO:teuthology.orchestra.run.smithi096:> set -ex 2020-11-28T21:45:34.875 INFO:teuthology.orchestra.run.smithi096:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-11-28T21:45:34.880 INFO:teuthology.orchestra.run.smithi096.stderr:+ dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-11-28T21:45:34.881 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records in 2020-11-28T21:45:34.882 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records out 2020-11-28T21:45:34.882 INFO:teuthology.orchestra.run.smithi096.stderr:408 bytes copied, 6.673e-05 s, 6.1 MB/s 2020-11-28T21:45:34.882 INFO:teuthology.orchestra.run.smithi096:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2020-11-28T21:45:34.961 INFO:tasks.ceph:Creating ceph cluster ceph... 2020-11-28T21:45:34.961 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'evicting unresponsive client', 'POOL_APP_NOT_ENABLED', 'has not responded to cap revoke by MDS for over', 'MDS_CLIENT_LATE_RELEASE', 'responding to mclientcaps'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2020-11-28T21:45:34.962 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2020-11-25_04:10:04-fs-octopus-distro-basic-smithi/5655905', 'branch': 'octopus', 'description': 'fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests}', 'email': 'ceph-qa@ceph.io', 'first_in_suite': False, 'job_id': '5655905', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'teuthology-2020-11-25_04:10:04-fs-octopus-distro-basic-smithi', 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 30}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '18.04', 'overrides': {'admin_socket': {'branch': 'octopus'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'evicting unresponsive client', 'POOL_APP_NOT_ENABLED', 'has not responded to cap revoke by MDS for over', 'MDS_CLIENT_LATE_RELEASE', 'responding to mclientcaps'], 'sha1': 'a83a3572a957356977342f461361c1cd3ec06dbd'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'sha1': 'a83a3572a957356977342f461361c1cd3ec06dbd'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'octopus', 'sha1': 'a83a3572a957356977342f461361c1cd3ec06dbd'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 1000, 'repo': 'git://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'], ['client.0'], ['client.1']], 'sha1': 'a83a3572a957356977342f461361c1cd3ec06dbd', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'octopus', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_octopus/qa', 'suite_relpath': 'qa', 'suite_repo': 'git://git.ceph.com/ceph.git', 'suite_sha1': 'a83a3572a957356977342f461361c1cd3ec06dbd', 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.lock_machines': (4, 'smithi')}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'sha1': 'a83a3572a957356977342f461361c1cd3ec06dbd'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_misc']}}], 'teuthology_branch': 'master', 'timestamp': '2020-11-25_04:10:04', 'tube': 'smithi', 'user': 'teuthology', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/worker.smithi.25733', 'targets': {'smithi162.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDR4skALubozLqiyWNdPiV3nXDJ59kK6uDCVCkvXxL23MBmGaGOBFcR13j69Vitu69OMVzysGee1ys5ovVzIHqiCzw/mKgxlYTlZ1r7wUXS8zxT6sn4towy+HnVWwUDi2VRguyHMTA2Tt0cIUa5xpiB1u5QRAIDxKF+sC+l/Rranvk2rMgIbnBUSldMtcfxzc/hGV7BRieV+Av2i+mf9hwVd8L6XMkwW+cwftwQaxGsk+0aspR/ANoNGW9IIP48uykU+mpv8seqzzloRNhSw0Wdh19BzFMkOPt+nfkOT7OxEV8G3ONu/bE0MxkBzhjwK9HBv2dhLXvuYdHl5dnivWsJ', 'smithi118.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDqeTK6CjSUKc3bD1q56WU85Yjz05zVszqohM/uxV1dA2+yWJwMqoSDcCtSr98rR1Q5a9LbPykWBlZ08UDRlPKjtYtM1wP+axpmXPRL/Mp2DaL7d98ZxPrVuCO2TwqwjLogQ1vr72jL8ze7mTdjei+I/QgFl9thSTa68NoCYflPkIAbhUhP+z7LNUSB32PHJibPtRmYSNQXkAC1aDFWfnb9Zzz/xHft2R9D7P9xXD0Oum+L2H+cFmCBA0e4qhP+HtIRFBOZ+fCaE5M/RUpRVnODQKCuyXxwkRYOIeQTUn4a4VGbcCq+rWnJelzTVG87HruHauGJRMxzJ6PuO6zz2osb', 'smithi192.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDG63qqli+Xx28jOQ/UofpMSrbdhqWqhJXbtgTZPAHScGFWeLmG2DjPrJlXvaJvE7cu1AXUj5VXo+zCpvxD1TgvykDEg/9CU+nXDLej/KX0ccpu0RJUHCBu9/c6X7VuCLoHMScuZ9d6v/sWLtRFHkCvKESo+dWbgQYh+Wa7ESay7X19VdoyJWJCiHraxZt4+uro5Ivq4pON1syctfHztqYEw/5d/TkgIqUkrmtauZHpChUJ0pcit03GAWZv3zKCKJDR4aw99hJ7nCiAegaxVJztjOxXPwvXXOMKGJbNWx0l/StPOQs83JkUXuOscggDlXWlau2XlrYcicpl2X1LrXod', 'smithi096.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDvbnZHmcs9Xr2dBzBkSrjn+U8zaCQ70Sp063V8Nq8wcY427+jzxcoOsW0cdDx+EvMYLu1rcO1zzQeNeX2zVCOI+8pCelRU0qINQigRedJsOc+4QxKDEMqyaa+mJJfO7L3sXCgsGxp9Vel3mpgq0UwEkyEOj0NENyVh5TbjVIBUJ5GNo94VW4OvwdaOtyZ7LoDUY5SIAhGp+IWzFUwb3ltgGtJnyJZ9sWtW6i8CWanTKaCWsYlVoUMtPTIiqXVI+90NkGBPRXtWmiMOyE4VDQqSQoop7UgcsUG7IFUzpTyOKZaFqFcgu0H2AMvrxcHnN8i13nN+nQ/UU90zSUDTEd1'}} 2020-11-28T21:45:34.963 INFO:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2020-11-28T21:45:35.007 INFO:teuthology.orchestra.run.smithi118:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2020-11-28T21:45:35.010 INFO:teuthology.orchestra.run.smithi162:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2020-11-28T21:45:35.012 INFO:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2020-11-28T21:45:35.018 INFO:teuthology.orchestra.run.smithi096:> sudo install -d -m0777 -- /var/run/ceph 2020-11-28T21:45:35.056 INFO:teuthology.orchestra.run.smithi118:> sudo install -d -m0777 -- /var/run/ceph 2020-11-28T21:45:35.064 INFO:teuthology.orchestra.run.smithi162:> sudo install -d -m0777 -- /var/run/ceph 2020-11-28T21:45:35.066 INFO:teuthology.orchestra.run.smithi192:> sudo install -d -m0777 -- /var/run/ceph 2020-11-28T21:45:35.081 INFO:teuthology.orchestra.run.smithi162:> set -ex 2020-11-28T21:45:35.082 INFO:teuthology.orchestra.run.smithi162:> dd if=/scratch_devs of=/dev/stdout 2020-11-28T21:45:35.127 INFO:teuthology.orchestra.run.smithi162.stderr:+ dd if=/scratch_devs of=/dev/stdout 2020-11-28T21:45:35.129 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/vg_nvme/lv_1 2020-11-28T21:45:35.129 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/vg_nvme/lv_2 2020-11-28T21:45:35.129 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/vg_nvme/lv_3 2020-11-28T21:45:35.129 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/vg_nvme/lv_4 2020-11-28T21:45:35.129 INFO:teuthology.orchestra.run.smithi162.stderr:0+1 records in 2020-11-28T21:45:35.130 INFO:teuthology.orchestra.run.smithi162.stderr:0+1 records out 2020-11-28T21:45:35.130 INFO:teuthology.orchestra.run.smithi162.stderr:72 bytes copied, 0.000176853 s, 407 kB/s 2020-11-28T21:45:35.130 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2020-11-28T21:45:35.131 INFO:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2020-11-28T21:45:35.177 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2020-11-28T21:45:35.178 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-28T21:45:35.178 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 6h/6d Inode: 344 Links: 1 2020-11-28T21:45:35.178 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-28T21:45:35.179 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2020-11-28 21:42:28.965348795 +0000 2020-11-28T21:45:35.179 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2020-11-28 21:42:28.961350796 +0000 2020-11-28T21:45:35.179 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2020-11-28 21:42:28.961350796 +0000 2020-11-28T21:45:35.179 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: - 2020-11-28T21:45:35.187 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2020-11-28T21:45:35.187 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2020-11-28T21:45:35.188 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000410889 s, 1.2 MB/s 2020-11-28T21:45:35.191 INFO:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2020-11-28T21:45:35.241 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2020-11-28T21:45:35.242 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-28T21:45:35.242 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 6h/6d Inode: 351 Links: 1 2020-11-28T21:45:35.242 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-28T21:45:35.242 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2020-11-28 21:42:29.281190789 +0000 2020-11-28T21:45:35.243 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2020-11-28 21:42:29.189236790 +0000 2020-11-28T21:45:35.243 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2020-11-28 21:42:29.189236790 +0000 2020-11-28T21:45:35.243 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: - 2020-11-28T21:45:35.251 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2020-11-28T21:45:35.251 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2020-11-28T21:45:35.252 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000356574 s, 1.4 MB/s 2020-11-28T21:45:35.256 INFO:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2020-11-28T21:45:35.305 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2020-11-28T21:45:35.306 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-28T21:45:35.306 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 6h/6d Inode: 357 Links: 1 2020-11-28T21:45:35.306 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-28T21:45:35.306 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2020-11-28 21:42:29.393134785 +0000 2020-11-28T21:45:35.307 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2020-11-28 21:42:29.389136786 +0000 2020-11-28T21:45:35.307 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2020-11-28 21:42:29.389136786 +0000 2020-11-28T21:45:35.307 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: - 2020-11-28T21:45:35.315 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2020-11-28T21:45:35.315 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2020-11-28T21:45:35.316 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000352358 s, 1.5 MB/s 2020-11-28T21:45:35.320 INFO:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2020-11-28T21:45:35.369 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2020-11-28T21:45:35.370 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-28T21:45:35.370 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 6h/6d Inode: 363 Links: 1 2020-11-28T21:45:35.370 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-28T21:45:35.370 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2020-11-28 21:42:29.676992780 +0000 2020-11-28T21:45:35.371 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2020-11-28 21:42:29.581040782 +0000 2020-11-28T21:45:35.371 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2020-11-28 21:42:29.581040782 +0000 2020-11-28T21:45:35.371 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: - 2020-11-28T21:45:35.379 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2020-11-28T21:45:35.379 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2020-11-28T21:45:35.379 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000401527 s, 1.3 MB/s 2020-11-28T21:45:35.384 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-11-28T21:45:35.384 INFO:teuthology.orchestra.run.smithi118:> set -ex 2020-11-28T21:45:35.384 INFO:teuthology.orchestra.run.smithi118:> dd if=/scratch_devs of=/dev/stdout 2020-11-28T21:45:35.389 INFO:teuthology.orchestra.run.smithi118.stderr:+ dd if=/scratch_devs of=/dev/stdout 2020-11-28T21:45:35.390 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/vg_nvme/lv_1 2020-11-28T21:45:35.391 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/vg_nvme/lv_2 2020-11-28T21:45:35.391 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/vg_nvme/lv_3 2020-11-28T21:45:35.391 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/vg_nvme/lv_4 2020-11-28T21:45:35.391 INFO:teuthology.orchestra.run.smithi118.stderr:0+1 records in 2020-11-28T21:45:35.392 INFO:teuthology.orchestra.run.smithi118.stderr:0+1 records out 2020-11-28T21:45:35.392 INFO:teuthology.orchestra.run.smithi118.stderr:72 bytes copied, 0.000152578 s, 472 kB/s 2020-11-28T21:45:35.392 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2020-11-28T21:45:35.392 INFO:teuthology.orchestra.run.smithi118:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2020-11-28T21:45:35.440 INFO:teuthology.orchestra.run.smithi118.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2020-11-28T21:45:35.440 INFO:teuthology.orchestra.run.smithi118.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-28T21:45:35.440 INFO:teuthology.orchestra.run.smithi118.stdout:Device: 6h/6d Inode: 341 Links: 1 2020-11-28T21:45:35.441 INFO:teuthology.orchestra.run.smithi118.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-28T21:45:35.441 INFO:teuthology.orchestra.run.smithi118.stdout:Access: 2020-11-28 21:42:24.672413667 +0000 2020-11-28T21:45:35.441 INFO:teuthology.orchestra.run.smithi118.stdout:Modify: 2020-11-28 21:42:24.560469671 +0000 2020-11-28T21:45:35.441 INFO:teuthology.orchestra.run.smithi118.stdout:Change: 2020-11-28 21:42:24.560469671 +0000 2020-11-28T21:45:35.441 INFO:teuthology.orchestra.run.smithi118.stdout: Birth: - 2020-11-28T21:45:35.450 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records in 2020-11-28T21:45:35.451 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records out 2020-11-28T21:45:35.451 INFO:teuthology.orchestra.run.smithi118.stderr:512 bytes copied, 0.000291517 s, 1.8 MB/s 2020-11-28T21:45:35.454 INFO:teuthology.orchestra.run.smithi118:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2020-11-28T21:45:35.504 INFO:teuthology.orchestra.run.smithi118.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2020-11-28T21:45:35.504 INFO:teuthology.orchestra.run.smithi118.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-28T21:45:35.504 INFO:teuthology.orchestra.run.smithi118.stdout:Device: 6h/6d Inode: 350 Links: 1 2020-11-28T21:45:35.505 INFO:teuthology.orchestra.run.smithi118.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-28T21:45:35.505 INFO:teuthology.orchestra.run.smithi118.stdout:Access: 2020-11-28 21:42:24.892303662 +0000 2020-11-28T21:45:35.505 INFO:teuthology.orchestra.run.smithi118.stdout:Modify: 2020-11-28 21:42:24.784357664 +0000 2020-11-28T21:45:35.505 INFO:teuthology.orchestra.run.smithi118.stdout:Change: 2020-11-28 21:42:24.784357664 +0000 2020-11-28T21:45:35.505 INFO:teuthology.orchestra.run.smithi118.stdout: Birth: - 2020-11-28T21:45:35.513 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records in 2020-11-28T21:45:35.514 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records out 2020-11-28T21:45:35.514 INFO:teuthology.orchestra.run.smithi118.stderr:512 bytes copied, 0.000288107 s, 1.8 MB/s 2020-11-28T21:45:35.518 INFO:teuthology.orchestra.run.smithi118:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2020-11-28T21:45:35.568 INFO:teuthology.orchestra.run.smithi118.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2020-11-28T21:45:35.568 INFO:teuthology.orchestra.run.smithi118.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-28T21:45:35.569 INFO:teuthology.orchestra.run.smithi118.stdout:Device: 6h/6d Inode: 356 Links: 1 2020-11-28T21:45:35.569 INFO:teuthology.orchestra.run.smithi118.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-28T21:45:35.569 INFO:teuthology.orchestra.run.smithi118.stdout:Access: 2020-11-28 21:42:25.100199656 +0000 2020-11-28T21:45:35.569 INFO:teuthology.orchestra.run.smithi118.stdout:Modify: 2020-11-28 21:42:25.000249659 +0000 2020-11-28T21:45:35.569 INFO:teuthology.orchestra.run.smithi118.stdout:Change: 2020-11-28 21:42:25.000249659 +0000 2020-11-28T21:45:35.570 INFO:teuthology.orchestra.run.smithi118.stdout: Birth: - 2020-11-28T21:45:35.577 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records in 2020-11-28T21:45:35.578 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records out 2020-11-28T21:45:35.578 INFO:teuthology.orchestra.run.smithi118.stderr:512 bytes copied, 0.000265322 s, 1.9 MB/s 2020-11-28T21:45:35.582 INFO:teuthology.orchestra.run.smithi118:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2020-11-28T21:45:35.636 INFO:teuthology.orchestra.run.smithi118.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2020-11-28T21:45:35.636 INFO:teuthology.orchestra.run.smithi118.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-28T21:45:35.636 INFO:teuthology.orchestra.run.smithi118.stdout:Device: 6h/6d Inode: 362 Links: 1 2020-11-28T21:45:35.637 INFO:teuthology.orchestra.run.smithi118.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-28T21:45:35.637 INFO:teuthology.orchestra.run.smithi118.stdout:Access: 2020-11-28 21:42:25.316091649 +0000 2020-11-28T21:45:35.637 INFO:teuthology.orchestra.run.smithi118.stdout:Modify: 2020-11-28 21:42:25.220139652 +0000 2020-11-28T21:45:35.637 INFO:teuthology.orchestra.run.smithi118.stdout:Change: 2020-11-28 21:42:25.220139652 +0000 2020-11-28T21:45:35.638 INFO:teuthology.orchestra.run.smithi118.stdout: Birth: - 2020-11-28T21:45:35.645 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records in 2020-11-28T21:45:35.645 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records out 2020-11-28T21:45:35.645 INFO:teuthology.orchestra.run.smithi118.stderr:512 bytes copied, 0.000278681 s, 1.8 MB/s 2020-11-28T21:45:35.648 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-11-28T21:45:35.648 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi162.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi118.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2020-11-28T21:45:35.648 INFO:tasks.ceph:Generating config... 2020-11-28T21:45:35.650 INFO:tasks.ceph:[client] client mount timeout = 600 2020-11-28T21:45:35.650 INFO:tasks.ceph:[client] debug client = 20 2020-11-28T21:45:35.650 INFO:tasks.ceph:[client] debug ms = 1 2020-11-28T21:45:35.651 INFO:tasks.ceph:[client] rados mon op timeout = 15m 2020-11-28T21:45:35.651 INFO:tasks.ceph:[client] rados osd op timeout = 15m 2020-11-28T21:45:35.651 INFO:tasks.ceph:[mds] debug mds = 20 2020-11-28T21:45:35.651 INFO:tasks.ceph:[mds] debug ms = 1 2020-11-28T21:45:35.652 INFO:tasks.ceph:[mds] mds bal frag = True 2020-11-28T21:45:35.652 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2020-11-28T21:45:35.652 INFO:tasks.ceph:[mds] mds bal merge size = 5 2020-11-28T21:45:35.652 INFO:tasks.ceph:[mds] mds bal split bits = 3 2020-11-28T21:45:35.653 INFO:tasks.ceph:[mds] mds bal split size = 100 2020-11-28T21:45:35.653 INFO:tasks.ceph:[mds] mds debug frag = True 2020-11-28T21:45:35.653 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2020-11-28T21:45:35.654 INFO:tasks.ceph:[mds] mds op complaint time = 180 2020-11-28T21:45:35.654 INFO:tasks.ceph:[mds] mds verify scatter = True 2020-11-28T21:45:35.654 INFO:tasks.ceph:[mds] osd op complaint time = 180 2020-11-28T21:45:35.654 INFO:tasks.ceph:[mds] rados mon op timeout = 15m 2020-11-28T21:45:35.655 INFO:tasks.ceph:[mds] rados osd op timeout = 15m 2020-11-28T21:45:35.655 INFO:tasks.ceph:[mgr] debug mgr = 20 2020-11-28T21:45:35.655 INFO:tasks.ceph:[mgr] debug ms = 1 2020-11-28T21:45:35.656 INFO:tasks.ceph:[mon] debug mon = 20 2020-11-28T21:45:35.656 INFO:tasks.ceph:[mon] debug ms = 1 2020-11-28T21:45:35.656 INFO:tasks.ceph:[mon] debug paxos = 20 2020-11-28T21:45:35.656 INFO:tasks.ceph:[mon] mon op complaint time = 120 2020-11-28T21:45:35.657 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2020-11-28T21:45:35.657 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2020-11-28T21:45:35.657 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2020-11-28T21:45:35.658 INFO:tasks.ceph:[osd] debug bluefs = 20 2020-11-28T21:45:35.658 INFO:tasks.ceph:[osd] debug bluestore = 20 2020-11-28T21:45:35.658 INFO:tasks.ceph:[osd] debug filestore = 20 2020-11-28T21:45:35.658 INFO:tasks.ceph:[osd] debug journal = 20 2020-11-28T21:45:35.659 INFO:tasks.ceph:[osd] debug ms = 20 2020-11-28T21:45:35.659 INFO:tasks.ceph:[osd] debug osd = 25 2020-11-28T21:45:35.659 INFO:tasks.ceph:[osd] debug rocksdb = 10 2020-11-28T21:45:35.660 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2020-11-28T21:45:35.660 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2020-11-28T21:45:35.660 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2020-11-28T21:45:35.660 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2020-11-28T21:45:35.661 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2020-11-28T21:45:35.661 INFO:tasks.ceph:[osd] osd op complaint time = 180 2020-11-28T21:45:35.661 INFO:tasks.ceph:Setting up mon.a... 2020-11-28T21:45:35.662 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2020-11-28T21:45:35.694 INFO:teuthology.orchestra.run.smithi162.stdout:creating /etc/ceph/ceph.keyring 2020-11-28T21:45:35.696 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2020-11-28T21:45:35.773 INFO:teuthology.orchestra.run.smithi162:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-28T21:45:35.828 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.162'), ('mon.b', '172.21.15.118'), ('mon.c', '[v2:172.21.15.118:3301,v1:172.21.15.118:6790]')] 2020-11-28T21:45:35.828 INFO:teuthology.orchestra.run.smithi162:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool --create --clobber --enable-all-features --add a 172.21.15.162 --add b 172.21.15.118 --addv c '[v2:172.21.15.118:3301,v1:172.21.15.118:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2020-11-28T21:45:35.907 INFO:teuthology.orchestra.run.smithi162.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2020-11-28T21:45:35.908 INFO:teuthology.orchestra.run.smithi162.stdout:monmaptool: generated fsid 7c321bcf-cd33-40f7-a39f-da5470b386cf 2020-11-28T21:45:35.908 INFO:teuthology.orchestra.run.smithi162.stdout:epoch 0 2020-11-28T21:45:35.908 INFO:teuthology.orchestra.run.smithi162.stdout:fsid 7c321bcf-cd33-40f7-a39f-da5470b386cf 2020-11-28T21:45:35.908 INFO:teuthology.orchestra.run.smithi162.stdout:last_changed 2020-11-28T21:45:35.909626+0000 2020-11-28T21:45:35.909 INFO:teuthology.orchestra.run.smithi162.stdout:created 2020-11-28T21:45:35.909626+0000 2020-11-28T21:45:35.909 INFO:teuthology.orchestra.run.smithi162.stdout:min_mon_release 0 (unknown) 2020-11-28T21:45:35.909 INFO:teuthology.orchestra.run.smithi162.stdout:0: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.a 2020-11-28T21:45:35.909 INFO:teuthology.orchestra.run.smithi162.stdout:1: [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] mon.b 2020-11-28T21:45:35.910 INFO:teuthology.orchestra.run.smithi162.stdout:2: [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] mon.c 2020-11-28T21:45:35.910 INFO:teuthology.orchestra.run.smithi162.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2020-11-28T21:45:35.911 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 7c321bcf-cd33-40f7-a39f-da5470b386cf... 2020-11-28T21:45:35.912 INFO:teuthology.orchestra.run.smithi096:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-28T21:45:35.915 INFO:teuthology.orchestra.run.smithi118:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-28T21:45:35.917 INFO:teuthology.orchestra.run.smithi162:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-28T21:45:35.953 INFO:teuthology.orchestra.run.smithi192:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-28T21:45:35.957 INFO:teuthology.orchestra.run.smithi118.stdout:[global] 2020-11-28T21:45:35.957 INFO:teuthology.orchestra.run.smithi118.stdout: chdir = "" 2020-11-28T21:45:35.957 INFO:teuthology.orchestra.run.smithi118.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-28T21:45:35.958 INFO:teuthology.orchestra.run.smithi118.stdout: auth supported = cephx 2020-11-28T21:45:35.958 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:45:35.958 INFO:teuthology.orchestra.run.smithi118.stdout: filestore xattr use omap = true 2020-11-28T21:45:35.958 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:45:35.958 INFO:teuthology.orchestra.run.smithi118.stdout: mon clock drift allowed = 1.000 2020-11-28T21:45:35.959 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:45:35.959 INFO:teuthology.orchestra.run.smithi118.stdout: osd crush chooseleaf type = 0 2020-11-28T21:45:35.959 INFO:teuthology.orchestra.run.smithi118.stdout: auth debug = true 2020-11-28T21:45:35.959 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:45:35.959 INFO:teuthology.orchestra.run.smithi118.stdout: ms die on old message = true 2020-11-28T21:45:35.960 INFO:teuthology.orchestra.run.smithi118.stdout: ms die on bug = true 2020-11-28T21:45:35.960 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:45:35.960 INFO:teuthology.orchestra.run.smithi118.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-28T21:45:35.960 INFO:teuthology.orchestra.run.smithi118.stdout: mon pg warn max object skew = 0 2020-11-28T21:45:35.960 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:45:35.961 INFO:teuthology.orchestra.run.smithi118.stdout: # disable pg_autoscaler by default for new pools 2020-11-28T21:45:35.961 INFO:teuthology.orchestra.run.smithi118.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-28T21:45:35.961 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:45:35.961 INFO:teuthology.orchestra.run.smithi118.stdout: osd pool default size = 2 2020-11-28T21:45:35.962 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:45:35.962 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd allow primary affinity = true 2020-11-28T21:45:35.962 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd allow pg remap = true 2020-11-28T21:45:35.962 INFO:teuthology.orchestra.run.smithi118.stdout: mon warn on legacy crush tunables = false 2020-11-28T21:45:35.962 INFO:teuthology.orchestra.run.smithi118.stdout: mon warn on crush straw calc version zero = false 2020-11-28T21:45:35.963 INFO:teuthology.orchestra.run.smithi118.stdout: mon warn on no sortbitwise = false 2020-11-28T21:45:35.963 INFO:teuthology.orchestra.run.smithi118.stdout: mon warn on osd down out interval zero = false 2020-11-28T21:45:35.963 INFO:teuthology.orchestra.run.smithi118.stdout: mon warn on too few osds = false 2020-11-28T21:45:35.963 INFO:teuthology.orchestra.run.smithi118.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-28T21:45:35.963 INFO:teuthology.orchestra.run.smithi118.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-28T21:45:35.964 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:45:35.964 INFO:teuthology.orchestra.run.smithi118.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-28T21:45:35.964 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:45:35.964 INFO:teuthology.orchestra.run.smithi118.stdout: osd default data pool replay window = 5 2020-11-28T21:45:35.964 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:45:35.965 INFO:teuthology.orchestra.run.smithi118.stdout: mon allow pool delete = true 2020-11-28T21:45:35.966 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:45:35.966 INFO:teuthology.orchestra.run.smithi118.stdout: mon cluster log file level = debug 2020-11-28T21:45:35.966 INFO:teuthology.orchestra.run.smithi118.stdout: debug asserts on shutdown = true 2020-11-28T21:45:35.966 INFO:teuthology.orchestra.run.smithi118.stdout: mon host = "172.21.15.162,172.21.15.118,[v2:172.21.15.118:3301,v1:172.21.15.118:6790]" 2020-11-28T21:45:35.966 INFO:teuthology.orchestra.run.smithi118.stdout: fsid = 7c321bcf-cd33-40f7-a39f-da5470b386cf 2020-11-28T21:45:35.967 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:45:35.967 INFO:teuthology.orchestra.run.smithi118.stdout:[osd] 2020-11-28T21:45:35.967 INFO:teuthology.orchestra.run.smithi118.stdout: osd journal size = 100 2020-11-28T21:45:35.967 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:45:35.968 INFO:teuthology.orchestra.run.smithi118.stdout: osd scrub load threshold = 5.0 2020-11-28T21:45:35.968 INFO:teuthology.orchestra.run.smithi118.stdout: osd scrub max interval = 600 2020-11-28T21:45:35.968 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:45:35.968 INFO:teuthology.orchestra.run.smithi118.stdout: osd recover clone overlap = true 2020-11-28T21:45:35.968 INFO:teuthology.orchestra.run.smithi118.stdout: osd recovery max chunk = 1048576 2020-11-28T21:45:35.969 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:45:35.969 INFO:teuthology.orchestra.run.smithi118.stdout: osd debug shutdown = true 2020-11-28T21:45:35.969 INFO:teuthology.orchestra.run.smithi118.stdout: osd debug op order = true 2020-11-28T21:45:35.969 INFO:teuthology.orchestra.run.smithi118.stdout: osd debug verify stray on activate = true 2020-11-28T21:45:35.969 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:45:35.970 INFO:teuthology.orchestra.run.smithi118.stdout: osd open classes on start = true 2020-11-28T21:45:35.970 INFO:teuthology.orchestra.run.smithi118.stdout: osd debug pg log writeout = true 2020-11-28T21:45:35.970 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:45:35.970 INFO:teuthology.orchestra.run.smithi118.stdout: osd deep scrub update digest min age = 30 2020-11-28T21:45:35.970 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:45:35.971 INFO:teuthology.orchestra.run.smithi118.stdout: osd map max advance = 10 2020-11-28T21:45:35.971 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:45:35.971 INFO:teuthology.orchestra.run.smithi118.stdout: journal zero on create = true 2020-11-28T21:45:35.971 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:45:35.971 INFO:teuthology.orchestra.run.smithi118.stdout: filestore ondisk finisher threads = 3 2020-11-28T21:45:35.972 INFO:teuthology.orchestra.run.smithi118.stdout: filestore apply finisher threads = 3 2020-11-28T21:45:35.972 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:45:35.972 INFO:teuthology.orchestra.run.smithi118.stdout: bdev debug aio = true 2020-11-28T21:45:35.972 INFO:teuthology.orchestra.run.smithi118.stdout: osd debug misdirected ops = true 2020-11-28T21:45:35.972 INFO:teuthology.orchestra.run.smithi118.stdout: bluestore block size = 96636764160 2020-11-28T21:45:35.973 INFO:teuthology.orchestra.run.smithi118.stdout: bluestore compression mode = aggressive 2020-11-28T21:45:35.973 INFO:teuthology.orchestra.run.smithi118.stdout: bluestore fsck on mount = True 2020-11-28T21:45:35.973 INFO:teuthology.orchestra.run.smithi118.stdout: debug bluefs = 20 2020-11-28T21:45:35.973 INFO:teuthology.orchestra.run.smithi118.stdout: debug bluestore = 20 2020-11-28T21:45:35.974 INFO:teuthology.orchestra.run.smithi118.stdout: debug filestore = 20 2020-11-28T21:45:35.974 INFO:teuthology.orchestra.run.smithi118.stdout: debug journal = 20 2020-11-28T21:45:35.974 INFO:teuthology.orchestra.run.smithi118.stdout: debug ms = 20 2020-11-28T21:45:35.974 INFO:teuthology.orchestra.run.smithi118.stdout: debug osd = 25 2020-11-28T21:45:35.975 INFO:teuthology.orchestra.run.smithi118.stdout: debug rocksdb = 10 2020-11-28T21:45:35.975 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd backfillfull_ratio = 0.85 2020-11-28T21:45:35.975 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd full ratio = 0.9 2020-11-28T21:45:35.975 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd nearfull ratio = 0.8 2020-11-28T21:45:35.975 INFO:teuthology.orchestra.run.smithi118.stdout: osd failsafe full ratio = 0.95 2020-11-28T21:45:35.976 INFO:teuthology.orchestra.run.smithi118.stdout: osd objectstore = bluestore 2020-11-28T21:45:35.976 INFO:teuthology.orchestra.run.smithi118.stdout: osd op complaint time = 180 2020-11-28T21:45:35.976 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:45:35.976 INFO:teuthology.orchestra.run.smithi118.stdout:[mgr] 2020-11-28T21:45:35.976 INFO:teuthology.orchestra.run.smithi118.stdout: debug ms = 1 2020-11-28T21:45:35.977 INFO:teuthology.orchestra.run.smithi118.stdout: debug mgr = 20 2020-11-28T21:45:35.977 INFO:teuthology.orchestra.run.smithi118.stdout: debug mon = 20 2020-11-28T21:45:35.977 INFO:teuthology.orchestra.run.smithi118.stdout: debug auth = 20 2020-11-28T21:45:35.977 INFO:teuthology.orchestra.run.smithi118.stdout: mon reweight min pgs per osd = 4 2020-11-28T21:45:35.977 INFO:teuthology.orchestra.run.smithi118.stdout: mon reweight min bytes per osd = 10 2020-11-28T21:45:35.978 INFO:teuthology.orchestra.run.smithi118.stdout: mgr/telemetry/nag = false 2020-11-28T21:45:35.978 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:45:35.978 INFO:teuthology.orchestra.run.smithi118.stdout:[mon] 2020-11-28T21:45:35.978 INFO:teuthology.orchestra.run.smithi118.stdout: debug ms = 1 2020-11-28T21:45:35.979 INFO:teuthology.orchestra.run.smithi118.stdout: debug mon = 20 2020-11-28T21:45:35.979 INFO:teuthology.orchestra.run.smithi118.stdout: debug paxos = 20 2020-11-28T21:45:35.979 INFO:teuthology.orchestra.run.smithi118.stdout: debug auth = 20 2020-11-28T21:45:35.979 INFO:teuthology.orchestra.run.smithi118.stdout: mon data avail warn = 5 2020-11-28T21:45:35.979 INFO:teuthology.orchestra.run.smithi118.stdout: mon mgr mkfs grace = 240 2020-11-28T21:45:35.980 INFO:teuthology.orchestra.run.smithi118.stdout: mon reweight min pgs per osd = 4 2020-11-28T21:45:35.980 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd reporter subtree level = osd 2020-11-28T21:45:35.980 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd prime pg temp = true 2020-11-28T21:45:35.980 INFO:teuthology.orchestra.run.smithi118.stdout: mon reweight min bytes per osd = 10 2020-11-28T21:45:35.981 INFO:teuthology.orchestra.run.smithi118.stdout: mon op complaint time = 120 2020-11-28T21:45:35.981 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:45:35.981 INFO:teuthology.orchestra.run.smithi118.stdout:[client] 2020-11-28T21:45:35.981 INFO:teuthology.orchestra.run.smithi118.stdout: rgw cache enabled = true 2020-11-28T21:45:35.981 INFO:teuthology.orchestra.run.smithi118.stdout: rgw enable ops log = true 2020-11-28T21:45:35.982 INFO:teuthology.orchestra.run.smithi118.stdout: rgw enable usage log = true 2020-11-28T21:45:35.982 INFO:teuthology.orchestra.run.smithi118.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-28T21:45:35.982 INFO:teuthology.orchestra.run.smithi118.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-28T21:45:35.982 INFO:teuthology.orchestra.run.smithi118.stdout: client mount timeout = 600 2020-11-28T21:45:35.982 INFO:teuthology.orchestra.run.smithi118.stdout: debug client = 20 2020-11-28T21:45:35.983 INFO:teuthology.orchestra.run.smithi118.stdout: debug ms = 1 2020-11-28T21:45:35.983 INFO:teuthology.orchestra.run.smithi118.stdout: rados mon op timeout = 15m 2020-11-28T21:45:35.983 INFO:teuthology.orchestra.run.smithi118.stdout: rados osd op timeout = 15m 2020-11-28T21:45:35.983 INFO:teuthology.orchestra.run.smithi118.stdout:[mon.a] 2020-11-28T21:45:35.983 INFO:teuthology.orchestra.run.smithi118.stdout:[mon.b] 2020-11-28T21:45:35.984 INFO:teuthology.orchestra.run.smithi118.stdout:[mon.c] 2020-11-28T21:45:35.984 INFO:teuthology.orchestra.run.smithi118.stdout:[mds.a] 2020-11-28T21:45:35.984 INFO:teuthology.orchestra.run.smithi118.stdout:[mds.c] 2020-11-28T21:45:35.984 INFO:teuthology.orchestra.run.smithi118.stdout:[mds.b] 2020-11-28T21:45:35.984 INFO:teuthology.orchestra.run.smithi118.stdout:[mds] 2020-11-28T21:45:35.985 INFO:teuthology.orchestra.run.smithi118.stdout: debug mds = 20 2020-11-28T21:45:35.985 INFO:teuthology.orchestra.run.smithi118.stdout: debug ms = 1 2020-11-28T21:45:35.985 INFO:teuthology.orchestra.run.smithi118.stdout: mds bal frag = True 2020-11-28T21:45:35.985 INFO:teuthology.orchestra.run.smithi118.stdout: mds bal fragment size max = 10000 2020-11-28T21:45:35.986 INFO:teuthology.orchestra.run.smithi118.stdout: mds bal merge size = 5 2020-11-28T21:45:35.986 INFO:teuthology.orchestra.run.smithi118.stdout: mds bal split bits = 3 2020-11-28T21:45:35.986 INFO:teuthology.orchestra.run.smithi118.stdout: mds bal split size = 100 2020-11-28T21:45:35.986 INFO:teuthology.orchestra.run.smithi118.stdout: mds debug frag = True 2020-11-28T21:45:35.986 INFO:teuthology.orchestra.run.smithi118.stdout: mds debug scatterstat = True 2020-11-28T21:45:35.987 INFO:teuthology.orchestra.run.smithi118.stdout: mds op complaint time = 180 2020-11-28T21:45:35.987 INFO:teuthology.orchestra.run.smithi118.stdout: mds verify scatter = True 2020-11-28T21:45:35.987 INFO:teuthology.orchestra.run.smithi118.stdout: osd op complaint time = 180 2020-11-28T21:45:35.987 INFO:teuthology.orchestra.run.smithi118.stdout: rados mon op timeout = 15m 2020-11-28T21:45:35.987 INFO:teuthology.orchestra.run.smithi118.stdout: rados osd op timeout = 15m 2020-11-28T21:45:35.988 INFO:teuthology.orchestra.run.smithi096.stdout:[global] 2020-11-28T21:45:35.988 INFO:teuthology.orchestra.run.smithi096.stdout: chdir = "" 2020-11-28T21:45:35.988 INFO:teuthology.orchestra.run.smithi096.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-28T21:45:35.988 INFO:teuthology.orchestra.run.smithi096.stdout: auth supported = cephx 2020-11-28T21:45:35.989 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:45:35.989 INFO:teuthology.orchestra.run.smithi096.stdout: filestore xattr use omap = true 2020-11-28T21:45:35.989 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:45:35.989 INFO:teuthology.orchestra.run.smithi096.stdout: mon clock drift allowed = 1.000 2020-11-28T21:45:35.989 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:45:35.990 INFO:teuthology.orchestra.run.smithi096.stdout: osd crush chooseleaf type = 0 2020-11-28T21:45:35.990 INFO:teuthology.orchestra.run.smithi096.stdout: auth debug = true 2020-11-28T21:45:35.990 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:45:35.990 INFO:teuthology.orchestra.run.smithi096.stdout: ms die on old message = true 2020-11-28T21:45:35.991 INFO:teuthology.orchestra.run.smithi096.stdout: ms die on bug = true 2020-11-28T21:45:35.991 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:45:35.991 INFO:teuthology.orchestra.run.smithi096.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-28T21:45:35.991 INFO:teuthology.orchestra.run.smithi096.stdout: mon pg warn max object skew = 0 2020-11-28T21:45:35.992 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:45:35.992 INFO:teuthology.orchestra.run.smithi096.stdout: # disable pg_autoscaler by default for new pools 2020-11-28T21:45:35.992 INFO:teuthology.orchestra.run.smithi096.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-28T21:45:35.992 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:45:35.993 INFO:teuthology.orchestra.run.smithi096.stdout: osd pool default size = 2 2020-11-28T21:45:35.993 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:45:35.993 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd allow primary affinity = true 2020-11-28T21:45:35.993 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd allow pg remap = true 2020-11-28T21:45:35.993 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on legacy crush tunables = false 2020-11-28T21:45:35.994 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on crush straw calc version zero = false 2020-11-28T21:45:35.994 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on no sortbitwise = false 2020-11-28T21:45:35.994 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on osd down out interval zero = false 2020-11-28T21:45:35.994 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on too few osds = false 2020-11-28T21:45:35.994 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-28T21:45:35.995 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-28T21:45:35.995 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:45:35.995 INFO:teuthology.orchestra.run.smithi096.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-28T21:45:35.995 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:45:35.995 INFO:teuthology.orchestra.run.smithi096.stdout: osd default data pool replay window = 5 2020-11-28T21:45:35.996 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:45:35.996 INFO:teuthology.orchestra.run.smithi096.stdout: mon allow pool delete = true 2020-11-28T21:45:35.996 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:45:35.996 INFO:teuthology.orchestra.run.smithi096.stdout: mon cluster log file level = debug 2020-11-28T21:45:35.996 INFO:teuthology.orchestra.run.smithi096.stdout: debug asserts on shutdown = true 2020-11-28T21:45:35.997 INFO:teuthology.orchestra.run.smithi096.stdout: mon host = "172.21.15.162,172.21.15.118,[v2:172.21.15.118:3301,v1:172.21.15.118:6790]" 2020-11-28T21:45:35.997 INFO:teuthology.orchestra.run.smithi096.stdout: fsid = 7c321bcf-cd33-40f7-a39f-da5470b386cf 2020-11-28T21:45:35.997 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:45:35.997 INFO:teuthology.orchestra.run.smithi096.stdout:[osd] 2020-11-28T21:45:35.997 INFO:teuthology.orchestra.run.smithi096.stdout: osd journal size = 100 2020-11-28T21:45:35.998 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:45:35.998 INFO:teuthology.orchestra.run.smithi096.stdout: osd scrub load threshold = 5.0 2020-11-28T21:45:35.998 INFO:teuthology.orchestra.run.smithi096.stdout: osd scrub max interval = 600 2020-11-28T21:45:35.998 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:45:35.999 INFO:teuthology.orchestra.run.smithi096.stdout: osd recover clone overlap = true 2020-11-28T21:45:35.999 INFO:teuthology.orchestra.run.smithi096.stdout: osd recovery max chunk = 1048576 2020-11-28T21:45:35.999 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:45:35.999 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug shutdown = true 2020-11-28T21:45:35.999 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug op order = true 2020-11-28T21:45:36.000 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug verify stray on activate = true 2020-11-28T21:45:36.000 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:45:36.000 INFO:teuthology.orchestra.run.smithi096.stdout: osd open classes on start = true 2020-11-28T21:45:36.000 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug pg log writeout = true 2020-11-28T21:45:36.000 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:45:36.001 INFO:teuthology.orchestra.run.smithi096.stdout: osd deep scrub update digest min age = 30 2020-11-28T21:45:36.001 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:45:36.001 INFO:teuthology.orchestra.run.smithi096.stdout: osd map max advance = 10 2020-11-28T21:45:36.001 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:45:36.001 INFO:teuthology.orchestra.run.smithi096.stdout: journal zero on create = true 2020-11-28T21:45:36.002 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:45:36.002 INFO:teuthology.orchestra.run.smithi096.stdout: filestore ondisk finisher threads = 3 2020-11-28T21:45:36.002 INFO:teuthology.orchestra.run.smithi096.stdout: filestore apply finisher threads = 3 2020-11-28T21:45:36.002 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:45:36.002 INFO:teuthology.orchestra.run.smithi096.stdout: bdev debug aio = true 2020-11-28T21:45:36.003 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug misdirected ops = true 2020-11-28T21:45:36.003 INFO:teuthology.orchestra.run.smithi096.stdout: bluestore block size = 96636764160 2020-11-28T21:45:36.003 INFO:teuthology.orchestra.run.smithi096.stdout: bluestore compression mode = aggressive 2020-11-28T21:45:36.003 INFO:teuthology.orchestra.run.smithi096.stdout: bluestore fsck on mount = True 2020-11-28T21:45:36.003 INFO:teuthology.orchestra.run.smithi096.stdout: debug bluefs = 20 2020-11-28T21:45:36.004 INFO:teuthology.orchestra.run.smithi096.stdout: debug bluestore = 20 2020-11-28T21:45:36.004 INFO:teuthology.orchestra.run.smithi096.stdout: debug filestore = 20 2020-11-28T21:45:36.004 INFO:teuthology.orchestra.run.smithi096.stdout: debug journal = 20 2020-11-28T21:45:36.004 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 20 2020-11-28T21:45:36.004 INFO:teuthology.orchestra.run.smithi096.stdout: debug osd = 25 2020-11-28T21:45:36.005 INFO:teuthology.orchestra.run.smithi096.stdout: debug rocksdb = 10 2020-11-28T21:45:36.005 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd backfillfull_ratio = 0.85 2020-11-28T21:45:36.005 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd full ratio = 0.9 2020-11-28T21:45:36.005 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd nearfull ratio = 0.8 2020-11-28T21:45:36.005 INFO:teuthology.orchestra.run.smithi096.stdout: osd failsafe full ratio = 0.95 2020-11-28T21:45:36.006 INFO:teuthology.orchestra.run.smithi096.stdout: osd objectstore = bluestore 2020-11-28T21:45:36.006 INFO:teuthology.orchestra.run.smithi096.stdout: osd op complaint time = 180 2020-11-28T21:45:36.006 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:45:36.006 INFO:teuthology.orchestra.run.smithi096.stdout:[mgr] 2020-11-28T21:45:36.006 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2020-11-28T21:45:36.007 INFO:teuthology.orchestra.run.smithi096.stdout: debug mgr = 20 2020-11-28T21:45:36.007 INFO:teuthology.orchestra.run.smithi096.stdout: debug mon = 20 2020-11-28T21:45:36.007 INFO:teuthology.orchestra.run.smithi096.stdout: debug auth = 20 2020-11-28T21:45:36.007 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min pgs per osd = 4 2020-11-28T21:45:36.007 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min bytes per osd = 10 2020-11-28T21:45:36.008 INFO:teuthology.orchestra.run.smithi096.stdout: mgr/telemetry/nag = false 2020-11-28T21:45:36.008 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:45:36.008 INFO:teuthology.orchestra.run.smithi096.stdout:[mon] 2020-11-28T21:45:36.008 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2020-11-28T21:45:36.008 INFO:teuthology.orchestra.run.smithi096.stdout: debug mon = 20 2020-11-28T21:45:36.009 INFO:teuthology.orchestra.run.smithi096.stdout: debug paxos = 20 2020-11-28T21:45:36.009 INFO:teuthology.orchestra.run.smithi096.stdout: debug auth = 20 2020-11-28T21:45:36.009 INFO:teuthology.orchestra.run.smithi096.stdout: mon data avail warn = 5 2020-11-28T21:45:36.009 INFO:teuthology.orchestra.run.smithi096.stdout: mon mgr mkfs grace = 240 2020-11-28T21:45:36.009 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min pgs per osd = 4 2020-11-28T21:45:36.010 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd reporter subtree level = osd 2020-11-28T21:45:36.010 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd prime pg temp = true 2020-11-28T21:45:36.010 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min bytes per osd = 10 2020-11-28T21:45:36.010 INFO:teuthology.orchestra.run.smithi096.stdout: mon op complaint time = 120 2020-11-28T21:45:36.011 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:45:36.011 INFO:teuthology.orchestra.run.smithi096.stdout:[client] 2020-11-28T21:45:36.011 INFO:teuthology.orchestra.run.smithi096.stdout: rgw cache enabled = true 2020-11-28T21:45:36.011 INFO:teuthology.orchestra.run.smithi096.stdout: rgw enable ops log = true 2020-11-28T21:45:36.011 INFO:teuthology.orchestra.run.smithi096.stdout: rgw enable usage log = true 2020-11-28T21:45:36.012 INFO:teuthology.orchestra.run.smithi096.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-28T21:45:36.012 INFO:teuthology.orchestra.run.smithi096.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-28T21:45:36.012 INFO:teuthology.orchestra.run.smithi096.stdout: client mount timeout = 600 2020-11-28T21:45:36.012 INFO:teuthology.orchestra.run.smithi096.stdout: debug client = 20 2020-11-28T21:45:36.012 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2020-11-28T21:45:36.013 INFO:teuthology.orchestra.run.smithi096.stdout: rados mon op timeout = 15m 2020-11-28T21:45:36.013 INFO:teuthology.orchestra.run.smithi096.stdout: rados osd op timeout = 15m 2020-11-28T21:45:36.013 INFO:teuthology.orchestra.run.smithi096.stdout:[mon.a] 2020-11-28T21:45:36.013 INFO:teuthology.orchestra.run.smithi096.stdout:[mon.b] 2020-11-28T21:45:36.013 INFO:teuthology.orchestra.run.smithi096.stdout:[mon.c] 2020-11-28T21:45:36.014 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.a] 2020-11-28T21:45:36.014 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.c] 2020-11-28T21:45:36.014 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.b] 2020-11-28T21:45:36.014 INFO:teuthology.orchestra.run.smithi096.stdout:[mds] 2020-11-28T21:45:36.015 INFO:teuthology.orchestra.run.smithi096.stdout: debug mds = 20 2020-11-28T21:45:36.015 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2020-11-28T21:45:36.015 INFO:teuthology.orchestra.run.smithi096.stdout: mds bal frag = True 2020-11-28T21:45:36.015 INFO:teuthology.orchestra.run.smithi096.stdout: mds bal fragment size max = 10000 2020-11-28T21:45:36.015 INFO:teuthology.orchestra.run.smithi096.stdout: mds bal merge size = 5 2020-11-28T21:45:36.016 INFO:teuthology.orchestra.run.smithi096.stdout: mds bal split bits = 3 2020-11-28T21:45:36.016 INFO:teuthology.orchestra.run.smithi096.stdout: mds bal split size = 100 2020-11-28T21:45:36.016 INFO:teuthology.orchestra.run.smithi096.stdout: mds debug frag = True 2020-11-28T21:45:36.016 INFO:teuthology.orchestra.run.smithi096.stdout: mds debug scatterstat = True 2020-11-28T21:45:36.016 INFO:teuthology.orchestra.run.smithi096.stdout: mds op complaint time = 180 2020-11-28T21:45:36.017 INFO:teuthology.orchestra.run.smithi096.stdout: mds verify scatter = True 2020-11-28T21:45:36.017 INFO:teuthology.orchestra.run.smithi096.stdout: osd op complaint time = 180 2020-11-28T21:45:36.017 INFO:teuthology.orchestra.run.smithi096.stdout: rados mon op timeout = 15m 2020-11-28T21:45:36.017 INFO:teuthology.orchestra.run.smithi096.stdout: rados osd op timeout = 15m 2020-11-28T21:45:36.020 INFO:teuthology.orchestra.run.smithi192.stdout:[global] 2020-11-28T21:45:36.020 INFO:teuthology.orchestra.run.smithi192.stdout: chdir = "" 2020-11-28T21:45:36.020 INFO:teuthology.orchestra.run.smithi192.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-28T21:45:36.021 INFO:teuthology.orchestra.run.smithi192.stdout: auth supported = cephx 2020-11-28T21:45:36.021 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:45:36.021 INFO:teuthology.orchestra.run.smithi192.stdout: filestore xattr use omap = true 2020-11-28T21:45:36.021 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:45:36.021 INFO:teuthology.orchestra.run.smithi192.stdout: mon clock drift allowed = 1.000 2020-11-28T21:45:36.022 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:45:36.022 INFO:teuthology.orchestra.run.smithi192.stdout: osd crush chooseleaf type = 0 2020-11-28T21:45:36.022 INFO:teuthology.orchestra.run.smithi192.stdout: auth debug = true 2020-11-28T21:45:36.022 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:45:36.022 INFO:teuthology.orchestra.run.smithi192.stdout: ms die on old message = true 2020-11-28T21:45:36.023 INFO:teuthology.orchestra.run.smithi192.stdout: ms die on bug = true 2020-11-28T21:45:36.023 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:45:36.023 INFO:teuthology.orchestra.run.smithi192.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-28T21:45:36.023 INFO:teuthology.orchestra.run.smithi192.stdout: mon pg warn max object skew = 0 2020-11-28T21:45:36.023 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:45:36.024 INFO:teuthology.orchestra.run.smithi192.stdout: # disable pg_autoscaler by default for new pools 2020-11-28T21:45:36.024 INFO:teuthology.orchestra.run.smithi192.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-28T21:45:36.024 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:45:36.024 INFO:teuthology.orchestra.run.smithi192.stdout: osd pool default size = 2 2020-11-28T21:45:36.024 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:45:36.025 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd allow primary affinity = true 2020-11-28T21:45:36.025 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd allow pg remap = true 2020-11-28T21:45:36.025 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on legacy crush tunables = false 2020-11-28T21:45:36.026 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on crush straw calc version zero = false 2020-11-28T21:45:36.026 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on no sortbitwise = false 2020-11-28T21:45:36.027 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on osd down out interval zero = false 2020-11-28T21:45:36.028 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on too few osds = false 2020-11-28T21:45:36.028 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-28T21:45:36.029 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-28T21:45:36.030 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:45:36.030 INFO:teuthology.orchestra.run.smithi192.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-28T21:45:36.031 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:45:36.032 INFO:teuthology.orchestra.run.smithi192.stdout: osd default data pool replay window = 5 2020-11-28T21:45:36.032 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:45:36.032 INFO:teuthology.orchestra.run.smithi192.stdout: mon allow pool delete = true 2020-11-28T21:45:36.032 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:45:36.033 INFO:teuthology.orchestra.run.smithi192.stdout: mon cluster log file level = debug 2020-11-28T21:45:36.033 INFO:teuthology.orchestra.run.smithi192.stdout: debug asserts on shutdown = true 2020-11-28T21:45:36.033 INFO:teuthology.orchestra.run.smithi192.stdout: mon host = "172.21.15.162,172.21.15.118,[v2:172.21.15.118:3301,v1:172.21.15.118:6790]" 2020-11-28T21:45:36.033 INFO:teuthology.orchestra.run.smithi192.stdout: fsid = 7c321bcf-cd33-40f7-a39f-da5470b386cf 2020-11-28T21:45:36.034 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:45:36.034 INFO:teuthology.orchestra.run.smithi192.stdout:[osd] 2020-11-28T21:45:36.034 INFO:teuthology.orchestra.run.smithi192.stdout: osd journal size = 100 2020-11-28T21:45:36.034 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:45:36.034 INFO:teuthology.orchestra.run.smithi192.stdout: osd scrub load threshold = 5.0 2020-11-28T21:45:36.035 INFO:teuthology.orchestra.run.smithi192.stdout: osd scrub max interval = 600 2020-11-28T21:45:36.035 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:45:36.035 INFO:teuthology.orchestra.run.smithi192.stdout: osd recover clone overlap = true 2020-11-28T21:45:36.035 INFO:teuthology.orchestra.run.smithi192.stdout: osd recovery max chunk = 1048576 2020-11-28T21:45:36.035 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:45:36.036 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug shutdown = true 2020-11-28T21:45:36.036 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug op order = true 2020-11-28T21:45:36.036 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug verify stray on activate = true 2020-11-28T21:45:36.036 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:45:36.036 INFO:teuthology.orchestra.run.smithi192.stdout: osd open classes on start = true 2020-11-28T21:45:36.037 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug pg log writeout = true 2020-11-28T21:45:36.037 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:45:36.037 INFO:teuthology.orchestra.run.smithi192.stdout: osd deep scrub update digest min age = 30 2020-11-28T21:45:36.037 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:45:36.037 INFO:teuthology.orchestra.run.smithi192.stdout: osd map max advance = 10 2020-11-28T21:45:36.038 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:45:36.038 INFO:teuthology.orchestra.run.smithi192.stdout: journal zero on create = true 2020-11-28T21:45:36.038 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:45:36.038 INFO:teuthology.orchestra.run.smithi192.stdout: filestore ondisk finisher threads = 3 2020-11-28T21:45:36.038 INFO:teuthology.orchestra.run.smithi192.stdout: filestore apply finisher threads = 3 2020-11-28T21:45:36.039 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:45:36.039 INFO:teuthology.orchestra.run.smithi192.stdout: bdev debug aio = true 2020-11-28T21:45:36.039 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug misdirected ops = true 2020-11-28T21:45:36.039 INFO:teuthology.orchestra.run.smithi192.stdout: bluestore block size = 96636764160 2020-11-28T21:45:36.039 INFO:teuthology.orchestra.run.smithi192.stdout: bluestore compression mode = aggressive 2020-11-28T21:45:36.040 INFO:teuthology.orchestra.run.smithi192.stdout: bluestore fsck on mount = True 2020-11-28T21:45:36.040 INFO:teuthology.orchestra.run.smithi192.stdout: debug bluefs = 20 2020-11-28T21:45:36.040 INFO:teuthology.orchestra.run.smithi192.stdout: debug bluestore = 20 2020-11-28T21:45:36.040 INFO:teuthology.orchestra.run.smithi192.stdout: debug filestore = 20 2020-11-28T21:45:36.040 INFO:teuthology.orchestra.run.smithi192.stdout: debug journal = 20 2020-11-28T21:45:36.041 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 20 2020-11-28T21:45:36.041 INFO:teuthology.orchestra.run.smithi192.stdout: debug osd = 25 2020-11-28T21:45:36.041 INFO:teuthology.orchestra.run.smithi192.stdout: debug rocksdb = 10 2020-11-28T21:45:36.041 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd backfillfull_ratio = 0.85 2020-11-28T21:45:36.041 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd full ratio = 0.9 2020-11-28T21:45:36.042 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd nearfull ratio = 0.8 2020-11-28T21:45:36.042 INFO:teuthology.orchestra.run.smithi192.stdout: osd failsafe full ratio = 0.95 2020-11-28T21:45:36.042 INFO:teuthology.orchestra.run.smithi192.stdout: osd objectstore = bluestore 2020-11-28T21:45:36.042 INFO:teuthology.orchestra.run.smithi192.stdout: osd op complaint time = 180 2020-11-28T21:45:36.042 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:45:36.043 INFO:teuthology.orchestra.run.smithi192.stdout:[mgr] 2020-11-28T21:45:36.043 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2020-11-28T21:45:36.043 INFO:teuthology.orchestra.run.smithi192.stdout: debug mgr = 20 2020-11-28T21:45:36.043 INFO:teuthology.orchestra.run.smithi192.stdout: debug mon = 20 2020-11-28T21:45:36.044 INFO:teuthology.orchestra.run.smithi192.stdout: debug auth = 20 2020-11-28T21:45:36.044 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min pgs per osd = 4 2020-11-28T21:45:36.044 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min bytes per osd = 10 2020-11-28T21:45:36.044 INFO:teuthology.orchestra.run.smithi192.stdout: mgr/telemetry/nag = false 2020-11-28T21:45:36.044 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:45:36.045 INFO:teuthology.orchestra.run.smithi192.stdout:[mon] 2020-11-28T21:45:36.045 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2020-11-28T21:45:36.045 INFO:teuthology.orchestra.run.smithi192.stdout: debug mon = 20 2020-11-28T21:45:36.045 INFO:teuthology.orchestra.run.smithi192.stdout: debug paxos = 20 2020-11-28T21:45:36.046 INFO:teuthology.orchestra.run.smithi192.stdout: debug auth = 20 2020-11-28T21:45:36.046 INFO:teuthology.orchestra.run.smithi192.stdout: mon data avail warn = 5 2020-11-28T21:45:36.046 INFO:teuthology.orchestra.run.smithi192.stdout: mon mgr mkfs grace = 240 2020-11-28T21:45:36.046 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min pgs per osd = 4 2020-11-28T21:45:36.046 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd reporter subtree level = osd 2020-11-28T21:45:36.047 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd prime pg temp = true 2020-11-28T21:45:36.047 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min bytes per osd = 10 2020-11-28T21:45:36.047 INFO:teuthology.orchestra.run.smithi192.stdout: mon op complaint time = 120 2020-11-28T21:45:36.047 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:45:36.047 INFO:teuthology.orchestra.run.smithi192.stdout:[client] 2020-11-28T21:45:36.048 INFO:teuthology.orchestra.run.smithi192.stdout: rgw cache enabled = true 2020-11-28T21:45:36.048 INFO:teuthology.orchestra.run.smithi192.stdout: rgw enable ops log = true 2020-11-28T21:45:36.048 INFO:teuthology.orchestra.run.smithi192.stdout: rgw enable usage log = true 2020-11-28T21:45:36.048 INFO:teuthology.orchestra.run.smithi192.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-28T21:45:36.048 INFO:teuthology.orchestra.run.smithi192.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-28T21:45:36.049 INFO:teuthology.orchestra.run.smithi192.stdout: client mount timeout = 600 2020-11-28T21:45:36.049 INFO:teuthology.orchestra.run.smithi192.stdout: debug client = 20 2020-11-28T21:45:36.049 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2020-11-28T21:45:36.049 INFO:teuthology.orchestra.run.smithi192.stdout: rados mon op timeout = 15m 2020-11-28T21:45:36.049 INFO:teuthology.orchestra.run.smithi192.stdout: rados osd op timeout = 15m 2020-11-28T21:45:36.050 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.a] 2020-11-28T21:45:36.050 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.b] 2020-11-28T21:45:36.050 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.c] 2020-11-28T21:45:36.050 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.a] 2020-11-28T21:45:36.050 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.c] 2020-11-28T21:45:36.051 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.b] 2020-11-28T21:45:36.051 INFO:teuthology.orchestra.run.smithi192.stdout:[mds] 2020-11-28T21:45:36.051 INFO:teuthology.orchestra.run.smithi192.stdout: debug mds = 20 2020-11-28T21:45:36.051 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2020-11-28T21:45:36.051 INFO:teuthology.orchestra.run.smithi192.stdout: mds bal frag = True 2020-11-28T21:45:36.052 INFO:teuthology.orchestra.run.smithi192.stdout: mds bal fragment size max = 10000 2020-11-28T21:45:36.052 INFO:teuthology.orchestra.run.smithi192.stdout: mds bal merge size = 5 2020-11-28T21:45:36.052 INFO:teuthology.orchestra.run.smithi192.stdout: mds bal split bits = 3 2020-11-28T21:45:36.052 INFO:teuthology.orchestra.run.smithi192.stdout: mds bal split size = 100 2020-11-28T21:45:36.052 INFO:teuthology.orchestra.run.smithi192.stdout: mds debug frag = True 2020-11-28T21:45:36.053 INFO:teuthology.orchestra.run.smithi192.stdout: mds debug scatterstat = True 2020-11-28T21:45:36.053 INFO:teuthology.orchestra.run.smithi192.stdout: mds op complaint time = 180 2020-11-28T21:45:36.053 INFO:teuthology.orchestra.run.smithi192.stdout: mds verify scatter = True 2020-11-28T21:45:36.053 INFO:teuthology.orchestra.run.smithi192.stdout: osd op complaint time = 180 2020-11-28T21:45:36.053 INFO:teuthology.orchestra.run.smithi192.stdout: rados mon op timeout = 15m 2020-11-28T21:45:36.054 INFO:teuthology.orchestra.run.smithi192.stdout: rados osd op timeout = 15m 2020-11-28T21:45:36.054 INFO:teuthology.orchestra.run.smithi162.stdout:[global] 2020-11-28T21:45:36.054 INFO:teuthology.orchestra.run.smithi162.stdout: chdir = "" 2020-11-28T21:45:36.054 INFO:teuthology.orchestra.run.smithi162.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-28T21:45:36.055 INFO:teuthology.orchestra.run.smithi162.stdout: auth supported = cephx 2020-11-28T21:45:36.055 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:45:36.055 INFO:teuthology.orchestra.run.smithi162.stdout: filestore xattr use omap = true 2020-11-28T21:45:36.056 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:45:36.056 INFO:teuthology.orchestra.run.smithi162.stdout: mon clock drift allowed = 1.000 2020-11-28T21:45:36.056 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:45:36.056 INFO:teuthology.orchestra.run.smithi162.stdout: osd crush chooseleaf type = 0 2020-11-28T21:45:36.056 INFO:teuthology.orchestra.run.smithi162.stdout: auth debug = true 2020-11-28T21:45:36.057 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:45:36.057 INFO:teuthology.orchestra.run.smithi162.stdout: ms die on old message = true 2020-11-28T21:45:36.057 INFO:teuthology.orchestra.run.smithi162.stdout: ms die on bug = true 2020-11-28T21:45:36.057 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:45:36.057 INFO:teuthology.orchestra.run.smithi162.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-28T21:45:36.058 INFO:teuthology.orchestra.run.smithi162.stdout: mon pg warn max object skew = 0 2020-11-28T21:45:36.058 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:45:36.058 INFO:teuthology.orchestra.run.smithi162.stdout: # disable pg_autoscaler by default for new pools 2020-11-28T21:45:36.058 INFO:teuthology.orchestra.run.smithi162.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-28T21:45:36.058 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:45:36.059 INFO:teuthology.orchestra.run.smithi162.stdout: osd pool default size = 2 2020-11-28T21:45:36.059 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:45:36.059 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd allow primary affinity = true 2020-11-28T21:45:36.059 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd allow pg remap = true 2020-11-28T21:45:36.059 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on legacy crush tunables = false 2020-11-28T21:45:36.060 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on crush straw calc version zero = false 2020-11-28T21:45:36.060 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on no sortbitwise = false 2020-11-28T21:45:36.060 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on osd down out interval zero = false 2020-11-28T21:45:36.060 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on too few osds = false 2020-11-28T21:45:36.060 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-28T21:45:36.061 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-28T21:45:36.061 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:45:36.061 INFO:teuthology.orchestra.run.smithi162.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-28T21:45:36.061 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:45:36.061 INFO:teuthology.orchestra.run.smithi162.stdout: osd default data pool replay window = 5 2020-11-28T21:45:36.062 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:45:36.062 INFO:teuthology.orchestra.run.smithi162.stdout: mon allow pool delete = true 2020-11-28T21:45:36.062 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:45:36.062 INFO:teuthology.orchestra.run.smithi162.stdout: mon cluster log file level = debug 2020-11-28T21:45:36.062 INFO:teuthology.orchestra.run.smithi162.stdout: debug asserts on shutdown = true 2020-11-28T21:45:36.063 INFO:teuthology.orchestra.run.smithi162.stdout: mon host = "172.21.15.162,172.21.15.118,[v2:172.21.15.118:3301,v1:172.21.15.118:6790]" 2020-11-28T21:45:36.063 INFO:teuthology.orchestra.run.smithi162.stdout: fsid = 7c321bcf-cd33-40f7-a39f-da5470b386cf 2020-11-28T21:45:36.063 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:45:36.063 INFO:teuthology.orchestra.run.smithi162.stdout:[osd] 2020-11-28T21:45:36.063 INFO:teuthology.orchestra.run.smithi162.stdout: osd journal size = 100 2020-11-28T21:45:36.064 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:45:36.064 INFO:teuthology.orchestra.run.smithi162.stdout: osd scrub load threshold = 5.0 2020-11-28T21:45:36.064 INFO:teuthology.orchestra.run.smithi162.stdout: osd scrub max interval = 600 2020-11-28T21:45:36.064 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:45:36.064 INFO:teuthology.orchestra.run.smithi162.stdout: osd recover clone overlap = true 2020-11-28T21:45:36.065 INFO:teuthology.orchestra.run.smithi162.stdout: osd recovery max chunk = 1048576 2020-11-28T21:45:36.065 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:45:36.065 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug shutdown = true 2020-11-28T21:45:36.065 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug op order = true 2020-11-28T21:45:36.065 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug verify stray on activate = true 2020-11-28T21:45:36.066 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:45:36.066 INFO:teuthology.orchestra.run.smithi162.stdout: osd open classes on start = true 2020-11-28T21:45:36.066 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug pg log writeout = true 2020-11-28T21:45:36.066 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:45:36.067 INFO:teuthology.orchestra.run.smithi162.stdout: osd deep scrub update digest min age = 30 2020-11-28T21:45:36.067 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:45:36.067 INFO:teuthology.orchestra.run.smithi162.stdout: osd map max advance = 10 2020-11-28T21:45:36.067 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:45:36.067 INFO:teuthology.orchestra.run.smithi162.stdout: journal zero on create = true 2020-11-28T21:45:36.068 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:45:36.068 INFO:teuthology.orchestra.run.smithi162.stdout: filestore ondisk finisher threads = 3 2020-11-28T21:45:36.068 INFO:teuthology.orchestra.run.smithi162.stdout: filestore apply finisher threads = 3 2020-11-28T21:45:36.068 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:45:36.068 INFO:teuthology.orchestra.run.smithi162.stdout: bdev debug aio = true 2020-11-28T21:45:36.069 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug misdirected ops = true 2020-11-28T21:45:36.069 INFO:teuthology.orchestra.run.smithi162.stdout: bluestore block size = 96636764160 2020-11-28T21:45:36.069 INFO:teuthology.orchestra.run.smithi162.stdout: bluestore compression mode = aggressive 2020-11-28T21:45:36.069 INFO:teuthology.orchestra.run.smithi162.stdout: bluestore fsck on mount = True 2020-11-28T21:45:36.069 INFO:teuthology.orchestra.run.smithi162.stdout: debug bluefs = 20 2020-11-28T21:45:36.070 INFO:teuthology.orchestra.run.smithi162.stdout: debug bluestore = 20 2020-11-28T21:45:36.070 INFO:teuthology.orchestra.run.smithi162.stdout: debug filestore = 20 2020-11-28T21:45:36.070 INFO:teuthology.orchestra.run.smithi162.stdout: debug journal = 20 2020-11-28T21:45:36.070 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 20 2020-11-28T21:45:36.070 INFO:teuthology.orchestra.run.smithi162.stdout: debug osd = 25 2020-11-28T21:45:36.071 INFO:teuthology.orchestra.run.smithi162.stdout: debug rocksdb = 10 2020-11-28T21:45:36.071 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd backfillfull_ratio = 0.85 2020-11-28T21:45:36.071 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd full ratio = 0.9 2020-11-28T21:45:36.071 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd nearfull ratio = 0.8 2020-11-28T21:45:36.071 INFO:teuthology.orchestra.run.smithi162.stdout: osd failsafe full ratio = 0.95 2020-11-28T21:45:36.072 INFO:teuthology.orchestra.run.smithi162.stdout: osd objectstore = bluestore 2020-11-28T21:45:36.072 INFO:teuthology.orchestra.run.smithi162.stdout: osd op complaint time = 180 2020-11-28T21:45:36.072 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:45:36.072 INFO:teuthology.orchestra.run.smithi162.stdout:[mgr] 2020-11-28T21:45:36.072 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2020-11-28T21:45:36.073 INFO:teuthology.orchestra.run.smithi162.stdout: debug mgr = 20 2020-11-28T21:45:36.073 INFO:teuthology.orchestra.run.smithi162.stdout: debug mon = 20 2020-11-28T21:45:36.073 INFO:teuthology.orchestra.run.smithi162.stdout: debug auth = 20 2020-11-28T21:45:36.073 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min pgs per osd = 4 2020-11-28T21:45:36.073 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min bytes per osd = 10 2020-11-28T21:45:36.074 INFO:teuthology.orchestra.run.smithi162.stdout: mgr/telemetry/nag = false 2020-11-28T21:45:36.074 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:45:36.074 INFO:teuthology.orchestra.run.smithi162.stdout:[mon] 2020-11-28T21:45:36.074 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2020-11-28T21:45:36.075 INFO:teuthology.orchestra.run.smithi162.stdout: debug mon = 20 2020-11-28T21:45:36.075 INFO:teuthology.orchestra.run.smithi162.stdout: debug paxos = 20 2020-11-28T21:45:36.075 INFO:teuthology.orchestra.run.smithi162.stdout: debug auth = 20 2020-11-28T21:45:36.075 INFO:teuthology.orchestra.run.smithi162.stdout: mon data avail warn = 5 2020-11-28T21:45:36.075 INFO:teuthology.orchestra.run.smithi162.stdout: mon mgr mkfs grace = 240 2020-11-28T21:45:36.076 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min pgs per osd = 4 2020-11-28T21:45:36.076 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd reporter subtree level = osd 2020-11-28T21:45:36.076 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd prime pg temp = true 2020-11-28T21:45:36.076 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min bytes per osd = 10 2020-11-28T21:45:36.076 INFO:teuthology.orchestra.run.smithi162.stdout: mon op complaint time = 120 2020-11-28T21:45:36.077 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:45:36.077 INFO:teuthology.orchestra.run.smithi162.stdout:[client] 2020-11-28T21:45:36.077 INFO:teuthology.orchestra.run.smithi162.stdout: rgw cache enabled = true 2020-11-28T21:45:36.077 INFO:teuthology.orchestra.run.smithi162.stdout: rgw enable ops log = true 2020-11-28T21:45:36.077 INFO:teuthology.orchestra.run.smithi162.stdout: rgw enable usage log = true 2020-11-28T21:45:36.078 INFO:teuthology.orchestra.run.smithi162.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-28T21:45:36.078 INFO:teuthology.orchestra.run.smithi162.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-28T21:45:36.078 INFO:teuthology.orchestra.run.smithi162.stdout: client mount timeout = 600 2020-11-28T21:45:36.078 INFO:teuthology.orchestra.run.smithi162.stdout: debug client = 20 2020-11-28T21:45:36.078 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2020-11-28T21:45:36.079 INFO:teuthology.orchestra.run.smithi162.stdout: rados mon op timeout = 15m 2020-11-28T21:45:36.079 INFO:teuthology.orchestra.run.smithi162.stdout: rados osd op timeout = 15m 2020-11-28T21:45:36.079 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.a] 2020-11-28T21:45:36.079 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.b] 2020-11-28T21:45:36.079 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.c] 2020-11-28T21:45:36.080 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.a] 2020-11-28T21:45:36.080 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.c] 2020-11-28T21:45:36.080 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.b] 2020-11-28T21:45:36.080 INFO:teuthology.orchestra.run.smithi162.stdout:[mds] 2020-11-28T21:45:36.081 INFO:teuthology.orchestra.run.smithi162.stdout: debug mds = 20 2020-11-28T21:45:36.081 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2020-11-28T21:45:36.081 INFO:teuthology.orchestra.run.smithi162.stdout: mds bal frag = True 2020-11-28T21:45:36.081 INFO:teuthology.orchestra.run.smithi162.stdout: mds bal fragment size max = 10000 2020-11-28T21:45:36.081 INFO:teuthology.orchestra.run.smithi162.stdout: mds bal merge size = 5 2020-11-28T21:45:36.082 INFO:teuthology.orchestra.run.smithi162.stdout: mds bal split bits = 3 2020-11-28T21:45:36.082 INFO:teuthology.orchestra.run.smithi162.stdout: mds bal split size = 100 2020-11-28T21:45:36.082 INFO:teuthology.orchestra.run.smithi162.stdout: mds debug frag = True 2020-11-28T21:45:36.082 INFO:teuthology.orchestra.run.smithi162.stdout: mds debug scatterstat = True 2020-11-28T21:45:36.083 INFO:teuthology.orchestra.run.smithi162.stdout: mds op complaint time = 180 2020-11-28T21:45:36.083 INFO:teuthology.orchestra.run.smithi162.stdout: mds verify scatter = True 2020-11-28T21:45:36.083 INFO:teuthology.orchestra.run.smithi162.stdout: osd op complaint time = 180 2020-11-28T21:45:36.083 INFO:teuthology.orchestra.run.smithi162.stdout: rados mon op timeout = 15m 2020-11-28T21:45:36.083 INFO:teuthology.orchestra.run.smithi162.stdout: rados osd op timeout = 15m 2020-11-28T21:45:36.084 INFO:tasks.ceph:Creating admin key on mon.a... 2020-11-28T21:45:36.084 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2020-11-28T21:45:36.132 INFO:tasks.ceph:Copying monmap to all nodes... 2020-11-28T21:45:36.138 DEBUG:teuthology.orchestra.remote:smithi162:/etc/ceph/ceph.keyring is 206B 2020-11-28T21:45:36.240 DEBUG:teuthology.orchestra.remote:smithi162:/home/ubuntu/cephtest/ceph.monmap is 379B 2020-11-28T21:45:36.295 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi162.front.sepia.ceph.com 2020-11-28T21:45:36.297 INFO:teuthology.orchestra.run.smithi162:> set -ex 2020-11-28T21:45:36.297 INFO:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/ceph/ceph.keyring 2020-11-28T21:45:36.298 INFO:teuthology.orchestra.run.smithi162:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-28T21:45:36.304 INFO:teuthology.orchestra.run.smithi162.stderr:+ sudo dd of=/etc/ceph/ceph.keyring 2020-11-28T21:45:36.313 INFO:teuthology.orchestra.run.smithi162.stderr:0+1 records in 2020-11-28T21:45:36.313 INFO:teuthology.orchestra.run.smithi162.stderr:0+1 records out 2020-11-28T21:45:36.314 INFO:teuthology.orchestra.run.smithi162.stderr:206 bytes copied, 0.00013359 s, 1.5 MB/s 2020-11-28T21:45:36.314 INFO:teuthology.orchestra.run.smithi162.stderr:+ sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-28T21:45:36.324 INFO:teuthology.orchestra.run.smithi162:> set -ex 2020-11-28T21:45:36.324 INFO:teuthology.orchestra.run.smithi162:> dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-28T21:45:36.371 INFO:teuthology.orchestra.run.smithi162.stderr:+ dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-28T21:45:36.373 INFO:teuthology.orchestra.run.smithi162.stderr:0+1 records in 2020-11-28T21:45:36.373 INFO:teuthology.orchestra.run.smithi162.stderr:0+1 records out 2020-11-28T21:45:36.373 INFO:teuthology.orchestra.run.smithi162.stderr:379 bytes copied, 0.000127663 s, 3.0 MB/s 2020-11-28T21:45:36.374 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi118.front.sepia.ceph.com 2020-11-28T21:45:36.374 INFO:teuthology.orchestra.run.smithi118:> set -ex 2020-11-28T21:45:36.375 INFO:teuthology.orchestra.run.smithi118:> sudo dd of=/etc/ceph/ceph.keyring 2020-11-28T21:45:36.375 INFO:teuthology.orchestra.run.smithi118:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-28T21:45:36.382 INFO:teuthology.orchestra.run.smithi118.stderr:+ sudo dd of=/etc/ceph/ceph.keyring 2020-11-28T21:45:36.391 INFO:teuthology.orchestra.run.smithi118.stderr:0+1 records in 2020-11-28T21:45:36.391 INFO:teuthology.orchestra.run.smithi118.stderr:0+1 records out 2020-11-28T21:45:36.392 INFO:teuthology.orchestra.run.smithi118.stderr:206 bytes copied, 6.8056e-05 s, 3.0 MB/s 2020-11-28T21:45:36.392 INFO:teuthology.orchestra.run.smithi118.stderr:+ sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-28T21:45:36.401 INFO:teuthology.orchestra.run.smithi118:> set -ex 2020-11-28T21:45:36.402 INFO:teuthology.orchestra.run.smithi118:> dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-28T21:45:36.447 INFO:teuthology.orchestra.run.smithi118.stderr:+ dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-28T21:45:36.448 INFO:teuthology.orchestra.run.smithi118.stderr:0+1 records in 2020-11-28T21:45:36.448 INFO:teuthology.orchestra.run.smithi118.stderr:0+1 records out 2020-11-28T21:45:36.448 INFO:teuthology.orchestra.run.smithi118.stderr:379 bytes copied, 8.0843e-05 s, 4.7 MB/s 2020-11-28T21:45:36.449 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi192.front.sepia.ceph.com 2020-11-28T21:45:36.449 INFO:teuthology.orchestra.run.smithi192:> set -ex 2020-11-28T21:45:36.449 INFO:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/ceph/ceph.keyring 2020-11-28T21:45:36.450 INFO:teuthology.orchestra.run.smithi192:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-28T21:45:36.455 INFO:teuthology.orchestra.run.smithi192.stderr:+ sudo dd of=/etc/ceph/ceph.keyring 2020-11-28T21:45:36.464 INFO:teuthology.orchestra.run.smithi192.stderr:0+1 records in 2020-11-28T21:45:36.464 INFO:teuthology.orchestra.run.smithi192.stderr:0+1 records out 2020-11-28T21:45:36.464 INFO:teuthology.orchestra.run.smithi192.stderr:206 bytes copied, 7.2324e-05 s, 2.8 MB/s 2020-11-28T21:45:36.465 INFO:teuthology.orchestra.run.smithi192.stderr:+ sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-28T21:45:36.473 INFO:teuthology.orchestra.run.smithi192:> set -ex 2020-11-28T21:45:36.474 INFO:teuthology.orchestra.run.smithi192:> dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-28T21:45:36.518 INFO:teuthology.orchestra.run.smithi192.stderr:+ dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-28T21:45:36.519 INFO:teuthology.orchestra.run.smithi192.stderr:0+1 records in 2020-11-28T21:45:36.520 INFO:teuthology.orchestra.run.smithi192.stderr:0+1 records out 2020-11-28T21:45:36.520 INFO:teuthology.orchestra.run.smithi192.stderr:379 bytes copied, 8.134e-05 s, 4.7 MB/s 2020-11-28T21:45:36.520 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi096.front.sepia.ceph.com 2020-11-28T21:45:36.520 INFO:teuthology.orchestra.run.smithi096:> set -ex 2020-11-28T21:45:36.521 INFO:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/ceph/ceph.keyring 2020-11-28T21:45:36.521 INFO:teuthology.orchestra.run.smithi096:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-28T21:45:36.526 INFO:teuthology.orchestra.run.smithi096.stderr:+ sudo dd of=/etc/ceph/ceph.keyring 2020-11-28T21:45:36.535 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records in 2020-11-28T21:45:36.535 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records out 2020-11-28T21:45:36.535 INFO:teuthology.orchestra.run.smithi096.stderr:206 bytes copied, 6.5656e-05 s, 3.1 MB/s 2020-11-28T21:45:36.536 INFO:teuthology.orchestra.run.smithi096.stderr:+ sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-28T21:45:36.545 INFO:teuthology.orchestra.run.smithi096:> set -ex 2020-11-28T21:45:36.545 INFO:teuthology.orchestra.run.smithi096:> dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-28T21:45:36.589 INFO:teuthology.orchestra.run.smithi096.stderr:+ dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-28T21:45:36.591 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records in 2020-11-28T21:45:36.591 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records out 2020-11-28T21:45:36.591 INFO:teuthology.orchestra.run.smithi096.stderr:379 bytes copied, 7.6097e-05 s, 5.0 MB/s 2020-11-28T21:45:36.592 INFO:tasks.ceph:Setting up mon nodes... 2020-11-28T21:45:36.592 INFO:tasks.ceph:Setting up mgr nodes... 2020-11-28T21:45:36.592 INFO:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2020-11-28T21:45:36.634 INFO:teuthology.orchestra.run.smithi162.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2020-11-28T21:45:36.636 INFO:teuthology.orchestra.run.smithi118:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2020-11-28T21:45:36.676 INFO:teuthology.orchestra.run.smithi118.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2020-11-28T21:45:36.679 INFO:tasks.ceph:Setting up mds nodes... 2020-11-28T21:45:36.679 INFO:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2020-11-28T21:45:36.724 INFO:teuthology.orchestra.run.smithi162.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2020-11-28T21:45:36.725 INFO:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2020-11-28T21:45:36.786 INFO:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2020-11-28T21:45:36.867 INFO:teuthology.orchestra.run.smithi162.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2020-11-28T21:45:36.869 INFO:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2020-11-28T21:45:36.926 INFO:teuthology.orchestra.run.smithi118:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2020-11-28T21:45:36.966 INFO:teuthology.orchestra.run.smithi118.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2020-11-28T21:45:36.969 INFO:teuthology.orchestra.run.smithi118:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2020-11-28T21:45:37.025 INFO:tasks.ceph_client:Setting up client nodes... 2020-11-28T21:45:37.025 INFO:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2020-11-28T21:45:37.058 INFO:teuthology.orchestra.run.smithi192.stdout:creating /etc/ceph/ceph.client.0.keyring 2020-11-28T21:45:37.070 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2020-11-28T21:45:37.102 INFO:teuthology.orchestra.run.smithi096.stdout:creating /etc/ceph/ceph.client.1.keyring 2020-11-28T21:45:37.113 INFO:tasks.ceph:Running mkfs on osd nodes... 2020-11-28T21:45:37.113 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi162.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi118.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2020-11-28T21:45:37.114 INFO:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2020-11-28T21:45:37.129 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-11-28T21:45:37.129 INFO:tasks.ceph:role: osd.0 2020-11-28T21:45:37.129 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi162.front.sepia.ceph.com 2020-11-28T21:45:37.130 INFO:teuthology.orchestra.run.smithi162:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2020-11-28T21:45:37.661 INFO:teuthology.orchestra.run.smithi162.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2020-11-28T21:45:37.662 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-28T21:45:37.662 INFO:teuthology.orchestra.run.smithi162.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-11-28T21:45:37.662 INFO:teuthology.orchestra.run.smithi162.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-28T21:45:37.662 INFO:teuthology.orchestra.run.smithi162.stdout: = sunit=0 swidth=0 blks 2020-11-28T21:45:37.663 INFO:teuthology.orchestra.run.smithi162.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-11-28T21:45:37.663 INFO:teuthology.orchestra.run.smithi162.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-28T21:45:37.663 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-28T21:45:37.663 INFO:teuthology.orchestra.run.smithi162.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-28T21:45:37.664 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi162.front.sepia.ceph.com -o noatime 2020-11-28T21:45:37.665 INFO:teuthology.orchestra.run.smithi162:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2020-11-28T21:45:37.740 INFO:teuthology.orchestra.run.smithi162:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2020-11-28T21:45:37.750 INFO:teuthology.orchestra.run.smithi162.stderr:sudo: /sbin/restorecon: command not found 2020-11-28T21:45:37.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:45:37.751 INFO:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2020-11-28T21:45:37.806 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-11-28T21:45:37.806 INFO:tasks.ceph:role: osd.1 2020-11-28T21:45:37.807 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi162.front.sepia.ceph.com 2020-11-28T21:45:37.807 INFO:teuthology.orchestra.run.smithi162:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2020-11-28T21:45:38.312 INFO:teuthology.orchestra.run.smithi162.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2020-11-28T21:45:38.312 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-28T21:45:38.313 INFO:teuthology.orchestra.run.smithi162.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-11-28T21:45:38.313 INFO:teuthology.orchestra.run.smithi162.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-28T21:45:38.313 INFO:teuthology.orchestra.run.smithi162.stdout: = sunit=0 swidth=0 blks 2020-11-28T21:45:38.313 INFO:teuthology.orchestra.run.smithi162.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-11-28T21:45:38.313 INFO:teuthology.orchestra.run.smithi162.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-28T21:45:38.314 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-28T21:45:38.314 INFO:teuthology.orchestra.run.smithi162.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-28T21:45:38.315 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi162.front.sepia.ceph.com -o noatime 2020-11-28T21:45:38.315 INFO:teuthology.orchestra.run.smithi162:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2020-11-28T21:45:38.382 INFO:teuthology.orchestra.run.smithi162:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2020-11-28T21:45:38.393 INFO:teuthology.orchestra.run.smithi162.stderr:sudo: /sbin/restorecon: command not found 2020-11-28T21:45:38.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:45:38.394 INFO:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2020-11-28T21:45:38.450 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-11-28T21:45:38.450 INFO:tasks.ceph:role: osd.2 2020-11-28T21:45:38.450 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi162.front.sepia.ceph.com 2020-11-28T21:45:38.451 INFO:teuthology.orchestra.run.smithi162:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2020-11-28T21:45:38.947 INFO:teuthology.orchestra.run.smithi162.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2020-11-28T21:45:38.947 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-28T21:45:38.948 INFO:teuthology.orchestra.run.smithi162.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-11-28T21:45:38.948 INFO:teuthology.orchestra.run.smithi162.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-28T21:45:38.949 INFO:teuthology.orchestra.run.smithi162.stdout: = sunit=0 swidth=0 blks 2020-11-28T21:45:38.949 INFO:teuthology.orchestra.run.smithi162.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-11-28T21:45:38.949 INFO:teuthology.orchestra.run.smithi162.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-28T21:45:38.950 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-28T21:45:38.950 INFO:teuthology.orchestra.run.smithi162.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-28T21:45:38.952 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi162.front.sepia.ceph.com -o noatime 2020-11-28T21:45:38.952 INFO:teuthology.orchestra.run.smithi162:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2020-11-28T21:45:39.011 INFO:teuthology.orchestra.run.smithi162:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2020-11-28T21:45:39.022 INFO:teuthology.orchestra.run.smithi162.stderr:sudo: /sbin/restorecon: command not found 2020-11-28T21:45:39.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:45:39.024 INFO:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2020-11-28T21:45:39.082 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-11-28T21:45:39.082 INFO:tasks.ceph:role: osd.3 2020-11-28T21:45:39.082 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi162.front.sepia.ceph.com 2020-11-28T21:45:39.083 INFO:teuthology.orchestra.run.smithi162:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2020-11-28T21:45:39.610 INFO:teuthology.orchestra.run.smithi162.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2020-11-28T21:45:39.610 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-28T21:45:39.611 INFO:teuthology.orchestra.run.smithi162.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-11-28T21:45:39.611 INFO:teuthology.orchestra.run.smithi162.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-28T21:45:39.612 INFO:teuthology.orchestra.run.smithi162.stdout: = sunit=0 swidth=0 blks 2020-11-28T21:45:39.612 INFO:teuthology.orchestra.run.smithi162.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-11-28T21:45:39.612 INFO:teuthology.orchestra.run.smithi162.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-28T21:45:39.613 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-28T21:45:39.613 INFO:teuthology.orchestra.run.smithi162.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-28T21:45:39.615 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi162.front.sepia.ceph.com -o noatime 2020-11-28T21:45:39.615 INFO:teuthology.orchestra.run.smithi162:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2020-11-28T21:45:39.657 INFO:teuthology.orchestra.run.smithi162:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2020-11-28T21:45:39.710 INFO:teuthology.orchestra.run.smithi162.stderr:sudo: /sbin/restorecon: command not found 2020-11-28T21:45:39.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:45:39.712 INFO:teuthology.orchestra.run.smithi162:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-28T21:45:39.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:45:39.793+0000 7f349e144d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2020-11-28T21:45:39.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:45:39.793+0000 7f349e144d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2020-11-28T21:45:39.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:45:39.793+0000 7f349e144d80 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-11-28T21:45:39.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:45:39.793+0000 7f349e144d80 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-11-28T21:45:39.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:45:39.793+0000 7f349e144d80 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-11-28T21:45:39.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:45:39.801+0000 7f349e144d80 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2020-11-28T21:45:39.843 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:45:39.837+0000 7f349e144d80 -1 freelist read_size_meta_from_db missing size meta in DB 2020-11-28T21:45:42.246 INFO:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2020-11-28T21:45:42.302 INFO:teuthology.orchestra.run.smithi162:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-28T21:45:42.379 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:45:42.377+0000 7fd1eadfcd80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2020-11-28T21:45:42.380 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:45:42.377+0000 7fd1eadfcd80 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2020-11-28T21:45:42.380 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:45:42.377+0000 7fd1eadfcd80 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-11-28T21:45:42.380 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:45:42.377+0000 7fd1eadfcd80 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-11-28T21:45:42.381 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:45:42.377+0000 7fd1eadfcd80 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-11-28T21:45:42.381 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:45:42.377+0000 7fd1eadfcd80 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2020-11-28T21:45:42.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:45:42.393+0000 7fd1eadfcd80 -1 freelist read_size_meta_from_db missing size meta in DB 2020-11-28T21:45:44.837 INFO:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2020-11-28T21:45:44.894 INFO:teuthology.orchestra.run.smithi162:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-28T21:45:44.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:45:44.969+0000 7efce31cad80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2020-11-28T21:45:44.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:45:44.969+0000 7efce31cad80 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2020-11-28T21:45:44.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:45:44.969+0000 7efce31cad80 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-11-28T21:45:44.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:45:44.969+0000 7efce31cad80 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-11-28T21:45:44.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:45:44.969+0000 7efce31cad80 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-11-28T21:45:44.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:45:44.969+0000 7efce31cad80 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2020-11-28T21:45:44.993 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:45:44.989+0000 7efce31cad80 -1 freelist read_size_meta_from_db missing size meta in DB 2020-11-28T21:45:47.426 INFO:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2020-11-28T21:45:47.480 INFO:teuthology.orchestra.run.smithi162:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-28T21:45:47.560 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:45:47.557+0000 7f922a6afd80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2020-11-28T21:45:47.561 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:45:47.557+0000 7f922a6afd80 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2020-11-28T21:45:47.561 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:45:47.557+0000 7f922a6afd80 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2020-11-28T21:45:47.561 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:45:47.557+0000 7f922a6afd80 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2020-11-28T21:45:47.562 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:45:47.557+0000 7f922a6afd80 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2020-11-28T21:45:47.562 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:45:47.557+0000 7f922a6afd80 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2020-11-28T21:45:47.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:45:47.577+0000 7f922a6afd80 -1 freelist read_size_meta_from_db missing size meta in DB 2020-11-28T21:45:50.010 INFO:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2020-11-28T21:45:50.066 INFO:teuthology.orchestra.run.smithi118:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2020-11-28T21:45:50.082 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-11-28T21:45:50.083 INFO:tasks.ceph:role: osd.4 2020-11-28T21:45:50.083 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi118.front.sepia.ceph.com 2020-11-28T21:45:50.084 INFO:teuthology.orchestra.run.smithi118:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2020-11-28T21:45:50.164 INFO:teuthology.orchestra.run.smithi118.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2020-11-28T21:45:50.165 INFO:teuthology.orchestra.run.smithi118.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-28T21:45:50.165 INFO:teuthology.orchestra.run.smithi118.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-11-28T21:45:50.165 INFO:teuthology.orchestra.run.smithi118.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-28T21:45:50.165 INFO:teuthology.orchestra.run.smithi118.stdout: = sunit=0 swidth=0 blks 2020-11-28T21:45:50.166 INFO:teuthology.orchestra.run.smithi118.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-11-28T21:45:50.166 INFO:teuthology.orchestra.run.smithi118.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-28T21:45:50.166 INFO:teuthology.orchestra.run.smithi118.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-28T21:45:50.166 INFO:teuthology.orchestra.run.smithi118.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-28T21:45:50.167 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi118.front.sepia.ceph.com -o noatime 2020-11-28T21:45:50.168 INFO:teuthology.orchestra.run.smithi118:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2020-11-28T21:45:50.229 INFO:teuthology.orchestra.run.smithi118:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2020-11-28T21:45:50.284 INFO:teuthology.orchestra.run.smithi118.stderr:sudo: /sbin/restorecon: command not found 2020-11-28T21:45:50.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:45:50.286 INFO:teuthology.orchestra.run.smithi118:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2020-11-28T21:45:50.343 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-11-28T21:45:50.344 INFO:tasks.ceph:role: osd.5 2020-11-28T21:45:50.344 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi118.front.sepia.ceph.com 2020-11-28T21:45:50.344 INFO:teuthology.orchestra.run.smithi118:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2020-11-28T21:45:50.426 INFO:teuthology.orchestra.run.smithi118.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2020-11-28T21:45:50.426 INFO:teuthology.orchestra.run.smithi118.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-28T21:45:50.426 INFO:teuthology.orchestra.run.smithi118.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-11-28T21:45:50.427 INFO:teuthology.orchestra.run.smithi118.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-28T21:45:50.427 INFO:teuthology.orchestra.run.smithi118.stdout: = sunit=0 swidth=0 blks 2020-11-28T21:45:50.427 INFO:teuthology.orchestra.run.smithi118.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-11-28T21:45:50.427 INFO:teuthology.orchestra.run.smithi118.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-28T21:45:50.429 INFO:teuthology.orchestra.run.smithi118.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-28T21:45:50.429 INFO:teuthology.orchestra.run.smithi118.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-28T21:45:50.430 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi118.front.sepia.ceph.com -o noatime 2020-11-28T21:45:50.431 INFO:teuthology.orchestra.run.smithi118:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2020-11-28T21:45:50.493 INFO:teuthology.orchestra.run.smithi118:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2020-11-28T21:45:50.549 INFO:teuthology.orchestra.run.smithi118.stderr:sudo: /sbin/restorecon: command not found 2020-11-28T21:45:50.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:45:50.552 INFO:teuthology.orchestra.run.smithi118:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2020-11-28T21:45:50.608 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-11-28T21:45:50.608 INFO:tasks.ceph:role: osd.6 2020-11-28T21:45:50.608 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi118.front.sepia.ceph.com 2020-11-28T21:45:50.609 INFO:teuthology.orchestra.run.smithi118:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2020-11-28T21:45:50.692 INFO:teuthology.orchestra.run.smithi118.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2020-11-28T21:45:50.692 INFO:teuthology.orchestra.run.smithi118.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-28T21:45:50.693 INFO:teuthology.orchestra.run.smithi118.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-11-28T21:45:50.693 INFO:teuthology.orchestra.run.smithi118.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-28T21:45:50.693 INFO:teuthology.orchestra.run.smithi118.stdout: = sunit=0 swidth=0 blks 2020-11-28T21:45:50.693 INFO:teuthology.orchestra.run.smithi118.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-11-28T21:45:50.693 INFO:teuthology.orchestra.run.smithi118.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-28T21:45:50.694 INFO:teuthology.orchestra.run.smithi118.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-28T21:45:50.694 INFO:teuthology.orchestra.run.smithi118.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-28T21:45:50.695 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi118.front.sepia.ceph.com -o noatime 2020-11-28T21:45:50.695 INFO:teuthology.orchestra.run.smithi118:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2020-11-28T21:45:50.756 INFO:teuthology.orchestra.run.smithi118:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2020-11-28T21:45:50.808 INFO:teuthology.orchestra.run.smithi118.stderr:sudo: /sbin/restorecon: command not found 2020-11-28T21:45:50.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:45:50.811 INFO:teuthology.orchestra.run.smithi118:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2020-11-28T21:45:50.868 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-11-28T21:45:50.868 INFO:tasks.ceph:role: osd.7 2020-11-28T21:45:50.869 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi118.front.sepia.ceph.com 2020-11-28T21:45:50.869 INFO:teuthology.orchestra.run.smithi118:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2020-11-28T21:45:50.950 INFO:teuthology.orchestra.run.smithi118.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2020-11-28T21:45:50.950 INFO:teuthology.orchestra.run.smithi118.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-28T21:45:50.951 INFO:teuthology.orchestra.run.smithi118.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-11-28T21:45:50.951 INFO:teuthology.orchestra.run.smithi118.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-28T21:45:50.951 INFO:teuthology.orchestra.run.smithi118.stdout: = sunit=0 swidth=0 blks 2020-11-28T21:45:50.952 INFO:teuthology.orchestra.run.smithi118.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-11-28T21:45:50.953 INFO:teuthology.orchestra.run.smithi118.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-28T21:45:50.953 INFO:teuthology.orchestra.run.smithi118.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-28T21:45:50.953 INFO:teuthology.orchestra.run.smithi118.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-28T21:45:50.954 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi118.front.sepia.ceph.com -o noatime 2020-11-28T21:45:50.955 INFO:teuthology.orchestra.run.smithi118:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2020-11-28T21:45:51.016 INFO:teuthology.orchestra.run.smithi118:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2020-11-28T21:45:51.067 INFO:teuthology.orchestra.run.smithi118.stderr:sudo: /sbin/restorecon: command not found 2020-11-28T21:45:51.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:45:51.068 INFO:teuthology.orchestra.run.smithi118:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-28T21:45:51.161 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:45:51.159+0000 7faf3eb66d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2020-11-28T21:45:51.162 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:45:51.159+0000 7faf3eb66d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2020-11-28T21:45:51.162 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:45:51.159+0000 7faf3eb66d80 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2020-11-28T21:45:51.162 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:45:51.159+0000 7faf3eb66d80 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2020-11-28T21:45:51.162 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:45:51.159+0000 7faf3eb66d80 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2020-11-28T21:45:51.163 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:45:51.159+0000 7faf3eb66d80 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2020-11-28T21:45:51.175 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:45:51.175+0000 7faf3eb66d80 -1 freelist read_size_meta_from_db missing size meta in DB 2020-11-28T21:45:53.596 INFO:teuthology.orchestra.run.smithi118:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2020-11-28T21:45:53.610 INFO:teuthology.orchestra.run.smithi118:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-28T21:45:53.695 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:45:53.695+0000 7f385693ed80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2020-11-28T21:45:53.695 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:45:53.695+0000 7f385693ed80 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2020-11-28T21:45:53.695 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:45:53.695+0000 7f385693ed80 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2020-11-28T21:45:53.696 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:45:53.695+0000 7f385693ed80 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2020-11-28T21:45:53.696 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:45:53.695+0000 7f385693ed80 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2020-11-28T21:45:53.696 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:45:53.695+0000 7f385693ed80 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2020-11-28T21:45:53.706 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:45:53.707+0000 7f385693ed80 -1 freelist read_size_meta_from_db missing size meta in DB 2020-11-28T21:45:56.136 INFO:teuthology.orchestra.run.smithi118:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2020-11-28T21:45:56.193 INFO:teuthology.orchestra.run.smithi118:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-28T21:45:56.275 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:45:56.275+0000 7f3cfc865d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2020-11-28T21:45:56.276 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:45:56.275+0000 7f3cfc865d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2020-11-28T21:45:56.276 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:45:56.275+0000 7f3cfc865d80 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2020-11-28T21:45:56.276 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:45:56.275+0000 7f3cfc865d80 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2020-11-28T21:45:56.276 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:45:56.275+0000 7f3cfc865d80 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2020-11-28T21:45:56.277 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:45:56.275+0000 7f3cfc865d80 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2020-11-28T21:45:56.287 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:45:56.287+0000 7f3cfc865d80 -1 freelist read_size_meta_from_db missing size meta in DB 2020-11-28T21:45:58.707 INFO:teuthology.orchestra.run.smithi118:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2020-11-28T21:45:58.765 INFO:teuthology.orchestra.run.smithi118:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-28T21:45:58.843 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:45:58.843+0000 7f98d04ffd80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2020-11-28T21:45:58.843 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:45:58.843+0000 7f98d04ffd80 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2020-11-28T21:45:58.843 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:45:58.843+0000 7f98d04ffd80 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2020-11-28T21:45:58.843 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:45:58.843+0000 7f98d04ffd80 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2020-11-28T21:45:58.844 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:45:58.843+0000 7f98d04ffd80 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2020-11-28T21:45:58.844 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:45:58.843+0000 7f98d04ffd80 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2020-11-28T21:45:58.853 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:45:58.855+0000 7f98d04ffd80 -1 freelist read_size_meta_from_db missing size meta in DB 2020-11-28T21:46:01.274 INFO:teuthology.orchestra.run.smithi118:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2020-11-28T21:46:01.330 INFO:tasks.ceph:Reading keys from all nodes... 2020-11-28T21:46:01.331 INFO:teuthology.orchestra.run.smithi162:> mktemp 2020-11-28T21:46:01.337 INFO:teuthology.orchestra.run.smithi162.stdout:/tmp/tmp.tdIWZbONDB 2020-11-28T21:46:01.338 INFO:teuthology.orchestra.run.smithi162:> sudo cp /var/lib/ceph/mgr/ceph-y/keyring /tmp/tmp.tdIWZbONDB 2020-11-28T21:46:01.394 INFO:teuthology.orchestra.run.smithi162:> sudo chmod 0666 /tmp/tmp.tdIWZbONDB 2020-11-28T21:46:01.498 DEBUG:teuthology.orchestra.remote:smithi162:/tmp/tmp.tdIWZbONDB is 56B 2020-11-28T21:46:01.509 INFO:teuthology.orchestra.run.smithi162:> rm -fr /tmp/tmp.tdIWZbONDB 2020-11-28T21:46:01.515 INFO:teuthology.orchestra.run.smithi162:> mktemp 2020-11-28T21:46:01.564 INFO:teuthology.orchestra.run.smithi162.stdout:/tmp/tmp.lA1WBwa2FN 2020-11-28T21:46:01.565 INFO:teuthology.orchestra.run.smithi162:> sudo cp /var/lib/ceph/mds/ceph-a/keyring /tmp/tmp.lA1WBwa2FN 2020-11-28T21:46:01.621 INFO:teuthology.orchestra.run.smithi162:> sudo chmod 0666 /tmp/tmp.lA1WBwa2FN 2020-11-28T21:46:01.730 DEBUG:teuthology.orchestra.remote:smithi162:/tmp/tmp.lA1WBwa2FN is 56B 2020-11-28T21:46:01.742 INFO:teuthology.orchestra.run.smithi162:> rm -fr /tmp/tmp.lA1WBwa2FN 2020-11-28T21:46:01.749 INFO:teuthology.orchestra.run.smithi162:> mktemp 2020-11-28T21:46:01.796 INFO:teuthology.orchestra.run.smithi162.stdout:/tmp/tmp.TevLDp5eIi 2020-11-28T21:46:01.797 INFO:teuthology.orchestra.run.smithi162:> sudo cp /var/lib/ceph/mds/ceph-c/keyring /tmp/tmp.TevLDp5eIi 2020-11-28T21:46:01.853 INFO:teuthology.orchestra.run.smithi162:> sudo chmod 0666 /tmp/tmp.TevLDp5eIi 2020-11-28T21:46:01.962 DEBUG:teuthology.orchestra.remote:smithi162:/tmp/tmp.TevLDp5eIi is 56B 2020-11-28T21:46:01.973 INFO:teuthology.orchestra.run.smithi162:> rm -fr /tmp/tmp.TevLDp5eIi 2020-11-28T21:46:01.978 INFO:teuthology.orchestra.run.smithi162:> mktemp 2020-11-28T21:46:02.024 INFO:teuthology.orchestra.run.smithi162.stdout:/tmp/tmp.ljcaGcDIVN 2020-11-28T21:46:02.024 INFO:teuthology.orchestra.run.smithi162:> sudo cp /var/lib/ceph/osd/ceph-0/keyring /tmp/tmp.ljcaGcDIVN 2020-11-28T21:46:02.081 INFO:teuthology.orchestra.run.smithi162:> sudo chmod 0666 /tmp/tmp.ljcaGcDIVN 2020-11-28T21:46:02.186 DEBUG:teuthology.orchestra.remote:smithi162:/tmp/tmp.ljcaGcDIVN is 56B 2020-11-28T21:46:02.200 INFO:teuthology.orchestra.run.smithi162:> rm -fr /tmp/tmp.ljcaGcDIVN 2020-11-28T21:46:02.206 INFO:teuthology.orchestra.run.smithi162:> mktemp 2020-11-28T21:46:02.252 INFO:teuthology.orchestra.run.smithi162.stdout:/tmp/tmp.bcknjnAbcF 2020-11-28T21:46:02.252 INFO:teuthology.orchestra.run.smithi162:> sudo cp /var/lib/ceph/osd/ceph-1/keyring /tmp/tmp.bcknjnAbcF 2020-11-28T21:46:02.310 INFO:teuthology.orchestra.run.smithi162:> sudo chmod 0666 /tmp/tmp.bcknjnAbcF 2020-11-28T21:46:02.415 DEBUG:teuthology.orchestra.remote:smithi162:/tmp/tmp.bcknjnAbcF is 56B 2020-11-28T21:46:02.471 INFO:teuthology.orchestra.run.smithi162:> rm -fr /tmp/tmp.bcknjnAbcF 2020-11-28T21:46:02.478 INFO:teuthology.orchestra.run.smithi162:> mktemp 2020-11-28T21:46:02.528 INFO:teuthology.orchestra.run.smithi162.stdout:/tmp/tmp.pQok0XfdOt 2020-11-28T21:46:02.529 INFO:teuthology.orchestra.run.smithi162:> sudo cp /var/lib/ceph/osd/ceph-2/keyring /tmp/tmp.pQok0XfdOt 2020-11-28T21:46:02.586 INFO:teuthology.orchestra.run.smithi162:> sudo chmod 0666 /tmp/tmp.pQok0XfdOt 2020-11-28T21:46:02.691 DEBUG:teuthology.orchestra.remote:smithi162:/tmp/tmp.pQok0XfdOt is 56B 2020-11-28T21:46:02.702 INFO:teuthology.orchestra.run.smithi162:> rm -fr /tmp/tmp.pQok0XfdOt 2020-11-28T21:46:02.709 INFO:teuthology.orchestra.run.smithi162:> mktemp 2020-11-28T21:46:02.756 INFO:teuthology.orchestra.run.smithi162.stdout:/tmp/tmp.84L284sg7I 2020-11-28T21:46:02.757 INFO:teuthology.orchestra.run.smithi162:> sudo cp /var/lib/ceph/osd/ceph-3/keyring /tmp/tmp.84L284sg7I 2020-11-28T21:46:02.814 INFO:teuthology.orchestra.run.smithi162:> sudo chmod 0666 /tmp/tmp.84L284sg7I 2020-11-28T21:46:02.918 DEBUG:teuthology.orchestra.remote:smithi162:/tmp/tmp.84L284sg7I is 56B 2020-11-28T21:46:02.929 INFO:teuthology.orchestra.run.smithi162:> rm -fr /tmp/tmp.84L284sg7I 2020-11-28T21:46:02.936 INFO:teuthology.orchestra.run.smithi118:> mktemp 2020-11-28T21:46:02.943 INFO:teuthology.orchestra.run.smithi118.stdout:/tmp/tmp.b0neYUh57S 2020-11-28T21:46:02.945 INFO:teuthology.orchestra.run.smithi118:> sudo cp /var/lib/ceph/mgr/ceph-x/keyring /tmp/tmp.b0neYUh57S 2020-11-28T21:46:03.000 INFO:teuthology.orchestra.run.smithi118:> sudo chmod 0666 /tmp/tmp.b0neYUh57S 2020-11-28T21:46:03.106 DEBUG:teuthology.orchestra.remote:smithi118:/tmp/tmp.b0neYUh57S is 56B 2020-11-28T21:46:03.160 INFO:teuthology.orchestra.run.smithi118:> rm -fr /tmp/tmp.b0neYUh57S 2020-11-28T21:46:03.167 INFO:teuthology.orchestra.run.smithi118:> mktemp 2020-11-28T21:46:03.214 INFO:teuthology.orchestra.run.smithi118.stdout:/tmp/tmp.5ugK1oIZm1 2020-11-28T21:46:03.215 INFO:teuthology.orchestra.run.smithi118:> sudo cp /var/lib/ceph/mds/ceph-b/keyring /tmp/tmp.5ugK1oIZm1 2020-11-28T21:46:03.274 INFO:teuthology.orchestra.run.smithi118:> sudo chmod 0666 /tmp/tmp.5ugK1oIZm1 2020-11-28T21:46:03.382 DEBUG:teuthology.orchestra.remote:smithi118:/tmp/tmp.5ugK1oIZm1 is 56B 2020-11-28T21:46:03.434 INFO:teuthology.orchestra.run.smithi118:> rm -fr /tmp/tmp.5ugK1oIZm1 2020-11-28T21:46:03.441 INFO:teuthology.orchestra.run.smithi118:> mktemp 2020-11-28T21:46:03.490 INFO:teuthology.orchestra.run.smithi118.stdout:/tmp/tmp.zJ07DNUKIK 2020-11-28T21:46:03.491 INFO:teuthology.orchestra.run.smithi118:> sudo cp /var/lib/ceph/osd/ceph-4/keyring /tmp/tmp.zJ07DNUKIK 2020-11-28T21:46:03.548 INFO:teuthology.orchestra.run.smithi118:> sudo chmod 0666 /tmp/tmp.zJ07DNUKIK 2020-11-28T21:46:03.766 DEBUG:teuthology.orchestra.remote:smithi118:/tmp/tmp.zJ07DNUKIK is 56B 2020-11-28T21:46:03.819 INFO:teuthology.orchestra.run.smithi118:> rm -fr /tmp/tmp.zJ07DNUKIK 2020-11-28T21:46:03.826 INFO:teuthology.orchestra.run.smithi118:> mktemp 2020-11-28T21:46:03.874 INFO:teuthology.orchestra.run.smithi118.stdout:/tmp/tmp.pfdHENHYVY 2020-11-28T21:46:03.875 INFO:teuthology.orchestra.run.smithi118:> sudo cp /var/lib/ceph/osd/ceph-5/keyring /tmp/tmp.pfdHENHYVY 2020-11-28T21:46:03.932 INFO:teuthology.orchestra.run.smithi118:> sudo chmod 0666 /tmp/tmp.pfdHENHYVY 2020-11-28T21:46:04.039 DEBUG:teuthology.orchestra.remote:smithi118:/tmp/tmp.pfdHENHYVY is 56B 2020-11-28T21:46:04.050 INFO:teuthology.orchestra.run.smithi118:> rm -fr /tmp/tmp.pfdHENHYVY 2020-11-28T21:46:04.056 INFO:teuthology.orchestra.run.smithi118:> mktemp 2020-11-28T21:46:04.102 INFO:teuthology.orchestra.run.smithi118.stdout:/tmp/tmp.Mb0vQGZJKt 2020-11-28T21:46:04.103 INFO:teuthology.orchestra.run.smithi118:> sudo cp /var/lib/ceph/osd/ceph-6/keyring /tmp/tmp.Mb0vQGZJKt 2020-11-28T21:46:04.159 INFO:teuthology.orchestra.run.smithi118:> sudo chmod 0666 /tmp/tmp.Mb0vQGZJKt 2020-11-28T21:46:04.266 DEBUG:teuthology.orchestra.remote:smithi118:/tmp/tmp.Mb0vQGZJKt is 56B 2020-11-28T21:46:04.278 INFO:teuthology.orchestra.run.smithi118:> rm -fr /tmp/tmp.Mb0vQGZJKt 2020-11-28T21:46:04.285 INFO:teuthology.orchestra.run.smithi118:> mktemp 2020-11-28T21:46:04.330 INFO:teuthology.orchestra.run.smithi118.stdout:/tmp/tmp.839TfayKQu 2020-11-28T21:46:04.331 INFO:teuthology.orchestra.run.smithi118:> sudo cp /var/lib/ceph/osd/ceph-7/keyring /tmp/tmp.839TfayKQu 2020-11-28T21:46:04.388 INFO:teuthology.orchestra.run.smithi118:> sudo chmod 0666 /tmp/tmp.839TfayKQu 2020-11-28T21:46:04.497 DEBUG:teuthology.orchestra.remote:smithi118:/tmp/tmp.839TfayKQu is 56B 2020-11-28T21:46:04.552 INFO:teuthology.orchestra.run.smithi118:> rm -fr /tmp/tmp.839TfayKQu 2020-11-28T21:46:04.568 DEBUG:teuthology.orchestra.remote:smithi192:/etc/ceph/ceph.client.0.keyring is 59B 2020-11-28T21:46:04.589 DEBUG:teuthology.orchestra.remote:smithi096:/etc/ceph/ceph.client.1.keyring is 59B 2020-11-28T21:46:04.601 INFO:tasks.ceph:Adding keys to all mons... 2020-11-28T21:46:04.602 INFO:teuthology.orchestra.run.smithi118:> sudo tee -a /etc/ceph/ceph.keyring 2020-11-28T21:46:04.605 INFO:teuthology.orchestra.run.smithi162:> sudo tee -a /etc/ceph/ceph.keyring 2020-11-28T21:46:04.616 INFO:teuthology.orchestra.run.smithi118.stdout:[mgr.y] 2020-11-28T21:46:04.617 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQAAxcJfs8fvJRAAP0Atp074EeMHiTtpCCVHuA== 2020-11-28T21:46:04.617 INFO:teuthology.orchestra.run.smithi118.stdout:[mds.a] 2020-11-28T21:46:04.618 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQAAxcJfWxUpKxAAIUDwQRKwz8H4Qk9qE356Bg== 2020-11-28T21:46:04.618 INFO:teuthology.orchestra.run.smithi118.stdout:[mds.c] 2020-11-28T21:46:04.619 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQAAxcJfdQbRMxAAWlcjesxTZzZnvKCQneSjjQ== 2020-11-28T21:46:04.619 INFO:teuthology.orchestra.run.smithi118.stdout:[osd.0] 2020-11-28T21:46:04.620 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQADxcJfF3WLLxAAxsB5YPjH1yjv/vhlRbijqw== 2020-11-28T21:46:04.620 INFO:teuthology.orchestra.run.smithi118.stdout:[osd.1] 2020-11-28T21:46:04.620 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQAGxcJfKYbBFhAA1x4DzFtkQQv6uWLIdOi1qQ== 2020-11-28T21:46:04.621 INFO:teuthology.orchestra.run.smithi118.stdout:[osd.2] 2020-11-28T21:46:04.621 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQAIxcJfvcYUOhAASENjM0HfAxxyFe13mmE36Q== 2020-11-28T21:46:04.622 INFO:teuthology.orchestra.run.smithi118.stdout:[osd.3] 2020-11-28T21:46:04.622 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQALxcJfc/OHIRAAFYx3s8Ssttt15hjoH04tow== 2020-11-28T21:46:04.623 INFO:teuthology.orchestra.run.smithi118.stdout:[mgr.x] 2020-11-28T21:46:04.623 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQAAxcJfZhdvKBAAT7zCCsD2OVCHjvddGLxQ/A== 2020-11-28T21:46:04.623 INFO:teuthology.orchestra.run.smithi118.stdout:[mds.b] 2020-11-28T21:46:04.624 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQAAxcJfKY2zORAAaO65etMZiYcaTDNX0nsEPw== 2020-11-28T21:46:04.624 INFO:teuthology.orchestra.run.smithi118.stdout:[osd.4] 2020-11-28T21:46:04.625 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQAPxcJffGWqCRAAb8CCN5oUPkdDh6BJH5HEnw== 2020-11-28T21:46:04.625 INFO:teuthology.orchestra.run.smithi118.stdout:[osd.5] 2020-11-28T21:46:04.626 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQARxcJfo8GKKRAAfbsCDb4L2wULiKmHyzu7/A== 2020-11-28T21:46:04.626 INFO:teuthology.orchestra.run.smithi118.stdout:[osd.6] 2020-11-28T21:46:04.626 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQAUxcJf8Y2MEBAAqQX6rX3LK0zodlI2wK4JrA== 2020-11-28T21:46:04.627 INFO:teuthology.orchestra.run.smithi118.stdout:[osd.7] 2020-11-28T21:46:04.627 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQAWxcJfCLBdMhAAf8QixQMQDrfZOGWN8rHe5g== 2020-11-28T21:46:04.628 INFO:teuthology.orchestra.run.smithi118.stdout:[client.0] 2020-11-28T21:46:04.628 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQABxcJfS9vDAxAAUk4FU8FVt4if/CRH5ZisOA== 2020-11-28T21:46:04.629 INFO:teuthology.orchestra.run.smithi118.stdout:[client.1] 2020-11-28T21:46:04.629 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQABxcJfuCU8BhAAXUSoSK3D0XRo58uHNzcjXg== 2020-11-28T21:46:04.632 INFO:teuthology.orchestra.run.smithi162.stdout:[mgr.y] 2020-11-28T21:46:04.632 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAAxcJfs8fvJRAAP0Atp074EeMHiTtpCCVHuA== 2020-11-28T21:46:04.633 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.a] 2020-11-28T21:46:04.633 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAAxcJfWxUpKxAAIUDwQRKwz8H4Qk9qE356Bg== 2020-11-28T21:46:04.633 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.c] 2020-11-28T21:46:04.634 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAAxcJfdQbRMxAAWlcjesxTZzZnvKCQneSjjQ== 2020-11-28T21:46:04.634 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.0] 2020-11-28T21:46:04.635 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQADxcJfF3WLLxAAxsB5YPjH1yjv/vhlRbijqw== 2020-11-28T21:46:04.635 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.1] 2020-11-28T21:46:04.636 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAGxcJfKYbBFhAA1x4DzFtkQQv6uWLIdOi1qQ== 2020-11-28T21:46:04.636 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.2] 2020-11-28T21:46:04.636 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAIxcJfvcYUOhAASENjM0HfAxxyFe13mmE36Q== 2020-11-28T21:46:04.637 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.3] 2020-11-28T21:46:04.637 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQALxcJfc/OHIRAAFYx3s8Ssttt15hjoH04tow== 2020-11-28T21:46:04.638 INFO:teuthology.orchestra.run.smithi162.stdout:[mgr.x] 2020-11-28T21:46:04.638 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAAxcJfZhdvKBAAT7zCCsD2OVCHjvddGLxQ/A== 2020-11-28T21:46:04.639 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.b] 2020-11-28T21:46:04.639 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAAxcJfKY2zORAAaO65etMZiYcaTDNX0nsEPw== 2020-11-28T21:46:04.639 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.4] 2020-11-28T21:46:04.640 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAPxcJffGWqCRAAb8CCN5oUPkdDh6BJH5HEnw== 2020-11-28T21:46:04.640 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.5] 2020-11-28T21:46:04.641 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQARxcJfo8GKKRAAfbsCDb4L2wULiKmHyzu7/A== 2020-11-28T21:46:04.641 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.6] 2020-11-28T21:46:04.642 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAUxcJf8Y2MEBAAqQX6rX3LK0zodlI2wK4JrA== 2020-11-28T21:46:04.642 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.7] 2020-11-28T21:46:04.642 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAWxcJfCLBdMhAAf8QixQMQDrfZOGWN8rHe5g== 2020-11-28T21:46:04.643 INFO:teuthology.orchestra.run.smithi162.stdout:[client.0] 2020-11-28T21:46:04.643 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQABxcJfS9vDAxAAUk4FU8FVt4if/CRH5ZisOA== 2020-11-28T21:46:04.644 INFO:teuthology.orchestra.run.smithi162.stdout:[client.1] 2020-11-28T21:46:04.644 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQABxcJfuCU8BhAAXUSoSK3D0XRo58uHNzcjXg== 2020-11-28T21:46:04.645 INFO:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-11-28T21:46:04.675 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-11-28T21:46:04.718 INFO:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-28T21:46:04.760 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-28T21:46:04.831 INFO:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-28T21:46:04.847 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-28T21:46:04.893 INFO:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-28T21:46:04.935 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-28T21:46:04.964 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:46:04.969 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:46:04.977 INFO:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-28T21:46:05.018 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:46:05.021 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-28T21:46:05.022 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:46:05.058 INFO:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-28T21:46:05.114 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-28T21:46:05.159 INFO:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-28T21:46:05.198 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-28T21:46:05.243 INFO:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-11-28T21:46:05.283 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-11-28T21:46:05.329 INFO:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-28T21:46:05.367 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-28T21:46:05.414 INFO:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-28T21:46:05.456 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-28T21:46:05.506 INFO:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-28T21:46:05.543 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-28T21:46:05.591 INFO:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-28T21:46:05.627 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-28T21:46:05.679 INFO:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-28T21:46:05.711 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-28T21:46:05.764 INFO:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-28T21:46:05.795 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-28T21:46:05.851 INFO:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-28T21:46:05.879 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-28T21:46:05.938 INFO:tasks.ceph:Running mkfs on mon nodes... 2020-11-28T21:46:05.939 INFO:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2020-11-28T21:46:05.993 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2020-11-28T21:46:06.106 INFO:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2020-11-28T21:46:06.122 INFO:teuthology.orchestra.run.smithi118:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2020-11-28T21:46:06.138 INFO:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2020-11-28T21:46:06.257 INFO:teuthology.orchestra.run.smithi118:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2020-11-28T21:46:06.270 INFO:teuthology.orchestra.run.smithi118:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2020-11-28T21:46:06.327 INFO:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2020-11-28T21:46:06.446 INFO:teuthology.orchestra.run.smithi118:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2020-11-28T21:46:06.461 INFO:teuthology.orchestra.run.smithi118:> rm -- /home/ubuntu/cephtest/ceph.monmap 2020-11-28T21:46:06.506 INFO:teuthology.orchestra.run.smithi162:> rm -- /home/ubuntu/cephtest/ceph.monmap 2020-11-28T21:46:06.512 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2020-11-28T21:46:06.513 INFO:tasks.ceph.mon.a:Restarting daemon 2020-11-28T21:46:06.513 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2020-11-28T21:46:06.556 INFO:tasks.ceph.mon.a:Started 2020-11-28T21:46:06.556 INFO:tasks.ceph.mon.b:Restarting daemon 2020-11-28T21:46:06.557 INFO:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2020-11-28T21:46:06.559 INFO:tasks.ceph.mon.b:Started 2020-11-28T21:46:06.560 INFO:tasks.ceph.mon.c:Restarting daemon 2020-11-28T21:46:06.560 INFO:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2020-11-28T21:46:06.562 INFO:tasks.ceph.mon.c:Started 2020-11-28T21:46:06.563 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2020-11-28T21:46:06.563 INFO:tasks.ceph.mgr.y:Restarting daemon 2020-11-28T21:46:06.563 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2020-11-28T21:46:06.566 INFO:tasks.ceph.mgr.y:Started 2020-11-28T21:46:06.566 INFO:tasks.ceph.mgr.x:Restarting daemon 2020-11-28T21:46:06.567 INFO:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2020-11-28T21:46:06.569 INFO:tasks.ceph.mgr.x:Started 2020-11-28T21:46:06.569 INFO:tasks.ceph:Setting crush tunables to default 2020-11-28T21:46:06.570 INFO:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd crush tunables default 2020-11-28T21:46:11.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:06.777+0000 7f61a374c700 1 Processor -- start 2020-11-28T21:46:11.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:06.777+0000 7f61a374c700 1 -- start start 2020-11-28T21:46:11.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:06.781+0000 7f61a374c700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f619c060af0 0x7f619c0b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:11.758 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:06.781+0000 7f61a374c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f619c060ed0 0x7f619c0602f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:11.758 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:06.781+0000 7f61a374c700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f619c0b4c30 con 0x7f619c080960 2020-11-28T21:46:11.758 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:06.781+0000 7f61a374c700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f619c0b53a0 con 0x7f619c060ed0 2020-11-28T21:46:11.758 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:06.781+0000 7f61a374c700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f619c0b4e60 con 0x7f619c060af0 2020-11-28T21:46:11.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:06.781+0000 7f61a14e8700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f619c080960 0x7f619c0a92d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:43448/0 (socket says 172.21.15.162:43448) 2020-11-28T21:46:11.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:06.781+0000 7f61a14e8700 1 -- 172.21.15.162:0/979096437 learned_addr learned my addr 172.21.15.162:0/979096437 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:11.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:06.781+0000 7f61a1ce9700 1 --2- 172.21.15.162:0/979096437 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f619c060af0 0x7f619c0b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:11.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:06.781+0000 7f61a24ea700 1 --2- 172.21.15.162:0/979096437 >> v2:172.21.15.118:3300/0 conn(0x7f619c060ed0 0x7f619c0602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:11.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:06.781+0000 7f61a1ce9700 1 -- 172.21.15.162:0/979096437 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f619c060af0 msgr2=0x7f619c0b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2020-11-28T21:46:11.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:06.781+0000 7f61a1ce9700 1 -- 172.21.15.162:0/979096437 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f619c060af0 msgr2=0x7f619c0b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-28T21:46:11.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:06.781+0000 7f61a1ce9700 1 --2- 172.21.15.162:0/979096437 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f619c060af0 0x7f619c0b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-28T21:46:11.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:06.781+0000 7f61a1ce9700 1 --2- 172.21.15.162:0/979096437 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f619c060af0 0x7f619c0b0600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-28T21:46:11.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:06.781+0000 7f61a24ea700 1 -- 172.21.15.162:0/979096437 >> v2:172.21.15.118:3300/0 conn(0x7f619c060ed0 msgr2=0x7f619c0602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-28T21:46:11.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:06.781+0000 7f61a24ea700 1 -- 172.21.15.162:0/979096437 >> v2:172.21.15.118:3300/0 conn(0x7f619c060ed0 msgr2=0x7f619c0602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-28T21:46:11.761 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:06.781+0000 7f61a24ea700 1 --2- 172.21.15.162:0/979096437 >> v2:172.21.15.118:3300/0 conn(0x7f619c060ed0 0x7f619c0602f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-28T21:46:11.761 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:06.781+0000 7f61a24ea700 1 --2- 172.21.15.162:0/979096437 >> v2:172.21.15.118:3300/0 conn(0x7f619c060ed0 0x7f619c0602f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-28T21:46:11.761 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:06.985+0000 7f61a1ce9700 1 --2- 172.21.15.162:0/979096437 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f619c060af0 0x7f619c0b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:11.761 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:06.985+0000 7f61a24ea700 1 --2- 172.21.15.162:0/979096437 >> v2:172.21.15.118:3300/0 conn(0x7f619c060ed0 0x7f619c0602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:11.761 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:06.985+0000 7f61a1ce9700 1 -- 172.21.15.162:0/979096437 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f619c060af0 msgr2=0x7f619c0b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2020-11-28T21:46:11.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:06.985+0000 7f61a1ce9700 1 -- 172.21.15.162:0/979096437 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f619c060af0 msgr2=0x7f619c0b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-28T21:46:11.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:06.985+0000 7f61a1ce9700 1 --2- 172.21.15.162:0/979096437 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f619c060af0 0x7f619c0b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-28T21:46:11.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:06.985+0000 7f61a1ce9700 1 --2- 172.21.15.162:0/979096437 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f619c060af0 0x7f619c0b0600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-28T21:46:11.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:06.985+0000 7f61a24ea700 1 -- 172.21.15.162:0/979096437 >> v2:172.21.15.118:3300/0 conn(0x7f619c060ed0 msgr2=0x7f619c0602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-28T21:46:11.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:06.985+0000 7f61a24ea700 1 -- 172.21.15.162:0/979096437 >> v2:172.21.15.118:3300/0 conn(0x7f619c060ed0 msgr2=0x7f619c0602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-28T21:46:11.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:06.985+0000 7f61a24ea700 1 --2- 172.21.15.162:0/979096437 >> v2:172.21.15.118:3300/0 conn(0x7f619c060ed0 0x7f619c0602f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-28T21:46:11.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:06.985+0000 7f61a24ea700 1 --2- 172.21.15.162:0/979096437 >> v2:172.21.15.118:3300/0 conn(0x7f619c060ed0 0x7f619c0602f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-28T21:46:11.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:07.389+0000 7f61a1ce9700 1 --2- 172.21.15.162:0/979096437 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f619c060af0 0x7f619c0b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:11.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:07.389+0000 7f61a24ea700 1 --2- 172.21.15.162:0/979096437 >> v2:172.21.15.118:3300/0 conn(0x7f619c060ed0 0x7f619c0602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:11.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:07.389+0000 7f61a1ce9700 1 -- 172.21.15.162:0/979096437 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f619c060af0 msgr2=0x7f619c0b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2020-11-28T21:46:11.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:07.389+0000 7f61a1ce9700 1 -- 172.21.15.162:0/979096437 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f619c060af0 msgr2=0x7f619c0b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-28T21:46:11.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:07.389+0000 7f61a1ce9700 1 --2- 172.21.15.162:0/979096437 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f619c060af0 0x7f619c0b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-28T21:46:11.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:07.389+0000 7f61a24ea700 1 -- 172.21.15.162:0/979096437 >> v2:172.21.15.118:3300/0 conn(0x7f619c060ed0 msgr2=0x7f619c0602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-28T21:46:11.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:07.389+0000 7f61a24ea700 1 -- 172.21.15.162:0/979096437 >> v2:172.21.15.118:3300/0 conn(0x7f619c060ed0 msgr2=0x7f619c0602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-28T21:46:11.765 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:07.389+0000 7f61a24ea700 1 --2- 172.21.15.162:0/979096437 >> v2:172.21.15.118:3300/0 conn(0x7f619c060ed0 0x7f619c0602f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-28T21:46:11.765 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:07.389+0000 7f61a1ce9700 1 --2- 172.21.15.162:0/979096437 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f619c060af0 0x7f619c0b0600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-28T21:46:11.765 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:07.389+0000 7f61a24ea700 1 --2- 172.21.15.162:0/979096437 >> v2:172.21.15.118:3300/0 conn(0x7f619c060ed0 0x7f619c0602f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-28T21:46:11.765 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:08.193+0000 7f61a1ce9700 1 --2- 172.21.15.162:0/979096437 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f619c060af0 0x7f619c0b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:11.765 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:08.193+0000 7f61a24ea700 1 --2- 172.21.15.162:0/979096437 >> v2:172.21.15.118:3300/0 conn(0x7f619c060ed0 0x7f619c0602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:11.766 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:08.193+0000 7f61a1ce9700 1 -- 172.21.15.162:0/979096437 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f619c060af0 msgr2=0x7f619c0b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2020-11-28T21:46:11.766 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:08.193+0000 7f61a1ce9700 1 -- 172.21.15.162:0/979096437 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f619c060af0 msgr2=0x7f619c0b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-28T21:46:11.766 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:08.193+0000 7f61a24ea700 1 -- 172.21.15.162:0/979096437 >> v2:172.21.15.118:3300/0 conn(0x7f619c060ed0 msgr2=0x7f619c0602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-28T21:46:11.766 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:08.193+0000 7f61a24ea700 1 -- 172.21.15.162:0/979096437 >> v2:172.21.15.118:3300/0 conn(0x7f619c060ed0 msgr2=0x7f619c0602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-28T21:46:11.766 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:08.193+0000 7f61a1ce9700 1 --2- 172.21.15.162:0/979096437 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f619c060af0 0x7f619c0b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-28T21:46:11.767 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:08.193+0000 7f61a24ea700 1 --2- 172.21.15.162:0/979096437 >> v2:172.21.15.118:3300/0 conn(0x7f619c060ed0 0x7f619c0602f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-28T21:46:11.767 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:08.193+0000 7f61a1ce9700 1 --2- 172.21.15.162:0/979096437 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f619c060af0 0x7f619c0b0600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-28T21:46:11.767 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:08.193+0000 7f61a24ea700 1 --2- 172.21.15.162:0/979096437 >> v2:172.21.15.118:3300/0 conn(0x7f619c060ed0 0x7f619c0602f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-28T21:46:11.767 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:09.781+0000 7f61937fe700 1 -- 172.21.15.162:0/979096437 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f619c060af0 msgr2=0x7f619c0b0600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-28T21:46:11.767 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:09.781+0000 7f61937fe700 1 --2- 172.21.15.162:0/979096437 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f619c060af0 0x7f619c0b0600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:11.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:09.781+0000 7f61937fe700 1 -- 172.21.15.162:0/979096437 >> v2:172.21.15.118:3300/0 conn(0x7f619c060ed0 msgr2=0x7f619c0602f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-28T21:46:11.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:09.781+0000 7f61937fe700 1 --2- 172.21.15.162:0/979096437 >> v2:172.21.15.118:3300/0 conn(0x7f619c060ed0 0x7f619c0602f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:11.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:09.781+0000 7f61937fe700 1 -- 172.21.15.162:0/979096437 >> v1:172.21.15.162:6789/0 conn(0x7f619c080960 legacy=0x7f619c0a92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:11.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:09.781+0000 7f61937fe700 1 --2- 172.21.15.162:0/979096437 >> v2:172.21.15.162:3300/0 conn(0x7f6180007050 0x7f61800094f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:11.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:09.781+0000 7f61937fe700 1 -- 172.21.15.162:0/979096437 --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f619c0b5100 con 0x7f61800033c0 2020-11-28T21:46:11.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:09.781+0000 7f61937fe700 1 -- 172.21.15.162:0/979096437 --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f619c0b53a0 con 0x7f6180007050 2020-11-28T21:46:11.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:09.781+0000 7f61a14e8700 1 --2- 172.21.15.162:0/979096437 >> v2:172.21.15.162:3300/0 conn(0x7f6180007050 0x7f61800094f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:11.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:09.781+0000 7f61a14e8700 1 -- 172.21.15.162:0/979096437 >> v2:172.21.15.162:3300/0 conn(0x7f6180007050 msgr2=0x7f61800094f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-28T21:46:11.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:09.781+0000 7f61a14e8700 1 -- 172.21.15.162:0/979096437 >> v2:172.21.15.162:3300/0 conn(0x7f6180007050 msgr2=0x7f61800094f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-28T21:46:11.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:09.781+0000 7f61a14e8700 1 --2- 172.21.15.162:0/979096437 >> v2:172.21.15.162:3300/0 conn(0x7f6180007050 0x7f61800094f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-28T21:46:11.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:09.781+0000 7f61a14e8700 1 --2- 172.21.15.162:0/979096437 >> v2:172.21.15.162:3300/0 conn(0x7f6180007050 0x7f61800094f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-28T21:46:11.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:09.985+0000 7f61a14e8700 1 --2- 172.21.15.162:0/979096437 >> v2:172.21.15.162:3300/0 conn(0x7f6180007050 0x7f61800094f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:11.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:09.985+0000 7f61a14e8700 1 -- 172.21.15.162:0/979096437 >> v2:172.21.15.162:3300/0 conn(0x7f6180007050 msgr2=0x7f61800094f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-28T21:46:11.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:09.985+0000 7f61a14e8700 1 -- 172.21.15.162:0/979096437 >> v2:172.21.15.162:3300/0 conn(0x7f6180007050 msgr2=0x7f61800094f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-28T21:46:11.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:09.985+0000 7f61a14e8700 1 --2- 172.21.15.162:0/979096437 >> v2:172.21.15.162:3300/0 conn(0x7f6180007050 0x7f61800094f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-28T21:46:11.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:09.985+0000 7f61a14e8700 1 --2- 172.21.15.162:0/979096437 >> v2:172.21.15.162:3300/0 conn(0x7f6180007050 0x7f61800094f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-28T21:46:11.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:10.389+0000 7f61a14e8700 1 --2- 172.21.15.162:0/979096437 >> v2:172.21.15.162:3300/0 conn(0x7f6180007050 0x7f61800094f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:11.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:10.389+0000 7f61a14e8700 1 -- 172.21.15.162:0/979096437 >> v2:172.21.15.162:3300/0 conn(0x7f6180007050 msgr2=0x7f61800094f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-28T21:46:11.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:10.389+0000 7f61a14e8700 1 -- 172.21.15.162:0/979096437 >> v2:172.21.15.162:3300/0 conn(0x7f6180007050 msgr2=0x7f61800094f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-28T21:46:11.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:10.389+0000 7f61a14e8700 1 --2- 172.21.15.162:0/979096437 >> v2:172.21.15.162:3300/0 conn(0x7f6180007050 0x7f61800094f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-28T21:46:11.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:10.389+0000 7f61a14e8700 1 --2- 172.21.15.162:0/979096437 >> v2:172.21.15.162:3300/0 conn(0x7f6180007050 0x7f61800094f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-28T21:46:11.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.193+0000 7f61a14e8700 1 --2- 172.21.15.162:0/979096437 >> v2:172.21.15.162:3300/0 conn(0x7f6180007050 0x7f61800094f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:11.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.193+0000 7f61a14e8700 1 -- 172.21.15.162:0/979096437 >> v2:172.21.15.162:3300/0 conn(0x7f6180007050 msgr2=0x7f61800094f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-28T21:46:11.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.193+0000 7f61a14e8700 1 -- 172.21.15.162:0/979096437 >> v2:172.21.15.162:3300/0 conn(0x7f6180007050 msgr2=0x7f61800094f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-28T21:46:11.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.193+0000 7f61a14e8700 1 --2- 172.21.15.162:0/979096437 >> v2:172.21.15.162:3300/0 conn(0x7f6180007050 0x7f61800094f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-28T21:46:11.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.193+0000 7f61a14e8700 1 --2- 172.21.15.162:0/979096437 >> v2:172.21.15.162:3300/0 conn(0x7f6180007050 0x7f61800094f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-28T21:46:11.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.745+0000 7f61a0ce7700 1 -- 172.21.15.162:0/979096437 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2400591145 0 0) 0x7f619c0b5100 con 0x7f61800033c0 2020-11-28T21:46:11.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.745+0000 7f61a0ce7700 1 -- 172.21.15.162:0/979096437 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6188003230 con 0x7f61800033c0 2020-11-28T21:46:11.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.745+0000 7f61a0ce7700 1 -- 172.21.15.162:0/979096437 <== mon.0 v1:172.21.15.118:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4155605192 0 0) 0x7f6188003230 con 0x7f61800033c0 2020-11-28T21:46:11.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.745+0000 7f61a0ce7700 1 -- 172.21.15.162:0/979096437 >> v2:172.21.15.162:3300/0 conn(0x7f6180007050 msgr2=0x7f61800094f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-28T21:46:11.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.745+0000 7f61a0ce7700 1 --2- 172.21.15.162:0/979096437 >> v2:172.21.15.162:3300/0 conn(0x7f6180007050 0x7f61800094f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:11.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.745+0000 7f61a0ce7700 1 -- 172.21.15.162:0/979096437 --> v1:172.21.15.118:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f618000ac70 con 0x7f61800033c0 2020-11-28T21:46:11.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.745+0000 7f61a0ce7700 1 -- 172.21.15.162:0/979096437 <== mon.0 v1:172.21.15.118:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f61980513c0 con 0x7f61800033c0 2020-11-28T21:46:11.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.749+0000 7f61a0ce7700 1 -- 172.21.15.162:0/979096437 >> v1:172.21.15.118:6789/0 conn(0x7f61800033c0 legacy=0x7f6180005820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:11.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.749+0000 7f61a24ea700 1 -- 172.21.15.162:0/979096437 reap_dead start 2020-11-28T21:46:11.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.749+0000 7f61a0ce7700 1 --2- 172.21.15.162:0/979096437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6188003720 0x7f6188003b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:11.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.749+0000 7f61a0ce7700 1 --2- 172.21.15.162:0/979096437 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f61880044d0 0x7f6188004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:11.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.749+0000 7f61a0ce7700 1 --2- 172.21.15.162:0/979096437 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6188004e40 0x7f618800b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:11.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.749+0000 7f61a0ce7700 1 -- 172.21.15.162:0/979096437 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619c0b53a0 con 0x7f61880044d0 2020-11-28T21:46:11.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.749+0000 7f61a0ce7700 1 -- 172.21.15.162:0/979096437 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6188005350 con 0x7f6188003720 2020-11-28T21:46:11.776 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.749+0000 7f61a0ce7700 1 -- 172.21.15.162:0/979096437 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f618800bc40 con 0x7f6188004e40 2020-11-28T21:46:11.776 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.749+0000 7f61a1ce9700 1 --2- 172.21.15.162:0/979096437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6188003720 0x7f6188003b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:11.776 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.749+0000 7f61a24ea700 1 --2- 172.21.15.162:0/979096437 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6188004e40 0x7f618800b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:11.776 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.749+0000 7f61a14e8700 1 --2- 172.21.15.162:0/979096437 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f61880044d0 0x7f6188004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:11.776 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.749+0000 7f61a1ce9700 1 -- 172.21.15.162:0/979096437 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6188004e40 msgr2=0x7f618800b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:11.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.749+0000 7f61a1ce9700 1 --2- 172.21.15.162:0/979096437 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6188004e40 0x7f618800b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:11.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.749+0000 7f61a1ce9700 1 -- 172.21.15.162:0/979096437 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f61880044d0 msgr2=0x7f6188004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:11.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.749+0000 7f61a1ce9700 1 --2- 172.21.15.162:0/979096437 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f61880044d0 0x7f6188004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:11.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.749+0000 7f61a1ce9700 1 -- 172.21.15.162:0/979096437 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6188003230 con 0x7f6188003720 2020-11-28T21:46:11.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.749+0000 7f61a14e8700 1 --2- 172.21.15.162:0/979096437 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f61880044d0 0x7f6188004950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:46:11.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.749+0000 7f61a1ce9700 1 --2- 172.21.15.162:0/979096437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6188003720 0x7f6188003b20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f6198000b20 tx=0x7f6198008040).ready entity=mon.1 client_cookie=fdde0d81831d0279 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:11.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.749+0000 7f61a0ce7700 1 -- 172.21.15.162:0/979096437 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6198012410 con 0x7f6188003720 2020-11-28T21:46:11.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.749+0000 7f61a0ce7700 1 -- 172.21.15.162:0/979096437 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6198012570 con 0x7f6188003720 2020-11-28T21:46:11.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.749+0000 7f61a0ce7700 1 -- 172.21.15.162:0/979096437 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61980128d0 con 0x7f6188003720 2020-11-28T21:46:11.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.749+0000 7f61a374c700 1 -- 172.21.15.162:0/979096437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6188003720 msgr2=0x7f6188003b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:11.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.749+0000 7f61a374c700 1 --2- 172.21.15.162:0/979096437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6188003720 0x7f6188003b20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f6198000b20 tx=0x7f6198008040).stop 2020-11-28T21:46:11.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.753+0000 7f61a374c700 1 -- 172.21.15.162:0/979096437 shutdown_connections 2020-11-28T21:46:11.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.753+0000 7f61a374c700 1 --2- 172.21.15.162:0/979096437 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6188004e40 0x7f618800b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:11.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.753+0000 7f61a374c700 1 --2- 172.21.15.162:0/979096437 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f61880044d0 0x7f6188004950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:11.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.753+0000 7f61a374c700 1 --2- 172.21.15.162:0/979096437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6188003720 0x7f6188003b20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:11.780 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.753+0000 7f61a374c700 1 -- 172.21.15.162:0/979096437 >> 172.21.15.162:0/979096437 conn(0x7f619c09a2f0 msgr2=0x7f619c09a700 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:11.780 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.753+0000 7f61a374c700 1 -- 172.21.15.162:0/979096437 shutdown_connections 2020-11-28T21:46:11.780 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.753+0000 7f61a374c700 1 -- 172.21.15.162:0/979096437 wait complete. 2020-11-28T21:46:11.780 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.753+0000 7f61a374c700 1 Processor -- start 2020-11-28T21:46:11.781 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.753+0000 7f61a374c700 1 -- start start 2020-11-28T21:46:11.781 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.757+0000 7f61a374c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6188003720 0x7f619c0a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:11.781 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.757+0000 7f61a374c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61880044d0 0x7f619c0a7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:11.781 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.757+0000 7f61a374c700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6188004e40 0x7f619c0a8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:11.781 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.757+0000 7f61a374c700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619c0af050 con 0x7f6188003720 2020-11-28T21:46:11.782 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.757+0000 7f61a374c700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619c145450 con 0x7f61880044d0 2020-11-28T21:46:11.782 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.757+0000 7f61a374c700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f619c1456f0 con 0x7f6188004e40 2020-11-28T21:46:11.782 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.757+0000 7f61a14e8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61880044d0 0x7f619c0a7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:11.782 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.757+0000 7f61a1ce9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6188003720 0x7f619c0a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:11.782 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.757+0000 7f61a14e8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61880044d0 0x7f619c0a7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:36666/0 (socket says 172.21.15.162:36666) 2020-11-28T21:46:11.783 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.757+0000 7f61a24ea700 1 --2- 172.21.15.162:0/2493055737 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6188004e40 0x7f619c0a8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:11.783 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.757+0000 7f61a14e8700 1 -- 172.21.15.162:0/2493055737 learned_addr learned my addr 172.21.15.162:0/2493055737 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:11.783 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.757+0000 7f61a14e8700 1 -- 172.21.15.162:0/2493055737 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6188004e40 msgr2=0x7f619c0a8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:11.783 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.757+0000 7f61a14e8700 1 --2- 172.21.15.162:0/2493055737 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6188004e40 0x7f619c0a8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:11.783 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.757+0000 7f61a14e8700 1 -- 172.21.15.162:0/2493055737 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6188003720 msgr2=0x7f619c0a3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:11.784 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.757+0000 7f61a14e8700 1 --2- 172.21.15.162:0/2493055737 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6188003720 0x7f619c0a3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:11.784 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.757+0000 7f61a14e8700 1 -- 172.21.15.162:0/2493055737 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f619c0a8540 con 0x7f61880044d0 2020-11-28T21:46:11.784 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.757+0000 7f61a1ce9700 1 --2- 172.21.15.162:0/2493055737 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6188003720 0x7f619c0a3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:46:11.784 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.757+0000 7f61a14e8700 1 --2- 172.21.15.162:0/2493055737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61880044d0 0x7f619c0a7b60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f618c00b000 tx=0x7f618c009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:11.785 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.761+0000 7f61927fc700 1 -- 172.21.15.162:0/2493055737 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f618c017620 con 0x7f61880044d0 2020-11-28T21:46:11.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.761+0000 7f61927fc700 1 -- 172.21.15.162:0/2493055737 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f618c017780 con 0x7f61880044d0 2020-11-28T21:46:11.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.761+0000 7f61a374c700 1 -- 172.21.15.162:0/2493055737 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f619c0ada70 con 0x7f61880044d0 2020-11-28T21:46:11.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.761+0000 7f61927fc700 1 -- 172.21.15.162:0/2493055737 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f618c017ae0 con 0x7f61880044d0 2020-11-28T21:46:11.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.761+0000 7f61a374c700 1 -- 172.21.15.162:0/2493055737 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f619c0ae7d0 con 0x7f61880044d0 2020-11-28T21:46:11.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.761+0000 7f61927fc700 1 -- 172.21.15.162:0/2493055737 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f618c013040 con 0x7f61880044d0 2020-11-28T21:46:11.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.761+0000 7f61927fc700 1 -- 172.21.15.162:0/2493055737 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f618c0169a0 con 0x7f61880044d0 2020-11-28T21:46:11.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.761+0000 7f61a374c700 1 -- 172.21.15.162:0/2493055737 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f619c057050 con 0x7f61880044d0 2020-11-28T21:46:11.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.769+0000 7f61927fc700 1 -- 172.21.15.162:0/2493055737 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f618c01c030 con 0x7f61880044d0 2020-11-28T21:46:11.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.845+0000 7f61a374c700 1 -- 172.21.15.162:0/2493055737 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f619c0af1c0 con 0x7f61880044d0 2020-11-28T21:46:11.853 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.853+0000 7f61927fc700 1 -- 172.21.15.162:0/2493055737 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f618c02f090 con 0x7f61880044d0 2020-11-28T21:46:11.859 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.853+0000 7f61a374c700 1 -- 172.21.15.162:0/2493055737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61880044d0 msgr2=0x7f619c0a7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:11.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.853+0000 7f61a374c700 1 --2- 172.21.15.162:0/2493055737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61880044d0 0x7f619c0a7b60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f618c00b000 tx=0x7f618c009040).stop 2020-11-28T21:46:11.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.853+0000 7f61a374c700 1 -- 172.21.15.162:0/2493055737 shutdown_connections 2020-11-28T21:46:11.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.853+0000 7f61a374c700 1 --2- 172.21.15.162:0/2493055737 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6188004e40 0x7f619c0a8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:11.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.853+0000 7f61a374c700 1 --2- 172.21.15.162:0/2493055737 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6188003720 0x7f619c0a3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:11.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.853+0000 7f61a374c700 1 --2- 172.21.15.162:0/2493055737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61880044d0 0x7f619c0a7b60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:11.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.853+0000 7f61a374c700 1 -- 172.21.15.162:0/2493055737 >> 172.21.15.162:0/2493055737 conn(0x7f619c09a2f0 msgr2=0x7f619c063390 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:11.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.853+0000 7f61a374c700 1 -- 172.21.15.162:0/2493055737 shutdown_connections 2020-11-28T21:46:11.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:11.853+0000 7f61a374c700 1 -- 172.21.15.162:0/2493055737 wait complete. 2020-11-28T21:46:11.861 INFO:teuthology.orchestra.run.smithi162.stderr:adjusted tunables profile to default 2020-11-28T21:46:11.870 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2020-11-28T21:46:11.870 INFO:teuthology.orchestra.run.smithi162:> mktemp 2020-11-28T21:46:11.875 INFO:teuthology.orchestra.run.smithi162.stdout:/tmp/tmp.xczcHEVz4V 2020-11-28T21:46:11.875 INFO:teuthology.orchestra.run.smithi162:> sudo cp /var/lib/ceph/osd/ceph-0/fsid /tmp/tmp.xczcHEVz4V 2020-11-28T21:46:11.934 INFO:teuthology.orchestra.run.smithi162:> sudo chmod 0666 /tmp/tmp.xczcHEVz4V 2020-11-28T21:46:12.038 DEBUG:teuthology.orchestra.remote:smithi162:/tmp/tmp.xczcHEVz4V is 37B 2020-11-28T21:46:12.049 INFO:teuthology.orchestra.run.smithi162:> rm -fr /tmp/tmp.xczcHEVz4V 2020-11-28T21:46:12.054 INFO:teuthology.orchestra.run.smithi162:> mktemp 2020-11-28T21:46:12.100 INFO:teuthology.orchestra.run.smithi162.stdout:/tmp/tmp.3mduXWVv2z 2020-11-28T21:46:12.101 INFO:teuthology.orchestra.run.smithi162:> sudo cp /var/lib/ceph/osd/ceph-1/fsid /tmp/tmp.3mduXWVv2z 2020-11-28T21:46:12.157 INFO:teuthology.orchestra.run.smithi162:> sudo chmod 0666 /tmp/tmp.3mduXWVv2z 2020-11-28T21:46:12.262 DEBUG:teuthology.orchestra.remote:smithi162:/tmp/tmp.3mduXWVv2z is 37B 2020-11-28T21:46:12.272 INFO:teuthology.orchestra.run.smithi162:> rm -fr /tmp/tmp.3mduXWVv2z 2020-11-28T21:46:12.278 INFO:teuthology.orchestra.run.smithi162:> mktemp 2020-11-28T21:46:12.324 INFO:teuthology.orchestra.run.smithi162.stdout:/tmp/tmp.bWOEbZIJZe 2020-11-28T21:46:12.325 INFO:teuthology.orchestra.run.smithi162:> sudo cp /var/lib/ceph/osd/ceph-2/fsid /tmp/tmp.bWOEbZIJZe 2020-11-28T21:46:12.381 INFO:teuthology.orchestra.run.smithi162:> sudo chmod 0666 /tmp/tmp.bWOEbZIJZe 2020-11-28T21:46:12.487 DEBUG:teuthology.orchestra.remote:smithi162:/tmp/tmp.bWOEbZIJZe is 37B 2020-11-28T21:46:12.537 INFO:teuthology.orchestra.run.smithi162:> rm -fr /tmp/tmp.bWOEbZIJZe 2020-11-28T21:46:12.543 INFO:teuthology.orchestra.run.smithi162:> mktemp 2020-11-28T21:46:12.593 INFO:teuthology.orchestra.run.smithi162.stdout:/tmp/tmp.7p0jHAHvc1 2020-11-28T21:46:12.593 INFO:teuthology.orchestra.run.smithi162:> sudo cp /var/lib/ceph/osd/ceph-3/fsid /tmp/tmp.7p0jHAHvc1 2020-11-28T21:46:12.649 INFO:teuthology.orchestra.run.smithi162:> sudo chmod 0666 /tmp/tmp.7p0jHAHvc1 2020-11-28T21:46:12.754 DEBUG:teuthology.orchestra.remote:smithi162:/tmp/tmp.7p0jHAHvc1 is 37B 2020-11-28T21:46:12.764 INFO:teuthology.orchestra.run.smithi162:> rm -fr /tmp/tmp.7p0jHAHvc1 2020-11-28T21:46:12.770 INFO:teuthology.orchestra.run.smithi118:> mktemp 2020-11-28T21:46:12.776 INFO:teuthology.orchestra.run.smithi118.stdout:/tmp/tmp.UAsEQHdfDJ 2020-11-28T21:46:12.776 INFO:teuthology.orchestra.run.smithi118:> sudo cp /var/lib/ceph/osd/ceph-4/fsid /tmp/tmp.UAsEQHdfDJ 2020-11-28T21:46:12.832 INFO:teuthology.orchestra.run.smithi118:> sudo chmod 0666 /tmp/tmp.UAsEQHdfDJ 2020-11-28T21:46:12.936 DEBUG:teuthology.orchestra.remote:smithi118:/tmp/tmp.UAsEQHdfDJ is 37B 2020-11-28T21:46:12.948 INFO:teuthology.orchestra.run.smithi118:> rm -fr /tmp/tmp.UAsEQHdfDJ 2020-11-28T21:46:12.955 INFO:teuthology.orchestra.run.smithi118:> mktemp 2020-11-28T21:46:13.003 INFO:teuthology.orchestra.run.smithi118.stdout:/tmp/tmp.w6tnrQ6Ygg 2020-11-28T21:46:13.004 INFO:teuthology.orchestra.run.smithi118:> sudo cp /var/lib/ceph/osd/ceph-5/fsid /tmp/tmp.w6tnrQ6Ygg 2020-11-28T21:46:13.059 INFO:teuthology.orchestra.run.smithi118:> sudo chmod 0666 /tmp/tmp.w6tnrQ6Ygg 2020-11-28T21:46:13.165 DEBUG:teuthology.orchestra.remote:smithi118:/tmp/tmp.w6tnrQ6Ygg is 37B 2020-11-28T21:46:13.176 INFO:teuthology.orchestra.run.smithi118:> rm -fr /tmp/tmp.w6tnrQ6Ygg 2020-11-28T21:46:13.183 INFO:teuthology.orchestra.run.smithi118:> mktemp 2020-11-28T21:46:13.230 INFO:teuthology.orchestra.run.smithi118.stdout:/tmp/tmp.gQbBj79i3j 2020-11-28T21:46:13.231 INFO:teuthology.orchestra.run.smithi118:> sudo cp /var/lib/ceph/osd/ceph-6/fsid /tmp/tmp.gQbBj79i3j 2020-11-28T21:46:13.288 INFO:teuthology.orchestra.run.smithi118:> sudo chmod 0666 /tmp/tmp.gQbBj79i3j 2020-11-28T21:46:13.393 DEBUG:teuthology.orchestra.remote:smithi118:/tmp/tmp.gQbBj79i3j is 37B 2020-11-28T21:46:13.403 INFO:teuthology.orchestra.run.smithi118:> rm -fr /tmp/tmp.gQbBj79i3j 2020-11-28T21:46:13.410 INFO:teuthology.orchestra.run.smithi118:> mktemp 2020-11-28T21:46:13.458 INFO:teuthology.orchestra.run.smithi118.stdout:/tmp/tmp.VLb1aDk2K0 2020-11-28T21:46:13.459 INFO:teuthology.orchestra.run.smithi118:> sudo cp /var/lib/ceph/osd/ceph-7/fsid /tmp/tmp.VLb1aDk2K0 2020-11-28T21:46:13.516 INFO:teuthology.orchestra.run.smithi118:> sudo chmod 0666 /tmp/tmp.VLb1aDk2K0 2020-11-28T21:46:13.621 DEBUG:teuthology.orchestra.remote:smithi118:/tmp/tmp.VLb1aDk2K0 is 37B 2020-11-28T21:46:13.670 INFO:teuthology.orchestra.run.smithi118:> rm -fr /tmp/tmp.VLb1aDk2K0 2020-11-28T21:46:13.676 INFO:teuthology.orchestra.run.smithi118:> sudo ceph --cluster ceph osd new 99c7d9ae-adfa-4fd7-880a-23e6baf3281b 0 2020-11-28T21:46:13.832 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.827+0000 7ff9715df700 1 Processor -- start 2020-11-28T21:46:13.832 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.827+0000 7ff9715df700 1 -- start start 2020-11-28T21:46:13.833 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.831+0000 7ff9715df700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff96c0b1240 0x7ff96c0ad790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:13.833 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.831+0000 7ff9715df700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff96c1530e0 0x7ff96c153550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:13.833 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.831+0000 7ff9715df700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff96c0638e0 con 0x7ff96c05f380 2020-11-28T21:46:13.833 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.831+0000 7ff9715df700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff96c0b3760 con 0x7ff96c1530e0 2020-11-28T21:46:13.834 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.831+0000 7ff9715df700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff96c0b1620 con 0x7ff96c0b1240 2020-11-28T21:46:13.834 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.831+0000 7ff972841700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff96c1530e0 0x7ff96c153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:13.834 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.831+0000 7ff96bfff700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7ff96c05f380 0x7ff96c0adc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:55718/0 (socket says 172.21.15.118:55718) 2020-11-28T21:46:13.835 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.831+0000 7ff96bfff700 1 -- 172.21.15.118:0/2458491907 learned_addr learned my addr 172.21.15.118:0/2458491907 (peer_addr_for_me v1:172.21.15.118:0/0) 2020-11-28T21:46:13.835 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.831+0000 7ff970dde700 1 --2- 172.21.15.118:0/2458491907 >> v2:172.21.15.162:3300/0 conn(0x7ff96c0b1240 0x7ff96c0ad790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:13.835 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.831+0000 7ff972841700 1 -- 172.21.15.118:0/2458491907 >> v2:172.21.15.162:3300/0 conn(0x7ff96c0b1240 msgr2=0x7ff96c0ad790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:13.835 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.831+0000 7ff972841700 1 --2- 172.21.15.118:0/2458491907 >> v2:172.21.15.162:3300/0 conn(0x7ff96c0b1240 0x7ff96c0ad790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:13.835 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.831+0000 7ff972841700 1 -- 172.21.15.118:0/2458491907 >> v1:172.21.15.118:6789/0 conn(0x7ff96c05f380 legacy=0x7ff96c0adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:13.836 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.831+0000 7ff96b7fe700 1 -- 172.21.15.118:0/2458491907 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 147950605 0 0) 0x7ff96c0638e0 con 0x7ff96c05f380 2020-11-28T21:46:13.836 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.831+0000 7ff972841700 1 -- 172.21.15.118:0/2458491907 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff96c0605b0 con 0x7ff96c1530e0 2020-11-28T21:46:13.836 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.831+0000 7ff972841700 1 --2- 172.21.15.118:0/2458491907 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff96c1530e0 0x7ff96c153550 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7ff9640032b0 tx=0x7ff9640091a0).ready entity=mon.0 client_cookie=ee5409ab654154ad server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:13.836 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.831+0000 7ff96b7fe700 1 -- 172.21.15.118:0/2458491907 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff96400b9a0 con 0x7ff96c1530e0 2020-11-28T21:46:13.836 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.831+0000 7ff96b7fe700 1 -- 172.21.15.118:0/2458491907 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff96400bb00 con 0x7ff96c1530e0 2020-11-28T21:46:13.837 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.831+0000 7ff96b7fe700 1 -- 172.21.15.118:0/2458491907 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff96400bc70 con 0x7ff96c1530e0 2020-11-28T21:46:13.837 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.831+0000 7ff9715df700 1 -- 172.21.15.118:0/2458491907 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff96c1530e0 msgr2=0x7ff96c153550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:13.837 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.831+0000 7ff9715df700 1 --2- 172.21.15.118:0/2458491907 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff96c1530e0 0x7ff96c153550 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7ff9640032b0 tx=0x7ff9640091a0).stop 2020-11-28T21:46:13.837 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.831+0000 7ff9715df700 1 -- 172.21.15.118:0/2458491907 shutdown_connections 2020-11-28T21:46:13.838 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.831+0000 7ff9715df700 1 --2- 172.21.15.118:0/2458491907 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff96c1530e0 0x7ff96c153550 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:13.838 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.831+0000 7ff9715df700 1 --2- 172.21.15.118:0/2458491907 >> v2:172.21.15.162:3300/0 conn(0x7ff96c0b1240 0x7ff96c0ad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:13.838 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.831+0000 7ff9715df700 1 -- 172.21.15.118:0/2458491907 >> 172.21.15.118:0/2458491907 conn(0x7ff96c063d80 msgr2=0x7ff96c064190 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:13.838 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.831+0000 7ff9715df700 1 -- 172.21.15.118:0/2458491907 shutdown_connections 2020-11-28T21:46:13.838 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.831+0000 7ff9715df700 1 -- 172.21.15.118:0/2458491907 wait complete. 2020-11-28T21:46:13.841 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.835+0000 7ff9715df700 1 Processor -- start 2020-11-28T21:46:13.841 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.835+0000 7ff9715df700 1 -- start start 2020-11-28T21:46:13.842 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.835+0000 7ff9715df700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff96c05f380 0x7ff96c139730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:13.842 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.835+0000 7ff9715df700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff96c0b1240 0x7ff96c13d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:13.842 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.835+0000 7ff9715df700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff96c1530e0 0x7ff96c12a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:13.842 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.835+0000 7ff9715df700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c129bf0 con 0x7ff96c0b1240 2020-11-28T21:46:13.843 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.835+0000 7ff9715df700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c129e90 con 0x7ff96c1530e0 2020-11-28T21:46:13.843 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.835+0000 7ff9715df700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c13cdc0 con 0x7ff96c05f380 2020-11-28T21:46:13.843 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.835+0000 7ff970dde700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff96c05f380 0x7ff96c139730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:13.843 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.835+0000 7ff96bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff96c0b1240 0x7ff96c13d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:13.843 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.835+0000 7ff970dde700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff96c05f380 0x7ff96c139730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.118:41384/0 (socket says 172.21.15.118:41384) 2020-11-28T21:46:13.844 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.835+0000 7ff96bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff96c0b1240 0x7ff96c13d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:57942/0 (socket says 172.21.15.118:57942) 2020-11-28T21:46:13.844 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.835+0000 7ff970dde700 1 -- 172.21.15.118:0/2653861796 learned_addr learned my addr 172.21.15.118:0/2653861796 (peer_addr_for_me v2:172.21.15.118:0/0) 2020-11-28T21:46:13.844 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.835+0000 7ff972841700 1 --2- 172.21.15.118:0/2653861796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff96c1530e0 0x7ff96c12a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:13.844 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.835+0000 7ff970dde700 1 -- 172.21.15.118:0/2653861796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff96c1530e0 msgr2=0x7ff96c12a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:13.844 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.835+0000 7ff970dde700 1 --2- 172.21.15.118:0/2653861796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff96c1530e0 0x7ff96c12a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:13.845 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.835+0000 7ff970dde700 1 -- 172.21.15.118:0/2653861796 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff96c0b1240 msgr2=0x7ff96c13d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:13.845 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.835+0000 7ff970dde700 1 --2- 172.21.15.118:0/2653861796 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff96c0b1240 0x7ff96c13d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:13.845 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.835+0000 7ff970dde700 1 -- 172.21.15.118:0/2653861796 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff96c13d060 con 0x7ff96c05f380 2020-11-28T21:46:13.845 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.835+0000 7ff970dde700 1 --2- 172.21.15.118:0/2653861796 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff96c05f380 0x7ff96c139730 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7ff96000c000 tx=0x7ff960006dc0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:13.846 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.835+0000 7ff9697fa700 1 -- 172.21.15.118:0/2653861796 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff96000ca10 con 0x7ff96c05f380 2020-11-28T21:46:13.846 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.835+0000 7ff9697fa700 1 -- 172.21.15.118:0/2653861796 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff96000cb70 con 0x7ff96c05f380 2020-11-28T21:46:13.846 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.835+0000 7ff9715df700 1 -- 172.21.15.118:0/2653861796 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff96c0acd40 con 0x7ff96c05f380 2020-11-28T21:46:13.847 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.835+0000 7ff9697fa700 1 -- 172.21.15.118:0/2653861796 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff96000b370 con 0x7ff96c05f380 2020-11-28T21:46:13.847 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.835+0000 7ff9715df700 1 -- 172.21.15.118:0/2653861796 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff96c13db90 con 0x7ff96c05f380 2020-11-28T21:46:13.847 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.835+0000 7ff9697fa700 1 -- 172.21.15.118:0/2653861796 <== mon.2 v2:172.21.15.118:3301/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7ff96001c630 con 0x7ff96c05f380 2020-11-28T21:46:13.848 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.835+0000 7ff9697fa700 1 -- 172.21.15.118:0/2653861796 <== mon.2 v2:172.21.15.118:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 754+0+0 (secure 0 0 0) 0x7ff960012760 con 0x7ff96c05f380 2020-11-28T21:46:13.848 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.839+0000 7ff9715df700 1 -- 172.21.15.118:0/2653861796 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff96c13d1f0 con 0x7ff96c05f380 2020-11-28T21:46:13.848 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.839+0000 7ff9697fa700 1 -- 172.21.15.118:0/2653861796 <== mon.2 v2:172.21.15.118:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7ff96001cd70 con 0x7ff96c05f380 2020-11-28T21:46:13.920 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.919+0000 7ff9715df700 1 -- 172.21.15.118:0/2653861796 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "osd new", "uuid": "99c7d9ae-adfa-4fd7-880a-23e6baf3281b", "id": 0} v 0) v1 -- 0x7ff96c057050 con 0x7ff96c05f380 2020-11-28T21:46:13.925 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.927+0000 7ff9697fa700 1 -- 172.21.15.118:0/2653861796 <== mon.2 v2:172.21.15.118:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "99c7d9ae-adfa-4fd7-880a-23e6baf3281b", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff960012920 con 0x7ff96c05f380 2020-11-28T21:46:13.926 INFO:teuthology.orchestra.run.smithi118.stdout:0 2020-11-28T21:46:13.927 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.927+0000 7ff9715df700 1 -- 172.21.15.118:0/2653861796 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff96c05f380 msgr2=0x7ff96c139730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:13.927 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.927+0000 7ff9715df700 1 --2- 172.21.15.118:0/2653861796 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff96c05f380 0x7ff96c139730 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7ff96000c000 tx=0x7ff960006dc0).stop 2020-11-28T21:46:13.927 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.927+0000 7ff9715df700 1 -- 172.21.15.118:0/2653861796 shutdown_connections 2020-11-28T21:46:13.927 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.927+0000 7ff9715df700 1 --2- 172.21.15.118:0/2653861796 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff96c05f380 0x7ff96c139730 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:13.928 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.927+0000 7ff9715df700 1 --2- 172.21.15.118:0/2653861796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff96c1530e0 0x7ff96c12a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:13.928 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.927+0000 7ff9715df700 1 --2- 172.21.15.118:0/2653861796 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff96c0b1240 0x7ff96c13d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:13.928 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.927+0000 7ff9715df700 1 -- 172.21.15.118:0/2653861796 >> 172.21.15.118:0/2653861796 conn(0x7ff96c063d80 msgr2=0x7ff96c159990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:13.928 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.927+0000 7ff9715df700 1 -- 172.21.15.118:0/2653861796 shutdown_connections 2020-11-28T21:46:13.929 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:13.927+0000 7ff9715df700 1 -- 172.21.15.118:0/2653861796 wait complete. 2020-11-28T21:46:13.937 INFO:teuthology.orchestra.run.smithi118:> sudo ceph --cluster ceph osd new 6ce6220a-00ef-4d13-abba-56f87636fb5f 1 2020-11-28T21:46:14.050 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.047+0000 7fc253461700 1 Processor -- start 2020-11-28T21:46:14.051 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.047+0000 7fc253461700 1 -- start start 2020-11-28T21:46:14.051 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.047+0000 7fc253461700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc24c0b1240 0x7fc24c0ad790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:14.051 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.047+0000 7fc253461700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc24c1530e0 0x7fc24c153550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:14.052 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.047+0000 7fc253461700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc24c0638e0 con 0x7fc24c05f380 2020-11-28T21:46:14.052 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.047+0000 7fc253461700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc24c0b3760 con 0x7fc24c0b1240 2020-11-28T21:46:14.052 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.047+0000 7fc253461700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c0b1620 con 0x7fc24c1530e0 2020-11-28T21:46:14.052 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.047+0000 7fc2546c3700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc24c1530e0 0x7fc24c153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:14.053 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.047+0000 7fc25245f700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fc24c05f380 0x7fc24c0adc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:55730/0 (socket says 172.21.15.118:55730) 2020-11-28T21:46:14.053 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.047+0000 7fc25245f700 1 -- 172.21.15.118:0/1794056172 learned_addr learned my addr 172.21.15.118:0/1794056172 (peer_addr_for_me v1:172.21.15.118:0/0) 2020-11-28T21:46:14.053 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.047+0000 7fc252c60700 1 --2- 172.21.15.118:0/1794056172 >> v2:172.21.15.162:3300/0 conn(0x7fc24c0b1240 0x7fc24c0ad790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:14.054 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.047+0000 7fc251c5e700 1 -- 172.21.15.118:0/1794056172 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4144513539 0 0) 0x7fc24c0638e0 con 0x7fc24c05f380 2020-11-28T21:46:14.054 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.047+0000 7fc251c5e700 1 -- 172.21.15.118:0/1794056172 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc234003230 con 0x7fc24c05f380 2020-11-28T21:46:14.054 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.047+0000 7fc2546c3700 1 -- 172.21.15.118:0/1794056172 >> v2:172.21.15.162:3300/0 conn(0x7fc24c0b1240 msgr2=0x7fc24c0ad790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:14.055 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.047+0000 7fc2546c3700 1 --2- 172.21.15.118:0/1794056172 >> v2:172.21.15.162:3300/0 conn(0x7fc24c0b1240 0x7fc24c0ad790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:14.055 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.047+0000 7fc2546c3700 1 -- 172.21.15.118:0/1794056172 >> v1:172.21.15.118:6789/0 conn(0x7fc24c05f380 legacy=0x7fc24c0adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:14.055 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.047+0000 7fc2546c3700 1 -- 172.21.15.118:0/1794056172 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc24c0605b0 con 0x7fc24c1530e0 2020-11-28T21:46:14.055 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.047+0000 7fc2546c3700 1 --2- 172.21.15.118:0/1794056172 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc24c1530e0 0x7fc24c153550 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fc2440032b0 tx=0x7fc2440091a0).ready entity=mon.2 client_cookie=f0844fc1bb12db47 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:14.056 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.047+0000 7fc251c5e700 1 -- 172.21.15.118:0/1794056172 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc24400b9a0 con 0x7fc24c1530e0 2020-11-28T21:46:14.056 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.047+0000 7fc251c5e700 1 -- 172.21.15.118:0/1794056172 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc24400bb00 con 0x7fc24c1530e0 2020-11-28T21:46:14.056 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.047+0000 7fc251c5e700 1 -- 172.21.15.118:0/1794056172 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc24400bc70 con 0x7fc24c1530e0 2020-11-28T21:46:14.057 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.047+0000 7fc253461700 1 -- 172.21.15.118:0/1794056172 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc24c1530e0 msgr2=0x7fc24c153550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:14.057 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.047+0000 7fc253461700 1 --2- 172.21.15.118:0/1794056172 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc24c1530e0 0x7fc24c153550 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fc2440032b0 tx=0x7fc2440091a0).stop 2020-11-28T21:46:14.057 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.047+0000 7fc253461700 1 -- 172.21.15.118:0/1794056172 shutdown_connections 2020-11-28T21:46:14.058 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.047+0000 7fc253461700 1 --2- 172.21.15.118:0/1794056172 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc24c1530e0 0x7fc24c153550 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:14.058 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.047+0000 7fc253461700 1 --2- 172.21.15.118:0/1794056172 >> v2:172.21.15.162:3300/0 conn(0x7fc24c0b1240 0x7fc24c0ad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:14.058 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.047+0000 7fc253461700 1 -- 172.21.15.118:0/1794056172 >> 172.21.15.118:0/1794056172 conn(0x7fc24c063d80 msgr2=0x7fc24c064190 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:14.058 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.047+0000 7fc253461700 1 -- 172.21.15.118:0/1794056172 shutdown_connections 2020-11-28T21:46:14.059 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.047+0000 7fc253461700 1 -- 172.21.15.118:0/1794056172 wait complete. 2020-11-28T21:46:14.059 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.051+0000 7fc253461700 1 Processor -- start 2020-11-28T21:46:14.059 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.051+0000 7fc253461700 1 -- start start 2020-11-28T21:46:14.063 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.051+0000 7fc253461700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc24c05f380 0x7fc24c153830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:14.063 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.051+0000 7fc253461700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc24c0b1240 0x7fc24c064a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:14.064 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.051+0000 7fc253461700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc24c1530e0 0x7fc24c0670a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:14.064 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.051+0000 7fc253461700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c0669d0 con 0x7fc24c0b1240 2020-11-28T21:46:14.064 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.051+0000 7fc253461700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c155940 con 0x7fc24c1530e0 2020-11-28T21:46:14.064 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.051+0000 7fc253461700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c0ae610 con 0x7fc24c05f380 2020-11-28T21:46:14.065 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.051+0000 7fc25245f700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc24c0b1240 0x7fc24c064a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:14.065 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.051+0000 7fc252c60700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc24c05f380 0x7fc24c153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:14.065 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.051+0000 7fc25245f700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc24c0b1240 0x7fc24c064a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:57950/0 (socket says 172.21.15.118:57950) 2020-11-28T21:46:14.066 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.051+0000 7fc25245f700 1 -- 172.21.15.118:0/1172457607 learned_addr learned my addr 172.21.15.118:0/1172457607 (peer_addr_for_me v2:172.21.15.118:0/0) 2020-11-28T21:46:14.066 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.051+0000 7fc2546c3700 1 --2- 172.21.15.118:0/1172457607 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc24c1530e0 0x7fc24c0670a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:14.067 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.051+0000 7fc252c60700 1 -- 172.21.15.118:0/1172457607 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc24c1530e0 msgr2=0x7fc24c0670a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:14.067 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.051+0000 7fc252c60700 1 --2- 172.21.15.118:0/1172457607 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc24c1530e0 0x7fc24c0670a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:14.068 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.051+0000 7fc252c60700 1 -- 172.21.15.118:0/1172457607 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc24c0b1240 msgr2=0x7fc24c064a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:14.068 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.051+0000 7fc252c60700 1 --2- 172.21.15.118:0/1172457607 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc24c0b1240 0x7fc24c064a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:14.068 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.051+0000 7fc252c60700 1 -- 172.21.15.118:0/1172457607 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc24c066c00 con 0x7fc24c05f380 2020-11-28T21:46:14.068 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.051+0000 7fc252c60700 1 --2- 172.21.15.118:0/1172457607 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc24c05f380 0x7fc24c153830 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fc24000a000 tx=0x7fc240008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:14.068 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.051+0000 7fc24b7fe700 1 -- 172.21.15.118:0/1172457607 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc24000f7e0 con 0x7fc24c05f380 2020-11-28T21:46:14.069 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.051+0000 7fc24b7fe700 1 -- 172.21.15.118:0/1172457607 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc24000f940 con 0x7fc24c05f380 2020-11-28T21:46:14.069 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.051+0000 7fc253461700 1 -- 172.21.15.118:0/1172457607 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc24c0aec40 con 0x7fc24c05f380 2020-11-28T21:46:14.069 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.051+0000 7fc24b7fe700 1 -- 172.21.15.118:0/1172457607 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc24000fca0 con 0x7fc24c05f380 2020-11-28T21:46:14.069 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.051+0000 7fc253461700 1 -- 172.21.15.118:0/1172457607 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc24c066670 con 0x7fc24c05f380 2020-11-28T21:46:14.070 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.051+0000 7fc24b7fe700 1 -- 172.21.15.118:0/1172457607 <== mon.2 v2:172.21.15.118:3301/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7fc240013480 con 0x7fc24c05f380 2020-11-28T21:46:14.070 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.051+0000 7fc24b7fe700 1 -- 172.21.15.118:0/1172457607 <== mon.2 v2:172.21.15.118:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 869+0+0 (secure 0 0 0) 0x7fc24001c3f0 con 0x7fc24c05f380 2020-11-28T21:46:14.070 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.055+0000 7fc253461700 1 -- 172.21.15.118:0/1172457607 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc24c057050 con 0x7fc24c05f380 2020-11-28T21:46:14.070 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.055+0000 7fc24b7fe700 1 -- 172.21.15.118:0/1172457607 <== mon.2 v2:172.21.15.118:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7fc240021060 con 0x7fc24c05f380 2020-11-28T21:46:14.135 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.135+0000 7fc253461700 1 -- 172.21.15.118:0/1172457607 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "osd new", "uuid": "6ce6220a-00ef-4d13-abba-56f87636fb5f", "id": 1} v 0) v1 -- 0x7fc24c044350 con 0x7fc24c05f380 2020-11-28T21:46:14.140 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.139+0000 7fc24b7fe700 1 -- 172.21.15.118:0/1172457607 <== mon.2 v2:172.21.15.118:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6ce6220a-00ef-4d13-abba-56f87636fb5f", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc240018070 con 0x7fc24c05f380 2020-11-28T21:46:14.141 INFO:teuthology.orchestra.run.smithi118.stdout:1 2020-11-28T21:46:14.142 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.143+0000 7fc253461700 1 -- 172.21.15.118:0/1172457607 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc24c05f380 msgr2=0x7fc24c153830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:14.142 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.143+0000 7fc253461700 1 --2- 172.21.15.118:0/1172457607 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc24c05f380 0x7fc24c153830 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fc24000a000 tx=0x7fc240008040).stop 2020-11-28T21:46:14.143 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.143+0000 7fc253461700 1 -- 172.21.15.118:0/1172457607 shutdown_connections 2020-11-28T21:46:14.143 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.143+0000 7fc253461700 1 --2- 172.21.15.118:0/1172457607 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc24c05f380 0x7fc24c153830 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:14.143 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.143+0000 7fc253461700 1 --2- 172.21.15.118:0/1172457607 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc24c1530e0 0x7fc24c0670a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:14.143 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.143+0000 7fc253461700 1 --2- 172.21.15.118:0/1172457607 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc24c0b1240 0x7fc24c064a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:14.144 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.143+0000 7fc253461700 1 -- 172.21.15.118:0/1172457607 >> 172.21.15.118:0/1172457607 conn(0x7fc24c063d80 msgr2=0x7fc24c12a870 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:14.144 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.143+0000 7fc253461700 1 -- 172.21.15.118:0/1172457607 shutdown_connections 2020-11-28T21:46:14.144 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.143+0000 7fc253461700 1 -- 172.21.15.118:0/1172457607 wait complete. 2020-11-28T21:46:14.154 INFO:teuthology.orchestra.run.smithi118:> sudo ceph --cluster ceph osd new db4e510d-27c1-47fc-a902-8b32e7dfd73e 2 2020-11-28T21:46:14.266 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.263+0000 7f000091f700 1 Processor -- start 2020-11-28T21:46:14.267 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.263+0000 7f000091f700 1 -- start start 2020-11-28T21:46:14.267 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.263+0000 7f000091f700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7efffc0b1240 0x7efffc0ad790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:14.267 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.263+0000 7f000091f700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7efffc05f380 0x7efffc0adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:14.268 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.263+0000 7f000091f700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efffc0638e0 con 0x7efffc024f80 2020-11-28T21:46:14.268 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.263+0000 7f000091f700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efffc0b3760 con 0x7efffc05f380 2020-11-28T21:46:14.268 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.263+0000 7f000091f700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efffc0b1620 con 0x7efffc0b1240 2020-11-28T21:46:14.268 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.263+0000 7efffad9d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7efffc05f380 0x7efffc0adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:14.269 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.263+0000 7efffad9d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7efffc05f380 0x7efffc0adc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:57958/0 (socket says 172.21.15.118:57958) 2020-11-28T21:46:14.269 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.263+0000 7efffad9d700 1 -- 172.21.15.118:0/500489833 learned_addr learned my addr 172.21.15.118:0/500489833 (peer_addr_for_me v2:172.21.15.118:0/0) 2020-11-28T21:46:14.269 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.263+0000 7efffb59e700 1 --2- 172.21.15.118:0/500489833 >> v2:172.21.15.162:3300/0 conn(0x7efffc0b1240 0x7efffc0ad790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:14.269 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.263+0000 7efffad9d700 1 -- 172.21.15.118:0/500489833 >> v2:172.21.15.162:3300/0 conn(0x7efffc0b1240 msgr2=0x7efffc0ad790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:14.269 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.263+0000 7efffad9d700 1 --2- 172.21.15.118:0/500489833 >> v2:172.21.15.162:3300/0 conn(0x7efffc0b1240 0x7efffc0ad790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:14.270 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.263+0000 7efffad9d700 1 -- 172.21.15.118:0/500489833 >> v1:172.21.15.162:6789/0 conn(0x7efffc024f80 legacy=0x7efffc0253f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:14.270 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.263+0000 7efffad9d700 1 -- 172.21.15.118:0/500489833 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efffc0605b0 con 0x7efffc05f380 2020-11-28T21:46:14.271 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.263+0000 7efffad9d700 1 --2- 172.21.15.118:0/500489833 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efffc05f380 0x7efffc0adc80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7effec002f30 tx=0x7effec006e80).ready entity=mon.0 client_cookie=6de67b0c35986e71 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:14.271 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.267+0000 7efffa59c700 1 -- 172.21.15.118:0/500489833 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effec0097f0 con 0x7efffc05f380 2020-11-28T21:46:14.271 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.267+0000 7efffa59c700 1 -- 172.21.15.118:0/500489833 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effec009950 con 0x7efffc05f380 2020-11-28T21:46:14.272 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.267+0000 7efffa59c700 1 -- 172.21.15.118:0/500489833 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effec009cb0 con 0x7efffc05f380 2020-11-28T21:46:14.272 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.267+0000 7f000091f700 1 -- 172.21.15.118:0/500489833 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efffc05f380 msgr2=0x7efffc0adc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:14.273 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.267+0000 7f000091f700 1 --2- 172.21.15.118:0/500489833 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efffc05f380 0x7efffc0adc80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7effec002f30 tx=0x7effec006e80).stop 2020-11-28T21:46:14.273 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.267+0000 7f000091f700 1 -- 172.21.15.118:0/500489833 shutdown_connections 2020-11-28T21:46:14.274 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.267+0000 7f000091f700 1 --2- 172.21.15.118:0/500489833 >> v2:172.21.15.162:3300/0 conn(0x7efffc0b1240 0x7efffc0ad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:14.274 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.267+0000 7f000091f700 1 --2- 172.21.15.118:0/500489833 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efffc05f380 0x7efffc0adc80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:14.274 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.267+0000 7f000091f700 1 -- 172.21.15.118:0/500489833 >> 172.21.15.118:0/500489833 conn(0x7efffc063d80 msgr2=0x7efffc064190 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:14.275 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.267+0000 7f000091f700 1 -- 172.21.15.118:0/500489833 shutdown_connections 2020-11-28T21:46:14.275 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.267+0000 7f000091f700 1 -- 172.21.15.118:0/500489833 wait complete. 2020-11-28T21:46:14.280 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.267+0000 7f000091f700 1 Processor -- start 2020-11-28T21:46:14.281 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.267+0000 7f000091f700 1 -- start start 2020-11-28T21:46:14.281 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.267+0000 7f000091f700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efffc024f80 0x7efffc07e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:14.281 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.267+0000 7f000091f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efffc05f380 0x7efffc0663d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:14.282 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.267+0000 7f000091f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efffc0b1240 0x7efffc0668c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:14.282 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.267+0000 7f000091f700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efffc12e140 con 0x7efffc024f80 2020-11-28T21:46:14.283 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.267+0000 7f000091f700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efffc066db0 con 0x7efffc0b1240 2020-11-28T21:46:14.283 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.267+0000 7f000091f700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efffc0661e0 con 0x7efffc05f380 2020-11-28T21:46:14.284 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.267+0000 7efffb59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efffc024f80 0x7efffc07e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:14.284 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.267+0000 7efffb59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efffc024f80 0x7efffc07e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:57962/0 (socket says 172.21.15.118:57962) 2020-11-28T21:46:14.284 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.267+0000 7efffb59e700 1 -- 172.21.15.118:0/1543730389 learned_addr learned my addr 172.21.15.118:0/1543730389 (peer_addr_for_me v2:172.21.15.118:0/0) 2020-11-28T21:46:14.285 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.271+0000 7efffad9d700 1 --2- 172.21.15.118:0/1543730389 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efffc05f380 0x7efffc0663d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:14.285 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.271+0000 7f0001120700 1 --2- 172.21.15.118:0/1543730389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efffc0b1240 0x7efffc0668c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:14.286 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.271+0000 7efffb59e700 1 -- 172.21.15.118:0/1543730389 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efffc05f380 msgr2=0x7efffc0663d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:14.286 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.271+0000 7efffb59e700 1 --2- 172.21.15.118:0/1543730389 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efffc05f380 0x7efffc0663d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:14.286 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.271+0000 7efffb59e700 1 -- 172.21.15.118:0/1543730389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efffc0b1240 msgr2=0x7efffc0668c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:14.287 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.271+0000 7efffb59e700 1 --2- 172.21.15.118:0/1543730389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efffc0b1240 0x7efffc0668c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:14.287 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.271+0000 7efffb59e700 1 -- 172.21.15.118:0/1543730389 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efffc130550 con 0x7efffc024f80 2020-11-28T21:46:14.288 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.271+0000 7efffb59e700 1 --2- 172.21.15.118:0/1543730389 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efffc024f80 0x7efffc07e380 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7effe400c000 tx=0x7effe4006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:14.288 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.271+0000 7efff3fff700 1 -- 172.21.15.118:0/1543730389 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effe400ca10 con 0x7efffc024f80 2020-11-28T21:46:14.289 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.271+0000 7efff3fff700 1 -- 172.21.15.118:0/1543730389 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effe400cb70 con 0x7efffc024f80 2020-11-28T21:46:14.289 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.271+0000 7efff3fff700 1 -- 172.21.15.118:0/1543730389 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effe400be50 con 0x7efffc024f80 2020-11-28T21:46:14.289 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.271+0000 7f000091f700 1 -- 172.21.15.118:0/1543730389 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efffc1307f0 con 0x7efffc024f80 2020-11-28T21:46:14.290 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.271+0000 7f000091f700 1 -- 172.21.15.118:0/1543730389 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efffc0b43d0 con 0x7efffc024f80 2020-11-28T21:46:14.290 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.271+0000 7efff3fff700 1 -- 172.21.15.118:0/1543730389 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7effe401b5e0 con 0x7efffc024f80 2020-11-28T21:46:14.291 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.271+0000 7efff3fff700 1 -- 172.21.15.118:0/1543730389 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 984+0+0 (secure 0 0 0) 0x7effe401b870 con 0x7efffc024f80 2020-11-28T21:46:14.291 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.271+0000 7f000091f700 1 -- 172.21.15.118:0/1543730389 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efffc0b45d0 con 0x7efffc024f80 2020-11-28T21:46:14.291 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.275+0000 7efff3fff700 1 -- 172.21.15.118:0/1543730389 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7efffc0b45d0 con 0x7efffc024f80 2020-11-28T21:46:14.352 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.351+0000 7f000091f700 1 -- 172.21.15.118:0/1543730389 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd new", "uuid": "db4e510d-27c1-47fc-a902-8b32e7dfd73e", "id": 2} v 0) v1 -- 0x7efffc130c90 con 0x7efffc024f80 2020-11-28T21:46:14.356 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.355+0000 7efff3fff700 1 -- 172.21.15.118:0/1543730389 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "db4e510d-27c1-47fc-a902-8b32e7dfd73e", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7effe401bcd0 con 0x7efffc024f80 2020-11-28T21:46:14.357 INFO:teuthology.orchestra.run.smithi118.stdout:2 2020-11-28T21:46:14.358 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.359+0000 7f000091f700 1 -- 172.21.15.118:0/1543730389 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efffc024f80 msgr2=0x7efffc07e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:14.358 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.359+0000 7f000091f700 1 --2- 172.21.15.118:0/1543730389 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efffc024f80 0x7efffc07e380 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7effe400c000 tx=0x7effe4006dc0).stop 2020-11-28T21:46:14.359 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.359+0000 7f000091f700 1 -- 172.21.15.118:0/1543730389 shutdown_connections 2020-11-28T21:46:14.359 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.359+0000 7f000091f700 1 --2- 172.21.15.118:0/1543730389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efffc0b1240 0x7efffc0668c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:14.359 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.359+0000 7f000091f700 1 --2- 172.21.15.118:0/1543730389 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efffc024f80 0x7efffc07e380 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:14.359 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.359+0000 7f000091f700 1 --2- 172.21.15.118:0/1543730389 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efffc05f380 0x7efffc0663d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:14.360 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.359+0000 7f000091f700 1 -- 172.21.15.118:0/1543730389 >> 172.21.15.118:0/1543730389 conn(0x7efffc063d80 msgr2=0x7efffc0b5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:14.360 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.359+0000 7f000091f700 1 -- 172.21.15.118:0/1543730389 shutdown_connections 2020-11-28T21:46:14.360 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.359+0000 7f000091f700 1 -- 172.21.15.118:0/1543730389 wait complete. 2020-11-28T21:46:14.369 INFO:teuthology.orchestra.run.smithi118:> sudo ceph --cluster ceph osd new 4094cb87-f04a-45e1-8710-fb4b651949f6 3 2020-11-28T21:46:14.484 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.479+0000 7f49a357d700 1 Processor -- start 2020-11-28T21:46:14.485 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.479+0000 7f49a357d700 1 -- start start 2020-11-28T21:46:14.485 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.479+0000 7f49a357d700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f499c0b1240 0x7f499c0ad790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:14.485 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.479+0000 7f49a357d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f499c1530e0 0x7f499c153530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:14.485 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.479+0000 7f49a357d700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f499c0638e0 con 0x7f499c05f380 2020-11-28T21:46:14.485 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.479+0000 7f49a357d700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f499c0b3760 con 0x7f499c1530e0 2020-11-28T21:46:14.486 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.479+0000 7f49a357d700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f499c0b1620 con 0x7f499c0b1240 2020-11-28T21:46:14.486 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.483+0000 7f49a47df700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f499c1530e0 0x7f499c153530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:14.486 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.483+0000 7f49a47df700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f499c1530e0 0x7f499c153530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:57970/0 (socket says 172.21.15.118:57970) 2020-11-28T21:46:14.486 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.483+0000 7f49a47df700 1 -- 172.21.15.118:0/956658946 learned_addr learned my addr 172.21.15.118:0/956658946 (peer_addr_for_me v2:172.21.15.118:0/0) 2020-11-28T21:46:14.487 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.483+0000 7f49a2d7c700 1 --2- 172.21.15.118:0/956658946 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f499c0b1240 0x7f499c0ad790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:14.487 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.483+0000 7f49a47df700 1 -- 172.21.15.118:0/956658946 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f499c0b1240 msgr2=0x7f499c0ad790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:14.487 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.483+0000 7f49a47df700 1 --2- 172.21.15.118:0/956658946 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f499c0b1240 0x7f499c0ad790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:14.487 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.483+0000 7f49a47df700 1 -- 172.21.15.118:0/956658946 >> v1:172.21.15.162:6789/0 conn(0x7f499c05f380 legacy=0x7f499c0adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:14.487 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.483+0000 7f49a47df700 1 -- 172.21.15.118:0/956658946 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f499c0605b0 con 0x7f499c1530e0 2020-11-28T21:46:14.488 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.483+0000 7f49a2d7c700 1 --2- 172.21.15.118:0/956658946 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f499c0b1240 0x7f499c0ad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:46:14.488 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.483+0000 7f49a47df700 1 --2- 172.21.15.118:0/956658946 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f499c1530e0 0x7f499c153530 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f4994002f30 tx=0x7f4994006e80).ready entity=mon.0 client_cookie=c36dfb8dc587785a server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:14.488 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.483+0000 7f49a1d7a700 1 -- 172.21.15.118:0/956658946 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49940097f0 con 0x7f499c1530e0 2020-11-28T21:46:14.488 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.483+0000 7f49a1d7a700 1 -- 172.21.15.118:0/956658946 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4994009950 con 0x7f499c1530e0 2020-11-28T21:46:14.488 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.483+0000 7f49a1d7a700 1 -- 172.21.15.118:0/956658946 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4994009cb0 con 0x7f499c1530e0 2020-11-28T21:46:14.489 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.483+0000 7f49a357d700 1 -- 172.21.15.118:0/956658946 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f499c1530e0 msgr2=0x7f499c153530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:14.489 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.483+0000 7f49a357d700 1 --2- 172.21.15.118:0/956658946 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f499c1530e0 0x7f499c153530 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f4994002f30 tx=0x7f4994006e80).stop 2020-11-28T21:46:14.489 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.483+0000 7f49a357d700 1 -- 172.21.15.118:0/956658946 shutdown_connections 2020-11-28T21:46:14.489 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.483+0000 7f49a357d700 1 --2- 172.21.15.118:0/956658946 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f499c1530e0 0x7f499c153530 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:14.489 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.483+0000 7f49a357d700 1 --2- 172.21.15.118:0/956658946 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f499c0b1240 0x7f499c0ad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:14.490 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.483+0000 7f49a357d700 1 -- 172.21.15.118:0/956658946 >> 172.21.15.118:0/956658946 conn(0x7f499c063d80 msgr2=0x7f499c064190 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:14.490 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.483+0000 7f49a357d700 1 -- 172.21.15.118:0/956658946 shutdown_connections 2020-11-28T21:46:14.490 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.483+0000 7f49a357d700 1 -- 172.21.15.118:0/956658946 wait complete. 2020-11-28T21:46:14.492 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.487+0000 7f49a357d700 1 Processor -- start 2020-11-28T21:46:14.492 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.487+0000 7f49a357d700 1 -- start start 2020-11-28T21:46:14.493 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.487+0000 7f49a357d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f499c05f380 0x7f499c0af9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:14.493 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.487+0000 7f49a357d700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f499c0b1240 0x7f499c129e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:14.493 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.487+0000 7f49a357d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f499c1530e0 0x7f499c064a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:14.493 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.487+0000 7f49a357d700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499c128ba0 con 0x7f499c1530e0 2020-11-28T21:46:14.493 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.487+0000 7f49a357d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499c129740 con 0x7f499c05f380 2020-11-28T21:46:14.494 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.487+0000 7f49a357d700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f499c1299a0 con 0x7f499c0b1240 2020-11-28T21:46:14.494 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.487+0000 7f49a257b700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f499c0b1240 0x7f499c129e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:14.494 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.487+0000 7f49a257b700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f499c0b1240 0x7f499c129e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.118:41422/0 (socket says 172.21.15.118:41422) 2020-11-28T21:46:14.494 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.487+0000 7f49a257b700 1 -- 172.21.15.118:0/1308792831 learned_addr learned my addr 172.21.15.118:0/1308792831 (peer_addr_for_me v2:172.21.15.118:0/0) 2020-11-28T21:46:14.494 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.487+0000 7f49a47df700 1 --2- 172.21.15.118:0/1308792831 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f499c1530e0 0x7f499c064a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:14.495 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.487+0000 7f49a2d7c700 1 --2- 172.21.15.118:0/1308792831 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f499c05f380 0x7f499c0af9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:14.495 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.487+0000 7f49a257b700 1 -- 172.21.15.118:0/1308792831 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f499c05f380 msgr2=0x7f499c0af9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:14.495 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.487+0000 7f49a257b700 1 --2- 172.21.15.118:0/1308792831 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f499c05f380 0x7f499c0af9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:14.495 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.487+0000 7f49a257b700 1 -- 172.21.15.118:0/1308792831 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f499c1530e0 msgr2=0x7f499c064a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:14.495 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.487+0000 7f49a257b700 1 --2- 172.21.15.118:0/1308792831 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f499c1530e0 0x7f499c064a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:14.496 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.487+0000 7f49a257b700 1 -- 172.21.15.118:0/1308792831 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f499c128dd0 con 0x7f499c0b1240 2020-11-28T21:46:14.496 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.487+0000 7f49a257b700 1 --2- 172.21.15.118:0/1308792831 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f499c0b1240 0x7f499c129e70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f499c0669c0 tx=0x7f4998007f30).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:14.496 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.487+0000 7f49937fe700 1 -- 172.21.15.118:0/1308792831 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f499800ce70 con 0x7f499c0b1240 2020-11-28T21:46:14.496 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.487+0000 7f49937fe700 1 -- 172.21.15.118:0/1308792831 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4998006a70 con 0x7f499c0b1240 2020-11-28T21:46:14.496 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.487+0000 7f49a357d700 1 -- 172.21.15.118:0/1308792831 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f499c067890 con 0x7f499c0b1240 2020-11-28T21:46:14.497 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.487+0000 7f49937fe700 1 -- 172.21.15.118:0/1308792831 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4998006dd0 con 0x7f499c0b1240 2020-11-28T21:46:14.497 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.487+0000 7f49a357d700 1 -- 172.21.15.118:0/1308792831 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f499c065150 con 0x7f499c0b1240 2020-11-28T21:46:14.497 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.487+0000 7f49937fe700 1 -- 172.21.15.118:0/1308792831 <== mon.2 v2:172.21.15.118:3301/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f4998019030 con 0x7f499c0b1240 2020-11-28T21:46:14.497 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.487+0000 7f49937fe700 1 -- 172.21.15.118:0/1308792831 <== mon.2 v2:172.21.15.118:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1099+0+0 (secure 0 0 0) 0x7f499800aa20 con 0x7f499c0b1240 2020-11-28T21:46:14.497 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.491+0000 7f49a357d700 1 -- 172.21.15.118:0/1308792831 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f499c129ba0 con 0x7f499c0b1240 2020-11-28T21:46:14.498 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.495+0000 7f49937fe700 1 -- 172.21.15.118:0/1308792831 <== mon.2 v2:172.21.15.118:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f4998017070 con 0x7f499c0b1240 2020-11-28T21:46:14.571 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.571+0000 7f49a357d700 1 -- 172.21.15.118:0/1308792831 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "osd new", "uuid": "4094cb87-f04a-45e1-8710-fb4b651949f6", "id": 3} v 0) v1 -- 0x7f499c057050 con 0x7f499c0b1240 2020-11-28T21:46:14.577 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.579+0000 7f49937fe700 1 -- 172.21.15.118:0/1308792831 <== mon.2 v2:172.21.15.118:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4094cb87-f04a-45e1-8710-fb4b651949f6", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f499800ac30 con 0x7f499c0b1240 2020-11-28T21:46:14.578 INFO:teuthology.orchestra.run.smithi118.stdout:3 2020-11-28T21:46:14.579 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.579+0000 7f49a357d700 1 -- 172.21.15.118:0/1308792831 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f499c0b1240 msgr2=0x7f499c129e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:14.579 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.579+0000 7f49a357d700 1 --2- 172.21.15.118:0/1308792831 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f499c0b1240 0x7f499c129e70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f499c0669c0 tx=0x7f4998007f30).stop 2020-11-28T21:46:14.579 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.579+0000 7f49a357d700 1 -- 172.21.15.118:0/1308792831 shutdown_connections 2020-11-28T21:46:14.580 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.579+0000 7f49a357d700 1 --2- 172.21.15.118:0/1308792831 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f499c1530e0 0x7f499c064a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:14.580 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.579+0000 7f49a357d700 1 --2- 172.21.15.118:0/1308792831 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f499c05f380 0x7f499c0af9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:14.580 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.579+0000 7f49a357d700 1 --2- 172.21.15.118:0/1308792831 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f499c0b1240 0x7f499c129e70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:14.580 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.579+0000 7f49a357d700 1 -- 172.21.15.118:0/1308792831 >> 172.21.15.118:0/1308792831 conn(0x7f499c063d80 msgr2=0x7f499c0b1760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:14.581 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.579+0000 7f49a357d700 1 -- 172.21.15.118:0/1308792831 shutdown_connections 2020-11-28T21:46:14.581 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.579+0000 7f49a357d700 1 -- 172.21.15.118:0/1308792831 wait complete. 2020-11-28T21:46:14.590 INFO:teuthology.orchestra.run.smithi118:> sudo ceph --cluster ceph osd new b907bf53-c297-4756-847b-3be96680eef5 4 2020-11-28T21:46:14.697 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.695+0000 7f2a3993b700 1 Processor -- start 2020-11-28T21:46:14.698 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.695+0000 7f2a3993b700 1 -- start start 2020-11-28T21:46:14.698 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.695+0000 7f2a3993b700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f2a3405f380 0x7f2a340adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:14.698 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.695+0000 7f2a3993b700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a340638e0 con 0x7f2a340b1240 2020-11-28T21:46:14.699 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.695+0000 7f2a3993b700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a340605b0 con 0x7f2a341530e0 2020-11-28T21:46:14.699 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.695+0000 7f2a3993b700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a340b3760 con 0x7f2a3405f380 2020-11-28T21:46:14.699 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.695+0000 7f2a3913a700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f2a340b1240 0x7f2a340ad790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:55764/0 (socket says 172.21.15.118:55764) 2020-11-28T21:46:14.699 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.695+0000 7f2a3913a700 1 -- 172.21.15.118:0/854613970 learned_addr learned my addr 172.21.15.118:0/854613970 (peer_addr_for_me v1:172.21.15.118:0/0) 2020-11-28T21:46:14.699 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.695+0000 7f2a38939700 1 --2- 172.21.15.118:0/854613970 >> v2:172.21.15.118:3300/0 conn(0x7f2a3405f380 0x7f2a340adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:14.700 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.695+0000 7f2a33fff700 1 -- 172.21.15.118:0/854613970 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3364561738 0 0) 0x7f2a340638e0 con 0x7f2a340b1240 2020-11-28T21:46:14.700 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.695+0000 7f2a33fff700 1 -- 172.21.15.118:0/854613970 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a1c003230 con 0x7f2a340b1240 2020-11-28T21:46:14.700 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.695+0000 7f2a38939700 1 -- 172.21.15.118:0/854613970 >> v1:172.21.15.162:6789/0 conn(0x7f2a341530e0 legacy=0x7f2a34153550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:14.700 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.695+0000 7f2a38939700 1 -- 172.21.15.118:0/854613970 >> v1:172.21.15.118:6789/0 conn(0x7f2a340b1240 legacy=0x7f2a340ad790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:14.701 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.695+0000 7f2a38939700 1 -- 172.21.15.118:0/854613970 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a340b1620 con 0x7f2a3405f380 2020-11-28T21:46:14.701 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.695+0000 7f2a38939700 1 --2- 172.21.15.118:0/854613970 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2a3405f380 0x7f2a340adea0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f2a28002f30 tx=0x7f2a28006dc0).ready entity=mon.0 client_cookie=a599d8de987542df server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:14.701 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.695+0000 7f2a33fff700 1 -- 172.21.15.118:0/854613970 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a28009800 con 0x7f2a3405f380 2020-11-28T21:46:14.701 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.695+0000 7f2a33fff700 1 -- 172.21.15.118:0/854613970 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a28009960 con 0x7f2a3405f380 2020-11-28T21:46:14.702 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.695+0000 7f2a33fff700 1 -- 172.21.15.118:0/854613970 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a28009cc0 con 0x7f2a3405f380 2020-11-28T21:46:14.702 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.695+0000 7f2a3993b700 1 -- 172.21.15.118:0/854613970 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2a3405f380 msgr2=0x7f2a340adea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:14.702 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.695+0000 7f2a3993b700 1 --2- 172.21.15.118:0/854613970 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2a3405f380 0x7f2a340adea0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f2a28002f30 tx=0x7f2a28006dc0).stop 2020-11-28T21:46:14.702 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.695+0000 7f2a3993b700 1 -- 172.21.15.118:0/854613970 shutdown_connections 2020-11-28T21:46:14.702 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.695+0000 7f2a3993b700 1 --2- 172.21.15.118:0/854613970 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2a3405f380 0x7f2a340adea0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:14.703 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.695+0000 7f2a3993b700 1 -- 172.21.15.118:0/854613970 >> 172.21.15.118:0/854613970 conn(0x7f2a34063d80 msgr2=0x7f2a34064190 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:14.703 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.695+0000 7f2a3993b700 1 -- 172.21.15.118:0/854613970 shutdown_connections 2020-11-28T21:46:14.703 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.695+0000 7f2a3993b700 1 -- 172.21.15.118:0/854613970 wait complete. 2020-11-28T21:46:14.705 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.699+0000 7f2a3993b700 1 Processor -- start 2020-11-28T21:46:14.706 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.699+0000 7f2a3993b700 1 -- start start 2020-11-28T21:46:14.706 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.699+0000 7f2a3993b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2a3405f380 0x7f2a340a92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:14.706 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.699+0000 7f2a3993b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a340b1240 0x7f2a341413e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:14.706 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.699+0000 7f2a3993b700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2a341530e0 0x7f2a340adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:14.707 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.699+0000 7f2a3993b700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a340ad810 con 0x7f2a3405f380 2020-11-28T21:46:14.707 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.699+0000 7f2a3993b700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a34132110 con 0x7f2a340b1240 2020-11-28T21:46:14.707 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.699+0000 7f2a3993b700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a34140a00 con 0x7f2a341530e0 2020-11-28T21:46:14.707 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.699+0000 7f2a3913a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2a3405f380 0x7f2a340a92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:14.707 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.699+0000 7f2a3913a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2a3405f380 0x7f2a340a92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:57986/0 (socket says 172.21.15.118:57986) 2020-11-28T21:46:14.708 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.699+0000 7f2a3913a700 1 -- 172.21.15.118:0/3432050897 learned_addr learned my addr 172.21.15.118:0/3432050897 (peer_addr_for_me v2:172.21.15.118:0/0) 2020-11-28T21:46:14.708 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.699+0000 7f2a3ab9d700 1 --2- 172.21.15.118:0/3432050897 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2a341530e0 0x7f2a340adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:14.708 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.699+0000 7f2a38939700 1 --2- 172.21.15.118:0/3432050897 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a340b1240 0x7f2a341413e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:14.708 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.699+0000 7f2a3913a700 1 -- 172.21.15.118:0/3432050897 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2a341530e0 msgr2=0x7f2a340adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:14.708 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.699+0000 7f2a3913a700 1 --2- 172.21.15.118:0/3432050897 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2a341530e0 0x7f2a340adfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:14.709 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.699+0000 7f2a3913a700 1 -- 172.21.15.118:0/3432050897 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a340b1240 msgr2=0x7f2a341413e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:14.709 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.699+0000 7f2a3913a700 1 --2- 172.21.15.118:0/3432050897 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a340b1240 0x7f2a341413e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:14.709 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.699+0000 7f2a3913a700 1 -- 172.21.15.118:0/3432050897 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a34140ca0 con 0x7f2a3405f380 2020-11-28T21:46:14.709 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.699+0000 7f2a38939700 1 --2- 172.21.15.118:0/3432050897 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a340b1240 0x7f2a341413e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:46:14.709 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.699+0000 7f2a3913a700 1 --2- 172.21.15.118:0/3432050897 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2a3405f380 0x7f2a340a92a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f2a24001450 tx=0x7f2a24007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:14.710 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.699+0000 7f2a31ffb700 1 -- 172.21.15.118:0/3432050897 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a24005b70 con 0x7f2a3405f380 2020-11-28T21:46:14.710 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.699+0000 7f2a3993b700 1 -- 172.21.15.118:0/3432050897 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a34140f40 con 0x7f2a3405f380 2020-11-28T21:46:14.710 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.699+0000 7f2a31ffb700 1 -- 172.21.15.118:0/3432050897 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a24005cd0 con 0x7f2a3405f380 2020-11-28T21:46:14.710 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.699+0000 7f2a31ffb700 1 -- 172.21.15.118:0/3432050897 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a24007300 con 0x7f2a3405f380 2020-11-28T21:46:14.710 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.699+0000 7f2a3993b700 1 -- 172.21.15.118:0/3432050897 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a340ad490 con 0x7f2a3405f380 2020-11-28T21:46:14.711 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.699+0000 7f2a31ffb700 1 -- 172.21.15.118:0/3432050897 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f2a24010030 con 0x7f2a3405f380 2020-11-28T21:46:14.711 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.699+0000 7f2a31ffb700 1 -- 172.21.15.118:0/3432050897 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1214+0+0 (secure 0 0 0) 0x7f2a24007630 con 0x7f2a3405f380 2020-11-28T21:46:14.711 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.703+0000 7f2a3993b700 1 -- 172.21.15.118:0/3432050897 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a341410d0 con 0x7f2a3405f380 2020-11-28T21:46:14.711 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.707+0000 7f2a31ffb700 1 -- 172.21.15.118:0/3432050897 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f2a2400a5a0 con 0x7f2a3405f380 2020-11-28T21:46:14.781 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.779+0000 7f2a3993b700 1 -- 172.21.15.118:0/3432050897 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b907bf53-c297-4756-847b-3be96680eef5", "id": 4} v 0) v1 -- 0x7f2a34057050 con 0x7f2a3405f380 2020-11-28T21:46:14.784 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.783+0000 7f2a31ffb700 1 -- 172.21.15.118:0/3432050897 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b907bf53-c297-4756-847b-3be96680eef5", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2a2400a740 con 0x7f2a3405f380 2020-11-28T21:46:14.784 INFO:teuthology.orchestra.run.smithi118.stdout:4 2020-11-28T21:46:14.785 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.787+0000 7f2a3993b700 1 -- 172.21.15.118:0/3432050897 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2a3405f380 msgr2=0x7f2a340a92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:14.786 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.787+0000 7f2a3993b700 1 --2- 172.21.15.118:0/3432050897 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2a3405f380 0x7f2a340a92a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f2a24001450 tx=0x7f2a24007e20).stop 2020-11-28T21:46:14.786 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.787+0000 7f2a3993b700 1 -- 172.21.15.118:0/3432050897 shutdown_connections 2020-11-28T21:46:14.786 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.787+0000 7f2a3993b700 1 --2- 172.21.15.118:0/3432050897 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2a341530e0 0x7f2a340adfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:14.786 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.787+0000 7f2a3993b700 1 --2- 172.21.15.118:0/3432050897 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2a3405f380 0x7f2a340a92a0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:14.787 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.787+0000 7f2a3993b700 1 --2- 172.21.15.118:0/3432050897 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a340b1240 0x7f2a341413e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:14.787 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.787+0000 7f2a3993b700 1 -- 172.21.15.118:0/3432050897 >> 172.21.15.118:0/3432050897 conn(0x7f2a34063d80 msgr2=0x7f2a341329e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:14.787 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.787+0000 7f2a3993b700 1 -- 172.21.15.118:0/3432050897 shutdown_connections 2020-11-28T21:46:14.787 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.787+0000 7f2a3993b700 1 -- 172.21.15.118:0/3432050897 wait complete. 2020-11-28T21:46:14.795 INFO:teuthology.orchestra.run.smithi118:> sudo ceph --cluster ceph osd new 55ba305d-0505-4212-aebb-631e8554b440 5 2020-11-28T21:46:14.900 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.895+0000 7fc42d029700 1 Processor -- start 2020-11-28T21:46:14.900 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.895+0000 7fc42d029700 1 -- start start 2020-11-28T21:46:14.901 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.899+0000 7fc42d029700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc4280b1240 0x7fc4280ad790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:14.901 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.899+0000 7fc42d029700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc42805f380 0x7fc4280adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:14.901 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.899+0000 7fc42d029700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fc428024f80 0x7fc4280253d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:14.901 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.899+0000 7fc42d029700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4280b3760 con 0x7fc428024f80 2020-11-28T21:46:14.902 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.899+0000 7fc42d029700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4280638e0 con 0x7fc4280b1240 2020-11-28T21:46:14.902 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.899+0000 7fc42d029700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4280605b0 con 0x7fc42805f380 2020-11-28T21:46:14.902 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.899+0000 7fc42e28b700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fc428024f80 0x7fc4280253d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:14.902 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.899+0000 7fc41ffff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc42805f380 0x7fc4280adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:14.903 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.899+0000 7fc42e28b700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fc428024f80 0x7fc4280253d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:57996/0 (socket says 172.21.15.118:57996) 2020-11-28T21:46:14.903 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.899+0000 7fc42e28b700 1 -- 172.21.15.118:0/2260860351 learned_addr learned my addr 172.21.15.118:0/2260860351 (peer_addr_for_me v2:172.21.15.118:0/0) 2020-11-28T21:46:14.903 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.899+0000 7fc42c828700 1 --2- 172.21.15.118:0/2260860351 >> v2:172.21.15.162:3300/0 conn(0x7fc4280b1240 0x7fc4280ad790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:14.903 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.899+0000 7fc42e28b700 1 -- 172.21.15.118:0/2260860351 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc42805f380 msgr2=0x7fc4280adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:14.904 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.899+0000 7fc42e28b700 1 --2- 172.21.15.118:0/2260860351 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc42805f380 0x7fc4280adc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:14.904 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.899+0000 7fc42e28b700 1 -- 172.21.15.118:0/2260860351 >> v2:172.21.15.162:3300/0 conn(0x7fc4280b1240 msgr2=0x7fc4280ad790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:14.904 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.899+0000 7fc42e28b700 1 --2- 172.21.15.118:0/2260860351 >> v2:172.21.15.162:3300/0 conn(0x7fc4280b1240 0x7fc4280ad790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:14.904 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.899+0000 7fc42e28b700 1 -- 172.21.15.118:0/2260860351 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4280b1620 con 0x7fc428024f80 2020-11-28T21:46:14.905 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.899+0000 7fc42e28b700 1 --2- 172.21.15.118:0/2260860351 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc428024f80 0x7fc4280253d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7fc4280b38a0 tx=0x7fc420009270).ready entity=mon.0 client_cookie=67d19cad45c583be server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:14.905 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.899+0000 7fc41f7fe700 1 -- 172.21.15.118:0/2260860351 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc420009e90 con 0x7fc428024f80 2020-11-28T21:46:14.905 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.899+0000 7fc41f7fe700 1 -- 172.21.15.118:0/2260860351 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc42000bbc0 con 0x7fc428024f80 2020-11-28T21:46:14.905 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.899+0000 7fc41f7fe700 1 -- 172.21.15.118:0/2260860351 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc42000bd30 con 0x7fc428024f80 2020-11-28T21:46:14.905 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.899+0000 7fc42d029700 1 -- 172.21.15.118:0/2260860351 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc428024f80 msgr2=0x7fc4280253d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:14.906 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.899+0000 7fc42d029700 1 --2- 172.21.15.118:0/2260860351 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc428024f80 0x7fc4280253d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7fc4280b38a0 tx=0x7fc420009270).stop 2020-11-28T21:46:14.906 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.899+0000 7fc42d029700 1 -- 172.21.15.118:0/2260860351 shutdown_connections 2020-11-28T21:46:14.906 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.899+0000 7fc42d029700 1 --2- 172.21.15.118:0/2260860351 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc428024f80 0x7fc4280253d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:14.906 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.899+0000 7fc42d029700 1 --2- 172.21.15.118:0/2260860351 >> v2:172.21.15.162:3300/0 conn(0x7fc4280b1240 0x7fc4280ad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:14.906 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.899+0000 7fc42d029700 1 --2- 172.21.15.118:0/2260860351 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc42805f380 0x7fc4280adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:14.907 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.899+0000 7fc42d029700 1 -- 172.21.15.118:0/2260860351 >> 172.21.15.118:0/2260860351 conn(0x7fc428063d80 msgr2=0x7fc428064190 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:14.907 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.899+0000 7fc42d029700 1 -- 172.21.15.118:0/2260860351 shutdown_connections 2020-11-28T21:46:14.907 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.899+0000 7fc42d029700 1 -- 172.21.15.118:0/2260860351 wait complete. 2020-11-28T21:46:14.910 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.903+0000 7fc42d029700 1 Processor -- start 2020-11-28T21:46:14.910 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.903+0000 7fc42d029700 1 -- start start 2020-11-28T21:46:14.910 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.903+0000 7fc42d029700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc428024f80 0x7fc428135310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:14.910 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.903+0000 7fc42d029700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc42805f380 0x7fc428140f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:14.911 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.903+0000 7fc42d029700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4280b1240 0x7fc4281413f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:14.911 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.903+0000 7fc42d029700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4280655a0 con 0x7fc428024f80 2020-11-28T21:46:14.911 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.903+0000 7fc42d029700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc428140b10 con 0x7fc4280b1240 2020-11-28T21:46:14.911 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.903+0000 7fc42d029700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc428064dc0 con 0x7fc42805f380 2020-11-28T21:46:14.912 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.903+0000 7fc42c828700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc428024f80 0x7fc428135310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:14.912 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.903+0000 7fc41ffff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc42805f380 0x7fc428140f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:14.912 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.903+0000 7fc42c828700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc428024f80 0x7fc428135310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:57998/0 (socket says 172.21.15.118:57998) 2020-11-28T21:46:14.912 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.903+0000 7fc42c828700 1 -- 172.21.15.118:0/2816301893 learned_addr learned my addr 172.21.15.118:0/2816301893 (peer_addr_for_me v2:172.21.15.118:0/0) 2020-11-28T21:46:14.912 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.903+0000 7fc42e28b700 1 --2- 172.21.15.118:0/2816301893 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4280b1240 0x7fc4281413f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:14.913 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.903+0000 7fc42c828700 1 -- 172.21.15.118:0/2816301893 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc42805f380 msgr2=0x7fc428140f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:14.913 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.903+0000 7fc42c828700 1 --2- 172.21.15.118:0/2816301893 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc42805f380 0x7fc428140f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:14.913 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.903+0000 7fc42c828700 1 -- 172.21.15.118:0/2816301893 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4280b1240 msgr2=0x7fc4281413f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:14.913 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.903+0000 7fc42c828700 1 --2- 172.21.15.118:0/2816301893 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4280b1240 0x7fc4281413f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:14.914 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.903+0000 7fc42c828700 1 -- 172.21.15.118:0/2816301893 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc428065060 con 0x7fc428024f80 2020-11-28T21:46:14.915 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.903+0000 7fc42e28b700 1 --2- 172.21.15.118:0/2816301893 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4280b1240 0x7fc4281413f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:46:14.915 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.903+0000 7fc42c828700 1 --2- 172.21.15.118:0/2816301893 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc428024f80 0x7fc428135310 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7fc424003070 tx=0x7fc424008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:14.915 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.903+0000 7fc41d7fa700 1 -- 172.21.15.118:0/2816301893 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc424009a50 con 0x7fc428024f80 2020-11-28T21:46:14.915 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.903+0000 7fc41d7fa700 1 -- 172.21.15.118:0/2816301893 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc424009bb0 con 0x7fc428024f80 2020-11-28T21:46:14.915 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.903+0000 7fc42d029700 1 -- 172.21.15.118:0/2816301893 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc428065300 con 0x7fc428024f80 2020-11-28T21:46:14.916 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.903+0000 7fc41d7fa700 1 -- 172.21.15.118:0/2816301893 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc424010440 con 0x7fc428024f80 2020-11-28T21:46:14.916 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.903+0000 7fc42d029700 1 -- 172.21.15.118:0/2816301893 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc428066fe0 con 0x7fc428024f80 2020-11-28T21:46:14.916 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.903+0000 7fc41d7fa700 1 -- 172.21.15.118:0/2816301893 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7fc42401f070 con 0x7fc428024f80 2020-11-28T21:46:14.916 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.903+0000 7fc41d7fa700 1 -- 172.21.15.118:0/2816301893 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1329+0+0 (secure 0 0 0) 0x7fc424010c70 con 0x7fc428024f80 2020-11-28T21:46:14.917 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.907+0000 7fc42d029700 1 -- 172.21.15.118:0/2816301893 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc428057050 con 0x7fc428024f80 2020-11-28T21:46:14.917 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.907+0000 7fc41d7fa700 1 -- 172.21.15.118:0/2816301893 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7fc42401d060 con 0x7fc428024f80 2020-11-28T21:46:14.984 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.983+0000 7fc42d029700 1 -- 172.21.15.118:0/2816301893 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd new", "uuid": "55ba305d-0505-4212-aebb-631e8554b440", "id": 5} v 0) v1 -- 0x7fc428044350 con 0x7fc428024f80 2020-11-28T21:46:14.988 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.987+0000 7fc41d7fa700 1 -- 172.21.15.118:0/2816301893 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "55ba305d-0505-4212-aebb-631e8554b440", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc424026020 con 0x7fc428024f80 2020-11-28T21:46:14.989 INFO:teuthology.orchestra.run.smithi118.stdout:5 2020-11-28T21:46:14.990 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.991+0000 7fc42d029700 1 -- 172.21.15.118:0/2816301893 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc428024f80 msgr2=0x7fc428135310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:14.990 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.991+0000 7fc42d029700 1 --2- 172.21.15.118:0/2816301893 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc428024f80 0x7fc428135310 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7fc424003070 tx=0x7fc424008040).stop 2020-11-28T21:46:14.991 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.991+0000 7fc42d029700 1 -- 172.21.15.118:0/2816301893 shutdown_connections 2020-11-28T21:46:14.991 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.991+0000 7fc42d029700 1 --2- 172.21.15.118:0/2816301893 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4280b1240 0x7fc4281413f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:14.991 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.991+0000 7fc42d029700 1 --2- 172.21.15.118:0/2816301893 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc428024f80 0x7fc428135310 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:14.991 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.991+0000 7fc42d029700 1 --2- 172.21.15.118:0/2816301893 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc42805f380 0x7fc428140f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:14.991 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.991+0000 7fc42d029700 1 -- 172.21.15.118:0/2816301893 >> 172.21.15.118:0/2816301893 conn(0x7fc428063d80 msgr2=0x7fc4280ab760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:14.992 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.991+0000 7fc42d029700 1 -- 172.21.15.118:0/2816301893 shutdown_connections 2020-11-28T21:46:14.992 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:14.991+0000 7fc42d029700 1 -- 172.21.15.118:0/2816301893 wait complete. 2020-11-28T21:46:15.001 INFO:teuthology.orchestra.run.smithi118:> sudo ceph --cluster ceph osd new 9ada6051-493b-46b5-ba22-1b401440490f 6 2020-11-28T21:46:15.107 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.103+0000 7f693a076700 1 Processor -- start 2020-11-28T21:46:15.107 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.103+0000 7f693a076700 1 -- start start 2020-11-28T21:46:15.108 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.107+0000 7f693a076700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f69340b1240 0x7f69340ad790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:15.108 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.107+0000 7f693a076700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f69341530e0 0x7f6934153550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:15.108 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.107+0000 7f693a076700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69340638e0 con 0x7f693405f380 2020-11-28T21:46:15.108 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.107+0000 7f693a076700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69340b3760 con 0x7f69341530e0 2020-11-28T21:46:15.108 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.107+0000 7f693a076700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69340b1620 con 0x7f69340b1240 2020-11-28T21:46:15.109 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.107+0000 7f693b2d8700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f69341530e0 0x7f6934153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:15.109 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.107+0000 7f6939074700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f693405f380 0x7f69340adc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:55790/0 (socket says 172.21.15.118:55790) 2020-11-28T21:46:15.109 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.107+0000 7f6939074700 1 -- 172.21.15.118:0/2529363989 learned_addr learned my addr 172.21.15.118:0/2529363989 (peer_addr_for_me v1:172.21.15.118:0/0) 2020-11-28T21:46:15.109 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.107+0000 7f6939875700 1 --2- 172.21.15.118:0/2529363989 >> v2:172.21.15.162:3300/0 conn(0x7f69340b1240 0x7f69340ad790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:15.109 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.107+0000 7f693b2d8700 1 -- 172.21.15.118:0/2529363989 >> v2:172.21.15.162:3300/0 conn(0x7f69340b1240 msgr2=0x7f69340ad790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:15.110 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.107+0000 7f693b2d8700 1 --2- 172.21.15.118:0/2529363989 >> v2:172.21.15.162:3300/0 conn(0x7f69340b1240 0x7f69340ad790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:15.110 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.107+0000 7f693b2d8700 1 -- 172.21.15.118:0/2529363989 >> v1:172.21.15.118:6789/0 conn(0x7f693405f380 legacy=0x7f69340adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:15.110 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.107+0000 7f6938873700 1 -- 172.21.15.118:0/2529363989 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 186052213 0 0) 0x7f69340638e0 con 0x7f693405f380 2020-11-28T21:46:15.110 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.107+0000 7f693b2d8700 1 -- 172.21.15.118:0/2529363989 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69340605b0 con 0x7f69341530e0 2020-11-28T21:46:15.110 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.107+0000 7f693b2d8700 1 --2- 172.21.15.118:0/2529363989 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f69341530e0 0x7f6934153550 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f692c0032b0 tx=0x7f692c0091a0).ready entity=mon.0 client_cookie=1c068a7ff2c3e766 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:15.111 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.107+0000 7f6938873700 1 -- 172.21.15.118:0/2529363989 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f692c00ba50 con 0x7f69341530e0 2020-11-28T21:46:15.111 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.107+0000 7f6938873700 1 -- 172.21.15.118:0/2529363989 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f692c00bbb0 con 0x7f69341530e0 2020-11-28T21:46:15.111 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.107+0000 7f6938873700 1 -- 172.21.15.118:0/2529363989 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f692c00bd20 con 0x7f69341530e0 2020-11-28T21:46:15.111 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.107+0000 7f693a076700 1 -- 172.21.15.118:0/2529363989 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f69341530e0 msgr2=0x7f6934153550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:15.111 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.107+0000 7f693a076700 1 --2- 172.21.15.118:0/2529363989 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f69341530e0 0x7f6934153550 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f692c0032b0 tx=0x7f692c0091a0).stop 2020-11-28T21:46:15.112 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.107+0000 7f693a076700 1 -- 172.21.15.118:0/2529363989 shutdown_connections 2020-11-28T21:46:15.112 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.107+0000 7f693a076700 1 --2- 172.21.15.118:0/2529363989 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f69341530e0 0x7f6934153550 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:15.112 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.107+0000 7f693a076700 1 --2- 172.21.15.118:0/2529363989 >> v2:172.21.15.162:3300/0 conn(0x7f69340b1240 0x7f69340ad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:15.112 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.107+0000 7f693a076700 1 -- 172.21.15.118:0/2529363989 >> 172.21.15.118:0/2529363989 conn(0x7f6934063d80 msgr2=0x7f6934064190 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:15.113 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.107+0000 7f693a076700 1 -- 172.21.15.118:0/2529363989 shutdown_connections 2020-11-28T21:46:15.113 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.107+0000 7f693a076700 1 -- 172.21.15.118:0/2529363989 wait complete. 2020-11-28T21:46:15.115 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.107+0000 7f693a076700 1 Processor -- start 2020-11-28T21:46:15.115 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.107+0000 7f693a076700 1 -- start start 2020-11-28T21:46:15.116 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.111+0000 7f693a076700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f693405f380 0x7f6934139730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:15.116 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.111+0000 7f693a076700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f69340b1240 0x7f693413d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:15.116 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.111+0000 7f693a076700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69341530e0 0x7f693412a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:15.116 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.111+0000 7f693a076700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6934129bf0 con 0x7f693405f380 2020-11-28T21:46:15.117 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.111+0000 7f693a076700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6934129e90 con 0x7f69341530e0 2020-11-28T21:46:15.117 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.111+0000 7f693a076700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f693413cdc0 con 0x7f69340b1240 2020-11-28T21:46:15.117 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.111+0000 7f6939875700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f693405f380 0x7f6934139730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:15.117 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.111+0000 7f6939074700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f69340b1240 0x7f693413d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:15.117 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.111+0000 7f6939875700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f693405f380 0x7f6934139730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58010/0 (socket says 172.21.15.118:58010) 2020-11-28T21:46:15.118 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.111+0000 7f6939875700 1 -- 172.21.15.118:0/645976528 learned_addr learned my addr 172.21.15.118:0/645976528 (peer_addr_for_me v2:172.21.15.118:0/0) 2020-11-28T21:46:15.118 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.111+0000 7f693b2d8700 1 --2- 172.21.15.118:0/645976528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69341530e0 0x7f693412a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:15.118 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.111+0000 7f6939875700 1 -- 172.21.15.118:0/645976528 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f69340b1240 msgr2=0x7f693413d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:15.118 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.111+0000 7f6939875700 1 --2- 172.21.15.118:0/645976528 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f69340b1240 0x7f693413d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:15.118 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.111+0000 7f6939875700 1 -- 172.21.15.118:0/645976528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69341530e0 msgr2=0x7f693412a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:15.119 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.111+0000 7f6939875700 1 --2- 172.21.15.118:0/645976528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69341530e0 0x7f693412a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:15.119 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.111+0000 7f6939875700 1 -- 172.21.15.118:0/645976528 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f693413d060 con 0x7f693405f380 2020-11-28T21:46:15.119 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.111+0000 7f6939875700 1 --2- 172.21.15.118:0/645976528 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f693405f380 0x7f6934139730 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f6930003070 tx=0x7f6930008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:15.119 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.111+0000 7f69267fc700 1 -- 172.21.15.118:0/645976528 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6930009790 con 0x7f693405f380 2020-11-28T21:46:15.120 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.111+0000 7f69267fc700 1 -- 172.21.15.118:0/645976528 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69300098f0 con 0x7f693405f380 2020-11-28T21:46:15.120 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.111+0000 7f693a076700 1 -- 172.21.15.118:0/645976528 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69340acd40 con 0x7f693405f380 2020-11-28T21:46:15.120 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.111+0000 7f69267fc700 1 -- 172.21.15.118:0/645976528 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6930009c50 con 0x7f693405f380 2020-11-28T21:46:15.120 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.111+0000 7f693a076700 1 -- 172.21.15.118:0/645976528 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f693413db90 con 0x7f693405f380 2020-11-28T21:46:15.120 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.111+0000 7f69267fc700 1 -- 172.21.15.118:0/645976528 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f6930013480 con 0x7f693405f380 2020-11-28T21:46:15.121 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.111+0000 7f69267fc700 1 -- 172.21.15.118:0/645976528 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1444+0+0 (secure 0 0 0) 0x7f6930013710 con 0x7f693405f380 2020-11-28T21:46:15.121 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.111+0000 7f693a076700 1 -- 172.21.15.118:0/645976528 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f693413d1f0 con 0x7f693405f380 2020-11-28T21:46:15.121 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.115+0000 7f69267fc700 1 -- 172.21.15.118:0/645976528 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f6930013b70 con 0x7f693405f380 2020-11-28T21:46:15.190 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.191+0000 7f693a076700 1 -- 172.21.15.118:0/645976528 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9ada6051-493b-46b5-ba22-1b401440490f", "id": 6} v 0) v1 -- 0x7f6934057050 con 0x7f693405f380 2020-11-28T21:46:15.196 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.195+0000 7f69267fc700 1 -- 172.21.15.118:0/645976528 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9ada6051-493b-46b5-ba22-1b401440490f", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6930013d10 con 0x7f693405f380 2020-11-28T21:46:15.196 INFO:teuthology.orchestra.run.smithi118.stdout:6 2020-11-28T21:46:15.197 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.199+0000 7f693a076700 1 -- 172.21.15.118:0/645976528 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f693405f380 msgr2=0x7f6934139730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:15.198 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.199+0000 7f693a076700 1 --2- 172.21.15.118:0/645976528 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f693405f380 0x7f6934139730 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f6930003070 tx=0x7f6930008040).stop 2020-11-28T21:46:15.198 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.199+0000 7f693a076700 1 -- 172.21.15.118:0/645976528 shutdown_connections 2020-11-28T21:46:15.198 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.199+0000 7f693a076700 1 --2- 172.21.15.118:0/645976528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69341530e0 0x7f693412a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:15.198 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.199+0000 7f693a076700 1 --2- 172.21.15.118:0/645976528 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f693405f380 0x7f6934139730 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:15.199 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.199+0000 7f693a076700 1 --2- 172.21.15.118:0/645976528 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f69340b1240 0x7f693413d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:15.199 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.199+0000 7f693a076700 1 -- 172.21.15.118:0/645976528 >> 172.21.15.118:0/645976528 conn(0x7f6934063d80 msgr2=0x7f6934159990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:15.199 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.199+0000 7f693a076700 1 -- 172.21.15.118:0/645976528 shutdown_connections 2020-11-28T21:46:15.199 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.199+0000 7f693a076700 1 -- 172.21.15.118:0/645976528 wait complete. 2020-11-28T21:46:15.213 INFO:teuthology.orchestra.run.smithi118:> sudo ceph --cluster ceph osd new c328b06d-a216-4cba-aad7-9ad081340613 7 2020-11-28T21:46:15.318 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.311+0000 7f11f359e700 1 Processor -- start 2020-11-28T21:46:15.319 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.311+0000 7f11f359e700 1 -- start start 2020-11-28T21:46:15.319 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.315+0000 7f11f359e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f11ec046c70 0x7f11ec0ee920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:15.319 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.315+0000 7f11f359e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f11ec006b80 0x7f11ec0417d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:15.320 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.315+0000 7f11f359e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11ec040300 con 0x7f11ec03f8c0 2020-11-28T21:46:15.320 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.315+0000 7f11f359e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11ec047050 con 0x7f11ec046c70 2020-11-28T21:46:15.320 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.315+0000 7f11f359e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11ec03d700 con 0x7f11ec006b80 2020-11-28T21:46:15.320 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.315+0000 7f11f2d9d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f11ec046c70 0x7f11ec0ee920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:15.320 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.315+0000 7f11f259c700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f11ec03f8c0 0x7f11ec0eee10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:55802/0 (socket says 172.21.15.118:55802) 2020-11-28T21:46:15.321 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.315+0000 7f11f2d9d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f11ec046c70 0x7f11ec0ee920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58016/0 (socket says 172.21.15.118:58016) 2020-11-28T21:46:15.321 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.315+0000 7f11f259c700 1 -- 172.21.15.118:0/1462942827 learned_addr learned my addr 172.21.15.118:0/1462942827 (peer_addr_for_me v1:172.21.15.118:0/0) 2020-11-28T21:46:15.321 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.315+0000 7f11f8cf5700 1 --2- 172.21.15.118:0/1462942827 >> v2:172.21.15.162:3300/0 conn(0x7f11ec006b80 0x7f11ec0417d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:15.321 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.315+0000 7f11f2d9d700 1 -- 172.21.15.118:0/1462942827 >> v2:172.21.15.162:3300/0 conn(0x7f11ec006b80 msgr2=0x7f11ec0417d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:15.321 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.315+0000 7f11f2d9d700 1 --2- 172.21.15.118:0/1462942827 >> v2:172.21.15.162:3300/0 conn(0x7f11ec006b80 0x7f11ec0417d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:15.322 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.315+0000 7f11f2d9d700 1 -- 172.21.15.118:0/1462942827 >> v1:172.21.15.118:6789/0 conn(0x7f11ec03f8c0 legacy=0x7f11ec0eee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:15.322 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.315+0000 7f11f1d9b700 1 -- 172.21.15.118:0/1462942827 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2654103178 0 0) 0x7f11ec040300 con 0x7f11ec03f8c0 2020-11-28T21:46:15.322 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.315+0000 7f11f2d9d700 1 -- 172.21.15.118:0/1462942827 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ec041410 con 0x7f11ec046c70 2020-11-28T21:46:15.322 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.315+0000 7f11f8cf5700 1 --2- 172.21.15.118:0/1462942827 >> v2:172.21.15.162:3300/0 conn(0x7f11ec006b80 0x7f11ec0417d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:46:15.322 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.315+0000 7f11f2d9d700 1 --2- 172.21.15.118:0/1462942827 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f11ec046c70 0x7f11ec0ee920 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f11e8002f30 tx=0x7f11e8006dc0).ready entity=mon.0 client_cookie=981ae14db912721d server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:15.323 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.315+0000 7f11f1d9b700 1 -- 172.21.15.118:0/1462942827 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11e80097f0 con 0x7f11ec046c70 2020-11-28T21:46:15.323 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.315+0000 7f11f1d9b700 1 -- 172.21.15.118:0/1462942827 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e8009950 con 0x7f11ec046c70 2020-11-28T21:46:15.323 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.315+0000 7f11f1d9b700 1 -- 172.21.15.118:0/1462942827 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11e8009cb0 con 0x7f11ec046c70 2020-11-28T21:46:15.323 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.315+0000 7f11f359e700 1 -- 172.21.15.118:0/1462942827 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f11ec046c70 msgr2=0x7f11ec0ee920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:15.324 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.315+0000 7f11f359e700 1 --2- 172.21.15.118:0/1462942827 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f11ec046c70 0x7f11ec0ee920 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f11e8002f30 tx=0x7f11e8006dc0).stop 2020-11-28T21:46:15.324 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.315+0000 7f11f359e700 1 -- 172.21.15.118:0/1462942827 shutdown_connections 2020-11-28T21:46:15.324 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.315+0000 7f11f359e700 1 --2- 172.21.15.118:0/1462942827 >> v2:172.21.15.162:3300/0 conn(0x7f11ec006b80 0x7f11ec0417d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:15.324 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.315+0000 7f11f359e700 1 --2- 172.21.15.118:0/1462942827 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f11ec046c70 0x7f11ec0ee920 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:15.325 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.315+0000 7f11f359e700 1 -- 172.21.15.118:0/1462942827 >> 172.21.15.118:0/1462942827 conn(0x7f11ec009e10 msgr2=0x7f11ec046220 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:15.325 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.315+0000 7f11f359e700 1 -- 172.21.15.118:0/1462942827 shutdown_connections 2020-11-28T21:46:15.325 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.315+0000 7f11f359e700 1 -- 172.21.15.118:0/1462942827 wait complete. 2020-11-28T21:46:15.325 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.315+0000 7f11f359e700 1 Processor -- start 2020-11-28T21:46:15.325 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.315+0000 7f11f359e700 1 -- start start 2020-11-28T21:46:15.326 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.319+0000 7f11f359e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11ec006b80 0x7f11ec0c9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:15.326 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.319+0000 7f11f359e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f11ec03f8c0 0x7f11ec0de7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:15.326 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.319+0000 7f11f359e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f11ec046c70 0x7f11ec0dece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:15.326 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.319+0000 7f11f359e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ec0df1d0 con 0x7f11ec046c70 2020-11-28T21:46:15.326 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.319+0000 7f11f359e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ec045290 con 0x7f11ec006b80 2020-11-28T21:46:15.327 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.319+0000 7f11f359e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11ec0ca180 con 0x7f11ec03f8c0 2020-11-28T21:46:15.327 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.319+0000 7f11f8cf5700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f11ec046c70 0x7f11ec0dece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:15.327 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.319+0000 7f11f259c700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f11ec03f8c0 0x7f11ec0de7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:15.327 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.319+0000 7f11f8cf5700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f11ec046c70 0x7f11ec0dece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58026/0 (socket says 172.21.15.118:58026) 2020-11-28T21:46:15.327 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.319+0000 7f11f8cf5700 1 -- 172.21.15.118:0/1271728085 learned_addr learned my addr 172.21.15.118:0/1271728085 (peer_addr_for_me v2:172.21.15.118:0/0) 2020-11-28T21:46:15.328 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.319+0000 7f11f2d9d700 1 --2- 172.21.15.118:0/1271728085 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11ec006b80 0x7f11ec0c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:15.328 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.319+0000 7f11f259c700 1 -- 172.21.15.118:0/1271728085 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11ec006b80 msgr2=0x7f11ec0c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:15.328 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.319+0000 7f11f259c700 1 --2- 172.21.15.118:0/1271728085 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11ec006b80 0x7f11ec0c9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:15.328 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.319+0000 7f11f259c700 1 -- 172.21.15.118:0/1271728085 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f11ec046c70 msgr2=0x7f11ec0dece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:15.328 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.319+0000 7f11f259c700 1 --2- 172.21.15.118:0/1271728085 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f11ec046c70 0x7f11ec0dece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:15.329 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.319+0000 7f11f259c700 1 -- 172.21.15.118:0/1271728085 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ec044170 con 0x7f11ec03f8c0 2020-11-28T21:46:15.329 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.319+0000 7f11f8cf5700 1 --2- 172.21.15.118:0/1271728085 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f11ec046c70 0x7f11ec0dece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:46:15.329 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.319+0000 7f11f259c700 1 --2- 172.21.15.118:0/1271728085 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f11ec03f8c0 0x7f11ec0de7f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f11ec0df310 tx=0x7f11e40079b0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:15.329 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.319+0000 7f11e37fe700 1 -- 172.21.15.118:0/1271728085 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11e40072e0 con 0x7f11ec03f8c0 2020-11-28T21:46:15.329 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.319+0000 7f11e37fe700 1 -- 172.21.15.118:0/1271728085 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e4007440 con 0x7f11ec03f8c0 2020-11-28T21:46:15.330 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.319+0000 7f11e37fe700 1 -- 172.21.15.118:0/1271728085 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11e40057d0 con 0x7f11ec03f8c0 2020-11-28T21:46:15.330 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.319+0000 7f11f359e700 1 -- 172.21.15.118:0/1271728085 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11ec0454c0 con 0x7f11ec03f8c0 2020-11-28T21:46:15.330 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.319+0000 7f11f359e700 1 -- 172.21.15.118:0/1271728085 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11ec0c1f00 con 0x7f11ec03f8c0 2020-11-28T21:46:15.330 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.319+0000 7f11e37fe700 1 -- 172.21.15.118:0/1271728085 <== mon.2 v2:172.21.15.118:3301/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f11e4010070 con 0x7f11ec03f8c0 2020-11-28T21:46:15.330 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.319+0000 7f11e37fe700 1 -- 172.21.15.118:0/1271728085 <== mon.2 v2:172.21.15.118:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1559+0+0 (secure 0 0 0) 0x7f11e4013ae0 con 0x7f11ec03f8c0 2020-11-28T21:46:15.331 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.319+0000 7f11f359e700 1 -- 172.21.15.118:0/1271728085 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11ec007de0 con 0x7f11ec03f8c0 2020-11-28T21:46:15.332 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.323+0000 7f11e37fe700 1 -- 172.21.15.118:0/1271728085 <== mon.2 v2:172.21.15.118:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f11ec007de0 con 0x7f11ec03f8c0 2020-11-28T21:46:15.401 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.403+0000 7f11f359e700 1 -- 172.21.15.118:0/1271728085 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c328b06d-a216-4cba-aad7-9ad081340613", "id": 7} v 0) v1 -- 0x7f11ec004cf0 con 0x7f11ec03f8c0 2020-11-28T21:46:15.407 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.407+0000 7f11e37fe700 1 -- 172.21.15.118:0/1271728085 <== mon.2 v2:172.21.15.118:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c328b06d-a216-4cba-aad7-9ad081340613", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f11e4018020 con 0x7f11ec03f8c0 2020-11-28T21:46:15.408 INFO:teuthology.orchestra.run.smithi118.stdout:7 2020-11-28T21:46:15.409 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.411+0000 7f11f359e700 1 -- 172.21.15.118:0/1271728085 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f11ec03f8c0 msgr2=0x7f11ec0de7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:15.410 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.411+0000 7f11f359e700 1 --2- 172.21.15.118:0/1271728085 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f11ec03f8c0 0x7f11ec0de7f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f11ec0df310 tx=0x7f11e40079b0).stop 2020-11-28T21:46:15.410 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.411+0000 7f11f359e700 1 -- 172.21.15.118:0/1271728085 shutdown_connections 2020-11-28T21:46:15.410 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.411+0000 7f11f359e700 1 --2- 172.21.15.118:0/1271728085 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f11ec046c70 0x7f11ec0dece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:15.410 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.411+0000 7f11f359e700 1 --2- 172.21.15.118:0/1271728085 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11ec006b80 0x7f11ec0c9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:15.411 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.411+0000 7f11f359e700 1 --2- 172.21.15.118:0/1271728085 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f11ec03f8c0 0x7f11ec0de7f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:15.411 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.411+0000 7f11f359e700 1 -- 172.21.15.118:0/1271728085 >> 172.21.15.118:0/1271728085 conn(0x7f11ec009e10 msgr2=0x7f11ec042ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:15.411 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.411+0000 7f11f359e700 1 -- 172.21.15.118:0/1271728085 shutdown_connections 2020-11-28T21:46:15.411 INFO:teuthology.orchestra.run.smithi118.stderr:2020-11-28T21:46:15.411+0000 7f11f359e700 1 -- 172.21.15.118:0/1271728085 wait complete. 2020-11-28T21:46:15.423 INFO:tasks.ceph.osd.0:Restarting daemon 2020-11-28T21:46:15.424 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2020-11-28T21:46:15.426 INFO:tasks.ceph.osd.0:Started 2020-11-28T21:46:15.427 INFO:tasks.ceph.osd.1:Restarting daemon 2020-11-28T21:46:15.427 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2020-11-28T21:46:15.429 INFO:tasks.ceph.osd.1:Started 2020-11-28T21:46:15.430 INFO:tasks.ceph.osd.2:Restarting daemon 2020-11-28T21:46:15.430 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2020-11-28T21:46:15.432 INFO:tasks.ceph.osd.2:Started 2020-11-28T21:46:15.433 INFO:tasks.ceph.osd.3:Restarting daemon 2020-11-28T21:46:15.433 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2020-11-28T21:46:15.435 INFO:tasks.ceph.osd.3:Started 2020-11-28T21:46:15.436 INFO:tasks.ceph.osd.4:Restarting daemon 2020-11-28T21:46:15.469 INFO:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2020-11-28T21:46:15.472 INFO:tasks.ceph.osd.4:Started 2020-11-28T21:46:15.472 INFO:tasks.ceph.osd.5:Restarting daemon 2020-11-28T21:46:15.472 INFO:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2020-11-28T21:46:15.474 INFO:tasks.ceph.osd.5:Started 2020-11-28T21:46:15.475 INFO:tasks.ceph.osd.6:Restarting daemon 2020-11-28T21:46:15.475 INFO:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2020-11-28T21:46:15.477 INFO:tasks.ceph.osd.6:Started 2020-11-28T21:46:15.478 INFO:tasks.ceph.osd.7:Restarting daemon 2020-11-28T21:46:15.478 INFO:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2020-11-28T21:46:15.480 INFO:tasks.ceph.osd.7:Started 2020-11-28T21:46:15.481 INFO:tasks.ceph:Waiting for OSDs to come up 2020-11-28T21:46:15.786 INFO:tasks.ceph.osd.1.smithi162.stderr:2020-11-28T21:46:15.783+0000 7fdb67bebd80 -1 Falling back to public interface 2020-11-28T21:46:15.802 INFO:tasks.ceph.osd.0.smithi162.stderr:2020-11-28T21:46:15.799+0000 7fc4ac414d80 -1 Falling back to public interface 2020-11-28T21:46:15.812 INFO:tasks.ceph.osd.2.smithi162.stderr:2020-11-28T21:46:15.807+0000 7f6d6f2bcd80 -1 Falling back to public interface 2020-11-28T21:46:15.831 INFO:tasks.ceph.osd.4.smithi118.stderr:2020-11-28T21:46:15.831+0000 7f8eb2cead80 -1 Falling back to public interface 2020-11-28T21:46:15.838 INFO:tasks.ceph.osd.5.smithi118.stderr:2020-11-28T21:46:15.839+0000 7f135c695d80 -1 Falling back to public interface 2020-11-28T21:46:15.855 INFO:tasks.ceph.osd.7.smithi118.stderr:2020-11-28T21:46:15.855+0000 7fac9db16d80 -1 Falling back to public interface 2020-11-28T21:46:16.054 INFO:tasks.ceph.osd.3.smithi162.stderr:2020-11-28T21:46:16.051+0000 7f36b8cbbd80 -1 Falling back to public interface 2020-11-28T21:46:16.108 INFO:tasks.ceph.osd.6.smithi118.stderr:2020-11-28T21:46:16.107+0000 7f84c6b75d80 -1 Falling back to public interface 2020-11-28T21:46:16.287 INFO:teuthology.orchestra.run.smithi162:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2020-11-28T21:46:16.390 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.383+0000 7f1b8a7c2700 1 Processor -- start 2020-11-28T21:46:16.390 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.383+0000 7f1b8a7c2700 1 -- start start 2020-11-28T21:46:16.391 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.383+0000 7f1b8a7c2700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1b8405f380 0x7f1b840adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:16.391 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.383+0000 7f1b8a7c2700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b840638e0 con 0x7f1b840b1240 2020-11-28T21:46:16.391 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.383+0000 7f1b8a7c2700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b840605b0 con 0x7f1b841530e0 2020-11-28T21:46:16.392 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.383+0000 7f1b8a7c2700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b840b3760 con 0x7f1b8405f380 2020-11-28T21:46:16.392 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.383+0000 7f1b897c0700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1b8405f380 0x7f1b840adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:16.393 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.383+0000 7f1b8ba24700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f1b841530e0 0x7f1b84153550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:43576/0 (socket says 172.21.15.162:43576) 2020-11-28T21:46:16.393 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.383+0000 7f1b8ba24700 1 -- 172.21.15.162:0/3085976251 learned_addr learned my addr 172.21.15.162:0/3085976251 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:16.393 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.383+0000 7f1b897c0700 1 -- 172.21.15.162:0/3085976251 >> v1:172.21.15.162:6789/0 conn(0x7f1b841530e0 legacy=0x7f1b84153550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:16.394 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.383+0000 7f1b897c0700 1 -- 172.21.15.162:0/3085976251 >> v1:172.21.15.118:6789/0 conn(0x7f1b840b1240 legacy=0x7f1b840ad790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:16.394 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.383+0000 7f1b897c0700 1 -- 172.21.15.162:0/3085976251 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b840b1620 con 0x7f1b8405f380 2020-11-28T21:46:16.394 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.383+0000 7f1b897c0700 1 --2- 172.21.15.162:0/3085976251 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b8405f380 0x7f1b840adea0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f1b74002f30 tx=0x7f1b74006dc0).ready entity=mon.1 client_cookie=cbe21fa425917307 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:16.395 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.387+0000 7f1b88fbf700 1 -- 172.21.15.162:0/3085976251 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b740097f0 con 0x7f1b8405f380 2020-11-28T21:46:16.395 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.387+0000 7f1b88fbf700 1 -- 172.21.15.162:0/3085976251 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b74009950 con 0x7f1b8405f380 2020-11-28T21:46:16.396 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.387+0000 7f1b88fbf700 1 -- 172.21.15.162:0/3085976251 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b74009cb0 con 0x7f1b8405f380 2020-11-28T21:46:16.396 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.387+0000 7f1b8a7c2700 1 -- 172.21.15.162:0/3085976251 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b8405f380 msgr2=0x7f1b840adea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:16.396 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.387+0000 7f1b8a7c2700 1 --2- 172.21.15.162:0/3085976251 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b8405f380 0x7f1b840adea0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f1b74002f30 tx=0x7f1b74006dc0).stop 2020-11-28T21:46:16.397 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.387+0000 7f1b8a7c2700 1 -- 172.21.15.162:0/3085976251 shutdown_connections 2020-11-28T21:46:16.397 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.387+0000 7f1b8a7c2700 1 --2- 172.21.15.162:0/3085976251 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b8405f380 0x7f1b840adea0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:16.397 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.387+0000 7f1b8a7c2700 1 -- 172.21.15.162:0/3085976251 >> 172.21.15.162:0/3085976251 conn(0x7f1b84063d80 msgr2=0x7f1b84064190 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:16.398 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.387+0000 7f1b8a7c2700 1 -- 172.21.15.162:0/3085976251 shutdown_connections 2020-11-28T21:46:16.398 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.387+0000 7f1b8a7c2700 1 -- 172.21.15.162:0/3085976251 wait complete. 2020-11-28T21:46:16.402 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.387+0000 7f1b8a7c2700 1 Processor -- start 2020-11-28T21:46:16.402 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.387+0000 7f1b8a7c2700 1 -- start start 2020-11-28T21:46:16.403 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.387+0000 7f1b8a7c2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b8405f380 0x7f1b840b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:16.403 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.387+0000 7f1b8a7c2700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1b840b1240 0x7f1b840aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:16.403 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.387+0000 7f1b8a7c2700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1b841530e0 0x7f1b84162b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:16.404 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.387+0000 7f1b8a7c2700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b84160550 con 0x7f1b840b1240 2020-11-28T21:46:16.404 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.387+0000 7f1b8a7c2700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b84156550 con 0x7f1b8405f380 2020-11-28T21:46:16.405 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.387+0000 7f1b8a7c2700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b840ae5a0 con 0x7f1b841530e0 2020-11-28T21:46:16.405 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.387+0000 7f1b89fc1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b8405f380 0x7f1b840b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:16.405 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.387+0000 7f1b89fc1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b8405f380 0x7f1b840b4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:36734/0 (socket says 172.21.15.162:36734) 2020-11-28T21:46:16.406 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.387+0000 7f1b89fc1700 1 -- 172.21.15.162:0/191117234 learned_addr learned my addr 172.21.15.162:0/191117234 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:16.406 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.387+0000 7f1b897c0700 1 --2- 172.21.15.162:0/191117234 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1b840b1240 0x7f1b840aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:16.406 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.387+0000 7f1b8ba24700 1 --2- 172.21.15.162:0/191117234 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1b841530e0 0x7f1b84162b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:16.407 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.391+0000 7f1b89fc1700 1 -- 172.21.15.162:0/191117234 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1b841530e0 msgr2=0x7f1b84162b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:16.407 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.391+0000 7f1b89fc1700 1 --2- 172.21.15.162:0/191117234 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1b841530e0 0x7f1b84162b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:16.407 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.391+0000 7f1b89fc1700 1 -- 172.21.15.162:0/191117234 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1b840b1240 msgr2=0x7f1b840aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:16.408 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.391+0000 7f1b89fc1700 1 --2- 172.21.15.162:0/191117234 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1b840b1240 0x7f1b840aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:16.408 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.391+0000 7f1b89fc1700 1 -- 172.21.15.162:0/191117234 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b840ae840 con 0x7f1b8405f380 2020-11-28T21:46:16.408 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.391+0000 7f1b89fc1700 1 --2- 172.21.15.162:0/191117234 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b8405f380 0x7f1b840b4f70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f1b84156820 tx=0x7f1b80007c20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:16.409 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.391+0000 7f1b7a7fc700 1 -- 172.21.15.162:0/191117234 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b8000e4a0 con 0x7f1b8405f380 2020-11-28T21:46:16.409 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.391+0000 7f1b7a7fc700 1 -- 172.21.15.162:0/191117234 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b8000e600 con 0x7f1b8405f380 2020-11-28T21:46:16.409 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.391+0000 7f1b7a7fc700 1 -- 172.21.15.162:0/191117234 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b8000e960 con 0x7f1b8405f380 2020-11-28T21:46:16.410 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.391+0000 7f1b8a7c2700 1 -- 172.21.15.162:0/191117234 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b840ac150 con 0x7f1b8405f380 2020-11-28T21:46:16.410 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.391+0000 7f1b8a7c2700 1 -- 172.21.15.162:0/191117234 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b84162ff0 con 0x7f1b8405f380 2020-11-28T21:46:16.411 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.391+0000 7f1b7a7fc700 1 -- 172.21.15.162:0/191117234 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 2) v1 ==== 33316+0+0 (secure 0 0 0) 0x7f1b80010030 con 0x7f1b8405f380 2020-11-28T21:46:16.411 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.391+0000 7f1b7a7fc700 1 -- 172.21.15.162:0/191117234 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1674+0+0 (secure 0 0 0) 0x7f1b8003b600 con 0x7f1b8405f380 2020-11-28T21:46:16.411 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.391+0000 7f1b8a7c2700 1 -- 172.21.15.162:0/191117234 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b840b0640 con 0x7f1b8405f380 2020-11-28T21:46:16.412 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.395+0000 7f1b7a7fc700 1 -- 172.21.15.162:0/191117234 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f1b8000eac0 con 0x7f1b8405f380 2020-11-28T21:46:16.457 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.455+0000 7f1b8a7c2700 1 -- 172.21.15.162:0/191117234 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1b840ae9d0 con 0x7f1b8405f380 2020-11-28T21:46:16.458 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.455+0000 7f1b7a7fc700 1 -- 172.21.15.162:0/191117234 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+6849 (secure 0 0 0) 0x7f1b80039090 con 0x7f1b8405f380 2020-11-28T21:46:16.459 INFO:teuthology.misc.health.smithi162.stdout: 2020-11-28T21:46:16.459 INFO:teuthology.misc.health.smithi162.stdout:{"epoch":10,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:46:15.405364+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:46:16.461 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.455+0000 7f1b8a7c2700 1 -- 172.21.15.162:0/191117234 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b8405f380 msgr2=0x7f1b840b4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:16.462 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.455+0000 7f1b8a7c2700 1 --2- 172.21.15.162:0/191117234 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b8405f380 0x7f1b840b4f70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f1b84156820 tx=0x7f1b80007c20).stop 2020-11-28T21:46:16.462 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.455+0000 7f1b8a7c2700 1 -- 172.21.15.162:0/191117234 shutdown_connections 2020-11-28T21:46:16.462 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.455+0000 7f1b8a7c2700 1 --2- 172.21.15.162:0/191117234 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1b841530e0 0x7f1b84162b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:16.463 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.455+0000 7f1b8a7c2700 1 --2- 172.21.15.162:0/191117234 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b8405f380 0x7f1b840b4f70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:16.463 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.455+0000 7f1b8a7c2700 1 --2- 172.21.15.162:0/191117234 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1b840b1240 0x7f1b840aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:16.464 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.455+0000 7f1b8a7c2700 1 -- 172.21.15.162:0/191117234 >> 172.21.15.162:0/191117234 conn(0x7f1b84063d80 msgr2=0x7f1b84025720 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:16.464 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.455+0000 7f1b8a7c2700 1 -- 172.21.15.162:0/191117234 shutdown_connections 2020-11-28T21:46:16.465 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:16.455+0000 7f1b8a7c2700 1 -- 172.21.15.162:0/191117234 wait complete. 2020-11-28T21:46:16.469 DEBUG:teuthology.misc:0 of 8 OSDs are up 2020-11-28T21:46:16.969 INFO:tasks.ceph.osd.1.smithi162.stderr:2020-11-28T21:46:16.967+0000 7fdb67bebd80 -1 osd.1 0 log_to_monitors {default=true} 2020-11-28T21:46:16.999 INFO:tasks.ceph.osd.0.smithi162.stderr:2020-11-28T21:46:16.995+0000 7fc4ac414d80 -1 osd.0 0 log_to_monitors {default=true} 2020-11-28T21:46:16.999 INFO:tasks.ceph.osd.2.smithi162.stderr:2020-11-28T21:46:16.995+0000 7f6d6f2bcd80 -1 osd.2 0 log_to_monitors {default=true} 2020-11-28T21:46:17.032 INFO:tasks.ceph.osd.4.smithi118.stderr:2020-11-28T21:46:17.031+0000 7f8eb2cead80 -1 osd.4 0 log_to_monitors {default=true} 2020-11-28T21:46:17.033 INFO:tasks.ceph.osd.5.smithi118.stderr:2020-11-28T21:46:17.031+0000 7f135c695d80 -1 osd.5 0 log_to_monitors {default=true} 2020-11-28T21:46:17.070 INFO:tasks.ceph.osd.7.smithi118.stderr:2020-11-28T21:46:17.071+0000 7fac9db16d80 -1 osd.7 0 log_to_monitors {default=true} 2020-11-28T21:46:17.208 INFO:tasks.ceph.osd.3.smithi162.stderr:2020-11-28T21:46:17.203+0000 7f36b8cbbd80 -1 osd.3 0 log_to_monitors {default=true} 2020-11-28T21:46:17.236 INFO:tasks.ceph.osd.6.smithi118.stderr:2020-11-28T21:46:17.235+0000 7f84c6b75d80 -1 osd.6 0 log_to_monitors {default=true} 2020-11-28T21:46:18.601 INFO:tasks.ceph.osd.4.smithi118.stderr:2020-11-28T21:46:18.599+0000 7f8e9bf35700 -1 osd.4 0 waiting for initial osdmap 2020-11-28T21:46:18.602 INFO:tasks.ceph.osd.5.smithi118.stderr:2020-11-28T21:46:18.603+0000 7f13458e0700 -1 osd.5 0 waiting for initial osdmap 2020-11-28T21:46:18.602 INFO:tasks.ceph.osd.6.smithi118.stderr:2020-11-28T21:46:18.603+0000 7f84afdc0700 -1 osd.6 0 waiting for initial osdmap 2020-11-28T21:46:18.603 INFO:tasks.ceph.osd.7.smithi118.stderr:2020-11-28T21:46:18.603+0000 7fac86d61700 -1 osd.7 0 waiting for initial osdmap 2020-11-28T21:46:18.603 INFO:tasks.ceph.osd.1.smithi162.stderr:2020-11-28T21:46:18.599+0000 7fdb50e36700 -1 osd.1 0 waiting for initial osdmap 2020-11-28T21:46:18.606 INFO:tasks.ceph.osd.0.smithi162.stderr:2020-11-28T21:46:18.603+0000 7fc49565f700 -1 osd.0 0 waiting for initial osdmap 2020-11-28T21:46:18.607 INFO:tasks.ceph.osd.2.smithi162.stderr:2020-11-28T21:46:18.603+0000 7f6d58507700 -1 osd.2 0 waiting for initial osdmap 2020-11-28T21:46:18.608 INFO:tasks.ceph.osd.3.smithi162.stderr:2020-11-28T21:46:18.599+0000 7f36a1f06700 -1 osd.3 0 waiting for initial osdmap 2020-11-28T21:46:23.275 INFO:teuthology.orchestra.run.smithi162:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2020-11-28T21:46:23.399 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.383+0000 7f7c99635700 1 Processor -- start 2020-11-28T21:46:23.401 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.383+0000 7f7c99635700 1 -- start start 2020-11-28T21:46:23.401 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.383+0000 7f7c99635700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f7c940adea0 0x7f7c941537f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:23.402 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.383+0000 7f7c99635700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c940638e0 con 0x7f7c940b1240 2020-11-28T21:46:23.402 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.383+0000 7f7c99635700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c940605b0 con 0x7f7c9405f380 2020-11-28T21:46:23.403 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.383+0000 7f7c99635700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c940ae310 con 0x7f7c940adea0 2020-11-28T21:46:23.403 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.383+0000 7f7c93fff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f7c9405f380 0x7f7c941530e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:43820/0 (socket says 172.21.15.162:43820) 2020-11-28T21:46:23.404 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.383+0000 7f7c93fff700 1 -- 172.21.15.162:0/3972222281 learned_addr learned my addr 172.21.15.162:0/3972222281 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:23.404 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.383+0000 7f7c9a897700 1 --2- 172.21.15.162:0/3972222281 >> v2:172.21.15.118:3300/0 conn(0x7f7c940adea0 0x7f7c941537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:23.404 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.383+0000 7f7c937fe700 1 -- 172.21.15.162:0/3972222281 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1112338955 0 0) 0x7f7c940605b0 con 0x7f7c9405f380 2020-11-28T21:46:23.405 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.383+0000 7f7c937fe700 1 -- 172.21.15.162:0/3972222281 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c80003230 con 0x7f7c9405f380 2020-11-28T21:46:23.405 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.383+0000 7f7c937fe700 1 -- 172.21.15.162:0/3972222281 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2084056541 0 0) 0x7f7c940638e0 con 0x7f7c940b1240 2020-11-28T21:46:23.405 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.383+0000 7f7c937fe700 1 -- 172.21.15.162:0/3972222281 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c940605b0 con 0x7f7c940b1240 2020-11-28T21:46:23.405 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.383+0000 7f7c937fe700 1 -- 172.21.15.162:0/3972222281 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2236044302 0 0) 0x7f7c80003230 con 0x7f7c9405f380 2020-11-28T21:46:23.406 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.383+0000 7f7c937fe700 1 -- 172.21.15.162:0/3972222281 >> v2:172.21.15.118:3300/0 conn(0x7f7c940adea0 msgr2=0x7f7c941537f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:23.406 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.383+0000 7f7c937fe700 1 --2- 172.21.15.162:0/3972222281 >> v2:172.21.15.118:3300/0 conn(0x7f7c940adea0 0x7f7c941537f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:23.406 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.383+0000 7f7c937fe700 1 -- 172.21.15.162:0/3972222281 >> v1:172.21.15.118:6789/0 conn(0x7f7c940b1240 legacy=0x7f7c940ad790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:23.407 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.383+0000 7f7c937fe700 1 -- 172.21.15.162:0/3972222281 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c940b1620 con 0x7f7c9405f380 2020-11-28T21:46:23.407 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.383+0000 7f7c937fe700 1 -- 172.21.15.162:0/3972222281 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f7c8c0016d0 con 0x7f7c9405f380 2020-11-28T21:46:23.407 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.383+0000 7f7c937fe700 1 -- 172.21.15.162:0/3972222281 >> v1:172.21.15.162:6789/0 conn(0x7f7c9405f380 legacy=0x7f7c941530e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:23.407 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.383+0000 7f7c937fe700 1 --2- 172.21.15.162:0/3972222281 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7c80003900 0x7f7c80003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:23.408 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.383+0000 7f7c937fe700 1 --2- 172.21.15.162:0/3972222281 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7c800044d0 0x7f7c80004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:23.408 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.383+0000 7f7c937fe700 1 --2- 172.21.15.162:0/3972222281 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c80004e40 0x7f7c8004f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:23.408 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.383+0000 7f7c937fe700 1 -- 172.21.15.162:0/3972222281 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c940ae310 con 0x7f7c80003900 2020-11-28T21:46:23.408 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.383+0000 7f7c937fe700 1 -- 172.21.15.162:0/3972222281 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8004fc20 con 0x7f7c80004e40 2020-11-28T21:46:23.408 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.383+0000 7f7c937fe700 1 -- 172.21.15.162:0/3972222281 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8004fd60 con 0x7f7c800044d0 2020-11-28T21:46:23.409 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.383+0000 7f7c98e34700 1 --2- 172.21.15.162:0/3972222281 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7c80003900 0x7f7c80003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:23.409 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.383+0000 7f7c93fff700 1 --2- 172.21.15.162:0/3972222281 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7c800044d0 0x7f7c80004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:23.409 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.383+0000 7f7c9a897700 1 --2- 172.21.15.162:0/3972222281 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c80004e40 0x7f7c8004f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:23.409 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.383+0000 7f7c98e34700 1 -- 172.21.15.162:0/3972222281 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7c800044d0 msgr2=0x7f7c80004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:23.410 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.383+0000 7f7c98e34700 1 --2- 172.21.15.162:0/3972222281 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7c800044d0 0x7f7c80004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:23.410 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.383+0000 7f7c98e34700 1 -- 172.21.15.162:0/3972222281 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c80004e40 msgr2=0x7f7c8004f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:23.410 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.383+0000 7f7c98e34700 1 --2- 172.21.15.162:0/3972222281 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c80004e40 0x7f7c8004f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:23.411 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.383+0000 7f7c98e34700 1 -- 172.21.15.162:0/3972222281 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c80003230 con 0x7f7c80003900 2020-11-28T21:46:23.411 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.383+0000 7f7c9a897700 1 -- 172.21.15.162:0/3972222281 reap_dead start 2020-11-28T21:46:23.411 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.387+0000 7f7c98e34700 1 --2- 172.21.15.162:0/3972222281 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7c80003900 0x7f7c80003d00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f7c7c001450 tx=0x7f7c7c007ee0).ready entity=mon.0 client_cookie=4af62acda310db64 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:23.411 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.387+0000 7f7c937fe700 1 -- 172.21.15.162:0/3972222281 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c7c00cbc0 con 0x7f7c80003900 2020-11-28T21:46:23.412 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.387+0000 7f7c937fe700 1 -- 172.21.15.162:0/3972222281 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c7c00cd20 con 0x7f7c80003900 2020-11-28T21:46:23.412 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.387+0000 7f7c937fe700 1 -- 172.21.15.162:0/3972222281 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c7c013600 con 0x7f7c80003900 2020-11-28T21:46:23.412 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.387+0000 7f7c99635700 1 -- 172.21.15.162:0/3972222281 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7c80003900 msgr2=0x7f7c80003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:23.412 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.387+0000 7f7c99635700 1 --2- 172.21.15.162:0/3972222281 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7c80003900 0x7f7c80003d00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f7c7c001450 tx=0x7f7c7c007ee0).stop 2020-11-28T21:46:23.412 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.387+0000 7f7c99635700 1 -- 172.21.15.162:0/3972222281 shutdown_connections 2020-11-28T21:46:23.413 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.387+0000 7f7c99635700 1 --2- 172.21.15.162:0/3972222281 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7c80003900 0x7f7c80003d00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:23.413 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.387+0000 7f7c99635700 1 -- 172.21.15.162:0/3972222281 >> 172.21.15.162:0/3972222281 conn(0x7f7c94063d80 msgr2=0x7f7c94064190 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:23.413 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.387+0000 7f7c99635700 1 -- 172.21.15.162:0/3972222281 shutdown_connections 2020-11-28T21:46:23.413 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.387+0000 7f7c99635700 1 -- 172.21.15.162:0/3972222281 wait complete. 2020-11-28T21:46:23.413 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.387+0000 7f7c99635700 1 Processor -- start 2020-11-28T21:46:23.414 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.387+0000 7f7c99635700 1 -- start start 2020-11-28T21:46:23.414 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.391+0000 7f7c99635700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7c80003900 0x7f7c94134c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:23.414 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.391+0000 7f7c99635700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7c940accf0 0x7f7c940aa1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:23.414 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.391+0000 7f7c99635700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c940aede0 0x7f7c940aa6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:23.415 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.391+0000 7f7c99635700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c940affa0 con 0x7f7c80003900 2020-11-28T21:46:23.415 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.391+0000 7f7c99635700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c940af360 con 0x7f7c940aede0 2020-11-28T21:46:23.415 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.391+0000 7f7c99635700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c940b01d0 con 0x7f7c940accf0 2020-11-28T21:46:23.415 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.391+0000 7f7c9a897700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c940aede0 0x7f7c940aa6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:23.415 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.391+0000 7f7c9a897700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c940aede0 0x7f7c940aa6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:36988/0 (socket says 172.21.15.162:36988) 2020-11-28T21:46:23.416 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.391+0000 7f7c9a897700 1 -- 172.21.15.162:0/3098547307 learned_addr learned my addr 172.21.15.162:0/3098547307 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:23.416 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.391+0000 7f7c98e34700 1 --2- 172.21.15.162:0/3098547307 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7c80003900 0x7f7c94134c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:23.416 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.391+0000 7f7c93fff700 1 --2- 172.21.15.162:0/3098547307 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7c940accf0 0x7f7c940aa1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:23.416 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.391+0000 7f7c9a897700 1 -- 172.21.15.162:0/3098547307 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7c940accf0 msgr2=0x7f7c940aa1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:23.417 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.391+0000 7f7c9a897700 1 --2- 172.21.15.162:0/3098547307 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7c940accf0 0x7f7c940aa1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:23.417 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.391+0000 7f7c9a897700 1 -- 172.21.15.162:0/3098547307 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7c80003900 msgr2=0x7f7c94134c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:23.417 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.391+0000 7f7c9a897700 1 --2- 172.21.15.162:0/3098547307 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7c80003900 0x7f7c94134c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:23.417 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.391+0000 7f7c9a897700 1 -- 172.21.15.162:0/3098547307 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c940af5c0 con 0x7f7c940aede0 2020-11-28T21:46:23.417 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.391+0000 7f7c9a897700 1 --2- 172.21.15.162:0/3098547307 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c940aede0 0x7f7c940aa6c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f7c940ac230 tx=0x7f7c84008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:23.418 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.391+0000 7f7c917fa700 1 -- 172.21.15.162:0/3098547307 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c84013750 con 0x7f7c940aede0 2020-11-28T21:46:23.418 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.391+0000 7f7c917fa700 1 -- 172.21.15.162:0/3098547307 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c840138b0 con 0x7f7c940aede0 2020-11-28T21:46:23.418 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.391+0000 7f7c99635700 1 -- 172.21.15.162:0/3098547307 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c940af860 con 0x7f7c940aede0 2020-11-28T21:46:23.418 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.391+0000 7f7c917fa700 1 -- 172.21.15.162:0/3098547307 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c84013c10 con 0x7f7c940aede0 2020-11-28T21:46:23.419 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.391+0000 7f7c99635700 1 -- 172.21.15.162:0/3098547307 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c940aadd0 con 0x7f7c940aede0 2020-11-28T21:46:23.419 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.391+0000 7f7c99635700 1 -- 172.21.15.162:0/3098547307 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c940af9f0 con 0x7f7c940aede0 2020-11-28T21:46:23.419 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.395+0000 7f7c917fa700 1 -- 172.21.15.162:0/3098547307 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7c84013d70 con 0x7f7c940aede0 2020-11-28T21:46:23.419 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.395+0000 7f7c917fa700 1 --2- 172.21.15.162:0/3098547307 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7c78055320 0x7f7c780577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:23.419 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.395+0000 7f7c917fa700 1 -- 172.21.15.162:0/3098547307 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4900+0+0 (secure 0 0 0) 0x7f7c8406e2d0 con 0x7f7c940aede0 2020-11-28T21:46:23.420 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.395+0000 7f7c98e34700 1 --2- 172.21.15.162:0/3098547307 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7c78055320 0x7f7c780577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:23.420 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.395+0000 7f7c98e34700 1 --2- 172.21.15.162:0/3098547307 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7c78055320 0x7f7c780577e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f7c7c01b000 tx=0x7f7c7c016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:23.420 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.403+0000 7f7c917fa700 1 -- 172.21.15.162:0/3098547307 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7c84044950 con 0x7f7c940aede0 2020-11-28T21:46:23.547 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.543+0000 7f7c99635700 1 -- 172.21.15.162:0/3098547307 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c94057050 con 0x7f7c940aede0 2020-11-28T21:46:23.549 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.547+0000 7f7c917fa700 1 -- 172.21.15.162:0/3098547307 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+11901 (secure 0 0 0) 0x7f7c84047f70 con 0x7f7c940aede0 2020-11-28T21:46:23.549 INFO:teuthology.misc.health.smithi162.stdout: 2020-11-28T21:46:23.549 INFO:teuthology.misc.health.smithi162.stdout:{"epoch":16,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:46:21.607785+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:46:23.551 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.547+0000 7f7c99635700 1 -- 172.21.15.162:0/3098547307 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7c78055320 msgr2=0x7f7c780577e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:23.551 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.547+0000 7f7c99635700 1 --2- 172.21.15.162:0/3098547307 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7c78055320 0x7f7c780577e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f7c7c01b000 tx=0x7f7c7c016000).stop 2020-11-28T21:46:23.551 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.547+0000 7f7c99635700 1 -- 172.21.15.162:0/3098547307 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c940aede0 msgr2=0x7f7c940aa6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:23.551 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.547+0000 7f7c99635700 1 --2- 172.21.15.162:0/3098547307 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c940aede0 0x7f7c940aa6c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f7c940ac230 tx=0x7f7c84008040).stop 2020-11-28T21:46:23.552 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.547+0000 7f7c99635700 1 -- 172.21.15.162:0/3098547307 shutdown_connections 2020-11-28T21:46:23.552 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.547+0000 7f7c99635700 1 --2- 172.21.15.162:0/3098547307 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7c78055320 0x7f7c780577e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:23.552 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.547+0000 7f7c99635700 1 --2- 172.21.15.162:0/3098547307 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c940aede0 0x7f7c940aa6c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:23.552 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.547+0000 7f7c99635700 1 --2- 172.21.15.162:0/3098547307 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7c80003900 0x7f7c94134c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:23.553 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.547+0000 7f7c99635700 1 --2- 172.21.15.162:0/3098547307 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7c940accf0 0x7f7c940aa1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:23.553 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.547+0000 7f7c99635700 1 -- 172.21.15.162:0/3098547307 >> 172.21.15.162:0/3098547307 conn(0x7f7c94063d80 msgr2=0x7f7c9405f810 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:23.553 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.547+0000 7f7c99635700 1 -- 172.21.15.162:0/3098547307 shutdown_connections 2020-11-28T21:46:23.553 INFO:teuthology.misc.health.smithi162.stderr:2020-11-28T21:46:23.547+0000 7f7c99635700 1 -- 172.21.15.162:0/3098547307 wait complete. 2020-11-28T21:46:23.563 DEBUG:teuthology.misc:8 of 8 OSDs are up 2020-11-28T21:46:23.564 INFO:tasks.ceph:Creating RBD pool 2020-11-28T21:46:23.564 INFO:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd pool create rbd 8 2020-11-28T21:46:23.685 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.679+0000 7fe103037700 1 Processor -- start 2020-11-28T21:46:23.686 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.679+0000 7fe103037700 1 -- start start 2020-11-28T21:46:23.686 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.679+0000 7fe103037700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe0fc0b2560 0x7fe0fc156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:23.686 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.679+0000 7fe103037700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe0fc05fa90 0x7fe0fc1593c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:23.686 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.679+0000 7fe103037700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0fc0ab0b0 con 0x7fe0fc0b2e80 2020-11-28T21:46:23.687 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.679+0000 7fe103037700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0fc0b3260 con 0x7fe0fc0b2560 2020-11-28T21:46:23.687 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.679+0000 7fe103037700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0fc0ab2e0 con 0x7fe0fc05fa90 2020-11-28T21:46:23.687 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.679+0000 7fe104299700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe0fc05fa90 0x7fe0fc1593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:23.687 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.679+0000 7fe104299700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe0fc05fa90 0x7fe0fc1593c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37000/0 (socket says 172.21.15.162:37000) 2020-11-28T21:46:23.687 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.679+0000 7fe104299700 1 -- 172.21.15.162:0/4221288383 learned_addr learned my addr 172.21.15.162:0/4221288383 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:23.688 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.679+0000 7fe102035700 1 --2- 172.21.15.162:0/4221288383 >> v2:172.21.15.118:3300/0 conn(0x7fe0fc0b2560 0x7fe0fc156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:23.688 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.679+0000 7fe104299700 1 -- 172.21.15.162:0/4221288383 >> v2:172.21.15.118:3300/0 conn(0x7fe0fc0b2560 msgr2=0x7fe0fc156ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:23.688 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.679+0000 7fe104299700 1 --2- 172.21.15.162:0/4221288383 >> v2:172.21.15.118:3300/0 conn(0x7fe0fc0b2560 0x7fe0fc156ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:23.688 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.679+0000 7fe104299700 1 -- 172.21.15.162:0/4221288383 >> v1:172.21.15.118:6789/0 conn(0x7fe0fc0b2e80 legacy=0x7fe0fc05f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:23.688 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.679+0000 7fe104299700 1 -- 172.21.15.162:0/4221288383 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0fc0256d0 con 0x7fe0fc05fa90 2020-11-28T21:46:23.689 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.679+0000 7fe101834700 1 -- 172.21.15.162:0/4221288383 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3475460324 0 0) 0x7fe0fc0ab0b0 con 0x7fe0fc0b2e80 2020-11-28T21:46:23.689 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.679+0000 7fe104299700 1 --2- 172.21.15.162:0/4221288383 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0fc05fa90 0x7fe0fc1593c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fe0ec002f30 tx=0x7fe0ec006e80).ready entity=mon.1 client_cookie=33e62716c688ca2f server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:23.689 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.679+0000 7fe101834700 1 -- 172.21.15.162:0/4221288383 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0ec0097f0 con 0x7fe0fc05fa90 2020-11-28T21:46:23.689 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.679+0000 7fe101834700 1 -- 172.21.15.162:0/4221288383 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0ec009950 con 0x7fe0fc05fa90 2020-11-28T21:46:23.690 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.679+0000 7fe101834700 1 -- 172.21.15.162:0/4221288383 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0ec009cb0 con 0x7fe0fc05fa90 2020-11-28T21:46:23.690 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.679+0000 7fe103037700 1 -- 172.21.15.162:0/4221288383 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0fc05fa90 msgr2=0x7fe0fc1593c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:23.691 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.679+0000 7fe103037700 1 --2- 172.21.15.162:0/4221288383 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0fc05fa90 0x7fe0fc1593c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fe0ec002f30 tx=0x7fe0ec006e80).stop 2020-11-28T21:46:23.691 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.683+0000 7fe103037700 1 -- 172.21.15.162:0/4221288383 shutdown_connections 2020-11-28T21:46:23.691 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.683+0000 7fe103037700 1 --2- 172.21.15.162:0/4221288383 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0fc05fa90 0x7fe0fc1593c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:23.691 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.683+0000 7fe103037700 1 --2- 172.21.15.162:0/4221288383 >> v2:172.21.15.118:3300/0 conn(0x7fe0fc0b2560 0x7fe0fc156ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:23.691 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.683+0000 7fe103037700 1 -- 172.21.15.162:0/4221288383 >> 172.21.15.162:0/4221288383 conn(0x7fe0fc063b60 msgr2=0x7fe0fc063f40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:23.692 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.683+0000 7fe103037700 1 -- 172.21.15.162:0/4221288383 shutdown_connections 2020-11-28T21:46:23.692 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.683+0000 7fe103037700 1 -- 172.21.15.162:0/4221288383 wait complete. 2020-11-28T21:46:23.695 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.683+0000 7fe103037700 1 Processor -- start 2020-11-28T21:46:23.695 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.683+0000 7fe103037700 1 -- start start 2020-11-28T21:46:23.695 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.687+0000 7fe103037700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0fc05fa90 0x7fe0fc130230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:23.695 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.687+0000 7fe103037700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe0fc0b2560 0x7fe0fc064840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:23.696 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.687+0000 7fe103037700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe0fc0b2e80 0x7fe0fc064d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:23.696 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.687+0000 7fe103037700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc09b590 con 0x7fe0fc0b2e80 2020-11-28T21:46:23.696 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.687+0000 7fe103037700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc09b830 con 0x7fe0fc05fa90 2020-11-28T21:46:23.696 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.687+0000 7fe103037700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc09bad0 con 0x7fe0fc0b2560 2020-11-28T21:46:23.696 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.687+0000 7fe102836700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0fc05fa90 0x7fe0fc130230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:23.697 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.687+0000 7fe102836700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0fc05fa90 0x7fe0fc130230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37002/0 (socket says 172.21.15.162:37002) 2020-11-28T21:46:23.697 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.687+0000 7fe102836700 1 -- 172.21.15.162:0/3586347611 learned_addr learned my addr 172.21.15.162:0/3586347611 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:23.697 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.687+0000 7fe104299700 1 --2- 172.21.15.162:0/3586347611 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe0fc0b2e80 0x7fe0fc064d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:23.697 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.687+0000 7fe102035700 1 --2- 172.21.15.162:0/3586347611 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe0fc0b2560 0x7fe0fc064840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:23.697 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.687+0000 7fe104299700 1 -- 172.21.15.162:0/3586347611 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe0fc0b2560 msgr2=0x7fe0fc064840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:23.698 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.687+0000 7fe104299700 1 --2- 172.21.15.162:0/3586347611 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe0fc0b2560 0x7fe0fc064840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:23.698 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.687+0000 7fe104299700 1 -- 172.21.15.162:0/3586347611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0fc05fa90 msgr2=0x7fe0fc130230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:23.698 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.687+0000 7fe104299700 1 --2- 172.21.15.162:0/3586347611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0fc05fa90 0x7fe0fc130230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:23.698 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.687+0000 7fe104299700 1 -- 172.21.15.162:0/3586347611 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0fc09bd70 con 0x7fe0fc0b2e80 2020-11-28T21:46:23.698 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.687+0000 7fe102035700 1 --2- 172.21.15.162:0/3586347611 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe0fc0b2560 0x7fe0fc064840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:46:23.699 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.687+0000 7fe102836700 1 --2- 172.21.15.162:0/3586347611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0fc05fa90 0x7fe0fc130230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:46:23.699 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.687+0000 7fe104299700 1 --2- 172.21.15.162:0/3586347611 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe0fc0b2e80 0x7fe0fc064d30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fe0ec002f30 tx=0x7fe0ec015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:23.699 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.687+0000 7fe0fb7fe700 1 -- 172.21.15.162:0/3586347611 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0ec009e60 con 0x7fe0fc0b2e80 2020-11-28T21:46:23.699 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.687+0000 7fe0fb7fe700 1 -- 172.21.15.162:0/3586347611 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0ec009c50 con 0x7fe0fc0b2e80 2020-11-28T21:46:23.699 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.687+0000 7fe0fb7fe700 1 -- 172.21.15.162:0/3586347611 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0ec01e650 con 0x7fe0fc0b2e80 2020-11-28T21:46:23.700 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.687+0000 7fe103037700 1 -- 172.21.15.162:0/3586347611 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0fc067520 con 0x7fe0fc0b2e80 2020-11-28T21:46:23.700 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.687+0000 7fe103037700 1 -- 172.21.15.162:0/3586347611 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0fc065440 con 0x7fe0fc0b2e80 2020-11-28T21:46:23.700 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.691+0000 7fe103037700 1 -- 172.21.15.162:0/3586347611 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0fc09bf00 con 0x7fe0fc0b2e80 2020-11-28T21:46:23.701 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.691+0000 7fe0fb7fe700 1 -- 172.21.15.162:0/3586347611 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe0ec01a040 con 0x7fe0fc0b2e80 2020-11-28T21:46:23.701 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.691+0000 7fe0fb7fe700 1 --2- 172.21.15.162:0/3586347611 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe0dc0554d0 0x7fe0dc057990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:23.701 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.691+0000 7fe0fb7fe700 1 -- 172.21.15.162:0/3586347611 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4900+0+0 (secure 0 0 0) 0x7fe0ec06ead0 con 0x7fe0fc0b2e80 2020-11-28T21:46:23.701 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.691+0000 7fe102836700 1 --2- 172.21.15.162:0/3586347611 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe0dc0554d0 0x7fe0dc057990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:23.701 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.691+0000 7fe102836700 1 --2- 172.21.15.162:0/3586347611 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe0dc0554d0 0x7fe0dc057990 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fe0f400d000 tx=0x7fe0f4009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:23.704 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.699+0000 7fe0fb7fe700 1 -- 172.21.15.162:0/3586347611 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe0ec0451c0 con 0x7fe0fc0b2e80 2020-11-28T21:46:23.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:23.843+0000 7fe103037700 1 -- 172.21.15.162:0/3586347611 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fe0fc057050 con 0x7fe0fc0b2e80 2020-11-28T21:46:24.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.635+0000 7fe0fb7fe700 1 -- 172.21.15.162:0/3586347611 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v17) v1 ==== 109+0+0 (secure 0 0 0) 0x7fe0ec0230a0 con 0x7fe0fc0b2e80 2020-11-28T21:46:24.639 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.635+0000 7fe103037700 1 -- 172.21.15.162:0/3586347611 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe0dc0554d0 msgr2=0x7fe0dc057990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:24.639 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.635+0000 7fe103037700 1 --2- 172.21.15.162:0/3586347611 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe0dc0554d0 0x7fe0dc057990 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fe0f400d000 tx=0x7fe0f4009040).stop 2020-11-28T21:46:24.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.635+0000 7fe103037700 1 -- 172.21.15.162:0/3586347611 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe0fc0b2e80 msgr2=0x7fe0fc064d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:24.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.635+0000 7fe103037700 1 --2- 172.21.15.162:0/3586347611 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe0fc0b2e80 0x7fe0fc064d30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fe0ec002f30 tx=0x7fe0ec015000).stop 2020-11-28T21:46:24.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.635+0000 7fe103037700 1 -- 172.21.15.162:0/3586347611 shutdown_connections 2020-11-28T21:46:24.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.635+0000 7fe103037700 1 --2- 172.21.15.162:0/3586347611 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe0dc0554d0 0x7fe0dc057990 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:24.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.635+0000 7fe103037700 1 --2- 172.21.15.162:0/3586347611 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe0fc0b2e80 0x7fe0fc064d30 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:24.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.635+0000 7fe103037700 1 --2- 172.21.15.162:0/3586347611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0fc05fa90 0x7fe0fc130230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:24.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.635+0000 7fe103037700 1 --2- 172.21.15.162:0/3586347611 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe0fc0b2560 0x7fe0fc064840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:24.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.635+0000 7fe103037700 1 -- 172.21.15.162:0/3586347611 >> 172.21.15.162:0/3586347611 conn(0x7fe0fc063b60 msgr2=0x7fe0fc0ac690 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:24.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.635+0000 7fe103037700 1 -- 172.21.15.162:0/3586347611 shutdown_connections 2020-11-28T21:46:24.642 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.635+0000 7fe103037700 1 -- 172.21.15.162:0/3586347611 wait complete. 2020-11-28T21:46:24.642 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'rbd' created 2020-11-28T21:46:24.654 INFO:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2020-11-28T21:46:24.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.763+0000 7ff1ee23f700 1 Processor -- start 2020-11-28T21:46:24.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.763+0000 7ff1ee23f700 1 -- start start 2020-11-28T21:46:24.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.767+0000 7ff1ee23f700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff1e80b0f60 0x7ff1e8156f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:24.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.767+0000 7ff1ee23f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff1e8157420 0x7ff1e8159810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:24.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.767+0000 7ff1ee23f700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1e80b3a50 con 0x7ff1e80b3670 2020-11-28T21:46:24.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.767+0000 7ff1ee23f700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1e8063620 con 0x7ff1e80b0f60 2020-11-28T21:46:24.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.767+0000 7ff1ee23f700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e80abd10 con 0x7ff1e8157420 2020-11-28T21:46:24.776 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.767+0000 7ff1ef4a1700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff1e8157420 0x7ff1e8159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:24.776 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.767+0000 7ff1ed23d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff1e80b0f60 0x7ff1e8156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:24.776 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.767+0000 7ff1ef4a1700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff1e8157420 0x7ff1e8159810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:34238/0 (socket says 172.21.15.162:34238) 2020-11-28T21:46:24.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.767+0000 7ff1ef4a1700 1 -- 172.21.15.162:0/844600247 learned_addr learned my addr 172.21.15.162:0/844600247 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:24.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.767+0000 7ff1ef4a1700 1 -- 172.21.15.162:0/844600247 >> v2:172.21.15.118:3300/0 conn(0x7ff1e80b0f60 msgr2=0x7ff1e8156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:24.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.767+0000 7ff1ef4a1700 1 --2- 172.21.15.162:0/844600247 >> v2:172.21.15.118:3300/0 conn(0x7ff1e80b0f60 0x7ff1e8156f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:24.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.767+0000 7ff1ef4a1700 1 -- 172.21.15.162:0/844600247 >> v1:172.21.15.118:6789/0 conn(0x7ff1e80b3670 legacy=0x7ff1e80ac170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:24.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.767+0000 7ff1ef4a1700 1 -- 172.21.15.162:0/844600247 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e8061a70 con 0x7ff1e8157420 2020-11-28T21:46:24.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.767+0000 7ff1ef4a1700 1 --2- 172.21.15.162:0/844600247 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff1e8157420 0x7ff1e8159810 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7ff1e00032b0 tx=0x7ff1e0009270).ready entity=mon.2 client_cookie=17d5bef3c48068a2 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:24.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.767+0000 7ff1eca3c700 1 -- 172.21.15.162:0/844600247 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1e0009e60 con 0x7ff1e8157420 2020-11-28T21:46:24.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.767+0000 7ff1eca3c700 1 -- 172.21.15.162:0/844600247 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e000bc00 con 0x7ff1e8157420 2020-11-28T21:46:24.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.767+0000 7ff1eca3c700 1 -- 172.21.15.162:0/844600247 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1e000bd70 con 0x7ff1e8157420 2020-11-28T21:46:24.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.767+0000 7ff1ee23f700 1 -- 172.21.15.162:0/844600247 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff1e8157420 msgr2=0x7ff1e8159810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:24.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.767+0000 7ff1ee23f700 1 --2- 172.21.15.162:0/844600247 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff1e8157420 0x7ff1e8159810 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7ff1e00032b0 tx=0x7ff1e0009270).stop 2020-11-28T21:46:24.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.767+0000 7ff1ee23f700 1 -- 172.21.15.162:0/844600247 shutdown_connections 2020-11-28T21:46:24.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.767+0000 7ff1ee23f700 1 --2- 172.21.15.162:0/844600247 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff1e8157420 0x7ff1e8159810 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:24.780 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.767+0000 7ff1ee23f700 1 --2- 172.21.15.162:0/844600247 >> v2:172.21.15.118:3300/0 conn(0x7ff1e80b0f60 0x7ff1e8156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:24.780 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.767+0000 7ff1ee23f700 1 -- 172.21.15.162:0/844600247 >> 172.21.15.162:0/844600247 conn(0x7ff1e80b6c50 msgr2=0x7ff1e80b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:24.781 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.767+0000 7ff1ee23f700 1 -- 172.21.15.162:0/844600247 shutdown_connections 2020-11-28T21:46:24.781 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.767+0000 7ff1ee23f700 1 -- 172.21.15.162:0/844600247 wait complete. 2020-11-28T21:46:24.784 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.771+0000 7ff1ee23f700 1 Processor -- start 2020-11-28T21:46:24.784 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.771+0000 7ff1ee23f700 1 -- start start 2020-11-28T21:46:24.784 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.771+0000 7ff1ee23f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1e80b0f60 0x7ff1e809b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:24.785 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.771+0000 7ff1ee23f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff1e80b3670 0x7ff1e8141d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:24.785 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.771+0000 7ff1ee23f700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff1e8157420 0x7ff1e8142250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:24.785 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.771+0000 7ff1ee23f700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e813e3f0 con 0x7ff1e8157420 2020-11-28T21:46:24.785 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.771+0000 7ff1ee23f700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e808fe90 con 0x7ff1e80b0f60 2020-11-28T21:46:24.785 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.771+0000 7ff1ee23f700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e8090130 con 0x7ff1e80b3670 2020-11-28T21:46:24.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.771+0000 7ff1eda3e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1e80b0f60 0x7ff1e809b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:24.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.771+0000 7ff1eda3e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1e80b0f60 0x7ff1e809b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37018/0 (socket says 172.21.15.162:37018) 2020-11-28T21:46:24.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.771+0000 7ff1eda3e700 1 -- 172.21.15.162:0/438916509 learned_addr learned my addr 172.21.15.162:0/438916509 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:24.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.771+0000 7ff1ef4a1700 1 --2- 172.21.15.162:0/438916509 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff1e8157420 0x7ff1e8142250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:24.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.771+0000 7ff1ed23d700 1 --2- 172.21.15.162:0/438916509 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff1e80b3670 0x7ff1e8141d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:24.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.775+0000 7ff1eda3e700 1 -- 172.21.15.162:0/438916509 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff1e80b3670 msgr2=0x7ff1e8141d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:24.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.775+0000 7ff1eda3e700 1 --2- 172.21.15.162:0/438916509 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff1e80b3670 0x7ff1e8141d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:24.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.775+0000 7ff1eda3e700 1 -- 172.21.15.162:0/438916509 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff1e8157420 msgr2=0x7ff1e8142250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:24.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.775+0000 7ff1eda3e700 1 --2- 172.21.15.162:0/438916509 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff1e8157420 0x7ff1e8142250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:24.788 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.775+0000 7ff1eda3e700 1 -- 172.21.15.162:0/438916509 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e80903d0 con 0x7ff1e80b0f60 2020-11-28T21:46:24.788 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.775+0000 7ff1ed23d700 1 --2- 172.21.15.162:0/438916509 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff1e80b3670 0x7ff1e8141d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:46:24.788 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.775+0000 7ff1eda3e700 1 --2- 172.21.15.162:0/438916509 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1e80b0f60 0x7ff1e809b7e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7ff1e8062d30 tx=0x7ff1e4007c20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:24.788 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.775+0000 7ff1de7fc700 1 -- 172.21.15.162:0/438916509 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1e400e4a0 con 0x7ff1e80b0f60 2020-11-28T21:46:24.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.775+0000 7ff1de7fc700 1 -- 172.21.15.162:0/438916509 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e400e600 con 0x7ff1e80b0f60 2020-11-28T21:46:24.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.775+0000 7ff1ee23f700 1 -- 172.21.15.162:0/438916509 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1e809bdf0 con 0x7ff1e80b0f60 2020-11-28T21:46:24.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.775+0000 7ff1de7fc700 1 -- 172.21.15.162:0/438916509 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1e400e960 con 0x7ff1e80b0f60 2020-11-28T21:46:24.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.775+0000 7ff1ee23f700 1 -- 172.21.15.162:0/438916509 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e8063fa0 con 0x7ff1e80b0f60 2020-11-28T21:46:24.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.775+0000 7ff1ee23f700 1 -- 172.21.15.162:0/438916509 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1e8057050 con 0x7ff1e80b0f60 2020-11-28T21:46:24.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.775+0000 7ff1de7fc700 1 -- 172.21.15.162:0/438916509 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff1e4010030 con 0x7ff1e80b0f60 2020-11-28T21:46:24.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.775+0000 7ff1de7fc700 1 --2- 172.21.15.162:0/438916509 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff1d0055500 0x7ff1d00579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:24.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.775+0000 7ff1de7fc700 1 -- 172.21.15.162:0/438916509 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5242+0+0 (secure 0 0 0) 0x7ff1e406dac0 con 0x7ff1e80b0f60 2020-11-28T21:46:24.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.775+0000 7ff1ed23d700 1 --2- 172.21.15.162:0/438916509 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff1d0055500 0x7ff1d00579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:24.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.779+0000 7ff1ed23d700 1 --2- 172.21.15.162:0/438916509 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff1d0055500 0x7ff1d00579c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7ff1d800b000 tx=0x7ff1d8008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:24.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.783+0000 7ff1de7fc700 1 -- 172.21.15.162:0/438916509 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff1e4044060 con 0x7ff1e80b0f60 2020-11-28T21:46:24.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:24.927+0000 7ff1ee23f700 1 -- 172.21.15.162:0/438916509 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff1e80b1710 con 0x7ff1e80b0f60 2020-11-28T21:46:25.646 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.643+0000 7ff1de7fc700 1 -- 172.21.15.162:0/438916509 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v18) v1 ==== 173+0+0 (secure 0 0 0) 0x7ff1e4047680 con 0x7ff1e80b0f60 2020-11-28T21:46:25.660 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.647+0000 7ff1ee23f700 1 -- 172.21.15.162:0/438916509 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff1d0055500 msgr2=0x7ff1d00579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:25.660 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.647+0000 7ff1ee23f700 1 --2- 172.21.15.162:0/438916509 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff1d0055500 0x7ff1d00579c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7ff1d800b000 tx=0x7ff1d8008040).stop 2020-11-28T21:46:25.661 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.647+0000 7ff1ee23f700 1 -- 172.21.15.162:0/438916509 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1e80b0f60 msgr2=0x7ff1e809b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:25.661 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.647+0000 7ff1ee23f700 1 --2- 172.21.15.162:0/438916509 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1e80b0f60 0x7ff1e809b7e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7ff1e8062d30 tx=0x7ff1e4007c20).stop 2020-11-28T21:46:25.661 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.647+0000 7ff1ee23f700 1 -- 172.21.15.162:0/438916509 shutdown_connections 2020-11-28T21:46:25.661 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.647+0000 7ff1ee23f700 1 --2- 172.21.15.162:0/438916509 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff1d0055500 0x7ff1d00579c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:25.662 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.647+0000 7ff1ee23f700 1 --2- 172.21.15.162:0/438916509 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff1e8157420 0x7ff1e8142250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:25.662 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.647+0000 7ff1ee23f700 1 --2- 172.21.15.162:0/438916509 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1e80b0f60 0x7ff1e809b7e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:25.662 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.647+0000 7ff1ee23f700 1 --2- 172.21.15.162:0/438916509 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff1e80b3670 0x7ff1e8141d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:25.662 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.647+0000 7ff1ee23f700 1 -- 172.21.15.162:0/438916509 >> 172.21.15.162:0/438916509 conn(0x7ff1e80b6c50 msgr2=0x7ff1e806a280 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:25.662 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.647+0000 7ff1ee23f700 1 -- 172.21.15.162:0/438916509 shutdown_connections 2020-11-28T21:46:25.663 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.647+0000 7ff1ee23f700 1 -- 172.21.15.162:0/438916509 wait complete. 2020-11-28T21:46:25.663 INFO:teuthology.orchestra.run.smithi162.stderr:enabled application 'rbd' on pool 'rbd' 2020-11-28T21:46:25.673 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2020-11-28T21:46:25.673 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-28T21:46:25.674 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-28T21:46:25.678 INFO:tasks.ceph.mds.a:Started 2020-11-28T21:46:25.679 INFO:tasks.ceph.mds.c:Restarting daemon 2020-11-28T21:46:25.679 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2020-11-28T21:46:25.684 INFO:tasks.ceph.mds.c:Started 2020-11-28T21:46:25.684 INFO:tasks.ceph.mds.b:Restarting daemon 2020-11-28T21:46:25.685 INFO:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-11-28T21:46:25.690 INFO:tasks.ceph.mds.b:Started 2020-11-28T21:46:25.690 INFO:tasks.ceph:Setting up CephFS filesystem... 2020-11-28T21:46:25.691 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:46:25.745 INFO:tasks.ceph.mds.a.smithi162.stdout:starting mds.a at 2020-11-28T21:46:25.755 INFO:tasks.ceph.mds.b.smithi118.stdout:starting mds.b at 2020-11-28T21:46:25.770 INFO:tasks.ceph.mds.c.smithi162.stdout:starting mds.c at 2020-11-28T21:46:25.812 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.807+0000 7f6d8b1e3700 1 Processor -- start 2020-11-28T21:46:25.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.807+0000 7f6d8b1e3700 1 -- start start 2020-11-28T21:46:25.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.811+0000 7f6d8b1e3700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f6d84146330 0x7f6d840b1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:25.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.811+0000 7f6d8b1e3700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6d840b1bf0 0x7f6d8405c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:25.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.811+0000 7f6d8b1e3700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d84146b00 con 0x7f6d8406aa70 2020-11-28T21:46:25.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.811+0000 7f6d8b1e3700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d84146da0 con 0x7f6d84146330 2020-11-28T21:46:25.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.811+0000 7f6d8b1e3700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d840b6610 con 0x7f6d840b1bf0 2020-11-28T21:46:25.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.811+0000 7f6d83fff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f6d8406aa70 0x7f6d8405b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:43896/0 (socket says 172.21.15.162:43896) 2020-11-28T21:46:25.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.811+0000 7f6d83fff700 1 -- 172.21.15.162:0/1704970034 learned_addr learned my addr 172.21.15.162:0/1704970034 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:25.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.811+0000 7f6d88f7f700 1 --2- 172.21.15.162:0/1704970034 >> v2:172.21.15.118:3300/0 conn(0x7f6d84146330 0x7f6d840b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:25.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.811+0000 7f6d89780700 1 --2- 172.21.15.162:0/1704970034 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6d840b1bf0 0x7f6d8405c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:25.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.811+0000 7f6d837fe700 1 -- 172.21.15.162:0/1704970034 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 642878670 0 0) 0x7f6d84146b00 con 0x7f6d8406aa70 2020-11-28T21:46:25.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.811+0000 7f6d837fe700 1 -- 172.21.15.162:0/1704970034 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d64003230 con 0x7f6d8406aa70 2020-11-28T21:46:25.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.811+0000 7f6d837fe700 1 -- 172.21.15.162:0/1704970034 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2268727085 0 0) 0x7f6d64003230 con 0x7f6d8406aa70 2020-11-28T21:46:25.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.811+0000 7f6d837fe700 1 -- 172.21.15.162:0/1704970034 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6d840b1bf0 msgr2=0x7f6d8405c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:25.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.811+0000 7f6d837fe700 1 --2- 172.21.15.162:0/1704970034 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6d840b1bf0 0x7f6d8405c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:25.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.811+0000 7f6d837fe700 1 -- 172.21.15.162:0/1704970034 >> v2:172.21.15.118:3300/0 conn(0x7f6d84146330 msgr2=0x7f6d840b1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:25.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.811+0000 7f6d837fe700 1 --2- 172.21.15.162:0/1704970034 >> v2:172.21.15.118:3300/0 conn(0x7f6d84146330 0x7f6d840b1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:25.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.811+0000 7f6d837fe700 1 -- 172.21.15.162:0/1704970034 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d840b0f80 con 0x7f6d8406aa70 2020-11-28T21:46:25.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.811+0000 7f6d837fe700 1 -- 172.21.15.162:0/1704970034 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f6d7c001a50 con 0x7f6d8406aa70 2020-11-28T21:46:25.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.811+0000 7f6d837fe700 1 -- 172.21.15.162:0/1704970034 >> v1:172.21.15.162:6789/0 conn(0x7f6d8406aa70 legacy=0x7f6d8405b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:25.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.811+0000 7f6d837fe700 1 --2- 172.21.15.162:0/1704970034 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6d840b1bf0 0x7f6d64003720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:25.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.811+0000 7f6d837fe700 1 --2- 172.21.15.162:0/1704970034 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d64003c30 0x7f6d640044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:25.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.811+0000 7f6d837fe700 1 --2- 172.21.15.162:0/1704970034 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6d640049c0 0x7f6d64004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:25.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.811+0000 7f6d837fe700 1 -- 172.21.15.162:0/1704970034 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d84146da0 con 0x7f6d640049c0 2020-11-28T21:46:25.838 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.811+0000 7f6d837fe700 1 -- 172.21.15.162:0/1704970034 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d840b6610 con 0x7f6d64003c30 2020-11-28T21:46:25.838 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.811+0000 7f6d837fe700 1 -- 172.21.15.162:0/1704970034 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d640041f0 con 0x7f6d840b1bf0 2020-11-28T21:46:25.839 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.811+0000 7f6d83fff700 1 --2- 172.21.15.162:0/1704970034 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d64003c30 0x7f6d640044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:25.839 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.811+0000 7f6d88f7f700 1 --2- 172.21.15.162:0/1704970034 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6d840b1bf0 0x7f6d64003720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:25.840 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.811+0000 7f6d89780700 1 --2- 172.21.15.162:0/1704970034 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6d640049c0 0x7f6d64004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:25.840 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.811+0000 7f6d83fff700 1 -- 172.21.15.162:0/1704970034 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6d840b1bf0 msgr2=0x7f6d64003720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:25.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.811+0000 7f6d83fff700 1 --2- 172.21.15.162:0/1704970034 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6d840b1bf0 0x7f6d64003720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:25.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.811+0000 7f6d83fff700 1 -- 172.21.15.162:0/1704970034 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6d640049c0 msgr2=0x7f6d64004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:25.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.811+0000 7f6d83fff700 1 --2- 172.21.15.162:0/1704970034 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6d640049c0 0x7f6d64004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:25.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.811+0000 7f6d83fff700 1 -- 172.21.15.162:0/1704970034 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d64003230 con 0x7f6d64003c30 2020-11-28T21:46:25.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.811+0000 7f6d83fff700 1 --2- 172.21.15.162:0/1704970034 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d64003c30 0x7f6d640044d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f6d7c001760 tx=0x7f6d7c007b40).ready entity=mon.1 client_cookie=405315ae9c1dd885 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:25.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.811+0000 7f6d837fe700 1 -- 172.21.15.162:0/1704970034 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d7c005b70 con 0x7f6d64003c30 2020-11-28T21:46:25.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.811+0000 7f6d837fe700 1 -- 172.21.15.162:0/1704970034 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d7c005cd0 con 0x7f6d64003c30 2020-11-28T21:46:25.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.811+0000 7f6d837fe700 1 -- 172.21.15.162:0/1704970034 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d7c0072c0 con 0x7f6d64003c30 2020-11-28T21:46:25.843 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.811+0000 7f6d8b1e3700 1 -- 172.21.15.162:0/1704970034 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d64003c30 msgr2=0x7f6d640044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:25.843 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.811+0000 7f6d8b1e3700 1 --2- 172.21.15.162:0/1704970034 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d64003c30 0x7f6d640044d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f6d7c001760 tx=0x7f6d7c007b40).stop 2020-11-28T21:46:25.843 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.811+0000 7f6d89780700 1 -- 172.21.15.162:0/1704970034 reap_dead start 2020-11-28T21:46:25.843 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.815+0000 7f6d8b1e3700 1 -- 172.21.15.162:0/1704970034 shutdown_connections 2020-11-28T21:46:25.844 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.815+0000 7f6d8b1e3700 1 -- 172.21.15.162:0/1704970034 >> 172.21.15.162:0/1704970034 conn(0x7f6d840b56f0 msgr2=0x7f6d840b5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:25.844 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.815+0000 7f6d8b1e3700 1 -- 172.21.15.162:0/1704970034 shutdown_connections 2020-11-28T21:46:25.844 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.815+0000 7f6d8b1e3700 1 -- 172.21.15.162:0/1704970034 wait complete. 2020-11-28T21:46:25.844 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.815+0000 7f6d8b1e3700 1 Processor -- start 2020-11-28T21:46:25.844 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.815+0000 7f6d8b1e3700 1 -- start start 2020-11-28T21:46:25.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.815+0000 7f6d8b1e3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d84155130 0x7f6d841429e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:25.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.815+0000 7f6d8b1e3700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6d840a8a00 0x7f6d84155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:25.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.815+0000 7f6d8b1e3700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6d84155a00 0x7f6d8413fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:25.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.815+0000 7f6d8b1e3700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d84063d00 con 0x7f6d840a8a00 2020-11-28T21:46:25.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.815+0000 7f6d8b1e3700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d84142ed0 con 0x7f6d84155130 2020-11-28T21:46:25.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.815+0000 7f6d8b1e3700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d84143390 con 0x7f6d84155a00 2020-11-28T21:46:25.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.815+0000 7f6d88f7f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d84155130 0x7f6d841429e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:25.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.815+0000 7f6d88f7f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d84155130 0x7f6d841429e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37062/0 (socket says 172.21.15.162:37062) 2020-11-28T21:46:25.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.815+0000 7f6d88f7f700 1 -- 172.21.15.162:0/2780931726 learned_addr learned my addr 172.21.15.162:0/2780931726 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:25.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.815+0000 7f6d89780700 1 --2- 172.21.15.162:0/2780931726 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6d84155a00 0x7f6d8413fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:25.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.819+0000 7f6d88f7f700 1 -- 172.21.15.162:0/2780931726 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6d84155a00 msgr2=0x7f6d8413fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:25.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.819+0000 7f6d88f7f700 1 --2- 172.21.15.162:0/2780931726 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6d84155a00 0x7f6d8413fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:25.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.819+0000 7f6d88f7f700 1 -- 172.21.15.162:0/2780931726 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6d840a8a00 msgr2=0x7f6d84155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:25.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.819+0000 7f6d88f7f700 1 --2- 172.21.15.162:0/2780931726 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6d840a8a00 0x7f6d84155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-28T21:46:25.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.819+0000 7f6d88f7f700 1 -- 172.21.15.162:0/2780931726 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d84143630 con 0x7f6d84155130 2020-11-28T21:46:25.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.819+0000 7f6d88f7f700 1 --2- 172.21.15.162:0/2780931726 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d84155130 0x7f6d841429e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f6d6c006750 tx=0x7f6d6c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:25.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.819+0000 7f6d81ffb700 1 -- 172.21.15.162:0/2780931726 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d6c011e40 con 0x7f6d84155130 2020-11-28T21:46:25.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.819+0000 7f6d81ffb700 1 -- 172.21.15.162:0/2780931726 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d6c00fca0 con 0x7f6d84155130 2020-11-28T21:46:25.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.819+0000 7f6d81ffb700 1 -- 172.21.15.162:0/2780931726 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d6c00fe10 con 0x7f6d84155130 2020-11-28T21:46:25.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.819+0000 7f6d8b1e3700 1 -- 172.21.15.162:0/2780931726 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d84063f30 con 0x7f6d84155130 2020-11-28T21:46:25.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.819+0000 7f6d8b1e3700 1 -- 172.21.15.162:0/2780931726 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d840ac070 con 0x7f6d84155130 2020-11-28T21:46:25.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.819+0000 7f6d8b1e3700 1 -- 172.21.15.162:0/2780931726 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d840640c0 con 0x7f6d84155130 2020-11-28T21:46:25.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.819+0000 7f6d81ffb700 1 -- 172.21.15.162:0/2780931726 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6d6c024070 con 0x7f6d84155130 2020-11-28T21:46:25.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.819+0000 7f6d81ffb700 1 --2- 172.21.15.162:0/2780931726 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6d68055350 0x7f6d68057810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:25.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.819+0000 7f6d83fff700 1 --2- 172.21.15.162:0/2780931726 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6d68055350 0x7f6d68057810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:25.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.819+0000 7f6d81ffb700 1 -- 172.21.15.162:0/2780931726 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f6d6c06ecc0 con 0x7f6d84155130 2020-11-28T21:46:25.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.823+0000 7f6d83fff700 1 --2- 172.21.15.162:0/2780931726 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6d68055350 0x7f6d68057810 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f6d7c001760 tx=0x7f6d7c015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:25.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.827+0000 7f6d81ffb700 1 -- 172.21.15.162:0/2780931726 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6d6c0451e0 con 0x7f6d84155130 2020-11-28T21:46:25.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.967+0000 7f6d8b1e3700 1 -- 172.21.15.162:0/2780931726 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6d84057050 con 0x7f6d84155130 2020-11-28T21:46:25.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.971+0000 7f6d81ffb700 1 -- 172.21.15.162:0/2780931726 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+13287 (secure 0 0 0) 0x7f6d6c02e400 con 0x7f6d84155130 2020-11-28T21:46:25.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.971+0000 7f6d8b1e3700 1 -- 172.21.15.162:0/2780931726 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6d68055350 msgr2=0x7f6d68057810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:25.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.971+0000 7f6d8b1e3700 1 --2- 172.21.15.162:0/2780931726 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6d68055350 0x7f6d68057810 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f6d7c001760 tx=0x7f6d7c015000).stop 2020-11-28T21:46:25.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.971+0000 7f6d8b1e3700 1 -- 172.21.15.162:0/2780931726 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d84155130 msgr2=0x7f6d841429e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:25.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.971+0000 7f6d8b1e3700 1 --2- 172.21.15.162:0/2780931726 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d84155130 0x7f6d841429e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f6d6c006750 tx=0x7f6d6c008040).stop 2020-11-28T21:46:25.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.971+0000 7f6d8b1e3700 1 -- 172.21.15.162:0/2780931726 shutdown_connections 2020-11-28T21:46:25.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.971+0000 7f6d8b1e3700 1 --2- 172.21.15.162:0/2780931726 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6d68055350 0x7f6d68057810 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:25.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.971+0000 7f6d8b1e3700 1 --2- 172.21.15.162:0/2780931726 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6d84155a00 0x7f6d8413fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:25.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.971+0000 7f6d8b1e3700 1 --2- 172.21.15.162:0/2780931726 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d84155130 0x7f6d841429e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:25.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.971+0000 7f6d8b1e3700 1 --2- 172.21.15.162:0/2780931726 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6d840a8a00 0x7f6d84155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-28T21:46:25.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.971+0000 7f6d8b1e3700 1 -- 172.21.15.162:0/2780931726 >> 172.21.15.162:0/2780931726 conn(0x7f6d840b56f0 msgr2=0x7f6d840b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:25.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.971+0000 7f6d8b1e3700 1 -- 172.21.15.162:0/2780931726 shutdown_connections 2020-11-28T21:46:25.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:25.971+0000 7f6d8b1e3700 1 -- 172.21.15.162:0/2780931726 wait complete. 2020-11-28T21:46:25.985 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:46:25.985 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":18,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:46:25.631822+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:46:25.988 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-28T21:46:25.989 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:46:26.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.103+0000 7f7f74bcd700 1 Processor -- start 2020-11-28T21:46:26.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.103+0000 7f7f74bcd700 1 -- start start 2020-11-28T21:46:26.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.107+0000 7f7f74bcd700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f7f700b1eb0 0x7f7f700b2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:26.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.107+0000 7f7f74bcd700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7f700af6a0 0x7f7f70060a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:26.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.107+0000 7f7f74bcd700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f701471f0 con 0x7f7f70144350 2020-11-28T21:46:26.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.107+0000 7f7f74bcd700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f70060090 con 0x7f7f700b1eb0 2020-11-28T21:46:26.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.107+0000 7f7f74bcd700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f70060330 con 0x7f7f700af6a0 2020-11-28T21:46:26.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.107+0000 7f7f6effd700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7f700af6a0 0x7f7f70060a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:26.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.107+0000 7f7f6f7fe700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f7f700b1eb0 0x7f7f700b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:26.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.107+0000 7f7f6effd700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7f700af6a0 0x7f7f70060a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:34294/0 (socket says 172.21.15.162:34294) 2020-11-28T21:46:26.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.107+0000 7f7f6effd700 1 -- 172.21.15.162:0/3134677511 learned_addr learned my addr 172.21.15.162:0/3134677511 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:26.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.107+0000 7f7f6effd700 1 -- 172.21.15.162:0/3134677511 >> v2:172.21.15.118:3300/0 conn(0x7f7f700b1eb0 msgr2=0x7f7f700b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:26.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.107+0000 7f7f6effd700 1 --2- 172.21.15.162:0/3134677511 >> v2:172.21.15.118:3300/0 conn(0x7f7f700b1eb0 0x7f7f700b2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:26.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.107+0000 7f7f6effd700 1 -- 172.21.15.162:0/3134677511 >> v1:172.21.15.118:6789/0 conn(0x7f7f70144350 legacy=0x7f7f7014a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:26.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.107+0000 7f7f6effd700 1 -- 172.21.15.162:0/3134677511 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f700605d0 con 0x7f7f700af6a0 2020-11-28T21:46:26.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.107+0000 7f7f6effd700 1 --2- 172.21.15.162:0/3134677511 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7f700af6a0 0x7f7f70060a70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f7f60002f30 tx=0x7f7f60006e80).ready entity=mon.2 client_cookie=36b64d771a5704d1 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:26.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.107+0000 7f7f6e7fc700 1 -- 172.21.15.162:0/3134677511 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f600097f0 con 0x7f7f700af6a0 2020-11-28T21:46:26.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.107+0000 7f7f6e7fc700 1 -- 172.21.15.162:0/3134677511 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f60009950 con 0x7f7f700af6a0 2020-11-28T21:46:26.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.107+0000 7f7f6e7fc700 1 -- 172.21.15.162:0/3134677511 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f60009cb0 con 0x7f7f700af6a0 2020-11-28T21:46:26.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.107+0000 7f7f74bcd700 1 -- 172.21.15.162:0/3134677511 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7f700af6a0 msgr2=0x7f7f70060a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:26.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.107+0000 7f7f74bcd700 1 --2- 172.21.15.162:0/3134677511 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7f700af6a0 0x7f7f70060a70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f7f60002f30 tx=0x7f7f60006e80).stop 2020-11-28T21:46:26.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.107+0000 7f7f74bcd700 1 -- 172.21.15.162:0/3134677511 shutdown_connections 2020-11-28T21:46:26.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.107+0000 7f7f74bcd700 1 --2- 172.21.15.162:0/3134677511 >> v2:172.21.15.118:3300/0 conn(0x7f7f700b1eb0 0x7f7f700b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:26.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.107+0000 7f7f74bcd700 1 --2- 172.21.15.162:0/3134677511 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7f700af6a0 0x7f7f70060a70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:26.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.107+0000 7f7f74bcd700 1 -- 172.21.15.162:0/3134677511 >> 172.21.15.162:0/3134677511 conn(0x7f7f700b57d0 msgr2=0x7f7f700b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:26.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.107+0000 7f7f74bcd700 1 -- 172.21.15.162:0/3134677511 shutdown_connections 2020-11-28T21:46:26.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.107+0000 7f7f74bcd700 1 -- 172.21.15.162:0/3134677511 wait complete. 2020-11-28T21:46:26.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.107+0000 7f7f74bcd700 1 Processor -- start 2020-11-28T21:46:26.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.111+0000 7f7f74bcd700 1 -- start start 2020-11-28T21:46:26.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.111+0000 7f7f74bcd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7f700af6a0 0x7f7f700656c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:26.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.111+0000 7f7f74bcd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f700b1eb0 0x7f7f70146b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:26.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.111+0000 7f7f74bcd700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7f70144350 0x7f7f70159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:26.129 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.111+0000 7f7f74bcd700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f700a0810 con 0x7f7f700af6a0 2020-11-28T21:46:26.129 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.111+0000 7f7f74bcd700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f700a0ab0 con 0x7f7f700b1eb0 2020-11-28T21:46:26.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.111+0000 7f7f74bcd700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f700a0d50 con 0x7f7f70144350 2020-11-28T21:46:26.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.111+0000 7f7f6effd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f700b1eb0 0x7f7f70146b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:26.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.111+0000 7f7f6effd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f700b1eb0 0x7f7f70146b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37078/0 (socket says 172.21.15.162:37078) 2020-11-28T21:46:26.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.111+0000 7f7f6effd700 1 -- 172.21.15.162:0/3257563629 learned_addr learned my addr 172.21.15.162:0/3257563629 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:26.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.111+0000 7f7f6f7fe700 1 --2- 172.21.15.162:0/3257563629 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7f700af6a0 0x7f7f700656c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:26.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.111+0000 7f7f6ffff700 1 --2- 172.21.15.162:0/3257563629 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7f70144350 0x7f7f70159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:26.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.111+0000 7f7f6effd700 1 -- 172.21.15.162:0/3257563629 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7f70144350 msgr2=0x7f7f70159c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:26.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.111+0000 7f7f6effd700 1 --2- 172.21.15.162:0/3257563629 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7f70144350 0x7f7f70159c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:26.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.111+0000 7f7f6effd700 1 -- 172.21.15.162:0/3257563629 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7f700af6a0 msgr2=0x7f7f700656c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:26.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.111+0000 7f7f6effd700 1 --2- 172.21.15.162:0/3257563629 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7f700af6a0 0x7f7f700656c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:26.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.111+0000 7f7f6effd700 1 -- 172.21.15.162:0/3257563629 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f70159790 con 0x7f7f700b1eb0 2020-11-28T21:46:26.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.111+0000 7f7f6effd700 1 --2- 172.21.15.162:0/3257563629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f700b1eb0 0x7f7f70146b60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f7f60002f30 tx=0x7f7f60015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:26.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.111+0000 7f7f6cff9700 1 -- 172.21.15.162:0/3257563629 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f60011bf0 con 0x7f7f700b1eb0 2020-11-28T21:46:26.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.111+0000 7f7f6cff9700 1 -- 172.21.15.162:0/3257563629 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f60011d50 con 0x7f7f700b1eb0 2020-11-28T21:46:26.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.111+0000 7f7f74bcd700 1 -- 172.21.15.162:0/3257563629 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f7009ee70 con 0x7f7f700b1eb0 2020-11-28T21:46:26.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.111+0000 7f7f6cff9700 1 -- 172.21.15.162:0/3257563629 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f6001e650 con 0x7f7f700b1eb0 2020-11-28T21:46:26.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.111+0000 7f7f74bcd700 1 -- 172.21.15.162:0/3257563629 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f700a04b0 con 0x7f7f700b1eb0 2020-11-28T21:46:26.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.115+0000 7f7f74bcd700 1 -- 172.21.15.162:0/3257563629 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f70159920 con 0x7f7f700b1eb0 2020-11-28T21:46:26.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.115+0000 7f7f6cff9700 1 -- 172.21.15.162:0/3257563629 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7f60025030 con 0x7f7f700b1eb0 2020-11-28T21:46:26.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.115+0000 7f7f6cff9700 1 --2- 172.21.15.162:0/3257563629 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7f50055440 0x7f7f50057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:26.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.115+0000 7f7f6cff9700 1 -- 172.21.15.162:0/3257563629 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f7f6006eff0 con 0x7f7f700b1eb0 2020-11-28T21:46:26.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.115+0000 7f7f6f7fe700 1 --2- 172.21.15.162:0/3257563629 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7f50055440 0x7f7f50057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:26.138 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.115+0000 7f7f6f7fe700 1 --2- 172.21.15.162:0/3257563629 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7f50055440 0x7f7f50057900 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f7f64008000 tx=0x7f7f64008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:26.138 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.119+0000 7f7f6cff9700 1 -- 172.21.15.162:0/3257563629 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7f60045510 con 0x7f7f700b1eb0 2020-11-28T21:46:26.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.259+0000 7f7f74bcd700 1 -- 172.21.15.162:0/3257563629 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f7f70057050 con 0x7f7f700b1eb0 2020-11-28T21:46:26.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.263+0000 7f7f6cff9700 1 -- 172.21.15.162:0/3257563629 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v18) v1 ==== 110+0+10 (secure 0 0 0) 0x7f7f60023020 con 0x7f7f700b1eb0 2020-11-28T21:46:26.264 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:46:26.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.263+0000 7f7f74bcd700 1 -- 172.21.15.162:0/3257563629 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7f50055440 msgr2=0x7f7f50057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:26.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.263+0000 7f7f74bcd700 1 --2- 172.21.15.162:0/3257563629 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7f50055440 0x7f7f50057900 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f7f64008000 tx=0x7f7f64008040).stop 2020-11-28T21:46:26.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.263+0000 7f7f74bcd700 1 -- 172.21.15.162:0/3257563629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f700b1eb0 msgr2=0x7f7f70146b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:26.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.263+0000 7f7f74bcd700 1 --2- 172.21.15.162:0/3257563629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f700b1eb0 0x7f7f70146b60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f7f60002f30 tx=0x7f7f60015000).stop 2020-11-28T21:46:26.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.263+0000 7f7f74bcd700 1 -- 172.21.15.162:0/3257563629 shutdown_connections 2020-11-28T21:46:26.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.263+0000 7f7f74bcd700 1 --2- 172.21.15.162:0/3257563629 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7f50055440 0x7f7f50057900 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:26.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.263+0000 7f7f74bcd700 1 --2- 172.21.15.162:0/3257563629 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7f70144350 0x7f7f70159c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:26.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.263+0000 7f7f74bcd700 1 --2- 172.21.15.162:0/3257563629 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7f700af6a0 0x7f7f700656c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:26.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.263+0000 7f7f74bcd700 1 --2- 172.21.15.162:0/3257563629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f700b1eb0 0x7f7f70146b60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:26.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.263+0000 7f7f74bcd700 1 -- 172.21.15.162:0/3257563629 >> 172.21.15.162:0/3257563629 conn(0x7f7f700b57d0 msgr2=0x7f7f700b3640 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:26.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.263+0000 7f7f74bcd700 1 -- 172.21.15.162:0/3257563629 shutdown_connections 2020-11-28T21:46:26.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.263+0000 7f7f74bcd700 1 -- 172.21.15.162:0/3257563629 wait complete. 2020-11-28T21:46:26.278 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:46:26.392 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.387+0000 7ff5bea2c700 1 Processor -- start 2020-11-28T21:46:26.392 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.387+0000 7ff5bea2c700 1 -- start start 2020-11-28T21:46:26.394 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.391+0000 7ff5bea2c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff5b8060b60 0x7ff5b814e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:26.395 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.391+0000 7ff5bea2c700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5b80aff20 con 0x7ff5b81404c0 2020-11-28T21:46:26.395 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.391+0000 7ff5bea2c700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5b80b25d0 con 0x7ff5b80b21f0 2020-11-28T21:46:26.395 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.391+0000 7ff5bea2c700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5b80af240 con 0x7ff5b8060b60 2020-11-28T21:46:26.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.391+0000 7ff5be22b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff5b8060b60 0x7ff5b814e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:26.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.391+0000 7ff5be22b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff5b8060b60 0x7ff5b814e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37088/0 (socket says 172.21.15.162:37088) 2020-11-28T21:46:26.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.391+0000 7ff5be22b700 1 -- 172.21.15.162:0/1351843522 learned_addr learned my addr 172.21.15.162:0/1351843522 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:26.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.391+0000 7ff5bda2a700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7ff5b80b21f0 0x7ff5b813fdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:43930/0 (socket says 172.21.15.162:43930) 2020-11-28T21:46:26.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.391+0000 7ff5be22b700 1 -- 172.21.15.162:0/1351843522 >> v1:172.21.15.162:6789/0 conn(0x7ff5b80b21f0 legacy=0x7ff5b813fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:26.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.391+0000 7ff5be22b700 1 -- 172.21.15.162:0/1351843522 >> v1:172.21.15.118:6789/0 conn(0x7ff5b81404c0 legacy=0x7ff5b8060450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:26.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.391+0000 7ff5be22b700 1 -- 172.21.15.162:0/1351843522 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b805fff0 con 0x7ff5b8060b60 2020-11-28T21:46:26.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.391+0000 7ff5be22b700 1 --2- 172.21.15.162:0/1351843522 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5b8060b60 0x7ff5b814e3e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7ff5b00032b0 tx=0x7ff5b0009270).ready entity=mon.1 client_cookie=aa3500c052b73347 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:26.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.391+0000 7ff5bca28700 1 -- 172.21.15.162:0/1351843522 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff5b000bc00 con 0x7ff5b8060b60 2020-11-28T21:46:26.398 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.391+0000 7ff5bca28700 1 -- 172.21.15.162:0/1351843522 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b000bd60 con 0x7ff5b8060b60 2020-11-28T21:46:26.398 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.391+0000 7ff5bea2c700 1 -- 172.21.15.162:0/1351843522 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5b8060b60 msgr2=0x7ff5b814e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:26.398 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.391+0000 7ff5bca28700 1 -- 172.21.15.162:0/1351843522 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff5b0011400 con 0x7ff5b8060b60 2020-11-28T21:46:26.398 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.391+0000 7ff5bea2c700 1 --2- 172.21.15.162:0/1351843522 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5b8060b60 0x7ff5b814e3e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7ff5b00032b0 tx=0x7ff5b0009270).stop 2020-11-28T21:46:26.399 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.391+0000 7ff5bea2c700 1 -- 172.21.15.162:0/1351843522 shutdown_connections 2020-11-28T21:46:26.399 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.391+0000 7ff5bea2c700 1 --2- 172.21.15.162:0/1351843522 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5b8060b60 0x7ff5b814e3e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:26.399 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.391+0000 7ff5bea2c700 1 -- 172.21.15.162:0/1351843522 >> 172.21.15.162:0/1351843522 conn(0x7ff5b80b57d0 msgr2=0x7ff5b80b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:26.399 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.391+0000 7ff5bea2c700 1 -- 172.21.15.162:0/1351843522 shutdown_connections 2020-11-28T21:46:26.399 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.391+0000 7ff5bea2c700 1 -- 172.21.15.162:0/1351843522 wait complete. 2020-11-28T21:46:26.402 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.391+0000 7ff5bea2c700 1 Processor -- start 2020-11-28T21:46:26.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.391+0000 7ff5bea2c700 1 -- start start 2020-11-28T21:46:26.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.395+0000 7ff5bea2c700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff5b8060b60 0x7ff5b8146370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:26.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.395+0000 7ff5bea2c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff5b80b21f0 0x7ff5b813e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:26.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.395+0000 7ff5bea2c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5b81404c0 0x7ff5b813e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:26.404 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.395+0000 7ff5bea2c700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b80a3670 con 0x7ff5b80b21f0 2020-11-28T21:46:26.404 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.395+0000 7ff5bea2c700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b80659d0 con 0x7ff5b81404c0 2020-11-28T21:46:26.404 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.395+0000 7ff5bea2c700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b8064bb0 con 0x7ff5b8060b60 2020-11-28T21:46:26.404 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.395+0000 7ff5be22b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5b81404c0 0x7ff5b813e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:26.404 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.395+0000 7ff5be22b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5b81404c0 0x7ff5b813e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37094/0 (socket says 172.21.15.162:37094) 2020-11-28T21:46:26.405 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.395+0000 7ff5be22b700 1 -- 172.21.15.162:0/2620041466 learned_addr learned my addr 172.21.15.162:0/2620041466 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:26.405 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.395+0000 7ff5bda2a700 1 --2- 172.21.15.162:0/2620041466 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff5b8060b60 0x7ff5b8146370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:26.405 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.395+0000 7ff5bd229700 1 --2- 172.21.15.162:0/2620041466 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff5b80b21f0 0x7ff5b813e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:26.405 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.395+0000 7ff5be22b700 1 -- 172.21.15.162:0/2620041466 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff5b8060b60 msgr2=0x7ff5b8146370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:26.405 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.395+0000 7ff5be22b700 1 --2- 172.21.15.162:0/2620041466 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff5b8060b60 0x7ff5b8146370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:26.406 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.395+0000 7ff5be22b700 1 -- 172.21.15.162:0/2620041466 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff5b80b21f0 msgr2=0x7ff5b813e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:26.406 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.395+0000 7ff5be22b700 1 --2- 172.21.15.162:0/2620041466 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff5b80b21f0 0x7ff5b813e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:26.406 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.395+0000 7ff5be22b700 1 -- 172.21.15.162:0/2620041466 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b8064e50 con 0x7ff5b81404c0 2020-11-28T21:46:26.406 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.395+0000 7ff5be22b700 1 --2- 172.21.15.162:0/2620041466 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5b81404c0 0x7ff5b813e4f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7ff5b8064970 tx=0x7ff5b0016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:26.407 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.395+0000 7ff5a6ffd700 1 -- 172.21.15.162:0/2620041466 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff5b0011a30 con 0x7ff5b81404c0 2020-11-28T21:46:26.407 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.395+0000 7ff5a6ffd700 1 -- 172.21.15.162:0/2620041466 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b0011b90 con 0x7ff5b81404c0 2020-11-28T21:46:26.407 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.395+0000 7ff5bea2c700 1 -- 172.21.15.162:0/2620041466 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5b80650f0 con 0x7ff5b81404c0 2020-11-28T21:46:26.407 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.395+0000 7ff5a6ffd700 1 -- 172.21.15.162:0/2620041466 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff5b0006040 con 0x7ff5b81404c0 2020-11-28T21:46:26.407 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.395+0000 7ff5bea2c700 1 -- 172.21.15.162:0/2620041466 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5b80a48a0 con 0x7ff5b81404c0 2020-11-28T21:46:26.408 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.395+0000 7ff5bea2c700 1 -- 172.21.15.162:0/2620041466 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5b8057050 con 0x7ff5b81404c0 2020-11-28T21:46:26.408 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.399+0000 7ff5a6ffd700 1 -- 172.21.15.162:0/2620041466 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff5b0026030 con 0x7ff5b81404c0 2020-11-28T21:46:26.408 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.399+0000 7ff5a6ffd700 1 --2- 172.21.15.162:0/2620041466 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff598055330 0x7ff5980577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:26.408 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.399+0000 7ff5bda2a700 1 --2- 172.21.15.162:0/2620041466 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff598055330 0x7ff5980577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:26.408 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.399+0000 7ff5a6ffd700 1 -- 172.21.15.162:0/2620041466 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5253+0+0 (secure 0 0 0) 0x7ff5b006e740 con 0x7ff5b81404c0 2020-11-28T21:46:26.409 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.399+0000 7ff5bda2a700 1 --2- 172.21.15.162:0/2620041466 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff598055330 0x7ff5980577f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7ff5b400d000 tx=0x7ff5b4007b10).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:26.409 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.403+0000 7ff5a6ffd700 1 -- 172.21.15.162:0/2620041466 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff5b0044cd0 con 0x7ff5b81404c0 2020-11-28T21:46:26.546 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.543+0000 7ff5bea2c700 1 -- 172.21.15.162:0/2620041466 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff5b805d250 con 0x7ff5b81404c0 2020-11-28T21:46:26.547 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.543+0000 7ff5a6ffd700 1 -- 172.21.15.162:0/2620041466 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v18) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff5b00150a0 con 0x7ff5b81404c0 2020-11-28T21:46:26.547 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:46:26.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.547+0000 7ff5bea2c700 1 -- 172.21.15.162:0/2620041466 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff598055330 msgr2=0x7ff5980577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:26.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.547+0000 7ff5bea2c700 1 --2- 172.21.15.162:0/2620041466 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff598055330 0x7ff5980577f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7ff5b400d000 tx=0x7ff5b4007b10).stop 2020-11-28T21:46:26.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.547+0000 7ff5bea2c700 1 -- 172.21.15.162:0/2620041466 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5b81404c0 msgr2=0x7ff5b813e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:26.550 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.547+0000 7ff5bea2c700 1 --2- 172.21.15.162:0/2620041466 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5b81404c0 0x7ff5b813e4f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7ff5b8064970 tx=0x7ff5b0016000).stop 2020-11-28T21:46:26.550 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.547+0000 7ff5bea2c700 1 -- 172.21.15.162:0/2620041466 shutdown_connections 2020-11-28T21:46:26.550 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.547+0000 7ff5bea2c700 1 --2- 172.21.15.162:0/2620041466 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff598055330 0x7ff5980577f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:26.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.547+0000 7ff5bea2c700 1 --2- 172.21.15.162:0/2620041466 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff5b8060b60 0x7ff5b8146370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:26.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.547+0000 7ff5bea2c700 1 --2- 172.21.15.162:0/2620041466 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5b81404c0 0x7ff5b813e4f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:26.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.547+0000 7ff5bea2c700 1 --2- 172.21.15.162:0/2620041466 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff5b80b21f0 0x7ff5b813e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:26.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.547+0000 7ff5bea2c700 1 -- 172.21.15.162:0/2620041466 >> 172.21.15.162:0/2620041466 conn(0x7ff5b80b57d0 msgr2=0x7ff5b813f450 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:26.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.547+0000 7ff5bea2c700 1 -- 172.21.15.162:0/2620041466 shutdown_connections 2020-11-28T21:46:26.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.547+0000 7ff5bea2c700 1 -- 172.21.15.162:0/2620041466 wait complete. 2020-11-28T21:46:26.561 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2020-11-28T21:46:26.673 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.671+0000 7f870c388700 1 Processor -- start 2020-11-28T21:46:26.674 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.671+0000 7f870c388700 1 -- start start 2020-11-28T21:46:26.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.671+0000 7f870c388700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f87040608f0 0x7f870413c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:26.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.671+0000 7f870c388700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f87040b6130 0x7f870413ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:26.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.671+0000 7f870c388700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f87040b54f0 0x7f8704143020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:26.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.671+0000 7f870c388700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87040ba0b0 con 0x7f87040b54f0 2020-11-28T21:46:26.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.671+0000 7f870c388700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87040b9e10 con 0x7f87040608f0 2020-11-28T21:46:26.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.671+0000 7f870c388700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8704142b80 con 0x7f87040b6130 2020-11-28T21:46:26.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.671+0000 7f870a124700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f87040608f0 0x7f870413c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:26.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.671+0000 7f870a124700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f87040608f0 0x7f870413c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37098/0 (socket says 172.21.15.162:37098) 2020-11-28T21:46:26.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.671+0000 7f870a124700 1 -- 172.21.15.162:0/3011910824 learned_addr learned my addr 172.21.15.162:0/3011910824 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:26.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.671+0000 7f8709923700 1 --2- 172.21.15.162:0/3011910824 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f87040b6130 0x7f870413ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:26.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.671+0000 7f870a925700 1 --2- 172.21.15.162:0/3011910824 >> v2:172.21.15.118:3300/0 conn(0x7f87040b54f0 0x7f8704143020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:26.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.671+0000 7f870a124700 1 -- 172.21.15.162:0/3011910824 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f87040b6130 msgr2=0x7f870413ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:26.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.671+0000 7f870a124700 1 --2- 172.21.15.162:0/3011910824 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f87040b6130 0x7f870413ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:26.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.671+0000 7f870a124700 1 -- 172.21.15.162:0/3011910824 >> v2:172.21.15.118:3300/0 conn(0x7f87040b54f0 msgr2=0x7f8704143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:26.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.671+0000 7f870a124700 1 --2- 172.21.15.162:0/3011910824 >> v2:172.21.15.118:3300/0 conn(0x7f87040b54f0 0x7f8704143020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:26.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.671+0000 7f870a124700 1 -- 172.21.15.162:0/3011910824 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87040b9b30 con 0x7f87040608f0 2020-11-28T21:46:26.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.671+0000 7f870a124700 1 --2- 172.21.15.162:0/3011910824 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87040608f0 0x7f870413c910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f86fc0032b0 tx=0x7f86fc009270).ready entity=mon.1 client_cookie=3ff43667f46fb6f6 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:26.681 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.671+0000 7f8709122700 1 -- 172.21.15.162:0/3011910824 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f86fc009e60 con 0x7f87040608f0 2020-11-28T21:46:26.681 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.671+0000 7f8709122700 1 -- 172.21.15.162:0/3011910824 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86fc00bb50 con 0x7f87040608f0 2020-11-28T21:46:26.682 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.671+0000 7f8709122700 1 -- 172.21.15.162:0/3011910824 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f86fc00bcc0 con 0x7f87040608f0 2020-11-28T21:46:26.682 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.671+0000 7f870c388700 1 -- 172.21.15.162:0/3011910824 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87040608f0 msgr2=0x7f870413c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:26.682 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.671+0000 7f870c388700 1 --2- 172.21.15.162:0/3011910824 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87040608f0 0x7f870413c910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f86fc0032b0 tx=0x7f86fc009270).stop 2020-11-28T21:46:26.682 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.675+0000 7f870c388700 1 -- 172.21.15.162:0/3011910824 shutdown_connections 2020-11-28T21:46:26.682 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.675+0000 7f870c388700 1 --2- 172.21.15.162:0/3011910824 >> v2:172.21.15.118:3300/0 conn(0x7f87040b54f0 0x7f8704143020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:26.683 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.675+0000 7f870c388700 1 --2- 172.21.15.162:0/3011910824 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87040608f0 0x7f870413c910 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:26.683 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.675+0000 7f870c388700 1 --2- 172.21.15.162:0/3011910824 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f87040b6130 0x7f870413ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:26.683 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.675+0000 7f870c388700 1 -- 172.21.15.162:0/3011910824 >> 172.21.15.162:0/3011910824 conn(0x7f8704066c70 msgr2=0x7f8704067050 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:26.683 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.675+0000 7f870c388700 1 -- 172.21.15.162:0/3011910824 shutdown_connections 2020-11-28T21:46:26.683 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.675+0000 7f870c388700 1 -- 172.21.15.162:0/3011910824 wait complete. 2020-11-28T21:46:26.684 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.675+0000 7f870c388700 1 Processor -- start 2020-11-28T21:46:26.684 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.675+0000 7f870c388700 1 -- start start 2020-11-28T21:46:26.684 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.675+0000 7f870c388700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f87040608f0 0x7f8704144890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:26.684 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.675+0000 7f870c388700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87040b54f0 0x7f8704155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:26.684 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.675+0000 7f870c388700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f87040b6130 0x7f8704155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:26.685 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.675+0000 7f870c388700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8704154fc0 con 0x7f87040b6130 2020-11-28T21:46:26.685 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.675+0000 7f870c388700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87040ad7b0 con 0x7f87040b54f0 2020-11-28T21:46:26.685 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.675+0000 7f870c388700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87040b3330 con 0x7f87040608f0 2020-11-28T21:46:26.685 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.675+0000 7f8709923700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87040b54f0 0x7f8704155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:26.686 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.675+0000 7f8709923700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87040b54f0 0x7f8704155650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37104/0 (socket says 172.21.15.162:37104) 2020-11-28T21:46:26.686 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.675+0000 7f8709923700 1 -- 172.21.15.162:0/27387950 learned_addr learned my addr 172.21.15.162:0/27387950 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:26.686 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.675+0000 7f870a925700 1 --2- 172.21.15.162:0/27387950 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f87040b6130 0x7f8704155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:26.686 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.675+0000 7f870a124700 1 --2- 172.21.15.162:0/27387950 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f87040608f0 0x7f8704144890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:26.686 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.675+0000 7f8709923700 1 -- 172.21.15.162:0/27387950 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f87040608f0 msgr2=0x7f8704144890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:26.687 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.675+0000 7f8709923700 1 --2- 172.21.15.162:0/27387950 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f87040608f0 0x7f8704144890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:26.687 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.675+0000 7f8709923700 1 -- 172.21.15.162:0/27387950 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f87040b6130 msgr2=0x7f8704155b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:26.687 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.675+0000 7f8709923700 1 --2- 172.21.15.162:0/27387950 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f87040b6130 0x7f8704155b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:26.687 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.675+0000 7f8709923700 1 -- 172.21.15.162:0/27387950 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87040b2300 con 0x7f87040b54f0 2020-11-28T21:46:26.687 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.675+0000 7f8709923700 1 --2- 172.21.15.162:0/27387950 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87040b54f0 0x7f8704155650 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f87040b91e0 tx=0x7f86f4006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:26.688 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.675+0000 7f87037fe700 1 -- 172.21.15.162:0/27387950 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f86f400ca10 con 0x7f87040b54f0 2020-11-28T21:46:26.688 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.675+0000 7f87037fe700 1 -- 172.21.15.162:0/27387950 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86f400cb70 con 0x7f87040b54f0 2020-11-28T21:46:26.688 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.675+0000 7f870c388700 1 -- 172.21.15.162:0/27387950 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87040b25a0 con 0x7f87040b54f0 2020-11-28T21:46:26.688 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.675+0000 7f87037fe700 1 -- 172.21.15.162:0/27387950 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f86f40123f0 con 0x7f87040b54f0 2020-11-28T21:46:26.688 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.675+0000 7f870c388700 1 -- 172.21.15.162:0/27387950 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87041551b0 con 0x7f87040b54f0 2020-11-28T21:46:26.689 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.679+0000 7f87037fe700 1 -- 172.21.15.162:0/27387950 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f86f4012550 con 0x7f87040b54f0 2020-11-28T21:46:26.689 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.679+0000 7f870c388700 1 -- 172.21.15.162:0/27387950 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8704057050 con 0x7f87040b54f0 2020-11-28T21:46:26.690 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.679+0000 7f87037fe700 1 --2- 172.21.15.162:0/27387950 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f86e80553b0 0x7f86e8057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:26.690 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.679+0000 7f87037fe700 1 -- 172.21.15.162:0/27387950 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f86f406e080 con 0x7f87040b54f0 2020-11-28T21:46:26.690 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.679+0000 7f870a124700 1 --2- 172.21.15.162:0/27387950 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f86e80553b0 0x7f86e8057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:26.690 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.679+0000 7f870a124700 1 --2- 172.21.15.162:0/27387950 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f86e80553b0 0x7f86e8057870 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f86fc018000 tx=0x7f86fc016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:26.690 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.687+0000 7f87037fe700 1 -- 172.21.15.162:0/27387950 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f86f4044650 con 0x7f87040b54f0 2020-11-28T21:46:26.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.827+0000 7f870c388700 1 -- 172.21.15.162:0/27387950 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f870405d530 con 0x7f87040b54f0 2020-11-28T21:46:26.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.827+0000 7f87037fe700 1 -- 172.21.15.162:0/27387950 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) v1 ==== 84+0+143 (secure 0 0 0) 0x7f86f4047c70 con 0x7f87040b54f0 2020-11-28T21:46:26.830 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:46:26.831 INFO:teuthology.orchestra.run.smithi162.stdout:[ 2020-11-28T21:46:26.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:26.831 INFO:teuthology.orchestra.run.smithi162.stdout: "poolnum": 1, 2020-11-28T21:46:26.831 INFO:teuthology.orchestra.run.smithi162.stdout: "poolname": "device_health_metrics" 2020-11-28T21:46:26.832 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:26.832 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:26.832 INFO:teuthology.orchestra.run.smithi162.stdout: "poolnum": 2, 2020-11-28T21:46:26.832 INFO:teuthology.orchestra.run.smithi162.stdout: "poolname": "rbd" 2020-11-28T21:46:26.833 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:26.833 INFO:teuthology.orchestra.run.smithi162.stdout:] 2020-11-28T21:46:26.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.827+0000 7f870c388700 1 -- 172.21.15.162:0/27387950 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f86e80553b0 msgr2=0x7f86e8057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:26.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.827+0000 7f870c388700 1 --2- 172.21.15.162:0/27387950 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f86e80553b0 0x7f86e8057870 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f86fc018000 tx=0x7f86fc016000).stop 2020-11-28T21:46:26.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.827+0000 7f870c388700 1 -- 172.21.15.162:0/27387950 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87040b54f0 msgr2=0x7f8704155650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:26.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.827+0000 7f870c388700 1 --2- 172.21.15.162:0/27387950 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87040b54f0 0x7f8704155650 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f87040b91e0 tx=0x7f86f4006dc0).stop 2020-11-28T21:46:26.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.827+0000 7f870c388700 1 -- 172.21.15.162:0/27387950 shutdown_connections 2020-11-28T21:46:26.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.827+0000 7f870c388700 1 --2- 172.21.15.162:0/27387950 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f86e80553b0 0x7f86e8057870 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:26.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.827+0000 7f870c388700 1 --2- 172.21.15.162:0/27387950 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f87040608f0 0x7f8704144890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:26.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.827+0000 7f870c388700 1 --2- 172.21.15.162:0/27387950 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f87040b6130 0x7f8704155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:26.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.827+0000 7f870c388700 1 --2- 172.21.15.162:0/27387950 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87040b54f0 0x7f8704155650 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:26.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.827+0000 7f870c388700 1 -- 172.21.15.162:0/27387950 >> 172.21.15.162:0/27387950 conn(0x7f8704066c70 msgr2=0x7f870413bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:26.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.827+0000 7f870c388700 1 -- 172.21.15.162:0/27387950 shutdown_connections 2020-11-28T21:46:26.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.827+0000 7f870c388700 1 -- 172.21.15.162:0/27387950 wait complete. 2020-11-28T21:46:26.843 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-11-28T21:46:26.844 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2020-11-28T21:46:26.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.951+0000 7f13be2bc700 1 Processor -- start 2020-11-28T21:46:26.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.951+0000 7f13be2bc700 1 -- start start 2020-11-28T21:46:26.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.955+0000 7f13be2bc700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f13b80b9ce0 0x7f13b81433c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:26.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.955+0000 7f13be2bc700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f13b80605c0 0x7f13b8067610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:26.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.955+0000 7f13be2bc700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f13b80b3c30 0x7f13b813fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:26.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.955+0000 7f13be2bc700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13b8143950 con 0x7f13b80b3c30 2020-11-28T21:46:26.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.955+0000 7f13be2bc700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13b80ba0c0 con 0x7f13b80605c0 2020-11-28T21:46:26.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.955+0000 7f13be2bc700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13b8066c30 con 0x7f13b80b9ce0 2020-11-28T21:46:26.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.955+0000 7f13b77fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f13b80605c0 0x7f13b8067610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:26.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.955+0000 7f13b77fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f13b80605c0 0x7f13b8067610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37114/0 (socket says 172.21.15.162:37114) 2020-11-28T21:46:26.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.955+0000 7f13b77fe700 1 -- 172.21.15.162:0/422706275 learned_addr learned my addr 172.21.15.162:0/422706275 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:26.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.955+0000 7f13bc859700 1 --2- 172.21.15.162:0/422706275 >> v2:172.21.15.118:3300/0 conn(0x7f13b80b3c30 0x7f13b813fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:26.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.955+0000 7f13b7fff700 1 --2- 172.21.15.162:0/422706275 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f13b80b9ce0 0x7f13b81433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:26.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.955+0000 7f13b77fe700 1 -- 172.21.15.162:0/422706275 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f13b80b9ce0 msgr2=0x7f13b81433c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:26.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.955+0000 7f13b77fe700 1 --2- 172.21.15.162:0/422706275 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f13b80b9ce0 0x7f13b81433c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:26.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.955+0000 7f13b77fe700 1 -- 172.21.15.162:0/422706275 >> v2:172.21.15.118:3300/0 conn(0x7f13b80b3c30 msgr2=0x7f13b813fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:26.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.955+0000 7f13b77fe700 1 --2- 172.21.15.162:0/422706275 >> v2:172.21.15.118:3300/0 conn(0x7f13b80b3c30 0x7f13b813fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:26.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.955+0000 7f13b77fe700 1 -- 172.21.15.162:0/422706275 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b8066ed0 con 0x7f13b80605c0 2020-11-28T21:46:26.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.955+0000 7f13b77fe700 1 --2- 172.21.15.162:0/422706275 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13b80605c0 0x7f13b8067610 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f13a8002f30 tx=0x7f13a8006e80).ready entity=mon.1 client_cookie=f3e42dae43594c40 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:26.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.955+0000 7f13b6ffd700 1 -- 172.21.15.162:0/422706275 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13a80097f0 con 0x7f13b80605c0 2020-11-28T21:46:26.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.955+0000 7f13b6ffd700 1 -- 172.21.15.162:0/422706275 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a8009950 con 0x7f13b80605c0 2020-11-28T21:46:26.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.955+0000 7f13b6ffd700 1 -- 172.21.15.162:0/422706275 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13a8009cb0 con 0x7f13b80605c0 2020-11-28T21:46:26.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.955+0000 7f13be2bc700 1 -- 172.21.15.162:0/422706275 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13b80605c0 msgr2=0x7f13b8067610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:26.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.955+0000 7f13be2bc700 1 --2- 172.21.15.162:0/422706275 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13b80605c0 0x7f13b8067610 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f13a8002f30 tx=0x7f13a8006e80).stop 2020-11-28T21:46:26.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.955+0000 7f13be2bc700 1 -- 172.21.15.162:0/422706275 shutdown_connections 2020-11-28T21:46:26.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.955+0000 7f13be2bc700 1 --2- 172.21.15.162:0/422706275 >> v2:172.21.15.118:3300/0 conn(0x7f13b80b3c30 0x7f13b813fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:26.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.955+0000 7f13be2bc700 1 --2- 172.21.15.162:0/422706275 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f13b80b9ce0 0x7f13b81433c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:26.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.955+0000 7f13be2bc700 1 --2- 172.21.15.162:0/422706275 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13b80605c0 0x7f13b8067610 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:26.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.955+0000 7f13be2bc700 1 -- 172.21.15.162:0/422706275 >> 172.21.15.162:0/422706275 conn(0x7f13b805c430 msgr2=0x7f13b80b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:26.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.955+0000 7f13be2bc700 1 -- 172.21.15.162:0/422706275 shutdown_connections 2020-11-28T21:46:26.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.955+0000 7f13be2bc700 1 -- 172.21.15.162:0/422706275 wait complete. 2020-11-28T21:46:26.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.959+0000 7f13be2bc700 1 Processor -- start 2020-11-28T21:46:26.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.959+0000 7f13be2bc700 1 -- start start 2020-11-28T21:46:26.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.959+0000 7f13be2bc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f13b80605c0 0x7f13b8145cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:26.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.959+0000 7f13be2bc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13b80b3c30 0x7f13b8152140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:26.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.959+0000 7f13be2bc700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f13b80b9ce0 0x7f13b809e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:26.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.959+0000 7f13be2bc700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b8151890 con 0x7f13b80605c0 2020-11-28T21:46:26.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.959+0000 7f13be2bc700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b8090d50 con 0x7f13b80b3c30 2020-11-28T21:46:26.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.959+0000 7f13be2bc700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13b8145a20 con 0x7f13b80b9ce0 2020-11-28T21:46:26.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.959+0000 7f13b77fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13b80b3c30 0x7f13b8152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:26.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.959+0000 7f13b77fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13b80b3c30 0x7f13b8152140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37122/0 (socket says 172.21.15.162:37122) 2020-11-28T21:46:26.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.959+0000 7f13b77fe700 1 -- 172.21.15.162:0/3825006581 learned_addr learned my addr 172.21.15.162:0/3825006581 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:26.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.959+0000 7f13b7fff700 1 --2- 172.21.15.162:0/3825006581 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f13b80605c0 0x7f13b8145cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:26.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.959+0000 7f13bc859700 1 --2- 172.21.15.162:0/3825006581 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f13b80b9ce0 0x7f13b809e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:26.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.959+0000 7f13b77fe700 1 -- 172.21.15.162:0/3825006581 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f13b80b9ce0 msgr2=0x7f13b809e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:26.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.959+0000 7f13b77fe700 1 --2- 172.21.15.162:0/3825006581 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f13b80b9ce0 0x7f13b809e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:26.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.959+0000 7f13b77fe700 1 -- 172.21.15.162:0/3825006581 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f13b80605c0 msgr2=0x7f13b8145cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:26.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.959+0000 7f13b77fe700 1 --2- 172.21.15.162:0/3825006581 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f13b80605c0 0x7f13b8145cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:26.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.959+0000 7f13b77fe700 1 -- 172.21.15.162:0/3825006581 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b81468a0 con 0x7f13b80b3c30 2020-11-28T21:46:26.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.959+0000 7f13b7fff700 1 --2- 172.21.15.162:0/3825006581 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f13b80605c0 0x7f13b8145cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:46:26.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.959+0000 7f13bc859700 1 --2- 172.21.15.162:0/3825006581 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f13b80b9ce0 0x7f13b809e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:46:26.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.959+0000 7f13b77fe700 1 --2- 172.21.15.162:0/3825006581 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13b80b3c30 0x7f13b8152140 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f13a8002f30 tx=0x7f13a8015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:26.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.959+0000 7f13b57fa700 1 -- 172.21.15.162:0/3825006581 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13a8011d30 con 0x7f13b80b3c30 2020-11-28T21:46:26.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.959+0000 7f13b57fa700 1 -- 172.21.15.162:0/3825006581 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a8011e90 con 0x7f13b80b3c30 2020-11-28T21:46:26.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.959+0000 7f13b57fa700 1 -- 172.21.15.162:0/3825006581 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13a801e650 con 0x7f13b80b3c30 2020-11-28T21:46:26.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.959+0000 7f13be2bc700 1 -- 172.21.15.162:0/3825006581 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13b8146b40 con 0x7f13b80b3c30 2020-11-28T21:46:26.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.963+0000 7f13be2bc700 1 -- 172.21.15.162:0/3825006581 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13b81455d0 con 0x7f13b80b3c30 2020-11-28T21:46:26.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.963+0000 7f13be2bc700 1 -- 172.21.15.162:0/3825006581 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13b8146cd0 con 0x7f13b80b3c30 2020-11-28T21:46:26.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.963+0000 7f13b57fa700 1 -- 172.21.15.162:0/3825006581 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f13a801a040 con 0x7f13b80b3c30 2020-11-28T21:46:26.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.963+0000 7f13b57fa700 1 --2- 172.21.15.162:0/3825006581 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f139c055330 0x7f139c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:26.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.963+0000 7f13b57fa700 1 -- 172.21.15.162:0/3825006581 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f13a806ecc0 con 0x7f13b80b3c30 2020-11-28T21:46:26.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.963+0000 7f13b7fff700 1 --2- 172.21.15.162:0/3825006581 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f139c055330 0x7f139c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:26.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.963+0000 7f13b7fff700 1 --2- 172.21.15.162:0/3825006581 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f139c055330 0x7f139c0577f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f13a0008000 tx=0x7f13a0008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:26.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:26.967+0000 7f13b57fa700 1 -- 172.21.15.162:0/3825006581 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f13a80452f0 con 0x7f13b80b3c30 2020-11-28T21:46:27.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.103+0000 7f13be2bc700 1 -- 172.21.15.162:0/3825006581 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f13b8057050 con 0x7f13b80b3c30 2020-11-28T21:46:27.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.771+0000 7f13b57fa700 1 -- 172.21.15.162:0/3825006581 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v20) v1 ==== 133+0+0 (secure 0 0 0) 0x7f13a8023070 con 0x7f13b80b3c30 2020-11-28T21:46:27.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.775+0000 7f13be2bc700 1 -- 172.21.15.162:0/3825006581 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f139c055330 msgr2=0x7f139c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:27.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.775+0000 7f13be2bc700 1 --2- 172.21.15.162:0/3825006581 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f139c055330 0x7f139c0577f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f13a0008000 tx=0x7f13a0008040).stop 2020-11-28T21:46:27.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.775+0000 7f13be2bc700 1 -- 172.21.15.162:0/3825006581 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13b80b3c30 msgr2=0x7f13b8152140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:27.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.775+0000 7f13be2bc700 1 --2- 172.21.15.162:0/3825006581 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13b80b3c30 0x7f13b8152140 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f13a8002f30 tx=0x7f13a8015000).stop 2020-11-28T21:46:27.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.775+0000 7f13be2bc700 1 -- 172.21.15.162:0/3825006581 shutdown_connections 2020-11-28T21:46:27.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.775+0000 7f13be2bc700 1 --2- 172.21.15.162:0/3825006581 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f139c055330 0x7f139c0577f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:27.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.775+0000 7f13be2bc700 1 --2- 172.21.15.162:0/3825006581 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f13b80b9ce0 0x7f13b809e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:27.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.775+0000 7f13be2bc700 1 --2- 172.21.15.162:0/3825006581 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f13b80605c0 0x7f13b8145cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:27.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.775+0000 7f13be2bc700 1 --2- 172.21.15.162:0/3825006581 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13b80b3c30 0x7f13b8152140 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:27.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.775+0000 7f13be2bc700 1 -- 172.21.15.162:0/3825006581 >> 172.21.15.162:0/3825006581 conn(0x7f13b805c430 msgr2=0x7f13b809d870 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:27.780 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.775+0000 7f13be2bc700 1 -- 172.21.15.162:0/3825006581 shutdown_connections 2020-11-28T21:46:27.780 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.775+0000 7f13be2bc700 1 -- 172.21.15.162:0/3825006581 wait complete. 2020-11-28T21:46:27.780 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_metadata' created 2020-11-28T21:46:27.792 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2020-11-28T21:46:27.918 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.915+0000 7fa02b56f700 1 Processor -- start 2020-11-28T21:46:27.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.915+0000 7fa02b56f700 1 -- start start 2020-11-28T21:46:27.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02b56f700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fa0240b1900 0x7fa0240b1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:27.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02b56f700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0241404c0 con 0x7fa024140040 2020-11-28T21:46:27.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02b56f700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0240b1080 con 0x7fa0240ad240 2020-11-28T21:46:27.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02b56f700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0240b1490 con 0x7fa0240b1900 2020-11-28T21:46:27.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02a56d700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fa0240ad240 0x7fa024142aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:43972/0 (socket says 172.21.15.162:43972) 2020-11-28T21:46:27.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02a56d700 1 -- 172.21.15.162:0/344113694 learned_addr learned my addr 172.21.15.162:0/344113694 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:27.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02ad6e700 1 --2- 172.21.15.162:0/344113694 >> v2:172.21.15.118:3300/0 conn(0x7fa0240b1900 0x7fa0240b1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:27.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02956b700 1 -- 172.21.15.162:0/344113694 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3349498809 0 0) 0x7fa0240b1080 con 0x7fa0240ad240 2020-11-28T21:46:27.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02956b700 1 -- 172.21.15.162:0/344113694 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa00c003230 con 0x7fa0240ad240 2020-11-28T21:46:27.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02956b700 1 -- 172.21.15.162:0/344113694 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 130837010 0 0) 0x7fa0140010d0 con 0x7fa024140040 2020-11-28T21:46:27.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02956b700 1 -- 172.21.15.162:0/344113694 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0240b1080 con 0x7fa024140040 2020-11-28T21:46:27.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02956b700 1 -- 172.21.15.162:0/344113694 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 974912674 0 0) 0x7fa00c003230 con 0x7fa0240ad240 2020-11-28T21:46:27.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02956b700 1 -- 172.21.15.162:0/344113694 >> v2:172.21.15.118:3300/0 conn(0x7fa0240b1900 msgr2=0x7fa0240b1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:27.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02956b700 1 --2- 172.21.15.162:0/344113694 >> v2:172.21.15.118:3300/0 conn(0x7fa0240b1900 0x7fa0240b1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:27.931 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02956b700 1 -- 172.21.15.162:0/344113694 >> v1:172.21.15.118:6789/0 conn(0x7fa024140040 legacy=0x7fa0240ad620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:27.931 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02956b700 1 -- 172.21.15.162:0/344113694 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa02413f660 con 0x7fa0240ad240 2020-11-28T21:46:27.931 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02956b700 1 -- 172.21.15.162:0/344113694 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7fa0200016d0 con 0x7fa0240ad240 2020-11-28T21:46:27.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02956b700 1 -- 172.21.15.162:0/344113694 >> v1:172.21.15.162:6789/0 conn(0x7fa0240ad240 legacy=0x7fa024142aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:27.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02956b700 1 --2- 172.21.15.162:0/344113694 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa00c0038b0 0x7fa00c003cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:27.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02956b700 1 --2- 172.21.15.162:0/344113694 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa00c0044d0 0x7fa00c004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:27.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02956b700 1 --2- 172.21.15.162:0/344113694 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa00c004e40 0x7fa00c00b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:27.933 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02956b700 1 -- 172.21.15.162:0/344113694 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0240b1490 con 0x7fa00c004e40 2020-11-28T21:46:27.933 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02956b700 1 -- 172.21.15.162:0/344113694 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa00c0053a0 con 0x7fa00c0038b0 2020-11-28T21:46:27.933 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02956b700 1 -- 172.21.15.162:0/344113694 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa00c00bc90 con 0x7fa00c0044d0 2020-11-28T21:46:27.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02ad6e700 1 --2- 172.21.15.162:0/344113694 >> v2:172.21.15.118:3300/0 conn(0x7fa0240b1900 0x7fa0240b1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:46:27.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02a56d700 1 --2- 172.21.15.162:0/344113694 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa00c0038b0 0x7fa00c003cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:27.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa029d6c700 1 --2- 172.21.15.162:0/344113694 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa00c0044d0 0x7fa00c004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:27.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02a56d700 1 -- 172.21.15.162:0/344113694 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa00c0044d0 msgr2=0x7fa00c004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:27.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02a56d700 1 --2- 172.21.15.162:0/344113694 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa00c0044d0 0x7fa00c004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:27.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02ad6e700 1 --2- 172.21.15.162:0/344113694 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa00c004e40 0x7fa00c00b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:27.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02a56d700 1 -- 172.21.15.162:0/344113694 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa00c004e40 msgr2=0x7fa00c00b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:27.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02a56d700 1 --2- 172.21.15.162:0/344113694 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa00c004e40 0x7fa00c00b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:27.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02a56d700 1 -- 172.21.15.162:0/344113694 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa00c003230 con 0x7fa00c0038b0 2020-11-28T21:46:27.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02ad6e700 1 --2- 172.21.15.162:0/344113694 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa00c004e40 0x7fa00c00b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:46:27.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02ad6e700 1 -- 172.21.15.162:0/344113694 reap_dead start 2020-11-28T21:46:27.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02a56d700 1 --2- 172.21.15.162:0/344113694 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa00c0038b0 0x7fa00c003cb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fa0200014a0 tx=0x7fa0240b1210).ready entity=mon.1 client_cookie=cbb5b0fc5c19bfef server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:27.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02956b700 1 -- 172.21.15.162:0/344113694 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0200070d0 con 0x7fa00c0038b0 2020-11-28T21:46:27.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02956b700 1 -- 172.21.15.162:0/344113694 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa020007230 con 0x7fa00c0038b0 2020-11-28T21:46:27.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02956b700 1 -- 172.21.15.162:0/344113694 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0200055e0 con 0x7fa00c0038b0 2020-11-28T21:46:27.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02b56f700 1 -- 172.21.15.162:0/344113694 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa00c0038b0 msgr2=0x7fa00c003cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:27.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02b56f700 1 --2- 172.21.15.162:0/344113694 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa00c0038b0 0x7fa00c003cb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fa0200014a0 tx=0x7fa0240b1210).stop 2020-11-28T21:46:27.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02b56f700 1 -- 172.21.15.162:0/344113694 shutdown_connections 2020-11-28T21:46:27.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02b56f700 1 --2- 172.21.15.162:0/344113694 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa00c0038b0 0x7fa00c003cb0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:27.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02b56f700 1 -- 172.21.15.162:0/344113694 >> 172.21.15.162:0/344113694 conn(0x7fa0240b2830 msgr2=0x7fa0240b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:27.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02b56f700 1 -- 172.21.15.162:0/344113694 shutdown_connections 2020-11-28T21:46:27.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.919+0000 7fa02b56f700 1 -- 172.21.15.162:0/344113694 wait complete. 2020-11-28T21:46:27.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.923+0000 7fa02b56f700 1 Processor -- start 2020-11-28T21:46:27.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.923+0000 7fa02b56f700 1 -- start start 2020-11-28T21:46:27.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.923+0000 7fa02b56f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa00c0038b0 0x7fa024141e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:27.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.923+0000 7fa02b56f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa024152db0 0x7fa024066e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:27.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.923+0000 7fa02b56f700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa02405b970 0x7fa02405be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:27.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.923+0000 7fa02b56f700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0241431b0 con 0x7fa02405b970 2020-11-28T21:46:27.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.923+0000 7fa02b56f700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0240b5460 con 0x7fa024152db0 2020-11-28T21:46:27.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.923+0000 7fa02b56f700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0240b4c30 con 0x7fa00c0038b0 2020-11-28T21:46:27.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.923+0000 7fa029d6c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa024152db0 0x7fa024066e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:27.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.927+0000 7fa029d6c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa024152db0 0x7fa024066e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37142/0 (socket says 172.21.15.162:37142) 2020-11-28T21:46:27.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.927+0000 7fa029d6c700 1 -- 172.21.15.162:0/2347116179 learned_addr learned my addr 172.21.15.162:0/2347116179 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:27.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.927+0000 7fa02a56d700 1 --2- 172.21.15.162:0/2347116179 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa00c0038b0 0x7fa024141e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:27.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.927+0000 7fa02ad6e700 1 --2- 172.21.15.162:0/2347116179 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa02405b970 0x7fa02405be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:27.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.927+0000 7fa029d6c700 1 -- 172.21.15.162:0/2347116179 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa00c0038b0 msgr2=0x7fa024141e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:27.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.927+0000 7fa029d6c700 1 --2- 172.21.15.162:0/2347116179 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa00c0038b0 0x7fa024141e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:27.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.927+0000 7fa029d6c700 1 -- 172.21.15.162:0/2347116179 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa02405b970 msgr2=0x7fa02405be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:27.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.927+0000 7fa029d6c700 1 --2- 172.21.15.162:0/2347116179 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa02405b970 0x7fa02405be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:27.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.927+0000 7fa029d6c700 1 -- 172.21.15.162:0/2347116179 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa024153230 con 0x7fa024152db0 2020-11-28T21:46:27.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.927+0000 7fa029d6c700 1 --2- 172.21.15.162:0/2347116179 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa024152db0 0x7fa024066e10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fa0240b50e0 tx=0x7fa014009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:27.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.927+0000 7fa0137fe700 1 -- 172.21.15.162:0/2347116179 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa014001cc0 con 0x7fa024152db0 2020-11-28T21:46:27.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.927+0000 7fa0137fe700 1 -- 172.21.15.162:0/2347116179 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa014001e20 con 0x7fa024152db0 2020-11-28T21:46:27.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.927+0000 7fa02b56f700 1 -- 172.21.15.162:0/2347116179 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0240669b0 con 0x7fa024152db0 2020-11-28T21:46:27.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.927+0000 7fa0137fe700 1 -- 172.21.15.162:0/2347116179 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa014003300 con 0x7fa024152db0 2020-11-28T21:46:27.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.927+0000 7fa02b56f700 1 -- 172.21.15.162:0/2347116179 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0240b4920 con 0x7fa024152db0 2020-11-28T21:46:27.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.927+0000 7fa0137fe700 1 -- 172.21.15.162:0/2347116179 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa014003460 con 0x7fa024152db0 2020-11-28T21:46:27.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.927+0000 7fa02b56f700 1 -- 172.21.15.162:0/2347116179 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa024057050 con 0x7fa024152db0 2020-11-28T21:46:27.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.927+0000 7fa0137fe700 1 --2- 172.21.15.162:0/2347116179 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa0040553b0 0x7fa004057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:27.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.927+0000 7fa0137fe700 1 -- 172.21.15.162:0/2347116179 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5607+0+0 (secure 0 0 0) 0x7fa01406d870 con 0x7fa024152db0 2020-11-28T21:46:27.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.927+0000 7fa02a56d700 1 --2- 172.21.15.162:0/2347116179 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa0040553b0 0x7fa004057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:27.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.931+0000 7fa02a56d700 1 --2- 172.21.15.162:0/2347116179 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa0040553b0 0x7fa004057870 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fa02001a000 tx=0x7fa020015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:27.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:27.935+0000 7fa0137fe700 1 -- 172.21.15.162:0/2347116179 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa014042df0 con 0x7fa024152db0 2020-11-28T21:46:28.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.079+0000 7fa02b56f700 1 -- 172.21.15.162:0/2347116179 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fa0241404c0 con 0x7fa024152db0 2020-11-28T21:46:28.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.783+0000 7fa0137fe700 1 -- 172.21.15.162:0/2347116179 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v21) v1 ==== 125+0+0 (secure 0 0 0) 0x7fa0241404c0 con 0x7fa024152db0 2020-11-28T21:46:28.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.791+0000 7fa02b56f700 1 -- 172.21.15.162:0/2347116179 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa0040553b0 msgr2=0x7fa004057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:28.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.791+0000 7fa02b56f700 1 --2- 172.21.15.162:0/2347116179 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa0040553b0 0x7fa004057870 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fa02001a000 tx=0x7fa020015000).stop 2020-11-28T21:46:28.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.791+0000 7fa02b56f700 1 -- 172.21.15.162:0/2347116179 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa024152db0 msgr2=0x7fa024066e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:28.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.791+0000 7fa02b56f700 1 --2- 172.21.15.162:0/2347116179 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa024152db0 0x7fa024066e10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fa0240b50e0 tx=0x7fa014009040).stop 2020-11-28T21:46:28.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.791+0000 7fa02b56f700 1 -- 172.21.15.162:0/2347116179 shutdown_connections 2020-11-28T21:46:28.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.791+0000 7fa02b56f700 1 --2- 172.21.15.162:0/2347116179 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa0040553b0 0x7fa004057870 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:28.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.791+0000 7fa02b56f700 1 --2- 172.21.15.162:0/2347116179 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa00c0038b0 0x7fa024141e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:28.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.791+0000 7fa02b56f700 1 --2- 172.21.15.162:0/2347116179 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa02405b970 0x7fa02405be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:28.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.791+0000 7fa02b56f700 1 --2- 172.21.15.162:0/2347116179 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa024152db0 0x7fa024066e10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:28.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.791+0000 7fa02b56f700 1 -- 172.21.15.162:0/2347116179 >> 172.21.15.162:0/2347116179 conn(0x7fa0240b2830 msgr2=0x7fa02413df40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:28.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.791+0000 7fa02b56f700 1 -- 172.21.15.162:0/2347116179 shutdown_connections 2020-11-28T21:46:28.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.791+0000 7fa02b56f700 1 -- 172.21.15.162:0/2347116179 wait complete. 2020-11-28T21:46:28.799 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_data' created 2020-11-28T21:46:28.810 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2020-11-28T21:46:28.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.919+0000 7f7da3caa700 1 Processor -- start 2020-11-28T21:46:28.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.919+0000 7f7da3caa700 1 -- start start 2020-11-28T21:46:28.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.919+0000 7f7da3caa700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f7d9c1433c0 0x7f7d9c0605c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:28.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.919+0000 7f7da3caa700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7d9c067610 0x7f7d9c13fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:28.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.919+0000 7f7da3caa700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d9c143950 con 0x7f7d9c0b9ce0 2020-11-28T21:46:28.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.919+0000 7f7da3caa700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d9c0b4340 con 0x7f7d9c1433c0 2020-11-28T21:46:28.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.919+0000 7f7da3caa700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d9c067ac0 con 0x7f7d9c067610 2020-11-28T21:46:28.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.923+0000 7f7da2247700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7d9c067610 0x7f7d9c13fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:28.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.923+0000 7f7da1a46700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f7d9c0b9ce0 0x7f7d9c0b3c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:43992/0 (socket says 172.21.15.162:43992) 2020-11-28T21:46:28.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.923+0000 7f7da1a46700 1 -- 172.21.15.162:0/3122754396 learned_addr learned my addr 172.21.15.162:0/3122754396 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:28.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.923+0000 7f7da1245700 1 --2- 172.21.15.162:0/3122754396 >> v2:172.21.15.118:3300/0 conn(0x7f7d9c1433c0 0x7f7d9c0605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:28.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.923+0000 7f7da2247700 1 -- 172.21.15.162:0/3122754396 >> v2:172.21.15.118:3300/0 conn(0x7f7d9c1433c0 msgr2=0x7f7d9c0605c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:28.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.923+0000 7f7da2247700 1 --2- 172.21.15.162:0/3122754396 >> v2:172.21.15.118:3300/0 conn(0x7f7d9c1433c0 0x7f7d9c0605c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:28.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.923+0000 7f7da2247700 1 -- 172.21.15.162:0/3122754396 >> v1:172.21.15.162:6789/0 conn(0x7f7d9c0b9ce0 legacy=0x7f7d9c0b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:28.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.923+0000 7f7da2247700 1 -- 172.21.15.162:0/3122754396 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d9c0ba0c0 con 0x7f7d9c067610 2020-11-28T21:46:28.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.923+0000 7f7da2247700 1 --2- 172.21.15.162:0/3122754396 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d9c067610 0x7f7d9c13fc80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f7d940032b0 tx=0x7f7d940091a0).ready entity=mon.1 client_cookie=ee3ad607788b7b2b server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:28.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.923+0000 7f7da0a44700 1 -- 172.21.15.162:0/3122754396 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d9400ba50 con 0x7f7d9c067610 2020-11-28T21:46:28.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.923+0000 7f7da0a44700 1 -- 172.21.15.162:0/3122754396 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d9400bbb0 con 0x7f7d9c067610 2020-11-28T21:46:28.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.923+0000 7f7da0a44700 1 -- 172.21.15.162:0/3122754396 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d9400bd20 con 0x7f7d9c067610 2020-11-28T21:46:28.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.923+0000 7f7da3caa700 1 -- 172.21.15.162:0/3122754396 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d9c067610 msgr2=0x7f7d9c13fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:28.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.923+0000 7f7da3caa700 1 --2- 172.21.15.162:0/3122754396 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d9c067610 0x7f7d9c13fc80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f7d940032b0 tx=0x7f7d940091a0).stop 2020-11-28T21:46:28.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.923+0000 7f7da3caa700 1 -- 172.21.15.162:0/3122754396 shutdown_connections 2020-11-28T21:46:28.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.923+0000 7f7da3caa700 1 --2- 172.21.15.162:0/3122754396 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d9c067610 0x7f7d9c13fc80 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:28.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.923+0000 7f7da3caa700 1 --2- 172.21.15.162:0/3122754396 >> v2:172.21.15.118:3300/0 conn(0x7f7d9c1433c0 0x7f7d9c0605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:28.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.923+0000 7f7da3caa700 1 -- 172.21.15.162:0/3122754396 >> 172.21.15.162:0/3122754396 conn(0x7f7d9c05c430 msgr2=0x7f7d9c0b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:28.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.923+0000 7f7da3caa700 1 -- 172.21.15.162:0/3122754396 shutdown_connections 2020-11-28T21:46:28.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.923+0000 7f7da3caa700 1 -- 172.21.15.162:0/3122754396 wait complete. 2020-11-28T21:46:28.931 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.923+0000 7f7da3caa700 1 Processor -- start 2020-11-28T21:46:28.931 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.923+0000 7f7da3caa700 1 -- start start 2020-11-28T21:46:28.933 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.927+0000 7f7da3caa700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d9c067610 0x7f7d9c0b5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:28.933 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.927+0000 7f7da3caa700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7d9c0b9ce0 0x7f7d9c1550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:28.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.927+0000 7f7da3caa700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7d9c1433c0 0x7f7d9c0af090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:28.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.927+0000 7f7da3caa700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9c0b0940 con 0x7f7d9c0b9ce0 2020-11-28T21:46:28.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.927+0000 7f7da3caa700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9c1559d0 con 0x7f7d9c067610 2020-11-28T21:46:28.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.927+0000 7f7da3caa700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9c0ad980 con 0x7f7d9c1433c0 2020-11-28T21:46:28.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.927+0000 7f7da1a46700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d9c067610 0x7f7d9c0b5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:28.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.927+0000 7f7da1a46700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d9c067610 0x7f7d9c0b5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37154/0 (socket says 172.21.15.162:37154) 2020-11-28T21:46:28.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.927+0000 7f7da1a46700 1 -- 172.21.15.162:0/883666330 learned_addr learned my addr 172.21.15.162:0/883666330 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:28.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.927+0000 7f7da2247700 1 --2- 172.21.15.162:0/883666330 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7d9c1433c0 0x7f7d9c0af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:28.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.927+0000 7f7da1245700 1 --2- 172.21.15.162:0/883666330 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7d9c0b9ce0 0x7f7d9c1550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:28.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.927+0000 7f7da1a46700 1 -- 172.21.15.162:0/883666330 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7d9c1433c0 msgr2=0x7f7d9c0af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:28.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.927+0000 7f7da1a46700 1 --2- 172.21.15.162:0/883666330 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7d9c1433c0 0x7f7d9c0af090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:28.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.927+0000 7f7da1a46700 1 -- 172.21.15.162:0/883666330 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7d9c0b9ce0 msgr2=0x7f7d9c1550b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:28.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.927+0000 7f7da1a46700 1 --2- 172.21.15.162:0/883666330 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7d9c0b9ce0 0x7f7d9c1550b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:28.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.927+0000 7f7da1a46700 1 -- 172.21.15.162:0/883666330 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d9c0adc20 con 0x7f7d9c067610 2020-11-28T21:46:28.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.927+0000 7f7da1245700 1 --2- 172.21.15.162:0/883666330 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7d9c0b9ce0 0x7f7d9c1550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:46:28.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.927+0000 7f7da1a46700 1 --2- 172.21.15.162:0/883666330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d9c067610 0x7f7d9c0b5410 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f7d9c0afd00 tx=0x7f7d8c007a40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:28.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.927+0000 7f7d8affd700 1 -- 172.21.15.162:0/883666330 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d8c0072e0 con 0x7f7d9c067610 2020-11-28T21:46:28.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.927+0000 7f7d8affd700 1 -- 172.21.15.162:0/883666330 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d8c0054b0 con 0x7f7d9c067610 2020-11-28T21:46:28.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.927+0000 7f7da3caa700 1 -- 172.21.15.162:0/883666330 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d9c1555a0 con 0x7f7d9c067610 2020-11-28T21:46:28.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.927+0000 7f7da3caa700 1 -- 172.21.15.162:0/883666330 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d9c0ae1c0 con 0x7f7d9c067610 2020-11-28T21:46:28.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.927+0000 7f7d8affd700 1 -- 172.21.15.162:0/883666330 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d8c005620 con 0x7f7d9c067610 2020-11-28T21:46:28.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.931+0000 7f7da3caa700 1 -- 172.21.15.162:0/883666330 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d9c0ad3f0 con 0x7f7d9c067610 2020-11-28T21:46:28.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.931+0000 7f7d8affd700 1 -- 172.21.15.162:0/883666330 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7d8c010070 con 0x7f7d9c067610 2020-11-28T21:46:28.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.931+0000 7f7d8affd700 1 --2- 172.21.15.162:0/883666330 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7d7c0554e0 0x7f7d7c0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:28.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.931+0000 7f7da1245700 1 --2- 172.21.15.162:0/883666330 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7d7c0554e0 0x7f7d7c0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:28.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.931+0000 7f7d8affd700 1 -- 172.21.15.162:0/883666330 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5957+0+0 (secure 0 0 0) 0x7f7d8c06d3d0 con 0x7f7d9c067610 2020-11-28T21:46:28.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.931+0000 7f7da1245700 1 --2- 172.21.15.162:0/883666330 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7d7c0554e0 0x7f7d7c0579a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f7d98008000 tx=0x7f7d98008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:28.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:28.935+0000 7f7d8affd700 1 -- 172.21.15.162:0/883666330 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7d9c0ad3f0 con 0x7f7d9c067610 2020-11-28T21:46:29.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.115+0000 7f7da3caa700 1 -- 172.21.15.162:0/883666330 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f7d9c0ad3f0 con 0x7f7d9c067610 2020-11-28T21:46:29.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.131+0000 7f7d8affd700 1 -- 172.21.15.162:0/883666330 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 187+0+0 (secure 0 0 0) 0x7f7d9c0ad3f0 con 0x7f7d9c067610 2020-11-28T21:46:29.138 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.135+0000 7f7da3caa700 1 -- 172.21.15.162:0/883666330 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7d7c0554e0 msgr2=0x7f7d7c0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:29.138 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.135+0000 7f7da3caa700 1 --2- 172.21.15.162:0/883666330 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7d7c0554e0 0x7f7d7c0579a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f7d98008000 tx=0x7f7d98008040).stop 2020-11-28T21:46:29.139 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.135+0000 7f7da3caa700 1 -- 172.21.15.162:0/883666330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d9c067610 msgr2=0x7f7d9c0b5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:29.139 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.135+0000 7f7da3caa700 1 --2- 172.21.15.162:0/883666330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d9c067610 0x7f7d9c0b5410 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f7d9c0afd00 tx=0x7f7d8c007a40).stop 2020-11-28T21:46:29.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.135+0000 7f7da3caa700 1 -- 172.21.15.162:0/883666330 shutdown_connections 2020-11-28T21:46:29.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.135+0000 7f7da3caa700 1 --2- 172.21.15.162:0/883666330 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7d7c0554e0 0x7f7d7c0579a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:29.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.135+0000 7f7da3caa700 1 --2- 172.21.15.162:0/883666330 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7d9c1433c0 0x7f7d9c0af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:29.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.135+0000 7f7da3caa700 1 --2- 172.21.15.162:0/883666330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d9c067610 0x7f7d9c0b5410 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:29.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.135+0000 7f7da3caa700 1 --2- 172.21.15.162:0/883666330 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7d9c0b9ce0 0x7f7d9c1550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:29.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.135+0000 7f7da3caa700 1 -- 172.21.15.162:0/883666330 >> 172.21.15.162:0/883666330 conn(0x7f7d9c05c430 msgr2=0x7f7d9c13e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:29.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.135+0000 7f7da3caa700 1 -- 172.21.15.162:0/883666330 shutdown_connections 2020-11-28T21:46:29.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.135+0000 7f7da3caa700 1 -- 172.21.15.162:0/883666330 wait complete. 2020-11-28T21:46:29.141 INFO:teuthology.orchestra.run.smithi162.stderr:new fs with metadata pool 3 and data pool 4 2020-11-28T21:46:29.155 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:46:29.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.251+0000 7f9efafc2700 1 Processor -- start 2020-11-28T21:46:29.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.251+0000 7f9efafc2700 1 -- start start 2020-11-28T21:46:29.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.255+0000 7f9efafc2700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9ef4142a00 0x7f9ef40b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:29.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.255+0000 7f9efafc2700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ef40b4110 con 0x7f9ef40603d0 2020-11-28T21:46:29.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.255+0000 7f9efafc2700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ef40b43f0 con 0x7f9ef40b4c70 2020-11-28T21:46:29.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.255+0000 7f9efafc2700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ef4060f60 con 0x7f9ef4142a00 2020-11-28T21:46:29.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.255+0000 7f9ef97bf700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9ef4142a00 0x7f9ef40b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:29.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.255+0000 7f9efa7c1700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f9ef40b4c70 0x7f9ef4143020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:44020/0 (socket says 172.21.15.162:44020) 2020-11-28T21:46:29.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.255+0000 7f9efa7c1700 1 -- 172.21.15.162:0/3573518814 learned_addr learned my addr 172.21.15.162:0/3573518814 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:29.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.255+0000 7f9ef97bf700 1 -- 172.21.15.162:0/3573518814 >> v1:172.21.15.162:6789/0 conn(0x7f9ef40b4c70 legacy=0x7f9ef4143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:29.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.255+0000 7f9ef8fbe700 1 -- 172.21.15.162:0/3573518814 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3714244292 0 0) 0x7f9ef40b43f0 con 0x7f9ef40b4c70 2020-11-28T21:46:29.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.255+0000 7f9ef97bf700 1 -- 172.21.15.162:0/3573518814 >> v1:172.21.15.118:6789/0 conn(0x7f9ef40603d0 legacy=0x7f9ef409cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:29.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.255+0000 7f9ef97bf700 1 -- 172.21.15.162:0/3573518814 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef40b4690 con 0x7f9ef4142a00 2020-11-28T21:46:29.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.255+0000 7f9ef97bf700 1 --2- 172.21.15.162:0/3573518814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ef4142a00 0x7f9ef40b3a70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f9ee0002f30 tx=0x7f9ee0006dc0).ready entity=mon.1 client_cookie=8a278ac4ab73038e server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:29.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.255+0000 7f9ef8fbe700 1 -- 172.21.15.162:0/3573518814 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ee00097f0 con 0x7f9ef4142a00 2020-11-28T21:46:29.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.255+0000 7f9ef8fbe700 1 -- 172.21.15.162:0/3573518814 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee0009950 con 0x7f9ef4142a00 2020-11-28T21:46:29.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.255+0000 7f9ef8fbe700 1 -- 172.21.15.162:0/3573518814 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ee0009cb0 con 0x7f9ef4142a00 2020-11-28T21:46:29.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.255+0000 7f9efafc2700 1 -- 172.21.15.162:0/3573518814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ef4142a00 msgr2=0x7f9ef40b3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:29.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.255+0000 7f9efafc2700 1 --2- 172.21.15.162:0/3573518814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ef4142a00 0x7f9ef40b3a70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f9ee0002f30 tx=0x7f9ee0006dc0).stop 2020-11-28T21:46:29.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.255+0000 7f9efafc2700 1 -- 172.21.15.162:0/3573518814 shutdown_connections 2020-11-28T21:46:29.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.255+0000 7f9efafc2700 1 --2- 172.21.15.162:0/3573518814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ef4142a00 0x7f9ef40b3a70 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:29.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.255+0000 7f9efafc2700 1 -- 172.21.15.162:0/3573518814 >> 172.21.15.162:0/3573518814 conn(0x7f9ef4066a40 msgr2=0x7f9ef4066e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:29.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.255+0000 7f9efafc2700 1 -- 172.21.15.162:0/3573518814 shutdown_connections 2020-11-28T21:46:29.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.255+0000 7f9efafc2700 1 -- 172.21.15.162:0/3573518814 wait complete. 2020-11-28T21:46:29.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.259+0000 7f9efafc2700 1 Processor -- start 2020-11-28T21:46:29.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.259+0000 7f9efafc2700 1 -- start start 2020-11-28T21:46:29.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.259+0000 7f9efafc2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ef40603d0 0x7f9ef4151830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:29.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.259+0000 7f9efafc2700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9ef40b4c70 0x7f9ef40b8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:29.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.259+0000 7f9efafc2700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9ef40b1100 0x7f9ef40b1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:29.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.259+0000 7f9efafc2700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef40b15b0 con 0x7f9ef40b4c70 2020-11-28T21:46:29.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.259+0000 7f9efafc2700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef40b0480 con 0x7f9ef40603d0 2020-11-28T21:46:29.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.259+0000 7f9efafc2700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef40b0720 con 0x7f9ef40b1100 2020-11-28T21:46:29.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.259+0000 7f9ef9fc0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ef40603d0 0x7f9ef4151830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:29.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.259+0000 7f9ef9fc0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ef40603d0 0x7f9ef4151830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37178/0 (socket says 172.21.15.162:37178) 2020-11-28T21:46:29.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.259+0000 7f9ef9fc0700 1 -- 172.21.15.162:0/1235533755 learned_addr learned my addr 172.21.15.162:0/1235533755 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:29.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.259+0000 7f9ef97bf700 1 --2- 172.21.15.162:0/1235533755 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9ef40b4c70 0x7f9ef40b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:29.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.259+0000 7f9efa7c1700 1 --2- 172.21.15.162:0/1235533755 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9ef40b1100 0x7f9ef40b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:29.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.259+0000 7f9ef9fc0700 1 -- 172.21.15.162:0/1235533755 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9ef40b1100 msgr2=0x7f9ef40b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:29.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.259+0000 7f9ef9fc0700 1 --2- 172.21.15.162:0/1235533755 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9ef40b1100 0x7f9ef40b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:29.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.259+0000 7f9ef9fc0700 1 -- 172.21.15.162:0/1235533755 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9ef40b4c70 msgr2=0x7f9ef40b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:29.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.259+0000 7f9ef9fc0700 1 --2- 172.21.15.162:0/1235533755 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9ef40b4c70 0x7f9ef40b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:29.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.259+0000 7f9ef9fc0700 1 -- 172.21.15.162:0/1235533755 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef40b09c0 con 0x7f9ef40603d0 2020-11-28T21:46:29.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.259+0000 7f9ef9fc0700 1 --2- 172.21.15.162:0/1235533755 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ef40603d0 0x7f9ef4151830 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f9ef40b9c10 tx=0x7f9ee8007f60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:29.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.259+0000 7f9ef2ffd700 1 -- 172.21.15.162:0/1235533755 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ee800ce10 con 0x7f9ef40603d0 2020-11-28T21:46:29.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.259+0000 7f9ef2ffd700 1 -- 172.21.15.162:0/1235533755 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee8006a70 con 0x7f9ef40603d0 2020-11-28T21:46:29.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.259+0000 7f9efafc2700 1 -- 172.21.15.162:0/1235533755 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ef40b0c60 con 0x7f9ef40603d0 2020-11-28T21:46:29.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.259+0000 7f9ef2ffd700 1 -- 172.21.15.162:0/1235533755 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ee8006dd0 con 0x7f9ef40603d0 2020-11-28T21:46:29.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.259+0000 7f9efafc2700 1 -- 172.21.15.162:0/1235533755 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ef40b1d10 con 0x7f9ef40603d0 2020-11-28T21:46:29.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.263+0000 7f9efafc2700 1 -- 172.21.15.162:0/1235533755 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ef40b0df0 con 0x7f9ef40603d0 2020-11-28T21:46:29.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.263+0000 7f9ef2ffd700 1 -- 172.21.15.162:0/1235533755 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9ee8010030 con 0x7f9ef40603d0 2020-11-28T21:46:29.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.263+0000 7f9ef2ffd700 1 --2- 172.21.15.162:0/1235533755 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9edc0554e0 0x7f9edc0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:29.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.263+0000 7f9ef2ffd700 1 -- 172.21.15.162:0/1235533755 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9ee806df40 con 0x7f9ef40603d0 2020-11-28T21:46:29.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.263+0000 7f9ef97bf700 1 --2- 172.21.15.162:0/1235533755 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9edc0554e0 0x7f9edc0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:29.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.263+0000 7f9ef97bf700 1 --2- 172.21.15.162:0/1235533755 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9edc0554e0 0x7f9edc0579a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f9ee001b000 tx=0x7f9ee0015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:29.273 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.267+0000 7f9ef2ffd700 1 -- 172.21.15.162:0/1235533755 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9ee8044240 con 0x7f9ef40603d0 2020-11-28T21:46:29.409 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.407+0000 7f9efafc2700 1 -- 172.21.15.162:0/1235533755 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ef4057050 con 0x7f9ef40603d0 2020-11-28T21:46:29.411 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.407+0000 7f9ef2ffd700 1 -- 172.21.15.162:0/1235533755 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16147 (secure 0 0 0) 0x7f9ee8047860 con 0x7f9ef40603d0 2020-11-28T21:46:29.411 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:46:29.412 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":22,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:46:29.120112+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:46:27.155497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-28T21:46:28.137474+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:46:29.413 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.411+0000 7f9efafc2700 1 -- 172.21.15.162:0/1235533755 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9edc0554e0 msgr2=0x7f9edc0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:29.413 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.411+0000 7f9efafc2700 1 --2- 172.21.15.162:0/1235533755 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9edc0554e0 0x7f9edc0579a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f9ee001b000 tx=0x7f9ee0015000).stop 2020-11-28T21:46:29.414 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.411+0000 7f9efafc2700 1 -- 172.21.15.162:0/1235533755 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ef40603d0 msgr2=0x7f9ef4151830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:29.414 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.411+0000 7f9efafc2700 1 --2- 172.21.15.162:0/1235533755 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ef40603d0 0x7f9ef4151830 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f9ef40b9c10 tx=0x7f9ee8007f60).stop 2020-11-28T21:46:29.414 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.411+0000 7f9efafc2700 1 -- 172.21.15.162:0/1235533755 shutdown_connections 2020-11-28T21:46:29.414 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.411+0000 7f9efafc2700 1 --2- 172.21.15.162:0/1235533755 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9edc0554e0 0x7f9edc0579a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:29.415 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.411+0000 7f9efafc2700 1 --2- 172.21.15.162:0/1235533755 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9ef40b1100 0x7f9ef40b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:29.415 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.411+0000 7f9efafc2700 1 --2- 172.21.15.162:0/1235533755 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ef40603d0 0x7f9ef4151830 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:29.415 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.411+0000 7f9efafc2700 1 --2- 172.21.15.162:0/1235533755 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9ef40b4c70 0x7f9ef40b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:29.415 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.411+0000 7f9efafc2700 1 -- 172.21.15.162:0/1235533755 >> 172.21.15.162:0/1235533755 conn(0x7f9ef4066a40 msgr2=0x7f9ef413e460 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:29.416 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.411+0000 7f9efafc2700 1 -- 172.21.15.162:0/1235533755 shutdown_connections 2020-11-28T21:46:29.416 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.411+0000 7f9efafc2700 1 -- 172.21.15.162:0/1235533755 wait complete. 2020-11-28T21:46:29.430 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:46:29.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.535+0000 7fc1bffff700 1 Processor -- start 2020-11-28T21:46:29.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.535+0000 7fc1bffff700 1 -- start start 2020-11-28T21:46:29.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.539+0000 7fc1bffff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc1c00a9bc0 0x7fc1c00a9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:29.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.539+0000 7fc1bffff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc1c00aa490 0x7fc1c00afb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:29.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.539+0000 7fc1bffff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fc1c00b1fd0 0x7fc1c00b0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:29.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.539+0000 7fc1bffff700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1c0061030 con 0x7fc1c00b1fd0 2020-11-28T21:46:29.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.539+0000 7fc1bffff700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1c00b0f30 con 0x7fc1c00a9bc0 2020-11-28T21:46:29.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.539+0000 7fc1bffff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c00af980 con 0x7fc1c00aa490 2020-11-28T21:46:29.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.539+0000 7fc1beffd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc1c00a9bc0 0x7fc1c00a9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:29.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.539+0000 7fc1be7fc700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc1c00aa490 0x7fc1c00afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:29.543 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.539+0000 7fc1beffd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc1c00a9bc0 0x7fc1c00a9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37188/0 (socket says 172.21.15.162:37188) 2020-11-28T21:46:29.543 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.539+0000 7fc1bf7fe700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fc1c00b1fd0 0x7fc1c00b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:29.543 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.539+0000 7fc1beffd700 1 -- 172.21.15.162:0/2182059894 learned_addr learned my addr 172.21.15.162:0/2182059894 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:29.547 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.539+0000 7fc1beffd700 1 -- 172.21.15.162:0/2182059894 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc1c00aa490 msgr2=0x7fc1c00afb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:29.547 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.539+0000 7fc1beffd700 1 --2- 172.21.15.162:0/2182059894 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc1c00aa490 0x7fc1c00afb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:29.547 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.539+0000 7fc1beffd700 1 -- 172.21.15.162:0/2182059894 >> v2:172.21.15.118:3300/0 conn(0x7fc1c00b1fd0 msgr2=0x7fc1c00b0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:29.548 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.539+0000 7fc1beffd700 1 --2- 172.21.15.162:0/2182059894 >> v2:172.21.15.118:3300/0 conn(0x7fc1c00b1fd0 0x7fc1c00b0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:29.548 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.539+0000 7fc1beffd700 1 -- 172.21.15.162:0/2182059894 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c00af6e0 con 0x7fc1c00a9bc0 2020-11-28T21:46:29.548 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.539+0000 7fc1beffd700 1 --2- 172.21.15.162:0/2182059894 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1c00a9bc0 0x7fc1c00a9fa0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fc1b0002f30 tx=0x7fc1b0006e80).ready entity=mon.1 client_cookie=83a5ad7a55abe5d8 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:29.548 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.539+0000 7fc1bdffb700 1 -- 172.21.15.162:0/2182059894 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1b0009640 con 0x7fc1c00a9bc0 2020-11-28T21:46:29.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.539+0000 7fc1bdffb700 1 -- 172.21.15.162:0/2182059894 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1b00097a0 con 0x7fc1c00a9bc0 2020-11-28T21:46:29.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.539+0000 7fc1bdffb700 1 -- 172.21.15.162:0/2182059894 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1b0009b00 con 0x7fc1c00a9bc0 2020-11-28T21:46:29.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.539+0000 7fc1bffff700 1 -- 172.21.15.162:0/2182059894 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1c00a9bc0 msgr2=0x7fc1c00a9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:29.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.539+0000 7fc1bffff700 1 --2- 172.21.15.162:0/2182059894 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1c00a9bc0 0x7fc1c00a9fa0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fc1b0002f30 tx=0x7fc1b0006e80).stop 2020-11-28T21:46:29.550 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.539+0000 7fc1bffff700 1 -- 172.21.15.162:0/2182059894 shutdown_connections 2020-11-28T21:46:29.550 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.539+0000 7fc1bffff700 1 --2- 172.21.15.162:0/2182059894 >> v2:172.21.15.118:3300/0 conn(0x7fc1c00b1fd0 0x7fc1c00b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:29.550 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.539+0000 7fc1bffff700 1 --2- 172.21.15.162:0/2182059894 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1c00a9bc0 0x7fc1c00a9fa0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:29.550 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.539+0000 7fc1bffff700 1 --2- 172.21.15.162:0/2182059894 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc1c00aa490 0x7fc1c00afb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:29.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.539+0000 7fc1bffff700 1 -- 172.21.15.162:0/2182059894 >> 172.21.15.162:0/2182059894 conn(0x7fc1c005b970 msgr2=0x7fc1c005bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:29.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.539+0000 7fc1bffff700 1 -- 172.21.15.162:0/2182059894 shutdown_connections 2020-11-28T21:46:29.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.539+0000 7fc1bffff700 1 -- 172.21.15.162:0/2182059894 wait complete. 2020-11-28T21:46:29.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.543+0000 7fc1bffff700 1 Processor -- start 2020-11-28T21:46:29.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.543+0000 7fc1bffff700 1 -- start start 2020-11-28T21:46:29.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.543+0000 7fc1bffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc1c00a9bc0 0x7fc1c00aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:29.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.543+0000 7fc1bffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1c00aa490 0x7fc1c0063340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:29.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.543+0000 7fc1bffff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc1c00b1fd0 0x7fc1c00ad650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:29.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.543+0000 7fc1bffff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c00b3a40 con 0x7fc1c00a9bc0 2020-11-28T21:46:29.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.543+0000 7fc1bffff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c00b3c30 con 0x7fc1c00aa490 2020-11-28T21:46:29.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.543+0000 7fc1bffff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c00b3ed0 con 0x7fc1c00b1fd0 2020-11-28T21:46:29.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.543+0000 7fc1be7fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1c00aa490 0x7fc1c0063340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:29.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.543+0000 7fc1be7fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1c00aa490 0x7fc1c0063340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37194/0 (socket says 172.21.15.162:37194) 2020-11-28T21:46:29.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.543+0000 7fc1be7fc700 1 -- 172.21.15.162:0/3647880719 learned_addr learned my addr 172.21.15.162:0/3647880719 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:29.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.543+0000 7fc1beffd700 1 --2- 172.21.15.162:0/3647880719 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc1c00a9bc0 0x7fc1c00aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:29.555 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.543+0000 7fc1bf7fe700 1 --2- 172.21.15.162:0/3647880719 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc1c00b1fd0 0x7fc1c00ad650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:29.555 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.543+0000 7fc1be7fc700 1 -- 172.21.15.162:0/3647880719 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc1c00b1fd0 msgr2=0x7fc1c00ad650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:29.555 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.543+0000 7fc1be7fc700 1 --2- 172.21.15.162:0/3647880719 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc1c00b1fd0 0x7fc1c00ad650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:29.555 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.543+0000 7fc1be7fc700 1 -- 172.21.15.162:0/3647880719 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc1c00a9bc0 msgr2=0x7fc1c00aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:29.555 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.543+0000 7fc1be7fc700 1 --2- 172.21.15.162:0/3647880719 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc1c00a9bc0 0x7fc1c00aedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:29.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.543+0000 7fc1be7fc700 1 -- 172.21.15.162:0/3647880719 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c00b4170 con 0x7fc1c00aa490 2020-11-28T21:46:29.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.543+0000 7fc1bf7fe700 1 --2- 172.21.15.162:0/3647880719 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc1c00b1fd0 0x7fc1c00ad650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:46:29.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.543+0000 7fc1be7fc700 1 --2- 172.21.15.162:0/3647880719 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1c00aa490 0x7fc1c0063340 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fc1ac00e000 tx=0x7fc1ac00f040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:29.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.543+0000 7fc1b7fff700 1 -- 172.21.15.162:0/3647880719 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1ac012040 con 0x7fc1c00aa490 2020-11-28T21:46:29.557 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.543+0000 7fc1b7fff700 1 -- 172.21.15.162:0/3647880719 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1ac010cc0 con 0x7fc1c00aa490 2020-11-28T21:46:29.557 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.543+0000 7fc1bffff700 1 -- 172.21.15.162:0/3647880719 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1c00ad1b0 con 0x7fc1c00aa490 2020-11-28T21:46:29.557 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.543+0000 7fc1b7fff700 1 -- 172.21.15.162:0/3647880719 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1ac017630 con 0x7fc1c00aa490 2020-11-28T21:46:29.557 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.543+0000 7fc1bffff700 1 -- 172.21.15.162:0/3647880719 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c0063830 con 0x7fc1c00aa490 2020-11-28T21:46:29.558 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.543+0000 7fc1bffff700 1 -- 172.21.15.162:0/3647880719 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1c00ad340 con 0x7fc1c00aa490 2020-11-28T21:46:29.559 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.547+0000 7fc1b7fff700 1 -- 172.21.15.162:0/3647880719 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc1ac017790 con 0x7fc1c00aa490 2020-11-28T21:46:29.559 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.547+0000 7fc1b7fff700 1 --2- 172.21.15.162:0/3647880719 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc19c0554e0 0x7fc19c0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:29.559 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.547+0000 7fc1b7fff700 1 -- 172.21.15.162:0/3647880719 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc1ac0705c0 con 0x7fc1c00aa490 2020-11-28T21:46:29.560 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.547+0000 7fc1beffd700 1 --2- 172.21.15.162:0/3647880719 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc19c0554e0 0x7fc19c0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:29.560 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.547+0000 7fc1beffd700 1 --2- 172.21.15.162:0/3647880719 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc19c0554e0 0x7fc19c0579a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fc1b0019000 tx=0x7fc1b0015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:29.560 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.551+0000 7fc1b7fff700 1 -- 172.21.15.162:0/3647880719 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc1ac046860 con 0x7fc1c00aa490 2020-11-28T21:46:29.693 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.691+0000 7fc1bffff700 1 -- 172.21.15.162:0/3647880719 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc1c0057050 con 0x7fc1c00aa490 2020-11-28T21:46:29.695 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.691+0000 7fc1b7fff700 1 -- 172.21.15.162:0/3647880719 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16147 (secure 0 0 0) 0x7fc1ac049e80 con 0x7fc1c00aa490 2020-11-28T21:46:29.695 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:46:29.696 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":22,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:46:29.120112+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:46:27.155497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-28T21:46:28.137474+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:46:29.697 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.695+0000 7fc1bffff700 1 -- 172.21.15.162:0/3647880719 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc19c0554e0 msgr2=0x7fc19c0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:29.697 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.695+0000 7fc1bffff700 1 --2- 172.21.15.162:0/3647880719 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc19c0554e0 0x7fc19c0579a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fc1b0019000 tx=0x7fc1b0015000).stop 2020-11-28T21:46:29.698 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.695+0000 7fc1bffff700 1 -- 172.21.15.162:0/3647880719 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1c00aa490 msgr2=0x7fc1c0063340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:29.698 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.695+0000 7fc1bffff700 1 --2- 172.21.15.162:0/3647880719 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1c00aa490 0x7fc1c0063340 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fc1ac00e000 tx=0x7fc1ac00f040).stop 2020-11-28T21:46:29.698 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.695+0000 7fc1bffff700 1 -- 172.21.15.162:0/3647880719 shutdown_connections 2020-11-28T21:46:29.698 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.695+0000 7fc1bffff700 1 --2- 172.21.15.162:0/3647880719 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc19c0554e0 0x7fc19c0579a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:29.699 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.695+0000 7fc1bffff700 1 --2- 172.21.15.162:0/3647880719 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc1c00b1fd0 0x7fc1c00ad650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:29.699 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.695+0000 7fc1bffff700 1 --2- 172.21.15.162:0/3647880719 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc1c00a9bc0 0x7fc1c00aedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:29.699 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.695+0000 7fc1bffff700 1 --2- 172.21.15.162:0/3647880719 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1c00aa490 0x7fc1c0063340 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:29.699 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.695+0000 7fc1bffff700 1 -- 172.21.15.162:0/3647880719 >> 172.21.15.162:0/3647880719 conn(0x7fc1c005b970 msgr2=0x7fc1c0140a20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:29.700 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.695+0000 7fc1bffff700 1 -- 172.21.15.162:0/3647880719 shutdown_connections 2020-11-28T21:46:29.700 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.695+0000 7fc1bffff700 1 -- 172.21.15.162:0/3647880719 wait complete. 2020-11-28T21:46:29.709 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2020-11-28T21:46:29.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.827+0000 7f1cba788700 1 Processor -- start 2020-11-28T21:46:29.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.827+0000 7f1cba788700 1 -- start start 2020-11-28T21:46:29.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.831+0000 7f1cba788700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1cb4140040 0x7f1cb40ad620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:29.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.831+0000 7f1cba788700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1cb40adb10 0x7f1cb40b1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:29.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.831+0000 7f1cba788700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cb41404c0 con 0x7f1cb40ad240 2020-11-28T21:46:29.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.831+0000 7f1cba788700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cb40b1490 con 0x7f1cb40adb10 2020-11-28T21:46:29.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.831+0000 7f1cba788700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb41431b0 con 0x7f1cb4140040 2020-11-28T21:46:29.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.831+0000 7f1cb9f87700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1cb40adb10 0x7f1cb40b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:29.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.831+0000 7f1cb9f87700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1cb40adb10 0x7f1cb40b1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37202/0 (socket says 172.21.15.162:37202) 2020-11-28T21:46:29.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.831+0000 7f1cb9f87700 1 -- 172.21.15.162:0/3495289163 learned_addr learned my addr 172.21.15.162:0/3495289163 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:29.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.831+0000 7f1cb8f85700 1 --2- 172.21.15.162:0/3495289163 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1cb4140040 0x7f1cb40ad620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:29.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.831+0000 7f1cb9f87700 1 -- 172.21.15.162:0/3495289163 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1cb4140040 msgr2=0x7f1cb40ad620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:29.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.831+0000 7f1cb9f87700 1 --2- 172.21.15.162:0/3495289163 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1cb4140040 0x7f1cb40ad620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:29.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.831+0000 7f1cb9f87700 1 -- 172.21.15.162:0/3495289163 >> v1:172.21.15.118:6789/0 conn(0x7f1cb40ad240 legacy=0x7f1cb4142aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:29.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.831+0000 7f1cb9f87700 1 -- 172.21.15.162:0/3495289163 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb40b1080 con 0x7f1cb40adb10 2020-11-28T21:46:29.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.831+0000 7f1cb9f87700 1 --2- 172.21.15.162:0/3495289163 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1cb40adb10 0x7f1cb40b1900 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f1cac002f30 tx=0x7f1cac006e80).ready entity=mon.1 client_cookie=f58127c28eb1d3af server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:29.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.831+0000 7f1cb3fff700 1 -- 172.21.15.162:0/3495289163 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1cac0097f0 con 0x7f1cb40adb10 2020-11-28T21:46:29.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.831+0000 7f1cb3fff700 1 -- 172.21.15.162:0/3495289163 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cac009950 con 0x7f1cb40adb10 2020-11-28T21:46:29.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.831+0000 7f1cb3fff700 1 -- 172.21.15.162:0/3495289163 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1cac009cb0 con 0x7f1cb40adb10 2020-11-28T21:46:29.838 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.831+0000 7f1cba788700 1 -- 172.21.15.162:0/3495289163 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1cb40adb10 msgr2=0x7f1cb40b1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:29.838 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.831+0000 7f1cba788700 1 --2- 172.21.15.162:0/3495289163 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1cb40adb10 0x7f1cb40b1900 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f1cac002f30 tx=0x7f1cac006e80).stop 2020-11-28T21:46:29.840 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.831+0000 7f1cba788700 1 -- 172.21.15.162:0/3495289163 shutdown_connections 2020-11-28T21:46:29.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.831+0000 7f1cba788700 1 --2- 172.21.15.162:0/3495289163 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1cb40adb10 0x7f1cb40b1900 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:29.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.831+0000 7f1cba788700 1 --2- 172.21.15.162:0/3495289163 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1cb4140040 0x7f1cb40ad620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:29.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.831+0000 7f1cba788700 1 -- 172.21.15.162:0/3495289163 >> 172.21.15.162:0/3495289163 conn(0x7f1cb40b2830 msgr2=0x7f1cb40b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:29.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.831+0000 7f1cba788700 1 -- 172.21.15.162:0/3495289163 shutdown_connections 2020-11-28T21:46:29.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.831+0000 7f1cba788700 1 -- 172.21.15.162:0/3495289163 wait complete. 2020-11-28T21:46:29.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.835+0000 7f1cba788700 1 Processor -- start 2020-11-28T21:46:29.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.835+0000 7f1cba788700 1 -- start start 2020-11-28T21:46:29.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.835+0000 7f1cba788700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1cb40ad240 0x7f1cb405fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:29.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.835+0000 7f1cba788700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1cb40adb10 0x7f1cb4063970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:29.843 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.835+0000 7f1cba788700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1cb4140040 0x7f1cb4065070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:29.843 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.835+0000 7f1cba788700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb4063e60 con 0x7f1cb40adb10 2020-11-28T21:46:29.843 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.835+0000 7f1cba788700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb40b0070 con 0x7f1cb4140040 2020-11-28T21:46:29.844 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.835+0000 7f1cba788700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb4064690 con 0x7f1cb40ad240 2020-11-28T21:46:29.844 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.835+0000 7f1cb9f87700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1cb4140040 0x7f1cb4065070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:29.844 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.835+0000 7f1cb9f87700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1cb4140040 0x7f1cb4065070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37210/0 (socket says 172.21.15.162:37210) 2020-11-28T21:46:29.844 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.835+0000 7f1cb9f87700 1 -- 172.21.15.162:0/179825683 learned_addr learned my addr 172.21.15.162:0/179825683 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:29.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.835+0000 7f1cb8f85700 1 --2- 172.21.15.162:0/179825683 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1cb40adb10 0x7f1cb4063970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:29.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.835+0000 7f1cb9786700 1 --2- 172.21.15.162:0/179825683 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1cb40ad240 0x7f1cb405fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:29.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.835+0000 7f1cb9f87700 1 -- 172.21.15.162:0/179825683 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1cb40ad240 msgr2=0x7f1cb405fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:29.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.835+0000 7f1cb9f87700 1 --2- 172.21.15.162:0/179825683 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1cb40ad240 0x7f1cb405fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:29.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.835+0000 7f1cb9f87700 1 -- 172.21.15.162:0/179825683 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1cb40adb10 msgr2=0x7f1cb4063970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:29.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.835+0000 7f1cb9f87700 1 --2- 172.21.15.162:0/179825683 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1cb40adb10 0x7f1cb4063970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:29.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.835+0000 7f1cb9f87700 1 -- 172.21.15.162:0/179825683 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb4064930 con 0x7f1cb4140040 2020-11-28T21:46:29.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.835+0000 7f1cb9f87700 1 --2- 172.21.15.162:0/179825683 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1cb4140040 0x7f1cb4065070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f1cb4067610 tx=0x7f1cac015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:29.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.835+0000 7f1cb27fc700 1 -- 172.21.15.162:0/179825683 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1cac011b40 con 0x7f1cb4140040 2020-11-28T21:46:29.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.835+0000 7f1cb27fc700 1 -- 172.21.15.162:0/179825683 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cac011ca0 con 0x7f1cb4140040 2020-11-28T21:46:29.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.835+0000 7f1cba788700 1 -- 172.21.15.162:0/179825683 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cb4064bd0 con 0x7f1cb4140040 2020-11-28T21:46:29.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.835+0000 7f1cb27fc700 1 -- 172.21.15.162:0/179825683 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1cac01e460 con 0x7f1cb4140040 2020-11-28T21:46:29.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.835+0000 7f1cba788700 1 -- 172.21.15.162:0/179825683 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cb4067860 con 0x7f1cb4140040 2020-11-28T21:46:29.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.839+0000 7f1cba788700 1 -- 172.21.15.162:0/179825683 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cb4064d60 con 0x7f1cb4140040 2020-11-28T21:46:29.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.839+0000 7f1cb27fc700 1 -- 172.21.15.162:0/179825683 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1cac025030 con 0x7f1cb4140040 2020-11-28T21:46:29.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.839+0000 7f1cb27fc700 1 --2- 172.21.15.162:0/179825683 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1c98055330 0x7f1c980577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:29.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.839+0000 7f1cb27fc700 1 -- 172.21.15.162:0/179825683 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1cac06f2c0 con 0x7f1cb4140040 2020-11-28T21:46:29.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.839+0000 7f1cb9786700 1 --2- 172.21.15.162:0/179825683 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1c98055330 0x7f1c980577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:29.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.839+0000 7f1cb9786700 1 --2- 172.21.15.162:0/179825683 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1c98055330 0x7f1c980577f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f1ca400d000 tx=0x7f1ca4000b80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:29.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:29.843+0000 7f1cb27fc700 1 -- 172.21.15.162:0/179825683 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1cac045430 con 0x7f1cb4140040 2020-11-28T21:46:30.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.027+0000 7f1cba788700 1 -- 172.21.15.162:0/179825683 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1cb4057050 con 0x7f1cb4140040 2020-11-28T21:46:30.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.127+0000 7f1cb27fc700 1 -- 172.21.15.162:0/179825683 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1cac0230b0 con 0x7f1cb4140040 2020-11-28T21:46:30.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.131+0000 7f1cba788700 1 -- 172.21.15.162:0/179825683 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1c98055330 msgr2=0x7f1c980577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:30.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.131+0000 7f1cba788700 1 --2- 172.21.15.162:0/179825683 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1c98055330 0x7f1c980577f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f1ca400d000 tx=0x7f1ca4000b80).stop 2020-11-28T21:46:30.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.131+0000 7f1cba788700 1 -- 172.21.15.162:0/179825683 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1cb4140040 msgr2=0x7f1cb4065070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:30.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.131+0000 7f1cba788700 1 --2- 172.21.15.162:0/179825683 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1cb4140040 0x7f1cb4065070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f1cb4067610 tx=0x7f1cac015000).stop 2020-11-28T21:46:30.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.131+0000 7f1cba788700 1 -- 172.21.15.162:0/179825683 shutdown_connections 2020-11-28T21:46:30.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.131+0000 7f1cba788700 1 --2- 172.21.15.162:0/179825683 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1c98055330 0x7f1c980577f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:30.138 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.131+0000 7f1cba788700 1 --2- 172.21.15.162:0/179825683 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1cb40ad240 0x7f1cb405fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:30.138 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.131+0000 7f1cba788700 1 --2- 172.21.15.162:0/179825683 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1cb4140040 0x7f1cb4065070 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:30.138 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.131+0000 7f1cba788700 1 --2- 172.21.15.162:0/179825683 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1cb40adb10 0x7f1cb4063970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:30.139 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.131+0000 7f1cba788700 1 -- 172.21.15.162:0/179825683 >> 172.21.15.162:0/179825683 conn(0x7f1cb40b2830 msgr2=0x7f1cb413e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:30.139 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.131+0000 7f1cba788700 1 -- 172.21.15.162:0/179825683 shutdown_connections 2020-11-28T21:46:30.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.131+0000 7f1cba788700 1 -- 172.21.15.162:0/179825683 wait complete. 2020-11-28T21:46:30.154 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:46:30.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.263+0000 7f073cfe1700 1 Processor -- start 2020-11-28T21:46:30.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.263+0000 7f073cfe1700 1 -- start start 2020-11-28T21:46:30.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.267+0000 7f073cfe1700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f07380611e0 0x7f07380615c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:30.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.267+0000 7f073cfe1700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f07380b10f0 0x7f07380b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:30.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.267+0000 7f073cfe1700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07380b1790 con 0x7f07380b4920 2020-11-28T21:46:30.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.267+0000 7f073cfe1700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07380b1580 con 0x7f07380b10f0 2020-11-28T21:46:30.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.267+0000 7f073cfe1700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07380b1a30 con 0x7f07380611e0 2020-11-28T21:46:30.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.267+0000 7f07377fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f07380611e0 0x7f07380615c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:30.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.267+0000 7f07377fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f07380611e0 0x7f07380615c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37214/0 (socket says 172.21.15.162:37214) 2020-11-28T21:46:30.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.267+0000 7f07377fe700 1 -- 172.21.15.162:0/3440940713 learned_addr learned my addr 172.21.15.162:0/3440940713 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:30.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.267+0000 7f072ffff700 1 --2- 172.21.15.162:0/3440940713 >> v2:172.21.15.118:3300/0 conn(0x7f07380b10f0 0x7f07380b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:30.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.267+0000 7f07377fe700 1 -- 172.21.15.162:0/3440940713 >> v2:172.21.15.118:3300/0 conn(0x7f07380b10f0 msgr2=0x7f07380b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:30.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.267+0000 7f07377fe700 1 --2- 172.21.15.162:0/3440940713 >> v2:172.21.15.118:3300/0 conn(0x7f07380b10f0 0x7f07380b4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:30.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.267+0000 7f07377fe700 1 -- 172.21.15.162:0/3440940713 >> v1:172.21.15.118:6789/0 conn(0x7f07380b4920 legacy=0x7f073813ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:30.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.267+0000 7f07377fe700 1 -- 172.21.15.162:0/3440940713 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07380b3f90 con 0x7f07380611e0 2020-11-28T21:46:30.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.267+0000 7f07377fe700 1 --2- 172.21.15.162:0/3440940713 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07380611e0 0x7f07380615c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f0728002f30 tx=0x7f0728006e80).ready entity=mon.1 client_cookie=e9f4e1006ff00dc3 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:30.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.267+0000 7f0736ffd700 1 -- 172.21.15.162:0/3440940713 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07280097f0 con 0x7f07380611e0 2020-11-28T21:46:30.273 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.267+0000 7f0736ffd700 1 -- 172.21.15.162:0/3440940713 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0728009950 con 0x7f07380611e0 2020-11-28T21:46:30.273 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.267+0000 7f0736ffd700 1 -- 172.21.15.162:0/3440940713 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0728009cb0 con 0x7f07380611e0 2020-11-28T21:46:30.273 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.267+0000 7f073cfe1700 1 -- 172.21.15.162:0/3440940713 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07380611e0 msgr2=0x7f07380615c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:30.273 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.267+0000 7f073cfe1700 1 --2- 172.21.15.162:0/3440940713 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07380611e0 0x7f07380615c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f0728002f30 tx=0x7f0728006e80).stop 2020-11-28T21:46:30.275 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.267+0000 7f073cfe1700 1 -- 172.21.15.162:0/3440940713 shutdown_connections 2020-11-28T21:46:30.275 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.267+0000 7f073cfe1700 1 --2- 172.21.15.162:0/3440940713 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07380611e0 0x7f07380615c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:30.276 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.267+0000 7f073cfe1700 1 --2- 172.21.15.162:0/3440940713 >> v2:172.21.15.118:3300/0 conn(0x7f07380b10f0 0x7f07380b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:30.276 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.267+0000 7f073cfe1700 1 -- 172.21.15.162:0/3440940713 >> 172.21.15.162:0/3440940713 conn(0x7f07380b54f0 msgr2=0x7f07380b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:30.276 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.267+0000 7f073cfe1700 1 -- 172.21.15.162:0/3440940713 shutdown_connections 2020-11-28T21:46:30.276 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.267+0000 7f073cfe1700 1 -- 172.21.15.162:0/3440940713 wait complete. 2020-11-28T21:46:30.276 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.267+0000 7f073cfe1700 1 Processor -- start 2020-11-28T21:46:30.276 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.267+0000 7f073cfe1700 1 -- start start 2020-11-28T21:46:30.277 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.271+0000 7f073cfe1700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f07380611e0 0x7f0738152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:30.277 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.271+0000 7f073cfe1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07380b10f0 0x7f073813de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:30.277 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.271+0000 7f073cfe1700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f07380b4920 0x7f0738154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:30.277 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.271+0000 7f073cfe1700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0738151700 con 0x7f07380b4920 2020-11-28T21:46:30.277 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.271+0000 7f073cfe1700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0738152560 con 0x7f07380b10f0 2020-11-28T21:46:30.278 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.271+0000 7f073cfe1700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07380af1a0 con 0x7f07380611e0 2020-11-28T21:46:30.278 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.271+0000 7f072ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07380b10f0 0x7f073813de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:30.278 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.271+0000 7f072ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07380b10f0 0x7f073813de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37224/0 (socket says 172.21.15.162:37224) 2020-11-28T21:46:30.278 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.271+0000 7f072ffff700 1 -- 172.21.15.162:0/1291197371 learned_addr learned my addr 172.21.15.162:0/1291197371 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:30.278 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.271+0000 7f07377fe700 1 --2- 172.21.15.162:0/1291197371 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f07380611e0 0x7f0738152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:30.279 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.271+0000 7f0737fff700 1 --2- 172.21.15.162:0/1291197371 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f07380b4920 0x7f0738154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:30.279 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.271+0000 7f072ffff700 1 -- 172.21.15.162:0/1291197371 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f07380611e0 msgr2=0x7f0738152070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:30.279 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.271+0000 7f072ffff700 1 --2- 172.21.15.162:0/1291197371 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f07380611e0 0x7f0738152070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:30.279 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.271+0000 7f072ffff700 1 -- 172.21.15.162:0/1291197371 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f07380b4920 msgr2=0x7f0738154ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:30.279 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.271+0000 7f072ffff700 1 --2- 172.21.15.162:0/1291197371 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f07380b4920 0x7f0738154ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:30.280 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.271+0000 7f072ffff700 1 -- 172.21.15.162:0/1291197371 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0738151930 con 0x7f07380b10f0 2020-11-28T21:46:30.280 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.271+0000 7f072ffff700 1 --2- 172.21.15.162:0/1291197371 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07380b10f0 0x7f073813de70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f072000a000 tx=0x7f0720008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:30.280 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.271+0000 7f07357fa700 1 -- 172.21.15.162:0/1291197371 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0720011780 con 0x7f07380b10f0 2020-11-28T21:46:30.280 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.271+0000 7f07357fa700 1 -- 172.21.15.162:0/1291197371 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07200118e0 con 0x7f07380b10f0 2020-11-28T21:46:30.280 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.271+0000 7f073cfe1700 1 -- 172.21.15.162:0/1291197371 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0738151bd0 con 0x7f07380b10f0 2020-11-28T21:46:30.280 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.271+0000 7f073cfe1700 1 -- 172.21.15.162:0/1291197371 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07380af440 con 0x7f07380b10f0 2020-11-28T21:46:30.281 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.271+0000 7f07357fa700 1 -- 172.21.15.162:0/1291197371 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0720011c40 con 0x7f07380b10f0 2020-11-28T21:46:30.281 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.271+0000 7f073cfe1700 1 -- 172.21.15.162:0/1291197371 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07100033e0 con 0x7f07380b10f0 2020-11-28T21:46:30.282 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.271+0000 7f07357fa700 1 -- 172.21.15.162:0/1291197371 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f072001c460 con 0x7f07380b10f0 2020-11-28T21:46:30.282 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.275+0000 7f07357fa700 1 --2- 172.21.15.162:0/1291197371 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f071c055440 0x7f071c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:30.282 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.275+0000 7f07357fa700 1 -- 172.21.15.162:0/1291197371 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f072006e200 con 0x7f07380b10f0 2020-11-28T21:46:30.283 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.275+0000 7f07377fe700 1 --2- 172.21.15.162:0/1291197371 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f071c055440 0x7f071c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:30.283 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.275+0000 7f07377fe700 1 --2- 172.21.15.162:0/1291197371 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f071c055440 0x7f071c057900 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f072801c000 tx=0x7f0728015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:30.284 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.279+0000 7f07357fa700 1 -- 172.21.15.162:0/1291197371 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f07200444a0 con 0x7f07380b10f0 2020-11-28T21:46:30.485 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.483+0000 7f073cfe1700 1 -- 172.21.15.162:0/1291197371 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0710003820 con 0x7f07380b10f0 2020-11-28T21:46:30.486 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.483+0000 7f07357fa700 1 -- 172.21.15.162:0/1291197371 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2495 (secure 0 0 0) 0x7f0720047ac0 con 0x7f07380b10f0 2020-11-28T21:46:30.487 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:46:30.487 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/1297721315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1297721315},{"type":"v1","addr":"172.21.15.118:6833","nonce":1297721315}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3},{"gid":4264,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2415281381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2415281381},{"type":"v1","addr":"172.21.15.162:6837","nonce":2415281381}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:46:29.120502+0000","modified":"2020-11-28T21:46:30.123428+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.162:6835/3696705934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3696705934},{"type":"v1","addr":"172.21.15.162:6835","nonce":3696705934}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-28T21:46:30.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.487+0000 7f073cfe1700 1 -- 172.21.15.162:0/1291197371 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f071c055440 msgr2=0x7f071c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:30.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.487+0000 7f073cfe1700 1 --2- 172.21.15.162:0/1291197371 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f071c055440 0x7f071c057900 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f072801c000 tx=0x7f0728015000).stop 2020-11-28T21:46:30.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.487+0000 7f073cfe1700 1 -- 172.21.15.162:0/1291197371 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07380b10f0 msgr2=0x7f073813de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:30.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.487+0000 7f073cfe1700 1 --2- 172.21.15.162:0/1291197371 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07380b10f0 0x7f073813de70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f072000a000 tx=0x7f0720008040).stop 2020-11-28T21:46:30.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.487+0000 7f073cfe1700 1 -- 172.21.15.162:0/1291197371 shutdown_connections 2020-11-28T21:46:30.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.487+0000 7f073cfe1700 1 --2- 172.21.15.162:0/1291197371 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f071c055440 0x7f071c057900 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:30.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.487+0000 7f073cfe1700 1 --2- 172.21.15.162:0/1291197371 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f07380611e0 0x7f0738152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:30.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.487+0000 7f073cfe1700 1 --2- 172.21.15.162:0/1291197371 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f07380b4920 0x7f0738154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:30.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.487+0000 7f073cfe1700 1 --2- 172.21.15.162:0/1291197371 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07380b10f0 0x7f073813de70 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:30.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.487+0000 7f073cfe1700 1 -- 172.21.15.162:0/1291197371 >> 172.21.15.162:0/1291197371 conn(0x7f07380b54f0 msgr2=0x7f073813b900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:30.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.487+0000 7f073cfe1700 1 -- 172.21.15.162:0/1291197371 shutdown_connections 2020-11-28T21:46:30.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.487+0000 7f073cfe1700 1 -- 172.21.15.162:0/1291197371 wait complete. 2020-11-28T21:46:30.492 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 5 2020-11-28T21:46:30.506 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:46:30.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.611+0000 7f1682b26700 1 Processor -- start 2020-11-28T21:46:30.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.611+0000 7f1682b26700 1 -- start start 2020-11-28T21:46:30.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.611+0000 7f1682b26700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f167c0603d0 0x7f167c0b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:30.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.611+0000 7f1682b26700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f167c142a00 0x7f167c09cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:30.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.611+0000 7f1682b26700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f167c0b4c70 0x7f167c09d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:30.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.611+0000 7f1682b26700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f167c060f60 con 0x7f167c142a00 2020-11-28T21:46:30.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.611+0000 7f1682b26700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f167c0b4110 con 0x7f167c0603d0 2020-11-28T21:46:30.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.611+0000 7f1682b26700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f167c0b43f0 con 0x7f167c0b4c70 2020-11-28T21:46:30.620 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.611+0000 7f1681b24700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f167c0603d0 0x7f167c0b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:30.620 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.611+0000 7f1681b24700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f167c0603d0 0x7f167c0b3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37228/0 (socket says 172.21.15.162:37228) 2020-11-28T21:46:30.620 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.611+0000 7f1681b24700 1 -- 172.21.15.162:0/2512489527 learned_addr learned my addr 172.21.15.162:0/2512489527 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:30.621 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.611+0000 7f1682325700 1 --2- 172.21.15.162:0/2512489527 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f167c0b4c70 0x7f167c09d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:30.621 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.611+0000 7f1681323700 1 --2- 172.21.15.162:0/2512489527 >> v2:172.21.15.118:3300/0 conn(0x7f167c142a00 0x7f167c09cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:30.621 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.611+0000 7f1681b24700 1 -- 172.21.15.162:0/2512489527 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f167c0b4c70 msgr2=0x7f167c09d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:30.622 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.611+0000 7f1681b24700 1 --2- 172.21.15.162:0/2512489527 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f167c0b4c70 0x7f167c09d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:30.622 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.611+0000 7f1681b24700 1 -- 172.21.15.162:0/2512489527 >> v2:172.21.15.118:3300/0 conn(0x7f167c142a00 msgr2=0x7f167c09cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:30.622 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.611+0000 7f1681b24700 1 --2- 172.21.15.162:0/2512489527 >> v2:172.21.15.118:3300/0 conn(0x7f167c142a00 0x7f167c09cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:30.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.611+0000 7f1681b24700 1 -- 172.21.15.162:0/2512489527 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f167c0b4690 con 0x7f167c0603d0 2020-11-28T21:46:30.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.611+0000 7f1681323700 1 --2- 172.21.15.162:0/2512489527 >> v2:172.21.15.118:3300/0 conn(0x7f167c142a00 0x7f167c09cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:46:30.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.611+0000 7f1682325700 1 --2- 172.21.15.162:0/2512489527 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f167c0b4c70 0x7f167c09d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:46:30.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.611+0000 7f1681b24700 1 --2- 172.21.15.162:0/2512489527 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f167c0603d0 0x7f167c0b3a70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f1674002f30 tx=0x7f1674006e80).ready entity=mon.1 client_cookie=646db8d7ad4f640d server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:30.624 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.615+0000 7f1680b22700 1 -- 172.21.15.162:0/2512489527 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f16740097f0 con 0x7f167c0603d0 2020-11-28T21:46:30.624 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.615+0000 7f1680b22700 1 -- 172.21.15.162:0/2512489527 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1674009950 con 0x7f167c0603d0 2020-11-28T21:46:30.624 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.615+0000 7f1680b22700 1 -- 172.21.15.162:0/2512489527 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1674009cb0 con 0x7f167c0603d0 2020-11-28T21:46:30.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.615+0000 7f1682b26700 1 -- 172.21.15.162:0/2512489527 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f167c0603d0 msgr2=0x7f167c0b3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:30.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.615+0000 7f1682b26700 1 --2- 172.21.15.162:0/2512489527 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f167c0603d0 0x7f167c0b3a70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f1674002f30 tx=0x7f1674006e80).stop 2020-11-28T21:46:30.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.615+0000 7f1682b26700 1 -- 172.21.15.162:0/2512489527 shutdown_connections 2020-11-28T21:46:30.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.615+0000 7f1682b26700 1 --2- 172.21.15.162:0/2512489527 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f167c0b4c70 0x7f167c09d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:30.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.615+0000 7f1682b26700 1 --2- 172.21.15.162:0/2512489527 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f167c0603d0 0x7f167c0b3a70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:30.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.615+0000 7f1682b26700 1 --2- 172.21.15.162:0/2512489527 >> v2:172.21.15.118:3300/0 conn(0x7f167c142a00 0x7f167c09cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:30.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.615+0000 7f1682b26700 1 -- 172.21.15.162:0/2512489527 >> 172.21.15.162:0/2512489527 conn(0x7f167c066a40 msgr2=0x7f167c066e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:30.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.615+0000 7f1682b26700 1 -- 172.21.15.162:0/2512489527 shutdown_connections 2020-11-28T21:46:30.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.615+0000 7f1682b26700 1 -- 172.21.15.162:0/2512489527 wait complete. 2020-11-28T21:46:30.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.615+0000 7f1682b26700 1 Processor -- start 2020-11-28T21:46:30.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.615+0000 7f1682b26700 1 -- start start 2020-11-28T21:46:30.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.615+0000 7f1682b26700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f167c0603d0 0x7f167c092eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:30.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.615+0000 7f1682b26700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f167c0b4c70 0x7f167c090df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:30.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.615+0000 7f1682b26700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f167c0933a0 0x7f167c0912e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:30.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.615+0000 7f1682b26700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167c092330 con 0x7f167c0b4c70 2020-11-28T21:46:30.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.615+0000 7f1682b26700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167c0afa80 con 0x7f167c0933a0 2020-11-28T21:46:30.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.615+0000 7f1682b26700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f167c0acf90 con 0x7f167c0603d0 2020-11-28T21:46:30.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.615+0000 7f1682325700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f167c0933a0 0x7f167c0912e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:30.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.615+0000 7f1682325700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f167c0933a0 0x7f167c0912e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37238/0 (socket says 172.21.15.162:37238) 2020-11-28T21:46:30.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.615+0000 7f1682325700 1 -- 172.21.15.162:0/2633677824 learned_addr learned my addr 172.21.15.162:0/2633677824 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:30.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.615+0000 7f1681323700 1 --2- 172.21.15.162:0/2633677824 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f167c0b4c70 0x7f167c090df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:30.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.615+0000 7f1682325700 1 -- 172.21.15.162:0/2633677824 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f167c0603d0 msgr2=0x7f167c092eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:30.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.615+0000 7f1681b24700 1 --2- 172.21.15.162:0/2633677824 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f167c0603d0 0x7f167c092eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:30.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.615+0000 7f1682325700 1 --2- 172.21.15.162:0/2633677824 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f167c0603d0 0x7f167c092eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:30.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.615+0000 7f1682325700 1 -- 172.21.15.162:0/2633677824 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f167c0b4c70 msgr2=0x7f167c090df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:30.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.615+0000 7f1682325700 1 --2- 172.21.15.162:0/2633677824 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f167c0b4c70 0x7f167c090df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:30.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.615+0000 7f1682325700 1 -- 172.21.15.162:0/2633677824 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f167c1516f0 con 0x7f167c0933a0 2020-11-28T21:46:30.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.615+0000 7f1681323700 1 --2- 172.21.15.162:0/2633677824 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f167c0b4c70 0x7f167c090df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:46:30.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.615+0000 7f1681b24700 1 --2- 172.21.15.162:0/2633677824 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f167c0603d0 0x7f167c092eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:46:30.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.615+0000 7f1682325700 1 --2- 172.21.15.162:0/2633677824 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f167c0933a0 0x7f167c0912e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f166c006dd0 tx=0x7f166c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:30.639 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.619+0000 7f1672ffd700 1 -- 172.21.15.162:0/2633677824 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f166c00c920 con 0x7f167c0933a0 2020-11-28T21:46:30.639 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.619+0000 7f1672ffd700 1 -- 172.21.15.162:0/2633677824 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f166c00ca80 con 0x7f167c0933a0 2020-11-28T21:46:30.639 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.619+0000 7f1682b26700 1 -- 172.21.15.162:0/2633677824 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f167c0917d0 con 0x7f167c0933a0 2020-11-28T21:46:30.639 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.619+0000 7f1672ffd700 1 -- 172.21.15.162:0/2633677824 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f166c00cde0 con 0x7f167c0933a0 2020-11-28T21:46:30.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.619+0000 7f1682b26700 1 -- 172.21.15.162:0/2633677824 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f167c151450 con 0x7f167c0933a0 2020-11-28T21:46:30.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.619+0000 7f1672ffd700 1 -- 172.21.15.162:0/2633677824 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f166c01d460 con 0x7f167c0933a0 2020-11-28T21:46:30.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.619+0000 7f1682b26700 1 -- 172.21.15.162:0/2633677824 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f167c0ad100 con 0x7f167c0933a0 2020-11-28T21:46:30.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.619+0000 7f1672ffd700 1 --2- 172.21.15.162:0/2633677824 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f165c055330 0x7f165c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:30.642 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.619+0000 7f1672ffd700 1 -- 172.21.15.162:0/2633677824 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f166c06e430 con 0x7f167c0933a0 2020-11-28T21:46:30.642 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.619+0000 7f1681b24700 1 --2- 172.21.15.162:0/2633677824 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f165c055330 0x7f165c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:30.643 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.619+0000 7f1681b24700 1 --2- 172.21.15.162:0/2633677824 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f165c055330 0x7f165c0577f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f1674017000 tx=0x7f1674015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:30.643 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.627+0000 7f1672ffd700 1 -- 172.21.15.162:0/2633677824 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f166c044630 con 0x7f167c0933a0 2020-11-28T21:46:30.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.767+0000 7f1682b26700 1 -- 172.21.15.162:0/2633677824 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f167c057050 con 0x7f167c0933a0 2020-11-28T21:46:30.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.771+0000 7f1672ffd700 1 -- 172.21.15.162:0/2633677824 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16134 (secure 0 0 0) 0x7f166c047c50 con 0x7f167c0933a0 2020-11-28T21:46:30.773 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:46:30.773 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":23,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:46:30.124354+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:46:27.155497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-28T21:46:28.137474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:46:30.776 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.771+0000 7f1682b26700 1 -- 172.21.15.162:0/2633677824 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f165c055330 msgr2=0x7f165c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:30.776 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.771+0000 7f1682b26700 1 --2- 172.21.15.162:0/2633677824 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f165c055330 0x7f165c0577f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f1674017000 tx=0x7f1674015000).stop 2020-11-28T21:46:30.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.771+0000 7f1682b26700 1 -- 172.21.15.162:0/2633677824 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f167c0933a0 msgr2=0x7f167c0912e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:30.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.771+0000 7f1682b26700 1 --2- 172.21.15.162:0/2633677824 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f167c0933a0 0x7f167c0912e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f166c006dd0 tx=0x7f166c008040).stop 2020-11-28T21:46:30.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.771+0000 7f1682b26700 1 -- 172.21.15.162:0/2633677824 shutdown_connections 2020-11-28T21:46:30.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.771+0000 7f1682b26700 1 --2- 172.21.15.162:0/2633677824 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f165c055330 0x7f165c0577f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:30.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.771+0000 7f1682b26700 1 --2- 172.21.15.162:0/2633677824 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f167c0603d0 0x7f167c092eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:30.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.771+0000 7f1682b26700 1 --2- 172.21.15.162:0/2633677824 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f167c0933a0 0x7f167c0912e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:30.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.771+0000 7f1682b26700 1 --2- 172.21.15.162:0/2633677824 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f167c0b4c70 0x7f167c090df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:30.780 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.771+0000 7f1682b26700 1 -- 172.21.15.162:0/2633677824 >> 172.21.15.162:0/2633677824 conn(0x7f167c066a40 msgr2=0x7f167c142260 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:30.780 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.771+0000 7f1682b26700 1 -- 172.21.15.162:0/2633677824 shutdown_connections 2020-11-28T21:46:30.781 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.771+0000 7f1682b26700 1 -- 172.21.15.162:0/2633677824 wait complete. 2020-11-28T21:46:30.788 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:46:30.789 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2020-11-28T21:46:30.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.923+0000 7f3259020700 1 Processor -- start 2020-11-28T21:46:30.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.923+0000 7f3259020700 1 -- start start 2020-11-28T21:46:30.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.923+0000 7f3259020700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f32540affc0 0x7f32540ae570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:30.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.923+0000 7f3259020700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3254060290 0x7f32540aea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:30.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.923+0000 7f3259020700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32540b03a0 con 0x7f32540aef50 2020-11-28T21:46:30.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.923+0000 7f3259020700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32540a8f10 con 0x7f32540affc0 2020-11-28T21:46:30.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.923+0000 7f3259020700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32540a91b0 con 0x7f3254060290 2020-11-28T21:46:30.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.923+0000 7f3252d9d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f32540affc0 0x7f32540ae570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:30.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.923+0000 7f3252d9d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f32540affc0 0x7f32540ae570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37244/0 (socket says 172.21.15.162:37244) 2020-11-28T21:46:30.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.923+0000 7f3252d9d700 1 -- 172.21.15.162:0/1583431907 learned_addr learned my addr 172.21.15.162:0/1583431907 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:30.931 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.923+0000 7f324bfff700 1 --2- 172.21.15.162:0/1583431907 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3254060290 0x7f32540aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:30.931 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.923+0000 7f3252d9d700 1 -- 172.21.15.162:0/1583431907 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3254060290 msgr2=0x7f32540aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:30.931 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.923+0000 7f3252d9d700 1 --2- 172.21.15.162:0/1583431907 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3254060290 0x7f32540aea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:30.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.923+0000 7f3252d9d700 1 -- 172.21.15.162:0/1583431907 >> v1:172.21.15.118:6789/0 conn(0x7f32540aef50 legacy=0x7f3254143ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:30.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.923+0000 7f3252d9d700 1 -- 172.21.15.162:0/1583431907 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32540a9450 con 0x7f32540affc0 2020-11-28T21:46:30.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.923+0000 7f3252d9d700 1 --2- 172.21.15.162:0/1583431907 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32540affc0 0x7f32540ae570 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f3244002f30 tx=0x7f3244006e80).ready entity=mon.1 client_cookie=1469b484fdee9409 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:30.933 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.923+0000 7f325259c700 1 -- 172.21.15.162:0/1583431907 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f32440097f0 con 0x7f32540affc0 2020-11-28T21:46:30.933 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.923+0000 7f325259c700 1 -- 172.21.15.162:0/1583431907 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3244009950 con 0x7f32540affc0 2020-11-28T21:46:30.933 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.923+0000 7f325259c700 1 -- 172.21.15.162:0/1583431907 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3244009cb0 con 0x7f32540affc0 2020-11-28T21:46:30.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.923+0000 7f3259020700 1 -- 172.21.15.162:0/1583431907 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32540affc0 msgr2=0x7f32540ae570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:30.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.923+0000 7f3259020700 1 --2- 172.21.15.162:0/1583431907 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32540affc0 0x7f32540ae570 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f3244002f30 tx=0x7f3244006e80).stop 2020-11-28T21:46:30.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.923+0000 7f3259020700 1 -- 172.21.15.162:0/1583431907 shutdown_connections 2020-11-28T21:46:30.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.923+0000 7f3259020700 1 --2- 172.21.15.162:0/1583431907 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32540affc0 0x7f32540ae570 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:30.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.923+0000 7f3259020700 1 --2- 172.21.15.162:0/1583431907 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3254060290 0x7f32540aea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:30.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.923+0000 7f3259020700 1 -- 172.21.15.162:0/1583431907 >> 172.21.15.162:0/1583431907 conn(0x7f325405b970 msgr2=0x7f325405bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:30.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.923+0000 7f3259020700 1 -- 172.21.15.162:0/1583431907 shutdown_connections 2020-11-28T21:46:30.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.923+0000 7f3259020700 1 -- 172.21.15.162:0/1583431907 wait complete. 2020-11-28T21:46:30.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.927+0000 7f3259020700 1 Processor -- start 2020-11-28T21:46:30.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.927+0000 7f3259020700 1 -- start start 2020-11-28T21:46:30.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.927+0000 7f3259020700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3254060290 0x7f32540b3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:30.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.927+0000 7f3259020700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f32540aef50 0x7f3254143560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:30.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.927+0000 7f3259020700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32540affc0 0x7f3254143a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:30.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.927+0000 7f3259020700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32540b3e40 con 0x7f3254060290 2020-11-28T21:46:30.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.927+0000 7f3259020700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325413e140 con 0x7f32540affc0 2020-11-28T21:46:30.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.927+0000 7f3259020700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f325413dea0 con 0x7f32540aef50 2020-11-28T21:46:30.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.927+0000 7f325359e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32540affc0 0x7f3254143a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:30.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.927+0000 7f325359e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32540affc0 0x7f3254143a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37252/0 (socket says 172.21.15.162:37252) 2020-11-28T21:46:30.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.927+0000 7f325359e700 1 -- 172.21.15.162:0/1469214636 learned_addr learned my addr 172.21.15.162:0/1469214636 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:30.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.927+0000 7f3252d9d700 1 --2- 172.21.15.162:0/1469214636 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3254060290 0x7f32540b3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:30.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.927+0000 7f324bfff700 1 --2- 172.21.15.162:0/1469214636 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f32540aef50 0x7f3254143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:30.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.931+0000 7f325359e700 1 -- 172.21.15.162:0/1469214636 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f32540aef50 msgr2=0x7f3254143560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:30.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.931+0000 7f325359e700 1 --2- 172.21.15.162:0/1469214636 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f32540aef50 0x7f3254143560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:30.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.931+0000 7f325359e700 1 -- 172.21.15.162:0/1469214636 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3254060290 msgr2=0x7f32540b3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:30.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.931+0000 7f325359e700 1 --2- 172.21.15.162:0/1469214636 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3254060290 0x7f32540b3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:30.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.931+0000 7f325359e700 1 -- 172.21.15.162:0/1469214636 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3254141d10 con 0x7f32540affc0 2020-11-28T21:46:30.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.931+0000 7f325359e700 1 --2- 172.21.15.162:0/1469214636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32540affc0 0x7f3254143a50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f325406b190 tx=0x7f324c007ed0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:30.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.931+0000 7f3250d99700 1 -- 172.21.15.162:0/1469214636 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f324c00a7b0 con 0x7f32540affc0 2020-11-28T21:46:30.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.931+0000 7f3250d99700 1 -- 172.21.15.162:0/1469214636 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f324c00a910 con 0x7f32540affc0 2020-11-28T21:46:30.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.931+0000 7f3259020700 1 -- 172.21.15.162:0/1469214636 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f325413cd30 con 0x7f32540affc0 2020-11-28T21:46:30.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.931+0000 7f3250d99700 1 -- 172.21.15.162:0/1469214636 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f324c00ac70 con 0x7f32540affc0 2020-11-28T21:46:30.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.931+0000 7f3259020700 1 -- 172.21.15.162:0/1469214636 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32541421b0 con 0x7f32540affc0 2020-11-28T21:46:30.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.931+0000 7f3259020700 1 -- 172.21.15.162:0/1469214636 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3254057050 con 0x7f32540affc0 2020-11-28T21:46:30.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.931+0000 7f3250d99700 1 -- 172.21.15.162:0/1469214636 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f324c019030 con 0x7f32540affc0 2020-11-28T21:46:30.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.931+0000 7f3250d99700 1 --2- 172.21.15.162:0/1469214636 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3238055440 0x7f3238057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:30.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.931+0000 7f3250d99700 1 -- 172.21.15.162:0/1469214636 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f324c06d7f0 con 0x7f32540affc0 2020-11-28T21:46:30.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.931+0000 7f3252d9d700 1 --2- 172.21.15.162:0/1469214636 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3238055440 0x7f3238057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:30.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.931+0000 7f3252d9d700 1 --2- 172.21.15.162:0/1469214636 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3238055440 0x7f3238057900 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f324401c000 tx=0x7f3244015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:30.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:30.939+0000 7f3250d99700 1 -- 172.21.15.162:0/1469214636 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f324c043af0 con 0x7f32540affc0 2020-11-28T21:46:31.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.079+0000 7f3259020700 1 -- 172.21.15.162:0/1469214636 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32540608f0 con 0x7f32540affc0 2020-11-28T21:46:31.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.079+0000 7f3250d99700 1 -- 172.21.15.162:0/1469214636 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16134 (secure 0 0 0) 0x7f324c047110 con 0x7f32540affc0 2020-11-28T21:46:31.085 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:46:31.086 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":23,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:46:30.124354+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:46:27.155497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-28T21:46:28.137474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:46:31.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.083+0000 7f3259020700 1 -- 172.21.15.162:0/1469214636 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3238055440 msgr2=0x7f3238057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:31.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.083+0000 7f3259020700 1 --2- 172.21.15.162:0/1469214636 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3238055440 0x7f3238057900 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f324401c000 tx=0x7f3244015000).stop 2020-11-28T21:46:31.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.083+0000 7f3259020700 1 -- 172.21.15.162:0/1469214636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32540affc0 msgr2=0x7f3254143a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:31.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.083+0000 7f3259020700 1 --2- 172.21.15.162:0/1469214636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32540affc0 0x7f3254143a50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f325406b190 tx=0x7f324c007ed0).stop 2020-11-28T21:46:31.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.083+0000 7f3259020700 1 -- 172.21.15.162:0/1469214636 shutdown_connections 2020-11-28T21:46:31.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.083+0000 7f3259020700 1 --2- 172.21.15.162:0/1469214636 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3238055440 0x7f3238057900 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:31.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.083+0000 7f3259020700 1 --2- 172.21.15.162:0/1469214636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32540affc0 0x7f3254143a50 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:31.094 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.083+0000 7f3259020700 1 --2- 172.21.15.162:0/1469214636 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3254060290 0x7f32540b3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:31.095 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.083+0000 7f3259020700 1 --2- 172.21.15.162:0/1469214636 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f32540aef50 0x7f3254143560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:31.095 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.083+0000 7f3259020700 1 -- 172.21.15.162:0/1469214636 >> 172.21.15.162:0/1469214636 conn(0x7f325405b970 msgr2=0x7f32540ae490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:31.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.083+0000 7f3259020700 1 -- 172.21.15.162:0/1469214636 shutdown_connections 2020-11-28T21:46:31.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.083+0000 7f3259020700 1 -- 172.21.15.162:0/1469214636 wait complete. 2020-11-28T21:46:31.098 INFO:tasks.ceph.ceph_manager.ceph:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-28T21:46:27.155497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-28T21:46:28.137474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-28T21:46:31.099 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:46:31.238 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.235+0000 7fafc1cac700 1 Processor -- start 2020-11-28T21:46:31.238 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.235+0000 7fafc1cac700 1 -- start start 2020-11-28T21:46:31.239 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.235+0000 7fafc1cac700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fafbc098860 0x7fafbc098c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:31.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.235+0000 7fafc1cac700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fafbc0610e0 0x7fafbc147aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:31.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.235+0000 7fafc1cac700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafbc0b0da0 con 0x7fafbc06ae10 2020-11-28T21:46:31.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.235+0000 7fafc1cac700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafbc146ed0 con 0x7fafbc0610e0 2020-11-28T21:46:31.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.235+0000 7fafc1cac700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafbc1470c0 con 0x7fafbc098860 2020-11-28T21:46:31.241 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.235+0000 7fafbbfff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fafbc06ae10 0x7fafbc14a420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:44100/0 (socket says 172.21.15.162:44100) 2020-11-28T21:46:31.241 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.235+0000 7fafbbfff700 1 -- 172.21.15.162:0/922445606 learned_addr learned my addr 172.21.15.162:0/922445606 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:31.241 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.235+0000 7fafc14ab700 1 --2- 172.21.15.162:0/922445606 >> v2:172.21.15.118:3300/0 conn(0x7fafbc0610e0 0x7fafbc147aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:31.241 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.235+0000 7fafc0caa700 1 --2- 172.21.15.162:0/922445606 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fafbc098860 0x7fafbc098c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:31.241 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.235+0000 7fafbb7fe700 1 -- 172.21.15.162:0/922445606 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1379526076 0 0) 0x7fafbc0b0da0 con 0x7fafbc06ae10 2020-11-28T21:46:31.242 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.235+0000 7fafbb7fe700 1 -- 172.21.15.162:0/922445606 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafa0003230 con 0x7fafbc06ae10 2020-11-28T21:46:31.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.235+0000 7fafbb7fe700 1 -- 172.21.15.162:0/922445606 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3759894855 0 0) 0x7fafa0003230 con 0x7fafbc06ae10 2020-11-28T21:46:31.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.235+0000 7fafbb7fe700 1 -- 172.21.15.162:0/922445606 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fafbc098860 msgr2=0x7fafbc098c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:31.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.235+0000 7fafbb7fe700 1 --2- 172.21.15.162:0/922445606 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fafbc098860 0x7fafbc098c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:31.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.235+0000 7fafbb7fe700 1 -- 172.21.15.162:0/922445606 >> v2:172.21.15.118:3300/0 conn(0x7fafbc0610e0 msgr2=0x7fafbc147aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:31.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.235+0000 7fafbb7fe700 1 --2- 172.21.15.162:0/922445606 >> v2:172.21.15.118:3300/0 conn(0x7fafbc0610e0 0x7fafbc147aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:31.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.235+0000 7fafbb7fe700 1 -- 172.21.15.162:0/922445606 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafbc147360 con 0x7fafbc06ae10 2020-11-28T21:46:31.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.235+0000 7fafbb7fe700 1 -- 172.21.15.162:0/922445606 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7fafac0016d0 con 0x7fafbc06ae10 2020-11-28T21:46:31.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.239+0000 7fafbb7fe700 1 -- 172.21.15.162:0/922445606 >> v1:172.21.15.162:6789/0 conn(0x7fafbc06ae10 legacy=0x7fafbc14a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:31.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.239+0000 7fafbb7fe700 1 --2- 172.21.15.162:0/922445606 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fafa0003720 0x7fafa0003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:31.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.239+0000 7fafbb7fe700 1 --2- 172.21.15.162:0/922445606 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafa00044d0 0x7fafa0004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:31.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.239+0000 7fafbb7fe700 1 --2- 172.21.15.162:0/922445606 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fafbc098860 0x7fafa0004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:31.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.239+0000 7fafbb7fe700 1 -- 172.21.15.162:0/922445606 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafbc1470c0 con 0x7fafa0003720 2020-11-28T21:46:31.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.239+0000 7fafbb7fe700 1 -- 172.21.15.162:0/922445606 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa0005380 con 0x7fafa00044d0 2020-11-28T21:46:31.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.239+0000 7fafbb7fe700 1 -- 172.21.15.162:0/922445606 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafa00042e0 con 0x7fafbc098860 2020-11-28T21:46:31.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.239+0000 7fafbbfff700 1 --2- 172.21.15.162:0/922445606 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafa00044d0 0x7fafa0004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:31.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.239+0000 7fafc0caa700 1 --2- 172.21.15.162:0/922445606 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fafa0003720 0x7fafa0003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:31.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.239+0000 7fafc14ab700 1 --2- 172.21.15.162:0/922445606 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fafbc098860 0x7fafa0004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:31.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.239+0000 7fafbbfff700 1 -- 172.21.15.162:0/922445606 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fafbc098860 msgr2=0x7fafa0004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:31.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.239+0000 7fafbbfff700 1 --2- 172.21.15.162:0/922445606 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fafbc098860 0x7fafa0004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:31.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.239+0000 7fafbbfff700 1 -- 172.21.15.162:0/922445606 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fafa0003720 msgr2=0x7fafa0003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:31.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.239+0000 7fafbbfff700 1 --2- 172.21.15.162:0/922445606 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fafa0003720 0x7fafa0003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:31.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.239+0000 7fafbbfff700 1 -- 172.21.15.162:0/922445606 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa0003230 con 0x7fafa00044d0 2020-11-28T21:46:31.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.239+0000 7fafbbfff700 1 --2- 172.21.15.162:0/922445606 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafa00044d0 0x7fafa0004950 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fafac0013e0 tx=0x7fafbc062a70).ready entity=mon.1 client_cookie=c8695bd95877f5c9 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:31.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.239+0000 7fafbb7fe700 1 -- 172.21.15.162:0/922445606 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafac0070d0 con 0x7fafa00044d0 2020-11-28T21:46:31.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.239+0000 7fafbb7fe700 1 -- 172.21.15.162:0/922445606 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafac007230 con 0x7fafa00044d0 2020-11-28T21:46:31.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.239+0000 7fafbb7fe700 1 -- 172.21.15.162:0/922445606 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafac0055e0 con 0x7fafa00044d0 2020-11-28T21:46:31.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.239+0000 7fafc1cac700 1 -- 172.21.15.162:0/922445606 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafa00044d0 msgr2=0x7fafa0004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:31.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.239+0000 7fafc1cac700 1 --2- 172.21.15.162:0/922445606 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafa00044d0 0x7fafa0004950 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fafac0013e0 tx=0x7fafbc062a70).stop 2020-11-28T21:46:31.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.239+0000 7fafc14ab700 1 -- 172.21.15.162:0/922445606 reap_dead start 2020-11-28T21:46:31.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.239+0000 7fafc1cac700 1 -- 172.21.15.162:0/922445606 shutdown_connections 2020-11-28T21:46:31.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.239+0000 7fafc1cac700 1 -- 172.21.15.162:0/922445606 >> 172.21.15.162:0/922445606 conn(0x7fafbc0b5520 msgr2=0x7fafbc0b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:31.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.239+0000 7fafc1cac700 1 -- 172.21.15.162:0/922445606 shutdown_connections 2020-11-28T21:46:31.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.239+0000 7fafc1cac700 1 -- 172.21.15.162:0/922445606 wait complete. 2020-11-28T21:46:31.253 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.243+0000 7fafc1cac700 1 Processor -- start 2020-11-28T21:46:31.253 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.243+0000 7fafc1cac700 1 -- start start 2020-11-28T21:46:31.253 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.243+0000 7fafc1cac700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fafbc1410c0 0x7fafbc145740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:31.253 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.243+0000 7fafc1cac700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafbc1462b0 0x7fafbc145c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:31.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.243+0000 7fafc1cac700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fafbc148210 0x7fafbc148690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:31.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.243+0000 7fafc1cac700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafbc1414a0 con 0x7fafbc148210 2020-11-28T21:46:31.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.243+0000 7fafc1cac700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafbc144d40 con 0x7fafbc1462b0 2020-11-28T21:46:31.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.243+0000 7fafc1cac700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafbc144fe0 con 0x7fafbc1410c0 2020-11-28T21:46:31.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.243+0000 7fafbbfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafbc1462b0 0x7fafbc145c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:31.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.243+0000 7fafbbfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafbc1462b0 0x7fafbc145c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37270/0 (socket says 172.21.15.162:37270) 2020-11-28T21:46:31.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.243+0000 7fafbbfff700 1 -- 172.21.15.162:0/1708088614 learned_addr learned my addr 172.21.15.162:0/1708088614 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:31.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.243+0000 7fafc0caa700 1 --2- 172.21.15.162:0/1708088614 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fafbc1410c0 0x7fafbc145740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:31.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.243+0000 7fafc14ab700 1 --2- 172.21.15.162:0/1708088614 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fafbc148210 0x7fafbc148690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:31.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.243+0000 7fafbbfff700 1 -- 172.21.15.162:0/1708088614 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fafbc1410c0 msgr2=0x7fafbc145740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:31.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.243+0000 7fafbbfff700 1 --2- 172.21.15.162:0/1708088614 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fafbc1410c0 0x7fafbc145740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:31.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.243+0000 7fafbbfff700 1 -- 172.21.15.162:0/1708088614 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fafbc148210 msgr2=0x7fafbc148690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:31.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.243+0000 7fafbbfff700 1 --2- 172.21.15.162:0/1708088614 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fafbc148210 0x7fafbc148690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:31.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.243+0000 7fafbbfff700 1 -- 172.21.15.162:0/1708088614 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafbc145280 con 0x7fafbc1462b0 2020-11-28T21:46:31.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.243+0000 7fafbbfff700 1 --2- 172.21.15.162:0/1708088614 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafbc1462b0 0x7fafbc145c30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fafac0013e0 tx=0x7fafac013000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:31.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.243+0000 7fafb9ffb700 1 -- 172.21.15.162:0/1708088614 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafac01a070 con 0x7fafbc1462b0 2020-11-28T21:46:31.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.243+0000 7fafb9ffb700 1 -- 172.21.15.162:0/1708088614 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafac005b40 con 0x7fafbc1462b0 2020-11-28T21:46:31.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.243+0000 7fafc1cac700 1 -- 172.21.15.162:0/1708088614 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafbc158ee0 con 0x7fafbc1462b0 2020-11-28T21:46:31.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.243+0000 7fafb9ffb700 1 -- 172.21.15.162:0/1708088614 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafac005ea0 con 0x7fafbc1462b0 2020-11-28T21:46:31.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.243+0000 7fafc1cac700 1 -- 172.21.15.162:0/1708088614 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafbc067610 con 0x7fafbc1462b0 2020-11-28T21:46:31.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.247+0000 7fafb9ffb700 1 -- 172.21.15.162:0/1708088614 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fafac026070 con 0x7fafbc1462b0 2020-11-28T21:46:31.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.247+0000 7fafc1cac700 1 -- 172.21.15.162:0/1708088614 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafbc057050 con 0x7fafbc1462b0 2020-11-28T21:46:31.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.247+0000 7fafb9ffb700 1 --2- 172.21.15.162:0/1708088614 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7faf94055440 0x7faf94057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:31.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.247+0000 7fafc0caa700 1 --2- 172.21.15.162:0/1708088614 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7faf94055440 0x7faf94057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:31.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.247+0000 7fafb9ffb700 1 -- 172.21.15.162:0/1708088614 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fafac06e760 con 0x7fafbc1462b0 2020-11-28T21:46:31.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.247+0000 7fafc0caa700 1 --2- 172.21.15.162:0/1708088614 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7faf94055440 0x7faf94057900 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fafa4008000 tx=0x7fafa4008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:31.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.251+0000 7fafb9ffb700 1 -- 172.21.15.162:0/1708088614 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fafac044950 con 0x7fafbc1462b0 2020-11-28T21:46:31.394 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.391+0000 7fafc1cac700 1 -- 172.21.15.162:0/1708088614 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fafbc159070 con 0x7fafbc1462b0 2020-11-28T21:46:31.395 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.391+0000 7fafb9ffb700 1 -- 172.21.15.162:0/1708088614 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v23) v1 ==== 110+0+10 (secure 0 0 0) 0x7fafac047f70 con 0x7fafbc1462b0 2020-11-28T21:46:31.395 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:46:31.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.395+0000 7fafc1cac700 1 -- 172.21.15.162:0/1708088614 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7faf94055440 msgr2=0x7faf94057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:31.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.395+0000 7fafc1cac700 1 --2- 172.21.15.162:0/1708088614 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7faf94055440 0x7faf94057900 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fafa4008000 tx=0x7fafa4008040).stop 2020-11-28T21:46:31.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.395+0000 7fafc1cac700 1 -- 172.21.15.162:0/1708088614 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafbc1462b0 msgr2=0x7fafbc145c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:31.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.395+0000 7fafc1cac700 1 --2- 172.21.15.162:0/1708088614 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafbc1462b0 0x7fafbc145c30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fafac0013e0 tx=0x7fafac013000).stop 2020-11-28T21:46:31.398 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.395+0000 7fafc1cac700 1 -- 172.21.15.162:0/1708088614 shutdown_connections 2020-11-28T21:46:31.398 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.395+0000 7fafc1cac700 1 --2- 172.21.15.162:0/1708088614 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7faf94055440 0x7faf94057900 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:31.399 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.395+0000 7fafc1cac700 1 --2- 172.21.15.162:0/1708088614 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fafbc1410c0 0x7fafbc145740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:31.399 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.395+0000 7fafc1cac700 1 --2- 172.21.15.162:0/1708088614 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fafbc148210 0x7fafbc148690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:31.400 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.395+0000 7fafc1cac700 1 --2- 172.21.15.162:0/1708088614 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafbc1462b0 0x7fafbc145c30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:31.400 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.395+0000 7fafc1cac700 1 -- 172.21.15.162:0/1708088614 >> 172.21.15.162:0/1708088614 conn(0x7fafbc0b5520 msgr2=0x7fafbc098f50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:31.400 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.395+0000 7fafc1cac700 1 -- 172.21.15.162:0/1708088614 shutdown_connections 2020-11-28T21:46:31.400 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.395+0000 7fafc1cac700 1 -- 172.21.15.162:0/1708088614 wait complete. 2020-11-28T21:46:31.410 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:46:31.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.531+0000 7f8a668ea700 1 Processor -- start 2020-11-28T21:46:31.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.531+0000 7f8a668ea700 1 -- start start 2020-11-28T21:46:31.537 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.531+0000 7f8a668ea700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8a600b18c0 0x7f8a60143190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:31.537 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.531+0000 7f8a668ea700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a600aac00 con 0x7f8a600b3c90 2020-11-28T21:46:31.537 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.531+0000 7f8a668ea700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a60060a10 con 0x7f8a6005f380 2020-11-28T21:46:31.538 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.531+0000 7f8a668ea700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a6013cf80 con 0x7f8a600b18c0 2020-11-28T21:46:31.538 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.531+0000 7f8a650e7700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8a600b18c0 0x7f8a60143190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:31.538 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.531+0000 7f8a660e9700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f8a6005f380 0x7f8a60146050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:44124/0 (socket says 172.21.15.162:44124) 2020-11-28T21:46:31.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.531+0000 7f8a660e9700 1 -- 172.21.15.162:0/1973943843 learned_addr learned my addr 172.21.15.162:0/1973943843 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:31.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.531+0000 7f8a650e7700 1 -- 172.21.15.162:0/1973943843 >> v1:172.21.15.162:6789/0 conn(0x7f8a6005f380 legacy=0x7f8a60146050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:31.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.531+0000 7f8a650e7700 1 -- 172.21.15.162:0/1973943843 >> v1:172.21.15.118:6789/0 conn(0x7f8a600b3c90 legacy=0x7f8a60142a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:31.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.531+0000 7f8a650e7700 1 -- 172.21.15.162:0/1973943843 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a600b4070 con 0x7f8a600b18c0 2020-11-28T21:46:31.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.531+0000 7f8a650e7700 1 --2- 172.21.15.162:0/1973943843 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a600b18c0 0x7f8a60143190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f8a580010d0 tx=0x7f8a50006dc0).ready entity=mon.1 client_cookie=f6327e4e013e3b38 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:31.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.531+0000 7f8a648e6700 1 -- 172.21.15.162:0/1973943843 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a500096f0 con 0x7f8a600b18c0 2020-11-28T21:46:31.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.531+0000 7f8a648e6700 1 -- 172.21.15.162:0/1973943843 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a50009850 con 0x7f8a600b18c0 2020-11-28T21:46:31.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.531+0000 7f8a648e6700 1 -- 172.21.15.162:0/1973943843 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a50009bb0 con 0x7f8a600b18c0 2020-11-28T21:46:31.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.535+0000 7f8a668ea700 1 -- 172.21.15.162:0/1973943843 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a600b18c0 msgr2=0x7f8a60143190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:31.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.535+0000 7f8a668ea700 1 --2- 172.21.15.162:0/1973943843 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a600b18c0 0x7f8a60143190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f8a580010d0 tx=0x7f8a50006dc0).stop 2020-11-28T21:46:31.543 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.535+0000 7f8a668ea700 1 -- 172.21.15.162:0/1973943843 shutdown_connections 2020-11-28T21:46:31.544 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.535+0000 7f8a668ea700 1 --2- 172.21.15.162:0/1973943843 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a600b18c0 0x7f8a60143190 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:31.544 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.535+0000 7f8a668ea700 1 -- 172.21.15.162:0/1973943843 >> 172.21.15.162:0/1973943843 conn(0x7f8a60062630 msgr2=0x7f8a60062a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:31.544 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.535+0000 7f8a668ea700 1 -- 172.21.15.162:0/1973943843 shutdown_connections 2020-11-28T21:46:31.544 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.535+0000 7f8a668ea700 1 -- 172.21.15.162:0/1973943843 wait complete. 2020-11-28T21:46:31.548 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.535+0000 7f8a668ea700 1 Processor -- start 2020-11-28T21:46:31.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.535+0000 7f8a668ea700 1 -- start start 2020-11-28T21:46:31.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.535+0000 7f8a668ea700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8a6005f380 0x7f8a6013a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:31.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.535+0000 7f8a668ea700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8a600b18c0 0x7f8a60155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:31.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.535+0000 7f8a668ea700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a600b3c90 0x7f8a60146300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:31.550 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.535+0000 7f8a668ea700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6013fb90 con 0x7f8a6005f380 2020-11-28T21:46:31.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.535+0000 7f8a668ea700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a60151eb0 con 0x7f8a600b3c90 2020-11-28T21:46:31.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.535+0000 7f8a668ea700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6013d6d0 con 0x7f8a600b18c0 2020-11-28T21:46:31.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.535+0000 7f8a660e9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a600b3c90 0x7f8a60146300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:31.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.535+0000 7f8a660e9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a600b3c90 0x7f8a60146300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37286/0 (socket says 172.21.15.162:37286) 2020-11-28T21:46:31.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.535+0000 7f8a660e9700 1 -- 172.21.15.162:0/2042869315 learned_addr learned my addr 172.21.15.162:0/2042869315 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:31.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.539+0000 7f8a658e8700 1 --2- 172.21.15.162:0/2042869315 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8a6005f380 0x7f8a6013a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:31.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.539+0000 7f8a660e9700 1 -- 172.21.15.162:0/2042869315 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8a600b18c0 msgr2=0x7f8a60155dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:31.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.539+0000 7f8a660e9700 1 --2- 172.21.15.162:0/2042869315 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8a600b18c0 0x7f8a60155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-28T21:46:31.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.539+0000 7f8a660e9700 1 -- 172.21.15.162:0/2042869315 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8a6005f380 msgr2=0x7f8a6013a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:31.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.539+0000 7f8a660e9700 1 --2- 172.21.15.162:0/2042869315 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8a6005f380 0x7f8a6013a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:31.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.539+0000 7f8a660e9700 1 -- 172.21.15.162:0/2042869315 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a60142660 con 0x7f8a600b3c90 2020-11-28T21:46:31.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.539+0000 7f8a660e9700 1 --2- 172.21.15.162:0/2042869315 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a600b3c90 0x7f8a60146300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f8a60141f00 tx=0x7f8a58007b50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:31.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.539+0000 7f8a4effd700 1 -- 172.21.15.162:0/2042869315 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a58007320 con 0x7f8a600b3c90 2020-11-28T21:46:31.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.539+0000 7f8a4effd700 1 -- 172.21.15.162:0/2042869315 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a58007480 con 0x7f8a600b3c90 2020-11-28T21:46:31.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.539+0000 7f8a4effd700 1 -- 172.21.15.162:0/2042869315 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a580057d0 con 0x7f8a600b3c90 2020-11-28T21:46:31.555 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.539+0000 7f8a668ea700 1 -- 172.21.15.162:0/2042869315 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a601428c0 con 0x7f8a600b3c90 2020-11-28T21:46:31.555 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.539+0000 7f8a668ea700 1 -- 172.21.15.162:0/2042869315 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a6013eb90 con 0x7f8a600b3c90 2020-11-28T21:46:31.555 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.539+0000 7f8a668ea700 1 -- 172.21.15.162:0/2042869315 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a60057050 con 0x7f8a600b3c90 2020-11-28T21:46:31.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.539+0000 7f8a4effd700 1 -- 172.21.15.162:0/2042869315 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8a58010030 con 0x7f8a600b3c90 2020-11-28T21:46:31.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.539+0000 7f8a4effd700 1 --2- 172.21.15.162:0/2042869315 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8a400554e0 0x7f8a400579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:31.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.539+0000 7f8a658e8700 1 --2- 172.21.15.162:0/2042869315 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8a400554e0 0x7f8a400579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:31.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.539+0000 7f8a4effd700 1 -- 172.21.15.162:0/2042869315 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8a5806deb0 con 0x7f8a600b3c90 2020-11-28T21:46:31.557 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.539+0000 7f8a658e8700 1 --2- 172.21.15.162:0/2042869315 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8a400554e0 0x7f8a400579a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f8a5c00b000 tx=0x7f8a5c009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:31.557 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.547+0000 7f8a4effd700 1 -- 172.21.15.162:0/2042869315 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8a580440a0 con 0x7f8a600b3c90 2020-11-28T21:46:31.687 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.683+0000 7f8a668ea700 1 -- 172.21.15.162:0/2042869315 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8a6005d250 con 0x7f8a600b3c90 2020-11-28T21:46:31.687 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.683+0000 7f8a4effd700 1 -- 172.21.15.162:0/2042869315 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8a580476c0 con 0x7f8a600b3c90 2020-11-28T21:46:31.688 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:46:31.689 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.687+0000 7f8a668ea700 1 -- 172.21.15.162:0/2042869315 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8a400554e0 msgr2=0x7f8a400579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:31.690 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.687+0000 7f8a668ea700 1 --2- 172.21.15.162:0/2042869315 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8a400554e0 0x7f8a400579a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f8a5c00b000 tx=0x7f8a5c009040).stop 2020-11-28T21:46:31.690 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.687+0000 7f8a668ea700 1 -- 172.21.15.162:0/2042869315 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a600b3c90 msgr2=0x7f8a60146300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:31.690 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.687+0000 7f8a668ea700 1 --2- 172.21.15.162:0/2042869315 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a600b3c90 0x7f8a60146300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f8a60141f00 tx=0x7f8a58007b50).stop 2020-11-28T21:46:31.690 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.687+0000 7f8a668ea700 1 -- 172.21.15.162:0/2042869315 shutdown_connections 2020-11-28T21:46:31.691 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.687+0000 7f8a668ea700 1 --2- 172.21.15.162:0/2042869315 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8a400554e0 0x7f8a400579a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:31.691 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.687+0000 7f8a668ea700 1 --2- 172.21.15.162:0/2042869315 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a600b3c90 0x7f8a60146300 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:31.691 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.687+0000 7f8a668ea700 1 --2- 172.21.15.162:0/2042869315 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8a6005f380 0x7f8a6013a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:31.691 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.687+0000 7f8a668ea700 1 --2- 172.21.15.162:0/2042869315 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8a600b18c0 0x7f8a60155dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-28T21:46:31.692 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.687+0000 7f8a668ea700 1 -- 172.21.15.162:0/2042869315 >> 172.21.15.162:0/2042869315 conn(0x7f8a60062630 msgr2=0x7f8a6013b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:31.692 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.687+0000 7f8a668ea700 1 -- 172.21.15.162:0/2042869315 shutdown_connections 2020-11-28T21:46:31.692 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.687+0000 7f8a668ea700 1 -- 172.21.15.162:0/2042869315 wait complete. 2020-11-28T21:46:31.701 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-28T21:46:31.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.815+0000 7fe72b59e700 1 Processor -- start 2020-11-28T21:46:31.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.815+0000 7fe72b59e700 1 -- start start 2020-11-28T21:46:31.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.819+0000 7fe72b59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe72c0b2d00 0x7fe72c05c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:31.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.819+0000 7fe72b59e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe72c0b30e0 0x7fe72c13fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:31.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.819+0000 7fe72b59e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe72c05baf0 con 0x7fe72c0b3ab0 2020-11-28T21:46:31.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.819+0000 7fe72b59e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe72c0b0e20 con 0x7fe72c0b30e0 2020-11-28T21:46:31.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.819+0000 7fe72b59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe72c05bd90 con 0x7fe72c0b2d00 2020-11-28T21:46:31.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.819+0000 7fe72ad9d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe72c0b30e0 0x7fe72c13fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:31.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.819+0000 7fe72ad9d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe72c0b30e0 0x7fe72c13fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37294/0 (socket says 172.21.15.162:37294) 2020-11-28T21:46:31.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.819+0000 7fe72ad9d700 1 -- 172.21.15.162:0/1378037146 learned_addr learned my addr 172.21.15.162:0/1378037146 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:31.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.819+0000 7fe72a59c700 1 --2- 172.21.15.162:0/1378037146 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe72c0b2d00 0x7fe72c05c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:31.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.819+0000 7fe72ad9d700 1 -- 172.21.15.162:0/1378037146 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe72c0b2d00 msgr2=0x7fe72c05c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:31.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.819+0000 7fe72ad9d700 1 --2- 172.21.15.162:0/1378037146 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe72c0b2d00 0x7fe72c05c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:31.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.819+0000 7fe72ad9d700 1 -- 172.21.15.162:0/1378037146 >> v1:172.21.15.118:6789/0 conn(0x7fe72c0b3ab0 legacy=0x7fe72c142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:31.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.819+0000 7fe72ad9d700 1 -- 172.21.15.162:0/1378037146 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe72c05c030 con 0x7fe72c0b30e0 2020-11-28T21:46:31.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.819+0000 7fe72a59c700 1 --2- 172.21.15.162:0/1378037146 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe72c0b2d00 0x7fe72c05c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:46:31.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.819+0000 7fe72ad9d700 1 --2- 172.21.15.162:0/1378037146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe72c0b30e0 0x7fe72c13fa70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fe7240032b0 tx=0x7fe724009270).ready entity=mon.1 client_cookie=4a9fd8d2de29f923 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:31.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.819+0000 7fe72959a700 1 -- 172.21.15.162:0/1378037146 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe72400bc00 con 0x7fe72c0b30e0 2020-11-28T21:46:31.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.819+0000 7fe72959a700 1 -- 172.21.15.162:0/1378037146 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe72400bd60 con 0x7fe72c0b30e0 2020-11-28T21:46:31.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.819+0000 7fe72959a700 1 -- 172.21.15.162:0/1378037146 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe724011400 con 0x7fe72c0b30e0 2020-11-28T21:46:31.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.819+0000 7fe72b59e700 1 -- 172.21.15.162:0/1378037146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe72c0b30e0 msgr2=0x7fe72c13fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:31.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.819+0000 7fe72b59e700 1 --2- 172.21.15.162:0/1378037146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe72c0b30e0 0x7fe72c13fa70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fe7240032b0 tx=0x7fe724009270).stop 2020-11-28T21:46:31.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.819+0000 7fe72b59e700 1 -- 172.21.15.162:0/1378037146 shutdown_connections 2020-11-28T21:46:31.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.819+0000 7fe72b59e700 1 --2- 172.21.15.162:0/1378037146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe72c0b30e0 0x7fe72c13fa70 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:31.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.819+0000 7fe72b59e700 1 --2- 172.21.15.162:0/1378037146 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe72c0b2d00 0x7fe72c05c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:31.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.819+0000 7fe72b59e700 1 -- 172.21.15.162:0/1378037146 >> 172.21.15.162:0/1378037146 conn(0x7fe72c0604c0 msgr2=0x7fe72c0608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:31.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.819+0000 7fe72b59e700 1 -- 172.21.15.162:0/1378037146 shutdown_connections 2020-11-28T21:46:31.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.819+0000 7fe72b59e700 1 -- 172.21.15.162:0/1378037146 wait complete. 2020-11-28T21:46:31.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.823+0000 7fe72b59e700 1 Processor -- start 2020-11-28T21:46:31.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.823+0000 7fe72b59e700 1 -- start start 2020-11-28T21:46:31.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.823+0000 7fe72b59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe72c0b2d00 0x7fe72c144210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:31.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.823+0000 7fe72b59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe72c0b30e0 0x7fe72c154db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:31.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.823+0000 7fe72b59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe72c0b3ab0 0x7fe72c145fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:31.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.823+0000 7fe72b59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe72c0b5e00 con 0x7fe72c0b2d00 2020-11-28T21:46:31.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.823+0000 7fe72b59e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe72c146b70 con 0x7fe72c0b30e0 2020-11-28T21:46:31.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.823+0000 7fe72b59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe72c146e10 con 0x7fe72c0b3ab0 2020-11-28T21:46:31.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.823+0000 7fe729d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe72c0b30e0 0x7fe72c154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:31.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.823+0000 7fe729d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe72c0b30e0 0x7fe72c154db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37298/0 (socket says 172.21.15.162:37298) 2020-11-28T21:46:31.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.823+0000 7fe729d9b700 1 -- 172.21.15.162:0/834072990 learned_addr learned my addr 172.21.15.162:0/834072990 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:31.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.823+0000 7fe72a59c700 1 --2- 172.21.15.162:0/834072990 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe72c0b2d00 0x7fe72c144210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:31.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.823+0000 7fe72ad9d700 1 --2- 172.21.15.162:0/834072990 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe72c0b3ab0 0x7fe72c145fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:31.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.823+0000 7fe729d9b700 1 -- 172.21.15.162:0/834072990 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe72c0b3ab0 msgr2=0x7fe72c145fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:31.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.823+0000 7fe729d9b700 1 --2- 172.21.15.162:0/834072990 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe72c0b3ab0 0x7fe72c145fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:31.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.823+0000 7fe729d9b700 1 -- 172.21.15.162:0/834072990 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe72c0b2d00 msgr2=0x7fe72c144210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:31.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.823+0000 7fe729d9b700 1 --2- 172.21.15.162:0/834072990 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe72c0b2d00 0x7fe72c144210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:31.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.823+0000 7fe729d9b700 1 -- 172.21.15.162:0/834072990 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe72c1458a0 con 0x7fe72c0b30e0 2020-11-28T21:46:31.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.823+0000 7fe72a59c700 1 --2- 172.21.15.162:0/834072990 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe72c0b2d00 0x7fe72c144210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:46:31.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.823+0000 7fe72ad9d700 1 --2- 172.21.15.162:0/834072990 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe72c0b3ab0 0x7fe72c145fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:46:31.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.823+0000 7fe729d9b700 1 --2- 172.21.15.162:0/834072990 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe72c0b30e0 0x7fe72c154db0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fe72c0611e0 tx=0x7fe71c007ba0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:31.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.823+0000 7fe7237fe700 1 -- 172.21.15.162:0/834072990 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe71c007030 con 0x7fe72c0b30e0 2020-11-28T21:46:31.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.823+0000 7fe7237fe700 1 -- 172.21.15.162:0/834072990 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe71c007190 con 0x7fe72c0b30e0 2020-11-28T21:46:31.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.823+0000 7fe7237fe700 1 -- 172.21.15.162:0/834072990 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe71c0064e0 con 0x7fe72c0b30e0 2020-11-28T21:46:31.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.823+0000 7fe72b59e700 1 -- 172.21.15.162:0/834072990 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe72c145b40 con 0x7fe72c0b30e0 2020-11-28T21:46:31.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.823+0000 7fe72b59e700 1 -- 172.21.15.162:0/834072990 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe72c1554c0 con 0x7fe72c0b30e0 2020-11-28T21:46:31.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.823+0000 7fe72b59e700 1 -- 172.21.15.162:0/834072990 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe72c145cd0 con 0x7fe72c0b30e0 2020-11-28T21:46:31.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.827+0000 7fe7237fe700 1 -- 172.21.15.162:0/834072990 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe71c010070 con 0x7fe72c0b30e0 2020-11-28T21:46:31.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.827+0000 7fe7237fe700 1 --2- 172.21.15.162:0/834072990 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe710055330 0x7fe7100577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:31.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.827+0000 7fe7237fe700 1 -- 172.21.15.162:0/834072990 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe71c06d670 con 0x7fe72c0b30e0 2020-11-28T21:46:31.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.827+0000 7fe72a59c700 1 --2- 172.21.15.162:0/834072990 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe710055330 0x7fe7100577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:31.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.827+0000 7fe72a59c700 1 --2- 172.21.15.162:0/834072990 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe710055330 0x7fe7100577f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fe71400b000 tx=0x7fe714008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:31.838 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.835+0000 7fe7237fe700 1 -- 172.21.15.162:0/834072990 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe71c043860 con 0x7fe72c0b30e0 2020-11-28T21:46:31.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.975+0000 7fe72b59e700 1 -- 172.21.15.162:0/834072990 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe72c057050 con 0x7fe72c0b30e0 2020-11-28T21:46:31.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.975+0000 7fe7237fe700 1 -- 172.21.15.162:0/834072990 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+10 (secure 0 0 0) 0x7fe71c046e80 con 0x7fe72c0b30e0 2020-11-28T21:46:31.980 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:46:31.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.979+0000 7fe72b59e700 1 -- 172.21.15.162:0/834072990 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe710055330 msgr2=0x7fe7100577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:31.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.979+0000 7fe72b59e700 1 --2- 172.21.15.162:0/834072990 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe710055330 0x7fe7100577f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fe71400b000 tx=0x7fe714008040).stop 2020-11-28T21:46:31.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.979+0000 7fe72b59e700 1 -- 172.21.15.162:0/834072990 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe72c0b30e0 msgr2=0x7fe72c154db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:31.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.979+0000 7fe72b59e700 1 --2- 172.21.15.162:0/834072990 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe72c0b30e0 0x7fe72c154db0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fe72c0611e0 tx=0x7fe71c007ba0).stop 2020-11-28T21:46:31.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.979+0000 7fe72b59e700 1 -- 172.21.15.162:0/834072990 shutdown_connections 2020-11-28T21:46:31.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.979+0000 7fe72b59e700 1 --2- 172.21.15.162:0/834072990 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe710055330 0x7fe7100577f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:31.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.979+0000 7fe72b59e700 1 --2- 172.21.15.162:0/834072990 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe72c0b3ab0 0x7fe72c145fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:31.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.979+0000 7fe72b59e700 1 --2- 172.21.15.162:0/834072990 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe72c0b2d00 0x7fe72c144210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:31.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.979+0000 7fe72b59e700 1 --2- 172.21.15.162:0/834072990 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe72c0b30e0 0x7fe72c154db0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:31.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.979+0000 7fe72b59e700 1 -- 172.21.15.162:0/834072990 >> 172.21.15.162:0/834072990 conn(0x7fe72c0604c0 msgr2=0x7fe72c13b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:31.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.979+0000 7fe72b59e700 1 -- 172.21.15.162:0/834072990 shutdown_connections 2020-11-28T21:46:31.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:31.979+0000 7fe72b59e700 1 -- 172.21.15.162:0/834072990 wait complete. 2020-11-28T21:46:31.993 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-28T21:46:32.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.103+0000 7fe879252700 1 Processor -- start 2020-11-28T21:46:32.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.103+0000 7fe879252700 1 -- start start 2020-11-28T21:46:32.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe879252700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe8740b1120 0x7fe874147660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:32.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe879252700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8740a8690 con 0x7fe8740b5730 2020-11-28T21:46:32.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe879252700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8740a88c0 con 0x7fe87414a440 2020-11-28T21:46:32.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe879252700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8741471c0 con 0x7fe8740b1120 2020-11-28T21:46:32.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe87259c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe8740b1120 0x7fe874147660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:32.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe87359e700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fe87414a440 0x7fe8741517c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:44152/0 (socket says 172.21.15.162:44152) 2020-11-28T21:46:32.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe87359e700 1 -- 172.21.15.162:0/407146368 learned_addr learned my addr 172.21.15.162:0/407146368 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:32.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe871d9b700 1 -- 172.21.15.162:0/407146368 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4247426913 0 0) 0x7fe8740a88c0 con 0x7fe87414a440 2020-11-28T21:46:32.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe871d9b700 1 -- 172.21.15.162:0/407146368 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe854003230 con 0x7fe87414a440 2020-11-28T21:46:32.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe871d9b700 1 -- 172.21.15.162:0/407146368 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1233363372 0 0) 0x7fe854003230 con 0x7fe87414a440 2020-11-28T21:46:32.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe871d9b700 1 -- 172.21.15.162:0/407146368 >> v2:172.21.15.162:3300/0 conn(0x7fe8740b1120 msgr2=0x7fe874147660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:32.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe871d9b700 1 --2- 172.21.15.162:0/407146368 >> v2:172.21.15.162:3300/0 conn(0x7fe8740b1120 0x7fe874147660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:32.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe871d9b700 1 -- 172.21.15.162:0/407146368 >> v1:172.21.15.118:6789/0 conn(0x7fe8740b5730 legacy=0x7fe87414ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:32.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe871d9b700 1 -- 172.21.15.162:0/407146368 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe87414b360 con 0x7fe87414a440 2020-11-28T21:46:32.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe871d9b700 1 -- 172.21.15.162:0/407146368 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7fe86c0016d0 con 0x7fe87414a440 2020-11-28T21:46:32.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe87259c700 1 --2- 172.21.15.162:0/407146368 >> v2:172.21.15.162:3300/0 conn(0x7fe8740b1120 0x7fe874147660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:46:32.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe871d9b700 1 -- 172.21.15.162:0/407146368 >> v1:172.21.15.162:6789/0 conn(0x7fe87414a440 legacy=0x7fe8741517c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:32.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe871d9b700 1 --2- 172.21.15.162:0/407146368 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe854003720 0x7fe854003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:32.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe871d9b700 1 --2- 172.21.15.162:0/407146368 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe8540044d0 0x7fe854004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:32.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe871d9b700 1 --2- 172.21.15.162:0/407146368 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe854004e40 0x7fe85400b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:32.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe871d9b700 1 -- 172.21.15.162:0/407146368 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8741471c0 con 0x7fe854004e40 2020-11-28T21:46:32.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe871d9b700 1 -- 172.21.15.162:0/407146368 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe854004250 con 0x7fe854003720 2020-11-28T21:46:32.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe871d9b700 1 -- 172.21.15.162:0/407146368 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe85400bc10 con 0x7fe8540044d0 2020-11-28T21:46:32.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe872d9d700 1 --2- 172.21.15.162:0/407146368 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe854003720 0x7fe854003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:32.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe87359e700 1 --2- 172.21.15.162:0/407146368 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe854004e40 0x7fe85400b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:32.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe87259c700 1 --2- 172.21.15.162:0/407146368 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe8540044d0 0x7fe854004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:32.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe872d9d700 1 -- 172.21.15.162:0/407146368 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe8540044d0 msgr2=0x7fe854004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:32.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe872d9d700 1 --2- 172.21.15.162:0/407146368 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe8540044d0 0x7fe854004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:32.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe872d9d700 1 -- 172.21.15.162:0/407146368 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe854004e40 msgr2=0x7fe85400b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:32.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe872d9d700 1 --2- 172.21.15.162:0/407146368 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe854004e40 0x7fe85400b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:32.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe872d9d700 1 -- 172.21.15.162:0/407146368 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe854003230 con 0x7fe854003720 2020-11-28T21:46:32.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe87359e700 1 -- 172.21.15.162:0/407146368 reap_dead start 2020-11-28T21:46:32.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe872d9d700 1 --2- 172.21.15.162:0/407146368 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe854003720 0x7fe854003b40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fe8740ac030 tx=0x7fe864007ee0).ready entity=mon.1 client_cookie=20825777987013b0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:32.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe871d9b700 1 -- 172.21.15.162:0/407146368 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe86400c9a0 con 0x7fe854003720 2020-11-28T21:46:32.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe871d9b700 1 -- 172.21.15.162:0/407146368 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe86400cb00 con 0x7fe854003720 2020-11-28T21:46:32.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe871d9b700 1 -- 172.21.15.162:0/407146368 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe86400ce60 con 0x7fe854003720 2020-11-28T21:46:32.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe879252700 1 -- 172.21.15.162:0/407146368 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe854003720 msgr2=0x7fe854003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:32.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe879252700 1 --2- 172.21.15.162:0/407146368 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe854003720 0x7fe854003b40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fe8740ac030 tx=0x7fe864007ee0).stop 2020-11-28T21:46:32.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe879252700 1 -- 172.21.15.162:0/407146368 shutdown_connections 2020-11-28T21:46:32.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe879252700 1 --2- 172.21.15.162:0/407146368 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe854003720 0x7fe854003b40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:32.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe879252700 1 -- 172.21.15.162:0/407146368 >> 172.21.15.162:0/407146368 conn(0x7fe87405c3f0 msgr2=0x7fe8740b4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:32.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe879252700 1 -- 172.21.15.162:0/407146368 shutdown_connections 2020-11-28T21:46:32.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.107+0000 7fe879252700 1 -- 172.21.15.162:0/407146368 wait complete. 2020-11-28T21:46:32.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.111+0000 7fe879252700 1 Processor -- start 2020-11-28T21:46:32.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.111+0000 7fe879252700 1 -- start start 2020-11-28T21:46:32.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.111+0000 7fe879252700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe854003720 0x7fe874149490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:32.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.111+0000 7fe879252700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe874146ba0 0x7fe874143bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:32.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.111+0000 7fe879252700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8740aee10 0x7fe8741440e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:32.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.111+0000 7fe879252700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8741445d0 con 0x7fe874146ba0 2020-11-28T21:46:32.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.111+0000 7fe879252700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8741467c0 con 0x7fe8740aee10 2020-11-28T21:46:32.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.111+0000 7fe879252700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8740ad8f0 con 0x7fe854003720 2020-11-28T21:46:32.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.111+0000 7fe87359e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8740aee10 0x7fe8741440e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:32.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.111+0000 7fe872d9d700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe854003720 0x7fe874149490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:32.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.111+0000 7fe87359e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8740aee10 0x7fe8741440e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37320/0 (socket says 172.21.15.162:37320) 2020-11-28T21:46:32.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.111+0000 7fe87359e700 1 -- 172.21.15.162:0/4283592199 learned_addr learned my addr 172.21.15.162:0/4283592199 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:32.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.111+0000 7fe87259c700 1 --2- 172.21.15.162:0/4283592199 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe874146ba0 0x7fe874143bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:32.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.111+0000 7fe87359e700 1 -- 172.21.15.162:0/4283592199 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe854003720 msgr2=0x7fe874149490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:32.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.111+0000 7fe87359e700 1 --2- 172.21.15.162:0/4283592199 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe854003720 0x7fe874149490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:32.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.111+0000 7fe87359e700 1 -- 172.21.15.162:0/4283592199 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe874146ba0 msgr2=0x7fe874143bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:32.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.111+0000 7fe87359e700 1 --2- 172.21.15.162:0/4283592199 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe874146ba0 0x7fe874143bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:32.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.111+0000 7fe87359e700 1 -- 172.21.15.162:0/4283592199 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe874146080 con 0x7fe8740aee10 2020-11-28T21:46:32.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.111+0000 7fe87359e700 1 --2- 172.21.15.162:0/4283592199 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8740aee10 0x7fe8741440e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fe86c0010d0 tx=0x7fe8740989e0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:32.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.111+0000 7fe86bfff700 1 -- 172.21.15.162:0/4283592199 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe86c00f870 con 0x7fe8740aee10 2020-11-28T21:46:32.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.111+0000 7fe86bfff700 1 -- 172.21.15.162:0/4283592199 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe86c00f9d0 con 0x7fe8740aee10 2020-11-28T21:46:32.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.111+0000 7fe86bfff700 1 -- 172.21.15.162:0/4283592199 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe86c00fd30 con 0x7fe8740aee10 2020-11-28T21:46:32.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.111+0000 7fe879252700 1 -- 172.21.15.162:0/4283592199 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe874146320 con 0x7fe8740aee10 2020-11-28T21:46:32.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.111+0000 7fe879252700 1 -- 172.21.15.162:0/4283592199 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8740ac900 con 0x7fe8740aee10 2020-11-28T21:46:32.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.115+0000 7fe879252700 1 -- 172.21.15.162:0/4283592199 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8741464b0 con 0x7fe8740aee10 2020-11-28T21:46:32.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.115+0000 7fe86bfff700 1 -- 172.21.15.162:0/4283592199 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe86c01c360 con 0x7fe8740aee10 2020-11-28T21:46:32.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.115+0000 7fe86bfff700 1 --2- 172.21.15.162:0/4283592199 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe858055470 0x7fe858057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:32.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.115+0000 7fe86bfff700 1 -- 172.21.15.162:0/4283592199 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe86c06d5c0 con 0x7fe8740aee10 2020-11-28T21:46:32.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.115+0000 7fe872d9d700 1 --2- 172.21.15.162:0/4283592199 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe858055470 0x7fe858057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:32.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.115+0000 7fe872d9d700 1 --2- 172.21.15.162:0/4283592199 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe858055470 0x7fe858057930 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fe86401a000 tx=0x7fe864016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:32.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.123+0000 7fe86bfff700 1 -- 172.21.15.162:0/4283592199 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe86c043860 con 0x7fe8740aee10 2020-11-28T21:46:32.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.263+0000 7fe879252700 1 -- 172.21.15.162:0/4283592199 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe874057050 con 0x7fe8740aee10 2020-11-28T21:46:32.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.263+0000 7fe86bfff700 1 -- 172.21.15.162:0/4283592199 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+10 (secure 0 0 0) 0x7fe86c019030 con 0x7fe8740aee10 2020-11-28T21:46:32.265 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:46:32.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.263+0000 7fe879252700 1 -- 172.21.15.162:0/4283592199 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe858055470 msgr2=0x7fe858057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:32.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.263+0000 7fe879252700 1 --2- 172.21.15.162:0/4283592199 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe858055470 0x7fe858057930 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fe86401a000 tx=0x7fe864016000).stop 2020-11-28T21:46:32.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.263+0000 7fe879252700 1 -- 172.21.15.162:0/4283592199 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8740aee10 msgr2=0x7fe8741440e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:32.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.263+0000 7fe879252700 1 --2- 172.21.15.162:0/4283592199 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8740aee10 0x7fe8741440e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fe86c0010d0 tx=0x7fe8740989e0).stop 2020-11-28T21:46:32.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.263+0000 7fe879252700 1 -- 172.21.15.162:0/4283592199 shutdown_connections 2020-11-28T21:46:32.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.263+0000 7fe879252700 1 --2- 172.21.15.162:0/4283592199 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe858055470 0x7fe858057930 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:32.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.263+0000 7fe879252700 1 --2- 172.21.15.162:0/4283592199 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe854003720 0x7fe874149490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:32.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.263+0000 7fe879252700 1 --2- 172.21.15.162:0/4283592199 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8740aee10 0x7fe8741440e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:32.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.263+0000 7fe879252700 1 --2- 172.21.15.162:0/4283592199 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe874146ba0 0x7fe874143bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:32.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.263+0000 7fe879252700 1 -- 172.21.15.162:0/4283592199 >> 172.21.15.162:0/4283592199 conn(0x7fe87405c3f0 msgr2=0x7fe8740b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:32.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.263+0000 7fe879252700 1 -- 172.21.15.162:0/4283592199 shutdown_connections 2020-11-28T21:46:32.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.263+0000 7fe879252700 1 -- 172.21.15.162:0/4283592199 wait complete. 2020-11-28T21:46:32.277 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2020-11-28T21:46:32.278 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2020-11-28T21:46:32.278 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early mgr dump --format=json 2020-11-28T21:46:32.387 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.383+0000 7f050a002700 1 Processor -- start 2020-11-28T21:46:32.387 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.383+0000 7f050a002700 1 -- start start 2020-11-28T21:46:32.387 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.383+0000 7f050a002700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f05040608f0 0x7f050413c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:32.388 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.383+0000 7f050a002700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f05040b6130 0x7f050413ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:32.388 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.383+0000 7f050a002700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f05040b54f0 0x7f0504143020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:32.388 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.383+0000 7f050a002700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05040ba0b0 con 0x7f05040b54f0 2020-11-28T21:46:32.388 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.383+0000 7f050a002700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05040b9e10 con 0x7f05040608f0 2020-11-28T21:46:32.389 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.383+0000 7f050a002700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0504142b80 con 0x7f05040b6130 2020-11-28T21:46:32.389 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.383+0000 7f05037fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f05040608f0 0x7f050413c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:32.389 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.383+0000 7f05037fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f05040608f0 0x7f050413c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37324/0 (socket says 172.21.15.162:37324) 2020-11-28T21:46:32.389 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.383+0000 7f05037fe700 1 -- 172.21.15.162:0/2524367879 learned_addr learned my addr 172.21.15.162:0/2524367879 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:32.393 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.383+0000 7f0502ffd700 1 --2- 172.21.15.162:0/2524367879 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f05040b6130 0x7f050413ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:32.393 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.383+0000 7f0503fff700 1 --2- 172.21.15.162:0/2524367879 >> v2:172.21.15.118:3300/0 conn(0x7f05040b54f0 0x7f0504143020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:32.393 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.383+0000 7f05037fe700 1 -- 172.21.15.162:0/2524367879 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f05040b6130 msgr2=0x7f050413ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:32.394 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.383+0000 7f05037fe700 1 --2- 172.21.15.162:0/2524367879 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f05040b6130 0x7f050413ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:32.394 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.383+0000 7f05037fe700 1 -- 172.21.15.162:0/2524367879 >> v2:172.21.15.118:3300/0 conn(0x7f05040b54f0 msgr2=0x7f0504143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:32.394 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.383+0000 7f05037fe700 1 --2- 172.21.15.162:0/2524367879 >> v2:172.21.15.118:3300/0 conn(0x7f05040b54f0 0x7f0504143020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:32.394 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.383+0000 7f05037fe700 1 -- 172.21.15.162:0/2524367879 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05040b9b30 con 0x7f05040608f0 2020-11-28T21:46:32.395 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.383+0000 7f0503fff700 1 --2- 172.21.15.162:0/2524367879 >> v2:172.21.15.118:3300/0 conn(0x7f05040b54f0 0x7f0504143020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:46:32.395 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.383+0000 7f0502ffd700 1 --2- 172.21.15.162:0/2524367879 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f05040b6130 0x7f050413ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:46:32.395 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.383+0000 7f05037fe700 1 --2- 172.21.15.162:0/2524367879 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05040608f0 0x7f050413c910 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f04f40032b0 tx=0x7f04f4009270).ready entity=mon.1 client_cookie=dbfd45a7b1de8cb5 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:32.395 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.383+0000 7f05027fc700 1 -- 172.21.15.162:0/2524367879 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04f400bc00 con 0x7f05040608f0 2020-11-28T21:46:32.395 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.387+0000 7f05027fc700 1 -- 172.21.15.162:0/2524367879 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04f400bd60 con 0x7f05040608f0 2020-11-28T21:46:32.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.387+0000 7f05027fc700 1 -- 172.21.15.162:0/2524367879 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04f4011400 con 0x7f05040608f0 2020-11-28T21:46:32.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.387+0000 7f050a002700 1 -- 172.21.15.162:0/2524367879 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05040608f0 msgr2=0x7f050413c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:32.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.387+0000 7f050a002700 1 --2- 172.21.15.162:0/2524367879 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05040608f0 0x7f050413c910 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f04f40032b0 tx=0x7f04f4009270).stop 2020-11-28T21:46:32.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.387+0000 7f050a002700 1 -- 172.21.15.162:0/2524367879 shutdown_connections 2020-11-28T21:46:32.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.387+0000 7f050a002700 1 --2- 172.21.15.162:0/2524367879 >> v2:172.21.15.118:3300/0 conn(0x7f05040b54f0 0x7f0504143020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:32.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.387+0000 7f050a002700 1 --2- 172.21.15.162:0/2524367879 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05040608f0 0x7f050413c910 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:32.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.387+0000 7f050a002700 1 --2- 172.21.15.162:0/2524367879 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f05040b6130 0x7f050413ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:32.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.387+0000 7f050a002700 1 -- 172.21.15.162:0/2524367879 >> 172.21.15.162:0/2524367879 conn(0x7f0504066c70 msgr2=0x7f0504067050 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:32.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.387+0000 7f050a002700 1 -- 172.21.15.162:0/2524367879 shutdown_connections 2020-11-28T21:46:32.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.387+0000 7f050a002700 1 -- 172.21.15.162:0/2524367879 wait complete. 2020-11-28T21:46:32.398 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.387+0000 7f050a002700 1 Processor -- start 2020-11-28T21:46:32.398 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.387+0000 7f050a002700 1 -- start start 2020-11-28T21:46:32.398 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.387+0000 7f050a002700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f05040608f0 0x7f05040b1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:32.398 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.387+0000 7f050a002700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f05040b54f0 0x7f0504155760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:32.399 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.387+0000 7f050a002700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05040b6130 0x7f0504139ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:32.399 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.387+0000 7f050a002700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0504154df0 con 0x7f05040b54f0 2020-11-28T21:46:32.399 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.387+0000 7f050a002700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05040b35f0 con 0x7f05040b6130 2020-11-28T21:46:32.399 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.387+0000 7f050a002700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f050413a6f0 con 0x7f05040608f0 2020-11-28T21:46:32.399 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.387+0000 7f0503fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05040b6130 0x7f0504139ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:32.400 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.387+0000 7f0503fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05040b6130 0x7f0504139ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37334/0 (socket says 172.21.15.162:37334) 2020-11-28T21:46:32.400 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.387+0000 7f0503fff700 1 -- 172.21.15.162:0/247687552 learned_addr learned my addr 172.21.15.162:0/247687552 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:32.400 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.387+0000 7f05037fe700 1 --2- 172.21.15.162:0/247687552 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f05040608f0 0x7f05040b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:32.400 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.387+0000 7f0502ffd700 1 --2- 172.21.15.162:0/247687552 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f05040b54f0 0x7f0504155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:32.400 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.391+0000 7f0503fff700 1 -- 172.21.15.162:0/247687552 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f05040608f0 msgr2=0x7f05040b1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:32.401 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.391+0000 7f0503fff700 1 --2- 172.21.15.162:0/247687552 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f05040608f0 0x7f05040b1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:32.401 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.391+0000 7f0503fff700 1 -- 172.21.15.162:0/247687552 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f05040b54f0 msgr2=0x7f0504155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:32.401 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.391+0000 7f0503fff700 1 --2- 172.21.15.162:0/247687552 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f05040b54f0 0x7f0504155760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:32.402 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.391+0000 7f0503fff700 1 -- 172.21.15.162:0/247687552 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0504155020 con 0x7f05040b6130 2020-11-28T21:46:32.402 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.391+0000 7f0503fff700 1 --2- 172.21.15.162:0/247687552 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05040b6130 0x7f0504139ec0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f04fc006910 tx=0x7f04fc008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:32.402 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.391+0000 7f0500ff9700 1 -- 172.21.15.162:0/247687552 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04fc0127e0 con 0x7f05040b6130 2020-11-28T21:46:32.402 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.391+0000 7f0500ff9700 1 -- 172.21.15.162:0/247687552 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04fc012940 con 0x7f05040b6130 2020-11-28T21:46:32.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.391+0000 7f050a002700 1 -- 172.21.15.162:0/247687552 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05041552c0 con 0x7f05040b6130 2020-11-28T21:46:32.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.391+0000 7f0500ff9700 1 -- 172.21.15.162:0/247687552 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04fc012ca0 con 0x7f05040b6130 2020-11-28T21:46:32.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.391+0000 7f050a002700 1 -- 172.21.15.162:0/247687552 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f050413a3b0 con 0x7f05040b6130 2020-11-28T21:46:32.404 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.391+0000 7f050a002700 1 -- 172.21.15.162:0/247687552 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0504155450 con 0x7f05040b6130 2020-11-28T21:46:32.404 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.391+0000 7f0500ff9700 1 -- 172.21.15.162:0/247687552 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f04fc013480 con 0x7f05040b6130 2020-11-28T21:46:32.405 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.391+0000 7f0500ff9700 1 --2- 172.21.15.162:0/247687552 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f04dc0553f0 0x7f04dc0578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:32.405 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.391+0000 7f0500ff9700 1 -- 172.21.15.162:0/247687552 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f04fc06e5d0 con 0x7f05040b6130 2020-11-28T21:46:32.405 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.395+0000 7f05037fe700 1 --2- 172.21.15.162:0/247687552 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f04dc0553f0 0x7f04dc0578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:32.405 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.395+0000 7f05037fe700 1 --2- 172.21.15.162:0/247687552 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f04dc0553f0 0x7f04dc0578b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f04f401a000 tx=0x7f04f4016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:32.405 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.399+0000 7f0500ff9700 1 -- 172.21.15.162:0/247687552 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f04fc0447d0 con 0x7f05040b6130 2020-11-28T21:46:32.575 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.571+0000 7f050a002700 1 -- 172.21.15.162:0/247687552 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f0504057050 con 0x7f05040b6130 2020-11-28T21:46:32.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.579+0000 7f0500ff9700 1 -- 172.21.15.162:0/247687552 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+133623 (secure 0 0 0) 0x7f04fc047df0 con 0x7f05040b6130 2020-11-28T21:46:32.583 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:46:32.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.583+0000 7f050a002700 1 -- 172.21.15.162:0/247687552 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f04dc0553f0 msgr2=0x7f04dc0578b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:32.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.583+0000 7f050a002700 1 --2- 172.21.15.162:0/247687552 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f04dc0553f0 0x7f04dc0578b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f04f401a000 tx=0x7f04f4016000).stop 2020-11-28T21:46:32.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.583+0000 7f050a002700 1 -- 172.21.15.162:0/247687552 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05040b6130 msgr2=0x7f0504139ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:32.586 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.583+0000 7f050a002700 1 --2- 172.21.15.162:0/247687552 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05040b6130 0x7f0504139ec0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f04fc006910 tx=0x7f04fc008040).stop 2020-11-28T21:46:32.586 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.583+0000 7f050a002700 1 -- 172.21.15.162:0/247687552 shutdown_connections 2020-11-28T21:46:32.586 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.583+0000 7f050a002700 1 --2- 172.21.15.162:0/247687552 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f04dc0553f0 0x7f04dc0578b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:32.586 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.583+0000 7f050a002700 1 --2- 172.21.15.162:0/247687552 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f05040608f0 0x7f05040b1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:32.586 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.583+0000 7f050a002700 1 --2- 172.21.15.162:0/247687552 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05040b6130 0x7f0504139ec0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:32.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.583+0000 7f050a002700 1 --2- 172.21.15.162:0/247687552 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f05040b54f0 0x7f0504155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:32.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.583+0000 7f050a002700 1 -- 172.21.15.162:0/247687552 >> 172.21.15.162:0/247687552 conn(0x7f0504066c70 msgr2=0x7f0504141690 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:32.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.583+0000 7f050a002700 1 -- 172.21.15.162:0/247687552 shutdown_connections 2020-11-28T21:46:32.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.583+0000 7f050a002700 1 -- 172.21.15.162:0/247687552 wait complete. 2020-11-28T21:46:32.601 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":5,"active_gid":4105,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":12156},{"type":"v1","addr":"172.21.15.162:6801","nonce":12156}]},"active_addr":"172.21.15.162:6801/12156","active_change":"2020-11-28T21:46:15.565586+0000","active_mgr_features":4540138292840890367,"available":true,"standbys":[{"gid":4107,"name":"x","mgr_features":4540138292840890367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading remoto library:No module named 'remoto'","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.6.2","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading remoto library:No module named 'remoto'","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.6.2","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.162:0","nonce":3474494274}]},{"addrvec":[{"type":"v2","addr":"172.21.15.162:0","nonce":1679707546}]},{"addrvec":[{"type":"v2","addr":"172.21.15.162:0","nonce":755316240}]}]}} 2020-11-28T21:46:32.605 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2020-11-28T21:46:32.605 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2020-11-28T21:46:32.606 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:46:32.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.715+0000 7f296b59d700 1 Processor -- start 2020-11-28T21:46:32.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.715+0000 7f296b59d700 1 -- start start 2020-11-28T21:46:32.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.715+0000 7f296b59d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f29640b1c20 0x7f29640b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:32.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.715+0000 7f296b59d700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f29640aabb0 0x7f2964067930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:32.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.715+0000 7f296b59d700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2964065b00 con 0x7f29640aa7d0 2020-11-28T21:46:32.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.715+0000 7f296b59d700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f296413e7f0 con 0x7f29640b1c20 2020-11-28T21:46:32.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.715+0000 7f296b59d700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2964066270 con 0x7f29640aabb0 2020-11-28T21:46:32.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.715+0000 7f2968b38700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f29640b1c20 0x7f29640b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:32.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.715+0000 7f2968b38700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f29640b1c20 0x7f29640b2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37340/0 (socket says 172.21.15.162:37340) 2020-11-28T21:46:32.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.715+0000 7f2969339700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f29640aa7d0 0x7f2964140ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.162:34330/0 (socket says 172.21.15.162:34330) 2020-11-28T21:46:32.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.715+0000 7f2968b38700 1 -- 172.21.15.162:0/3013493257 learned_addr learned my addr 172.21.15.162:0/3013493257 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:32.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.715+0000 7f2969b3a700 1 --2- 172.21.15.162:0/3013493257 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f29640aabb0 0x7f2964067930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:32.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.715+0000 7f2968b38700 1 -- 172.21.15.162:0/3013493257 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f29640aabb0 msgr2=0x7f2964067930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:32.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.715+0000 7f2963fff700 1 -- 172.21.15.162:0/3013493257 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2279737862 0 0) 0x7f2964065b00 con 0x7f29640aa7d0 2020-11-28T21:46:32.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.715+0000 7f2968b38700 1 --2- 172.21.15.162:0/3013493257 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f29640aabb0 0x7f2964067930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:32.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.715+0000 7f2968b38700 1 -- 172.21.15.162:0/3013493257 >> v1:172.21.15.118:6789/0 conn(0x7f29640aa7d0 legacy=0x7f2964140ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:32.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.715+0000 7f2968b38700 1 -- 172.21.15.162:0/3013493257 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2964067e20 con 0x7f29640b1c20 2020-11-28T21:46:32.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.715+0000 7f2969b3a700 1 --2- 172.21.15.162:0/3013493257 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f29640aabb0 0x7f2964067930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:46:32.723 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.715+0000 7f2968b38700 1 --2- 172.21.15.162:0/3013493257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29640b1c20 0x7f29640b2370 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f29640663b0 tx=0x7f295c006e20).ready entity=mon.1 client_cookie=856a89cb15510a13 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:32.724 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.715+0000 7f2963fff700 1 -- 172.21.15.162:0/3013493257 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f295c0097b0 con 0x7f29640b1c20 2020-11-28T21:46:32.724 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.715+0000 7f2963fff700 1 -- 172.21.15.162:0/3013493257 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f295c009910 con 0x7f29640b1c20 2020-11-28T21:46:32.724 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.715+0000 7f296b59d700 1 -- 172.21.15.162:0/3013493257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29640b1c20 msgr2=0x7f29640b2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:32.724 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.715+0000 7f2963fff700 1 -- 172.21.15.162:0/3013493257 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f295c009c70 con 0x7f29640b1c20 2020-11-28T21:46:32.725 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.715+0000 7f296b59d700 1 --2- 172.21.15.162:0/3013493257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29640b1c20 0x7f29640b2370 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f29640663b0 tx=0x7f295c006e20).stop 2020-11-28T21:46:32.725 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.715+0000 7f296b59d700 1 -- 172.21.15.162:0/3013493257 shutdown_connections 2020-11-28T21:46:32.725 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.715+0000 7f296b59d700 1 --2- 172.21.15.162:0/3013493257 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f29640aabb0 0x7f2964067930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:32.725 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.715+0000 7f296b59d700 1 --2- 172.21.15.162:0/3013493257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29640b1c20 0x7f29640b2370 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:32.725 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.715+0000 7f296b59d700 1 -- 172.21.15.162:0/3013493257 >> 172.21.15.162:0/3013493257 conn(0x7f2964062b50 msgr2=0x7f2964062f30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:32.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.715+0000 7f296b59d700 1 -- 172.21.15.162:0/3013493257 shutdown_connections 2020-11-28T21:46:32.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.715+0000 7f296b59d700 1 -- 172.21.15.162:0/3013493257 wait complete. 2020-11-28T21:46:32.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.719+0000 7f296b59d700 1 Processor -- start 2020-11-28T21:46:32.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.719+0000 7f296b59d700 1 -- start start 2020-11-28T21:46:32.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.719+0000 7f296b59d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29640aa7d0 0x7f2964060710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:32.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.719+0000 7f296b59d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f29640aabb0 0x7f29640ab7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:32.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.719+0000 7f296b59d700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f29640b1c20 0x7f29640abca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:32.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.719+0000 7f296b59d700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29640ad250 con 0x7f29640aabb0 2020-11-28T21:46:32.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.719+0000 7f296b59d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2964142770 con 0x7f29640aa7d0 2020-11-28T21:46:32.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.719+0000 7f296b59d700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2964131d70 con 0x7f29640b1c20 2020-11-28T21:46:32.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.719+0000 7f2969339700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29640aa7d0 0x7f2964060710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:32.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.719+0000 7f2969339700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29640aa7d0 0x7f2964060710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37344/0 (socket says 172.21.15.162:37344) 2020-11-28T21:46:32.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.719+0000 7f2969339700 1 -- 172.21.15.162:0/164038815 learned_addr learned my addr 172.21.15.162:0/164038815 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:32.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.719+0000 7f2969b3a700 1 --2- 172.21.15.162:0/164038815 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f29640b1c20 0x7f29640abca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:32.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.719+0000 7f2968b38700 1 --2- 172.21.15.162:0/164038815 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f29640aabb0 0x7f29640ab7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:32.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.719+0000 7f2969339700 1 -- 172.21.15.162:0/164038815 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f29640b1c20 msgr2=0x7f29640abca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:32.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.719+0000 7f2969339700 1 --2- 172.21.15.162:0/164038815 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f29640b1c20 0x7f29640abca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:32.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.719+0000 7f2969339700 1 -- 172.21.15.162:0/164038815 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f29640aabb0 msgr2=0x7f29640ab7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:32.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.719+0000 7f2969339700 1 --2- 172.21.15.162:0/164038815 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f29640aabb0 0x7f29640ab7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:32.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.719+0000 7f2969339700 1 -- 172.21.15.162:0/164038815 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29640ab090 con 0x7f29640aa7d0 2020-11-28T21:46:32.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.719+0000 7f2969339700 1 --2- 172.21.15.162:0/164038815 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29640aa7d0 0x7f2964060710 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f29640adef0 tx=0x7f2954007e10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:32.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.719+0000 7f29627fc700 1 -- 172.21.15.162:0/164038815 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2954005ab0 con 0x7f29640aa7d0 2020-11-28T21:46:32.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.719+0000 7f29627fc700 1 -- 172.21.15.162:0/164038815 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2954005c10 con 0x7f29640aa7d0 2020-11-28T21:46:32.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.719+0000 7f296b59d700 1 -- 172.21.15.162:0/164038815 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29640b3ba0 con 0x7f29640aa7d0 2020-11-28T21:46:32.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.719+0000 7f29627fc700 1 -- 172.21.15.162:0/164038815 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2954007110 con 0x7f29640aa7d0 2020-11-28T21:46:32.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.719+0000 7f296b59d700 1 -- 172.21.15.162:0/164038815 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29640adfa0 con 0x7f29640aa7d0 2020-11-28T21:46:32.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.723+0000 7f296b59d700 1 -- 172.21.15.162:0/164038815 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29640b0d30 con 0x7f29640aa7d0 2020-11-28T21:46:32.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.723+0000 7f29627fc700 1 -- 172.21.15.162:0/164038815 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2954010030 con 0x7f29640aa7d0 2020-11-28T21:46:32.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.723+0000 7f29627fc700 1 --2- 172.21.15.162:0/164038815 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f29440553b0 0x7f2944057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:32.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.723+0000 7f2968b38700 1 --2- 172.21.15.162:0/164038815 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f29440553b0 0x7f2944057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:32.733 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.723+0000 7f29627fc700 1 -- 172.21.15.162:0/164038815 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f295406dff0 con 0x7f29640aa7d0 2020-11-28T21:46:32.733 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.723+0000 7f2968b38700 1 --2- 172.21.15.162:0/164038815 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f29440553b0 0x7f2944057870 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f295c01b000 tx=0x7f295c015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:32.733 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.727+0000 7f29627fc700 1 -- 172.21.15.162:0/164038815 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f2954044200 con 0x7f29640aa7d0 2020-11-28T21:46:32.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.871+0000 7f296b59d700 1 -- 172.21.15.162:0/164038815 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2964057050 con 0x7f29640aa7d0 2020-11-28T21:46:32.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.875+0000 7f29627fc700 1 -- 172.21.15.162:0/164038815 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16134 (secure 0 0 0) 0x7f2954047820 con 0x7f29640aa7d0 2020-11-28T21:46:32.877 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:46:32.877 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":23,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:46:30.124354+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:46:27.155497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-28T21:46:28.137474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:46:32.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.875+0000 7f296b59d700 1 -- 172.21.15.162:0/164038815 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f29440553b0 msgr2=0x7f2944057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:32.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.875+0000 7f296b59d700 1 --2- 172.21.15.162:0/164038815 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f29440553b0 0x7f2944057870 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f295c01b000 tx=0x7f295c015000).stop 2020-11-28T21:46:32.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.875+0000 7f296b59d700 1 -- 172.21.15.162:0/164038815 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29640aa7d0 msgr2=0x7f2964060710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:32.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.875+0000 7f296b59d700 1 --2- 172.21.15.162:0/164038815 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29640aa7d0 0x7f2964060710 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f29640adef0 tx=0x7f2954007e10).stop 2020-11-28T21:46:32.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.875+0000 7f296b59d700 1 -- 172.21.15.162:0/164038815 shutdown_connections 2020-11-28T21:46:32.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.875+0000 7f296b59d700 1 --2- 172.21.15.162:0/164038815 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f29440553b0 0x7f2944057870 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:32.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.875+0000 7f296b59d700 1 --2- 172.21.15.162:0/164038815 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f29640b1c20 0x7f29640abca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:32.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.875+0000 7f296b59d700 1 --2- 172.21.15.162:0/164038815 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29640aa7d0 0x7f2964060710 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:32.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.875+0000 7f296b59d700 1 --2- 172.21.15.162:0/164038815 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f29640aabb0 0x7f29640ab7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:32.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.875+0000 7f296b59d700 1 -- 172.21.15.162:0/164038815 >> 172.21.15.162:0/164038815 conn(0x7f2964062b50 msgr2=0x7f2964062fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:32.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.875+0000 7f296b59d700 1 -- 172.21.15.162:0/164038815 shutdown_connections 2020-11-28T21:46:32.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.875+0000 7f296b59d700 1 -- 172.21.15.162:0/164038815 wait complete. 2020-11-28T21:46:32.894 INFO:tasks.ceph.ceph_manager.ceph:all up! 2020-11-28T21:46:32.895 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:46:33.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.999+0000 7f570dbc9700 1 Processor -- start 2020-11-28T21:46:33.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:32.999+0000 7f570dbc9700 1 -- start start 2020-11-28T21:46:33.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.003+0000 7f570dbc9700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f57080b7320 0x7f570813d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.003+0000 7f570dbc9700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57080b7700 con 0x7f57080b19f0 2020-11-28T21:46:33.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.003+0000 7f570dbc9700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5708067b50 con 0x7f57080b1f00 2020-11-28T21:46:33.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.003+0000 7f570dbc9700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5708067df0 con 0x7f57080b7320 2020-11-28T21:46:33.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.003+0000 7f5707fff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f57080b1f00 0x7f57080b2ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:44198/0 (socket says 172.21.15.162:44198) 2020-11-28T21:46:33.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.003+0000 7f5707fff700 1 -- 172.21.15.162:0/1403459273 learned_addr learned my addr 172.21.15.162:0/1403459273 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:33.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.003+0000 7f57077fe700 1 --2- 172.21.15.162:0/1403459273 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f57080b7320 0x7f570813d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.003+0000 7f57067fc700 1 -- 172.21.15.162:0/1403459273 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2579884443 0 0) 0x7f5708067b50 con 0x7f57080b1f00 2020-11-28T21:46:33.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.003+0000 7f57067fc700 1 -- 172.21.15.162:0/1403459273 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56ec003230 con 0x7f57080b1f00 2020-11-28T21:46:33.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.003+0000 7f57067fc700 1 -- 172.21.15.162:0/1403459273 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2215448909 0 0) 0x7f57080b7700 con 0x7f57080b19f0 2020-11-28T21:46:33.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.003+0000 7f57067fc700 1 -- 172.21.15.162:0/1403459273 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5708067b50 con 0x7f57080b19f0 2020-11-28T21:46:33.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.003+0000 7f57067fc700 1 -- 172.21.15.162:0/1403459273 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3120227581 0 0) 0x7f56ec003230 con 0x7f57080b1f00 2020-11-28T21:46:33.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.003+0000 7f57067fc700 1 -- 172.21.15.162:0/1403459273 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f57080b7320 msgr2=0x7f570813d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:33.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.003+0000 7f57067fc700 1 --2- 172.21.15.162:0/1403459273 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f57080b7320 0x7f570813d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:33.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.003+0000 7f57067fc700 1 -- 172.21.15.162:0/1403459273 >> v1:172.21.15.118:6789/0 conn(0x7f57080b19f0 legacy=0x7f57080b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:33.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.003+0000 7f57067fc700 1 -- 172.21.15.162:0/1403459273 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5708065c80 con 0x7f57080b1f00 2020-11-28T21:46:33.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.003+0000 7f57067fc700 1 -- 172.21.15.162:0/1403459273 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f5700001a50 con 0x7f57080b1f00 2020-11-28T21:46:33.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.003+0000 7f57067fc700 1 -- 172.21.15.162:0/1403459273 >> v1:172.21.15.162:6789/0 conn(0x7f57080b1f00 legacy=0x7f57080b2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:33.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.003+0000 7f57067fc700 1 --2- 172.21.15.162:0/1403459273 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f57080b7320 0x7f56ec0038b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.003+0000 7f57067fc700 1 --2- 172.21.15.162:0/1403459273 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f56ec003df0 0x7f56ec0044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.003+0000 7f57067fc700 1 --2- 172.21.15.162:0/1403459273 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56ec0049c0 0x7f56ec004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.003+0000 7f57067fc700 1 -- 172.21.15.162:0/1403459273 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5708067df0 con 0x7f56ec003df0 2020-11-28T21:46:33.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.003+0000 7f57067fc700 1 -- 172.21.15.162:0/1403459273 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec005390 con 0x7f56ec0049c0 2020-11-28T21:46:33.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.003+0000 7f57067fc700 1 -- 172.21.15.162:0/1403459273 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec00b7a0 con 0x7f57080b7320 2020-11-28T21:46:33.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.003+0000 7f5707fff700 1 --2- 172.21.15.162:0/1403459273 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56ec0049c0 0x7f56ec004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.003+0000 7f57077fe700 1 --2- 172.21.15.162:0/1403459273 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f57080b7320 0x7f56ec0038b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.003+0000 7f5706ffd700 1 --2- 172.21.15.162:0/1403459273 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f56ec003df0 0x7f56ec0044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.003+0000 7f5707fff700 1 -- 172.21.15.162:0/1403459273 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f57080b7320 msgr2=0x7f56ec0038b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:33.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.003+0000 7f5707fff700 1 --2- 172.21.15.162:0/1403459273 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f57080b7320 0x7f56ec0038b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:33.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.003+0000 7f5707fff700 1 -- 172.21.15.162:0/1403459273 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f56ec003df0 msgr2=0x7f56ec0044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:33.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.003+0000 7f5707fff700 1 --2- 172.21.15.162:0/1403459273 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f56ec003df0 0x7f56ec0044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:33.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.003+0000 7f5707fff700 1 -- 172.21.15.162:0/1403459273 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ec003230 con 0x7f56ec0049c0 2020-11-28T21:46:33.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.003+0000 7f5707fff700 1 --2- 172.21.15.162:0/1403459273 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56ec0049c0 0x7f56ec004e40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f5700001820 tx=0x7f5700007a10).ready entity=mon.1 client_cookie=68f028593a12e89c server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:33.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.003+0000 7f57067fc700 1 -- 172.21.15.162:0/1403459273 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f570000ce20 con 0x7f56ec0049c0 2020-11-28T21:46:33.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.003+0000 7f57067fc700 1 -- 172.21.15.162:0/1403459273 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5700005b70 con 0x7f56ec0049c0 2020-11-28T21:46:33.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.003+0000 7f57067fc700 1 -- 172.21.15.162:0/1403459273 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57000070d0 con 0x7f56ec0049c0 2020-11-28T21:46:33.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.003+0000 7f570dbc9700 1 -- 172.21.15.162:0/1403459273 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56ec0049c0 msgr2=0x7f56ec004e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:33.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.003+0000 7f570dbc9700 1 --2- 172.21.15.162:0/1403459273 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56ec0049c0 0x7f56ec004e40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f5700001820 tx=0x7f5700007a10).stop 2020-11-28T21:46:33.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.003+0000 7f5707fff700 1 -- 172.21.15.162:0/1403459273 reap_dead start 2020-11-28T21:46:33.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.007+0000 7f570dbc9700 1 -- 172.21.15.162:0/1403459273 shutdown_connections 2020-11-28T21:46:33.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.007+0000 7f570dbc9700 1 -- 172.21.15.162:0/1403459273 >> 172.21.15.162:0/1403459273 conn(0x7f5708062830 msgr2=0x7f5708062c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:33.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.007+0000 7f570dbc9700 1 -- 172.21.15.162:0/1403459273 shutdown_connections 2020-11-28T21:46:33.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.007+0000 7f570dbc9700 1 -- 172.21.15.162:0/1403459273 wait complete. 2020-11-28T21:46:33.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.007+0000 7f570dbc9700 1 Processor -- start 2020-11-28T21:46:33.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.007+0000 7f570dbc9700 1 -- start start 2020-11-28T21:46:33.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.011+0000 7f570dbc9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57081533a0 0x7f57080afce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.011+0000 7f570dbc9700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5708141c30 0x7f57080af530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.011+0000 7f570dbc9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f57081391a0 0x7f5708139630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.011+0000 7f570dbc9700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57080b0970 con 0x7f57081391a0 2020-11-28T21:46:33.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.011+0000 7f570dbc9700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5708143990 con 0x7f57081533a0 2020-11-28T21:46:33.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.011+0000 7f570dbc9700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5708144440 con 0x7f5708141c30 2020-11-28T21:46:33.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.011+0000 7f57077fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57081533a0 0x7f57080afce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.011+0000 7f57077fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57081533a0 0x7f57080afce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37364/0 (socket says 172.21.15.162:37364) 2020-11-28T21:46:33.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.011+0000 7f57077fe700 1 -- 172.21.15.162:0/1351068229 learned_addr learned my addr 172.21.15.162:0/1351068229 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:33.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.011+0000 7f5706ffd700 1 --2- 172.21.15.162:0/1351068229 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5708141c30 0x7f57080af530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.011+0000 7f5707fff700 1 --2- 172.21.15.162:0/1351068229 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f57081391a0 0x7f5708139630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.011+0000 7f57077fe700 1 -- 172.21.15.162:0/1351068229 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5708141c30 msgr2=0x7f57080af530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:33.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.011+0000 7f57077fe700 1 --2- 172.21.15.162:0/1351068229 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5708141c30 0x7f57080af530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:33.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.011+0000 7f57077fe700 1 -- 172.21.15.162:0/1351068229 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f57081391a0 msgr2=0x7f5708139630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:33.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.011+0000 7f57077fe700 1 --2- 172.21.15.162:0/1351068229 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f57081391a0 0x7f5708139630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:33.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.011+0000 7f57077fe700 1 -- 172.21.15.162:0/1351068229 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5708153ac0 con 0x7f57081533a0 2020-11-28T21:46:33.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.011+0000 7f57077fe700 1 --2- 172.21.15.162:0/1351068229 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57081533a0 0x7f57080afce0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f56f8006750 tx=0x7f56f8008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:33.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.011+0000 7f5704ff9700 1 -- 172.21.15.162:0/1351068229 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56f800f980 con 0x7f57081533a0 2020-11-28T21:46:33.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.011+0000 7f5704ff9700 1 -- 172.21.15.162:0/1351068229 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f800fae0 con 0x7f57081533a0 2020-11-28T21:46:33.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.011+0000 7f570dbc9700 1 -- 172.21.15.162:0/1351068229 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5708153d60 con 0x7f57081533a0 2020-11-28T21:46:33.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.011+0000 7f5704ff9700 1 -- 172.21.15.162:0/1351068229 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56f800fc50 con 0x7f57081533a0 2020-11-28T21:46:33.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.011+0000 7f570dbc9700 1 -- 172.21.15.162:0/1351068229 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57081420b0 con 0x7f57081533a0 2020-11-28T21:46:33.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.011+0000 7f570dbc9700 1 -- 172.21.15.162:0/1351068229 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5708153920 con 0x7f57081533a0 2020-11-28T21:46:33.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.011+0000 7f5704ff9700 1 -- 172.21.15.162:0/1351068229 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f56f8024070 con 0x7f57081533a0 2020-11-28T21:46:33.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.015+0000 7f5704ff9700 1 --2- 172.21.15.162:0/1351068229 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f56e00554e0 0x7f56e00579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.015+0000 7f5704ff9700 1 -- 172.21.15.162:0/1351068229 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f56f806e850 con 0x7f57081533a0 2020-11-28T21:46:33.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.015+0000 7f5706ffd700 1 --2- 172.21.15.162:0/1351068229 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f56e00554e0 0x7f56e00579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.015+0000 7f5706ffd700 1 --2- 172.21.15.162:0/1351068229 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f56e00554e0 0x7f56e00579a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f56f0003000 tx=0x7f56f0007b60).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:33.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.019+0000 7f5704ff9700 1 -- 172.21.15.162:0/1351068229 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5708153920 con 0x7f57081533a0 2020-11-28T21:46:33.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.159+0000 7f570dbc9700 1 -- 172.21.15.162:0/1351068229 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5708153920 con 0x7f57081533a0 2020-11-28T21:46:33.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.163+0000 7f5704ff9700 1 -- 172.21.15.162:0/1351068229 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16134 (secure 0 0 0) 0x7f5708153920 con 0x7f57081533a0 2020-11-28T21:46:33.164 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:46:33.165 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":23,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:46:30.124354+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:46:27.155497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-28T21:46:28.137474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:46:33.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.163+0000 7f570dbc9700 1 -- 172.21.15.162:0/1351068229 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f56e00554e0 msgr2=0x7f56e00579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:33.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.163+0000 7f570dbc9700 1 --2- 172.21.15.162:0/1351068229 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f56e00554e0 0x7f56e00579a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f56f0003000 tx=0x7f56f0007b60).stop 2020-11-28T21:46:33.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.163+0000 7f570dbc9700 1 -- 172.21.15.162:0/1351068229 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57081533a0 msgr2=0x7f57080afce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:33.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.163+0000 7f570dbc9700 1 --2- 172.21.15.162:0/1351068229 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57081533a0 0x7f57080afce0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f56f8006750 tx=0x7f56f8008040).stop 2020-11-28T21:46:33.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.163+0000 7f570dbc9700 1 -- 172.21.15.162:0/1351068229 shutdown_connections 2020-11-28T21:46:33.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.163+0000 7f570dbc9700 1 --2- 172.21.15.162:0/1351068229 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f56e00554e0 0x7f56e00579a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:33.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.163+0000 7f570dbc9700 1 --2- 172.21.15.162:0/1351068229 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f57081391a0 0x7f5708139630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:33.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.163+0000 7f570dbc9700 1 --2- 172.21.15.162:0/1351068229 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57081533a0 0x7f57080afce0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:33.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.163+0000 7f570dbc9700 1 --2- 172.21.15.162:0/1351068229 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5708141c30 0x7f57080af530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:33.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.163+0000 7f570dbc9700 1 -- 172.21.15.162:0/1351068229 >> 172.21.15.162:0/1351068229 conn(0x7f5708062830 msgr2=0x7f57080b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:33.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.163+0000 7f570dbc9700 1 -- 172.21.15.162:0/1351068229 shutdown_connections 2020-11-28T21:46:33.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.163+0000 7f570dbc9700 1 -- 172.21.15.162:0/1351068229 wait complete. 2020-11-28T21:46:33.185 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.0 flush_pg_stats 2020-11-28T21:46:33.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.295+0000 7fbd09e73700 1 Processor -- start 2020-11-28T21:46:33.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.295+0000 7fbd09e73700 1 -- start start 2020-11-28T21:46:33.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.295+0000 7fbd09e73700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbd04062cd0 0x7fbd041406c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.295+0000 7fbd09e73700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbd040664a0 0x7fbd040668e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.295+0000 7fbd09e73700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd04140bb0 con 0x7fbd040b3a50 2020-11-28T21:46:33.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.295+0000 7fbd09e73700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd04144bf0 con 0x7fbd04062cd0 2020-11-28T21:46:33.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.295+0000 7fbd09e73700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd040b3230 con 0x7fbd040664a0 2020-11-28T21:46:33.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.295+0000 7fbd08e71700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fbd040b3a50 0x7fbd041444e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:44216/0 (socket says 172.21.15.162:44216) 2020-11-28T21:46:33.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.295+0000 7fbd08e71700 1 -- 172.21.15.162:0/748144593 learned_addr learned my addr 172.21.15.162:0/748144593 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:33.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.295+0000 7fbd03fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbd04062cd0 0x7fbd041406c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.295+0000 7fbd09672700 1 --2- 172.21.15.162:0/748144593 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbd040664a0 0x7fbd040668e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.295+0000 7fbd037fe700 1 -- 172.21.15.162:0/748144593 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3547649050 0 0) 0x7fbcf4001450 con 0x7fbd040b3a50 2020-11-28T21:46:33.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.295+0000 7fbd037fe700 1 -- 172.21.15.162:0/748144593 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbce4003230 con 0x7fbd040b3a50 2020-11-28T21:46:33.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.295+0000 7fbd03fff700 1 -- 172.21.15.162:0/748144593 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbd040664a0 msgr2=0x7fbd040668e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:33.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.295+0000 7fbd03fff700 1 --2- 172.21.15.162:0/748144593 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbd040664a0 0x7fbd040668e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:33.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.295+0000 7fbd03fff700 1 -- 172.21.15.162:0/748144593 >> v1:172.21.15.162:6789/0 conn(0x7fbd040b3a50 legacy=0x7fbd041444e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:33.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.295+0000 7fbd03fff700 1 -- 172.21.15.162:0/748144593 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd04143da0 con 0x7fbd04062cd0 2020-11-28T21:46:33.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.295+0000 7fbd09672700 1 --2- 172.21.15.162:0/748144593 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbd040664a0 0x7fbd040668e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:46:33.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.295+0000 7fbd03fff700 1 --2- 172.21.15.162:0/748144593 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd04062cd0 0x7fbd041406c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fbcec002f30 tx=0x7fbcec006dc0).ready entity=mon.1 client_cookie=3341d14e7f226411 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:33.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.295+0000 7fbd037fe700 1 -- 172.21.15.162:0/748144593 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbcec009760 con 0x7fbd04062cd0 2020-11-28T21:46:33.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.295+0000 7fbd037fe700 1 -- 172.21.15.162:0/748144593 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcec0098c0 con 0x7fbd04062cd0 2020-11-28T21:46:33.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.295+0000 7fbd037fe700 1 -- 172.21.15.162:0/748144593 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbcec009c20 con 0x7fbd04062cd0 2020-11-28T21:46:33.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.295+0000 7fbd09e73700 1 -- 172.21.15.162:0/748144593 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd04062cd0 msgr2=0x7fbd041406c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:33.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.295+0000 7fbd09e73700 1 --2- 172.21.15.162:0/748144593 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd04062cd0 0x7fbd041406c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fbcec002f30 tx=0x7fbcec006dc0).stop 2020-11-28T21:46:33.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.295+0000 7fbd09e73700 1 -- 172.21.15.162:0/748144593 shutdown_connections 2020-11-28T21:46:33.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.295+0000 7fbd09e73700 1 --2- 172.21.15.162:0/748144593 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbd040664a0 0x7fbd040668e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:33.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.295+0000 7fbd09e73700 1 --2- 172.21.15.162:0/748144593 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd04062cd0 0x7fbd041406c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:33.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.295+0000 7fbd09e73700 1 -- 172.21.15.162:0/748144593 >> 172.21.15.162:0/748144593 conn(0x7fbd0405b970 msgr2=0x7fbd0405bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:33.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.295+0000 7fbd09e73700 1 -- 172.21.15.162:0/748144593 shutdown_connections 2020-11-28T21:46:33.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.295+0000 7fbd09e73700 1 -- 172.21.15.162:0/748144593 wait complete. 2020-11-28T21:46:33.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.299+0000 7fbd09e73700 1 Processor -- start 2020-11-28T21:46:33.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.299+0000 7fbd09e73700 1 -- start start 2020-11-28T21:46:33.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.299+0000 7fbd09e73700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbd04062cd0 0x7fbd04145000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.299+0000 7fbd09e73700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd040664a0 0x7fbd040b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.299+0000 7fbd09e73700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbd040b3a50 0x7fbd041472b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.299+0000 7fbd09e73700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd040b06e0 con 0x7fbd04062cd0 2020-11-28T21:46:33.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.299+0000 7fbd09e73700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd04146b70 con 0x7fbd040664a0 2020-11-28T21:46:33.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.299+0000 7fbd09e73700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd04146e10 con 0x7fbd040b3a50 2020-11-28T21:46:33.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.299+0000 7fbd03fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd040664a0 0x7fbd040b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.299+0000 7fbd03fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd040664a0 0x7fbd040b27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37382/0 (socket says 172.21.15.162:37382) 2020-11-28T21:46:33.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.299+0000 7fbd03fff700 1 -- 172.21.15.162:0/1156893236 learned_addr learned my addr 172.21.15.162:0/1156893236 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:33.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.299+0000 7fbd09672700 1 --2- 172.21.15.162:0/1156893236 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbd040b3a50 0x7fbd041472b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.299+0000 7fbd08e71700 1 --2- 172.21.15.162:0/1156893236 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbd04062cd0 0x7fbd04145000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.299+0000 7fbd03fff700 1 -- 172.21.15.162:0/1156893236 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbd040b3a50 msgr2=0x7fbd041472b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:33.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.299+0000 7fbd03fff700 1 --2- 172.21.15.162:0/1156893236 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbd040b3a50 0x7fbd041472b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:33.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.299+0000 7fbd03fff700 1 -- 172.21.15.162:0/1156893236 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbd04062cd0 msgr2=0x7fbd04145000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:33.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.299+0000 7fbd03fff700 1 --2- 172.21.15.162:0/1156893236 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbd04062cd0 0x7fbd04145000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:33.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.299+0000 7fbd03fff700 1 -- 172.21.15.162:0/1156893236 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd04149860 con 0x7fbd040664a0 2020-11-28T21:46:33.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.299+0000 7fbd08e71700 1 --2- 172.21.15.162:0/1156893236 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbd04062cd0 0x7fbd04145000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:46:33.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.299+0000 7fbd03fff700 1 --2- 172.21.15.162:0/1156893236 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd040664a0 0x7fbd040b27d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fbcec002f30 tx=0x7fbcec015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:33.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.299+0000 7fbd01ffb700 1 -- 172.21.15.162:0/1156893236 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbcec011980 con 0x7fbd040664a0 2020-11-28T21:46:33.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.299+0000 7fbd01ffb700 1 -- 172.21.15.162:0/1156893236 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcec011ae0 con 0x7fbd040664a0 2020-11-28T21:46:33.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.299+0000 7fbd01ffb700 1 -- 172.21.15.162:0/1156893236 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbcec011e40 con 0x7fbd040664a0 2020-11-28T21:46:33.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.299+0000 7fbd09e73700 1 -- 172.21.15.162:0/1156893236 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd04141ac0 con 0x7fbd040664a0 2020-11-28T21:46:33.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.299+0000 7fbd09e73700 1 -- 172.21.15.162:0/1156893236 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd041479c0 con 0x7fbd040664a0 2020-11-28T21:46:33.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.303+0000 7fbd09e73700 1 -- 172.21.15.162:0/1156893236 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbd040b08a0 con 0x7fbd040664a0 2020-11-28T21:46:33.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.303+0000 7fbd01ffb700 1 -- 172.21.15.162:0/1156893236 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fbcec024030 con 0x7fbd040664a0 2020-11-28T21:46:33.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.303+0000 7fbd01ffb700 1 --2- 172.21.15.162:0/1156893236 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbce8055350 0x7fbce8057810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.303+0000 7fbd01ffb700 1 -- 172.21.15.162:0/1156893236 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbcec06eef0 con 0x7fbd040664a0 2020-11-28T21:46:33.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.303+0000 7fbd08e71700 1 --2- 172.21.15.162:0/1156893236 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbce8055350 0x7fbce8057810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.303+0000 7fbd01ffb700 1 --2- 172.21.15.162:0/1156893236 >> [v2:172.21.15.162:6810/12313,v1:172.21.15.162:6811/12313] conn(0x7fbce805c7e0 0x7fbce805ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.303+0000 7fbd01ffb700 1 -- 172.21.15.162:0/1156893236 --> [v2:172.21.15.162:6810/12313,v1:172.21.15.162:6811/12313] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbce805f260 con 0x7fbce805c7e0 2020-11-28T21:46:33.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.303+0000 7fbd01ffb700 1 -- 172.21.15.162:0/1156893236 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbcec071410 con 0x7fbd040664a0 2020-11-28T21:46:33.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.303+0000 7fbd09672700 1 --2- 172.21.15.162:0/1156893236 >> [v2:172.21.15.162:6810/12313,v1:172.21.15.162:6811/12313] conn(0x7fbce805c7e0 0x7fbce805ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.303+0000 7fbd08e71700 1 --2- 172.21.15.162:0/1156893236 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbce8055350 0x7fbce8057810 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fbcf400b000 tx=0x7fbcf4009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:33.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.307+0000 7fbd09672700 1 --2- 172.21.15.162:0/1156893236 >> [v2:172.21.15.162:6810/12313,v1:172.21.15.162:6811/12313] conn(0x7fbce805c7e0 0x7fbce805ec00 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:33.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.307+0000 7fbd01ffb700 1 -- 172.21.15.162:0/1156893236 <== osd.0 v2:172.21.15.162:6810/12313 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7fbce805f260 con 0x7fbce805c7e0 2020-11-28T21:46:33.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.327+0000 7fbd09e73700 1 -- 172.21.15.162:0/1156893236 --> [v2:172.21.15.162:6810/12313,v1:172.21.15.162:6811/12313] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbd041469f0 con 0x7fbce805c7e0 2020-11-28T21:46:33.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.331+0000 7fbd01ffb700 1 -- 172.21.15.162:0/1156893236 <== osd.0 v2:172.21.15.162:6810/12313 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbcfc00ba90 con 0x7fbce805c7e0 2020-11-28T21:46:33.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.331+0000 7fbd09e73700 1 -- 172.21.15.162:0/1156893236 >> [v2:172.21.15.162:6810/12313,v1:172.21.15.162:6811/12313] conn(0x7fbce805c7e0 msgr2=0x7fbce805ec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:33.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.331+0000 7fbd09e73700 1 --2- 172.21.15.162:0/1156893236 >> [v2:172.21.15.162:6810/12313,v1:172.21.15.162:6811/12313] conn(0x7fbce805c7e0 0x7fbce805ec00 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:33.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.331+0000 7fbd09e73700 1 -- 172.21.15.162:0/1156893236 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbce8055350 msgr2=0x7fbce8057810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:33.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.331+0000 7fbd09e73700 1 --2- 172.21.15.162:0/1156893236 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbce8055350 0x7fbce8057810 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fbcf400b000 tx=0x7fbcf4009040).stop 2020-11-28T21:46:33.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.331+0000 7fbd09e73700 1 -- 172.21.15.162:0/1156893236 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd040664a0 msgr2=0x7fbd040b27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:33.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.331+0000 7fbd09e73700 1 --2- 172.21.15.162:0/1156893236 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd040664a0 0x7fbd040b27d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fbcec002f30 tx=0x7fbcec015000).stop 2020-11-28T21:46:33.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.331+0000 7fbd09672700 1 -- 172.21.15.162:0/1156893236 reap_dead start 2020-11-28T21:46:33.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.331+0000 7fbd09e73700 1 -- 172.21.15.162:0/1156893236 shutdown_connections 2020-11-28T21:46:33.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.331+0000 7fbd09e73700 1 -- 172.21.15.162:0/1156893236 >> 172.21.15.162:0/1156893236 conn(0x7fbd0405b970 msgr2=0x7fbd040b19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:33.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.331+0000 7fbd09e73700 1 -- 172.21.15.162:0/1156893236 shutdown_connections 2020-11-28T21:46:33.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.331+0000 7fbd09e73700 1 -- 172.21.15.162:0/1156893236 wait complete. 2020-11-28T21:46:33.344 INFO:teuthology.orchestra.run.smithi162.stdout:60129542148 2020-11-28T21:46:33.345 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.1 flush_pg_stats 2020-11-28T21:46:33.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.459+0000 7f6090a39700 1 Processor -- start 2020-11-28T21:46:33.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.459+0000 7f6090a39700 1 -- start start 2020-11-28T21:46:33.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.459+0000 7f6090a39700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f608c0b3290 0x7f608c0b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.459+0000 7f6090a39700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f608c1467f0 0x7f608c1429d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.459+0000 7f6090a39700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f608c0b3670 con 0x7f608c142ec0 2020-11-28T21:46:33.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.459+0000 7f6090a39700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f608c0b0850 con 0x7f608c1467f0 2020-11-28T21:46:33.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.459+0000 7f6090a39700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f608c0b0af0 con 0x7f608c0b3290 2020-11-28T21:46:33.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.459+0000 7f608affd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f608c1467f0 0x7f608c1429d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.459+0000 7f608bfff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f608c142ec0 0x7f608c05b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:44234/0 (socket says 172.21.15.162:44234) 2020-11-28T21:46:33.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.459+0000 7f608bfff700 1 -- 172.21.15.162:0/2388643850 learned_addr learned my addr 172.21.15.162:0/2388643850 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:33.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.459+0000 7f608b7fe700 1 --2- 172.21.15.162:0/2388643850 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f608c0b3290 0x7f608c0b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.459+0000 7f608affd700 1 -- 172.21.15.162:0/2388643850 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f608c0b3290 msgr2=0x7f608c0b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:33.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.459+0000 7f608a7fc700 1 -- 172.21.15.162:0/2388643850 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1682427210 0 0) 0x7f608c0b3670 con 0x7f608c142ec0 2020-11-28T21:46:33.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.459+0000 7f608affd700 1 --2- 172.21.15.162:0/2388643850 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f608c0b3290 0x7f608c0b14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:33.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.459+0000 7f608affd700 1 -- 172.21.15.162:0/2388643850 >> v1:172.21.15.162:6789/0 conn(0x7f608c142ec0 legacy=0x7f608c05b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:33.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.459+0000 7f608affd700 1 -- 172.21.15.162:0/2388643850 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f608c0b0d90 con 0x7f608c1467f0 2020-11-28T21:46:33.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.459+0000 7f608affd700 1 --2- 172.21.15.162:0/2388643850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f608c1467f0 0x7f608c1429d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f6074002f30 tx=0x7f6074006dc0).ready entity=mon.1 client_cookie=72128a54f62738d3 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:33.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.459+0000 7f608a7fc700 1 -- 172.21.15.162:0/2388643850 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6074009760 con 0x7f608c1467f0 2020-11-28T21:46:33.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.459+0000 7f608a7fc700 1 -- 172.21.15.162:0/2388643850 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60740098c0 con 0x7f608c1467f0 2020-11-28T21:46:33.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.459+0000 7f608a7fc700 1 -- 172.21.15.162:0/2388643850 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6074009c20 con 0x7f608c1467f0 2020-11-28T21:46:33.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.459+0000 7f6090a39700 1 -- 172.21.15.162:0/2388643850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f608c1467f0 msgr2=0x7f608c1429d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:33.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.459+0000 7f6090a39700 1 --2- 172.21.15.162:0/2388643850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f608c1467f0 0x7f608c1429d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f6074002f30 tx=0x7f6074006dc0).stop 2020-11-28T21:46:33.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.463+0000 7f6090a39700 1 -- 172.21.15.162:0/2388643850 shutdown_connections 2020-11-28T21:46:33.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.463+0000 7f6090a39700 1 --2- 172.21.15.162:0/2388643850 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f608c0b3290 0x7f608c0b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:33.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.463+0000 7f6090a39700 1 --2- 172.21.15.162:0/2388643850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f608c1467f0 0x7f608c1429d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:33.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.463+0000 7f6090a39700 1 -- 172.21.15.162:0/2388643850 >> 172.21.15.162:0/2388643850 conn(0x7f608c0b6870 msgr2=0x7f608c0b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:33.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.463+0000 7f6090a39700 1 -- 172.21.15.162:0/2388643850 shutdown_connections 2020-11-28T21:46:33.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.463+0000 7f6090a39700 1 -- 172.21.15.162:0/2388643850 wait complete. 2020-11-28T21:46:33.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.463+0000 7f6090a39700 1 Processor -- start 2020-11-28T21:46:33.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.463+0000 7f6090a39700 1 -- start start 2020-11-28T21:46:33.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.463+0000 7f6090a39700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f608c0b3290 0x7f608c13e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.463+0000 7f6090a39700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f608c142ec0 0x7f608c1515e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.463+0000 7f6090a39700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f608c1467f0 0x7f608c151ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.463+0000 7f6090a39700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608c13bac0 con 0x7f608c0b3290 2020-11-28T21:46:33.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.463+0000 7f6090a39700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608c148f30 con 0x7f608c1467f0 2020-11-28T21:46:33.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.463+0000 7f6090a39700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f608c149190 con 0x7f608c142ec0 2020-11-28T21:46:33.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.463+0000 7f608bfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f608c1467f0 0x7f608c151ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.463+0000 7f608bfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f608c1467f0 0x7f608c151ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37398/0 (socket says 172.21.15.162:37398) 2020-11-28T21:46:33.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.463+0000 7f608bfff700 1 -- 172.21.15.162:0/3434250193 learned_addr learned my addr 172.21.15.162:0/3434250193 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:33.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.463+0000 7f608affd700 1 --2- 172.21.15.162:0/3434250193 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f608c142ec0 0x7f608c1515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.463+0000 7f608b7fe700 1 --2- 172.21.15.162:0/3434250193 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f608c0b3290 0x7f608c13e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.463+0000 7f608bfff700 1 -- 172.21.15.162:0/3434250193 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f608c142ec0 msgr2=0x7f608c1515e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:33.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.463+0000 7f608bfff700 1 --2- 172.21.15.162:0/3434250193 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f608c142ec0 0x7f608c1515e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:33.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.463+0000 7f608bfff700 1 -- 172.21.15.162:0/3434250193 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f608c0b3290 msgr2=0x7f608c13e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:33.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.463+0000 7f608bfff700 1 --2- 172.21.15.162:0/3434250193 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f608c0b3290 0x7f608c13e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:33.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.463+0000 7f608bfff700 1 -- 172.21.15.162:0/3434250193 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f608c152350 con 0x7f608c1467f0 2020-11-28T21:46:33.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.463+0000 7f608b7fe700 1 --2- 172.21.15.162:0/3434250193 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f608c0b3290 0x7f608c13e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:46:33.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.467+0000 7f608bfff700 1 --2- 172.21.15.162:0/3434250193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f608c1467f0 0x7f608c151ad0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f608c0645c0 tx=0x7f6084007d70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:33.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.467+0000 7f6088ff9700 1 -- 172.21.15.162:0/3434250193 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6084005c40 con 0x7f608c1467f0 2020-11-28T21:46:33.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.467+0000 7f6088ff9700 1 -- 172.21.15.162:0/3434250193 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6084005da0 con 0x7f608c1467f0 2020-11-28T21:46:33.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.467+0000 7f6090a39700 1 -- 172.21.15.162:0/3434250193 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f608c13e9b0 con 0x7f608c1467f0 2020-11-28T21:46:33.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.467+0000 7f6088ff9700 1 -- 172.21.15.162:0/3434250193 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6084007300 con 0x7f608c1467f0 2020-11-28T21:46:33.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.467+0000 7f6090a39700 1 -- 172.21.15.162:0/3434250193 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f608c13da70 con 0x7f608c1467f0 2020-11-28T21:46:33.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.467+0000 7f6090a39700 1 -- 172.21.15.162:0/3434250193 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f608c1521e0 con 0x7f608c1467f0 2020-11-28T21:46:33.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.467+0000 7f6088ff9700 1 -- 172.21.15.162:0/3434250193 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6084010030 con 0x7f608c1467f0 2020-11-28T21:46:33.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.467+0000 7f6088ff9700 1 --2- 172.21.15.162:0/3434250193 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6070055330 0x7f60700577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.467+0000 7f6088ff9700 1 -- 172.21.15.162:0/3434250193 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f608406e900 con 0x7f608c1467f0 2020-11-28T21:46:33.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.467+0000 7f608b7fe700 1 --2- 172.21.15.162:0/3434250193 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6070055330 0x7f60700577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.467+0000 7f6088ff9700 1 --2- 172.21.15.162:0/3434250193 >> [v2:172.21.15.162:6802/12311,v1:172.21.15.162:6803/12311] conn(0x7f607005c740 0x7f607005eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.467+0000 7f6088ff9700 1 -- 172.21.15.162:0/3434250193 --> [v2:172.21.15.162:6802/12311,v1:172.21.15.162:6803/12311] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f607005f1c0 con 0x7f607005c740 2020-11-28T21:46:33.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.467+0000 7f6088ff9700 1 -- 172.21.15.162:0/3434250193 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f608406ecc0 con 0x7f608c1467f0 2020-11-28T21:46:33.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.467+0000 7f608affd700 1 --2- 172.21.15.162:0/3434250193 >> [v2:172.21.15.162:6802/12311,v1:172.21.15.162:6803/12311] conn(0x7f607005c740 0x7f607005eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.467+0000 7f608b7fe700 1 --2- 172.21.15.162:0/3434250193 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6070055330 0x7f60700577f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f607c00b000 tx=0x7f607c008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:33.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.467+0000 7f608affd700 1 --2- 172.21.15.162:0/3434250193 >> [v2:172.21.15.162:6802/12311,v1:172.21.15.162:6803/12311] conn(0x7f607005c740 0x7f607005eb60 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:33.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.471+0000 7f6088ff9700 1 -- 172.21.15.162:0/3434250193 <== osd.1 v2:172.21.15.162:6802/12311 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7f607005f1c0 con 0x7f607005c740 2020-11-28T21:46:33.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.491+0000 7f6090a39700 1 -- 172.21.15.162:0/3434250193 --> [v2:172.21.15.162:6802/12311,v1:172.21.15.162:6803/12311] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f608c1479d0 con 0x7f607005c740 2020-11-28T21:46:33.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.491+0000 7f6088ff9700 1 -- 172.21.15.162:0/3434250193 <== osd.1 v2:172.21.15.162:6802/12311 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f607400b5e0 con 0x7f607005c740 2020-11-28T21:46:33.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.491+0000 7f6090a39700 1 -- 172.21.15.162:0/3434250193 >> [v2:172.21.15.162:6802/12311,v1:172.21.15.162:6803/12311] conn(0x7f607005c740 msgr2=0x7f607005eb60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:33.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.491+0000 7f6090a39700 1 --2- 172.21.15.162:0/3434250193 >> [v2:172.21.15.162:6802/12311,v1:172.21.15.162:6803/12311] conn(0x7f607005c740 0x7f607005eb60 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:33.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.491+0000 7f6090a39700 1 -- 172.21.15.162:0/3434250193 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6070055330 msgr2=0x7f60700577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:33.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.491+0000 7f6090a39700 1 --2- 172.21.15.162:0/3434250193 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6070055330 0x7f60700577f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f607c00b000 tx=0x7f607c008040).stop 2020-11-28T21:46:33.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.491+0000 7f6090a39700 1 -- 172.21.15.162:0/3434250193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f608c1467f0 msgr2=0x7f608c151ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:33.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.491+0000 7f6090a39700 1 --2- 172.21.15.162:0/3434250193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f608c1467f0 0x7f608c151ad0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f608c0645c0 tx=0x7f6084007d70).stop 2020-11-28T21:46:33.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.491+0000 7f608bfff700 1 -- 172.21.15.162:0/3434250193 reap_dead start 2020-11-28T21:46:33.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.491+0000 7f6090a39700 1 -- 172.21.15.162:0/3434250193 shutdown_connections 2020-11-28T21:46:33.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.491+0000 7f6090a39700 1 -- 172.21.15.162:0/3434250193 >> 172.21.15.162:0/3434250193 conn(0x7f608c0b6870 msgr2=0x7f608c061440 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:33.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.491+0000 7f6090a39700 1 -- 172.21.15.162:0/3434250193 shutdown_connections 2020-11-28T21:46:33.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.491+0000 7f6090a39700 1 -- 172.21.15.162:0/3434250193 wait complete. 2020-11-28T21:46:33.510 INFO:teuthology.orchestra.run.smithi162.stdout:60129542149 2020-11-28T21:46:33.511 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.2 flush_pg_stats 2020-11-28T21:46:33.624 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.619+0000 7fe8b1ef2700 1 Processor -- start 2020-11-28T21:46:33.624 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.619+0000 7fe8b1ef2700 1 -- start start 2020-11-28T21:46:33.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.623+0000 7fe8b1ef2700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe8ac098b30 0x7fe8ac0994f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.623+0000 7fe8b1ef2700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe8ac060710 0x7fe8ac0b1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.623+0000 7fe8b1ef2700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8ac098f10 con 0x7fe8ac0b0000 2020-11-28T21:46:33.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.623+0000 7fe8b1ef2700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8ac0b1030 con 0x7fe8ac060710 2020-11-28T21:46:33.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.623+0000 7fe8b1ef2700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac0b0480 con 0x7fe8ac098b30 2020-11-28T21:46:33.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.623+0000 7fe8b0ef0700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe8ac098b30 0x7fe8ac0994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.623+0000 7fe8b16f1700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe8ac060710 0x7fe8ac0b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.623+0000 7fe8b0ef0700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe8ac098b30 0x7fe8ac0994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:34628/0 (socket says 172.21.15.162:34628) 2020-11-28T21:46:33.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.623+0000 7fe8b0ef0700 1 -- 172.21.15.162:0/4278457220 learned_addr learned my addr 172.21.15.162:0/4278457220 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:33.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.623+0000 7fe8b0ef0700 1 -- 172.21.15.162:0/4278457220 >> v2:172.21.15.118:3300/0 conn(0x7fe8ac060710 msgr2=0x7fe8ac0b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:33.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.623+0000 7fe8b0ef0700 1 --2- 172.21.15.162:0/4278457220 >> v2:172.21.15.118:3300/0 conn(0x7fe8ac060710 0x7fe8ac0b1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:33.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.623+0000 7fe8b0ef0700 1 -- 172.21.15.162:0/4278457220 >> v1:172.21.15.118:6789/0 conn(0x7fe8ac0b0000 legacy=0x7fe8ac060000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:33.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.623+0000 7fe8b0ef0700 1 -- 172.21.15.162:0/4278457220 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ac0b0680 con 0x7fe8ac098b30 2020-11-28T21:46:33.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.623+0000 7fe8b0ef0700 1 --2- 172.21.15.162:0/4278457220 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe8ac098b30 0x7fe8ac0994f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fe8ac099be0 tx=0x7fe8a0006e80).ready entity=mon.2 client_cookie=b5637aa6568a411 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:33.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.623+0000 7fe8ab7fe700 1 -- 172.21.15.162:0/4278457220 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe8a00097b0 con 0x7fe8ac098b30 2020-11-28T21:46:33.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.623+0000 7fe8ab7fe700 1 -- 172.21.15.162:0/4278457220 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a0009910 con 0x7fe8ac098b30 2020-11-28T21:46:33.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.623+0000 7fe8ab7fe700 1 -- 172.21.15.162:0/4278457220 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe8a0009c70 con 0x7fe8ac098b30 2020-11-28T21:46:33.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.623+0000 7fe8b1ef2700 1 -- 172.21.15.162:0/4278457220 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe8ac098b30 msgr2=0x7fe8ac0994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:33.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.623+0000 7fe8b1ef2700 1 --2- 172.21.15.162:0/4278457220 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe8ac098b30 0x7fe8ac0994f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fe8ac099be0 tx=0x7fe8a0006e80).stop 2020-11-28T21:46:33.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.623+0000 7fe8b1ef2700 1 -- 172.21.15.162:0/4278457220 shutdown_connections 2020-11-28T21:46:33.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.623+0000 7fe8b1ef2700 1 --2- 172.21.15.162:0/4278457220 >> v2:172.21.15.118:3300/0 conn(0x7fe8ac060710 0x7fe8ac0b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:33.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.623+0000 7fe8b1ef2700 1 --2- 172.21.15.162:0/4278457220 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe8ac098b30 0x7fe8ac0994f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:33.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.623+0000 7fe8b1ef2700 1 -- 172.21.15.162:0/4278457220 >> 172.21.15.162:0/4278457220 conn(0x7fe8ac05b970 msgr2=0x7fe8ac05bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:33.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.623+0000 7fe8b1ef2700 1 -- 172.21.15.162:0/4278457220 shutdown_connections 2020-11-28T21:46:33.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.623+0000 7fe8b1ef2700 1 -- 172.21.15.162:0/4278457220 wait complete. 2020-11-28T21:46:33.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.627+0000 7fe8b1ef2700 1 Processor -- start 2020-11-28T21:46:33.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.627+0000 7fe8b1ef2700 1 -- start start 2020-11-28T21:46:33.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.627+0000 7fe8b1ef2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8ac060710 0x7fe8ac0aeca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.627+0000 7fe8b1ef2700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe8ac098b30 0x7fe8ac0aad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.627+0000 7fe8b1ef2700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe8ac0b0000 0x7fe8ac0635e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.627+0000 7fe8b1ef2700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac0675c0 con 0x7fe8ac098b30 2020-11-28T21:46:33.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.627+0000 7fe8b1ef2700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac0ad8b0 con 0x7fe8ac060710 2020-11-28T21:46:33.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.627+0000 7fe8b1ef2700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac0ad370 con 0x7fe8ac0b0000 2020-11-28T21:46:33.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.627+0000 7fe8b0ef0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8ac060710 0x7fe8ac0aeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.627+0000 7fe8b0ef0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8ac060710 0x7fe8ac0aeca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37410/0 (socket says 172.21.15.162:37410) 2020-11-28T21:46:33.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.627+0000 7fe8b0ef0700 1 -- 172.21.15.162:0/415204952 learned_addr learned my addr 172.21.15.162:0/415204952 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:33.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.627+0000 7fe8b16f1700 1 --2- 172.21.15.162:0/415204952 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe8ac0b0000 0x7fe8ac0635e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.627+0000 7fe8abfff700 1 --2- 172.21.15.162:0/415204952 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe8ac098b30 0x7fe8ac0aad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.627+0000 7fe8b0ef0700 1 -- 172.21.15.162:0/415204952 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe8ac0b0000 msgr2=0x7fe8ac0635e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:33.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.627+0000 7fe8b0ef0700 1 --2- 172.21.15.162:0/415204952 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe8ac0b0000 0x7fe8ac0635e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:33.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.627+0000 7fe8b0ef0700 1 -- 172.21.15.162:0/415204952 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe8ac098b30 msgr2=0x7fe8ac0aad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:33.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.627+0000 7fe8b0ef0700 1 --2- 172.21.15.162:0/415204952 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe8ac098b30 0x7fe8ac0aad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:33.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.627+0000 7fe8b0ef0700 1 -- 172.21.15.162:0/415204952 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ac0ad610 con 0x7fe8ac060710 2020-11-28T21:46:33.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.627+0000 7fe8b0ef0700 1 --2- 172.21.15.162:0/415204952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8ac060710 0x7fe8ac0aeca0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fe8ac099be0 tx=0x7fe8a0015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:33.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.627+0000 7fe8a9ffb700 1 -- 172.21.15.162:0/415204952 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe8a0002f30 con 0x7fe8ac060710 2020-11-28T21:46:33.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.631+0000 7fe8a9ffb700 1 -- 172.21.15.162:0/415204952 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a0011b80 con 0x7fe8ac060710 2020-11-28T21:46:33.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.631+0000 7fe8a9ffb700 1 -- 172.21.15.162:0/415204952 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe8a001e460 con 0x7fe8ac060710 2020-11-28T21:46:33.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.631+0000 7fe8b1ef2700 1 -- 172.21.15.162:0/415204952 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8ac063ad0 con 0x7fe8ac060710 2020-11-28T21:46:33.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.631+0000 7fe8b1ef2700 1 -- 172.21.15.162:0/415204952 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8ac067240 con 0x7fe8ac060710 2020-11-28T21:46:33.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.631+0000 7fe8b1ef2700 1 -- 172.21.15.162:0/415204952 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe8ac05aa30 con 0x7fe8ac060710 2020-11-28T21:46:33.639 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.631+0000 7fe8a9ffb700 1 -- 172.21.15.162:0/415204952 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe8a0025030 con 0x7fe8ac060710 2020-11-28T21:46:33.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.631+0000 7fe8a9ffb700 1 --2- 172.21.15.162:0/415204952 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe8940553b0 0x7fe894057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.631+0000 7fe8a9ffb700 1 -- 172.21.15.162:0/415204952 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe8a006f6f0 con 0x7fe8ac060710 2020-11-28T21:46:33.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.631+0000 7fe8abfff700 1 --2- 172.21.15.162:0/415204952 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe8940553b0 0x7fe894057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.631+0000 7fe8a9ffb700 1 --2- 172.21.15.162:0/415204952 >> [v2:172.21.15.162:6818/12312,v1:172.21.15.162:6819/12312] conn(0x7fe89405c720 0x7fe89405eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.631+0000 7fe8a9ffb700 1 -- 172.21.15.162:0/415204952 --> [v2:172.21.15.162:6818/12312,v1:172.21.15.162:6819/12312] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe89405f1a0 con 0x7fe89405c720 2020-11-28T21:46:33.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.631+0000 7fe8a9ffb700 1 -- 172.21.15.162:0/415204952 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8a006fab0 con 0x7fe8ac060710 2020-11-28T21:46:33.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.631+0000 7fe8b16f1700 1 --2- 172.21.15.162:0/415204952 >> [v2:172.21.15.162:6818/12312,v1:172.21.15.162:6819/12312] conn(0x7fe89405c720 0x7fe89405eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.635+0000 7fe8abfff700 1 --2- 172.21.15.162:0/415204952 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe8940553b0 0x7fe894057870 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fe89800d000 tx=0x7fe898007b40).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:33.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.635+0000 7fe8b16f1700 1 --2- 172.21.15.162:0/415204952 >> [v2:172.21.15.162:6818/12312,v1:172.21.15.162:6819/12312] conn(0x7fe89405c720 0x7fe89405eb40 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:33.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.635+0000 7fe8a9ffb700 1 -- 172.21.15.162:0/415204952 <== osd.2 v2:172.21.15.162:6818/12312 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7fe89405f1a0 con 0x7fe89405c720 2020-11-28T21:46:33.660 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.659+0000 7fe8b1ef2700 1 -- 172.21.15.162:0/415204952 --> [v2:172.21.15.162:6818/12312,v1:172.21.15.162:6819/12312] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe8ac063340 con 0x7fe89405c720 2020-11-28T21:46:33.662 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.659+0000 7fe8a9ffb700 1 -- 172.21.15.162:0/415204952 <== osd.2 v2:172.21.15.162:6818/12312 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe8a4009e00 con 0x7fe89405c720 2020-11-28T21:46:33.662 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.659+0000 7fe8b1ef2700 1 -- 172.21.15.162:0/415204952 >> [v2:172.21.15.162:6818/12312,v1:172.21.15.162:6819/12312] conn(0x7fe89405c720 msgr2=0x7fe89405eb40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:33.662 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.659+0000 7fe8b1ef2700 1 --2- 172.21.15.162:0/415204952 >> [v2:172.21.15.162:6818/12312,v1:172.21.15.162:6819/12312] conn(0x7fe89405c720 0x7fe89405eb40 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:33.662 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.659+0000 7fe8b1ef2700 1 -- 172.21.15.162:0/415204952 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe8940553b0 msgr2=0x7fe894057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:33.662 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.659+0000 7fe8b1ef2700 1 --2- 172.21.15.162:0/415204952 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe8940553b0 0x7fe894057870 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fe89800d000 tx=0x7fe898007b40).stop 2020-11-28T21:46:33.663 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.659+0000 7fe8b1ef2700 1 -- 172.21.15.162:0/415204952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8ac060710 msgr2=0x7fe8ac0aeca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:33.663 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.659+0000 7fe8b1ef2700 1 --2- 172.21.15.162:0/415204952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8ac060710 0x7fe8ac0aeca0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fe8ac099be0 tx=0x7fe8a0015000).stop 2020-11-28T21:46:33.663 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.659+0000 7fe8b16f1700 1 -- 172.21.15.162:0/415204952 reap_dead start 2020-11-28T21:46:33.663 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.659+0000 7fe8b1ef2700 1 -- 172.21.15.162:0/415204952 shutdown_connections 2020-11-28T21:46:33.664 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.659+0000 7fe8b1ef2700 1 -- 172.21.15.162:0/415204952 >> 172.21.15.162:0/415204952 conn(0x7fe8ac05b970 msgr2=0x7fe8ac0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:33.664 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.659+0000 7fe8b1ef2700 1 -- 172.21.15.162:0/415204952 shutdown_connections 2020-11-28T21:46:33.664 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.659+0000 7fe8b1ef2700 1 -- 172.21.15.162:0/415204952 wait complete. 2020-11-28T21:46:33.674 INFO:teuthology.orchestra.run.smithi162.stdout:60129542148 2020-11-28T21:46:33.674 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.3 flush_pg_stats 2020-11-28T21:46:33.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.795+0000 7fce22811700 1 Processor -- start 2020-11-28T21:46:33.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.795+0000 7fce22811700 1 -- start start 2020-11-28T21:46:33.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.795+0000 7fce22811700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fce1c0b5bf0 0x7fce1c142980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.795+0000 7fce22811700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce1c0b3a60 0x7fce1c13fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.795+0000 7fce22811700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce1c05f380 con 0x7fce1c060cb0 2020-11-28T21:46:33.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.795+0000 7fce22811700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce1c143930 con 0x7fce1c0b5bf0 2020-11-28T21:46:33.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.795+0000 7fce22811700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c05f5b0 con 0x7fce1c0b3a60 2020-11-28T21:46:33.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.795+0000 7fce2100e700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fce1c060cb0 0x7fce1c060230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:44264/0 (socket says 172.21.15.162:44264) 2020-11-28T21:46:33.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.795+0000 7fce2100e700 1 -- 172.21.15.162:0/3836140972 learned_addr learned my addr 172.21.15.162:0/3836140972 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:33.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.795+0000 7fce22010700 1 --2- 172.21.15.162:0/3836140972 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce1c0b3a60 0x7fce1c13fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.795+0000 7fce2180f700 1 --2- 172.21.15.162:0/3836140972 >> v2:172.21.15.118:3300/0 conn(0x7fce1c0b5bf0 0x7fce1c142980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.795+0000 7fce22010700 1 -- 172.21.15.162:0/3836140972 >> v2:172.21.15.118:3300/0 conn(0x7fce1c0b5bf0 msgr2=0x7fce1c142980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:33.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.795+0000 7fce22010700 1 --2- 172.21.15.162:0/3836140972 >> v2:172.21.15.118:3300/0 conn(0x7fce1c0b5bf0 0x7fce1c142980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:33.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.795+0000 7fce22010700 1 -- 172.21.15.162:0/3836140972 >> v1:172.21.15.162:6789/0 conn(0x7fce1c060cb0 legacy=0x7fce1c060230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:33.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.795+0000 7fce22010700 1 -- 172.21.15.162:0/3836140972 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce1c05f850 con 0x7fce1c0b3a60 2020-11-28T21:46:33.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.799+0000 7fce22010700 1 --2- 172.21.15.162:0/3836140972 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce1c0b3a60 0x7fce1c13fbe0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fce0c002f30 tx=0x7fce0c006dc0).ready entity=mon.2 client_cookie=ec1539c59ab687e3 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:33.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.799+0000 7fce2080d700 1 -- 172.21.15.162:0/3836140972 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce0c009760 con 0x7fce1c0b3a60 2020-11-28T21:46:33.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.799+0000 7fce2080d700 1 -- 172.21.15.162:0/3836140972 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce0c0098c0 con 0x7fce1c0b3a60 2020-11-28T21:46:33.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.799+0000 7fce2080d700 1 -- 172.21.15.162:0/3836140972 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce0c009c20 con 0x7fce1c0b3a60 2020-11-28T21:46:33.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.799+0000 7fce22811700 1 -- 172.21.15.162:0/3836140972 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce1c0b3a60 msgr2=0x7fce1c13fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:33.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.799+0000 7fce22811700 1 --2- 172.21.15.162:0/3836140972 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce1c0b3a60 0x7fce1c13fbe0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fce0c002f30 tx=0x7fce0c006dc0).stop 2020-11-28T21:46:33.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.799+0000 7fce22811700 1 -- 172.21.15.162:0/3836140972 shutdown_connections 2020-11-28T21:46:33.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.799+0000 7fce22811700 1 --2- 172.21.15.162:0/3836140972 >> v2:172.21.15.118:3300/0 conn(0x7fce1c0b5bf0 0x7fce1c142980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:33.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.799+0000 7fce22811700 1 --2- 172.21.15.162:0/3836140972 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce1c0b3a60 0x7fce1c13fbe0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:33.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.799+0000 7fce22811700 1 -- 172.21.15.162:0/3836140972 >> 172.21.15.162:0/3836140972 conn(0x7fce1c05b970 msgr2=0x7fce1c05bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:33.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.799+0000 7fce22811700 1 -- 172.21.15.162:0/3836140972 shutdown_connections 2020-11-28T21:46:33.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.799+0000 7fce22811700 1 -- 172.21.15.162:0/3836140972 wait complete. 2020-11-28T21:46:33.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.803+0000 7fce22811700 1 Processor -- start 2020-11-28T21:46:33.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.803+0000 7fce22811700 1 -- start start 2020-11-28T21:46:33.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.803+0000 7fce22811700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce1c060cb0 0x7fce1c09dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.810 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.803+0000 7fce22811700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce1c0b3a60 0x7fce1c09e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.810 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.803+0000 7fce22811700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce1c0b5bf0 0x7fce1c143e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.810 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.803+0000 7fce22811700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c0af580 con 0x7fce1c060cb0 2020-11-28T21:46:33.811 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.803+0000 7fce22811700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c09e650 con 0x7fce1c0b3a60 2020-11-28T21:46:33.811 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.803+0000 7fce22811700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c13e2c0 con 0x7fce1c0b5bf0 2020-11-28T21:46:33.811 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.803+0000 7fce2100e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce1c0b3a60 0x7fce1c09e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.811 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.803+0000 7fce2100e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce1c0b3a60 0x7fce1c09e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37426/0 (socket says 172.21.15.162:37426) 2020-11-28T21:46:33.812 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.803+0000 7fce2100e700 1 -- 172.21.15.162:0/2289999412 learned_addr learned my addr 172.21.15.162:0/2289999412 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:33.812 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.803+0000 7fce22010700 1 --2- 172.21.15.162:0/2289999412 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce1c0b5bf0 0x7fce1c143e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.812 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.803+0000 7fce2180f700 1 --2- 172.21.15.162:0/2289999412 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce1c060cb0 0x7fce1c09dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.812 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.803+0000 7fce2100e700 1 -- 172.21.15.162:0/2289999412 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce1c0b5bf0 msgr2=0x7fce1c143e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:33.812 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.803+0000 7fce2100e700 1 --2- 172.21.15.162:0/2289999412 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce1c0b5bf0 0x7fce1c143e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:33.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.803+0000 7fce2100e700 1 -- 172.21.15.162:0/2289999412 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce1c060cb0 msgr2=0x7fce1c09dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:33.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.803+0000 7fce2100e700 1 --2- 172.21.15.162:0/2289999412 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce1c060cb0 0x7fce1c09dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:33.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.803+0000 7fce2100e700 1 -- 172.21.15.162:0/2289999412 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce1c0acee0 con 0x7fce1c0b3a60 2020-11-28T21:46:33.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.803+0000 7fce2100e700 1 --2- 172.21.15.162:0/2289999412 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce1c0b3a60 0x7fce1c09e160 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fce1c09f0d0 tx=0x7fce14007f50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:33.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.803+0000 7fce12ffd700 1 -- 172.21.15.162:0/2289999412 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce14005b70 con 0x7fce1c0b3a60 2020-11-28T21:46:33.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.803+0000 7fce12ffd700 1 -- 172.21.15.162:0/2289999412 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce14005cd0 con 0x7fce1c0b3a60 2020-11-28T21:46:33.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.803+0000 7fce22811700 1 -- 172.21.15.162:0/2289999412 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce1c152880 con 0x7fce1c0b3a60 2020-11-28T21:46:33.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.803+0000 7fce12ffd700 1 -- 172.21.15.162:0/2289999412 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce14007300 con 0x7fce1c0b3a60 2020-11-28T21:46:33.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.803+0000 7fce22811700 1 -- 172.21.15.162:0/2289999412 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce1c0b7f30 con 0x7fce1c0b3a60 2020-11-28T21:46:33.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.807+0000 7fce22811700 1 -- 172.21.15.162:0/2289999412 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fce1c0b8570 con 0x7fce1c0b3a60 2020-11-28T21:46:33.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.807+0000 7fce12ffd700 1 -- 172.21.15.162:0/2289999412 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fce14010030 con 0x7fce1c0b3a60 2020-11-28T21:46:33.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.807+0000 7fce12ffd700 1 --2- 172.21.15.162:0/2289999412 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcdfc0554e0 0x7fcdfc0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.807+0000 7fce12ffd700 1 -- 172.21.15.162:0/2289999412 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fce1406df20 con 0x7fce1c0b3a60 2020-11-28T21:46:33.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.807+0000 7fce2180f700 1 --2- 172.21.15.162:0/2289999412 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcdfc0554e0 0x7fcdfc0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.807+0000 7fce12ffd700 1 --2- 172.21.15.162:0/2289999412 >> [v2:172.21.15.162:6826/12323,v1:172.21.15.162:6827/12323] conn(0x7fcdfc05c850 0x7fcdfc05ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.807+0000 7fce12ffd700 1 -- 172.21.15.162:0/2289999412 --> [v2:172.21.15.162:6826/12323,v1:172.21.15.162:6827/12323] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcdfc05f230 con 0x7fcdfc05c850 2020-11-28T21:46:33.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.807+0000 7fce12ffd700 1 -- 172.21.15.162:0/2289999412 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce14070590 con 0x7fce1c0b3a60 2020-11-28T21:46:33.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.807+0000 7fce22010700 1 --2- 172.21.15.162:0/2289999412 >> [v2:172.21.15.162:6826/12323,v1:172.21.15.162:6827/12323] conn(0x7fcdfc05c850 0x7fcdfc05ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.807+0000 7fce2180f700 1 --2- 172.21.15.162:0/2289999412 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcdfc0554e0 0x7fcdfc0579a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fce1800b000 tx=0x7fce18006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:33.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.807+0000 7fce22010700 1 --2- 172.21.15.162:0/2289999412 >> [v2:172.21.15.162:6826/12323,v1:172.21.15.162:6827/12323] conn(0x7fcdfc05c850 0x7fcdfc05ec70 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:33.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.811+0000 7fce12ffd700 1 -- 172.21.15.162:0/2289999412 <== osd.3 v2:172.21.15.162:6826/12323 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7fcdfc05f230 con 0x7fcdfc05c850 2020-11-28T21:46:33.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.827+0000 7fce22811700 1 -- 172.21.15.162:0/2289999412 --> [v2:172.21.15.162:6826/12323,v1:172.21.15.162:6827/12323] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fce1c057050 con 0x7fcdfc05c850 2020-11-28T21:46:33.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.827+0000 7fce12ffd700 1 -- 172.21.15.162:0/2289999412 <== osd.3 v2:172.21.15.162:6826/12323 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fce1c057050 con 0x7fcdfc05c850 2020-11-28T21:46:33.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.831+0000 7fce22811700 1 -- 172.21.15.162:0/2289999412 >> [v2:172.21.15.162:6826/12323,v1:172.21.15.162:6827/12323] conn(0x7fcdfc05c850 msgr2=0x7fcdfc05ec70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:33.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.831+0000 7fce22811700 1 --2- 172.21.15.162:0/2289999412 >> [v2:172.21.15.162:6826/12323,v1:172.21.15.162:6827/12323] conn(0x7fcdfc05c850 0x7fcdfc05ec70 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:33.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.831+0000 7fce22811700 1 -- 172.21.15.162:0/2289999412 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcdfc0554e0 msgr2=0x7fcdfc0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:33.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.831+0000 7fce22811700 1 --2- 172.21.15.162:0/2289999412 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcdfc0554e0 0x7fcdfc0579a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fce1800b000 tx=0x7fce18006dc0).stop 2020-11-28T21:46:33.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.831+0000 7fce22811700 1 -- 172.21.15.162:0/2289999412 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce1c0b3a60 msgr2=0x7fce1c09e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:33.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.831+0000 7fce22811700 1 --2- 172.21.15.162:0/2289999412 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce1c0b3a60 0x7fce1c09e160 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fce1c09f0d0 tx=0x7fce14007f50).stop 2020-11-28T21:46:33.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.831+0000 7fce22010700 1 -- 172.21.15.162:0/2289999412 reap_dead start 2020-11-28T21:46:33.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.831+0000 7fce22811700 1 -- 172.21.15.162:0/2289999412 shutdown_connections 2020-11-28T21:46:33.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.831+0000 7fce22811700 1 -- 172.21.15.162:0/2289999412 >> 172.21.15.162:0/2289999412 conn(0x7fce1c05b970 msgr2=0x7fce1c140ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:33.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.831+0000 7fce22811700 1 -- 172.21.15.162:0/2289999412 shutdown_connections 2020-11-28T21:46:33.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.831+0000 7fce22811700 1 -- 172.21.15.162:0/2289999412 wait complete. 2020-11-28T21:46:33.845 INFO:teuthology.orchestra.run.smithi162.stdout:60129542148 2020-11-28T21:46:33.845 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.4 flush_pg_stats 2020-11-28T21:46:33.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.963+0000 7f0deffff700 1 Processor -- start 2020-11-28T21:46:33.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.963+0000 7f0deffff700 1 -- start start 2020-11-28T21:46:33.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.963+0000 7f0deffff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0df005bcb0 0x7f0df0143530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.963+0000 7f0deffff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0df013c630 0x7f0df013cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.963+0000 7f0deffff700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0df00b0e50 con 0x7f0df00b1c00 2020-11-28T21:46:33.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.963+0000 7f0deffff700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0df005fe40 con 0x7f0df005bcb0 2020-11-28T21:46:33.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.963+0000 7f0deffff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0df00b2080 con 0x7f0df013c630 2020-11-28T21:46:33.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.963+0000 7f0deeffd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0df005bcb0 0x7f0df0143530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.963+0000 7f0dee7fc700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f0df00b1c00 0x7f0df00b0740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:44280/0 (socket says 172.21.15.162:44280) 2020-11-28T21:46:33.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.963+0000 7f0dee7fc700 1 -- 172.21.15.162:0/329552065 learned_addr learned my addr 172.21.15.162:0/329552065 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:33.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.967+0000 7f0def7fe700 1 --2- 172.21.15.162:0/329552065 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0df013c630 0x7f0df013cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.967+0000 7f0deeffd700 1 -- 172.21.15.162:0/329552065 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0df013c630 msgr2=0x7f0df013cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:33.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.967+0000 7f0deeffd700 1 --2- 172.21.15.162:0/329552065 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0df013c630 0x7f0df013cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:33.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.967+0000 7f0deeffd700 1 -- 172.21.15.162:0/329552065 >> v1:172.21.15.162:6789/0 conn(0x7f0df00b1c00 legacy=0x7f0df00b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:33.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.967+0000 7f0deeffd700 1 -- 172.21.15.162:0/329552065 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df005c090 con 0x7f0df005bcb0 2020-11-28T21:46:33.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.967+0000 7f0deeffd700 1 --2- 172.21.15.162:0/329552065 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0df005bcb0 0x7f0df0143530 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f0df005ff80 tx=0x7f0de0006dc0).ready entity=mon.1 client_cookie=e02180a6ee881baa server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:33.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.967+0000 7f0dedffb700 1 -- 172.21.15.162:0/329552065 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0de00097b0 con 0x7f0df005bcb0 2020-11-28T21:46:33.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.967+0000 7f0dedffb700 1 -- 172.21.15.162:0/329552065 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0de0009910 con 0x7f0df005bcb0 2020-11-28T21:46:33.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.967+0000 7f0dedffb700 1 -- 172.21.15.162:0/329552065 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0de0009c70 con 0x7f0df005bcb0 2020-11-28T21:46:33.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.967+0000 7f0deffff700 1 -- 172.21.15.162:0/329552065 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0df005bcb0 msgr2=0x7f0df0143530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:33.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.967+0000 7f0deffff700 1 --2- 172.21.15.162:0/329552065 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0df005bcb0 0x7f0df0143530 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f0df005ff80 tx=0x7f0de0006dc0).stop 2020-11-28T21:46:33.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.967+0000 7f0deffff700 1 -- 172.21.15.162:0/329552065 shutdown_connections 2020-11-28T21:46:33.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.967+0000 7f0deffff700 1 --2- 172.21.15.162:0/329552065 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0df005bcb0 0x7f0df0143530 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:33.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.967+0000 7f0deffff700 1 --2- 172.21.15.162:0/329552065 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0df013c630 0x7f0df013cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:33.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.967+0000 7f0deffff700 1 -- 172.21.15.162:0/329552065 >> 172.21.15.162:0/329552065 conn(0x7f0df00b5da0 msgr2=0x7f0df00b6180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:33.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.967+0000 7f0deffff700 1 -- 172.21.15.162:0/329552065 shutdown_connections 2020-11-28T21:46:33.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.967+0000 7f0deffff700 1 -- 172.21.15.162:0/329552065 wait complete. 2020-11-28T21:46:33.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.967+0000 7f0deffff700 1 Processor -- start 2020-11-28T21:46:33.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.967+0000 7f0deffff700 1 -- start start 2020-11-28T21:46:33.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.967+0000 7f0deffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0df005bcb0 0x7f0df0143e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.967+0000 7f0deffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0df00b1c00 0x7f0df0151930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.967+0000 7f0deffff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0df013c630 0x7f0df0151e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.967+0000 7f0deffff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df0144f60 con 0x7f0df005bcb0 2020-11-28T21:46:33.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.967+0000 7f0deffff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df00af8d0 con 0x7f0df00b1c00 2020-11-28T21:46:33.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.967+0000 7f0deffff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0df0060fd0 con 0x7f0df013c630 2020-11-28T21:46:33.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.967+0000 7f0dee7fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0df00b1c00 0x7f0df0151930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.971+0000 7f0dee7fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0df00b1c00 0x7f0df0151930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37444/0 (socket says 172.21.15.162:37444) 2020-11-28T21:46:33.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.971+0000 7f0dee7fc700 1 -- 172.21.15.162:0/2040475026 learned_addr learned my addr 172.21.15.162:0/2040475026 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:33.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.971+0000 7f0dee7fc700 1 -- 172.21.15.162:0/2040475026 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0df013c630 msgr2=0x7f0df0151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:33.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.971+0000 7f0deeffd700 1 --2- 172.21.15.162:0/2040475026 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0df005bcb0 0x7f0df0143e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.971+0000 7f0def7fe700 1 --2- 172.21.15.162:0/2040475026 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0df013c630 0x7f0df0151e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.971+0000 7f0dee7fc700 1 --2- 172.21.15.162:0/2040475026 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0df013c630 0x7f0df0151e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:33.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.971+0000 7f0dee7fc700 1 -- 172.21.15.162:0/2040475026 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0df005bcb0 msgr2=0x7f0df0143e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:33.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.971+0000 7f0dee7fc700 1 --2- 172.21.15.162:0/2040475026 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0df005bcb0 0x7f0df0143e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:33.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.971+0000 7f0dee7fc700 1 -- 172.21.15.162:0/2040475026 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df0061d90 con 0x7f0df00b1c00 2020-11-28T21:46:33.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.971+0000 7f0deeffd700 1 --2- 172.21.15.162:0/2040475026 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0df005bcb0 0x7f0df0143e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:46:33.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.971+0000 7f0def7fe700 1 --2- 172.21.15.162:0/2040475026 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0df013c630 0x7f0df0151e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:46:33.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.971+0000 7f0dee7fc700 1 --2- 172.21.15.162:0/2040475026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0df00b1c00 0x7f0df0151930 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f0df00b6b90 tx=0x7f0de4007c30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:33.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.971+0000 7f0ddffff700 1 -- 172.21.15.162:0/2040475026 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0de4007320 con 0x7f0df00b1c00 2020-11-28T21:46:33.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.971+0000 7f0ddffff700 1 -- 172.21.15.162:0/2040475026 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0de4007480 con 0x7f0df00b1c00 2020-11-28T21:46:33.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.971+0000 7f0deffff700 1 -- 172.21.15.162:0/2040475026 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0df00aff80 con 0x7f0df00b1c00 2020-11-28T21:46:33.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.971+0000 7f0ddffff700 1 -- 172.21.15.162:0/2040475026 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0de4005220 con 0x7f0df00b1c00 2020-11-28T21:46:33.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.971+0000 7f0deffff700 1 -- 172.21.15.162:0/2040475026 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0df0061200 con 0x7f0df00b1c00 2020-11-28T21:46:33.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.971+0000 7f0deffff700 1 -- 172.21.15.162:0/2040475026 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0df0061400 con 0x7f0df00b1c00 2020-11-28T21:46:33.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.971+0000 7f0ddffff700 1 -- 172.21.15.162:0/2040475026 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0de401b070 con 0x7f0df00b1c00 2020-11-28T21:46:33.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.971+0000 7f0ddffff700 1 --2- 172.21.15.162:0/2040475026 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0dcc0553b0 0x7f0dcc057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.971+0000 7f0ddffff700 1 -- 172.21.15.162:0/2040475026 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0de406d440 con 0x7f0df00b1c00 2020-11-28T21:46:33.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.971+0000 7f0deeffd700 1 --2- 172.21.15.162:0/2040475026 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0dcc0553b0 0x7f0dcc057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.971+0000 7f0ddffff700 1 --2- 172.21.15.162:0/2040475026 >> [v2:172.21.15.118:6800/12527,v1:172.21.15.118:6801/12527] conn(0x7f0dcc05c7a0 0x7f0dcc05ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:33.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.971+0000 7f0ddffff700 1 -- 172.21.15.162:0/2040475026 --> [v2:172.21.15.118:6800/12527,v1:172.21.15.118:6801/12527] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0dcc05f220 con 0x7f0dcc05c7a0 2020-11-28T21:46:33.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.971+0000 7f0ddffff700 1 -- 172.21.15.162:0/2040475026 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0de406fad0 con 0x7f0df00b1c00 2020-11-28T21:46:33.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.975+0000 7f0deeffd700 1 --2- 172.21.15.162:0/2040475026 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0dcc0553b0 0x7f0dcc057870 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f0de0016000 tx=0x7f0de0015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:33.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.975+0000 7f0def7fe700 1 --2- 172.21.15.162:0/2040475026 >> [v2:172.21.15.118:6800/12527,v1:172.21.15.118:6801/12527] conn(0x7f0dcc05c7a0 0x7f0dcc05ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:33.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.975+0000 7f0def7fe700 1 --2- 172.21.15.162:0/2040475026 >> [v2:172.21.15.118:6800/12527,v1:172.21.15.118:6801/12527] conn(0x7f0dcc05c7a0 0x7f0dcc05ebc0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:33.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.975+0000 7f0ddffff700 1 -- 172.21.15.162:0/2040475026 <== osd.4 v2:172.21.15.118:6800/12527 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7f0dcc05f220 con 0x7f0dcc05c7a0 2020-11-28T21:46:33.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.995+0000 7f0deffff700 1 -- 172.21.15.162:0/2040475026 --> [v2:172.21.15.118:6800/12527,v1:172.21.15.118:6801/12527] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0df0061800 con 0x7f0dcc05c7a0 2020-11-28T21:46:34.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.999+0000 7f0ddffff700 1 -- 172.21.15.162:0/2040475026 <== osd.4 v2:172.21.15.118:6800/12527 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0df0061800 con 0x7f0dcc05c7a0 2020-11-28T21:46:34.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.999+0000 7f0deffff700 1 -- 172.21.15.162:0/2040475026 >> [v2:172.21.15.118:6800/12527,v1:172.21.15.118:6801/12527] conn(0x7f0dcc05c7a0 msgr2=0x7f0dcc05ebc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:34.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.999+0000 7f0deffff700 1 --2- 172.21.15.162:0/2040475026 >> [v2:172.21.15.118:6800/12527,v1:172.21.15.118:6801/12527] conn(0x7f0dcc05c7a0 0x7f0dcc05ebc0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:34.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.999+0000 7f0deffff700 1 -- 172.21.15.162:0/2040475026 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0dcc0553b0 msgr2=0x7f0dcc057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:34.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.999+0000 7f0deffff700 1 --2- 172.21.15.162:0/2040475026 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0dcc0553b0 0x7f0dcc057870 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f0de0016000 tx=0x7f0de0015000).stop 2020-11-28T21:46:34.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.999+0000 7f0deffff700 1 -- 172.21.15.162:0/2040475026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0df00b1c00 msgr2=0x7f0df0151930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:34.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.999+0000 7f0deffff700 1 --2- 172.21.15.162:0/2040475026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0df00b1c00 0x7f0df0151930 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f0df00b6b90 tx=0x7f0de4007c30).stop 2020-11-28T21:46:34.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.999+0000 7f0def7fe700 1 -- 172.21.15.162:0/2040475026 reap_dead start 2020-11-28T21:46:34.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.999+0000 7f0deffff700 1 -- 172.21.15.162:0/2040475026 shutdown_connections 2020-11-28T21:46:34.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.999+0000 7f0deffff700 1 -- 172.21.15.162:0/2040475026 >> 172.21.15.162:0/2040475026 conn(0x7f0df00b5da0 msgr2=0x7f0df0142200 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:34.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.999+0000 7f0deffff700 1 -- 172.21.15.162:0/2040475026 shutdown_connections 2020-11-28T21:46:34.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:33.999+0000 7f0deffff700 1 -- 172.21.15.162:0/2040475026 wait complete. 2020-11-28T21:46:34.014 INFO:teuthology.orchestra.run.smithi162.stdout:60129542148 2020-11-28T21:46:34.015 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.5 flush_pg_stats 2020-11-28T21:46:34.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.135+0000 7f223b26b700 1 Processor -- start 2020-11-28T21:46:34.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.135+0000 7f223b26b700 1 -- start start 2020-11-28T21:46:34.139 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.135+0000 7f223b26b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f22340b5bf0 0x7f2234142980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:34.139 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.135+0000 7f223b26b700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f22340b3a60 0x7f223413fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:34.139 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.135+0000 7f223b26b700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f223405f380 con 0x7f2234060cb0 2020-11-28T21:46:34.139 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.135+0000 7f223b26b700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2234143930 con 0x7f22340b3a60 2020-11-28T21:46:34.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.135+0000 7f223b26b700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f223405f5b0 con 0x7f22340b5bf0 2020-11-28T21:46:34.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.135+0000 7f223a269700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f22340b5bf0 0x7f2234142980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:34.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.135+0000 7f223a269700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f22340b5bf0 0x7f2234142980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37452/0 (socket says 172.21.15.162:37452) 2020-11-28T21:46:34.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.135+0000 7f2239a68700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f2234060cb0 0x7f2234060230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:44298/0 (socket says 172.21.15.162:44298) 2020-11-28T21:46:34.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.135+0000 7f223a269700 1 -- 172.21.15.162:0/862487399 learned_addr learned my addr 172.21.15.162:0/862487399 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:34.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.135+0000 7f223aa6a700 1 --2- 172.21.15.162:0/862487399 >> v2:172.21.15.118:3300/0 conn(0x7f22340b3a60 0x7f223413fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:34.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.135+0000 7f223a269700 1 -- 172.21.15.162:0/862487399 >> v2:172.21.15.118:3300/0 conn(0x7f22340b3a60 msgr2=0x7f223413fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:34.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.135+0000 7f223a269700 1 --2- 172.21.15.162:0/862487399 >> v2:172.21.15.118:3300/0 conn(0x7f22340b3a60 0x7f223413fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:34.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.135+0000 7f223a269700 1 -- 172.21.15.162:0/862487399 >> v1:172.21.15.162:6789/0 conn(0x7f2234060cb0 legacy=0x7f2234060230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:34.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.135+0000 7f223a269700 1 -- 172.21.15.162:0/862487399 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f223405f850 con 0x7f22340b5bf0 2020-11-28T21:46:34.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.135+0000 7f223aa6a700 1 --2- 172.21.15.162:0/862487399 >> v2:172.21.15.118:3300/0 conn(0x7f22340b3a60 0x7f223413fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:46:34.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.135+0000 7f2239267700 1 -- 172.21.15.162:0/862487399 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2479306464 0 0) 0x7f223405f380 con 0x7f2234060cb0 2020-11-28T21:46:34.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.135+0000 7f223a269700 1 --2- 172.21.15.162:0/862487399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22340b5bf0 0x7f2234142980 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f22300032b0 tx=0x7f2230009270).ready entity=mon.1 client_cookie=a571b2938141d008 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:34.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.135+0000 7f2239267700 1 -- 172.21.15.162:0/862487399 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f223000bc00 con 0x7f22340b5bf0 2020-11-28T21:46:34.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.135+0000 7f2239267700 1 -- 172.21.15.162:0/862487399 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f223000bd60 con 0x7f22340b5bf0 2020-11-28T21:46:34.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.135+0000 7f2239267700 1 -- 172.21.15.162:0/862487399 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2230011400 con 0x7f22340b5bf0 2020-11-28T21:46:34.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.135+0000 7f223b26b700 1 -- 172.21.15.162:0/862487399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22340b5bf0 msgr2=0x7f2234142980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:34.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.135+0000 7f223b26b700 1 --2- 172.21.15.162:0/862487399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22340b5bf0 0x7f2234142980 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f22300032b0 tx=0x7f2230009270).stop 2020-11-28T21:46:34.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.135+0000 7f223b26b700 1 -- 172.21.15.162:0/862487399 shutdown_connections 2020-11-28T21:46:34.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.135+0000 7f223b26b700 1 --2- 172.21.15.162:0/862487399 >> v2:172.21.15.118:3300/0 conn(0x7f22340b3a60 0x7f223413fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:34.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.135+0000 7f223b26b700 1 --2- 172.21.15.162:0/862487399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22340b5bf0 0x7f2234142980 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:34.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.135+0000 7f223b26b700 1 -- 172.21.15.162:0/862487399 >> 172.21.15.162:0/862487399 conn(0x7f223405b970 msgr2=0x7f223405bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:34.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.135+0000 7f223b26b700 1 -- 172.21.15.162:0/862487399 shutdown_connections 2020-11-28T21:46:34.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.135+0000 7f223b26b700 1 -- 172.21.15.162:0/862487399 wait complete. 2020-11-28T21:46:34.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.139+0000 7f223b26b700 1 Processor -- start 2020-11-28T21:46:34.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.139+0000 7f223b26b700 1 -- start start 2020-11-28T21:46:34.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.139+0000 7f223b26b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2234060cb0 0x7f22340b7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:34.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.139+0000 7f223b26b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22340b3a60 0x7f2234154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:34.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.139+0000 7f223b26b700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f22340b5bf0 0x7f2234155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:34.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.139+0000 7f223b26b700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22340b02d0 con 0x7f2234060cb0 2020-11-28T21:46:34.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.139+0000 7f223b26b700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22340b0500 con 0x7f22340b3a60 2020-11-28T21:46:34.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.139+0000 7f223b26b700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22340b2d10 con 0x7f22340b5bf0 2020-11-28T21:46:34.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.139+0000 7f2239a68700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22340b3a60 0x7f2234154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:34.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.139+0000 7f2239a68700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22340b3a60 0x7f2234154ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37460/0 (socket says 172.21.15.162:37460) 2020-11-28T21:46:34.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.139+0000 7f2239a68700 1 -- 172.21.15.162:0/3123510588 learned_addr learned my addr 172.21.15.162:0/3123510588 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:34.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.139+0000 7f223aa6a700 1 --2- 172.21.15.162:0/3123510588 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f22340b5bf0 0x7f2234155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:34.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.139+0000 7f223a269700 1 --2- 172.21.15.162:0/3123510588 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2234060cb0 0x7f22340b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:34.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.139+0000 7f2239a68700 1 -- 172.21.15.162:0/3123510588 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f22340b5bf0 msgr2=0x7f2234155090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:34.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.139+0000 7f2239a68700 1 --2- 172.21.15.162:0/3123510588 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f22340b5bf0 0x7f2234155090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:34.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.139+0000 7f2239a68700 1 -- 172.21.15.162:0/3123510588 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2234060cb0 msgr2=0x7f22340b7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:34.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.139+0000 7f2239a68700 1 --2- 172.21.15.162:0/3123510588 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2234060cb0 0x7f22340b7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:34.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.139+0000 7f2239a68700 1 -- 172.21.15.162:0/3123510588 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22340b09a0 con 0x7f22340b3a60 2020-11-28T21:46:34.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.139+0000 7f223a269700 1 --2- 172.21.15.162:0/3123510588 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2234060cb0 0x7f22340b7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:46:34.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.139+0000 7f2239a68700 1 --2- 172.21.15.162:0/3123510588 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22340b3a60 0x7f2234154ba0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f22340b1f80 tx=0x7f2224007a90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:34.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.139+0000 7f222b7fe700 1 -- 172.21.15.162:0/3123510588 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2224007320 con 0x7f22340b3a60 2020-11-28T21:46:34.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.139+0000 7f222b7fe700 1 -- 172.21.15.162:0/3123510588 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2224005520 con 0x7f22340b3a60 2020-11-28T21:46:34.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.139+0000 7f223b26b700 1 -- 172.21.15.162:0/3123510588 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2234155580 con 0x7f22340b3a60 2020-11-28T21:46:34.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.139+0000 7f222b7fe700 1 -- 172.21.15.162:0/3123510588 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2224005880 con 0x7f22340b3a60 2020-11-28T21:46:34.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.139+0000 7f223b26b700 1 -- 172.21.15.162:0/3123510588 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2234149600 con 0x7f22340b3a60 2020-11-28T21:46:34.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.143+0000 7f223b26b700 1 -- 172.21.15.162:0/3123510588 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f22340b0700 con 0x7f22340b3a60 2020-11-28T21:46:34.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.143+0000 7f222b7fe700 1 -- 172.21.15.162:0/3123510588 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2224010030 con 0x7f22340b3a60 2020-11-28T21:46:34.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.143+0000 7f222b7fe700 1 --2- 172.21.15.162:0/3123510588 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2214055330 0x7f22140577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:34.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.143+0000 7f222b7fe700 1 -- 172.21.15.162:0/3123510588 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f222406e570 con 0x7f22340b3a60 2020-11-28T21:46:34.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.143+0000 7f223a269700 1 --2- 172.21.15.162:0/3123510588 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2214055330 0x7f22140577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:34.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.143+0000 7f222b7fe700 1 --2- 172.21.15.162:0/3123510588 >> [v2:172.21.15.118:6808/12529,v1:172.21.15.118:6809/12529] conn(0x7f221405c740 0x7f221405eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:34.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.143+0000 7f222b7fe700 1 -- 172.21.15.162:0/3123510588 --> [v2:172.21.15.118:6808/12529,v1:172.21.15.118:6809/12529] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f221405f1c0 con 0x7f221405c740 2020-11-28T21:46:34.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.143+0000 7f222b7fe700 1 -- 172.21.15.162:0/3123510588 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f222406e930 con 0x7f22340b3a60 2020-11-28T21:46:34.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.143+0000 7f223aa6a700 1 --2- 172.21.15.162:0/3123510588 >> [v2:172.21.15.118:6808/12529,v1:172.21.15.118:6809/12529] conn(0x7f221405c740 0x7f221405eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:34.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.143+0000 7f223a269700 1 --2- 172.21.15.162:0/3123510588 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2214055330 0x7f22140577f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f2230017000 tx=0x7f2230016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:34.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.143+0000 7f223aa6a700 1 --2- 172.21.15.162:0/3123510588 >> [v2:172.21.15.118:6808/12529,v1:172.21.15.118:6809/12529] conn(0x7f221405c740 0x7f221405eb60 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:34.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.147+0000 7f222b7fe700 1 -- 172.21.15.162:0/3123510588 <== osd.5 v2:172.21.15.118:6808/12529 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7f221405f1c0 con 0x7f221405c740 2020-11-28T21:46:34.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.163+0000 7f223b26b700 1 -- 172.21.15.162:0/3123510588 --> [v2:172.21.15.118:6808/12529,v1:172.21.15.118:6809/12529] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2234057050 con 0x7f221405c740 2020-11-28T21:46:34.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.167+0000 7f222b7fe700 1 -- 172.21.15.162:0/3123510588 <== osd.5 v2:172.21.15.118:6808/12529 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2234057050 con 0x7f221405c740 2020-11-28T21:46:34.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.167+0000 7f223b26b700 1 -- 172.21.15.162:0/3123510588 >> [v2:172.21.15.118:6808/12529,v1:172.21.15.118:6809/12529] conn(0x7f221405c740 msgr2=0x7f221405eb60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:34.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.167+0000 7f223b26b700 1 --2- 172.21.15.162:0/3123510588 >> [v2:172.21.15.118:6808/12529,v1:172.21.15.118:6809/12529] conn(0x7f221405c740 0x7f221405eb60 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:34.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.167+0000 7f223b26b700 1 -- 172.21.15.162:0/3123510588 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2214055330 msgr2=0x7f22140577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:34.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.167+0000 7f223b26b700 1 --2- 172.21.15.162:0/3123510588 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2214055330 0x7f22140577f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f2230017000 tx=0x7f2230016000).stop 2020-11-28T21:46:34.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.167+0000 7f223b26b700 1 -- 172.21.15.162:0/3123510588 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22340b3a60 msgr2=0x7f2234154ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:34.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.167+0000 7f223b26b700 1 --2- 172.21.15.162:0/3123510588 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22340b3a60 0x7f2234154ba0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f22340b1f80 tx=0x7f2224007a90).stop 2020-11-28T21:46:34.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.167+0000 7f223aa6a700 1 -- 172.21.15.162:0/3123510588 reap_dead start 2020-11-28T21:46:34.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.167+0000 7f223b26b700 1 -- 172.21.15.162:0/3123510588 shutdown_connections 2020-11-28T21:46:34.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.167+0000 7f223b26b700 1 -- 172.21.15.162:0/3123510588 >> 172.21.15.162:0/3123510588 conn(0x7f223405b970 msgr2=0x7f22340b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:34.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.167+0000 7f223b26b700 1 -- 172.21.15.162:0/3123510588 shutdown_connections 2020-11-28T21:46:34.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.167+0000 7f223b26b700 1 -- 172.21.15.162:0/3123510588 wait complete. 2020-11-28T21:46:34.180 INFO:teuthology.orchestra.run.smithi162.stdout:60129542148 2020-11-28T21:46:34.180 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.6 flush_pg_stats 2020-11-28T21:46:34.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.295+0000 7f7158ec2700 1 Processor -- start 2020-11-28T21:46:34.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.295+0000 7f7158ec2700 1 -- start start 2020-11-28T21:46:34.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.299+0000 7f7158ec2700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f71540603d0 0x7f71540b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:34.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.299+0000 7f7158ec2700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f7154142a00 0x7f715409cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:34.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.299+0000 7f7158ec2700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71540b4110 con 0x7f71540b4c70 2020-11-28T21:46:34.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.299+0000 7f7158ec2700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7154060f60 con 0x7f7154142a00 2020-11-28T21:46:34.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.299+0000 7f7158ec2700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71540b43f0 con 0x7f71540603d0 2020-11-28T21:46:34.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.299+0000 7f7152ffd700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f7154142a00 0x7f715409cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:34.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.299+0000 7f7152ffd700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f7154142a00 0x7f715409cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.162:44456/0 (socket says 172.21.15.162:44456) 2020-11-28T21:46:34.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.299+0000 7f7152ffd700 1 -- 172.21.15.162:0/984802541 learned_addr learned my addr 172.21.15.162:0/984802541 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:34.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.299+0000 7f71537fe700 1 --2- 172.21.15.162:0/984802541 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f71540603d0 0x7f71540b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:34.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.299+0000 7f7152ffd700 1 -- 172.21.15.162:0/984802541 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f71540603d0 msgr2=0x7f71540b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:34.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.299+0000 7f7152ffd700 1 --2- 172.21.15.162:0/984802541 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f71540603d0 0x7f71540b3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:34.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.299+0000 7f7152ffd700 1 -- 172.21.15.162:0/984802541 >> v1:172.21.15.118:6789/0 conn(0x7f71540b4c70 legacy=0x7f7154143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:34.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.299+0000 7f7152ffd700 1 -- 172.21.15.162:0/984802541 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71540b4690 con 0x7f7154142a00 2020-11-28T21:46:34.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.299+0000 7f7152ffd700 1 --2- 172.21.15.162:0/984802541 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7154142a00 0x7f715409cfc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f7144002f30 tx=0x7f7144006e80).ready entity=mon.0 client_cookie=e2c7412203f0f495 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:34.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.299+0000 7f71527fc700 1 -- 172.21.15.162:0/984802541 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71440097f0 con 0x7f7154142a00 2020-11-28T21:46:34.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.299+0000 7f71527fc700 1 -- 172.21.15.162:0/984802541 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7144009950 con 0x7f7154142a00 2020-11-28T21:46:34.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.299+0000 7f7158ec2700 1 -- 172.21.15.162:0/984802541 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7154142a00 msgr2=0x7f715409cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:34.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.299+0000 7f71527fc700 1 -- 172.21.15.162:0/984802541 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7144009cb0 con 0x7f7154142a00 2020-11-28T21:46:34.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.299+0000 7f7158ec2700 1 --2- 172.21.15.162:0/984802541 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7154142a00 0x7f715409cfc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f7144002f30 tx=0x7f7144006e80).stop 2020-11-28T21:46:34.307 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.299+0000 7f7158ec2700 1 -- 172.21.15.162:0/984802541 shutdown_connections 2020-11-28T21:46:34.307 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.299+0000 7f7158ec2700 1 --2- 172.21.15.162:0/984802541 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f71540603d0 0x7f71540b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:34.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.299+0000 7f7158ec2700 1 --2- 172.21.15.162:0/984802541 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7154142a00 0x7f715409cfc0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:34.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.299+0000 7f7158ec2700 1 -- 172.21.15.162:0/984802541 >> 172.21.15.162:0/984802541 conn(0x7f7154066a40 msgr2=0x7f7154066e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:34.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.299+0000 7f7158ec2700 1 -- 172.21.15.162:0/984802541 shutdown_connections 2020-11-28T21:46:34.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.299+0000 7f7158ec2700 1 -- 172.21.15.162:0/984802541 wait complete. 2020-11-28T21:46:34.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.303+0000 7f7158ec2700 1 Processor -- start 2020-11-28T21:46:34.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.303+0000 7f7158ec2700 1 -- start start 2020-11-28T21:46:34.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.303+0000 7f7158ec2700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f71540603d0 0x7f71540b8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:34.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.303+0000 7f7158ec2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71540b4c70 0x7f7154145d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:34.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.303+0000 7f7158ec2700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7154146620 0x7f7154151720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:34.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.303+0000 7f7158ec2700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71541461f0 con 0x7f7154146620 2020-11-28T21:46:34.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.303+0000 7f7158ec2700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7154155790 con 0x7f71540b4c70 2020-11-28T21:46:34.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.303+0000 7f7158ec2700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71540b3170 con 0x7f71540603d0 2020-11-28T21:46:34.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.303+0000 7f7152ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71540b4c70 0x7f7154145d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:34.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.303+0000 7f7152ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71540b4c70 0x7f7154145d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37476/0 (socket says 172.21.15.162:37476) 2020-11-28T21:46:34.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.303+0000 7f7152ffd700 1 -- 172.21.15.162:0/1674139129 learned_addr learned my addr 172.21.15.162:0/1674139129 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:34.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.303+0000 7f71537fe700 1 --2- 172.21.15.162:0/1674139129 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f71540603d0 0x7f71540b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:34.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.303+0000 7f7153fff700 1 --2- 172.21.15.162:0/1674139129 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7154146620 0x7f7154151720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:34.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.303+0000 7f7152ffd700 1 -- 172.21.15.162:0/1674139129 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f71540603d0 msgr2=0x7f71540b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:34.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.303+0000 7f7152ffd700 1 --2- 172.21.15.162:0/1674139129 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f71540603d0 0x7f71540b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:34.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.303+0000 7f7152ffd700 1 -- 172.21.15.162:0/1674139129 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7154146620 msgr2=0x7f7154151720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:34.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.303+0000 7f7152ffd700 1 --2- 172.21.15.162:0/1674139129 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7154146620 0x7f7154151720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:34.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.303+0000 7f7152ffd700 1 -- 172.21.15.162:0/1674139129 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71540b2ed0 con 0x7f71540b4c70 2020-11-28T21:46:34.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.303+0000 7f71537fe700 1 --2- 172.21.15.162:0/1674139129 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f71540603d0 0x7f71540b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:46:34.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.303+0000 7f7152ffd700 1 --2- 172.21.15.162:0/1674139129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71540b4c70 0x7f7154145d00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f71540b3300 tx=0x7f7144015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:34.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.307+0000 7f7150ff9700 1 -- 172.21.15.162:0/1674139129 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7144011b50 con 0x7f71540b4c70 2020-11-28T21:46:34.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.307+0000 7f7150ff9700 1 -- 172.21.15.162:0/1674139129 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7144011cb0 con 0x7f71540b4c70 2020-11-28T21:46:34.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.307+0000 7f7158ec2700 1 -- 172.21.15.162:0/1674139129 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71541559f0 con 0x7f71540b4c70 2020-11-28T21:46:34.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.307+0000 7f7150ff9700 1 -- 172.21.15.162:0/1674139129 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f714401e650 con 0x7f71540b4c70 2020-11-28T21:46:34.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.307+0000 7f7158ec2700 1 -- 172.21.15.162:0/1674139129 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7154151c10 con 0x7f71540b4c70 2020-11-28T21:46:34.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.307+0000 7f7158ec2700 1 -- 172.21.15.162:0/1674139129 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7154057050 con 0x7f71540b4c70 2020-11-28T21:46:34.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.307+0000 7f7150ff9700 1 -- 172.21.15.162:0/1674139129 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7144025030 con 0x7f71540b4c70 2020-11-28T21:46:34.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.307+0000 7f7150ff9700 1 --2- 172.21.15.162:0/1674139129 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7130055330 0x7f71300577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:34.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.307+0000 7f7150ff9700 1 -- 172.21.15.162:0/1674139129 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f714406f420 con 0x7f71540b4c70 2020-11-28T21:46:34.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.307+0000 7f71537fe700 1 --2- 172.21.15.162:0/1674139129 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7130055330 0x7f71300577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:34.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.307+0000 7f7150ff9700 1 --2- 172.21.15.162:0/1674139129 >> [v2:172.21.15.118:6824/12530,v1:172.21.15.118:6825/12530] conn(0x7f713005c7c0 0x7f713005ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:34.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.307+0000 7f7150ff9700 1 -- 172.21.15.162:0/1674139129 --> [v2:172.21.15.118:6824/12530,v1:172.21.15.118:6825/12530] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f713005f240 con 0x7f713005c7c0 2020-11-28T21:46:34.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.307+0000 7f7150ff9700 1 -- 172.21.15.162:0/1674139129 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7144071a30 con 0x7f71540b4c70 2020-11-28T21:46:34.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.311+0000 7f71537fe700 1 --2- 172.21.15.162:0/1674139129 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7130055330 0x7f71300577f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f714000b000 tx=0x7f7140008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:34.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.311+0000 7f7153fff700 1 --2- 172.21.15.162:0/1674139129 >> [v2:172.21.15.118:6824/12530,v1:172.21.15.118:6825/12530] conn(0x7f713005c7c0 0x7f713005ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:34.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.311+0000 7f7153fff700 1 --2- 172.21.15.162:0/1674139129 >> [v2:172.21.15.118:6824/12530,v1:172.21.15.118:6825/12530] conn(0x7f713005c7c0 0x7f713005ebe0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:34.318 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.311+0000 7f7150ff9700 1 -- 172.21.15.162:0/1674139129 <== osd.6 v2:172.21.15.118:6824/12530 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7f713005f240 con 0x7f713005c7c0 2020-11-28T21:46:34.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.335+0000 7f7158ec2700 1 -- 172.21.15.162:0/1674139129 --> [v2:172.21.15.118:6824/12530,v1:172.21.15.118:6825/12530] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7154142cc0 con 0x7f713005c7c0 2020-11-28T21:46:34.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.335+0000 7f7150ff9700 1 -- 172.21.15.162:0/1674139129 <== osd.6 v2:172.21.15.118:6824/12530 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7154142cc0 con 0x7f713005c7c0 2020-11-28T21:46:34.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.335+0000 7f7158ec2700 1 -- 172.21.15.162:0/1674139129 >> [v2:172.21.15.118:6824/12530,v1:172.21.15.118:6825/12530] conn(0x7f713005c7c0 msgr2=0x7f713005ebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:34.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.335+0000 7f7158ec2700 1 --2- 172.21.15.162:0/1674139129 >> [v2:172.21.15.118:6824/12530,v1:172.21.15.118:6825/12530] conn(0x7f713005c7c0 0x7f713005ebe0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:34.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.335+0000 7f7158ec2700 1 -- 172.21.15.162:0/1674139129 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7130055330 msgr2=0x7f71300577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:34.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.335+0000 7f7158ec2700 1 --2- 172.21.15.162:0/1674139129 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7130055330 0x7f71300577f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f714000b000 tx=0x7f7140008040).stop 2020-11-28T21:46:34.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.335+0000 7f7158ec2700 1 -- 172.21.15.162:0/1674139129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71540b4c70 msgr2=0x7f7154145d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:34.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.335+0000 7f7158ec2700 1 --2- 172.21.15.162:0/1674139129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71540b4c70 0x7f7154145d00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f71540b3300 tx=0x7f7144015000).stop 2020-11-28T21:46:34.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.335+0000 7f7153fff700 1 -- 172.21.15.162:0/1674139129 reap_dead start 2020-11-28T21:46:34.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.335+0000 7f7158ec2700 1 -- 172.21.15.162:0/1674139129 shutdown_connections 2020-11-28T21:46:34.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.335+0000 7f7158ec2700 1 -- 172.21.15.162:0/1674139129 >> 172.21.15.162:0/1674139129 conn(0x7f7154066a40 msgr2=0x7f71540b88e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:34.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.335+0000 7f7158ec2700 1 -- 172.21.15.162:0/1674139129 shutdown_connections 2020-11-28T21:46:34.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.335+0000 7f7158ec2700 1 -- 172.21.15.162:0/1674139129 wait complete. 2020-11-28T21:46:34.351 INFO:teuthology.orchestra.run.smithi162.stdout:60129542148 2020-11-28T21:46:34.351 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.7 flush_pg_stats 2020-11-28T21:46:34.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.471+0000 7fefe3df9700 1 Processor -- start 2020-11-28T21:46:34.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.471+0000 7fefe3df9700 1 -- start start 2020-11-28T21:46:34.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.475+0000 7fefe3df9700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fefdc0b1ae0 0x7fefdc13cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:34.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.475+0000 7fefe3df9700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fefdc13f9e0 0x7fefdc13d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:34.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.475+0000 7fefe3df9700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefdc0b1ec0 con 0x7fefdc0aef20 2020-11-28T21:46:34.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.475+0000 7fefe3df9700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefdc13fef0 con 0x7fefdc0b1ae0 2020-11-28T21:46:34.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.475+0000 7fefe3df9700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefdc140190 con 0x7fefdc13f9e0 2020-11-28T21:46:34.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.475+0000 7fefe1b95700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fefdc0b1ae0 0x7fefdc13cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:34.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.475+0000 7fefe1b95700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fefdc0b1ae0 0x7fefdc13cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37484/0 (socket says 172.21.15.162:37484) 2020-11-28T21:46:34.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.475+0000 7fefe1b95700 1 -- 172.21.15.162:0/932971423 learned_addr learned my addr 172.21.15.162:0/932971423 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:34.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.475+0000 7fefe2396700 1 --2- 172.21.15.162:0/932971423 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fefdc13f9e0 0x7fefdc13d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:34.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.475+0000 7fefe1b95700 1 -- 172.21.15.162:0/932971423 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fefdc13f9e0 msgr2=0x7fefdc13d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:34.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.475+0000 7fefe1b95700 1 --2- 172.21.15.162:0/932971423 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fefdc13f9e0 0x7fefdc13d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:34.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.475+0000 7fefe1b95700 1 -- 172.21.15.162:0/932971423 >> v1:172.21.15.118:6789/0 conn(0x7fefdc0aef20 legacy=0x7fefdc0ac2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:34.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.475+0000 7fefe1b95700 1 -- 172.21.15.162:0/932971423 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefdc140430 con 0x7fefdc0b1ae0 2020-11-28T21:46:34.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.475+0000 7fefe1b95700 1 --2- 172.21.15.162:0/932971423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fefdc0b1ae0 0x7fefdc13cc90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fefd8002f30 tx=0x7fefd8006e80).ready entity=mon.1 client_cookie=2a1cba8bdfc43ef7 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:34.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.475+0000 7fefe0b93700 1 -- 172.21.15.162:0/932971423 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefd80097f0 con 0x7fefdc0b1ae0 2020-11-28T21:46:34.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.475+0000 7fefe0b93700 1 -- 172.21.15.162:0/932971423 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd8009950 con 0x7fefdc0b1ae0 2020-11-28T21:46:34.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.475+0000 7fefe0b93700 1 -- 172.21.15.162:0/932971423 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefd8009cb0 con 0x7fefdc0b1ae0 2020-11-28T21:46:34.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.475+0000 7fefe3df9700 1 -- 172.21.15.162:0/932971423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fefdc0b1ae0 msgr2=0x7fefdc13cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:34.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.475+0000 7fefe3df9700 1 --2- 172.21.15.162:0/932971423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fefdc0b1ae0 0x7fefdc13cc90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fefd8002f30 tx=0x7fefd8006e80).stop 2020-11-28T21:46:34.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.475+0000 7fefe3df9700 1 -- 172.21.15.162:0/932971423 shutdown_connections 2020-11-28T21:46:34.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.475+0000 7fefe3df9700 1 --2- 172.21.15.162:0/932971423 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fefdc13f9e0 0x7fefdc13d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:34.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.475+0000 7fefe3df9700 1 --2- 172.21.15.162:0/932971423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fefdc0b1ae0 0x7fefdc13cc90 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:34.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.475+0000 7fefe3df9700 1 -- 172.21.15.162:0/932971423 >> 172.21.15.162:0/932971423 conn(0x7fefdc05b970 msgr2=0x7fefdc05bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:34.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.475+0000 7fefe3df9700 1 -- 172.21.15.162:0/932971423 shutdown_connections 2020-11-28T21:46:34.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.475+0000 7fefe3df9700 1 -- 172.21.15.162:0/932971423 wait complete. 2020-11-28T21:46:34.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.479+0000 7fefe3df9700 1 Processor -- start 2020-11-28T21:46:34.486 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.479+0000 7fefe3df9700 1 -- start start 2020-11-28T21:46:34.486 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.479+0000 7fefe3df9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fefdc0aef20 0x7fefdc065cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:34.486 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.479+0000 7fefe3df9700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fefdc0b1ae0 0x7fefdc153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:34.487 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.479+0000 7fefe3df9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fefdc13f9e0 0x7fefdc0637e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:34.487 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.479+0000 7fefe3df9700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefdc064720 con 0x7fefdc0aef20 2020-11-28T21:46:34.487 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.479+0000 7fefe3df9700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefdc064480 con 0x7fefdc13f9e0 2020-11-28T21:46:34.487 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.479+0000 7fefe3df9700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefdc153990 con 0x7fefdc0b1ae0 2020-11-28T21:46:34.487 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.479+0000 7fefe2396700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fefdc13f9e0 0x7fefdc0637e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:34.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.479+0000 7fefe1b95700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fefdc0aef20 0x7fefdc065cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:34.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.479+0000 7fefe2396700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fefdc13f9e0 0x7fefdc0637e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37492/0 (socket says 172.21.15.162:37492) 2020-11-28T21:46:34.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.479+0000 7fefe1394700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fefdc0b1ae0 0x7fefdc153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:34.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.479+0000 7fefe2396700 1 -- 172.21.15.162:0/1341536901 learned_addr learned my addr 172.21.15.162:0/1341536901 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:34.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.479+0000 7fefe2396700 1 -- 172.21.15.162:0/1341536901 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fefdc0b1ae0 msgr2=0x7fefdc153190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:34.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.479+0000 7fefe2396700 1 --2- 172.21.15.162:0/1341536901 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fefdc0b1ae0 0x7fefdc153190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:34.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.479+0000 7fefe2396700 1 -- 172.21.15.162:0/1341536901 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fefdc0aef20 msgr2=0x7fefdc065cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:34.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.479+0000 7fefe2396700 1 --2- 172.21.15.162:0/1341536901 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fefdc0aef20 0x7fefdc065cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:34.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.479+0000 7fefe2396700 1 -- 172.21.15.162:0/1341536901 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefdc153680 con 0x7fefdc13f9e0 2020-11-28T21:46:34.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.479+0000 7fefe2396700 1 --2- 172.21.15.162:0/1341536901 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fefdc13f9e0 0x7fefdc0637e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fefd4006dd0 tx=0x7fefd40091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:34.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.479+0000 7fefcaffd700 1 -- 172.21.15.162:0/1341536901 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefd400c8f0 con 0x7fefdc13f9e0 2020-11-28T21:46:34.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.479+0000 7fefcaffd700 1 -- 172.21.15.162:0/1341536901 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd400ca50 con 0x7fefdc13f9e0 2020-11-28T21:46:34.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.479+0000 7fefe3df9700 1 -- 172.21.15.162:0/1341536901 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefdc13e570 con 0x7fefdc13f9e0 2020-11-28T21:46:34.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.479+0000 7fefcaffd700 1 -- 172.21.15.162:0/1341536901 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefd400cdb0 con 0x7fefdc13f9e0 2020-11-28T21:46:34.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.479+0000 7fefe3df9700 1 -- 172.21.15.162:0/1341536901 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefdc0a8970 con 0x7fefdc13f9e0 2020-11-28T21:46:34.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.483+0000 7fefe3df9700 1 -- 172.21.15.162:0/1341536901 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fefdc064300 con 0x7fefdc13f9e0 2020-11-28T21:46:34.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.483+0000 7fefcaffd700 1 -- 172.21.15.162:0/1341536901 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fefd4022070 con 0x7fefdc13f9e0 2020-11-28T21:46:34.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.483+0000 7fefcaffd700 1 --2- 172.21.15.162:0/1341536901 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fefbc055330 0x7fefbc0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:34.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.483+0000 7fefcaffd700 1 -- 172.21.15.162:0/1341536901 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fefd406ecc0 con 0x7fefdc13f9e0 2020-11-28T21:46:34.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.483+0000 7fefe1b95700 1 --2- 172.21.15.162:0/1341536901 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fefbc055330 0x7fefbc0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:34.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.483+0000 7fefcaffd700 1 --2- 172.21.15.162:0/1341536901 >> [v2:172.21.15.118:6816/12528,v1:172.21.15.118:6817/12528] conn(0x7fefbc05c7c0 0x7fefbc05ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:34.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.483+0000 7fefcaffd700 1 -- 172.21.15.162:0/1341536901 --> [v2:172.21.15.118:6816/12528,v1:172.21.15.118:6817/12528] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fefbc05f240 con 0x7fefbc05c7c0 2020-11-28T21:46:34.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.483+0000 7fefcaffd700 1 -- 172.21.15.162:0/1341536901 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefd406ef90 con 0x7fefdc13f9e0 2020-11-28T21:46:34.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.483+0000 7fefe1394700 1 --2- 172.21.15.162:0/1341536901 >> [v2:172.21.15.118:6816/12528,v1:172.21.15.118:6817/12528] conn(0x7fefbc05c7c0 0x7fefbc05ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:34.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.483+0000 7fefe1b95700 1 --2- 172.21.15.162:0/1341536901 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fefbc055330 0x7fefbc0577f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fefd801c000 tx=0x7fefd8015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:34.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.487+0000 7fefe1394700 1 --2- 172.21.15.162:0/1341536901 >> [v2:172.21.15.118:6816/12528,v1:172.21.15.118:6817/12528] conn(0x7fefbc05c7c0 0x7fefbc05ebe0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:34.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.487+0000 7fefcaffd700 1 -- 172.21.15.162:0/1341536901 <== osd.7 v2:172.21.15.118:6816/12528 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7fefbc05f240 con 0x7fefbc05c7c0 2020-11-28T21:46:34.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.507+0000 7fefe3df9700 1 -- 172.21.15.162:0/1341536901 --> [v2:172.21.15.118:6816/12528,v1:172.21.15.118:6817/12528] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fefdc0ac840 con 0x7fefbc05c7c0 2020-11-28T21:46:34.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.511+0000 7fefcaffd700 1 -- 172.21.15.162:0/1341536901 <== osd.7 v2:172.21.15.118:6816/12528 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fefdc0ac840 con 0x7fefbc05c7c0 2020-11-28T21:46:34.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.511+0000 7fefe3df9700 1 -- 172.21.15.162:0/1341536901 >> [v2:172.21.15.118:6816/12528,v1:172.21.15.118:6817/12528] conn(0x7fefbc05c7c0 msgr2=0x7fefbc05ebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:34.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.511+0000 7fefe3df9700 1 --2- 172.21.15.162:0/1341536901 >> [v2:172.21.15.118:6816/12528,v1:172.21.15.118:6817/12528] conn(0x7fefbc05c7c0 0x7fefbc05ebe0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:34.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.511+0000 7fefe3df9700 1 -- 172.21.15.162:0/1341536901 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fefbc055330 msgr2=0x7fefbc0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:34.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.511+0000 7fefe3df9700 1 --2- 172.21.15.162:0/1341536901 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fefbc055330 0x7fefbc0577f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fefd801c000 tx=0x7fefd8015000).stop 2020-11-28T21:46:34.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.511+0000 7fefe3df9700 1 -- 172.21.15.162:0/1341536901 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fefdc13f9e0 msgr2=0x7fefdc0637e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:34.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.511+0000 7fefe3df9700 1 --2- 172.21.15.162:0/1341536901 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fefdc13f9e0 0x7fefdc0637e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fefd4006dd0 tx=0x7fefd40091a0).stop 2020-11-28T21:46:34.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.511+0000 7fefe2396700 1 -- 172.21.15.162:0/1341536901 reap_dead start 2020-11-28T21:46:34.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.511+0000 7fefe3df9700 1 -- 172.21.15.162:0/1341536901 shutdown_connections 2020-11-28T21:46:34.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.511+0000 7fefe3df9700 1 -- 172.21.15.162:0/1341536901 >> 172.21.15.162:0/1341536901 conn(0x7fefdc05b970 msgr2=0x7fefdc0ab410 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:34.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.511+0000 7fefe3df9700 1 -- 172.21.15.162:0/1341536901 shutdown_connections 2020-11-28T21:46:34.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.511+0000 7fefe3df9700 1 -- 172.21.15.162:0/1341536901 wait complete. 2020-11-28T21:46:34.526 INFO:teuthology.orchestra.run.smithi162.stdout:60129542148 2020-11-28T21:46:34.527 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.0 2020-11-28T21:46:34.648 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.647+0000 7f2fe4aa2700 1 Processor -- start 2020-11-28T21:46:34.649 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.647+0000 7f2fe4aa2700 1 -- start start 2020-11-28T21:46:34.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.647+0000 7f2fe4aa2700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2fe0067600 0x7f2fe013f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:34.651 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.647+0000 7f2fe4aa2700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2fe013faf0 0x7f2fe013ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:34.651 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.647+0000 7f2fe4aa2700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe00b5f70 con 0x7f2fe00b5b90 2020-11-28T21:46:34.652 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.647+0000 7f2fe4aa2700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fe00abbf0 con 0x7f2fe0067600 2020-11-28T21:46:34.652 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.647+0000 7f2fe4aa2700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe0140490 con 0x7f2fe013faf0 2020-11-28T21:46:34.652 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.647+0000 7f2fd6ffd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2fe0067600 0x7f2fe013f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:34.652 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.647+0000 7f2fd6ffd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2fe0067600 0x7f2fe013f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37500/0 (socket says 172.21.15.162:37500) 2020-11-28T21:46:34.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.647+0000 7f2fd6ffd700 1 -- 172.21.15.162:0/193011506 learned_addr learned my addr 172.21.15.162:0/193011506 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:34.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.647+0000 7f2fdffff700 1 --2- 172.21.15.162:0/193011506 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2fe013faf0 0x7f2fe013ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:34.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.647+0000 7f2fd6ffd700 1 -- 172.21.15.162:0/193011506 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2fe013faf0 msgr2=0x7f2fe013ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:34.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.647+0000 7f2fd6ffd700 1 --2- 172.21.15.162:0/193011506 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2fe013faf0 0x7f2fe013ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:34.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.647+0000 7f2fd6ffd700 1 -- 172.21.15.162:0/193011506 >> v1:172.21.15.118:6789/0 conn(0x7f2fe00b5b90 legacy=0x7f2fe0066ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:34.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.647+0000 7f2fd6ffd700 1 -- 172.21.15.162:0/193011506 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe00b1490 con 0x7f2fe0067600 2020-11-28T21:46:34.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.647+0000 7f2fd6ffd700 1 --2- 172.21.15.162:0/193011506 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2fe0067600 0x7f2fe013f600 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f2fe0060600 tx=0x7f2fc8006e80).ready entity=mon.1 client_cookie=36d44f6dfe04fb56 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:34.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.647+0000 7f2fdeffd700 1 -- 172.21.15.162:0/193011506 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fc80097b0 con 0x7f2fe0067600 2020-11-28T21:46:34.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.647+0000 7f2fdeffd700 1 -- 172.21.15.162:0/193011506 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc8009910 con 0x7f2fe0067600 2020-11-28T21:46:34.655 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.647+0000 7f2fdeffd700 1 -- 172.21.15.162:0/193011506 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fc8009c70 con 0x7f2fe0067600 2020-11-28T21:46:34.655 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.647+0000 7f2fe4aa2700 1 -- 172.21.15.162:0/193011506 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2fe0067600 msgr2=0x7f2fe013f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:34.655 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.647+0000 7f2fe4aa2700 1 --2- 172.21.15.162:0/193011506 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2fe0067600 0x7f2fe013f600 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f2fe0060600 tx=0x7f2fc8006e80).stop 2020-11-28T21:46:34.658 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.647+0000 7f2fe4aa2700 1 -- 172.21.15.162:0/193011506 shutdown_connections 2020-11-28T21:46:34.658 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.647+0000 7f2fe4aa2700 1 --2- 172.21.15.162:0/193011506 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2fe013faf0 0x7f2fe013ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:34.659 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.647+0000 7f2fe4aa2700 1 --2- 172.21.15.162:0/193011506 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2fe0067600 0x7f2fe013f600 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:34.659 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.647+0000 7f2fe4aa2700 1 -- 172.21.15.162:0/193011506 >> 172.21.15.162:0/193011506 conn(0x7f2fe005c440 msgr2=0x7f2fe00b5140 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:34.659 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.647+0000 7f2fe4aa2700 1 -- 172.21.15.162:0/193011506 shutdown_connections 2020-11-28T21:46:34.659 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.647+0000 7f2fe4aa2700 1 -- 172.21.15.162:0/193011506 wait complete. 2020-11-28T21:46:34.660 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.651+0000 7f2fe4aa2700 1 Processor -- start 2020-11-28T21:46:34.661 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.651+0000 7f2fe4aa2700 1 -- start start 2020-11-28T21:46:34.662 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.651+0000 7f2fe4aa2700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2fe0067600 0x7f2fe00b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:34.662 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.651+0000 7f2fe4aa2700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2fe00b5b90 0x7f2fe0148290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:34.662 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.651+0000 7f2fe4aa2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2fe013faf0 0x7f2fe0151740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:34.663 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.651+0000 7f2fe4aa2700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe00b0070 con 0x7f2fe00b5b90 2020-11-28T21:46:34.663 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.651+0000 7f2fe4aa2700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe013d650 con 0x7f2fe013faf0 2020-11-28T21:46:34.663 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.651+0000 7f2fe4aa2700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe013d8f0 con 0x7f2fe0067600 2020-11-28T21:46:34.663 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.651+0000 7f2fdffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2fe013faf0 0x7f2fe0151740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:34.663 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.651+0000 7f2fdffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2fe013faf0 0x7f2fe0151740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37508/0 (socket says 172.21.15.162:37508) 2020-11-28T21:46:34.664 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.651+0000 7f2fdffff700 1 -- 172.21.15.162:0/3532281297 learned_addr learned my addr 172.21.15.162:0/3532281297 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:34.664 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.651+0000 7f2fdf7fe700 1 --2- 172.21.15.162:0/3532281297 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2fe0067600 0x7f2fe00b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:34.664 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.651+0000 7f2fd6ffd700 1 --2- 172.21.15.162:0/3532281297 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2fe00b5b90 0x7f2fe0148290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:34.664 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.651+0000 7f2fdffff700 1 -- 172.21.15.162:0/3532281297 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2fe0067600 msgr2=0x7f2fe00b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:34.664 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.651+0000 7f2fdffff700 1 --2- 172.21.15.162:0/3532281297 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2fe0067600 0x7f2fe00b30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:34.665 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.651+0000 7f2fdffff700 1 -- 172.21.15.162:0/3532281297 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2fe00b5b90 msgr2=0x7f2fe0148290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:34.665 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.651+0000 7f2fdffff700 1 --2- 172.21.15.162:0/3532281297 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2fe00b5b90 0x7f2fe0148290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:34.665 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.651+0000 7f2fdffff700 1 -- 172.21.15.162:0/3532281297 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe013db90 con 0x7f2fe013faf0 2020-11-28T21:46:34.665 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.655+0000 7f2fdffff700 1 --2- 172.21.15.162:0/3532281297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2fe013faf0 0x7f2fe0151740 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f2fd800a000 tx=0x7f2fd8008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:34.665 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.655+0000 7f2fdd7fa700 1 -- 172.21.15.162:0/3532281297 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fd800b980 con 0x7f2fe013faf0 2020-11-28T21:46:34.666 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.655+0000 7f2fdd7fa700 1 -- 172.21.15.162:0/3532281297 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd800bae0 con 0x7f2fe013faf0 2020-11-28T21:46:34.666 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.655+0000 7f2fe4aa2700 1 -- 172.21.15.162:0/3532281297 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe0148780 con 0x7f2fe013faf0 2020-11-28T21:46:34.666 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.655+0000 7f2fdd7fa700 1 -- 172.21.15.162:0/3532281297 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fd800be40 con 0x7f2fe013faf0 2020-11-28T21:46:34.666 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.655+0000 7f2fe4aa2700 1 -- 172.21.15.162:0/3532281297 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe0135740 con 0x7f2fe013faf0 2020-11-28T21:46:34.667 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.655+0000 7f2fe4aa2700 1 -- 172.21.15.162:0/3532281297 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fe00b2e00 con 0x7f2fe013faf0 2020-11-28T21:46:34.668 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.655+0000 7f2fdd7fa700 1 -- 172.21.15.162:0/3532281297 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2fd801c460 con 0x7f2fe013faf0 2020-11-28T21:46:34.668 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.655+0000 7f2fdd7fa700 1 --2- 172.21.15.162:0/3532281297 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2fc4055500 0x7f2fc40579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:34.668 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.655+0000 7f2fdd7fa700 1 -- 172.21.15.162:0/3532281297 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2fd806e640 con 0x7f2fe013faf0 2020-11-28T21:46:34.668 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.655+0000 7f2fdf7fe700 1 --2- 172.21.15.162:0/3532281297 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2fc4055500 0x7f2fc40579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:34.668 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.655+0000 7f2fdf7fe700 1 --2- 172.21.15.162:0/3532281297 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2fc4055500 0x7f2fc40579c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f2fd000d000 tx=0x7f2fd0007b40).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:34.669 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.663+0000 7f2fdd7fa700 1 -- 172.21.15.162:0/3532281297 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f2fe00b2e00 con 0x7f2fe013faf0 2020-11-28T21:46:34.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.807+0000 7f2fe4aa2700 1 -- 172.21.15.162:0/3532281297 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2fe00b2e00 con 0x7f2fe013faf0 2020-11-28T21:46:34.812 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.807+0000 7f2fdd7fa700 1 -- 172.21.15.162:0/3532281297 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2fe00b2e00 con 0x7f2fe013faf0 2020-11-28T21:46:34.812 INFO:teuthology.orchestra.run.smithi162.stdout:60129542148 2020-11-28T21:46:34.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.811+0000 7f2fe4aa2700 1 -- 172.21.15.162:0/3532281297 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2fc4055500 msgr2=0x7f2fc40579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:34.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.811+0000 7f2fe4aa2700 1 --2- 172.21.15.162:0/3532281297 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2fc4055500 0x7f2fc40579c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f2fd000d000 tx=0x7f2fd0007b40).stop 2020-11-28T21:46:34.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.811+0000 7f2fe4aa2700 1 -- 172.21.15.162:0/3532281297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2fe013faf0 msgr2=0x7f2fe0151740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:34.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.811+0000 7f2fe4aa2700 1 --2- 172.21.15.162:0/3532281297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2fe013faf0 0x7f2fe0151740 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f2fd800a000 tx=0x7f2fd8008040).stop 2020-11-28T21:46:34.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.811+0000 7f2fe4aa2700 1 -- 172.21.15.162:0/3532281297 shutdown_connections 2020-11-28T21:46:34.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.811+0000 7f2fe4aa2700 1 --2- 172.21.15.162:0/3532281297 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2fc4055500 0x7f2fc40579c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:34.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.811+0000 7f2fe4aa2700 1 --2- 172.21.15.162:0/3532281297 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2fe0067600 0x7f2fe00b30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:34.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.811+0000 7f2fe4aa2700 1 --2- 172.21.15.162:0/3532281297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2fe013faf0 0x7f2fe0151740 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:34.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.811+0000 7f2fe4aa2700 1 --2- 172.21.15.162:0/3532281297 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2fe00b5b90 0x7f2fe0148290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:34.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.811+0000 7f2fe4aa2700 1 -- 172.21.15.162:0/3532281297 >> 172.21.15.162:0/3532281297 conn(0x7f2fe005c440 msgr2=0x7f2fe006a280 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:34.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.811+0000 7f2fe4aa2700 1 -- 172.21.15.162:0/3532281297 shutdown_connections 2020-11-28T21:46:34.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.811+0000 7f2fe4aa2700 1 -- 172.21.15.162:0/3532281297 wait complete. 2020-11-28T21:46:34.829 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.0 2020-11-28T21:46:34.830 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.1 2020-11-28T21:46:34.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.943+0000 7fe47ffff700 1 Processor -- start 2020-11-28T21:46:34.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.943+0000 7fe47ffff700 1 -- start start 2020-11-28T21:46:34.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.943+0000 7fe47ffff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe480065750 0x7fe48014a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:34.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.943+0000 7fe47ffff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe480065050 con 0x7fe480064c70 2020-11-28T21:46:34.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.943+0000 7fe47ffff700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe48013fc40 con 0x7fe48013f6c0 2020-11-28T21:46:34.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.943+0000 7fe47ffff700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe48013fee0 con 0x7fe480065750 2020-11-28T21:46:34.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.943+0000 7fe47e7fc700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fe48013f6c0 0x7fe4800669a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:44358/0 (socket says 172.21.15.162:44358) 2020-11-28T21:46:34.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.943+0000 7fe47e7fc700 1 -- 172.21.15.162:0/3646489026 learned_addr learned my addr 172.21.15.162:0/3646489026 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:34.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.943+0000 7fe47f7fe700 1 --2- 172.21.15.162:0/3646489026 >> v2:172.21.15.118:3300/0 conn(0x7fe480065750 0x7fe48014a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:34.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.943+0000 7fe47dffb700 1 -- 172.21.15.162:0/3646489026 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3970337405 0 0) 0x7fe48013fc40 con 0x7fe48013f6c0 2020-11-28T21:46:34.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.943+0000 7fe47dffb700 1 -- 172.21.15.162:0/3646489026 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe464003230 con 0x7fe48013f6c0 2020-11-28T21:46:34.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.943+0000 7fe47dffb700 1 -- 172.21.15.162:0/3646489026 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4240723353 0 0) 0x7fe480065050 con 0x7fe480064c70 2020-11-28T21:46:34.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.947+0000 7fe47dffb700 1 -- 172.21.15.162:0/3646489026 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe48013fc40 con 0x7fe480064c70 2020-11-28T21:46:34.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.947+0000 7fe47dffb700 1 -- 172.21.15.162:0/3646489026 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2515275480 0 0) 0x7fe464003230 con 0x7fe48013f6c0 2020-11-28T21:46:34.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.947+0000 7fe47dffb700 1 -- 172.21.15.162:0/3646489026 >> v2:172.21.15.118:3300/0 conn(0x7fe480065750 msgr2=0x7fe48014a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:34.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.947+0000 7fe47dffb700 1 --2- 172.21.15.162:0/3646489026 >> v2:172.21.15.118:3300/0 conn(0x7fe480065750 0x7fe48014a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:34.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.947+0000 7fe47dffb700 1 -- 172.21.15.162:0/3646489026 >> v1:172.21.15.118:6789/0 conn(0x7fe480064c70 legacy=0x7fe480066290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:34.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.947+0000 7fe47dffb700 1 -- 172.21.15.162:0/3646489026 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe480140180 con 0x7fe48013f6c0 2020-11-28T21:46:34.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.947+0000 7fe47dffb700 1 -- 172.21.15.162:0/3646489026 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7fe4740016d0 con 0x7fe48013f6c0 2020-11-28T21:46:34.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.947+0000 7fe47dffb700 1 -- 172.21.15.162:0/3646489026 >> v1:172.21.15.162:6789/0 conn(0x7fe48013f6c0 legacy=0x7fe4800669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:34.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.947+0000 7fe47dffb700 1 --2- 172.21.15.162:0/3646489026 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe4640038b0 0x7fe464003c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:34.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.947+0000 7fe47dffb700 1 --2- 172.21.15.162:0/3646489026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4640044d0 0x7fe464004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:34.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.947+0000 7fe47dffb700 1 --2- 172.21.15.162:0/3646489026 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe464004e40 0x7fe46400b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:34.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.947+0000 7fe47dffb700 1 -- 172.21.15.162:0/3646489026 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48013fee0 con 0x7fe464004e40 2020-11-28T21:46:34.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.947+0000 7fe47dffb700 1 -- 172.21.15.162:0/3646489026 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4640053a0 con 0x7fe4640044d0 2020-11-28T21:46:34.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.947+0000 7fe47e7fc700 1 --2- 172.21.15.162:0/3646489026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4640044d0 0x7fe464004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:34.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.947+0000 7fe47dffb700 1 -- 172.21.15.162:0/3646489026 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe46400bc60 con 0x7fe4640038b0 2020-11-28T21:46:34.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.947+0000 7fe47effd700 1 --2- 172.21.15.162:0/3646489026 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe4640038b0 0x7fe464003c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:34.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.947+0000 7fe47f7fe700 1 --2- 172.21.15.162:0/3646489026 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe464004e40 0x7fe46400b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:34.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.947+0000 7fe47e7fc700 1 -- 172.21.15.162:0/3646489026 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe4640038b0 msgr2=0x7fe464003c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:34.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.947+0000 7fe47e7fc700 1 --2- 172.21.15.162:0/3646489026 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe4640038b0 0x7fe464003c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:34.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.947+0000 7fe47e7fc700 1 -- 172.21.15.162:0/3646489026 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe464004e40 msgr2=0x7fe46400b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:34.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.947+0000 7fe47e7fc700 1 --2- 172.21.15.162:0/3646489026 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe464004e40 0x7fe46400b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:34.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.947+0000 7fe47f7fe700 1 -- 172.21.15.162:0/3646489026 reap_dead start 2020-11-28T21:46:34.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.947+0000 7fe47e7fc700 1 -- 172.21.15.162:0/3646489026 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe464003230 con 0x7fe4640044d0 2020-11-28T21:46:34.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.947+0000 7fe47e7fc700 1 --2- 172.21.15.162:0/3646489026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4640044d0 0x7fe464004950 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fe4740010d0 tx=0x7fe48013fdd0).ready entity=mon.1 client_cookie=dc72d2dbfb1afa35 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:34.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.947+0000 7fe47dffb700 1 -- 172.21.15.162:0/3646489026 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe474008b90 con 0x7fe4640044d0 2020-11-28T21:46:34.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.947+0000 7fe47dffb700 1 -- 172.21.15.162:0/3646489026 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe474008170 con 0x7fe4640044d0 2020-11-28T21:46:34.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.947+0000 7fe47dffb700 1 -- 172.21.15.162:0/3646489026 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe4740084d0 con 0x7fe4640044d0 2020-11-28T21:46:34.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.947+0000 7fe47ffff700 1 -- 172.21.15.162:0/3646489026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4640044d0 msgr2=0x7fe464004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:34.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.947+0000 7fe47ffff700 1 --2- 172.21.15.162:0/3646489026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4640044d0 0x7fe464004950 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fe4740010d0 tx=0x7fe48013fdd0).stop 2020-11-28T21:46:34.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.947+0000 7fe47ffff700 1 -- 172.21.15.162:0/3646489026 shutdown_connections 2020-11-28T21:46:34.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.947+0000 7fe47ffff700 1 --2- 172.21.15.162:0/3646489026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4640044d0 0x7fe464004950 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:34.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.947+0000 7fe47ffff700 1 -- 172.21.15.162:0/3646489026 >> 172.21.15.162:0/3646489026 conn(0x7fe4800b5da0 msgr2=0x7fe4800b6180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:34.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.947+0000 7fe47ffff700 1 -- 172.21.15.162:0/3646489026 shutdown_connections 2020-11-28T21:46:34.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.947+0000 7fe47ffff700 1 -- 172.21.15.162:0/3646489026 wait complete. 2020-11-28T21:46:34.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.951+0000 7fe47ffff700 1 Processor -- start 2020-11-28T21:46:34.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.951+0000 7fe47ffff700 1 -- start start 2020-11-28T21:46:34.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.951+0000 7fe47ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4640044d0 0x7fe480142690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:34.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.951+0000 7fe47ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe480143330 0x7fe480066c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:34.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.951+0000 7fe47ffff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe480143910 0x7fe480140d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:34.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.951+0000 7fe47ffff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe480145c80 con 0x7fe480143330 2020-11-28T21:46:34.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.951+0000 7fe47ffff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe480145a00 con 0x7fe4640044d0 2020-11-28T21:46:34.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.951+0000 7fe47ffff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe480145020 con 0x7fe480143910 2020-11-28T21:46:34.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.951+0000 7fe47effd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4640044d0 0x7fe480142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:34.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.951+0000 7fe47effd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4640044d0 0x7fe480142690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37524/0 (socket says 172.21.15.162:37524) 2020-11-28T21:46:34.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.951+0000 7fe47effd700 1 -- 172.21.15.162:0/2387005181 learned_addr learned my addr 172.21.15.162:0/2387005181 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:34.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.951+0000 7fe47e7fc700 1 --2- 172.21.15.162:0/2387005181 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe480143330 0x7fe480066c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:34.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.951+0000 7fe47f7fe700 1 --2- 172.21.15.162:0/2387005181 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe480143910 0x7fe480140d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:34.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.951+0000 7fe47effd700 1 -- 172.21.15.162:0/2387005181 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe480143910 msgr2=0x7fe480140d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:34.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.951+0000 7fe47effd700 1 --2- 172.21.15.162:0/2387005181 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe480143910 0x7fe480140d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:34.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.951+0000 7fe47effd700 1 -- 172.21.15.162:0/2387005181 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe480143330 msgr2=0x7fe480066c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:34.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.951+0000 7fe47effd700 1 --2- 172.21.15.162:0/2387005181 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe480143330 0x7fe480066c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:34.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.951+0000 7fe47effd700 1 -- 172.21.15.162:0/2387005181 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4801452c0 con 0x7fe4640044d0 2020-11-28T21:46:34.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.951+0000 7fe47f7fe700 1 --2- 172.21.15.162:0/2387005181 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe480143910 0x7fe480140d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:46:34.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.951+0000 7fe47effd700 1 --2- 172.21.15.162:0/2387005181 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4640044d0 0x7fe480142690 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fe480145dc0 tx=0x7fe470009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:34.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.951+0000 7fe46ffff700 1 -- 172.21.15.162:0/2387005181 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe470003320 con 0x7fe4640044d0 2020-11-28T21:46:34.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.951+0000 7fe46ffff700 1 -- 172.21.15.162:0/2387005181 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe470003480 con 0x7fe4640044d0 2020-11-28T21:46:34.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.951+0000 7fe47ffff700 1 -- 172.21.15.162:0/2387005181 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe480145760 con 0x7fe4640044d0 2020-11-28T21:46:34.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.951+0000 7fe46ffff700 1 -- 172.21.15.162:0/2387005181 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe47000cb20 con 0x7fe4640044d0 2020-11-28T21:46:34.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.951+0000 7fe47ffff700 1 -- 172.21.15.162:0/2387005181 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe480141280 con 0x7fe4640044d0 2020-11-28T21:46:34.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.955+0000 7fe46ffff700 1 -- 172.21.15.162:0/2387005181 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe470022070 con 0x7fe4640044d0 2020-11-28T21:46:34.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.955+0000 7fe47ffff700 1 -- 172.21.15.162:0/2387005181 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe480057050 con 0x7fe4640044d0 2020-11-28T21:46:34.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.955+0000 7fe46ffff700 1 --2- 172.21.15.162:0/2387005181 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe45c0553b0 0x7fe45c057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:34.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.955+0000 7fe46ffff700 1 -- 172.21.15.162:0/2387005181 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe470070350 con 0x7fe4640044d0 2020-11-28T21:46:34.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.955+0000 7fe47e7fc700 1 --2- 172.21.15.162:0/2387005181 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe45c0553b0 0x7fe45c057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:34.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.955+0000 7fe47e7fc700 1 --2- 172.21.15.162:0/2387005181 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe45c0553b0 0x7fe45c057870 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fe47401a000 tx=0x7fe474014000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:34.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:34.959+0000 7fe46ffff700 1 -- 172.21.15.162:0/2387005181 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe470046650 con 0x7fe4640044d0 2020-11-28T21:46:35.081 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:46:35.084 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:46:35.086 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:46:35.088 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:46:35.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.103+0000 7fe47ffff700 1 -- 172.21.15.162:0/2387005181 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe480063480 con 0x7fe4640044d0 2020-11-28T21:46:35.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.103+0000 7fe46ffff700 1 -- 172.21.15.162:0/2387005181 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe470016070 con 0x7fe4640044d0 2020-11-28T21:46:35.107 INFO:teuthology.orchestra.run.smithi162.stdout:60129542149 2020-11-28T21:46:35.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.103+0000 7fe47ffff700 1 -- 172.21.15.162:0/2387005181 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe45c0553b0 msgr2=0x7fe45c057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:35.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.103+0000 7fe47ffff700 1 --2- 172.21.15.162:0/2387005181 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe45c0553b0 0x7fe45c057870 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fe47401a000 tx=0x7fe474014000).stop 2020-11-28T21:46:35.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.103+0000 7fe47ffff700 1 -- 172.21.15.162:0/2387005181 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4640044d0 msgr2=0x7fe480142690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:35.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.103+0000 7fe47ffff700 1 --2- 172.21.15.162:0/2387005181 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4640044d0 0x7fe480142690 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fe480145dc0 tx=0x7fe470009040).stop 2020-11-28T21:46:35.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.103+0000 7fe47ffff700 1 -- 172.21.15.162:0/2387005181 shutdown_connections 2020-11-28T21:46:35.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.103+0000 7fe47ffff700 1 --2- 172.21.15.162:0/2387005181 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe45c0553b0 0x7fe45c057870 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:35.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.103+0000 7fe47ffff700 1 --2- 172.21.15.162:0/2387005181 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe480143910 0x7fe480140d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:35.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.103+0000 7fe47ffff700 1 --2- 172.21.15.162:0/2387005181 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4640044d0 0x7fe480142690 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:35.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.103+0000 7fe47ffff700 1 --2- 172.21.15.162:0/2387005181 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe480143330 0x7fe480066c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:35.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.103+0000 7fe47ffff700 1 -- 172.21.15.162:0/2387005181 >> 172.21.15.162:0/2387005181 conn(0x7fe4800b5da0 msgr2=0x7fe4800610b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:35.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.103+0000 7fe47ffff700 1 -- 172.21.15.162:0/2387005181 shutdown_connections 2020-11-28T21:46:35.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.103+0000 7fe47ffff700 1 -- 172.21.15.162:0/2387005181 wait complete. 2020-11-28T21:46:35.120 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.1 2020-11-28T21:46:35.121 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.2 2020-11-28T21:46:35.280 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.279+0000 7fd344a22700 1 Processor -- start 2020-11-28T21:46:35.280 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.279+0000 7fd344a22700 1 -- start start 2020-11-28T21:46:35.282 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.279+0000 7fd344a22700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd3400b5a70 0x7fd340143360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:35.282 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.279+0000 7fd344a22700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd3400b5e50 0x7fd3400b6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:35.283 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.279+0000 7fd344a22700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd3400b5430 0x7fd34009d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:35.283 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.279+0000 7fd344a22700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3400b8040 con 0x7fd3400b5a70 2020-11-28T21:46:35.283 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.279+0000 7fd344a22700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3400b8720 con 0x7fd3400b5e50 2020-11-28T21:46:35.283 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.279+0000 7fd344a22700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd340142980 con 0x7fd3400b5430 2020-11-28T21:46:35.284 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.279+0000 7fd33e59c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd3400b5e50 0x7fd3400b6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:35.284 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.279+0000 7fd33e59c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd3400b5e50 0x7fd3400b6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37534/0 (socket says 172.21.15.162:37534) 2020-11-28T21:46:35.285 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.279+0000 7fd33e59c700 1 -- 172.21.15.162:0/3099351016 learned_addr learned my addr 172.21.15.162:0/3099351016 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:35.285 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.279+0000 7fd33f59e700 1 --2- 172.21.15.162:0/3099351016 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd3400b5430 0x7fd34009d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:35.285 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.279+0000 7fd33ed9d700 1 --2- 172.21.15.162:0/3099351016 >> v2:172.21.15.118:3300/0 conn(0x7fd3400b5a70 0x7fd340143360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:35.286 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.279+0000 7fd33e59c700 1 -- 172.21.15.162:0/3099351016 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd3400b5430 msgr2=0x7fd34009d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:35.286 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.279+0000 7fd33e59c700 1 --2- 172.21.15.162:0/3099351016 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd3400b5430 0x7fd34009d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:35.287 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.279+0000 7fd33e59c700 1 -- 172.21.15.162:0/3099351016 >> v2:172.21.15.118:3300/0 conn(0x7fd3400b5a70 msgr2=0x7fd340143360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:35.287 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.279+0000 7fd33e59c700 1 --2- 172.21.15.162:0/3099351016 >> v2:172.21.15.118:3300/0 conn(0x7fd3400b5a70 0x7fd340143360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:35.288 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.279+0000 7fd33e59c700 1 -- 172.21.15.162:0/3099351016 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd340142c20 con 0x7fd3400b5e50 2020-11-28T21:46:35.288 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.279+0000 7fd33f59e700 1 --2- 172.21.15.162:0/3099351016 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd3400b5430 0x7fd34009d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:46:35.288 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.279+0000 7fd33e59c700 1 --2- 172.21.15.162:0/3099351016 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3400b5e50 0x7fd3400b6ef0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fd330002f30 tx=0x7fd330006e80).ready entity=mon.1 client_cookie=a028ebec3c90a7b1 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:35.294 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.279+0000 7fd33dd9b700 1 -- 172.21.15.162:0/3099351016 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3300097f0 con 0x7fd3400b5e50 2020-11-28T21:46:35.295 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.279+0000 7fd33dd9b700 1 -- 172.21.15.162:0/3099351016 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd330009950 con 0x7fd3400b5e50 2020-11-28T21:46:35.295 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.279+0000 7fd33dd9b700 1 -- 172.21.15.162:0/3099351016 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd330009cb0 con 0x7fd3400b5e50 2020-11-28T21:46:35.295 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.279+0000 7fd344a22700 1 -- 172.21.15.162:0/3099351016 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3400b5e50 msgr2=0x7fd3400b6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:35.295 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.279+0000 7fd344a22700 1 --2- 172.21.15.162:0/3099351016 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3400b5e50 0x7fd3400b6ef0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fd330002f30 tx=0x7fd330006e80).stop 2020-11-28T21:46:35.296 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.279+0000 7fd344a22700 1 -- 172.21.15.162:0/3099351016 shutdown_connections 2020-11-28T21:46:35.296 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.279+0000 7fd344a22700 1 --2- 172.21.15.162:0/3099351016 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd3400b5430 0x7fd34009d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:35.296 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.279+0000 7fd344a22700 1 --2- 172.21.15.162:0/3099351016 >> v2:172.21.15.118:3300/0 conn(0x7fd3400b5a70 0x7fd340143360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:35.296 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.279+0000 7fd344a22700 1 --2- 172.21.15.162:0/3099351016 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3400b5e50 0x7fd3400b6ef0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:35.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.279+0000 7fd344a22700 1 -- 172.21.15.162:0/3099351016 >> 172.21.15.162:0/3099351016 conn(0x7fd34005b970 msgr2=0x7fd34005bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:35.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.279+0000 7fd344a22700 1 -- 172.21.15.162:0/3099351016 shutdown_connections 2020-11-28T21:46:35.298 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.279+0000 7fd344a22700 1 -- 172.21.15.162:0/3099351016 wait complete. 2020-11-28T21:46:35.298 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.283+0000 7fd344a22700 1 Processor -- start 2020-11-28T21:46:35.298 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.283+0000 7fd344a22700 1 -- start start 2020-11-28T21:46:35.298 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.283+0000 7fd344a22700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd3400b5430 0x7fd34009ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:35.298 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.283+0000 7fd344a22700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd3400b5a70 0x7fd340152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:35.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.283+0000 7fd344a22700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3400b5e50 0x7fd340144c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:35.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.283+0000 7fd344a22700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3400aeeb0 con 0x7fd3400b5a70 2020-11-28T21:46:35.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.283+0000 7fd344a22700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3400ae6d0 con 0x7fd3400b5e50 2020-11-28T21:46:35.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.283+0000 7fd344a22700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3400ae970 con 0x7fd3400b5430 2020-11-28T21:46:35.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.283+0000 7fd33f59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3400b5e50 0x7fd340144c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:35.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.283+0000 7fd33f59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3400b5e50 0x7fd340144c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37538/0 (socket says 172.21.15.162:37538) 2020-11-28T21:46:35.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.283+0000 7fd33f59e700 1 -- 172.21.15.162:0/1564495618 learned_addr learned my addr 172.21.15.162:0/1564495618 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:35.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.283+0000 7fd33ed9d700 1 --2- 172.21.15.162:0/1564495618 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd3400b5430 0x7fd34009ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:35.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.283+0000 7fd33e59c700 1 --2- 172.21.15.162:0/1564495618 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd3400b5a70 0x7fd340152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:35.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.283+0000 7fd33f59e700 1 -- 172.21.15.162:0/1564495618 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd3400b5430 msgr2=0x7fd34009ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:35.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.283+0000 7fd33f59e700 1 --2- 172.21.15.162:0/1564495618 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd3400b5430 0x7fd34009ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:35.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.283+0000 7fd33f59e700 1 -- 172.21.15.162:0/1564495618 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd3400b5a70 msgr2=0x7fd340152000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:35.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.283+0000 7fd33f59e700 1 --2- 172.21.15.162:0/1564495618 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd3400b5a70 0x7fd340152000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:35.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.283+0000 7fd33f59e700 1 -- 172.21.15.162:0/1564495618 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3400aec10 con 0x7fd3400b5e50 2020-11-28T21:46:35.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.283+0000 7fd33e59c700 1 --2- 172.21.15.162:0/1564495618 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd3400b5a70 0x7fd340152000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:46:35.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.283+0000 7fd33f59e700 1 --2- 172.21.15.162:0/1564495618 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3400b5e50 0x7fd340144c80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fd338006dd0 tx=0x7fd338008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:35.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.283+0000 7fd337fff700 1 -- 172.21.15.162:0/1564495618 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd33800c920 con 0x7fd3400b5e50 2020-11-28T21:46:35.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.283+0000 7fd337fff700 1 -- 172.21.15.162:0/1564495618 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd33800ca80 con 0x7fd3400b5e50 2020-11-28T21:46:35.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.283+0000 7fd344a22700 1 -- 172.21.15.162:0/1564495618 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd340061140 con 0x7fd3400b5e50 2020-11-28T21:46:35.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.283+0000 7fd337fff700 1 -- 172.21.15.162:0/1564495618 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd33800cde0 con 0x7fd3400b5e50 2020-11-28T21:46:35.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.283+0000 7fd344a22700 1 -- 172.21.15.162:0/1564495618 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3401524f0 con 0x7fd3400b5e50 2020-11-28T21:46:35.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.287+0000 7fd344a22700 1 -- 172.21.15.162:0/1564495618 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3401457b0 con 0x7fd3400b5e50 2020-11-28T21:46:35.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.287+0000 7fd337fff700 1 -- 172.21.15.162:0/1564495618 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd33801d460 con 0x7fd3400b5e50 2020-11-28T21:46:35.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.287+0000 7fd337fff700 1 --2- 172.21.15.162:0/1564495618 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd3240554e0 0x7fd3240579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:35.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.287+0000 7fd337fff700 1 -- 172.21.15.162:0/1564495618 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd33806e520 con 0x7fd3400b5e50 2020-11-28T21:46:35.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.287+0000 7fd33ed9d700 1 --2- 172.21.15.162:0/1564495618 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd3240554e0 0x7fd3240579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:35.306 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.287+0000 7fd33ed9d700 1 --2- 172.21.15.162:0/1564495618 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd3240554e0 0x7fd3240579a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fd32800b000 tx=0x7fd328006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:35.306 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.295+0000 7fd337fff700 1 -- 172.21.15.162:0/1564495618 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd3380447c0 con 0x7fd3400b5e50 2020-11-28T21:46:35.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.439+0000 7fd344a22700 1 -- 172.21.15.162:0/1564495618 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd340057050 con 0x7fd3400b5e50 2020-11-28T21:46:35.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.439+0000 7fd337fff700 1 -- 172.21.15.162:0/1564495618 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd338047de0 con 0x7fd3400b5e50 2020-11-28T21:46:35.443 INFO:teuthology.orchestra.run.smithi162.stdout:60129542147 2020-11-28T21:46:35.445 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.439+0000 7fd344a22700 1 -- 172.21.15.162:0/1564495618 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd3240554e0 msgr2=0x7fd3240579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:35.445 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.439+0000 7fd344a22700 1 --2- 172.21.15.162:0/1564495618 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd3240554e0 0x7fd3240579a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fd32800b000 tx=0x7fd328006dc0).stop 2020-11-28T21:46:35.446 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.439+0000 7fd344a22700 1 -- 172.21.15.162:0/1564495618 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3400b5e50 msgr2=0x7fd340144c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:35.446 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.439+0000 7fd344a22700 1 --2- 172.21.15.162:0/1564495618 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3400b5e50 0x7fd340144c80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fd338006dd0 tx=0x7fd338008040).stop 2020-11-28T21:46:35.446 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.443+0000 7fd344a22700 1 -- 172.21.15.162:0/1564495618 shutdown_connections 2020-11-28T21:46:35.447 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.443+0000 7fd344a22700 1 --2- 172.21.15.162:0/1564495618 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd3240554e0 0x7fd3240579a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:35.447 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.443+0000 7fd344a22700 1 --2- 172.21.15.162:0/1564495618 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd3400b5430 0x7fd34009ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:35.448 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.443+0000 7fd344a22700 1 --2- 172.21.15.162:0/1564495618 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3400b5e50 0x7fd340144c80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:35.448 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.443+0000 7fd344a22700 1 --2- 172.21.15.162:0/1564495618 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd3400b5a70 0x7fd340152000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:35.449 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.443+0000 7fd344a22700 1 -- 172.21.15.162:0/1564495618 >> 172.21.15.162:0/1564495618 conn(0x7fd34005b970 msgr2=0x7fd34013f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:35.449 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.443+0000 7fd344a22700 1 -- 172.21.15.162:0/1564495618 shutdown_connections 2020-11-28T21:46:35.449 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:35.443+0000 7fd344a22700 1 -- 172.21.15.162:0/1564495618 wait complete. 2020-11-28T21:46:35.456 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.2 2020-11-28T21:46:36.457 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.2 2020-11-28T21:46:36.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.575+0000 7f6dc53b0700 1 Processor -- start 2020-11-28T21:46:36.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.575+0000 7f6dc53b0700 1 -- start start 2020-11-28T21:46:36.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.579+0000 7f6dc53b0700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6dc005bcb0 0x7f6dc0143530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:36.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.579+0000 7f6dc53b0700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6dc00b1c00 0x7f6dc00b0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:36.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.579+0000 7f6dc53b0700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f6dc013c630 0x7f6dc013cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:36.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.579+0000 7f6dc53b0700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dc005fe40 con 0x7f6dc013c630 2020-11-28T21:46:36.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.579+0000 7f6dc53b0700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dc00b2080 con 0x7f6dc00b1c00 2020-11-28T21:46:36.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.579+0000 7f6dc53b0700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc005c090 con 0x7f6dc005bcb0 2020-11-28T21:46:36.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.579+0000 7f6dbf7fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6dc00b1c00 0x7f6dc00b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:36.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.579+0000 7f6dbf7fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6dc00b1c00 0x7f6dc00b0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37544/0 (socket says 172.21.15.162:37544) 2020-11-28T21:46:36.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.579+0000 7f6dbf7fe700 1 -- 172.21.15.162:0/1421127844 learned_addr learned my addr 172.21.15.162:0/1421127844 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:36.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.579+0000 7f6dbffff700 1 --2- 172.21.15.162:0/1421127844 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6dc005bcb0 0x7f6dc0143530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:36.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.579+0000 7f6dc4baf700 1 --2- 172.21.15.162:0/1421127844 >> v2:172.21.15.118:3300/0 conn(0x7f6dc013c630 0x7f6dc013cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:36.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.579+0000 7f6dbf7fe700 1 -- 172.21.15.162:0/1421127844 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6dc005bcb0 msgr2=0x7f6dc0143530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:36.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.579+0000 7f6dbf7fe700 1 --2- 172.21.15.162:0/1421127844 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6dc005bcb0 0x7f6dc0143530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:36.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.579+0000 7f6dbf7fe700 1 -- 172.21.15.162:0/1421127844 >> v2:172.21.15.118:3300/0 conn(0x7f6dc013c630 msgr2=0x7f6dc013cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:36.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.579+0000 7f6dbf7fe700 1 --2- 172.21.15.162:0/1421127844 >> v2:172.21.15.118:3300/0 conn(0x7f6dc013c630 0x7f6dc013cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:36.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.579+0000 7f6dbf7fe700 1 -- 172.21.15.162:0/1421127844 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc00b22a0 con 0x7f6dc00b1c00 2020-11-28T21:46:36.586 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.579+0000 7f6dbf7fe700 1 --2- 172.21.15.162:0/1421127844 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6dc00b1c00 0x7f6dc00b0740 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f6db0002f30 tx=0x7f6db0006e80).ready entity=mon.1 client_cookie=e2e385d60344ecba server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:36.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.579+0000 7f6dbeffd700 1 -- 172.21.15.162:0/1421127844 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6db0009640 con 0x7f6dc00b1c00 2020-11-28T21:46:36.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.579+0000 7f6dbeffd700 1 -- 172.21.15.162:0/1421127844 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db00097a0 con 0x7f6dc00b1c00 2020-11-28T21:46:36.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.579+0000 7f6dbeffd700 1 -- 172.21.15.162:0/1421127844 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6db0009b00 con 0x7f6dc00b1c00 2020-11-28T21:46:36.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.579+0000 7f6dc53b0700 1 -- 172.21.15.162:0/1421127844 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6dc00b1c00 msgr2=0x7f6dc00b0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:36.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.579+0000 7f6dc53b0700 1 --2- 172.21.15.162:0/1421127844 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6dc00b1c00 0x7f6dc00b0740 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f6db0002f30 tx=0x7f6db0006e80).stop 2020-11-28T21:46:36.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.579+0000 7f6dc53b0700 1 -- 172.21.15.162:0/1421127844 shutdown_connections 2020-11-28T21:46:36.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.579+0000 7f6dc53b0700 1 --2- 172.21.15.162:0/1421127844 >> v2:172.21.15.118:3300/0 conn(0x7f6dc013c630 0x7f6dc013cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:36.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.579+0000 7f6dc53b0700 1 --2- 172.21.15.162:0/1421127844 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6dc005bcb0 0x7f6dc0143530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:36.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.579+0000 7f6dc53b0700 1 --2- 172.21.15.162:0/1421127844 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6dc00b1c00 0x7f6dc00b0740 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:36.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.579+0000 7f6dc53b0700 1 -- 172.21.15.162:0/1421127844 >> 172.21.15.162:0/1421127844 conn(0x7f6dc00b5da0 msgr2=0x7f6dc00b6180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:36.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.579+0000 7f6dc53b0700 1 -- 172.21.15.162:0/1421127844 shutdown_connections 2020-11-28T21:46:36.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.579+0000 7f6dc53b0700 1 -- 172.21.15.162:0/1421127844 wait complete. 2020-11-28T21:46:36.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.583+0000 7f6dc53b0700 1 Processor -- start 2020-11-28T21:46:36.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.583+0000 7f6dc53b0700 1 -- start start 2020-11-28T21:46:36.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.583+0000 7f6dc53b0700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6dc005bcb0 0x7f6dc0060560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:36.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.583+0000 7f6dc53b0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6dc00b1c00 0x7f6dc0151990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:36.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.583+0000 7f6dc53b0700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6dc013c630 0x7f6dc0151e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:36.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.583+0000 7f6dc53b0700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc01403c0 con 0x7f6dc013c630 2020-11-28T21:46:36.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.583+0000 7f6dc53b0700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc013f9e0 con 0x7f6dc00b1c00 2020-11-28T21:46:36.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.583+0000 7f6dc53b0700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc013fc80 con 0x7f6dc005bcb0 2020-11-28T21:46:36.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.583+0000 7f6dbf7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6dc00b1c00 0x7f6dc0151990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:36.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.583+0000 7f6dbf7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6dc00b1c00 0x7f6dc0151990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37552/0 (socket says 172.21.15.162:37552) 2020-11-28T21:46:36.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.583+0000 7f6dbf7fe700 1 -- 172.21.15.162:0/3533695120 learned_addr learned my addr 172.21.15.162:0/3533695120 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:36.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.583+0000 7f6dc4baf700 1 --2- 172.21.15.162:0/3533695120 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6dc013c630 0x7f6dc0151e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:36.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.583+0000 7f6dbffff700 1 --2- 172.21.15.162:0/3533695120 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6dc005bcb0 0x7f6dc0060560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:36.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.583+0000 7f6dbf7fe700 1 -- 172.21.15.162:0/3533695120 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6dc005bcb0 msgr2=0x7f6dc0060560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:36.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.583+0000 7f6dbf7fe700 1 --2- 172.21.15.162:0/3533695120 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6dc005bcb0 0x7f6dc0060560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:36.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.583+0000 7f6dbf7fe700 1 -- 172.21.15.162:0/3533695120 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6dc013c630 msgr2=0x7f6dc0151e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:36.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.583+0000 7f6dbf7fe700 1 --2- 172.21.15.162:0/3533695120 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6dc013c630 0x7f6dc0151e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:36.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.583+0000 7f6dbf7fe700 1 -- 172.21.15.162:0/3533695120 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc013ff20 con 0x7f6dc00b1c00 2020-11-28T21:46:36.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.583+0000 7f6dbf7fe700 1 --2- 172.21.15.162:0/3533695120 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6dc00b1c00 0x7f6dc0151990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f6db0002f30 tx=0x7f6db0015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:36.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.587+0000 7f6dbd7fa700 1 -- 172.21.15.162:0/3533695120 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6db0009e40 con 0x7f6dc00b1c00 2020-11-28T21:46:36.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.587+0000 7f6dbd7fa700 1 -- 172.21.15.162:0/3533695120 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db0011bf0 con 0x7f6dc00b1c00 2020-11-28T21:46:36.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.587+0000 7f6dc53b0700 1 -- 172.21.15.162:0/3533695120 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dc013d7b0 con 0x7f6dc00b1c00 2020-11-28T21:46:36.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.587+0000 7f6dbd7fa700 1 -- 172.21.15.162:0/3533695120 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6db001d450 con 0x7f6dc00b1c00 2020-11-28T21:46:36.597 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.587+0000 7f6dc53b0700 1 -- 172.21.15.162:0/3533695120 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dc0152370 con 0x7f6dc00b1c00 2020-11-28T21:46:36.597 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.587+0000 7f6dc53b0700 1 -- 172.21.15.162:0/3533695120 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dc01400b0 con 0x7f6dc00b1c00 2020-11-28T21:46:36.598 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.587+0000 7f6dbd7fa700 1 -- 172.21.15.162:0/3533695120 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6db000f070 con 0x7f6dc00b1c00 2020-11-28T21:46:36.598 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.587+0000 7f6dbd7fa700 1 --2- 172.21.15.162:0/3533695120 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6da0055330 0x7f6da00577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:36.598 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.587+0000 7f6dbd7fa700 1 -- 172.21.15.162:0/3533695120 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6db006f840 con 0x7f6dc00b1c00 2020-11-28T21:46:36.598 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.587+0000 7f6dbffff700 1 --2- 172.21.15.162:0/3533695120 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6da0055330 0x7f6da00577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:36.599 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.587+0000 7f6dbffff700 1 --2- 172.21.15.162:0/3533695120 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6da0055330 0x7f6da00577f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f6db400b000 tx=0x7f6db4006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:36.599 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.595+0000 7f6dbd7fa700 1 -- 172.21.15.162:0/3533695120 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6db0045b80 con 0x7f6dc00b1c00 2020-11-28T21:46:36.739 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.735+0000 7f6dc53b0700 1 -- 172.21.15.162:0/3533695120 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6dc0057050 con 0x7f6dc00b1c00 2020-11-28T21:46:36.740 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.735+0000 7f6dbd7fa700 1 -- 172.21.15.162:0/3533695120 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6db0049160 con 0x7f6dc00b1c00 2020-11-28T21:46:36.740 INFO:teuthology.orchestra.run.smithi162.stdout:60129542149 2020-11-28T21:46:36.742 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.739+0000 7f6dc53b0700 1 -- 172.21.15.162:0/3533695120 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6da0055330 msgr2=0x7f6da00577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:36.742 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.739+0000 7f6dc53b0700 1 --2- 172.21.15.162:0/3533695120 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6da0055330 0x7f6da00577f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f6db400b000 tx=0x7f6db4006dc0).stop 2020-11-28T21:46:36.742 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.739+0000 7f6dc53b0700 1 -- 172.21.15.162:0/3533695120 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6dc00b1c00 msgr2=0x7f6dc0151990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:36.742 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.739+0000 7f6dc53b0700 1 --2- 172.21.15.162:0/3533695120 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6dc00b1c00 0x7f6dc0151990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f6db0002f30 tx=0x7f6db0015000).stop 2020-11-28T21:46:36.743 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.739+0000 7f6dc53b0700 1 -- 172.21.15.162:0/3533695120 shutdown_connections 2020-11-28T21:46:36.743 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.739+0000 7f6dc53b0700 1 --2- 172.21.15.162:0/3533695120 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6da0055330 0x7f6da00577f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:36.743 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.739+0000 7f6dc53b0700 1 --2- 172.21.15.162:0/3533695120 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6dc005bcb0 0x7f6dc0060560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:36.743 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.739+0000 7f6dc53b0700 1 --2- 172.21.15.162:0/3533695120 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6dc013c630 0x7f6dc0151e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:36.744 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.739+0000 7f6dc53b0700 1 --2- 172.21.15.162:0/3533695120 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6dc00b1c00 0x7f6dc0151990 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:36.744 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.739+0000 7f6dc53b0700 1 -- 172.21.15.162:0/3533695120 >> 172.21.15.162:0/3533695120 conn(0x7f6dc00b5da0 msgr2=0x7f6dc00b0440 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:36.744 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.739+0000 7f6dc53b0700 1 -- 172.21.15.162:0/3533695120 shutdown_connections 2020-11-28T21:46:36.744 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.739+0000 7f6dc53b0700 1 -- 172.21.15.162:0/3533695120 wait complete. 2020-11-28T21:46:36.753 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542149 for osd.2 2020-11-28T21:46:36.754 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.3 2020-11-28T21:46:36.864 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.863+0000 7f932e517700 1 Processor -- start 2020-11-28T21:46:36.865 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.863+0000 7f932e517700 1 -- start start 2020-11-28T21:46:36.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.863+0000 7f932e517700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f93280b69a0 0x7f93280b09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:36.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.863+0000 7f932e517700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f93281462b0 0x7f9328146760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:36.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.863+0000 7f932e517700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93280b6d80 con 0x7f932805fa90 2020-11-28T21:46:36.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.863+0000 7f932e517700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93280b15d0 con 0x7f93280b69a0 2020-11-28T21:46:36.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.863+0000 7f932e517700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93280b2200 con 0x7f93281462b0 2020-11-28T21:46:36.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.863+0000 7f932cab4700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f93281462b0 0x7f9328146760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:36.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.863+0000 7f932cab4700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f93281462b0 0x7f9328146760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37560/0 (socket says 172.21.15.162:37560) 2020-11-28T21:46:36.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.863+0000 7f932cab4700 1 -- 172.21.15.162:0/340953926 learned_addr learned my addr 172.21.15.162:0/340953926 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:36.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.863+0000 7f9327fff700 1 --2- 172.21.15.162:0/340953926 >> v2:172.21.15.118:3300/0 conn(0x7f93280b69a0 0x7f93280b09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:36.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.863+0000 7f932cab4700 1 -- 172.21.15.162:0/340953926 >> v2:172.21.15.118:3300/0 conn(0x7f93280b69a0 msgr2=0x7f93280b09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:36.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.863+0000 7f932cab4700 1 --2- 172.21.15.162:0/340953926 >> v2:172.21.15.118:3300/0 conn(0x7f93280b69a0 0x7f93280b09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:36.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.863+0000 7f932cab4700 1 -- 172.21.15.162:0/340953926 >> v1:172.21.15.118:6789/0 conn(0x7f932805fa90 legacy=0x7f93280b0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:36.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.863+0000 7f932cab4700 1 -- 172.21.15.162:0/340953926 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93280b1990 con 0x7f93281462b0 2020-11-28T21:46:36.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.863+0000 7f932cab4700 1 --2- 172.21.15.162:0/340953926 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93281462b0 0x7f9328146760 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f93200032b0 tx=0x7f9320009270).ready entity=mon.1 client_cookie=c40af22c4519de4c server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:36.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.863+0000 7f9326ffd700 1 -- 172.21.15.162:0/340953926 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9320009e60 con 0x7f93281462b0 2020-11-28T21:46:36.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.863+0000 7f9326ffd700 1 -- 172.21.15.162:0/340953926 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f932000bc00 con 0x7f93281462b0 2020-11-28T21:46:36.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.863+0000 7f9326ffd700 1 -- 172.21.15.162:0/340953926 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f932000bd70 con 0x7f93281462b0 2020-11-28T21:46:36.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.863+0000 7f932e517700 1 -- 172.21.15.162:0/340953926 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93281462b0 msgr2=0x7f9328146760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:36.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.863+0000 7f932e517700 1 --2- 172.21.15.162:0/340953926 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93281462b0 0x7f9328146760 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f93200032b0 tx=0x7f9320009270).stop 2020-11-28T21:46:36.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.867+0000 7f932e517700 1 -- 172.21.15.162:0/340953926 shutdown_connections 2020-11-28T21:46:36.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.867+0000 7f932e517700 1 --2- 172.21.15.162:0/340953926 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93281462b0 0x7f9328146760 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:36.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.867+0000 7f932e517700 1 --2- 172.21.15.162:0/340953926 >> v2:172.21.15.118:3300/0 conn(0x7f93280b69a0 0x7f93280b09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:36.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.867+0000 7f932e517700 1 -- 172.21.15.162:0/340953926 >> 172.21.15.162:0/340953926 conn(0x7f9328062170 msgr2=0x7f93280b5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:36.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.867+0000 7f932e517700 1 -- 172.21.15.162:0/340953926 shutdown_connections 2020-11-28T21:46:36.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.867+0000 7f932e517700 1 -- 172.21.15.162:0/340953926 wait complete. 2020-11-28T21:46:36.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.867+0000 7f932e517700 1 Processor -- start 2020-11-28T21:46:36.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.867+0000 7f932e517700 1 -- start start 2020-11-28T21:46:36.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.867+0000 7f932e517700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f932805fa90 0x7f93280b57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:36.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.867+0000 7f932e517700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93280b69a0 0x7f9328142a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:36.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.867+0000 7f932e517700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f93281462b0 0x7f93281455a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:36.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.867+0000 7f932e517700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93281489c0 con 0x7f93280b69a0 2020-11-28T21:46:36.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.867+0000 7f932e517700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9328064d50 con 0x7f932805fa90 2020-11-28T21:46:36.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.867+0000 7f932e517700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9328149540 con 0x7f93281462b0 2020-11-28T21:46:36.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.867+0000 7f9327fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f932805fa90 0x7f93280b57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:36.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.871+0000 7f9327fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f932805fa90 0x7f93280b57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37564/0 (socket says 172.21.15.162:37564) 2020-11-28T21:46:36.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.871+0000 7f9327fff700 1 -- 172.21.15.162:0/3430993553 learned_addr learned my addr 172.21.15.162:0/3430993553 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:36.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.871+0000 7f932cab4700 1 --2- 172.21.15.162:0/3430993553 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f93281462b0 0x7f93281455a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:36.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.871+0000 7f93277fe700 1 --2- 172.21.15.162:0/3430993553 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93280b69a0 0x7f9328142a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:36.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.871+0000 7f9327fff700 1 -- 172.21.15.162:0/3430993553 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f93281462b0 msgr2=0x7f93281455a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:36.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.871+0000 7f9327fff700 1 --2- 172.21.15.162:0/3430993553 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f93281462b0 0x7f93281455a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:36.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.871+0000 7f9327fff700 1 -- 172.21.15.162:0/3430993553 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93280b69a0 msgr2=0x7f9328142a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:36.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.871+0000 7f9327fff700 1 --2- 172.21.15.162:0/3430993553 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93280b69a0 0x7f9328142a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:36.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.871+0000 7f9327fff700 1 -- 172.21.15.162:0/3430993553 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93281550b0 con 0x7f932805fa90 2020-11-28T21:46:36.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.871+0000 7f9327fff700 1 --2- 172.21.15.162:0/3430993553 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f932805fa90 0x7f93280b57e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f931800c000 tx=0x7f9318006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:36.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.871+0000 7f93257fa700 1 -- 172.21.15.162:0/3430993553 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f931800ca10 con 0x7f932805fa90 2020-11-28T21:46:36.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.871+0000 7f93257fa700 1 -- 172.21.15.162:0/3430993553 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931800cb70 con 0x7f932805fa90 2020-11-28T21:46:36.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.871+0000 7f93257fa700 1 -- 172.21.15.162:0/3430993553 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f931800b370 con 0x7f932805fa90 2020-11-28T21:46:36.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.871+0000 7f932e517700 1 -- 172.21.15.162:0/3430993553 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9328065d40 con 0x7f932805fa90 2020-11-28T21:46:36.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.871+0000 7f932e517700 1 -- 172.21.15.162:0/3430993553 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93281499e0 con 0x7f932805fa90 2020-11-28T21:46:36.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.871+0000 7f932e517700 1 -- 172.21.15.162:0/3430993553 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9328064f10 con 0x7f932805fa90 2020-11-28T21:46:36.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.871+0000 7f93257fa700 1 -- 172.21.15.162:0/3430993553 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f931801c460 con 0x7f932805fa90 2020-11-28T21:46:36.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.875+0000 7f93257fa700 1 --2- 172.21.15.162:0/3430993553 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f930c0553f0 0x7f930c0578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:36.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.875+0000 7f93257fa700 1 -- 172.21.15.162:0/3430993553 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f931806e420 con 0x7f932805fa90 2020-11-28T21:46:36.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.875+0000 7f93277fe700 1 --2- 172.21.15.162:0/3430993553 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f930c0553f0 0x7f930c0578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:36.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.875+0000 7f93277fe700 1 --2- 172.21.15.162:0/3430993553 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f930c0553f0 0x7f930c0578b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f931000d000 tx=0x7f9310000b80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:36.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:36.879+0000 7f93257fa700 1 -- 172.21.15.162:0/3430993553 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9318044530 con 0x7f932805fa90 2020-11-28T21:46:37.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.023+0000 7f932e517700 1 -- 172.21.15.162:0/3430993553 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9328057050 con 0x7f932805fa90 2020-11-28T21:46:37.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.023+0000 7f93257fa700 1 -- 172.21.15.162:0/3430993553 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9318047b50 con 0x7f932805fa90 2020-11-28T21:46:37.025 INFO:teuthology.orchestra.run.smithi162.stdout:60129542149 2020-11-28T21:46:37.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.023+0000 7f932e517700 1 -- 172.21.15.162:0/3430993553 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f930c0553f0 msgr2=0x7f930c0578b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:37.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.023+0000 7f932e517700 1 --2- 172.21.15.162:0/3430993553 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f930c0553f0 0x7f930c0578b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f931000d000 tx=0x7f9310000b80).stop 2020-11-28T21:46:37.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.023+0000 7f932e517700 1 -- 172.21.15.162:0/3430993553 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f932805fa90 msgr2=0x7f93280b57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:37.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.023+0000 7f932e517700 1 --2- 172.21.15.162:0/3430993553 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f932805fa90 0x7f93280b57e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f931800c000 tx=0x7f9318006dc0).stop 2020-11-28T21:46:37.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.023+0000 7f932e517700 1 -- 172.21.15.162:0/3430993553 shutdown_connections 2020-11-28T21:46:37.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.023+0000 7f932e517700 1 --2- 172.21.15.162:0/3430993553 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f930c0553f0 0x7f930c0578b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:37.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.023+0000 7f932e517700 1 --2- 172.21.15.162:0/3430993553 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f93281462b0 0x7f93281455a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:37.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.023+0000 7f932e517700 1 --2- 172.21.15.162:0/3430993553 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f932805fa90 0x7f93280b57e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:37.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.023+0000 7f932e517700 1 --2- 172.21.15.162:0/3430993553 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93280b69a0 0x7f9328142a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:37.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.023+0000 7f932e517700 1 -- 172.21.15.162:0/3430993553 >> 172.21.15.162:0/3430993553 conn(0x7f9328062170 msgr2=0x7f932805f380 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:37.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.023+0000 7f932e517700 1 -- 172.21.15.162:0/3430993553 shutdown_connections 2020-11-28T21:46:37.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.023+0000 7f932e517700 1 -- 172.21.15.162:0/3430993553 wait complete. 2020-11-28T21:46:37.038 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542149 for osd.3 2020-11-28T21:46:37.038 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.4 2020-11-28T21:46:37.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.147+0000 7f5fe18c3700 1 Processor -- start 2020-11-28T21:46:37.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.147+0000 7f5fe18c3700 1 -- start start 2020-11-28T21:46:37.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.151+0000 7f5fe18c3700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5fdc0603d0 0x7f5fdc0b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:37.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.151+0000 7f5fe18c3700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5fdc142a00 0x7f5fdc09cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:37.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.151+0000 7f5fe18c3700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5fdc0b4c70 0x7f5fdc09d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:37.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.151+0000 7f5fe18c3700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fdc060f60 con 0x7f5fdc0b4c70 2020-11-28T21:46:37.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.151+0000 7f5fe18c3700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fdc0b4110 con 0x7f5fdc142a00 2020-11-28T21:46:37.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.151+0000 7f5fe18c3700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fdc0b43f0 con 0x7f5fdc0603d0 2020-11-28T21:46:37.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.151+0000 7f5fdbfff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5fdc142a00 0x7f5fdc09cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:37.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.151+0000 7f5fe08c1700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5fdc0603d0 0x7f5fdc0b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:37.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.151+0000 7f5fe10c2700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5fdc0b4c70 0x7f5fdc09d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:37.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.151+0000 7f5fdbfff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5fdc142a00 0x7f5fdc09cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37576/0 (socket says 172.21.15.162:37576) 2020-11-28T21:46:37.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.151+0000 7f5fe08c1700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5fdc0603d0 0x7f5fdc0b3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:34796/0 (socket says 172.21.15.162:34796) 2020-11-28T21:46:37.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.151+0000 7f5fdbfff700 1 -- 172.21.15.162:0/2285041211 learned_addr learned my addr 172.21.15.162:0/2285041211 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:37.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.151+0000 7f5fdbfff700 1 -- 172.21.15.162:0/2285041211 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5fdc0603d0 msgr2=0x7f5fdc0b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:37.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.151+0000 7f5fdbfff700 1 --2- 172.21.15.162:0/2285041211 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5fdc0603d0 0x7f5fdc0b3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:37.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.151+0000 7f5fdbfff700 1 -- 172.21.15.162:0/2285041211 >> v2:172.21.15.118:3300/0 conn(0x7f5fdc0b4c70 msgr2=0x7f5fdc09d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:37.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.151+0000 7f5fdbfff700 1 --2- 172.21.15.162:0/2285041211 >> v2:172.21.15.118:3300/0 conn(0x7f5fdc0b4c70 0x7f5fdc09d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:37.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.151+0000 7f5fdbfff700 1 -- 172.21.15.162:0/2285041211 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fdc0b4690 con 0x7f5fdc142a00 2020-11-28T21:46:37.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.151+0000 7f5fdbfff700 1 --2- 172.21.15.162:0/2285041211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fdc142a00 0x7f5fdc09cfc0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f5fc4002f30 tx=0x7f5fc4006e80).ready entity=mon.1 client_cookie=3c0d101f0b0746db server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:37.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.151+0000 7f5fdb7fe700 1 -- 172.21.15.162:0/2285041211 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fc4009640 con 0x7f5fdc142a00 2020-11-28T21:46:37.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.151+0000 7f5fdb7fe700 1 -- 172.21.15.162:0/2285041211 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc40097a0 con 0x7f5fdc142a00 2020-11-28T21:46:37.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.151+0000 7f5fdb7fe700 1 -- 172.21.15.162:0/2285041211 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fc4009b00 con 0x7f5fdc142a00 2020-11-28T21:46:37.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.151+0000 7f5fe18c3700 1 -- 172.21.15.162:0/2285041211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fdc142a00 msgr2=0x7f5fdc09cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:37.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.151+0000 7f5fe18c3700 1 --2- 172.21.15.162:0/2285041211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fdc142a00 0x7f5fdc09cfc0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f5fc4002f30 tx=0x7f5fc4006e80).stop 2020-11-28T21:46:37.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.151+0000 7f5fe18c3700 1 -- 172.21.15.162:0/2285041211 shutdown_connections 2020-11-28T21:46:37.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.151+0000 7f5fe18c3700 1 --2- 172.21.15.162:0/2285041211 >> v2:172.21.15.118:3300/0 conn(0x7f5fdc0b4c70 0x7f5fdc09d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:37.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.151+0000 7f5fe18c3700 1 --2- 172.21.15.162:0/2285041211 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5fdc0603d0 0x7f5fdc0b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:37.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.151+0000 7f5fe18c3700 1 --2- 172.21.15.162:0/2285041211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fdc142a00 0x7f5fdc09cfc0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:37.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.151+0000 7f5fe18c3700 1 -- 172.21.15.162:0/2285041211 >> 172.21.15.162:0/2285041211 conn(0x7f5fdc066a40 msgr2=0x7f5fdc066e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:37.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.151+0000 7f5fe18c3700 1 -- 172.21.15.162:0/2285041211 shutdown_connections 2020-11-28T21:46:37.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.151+0000 7f5fe18c3700 1 -- 172.21.15.162:0/2285041211 wait complete. 2020-11-28T21:46:37.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.155+0000 7f5fe18c3700 1 Processor -- start 2020-11-28T21:46:37.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.155+0000 7f5fe18c3700 1 -- start start 2020-11-28T21:46:37.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.155+0000 7f5fe18c3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fdc0603d0 0x7f5fdc0b7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:37.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.155+0000 7f5fe18c3700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5fdc0b4c70 0x7f5fdc143d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:37.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.155+0000 7f5fe18c3700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5fdc144270 0x7f5fdc0b0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:37.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.155+0000 7f5fe18c3700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fdc151740 con 0x7f5fdc0b4c70 2020-11-28T21:46:37.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.155+0000 7f5fe18c3700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fdc151930 con 0x7f5fdc0603d0 2020-11-28T21:46:37.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.155+0000 7f5fe18c3700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fdc151bd0 con 0x7f5fdc144270 2020-11-28T21:46:37.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.155+0000 7f5fe08c1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fdc0603d0 0x7f5fdc0b7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:37.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.155+0000 7f5fe08c1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fdc0603d0 0x7f5fdc0b7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37578/0 (socket says 172.21.15.162:37578) 2020-11-28T21:46:37.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.155+0000 7f5fe08c1700 1 -- 172.21.15.162:0/626835856 learned_addr learned my addr 172.21.15.162:0/626835856 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:37.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.155+0000 7f5fdbfff700 1 --2- 172.21.15.162:0/626835856 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5fdc0b4c70 0x7f5fdc143d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:37.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.155+0000 7f5fe10c2700 1 --2- 172.21.15.162:0/626835856 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5fdc144270 0x7f5fdc0b0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:37.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.155+0000 7f5fe08c1700 1 -- 172.21.15.162:0/626835856 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5fdc144270 msgr2=0x7f5fdc0b0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:37.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.155+0000 7f5fe08c1700 1 --2- 172.21.15.162:0/626835856 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5fdc144270 0x7f5fdc0b0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:37.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.155+0000 7f5fe08c1700 1 -- 172.21.15.162:0/626835856 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5fdc0b4c70 msgr2=0x7f5fdc143d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:37.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.155+0000 7f5fe08c1700 1 --2- 172.21.15.162:0/626835856 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5fdc0b4c70 0x7f5fdc143d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:37.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.155+0000 7f5fe08c1700 1 -- 172.21.15.162:0/626835856 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fdc151e70 con 0x7f5fdc0603d0 2020-11-28T21:46:37.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.155+0000 7f5fe08c1700 1 --2- 172.21.15.162:0/626835856 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fdc0603d0 0x7f5fdc0b7fa0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f5fdc144700 tx=0x7f5fd40091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:37.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.155+0000 7f5fd9ffb700 1 -- 172.21.15.162:0/626835856 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fd400cb30 con 0x7f5fdc0603d0 2020-11-28T21:46:37.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.155+0000 7f5fd9ffb700 1 -- 172.21.15.162:0/626835856 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd400cc90 con 0x7f5fdc0603d0 2020-11-28T21:46:37.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.155+0000 7f5fe18c3700 1 -- 172.21.15.162:0/626835856 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fdc152110 con 0x7f5fdc0603d0 2020-11-28T21:46:37.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.155+0000 7f5fd9ffb700 1 -- 172.21.15.162:0/626835856 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fd40123f0 con 0x7f5fdc0603d0 2020-11-28T21:46:37.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.155+0000 7f5fe18c3700 1 -- 172.21.15.162:0/626835856 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fdc0b0910 con 0x7f5fdc0603d0 2020-11-28T21:46:37.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.155+0000 7f5fd9ffb700 1 -- 172.21.15.162:0/626835856 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5fd4022070 con 0x7f5fdc0603d0 2020-11-28T21:46:37.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.155+0000 7f5fe18c3700 1 -- 172.21.15.162:0/626835856 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fdc0b0020 con 0x7f5fdc0603d0 2020-11-28T21:46:37.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.155+0000 7f5fd9ffb700 1 --2- 172.21.15.162:0/626835856 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5fc0055350 0x7f5fc0057810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:37.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.155+0000 7f5fd9ffb700 1 -- 172.21.15.162:0/626835856 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5fd406e5c0 con 0x7f5fdc0603d0 2020-11-28T21:46:37.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.155+0000 7f5fdbfff700 1 --2- 172.21.15.162:0/626835856 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5fc0055350 0x7f5fc0057810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:37.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.159+0000 7f5fdbfff700 1 --2- 172.21.15.162:0/626835856 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5fc0055350 0x7f5fc0057810 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f5fc401b000 tx=0x7f5fc4015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:37.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.163+0000 7f5fd9ffb700 1 -- 172.21.15.162:0/626835856 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5fd40448c0 con 0x7f5fdc0603d0 2020-11-28T21:46:37.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.307+0000 7f5fe18c3700 1 -- 172.21.15.162:0/626835856 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f5fdc1522a0 con 0x7f5fdc0603d0 2020-11-28T21:46:37.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.307+0000 7f5fd9ffb700 1 -- 172.21.15.162:0/626835856 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5fd4047ee0 con 0x7f5fdc0603d0 2020-11-28T21:46:37.310 INFO:teuthology.orchestra.run.smithi162.stdout:60129542149 2020-11-28T21:46:37.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.307+0000 7f5fe18c3700 1 -- 172.21.15.162:0/626835856 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5fc0055350 msgr2=0x7f5fc0057810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:37.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.307+0000 7f5fe18c3700 1 --2- 172.21.15.162:0/626835856 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5fc0055350 0x7f5fc0057810 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f5fc401b000 tx=0x7f5fc4015000).stop 2020-11-28T21:46:37.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.307+0000 7f5fe18c3700 1 -- 172.21.15.162:0/626835856 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fdc0603d0 msgr2=0x7f5fdc0b7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:37.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.307+0000 7f5fe18c3700 1 --2- 172.21.15.162:0/626835856 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fdc0603d0 0x7f5fdc0b7fa0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f5fdc144700 tx=0x7f5fd40091a0).stop 2020-11-28T21:46:37.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.307+0000 7f5fe18c3700 1 -- 172.21.15.162:0/626835856 shutdown_connections 2020-11-28T21:46:37.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.307+0000 7f5fe18c3700 1 --2- 172.21.15.162:0/626835856 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5fc0055350 0x7f5fc0057810 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:37.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.307+0000 7f5fe18c3700 1 --2- 172.21.15.162:0/626835856 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5fdc144270 0x7f5fdc0b0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:37.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.307+0000 7f5fe18c3700 1 --2- 172.21.15.162:0/626835856 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fdc0603d0 0x7f5fdc0b7fa0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:37.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.307+0000 7f5fe18c3700 1 --2- 172.21.15.162:0/626835856 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5fdc0b4c70 0x7f5fdc143d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:37.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.307+0000 7f5fe18c3700 1 -- 172.21.15.162:0/626835856 >> 172.21.15.162:0/626835856 conn(0x7f5fdc066a40 msgr2=0x7f5fdc0b6360 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:37.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.307+0000 7f5fe18c3700 1 -- 172.21.15.162:0/626835856 shutdown_connections 2020-11-28T21:46:37.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.307+0000 7f5fe18c3700 1 -- 172.21.15.162:0/626835856 wait complete. 2020-11-28T21:46:37.323 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542149 for osd.4 2020-11-28T21:46:37.324 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.5 2020-11-28T21:46:37.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.431+0000 7f721f4c2700 1 Processor -- start 2020-11-28T21:46:37.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.435+0000 7f721f4c2700 1 -- start start 2020-11-28T21:46:37.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.435+0000 7f721f4c2700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f72180b4c70 0x7f72180b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:37.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.435+0000 7f721f4c2700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72180b4110 con 0x7f7218142a00 2020-11-28T21:46:37.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.435+0000 7f721f4c2700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72180b43f0 con 0x7f72180603d0 2020-11-28T21:46:37.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.435+0000 7f721f4c2700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7218060f60 con 0x7f72180b4c70 2020-11-28T21:46:37.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.435+0000 7f721ecc1700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f72180b4c70 0x7f72180b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:37.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.435+0000 7f721ecc1700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f72180b4c70 0x7f72180b3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37590/0 (socket says 172.21.15.162:37590) 2020-11-28T21:46:37.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.435+0000 7f721e4c0700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f72180603d0 0x7f721809cfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:44432/0 (socket says 172.21.15.162:44432) 2020-11-28T21:46:37.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.435+0000 7f721ecc1700 1 -- 172.21.15.162:0/2907283375 learned_addr learned my addr 172.21.15.162:0/2907283375 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:37.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.435+0000 7f721ecc1700 1 -- 172.21.15.162:0/2907283375 >> v1:172.21.15.162:6789/0 conn(0x7f72180603d0 legacy=0x7f721809cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:37.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.435+0000 7f721ecc1700 1 -- 172.21.15.162:0/2907283375 >> v1:172.21.15.118:6789/0 conn(0x7f7218142a00 legacy=0x7f7218143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:37.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.435+0000 7f721ecc1700 1 -- 172.21.15.162:0/2907283375 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72180b4690 con 0x7f72180b4c70 2020-11-28T21:46:37.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.435+0000 7f721ecc1700 1 --2- 172.21.15.162:0/2907283375 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f72180b4c70 0x7f72180b3a70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f72100032b0 tx=0x7f7210009270).ready entity=mon.1 client_cookie=b56db562158a5b9e server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:37.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.435+0000 7f721d4be700 1 -- 172.21.15.162:0/2907283375 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f721000bc00 con 0x7f72180b4c70 2020-11-28T21:46:37.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.435+0000 7f721d4be700 1 -- 172.21.15.162:0/2907283375 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721000bd60 con 0x7f72180b4c70 2020-11-28T21:46:37.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.435+0000 7f721d4be700 1 -- 172.21.15.162:0/2907283375 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7210011400 con 0x7f72180b4c70 2020-11-28T21:46:37.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.435+0000 7f721f4c2700 1 -- 172.21.15.162:0/2907283375 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f72180b4c70 msgr2=0x7f72180b3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:37.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.435+0000 7f721f4c2700 1 --2- 172.21.15.162:0/2907283375 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f72180b4c70 0x7f72180b3a70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f72100032b0 tx=0x7f7210009270).stop 2020-11-28T21:46:37.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.435+0000 7f721f4c2700 1 -- 172.21.15.162:0/2907283375 shutdown_connections 2020-11-28T21:46:37.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.435+0000 7f721f4c2700 1 --2- 172.21.15.162:0/2907283375 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f72180b4c70 0x7f72180b3a70 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:37.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.435+0000 7f721f4c2700 1 -- 172.21.15.162:0/2907283375 >> 172.21.15.162:0/2907283375 conn(0x7f7218066a40 msgr2=0x7f7218066e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:37.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.435+0000 7f721f4c2700 1 -- 172.21.15.162:0/2907283375 shutdown_connections 2020-11-28T21:46:37.444 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.435+0000 7f721f4c2700 1 -- 172.21.15.162:0/2907283375 wait complete. 2020-11-28T21:46:37.444 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.439+0000 7f721f4c2700 1 Processor -- start 2020-11-28T21:46:37.444 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.439+0000 7f721f4c2700 1 -- start start 2020-11-28T21:46:37.447 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.439+0000 7f721f4c2700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f72180603d0 0x7f72180b8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:37.447 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.439+0000 7f721f4c2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f72180b4c70 0x7f72180ae580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:37.447 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.439+0000 7f721f4c2700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f72180acef0 0x7f72180aea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:37.447 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.439+0000 7f721f4c2700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72181469e0 con 0x7f72180603d0 2020-11-28T21:46:37.448 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.439+0000 7f721f4c2700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7218140aa0 con 0x7f72180b4c70 2020-11-28T21:46:37.448 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.439+0000 7f721f4c2700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72180af6e0 con 0x7f72180acef0 2020-11-28T21:46:37.448 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.439+0000 7f721dcbf700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f72180b4c70 0x7f72180ae580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:37.448 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.439+0000 7f721dcbf700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f72180b4c70 0x7f72180ae580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37592/0 (socket says 172.21.15.162:37592) 2020-11-28T21:46:37.448 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.439+0000 7f721dcbf700 1 -- 172.21.15.162:0/2125796975 learned_addr learned my addr 172.21.15.162:0/2125796975 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:37.449 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.439+0000 7f721e4c0700 1 --2- 172.21.15.162:0/2125796975 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f72180603d0 0x7f72180b8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:37.449 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.439+0000 7f721ecc1700 1 --2- 172.21.15.162:0/2125796975 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f72180acef0 0x7f72180aea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:37.449 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.439+0000 7f721dcbf700 1 -- 172.21.15.162:0/2125796975 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f72180acef0 msgr2=0x7f72180aea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:37.449 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.439+0000 7f721dcbf700 1 --2- 172.21.15.162:0/2125796975 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f72180acef0 0x7f72180aea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:37.449 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.439+0000 7f721dcbf700 1 -- 172.21.15.162:0/2125796975 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f72180603d0 msgr2=0x7f72180b8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:37.450 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.439+0000 7f721dcbf700 1 --2- 172.21.15.162:0/2125796975 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f72180603d0 0x7f72180b8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:37.450 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.439+0000 7f721dcbf700 1 -- 172.21.15.162:0/2125796975 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72180af980 con 0x7f72180b4c70 2020-11-28T21:46:37.450 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.439+0000 7f721e4c0700 1 --2- 172.21.15.162:0/2125796975 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f72180603d0 0x7f72180b8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:46:37.450 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.439+0000 7f721dcbf700 1 --2- 172.21.15.162:0/2125796975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f72180b4c70 0x7f72180ae580 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f72180b9c10 tx=0x7f7214007c20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:37.450 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.439+0000 7f720b7fe700 1 -- 172.21.15.162:0/2125796975 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f721400e4a0 con 0x7f72180b4c70 2020-11-28T21:46:37.451 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.439+0000 7f720b7fe700 1 -- 172.21.15.162:0/2125796975 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721400e600 con 0x7f72180b4c70 2020-11-28T21:46:37.451 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.439+0000 7f720b7fe700 1 -- 172.21.15.162:0/2125796975 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f721400e960 con 0x7f72180b4c70 2020-11-28T21:46:37.451 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.439+0000 7f721f4c2700 1 -- 172.21.15.162:0/2125796975 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72180afc20 con 0x7f72180b4c70 2020-11-28T21:46:37.452 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.439+0000 7f721f4c2700 1 -- 172.21.15.162:0/2125796975 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72180b89f0 con 0x7f72180b4c70 2020-11-28T21:46:37.452 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.439+0000 7f721f4c2700 1 -- 172.21.15.162:0/2125796975 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72180af100 con 0x7f72180b4c70 2020-11-28T21:46:37.452 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.443+0000 7f720b7fe700 1 -- 172.21.15.162:0/2125796975 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7214010030 con 0x7f72180b4c70 2020-11-28T21:46:37.452 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.443+0000 7f720b7fe700 1 --2- 172.21.15.162:0/2125796975 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f71f80554e0 0x7f71f80579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:37.452 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.443+0000 7f720b7fe700 1 -- 172.21.15.162:0/2125796975 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f721406de30 con 0x7f72180b4c70 2020-11-28T21:46:37.453 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.443+0000 7f721e4c0700 1 --2- 172.21.15.162:0/2125796975 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f71f80554e0 0x7f71f80579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:37.453 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.443+0000 7f721e4c0700 1 --2- 172.21.15.162:0/2125796975 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f71f80554e0 0x7f71f80579a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f720c00b000 tx=0x7f720c000b80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:37.453 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.447+0000 7f720b7fe700 1 -- 172.21.15.162:0/2125796975 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7214044090 con 0x7f72180b4c70 2020-11-28T21:46:37.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.591+0000 7f721f4c2700 1 -- 172.21.15.162:0/2125796975 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f7218057050 con 0x7f72180b4c70 2020-11-28T21:46:37.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.591+0000 7f720b7fe700 1 -- 172.21.15.162:0/2125796975 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f72140476b0 con 0x7f72180b4c70 2020-11-28T21:46:37.593 INFO:teuthology.orchestra.run.smithi162.stdout:60129542149 2020-11-28T21:46:37.594 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.591+0000 7f721f4c2700 1 -- 172.21.15.162:0/2125796975 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f71f80554e0 msgr2=0x7f71f80579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:37.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.591+0000 7f721f4c2700 1 --2- 172.21.15.162:0/2125796975 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f71f80554e0 0x7f71f80579a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f720c00b000 tx=0x7f720c000b80).stop 2020-11-28T21:46:37.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.591+0000 7f721f4c2700 1 -- 172.21.15.162:0/2125796975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f72180b4c70 msgr2=0x7f72180ae580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:37.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.591+0000 7f721f4c2700 1 --2- 172.21.15.162:0/2125796975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f72180b4c70 0x7f72180ae580 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f72180b9c10 tx=0x7f7214007c20).stop 2020-11-28T21:46:37.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.591+0000 7f721f4c2700 1 -- 172.21.15.162:0/2125796975 shutdown_connections 2020-11-28T21:46:37.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.591+0000 7f721f4c2700 1 --2- 172.21.15.162:0/2125796975 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f71f80554e0 0x7f71f80579a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:37.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.591+0000 7f721f4c2700 1 --2- 172.21.15.162:0/2125796975 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f72180acef0 0x7f72180aea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:37.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.591+0000 7f721f4c2700 1 --2- 172.21.15.162:0/2125796975 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f72180603d0 0x7f72180b8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:37.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.591+0000 7f721f4c2700 1 --2- 172.21.15.162:0/2125796975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f72180b4c70 0x7f72180ae580 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:37.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.591+0000 7f721f4c2700 1 -- 172.21.15.162:0/2125796975 >> 172.21.15.162:0/2125796975 conn(0x7f7218066a40 msgr2=0x7f721809e450 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:37.597 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.591+0000 7f721f4c2700 1 -- 172.21.15.162:0/2125796975 shutdown_connections 2020-11-28T21:46:37.597 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.591+0000 7f721f4c2700 1 -- 172.21.15.162:0/2125796975 wait complete. 2020-11-28T21:46:37.614 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542149 for osd.5 2020-11-28T21:46:37.615 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.6 2020-11-28T21:46:37.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.723+0000 7f0d027e4700 1 Processor -- start 2020-11-28T21:46:37.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.723+0000 7f0d027e4700 1 -- start start 2020-11-28T21:46:37.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.727+0000 7f0d027e4700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f0cfc0994f0 0x7f0cfc0b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:37.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.727+0000 7f0d027e4700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cfc098f10 con 0x7f0cfc060710 2020-11-28T21:46:37.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.727+0000 7f0d027e4700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cfc0b0680 con 0x7f0cfc098b30 2020-11-28T21:46:37.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.727+0000 7f0d027e4700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cfc0b1030 con 0x7f0cfc0994f0 2020-11-28T21:46:37.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.727+0000 7f0d017e2700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f0cfc098b30 0x7f0cfc060000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:44444/0 (socket says 172.21.15.162:44444) 2020-11-28T21:46:37.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.727+0000 7f0d017e2700 1 -- 172.21.15.162:0/2480591116 learned_addr learned my addr 172.21.15.162:0/2480591116 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:37.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.727+0000 7f0d00fe1700 1 --2- 172.21.15.162:0/2480591116 >> v2:172.21.15.118:3300/0 conn(0x7f0cfc0994f0 0x7f0cfc0b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:37.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.727+0000 7f0cfbfff700 1 -- 172.21.15.162:0/2480591116 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2824691632 0 0) 0x7f0cfc0b0680 con 0x7f0cfc098b30 2020-11-28T21:46:37.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.727+0000 7f0cfbfff700 1 -- 172.21.15.162:0/2480591116 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ce8003230 con 0x7f0cfc098b30 2020-11-28T21:46:37.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.727+0000 7f0d00fe1700 1 -- 172.21.15.162:0/2480591116 >> v1:172.21.15.162:6789/0 conn(0x7f0cfc098b30 legacy=0x7f0cfc060000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:37.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.727+0000 7f0d00fe1700 1 -- 172.21.15.162:0/2480591116 >> v1:172.21.15.118:6789/0 conn(0x7f0cfc060710 legacy=0x7f0cfc0b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:37.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.727+0000 7f0cfbfff700 1 -- 172.21.15.162:0/2480591116 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 997723582 0 0) 0x7f0cfc098f10 con 0x7f0cfc060710 2020-11-28T21:46:37.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.727+0000 7f0d00fe1700 1 -- 172.21.15.162:0/2480591116 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cfc0b0920 con 0x7f0cfc0994f0 2020-11-28T21:46:37.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.727+0000 7f0d00fe1700 1 --2- 172.21.15.162:0/2480591116 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0cfc0994f0 0x7f0cfc0b0000 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f0cf0002f30 tx=0x7f0cf0006dc0).ready entity=mon.0 client_cookie=7e77322a0d2209c1 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:37.735 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.727+0000 7f0cfbfff700 1 -- 172.21.15.162:0/2480591116 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cf0009800 con 0x7f0cfc0994f0 2020-11-28T21:46:37.735 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.727+0000 7f0cfbfff700 1 -- 172.21.15.162:0/2480591116 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf0009960 con 0x7f0cfc0994f0 2020-11-28T21:46:37.736 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.727+0000 7f0cfbfff700 1 -- 172.21.15.162:0/2480591116 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cf0009cc0 con 0x7f0cfc0994f0 2020-11-28T21:46:37.736 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.727+0000 7f0d027e4700 1 -- 172.21.15.162:0/2480591116 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0cfc0994f0 msgr2=0x7f0cfc0b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:37.736 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.727+0000 7f0d027e4700 1 --2- 172.21.15.162:0/2480591116 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0cfc0994f0 0x7f0cfc0b0000 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f0cf0002f30 tx=0x7f0cf0006dc0).stop 2020-11-28T21:46:37.736 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.727+0000 7f0d027e4700 1 -- 172.21.15.162:0/2480591116 shutdown_connections 2020-11-28T21:46:37.736 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.727+0000 7f0d027e4700 1 --2- 172.21.15.162:0/2480591116 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0cfc0994f0 0x7f0cfc0b0000 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:37.737 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.727+0000 7f0d027e4700 1 -- 172.21.15.162:0/2480591116 >> 172.21.15.162:0/2480591116 conn(0x7f0cfc05b970 msgr2=0x7f0cfc05bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:37.737 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.727+0000 7f0d027e4700 1 -- 172.21.15.162:0/2480591116 shutdown_connections 2020-11-28T21:46:37.737 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.727+0000 7f0d027e4700 1 -- 172.21.15.162:0/2480591116 wait complete. 2020-11-28T21:46:37.737 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.727+0000 7f0d027e4700 1 Processor -- start 2020-11-28T21:46:37.738 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.727+0000 7f0d027e4700 1 -- start start 2020-11-28T21:46:37.738 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.731+0000 7f0d027e4700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0cfc060710 0x7f0cfc0aabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:37.738 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.731+0000 7f0d027e4700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0cfc098b30 0x7f0cfc155be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:37.738 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.731+0000 7f0d027e4700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0cfc0994f0 0x7f0cfc1560d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:37.738 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.731+0000 7f0d027e4700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cfc14cf50 con 0x7f0cfc060710 2020-11-28T21:46:37.739 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.731+0000 7f0d027e4700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cfc0ab640 con 0x7f0cfc0994f0 2020-11-28T21:46:37.739 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.731+0000 7f0d027e4700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cfc14d9f0 con 0x7f0cfc098b30 2020-11-28T21:46:37.739 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.731+0000 7f0d01fe3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0cfc0994f0 0x7f0cfc1560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:37.739 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.731+0000 7f0d01fe3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0cfc0994f0 0x7f0cfc1560d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37610/0 (socket says 172.21.15.162:37610) 2020-11-28T21:46:37.739 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.731+0000 7f0d00fe1700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0cfc098b30 0x7f0cfc155be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:37.740 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.731+0000 7f0d017e2700 1 --2- 172.21.15.162:0/1896794618 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0cfc060710 0x7f0cfc0aabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:37.740 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.731+0000 7f0d01fe3700 1 -- 172.21.15.162:0/1896794618 learned_addr learned my addr 172.21.15.162:0/1896794618 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:37.740 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.731+0000 7f0d01fe3700 1 -- 172.21.15.162:0/1896794618 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0cfc098b30 msgr2=0x7f0cfc155be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:37.740 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.731+0000 7f0d01fe3700 1 --2- 172.21.15.162:0/1896794618 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0cfc098b30 0x7f0cfc155be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:37.740 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.731+0000 7f0d01fe3700 1 -- 172.21.15.162:0/1896794618 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0cfc060710 msgr2=0x7f0cfc0aabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:37.741 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.731+0000 7f0d01fe3700 1 --2- 172.21.15.162:0/1896794618 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0cfc060710 0x7f0cfc0aabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:37.741 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.731+0000 7f0d01fe3700 1 -- 172.21.15.162:0/1896794618 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cfc14dc90 con 0x7f0cfc0994f0 2020-11-28T21:46:37.741 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.731+0000 7f0d01fe3700 1 --2- 172.21.15.162:0/1896794618 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0cfc0994f0 0x7f0cfc1560d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f0cf4000f00 tx=0x7f0cf4007d30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:37.741 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.731+0000 7f0cfa7fc700 1 -- 172.21.15.162:0/1896794618 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cf400a5a0 con 0x7f0cfc0994f0 2020-11-28T21:46:37.741 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.731+0000 7f0cfa7fc700 1 -- 172.21.15.162:0/1896794618 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf400a700 con 0x7f0cfc0994f0 2020-11-28T21:46:37.742 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.731+0000 7f0d027e4700 1 -- 172.21.15.162:0/1896794618 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cfc14df30 con 0x7f0cfc0994f0 2020-11-28T21:46:37.742 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.731+0000 7f0cfa7fc700 1 -- 172.21.15.162:0/1896794618 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cf400aa60 con 0x7f0cfc0994f0 2020-11-28T21:46:37.742 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.731+0000 7f0d027e4700 1 -- 172.21.15.162:0/1896794618 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cfc14d5a0 con 0x7f0cfc0994f0 2020-11-28T21:46:37.743 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.735+0000 7f0d027e4700 1 -- 172.21.15.162:0/1896794618 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cfc057050 con 0x7f0cfc0994f0 2020-11-28T21:46:37.743 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.735+0000 7f0cfa7fc700 1 -- 172.21.15.162:0/1896794618 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0cf4010070 con 0x7f0cfc0994f0 2020-11-28T21:46:37.744 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.735+0000 7f0cfa7fc700 1 --2- 172.21.15.162:0/1896794618 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0cdc055330 0x7f0cdc0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:37.744 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.735+0000 7f0cfa7fc700 1 -- 172.21.15.162:0/1896794618 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0cf406d510 con 0x7f0cfc0994f0 2020-11-28T21:46:37.744 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.735+0000 7f0d017e2700 1 --2- 172.21.15.162:0/1896794618 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0cdc055330 0x7f0cdc0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:37.744 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.735+0000 7f0d017e2700 1 --2- 172.21.15.162:0/1896794618 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0cdc055330 0x7f0cdc0577f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f0cec00d000 tx=0x7f0cec009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:37.747 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.743+0000 7f0cfa7fc700 1 -- 172.21.15.162:0/1896794618 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0cf4043770 con 0x7f0cfc0994f0 2020-11-28T21:46:37.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.887+0000 7f0d027e4700 1 -- 172.21.15.162:0/1896794618 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f0cfc14e0c0 con 0x7f0cfc0994f0 2020-11-28T21:46:37.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.887+0000 7f0cfa7fc700 1 -- 172.21.15.162:0/1896794618 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0cf4046d90 con 0x7f0cfc0994f0 2020-11-28T21:46:37.891 INFO:teuthology.orchestra.run.smithi162.stdout:60129542149 2020-11-28T21:46:37.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.891+0000 7f0d027e4700 1 -- 172.21.15.162:0/1896794618 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0cdc055330 msgr2=0x7f0cdc0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:37.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.891+0000 7f0d027e4700 1 --2- 172.21.15.162:0/1896794618 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0cdc055330 0x7f0cdc0577f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f0cec00d000 tx=0x7f0cec009040).stop 2020-11-28T21:46:37.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.891+0000 7f0d027e4700 1 -- 172.21.15.162:0/1896794618 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0cfc0994f0 msgr2=0x7f0cfc1560d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:37.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.891+0000 7f0d027e4700 1 --2- 172.21.15.162:0/1896794618 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0cfc0994f0 0x7f0cfc1560d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f0cf4000f00 tx=0x7f0cf4007d30).stop 2020-11-28T21:46:37.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.891+0000 7f0d027e4700 1 -- 172.21.15.162:0/1896794618 shutdown_connections 2020-11-28T21:46:37.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.891+0000 7f0d027e4700 1 --2- 172.21.15.162:0/1896794618 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0cdc055330 0x7f0cdc0577f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:37.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.891+0000 7f0d027e4700 1 --2- 172.21.15.162:0/1896794618 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0cfc0994f0 0x7f0cfc1560d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:37.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.891+0000 7f0d027e4700 1 --2- 172.21.15.162:0/1896794618 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0cfc060710 0x7f0cfc0aabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:37.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.891+0000 7f0d027e4700 1 --2- 172.21.15.162:0/1896794618 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0cfc098b30 0x7f0cfc155be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:37.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.891+0000 7f0d027e4700 1 -- 172.21.15.162:0/1896794618 >> 172.21.15.162:0/1896794618 conn(0x7f0cfc05b970 msgr2=0x7f0cfc05bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:37.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.891+0000 7f0d027e4700 1 -- 172.21.15.162:0/1896794618 shutdown_connections 2020-11-28T21:46:37.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:37.891+0000 7f0d027e4700 1 -- 172.21.15.162:0/1896794618 wait complete. 2020-11-28T21:46:37.904 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542149 for osd.6 2020-11-28T21:46:37.905 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.7 2020-11-28T21:46:38.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.015+0000 7f6933fff700 1 Processor -- start 2020-11-28T21:46:38.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.019+0000 7f6933fff700 1 -- start start 2020-11-28T21:46:38.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.019+0000 7f6933fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f69340603d0 0x7f69340b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:38.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.019+0000 7f6933fff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f6934142a00 0x7f693409cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:38.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.019+0000 7f6933fff700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69340b4110 con 0x7f69340b4c70 2020-11-28T21:46:38.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.019+0000 7f6933fff700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6934060f60 con 0x7f6934142a00 2020-11-28T21:46:38.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.019+0000 7f6933fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69340b43f0 con 0x7f69340603d0 2020-11-28T21:46:38.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.019+0000 7f69337fe700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f69340b4c70 0x7f6934143020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:44460/0 (socket says 172.21.15.162:44460) 2020-11-28T21:46:38.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.019+0000 7f69337fe700 1 -- 172.21.15.162:0/3258654087 learned_addr learned my addr 172.21.15.162:0/3258654087 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:38.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.019+0000 7f69327fc700 1 --2- 172.21.15.162:0/3258654087 >> v2:172.21.15.118:3300/0 conn(0x7f6934142a00 0x7f693409cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:38.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.019+0000 7f6932ffd700 1 --2- 172.21.15.162:0/3258654087 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f69340603d0 0x7f69340b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:38.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.019+0000 7f6931ffb700 1 -- 172.21.15.162:0/3258654087 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2689273297 0 0) 0x7f69340b4110 con 0x7f69340b4c70 2020-11-28T21:46:38.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.019+0000 7f6931ffb700 1 -- 172.21.15.162:0/3258654087 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6918003230 con 0x7f69340b4c70 2020-11-28T21:46:38.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.019+0000 7f69327fc700 1 -- 172.21.15.162:0/3258654087 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f69340603d0 msgr2=0x7f69340b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:38.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.019+0000 7f69327fc700 1 --2- 172.21.15.162:0/3258654087 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f69340603d0 0x7f69340b3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:38.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.019+0000 7f69327fc700 1 -- 172.21.15.162:0/3258654087 >> v1:172.21.15.162:6789/0 conn(0x7f69340b4c70 legacy=0x7f6934143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:38.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.019+0000 7f6931ffb700 1 -- 172.21.15.162:0/3258654087 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2290606807 0 0) 0x7f6918003230 con 0x7f69340b4c70 2020-11-28T21:46:38.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.019+0000 7f69327fc700 1 -- 172.21.15.162:0/3258654087 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69340b4690 con 0x7f6934142a00 2020-11-28T21:46:38.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.019+0000 7f69327fc700 1 --2- 172.21.15.162:0/3258654087 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6934142a00 0x7f693409cfc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f692c001bf0 tx=0x7f6924006dc0).ready entity=mon.0 client_cookie=845cf8c1d62ec9b server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:38.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.019+0000 7f6931ffb700 1 -- 172.21.15.162:0/3258654087 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f692c001a50 con 0x7f6934142a00 2020-11-28T21:46:38.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.019+0000 7f6931ffb700 1 -- 172.21.15.162:0/3258654087 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f692400d570 con 0x7f6934142a00 2020-11-28T21:46:38.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.019+0000 7f6931ffb700 1 -- 172.21.15.162:0/3258654087 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f692400d6e0 con 0x7f6934142a00 2020-11-28T21:46:38.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.019+0000 7f6933fff700 1 -- 172.21.15.162:0/3258654087 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6934142a00 msgr2=0x7f693409cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:38.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.019+0000 7f6933fff700 1 --2- 172.21.15.162:0/3258654087 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6934142a00 0x7f693409cfc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f692c001bf0 tx=0x7f6924006dc0).stop 2020-11-28T21:46:38.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.019+0000 7f6933fff700 1 -- 172.21.15.162:0/3258654087 shutdown_connections 2020-11-28T21:46:38.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.019+0000 7f6933fff700 1 --2- 172.21.15.162:0/3258654087 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f69340603d0 0x7f69340b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:38.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.019+0000 7f6933fff700 1 --2- 172.21.15.162:0/3258654087 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6934142a00 0x7f693409cfc0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:38.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.019+0000 7f6933fff700 1 -- 172.21.15.162:0/3258654087 >> 172.21.15.162:0/3258654087 conn(0x7f6934066a40 msgr2=0x7f6934066e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:38.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.019+0000 7f6933fff700 1 -- 172.21.15.162:0/3258654087 shutdown_connections 2020-11-28T21:46:38.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.019+0000 7f6933fff700 1 -- 172.21.15.162:0/3258654087 wait complete. 2020-11-28T21:46:38.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.023+0000 7f6933fff700 1 Processor -- start 2020-11-28T21:46:38.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.023+0000 7f6933fff700 1 -- start start 2020-11-28T21:46:38.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.023+0000 7f6933fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f69340603d0 0x7f6934151c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:38.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.023+0000 7f6933fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69340b4c70 0x7f6934152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:38.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.023+0000 7f6933fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f69341470a0 0x7f6934147530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:38.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.023+0000 7f6933fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6934151570 con 0x7f69341470a0 2020-11-28T21:46:38.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.023+0000 7f6933fff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6934140e30 con 0x7f69340b4c70 2020-11-28T21:46:38.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.023+0000 7f6933fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69340aef80 con 0x7f69340603d0 2020-11-28T21:46:38.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.023+0000 7f69327fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69340b4c70 0x7f6934152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:38.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.023+0000 7f69327fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69340b4c70 0x7f6934152120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37620/0 (socket says 172.21.15.162:37620) 2020-11-28T21:46:38.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.023+0000 7f69327fc700 1 -- 172.21.15.162:0/2759418263 learned_addr learned my addr 172.21.15.162:0/2759418263 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:38.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.023+0000 7f69337fe700 1 --2- 172.21.15.162:0/2759418263 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f69341470a0 0x7f6934147530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:38.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.023+0000 7f6932ffd700 1 --2- 172.21.15.162:0/2759418263 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f69340603d0 0x7f6934151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:38.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.023+0000 7f69337fe700 1 -- 172.21.15.162:0/2759418263 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f69340603d0 msgr2=0x7f6934151c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:38.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.023+0000 7f69337fe700 1 --2- 172.21.15.162:0/2759418263 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f69340603d0 0x7f6934151c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:38.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.023+0000 7f69337fe700 1 -- 172.21.15.162:0/2759418263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69340b4c70 msgr2=0x7f6934152120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:38.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.023+0000 7f69337fe700 1 --2- 172.21.15.162:0/2759418263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69340b4c70 0x7f6934152120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:38.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.023+0000 7f69337fe700 1 -- 172.21.15.162:0/2759418263 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6934147a20 con 0x7f69341470a0 2020-11-28T21:46:38.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.023+0000 7f69327fc700 1 --2- 172.21.15.162:0/2759418263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69340b4c70 0x7f6934152120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:46:38.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.023+0000 7f69337fe700 1 --2- 172.21.15.162:0/2759418263 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f69341470a0 0x7f6934147530 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f6934146e50 tx=0x7f692c0019e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:38.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.023+0000 7f6923fff700 1 -- 172.21.15.162:0/2759418263 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f692c004a80 con 0x7f69341470a0 2020-11-28T21:46:38.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.023+0000 7f6923fff700 1 -- 172.21.15.162:0/2759418263 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f692c004be0 con 0x7f69341470a0 2020-11-28T21:46:38.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.023+0000 7f6933fff700 1 -- 172.21.15.162:0/2759418263 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69340af990 con 0x7f69341470a0 2020-11-28T21:46:38.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.023+0000 7f6923fff700 1 -- 172.21.15.162:0/2759418263 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f692c007170 con 0x7f69341470a0 2020-11-28T21:46:38.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.027+0000 7f6933fff700 1 -- 172.21.15.162:0/2759418263 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69340ae950 con 0x7f69341470a0 2020-11-28T21:46:38.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.027+0000 7f6933fff700 1 -- 172.21.15.162:0/2759418263 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6934057050 con 0x7f69341470a0 2020-11-28T21:46:38.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.027+0000 7f6923fff700 1 -- 172.21.15.162:0/2759418263 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f692c010070 con 0x7f69341470a0 2020-11-28T21:46:38.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.027+0000 7f6923fff700 1 --2- 172.21.15.162:0/2759418263 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6910055330 0x7f69100577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:38.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.027+0000 7f6923fff700 1 -- 172.21.15.162:0/2759418263 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f692c06e1b0 con 0x7f69341470a0 2020-11-28T21:46:38.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.027+0000 7f6932ffd700 1 --2- 172.21.15.162:0/2759418263 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6910055330 0x7f69100577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:38.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.027+0000 7f6932ffd700 1 --2- 172.21.15.162:0/2759418263 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6910055330 0x7f69100577f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f692800b000 tx=0x7f6928006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:38.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.035+0000 7f6923fff700 1 -- 172.21.15.162:0/2759418263 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f692c0444b0 con 0x7f69341470a0 2020-11-28T21:46:38.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.179+0000 7f6933fff700 1 -- 172.21.15.162:0/2759418263 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f6934069b60 con 0x7f69341470a0 2020-11-28T21:46:38.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.179+0000 7f6923fff700 1 -- 172.21.15.162:0/2759418263 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f692c047ad0 con 0x7f69341470a0 2020-11-28T21:46:38.184 INFO:teuthology.orchestra.run.smithi162.stdout:60129542149 2020-11-28T21:46:38.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.183+0000 7f6933fff700 1 -- 172.21.15.162:0/2759418263 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6910055330 msgr2=0x7f69100577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:38.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.183+0000 7f6933fff700 1 --2- 172.21.15.162:0/2759418263 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6910055330 0x7f69100577f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f692800b000 tx=0x7f6928006dc0).stop 2020-11-28T21:46:38.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.183+0000 7f6933fff700 1 -- 172.21.15.162:0/2759418263 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f69341470a0 msgr2=0x7f6934147530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:38.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.183+0000 7f6933fff700 1 --2- 172.21.15.162:0/2759418263 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f69341470a0 0x7f6934147530 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f6934146e50 tx=0x7f692c0019e0).stop 2020-11-28T21:46:38.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.183+0000 7f6933fff700 1 -- 172.21.15.162:0/2759418263 shutdown_connections 2020-11-28T21:46:38.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.183+0000 7f6933fff700 1 --2- 172.21.15.162:0/2759418263 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6910055330 0x7f69100577f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:38.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.183+0000 7f6933fff700 1 --2- 172.21.15.162:0/2759418263 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f69340603d0 0x7f6934151c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:38.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.183+0000 7f6933fff700 1 --2- 172.21.15.162:0/2759418263 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f69341470a0 0x7f6934147530 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:38.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.183+0000 7f6933fff700 1 --2- 172.21.15.162:0/2759418263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69340b4c70 0x7f6934152120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:38.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.183+0000 7f6933fff700 1 -- 172.21.15.162:0/2759418263 >> 172.21.15.162:0/2759418263 conn(0x7f6934066a40 msgr2=0x7f69340b63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:38.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.183+0000 7f6933fff700 1 -- 172.21.15.162:0/2759418263 shutdown_connections 2020-11-28T21:46:38.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.183+0000 7f6933fff700 1 -- 172.21.15.162:0/2759418263 wait complete. 2020-11-28T21:46:38.204 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542149 for osd.7 2020-11-28T21:46:38.204 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2020-11-28T21:46:38.205 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-11-28T21:46:38.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.315+0000 7f0cabfff700 1 Processor -- start 2020-11-28T21:46:38.318 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.315+0000 7f0cabfff700 1 -- start start 2020-11-28T21:46:38.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.315+0000 7f0cabfff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0cac142a00 0x7f0cac0b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:38.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.315+0000 7f0cabfff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cac0b4110 con 0x7f0cac0b4c70 2020-11-28T21:46:38.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.315+0000 7f0cabfff700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cac0b43f0 con 0x7f0cac0603d0 2020-11-28T21:46:38.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.315+0000 7f0cabfff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cac060f60 con 0x7f0cac142a00 2020-11-28T21:46:38.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.315+0000 7f0caaffd700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f0cac0603d0 0x7f0cac09cfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:44470/0 (socket says 172.21.15.162:44470) 2020-11-28T21:46:38.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.315+0000 7f0caaffd700 1 -- 172.21.15.162:0/4290158050 learned_addr learned my addr 172.21.15.162:0/4290158050 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:38.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.315+0000 7f0ca27fc700 1 --2- 172.21.15.162:0/4290158050 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0cac142a00 0x7f0cac0b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:38.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.315+0000 7f0caa7fc700 1 -- 172.21.15.162:0/4290158050 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 244110623 0 0) 0x7f0cac0b43f0 con 0x7f0cac0603d0 2020-11-28T21:46:38.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.315+0000 7f0caa7fc700 1 -- 172.21.15.162:0/4290158050 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c8c003230 con 0x7f0cac0603d0 2020-11-28T21:46:38.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.315+0000 7f0caa7fc700 1 -- 172.21.15.162:0/4290158050 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2453513991 0 0) 0x7f0cac0b4110 con 0x7f0cac0b4c70 2020-11-28T21:46:38.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.315+0000 7f0caa7fc700 1 -- 172.21.15.162:0/4290158050 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cac0b43f0 con 0x7f0cac0b4c70 2020-11-28T21:46:38.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.315+0000 7f0ca27fc700 1 -- 172.21.15.162:0/4290158050 >> v1:172.21.15.162:6789/0 conn(0x7f0cac0603d0 legacy=0x7f0cac09cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:38.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.315+0000 7f0ca27fc700 1 -- 172.21.15.162:0/4290158050 >> v1:172.21.15.118:6789/0 conn(0x7f0cac0b4c70 legacy=0x7f0cac143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:38.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.315+0000 7f0ca27fc700 1 -- 172.21.15.162:0/4290158050 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cac0b4690 con 0x7f0cac142a00 2020-11-28T21:46:38.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.315+0000 7f0ca27fc700 1 --2- 172.21.15.162:0/4290158050 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0cac142a00 0x7f0cac0b3a70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f0c94002f30 tx=0x7f0c94006dc0).ready entity=mon.2 client_cookie=b1b7519c2dc32e93 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:38.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.319+0000 7f0caa7fc700 1 -- 172.21.15.162:0/4290158050 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c94009730 con 0x7f0cac142a00 2020-11-28T21:46:38.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.319+0000 7f0caa7fc700 1 -- 172.21.15.162:0/4290158050 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c94009890 con 0x7f0cac142a00 2020-11-28T21:46:38.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.319+0000 7f0caa7fc700 1 -- 172.21.15.162:0/4290158050 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c94009bf0 con 0x7f0cac142a00 2020-11-28T21:46:38.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.319+0000 7f0cabfff700 1 -- 172.21.15.162:0/4290158050 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0cac142a00 msgr2=0x7f0cac0b3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:38.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.319+0000 7f0cabfff700 1 --2- 172.21.15.162:0/4290158050 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0cac142a00 0x7f0cac0b3a70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f0c94002f30 tx=0x7f0c94006dc0).stop 2020-11-28T21:46:38.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.319+0000 7f0cabfff700 1 -- 172.21.15.162:0/4290158050 shutdown_connections 2020-11-28T21:46:38.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.319+0000 7f0cabfff700 1 --2- 172.21.15.162:0/4290158050 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0cac142a00 0x7f0cac0b3a70 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:38.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.319+0000 7f0cabfff700 1 -- 172.21.15.162:0/4290158050 >> 172.21.15.162:0/4290158050 conn(0x7f0cac066a40 msgr2=0x7f0cac066e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:38.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.319+0000 7f0cabfff700 1 -- 172.21.15.162:0/4290158050 shutdown_connections 2020-11-28T21:46:38.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.319+0000 7f0cabfff700 1 -- 172.21.15.162:0/4290158050 wait complete. 2020-11-28T21:46:38.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.319+0000 7f0cabfff700 1 Processor -- start 2020-11-28T21:46:38.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.319+0000 7f0cabfff700 1 -- start start 2020-11-28T21:46:38.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.319+0000 7f0cabfff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0cac0603d0 0x7f0cac0b8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:38.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.319+0000 7f0cabfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0cac0b4c70 0x7f0cac090240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:38.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.319+0000 7f0cabfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0cac151480 0x7f0cac090730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:38.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.319+0000 7f0cabfff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cac149aa0 con 0x7f0cac151480 2020-11-28T21:46:38.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.319+0000 7f0cabfff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cac140c90 con 0x7f0cac0b4c70 2020-11-28T21:46:38.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.319+0000 7f0cabfff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cac147a20 con 0x7f0cac0603d0 2020-11-28T21:46:38.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.319+0000 7f0ca27fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0cac0b4c70 0x7f0cac090240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:38.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.319+0000 7f0ca27fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0cac0b4c70 0x7f0cac090240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37634/0 (socket says 172.21.15.162:37634) 2020-11-28T21:46:38.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.319+0000 7f0ca27fc700 1 -- 172.21.15.162:0/1273506973 learned_addr learned my addr 172.21.15.162:0/1273506973 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:38.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.323+0000 7f0caaffd700 1 --2- 172.21.15.162:0/1273506973 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0cac0603d0 0x7f0cac0b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:38.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.323+0000 7f0cab7fe700 1 --2- 172.21.15.162:0/1273506973 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0cac151480 0x7f0cac090730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:38.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.323+0000 7f0ca27fc700 1 -- 172.21.15.162:0/1273506973 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0cac0603d0 msgr2=0x7f0cac0b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:38.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.323+0000 7f0ca27fc700 1 --2- 172.21.15.162:0/1273506973 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0cac0603d0 0x7f0cac0b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:38.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.323+0000 7f0ca27fc700 1 -- 172.21.15.162:0/1273506973 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0cac151480 msgr2=0x7f0cac090730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:38.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.323+0000 7f0ca27fc700 1 --2- 172.21.15.162:0/1273506973 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0cac151480 0x7f0cac090730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:38.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.323+0000 7f0ca27fc700 1 -- 172.21.15.162:0/1273506973 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cac1476a0 con 0x7f0cac0b4c70 2020-11-28T21:46:38.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.323+0000 7f0ca27fc700 1 --2- 172.21.15.162:0/1273506973 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0cac0b4c70 0x7f0cac090240 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f0c94002f30 tx=0x7f0c94015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:38.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.323+0000 7f0ca8ff9700 1 -- 172.21.15.162:0/1273506973 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c94011b70 con 0x7f0cac0b4c70 2020-11-28T21:46:38.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.323+0000 7f0ca8ff9700 1 -- 172.21.15.162:0/1273506973 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c94011cd0 con 0x7f0cac0b4c70 2020-11-28T21:46:38.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.323+0000 7f0cabfff700 1 -- 172.21.15.162:0/1273506973 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cac146910 con 0x7f0cac0b4c70 2020-11-28T21:46:38.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.323+0000 7f0ca8ff9700 1 -- 172.21.15.162:0/1273506973 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c9401d640 con 0x7f0cac0b4c70 2020-11-28T21:46:38.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.323+0000 7f0cabfff700 1 -- 172.21.15.162:0/1273506973 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cac146e50 con 0x7f0cac0b4c70 2020-11-28T21:46:38.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.323+0000 7f0cabfff700 1 -- 172.21.15.162:0/1273506973 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cac057050 con 0x7f0cac0b4c70 2020-11-28T21:46:38.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.323+0000 7f0ca8ff9700 1 -- 172.21.15.162:0/1273506973 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0c9400f070 con 0x7f0cac0b4c70 2020-11-28T21:46:38.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.323+0000 7f0ca8ff9700 1 --2- 172.21.15.162:0/1273506973 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0c90055440 0x7f0c90057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:38.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.323+0000 7f0ca8ff9700 1 -- 172.21.15.162:0/1273506973 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0c9406fa30 con 0x7f0cac0b4c70 2020-11-28T21:46:38.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.323+0000 7f0caaffd700 1 --2- 172.21.15.162:0/1273506973 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0c90055440 0x7f0c90057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:38.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.323+0000 7f0caaffd700 1 --2- 172.21.15.162:0/1273506973 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0c90055440 0x7f0c90057900 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f0c9c00d000 tx=0x7f0c9c009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:38.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.327+0000 7f0ca8ff9700 1 -- 172.21.15.162:0/1273506973 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0c94045c90 con 0x7f0cac0b4c70 2020-11-28T21:46:38.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.467+0000 7f0cabfff700 1 -- 172.21.15.162:0/1273506973 --> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0cac069b60 con 0x7f0c90055440 2020-11-28T21:46:38.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.471+0000 7f0ca8ff9700 1 -- 172.21.15.162:0/1273506973 <== mgr.4105 v2:172.21.15.162:6800/12156 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82285 (secure 0 0 0) 0x7f0cac069b60 con 0x7f0c90055440 2020-11-28T21:46:38.477 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:46:38.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.475+0000 7f0cabfff700 1 -- 172.21.15.162:0/1273506973 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0c90055440 msgr2=0x7f0c90057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:38.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.475+0000 7f0cabfff700 1 --2- 172.21.15.162:0/1273506973 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0c90055440 0x7f0c90057900 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f0c9c00d000 tx=0x7f0c9c009040).stop 2020-11-28T21:46:38.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.475+0000 7f0cabfff700 1 -- 172.21.15.162:0/1273506973 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0cac0b4c70 msgr2=0x7f0cac090240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:38.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.475+0000 7f0cabfff700 1 --2- 172.21.15.162:0/1273506973 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0cac0b4c70 0x7f0cac090240 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f0c94002f30 tx=0x7f0c94015000).stop 2020-11-28T21:46:38.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.475+0000 7f0cabfff700 1 -- 172.21.15.162:0/1273506973 shutdown_connections 2020-11-28T21:46:38.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.475+0000 7f0cabfff700 1 --2- 172.21.15.162:0/1273506973 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0c90055440 0x7f0c90057900 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:38.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.475+0000 7f0cabfff700 1 --2- 172.21.15.162:0/1273506973 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0cac0603d0 0x7f0cac0b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:38.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.475+0000 7f0cabfff700 1 --2- 172.21.15.162:0/1273506973 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0cac151480 0x7f0cac090730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:38.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.475+0000 7f0cabfff700 1 --2- 172.21.15.162:0/1273506973 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0cac0b4c70 0x7f0cac090240 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:38.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.475+0000 7f0cabfff700 1 -- 172.21.15.162:0/1273506973 >> 172.21.15.162:0/1273506973 conn(0x7f0cac066a40 msgr2=0x7f0cac0b3980 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:38.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.475+0000 7f0cabfff700 1 -- 172.21.15.162:0/1273506973 shutdown_connections 2020-11-28T21:46:38.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.475+0000 7f0cabfff700 1 -- 172.21.15.162:0/1273506973 wait complete. 2020-11-28T21:46:38.481 INFO:teuthology.orchestra.run.smithi162.stderr:dumped all 2020-11-28T21:46:38.496 INFO:teuthology.orchestra.run.smithi162.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2020-11-28T21:46:37.576538+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2304,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":46,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8390588,"kb_used_data":1916,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746584132,"statfs":{"total":773094113280,"available":764502151168,"internally_reserved":8589934592,"allocated":1961984,"data_stored":710013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.449391"},"pg_stats":[{"pgid":"4.2","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.146922+0000","last_change":"2020-11-28T21:46:29.143409+0000","last_active":"2020-11-28T21:46:29.146922+0000","last_peered":"2020-11-28T21:46:29.146922+0000","last_clean":"2020-11-28T21:46:29.146922+0000","last_became_active":"2020-11-28T21:46:29.142911+0000","last_became_peered":"2020-11-28T21:46:29.142911+0000","last_unstale":"2020-11-28T21:46:29.146922+0000","last_undegraded":"2020-11-28T21:46:29.146922+0000","last_fullsized":"2020-11-28T21:46:29.146922+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:28.767144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:28.767144+0000","last_clean_scrub_stamp":"2020-11-28T21:46:28.767144+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"12","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.130289+0000","last_change":"2020-11-28T21:46:25.652727+0000","last_active":"2020-11-28T21:46:29.130289+0000","last_peered":"2020-11-28T21:46:29.130289+0000","last_clean":"2020-11-28T21:46:29.130289+0000","last_became_active":"2020-11-28T21:46:25.652265+0000","last_became_peered":"2020-11-28T21:46:25.652265+0000","last_unstale":"2020-11-28T21:46:29.130289+0000","last_undegraded":"2020-11-28T21:46:29.130289+0000","last_fullsized":"2020-11-28T21:46:29.130289+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_clean_scrub_stamp":"2020-11-28T21:46:24.622452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"22'6","reported_seq":"16","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.148880+0000","last_change":"2020-11-28T21:46:28.798375+0000","last_active":"2020-11-28T21:46:29.148880+0000","last_peered":"2020-11-28T21:46:29.148880+0000","last_clean":"2020-11-28T21:46:29.148880+0000","last_became_active":"2020-11-28T21:46:28.797984+0000","last_became_peered":"2020-11-28T21:46:28.797984+0000","last_unstale":"2020-11-28T21:46:29.148880+0000","last_undegraded":"2020-11-28T21:46:29.148880+0000","last_fullsized":"2020-11-28T21:46:29.148880+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:27.758912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:27.758912+0000","last_clean_scrub_stamp":"2020-11-28T21:46:27.758912+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.150049+0000","last_change":"2020-11-28T21:46:29.147464+0000","last_active":"2020-11-28T21:46:29.150049+0000","last_peered":"2020-11-28T21:46:29.150049+0000","last_clean":"2020-11-28T21:46:29.150049+0000","last_became_active":"2020-11-28T21:46:29.147036+0000","last_became_peered":"2020-11-28T21:46:29.147036+0000","last_unstale":"2020-11-28T21:46:29.150049+0000","last_undegraded":"2020-11-28T21:46:29.150049+0000","last_fullsized":"2020-11-28T21:46:29.150049+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:28.767144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:28.767144+0000","last_clean_scrub_stamp":"2020-11-28T21:46:28.767144+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"12","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.137303+0000","last_change":"2020-11-28T21:46:25.654101+0000","last_active":"2020-11-28T21:46:29.137303+0000","last_peered":"2020-11-28T21:46:29.137303+0000","last_clean":"2020-11-28T21:46:29.137303+0000","last_became_active":"2020-11-28T21:46:25.653417+0000","last_became_peered":"2020-11-28T21:46:25.653417+0000","last_unstale":"2020-11-28T21:46:29.137303+0000","last_undegraded":"2020-11-28T21:46:29.137303+0000","last_fullsized":"2020-11-28T21:46:29.137303+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_clean_scrub_stamp":"2020-11-28T21:46:24.622452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"22'5","reported_seq":"15","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.153309+0000","last_change":"2020-11-28T21:46:28.797447+0000","last_active":"2020-11-28T21:46:29.153309+0000","last_peered":"2020-11-28T21:46:29.153309+0000","last_clean":"2020-11-28T21:46:29.153309+0000","last_became_active":"2020-11-28T21:46:28.796944+0000","last_became_peered":"2020-11-28T21:46:28.796944+0000","last_unstale":"2020-11-28T21:46:29.153309+0000","last_undegraded":"2020-11-28T21:46:29.153309+0000","last_fullsized":"2020-11-28T21:46:29.153309+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:27.758912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:27.758912+0000","last_clean_scrub_stamp":"2020-11-28T21:46:27.758912+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.151290+0000","last_change":"2020-11-28T21:46:29.150407+0000","last_active":"2020-11-28T21:46:29.151290+0000","last_peered":"2020-11-28T21:46:29.151290+0000","last_clean":"2020-11-28T21:46:29.151290+0000","last_became_active":"2020-11-28T21:46:29.149655+0000","last_became_peered":"2020-11-28T21:46:29.149655+0000","last_unstale":"2020-11-28T21:46:29.151290+0000","last_undegraded":"2020-11-28T21:46:29.151290+0000","last_fullsized":"2020-11-28T21:46:29.151290+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:28.767144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:28.767144+0000","last_clean_scrub_stamp":"2020-11-28T21:46:28.767144+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"12","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.141495+0000","last_change":"2020-11-28T21:46:25.652888+0000","last_active":"2020-11-28T21:46:29.141495+0000","last_peered":"2020-11-28T21:46:29.141495+0000","last_clean":"2020-11-28T21:46:29.141495+0000","last_became_active":"2020-11-28T21:46:25.652295+0000","last_became_peered":"2020-11-28T21:46:25.652295+0000","last_unstale":"2020-11-28T21:46:29.141495+0000","last_undegraded":"2020-11-28T21:46:29.141495+0000","last_fullsized":"2020-11-28T21:46:29.141495+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_clean_scrub_stamp":"2020-11-28T21:46:24.622452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"22'8","reported_seq":"18","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.151518+0000","last_change":"2020-11-28T21:46:28.795437+0000","last_active":"2020-11-28T21:46:29.151518+0000","last_peered":"2020-11-28T21:46:29.151518+0000","last_clean":"2020-11-28T21:46:29.151518+0000","last_became_active":"2020-11-28T21:46:28.795076+0000","last_became_peered":"2020-11-28T21:46:28.795076+0000","last_unstale":"2020-11-28T21:46:29.151518+0000","last_undegraded":"2020-11-28T21:46:29.151518+0000","last_fullsized":"2020-11-28T21:46:29.151518+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:27.758912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:27.758912+0000","last_clean_scrub_stamp":"2020-11-28T21:46:27.758912+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.145558+0000","last_change":"2020-11-28T21:46:29.143471+0000","last_active":"2020-11-28T21:46:29.145558+0000","last_peered":"2020-11-28T21:46:29.145558+0000","last_clean":"2020-11-28T21:46:29.145558+0000","last_became_active":"2020-11-28T21:46:29.143023+0000","last_became_peered":"2020-11-28T21:46:29.143023+0000","last_unstale":"2020-11-28T21:46:29.145558+0000","last_undegraded":"2020-11-28T21:46:29.145558+0000","last_fullsized":"2020-11-28T21:46:29.145558+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:28.767144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:28.767144+0000","last_clean_scrub_stamp":"2020-11-28T21:46:28.767144+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"12","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.137448+0000","last_change":"2020-11-28T21:46:26.008971+0000","last_active":"2020-11-28T21:46:29.137448+0000","last_peered":"2020-11-28T21:46:29.137448+0000","last_clean":"2020-11-28T21:46:29.137448+0000","last_became_active":"2020-11-28T21:46:26.007744+0000","last_became_peered":"2020-11-28T21:46:26.007744+0000","last_unstale":"2020-11-28T21:46:29.137448+0000","last_undegraded":"2020-11-28T21:46:29.137448+0000","last_fullsized":"2020-11-28T21:46:29.137448+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_clean_scrub_stamp":"2020-11-28T21:46:24.622452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"16","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.135778+0000","last_change":"2020-11-28T21:46:20.941819+0000","last_active":"2020-11-28T21:46:29.135778+0000","last_peered":"2020-11-28T21:46:29.135778+0000","last_clean":"2020-11-28T21:46:29.135778+0000","last_became_active":"2020-11-28T21:46:20.941467+0000","last_became_peered":"2020-11-28T21:46:20.941467+0000","last_unstale":"2020-11-28T21:46:29.135778+0000","last_undegraded":"2020-11-28T21:46:29.135778+0000","last_fullsized":"2020-11-28T21:46:29.135778+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:16.570066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:16.570066+0000","last_clean_scrub_stamp":"2020-11-28T21:46:16.570066+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"22'1","reported_seq":"11","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.149810+0000","last_change":"2020-11-28T21:46:28.795437+0000","last_active":"2020-11-28T21:46:29.149810+0000","last_peered":"2020-11-28T21:46:29.149810+0000","last_clean":"2020-11-28T21:46:29.149810+0000","last_became_active":"2020-11-28T21:46:28.795024+0000","last_became_peered":"2020-11-28T21:46:28.795024+0000","last_unstale":"2020-11-28T21:46:29.149810+0000","last_undegraded":"2020-11-28T21:46:29.149810+0000","last_fullsized":"2020-11-28T21:46:29.149810+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:27.758912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:27.758912+0000","last_clean_scrub_stamp":"2020-11-28T21:46:27.758912+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.150192+0000","last_change":"2020-11-28T21:46:29.149147+0000","last_active":"2020-11-28T21:46:29.150192+0000","last_peered":"2020-11-28T21:46:29.150192+0000","last_clean":"2020-11-28T21:46:29.150192+0000","last_became_active":"2020-11-28T21:46:29.148658+0000","last_became_peered":"2020-11-28T21:46:29.148658+0000","last_unstale":"2020-11-28T21:46:29.150192+0000","last_undegraded":"2020-11-28T21:46:29.150192+0000","last_fullsized":"2020-11-28T21:46:29.150192+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:28.767144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:28.767144+0000","last_clean_scrub_stamp":"2020-11-28T21:46:28.767144+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"12","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.136616+0000","last_change":"2020-11-28T21:46:25.654099+0000","last_active":"2020-11-28T21:46:29.136616+0000","last_peered":"2020-11-28T21:46:29.136616+0000","last_clean":"2020-11-28T21:46:29.136616+0000","last_became_active":"2020-11-28T21:46:25.653652+0000","last_became_peered":"2020-11-28T21:46:25.653652+0000","last_unstale":"2020-11-28T21:46:29.136616+0000","last_undegraded":"2020-11-28T21:46:29.136616+0000","last_fullsized":"2020-11-28T21:46:29.136616+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_clean_scrub_stamp":"2020-11-28T21:46:24.622452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"10","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.128896+0000","last_change":"2020-11-28T21:46:28.794379+0000","last_active":"2020-11-28T21:46:29.128896+0000","last_peered":"2020-11-28T21:46:29.128896+0000","last_clean":"2020-11-28T21:46:29.128896+0000","last_became_active":"2020-11-28T21:46:28.793915+0000","last_became_peered":"2020-11-28T21:46:28.793915+0000","last_unstale":"2020-11-28T21:46:29.128896+0000","last_undegraded":"2020-11-28T21:46:29.128896+0000","last_fullsized":"2020-11-28T21:46:29.128896+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:27.758912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:27.758912+0000","last_clean_scrub_stamp":"2020-11-28T21:46:27.758912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.150922+0000","last_change":"2020-11-28T21:46:29.149491+0000","last_active":"2020-11-28T21:46:29.150922+0000","last_peered":"2020-11-28T21:46:29.150922+0000","last_clean":"2020-11-28T21:46:29.150922+0000","last_became_active":"2020-11-28T21:46:29.149008+0000","last_became_peered":"2020-11-28T21:46:29.149008+0000","last_unstale":"2020-11-28T21:46:29.150922+0000","last_undegraded":"2020-11-28T21:46:29.150922+0000","last_fullsized":"2020-11-28T21:46:29.150922+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:28.767144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:28.767144+0000","last_clean_scrub_stamp":"2020-11-28T21:46:28.767144+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"12","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.138052+0000","last_change":"2020-11-28T21:46:25.653672+0000","last_active":"2020-11-28T21:46:29.138052+0000","last_peered":"2020-11-28T21:46:29.138052+0000","last_clean":"2020-11-28T21:46:29.138052+0000","last_became_active":"2020-11-28T21:46:25.653222+0000","last_became_peered":"2020-11-28T21:46:25.653222+0000","last_unstale":"2020-11-28T21:46:29.138052+0000","last_undegraded":"2020-11-28T21:46:29.138052+0000","last_fullsized":"2020-11-28T21:46:29.138052+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_clean_scrub_stamp":"2020-11-28T21:46:24.622452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"22'8","reported_seq":"18","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.154177+0000","last_change":"2020-11-28T21:46:28.793524+0000","last_active":"2020-11-28T21:46:29.154177+0000","last_peered":"2020-11-28T21:46:29.154177+0000","last_clean":"2020-11-28T21:46:29.154177+0000","last_became_active":"2020-11-28T21:46:28.793077+0000","last_became_peered":"2020-11-28T21:46:28.793077+0000","last_unstale":"2020-11-28T21:46:29.154177+0000","last_undegraded":"2020-11-28T21:46:29.154177+0000","last_fullsized":"2020-11-28T21:46:29.154177+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:27.758912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:27.758912+0000","last_clean_scrub_stamp":"2020-11-28T21:46:27.758912+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1535,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.151553+0000","last_change":"2020-11-28T21:46:29.150884+0000","last_active":"2020-11-28T21:46:29.151553+0000","last_peered":"2020-11-28T21:46:29.151553+0000","last_clean":"2020-11-28T21:46:29.151553+0000","last_became_active":"2020-11-28T21:46:29.150424+0000","last_became_peered":"2020-11-28T21:46:29.150424+0000","last_unstale":"2020-11-28T21:46:29.151553+0000","last_undegraded":"2020-11-28T21:46:29.151553+0000","last_fullsized":"2020-11-28T21:46:29.151553+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:28.767144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:28.767144+0000","last_clean_scrub_stamp":"2020-11-28T21:46:28.767144+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"22'7","reported_seq":"17","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.151619+0000","last_change":"2020-11-28T21:46:28.791205+0000","last_active":"2020-11-28T21:46:29.151619+0000","last_peered":"2020-11-28T21:46:29.151619+0000","last_clean":"2020-11-28T21:46:29.151619+0000","last_became_active":"2020-11-28T21:46:28.790564+0000","last_became_peered":"2020-11-28T21:46:28.790564+0000","last_unstale":"2020-11-28T21:46:29.151619+0000","last_undegraded":"2020-11-28T21:46:29.151619+0000","last_fullsized":"2020-11-28T21:46:29.151619+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:27.758912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:27.758912+0000","last_clean_scrub_stamp":"2020-11-28T21:46:27.758912+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":575,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.136302+0000","last_change":"2020-11-28T21:46:25.838546+0000","last_active":"2020-11-28T21:46:29.136302+0000","last_peered":"2020-11-28T21:46:29.136302+0000","last_clean":"2020-11-28T21:46:29.136302+0000","last_became_active":"2020-11-28T21:46:25.837556+0000","last_became_peered":"2020-11-28T21:46:25.837556+0000","last_unstale":"2020-11-28T21:46:29.136302+0000","last_undegraded":"2020-11-28T21:46:29.136302+0000","last_fullsized":"2020-11-28T21:46:29.136302+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_clean_scrub_stamp":"2020-11-28T21:46:24.622452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.151643+0000","last_change":"2020-11-28T21:46:29.144515+0000","last_active":"2020-11-28T21:46:29.151643+0000","last_peered":"2020-11-28T21:46:29.151643+0000","last_clean":"2020-11-28T21:46:29.151643+0000","last_became_active":"2020-11-28T21:46:29.144141+0000","last_became_peered":"2020-11-28T21:46:29.144141+0000","last_unstale":"2020-11-28T21:46:29.151643+0000","last_undegraded":"2020-11-28T21:46:29.151643+0000","last_fullsized":"2020-11-28T21:46:29.151643+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:28.767144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:28.767144+0000","last_clean_scrub_stamp":"2020-11-28T21:46:28.767144+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"12","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.137554+0000","last_change":"2020-11-28T21:46:25.838687+0000","last_active":"2020-11-28T21:46:29.137554+0000","last_peered":"2020-11-28T21:46:29.137554+0000","last_clean":"2020-11-28T21:46:29.137554+0000","last_became_active":"2020-11-28T21:46:25.837943+0000","last_became_peered":"2020-11-28T21:46:25.837943+0000","last_unstale":"2020-11-28T21:46:29.137554+0000","last_undegraded":"2020-11-28T21:46:29.137554+0000","last_fullsized":"2020-11-28T21:46:29.137554+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_clean_scrub_stamp":"2020-11-28T21:46:24.622452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"23'4","reported_seq":"14","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-28T21:46:30.137532+0000","last_change":"2020-11-28T21:46:28.791512+0000","last_active":"2020-11-28T21:46:30.137532+0000","last_peered":"2020-11-28T21:46:30.137532+0000","last_clean":"2020-11-28T21:46:30.137532+0000","last_became_active":"2020-11-28T21:46:28.790599+0000","last_became_peered":"2020-11-28T21:46:28.790599+0000","last_unstale":"2020-11-28T21:46:30.137532+0000","last_undegraded":"2020-11-28T21:46:30.137532+0000","last_fullsized":"2020-11-28T21:46:30.137532+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:27.758912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:27.758912+0000","last_clean_scrub_stamp":"2020-11-28T21:46:27.758912+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048816,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323024,"statfs":{"total":96636764160,"available":95562776576,"internally_reserved":1073741824,"allocated":237568,"data_stored":88251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.706}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6459999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4319999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7349999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9460000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.0090000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.8550000000000004}]}]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048844,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322996,"statfs":{"total":96636764160,"available":95562747904,"internally_reserved":1073741824,"allocated":266240,"data_stored":90521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.659}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8660000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2389999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1019999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2320000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.379}]}]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048816,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323024,"statfs":{"total":96636764160,"available":95562776576,"internally_reserved":1073741824,"allocated":237568,"data_stored":88251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.325}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.383}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0910000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6879999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0150000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5470000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7589999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048820,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323020,"statfs":{"total":96636764160,"available":95562772480,"internally_reserved":1073741824,"allocated":241664,"data_stored":88285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3380000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3650000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.575}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6850000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.758}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.8019999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.319}]}]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048828,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323012,"statfs":{"total":96636764160,"available":95562764288,"internally_reserved":1073741824,"allocated":249856,"data_stored":88445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7090000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.923}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1459999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8620000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.4009999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3260000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.5419999999999998}]}]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048816,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323024,"statfs":{"total":96636764160,"available":95562776576,"internally_reserved":1073741824,"allocated":237568,"data_stored":88251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.421}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5329999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0819999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.0330000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.6629999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.9580000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0899999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048824,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323016,"statfs":{"total":96636764160,"available":95562768384,"internally_reserved":1073741824,"allocated":245760,"data_stored":88826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.484}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.595}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8620000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.863}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0419999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9809999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.694}]}]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048824,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323016,"statfs":{"total":96636764160,"available":95562768384,"internally_reserved":1073741824,"allocated":245760,"data_stored":89183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5409999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4710000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0449999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.25}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1629999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.5590000000000002}]}]}]}} 2020-11-28T21:46:38.499 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-11-28T21:46:38.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.611+0000 7effd759e700 1 Processor -- start 2020-11-28T21:46:38.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.611+0000 7effd759e700 1 -- start start 2020-11-28T21:46:38.616 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.611+0000 7effd759e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7effd8142a00 0x7effd80b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:38.616 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.611+0000 7effd759e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effd80b4110 con 0x7effd80b4c70 2020-11-28T21:46:38.616 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.611+0000 7effd759e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effd80b43f0 con 0x7effd80603d0 2020-11-28T21:46:38.616 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.611+0000 7effd759e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effd8060f60 con 0x7effd8142a00 2020-11-28T21:46:38.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.611+0000 7effcffff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7effd8142a00 0x7effd80b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:38.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.611+0000 7effd659c700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7effd80603d0 0x7effd809cfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:44484/0 (socket says 172.21.15.162:44484) 2020-11-28T21:46:38.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.611+0000 7effd659c700 1 -- 172.21.15.162:0/3868788336 learned_addr learned my addr 172.21.15.162:0/3868788336 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:38.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.611+0000 7effcffff700 1 -- 172.21.15.162:0/3868788336 >> v1:172.21.15.162:6789/0 conn(0x7effd80603d0 legacy=0x7effd809cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:38.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.611+0000 7effd5d9b700 1 -- 172.21.15.162:0/3868788336 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2162553505 0 0) 0x7effd80b43f0 con 0x7effd80603d0 2020-11-28T21:46:38.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.611+0000 7effcffff700 1 -- 172.21.15.162:0/3868788336 >> v1:172.21.15.118:6789/0 conn(0x7effd80b4c70 legacy=0x7effd8143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:38.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.611+0000 7effcffff700 1 -- 172.21.15.162:0/3868788336 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd80b4690 con 0x7effd8142a00 2020-11-28T21:46:38.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.611+0000 7effcffff700 1 --2- 172.21.15.162:0/3868788336 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7effd8142a00 0x7effd80b3a70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7effc0002f30 tx=0x7effc0006dc0).ready entity=mon.1 client_cookie=e668dcf84366862a server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:38.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.611+0000 7effd5d9b700 1 -- 172.21.15.162:0/3868788336 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effc00097f0 con 0x7effd8142a00 2020-11-28T21:46:38.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.611+0000 7effd5d9b700 1 -- 172.21.15.162:0/3868788336 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc0009950 con 0x7effd8142a00 2020-11-28T21:46:38.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.611+0000 7effd759e700 1 -- 172.21.15.162:0/3868788336 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7effd8142a00 msgr2=0x7effd80b3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:38.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.611+0000 7effd5d9b700 1 -- 172.21.15.162:0/3868788336 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effc0009cb0 con 0x7effd8142a00 2020-11-28T21:46:38.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.611+0000 7effd759e700 1 --2- 172.21.15.162:0/3868788336 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7effd8142a00 0x7effd80b3a70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7effc0002f30 tx=0x7effc0006dc0).stop 2020-11-28T21:46:38.622 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.615+0000 7effd759e700 1 -- 172.21.15.162:0/3868788336 shutdown_connections 2020-11-28T21:46:38.622 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.615+0000 7effd759e700 1 --2- 172.21.15.162:0/3868788336 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7effd8142a00 0x7effd80b3a70 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:38.622 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.615+0000 7effd759e700 1 -- 172.21.15.162:0/3868788336 >> 172.21.15.162:0/3868788336 conn(0x7effd8066a40 msgr2=0x7effd8066e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:38.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.615+0000 7effd759e700 1 -- 172.21.15.162:0/3868788336 shutdown_connections 2020-11-28T21:46:38.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.615+0000 7effd759e700 1 -- 172.21.15.162:0/3868788336 wait complete. 2020-11-28T21:46:38.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.615+0000 7effd759e700 1 Processor -- start 2020-11-28T21:46:38.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.615+0000 7effd759e700 1 -- start start 2020-11-28T21:46:38.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.615+0000 7effd759e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7effd80603d0 0x7effd80b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:38.624 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.615+0000 7effd759e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7effd80b4c70 0x7effd81467d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:38.624 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.615+0000 7effd759e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7effd80b0f40 0x7effd81408a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:38.624 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.615+0000 7effd759e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd80b25b0 con 0x7effd80603d0 2020-11-28T21:46:38.624 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.615+0000 7effd759e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd81521e0 con 0x7effd80b4c70 2020-11-28T21:46:38.624 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.615+0000 7effd759e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effd80b0780 con 0x7effd80b0f40 2020-11-28T21:46:38.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.615+0000 7effcffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7effd80b4c70 0x7effd81467d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:38.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.615+0000 7effcffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7effd80b4c70 0x7effd81467d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37648/0 (socket says 172.21.15.162:37648) 2020-11-28T21:46:38.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.615+0000 7effcffff700 1 -- 172.21.15.162:0/1800513031 learned_addr learned my addr 172.21.15.162:0/1800513031 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:38.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.615+0000 7effd6d9d700 1 --2- 172.21.15.162:0/1800513031 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7effd80b0f40 0x7effd81408a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:38.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.615+0000 7effd659c700 1 --2- 172.21.15.162:0/1800513031 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7effd80603d0 0x7effd80b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:38.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.615+0000 7effcffff700 1 -- 172.21.15.162:0/1800513031 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7effd80b0f40 msgr2=0x7effd81408a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:38.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.615+0000 7effcffff700 1 --2- 172.21.15.162:0/1800513031 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7effd80b0f40 0x7effd81408a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:38.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.615+0000 7effcffff700 1 -- 172.21.15.162:0/1800513031 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7effd80603d0 msgr2=0x7effd80b13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:38.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.615+0000 7effcffff700 1 --2- 172.21.15.162:0/1800513031 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7effd80603d0 0x7effd80b13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:38.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.615+0000 7effcffff700 1 -- 172.21.15.162:0/1800513031 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd80b04e0 con 0x7effd80b4c70 2020-11-28T21:46:38.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.615+0000 7effd659c700 1 --2- 172.21.15.162:0/1800513031 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7effd80603d0 0x7effd80b13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:46:38.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.615+0000 7effcffff700 1 --2- 172.21.15.162:0/1800513031 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7effd80b4c70 0x7effd81467d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7effd80b0910 tx=0x7effc0015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:38.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.615+0000 7effcf7fe700 1 -- 172.21.15.162:0/1800513031 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effc0021070 con 0x7effd80b4c70 2020-11-28T21:46:38.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.615+0000 7effcf7fe700 1 -- 172.21.15.162:0/1800513031 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc001d450 con 0x7effd80b4c70 2020-11-28T21:46:38.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.615+0000 7effcf7fe700 1 -- 172.21.15.162:0/1800513031 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effc001d7b0 con 0x7effd80b4c70 2020-11-28T21:46:38.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.615+0000 7effd759e700 1 -- 172.21.15.162:0/1800513031 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effd8152440 con 0x7effd80b4c70 2020-11-28T21:46:38.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.615+0000 7effd759e700 1 -- 172.21.15.162:0/1800513031 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effd8146cc0 con 0x7effd80b4c70 2020-11-28T21:46:38.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.619+0000 7effd759e700 1 -- 172.21.15.162:0/1800513031 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effd8140f30 con 0x7effd80b4c70 2020-11-28T21:46:38.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.619+0000 7effcf7fe700 1 -- 172.21.15.162:0/1800513031 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7effc000f070 con 0x7effd80b4c70 2020-11-28T21:46:38.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.619+0000 7effcf7fe700 1 --2- 172.21.15.162:0/1800513031 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7effbc0554e0 0x7effbc0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:38.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.619+0000 7effcf7fe700 1 -- 172.21.15.162:0/1800513031 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7effc006f380 con 0x7effd80b4c70 2020-11-28T21:46:38.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.619+0000 7effd659c700 1 --2- 172.21.15.162:0/1800513031 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7effbc0554e0 0x7effbc0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:38.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.619+0000 7effd659c700 1 --2- 172.21.15.162:0/1800513031 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7effbc0554e0 0x7effbc0579a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7effc8008000 tx=0x7effc8009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:38.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.623+0000 7effcf7fe700 1 -- 172.21.15.162:0/1800513031 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7effd8140f30 con 0x7effd80b4c70 2020-11-28T21:46:38.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.759+0000 7effd759e700 1 -- 172.21.15.162:0/1800513031 --> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7effd8140f30 con 0x7effbc0554e0 2020-11-28T21:46:38.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.767+0000 7effcf7fe700 1 -- 172.21.15.162:0/1800513031 <== mgr.4105 v2:172.21.15.162:6800/12156 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82285 (secure 0 0 0) 0x7effd8140f30 con 0x7effbc0554e0 2020-11-28T21:46:38.774 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:46:38.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.771+0000 7effd759e700 1 -- 172.21.15.162:0/1800513031 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7effbc0554e0 msgr2=0x7effbc0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:38.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.771+0000 7effd759e700 1 --2- 172.21.15.162:0/1800513031 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7effbc0554e0 0x7effbc0579a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7effc8008000 tx=0x7effc8009040).stop 2020-11-28T21:46:38.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.771+0000 7effd759e700 1 -- 172.21.15.162:0/1800513031 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7effd80b4c70 msgr2=0x7effd81467d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:38.776 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.771+0000 7effd759e700 1 --2- 172.21.15.162:0/1800513031 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7effd80b4c70 0x7effd81467d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7effd80b0910 tx=0x7effc0015000).stop 2020-11-28T21:46:38.776 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.771+0000 7effd759e700 1 -- 172.21.15.162:0/1800513031 shutdown_connections 2020-11-28T21:46:38.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.771+0000 7effd759e700 1 --2- 172.21.15.162:0/1800513031 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7effbc0554e0 0x7effbc0579a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:38.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.771+0000 7effd759e700 1 --2- 172.21.15.162:0/1800513031 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7effd80b0f40 0x7effd81408a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:38.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.771+0000 7effd759e700 1 --2- 172.21.15.162:0/1800513031 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7effd80603d0 0x7effd80b13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:38.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.771+0000 7effd759e700 1 --2- 172.21.15.162:0/1800513031 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7effd80b4c70 0x7effd81467d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:38.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.771+0000 7effd759e700 1 -- 172.21.15.162:0/1800513031 >> 172.21.15.162:0/1800513031 conn(0x7effd8066a40 msgr2=0x7effd809cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:38.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.771+0000 7effd759e700 1 -- 172.21.15.162:0/1800513031 shutdown_connections 2020-11-28T21:46:38.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.771+0000 7effd759e700 1 -- 172.21.15.162:0/1800513031 wait complete. 2020-11-28T21:46:38.780 INFO:teuthology.orchestra.run.smithi162.stderr:dumped all 2020-11-28T21:46:38.785 INFO:teuthology.orchestra.run.smithi162.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2020-11-28T21:46:37.576538+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2304,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":46,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8390588,"kb_used_data":1916,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746584132,"statfs":{"total":773094113280,"available":764502151168,"internally_reserved":8589934592,"allocated":1961984,"data_stored":710013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.449391"},"pg_stats":[{"pgid":"4.2","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.146922+0000","last_change":"2020-11-28T21:46:29.143409+0000","last_active":"2020-11-28T21:46:29.146922+0000","last_peered":"2020-11-28T21:46:29.146922+0000","last_clean":"2020-11-28T21:46:29.146922+0000","last_became_active":"2020-11-28T21:46:29.142911+0000","last_became_peered":"2020-11-28T21:46:29.142911+0000","last_unstale":"2020-11-28T21:46:29.146922+0000","last_undegraded":"2020-11-28T21:46:29.146922+0000","last_fullsized":"2020-11-28T21:46:29.146922+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:28.767144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:28.767144+0000","last_clean_scrub_stamp":"2020-11-28T21:46:28.767144+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"12","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.130289+0000","last_change":"2020-11-28T21:46:25.652727+0000","last_active":"2020-11-28T21:46:29.130289+0000","last_peered":"2020-11-28T21:46:29.130289+0000","last_clean":"2020-11-28T21:46:29.130289+0000","last_became_active":"2020-11-28T21:46:25.652265+0000","last_became_peered":"2020-11-28T21:46:25.652265+0000","last_unstale":"2020-11-28T21:46:29.130289+0000","last_undegraded":"2020-11-28T21:46:29.130289+0000","last_fullsized":"2020-11-28T21:46:29.130289+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_clean_scrub_stamp":"2020-11-28T21:46:24.622452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"22'6","reported_seq":"16","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.148880+0000","last_change":"2020-11-28T21:46:28.798375+0000","last_active":"2020-11-28T21:46:29.148880+0000","last_peered":"2020-11-28T21:46:29.148880+0000","last_clean":"2020-11-28T21:46:29.148880+0000","last_became_active":"2020-11-28T21:46:28.797984+0000","last_became_peered":"2020-11-28T21:46:28.797984+0000","last_unstale":"2020-11-28T21:46:29.148880+0000","last_undegraded":"2020-11-28T21:46:29.148880+0000","last_fullsized":"2020-11-28T21:46:29.148880+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:27.758912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:27.758912+0000","last_clean_scrub_stamp":"2020-11-28T21:46:27.758912+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.150049+0000","last_change":"2020-11-28T21:46:29.147464+0000","last_active":"2020-11-28T21:46:29.150049+0000","last_peered":"2020-11-28T21:46:29.150049+0000","last_clean":"2020-11-28T21:46:29.150049+0000","last_became_active":"2020-11-28T21:46:29.147036+0000","last_became_peered":"2020-11-28T21:46:29.147036+0000","last_unstale":"2020-11-28T21:46:29.150049+0000","last_undegraded":"2020-11-28T21:46:29.150049+0000","last_fullsized":"2020-11-28T21:46:29.150049+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:28.767144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:28.767144+0000","last_clean_scrub_stamp":"2020-11-28T21:46:28.767144+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"12","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.137303+0000","last_change":"2020-11-28T21:46:25.654101+0000","last_active":"2020-11-28T21:46:29.137303+0000","last_peered":"2020-11-28T21:46:29.137303+0000","last_clean":"2020-11-28T21:46:29.137303+0000","last_became_active":"2020-11-28T21:46:25.653417+0000","last_became_peered":"2020-11-28T21:46:25.653417+0000","last_unstale":"2020-11-28T21:46:29.137303+0000","last_undegraded":"2020-11-28T21:46:29.137303+0000","last_fullsized":"2020-11-28T21:46:29.137303+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_clean_scrub_stamp":"2020-11-28T21:46:24.622452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"22'5","reported_seq":"15","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.153309+0000","last_change":"2020-11-28T21:46:28.797447+0000","last_active":"2020-11-28T21:46:29.153309+0000","last_peered":"2020-11-28T21:46:29.153309+0000","last_clean":"2020-11-28T21:46:29.153309+0000","last_became_active":"2020-11-28T21:46:28.796944+0000","last_became_peered":"2020-11-28T21:46:28.796944+0000","last_unstale":"2020-11-28T21:46:29.153309+0000","last_undegraded":"2020-11-28T21:46:29.153309+0000","last_fullsized":"2020-11-28T21:46:29.153309+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:27.758912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:27.758912+0000","last_clean_scrub_stamp":"2020-11-28T21:46:27.758912+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.151290+0000","last_change":"2020-11-28T21:46:29.150407+0000","last_active":"2020-11-28T21:46:29.151290+0000","last_peered":"2020-11-28T21:46:29.151290+0000","last_clean":"2020-11-28T21:46:29.151290+0000","last_became_active":"2020-11-28T21:46:29.149655+0000","last_became_peered":"2020-11-28T21:46:29.149655+0000","last_unstale":"2020-11-28T21:46:29.151290+0000","last_undegraded":"2020-11-28T21:46:29.151290+0000","last_fullsized":"2020-11-28T21:46:29.151290+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:28.767144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:28.767144+0000","last_clean_scrub_stamp":"2020-11-28T21:46:28.767144+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"12","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.141495+0000","last_change":"2020-11-28T21:46:25.652888+0000","last_active":"2020-11-28T21:46:29.141495+0000","last_peered":"2020-11-28T21:46:29.141495+0000","last_clean":"2020-11-28T21:46:29.141495+0000","last_became_active":"2020-11-28T21:46:25.652295+0000","last_became_peered":"2020-11-28T21:46:25.652295+0000","last_unstale":"2020-11-28T21:46:29.141495+0000","last_undegraded":"2020-11-28T21:46:29.141495+0000","last_fullsized":"2020-11-28T21:46:29.141495+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_clean_scrub_stamp":"2020-11-28T21:46:24.622452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"22'8","reported_seq":"18","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.151518+0000","last_change":"2020-11-28T21:46:28.795437+0000","last_active":"2020-11-28T21:46:29.151518+0000","last_peered":"2020-11-28T21:46:29.151518+0000","last_clean":"2020-11-28T21:46:29.151518+0000","last_became_active":"2020-11-28T21:46:28.795076+0000","last_became_peered":"2020-11-28T21:46:28.795076+0000","last_unstale":"2020-11-28T21:46:29.151518+0000","last_undegraded":"2020-11-28T21:46:29.151518+0000","last_fullsized":"2020-11-28T21:46:29.151518+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:27.758912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:27.758912+0000","last_clean_scrub_stamp":"2020-11-28T21:46:27.758912+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.145558+0000","last_change":"2020-11-28T21:46:29.143471+0000","last_active":"2020-11-28T21:46:29.145558+0000","last_peered":"2020-11-28T21:46:29.145558+0000","last_clean":"2020-11-28T21:46:29.145558+0000","last_became_active":"2020-11-28T21:46:29.143023+0000","last_became_peered":"2020-11-28T21:46:29.143023+0000","last_unstale":"2020-11-28T21:46:29.145558+0000","last_undegraded":"2020-11-28T21:46:29.145558+0000","last_fullsized":"2020-11-28T21:46:29.145558+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:28.767144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:28.767144+0000","last_clean_scrub_stamp":"2020-11-28T21:46:28.767144+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"12","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.137448+0000","last_change":"2020-11-28T21:46:26.008971+0000","last_active":"2020-11-28T21:46:29.137448+0000","last_peered":"2020-11-28T21:46:29.137448+0000","last_clean":"2020-11-28T21:46:29.137448+0000","last_became_active":"2020-11-28T21:46:26.007744+0000","last_became_peered":"2020-11-28T21:46:26.007744+0000","last_unstale":"2020-11-28T21:46:29.137448+0000","last_undegraded":"2020-11-28T21:46:29.137448+0000","last_fullsized":"2020-11-28T21:46:29.137448+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_clean_scrub_stamp":"2020-11-28T21:46:24.622452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"16","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.135778+0000","last_change":"2020-11-28T21:46:20.941819+0000","last_active":"2020-11-28T21:46:29.135778+0000","last_peered":"2020-11-28T21:46:29.135778+0000","last_clean":"2020-11-28T21:46:29.135778+0000","last_became_active":"2020-11-28T21:46:20.941467+0000","last_became_peered":"2020-11-28T21:46:20.941467+0000","last_unstale":"2020-11-28T21:46:29.135778+0000","last_undegraded":"2020-11-28T21:46:29.135778+0000","last_fullsized":"2020-11-28T21:46:29.135778+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:16.570066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:16.570066+0000","last_clean_scrub_stamp":"2020-11-28T21:46:16.570066+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"22'1","reported_seq":"11","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.149810+0000","last_change":"2020-11-28T21:46:28.795437+0000","last_active":"2020-11-28T21:46:29.149810+0000","last_peered":"2020-11-28T21:46:29.149810+0000","last_clean":"2020-11-28T21:46:29.149810+0000","last_became_active":"2020-11-28T21:46:28.795024+0000","last_became_peered":"2020-11-28T21:46:28.795024+0000","last_unstale":"2020-11-28T21:46:29.149810+0000","last_undegraded":"2020-11-28T21:46:29.149810+0000","last_fullsized":"2020-11-28T21:46:29.149810+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:27.758912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:27.758912+0000","last_clean_scrub_stamp":"2020-11-28T21:46:27.758912+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.150192+0000","last_change":"2020-11-28T21:46:29.149147+0000","last_active":"2020-11-28T21:46:29.150192+0000","last_peered":"2020-11-28T21:46:29.150192+0000","last_clean":"2020-11-28T21:46:29.150192+0000","last_became_active":"2020-11-28T21:46:29.148658+0000","last_became_peered":"2020-11-28T21:46:29.148658+0000","last_unstale":"2020-11-28T21:46:29.150192+0000","last_undegraded":"2020-11-28T21:46:29.150192+0000","last_fullsized":"2020-11-28T21:46:29.150192+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:28.767144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:28.767144+0000","last_clean_scrub_stamp":"2020-11-28T21:46:28.767144+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"12","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.136616+0000","last_change":"2020-11-28T21:46:25.654099+0000","last_active":"2020-11-28T21:46:29.136616+0000","last_peered":"2020-11-28T21:46:29.136616+0000","last_clean":"2020-11-28T21:46:29.136616+0000","last_became_active":"2020-11-28T21:46:25.653652+0000","last_became_peered":"2020-11-28T21:46:25.653652+0000","last_unstale":"2020-11-28T21:46:29.136616+0000","last_undegraded":"2020-11-28T21:46:29.136616+0000","last_fullsized":"2020-11-28T21:46:29.136616+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_clean_scrub_stamp":"2020-11-28T21:46:24.622452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"10","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.128896+0000","last_change":"2020-11-28T21:46:28.794379+0000","last_active":"2020-11-28T21:46:29.128896+0000","last_peered":"2020-11-28T21:46:29.128896+0000","last_clean":"2020-11-28T21:46:29.128896+0000","last_became_active":"2020-11-28T21:46:28.793915+0000","last_became_peered":"2020-11-28T21:46:28.793915+0000","last_unstale":"2020-11-28T21:46:29.128896+0000","last_undegraded":"2020-11-28T21:46:29.128896+0000","last_fullsized":"2020-11-28T21:46:29.128896+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:27.758912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:27.758912+0000","last_clean_scrub_stamp":"2020-11-28T21:46:27.758912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.150922+0000","last_change":"2020-11-28T21:46:29.149491+0000","last_active":"2020-11-28T21:46:29.150922+0000","last_peered":"2020-11-28T21:46:29.150922+0000","last_clean":"2020-11-28T21:46:29.150922+0000","last_became_active":"2020-11-28T21:46:29.149008+0000","last_became_peered":"2020-11-28T21:46:29.149008+0000","last_unstale":"2020-11-28T21:46:29.150922+0000","last_undegraded":"2020-11-28T21:46:29.150922+0000","last_fullsized":"2020-11-28T21:46:29.150922+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:28.767144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:28.767144+0000","last_clean_scrub_stamp":"2020-11-28T21:46:28.767144+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"12","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.138052+0000","last_change":"2020-11-28T21:46:25.653672+0000","last_active":"2020-11-28T21:46:29.138052+0000","last_peered":"2020-11-28T21:46:29.138052+0000","last_clean":"2020-11-28T21:46:29.138052+0000","last_became_active":"2020-11-28T21:46:25.653222+0000","last_became_peered":"2020-11-28T21:46:25.653222+0000","last_unstale":"2020-11-28T21:46:29.138052+0000","last_undegraded":"2020-11-28T21:46:29.138052+0000","last_fullsized":"2020-11-28T21:46:29.138052+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_clean_scrub_stamp":"2020-11-28T21:46:24.622452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"22'8","reported_seq":"18","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.154177+0000","last_change":"2020-11-28T21:46:28.793524+0000","last_active":"2020-11-28T21:46:29.154177+0000","last_peered":"2020-11-28T21:46:29.154177+0000","last_clean":"2020-11-28T21:46:29.154177+0000","last_became_active":"2020-11-28T21:46:28.793077+0000","last_became_peered":"2020-11-28T21:46:28.793077+0000","last_unstale":"2020-11-28T21:46:29.154177+0000","last_undegraded":"2020-11-28T21:46:29.154177+0000","last_fullsized":"2020-11-28T21:46:29.154177+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:27.758912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:27.758912+0000","last_clean_scrub_stamp":"2020-11-28T21:46:27.758912+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1535,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.151553+0000","last_change":"2020-11-28T21:46:29.150884+0000","last_active":"2020-11-28T21:46:29.151553+0000","last_peered":"2020-11-28T21:46:29.151553+0000","last_clean":"2020-11-28T21:46:29.151553+0000","last_became_active":"2020-11-28T21:46:29.150424+0000","last_became_peered":"2020-11-28T21:46:29.150424+0000","last_unstale":"2020-11-28T21:46:29.151553+0000","last_undegraded":"2020-11-28T21:46:29.151553+0000","last_fullsized":"2020-11-28T21:46:29.151553+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:28.767144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:28.767144+0000","last_clean_scrub_stamp":"2020-11-28T21:46:28.767144+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"22'7","reported_seq":"17","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.151619+0000","last_change":"2020-11-28T21:46:28.791205+0000","last_active":"2020-11-28T21:46:29.151619+0000","last_peered":"2020-11-28T21:46:29.151619+0000","last_clean":"2020-11-28T21:46:29.151619+0000","last_became_active":"2020-11-28T21:46:28.790564+0000","last_became_peered":"2020-11-28T21:46:28.790564+0000","last_unstale":"2020-11-28T21:46:29.151619+0000","last_undegraded":"2020-11-28T21:46:29.151619+0000","last_fullsized":"2020-11-28T21:46:29.151619+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:27.758912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:27.758912+0000","last_clean_scrub_stamp":"2020-11-28T21:46:27.758912+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":575,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.136302+0000","last_change":"2020-11-28T21:46:25.838546+0000","last_active":"2020-11-28T21:46:29.136302+0000","last_peered":"2020-11-28T21:46:29.136302+0000","last_clean":"2020-11-28T21:46:29.136302+0000","last_became_active":"2020-11-28T21:46:25.837556+0000","last_became_peered":"2020-11-28T21:46:25.837556+0000","last_unstale":"2020-11-28T21:46:29.136302+0000","last_undegraded":"2020-11-28T21:46:29.136302+0000","last_fullsized":"2020-11-28T21:46:29.136302+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_clean_scrub_stamp":"2020-11-28T21:46:24.622452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.151643+0000","last_change":"2020-11-28T21:46:29.144515+0000","last_active":"2020-11-28T21:46:29.151643+0000","last_peered":"2020-11-28T21:46:29.151643+0000","last_clean":"2020-11-28T21:46:29.151643+0000","last_became_active":"2020-11-28T21:46:29.144141+0000","last_became_peered":"2020-11-28T21:46:29.144141+0000","last_unstale":"2020-11-28T21:46:29.151643+0000","last_undegraded":"2020-11-28T21:46:29.151643+0000","last_fullsized":"2020-11-28T21:46:29.151643+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:28.767144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:28.767144+0000","last_clean_scrub_stamp":"2020-11-28T21:46:28.767144+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"12","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-28T21:46:29.137554+0000","last_change":"2020-11-28T21:46:25.838687+0000","last_active":"2020-11-28T21:46:29.137554+0000","last_peered":"2020-11-28T21:46:29.137554+0000","last_clean":"2020-11-28T21:46:29.137554+0000","last_became_active":"2020-11-28T21:46:25.837943+0000","last_became_peered":"2020-11-28T21:46:25.837943+0000","last_unstale":"2020-11-28T21:46:29.137554+0000","last_undegraded":"2020-11-28T21:46:29.137554+0000","last_fullsized":"2020-11-28T21:46:29.137554+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_clean_scrub_stamp":"2020-11-28T21:46:24.622452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"23'4","reported_seq":"14","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-28T21:46:30.137532+0000","last_change":"2020-11-28T21:46:28.791512+0000","last_active":"2020-11-28T21:46:30.137532+0000","last_peered":"2020-11-28T21:46:30.137532+0000","last_clean":"2020-11-28T21:46:30.137532+0000","last_became_active":"2020-11-28T21:46:28.790599+0000","last_became_peered":"2020-11-28T21:46:28.790599+0000","last_unstale":"2020-11-28T21:46:30.137532+0000","last_undegraded":"2020-11-28T21:46:30.137532+0000","last_fullsized":"2020-11-28T21:46:30.137532+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:27.758912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:27.758912+0000","last_clean_scrub_stamp":"2020-11-28T21:46:27.758912+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048816,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323024,"statfs":{"total":96636764160,"available":95562776576,"internally_reserved":1073741824,"allocated":237568,"data_stored":88251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.706}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6459999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4319999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7349999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9460000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.0090000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.8550000000000004}]}]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048844,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322996,"statfs":{"total":96636764160,"available":95562747904,"internally_reserved":1073741824,"allocated":266240,"data_stored":90521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.659}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8660000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2389999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1019999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2320000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.379}]}]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048816,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323024,"statfs":{"total":96636764160,"available":95562776576,"internally_reserved":1073741824,"allocated":237568,"data_stored":88251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.325}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.383}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0910000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6879999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0150000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5470000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7589999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048820,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323020,"statfs":{"total":96636764160,"available":95562772480,"internally_reserved":1073741824,"allocated":241664,"data_stored":88285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3380000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3650000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.575}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6850000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.758}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.8019999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.319}]}]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048828,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323012,"statfs":{"total":96636764160,"available":95562764288,"internally_reserved":1073741824,"allocated":249856,"data_stored":88445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7090000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.923}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1459999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8620000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.4009999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3260000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.5419999999999998}]}]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048816,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323024,"statfs":{"total":96636764160,"available":95562776576,"internally_reserved":1073741824,"allocated":237568,"data_stored":88251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.421}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5329999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0819999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.0330000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.6629999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.9580000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0899999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048824,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323016,"statfs":{"total":96636764160,"available":95562768384,"internally_reserved":1073741824,"allocated":245760,"data_stored":88826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.484}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.595}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8620000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.863}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0419999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9809999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.694}]}]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048824,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323016,"statfs":{"total":96636764160,"available":95562768384,"internally_reserved":1073741824,"allocated":245760,"data_stored":89183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5409999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4710000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0449999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.25}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1629999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.5590000000000002}]}]}]}} 2020-11-28T21:46:38.789 INFO:tasks.ceph.ceph_manager.ceph:clean! 2020-11-28T21:46:38.789 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2020-11-28T21:46:38.790 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2020-11-28T21:46:38.790 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early health --format=json 2020-11-28T21:46:38.898 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.895+0000 7efcf6faa700 1 Processor -- start 2020-11-28T21:46:38.898 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.895+0000 7efcf6faa700 1 -- start start 2020-11-28T21:46:38.902 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.895+0000 7efcf6faa700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7efcf00603d0 0x7efcf00b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:38.902 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.895+0000 7efcf6faa700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efcf0142a00 0x7efcf009cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:38.903 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.895+0000 7efcf6faa700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7efcf00b4c70 0x7efcf009d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:38.903 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.895+0000 7efcf6faa700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcf0060f60 con 0x7efcf00603d0 2020-11-28T21:46:38.903 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.895+0000 7efcf6faa700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcf00b4110 con 0x7efcf00b4c70 2020-11-28T21:46:38.904 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.895+0000 7efcf6faa700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcf00b43f0 con 0x7efcf0142a00 2020-11-28T21:46:38.904 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.895+0000 7efcf67a9700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7efcf00b4c70 0x7efcf009d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:38.905 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.895+0000 7efcf67a9700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7efcf00b4c70 0x7efcf009d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37658/0 (socket says 172.21.15.162:37658) 2020-11-28T21:46:38.905 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.895+0000 7efcf67a9700 1 -- 172.21.15.162:0/78289438 learned_addr learned my addr 172.21.15.162:0/78289438 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:38.905 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.895+0000 7efcf57a7700 1 --2- 172.21.15.162:0/78289438 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efcf0142a00 0x7efcf009cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:38.906 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.895+0000 7efcf5fa8700 1 --2- 172.21.15.162:0/78289438 >> v2:172.21.15.118:3300/0 conn(0x7efcf00603d0 0x7efcf00b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:38.906 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.895+0000 7efcf67a9700 1 -- 172.21.15.162:0/78289438 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efcf0142a00 msgr2=0x7efcf009cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:38.906 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.895+0000 7efcf67a9700 1 --2- 172.21.15.162:0/78289438 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efcf0142a00 0x7efcf009cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:38.907 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.895+0000 7efcf67a9700 1 -- 172.21.15.162:0/78289438 >> v2:172.21.15.118:3300/0 conn(0x7efcf00603d0 msgr2=0x7efcf00b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:38.907 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.895+0000 7efcf67a9700 1 --2- 172.21.15.162:0/78289438 >> v2:172.21.15.118:3300/0 conn(0x7efcf00603d0 0x7efcf00b3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:38.907 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.895+0000 7efcf67a9700 1 -- 172.21.15.162:0/78289438 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf00b4690 con 0x7efcf00b4c70 2020-11-28T21:46:38.907 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.895+0000 7efcf67a9700 1 --2- 172.21.15.162:0/78289438 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcf00b4c70 0x7efcf009d4b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7efce80032b0 tx=0x7efce8009270).ready entity=mon.1 client_cookie=11ff19fd3e0a8fe4 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:38.908 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.895+0000 7efcf4fa6700 1 -- 172.21.15.162:0/78289438 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efce8009e60 con 0x7efcf00b4c70 2020-11-28T21:46:38.908 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.899+0000 7efcf4fa6700 1 -- 172.21.15.162:0/78289438 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce800bc00 con 0x7efcf00b4c70 2020-11-28T21:46:38.908 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.899+0000 7efcf4fa6700 1 -- 172.21.15.162:0/78289438 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efce800bd70 con 0x7efcf00b4c70 2020-11-28T21:46:38.908 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.899+0000 7efcf6faa700 1 -- 172.21.15.162:0/78289438 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcf00b4c70 msgr2=0x7efcf009d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:38.909 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.899+0000 7efcf6faa700 1 --2- 172.21.15.162:0/78289438 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcf00b4c70 0x7efcf009d4b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7efce80032b0 tx=0x7efce8009270).stop 2020-11-28T21:46:38.909 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.899+0000 7efcf6faa700 1 -- 172.21.15.162:0/78289438 shutdown_connections 2020-11-28T21:46:38.909 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.899+0000 7efcf6faa700 1 --2- 172.21.15.162:0/78289438 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcf00b4c70 0x7efcf009d4b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:38.909 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.899+0000 7efcf6faa700 1 --2- 172.21.15.162:0/78289438 >> v2:172.21.15.118:3300/0 conn(0x7efcf00603d0 0x7efcf00b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:38.910 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.899+0000 7efcf6faa700 1 --2- 172.21.15.162:0/78289438 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efcf0142a00 0x7efcf009cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:38.910 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.899+0000 7efcf6faa700 1 -- 172.21.15.162:0/78289438 >> 172.21.15.162:0/78289438 conn(0x7efcf0066a40 msgr2=0x7efcf0066e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:38.910 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.899+0000 7efcf6faa700 1 -- 172.21.15.162:0/78289438 shutdown_connections 2020-11-28T21:46:38.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.899+0000 7efcf6faa700 1 -- 172.21.15.162:0/78289438 wait complete. 2020-11-28T21:46:38.917 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.899+0000 7efcf6faa700 1 Processor -- start 2020-11-28T21:46:38.917 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.899+0000 7efcf6faa700 1 -- start start 2020-11-28T21:46:38.918 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.903+0000 7efcf6faa700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcf00603d0 0x7efcf0146d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:38.918 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.903+0000 7efcf6faa700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcf00b4c70 0x7efcf01520d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:38.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.903+0000 7efcf6faa700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efcf00afb30 0x7efcf0154c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:38.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.903+0000 7efcf6faa700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf00b3400 con 0x7efcf00b4c70 2020-11-28T21:46:38.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.903+0000 7efcf6faa700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf00b2ec0 con 0x7efcf00603d0 2020-11-28T21:46:38.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.903+0000 7efcf6faa700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcf00b3160 con 0x7efcf00afb30 2020-11-28T21:46:38.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.903+0000 7efcf5fa8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcf00603d0 0x7efcf0146d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:38.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.903+0000 7efcf5fa8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcf00603d0 0x7efcf0146d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37660/0 (socket says 172.21.15.162:37660) 2020-11-28T21:46:38.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.903+0000 7efcf5fa8700 1 -- 172.21.15.162:0/1054041120 learned_addr learned my addr 172.21.15.162:0/1054041120 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:38.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.903+0000 7efcf57a7700 1 --2- 172.21.15.162:0/1054041120 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcf00b4c70 0x7efcf01520d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:38.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.903+0000 7efcf67a9700 1 --2- 172.21.15.162:0/1054041120 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efcf00afb30 0x7efcf0154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:38.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.903+0000 7efcf5fa8700 1 -- 172.21.15.162:0/1054041120 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efcf00afb30 msgr2=0x7efcf0154c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:38.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.903+0000 7efcf5fa8700 1 --2- 172.21.15.162:0/1054041120 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efcf00afb30 0x7efcf0154c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:38.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.903+0000 7efcf5fa8700 1 -- 172.21.15.162:0/1054041120 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcf00b4c70 msgr2=0x7efcf01520d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:38.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.903+0000 7efcf5fa8700 1 --2- 172.21.15.162:0/1054041120 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcf00b4c70 0x7efcf01520d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:38.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.903+0000 7efcf5fa8700 1 -- 172.21.15.162:0/1054041120 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf00b2620 con 0x7efcf00603d0 2020-11-28T21:46:38.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.903+0000 7efcf67a9700 1 --2- 172.21.15.162:0/1054041120 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efcf00afb30 0x7efcf0154c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:46:38.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.903+0000 7efcf5fa8700 1 --2- 172.21.15.162:0/1054041120 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcf00603d0 0x7efcf0146d60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7efce0003070 tx=0x7efce0008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:38.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.903+0000 7efcdeffd700 1 -- 172.21.15.162:0/1054041120 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efce0009980 con 0x7efcf00603d0 2020-11-28T21:46:38.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.903+0000 7efcdeffd700 1 -- 172.21.15.162:0/1054041120 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce0009ae0 con 0x7efcf00603d0 2020-11-28T21:46:38.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.903+0000 7efcf6faa700 1 -- 172.21.15.162:0/1054041120 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcf01525c0 con 0x7efcf00603d0 2020-11-28T21:46:38.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.903+0000 7efcdeffd700 1 -- 172.21.15.162:0/1054041120 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efce0009e40 con 0x7efcf00603d0 2020-11-28T21:46:38.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.903+0000 7efcf6faa700 1 -- 172.21.15.162:0/1054041120 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcf0155100 con 0x7efcf00603d0 2020-11-28T21:46:38.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.903+0000 7efcf6faa700 1 -- 172.21.15.162:0/1054041120 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcf00b27b0 con 0x7efcf00603d0 2020-11-28T21:46:38.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.907+0000 7efcdeffd700 1 -- 172.21.15.162:0/1054041120 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7efce0013480 con 0x7efcf00603d0 2020-11-28T21:46:38.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.907+0000 7efcdeffd700 1 --2- 172.21.15.162:0/1054041120 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7efcd0055330 0x7efcd00577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:38.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.907+0000 7efcdeffd700 1 -- 172.21.15.162:0/1054041120 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7efce006e7d0 con 0x7efcf00603d0 2020-11-28T21:46:38.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.907+0000 7efcf57a7700 1 --2- 172.21.15.162:0/1054041120 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7efcd0055330 0x7efcd00577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:38.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.907+0000 7efcf57a7700 1 --2- 172.21.15.162:0/1054041120 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7efcd0055330 0x7efcd00577f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7efcec00b000 tx=0x7efcec008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:38.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:38.911+0000 7efcdeffd700 1 -- 172.21.15.162:0/1054041120 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7efce0044960 con 0x7efcf00603d0 2020-11-28T21:46:39.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.099+0000 7efcf6faa700 1 -- 172.21.15.162:0/1054041120 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7efcf0057050 con 0x7efcf00603d0 2020-11-28T21:46:39.104 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.103+0000 7efcdeffd700 1 -- 172.21.15.162:0/1054041120 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+45 (secure 0 0 0) 0x7efce0047f80 con 0x7efcf00603d0 2020-11-28T21:46:39.104 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:46:39.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.103+0000 7efcf6faa700 1 -- 172.21.15.162:0/1054041120 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7efcd0055330 msgr2=0x7efcd00577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:39.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.103+0000 7efcf6faa700 1 --2- 172.21.15.162:0/1054041120 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7efcd0055330 0x7efcd00577f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7efcec00b000 tx=0x7efcec008040).stop 2020-11-28T21:46:39.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.103+0000 7efcf6faa700 1 -- 172.21.15.162:0/1054041120 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcf00603d0 msgr2=0x7efcf0146d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:39.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.103+0000 7efcf6faa700 1 --2- 172.21.15.162:0/1054041120 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcf00603d0 0x7efcf0146d60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7efce0003070 tx=0x7efce0008040).stop 2020-11-28T21:46:39.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.103+0000 7efcf6faa700 1 -- 172.21.15.162:0/1054041120 shutdown_connections 2020-11-28T21:46:39.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.103+0000 7efcf6faa700 1 --2- 172.21.15.162:0/1054041120 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7efcd0055330 0x7efcd00577f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:39.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.103+0000 7efcf6faa700 1 --2- 172.21.15.162:0/1054041120 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efcf00afb30 0x7efcf0154c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:39.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.103+0000 7efcf6faa700 1 --2- 172.21.15.162:0/1054041120 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcf00603d0 0x7efcf0146d60 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:39.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.103+0000 7efcf6faa700 1 --2- 172.21.15.162:0/1054041120 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcf00b4c70 0x7efcf01520d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:39.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.103+0000 7efcf6faa700 1 -- 172.21.15.162:0/1054041120 >> 172.21.15.162:0/1054041120 conn(0x7efcf0066a40 msgr2=0x7efcf00b5e90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:39.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.103+0000 7efcf6faa700 1 -- 172.21.15.162:0/1054041120 shutdown_connections 2020-11-28T21:46:39.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.103+0000 7efcf6faa700 1 -- 172.21.15.162:0/1054041120 wait complete. 2020-11-28T21:46:39.124 INFO:teuthology.orchestra.run.smithi162.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2020-11-28T21:46:39.125 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2020-11-28T21:46:39.125 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:46:39.236 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.235+0000 7f3c54902700 1 Processor -- start 2020-11-28T21:46:39.237 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.235+0000 7f3c54902700 1 -- start start 2020-11-28T21:46:39.238 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.235+0000 7f3c54902700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3c500a9bc0 0x7f3c500a9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:39.238 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.235+0000 7f3c54902700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f3c500aa490 0x7f3c500afb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:39.239 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.235+0000 7f3c54902700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c500b0f30 con 0x7f3c500b0060 2020-11-28T21:46:39.239 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.235+0000 7f3c54902700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c50061030 con 0x7f3c500aa490 2020-11-28T21:46:39.239 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.235+0000 7f3c54902700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c500af980 con 0x7f3c500a9bc0 2020-11-28T21:46:39.239 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.235+0000 7f3c4f7fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3c500a9bc0 0x7f3c500a9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:39.239 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.235+0000 7f3c4f7fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3c500a9bc0 0x7f3c500a9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37668/0 (socket says 172.21.15.162:37668) 2020-11-28T21:46:39.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.235+0000 7f3c4f7fe700 1 -- 172.21.15.162:0/3059756440 learned_addr learned my addr 172.21.15.162:0/3059756440 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:39.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.235+0000 7f3c4effd700 1 --2- 172.21.15.162:0/3059756440 >> v2:172.21.15.118:3300/0 conn(0x7f3c500aa490 0x7f3c500afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:39.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.235+0000 7f3c4f7fe700 1 -- 172.21.15.162:0/3059756440 >> v2:172.21.15.118:3300/0 conn(0x7f3c500aa490 msgr2=0x7f3c500afb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:39.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.235+0000 7f3c4f7fe700 1 --2- 172.21.15.162:0/3059756440 >> v2:172.21.15.118:3300/0 conn(0x7f3c500aa490 0x7f3c500afb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:39.241 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.235+0000 7f3c4f7fe700 1 -- 172.21.15.162:0/3059756440 >> v1:172.21.15.118:6789/0 conn(0x7f3c500b0060 legacy=0x7f3c500b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:39.241 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.235+0000 7f3c4f7fe700 1 -- 172.21.15.162:0/3059756440 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c500af6e0 con 0x7f3c500a9bc0 2020-11-28T21:46:39.241 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.235+0000 7f3c4effd700 1 --2- 172.21.15.162:0/3059756440 >> v2:172.21.15.118:3300/0 conn(0x7f3c500aa490 0x7f3c500afb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:46:39.241 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.235+0000 7f3c4f7fe700 1 --2- 172.21.15.162:0/3059756440 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c500a9bc0 0x7f3c500a9fa0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f3c50067ea0 tx=0x7f3c40006e80).ready entity=mon.1 client_cookie=bd7aead72bb6f858 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:39.241 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.235+0000 7f3c4e7fc700 1 -- 172.21.15.162:0/3059756440 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c400097b0 con 0x7f3c500a9bc0 2020-11-28T21:46:39.242 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.235+0000 7f3c4e7fc700 1 -- 172.21.15.162:0/3059756440 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c40009910 con 0x7f3c500a9bc0 2020-11-28T21:46:39.242 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.235+0000 7f3c4e7fc700 1 -- 172.21.15.162:0/3059756440 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c40009a80 con 0x7f3c500a9bc0 2020-11-28T21:46:39.242 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.235+0000 7f3c54902700 1 -- 172.21.15.162:0/3059756440 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c500a9bc0 msgr2=0x7f3c500a9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:39.242 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.235+0000 7f3c54902700 1 --2- 172.21.15.162:0/3059756440 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c500a9bc0 0x7f3c500a9fa0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f3c50067ea0 tx=0x7f3c40006e80).stop 2020-11-28T21:46:39.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.235+0000 7f3c54902700 1 -- 172.21.15.162:0/3059756440 shutdown_connections 2020-11-28T21:46:39.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.235+0000 7f3c54902700 1 --2- 172.21.15.162:0/3059756440 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c500a9bc0 0x7f3c500a9fa0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:39.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.235+0000 7f3c54902700 1 --2- 172.21.15.162:0/3059756440 >> v2:172.21.15.118:3300/0 conn(0x7f3c500aa490 0x7f3c500afb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:39.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.235+0000 7f3c54902700 1 -- 172.21.15.162:0/3059756440 >> 172.21.15.162:0/3059756440 conn(0x7f3c5005b970 msgr2=0x7f3c5005bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:39.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.235+0000 7f3c54902700 1 -- 172.21.15.162:0/3059756440 shutdown_connections 2020-11-28T21:46:39.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.235+0000 7f3c54902700 1 -- 172.21.15.162:0/3059756440 wait complete. 2020-11-28T21:46:39.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.239+0000 7f3c54902700 1 Processor -- start 2020-11-28T21:46:39.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.239+0000 7f3c54902700 1 -- start start 2020-11-28T21:46:39.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.239+0000 7f3c54902700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3c500a9bc0 0x7f3c50142dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:39.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.239+0000 7f3c54902700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3c500aa490 0x7f3c50152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:39.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.239+0000 7f3c54902700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c500b0060 0x7f3c501531d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:39.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.239+0000 7f3c54902700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c500645a0 con 0x7f3c500aa490 2020-11-28T21:46:39.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.239+0000 7f3c54902700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c50064300 con 0x7f3c500b0060 2020-11-28T21:46:39.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.239+0000 7f3c54902700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c50064a40 con 0x7f3c500a9bc0 2020-11-28T21:46:39.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.239+0000 7f3c4ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c500b0060 0x7f3c501531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:39.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.239+0000 7f3c4effd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3c500aa490 0x7f3c50152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:39.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.239+0000 7f3c4ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c500b0060 0x7f3c501531d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37676/0 (socket says 172.21.15.162:37676) 2020-11-28T21:46:39.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.239+0000 7f3c4ffff700 1 -- 172.21.15.162:0/1506213140 learned_addr learned my addr 172.21.15.162:0/1506213140 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:39.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.239+0000 7f3c4f7fe700 1 --2- 172.21.15.162:0/1506213140 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3c500a9bc0 0x7f3c50142dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:39.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.239+0000 7f3c4ffff700 1 -- 172.21.15.162:0/1506213140 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3c500a9bc0 msgr2=0x7f3c50142dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:39.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.239+0000 7f3c4ffff700 1 --2- 172.21.15.162:0/1506213140 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3c500a9bc0 0x7f3c50142dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:39.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.239+0000 7f3c4ffff700 1 -- 172.21.15.162:0/1506213140 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3c500aa490 msgr2=0x7f3c50152ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:39.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.239+0000 7f3c4ffff700 1 --2- 172.21.15.162:0/1506213140 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3c500aa490 0x7f3c50152ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:39.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.239+0000 7f3c4ffff700 1 -- 172.21.15.162:0/1506213140 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c50064ce0 con 0x7f3c500b0060 2020-11-28T21:46:39.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.239+0000 7f3c4ffff700 1 --2- 172.21.15.162:0/1506213140 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c500b0060 0x7f3c501531d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f3c38002ef0 tx=0x7f3c38006cc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:39.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.239+0000 7f3c4cff9700 1 -- 172.21.15.162:0/1506213140 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c38002320 con 0x7f3c500b0060 2020-11-28T21:46:39.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.239+0000 7f3c54902700 1 -- 172.21.15.162:0/1506213140 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c50063340 con 0x7f3c500b0060 2020-11-28T21:46:39.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.239+0000 7f3c4cff9700 1 -- 172.21.15.162:0/1506213140 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c38000e90 con 0x7f3c500b0060 2020-11-28T21:46:39.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.239+0000 7f3c54902700 1 -- 172.21.15.162:0/1506213140 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c50065340 con 0x7f3c500b0060 2020-11-28T21:46:39.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.239+0000 7f3c4cff9700 1 -- 172.21.15.162:0/1506213140 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c38002670 con 0x7f3c500b0060 2020-11-28T21:46:39.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.239+0000 7f3c54902700 1 -- 172.21.15.162:0/1506213140 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c50064e70 con 0x7f3c500b0060 2020-11-28T21:46:39.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.243+0000 7f3c4cff9700 1 -- 172.21.15.162:0/1506213140 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f3c3800f030 con 0x7f3c500b0060 2020-11-28T21:46:39.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.243+0000 7f3c4cff9700 1 --2- 172.21.15.162:0/1506213140 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3c280553b0 0x7f3c28057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:39.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.243+0000 7f3c4cff9700 1 -- 172.21.15.162:0/1506213140 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3c3806e070 con 0x7f3c500b0060 2020-11-28T21:46:39.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.243+0000 7f3c4f7fe700 1 --2- 172.21.15.162:0/1506213140 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3c280553b0 0x7f3c28057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:39.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.243+0000 7f3c4f7fe700 1 --2- 172.21.15.162:0/1506213140 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3c280553b0 0x7f3c28057870 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f3c4001b000 tx=0x7f3c40016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:39.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.247+0000 7f3c4cff9700 1 -- 172.21.15.162:0/1506213140 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f3c38044370 con 0x7f3c500b0060 2020-11-28T21:46:39.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.383+0000 7f3c54902700 1 -- 172.21.15.162:0/1506213140 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3c50057050 con 0x7f3c500b0060 2020-11-28T21:46:39.387 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.383+0000 7f3c4cff9700 1 -- 172.21.15.162:0/1506213140 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16134 (secure 0 0 0) 0x7f3c38047990 con 0x7f3c500b0060 2020-11-28T21:46:39.388 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:46:39.388 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":23,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:46:30.124354+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:46:27.155497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-28T21:46:28.137474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:46:39.389 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.387+0000 7f3c54902700 1 -- 172.21.15.162:0/1506213140 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3c280553b0 msgr2=0x7f3c28057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:39.389 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.387+0000 7f3c54902700 1 --2- 172.21.15.162:0/1506213140 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3c280553b0 0x7f3c28057870 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f3c4001b000 tx=0x7f3c40016000).stop 2020-11-28T21:46:39.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.387+0000 7f3c54902700 1 -- 172.21.15.162:0/1506213140 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c500b0060 msgr2=0x7f3c501531d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:39.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.387+0000 7f3c54902700 1 --2- 172.21.15.162:0/1506213140 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c500b0060 0x7f3c501531d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f3c38002ef0 tx=0x7f3c38006cc0).stop 2020-11-28T21:46:39.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.387+0000 7f3c54902700 1 -- 172.21.15.162:0/1506213140 shutdown_connections 2020-11-28T21:46:39.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.387+0000 7f3c54902700 1 --2- 172.21.15.162:0/1506213140 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3c280553b0 0x7f3c28057870 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:39.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.387+0000 7f3c54902700 1 --2- 172.21.15.162:0/1506213140 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3c500a9bc0 0x7f3c50142dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:39.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.387+0000 7f3c54902700 1 --2- 172.21.15.162:0/1506213140 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c500b0060 0x7f3c501531d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:39.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.387+0000 7f3c54902700 1 --2- 172.21.15.162:0/1506213140 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3c500aa490 0x7f3c50152ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:39.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.387+0000 7f3c54902700 1 -- 172.21.15.162:0/1506213140 >> 172.21.15.162:0/1506213140 conn(0x7f3c5005b970 msgr2=0x7f3c501470f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:39.392 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.387+0000 7f3c54902700 1 -- 172.21.15.162:0/1506213140 shutdown_connections 2020-11-28T21:46:39.392 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.387+0000 7f3c54902700 1 -- 172.21.15.162:0/1506213140 wait complete. 2020-11-28T21:46:39.402 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-28T21:46:27.155497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-28T21:46:28.137474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-28T21:46:39.402 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:46:39.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.515+0000 7f67b0c46700 1 Processor -- start 2020-11-28T21:46:39.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.515+0000 7f67b0c46700 1 -- start start 2020-11-28T21:46:39.521 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.515+0000 7f67b0c46700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f67ac0b34a0 0x7f67ac0b1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:39.522 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.515+0000 7f67b0c46700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67ac0b2060 con 0x7f67ac0b7190 2020-11-28T21:46:39.522 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.515+0000 7f67b0c46700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67ac0b2560 con 0x7f67ac067300 2020-11-28T21:46:39.522 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.515+0000 7f67b0c46700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67ac141950 con 0x7f67ac0b34a0 2020-11-28T21:46:39.522 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.515+0000 7f67aad9d700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f67ac067300 0x7f67ac14b2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:44528/0 (socket says 172.21.15.162:44528) 2020-11-28T21:46:39.522 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.515+0000 7f67aad9d700 1 -- 172.21.15.162:0/3695660394 learned_addr learned my addr 172.21.15.162:0/3695660394 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:39.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.515+0000 7f67a9d9b700 1 --2- 172.21.15.162:0/3695660394 >> v2:172.21.15.118:3300/0 conn(0x7f67ac0b34a0 0x7f67ac0b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:39.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.515+0000 7f67a959a700 1 -- 172.21.15.162:0/3695660394 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1325502050 0 0) 0x7f67ac0b2560 con 0x7f67ac067300 2020-11-28T21:46:39.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.515+0000 7f67a959a700 1 -- 172.21.15.162:0/3695660394 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6790003230 con 0x7f67ac067300 2020-11-28T21:46:39.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.515+0000 7f67a959a700 1 -- 172.21.15.162:0/3695660394 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3645279151 0 0) 0x7f6790003230 con 0x7f67ac067300 2020-11-28T21:46:39.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.515+0000 7f67a959a700 1 -- 172.21.15.162:0/3695660394 >> v2:172.21.15.118:3300/0 conn(0x7f67ac0b34a0 msgr2=0x7f67ac0b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:39.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.515+0000 7f67a959a700 1 --2- 172.21.15.162:0/3695660394 >> v2:172.21.15.118:3300/0 conn(0x7f67ac0b34a0 0x7f67ac0b1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:39.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.515+0000 7f67a959a700 1 -- 172.21.15.162:0/3695660394 >> v1:172.21.15.118:6789/0 conn(0x7f67ac0b7190 legacy=0x7f67ac13dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:39.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.515+0000 7f67a959a700 1 -- 172.21.15.162:0/3695660394 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ac0b2800 con 0x7f67ac067300 2020-11-28T21:46:39.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.515+0000 7f67a959a700 1 -- 172.21.15.162:0/3695660394 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f67a40016d0 con 0x7f67ac067300 2020-11-28T21:46:39.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.515+0000 7f67a959a700 1 -- 172.21.15.162:0/3695660394 >> v1:172.21.15.162:6789/0 conn(0x7f67ac067300 legacy=0x7f67ac14b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:39.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.515+0000 7f67a959a700 1 --2- 172.21.15.162:0/3695660394 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6790003720 0x7f6790003b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:39.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.515+0000 7f67a959a700 1 --2- 172.21.15.162:0/3695660394 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f67900044d0 0x7f6790004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:39.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.515+0000 7f67a959a700 1 --2- 172.21.15.162:0/3695660394 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6790004e40 0x7f679000b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:39.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.515+0000 7f67a959a700 1 -- 172.21.15.162:0/3695660394 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ac141950 con 0x7f67900044d0 2020-11-28T21:46:39.526 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.515+0000 7f67a959a700 1 -- 172.21.15.162:0/3695660394 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6790004270 con 0x7f6790003720 2020-11-28T21:46:39.526 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.515+0000 7f67a959a700 1 -- 172.21.15.162:0/3695660394 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f679000bc10 con 0x7f6790004e40 2020-11-28T21:46:39.526 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.515+0000 7f67aa59c700 1 --2- 172.21.15.162:0/3695660394 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6790003720 0x7f6790003b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:39.526 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.515+0000 7f67a9d9b700 1 --2- 172.21.15.162:0/3695660394 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f67900044d0 0x7f6790004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:39.526 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.515+0000 7f67aad9d700 1 --2- 172.21.15.162:0/3695660394 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6790004e40 0x7f679000b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:39.527 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.515+0000 7f67aa59c700 1 -- 172.21.15.162:0/3695660394 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6790004e40 msgr2=0x7f679000b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:39.527 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.515+0000 7f67aa59c700 1 --2- 172.21.15.162:0/3695660394 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6790004e40 0x7f679000b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:39.527 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.515+0000 7f67aa59c700 1 -- 172.21.15.162:0/3695660394 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f67900044d0 msgr2=0x7f6790004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:39.527 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.515+0000 7f67aa59c700 1 --2- 172.21.15.162:0/3695660394 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f67900044d0 0x7f6790004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:39.527 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.515+0000 7f67aa59c700 1 -- 172.21.15.162:0/3695660394 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6790003230 con 0x7f6790003720 2020-11-28T21:46:39.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.515+0000 7f67aad9d700 1 -- 172.21.15.162:0/3695660394 reap_dead start 2020-11-28T21:46:39.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.519+0000 7f67aa59c700 1 --2- 172.21.15.162:0/3695660394 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6790003720 0x7f6790003b60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f67ac0b7a00 tx=0x7f67a0007b60).ready entity=mon.1 client_cookie=2ee7648ef758f1bf server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:39.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.519+0000 7f67a959a700 1 -- 172.21.15.162:0/3695660394 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f67a000c660 con 0x7f6790003720 2020-11-28T21:46:39.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.519+0000 7f67a959a700 1 -- 172.21.15.162:0/3695660394 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a000c7c0 con 0x7f6790003720 2020-11-28T21:46:39.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.519+0000 7f67a959a700 1 -- 172.21.15.162:0/3695660394 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f67a000cb20 con 0x7f6790003720 2020-11-28T21:46:39.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.519+0000 7f67b0c46700 1 -- 172.21.15.162:0/3695660394 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6790003720 msgr2=0x7f6790003b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:39.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.519+0000 7f67b0c46700 1 --2- 172.21.15.162:0/3695660394 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6790003720 0x7f6790003b60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f67ac0b7a00 tx=0x7f67a0007b60).stop 2020-11-28T21:46:39.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.519+0000 7f67b0c46700 1 -- 172.21.15.162:0/3695660394 shutdown_connections 2020-11-28T21:46:39.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.519+0000 7f67b0c46700 1 --2- 172.21.15.162:0/3695660394 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6790003720 0x7f6790003b60 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:39.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.519+0000 7f67b0c46700 1 -- 172.21.15.162:0/3695660394 >> 172.21.15.162:0/3695660394 conn(0x7f67ac064990 msgr2=0x7f67ac064d70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:39.530 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.519+0000 7f67b0c46700 1 -- 172.21.15.162:0/3695660394 shutdown_connections 2020-11-28T21:46:39.530 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.519+0000 7f67b0c46700 1 -- 172.21.15.162:0/3695660394 wait complete. 2020-11-28T21:46:39.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.519+0000 7f67b0c46700 1 Processor -- start 2020-11-28T21:46:39.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.519+0000 7f67b0c46700 1 -- start start 2020-11-28T21:46:39.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.519+0000 7f67b0c46700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6790003720 0x7f67ac14fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:39.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.519+0000 7f67b0c46700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67ac1466a0 0x7f67ac139350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:39.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.519+0000 7f67b0c46700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f67ac143970 0x7f67ac153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:39.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.519+0000 7f67b0c46700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ac146b40 con 0x7f6790003720 2020-11-28T21:46:39.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.519+0000 7f67b0c46700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ac146de0 con 0x7f67ac1466a0 2020-11-28T21:46:39.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.519+0000 7f67b0c46700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67ac1531e0 con 0x7f67ac143970 2020-11-28T21:46:39.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.519+0000 7f67a9d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67ac1466a0 0x7f67ac139350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:39.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.519+0000 7f67a9d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67ac1466a0 0x7f67ac139350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37692/0 (socket says 172.21.15.162:37692) 2020-11-28T21:46:39.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.519+0000 7f67a9d9b700 1 -- 172.21.15.162:0/3146015184 learned_addr learned my addr 172.21.15.162:0/3146015184 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:39.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.519+0000 7f67aa59c700 1 --2- 172.21.15.162:0/3146015184 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6790003720 0x7f67ac14fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:39.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.523+0000 7f67aad9d700 1 --2- 172.21.15.162:0/3146015184 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f67ac143970 0x7f67ac153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:39.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.523+0000 7f67a9d9b700 1 -- 172.21.15.162:0/3146015184 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f67ac143970 msgr2=0x7f67ac153920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:39.537 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.523+0000 7f67a9d9b700 1 --2- 172.21.15.162:0/3146015184 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f67ac143970 0x7f67ac153920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:39.537 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.523+0000 7f67a9d9b700 1 -- 172.21.15.162:0/3146015184 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6790003720 msgr2=0x7f67ac14fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:39.537 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.523+0000 7f67a9d9b700 1 --2- 172.21.15.162:0/3146015184 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6790003720 0x7f67ac14fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:39.537 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.523+0000 7f67a9d9b700 1 -- 172.21.15.162:0/3146015184 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ac153480 con 0x7f67ac1466a0 2020-11-28T21:46:39.537 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.523+0000 7f67a9d9b700 1 --2- 172.21.15.162:0/3146015184 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67ac1466a0 0x7f67ac139350 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f679c00b000 tx=0x7f679c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:39.538 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.523+0000 7f679b7fe700 1 -- 172.21.15.162:0/3146015184 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f679c00eb60 con 0x7f67ac1466a0 2020-11-28T21:46:39.538 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.523+0000 7f679b7fe700 1 -- 172.21.15.162:0/3146015184 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f679c00ecc0 con 0x7f67ac1466a0 2020-11-28T21:46:39.538 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.523+0000 7f67b0c46700 1 -- 172.21.15.162:0/3146015184 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67ac139840 con 0x7f67ac1466a0 2020-11-28T21:46:39.538 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.523+0000 7f679b7fe700 1 -- 172.21.15.162:0/3146015184 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f679c014620 con 0x7f67ac1466a0 2020-11-28T21:46:39.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.523+0000 7f67b0c46700 1 -- 172.21.15.162:0/3146015184 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67ac153e10 con 0x7f67ac1466a0 2020-11-28T21:46:39.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.523+0000 7f67b0c46700 1 -- 172.21.15.162:0/3146015184 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67ac057050 con 0x7f67ac1466a0 2020-11-28T21:46:39.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.523+0000 7f679b7fe700 1 -- 172.21.15.162:0/3146015184 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f679c024070 con 0x7f67ac1466a0 2020-11-28T21:46:39.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.523+0000 7f679b7fe700 1 --2- 172.21.15.162:0/3146015184 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f67880553b0 0x7f6788057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:39.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.523+0000 7f679b7fe700 1 -- 172.21.15.162:0/3146015184 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f679c06e8f0 con 0x7f67ac1466a0 2020-11-28T21:46:39.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.523+0000 7f67aa59c700 1 --2- 172.21.15.162:0/3146015184 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f67880553b0 0x7f6788057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:39.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.523+0000 7f67aa59c700 1 --2- 172.21.15.162:0/3146015184 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f67880553b0 0x7f6788057870 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f67a001b000 tx=0x7f67a0017000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:39.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.527+0000 7f679b7fe700 1 -- 172.21.15.162:0/3146015184 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f679c044b50 con 0x7f67ac1466a0 2020-11-28T21:46:39.668 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.667+0000 7f67b0c46700 1 -- 172.21.15.162:0/3146015184 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f67ac141b30 con 0x7f67ac1466a0 2020-11-28T21:46:39.669 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.667+0000 7f679b7fe700 1 -- 172.21.15.162:0/3146015184 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v23) v1 ==== 110+0+10 (secure 0 0 0) 0x7f679c048170 con 0x7f67ac1466a0 2020-11-28T21:46:39.670 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:46:39.672 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.667+0000 7f67b0c46700 1 -- 172.21.15.162:0/3146015184 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f67880553b0 msgr2=0x7f6788057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:39.672 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.667+0000 7f67b0c46700 1 --2- 172.21.15.162:0/3146015184 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f67880553b0 0x7f6788057870 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f67a001b000 tx=0x7f67a0017000).stop 2020-11-28T21:46:39.673 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.667+0000 7f67b0c46700 1 -- 172.21.15.162:0/3146015184 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67ac1466a0 msgr2=0x7f67ac139350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:39.673 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.667+0000 7f67b0c46700 1 --2- 172.21.15.162:0/3146015184 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67ac1466a0 0x7f67ac139350 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f679c00b000 tx=0x7f679c008040).stop 2020-11-28T21:46:39.673 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.667+0000 7f67b0c46700 1 -- 172.21.15.162:0/3146015184 shutdown_connections 2020-11-28T21:46:39.674 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.667+0000 7f67b0c46700 1 --2- 172.21.15.162:0/3146015184 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f67880553b0 0x7f6788057870 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:39.674 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.667+0000 7f67b0c46700 1 --2- 172.21.15.162:0/3146015184 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f67ac143970 0x7f67ac153920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:39.674 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.667+0000 7f67b0c46700 1 --2- 172.21.15.162:0/3146015184 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6790003720 0x7f67ac14fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:39.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.667+0000 7f67b0c46700 1 --2- 172.21.15.162:0/3146015184 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67ac1466a0 0x7f67ac139350 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:39.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.667+0000 7f67b0c46700 1 -- 172.21.15.162:0/3146015184 >> 172.21.15.162:0/3146015184 conn(0x7f67ac064990 msgr2=0x7f67ac0b63f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:39.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.667+0000 7f67b0c46700 1 -- 172.21.15.162:0/3146015184 shutdown_connections 2020-11-28T21:46:39.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.667+0000 7f67b0c46700 1 -- 172.21.15.162:0/3146015184 wait complete. 2020-11-28T21:46:39.683 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:46:39.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.799+0000 7f112759e700 1 Processor -- start 2020-11-28T21:46:39.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.799+0000 7f112759e700 1 -- start start 2020-11-28T21:46:39.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.799+0000 7f112759e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f11280b09f0 0x7f1128099d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:39.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.799+0000 7f112759e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11280afec0 con 0x7f11280991d0 2020-11-28T21:46:39.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.799+0000 7f112759e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11280b0160 con 0x7f1128060d10 2020-11-28T21:46:39.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.799+0000 7f112759e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11280608b0 con 0x7f11280b09f0 2020-11-28T21:46:39.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.799+0000 7f112659c700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f1128060d10 0x7f1128098ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:44544/0 (socket says 172.21.15.162:44544) 2020-11-28T21:46:39.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.799+0000 7f112659c700 1 -- 172.21.15.162:0/452968470 learned_addr learned my addr 172.21.15.162:0/452968470 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:39.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.799+0000 7f1126d9d700 1 --2- 172.21.15.162:0/452968470 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f11280b09f0 0x7f1128099d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:39.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.799+0000 7f1125d9b700 1 -- 172.21.15.162:0/452968470 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1675409409 0 0) 0x7f11280b0160 con 0x7f1128060d10 2020-11-28T21:46:39.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.799+0000 7f1125d9b700 1 -- 172.21.15.162:0/452968470 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1108003230 con 0x7f1128060d10 2020-11-28T21:46:39.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.799+0000 7f1125d9b700 1 -- 172.21.15.162:0/452968470 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3990385105 0 0) 0x7f11280afec0 con 0x7f11280991d0 2020-11-28T21:46:39.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.799+0000 7f1125d9b700 1 -- 172.21.15.162:0/452968470 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11280b0160 con 0x7f11280991d0 2020-11-28T21:46:39.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.799+0000 7f1126d9d700 1 -- 172.21.15.162:0/452968470 >> v1:172.21.15.162:6789/0 conn(0x7f1128060d10 legacy=0x7f1128098ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:39.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.799+0000 7f1125d9b700 1 -- 172.21.15.162:0/452968470 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2611438133 0 0) 0x7f1108003230 con 0x7f1128060d10 2020-11-28T21:46:39.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.799+0000 7f1126d9d700 1 -- 172.21.15.162:0/452968470 >> v1:172.21.15.118:6789/0 conn(0x7f11280991d0 legacy=0x7f1128099600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:39.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.799+0000 7f1126d9d700 1 -- 172.21.15.162:0/452968470 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11280b0400 con 0x7f11280b09f0 2020-11-28T21:46:39.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.799+0000 7f1126d9d700 1 --2- 172.21.15.162:0/452968470 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f11280b09f0 0x7f1128099d10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f1118001110 tx=0x7f11200091a0).ready entity=mon.2 client_cookie=1bc5621f08449819 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:39.810 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.799+0000 7f1125d9b700 1 -- 172.21.15.162:0/452968470 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1118001710 con 0x7f11280b09f0 2020-11-28T21:46:39.810 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.799+0000 7f1125d9b700 1 -- 172.21.15.162:0/452968470 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112000b880 con 0x7f11280b09f0 2020-11-28T21:46:39.810 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.799+0000 7f1125d9b700 1 -- 172.21.15.162:0/452968470 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f112000b9f0 con 0x7f11280b09f0 2020-11-28T21:46:39.811 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.799+0000 7f112759e700 1 -- 172.21.15.162:0/452968470 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f11280b09f0 msgr2=0x7f1128099d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:39.811 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.799+0000 7f112759e700 1 --2- 172.21.15.162:0/452968470 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f11280b09f0 0x7f1128099d10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f1118001110 tx=0x7f11200091a0).stop 2020-11-28T21:46:39.812 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.803+0000 7f112759e700 1 -- 172.21.15.162:0/452968470 shutdown_connections 2020-11-28T21:46:39.812 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.803+0000 7f112759e700 1 --2- 172.21.15.162:0/452968470 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f11280b09f0 0x7f1128099d10 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:39.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.803+0000 7f112759e700 1 -- 172.21.15.162:0/452968470 >> 172.21.15.162:0/452968470 conn(0x7f11280b5520 msgr2=0x7f11280b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:39.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.803+0000 7f112759e700 1 -- 172.21.15.162:0/452968470 shutdown_connections 2020-11-28T21:46:39.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.803+0000 7f112759e700 1 -- 172.21.15.162:0/452968470 wait complete. 2020-11-28T21:46:39.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.803+0000 7f112759e700 1 Processor -- start 2020-11-28T21:46:39.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.803+0000 7f112759e700 1 -- start start 2020-11-28T21:46:39.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.807+0000 7f112759e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1128060d10 0x7f11280aa150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:39.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.807+0000 7f112759e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f11280991d0 0x7f11280aa930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:39.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.807+0000 7f112759e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11280b09f0 0x7f1128151d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:39.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.807+0000 7f112759e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11281518c0 con 0x7f11280991d0 2020-11-28T21:46:39.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.807+0000 7f112759e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11280aae20 con 0x7f11280b09f0 2020-11-28T21:46:39.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.807+0000 7f112759e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11280aea20 con 0x7f1128060d10 2020-11-28T21:46:39.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.807+0000 7f1126d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11280b09f0 0x7f1128151d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:39.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.807+0000 7f1126d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11280b09f0 0x7f1128151d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37710/0 (socket says 172.21.15.162:37710) 2020-11-28T21:46:39.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.807+0000 7f1126d9d700 1 -- 172.21.15.162:0/300791017 learned_addr learned my addr 172.21.15.162:0/300791017 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:39.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.807+0000 7f112659c700 1 --2- 172.21.15.162:0/300791017 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1128060d10 0x7f11280aa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:39.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.807+0000 7f111ffff700 1 --2- 172.21.15.162:0/300791017 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f11280991d0 0x7f11280aa930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:39.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.807+0000 7f1126d9d700 1 -- 172.21.15.162:0/300791017 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1128060d10 msgr2=0x7f11280aa150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:39.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.807+0000 7f1126d9d700 1 --2- 172.21.15.162:0/300791017 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1128060d10 0x7f11280aa150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:39.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.807+0000 7f1126d9d700 1 -- 172.21.15.162:0/300791017 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f11280991d0 msgr2=0x7f11280aa930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:39.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.807+0000 7f1126d9d700 1 --2- 172.21.15.162:0/300791017 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f11280991d0 0x7f11280aa930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:39.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.807+0000 7f1126d9d700 1 -- 172.21.15.162:0/300791017 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11280aecc0 con 0x7f11280b09f0 2020-11-28T21:46:39.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.807+0000 7f111ffff700 1 --2- 172.21.15.162:0/300791017 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f11280991d0 0x7f11280aa930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:46:39.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.807+0000 7f1126d9d700 1 --2- 172.21.15.162:0/300791017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11280b09f0 0x7f1128151d70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f11280af8d0 tx=0x7f1120016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:39.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.807+0000 7f111f7fe700 1 -- 172.21.15.162:0/300791017 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1120011c80 con 0x7f11280b09f0 2020-11-28T21:46:39.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.807+0000 7f111f7fe700 1 -- 172.21.15.162:0/300791017 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1120011de0 con 0x7f11280b09f0 2020-11-28T21:46:39.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.807+0000 7f112759e700 1 -- 172.21.15.162:0/300791017 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11280aef60 con 0x7f11280b09f0 2020-11-28T21:46:39.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.807+0000 7f111f7fe700 1 -- 172.21.15.162:0/300791017 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11200205f0 con 0x7f11280b09f0 2020-11-28T21:46:39.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.807+0000 7f112759e700 1 -- 172.21.15.162:0/300791017 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f112808f910 con 0x7f11280b09f0 2020-11-28T21:46:39.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.807+0000 7f112759e700 1 -- 172.21.15.162:0/300791017 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1128057050 con 0x7f11280b09f0 2020-11-28T21:46:39.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.807+0000 7f111f7fe700 1 -- 172.21.15.162:0/300791017 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1120027030 con 0x7f11280b09f0 2020-11-28T21:46:39.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.811+0000 7f111f7fe700 1 --2- 172.21.15.162:0/300791017 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f110c0553b0 0x7f110c057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:39.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.811+0000 7f111f7fe700 1 -- 172.21.15.162:0/300791017 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f112006ea60 con 0x7f11280b09f0 2020-11-28T21:46:39.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.811+0000 7f112659c700 1 --2- 172.21.15.162:0/300791017 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f110c0553b0 0x7f110c057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:39.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.811+0000 7f112659c700 1 --2- 172.21.15.162:0/300791017 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f110c0553b0 0x7f110c057870 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f111800d000 tx=0x7f1118000b80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:39.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.815+0000 7f111f7fe700 1 -- 172.21.15.162:0/300791017 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1120044d60 con 0x7f11280b09f0 2020-11-28T21:46:39.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.955+0000 7f112759e700 1 -- 172.21.15.162:0/300791017 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f112805d040 con 0x7f11280b09f0 2020-11-28T21:46:39.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.959+0000 7f111f7fe700 1 -- 172.21.15.162:0/300791017 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f11200250a0 con 0x7f11280b09f0 2020-11-28T21:46:39.960 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:46:39.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.959+0000 7f112759e700 1 -- 172.21.15.162:0/300791017 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f110c0553b0 msgr2=0x7f110c057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:39.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.959+0000 7f112759e700 1 --2- 172.21.15.162:0/300791017 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f110c0553b0 0x7f110c057870 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f111800d000 tx=0x7f1118000b80).stop 2020-11-28T21:46:39.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.959+0000 7f112759e700 1 -- 172.21.15.162:0/300791017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11280b09f0 msgr2=0x7f1128151d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:39.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.959+0000 7f112759e700 1 --2- 172.21.15.162:0/300791017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11280b09f0 0x7f1128151d70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f11280af8d0 tx=0x7f1120016000).stop 2020-11-28T21:46:39.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.959+0000 7f112759e700 1 -- 172.21.15.162:0/300791017 shutdown_connections 2020-11-28T21:46:39.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.959+0000 7f112759e700 1 --2- 172.21.15.162:0/300791017 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f110c0553b0 0x7f110c057870 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:39.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.959+0000 7f112759e700 1 --2- 172.21.15.162:0/300791017 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1128060d10 0x7f11280aa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:39.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.959+0000 7f112759e700 1 --2- 172.21.15.162:0/300791017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11280b09f0 0x7f1128151d70 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:39.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.959+0000 7f112759e700 1 --2- 172.21.15.162:0/300791017 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f11280991d0 0x7f11280aa930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:39.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.959+0000 7f112759e700 1 -- 172.21.15.162:0/300791017 >> 172.21.15.162:0/300791017 conn(0x7f11280b5520 msgr2=0x7f11280995e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:39.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.959+0000 7f112759e700 1 -- 172.21.15.162:0/300791017 shutdown_connections 2020-11-28T21:46:39.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:39.959+0000 7f112759e700 1 -- 172.21.15.162:0/300791017 wait complete. 2020-11-28T21:46:39.973 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-28T21:46:40.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.091+0000 7f447069e700 1 Processor -- start 2020-11-28T21:46:40.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.091+0000 7f447069e700 1 -- start start 2020-11-28T21:46:40.094 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.091+0000 7f447069e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f44681433c0 0x7f44680605c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:40.095 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.091+0000 7f447069e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4468067610 0x7f446813fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:40.095 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.091+0000 7f447069e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4468143950 con 0x7f44680b9ce0 2020-11-28T21:46:40.095 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.091+0000 7f447069e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44680b4340 con 0x7f44681433c0 2020-11-28T21:46:40.095 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.091+0000 7f447069e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4468067ac0 con 0x7f4468067610 2020-11-28T21:46:40.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.091+0000 7f446ec3b700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4468067610 0x7f446813fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:40.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.091+0000 7f446dc39700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f44681433c0 0x7f44680605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:40.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.091+0000 7f446ec3b700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4468067610 0x7f446813fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:34942/0 (socket says 172.21.15.162:34942) 2020-11-28T21:46:40.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.091+0000 7f446ec3b700 1 -- 172.21.15.162:0/3325309096 learned_addr learned my addr 172.21.15.162:0/3325309096 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:40.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.091+0000 7f446ec3b700 1 -- 172.21.15.162:0/3325309096 >> v2:172.21.15.118:3300/0 conn(0x7f44681433c0 msgr2=0x7f44680605c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:40.098 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.091+0000 7f446ec3b700 1 --2- 172.21.15.162:0/3325309096 >> v2:172.21.15.118:3300/0 conn(0x7f44681433c0 0x7f44680605c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:40.098 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.091+0000 7f446ec3b700 1 -- 172.21.15.162:0/3325309096 >> v1:172.21.15.118:6789/0 conn(0x7f44680b9ce0 legacy=0x7f44680b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:40.098 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.091+0000 7f446ec3b700 1 -- 172.21.15.162:0/3325309096 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44680ba0c0 con 0x7f4468067610 2020-11-28T21:46:40.098 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.091+0000 7f446ec3b700 1 --2- 172.21.15.162:0/3325309096 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4468067610 0x7f446813fc80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f44600032b0 tx=0x7f4460009270).ready entity=mon.2 client_cookie=e44c3e2c01de3908 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:40.098 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.095+0000 7f446d438700 1 -- 172.21.15.162:0/3325309096 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4460009e60 con 0x7f4468067610 2020-11-28T21:46:40.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.095+0000 7f446d438700 1 -- 172.21.15.162:0/3325309096 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446000bc00 con 0x7f4468067610 2020-11-28T21:46:40.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.095+0000 7f446d438700 1 -- 172.21.15.162:0/3325309096 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f446000bd70 con 0x7f4468067610 2020-11-28T21:46:40.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.095+0000 7f447069e700 1 -- 172.21.15.162:0/3325309096 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4468067610 msgr2=0x7f446813fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:40.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.095+0000 7f447069e700 1 --2- 172.21.15.162:0/3325309096 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4468067610 0x7f446813fc80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f44600032b0 tx=0x7f4460009270).stop 2020-11-28T21:46:40.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.095+0000 7f447069e700 1 -- 172.21.15.162:0/3325309096 shutdown_connections 2020-11-28T21:46:40.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.095+0000 7f447069e700 1 --2- 172.21.15.162:0/3325309096 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4468067610 0x7f446813fc80 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:40.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.095+0000 7f447069e700 1 --2- 172.21.15.162:0/3325309096 >> v2:172.21.15.118:3300/0 conn(0x7f44681433c0 0x7f44680605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:40.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.095+0000 7f447069e700 1 -- 172.21.15.162:0/3325309096 >> 172.21.15.162:0/3325309096 conn(0x7f446805c430 msgr2=0x7f44680b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:40.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.095+0000 7f447069e700 1 -- 172.21.15.162:0/3325309096 shutdown_connections 2020-11-28T21:46:40.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.095+0000 7f447069e700 1 -- 172.21.15.162:0/3325309096 wait complete. 2020-11-28T21:46:40.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.095+0000 7f447069e700 1 Processor -- start 2020-11-28T21:46:40.104 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.095+0000 7f447069e700 1 -- start start 2020-11-28T21:46:40.104 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.099+0000 7f447069e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4468067610 0x7f44680b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:40.104 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.099+0000 7f447069e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f44680b9ce0 0x7f44680b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:40.104 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.099+0000 7f447069e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44681433c0 0x7f44680b26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:40.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.099+0000 7f447069e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4468151d10 con 0x7f4468067610 2020-11-28T21:46:40.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.099+0000 7f447069e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4468151fb0 con 0x7f44681433c0 2020-11-28T21:46:40.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.099+0000 7f447069e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4468152250 con 0x7f44680b9ce0 2020-11-28T21:46:40.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.099+0000 7f446ec3b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44681433c0 0x7f44680b26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:40.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.099+0000 7f446e43a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4468067610 0x7f44680b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:40.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.099+0000 7f446ec3b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44681433c0 0x7f44680b26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37724/0 (socket says 172.21.15.162:37724) 2020-11-28T21:46:40.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.099+0000 7f446dc39700 1 --2- 172.21.15.162:0/3847151507 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f44680b9ce0 0x7f44680b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:40.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.099+0000 7f446ec3b700 1 -- 172.21.15.162:0/3847151507 learned_addr learned my addr 172.21.15.162:0/3847151507 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:40.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.099+0000 7f446ec3b700 1 -- 172.21.15.162:0/3847151507 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f44680b9ce0 msgr2=0x7f44680b21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:40.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.099+0000 7f446ec3b700 1 --2- 172.21.15.162:0/3847151507 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f44680b9ce0 0x7f44680b21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:40.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.099+0000 7f446ec3b700 1 -- 172.21.15.162:0/3847151507 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4468067610 msgr2=0x7f44680b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:40.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.099+0000 7f446ec3b700 1 --2- 172.21.15.162:0/3847151507 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4468067610 0x7f44680b57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:40.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.099+0000 7f446ec3b700 1 -- 172.21.15.162:0/3847151507 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44681524f0 con 0x7f44681433c0 2020-11-28T21:46:40.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.099+0000 7f446e43a700 1 --2- 172.21.15.162:0/3847151507 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4468067610 0x7f44680b57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:46:40.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.099+0000 7f446ec3b700 1 --2- 172.21.15.162:0/3847151507 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44681433c0 0x7f44680b26e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f44600032b0 tx=0x7f4460016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:40.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.099+0000 7f445b7fe700 1 -- 172.21.15.162:0/3847151507 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4460009e60 con 0x7f44681433c0 2020-11-28T21:46:40.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.099+0000 7f445b7fe700 1 -- 172.21.15.162:0/3847151507 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44600105b0 con 0x7f44681433c0 2020-11-28T21:46:40.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.099+0000 7f447069e700 1 -- 172.21.15.162:0/3847151507 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4468152790 con 0x7f44681433c0 2020-11-28T21:46:40.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.099+0000 7f445b7fe700 1 -- 172.21.15.162:0/3847151507 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4460010910 con 0x7f44681433c0 2020-11-28T21:46:40.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.099+0000 7f447069e700 1 -- 172.21.15.162:0/3847151507 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44680b5c90 con 0x7f44681433c0 2020-11-28T21:46:40.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.099+0000 7f447069e700 1 -- 172.21.15.162:0/3847151507 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4468152920 con 0x7f44681433c0 2020-11-28T21:46:40.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.099+0000 7f445b7fe700 1 -- 172.21.15.162:0/3847151507 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f446000f070 con 0x7f44681433c0 2020-11-28T21:46:40.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.099+0000 7f445b7fe700 1 --2- 172.21.15.162:0/3847151507 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f44480554e0 0x7f44480579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:40.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.099+0000 7f445b7fe700 1 -- 172.21.15.162:0/3847151507 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f446006f560 con 0x7f44681433c0 2020-11-28T21:46:40.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.099+0000 7f446e43a700 1 --2- 172.21.15.162:0/3847151507 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f44480554e0 0x7f44480579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:40.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.103+0000 7f446e43a700 1 --2- 172.21.15.162:0/3847151507 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f44480554e0 0x7f44480579a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f445c00d000 tx=0x7f445c009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:40.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.107+0000 7f445b7fe700 1 -- 172.21.15.162:0/3847151507 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4460045860 con 0x7f44681433c0 2020-11-28T21:46:40.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.247+0000 7f447069e700 1 -- 172.21.15.162:0/3847151507 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4468057050 con 0x7f44681433c0 2020-11-28T21:46:40.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.251+0000 7f445b7fe700 1 -- 172.21.15.162:0/3847151507 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+10 (secure 0 0 0) 0x7f4460015020 con 0x7f44681433c0 2020-11-28T21:46:40.252 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:46:40.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.251+0000 7f447069e700 1 -- 172.21.15.162:0/3847151507 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f44480554e0 msgr2=0x7f44480579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:40.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.251+0000 7f447069e700 1 --2- 172.21.15.162:0/3847151507 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f44480554e0 0x7f44480579a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f445c00d000 tx=0x7f445c009040).stop 2020-11-28T21:46:40.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.251+0000 7f447069e700 1 -- 172.21.15.162:0/3847151507 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44681433c0 msgr2=0x7f44680b26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:40.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.251+0000 7f447069e700 1 --2- 172.21.15.162:0/3847151507 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44681433c0 0x7f44680b26e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f44600032b0 tx=0x7f4460016000).stop 2020-11-28T21:46:40.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.251+0000 7f447069e700 1 -- 172.21.15.162:0/3847151507 shutdown_connections 2020-11-28T21:46:40.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.251+0000 7f447069e700 1 --2- 172.21.15.162:0/3847151507 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f44480554e0 0x7f44480579a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:40.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.251+0000 7f447069e700 1 --2- 172.21.15.162:0/3847151507 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44681433c0 0x7f44680b26e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:40.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.251+0000 7f447069e700 1 --2- 172.21.15.162:0/3847151507 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4468067610 0x7f44680b57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:40.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.251+0000 7f447069e700 1 --2- 172.21.15.162:0/3847151507 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f44680b9ce0 0x7f44680b21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:40.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.251+0000 7f447069e700 1 -- 172.21.15.162:0/3847151507 >> 172.21.15.162:0/3847151507 conn(0x7f446805c430 msgr2=0x7f446809d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:40.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.251+0000 7f447069e700 1 -- 172.21.15.162:0/3847151507 shutdown_connections 2020-11-28T21:46:40.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.251+0000 7f447069e700 1 -- 172.21.15.162:0/3847151507 wait complete. 2020-11-28T21:46:40.266 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-28T21:46:40.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.383+0000 7f20dae6e700 1 Processor -- start 2020-11-28T21:46:40.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.383+0000 7f20dae6e700 1 -- start start 2020-11-28T21:46:40.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.383+0000 7f20dae6e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f20d40b1120 0x7f20d4147660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:40.398 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.383+0000 7f20dae6e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20d40a8690 con 0x7f20d414a440 2020-11-28T21:46:40.398 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.383+0000 7f20dae6e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20d40a88c0 con 0x7f20d40b5730 2020-11-28T21:46:40.399 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.383+0000 7f20dae6e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d41471c0 con 0x7f20d40b1120 2020-11-28T21:46:40.399 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.383+0000 7f20d8c0a700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f20d40b5730 0x7f20d414ac50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:44572/0 (socket says 172.21.15.162:44572) 2020-11-28T21:46:40.400 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.383+0000 7f20d8c0a700 1 -- 172.21.15.162:0/2029859489 learned_addr learned my addr 172.21.15.162:0/2029859489 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:40.400 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.383+0000 7f20d3fff700 1 --2- 172.21.15.162:0/2029859489 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f20d40b1120 0x7f20d4147660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:40.401 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.383+0000 7f20d37fe700 1 -- 172.21.15.162:0/2029859489 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 594842598 0 0) 0x7f20d40a88c0 con 0x7f20d40b5730 2020-11-28T21:46:40.401 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.383+0000 7f20d37fe700 1 -- 172.21.15.162:0/2029859489 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20c0003230 con 0x7f20d40b5730 2020-11-28T21:46:40.402 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.383+0000 7f20d37fe700 1 -- 172.21.15.162:0/2029859489 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3999366825 0 0) 0x7f20c0003230 con 0x7f20d40b5730 2020-11-28T21:46:40.402 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.383+0000 7f20d37fe700 1 -- 172.21.15.162:0/2029859489 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f20d40b1120 msgr2=0x7f20d4147660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:40.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.383+0000 7f20d37fe700 1 --2- 172.21.15.162:0/2029859489 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f20d40b1120 0x7f20d4147660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:40.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.383+0000 7f20d37fe700 1 -- 172.21.15.162:0/2029859489 >> v1:172.21.15.118:6789/0 conn(0x7f20d414a440 legacy=0x7f20d41517c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:40.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.383+0000 7f20d37fe700 1 -- 172.21.15.162:0/2029859489 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d414b360 con 0x7f20d40b5730 2020-11-28T21:46:40.404 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.383+0000 7f20d37fe700 1 -- 172.21.15.162:0/2029859489 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f20c40016d0 con 0x7f20d40b5730 2020-11-28T21:46:40.404 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.383+0000 7f20d37fe700 1 -- 172.21.15.162:0/2029859489 >> v1:172.21.15.162:6789/0 conn(0x7f20d40b5730 legacy=0x7f20d414ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:40.405 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.383+0000 7f20d37fe700 1 --2- 172.21.15.162:0/2029859489 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f20d40b1120 0x7f20c0003720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:40.405 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.383+0000 7f20d37fe700 1 --2- 172.21.15.162:0/2029859489 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20c0003c10 0x7f20c00044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:40.405 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.383+0000 7f20d37fe700 1 --2- 172.21.15.162:0/2029859489 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f20c00049c0 0x7f20c0004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:40.406 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.383+0000 7f20d37fe700 1 -- 172.21.15.162:0/2029859489 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d41471c0 con 0x7f20c00049c0 2020-11-28T21:46:40.406 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.383+0000 7f20d37fe700 1 -- 172.21.15.162:0/2029859489 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c0004130 con 0x7f20c0003c10 2020-11-28T21:46:40.407 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.383+0000 7f20d37fe700 1 -- 172.21.15.162:0/2029859489 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c00042f0 con 0x7f20d40b1120 2020-11-28T21:46:40.407 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.383+0000 7f20d3fff700 1 --2- 172.21.15.162:0/2029859489 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20c0003c10 0x7f20c00044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:40.407 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.383+0000 7f20d8c0a700 1 --2- 172.21.15.162:0/2029859489 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f20d40b1120 0x7f20c0003720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:40.408 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.383+0000 7f20d940b700 1 --2- 172.21.15.162:0/2029859489 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f20c00049c0 0x7f20c0004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:40.408 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.383+0000 7f20d3fff700 1 -- 172.21.15.162:0/2029859489 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f20d40b1120 msgr2=0x7f20c0003720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:40.408 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.383+0000 7f20d3fff700 1 --2- 172.21.15.162:0/2029859489 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f20d40b1120 0x7f20c0003720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:40.409 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.383+0000 7f20d3fff700 1 -- 172.21.15.162:0/2029859489 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f20c00049c0 msgr2=0x7f20c0004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:40.409 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.383+0000 7f20d3fff700 1 --2- 172.21.15.162:0/2029859489 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f20c00049c0 0x7f20c0004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:40.410 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.383+0000 7f20d3fff700 1 -- 172.21.15.162:0/2029859489 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c0003230 con 0x7f20c0003c10 2020-11-28T21:46:40.410 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.383+0000 7f20d940b700 1 --2- 172.21.15.162:0/2029859489 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f20c00049c0 0x7f20c0004e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:46:40.410 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.383+0000 7f20d3fff700 1 --2- 172.21.15.162:0/2029859489 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20c0003c10 0x7f20c00044d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f20c800c000 tx=0x7f20c8008040).ready entity=mon.1 client_cookie=79b9aaf13226c36a server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:40.411 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.387+0000 7f20d37fe700 1 -- 172.21.15.162:0/2029859489 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f20c800e970 con 0x7f20c0003c10 2020-11-28T21:46:40.411 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.387+0000 7f20d37fe700 1 -- 172.21.15.162:0/2029859489 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c800ead0 con 0x7f20c0003c10 2020-11-28T21:46:40.412 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.387+0000 7f20d37fe700 1 -- 172.21.15.162:0/2029859489 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f20c800ee30 con 0x7f20c0003c10 2020-11-28T21:46:40.412 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.387+0000 7f20dae6e700 1 -- 172.21.15.162:0/2029859489 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20c0003c10 msgr2=0x7f20c00044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:40.413 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.387+0000 7f20dae6e700 1 --2- 172.21.15.162:0/2029859489 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20c0003c10 0x7f20c00044d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f20c800c000 tx=0x7f20c8008040).stop 2020-11-28T21:46:40.413 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.387+0000 7f20d940b700 1 -- 172.21.15.162:0/2029859489 reap_dead start 2020-11-28T21:46:40.413 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.387+0000 7f20dae6e700 1 -- 172.21.15.162:0/2029859489 shutdown_connections 2020-11-28T21:46:40.414 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.387+0000 7f20dae6e700 1 -- 172.21.15.162:0/2029859489 >> 172.21.15.162:0/2029859489 conn(0x7f20d405c3f0 msgr2=0x7f20d40b4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:40.415 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.387+0000 7f20dae6e700 1 -- 172.21.15.162:0/2029859489 shutdown_connections 2020-11-28T21:46:40.415 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.387+0000 7f20dae6e700 1 -- 172.21.15.162:0/2029859489 wait complete. 2020-11-28T21:46:40.415 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.387+0000 7f20dae6e700 1 Processor -- start 2020-11-28T21:46:40.416 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.387+0000 7f20dae6e700 1 -- start start 2020-11-28T21:46:40.416 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.391+0000 7f20dae6e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f20d40b32b0 0x7f20d40ac1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:40.416 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.391+0000 7f20dae6e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20d40aacf0 0x7f20d4159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:40.417 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.391+0000 7f20dae6e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f20d4159e70 0x7f20d40a8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:40.417 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.391+0000 7f20dae6e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d41592b0 con 0x7f20d4159e70 2020-11-28T21:46:40.418 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.391+0000 7f20dae6e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d40b3690 con 0x7f20d40aacf0 2020-11-28T21:46:40.418 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.391+0000 7f20dae6e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d414e010 con 0x7f20d40b32b0 2020-11-28T21:46:40.419 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.391+0000 7f20d3fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20d40aacf0 0x7f20d4159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:40.419 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.391+0000 7f20d3fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20d40aacf0 0x7f20d4159980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37742/0 (socket says 172.21.15.162:37742) 2020-11-28T21:46:40.419 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.391+0000 7f20d3fff700 1 -- 172.21.15.162:0/1852012825 learned_addr learned my addr 172.21.15.162:0/1852012825 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:40.420 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.391+0000 7f20d8c0a700 1 --2- 172.21.15.162:0/1852012825 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f20d40b32b0 0x7f20d40ac1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:40.420 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.391+0000 7f20d940b700 1 --2- 172.21.15.162:0/1852012825 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f20d4159e70 0x7f20d40a8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:40.421 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.391+0000 7f20d3fff700 1 -- 172.21.15.162:0/1852012825 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f20d40b32b0 msgr2=0x7f20d40ac1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:40.421 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.391+0000 7f20d3fff700 1 --2- 172.21.15.162:0/1852012825 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f20d40b32b0 0x7f20d40ac1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:40.421 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.391+0000 7f20d3fff700 1 -- 172.21.15.162:0/1852012825 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f20d4159e70 msgr2=0x7f20d40a8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:40.422 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.391+0000 7f20d3fff700 1 --2- 172.21.15.162:0/1852012825 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f20d4159e70 0x7f20d40a8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:40.422 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.391+0000 7f20d3fff700 1 -- 172.21.15.162:0/1852012825 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d414e2b0 con 0x7f20d40aacf0 2020-11-28T21:46:40.422 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.391+0000 7f20d3fff700 1 --2- 172.21.15.162:0/1852012825 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20d40aacf0 0x7f20d4159980 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f20d4159860 tx=0x7f20c801a000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:40.423 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.391+0000 7f20d1ffb700 1 -- 172.21.15.162:0/1852012825 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f20c8012e60 con 0x7f20d40aacf0 2020-11-28T21:46:40.423 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.391+0000 7f20d1ffb700 1 -- 172.21.15.162:0/1852012825 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c801c430 con 0x7f20d40aacf0 2020-11-28T21:46:40.423 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.391+0000 7f20dae6e700 1 -- 172.21.15.162:0/1852012825 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20d41594e0 con 0x7f20d40aacf0 2020-11-28T21:46:40.424 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.391+0000 7f20d1ffb700 1 -- 172.21.15.162:0/1852012825 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f20c801c790 con 0x7f20d40aacf0 2020-11-28T21:46:40.424 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.391+0000 7f20dae6e700 1 -- 172.21.15.162:0/1852012825 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20d40a9310 con 0x7f20d40aacf0 2020-11-28T21:46:40.424 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.391+0000 7f20d1ffb700 1 -- 172.21.15.162:0/1852012825 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f20c801c8f0 con 0x7f20d40aacf0 2020-11-28T21:46:40.424 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.391+0000 7f20d1ffb700 1 --2- 172.21.15.162:0/1852012825 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f20b40553b0 0x7f20b4057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:40.425 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.391+0000 7f20dae6e700 1 -- 172.21.15.162:0/1852012825 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20d4159670 con 0x7f20d40aacf0 2020-11-28T21:46:40.425 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.395+0000 7f20d8c0a700 1 --2- 172.21.15.162:0/1852012825 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f20b40553b0 0x7f20b4057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:40.425 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.395+0000 7f20d1ffb700 1 -- 172.21.15.162:0/1852012825 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f20c806fb50 con 0x7f20d40aacf0 2020-11-28T21:46:40.425 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.395+0000 7f20d8c0a700 1 --2- 172.21.15.162:0/1852012825 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f20b40553b0 0x7f20b4057870 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f20c4005000 tx=0x7f20d4098f90).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:40.426 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.399+0000 7f20d1ffb700 1 -- 172.21.15.162:0/1852012825 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f20c8045d70 con 0x7f20d40aacf0 2020-11-28T21:46:40.545 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.543+0000 7f20dae6e700 1 -- 172.21.15.162:0/1852012825 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f20d4057050 con 0x7f20d40aacf0 2020-11-28T21:46:40.546 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.543+0000 7f20d1ffb700 1 -- 172.21.15.162:0/1852012825 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+10 (secure 0 0 0) 0x7f20c8015170 con 0x7f20d40aacf0 2020-11-28T21:46:40.547 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:46:40.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.547+0000 7f20dae6e700 1 -- 172.21.15.162:0/1852012825 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f20b40553b0 msgr2=0x7f20b4057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:40.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.547+0000 7f20dae6e700 1 --2- 172.21.15.162:0/1852012825 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f20b40553b0 0x7f20b4057870 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f20c4005000 tx=0x7f20d4098f90).stop 2020-11-28T21:46:40.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.547+0000 7f20dae6e700 1 -- 172.21.15.162:0/1852012825 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20d40aacf0 msgr2=0x7f20d4159980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:40.550 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.547+0000 7f20dae6e700 1 --2- 172.21.15.162:0/1852012825 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20d40aacf0 0x7f20d4159980 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f20d4159860 tx=0x7f20c801a000).stop 2020-11-28T21:46:40.550 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.547+0000 7f20dae6e700 1 -- 172.21.15.162:0/1852012825 shutdown_connections 2020-11-28T21:46:40.550 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.547+0000 7f20dae6e700 1 --2- 172.21.15.162:0/1852012825 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f20b40553b0 0x7f20b4057870 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:40.550 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.547+0000 7f20dae6e700 1 --2- 172.21.15.162:0/1852012825 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f20d40b32b0 0x7f20d40ac1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:40.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.547+0000 7f20dae6e700 1 --2- 172.21.15.162:0/1852012825 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f20d4159e70 0x7f20d40a8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:40.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.547+0000 7f20dae6e700 1 --2- 172.21.15.162:0/1852012825 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20d40aacf0 0x7f20d4159980 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:40.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.547+0000 7f20dae6e700 1 -- 172.21.15.162:0/1852012825 >> 172.21.15.162:0/1852012825 conn(0x7f20d405c3f0 msgr2=0x7f20d4099000 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:40.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.547+0000 7f20dae6e700 1 -- 172.21.15.162:0/1852012825 shutdown_connections 2020-11-28T21:46:40.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.547+0000 7f20dae6e700 1 -- 172.21.15.162:0/1852012825 wait complete. 2020-11-28T21:46:40.560 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:46:40.669 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.667+0000 7f48bb375700 1 Processor -- start 2020-11-28T21:46:40.669 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.667+0000 7f48bb375700 1 -- start start 2020-11-28T21:46:40.674 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.667+0000 7f48bb375700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f48b40b6130 0x7f48b4143020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:40.674 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.667+0000 7f48bb375700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f48b40b54f0 0x7f48b4143510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:40.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.667+0000 7f48bb375700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48b40ba0b0 con 0x7f48b40608f0 2020-11-28T21:46:40.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.667+0000 7f48bb375700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48b4143a00 con 0x7f48b40b6130 2020-11-28T21:46:40.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.667+0000 7f48bb375700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48b40b9e10 con 0x7f48b40b54f0 2020-11-28T21:46:40.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.667+0000 7f48b9912700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f48b40b54f0 0x7f48b4143510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:40.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.667+0000 7f48b9912700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f48b40b54f0 0x7f48b4143510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37752/0 (socket says 172.21.15.162:37752) 2020-11-28T21:46:40.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.667+0000 7f48b9912700 1 -- 172.21.15.162:0/2036215561 learned_addr learned my addr 172.21.15.162:0/2036215561 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:40.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.667+0000 7f48b8910700 1 --2- 172.21.15.162:0/2036215561 >> v2:172.21.15.118:3300/0 conn(0x7f48b40b6130 0x7f48b4143020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:40.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.667+0000 7f48b9912700 1 -- 172.21.15.162:0/2036215561 >> v2:172.21.15.118:3300/0 conn(0x7f48b40b6130 msgr2=0x7f48b4143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:40.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.667+0000 7f48b9912700 1 --2- 172.21.15.162:0/2036215561 >> v2:172.21.15.118:3300/0 conn(0x7f48b40b6130 0x7f48b4143020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:40.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.667+0000 7f48b9912700 1 -- 172.21.15.162:0/2036215561 >> v1:172.21.15.118:6789/0 conn(0x7f48b40608f0 legacy=0x7f48b413c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:40.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.667+0000 7f48b9912700 1 -- 172.21.15.162:0/2036215561 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b4142b80 con 0x7f48b40b54f0 2020-11-28T21:46:40.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.667+0000 7f48b8910700 1 --2- 172.21.15.162:0/2036215561 >> v2:172.21.15.118:3300/0 conn(0x7f48b40b6130 0x7f48b4143020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:46:40.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.667+0000 7f48b9912700 1 --2- 172.21.15.162:0/2036215561 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48b40b54f0 0x7f48b4143510 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f48b40b6560 tx=0x7f48ac009270).ready entity=mon.1 client_cookie=5e943e9a32e48fc6 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:40.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.667+0000 7f48abfff700 1 -- 172.21.15.162:0/2036215561 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48ac00bbc0 con 0x7f48b40b54f0 2020-11-28T21:46:40.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.667+0000 7f48abfff700 1 -- 172.21.15.162:0/2036215561 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48ac00bd20 con 0x7f48b40b54f0 2020-11-28T21:46:40.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.667+0000 7f48bb375700 1 -- 172.21.15.162:0/2036215561 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48b40b54f0 msgr2=0x7f48b4143510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:40.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.667+0000 7f48abfff700 1 -- 172.21.15.162:0/2036215561 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48ac011400 con 0x7f48b40b54f0 2020-11-28T21:46:40.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.667+0000 7f48bb375700 1 --2- 172.21.15.162:0/2036215561 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48b40b54f0 0x7f48b4143510 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f48b40b6560 tx=0x7f48ac009270).stop 2020-11-28T21:46:40.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.667+0000 7f48bb375700 1 -- 172.21.15.162:0/2036215561 shutdown_connections 2020-11-28T21:46:40.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.667+0000 7f48bb375700 1 --2- 172.21.15.162:0/2036215561 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48b40b54f0 0x7f48b4143510 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:40.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.667+0000 7f48bb375700 1 --2- 172.21.15.162:0/2036215561 >> v2:172.21.15.118:3300/0 conn(0x7f48b40b6130 0x7f48b4143020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:40.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.667+0000 7f48bb375700 1 -- 172.21.15.162:0/2036215561 >> 172.21.15.162:0/2036215561 conn(0x7f48b4066c70 msgr2=0x7f48b4067050 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:40.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.667+0000 7f48bb375700 1 -- 172.21.15.162:0/2036215561 shutdown_connections 2020-11-28T21:46:40.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.667+0000 7f48bb375700 1 -- 172.21.15.162:0/2036215561 wait complete. 2020-11-28T21:46:40.680 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.671+0000 7f48bb375700 1 Processor -- start 2020-11-28T21:46:40.680 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.671+0000 7f48bb375700 1 -- start start 2020-11-28T21:46:40.682 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.671+0000 7f48bb375700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48b40608f0 0x7f48b413d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:40.682 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.671+0000 7f48bb375700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f48b40b54f0 0x7f48b413eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:40.683 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.671+0000 7f48bb375700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48b40b6130 0x7f48b413eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:40.683 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.671+0000 7f48bb375700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b4151b70 con 0x7f48b40608f0 2020-11-28T21:46:40.683 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.671+0000 7f48bb375700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b40b9170 con 0x7f48b40b6130 2020-11-28T21:46:40.683 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.671+0000 7f48bb375700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b413f8a0 con 0x7f48b40b54f0 2020-11-28T21:46:40.684 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.671+0000 7f48b9912700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48b40b6130 0x7f48b413eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:40.684 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.671+0000 7f48b9912700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48b40b6130 0x7f48b413eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37756/0 (socket says 172.21.15.162:37756) 2020-11-28T21:46:40.684 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.671+0000 7f48b9912700 1 -- 172.21.15.162:0/1634512881 learned_addr learned my addr 172.21.15.162:0/1634512881 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:40.684 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.671+0000 7f48b9111700 1 --2- 172.21.15.162:0/1634512881 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48b40608f0 0x7f48b413d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:40.685 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.671+0000 7f48b8910700 1 --2- 172.21.15.162:0/1634512881 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f48b40b54f0 0x7f48b413eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:40.685 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.671+0000 7f48b9912700 1 -- 172.21.15.162:0/1634512881 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f48b40b54f0 msgr2=0x7f48b413eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:40.685 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.671+0000 7f48b9912700 1 --2- 172.21.15.162:0/1634512881 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f48b40b54f0 0x7f48b413eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:40.685 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.671+0000 7f48b9912700 1 -- 172.21.15.162:0/1634512881 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48b40608f0 msgr2=0x7f48b413d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:40.686 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.671+0000 7f48b9912700 1 --2- 172.21.15.162:0/1634512881 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48b40608f0 0x7f48b413d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:40.687 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.671+0000 7f48b9912700 1 -- 172.21.15.162:0/1634512881 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b413fb40 con 0x7f48b40b6130 2020-11-28T21:46:40.688 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.671+0000 7f48b9111700 1 --2- 172.21.15.162:0/1634512881 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48b40608f0 0x7f48b413d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:46:40.688 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.671+0000 7f48b9912700 1 --2- 172.21.15.162:0/1634512881 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48b40b6130 0x7f48b413eff0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f48b40b6560 tx=0x7f48ac016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:40.688 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.671+0000 7f48aa7fc700 1 -- 172.21.15.162:0/1634512881 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48ac01e610 con 0x7f48b40b6130 2020-11-28T21:46:40.688 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.671+0000 7f48aa7fc700 1 -- 172.21.15.162:0/1634512881 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48ac01e770 con 0x7f48b40b6130 2020-11-28T21:46:40.689 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.671+0000 7f48bb375700 1 -- 172.21.15.162:0/1634512881 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48b413fde0 con 0x7f48b40b6130 2020-11-28T21:46:40.689 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.671+0000 7f48aa7fc700 1 -- 172.21.15.162:0/1634512881 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48ac01ead0 con 0x7f48b40b6130 2020-11-28T21:46:40.690 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.671+0000 7f48bb375700 1 -- 172.21.15.162:0/1634512881 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b413f4e0 con 0x7f48b40b6130 2020-11-28T21:46:40.690 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.675+0000 7f48bb375700 1 -- 172.21.15.162:0/1634512881 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48b41401a0 con 0x7f48b40b6130 2020-11-28T21:46:40.690 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.675+0000 7f48aa7fc700 1 -- 172.21.15.162:0/1634512881 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f48ac00f070 con 0x7f48b40b6130 2020-11-28T21:46:40.691 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.675+0000 7f48aa7fc700 1 --2- 172.21.15.162:0/1634512881 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f489c055330 0x7f489c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:40.691 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.675+0000 7f48b9111700 1 --2- 172.21.15.162:0/1634512881 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f489c055330 0x7f489c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:40.691 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.675+0000 7f48aa7fc700 1 -- 172.21.15.162:0/1634512881 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f48ac06f970 con 0x7f48b40b6130 2020-11-28T21:46:40.691 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.675+0000 7f48b9111700 1 --2- 172.21.15.162:0/1634512881 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f489c055330 0x7f489c0577f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f48b000b000 tx=0x7f48b0009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:40.691 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.679+0000 7f48aa7fc700 1 -- 172.21.15.162:0/1634512881 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f48ac045b60 con 0x7f48b40b6130 2020-11-28T21:46:40.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.863+0000 7f48bb375700 1 -- 172.21.15.162:0/1634512881 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48b41401a0 con 0x7f48b40b6130 2020-11-28T21:46:40.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.863+0000 7f48aa7fc700 1 -- 172.21.15.162:0/1634512881 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2495 (secure 0 0 0) 0x7f48ac0159d0 con 0x7f48b40b6130 2020-11-28T21:46:40.868 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:46:40.868 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/1297721315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1297721315},{"type":"v1","addr":"172.21.15.118:6833","nonce":1297721315}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3},{"gid":4264,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2415281381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2415281381},{"type":"v1","addr":"172.21.15.162:6837","nonce":2415281381}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:46:29.120502+0000","modified":"2020-11-28T21:46:30.123428+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.162:6835/3696705934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3696705934},{"type":"v1","addr":"172.21.15.162:6835","nonce":3696705934}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-28T21:46:40.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.867+0000 7f48bb375700 1 -- 172.21.15.162:0/1634512881 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f489c055330 msgr2=0x7f489c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:40.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.867+0000 7f48bb375700 1 --2- 172.21.15.162:0/1634512881 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f489c055330 0x7f489c0577f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f48b000b000 tx=0x7f48b0009040).stop 2020-11-28T21:46:40.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.867+0000 7f48bb375700 1 -- 172.21.15.162:0/1634512881 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48b40b6130 msgr2=0x7f48b413eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:40.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.867+0000 7f48bb375700 1 --2- 172.21.15.162:0/1634512881 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48b40b6130 0x7f48b413eff0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f48b40b6560 tx=0x7f48ac016000).stop 2020-11-28T21:46:40.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.867+0000 7f48bb375700 1 -- 172.21.15.162:0/1634512881 shutdown_connections 2020-11-28T21:46:40.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.867+0000 7f48bb375700 1 --2- 172.21.15.162:0/1634512881 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f489c055330 0x7f489c0577f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:40.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.867+0000 7f48bb375700 1 --2- 172.21.15.162:0/1634512881 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48b40b6130 0x7f48b413eff0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:40.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.867+0000 7f48bb375700 1 --2- 172.21.15.162:0/1634512881 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48b40608f0 0x7f48b413d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:40.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.867+0000 7f48bb375700 1 --2- 172.21.15.162:0/1634512881 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f48b40b54f0 0x7f48b413eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:40.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.867+0000 7f48bb375700 1 -- 172.21.15.162:0/1634512881 >> 172.21.15.162:0/1634512881 conn(0x7f48b4066c70 msgr2=0x7f48b4067050 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:40.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.867+0000 7f48bb375700 1 -- 172.21.15.162:0/1634512881 shutdown_connections 2020-11-28T21:46:40.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:40.867+0000 7f48bb375700 1 -- 172.21.15.162:0/1634512881 wait complete. 2020-11-28T21:46:40.875 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 5 2020-11-28T21:46:41.182 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 5, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-28T21:46:29.120502+0000', 'modified': '2020-11-28T21:46:30.123428+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4284}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4284': {'gid': 4284, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.15.162:6835/3696705934', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6834', 'nonce': 3696705934}, {'type': 'v1', 'addr': '172.21.15.162:6835', 'nonce': 3696705934}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-28T21:46:41.183 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-28T21:46:41.184 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-28T21:46:41.313 INFO:teuthology.orchestra.run.smithi162.stdout:{"cluster_fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","whoami":0,"id":4284,"want_state":"up:active","state":"up:active","rank_uptime":12.177228646,"mdsmap_epoch":5,"osdmap_epoch":23,"osdmap_epoch_barrier":22,"uptime":15.555599912} 2020-11-28T21:46:41.313 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", "id": 4284, "mdsmap_epoch": 5, "osdmap_epoch": 23, "osdmap_epoch_barrier": 22, "rank_uptime": 12.177228646, "state": "up:active", "uptime": 15.555599912, "want_state": "up:active", "whoami": 0 } 2020-11-28T21:46:41.314 INFO:teuthology.run_tasks:Running task ceph-fuse... 2020-11-28T21:46:41.334 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2020-11-28T21:46:41.335 INFO:tasks.ceph_fuse:config is None 2020-11-28T21:46:41.335 INFO:tasks.ceph_fuse:new config is {'client.0': None, 'client.1': None} 2020-11-28T21:46:41.336 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2020-11-28T21:46:41.337 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:46:41.451 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.447+0000 7f12f3fff700 1 Processor -- start 2020-11-28T21:46:41.451 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.447+0000 7f12f3fff700 1 -- start start 2020-11-28T21:46:41.454 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.447+0000 7f12f3fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12f4098b30 0x7f12f40994f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:41.454 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.447+0000 7f12f3fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f12f40b0000 0x7f12f4060000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:41.455 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.447+0000 7f12f3fff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f12f40604f0 0x7f12f4060980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:41.455 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.447+0000 7f12f3fff700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12f40b1030 con 0x7f12f40604f0 2020-11-28T21:46:41.455 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.447+0000 7f12f3fff700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12f40b0480 con 0x7f12f40b0000 2020-11-28T21:46:41.455 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.447+0000 7f12f3fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12f4098f10 con 0x7f12f4098b30 2020-11-28T21:46:41.456 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.447+0000 7f12f27fc700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f12f40b0000 0x7f12f4060000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:41.456 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.447+0000 7f12f27fc700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f12f40b0000 0x7f12f4060000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37766/0 (socket says 172.21.15.162:37766) 2020-11-28T21:46:41.456 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.447+0000 7f12f27fc700 1 -- 172.21.15.162:0/934122195 learned_addr learned my addr 172.21.15.162:0/934122195 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:41.456 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.447+0000 7f12f2ffd700 1 --2- 172.21.15.162:0/934122195 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12f4098b30 0x7f12f40994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:41.457 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.447+0000 7f12f27fc700 1 -- 172.21.15.162:0/934122195 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12f4098b30 msgr2=0x7f12f40994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:41.457 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.447+0000 7f12f37fe700 1 --2- 172.21.15.162:0/934122195 >> v2:172.21.15.118:3300/0 conn(0x7f12f40604f0 0x7f12f4060980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:41.457 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.451+0000 7f12f27fc700 1 --2- 172.21.15.162:0/934122195 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12f4098b30 0x7f12f40994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:41.457 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.451+0000 7f12f27fc700 1 -- 172.21.15.162:0/934122195 >> v2:172.21.15.118:3300/0 conn(0x7f12f40604f0 msgr2=0x7f12f4060980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:41.458 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.451+0000 7f12f27fc700 1 --2- 172.21.15.162:0/934122195 >> v2:172.21.15.118:3300/0 conn(0x7f12f40604f0 0x7f12f4060980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:41.458 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.451+0000 7f12f27fc700 1 -- 172.21.15.162:0/934122195 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f40b0680 con 0x7f12f40b0000 2020-11-28T21:46:41.458 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.451+0000 7f12f37fe700 1 --2- 172.21.15.162:0/934122195 >> v2:172.21.15.118:3300/0 conn(0x7f12f40604f0 0x7f12f4060980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:46:41.458 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.451+0000 7f12f2ffd700 1 --2- 172.21.15.162:0/934122195 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12f4098b30 0x7f12f40994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:46:41.458 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.451+0000 7f12f27fc700 1 --2- 172.21.15.162:0/934122195 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12f40b0000 0x7f12f4060000 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f12f4099be0 tx=0x7f12dc006e80).ready entity=mon.1 client_cookie=e34dbcbd7c114292 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:41.459 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.451+0000 7f12f1ffb700 1 -- 172.21.15.162:0/934122195 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12dc0097b0 con 0x7f12f40b0000 2020-11-28T21:46:41.459 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.451+0000 7f12f1ffb700 1 -- 172.21.15.162:0/934122195 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12dc009910 con 0x7f12f40b0000 2020-11-28T21:46:41.460 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.451+0000 7f12f1ffb700 1 -- 172.21.15.162:0/934122195 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12dc009c70 con 0x7f12f40b0000 2020-11-28T21:46:41.460 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.451+0000 7f12f3fff700 1 -- 172.21.15.162:0/934122195 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12f40b0000 msgr2=0x7f12f4060000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:41.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.451+0000 7f12f3fff700 1 --2- 172.21.15.162:0/934122195 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12f40b0000 0x7f12f4060000 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f12f4099be0 tx=0x7f12dc006e80).stop 2020-11-28T21:46:41.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.451+0000 7f12f3fff700 1 -- 172.21.15.162:0/934122195 shutdown_connections 2020-11-28T21:46:41.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.451+0000 7f12f3fff700 1 --2- 172.21.15.162:0/934122195 >> v2:172.21.15.118:3300/0 conn(0x7f12f40604f0 0x7f12f4060980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:41.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.451+0000 7f12f3fff700 1 --2- 172.21.15.162:0/934122195 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12f4098b30 0x7f12f40994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:41.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.451+0000 7f12f3fff700 1 --2- 172.21.15.162:0/934122195 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12f40b0000 0x7f12f4060000 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:41.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.451+0000 7f12f3fff700 1 -- 172.21.15.162:0/934122195 >> 172.21.15.162:0/934122195 conn(0x7f12f405b970 msgr2=0x7f12f405bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:41.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.451+0000 7f12f3fff700 1 -- 172.21.15.162:0/934122195 shutdown_connections 2020-11-28T21:46:41.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.451+0000 7f12f3fff700 1 -- 172.21.15.162:0/934122195 wait complete. 2020-11-28T21:46:41.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.451+0000 7f12f3fff700 1 Processor -- start 2020-11-28T21:46:41.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.451+0000 7f12f3fff700 1 -- start start 2020-11-28T21:46:41.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.455+0000 7f12f3fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12f40604f0 0x7f12f4148ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:41.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.455+0000 7f12f3fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f12f4098b30 0x7f12f40644b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:41.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.455+0000 7f12f3fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12f40b0000 0x7f12f40649a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:41.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.455+0000 7f12f3fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f40aecb0 con 0x7f12f4098b30 2020-11-28T21:46:41.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.455+0000 7f12f3fff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f40640d0 con 0x7f12f40604f0 2020-11-28T21:46:41.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.455+0000 7f12f3fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12f40aeea0 con 0x7f12f40b0000 2020-11-28T21:46:41.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.455+0000 7f12f2ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12f40604f0 0x7f12f4148ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:41.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.455+0000 7f12f2ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12f40604f0 0x7f12f4148ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37768/0 (socket says 172.21.15.162:37768) 2020-11-28T21:46:41.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.455+0000 7f12f2ffd700 1 -- 172.21.15.162:0/1264598851 learned_addr learned my addr 172.21.15.162:0/1264598851 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:41.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.455+0000 7f12f27fc700 1 --2- 172.21.15.162:0/1264598851 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f12f4098b30 0x7f12f40644b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:41.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.455+0000 7f12f37fe700 1 --2- 172.21.15.162:0/1264598851 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12f40b0000 0x7f12f40649a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:41.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.455+0000 7f12f2ffd700 1 -- 172.21.15.162:0/1264598851 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12f40b0000 msgr2=0x7f12f40649a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:41.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.455+0000 7f12f2ffd700 1 --2- 172.21.15.162:0/1264598851 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12f40b0000 0x7f12f40649a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:41.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.455+0000 7f12f2ffd700 1 -- 172.21.15.162:0/1264598851 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f12f4098b30 msgr2=0x7f12f40644b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:41.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.455+0000 7f12f2ffd700 1 --2- 172.21.15.162:0/1264598851 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f12f4098b30 0x7f12f40644b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:41.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.455+0000 7f12f2ffd700 1 -- 172.21.15.162:0/1264598851 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f4063ae0 con 0x7f12f40604f0 2020-11-28T21:46:41.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.455+0000 7f12f2ffd700 1 --2- 172.21.15.162:0/1264598851 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12f40604f0 0x7f12f4148ef0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f12e4007000 tx=0x7f12e4008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:41.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.455+0000 7f12ebfff700 1 -- 172.21.15.162:0/1264598851 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12e400a8e0 con 0x7f12f40604f0 2020-11-28T21:46:41.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.455+0000 7f12f3fff700 1 -- 172.21.15.162:0/1264598851 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12f4063d80 con 0x7f12f40604f0 2020-11-28T21:46:41.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.455+0000 7f12ebfff700 1 -- 172.21.15.162:0/1264598851 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12e400aa40 con 0x7f12f40604f0 2020-11-28T21:46:41.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.455+0000 7f12ebfff700 1 -- 172.21.15.162:0/1264598851 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12e400ada0 con 0x7f12f40604f0 2020-11-28T21:46:41.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.455+0000 7f12f3fff700 1 -- 172.21.15.162:0/1264598851 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12f40af1e0 con 0x7f12f40604f0 2020-11-28T21:46:41.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.459+0000 7f12f3fff700 1 -- 172.21.15.162:0/1264598851 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12f4057050 con 0x7f12f40604f0 2020-11-28T21:46:41.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.459+0000 7f12ebfff700 1 -- 172.21.15.162:0/1264598851 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f12e401b460 con 0x7f12f40604f0 2020-11-28T21:46:41.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.459+0000 7f12ebfff700 1 --2- 172.21.15.162:0/1264598851 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f12cc055330 0x7f12cc0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:41.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.459+0000 7f12ebfff700 1 -- 172.21.15.162:0/1264598851 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f12e406e4c0 con 0x7f12f40604f0 2020-11-28T21:46:41.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.459+0000 7f12f27fc700 1 --2- 172.21.15.162:0/1264598851 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f12cc055330 0x7f12cc0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:41.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.459+0000 7f12f27fc700 1 --2- 172.21.15.162:0/1264598851 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f12cc055330 0x7f12cc0577f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f12dc01c000 tx=0x7f12dc015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:41.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.463+0000 7f12ebfff700 1 -- 172.21.15.162:0/1264598851 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f12e40446c0 con 0x7f12f40604f0 2020-11-28T21:46:41.605 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.603+0000 7f12f3fff700 1 -- 172.21.15.162:0/1264598851 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12f4068350 con 0x7f12f40604f0 2020-11-28T21:46:41.606 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.603+0000 7f12ebfff700 1 -- 172.21.15.162:0/1264598851 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16134 (secure 0 0 0) 0x7f12e4047ce0 con 0x7f12f40604f0 2020-11-28T21:46:41.607 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:46:41.607 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":23,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:46:30.124354+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:46:27.155497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-28T21:46:28.137474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:46:41.610 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.603+0000 7f12f3fff700 1 -- 172.21.15.162:0/1264598851 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f12cc055330 msgr2=0x7f12cc0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:41.610 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.603+0000 7f12f3fff700 1 --2- 172.21.15.162:0/1264598851 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f12cc055330 0x7f12cc0577f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f12dc01c000 tx=0x7f12dc015000).stop 2020-11-28T21:46:41.611 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.603+0000 7f12f3fff700 1 -- 172.21.15.162:0/1264598851 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12f40604f0 msgr2=0x7f12f4148ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:41.611 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.603+0000 7f12f3fff700 1 --2- 172.21.15.162:0/1264598851 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12f40604f0 0x7f12f4148ef0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f12e4007000 tx=0x7f12e4008040).stop 2020-11-28T21:46:41.612 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.603+0000 7f12f3fff700 1 -- 172.21.15.162:0/1264598851 shutdown_connections 2020-11-28T21:46:41.612 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.603+0000 7f12f3fff700 1 --2- 172.21.15.162:0/1264598851 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f12cc055330 0x7f12cc0577f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:41.612 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.603+0000 7f12f3fff700 1 --2- 172.21.15.162:0/1264598851 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12f40b0000 0x7f12f40649a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:41.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.603+0000 7f12f3fff700 1 --2- 172.21.15.162:0/1264598851 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12f40604f0 0x7f12f4148ef0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:41.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.603+0000 7f12f3fff700 1 --2- 172.21.15.162:0/1264598851 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f12f4098b30 0x7f12f40644b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:41.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.603+0000 7f12f3fff700 1 -- 172.21.15.162:0/1264598851 >> 172.21.15.162:0/1264598851 conn(0x7f12f405b970 msgr2=0x7f12f40ab5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:41.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.603+0000 7f12f3fff700 1 -- 172.21.15.162:0/1264598851 shutdown_connections 2020-11-28T21:46:41.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.603+0000 7f12f3fff700 1 -- 172.21.15.162:0/1264598851 wait complete. 2020-11-28T21:46:41.621 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-28T21:46:27.155497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-28T21:46:28.137474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-28T21:46:41.622 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:46:41.737 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.735+0000 7f53c37f6700 1 Processor -- start 2020-11-28T21:46:41.737 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.735+0000 7f53c37f6700 1 -- start start 2020-11-28T21:46:41.746 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.735+0000 7f53c37f6700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f53bc0b5730 0x7f53bc14ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:41.746 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.735+0000 7f53c37f6700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f53bc14b140 0x7f53bc147660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:41.747 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.735+0000 7f53c37f6700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f53bc0b1120 0x7f53bc0af820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:41.747 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.735+0000 7f53c37f6700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53bc0a8690 con 0x7f53bc0b1120 2020-11-28T21:46:41.747 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.735+0000 7f53c37f6700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53bc0a88c0 con 0x7f53bc0b5730 2020-11-28T21:46:41.748 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.735+0000 7f53c37f6700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc1471c0 con 0x7f53bc14b140 2020-11-28T21:46:41.748 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.735+0000 7f53c1592700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f53bc0b5730 0x7f53bc14ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:41.749 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.735+0000 7f53c1592700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f53bc0b5730 0x7f53bc14ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37776/0 (socket says 172.21.15.162:37776) 2020-11-28T21:46:41.749 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.735+0000 7f53c1592700 1 -- 172.21.15.162:0/2517083254 learned_addr learned my addr 172.21.15.162:0/2517083254 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:41.749 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.735+0000 7f53c1d93700 1 --2- 172.21.15.162:0/2517083254 >> v2:172.21.15.118:3300/0 conn(0x7f53bc0b1120 0x7f53bc0af820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:41.750 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.735+0000 7f53c0d91700 1 --2- 172.21.15.162:0/2517083254 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f53bc14b140 0x7f53bc147660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:41.750 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.735+0000 7f53c1592700 1 -- 172.21.15.162:0/2517083254 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f53bc14b140 msgr2=0x7f53bc147660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:41.751 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.735+0000 7f53c1592700 1 --2- 172.21.15.162:0/2517083254 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f53bc14b140 0x7f53bc147660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:41.751 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.735+0000 7f53c1592700 1 -- 172.21.15.162:0/2517083254 >> v2:172.21.15.118:3300/0 conn(0x7f53bc0b1120 msgr2=0x7f53bc0af820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:41.751 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.735+0000 7f53c1592700 1 --2- 172.21.15.162:0/2517083254 >> v2:172.21.15.118:3300/0 conn(0x7f53bc0b1120 0x7f53bc0af820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:41.752 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.735+0000 7f53c1592700 1 -- 172.21.15.162:0/2517083254 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53bc146ef0 con 0x7f53bc0b5730 2020-11-28T21:46:41.752 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.735+0000 7f53c1d93700 1 --2- 172.21.15.162:0/2517083254 >> v2:172.21.15.118:3300/0 conn(0x7f53bc0b1120 0x7f53bc0af820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:46:41.752 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.735+0000 7f53c0d91700 1 --2- 172.21.15.162:0/2517083254 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f53bc14b140 0x7f53bc147660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:46:41.753 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.735+0000 7f53c1592700 1 --2- 172.21.15.162:0/2517083254 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f53bc0b5730 0x7f53bc14ac50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f53b8002f30 tx=0x7f53b8006e80).ready entity=mon.1 client_cookie=5d3a06d52446a071 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:41.753 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.735+0000 7f53b3fff700 1 -- 172.21.15.162:0/2517083254 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f53b80097f0 con 0x7f53bc0b5730 2020-11-28T21:46:41.754 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.735+0000 7f53b3fff700 1 -- 172.21.15.162:0/2517083254 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b8009950 con 0x7f53bc0b5730 2020-11-28T21:46:41.754 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.735+0000 7f53b3fff700 1 -- 172.21.15.162:0/2517083254 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f53b8009cb0 con 0x7f53bc0b5730 2020-11-28T21:46:41.754 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.735+0000 7f53c37f6700 1 -- 172.21.15.162:0/2517083254 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f53bc0b5730 msgr2=0x7f53bc14ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:41.755 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.735+0000 7f53c37f6700 1 --2- 172.21.15.162:0/2517083254 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f53bc0b5730 0x7f53bc14ac50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f53b8002f30 tx=0x7f53b8006e80).stop 2020-11-28T21:46:41.755 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.739+0000 7f53c37f6700 1 -- 172.21.15.162:0/2517083254 shutdown_connections 2020-11-28T21:46:41.756 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.739+0000 7f53c37f6700 1 --2- 172.21.15.162:0/2517083254 >> v2:172.21.15.118:3300/0 conn(0x7f53bc0b1120 0x7f53bc0af820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:41.756 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.739+0000 7f53c37f6700 1 --2- 172.21.15.162:0/2517083254 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f53bc0b5730 0x7f53bc14ac50 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:41.756 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.739+0000 7f53c37f6700 1 --2- 172.21.15.162:0/2517083254 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f53bc14b140 0x7f53bc147660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:41.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.739+0000 7f53c37f6700 1 -- 172.21.15.162:0/2517083254 >> 172.21.15.162:0/2517083254 conn(0x7f53bc05c3f0 msgr2=0x7f53bc0b4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:41.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.739+0000 7f53c37f6700 1 -- 172.21.15.162:0/2517083254 shutdown_connections 2020-11-28T21:46:41.758 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.739+0000 7f53c37f6700 1 -- 172.21.15.162:0/2517083254 wait complete. 2020-11-28T21:46:41.758 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.739+0000 7f53c37f6700 1 Processor -- start 2020-11-28T21:46:41.758 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.739+0000 7f53c37f6700 1 -- start start 2020-11-28T21:46:41.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.739+0000 7f53c37f6700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f53bc0b1120 0x7f53bc148f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:41.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.739+0000 7f53c37f6700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f53bc0b5730 0x7f53bc13eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:41.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.739+0000 7f53c37f6700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f53bc14b140 0x7f53bc13f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:41.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.739+0000 7f53c37f6700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc0adab0 con 0x7f53bc14b140 2020-11-28T21:46:41.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.739+0000 7f53c37f6700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc13f580 con 0x7f53bc0b5730 2020-11-28T21:46:41.761 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.739+0000 7f53c37f6700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc0b5110 con 0x7f53bc0b1120 2020-11-28T21:46:41.761 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.739+0000 7f53c0d91700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f53bc0b5730 0x7f53bc13eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:41.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.739+0000 7f53c0d91700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f53bc0b5730 0x7f53bc13eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37784/0 (socket says 172.21.15.162:37784) 2020-11-28T21:46:41.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.739+0000 7f53c0d91700 1 -- 172.21.15.162:0/3429017555 learned_addr learned my addr 172.21.15.162:0/3429017555 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:41.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.739+0000 7f53c1592700 1 --2- 172.21.15.162:0/3429017555 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f53bc0b1120 0x7f53bc148f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:41.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.739+0000 7f53c1d93700 1 --2- 172.21.15.162:0/3429017555 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f53bc14b140 0x7f53bc13f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:41.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.739+0000 7f53c0d91700 1 -- 172.21.15.162:0/3429017555 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f53bc0b1120 msgr2=0x7f53bc148f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:41.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.739+0000 7f53c0d91700 1 --2- 172.21.15.162:0/3429017555 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f53bc0b1120 0x7f53bc148f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:41.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.739+0000 7f53c0d91700 1 -- 172.21.15.162:0/3429017555 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f53bc14b140 msgr2=0x7f53bc13f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:41.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.739+0000 7f53c0d91700 1 --2- 172.21.15.162:0/3429017555 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f53bc14b140 0x7f53bc13f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:41.765 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.739+0000 7f53c0d91700 1 -- 172.21.15.162:0/3429017555 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53bc1464b0 con 0x7f53bc0b5730 2020-11-28T21:46:41.765 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.739+0000 7f53c1592700 1 --2- 172.21.15.162:0/3429017555 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f53bc0b1120 0x7f53bc148f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:46:41.766 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.739+0000 7f53c1d93700 1 --2- 172.21.15.162:0/3429017555 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f53bc14b140 0x7f53bc13f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:46:41.766 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.739+0000 7f53c0d91700 1 --2- 172.21.15.162:0/3429017555 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f53bc0b5730 0x7f53bc13eba0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f53bc0b4630 tx=0x7f53ac008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:41.766 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.739+0000 7f53b27fc700 1 -- 172.21.15.162:0/3429017555 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f53ac00ce60 con 0x7f53bc0b5730 2020-11-28T21:46:41.767 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.739+0000 7f53b27fc700 1 -- 172.21.15.162:0/3429017555 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53ac007710 con 0x7f53bc0b5730 2020-11-28T21:46:41.767 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.739+0000 7f53c37f6700 1 -- 172.21.15.162:0/3429017555 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53bc146750 con 0x7f53bc0b5730 2020-11-28T21:46:41.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.739+0000 7f53b27fc700 1 -- 172.21.15.162:0/3429017555 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f53ac007a70 con 0x7f53bc0b5730 2020-11-28T21:46:41.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.739+0000 7f53c37f6700 1 -- 172.21.15.162:0/3429017555 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53bc0b4d90 con 0x7f53bc0b5730 2020-11-28T21:46:41.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.743+0000 7f53c37f6700 1 -- 172.21.15.162:0/3429017555 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53bc1485b0 con 0x7f53bc0b5730 2020-11-28T21:46:41.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.743+0000 7f53b27fc700 1 -- 172.21.15.162:0/3429017555 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f53ac007bd0 con 0x7f53bc0b5730 2020-11-28T21:46:41.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.743+0000 7f53b27fc700 1 --2- 172.21.15.162:0/3429017555 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f53a40553b0 0x7f53a4057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:41.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.743+0000 7f53b27fc700 1 -- 172.21.15.162:0/3429017555 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f53ac06e0d0 con 0x7f53bc0b5730 2020-11-28T21:46:41.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.743+0000 7f53c1592700 1 --2- 172.21.15.162:0/3429017555 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f53a40553b0 0x7f53a4057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:41.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.747+0000 7f53c1592700 1 --2- 172.21.15.162:0/3429017555 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f53a40553b0 0x7f53a4057870 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f53b801a000 tx=0x7f53b8015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:41.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.755+0000 7f53b27fc700 1 -- 172.21.15.162:0/3429017555 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f53bc1485b0 con 0x7f53bc0b5730 2020-11-28T21:46:41.899 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.895+0000 7f53c37f6700 1 -- 172.21.15.162:0/3429017555 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f53bc1485b0 con 0x7f53bc0b5730 2020-11-28T21:46:41.900 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.899+0000 7f53b27fc700 1 -- 172.21.15.162:0/3429017555 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v23) v1 ==== 110+0+10 (secure 0 0 0) 0x7f53bc1485b0 con 0x7f53bc0b5730 2020-11-28T21:46:41.900 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:46:41.902 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.899+0000 7f53c37f6700 1 -- 172.21.15.162:0/3429017555 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f53a40553b0 msgr2=0x7f53a4057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:41.903 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.899+0000 7f53c37f6700 1 --2- 172.21.15.162:0/3429017555 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f53a40553b0 0x7f53a4057870 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f53b801a000 tx=0x7f53b8015000).stop 2020-11-28T21:46:41.903 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.899+0000 7f53c37f6700 1 -- 172.21.15.162:0/3429017555 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f53bc0b5730 msgr2=0x7f53bc13eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:41.903 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.899+0000 7f53c37f6700 1 --2- 172.21.15.162:0/3429017555 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f53bc0b5730 0x7f53bc13eba0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f53bc0b4630 tx=0x7f53ac008040).stop 2020-11-28T21:46:41.903 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.899+0000 7f53c37f6700 1 -- 172.21.15.162:0/3429017555 shutdown_connections 2020-11-28T21:46:41.904 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.899+0000 7f53c37f6700 1 --2- 172.21.15.162:0/3429017555 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f53a40553b0 0x7f53a4057870 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:41.904 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.899+0000 7f53c37f6700 1 --2- 172.21.15.162:0/3429017555 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f53bc0b1120 0x7f53bc148f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:41.904 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.899+0000 7f53c37f6700 1 --2- 172.21.15.162:0/3429017555 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f53bc14b140 0x7f53bc13f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:41.904 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.899+0000 7f53c37f6700 1 --2- 172.21.15.162:0/3429017555 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f53bc0b5730 0x7f53bc13eba0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:41.904 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.899+0000 7f53c37f6700 1 -- 172.21.15.162:0/3429017555 >> 172.21.15.162:0/3429017555 conn(0x7f53bc05c3f0 msgr2=0x7f53bc0af690 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:41.905 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.899+0000 7f53c37f6700 1 -- 172.21.15.162:0/3429017555 shutdown_connections 2020-11-28T21:46:41.905 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:41.899+0000 7f53c37f6700 1 -- 172.21.15.162:0/3429017555 wait complete. 2020-11-28T21:46:41.914 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:46:42.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.027+0000 7f78bea93700 1 Processor -- start 2020-11-28T21:46:42.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.027+0000 7f78bea93700 1 -- start start 2020-11-28T21:46:42.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.027+0000 7f78bea93700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f78b80b2d00 0x7f78b805c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:42.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.027+0000 7f78bea93700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f78b80b30e0 0x7f78b813fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:42.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.027+0000 7f78bea93700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78b805baf0 con 0x7f78b80b3ab0 2020-11-28T21:46:42.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.027+0000 7f78bea93700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78b80b0e20 con 0x7f78b80b2d00 2020-11-28T21:46:42.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.027+0000 7f78bea93700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78b805bd90 con 0x7f78b80b30e0 2020-11-28T21:46:42.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.027+0000 7f78be292700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f78b80b30e0 0x7f78b813fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:42.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.027+0000 7f78be292700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f78b80b30e0 0x7f78b813fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37794/0 (socket says 172.21.15.162:37794) 2020-11-28T21:46:42.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.027+0000 7f78be292700 1 -- 172.21.15.162:0/4027842382 learned_addr learned my addr 172.21.15.162:0/4027842382 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:42.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.031+0000 7f78bda91700 1 --2- 172.21.15.162:0/4027842382 >> v2:172.21.15.118:3300/0 conn(0x7f78b80b2d00 0x7f78b805c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:42.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.031+0000 7f78be292700 1 -- 172.21.15.162:0/4027842382 >> v2:172.21.15.118:3300/0 conn(0x7f78b80b2d00 msgr2=0x7f78b805c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:42.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.031+0000 7f78be292700 1 --2- 172.21.15.162:0/4027842382 >> v2:172.21.15.118:3300/0 conn(0x7f78b80b2d00 0x7f78b805c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:42.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.031+0000 7f78be292700 1 -- 172.21.15.162:0/4027842382 >> v1:172.21.15.118:6789/0 conn(0x7f78b80b3ab0 legacy=0x7f78b8142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:42.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.031+0000 7f78be292700 1 -- 172.21.15.162:0/4027842382 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b805c030 con 0x7f78b80b30e0 2020-11-28T21:46:42.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.031+0000 7f78be292700 1 --2- 172.21.15.162:0/4027842382 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78b80b30e0 0x7f78b813fa70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f78b80acde0 tx=0x7f78b0009270).ready entity=mon.1 client_cookie=f76e2589e31047d4 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:42.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.031+0000 7f78bca8f700 1 -- 172.21.15.162:0/4027842382 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78b0009e60 con 0x7f78b80b30e0 2020-11-28T21:46:42.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.031+0000 7f78bca8f700 1 -- 172.21.15.162:0/4027842382 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78b000bbc0 con 0x7f78b80b30e0 2020-11-28T21:46:42.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.031+0000 7f78bca8f700 1 -- 172.21.15.162:0/4027842382 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78b000bd30 con 0x7f78b80b30e0 2020-11-28T21:46:42.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.031+0000 7f78bea93700 1 -- 172.21.15.162:0/4027842382 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78b80b30e0 msgr2=0x7f78b813fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:42.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.031+0000 7f78bea93700 1 --2- 172.21.15.162:0/4027842382 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78b80b30e0 0x7f78b813fa70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f78b80acde0 tx=0x7f78b0009270).stop 2020-11-28T21:46:42.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.031+0000 7f78bea93700 1 -- 172.21.15.162:0/4027842382 shutdown_connections 2020-11-28T21:46:42.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.031+0000 7f78bea93700 1 --2- 172.21.15.162:0/4027842382 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78b80b30e0 0x7f78b813fa70 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:42.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.031+0000 7f78bea93700 1 --2- 172.21.15.162:0/4027842382 >> v2:172.21.15.118:3300/0 conn(0x7f78b80b2d00 0x7f78b805c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:42.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.031+0000 7f78bea93700 1 -- 172.21.15.162:0/4027842382 >> 172.21.15.162:0/4027842382 conn(0x7f78b80604c0 msgr2=0x7f78b80608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:42.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.031+0000 7f78bea93700 1 -- 172.21.15.162:0/4027842382 shutdown_connections 2020-11-28T21:46:42.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.031+0000 7f78bea93700 1 -- 172.21.15.162:0/4027842382 wait complete. 2020-11-28T21:46:42.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.031+0000 7f78bea93700 1 Processor -- start 2020-11-28T21:46:42.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.031+0000 7f78bea93700 1 -- start start 2020-11-28T21:46:42.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.035+0000 7f78bea93700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f78b80b2d00 0x7f78b8145820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:42.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.035+0000 7f78bea93700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78b80b30e0 0x7f78b81378b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:42.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.035+0000 7f78bea93700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f78b80b3ab0 0x7f78b8137da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:42.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.035+0000 7f78bea93700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b80aeee0 con 0x7f78b80b3ab0 2020-11-28T21:46:42.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.035+0000 7f78bea93700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b80af110 con 0x7f78b80b30e0 2020-11-28T21:46:42.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.035+0000 7f78bea93700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78b80af3b0 con 0x7f78b80b2d00 2020-11-28T21:46:42.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.035+0000 7f78bd290700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78b80b30e0 0x7f78b81378b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:42.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.035+0000 7f78bda91700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f78b80b2d00 0x7f78b8145820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:42.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.035+0000 7f78bd290700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78b80b30e0 0x7f78b81378b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37798/0 (socket says 172.21.15.162:37798) 2020-11-28T21:46:42.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.035+0000 7f78bd290700 1 -- 172.21.15.162:0/3296611032 learned_addr learned my addr 172.21.15.162:0/3296611032 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:42.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.035+0000 7f78be292700 1 --2- 172.21.15.162:0/3296611032 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f78b80b3ab0 0x7f78b8137da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:42.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.035+0000 7f78bda91700 1 -- 172.21.15.162:0/3296611032 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78b80b30e0 msgr2=0x7f78b81378b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:42.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.035+0000 7f78bda91700 1 --2- 172.21.15.162:0/3296611032 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78b80b30e0 0x7f78b81378b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:42.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.035+0000 7f78bda91700 1 -- 172.21.15.162:0/3296611032 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f78b80b3ab0 msgr2=0x7f78b8137da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:42.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.035+0000 7f78bda91700 1 --2- 172.21.15.162:0/3296611032 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f78b80b3ab0 0x7f78b8137da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:42.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.035+0000 7f78bda91700 1 -- 172.21.15.162:0/3296611032 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b80af650 con 0x7f78b80b2d00 2020-11-28T21:46:42.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.035+0000 7f78bda91700 1 --2- 172.21.15.162:0/3296611032 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f78b80b2d00 0x7f78b8145820 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f78b400c000 tx=0x7f78b4006dc0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:42.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.035+0000 7f78a6ffd700 1 -- 172.21.15.162:0/3296611032 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78b400ca10 con 0x7f78b80b2d00 2020-11-28T21:46:42.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.035+0000 7f78a6ffd700 1 -- 172.21.15.162:0/3296611032 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78b400cb70 con 0x7f78b80b2d00 2020-11-28T21:46:42.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.035+0000 7f78bea93700 1 -- 172.21.15.162:0/3296611032 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78b80af8f0 con 0x7f78b80b2d00 2020-11-28T21:46:42.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.035+0000 7f78a6ffd700 1 -- 172.21.15.162:0/3296611032 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78b400be40 con 0x7f78b80b2d00 2020-11-28T21:46:42.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.035+0000 7f78bea93700 1 -- 172.21.15.162:0/3296611032 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78b80ad930 con 0x7f78b80b2d00 2020-11-28T21:46:42.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.035+0000 7f78bea93700 1 -- 172.21.15.162:0/3296611032 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78b8057050 con 0x7f78b80b2d00 2020-11-28T21:46:42.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.039+0000 7f78a6ffd700 1 -- 172.21.15.162:0/3296611032 <== mon.2 v2:172.21.15.118:3301/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f78b4025430 con 0x7f78b80b2d00 2020-11-28T21:46:42.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.039+0000 7f78a6ffd700 1 --2- 172.21.15.162:0/3296611032 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f78980553b0 0x7f7898057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:42.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.039+0000 7f78a6ffd700 1 -- 172.21.15.162:0/3296611032 <== mon.2 v2:172.21.15.118:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f78b406eac0 con 0x7f78b80b2d00 2020-11-28T21:46:42.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.039+0000 7f78bd290700 1 --2- 172.21.15.162:0/3296611032 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f78980553b0 0x7f7898057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:42.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.039+0000 7f78bd290700 1 --2- 172.21.15.162:0/3296611032 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f78980553b0 0x7f7898057870 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f78a800c000 tx=0x7f78a8008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:42.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.047+0000 7f78a6ffd700 1 -- 172.21.15.162:0/3296611032 <== mon.2 v2:172.21.15.118:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f78b4044d60 con 0x7f78b80b2d00 2020-11-28T21:46:42.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.187+0000 7f78bea93700 1 -- 172.21.15.162:0/3296611032 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f78b805d250 con 0x7f78b80b2d00 2020-11-28T21:46:42.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.191+0000 7f78a6ffd700 1 -- 172.21.15.162:0/3296611032 <== mon.2 v2:172.21.15.118:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f78b4048380 con 0x7f78b80b2d00 2020-11-28T21:46:42.193 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:46:42.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.191+0000 7f78bea93700 1 -- 172.21.15.162:0/3296611032 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f78980553b0 msgr2=0x7f7898057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:42.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.191+0000 7f78bea93700 1 --2- 172.21.15.162:0/3296611032 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f78980553b0 0x7f7898057870 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f78a800c000 tx=0x7f78a8008040).stop 2020-11-28T21:46:42.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.191+0000 7f78bea93700 1 -- 172.21.15.162:0/3296611032 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f78b80b2d00 msgr2=0x7f78b8145820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:42.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.191+0000 7f78bea93700 1 --2- 172.21.15.162:0/3296611032 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f78b80b2d00 0x7f78b8145820 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f78b400c000 tx=0x7f78b4006dc0).stop 2020-11-28T21:46:42.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.191+0000 7f78bea93700 1 -- 172.21.15.162:0/3296611032 shutdown_connections 2020-11-28T21:46:42.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.191+0000 7f78bea93700 1 --2- 172.21.15.162:0/3296611032 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f78980553b0 0x7f7898057870 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:42.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.191+0000 7f78bea93700 1 --2- 172.21.15.162:0/3296611032 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f78b80b2d00 0x7f78b8145820 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:42.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.191+0000 7f78bea93700 1 --2- 172.21.15.162:0/3296611032 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f78b80b3ab0 0x7f78b8137da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:42.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.191+0000 7f78bea93700 1 --2- 172.21.15.162:0/3296611032 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78b80b30e0 0x7f78b81378b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:42.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.191+0000 7f78bea93700 1 -- 172.21.15.162:0/3296611032 >> 172.21.15.162:0/3296611032 conn(0x7f78b80604c0 msgr2=0x7f78b813abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:42.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.191+0000 7f78bea93700 1 -- 172.21.15.162:0/3296611032 shutdown_connections 2020-11-28T21:46:42.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.191+0000 7f78bea93700 1 -- 172.21.15.162:0/3296611032 wait complete. 2020-11-28T21:46:42.206 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-28T21:46:42.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.319+0000 7f3b33462700 1 Processor -- start 2020-11-28T21:46:42.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.319+0000 7f3b33462700 1 -- start start 2020-11-28T21:46:42.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.319+0000 7f3b33462700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3b2c0b15e0 0x7f3b2c099b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:42.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.319+0000 7f3b33462700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b2c06aca0 con 0x7f3b2c09a050 2020-11-28T21:46:42.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.319+0000 7f3b33462700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b2c06af40 con 0x7f3b2c13d0c0 2020-11-28T21:46:42.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.319+0000 7f3b33462700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b2c13d4a0 con 0x7f3b2c0b15e0 2020-11-28T21:46:42.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.319+0000 7f3b32460700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f3b2c13d0c0 0x7f3b2c0b0ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:44648/0 (socket says 172.21.15.162:44648) 2020-11-28T21:46:42.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.319+0000 7f3b32460700 1 -- 172.21.15.162:0/2122420994 learned_addr learned my addr 172.21.15.162:0/2122420994 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:42.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.319+0000 7f3b31c5f700 1 --2- 172.21.15.162:0/2122420994 >> v2:172.21.15.162:3300/0 conn(0x7f3b2c0b15e0 0x7f3b2c099b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:42.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.319+0000 7f3b3145e700 1 -- 172.21.15.162:0/2122420994 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1514986047 0 0) 0x7f3b2c06af40 con 0x7f3b2c13d0c0 2020-11-28T21:46:42.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.319+0000 7f3b3145e700 1 -- 172.21.15.162:0/2122420994 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b14003230 con 0x7f3b2c13d0c0 2020-11-28T21:46:42.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.319+0000 7f3b31c5f700 1 -- 172.21.15.162:0/2122420994 >> v1:172.21.15.162:6789/0 conn(0x7f3b2c13d0c0 legacy=0x7f3b2c0b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:42.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.319+0000 7f3b31c5f700 1 -- 172.21.15.162:0/2122420994 >> v1:172.21.15.118:6789/0 conn(0x7f3b2c09a050 legacy=0x7f3b2c05bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:42.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.319+0000 7f3b31c5f700 1 -- 172.21.15.162:0/2122420994 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b2c05baf0 con 0x7f3b2c0b15e0 2020-11-28T21:46:42.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.323+0000 7f3b31c5f700 1 --2- 172.21.15.162:0/2122420994 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b2c0b15e0 0x7f3b2c099b60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f3b1c002f30 tx=0x7f3b1c006dc0).ready entity=mon.1 client_cookie=7e7f9d47de6118d0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:42.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.323+0000 7f3b3145e700 1 -- 172.21.15.162:0/2122420994 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b1c009800 con 0x7f3b2c0b15e0 2020-11-28T21:46:42.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.323+0000 7f3b3145e700 1 -- 172.21.15.162:0/2122420994 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b1c009960 con 0x7f3b2c0b15e0 2020-11-28T21:46:42.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.323+0000 7f3b3145e700 1 -- 172.21.15.162:0/2122420994 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b1c009cc0 con 0x7f3b2c0b15e0 2020-11-28T21:46:42.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.323+0000 7f3b33462700 1 -- 172.21.15.162:0/2122420994 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b2c0b15e0 msgr2=0x7f3b2c099b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:42.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.323+0000 7f3b33462700 1 --2- 172.21.15.162:0/2122420994 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b2c0b15e0 0x7f3b2c099b60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f3b1c002f30 tx=0x7f3b1c006dc0).stop 2020-11-28T21:46:42.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.323+0000 7f3b33462700 1 -- 172.21.15.162:0/2122420994 shutdown_connections 2020-11-28T21:46:42.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.323+0000 7f3b33462700 1 --2- 172.21.15.162:0/2122420994 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b2c0b15e0 0x7f3b2c099b60 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:42.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.323+0000 7f3b33462700 1 -- 172.21.15.162:0/2122420994 >> 172.21.15.162:0/2122420994 conn(0x7f3b2c0b5520 msgr2=0x7f3b2c0b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:42.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.323+0000 7f3b33462700 1 -- 172.21.15.162:0/2122420994 shutdown_connections 2020-11-28T21:46:42.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.323+0000 7f3b33462700 1 -- 172.21.15.162:0/2122420994 wait complete. 2020-11-28T21:46:42.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.323+0000 7f3b33462700 1 Processor -- start 2020-11-28T21:46:42.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.323+0000 7f3b33462700 1 -- start start 2020-11-28T21:46:42.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.327+0000 7f3b33462700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b2c09a050 0x7f3b2c0b2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:42.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.327+0000 7f3b33462700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3b2c0b15e0 0x7f3b2c0b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:42.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.327+0000 7f3b33462700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3b2c13d0c0 0x7f3b2c13dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:42.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.327+0000 7f3b33462700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2c098da0 con 0x7f3b2c13d0c0 2020-11-28T21:46:42.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.327+0000 7f3b33462700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2c0b4a30 con 0x7f3b2c09a050 2020-11-28T21:46:42.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.327+0000 7f3b33462700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2c0b3a80 con 0x7f3b2c0b15e0 2020-11-28T21:46:42.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.327+0000 7f3b32460700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b2c09a050 0x7f3b2c0b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:42.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.327+0000 7f3b32460700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b2c09a050 0x7f3b2c0b2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37810/0 (socket says 172.21.15.162:37810) 2020-11-28T21:46:42.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.327+0000 7f3b32460700 1 -- 172.21.15.162:0/3111316104 learned_addr learned my addr 172.21.15.162:0/3111316104 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:42.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.327+0000 7f3b31c5f700 1 --2- 172.21.15.162:0/3111316104 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3b2c0b15e0 0x7f3b2c0b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:42.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.327+0000 7f3b32c61700 1 --2- 172.21.15.162:0/3111316104 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3b2c13d0c0 0x7f3b2c13dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:42.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.327+0000 7f3b32460700 1 -- 172.21.15.162:0/3111316104 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3b2c0b15e0 msgr2=0x7f3b2c0b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:42.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.327+0000 7f3b32460700 1 --2- 172.21.15.162:0/3111316104 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3b2c0b15e0 0x7f3b2c0b41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:42.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.327+0000 7f3b32460700 1 -- 172.21.15.162:0/3111316104 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3b2c13d0c0 msgr2=0x7f3b2c13dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:42.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.327+0000 7f3b32460700 1 --2- 172.21.15.162:0/3111316104 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3b2c13d0c0 0x7f3b2c13dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:42.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.327+0000 7f3b32460700 1 -- 172.21.15.162:0/3111316104 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b2c0b3d20 con 0x7f3b2c09a050 2020-11-28T21:46:42.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.327+0000 7f3b32c61700 1 --2- 172.21.15.162:0/3111316104 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3b2c13d0c0 0x7f3b2c13dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:46:42.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.327+0000 7f3b32460700 1 --2- 172.21.15.162:0/3111316104 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b2c09a050 0x7f3b2c0b2050 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f3b2c0b4f90 tx=0x7f3b28007a70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:42.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.327+0000 7f3b1b7fe700 1 -- 172.21.15.162:0/3111316104 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b280072e0 con 0x7f3b2c09a050 2020-11-28T21:46:42.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.327+0000 7f3b1b7fe700 1 -- 172.21.15.162:0/3111316104 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b28007440 con 0x7f3b2c09a050 2020-11-28T21:46:42.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.327+0000 7f3b1b7fe700 1 -- 172.21.15.162:0/3111316104 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b280057d0 con 0x7f3b2c09a050 2020-11-28T21:46:42.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.327+0000 7f3b33462700 1 -- 172.21.15.162:0/3111316104 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b2c0b2bf0 con 0x7f3b2c09a050 2020-11-28T21:46:42.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.327+0000 7f3b33462700 1 -- 172.21.15.162:0/3111316104 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b2c13e970 con 0x7f3b2c09a050 2020-11-28T21:46:42.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.327+0000 7f3b33462700 1 -- 172.21.15.162:0/3111316104 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b2c13ec50 con 0x7f3b2c09a050 2020-11-28T21:46:42.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.331+0000 7f3b1b7fe700 1 -- 172.21.15.162:0/3111316104 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f3b28010070 con 0x7f3b2c09a050 2020-11-28T21:46:42.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.331+0000 7f3b1b7fe700 1 --2- 172.21.15.162:0/3111316104 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3b0c0554e0 0x7f3b0c0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:42.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.331+0000 7f3b1b7fe700 1 -- 172.21.15.162:0/3111316104 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3b2806d660 con 0x7f3b2c09a050 2020-11-28T21:46:42.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.331+0000 7f3b31c5f700 1 --2- 172.21.15.162:0/3111316104 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3b0c0554e0 0x7f3b0c0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:42.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.331+0000 7f3b31c5f700 1 --2- 172.21.15.162:0/3111316104 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3b0c0554e0 0x7f3b0c0579a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f3b1c018000 tx=0x7f3b1c015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:42.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.335+0000 7f3b1b7fe700 1 -- 172.21.15.162:0/3111316104 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f3b28043860 con 0x7f3b2c09a050 2020-11-28T21:46:42.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.471+0000 7f3b33462700 1 -- 172.21.15.162:0/3111316104 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3b2c0b3eb0 con 0x7f3b2c09a050 2020-11-28T21:46:42.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.471+0000 7f3b1b7fe700 1 -- 172.21.15.162:0/3111316104 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+10 (secure 0 0 0) 0x7f3b28018020 con 0x7f3b2c09a050 2020-11-28T21:46:42.476 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:46:42.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.475+0000 7f3b33462700 1 -- 172.21.15.162:0/3111316104 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3b0c0554e0 msgr2=0x7f3b0c0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:42.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.475+0000 7f3b33462700 1 --2- 172.21.15.162:0/3111316104 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3b0c0554e0 0x7f3b0c0579a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f3b1c018000 tx=0x7f3b1c015000).stop 2020-11-28T21:46:42.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.475+0000 7f3b33462700 1 -- 172.21.15.162:0/3111316104 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b2c09a050 msgr2=0x7f3b2c0b2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:42.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.475+0000 7f3b33462700 1 --2- 172.21.15.162:0/3111316104 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b2c09a050 0x7f3b2c0b2050 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f3b2c0b4f90 tx=0x7f3b28007a70).stop 2020-11-28T21:46:42.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.475+0000 7f3b33462700 1 -- 172.21.15.162:0/3111316104 shutdown_connections 2020-11-28T21:46:42.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.475+0000 7f3b33462700 1 --2- 172.21.15.162:0/3111316104 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3b0c0554e0 0x7f3b0c0579a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:42.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.475+0000 7f3b33462700 1 --2- 172.21.15.162:0/3111316104 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3b2c13d0c0 0x7f3b2c13dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:42.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.475+0000 7f3b33462700 1 --2- 172.21.15.162:0/3111316104 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b2c09a050 0x7f3b2c0b2050 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:42.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.475+0000 7f3b33462700 1 --2- 172.21.15.162:0/3111316104 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3b2c0b15e0 0x7f3b2c0b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:42.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.475+0000 7f3b33462700 1 -- 172.21.15.162:0/3111316104 >> 172.21.15.162:0/3111316104 conn(0x7f3b2c0b5520 msgr2=0x7f3b2c13f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:42.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.475+0000 7f3b33462700 1 -- 172.21.15.162:0/3111316104 shutdown_connections 2020-11-28T21:46:42.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.475+0000 7f3b33462700 1 -- 172.21.15.162:0/3111316104 wait complete. 2020-11-28T21:46:42.490 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-28T21:46:42.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.599+0000 7f33acc34700 1 Processor -- start 2020-11-28T21:46:42.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.599+0000 7f33acc34700 1 -- start start 2020-11-28T21:46:42.603 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.599+0000 7f33acc34700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f33a80b3ab0 0x7f33a805c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:42.603 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.599+0000 7f33acc34700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33a805baf0 con 0x7f33a80b30e0 2020-11-28T21:46:42.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.599+0000 7f33acc34700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33a805bd90 con 0x7f33a80b2d00 2020-11-28T21:46:42.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.599+0000 7f33acc34700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33a80b0e20 con 0x7f33a80b3ab0 2020-11-28T21:46:42.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.599+0000 7f33a77fe700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f33a80b2d00 0x7f33a8142ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:44662/0 (socket says 172.21.15.162:44662) 2020-11-28T21:46:42.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.599+0000 7f33a77fe700 1 -- 172.21.15.162:0/3587535172 learned_addr learned my addr 172.21.15.162:0/3587535172 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:42.605 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.599+0000 7f33a6ffd700 1 --2- 172.21.15.162:0/3587535172 >> v2:172.21.15.118:3300/0 conn(0x7f33a80b3ab0 0x7f33a805c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:42.606 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.599+0000 7f33a67fc700 1 -- 172.21.15.162:0/3587535172 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1114547530 0 0) 0x7f33a805bd90 con 0x7f33a80b2d00 2020-11-28T21:46:42.606 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.603+0000 7f33a67fc700 1 -- 172.21.15.162:0/3587535172 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3394003230 con 0x7f33a80b2d00 2020-11-28T21:46:42.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.603+0000 7f33a67fc700 1 -- 172.21.15.162:0/3587535172 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1125854048 0 0) 0x7f33a805baf0 con 0x7f33a80b30e0 2020-11-28T21:46:42.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.603+0000 7f33a67fc700 1 -- 172.21.15.162:0/3587535172 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33a805bd90 con 0x7f33a80b30e0 2020-11-28T21:46:42.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.603+0000 7f33a6ffd700 1 -- 172.21.15.162:0/3587535172 >> v1:172.21.15.162:6789/0 conn(0x7f33a80b2d00 legacy=0x7f33a8142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:42.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.603+0000 7f33a67fc700 1 -- 172.21.15.162:0/3587535172 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 624746120 0 0) 0x7f3394003230 con 0x7f33a80b2d00 2020-11-28T21:46:42.608 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.603+0000 7f33a6ffd700 1 -- 172.21.15.162:0/3587535172 >> v1:172.21.15.118:6789/0 conn(0x7f33a80b30e0 legacy=0x7f33a813fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:42.608 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.603+0000 7f33a6ffd700 1 -- 172.21.15.162:0/3587535172 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a805c030 con 0x7f33a80b3ab0 2020-11-28T21:46:42.608 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.603+0000 7f33a6ffd700 1 --2- 172.21.15.162:0/3587535172 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f33a80b3ab0 0x7f33a805c4d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f339c0010d0 tx=0x7f3398006dc0).ready entity=mon.0 client_cookie=7118abaa50fb9b0a server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:42.608 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.603+0000 7f33a67fc700 1 -- 172.21.15.162:0/3587535172 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f339c0016d0 con 0x7f33a80b3ab0 2020-11-28T21:46:42.609 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.603+0000 7f33a67fc700 1 -- 172.21.15.162:0/3587535172 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f339800d610 con 0x7f33a80b3ab0 2020-11-28T21:46:42.609 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.603+0000 7f33a67fc700 1 -- 172.21.15.162:0/3587535172 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f339800d780 con 0x7f33a80b3ab0 2020-11-28T21:46:42.609 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.603+0000 7f33acc34700 1 -- 172.21.15.162:0/3587535172 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f33a80b3ab0 msgr2=0x7f33a805c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:42.609 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.603+0000 7f33acc34700 1 --2- 172.21.15.162:0/3587535172 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f33a80b3ab0 0x7f33a805c4d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f339c0010d0 tx=0x7f3398006dc0).stop 2020-11-28T21:46:42.609 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.603+0000 7f33acc34700 1 -- 172.21.15.162:0/3587535172 shutdown_connections 2020-11-28T21:46:42.610 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.603+0000 7f33acc34700 1 --2- 172.21.15.162:0/3587535172 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f33a80b3ab0 0x7f33a805c4d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:42.610 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.603+0000 7f33acc34700 1 -- 172.21.15.162:0/3587535172 >> 172.21.15.162:0/3587535172 conn(0x7f33a80604c0 msgr2=0x7f33a80608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:42.610 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.603+0000 7f33acc34700 1 -- 172.21.15.162:0/3587535172 shutdown_connections 2020-11-28T21:46:42.610 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.603+0000 7f33acc34700 1 -- 172.21.15.162:0/3587535172 wait complete. 2020-11-28T21:46:42.612 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.603+0000 7f33acc34700 1 Processor -- start 2020-11-28T21:46:42.612 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.603+0000 7f33acc34700 1 -- start start 2020-11-28T21:46:42.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.607+0000 7f33acc34700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33a80b2d00 0x7f33a8140e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:42.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.607+0000 7f33acc34700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f33a80b30e0 0x7f33a8139db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:42.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.607+0000 7f33acc34700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f33a80b3ab0 0x7f33a80b5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:42.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.607+0000 7f33acc34700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a8141990 con 0x7f33a80b3ab0 2020-11-28T21:46:42.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.607+0000 7f33acc34700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a80b9740 con 0x7f33a80b2d00 2020-11-28T21:46:42.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.607+0000 7f33acc34700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a8139b00 con 0x7f33a80b30e0 2020-11-28T21:46:42.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.607+0000 7f33a77fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33a80b2d00 0x7f33a8140e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:42.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.607+0000 7f33a77fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33a80b2d00 0x7f33a8140e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37824/0 (socket says 172.21.15.162:37824) 2020-11-28T21:46:42.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.607+0000 7f33a77fe700 1 -- 172.21.15.162:0/2032806811 learned_addr learned my addr 172.21.15.162:0/2032806811 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:42.615 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.607+0000 7f33a7fff700 1 --2- 172.21.15.162:0/2032806811 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f33a80b3ab0 0x7f33a80b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:42.615 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.607+0000 7f33a6ffd700 1 --2- 172.21.15.162:0/2032806811 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f33a80b30e0 0x7f33a8139db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:42.615 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.607+0000 7f33a77fe700 1 -- 172.21.15.162:0/2032806811 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f33a80b30e0 msgr2=0x7f33a8139db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:42.615 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.607+0000 7f33a77fe700 1 --2- 172.21.15.162:0/2032806811 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f33a80b30e0 0x7f33a8139db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:42.615 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.607+0000 7f33a77fe700 1 -- 172.21.15.162:0/2032806811 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f33a80b3ab0 msgr2=0x7f33a80b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:42.616 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.607+0000 7f33a77fe700 1 --2- 172.21.15.162:0/2032806811 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f33a80b3ab0 0x7f33a80b5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:42.616 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.607+0000 7f33a77fe700 1 -- 172.21.15.162:0/2032806811 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a80b9970 con 0x7f33a80b2d00 2020-11-28T21:46:42.616 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.607+0000 7f33a77fe700 1 --2- 172.21.15.162:0/2032806811 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33a80b2d00 0x7f33a8140e10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f33a81518a0 tx=0x7f339c001280).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:42.616 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.607+0000 7f33a4ff9700 1 -- 172.21.15.162:0/2032806811 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f339c007170 con 0x7f33a80b2d00 2020-11-28T21:46:42.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.607+0000 7f33a4ff9700 1 -- 172.21.15.162:0/2032806811 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f339c0072d0 con 0x7f33a80b2d00 2020-11-28T21:46:42.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.607+0000 7f33acc34700 1 -- 172.21.15.162:0/2032806811 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a80b9c10 con 0x7f33a80b2d00 2020-11-28T21:46:42.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.607+0000 7f33a4ff9700 1 -- 172.21.15.162:0/2032806811 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f339c0048f0 con 0x7f33a80b2d00 2020-11-28T21:46:42.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.607+0000 7f33acc34700 1 -- 172.21.15.162:0/2032806811 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a813a5d0 con 0x7f33a80b2d00 2020-11-28T21:46:42.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.607+0000 7f33acc34700 1 -- 172.21.15.162:0/2032806811 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33a8057050 con 0x7f33a80b2d00 2020-11-28T21:46:42.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.611+0000 7f33a4ff9700 1 -- 172.21.15.162:0/2032806811 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f339c01b070 con 0x7f33a80b2d00 2020-11-28T21:46:42.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.611+0000 7f33a4ff9700 1 --2- 172.21.15.162:0/2032806811 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f33880554e0 0x7f33880579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:42.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.611+0000 7f33a4ff9700 1 -- 172.21.15.162:0/2032806811 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f339c06d680 con 0x7f33a80b2d00 2020-11-28T21:46:42.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.611+0000 7f33a6ffd700 1 --2- 172.21.15.162:0/2032806811 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f33880554e0 0x7f33880579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:42.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.611+0000 7f33a6ffd700 1 --2- 172.21.15.162:0/2032806811 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f33880554e0 0x7f33880579a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f339801a000 tx=0x7f3398015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:42.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.615+0000 7f33a4ff9700 1 -- 172.21.15.162:0/2032806811 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f339c0438e0 con 0x7f33a80b2d00 2020-11-28T21:46:42.761 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.759+0000 7f33acc34700 1 -- 172.21.15.162:0/2032806811 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f33a805d250 con 0x7f33a80b2d00 2020-11-28T21:46:42.761 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.759+0000 7f33a4ff9700 1 -- 172.21.15.162:0/2032806811 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+10 (secure 0 0 0) 0x7f339c046f00 con 0x7f33a80b2d00 2020-11-28T21:46:42.762 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:46:42.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.759+0000 7f33acc34700 1 -- 172.21.15.162:0/2032806811 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f33880554e0 msgr2=0x7f33880579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:42.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.759+0000 7f33acc34700 1 --2- 172.21.15.162:0/2032806811 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f33880554e0 0x7f33880579a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f339801a000 tx=0x7f3398015000).stop 2020-11-28T21:46:42.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.759+0000 7f33acc34700 1 -- 172.21.15.162:0/2032806811 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33a80b2d00 msgr2=0x7f33a8140e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:42.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.759+0000 7f33acc34700 1 --2- 172.21.15.162:0/2032806811 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33a80b2d00 0x7f33a8140e10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f33a81518a0 tx=0x7f339c001280).stop 2020-11-28T21:46:42.765 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.759+0000 7f33acc34700 1 -- 172.21.15.162:0/2032806811 shutdown_connections 2020-11-28T21:46:42.765 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.759+0000 7f33acc34700 1 --2- 172.21.15.162:0/2032806811 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f33880554e0 0x7f33880579a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:42.765 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.759+0000 7f33acc34700 1 --2- 172.21.15.162:0/2032806811 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f33a80b3ab0 0x7f33a80b5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:42.765 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.759+0000 7f33acc34700 1 --2- 172.21.15.162:0/2032806811 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33a80b2d00 0x7f33a8140e10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:42.766 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.759+0000 7f33acc34700 1 --2- 172.21.15.162:0/2032806811 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f33a80b30e0 0x7f33a8139db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:42.766 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.759+0000 7f33acc34700 1 -- 172.21.15.162:0/2032806811 >> 172.21.15.162:0/2032806811 conn(0x7f33a80604c0 msgr2=0x7f33a80b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:42.766 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.759+0000 7f33acc34700 1 -- 172.21.15.162:0/2032806811 shutdown_connections 2020-11-28T21:46:42.766 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.759+0000 7f33acc34700 1 -- 172.21.15.162:0/2032806811 wait complete. 2020-11-28T21:46:42.775 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-28T21:46:42.776 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:46:42.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.879+0000 7f21863ea700 1 Processor -- start 2020-11-28T21:46:42.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.879+0000 7f21863ea700 1 -- start start 2020-11-28T21:46:42.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.879+0000 7f21863ea700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f21800b69a0 0x7f218013f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:42.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.879+0000 7f21863ea700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f218013cb20 0x7f2180149f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:42.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.879+0000 7f21863ea700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2180067e90 con 0x7f218013fae0 2020-11-28T21:46:42.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.879+0000 7f21863ea700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21800ae270 con 0x7f21800b69a0 2020-11-28T21:46:42.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.879+0000 7f21863ea700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2180142ed0 con 0x7f218013cb20 2020-11-28T21:46:42.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.879+0000 7f217ffff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f21800b69a0 0x7f218013f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:42.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.879+0000 7f217ffff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f21800b69a0 0x7f218013f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37832/0 (socket says 172.21.15.162:37832) 2020-11-28T21:46:42.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.879+0000 7f217ffff700 1 -- 172.21.15.162:0/2451130342 learned_addr learned my addr 172.21.15.162:0/2451130342 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:42.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.879+0000 7f217f7fe700 1 --1- 172.21.15.162:0/2451130342 >> v1:172.21.15.162:6789/0 conn(0x7f218013fae0 0x7f218013c410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:44678/0 (socket says 172.21.15.162:44678) 2020-11-28T21:46:42.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.879+0000 7f2184987700 1 --2- 172.21.15.162:0/2451130342 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f218013cb20 0x7f2180149f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:42.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.879+0000 7f217ffff700 1 -- 172.21.15.162:0/2451130342 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f218013cb20 msgr2=0x7f2180149f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:42.886 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.879+0000 7f217ffff700 1 --2- 172.21.15.162:0/2451130342 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f218013cb20 0x7f2180149f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:42.886 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.879+0000 7f217ffff700 1 -- 172.21.15.162:0/2451130342 >> v1:172.21.15.162:6789/0 conn(0x7f218013fae0 legacy=0x7f218013c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:42.886 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.879+0000 7f217ffff700 1 -- 172.21.15.162:0/2451130342 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21800b6d80 con 0x7f21800b69a0 2020-11-28T21:46:42.886 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.879+0000 7f217ffff700 1 --2- 172.21.15.162:0/2451130342 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21800b69a0 0x7f218013f5f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f21800b5f10 tx=0x7f2170006e80).ready entity=mon.1 client_cookie=d7cbd6b12fcc51a7 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:42.886 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.879+0000 7f217effd700 1 -- 172.21.15.162:0/2451130342 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21700097b0 con 0x7f21800b69a0 2020-11-28T21:46:42.887 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.879+0000 7f217effd700 1 -- 172.21.15.162:0/2451130342 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2170009910 con 0x7f21800b69a0 2020-11-28T21:46:42.887 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.879+0000 7f21863ea700 1 -- 172.21.15.162:0/2451130342 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21800b69a0 msgr2=0x7f218013f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:42.887 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.879+0000 7f217effd700 1 -- 172.21.15.162:0/2451130342 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2170009c70 con 0x7f21800b69a0 2020-11-28T21:46:42.887 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.879+0000 7f21863ea700 1 --2- 172.21.15.162:0/2451130342 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21800b69a0 0x7f218013f5f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f21800b5f10 tx=0x7f2170006e80).stop 2020-11-28T21:46:42.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.883+0000 7f21863ea700 1 -- 172.21.15.162:0/2451130342 shutdown_connections 2020-11-28T21:46:42.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.883+0000 7f21863ea700 1 --2- 172.21.15.162:0/2451130342 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21800b69a0 0x7f218013f5f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:42.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.883+0000 7f21863ea700 1 --2- 172.21.15.162:0/2451130342 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f218013cb20 0x7f2180149f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:42.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.883+0000 7f21863ea700 1 -- 172.21.15.162:0/2451130342 >> 172.21.15.162:0/2451130342 conn(0x7f2180062170 msgr2=0x7f21800b5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:42.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.883+0000 7f21863ea700 1 -- 172.21.15.162:0/2451130342 shutdown_connections 2020-11-28T21:46:42.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.883+0000 7f21863ea700 1 -- 172.21.15.162:0/2451130342 wait complete. 2020-11-28T21:46:42.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.883+0000 7f21863ea700 1 Processor -- start 2020-11-28T21:46:42.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.883+0000 7f21863ea700 1 -- start start 2020-11-28T21:46:42.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.883+0000 7f21863ea700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21800b69a0 0x7f21800b4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:42.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.883+0000 7f21863ea700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f218013cb20 0x7f2180152e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:42.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.883+0000 7f21863ea700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f218013fae0 0x7f2180153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:42.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.883+0000 7f21863ea700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2180064e60 con 0x7f218013fae0 2020-11-28T21:46:42.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.883+0000 7f21863ea700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2180066e90 con 0x7f21800b69a0 2020-11-28T21:46:42.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.883+0000 7f21863ea700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2180067130 con 0x7f218013cb20 2020-11-28T21:46:42.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.883+0000 7f217ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21800b69a0 0x7f21800b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:42.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.883+0000 7f217ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21800b69a0 0x7f21800b4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37838/0 (socket says 172.21.15.162:37838) 2020-11-28T21:46:42.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.883+0000 7f217ffff700 1 -- 172.21.15.162:0/494903663 learned_addr learned my addr 172.21.15.162:0/494903663 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:42.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.883+0000 7f2184987700 1 --2- 172.21.15.162:0/494903663 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f218013fae0 0x7f2180153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:42.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.883+0000 7f217f7fe700 1 --2- 172.21.15.162:0/494903663 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f218013cb20 0x7f2180152e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:42.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.887+0000 7f217ffff700 1 -- 172.21.15.162:0/494903663 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f218013cb20 msgr2=0x7f2180152e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:42.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.887+0000 7f217ffff700 1 --2- 172.21.15.162:0/494903663 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f218013cb20 0x7f2180152e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:42.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.887+0000 7f217ffff700 1 -- 172.21.15.162:0/494903663 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f218013fae0 msgr2=0x7f2180153350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:42.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.887+0000 7f217ffff700 1 --2- 172.21.15.162:0/494903663 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f218013fae0 0x7f2180153350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:42.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.887+0000 7f217ffff700 1 -- 172.21.15.162:0/494903663 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21800650c0 con 0x7f21800b69a0 2020-11-28T21:46:42.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.887+0000 7f217ffff700 1 --2- 172.21.15.162:0/494903663 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21800b69a0 0x7f21800b4700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f21800b5f10 tx=0x7f2170015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:42.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.887+0000 7f217d7fa700 1 -- 172.21.15.162:0/494903663 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2170011ce0 con 0x7f21800b69a0 2020-11-28T21:46:42.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.887+0000 7f21863ea700 1 -- 172.21.15.162:0/494903663 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2180065360 con 0x7f21800b69a0 2020-11-28T21:46:42.896 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.887+0000 7f217d7fa700 1 -- 172.21.15.162:0/494903663 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2170011e40 con 0x7f21800b69a0 2020-11-28T21:46:42.896 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.887+0000 7f217d7fa700 1 -- 172.21.15.162:0/494903663 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f217001e650 con 0x7f21800b69a0 2020-11-28T21:46:42.896 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.887+0000 7f21863ea700 1 -- 172.21.15.162:0/494903663 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2180153840 con 0x7f21800b69a0 2020-11-28T21:46:42.896 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.887+0000 7f21863ea700 1 -- 172.21.15.162:0/494903663 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2180057050 con 0x7f21800b69a0 2020-11-28T21:46:42.897 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.887+0000 7f217d7fa700 1 -- 172.21.15.162:0/494903663 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f217001a040 con 0x7f21800b69a0 2020-11-28T21:46:42.898 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.887+0000 7f217d7fa700 1 --2- 172.21.15.162:0/494903663 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2160055440 0x7f2160057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:42.898 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.891+0000 7f217f7fe700 1 --2- 172.21.15.162:0/494903663 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2160055440 0x7f2160057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:42.898 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.891+0000 7f217d7fa700 1 -- 172.21.15.162:0/494903663 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f217006f2b0 con 0x7f21800b69a0 2020-11-28T21:46:42.898 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.891+0000 7f217f7fe700 1 --2- 172.21.15.162:0/494903663 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2160055440 0x7f2160057900 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f217400b000 tx=0x7f2174000b80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:42.899 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:42.895+0000 7f217d7fa700 1 -- 172.21.15.162:0/494903663 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f2170045420 con 0x7f21800b69a0 2020-11-28T21:46:43.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:43.075+0000 7f21863ea700 1 -- 172.21.15.162:0/494903663 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f218006b330 con 0x7f21800b69a0 2020-11-28T21:46:43.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:43.079+0000 7f217d7fa700 1 -- 172.21.15.162:0/494903663 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2495 (secure 0 0 0) 0x7f21700239d0 con 0x7f21800b69a0 2020-11-28T21:46:43.081 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:46:43.082 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/1297721315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1297721315},{"type":"v1","addr":"172.21.15.118:6833","nonce":1297721315}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3},{"gid":4264,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2415281381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2415281381},{"type":"v1","addr":"172.21.15.162:6837","nonce":2415281381}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:46:29.120502+0000","modified":"2020-11-28T21:46:30.123428+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.162:6835/3696705934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3696705934},{"type":"v1","addr":"172.21.15.162:6835","nonce":3696705934}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-28T21:46:43.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:43.079+0000 7f21863ea700 1 -- 172.21.15.162:0/494903663 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2160055440 msgr2=0x7f2160057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:43.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:43.079+0000 7f21863ea700 1 --2- 172.21.15.162:0/494903663 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2160055440 0x7f2160057900 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f217400b000 tx=0x7f2174000b80).stop 2020-11-28T21:46:43.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:43.079+0000 7f21863ea700 1 -- 172.21.15.162:0/494903663 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21800b69a0 msgr2=0x7f21800b4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:43.086 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:43.079+0000 7f21863ea700 1 --2- 172.21.15.162:0/494903663 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21800b69a0 0x7f21800b4700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f21800b5f10 tx=0x7f2170015000).stop 2020-11-28T21:46:43.086 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:43.079+0000 7f21863ea700 1 -- 172.21.15.162:0/494903663 shutdown_connections 2020-11-28T21:46:43.086 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:43.079+0000 7f21863ea700 1 --2- 172.21.15.162:0/494903663 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2160055440 0x7f2160057900 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:43.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:43.079+0000 7f21863ea700 1 --2- 172.21.15.162:0/494903663 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f218013fae0 0x7f2180153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:43.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:43.079+0000 7f21863ea700 1 --2- 172.21.15.162:0/494903663 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21800b69a0 0x7f21800b4700 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:43.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:43.079+0000 7f21863ea700 1 --2- 172.21.15.162:0/494903663 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f218013cb20 0x7f2180152e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:43.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:43.079+0000 7f21863ea700 1 -- 172.21.15.162:0/494903663 >> 172.21.15.162:0/494903663 conn(0x7f2180062170 msgr2=0x7f21800b53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:43.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:43.079+0000 7f21863ea700 1 -- 172.21.15.162:0/494903663 shutdown_connections 2020-11-28T21:46:43.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:43.079+0000 7f21863ea700 1 -- 172.21.15.162:0/494903663 wait complete. 2020-11-28T21:46:43.089 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 5 2020-11-28T21:46:43.398 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 5, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-28T21:46:29.120502+0000', 'modified': '2020-11-28T21:46:30.123428+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4284}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4284': {'gid': 4284, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.15.162:6835/3696705934', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6834', 'nonce': 3696705934}, {'type': 'v1', 'addr': '172.21.15.162:6835', 'nonce': 3696705934}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-28T21:46:43.398 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-28T21:46:43.399 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-28T21:46:43.529 INFO:teuthology.orchestra.run.smithi162.stdout:{"cluster_fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","whoami":0,"id":4284,"want_state":"up:active","state":"up:active","rank_uptime":14.394302819,"mdsmap_epoch":5,"osdmap_epoch":23,"osdmap_epoch_barrier":22,"uptime":17.772673652000002} 2020-11-28T21:46:43.530 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", "id": 4284, "mdsmap_epoch": 5, "osdmap_epoch": 23, "osdmap_epoch_barrier": 22, "rank_uptime": 14.394302819, "state": "up:active", "uptime": 17.772673652, "want_state": "up:active", "whoami": 0 } 2020-11-28T21:46:43.531 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-11-28T21:46:43.531 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-11-28T21:46:43.532 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi192.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-11-28T21:46:43.532 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:46:43.532 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:46:43.541 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:46:43.599 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:46:43.599 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:46:43.660 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:46:43.660 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:46:43.661 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:46:43.661 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:46:43.712 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-11-28T21:46:43.712 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:46:43.756 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:46:43.773 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:46:43.773 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:46:43.828 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:46:43.829 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:46:43.830 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:46:43.830 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:46:43.845 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:2020-11-28T21:46:43.841+0000 7fbd957410c0 -1 init, newargv = 0x5614fc6f8b80 newargc=9ceph-fuse[11346]: starting ceph client 2020-11-28T21:46:43.845 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr: 2020-11-28T21:46:43.856 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:ceph-fuse[11346]: starting fuse 2020-11-28T21:46:44.952 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:46:44.968 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:46:44.968 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:46:45.024 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:46:45.025 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:46:45.025 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:46:45.025 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:46:45.071 INFO:teuthology.orchestra.run.smithi192.stdout:49 2020-11-28T21:46:45.072 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2020-11-28T21:46:45.072 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:46:45.072 INFO:teuthology.orchestra.run.smithi192:> sudo python3 -c ' 2020-11-28T21:46:45.073 INFO:teuthology.orchestra.run.smithi192:> import glob 2020-11-28T21:46:45.073 INFO:teuthology.orchestra.run.smithi192:> import re 2020-11-28T21:46:45.073 INFO:teuthology.orchestra.run.smithi192:> import os 2020-11-28T21:46:45.074 INFO:teuthology.orchestra.run.smithi192:> import subprocess 2020-11-28T21:46:45.074 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:46:45.074 INFO:teuthology.orchestra.run.smithi192:> def find_socket(client_name): 2020-11-28T21:46:45.075 INFO:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-11-28T21:46:45.075 INFO:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2020-11-28T21:46:45.075 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:46:45.076 INFO:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2020-11-28T21:46:45.076 INFO:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2020-11-28T21:46:45.076 INFO:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2020-11-28T21:46:45.077 INFO:teuthology.orchestra.run.smithi192:> return files[0] 2020-11-28T21:46:45.077 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:46:45.077 INFO:teuthology.orchestra.run.smithi192:> for f in files: 2020-11-28T21:46:45.077 INFO:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-11-28T21:46:45.078 INFO:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2020-11-28T21:46:45.078 INFO:teuthology.orchestra.run.smithi192:> return f 2020-11-28T21:46:45.078 INFO:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-11-28T21:46:45.079 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:46:45.079 INFO:teuthology.orchestra.run.smithi192:> print(find_socket("client.0")) 2020-11-28T21:46:45.079 INFO:teuthology.orchestra.run.smithi192:> ' 2020-11-28T21:46:45.162 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.0.11346.asok 2020-11-28T21:46:45.167 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.11346.asok 2020-11-28T21:46:45.168 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:46:45.168 INFO:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.11346.asok status 2020-11-28T21:46:45.364 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2020-11-28T21:46:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2020-11-28T21:46:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "a83a3572a957356977342f461361c1cd3ec06dbd", 2020-11-28T21:46:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 15.2.6-595-ga83a3572 (a83a3572a957356977342f461361c1cd3ec06dbd) octopus (stable)", 2020-11-28T21:46:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "0", 2020-11-28T21:46:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2020-11-28T21:46:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-11-28T21:46:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "11346", 2020-11-28T21:46:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2020-11-28T21:46:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:46:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2020-11-28T21:46:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2020-11-28T21:46:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 4609, 2020-11-28T21:46:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2020-11-28T21:46:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2020-11-28T21:46:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2020-11-28T21:46:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 4609 2020-11-28T21:46:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:46:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2020-11-28T21:46:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2020-11-28T21:46:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2020-11-28T21:46:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 733565403 2020-11-28T21:46:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: } 2020-11-28T21:46:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:46:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2020-11-28T21:46:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2020-11-28T21:46:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2020-11-28T21:46:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 733565403 2020-11-28T21:46:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:46:45.371 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.4609 172.21.15.192:0/733565403", 2020-11-28T21:46:45.371 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "172.21.15.192:0/733565403", 2020-11-28T21:46:45.371 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2020-11-28T21:46:45.371 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 5, 2020-11-28T21:46:45.371 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 23, 2020-11-28T21:46:45.372 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2020-11-28T21:46:45.372 INFO:teuthology.orchestra.run.smithi192.stdout: "blacklisted": false 2020-11-28T21:46:45.372 INFO:teuthology.orchestra.run.smithi192.stdout:} 2020-11-28T21:46:45.375 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:46:45.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.495+0000 7f229a553700 1 Processor -- start 2020-11-28T21:46:45.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.495+0000 7f229a553700 1 -- start start 2020-11-28T21:46:45.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.495+0000 7f229a553700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f22940994f0 0x7f22940b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:45.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.495+0000 7f229a553700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2294060710 0x7f22940b1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:45.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.495+0000 7f229a553700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2294098f10 con 0x7f2294098b30 2020-11-28T21:46:45.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.495+0000 7f229a553700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22940b1030 con 0x7f2294060710 2020-11-28T21:46:45.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.495+0000 7f229a553700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22940b0680 con 0x7f22940994f0 2020-11-28T21:46:45.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.495+0000 7f2299d52700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2294060710 0x7f22940b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:45.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.495+0000 7f2299d52700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2294060710 0x7f22940b1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37848/0 (socket says 172.21.15.162:37848) 2020-11-28T21:46:45.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.495+0000 7f2299d52700 1 -- 172.21.15.162:0/807352000 learned_addr learned my addr 172.21.15.162:0/807352000 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:45.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.495+0000 7f2298d50700 1 --2- 172.21.15.162:0/807352000 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f22940994f0 0x7f22940b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:45.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.495+0000 7f2299d52700 1 -- 172.21.15.162:0/807352000 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f22940994f0 msgr2=0x7f22940b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:45.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.495+0000 7f2299d52700 1 --2- 172.21.15.162:0/807352000 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f22940994f0 0x7f22940b0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:45.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.495+0000 7f2299d52700 1 -- 172.21.15.162:0/807352000 >> v1:172.21.15.118:6789/0 conn(0x7f2294098b30 legacy=0x7f2294060000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:45.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.495+0000 7f2299d52700 1 -- 172.21.15.162:0/807352000 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22940b0920 con 0x7f2294060710 2020-11-28T21:46:45.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.495+0000 7f2298d50700 1 --2- 172.21.15.162:0/807352000 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f22940994f0 0x7f22940b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:46:45.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.495+0000 7f2299d52700 1 --2- 172.21.15.162:0/807352000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2294060710 0x7f22940b1580 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f228c002f30 tx=0x7f228c006e80).ready entity=mon.1 client_cookie=5812ce86151fa5ac server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:45.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.495+0000 7f2287fff700 1 -- 172.21.15.162:0/807352000 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f228c0097f0 con 0x7f2294060710 2020-11-28T21:46:45.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.495+0000 7f2287fff700 1 -- 172.21.15.162:0/807352000 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f228c009950 con 0x7f2294060710 2020-11-28T21:46:45.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.495+0000 7f229a553700 1 -- 172.21.15.162:0/807352000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2294060710 msgr2=0x7f22940b1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:45.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.495+0000 7f229a553700 1 --2- 172.21.15.162:0/807352000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2294060710 0x7f22940b1580 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f228c002f30 tx=0x7f228c006e80).stop 2020-11-28T21:46:45.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.495+0000 7f2287fff700 1 -- 172.21.15.162:0/807352000 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f228c009cb0 con 0x7f2294060710 2020-11-28T21:46:45.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.499+0000 7f229a553700 1 -- 172.21.15.162:0/807352000 shutdown_connections 2020-11-28T21:46:45.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.499+0000 7f229a553700 1 --2- 172.21.15.162:0/807352000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2294060710 0x7f22940b1580 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:45.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.499+0000 7f229a553700 1 --2- 172.21.15.162:0/807352000 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f22940994f0 0x7f22940b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:45.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.499+0000 7f229a553700 1 -- 172.21.15.162:0/807352000 >> 172.21.15.162:0/807352000 conn(0x7f229405b970 msgr2=0x7f229405bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:45.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.499+0000 7f229a553700 1 -- 172.21.15.162:0/807352000 shutdown_connections 2020-11-28T21:46:45.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.499+0000 7f229a553700 1 -- 172.21.15.162:0/807352000 wait complete. 2020-11-28T21:46:45.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.499+0000 7f229a553700 1 Processor -- start 2020-11-28T21:46:45.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.499+0000 7f229a553700 1 -- start start 2020-11-28T21:46:45.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.499+0000 7f229a553700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2294060710 0x7f2294145fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:45.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.499+0000 7f229a553700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2294098b30 0x7f22941590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:45.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.499+0000 7f229a553700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f22940994f0 0x7f22940b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:45.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.499+0000 7f229a553700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f229414d110 con 0x7f22940994f0 2020-11-28T21:46:45.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.499+0000 7f229a553700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f229414d340 con 0x7f2294060710 2020-11-28T21:46:45.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.499+0000 7f229a553700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f229414cd10 con 0x7f2294098b30 2020-11-28T21:46:45.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.499+0000 7f2299551700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2294060710 0x7f2294145fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:45.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.499+0000 7f2299551700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2294060710 0x7f2294145fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37852/0 (socket says 172.21.15.162:37852) 2020-11-28T21:46:45.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.499+0000 7f2299551700 1 -- 172.21.15.162:0/1135484661 learned_addr learned my addr 172.21.15.162:0/1135484661 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:45.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.499+0000 7f2299d52700 1 --2- 172.21.15.162:0/1135484661 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f22940994f0 0x7f22940b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:45.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.499+0000 7f2298d50700 1 --2- 172.21.15.162:0/1135484661 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2294098b30 0x7f22941590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:45.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.499+0000 7f2299551700 1 -- 172.21.15.162:0/1135484661 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2294098b30 msgr2=0x7f22941590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:45.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.499+0000 7f2299551700 1 --2- 172.21.15.162:0/1135484661 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2294098b30 0x7f22941590e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:45.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.499+0000 7f2299551700 1 -- 172.21.15.162:0/1135484661 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f22940994f0 msgr2=0x7f22940b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:45.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.499+0000 7f2299551700 1 --2- 172.21.15.162:0/1135484661 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f22940994f0 0x7f22940b3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:45.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.499+0000 7f2299551700 1 -- 172.21.15.162:0/1135484661 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f229414e470 con 0x7f2294060710 2020-11-28T21:46:45.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.499+0000 7f2299551700 1 --2- 172.21.15.162:0/1135484661 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2294060710 0x7f2294145fc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f229405fee0 tx=0x7f2288007be0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:45.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.499+0000 7f2286ffd700 1 -- 172.21.15.162:0/1135484661 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2288007240 con 0x7f2294060710 2020-11-28T21:46:45.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.499+0000 7f229a553700 1 -- 172.21.15.162:0/1135484661 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22940adce0 con 0x7f2294060710 2020-11-28T21:46:45.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.499+0000 7f2286ffd700 1 -- 172.21.15.162:0/1135484661 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22880073a0 con 0x7f2294060710 2020-11-28T21:46:45.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.499+0000 7f2286ffd700 1 -- 172.21.15.162:0/1135484661 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f22880066d0 con 0x7f2294060710 2020-11-28T21:46:45.521 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.499+0000 7f229a553700 1 -- 172.21.15.162:0/1135484661 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f229414bdf0 con 0x7f2294060710 2020-11-28T21:46:45.521 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.503+0000 7f229a553700 1 -- 172.21.15.162:0/1135484661 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f229414e600 con 0x7f2294060710 2020-11-28T21:46:45.521 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.503+0000 7f2286ffd700 1 -- 172.21.15.162:0/1135484661 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2288019020 con 0x7f2294060710 2020-11-28T21:46:45.521 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.503+0000 7f2286ffd700 1 --2- 172.21.15.162:0/1135484661 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2270055440 0x7f2270057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:45.521 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.503+0000 7f2286ffd700 1 -- 172.21.15.162:0/1135484661 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f228806d7a0 con 0x7f2294060710 2020-11-28T21:46:45.522 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.503+0000 7f2298d50700 1 --2- 172.21.15.162:0/1135484661 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2270055440 0x7f2270057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:45.522 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.503+0000 7f2298d50700 1 --2- 172.21.15.162:0/1135484661 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2270055440 0x7f2270057900 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f229000c000 tx=0x7f2290008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:45.522 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.511+0000 7f2286ffd700 1 -- 172.21.15.162:0/1135484661 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f2288043a00 con 0x7f2294060710 2020-11-28T21:46:45.656 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.651+0000 7f229a553700 1 -- 172.21.15.162:0/1135484661 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2294057050 con 0x7f2294060710 2020-11-28T21:46:45.657 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.655+0000 7f2286ffd700 1 -- 172.21.15.162:0/1135484661 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16134 (secure 0 0 0) 0x7f2288047020 con 0x7f2294060710 2020-11-28T21:46:45.657 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:46:45.657 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":23,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:46:30.124354+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:46:27.155497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-28T21:46:28.137474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:46:45.658 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.655+0000 7f229a553700 1 -- 172.21.15.162:0/1135484661 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2270055440 msgr2=0x7f2270057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:45.659 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.655+0000 7f229a553700 1 --2- 172.21.15.162:0/1135484661 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2270055440 0x7f2270057900 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f229000c000 tx=0x7f2290008040).stop 2020-11-28T21:46:45.659 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.655+0000 7f229a553700 1 -- 172.21.15.162:0/1135484661 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2294060710 msgr2=0x7f2294145fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:45.659 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.655+0000 7f229a553700 1 --2- 172.21.15.162:0/1135484661 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2294060710 0x7f2294145fc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f229405fee0 tx=0x7f2288007be0).stop 2020-11-28T21:46:45.659 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.655+0000 7f229a553700 1 -- 172.21.15.162:0/1135484661 shutdown_connections 2020-11-28T21:46:45.660 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.655+0000 7f229a553700 1 --2- 172.21.15.162:0/1135484661 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2270055440 0x7f2270057900 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:45.660 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.655+0000 7f229a553700 1 --2- 172.21.15.162:0/1135484661 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f22940994f0 0x7f22940b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:45.660 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.655+0000 7f229a553700 1 --2- 172.21.15.162:0/1135484661 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2294060710 0x7f2294145fc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:45.660 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.655+0000 7f229a553700 1 --2- 172.21.15.162:0/1135484661 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2294098b30 0x7f22941590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:45.660 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.655+0000 7f229a553700 1 -- 172.21.15.162:0/1135484661 >> 172.21.15.162:0/1135484661 conn(0x7f229405b970 msgr2=0x7f22940a84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:45.661 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.655+0000 7f229a553700 1 -- 172.21.15.162:0/1135484661 shutdown_connections 2020-11-28T21:46:45.661 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.655+0000 7f229a553700 1 -- 172.21.15.162:0/1135484661 wait complete. 2020-11-28T21:46:45.674 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-28T21:46:27.155497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-28T21:46:28.137474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-28T21:46:45.675 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:46:45.785 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.783+0000 7f9a3b344700 1 Processor -- start 2020-11-28T21:46:45.785 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.783+0000 7f9a3b344700 1 -- start start 2020-11-28T21:46:45.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.783+0000 7f9a3b344700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f9a340b9ce0 0x7f9a341433c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:45.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.783+0000 7f9a3b344700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9a340605c0 0x7f9a34067610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:45.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.783+0000 7f9a3b344700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a34143950 con 0x7f9a340b3c30 2020-11-28T21:46:45.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.783+0000 7f9a3b344700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a340ba0c0 con 0x7f9a340b9ce0 2020-11-28T21:46:45.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.783+0000 7f9a3b344700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a34066c30 con 0x7f9a340605c0 2020-11-28T21:46:45.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.783+0000 7f9a388df700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9a340605c0 0x7f9a34067610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:45.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.783+0000 7f9a390e0700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f9a340b9ce0 0x7f9a341433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:45.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.783+0000 7f9a388df700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9a340605c0 0x7f9a34067610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:35086/0 (socket says 172.21.15.162:35086) 2020-11-28T21:46:45.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.783+0000 7f9a390e0700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f9a340b9ce0 0x7f9a341433c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.162:44846/0 (socket says 172.21.15.162:44846) 2020-11-28T21:46:45.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.783+0000 7f9a398e1700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f9a340b3c30 0x7f9a3413fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.162:34852/0 (socket says 172.21.15.162:34852) 2020-11-28T21:46:45.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.783+0000 7f9a388df700 1 -- 172.21.15.162:0/2575140974 learned_addr learned my addr 172.21.15.162:0/2575140974 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:45.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.783+0000 7f9a388df700 1 -- 172.21.15.162:0/2575140974 >> v2:172.21.15.118:3300/0 conn(0x7f9a340b9ce0 msgr2=0x7f9a341433c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:45.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.783+0000 7f9a388df700 1 --2- 172.21.15.162:0/2575140974 >> v2:172.21.15.118:3300/0 conn(0x7f9a340b9ce0 0x7f9a341433c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:45.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.783+0000 7f9a388df700 1 -- 172.21.15.162:0/2575140974 >> v1:172.21.15.118:6789/0 conn(0x7f9a340b3c30 legacy=0x7f9a3413fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:45.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.783+0000 7f9a388df700 1 -- 172.21.15.162:0/2575140974 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a34066ed0 con 0x7f9a340605c0 2020-11-28T21:46:45.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.783+0000 7f9a388df700 1 --2- 172.21.15.162:0/2575140974 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9a340605c0 0x7f9a34067610 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f9a24002f30 tx=0x7f9a24006e80).ready entity=mon.2 client_cookie=94d69b15576738ee server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:45.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.783+0000 7f9a2bfff700 1 -- 172.21.15.162:0/2575140974 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a240097f0 con 0x7f9a340605c0 2020-11-28T21:46:45.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.783+0000 7f9a2bfff700 1 -- 172.21.15.162:0/2575140974 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a24009950 con 0x7f9a340605c0 2020-11-28T21:46:45.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.783+0000 7f9a2bfff700 1 -- 172.21.15.162:0/2575140974 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a24009cb0 con 0x7f9a340605c0 2020-11-28T21:46:45.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.783+0000 7f9a3b344700 1 -- 172.21.15.162:0/2575140974 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9a340605c0 msgr2=0x7f9a34067610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:45.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.787+0000 7f9a3b344700 1 --2- 172.21.15.162:0/2575140974 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9a340605c0 0x7f9a34067610 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f9a24002f30 tx=0x7f9a24006e80).stop 2020-11-28T21:46:45.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.787+0000 7f9a3b344700 1 -- 172.21.15.162:0/2575140974 shutdown_connections 2020-11-28T21:46:45.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.787+0000 7f9a3b344700 1 --2- 172.21.15.162:0/2575140974 >> v2:172.21.15.118:3300/0 conn(0x7f9a340b9ce0 0x7f9a341433c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:45.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.787+0000 7f9a3b344700 1 --2- 172.21.15.162:0/2575140974 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9a340605c0 0x7f9a34067610 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:45.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.787+0000 7f9a3b344700 1 -- 172.21.15.162:0/2575140974 >> 172.21.15.162:0/2575140974 conn(0x7f9a3405c430 msgr2=0x7f9a340b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:45.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.787+0000 7f9a3b344700 1 -- 172.21.15.162:0/2575140974 shutdown_connections 2020-11-28T21:46:45.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.787+0000 7f9a3b344700 1 -- 172.21.15.162:0/2575140974 wait complete. 2020-11-28T21:46:45.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.787+0000 7f9a3b344700 1 Processor -- start 2020-11-28T21:46:45.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.787+0000 7f9a3b344700 1 -- start start 2020-11-28T21:46:45.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.787+0000 7f9a3b344700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9a340605c0 0x7f9a340ae060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:45.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.787+0000 7f9a3b344700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9a340b3c30 0x7f9a340ae550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:45.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.787+0000 7f9a3b344700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a340b9ce0 0x7f9a3413eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:45.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.787+0000 7f9a3b344700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a340adb30 con 0x7f9a340605c0 2020-11-28T21:46:45.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.787+0000 7f9a3b344700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a340ad460 con 0x7f9a340b9ce0 2020-11-28T21:46:45.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.787+0000 7f9a3b344700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a34145da0 con 0x7f9a340b3c30 2020-11-28T21:46:45.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.787+0000 7f9a398e1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a340b9ce0 0x7f9a3413eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:45.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.787+0000 7f9a398e1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a340b9ce0 0x7f9a3413eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37868/0 (socket says 172.21.15.162:37868) 2020-11-28T21:46:45.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.787+0000 7f9a398e1700 1 -- 172.21.15.162:0/3593500623 learned_addr learned my addr 172.21.15.162:0/3593500623 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:45.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.787+0000 7f9a390e0700 1 --2- 172.21.15.162:0/3593500623 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9a340605c0 0x7f9a340ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:45.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.787+0000 7f9a388df700 1 --2- 172.21.15.162:0/3593500623 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9a340b3c30 0x7f9a340ae550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:45.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.791+0000 7f9a398e1700 1 -- 172.21.15.162:0/3593500623 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9a340b3c30 msgr2=0x7f9a340ae550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:45.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.791+0000 7f9a398e1700 1 --2- 172.21.15.162:0/3593500623 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9a340b3c30 0x7f9a340ae550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:45.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.791+0000 7f9a398e1700 1 -- 172.21.15.162:0/3593500623 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9a340605c0 msgr2=0x7f9a340ae060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:45.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.791+0000 7f9a398e1700 1 --2- 172.21.15.162:0/3593500623 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9a340605c0 0x7f9a340ae060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:45.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.791+0000 7f9a398e1700 1 -- 172.21.15.162:0/3593500623 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a3409b810 con 0x7f9a340b9ce0 2020-11-28T21:46:45.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.791+0000 7f9a390e0700 1 --2- 172.21.15.162:0/3593500623 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9a340605c0 0x7f9a340ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:46:45.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.791+0000 7f9a398e1700 1 --2- 172.21.15.162:0/3593500623 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a340b9ce0 0x7f9a3413eef0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f9a341401e0 tx=0x7f9a2c007f10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:45.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.791+0000 7f9a2a7fc700 1 -- 172.21.15.162:0/3593500623 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a2c00ce70 con 0x7f9a340b9ce0 2020-11-28T21:46:45.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.791+0000 7f9a2a7fc700 1 -- 172.21.15.162:0/3593500623 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a2c005ab0 con 0x7f9a340b9ce0 2020-11-28T21:46:45.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.791+0000 7f9a2a7fc700 1 -- 172.21.15.162:0/3593500623 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a2c005e10 con 0x7f9a340b9ce0 2020-11-28T21:46:45.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.791+0000 7f9a3b344700 1 -- 172.21.15.162:0/3593500623 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a341457b0 con 0x7f9a340b9ce0 2020-11-28T21:46:45.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.791+0000 7f9a3b344700 1 -- 172.21.15.162:0/3593500623 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a340ad690 con 0x7f9a340b9ce0 2020-11-28T21:46:45.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.791+0000 7f9a3b344700 1 -- 172.21.15.162:0/3593500623 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a341466e0 con 0x7f9a340b9ce0 2020-11-28T21:46:45.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.791+0000 7f9a2a7fc700 1 -- 172.21.15.162:0/3593500623 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9a2c010030 con 0x7f9a340b9ce0 2020-11-28T21:46:45.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.791+0000 7f9a2a7fc700 1 --2- 172.21.15.162:0/3593500623 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9a1c0553b0 0x7f9a1c057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:45.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.791+0000 7f9a2a7fc700 1 -- 172.21.15.162:0/3593500623 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9a2c06df20 con 0x7f9a340b9ce0 2020-11-28T21:46:45.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.791+0000 7f9a390e0700 1 --2- 172.21.15.162:0/3593500623 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9a1c0553b0 0x7f9a1c057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:45.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.795+0000 7f9a390e0700 1 --2- 172.21.15.162:0/3593500623 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9a1c0553b0 0x7f9a1c057870 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f9a30006000 tx=0x7f9a30006d60).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:45.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.799+0000 7f9a2a7fc700 1 -- 172.21.15.162:0/3593500623 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9a341466e0 con 0x7f9a340b9ce0 2020-11-28T21:46:45.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.935+0000 7f9a3b344700 1 -- 172.21.15.162:0/3593500623 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f9a341466e0 con 0x7f9a340b9ce0 2020-11-28T21:46:45.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.939+0000 7f9a2a7fc700 1 -- 172.21.15.162:0/3593500623 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v23) v1 ==== 110+0+10 (secure 0 0 0) 0x7f9a341466e0 con 0x7f9a340b9ce0 2020-11-28T21:46:45.941 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:46:45.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.939+0000 7f9a3b344700 1 -- 172.21.15.162:0/3593500623 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9a1c0553b0 msgr2=0x7f9a1c057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:45.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.939+0000 7f9a3b344700 1 --2- 172.21.15.162:0/3593500623 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9a1c0553b0 0x7f9a1c057870 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f9a30006000 tx=0x7f9a30006d60).stop 2020-11-28T21:46:45.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.939+0000 7f9a3b344700 1 -- 172.21.15.162:0/3593500623 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a340b9ce0 msgr2=0x7f9a3413eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:45.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.939+0000 7f9a3b344700 1 --2- 172.21.15.162:0/3593500623 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a340b9ce0 0x7f9a3413eef0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f9a341401e0 tx=0x7f9a2c007f10).stop 2020-11-28T21:46:45.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.939+0000 7f9a3b344700 1 -- 172.21.15.162:0/3593500623 shutdown_connections 2020-11-28T21:46:45.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.939+0000 7f9a3b344700 1 --2- 172.21.15.162:0/3593500623 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9a1c0553b0 0x7f9a1c057870 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:45.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.939+0000 7f9a3b344700 1 --2- 172.21.15.162:0/3593500623 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a340b9ce0 0x7f9a3413eef0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:45.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.939+0000 7f9a3b344700 1 --2- 172.21.15.162:0/3593500623 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9a340605c0 0x7f9a340ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:45.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.939+0000 7f9a3b344700 1 --2- 172.21.15.162:0/3593500623 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9a340b3c30 0x7f9a340ae550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:45.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.939+0000 7f9a3b344700 1 -- 172.21.15.162:0/3593500623 >> 172.21.15.162:0/3593500623 conn(0x7f9a3405c430 msgr2=0x7f9a3409eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:45.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.939+0000 7f9a3b344700 1 -- 172.21.15.162:0/3593500623 shutdown_connections 2020-11-28T21:46:45.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:45.939+0000 7f9a3b344700 1 -- 172.21.15.162:0/3593500623 wait complete. 2020-11-28T21:46:45.954 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:46:46.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.063+0000 7f8ae74d6700 1 Processor -- start 2020-11-28T21:46:46.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.063+0000 7f8ae74d6700 1 -- start start 2020-11-28T21:46:46.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.063+0000 7f8ae74d6700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8ae00ae230 0x7f8ae00a8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:46.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.063+0000 7f8ae74d6700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8ae0142dc0 0x7f8ae0149c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:46.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.063+0000 7f8ae74d6700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ae005f380 con 0x7f8ae00ade50 2020-11-28T21:46:46.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.063+0000 7f8ae74d6700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ae00acda0 con 0x7f8ae00ae230 2020-11-28T21:46:46.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.063+0000 7f8ae74d6700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae00a8a60 con 0x7f8ae0142dc0 2020-11-28T21:46:46.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.063+0000 7f8ae4a71700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8ae00ae230 0x7f8ae00a8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:46.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.063+0000 7f8ae4a71700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8ae00ae230 0x7f8ae00a8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37876/0 (socket says 172.21.15.162:37876) 2020-11-28T21:46:46.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.063+0000 7f8ae4a71700 1 -- 172.21.15.162:0/3020852642 learned_addr learned my addr 172.21.15.162:0/3020852642 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:46.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.063+0000 7f8ae5a73700 1 --2- 172.21.15.162:0/3020852642 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8ae0142dc0 0x7f8ae0149c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:46.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.063+0000 7f8ae4a71700 1 -- 172.21.15.162:0/3020852642 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8ae0142dc0 msgr2=0x7f8ae0149c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:46.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.063+0000 7f8ae4a71700 1 --2- 172.21.15.162:0/3020852642 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8ae0142dc0 0x7f8ae0149c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:46.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.063+0000 7f8ae4a71700 1 -- 172.21.15.162:0/3020852642 >> v1:172.21.15.118:6789/0 conn(0x7f8ae00ade50 legacy=0x7f8ae01426b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:46.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.063+0000 7f8ae4a71700 1 -- 172.21.15.162:0/3020852642 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae0061240 con 0x7f8ae00ae230 2020-11-28T21:46:46.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.063+0000 7f8ae4a71700 1 --2- 172.21.15.162:0/3020852642 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ae00ae230 0x7f8ae00a8570 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f8ae00abfe0 tx=0x7f8adc006f10).ready entity=mon.1 client_cookie=3d21ed8f085f9a57 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:46.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.067+0000 7f8ad7fff700 1 -- 172.21.15.162:0/3020852642 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8adc00cbe0 con 0x7f8ae00ae230 2020-11-28T21:46:46.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.067+0000 7f8ad7fff700 1 -- 172.21.15.162:0/3020852642 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8adc00cd40 con 0x7f8ae00ae230 2020-11-28T21:46:46.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.067+0000 7f8ad7fff700 1 -- 172.21.15.162:0/3020852642 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8adc010600 con 0x7f8ae00ae230 2020-11-28T21:46:46.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.067+0000 7f8ae74d6700 1 -- 172.21.15.162:0/3020852642 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ae00ae230 msgr2=0x7f8ae00a8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:46.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.067+0000 7f8ae74d6700 1 --2- 172.21.15.162:0/3020852642 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ae00ae230 0x7f8ae00a8570 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f8ae00abfe0 tx=0x7f8adc006f10).stop 2020-11-28T21:46:46.083 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.067+0000 7f8ae74d6700 1 -- 172.21.15.162:0/3020852642 shutdown_connections 2020-11-28T21:46:46.083 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.067+0000 7f8ae74d6700 1 --2- 172.21.15.162:0/3020852642 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8ae0142dc0 0x7f8ae0149c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:46.083 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.067+0000 7f8ae74d6700 1 --2- 172.21.15.162:0/3020852642 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ae00ae230 0x7f8ae00a8570 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:46.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.067+0000 7f8ae74d6700 1 -- 172.21.15.162:0/3020852642 >> 172.21.15.162:0/3020852642 conn(0x7f8ae005c3f0 msgr2=0x7f8ae00b4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:46.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.067+0000 7f8ae74d6700 1 -- 172.21.15.162:0/3020852642 shutdown_connections 2020-11-28T21:46:46.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.067+0000 7f8ae74d6700 1 -- 172.21.15.162:0/3020852642 wait complete. 2020-11-28T21:46:46.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.067+0000 7f8ae74d6700 1 Processor -- start 2020-11-28T21:46:46.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.067+0000 7f8ae74d6700 1 -- start start 2020-11-28T21:46:46.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.067+0000 7f8ae74d6700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8ae00ade50 0x7f8ae00669f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:46.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.067+0000 7f8ae74d6700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8ae00ae230 0x7f8ae0093ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:46.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.067+0000 7f8ae74d6700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ae0142dc0 0x7f8ae00943d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:46.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.067+0000 7f8ae74d6700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae00948c0 con 0x7f8ae00ae230 2020-11-28T21:46:46.086 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.067+0000 7f8ae74d6700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae00aab30 con 0x7f8ae0142dc0 2020-11-28T21:46:46.086 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.067+0000 7f8ae74d6700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae00aa410 con 0x7f8ae00ade50 2020-11-28T21:46:46.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.067+0000 7f8ae5a73700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ae0142dc0 0x7f8ae00943d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:46.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.067+0000 7f8ae5a73700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ae0142dc0 0x7f8ae00943d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37882/0 (socket says 172.21.15.162:37882) 2020-11-28T21:46:46.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.067+0000 7f8ae5a73700 1 -- 172.21.15.162:0/657289217 learned_addr learned my addr 172.21.15.162:0/657289217 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:46.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.067+0000 7f8ae4a71700 1 --2- 172.21.15.162:0/657289217 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8ae00ae230 0x7f8ae0093ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:46.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.071+0000 7f8ae5272700 1 --2- 172.21.15.162:0/657289217 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8ae00ade50 0x7f8ae00669f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:46.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.071+0000 7f8ae5a73700 1 -- 172.21.15.162:0/657289217 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8ae00ade50 msgr2=0x7f8ae00669f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:46.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.071+0000 7f8ae5a73700 1 --2- 172.21.15.162:0/657289217 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8ae00ade50 0x7f8ae00669f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:46.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.071+0000 7f8ae5a73700 1 -- 172.21.15.162:0/657289217 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8ae00ae230 msgr2=0x7f8ae0093ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:46.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.071+0000 7f8ae5a73700 1 --2- 172.21.15.162:0/657289217 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8ae00ae230 0x7f8ae0093ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:46.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.071+0000 7f8ae5a73700 1 -- 172.21.15.162:0/657289217 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae00aa6b0 con 0x7f8ae0142dc0 2020-11-28T21:46:46.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.071+0000 7f8ae5272700 1 --2- 172.21.15.162:0/657289217 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8ae00ade50 0x7f8ae00669f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:46:46.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.071+0000 7f8ae5a73700 1 --2- 172.21.15.162:0/657289217 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ae0142dc0 0x7f8ae00943d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f8ad800a000 tx=0x7f8ad8008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:46.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.071+0000 7f8ad67fc700 1 -- 172.21.15.162:0/657289217 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ad800ba70 con 0x7f8ae0142dc0 2020-11-28T21:46:46.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.071+0000 7f8ad67fc700 1 -- 172.21.15.162:0/657289217 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad800bbd0 con 0x7f8ae0142dc0 2020-11-28T21:46:46.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.071+0000 7f8ad67fc700 1 -- 172.21.15.162:0/657289217 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ad8012420 con 0x7f8ae0142dc0 2020-11-28T21:46:46.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.071+0000 7f8ae74d6700 1 -- 172.21.15.162:0/657289217 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ae0096c20 con 0x7f8ae0142dc0 2020-11-28T21:46:46.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.071+0000 7f8ae74d6700 1 -- 172.21.15.162:0/657289217 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae0067780 con 0x7f8ae0142dc0 2020-11-28T21:46:46.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.071+0000 7f8ae74d6700 1 -- 172.21.15.162:0/657289217 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ae0093d20 con 0x7f8ae0142dc0 2020-11-28T21:46:46.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.071+0000 7f8ad67fc700 1 -- 172.21.15.162:0/657289217 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8ad8012580 con 0x7f8ae0142dc0 2020-11-28T21:46:46.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.071+0000 7f8ad67fc700 1 --2- 172.21.15.162:0/657289217 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8ac80553b0 0x7f8ac8057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:46.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.071+0000 7f8ad67fc700 1 -- 172.21.15.162:0/657289217 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8ad806e6d0 con 0x7f8ae0142dc0 2020-11-28T21:46:46.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.071+0000 7f8ae5272700 1 --2- 172.21.15.162:0/657289217 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8ac80553b0 0x7f8ac8057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:46.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.075+0000 7f8ae5272700 1 --2- 172.21.15.162:0/657289217 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8ac80553b0 0x7f8ac8057870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f8ad0008000 tx=0x7f8ad0009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:46.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.079+0000 7f8ad67fc700 1 -- 172.21.15.162:0/657289217 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8ad8044970 con 0x7f8ae0142dc0 2020-11-28T21:46:46.218 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.215+0000 7f8ae74d6700 1 -- 172.21.15.162:0/657289217 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8ae0057050 con 0x7f8ae0142dc0 2020-11-28T21:46:46.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.215+0000 7f8ad67fc700 1 -- 172.21.15.162:0/657289217 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8ad8047f90 con 0x7f8ae0142dc0 2020-11-28T21:46:46.219 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:46:46.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.219+0000 7f8ae74d6700 1 -- 172.21.15.162:0/657289217 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8ac80553b0 msgr2=0x7f8ac8057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:46.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.219+0000 7f8ae74d6700 1 --2- 172.21.15.162:0/657289217 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8ac80553b0 0x7f8ac8057870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f8ad0008000 tx=0x7f8ad0009040).stop 2020-11-28T21:46:46.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.219+0000 7f8ae74d6700 1 -- 172.21.15.162:0/657289217 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ae0142dc0 msgr2=0x7f8ae00943d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:46.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.219+0000 7f8ae74d6700 1 --2- 172.21.15.162:0/657289217 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ae0142dc0 0x7f8ae00943d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f8ad800a000 tx=0x7f8ad8008040).stop 2020-11-28T21:46:46.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.219+0000 7f8ae74d6700 1 -- 172.21.15.162:0/657289217 shutdown_connections 2020-11-28T21:46:46.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.219+0000 7f8ae74d6700 1 --2- 172.21.15.162:0/657289217 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8ac80553b0 0x7f8ac8057870 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:46.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.219+0000 7f8ae74d6700 1 --2- 172.21.15.162:0/657289217 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8ae00ade50 0x7f8ae00669f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:46.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.219+0000 7f8ae74d6700 1 --2- 172.21.15.162:0/657289217 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ae0142dc0 0x7f8ae00943d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:46.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.219+0000 7f8ae74d6700 1 --2- 172.21.15.162:0/657289217 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8ae00ae230 0x7f8ae0093ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:46.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.219+0000 7f8ae74d6700 1 -- 172.21.15.162:0/657289217 >> 172.21.15.162:0/657289217 conn(0x7f8ae005c3f0 msgr2=0x7f8ae00b1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:46.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.219+0000 7f8ae74d6700 1 -- 172.21.15.162:0/657289217 shutdown_connections 2020-11-28T21:46:46.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.219+0000 7f8ae74d6700 1 -- 172.21.15.162:0/657289217 wait complete. 2020-11-28T21:46:46.233 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-28T21:46:46.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.351+0000 7f4baf110700 1 Processor -- start 2020-11-28T21:46:46.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.351+0000 7f4baf110700 1 -- start start 2020-11-28T21:46:46.355 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.351+0000 7f4baf110700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4ba80b18c0 0x7f4ba80b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:46.355 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.351+0000 7f4baf110700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f4ba80b1dd0 0x7f4ba8146e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:46.356 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.351+0000 7f4baf110700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ba814a440 con 0x7f4ba80b3950 2020-11-28T21:46:46.356 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.351+0000 7f4baf110700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ba814b2f0 con 0x7f4ba80b1dd0 2020-11-28T21:46:46.356 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.351+0000 7f4baf110700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ba814a670 con 0x7f4ba80b18c0 2020-11-28T21:46:46.356 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.351+0000 7f4bad90d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4ba80b18c0 0x7f4ba80b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:46.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.351+0000 7f4bad90d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4ba80b18c0 0x7f4ba80b3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37888/0 (socket says 172.21.15.162:37888) 2020-11-28T21:46:46.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.351+0000 7f4bad90d700 1 -- 172.21.15.162:0/1783292604 learned_addr learned my addr 172.21.15.162:0/1783292604 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:46.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.351+0000 7f4bae90f700 1 --2- 172.21.15.162:0/1783292604 >> v2:172.21.15.118:3300/0 conn(0x7f4ba80b1dd0 0x7f4ba8146e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:46.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.351+0000 7f4bad90d700 1 -- 172.21.15.162:0/1783292604 >> v2:172.21.15.118:3300/0 conn(0x7f4ba80b1dd0 msgr2=0x7f4ba8146e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:46.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.351+0000 7f4bad90d700 1 --2- 172.21.15.162:0/1783292604 >> v2:172.21.15.118:3300/0 conn(0x7f4ba80b1dd0 0x7f4ba8146e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:46.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.351+0000 7f4bad90d700 1 -- 172.21.15.162:0/1783292604 >> v1:172.21.15.118:6789/0 conn(0x7f4ba80b3950 legacy=0x7f4ba8060bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:46.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.351+0000 7f4bad90d700 1 -- 172.21.15.162:0/1783292604 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba814a910 con 0x7f4ba80b18c0 2020-11-28T21:46:46.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.351+0000 7f4bad90d700 1 --2- 172.21.15.162:0/1783292604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ba80b18c0 0x7f4ba80b3d30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f4ba4002f30 tx=0x7f4ba4006e80).ready entity=mon.1 client_cookie=febcf9a55663b1c6 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:46.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.351+0000 7f4bad10c700 1 -- 172.21.15.162:0/1783292604 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ba40097f0 con 0x7f4ba80b18c0 2020-11-28T21:46:46.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.351+0000 7f4bad10c700 1 -- 172.21.15.162:0/1783292604 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba4009950 con 0x7f4ba80b18c0 2020-11-28T21:46:46.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.351+0000 7f4bad10c700 1 -- 172.21.15.162:0/1783292604 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ba4009cb0 con 0x7f4ba80b18c0 2020-11-28T21:46:46.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.351+0000 7f4baf110700 1 -- 172.21.15.162:0/1783292604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ba80b18c0 msgr2=0x7f4ba80b3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:46.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.351+0000 7f4baf110700 1 --2- 172.21.15.162:0/1783292604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ba80b18c0 0x7f4ba80b3d30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f4ba4002f30 tx=0x7f4ba4006e80).stop 2020-11-28T21:46:46.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.355+0000 7f4baf110700 1 -- 172.21.15.162:0/1783292604 shutdown_connections 2020-11-28T21:46:46.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.355+0000 7f4baf110700 1 --2- 172.21.15.162:0/1783292604 >> v2:172.21.15.118:3300/0 conn(0x7f4ba80b1dd0 0x7f4ba8146e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:46.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.355+0000 7f4baf110700 1 --2- 172.21.15.162:0/1783292604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ba80b18c0 0x7f4ba80b3d30 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:46.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.355+0000 7f4baf110700 1 -- 172.21.15.162:0/1783292604 >> 172.21.15.162:0/1783292604 conn(0x7f4ba8062630 msgr2=0x7f4ba8062a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:46.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.355+0000 7f4baf110700 1 -- 172.21.15.162:0/1783292604 shutdown_connections 2020-11-28T21:46:46.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.355+0000 7f4baf110700 1 -- 172.21.15.162:0/1783292604 wait complete. 2020-11-28T21:46:46.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.355+0000 7f4baf110700 1 Processor -- start 2020-11-28T21:46:46.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.355+0000 7f4baf110700 1 -- start start 2020-11-28T21:46:46.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.355+0000 7f4baf110700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ba80b18c0 0x7f4ba8142a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:46.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.355+0000 7f4baf110700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ba80b1dd0 0x7f4ba8144d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:46.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.355+0000 7f4baf110700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4ba80b3950 0x7f4ba80ac8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:46.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.355+0000 7f4baf110700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba81432c0 con 0x7f4ba80b1dd0 2020-11-28T21:46:46.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.355+0000 7f4baf110700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba80ac490 con 0x7f4ba80b18c0 2020-11-28T21:46:46.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.355+0000 7f4baf110700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba815a080 con 0x7f4ba80b3950 2020-11-28T21:46:46.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.359+0000 7f4bae10e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ba80b18c0 0x7f4ba8142a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:46.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.359+0000 7f4bae90f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4ba80b3950 0x7f4ba80ac8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:46.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.359+0000 7f4bae10e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ba80b18c0 0x7f4ba8142a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37892/0 (socket says 172.21.15.162:37892) 2020-11-28T21:46:46.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.359+0000 7f4bae10e700 1 -- 172.21.15.162:0/123236751 learned_addr learned my addr 172.21.15.162:0/123236751 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:46.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.359+0000 7f4bad90d700 1 --2- 172.21.15.162:0/123236751 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ba80b1dd0 0x7f4ba8144d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:46.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.359+0000 7f4bae10e700 1 -- 172.21.15.162:0/123236751 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4ba80b3950 msgr2=0x7f4ba80ac8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:46.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.359+0000 7f4bae10e700 1 --2- 172.21.15.162:0/123236751 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4ba80b3950 0x7f4ba80ac8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:46.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.359+0000 7f4bae10e700 1 -- 172.21.15.162:0/123236751 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ba80b1dd0 msgr2=0x7f4ba8144d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:46.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.359+0000 7f4bae10e700 1 --2- 172.21.15.162:0/123236751 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ba80b1dd0 0x7f4ba8144d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:46.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.359+0000 7f4bae10e700 1 -- 172.21.15.162:0/123236751 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba815a320 con 0x7f4ba80b18c0 2020-11-28T21:46:46.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.359+0000 7f4bae10e700 1 --2- 172.21.15.162:0/123236751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ba80b18c0 0x7f4ba8142a80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f4ba8067d80 tx=0x7f4b9c007c20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:46.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.359+0000 7f4b9b7fe700 1 -- 172.21.15.162:0/123236751 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b9c013410 con 0x7f4ba80b18c0 2020-11-28T21:46:46.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.359+0000 7f4baf110700 1 -- 172.21.15.162:0/123236751 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ba80ab0a0 con 0x7f4ba80b18c0 2020-11-28T21:46:46.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.359+0000 7f4baf110700 1 -- 172.21.15.162:0/123236751 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba81461d0 con 0x7f4ba80b18c0 2020-11-28T21:46:46.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.359+0000 7f4b9b7fe700 1 -- 172.21.15.162:0/123236751 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b9c013570 con 0x7f4ba80b18c0 2020-11-28T21:46:46.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.359+0000 7f4b9b7fe700 1 -- 172.21.15.162:0/123236751 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b9c0138d0 con 0x7f4ba80b18c0 2020-11-28T21:46:46.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.359+0000 7f4baf110700 1 -- 172.21.15.162:0/123236751 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ba80ac600 con 0x7f4ba80b18c0 2020-11-28T21:46:46.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.359+0000 7f4b9b7fe700 1 -- 172.21.15.162:0/123236751 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4b9c010030 con 0x7f4ba80b18c0 2020-11-28T21:46:46.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.359+0000 7f4b9b7fe700 1 --2- 172.21.15.162:0/123236751 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4b88055330 0x7f4b880577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:46.371 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.359+0000 7f4b9b7fe700 1 -- 172.21.15.162:0/123236751 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4b9c06dd90 con 0x7f4ba80b18c0 2020-11-28T21:46:46.371 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.359+0000 7f4bad90d700 1 --2- 172.21.15.162:0/123236751 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4b88055330 0x7f4b880577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:46.371 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.363+0000 7f4bad90d700 1 --2- 172.21.15.162:0/123236751 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4b88055330 0x7f4b880577f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f4ba401c000 tx=0x7f4ba4015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:46.371 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.367+0000 7f4b9b7fe700 1 -- 172.21.15.162:0/123236751 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4b9c043ff0 con 0x7f4ba80b18c0 2020-11-28T21:46:46.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.511+0000 7f4baf110700 1 -- 172.21.15.162:0/123236751 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4ba8057050 con 0x7f4ba80b18c0 2020-11-28T21:46:46.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.511+0000 7f4b9b7fe700 1 -- 172.21.15.162:0/123236751 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+10 (secure 0 0 0) 0x7f4b9c047610 con 0x7f4ba80b18c0 2020-11-28T21:46:46.514 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:46:46.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.511+0000 7f4baf110700 1 -- 172.21.15.162:0/123236751 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4b88055330 msgr2=0x7f4b880577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:46.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.511+0000 7f4baf110700 1 --2- 172.21.15.162:0/123236751 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4b88055330 0x7f4b880577f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f4ba401c000 tx=0x7f4ba4015000).stop 2020-11-28T21:46:46.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.511+0000 7f4baf110700 1 -- 172.21.15.162:0/123236751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ba80b18c0 msgr2=0x7f4ba8142a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:46.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.511+0000 7f4baf110700 1 --2- 172.21.15.162:0/123236751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ba80b18c0 0x7f4ba8142a80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f4ba8067d80 tx=0x7f4b9c007c20).stop 2020-11-28T21:46:46.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.511+0000 7f4baf110700 1 -- 172.21.15.162:0/123236751 shutdown_connections 2020-11-28T21:46:46.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.511+0000 7f4baf110700 1 --2- 172.21.15.162:0/123236751 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4b88055330 0x7f4b880577f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:46.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.511+0000 7f4baf110700 1 --2- 172.21.15.162:0/123236751 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4ba80b3950 0x7f4ba80ac8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:46.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.511+0000 7f4baf110700 1 --2- 172.21.15.162:0/123236751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ba80b18c0 0x7f4ba8142a80 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:46.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.511+0000 7f4baf110700 1 --2- 172.21.15.162:0/123236751 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ba80b1dd0 0x7f4ba8144d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:46.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.511+0000 7f4baf110700 1 -- 172.21.15.162:0/123236751 >> 172.21.15.162:0/123236751 conn(0x7f4ba8062630 msgr2=0x7f4ba805c210 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:46.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.511+0000 7f4baf110700 1 -- 172.21.15.162:0/123236751 shutdown_connections 2020-11-28T21:46:46.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.511+0000 7f4baf110700 1 -- 172.21.15.162:0/123236751 wait complete. 2020-11-28T21:46:46.527 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-28T21:46:46.646 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.643+0000 7fbd1c87f700 1 Processor -- start 2020-11-28T21:46:46.646 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.643+0000 7fbd1c87f700 1 -- start start 2020-11-28T21:46:46.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.643+0000 7fbd1c87f700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbd180b2d00 0x7fbd1805c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:46.655 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.643+0000 7fbd1c87f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbd180b30e0 0x7fbd1813fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:46.655 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.643+0000 7fbd1c87f700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd1805baf0 con 0x7fbd180b3ab0 2020-11-28T21:46:46.656 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.643+0000 7fbd1c87f700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd180b0e20 con 0x7fbd180b2d00 2020-11-28T21:46:46.656 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.643+0000 7fbd1c87f700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1805bd90 con 0x7fbd180b30e0 2020-11-28T21:46:46.657 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.643+0000 7fbd16ffd700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fbd180b3ab0 0x7fbd18142ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:44744/0 (socket says 172.21.15.162:44744) 2020-11-28T21:46:46.657 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.643+0000 7fbd16ffd700 1 -- 172.21.15.162:0/4086268460 learned_addr learned my addr 172.21.15.162:0/4086268460 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:46.657 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.643+0000 7fbd177fe700 1 --2- 172.21.15.162:0/4086268460 >> v2:172.21.15.162:3300/0 conn(0x7fbd180b2d00 0x7fbd1805c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:46.658 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.643+0000 7fbd17fff700 1 --2- 172.21.15.162:0/4086268460 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbd180b30e0 0x7fbd1813fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:46.658 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.643+0000 7fbd167fc700 1 -- 172.21.15.162:0/4086268460 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 184762063 0 0) 0x7fbd1805baf0 con 0x7fbd180b3ab0 2020-11-28T21:46:46.659 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.643+0000 7fbd167fc700 1 -- 172.21.15.162:0/4086268460 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcf8003230 con 0x7fbd180b3ab0 2020-11-28T21:46:46.659 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.647+0000 7fbd177fe700 1 -- 172.21.15.162:0/4086268460 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbd180b30e0 msgr2=0x7fbd1813fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:46.659 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.647+0000 7fbd177fe700 1 --2- 172.21.15.162:0/4086268460 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbd180b30e0 0x7fbd1813fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:46.660 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.647+0000 7fbd177fe700 1 -- 172.21.15.162:0/4086268460 >> v1:172.21.15.162:6789/0 conn(0x7fbd180b3ab0 legacy=0x7fbd18142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:46.660 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.647+0000 7fbd167fc700 1 -- 172.21.15.162:0/4086268460 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1131744980 0 0) 0x7fbcf8003230 con 0x7fbd180b3ab0 2020-11-28T21:46:46.660 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.647+0000 7fbd177fe700 1 -- 172.21.15.162:0/4086268460 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd1805c030 con 0x7fbd180b2d00 2020-11-28T21:46:46.661 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.647+0000 7fbd177fe700 1 --2- 172.21.15.162:0/4086268460 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd180b2d00 0x7fbd1805c4d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7fbd00001870 tx=0x7fbd08006dc0).ready entity=mon.1 client_cookie=6a4208ff81b30d98 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:46.661 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.647+0000 7fbd167fc700 1 -- 172.21.15.162:0/4086268460 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd000016d0 con 0x7fbd180b2d00 2020-11-28T21:46:46.662 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.647+0000 7fbd167fc700 1 -- 172.21.15.162:0/4086268460 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd080095d0 con 0x7fbd180b2d00 2020-11-28T21:46:46.662 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.647+0000 7fbd167fc700 1 -- 172.21.15.162:0/4086268460 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd08009740 con 0x7fbd180b2d00 2020-11-28T21:46:46.662 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.647+0000 7fbd1c87f700 1 -- 172.21.15.162:0/4086268460 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd180b2d00 msgr2=0x7fbd1805c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:46.663 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.647+0000 7fbd1c87f700 1 --2- 172.21.15.162:0/4086268460 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd180b2d00 0x7fbd1805c4d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7fbd00001870 tx=0x7fbd08006dc0).stop 2020-11-28T21:46:46.663 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.647+0000 7fbd1c87f700 1 -- 172.21.15.162:0/4086268460 shutdown_connections 2020-11-28T21:46:46.664 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.647+0000 7fbd1c87f700 1 --2- 172.21.15.162:0/4086268460 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd180b2d00 0x7fbd1805c4d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:46.664 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.647+0000 7fbd1c87f700 1 --2- 172.21.15.162:0/4086268460 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbd180b30e0 0x7fbd1813fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:46.664 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.647+0000 7fbd1c87f700 1 -- 172.21.15.162:0/4086268460 >> 172.21.15.162:0/4086268460 conn(0x7fbd180604c0 msgr2=0x7fbd180608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:46.665 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.647+0000 7fbd1c87f700 1 -- 172.21.15.162:0/4086268460 shutdown_connections 2020-11-28T21:46:46.665 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.647+0000 7fbd1c87f700 1 -- 172.21.15.162:0/4086268460 wait complete. 2020-11-28T21:46:46.665 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.647+0000 7fbd1c87f700 1 Processor -- start 2020-11-28T21:46:46.665 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.647+0000 7fbd1c87f700 1 -- start start 2020-11-28T21:46:46.666 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.651+0000 7fbd1c87f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd180b2d00 0x7fbd18145470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:46.666 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.651+0000 7fbd1c87f700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbd180b30e0 0x7fbd18140720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:46.666 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.651+0000 7fbd1c87f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbd180b3ab0 0x7fbd18142960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:46.666 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.651+0000 7fbd1c87f700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd180b9980 con 0x7fbd180b30e0 2020-11-28T21:46:46.667 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.651+0000 7fbd1c87f700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd180b9bb0 con 0x7fbd180b2d00 2020-11-28T21:46:46.667 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.651+0000 7fbd1c87f700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd180b9e50 con 0x7fbd180b3ab0 2020-11-28T21:46:46.667 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.651+0000 7fbd177fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd180b2d00 0x7fbd18145470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:46.667 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.651+0000 7fbd177fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd180b2d00 0x7fbd18145470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37906/0 (socket says 172.21.15.162:37906) 2020-11-28T21:46:46.668 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.651+0000 7fbd177fe700 1 -- 172.21.15.162:0/3794969863 learned_addr learned my addr 172.21.15.162:0/3794969863 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:46.668 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.651+0000 7fbd17fff700 1 --2- 172.21.15.162:0/3794969863 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbd180b3ab0 0x7fbd18142960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:46.668 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.651+0000 7fbd16ffd700 1 --2- 172.21.15.162:0/3794969863 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbd180b30e0 0x7fbd18140720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:46.669 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.651+0000 7fbd177fe700 1 -- 172.21.15.162:0/3794969863 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbd180b3ab0 msgr2=0x7fbd18142960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:46.669 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.651+0000 7fbd177fe700 1 --2- 172.21.15.162:0/3794969863 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbd180b3ab0 0x7fbd18142960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:46.670 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.651+0000 7fbd177fe700 1 -- 172.21.15.162:0/3794969863 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbd180b30e0 msgr2=0x7fbd18140720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:46.670 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.651+0000 7fbd177fe700 1 --2- 172.21.15.162:0/3794969863 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbd180b30e0 0x7fbd18140720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:46.670 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.651+0000 7fbd177fe700 1 -- 172.21.15.162:0/3794969863 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd181424c0 con 0x7fbd180b2d00 2020-11-28T21:46:46.670 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.651+0000 7fbd177fe700 1 --2- 172.21.15.162:0/3794969863 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd180b2d00 0x7fbd18145470 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7fbd00001870 tx=0x7fbd08015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:46.671 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.651+0000 7fbd14ff9700 1 -- 172.21.15.162:0/3794969863 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd080119a0 con 0x7fbd180b2d00 2020-11-28T21:46:46.671 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.651+0000 7fbd14ff9700 1 -- 172.21.15.162:0/3794969863 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd08011b00 con 0x7fbd180b2d00 2020-11-28T21:46:46.671 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.651+0000 7fbd14ff9700 1 -- 172.21.15.162:0/3794969863 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd08011e60 con 0x7fbd180b2d00 2020-11-28T21:46:46.671 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.651+0000 7fbd1c87f700 1 -- 172.21.15.162:0/3794969863 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd180b9620 con 0x7fbd180b2d00 2020-11-28T21:46:46.671 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.651+0000 7fbd1c87f700 1 -- 172.21.15.162:0/3794969863 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd180b83a0 con 0x7fbd180b2d00 2020-11-28T21:46:46.672 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.651+0000 7fbd1c87f700 1 -- 172.21.15.162:0/3794969863 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd180b8f80 con 0x7fbd180b2d00 2020-11-28T21:46:46.673 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.655+0000 7fbd14ff9700 1 -- 172.21.15.162:0/3794969863 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fbd08017070 con 0x7fbd180b2d00 2020-11-28T21:46:46.673 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.655+0000 7fbd14ff9700 1 --2- 172.21.15.162:0/3794969863 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbcfc055330 0x7fbcfc0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:46.673 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.655+0000 7fbd14ff9700 1 -- 172.21.15.162:0/3794969863 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbd0806eaf0 con 0x7fbd180b2d00 2020-11-28T21:46:46.673 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.655+0000 7fbd16ffd700 1 --2- 172.21.15.162:0/3794969863 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbcfc055330 0x7fbcfc0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:46.673 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.655+0000 7fbd16ffd700 1 --2- 172.21.15.162:0/3794969863 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbcfc055330 0x7fbcfc0577f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fbd0000d000 tx=0x7fbd00001660).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:46.674 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.663+0000 7fbd14ff9700 1 -- 172.21.15.162:0/3794969863 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fbd180b8f80 con 0x7fbd180b2d00 2020-11-28T21:46:46.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.799+0000 7fbd1c87f700 1 -- 172.21.15.162:0/3794969863 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbd180b8f80 con 0x7fbd180b2d00 2020-11-28T21:46:46.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.799+0000 7fbd14ff9700 1 -- 172.21.15.162:0/3794969863 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+10 (secure 0 0 0) 0x7fbd180b8f80 con 0x7fbd180b2d00 2020-11-28T21:46:46.803 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:46:46.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.803+0000 7fbd1c87f700 1 -- 172.21.15.162:0/3794969863 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbcfc055330 msgr2=0x7fbcfc0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:46.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.803+0000 7fbd1c87f700 1 --2- 172.21.15.162:0/3794969863 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbcfc055330 0x7fbcfc0577f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fbd0000d000 tx=0x7fbd00001660).stop 2020-11-28T21:46:46.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.803+0000 7fbd1c87f700 1 -- 172.21.15.162:0/3794969863 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd180b2d00 msgr2=0x7fbd18145470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:46.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.803+0000 7fbd1c87f700 1 --2- 172.21.15.162:0/3794969863 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd180b2d00 0x7fbd18145470 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7fbd00001870 tx=0x7fbd08015000).stop 2020-11-28T21:46:46.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.803+0000 7fbd1c87f700 1 -- 172.21.15.162:0/3794969863 shutdown_connections 2020-11-28T21:46:46.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.803+0000 7fbd1c87f700 1 --2- 172.21.15.162:0/3794969863 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbcfc055330 0x7fbcfc0577f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:46.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.803+0000 7fbd1c87f700 1 --2- 172.21.15.162:0/3794969863 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbd180b3ab0 0x7fbd18142960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:46.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.803+0000 7fbd1c87f700 1 --2- 172.21.15.162:0/3794969863 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd180b2d00 0x7fbd18145470 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:46.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.803+0000 7fbd1c87f700 1 --2- 172.21.15.162:0/3794969863 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbd180b30e0 0x7fbd18140720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:46.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.803+0000 7fbd1c87f700 1 -- 172.21.15.162:0/3794969863 >> 172.21.15.162:0/3794969863 conn(0x7fbd180604c0 msgr2=0x7fbd180b4710 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:46.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.803+0000 7fbd1c87f700 1 -- 172.21.15.162:0/3794969863 shutdown_connections 2020-11-28T21:46:46.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.803+0000 7fbd1c87f700 1 -- 172.21.15.162:0/3794969863 wait complete. 2020-11-28T21:46:46.818 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-28T21:46:46.819 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:46:46.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.927+0000 7fdb51941700 1 Processor -- start 2020-11-28T21:46:46.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.927+0000 7fdb51941700 1 -- start start 2020-11-28T21:46:46.933 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.927+0000 7fdb51941700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fdb4c142980 0x7fdb4c1402f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:46.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.927+0000 7fdb51941700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb4c05f380 con 0x7fdb4c0b5bf0 2020-11-28T21:46:46.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.927+0000 7fdb51941700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb4c143930 con 0x7fdb4c060cb0 2020-11-28T21:46:46.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.927+0000 7fdb51941700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb4c05f5b0 con 0x7fdb4c142980 2020-11-28T21:46:46.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.927+0000 7fdb4bfff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fdb4c060cb0 0x7fdb4c13fbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:44760/0 (socket says 172.21.15.162:44760) 2020-11-28T21:46:46.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.927+0000 7fdb4bfff700 1 -- 172.21.15.162:0/3419133456 learned_addr learned my addr 172.21.15.162:0/3419133456 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:46.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.927+0000 7fdb51140700 1 --2- 172.21.15.162:0/3419133456 >> v2:172.21.15.118:3300/0 conn(0x7fdb4c142980 0x7fdb4c1402f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:46.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.927+0000 7fdb4b7fe700 1 -- 172.21.15.162:0/3419133456 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3832622285 0 0) 0x7fdb400010d0 con 0x7fdb4c060cb0 2020-11-28T21:46:46.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.927+0000 7fdb4b7fe700 1 -- 172.21.15.162:0/3419133456 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb38003230 con 0x7fdb4c060cb0 2020-11-28T21:46:46.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.927+0000 7fdb51140700 1 -- 172.21.15.162:0/3419133456 >> v1:172.21.15.162:6789/0 conn(0x7fdb4c060cb0 legacy=0x7fdb4c13fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:46.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.927+0000 7fdb51140700 1 -- 172.21.15.162:0/3419133456 >> v1:172.21.15.118:6789/0 conn(0x7fdb4c0b5bf0 legacy=0x7fdb4c060230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:46.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.927+0000 7fdb51140700 1 -- 172.21.15.162:0/3419133456 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb4c05f850 con 0x7fdb4c142980 2020-11-28T21:46:46.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.927+0000 7fdb51140700 1 --2- 172.21.15.162:0/3419133456 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdb4c142980 0x7fdb4c1402f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fdb3c002f30 tx=0x7fdb3c006dc0).ready entity=mon.0 client_cookie=bc3949383909e218 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:46.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.927+0000 7fdb4b7fe700 1 -- 172.21.15.162:0/3419133456 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb3c009800 con 0x7fdb4c142980 2020-11-28T21:46:46.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.927+0000 7fdb4b7fe700 1 -- 172.21.15.162:0/3419133456 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb3c009960 con 0x7fdb4c142980 2020-11-28T21:46:46.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.927+0000 7fdb4b7fe700 1 -- 172.21.15.162:0/3419133456 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb3c009cc0 con 0x7fdb4c142980 2020-11-28T21:46:46.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.927+0000 7fdb51941700 1 -- 172.21.15.162:0/3419133456 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdb4c142980 msgr2=0x7fdb4c1402f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:46.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.927+0000 7fdb51941700 1 --2- 172.21.15.162:0/3419133456 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdb4c142980 0x7fdb4c1402f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fdb3c002f30 tx=0x7fdb3c006dc0).stop 2020-11-28T21:46:46.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.931+0000 7fdb51941700 1 -- 172.21.15.162:0/3419133456 shutdown_connections 2020-11-28T21:46:46.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.931+0000 7fdb51941700 1 --2- 172.21.15.162:0/3419133456 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdb4c142980 0x7fdb4c1402f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:46.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.931+0000 7fdb51941700 1 -- 172.21.15.162:0/3419133456 >> 172.21.15.162:0/3419133456 conn(0x7fdb4c05b970 msgr2=0x7fdb4c05bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:46.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.931+0000 7fdb51941700 1 -- 172.21.15.162:0/3419133456 shutdown_connections 2020-11-28T21:46:46.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.931+0000 7fdb51941700 1 -- 172.21.15.162:0/3419133456 wait complete. 2020-11-28T21:46:46.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.931+0000 7fdb51941700 1 Processor -- start 2020-11-28T21:46:46.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.931+0000 7fdb51941700 1 -- start start 2020-11-28T21:46:46.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.931+0000 7fdb51941700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdb4c060cb0 0x7fdb4c148f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:46.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.931+0000 7fdb51941700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdb4c0b5bf0 0x7fdb4c13ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:46.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.931+0000 7fdb51941700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdb4c142980 0x7fdb4c13f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:46.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.931+0000 7fdb51941700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c0b2c40 con 0x7fdb4c142980 2020-11-28T21:46:46.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.931+0000 7fdb51941700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c1442e0 con 0x7fdb4c0b5bf0 2020-11-28T21:46:46.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.931+0000 7fdb51941700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c0b1b20 con 0x7fdb4c060cb0 2020-11-28T21:46:46.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.931+0000 7fdb4bfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdb4c0b5bf0 0x7fdb4c13ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:46.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.931+0000 7fdb4bfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdb4c0b5bf0 0x7fdb4c13ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37922/0 (socket says 172.21.15.162:37922) 2020-11-28T21:46:46.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.931+0000 7fdb4bfff700 1 -- 172.21.15.162:0/510889203 learned_addr learned my addr 172.21.15.162:0/510889203 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:46.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.931+0000 7fdb51140700 1 --2- 172.21.15.162:0/510889203 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdb4c142980 0x7fdb4c13f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:46.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.931+0000 7fdb5093f700 1 --2- 172.21.15.162:0/510889203 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdb4c060cb0 0x7fdb4c148f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:46.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.931+0000 7fdb4bfff700 1 -- 172.21.15.162:0/510889203 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdb4c060cb0 msgr2=0x7fdb4c148f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:46.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.931+0000 7fdb4bfff700 1 --2- 172.21.15.162:0/510889203 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdb4c060cb0 0x7fdb4c148f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:46.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.931+0000 7fdb4bfff700 1 -- 172.21.15.162:0/510889203 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdb4c142980 msgr2=0x7fdb4c13f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:46.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.931+0000 7fdb4bfff700 1 --2- 172.21.15.162:0/510889203 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdb4c142980 0x7fdb4c13f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:46.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.931+0000 7fdb4bfff700 1 -- 172.21.15.162:0/510889203 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb4c13f690 con 0x7fdb4c0b5bf0 2020-11-28T21:46:46.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.931+0000 7fdb4bfff700 1 --2- 172.21.15.162:0/510889203 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdb4c0b5bf0 0x7fdb4c13ecb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fdb40001260 tx=0x7fdb400078d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:46.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.931+0000 7fdb49ffb700 1 -- 172.21.15.162:0/510889203 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb400072e0 con 0x7fdb4c0b5bf0 2020-11-28T21:46:46.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.931+0000 7fdb49ffb700 1 -- 172.21.15.162:0/510889203 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb40007440 con 0x7fdb4c0b5bf0 2020-11-28T21:46:46.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.931+0000 7fdb51941700 1 -- 172.21.15.162:0/510889203 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb4c1467b0 con 0x7fdb4c0b5bf0 2020-11-28T21:46:46.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.931+0000 7fdb49ffb700 1 -- 172.21.15.162:0/510889203 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb400057d0 con 0x7fdb4c0b5bf0 2020-11-28T21:46:46.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.931+0000 7fdb51941700 1 -- 172.21.15.162:0/510889203 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb4c1448b0 con 0x7fdb4c0b5bf0 2020-11-28T21:46:46.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.935+0000 7fdb51941700 1 -- 172.21.15.162:0/510889203 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb4c0b2fb0 con 0x7fdb4c0b5bf0 2020-11-28T21:46:46.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.935+0000 7fdb49ffb700 1 -- 172.21.15.162:0/510889203 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fdb40010070 con 0x7fdb4c0b5bf0 2020-11-28T21:46:46.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.935+0000 7fdb49ffb700 1 --2- 172.21.15.162:0/510889203 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fdb2c0553b0 0x7fdb2c057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:46.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.935+0000 7fdb49ffb700 1 -- 172.21.15.162:0/510889203 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdb4006d400 con 0x7fdb4c0b5bf0 2020-11-28T21:46:46.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.935+0000 7fdb5093f700 1 --2- 172.21.15.162:0/510889203 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fdb2c0553b0 0x7fdb2c057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:46.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.935+0000 7fdb5093f700 1 --2- 172.21.15.162:0/510889203 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fdb2c0553b0 0x7fdb2c057870 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fdb4400b000 tx=0x7fdb44009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:46.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:46.939+0000 7fdb49ffb700 1 -- 172.21.15.162:0/510889203 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fdb40043700 con 0x7fdb4c0b5bf0 2020-11-28T21:46:47.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:47.123+0000 7fdb51941700 1 -- 172.21.15.162:0/510889203 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb4c143930 con 0x7fdb4c0b5bf0 2020-11-28T21:46:47.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:47.123+0000 7fdb49ffb700 1 -- 172.21.15.162:0/510889203 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2495 (secure 0 0 0) 0x7fdb4c143930 con 0x7fdb4c0b5bf0 2020-11-28T21:46:47.128 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:46:47.128 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/1297721315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1297721315},{"type":"v1","addr":"172.21.15.118:6833","nonce":1297721315}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3},{"gid":4264,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2415281381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2415281381},{"type":"v1","addr":"172.21.15.162:6837","nonce":2415281381}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:46:29.120502+0000","modified":"2020-11-28T21:46:30.123428+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.162:6835/3696705934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3696705934},{"type":"v1","addr":"172.21.15.162:6835","nonce":3696705934}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-28T21:46:47.129 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:47.127+0000 7fdb51941700 1 -- 172.21.15.162:0/510889203 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fdb2c0553b0 msgr2=0x7fdb2c057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:47.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:47.127+0000 7fdb51941700 1 --2- 172.21.15.162:0/510889203 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fdb2c0553b0 0x7fdb2c057870 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fdb4400b000 tx=0x7fdb44009040).stop 2020-11-28T21:46:47.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:47.127+0000 7fdb51941700 1 -- 172.21.15.162:0/510889203 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdb4c0b5bf0 msgr2=0x7fdb4c13ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:47.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:47.127+0000 7fdb51941700 1 --2- 172.21.15.162:0/510889203 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdb4c0b5bf0 0x7fdb4c13ecb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fdb40001260 tx=0x7fdb400078d0).stop 2020-11-28T21:46:47.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:47.127+0000 7fdb51941700 1 -- 172.21.15.162:0/510889203 shutdown_connections 2020-11-28T21:46:47.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:47.127+0000 7fdb51941700 1 --2- 172.21.15.162:0/510889203 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fdb2c0553b0 0x7fdb2c057870 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:47.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:47.127+0000 7fdb51941700 1 --2- 172.21.15.162:0/510889203 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdb4c060cb0 0x7fdb4c148f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:47.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:47.127+0000 7fdb51941700 1 --2- 172.21.15.162:0/510889203 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdb4c142980 0x7fdb4c13f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:47.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:47.127+0000 7fdb51941700 1 --2- 172.21.15.162:0/510889203 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdb4c0b5bf0 0x7fdb4c13ecb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:47.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:47.127+0000 7fdb51941700 1 -- 172.21.15.162:0/510889203 >> 172.21.15.162:0/510889203 conn(0x7fdb4c05b970 msgr2=0x7fdb4c141350 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:47.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:47.127+0000 7fdb51941700 1 -- 172.21.15.162:0/510889203 shutdown_connections 2020-11-28T21:46:47.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:47.127+0000 7fdb51941700 1 -- 172.21.15.162:0/510889203 wait complete. 2020-11-28T21:46:47.132 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 5 2020-11-28T21:46:47.449 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 5, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-28T21:46:29.120502+0000', 'modified': '2020-11-28T21:46:30.123428+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4284}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4284': {'gid': 4284, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.15.162:6835/3696705934', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6834', 'nonce': 3696705934}, {'type': 'v1', 'addr': '172.21.15.162:6835', 'nonce': 3696705934}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-28T21:46:47.449 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-28T21:46:47.450 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-28T21:46:47.580 INFO:teuthology.orchestra.run.smithi162.stdout:{"cluster_fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","whoami":0,"id":4284,"want_state":"up:active","state":"up:active","rank_uptime":18.445800253000002,"mdsmap_epoch":5,"osdmap_epoch":23,"osdmap_epoch_barrier":22,"uptime":21.824171157999999} 2020-11-28T21:46:47.581 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", "id": 4284, "mdsmap_epoch": 5, "osdmap_epoch": 23, "osdmap_epoch_barrier": 22, "rank_uptime": 18.445800253, "state": "up:active", "uptime": 21.824171158, "want_state": "up:active", "whoami": 0 } 2020-11-28T21:46:47.581 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-11-28T21:46:47.582 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2020-11-28T21:46:47.582 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi096.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2020-11-28T21:46:47.583 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:46:47.583 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:46:47.592 INFO:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2020-11-28T21:46:47.650 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:46:47.651 INFO:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:46:47.709 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:46:47.710 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:46:47.710 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:46:47.711 INFO:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2020-11-28T21:46:47.756 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-11-28T21:46:47.757 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:46:47.800 INFO:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2020-11-28T21:46:47.816 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:46:47.816 INFO:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:46:47.878 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:46:47.879 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:46:47.880 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:46:47.880 INFO:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2020-11-28T21:46:47.884 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:2020-11-28T21:46:47.883+0000 7f31ca8bd0c0 -1 init, newargv = 0x562f862ecd50 newargc=9 2020-11-28T21:46:47.884 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[11365]: starting ceph client 2020-11-28T21:46:47.896 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[11365]: starting fuse 2020-11-28T21:46:48.990 INFO:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2020-11-28T21:46:49.006 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:46:49.006 INFO:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:46:49.061 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:46:49.062 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:46:49.062 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:46:49.063 INFO:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2020-11-28T21:46:49.109 INFO:teuthology.orchestra.run.smithi096.stdout:49 2020-11-28T21:46:49.109 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2020-11-28T21:46:49.109 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:46:49.110 INFO:teuthology.orchestra.run.smithi096:> sudo python3 -c ' 2020-11-28T21:46:49.110 INFO:teuthology.orchestra.run.smithi096:> import glob 2020-11-28T21:46:49.110 INFO:teuthology.orchestra.run.smithi096:> import re 2020-11-28T21:46:49.111 INFO:teuthology.orchestra.run.smithi096:> import os 2020-11-28T21:46:49.111 INFO:teuthology.orchestra.run.smithi096:> import subprocess 2020-11-28T21:46:49.111 INFO:teuthology.orchestra.run.smithi096:> 2020-11-28T21:46:49.112 INFO:teuthology.orchestra.run.smithi096:> def find_socket(client_name): 2020-11-28T21:46:49.112 INFO:teuthology.orchestra.run.smithi096:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2020-11-28T21:46:49.113 INFO:teuthology.orchestra.run.smithi096:> files = glob.glob(asok_path) 2020-11-28T21:46:49.113 INFO:teuthology.orchestra.run.smithi096:> 2020-11-28T21:46:49.113 INFO:teuthology.orchestra.run.smithi096:> # Given a non-glob path, it better be there 2020-11-28T21:46:49.114 INFO:teuthology.orchestra.run.smithi096:> if "*" not in asok_path: 2020-11-28T21:46:49.114 INFO:teuthology.orchestra.run.smithi096:> assert(len(files) == 1) 2020-11-28T21:46:49.114 INFO:teuthology.orchestra.run.smithi096:> return files[0] 2020-11-28T21:46:49.115 INFO:teuthology.orchestra.run.smithi096:> 2020-11-28T21:46:49.115 INFO:teuthology.orchestra.run.smithi096:> for f in files: 2020-11-28T21:46:49.115 INFO:teuthology.orchestra.run.smithi096:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-11-28T21:46:49.116 INFO:teuthology.orchestra.run.smithi096:> if os.path.exists("/proc/{0}".format(pid)): 2020-11-28T21:46:49.116 INFO:teuthology.orchestra.run.smithi096:> return f 2020-11-28T21:46:49.116 INFO:teuthology.orchestra.run.smithi096:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-11-28T21:46:49.116 INFO:teuthology.orchestra.run.smithi096:> 2020-11-28T21:46:49.117 INFO:teuthology.orchestra.run.smithi096:> print(find_socket("client.1")) 2020-11-28T21:46:49.117 INFO:teuthology.orchestra.run.smithi096:> ' 2020-11-28T21:46:49.200 INFO:teuthology.orchestra.run.smithi096.stdout:/var/run/ceph/ceph-client.1.11365.asok 2020-11-28T21:46:49.205 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.11365.asok 2020-11-28T21:46:49.206 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:46:49.206 INFO:teuthology.orchestra.run.smithi096:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.11365.asok status 2020-11-28T21:46:49.702 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-11-28T21:46:49.703 INFO:teuthology.orchestra.run.smithi096.stdout: "metadata": { 2020-11-28T21:46:49.703 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_sha1": "a83a3572a957356977342f461361c1cd3ec06dbd", 2020-11-28T21:46:49.703 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_version": "ceph version 15.2.6-595-ga83a3572 (a83a3572a957356977342f461361c1cd3ec06dbd) octopus (stable)", 2020-11-28T21:46:49.704 INFO:teuthology.orchestra.run.smithi096.stdout: "entity_id": "1", 2020-11-28T21:46:49.705 INFO:teuthology.orchestra.run.smithi096.stdout: "hostname": "smithi096", 2020-11-28T21:46:49.705 INFO:teuthology.orchestra.run.smithi096.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2020-11-28T21:46:49.705 INFO:teuthology.orchestra.run.smithi096.stdout: "pid": "11365", 2020-11-28T21:46:49.706 INFO:teuthology.orchestra.run.smithi096.stdout: "root": "/" 2020-11-28T21:46:49.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-11-28T21:46:49.707 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_count": 0, 2020-11-28T21:46:49.707 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_pinned_count": 0, 2020-11-28T21:46:49.707 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 4651, 2020-11-28T21:46:49.709 INFO:teuthology.orchestra.run.smithi096.stdout: "inst": { 2020-11-28T21:46:49.710 INFO:teuthology.orchestra.run.smithi096.stdout: "name": { 2020-11-28T21:46:49.710 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "client", 2020-11-28T21:46:49.710 INFO:teuthology.orchestra.run.smithi096.stdout: "num": 4651 2020-11-28T21:46:49.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-11-28T21:46:49.711 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2020-11-28T21:46:49.711 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2020-11-28T21:46:49.711 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2020-11-28T21:46:49.711 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3692518311 2020-11-28T21:46:49.711 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-11-28T21:46:49.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-11-28T21:46:49.712 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2020-11-28T21:46:49.712 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2020-11-28T21:46:49.712 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2020-11-28T21:46:49.712 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3692518311 2020-11-28T21:46:49.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-11-28T21:46:49.713 INFO:teuthology.orchestra.run.smithi096.stdout: "inst_str": "client.4651 172.21.15.96:0/3692518311", 2020-11-28T21:46:49.713 INFO:teuthology.orchestra.run.smithi096.stdout: "addr_str": "172.21.15.96:0/3692518311", 2020-11-28T21:46:49.713 INFO:teuthology.orchestra.run.smithi096.stdout: "inode_count": 1, 2020-11-28T21:46:49.713 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_epoch": 5, 2020-11-28T21:46:49.714 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch": 23, 2020-11-28T21:46:49.714 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch_barrier": 0, 2020-11-28T21:46:49.714 INFO:teuthology.orchestra.run.smithi096.stdout: "blacklisted": false 2020-11-28T21:46:49.714 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-11-28T21:46:49.715 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:46:49.716 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:46:49.716 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:46:49.724 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2020-11-28T21:46:49.725 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-28T21:46:49.725 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:46:49.726 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:46:49.784 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:46:49.784 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:46:49.785 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:46:49.793 INFO:teuthology.orchestra.run.smithi096.stdout:fuseblk 2020-11-28T21:46:49.794 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-11-28T21:46:49.794 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:46:49.795 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:46:49.859 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2020-11-28T21:46:49.877 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:46:50.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:49.999+0000 7f93d48f7700 1 Processor -- start 2020-11-28T21:46:50.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:49.999+0000 7f93d48f7700 1 -- start start 2020-11-28T21:46:50.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:49.999+0000 7f93d48f7700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f93d00611e0 0x7f93d00615c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:50.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:49.999+0000 7f93d48f7700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f93d00b10f0 0x7f93d00b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:50.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:49.999+0000 7f93d48f7700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f93d00b4920 0x7f93d00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:50.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:49.999+0000 7f93d48f7700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93d00b1580 con 0x7f93d00611e0 2020-11-28T21:46:50.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:49.999+0000 7f93d48f7700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93d00b1790 con 0x7f93d00b10f0 2020-11-28T21:46:50.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:49.999+0000 7f93d48f7700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93d00b1a30 con 0x7f93d00b4920 2020-11-28T21:46:50.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:49.999+0000 7f93c659c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f93d00b10f0 0x7f93d00b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:50.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:49.999+0000 7f93c659c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f93d00b10f0 0x7f93d00b4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37930/0 (socket says 172.21.15.162:37930) 2020-11-28T21:46:50.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:49.999+0000 7f93c659c700 1 -- 172.21.15.162:0/952960152 learned_addr learned my addr 172.21.15.162:0/952960152 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:50.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:49.999+0000 7f93ced9d700 1 --2- 172.21.15.162:0/952960152 >> v2:172.21.15.118:3300/0 conn(0x7f93d00611e0 0x7f93d00615c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:50.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:49.999+0000 7f93cf59e700 1 --2- 172.21.15.162:0/952960152 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f93d00b4920 0x7f93d00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:50.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:49.999+0000 7f93c659c700 1 -- 172.21.15.162:0/952960152 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f93d00b4920 msgr2=0x7f93d00b4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:50.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:49.999+0000 7f93c659c700 1 --2- 172.21.15.162:0/952960152 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f93d00b4920 0x7f93d00b4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:50.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:49.999+0000 7f93c659c700 1 -- 172.21.15.162:0/952960152 >> v2:172.21.15.118:3300/0 conn(0x7f93d00611e0 msgr2=0x7f93d00615c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:50.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:49.999+0000 7f93c659c700 1 --2- 172.21.15.162:0/952960152 >> v2:172.21.15.118:3300/0 conn(0x7f93d00611e0 0x7f93d00615c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:50.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:49.999+0000 7f93c659c700 1 -- 172.21.15.162:0/952960152 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d00b3f90 con 0x7f93d00b10f0 2020-11-28T21:46:50.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:49.999+0000 7f93cf59e700 1 --2- 172.21.15.162:0/952960152 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f93d00b4920 0x7f93d00b4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:46:50.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:49.999+0000 7f93c659c700 1 --2- 172.21.15.162:0/952960152 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93d00b10f0 0x7f93d00b4430 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f93b8002f30 tx=0x7f93b8006f10).ready entity=mon.1 client_cookie=308807a2a33b118d server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:50.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.003+0000 7f93ce59c700 1 -- 172.21.15.162:0/952960152 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93b800de50 con 0x7f93d00b10f0 2020-11-28T21:46:50.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.003+0000 7f93ce59c700 1 -- 172.21.15.162:0/952960152 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b8010470 con 0x7f93d00b10f0 2020-11-28T21:46:50.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.003+0000 7f93ce59c700 1 -- 172.21.15.162:0/952960152 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93b80107d0 con 0x7f93d00b10f0 2020-11-28T21:46:50.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.003+0000 7f93d48f7700 1 -- 172.21.15.162:0/952960152 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93d00b10f0 msgr2=0x7f93d00b4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:50.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.003+0000 7f93d48f7700 1 --2- 172.21.15.162:0/952960152 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93d00b10f0 0x7f93d00b4430 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f93b8002f30 tx=0x7f93b8006f10).stop 2020-11-28T21:46:50.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.003+0000 7f93d48f7700 1 -- 172.21.15.162:0/952960152 shutdown_connections 2020-11-28T21:46:50.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.003+0000 7f93d48f7700 1 --2- 172.21.15.162:0/952960152 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f93d00b4920 0x7f93d00b4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:50.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.003+0000 7f93d48f7700 1 --2- 172.21.15.162:0/952960152 >> v2:172.21.15.118:3300/0 conn(0x7f93d00611e0 0x7f93d00615c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:50.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.003+0000 7f93d48f7700 1 --2- 172.21.15.162:0/952960152 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93d00b10f0 0x7f93d00b4430 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:50.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.003+0000 7f93d48f7700 1 -- 172.21.15.162:0/952960152 >> 172.21.15.162:0/952960152 conn(0x7f93d00b54f0 msgr2=0x7f93d00b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:50.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.003+0000 7f93d48f7700 1 -- 172.21.15.162:0/952960152 shutdown_connections 2020-11-28T21:46:50.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.003+0000 7f93d48f7700 1 -- 172.21.15.162:0/952960152 wait complete. 2020-11-28T21:46:50.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.003+0000 7f93d48f7700 1 Processor -- start 2020-11-28T21:46:50.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.003+0000 7f93d48f7700 1 -- start start 2020-11-28T21:46:50.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.003+0000 7f93d48f7700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93d00611e0 0x7f93d009af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:50.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.003+0000 7f93d48f7700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f93d00b10f0 0x7f93d0143da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:50.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.003+0000 7f93d48f7700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93d00b4920 0x7f93d0066b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:50.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.003+0000 7f93d48f7700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d0154f60 con 0x7f93d00b4920 2020-11-28T21:46:50.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.003+0000 7f93d48f7700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d0155200 con 0x7f93d00611e0 2020-11-28T21:46:50.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.003+0000 7f93d48f7700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93d01554a0 con 0x7f93d00b10f0 2020-11-28T21:46:50.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.003+0000 7f93ced9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93d00611e0 0x7f93d009af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:50.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.003+0000 7f93ced9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93d00611e0 0x7f93d009af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37934/0 (socket says 172.21.15.162:37934) 2020-11-28T21:46:50.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.003+0000 7f93ced9d700 1 -- 172.21.15.162:0/910902171 learned_addr learned my addr 172.21.15.162:0/910902171 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:50.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.007+0000 7f93cf59e700 1 --2- 172.21.15.162:0/910902171 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93d00b4920 0x7f93d0066b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:50.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.007+0000 7f93ced9d700 1 -- 172.21.15.162:0/910902171 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f93d00b10f0 msgr2=0x7f93d0143da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:50.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.007+0000 7f93c659c700 1 --2- 172.21.15.162:0/910902171 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f93d00b10f0 0x7f93d0143da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:50.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.007+0000 7f93ced9d700 1 --2- 172.21.15.162:0/910902171 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f93d00b10f0 0x7f93d0143da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:50.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.007+0000 7f93ced9d700 1 -- 172.21.15.162:0/910902171 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93d00b4920 msgr2=0x7f93d0066b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:50.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.007+0000 7f93ced9d700 1 --2- 172.21.15.162:0/910902171 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93d00b4920 0x7f93d0066b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:50.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.007+0000 7f93ced9d700 1 -- 172.21.15.162:0/910902171 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d0155740 con 0x7f93d00611e0 2020-11-28T21:46:50.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.007+0000 7f93c659c700 1 --2- 172.21.15.162:0/910902171 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f93d00b10f0 0x7f93d0143da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:46:50.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.007+0000 7f93ced9d700 1 --2- 172.21.15.162:0/910902171 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93d00611e0 0x7f93d009af90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f93c0003070 tx=0x7f93c0008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:50.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.007+0000 7f93ccd99700 1 -- 172.21.15.162:0/910902171 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93c0009980 con 0x7f93d00611e0 2020-11-28T21:46:50.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.007+0000 7f93ccd99700 1 -- 172.21.15.162:0/910902171 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93c0009ae0 con 0x7f93d00611e0 2020-11-28T21:46:50.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.007+0000 7f93d48f7700 1 -- 172.21.15.162:0/910902171 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93d00aee70 con 0x7f93d00611e0 2020-11-28T21:46:50.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.007+0000 7f93ccd99700 1 -- 172.21.15.162:0/910902171 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93c0009e40 con 0x7f93d00611e0 2020-11-28T21:46:50.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.007+0000 7f93d48f7700 1 -- 172.21.15.162:0/910902171 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93d0154c70 con 0x7f93d00611e0 2020-11-28T21:46:50.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.007+0000 7f93ccd99700 1 -- 172.21.15.162:0/910902171 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f93c0013480 con 0x7f93d00611e0 2020-11-28T21:46:50.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.007+0000 7f93d48f7700 1 -- 172.21.15.162:0/910902171 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93d01558d0 con 0x7f93d00611e0 2020-11-28T21:46:50.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.007+0000 7f93ccd99700 1 --2- 172.21.15.162:0/910902171 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f93b40553b0 0x7f93b4057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:50.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.007+0000 7f93ccd99700 1 -- 172.21.15.162:0/910902171 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f93c006e5b0 con 0x7f93d00611e0 2020-11-28T21:46:50.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.007+0000 7f93c659c700 1 --2- 172.21.15.162:0/910902171 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f93b40553b0 0x7f93b4057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:50.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.011+0000 7f93c659c700 1 --2- 172.21.15.162:0/910902171 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f93b40553b0 0x7f93b4057870 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f93b8007000 tx=0x7f93b8006f10).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:50.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.015+0000 7f93ccd99700 1 -- 172.21.15.162:0/910902171 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f93c00481b0 con 0x7f93d00611e0 2020-11-28T21:46:50.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.175+0000 7f93d48f7700 1 -- 172.21.15.162:0/910902171 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93d0057050 con 0x7f93d00611e0 2020-11-28T21:46:50.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.179+0000 7f93ccd99700 1 -- 172.21.15.162:0/910902171 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16134 (secure 0 0 0) 0x7f93c0047da0 con 0x7f93d00611e0 2020-11-28T21:46:50.180 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:46:50.181 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":23,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:46:30.124354+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:46:27.155497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-28T21:46:28.137474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:46:50.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.179+0000 7f93d48f7700 1 -- 172.21.15.162:0/910902171 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f93b40553b0 msgr2=0x7f93b4057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:50.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.179+0000 7f93d48f7700 1 --2- 172.21.15.162:0/910902171 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f93b40553b0 0x7f93b4057870 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f93b8007000 tx=0x7f93b8006f10).stop 2020-11-28T21:46:50.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.179+0000 7f93d48f7700 1 -- 172.21.15.162:0/910902171 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93d00611e0 msgr2=0x7f93d009af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:50.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.179+0000 7f93d48f7700 1 --2- 172.21.15.162:0/910902171 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93d00611e0 0x7f93d009af90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f93c0003070 tx=0x7f93c0008040).stop 2020-11-28T21:46:50.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.179+0000 7f93d48f7700 1 -- 172.21.15.162:0/910902171 shutdown_connections 2020-11-28T21:46:50.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.179+0000 7f93d48f7700 1 --2- 172.21.15.162:0/910902171 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f93b40553b0 0x7f93b4057870 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:50.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.179+0000 7f93d48f7700 1 --2- 172.21.15.162:0/910902171 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93d00b4920 0x7f93d0066b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:50.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.179+0000 7f93d48f7700 1 --2- 172.21.15.162:0/910902171 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93d00611e0 0x7f93d009af90 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:50.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.179+0000 7f93d48f7700 1 --2- 172.21.15.162:0/910902171 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f93d00b10f0 0x7f93d0143da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:50.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.179+0000 7f93d48f7700 1 -- 172.21.15.162:0/910902171 >> 172.21.15.162:0/910902171 conn(0x7f93d00b54f0 msgr2=0x7f93d013e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:50.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.179+0000 7f93d48f7700 1 -- 172.21.15.162:0/910902171 shutdown_connections 2020-11-28T21:46:50.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.179+0000 7f93d48f7700 1 -- 172.21.15.162:0/910902171 wait complete. 2020-11-28T21:46:50.200 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-28T21:46:27.155497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-28T21:46:28.137474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-28T21:46:50.201 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:46:50.318 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.315+0000 7fa5d7fff700 1 Processor -- start 2020-11-28T21:46:50.318 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.315+0000 7fa5d7fff700 1 -- start start 2020-11-28T21:46:50.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.315+0000 7fa5d7fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa5d80b3950 0x7fa5d80b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:50.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.315+0000 7fa5d7fff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5d80b1f50 con 0x7fa5d80b44e0 2020-11-28T21:46:50.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.315+0000 7fa5d7fff700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5d8143e90 con 0x7fa5d80b18c0 2020-11-28T21:46:50.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.315+0000 7fa5d7fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d80aab50 con 0x7fa5d80b3950 2020-11-28T21:46:50.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.315+0000 7fa5d67fc700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fa5d80b18c0 0x7fa5d813dd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:44790/0 (socket says 172.21.15.162:44790) 2020-11-28T21:46:50.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.315+0000 7fa5d67fc700 1 -- 172.21.15.162:0/601299058 learned_addr learned my addr 172.21.15.162:0/601299058 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:50.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.315+0000 7fa5d6ffd700 1 --2- 172.21.15.162:0/601299058 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa5d80b3950 0x7fa5d80b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:50.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.319+0000 7fa5d5ffb700 1 -- 172.21.15.162:0/601299058 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 268687610 0 0) 0x7fa5d8143e90 con 0x7fa5d80b18c0 2020-11-28T21:46:50.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.319+0000 7fa5d5ffb700 1 -- 172.21.15.162:0/601299058 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5bc003230 con 0x7fa5d80b18c0 2020-11-28T21:46:50.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.319+0000 7fa5d5ffb700 1 -- 172.21.15.162:0/601299058 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 16104257 0 0) 0x7fa5d80b1f50 con 0x7fa5d80b44e0 2020-11-28T21:46:50.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.319+0000 7fa5d6ffd700 1 -- 172.21.15.162:0/601299058 >> v1:172.21.15.162:6789/0 conn(0x7fa5d80b18c0 legacy=0x7fa5d813dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:50.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.319+0000 7fa5d6ffd700 1 -- 172.21.15.162:0/601299058 >> v1:172.21.15.118:6789/0 conn(0x7fa5d80b44e0 legacy=0x7fa5d80b2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:50.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.319+0000 7fa5d6ffd700 1 -- 172.21.15.162:0/601299058 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d80aa8b0 con 0x7fa5d80b3950 2020-11-28T21:46:50.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.319+0000 7fa5d6ffd700 1 --2- 172.21.15.162:0/601299058 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa5d80b3950 0x7fa5d80b3d30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fa5c8002f30 tx=0x7fa5c8006dc0).ready entity=mon.2 client_cookie=5d49e3a846dc6e7f server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:50.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.319+0000 7fa5d5ffb700 1 -- 172.21.15.162:0/601299058 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5c80097b0 con 0x7fa5d80b3950 2020-11-28T21:46:50.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.319+0000 7fa5d5ffb700 1 -- 172.21.15.162:0/601299058 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c8009910 con 0x7fa5d80b3950 2020-11-28T21:46:50.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.319+0000 7fa5d5ffb700 1 -- 172.21.15.162:0/601299058 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5c8009c70 con 0x7fa5d80b3950 2020-11-28T21:46:50.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.319+0000 7fa5d7fff700 1 -- 172.21.15.162:0/601299058 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa5d80b3950 msgr2=0x7fa5d80b3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:50.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.319+0000 7fa5d7fff700 1 --2- 172.21.15.162:0/601299058 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa5d80b3950 0x7fa5d80b3d30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fa5c8002f30 tx=0x7fa5c8006dc0).stop 2020-11-28T21:46:50.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.319+0000 7fa5d7fff700 1 -- 172.21.15.162:0/601299058 shutdown_connections 2020-11-28T21:46:50.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.319+0000 7fa5d7fff700 1 --2- 172.21.15.162:0/601299058 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa5d80b3950 0x7fa5d80b3d30 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:50.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.319+0000 7fa5d7fff700 1 -- 172.21.15.162:0/601299058 >> 172.21.15.162:0/601299058 conn(0x7fa5d8062630 msgr2=0x7fa5d8062a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:50.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.319+0000 7fa5d7fff700 1 -- 172.21.15.162:0/601299058 shutdown_connections 2020-11-28T21:46:50.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.319+0000 7fa5d7fff700 1 -- 172.21.15.162:0/601299058 wait complete. 2020-11-28T21:46:50.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.319+0000 7fa5d7fff700 1 Processor -- start 2020-11-28T21:46:50.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.319+0000 7fa5d7fff700 1 -- start start 2020-11-28T21:46:50.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.319+0000 7fa5d7fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa5d80b18c0 0x7fa5d80b6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:50.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.319+0000 7fa5d7fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa5d80b3950 0x7fa5d8146cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:50.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.319+0000 7fa5d7fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5d80b44e0 0x7fa5d81471b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:50.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.319+0000 7fa5d7fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d80ae7d0 con 0x7fa5d80b3950 2020-11-28T21:46:50.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.323+0000 7fa5d7fff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d8147e80 con 0x7fa5d80b44e0 2020-11-28T21:46:50.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.323+0000 7fa5d7fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d8147be0 con 0x7fa5d80b18c0 2020-11-28T21:46:50.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.323+0000 7fa5d77fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5d80b44e0 0x7fa5d81471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:50.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.323+0000 7fa5d77fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5d80b44e0 0x7fa5d81471b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37948/0 (socket says 172.21.15.162:37948) 2020-11-28T21:46:50.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.323+0000 7fa5d77fe700 1 -- 172.21.15.162:0/1407057219 learned_addr learned my addr 172.21.15.162:0/1407057219 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:50.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.323+0000 7fa5d67fc700 1 --2- 172.21.15.162:0/1407057219 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa5d80b3950 0x7fa5d8146cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:50.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.323+0000 7fa5d6ffd700 1 --2- 172.21.15.162:0/1407057219 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa5d80b18c0 0x7fa5d80b6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:50.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.323+0000 7fa5d77fe700 1 -- 172.21.15.162:0/1407057219 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa5d80b18c0 msgr2=0x7fa5d80b6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:50.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.323+0000 7fa5d77fe700 1 --2- 172.21.15.162:0/1407057219 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa5d80b18c0 0x7fa5d80b6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:50.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.323+0000 7fa5d77fe700 1 -- 172.21.15.162:0/1407057219 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa5d80b3950 msgr2=0x7fa5d8146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:50.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.323+0000 7fa5d77fe700 1 --2- 172.21.15.162:0/1407057219 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa5d80b3950 0x7fa5d8146cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:50.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.323+0000 7fa5d77fe700 1 -- 172.21.15.162:0/1407057219 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d80aea30 con 0x7fa5d80b44e0 2020-11-28T21:46:50.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.323+0000 7fa5d77fe700 1 --2- 172.21.15.162:0/1407057219 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5d80b44e0 0x7fa5d81471b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7fa5d0001210 tx=0x7fa5d0007f20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:50.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.323+0000 7fa5c7fff700 1 -- 172.21.15.162:0/1407057219 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5d00030c0 con 0x7fa5d80b44e0 2020-11-28T21:46:50.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.323+0000 7fa5d7fff700 1 -- 172.21.15.162:0/1407057219 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5d80b0cd0 con 0x7fa5d80b44e0 2020-11-28T21:46:50.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.323+0000 7fa5c7fff700 1 -- 172.21.15.162:0/1407057219 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d0006a70 con 0x7fa5d80b44e0 2020-11-28T21:46:50.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.323+0000 7fa5c7fff700 1 -- 172.21.15.162:0/1407057219 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5d0006dd0 con 0x7fa5d80b44e0 2020-11-28T21:46:50.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.323+0000 7fa5d7fff700 1 -- 172.21.15.162:0/1407057219 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5d80ae290 con 0x7fa5d80b44e0 2020-11-28T21:46:50.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.323+0000 7fa5d7fff700 1 -- 172.21.15.162:0/1407057219 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5d80aedf0 con 0x7fa5d80b44e0 2020-11-28T21:46:50.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.323+0000 7fa5c7fff700 1 -- 172.21.15.162:0/1407057219 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa5d0010030 con 0x7fa5d80b44e0 2020-11-28T21:46:50.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.323+0000 7fa5c7fff700 1 --2- 172.21.15.162:0/1407057219 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa5b4055330 0x7fa5b40577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:50.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.323+0000 7fa5c7fff700 1 -- 172.21.15.162:0/1407057219 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa5d006deb0 con 0x7fa5d80b44e0 2020-11-28T21:46:50.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.323+0000 7fa5d6ffd700 1 --2- 172.21.15.162:0/1407057219 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa5b4055330 0x7fa5b40577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:50.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.327+0000 7fa5d6ffd700 1 --2- 172.21.15.162:0/1407057219 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa5b4055330 0x7fa5b40577f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fa5c8018000 tx=0x7fa5c8015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:50.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.331+0000 7fa5c7fff700 1 -- 172.21.15.162:0/1407057219 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa5d0044110 con 0x7fa5d80b44e0 2020-11-28T21:46:50.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.471+0000 7fa5d7fff700 1 -- 172.21.15.162:0/1407057219 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fa5d80aebc0 con 0x7fa5d80b44e0 2020-11-28T21:46:50.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.471+0000 7fa5c7fff700 1 -- 172.21.15.162:0/1407057219 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v23) v1 ==== 110+0+10 (secure 0 0 0) 0x7fa5d0047730 con 0x7fa5d80b44e0 2020-11-28T21:46:50.473 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:46:50.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.471+0000 7fa5d7fff700 1 -- 172.21.15.162:0/1407057219 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa5b4055330 msgr2=0x7fa5b40577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:50.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.471+0000 7fa5d7fff700 1 --2- 172.21.15.162:0/1407057219 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa5b4055330 0x7fa5b40577f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fa5c8018000 tx=0x7fa5c8015000).stop 2020-11-28T21:46:50.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.471+0000 7fa5d7fff700 1 -- 172.21.15.162:0/1407057219 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5d80b44e0 msgr2=0x7fa5d81471b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:50.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.471+0000 7fa5d7fff700 1 --2- 172.21.15.162:0/1407057219 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5d80b44e0 0x7fa5d81471b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7fa5d0001210 tx=0x7fa5d0007f20).stop 2020-11-28T21:46:50.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.471+0000 7fa5d7fff700 1 -- 172.21.15.162:0/1407057219 shutdown_connections 2020-11-28T21:46:50.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.471+0000 7fa5d7fff700 1 --2- 172.21.15.162:0/1407057219 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa5b4055330 0x7fa5b40577f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:50.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.471+0000 7fa5d7fff700 1 --2- 172.21.15.162:0/1407057219 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa5d80b18c0 0x7fa5d80b6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:50.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.471+0000 7fa5d7fff700 1 --2- 172.21.15.162:0/1407057219 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5d80b44e0 0x7fa5d81471b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:50.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.471+0000 7fa5d7fff700 1 --2- 172.21.15.162:0/1407057219 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa5d80b3950 0x7fa5d8146cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:50.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.471+0000 7fa5d7fff700 1 -- 172.21.15.162:0/1407057219 >> 172.21.15.162:0/1407057219 conn(0x7fa5d8062630 msgr2=0x7fa5d8062a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:50.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.471+0000 7fa5d7fff700 1 -- 172.21.15.162:0/1407057219 shutdown_connections 2020-11-28T21:46:50.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.471+0000 7fa5d7fff700 1 -- 172.21.15.162:0/1407057219 wait complete. 2020-11-28T21:46:50.486 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:46:50.601 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.599+0000 7f4ac36c3700 1 Processor -- start 2020-11-28T21:46:50.601 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.599+0000 7f4ac36c3700 1 -- start start 2020-11-28T21:46:50.603 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.599+0000 7f4ac36c3700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4abc0b9ce0 0x7f4abc1433c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:50.603 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.599+0000 7f4ac36c3700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f4abc067610 0x7f4abc13fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:50.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.599+0000 7f4ac36c3700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4abc143950 con 0x7f4abc0605c0 2020-11-28T21:46:50.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.599+0000 7f4ac36c3700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4abc0b4340 con 0x7f4abc067610 2020-11-28T21:46:50.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.599+0000 7f4ac36c3700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4abc0ba0c0 con 0x7f4abc0b9ce0 2020-11-28T21:46:50.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.599+0000 7f4ac145f700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4abc0b9ce0 0x7f4abc1433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:50.605 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.599+0000 7f4ac145f700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4abc0b9ce0 0x7f4abc1433c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37956/0 (socket says 172.21.15.162:37956) 2020-11-28T21:46:50.605 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.599+0000 7f4ac145f700 1 -- 172.21.15.162:0/2800882501 learned_addr learned my addr 172.21.15.162:0/2800882501 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:50.605 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.599+0000 7f4ac1c60700 1 --2- 172.21.15.162:0/2800882501 >> v2:172.21.15.118:3300/0 conn(0x7f4abc067610 0x7f4abc13fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:50.605 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.599+0000 7f4ac145f700 1 -- 172.21.15.162:0/2800882501 >> v2:172.21.15.118:3300/0 conn(0x7f4abc067610 msgr2=0x7f4abc13fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:50.605 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.599+0000 7f4ac145f700 1 --2- 172.21.15.162:0/2800882501 >> v2:172.21.15.118:3300/0 conn(0x7f4abc067610 0x7f4abc13fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:50.606 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.599+0000 7f4ac145f700 1 -- 172.21.15.162:0/2800882501 >> v1:172.21.15.162:6789/0 conn(0x7f4abc0605c0 legacy=0x7f4abc0b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:50.606 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.599+0000 7f4ac145f700 1 -- 172.21.15.162:0/2800882501 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4abc067aa0 con 0x7f4abc0b9ce0 2020-11-28T21:46:50.606 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.599+0000 7f4ac1c60700 1 --2- 172.21.15.162:0/2800882501 >> v2:172.21.15.118:3300/0 conn(0x7f4abc067610 0x7f4abc13fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:46:50.606 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.599+0000 7f4ac145f700 1 --2- 172.21.15.162:0/2800882501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4abc0b9ce0 0x7f4abc1433c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f4ab8002f30 tx=0x7f4ab8006e20).ready entity=mon.1 client_cookie=5a753e3aa4bd9171 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:50.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.599+0000 7f4ab3fff700 1 -- 172.21.15.162:0/2800882501 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ab8009960 con 0x7f4abc0b9ce0 2020-11-28T21:46:50.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.599+0000 7f4ab3fff700 1 -- 172.21.15.162:0/2800882501 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab8009ac0 con 0x7f4abc0b9ce0 2020-11-28T21:46:50.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.599+0000 7f4ab3fff700 1 -- 172.21.15.162:0/2800882501 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ab8009e20 con 0x7f4abc0b9ce0 2020-11-28T21:46:50.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.599+0000 7f4ac36c3700 1 -- 172.21.15.162:0/2800882501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4abc0b9ce0 msgr2=0x7f4abc1433c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:50.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.599+0000 7f4ac36c3700 1 --2- 172.21.15.162:0/2800882501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4abc0b9ce0 0x7f4abc1433c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f4ab8002f30 tx=0x7f4ab8006e20).stop 2020-11-28T21:46:50.608 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.603+0000 7f4ac36c3700 1 -- 172.21.15.162:0/2800882501 shutdown_connections 2020-11-28T21:46:50.608 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.603+0000 7f4ac36c3700 1 --2- 172.21.15.162:0/2800882501 >> v2:172.21.15.118:3300/0 conn(0x7f4abc067610 0x7f4abc13fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:50.609 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.603+0000 7f4ac36c3700 1 --2- 172.21.15.162:0/2800882501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4abc0b9ce0 0x7f4abc1433c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:50.609 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.603+0000 7f4ac36c3700 1 -- 172.21.15.162:0/2800882501 >> 172.21.15.162:0/2800882501 conn(0x7f4abc05c430 msgr2=0x7f4abc0b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:50.609 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.603+0000 7f4ac36c3700 1 -- 172.21.15.162:0/2800882501 shutdown_connections 2020-11-28T21:46:50.609 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.603+0000 7f4ac36c3700 1 -- 172.21.15.162:0/2800882501 wait complete. 2020-11-28T21:46:50.610 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.603+0000 7f4ac36c3700 1 Processor -- start 2020-11-28T21:46:50.611 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.603+0000 7f4ac36c3700 1 -- start start 2020-11-28T21:46:50.612 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.607+0000 7f4ac36c3700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4abc0605c0 0x7f4abc145290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:50.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.607+0000 7f4ac36c3700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4abc067610 0x7f4abc141d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:50.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.607+0000 7f4ac36c3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4abc0b9ce0 0x7f4abc09e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:50.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.607+0000 7f4ac36c3700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4abc144bc0 con 0x7f4abc067610 2020-11-28T21:46:50.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.607+0000 7f4ac36c3700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4abc1524f0 con 0x7f4abc0b9ce0 2020-11-28T21:46:50.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.607+0000 7f4ac36c3700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4abc152790 con 0x7f4abc0605c0 2020-11-28T21:46:50.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.607+0000 7f4ac1c60700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4abc0b9ce0 0x7f4abc09e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:50.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.607+0000 7f4ac1c60700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4abc0b9ce0 0x7f4abc09e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37966/0 (socket says 172.21.15.162:37966) 2020-11-28T21:46:50.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.607+0000 7f4ac145f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4abc0605c0 0x7f4abc145290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:50.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.607+0000 7f4ac1c60700 1 -- 172.21.15.162:0/616063528 learned_addr learned my addr 172.21.15.162:0/616063528 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:50.615 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.607+0000 7f4ac0c5e700 1 --2- 172.21.15.162:0/616063528 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4abc067610 0x7f4abc141d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:50.615 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.607+0000 7f4ac1c60700 1 -- 172.21.15.162:0/616063528 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4abc0605c0 msgr2=0x7f4abc145290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:50.615 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.607+0000 7f4ac1c60700 1 --2- 172.21.15.162:0/616063528 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4abc0605c0 0x7f4abc145290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:50.615 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.607+0000 7f4ac1c60700 1 -- 172.21.15.162:0/616063528 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4abc067610 msgr2=0x7f4abc141d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:50.616 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.607+0000 7f4ac1c60700 1 --2- 172.21.15.162:0/616063528 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4abc067610 0x7f4abc141d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:50.616 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.607+0000 7f4ac1c60700 1 -- 172.21.15.162:0/616063528 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4abc144df0 con 0x7f4abc0b9ce0 2020-11-28T21:46:50.616 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.607+0000 7f4ac0c5e700 1 --2- 172.21.15.162:0/616063528 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4abc067610 0x7f4abc141d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:46:50.616 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.607+0000 7f4ac1c60700 1 --2- 172.21.15.162:0/616063528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4abc0b9ce0 0x7f4abc09e020 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f4ab4007000 tx=0x7f4ab4008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:50.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.607+0000 7f4ab27fc700 1 -- 172.21.15.162:0/616063528 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ab400a820 con 0x7f4abc0b9ce0 2020-11-28T21:46:50.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.607+0000 7f4ab27fc700 1 -- 172.21.15.162:0/616063528 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab400a980 con 0x7f4abc0b9ce0 2020-11-28T21:46:50.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.607+0000 7f4ac36c3700 1 -- 172.21.15.162:0/616063528 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4abc142250 con 0x7f4abc0b9ce0 2020-11-28T21:46:50.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.607+0000 7f4ab27fc700 1 -- 172.21.15.162:0/616063528 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ab400ace0 con 0x7f4abc0b9ce0 2020-11-28T21:46:50.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.607+0000 7f4ac36c3700 1 -- 172.21.15.162:0/616063528 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4abc09e510 con 0x7f4abc0b9ce0 2020-11-28T21:46:50.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.611+0000 7f4ac36c3700 1 -- 172.21.15.162:0/616063528 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4abc144f80 con 0x7f4abc0b9ce0 2020-11-28T21:46:50.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.611+0000 7f4ab27fc700 1 -- 172.21.15.162:0/616063528 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4ab4011410 con 0x7f4abc0b9ce0 2020-11-28T21:46:50.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.611+0000 7f4ab27fc700 1 --2- 172.21.15.162:0/616063528 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4aa40554e0 0x7f4aa40579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:50.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.611+0000 7f4ab27fc700 1 -- 172.21.15.162:0/616063528 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4ab406e410 con 0x7f4abc0b9ce0 2020-11-28T21:46:50.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.611+0000 7f4ac145f700 1 --2- 172.21.15.162:0/616063528 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4aa40554e0 0x7f4aa40579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:50.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.611+0000 7f4ac145f700 1 --2- 172.21.15.162:0/616063528 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4aa40554e0 0x7f4aa40579a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f4ab801b000 tx=0x7f4ab8015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:50.620 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.619+0000 7f4ab27fc700 1 -- 172.21.15.162:0/616063528 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4ab40446b0 con 0x7f4abc0b9ce0 2020-11-28T21:46:50.761 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.759+0000 7f4ac36c3700 1 -- 172.21.15.162:0/616063528 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4abc057050 con 0x7f4abc0b9ce0 2020-11-28T21:46:50.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.759+0000 7f4ab27fc700 1 -- 172.21.15.162:0/616063528 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4ab4047cd0 con 0x7f4abc0b9ce0 2020-11-28T21:46:50.763 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:46:50.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.759+0000 7f4ac36c3700 1 -- 172.21.15.162:0/616063528 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4aa40554e0 msgr2=0x7f4aa40579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:50.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.759+0000 7f4ac36c3700 1 --2- 172.21.15.162:0/616063528 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4aa40554e0 0x7f4aa40579a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f4ab801b000 tx=0x7f4ab8015000).stop 2020-11-28T21:46:50.765 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.759+0000 7f4ac36c3700 1 -- 172.21.15.162:0/616063528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4abc0b9ce0 msgr2=0x7f4abc09e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:50.765 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.759+0000 7f4ac36c3700 1 --2- 172.21.15.162:0/616063528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4abc0b9ce0 0x7f4abc09e020 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f4ab4007000 tx=0x7f4ab4008040).stop 2020-11-28T21:46:50.765 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.763+0000 7f4ac36c3700 1 -- 172.21.15.162:0/616063528 shutdown_connections 2020-11-28T21:46:50.765 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.763+0000 7f4ac36c3700 1 --2- 172.21.15.162:0/616063528 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4aa40554e0 0x7f4aa40579a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:50.765 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.763+0000 7f4ac36c3700 1 --2- 172.21.15.162:0/616063528 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4abc0605c0 0x7f4abc145290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:50.766 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.763+0000 7f4ac36c3700 1 --2- 172.21.15.162:0/616063528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4abc0b9ce0 0x7f4abc09e020 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:50.766 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.763+0000 7f4ac36c3700 1 --2- 172.21.15.162:0/616063528 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4abc067610 0x7f4abc141d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:50.766 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.763+0000 7f4ac36c3700 1 -- 172.21.15.162:0/616063528 >> 172.21.15.162:0/616063528 conn(0x7f4abc05c430 msgr2=0x7f4abc0b6e90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:50.766 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.763+0000 7f4ac36c3700 1 -- 172.21.15.162:0/616063528 shutdown_connections 2020-11-28T21:46:50.767 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.763+0000 7f4ac36c3700 1 -- 172.21.15.162:0/616063528 wait complete. 2020-11-28T21:46:50.776 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-28T21:46:50.897 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.895+0000 7ff79a791700 1 Processor -- start 2020-11-28T21:46:50.897 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.895+0000 7ff79a791700 1 -- start start 2020-11-28T21:46:50.899 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.895+0000 7ff79a791700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff7940ac500 0x7ff79405fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:50.899 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.895+0000 7ff79a791700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff79413f560 0x7ff79413f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:50.900 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.895+0000 7ff79a791700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7940ac8e0 con 0x7ff7940ae9d0 2020-11-28T21:46:50.900 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.895+0000 7ff79a791700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff79413ccf0 con 0x7ff7940ac500 2020-11-28T21:46:50.900 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.895+0000 7ff79a791700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff794060690 con 0x7ff79413f560 2020-11-28T21:46:50.900 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.895+0000 7ff799f90700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff79413f560 0x7ff79413f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:50.901 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.895+0000 7ff799f90700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff79413f560 0x7ff79413f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37974/0 (socket says 172.21.15.162:37974) 2020-11-28T21:46:50.901 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.895+0000 7ff799f90700 1 -- 172.21.15.162:0/1958504610 learned_addr learned my addr 172.21.15.162:0/1958504610 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:50.901 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.895+0000 7ff79978f700 1 --2- 172.21.15.162:0/1958504610 >> v2:172.21.15.118:3300/0 conn(0x7ff7940ac500 0x7ff79405fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:50.902 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.895+0000 7ff799f90700 1 -- 172.21.15.162:0/1958504610 >> v2:172.21.15.118:3300/0 conn(0x7ff7940ac500 msgr2=0x7ff79405fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:50.902 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.895+0000 7ff799f90700 1 --2- 172.21.15.162:0/1958504610 >> v2:172.21.15.118:3300/0 conn(0x7ff7940ac500 0x7ff79405fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:50.902 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.895+0000 7ff799f90700 1 -- 172.21.15.162:0/1958504610 >> v1:172.21.15.118:6789/0 conn(0x7ff7940ae9d0 legacy=0x7ff794141b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:50.902 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.895+0000 7ff799f90700 1 -- 172.21.15.162:0/1958504610 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff794060930 con 0x7ff79413f560 2020-11-28T21:46:50.903 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.895+0000 7ff79978f700 1 --2- 172.21.15.162:0/1958504610 >> v2:172.21.15.118:3300/0 conn(0x7ff7940ac500 0x7ff79405fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:46:50.903 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.899+0000 7ff799f90700 1 --2- 172.21.15.162:0/1958504610 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff79413f560 0x7ff79413f9d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7ff78c0032b0 tx=0x7ff78c009270).ready entity=mon.1 client_cookie=64be1f1be8e1c007 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:50.906 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.899+0000 7ff78bfff700 1 -- 172.21.15.162:0/1958504610 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff78c00bc00 con 0x7ff79413f560 2020-11-28T21:46:50.907 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.899+0000 7ff78bfff700 1 -- 172.21.15.162:0/1958504610 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78c00bd60 con 0x7ff79413f560 2020-11-28T21:46:50.907 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.899+0000 7ff78bfff700 1 -- 172.21.15.162:0/1958504610 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff78c011400 con 0x7ff79413f560 2020-11-28T21:46:50.907 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.899+0000 7ff79a791700 1 -- 172.21.15.162:0/1958504610 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff79413f560 msgr2=0x7ff79413f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:50.908 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.899+0000 7ff79a791700 1 --2- 172.21.15.162:0/1958504610 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff79413f560 0x7ff79413f9d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7ff78c0032b0 tx=0x7ff78c009270).stop 2020-11-28T21:46:50.908 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.899+0000 7ff79a791700 1 -- 172.21.15.162:0/1958504610 shutdown_connections 2020-11-28T21:46:50.908 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.899+0000 7ff79a791700 1 --2- 172.21.15.162:0/1958504610 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff79413f560 0x7ff79413f9d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:50.908 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.899+0000 7ff79a791700 1 --2- 172.21.15.162:0/1958504610 >> v2:172.21.15.118:3300/0 conn(0x7ff7940ac500 0x7ff79405fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:50.909 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.899+0000 7ff79a791700 1 -- 172.21.15.162:0/1958504610 >> 172.21.15.162:0/1958504610 conn(0x7ff7940b57d0 msgr2=0x7ff7940b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:50.909 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.899+0000 7ff79a791700 1 -- 172.21.15.162:0/1958504610 shutdown_connections 2020-11-28T21:46:50.909 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.899+0000 7ff79a791700 1 -- 172.21.15.162:0/1958504610 wait complete. 2020-11-28T21:46:50.910 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.899+0000 7ff79a791700 1 Processor -- start 2020-11-28T21:46:50.910 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.899+0000 7ff79a791700 1 -- start start 2020-11-28T21:46:50.910 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.903+0000 7ff79a791700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7940ac500 0x7ff7940a9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:50.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.903+0000 7ff79a791700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff7940ae9d0 0x7ff7940ab250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:50.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.903+0000 7ff79a791700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff79413f560 0x7ff7940ab740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:50.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.903+0000 7ff79a791700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7940aac40 con 0x7ff7940ae9d0 2020-11-28T21:46:50.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.903+0000 7ff79a791700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff794063a80 con 0x7ff7940ac500 2020-11-28T21:46:50.912 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.903+0000 7ff79a791700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7941425c0 con 0x7ff79413f560 2020-11-28T21:46:50.912 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.903+0000 7ff79978f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7940ac500 0x7ff7940a9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:50.912 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.903+0000 7ff79978f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7940ac500 0x7ff7940a9b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:37976/0 (socket says 172.21.15.162:37976) 2020-11-28T21:46:50.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.903+0000 7ff79978f700 1 -- 172.21.15.162:0/3864025806 learned_addr learned my addr 172.21.15.162:0/3864025806 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:50.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.903+0000 7ff798f8e700 1 --2- 172.21.15.162:0/3864025806 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff7940ae9d0 0x7ff7940ab250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:50.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.903+0000 7ff799f90700 1 --2- 172.21.15.162:0/3864025806 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff79413f560 0x7ff7940ab740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:50.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.903+0000 7ff79978f700 1 -- 172.21.15.162:0/3864025806 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff79413f560 msgr2=0x7ff7940ab740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:50.914 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.903+0000 7ff79978f700 1 --2- 172.21.15.162:0/3864025806 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff79413f560 0x7ff7940ab740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:50.914 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.903+0000 7ff79978f700 1 -- 172.21.15.162:0/3864025806 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff7940ae9d0 msgr2=0x7ff7940ab250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:50.914 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.903+0000 7ff79978f700 1 --2- 172.21.15.162:0/3864025806 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff7940ae9d0 0x7ff7940ab250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:50.915 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.903+0000 7ff79978f700 1 -- 172.21.15.162:0/3864025806 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff794142860 con 0x7ff7940ac500 2020-11-28T21:46:50.915 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.903+0000 7ff79978f700 1 --2- 172.21.15.162:0/3864025806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7940ac500 0x7ff7940a9b50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7ff784006910 tx=0x7ff784008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:50.915 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.903+0000 7ff78a7fc700 1 -- 172.21.15.162:0/3864025806 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff784009980 con 0x7ff7940ac500 2020-11-28T21:46:50.915 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.903+0000 7ff78a7fc700 1 -- 172.21.15.162:0/3864025806 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff784009ae0 con 0x7ff7940ac500 2020-11-28T21:46:50.916 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.903+0000 7ff79a791700 1 -- 172.21.15.162:0/3864025806 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff794142b00 con 0x7ff7940ac500 2020-11-28T21:46:50.916 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.903+0000 7ff78a7fc700 1 -- 172.21.15.162:0/3864025806 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff784009e40 con 0x7ff7940ac500 2020-11-28T21:46:50.916 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.903+0000 7ff79a791700 1 -- 172.21.15.162:0/3864025806 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7940b3b80 con 0x7ff7940ac500 2020-11-28T21:46:50.918 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.903+0000 7ff79a791700 1 -- 172.21.15.162:0/3864025806 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7940aafb0 con 0x7ff7940ac500 2020-11-28T21:46:50.918 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.907+0000 7ff78a7fc700 1 -- 172.21.15.162:0/3864025806 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff784013480 con 0x7ff7940ac500 2020-11-28T21:46:50.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.907+0000 7ff78a7fc700 1 --2- 172.21.15.162:0/3864025806 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff77c055440 0x7ff77c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:50.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.907+0000 7ff78a7fc700 1 -- 172.21.15.162:0/3864025806 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff78406e5f0 con 0x7ff7940ac500 2020-11-28T21:46:50.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.907+0000 7ff798f8e700 1 --2- 172.21.15.162:0/3864025806 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff77c055440 0x7ff77c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:50.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.907+0000 7ff798f8e700 1 --2- 172.21.15.162:0/3864025806 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff77c055440 0x7ff77c057900 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7ff79000d000 tx=0x7ff790000b80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:50.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:50.911+0000 7ff78a7fc700 1 -- 172.21.15.162:0/3864025806 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff7940aafb0 con 0x7ff7940ac500 2020-11-28T21:46:51.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.051+0000 7ff79a791700 1 -- 172.21.15.162:0/3864025806 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff7940aafb0 con 0x7ff7940ac500 2020-11-28T21:46:51.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.051+0000 7ff78a7fc700 1 -- 172.21.15.162:0/3864025806 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+10 (secure 0 0 0) 0x7ff7940aafb0 con 0x7ff7940ac500 2020-11-28T21:46:51.053 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:46:51.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.051+0000 7ff79a791700 1 -- 172.21.15.162:0/3864025806 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff77c055440 msgr2=0x7ff77c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:51.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.051+0000 7ff79a791700 1 --2- 172.21.15.162:0/3864025806 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff77c055440 0x7ff77c057900 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7ff79000d000 tx=0x7ff790000b80).stop 2020-11-28T21:46:51.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.051+0000 7ff79a791700 1 -- 172.21.15.162:0/3864025806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7940ac500 msgr2=0x7ff7940a9b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:51.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.051+0000 7ff79a791700 1 --2- 172.21.15.162:0/3864025806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7940ac500 0x7ff7940a9b50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7ff784006910 tx=0x7ff784008040).stop 2020-11-28T21:46:51.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.051+0000 7ff79a791700 1 -- 172.21.15.162:0/3864025806 shutdown_connections 2020-11-28T21:46:51.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.051+0000 7ff79a791700 1 --2- 172.21.15.162:0/3864025806 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff77c055440 0x7ff77c057900 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:51.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.051+0000 7ff79a791700 1 --2- 172.21.15.162:0/3864025806 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff79413f560 0x7ff7940ab740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:51.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.051+0000 7ff79a791700 1 --2- 172.21.15.162:0/3864025806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7940ac500 0x7ff7940a9b50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:51.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.051+0000 7ff79a791700 1 --2- 172.21.15.162:0/3864025806 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff7940ae9d0 0x7ff7940ab250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:51.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.051+0000 7ff79a791700 1 -- 172.21.15.162:0/3864025806 >> 172.21.15.162:0/3864025806 conn(0x7ff7940b57d0 msgr2=0x7ff7940b2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:51.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.051+0000 7ff79a791700 1 -- 172.21.15.162:0/3864025806 shutdown_connections 2020-11-28T21:46:51.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.051+0000 7ff79a791700 1 -- 172.21.15.162:0/3864025806 wait complete. 2020-11-28T21:46:51.067 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-28T21:46:51.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.179+0000 7fc2c196a700 1 Processor -- start 2020-11-28T21:46:51.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.179+0000 7fc2c196a700 1 -- start start 2020-11-28T21:46:51.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.179+0000 7fc2c196a700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fc2bc05c4d0 0x7fc2bc0b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:51.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.183+0000 7fc2c196a700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2bc05baf0 con 0x7fc2bc0b3ab0 2020-11-28T21:46:51.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.183+0000 7fc2c196a700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2bc05bd90 con 0x7fc2bc0b2d00 2020-11-28T21:46:51.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.183+0000 7fc2c196a700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2bc0b0e20 con 0x7fc2bc05c4d0 2020-11-28T21:46:51.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.183+0000 7fc2c0968700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fc2bc0b2d00 0x7fc2bc142ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:44828/0 (socket says 172.21.15.162:44828) 2020-11-28T21:46:51.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.183+0000 7fc2c0968700 1 -- 172.21.15.162:0/2966471213 learned_addr learned my addr 172.21.15.162:0/2966471213 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:51.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.183+0000 7fc2c1169700 1 --2- 172.21.15.162:0/2966471213 >> v2:172.21.15.118:3300/0 conn(0x7fc2bc05c4d0 0x7fc2bc0b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:51.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.183+0000 7fc2bb7fe700 1 -- 172.21.15.162:0/2966471213 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 993683893 0 0) 0x7fc2bc05bd90 con 0x7fc2bc0b2d00 2020-11-28T21:46:51.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.183+0000 7fc2bb7fe700 1 -- 172.21.15.162:0/2966471213 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc29c003230 con 0x7fc2bc0b2d00 2020-11-28T21:46:51.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.183+0000 7fc2bb7fe700 1 -- 172.21.15.162:0/2966471213 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4047988164 0 0) 0x7fc29c003230 con 0x7fc2bc0b2d00 2020-11-28T21:46:51.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.183+0000 7fc2bb7fe700 1 -- 172.21.15.162:0/2966471213 >> v2:172.21.15.118:3300/0 conn(0x7fc2bc05c4d0 msgr2=0x7fc2bc0b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:51.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.183+0000 7fc2bb7fe700 1 --2- 172.21.15.162:0/2966471213 >> v2:172.21.15.118:3300/0 conn(0x7fc2bc05c4d0 0x7fc2bc0b30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:51.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.183+0000 7fc2bb7fe700 1 -- 172.21.15.162:0/2966471213 >> v1:172.21.15.118:6789/0 conn(0x7fc2bc0b3ab0 legacy=0x7fc2bc13fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:51.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.183+0000 7fc2bb7fe700 1 -- 172.21.15.162:0/2966471213 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2bc05c030 con 0x7fc2bc0b2d00 2020-11-28T21:46:51.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.183+0000 7fc2bb7fe700 1 -- 172.21.15.162:0/2966471213 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7fc2a40016d0 con 0x7fc2bc0b2d00 2020-11-28T21:46:51.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.183+0000 7fc2bb7fe700 1 -- 172.21.15.162:0/2966471213 >> v1:172.21.15.162:6789/0 conn(0x7fc2bc0b2d00 legacy=0x7fc2bc142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:51.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.183+0000 7fc2bb7fe700 1 --2- 172.21.15.162:0/2966471213 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc29c003720 0x7fc29c003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:51.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.183+0000 7fc2bb7fe700 1 --2- 172.21.15.162:0/2966471213 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc29c0044d0 0x7fc29c004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:51.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.183+0000 7fc2bb7fe700 1 --2- 172.21.15.162:0/2966471213 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc29c004e40 0x7fc29c00b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:51.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.183+0000 7fc2bb7fe700 1 -- 172.21.15.162:0/2966471213 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2bc0b0e20 con 0x7fc29c003720 2020-11-28T21:46:51.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.183+0000 7fc2bb7fe700 1 -- 172.21.15.162:0/2966471213 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29c004250 con 0x7fc29c0044d0 2020-11-28T21:46:51.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.183+0000 7fc2bb7fe700 1 -- 172.21.15.162:0/2966471213 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc29c00bc10 con 0x7fc29c004e40 2020-11-28T21:46:51.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.183+0000 7fc2bbfff700 1 --2- 172.21.15.162:0/2966471213 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc29c0044d0 0x7fc29c004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:51.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.183+0000 7fc2c0968700 1 --2- 172.21.15.162:0/2966471213 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc29c003720 0x7fc29c003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:51.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.183+0000 7fc2c1169700 1 --2- 172.21.15.162:0/2966471213 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc29c004e40 0x7fc29c00b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:51.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.183+0000 7fc2bbfff700 1 -- 172.21.15.162:0/2966471213 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc29c004e40 msgr2=0x7fc29c00b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:51.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.183+0000 7fc2bbfff700 1 --2- 172.21.15.162:0/2966471213 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc29c004e40 0x7fc29c00b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:51.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.183+0000 7fc2bbfff700 1 -- 172.21.15.162:0/2966471213 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc29c003720 msgr2=0x7fc29c003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:51.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.183+0000 7fc2bbfff700 1 --2- 172.21.15.162:0/2966471213 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc29c003720 0x7fc29c003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:51.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.183+0000 7fc2c1169700 1 -- 172.21.15.162:0/2966471213 reap_dead start 2020-11-28T21:46:51.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.183+0000 7fc2bbfff700 1 -- 172.21.15.162:0/2966471213 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc29c003230 con 0x7fc29c0044d0 2020-11-28T21:46:51.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.183+0000 7fc2bbfff700 1 --2- 172.21.15.162:0/2966471213 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc29c0044d0 0x7fc29c004950 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7fc2bc0acde0 tx=0x7fc2ac007b60).ready entity=mon.1 client_cookie=363abc8f3b78aae8 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:51.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.183+0000 7fc2bb7fe700 1 -- 172.21.15.162:0/2966471213 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc2ac00c870 con 0x7fc29c0044d0 2020-11-28T21:46:51.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.183+0000 7fc2bb7fe700 1 -- 172.21.15.162:0/2966471213 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ac00c9d0 con 0x7fc29c0044d0 2020-11-28T21:46:51.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.183+0000 7fc2c196a700 1 -- 172.21.15.162:0/2966471213 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc29c0044d0 msgr2=0x7fc29c004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:51.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.183+0000 7fc2bb7fe700 1 -- 172.21.15.162:0/2966471213 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc2ac00cd30 con 0x7fc29c0044d0 2020-11-28T21:46:51.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.183+0000 7fc2c196a700 1 --2- 172.21.15.162:0/2966471213 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc29c0044d0 0x7fc29c004950 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7fc2bc0acde0 tx=0x7fc2ac007b60).stop 2020-11-28T21:46:51.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.183+0000 7fc2c196a700 1 -- 172.21.15.162:0/2966471213 shutdown_connections 2020-11-28T21:46:51.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.183+0000 7fc2c196a700 1 --2- 172.21.15.162:0/2966471213 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc29c0044d0 0x7fc29c004950 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:51.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.183+0000 7fc2c196a700 1 -- 172.21.15.162:0/2966471213 >> 172.21.15.162:0/2966471213 conn(0x7fc2bc0604c0 msgr2=0x7fc2bc0608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:51.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.183+0000 7fc2c196a700 1 -- 172.21.15.162:0/2966471213 shutdown_connections 2020-11-28T21:46:51.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.183+0000 7fc2c196a700 1 -- 172.21.15.162:0/2966471213 wait complete. 2020-11-28T21:46:51.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.187+0000 7fc2c196a700 1 Processor -- start 2020-11-28T21:46:51.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.187+0000 7fc2c196a700 1 -- start start 2020-11-28T21:46:51.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.187+0000 7fc2c196a700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc29c0044d0 0x7fc2bc13eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:51.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.187+0000 7fc2c196a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc2bc0acf60 0x7fc2bc155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:51.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.187+0000 7fc2c196a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2bc155d00 0x7fc2bc0b7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:51.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.187+0000 7fc2c196a700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2bc0ada40 con 0x7fc2bc0acf60 2020-11-28T21:46:51.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.187+0000 7fc2c196a700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2bc0adc70 con 0x7fc2bc155d00 2020-11-28T21:46:51.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.187+0000 7fc2c196a700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2bc0adf10 con 0x7fc29c0044d0 2020-11-28T21:46:51.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.187+0000 7fc2c1169700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2bc155d00 0x7fc2bc0b7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:51.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.187+0000 7fc2c1169700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2bc155d00 0x7fc2bc0b7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38000/0 (socket says 172.21.15.162:38000) 2020-11-28T21:46:51.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.187+0000 7fc2c1169700 1 -- 172.21.15.162:0/220506906 learned_addr learned my addr 172.21.15.162:0/220506906 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:51.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.187+0000 7fc2bbfff700 1 --2- 172.21.15.162:0/220506906 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc2bc0acf60 0x7fc2bc155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:51.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.187+0000 7fc2c0968700 1 --2- 172.21.15.162:0/220506906 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc29c0044d0 0x7fc2bc13eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:51.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.187+0000 7fc2c1169700 1 -- 172.21.15.162:0/220506906 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc29c0044d0 msgr2=0x7fc2bc13eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:51.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.187+0000 7fc2c1169700 1 --2- 172.21.15.162:0/220506906 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc29c0044d0 0x7fc2bc13eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:51.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.187+0000 7fc2c1169700 1 -- 172.21.15.162:0/220506906 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc2bc0acf60 msgr2=0x7fc2bc155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:51.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.187+0000 7fc2c1169700 1 --2- 172.21.15.162:0/220506906 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc2bc0acf60 0x7fc2bc155810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:51.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.187+0000 7fc2c1169700 1 -- 172.21.15.162:0/220506906 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2bc155370 con 0x7fc2bc155d00 2020-11-28T21:46:51.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.187+0000 7fc2c1169700 1 --2- 172.21.15.162:0/220506906 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2bc155d00 0x7fc2bc0b7eb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7fc2b400b000 tx=0x7fc2b40091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:51.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.187+0000 7fc2b9ffb700 1 -- 172.21.15.162:0/220506906 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc2b400fab0 con 0x7fc2bc155d00 2020-11-28T21:46:51.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.187+0000 7fc2b9ffb700 1 -- 172.21.15.162:0/220506906 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b400fc10 con 0x7fc2bc155d00 2020-11-28T21:46:51.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.187+0000 7fc2b9ffb700 1 -- 172.21.15.162:0/220506906 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc2b4014440 con 0x7fc2bc155d00 2020-11-28T21:46:51.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.187+0000 7fc2c196a700 1 -- 172.21.15.162:0/220506906 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2bc1451a0 con 0x7fc2bc155d00 2020-11-28T21:46:51.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.187+0000 7fc2c196a700 1 -- 172.21.15.162:0/220506906 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2bc0b83a0 con 0x7fc2bc155d00 2020-11-28T21:46:51.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.191+0000 7fc2c196a700 1 -- 172.21.15.162:0/220506906 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2bc0ae0d0 con 0x7fc2bc155d00 2020-11-28T21:46:51.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.191+0000 7fc2b9ffb700 1 -- 172.21.15.162:0/220506906 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc2b40145a0 con 0x7fc2bc155d00 2020-11-28T21:46:51.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.191+0000 7fc2b9ffb700 1 --2- 172.21.15.162:0/220506906 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc2a00553b0 0x7fc2a0057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:51.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.191+0000 7fc2b9ffb700 1 -- 172.21.15.162:0/220506906 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc2b406e750 con 0x7fc2bc155d00 2020-11-28T21:46:51.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.191+0000 7fc2c0968700 1 --2- 172.21.15.162:0/220506906 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc2a00553b0 0x7fc2a0057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:51.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.191+0000 7fc2c0968700 1 --2- 172.21.15.162:0/220506906 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc2a00553b0 0x7fc2a0057870 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fc2a4005000 tx=0x7fc2a4007770).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:51.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.195+0000 7fc2b9ffb700 1 -- 172.21.15.162:0/220506906 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc2b40449f0 con 0x7fc2bc155d00 2020-11-28T21:46:51.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.335+0000 7fc2c196a700 1 -- 172.21.15.162:0/220506906 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc2bc155500 con 0x7fc2bc155d00 2020-11-28T21:46:51.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.335+0000 7fc2b9ffb700 1 -- 172.21.15.162:0/220506906 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+10 (secure 0 0 0) 0x7fc2b4018020 con 0x7fc2bc155d00 2020-11-28T21:46:51.340 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:46:51.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.339+0000 7fc2c196a700 1 -- 172.21.15.162:0/220506906 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc2a00553b0 msgr2=0x7fc2a0057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:51.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.339+0000 7fc2c196a700 1 --2- 172.21.15.162:0/220506906 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc2a00553b0 0x7fc2a0057870 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fc2a4005000 tx=0x7fc2a4007770).stop 2020-11-28T21:46:51.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.339+0000 7fc2c196a700 1 -- 172.21.15.162:0/220506906 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2bc155d00 msgr2=0x7fc2bc0b7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:51.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.339+0000 7fc2c196a700 1 --2- 172.21.15.162:0/220506906 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2bc155d00 0x7fc2bc0b7eb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7fc2b400b000 tx=0x7fc2b40091a0).stop 2020-11-28T21:46:51.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.339+0000 7fc2c196a700 1 -- 172.21.15.162:0/220506906 shutdown_connections 2020-11-28T21:46:51.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.339+0000 7fc2c196a700 1 --2- 172.21.15.162:0/220506906 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc2a00553b0 0x7fc2a0057870 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:51.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.339+0000 7fc2c196a700 1 --2- 172.21.15.162:0/220506906 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc29c0044d0 0x7fc2bc13eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:51.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.339+0000 7fc2c196a700 1 --2- 172.21.15.162:0/220506906 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2bc155d00 0x7fc2bc0b7eb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:51.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.339+0000 7fc2c196a700 1 --2- 172.21.15.162:0/220506906 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc2bc0acf60 0x7fc2bc155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:51.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.339+0000 7fc2c196a700 1 -- 172.21.15.162:0/220506906 >> 172.21.15.162:0/220506906 conn(0x7fc2bc0604c0 msgr2=0x7fc2bc142b30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:51.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.339+0000 7fc2c196a700 1 -- 172.21.15.162:0/220506906 shutdown_connections 2020-11-28T21:46:51.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.339+0000 7fc2c196a700 1 -- 172.21.15.162:0/220506906 wait complete. 2020-11-28T21:46:51.353 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:46:51.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.463+0000 7f6e6e316700 1 Processor -- start 2020-11-28T21:46:51.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.463+0000 7f6e6e316700 1 -- start start 2020-11-28T21:46:51.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.463+0000 7f6e6e316700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6e680b1870 0x7f6e68142f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:51.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.463+0000 7f6e6e316700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f6e68143440 0x7f6e680b0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:51.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.463+0000 7f6e6e316700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e680b1c50 con 0x7f6e680b4720 2020-11-28T21:46:51.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.463+0000 7f6e6e316700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e680615a0 con 0x7f6e68143440 2020-11-28T21:46:51.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.463+0000 7f6e6e316700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e680afff0 con 0x7f6e680b1870 2020-11-28T21:46:51.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.463+0000 7f6e6d314700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6e680b1870 0x7f6e68142f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:51.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.463+0000 7f6e6d314700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6e680b1870 0x7f6e68142f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38004/0 (socket says 172.21.15.162:38004) 2020-11-28T21:46:51.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.463+0000 7f6e6d314700 1 -- 172.21.15.162:0/1136356953 learned_addr learned my addr 172.21.15.162:0/1136356953 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:51.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.463+0000 7f6e6cb13700 1 --2- 172.21.15.162:0/1136356953 >> v2:172.21.15.118:3300/0 conn(0x7f6e68143440 0x7f6e680b0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:51.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.463+0000 7f6e6d314700 1 -- 172.21.15.162:0/1136356953 >> v2:172.21.15.118:3300/0 conn(0x7f6e68143440 msgr2=0x7f6e680b0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:51.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.463+0000 7f6e6d314700 1 --2- 172.21.15.162:0/1136356953 >> v2:172.21.15.118:3300/0 conn(0x7f6e68143440 0x7f6e680b0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:51.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.463+0000 7f6e6d314700 1 -- 172.21.15.162:0/1136356953 >> v1:172.21.15.118:6789/0 conn(0x7f6e680b4720 legacy=0x7f6e680b0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:51.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.463+0000 7f6e6d314700 1 -- 172.21.15.162:0/1136356953 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e68060a60 con 0x7f6e680b1870 2020-11-28T21:46:51.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.463+0000 7f6e6d314700 1 --2- 172.21.15.162:0/1136356953 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e680b1870 0x7f6e68142f50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f6e64002f30 tx=0x7f6e64006e80).ready entity=mon.1 client_cookie=1b780e6ce0165056 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:51.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.463+0000 7f6e57fff700 1 -- 172.21.15.162:0/1136356953 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e640097f0 con 0x7f6e680b1870 2020-11-28T21:46:51.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.463+0000 7f6e57fff700 1 -- 172.21.15.162:0/1136356953 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e64009950 con 0x7f6e680b1870 2020-11-28T21:46:51.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.463+0000 7f6e57fff700 1 -- 172.21.15.162:0/1136356953 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e64009cb0 con 0x7f6e680b1870 2020-11-28T21:46:51.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.463+0000 7f6e6e316700 1 -- 172.21.15.162:0/1136356953 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e680b1870 msgr2=0x7f6e68142f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:51.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.463+0000 7f6e6e316700 1 --2- 172.21.15.162:0/1136356953 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e680b1870 0x7f6e68142f50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f6e64002f30 tx=0x7f6e64006e80).stop 2020-11-28T21:46:51.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.467+0000 7f6e6e316700 1 -- 172.21.15.162:0/1136356953 shutdown_connections 2020-11-28T21:46:51.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.467+0000 7f6e6e316700 1 --2- 172.21.15.162:0/1136356953 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e680b1870 0x7f6e68142f50 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:51.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.467+0000 7f6e6e316700 1 --2- 172.21.15.162:0/1136356953 >> v2:172.21.15.118:3300/0 conn(0x7f6e68143440 0x7f6e680b0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:51.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.467+0000 7f6e6e316700 1 -- 172.21.15.162:0/1136356953 >> 172.21.15.162:0/1136356953 conn(0x7f6e680b54f0 msgr2=0x7f6e680b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:51.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.467+0000 7f6e6e316700 1 -- 172.21.15.162:0/1136356953 shutdown_connections 2020-11-28T21:46:51.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.467+0000 7f6e6e316700 1 -- 172.21.15.162:0/1136356953 wait complete. 2020-11-28T21:46:51.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.467+0000 7f6e6e316700 1 Processor -- start 2020-11-28T21:46:51.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.467+0000 7f6e6e316700 1 -- start start 2020-11-28T21:46:51.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.471+0000 7f6e6e316700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6e680b1870 0x7f6e68093d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:51.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.471+0000 7f6e6e316700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e680b4720 0x7f6e6809a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:51.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.471+0000 7f6e6e316700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6e68143440 0x7f6e681519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:51.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.471+0000 7f6e6e316700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e681420a0 con 0x7f6e680b1870 2020-11-28T21:46:51.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.471+0000 7f6e6e316700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e68099e70 con 0x7f6e680b4720 2020-11-28T21:46:51.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.471+0000 7f6e6e316700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e680af190 con 0x7f6e68143440 2020-11-28T21:46:51.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.471+0000 7f6e6cb13700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e680b4720 0x7f6e6809a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:51.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.471+0000 7f6e6cb13700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e680b4720 0x7f6e6809a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38014/0 (socket says 172.21.15.162:38014) 2020-11-28T21:46:51.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.471+0000 7f6e6cb13700 1 -- 172.21.15.162:0/225869967 learned_addr learned my addr 172.21.15.162:0/225869967 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:51.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.471+0000 7f6e6d314700 1 --2- 172.21.15.162:0/225869967 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6e680b1870 0x7f6e68093d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:51.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.471+0000 7f6e6db15700 1 --2- 172.21.15.162:0/225869967 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6e68143440 0x7f6e681519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:51.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.471+0000 7f6e6cb13700 1 -- 172.21.15.162:0/225869967 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6e68143440 msgr2=0x7f6e681519b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:51.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.471+0000 7f6e6cb13700 1 --2- 172.21.15.162:0/225869967 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6e68143440 0x7f6e681519b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:51.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.471+0000 7f6e6cb13700 1 -- 172.21.15.162:0/225869967 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6e680b1870 msgr2=0x7f6e68093d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:51.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.471+0000 7f6e6cb13700 1 --2- 172.21.15.162:0/225869967 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6e680b1870 0x7f6e68093d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:51.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.471+0000 7f6e6cb13700 1 -- 172.21.15.162:0/225869967 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e680af430 con 0x7f6e680b4720 2020-11-28T21:46:51.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.471+0000 7f6e6cb13700 1 --2- 172.21.15.162:0/225869967 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e680b4720 0x7f6e6809a6f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f6e680af0e0 tx=0x7f6e58006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:51.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.471+0000 7f6e567fc700 1 -- 172.21.15.162:0/225869967 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e5800ca10 con 0x7f6e680b4720 2020-11-28T21:46:51.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.471+0000 7f6e567fc700 1 -- 172.21.15.162:0/225869967 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e5800cb70 con 0x7f6e680b4720 2020-11-28T21:46:51.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.471+0000 7f6e6e316700 1 -- 172.21.15.162:0/225869967 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e681525b0 con 0x7f6e680b4720 2020-11-28T21:46:51.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.471+0000 7f6e567fc700 1 -- 172.21.15.162:0/225869967 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e5800be30 con 0x7f6e680b4720 2020-11-28T21:46:51.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.471+0000 7f6e6e316700 1 -- 172.21.15.162:0/225869967 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e6809ae00 con 0x7f6e680b4720 2020-11-28T21:46:51.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.471+0000 7f6e6e316700 1 -- 172.21.15.162:0/225869967 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e68152740 con 0x7f6e680b4720 2020-11-28T21:46:51.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.475+0000 7f6e567fc700 1 -- 172.21.15.162:0/225869967 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6e5801c460 con 0x7f6e680b4720 2020-11-28T21:46:51.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.475+0000 7f6e567fc700 1 --2- 172.21.15.162:0/225869967 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6e48055330 0x7f6e480577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:51.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.475+0000 7f6e567fc700 1 -- 172.21.15.162:0/225869967 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6e5806e320 con 0x7f6e680b4720 2020-11-28T21:46:51.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.475+0000 7f6e6d314700 1 --2- 172.21.15.162:0/225869967 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6e48055330 0x7f6e480577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:51.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.475+0000 7f6e6d314700 1 --2- 172.21.15.162:0/225869967 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6e48055330 0x7f6e480577f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f6e6401c000 tx=0x7f6e64015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:51.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.483+0000 7f6e567fc700 1 -- 172.21.15.162:0/225869967 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6e580445d0 con 0x7f6e680b4720 2020-11-28T21:46:51.624 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.623+0000 7f6e6e316700 1 -- 172.21.15.162:0/225869967 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e68057050 con 0x7f6e680b4720 2020-11-28T21:46:51.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.623+0000 7f6e567fc700 1 -- 172.21.15.162:0/225869967 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16134 (secure 0 0 0) 0x7f6e58047bf0 con 0x7f6e680b4720 2020-11-28T21:46:51.626 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:46:51.627 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":23,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:46:30.124354+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:46:27.155497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-28T21:46:28.137474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:46:51.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.623+0000 7f6e6e316700 1 -- 172.21.15.162:0/225869967 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6e48055330 msgr2=0x7f6e480577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:51.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.623+0000 7f6e6e316700 1 --2- 172.21.15.162:0/225869967 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6e48055330 0x7f6e480577f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f6e6401c000 tx=0x7f6e64015000).stop 2020-11-28T21:46:51.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.623+0000 7f6e6e316700 1 -- 172.21.15.162:0/225869967 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e680b4720 msgr2=0x7f6e6809a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:51.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.623+0000 7f6e6e316700 1 --2- 172.21.15.162:0/225869967 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e680b4720 0x7f6e6809a6f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f6e680af0e0 tx=0x7f6e58006dc0).stop 2020-11-28T21:46:51.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.623+0000 7f6e6e316700 1 -- 172.21.15.162:0/225869967 shutdown_connections 2020-11-28T21:46:51.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.623+0000 7f6e6e316700 1 --2- 172.21.15.162:0/225869967 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6e48055330 0x7f6e480577f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:51.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.623+0000 7f6e6e316700 1 --2- 172.21.15.162:0/225869967 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6e68143440 0x7f6e681519b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:51.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.623+0000 7f6e6e316700 1 --2- 172.21.15.162:0/225869967 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6e680b1870 0x7f6e68093d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:51.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.623+0000 7f6e6e316700 1 --2- 172.21.15.162:0/225869967 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e680b4720 0x7f6e6809a6f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:51.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.623+0000 7f6e6e316700 1 -- 172.21.15.162:0/225869967 >> 172.21.15.162:0/225869967 conn(0x7f6e680b54f0 msgr2=0x7f6e6813fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:51.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.627+0000 7f6e6e316700 1 -- 172.21.15.162:0/225869967 shutdown_connections 2020-11-28T21:46:51.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.627+0000 7f6e6e316700 1 -- 172.21.15.162:0/225869967 wait complete. 2020-11-28T21:46:51.641 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-28T21:46:27.155497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-28T21:46:28.137474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-28T21:46:51.642 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:46:51.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.755+0000 7f261b30c700 1 Processor -- start 2020-11-28T21:46:51.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.755+0000 7f261b30c700 1 -- start start 2020-11-28T21:46:51.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.759+0000 7f261b30c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f26140b5730 0x7f261414ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:51.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.759+0000 7f261b30c700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f261414b140 0x7f2614147660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:51.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.759+0000 7f261b30c700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26140a8690 con 0x7f26140b1120 2020-11-28T21:46:51.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.759+0000 7f261b30c700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26140a88c0 con 0x7f26140b5730 2020-11-28T21:46:51.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.759+0000 7f261b30c700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26141471c0 con 0x7f261414b140 2020-11-28T21:46:51.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.759+0000 7f26198a9700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f26140b1120 0x7f26140af820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:44864/0 (socket says 172.21.15.162:44864) 2020-11-28T21:46:51.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.759+0000 7f26198a9700 1 -- 172.21.15.162:0/4031271081 learned_addr learned my addr 172.21.15.162:0/4031271081 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:51.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.759+0000 7f26188a7700 1 --2- 172.21.15.162:0/4031271081 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f261414b140 0x7f2614147660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:51.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.759+0000 7f26190a8700 1 --2- 172.21.15.162:0/4031271081 >> v2:172.21.15.118:3300/0 conn(0x7f26140b5730 0x7f261414ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:51.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.759+0000 7f2613fff700 1 -- 172.21.15.162:0/4031271081 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3156166147 0 0) 0x7f26140a8690 con 0x7f26140b1120 2020-11-28T21:46:51.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.759+0000 7f2613fff700 1 -- 172.21.15.162:0/4031271081 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25f4003230 con 0x7f26140b1120 2020-11-28T21:46:51.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.759+0000 7f26190a8700 1 -- 172.21.15.162:0/4031271081 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f261414b140 msgr2=0x7f2614147660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:51.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.759+0000 7f26190a8700 1 --2- 172.21.15.162:0/4031271081 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f261414b140 0x7f2614147660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:51.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.759+0000 7f26190a8700 1 -- 172.21.15.162:0/4031271081 >> v1:172.21.15.162:6789/0 conn(0x7f26140b1120 legacy=0x7f26140af820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:51.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.759+0000 7f26190a8700 1 -- 172.21.15.162:0/4031271081 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2614146ef0 con 0x7f26140b5730 2020-11-28T21:46:51.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.759+0000 7f26188a7700 1 --2- 172.21.15.162:0/4031271081 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f261414b140 0x7f2614147660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:46:51.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.759+0000 7f26190a8700 1 --2- 172.21.15.162:0/4031271081 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26140b5730 0x7f261414ac50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f25fc002f30 tx=0x7f25fc006dc0).ready entity=mon.0 client_cookie=7fd2588a161e42e5 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:51.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.759+0000 7f2613fff700 1 -- 172.21.15.162:0/4031271081 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f25fc0096a0 con 0x7f26140b5730 2020-11-28T21:46:51.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.759+0000 7f2613fff700 1 -- 172.21.15.162:0/4031271081 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25fc009800 con 0x7f26140b5730 2020-11-28T21:46:51.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.759+0000 7f2613fff700 1 -- 172.21.15.162:0/4031271081 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f25fc009b60 con 0x7f26140b5730 2020-11-28T21:46:51.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.759+0000 7f261b30c700 1 -- 172.21.15.162:0/4031271081 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26140b5730 msgr2=0x7f261414ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:51.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.759+0000 7f261b30c700 1 --2- 172.21.15.162:0/4031271081 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26140b5730 0x7f261414ac50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f25fc002f30 tx=0x7f25fc006dc0).stop 2020-11-28T21:46:51.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.759+0000 7f261b30c700 1 -- 172.21.15.162:0/4031271081 shutdown_connections 2020-11-28T21:46:51.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.759+0000 7f261b30c700 1 --2- 172.21.15.162:0/4031271081 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26140b5730 0x7f261414ac50 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:51.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.759+0000 7f261b30c700 1 --2- 172.21.15.162:0/4031271081 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f261414b140 0x7f2614147660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:51.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.759+0000 7f261b30c700 1 -- 172.21.15.162:0/4031271081 >> 172.21.15.162:0/4031271081 conn(0x7f261405c3f0 msgr2=0x7f26140b4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:51.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.759+0000 7f261b30c700 1 -- 172.21.15.162:0/4031271081 shutdown_connections 2020-11-28T21:46:51.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.759+0000 7f261b30c700 1 -- 172.21.15.162:0/4031271081 wait complete. 2020-11-28T21:46:51.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.763+0000 7f261b30c700 1 Processor -- start 2020-11-28T21:46:51.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.763+0000 7f261b30c700 1 -- start start 2020-11-28T21:46:51.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.763+0000 7f261b30c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26140b1120 0x7f2614156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:51.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.763+0000 7f261b30c700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f26140b5730 0x7f261409a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:51.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.763+0000 7f261b30c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f261414b140 0x7f2614148270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:51.776 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.763+0000 7f261b30c700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f261414a130 con 0x7f261414b140 2020-11-28T21:46:51.776 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.763+0000 7f261b30c700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26141560f0 con 0x7f26140b1120 2020-11-28T21:46:51.776 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.763+0000 7f261b30c700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2614156390 con 0x7f26140b5730 2020-11-28T21:46:51.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.763+0000 7f26190a8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26140b1120 0x7f2614156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:51.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.763+0000 7f26190a8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26140b1120 0x7f2614156ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38024/0 (socket says 172.21.15.162:38024) 2020-11-28T21:46:51.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.763+0000 7f26190a8700 1 -- 172.21.15.162:0/936411062 learned_addr learned my addr 172.21.15.162:0/936411062 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:51.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.763+0000 7f26188a7700 1 --2- 172.21.15.162:0/936411062 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f26140b5730 0x7f261409a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:51.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.763+0000 7f26198a9700 1 --2- 172.21.15.162:0/936411062 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f261414b140 0x7f2614148270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:51.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.763+0000 7f26190a8700 1 -- 172.21.15.162:0/936411062 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f26140b5730 msgr2=0x7f261409a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:51.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.763+0000 7f26190a8700 1 --2- 172.21.15.162:0/936411062 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f26140b5730 0x7f261409a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:51.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.763+0000 7f26190a8700 1 -- 172.21.15.162:0/936411062 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f261414b140 msgr2=0x7f2614148270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:51.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.763+0000 7f26190a8700 1 --2- 172.21.15.162:0/936411062 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f261414b140 0x7f2614148270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:51.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.763+0000 7f26190a8700 1 -- 172.21.15.162:0/936411062 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2614156630 con 0x7f26140b1120 2020-11-28T21:46:51.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.763+0000 7f26190a8700 1 --2- 172.21.15.162:0/936411062 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26140b1120 0x7f2614156ad0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f25fc002f30 tx=0x7f25fc015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:51.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.763+0000 7f26127fc700 1 -- 172.21.15.162:0/936411062 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f25fc011b80 con 0x7f26140b1120 2020-11-28T21:46:51.780 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.763+0000 7f26127fc700 1 -- 172.21.15.162:0/936411062 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25fc011ce0 con 0x7f26140b1120 2020-11-28T21:46:51.780 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.763+0000 7f261b30c700 1 -- 172.21.15.162:0/936411062 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f261409a990 con 0x7f26140b1120 2020-11-28T21:46:51.780 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.763+0000 7f26127fc700 1 -- 172.21.15.162:0/936411062 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f25fc01d640 con 0x7f26140b1120 2020-11-28T21:46:51.780 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.763+0000 7f261b30c700 1 -- 172.21.15.162:0/936411062 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26140abbd0 con 0x7f26140b1120 2020-11-28T21:46:51.781 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.767+0000 7f261b30c700 1 -- 172.21.15.162:0/936411062 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2614057050 con 0x7f26140b1120 2020-11-28T21:46:51.782 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.767+0000 7f26127fc700 1 -- 172.21.15.162:0/936411062 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f25fc00f070 con 0x7f26140b1120 2020-11-28T21:46:51.782 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.767+0000 7f26127fc700 1 --2- 172.21.15.162:0/936411062 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f25f80553b0 0x7f25f8057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:51.782 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.767+0000 7f26127fc700 1 -- 172.21.15.162:0/936411062 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f25fc06f800 con 0x7f26140b1120 2020-11-28T21:46:51.782 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.767+0000 7f26188a7700 1 --2- 172.21.15.162:0/936411062 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f25f80553b0 0x7f25f8057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:51.783 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.767+0000 7f26188a7700 1 --2- 172.21.15.162:0/936411062 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f25f80553b0 0x7f25f8057870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f260400b000 tx=0x7f2604008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:51.783 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.775+0000 7f26127fc700 1 -- 172.21.15.162:0/936411062 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f25fc045a60 con 0x7f26140b1120 2020-11-28T21:46:51.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.919+0000 7f261b30c700 1 -- 172.21.15.162:0/936411062 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f26140983d0 con 0x7f26140b1120 2020-11-28T21:46:51.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.919+0000 7f26127fc700 1 -- 172.21.15.162:0/936411062 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v23) v1 ==== 110+0+10 (secure 0 0 0) 0x7f25fc049080 con 0x7f26140b1120 2020-11-28T21:46:51.922 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:46:51.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.919+0000 7f261b30c700 1 -- 172.21.15.162:0/936411062 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f25f80553b0 msgr2=0x7f25f8057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:51.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.919+0000 7f261b30c700 1 --2- 172.21.15.162:0/936411062 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f25f80553b0 0x7f25f8057870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f260400b000 tx=0x7f2604008040).stop 2020-11-28T21:46:51.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.919+0000 7f261b30c700 1 -- 172.21.15.162:0/936411062 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26140b1120 msgr2=0x7f2614156ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:51.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.919+0000 7f261b30c700 1 --2- 172.21.15.162:0/936411062 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26140b1120 0x7f2614156ad0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f25fc002f30 tx=0x7f25fc015000).stop 2020-11-28T21:46:51.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.919+0000 7f261b30c700 1 -- 172.21.15.162:0/936411062 shutdown_connections 2020-11-28T21:46:51.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.919+0000 7f261b30c700 1 --2- 172.21.15.162:0/936411062 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f25f80553b0 0x7f25f8057870 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:51.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.919+0000 7f261b30c700 1 --2- 172.21.15.162:0/936411062 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f261414b140 0x7f2614148270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:51.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.919+0000 7f261b30c700 1 --2- 172.21.15.162:0/936411062 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26140b1120 0x7f2614156ad0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:51.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.919+0000 7f261b30c700 1 --2- 172.21.15.162:0/936411062 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f26140b5730 0x7f261409a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:51.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.919+0000 7f261b30c700 1 -- 172.21.15.162:0/936411062 >> 172.21.15.162:0/936411062 conn(0x7f261405c3f0 msgr2=0x7f2614060dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:51.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.919+0000 7f261b30c700 1 -- 172.21.15.162:0/936411062 shutdown_connections 2020-11-28T21:46:51.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:51.919+0000 7f261b30c700 1 -- 172.21.15.162:0/936411062 wait complete. 2020-11-28T21:46:51.936 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:46:52.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.047+0000 7f4975469700 1 Processor -- start 2020-11-28T21:46:52.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.047+0000 7f4975469700 1 -- start start 2020-11-28T21:46:52.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.047+0000 7f4975469700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f49700b3ab0 0x7f497005c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:52.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.047+0000 7f4975469700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f49700b30e0 0x7f497013fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:52.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.047+0000 7f4975469700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f497005baf0 con 0x7f49700b2d00 2020-11-28T21:46:52.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.047+0000 7f4975469700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49700b0e20 con 0x7f49700b3ab0 2020-11-28T21:46:52.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.047+0000 7f4975469700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f497005bd90 con 0x7f49700b30e0 2020-11-28T21:46:52.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.047+0000 7f496ffff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f49700b2d00 0x7f4970142ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:44876/0 (socket says 172.21.15.162:44876) 2020-11-28T21:46:52.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.047+0000 7f496ffff700 1 -- 172.21.15.162:0/80196420 learned_addr learned my addr 172.21.15.162:0/80196420 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:52.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.047+0000 7f496f7fe700 1 --2- 172.21.15.162:0/80196420 >> v2:172.21.15.118:3300/0 conn(0x7f49700b3ab0 0x7f497005c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:52.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.047+0000 7f4974c68700 1 --2- 172.21.15.162:0/80196420 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f49700b30e0 0x7f497013fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:52.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.047+0000 7f496effd700 1 -- 172.21.15.162:0/80196420 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2774746138 0 0) 0x7f497005baf0 con 0x7f49700b2d00 2020-11-28T21:46:52.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.047+0000 7f496effd700 1 -- 172.21.15.162:0/80196420 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4950003230 con 0x7f49700b2d00 2020-11-28T21:46:52.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.047+0000 7f496effd700 1 -- 172.21.15.162:0/80196420 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1151532250 0 0) 0x7f4950003230 con 0x7f49700b2d00 2020-11-28T21:46:52.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.047+0000 7f496f7fe700 1 -- 172.21.15.162:0/80196420 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f49700b30e0 msgr2=0x7f497013fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:52.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.047+0000 7f496f7fe700 1 --2- 172.21.15.162:0/80196420 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f49700b30e0 0x7f497013fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:52.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.047+0000 7f496f7fe700 1 -- 172.21.15.162:0/80196420 >> v1:172.21.15.162:6789/0 conn(0x7f49700b2d00 legacy=0x7f4970142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:52.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.047+0000 7f496f7fe700 1 -- 172.21.15.162:0/80196420 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f497005c030 con 0x7f49700b3ab0 2020-11-28T21:46:52.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.051+0000 7f496f7fe700 1 --2- 172.21.15.162:0/80196420 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49700b3ab0 0x7f497005c4d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f4968001870 tx=0x7f4958006dc0).ready entity=mon.0 client_cookie=5120031a54d8cef9 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:52.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.051+0000 7f496effd700 1 -- 172.21.15.162:0/80196420 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49680016d0 con 0x7f49700b3ab0 2020-11-28T21:46:52.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.051+0000 7f496effd700 1 -- 172.21.15.162:0/80196420 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f495800d570 con 0x7f49700b3ab0 2020-11-28T21:46:52.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.051+0000 7f496effd700 1 -- 172.21.15.162:0/80196420 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f495800d6e0 con 0x7f49700b3ab0 2020-11-28T21:46:52.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.051+0000 7f4975469700 1 -- 172.21.15.162:0/80196420 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49700b3ab0 msgr2=0x7f497005c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:52.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.051+0000 7f4975469700 1 --2- 172.21.15.162:0/80196420 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49700b3ab0 0x7f497005c4d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f4968001870 tx=0x7f4958006dc0).stop 2020-11-28T21:46:52.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.051+0000 7f4975469700 1 -- 172.21.15.162:0/80196420 shutdown_connections 2020-11-28T21:46:52.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.051+0000 7f4975469700 1 --2- 172.21.15.162:0/80196420 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f49700b30e0 0x7f497013fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:52.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.051+0000 7f4975469700 1 --2- 172.21.15.162:0/80196420 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49700b3ab0 0x7f497005c4d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:52.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.051+0000 7f4975469700 1 -- 172.21.15.162:0/80196420 >> 172.21.15.162:0/80196420 conn(0x7f49700604c0 msgr2=0x7f49700608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:52.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.051+0000 7f4975469700 1 -- 172.21.15.162:0/80196420 shutdown_connections 2020-11-28T21:46:52.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.051+0000 7f4975469700 1 -- 172.21.15.162:0/80196420 wait complete. 2020-11-28T21:46:52.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.051+0000 7f4975469700 1 Processor -- start 2020-11-28T21:46:52.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.051+0000 7f4975469700 1 -- start start 2020-11-28T21:46:52.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.051+0000 7f4975469700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49700b2d00 0x7f497013ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:52.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.051+0000 7f4975469700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f49700b30e0 0x7f4970140970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:52.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.051+0000 7f4975469700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49700b3ab0 0x7f49701425e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:52.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.051+0000 7f4975469700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49700b9220 con 0x7f49700b3ab0 2020-11-28T21:46:52.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.051+0000 7f4975469700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49700b94c0 con 0x7f49700b2d00 2020-11-28T21:46:52.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.051+0000 7f4975469700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49700b9760 con 0x7f49700b30e0 2020-11-28T21:46:52.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.051+0000 7f496ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49700b2d00 0x7f497013ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:52.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.051+0000 7f496ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49700b2d00 0x7f497013ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38042/0 (socket says 172.21.15.162:38042) 2020-11-28T21:46:52.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.051+0000 7f496ffff700 1 -- 172.21.15.162:0/2565198630 learned_addr learned my addr 172.21.15.162:0/2565198630 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:52.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.051+0000 7f496f7fe700 1 --2- 172.21.15.162:0/2565198630 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f49700b30e0 0x7f4970140970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:52.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.051+0000 7f4974c68700 1 --2- 172.21.15.162:0/2565198630 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49700b3ab0 0x7f49701425e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:52.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.055+0000 7f496ffff700 1 -- 172.21.15.162:0/2565198630 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f49700b30e0 msgr2=0x7f4970140970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:52.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.055+0000 7f496ffff700 1 --2- 172.21.15.162:0/2565198630 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f49700b30e0 0x7f4970140970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:52.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.055+0000 7f496ffff700 1 -- 172.21.15.162:0/2565198630 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49700b3ab0 msgr2=0x7f49701425e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:52.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.055+0000 7f496ffff700 1 --2- 172.21.15.162:0/2565198630 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49700b3ab0 0x7f49701425e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:52.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.055+0000 7f496ffff700 1 -- 172.21.15.162:0/2565198630 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49700b9a00 con 0x7f49700b2d00 2020-11-28T21:46:52.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.055+0000 7f496ffff700 1 --2- 172.21.15.162:0/2565198630 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49700b2d00 0x7f497013ef70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f49680018b0 tx=0x7f4968001770).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:52.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.055+0000 7f496d7fa700 1 -- 172.21.15.162:0/2565198630 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4968007170 con 0x7f49700b2d00 2020-11-28T21:46:52.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.055+0000 7f496d7fa700 1 -- 172.21.15.162:0/2565198630 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49680072d0 con 0x7f49700b2d00 2020-11-28T21:46:52.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.055+0000 7f4975469700 1 -- 172.21.15.162:0/2565198630 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49701517c0 con 0x7f49700b2d00 2020-11-28T21:46:52.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.055+0000 7f496d7fa700 1 -- 172.21.15.162:0/2565198630 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49680048f0 con 0x7f49700b2d00 2020-11-28T21:46:52.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.055+0000 7f4975469700 1 -- 172.21.15.162:0/2565198630 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4970151ad0 con 0x7f49700b2d00 2020-11-28T21:46:52.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.055+0000 7f4975469700 1 -- 172.21.15.162:0/2565198630 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4970142e10 con 0x7f49700b2d00 2020-11-28T21:46:52.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.055+0000 7f496d7fa700 1 -- 172.21.15.162:0/2565198630 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f496801c030 con 0x7f49700b2d00 2020-11-28T21:46:52.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.055+0000 7f496d7fa700 1 --2- 172.21.15.162:0/2565198630 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4954055500 0x7f49540579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:52.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.055+0000 7f496f7fe700 1 --2- 172.21.15.162:0/2565198630 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4954055500 0x7f49540579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:52.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.055+0000 7f496d7fa700 1 -- 172.21.15.162:0/2565198630 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f496806de20 con 0x7f49700b2d00 2020-11-28T21:46:52.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.055+0000 7f496f7fe700 1 --2- 172.21.15.162:0/2565198630 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4954055500 0x7f49540579c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f495801a000 tx=0x7f4958015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:52.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.063+0000 7f496d7fa700 1 -- 172.21.15.162:0/2565198630 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4968044120 con 0x7f49700b2d00 2020-11-28T21:46:52.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.203+0000 7f4975469700 1 -- 172.21.15.162:0/2565198630 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4970057050 con 0x7f49700b2d00 2020-11-28T21:46:52.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.207+0000 7f496d7fa700 1 -- 172.21.15.162:0/2565198630 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4968047740 con 0x7f49700b2d00 2020-11-28T21:46:52.208 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:46:52.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.207+0000 7f4975469700 1 -- 172.21.15.162:0/2565198630 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4954055500 msgr2=0x7f49540579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:52.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.207+0000 7f4975469700 1 --2- 172.21.15.162:0/2565198630 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4954055500 0x7f49540579c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f495801a000 tx=0x7f4958015000).stop 2020-11-28T21:46:52.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.207+0000 7f4975469700 1 -- 172.21.15.162:0/2565198630 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49700b2d00 msgr2=0x7f497013ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:52.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.207+0000 7f4975469700 1 --2- 172.21.15.162:0/2565198630 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49700b2d00 0x7f497013ef70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f49680018b0 tx=0x7f4968001770).stop 2020-11-28T21:46:52.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.207+0000 7f4975469700 1 -- 172.21.15.162:0/2565198630 shutdown_connections 2020-11-28T21:46:52.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.207+0000 7f4975469700 1 --2- 172.21.15.162:0/2565198630 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4954055500 0x7f49540579c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:52.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.207+0000 7f4975469700 1 --2- 172.21.15.162:0/2565198630 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49700b3ab0 0x7f49701425e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:52.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.207+0000 7f4975469700 1 --2- 172.21.15.162:0/2565198630 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49700b2d00 0x7f497013ef70 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:52.212 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.207+0000 7f4975469700 1 --2- 172.21.15.162:0/2565198630 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f49700b30e0 0x7f4970140970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:52.212 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.207+0000 7f4975469700 1 -- 172.21.15.162:0/2565198630 >> 172.21.15.162:0/2565198630 conn(0x7f49700604c0 msgr2=0x7f49700b5800 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:52.212 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.207+0000 7f4975469700 1 -- 172.21.15.162:0/2565198630 shutdown_connections 2020-11-28T21:46:52.212 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.207+0000 7f4975469700 1 -- 172.21.15.162:0/2565198630 wait complete. 2020-11-28T21:46:52.221 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-28T21:46:52.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.335+0000 7f01f4809700 1 Processor -- start 2020-11-28T21:46:52.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.335+0000 7f01f4809700 1 -- start start 2020-11-28T21:46:52.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.335+0000 7f01f4809700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f01f00b2d00 0x7f01f005c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:52.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.335+0000 7f01f4809700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f01f00b3ab0 0x7f01f00b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:52.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.335+0000 7f01f4809700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01f0142ad0 0x7f01f0142f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:52.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.335+0000 7f01f4809700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01f00b0e20 con 0x7f01f00b2d00 2020-11-28T21:46:52.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.335+0000 7f01f4809700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01f005baf0 con 0x7f01f00b3ab0 2020-11-28T21:46:52.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.335+0000 7f01f4809700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f005bd90 con 0x7f01f0142ad0 2020-11-28T21:46:52.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.335+0000 7f01ee59c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f01f00b3ab0 0x7f01f00b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:52.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.335+0000 7f01ee59c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f01f00b3ab0 0x7f01f00b30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38050/0 (socket says 172.21.15.162:38050) 2020-11-28T21:46:52.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.335+0000 7f01ee59c700 1 -- 172.21.15.162:0/2696733728 learned_addr learned my addr 172.21.15.162:0/2696733728 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:52.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.335+0000 7f01ef59e700 1 --2- 172.21.15.162:0/2696733728 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01f0142ad0 0x7f01f0142f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:52.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.335+0000 7f01eed9d700 1 --2- 172.21.15.162:0/2696733728 >> v2:172.21.15.118:3300/0 conn(0x7f01f00b2d00 0x7f01f005c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:52.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.335+0000 7f01ee59c700 1 -- 172.21.15.162:0/2696733728 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01f0142ad0 msgr2=0x7f01f0142f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:52.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.335+0000 7f01ee59c700 1 --2- 172.21.15.162:0/2696733728 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01f0142ad0 0x7f01f0142f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:52.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.335+0000 7f01ee59c700 1 -- 172.21.15.162:0/2696733728 >> v2:172.21.15.118:3300/0 conn(0x7f01f00b2d00 msgr2=0x7f01f005c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:52.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.335+0000 7f01ee59c700 1 --2- 172.21.15.162:0/2696733728 >> v2:172.21.15.118:3300/0 conn(0x7f01f00b2d00 0x7f01f005c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:52.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.335+0000 7f01ee59c700 1 -- 172.21.15.162:0/2696733728 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f005c030 con 0x7f01f00b3ab0 2020-11-28T21:46:52.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.335+0000 7f01ee59c700 1 --2- 172.21.15.162:0/2696733728 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01f00b3ab0 0x7f01f00b30e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f01e0002f30 tx=0x7f01e0006e80).ready entity=mon.1 client_cookie=d8be57c28df15489 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:52.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.335+0000 7f01edd9b700 1 -- 172.21.15.162:0/2696733728 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01e00097f0 con 0x7f01f00b3ab0 2020-11-28T21:46:52.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.335+0000 7f01edd9b700 1 -- 172.21.15.162:0/2696733728 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e0009950 con 0x7f01f00b3ab0 2020-11-28T21:46:52.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.335+0000 7f01edd9b700 1 -- 172.21.15.162:0/2696733728 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01e0009cb0 con 0x7f01f00b3ab0 2020-11-28T21:46:52.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.335+0000 7f01f4809700 1 -- 172.21.15.162:0/2696733728 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01f00b3ab0 msgr2=0x7f01f00b30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:52.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.335+0000 7f01f4809700 1 --2- 172.21.15.162:0/2696733728 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01f00b3ab0 0x7f01f00b30e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f01e0002f30 tx=0x7f01e0006e80).stop 2020-11-28T21:46:52.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.339+0000 7f01f4809700 1 -- 172.21.15.162:0/2696733728 shutdown_connections 2020-11-28T21:46:52.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.339+0000 7f01f4809700 1 --2- 172.21.15.162:0/2696733728 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01f0142ad0 0x7f01f0142f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:52.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.339+0000 7f01f4809700 1 --2- 172.21.15.162:0/2696733728 >> v2:172.21.15.118:3300/0 conn(0x7f01f00b2d00 0x7f01f005c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:52.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.339+0000 7f01f4809700 1 --2- 172.21.15.162:0/2696733728 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01f00b3ab0 0x7f01f00b30e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:52.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.339+0000 7f01f4809700 1 -- 172.21.15.162:0/2696733728 >> 172.21.15.162:0/2696733728 conn(0x7f01f00604c0 msgr2=0x7f01f00608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:52.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.339+0000 7f01f4809700 1 -- 172.21.15.162:0/2696733728 shutdown_connections 2020-11-28T21:46:52.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.339+0000 7f01f4809700 1 -- 172.21.15.162:0/2696733728 wait complete. 2020-11-28T21:46:52.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.339+0000 7f01f4809700 1 Processor -- start 2020-11-28T21:46:52.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.339+0000 7f01f4809700 1 -- start start 2020-11-28T21:46:52.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.343+0000 7f01f4809700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01f00b2d00 0x7f01f00b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:52.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.343+0000 7f01f4809700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f01f00b3ab0 0x7f01f00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:52.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.343+0000 7f01f4809700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01f0142ad0 0x7f01f01364f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:52.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.343+0000 7f01f4809700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f0135600 con 0x7f01f00b3ab0 2020-11-28T21:46:52.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.343+0000 7f01f4809700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f0136010 con 0x7f01f0142ad0 2020-11-28T21:46:52.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.343+0000 7f01f4809700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f0135830 con 0x7f01f00b2d00 2020-11-28T21:46:52.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.343+0000 7f01ef59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01f0142ad0 0x7f01f01364f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:52.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.343+0000 7f01ef59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01f0142ad0 0x7f01f01364f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38056/0 (socket says 172.21.15.162:38056) 2020-11-28T21:46:52.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.343+0000 7f01ef59e700 1 -- 172.21.15.162:0/2094028059 learned_addr learned my addr 172.21.15.162:0/2094028059 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:52.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.343+0000 7f01eed9d700 1 --2- 172.21.15.162:0/2094028059 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01f00b2d00 0x7f01f00b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:52.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.343+0000 7f01ee59c700 1 --2- 172.21.15.162:0/2094028059 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f01f00b3ab0 0x7f01f00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:52.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.343+0000 7f01ef59e700 1 -- 172.21.15.162:0/2094028059 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01f00b2d00 msgr2=0x7f01f00b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:52.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.343+0000 7f01ef59e700 1 --2- 172.21.15.162:0/2094028059 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01f00b2d00 0x7f01f00b5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:52.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.343+0000 7f01ef59e700 1 -- 172.21.15.162:0/2094028059 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f01f00b3ab0 msgr2=0x7f01f00b4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:52.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.343+0000 7f01ef59e700 1 --2- 172.21.15.162:0/2094028059 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f01f00b3ab0 0x7f01f00b4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:52.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.343+0000 7f01ef59e700 1 -- 172.21.15.162:0/2094028059 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f0135ad0 con 0x7f01f0142ad0 2020-11-28T21:46:52.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.343+0000 7f01ef59e700 1 --2- 172.21.15.162:0/2094028059 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01f0142ad0 0x7f01f01364f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f01f00b4540 tx=0x7f01e8008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:52.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.343+0000 7f01e7fff700 1 -- 172.21.15.162:0/2094028059 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01e800b9a0 con 0x7f01f0142ad0 2020-11-28T21:46:52.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.343+0000 7f01e7fff700 1 -- 172.21.15.162:0/2094028059 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e800bb00 con 0x7f01f0142ad0 2020-11-28T21:46:52.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.343+0000 7f01f4809700 1 -- 172.21.15.162:0/2094028059 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f0135d70 con 0x7f01f0142ad0 2020-11-28T21:46:52.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.343+0000 7f01e7fff700 1 -- 172.21.15.162:0/2094028059 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01e800be60 con 0x7f01f0142ad0 2020-11-28T21:46:52.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.343+0000 7f01f4809700 1 -- 172.21.15.162:0/2094028059 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f0136c00 con 0x7f01f0142ad0 2020-11-28T21:46:52.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.347+0000 7f01f4809700 1 -- 172.21.15.162:0/2094028059 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01f0057050 con 0x7f01f0142ad0 2020-11-28T21:46:52.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.347+0000 7f01e7fff700 1 -- 172.21.15.162:0/2094028059 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f01e801c460 con 0x7f01f0142ad0 2020-11-28T21:46:52.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.347+0000 7f01e7fff700 1 --2- 172.21.15.162:0/2094028059 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f01d4055440 0x7f01d4057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:52.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.347+0000 7f01e7fff700 1 -- 172.21.15.162:0/2094028059 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f01e806e5b0 con 0x7f01f0142ad0 2020-11-28T21:46:52.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.347+0000 7f01eed9d700 1 --2- 172.21.15.162:0/2094028059 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f01d4055440 0x7f01d4057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:52.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.347+0000 7f01eed9d700 1 --2- 172.21.15.162:0/2094028059 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f01d4055440 0x7f01d4057900 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f01d800c000 tx=0x7f01d8008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:52.355 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.351+0000 7f01e7fff700 1 -- 172.21.15.162:0/2094028059 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f01e80447b0 con 0x7f01f0142ad0 2020-11-28T21:46:52.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.495+0000 7f01f4809700 1 -- 172.21.15.162:0/2094028059 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f01f00b0b30 con 0x7f01f0142ad0 2020-11-28T21:46:52.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.495+0000 7f01e7fff700 1 -- 172.21.15.162:0/2094028059 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+10 (secure 0 0 0) 0x7f01e8047dd0 con 0x7f01f0142ad0 2020-11-28T21:46:52.497 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:46:52.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.495+0000 7f01f4809700 1 -- 172.21.15.162:0/2094028059 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f01d4055440 msgr2=0x7f01d4057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:52.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.495+0000 7f01f4809700 1 --2- 172.21.15.162:0/2094028059 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f01d4055440 0x7f01d4057900 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f01d800c000 tx=0x7f01d8008040).stop 2020-11-28T21:46:52.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.495+0000 7f01f4809700 1 -- 172.21.15.162:0/2094028059 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01f0142ad0 msgr2=0x7f01f01364f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:52.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.495+0000 7f01f4809700 1 --2- 172.21.15.162:0/2094028059 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01f0142ad0 0x7f01f01364f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f01f00b4540 tx=0x7f01e8008040).stop 2020-11-28T21:46:52.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.495+0000 7f01f4809700 1 -- 172.21.15.162:0/2094028059 shutdown_connections 2020-11-28T21:46:52.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.495+0000 7f01f4809700 1 --2- 172.21.15.162:0/2094028059 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f01d4055440 0x7f01d4057900 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:52.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.495+0000 7f01f4809700 1 --2- 172.21.15.162:0/2094028059 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01f00b2d00 0x7f01f00b5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:52.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.495+0000 7f01f4809700 1 --2- 172.21.15.162:0/2094028059 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01f0142ad0 0x7f01f01364f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:52.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.495+0000 7f01f4809700 1 --2- 172.21.15.162:0/2094028059 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f01f00b3ab0 0x7f01f00b4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:52.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.495+0000 7f01f4809700 1 -- 172.21.15.162:0/2094028059 >> 172.21.15.162:0/2094028059 conn(0x7f01f00604c0 msgr2=0x7f01f00608a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:52.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.495+0000 7f01f4809700 1 -- 172.21.15.162:0/2094028059 shutdown_connections 2020-11-28T21:46:52.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.495+0000 7f01f4809700 1 -- 172.21.15.162:0/2094028059 wait complete. 2020-11-28T21:46:52.511 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-28T21:46:52.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.627+0000 7fd76a803700 1 Processor -- start 2020-11-28T21:46:52.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.627+0000 7fd76a803700 1 -- start start 2020-11-28T21:46:52.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.627+0000 7fd76a803700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd7640b3c90 0x7fd764142a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:52.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.627+0000 7fd76a803700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd76405f380 0x7fd764146050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:52.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.627+0000 7fd76a803700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7640aac00 con 0x7fd7640b18c0 2020-11-28T21:46:52.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.627+0000 7fd76a803700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd76413cf80 con 0x7fd7640b3c90 2020-11-28T21:46:52.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.627+0000 7fd76a803700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7640b4070 con 0x7fd76405f380 2020-11-28T21:46:52.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.627+0000 7fd769000700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fd7640b18c0 0x7fd764142f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:44906/0 (socket says 172.21.15.162:44906) 2020-11-28T21:46:52.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.627+0000 7fd769801700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd7640b3c90 0x7fd764142a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:52.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.627+0000 7fd769000700 1 -- 172.21.15.162:0/1910280789 learned_addr learned my addr 172.21.15.162:0/1910280789 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:52.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.627+0000 7fd76a002700 1 --2- 172.21.15.162:0/1910280789 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd76405f380 0x7fd764146050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:52.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.627+0000 7fd763fff700 1 -- 172.21.15.162:0/1910280789 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1005406112 0 0) 0x7fd7640aac00 con 0x7fd7640b18c0 2020-11-28T21:46:52.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.627+0000 7fd763fff700 1 -- 172.21.15.162:0/1910280789 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd750003230 con 0x7fd7640b18c0 2020-11-28T21:46:52.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.627+0000 7fd769801700 1 -- 172.21.15.162:0/1910280789 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd76405f380 msgr2=0x7fd764146050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:52.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.627+0000 7fd769801700 1 --2- 172.21.15.162:0/1910280789 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd76405f380 0x7fd764146050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:52.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.627+0000 7fd769801700 1 -- 172.21.15.162:0/1910280789 >> v1:172.21.15.162:6789/0 conn(0x7fd7640b18c0 legacy=0x7fd764142f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:52.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.631+0000 7fd769801700 1 -- 172.21.15.162:0/1910280789 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd764060a10 con 0x7fd7640b3c90 2020-11-28T21:46:52.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.631+0000 7fd769801700 1 --2- 172.21.15.162:0/1910280789 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7640b3c90 0x7fd764142a80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7fd754002f30 tx=0x7fd754006dc0).ready entity=mon.1 client_cookie=14ef3736f6677206 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:52.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.631+0000 7fd763fff700 1 -- 172.21.15.162:0/1910280789 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7540096a0 con 0x7fd7640b3c90 2020-11-28T21:46:52.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.631+0000 7fd763fff700 1 -- 172.21.15.162:0/1910280789 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd754009800 con 0x7fd7640b3c90 2020-11-28T21:46:52.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.631+0000 7fd763fff700 1 -- 172.21.15.162:0/1910280789 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd754009b60 con 0x7fd7640b3c90 2020-11-28T21:46:52.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.631+0000 7fd76a803700 1 -- 172.21.15.162:0/1910280789 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7640b3c90 msgr2=0x7fd764142a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:52.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.631+0000 7fd76a803700 1 --2- 172.21.15.162:0/1910280789 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7640b3c90 0x7fd764142a80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7fd754002f30 tx=0x7fd754006dc0).stop 2020-11-28T21:46:52.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.631+0000 7fd76a803700 1 -- 172.21.15.162:0/1910280789 shutdown_connections 2020-11-28T21:46:52.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.631+0000 7fd76a803700 1 --2- 172.21.15.162:0/1910280789 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7640b3c90 0x7fd764142a80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:52.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.631+0000 7fd76a803700 1 --2- 172.21.15.162:0/1910280789 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd76405f380 0x7fd764146050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:52.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.631+0000 7fd76a803700 1 -- 172.21.15.162:0/1910280789 >> 172.21.15.162:0/1910280789 conn(0x7fd764062630 msgr2=0x7fd764062a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:52.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.631+0000 7fd76a803700 1 -- 172.21.15.162:0/1910280789 shutdown_connections 2020-11-28T21:46:52.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.631+0000 7fd76a803700 1 -- 172.21.15.162:0/1910280789 wait complete. 2020-11-28T21:46:52.639 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.631+0000 7fd76a803700 1 Processor -- start 2020-11-28T21:46:52.639 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.631+0000 7fd76a803700 1 -- start start 2020-11-28T21:46:52.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.631+0000 7fd76a803700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd76405f380 0x7fd764139a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:52.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.631+0000 7fd76a803700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7640b18c0 0x7fd764137360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:52.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.631+0000 7fd76a803700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd7640b3c90 0x7fd764137850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:52.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.631+0000 7fd76a803700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd764146ee0 con 0x7fd7640b3c90 2020-11-28T21:46:52.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.631+0000 7fd76a803700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7641380d0 con 0x7fd7640b18c0 2020-11-28T21:46:52.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.631+0000 7fd76a803700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd76413a440 con 0x7fd76405f380 2020-11-28T21:46:52.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.635+0000 7fd769000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7640b18c0 0x7fd764137360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:52.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.635+0000 7fd769000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7640b18c0 0x7fd764137360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38068/0 (socket says 172.21.15.162:38068) 2020-11-28T21:46:52.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.635+0000 7fd769000700 1 -- 172.21.15.162:0/4127004287 learned_addr learned my addr 172.21.15.162:0/4127004287 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:52.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.635+0000 7fd76a002700 1 --2- 172.21.15.162:0/4127004287 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd7640b3c90 0x7fd764137850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:52.642 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.635+0000 7fd769801700 1 --2- 172.21.15.162:0/4127004287 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd76405f380 0x7fd764139a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:52.642 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.635+0000 7fd769000700 1 -- 172.21.15.162:0/4127004287 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd76405f380 msgr2=0x7fd764139a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:52.642 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.635+0000 7fd769000700 1 --2- 172.21.15.162:0/4127004287 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd76405f380 0x7fd764139a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:52.642 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.635+0000 7fd769000700 1 -- 172.21.15.162:0/4127004287 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd7640b3c90 msgr2=0x7fd764137850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:52.642 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.635+0000 7fd769000700 1 --2- 172.21.15.162:0/4127004287 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd7640b3c90 0x7fd764137850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:52.643 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.635+0000 7fd769000700 1 -- 172.21.15.162:0/4127004287 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd764146c40 con 0x7fd7640b18c0 2020-11-28T21:46:52.643 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.635+0000 7fd769000700 1 --2- 172.21.15.162:0/4127004287 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7640b18c0 0x7fd764137360 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7fd758001450 tx=0x7fd758007e20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:52.643 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.635+0000 7fd7627fc700 1 -- 172.21.15.162:0/4127004287 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd758005b70 con 0x7fd7640b18c0 2020-11-28T21:46:52.643 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.635+0000 7fd7627fc700 1 -- 172.21.15.162:0/4127004287 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd758005cd0 con 0x7fd7640b18c0 2020-11-28T21:46:52.643 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.635+0000 7fd76a803700 1 -- 172.21.15.162:0/4127004287 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd764138360 con 0x7fd7640b18c0 2020-11-28T21:46:52.644 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.635+0000 7fd7627fc700 1 -- 172.21.15.162:0/4127004287 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd758007300 con 0x7fd7640b18c0 2020-11-28T21:46:52.644 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.635+0000 7fd76a803700 1 -- 172.21.15.162:0/4127004287 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7641434a0 con 0x7fd7640b18c0 2020-11-28T21:46:52.644 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.635+0000 7fd76a803700 1 -- 172.21.15.162:0/4127004287 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd764057050 con 0x7fd7640b18c0 2020-11-28T21:46:52.644 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.635+0000 7fd7627fc700 1 -- 172.21.15.162:0/4127004287 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd758010030 con 0x7fd7640b18c0 2020-11-28T21:46:52.645 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.639+0000 7fd7627fc700 1 --2- 172.21.15.162:0/4127004287 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd744055440 0x7fd744057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:52.645 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.639+0000 7fd7627fc700 1 -- 172.21.15.162:0/4127004287 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd75806e020 con 0x7fd7640b18c0 2020-11-28T21:46:52.645 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.639+0000 7fd769801700 1 --2- 172.21.15.162:0/4127004287 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd744055440 0x7fd744057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:52.646 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.639+0000 7fd769801700 1 --2- 172.21.15.162:0/4127004287 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd744055440 0x7fd744057900 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fd754019000 tx=0x7fd754015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:52.646 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.643+0000 7fd7627fc700 1 -- 172.21.15.162:0/4127004287 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd758044320 con 0x7fd7640b18c0 2020-11-28T21:46:52.788 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.787+0000 7fd76a803700 1 -- 172.21.15.162:0/4127004287 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd764060030 con 0x7fd7640b18c0 2020-11-28T21:46:52.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.787+0000 7fd7627fc700 1 -- 172.21.15.162:0/4127004287 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+10 (secure 0 0 0) 0x7fd758047940 con 0x7fd7640b18c0 2020-11-28T21:46:52.789 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:46:52.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.787+0000 7fd76a803700 1 -- 172.21.15.162:0/4127004287 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd744055440 msgr2=0x7fd744057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:52.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.787+0000 7fd76a803700 1 --2- 172.21.15.162:0/4127004287 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd744055440 0x7fd744057900 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fd754019000 tx=0x7fd754015000).stop 2020-11-28T21:46:52.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.787+0000 7fd76a803700 1 -- 172.21.15.162:0/4127004287 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7640b18c0 msgr2=0x7fd764137360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:52.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.787+0000 7fd76a803700 1 --2- 172.21.15.162:0/4127004287 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7640b18c0 0x7fd764137360 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7fd758001450 tx=0x7fd758007e20).stop 2020-11-28T21:46:52.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.787+0000 7fd76a803700 1 -- 172.21.15.162:0/4127004287 shutdown_connections 2020-11-28T21:46:52.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.787+0000 7fd76a803700 1 --2- 172.21.15.162:0/4127004287 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd744055440 0x7fd744057900 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:52.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.787+0000 7fd76a803700 1 --2- 172.21.15.162:0/4127004287 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd76405f380 0x7fd764139a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:52.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.787+0000 7fd76a803700 1 --2- 172.21.15.162:0/4127004287 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd7640b3c90 0x7fd764137850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:52.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.787+0000 7fd76a803700 1 --2- 172.21.15.162:0/4127004287 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7640b18c0 0x7fd764137360 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:52.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.787+0000 7fd76a803700 1 -- 172.21.15.162:0/4127004287 >> 172.21.15.162:0/4127004287 conn(0x7fd764062630 msgr2=0x7fd7640b53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:52.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.787+0000 7fd76a803700 1 -- 172.21.15.162:0/4127004287 shutdown_connections 2020-11-28T21:46:52.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.787+0000 7fd76a803700 1 -- 172.21.15.162:0/4127004287 wait complete. 2020-11-28T21:46:52.803 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:46:52.905 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.903+0000 7fdab4a4f700 1 Processor -- start 2020-11-28T21:46:52.905 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.903+0000 7fdab4a4f700 1 -- start start 2020-11-28T21:46:52.908 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.903+0000 7fdab4a4f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdab00b5a70 0x7fdab0143360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:52.908 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.903+0000 7fdab4a4f700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fdab00b5430 0x7fdab009d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:52.909 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.903+0000 7fdab4a4f700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdab00b8040 con 0x7fdab00b5e50 2020-11-28T21:46:52.909 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.903+0000 7fdab4a4f700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdab00b7600 con 0x7fdab00b5430 2020-11-28T21:46:52.909 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.903+0000 7fdab4a4f700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdab00b8720 con 0x7fdab00b5a70 2020-11-28T21:46:52.909 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.903+0000 7fdaaeffd700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fdab00b5e50 0x7fdab00b6ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:44918/0 (socket says 172.21.15.162:44918) 2020-11-28T21:46:52.909 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.903+0000 7fdaaeffd700 1 -- 172.21.15.162:0/3096590732 learned_addr learned my addr 172.21.15.162:0/3096590732 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:52.910 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.903+0000 7fdaaf7fe700 1 --2- 172.21.15.162:0/3096590732 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdab00b5a70 0x7fdab0143360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:52.910 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.903+0000 7fdaaffff700 1 --2- 172.21.15.162:0/3096590732 >> v2:172.21.15.118:3300/0 conn(0x7fdab00b5430 0x7fdab009d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:52.910 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.903+0000 7fdaae7fc700 1 -- 172.21.15.162:0/3096590732 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3364384537 0 0) 0x7fdab00b8040 con 0x7fdab00b5e50 2020-11-28T21:46:52.910 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.903+0000 7fdaae7fc700 1 -- 172.21.15.162:0/3096590732 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda90003230 con 0x7fdab00b5e50 2020-11-28T21:46:52.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.903+0000 7fdaae7fc700 1 -- 172.21.15.162:0/3096590732 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 977243822 0 0) 0x7fda90003230 con 0x7fdab00b5e50 2020-11-28T21:46:52.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.903+0000 7fdaae7fc700 1 -- 172.21.15.162:0/3096590732 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdab00b5a70 msgr2=0x7fdab0143360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:52.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.903+0000 7fdaae7fc700 1 --2- 172.21.15.162:0/3096590732 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdab00b5a70 0x7fdab0143360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:52.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.903+0000 7fdaae7fc700 1 -- 172.21.15.162:0/3096590732 >> v2:172.21.15.118:3300/0 conn(0x7fdab00b5430 msgr2=0x7fdab009d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:52.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.903+0000 7fdaae7fc700 1 --2- 172.21.15.162:0/3096590732 >> v2:172.21.15.118:3300/0 conn(0x7fdab00b5430 0x7fdab009d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:52.912 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.903+0000 7fdaae7fc700 1 -- 172.21.15.162:0/3096590732 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab0142980 con 0x7fdab00b5e50 2020-11-28T21:46:52.912 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.903+0000 7fdaae7fc700 1 -- 172.21.15.162:0/3096590732 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7fda980016d0 con 0x7fdab00b5e50 2020-11-28T21:46:52.912 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.903+0000 7fdaae7fc700 1 -- 172.21.15.162:0/3096590732 >> v1:172.21.15.162:6789/0 conn(0x7fdab00b5e50 legacy=0x7fdab00b6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:52.912 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.903+0000 7fdaae7fc700 1 --2- 172.21.15.162:0/3096590732 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdab00b5a70 0x7fda90003720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:52.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.903+0000 7fdaae7fc700 1 --2- 172.21.15.162:0/3096590732 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fda90003c30 0x7fda900044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:52.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.903+0000 7fdaae7fc700 1 --2- 172.21.15.162:0/3096590732 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda900049c0 0x7fda90004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:52.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.903+0000 7fdaae7fc700 1 -- 172.21.15.162:0/3096590732 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab00b7600 con 0x7fda90003c30 2020-11-28T21:46:52.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.903+0000 7fdaae7fc700 1 -- 172.21.15.162:0/3096590732 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab00b8720 con 0x7fda900049c0 2020-11-28T21:46:52.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.903+0000 7fdaae7fc700 1 -- 172.21.15.162:0/3096590732 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda900042a0 con 0x7fdab00b5a70 2020-11-28T21:46:52.914 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.903+0000 7fdaaffff700 1 --2- 172.21.15.162:0/3096590732 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda900049c0 0x7fda90004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:52.914 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.903+0000 7fdaaf7fe700 1 --2- 172.21.15.162:0/3096590732 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdab00b5a70 0x7fda90003720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:52.914 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.903+0000 7fdaaeffd700 1 --2- 172.21.15.162:0/3096590732 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fda90003c30 0x7fda900044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:52.914 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.903+0000 7fdaaffff700 1 -- 172.21.15.162:0/3096590732 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdab00b5a70 msgr2=0x7fda90003720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:52.915 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.903+0000 7fdaaffff700 1 --2- 172.21.15.162:0/3096590732 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdab00b5a70 0x7fda90003720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:52.915 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.903+0000 7fdaaffff700 1 -- 172.21.15.162:0/3096590732 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fda90003c30 msgr2=0x7fda900044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:52.915 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.903+0000 7fdaaffff700 1 --2- 172.21.15.162:0/3096590732 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fda90003c30 0x7fda900044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:52.915 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.903+0000 7fdaaffff700 1 -- 172.21.15.162:0/3096590732 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda90003230 con 0x7fda900049c0 2020-11-28T21:46:52.916 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.903+0000 7fdaaf7fe700 1 --2- 172.21.15.162:0/3096590732 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdab00b5a70 0x7fda90003720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:46:52.916 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.907+0000 7fdaaffff700 1 --2- 172.21.15.162:0/3096590732 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda900049c0 0x7fda90004e40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7fdaa800c000 tx=0x7fdaa80091a0).ready entity=mon.1 client_cookie=afe5d4ff747111f9 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:52.916 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.907+0000 7fdaae7fc700 1 -- 172.21.15.162:0/3096590732 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdaa800ed00 con 0x7fda900049c0 2020-11-28T21:46:52.916 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.907+0000 7fdaae7fc700 1 -- 172.21.15.162:0/3096590732 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaa800ee60 con 0x7fda900049c0 2020-11-28T21:46:52.916 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.907+0000 7fdaae7fc700 1 -- 172.21.15.162:0/3096590732 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdaa80125e0 con 0x7fda900049c0 2020-11-28T21:46:52.917 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.907+0000 7fdab4a4f700 1 -- 172.21.15.162:0/3096590732 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda900049c0 msgr2=0x7fda90004e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:52.917 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.907+0000 7fdab4a4f700 1 --2- 172.21.15.162:0/3096590732 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda900049c0 0x7fda90004e40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7fdaa800c000 tx=0x7fdaa80091a0).stop 2020-11-28T21:46:52.917 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.907+0000 7fdaaffff700 1 -- 172.21.15.162:0/3096590732 reap_dead start 2020-11-28T21:46:52.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.907+0000 7fdab4a4f700 1 -- 172.21.15.162:0/3096590732 shutdown_connections 2020-11-28T21:46:52.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.907+0000 7fdab4a4f700 1 -- 172.21.15.162:0/3096590732 >> 172.21.15.162:0/3096590732 conn(0x7fdab005b970 msgr2=0x7fdab005bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:52.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.907+0000 7fdab4a4f700 1 -- 172.21.15.162:0/3096590732 shutdown_connections 2020-11-28T21:46:52.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.907+0000 7fdab4a4f700 1 -- 172.21.15.162:0/3096590732 wait complete. 2020-11-28T21:46:52.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.907+0000 7fdab4a4f700 1 Processor -- start 2020-11-28T21:46:52.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.907+0000 7fdab4a4f700 1 -- start start 2020-11-28T21:46:52.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.911+0000 7fdab4a4f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdab0143fb0 0x7fdab0144390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:52.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.911+0000 7fdab4a4f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdab0147640 0x7fdab0144a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:52.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.911+0000 7fdab4a4f700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdab0144f80 0x7fdab009e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:52.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.911+0000 7fdab4a4f700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab01529a0 con 0x7fdab0144f80 2020-11-28T21:46:52.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.911+0000 7fdab4a4f700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab006a530 con 0x7fdab0143fb0 2020-11-28T21:46:52.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.911+0000 7fdab4a4f700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdab0152700 con 0x7fdab0147640 2020-11-28T21:46:52.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.911+0000 7fdaaf7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdab0143fb0 0x7fdab0144390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:52.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.911+0000 7fdaaf7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdab0143fb0 0x7fdab0144390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38086/0 (socket says 172.21.15.162:38086) 2020-11-28T21:46:52.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.911+0000 7fdaaf7fe700 1 -- 172.21.15.162:0/2583338151 learned_addr learned my addr 172.21.15.162:0/2583338151 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:52.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.911+0000 7fdaaeffd700 1 --2- 172.21.15.162:0/2583338151 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdab0147640 0x7fdab0144a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:52.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.911+0000 7fdaaffff700 1 --2- 172.21.15.162:0/2583338151 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdab0144f80 0x7fdab009e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:52.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.911+0000 7fdaaf7fe700 1 -- 172.21.15.162:0/2583338151 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdab0147640 msgr2=0x7fdab0144a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:52.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.911+0000 7fdaaf7fe700 1 --2- 172.21.15.162:0/2583338151 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdab0147640 0x7fdab0144a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:52.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.911+0000 7fdaaf7fe700 1 -- 172.21.15.162:0/2583338151 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdab0144f80 msgr2=0x7fdab009e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:52.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.911+0000 7fdaaf7fe700 1 --2- 172.21.15.162:0/2583338151 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdab0144f80 0x7fdab009e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:52.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.911+0000 7fdaaf7fe700 1 -- 172.21.15.162:0/2583338151 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab0146f00 con 0x7fdab0143fb0 2020-11-28T21:46:52.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.911+0000 7fdaaffff700 1 --2- 172.21.15.162:0/2583338151 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdab0144f80 0x7fdab009e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:46:52.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.911+0000 7fdaaf7fe700 1 --2- 172.21.15.162:0/2583338151 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdab0143fb0 0x7fdab0144390 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7fdaa0008000 tx=0x7fdaa000b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:52.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.911+0000 7fdaacff9700 1 -- 172.21.15.162:0/2583338151 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdaa0011860 con 0x7fdab0143fb0 2020-11-28T21:46:52.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.911+0000 7fdaacff9700 1 -- 172.21.15.162:0/2583338151 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaa00119c0 con 0x7fdab0143fb0 2020-11-28T21:46:52.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.911+0000 7fdaacff9700 1 -- 172.21.15.162:0/2583338151 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdaa0011d20 con 0x7fdab0143fb0 2020-11-28T21:46:52.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.911+0000 7fdab4a4f700 1 -- 172.21.15.162:0/2583338151 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdab01471a0 con 0x7fdab0143fb0 2020-11-28T21:46:52.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.911+0000 7fdab4a4f700 1 -- 172.21.15.162:0/2583338151 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdab00b3260 con 0x7fdab0143fb0 2020-11-28T21:46:52.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.911+0000 7fdab4a4f700 1 -- 172.21.15.162:0/2583338151 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdab0057050 con 0x7fdab0143fb0 2020-11-28T21:46:52.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.915+0000 7fdaacff9700 1 -- 172.21.15.162:0/2583338151 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fdaa0014460 con 0x7fdab0143fb0 2020-11-28T21:46:52.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.915+0000 7fdaacff9700 1 --2- 172.21.15.162:0/2583338151 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fda940554e0 0x7fda940579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:52.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.915+0000 7fdaacff9700 1 -- 172.21.15.162:0/2583338151 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdaa006dda0 con 0x7fdab0143fb0 2020-11-28T21:46:52.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.915+0000 7fdaaeffd700 1 --2- 172.21.15.162:0/2583338151 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fda940554e0 0x7fda940579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:52.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.915+0000 7fdaaeffd700 1 --2- 172.21.15.162:0/2583338151 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fda940554e0 0x7fda940579a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fda9800d000 tx=0x7fda98007040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:52.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:52.919+0000 7fdaacff9700 1 -- 172.21.15.162:0/2583338151 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fdaa0044040 con 0x7fdab0143fb0 2020-11-28T21:46:53.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.055+0000 7fdab4a4f700 1 -- 172.21.15.162:0/2583338151 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdab009f240 con 0x7fdab0143fb0 2020-11-28T21:46:53.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.059+0000 7fdaacff9700 1 -- 172.21.15.162:0/2583338151 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16134 (secure 0 0 0) 0x7fdaa0047660 con 0x7fdab0143fb0 2020-11-28T21:46:53.061 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:46:53.061 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":23,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:46:30.124354+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:46:27.155497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-28T21:46:28.137474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:46:53.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.059+0000 7fdab4a4f700 1 -- 172.21.15.162:0/2583338151 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fda940554e0 msgr2=0x7fda940579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:53.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.059+0000 7fdab4a4f700 1 --2- 172.21.15.162:0/2583338151 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fda940554e0 0x7fda940579a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fda9800d000 tx=0x7fda98007040).stop 2020-11-28T21:46:53.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.059+0000 7fdab4a4f700 1 -- 172.21.15.162:0/2583338151 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdab0143fb0 msgr2=0x7fdab0144390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:53.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.059+0000 7fdab4a4f700 1 --2- 172.21.15.162:0/2583338151 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdab0143fb0 0x7fdab0144390 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7fdaa0008000 tx=0x7fdaa000b040).stop 2020-11-28T21:46:53.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.059+0000 7fdab4a4f700 1 -- 172.21.15.162:0/2583338151 shutdown_connections 2020-11-28T21:46:53.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.059+0000 7fdab4a4f700 1 --2- 172.21.15.162:0/2583338151 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fda940554e0 0x7fda940579a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:53.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.059+0000 7fdab4a4f700 1 --2- 172.21.15.162:0/2583338151 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdab0144f80 0x7fdab009e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:53.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.059+0000 7fdab4a4f700 1 --2- 172.21.15.162:0/2583338151 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdab0143fb0 0x7fdab0144390 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:53.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.059+0000 7fdab4a4f700 1 --2- 172.21.15.162:0/2583338151 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdab0147640 0x7fdab0144a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:53.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.063+0000 7fdab4a4f700 1 -- 172.21.15.162:0/2583338151 >> 172.21.15.162:0/2583338151 conn(0x7fdab005b970 msgr2=0x7fdab0060ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:53.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.063+0000 7fdab4a4f700 1 -- 172.21.15.162:0/2583338151 shutdown_connections 2020-11-28T21:46:53.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.063+0000 7fdab4a4f700 1 -- 172.21.15.162:0/2583338151 wait complete. 2020-11-28T21:46:53.078 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-28T21:46:27.155497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-28T21:46:28.137474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-28T21:46:53.079 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:46:53.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.195+0000 7f6411232700 1 Processor -- start 2020-11-28T21:46:53.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.195+0000 7f6411232700 1 -- start start 2020-11-28T21:46:53.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.195+0000 7f6411232700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f640c13dbe0 0x7f640c0b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:53.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.195+0000 7f6411232700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f640c13dfc0 0x7f640c140590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:53.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.199+0000 7f6411232700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f640c0b0ca0 0x7f640c060920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:53.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.199+0000 7f6411232700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f640c0b0040 con 0x7f640c13dbe0 2020-11-28T21:46:53.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.199+0000 7f6411232700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f640c0b1420 con 0x7f640c0b0ca0 2020-11-28T21:46:53.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.199+0000 7f6411232700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f640c0b16c0 con 0x7f640c13dfc0 2020-11-28T21:46:53.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.199+0000 7f6410a31700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f640c0b0ca0 0x7f640c060920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:53.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.199+0000 7f6410a31700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f640c0b0ca0 0x7f640c060920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38098/0 (socket says 172.21.15.162:38098) 2020-11-28T21:46:53.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.199+0000 7f6410a31700 1 -- 172.21.15.162:0/1779209428 learned_addr learned my addr 172.21.15.162:0/1779209428 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:53.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.199+0000 7f640bfff700 1 --2- 172.21.15.162:0/1779209428 >> v2:172.21.15.118:3300/0 conn(0x7f640c13dbe0 0x7f640c0b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:53.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.199+0000 7f640b7fe700 1 --2- 172.21.15.162:0/1779209428 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f640c13dfc0 0x7f640c140590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:53.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.199+0000 7f6410a31700 1 -- 172.21.15.162:0/1779209428 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f640c13dfc0 msgr2=0x7f640c140590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:53.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.199+0000 7f6410a31700 1 --2- 172.21.15.162:0/1779209428 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f640c13dfc0 0x7f640c140590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:53.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.199+0000 7f6410a31700 1 -- 172.21.15.162:0/1779209428 >> v2:172.21.15.118:3300/0 conn(0x7f640c13dbe0 msgr2=0x7f640c0b20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:53.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.199+0000 7f6410a31700 1 --2- 172.21.15.162:0/1779209428 >> v2:172.21.15.118:3300/0 conn(0x7f640c13dbe0 0x7f640c0b20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:53.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.199+0000 7f6410a31700 1 -- 172.21.15.162:0/1779209428 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f640c0b1960 con 0x7f640c0b0ca0 2020-11-28T21:46:53.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.199+0000 7f6410a31700 1 --2- 172.21.15.162:0/1779209428 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f640c0b0ca0 0x7f640c060920 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f64040032b0 tx=0x7f6404009270).ready entity=mon.1 client_cookie=2528f89f3511e460 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:53.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.199+0000 7f640affd700 1 -- 172.21.15.162:0/1779209428 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6404009e60 con 0x7f640c0b0ca0 2020-11-28T21:46:53.212 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.199+0000 7f640affd700 1 -- 172.21.15.162:0/1779209428 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f640400bc00 con 0x7f640c0b0ca0 2020-11-28T21:46:53.212 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.199+0000 7f640affd700 1 -- 172.21.15.162:0/1779209428 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f640400bd70 con 0x7f640c0b0ca0 2020-11-28T21:46:53.212 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.199+0000 7f6411232700 1 -- 172.21.15.162:0/1779209428 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f640c0b0ca0 msgr2=0x7f640c060920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:53.213 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.199+0000 7f6411232700 1 --2- 172.21.15.162:0/1779209428 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f640c0b0ca0 0x7f640c060920 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f64040032b0 tx=0x7f6404009270).stop 2020-11-28T21:46:53.213 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.199+0000 7f6411232700 1 -- 172.21.15.162:0/1779209428 shutdown_connections 2020-11-28T21:46:53.213 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.199+0000 7f6411232700 1 --2- 172.21.15.162:0/1779209428 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f640c0b0ca0 0x7f640c060920 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:53.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.199+0000 7f6411232700 1 --2- 172.21.15.162:0/1779209428 >> v2:172.21.15.118:3300/0 conn(0x7f640c13dbe0 0x7f640c0b20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:53.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.199+0000 7f6411232700 1 --2- 172.21.15.162:0/1779209428 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f640c13dfc0 0x7f640c140590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:53.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.199+0000 7f6411232700 1 -- 172.21.15.162:0/1779209428 >> 172.21.15.162:0/1779209428 conn(0x7f640c0b2830 msgr2=0x7f640c0b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:53.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.199+0000 7f6411232700 1 -- 172.21.15.162:0/1779209428 shutdown_connections 2020-11-28T21:46:53.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.199+0000 7f6411232700 1 -- 172.21.15.162:0/1779209428 wait complete. 2020-11-28T21:46:53.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.203+0000 7f6411232700 1 Processor -- start 2020-11-28T21:46:53.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.203+0000 7f6411232700 1 -- start start 2020-11-28T21:46:53.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.203+0000 7f6411232700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f640c0b0ca0 0x7f640c13cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:53.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.203+0000 7f6411232700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f640c13dbe0 0x7f640c152930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:53.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.203+0000 7f6411232700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f640c13dfc0 0x7f640c152e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:53.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.203+0000 7f6411232700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640c13f7e0 con 0x7f640c0b0ca0 2020-11-28T21:46:53.224 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.203+0000 7f6411232700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640c1446a0 con 0x7f640c13dbe0 2020-11-28T21:46:53.224 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.203+0000 7f6411232700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f640c13fa10 con 0x7f640c13dfc0 2020-11-28T21:46:53.225 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.203+0000 7f640b7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f640c13dbe0 0x7f640c152930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:53.225 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.203+0000 7f640b7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f640c13dbe0 0x7f640c152930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38104/0 (socket says 172.21.15.162:38104) 2020-11-28T21:46:53.225 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.203+0000 7f640b7fe700 1 -- 172.21.15.162:0/1256643296 learned_addr learned my addr 172.21.15.162:0/1256643296 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:53.225 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.203+0000 7f640bfff700 1 --2- 172.21.15.162:0/1256643296 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f640c0b0ca0 0x7f640c13cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:53.226 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.203+0000 7f6410a31700 1 --2- 172.21.15.162:0/1256643296 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f640c13dfc0 0x7f640c152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:53.226 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.203+0000 7f640b7fe700 1 -- 172.21.15.162:0/1256643296 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f640c13dfc0 msgr2=0x7f640c152e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:53.226 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.203+0000 7f640b7fe700 1 --2- 172.21.15.162:0/1256643296 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f640c13dfc0 0x7f640c152e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:53.227 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.203+0000 7f640b7fe700 1 -- 172.21.15.162:0/1256643296 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f640c0b0ca0 msgr2=0x7f640c13cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:53.227 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.203+0000 7f640b7fe700 1 --2- 172.21.15.162:0/1256643296 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f640c0b0ca0 0x7f640c13cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:53.227 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.203+0000 7f640b7fe700 1 -- 172.21.15.162:0/1256643296 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f640c144940 con 0x7f640c13dbe0 2020-11-28T21:46:53.227 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.203+0000 7f640b7fe700 1 --2- 172.21.15.162:0/1256643296 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f640c13dbe0 0x7f640c152930 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f63f400a000 tx=0x7f63f4008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:53.227 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.203+0000 7f64097fa700 1 -- 172.21.15.162:0/1256643296 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63f4011850 con 0x7f640c13dbe0 2020-11-28T21:46:53.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.203+0000 7f64097fa700 1 -- 172.21.15.162:0/1256643296 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63f40119b0 con 0x7f640c13dbe0 2020-11-28T21:46:53.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.203+0000 7f6411232700 1 -- 172.21.15.162:0/1256643296 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f640c13f3b0 con 0x7f640c13dbe0 2020-11-28T21:46:53.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.203+0000 7f64097fa700 1 -- 172.21.15.162:0/1256643296 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63f4011d10 con 0x7f640c13dbe0 2020-11-28T21:46:53.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.203+0000 7f6411232700 1 -- 172.21.15.162:0/1256643296 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f640c1534b0 con 0x7f640c13dbe0 2020-11-28T21:46:53.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.207+0000 7f6411232700 1 -- 172.21.15.162:0/1256643296 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f640c13fbd0 con 0x7f640c13dbe0 2020-11-28T21:46:53.229 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.207+0000 7f64097fa700 1 -- 172.21.15.162:0/1256643296 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f63f4012420 con 0x7f640c13dbe0 2020-11-28T21:46:53.229 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.207+0000 7f64097fa700 1 --2- 172.21.15.162:0/1256643296 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f63f00553b0 0x7f63f0057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:53.229 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.207+0000 7f64097fa700 1 -- 172.21.15.162:0/1256643296 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f63f406e180 con 0x7f640c13dbe0 2020-11-28T21:46:53.229 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.207+0000 7f640bfff700 1 --2- 172.21.15.162:0/1256643296 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f63f00553b0 0x7f63f0057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:53.230 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.207+0000 7f640bfff700 1 --2- 172.21.15.162:0/1256643296 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f63f00553b0 0x7f63f0057870 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f63fc00b000 tx=0x7f63fc006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:53.230 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.215+0000 7f64097fa700 1 -- 172.21.15.162:0/1256643296 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f63f4044380 con 0x7f640c13dbe0 2020-11-28T21:46:53.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.355+0000 7f6411232700 1 -- 172.21.15.162:0/1256643296 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f640c057050 con 0x7f640c13dbe0 2020-11-28T21:46:53.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.355+0000 7f64097fa700 1 -- 172.21.15.162:0/1256643296 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v23) v1 ==== 110+0+10 (secure 0 0 0) 0x7f63f40479a0 con 0x7f640c13dbe0 2020-11-28T21:46:53.358 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:46:53.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.355+0000 7f6411232700 1 -- 172.21.15.162:0/1256643296 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f63f00553b0 msgr2=0x7f63f0057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:53.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.355+0000 7f6411232700 1 --2- 172.21.15.162:0/1256643296 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f63f00553b0 0x7f63f0057870 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f63fc00b000 tx=0x7f63fc006dc0).stop 2020-11-28T21:46:53.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.355+0000 7f6411232700 1 -- 172.21.15.162:0/1256643296 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f640c13dbe0 msgr2=0x7f640c152930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:53.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.355+0000 7f6411232700 1 --2- 172.21.15.162:0/1256643296 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f640c13dbe0 0x7f640c152930 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f63f400a000 tx=0x7f63f4008040).stop 2020-11-28T21:46:53.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.359+0000 7f6411232700 1 -- 172.21.15.162:0/1256643296 shutdown_connections 2020-11-28T21:46:53.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.359+0000 7f6411232700 1 --2- 172.21.15.162:0/1256643296 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f63f00553b0 0x7f63f0057870 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:53.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.359+0000 7f6411232700 1 --2- 172.21.15.162:0/1256643296 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f640c13dfc0 0x7f640c152e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:53.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.359+0000 7f6411232700 1 --2- 172.21.15.162:0/1256643296 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f640c0b0ca0 0x7f640c13cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:53.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.359+0000 7f6411232700 1 --2- 172.21.15.162:0/1256643296 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f640c13dbe0 0x7f640c152930 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:53.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.359+0000 7f6411232700 1 -- 172.21.15.162:0/1256643296 >> 172.21.15.162:0/1256643296 conn(0x7f640c0b2830 msgr2=0x7f640c0ae9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:53.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.359+0000 7f6411232700 1 -- 172.21.15.162:0/1256643296 shutdown_connections 2020-11-28T21:46:53.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.359+0000 7f6411232700 1 -- 172.21.15.162:0/1256643296 wait complete. 2020-11-28T21:46:53.371 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:46:53.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.487+0000 7f2e2f283700 1 Processor -- start 2020-11-28T21:46:53.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.487+0000 7f2e2f283700 1 -- start start 2020-11-28T21:46:53.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.487+0000 7f2e2f283700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2e280b9ce0 0x7f2e281433c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:53.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.487+0000 7f2e2f283700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f2e28067610 0x7f2e2813fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:53.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.487+0000 7f2e2f283700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e28143950 con 0x7f2e280605c0 2020-11-28T21:46:53.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.487+0000 7f2e2f283700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e280b4340 con 0x7f2e28067610 2020-11-28T21:46:53.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.487+0000 7f2e2f283700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e280ba0c0 con 0x7f2e280b9ce0 2020-11-28T21:46:53.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.487+0000 7f2e2c81e700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f2e280605c0 0x7f2e280b3c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:44956/0 (socket says 172.21.15.162:44956) 2020-11-28T21:46:53.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.487+0000 7f2e2c81e700 1 -- 172.21.15.162:0/2173589853 learned_addr learned my addr 172.21.15.162:0/2173589853 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:53.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.487+0000 7f2e2d820700 1 --2- 172.21.15.162:0/2173589853 >> v2:172.21.15.118:3300/0 conn(0x7f2e28067610 0x7f2e2813fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:53.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.487+0000 7f2e2d01f700 1 --2- 172.21.15.162:0/2173589853 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2e280b9ce0 0x7f2e281433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:53.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.487+0000 7f2e1ffff700 1 -- 172.21.15.162:0/2173589853 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 881764759 0 0) 0x7f2e28143950 con 0x7f2e280605c0 2020-11-28T21:46:53.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.487+0000 7f2e1ffff700 1 -- 172.21.15.162:0/2173589853 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e0c003230 con 0x7f2e280605c0 2020-11-28T21:46:53.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.491+0000 7f2e2d820700 1 -- 172.21.15.162:0/2173589853 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2e280b9ce0 msgr2=0x7f2e281433c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:53.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.491+0000 7f2e2d820700 1 --2- 172.21.15.162:0/2173589853 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2e280b9ce0 0x7f2e281433c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:53.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.491+0000 7f2e2d820700 1 -- 172.21.15.162:0/2173589853 >> v1:172.21.15.162:6789/0 conn(0x7f2e280605c0 legacy=0x7f2e280b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:53.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.491+0000 7f2e1ffff700 1 -- 172.21.15.162:0/2173589853 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2875144934 0 0) 0x7f2e0c003230 con 0x7f2e280605c0 2020-11-28T21:46:53.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.491+0000 7f2e2d820700 1 -- 172.21.15.162:0/2173589853 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e28067aa0 con 0x7f2e28067610 2020-11-28T21:46:53.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.491+0000 7f2e2d01f700 1 --2- 172.21.15.162:0/2173589853 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2e280b9ce0 0x7f2e281433c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:46:53.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.491+0000 7f2e2d820700 1 --2- 172.21.15.162:0/2173589853 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2e28067610 0x7f2e2813fc80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f2e18001870 tx=0x7f2e200091a0).ready entity=mon.0 client_cookie=29afee7293e032a8 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:53.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.491+0000 7f2e1ffff700 1 -- 172.21.15.162:0/2173589853 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e180016d0 con 0x7f2e28067610 2020-11-28T21:46:53.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.491+0000 7f2e1ffff700 1 -- 172.21.15.162:0/2173589853 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e2000b800 con 0x7f2e28067610 2020-11-28T21:46:53.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.491+0000 7f2e1ffff700 1 -- 172.21.15.162:0/2173589853 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e2000b970 con 0x7f2e28067610 2020-11-28T21:46:53.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.491+0000 7f2e2f283700 1 -- 172.21.15.162:0/2173589853 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2e28067610 msgr2=0x7f2e2813fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:53.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.491+0000 7f2e2f283700 1 --2- 172.21.15.162:0/2173589853 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2e28067610 0x7f2e2813fc80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f2e18001870 tx=0x7f2e200091a0).stop 2020-11-28T21:46:53.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.491+0000 7f2e2f283700 1 -- 172.21.15.162:0/2173589853 shutdown_connections 2020-11-28T21:46:53.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.491+0000 7f2e2f283700 1 --2- 172.21.15.162:0/2173589853 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2e28067610 0x7f2e2813fc80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:53.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.491+0000 7f2e2f283700 1 --2- 172.21.15.162:0/2173589853 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2e280b9ce0 0x7f2e281433c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:53.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.491+0000 7f2e2f283700 1 -- 172.21.15.162:0/2173589853 >> 172.21.15.162:0/2173589853 conn(0x7f2e2805c430 msgr2=0x7f2e280b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:53.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.491+0000 7f2e2f283700 1 -- 172.21.15.162:0/2173589853 shutdown_connections 2020-11-28T21:46:53.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.491+0000 7f2e2f283700 1 -- 172.21.15.162:0/2173589853 wait complete. 2020-11-28T21:46:53.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.491+0000 7f2e2f283700 1 Processor -- start 2020-11-28T21:46:53.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.491+0000 7f2e2f283700 1 -- start start 2020-11-28T21:46:53.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.495+0000 7f2e2f283700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2e280605c0 0x7f2e280b5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:53.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.495+0000 7f2e2f283700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2e28067610 0x7f2e280b06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:53.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.495+0000 7f2e2f283700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e280b9ce0 0x7f2e28154dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:53.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.495+0000 7f2e2f283700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e280b1270 con 0x7f2e28067610 2020-11-28T21:46:53.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.495+0000 7f2e2f283700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e280b1510 con 0x7f2e280b9ce0 2020-11-28T21:46:53.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.495+0000 7f2e2f283700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e280b17b0 con 0x7f2e280605c0 2020-11-28T21:46:53.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.495+0000 7f2e2d820700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e280b9ce0 0x7f2e28154dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:53.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.495+0000 7f2e2d820700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e280b9ce0 0x7f2e28154dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38118/0 (socket says 172.21.15.162:38118) 2020-11-28T21:46:53.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.495+0000 7f2e2d820700 1 -- 172.21.15.162:0/1947086184 learned_addr learned my addr 172.21.15.162:0/1947086184 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:53.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.495+0000 7f2e2c81e700 1 --2- 172.21.15.162:0/1947086184 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2e28067610 0x7f2e280b06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:53.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.495+0000 7f2e2d01f700 1 --2- 172.21.15.162:0/1947086184 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2e280605c0 0x7f2e280b5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:53.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.495+0000 7f2e2d820700 1 -- 172.21.15.162:0/1947086184 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2e280605c0 msgr2=0x7f2e280b5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:53.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.495+0000 7f2e2d820700 1 --2- 172.21.15.162:0/1947086184 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2e280605c0 0x7f2e280b5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:53.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.495+0000 7f2e2d820700 1 -- 172.21.15.162:0/1947086184 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2e28067610 msgr2=0x7f2e280b06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:53.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.495+0000 7f2e2d820700 1 --2- 172.21.15.162:0/1947086184 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2e28067610 0x7f2e280b06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:53.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.495+0000 7f2e2d820700 1 -- 172.21.15.162:0/1947086184 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e280b1a50 con 0x7f2e280b9ce0 2020-11-28T21:46:53.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.495+0000 7f2e2d01f700 1 --2- 172.21.15.162:0/1947086184 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2e280605c0 0x7f2e280b5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:46:53.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.495+0000 7f2e2d820700 1 --2- 172.21.15.162:0/1947086184 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e280b9ce0 0x7f2e28154dc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f2e18001870 tx=0x7f2e20016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:53.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.495+0000 7f2e1e7fc700 1 -- 172.21.15.162:0/1947086184 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e20011ad0 con 0x7f2e280b9ce0 2020-11-28T21:46:53.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.495+0000 7f2e1e7fc700 1 -- 172.21.15.162:0/1947086184 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e20011c30 con 0x7f2e280b9ce0 2020-11-28T21:46:53.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.495+0000 7f2e2f283700 1 -- 172.21.15.162:0/1947086184 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e280b1cf0 con 0x7f2e280b9ce0 2020-11-28T21:46:53.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.495+0000 7f2e1e7fc700 1 -- 172.21.15.162:0/1947086184 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e2001f3f0 con 0x7f2e280b9ce0 2020-11-28T21:46:53.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.495+0000 7f2e2f283700 1 -- 172.21.15.162:0/1947086184 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e280b0dc0 con 0x7f2e280b9ce0 2020-11-28T21:46:53.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.495+0000 7f2e2f283700 1 -- 172.21.15.162:0/1947086184 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e28154bc0 con 0x7f2e280b9ce0 2020-11-28T21:46:53.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.499+0000 7f2e1e7fc700 1 -- 172.21.15.162:0/1947086184 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2e20026030 con 0x7f2e280b9ce0 2020-11-28T21:46:53.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.499+0000 7f2e1e7fc700 1 --2- 172.21.15.162:0/1947086184 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2e100554e0 0x7f2e100579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:53.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.499+0000 7f2e2d01f700 1 --2- 172.21.15.162:0/1947086184 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2e100554e0 0x7f2e100579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:53.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.499+0000 7f2e1e7fc700 1 -- 172.21.15.162:0/1947086184 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2e2006e540 con 0x7f2e280b9ce0 2020-11-28T21:46:53.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.499+0000 7f2e2d01f700 1 --2- 172.21.15.162:0/1947086184 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2e100554e0 0x7f2e100579a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f2e24008000 tx=0x7f2e24008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:53.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.503+0000 7f2e1e7fc700 1 -- 172.21.15.162:0/1947086184 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f2e20044840 con 0x7f2e280b9ce0 2020-11-28T21:46:53.646 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.643+0000 7f2e2f283700 1 -- 172.21.15.162:0/1947086184 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2e28057050 con 0x7f2e280b9ce0 2020-11-28T21:46:53.647 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.643+0000 7f2e1e7fc700 1 -- 172.21.15.162:0/1947086184 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2e200150a0 con 0x7f2e280b9ce0 2020-11-28T21:46:53.647 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:46:53.648 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.647+0000 7f2e2f283700 1 -- 172.21.15.162:0/1947086184 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2e100554e0 msgr2=0x7f2e100579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:53.649 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.647+0000 7f2e2f283700 1 --2- 172.21.15.162:0/1947086184 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2e100554e0 0x7f2e100579a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f2e24008000 tx=0x7f2e24008040).stop 2020-11-28T21:46:53.649 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.647+0000 7f2e2f283700 1 -- 172.21.15.162:0/1947086184 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e280b9ce0 msgr2=0x7f2e28154dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:53.649 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.647+0000 7f2e2f283700 1 --2- 172.21.15.162:0/1947086184 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e280b9ce0 0x7f2e28154dc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f2e18001870 tx=0x7f2e20016000).stop 2020-11-28T21:46:53.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.647+0000 7f2e2f283700 1 -- 172.21.15.162:0/1947086184 shutdown_connections 2020-11-28T21:46:53.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.647+0000 7f2e2f283700 1 --2- 172.21.15.162:0/1947086184 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2e100554e0 0x7f2e100579a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:53.651 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.647+0000 7f2e2f283700 1 --2- 172.21.15.162:0/1947086184 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2e280605c0 0x7f2e280b5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:53.651 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.647+0000 7f2e2f283700 1 --2- 172.21.15.162:0/1947086184 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e280b9ce0 0x7f2e28154dc0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:53.651 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.647+0000 7f2e2f283700 1 --2- 172.21.15.162:0/1947086184 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2e28067610 0x7f2e280b06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:53.651 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.647+0000 7f2e2f283700 1 -- 172.21.15.162:0/1947086184 >> 172.21.15.162:0/1947086184 conn(0x7f2e2805c430 msgr2=0x7f2e280b7dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:53.652 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.647+0000 7f2e2f283700 1 -- 172.21.15.162:0/1947086184 shutdown_connections 2020-11-28T21:46:53.652 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.647+0000 7f2e2f283700 1 -- 172.21.15.162:0/1947086184 wait complete. 2020-11-28T21:46:53.661 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-28T21:46:53.780 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.779+0000 7f371a30a700 1 Processor -- start 2020-11-28T21:46:53.781 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.779+0000 7f371a30a700 1 -- start start 2020-11-28T21:46:53.782 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.779+0000 7f371a30a700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f37140b9ce0 0x7f37141433c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:53.782 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.779+0000 7f371a30a700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3714143950 con 0x7f3714067610 2020-11-28T21:46:53.782 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.779+0000 7f371a30a700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3714067aa0 con 0x7f37140605c0 2020-11-28T21:46:53.782 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.779+0000 7f371a30a700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37140b4340 con 0x7f37140b9ce0 2020-11-28T21:46:53.783 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.779+0000 7f37137fe700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f37140605c0 0x7f37140b3c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:44968/0 (socket says 172.21.15.162:44968) 2020-11-28T21:46:53.783 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.779+0000 7f37137fe700 1 -- 172.21.15.162:0/2838763733 learned_addr learned my addr 172.21.15.162:0/2838763733 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:53.783 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.779+0000 7f3713fff700 1 --2- 172.21.15.162:0/2838763733 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f37140b9ce0 0x7f37141433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:53.784 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.779+0000 7f3712ffd700 1 -- 172.21.15.162:0/2838763733 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 679382450 0 0) 0x7f3714067aa0 con 0x7f37140605c0 2020-11-28T21:46:53.785 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.779+0000 7f3712ffd700 1 -- 172.21.15.162:0/2838763733 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3700003230 con 0x7f37140605c0 2020-11-28T21:46:53.785 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.779+0000 7f3713fff700 1 -- 172.21.15.162:0/2838763733 >> v1:172.21.15.162:6789/0 conn(0x7f37140605c0 legacy=0x7f37140b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:53.785 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.779+0000 7f3712ffd700 1 -- 172.21.15.162:0/2838763733 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4056051134 0 0) 0x7f3714143950 con 0x7f3714067610 2020-11-28T21:46:53.785 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.779+0000 7f3713fff700 1 -- 172.21.15.162:0/2838763733 >> v1:172.21.15.118:6789/0 conn(0x7f3714067610 legacy=0x7f371413fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:53.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.779+0000 7f3713fff700 1 -- 172.21.15.162:0/2838763733 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37140ba0c0 con 0x7f37140b9ce0 2020-11-28T21:46:53.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.779+0000 7f3713fff700 1 --2- 172.21.15.162:0/2838763733 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f37140b9ce0 0x7f37141433c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f3704002f30 tx=0x7f3704006d60).ready entity=mon.2 client_cookie=c18f6a0c56bb9f4 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:53.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.779+0000 7f3712ffd700 1 -- 172.21.15.162:0/2838763733 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3704009640 con 0x7f37140b9ce0 2020-11-28T21:46:53.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.779+0000 7f3712ffd700 1 -- 172.21.15.162:0/2838763733 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37040097a0 con 0x7f37140b9ce0 2020-11-28T21:46:53.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.779+0000 7f3712ffd700 1 -- 172.21.15.162:0/2838763733 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3704009910 con 0x7f37140b9ce0 2020-11-28T21:46:53.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.779+0000 7f371a30a700 1 -- 172.21.15.162:0/2838763733 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f37140b9ce0 msgr2=0x7f37141433c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:53.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.779+0000 7f371a30a700 1 --2- 172.21.15.162:0/2838763733 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f37140b9ce0 0x7f37141433c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f3704002f30 tx=0x7f3704006d60).stop 2020-11-28T21:46:53.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.783+0000 7f371a30a700 1 -- 172.21.15.162:0/2838763733 shutdown_connections 2020-11-28T21:46:53.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.783+0000 7f371a30a700 1 --2- 172.21.15.162:0/2838763733 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f37140b9ce0 0x7f37141433c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:53.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.783+0000 7f371a30a700 1 -- 172.21.15.162:0/2838763733 >> 172.21.15.162:0/2838763733 conn(0x7f371405c430 msgr2=0x7f37140b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:53.788 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.783+0000 7f371a30a700 1 -- 172.21.15.162:0/2838763733 shutdown_connections 2020-11-28T21:46:53.788 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.783+0000 7f371a30a700 1 -- 172.21.15.162:0/2838763733 wait complete. 2020-11-28T21:46:53.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.783+0000 7f371a30a700 1 Processor -- start 2020-11-28T21:46:53.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.783+0000 7f371a30a700 1 -- start start 2020-11-28T21:46:53.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.783+0000 7f371a30a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37140605c0 0x7f371413f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:53.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.783+0000 7f371a30a700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3714067610 0x7f3714143fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:53.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.783+0000 7f371a30a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37140b9ce0 0x7f37140abc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:53.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.783+0000 7f371a30a700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371413fc80 con 0x7f37140b9ce0 2020-11-28T21:46:53.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.783+0000 7f371a30a700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37141455b0 con 0x7f37140605c0 2020-11-28T21:46:53.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.783+0000 7f371a30a700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37140b13c0 con 0x7f3714067610 2020-11-28T21:46:53.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.783+0000 7f3713fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37140605c0 0x7f371413f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:53.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.783+0000 7f3713fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37140605c0 0x7f371413f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38130/0 (socket says 172.21.15.162:38130) 2020-11-28T21:46:53.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.783+0000 7f3713fff700 1 -- 172.21.15.162:0/2285347167 learned_addr learned my addr 172.21.15.162:0/2285347167 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:53.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.783+0000 7f37137fe700 1 --2- 172.21.15.162:0/2285347167 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3714067610 0x7f3714143fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:53.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.783+0000 7f37188a7700 1 --2- 172.21.15.162:0/2285347167 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37140b9ce0 0x7f37140abc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:53.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.783+0000 7f3713fff700 1 -- 172.21.15.162:0/2285347167 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3714067610 msgr2=0x7f3714143fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:53.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.783+0000 7f3713fff700 1 --2- 172.21.15.162:0/2285347167 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3714067610 0x7f3714143fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:53.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.783+0000 7f3713fff700 1 -- 172.21.15.162:0/2285347167 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37140b9ce0 msgr2=0x7f37140abc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:53.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.783+0000 7f3713fff700 1 --2- 172.21.15.162:0/2285347167 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37140b9ce0 0x7f37140abc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:53.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.783+0000 7f3713fff700 1 -- 172.21.15.162:0/2285347167 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37140b1660 con 0x7f37140605c0 2020-11-28T21:46:53.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.783+0000 7f37137fe700 1 --2- 172.21.15.162:0/2285347167 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3714067610 0x7f3714143fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:46:53.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.783+0000 7f37188a7700 1 --2- 172.21.15.162:0/2285347167 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37140b9ce0 0x7f37140abc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:46:53.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.783+0000 7f3713fff700 1 --2- 172.21.15.162:0/2285347167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37140605c0 0x7f371413f4e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f3704002f30 tx=0x7f3704016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:53.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.783+0000 7f37117fa700 1 -- 172.21.15.162:0/2285347167 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3704006220 con 0x7f37140605c0 2020-11-28T21:46:53.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.783+0000 7f37117fa700 1 -- 172.21.15.162:0/2285347167 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3704006380 con 0x7f37140605c0 2020-11-28T21:46:53.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.783+0000 7f371a30a700 1 -- 172.21.15.162:0/2285347167 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37140b1900 con 0x7f37140605c0 2020-11-28T21:46:53.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.783+0000 7f37117fa700 1 -- 172.21.15.162:0/2285347167 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f37040056c0 con 0x7f37140605c0 2020-11-28T21:46:53.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.783+0000 7f371a30a700 1 -- 172.21.15.162:0/2285347167 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37141452a0 con 0x7f37140605c0 2020-11-28T21:46:53.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.787+0000 7f371a30a700 1 -- 172.21.15.162:0/2285347167 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3714057050 con 0x7f37140605c0 2020-11-28T21:46:53.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.787+0000 7f37117fa700 1 -- 172.21.15.162:0/2285347167 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f370400f070 con 0x7f37140605c0 2020-11-28T21:46:53.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.787+0000 7f37117fa700 1 --2- 172.21.15.162:0/2285347167 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f36f40554e0 0x7f36f40579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:53.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.787+0000 7f37117fa700 1 -- 172.21.15.162:0/2285347167 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f370406f280 con 0x7f37140605c0 2020-11-28T21:46:53.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.787+0000 7f37137fe700 1 --2- 172.21.15.162:0/2285347167 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f36f40554e0 0x7f36f40579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:53.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.787+0000 7f37137fe700 1 --2- 172.21.15.162:0/2285347167 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f36f40554e0 0x7f36f40579a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f3708008000 tx=0x7f3708009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:53.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.791+0000 7f37117fa700 1 -- 172.21.15.162:0/2285347167 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f3704045630 con 0x7f37140605c0 2020-11-28T21:46:53.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.931+0000 7f371a30a700 1 -- 172.21.15.162:0/2285347167 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3714068350 con 0x7f37140605c0 2020-11-28T21:46:53.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.935+0000 7f37117fa700 1 -- 172.21.15.162:0/2285347167 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+10 (secure 0 0 0) 0x7f3704015020 con 0x7f37140605c0 2020-11-28T21:46:53.936 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:46:53.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.935+0000 7f371a30a700 1 -- 172.21.15.162:0/2285347167 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f36f40554e0 msgr2=0x7f36f40579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:53.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.935+0000 7f371a30a700 1 --2- 172.21.15.162:0/2285347167 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f36f40554e0 0x7f36f40579a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f3708008000 tx=0x7f3708009040).stop 2020-11-28T21:46:53.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.935+0000 7f371a30a700 1 -- 172.21.15.162:0/2285347167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37140605c0 msgr2=0x7f371413f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:53.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.935+0000 7f371a30a700 1 --2- 172.21.15.162:0/2285347167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37140605c0 0x7f371413f4e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f3704002f30 tx=0x7f3704016000).stop 2020-11-28T21:46:53.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.935+0000 7f371a30a700 1 -- 172.21.15.162:0/2285347167 shutdown_connections 2020-11-28T21:46:53.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.935+0000 7f371a30a700 1 --2- 172.21.15.162:0/2285347167 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f36f40554e0 0x7f36f40579a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:53.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.935+0000 7f371a30a700 1 --2- 172.21.15.162:0/2285347167 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37140b9ce0 0x7f37140abc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:53.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.935+0000 7f371a30a700 1 --2- 172.21.15.162:0/2285347167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37140605c0 0x7f371413f4e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:53.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.935+0000 7f371a30a700 1 --2- 172.21.15.162:0/2285347167 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3714067610 0x7f3714143fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:53.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.935+0000 7f371a30a700 1 -- 172.21.15.162:0/2285347167 >> 172.21.15.162:0/2285347167 conn(0x7f371405c430 msgr2=0x7f371413e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:53.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.935+0000 7f371a30a700 1 -- 172.21.15.162:0/2285347167 shutdown_connections 2020-11-28T21:46:53.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:53.935+0000 7f371a30a700 1 -- 172.21.15.162:0/2285347167 wait complete. 2020-11-28T21:46:53.949 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-28T21:46:54.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.063+0000 7f15198de700 1 Processor -- start 2020-11-28T21:46:54.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.063+0000 7f15198de700 1 -- start start 2020-11-28T21:46:54.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.063+0000 7f15198de700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f15140b9ce0 0x7f15141433c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:54.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.063+0000 7f15198de700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1514067610 0x7f151413fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:54.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.063+0000 7f15198de700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1514143950 con 0x7f15140605c0 2020-11-28T21:46:54.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.063+0000 7f15198de700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15140b4340 con 0x7f1514067610 2020-11-28T21:46:54.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.063+0000 7f15198de700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15140ba0c0 con 0x7f15140b9ce0 2020-11-28T21:46:54.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.063+0000 7f1512ffd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f15140b9ce0 0x7f15141433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:54.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.063+0000 7f15127fc700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f15140605c0 0x7f15140b3c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:44980/0 (socket says 172.21.15.162:44980) 2020-11-28T21:46:54.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.063+0000 7f15127fc700 1 -- 172.21.15.162:0/3535349123 learned_addr learned my addr 172.21.15.162:0/3535349123 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:54.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.063+0000 7f15137fe700 1 --2- 172.21.15.162:0/3535349123 >> v2:172.21.15.118:3300/0 conn(0x7f1514067610 0x7f151413fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:54.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.063+0000 7f1511ffb700 1 -- 172.21.15.162:0/3535349123 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 849441141 0 0) 0x7f1514143950 con 0x7f15140605c0 2020-11-28T21:46:54.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.063+0000 7f1511ffb700 1 -- 172.21.15.162:0/3535349123 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14f8003230 con 0x7f15140605c0 2020-11-28T21:46:54.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.063+0000 7f1512ffd700 1 -- 172.21.15.162:0/3535349123 >> v2:172.21.15.118:3300/0 conn(0x7f1514067610 msgr2=0x7f151413fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:54.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.063+0000 7f1512ffd700 1 --2- 172.21.15.162:0/3535349123 >> v2:172.21.15.118:3300/0 conn(0x7f1514067610 0x7f151413fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:54.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.063+0000 7f1512ffd700 1 -- 172.21.15.162:0/3535349123 >> v1:172.21.15.162:6789/0 conn(0x7f15140605c0 legacy=0x7f15140b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:54.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.063+0000 7f1512ffd700 1 -- 172.21.15.162:0/3535349123 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1514067aa0 con 0x7f15140b9ce0 2020-11-28T21:46:54.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.067+0000 7f1512ffd700 1 --2- 172.21.15.162:0/3535349123 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15140b9ce0 0x7f15141433c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f1508002f30 tx=0x7f1508006d60).ready entity=mon.1 client_cookie=5226003ab11b5b45 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:54.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.067+0000 7f1511ffb700 1 -- 172.21.15.162:0/3535349123 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1508009580 con 0x7f15140b9ce0 2020-11-28T21:46:54.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.067+0000 7f1511ffb700 1 -- 172.21.15.162:0/3535349123 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15080096e0 con 0x7f15140b9ce0 2020-11-28T21:46:54.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.067+0000 7f1511ffb700 1 -- 172.21.15.162:0/3535349123 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1508009850 con 0x7f15140b9ce0 2020-11-28T21:46:54.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.067+0000 7f15198de700 1 -- 172.21.15.162:0/3535349123 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15140b9ce0 msgr2=0x7f15141433c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:54.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.067+0000 7f15198de700 1 --2- 172.21.15.162:0/3535349123 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15140b9ce0 0x7f15141433c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f1508002f30 tx=0x7f1508006d60).stop 2020-11-28T21:46:54.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.067+0000 7f15198de700 1 -- 172.21.15.162:0/3535349123 shutdown_connections 2020-11-28T21:46:54.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.067+0000 7f15198de700 1 --2- 172.21.15.162:0/3535349123 >> v2:172.21.15.118:3300/0 conn(0x7f1514067610 0x7f151413fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:54.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.067+0000 7f15198de700 1 --2- 172.21.15.162:0/3535349123 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15140b9ce0 0x7f15141433c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:54.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.067+0000 7f15198de700 1 -- 172.21.15.162:0/3535349123 >> 172.21.15.162:0/3535349123 conn(0x7f151405c430 msgr2=0x7f15140b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:54.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.067+0000 7f15198de700 1 -- 172.21.15.162:0/3535349123 shutdown_connections 2020-11-28T21:46:54.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.067+0000 7f15198de700 1 -- 172.21.15.162:0/3535349123 wait complete. 2020-11-28T21:46:54.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.067+0000 7f15198de700 1 Processor -- start 2020-11-28T21:46:54.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.067+0000 7f15198de700 1 -- start start 2020-11-28T21:46:54.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.067+0000 7f15198de700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f15140605c0 0x7f15140b8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:54.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.067+0000 7f15198de700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1514067610 0x7f1514144d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:54.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.067+0000 7f15198de700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15140b9ce0 0x7f1514151fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:54.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.067+0000 7f15198de700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15140b2b20 con 0x7f15140605c0 2020-11-28T21:46:54.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.067+0000 7f15198de700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1514151930 con 0x7f15140b9ce0 2020-11-28T21:46:54.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.067+0000 7f15198de700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15141453f0 con 0x7f1514067610 2020-11-28T21:46:54.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.071+0000 7f15137fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15140b9ce0 0x7f1514151fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:54.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.071+0000 7f15137fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15140b9ce0 0x7f1514151fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38146/0 (socket says 172.21.15.162:38146) 2020-11-28T21:46:54.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.071+0000 7f15137fe700 1 -- 172.21.15.162:0/3532895177 learned_addr learned my addr 172.21.15.162:0/3532895177 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:54.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.071+0000 7f1512ffd700 1 --2- 172.21.15.162:0/3532895177 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f15140605c0 0x7f15140b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:54.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.071+0000 7f15127fc700 1 --2- 172.21.15.162:0/3532895177 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1514067610 0x7f1514144d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:54.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.071+0000 7f15137fe700 1 -- 172.21.15.162:0/3532895177 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1514067610 msgr2=0x7f1514144d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:54.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.071+0000 7f15137fe700 1 --2- 172.21.15.162:0/3532895177 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1514067610 0x7f1514144d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:54.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.071+0000 7f15137fe700 1 -- 172.21.15.162:0/3532895177 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f15140605c0 msgr2=0x7f15140b8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:54.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.071+0000 7f15137fe700 1 --2- 172.21.15.162:0/3532895177 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f15140605c0 0x7f15140b8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:54.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.071+0000 7f15137fe700 1 -- 172.21.15.162:0/3532895177 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1514145690 con 0x7f15140b9ce0 2020-11-28T21:46:54.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.071+0000 7f15137fe700 1 --2- 172.21.15.162:0/3532895177 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15140b9ce0 0x7f1514151fc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f1514144ca0 tx=0x7f150c0091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:54.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.071+0000 7f1503fff700 1 -- 172.21.15.162:0/3532895177 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f150c00ee60 con 0x7f15140b9ce0 2020-11-28T21:46:54.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.071+0000 7f1503fff700 1 -- 172.21.15.162:0/3532895177 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f150c00cb00 con 0x7f15140b9ce0 2020-11-28T21:46:54.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.071+0000 7f15198de700 1 -- 172.21.15.162:0/3532895177 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1514151b20 con 0x7f15140b9ce0 2020-11-28T21:46:54.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.071+0000 7f1503fff700 1 -- 172.21.15.162:0/3532895177 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f150c00ce60 con 0x7f15140b9ce0 2020-11-28T21:46:54.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.071+0000 7f15198de700 1 -- 172.21.15.162:0/3532895177 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f151409e050 con 0x7f15140b9ce0 2020-11-28T21:46:54.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.071+0000 7f15198de700 1 -- 172.21.15.162:0/3532895177 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1514151cb0 con 0x7f15140b9ce0 2020-11-28T21:46:54.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.071+0000 7f1503fff700 1 -- 172.21.15.162:0/3532895177 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f150c022070 con 0x7f15140b9ce0 2020-11-28T21:46:54.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.071+0000 7f1503fff700 1 --2- 172.21.15.162:0/3532895177 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f14f00554e0 0x7f14f00579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:54.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.071+0000 7f1503fff700 1 -- 172.21.15.162:0/3532895177 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f150c06e540 con 0x7f15140b9ce0 2020-11-28T21:46:54.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.075+0000 7f1512ffd700 1 --2- 172.21.15.162:0/3532895177 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f14f00554e0 0x7f14f00579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:54.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.075+0000 7f1512ffd700 1 --2- 172.21.15.162:0/3532895177 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f14f00554e0 0x7f14f00579a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f150801b000 tx=0x7f1508016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:54.083 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.075+0000 7f1503fff700 1 -- 172.21.15.162:0/3532895177 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f150c044840 con 0x7f15140b9ce0 2020-11-28T21:46:54.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.215+0000 7f15198de700 1 -- 172.21.15.162:0/3532895177 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1514057050 con 0x7f15140b9ce0 2020-11-28T21:46:54.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.215+0000 7f1503fff700 1 -- 172.21.15.162:0/3532895177 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+10 (secure 0 0 0) 0x7f150c016070 con 0x7f15140b9ce0 2020-11-28T21:46:54.220 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:46:54.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.219+0000 7f15198de700 1 -- 172.21.15.162:0/3532895177 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f14f00554e0 msgr2=0x7f14f00579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:54.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.219+0000 7f15198de700 1 --2- 172.21.15.162:0/3532895177 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f14f00554e0 0x7f14f00579a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f150801b000 tx=0x7f1508016000).stop 2020-11-28T21:46:54.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.219+0000 7f15198de700 1 -- 172.21.15.162:0/3532895177 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15140b9ce0 msgr2=0x7f1514151fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:54.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.219+0000 7f15198de700 1 --2- 172.21.15.162:0/3532895177 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15140b9ce0 0x7f1514151fc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f1514144ca0 tx=0x7f150c0091a0).stop 2020-11-28T21:46:54.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.219+0000 7f15198de700 1 -- 172.21.15.162:0/3532895177 shutdown_connections 2020-11-28T21:46:54.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.219+0000 7f15198de700 1 --2- 172.21.15.162:0/3532895177 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f14f00554e0 0x7f14f00579a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:54.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.219+0000 7f15198de700 1 --2- 172.21.15.162:0/3532895177 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15140b9ce0 0x7f1514151fc0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:54.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.219+0000 7f15198de700 1 --2- 172.21.15.162:0/3532895177 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f15140605c0 0x7f15140b8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:54.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.219+0000 7f15198de700 1 --2- 172.21.15.162:0/3532895177 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1514067610 0x7f1514144d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:54.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.219+0000 7f15198de700 1 -- 172.21.15.162:0/3532895177 >> 172.21.15.162:0/3532895177 conn(0x7f151405c430 msgr2=0x7f1514140ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:54.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.219+0000 7f15198de700 1 -- 172.21.15.162:0/3532895177 shutdown_connections 2020-11-28T21:46:54.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.219+0000 7f15198de700 1 -- 172.21.15.162:0/3532895177 wait complete. 2020-11-28T21:46:54.238 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:46:54.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.351+0000 7f8661507700 1 Processor -- start 2020-11-28T21:46:54.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.351+0000 7f8661507700 1 -- start start 2020-11-28T21:46:54.356 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.351+0000 7f8661507700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f865c098b30 0x7f865c0994f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:54.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.351+0000 7f8661507700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f865c098f10 con 0x7f865c0b0000 2020-11-28T21:46:54.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.351+0000 7f8661507700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f865c0b0680 con 0x7f865c060710 2020-11-28T21:46:54.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.351+0000 7f8661507700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f865c0b1030 con 0x7f865c098b30 2020-11-28T21:46:54.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.351+0000 7f8660d06700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f865c060710 0x7f865c0b1580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:44998/0 (socket says 172.21.15.162:44998) 2020-11-28T21:46:54.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.351+0000 7f8660d06700 1 -- 172.21.15.162:0/3643805963 learned_addr learned my addr 172.21.15.162:0/3643805963 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:54.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.351+0000 7f865bfff700 1 --2- 172.21.15.162:0/3643805963 >> v2:172.21.15.118:3300/0 conn(0x7f865c098b30 0x7f865c0994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:54.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.351+0000 7f865affd700 1 -- 172.21.15.162:0/3643805963 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 584943319 0 0) 0x7f865c0b0680 con 0x7f865c060710 2020-11-28T21:46:54.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.351+0000 7f865affd700 1 -- 172.21.15.162:0/3643805963 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f863c003230 con 0x7f865c060710 2020-11-28T21:46:54.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.351+0000 7f865affd700 1 -- 172.21.15.162:0/3643805963 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1579051356 0 0) 0x7f865c098f10 con 0x7f865c0b0000 2020-11-28T21:46:54.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.351+0000 7f865affd700 1 -- 172.21.15.162:0/3643805963 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f865c0b0680 con 0x7f865c0b0000 2020-11-28T21:46:54.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.351+0000 7f865affd700 1 -- 172.21.15.162:0/3643805963 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 322592601 0 0) 0x7f863c003230 con 0x7f865c060710 2020-11-28T21:46:54.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.351+0000 7f865affd700 1 -- 172.21.15.162:0/3643805963 >> v2:172.21.15.118:3300/0 conn(0x7f865c098b30 msgr2=0x7f865c0994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:54.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.351+0000 7f865affd700 1 --2- 172.21.15.162:0/3643805963 >> v2:172.21.15.118:3300/0 conn(0x7f865c098b30 0x7f865c0994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:54.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.351+0000 7f865affd700 1 -- 172.21.15.162:0/3643805963 >> v1:172.21.15.118:6789/0 conn(0x7f865c0b0000 legacy=0x7f865c060000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:54.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.351+0000 7f865affd700 1 -- 172.21.15.162:0/3643805963 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f865c0b0920 con 0x7f865c060710 2020-11-28T21:46:54.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.351+0000 7f865affd700 1 -- 172.21.15.162:0/3643805963 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f86540016d0 con 0x7f865c060710 2020-11-28T21:46:54.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.351+0000 7f865affd700 1 -- 172.21.15.162:0/3643805963 >> v1:172.21.15.162:6789/0 conn(0x7f865c060710 legacy=0x7f865c0b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:54.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.351+0000 7f865affd700 1 --2- 172.21.15.162:0/3643805963 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f863c004910 0x7f863c004d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:54.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.351+0000 7f865affd700 1 --2- 172.21.15.162:0/3643805963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f863c003470 0x7f863c0038f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:54.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.351+0000 7f865affd700 1 --2- 172.21.15.162:0/3643805963 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f863c003de0 0x7f863c00b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:54.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.351+0000 7f865affd700 1 -- 172.21.15.162:0/3643805963 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c0b1030 con 0x7f863c004910 2020-11-28T21:46:54.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.351+0000 7f865affd700 1 -- 172.21.15.162:0/3643805963 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863c004340 con 0x7f863c003470 2020-11-28T21:46:54.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.351+0000 7f865affd700 1 -- 172.21.15.162:0/3643805963 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f863c00bc90 con 0x7f863c003de0 2020-11-28T21:46:54.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.351+0000 7f865b7fe700 1 --2- 172.21.15.162:0/3643805963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f863c003470 0x7f863c0038f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:54.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.351+0000 7f865bfff700 1 --2- 172.21.15.162:0/3643805963 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f863c004910 0x7f863c004d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:54.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.355+0000 7f8660d06700 1 --2- 172.21.15.162:0/3643805963 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f863c003de0 0x7f863c00b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:54.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.355+0000 7f865b7fe700 1 -- 172.21.15.162:0/3643805963 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f863c003de0 msgr2=0x7f863c00b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:54.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.355+0000 7f865b7fe700 1 --2- 172.21.15.162:0/3643805963 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f863c003de0 0x7f863c00b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:54.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.355+0000 7f865b7fe700 1 -- 172.21.15.162:0/3643805963 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f863c004910 msgr2=0x7f863c004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:54.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.355+0000 7f865b7fe700 1 --2- 172.21.15.162:0/3643805963 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f863c004910 0x7f863c004d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:54.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.355+0000 7f865b7fe700 1 -- 172.21.15.162:0/3643805963 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f863c003230 con 0x7f863c003470 2020-11-28T21:46:54.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.355+0000 7f8660d06700 1 -- 172.21.15.162:0/3643805963 reap_dead start 2020-11-28T21:46:54.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.355+0000 7f865b7fe700 1 --2- 172.21.15.162:0/3643805963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f863c003470 0x7f863c0038f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f86440010d0 tx=0x7f8644007b60).ready entity=mon.1 client_cookie=119bd1d8d36d0771 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:54.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.355+0000 7f865affd700 1 -- 172.21.15.162:0/3643805963 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f864400c870 con 0x7f863c003470 2020-11-28T21:46:54.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.355+0000 7f865affd700 1 -- 172.21.15.162:0/3643805963 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f864400c9d0 con 0x7f863c003470 2020-11-28T21:46:54.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.355+0000 7f865affd700 1 -- 172.21.15.162:0/3643805963 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f864400cd30 con 0x7f863c003470 2020-11-28T21:46:54.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.355+0000 7f8661507700 1 -- 172.21.15.162:0/3643805963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f863c003470 msgr2=0x7f863c0038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:54.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.355+0000 7f8661507700 1 --2- 172.21.15.162:0/3643805963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f863c003470 0x7f863c0038f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f86440010d0 tx=0x7f8644007b60).stop 2020-11-28T21:46:54.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.355+0000 7f8661507700 1 -- 172.21.15.162:0/3643805963 shutdown_connections 2020-11-28T21:46:54.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.355+0000 7f8661507700 1 --2- 172.21.15.162:0/3643805963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f863c003470 0x7f863c0038f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:54.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.355+0000 7f8661507700 1 -- 172.21.15.162:0/3643805963 >> 172.21.15.162:0/3643805963 conn(0x7f865c05b970 msgr2=0x7f865c05bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:54.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.355+0000 7f8661507700 1 -- 172.21.15.162:0/3643805963 shutdown_connections 2020-11-28T21:46:54.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.355+0000 7f8661507700 1 -- 172.21.15.162:0/3643805963 wait complete. 2020-11-28T21:46:54.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.355+0000 7f8661507700 1 Processor -- start 2020-11-28T21:46:54.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.355+0000 7f8661507700 1 -- start start 2020-11-28T21:46:54.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.355+0000 7f8661507700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f863c003470 0x7f865c0ab8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:54.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.355+0000 7f8661507700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f865c063340 0x7f865c0aaa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:54.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.355+0000 7f8661507700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f865c0ae350 0x7f865c0ae800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:54.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.359+0000 7f8661507700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c0b4460 con 0x7f865c0ae350 2020-11-28T21:46:54.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.359+0000 7f8661507700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c0ac010 con 0x7f865c063340 2020-11-28T21:46:54.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.359+0000 7f8661507700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f865c1566a0 con 0x7f863c003470 2020-11-28T21:46:54.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.359+0000 7f865b7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f865c063340 0x7f865c0aaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:54.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.359+0000 7f865b7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f865c063340 0x7f865c0aaa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38164/0 (socket says 172.21.15.162:38164) 2020-11-28T21:46:54.371 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.359+0000 7f865b7fe700 1 -- 172.21.15.162:0/236918587 learned_addr learned my addr 172.21.15.162:0/236918587 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:54.371 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.359+0000 7f865bfff700 1 --2- 172.21.15.162:0/236918587 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f863c003470 0x7f865c0ab8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:54.371 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.359+0000 7f8660d06700 1 --2- 172.21.15.162:0/236918587 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f865c0ae350 0x7f865c0ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:54.371 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.359+0000 7f865b7fe700 1 -- 172.21.15.162:0/236918587 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f863c003470 msgr2=0x7f865c0ab8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:54.371 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.359+0000 7f865b7fe700 1 --2- 172.21.15.162:0/236918587 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f863c003470 0x7f865c0ab8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:54.372 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.359+0000 7f865b7fe700 1 -- 172.21.15.162:0/236918587 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f865c0ae350 msgr2=0x7f865c0ae800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:54.372 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.359+0000 7f865b7fe700 1 --2- 172.21.15.162:0/236918587 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f865c0ae350 0x7f865c0ae800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:54.372 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.359+0000 7f865b7fe700 1 -- 172.21.15.162:0/236918587 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f865c156940 con 0x7f865c063340 2020-11-28T21:46:54.372 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.359+0000 7f865bfff700 1 --2- 172.21.15.162:0/236918587 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f863c003470 0x7f865c0ab8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:46:54.372 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.359+0000 7f865b7fe700 1 --2- 172.21.15.162:0/236918587 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f865c063340 0x7f865c0aaa30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f865c0ae230 tx=0x7f8644017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:54.373 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.359+0000 7f86597fa700 1 -- 172.21.15.162:0/236918587 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8644007430 con 0x7f865c063340 2020-11-28T21:46:54.373 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.359+0000 7f86597fa700 1 -- 172.21.15.162:0/236918587 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8644012ac0 con 0x7f865c063340 2020-11-28T21:46:54.373 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.359+0000 7f8661507700 1 -- 172.21.15.162:0/236918587 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f865c0adeb0 con 0x7f865c063340 2020-11-28T21:46:54.373 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.359+0000 7f86597fa700 1 -- 172.21.15.162:0/236918587 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8644012e20 con 0x7f865c063340 2020-11-28T21:46:54.373 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.359+0000 7f8661507700 1 -- 172.21.15.162:0/236918587 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f865c0aecf0 con 0x7f865c063340 2020-11-28T21:46:54.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.359+0000 7f8661507700 1 -- 172.21.15.162:0/236918587 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f865c0ae040 con 0x7f865c063340 2020-11-28T21:46:54.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.359+0000 7f86597fa700 1 -- 172.21.15.162:0/236918587 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8644010070 con 0x7f865c063340 2020-11-28T21:46:54.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.359+0000 7f86597fa700 1 --2- 172.21.15.162:0/236918587 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8640055440 0x7f8640057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:54.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.359+0000 7f865bfff700 1 --2- 172.21.15.162:0/236918587 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8640055440 0x7f8640057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:54.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.359+0000 7f86597fa700 1 -- 172.21.15.162:0/236918587 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f864406eb40 con 0x7f865c063340 2020-11-28T21:46:54.375 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.363+0000 7f865bfff700 1 --2- 172.21.15.162:0/236918587 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8640055440 0x7f8640057900 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f864c00b000 tx=0x7f864c008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:54.375 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.367+0000 7f86597fa700 1 -- 172.21.15.162:0/236918587 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8644044e40 con 0x7f865c063340 2020-11-28T21:46:54.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.507+0000 7f8661507700 1 -- 172.21.15.162:0/236918587 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f865c057050 con 0x7f865c063340 2020-11-28T21:46:54.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.511+0000 7f86597fa700 1 -- 172.21.15.162:0/236918587 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16134 (secure 0 0 0) 0x7f8644048460 con 0x7f865c063340 2020-11-28T21:46:54.513 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:46:54.514 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":23,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:46:30.124354+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:46:27.155497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-28T21:46:28.137474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:46:54.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.511+0000 7f8661507700 1 -- 172.21.15.162:0/236918587 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8640055440 msgr2=0x7f8640057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:54.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.511+0000 7f8661507700 1 --2- 172.21.15.162:0/236918587 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8640055440 0x7f8640057900 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f864c00b000 tx=0x7f864c008040).stop 2020-11-28T21:46:54.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.511+0000 7f8661507700 1 -- 172.21.15.162:0/236918587 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f865c063340 msgr2=0x7f865c0aaa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:54.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.511+0000 7f8661507700 1 --2- 172.21.15.162:0/236918587 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f865c063340 0x7f865c0aaa30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f865c0ae230 tx=0x7f8644017000).stop 2020-11-28T21:46:54.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.511+0000 7f8661507700 1 -- 172.21.15.162:0/236918587 shutdown_connections 2020-11-28T21:46:54.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.511+0000 7f8661507700 1 --2- 172.21.15.162:0/236918587 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8640055440 0x7f8640057900 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:54.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.511+0000 7f8661507700 1 --2- 172.21.15.162:0/236918587 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f863c003470 0x7f865c0ab8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:54.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.511+0000 7f8661507700 1 --2- 172.21.15.162:0/236918587 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f865c0ae350 0x7f865c0ae800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:54.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.511+0000 7f8661507700 1 --2- 172.21.15.162:0/236918587 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f865c063340 0x7f865c0aaa30 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:54.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.511+0000 7f8661507700 1 -- 172.21.15.162:0/236918587 >> 172.21.15.162:0/236918587 conn(0x7f865c05b970 msgr2=0x7f865c148c20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:54.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.511+0000 7f8661507700 1 -- 172.21.15.162:0/236918587 shutdown_connections 2020-11-28T21:46:54.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.511+0000 7f8661507700 1 -- 172.21.15.162:0/236918587 wait complete. 2020-11-28T21:46:54.527 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-28T21:46:27.155497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-28T21:46:28.137474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-28T21:46:54.528 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:46:54.642 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.639+0000 7f422b59e700 1 Processor -- start 2020-11-28T21:46:54.643 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.639+0000 7f422b59e700 1 -- start start 2020-11-28T21:46:54.644 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.643+0000 7f422b59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f422c0ac500 0x7f422c05fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:54.644 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.643+0000 7f422b59e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f422c0ac8e0 con 0x7f422c0ae9d0 2020-11-28T21:46:54.644 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.643+0000 7f422b59e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f422c060690 con 0x7f422c13f560 2020-11-28T21:46:54.645 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.643+0000 7f422b59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f422c13ccf0 con 0x7f422c0ac500 2020-11-28T21:46:54.645 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.643+0000 7f422ad9d700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f422c13f560 0x7f422c13f9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:45018/0 (socket says 172.21.15.162:45018) 2020-11-28T21:46:54.645 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.643+0000 7f422ad9d700 1 -- 172.21.15.162:0/3385481762 learned_addr learned my addr 172.21.15.162:0/3385481762 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:54.645 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.643+0000 7f422a59c700 1 --2- 172.21.15.162:0/3385481762 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f422c0ac500 0x7f422c05fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:54.647 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.643+0000 7f4229d9b700 1 -- 172.21.15.162:0/3385481762 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1923500362 0 0) 0x7f422c060690 con 0x7f422c13f560 2020-11-28T21:46:54.647 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.643+0000 7f4229d9b700 1 -- 172.21.15.162:0/3385481762 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f420c003230 con 0x7f422c13f560 2020-11-28T21:46:54.647 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.643+0000 7f4229d9b700 1 -- 172.21.15.162:0/3385481762 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1229335693 0 0) 0x7f422c0ac8e0 con 0x7f422c0ae9d0 2020-11-28T21:46:54.648 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.643+0000 7f4229d9b700 1 -- 172.21.15.162:0/3385481762 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f422c060690 con 0x7f422c0ae9d0 2020-11-28T21:46:54.648 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.643+0000 7f422a59c700 1 -- 172.21.15.162:0/3385481762 >> v1:172.21.15.162:6789/0 conn(0x7f422c13f560 legacy=0x7f422c13f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:54.648 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.643+0000 7f422a59c700 1 -- 172.21.15.162:0/3385481762 >> v1:172.21.15.118:6789/0 conn(0x7f422c0ae9d0 legacy=0x7f422c141b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:54.648 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.643+0000 7f422a59c700 1 -- 172.21.15.162:0/3385481762 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f422c060930 con 0x7f422c0ac500 2020-11-28T21:46:54.649 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.643+0000 7f422a59c700 1 --2- 172.21.15.162:0/3385481762 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f422c0ac500 0x7f422c05fff0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f421c002f30 tx=0x7f421c006e50).ready entity=mon.2 client_cookie=b5b9fb2d334dab40 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:54.649 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.643+0000 7f4229d9b700 1 -- 172.21.15.162:0/3385481762 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f421c014070 con 0x7f422c0ac500 2020-11-28T21:46:54.649 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.643+0000 7f4229d9b700 1 -- 172.21.15.162:0/3385481762 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f421c00aa50 con 0x7f422c0ac500 2020-11-28T21:46:54.649 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.643+0000 7f4229d9b700 1 -- 172.21.15.162:0/3385481762 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f421c00adb0 con 0x7f422c0ac500 2020-11-28T21:46:54.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.643+0000 7f422b59e700 1 -- 172.21.15.162:0/3385481762 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f422c0ac500 msgr2=0x7f422c05fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:54.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.643+0000 7f422b59e700 1 --2- 172.21.15.162:0/3385481762 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f422c0ac500 0x7f422c05fff0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f421c002f30 tx=0x7f421c006e50).stop 2020-11-28T21:46:54.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.643+0000 7f422b59e700 1 -- 172.21.15.162:0/3385481762 shutdown_connections 2020-11-28T21:46:54.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.643+0000 7f422b59e700 1 --2- 172.21.15.162:0/3385481762 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f422c0ac500 0x7f422c05fff0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:54.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.643+0000 7f422b59e700 1 -- 172.21.15.162:0/3385481762 >> 172.21.15.162:0/3385481762 conn(0x7f422c0b57d0 msgr2=0x7f422c0b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:54.651 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.643+0000 7f422b59e700 1 -- 172.21.15.162:0/3385481762 shutdown_connections 2020-11-28T21:46:54.651 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.643+0000 7f422b59e700 1 -- 172.21.15.162:0/3385481762 wait complete. 2020-11-28T21:46:54.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.647+0000 7f422b59e700 1 Processor -- start 2020-11-28T21:46:54.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.647+0000 7f422b59e700 1 -- start start 2020-11-28T21:46:54.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.647+0000 7f422b59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f422c0ac500 0x7f422c148840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:54.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.647+0000 7f422b59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f422c0ae9d0 0x7f422c0b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:54.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.647+0000 7f422b59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f422c13f560 0x7f422c154c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:54.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.647+0000 7f422b59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422c0b2410 con 0x7f422c13f560 2020-11-28T21:46:54.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.647+0000 7f422b59e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422c0b08e0 con 0x7f422c0ae9d0 2020-11-28T21:46:54.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.647+0000 7f422b59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f422c0b1a30 con 0x7f422c0ac500 2020-11-28T21:46:54.655 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.651+0000 7f4221d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f422c0ae9d0 0x7f422c0b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:54.655 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.651+0000 7f4221d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f422c0ae9d0 0x7f422c0b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38178/0 (socket says 172.21.15.162:38178) 2020-11-28T21:46:54.655 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.651+0000 7f4221d9b700 1 -- 172.21.15.162:0/3304971998 learned_addr learned my addr 172.21.15.162:0/3304971998 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:54.655 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.651+0000 7f422ad9d700 1 --2- 172.21.15.162:0/3304971998 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f422c13f560 0x7f422c154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:54.655 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.651+0000 7f422a59c700 1 --2- 172.21.15.162:0/3304971998 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f422c0ac500 0x7f422c148840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:54.656 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.651+0000 7f4221d9b700 1 -- 172.21.15.162:0/3304971998 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f422c0ac500 msgr2=0x7f422c148840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:54.656 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.651+0000 7f4221d9b700 1 --2- 172.21.15.162:0/3304971998 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f422c0ac500 0x7f422c148840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:54.656 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.651+0000 7f4221d9b700 1 -- 172.21.15.162:0/3304971998 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f422c13f560 msgr2=0x7f422c154c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:54.656 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.651+0000 7f4221d9b700 1 --2- 172.21.15.162:0/3304971998 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f422c13f560 0x7f422c154c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:54.657 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.651+0000 7f4221d9b700 1 -- 172.21.15.162:0/3304971998 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f422c0b1cd0 con 0x7f422c0ae9d0 2020-11-28T21:46:54.657 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.651+0000 7f4221d9b700 1 --2- 172.21.15.162:0/3304971998 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f422c0ae9d0 0x7f422c0b03f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f422c0b2550 tx=0x7f4214007ab0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:54.657 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.651+0000 7f4223fff700 1 -- 172.21.15.162:0/3304971998 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f421400e4a0 con 0x7f422c0ae9d0 2020-11-28T21:46:54.657 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.651+0000 7f4223fff700 1 -- 172.21.15.162:0/3304971998 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f421400e600 con 0x7f422c0ae9d0 2020-11-28T21:46:54.658 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.651+0000 7f422b59e700 1 -- 172.21.15.162:0/3304971998 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f422c0b1f70 con 0x7f422c0ae9d0 2020-11-28T21:46:54.658 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.651+0000 7f4223fff700 1 -- 172.21.15.162:0/3304971998 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f421400e960 con 0x7f422c0ae9d0 2020-11-28T21:46:54.658 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.651+0000 7f422b59e700 1 -- 172.21.15.162:0/3304971998 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f422c0b0f20 con 0x7f422c0ae9d0 2020-11-28T21:46:54.659 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.651+0000 7f422b59e700 1 -- 172.21.15.162:0/3304971998 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f422c0b2100 con 0x7f422c0ae9d0 2020-11-28T21:46:54.659 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.651+0000 7f4223fff700 1 -- 172.21.15.162:0/3304971998 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4214019030 con 0x7f422c0ae9d0 2020-11-28T21:46:54.660 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.651+0000 7f4223fff700 1 --2- 172.21.15.162:0/3304971998 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4210055440 0x7f4210057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:54.660 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.655+0000 7f422a59c700 1 --2- 172.21.15.162:0/3304971998 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4210055440 0x7f4210057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:54.660 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.655+0000 7f4223fff700 1 -- 172.21.15.162:0/3304971998 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f421406d410 con 0x7f422c0ae9d0 2020-11-28T21:46:54.660 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.655+0000 7f422a59c700 1 --2- 172.21.15.162:0/3304971998 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4210055440 0x7f4210057900 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f421c01a000 tx=0x7f421c006e50).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:54.661 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.659+0000 7f4223fff700 1 -- 172.21.15.162:0/3304971998 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4214043710 con 0x7f422c0ae9d0 2020-11-28T21:46:54.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.803+0000 7f422b59e700 1 -- 172.21.15.162:0/3304971998 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f422c057050 con 0x7f422c0ae9d0 2020-11-28T21:46:54.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.803+0000 7f4223fff700 1 -- 172.21.15.162:0/3304971998 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v23) v1 ==== 110+0+10 (secure 0 0 0) 0x7f4214007bd0 con 0x7f422c0ae9d0 2020-11-28T21:46:54.805 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:46:54.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.803+0000 7f422b59e700 1 -- 172.21.15.162:0/3304971998 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4210055440 msgr2=0x7f4210057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:54.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.803+0000 7f422b59e700 1 --2- 172.21.15.162:0/3304971998 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4210055440 0x7f4210057900 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f421c01a000 tx=0x7f421c006e50).stop 2020-11-28T21:46:54.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.803+0000 7f422b59e700 1 -- 172.21.15.162:0/3304971998 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f422c0ae9d0 msgr2=0x7f422c0b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:54.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.803+0000 7f422b59e700 1 --2- 172.21.15.162:0/3304971998 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f422c0ae9d0 0x7f422c0b03f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f422c0b2550 tx=0x7f4214007ab0).stop 2020-11-28T21:46:54.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.803+0000 7f422b59e700 1 -- 172.21.15.162:0/3304971998 shutdown_connections 2020-11-28T21:46:54.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.803+0000 7f422b59e700 1 --2- 172.21.15.162:0/3304971998 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4210055440 0x7f4210057900 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:54.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.803+0000 7f422b59e700 1 --2- 172.21.15.162:0/3304971998 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f422c0ac500 0x7f422c148840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:54.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.803+0000 7f422b59e700 1 --2- 172.21.15.162:0/3304971998 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f422c13f560 0x7f422c154c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:54.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.803+0000 7f422b59e700 1 --2- 172.21.15.162:0/3304971998 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f422c0ae9d0 0x7f422c0b03f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:54.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.803+0000 7f422b59e700 1 -- 172.21.15.162:0/3304971998 >> 172.21.15.162:0/3304971998 conn(0x7f422c0b57d0 msgr2=0x7f422c13f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:54.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.803+0000 7f422b59e700 1 -- 172.21.15.162:0/3304971998 shutdown_connections 2020-11-28T21:46:54.810 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.803+0000 7f422b59e700 1 -- 172.21.15.162:0/3304971998 wait complete. 2020-11-28T21:46:54.819 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:46:54.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.935+0000 7f480b127700 1 Processor -- start 2020-11-28T21:46:54.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.935+0000 7f480b127700 1 -- start start 2020-11-28T21:46:54.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.939+0000 7f480b127700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f480413d0c0 0x7f48040b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:54.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.939+0000 7f480b127700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f480409a670 con 0x7f48040b13c0 2020-11-28T21:46:54.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.939+0000 7f480b127700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f480406aca0 con 0x7f480409a270 2020-11-28T21:46:54.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.939+0000 7f480b127700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f480413d4a0 con 0x7f480413d0c0 2020-11-28T21:46:54.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.939+0000 7f480a926700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f480409a270 0x7f480405bf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:45032/0 (socket says 172.21.15.162:45032) 2020-11-28T21:46:54.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.939+0000 7f480a926700 1 -- 172.21.15.162:0/1655790335 learned_addr learned my addr 172.21.15.162:0/1655790335 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:54.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.939+0000 7f480a125700 1 --2- 172.21.15.162:0/1655790335 >> v2:172.21.15.118:3300/0 conn(0x7f480413d0c0 0x7f48040b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:54.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.939+0000 7f4809123700 1 -- 172.21.15.162:0/1655790335 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4054367058 0 0) 0x7f480406aca0 con 0x7f480409a270 2020-11-28T21:46:54.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.939+0000 7f4809123700 1 -- 172.21.15.162:0/1655790335 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47ec003230 con 0x7f480409a270 2020-11-28T21:46:54.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.939+0000 7f4809123700 1 -- 172.21.15.162:0/1655790335 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2688470575 0 0) 0x7f47ec003230 con 0x7f480409a270 2020-11-28T21:46:54.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.939+0000 7f4809123700 1 -- 172.21.15.162:0/1655790335 >> v2:172.21.15.118:3300/0 conn(0x7f480413d0c0 msgr2=0x7f48040b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:54.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.939+0000 7f4809123700 1 --2- 172.21.15.162:0/1655790335 >> v2:172.21.15.118:3300/0 conn(0x7f480413d0c0 0x7f48040b0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:54.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.939+0000 7f4809123700 1 -- 172.21.15.162:0/1655790335 >> v1:172.21.15.118:6789/0 conn(0x7f48040b13c0 legacy=0x7f4804099b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:54.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.939+0000 7f4809123700 1 -- 172.21.15.162:0/1655790335 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f480406af40 con 0x7f480409a270 2020-11-28T21:46:54.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.939+0000 7f4809123700 1 -- 172.21.15.162:0/1655790335 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f47fc0016d0 con 0x7f480409a270 2020-11-28T21:46:54.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.939+0000 7f4809123700 1 -- 172.21.15.162:0/1655790335 >> v1:172.21.15.162:6789/0 conn(0x7f480409a270 legacy=0x7f480405bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:54.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.939+0000 7f4809123700 1 --2- 172.21.15.162:0/1655790335 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47ec003770 0x7f47ec003b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:54.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.939+0000 7f4809123700 1 --2- 172.21.15.162:0/1655790335 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f47ec0044d0 0x7f47ec004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:54.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.939+0000 7f4809123700 1 --2- 172.21.15.162:0/1655790335 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f47ec004e40 0x7f47ec00b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:54.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.939+0000 7f4809123700 1 -- 172.21.15.162:0/1655790335 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480413d4a0 con 0x7f47ec0044d0 2020-11-28T21:46:54.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.939+0000 7f4809123700 1 -- 172.21.15.162:0/1655790335 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ec0052f0 con 0x7f47ec003770 2020-11-28T21:46:54.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.939+0000 7f4809123700 1 -- 172.21.15.162:0/1655790335 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47ec00bc10 con 0x7f47ec004e40 2020-11-28T21:46:54.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.939+0000 7f480a125700 1 --2- 172.21.15.162:0/1655790335 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47ec003770 0x7f47ec003b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:54.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.939+0000 7f4809924700 1 --2- 172.21.15.162:0/1655790335 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f47ec0044d0 0x7f47ec004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:54.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.939+0000 7f480a926700 1 --2- 172.21.15.162:0/1655790335 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f47ec004e40 0x7f47ec00b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:54.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.939+0000 7f480a125700 1 -- 172.21.15.162:0/1655790335 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f47ec004e40 msgr2=0x7f47ec00b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:54.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.939+0000 7f480a125700 1 --2- 172.21.15.162:0/1655790335 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f47ec004e40 0x7f47ec00b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:54.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.939+0000 7f480a125700 1 -- 172.21.15.162:0/1655790335 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f47ec0044d0 msgr2=0x7f47ec004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:54.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.939+0000 7f480a125700 1 --2- 172.21.15.162:0/1655790335 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f47ec0044d0 0x7f47ec004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:54.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.939+0000 7f480a125700 1 -- 172.21.15.162:0/1655790335 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ec003230 con 0x7f47ec003770 2020-11-28T21:46:54.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.939+0000 7f480a926700 1 --2- 172.21.15.162:0/1655790335 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f47ec004e40 0x7f47ec00b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:46:54.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.939+0000 7f480a926700 1 -- 172.21.15.162:0/1655790335 reap_dead start 2020-11-28T21:46:54.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.943+0000 7f480a125700 1 --2- 172.21.15.162:0/1655790335 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47ec003770 0x7f47ec003b90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f480000c000 tx=0x7f48000091a0).ready entity=mon.1 client_cookie=eb57b630e99980b4 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:54.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.943+0000 7f4809123700 1 -- 172.21.15.162:0/1655790335 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f480000ec30 con 0x7f47ec003770 2020-11-28T21:46:54.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.943+0000 7f4809123700 1 -- 172.21.15.162:0/1655790335 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480000ed90 con 0x7f47ec003770 2020-11-28T21:46:54.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.943+0000 7f4809123700 1 -- 172.21.15.162:0/1655790335 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48000125e0 con 0x7f47ec003770 2020-11-28T21:46:54.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.943+0000 7f480b127700 1 -- 172.21.15.162:0/1655790335 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47ec003770 msgr2=0x7f47ec003b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:54.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.943+0000 7f480b127700 1 --2- 172.21.15.162:0/1655790335 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47ec003770 0x7f47ec003b90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f480000c000 tx=0x7f48000091a0).stop 2020-11-28T21:46:54.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.943+0000 7f480b127700 1 -- 172.21.15.162:0/1655790335 shutdown_connections 2020-11-28T21:46:54.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.943+0000 7f480b127700 1 --2- 172.21.15.162:0/1655790335 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47ec003770 0x7f47ec003b90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:54.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.943+0000 7f480b127700 1 -- 172.21.15.162:0/1655790335 >> 172.21.15.162:0/1655790335 conn(0x7f48040b5520 msgr2=0x7f48040b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:54.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.943+0000 7f480b127700 1 -- 172.21.15.162:0/1655790335 shutdown_connections 2020-11-28T21:46:54.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.943+0000 7f480b127700 1 -- 172.21.15.162:0/1655790335 wait complete. 2020-11-28T21:46:54.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.943+0000 7f480b127700 1 Processor -- start 2020-11-28T21:46:54.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.943+0000 7f480b127700 1 -- start start 2020-11-28T21:46:54.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.947+0000 7f480b127700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f47ec003770 0x7f480413ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:54.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.947+0000 7f480b127700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4804064df0 0x7f48040663d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:54.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.947+0000 7f480b127700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f48040668c0 0x7f4804063ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:54.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.947+0000 7f480b127700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4804098da0 con 0x7f47ec003770 2020-11-28T21:46:54.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.947+0000 7f480b127700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48040aa360 con 0x7f4804064df0 2020-11-28T21:46:54.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.947+0000 7f480b127700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48040aa600 con 0x7f48040668c0 2020-11-28T21:46:54.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.947+0000 7f4809924700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4804064df0 0x7f48040663d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:54.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.947+0000 7f4809924700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4804064df0 0x7f48040663d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38198/0 (socket says 172.21.15.162:38198) 2020-11-28T21:46:54.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.947+0000 7f4809924700 1 -- 172.21.15.162:0/3348920693 learned_addr learned my addr 172.21.15.162:0/3348920693 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:54.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.947+0000 7f480a926700 1 --2- 172.21.15.162:0/3348920693 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f48040668c0 0x7f4804063ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:54.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.947+0000 7f480a125700 1 --2- 172.21.15.162:0/3348920693 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f47ec003770 0x7f480413ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:54.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.947+0000 7f4809924700 1 -- 172.21.15.162:0/3348920693 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f48040668c0 msgr2=0x7f4804063ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:54.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.947+0000 7f4809924700 1 --2- 172.21.15.162:0/3348920693 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f48040668c0 0x7f4804063ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:54.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.947+0000 7f4809924700 1 -- 172.21.15.162:0/3348920693 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f47ec003770 msgr2=0x7f480413ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:54.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.947+0000 7f4809924700 1 --2- 172.21.15.162:0/3348920693 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f47ec003770 0x7f480413ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:54.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.947+0000 7f4809924700 1 -- 172.21.15.162:0/3348920693 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48040aa8a0 con 0x7f4804064df0 2020-11-28T21:46:54.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.947+0000 7f4809924700 1 --2- 172.21.15.162:0/3348920693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4804064df0 0x7f48040663d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f4804135d50 tx=0x7f47f4007b60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:54.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.947+0000 7f47f37fe700 1 -- 172.21.15.162:0/3348920693 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47f40028b0 con 0x7f4804064df0 2020-11-28T21:46:54.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.947+0000 7f47f37fe700 1 -- 172.21.15.162:0/3348920693 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f4002a10 con 0x7f4804064df0 2020-11-28T21:46:54.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.947+0000 7f480b127700 1 -- 172.21.15.162:0/3348920693 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48040aab40 con 0x7f4804064df0 2020-11-28T21:46:54.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.947+0000 7f47f37fe700 1 -- 172.21.15.162:0/3348920693 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47f4002d70 con 0x7f4804064df0 2020-11-28T21:46:54.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.947+0000 7f480b127700 1 -- 172.21.15.162:0/3348920693 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4804065270 con 0x7f4804064df0 2020-11-28T21:46:54.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.951+0000 7f480b127700 1 -- 172.21.15.162:0/3348920693 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48040670b0 con 0x7f4804064df0 2020-11-28T21:46:54.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.951+0000 7f47f37fe700 1 -- 172.21.15.162:0/3348920693 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f47f401b030 con 0x7f4804064df0 2020-11-28T21:46:54.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.951+0000 7f47f37fe700 1 --2- 172.21.15.162:0/3348920693 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f47e4055330 0x7f47e40577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:54.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.951+0000 7f47f37fe700 1 -- 172.21.15.162:0/3348920693 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f47f406dca0 con 0x7f4804064df0 2020-11-28T21:46:54.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.951+0000 7f480a125700 1 --2- 172.21.15.162:0/3348920693 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f47e4055330 0x7f47e40577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:54.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.951+0000 7f480a125700 1 --2- 172.21.15.162:0/3348920693 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f47e4055330 0x7f47e40577f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f480001d000 tx=0x7f480001a000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:54.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:54.959+0000 7f47f37fe700 1 -- 172.21.15.162:0/3348920693 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f47f4072070 con 0x7f4804064df0 2020-11-28T21:46:55.104 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.103+0000 7f480b127700 1 -- 172.21.15.162:0/3348920693 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f48040aacd0 con 0x7f4804064df0 2020-11-28T21:46:55.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.103+0000 7f47f37fe700 1 -- 172.21.15.162:0/3348920693 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f47f4043fb0 con 0x7f4804064df0 2020-11-28T21:46:55.105 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:46:55.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.103+0000 7f480b127700 1 -- 172.21.15.162:0/3348920693 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f47e4055330 msgr2=0x7f47e40577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:55.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.103+0000 7f480b127700 1 --2- 172.21.15.162:0/3348920693 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f47e4055330 0x7f47e40577f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f480001d000 tx=0x7f480001a000).stop 2020-11-28T21:46:55.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.103+0000 7f480b127700 1 -- 172.21.15.162:0/3348920693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4804064df0 msgr2=0x7f48040663d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:55.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.103+0000 7f480b127700 1 --2- 172.21.15.162:0/3348920693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4804064df0 0x7f48040663d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f4804135d50 tx=0x7f47f4007b60).stop 2020-11-28T21:46:55.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.103+0000 7f480b127700 1 -- 172.21.15.162:0/3348920693 shutdown_connections 2020-11-28T21:46:55.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.103+0000 7f480b127700 1 --2- 172.21.15.162:0/3348920693 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f47e4055330 0x7f47e40577f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:55.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.103+0000 7f480b127700 1 --2- 172.21.15.162:0/3348920693 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f48040668c0 0x7f4804063ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:55.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.103+0000 7f480b127700 1 --2- 172.21.15.162:0/3348920693 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f47ec003770 0x7f480413ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:55.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.103+0000 7f480b127700 1 --2- 172.21.15.162:0/3348920693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4804064df0 0x7f48040663d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:55.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.103+0000 7f480b127700 1 -- 172.21.15.162:0/3348920693 >> 172.21.15.162:0/3348920693 conn(0x7f48040b5520 msgr2=0x7f48040b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:55.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.103+0000 7f480b127700 1 -- 172.21.15.162:0/3348920693 shutdown_connections 2020-11-28T21:46:55.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.103+0000 7f480b127700 1 -- 172.21.15.162:0/3348920693 wait complete. 2020-11-28T21:46:55.124 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-28T21:46:55.242 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.239+0000 7f04cd4ac700 1 Processor -- start 2020-11-28T21:46:55.242 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.239+0000 7f04cd4ac700 1 -- start start 2020-11-28T21:46:55.243 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.239+0000 7f04cd4ac700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f04c80b6a20 0x7f04c81402c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:55.243 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.239+0000 7f04cd4ac700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f04c805f8f0 0x7f04c80b1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:55.243 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.239+0000 7f04cd4ac700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04c80b1f50 con 0x7f04c8060d90 2020-11-28T21:46:55.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.239+0000 7f04cd4ac700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04c80b6e00 con 0x7f04c80b6a20 2020-11-28T21:46:55.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.239+0000 7f04cd4ac700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04c80b2180 con 0x7f04c805f8f0 2020-11-28T21:46:55.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.239+0000 7f04c67fc700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f04c805f8f0 0x7f04c80b1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:55.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.243+0000 7f04c6ffd700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f04c80b6a20 0x7f04c81402c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:55.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.243+0000 7f04c67fc700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f04c805f8f0 0x7f04c80b1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:35430/0 (socket says 172.21.15.162:35430) 2020-11-28T21:46:55.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.243+0000 7f04c67fc700 1 -- 172.21.15.162:0/1992033475 learned_addr learned my addr 172.21.15.162:0/1992033475 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:55.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.243+0000 7f04c77fe700 1 --1- 172.21.15.162:0/1992033475 >> v1:172.21.15.118:6789/0 conn(0x7f04c8060d90 0x7f04c805ff50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.162:35200/0 (socket says 172.21.15.162:35200) 2020-11-28T21:46:55.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.243+0000 7f04c6ffd700 1 -- 172.21.15.162:0/1992033475 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f04c805f8f0 msgr2=0x7f04c80b1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:55.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.243+0000 7f04c6ffd700 1 --2- 172.21.15.162:0/1992033475 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f04c805f8f0 0x7f04c80b1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:55.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.243+0000 7f04c6ffd700 1 -- 172.21.15.162:0/1992033475 >> v1:172.21.15.118:6789/0 conn(0x7f04c8060d90 legacy=0x7f04c805ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:55.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.243+0000 7f04c6ffd700 1 -- 172.21.15.162:0/1992033475 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c80b0970 con 0x7f04c80b6a20 2020-11-28T21:46:55.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.243+0000 7f04c5ffb700 1 -- 172.21.15.162:0/1992033475 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 670167334 0 0) 0x7f04c80b1f50 con 0x7f04c8060d90 2020-11-28T21:46:55.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.243+0000 7f04c6ffd700 1 --2- 172.21.15.162:0/1992033475 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f04c80b6a20 0x7f04c81402c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f04b0002f30 tx=0x7f04b0006d60).ready entity=mon.0 client_cookie=fe6f08e45dac6a8 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:55.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.243+0000 7f04c5ffb700 1 -- 172.21.15.162:0/1992033475 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04b0009600 con 0x7f04c80b6a20 2020-11-28T21:46:55.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.243+0000 7f04c5ffb700 1 -- 172.21.15.162:0/1992033475 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b0009760 con 0x7f04c80b6a20 2020-11-28T21:46:55.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.243+0000 7f04c5ffb700 1 -- 172.21.15.162:0/1992033475 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04b00098d0 con 0x7f04c80b6a20 2020-11-28T21:46:55.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.243+0000 7f04cd4ac700 1 -- 172.21.15.162:0/1992033475 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f04c80b6a20 msgr2=0x7f04c81402c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:55.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.243+0000 7f04cd4ac700 1 --2- 172.21.15.162:0/1992033475 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f04c80b6a20 0x7f04c81402c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f04b0002f30 tx=0x7f04b0006d60).stop 2020-11-28T21:46:55.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.243+0000 7f04cd4ac700 1 -- 172.21.15.162:0/1992033475 shutdown_connections 2020-11-28T21:46:55.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.243+0000 7f04cd4ac700 1 --2- 172.21.15.162:0/1992033475 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f04c80b6a20 0x7f04c81402c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:55.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.243+0000 7f04cd4ac700 1 --2- 172.21.15.162:0/1992033475 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f04c805f8f0 0x7f04c80b1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:55.253 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.243+0000 7f04cd4ac700 1 -- 172.21.15.162:0/1992033475 >> 172.21.15.162:0/1992033475 conn(0x7f04c8062150 msgr2=0x7f04c80b5fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:55.253 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.243+0000 7f04cd4ac700 1 -- 172.21.15.162:0/1992033475 shutdown_connections 2020-11-28T21:46:55.253 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.243+0000 7f04cd4ac700 1 -- 172.21.15.162:0/1992033475 wait complete. 2020-11-28T21:46:55.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.247+0000 7f04cd4ac700 1 Processor -- start 2020-11-28T21:46:55.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.247+0000 7f04cd4ac700 1 -- start start 2020-11-28T21:46:55.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.247+0000 7f04cd4ac700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f04c805f8f0 0x7f04c80ac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:55.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.247+0000 7f04cd4ac700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f04c8060d90 0x7f04c8138bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:55.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.247+0000 7f04cd4ac700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f04c80b6a20 0x7f04c814e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:55.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.247+0000 7f04cd4ac700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c81519c0 con 0x7f04c80b6a20 2020-11-28T21:46:55.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.247+0000 7f04cd4ac700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c80acb90 con 0x7f04c805f8f0 2020-11-28T21:46:55.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.247+0000 7f04cd4ac700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04c8151bf0 con 0x7f04c8060d90 2020-11-28T21:46:55.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.247+0000 7f04c6ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f04c805f8f0 0x7f04c80ac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:55.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.247+0000 7f04c6ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f04c805f8f0 0x7f04c80ac6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38210/0 (socket says 172.21.15.162:38210) 2020-11-28T21:46:55.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.247+0000 7f04c6ffd700 1 -- 172.21.15.162:0/2458038667 learned_addr learned my addr 172.21.15.162:0/2458038667 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:55.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.247+0000 7f04c67fc700 1 --2- 172.21.15.162:0/2458038667 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f04c8060d90 0x7f04c8138bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:55.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.247+0000 7f04c77fe700 1 --2- 172.21.15.162:0/2458038667 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f04c80b6a20 0x7f04c814e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:55.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.247+0000 7f04c6ffd700 1 -- 172.21.15.162:0/2458038667 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f04c8060d90 msgr2=0x7f04c8138bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:55.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.247+0000 7f04c6ffd700 1 --2- 172.21.15.162:0/2458038667 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f04c8060d90 0x7f04c8138bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:55.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.247+0000 7f04c6ffd700 1 -- 172.21.15.162:0/2458038667 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f04c80b6a20 msgr2=0x7f04c814e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:55.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.247+0000 7f04c6ffd700 1 --2- 172.21.15.162:0/2458038667 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f04c80b6a20 0x7f04c814e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:55.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.247+0000 7f04c6ffd700 1 -- 172.21.15.162:0/2458038667 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c8151e90 con 0x7f04c805f8f0 2020-11-28T21:46:55.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.247+0000 7f04c6ffd700 1 --2- 172.21.15.162:0/2458038667 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f04c805f8f0 0x7f04c80ac6a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f04c80ace50 tx=0x7f04b0016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:55.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.247+0000 7f04bffff700 1 -- 172.21.15.162:0/2458038667 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04b00056c0 con 0x7f04c805f8f0 2020-11-28T21:46:55.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.247+0000 7f04bffff700 1 -- 172.21.15.162:0/2458038667 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b0005820 con 0x7f04c805f8f0 2020-11-28T21:46:55.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.247+0000 7f04cd4ac700 1 -- 172.21.15.162:0/2458038667 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04c8152130 con 0x7f04c805f8f0 2020-11-28T21:46:55.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.247+0000 7f04bffff700 1 -- 172.21.15.162:0/2458038667 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04b0005b80 con 0x7f04c805f8f0 2020-11-28T21:46:55.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.247+0000 7f04cd4ac700 1 -- 172.21.15.162:0/2458038667 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c81525d0 con 0x7f04c805f8f0 2020-11-28T21:46:55.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.251+0000 7f04cd4ac700 1 -- 172.21.15.162:0/2458038667 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04c81522c0 con 0x7f04c805f8f0 2020-11-28T21:46:55.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.251+0000 7f04bffff700 1 -- 172.21.15.162:0/2458038667 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f04b000f070 con 0x7f04c805f8f0 2020-11-28T21:46:55.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.251+0000 7f04bffff700 1 --2- 172.21.15.162:0/2458038667 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f04ac055330 0x7f04ac0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:55.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.251+0000 7f04bffff700 1 -- 172.21.15.162:0/2458038667 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f04b006dd00 con 0x7f04c805f8f0 2020-11-28T21:46:55.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.251+0000 7f04c67fc700 1 --2- 172.21.15.162:0/2458038667 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f04ac055330 0x7f04ac0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:55.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.251+0000 7f04c67fc700 1 --2- 172.21.15.162:0/2458038667 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f04ac055330 0x7f04ac0577f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f04b8008000 tx=0x7f04b8008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:55.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.255+0000 7f04bffff700 1 -- 172.21.15.162:0/2458038667 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f04b0071030 con 0x7f04c805f8f0 2020-11-28T21:46:55.399 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.395+0000 7f04cd4ac700 1 -- 172.21.15.162:0/2458038667 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f04c8057050 con 0x7f04c805f8f0 2020-11-28T21:46:55.400 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.395+0000 7f04bffff700 1 -- 172.21.15.162:0/2458038667 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+10 (secure 0 0 0) 0x7f04b0015070 con 0x7f04c805f8f0 2020-11-28T21:46:55.400 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:46:55.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.399+0000 7f04cd4ac700 1 -- 172.21.15.162:0/2458038667 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f04ac055330 msgr2=0x7f04ac0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:55.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.399+0000 7f04cd4ac700 1 --2- 172.21.15.162:0/2458038667 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f04ac055330 0x7f04ac0577f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f04b8008000 tx=0x7f04b8008040).stop 2020-11-28T21:46:55.404 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.399+0000 7f04cd4ac700 1 -- 172.21.15.162:0/2458038667 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f04c805f8f0 msgr2=0x7f04c80ac6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:55.405 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.399+0000 7f04cd4ac700 1 --2- 172.21.15.162:0/2458038667 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f04c805f8f0 0x7f04c80ac6a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f04c80ace50 tx=0x7f04b0016000).stop 2020-11-28T21:46:55.405 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.399+0000 7f04cd4ac700 1 -- 172.21.15.162:0/2458038667 shutdown_connections 2020-11-28T21:46:55.405 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.399+0000 7f04cd4ac700 1 --2- 172.21.15.162:0/2458038667 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f04ac055330 0x7f04ac0577f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:55.406 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.399+0000 7f04cd4ac700 1 --2- 172.21.15.162:0/2458038667 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f04c80b6a20 0x7f04c814e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:55.406 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.399+0000 7f04cd4ac700 1 --2- 172.21.15.162:0/2458038667 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f04c805f8f0 0x7f04c80ac6a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:55.407 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.399+0000 7f04cd4ac700 1 --2- 172.21.15.162:0/2458038667 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f04c8060d90 0x7f04c8138bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:55.407 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.399+0000 7f04cd4ac700 1 -- 172.21.15.162:0/2458038667 >> 172.21.15.162:0/2458038667 conn(0x7f04c8062150 msgr2=0x7f04c8137ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:55.407 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.399+0000 7f04cd4ac700 1 -- 172.21.15.162:0/2458038667 shutdown_connections 2020-11-28T21:46:55.408 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.399+0000 7f04cd4ac700 1 -- 172.21.15.162:0/2458038667 wait complete. 2020-11-28T21:46:55.417 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-28T21:46:55.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.531+0000 7f39d9ea2700 1 Processor -- start 2020-11-28T21:46:55.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.531+0000 7f39d9ea2700 1 -- start start 2020-11-28T21:46:55.538 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.535+0000 7f39d9ea2700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f39d40b1eb0 0x7f39d40b2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:55.538 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.535+0000 7f39d9ea2700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f39d40af6a0 0x7f39d4060a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:55.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.535+0000 7f39d9ea2700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39d41471f0 con 0x7f39d4144350 2020-11-28T21:46:55.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.535+0000 7f39d9ea2700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39d4060090 con 0x7f39d40b1eb0 2020-11-28T21:46:55.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.535+0000 7f39d9ea2700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39d4060330 con 0x7f39d40af6a0 2020-11-28T21:46:55.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.535+0000 7f39d3fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f39d40af6a0 0x7f39d4060a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:55.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.535+0000 7f39d96a1700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f39d4144350 0x7f39d414a440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.162:35214/0 (socket says 172.21.15.162:35214) 2020-11-28T21:46:55.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.535+0000 7f39d3fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f39d40af6a0 0x7f39d4060a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:35442/0 (socket says 172.21.15.162:35442) 2020-11-28T21:46:55.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.535+0000 7f39d96a1700 1 -- 172.21.15.162:0/1694824406 learned_addr learned my addr 172.21.15.162:0/1694824406 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:55.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.535+0000 7f39d8ea0700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f39d40b1eb0 0x7f39d40b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:55.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.535+0000 7f39d37fe700 1 -- 172.21.15.162:0/1694824406 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2950715545 0 0) 0x7f39d41471f0 con 0x7f39d4144350 2020-11-28T21:46:55.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.535+0000 7f39d37fe700 1 -- 172.21.15.162:0/1694824406 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39b4003230 con 0x7f39d4144350 2020-11-28T21:46:55.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.535+0000 7f39d3fff700 1 -- 172.21.15.162:0/1694824406 >> v2:172.21.15.118:3300/0 conn(0x7f39d40b1eb0 msgr2=0x7f39d40b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:55.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.535+0000 7f39d3fff700 1 --2- 172.21.15.162:0/1694824406 >> v2:172.21.15.118:3300/0 conn(0x7f39d40b1eb0 0x7f39d40b2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:55.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.535+0000 7f39d3fff700 1 -- 172.21.15.162:0/1694824406 >> v1:172.21.15.118:6789/0 conn(0x7f39d4144350 legacy=0x7f39d414a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:55.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.535+0000 7f39d3fff700 1 -- 172.21.15.162:0/1694824406 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d40605d0 con 0x7f39d40af6a0 2020-11-28T21:46:55.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.535+0000 7f39d8ea0700 1 --2- 172.21.15.162:0/1694824406 >> v2:172.21.15.118:3300/0 conn(0x7f39d40b1eb0 0x7f39d40b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:46:55.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.535+0000 7f39d3fff700 1 --2- 172.21.15.162:0/1694824406 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f39d40af6a0 0x7f39d4060a70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f39bc002f30 tx=0x7f39bc006dc0).ready entity=mon.2 client_cookie=d12f7788c1e70faa server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:55.543 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.535+0000 7f39d37fe700 1 -- 172.21.15.162:0/1694824406 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f39bc0096a0 con 0x7f39d40af6a0 2020-11-28T21:46:55.544 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.535+0000 7f39d37fe700 1 -- 172.21.15.162:0/1694824406 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39bc009800 con 0x7f39d40af6a0 2020-11-28T21:46:55.544 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.535+0000 7f39d37fe700 1 -- 172.21.15.162:0/1694824406 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f39bc009b60 con 0x7f39d40af6a0 2020-11-28T21:46:55.545 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.535+0000 7f39d9ea2700 1 -- 172.21.15.162:0/1694824406 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f39d40af6a0 msgr2=0x7f39d4060a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:55.545 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.535+0000 7f39d9ea2700 1 --2- 172.21.15.162:0/1694824406 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f39d40af6a0 0x7f39d4060a70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f39bc002f30 tx=0x7f39bc006dc0).stop 2020-11-28T21:46:55.548 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.535+0000 7f39d9ea2700 1 -- 172.21.15.162:0/1694824406 shutdown_connections 2020-11-28T21:46:55.548 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.535+0000 7f39d9ea2700 1 --2- 172.21.15.162:0/1694824406 >> v2:172.21.15.118:3300/0 conn(0x7f39d40b1eb0 0x7f39d40b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:55.548 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.535+0000 7f39d9ea2700 1 --2- 172.21.15.162:0/1694824406 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f39d40af6a0 0x7f39d4060a70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:55.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.535+0000 7f39d9ea2700 1 -- 172.21.15.162:0/1694824406 >> 172.21.15.162:0/1694824406 conn(0x7f39d40b57d0 msgr2=0x7f39d40b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:55.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.535+0000 7f39d9ea2700 1 -- 172.21.15.162:0/1694824406 shutdown_connections 2020-11-28T21:46:55.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.535+0000 7f39d9ea2700 1 -- 172.21.15.162:0/1694824406 wait complete. 2020-11-28T21:46:55.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.539+0000 7f39d9ea2700 1 Processor -- start 2020-11-28T21:46:55.550 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.539+0000 7f39d9ea2700 1 -- start start 2020-11-28T21:46:55.550 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.539+0000 7f39d9ea2700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f39d40af6a0 0x7f39d409ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:55.550 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.539+0000 7f39d9ea2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f39d40b1eb0 0x7f39d405c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:55.550 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.539+0000 7f39d9ea2700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f39d4144350 0x7f39d4155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:55.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.539+0000 7f39d9ea2700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d4065d70 con 0x7f39d40af6a0 2020-11-28T21:46:55.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.539+0000 7f39d9ea2700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d409e490 con 0x7f39d40b1eb0 2020-11-28T21:46:55.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.539+0000 7f39d9ea2700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39d409e730 con 0x7f39d4144350 2020-11-28T21:46:55.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.539+0000 7f39d3fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f39d40b1eb0 0x7f39d405c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:55.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.539+0000 7f39d3fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f39d40b1eb0 0x7f39d405c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38226/0 (socket says 172.21.15.162:38226) 2020-11-28T21:46:55.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.539+0000 7f39d3fff700 1 -- 172.21.15.162:0/3825014768 learned_addr learned my addr 172.21.15.162:0/3825014768 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:55.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.539+0000 7f39d96a1700 1 --2- 172.21.15.162:0/3825014768 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f39d4144350 0x7f39d4155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:55.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.539+0000 7f39d8ea0700 1 --2- 172.21.15.162:0/3825014768 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f39d40af6a0 0x7f39d409ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:55.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.539+0000 7f39d3fff700 1 -- 172.21.15.162:0/3825014768 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f39d4144350 msgr2=0x7f39d4155fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:55.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.539+0000 7f39d3fff700 1 --2- 172.21.15.162:0/3825014768 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f39d4144350 0x7f39d4155fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:55.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.539+0000 7f39d3fff700 1 -- 172.21.15.162:0/3825014768 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f39d40af6a0 msgr2=0x7f39d409ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:55.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.539+0000 7f39d3fff700 1 --2- 172.21.15.162:0/3825014768 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f39d40af6a0 0x7f39d409ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:55.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.539+0000 7f39d3fff700 1 -- 172.21.15.162:0/3825014768 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d409e9d0 con 0x7f39d40b1eb0 2020-11-28T21:46:55.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.539+0000 7f39d8ea0700 1 --2- 172.21.15.162:0/3825014768 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f39d40af6a0 0x7f39d409ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:46:55.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.539+0000 7f39d3fff700 1 --2- 172.21.15.162:0/3825014768 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f39d40b1eb0 0x7f39d405c400 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f39bc002f30 tx=0x7f39bc015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:55.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.539+0000 7f39d1ffb700 1 -- 172.21.15.162:0/3825014768 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f39bc011ad0 con 0x7f39d40b1eb0 2020-11-28T21:46:55.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.539+0000 7f39d1ffb700 1 -- 172.21.15.162:0/3825014768 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39bc011c30 con 0x7f39d40b1eb0 2020-11-28T21:46:55.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.539+0000 7f39d9ea2700 1 -- 172.21.15.162:0/3825014768 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39d40b0160 con 0x7f39d40b1eb0 2020-11-28T21:46:55.555 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.539+0000 7f39d1ffb700 1 -- 172.21.15.162:0/3825014768 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f39bc01d450 con 0x7f39d40b1eb0 2020-11-28T21:46:55.555 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.539+0000 7f39d9ea2700 1 -- 172.21.15.162:0/3825014768 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39d40659f0 con 0x7f39d40b1eb0 2020-11-28T21:46:55.555 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.543+0000 7f39d1ffb700 1 -- 172.21.15.162:0/3825014768 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f39bc024030 con 0x7f39d40b1eb0 2020-11-28T21:46:55.555 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.543+0000 7f39d9ea2700 1 -- 172.21.15.162:0/3825014768 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39d409eb60 con 0x7f39d40b1eb0 2020-11-28T21:46:55.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.543+0000 7f39d1ffb700 1 --2- 172.21.15.162:0/3825014768 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f39b80553b0 0x7f39b8057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:55.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.543+0000 7f39d1ffb700 1 -- 172.21.15.162:0/3825014768 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f39bc06ef70 con 0x7f39d40b1eb0 2020-11-28T21:46:55.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.543+0000 7f39d8ea0700 1 --2- 172.21.15.162:0/3825014768 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f39b80553b0 0x7f39b8057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:55.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.543+0000 7f39d8ea0700 1 --2- 172.21.15.162:0/3825014768 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f39b80553b0 0x7f39b8057870 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f39c400b000 tx=0x7f39c4008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:55.557 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.547+0000 7f39d1ffb700 1 -- 172.21.15.162:0/3825014768 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f39bc045160 con 0x7f39d40b1eb0 2020-11-28T21:46:55.689 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.687+0000 7f39d9ea2700 1 -- 172.21.15.162:0/3825014768 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f39d4057050 con 0x7f39d40b1eb0 2020-11-28T21:46:55.690 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.687+0000 7f39d1ffb700 1 -- 172.21.15.162:0/3825014768 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+10 (secure 0 0 0) 0x7f39bc022070 con 0x7f39d40b1eb0 2020-11-28T21:46:55.694 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:46:55.695 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.691+0000 7f39d9ea2700 1 -- 172.21.15.162:0/3825014768 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f39b80553b0 msgr2=0x7f39b8057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:55.695 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.691+0000 7f39d9ea2700 1 --2- 172.21.15.162:0/3825014768 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f39b80553b0 0x7f39b8057870 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f39c400b000 tx=0x7f39c4008040).stop 2020-11-28T21:46:55.695 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.691+0000 7f39d9ea2700 1 -- 172.21.15.162:0/3825014768 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f39d40b1eb0 msgr2=0x7f39d405c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:55.696 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.691+0000 7f39d9ea2700 1 --2- 172.21.15.162:0/3825014768 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f39d40b1eb0 0x7f39d405c400 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f39bc002f30 tx=0x7f39bc015000).stop 2020-11-28T21:46:55.696 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.691+0000 7f39d9ea2700 1 -- 172.21.15.162:0/3825014768 shutdown_connections 2020-11-28T21:46:55.696 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.691+0000 7f39d9ea2700 1 --2- 172.21.15.162:0/3825014768 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f39b80553b0 0x7f39b8057870 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:55.696 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.691+0000 7f39d9ea2700 1 --2- 172.21.15.162:0/3825014768 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f39d4144350 0x7f39d4155fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:55.696 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.691+0000 7f39d9ea2700 1 --2- 172.21.15.162:0/3825014768 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f39d40af6a0 0x7f39d409ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:55.697 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.691+0000 7f39d9ea2700 1 --2- 172.21.15.162:0/3825014768 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f39d40b1eb0 0x7f39d405c400 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:55.697 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.691+0000 7f39d9ea2700 1 -- 172.21.15.162:0/3825014768 >> 172.21.15.162:0/3825014768 conn(0x7f39d40b57d0 msgr2=0x7f39d40b0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:55.697 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.691+0000 7f39d9ea2700 1 -- 172.21.15.162:0/3825014768 shutdown_connections 2020-11-28T21:46:55.697 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.691+0000 7f39d9ea2700 1 -- 172.21.15.162:0/3825014768 wait complete. 2020-11-28T21:46:55.716 INFO:tasks.cephfs_test_runner:Starting test: test_drop_cache_command (tasks.cephfs.test_misc.TestCacheDrop) 2020-11-28T21:46:55.717 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command' 2020-11-28T21:46:55.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.831+0000 7f04b48bd700 1 Processor -- start 2020-11-28T21:46:55.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.831+0000 7f04b48bd700 1 -- start start 2020-11-28T21:46:55.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.831+0000 7f04b48bd700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f04b013f6c0 0x7f04b00669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:55.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.831+0000 7f04b48bd700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04b0065050 con 0x7f04b0064c70 2020-11-28T21:46:55.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.831+0000 7f04b48bd700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04b013fc40 con 0x7f04b0065750 2020-11-28T21:46:55.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.831+0000 7f04b48bd700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04b013fee0 con 0x7f04b013f6c0 2020-11-28T21:46:55.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.831+0000 7f04affff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f04b0065750 0x7f04b013c260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:45080/0 (socket says 172.21.15.162:45080) 2020-11-28T21:46:55.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.831+0000 7f04affff700 1 -- 172.21.15.162:0/75186475 learned_addr learned my addr 172.21.15.162:0/75186475 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:55.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.831+0000 7f04aeffd700 1 --2- 172.21.15.162:0/75186475 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f04b013f6c0 0x7f04b00669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:55.838 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.831+0000 7f04ae7fc700 1 -- 172.21.15.162:0/75186475 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 120653702 0 0) 0x7f04b013fc40 con 0x7f04b0065750 2020-11-28T21:46:55.838 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.831+0000 7f04ae7fc700 1 -- 172.21.15.162:0/75186475 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f049c003230 con 0x7f04b0065750 2020-11-28T21:46:55.839 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.831+0000 7f04ae7fc700 1 -- 172.21.15.162:0/75186475 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2993093246 0 0) 0x7f04b0065050 con 0x7f04b0064c70 2020-11-28T21:46:55.839 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.831+0000 7f04aeffd700 1 -- 172.21.15.162:0/75186475 >> v1:172.21.15.162:6789/0 conn(0x7f04b0065750 legacy=0x7f04b013c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:55.839 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.831+0000 7f04aeffd700 1 -- 172.21.15.162:0/75186475 >> v1:172.21.15.118:6789/0 conn(0x7f04b0064c70 legacy=0x7f04b0066290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:55.839 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.831+0000 7f04aeffd700 1 -- 172.21.15.162:0/75186475 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b0140180 con 0x7f04b013f6c0 2020-11-28T21:46:55.840 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.835+0000 7f04aeffd700 1 --2- 172.21.15.162:0/75186475 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f04b013f6c0 0x7f04b00669a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f04a4002f30 tx=0x7f04a4006dc0).ready entity=mon.2 client_cookie=e520c5a2809f05a3 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:55.840 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.835+0000 7f04ae7fc700 1 -- 172.21.15.162:0/75186475 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04a40097b0 con 0x7f04b013f6c0 2020-11-28T21:46:55.840 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.835+0000 7f04ae7fc700 1 -- 172.21.15.162:0/75186475 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04a4009910 con 0x7f04b013f6c0 2020-11-28T21:46:55.840 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.835+0000 7f04ae7fc700 1 -- 172.21.15.162:0/75186475 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04a4009c70 con 0x7f04b013f6c0 2020-11-28T21:46:55.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.835+0000 7f04b48bd700 1 -- 172.21.15.162:0/75186475 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f04b013f6c0 msgr2=0x7f04b00669a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:55.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.835+0000 7f04b48bd700 1 --2- 172.21.15.162:0/75186475 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f04b013f6c0 0x7f04b00669a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f04a4002f30 tx=0x7f04a4006dc0).stop 2020-11-28T21:46:55.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.835+0000 7f04b48bd700 1 -- 172.21.15.162:0/75186475 shutdown_connections 2020-11-28T21:46:55.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.835+0000 7f04b48bd700 1 --2- 172.21.15.162:0/75186475 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f04b013f6c0 0x7f04b00669a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:55.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.835+0000 7f04b48bd700 1 -- 172.21.15.162:0/75186475 >> 172.21.15.162:0/75186475 conn(0x7f04b00b5da0 msgr2=0x7f04b00b6180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:55.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.835+0000 7f04b48bd700 1 -- 172.21.15.162:0/75186475 shutdown_connections 2020-11-28T21:46:55.843 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.835+0000 7f04b48bd700 1 -- 172.21.15.162:0/75186475 wait complete. 2020-11-28T21:46:55.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.839+0000 7f04b48bd700 1 Processor -- start 2020-11-28T21:46:55.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.839+0000 7f04b48bd700 1 -- start start 2020-11-28T21:46:55.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.839+0000 7f04b48bd700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f04b0064c70 0x7f04b00664f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:55.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.839+0000 7f04b48bd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f04b0065750 0x7f04b0151270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:55.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.839+0000 7f04b48bd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f04b013f6c0 0x7f04b0151760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:55.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.839+0000 7f04b48bd700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b00b3bb0 con 0x7f04b013f6c0 2020-11-28T21:46:55.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.839+0000 7f04b48bd700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b00b3e50 con 0x7f04b0065750 2020-11-28T21:46:55.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.839+0000 7f04b48bd700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04b00b40f0 con 0x7f04b0064c70 2020-11-28T21:46:55.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.839+0000 7f04aeffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f04b0065750 0x7f04b0151270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:55.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.839+0000 7f04aeffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f04b0065750 0x7f04b0151270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38240/0 (socket says 172.21.15.162:38240) 2020-11-28T21:46:55.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.839+0000 7f04aeffd700 1 -- 172.21.15.162:0/859568659 learned_addr learned my addr 172.21.15.162:0/859568659 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:55.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.839+0000 7f04affff700 1 --2- 172.21.15.162:0/859568659 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f04b013f6c0 0x7f04b0151760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:55.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.839+0000 7f04af7fe700 1 --2- 172.21.15.162:0/859568659 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f04b0064c70 0x7f04b00664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:55.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.839+0000 7f04aeffd700 1 -- 172.21.15.162:0/859568659 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f04b0064c70 msgr2=0x7f04b00664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:55.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.839+0000 7f04aeffd700 1 --2- 172.21.15.162:0/859568659 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f04b0064c70 0x7f04b00664f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:55.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.839+0000 7f04aeffd700 1 -- 172.21.15.162:0/859568659 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f04b013f6c0 msgr2=0x7f04b0151760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:55.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.839+0000 7f04aeffd700 1 --2- 172.21.15.162:0/859568659 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f04b013f6c0 0x7f04b0151760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:55.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.839+0000 7f04aeffd700 1 -- 172.21.15.162:0/859568659 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b0150dd0 con 0x7f04b0065750 2020-11-28T21:46:55.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.839+0000 7f04aeffd700 1 --2- 172.21.15.162:0/859568659 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f04b0065750 0x7f04b0151270 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f04a4002f30 tx=0x7f04a4015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:55.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.839+0000 7f04acff9700 1 -- 172.21.15.162:0/859568659 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04a4011e60 con 0x7f04b0065750 2020-11-28T21:46:55.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.839+0000 7f04b48bd700 1 -- 172.21.15.162:0/859568659 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04b00b4b90 con 0x7f04b0065750 2020-11-28T21:46:55.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.839+0000 7f04acff9700 1 -- 172.21.15.162:0/859568659 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04a4009e20 con 0x7f04b0065750 2020-11-28T21:46:55.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.839+0000 7f04acff9700 1 -- 172.21.15.162:0/859568659 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04a401d640 con 0x7f04b0065750 2020-11-28T21:46:55.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.839+0000 7f04b48bd700 1 -- 172.21.15.162:0/859568659 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04b0144220 con 0x7f04b0065750 2020-11-28T21:46:55.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.843+0000 7f04b48bd700 1 -- 172.21.15.162:0/859568659 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04b00b4880 con 0x7f04b0065750 2020-11-28T21:46:55.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.843+0000 7f04acff9700 1 -- 172.21.15.162:0/859568659 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f04a4024030 con 0x7f04b0065750 2020-11-28T21:46:55.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.843+0000 7f04acff9700 1 --2- 172.21.15.162:0/859568659 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f04900554e0 0x7f04900579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:55.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.843+0000 7f04acff9700 1 -- 172.21.15.162:0/859568659 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f04a406f2f0 con 0x7f04b0065750 2020-11-28T21:46:55.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.843+0000 7f04af7fe700 1 --2- 172.21.15.162:0/859568659 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f04900554e0 0x7f04900579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:55.852 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.843+0000 7f04af7fe700 1 --2- 172.21.15.162:0/859568659 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f04900554e0 0x7f04900579a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f04a000b000 tx=0x7f04a0009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:55.853 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:55.851+0000 7f04acff9700 1 -- 172.21.15.162:0/859568659 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f04b00b4880 con 0x7f04b0065750 2020-11-28T21:46:56.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.039+0000 7f04b48bd700 1 -- 172.21.15.162:0/859568659 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command"]} v 0) v1 -- 0x7f04b00b4880 con 0x7f04b0065750 2020-11-28T21:46:56.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.719+0000 7f04acff9700 1 -- 172.21.15.162:0/859568659 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command"]}]=0 v43) v1 ==== 142+0+0 (secure 0 0 0) 0x7f04b00b4880 con 0x7f04b0065750 2020-11-28T21:46:56.724 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.723+0000 7f04b48bd700 1 -- 172.21.15.162:0/859568659 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f04900554e0 msgr2=0x7f04900579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:56.724 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.723+0000 7f04b48bd700 1 --2- 172.21.15.162:0/859568659 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f04900554e0 0x7f04900579a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f04a000b000 tx=0x7f04a0009040).stop 2020-11-28T21:46:56.725 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.723+0000 7f04b48bd700 1 -- 172.21.15.162:0/859568659 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f04b0065750 msgr2=0x7f04b0151270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:56.725 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.723+0000 7f04b48bd700 1 --2- 172.21.15.162:0/859568659 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f04b0065750 0x7f04b0151270 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f04a4002f30 tx=0x7f04a4015000).stop 2020-11-28T21:46:56.725 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.723+0000 7f04b48bd700 1 -- 172.21.15.162:0/859568659 shutdown_connections 2020-11-28T21:46:56.725 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.723+0000 7f04b48bd700 1 --2- 172.21.15.162:0/859568659 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f04900554e0 0x7f04900579a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:56.725 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.723+0000 7f04b48bd700 1 --2- 172.21.15.162:0/859568659 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f04b0064c70 0x7f04b00664f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:56.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.723+0000 7f04b48bd700 1 --2- 172.21.15.162:0/859568659 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f04b013f6c0 0x7f04b0151760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:56.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.723+0000 7f04b48bd700 1 --2- 172.21.15.162:0/859568659 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f04b0065750 0x7f04b0151270 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:56.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.723+0000 7f04b48bd700 1 -- 172.21.15.162:0/859568659 >> 172.21.15.162:0/859568659 conn(0x7f04b00b5da0 msgr2=0x7f04b013e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:56.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.723+0000 7f04b48bd700 1 -- 172.21.15.162:0/859568659 shutdown_connections 2020-11-28T21:46:56.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.723+0000 7f04b48bd700 1 -- 172.21.15.162:0/859568659 wait complete. 2020-11-28T21:46:56.741 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config set mon mon_allow_pool_delete True 2020-11-28T21:46:56.862 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.859+0000 7f9ae88d7700 1 Processor -- start 2020-11-28T21:46:56.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.859+0000 7f9ae88d7700 1 -- start start 2020-11-28T21:46:56.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.863+0000 7f9ae88d7700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9ae40605c0 0x7f9ae4140390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:56.864 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.863+0000 7f9ae88d7700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ae4143950 con 0x7f9ae40b9ce0 2020-11-28T21:46:56.864 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.863+0000 7f9ae88d7700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ae40ba0c0 con 0x7f9ae41433c0 2020-11-28T21:46:56.864 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.863+0000 7f9ae88d7700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae40b4340 con 0x7f9ae40605c0 2020-11-28T21:46:56.864 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.863+0000 7f9ae1d9b700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f9ae41433c0 0x7f9ae413fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:45092/0 (socket says 172.21.15.162:45092) 2020-11-28T21:46:56.865 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.863+0000 7f9ae1d9b700 1 -- 172.21.15.162:0/221844269 learned_addr learned my addr 172.21.15.162:0/221844269 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:56.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.863+0000 7f9ae2d9d700 1 --2- 172.21.15.162:0/221844269 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9ae40605c0 0x7f9ae4140390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:56.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.863+0000 7f9ae159a700 1 -- 172.21.15.162:0/221844269 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 874388431 0 0) 0x7f9ae40ba0c0 con 0x7f9ae41433c0 2020-11-28T21:46:56.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.863+0000 7f9ae159a700 1 -- 172.21.15.162:0/221844269 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ac4003230 con 0x7f9ae41433c0 2020-11-28T21:46:56.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.863+0000 7f9ae159a700 1 -- 172.21.15.162:0/221844269 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1123452060 0 0) 0x7f9ac4003230 con 0x7f9ae41433c0 2020-11-28T21:46:56.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.863+0000 7f9ae2d9d700 1 -- 172.21.15.162:0/221844269 >> v1:172.21.15.162:6789/0 conn(0x7f9ae41433c0 legacy=0x7f9ae413fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:56.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.863+0000 7f9ae2d9d700 1 -- 172.21.15.162:0/221844269 >> v1:172.21.15.118:6789/0 conn(0x7f9ae40b9ce0 legacy=0x7f9ae40b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:56.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.863+0000 7f9ae2d9d700 1 -- 172.21.15.162:0/221844269 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae4066c30 con 0x7f9ae40605c0 2020-11-28T21:46:56.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.863+0000 7f9ae2d9d700 1 --2- 172.21.15.162:0/221844269 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9ae40605c0 0x7f9ae4140390 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f9ad4001120 tx=0x7f9adc0091a0).ready entity=mon.2 client_cookie=d076e7df9b09cb6d server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:56.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.863+0000 7f9ae159a700 1 -- 172.21.15.162:0/221844269 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9acc0016d0 con 0x7f9ae40605c0 2020-11-28T21:46:56.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.863+0000 7f9ae159a700 1 -- 172.21.15.162:0/221844269 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9adc00b8b0 con 0x7f9ae40605c0 2020-11-28T21:46:56.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.863+0000 7f9ae88d7700 1 -- 172.21.15.162:0/221844269 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9ae40605c0 msgr2=0x7f9ae4140390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:56.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.863+0000 7f9ae159a700 1 -- 172.21.15.162:0/221844269 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9adc00ba20 con 0x7f9ae40605c0 2020-11-28T21:46:56.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.863+0000 7f9ae88d7700 1 --2- 172.21.15.162:0/221844269 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9ae40605c0 0x7f9ae4140390 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f9ad4001120 tx=0x7f9adc0091a0).stop 2020-11-28T21:46:56.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.863+0000 7f9ae88d7700 1 -- 172.21.15.162:0/221844269 shutdown_connections 2020-11-28T21:46:56.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.863+0000 7f9ae88d7700 1 --2- 172.21.15.162:0/221844269 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9ae40605c0 0x7f9ae4140390 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:56.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.863+0000 7f9ae88d7700 1 -- 172.21.15.162:0/221844269 >> 172.21.15.162:0/221844269 conn(0x7f9ae405c430 msgr2=0x7f9ae40b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:56.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.863+0000 7f9ae88d7700 1 -- 172.21.15.162:0/221844269 shutdown_connections 2020-11-28T21:46:56.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.863+0000 7f9ae88d7700 1 -- 172.21.15.162:0/221844269 wait complete. 2020-11-28T21:46:56.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.867+0000 7f9ae88d7700 1 Processor -- start 2020-11-28T21:46:56.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.867+0000 7f9ae88d7700 1 -- start start 2020-11-28T21:46:56.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.867+0000 7f9ae88d7700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9ae40605c0 0x7f9ae40b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:56.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.867+0000 7f9ae88d7700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ae40b9ce0 0x7f9ae4148bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:56.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.867+0000 7f9ae88d7700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9ae41433c0 0x7f9ae40af610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:56.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.867+0000 7f9ae88d7700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae40b1ae0 con 0x7f9ae41433c0 2020-11-28T21:46:56.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.867+0000 7f9ae88d7700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae40b1d70 con 0x7f9ae40b9ce0 2020-11-28T21:46:56.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.867+0000 7f9ae88d7700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae40b2010 con 0x7f9ae40605c0 2020-11-28T21:46:56.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.867+0000 7f9ae1d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ae40b9ce0 0x7f9ae4148bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:56.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.867+0000 7f9ae1d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ae40b9ce0 0x7f9ae4148bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38252/0 (socket says 172.21.15.162:38252) 2020-11-28T21:46:56.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.867+0000 7f9ae1d9b700 1 -- 172.21.15.162:0/288296907 learned_addr learned my addr 172.21.15.162:0/288296907 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:56.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.867+0000 7f9ae1d9b700 1 -- 172.21.15.162:0/288296907 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9ae40605c0 msgr2=0x7f9ae40b15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:56.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.867+0000 7f9ae1d9b700 1 --2- 172.21.15.162:0/288296907 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9ae40605c0 0x7f9ae40b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-28T21:46:56.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.867+0000 7f9ae1d9b700 1 -- 172.21.15.162:0/288296907 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9ae41433c0 msgr2=0x7f9ae40af610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:56.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.867+0000 7f9ae1d9b700 1 --2- 172.21.15.162:0/288296907 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9ae41433c0 0x7f9ae40af610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-28T21:46:56.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.867+0000 7f9ae1d9b700 1 -- 172.21.15.162:0/288296907 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae40aeed0 con 0x7f9ae40b9ce0 2020-11-28T21:46:56.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.867+0000 7f9ae1d9b700 1 --2- 172.21.15.162:0/288296907 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ae40b9ce0 0x7f9ae4148bc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f9acc001930 tx=0x7f9acc0011f0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:56.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.867+0000 7f9adb7fe700 1 -- 172.21.15.162:0/288296907 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9acc007170 con 0x7f9ae40b9ce0 2020-11-28T21:46:56.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.867+0000 7f9adb7fe700 1 -- 172.21.15.162:0/288296907 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9acc0072d0 con 0x7f9ae40b9ce0 2020-11-28T21:46:56.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.867+0000 7f9ae88d7700 1 -- 172.21.15.162:0/288296907 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ae40af170 con 0x7f9ae40b9ce0 2020-11-28T21:46:56.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.867+0000 7f9adb7fe700 1 -- 172.21.15.162:0/288296907 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9acc0048f0 con 0x7f9ae40b9ce0 2020-11-28T21:46:56.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.867+0000 7f9ae88d7700 1 -- 172.21.15.162:0/288296907 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae40afb00 con 0x7f9ae40b9ce0 2020-11-28T21:46:56.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.871+0000 7f9ae88d7700 1 -- 172.21.15.162:0/288296907 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ae4149ad0 con 0x7f9ae40b9ce0 2020-11-28T21:46:56.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.871+0000 7f9adb7fe700 1 -- 172.21.15.162:0/288296907 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9acc010070 con 0x7f9ae40b9ce0 2020-11-28T21:46:56.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.871+0000 7f9adb7fe700 1 --2- 172.21.15.162:0/288296907 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9ac80554e0 0x7f9ac80579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:56.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.871+0000 7f9adb7fe700 1 -- 172.21.15.162:0/288296907 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9acc06dac0 con 0x7f9ae40b9ce0 2020-11-28T21:46:56.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.871+0000 7f9ae259c700 1 --2- 172.21.15.162:0/288296907 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9ac80554e0 0x7f9ac80579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:56.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.871+0000 7f9ae259c700 1 --2- 172.21.15.162:0/288296907 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9ac80554e0 0x7f9ac80579a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f9ad4001160 tx=0x7f9ad4007730).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:56.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:56.875+0000 7f9adb7fe700 1 -- 172.21.15.162:0/288296907 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9ae4149ad0 con 0x7f9ae40b9ce0 2020-11-28T21:46:57.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.019+0000 7f9ae88d7700 1 -- 172.21.15.162:0/288296907 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7f9ae4149ad0 con 0x7f9ae40b9ce0 2020-11-28T21:46:57.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.027+0000 7f9adb7fe700 1 -- 172.21.15.162:0/288296907 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v2) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9ae4149ad0 con 0x7f9ae40b9ce0 2020-11-28T21:46:57.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.027+0000 7f9ae88d7700 1 -- 172.21.15.162:0/288296907 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9ac80554e0 msgr2=0x7f9ac80579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:57.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.027+0000 7f9ae88d7700 1 --2- 172.21.15.162:0/288296907 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9ac80554e0 0x7f9ac80579a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f9ad4001160 tx=0x7f9ad4007730).stop 2020-11-28T21:46:57.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.027+0000 7f9ae88d7700 1 -- 172.21.15.162:0/288296907 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ae40b9ce0 msgr2=0x7f9ae4148bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:57.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.027+0000 7f9ae88d7700 1 --2- 172.21.15.162:0/288296907 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ae40b9ce0 0x7f9ae4148bc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f9acc001930 tx=0x7f9acc0011f0).stop 2020-11-28T21:46:57.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.027+0000 7f9ae88d7700 1 -- 172.21.15.162:0/288296907 shutdown_connections 2020-11-28T21:46:57.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.027+0000 7f9ae88d7700 1 --2- 172.21.15.162:0/288296907 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9ac80554e0 0x7f9ac80579a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:57.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.027+0000 7f9ae88d7700 1 --2- 172.21.15.162:0/288296907 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9ae40605c0 0x7f9ae40b15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-28T21:46:57.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.027+0000 7f9ae88d7700 1 --2- 172.21.15.162:0/288296907 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9ae41433c0 0x7f9ae40af610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-28T21:46:57.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.027+0000 7f9ae88d7700 1 --2- 172.21.15.162:0/288296907 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ae40b9ce0 0x7f9ae4148bc0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:57.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.027+0000 7f9ae88d7700 1 -- 172.21.15.162:0/288296907 >> 172.21.15.162:0/288296907 conn(0x7f9ae405c430 msgr2=0x7f9ae409e220 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:57.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.027+0000 7f9ae88d7700 1 -- 172.21.15.162:0/288296907 shutdown_connections 2020-11-28T21:46:57.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.027+0000 7f9ae88d7700 1 -- 172.21.15.162:0/288296907 wait complete. 2020-11-28T21:46:57.048 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-28T21:46:57.049 INFO:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:46:57.068 INFO:teuthology.orchestra.run.smithi118:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:46:57.086 INFO:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:46:57.104 INFO:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:46:57.123 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-28T21:46:57.123 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:46:57.124 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:46:57.124 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:46:57.173 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2020-11-28T21:46:57.174 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-28T21:46:57.174 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:46:57.176 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:46:57.176 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:46:57.224 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2020-11-28T21:46:57.225 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-28T21:46:57.226 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:46:57.226 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:46:57.226 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:46:57.275 INFO:teuthology.orchestra.run.smithi192.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Transport endpoint is not connected 2020-11-28T21:46:57.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:46:57.277 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2020-11-28T21:46:57.277 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2020-11-28T21:46:57.278 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-28T21:46:57.278 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:46:57.367 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:46:57.368 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:46:57.369 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:46:57.415 INFO:teuthology.orchestra.run.smithi192.stdout:ext2/ext3 2020-11-28T21:46:57.416 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-11-28T21:46:57.417 INFO:teuthology.orchestra.run:waiting for 900 2020-11-28T21:46:57.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-28T21:46:57.418 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:46:57.470 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:46:57.470 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:46:57.471 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:46:57.479 INFO:teuthology.orchestra.run.smithi096.stdout:fuseblk 2020-11-28T21:46:57.480 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-11-28T21:46:57.480 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:46:57.481 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:46:57.481 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:46:57.530 INFO:teuthology.orchestra.run.smithi096.stdout:fuseblk 2020-11-28T21:46:57.531 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-11-28T21:46:57.532 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:46:57.533 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:46:57.534 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:46:57.581 INFO:teuthology.orchestra.run.smithi096.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': Transport endpoint is not connected 2020-11-28T21:46:57.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:46:57.583 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.1 2020-11-28T21:46:57.584 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi096.front.sepia.ceph.com... 2020-11-28T21:46:57.585 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-28T21:46:57.585 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:46:57.701 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:46:57.702 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:46:57.703 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:46:57.710 INFO:teuthology.orchestra.run.smithi096.stdout:ext2/ext3 2020-11-28T21:46:57.711 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-11-28T21:46:57.712 INFO:teuthology.orchestra.run:waiting for 900 2020-11-28T21:46:57.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-28T21:46:57.713 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:46:57.764 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2020-11-28T21:46:57.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.883+0000 7f56ba977700 1 Processor -- start 2020-11-28T21:46:57.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.883+0000 7f56ba977700 1 -- start start 2020-11-28T21:46:57.886 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.883+0000 7f56ba977700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f56b405fbd0 0x7f56b40b07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:57.887 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.883+0000 7f56ba977700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f56b4146dc0 0x7f56b40b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:57.887 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.883+0000 7f56ba977700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56b4147260 con 0x7f56b405bb10 2020-11-28T21:46:57.887 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.883+0000 7f56ba977700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56b4147500 con 0x7f56b405fbd0 2020-11-28T21:46:57.887 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.883+0000 7f56ba977700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56b41477a0 con 0x7f56b4146dc0 2020-11-28T21:46:57.888 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.883+0000 7f56ba176700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f56b4146dc0 0x7f56b40b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:57.888 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.883+0000 7f56b9975700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f56b405bb10 0x7f56b405f4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:45104/0 (socket says 172.21.15.162:45104) 2020-11-28T21:46:57.888 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.883+0000 7f56b9975700 1 -- 172.21.15.162:0/222180775 learned_addr learned my addr 172.21.15.162:0/222180775 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:57.888 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.883+0000 7f56b8973700 1 -- 172.21.15.162:0/222180775 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3936259866 0 0) 0x7f56b4147260 con 0x7f56b405bb10 2020-11-28T21:46:57.888 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.883+0000 7f56b8973700 1 -- 172.21.15.162:0/222180775 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56a0003230 con 0x7f56b405bb10 2020-11-28T21:46:57.889 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.883+0000 7f56b9174700 1 --2- 172.21.15.162:0/222180775 >> v2:172.21.15.118:3300/0 conn(0x7f56b405fbd0 0x7f56b40b07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:57.889 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.883+0000 7f56ba176700 1 -- 172.21.15.162:0/222180775 >> v2:172.21.15.118:3300/0 conn(0x7f56b405fbd0 msgr2=0x7f56b40b07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:57.889 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.883+0000 7f56ba176700 1 --2- 172.21.15.162:0/222180775 >> v2:172.21.15.118:3300/0 conn(0x7f56b405fbd0 0x7f56b40b07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:57.889 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.883+0000 7f56ba176700 1 -- 172.21.15.162:0/222180775 >> v1:172.21.15.162:6789/0 conn(0x7f56b405bb10 legacy=0x7f56b405f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:57.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.883+0000 7f56ba176700 1 -- 172.21.15.162:0/222180775 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b4147a40 con 0x7f56b4146dc0 2020-11-28T21:46:57.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.883+0000 7f56b9174700 1 --2- 172.21.15.162:0/222180775 >> v2:172.21.15.118:3300/0 conn(0x7f56b405fbd0 0x7f56b40b07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:46:57.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.883+0000 7f56ba176700 1 --2- 172.21.15.162:0/222180775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56b4146dc0 0x7f56b40b0ca0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f56ac002f30 tx=0x7f56ac006dc0).ready entity=mon.1 client_cookie=6954eeec68d1ca27 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:57.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.883+0000 7f56b8973700 1 -- 172.21.15.162:0/222180775 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56ac0097b0 con 0x7f56b4146dc0 2020-11-28T21:46:57.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.883+0000 7f56b8973700 1 -- 172.21.15.162:0/222180775 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56ac009910 con 0x7f56b4146dc0 2020-11-28T21:46:57.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.883+0000 7f56ba977700 1 -- 172.21.15.162:0/222180775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56b4146dc0 msgr2=0x7f56b40b0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:57.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.883+0000 7f56ba977700 1 --2- 172.21.15.162:0/222180775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56b4146dc0 0x7f56b40b0ca0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f56ac002f30 tx=0x7f56ac006dc0).stop 2020-11-28T21:46:57.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.883+0000 7f56ba977700 1 -- 172.21.15.162:0/222180775 shutdown_connections 2020-11-28T21:46:57.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.883+0000 7f56ba977700 1 --2- 172.21.15.162:0/222180775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56b4146dc0 0x7f56b40b0ca0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:57.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.883+0000 7f56ba977700 1 --2- 172.21.15.162:0/222180775 >> v2:172.21.15.118:3300/0 conn(0x7f56b405fbd0 0x7f56b40b07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:57.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.883+0000 7f56ba977700 1 -- 172.21.15.162:0/222180775 >> 172.21.15.162:0/222180775 conn(0x7f56b40b5da0 msgr2=0x7f56b40b6180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:57.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.883+0000 7f56ba977700 1 -- 172.21.15.162:0/222180775 shutdown_connections 2020-11-28T21:46:57.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.883+0000 7f56ba977700 1 -- 172.21.15.162:0/222180775 wait complete. 2020-11-28T21:46:57.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.887+0000 7f56ba977700 1 Processor -- start 2020-11-28T21:46:57.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.887+0000 7f56ba977700 1 -- start start 2020-11-28T21:46:57.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.887+0000 7f56ba977700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56b405bb10 0x7f56b40aee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:57.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.887+0000 7f56ba977700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f56b4146dc0 0x7f56b4143fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:57.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.887+0000 7f56ba977700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f56b41444b0 0x7f56b41121a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:57.896 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.887+0000 7f56ba977700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b4144960 con 0x7f56b4146dc0 2020-11-28T21:46:57.896 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.887+0000 7f56ba977700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b4143bb0 con 0x7f56b405bb10 2020-11-28T21:46:57.896 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.887+0000 7f56ba977700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56b4142630 con 0x7f56b41444b0 2020-11-28T21:46:57.896 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.887+0000 7f56b9975700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56b405bb10 0x7f56b40aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:57.897 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.887+0000 7f56b9975700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56b405bb10 0x7f56b40aee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38266/0 (socket says 172.21.15.162:38266) 2020-11-28T21:46:57.897 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.887+0000 7f56b9975700 1 -- 172.21.15.162:0/2265356915 learned_addr learned my addr 172.21.15.162:0/2265356915 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:57.897 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.887+0000 7f56ba176700 1 --2- 172.21.15.162:0/2265356915 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f56b41444b0 0x7f56b41121a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:57.897 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.887+0000 7f56b9174700 1 --2- 172.21.15.162:0/2265356915 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f56b4146dc0 0x7f56b4143fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:57.897 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.887+0000 7f56b9975700 1 -- 172.21.15.162:0/2265356915 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f56b41444b0 msgr2=0x7f56b41121a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:57.898 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.887+0000 7f56b9975700 1 --2- 172.21.15.162:0/2265356915 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f56b41444b0 0x7f56b41121a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:57.898 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.887+0000 7f56b9975700 1 -- 172.21.15.162:0/2265356915 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f56b4146dc0 msgr2=0x7f56b4143fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:57.898 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.887+0000 7f56b9975700 1 --2- 172.21.15.162:0/2265356915 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f56b4146dc0 0x7f56b4143fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:57.898 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.887+0000 7f56b9975700 1 -- 172.21.15.162:0/2265356915 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b41428d0 con 0x7f56b405bb10 2020-11-28T21:46:57.898 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.887+0000 7f56b9975700 1 --2- 172.21.15.162:0/2265356915 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56b405bb10 0x7f56b40aee10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f56a40010d0 tx=0x7f56a4007aa0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:57.899 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.887+0000 7f56aaffd700 1 -- 172.21.15.162:0/2265356915 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56a40072e0 con 0x7f56b405bb10 2020-11-28T21:46:57.899 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.887+0000 7f56aaffd700 1 -- 172.21.15.162:0/2265356915 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56a4007440 con 0x7f56b405bb10 2020-11-28T21:46:57.899 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.887+0000 7f56aaffd700 1 -- 172.21.15.162:0/2265356915 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56a40057d0 con 0x7f56b405bb10 2020-11-28T21:46:57.899 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.887+0000 7f56ba977700 1 -- 172.21.15.162:0/2265356915 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56b4142b70 con 0x7f56b405bb10 2020-11-28T21:46:57.899 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.887+0000 7f56ba977700 1 -- 172.21.15.162:0/2265356915 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56b4110e60 con 0x7f56b405bb10 2020-11-28T21:46:57.900 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.891+0000 7f56ba977700 1 -- 172.21.15.162:0/2265356915 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56b4143d20 con 0x7f56b405bb10 2020-11-28T21:46:57.900 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.891+0000 7f56aaffd700 1 -- 172.21.15.162:0/2265356915 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f56a4010070 con 0x7f56b405bb10 2020-11-28T21:46:57.900 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.891+0000 7f56aaffd700 1 --2- 172.21.15.162:0/2265356915 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5694055440 0x7f5694057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:57.900 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.891+0000 7f56b9174700 1 --2- 172.21.15.162:0/2265356915 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5694055440 0x7f5694057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:57.901 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.891+0000 7f56aaffd700 1 -- 172.21.15.162:0/2265356915 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f56a406d6f0 con 0x7f56b405bb10 2020-11-28T21:46:57.901 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.891+0000 7f56b9174700 1 --2- 172.21.15.162:0/2265356915 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5694055440 0x7f5694057900 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f56b000d000 tx=0x7f56b0008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:57.902 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:57.895+0000 7f56aaffd700 1 -- 172.21.15.162:0/2265356915 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f56b4143d20 con 0x7f56b405bb10 2020-11-28T21:46:58.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.035+0000 7f56ba977700 1 -- 172.21.15.162:0/2265356915 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f56b4143cf0 con 0x7f56b405bb10 2020-11-28T21:46:58.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.035+0000 7f56aaffd700 1 -- 172.21.15.162:0/2265356915 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v23) v1 ==== 81+0+33741 (secure 0 0 0) 0x7f56a40438e0 con 0x7f56b405bb10 2020-11-28T21:46:58.041 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:46:58.041 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2020-11-28T21:46:58.042 INFO:teuthology.orchestra.run.smithi162.stdout: "epoch": 23, 2020-11-28T21:46:58.042 INFO:teuthology.orchestra.run.smithi162.stdout: "fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", 2020-11-28T21:46:58.042 INFO:teuthology.orchestra.run.smithi162.stdout: "created": "2020-11-28T21:46:11.724372+0000", 2020-11-28T21:46:58.043 INFO:teuthology.orchestra.run.smithi162.stdout: "modified": "2020-11-28T21:46:30.124354+0000", 2020-11-28T21:46:58.043 INFO:teuthology.orchestra.run.smithi162.stdout: "last_up_change": "2020-11-28T21:46:19.591650+0000", 2020-11-28T21:46:58.044 INFO:teuthology.orchestra.run.smithi162.stdout: "last_in_change": "2020-11-28T21:46:19.591650+0000", 2020-11-28T21:46:58.044 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-11-28T21:46:58.044 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_num": 5799936, 2020-11-28T21:46:58.045 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_set": [ 2020-11-28T21:46:58.045 INFO:teuthology.orchestra.run.smithi162.stdout: "pglog_hardlimit", 2020-11-28T21:46:58.046 INFO:teuthology.orchestra.run.smithi162.stdout: "purged_snapdirs", 2020-11-28T21:46:58.046 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_deletes", 2020-11-28T21:46:58.046 INFO:teuthology.orchestra.run.smithi162.stdout: "sortbitwise" 2020-11-28T21:46:58.047 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:46:58.047 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_version": 4, 2020-11-28T21:46:58.048 INFO:teuthology.orchestra.run.smithi162.stdout: "full_ratio": 0.94999998807907104, 2020-11-28T21:46:58.048 INFO:teuthology.orchestra.run.smithi162.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-11-28T21:46:58.048 INFO:teuthology.orchestra.run.smithi162.stdout: "nearfull_ratio": 0.85000002384185791, 2020-11-28T21:46:58.049 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_snapshot": "", 2020-11-28T21:46:58.049 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_max": 4, 2020-11-28T21:46:58.050 INFO:teuthology.orchestra.run.smithi162.stdout: "max_osd": 8, 2020-11-28T21:46:58.050 INFO:teuthology.orchestra.run.smithi162.stdout: "require_min_compat_client": "jewel", 2020-11-28T21:46:58.050 INFO:teuthology.orchestra.run.smithi162.stdout: "min_compat_client": "jewel", 2020-11-28T21:46:58.051 INFO:teuthology.orchestra.run.smithi162.stdout: "require_osd_release": "octopus", 2020-11-28T21:46:58.051 INFO:teuthology.orchestra.run.smithi162.stdout: "pools": [ 2020-11-28T21:46:58.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.052 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 1, 2020-11-28T21:46:58.052 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "device_health_metrics", 2020-11-28T21:46:58.053 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:46:15.639551+0000", 2020-11-28T21:46:58.053 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:46:58.054 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:46:58.054 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:46:58.054 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:46:58.055 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:46:58.055 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:46:58.056 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:46:58.056 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:46:58.056 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 1, 2020-11-28T21:46:58.057 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 1, 2020-11-28T21:46:58.057 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 1, 2020-11-28T21:46:58.057 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 1, 2020-11-28T21:46:58.058 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 1, 2020-11-28T21:46:58.058 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:46:58.058 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:46:58.058 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:46:58.058 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:46:58.059 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:46:58.059 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:46:58.059 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:46:58.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.060 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "16", 2020-11-28T21:46:58.060 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:46:58.060 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:46:58.060 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:46:58.061 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:46:58.061 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:46:58.061 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:46:58.061 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:46:58.061 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:46:58.062 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:46:58.062 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:46:58.062 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:46:58.062 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:46:58.062 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:46:58.063 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:46:58.063 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:46:58.063 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:46:58.063 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:46:58.064 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:46:58.064 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:46:58.064 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:46:58.064 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:46:58.065 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:46:58.065 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:46:58.065 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:46:58.065 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:46:58.065 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:46:58.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.066 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:46:58.066 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:46:58.066 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:46:58.067 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:46:58.067 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:46:58.067 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:46:58.067 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:46:58.067 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:46:58.068 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:46:58.068 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:46:58.068 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:46:58.068 INFO:teuthology.orchestra.run.smithi162.stdout: "options": { 2020-11-28T21:46:58.068 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_min": 1 2020-11-28T21:46:58.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.069 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:46:58.069 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr_devicehealth": {} 2020-11-28T21:46:58.069 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:58.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.070 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 2, 2020-11-28T21:46:58.070 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "rbd", 2020-11-28T21:46:58.070 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:46:23.901366+0000", 2020-11-28T21:46:58.070 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:46:58.071 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:46:58.071 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:46:58.071 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:46:58.071 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:46:58.071 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:46:58.072 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:46:58.072 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:46:58.072 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2020-11-28T21:46:58.072 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2020-11-28T21:46:58.072 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2020-11-28T21:46:58.073 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2020-11-28T21:46:58.073 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2020-11-28T21:46:58.073 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:46:58.073 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:46:58.073 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:46:58.074 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:46:58.074 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:46:58.074 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:46:58.074 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:46:58.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.075 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "19", 2020-11-28T21:46:58.075 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:46:58.075 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:46:58.075 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:46:58.075 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:46:58.076 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:46:58.076 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:46:58.076 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:46:58.076 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:46:58.076 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:46:58.077 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:46:58.077 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:46:58.077 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:46:58.077 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:46:58.078 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:46:58.078 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:46:58.078 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:46:58.078 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:46:58.078 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:46:58.079 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:46:58.079 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:46:58.079 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:46:58.079 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:46:58.080 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:46:58.080 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:46:58.080 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:46:58.080 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:46:58.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.081 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:46:58.081 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:46:58.081 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:46:58.081 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:46:58.082 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:46:58.082 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:46:58.082 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:46:58.082 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:46:58.082 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:46:58.083 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:46:58.083 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:46:58.083 INFO:teuthology.orchestra.run.smithi162.stdout: "options": {}, 2020-11-28T21:46:58.083 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:46:58.083 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd": {} 2020-11-28T21:46:58.084 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:58.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.084 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 3, 2020-11-28T21:46:58.084 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "cephfs_metadata", 2020-11-28T21:46:58.085 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:46:27.155497+0000", 2020-11-28T21:46:58.085 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:46:58.085 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:46:58.085 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:46:58.086 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:46:58.086 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:46:58.086 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:46:58.086 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:46:58.087 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:46:58.087 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2020-11-28T21:46:58.088 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2020-11-28T21:46:58.088 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2020-11-28T21:46:58.088 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2020-11-28T21:46:58.088 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2020-11-28T21:46:58.089 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:46:58.089 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:46:58.089 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:46:58.089 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:46:58.089 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:46:58.090 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:46:58.090 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:46:58.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.090 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "22", 2020-11-28T21:46:58.090 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:46:58.091 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:46:58.091 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:46:58.091 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:46:58.091 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:46:58.092 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:46:58.092 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:46:58.092 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:46:58.092 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:46:58.093 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:46:58.093 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:46:58.093 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:46:58.094 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:46:58.094 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:46:58.095 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:46:58.095 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:46:58.095 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:46:58.096 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:46:58.096 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:46:58.097 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:46:58.097 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:46:58.097 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:46:58.098 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:46:58.098 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:46:58.099 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:46:58.099 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:46:58.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.099 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:46:58.100 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:46:58.100 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:46:58.100 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:46:58.100 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:46:58.100 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:46:58.101 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:46:58.101 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:46:58.101 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:46:58.101 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:46:58.101 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:46:58.102 INFO:teuthology.orchestra.run.smithi162.stdout: "options": { 2020-11-28T21:46:58.102 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_bias": 4, 2020-11-28T21:46:58.102 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_min": 16, 2020-11-28T21:46:58.102 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_priority": 5 2020-11-28T21:46:58.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.103 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:46:58.103 INFO:teuthology.orchestra.run.smithi162.stdout: "cephfs": { 2020-11-28T21:46:58.103 INFO:teuthology.orchestra.run.smithi162.stdout: "metadata": "cephfs" 2020-11-28T21:46:58.103 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:58.103 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:58.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.104 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 4, 2020-11-28T21:46:58.104 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "cephfs_data", 2020-11-28T21:46:58.104 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:46:28.137474+0000", 2020-11-28T21:46:58.105 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:46:58.105 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:46:58.105 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:46:58.105 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:46:58.105 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:46:58.106 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:46:58.106 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:46:58.106 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:46:58.106 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2020-11-28T21:46:58.106 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2020-11-28T21:46:58.107 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2020-11-28T21:46:58.107 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2020-11-28T21:46:58.107 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2020-11-28T21:46:58.107 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:46:58.107 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:46:58.108 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:46:58.108 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:46:58.108 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:46:58.108 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:46:58.108 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:46:58.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.109 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "23", 2020-11-28T21:46:58.109 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:46:58.109 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:46:58.109 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:46:58.110 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:46:58.110 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:46:58.110 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:46:58.110 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:46:58.110 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:46:58.111 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:46:58.111 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:46:58.111 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:46:58.111 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:46:58.111 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:46:58.112 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:46:58.112 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:46:58.112 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:46:58.112 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:46:58.112 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:46:58.113 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:46:58.113 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:46:58.113 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:46:58.113 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:46:58.113 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:46:58.114 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:46:58.114 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:46:58.114 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:46:58.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.114 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:46:58.115 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:46:58.115 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:46:58.115 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:46:58.115 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:46:58.115 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:46:58.116 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:46:58.116 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:46:58.116 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:46:58.116 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:46:58.116 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:46:58.117 INFO:teuthology.orchestra.run.smithi162.stdout: "options": {}, 2020-11-28T21:46:58.117 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:46:58.117 INFO:teuthology.orchestra.run.smithi162.stdout: "cephfs": { 2020-11-28T21:46:58.117 INFO:teuthology.orchestra.run.smithi162.stdout: "data": "cephfs" 2020-11-28T21:46:58.117 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:58.118 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:58.118 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:58.118 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:46:58.118 INFO:teuthology.orchestra.run.smithi162.stdout: "osds": [ 2020-11-28T21:46:58.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.119 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 0, 2020-11-28T21:46:58.119 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "99c7d9ae-adfa-4fd7-880a-23e6baf3281b", 2020-11-28T21:46:58.119 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:46:58.119 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:46:58.119 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:46:58.120 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:46:58.120 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:46:58.120 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:46:58.120 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:46:58.120 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 20, 2020-11-28T21:46:58.121 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:46:58.121 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:46:58.121 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:46:58.121 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:46:58.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.122 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:46:58.122 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6810", 2020-11-28T21:46:58.122 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:46:58.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.123 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:46:58.123 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6811", 2020-11-28T21:46:58.123 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:46:58.123 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:58.123 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:46:58.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.124 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:46:58.124 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:46:58.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.124 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:46:58.125 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6812", 2020-11-28T21:46:58.125 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:46:58.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.125 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:46:58.126 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6813", 2020-11-28T21:46:58.126 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:46:58.126 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:58.126 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:46:58.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.127 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:46:58.127 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:46:58.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.127 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:46:58.127 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6816", 2020-11-28T21:46:58.128 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:46:58.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.128 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:46:58.128 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6817", 2020-11-28T21:46:58.129 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:46:58.129 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:58.129 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:46:58.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.129 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:46:58.130 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:46:58.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.130 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:46:58.130 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6814", 2020-11-28T21:46:58.130 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:46:58.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.131 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:46:58.131 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6815", 2020-11-28T21:46:58.131 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:46:58.132 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:58.132 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:46:58.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.132 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6811/12313", 2020-11-28T21:46:58.132 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6813/12313", 2020-11-28T21:46:58.133 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6817/12313", 2020-11-28T21:46:58.133 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6815/12313", 2020-11-28T21:46:58.133 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:46:58.133 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:46:58.133 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:46:58.134 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:46:58.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.134 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 1, 2020-11-28T21:46:58.134 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "6ce6220a-00ef-4d13-abba-56f87636fb5f", 2020-11-28T21:46:58.135 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:46:58.135 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:46:58.135 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:46:58.135 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:46:58.135 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:46:58.136 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:46:58.136 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:46:58.136 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 14, 2020-11-28T21:46:58.136 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:46:58.136 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:46:58.137 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:46:58.137 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:46:58.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.137 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:46:58.137 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6802", 2020-11-28T21:46:58.138 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:46:58.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.138 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:46:58.138 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6803", 2020-11-28T21:46:58.139 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:46:58.139 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:58.139 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:46:58.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.139 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:46:58.140 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:46:58.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.140 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:46:58.140 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6804", 2020-11-28T21:46:58.140 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:46:58.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.141 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:46:58.141 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6805", 2020-11-28T21:46:58.141 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:46:58.142 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:58.142 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:46:58.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.142 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:46:58.142 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:46:58.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.143 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:46:58.143 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6808", 2020-11-28T21:46:58.143 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:46:58.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.144 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:46:58.144 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6809", 2020-11-28T21:46:58.144 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:46:58.144 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:58.145 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:46:58.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.145 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:46:58.145 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:46:58.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.146 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:46:58.146 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6806", 2020-11-28T21:46:58.146 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:46:58.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.147 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:46:58.147 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6807", 2020-11-28T21:46:58.147 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:46:58.147 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:58.147 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:46:58.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.148 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6803/12311", 2020-11-28T21:46:58.148 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6805/12311", 2020-11-28T21:46:58.148 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6809/12311", 2020-11-28T21:46:58.148 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6807/12311", 2020-11-28T21:46:58.148 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:46:58.149 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:46:58.149 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:46:58.149 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:46:58.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.150 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 2, 2020-11-28T21:46:58.150 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "db4e510d-27c1-47fc-a902-8b32e7dfd73e", 2020-11-28T21:46:58.150 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:46:58.150 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:46:58.150 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:46:58.151 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:46:58.151 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:46:58.151 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:46:58.151 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:46:58.151 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 21, 2020-11-28T21:46:58.152 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:46:58.152 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:46:58.152 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:46:58.152 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:46:58.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.153 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:46:58.153 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6818", 2020-11-28T21:46:58.153 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:46:58.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.154 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:46:58.154 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6819", 2020-11-28T21:46:58.154 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:46:58.154 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:58.154 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:46:58.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.155 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:46:58.155 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:46:58.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.155 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:46:58.156 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6820", 2020-11-28T21:46:58.156 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:46:58.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.156 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:46:58.157 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6821", 2020-11-28T21:46:58.157 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:46:58.157 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:58.157 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:46:58.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.158 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:46:58.158 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:46:58.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.158 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:46:58.158 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6824", 2020-11-28T21:46:58.159 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:46:58.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.159 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:46:58.159 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6825", 2020-11-28T21:46:58.160 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:46:58.160 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:58.160 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:46:58.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.160 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:46:58.161 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:46:58.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.161 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:46:58.161 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6822", 2020-11-28T21:46:58.161 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:46:58.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.162 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:46:58.162 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6823", 2020-11-28T21:46:58.162 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:46:58.163 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:58.163 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:46:58.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.163 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6819/12312", 2020-11-28T21:46:58.163 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6821/12312", 2020-11-28T21:46:58.164 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6825/12312", 2020-11-28T21:46:58.164 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6823/12312", 2020-11-28T21:46:58.164 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:46:58.164 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:46:58.164 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:46:58.165 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:46:58.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.165 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 3, 2020-11-28T21:46:58.165 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "4094cb87-f04a-45e1-8710-fb4b651949f6", 2020-11-28T21:46:58.166 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:46:58.166 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:46:58.166 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:46:58.166 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:46:58.166 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:46:58.167 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:46:58.167 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:46:58.167 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 21, 2020-11-28T21:46:58.167 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:46:58.167 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:46:58.168 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:46:58.168 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:46:58.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.168 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:46:58.168 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6826", 2020-11-28T21:46:58.169 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:46:58.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.169 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:46:58.169 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6827", 2020-11-28T21:46:58.170 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:46:58.170 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:58.170 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:46:58.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.170 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:46:58.171 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:46:58.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.171 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:46:58.171 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6828", 2020-11-28T21:46:58.171 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:46:58.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.172 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:46:58.172 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6829", 2020-11-28T21:46:58.172 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:46:58.173 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:58.173 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:46:58.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.173 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:46:58.173 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:46:58.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.174 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:46:58.174 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6832", 2020-11-28T21:46:58.174 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:46:58.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.175 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:46:58.175 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6833", 2020-11-28T21:46:58.175 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:46:58.175 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:58.176 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:46:58.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.176 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:46:58.176 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:46:58.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.177 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:46:58.177 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6830", 2020-11-28T21:46:58.177 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:46:58.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.178 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:46:58.178 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6831", 2020-11-28T21:46:58.178 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:46:58.178 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:58.178 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:46:58.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.179 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6827/12323", 2020-11-28T21:46:58.179 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6829/12323", 2020-11-28T21:46:58.179 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6833/12323", 2020-11-28T21:46:58.179 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6831/12323", 2020-11-28T21:46:58.180 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:46:58.180 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:46:58.180 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:46:58.180 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:46:58.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.181 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 4, 2020-11-28T21:46:58.181 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "b907bf53-c297-4756-847b-3be96680eef5", 2020-11-28T21:46:58.181 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:46:58.181 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:46:58.182 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:46:58.182 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:46:58.182 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:46:58.182 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:46:58.182 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:46:58.183 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 21, 2020-11-28T21:46:58.183 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:46:58.183 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:46:58.183 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:46:58.183 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:46:58.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.184 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:46:58.184 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6800", 2020-11-28T21:46:58.184 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:46:58.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.185 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:46:58.185 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6801", 2020-11-28T21:46:58.185 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:46:58.185 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:58.186 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:46:58.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.186 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:46:58.186 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:46:58.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.187 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:46:58.187 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6802", 2020-11-28T21:46:58.187 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:46:58.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.188 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:46:58.188 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6803", 2020-11-28T21:46:58.188 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:46:58.188 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:58.188 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:46:58.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.189 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:46:58.189 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:46:58.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.189 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:46:58.190 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6806", 2020-11-28T21:46:58.190 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:46:58.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.190 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:46:58.191 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6807", 2020-11-28T21:46:58.191 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:46:58.191 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:58.191 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:46:58.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.192 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:46:58.192 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:46:58.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.192 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:46:58.192 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6804", 2020-11-28T21:46:58.193 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:46:58.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.193 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:46:58.193 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6805", 2020-11-28T21:46:58.194 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:46:58.194 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:58.194 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:46:58.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.194 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6801/12527", 2020-11-28T21:46:58.195 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6803/12527", 2020-11-28T21:46:58.195 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6807/12527", 2020-11-28T21:46:58.195 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6805/12527", 2020-11-28T21:46:58.195 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:46:58.196 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:46:58.196 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:46:58.196 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:46:58.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.197 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 5, 2020-11-28T21:46:58.197 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "55ba305d-0505-4212-aebb-631e8554b440", 2020-11-28T21:46:58.197 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:46:58.197 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:46:58.197 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:46:58.198 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:46:58.198 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:46:58.198 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:46:58.198 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:46:58.198 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 21, 2020-11-28T21:46:58.199 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:46:58.199 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:46:58.199 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:46:58.199 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:46:58.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.200 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:46:58.200 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6808", 2020-11-28T21:46:58.200 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:46:58.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.201 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:46:58.201 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6809", 2020-11-28T21:46:58.201 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:46:58.201 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:58.201 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:46:58.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.202 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:46:58.202 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:46:58.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.202 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:46:58.203 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6810", 2020-11-28T21:46:58.203 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:46:58.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.203 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:46:58.204 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6811", 2020-11-28T21:46:58.204 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:46:58.204 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:58.204 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:46:58.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.204 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:46:58.205 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:46:58.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.205 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:46:58.205 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6814", 2020-11-28T21:46:58.205 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:46:58.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.206 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:46:58.206 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6815", 2020-11-28T21:46:58.207 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:46:58.207 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:58.207 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:46:58.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.207 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:46:58.208 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:46:58.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.208 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:46:58.208 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6812", 2020-11-28T21:46:58.208 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:46:58.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.209 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:46:58.209 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6813", 2020-11-28T21:46:58.209 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:46:58.210 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:58.210 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:46:58.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.210 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6809/12529", 2020-11-28T21:46:58.210 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6811/12529", 2020-11-28T21:46:58.211 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6815/12529", 2020-11-28T21:46:58.211 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6813/12529", 2020-11-28T21:46:58.211 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:46:58.211 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:46:58.211 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:46:58.212 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:46:58.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.212 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 6, 2020-11-28T21:46:58.212 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "9ada6051-493b-46b5-ba22-1b401440490f", 2020-11-28T21:46:58.213 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:46:58.213 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:46:58.213 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:46:58.213 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:46:58.213 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:46:58.214 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:46:58.214 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:46:58.214 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 21, 2020-11-28T21:46:58.214 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:46:58.214 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:46:58.215 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:46:58.215 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:46:58.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.215 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:46:58.215 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6824", 2020-11-28T21:46:58.216 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:46:58.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.216 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:46:58.216 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6825", 2020-11-28T21:46:58.217 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:46:58.217 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:58.217 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:46:58.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.217 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:46:58.218 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:46:58.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.218 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:46:58.218 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6826", 2020-11-28T21:46:58.218 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:46:58.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.219 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:46:58.219 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6827", 2020-11-28T21:46:58.219 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:46:58.220 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:58.220 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:46:58.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.220 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:46:58.220 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:46:58.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.221 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:46:58.221 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6830", 2020-11-28T21:46:58.221 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:46:58.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.222 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:46:58.222 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6831", 2020-11-28T21:46:58.222 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:46:58.222 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:58.223 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:46:58.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.223 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:46:58.223 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:46:58.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.224 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:46:58.224 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6828", 2020-11-28T21:46:58.224 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:46:58.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.225 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:46:58.225 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6829", 2020-11-28T21:46:58.225 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:46:58.225 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:58.225 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:46:58.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.226 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6825/12530", 2020-11-28T21:46:58.226 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6827/12530", 2020-11-28T21:46:58.226 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6831/12530", 2020-11-28T21:46:58.226 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6829/12530", 2020-11-28T21:46:58.227 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:46:58.227 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:46:58.227 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:46:58.227 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:46:58.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.228 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 7, 2020-11-28T21:46:58.228 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "c328b06d-a216-4cba-aad7-9ad081340613", 2020-11-28T21:46:58.228 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:46:58.228 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:46:58.229 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:46:58.229 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:46:58.229 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:46:58.229 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:46:58.229 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:46:58.230 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 17, 2020-11-28T21:46:58.230 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:46:58.230 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:46:58.230 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:46:58.230 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:46:58.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.231 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:46:58.231 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6816", 2020-11-28T21:46:58.231 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:46:58.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.232 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:46:58.232 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6817", 2020-11-28T21:46:58.232 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:46:58.232 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:58.233 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:46:58.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.233 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:46:58.233 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:46:58.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.234 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:46:58.234 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6818", 2020-11-28T21:46:58.234 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:46:58.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.235 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:46:58.235 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6819", 2020-11-28T21:46:58.235 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:46:58.235 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:58.235 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:46:58.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.236 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:46:58.236 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:46:58.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.236 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:46:58.237 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6822", 2020-11-28T21:46:58.237 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:46:58.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.237 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:46:58.238 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6823", 2020-11-28T21:46:58.238 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:46:58.238 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:58.238 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:46:58.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.239 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:46:58.239 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:46:58.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.239 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:46:58.239 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6820", 2020-11-28T21:46:58.240 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:46:58.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.240 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:46:58.240 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6821", 2020-11-28T21:46:58.241 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:46:58.241 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:58.241 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:46:58.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.241 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6817/12528", 2020-11-28T21:46:58.242 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6819/12528", 2020-11-28T21:46:58.242 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6823/12528", 2020-11-28T21:46:58.242 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6821/12528", 2020-11-28T21:46:58.242 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:46:58.242 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:46:58.243 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:46:58.243 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:46:58.243 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:58.243 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:46:58.243 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_xinfo": [ 2020-11-28T21:46:58.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.244 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 0, 2020-11-28T21:46:58.244 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:46:58.244 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:46:58.244 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:46:58.245 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:46:58.245 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:46:58.245 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.038052+0000", 2020-11-28T21:46:58.245 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:46:58.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.246 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 1, 2020-11-28T21:46:58.246 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:46:58.246 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:46:58.246 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:46:58.247 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:46:58.247 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:46:58.247 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:17.947136+0000", 2020-11-28T21:46:58.247 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:46:58.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.248 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 2, 2020-11-28T21:46:58.248 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:46:58.248 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:46:58.248 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:46:58.249 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:46:58.249 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:46:58.249 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.001186+0000", 2020-11-28T21:46:58.249 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:46:58.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.250 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 3, 2020-11-28T21:46:58.250 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:46:58.250 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:46:58.250 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:46:58.251 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:46:58.251 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:46:58.251 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.199284+0000", 2020-11-28T21:46:58.251 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:46:58.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.252 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 4, 2020-11-28T21:46:58.252 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:46:58.252 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:46:58.252 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:46:58.253 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:46:58.253 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:46:58.253 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.038823+0000", 2020-11-28T21:46:58.253 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:46:58.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 5, 2020-11-28T21:46:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:46:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:46:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:46:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:46:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:46:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.009322+0000", 2020-11-28T21:46:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:46:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 6, 2020-11-28T21:46:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:46:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:46:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:46:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:46:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:46:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.238927+0000", 2020-11-28T21:46:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:46:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:46:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 7, 2020-11-28T21:46:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:46:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:46:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:46:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:46:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:46:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.029839+0000", 2020-11-28T21:46:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:46:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:46:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_upmap": [], 2020-11-28T21:46:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_upmap_items": [], 2020-11-28T21:46:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_temp": [], 2020-11-28T21:46:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_temp": [], 2020-11-28T21:46:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: "blacklist": {}, 2020-11-28T21:46:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profiles": { 2020-11-28T21:46:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: "default": { 2020-11-28T21:46:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: "crush-failure-domain": "osd", 2020-11-28T21:46:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: "k": "2", 2020-11-28T21:46:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: "m": "1", 2020-11-28T21:46:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: "plugin": "jerasure", 2020-11-28T21:46:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: "ruleset-failure-domain": "osd", 2020-11-28T21:46:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: "technique": "reed_sol_van" 2020-11-28T21:46:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:46:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:46:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps_queue": [], 2020-11-28T21:46:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: "new_removed_snaps": [], 2020-11-28T21:46:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: "new_purged_snaps": [], 2020-11-28T21:46:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_node_flags": {}, 2020-11-28T21:46:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: "device_class_flags": {} 2020-11-28T21:46:58.264 INFO:teuthology.orchestra.run.smithi162.stdout:} 2020-11-28T21:46:58.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.039+0000 7f56ba977700 1 -- 172.21.15.162:0/2265356915 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5694055440 msgr2=0x7f5694057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:58.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.039+0000 7f56ba977700 1 --2- 172.21.15.162:0/2265356915 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5694055440 0x7f5694057900 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f56b000d000 tx=0x7f56b0008040).stop 2020-11-28T21:46:58.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.039+0000 7f56ba977700 1 -- 172.21.15.162:0/2265356915 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56b405bb10 msgr2=0x7f56b40aee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:58.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.039+0000 7f56ba977700 1 --2- 172.21.15.162:0/2265356915 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56b405bb10 0x7f56b40aee10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f56a40010d0 tx=0x7f56a4007aa0).stop 2020-11-28T21:46:58.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.039+0000 7f56ba977700 1 -- 172.21.15.162:0/2265356915 shutdown_connections 2020-11-28T21:46:58.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.039+0000 7f56ba977700 1 --2- 172.21.15.162:0/2265356915 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5694055440 0x7f5694057900 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:58.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.039+0000 7f56ba977700 1 --2- 172.21.15.162:0/2265356915 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f56b41444b0 0x7f56b41121a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:58.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.039+0000 7f56ba977700 1 --2- 172.21.15.162:0/2265356915 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56b405bb10 0x7f56b40aee10 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:58.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.039+0000 7f56ba977700 1 --2- 172.21.15.162:0/2265356915 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f56b4146dc0 0x7f56b4143fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:58.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.039+0000 7f56ba977700 1 -- 172.21.15.162:0/2265356915 >> 172.21.15.162:0/2265356915 conn(0x7f56b40b5da0 msgr2=0x7f56b4066480 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:58.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.039+0000 7f56ba977700 1 -- 172.21.15.162:0/2265356915 shutdown_connections 2020-11-28T21:46:58.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.039+0000 7f56ba977700 1 -- 172.21.15.162:0/2265356915 wait complete. 2020-11-28T21:46:58.269 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:46:58.389 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.387+0000 7ff3588de700 1 Processor -- start 2020-11-28T21:46:58.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.387+0000 7ff3588de700 1 -- start start 2020-11-28T21:46:58.401 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.387+0000 7ff3588de700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff3540b1990 0x7ff354140890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:58.401 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.387+0000 7ff3588de700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff3540b11b0 0x7ff354140d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:58.402 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.387+0000 7ff3588de700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff3540b1d70 0x7ff35414a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:58.402 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.387+0000 7ff3588de700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff354146e20 con 0x7ff3540b1990 2020-11-28T21:46:58.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.387+0000 7ff3588de700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3540b2870 con 0x7ff3540b1d70 2020-11-28T21:46:58.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.387+0000 7ff3588de700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3540b22e0 con 0x7ff3540b11b0 2020-11-28T21:46:58.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.387+0000 7ff352d9d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff3540b1d70 0x7ff35414a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:58.404 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.387+0000 7ff352d9d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff3540b1d70 0x7ff35414a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38278/0 (socket says 172.21.15.162:38278) 2020-11-28T21:46:58.404 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.387+0000 7ff352d9d700 1 -- 172.21.15.162:0/2666033412 learned_addr learned my addr 172.21.15.162:0/2666033412 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:58.405 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.387+0000 7ff351d9b700 1 --2- 172.21.15.162:0/2666033412 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff3540b11b0 0x7ff354140d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:58.405 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.387+0000 7ff35259c700 1 --2- 172.21.15.162:0/2666033412 >> v2:172.21.15.118:3300/0 conn(0x7ff3540b1990 0x7ff354140890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:58.405 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.387+0000 7ff352d9d700 1 -- 172.21.15.162:0/2666033412 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff3540b11b0 msgr2=0x7ff354140d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:58.406 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.387+0000 7ff352d9d700 1 --2- 172.21.15.162:0/2666033412 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff3540b11b0 0x7ff354140d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:58.406 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.387+0000 7ff352d9d700 1 -- 172.21.15.162:0/2666033412 >> v2:172.21.15.118:3300/0 conn(0x7ff3540b1990 msgr2=0x7ff354140890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:46:58.407 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.387+0000 7ff352d9d700 1 --2- 172.21.15.162:0/2666033412 >> v2:172.21.15.118:3300/0 conn(0x7ff3540b1990 0x7ff354140890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:58.407 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.387+0000 7ff352d9d700 1 -- 172.21.15.162:0/2666033412 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3540b0740 con 0x7ff3540b1d70 2020-11-28T21:46:58.407 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.387+0000 7ff351d9b700 1 --2- 172.21.15.162:0/2666033412 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff3540b11b0 0x7ff354140d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:46:58.408 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.387+0000 7ff352d9d700 1 --2- 172.21.15.162:0/2666033412 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3540b1d70 0x7ff35414a770 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7ff34c0032b0 tx=0x7ff34c009270).ready entity=mon.1 client_cookie=c9d626836d9fcc09 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:58.408 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.391+0000 7ff35159a700 1 -- 172.21.15.162:0/2666033412 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff34c009e60 con 0x7ff3540b1d70 2020-11-28T21:46:58.409 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.391+0000 7ff35159a700 1 -- 172.21.15.162:0/2666033412 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff34c00bb50 con 0x7ff3540b1d70 2020-11-28T21:46:58.409 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.391+0000 7ff35159a700 1 -- 172.21.15.162:0/2666033412 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff34c00bcc0 con 0x7ff3540b1d70 2020-11-28T21:46:58.409 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.391+0000 7ff3588de700 1 -- 172.21.15.162:0/2666033412 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3540b1d70 msgr2=0x7ff35414a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:58.410 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.391+0000 7ff3588de700 1 --2- 172.21.15.162:0/2666033412 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3540b1d70 0x7ff35414a770 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7ff34c0032b0 tx=0x7ff34c009270).stop 2020-11-28T21:46:58.410 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.391+0000 7ff3588de700 1 -- 172.21.15.162:0/2666033412 shutdown_connections 2020-11-28T21:46:58.411 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.391+0000 7ff3588de700 1 --2- 172.21.15.162:0/2666033412 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3540b1d70 0x7ff35414a770 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:58.411 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.391+0000 7ff3588de700 1 --2- 172.21.15.162:0/2666033412 >> v2:172.21.15.118:3300/0 conn(0x7ff3540b1990 0x7ff354140890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:58.411 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.391+0000 7ff3588de700 1 --2- 172.21.15.162:0/2666033412 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff3540b11b0 0x7ff354140d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:58.412 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.391+0000 7ff3588de700 1 -- 172.21.15.162:0/2666033412 >> 172.21.15.162:0/2666033412 conn(0x7ff35405b970 msgr2=0x7ff35405bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:58.412 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.391+0000 7ff3588de700 1 -- 172.21.15.162:0/2666033412 shutdown_connections 2020-11-28T21:46:58.413 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.391+0000 7ff3588de700 1 -- 172.21.15.162:0/2666033412 wait complete. 2020-11-28T21:46:58.413 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.391+0000 7ff3588de700 1 Processor -- start 2020-11-28T21:46:58.414 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.391+0000 7ff3588de700 1 -- start start 2020-11-28T21:46:58.414 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.391+0000 7ff3588de700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff3540b11b0 0x7ff354066d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:58.414 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.391+0000 7ff3588de700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3540b1990 0x7ff354140a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:58.415 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.391+0000 7ff3588de700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff3540b1d70 0x7ff354140ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:58.415 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.391+0000 7ff3588de700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3541413e0 con 0x7ff3540b11b0 2020-11-28T21:46:58.416 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.391+0000 7ff3588de700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff354143690 con 0x7ff3540b1990 2020-11-28T21:46:58.416 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.391+0000 7ff3588de700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff354159cc0 con 0x7ff3540b1d70 2020-11-28T21:46:58.417 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.391+0000 7ff351d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3540b1990 0x7ff354140a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:58.417 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.391+0000 7ff351d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3540b1990 0x7ff354140a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38280/0 (socket says 172.21.15.162:38280) 2020-11-28T21:46:58.417 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.391+0000 7ff351d9b700 1 -- 172.21.15.162:0/1505236888 learned_addr learned my addr 172.21.15.162:0/1505236888 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:58.418 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.391+0000 7ff352d9d700 1 --2- 172.21.15.162:0/1505236888 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff3540b1d70 0x7ff354140ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:58.426 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.391+0000 7ff35259c700 1 --2- 172.21.15.162:0/1505236888 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff3540b11b0 0x7ff354066d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:58.448 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.391+0000 7ff351d9b700 1 -- 172.21.15.162:0/1505236888 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff3540b1d70 msgr2=0x7ff354140ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:58.448 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.391+0000 7ff351d9b700 1 --2- 172.21.15.162:0/1505236888 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff3540b1d70 0x7ff354140ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:58.448 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.391+0000 7ff351d9b700 1 -- 172.21.15.162:0/1505236888 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff3540b11b0 msgr2=0x7ff354066d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:58.449 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.391+0000 7ff351d9b700 1 --2- 172.21.15.162:0/1505236888 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff3540b11b0 0x7ff354066d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:58.449 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.391+0000 7ff351d9b700 1 -- 172.21.15.162:0/1505236888 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3541438c0 con 0x7ff3540b1990 2020-11-28T21:46:58.449 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.395+0000 7ff351d9b700 1 --2- 172.21.15.162:0/1505236888 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3540b1990 0x7ff354140a00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7ff354141520 tx=0x7ff344006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:58.450 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.395+0000 7ff34b7fe700 1 -- 172.21.15.162:0/1505236888 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff34400cda0 con 0x7ff3540b1990 2020-11-28T21:46:58.450 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.395+0000 7ff34b7fe700 1 -- 172.21.15.162:0/1505236888 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff34400ee80 con 0x7ff3540b1990 2020-11-28T21:46:58.451 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.395+0000 7ff3588de700 1 -- 172.21.15.162:0/1505236888 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff354143b60 con 0x7ff3540b1990 2020-11-28T21:46:58.451 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.395+0000 7ff34b7fe700 1 -- 172.21.15.162:0/1505236888 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3440125e0 con 0x7ff3540b1990 2020-11-28T21:46:58.451 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.395+0000 7ff3588de700 1 -- 172.21.15.162:0/1505236888 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff354144000 con 0x7ff3540b1990 2020-11-28T21:46:58.452 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.395+0000 7ff3588de700 1 -- 172.21.15.162:0/1505236888 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff354143cf0 con 0x7ff3540b1990 2020-11-28T21:46:58.452 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.395+0000 7ff34b7fe700 1 -- 172.21.15.162:0/1505236888 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff344012740 con 0x7ff3540b1990 2020-11-28T21:46:58.452 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.395+0000 7ff34b7fe700 1 --2- 172.21.15.162:0/1505236888 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff338055330 0x7ff3380577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:58.452 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.395+0000 7ff34b7fe700 1 -- 172.21.15.162:0/1505236888 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff34406e6a0 con 0x7ff3540b1990 2020-11-28T21:46:58.452 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.395+0000 7ff35259c700 1 --2- 172.21.15.162:0/1505236888 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff338055330 0x7ff3380577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:58.453 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.399+0000 7ff35259c700 1 --2- 172.21.15.162:0/1505236888 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff338055330 0x7ff3380577f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7ff33c00b000 tx=0x7ff33c006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:58.453 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.407+0000 7ff34b7fe700 1 -- 172.21.15.162:0/1505236888 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff3440449a0 con 0x7ff3540b1990 2020-11-28T21:46:58.586 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.583+0000 7ff3588de700 1 -- 172.21.15.162:0/1505236888 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff354057050 con 0x7ff3540b1990 2020-11-28T21:46:58.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.583+0000 7ff34b7fe700 1 -- 172.21.15.162:0/1505236888 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2495 (secure 0 0 0) 0x7ff344047fc0 con 0x7ff3540b1990 2020-11-28T21:46:58.587 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:46:58.588 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/1297721315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1297721315},{"type":"v1","addr":"172.21.15.118:6833","nonce":1297721315}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3},{"gid":4264,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2415281381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2415281381},{"type":"v1","addr":"172.21.15.162:6837","nonce":2415281381}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:46:29.120502+0000","modified":"2020-11-28T21:46:30.123428+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.162:6835/3696705934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3696705934},{"type":"v1","addr":"172.21.15.162:6835","nonce":3696705934}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-28T21:46:58.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.587+0000 7ff3588de700 1 -- 172.21.15.162:0/1505236888 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff338055330 msgr2=0x7ff3380577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:58.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.587+0000 7ff3588de700 1 --2- 172.21.15.162:0/1505236888 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff338055330 0x7ff3380577f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7ff33c00b000 tx=0x7ff33c006dc0).stop 2020-11-28T21:46:58.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.587+0000 7ff3588de700 1 -- 172.21.15.162:0/1505236888 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3540b1990 msgr2=0x7ff354140a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:58.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.587+0000 7ff3588de700 1 --2- 172.21.15.162:0/1505236888 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3540b1990 0x7ff354140a00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7ff354141520 tx=0x7ff344006dc0).stop 2020-11-28T21:46:58.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.587+0000 7ff3588de700 1 -- 172.21.15.162:0/1505236888 shutdown_connections 2020-11-28T21:46:58.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.587+0000 7ff3588de700 1 --2- 172.21.15.162:0/1505236888 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff338055330 0x7ff3380577f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:58.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.587+0000 7ff3588de700 1 --2- 172.21.15.162:0/1505236888 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff3540b1d70 0x7ff354140ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:58.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.587+0000 7ff3588de700 1 --2- 172.21.15.162:0/1505236888 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff3540b11b0 0x7ff354066d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:58.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.587+0000 7ff3588de700 1 --2- 172.21.15.162:0/1505236888 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3540b1990 0x7ff354140a00 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:58.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.587+0000 7ff3588de700 1 -- 172.21.15.162:0/1505236888 >> 172.21.15.162:0/1505236888 conn(0x7ff35405b970 msgr2=0x7ff3541481e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:58.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.587+0000 7ff3588de700 1 -- 172.21.15.162:0/1505236888 shutdown_connections 2020-11-28T21:46:58.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.587+0000 7ff3588de700 1 -- 172.21.15.162:0/1505236888 wait complete. 2020-11-28T21:46:58.594 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 5 2020-11-28T21:46:58.600 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2020-11-28T21:46:58.710 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.707+0000 7feac7fff700 1 Processor -- start 2020-11-28T21:46:58.711 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.707+0000 7feac7fff700 1 -- start start 2020-11-28T21:46:58.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.707+0000 7feac7fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7feac80b2290 0x7feac80ae4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:58.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.707+0000 7feac7fff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feac80b2670 con 0x7feac80a8550 2020-11-28T21:46:58.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.707+0000 7feac7fff700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feac80afa00 con 0x7feac80af480 2020-11-28T21:46:58.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.707+0000 7feac7fff700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feac80afca0 con 0x7feac80b2290 2020-11-28T21:46:58.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.707+0000 7feac6ffd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7feac80b2290 0x7feac80ae4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:58.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.707+0000 7feac77fe700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7feac80af480 0x7feac81417a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:45136/0 (socket says 172.21.15.162:45136) 2020-11-28T21:46:58.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.707+0000 7feac77fe700 1 -- 172.21.15.162:0/2527891609 learned_addr learned my addr 172.21.15.162:0/2527891609 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:58.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.711+0000 7feac6ffd700 1 -- 172.21.15.162:0/2527891609 >> v1:172.21.15.162:6789/0 conn(0x7feac80af480 legacy=0x7feac81417a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:58.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.711+0000 7feac6ffd700 1 -- 172.21.15.162:0/2527891609 >> v1:172.21.15.118:6789/0 conn(0x7feac80a8550 legacy=0x7feac80ae9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:58.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.711+0000 7feac67fc700 1 -- 172.21.15.162:0/2527891609 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2107588096 0 0) 0x7feac80b2670 con 0x7feac80a8550 2020-11-28T21:46:58.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.711+0000 7feac6ffd700 1 -- 172.21.15.162:0/2527891609 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac80acb80 con 0x7feac80b2290 2020-11-28T21:46:58.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.711+0000 7feac6ffd700 1 --2- 172.21.15.162:0/2527891609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feac80b2290 0x7feac80ae4c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7feac0001450 tx=0x7feab0006dc0).ready entity=mon.1 client_cookie=b3121ea388ed4aa2 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:58.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.711+0000 7feac67fc700 1 -- 172.21.15.162:0/2527891609 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feab00096f0 con 0x7feac80b2290 2020-11-28T21:46:58.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.711+0000 7feac67fc700 1 -- 172.21.15.162:0/2527891609 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab0009850 con 0x7feac80b2290 2020-11-28T21:46:58.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.711+0000 7feac67fc700 1 -- 172.21.15.162:0/2527891609 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feab0009bb0 con 0x7feac80b2290 2020-11-28T21:46:58.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.711+0000 7feac7fff700 1 -- 172.21.15.162:0/2527891609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feac80b2290 msgr2=0x7feac80ae4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:58.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.711+0000 7feac7fff700 1 --2- 172.21.15.162:0/2527891609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feac80b2290 0x7feac80ae4c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7feac0001450 tx=0x7feab0006dc0).stop 2020-11-28T21:46:58.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.711+0000 7feac7fff700 1 -- 172.21.15.162:0/2527891609 shutdown_connections 2020-11-28T21:46:58.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.711+0000 7feac7fff700 1 --2- 172.21.15.162:0/2527891609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feac80b2290 0x7feac80ae4c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:58.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.711+0000 7feac7fff700 1 -- 172.21.15.162:0/2527891609 >> 172.21.15.162:0/2527891609 conn(0x7feac8061320 msgr2=0x7feac8062e50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:58.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.711+0000 7feac7fff700 1 -- 172.21.15.162:0/2527891609 shutdown_connections 2020-11-28T21:46:58.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.711+0000 7feac7fff700 1 -- 172.21.15.162:0/2527891609 wait complete. 2020-11-28T21:46:58.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.715+0000 7feac7fff700 1 Processor -- start 2020-11-28T21:46:58.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.715+0000 7feac7fff700 1 -- start start 2020-11-28T21:46:58.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.715+0000 7feac7fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feac80a8550 0x7feac80664c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:58.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.715+0000 7feac7fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feac80af480 0x7feac813e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:58.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.715+0000 7feac7fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7feac80b2290 0x7feac8065b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:58.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.715+0000 7feac7fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac8064cf0 con 0x7feac80a8550 2020-11-28T21:46:58.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.715+0000 7feac7fff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac813f630 con 0x7feac80af480 2020-11-28T21:46:58.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.715+0000 7feac7fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac813fb10 con 0x7feac80b2290 2020-11-28T21:46:58.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.715+0000 7feabffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feac80af480 0x7feac813e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:58.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.715+0000 7feabffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feac80af480 0x7feac813e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38296/0 (socket says 172.21.15.162:38296) 2020-11-28T21:46:58.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.715+0000 7feabffff700 1 -- 172.21.15.162:0/2488791870 learned_addr learned my addr 172.21.15.162:0/2488791870 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:58.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.715+0000 7feac6ffd700 1 --2- 172.21.15.162:0/2488791870 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feac80a8550 0x7feac80664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:58.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.715+0000 7feac77fe700 1 --2- 172.21.15.162:0/2488791870 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7feac80b2290 0x7feac8065b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:58.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.715+0000 7feabffff700 1 -- 172.21.15.162:0/2488791870 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7feac80b2290 msgr2=0x7feac8065b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:58.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.715+0000 7feabffff700 1 --2- 172.21.15.162:0/2488791870 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7feac80b2290 0x7feac8065b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:58.723 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.715+0000 7feabffff700 1 -- 172.21.15.162:0/2488791870 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feac80a8550 msgr2=0x7feac80664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:58.723 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.715+0000 7feabffff700 1 --2- 172.21.15.162:0/2488791870 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feac80a8550 0x7feac80664c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:58.723 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.715+0000 7feabffff700 1 -- 172.21.15.162:0/2488791870 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac813f230 con 0x7feac80af480 2020-11-28T21:46:58.723 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.715+0000 7feac6ffd700 1 --2- 172.21.15.162:0/2488791870 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feac80a8550 0x7feac80664c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:46:58.724 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.715+0000 7feac77fe700 1 --2- 172.21.15.162:0/2488791870 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7feac80b2290 0x7feac8065b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:46:58.724 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.715+0000 7feabffff700 1 --2- 172.21.15.162:0/2488791870 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feac80af480 0x7feac813e810 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7feab8001190 tx=0x7feab8007b20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:58.724 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.715+0000 7feac4ff9700 1 -- 172.21.15.162:0/2488791870 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feab8010510 con 0x7feac80af480 2020-11-28T21:46:58.724 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.715+0000 7feac4ff9700 1 -- 172.21.15.162:0/2488791870 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab8010670 con 0x7feac80af480 2020-11-28T21:46:58.724 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.715+0000 7feac7fff700 1 -- 172.21.15.162:0/2488791870 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feac813ed00 con 0x7feac80af480 2020-11-28T21:46:58.725 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.715+0000 7feac4ff9700 1 -- 172.21.15.162:0/2488791870 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feab80107e0 con 0x7feac80af480 2020-11-28T21:46:58.725 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.715+0000 7feac7fff700 1 -- 172.21.15.162:0/2488791870 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac813ff10 con 0x7feac80af480 2020-11-28T21:46:58.725 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.719+0000 7feac7fff700 1 -- 172.21.15.162:0/2488791870 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feac813f090 con 0x7feac80af480 2020-11-28T21:46:58.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.719+0000 7feac4ff9700 1 -- 172.21.15.162:0/2488791870 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7feab8010070 con 0x7feac80af480 2020-11-28T21:46:58.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.719+0000 7feac4ff9700 1 --2- 172.21.15.162:0/2488791870 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7feaac0554e0 0x7feaac0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:58.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.719+0000 7feac4ff9700 1 -- 172.21.15.162:0/2488791870 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7feab806d9e0 con 0x7feac80af480 2020-11-28T21:46:58.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.719+0000 7feac6ffd700 1 --2- 172.21.15.162:0/2488791870 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7feaac0554e0 0x7feaac0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:58.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.719+0000 7feac6ffd700 1 --2- 172.21.15.162:0/2488791870 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7feaac0554e0 0x7feaac0579a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7feab0019000 tx=0x7feab0015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:58.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.723+0000 7feac4ff9700 1 -- 172.21.15.162:0/2488791870 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7feac813f090 con 0x7feac80af480 2020-11-28T21:46:58.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:58.911+0000 7feac7fff700 1 -- 172.21.15.162:0/2488791870 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7feac813f090 con 0x7feac80af480 2020-11-28T21:46:59.722 INFO:tasks.ceph.mon.b.smithi118.stderr:2020-11-28T21:46:59.718+0000 7ff4e1520700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-11-28T21:46:59.736 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.735+0000 7feac4ff9700 1 -- 172.21.15.162:0/2488791870 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v6) v1 ==== 161+0+0 (secure 0 0 0) 0x7feac813f090 con 0x7feac80af480 2020-11-28T21:46:59.739 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.735+0000 7feac7fff700 1 -- 172.21.15.162:0/2488791870 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7feaac0554e0 msgr2=0x7feaac0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:59.739 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.735+0000 7feac7fff700 1 --2- 172.21.15.162:0/2488791870 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7feaac0554e0 0x7feaac0579a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7feab0019000 tx=0x7feab0015000).stop 2020-11-28T21:46:59.739 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.735+0000 7feac7fff700 1 -- 172.21.15.162:0/2488791870 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feac80af480 msgr2=0x7feac813e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:59.739 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.735+0000 7feac7fff700 1 --2- 172.21.15.162:0/2488791870 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feac80af480 0x7feac813e810 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7feab8001190 tx=0x7feab8007b20).stop 2020-11-28T21:46:59.740 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.735+0000 7feac7fff700 1 -- 172.21.15.162:0/2488791870 shutdown_connections 2020-11-28T21:46:59.740 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.735+0000 7feac7fff700 1 --2- 172.21.15.162:0/2488791870 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7feaac0554e0 0x7feaac0579a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:59.740 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.735+0000 7feac7fff700 1 --2- 172.21.15.162:0/2488791870 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7feac80b2290 0x7feac8065b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:59.740 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.735+0000 7feac7fff700 1 --2- 172.21.15.162:0/2488791870 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feac80a8550 0x7feac80664c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:59.741 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.735+0000 7feac7fff700 1 --2- 172.21.15.162:0/2488791870 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feac80af480 0x7feac813e810 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:59.741 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.735+0000 7feac7fff700 1 -- 172.21.15.162:0/2488791870 >> 172.21.15.162:0/2488791870 conn(0x7feac8061320 msgr2=0x7feac8141eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:59.741 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.735+0000 7feac7fff700 1 -- 172.21.15.162:0/2488791870 shutdown_connections 2020-11-28T21:46:59.741 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.735+0000 7feac7fff700 1 -- 172.21.15.162:0/2488791870 wait complete. 2020-11-28T21:46:59.742 INFO:teuthology.orchestra.run.smithi162.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-11-28T21:46:59.754 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:46:59.763 INFO:tasks.ceph.mds.a.smithi162.stdout:starting mds.a at 2020-11-28T21:46:59.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.859+0000 7f19dffff700 1 Processor -- start 2020-11-28T21:46:59.864 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.859+0000 7f19dffff700 1 -- start start 2020-11-28T21:46:59.865 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.863+0000 7f19dffff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f19e00b4720 0x7f19e00b0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:59.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.863+0000 7f19dffff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19e00b1c50 con 0x7f19e00b0d20 2020-11-28T21:46:59.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.863+0000 7f19dffff700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19e0060a60 con 0x7f19e00b1870 2020-11-28T21:46:59.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.863+0000 7f19dffff700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19e00615a0 con 0x7f19e00b4720 2020-11-28T21:46:59.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.863+0000 7f19deffd700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f19e00b1870 0x7f19e0142f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:45158/0 (socket says 172.21.15.162:45158) 2020-11-28T21:46:59.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.863+0000 7f19deffd700 1 -- 172.21.15.162:0/2212133660 learned_addr learned my addr 172.21.15.162:0/2212133660 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:46:59.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.863+0000 7f19d7fff700 1 --2- 172.21.15.162:0/2212133660 >> v2:172.21.15.118:3300/0 conn(0x7f19e00b4720 0x7f19e00b0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:59.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.863+0000 7f19de7fc700 1 -- 172.21.15.162:0/2212133660 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 844215230 0 0) 0x7f19e0060a60 con 0x7f19e00b1870 2020-11-28T21:46:59.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.863+0000 7f19de7fc700 1 -- 172.21.15.162:0/2212133660 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19c0003230 con 0x7f19e00b1870 2020-11-28T21:46:59.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.863+0000 7f19d7fff700 1 -- 172.21.15.162:0/2212133660 >> v1:172.21.15.162:6789/0 conn(0x7f19e00b1870 legacy=0x7f19e0142f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:59.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.863+0000 7f19de7fc700 1 -- 172.21.15.162:0/2212133660 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3127923140 0 0) 0x7f19c0003230 con 0x7f19e00b1870 2020-11-28T21:46:59.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.863+0000 7f19d7fff700 1 -- 172.21.15.162:0/2212133660 >> v1:172.21.15.118:6789/0 conn(0x7f19e00b0d20 legacy=0x7f19e014cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:59.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.863+0000 7f19d7fff700 1 -- 172.21.15.162:0/2212133660 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e0143660 con 0x7f19e00b4720 2020-11-28T21:46:59.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.863+0000 7f19d7fff700 1 --2- 172.21.15.162:0/2212133660 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f19e00b4720 0x7f19e00b0830 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f19d8001450 tx=0x7f19c8006dc0).ready entity=mon.0 client_cookie=740be27520531419 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:59.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.863+0000 7f19de7fc700 1 -- 172.21.15.162:0/2212133660 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19d00016f0 con 0x7f19e00b4720 2020-11-28T21:46:59.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.863+0000 7f19de7fc700 1 -- 172.21.15.162:0/2212133660 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19c8009620 con 0x7f19e00b4720 2020-11-28T21:46:59.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.863+0000 7f19dffff700 1 -- 172.21.15.162:0/2212133660 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f19e00b4720 msgr2=0x7f19e00b0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:59.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.863+0000 7f19de7fc700 1 -- 172.21.15.162:0/2212133660 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19c8009790 con 0x7f19e00b4720 2020-11-28T21:46:59.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.863+0000 7f19dffff700 1 --2- 172.21.15.162:0/2212133660 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f19e00b4720 0x7f19e00b0830 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f19d8001450 tx=0x7f19c8006dc0).stop 2020-11-28T21:46:59.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.867+0000 7f19dffff700 1 -- 172.21.15.162:0/2212133660 shutdown_connections 2020-11-28T21:46:59.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.867+0000 7f19dffff700 1 --2- 172.21.15.162:0/2212133660 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f19e00b4720 0x7f19e00b0830 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:59.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.867+0000 7f19dffff700 1 -- 172.21.15.162:0/2212133660 >> 172.21.15.162:0/2212133660 conn(0x7f19e00b54f0 msgr2=0x7f19e00b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:46:59.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.867+0000 7f19dffff700 1 -- 172.21.15.162:0/2212133660 shutdown_connections 2020-11-28T21:46:59.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.867+0000 7f19dffff700 1 -- 172.21.15.162:0/2212133660 wait complete. 2020-11-28T21:46:59.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.867+0000 7f19dffff700 1 Processor -- start 2020-11-28T21:46:59.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.867+0000 7f19dffff700 1 -- start start 2020-11-28T21:46:59.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.867+0000 7f19dffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f19e00b0d20 0x7f19e009b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:59.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.867+0000 7f19dffff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f19e00b1870 0x7f19e013e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:59.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.867+0000 7f19dffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f19e00b4720 0x7f19e0098590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:59.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.867+0000 7f19dffff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e009a070 con 0x7f19e00b0d20 2020-11-28T21:46:59.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.867+0000 7f19dffff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e013cd60 con 0x7f19e00b4720 2020-11-28T21:46:59.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.867+0000 7f19dffff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e00978c0 con 0x7f19e00b1870 2020-11-28T21:46:59.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.867+0000 7f19df7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f19e00b4720 0x7f19e0098590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:59.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.867+0000 7f19df7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f19e00b4720 0x7f19e0098590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38324/0 (socket says 172.21.15.162:38324) 2020-11-28T21:46:59.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.867+0000 7f19df7fe700 1 -- 172.21.15.162:0/1317568913 learned_addr learned my addr 172.21.15.162:0/1317568913 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:46:59.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.867+0000 7f19deffd700 1 --2- 172.21.15.162:0/1317568913 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f19e00b0d20 0x7f19e009b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:59.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.867+0000 7f19d7fff700 1 --2- 172.21.15.162:0/1317568913 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f19e00b1870 0x7f19e013e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:59.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.871+0000 7f19df7fe700 1 -- 172.21.15.162:0/1317568913 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f19e00b1870 msgr2=0x7f19e013e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:59.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.871+0000 7f19df7fe700 1 --2- 172.21.15.162:0/1317568913 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f19e00b1870 0x7f19e013e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:59.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.871+0000 7f19df7fe700 1 -- 172.21.15.162:0/1317568913 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f19e00b0d20 msgr2=0x7f19e009b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:46:59.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.871+0000 7f19df7fe700 1 --2- 172.21.15.162:0/1317568913 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f19e00b0d20 0x7f19e009b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:46:59.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.871+0000 7f19df7fe700 1 -- 172.21.15.162:0/1317568913 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e0097b60 con 0x7f19e00b4720 2020-11-28T21:46:59.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.871+0000 7f19d7fff700 1 --2- 172.21.15.162:0/1317568913 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f19e00b1870 0x7f19e013e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:46:59.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.871+0000 7f19df7fe700 1 --2- 172.21.15.162:0/1317568913 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f19e00b4720 0x7f19e0098590 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f19d8001490 tx=0x7f19d8007ee0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:59.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.871+0000 7f19dcff9700 1 -- 172.21.15.162:0/1317568913 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19d800a7b0 con 0x7f19e00b4720 2020-11-28T21:46:59.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.871+0000 7f19dcff9700 1 -- 172.21.15.162:0/1317568913 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d800a910 con 0x7f19e00b4720 2020-11-28T21:46:59.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.871+0000 7f19dffff700 1 -- 172.21.15.162:0/1317568913 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19e009a990 con 0x7f19e00b4720 2020-11-28T21:46:59.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.871+0000 7f19dcff9700 1 -- 172.21.15.162:0/1317568913 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19d800aa80 con 0x7f19e00b4720 2020-11-28T21:46:59.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.871+0000 7f19dffff700 1 -- 172.21.15.162:0/1317568913 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e013eb60 con 0x7f19e00b4720 2020-11-28T21:46:59.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.871+0000 7f19dffff700 1 -- 172.21.15.162:0/1317568913 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19e0057050 con 0x7f19e00b4720 2020-11-28T21:46:59.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.871+0000 7f19dcff9700 1 -- 172.21.15.162:0/1317568913 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f19d8010070 con 0x7f19e00b4720 2020-11-28T21:46:59.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.871+0000 7f19dcff9700 1 --2- 172.21.15.162:0/1317568913 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f19c4055350 0x7f19c4057810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:46:59.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.871+0000 7f19dcff9700 1 -- 172.21.15.162:0/1317568913 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f19d806dca0 con 0x7f19e00b4720 2020-11-28T21:46:59.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.871+0000 7f19deffd700 1 --2- 172.21.15.162:0/1317568913 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f19c4055350 0x7f19c4057810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:46:59.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.871+0000 7f19deffd700 1 --2- 172.21.15.162:0/1317568913 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f19c4055350 0x7f19c4057810 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f19d000d000 tx=0x7f19d0000b80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:46:59.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:46:59.879+0000 7f19dcff9700 1 -- 172.21.15.162:0/1317568913 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f19d8072070 con 0x7f19e00b4720 2020-11-28T21:47:00.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.067+0000 7f19dffff700 1 -- 172.21.15.162:0/1317568913 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19e0061a60 con 0x7f19e00b4720 2020-11-28T21:47:00.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.067+0000 7f19dcff9700 1 -- 172.21.15.162:0/1317568913 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2132 (secure 0 0 0) 0x7f19d8043eb0 con 0x7f19e00b4720 2020-11-28T21:47:00.071 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:47:00.071 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/1297721315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1297721315},{"type":"v1","addr":"172.21.15.118:6833","nonce":1297721315}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3},{"gid":4264,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2415281381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2415281381},{"type":"v1","addr":"172.21.15.162:6837","nonce":2415281381}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:46:29.120502+0000","modified":"2020-11-28T21:46:59.721544+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":24,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-28T21:47:00.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.071+0000 7f19dffff700 1 -- 172.21.15.162:0/1317568913 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f19c4055350 msgr2=0x7f19c4057810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:00.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.071+0000 7f19dffff700 1 --2- 172.21.15.162:0/1317568913 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f19c4055350 0x7f19c4057810 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f19d000d000 tx=0x7f19d0000b80).stop 2020-11-28T21:47:00.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.071+0000 7f19dffff700 1 -- 172.21.15.162:0/1317568913 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f19e00b4720 msgr2=0x7f19e0098590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:00.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.071+0000 7f19dffff700 1 --2- 172.21.15.162:0/1317568913 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f19e00b4720 0x7f19e0098590 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f19d8001490 tx=0x7f19d8007ee0).stop 2020-11-28T21:47:00.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.071+0000 7f19dffff700 1 -- 172.21.15.162:0/1317568913 shutdown_connections 2020-11-28T21:47:00.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.071+0000 7f19dffff700 1 --2- 172.21.15.162:0/1317568913 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f19c4055350 0x7f19c4057810 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:00.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.071+0000 7f19dffff700 1 --2- 172.21.15.162:0/1317568913 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f19e00b4720 0x7f19e0098590 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:00.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.071+0000 7f19dffff700 1 --2- 172.21.15.162:0/1317568913 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f19e00b0d20 0x7f19e009b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:00.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.071+0000 7f19dffff700 1 --2- 172.21.15.162:0/1317568913 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f19e00b1870 0x7f19e013e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:00.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.071+0000 7f19dffff700 1 -- 172.21.15.162:0/1317568913 >> 172.21.15.162:0/1317568913 conn(0x7f19e00b54f0 msgr2=0x7f19e0141f10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:00.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.071+0000 7f19dffff700 1 -- 172.21.15.162:0/1317568913 shutdown_connections 2020-11-28T21:47:00.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.071+0000 7f19dffff700 1 -- 172.21.15.162:0/1317568913 wait complete. 2020-11-28T21:47:00.076 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 6 2020-11-28T21:47:00.084 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2020-11-28T21:47:00.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.243+0000 7fae8b1d9700 1 Processor -- start 2020-11-28T21:47:00.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.243+0000 7fae8b1d9700 1 -- start start 2020-11-28T21:47:00.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.243+0000 7fae8b1d9700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fae840b5bf0 0x7fae84142980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:00.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.243+0000 7fae8b1d9700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fae84060cb0 0x7fae840b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:00.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.243+0000 7fae8b1d9700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae8405f380 con 0x7fae84060230 2020-11-28T21:47:00.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.243+0000 7fae8b1d9700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae84143930 con 0x7fae840b5bf0 2020-11-28T21:47:00.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.243+0000 7fae8b1d9700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae840b3f50 con 0x7fae84060cb0 2020-11-28T21:47:00.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.243+0000 7fae8a1d7700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fae840b5bf0 0x7fae84142980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:00.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.243+0000 7fae8a9d8700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fae84060230 0x7fae8413fbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.162:35320/0 (socket says 172.21.15.162:35320) 2020-11-28T21:47:00.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.243+0000 7fae899d6700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fae84060cb0 0x7fae840b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:00.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.243+0000 7fae8a9d8700 1 -- 172.21.15.162:0/2024876681 learned_addr learned my addr 172.21.15.162:0/2024876681 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:47:00.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.247+0000 7fae891d5700 1 -- 172.21.15.162:0/2024876681 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 578562963 0 0) 0x7fae8405f380 con 0x7fae84060230 2020-11-28T21:47:00.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.247+0000 7fae891d5700 1 -- 172.21.15.162:0/2024876681 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae6c003230 con 0x7fae84060230 2020-11-28T21:47:00.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.247+0000 7fae899d6700 1 -- 172.21.15.162:0/2024876681 >> v2:172.21.15.118:3300/0 conn(0x7fae840b5bf0 msgr2=0x7fae84142980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:00.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.247+0000 7fae899d6700 1 --2- 172.21.15.162:0/2024876681 >> v2:172.21.15.118:3300/0 conn(0x7fae840b5bf0 0x7fae84142980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:00.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.247+0000 7fae899d6700 1 -- 172.21.15.162:0/2024876681 >> v1:172.21.15.118:6789/0 conn(0x7fae84060230 legacy=0x7fae8413fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:00.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.247+0000 7fae899d6700 1 -- 172.21.15.162:0/2024876681 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae8405f5b0 con 0x7fae84060cb0 2020-11-28T21:47:00.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.247+0000 7fae899d6700 1 --2- 172.21.15.162:0/2024876681 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fae84060cb0 0x7fae840b3a60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fae78002f30 tx=0x7fae78006dc0).ready entity=mon.2 client_cookie=2cf9cf3d554311b6 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:00.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.247+0000 7fae891d5700 1 -- 172.21.15.162:0/2024876681 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae780096a0 con 0x7fae84060cb0 2020-11-28T21:47:00.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.247+0000 7fae891d5700 1 -- 172.21.15.162:0/2024876681 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae78009800 con 0x7fae84060cb0 2020-11-28T21:47:00.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.247+0000 7fae891d5700 1 -- 172.21.15.162:0/2024876681 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae78009b60 con 0x7fae84060cb0 2020-11-28T21:47:00.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.247+0000 7fae8b1d9700 1 -- 172.21.15.162:0/2024876681 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fae84060cb0 msgr2=0x7fae840b3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:00.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.247+0000 7fae8b1d9700 1 --2- 172.21.15.162:0/2024876681 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fae84060cb0 0x7fae840b3a60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fae78002f30 tx=0x7fae78006dc0).stop 2020-11-28T21:47:00.253 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.247+0000 7fae8b1d9700 1 -- 172.21.15.162:0/2024876681 shutdown_connections 2020-11-28T21:47:00.253 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.247+0000 7fae8b1d9700 1 --2- 172.21.15.162:0/2024876681 >> v2:172.21.15.118:3300/0 conn(0x7fae840b5bf0 0x7fae84142980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:00.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.247+0000 7fae8b1d9700 1 --2- 172.21.15.162:0/2024876681 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fae84060cb0 0x7fae840b3a60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:00.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.247+0000 7fae8b1d9700 1 -- 172.21.15.162:0/2024876681 >> 172.21.15.162:0/2024876681 conn(0x7fae8405b970 msgr2=0x7fae8405bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:00.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.247+0000 7fae8b1d9700 1 -- 172.21.15.162:0/2024876681 shutdown_connections 2020-11-28T21:47:00.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.247+0000 7fae8b1d9700 1 -- 172.21.15.162:0/2024876681 wait complete. 2020-11-28T21:47:00.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.251+0000 7fae8b1d9700 1 Processor -- start 2020-11-28T21:47:00.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.251+0000 7fae8b1d9700 1 -- start start 2020-11-28T21:47:00.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.251+0000 7fae8b1d9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fae84060230 0x7fae840ac950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:00.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.251+0000 7fae8b1d9700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fae84060cb0 0x7fae840ae240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:00.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.251+0000 7fae8b1d9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae840b5bf0 0x7fae840b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:00.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.251+0000 7fae8b1d9700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae840aece0 con 0x7fae84060230 2020-11-28T21:47:00.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.251+0000 7fae8b1d9700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae840aff30 con 0x7fae840b5bf0 2020-11-28T21:47:00.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.251+0000 7fae8b1d9700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae840af9f0 con 0x7fae84060cb0 2020-11-28T21:47:00.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.251+0000 7fae8a9d8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae840b5bf0 0x7fae840b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:00.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.251+0000 7fae8a9d8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae840b5bf0 0x7fae840b1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38336/0 (socket says 172.21.15.162:38336) 2020-11-28T21:47:00.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.251+0000 7fae8a9d8700 1 -- 172.21.15.162:0/2950084199 learned_addr learned my addr 172.21.15.162:0/2950084199 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:00.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.251+0000 7fae8a1d7700 1 --2- 172.21.15.162:0/2950084199 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fae84060230 0x7fae840ac950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:00.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.251+0000 7fae899d6700 1 --2- 172.21.15.162:0/2950084199 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fae84060cb0 0x7fae840ae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:00.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.251+0000 7fae8a9d8700 1 -- 172.21.15.162:0/2950084199 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fae84060cb0 msgr2=0x7fae840ae240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:00.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.251+0000 7fae8a9d8700 1 --2- 172.21.15.162:0/2950084199 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fae84060cb0 0x7fae840ae240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:00.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.251+0000 7fae8a9d8700 1 -- 172.21.15.162:0/2950084199 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fae84060230 msgr2=0x7fae840ac950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:00.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.251+0000 7fae8a9d8700 1 --2- 172.21.15.162:0/2950084199 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fae84060230 0x7fae840ac950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:00.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.251+0000 7fae8a9d8700 1 -- 172.21.15.162:0/2950084199 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae840afc90 con 0x7fae840b5bf0 2020-11-28T21:47:00.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.251+0000 7fae8a9d8700 1 --2- 172.21.15.162:0/2950084199 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae840b5bf0 0x7fae840b1f40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7fae7c001450 tx=0x7fae7c007dd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:00.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.251+0000 7fae777fe700 1 -- 172.21.15.162:0/2950084199 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae7c005b70 con 0x7fae840b5bf0 2020-11-28T21:47:00.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.255+0000 7fae777fe700 1 -- 172.21.15.162:0/2950084199 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae7c005cd0 con 0x7fae840b5bf0 2020-11-28T21:47:00.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.255+0000 7fae8b1d9700 1 -- 172.21.15.162:0/2950084199 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae84146420 con 0x7fae840b5bf0 2020-11-28T21:47:00.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.255+0000 7fae777fe700 1 -- 172.21.15.162:0/2950084199 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae7c007300 con 0x7fae840b5bf0 2020-11-28T21:47:00.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.255+0000 7fae8b1d9700 1 -- 172.21.15.162:0/2950084199 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae840aea60 con 0x7fae840b5bf0 2020-11-28T21:47:00.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.255+0000 7fae8b1d9700 1 -- 172.21.15.162:0/2950084199 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae840adfa0 con 0x7fae840b5bf0 2020-11-28T21:47:00.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.255+0000 7fae777fe700 1 -- 172.21.15.162:0/2950084199 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fae7c010030 con 0x7fae840b5bf0 2020-11-28T21:47:00.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.255+0000 7fae777fe700 1 --2- 172.21.15.162:0/2950084199 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fae64055330 0x7fae640577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:00.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.255+0000 7fae777fe700 1 -- 172.21.15.162:0/2950084199 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fae7c06e010 con 0x7fae840b5bf0 2020-11-28T21:47:00.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.255+0000 7fae8a1d7700 1 --2- 172.21.15.162:0/2950084199 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fae64055330 0x7fae640577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:00.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.255+0000 7fae8a1d7700 1 --2- 172.21.15.162:0/2950084199 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fae64055330 0x7fae640577f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fae8000b000 tx=0x7fae80006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:00.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.263+0000 7fae777fe700 1 -- 172.21.15.162:0/2950084199 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fae840adfa0 con 0x7fae840b5bf0 2020-11-28T21:47:00.448 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.447+0000 7fae8b1d9700 1 -- 172.21.15.162:0/2950084199 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fae840adfa0 con 0x7fae840b5bf0 2020-11-28T21:47:00.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.927+0000 7fae777fe700 1 -- 172.21.15.162:0/2950084199 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v7) v1 ==== 104+0+0 (secure 0 0 0) 0x7fae840adfa0 con 0x7fae840b5bf0 2020-11-28T21:47:00.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.927+0000 7fae8b1d9700 1 -- 172.21.15.162:0/2950084199 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fae64055330 msgr2=0x7fae640577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:00.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.927+0000 7fae8b1d9700 1 --2- 172.21.15.162:0/2950084199 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fae64055330 0x7fae640577f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fae8000b000 tx=0x7fae80006dc0).stop 2020-11-28T21:47:00.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.927+0000 7fae8b1d9700 1 -- 172.21.15.162:0/2950084199 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae840b5bf0 msgr2=0x7fae840b1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:00.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.927+0000 7fae8b1d9700 1 --2- 172.21.15.162:0/2950084199 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae840b5bf0 0x7fae840b1f40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7fae7c001450 tx=0x7fae7c007dd0).stop 2020-11-28T21:47:00.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.931+0000 7fae8b1d9700 1 -- 172.21.15.162:0/2950084199 shutdown_connections 2020-11-28T21:47:00.933 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.931+0000 7fae8b1d9700 1 --2- 172.21.15.162:0/2950084199 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fae64055330 0x7fae640577f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:00.933 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.931+0000 7fae8b1d9700 1 --2- 172.21.15.162:0/2950084199 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae840b5bf0 0x7fae840b1f40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:00.933 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.931+0000 7fae8b1d9700 1 --2- 172.21.15.162:0/2950084199 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fae84060230 0x7fae840ac950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:00.933 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.931+0000 7fae8b1d9700 1 --2- 172.21.15.162:0/2950084199 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fae84060cb0 0x7fae840ae240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:00.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.931+0000 7fae8b1d9700 1 -- 172.21.15.162:0/2950084199 >> 172.21.15.162:0/2950084199 conn(0x7fae8405b970 msgr2=0x7fae840b40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:00.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.931+0000 7fae8b1d9700 1 -- 172.21.15.162:0/2950084199 shutdown_connections 2020-11-28T21:47:00.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:00.931+0000 7fae8b1d9700 1 -- 172.21.15.162:0/2950084199 wait complete. 2020-11-28T21:47:00.950 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-11-28T21:47:01.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.071+0000 7fa20e3aa700 1 Processor -- start 2020-11-28T21:47:01.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.071+0000 7fa20e3aa700 1 -- start start 2020-11-28T21:47:01.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.071+0000 7fa20e3aa700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fa2080605c0 0x7fa208140390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:01.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.071+0000 7fa20e3aa700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa208143950 con 0x7fa2080b9ce0 2020-11-28T21:47:01.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.071+0000 7fa20e3aa700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2080ba0c0 con 0x7fa2081433c0 2020-11-28T21:47:01.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.071+0000 7fa20e3aa700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2080b4340 con 0x7fa2080605c0 2020-11-28T21:47:01.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.071+0000 7fa2077fe700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fa2081433c0 0x7fa20813fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:45186/0 (socket says 172.21.15.162:45186) 2020-11-28T21:47:01.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.071+0000 7fa2077fe700 1 -- 172.21.15.162:0/3546595064 learned_addr learned my addr 172.21.15.162:0/3546595064 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:47:01.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.071+0000 7fa20c947700 1 --2- 172.21.15.162:0/3546595064 >> v2:172.21.15.118:3300/0 conn(0x7fa2080605c0 0x7fa208140390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:01.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.071+0000 7fa206ffd700 1 -- 172.21.15.162:0/3546595064 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2056185597 0 0) 0x7fa2080ba0c0 con 0x7fa2081433c0 2020-11-28T21:47:01.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.071+0000 7fa206ffd700 1 -- 172.21.15.162:0/3546595064 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1f4003230 con 0x7fa2081433c0 2020-11-28T21:47:01.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.071+0000 7fa20c947700 1 -- 172.21.15.162:0/3546595064 >> v1:172.21.15.162:6789/0 conn(0x7fa2081433c0 legacy=0x7fa20813fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:01.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.071+0000 7fa206ffd700 1 -- 172.21.15.162:0/3546595064 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2166648873 0 0) 0x7fa1f4003230 con 0x7fa2081433c0 2020-11-28T21:47:01.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.071+0000 7fa20c947700 1 -- 172.21.15.162:0/3546595064 >> v1:172.21.15.118:6789/0 conn(0x7fa2080b9ce0 legacy=0x7fa2080b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:01.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.075+0000 7fa20c947700 1 -- 172.21.15.162:0/3546595064 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa208066c30 con 0x7fa2080605c0 2020-11-28T21:47:01.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.075+0000 7fa20c947700 1 --2- 172.21.15.162:0/3546595064 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa2080605c0 0x7fa208140390 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fa1fc001120 tx=0x7fa2000091a0).ready entity=mon.0 client_cookie=17762cd5c11ad194 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:01.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.075+0000 7fa206ffd700 1 -- 172.21.15.162:0/3546595064 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1f80016d0 con 0x7fa2080605c0 2020-11-28T21:47:01.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.075+0000 7fa206ffd700 1 -- 172.21.15.162:0/3546595064 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa20000b860 con 0x7fa2080605c0 2020-11-28T21:47:01.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.075+0000 7fa206ffd700 1 -- 172.21.15.162:0/3546595064 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa20000b9d0 con 0x7fa2080605c0 2020-11-28T21:47:01.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.075+0000 7fa20e3aa700 1 -- 172.21.15.162:0/3546595064 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa2080605c0 msgr2=0x7fa208140390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:01.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.075+0000 7fa20e3aa700 1 --2- 172.21.15.162:0/3546595064 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa2080605c0 0x7fa208140390 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fa1fc001120 tx=0x7fa2000091a0).stop 2020-11-28T21:47:01.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.075+0000 7fa20e3aa700 1 -- 172.21.15.162:0/3546595064 shutdown_connections 2020-11-28T21:47:01.083 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.075+0000 7fa20e3aa700 1 --2- 172.21.15.162:0/3546595064 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa2080605c0 0x7fa208140390 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:01.083 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.075+0000 7fa20e3aa700 1 -- 172.21.15.162:0/3546595064 >> 172.21.15.162:0/3546595064 conn(0x7fa20805c430 msgr2=0x7fa2080b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:01.083 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.075+0000 7fa20e3aa700 1 -- 172.21.15.162:0/3546595064 shutdown_connections 2020-11-28T21:47:01.083 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.075+0000 7fa20e3aa700 1 -- 172.21.15.162:0/3546595064 wait complete. 2020-11-28T21:47:01.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.075+0000 7fa20e3aa700 1 Processor -- start 2020-11-28T21:47:01.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.075+0000 7fa20e3aa700 1 -- start start 2020-11-28T21:47:01.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.079+0000 7fa20e3aa700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2080605c0 0x7fa2081524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:01.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.079+0000 7fa20e3aa700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa2080b9ce0 0x7fa2080b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:01.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.079+0000 7fa20e3aa700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa2081433c0 0x7fa2080b0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:01.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.079+0000 7fa20e3aa700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa208152990 con 0x7fa2080b9ce0 2020-11-28T21:47:01.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.079+0000 7fa20e3aa700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa208145740 con 0x7fa2080605c0 2020-11-28T21:47:01.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.079+0000 7fa20e3aa700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa208151d60 con 0x7fa2081433c0 2020-11-28T21:47:01.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.079+0000 7fa207fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2080605c0 0x7fa2081524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:01.086 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.079+0000 7fa20c947700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa2081433c0 0x7fa2080b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:01.086 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.079+0000 7fa207fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2080605c0 0x7fa2081524a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38350/0 (socket says 172.21.15.162:38350) 2020-11-28T21:47:01.086 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.079+0000 7fa207fff700 1 -- 172.21.15.162:0/3736186123 learned_addr learned my addr 172.21.15.162:0/3736186123 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:01.086 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.079+0000 7fa2077fe700 1 --2- 172.21.15.162:0/3736186123 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa2080b9ce0 0x7fa2080b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:01.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.079+0000 7fa207fff700 1 -- 172.21.15.162:0/3736186123 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa2081433c0 msgr2=0x7fa2080b0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:01.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.079+0000 7fa207fff700 1 --2- 172.21.15.162:0/3736186123 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa2081433c0 0x7fa2080b0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:01.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.079+0000 7fa207fff700 1 -- 172.21.15.162:0/3736186123 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa2080b9ce0 msgr2=0x7fa2080b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:01.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.079+0000 7fa207fff700 1 --2- 172.21.15.162:0/3736186123 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa2080b9ce0 0x7fa2080b57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:01.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.079+0000 7fa207fff700 1 -- 172.21.15.162:0/3736186123 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa208152000 con 0x7fa2080605c0 2020-11-28T21:47:01.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.079+0000 7fa207fff700 1 --2- 172.21.15.162:0/3736186123 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2080605c0 0x7fa2081524a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7fa1fc001160 tx=0x7fa1fc007b70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:01.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.079+0000 7fa2057fa700 1 -- 172.21.15.162:0/3736186123 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1fc00e4a0 con 0x7fa2080605c0 2020-11-28T21:47:01.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.079+0000 7fa2057fa700 1 -- 172.21.15.162:0/3736186123 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1fc00e600 con 0x7fa2080605c0 2020-11-28T21:47:01.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.079+0000 7fa20e3aa700 1 -- 172.21.15.162:0/3736186123 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2080b5c90 con 0x7fa2080605c0 2020-11-28T21:47:01.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.079+0000 7fa2057fa700 1 -- 172.21.15.162:0/3736186123 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1fc00e9a0 con 0x7fa2080605c0 2020-11-28T21:47:01.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.079+0000 7fa20e3aa700 1 -- 172.21.15.162:0/3736186123 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2080b0a10 con 0x7fa2080605c0 2020-11-28T21:47:01.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.079+0000 7fa20e3aa700 1 -- 172.21.15.162:0/3736186123 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa208152190 con 0x7fa2080605c0 2020-11-28T21:47:01.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.079+0000 7fa2057fa700 1 -- 172.21.15.162:0/3736186123 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa1fc019030 con 0x7fa2080605c0 2020-11-28T21:47:01.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.083+0000 7fa2057fa700 1 --2- 172.21.15.162:0/3736186123 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa1e80554e0 0x7fa1e80579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:01.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.083+0000 7fa2057fa700 1 -- 172.21.15.162:0/3736186123 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fa1fc06d400 con 0x7fa2080605c0 2020-11-28T21:47:01.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.083+0000 7fa2077fe700 1 --2- 172.21.15.162:0/3736186123 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa1e80554e0 0x7fa1e80579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:01.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.083+0000 7fa2077fe700 1 --2- 172.21.15.162:0/3736186123 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa1e80554e0 0x7fa1e80579a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fa1f800d000 tx=0x7fa1f8000b80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:01.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.091+0000 7fa2057fa700 1 -- 172.21.15.162:0/3736186123 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa1fc041df0 con 0x7fa2080605c0 2020-11-28T21:47:01.236 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.235+0000 7fa20e3aa700 1 -- 172.21.15.162:0/3736186123 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa208057050 con 0x7fa2080605c0 2020-11-28T21:47:01.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.935+0000 7fa2057fa700 1 -- 172.21.15.162:0/3736186123 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) v1 ==== 185+0+0 (secure 0 0 0) 0x7fa1fc043650 con 0x7fa2080605c0 2020-11-28T21:47:01.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.939+0000 7fa20e3aa700 1 -- 172.21.15.162:0/3736186123 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa1e80554e0 msgr2=0x7fa1e80579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:01.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.939+0000 7fa20e3aa700 1 --2- 172.21.15.162:0/3736186123 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa1e80554e0 0x7fa1e80579a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fa1f800d000 tx=0x7fa1f8000b80).stop 2020-11-28T21:47:01.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.939+0000 7fa20e3aa700 1 -- 172.21.15.162:0/3736186123 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2080605c0 msgr2=0x7fa2081524a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:01.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.939+0000 7fa20e3aa700 1 --2- 172.21.15.162:0/3736186123 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2080605c0 0x7fa2081524a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7fa1fc001160 tx=0x7fa1fc007b70).stop 2020-11-28T21:47:01.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.939+0000 7fa20e3aa700 1 -- 172.21.15.162:0/3736186123 shutdown_connections 2020-11-28T21:47:01.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.939+0000 7fa20e3aa700 1 --2- 172.21.15.162:0/3736186123 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa1e80554e0 0x7fa1e80579a0 secure :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0x7fa1f800d000 tx=0x7fa1f8000b80).stop 2020-11-28T21:47:01.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.939+0000 7fa20e3aa700 1 --2- 172.21.15.162:0/3736186123 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa2081433c0 0x7fa2080b0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:01.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.939+0000 7fa20e3aa700 1 --2- 172.21.15.162:0/3736186123 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2080605c0 0x7fa2081524a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:01.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.939+0000 7fa20e3aa700 1 --2- 172.21.15.162:0/3736186123 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa2080b9ce0 0x7fa2080b57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:01.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.939+0000 7fa20e3aa700 1 -- 172.21.15.162:0/3736186123 >> 172.21.15.162:0/3736186123 conn(0x7fa20805c430 msgr2=0x7fa20813dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:01.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.939+0000 7fa20e3aa700 1 -- 172.21.15.162:0/3736186123 shutdown_connections 2020-11-28T21:47:01.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:01.939+0000 7fa20e3aa700 1 -- 172.21.15.162:0/3736186123 wait complete. 2020-11-28T21:47:01.947 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_metadata' removed 2020-11-28T21:47:01.958 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-11-28T21:47:02.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.123+0000 7fd196ddc700 1 Processor -- start 2020-11-28T21:47:02.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.123+0000 7fd196ddc700 1 -- start start 2020-11-28T21:47:02.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.127+0000 7fd196ddc700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd1900b15e0 0x7fd190099b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:02.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.127+0000 7fd196ddc700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd19009a050 0x7fd19005bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:02.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.127+0000 7fd196ddc700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd19006aca0 con 0x7fd19013d0c0 2020-11-28T21:47:02.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.127+0000 7fd196ddc700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd19013d4a0 con 0x7fd19009a050 2020-11-28T21:47:02.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.127+0000 7fd196ddc700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd190098da0 con 0x7fd1900b15e0 2020-11-28T21:47:02.138 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.127+0000 7fd1965db700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd19009a050 0x7fd19005bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:02.138 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.127+0000 7fd195dda700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fd19013d0c0 0x7fd1900b0ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.162:35346/0 (socket says 172.21.15.162:35346) 2020-11-28T21:47:02.138 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.127+0000 7fd1955d9700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd1900b15e0 0x7fd190099b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:02.138 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.127+0000 7fd195dda700 1 -- 172.21.15.162:0/1061318444 learned_addr learned my addr 172.21.15.162:0/1061318444 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:47:02.138 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.127+0000 7fd1965db700 1 -- 172.21.15.162:0/1061318444 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd1900b15e0 msgr2=0x7fd190099b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:02.139 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.127+0000 7fd1965db700 1 --2- 172.21.15.162:0/1061318444 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd1900b15e0 0x7fd190099b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:02.139 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.127+0000 7fd1965db700 1 -- 172.21.15.162:0/1061318444 >> v1:172.21.15.118:6789/0 conn(0x7fd19013d0c0 legacy=0x7fd1900b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:02.139 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.127+0000 7fd1965db700 1 -- 172.21.15.162:0/1061318444 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd19006af40 con 0x7fd19009a050 2020-11-28T21:47:02.139 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.127+0000 7fd1965db700 1 --2- 172.21.15.162:0/1061318444 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd19009a050 0x7fd19005bf90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fd188002f30 tx=0x7fd188006dc0).ready entity=mon.0 client_cookie=7010db610526172 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:02.139 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.127+0000 7fd194dd8700 1 -- 172.21.15.162:0/1061318444 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd18800f9c0 con 0x7fd19009a050 2020-11-28T21:47:02.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.127+0000 7fd194dd8700 1 -- 172.21.15.162:0/1061318444 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd18800fb20 con 0x7fd19009a050 2020-11-28T21:47:02.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.127+0000 7fd194dd8700 1 -- 172.21.15.162:0/1061318444 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd18800fe80 con 0x7fd19009a050 2020-11-28T21:47:02.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.127+0000 7fd196ddc700 1 -- 172.21.15.162:0/1061318444 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd19009a050 msgr2=0x7fd19005bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:02.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.127+0000 7fd196ddc700 1 --2- 172.21.15.162:0/1061318444 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd19009a050 0x7fd19005bf90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fd188002f30 tx=0x7fd188006dc0).stop 2020-11-28T21:47:02.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.127+0000 7fd196ddc700 1 -- 172.21.15.162:0/1061318444 shutdown_connections 2020-11-28T21:47:02.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.127+0000 7fd196ddc700 1 --2- 172.21.15.162:0/1061318444 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd19009a050 0x7fd19005bf90 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:02.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.127+0000 7fd196ddc700 1 --2- 172.21.15.162:0/1061318444 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd1900b15e0 0x7fd190099b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:02.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.127+0000 7fd196ddc700 1 -- 172.21.15.162:0/1061318444 >> 172.21.15.162:0/1061318444 conn(0x7fd1900b5520 msgr2=0x7fd1900b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:02.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.127+0000 7fd196ddc700 1 -- 172.21.15.162:0/1061318444 shutdown_connections 2020-11-28T21:47:02.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.127+0000 7fd196ddc700 1 -- 172.21.15.162:0/1061318444 wait complete. 2020-11-28T21:47:02.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.131+0000 7fd196ddc700 1 Processor -- start 2020-11-28T21:47:02.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.131+0000 7fd196ddc700 1 -- start start 2020-11-28T21:47:02.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.131+0000 7fd196ddc700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd19009a050 0x7fd19013f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:02.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.131+0000 7fd196ddc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd1900b15e0 0x7fd190155180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:02.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.131+0000 7fd196ddc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd19013d0c0 0x7fd1900ad740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:02.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.131+0000 7fd196ddc700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd190155d70 con 0x7fd1900b15e0 2020-11-28T21:47:02.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.131+0000 7fd196ddc700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd190155830 con 0x7fd19013d0c0 2020-11-28T21:47:02.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.131+0000 7fd196ddc700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd190155ad0 con 0x7fd19009a050 2020-11-28T21:47:02.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.135+0000 7fd1965db700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd19013d0c0 0x7fd1900ad740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:02.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.135+0000 7fd1965db700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd19013d0c0 0x7fd1900ad740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38364/0 (socket says 172.21.15.162:38364) 2020-11-28T21:47:02.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.135+0000 7fd1965db700 1 -- 172.21.15.162:0/15638288 learned_addr learned my addr 172.21.15.162:0/15638288 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:02.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.135+0000 7fd195dda700 1 --2- 172.21.15.162:0/15638288 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd19009a050 0x7fd19013f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:02.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.135+0000 7fd1955d9700 1 --2- 172.21.15.162:0/15638288 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd1900b15e0 0x7fd190155180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:02.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.135+0000 7fd1965db700 1 -- 172.21.15.162:0/15638288 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd19009a050 msgr2=0x7fd19013f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:02.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.135+0000 7fd1965db700 1 --2- 172.21.15.162:0/15638288 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd19009a050 0x7fd19013f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:02.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.135+0000 7fd1965db700 1 -- 172.21.15.162:0/15638288 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd1900b15e0 msgr2=0x7fd190155180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:02.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.135+0000 7fd1965db700 1 --2- 172.21.15.162:0/15638288 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd1900b15e0 0x7fd190155180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:02.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.135+0000 7fd1965db700 1 -- 172.21.15.162:0/15638288 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd19013f410 con 0x7fd19013d0c0 2020-11-28T21:47:02.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.135+0000 7fd1965db700 1 --2- 172.21.15.162:0/15638288 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd19013d0c0 0x7fd1900ad740 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7fd188002f30 tx=0x7fd188017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:02.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.135+0000 7fd17effd700 1 -- 172.21.15.162:0/15638288 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd188010e90 con 0x7fd19013d0c0 2020-11-28T21:47:02.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.135+0000 7fd17effd700 1 -- 172.21.15.162:0/15638288 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd18801e070 con 0x7fd19013d0c0 2020-11-28T21:47:02.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.135+0000 7fd196ddc700 1 -- 172.21.15.162:0/15638288 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd19005d0f0 con 0x7fd19013d0c0 2020-11-28T21:47:02.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.135+0000 7fd17effd700 1 -- 172.21.15.162:0/15638288 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd18801a640 con 0x7fd19013d0c0 2020-11-28T21:47:02.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.135+0000 7fd196ddc700 1 -- 172.21.15.162:0/15638288 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1900addd0 con 0x7fd19013d0c0 2020-11-28T21:47:02.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.135+0000 7fd196ddc700 1 -- 172.21.15.162:0/15638288 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd19013f5a0 con 0x7fd19013d0c0 2020-11-28T21:47:02.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.135+0000 7fd17effd700 1 -- 172.21.15.162:0/15638288 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd18802a070 con 0x7fd19013d0c0 2020-11-28T21:47:02.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.139+0000 7fd17effd700 1 --2- 172.21.15.162:0/15638288 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd170055350 0x7fd170057810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:02.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.139+0000 7fd17effd700 1 -- 172.21.15.162:0/15638288 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5721+0+0 (secure 0 0 0) 0x7fd18806f570 con 0x7fd19013d0c0 2020-11-28T21:47:02.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.139+0000 7fd195dda700 1 --2- 172.21.15.162:0/15638288 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd170055350 0x7fd170057810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:02.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.139+0000 7fd195dda700 1 --2- 172.21.15.162:0/15638288 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd170055350 0x7fd170057810 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fd18c00d000 tx=0x7fd18c007f30).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:02.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.143+0000 7fd17effd700 1 -- 172.21.15.162:0/15638288 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd1880459d0 con 0x7fd19013d0c0 2020-11-28T21:47:02.280 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.279+0000 7fd196ddc700 1 -- 172.21.15.162:0/15638288 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd190057050 con 0x7fd19013d0c0 2020-11-28T21:47:02.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.939+0000 7fd17effd700 1 -- 172.21.15.162:0/15638288 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) v1 ==== 173+0+0 (secure 0 0 0) 0x7fd188048ff0 con 0x7fd19013d0c0 2020-11-28T21:47:02.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.943+0000 7fd196ddc700 1 -- 172.21.15.162:0/15638288 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd170055350 msgr2=0x7fd170057810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:02.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.943+0000 7fd196ddc700 1 --2- 172.21.15.162:0/15638288 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd170055350 0x7fd170057810 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fd18c00d000 tx=0x7fd18c007f30).stop 2020-11-28T21:47:02.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.943+0000 7fd196ddc700 1 -- 172.21.15.162:0/15638288 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd19013d0c0 msgr2=0x7fd1900ad740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:02.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.943+0000 7fd196ddc700 1 --2- 172.21.15.162:0/15638288 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd19013d0c0 0x7fd1900ad740 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7fd188002f30 tx=0x7fd188017000).stop 2020-11-28T21:47:02.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.943+0000 7fd196ddc700 1 -- 172.21.15.162:0/15638288 shutdown_connections 2020-11-28T21:47:02.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.943+0000 7fd196ddc700 1 --2- 172.21.15.162:0/15638288 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd170055350 0x7fd170057810 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:02.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.943+0000 7fd196ddc700 1 --2- 172.21.15.162:0/15638288 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd19009a050 0x7fd19013f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:02.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.943+0000 7fd196ddc700 1 --2- 172.21.15.162:0/15638288 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd19013d0c0 0x7fd1900ad740 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:02.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.943+0000 7fd196ddc700 1 --2- 172.21.15.162:0/15638288 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd1900b15e0 0x7fd190155180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:02.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.943+0000 7fd196ddc700 1 -- 172.21.15.162:0/15638288 >> 172.21.15.162:0/15638288 conn(0x7fd1900b5520 msgr2=0x7fd1900b1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:02.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.943+0000 7fd196ddc700 1 -- 172.21.15.162:0/15638288 shutdown_connections 2020-11-28T21:47:02.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:02.943+0000 7fd196ddc700 1 -- 172.21.15.162:0/15638288 wait complete. 2020-11-28T21:47:02.950 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_data' removed 2020-11-28T21:47:02.961 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-28T21:47:02.961 INFO:tasks.ceph.mds.a:Stopping old one... 2020-11-28T21:47:02.962 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-11-28T21:47:02.962 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T21:47:02.962 INFO:tasks.ceph.mds.c:Restarting daemon 2020-11-28T21:47:02.962 INFO:tasks.ceph.mds.c:Stopping old one... 2020-11-28T21:47:02.963 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2020-11-28T21:47:02.963 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T21:47:02.963 INFO:tasks.ceph.mds.b:Restarting daemon 2020-11-28T21:47:02.964 INFO:tasks.ceph.mds.b:Stopping old one... 2020-11-28T21:47:02.964 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-11-28T21:47:02.964 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T21:47:02.975 INFO:tasks.ceph.mds.a:Stopped 2020-11-28T21:47:02.975 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-28T21:47:02.976 INFO:tasks.ceph.mds.c:Stopped 2020-11-28T21:47:02.977 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2020-11-28T21:47:02.978 INFO:tasks.ceph.mds.b:Stopped 2020-11-28T21:47:02.978 INFO:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-11-28T21:47:03.020 INFO:tasks.ceph.mds.a:Started 2020-11-28T21:47:03.021 DEBUG:teuthology.parallel:result is None 2020-11-28T21:47:03.022 INFO:tasks.ceph.mds.c:Started 2020-11-28T21:47:03.022 DEBUG:teuthology.parallel:result is None 2020-11-28T21:47:03.023 INFO:tasks.ceph.mds.b:Started 2020-11-28T21:47:03.023 DEBUG:teuthology.parallel:result is None 2020-11-28T21:47:03.024 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist clear 2020-11-28T21:47:03.098 INFO:tasks.ceph.mds.c.smithi162.stdout:starting mds.c at 2020-11-28T21:47:03.099 INFO:tasks.ceph.mds.a.smithi162.stdout:starting mds.a at 2020-11-28T21:47:03.103 INFO:tasks.ceph.mds.b.smithi118.stdout:starting mds.b at 2020-11-28T21:47:03.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.139+0000 7f068a7bb700 1 Processor -- start 2020-11-28T21:47:03.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.139+0000 7f068a7bb700 1 -- start start 2020-11-28T21:47:03.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.139+0000 7f068a7bb700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f06840b10e0 0x7f06841469c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:03.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.139+0000 7f068a7bb700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06840b1590 con 0x7f068405fa90 2020-11-28T21:47:03.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.139+0000 7f068a7bb700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06840b6d80 con 0x7f06840b69a0 2020-11-28T21:47:03.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.139+0000 7f068a7bb700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06840b2200 con 0x7f06840b10e0 2020-11-28T21:47:03.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.139+0000 7f0683fff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f06840b69a0 0x7f06840b09d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:45236/0 (socket says 172.21.15.162:45236) 2020-11-28T21:47:03.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.139+0000 7f0683fff700 1 -- 172.21.15.162:0/1043930126 learned_addr learned my addr 172.21.15.162:0/1043930126 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:47:03.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.139+0000 7f0688d58700 1 --2- 172.21.15.162:0/1043930126 >> v2:172.21.15.162:3300/0 conn(0x7f06840b10e0 0x7f06841469c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:03.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.139+0000 7f06837fe700 1 -- 172.21.15.162:0/1043930126 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2970349997 0 0) 0x7f06840b6d80 con 0x7f06840b69a0 2020-11-28T21:47:03.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.139+0000 7f06837fe700 1 -- 172.21.15.162:0/1043930126 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0664003230 con 0x7f06840b69a0 2020-11-28T21:47:03.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.139+0000 7f0688d58700 1 -- 172.21.15.162:0/1043930126 >> v1:172.21.15.162:6789/0 conn(0x7f06840b69a0 legacy=0x7f06840b09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:03.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.139+0000 7f0688d58700 1 -- 172.21.15.162:0/1043930126 >> v1:172.21.15.118:6789/0 conn(0x7f068405fa90 legacy=0x7f06841462b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:03.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.139+0000 7f0688d58700 1 -- 172.21.15.162:0/1043930126 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06840b1990 con 0x7f06840b10e0 2020-11-28T21:47:03.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.143+0000 7f0688d58700 1 --2- 172.21.15.162:0/1043930126 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06840b10e0 0x7f06841469c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f067c0032b0 tx=0x7f067c0091a0).ready entity=mon.1 client_cookie=4f08d7addeedb9f7 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:03.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.143+0000 7f06837fe700 1 -- 172.21.15.162:0/1043930126 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f067c00b9a0 con 0x7f06840b10e0 2020-11-28T21:47:03.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.143+0000 7f06837fe700 1 -- 172.21.15.162:0/1043930126 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f067c00bb00 con 0x7f06840b10e0 2020-11-28T21:47:03.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.143+0000 7f06837fe700 1 -- 172.21.15.162:0/1043930126 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f067c00bc70 con 0x7f06840b10e0 2020-11-28T21:47:03.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.143+0000 7f068a7bb700 1 -- 172.21.15.162:0/1043930126 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06840b10e0 msgr2=0x7f06841469c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:03.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.143+0000 7f068a7bb700 1 --2- 172.21.15.162:0/1043930126 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06840b10e0 0x7f06841469c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f067c0032b0 tx=0x7f067c0091a0).stop 2020-11-28T21:47:03.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.143+0000 7f068a7bb700 1 -- 172.21.15.162:0/1043930126 shutdown_connections 2020-11-28T21:47:03.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.143+0000 7f068a7bb700 1 --2- 172.21.15.162:0/1043930126 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06840b10e0 0x7f06841469c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:03.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.143+0000 7f068a7bb700 1 -- 172.21.15.162:0/1043930126 >> 172.21.15.162:0/1043930126 conn(0x7f0684062170 msgr2=0x7f06840b5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:03.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.143+0000 7f068a7bb700 1 -- 172.21.15.162:0/1043930126 shutdown_connections 2020-11-28T21:47:03.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.143+0000 7f068a7bb700 1 -- 172.21.15.162:0/1043930126 wait complete. 2020-11-28T21:47:03.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.143+0000 7f068a7bb700 1 Processor -- start 2020-11-28T21:47:03.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.143+0000 7f068a7bb700 1 -- start start 2020-11-28T21:47:03.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.143+0000 7f068a7bb700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f068405fa90 0x7f06840b2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:03.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.143+0000 7f068a7bb700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f06840b10e0 0x7f0684147390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:03.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.143+0000 7f068a7bb700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f06840b69a0 0x7f0684147880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:03.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.143+0000 7f068a7bb700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06840b42e0 con 0x7f06840b69a0 2020-11-28T21:47:03.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.143+0000 7f068a7bb700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06840b5160 con 0x7f068405fa90 2020-11-28T21:47:03.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.143+0000 7f068a7bb700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06840b5400 con 0x7f06840b10e0 2020-11-28T21:47:03.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.143+0000 7f0683fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f068405fa90 0x7f06840b2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:03.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.143+0000 7f0683fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f068405fa90 0x7f06840b2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38400/0 (socket says 172.21.15.162:38400) 2020-11-28T21:47:03.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.143+0000 7f0683fff700 1 -- 172.21.15.162:0/3541798429 learned_addr learned my addr 172.21.15.162:0/3541798429 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:03.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.143+0000 7f067b7fe700 1 --2- 172.21.15.162:0/3541798429 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f06840b10e0 0x7f0684147390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:03.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.143+0000 7f0688d58700 1 --2- 172.21.15.162:0/3541798429 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f06840b69a0 0x7f0684147880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:03.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.147+0000 7f0683fff700 1 -- 172.21.15.162:0/3541798429 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f06840b10e0 msgr2=0x7f0684147390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:03.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.147+0000 7f0683fff700 1 --2- 172.21.15.162:0/3541798429 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f06840b10e0 0x7f0684147390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:03.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.147+0000 7f0683fff700 1 -- 172.21.15.162:0/3541798429 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f06840b69a0 msgr2=0x7f0684147880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:03.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.147+0000 7f0683fff700 1 --2- 172.21.15.162:0/3541798429 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f06840b69a0 0x7f0684147880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:03.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.147+0000 7f0683fff700 1 -- 172.21.15.162:0/3541798429 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06840b4510 con 0x7f068405fa90 2020-11-28T21:47:03.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.147+0000 7f0683fff700 1 --2- 172.21.15.162:0/3541798429 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f068405fa90 0x7f06840b2b80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f0684149030 tx=0x7f0674007aa0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:03.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.147+0000 7f0681ffb700 1 -- 172.21.15.162:0/3541798429 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06740072e0 con 0x7f068405fa90 2020-11-28T21:47:03.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.147+0000 7f0681ffb700 1 -- 172.21.15.162:0/3541798429 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0674007440 con 0x7f068405fa90 2020-11-28T21:47:03.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.147+0000 7f068a7bb700 1 -- 172.21.15.162:0/3541798429 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06840b47b0 con 0x7f068405fa90 2020-11-28T21:47:03.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.147+0000 7f0681ffb700 1 -- 172.21.15.162:0/3541798429 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06740057d0 con 0x7f068405fa90 2020-11-28T21:47:03.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.147+0000 7f068a7bb700 1 -- 172.21.15.162:0/3541798429 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f068414a0d0 con 0x7f068405fa90 2020-11-28T21:47:03.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.147+0000 7f068a7bb700 1 -- 172.21.15.162:0/3541798429 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06840b4940 con 0x7f068405fa90 2020-11-28T21:47:03.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.147+0000 7f0681ffb700 1 -- 172.21.15.162:0/3541798429 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f067401a060 con 0x7f068405fa90 2020-11-28T21:47:03.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.147+0000 7f0681ffb700 1 --2- 172.21.15.162:0/3541798429 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f06680553b0 0x7f0668057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:03.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.147+0000 7f0681ffb700 1 -- 172.21.15.162:0/3541798429 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f067406d290 con 0x7f068405fa90 2020-11-28T21:47:03.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.147+0000 7f067b7fe700 1 --2- 172.21.15.162:0/3541798429 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f06680553b0 0x7f0668057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:03.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.147+0000 7f067b7fe700 1 --2- 172.21.15.162:0/3541798429 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f06680553b0 0x7f0668057870 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f066c00d000 tx=0x7f066c007b60).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:03.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.155+0000 7f0681ffb700 1 -- 172.21.15.162:0/3541798429 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0674043810 con 0x7f068405fa90 2020-11-28T21:47:03.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:03.307+0000 7f068a7bb700 1 -- 172.21.15.162:0/3541798429 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f0684057050 con 0x7f068405fa90 2020-11-28T21:47:04.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.151+0000 7f0681ffb700 1 -- 172.21.15.162:0/3541798429 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v29) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0674018020 con 0x7f068405fa90 2020-11-28T21:47:04.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.151+0000 7f068a7bb700 1 -- 172.21.15.162:0/3541798429 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f06680553b0 msgr2=0x7f0668057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:04.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.151+0000 7f068a7bb700 1 --2- 172.21.15.162:0/3541798429 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f06680553b0 0x7f0668057870 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f066c00d000 tx=0x7f066c007b60).stop 2020-11-28T21:47:04.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.151+0000 7f068a7bb700 1 -- 172.21.15.162:0/3541798429 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f068405fa90 msgr2=0x7f06840b2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:04.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.151+0000 7f068a7bb700 1 --2- 172.21.15.162:0/3541798429 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f068405fa90 0x7f06840b2b80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f0684149030 tx=0x7f0674007aa0).stop 2020-11-28T21:47:04.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.151+0000 7f068a7bb700 1 -- 172.21.15.162:0/3541798429 shutdown_connections 2020-11-28T21:47:04.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.151+0000 7f068a7bb700 1 --2- 172.21.15.162:0/3541798429 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f06680553b0 0x7f0668057870 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:04.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.151+0000 7f068a7bb700 1 --2- 172.21.15.162:0/3541798429 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f06840b69a0 0x7f0684147880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:04.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.151+0000 7f068a7bb700 1 --2- 172.21.15.162:0/3541798429 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f068405fa90 0x7f06840b2b80 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:04.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.151+0000 7f068a7bb700 1 --2- 172.21.15.162:0/3541798429 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f06840b10e0 0x7f0684147390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:04.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.151+0000 7f068a7bb700 1 -- 172.21.15.162:0/3541798429 >> 172.21.15.162:0/3541798429 conn(0x7f0684062170 msgr2=0x7f06840ae6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:04.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.155+0000 7f068a7bb700 1 -- 172.21.15.162:0/3541798429 shutdown_connections 2020-11-28T21:47:04.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.155+0000 7f068a7bb700 1 -- 172.21.15.162:0/3541798429 wait complete. 2020-11-28T21:47:04.162 INFO:teuthology.orchestra.run.smithi162.stderr: removed all blacklist entries 2020-11-28T21:47:04.170 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2020-11-28T21:47:04.171 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth ls --format=json-pretty 2020-11-28T21:47:04.284 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.283+0000 7f2e19bf6700 1 Processor -- start 2020-11-28T21:47:04.285 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.283+0000 7f2e19bf6700 1 -- start start 2020-11-28T21:47:04.289 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.283+0000 7f2e19bf6700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f2e14098b30 0x7f2e140994f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:04.289 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.283+0000 7f2e19bf6700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2e140b0000 0x7f2e14060000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:04.289 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.283+0000 7f2e19bf6700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2e140604f0 0x7f2e14060980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:04.290 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.283+0000 7f2e19bf6700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e140b1030 con 0x7f2e14098b30 2020-11-28T21:47:04.290 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.283+0000 7f2e19bf6700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e140b0480 con 0x7f2e140b0000 2020-11-28T21:47:04.290 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.283+0000 7f2e19bf6700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e14098f10 con 0x7f2e140604f0 2020-11-28T21:47:04.291 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.283+0000 7f2e13fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2e140b0000 0x7f2e14060000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:04.291 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.283+0000 7f2e13fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2e140b0000 0x7f2e14060000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38412/0 (socket says 172.21.15.162:38412) 2020-11-28T21:47:04.291 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.283+0000 7f2e13fff700 1 -- 172.21.15.162:0/1594303749 learned_addr learned my addr 172.21.15.162:0/1594303749 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:04.291 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.283+0000 7f2e18bf4700 1 --2- 172.21.15.162:0/1594303749 >> v2:172.21.15.118:3300/0 conn(0x7f2e14098b30 0x7f2e140994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:04.292 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.283+0000 7f2e193f5700 1 --2- 172.21.15.162:0/1594303749 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2e140604f0 0x7f2e14060980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:04.292 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.283+0000 7f2e13fff700 1 -- 172.21.15.162:0/1594303749 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2e140604f0 msgr2=0x7f2e14060980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:04.292 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.283+0000 7f2e13fff700 1 --2- 172.21.15.162:0/1594303749 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2e140604f0 0x7f2e14060980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:04.292 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.283+0000 7f2e13fff700 1 -- 172.21.15.162:0/1594303749 >> v2:172.21.15.118:3300/0 conn(0x7f2e14098b30 msgr2=0x7f2e140994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:04.293 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.283+0000 7f2e13fff700 1 --2- 172.21.15.162:0/1594303749 >> v2:172.21.15.118:3300/0 conn(0x7f2e14098b30 0x7f2e140994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:04.293 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.283+0000 7f2e13fff700 1 -- 172.21.15.162:0/1594303749 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e140b0680 con 0x7f2e140b0000 2020-11-28T21:47:04.293 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.283+0000 7f2e13fff700 1 --2- 172.21.15.162:0/1594303749 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e140b0000 0x7f2e14060000 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f2e08002f30 tx=0x7f2e08006e80).ready entity=mon.1 client_cookie=9f80fc461c636f09 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:04.293 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.283+0000 7f2e137fe700 1 -- 172.21.15.162:0/1594303749 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e08009640 con 0x7f2e140b0000 2020-11-28T21:47:04.293 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.283+0000 7f2e137fe700 1 -- 172.21.15.162:0/1594303749 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e080097a0 con 0x7f2e140b0000 2020-11-28T21:47:04.294 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.283+0000 7f2e137fe700 1 -- 172.21.15.162:0/1594303749 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e08009b00 con 0x7f2e140b0000 2020-11-28T21:47:04.294 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.283+0000 7f2e19bf6700 1 -- 172.21.15.162:0/1594303749 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e140b0000 msgr2=0x7f2e14060000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:04.294 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.283+0000 7f2e19bf6700 1 --2- 172.21.15.162:0/1594303749 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e140b0000 0x7f2e14060000 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f2e08002f30 tx=0x7f2e08006e80).stop 2020-11-28T21:47:04.294 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.287+0000 7f2e19bf6700 1 -- 172.21.15.162:0/1594303749 shutdown_connections 2020-11-28T21:47:04.295 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.287+0000 7f2e19bf6700 1 --2- 172.21.15.162:0/1594303749 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2e140604f0 0x7f2e14060980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:04.295 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.287+0000 7f2e19bf6700 1 --2- 172.21.15.162:0/1594303749 >> v2:172.21.15.118:3300/0 conn(0x7f2e14098b30 0x7f2e140994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:04.295 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.287+0000 7f2e19bf6700 1 --2- 172.21.15.162:0/1594303749 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e140b0000 0x7f2e14060000 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:04.295 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.287+0000 7f2e19bf6700 1 -- 172.21.15.162:0/1594303749 >> 172.21.15.162:0/1594303749 conn(0x7f2e1405b970 msgr2=0x7f2e1405bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:04.295 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.287+0000 7f2e19bf6700 1 -- 172.21.15.162:0/1594303749 shutdown_connections 2020-11-28T21:47:04.296 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.287+0000 7f2e19bf6700 1 -- 172.21.15.162:0/1594303749 wait complete. 2020-11-28T21:47:04.296 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.287+0000 7f2e19bf6700 1 Processor -- start 2020-11-28T21:47:04.296 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.287+0000 7f2e19bf6700 1 -- start start 2020-11-28T21:47:04.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.287+0000 7f2e19bf6700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2e140604f0 0x7f2e140adb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:04.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.287+0000 7f2e19bf6700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2e14098b30 0x7f2e140657d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:04.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.287+0000 7f2e19bf6700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e140b0000 0x7f2e140aa950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:04.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.287+0000 7f2e19bf6700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e140637e0 con 0x7f2e140604f0 2020-11-28T21:47:04.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.287+0000 7f2e19bf6700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e14063a80 con 0x7f2e140b0000 2020-11-28T21:47:04.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.287+0000 7f2e19bf6700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e14065090 con 0x7f2e14098b30 2020-11-28T21:47:04.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.287+0000 7f2e193f5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e140b0000 0x7f2e140aa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:04.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.287+0000 7f2e193f5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e140b0000 0x7f2e140aa950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38420/0 (socket says 172.21.15.162:38420) 2020-11-28T21:47:04.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.287+0000 7f2e193f5700 1 -- 172.21.15.162:0/4207104917 learned_addr learned my addr 172.21.15.162:0/4207104917 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:04.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.287+0000 7f2e18bf4700 1 --2- 172.21.15.162:0/4207104917 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2e140604f0 0x7f2e140adb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:04.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.287+0000 7f2e13fff700 1 --2- 172.21.15.162:0/4207104917 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2e14098b30 0x7f2e140657d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:04.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.287+0000 7f2e193f5700 1 -- 172.21.15.162:0/4207104917 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2e14098b30 msgr2=0x7f2e140657d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:04.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.287+0000 7f2e193f5700 1 --2- 172.21.15.162:0/4207104917 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2e14098b30 0x7f2e140657d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:04.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.287+0000 7f2e193f5700 1 -- 172.21.15.162:0/4207104917 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2e140604f0 msgr2=0x7f2e140adb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:04.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.287+0000 7f2e193f5700 1 --2- 172.21.15.162:0/4207104917 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2e140604f0 0x7f2e140adb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:04.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.287+0000 7f2e193f5700 1 -- 172.21.15.162:0/4207104917 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e14065330 con 0x7f2e140b0000 2020-11-28T21:47:04.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.287+0000 7f2e193f5700 1 --2- 172.21.15.162:0/4207104917 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e140b0000 0x7f2e140aa950 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f2e0c006a50 tx=0x7f2e0c006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:04.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.287+0000 7f2e11ffb700 1 -- 172.21.15.162:0/4207104917 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e0c00ca10 con 0x7f2e140b0000 2020-11-28T21:47:04.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.287+0000 7f2e11ffb700 1 -- 172.21.15.162:0/4207104917 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e0c00be90 con 0x7f2e140b0000 2020-11-28T21:47:04.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.287+0000 7f2e19bf6700 1 -- 172.21.15.162:0/4207104917 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e140ae080 con 0x7f2e140b0000 2020-11-28T21:47:04.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.287+0000 7f2e11ffb700 1 -- 172.21.15.162:0/4207104917 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e0c00cd60 con 0x7f2e140b0000 2020-11-28T21:47:04.306 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.287+0000 7f2e19bf6700 1 -- 172.21.15.162:0/4207104917 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e140634e0 con 0x7f2e140b0000 2020-11-28T21:47:04.306 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.291+0000 7f2e11ffb700 1 -- 172.21.15.162:0/4207104917 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2e0c01c460 con 0x7f2e140b0000 2020-11-28T21:47:04.306 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.291+0000 7f2e19bf6700 1 -- 172.21.15.162:0/4207104917 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e140654c0 con 0x7f2e140b0000 2020-11-28T21:47:04.307 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.291+0000 7f2e11ffb700 1 --2- 172.21.15.162:0/4207104917 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2df40553b0 0x7f2df4057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:04.307 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.291+0000 7f2e18bf4700 1 --2- 172.21.15.162:0/4207104917 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2df40553b0 0x7f2df4057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:04.307 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.291+0000 7f2e11ffb700 1 -- 172.21.15.162:0/4207104917 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f2e0c06dfe0 con 0x7f2e140b0000 2020-11-28T21:47:04.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.291+0000 7f2e18bf4700 1 --2- 172.21.15.162:0/4207104917 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2df40553b0 0x7f2df4057870 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f2e0400b000 tx=0x7f2e040091a0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:04.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.299+0000 7f2e11ffb700 1 -- 172.21.15.162:0/4207104917 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f2e0c044500 con 0x7f2e140b0000 2020-11-28T21:47:04.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.487+0000 7f2e19bf6700 1 -- 172.21.15.162:0/4207104917 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f2e14057050 con 0x7f2e140b0000 2020-11-28T21:47:04.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.487+0000 7f2e11ffb700 1 -- 172.21.15.162:0/4207104917 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+5621 (secure 0 0 0) 0x7f2e0c047b20 con 0x7f2e140b0000 2020-11-28T21:47:04.491 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:47:04.492 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2020-11-28T21:47:04.492 INFO:teuthology.orchestra.run.smithi162.stdout: "auth_dump": [ 2020-11-28T21:47:04.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:47:04.493 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mds.a", 2020-11-28T21:47:04.493 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfWxUpKxAAIUDwQRKwz8H4Qk9qE356Bg==", 2020-11-28T21:47:04.494 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:47:04.494 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:47:04.494 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:47:04.495 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:47:04.495 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:47:04.496 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:47:04.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:47:04.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:47:04.498 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mds.b", 2020-11-28T21:47:04.499 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfKY2zORAAaO65etMZiYcaTDNX0nsEPw==", 2020-11-28T21:47:04.499 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:47:04.500 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:47:04.500 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:47:04.500 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:47:04.501 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:47:04.501 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:47:04.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:47:04.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:47:04.502 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mds.c", 2020-11-28T21:47:04.503 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfdQbRMxAAWlcjesxTZzZnvKCQneSjjQ==", 2020-11-28T21:47:04.503 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:47:04.504 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:47:04.504 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:47:04.504 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:47:04.505 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:47:04.505 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:47:04.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:47:04.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:47:04.506 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.0", 2020-11-28T21:47:04.507 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQADxcJfF3WLLxAAxsB5YPjH1yjv/vhlRbijqw==", 2020-11-28T21:47:04.507 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:47:04.508 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:47:04.508 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:47:04.508 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:47:04.509 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:47:04.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:47:04.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:47:04.510 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.1", 2020-11-28T21:47:04.510 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAGxcJfKYbBFhAA1x4DzFtkQQv6uWLIdOi1qQ==", 2020-11-28T21:47:04.511 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:47:04.511 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:47:04.512 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:47:04.512 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:47:04.512 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:47:04.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:47:04.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:47:04.513 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.2", 2020-11-28T21:47:04.514 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAIxcJfvcYUOhAASENjM0HfAxxyFe13mmE36Q==", 2020-11-28T21:47:04.514 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:47:04.515 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:47:04.515 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:47:04.515 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:47:04.516 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:47:04.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:47:04.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:47:04.517 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.3", 2020-11-28T21:47:04.517 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQALxcJfc/OHIRAAFYx3s8Ssttt15hjoH04tow==", 2020-11-28T21:47:04.518 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:47:04.518 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:47:04.519 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:47:04.519 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:47:04.520 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:47:04.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:47:04.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:47:04.521 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.4", 2020-11-28T21:47:04.521 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAPxcJffGWqCRAAb8CCN5oUPkdDh6BJH5HEnw==", 2020-11-28T21:47:04.522 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:47:04.522 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:47:04.523 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:47:04.523 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:47:04.524 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:47:04.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:47:04.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:47:04.525 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.5", 2020-11-28T21:47:04.525 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQARxcJfo8GKKRAAfbsCDb4L2wULiKmHyzu7/A==", 2020-11-28T21:47:04.526 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:47:04.526 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:47:04.527 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:47:04.527 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:47:04.527 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:47:04.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:47:04.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:47:04.529 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.6", 2020-11-28T21:47:04.529 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAUxcJf8Y2MEBAAqQX6rX3LK0zodlI2wK4JrA==", 2020-11-28T21:47:04.529 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:47:04.530 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:47:04.530 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:47:04.531 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:47:04.531 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:47:04.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:47:04.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:47:04.533 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.7", 2020-11-28T21:47:04.533 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAWxcJfCLBdMhAAf8QixQMQDrfZOGWN8rHe5g==", 2020-11-28T21:47:04.533 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:47:04.534 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:47:04.534 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:47:04.535 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:47:04.535 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:47:04.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:47:04.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:47:04.536 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "client.0", 2020-11-28T21:47:04.537 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQABxcJfS9vDAxAAUk4FU8FVt4if/CRH5ZisOA==", 2020-11-28T21:47:04.537 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:47:04.538 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:47:04.538 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow r", 2020-11-28T21:47:04.539 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow rw", 2020-11-28T21:47:04.539 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow rwx" 2020-11-28T21:47:04.539 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:47:04.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:47:04.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:47:04.541 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "client.1", 2020-11-28T21:47:04.541 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQABxcJfuCU8BhAAXUSoSK3D0XRo58uHNzcjXg==", 2020-11-28T21:47:04.542 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:47:04.542 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:47:04.543 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow r", 2020-11-28T21:47:04.544 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow rw", 2020-11-28T21:47:04.544 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow rwx" 2020-11-28T21:47:04.544 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:47:04.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:47:04.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:47:04.546 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "client.admin", 2020-11-28T21:47:04.547 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfdlTWBxAApzAxlVjzdnETQF//Vnsg5w==", 2020-11-28T21:47:04.547 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:47:04.547 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow *", 2020-11-28T21:47:04.547 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:47:04.548 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:47:04.548 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:47:04.549 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:47:04.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:47:04.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:47:04.549 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "client.bootstrap-mds", 2020-11-28T21:47:04.550 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAjxcJf2adNKxAAgbvCyetga3AX5xF4Jpmp3A==", 2020-11-28T21:47:04.550 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:47:04.551 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow profile bootstrap-mds" 2020-11-28T21:47:04.552 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:47:04.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:47:04.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:47:04.553 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "client.bootstrap-mgr", 2020-11-28T21:47:04.553 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAjxcJfhcRNKxAAfHgBXIompprMaQCtB9bV4g==", 2020-11-28T21:47:04.554 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:47:04.554 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow profile bootstrap-mgr" 2020-11-28T21:47:04.554 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:47:04.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:47:04.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:47:04.556 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "client.bootstrap-osd", 2020-11-28T21:47:04.556 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAjxcJfn9xNKxAAWcnDi2inaGsZ9wnSWUlhpA==", 2020-11-28T21:47:04.556 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:47:04.557 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow profile bootstrap-osd" 2020-11-28T21:47:04.557 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:47:04.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:47:04.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:47:04.558 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "client.bootstrap-rbd", 2020-11-28T21:47:04.559 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAjxcJfC/RNKxAArG2jK2OWIQa6nayaQVdZtg==", 2020-11-28T21:47:04.559 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:47:04.560 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow profile bootstrap-rbd" 2020-11-28T21:47:04.560 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:47:04.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:47:04.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:47:04.561 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "client.bootstrap-rbd-mirror", 2020-11-28T21:47:04.562 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAjxcJfqQ9OKxAAD2xqqM7aiVs1b56jIxWWAA==", 2020-11-28T21:47:04.563 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:47:04.563 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2020-11-28T21:47:04.563 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:47:04.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:47:04.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:47:04.564 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "client.bootstrap-rgw", 2020-11-28T21:47:04.564 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAjxcJf9SdOKxAAAI2pTn4ZFMiMthktZbkI4Q==", 2020-11-28T21:47:04.564 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:47:04.564 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow profile bootstrap-rgw" 2020-11-28T21:47:04.565 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:47:04.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:47:04.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:47:04.565 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mgr.x", 2020-11-28T21:47:04.565 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfZhdvKBAAT7zCCsD2OVCHjvddGLxQ/A==", 2020-11-28T21:47:04.566 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:47:04.566 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow *", 2020-11-28T21:47:04.566 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow profile mgr", 2020-11-28T21:47:04.566 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:47:04.566 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:47:04.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:47:04.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:47:04.567 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mgr.y", 2020-11-28T21:47:04.567 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfs8fvJRAAP0Atp074EeMHiTtpCCVHuA==", 2020-11-28T21:47:04.567 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:47:04.568 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow *", 2020-11-28T21:47:04.568 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow profile mgr", 2020-11-28T21:47:04.568 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:47:04.568 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:47:04.568 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:47:04.569 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:47:04.569 INFO:teuthology.orchestra.run.smithi162.stdout:} 2020-11-28T21:47:04.571 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.491+0000 7f2e19bf6700 1 -- 172.21.15.162:0/4207104917 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2df40553b0 msgr2=0x7f2df4057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:04.571 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.491+0000 7f2e19bf6700 1 --2- 172.21.15.162:0/4207104917 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2df40553b0 0x7f2df4057870 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f2e0400b000 tx=0x7f2e040091a0).stop 2020-11-28T21:47:04.571 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.491+0000 7f2e19bf6700 1 -- 172.21.15.162:0/4207104917 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e140b0000 msgr2=0x7f2e140aa950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:04.571 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.491+0000 7f2e19bf6700 1 --2- 172.21.15.162:0/4207104917 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e140b0000 0x7f2e140aa950 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f2e0c006a50 tx=0x7f2e0c006dc0).stop 2020-11-28T21:47:04.572 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.491+0000 7f2e19bf6700 1 -- 172.21.15.162:0/4207104917 shutdown_connections 2020-11-28T21:47:04.572 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.491+0000 7f2e19bf6700 1 --2- 172.21.15.162:0/4207104917 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2df40553b0 0x7f2df4057870 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:04.572 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.491+0000 7f2e19bf6700 1 --2- 172.21.15.162:0/4207104917 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e140b0000 0x7f2e140aa950 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:04.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.491+0000 7f2e19bf6700 1 --2- 172.21.15.162:0/4207104917 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2e140604f0 0x7f2e140adb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:04.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.491+0000 7f2e19bf6700 1 --2- 172.21.15.162:0/4207104917 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2e14098b30 0x7f2e140657d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:04.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.491+0000 7f2e19bf6700 1 -- 172.21.15.162:0/4207104917 >> 172.21.15.162:0/4207104917 conn(0x7f2e1405b970 msgr2=0x7f2e14149d70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:04.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.491+0000 7f2e19bf6700 1 -- 172.21.15.162:0/4207104917 shutdown_connections 2020-11-28T21:47:04.574 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.491+0000 7f2e19bf6700 1 -- 172.21.15.162:0/4207104917 wait complete. 2020-11-28T21:47:04.574 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth del client.bootstrap-mds 2020-11-28T21:47:04.694 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.691+0000 7f7c4a5b6700 1 Processor -- start 2020-11-28T21:47:04.695 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.691+0000 7f7c4a5b6700 1 -- start start 2020-11-28T21:47:04.699 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.691+0000 7f7c4a5b6700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7c440b2290 0x7f7c440ae4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:04.699 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.691+0000 7f7c4a5b6700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f7c440a8550 0x7f7c440ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:04.699 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.691+0000 7f7c4a5b6700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c440b2670 con 0x7f7c440af480 2020-11-28T21:47:04.699 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.691+0000 7f7c4a5b6700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c440afca0 con 0x7f7c440a8550 2020-11-28T21:47:04.700 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.691+0000 7f7c4a5b6700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c440ad740 con 0x7f7c440b2290 2020-11-28T21:47:04.700 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.695+0000 7f7c49db5700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f7c440af480 0x7f7c441417a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:45268/0 (socket says 172.21.15.162:45268) 2020-11-28T21:47:04.700 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.695+0000 7f7c49db5700 1 -- 172.21.15.162:0/2149763651 learned_addr learned my addr 172.21.15.162:0/2149763651 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:47:04.700 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.695+0000 7f7c48db3700 1 --2- 172.21.15.162:0/2149763651 >> v2:172.21.15.118:3300/0 conn(0x7f7c440a8550 0x7f7c440ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:04.701 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.695+0000 7f7c43fff700 1 -- 172.21.15.162:0/2149763651 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2610512570 0 0) 0x7f7c440b2670 con 0x7f7c440af480 2020-11-28T21:47:04.701 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.695+0000 7f7c43fff700 1 -- 172.21.15.162:0/2149763651 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c24003230 con 0x7f7c440af480 2020-11-28T21:47:04.701 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.695+0000 7f7c495b4700 1 --2- 172.21.15.162:0/2149763651 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7c440b2290 0x7f7c440ae4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:04.701 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.695+0000 7f7c43fff700 1 -- 172.21.15.162:0/2149763651 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2416789414 0 0) 0x7f7c24003230 con 0x7f7c440af480 2020-11-28T21:47:04.701 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.695+0000 7f7c43fff700 1 -- 172.21.15.162:0/2149763651 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7c440b2290 msgr2=0x7f7c440ae4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:04.702 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.695+0000 7f7c43fff700 1 --2- 172.21.15.162:0/2149763651 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7c440b2290 0x7f7c440ae4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:04.702 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.695+0000 7f7c43fff700 1 -- 172.21.15.162:0/2149763651 >> v2:172.21.15.118:3300/0 conn(0x7f7c440a8550 msgr2=0x7f7c440ae9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:04.702 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.695+0000 7f7c43fff700 1 --2- 172.21.15.162:0/2149763651 >> v2:172.21.15.118:3300/0 conn(0x7f7c440a8550 0x7f7c440ae9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:04.702 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.695+0000 7f7c43fff700 1 -- 172.21.15.162:0/2149763651 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c440afa00 con 0x7f7c440af480 2020-11-28T21:47:04.703 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.695+0000 7f7c43fff700 1 -- 172.21.15.162:0/2149763651 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f7c340016d0 con 0x7f7c440af480 2020-11-28T21:47:04.703 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.695+0000 7f7c43fff700 1 -- 172.21.15.162:0/2149763651 >> v1:172.21.15.162:6789/0 conn(0x7f7c440af480 legacy=0x7f7c441417a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:04.703 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.695+0000 7f7c43fff700 1 --2- 172.21.15.162:0/2149763651 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7c24003720 0x7f7c24003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:04.703 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.695+0000 7f7c43fff700 1 --2- 172.21.15.162:0/2149763651 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c240044d0 0x7f7c24004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:04.703 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.695+0000 7f7c43fff700 1 --2- 172.21.15.162:0/2149763651 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7c440b2290 0x7f7c24004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:04.704 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.695+0000 7f7c43fff700 1 -- 172.21.15.162:0/2149763651 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c440ad740 con 0x7f7c24003720 2020-11-28T21:47:04.704 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.695+0000 7f7c43fff700 1 -- 172.21.15.162:0/2149763651 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c24005380 con 0x7f7c240044d0 2020-11-28T21:47:04.704 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.695+0000 7f7c43fff700 1 -- 172.21.15.162:0/2149763651 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c240042e0 con 0x7f7c440b2290 2020-11-28T21:47:04.704 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.695+0000 7f7c48db3700 1 --2- 172.21.15.162:0/2149763651 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c240044d0 0x7f7c24004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:04.704 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.695+0000 7f7c49db5700 1 --2- 172.21.15.162:0/2149763651 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7c440b2290 0x7f7c24004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:04.705 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.695+0000 7f7c495b4700 1 --2- 172.21.15.162:0/2149763651 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7c24003720 0x7f7c24003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:04.705 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.695+0000 7f7c48db3700 1 -- 172.21.15.162:0/2149763651 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7c440b2290 msgr2=0x7f7c24004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:04.705 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.695+0000 7f7c48db3700 1 --2- 172.21.15.162:0/2149763651 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7c440b2290 0x7f7c24004e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:04.705 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.695+0000 7f7c48db3700 1 -- 172.21.15.162:0/2149763651 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7c24003720 msgr2=0x7f7c24003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:04.705 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.695+0000 7f7c48db3700 1 --2- 172.21.15.162:0/2149763651 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7c24003720 0x7f7c24003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:04.706 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.695+0000 7f7c48db3700 1 -- 172.21.15.162:0/2149763651 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c24003230 con 0x7f7c240044d0 2020-11-28T21:47:04.706 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.695+0000 7f7c495b4700 1 --2- 172.21.15.162:0/2149763651 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7c24003720 0x7f7c24003b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:47:04.706 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.695+0000 7f7c48db3700 1 --2- 172.21.15.162:0/2149763651 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c240044d0 0x7f7c24004950 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f7c3c00c000 tx=0x7f7c3c0091a0).ready entity=mon.1 client_cookie=1e935372ba4a5942 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:04.706 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.695+0000 7f7c43fff700 1 -- 172.21.15.162:0/2149763651 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c3c00eca0 con 0x7f7c240044d0 2020-11-28T21:47:04.707 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.695+0000 7f7c43fff700 1 -- 172.21.15.162:0/2149763651 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c3c00ee00 con 0x7f7c240044d0 2020-11-28T21:47:04.707 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.695+0000 7f7c43fff700 1 -- 172.21.15.162:0/2149763651 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c3c0125e0 con 0x7f7c240044d0 2020-11-28T21:47:04.707 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.695+0000 7f7c4a5b6700 1 -- 172.21.15.162:0/2149763651 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c240044d0 msgr2=0x7f7c24004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:04.707 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.695+0000 7f7c4a5b6700 1 --2- 172.21.15.162:0/2149763651 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c240044d0 0x7f7c24004950 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f7c3c00c000 tx=0x7f7c3c0091a0).stop 2020-11-28T21:47:04.707 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.695+0000 7f7c49db5700 1 -- 172.21.15.162:0/2149763651 reap_dead start 2020-11-28T21:47:04.708 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.695+0000 7f7c4a5b6700 1 -- 172.21.15.162:0/2149763651 shutdown_connections 2020-11-28T21:47:04.708 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.695+0000 7f7c4a5b6700 1 -- 172.21.15.162:0/2149763651 >> 172.21.15.162:0/2149763651 conn(0x7f7c44061320 msgr2=0x7f7c44062e50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:04.708 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.695+0000 7f7c4a5b6700 1 -- 172.21.15.162:0/2149763651 shutdown_connections 2020-11-28T21:47:04.708 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.695+0000 7f7c4a5b6700 1 -- 172.21.15.162:0/2149763651 wait complete. 2020-11-28T21:47:04.708 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.695+0000 7f7c4a5b6700 1 Processor -- start 2020-11-28T21:47:04.709 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.695+0000 7f7c4a5b6700 1 -- start start 2020-11-28T21:47:04.711 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.699+0000 7f7c4a5b6700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7c440a9650 0x7f7c44149fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:04.711 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.699+0000 7f7c4a5b6700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c44154fb0 0x7f7c44155410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:04.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.699+0000 7f7c4a5b6700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7c441489e0 0x7f7c44148e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:04.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.699+0000 7f7c4a5b6700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c441336c0 con 0x7f7c440a9650 2020-11-28T21:47:04.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.699+0000 7f7c4a5b6700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c440aadf0 con 0x7f7c44154fb0 2020-11-28T21:47:04.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.699+0000 7f7c4a5b6700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c440a9a30 con 0x7f7c441489e0 2020-11-28T21:47:04.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.699+0000 7f7c48db3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c44154fb0 0x7f7c44155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:04.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.699+0000 7f7c48db3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c44154fb0 0x7f7c44155410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38438/0 (socket says 172.21.15.162:38438) 2020-11-28T21:47:04.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.699+0000 7f7c48db3700 1 -- 172.21.15.162:0/3375692012 learned_addr learned my addr 172.21.15.162:0/3375692012 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:04.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.699+0000 7f7c49db5700 1 --2- 172.21.15.162:0/3375692012 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7c441489e0 0x7f7c44148e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:04.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.699+0000 7f7c495b4700 1 --2- 172.21.15.162:0/3375692012 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7c440a9650 0x7f7c44149fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:04.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.699+0000 7f7c48db3700 1 -- 172.21.15.162:0/3375692012 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7c441489e0 msgr2=0x7f7c44148e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:04.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.699+0000 7f7c48db3700 1 --2- 172.21.15.162:0/3375692012 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7c441489e0 0x7f7c44148e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:04.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.699+0000 7f7c48db3700 1 -- 172.21.15.162:0/3375692012 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7c440a9650 msgr2=0x7f7c44149fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:04.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.699+0000 7f7c48db3700 1 --2- 172.21.15.162:0/3375692012 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7c440a9650 0x7f7c44149fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:04.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.699+0000 7f7c48db3700 1 -- 172.21.15.162:0/3375692012 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c440ab020 con 0x7f7c44154fb0 2020-11-28T21:47:04.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.699+0000 7f7c495b4700 1 --2- 172.21.15.162:0/3375692012 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7c440a9650 0x7f7c44149fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:47:04.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.699+0000 7f7c48db3700 1 --2- 172.21.15.162:0/3375692012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c44154fb0 0x7f7c44155410 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f7c440ab8e0 tx=0x7f7c3c01a000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:04.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.699+0000 7f7c427fc700 1 -- 172.21.15.162:0/3375692012 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c3c0245e0 con 0x7f7c44154fb0 2020-11-28T21:47:04.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.699+0000 7f7c427fc700 1 -- 172.21.15.162:0/3375692012 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c3c024740 con 0x7f7c44154fb0 2020-11-28T21:47:04.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.699+0000 7f7c4a5b6700 1 -- 172.21.15.162:0/3375692012 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c440ab2c0 con 0x7f7c44154fb0 2020-11-28T21:47:04.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.699+0000 7f7c427fc700 1 -- 172.21.15.162:0/3375692012 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c3c024aa0 con 0x7f7c44154fb0 2020-11-28T21:47:04.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.699+0000 7f7c4a5b6700 1 -- 172.21.15.162:0/3375692012 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c44155900 con 0x7f7c44154fb0 2020-11-28T21:47:04.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.699+0000 7f7c4a5b6700 1 -- 172.21.15.162:0/3375692012 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c44149dc0 con 0x7f7c44154fb0 2020-11-28T21:47:04.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.699+0000 7f7c427fc700 1 -- 172.21.15.162:0/3375692012 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7c3c024c00 con 0x7f7c44154fb0 2020-11-28T21:47:04.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.699+0000 7f7c427fc700 1 --2- 172.21.15.162:0/3375692012 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7c280553f0 0x7f7c280578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:04.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.699+0000 7f7c427fc700 1 -- 172.21.15.162:0/3375692012 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f7c3c06f950 con 0x7f7c44154fb0 2020-11-28T21:47:04.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.699+0000 7f7c495b4700 1 --2- 172.21.15.162:0/3375692012 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7c280553f0 0x7f7c280578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:04.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.703+0000 7f7c495b4700 1 --2- 172.21.15.162:0/3375692012 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7c280553f0 0x7f7c280578b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f7c2c009000 tx=0x7f7c2c008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:04.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.707+0000 7f7c427fc700 1 -- 172.21.15.162:0/3375692012 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7c3c045ee0 con 0x7f7c44154fb0 2020-11-28T21:47:04.899 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.895+0000 7f7c4a5b6700 1 -- 172.21.15.162:0/3375692012 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f7c44057050 con 0x7f7c44154fb0 2020-11-28T21:47:04.908 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.907+0000 7f7c427fc700 1 -- 172.21.15.162:0/3375692012 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 updated v5) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7c3c0160a0 con 0x7f7c44154fb0 2020-11-28T21:47:04.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.907+0000 7f7c4a5b6700 1 -- 172.21.15.162:0/3375692012 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7c280553f0 msgr2=0x7f7c280578b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:04.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.907+0000 7f7c4a5b6700 1 --2- 172.21.15.162:0/3375692012 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7c280553f0 0x7f7c280578b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f7c2c009000 tx=0x7f7c2c008040).stop 2020-11-28T21:47:04.912 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.907+0000 7f7c4a5b6700 1 -- 172.21.15.162:0/3375692012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c44154fb0 msgr2=0x7f7c44155410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:04.912 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.907+0000 7f7c4a5b6700 1 --2- 172.21.15.162:0/3375692012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c44154fb0 0x7f7c44155410 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f7c440ab8e0 tx=0x7f7c3c01a000).stop 2020-11-28T21:47:04.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.907+0000 7f7c4a5b6700 1 -- 172.21.15.162:0/3375692012 shutdown_connections 2020-11-28T21:47:04.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.907+0000 7f7c4a5b6700 1 --2- 172.21.15.162:0/3375692012 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7c280553f0 0x7f7c280578b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:04.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.907+0000 7f7c4a5b6700 1 --2- 172.21.15.162:0/3375692012 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7c441489e0 0x7f7c44148e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:04.914 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.907+0000 7f7c4a5b6700 1 --2- 172.21.15.162:0/3375692012 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7c440a9650 0x7f7c44149fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:04.914 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.907+0000 7f7c4a5b6700 1 --2- 172.21.15.162:0/3375692012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c44154fb0 0x7f7c44155410 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:04.915 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.907+0000 7f7c4a5b6700 1 -- 172.21.15.162:0/3375692012 >> 172.21.15.162:0/3375692012 conn(0x7f7c44061320 msgr2=0x7f7c440b3100 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:04.915 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.907+0000 7f7c4a5b6700 1 -- 172.21.15.162:0/3375692012 shutdown_connections 2020-11-28T21:47:04.915 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:04.907+0000 7f7c4a5b6700 1 -- 172.21.15.162:0/3375692012 wait complete. 2020-11-28T21:47:04.916 INFO:teuthology.orchestra.run.smithi162.stderr:updated 2020-11-28T21:47:04.925 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth del client.bootstrap-mgr 2020-11-28T21:47:05.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.035+0000 7fbd60ec3700 1 Processor -- start 2020-11-28T21:47:05.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.035+0000 7fbd60ec3700 1 -- start start 2020-11-28T21:47:05.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.035+0000 7fbd60ec3700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fbd5c0b7320 0x7fbd5c13d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:05.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.035+0000 7fbd60ec3700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbd5c0b19f0 0x7fbd5c13dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:05.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.035+0000 7fbd60ec3700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd5c0b7700 con 0x7fbd5c0b1f00 2020-11-28T21:47:05.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.035+0000 7fbd60ec3700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd5c067b50 con 0x7fbd5c0b7320 2020-11-28T21:47:05.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.035+0000 7fbd60ec3700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd5c067df0 con 0x7fbd5c0b19f0 2020-11-28T21:47:05.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.035+0000 7fbd51d9b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbd5c0b19f0 0x7fbd5c13dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:05.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.035+0000 7fbd51d9b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbd5c0b19f0 0x7fbd5c13dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38444/0 (socket says 172.21.15.162:38444) 2020-11-28T21:47:05.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.035+0000 7fbd51d9b700 1 -- 172.21.15.162:0/662170644 learned_addr learned my addr 172.21.15.162:0/662170644 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:05.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.035+0000 7fbd5a59c700 1 --2- 172.21.15.162:0/662170644 >> v2:172.21.15.118:3300/0 conn(0x7fbd5c0b7320 0x7fbd5c13d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:05.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.035+0000 7fbd51d9b700 1 -- 172.21.15.162:0/662170644 >> v2:172.21.15.118:3300/0 conn(0x7fbd5c0b7320 msgr2=0x7fbd5c13d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:05.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.035+0000 7fbd51d9b700 1 --2- 172.21.15.162:0/662170644 >> v2:172.21.15.118:3300/0 conn(0x7fbd5c0b7320 0x7fbd5c13d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:05.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.035+0000 7fbd51d9b700 1 -- 172.21.15.162:0/662170644 >> v1:172.21.15.118:6789/0 conn(0x7fbd5c0b1f00 legacy=0x7fbd5c0b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:05.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.035+0000 7fbd51d9b700 1 -- 172.21.15.162:0/662170644 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd5c065c80 con 0x7fbd5c0b19f0 2020-11-28T21:47:05.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.035+0000 7fbd51d9b700 1 --2- 172.21.15.162:0/662170644 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd5c0b19f0 0x7fbd5c13dec0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7fbd44002f30 tx=0x7fbd44006e80).ready entity=mon.1 client_cookie=9546e946efe12830 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:05.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.035+0000 7fbd59d9b700 1 -- 172.21.15.162:0/662170644 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd440097f0 con 0x7fbd5c0b19f0 2020-11-28T21:47:05.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.035+0000 7fbd59d9b700 1 -- 172.21.15.162:0/662170644 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd44009950 con 0x7fbd5c0b19f0 2020-11-28T21:47:05.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.035+0000 7fbd59d9b700 1 -- 172.21.15.162:0/662170644 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd44009cf0 con 0x7fbd5c0b19f0 2020-11-28T21:47:05.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.035+0000 7fbd60ec3700 1 -- 172.21.15.162:0/662170644 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd5c0b19f0 msgr2=0x7fbd5c13dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:05.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.035+0000 7fbd60ec3700 1 --2- 172.21.15.162:0/662170644 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd5c0b19f0 0x7fbd5c13dec0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7fbd44002f30 tx=0x7fbd44006e80).stop 2020-11-28T21:47:05.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.035+0000 7fbd60ec3700 1 -- 172.21.15.162:0/662170644 shutdown_connections 2020-11-28T21:47:05.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.035+0000 7fbd60ec3700 1 --2- 172.21.15.162:0/662170644 >> v2:172.21.15.118:3300/0 conn(0x7fbd5c0b7320 0x7fbd5c13d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:05.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.035+0000 7fbd60ec3700 1 --2- 172.21.15.162:0/662170644 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd5c0b19f0 0x7fbd5c13dec0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:05.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.035+0000 7fbd60ec3700 1 -- 172.21.15.162:0/662170644 >> 172.21.15.162:0/662170644 conn(0x7fbd5c062830 msgr2=0x7fbd5c062c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:05.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.035+0000 7fbd60ec3700 1 -- 172.21.15.162:0/662170644 shutdown_connections 2020-11-28T21:47:05.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.035+0000 7fbd60ec3700 1 -- 172.21.15.162:0/662170644 wait complete. 2020-11-28T21:47:05.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.039+0000 7fbd60ec3700 1 Processor -- start 2020-11-28T21:47:05.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.039+0000 7fbd60ec3700 1 -- start start 2020-11-28T21:47:05.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.039+0000 7fbd60ec3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd5c0b19f0 0x7fbd5c13c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:05.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.039+0000 7fbd60ec3700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbd5c0b1f00 0x7fbd5c0b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:05.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.039+0000 7fbd60ec3700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbd5c0b7320 0x7fbd5c14f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:05.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.039+0000 7fbd60ec3700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c150960 con 0x7fbd5c0b7320 2020-11-28T21:47:05.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.039+0000 7fbd60ec3700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c0b0c20 con 0x7fbd5c0b19f0 2020-11-28T21:47:05.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.039+0000 7fbd60ec3700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c1539b0 con 0x7fbd5c0b1f00 2020-11-28T21:47:05.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.039+0000 7fbd5a59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd5c0b19f0 0x7fbd5c13c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:05.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.039+0000 7fbd5a59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd5c0b19f0 0x7fbd5c13c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38450/0 (socket says 172.21.15.162:38450) 2020-11-28T21:47:05.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.039+0000 7fbd5a59c700 1 -- 172.21.15.162:0/798100732 learned_addr learned my addr 172.21.15.162:0/798100732 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:05.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.039+0000 7fbd5a59c700 1 -- 172.21.15.162:0/798100732 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbd5c0b1f00 msgr2=0x7fbd5c0b02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:05.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.039+0000 7fbd5a59c700 1 --2- 172.21.15.162:0/798100732 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbd5c0b1f00 0x7fbd5c0b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-28T21:47:05.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.039+0000 7fbd5a59c700 1 -- 172.21.15.162:0/798100732 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbd5c0b7320 msgr2=0x7fbd5c14f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:05.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.039+0000 7fbd5ad9d700 1 --2- 172.21.15.162:0/798100732 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbd5c0b7320 0x7fbd5c14f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:05.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.039+0000 7fbd5a59c700 1 --2- 172.21.15.162:0/798100732 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbd5c0b7320 0x7fbd5c14f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:05.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.039+0000 7fbd5a59c700 1 -- 172.21.15.162:0/798100732 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd5c153c50 con 0x7fbd5c0b19f0 2020-11-28T21:47:05.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.039+0000 7fbd5ad9d700 1 --2- 172.21.15.162:0/798100732 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbd5c0b7320 0x7fbd5c14f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:47:05.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.039+0000 7fbd5a59c700 1 --2- 172.21.15.162:0/798100732 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd5c0b19f0 0x7fbd5c13c430 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7fbd4c00a000 tx=0x7fbd4c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:05.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.039+0000 7fbd53fff700 1 -- 172.21.15.162:0/798100732 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd4c011850 con 0x7fbd5c0b19f0 2020-11-28T21:47:05.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.043+0000 7fbd53fff700 1 -- 172.21.15.162:0/798100732 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd4c0119b0 con 0x7fbd5c0b19f0 2020-11-28T21:47:05.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.043+0000 7fbd60ec3700 1 -- 172.21.15.162:0/798100732 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd5c141c30 con 0x7fbd5c0b19f0 2020-11-28T21:47:05.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.043+0000 7fbd53fff700 1 -- 172.21.15.162:0/798100732 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd4c011d10 con 0x7fbd5c0b19f0 2020-11-28T21:47:05.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.043+0000 7fbd60ec3700 1 -- 172.21.15.162:0/798100732 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd5c0b0790 con 0x7fbd5c0b19f0 2020-11-28T21:47:05.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.043+0000 7fbd60ec3700 1 -- 172.21.15.162:0/798100732 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd5c057050 con 0x7fbd5c0b19f0 2020-11-28T21:47:05.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.043+0000 7fbd53fff700 1 -- 172.21.15.162:0/798100732 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fbd4c01c460 con 0x7fbd5c0b19f0 2020-11-28T21:47:05.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.043+0000 7fbd53fff700 1 --2- 172.21.15.162:0/798100732 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbd40055330 0x7fbd400577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:05.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.043+0000 7fbd53fff700 1 -- 172.21.15.162:0/798100732 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fbd4c06e1c0 con 0x7fbd5c0b19f0 2020-11-28T21:47:05.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.043+0000 7fbd51d9b700 1 --2- 172.21.15.162:0/798100732 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbd40055330 0x7fbd400577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:05.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.043+0000 7fbd51d9b700 1 --2- 172.21.15.162:0/798100732 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbd40055330 0x7fbd400577f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7fbd44002f30 tx=0x7fbd44015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:05.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.051+0000 7fbd53fff700 1 -- 172.21.15.162:0/798100732 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fbd4c0446f0 con 0x7fbd5c0b19f0 2020-11-28T21:47:05.130 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:47:05.134 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:47:05.136 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:47:05.139 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:47:05.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.243+0000 7fbd60ec3700 1 -- 172.21.15.162:0/798100732 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fbd5c1420d0 con 0x7fbd5c0b19f0 2020-11-28T21:47:05.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.251+0000 7fbd53fff700 1 -- 172.21.15.162:0/798100732 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 updated v6) v1 ==== 97+0+0 (secure 0 0 0) 0x7fbd5c1420d0 con 0x7fbd5c0b19f0 2020-11-28T21:47:05.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.255+0000 7fbd60ec3700 1 -- 172.21.15.162:0/798100732 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbd40055330 msgr2=0x7fbd400577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:05.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.255+0000 7fbd60ec3700 1 --2- 172.21.15.162:0/798100732 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbd40055330 0x7fbd400577f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7fbd44002f30 tx=0x7fbd44015000).stop 2020-11-28T21:47:05.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.255+0000 7fbd60ec3700 1 -- 172.21.15.162:0/798100732 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd5c0b19f0 msgr2=0x7fbd5c13c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:05.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.255+0000 7fbd60ec3700 1 --2- 172.21.15.162:0/798100732 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd5c0b19f0 0x7fbd5c13c430 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7fbd4c00a000 tx=0x7fbd4c008040).stop 2020-11-28T21:47:05.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.255+0000 7fbd60ec3700 1 -- 172.21.15.162:0/798100732 shutdown_connections 2020-11-28T21:47:05.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.255+0000 7fbd60ec3700 1 --2- 172.21.15.162:0/798100732 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbd40055330 0x7fbd400577f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:05.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.255+0000 7fbd60ec3700 1 --2- 172.21.15.162:0/798100732 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbd5c0b7320 0x7fbd5c14f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:05.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.255+0000 7fbd60ec3700 1 --2- 172.21.15.162:0/798100732 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd5c0b19f0 0x7fbd5c13c430 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:05.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.255+0000 7fbd60ec3700 1 --2- 172.21.15.162:0/798100732 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbd5c0b1f00 0x7fbd5c0b02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-28T21:47:05.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.255+0000 7fbd60ec3700 1 -- 172.21.15.162:0/798100732 >> 172.21.15.162:0/798100732 conn(0x7fbd5c062830 msgr2=0x7fbd5c0b4180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:05.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.255+0000 7fbd60ec3700 1 -- 172.21.15.162:0/798100732 shutdown_connections 2020-11-28T21:47:05.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.255+0000 7fbd60ec3700 1 -- 172.21.15.162:0/798100732 wait complete. 2020-11-28T21:47:05.260 INFO:teuthology.orchestra.run.smithi162.stderr:updated 2020-11-28T21:47:05.270 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth del client.bootstrap-osd 2020-11-28T21:47:05.432 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.431+0000 7f94e89a9700 1 Processor -- start 2020-11-28T21:47:05.433 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.431+0000 7f94e89a9700 1 -- start start 2020-11-28T21:47:05.435 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.431+0000 7f94e89a9700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f94e40611e0 0x7f94e40615c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:05.435 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.431+0000 7f94e89a9700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f94e40b10f0 0x7f94e40b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:05.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.431+0000 7f94e89a9700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94e40b1790 con 0x7f94e40b4920 2020-11-28T21:47:05.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.431+0000 7f94e89a9700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94e40b1580 con 0x7f94e40b10f0 2020-11-28T21:47:05.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.431+0000 7f94e89a9700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94e40b1a30 con 0x7f94e40611e0 2020-11-28T21:47:05.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.431+0000 7f94e2d9d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f94e40611e0 0x7f94e40615c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:05.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.431+0000 7f94e2d9d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f94e40611e0 0x7f94e40615c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38458/0 (socket says 172.21.15.162:38458) 2020-11-28T21:47:05.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.431+0000 7f94e2d9d700 1 -- 172.21.15.162:0/3973360419 learned_addr learned my addr 172.21.15.162:0/3973360419 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:05.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.431+0000 7f94da59c700 1 --2- 172.21.15.162:0/3973360419 >> v2:172.21.15.118:3300/0 conn(0x7f94e40b10f0 0x7f94e40b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:05.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.431+0000 7f94e2d9d700 1 -- 172.21.15.162:0/3973360419 >> v2:172.21.15.118:3300/0 conn(0x7f94e40b10f0 msgr2=0x7f94e40b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:05.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.431+0000 7f94e2d9d700 1 --2- 172.21.15.162:0/3973360419 >> v2:172.21.15.118:3300/0 conn(0x7f94e40b10f0 0x7f94e40b4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:05.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.431+0000 7f94e2d9d700 1 -- 172.21.15.162:0/3973360419 >> v1:172.21.15.162:6789/0 conn(0x7f94e40b4920 legacy=0x7f94e413ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:05.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.435+0000 7f94e2d9d700 1 -- 172.21.15.162:0/3973360419 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e40b3f90 con 0x7f94e40611e0 2020-11-28T21:47:05.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.435+0000 7f94e2d9d700 1 --2- 172.21.15.162:0/3973360419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94e40611e0 0x7f94e40615c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f94d4002f30 tx=0x7f94d4006f10).ready entity=mon.1 client_cookie=b2cb65da43ed0540 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:05.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.435+0000 7f94e259c700 1 -- 172.21.15.162:0/3973360419 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94d400cc20 con 0x7f94e40611e0 2020-11-28T21:47:05.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.435+0000 7f94e259c700 1 -- 172.21.15.162:0/3973360419 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94d400cd80 con 0x7f94e40611e0 2020-11-28T21:47:05.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.435+0000 7f94e259c700 1 -- 172.21.15.162:0/3973360419 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94d4010600 con 0x7f94e40611e0 2020-11-28T21:47:05.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.435+0000 7f94e89a9700 1 -- 172.21.15.162:0/3973360419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94e40611e0 msgr2=0x7f94e40615c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:05.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.435+0000 7f94e89a9700 1 --2- 172.21.15.162:0/3973360419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94e40611e0 0x7f94e40615c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f94d4002f30 tx=0x7f94d4006f10).stop 2020-11-28T21:47:05.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.435+0000 7f94e89a9700 1 -- 172.21.15.162:0/3973360419 shutdown_connections 2020-11-28T21:47:05.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.435+0000 7f94e89a9700 1 --2- 172.21.15.162:0/3973360419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94e40611e0 0x7f94e40615c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:05.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.435+0000 7f94e89a9700 1 --2- 172.21.15.162:0/3973360419 >> v2:172.21.15.118:3300/0 conn(0x7f94e40b10f0 0x7f94e40b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:05.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.435+0000 7f94e89a9700 1 -- 172.21.15.162:0/3973360419 >> 172.21.15.162:0/3973360419 conn(0x7f94e40b54f0 msgr2=0x7f94e40b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:05.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.435+0000 7f94e89a9700 1 -- 172.21.15.162:0/3973360419 shutdown_connections 2020-11-28T21:47:05.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.435+0000 7f94e89a9700 1 -- 172.21.15.162:0/3973360419 wait complete. 2020-11-28T21:47:05.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.435+0000 7f94e89a9700 1 Processor -- start 2020-11-28T21:47:05.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.435+0000 7f94e89a9700 1 -- start start 2020-11-28T21:47:05.444 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.439+0000 7f94e89a9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f94e40611e0 0x7f94e4066d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:05.444 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.439+0000 7f94e89a9700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f94e40b10f0 0x7f94e4150640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:05.444 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.439+0000 7f94e89a9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94e40b4920 0x7f94e4150b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:05.444 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.439+0000 7f94e89a9700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e4151020 con 0x7f94e40611e0 2020-11-28T21:47:05.445 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.439+0000 7f94e89a9700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e40aa880 con 0x7f94e40b4920 2020-11-28T21:47:05.445 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.439+0000 7f94e89a9700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94e40aaab0 con 0x7f94e40b10f0 2020-11-28T21:47:05.445 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.439+0000 7f94e359e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94e40b4920 0x7f94e4150b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:05.445 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.439+0000 7f94e359e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94e40b4920 0x7f94e4150b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38468/0 (socket says 172.21.15.162:38468) 2020-11-28T21:47:05.445 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.439+0000 7f94e359e700 1 -- 172.21.15.162:0/3281834672 learned_addr learned my addr 172.21.15.162:0/3281834672 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:05.446 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.439+0000 7f94e2d9d700 1 --2- 172.21.15.162:0/3281834672 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f94e40611e0 0x7f94e4066d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:05.446 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.439+0000 7f94da59c700 1 --2- 172.21.15.162:0/3281834672 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f94e40b10f0 0x7f94e4150640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:05.446 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.439+0000 7f94e359e700 1 -- 172.21.15.162:0/3281834672 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f94e40b10f0 msgr2=0x7f94e4150640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:05.446 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.439+0000 7f94e359e700 1 --2- 172.21.15.162:0/3281834672 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f94e40b10f0 0x7f94e4150640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:05.447 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.439+0000 7f94e359e700 1 -- 172.21.15.162:0/3281834672 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f94e40611e0 msgr2=0x7f94e4066d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:05.447 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.439+0000 7f94e359e700 1 --2- 172.21.15.162:0/3281834672 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f94e40611e0 0x7f94e4066d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:05.447 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.439+0000 7f94e359e700 1 -- 172.21.15.162:0/3281834672 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e40aad50 con 0x7f94e40b4920 2020-11-28T21:47:05.447 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.439+0000 7f94e2d9d700 1 --2- 172.21.15.162:0/3281834672 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f94e40611e0 0x7f94e4066d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:47:05.448 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.439+0000 7f94e359e700 1 --2- 172.21.15.162:0/3281834672 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94e40b4920 0x7f94e4150b30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f94e405d0f0 tx=0x7f94dc007f70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:05.448 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.439+0000 7f94e0d99700 1 -- 172.21.15.162:0/3281834672 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94dc003080 con 0x7f94e40b4920 2020-11-28T21:47:05.448 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.439+0000 7f94e0d99700 1 -- 172.21.15.162:0/3281834672 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94dc00a770 con 0x7f94e40b4920 2020-11-28T21:47:05.448 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.439+0000 7f94e89a9700 1 -- 172.21.15.162:0/3281834672 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94e40aaff0 con 0x7f94e40b4920 2020-11-28T21:47:05.448 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.439+0000 7f94e0d99700 1 -- 172.21.15.162:0/3281834672 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94dc00aad0 con 0x7f94e40b4920 2020-11-28T21:47:05.449 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.439+0000 7f94e89a9700 1 -- 172.21.15.162:0/3281834672 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94e40ab490 con 0x7f94e40b4920 2020-11-28T21:47:05.449 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.439+0000 7f94e0d99700 1 -- 172.21.15.162:0/3281834672 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f94dc019030 con 0x7f94e40b4920 2020-11-28T21:47:05.449 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.439+0000 7f94e89a9700 1 -- 172.21.15.162:0/3281834672 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94e40ab180 con 0x7f94e40b4920 2020-11-28T21:47:05.449 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.443+0000 7f94e0d99700 1 --2- 172.21.15.162:0/3281834672 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f94c8055440 0x7f94c8057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:05.449 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.443+0000 7f94e2d9d700 1 --2- 172.21.15.162:0/3281834672 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f94c8055440 0x7f94c8057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:05.450 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.443+0000 7f94e0d99700 1 -- 172.21.15.162:0/3281834672 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f94dc06d2b0 con 0x7f94e40b4920 2020-11-28T21:47:05.450 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.443+0000 7f94e2d9d700 1 --2- 172.21.15.162:0/3281834672 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f94c8055440 0x7f94c8057900 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f94d401b000 tx=0x7f94e4060a50).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:05.451 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.447+0000 7f94e0d99700 1 -- 172.21.15.162:0/3281834672 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f94dc0438e0 con 0x7f94e40b4920 2020-11-28T21:47:05.639 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.639+0000 7f94e89a9700 1 -- 172.21.15.162:0/3281834672 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f94e4057050 con 0x7f94e40b4920 2020-11-28T21:47:05.648 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.647+0000 7f94e0d99700 1 -- 172.21.15.162:0/3281834672 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 updated v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7f94dc046f00 con 0x7f94e40b4920 2020-11-28T21:47:05.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.647+0000 7f94e89a9700 1 -- 172.21.15.162:0/3281834672 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f94c8055440 msgr2=0x7f94c8057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:05.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.647+0000 7f94e89a9700 1 --2- 172.21.15.162:0/3281834672 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f94c8055440 0x7f94c8057900 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f94d401b000 tx=0x7f94e4060a50).stop 2020-11-28T21:47:05.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.647+0000 7f94e89a9700 1 -- 172.21.15.162:0/3281834672 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94e40b4920 msgr2=0x7f94e4150b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:05.651 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.647+0000 7f94e89a9700 1 --2- 172.21.15.162:0/3281834672 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94e40b4920 0x7f94e4150b30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f94e405d0f0 tx=0x7f94dc007f70).stop 2020-11-28T21:47:05.651 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.647+0000 7f94e89a9700 1 -- 172.21.15.162:0/3281834672 shutdown_connections 2020-11-28T21:47:05.651 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.647+0000 7f94e89a9700 1 --2- 172.21.15.162:0/3281834672 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f94c8055440 0x7f94c8057900 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:05.651 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.647+0000 7f94e89a9700 1 --2- 172.21.15.162:0/3281834672 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94e40b4920 0x7f94e4150b30 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:05.652 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.647+0000 7f94e89a9700 1 --2- 172.21.15.162:0/3281834672 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f94e40611e0 0x7f94e4066d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:05.652 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.647+0000 7f94e89a9700 1 --2- 172.21.15.162:0/3281834672 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f94e40b10f0 0x7f94e4150640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:05.652 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.647+0000 7f94e89a9700 1 -- 172.21.15.162:0/3281834672 >> 172.21.15.162:0/3281834672 conn(0x7f94e40b54f0 msgr2=0x7f94e405c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:05.652 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.647+0000 7f94e89a9700 1 -- 172.21.15.162:0/3281834672 shutdown_connections 2020-11-28T21:47:05.652 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.647+0000 7f94e89a9700 1 -- 172.21.15.162:0/3281834672 wait complete. 2020-11-28T21:47:05.653 INFO:teuthology.orchestra.run.smithi162.stderr:updated 2020-11-28T21:47:05.663 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth del client.bootstrap-rbd 2020-11-28T21:47:05.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.771+0000 7fcd5d4a5700 1 Processor -- start 2020-11-28T21:47:05.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.771+0000 7fcd5d4a5700 1 -- start start 2020-11-28T21:47:05.776 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.771+0000 7fcd5d4a5700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcd580994f0 0x7fcd580b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:05.776 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.771+0000 7fcd5d4a5700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd58098f10 con 0x7fcd58098b30 2020-11-28T21:47:05.776 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.771+0000 7fcd5d4a5700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd580b0680 con 0x7fcd58060710 2020-11-28T21:47:05.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.771+0000 7fcd5d4a5700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd580b1030 con 0x7fcd580994f0 2020-11-28T21:47:05.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.771+0000 7fcd5cca4700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fcd58060710 0x7fcd580b1580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:45320/0 (socket says 172.21.15.162:45320) 2020-11-28T21:47:05.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.771+0000 7fcd5cca4700 1 -- 172.21.15.162:0/3691147606 learned_addr learned my addr 172.21.15.162:0/3691147606 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:47:05.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.771+0000 7fcd577fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcd580994f0 0x7fcd580b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:05.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.775+0000 7fcd56ffd700 1 -- 172.21.15.162:0/3691147606 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2455632420 0 0) 0x7fcd580b0680 con 0x7fcd58060710 2020-11-28T21:47:05.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.775+0000 7fcd56ffd700 1 -- 172.21.15.162:0/3691147606 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd3c003230 con 0x7fcd58060710 2020-11-28T21:47:05.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.775+0000 7fcd577fe700 1 -- 172.21.15.162:0/3691147606 >> v1:172.21.15.162:6789/0 conn(0x7fcd58060710 legacy=0x7fcd580b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:05.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.775+0000 7fcd577fe700 1 -- 172.21.15.162:0/3691147606 >> v1:172.21.15.118:6789/0 conn(0x7fcd58098b30 legacy=0x7fcd58060000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:05.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.775+0000 7fcd577fe700 1 -- 172.21.15.162:0/3691147606 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd580b0920 con 0x7fcd580994f0 2020-11-28T21:47:05.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.775+0000 7fcd577fe700 1 --2- 172.21.15.162:0/3691147606 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd580994f0 0x7fcd580b0000 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7fcd40002f30 tx=0x7fcd40006dc0).ready entity=mon.1 client_cookie=9745f6c487824903 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:05.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.775+0000 7fcd56ffd700 1 -- 172.21.15.162:0/3691147606 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd40009800 con 0x7fcd580994f0 2020-11-28T21:47:05.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.775+0000 7fcd56ffd700 1 -- 172.21.15.162:0/3691147606 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd40009960 con 0x7fcd580994f0 2020-11-28T21:47:05.780 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.775+0000 7fcd56ffd700 1 -- 172.21.15.162:0/3691147606 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd40009cc0 con 0x7fcd580994f0 2020-11-28T21:47:05.780 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.775+0000 7fcd5d4a5700 1 -- 172.21.15.162:0/3691147606 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd580994f0 msgr2=0x7fcd580b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:05.780 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.775+0000 7fcd5d4a5700 1 --2- 172.21.15.162:0/3691147606 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd580994f0 0x7fcd580b0000 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7fcd40002f30 tx=0x7fcd40006dc0).stop 2020-11-28T21:47:05.783 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.775+0000 7fcd5d4a5700 1 -- 172.21.15.162:0/3691147606 shutdown_connections 2020-11-28T21:47:05.783 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.775+0000 7fcd5d4a5700 1 --2- 172.21.15.162:0/3691147606 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd580994f0 0x7fcd580b0000 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:05.784 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.775+0000 7fcd5d4a5700 1 -- 172.21.15.162:0/3691147606 >> 172.21.15.162:0/3691147606 conn(0x7fcd5805b970 msgr2=0x7fcd5805bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:05.784 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.775+0000 7fcd5d4a5700 1 -- 172.21.15.162:0/3691147606 shutdown_connections 2020-11-28T21:47:05.784 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.775+0000 7fcd5d4a5700 1 -- 172.21.15.162:0/3691147606 wait complete. 2020-11-28T21:47:05.784 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.775+0000 7fcd5d4a5700 1 Processor -- start 2020-11-28T21:47:05.784 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.775+0000 7fcd5d4a5700 1 -- start start 2020-11-28T21:47:05.785 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.779+0000 7fcd5d4a5700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcd58060710 0x7fcd58066800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:05.785 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.779+0000 7fcd5d4a5700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcd58098b30 0x7fcd58155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:05.785 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.779+0000 7fcd5d4a5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd580994f0 0x7fcd58155f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:05.785 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.779+0000 7fcd5d4a5700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd58067040 con 0x7fcd58098b30 2020-11-28T21:47:05.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.779+0000 7fcd5d4a5700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd58064fb0 con 0x7fcd580994f0 2020-11-28T21:47:05.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.779+0000 7fcd5d4a5700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd58063b50 con 0x7fcd58060710 2020-11-28T21:47:05.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.779+0000 7fcd5cca4700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd580994f0 0x7fcd58155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:05.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.779+0000 7fcd5cca4700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd580994f0 0x7fcd58155f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38482/0 (socket says 172.21.15.162:38482) 2020-11-28T21:47:05.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.779+0000 7fcd5cca4700 1 -- 172.21.15.162:0/1743366540 learned_addr learned my addr 172.21.15.162:0/1743366540 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:05.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.779+0000 7fcd57fff700 1 --2- 172.21.15.162:0/1743366540 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcd58060710 0x7fcd58066800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:05.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.779+0000 7fcd577fe700 1 --2- 172.21.15.162:0/1743366540 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcd58098b30 0x7fcd58155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:05.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.779+0000 7fcd5cca4700 1 -- 172.21.15.162:0/1743366540 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcd58060710 msgr2=0x7fcd58066800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:05.788 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.779+0000 7fcd5cca4700 1 --2- 172.21.15.162:0/1743366540 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcd58060710 0x7fcd58066800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:05.788 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.779+0000 7fcd5cca4700 1 -- 172.21.15.162:0/1743366540 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcd58098b30 msgr2=0x7fcd58155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:05.788 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.779+0000 7fcd5cca4700 1 --2- 172.21.15.162:0/1743366540 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcd58098b30 0x7fcd58155a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:05.788 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.779+0000 7fcd5cca4700 1 -- 172.21.15.162:0/1743366540 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd58063df0 con 0x7fcd580994f0 2020-11-28T21:47:05.788 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.779+0000 7fcd57fff700 1 --2- 172.21.15.162:0/1743366540 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcd58060710 0x7fcd58066800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:47:05.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.779+0000 7fcd5cca4700 1 --2- 172.21.15.162:0/1743366540 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd580994f0 0x7fcd58155f30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7fcd500010d0 tx=0x7fcd500079f0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:05.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.779+0000 7fcd557fa700 1 -- 172.21.15.162:0/1743366540 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd500072e0 con 0x7fcd580994f0 2020-11-28T21:47:05.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.779+0000 7fcd557fa700 1 -- 172.21.15.162:0/1743366540 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd50007440 con 0x7fcd580994f0 2020-11-28T21:47:05.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.779+0000 7fcd557fa700 1 -- 172.21.15.162:0/1743366540 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd500057d0 con 0x7fcd580994f0 2020-11-28T21:47:05.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.779+0000 7fcd5d4a5700 1 -- 172.21.15.162:0/1743366540 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd580aeeb0 con 0x7fcd580994f0 2020-11-28T21:47:05.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.779+0000 7fcd5d4a5700 1 -- 172.21.15.162:0/1743366540 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd58066cf0 con 0x7fcd580994f0 2020-11-28T21:47:05.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.779+0000 7fcd557fa700 1 -- 172.21.15.162:0/1743366540 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fcd50010070 con 0x7fcd580994f0 2020-11-28T21:47:05.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.779+0000 7fcd5d4a5700 1 -- 172.21.15.162:0/1743366540 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd58064bf0 con 0x7fcd580994f0 2020-11-28T21:47:05.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.779+0000 7fcd557fa700 1 --2- 172.21.15.162:0/1743366540 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcd300553b0 0x7fcd30057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:05.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.779+0000 7fcd557fa700 1 -- 172.21.15.162:0/1743366540 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fcd5006d250 con 0x7fcd580994f0 2020-11-28T21:47:05.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.779+0000 7fcd57fff700 1 --2- 172.21.15.162:0/1743366540 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcd300553b0 0x7fcd30057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:05.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.779+0000 7fcd57fff700 1 --2- 172.21.15.162:0/1743366540 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcd300553b0 0x7fcd30057870 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fcd4800b000 tx=0x7fcd48009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:05.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.787+0000 7fcd557fa700 1 -- 172.21.15.162:0/1743366540 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fcd50043880 con 0x7fcd580994f0 2020-11-28T21:47:05.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.979+0000 7fcd5d4a5700 1 -- 172.21.15.162:0/1743366540 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fcd58057050 con 0x7fcd580994f0 2020-11-28T21:47:05.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.987+0000 7fcd557fa700 1 -- 172.21.15.162:0/1743366540 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 updated v8) v1 ==== 97+0+0 (secure 0 0 0) 0x7fcd50018070 con 0x7fcd580994f0 2020-11-28T21:47:05.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.987+0000 7fcd5d4a5700 1 -- 172.21.15.162:0/1743366540 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcd300553b0 msgr2=0x7fcd30057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:05.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.987+0000 7fcd5d4a5700 1 --2- 172.21.15.162:0/1743366540 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcd300553b0 0x7fcd30057870 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fcd4800b000 tx=0x7fcd48009040).stop 2020-11-28T21:47:05.992 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.987+0000 7fcd5d4a5700 1 -- 172.21.15.162:0/1743366540 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd580994f0 msgr2=0x7fcd58155f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:05.992 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.991+0000 7fcd5d4a5700 1 --2- 172.21.15.162:0/1743366540 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd580994f0 0x7fcd58155f30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7fcd500010d0 tx=0x7fcd500079f0).stop 2020-11-28T21:47:05.993 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.991+0000 7fcd5d4a5700 1 -- 172.21.15.162:0/1743366540 shutdown_connections 2020-11-28T21:47:05.993 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.991+0000 7fcd5d4a5700 1 --2- 172.21.15.162:0/1743366540 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcd300553b0 0x7fcd30057870 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:05.993 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.991+0000 7fcd5d4a5700 1 --2- 172.21.15.162:0/1743366540 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcd58060710 0x7fcd58066800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:05.993 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.991+0000 7fcd5d4a5700 1 --2- 172.21.15.162:0/1743366540 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd580994f0 0x7fcd58155f30 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:05.993 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.991+0000 7fcd5d4a5700 1 --2- 172.21.15.162:0/1743366540 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcd58098b30 0x7fcd58155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:05.994 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.991+0000 7fcd5d4a5700 1 -- 172.21.15.162:0/1743366540 >> 172.21.15.162:0/1743366540 conn(0x7fcd5805b970 msgr2=0x7fcd58149ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:05.994 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.991+0000 7fcd5d4a5700 1 -- 172.21.15.162:0/1743366540 shutdown_connections 2020-11-28T21:47:05.994 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:05.991+0000 7fcd5d4a5700 1 -- 172.21.15.162:0/1743366540 wait complete. 2020-11-28T21:47:05.995 INFO:teuthology.orchestra.run.smithi162.stderr:updated 2020-11-28T21:47:06.006 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth del client.bootstrap-rbd-mirror 2020-11-28T21:47:06.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.119+0000 7f7cac875700 1 Processor -- start 2020-11-28T21:47:06.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.119+0000 7f7cac875700 1 -- start start 2020-11-28T21:47:06.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.119+0000 7f7cac875700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f7ca8098b30 0x7f7ca80994f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:06.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.119+0000 7f7cac875700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7ca8060710 0x7f7ca80b1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:06.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.119+0000 7f7cac875700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ca8098f10 con 0x7f7ca80b0000 2020-11-28T21:47:06.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.119+0000 7f7cac875700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ca80b1030 con 0x7f7ca8098b30 2020-11-28T21:47:06.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.119+0000 7f7cac875700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ca80b0480 con 0x7f7ca8060710 2020-11-28T21:47:06.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.119+0000 7f7ca759e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7ca8060710 0x7f7ca80b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:06.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.119+0000 7f7ca759e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7ca8060710 0x7f7ca80b1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38488/0 (socket says 172.21.15.162:38488) 2020-11-28T21:47:06.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.119+0000 7f7ca759e700 1 -- 172.21.15.162:0/3175233155 learned_addr learned my addr 172.21.15.162:0/3175233155 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:06.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.119+0000 7f7ca6d9d700 1 --2- 172.21.15.162:0/3175233155 >> v2:172.21.15.118:3300/0 conn(0x7f7ca8098b30 0x7f7ca80994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:06.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.119+0000 7f7ca759e700 1 -- 172.21.15.162:0/3175233155 >> v2:172.21.15.118:3300/0 conn(0x7f7ca8098b30 msgr2=0x7f7ca80994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:06.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.119+0000 7f7ca759e700 1 --2- 172.21.15.162:0/3175233155 >> v2:172.21.15.118:3300/0 conn(0x7f7ca8098b30 0x7f7ca80994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:06.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.119+0000 7f7ca759e700 1 -- 172.21.15.162:0/3175233155 >> v1:172.21.15.118:6789/0 conn(0x7f7ca80b0000 legacy=0x7f7ca8060000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:06.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.119+0000 7f7ca759e700 1 -- 172.21.15.162:0/3175233155 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca80b0680 con 0x7f7ca8060710 2020-11-28T21:47:06.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.119+0000 7f7ca6d9d700 1 --2- 172.21.15.162:0/3175233155 >> v2:172.21.15.118:3300/0 conn(0x7f7ca8098b30 0x7f7ca80994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:47:06.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.119+0000 7f7ca759e700 1 --2- 172.21.15.162:0/3175233155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ca8060710 0x7f7ca80b1580 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f7ca8061f00 tx=0x7f7c98006e80).ready entity=mon.1 client_cookie=f74f7d2caa0c05d3 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:06.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.119+0000 7f7ca5d9b700 1 -- 172.21.15.162:0/3175233155 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c980097b0 con 0x7f7ca8060710 2020-11-28T21:47:06.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.119+0000 7f7ca5d9b700 1 -- 172.21.15.162:0/3175233155 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c98009910 con 0x7f7ca8060710 2020-11-28T21:47:06.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.119+0000 7f7ca5d9b700 1 -- 172.21.15.162:0/3175233155 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c98009a80 con 0x7f7ca8060710 2020-11-28T21:47:06.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.119+0000 7f7cac875700 1 -- 172.21.15.162:0/3175233155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ca8060710 msgr2=0x7f7ca80b1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:06.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.119+0000 7f7cac875700 1 --2- 172.21.15.162:0/3175233155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ca8060710 0x7f7ca80b1580 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f7ca8061f00 tx=0x7f7c98006e80).stop 2020-11-28T21:47:06.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.119+0000 7f7cac875700 1 -- 172.21.15.162:0/3175233155 shutdown_connections 2020-11-28T21:47:06.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.119+0000 7f7cac875700 1 --2- 172.21.15.162:0/3175233155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ca8060710 0x7f7ca80b1580 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:06.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.119+0000 7f7cac875700 1 --2- 172.21.15.162:0/3175233155 >> v2:172.21.15.118:3300/0 conn(0x7f7ca8098b30 0x7f7ca80994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:06.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.119+0000 7f7cac875700 1 -- 172.21.15.162:0/3175233155 >> 172.21.15.162:0/3175233155 conn(0x7f7ca805b970 msgr2=0x7f7ca805bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:06.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.119+0000 7f7cac875700 1 -- 172.21.15.162:0/3175233155 shutdown_connections 2020-11-28T21:47:06.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.119+0000 7f7cac875700 1 -- 172.21.15.162:0/3175233155 wait complete. 2020-11-28T21:47:06.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.123+0000 7f7cac875700 1 Processor -- start 2020-11-28T21:47:06.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.123+0000 7f7cac875700 1 -- start start 2020-11-28T21:47:06.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.123+0000 7f7cac875700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7ca8060710 0x7f7ca80aea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:06.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.123+0000 7f7cac875700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7ca8098b30 0x7f7ca80ad780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:06.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.123+0000 7f7cac875700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ca80b0000 0x7f7ca8155b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:06.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.123+0000 7f7cac875700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca80adf30 con 0x7f7ca8098b30 2020-11-28T21:47:06.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.123+0000 7f7cac875700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca8065800 con 0x7f7ca80b0000 2020-11-28T21:47:06.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.123+0000 7f7cac875700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca8065aa0 con 0x7f7ca8060710 2020-11-28T21:47:06.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.123+0000 7f7ca759e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ca80b0000 0x7f7ca8155b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:06.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.123+0000 7f7ca759e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ca80b0000 0x7f7ca8155b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38496/0 (socket says 172.21.15.162:38496) 2020-11-28T21:47:06.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.123+0000 7f7ca759e700 1 -- 172.21.15.162:0/2230036635 learned_addr learned my addr 172.21.15.162:0/2230036635 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:06.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.123+0000 7f7ca6d9d700 1 --2- 172.21.15.162:0/2230036635 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7ca8060710 0x7f7ca80aea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:06.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.123+0000 7f7ca659c700 1 --2- 172.21.15.162:0/2230036635 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7ca8098b30 0x7f7ca80ad780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:06.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.123+0000 7f7ca759e700 1 -- 172.21.15.162:0/2230036635 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7ca8060710 msgr2=0x7f7ca80aea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:06.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.123+0000 7f7ca759e700 1 --2- 172.21.15.162:0/2230036635 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7ca8060710 0x7f7ca80aea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:06.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.123+0000 7f7ca759e700 1 -- 172.21.15.162:0/2230036635 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7ca8098b30 msgr2=0x7f7ca80ad780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:06.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.123+0000 7f7ca759e700 1 --2- 172.21.15.162:0/2230036635 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7ca8098b30 0x7f7ca80ad780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:06.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.123+0000 7f7ca759e700 1 -- 172.21.15.162:0/2230036635 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca80adc70 con 0x7f7ca80b0000 2020-11-28T21:47:06.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.123+0000 7f7ca759e700 1 --2- 172.21.15.162:0/2230036635 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ca80b0000 0x7f7ca8155b80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f7ca8061f00 tx=0x7f7c98016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:06.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.123+0000 7f7c9ffff700 1 -- 172.21.15.162:0/2230036635 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c98011d50 con 0x7f7ca80b0000 2020-11-28T21:47:06.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.123+0000 7f7c9ffff700 1 -- 172.21.15.162:0/2230036635 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c98021440 con 0x7f7ca80b0000 2020-11-28T21:47:06.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.123+0000 7f7cac875700 1 -- 172.21.15.162:0/2230036635 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ca8065f40 con 0x7f7ca80b0000 2020-11-28T21:47:06.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.123+0000 7f7c9ffff700 1 -- 172.21.15.162:0/2230036635 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c980217a0 con 0x7f7ca80b0000 2020-11-28T21:47:06.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.123+0000 7f7cac875700 1 -- 172.21.15.162:0/2230036635 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca80a8760 con 0x7f7ca80b0000 2020-11-28T21:47:06.138 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.127+0000 7f7cac875700 1 -- 172.21.15.162:0/2230036635 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ca80ae0a0 con 0x7f7ca80b0000 2020-11-28T21:47:06.138 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.127+0000 7f7c9ffff700 1 -- 172.21.15.162:0/2230036635 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7c9800f070 con 0x7f7ca80b0000 2020-11-28T21:47:06.138 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.127+0000 7f7c9ffff700 1 --2- 172.21.15.162:0/2230036635 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7c8c055330 0x7f7c8c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:06.138 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.127+0000 7f7ca6d9d700 1 --2- 172.21.15.162:0/2230036635 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7c8c055330 0x7f7c8c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:06.139 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.127+0000 7f7c9ffff700 1 -- 172.21.15.162:0/2230036635 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f7c9806e010 con 0x7f7ca80b0000 2020-11-28T21:47:06.139 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.127+0000 7f7ca6d9d700 1 --2- 172.21.15.162:0/2230036635 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7c8c055330 0x7f7c8c0577f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f7ca000d000 tx=0x7f7ca0008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:06.139 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.135+0000 7f7c9ffff700 1 -- 172.21.15.162:0/2230036635 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7ca80ae0a0 con 0x7f7ca80b0000 2020-11-28T21:47:06.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.323+0000 7f7cac875700 1 -- 172.21.15.162:0/2230036635 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f7ca80ae0a0 con 0x7f7ca80b0000 2020-11-28T21:47:06.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.331+0000 7f7c9ffff700 1 -- 172.21.15.162:0/2230036635 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 updated v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7ca80ae0a0 con 0x7f7ca80b0000 2020-11-28T21:47:06.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.331+0000 7f7cac875700 1 -- 172.21.15.162:0/2230036635 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7c8c055330 msgr2=0x7f7c8c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:06.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.331+0000 7f7cac875700 1 --2- 172.21.15.162:0/2230036635 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7c8c055330 0x7f7c8c0577f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f7ca000d000 tx=0x7f7ca0008040).stop 2020-11-28T21:47:06.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.331+0000 7f7cac875700 1 -- 172.21.15.162:0/2230036635 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ca80b0000 msgr2=0x7f7ca8155b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:06.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.331+0000 7f7cac875700 1 --2- 172.21.15.162:0/2230036635 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ca80b0000 0x7f7ca8155b80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f7ca8061f00 tx=0x7f7c98016000).stop 2020-11-28T21:47:06.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.331+0000 7f7cac875700 1 -- 172.21.15.162:0/2230036635 shutdown_connections 2020-11-28T21:47:06.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.331+0000 7f7cac875700 1 --2- 172.21.15.162:0/2230036635 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7c8c055330 0x7f7c8c0577f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:06.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.331+0000 7f7cac875700 1 --2- 172.21.15.162:0/2230036635 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7ca8060710 0x7f7ca80aea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:06.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.331+0000 7f7cac875700 1 --2- 172.21.15.162:0/2230036635 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ca80b0000 0x7f7ca8155b80 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:06.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.331+0000 7f7cac875700 1 --2- 172.21.15.162:0/2230036635 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7ca8098b30 0x7f7ca80ad780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:06.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.331+0000 7f7cac875700 1 -- 172.21.15.162:0/2230036635 >> 172.21.15.162:0/2230036635 conn(0x7f7ca805b970 msgr2=0x7f7ca80b1450 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:06.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.331+0000 7f7cac875700 1 -- 172.21.15.162:0/2230036635 shutdown_connections 2020-11-28T21:47:06.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.331+0000 7f7cac875700 1 -- 172.21.15.162:0/2230036635 wait complete. 2020-11-28T21:47:06.338 INFO:teuthology.orchestra.run.smithi162.stderr:updated 2020-11-28T21:47:06.349 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth del client.bootstrap-rgw 2020-11-28T21:47:06.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.463+0000 7f2800034700 1 Processor -- start 2020-11-28T21:47:06.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.463+0000 7f2800034700 1 -- start start 2020-11-28T21:47:06.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.463+0000 7f2800034700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f27f80b11b0 0x7f27f8140fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:06.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.463+0000 7f2800034700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27f80b2870 con 0x7f27f80b1d70 2020-11-28T21:47:06.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.463+0000 7f2800034700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27f80b22e0 con 0x7f27f80b1990 2020-11-28T21:47:06.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.463+0000 7f2800034700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27f8146e20 con 0x7f27f80b11b0 2020-11-28T21:47:06.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.463+0000 7f27fddd0700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f27f80b1990 0x7f27f8140890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:45344/0 (socket says 172.21.15.162:45344) 2020-11-28T21:47:06.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.463+0000 7f27fddd0700 1 -- 172.21.15.162:0/1185271785 learned_addr learned my addr 172.21.15.162:0/1185271785 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:47:06.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.463+0000 7f27fd5cf700 1 --2- 172.21.15.162:0/1185271785 >> v2:172.21.15.162:3300/0 conn(0x7f27f80b11b0 0x7f27f8140fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:06.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.463+0000 7f27fcdce700 1 -- 172.21.15.162:0/1185271785 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 414131294 0 0) 0x7f27f80b22e0 con 0x7f27f80b1990 2020-11-28T21:47:06.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.463+0000 7f27fcdce700 1 -- 172.21.15.162:0/1185271785 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27dc003230 con 0x7f27f80b1990 2020-11-28T21:47:06.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.463+0000 7f27fd5cf700 1 -- 172.21.15.162:0/1185271785 >> v1:172.21.15.162:6789/0 conn(0x7f27f80b1990 legacy=0x7f27f8140890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:06.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.463+0000 7f27fd5cf700 1 -- 172.21.15.162:0/1185271785 >> v1:172.21.15.118:6789/0 conn(0x7f27f80b1d70 legacy=0x7f27f814a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:06.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.463+0000 7f27fd5cf700 1 -- 172.21.15.162:0/1185271785 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f80b0740 con 0x7f27f80b11b0 2020-11-28T21:47:06.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.463+0000 7f27fd5cf700 1 --2- 172.21.15.162:0/1185271785 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f27f80b11b0 0x7f27f8140fa0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f27e8002f30 tx=0x7f27e8006dc0).ready entity=mon.1 client_cookie=576be115795e70fc server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:06.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.463+0000 7f27fcdce700 1 -- 172.21.15.162:0/1185271785 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f27e8009800 con 0x7f27f80b11b0 2020-11-28T21:47:06.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.463+0000 7f27fcdce700 1 -- 172.21.15.162:0/1185271785 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e8009960 con 0x7f27f80b11b0 2020-11-28T21:47:06.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.463+0000 7f27fcdce700 1 -- 172.21.15.162:0/1185271785 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f27e8009cc0 con 0x7f27f80b11b0 2020-11-28T21:47:06.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.463+0000 7f2800034700 1 -- 172.21.15.162:0/1185271785 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f27f80b11b0 msgr2=0x7f27f8140fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:06.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.463+0000 7f2800034700 1 --2- 172.21.15.162:0/1185271785 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f27f80b11b0 0x7f27f8140fa0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f27e8002f30 tx=0x7f27e8006dc0).stop 2020-11-28T21:47:06.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.463+0000 7f2800034700 1 -- 172.21.15.162:0/1185271785 shutdown_connections 2020-11-28T21:47:06.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.463+0000 7f2800034700 1 --2- 172.21.15.162:0/1185271785 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f27f80b11b0 0x7f27f8140fa0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:06.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.463+0000 7f2800034700 1 -- 172.21.15.162:0/1185271785 >> 172.21.15.162:0/1185271785 conn(0x7f27f805b970 msgr2=0x7f27f805bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:06.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.463+0000 7f2800034700 1 -- 172.21.15.162:0/1185271785 shutdown_connections 2020-11-28T21:47:06.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.463+0000 7f2800034700 1 -- 172.21.15.162:0/1185271785 wait complete. 2020-11-28T21:47:06.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.467+0000 7f2800034700 1 Processor -- start 2020-11-28T21:47:06.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.467+0000 7f2800034700 1 -- start start 2020-11-28T21:47:06.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.467+0000 7f2800034700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f27f80b11b0 0x7f27f80b4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:06.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.467+0000 7f2800034700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f27f80b1990 0x7f27f814dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:06.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.467+0000 7f2800034700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f27f80b1d70 0x7f27f813f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:06.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.467+0000 7f2800034700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f814e0d0 con 0x7f27f80b1d70 2020-11-28T21:47:06.485 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.467+0000 7f2800034700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f814d200 con 0x7f27f80b11b0 2020-11-28T21:47:06.485 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.467+0000 7f2800034700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27f814d4a0 con 0x7f27f80b1990 2020-11-28T21:47:06.485 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.467+0000 7f27fddd0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f27f80b11b0 0x7f27f80b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:06.486 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.467+0000 7f27fddd0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f27f80b11b0 0x7f27f80b4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38508/0 (socket says 172.21.15.162:38508) 2020-11-28T21:47:06.486 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.467+0000 7f27fddd0700 1 -- 172.21.15.162:0/2562289984 learned_addr learned my addr 172.21.15.162:0/2562289984 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:06.486 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.467+0000 7f27fe5d1700 1 --2- 172.21.15.162:0/2562289984 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f27f80b1d70 0x7f27f813f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:06.487 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.467+0000 7f27fd5cf700 1 --2- 172.21.15.162:0/2562289984 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f27f80b1990 0x7f27f814dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:06.487 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.467+0000 7f27fddd0700 1 -- 172.21.15.162:0/2562289984 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f27f80b1990 msgr2=0x7f27f814dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:06.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.467+0000 7f27fddd0700 1 --2- 172.21.15.162:0/2562289984 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f27f80b1990 0x7f27f814dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:06.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.467+0000 7f27fddd0700 1 -- 172.21.15.162:0/2562289984 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f27f80b1d70 msgr2=0x7f27f813f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:06.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.467+0000 7f27fddd0700 1 --2- 172.21.15.162:0/2562289984 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f27f80b1d70 0x7f27f813f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:06.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.467+0000 7f27fddd0700 1 -- 172.21.15.162:0/2562289984 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f814d740 con 0x7f27f80b11b0 2020-11-28T21:47:06.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.467+0000 7f27fddd0700 1 --2- 172.21.15.162:0/2562289984 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f27f80b11b0 0x7f27f80b4290 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f27f40010d0 tx=0x7f27f4007a70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:06.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.467+0000 7f27eeffd700 1 -- 172.21.15.162:0/2562289984 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f27f40072e0 con 0x7f27f80b11b0 2020-11-28T21:47:06.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.467+0000 7f2800034700 1 -- 172.21.15.162:0/2562289984 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27f814e2e0 con 0x7f27f80b11b0 2020-11-28T21:47:06.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.467+0000 7f2800034700 1 -- 172.21.15.162:0/2562289984 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27f813fd00 con 0x7f27f80b11b0 2020-11-28T21:47:06.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.467+0000 7f27eeffd700 1 -- 172.21.15.162:0/2562289984 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f4007440 con 0x7f27f80b11b0 2020-11-28T21:47:06.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.471+0000 7f27eeffd700 1 -- 172.21.15.162:0/2562289984 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f27f40055c0 con 0x7f27f80b11b0 2020-11-28T21:47:06.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.471+0000 7f2800034700 1 -- 172.21.15.162:0/2562289984 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27d40033e0 con 0x7f27f80b11b0 2020-11-28T21:47:06.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.471+0000 7f27eeffd700 1 -- 172.21.15.162:0/2562289984 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f27f4010070 con 0x7f27f80b11b0 2020-11-28T21:47:06.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.471+0000 7f27eeffd700 1 --2- 172.21.15.162:0/2562289984 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f27e00553b0 0x7f27e0057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:06.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.471+0000 7f27eeffd700 1 -- 172.21.15.162:0/2562289984 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f27f406cfd0 con 0x7f27f80b11b0 2020-11-28T21:47:06.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.471+0000 7f27fd5cf700 1 --2- 172.21.15.162:0/2562289984 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f27e00553b0 0x7f27e0057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:06.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.471+0000 7f27fd5cf700 1 --2- 172.21.15.162:0/2562289984 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f27e00553b0 0x7f27e0057870 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f27e801a000 tx=0x7f27e8015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:06.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.479+0000 7f27eeffd700 1 -- 172.21.15.162:0/2562289984 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f27f4043560 con 0x7f27f80b11b0 2020-11-28T21:47:06.672 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.671+0000 7f2800034700 1 -- 172.21.15.162:0/2562289984 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f27d40022a0 con 0x7f27f80b11b0 2020-11-28T21:47:06.681 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.679+0000 7f27eeffd700 1 -- 172.21.15.162:0/2562289984 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 updated v10) v1 ==== 97+0+0 (secure 0 0 0) 0x7f27f4046b80 con 0x7f27f80b11b0 2020-11-28T21:47:06.684 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.683+0000 7f2800034700 1 -- 172.21.15.162:0/2562289984 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f27e00553b0 msgr2=0x7f27e0057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:06.685 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.683+0000 7f2800034700 1 --2- 172.21.15.162:0/2562289984 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f27e00553b0 0x7f27e0057870 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f27e801a000 tx=0x7f27e8015000).stop 2020-11-28T21:47:06.685 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.683+0000 7f2800034700 1 -- 172.21.15.162:0/2562289984 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f27f80b11b0 msgr2=0x7f27f80b4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:06.685 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.683+0000 7f2800034700 1 --2- 172.21.15.162:0/2562289984 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f27f80b11b0 0x7f27f80b4290 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f27f40010d0 tx=0x7f27f4007a70).stop 2020-11-28T21:47:06.685 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.683+0000 7f2800034700 1 -- 172.21.15.162:0/2562289984 shutdown_connections 2020-11-28T21:47:06.686 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.683+0000 7f2800034700 1 --2- 172.21.15.162:0/2562289984 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f27e00553b0 0x7f27e0057870 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:06.686 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.683+0000 7f2800034700 1 --2- 172.21.15.162:0/2562289984 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f27f80b1d70 0x7f27f813f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:06.686 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.683+0000 7f2800034700 1 --2- 172.21.15.162:0/2562289984 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f27f80b11b0 0x7f27f80b4290 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:06.686 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.683+0000 7f2800034700 1 --2- 172.21.15.162:0/2562289984 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f27f80b1990 0x7f27f814dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:06.686 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.683+0000 7f2800034700 1 -- 172.21.15.162:0/2562289984 >> 172.21.15.162:0/2562289984 conn(0x7f27f805b970 msgr2=0x7f27f8140660 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:06.687 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.683+0000 7f2800034700 1 -- 172.21.15.162:0/2562289984 shutdown_connections 2020-11-28T21:47:06.687 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.683+0000 7f2800034700 1 -- 172.21.15.162:0/2562289984 wait complete. 2020-11-28T21:47:06.687 INFO:teuthology.orchestra.run.smithi162.stderr:updated 2020-11-28T21:47:06.701 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:47:06.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.803+0000 7f014b59e700 1 Processor -- start 2020-11-28T21:47:06.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.803+0000 7f014b59e700 1 -- start start 2020-11-28T21:47:06.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.803+0000 7f014b59e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f014c13f6c0 0x7f014c0669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:06.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.803+0000 7f014b59e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f014c065050 con 0x7f014c064c70 2020-11-28T21:47:06.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.803+0000 7f014b59e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f014c13fc40 con 0x7f014c065750 2020-11-28T21:47:06.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.803+0000 7f014b59e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f014c13fee0 con 0x7f014c13f6c0 2020-11-28T21:47:06.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.803+0000 7f0149d9b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f014c13f6c0 0x7f014c0669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:06.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.803+0000 7f014ad9d700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f014c065750 0x7f014c13c260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:45362/0 (socket says 172.21.15.162:45362) 2020-11-28T21:47:06.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.803+0000 7f014ad9d700 1 -- 172.21.15.162:0/3675929428 learned_addr learned my addr 172.21.15.162:0/3675929428 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:47:06.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.807+0000 7f0149d9b700 1 -- 172.21.15.162:0/3675929428 >> v1:172.21.15.162:6789/0 conn(0x7f014c065750 legacy=0x7f014c13c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:06.810 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.807+0000 7f0149d9b700 1 -- 172.21.15.162:0/3675929428 >> v1:172.21.15.118:6789/0 conn(0x7f014c064c70 legacy=0x7f014c066290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:06.810 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.807+0000 7f014959a700 1 -- 172.21.15.162:0/3675929428 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1808663126 0 0) 0x7f014c065050 con 0x7f014c064c70 2020-11-28T21:47:06.810 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.807+0000 7f0149d9b700 1 -- 172.21.15.162:0/3675929428 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f014c140180 con 0x7f014c13f6c0 2020-11-28T21:47:06.810 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.807+0000 7f0149d9b700 1 --2- 172.21.15.162:0/3675929428 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f014c13f6c0 0x7f014c0669a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f013c001000 tx=0x7f0140006dc0).ready entity=mon.1 client_cookie=7d4de8fc96c098e9 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:06.811 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.807+0000 7f014959a700 1 -- 172.21.15.162:0/3675929428 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f014000fa10 con 0x7f014c13f6c0 2020-11-28T21:47:06.811 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.807+0000 7f014959a700 1 -- 172.21.15.162:0/3675929428 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f014000fb70 con 0x7f014c13f6c0 2020-11-28T21:47:06.811 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.807+0000 7f014959a700 1 -- 172.21.15.162:0/3675929428 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0140010470 con 0x7f014c13f6c0 2020-11-28T21:47:06.811 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.807+0000 7f014b59e700 1 -- 172.21.15.162:0/3675929428 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f014c13f6c0 msgr2=0x7f014c0669a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:06.811 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.807+0000 7f014b59e700 1 --2- 172.21.15.162:0/3675929428 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f014c13f6c0 0x7f014c0669a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f013c001000 tx=0x7f0140006dc0).stop 2020-11-28T21:47:06.812 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.807+0000 7f014b59e700 1 -- 172.21.15.162:0/3675929428 shutdown_connections 2020-11-28T21:47:06.812 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.807+0000 7f014b59e700 1 --2- 172.21.15.162:0/3675929428 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f014c13f6c0 0x7f014c0669a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:06.812 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.807+0000 7f014b59e700 1 -- 172.21.15.162:0/3675929428 >> 172.21.15.162:0/3675929428 conn(0x7f014c0b5da0 msgr2=0x7f014c0b6180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:06.812 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.807+0000 7f014b59e700 1 -- 172.21.15.162:0/3675929428 shutdown_connections 2020-11-28T21:47:06.812 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.807+0000 7f014b59e700 1 -- 172.21.15.162:0/3675929428 wait complete. 2020-11-28T21:47:06.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.807+0000 7f014b59e700 1 Processor -- start 2020-11-28T21:47:06.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.807+0000 7f014b59e700 1 -- start start 2020-11-28T21:47:06.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.811+0000 7f014b59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f014c064c70 0x7f014c136d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:06.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.811+0000 7f014b59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f014c065750 0x7f014c14d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:06.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.811+0000 7f014b59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f014c13f6c0 0x7f014c0b5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:06.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.811+0000 7f014b59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014c14dbf0 con 0x7f014c13f6c0 2020-11-28T21:47:06.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.811+0000 7f014b59e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014c144af0 con 0x7f014c065750 2020-11-28T21:47:06.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.811+0000 7f014b59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f014c0b3c50 con 0x7f014c064c70 2020-11-28T21:47:06.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.811+0000 7f0149d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f014c065750 0x7f014c14d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:06.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.811+0000 7f0149d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f014c065750 0x7f014c14d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38522/0 (socket says 172.21.15.162:38522) 2020-11-28T21:47:06.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.811+0000 7f0149d9b700 1 -- 172.21.15.162:0/3790956372 learned_addr learned my addr 172.21.15.162:0/3790956372 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:06.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.811+0000 7f014a59c700 1 --2- 172.21.15.162:0/3790956372 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f014c064c70 0x7f014c136d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:06.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.811+0000 7f014ad9d700 1 --2- 172.21.15.162:0/3790956372 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f014c13f6c0 0x7f014c0b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:06.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.811+0000 7f0149d9b700 1 -- 172.21.15.162:0/3790956372 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f014c064c70 msgr2=0x7f014c136d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:06.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.811+0000 7f0149d9b700 1 --2- 172.21.15.162:0/3790956372 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f014c064c70 0x7f014c136d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:06.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.811+0000 7f0149d9b700 1 -- 172.21.15.162:0/3790956372 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f014c13f6c0 msgr2=0x7f014c0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:06.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.811+0000 7f0149d9b700 1 --2- 172.21.15.162:0/3790956372 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f014c13f6c0 0x7f014c0b5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:06.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.811+0000 7f0149d9b700 1 -- 172.21.15.162:0/3790956372 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f014c144d50 con 0x7f014c065750 2020-11-28T21:47:06.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.811+0000 7f0149d9b700 1 --2- 172.21.15.162:0/3790956372 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f014c065750 0x7f014c14d700 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f013c001000 tx=0x7f0140017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:06.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.811+0000 7f013b7fe700 1 -- 172.21.15.162:0/3790956372 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f014001f070 con 0x7f014c065750 2020-11-28T21:47:06.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.811+0000 7f013b7fe700 1 -- 172.21.15.162:0/3790956372 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0140006770 con 0x7f014c065750 2020-11-28T21:47:06.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.811+0000 7f014b59e700 1 -- 172.21.15.162:0/3790956372 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f014c0b3ef0 con 0x7f014c065750 2020-11-28T21:47:06.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.811+0000 7f013b7fe700 1 -- 172.21.15.162:0/3790956372 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0140019660 con 0x7f014c065750 2020-11-28T21:47:06.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.811+0000 7f014b59e700 1 -- 172.21.15.162:0/3790956372 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f014c140ac0 con 0x7f014c065750 2020-11-28T21:47:06.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.815+0000 7f014b59e700 1 -- 172.21.15.162:0/3790956372 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f014c144ee0 con 0x7f014c065750 2020-11-28T21:47:06.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.815+0000 7f013b7fe700 1 -- 172.21.15.162:0/3790956372 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f01400197c0 con 0x7f014c065750 2020-11-28T21:47:06.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.815+0000 7f013b7fe700 1 --2- 172.21.15.162:0/3790956372 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0128055440 0x7f0128057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:06.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.815+0000 7f013b7fe700 1 -- 172.21.15.162:0/3790956372 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f0140070ab0 con 0x7f014c065750 2020-11-28T21:47:06.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.815+0000 7f014a59c700 1 --2- 172.21.15.162:0/3790956372 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0128055440 0x7f0128057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:06.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.815+0000 7f014a59c700 1 --2- 172.21.15.162:0/3790956372 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0128055440 0x7f0128057900 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f014400d000 tx=0x7f0144007d00).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:06.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.819+0000 7f013b7fe700 1 -- 172.21.15.162:0/3790956372 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f01400470f0 con 0x7f014c065750 2020-11-28T21:47:06.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.959+0000 7f014b59e700 1 -- 172.21.15.162:0/3790956372 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f014c057050 con 0x7f014c065750 2020-11-28T21:47:06.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.963+0000 7f013b7fe700 1 -- 172.21.15.162:0/3790956372 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+13277 (secure 0 0 0) 0x7f014004a710 con 0x7f014c065750 2020-11-28T21:47:06.964 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:47:06.964 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":29,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:47:04.138232+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:47:06.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.963+0000 7f014b59e700 1 -- 172.21.15.162:0/3790956372 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0128055440 msgr2=0x7f0128057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:06.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.963+0000 7f014b59e700 1 --2- 172.21.15.162:0/3790956372 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0128055440 0x7f0128057900 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f014400d000 tx=0x7f0144007d00).stop 2020-11-28T21:47:06.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.963+0000 7f014b59e700 1 -- 172.21.15.162:0/3790956372 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f014c065750 msgr2=0x7f014c14d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:06.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.963+0000 7f014b59e700 1 --2- 172.21.15.162:0/3790956372 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f014c065750 0x7f014c14d700 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f013c001000 tx=0x7f0140017000).stop 2020-11-28T21:47:06.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.963+0000 7f014b59e700 1 -- 172.21.15.162:0/3790956372 shutdown_connections 2020-11-28T21:47:06.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.963+0000 7f014b59e700 1 --2- 172.21.15.162:0/3790956372 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0128055440 0x7f0128057900 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:06.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.963+0000 7f014b59e700 1 --2- 172.21.15.162:0/3790956372 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f014c064c70 0x7f014c136d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:06.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.963+0000 7f014b59e700 1 --2- 172.21.15.162:0/3790956372 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f014c13f6c0 0x7f014c0b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:06.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.963+0000 7f014b59e700 1 --2- 172.21.15.162:0/3790956372 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f014c065750 0x7f014c14d700 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:06.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.963+0000 7f014b59e700 1 -- 172.21.15.162:0/3790956372 >> 172.21.15.162:0/3790956372 conn(0x7f014c0b5da0 msgr2=0x7f014c0b0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:06.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.963+0000 7f014b59e700 1 -- 172.21.15.162:0/3790956372 shutdown_connections 2020-11-28T21:47:06.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:06.963+0000 7f014b59e700 1 -- 172.21.15.162:0/3790956372 wait complete. 2020-11-28T21:47:06.981 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-28T21:47:07.025 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:47:07.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.163+0000 7f920251e700 1 Processor -- start 2020-11-28T21:47:07.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.163+0000 7f920251e700 1 -- start start 2020-11-28T21:47:07.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.163+0000 7f920251e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f91fc0605c0 0x7f91fc140390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:07.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.163+0000 7f920251e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91fc143950 con 0x7f91fc0b9ce0 2020-11-28T21:47:07.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.163+0000 7f920251e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91fc0ba0c0 con 0x7f91fc1433c0 2020-11-28T21:47:07.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.163+0000 7f920251e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91fc0b4340 con 0x7f91fc0605c0 2020-11-28T21:47:07.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.163+0000 7f91fb7fe700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f91fc1433c0 0x7f91fc13fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:45376/0 (socket says 172.21.15.162:45376) 2020-11-28T21:47:07.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.163+0000 7f91fb7fe700 1 -- 172.21.15.162:0/3698829387 learned_addr learned my addr 172.21.15.162:0/3698829387 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:47:07.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.163+0000 7f9200abb700 1 --2- 172.21.15.162:0/3698829387 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f91fc0605c0 0x7f91fc140390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:07.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.163+0000 7f91faffd700 1 -- 172.21.15.162:0/3698829387 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1721556657 0 0) 0x7f91fc0ba0c0 con 0x7f91fc1433c0 2020-11-28T21:47:07.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.163+0000 7f91faffd700 1 -- 172.21.15.162:0/3698829387 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91dc003230 con 0x7f91fc1433c0 2020-11-28T21:47:07.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.163+0000 7f91faffd700 1 -- 172.21.15.162:0/3698829387 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1428712115 0 0) 0x7f91fc143950 con 0x7f91fc0b9ce0 2020-11-28T21:47:07.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.163+0000 7f91faffd700 1 -- 172.21.15.162:0/3698829387 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91fc0ba0c0 con 0x7f91fc0b9ce0 2020-11-28T21:47:07.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.163+0000 7f9200abb700 1 -- 172.21.15.162:0/3698829387 >> v1:172.21.15.162:6789/0 conn(0x7f91fc1433c0 legacy=0x7f91fc13fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:07.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.163+0000 7f9200abb700 1 -- 172.21.15.162:0/3698829387 >> v1:172.21.15.118:6789/0 conn(0x7f91fc0b9ce0 legacy=0x7f91fc0b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:07.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.163+0000 7f9200abb700 1 -- 172.21.15.162:0/3698829387 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91fc066c30 con 0x7f91fc0605c0 2020-11-28T21:47:07.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.163+0000 7f9200abb700 1 --2- 172.21.15.162:0/3698829387 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f91fc0605c0 0x7f91fc140390 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f91f4002f30 tx=0x7f91f4006dc0).ready entity=mon.2 client_cookie=838a7905e1f71e8f server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:07.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.163+0000 7f91faffd700 1 -- 172.21.15.162:0/3698829387 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91f4009730 con 0x7f91fc0605c0 2020-11-28T21:47:07.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.163+0000 7f91faffd700 1 -- 172.21.15.162:0/3698829387 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f4009890 con 0x7f91fc0605c0 2020-11-28T21:47:07.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.163+0000 7f91faffd700 1 -- 172.21.15.162:0/3698829387 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91f4009bf0 con 0x7f91fc0605c0 2020-11-28T21:47:07.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.163+0000 7f920251e700 1 -- 172.21.15.162:0/3698829387 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f91fc0605c0 msgr2=0x7f91fc140390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:07.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.163+0000 7f920251e700 1 --2- 172.21.15.162:0/3698829387 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f91fc0605c0 0x7f91fc140390 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f91f4002f30 tx=0x7f91f4006dc0).stop 2020-11-28T21:47:07.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.167+0000 7f920251e700 1 -- 172.21.15.162:0/3698829387 shutdown_connections 2020-11-28T21:47:07.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.167+0000 7f920251e700 1 --2- 172.21.15.162:0/3698829387 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f91fc0605c0 0x7f91fc140390 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:07.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.167+0000 7f920251e700 1 -- 172.21.15.162:0/3698829387 >> 172.21.15.162:0/3698829387 conn(0x7f91fc05c430 msgr2=0x7f91fc0b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:07.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.167+0000 7f920251e700 1 -- 172.21.15.162:0/3698829387 shutdown_connections 2020-11-28T21:47:07.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.167+0000 7f920251e700 1 -- 172.21.15.162:0/3698829387 wait complete. 2020-11-28T21:47:07.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.167+0000 7f920251e700 1 Processor -- start 2020-11-28T21:47:07.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.167+0000 7f920251e700 1 -- start start 2020-11-28T21:47:07.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.167+0000 7f920251e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91fc0605c0 0x7f91fc154fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:07.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.167+0000 7f920251e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f91fc0b9ce0 0x7f91fc0ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:07.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.167+0000 7f920251e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f91fc1433c0 0x7f91fc0add90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:07.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.167+0000 7f920251e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91fc0ae900 con 0x7f91fc1433c0 2020-11-28T21:47:07.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.167+0000 7f920251e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91fc155890 con 0x7f91fc0605c0 2020-11-28T21:47:07.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.167+0000 7f920251e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91fc155b30 con 0x7f91fc0b9ce0 2020-11-28T21:47:07.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.167+0000 7f91fbfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91fc0605c0 0x7f91fc154fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:07.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.167+0000 7f91fbfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91fc0605c0 0x7f91fc154fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38536/0 (socket says 172.21.15.162:38536) 2020-11-28T21:47:07.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.167+0000 7f91fbfff700 1 -- 172.21.15.162:0/1329092864 learned_addr learned my addr 172.21.15.162:0/1329092864 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:07.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.167+0000 7f91fb7fe700 1 --2- 172.21.15.162:0/1329092864 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f91fc0b9ce0 0x7f91fc0ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:07.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.167+0000 7f9200abb700 1 --2- 172.21.15.162:0/1329092864 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f91fc1433c0 0x7f91fc0add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:07.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.167+0000 7f91fbfff700 1 -- 172.21.15.162:0/1329092864 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f91fc0b9ce0 msgr2=0x7f91fc0ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:07.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.167+0000 7f91fbfff700 1 --2- 172.21.15.162:0/1329092864 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f91fc0b9ce0 0x7f91fc0ad8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:07.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.167+0000 7f91fbfff700 1 -- 172.21.15.162:0/1329092864 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f91fc1433c0 msgr2=0x7f91fc0add90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:07.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.167+0000 7f91fbfff700 1 --2- 172.21.15.162:0/1329092864 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f91fc1433c0 0x7f91fc0add90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:07.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.167+0000 7f91fbfff700 1 -- 172.21.15.162:0/1329092864 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91fc0ae580 con 0x7f91fc0605c0 2020-11-28T21:47:07.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.167+0000 7f9200abb700 1 --2- 172.21.15.162:0/1329092864 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f91fc1433c0 0x7f91fc0add90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:47:07.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.167+0000 7f91fbfff700 1 --2- 172.21.15.162:0/1329092864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91fc0605c0 0x7f91fc154fd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f91ec0014a0 tx=0x7f91ec007dd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:07.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.167+0000 7f91f97fa700 1 -- 172.21.15.162:0/1329092864 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91ec002240 con 0x7f91fc0605c0 2020-11-28T21:47:07.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.167+0000 7f91f97fa700 1 -- 172.21.15.162:0/1329092864 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91ec0023a0 con 0x7f91fc0605c0 2020-11-28T21:47:07.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.167+0000 7f920251e700 1 -- 172.21.15.162:0/1329092864 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91fc1554c0 con 0x7f91fc0605c0 2020-11-28T21:47:07.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.167+0000 7f91f97fa700 1 -- 172.21.15.162:0/1329092864 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91ec001660 con 0x7f91fc0605c0 2020-11-28T21:47:07.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.167+0000 7f920251e700 1 -- 172.21.15.162:0/1329092864 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91fc0b0130 con 0x7f91fc0605c0 2020-11-28T21:47:07.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.171+0000 7f920251e700 1 -- 172.21.15.162:0/1329092864 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91fc0b6310 con 0x7f91fc0605c0 2020-11-28T21:47:07.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.171+0000 7f91f97fa700 1 -- 172.21.15.162:0/1329092864 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f91ec019030 con 0x7f91fc0605c0 2020-11-28T21:47:07.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.171+0000 7f91f97fa700 1 --2- 172.21.15.162:0/1329092864 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f91e00554e0 0x7f91e00579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:07.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.171+0000 7f91fb7fe700 1 --2- 172.21.15.162:0/1329092864 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f91e00554e0 0x7f91e00579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:07.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.171+0000 7f91f97fa700 1 -- 172.21.15.162:0/1329092864 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f91ec06d350 con 0x7f91fc0605c0 2020-11-28T21:47:07.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.171+0000 7f91fb7fe700 1 --2- 172.21.15.162:0/1329092864 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f91e00554e0 0x7f91e00579a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f91e400d000 tx=0x7f91e4009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:07.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.179+0000 7f91f97fa700 1 -- 172.21.15.162:0/1329092864 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f91fc0b6310 con 0x7f91fc0605c0 2020-11-28T21:47:07.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.319+0000 7f920251e700 1 -- 172.21.15.162:0/1329092864 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f91fc0b0130 con 0x7f91fc0605c0 2020-11-28T21:47:07.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.319+0000 7f91f97fa700 1 -- 172.21.15.162:0/1329092864 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v29) v1 ==== 110+0+10 (secure 0 0 0) 0x7f91fc0b0130 con 0x7f91fc0605c0 2020-11-28T21:47:07.323 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:47:07.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.323+0000 7f920251e700 1 -- 172.21.15.162:0/1329092864 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f91e00554e0 msgr2=0x7f91e00579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:07.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.323+0000 7f920251e700 1 --2- 172.21.15.162:0/1329092864 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f91e00554e0 0x7f91e00579a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f91e400d000 tx=0x7f91e4009040).stop 2020-11-28T21:47:07.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.323+0000 7f920251e700 1 -- 172.21.15.162:0/1329092864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91fc0605c0 msgr2=0x7f91fc154fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:07.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.323+0000 7f920251e700 1 --2- 172.21.15.162:0/1329092864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91fc0605c0 0x7f91fc154fd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f91ec0014a0 tx=0x7f91ec007dd0).stop 2020-11-28T21:47:07.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.323+0000 7f920251e700 1 -- 172.21.15.162:0/1329092864 shutdown_connections 2020-11-28T21:47:07.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.323+0000 7f920251e700 1 --2- 172.21.15.162:0/1329092864 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f91e00554e0 0x7f91e00579a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:07.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.323+0000 7f920251e700 1 --2- 172.21.15.162:0/1329092864 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f91fc1433c0 0x7f91fc0add90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:07.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.323+0000 7f920251e700 1 --2- 172.21.15.162:0/1329092864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91fc0605c0 0x7f91fc154fd0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:07.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.323+0000 7f920251e700 1 --2- 172.21.15.162:0/1329092864 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f91fc0b9ce0 0x7f91fc0ad8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:07.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.323+0000 7f920251e700 1 -- 172.21.15.162:0/1329092864 >> 172.21.15.162:0/1329092864 conn(0x7f91fc05c430 msgr2=0x7f91fc05f750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:07.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.323+0000 7f920251e700 1 -- 172.21.15.162:0/1329092864 shutdown_connections 2020-11-28T21:47:07.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.323+0000 7f920251e700 1 -- 172.21.15.162:0/1329092864 wait complete. 2020-11-28T21:47:07.337 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:47:07.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.463+0000 7f99eaa00700 1 Processor -- start 2020-11-28T21:47:07.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.463+0000 7f99eaa00700 1 -- start start 2020-11-28T21:47:07.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.467+0000 7f99eaa00700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f99e414a1e0 0x7f99e414c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:07.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.467+0000 7f99eaa00700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99e40a8f50 con 0x7f99e40a9960 2020-11-28T21:47:07.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.467+0000 7f99eaa00700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99e40981d0 con 0x7f99e40a9d40 2020-11-28T21:47:07.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.467+0000 7f99eaa00700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99e40aa1a0 con 0x7f99e414a1e0 2020-11-28T21:47:07.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.467+0000 7f99ea1ff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f99e414a1e0 0x7f99e414c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:07.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.467+0000 7f99ea1ff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f99e414a1e0 0x7f99e414c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38546/0 (socket says 172.21.15.162:38546) 2020-11-28T21:47:07.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.467+0000 7f99e91fd700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f99e40a9d40 0x7f99e4141330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:45388/0 (socket says 172.21.15.162:45388) 2020-11-28T21:47:07.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.467+0000 7f99ea1ff700 1 -- 172.21.15.162:0/2180559969 learned_addr learned my addr 172.21.15.162:0/2180559969 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:07.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.467+0000 7f99ea1ff700 1 -- 172.21.15.162:0/2180559969 >> v1:172.21.15.162:6789/0 conn(0x7f99e40a9d40 legacy=0x7f99e4141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:07.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.467+0000 7f99ea1ff700 1 -- 172.21.15.162:0/2180559969 >> v1:172.21.15.118:6789/0 conn(0x7f99e40a9960 legacy=0x7f99e4097ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:07.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.467+0000 7f99ea1ff700 1 -- 172.21.15.162:0/2180559969 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e40b0b30 con 0x7f99e414a1e0 2020-11-28T21:47:07.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.467+0000 7f99ea1ff700 1 --2- 172.21.15.162:0/2180559969 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99e414a1e0 0x7f99e414c5d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f99e40629b0 tx=0x7f99dc009270).ready entity=mon.1 client_cookie=bca09ac635a31f8e server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:07.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.467+0000 7f99e89fc700 1 -- 172.21.15.162:0/2180559969 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f99dc00bbc0 con 0x7f99e414a1e0 2020-11-28T21:47:07.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.467+0000 7f99e89fc700 1 -- 172.21.15.162:0/2180559969 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99dc00bd20 con 0x7f99e414a1e0 2020-11-28T21:47:07.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.467+0000 7f99e89fc700 1 -- 172.21.15.162:0/2180559969 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f99dc00be90 con 0x7f99e414a1e0 2020-11-28T21:47:07.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.467+0000 7f99eaa00700 1 -- 172.21.15.162:0/2180559969 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99e414a1e0 msgr2=0x7f99e414c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:07.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.467+0000 7f99eaa00700 1 --2- 172.21.15.162:0/2180559969 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99e414a1e0 0x7f99e414c5d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f99e40629b0 tx=0x7f99dc009270).stop 2020-11-28T21:47:07.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.467+0000 7f99eaa00700 1 -- 172.21.15.162:0/2180559969 shutdown_connections 2020-11-28T21:47:07.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.467+0000 7f99eaa00700 1 --2- 172.21.15.162:0/2180559969 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99e414a1e0 0x7f99e414c5d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:07.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.467+0000 7f99eaa00700 1 -- 172.21.15.162:0/2180559969 >> 172.21.15.162:0/2180559969 conn(0x7f99e40b5520 msgr2=0x7f99e40b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:07.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.467+0000 7f99eaa00700 1 -- 172.21.15.162:0/2180559969 shutdown_connections 2020-11-28T21:47:07.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.467+0000 7f99eaa00700 1 -- 172.21.15.162:0/2180559969 wait complete. 2020-11-28T21:47:07.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.471+0000 7f99eaa00700 1 Processor -- start 2020-11-28T21:47:07.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.471+0000 7f99eaa00700 1 -- start start 2020-11-28T21:47:07.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.471+0000 7f99eaa00700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f99e40a9960 0x7f99e41496f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:07.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.471+0000 7f99eaa00700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f99e40a9d40 0x7f99e4065440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:07.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.471+0000 7f99eaa00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99e414a1e0 0x7f99e41524e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:07.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.471+0000 7f99eaa00700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e41560b0 con 0x7f99e40a9960 2020-11-28T21:47:07.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.471+0000 7f99eaa00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e4151bb0 con 0x7f99e414a1e0 2020-11-28T21:47:07.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.471+0000 7f99eaa00700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99e41552c0 con 0x7f99e40a9d40 2020-11-28T21:47:07.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.471+0000 7f99ea1ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99e414a1e0 0x7f99e41524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:07.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.471+0000 7f99ea1ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99e414a1e0 0x7f99e41524e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38552/0 (socket says 172.21.15.162:38552) 2020-11-28T21:47:07.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.471+0000 7f99ea1ff700 1 -- 172.21.15.162:0/3219275422 learned_addr learned my addr 172.21.15.162:0/3219275422 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:07.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.471+0000 7f99e91fd700 1 --2- 172.21.15.162:0/3219275422 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f99e40a9d40 0x7f99e4065440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:07.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.471+0000 7f99e99fe700 1 --2- 172.21.15.162:0/3219275422 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f99e40a9960 0x7f99e41496f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:07.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.471+0000 7f99ea1ff700 1 -- 172.21.15.162:0/3219275422 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f99e40a9d40 msgr2=0x7f99e4065440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:07.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.471+0000 7f99ea1ff700 1 --2- 172.21.15.162:0/3219275422 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f99e40a9d40 0x7f99e4065440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:07.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.471+0000 7f99ea1ff700 1 -- 172.21.15.162:0/3219275422 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f99e40a9960 msgr2=0x7f99e41496f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:07.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.471+0000 7f99ea1ff700 1 --2- 172.21.15.162:0/3219275422 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f99e40a9960 0x7f99e41496f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:07.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.471+0000 7f99ea1ff700 1 -- 172.21.15.162:0/3219275422 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e4151da0 con 0x7f99e414a1e0 2020-11-28T21:47:07.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.471+0000 7f99e91fd700 1 --2- 172.21.15.162:0/3219275422 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f99e40a9d40 0x7f99e4065440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:47:07.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.471+0000 7f99e99fe700 1 --2- 172.21.15.162:0/3219275422 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f99e40a9960 0x7f99e41496f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:47:07.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.471+0000 7f99ea1ff700 1 --2- 172.21.15.162:0/3219275422 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99e414a1e0 0x7f99e41524e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f99e40629b0 tx=0x7f99dc016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:07.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.475+0000 7f99d2ffd700 1 -- 172.21.15.162:0/3219275422 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f99dc01f5e0 con 0x7f99e414a1e0 2020-11-28T21:47:07.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.475+0000 7f99d2ffd700 1 -- 172.21.15.162:0/3219275422 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99dc01f740 con 0x7f99e414a1e0 2020-11-28T21:47:07.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.475+0000 7f99eaa00700 1 -- 172.21.15.162:0/3219275422 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99e4152040 con 0x7f99e414a1e0 2020-11-28T21:47:07.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.475+0000 7f99d2ffd700 1 -- 172.21.15.162:0/3219275422 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f99dc01faa0 con 0x7f99e414a1e0 2020-11-28T21:47:07.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.475+0000 7f99eaa00700 1 -- 172.21.15.162:0/3219275422 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99e409a300 con 0x7f99e414a1e0 2020-11-28T21:47:07.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.475+0000 7f99eaa00700 1 -- 172.21.15.162:0/3219275422 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99e4057050 con 0x7f99e414a1e0 2020-11-28T21:47:07.485 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.475+0000 7f99d2ffd700 1 -- 172.21.15.162:0/3219275422 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f99dc026030 con 0x7f99e414a1e0 2020-11-28T21:47:07.485 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.475+0000 7f99d2ffd700 1 --2- 172.21.15.162:0/3219275422 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f99c40553b0 0x7f99c4057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:07.485 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.475+0000 7f99d2ffd700 1 -- 172.21.15.162:0/3219275422 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f99dc06ec70 con 0x7f99e414a1e0 2020-11-28T21:47:07.486 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.475+0000 7f99e99fe700 1 --2- 172.21.15.162:0/3219275422 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f99c40553b0 0x7f99c4057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:07.486 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.475+0000 7f99e99fe700 1 --2- 172.21.15.162:0/3219275422 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f99c40553b0 0x7f99c4057870 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f99e0008000 tx=0x7f99e0009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:07.487 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.483+0000 7f99d2ffd700 1 -- 172.21.15.162:0/3219275422 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f99dc045190 con 0x7f99e414a1e0 2020-11-28T21:47:07.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.615+0000 7f99eaa00700 1 -- 172.21.15.162:0/3219275422 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f99e405d040 con 0x7f99e414a1e0 2020-11-28T21:47:07.620 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.619+0000 7f99d2ffd700 1 -- 172.21.15.162:0/3219275422 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f99dc024020 con 0x7f99e414a1e0 2020-11-28T21:47:07.620 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:47:07.622 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.619+0000 7f99eaa00700 1 -- 172.21.15.162:0/3219275422 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f99c40553b0 msgr2=0x7f99c4057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:07.622 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.619+0000 7f99eaa00700 1 --2- 172.21.15.162:0/3219275422 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f99c40553b0 0x7f99c4057870 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f99e0008000 tx=0x7f99e0009040).stop 2020-11-28T21:47:07.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.619+0000 7f99eaa00700 1 -- 172.21.15.162:0/3219275422 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99e414a1e0 msgr2=0x7f99e41524e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:07.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.619+0000 7f99eaa00700 1 --2- 172.21.15.162:0/3219275422 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99e414a1e0 0x7f99e41524e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f99e40629b0 tx=0x7f99dc016000).stop 2020-11-28T21:47:07.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.619+0000 7f99eaa00700 1 -- 172.21.15.162:0/3219275422 shutdown_connections 2020-11-28T21:47:07.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.619+0000 7f99eaa00700 1 --2- 172.21.15.162:0/3219275422 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f99c40553b0 0x7f99c4057870 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:07.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.619+0000 7f99eaa00700 1 --2- 172.21.15.162:0/3219275422 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99e414a1e0 0x7f99e41524e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:07.624 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.619+0000 7f99eaa00700 1 --2- 172.21.15.162:0/3219275422 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f99e40a9960 0x7f99e41496f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:07.624 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.619+0000 7f99eaa00700 1 --2- 172.21.15.162:0/3219275422 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f99e40a9d40 0x7f99e4065440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:07.624 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.619+0000 7f99eaa00700 1 -- 172.21.15.162:0/3219275422 >> 172.21.15.162:0/3219275422 conn(0x7f99e40b5520 msgr2=0x7f99e405b970 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:07.624 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.619+0000 7f99eaa00700 1 -- 172.21.15.162:0/3219275422 shutdown_connections 2020-11-28T21:47:07.624 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.619+0000 7f99eaa00700 1 -- 172.21.15.162:0/3219275422 wait complete. 2020-11-28T21:47:07.634 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2020-11-28T21:47:07.748 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.747+0000 7f6f80ff5700 1 Processor -- start 2020-11-28T21:47:07.748 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.747+0000 7f6f80ff5700 1 -- start start 2020-11-28T21:47:07.749 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.747+0000 7f6f80ff5700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f6f7c0b1990 0x7f6f7c140890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:07.749 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.747+0000 7f6f80ff5700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6f7c0b11b0 0x7f6f7c140d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:07.750 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.747+0000 7f6f80ff5700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f7c0b2870 con 0x7f6f7c0b1d70 2020-11-28T21:47:07.750 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.747+0000 7f6f80ff5700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f7c146e20 con 0x7f6f7c0b1990 2020-11-28T21:47:07.750 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.747+0000 7f6f80ff5700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7c0b22e0 con 0x7f6f7c0b11b0 2020-11-28T21:47:07.750 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.747+0000 7f6f7a59c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f6f7c0b1990 0x7f6f7c140890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:07.751 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.747+0000 7f6f7a59c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f6f7c0b1990 0x7f6f7c140890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.162:45544/0 (socket says 172.21.15.162:45544) 2020-11-28T21:47:07.751 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.747+0000 7f6f79d9b700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6f7c0b11b0 0x7f6f7c140d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:07.751 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.747+0000 7f6f7a59c700 1 -- 172.21.15.162:0/956598355 learned_addr learned my addr 172.21.15.162:0/956598355 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:07.751 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.747+0000 7f6f7a59c700 1 -- 172.21.15.162:0/956598355 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6f7c0b11b0 msgr2=0x7f6f7c140d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:07.751 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.747+0000 7f6f7a59c700 1 --2- 172.21.15.162:0/956598355 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6f7c0b11b0 0x7f6f7c140d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:07.752 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.747+0000 7f6f7a59c700 1 -- 172.21.15.162:0/956598355 >> v1:172.21.15.118:6789/0 conn(0x7f6f7c0b1d70 legacy=0x7f6f7c14a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:07.752 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.747+0000 7f6f7a59c700 1 -- 172.21.15.162:0/956598355 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f7c0b0740 con 0x7f6f7c0b1990 2020-11-28T21:47:07.753 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.747+0000 7f6f7a59c700 1 --2- 172.21.15.162:0/956598355 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f7c0b1990 0x7f6f7c140890 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f6f6c0032b0 tx=0x7f6f6c009270).ready entity=mon.0 client_cookie=868aecd87b5d95a2 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:07.753 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.747+0000 7f6f7959a700 1 -- 172.21.15.162:0/956598355 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f6c009e60 con 0x7f6f7c0b1990 2020-11-28T21:47:07.754 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.747+0000 7f6f7959a700 1 -- 172.21.15.162:0/956598355 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6c00bc00 con 0x7f6f7c0b1990 2020-11-28T21:47:07.754 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.747+0000 7f6f7959a700 1 -- 172.21.15.162:0/956598355 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f6c00bd70 con 0x7f6f7c0b1990 2020-11-28T21:47:07.754 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.747+0000 7f6f80ff5700 1 -- 172.21.15.162:0/956598355 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f7c0b1990 msgr2=0x7f6f7c140890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:07.754 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.747+0000 7f6f80ff5700 1 --2- 172.21.15.162:0/956598355 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f7c0b1990 0x7f6f7c140890 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f6f6c0032b0 tx=0x7f6f6c009270).stop 2020-11-28T21:47:07.755 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.751+0000 7f6f80ff5700 1 -- 172.21.15.162:0/956598355 shutdown_connections 2020-11-28T21:47:07.755 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.751+0000 7f6f80ff5700 1 --2- 172.21.15.162:0/956598355 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f7c0b1990 0x7f6f7c140890 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:07.755 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.751+0000 7f6f80ff5700 1 --2- 172.21.15.162:0/956598355 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6f7c0b11b0 0x7f6f7c140d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:07.755 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.751+0000 7f6f80ff5700 1 -- 172.21.15.162:0/956598355 >> 172.21.15.162:0/956598355 conn(0x7f6f7c05b970 msgr2=0x7f6f7c05bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:07.755 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.751+0000 7f6f80ff5700 1 -- 172.21.15.162:0/956598355 shutdown_connections 2020-11-28T21:47:07.756 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.751+0000 7f6f80ff5700 1 -- 172.21.15.162:0/956598355 wait complete. 2020-11-28T21:47:07.756 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.751+0000 7f6f80ff5700 1 Processor -- start 2020-11-28T21:47:07.756 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.751+0000 7f6f80ff5700 1 -- start start 2020-11-28T21:47:07.756 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.751+0000 7f6f80ff5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f7c0b11b0 0x7f6f7c0662f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:07.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.751+0000 7f6f80ff5700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f7c0b1990 0x7f6f7c063330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:07.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.751+0000 7f6f80ff5700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6f7c0b1d70 0x7f6f7c063820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:07.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.751+0000 7f6f80ff5700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7c0abc30 con 0x7f6f7c0b1990 2020-11-28T21:47:07.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.751+0000 7f6f80ff5700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7c1567c0 con 0x7f6f7c0b11b0 2020-11-28T21:47:07.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.751+0000 7f6f80ff5700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7c156a60 con 0x7f6f7c0b1d70 2020-11-28T21:47:07.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.751+0000 7f6f7a59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f7c0b11b0 0x7f6f7c0662f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:07.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.751+0000 7f6f7a59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f7c0b11b0 0x7f6f7c0662f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38560/0 (socket says 172.21.15.162:38560) 2020-11-28T21:47:07.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.751+0000 7f6f7a59c700 1 -- 172.21.15.162:0/1422792235 learned_addr learned my addr 172.21.15.162:0/1422792235 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:07.761 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.751+0000 7f6f79d9b700 1 --2- 172.21.15.162:0/1422792235 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f7c0b1990 0x7f6f7c063330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:07.761 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.751+0000 7f6f7ad9d700 1 --2- 172.21.15.162:0/1422792235 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6f7c0b1d70 0x7f6f7c063820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:07.761 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.751+0000 7f6f7a59c700 1 -- 172.21.15.162:0/1422792235 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6f7c0b1d70 msgr2=0x7f6f7c063820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:07.761 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.751+0000 7f6f7a59c700 1 --2- 172.21.15.162:0/1422792235 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6f7c0b1d70 0x7f6f7c063820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:07.761 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.751+0000 7f6f7a59c700 1 -- 172.21.15.162:0/1422792235 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f7c0b1990 msgr2=0x7f6f7c063330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:07.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.751+0000 7f6f7a59c700 1 --2- 172.21.15.162:0/1422792235 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f7c0b1990 0x7f6f7c063330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:07.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.751+0000 7f6f7a59c700 1 -- 172.21.15.162:0/1422792235 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f7c0ab8b0 con 0x7f6f7c0b11b0 2020-11-28T21:47:07.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.751+0000 7f6f79d9b700 1 --2- 172.21.15.162:0/1422792235 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f7c0b1990 0x7f6f7c063330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:47:07.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.751+0000 7f6f7a59c700 1 --2- 172.21.15.162:0/1422792235 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f7c0b11b0 0x7f6f7c0662f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7f6f6c0032b0 tx=0x7f6f6c016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:07.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.751+0000 7f6f737fe700 1 -- 172.21.15.162:0/1422792235 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f6c022070 con 0x7f6f7c0b11b0 2020-11-28T21:47:07.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.751+0000 7f6f737fe700 1 -- 172.21.15.162:0/1422792235 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6c0107a0 con 0x7f6f7c0b11b0 2020-11-28T21:47:07.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.751+0000 7f6f80ff5700 1 -- 172.21.15.162:0/1422792235 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f7c066cb0 con 0x7f6f7c0b11b0 2020-11-28T21:47:07.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.751+0000 7f6f737fe700 1 -- 172.21.15.162:0/1422792235 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f6c010b00 con 0x7f6f7c0b11b0 2020-11-28T21:47:07.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.751+0000 7f6f80ff5700 1 -- 172.21.15.162:0/1422792235 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f7c143fb0 con 0x7f6f7c0b11b0 2020-11-28T21:47:07.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.755+0000 7f6f80ff5700 1 -- 172.21.15.162:0/1422792235 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f7c057050 con 0x7f6f7c0b11b0 2020-11-28T21:47:07.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.755+0000 7f6f737fe700 1 -- 172.21.15.162:0/1422792235 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6f6c00f070 con 0x7f6f7c0b11b0 2020-11-28T21:47:07.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.755+0000 7f6f737fe700 1 --2- 172.21.15.162:0/1422792235 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6f60055440 0x7f6f60057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:07.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.755+0000 7f6f79d9b700 1 --2- 172.21.15.162:0/1422792235 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6f60055440 0x7f6f60057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:07.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.755+0000 7f6f737fe700 1 -- 172.21.15.162:0/1422792235 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f6f6c06f2c0 con 0x7f6f7c0b11b0 2020-11-28T21:47:07.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.755+0000 7f6f79d9b700 1 --2- 172.21.15.162:0/1422792235 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6f60055440 0x7f6f60057900 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f6f6400b000 tx=0x7f6f64008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:07.765 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.763+0000 7f6f737fe700 1 -- 172.21.15.162:0/1422792235 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6f6c045920 con 0x7f6f7c0b11b0 2020-11-28T21:47:07.907 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.903+0000 7f6f80ff5700 1 -- 172.21.15.162:0/1422792235 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f6f7c05d530 con 0x7f6f7c0b11b0 2020-11-28T21:47:07.907 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.907+0000 7f6f737fe700 1 -- 172.21.15.162:0/1422792235 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) v1 ==== 84+0+143 (secure 0 0 0) 0x7f6f6c0150a0 con 0x7f6f7c0b11b0 2020-11-28T21:47:07.908 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:47:07.908 INFO:teuthology.orchestra.run.smithi162.stdout:[ 2020-11-28T21:47:07.908 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:47:07.909 INFO:teuthology.orchestra.run.smithi162.stdout: "poolnum": 1, 2020-11-28T21:47:07.909 INFO:teuthology.orchestra.run.smithi162.stdout: "poolname": "device_health_metrics" 2020-11-28T21:47:07.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:47:07.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:47:07.909 INFO:teuthology.orchestra.run.smithi162.stdout: "poolnum": 2, 2020-11-28T21:47:07.910 INFO:teuthology.orchestra.run.smithi162.stdout: "poolname": "rbd" 2020-11-28T21:47:07.910 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:47:07.910 INFO:teuthology.orchestra.run.smithi162.stdout:] 2020-11-28T21:47:07.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.907+0000 7f6f80ff5700 1 -- 172.21.15.162:0/1422792235 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6f60055440 msgr2=0x7f6f60057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:07.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.907+0000 7f6f80ff5700 1 --2- 172.21.15.162:0/1422792235 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6f60055440 0x7f6f60057900 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f6f6400b000 tx=0x7f6f64008040).stop 2020-11-28T21:47:07.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.907+0000 7f6f80ff5700 1 -- 172.21.15.162:0/1422792235 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f7c0b11b0 msgr2=0x7f6f7c0662f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:07.912 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.907+0000 7f6f80ff5700 1 --2- 172.21.15.162:0/1422792235 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f7c0b11b0 0x7f6f7c0662f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7f6f6c0032b0 tx=0x7f6f6c016000).stop 2020-11-28T21:47:07.912 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.907+0000 7f6f80ff5700 1 -- 172.21.15.162:0/1422792235 shutdown_connections 2020-11-28T21:47:07.912 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.907+0000 7f6f80ff5700 1 --2- 172.21.15.162:0/1422792235 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6f60055440 0x7f6f60057900 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:07.912 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.907+0000 7f6f80ff5700 1 --2- 172.21.15.162:0/1422792235 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6f7c0b1d70 0x7f6f7c063820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:07.912 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.907+0000 7f6f80ff5700 1 --2- 172.21.15.162:0/1422792235 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f7c0b11b0 0x7f6f7c0662f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:07.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.907+0000 7f6f80ff5700 1 --2- 172.21.15.162:0/1422792235 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f7c0b1990 0x7f6f7c063330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:07.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.907+0000 7f6f80ff5700 1 -- 172.21.15.162:0/1422792235 >> 172.21.15.162:0/1422792235 conn(0x7f6f7c05b970 msgr2=0x7f6f7c146f70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:07.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.907+0000 7f6f80ff5700 1 -- 172.21.15.162:0/1422792235 shutdown_connections 2020-11-28T21:47:07.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:07.907+0000 7f6f80ff5700 1 -- 172.21.15.162:0/1422792235 wait complete. 2020-11-28T21:47:07.921 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-11-28T21:47:07.921 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2020-11-28T21:47:08.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.035+0000 7f0031845700 1 Processor -- start 2020-11-28T21:47:08.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.035+0000 7f0031845700 1 -- start start 2020-11-28T21:47:08.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.039+0000 7f0031845700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f002c0b44e0 0x7f002c0b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:08.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.039+0000 7f0031845700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f002c0b1f50 con 0x7f002c0b18c0 2020-11-28T21:47:08.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.039+0000 7f0031845700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f002c143e90 con 0x7f002c0b3950 2020-11-28T21:47:08.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.039+0000 7f0031845700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f002c0aab50 con 0x7f002c0b44e0 2020-11-28T21:47:08.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.039+0000 7f0030843700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f002c0b3950 0x7f002c13dd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:45412/0 (socket says 172.21.15.162:45412) 2020-11-28T21:47:08.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.039+0000 7f0030843700 1 -- 172.21.15.162:0/1675191417 learned_addr learned my addr 172.21.15.162:0/1675191417 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:47:08.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.039+0000 7f0031044700 1 --2- 172.21.15.162:0/1675191417 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f002c0b44e0 0x7f002c0b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:08.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.039+0000 7f002b7fe700 1 -- 172.21.15.162:0/1675191417 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1274186154 0 0) 0x7f002c143e90 con 0x7f002c0b3950 2020-11-28T21:47:08.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.039+0000 7f002b7fe700 1 -- 172.21.15.162:0/1675191417 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0018003230 con 0x7f002c0b3950 2020-11-28T21:47:08.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.039+0000 7f002b7fe700 1 -- 172.21.15.162:0/1675191417 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2897581275 0 0) 0x7f002c0b1f50 con 0x7f002c0b18c0 2020-11-28T21:47:08.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.039+0000 7f002b7fe700 1 -- 172.21.15.162:0/1675191417 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f002c143e90 con 0x7f002c0b18c0 2020-11-28T21:47:08.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.039+0000 7f002b7fe700 1 -- 172.21.15.162:0/1675191417 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3632687199 0 0) 0x7f0018003230 con 0x7f002c0b3950 2020-11-28T21:47:08.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.039+0000 7f0031044700 1 -- 172.21.15.162:0/1675191417 >> v1:172.21.15.162:6789/0 conn(0x7f002c0b3950 legacy=0x7f002c13dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:08.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.039+0000 7f0031044700 1 -- 172.21.15.162:0/1675191417 >> v1:172.21.15.118:6789/0 conn(0x7f002c0b18c0 legacy=0x7f002c0b2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:08.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.039+0000 7f0031044700 1 -- 172.21.15.162:0/1675191417 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f002c0aa8b0 con 0x7f002c0b44e0 2020-11-28T21:47:08.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.039+0000 7f0031044700 1 --2- 172.21.15.162:0/1675191417 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f002c0b44e0 0x7f002c0b3d30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f00200010d0 tx=0x7f00240091a0).ready entity=mon.2 client_cookie=e039fbbefc98f8b3 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:08.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.039+0000 7f002b7fe700 1 -- 172.21.15.162:0/1675191417 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00200016d0 con 0x7f002c0b44e0 2020-11-28T21:47:08.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.039+0000 7f002b7fe700 1 -- 172.21.15.162:0/1675191417 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f002400b880 con 0x7f002c0b44e0 2020-11-28T21:47:08.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.039+0000 7f002b7fe700 1 -- 172.21.15.162:0/1675191417 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f002400b9f0 con 0x7f002c0b44e0 2020-11-28T21:47:08.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.039+0000 7f0031845700 1 -- 172.21.15.162:0/1675191417 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f002c0b44e0 msgr2=0x7f002c0b3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:08.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.039+0000 7f0031845700 1 --2- 172.21.15.162:0/1675191417 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f002c0b44e0 0x7f002c0b3d30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f00200010d0 tx=0x7f00240091a0).stop 2020-11-28T21:47:08.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.039+0000 7f0031845700 1 -- 172.21.15.162:0/1675191417 shutdown_connections 2020-11-28T21:47:08.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.039+0000 7f0031845700 1 --2- 172.21.15.162:0/1675191417 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f002c0b44e0 0x7f002c0b3d30 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:08.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.039+0000 7f0031845700 1 -- 172.21.15.162:0/1675191417 >> 172.21.15.162:0/1675191417 conn(0x7f002c062630 msgr2=0x7f002c062a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:08.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.039+0000 7f0031845700 1 -- 172.21.15.162:0/1675191417 shutdown_connections 2020-11-28T21:47:08.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.039+0000 7f0031845700 1 -- 172.21.15.162:0/1675191417 wait complete. 2020-11-28T21:47:08.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.043+0000 7f0031845700 1 Processor -- start 2020-11-28T21:47:08.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.043+0000 7f0031845700 1 -- start start 2020-11-28T21:47:08.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.043+0000 7f0031845700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f002c0b18c0 0x7f002c0b2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:08.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.043+0000 7f0031845700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f002c0b3950 0x7f002c14f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:08.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.043+0000 7f0031845700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f002c0b44e0 0x7f002c14fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:08.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.043+0000 7f0031845700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002c150170 con 0x7f002c0b3950 2020-11-28T21:47:08.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.043+0000 7f0031845700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002c0ad6a0 con 0x7f002c0b18c0 2020-11-28T21:47:08.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.043+0000 7f0031845700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f002c141370 con 0x7f002c0b44e0 2020-11-28T21:47:08.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.043+0000 7f0030843700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f002c0b18c0 0x7f002c0b2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:08.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.043+0000 7f0030843700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f002c0b18c0 0x7f002c0b2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38574/0 (socket says 172.21.15.162:38574) 2020-11-28T21:47:08.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.043+0000 7f0030843700 1 -- 172.21.15.162:0/3870119337 learned_addr learned my addr 172.21.15.162:0/3870119337 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:08.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.043+0000 7f0031044700 1 --2- 172.21.15.162:0/3870119337 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f002c0b44e0 0x7f002c14fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:08.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.043+0000 7f002bfff700 1 --2- 172.21.15.162:0/3870119337 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f002c0b3950 0x7f002c14f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:08.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.043+0000 7f0030843700 1 -- 172.21.15.162:0/3870119337 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f002c0b44e0 msgr2=0x7f002c14fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:08.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.043+0000 7f0030843700 1 --2- 172.21.15.162:0/3870119337 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f002c0b44e0 0x7f002c14fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:08.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.043+0000 7f0030843700 1 -- 172.21.15.162:0/3870119337 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f002c0b3950 msgr2=0x7f002c14f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:08.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.043+0000 7f0030843700 1 --2- 172.21.15.162:0/3870119337 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f002c0b3950 0x7f002c14f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:08.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.043+0000 7f0030843700 1 -- 172.21.15.162:0/3870119337 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f002c153710 con 0x7f002c0b18c0 2020-11-28T21:47:08.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.043+0000 7f0030843700 1 --2- 172.21.15.162:0/3870119337 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f002c0b18c0 0x7f002c0b2f70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f00200018b0 tx=0x7f0020001aa0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:08.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.047+0000 7f0029ffb700 1 -- 172.21.15.162:0/3870119337 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0020007170 con 0x7f002c0b18c0 2020-11-28T21:47:08.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.047+0000 7f0029ffb700 1 -- 172.21.15.162:0/3870119337 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00200072d0 con 0x7f002c0b18c0 2020-11-28T21:47:08.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.047+0000 7f0031845700 1 -- 172.21.15.162:0/3870119337 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f002c1539b0 con 0x7f002c0b18c0 2020-11-28T21:47:08.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.047+0000 7f0029ffb700 1 -- 172.21.15.162:0/3870119337 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f002000a7d0 con 0x7f002c0b18c0 2020-11-28T21:47:08.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.047+0000 7f0031845700 1 -- 172.21.15.162:0/3870119337 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f002c0adad0 con 0x7f002c0b18c0 2020-11-28T21:47:08.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.047+0000 7f0031845700 1 -- 172.21.15.162:0/3870119337 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f002c153580 con 0x7f002c0b18c0 2020-11-28T21:47:08.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.047+0000 7f0029ffb700 1 -- 172.21.15.162:0/3870119337 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f002001c030 con 0x7f002c0b18c0 2020-11-28T21:47:08.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.047+0000 7f0029ffb700 1 --2- 172.21.15.162:0/3870119337 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f000c0554e0 0x7f000c0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:08.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.047+0000 7f0029ffb700 1 -- 172.21.15.162:0/3870119337 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f002006dd70 con 0x7f002c0b18c0 2020-11-28T21:47:08.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.047+0000 7f002bfff700 1 --2- 172.21.15.162:0/3870119337 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f000c0554e0 0x7f000c0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:08.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.051+0000 7f002bfff700 1 --2- 172.21.15.162:0/3870119337 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f000c0554e0 0x7f000c0579a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f001c00d000 tx=0x7f001c0079f0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:08.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.055+0000 7f0029ffb700 1 -- 172.21.15.162:0/3870119337 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0020044290 con 0x7f002c0b18c0 2020-11-28T21:47:08.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.195+0000 7f0031845700 1 -- 172.21.15.162:0/3870119337 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f002c0ad7e0 con 0x7f002c0b18c0 2020-11-28T21:47:08.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.979+0000 7f0029ffb700 1 -- 172.21.15.162:0/3870119337 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v30) v1 ==== 133+0+0 (secure 0 0 0) 0x7f00200478b0 con 0x7f002c0b18c0 2020-11-28T21:47:08.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.983+0000 7f0031845700 1 -- 172.21.15.162:0/3870119337 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f000c0554e0 msgr2=0x7f000c0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:08.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.983+0000 7f0031845700 1 --2- 172.21.15.162:0/3870119337 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f000c0554e0 0x7f000c0579a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f001c00d000 tx=0x7f001c0079f0).stop 2020-11-28T21:47:08.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.983+0000 7f0031845700 1 -- 172.21.15.162:0/3870119337 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f002c0b18c0 msgr2=0x7f002c0b2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:08.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.983+0000 7f0031845700 1 --2- 172.21.15.162:0/3870119337 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f002c0b18c0 0x7f002c0b2f70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f00200018b0 tx=0x7f0020001aa0).stop 2020-11-28T21:47:08.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.983+0000 7f0031845700 1 -- 172.21.15.162:0/3870119337 shutdown_connections 2020-11-28T21:47:08.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.983+0000 7f0031845700 1 --2- 172.21.15.162:0/3870119337 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f000c0554e0 0x7f000c0579a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:08.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.983+0000 7f0031845700 1 --2- 172.21.15.162:0/3870119337 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f002c0b44e0 0x7f002c14fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:08.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.983+0000 7f0031845700 1 --2- 172.21.15.162:0/3870119337 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f002c0b18c0 0x7f002c0b2f70 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:08.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.983+0000 7f0031845700 1 --2- 172.21.15.162:0/3870119337 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f002c0b3950 0x7f002c14f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:08.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.983+0000 7f0031845700 1 -- 172.21.15.162:0/3870119337 >> 172.21.15.162:0/3870119337 conn(0x7f002c062630 msgr2=0x7f002c062a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:08.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.983+0000 7f0031845700 1 -- 172.21.15.162:0/3870119337 shutdown_connections 2020-11-28T21:47:08.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:08.983+0000 7f0031845700 1 -- 172.21.15.162:0/3870119337 wait complete. 2020-11-28T21:47:08.988 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_metadata' created 2020-11-28T21:47:09.000 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2020-11-28T21:47:09.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.111+0000 7fc2af020700 1 Processor -- start 2020-11-28T21:47:09.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.111+0000 7fc2af020700 1 -- start start 2020-11-28T21:47:09.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.115+0000 7fc2af020700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fc2a805cc30 0x7fc2a80b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:09.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.115+0000 7fc2af020700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2a80b1fb0 con 0x7fc2a80b2370 2020-11-28T21:47:09.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.115+0000 7fc2af020700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2a80b0e30 con 0x7fc2a80b0140 2020-11-28T21:47:09.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.115+0000 7fc2af020700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2a80b10d0 con 0x7fc2a805cc30 2020-11-28T21:47:09.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.115+0000 7fc2a7fff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fc2a80b0140 0x7fc2a80a8b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:45428/0 (socket says 172.21.15.162:45428) 2020-11-28T21:47:09.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.115+0000 7fc2a7fff700 1 -- 172.21.15.162:0/2161990902 learned_addr learned my addr 172.21.15.162:0/2161990902 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:47:09.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.115+0000 7fc2ad5bd700 1 --2- 172.21.15.162:0/2161990902 >> v2:172.21.15.118:3300/0 conn(0x7fc2a805cc30 0x7fc2a80b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:09.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.115+0000 7fc2a77fe700 1 -- 172.21.15.162:0/2161990902 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 506351530 0 0) 0x7fc2a80b0e30 con 0x7fc2a80b0140 2020-11-28T21:47:09.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.115+0000 7fc2a77fe700 1 -- 172.21.15.162:0/2161990902 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc288003230 con 0x7fc2a80b0140 2020-11-28T21:47:09.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.115+0000 7fc2a77fe700 1 -- 172.21.15.162:0/2161990902 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2529357558 0 0) 0x7fc2a80b1fb0 con 0x7fc2a80b2370 2020-11-28T21:47:09.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.115+0000 7fc2a77fe700 1 -- 172.21.15.162:0/2161990902 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2a80b0e30 con 0x7fc2a80b2370 2020-11-28T21:47:09.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.115+0000 7fc2ad5bd700 1 -- 172.21.15.162:0/2161990902 >> v1:172.21.15.162:6789/0 conn(0x7fc2a80b0140 legacy=0x7fc2a80a8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:09.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.115+0000 7fc2ad5bd700 1 -- 172.21.15.162:0/2161990902 >> v1:172.21.15.118:6789/0 conn(0x7fc2a80b2370 legacy=0x7fc2a8142d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:09.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.115+0000 7fc2ad5bd700 1 -- 172.21.15.162:0/2161990902 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a80b1620 con 0x7fc2a805cc30 2020-11-28T21:47:09.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.115+0000 7fc2ad5bd700 1 --2- 172.21.15.162:0/2161990902 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc2a805cc30 0x7fc2a80b1ac0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7fc2a0002f30 tx=0x7fc2a0006dc0).ready entity=mon.0 client_cookie=307c0acb7d61f65b server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:09.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.115+0000 7fc2a77fe700 1 -- 172.21.15.162:0/2161990902 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc2a0009740 con 0x7fc2a805cc30 2020-11-28T21:47:09.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.115+0000 7fc2a77fe700 1 -- 172.21.15.162:0/2161990902 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a00098a0 con 0x7fc2a805cc30 2020-11-28T21:47:09.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.115+0000 7fc2a77fe700 1 -- 172.21.15.162:0/2161990902 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc2a0009c00 con 0x7fc2a805cc30 2020-11-28T21:47:09.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.115+0000 7fc2af020700 1 -- 172.21.15.162:0/2161990902 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc2a805cc30 msgr2=0x7fc2a80b1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:09.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.115+0000 7fc2af020700 1 --2- 172.21.15.162:0/2161990902 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc2a805cc30 0x7fc2a80b1ac0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7fc2a0002f30 tx=0x7fc2a0006dc0).stop 2020-11-28T21:47:09.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.119+0000 7fc2af020700 1 -- 172.21.15.162:0/2161990902 shutdown_connections 2020-11-28T21:47:09.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.119+0000 7fc2af020700 1 --2- 172.21.15.162:0/2161990902 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc2a805cc30 0x7fc2a80b1ac0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:09.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.119+0000 7fc2af020700 1 -- 172.21.15.162:0/2161990902 >> 172.21.15.162:0/2161990902 conn(0x7fc2a8063140 msgr2=0x7fc2a8063520 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:09.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.119+0000 7fc2af020700 1 -- 172.21.15.162:0/2161990902 shutdown_connections 2020-11-28T21:47:09.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.119+0000 7fc2af020700 1 -- 172.21.15.162:0/2161990902 wait complete. 2020-11-28T21:47:09.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.119+0000 7fc2af020700 1 Processor -- start 2020-11-28T21:47:09.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.119+0000 7fc2af020700 1 -- start start 2020-11-28T21:47:09.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.123+0000 7fc2af020700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2a805cc30 0x7fc2a80a9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:09.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.123+0000 7fc2af020700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc2a80b0140 0x7fc2a8151b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:09.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.123+0000 7fc2af020700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc2a80b2370 0x7fc2a8152050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:09.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.123+0000 7fc2af020700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a80a9f40 con 0x7fc2a80b0140 2020-11-28T21:47:09.129 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.123+0000 7fc2af020700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a8067e30 con 0x7fc2a805cc30 2020-11-28T21:47:09.129 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.123+0000 7fc2af020700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a8152540 con 0x7fc2a80b2370 2020-11-28T21:47:09.129 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.123+0000 7fc2acdbc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2a805cc30 0x7fc2a80a9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:09.129 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.123+0000 7fc2acdbc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2a805cc30 0x7fc2a80a9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38588/0 (socket says 172.21.15.162:38588) 2020-11-28T21:47:09.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.123+0000 7fc2acdbc700 1 -- 172.21.15.162:0/3338925990 learned_addr learned my addr 172.21.15.162:0/3338925990 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:09.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.123+0000 7fc2ad5bd700 1 --2- 172.21.15.162:0/3338925990 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc2a80b2370 0x7fc2a8152050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:09.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.123+0000 7fc2a7fff700 1 --2- 172.21.15.162:0/3338925990 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc2a80b0140 0x7fc2a8151b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:09.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.123+0000 7fc2acdbc700 1 -- 172.21.15.162:0/3338925990 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc2a80b2370 msgr2=0x7fc2a8152050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:09.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.123+0000 7fc2acdbc700 1 --2- 172.21.15.162:0/3338925990 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc2a80b2370 0x7fc2a8152050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:09.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.123+0000 7fc2acdbc700 1 -- 172.21.15.162:0/3338925990 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc2a80b0140 msgr2=0x7fc2a8151b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:09.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.123+0000 7fc2acdbc700 1 --2- 172.21.15.162:0/3338925990 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc2a80b0140 0x7fc2a8151b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:09.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.123+0000 7fc2acdbc700 1 -- 172.21.15.162:0/3338925990 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a8143b10 con 0x7fc2a805cc30 2020-11-28T21:47:09.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.123+0000 7fc2a7fff700 1 --2- 172.21.15.162:0/3338925990 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc2a80b0140 0x7fc2a8151b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:47:09.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.123+0000 7fc2acdbc700 1 --2- 172.21.15.162:0/3338925990 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2a805cc30 0x7fc2a80a9960 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7fc2a80a9e80 tx=0x7fc298007e30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:09.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.123+0000 7fc2a5ffb700 1 -- 172.21.15.162:0/3338925990 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc298002240 con 0x7fc2a805cc30 2020-11-28T21:47:09.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.123+0000 7fc2a5ffb700 1 -- 172.21.15.162:0/3338925990 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2980023a0 con 0x7fc2a805cc30 2020-11-28T21:47:09.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.123+0000 7fc2af020700 1 -- 172.21.15.162:0/3338925990 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a8152760 con 0x7fc2a805cc30 2020-11-28T21:47:09.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.123+0000 7fc2a5ffb700 1 -- 172.21.15.162:0/3338925990 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc29800a750 con 0x7fc2a805cc30 2020-11-28T21:47:09.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.123+0000 7fc2af020700 1 -- 172.21.15.162:0/3338925990 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a80aa130 con 0x7fc2a805cc30 2020-11-28T21:47:09.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.123+0000 7fc2af020700 1 -- 172.21.15.162:0/3338925990 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2a80aa2f0 con 0x7fc2a805cc30 2020-11-28T21:47:09.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.123+0000 7fc2a5ffb700 1 -- 172.21.15.162:0/3338925990 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc298010070 con 0x7fc2a805cc30 2020-11-28T21:47:09.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.127+0000 7fc2a5ffb700 1 --2- 172.21.15.162:0/3338925990 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc28c0554e0 0x7fc28c0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:09.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.127+0000 7fc2a5ffb700 1 -- 172.21.15.162:0/3338925990 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5607+0+0 (secure 0 0 0) 0x7fc298004020 con 0x7fc2a805cc30 2020-11-28T21:47:09.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.127+0000 7fc2a7fff700 1 --2- 172.21.15.162:0/3338925990 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc28c0554e0 0x7fc28c0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:09.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.127+0000 7fc2a7fff700 1 --2- 172.21.15.162:0/3338925990 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc28c0554e0 0x7fc28c0579a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fc29000b000 tx=0x7fc290009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:09.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.131+0000 7fc2a5ffb700 1 -- 172.21.15.162:0/3338925990 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc298043780 con 0x7fc2a805cc30 2020-11-28T21:47:09.274 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.271+0000 7fc2af020700 1 -- 172.21.15.162:0/3338925990 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fc2a8057050 con 0x7fc2a805cc30 2020-11-28T21:47:09.993 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.987+0000 7fc2a5ffb700 1 -- 172.21.15.162:0/3338925990 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v31) v1 ==== 125+0+0 (secure 0 0 0) 0x7fc298046da0 con 0x7fc2a805cc30 2020-11-28T21:47:09.994 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.991+0000 7fc2af020700 1 -- 172.21.15.162:0/3338925990 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc28c0554e0 msgr2=0x7fc28c0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:09.994 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.991+0000 7fc2af020700 1 --2- 172.21.15.162:0/3338925990 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc28c0554e0 0x7fc28c0579a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fc29000b000 tx=0x7fc290009040).stop 2020-11-28T21:47:09.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.991+0000 7fc2af020700 1 -- 172.21.15.162:0/3338925990 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2a805cc30 msgr2=0x7fc2a80a9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:09.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.991+0000 7fc2af020700 1 --2- 172.21.15.162:0/3338925990 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2a805cc30 0x7fc2a80a9960 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7fc2a80a9e80 tx=0x7fc298007e30).stop 2020-11-28T21:47:09.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.991+0000 7fc2af020700 1 -- 172.21.15.162:0/3338925990 shutdown_connections 2020-11-28T21:47:09.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.991+0000 7fc2af020700 1 --2- 172.21.15.162:0/3338925990 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc28c0554e0 0x7fc28c0579a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:09.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.991+0000 7fc2af020700 1 --2- 172.21.15.162:0/3338925990 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc2a80b2370 0x7fc2a8152050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:09.996 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.991+0000 7fc2af020700 1 --2- 172.21.15.162:0/3338925990 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2a805cc30 0x7fc2a80a9960 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:09.996 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.991+0000 7fc2af020700 1 --2- 172.21.15.162:0/3338925990 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc2a80b0140 0x7fc2a8151b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:09.996 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.991+0000 7fc2af020700 1 -- 172.21.15.162:0/3338925990 >> 172.21.15.162:0/3338925990 conn(0x7fc2a8063140 msgr2=0x7fc2a813da70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:09.997 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.991+0000 7fc2af020700 1 -- 172.21.15.162:0/3338925990 shutdown_connections 2020-11-28T21:47:09.997 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:09.995+0000 7fc2af020700 1 -- 172.21.15.162:0/3338925990 wait complete. 2020-11-28T21:47:10.002 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_data' created 2020-11-28T21:47:10.013 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2020-11-28T21:47:10.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.123+0000 7fe316f0d700 1 Processor -- start 2020-11-28T21:47:10.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.123+0000 7fe316f0d700 1 -- start start 2020-11-28T21:47:10.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.127+0000 7fe316f0d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe3100af6a0 0x7fe3100b2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:10.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.127+0000 7fe316f0d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe310060a70 0x7fe3100afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:10.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.127+0000 7fe316f0d700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe310060ee0 con 0x7fe3100b1eb0 2020-11-28T21:47:10.129 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.127+0000 7fe316f0d700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3101471f0 con 0x7fe3100af6a0 2020-11-28T21:47:10.129 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.127+0000 7fe316f0d700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe310060090 con 0x7fe310060a70 2020-11-28T21:47:10.129 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.127+0000 7fe31670c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe310060a70 0x7fe3100afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:10.129 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.127+0000 7fe315f0b700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fe3100b1eb0 0x7fe310144350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:45440/0 (socket says 172.21.15.162:45440) 2020-11-28T21:47:10.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.127+0000 7fe315f0b700 1 -- 172.21.15.162:0/1373050805 learned_addr learned my addr 172.21.15.162:0/1373050805 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:47:10.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.127+0000 7fe31570a700 1 --2- 172.21.15.162:0/1373050805 >> v2:172.21.15.118:3300/0 conn(0x7fe3100af6a0 0x7fe3100b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:10.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.127+0000 7fe31670c700 1 -- 172.21.15.162:0/1373050805 >> v2:172.21.15.118:3300/0 conn(0x7fe3100af6a0 msgr2=0x7fe3100b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:10.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.127+0000 7fe31670c700 1 --2- 172.21.15.162:0/1373050805 >> v2:172.21.15.118:3300/0 conn(0x7fe3100af6a0 0x7fe3100b2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:10.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.127+0000 7fe31670c700 1 -- 172.21.15.162:0/1373050805 >> v1:172.21.15.162:6789/0 conn(0x7fe3100b1eb0 legacy=0x7fe310144350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:10.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.127+0000 7fe31670c700 1 -- 172.21.15.162:0/1373050805 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe310060330 con 0x7fe310060a70 2020-11-28T21:47:10.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.127+0000 7fe314f09700 1 -- 172.21.15.162:0/1373050805 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1639830545 0 0) 0x7fe310060ee0 con 0x7fe3100b1eb0 2020-11-28T21:47:10.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.127+0000 7fe31570a700 1 --2- 172.21.15.162:0/1373050805 >> v2:172.21.15.118:3300/0 conn(0x7fe3100af6a0 0x7fe3100b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:47:10.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.127+0000 7fe31670c700 1 --2- 172.21.15.162:0/1373050805 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe310060a70 0x7fe3100afdc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7fe30c002f30 tx=0x7fe30c006dc0).ready entity=mon.1 client_cookie=6634d73a956c617 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:10.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.127+0000 7fe314f09700 1 -- 172.21.15.162:0/1373050805 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe30c0097f0 con 0x7fe310060a70 2020-11-28T21:47:10.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.127+0000 7fe314f09700 1 -- 172.21.15.162:0/1373050805 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe30c009950 con 0x7fe310060a70 2020-11-28T21:47:10.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.127+0000 7fe316f0d700 1 -- 172.21.15.162:0/1373050805 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe310060a70 msgr2=0x7fe3100afdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:10.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.127+0000 7fe316f0d700 1 --2- 172.21.15.162:0/1373050805 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe310060a70 0x7fe3100afdc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7fe30c002f30 tx=0x7fe30c006dc0).stop 2020-11-28T21:47:10.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.127+0000 7fe314f09700 1 -- 172.21.15.162:0/1373050805 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe30c009cb0 con 0x7fe310060a70 2020-11-28T21:47:10.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.127+0000 7fe316f0d700 1 -- 172.21.15.162:0/1373050805 shutdown_connections 2020-11-28T21:47:10.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.127+0000 7fe316f0d700 1 --2- 172.21.15.162:0/1373050805 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe310060a70 0x7fe3100afdc0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:10.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.127+0000 7fe316f0d700 1 --2- 172.21.15.162:0/1373050805 >> v2:172.21.15.118:3300/0 conn(0x7fe3100af6a0 0x7fe3100b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:10.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.127+0000 7fe316f0d700 1 -- 172.21.15.162:0/1373050805 >> 172.21.15.162:0/1373050805 conn(0x7fe3100b57d0 msgr2=0x7fe3100b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:10.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.127+0000 7fe316f0d700 1 -- 172.21.15.162:0/1373050805 shutdown_connections 2020-11-28T21:47:10.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.127+0000 7fe316f0d700 1 -- 172.21.15.162:0/1373050805 wait complete. 2020-11-28T21:47:10.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.127+0000 7fe316f0d700 1 Processor -- start 2020-11-28T21:47:10.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.131+0000 7fe316f0d700 1 -- start start 2020-11-28T21:47:10.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.131+0000 7fe316f0d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe310060a70 0x7fe3101499c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:10.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.131+0000 7fe316f0d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe3100af6a0 0x7fe3101599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:10.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.131+0000 7fe316f0d700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe3100b1eb0 0x7fe3101484e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:10.138 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.131+0000 7fe316f0d700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3100ab3e0 con 0x7fe3100af6a0 2020-11-28T21:47:10.138 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.131+0000 7fe316f0d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3100ab610 con 0x7fe310060a70 2020-11-28T21:47:10.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.131+0000 7fe316f0d700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe31015a1e0 con 0x7fe3100b1eb0 2020-11-28T21:47:10.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.131+0000 7fe315f0b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe310060a70 0x7fe3101499c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:10.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.131+0000 7fe315f0b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe310060a70 0x7fe3101499c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38604/0 (socket says 172.21.15.162:38604) 2020-11-28T21:47:10.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.131+0000 7fe315f0b700 1 -- 172.21.15.162:0/3415607524 learned_addr learned my addr 172.21.15.162:0/3415607524 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:10.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.131+0000 7fe31670c700 1 --2- 172.21.15.162:0/3415607524 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe3100b1eb0 0x7fe3101484e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:10.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.131+0000 7fe31570a700 1 --2- 172.21.15.162:0/3415607524 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe3100af6a0 0x7fe3101599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:10.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.131+0000 7fe315f0b700 1 -- 172.21.15.162:0/3415607524 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe3100b1eb0 msgr2=0x7fe3101484e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:10.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.131+0000 7fe315f0b700 1 --2- 172.21.15.162:0/3415607524 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe3100b1eb0 0x7fe3101484e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:10.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.131+0000 7fe315f0b700 1 -- 172.21.15.162:0/3415607524 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe3100af6a0 msgr2=0x7fe3101599a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:10.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.131+0000 7fe315f0b700 1 --2- 172.21.15.162:0/3415607524 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe3100af6a0 0x7fe3101599a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:10.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.131+0000 7fe315f0b700 1 -- 172.21.15.162:0/3415607524 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3100ab0c0 con 0x7fe310060a70 2020-11-28T21:47:10.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.131+0000 7fe31570a700 1 --2- 172.21.15.162:0/3415607524 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe3100af6a0 0x7fe3101599a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:47:10.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.131+0000 7fe315f0b700 1 --2- 172.21.15.162:0/3415607524 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe310060a70 0x7fe3101499c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7fe300000e90 tx=0x7fe300007cb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:10.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.131+0000 7fe2feffd700 1 -- 172.21.15.162:0/3415607524 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3000072e0 con 0x7fe310060a70 2020-11-28T21:47:10.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.131+0000 7fe2feffd700 1 -- 172.21.15.162:0/3415607524 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe300007440 con 0x7fe310060a70 2020-11-28T21:47:10.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.131+0000 7fe316f0d700 1 -- 172.21.15.162:0/3415607524 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3100ab960 con 0x7fe310060a70 2020-11-28T21:47:10.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.131+0000 7fe2feffd700 1 -- 172.21.15.162:0/3415607524 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3000057d0 con 0x7fe310060a70 2020-11-28T21:47:10.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.131+0000 7fe316f0d700 1 -- 172.21.15.162:0/3415607524 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe31014b170 con 0x7fe310060a70 2020-11-28T21:47:10.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.131+0000 7fe316f0d700 1 -- 172.21.15.162:0/3415607524 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe310057050 con 0x7fe310060a70 2020-11-28T21:47:10.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.135+0000 7fe2feffd700 1 -- 172.21.15.162:0/3415607524 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe300010070 con 0x7fe310060a70 2020-11-28T21:47:10.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.135+0000 7fe2feffd700 1 --2- 172.21.15.162:0/3415607524 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe2f0055440 0x7fe2f0057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:10.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.135+0000 7fe2feffd700 1 -- 172.21.15.162:0/3415607524 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5957+0+0 (secure 0 0 0) 0x7fe30006d500 con 0x7fe310060a70 2020-11-28T21:47:10.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.135+0000 7fe31570a700 1 --2- 172.21.15.162:0/3415607524 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe2f0055440 0x7fe2f0057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:10.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.135+0000 7fe31570a700 1 --2- 172.21.15.162:0/3415607524 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe2f0055440 0x7fe2f0057900 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fe30800d000 tx=0x7fe308008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:10.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.139+0000 7fe2feffd700 1 -- 172.21.15.162:0/3415607524 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe3000437d0 con 0x7fe310060a70 2020-11-28T21:47:10.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.327+0000 7fe316f0d700 1 -- 172.21.15.162:0/3415607524 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fe3100af1a0 con 0x7fe310060a70 2020-11-28T21:47:10.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.339+0000 7fe2feffd700 1 -- 172.21.15.162:0/3415607524 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 5 and data pool 6 v9) v1 ==== 187+0+0 (secure 0 0 0) 0x7fe300007dd0 con 0x7fe310060a70 2020-11-28T21:47:10.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.347+0000 7fe316f0d700 1 -- 172.21.15.162:0/3415607524 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe2f0055440 msgr2=0x7fe2f0057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:10.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.347+0000 7fe316f0d700 1 --2- 172.21.15.162:0/3415607524 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe2f0055440 0x7fe2f0057900 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fe30800d000 tx=0x7fe308008040).stop 2020-11-28T21:47:10.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.347+0000 7fe316f0d700 1 -- 172.21.15.162:0/3415607524 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe310060a70 msgr2=0x7fe3101499c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:10.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.347+0000 7fe316f0d700 1 --2- 172.21.15.162:0/3415607524 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe310060a70 0x7fe3101499c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7fe300000e90 tx=0x7fe300007cb0).stop 2020-11-28T21:47:10.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.347+0000 7fe316f0d700 1 -- 172.21.15.162:0/3415607524 shutdown_connections 2020-11-28T21:47:10.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.347+0000 7fe316f0d700 1 --2- 172.21.15.162:0/3415607524 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe2f0055440 0x7fe2f0057900 secure :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0x7fe30800d000 tx=0x7fe308008040).stop 2020-11-28T21:47:10.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.347+0000 7fe316f0d700 1 --2- 172.21.15.162:0/3415607524 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe3100b1eb0 0x7fe3101484e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:10.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.347+0000 7fe316f0d700 1 --2- 172.21.15.162:0/3415607524 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe310060a70 0x7fe3101499c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:10.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.347+0000 7fe316f0d700 1 --2- 172.21.15.162:0/3415607524 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe3100af6a0 0x7fe3101599a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:10.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.347+0000 7fe316f0d700 1 -- 172.21.15.162:0/3415607524 >> 172.21.15.162:0/3415607524 conn(0x7fe3100b57d0 msgr2=0x7fe3100abd70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:10.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.351+0000 7fe316f0d700 1 -- 172.21.15.162:0/3415607524 shutdown_connections 2020-11-28T21:47:10.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.351+0000 7fe316f0d700 1 -- 172.21.15.162:0/3415607524 wait complete. 2020-11-28T21:47:10.353 INFO:teuthology.orchestra.run.smithi162.stderr:new fs with metadata pool 5 and data pool 6 2020-11-28T21:47:10.365 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:47:10.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.467+0000 7f9b8c9c8700 1 Processor -- start 2020-11-28T21:47:10.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.467+0000 7f9b8c9c8700 1 -- start start 2020-11-28T21:47:10.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.467+0000 7f9b8c9c8700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9b88062cd0 0x7f9b881406c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:10.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.467+0000 7f9b8c9c8700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b88140bb0 con 0x7f9b880664a0 2020-11-28T21:47:10.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.467+0000 7f9b8c9c8700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b880b3230 con 0x7f9b880b3a50 2020-11-28T21:47:10.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.467+0000 7f9b8c9c8700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b88144bf0 con 0x7f9b88062cd0 2020-11-28T21:47:10.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.467+0000 7f9b877fe700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f9b880b3a50 0x7f9b881444e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:45454/0 (socket says 172.21.15.162:45454) 2020-11-28T21:47:10.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.467+0000 7f9b877fe700 1 -- 172.21.15.162:0/20780461 learned_addr learned my addr 172.21.15.162:0/20780461 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:47:10.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.467+0000 7f9b86ffd700 1 --2- 172.21.15.162:0/20780461 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9b88062cd0 0x7f9b881406c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:10.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.467+0000 7f9b867fc700 1 -- 172.21.15.162:0/20780461 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3924005948 0 0) 0x7f9b880b3230 con 0x7f9b880b3a50 2020-11-28T21:47:10.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.467+0000 7f9b867fc700 1 -- 172.21.15.162:0/20780461 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b74003230 con 0x7f9b880b3a50 2020-11-28T21:47:10.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.467+0000 7f9b867fc700 1 -- 172.21.15.162:0/20780461 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 883057726 0 0) 0x7f9b74003230 con 0x7f9b880b3a50 2020-11-28T21:47:10.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.467+0000 7f9b867fc700 1 -- 172.21.15.162:0/20780461 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9b88062cd0 msgr2=0x7f9b881406c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:10.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.467+0000 7f9b867fc700 1 --2- 172.21.15.162:0/20780461 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9b88062cd0 0x7f9b881406c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:10.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.467+0000 7f9b867fc700 1 -- 172.21.15.162:0/20780461 >> v1:172.21.15.118:6789/0 conn(0x7f9b880664a0 legacy=0x7f9b88066910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:10.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.467+0000 7f9b867fc700 1 -- 172.21.15.162:0/20780461 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b88143da0 con 0x7f9b880b3a50 2020-11-28T21:47:10.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.467+0000 7f9b867fc700 1 -- 172.21.15.162:0/20780461 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f9b78001a50 con 0x7f9b880b3a50 2020-11-28T21:47:10.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.467+0000 7f9b867fc700 1 -- 172.21.15.162:0/20780461 >> v1:172.21.15.162:6789/0 conn(0x7f9b880b3a50 legacy=0x7f9b881444e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:10.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.467+0000 7f9b867fc700 1 --2- 172.21.15.162:0/20780461 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b74003720 0x7f9b74003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:10.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.467+0000 7f9b867fc700 1 --2- 172.21.15.162:0/20780461 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9b88062cd0 0x7f9b740044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:10.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.467+0000 7f9b867fc700 1 --2- 172.21.15.162:0/20780461 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b740049c0 0x7f9b74004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:10.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.467+0000 7f9b867fc700 1 -- 172.21.15.162:0/20780461 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b88144bf0 con 0x7f9b740049c0 2020-11-28T21:47:10.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.467+0000 7f9b867fc700 1 -- 172.21.15.162:0/20780461 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b740041c0 con 0x7f9b74003720 2020-11-28T21:47:10.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.467+0000 7f9b867fc700 1 -- 172.21.15.162:0/20780461 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7400b720 con 0x7f9b88062cd0 2020-11-28T21:47:10.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.467+0000 7f9b877fe700 1 --2- 172.21.15.162:0/20780461 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b74003720 0x7f9b74003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:10.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.467+0000 7f9b87fff700 1 --2- 172.21.15.162:0/20780461 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b740049c0 0x7f9b74004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:10.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.467+0000 7f9b86ffd700 1 --2- 172.21.15.162:0/20780461 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9b88062cd0 0x7f9b740044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:10.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.467+0000 7f9b877fe700 1 -- 172.21.15.162:0/20780461 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9b88062cd0 msgr2=0x7f9b740044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:10.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.471+0000 7f9b877fe700 1 --2- 172.21.15.162:0/20780461 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9b88062cd0 0x7f9b740044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:10.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.471+0000 7f9b877fe700 1 -- 172.21.15.162:0/20780461 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b740049c0 msgr2=0x7f9b74004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:10.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.471+0000 7f9b877fe700 1 --2- 172.21.15.162:0/20780461 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b740049c0 0x7f9b74004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:10.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.471+0000 7f9b877fe700 1 -- 172.21.15.162:0/20780461 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b74003230 con 0x7f9b74003720 2020-11-28T21:47:10.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.471+0000 7f9b87fff700 1 --2- 172.21.15.162:0/20780461 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b740049c0 0x7f9b74004e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:47:10.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.471+0000 7f9b86ffd700 1 --2- 172.21.15.162:0/20780461 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9b88062cd0 0x7f9b740044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:47:10.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.471+0000 7f9b877fe700 1 --2- 172.21.15.162:0/20780461 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b74003720 0x7f9b74003b40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f9b78001760 tx=0x7f9b78007af0).ready entity=mon.1 client_cookie=556adb4e5f1dee68 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:10.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.471+0000 7f9b867fc700 1 -- 172.21.15.162:0/20780461 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b78005b70 con 0x7f9b74003720 2020-11-28T21:47:10.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.471+0000 7f9b867fc700 1 -- 172.21.15.162:0/20780461 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b78005cd0 con 0x7f9b74003720 2020-11-28T21:47:10.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.471+0000 7f9b867fc700 1 -- 172.21.15.162:0/20780461 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b780072c0 con 0x7f9b74003720 2020-11-28T21:47:10.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.471+0000 7f9b8c9c8700 1 -- 172.21.15.162:0/20780461 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b74003720 msgr2=0x7f9b74003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:10.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.471+0000 7f9b8c9c8700 1 --2- 172.21.15.162:0/20780461 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b74003720 0x7f9b74003b40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f9b78001760 tx=0x7f9b78007af0).stop 2020-11-28T21:47:10.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.471+0000 7f9b87fff700 1 -- 172.21.15.162:0/20780461 reap_dead start 2020-11-28T21:47:10.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.471+0000 7f9b8c9c8700 1 -- 172.21.15.162:0/20780461 shutdown_connections 2020-11-28T21:47:10.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.471+0000 7f9b8c9c8700 1 -- 172.21.15.162:0/20780461 >> 172.21.15.162:0/20780461 conn(0x7f9b8805b970 msgr2=0x7f9b8805bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:10.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.471+0000 7f9b8c9c8700 1 -- 172.21.15.162:0/20780461 shutdown_connections 2020-11-28T21:47:10.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.471+0000 7f9b8c9c8700 1 -- 172.21.15.162:0/20780461 wait complete. 2020-11-28T21:47:10.485 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.471+0000 7f9b8c9c8700 1 Processor -- start 2020-11-28T21:47:10.485 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.471+0000 7f9b8c9c8700 1 -- start start 2020-11-28T21:47:10.486 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.475+0000 7f9b8c9c8700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b88140030 0x7f9b880abfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:10.486 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.475+0000 7f9b8c9c8700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9b8805f380 0x7f9b880606e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:10.486 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.475+0000 7f9b8c9c8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b880ac8a0 0x7f9b880acd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:10.487 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.475+0000 7f9b8c9c8700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b880aaa30 con 0x7f9b88140030 2020-11-28T21:47:10.487 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.475+0000 7f9b8c9c8700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b880ac4d0 con 0x7f9b880ac8a0 2020-11-28T21:47:10.487 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.475+0000 7f9b8c9c8700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b88065530 con 0x7f9b8805f380 2020-11-28T21:47:10.487 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.475+0000 7f9b87fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b880ac8a0 0x7f9b880acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:10.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.475+0000 7f9b87fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b880ac8a0 0x7f9b880acd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38626/0 (socket says 172.21.15.162:38626) 2020-11-28T21:47:10.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.475+0000 7f9b87fff700 1 -- 172.21.15.162:0/801469037 learned_addr learned my addr 172.21.15.162:0/801469037 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:10.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.475+0000 7f9b877fe700 1 --2- 172.21.15.162:0/801469037 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b88140030 0x7f9b880abfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:10.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.475+0000 7f9b86ffd700 1 --2- 172.21.15.162:0/801469037 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9b8805f380 0x7f9b880606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:10.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.475+0000 7f9b87fff700 1 -- 172.21.15.162:0/801469037 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9b8805f380 msgr2=0x7f9b880606e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:10.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.475+0000 7f9b87fff700 1 --2- 172.21.15.162:0/801469037 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9b8805f380 0x7f9b880606e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:10.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.475+0000 7f9b87fff700 1 -- 172.21.15.162:0/801469037 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b88140030 msgr2=0x7f9b880abfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:10.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.475+0000 7f9b87fff700 1 --2- 172.21.15.162:0/801469037 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b88140030 0x7f9b880abfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:10.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.475+0000 7f9b87fff700 1 -- 172.21.15.162:0/801469037 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b88060bd0 con 0x7f9b880ac8a0 2020-11-28T21:47:10.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.475+0000 7f9b87fff700 1 --2- 172.21.15.162:0/801469037 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b880ac8a0 0x7f9b880acd30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f9b880aab70 tx=0x7f9b80009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:10.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.475+0000 7f9b84ff9700 1 -- 172.21.15.162:0/801469037 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b80013420 con 0x7f9b880ac8a0 2020-11-28T21:47:10.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.475+0000 7f9b84ff9700 1 -- 172.21.15.162:0/801469037 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b80013580 con 0x7f9b880ac8a0 2020-11-28T21:47:10.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.475+0000 7f9b8c9c8700 1 -- 172.21.15.162:0/801469037 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b8805f890 con 0x7f9b880ac8a0 2020-11-28T21:47:10.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.475+0000 7f9b84ff9700 1 -- 172.21.15.162:0/801469037 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b800138e0 con 0x7f9b880ac8a0 2020-11-28T21:47:10.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.475+0000 7f9b8c9c8700 1 -- 172.21.15.162:0/801469037 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b88140800 con 0x7f9b880ac8a0 2020-11-28T21:47:10.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.475+0000 7f9b8c9c8700 1 -- 172.21.15.162:0/801469037 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b88057050 con 0x7f9b880ac8a0 2020-11-28T21:47:10.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.475+0000 7f9b84ff9700 1 -- 172.21.15.162:0/801469037 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9b80013a40 con 0x7f9b880ac8a0 2020-11-28T21:47:10.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.475+0000 7f9b84ff9700 1 --2- 172.21.15.162:0/801469037 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9b680554e0 0x7f9b680579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:10.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.479+0000 7f9b84ff9700 1 -- 172.21.15.162:0/801469037 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9b8006d520 con 0x7f9b880ac8a0 2020-11-28T21:47:10.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.479+0000 7f9b877fe700 1 --2- 172.21.15.162:0/801469037 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9b680554e0 0x7f9b680579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:10.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.479+0000 7f9b877fe700 1 --2- 172.21.15.162:0/801469037 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9b680554e0 0x7f9b680579a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f9b7801a000 tx=0x7f9b78015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:10.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.483+0000 7f9b84ff9700 1 -- 172.21.15.162:0/801469037 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9b80043820 con 0x7f9b880ac8a0 2020-11-28T21:47:10.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.623+0000 7f9b8c9c8700 1 -- 172.21.15.162:0/801469037 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9b88140bb0 con 0x7f9b880ac8a0 2020-11-28T21:47:10.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.627+0000 7f9b84ff9700 1 -- 172.21.15.162:0/801469037 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16147 (secure 0 0 0) 0x7f9b88140bb0 con 0x7f9b880ac8a0 2020-11-28T21:47:10.629 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:47:10.629 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":32,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:47:10.330274+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:47:08.251301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2020-11-28T21:47:09.333148+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:47:10.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.627+0000 7f9b8c9c8700 1 -- 172.21.15.162:0/801469037 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9b680554e0 msgr2=0x7f9b680579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:10.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.627+0000 7f9b8c9c8700 1 --2- 172.21.15.162:0/801469037 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9b680554e0 0x7f9b680579a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f9b7801a000 tx=0x7f9b78015000).stop 2020-11-28T21:47:10.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.627+0000 7f9b8c9c8700 1 -- 172.21.15.162:0/801469037 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b880ac8a0 msgr2=0x7f9b880acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:10.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.627+0000 7f9b8c9c8700 1 --2- 172.21.15.162:0/801469037 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b880ac8a0 0x7f9b880acd30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f9b880aab70 tx=0x7f9b80009040).stop 2020-11-28T21:47:10.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.627+0000 7f9b8c9c8700 1 -- 172.21.15.162:0/801469037 shutdown_connections 2020-11-28T21:47:10.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.627+0000 7f9b8c9c8700 1 --2- 172.21.15.162:0/801469037 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9b680554e0 0x7f9b680579a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:10.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.627+0000 7f9b8c9c8700 1 --2- 172.21.15.162:0/801469037 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b880ac8a0 0x7f9b880acd30 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:10.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.627+0000 7f9b8c9c8700 1 --2- 172.21.15.162:0/801469037 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b88140030 0x7f9b880abfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:10.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.627+0000 7f9b8c9c8700 1 --2- 172.21.15.162:0/801469037 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9b8805f380 0x7f9b880606e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:10.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.627+0000 7f9b8c9c8700 1 -- 172.21.15.162:0/801469037 >> 172.21.15.162:0/801469037 conn(0x7f9b8805b970 msgr2=0x7f9b8813c090 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:10.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.627+0000 7f9b8c9c8700 1 -- 172.21.15.162:0/801469037 shutdown_connections 2020-11-28T21:47:10.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.627+0000 7f9b8c9c8700 1 -- 172.21.15.162:0/801469037 wait complete. 2020-11-28T21:47:10.646 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:47:10.755 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.751+0000 7f3f4359e700 1 Processor -- start 2020-11-28T21:47:10.755 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.751+0000 7f3f4359e700 1 -- start start 2020-11-28T21:47:10.756 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.755+0000 7f3f4359e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3f44098b30 0x7f3f440994f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:10.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.755+0000 7f3f4359e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3f440b0000 0x7f3f44060000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:10.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.755+0000 7f3f4359e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f44098f10 con 0x7f3f440604f0 2020-11-28T21:47:10.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.755+0000 7f3f4359e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f440b1030 con 0x7f3f44098b30 2020-11-28T21:47:10.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.755+0000 7f3f4359e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f440b0480 con 0x7f3f440b0000 2020-11-28T21:47:10.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.755+0000 7f3f4259c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3f44098b30 0x7f3f440994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:10.758 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.755+0000 7f3f4259c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3f44098b30 0x7f3f440994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38634/0 (socket says 172.21.15.162:38634) 2020-11-28T21:47:10.758 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.755+0000 7f3f39d9b700 1 --2- 172.21.15.162:0/3885420273 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3f440b0000 0x7f3f44060000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:10.758 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.755+0000 7f3f4259c700 1 -- 172.21.15.162:0/3885420273 learned_addr learned my addr 172.21.15.162:0/3885420273 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:10.758 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.755+0000 7f3f4259c700 1 -- 172.21.15.162:0/3885420273 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3f440b0000 msgr2=0x7f3f44060000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:10.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.755+0000 7f3f4259c700 1 --2- 172.21.15.162:0/3885420273 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3f440b0000 0x7f3f44060000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:10.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.755+0000 7f3f4259c700 1 -- 172.21.15.162:0/3885420273 >> v1:172.21.15.118:6789/0 conn(0x7f3f440604f0 legacy=0x7f3f4414a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:10.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.755+0000 7f3f4259c700 1 -- 172.21.15.162:0/3885420273 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f440b0680 con 0x7f3f44098b30 2020-11-28T21:47:10.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.755+0000 7f3f4259c700 1 --2- 172.21.15.162:0/3885420273 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f44098b30 0x7f3f440994f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f3f44099be0 tx=0x7f3f2c006e80).ready entity=mon.1 client_cookie=45d12860b6205b9b server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:10.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.755+0000 7f3f41d9b700 1 -- 172.21.15.162:0/3885420273 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f2c0097b0 con 0x7f3f44098b30 2020-11-28T21:47:10.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.755+0000 7f3f41d9b700 1 -- 172.21.15.162:0/3885420273 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f2c009910 con 0x7f3f44098b30 2020-11-28T21:47:10.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.755+0000 7f3f4359e700 1 -- 172.21.15.162:0/3885420273 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f44098b30 msgr2=0x7f3f440994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:10.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.755+0000 7f3f41d9b700 1 -- 172.21.15.162:0/3885420273 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f2c009c70 con 0x7f3f44098b30 2020-11-28T21:47:10.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.755+0000 7f3f4359e700 1 --2- 172.21.15.162:0/3885420273 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f44098b30 0x7f3f440994f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f3f44099be0 tx=0x7f3f2c006e80).stop 2020-11-28T21:47:10.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.755+0000 7f3f4359e700 1 -- 172.21.15.162:0/3885420273 shutdown_connections 2020-11-28T21:47:10.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.755+0000 7f3f4359e700 1 --2- 172.21.15.162:0/3885420273 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f44098b30 0x7f3f440994f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:10.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.755+0000 7f3f4359e700 1 --2- 172.21.15.162:0/3885420273 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3f440b0000 0x7f3f44060000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:10.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.755+0000 7f3f4359e700 1 -- 172.21.15.162:0/3885420273 >> 172.21.15.162:0/3885420273 conn(0x7f3f4405b970 msgr2=0x7f3f4405bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:10.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.755+0000 7f3f4359e700 1 -- 172.21.15.162:0/3885420273 shutdown_connections 2020-11-28T21:47:10.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.755+0000 7f3f4359e700 1 -- 172.21.15.162:0/3885420273 wait complete. 2020-11-28T21:47:10.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.755+0000 7f3f4359e700 1 Processor -- start 2020-11-28T21:47:10.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.759+0000 7f3f4359e700 1 -- start start 2020-11-28T21:47:10.765 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.759+0000 7f3f4359e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3f440604f0 0x7f3f44147d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:10.765 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.759+0000 7f3f4359e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f44098b30 0x7f3f44159ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:10.765 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.759+0000 7f3f4359e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3f440b0000 0x7f3f44159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:10.765 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.759+0000 7f3f4359e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f440ae240 con 0x7f3f440b0000 2020-11-28T21:47:10.765 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.759+0000 7f3f4359e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f440ae4e0 con 0x7f3f44098b30 2020-11-28T21:47:10.766 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.759+0000 7f3f4359e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f44159640 con 0x7f3f440604f0 2020-11-28T21:47:10.766 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.759+0000 7f3f39d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f44098b30 0x7f3f44159ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:10.766 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.759+0000 7f3f39d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f44098b30 0x7f3f44159ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38640/0 (socket says 172.21.15.162:38640) 2020-11-28T21:47:10.766 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.759+0000 7f3f39d9b700 1 -- 172.21.15.162:0/1652077320 learned_addr learned my addr 172.21.15.162:0/1652077320 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:10.766 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.759+0000 7f3f4259c700 1 --2- 172.21.15.162:0/1652077320 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3f440604f0 0x7f3f44147d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:10.767 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.759+0000 7f3f42d9d700 1 --2- 172.21.15.162:0/1652077320 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3f440b0000 0x7f3f44159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:10.767 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.759+0000 7f3f39d9b700 1 -- 172.21.15.162:0/1652077320 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3f440604f0 msgr2=0x7f3f44147d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:10.767 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.759+0000 7f3f39d9b700 1 --2- 172.21.15.162:0/1652077320 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3f440604f0 0x7f3f44147d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:10.767 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.759+0000 7f3f39d9b700 1 -- 172.21.15.162:0/1652077320 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3f440b0000 msgr2=0x7f3f44159fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:10.767 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.759+0000 7f3f39d9b700 1 --2- 172.21.15.162:0/1652077320 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3f440b0000 0x7f3f44159fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:10.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.759+0000 7f3f39d9b700 1 -- 172.21.15.162:0/1652077320 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f44148650 con 0x7f3f44098b30 2020-11-28T21:47:10.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.759+0000 7f3f4259c700 1 --2- 172.21.15.162:0/1652077320 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3f440604f0 0x7f3f44147d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:47:10.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.759+0000 7f3f42d9d700 1 --2- 172.21.15.162:0/1652077320 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3f440b0000 0x7f3f44159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:47:10.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.759+0000 7f3f39d9b700 1 --2- 172.21.15.162:0/1652077320 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f44098b30 0x7f3f44159ae0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f3f3400c000 tx=0x7f3f34006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:10.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.759+0000 7f3f3bfff700 1 -- 172.21.15.162:0/1652077320 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f3400ca10 con 0x7f3f44098b30 2020-11-28T21:47:10.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.759+0000 7f3f4359e700 1 -- 172.21.15.162:0/1652077320 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f44066120 con 0x7f3f44098b30 2020-11-28T21:47:10.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.759+0000 7f3f4359e700 1 -- 172.21.15.162:0/1652077320 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f440add00 con 0x7f3f44098b30 2020-11-28T21:47:10.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.759+0000 7f3f3bfff700 1 -- 172.21.15.162:0/1652077320 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f3400cb70 con 0x7f3f44098b30 2020-11-28T21:47:10.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.759+0000 7f3f3bfff700 1 -- 172.21.15.162:0/1652077320 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f3401b410 con 0x7f3f44098b30 2020-11-28T21:47:10.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.759+0000 7f3f4359e700 1 -- 172.21.15.162:0/1652077320 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f200033e0 con 0x7f3f44098b30 2020-11-28T21:47:10.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.759+0000 7f3f3bfff700 1 -- 172.21.15.162:0/1652077320 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f3f340123f0 con 0x7f3f44098b30 2020-11-28T21:47:10.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.763+0000 7f3f3bfff700 1 --2- 172.21.15.162:0/1652077320 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3f1c055440 0x7f3f1c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:10.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.763+0000 7f3f3bfff700 1 -- 172.21.15.162:0/1652077320 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3f3406e200 con 0x7f3f44098b30 2020-11-28T21:47:10.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.763+0000 7f3f4259c700 1 --2- 172.21.15.162:0/1652077320 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3f1c055440 0x7f3f1c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:10.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.763+0000 7f3f4259c700 1 --2- 172.21.15.162:0/1652077320 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3f1c055440 0x7f3f1c057900 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f3f2c017000 tx=0x7f3f2c015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:10.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.767+0000 7f3f3bfff700 1 -- 172.21.15.162:0/1652077320 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f3f34044400 con 0x7f3f44098b30 2020-11-28T21:47:10.909 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.907+0000 7f3f4359e700 1 -- 172.21.15.162:0/1652077320 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3f20002270 con 0x7f3f44098b30 2020-11-28T21:47:10.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.907+0000 7f3f3bfff700 1 -- 172.21.15.162:0/1652077320 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16147 (secure 0 0 0) 0x7f3f34047a20 con 0x7f3f44098b30 2020-11-28T21:47:10.911 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:47:10.912 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":32,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:47:10.330274+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:47:08.251301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2020-11-28T21:47:09.333148+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:47:10.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.911+0000 7f3f4359e700 1 -- 172.21.15.162:0/1652077320 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3f1c055440 msgr2=0x7f3f1c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:10.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.911+0000 7f3f4359e700 1 --2- 172.21.15.162:0/1652077320 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3f1c055440 0x7f3f1c057900 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f3f2c017000 tx=0x7f3f2c015000).stop 2020-11-28T21:47:10.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.911+0000 7f3f4359e700 1 -- 172.21.15.162:0/1652077320 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f44098b30 msgr2=0x7f3f44159ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:10.914 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.911+0000 7f3f4359e700 1 --2- 172.21.15.162:0/1652077320 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f44098b30 0x7f3f44159ae0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f3f3400c000 tx=0x7f3f34006dc0).stop 2020-11-28T21:47:10.914 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.911+0000 7f3f4359e700 1 -- 172.21.15.162:0/1652077320 shutdown_connections 2020-11-28T21:47:10.914 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.911+0000 7f3f4359e700 1 --2- 172.21.15.162:0/1652077320 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3f1c055440 0x7f3f1c057900 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:10.914 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.911+0000 7f3f4359e700 1 --2- 172.21.15.162:0/1652077320 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3f440604f0 0x7f3f44147d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:10.915 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.911+0000 7f3f4359e700 1 --2- 172.21.15.162:0/1652077320 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3f440b0000 0x7f3f44159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:10.915 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.911+0000 7f3f4359e700 1 --2- 172.21.15.162:0/1652077320 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f44098b30 0x7f3f44159ae0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:10.915 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.911+0000 7f3f4359e700 1 -- 172.21.15.162:0/1652077320 >> 172.21.15.162:0/1652077320 conn(0x7f3f4405b970 msgr2=0x7f3f440b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:10.915 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.911+0000 7f3f4359e700 1 -- 172.21.15.162:0/1652077320 shutdown_connections 2020-11-28T21:47:10.915 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:10.911+0000 7f3f4359e700 1 -- 172.21.15.162:0/1652077320 wait complete. 2020-11-28T21:47:10.929 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2020-11-28T21:47:11.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.039+0000 7f3ba7a47700 1 Processor -- start 2020-11-28T21:47:11.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.039+0000 7f3ba7a47700 1 -- start start 2020-11-28T21:47:11.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.043+0000 7f3ba7a47700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f3ba00649c0 0x7f3ba014a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:11.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.043+0000 7f3ba7a47700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3ba00b0940 0x7f3ba005c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:11.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.043+0000 7f3ba7a47700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ba0064da0 con 0x7f3ba00b0f40 2020-11-28T21:47:11.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.043+0000 7f3ba7a47700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ba0062510 con 0x7f3ba00649c0 2020-11-28T21:47:11.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.043+0000 7f3ba7a47700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ba005ba10 con 0x7f3ba00b0940 2020-11-28T21:47:11.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.043+0000 7f3ba5fe4700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f3ba00b0f40 0x7f3ba0144030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:45492/0 (socket says 172.21.15.162:45492) 2020-11-28T21:47:11.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.043+0000 7f3ba4fe2700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3ba00b0940 0x7f3ba005c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:11.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.043+0000 7f3ba5fe4700 1 -- 172.21.15.162:0/827585818 learned_addr learned my addr 172.21.15.162:0/827585818 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:47:11.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.043+0000 7f3ba57e3700 1 --2- 172.21.15.162:0/827585818 >> v2:172.21.15.118:3300/0 conn(0x7f3ba00649c0 0x7f3ba014a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:11.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.043+0000 7f3ba4fe2700 1 -- 172.21.15.162:0/827585818 >> v2:172.21.15.118:3300/0 conn(0x7f3ba00649c0 msgr2=0x7f3ba014a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:11.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.043+0000 7f3ba4fe2700 1 --2- 172.21.15.162:0/827585818 >> v2:172.21.15.118:3300/0 conn(0x7f3ba00649c0 0x7f3ba014a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:11.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.043+0000 7f3ba4fe2700 1 -- 172.21.15.162:0/827585818 >> v1:172.21.15.162:6789/0 conn(0x7f3ba00b0f40 legacy=0x7f3ba0144030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:11.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.043+0000 7f3ba4fe2700 1 -- 172.21.15.162:0/827585818 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba014a870 con 0x7f3ba00b0940 2020-11-28T21:47:11.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.043+0000 7f3ba4fe2700 1 --2- 172.21.15.162:0/827585818 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ba00b0940 0x7f3ba005c110 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f3b90002f30 tx=0x7f3b90006dc0).ready entity=mon.1 client_cookie=29aa7146433cc1a server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:11.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.043+0000 7f3b97fff700 1 -- 172.21.15.162:0/827585818 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b900097b0 con 0x7f3ba00b0940 2020-11-28T21:47:11.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.043+0000 7f3b97fff700 1 -- 172.21.15.162:0/827585818 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b90009910 con 0x7f3ba00b0940 2020-11-28T21:47:11.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.043+0000 7f3ba7a47700 1 -- 172.21.15.162:0/827585818 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ba00b0940 msgr2=0x7f3ba005c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:11.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.043+0000 7f3b97fff700 1 -- 172.21.15.162:0/827585818 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b90009c70 con 0x7f3ba00b0940 2020-11-28T21:47:11.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.043+0000 7f3ba7a47700 1 --2- 172.21.15.162:0/827585818 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ba00b0940 0x7f3ba005c110 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f3b90002f30 tx=0x7f3b90006dc0).stop 2020-11-28T21:47:11.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.043+0000 7f3ba7a47700 1 -- 172.21.15.162:0/827585818 shutdown_connections 2020-11-28T21:47:11.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.043+0000 7f3ba7a47700 1 --2- 172.21.15.162:0/827585818 >> v2:172.21.15.118:3300/0 conn(0x7f3ba00649c0 0x7f3ba014a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:11.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.043+0000 7f3ba7a47700 1 --2- 172.21.15.162:0/827585818 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ba00b0940 0x7f3ba005c110 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:11.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.043+0000 7f3ba7a47700 1 -- 172.21.15.162:0/827585818 >> 172.21.15.162:0/827585818 conn(0x7f3ba0061740 msgr2=0x7f3ba0061b50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:11.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.043+0000 7f3ba7a47700 1 -- 172.21.15.162:0/827585818 shutdown_connections 2020-11-28T21:47:11.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.043+0000 7f3ba7a47700 1 -- 172.21.15.162:0/827585818 wait complete. 2020-11-28T21:47:11.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.047+0000 7f3ba7a47700 1 Processor -- start 2020-11-28T21:47:11.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.047+0000 7f3ba7a47700 1 -- start start 2020-11-28T21:47:11.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.047+0000 7f3ba7a47700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ba00649c0 0x7f3ba0149b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:11.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.047+0000 7f3ba7a47700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3ba00b0940 0x7f3ba00b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:11.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.047+0000 7f3ba7a47700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3ba00b0f40 0x7f3ba00aec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:11.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.047+0000 7f3ba7a47700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba0148b70 con 0x7f3ba00b0940 2020-11-28T21:47:11.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.047+0000 7f3ba7a47700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba0148da0 con 0x7f3ba00649c0 2020-11-28T21:47:11.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.047+0000 7f3ba7a47700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba0149040 con 0x7f3ba00b0f40 2020-11-28T21:47:11.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.047+0000 7f3ba57e3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ba00649c0 0x7f3ba0149b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:11.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.047+0000 7f3ba57e3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ba00649c0 0x7f3ba0149b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38650/0 (socket says 172.21.15.162:38650) 2020-11-28T21:47:11.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.047+0000 7f3ba57e3700 1 -- 172.21.15.162:0/306487769 learned_addr learned my addr 172.21.15.162:0/306487769 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:11.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.047+0000 7f3ba4fe2700 1 --2- 172.21.15.162:0/306487769 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3ba00b0940 0x7f3ba00b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:11.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.047+0000 7f3ba5fe4700 1 --2- 172.21.15.162:0/306487769 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3ba00b0f40 0x7f3ba00aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:11.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.047+0000 7f3ba57e3700 1 -- 172.21.15.162:0/306487769 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3ba00b0f40 msgr2=0x7f3ba00aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:11.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.047+0000 7f3ba57e3700 1 --2- 172.21.15.162:0/306487769 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3ba00b0f40 0x7f3ba00aec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:11.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.047+0000 7f3ba57e3700 1 -- 172.21.15.162:0/306487769 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3ba00b0940 msgr2=0x7f3ba00b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:11.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.047+0000 7f3ba57e3700 1 --2- 172.21.15.162:0/306487769 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3ba00b0940 0x7f3ba00b55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:11.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.047+0000 7f3ba57e3700 1 -- 172.21.15.162:0/306487769 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba01482d0 con 0x7f3ba00649c0 2020-11-28T21:47:11.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.047+0000 7f3ba4fe2700 1 --2- 172.21.15.162:0/306487769 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3ba00b0940 0x7f3ba00b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:47:11.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.047+0000 7f3ba57e3700 1 --2- 172.21.15.162:0/306487769 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ba00649c0 0x7f3ba0149b00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f3b9c00c000 tx=0x7f3b9c006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:11.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.047+0000 7f3b967fc700 1 -- 172.21.15.162:0/306487769 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b9c00ca80 con 0x7f3ba00649c0 2020-11-28T21:47:11.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.047+0000 7f3b967fc700 1 -- 172.21.15.162:0/306487769 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b9c00cbe0 con 0x7f3ba00649c0 2020-11-28T21:47:11.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.047+0000 7f3ba7a47700 1 -- 172.21.15.162:0/306487769 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ba00ad110 con 0x7f3ba00649c0 2020-11-28T21:47:11.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.047+0000 7f3b967fc700 1 -- 172.21.15.162:0/306487769 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b9c0123f0 con 0x7f3ba00649c0 2020-11-28T21:47:11.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.047+0000 7f3ba7a47700 1 -- 172.21.15.162:0/306487769 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba00af310 con 0x7f3ba00649c0 2020-11-28T21:47:11.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.047+0000 7f3ba7a47700 1 -- 172.21.15.162:0/306487769 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba0149200 con 0x7f3ba00649c0 2020-11-28T21:47:11.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.051+0000 7f3b967fc700 1 -- 172.21.15.162:0/306487769 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f3b9c012550 con 0x7f3ba00649c0 2020-11-28T21:47:11.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.051+0000 7f3b967fc700 1 --2- 172.21.15.162:0/306487769 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3b88055440 0x7f3b88057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:11.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.051+0000 7f3b967fc700 1 -- 172.21.15.162:0/306487769 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3b9c06e3a0 con 0x7f3ba00649c0 2020-11-28T21:47:11.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.051+0000 7f3ba4fe2700 1 --2- 172.21.15.162:0/306487769 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3b88055440 0x7f3b88057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:11.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.051+0000 7f3ba4fe2700 1 --2- 172.21.15.162:0/306487769 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3b88055440 0x7f3b88057900 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f3b90018000 tx=0x7f3b90015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:11.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.055+0000 7f3b967fc700 1 -- 172.21.15.162:0/306487769 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f3b9c044640 con 0x7f3ba00649c0 2020-11-28T21:47:11.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.247+0000 7f3ba7a47700 1 -- 172.21.15.162:0/306487769 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f3ba0057050 con 0x7f3ba00649c0 2020-11-28T21:47:11.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.339+0000 7f3b967fc700 1 -- 172.21.15.162:0/306487769 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v10) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3b9c047c60 con 0x7f3ba00649c0 2020-11-28T21:47:11.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.339+0000 7f3ba7a47700 1 -- 172.21.15.162:0/306487769 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3b88055440 msgr2=0x7f3b88057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:11.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.339+0000 7f3ba7a47700 1 --2- 172.21.15.162:0/306487769 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3b88055440 0x7f3b88057900 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f3b90018000 tx=0x7f3b90015000).stop 2020-11-28T21:47:11.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.339+0000 7f3ba7a47700 1 -- 172.21.15.162:0/306487769 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ba00649c0 msgr2=0x7f3ba0149b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:11.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.339+0000 7f3ba7a47700 1 --2- 172.21.15.162:0/306487769 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ba00649c0 0x7f3ba0149b00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f3b9c00c000 tx=0x7f3b9c006dc0).stop 2020-11-28T21:47:11.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.339+0000 7f3ba7a47700 1 -- 172.21.15.162:0/306487769 shutdown_connections 2020-11-28T21:47:11.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.339+0000 7f3ba7a47700 1 --2- 172.21.15.162:0/306487769 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3b88055440 0x7f3b88057900 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:11.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.339+0000 7f3ba7a47700 1 --2- 172.21.15.162:0/306487769 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3ba00b0f40 0x7f3ba00aec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:11.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.343+0000 7f3ba7a47700 1 --2- 172.21.15.162:0/306487769 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ba00649c0 0x7f3ba0149b00 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:11.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.343+0000 7f3ba7a47700 1 --2- 172.21.15.162:0/306487769 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3ba00b0940 0x7f3ba00b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:11.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.343+0000 7f3ba7a47700 1 -- 172.21.15.162:0/306487769 >> 172.21.15.162:0/306487769 conn(0x7f3ba0061740 msgr2=0x7f3ba00b3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:11.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.343+0000 7f3ba7a47700 1 -- 172.21.15.162:0/306487769 shutdown_connections 2020-11-28T21:47:11.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.343+0000 7f3ba7a47700 1 -- 172.21.15.162:0/306487769 wait complete. 2020-11-28T21:47:11.359 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:47:11.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.463+0000 7fe1d359e700 1 Processor -- start 2020-11-28T21:47:11.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.463+0000 7fe1d359e700 1 -- start start 2020-11-28T21:47:11.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.463+0000 7fe1d359e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe1d40b7920 0x7fe1d413f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:11.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.463+0000 7fe1d359e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe1d40b2da0 0x7fe1d413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:11.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.463+0000 7fe1d359e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1d4140360 con 0x7fe1d4066fa0 2020-11-28T21:47:11.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.463+0000 7fe1d359e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1d40b7d00 con 0x7fe1d40b2da0 2020-11-28T21:47:11.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.463+0000 7fe1d359e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1d4143300 con 0x7fe1d40b7920 2020-11-28T21:47:11.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.463+0000 7fe1d2d9d700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fe1d4066fa0 0x7fe1d40b1db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:45502/0 (socket says 172.21.15.162:45502) 2020-11-28T21:47:11.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.463+0000 7fe1d2d9d700 1 -- 172.21.15.162:0/1540052619 learned_addr learned my addr 172.21.15.162:0/1540052619 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:47:11.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.463+0000 7fe1d259c700 1 --2- 172.21.15.162:0/1540052619 >> v2:172.21.15.162:3300/0 conn(0x7fe1d40b7920 0x7fe1d413f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:11.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.467+0000 7fe1d1d9b700 1 --2- 172.21.15.162:0/1540052619 >> v2:172.21.15.118:3300/0 conn(0x7fe1d40b2da0 0x7fe1d413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:11.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.467+0000 7fe1d159a700 1 -- 172.21.15.162:0/1540052619 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1066652984 0 0) 0x7fe1cc001450 con 0x7fe1d4066fa0 2020-11-28T21:47:11.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.467+0000 7fe1d159a700 1 -- 172.21.15.162:0/1540052619 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1b4003230 con 0x7fe1d4066fa0 2020-11-28T21:47:11.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.467+0000 7fe1d159a700 1 -- 172.21.15.162:0/1540052619 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 132189344 0 0) 0x7fe1b4003230 con 0x7fe1d4066fa0 2020-11-28T21:47:11.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.467+0000 7fe1d259c700 1 -- 172.21.15.162:0/1540052619 >> v2:172.21.15.118:3300/0 conn(0x7fe1d40b2da0 msgr2=0x7fe1d413fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:11.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.467+0000 7fe1d259c700 1 --2- 172.21.15.162:0/1540052619 >> v2:172.21.15.118:3300/0 conn(0x7fe1d40b2da0 0x7fe1d413fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:11.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.467+0000 7fe1d259c700 1 -- 172.21.15.162:0/1540052619 >> v1:172.21.15.162:6789/0 conn(0x7fe1d4066fa0 legacy=0x7fe1d40b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:11.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.467+0000 7fe1d259c700 1 -- 172.21.15.162:0/1540052619 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d4143560 con 0x7fe1d40b7920 2020-11-28T21:47:11.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.467+0000 7fe1d259c700 1 --2- 172.21.15.162:0/1540052619 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1d40b7920 0x7fe1d413f980 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7fe1cc001d80 tx=0x7fe1c4006dc0).ready entity=mon.1 client_cookie=918117871e60150 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:11.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.467+0000 7fe1d159a700 1 -- 172.21.15.162:0/1540052619 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1cc001be0 con 0x7fe1d40b7920 2020-11-28T21:47:11.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.467+0000 7fe1d159a700 1 -- 172.21.15.162:0/1540052619 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c400d5c0 con 0x7fe1d40b7920 2020-11-28T21:47:11.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.467+0000 7fe1d159a700 1 -- 172.21.15.162:0/1540052619 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1c400d730 con 0x7fe1d40b7920 2020-11-28T21:47:11.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.467+0000 7fe1d359e700 1 -- 172.21.15.162:0/1540052619 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1d40b7920 msgr2=0x7fe1d413f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:11.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.467+0000 7fe1d359e700 1 --2- 172.21.15.162:0/1540052619 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1d40b7920 0x7fe1d413f980 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7fe1cc001d80 tx=0x7fe1c4006dc0).stop 2020-11-28T21:47:11.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.467+0000 7fe1d359e700 1 -- 172.21.15.162:0/1540052619 shutdown_connections 2020-11-28T21:47:11.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.467+0000 7fe1d359e700 1 --2- 172.21.15.162:0/1540052619 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1d40b7920 0x7fe1d413f980 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:11.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.467+0000 7fe1d359e700 1 --2- 172.21.15.162:0/1540052619 >> v2:172.21.15.118:3300/0 conn(0x7fe1d40b2da0 0x7fe1d413fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:11.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.467+0000 7fe1d359e700 1 -- 172.21.15.162:0/1540052619 >> 172.21.15.162:0/1540052619 conn(0x7fe1d405c440 msgr2=0x7fe1d40b6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:11.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.467+0000 7fe1d359e700 1 -- 172.21.15.162:0/1540052619 shutdown_connections 2020-11-28T21:47:11.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.467+0000 7fe1d359e700 1 -- 172.21.15.162:0/1540052619 wait complete. 2020-11-28T21:47:11.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.471+0000 7fe1d359e700 1 Processor -- start 2020-11-28T21:47:11.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.471+0000 7fe1d359e700 1 -- start start 2020-11-28T21:47:11.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.471+0000 7fe1d359e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe1d4066fa0 0x7fe1d40b0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:11.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.471+0000 7fe1d359e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe1d40b2da0 0x7fe1d40ace20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:11.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.471+0000 7fe1d359e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1d40b7920 0x7fe1d40ae4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:11.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.471+0000 7fe1d359e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d40b03c0 con 0x7fe1d4066fa0 2020-11-28T21:47:11.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.471+0000 7fe1d359e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d4147120 con 0x7fe1d40b7920 2020-11-28T21:47:11.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.471+0000 7fe1d359e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d40adb00 con 0x7fe1d40b2da0 2020-11-28T21:47:11.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.471+0000 7fe1d2d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1d40b7920 0x7fe1d40ae4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:11.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.471+0000 7fe1d2d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1d40b7920 0x7fe1d40ae4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38666/0 (socket says 172.21.15.162:38666) 2020-11-28T21:47:11.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.471+0000 7fe1d2d9d700 1 -- 172.21.15.162:0/2726950719 learned_addr learned my addr 172.21.15.162:0/2726950719 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:11.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.471+0000 7fe1d259c700 1 --2- 172.21.15.162:0/2726950719 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe1d4066fa0 0x7fe1d40b0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:11.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.471+0000 7fe1d1d9b700 1 --2- 172.21.15.162:0/2726950719 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe1d40b2da0 0x7fe1d40ace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:11.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.471+0000 7fe1d2d9d700 1 -- 172.21.15.162:0/2726950719 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe1d40b2da0 msgr2=0x7fe1d40ace20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:11.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.471+0000 7fe1d2d9d700 1 --2- 172.21.15.162:0/2726950719 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe1d40b2da0 0x7fe1d40ace20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:11.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.471+0000 7fe1d2d9d700 1 -- 172.21.15.162:0/2726950719 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe1d4066fa0 msgr2=0x7fe1d40b0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:11.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.471+0000 7fe1d2d9d700 1 --2- 172.21.15.162:0/2726950719 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe1d4066fa0 0x7fe1d40b0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:11.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.471+0000 7fe1d2d9d700 1 -- 172.21.15.162:0/2726950719 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d40adda0 con 0x7fe1d40b7920 2020-11-28T21:47:11.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.471+0000 7fe1d2d9d700 1 --2- 172.21.15.162:0/2726950719 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1d40b7920 0x7fe1d40ae4f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7fe1cc001e40 tx=0x7fe1cc001700).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:11.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.471+0000 7fe1cb7fe700 1 -- 172.21.15.162:0/2726950719 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1cc004b50 con 0x7fe1d40b7920 2020-11-28T21:47:11.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.471+0000 7fe1cb7fe700 1 -- 172.21.15.162:0/2726950719 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1cc004cb0 con 0x7fe1d40b7920 2020-11-28T21:47:11.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.471+0000 7fe1d359e700 1 -- 172.21.15.162:0/2726950719 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1d40b0120 con 0x7fe1d40b7920 2020-11-28T21:47:11.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.471+0000 7fe1cb7fe700 1 -- 172.21.15.162:0/2726950719 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1cc007360 con 0x7fe1d40b7920 2020-11-28T21:47:11.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.471+0000 7fe1d359e700 1 -- 172.21.15.162:0/2726950719 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1d40afc40 con 0x7fe1d40b7920 2020-11-28T21:47:11.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.475+0000 7fe1d359e700 1 -- 172.21.15.162:0/2726950719 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1d4057050 con 0x7fe1d40b7920 2020-11-28T21:47:11.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.475+0000 7fe1cb7fe700 1 -- 172.21.15.162:0/2726950719 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe1cc010070 con 0x7fe1d40b7920 2020-11-28T21:47:11.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.475+0000 7fe1cb7fe700 1 --2- 172.21.15.162:0/2726950719 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe1b8055440 0x7fe1b8057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:11.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.475+0000 7fe1cb7fe700 1 -- 172.21.15.162:0/2726950719 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe1cc06db00 con 0x7fe1d40b7920 2020-11-28T21:47:11.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.475+0000 7fe1d259c700 1 --2- 172.21.15.162:0/2726950719 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe1b8055440 0x7fe1b8057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:11.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.475+0000 7fe1d259c700 1 --2- 172.21.15.162:0/2726950719 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe1b8055440 0x7fe1b8057900 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fe1c401a000 tx=0x7fe1c4015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:11.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.483+0000 7fe1cb7fe700 1 -- 172.21.15.162:0/2726950719 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe1cc043e00 con 0x7fe1d40b7920 2020-11-28T21:47:11.674 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.671+0000 7fe1d359e700 1 -- 172.21.15.162:0/2726950719 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1d4140360 con 0x7fe1d40b7920 2020-11-28T21:47:11.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.675+0000 7fe1cb7fe700 1 -- 172.21.15.162:0/2726950719 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+2497 (secure 0 0 0) 0x7fe1d4140360 con 0x7fe1d40b7920 2020-11-28T21:47:11.677 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:47:11.677 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":10,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4839,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/1153453343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1153453343},{"type":"v1","addr":"172.21.15.162:6836","nonce":1153453343}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8},{"gid":4849,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2101329586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":2101329586},{"type":"v1","addr":"172.21.15.162:6837","nonce":2101329586}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:47:10.330722+0000","modified":"2020-11-28T21:47:11.333518+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4855},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4855":{"gid":4855,"name":"b","rank":0,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.118:6833/3038031199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3038031199},{"type":"v1","addr":"172.21.15.118:6833","nonce":3038031199}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-28T21:47:11.680 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.679+0000 7fe1d359e700 1 -- 172.21.15.162:0/2726950719 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe1b8055440 msgr2=0x7fe1b8057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:11.680 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.679+0000 7fe1d359e700 1 --2- 172.21.15.162:0/2726950719 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe1b8055440 0x7fe1b8057900 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fe1c401a000 tx=0x7fe1c4015000).stop 2020-11-28T21:47:11.680 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.679+0000 7fe1d359e700 1 -- 172.21.15.162:0/2726950719 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1d40b7920 msgr2=0x7fe1d40ae4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:11.680 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.679+0000 7fe1d359e700 1 --2- 172.21.15.162:0/2726950719 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1d40b7920 0x7fe1d40ae4f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7fe1cc001e40 tx=0x7fe1cc001700).stop 2020-11-28T21:47:11.681 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.679+0000 7fe1d359e700 1 -- 172.21.15.162:0/2726950719 shutdown_connections 2020-11-28T21:47:11.681 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.679+0000 7fe1d359e700 1 --2- 172.21.15.162:0/2726950719 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe1b8055440 0x7fe1b8057900 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:11.681 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.679+0000 7fe1d359e700 1 --2- 172.21.15.162:0/2726950719 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1d40b7920 0x7fe1d40ae4f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:11.681 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.679+0000 7fe1d359e700 1 --2- 172.21.15.162:0/2726950719 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe1d4066fa0 0x7fe1d40b0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:11.682 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.679+0000 7fe1d359e700 1 --2- 172.21.15.162:0/2726950719 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe1d40b2da0 0x7fe1d40ace20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:11.682 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.679+0000 7fe1d359e700 1 -- 172.21.15.162:0/2726950719 >> 172.21.15.162:0/2726950719 conn(0x7fe1d405c440 msgr2=0x7fe1d4141cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:11.682 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.679+0000 7fe1d359e700 1 -- 172.21.15.162:0/2726950719 shutdown_connections 2020-11-28T21:47:11.682 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.679+0000 7fe1d359e700 1 -- 172.21.15.162:0/2726950719 wait complete. 2020-11-28T21:47:11.683 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 10 2020-11-28T21:47:11.697 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:47:11.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.803+0000 7f75ec8fc700 1 Processor -- start 2020-11-28T21:47:11.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.803+0000 7f75ec8fc700 1 -- start start 2020-11-28T21:47:11.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.803+0000 7f75ec8fc700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f75e80b2290 0x7f75e80ae4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:11.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.803+0000 7f75ec8fc700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f75e80af480 0x7f75e81417a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:11.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.803+0000 7f75ec8fc700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75e80b2670 con 0x7f75e80a8550 2020-11-28T21:47:11.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.803+0000 7f75ec8fc700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75e80afca0 con 0x7f75e80af480 2020-11-28T21:47:11.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.803+0000 7f75ec8fc700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75e80ad740 con 0x7f75e80b2290 2020-11-28T21:47:11.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.803+0000 7f75e6ffd700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f75e80a8550 0x7f75e80ae9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:45516/0 (socket says 172.21.15.162:45516) 2020-11-28T21:47:11.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.803+0000 7f75e6ffd700 1 -- 172.21.15.162:0/2016200390 learned_addr learned my addr 172.21.15.162:0/2016200390 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:47:11.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.803+0000 7f75e77fe700 1 --2- 172.21.15.162:0/2016200390 >> v2:172.21.15.162:3300/0 conn(0x7f75e80b2290 0x7f75e80ae4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:11.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.803+0000 7f75e7fff700 1 --2- 172.21.15.162:0/2016200390 >> v2:172.21.15.118:3300/0 conn(0x7f75e80af480 0x7f75e81417a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:11.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.803+0000 7f75e67fc700 1 -- 172.21.15.162:0/2016200390 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1482127424 0 0) 0x7f75e80b2670 con 0x7f75e80a8550 2020-11-28T21:47:11.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.803+0000 7f75e67fc700 1 -- 172.21.15.162:0/2016200390 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75cc003230 con 0x7f75e80a8550 2020-11-28T21:47:11.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.803+0000 7f75e77fe700 1 -- 172.21.15.162:0/2016200390 >> v2:172.21.15.118:3300/0 conn(0x7f75e80af480 msgr2=0x7f75e81417a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:11.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.803+0000 7f75e67fc700 1 -- 172.21.15.162:0/2016200390 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 339025077 0 0) 0x7f75cc003230 con 0x7f75e80a8550 2020-11-28T21:47:11.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.803+0000 7f75e77fe700 1 --2- 172.21.15.162:0/2016200390 >> v2:172.21.15.118:3300/0 conn(0x7f75e80af480 0x7f75e81417a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:11.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.803+0000 7f75e77fe700 1 -- 172.21.15.162:0/2016200390 >> v1:172.21.15.162:6789/0 conn(0x7f75e80a8550 legacy=0x7f75e80ae9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:11.810 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.803+0000 7f75e77fe700 1 -- 172.21.15.162:0/2016200390 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e80afa00 con 0x7f75e80b2290 2020-11-28T21:47:11.812 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.803+0000 7f75e77fe700 1 --2- 172.21.15.162:0/2016200390 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75e80b2290 0x7f75e80ae4c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7f75d8001870 tx=0x7f75d0006dc0).ready entity=mon.1 client_cookie=6b3061ee3e1acefd server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:11.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.807+0000 7f75e67fc700 1 -- 172.21.15.162:0/2016200390 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75d80016d0 con 0x7f75e80b2290 2020-11-28T21:47:11.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.807+0000 7f75e67fc700 1 -- 172.21.15.162:0/2016200390 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d000d5c0 con 0x7f75e80b2290 2020-11-28T21:47:11.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.807+0000 7f75e67fc700 1 -- 172.21.15.162:0/2016200390 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75d000d730 con 0x7f75e80b2290 2020-11-28T21:47:11.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.807+0000 7f75ec8fc700 1 -- 172.21.15.162:0/2016200390 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75e80b2290 msgr2=0x7f75e80ae4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:11.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.807+0000 7f75ec8fc700 1 --2- 172.21.15.162:0/2016200390 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75e80b2290 0x7f75e80ae4c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7f75d8001870 tx=0x7f75d0006dc0).stop 2020-11-28T21:47:11.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.807+0000 7f75ec8fc700 1 -- 172.21.15.162:0/2016200390 shutdown_connections 2020-11-28T21:47:11.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.807+0000 7f75ec8fc700 1 --2- 172.21.15.162:0/2016200390 >> v2:172.21.15.118:3300/0 conn(0x7f75e80af480 0x7f75e81417a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:11.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.807+0000 7f75ec8fc700 1 --2- 172.21.15.162:0/2016200390 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75e80b2290 0x7f75e80ae4c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:11.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.807+0000 7f75ec8fc700 1 -- 172.21.15.162:0/2016200390 >> 172.21.15.162:0/2016200390 conn(0x7f75e8061320 msgr2=0x7f75e8062e50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:11.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.807+0000 7f75ec8fc700 1 -- 172.21.15.162:0/2016200390 shutdown_connections 2020-11-28T21:47:11.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.807+0000 7f75ec8fc700 1 -- 172.21.15.162:0/2016200390 wait complete. 2020-11-28T21:47:11.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.807+0000 7f75ec8fc700 1 Processor -- start 2020-11-28T21:47:11.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.807+0000 7f75ec8fc700 1 -- start start 2020-11-28T21:47:11.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.807+0000 7f75ec8fc700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f75e80a8550 0x7f75e80b4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:11.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.807+0000 7f75ec8fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75e80af480 0x7f75e80b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:11.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.807+0000 7f75ec8fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f75e80b2290 0x7f75e80b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:11.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.807+0000 7f75ec8fc700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e8133f80 con 0x7f75e80b2290 2020-11-28T21:47:11.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.807+0000 7f75ec8fc700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e812bf70 con 0x7f75e80af480 2020-11-28T21:47:11.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.807+0000 7f75ec8fc700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e81341b0 con 0x7f75e80a8550 2020-11-28T21:47:11.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.807+0000 7f75e6ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75e80af480 0x7f75e80b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:11.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.807+0000 7f75e6ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75e80af480 0x7f75e80b3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38680/0 (socket says 172.21.15.162:38680) 2020-11-28T21:47:11.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.807+0000 7f75e6ffd700 1 -- 172.21.15.162:0/3832165266 learned_addr learned my addr 172.21.15.162:0/3832165266 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:11.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.811+0000 7f75e7fff700 1 --2- 172.21.15.162:0/3832165266 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f75e80b2290 0x7f75e80b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:11.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.811+0000 7f75e77fe700 1 --2- 172.21.15.162:0/3832165266 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f75e80a8550 0x7f75e80b4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:11.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.811+0000 7f75e6ffd700 1 -- 172.21.15.162:0/3832165266 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f75e80a8550 msgr2=0x7f75e80b4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:11.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.811+0000 7f75e6ffd700 1 --2- 172.21.15.162:0/3832165266 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f75e80a8550 0x7f75e80b4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:11.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.811+0000 7f75e6ffd700 1 -- 172.21.15.162:0/3832165266 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f75e80b2290 msgr2=0x7f75e80b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:11.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.811+0000 7f75e6ffd700 1 --2- 172.21.15.162:0/3832165266 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f75e80b2290 0x7f75e80b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:11.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.811+0000 7f75e6ffd700 1 -- 172.21.15.162:0/3832165266 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e812c1d0 con 0x7f75e80af480 2020-11-28T21:47:11.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.811+0000 7f75e6ffd700 1 --2- 172.21.15.162:0/3832165266 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75e80af480 0x7f75e80b3a90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f75e81368e0 tx=0x7f75d8001280).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:11.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.811+0000 7f75e4ff9700 1 -- 172.21.15.162:0/3832165266 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75d8007170 con 0x7f75e80af480 2020-11-28T21:47:11.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.811+0000 7f75e4ff9700 1 -- 172.21.15.162:0/3832165266 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d80072d0 con 0x7f75e80af480 2020-11-28T21:47:11.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.811+0000 7f75ec8fc700 1 -- 172.21.15.162:0/3832165266 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75e8136610 con 0x7f75e80af480 2020-11-28T21:47:11.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.811+0000 7f75e4ff9700 1 -- 172.21.15.162:0/3832165266 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75d80048f0 con 0x7f75e80af480 2020-11-28T21:47:11.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.811+0000 7f75ec8fc700 1 -- 172.21.15.162:0/3832165266 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e812a260 con 0x7f75e80af480 2020-11-28T21:47:11.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.811+0000 7f75ec8fc700 1 -- 172.21.15.162:0/3832165266 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75e8057050 con 0x7f75e80af480 2020-11-28T21:47:11.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.811+0000 7f75e4ff9700 1 -- 172.21.15.162:0/3832165266 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f75d801b070 con 0x7f75e80af480 2020-11-28T21:47:11.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.811+0000 7f75e4ff9700 1 --2- 172.21.15.162:0/3832165266 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f75c00554e0 0x7f75c00579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:11.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.811+0000 7f75e77fe700 1 --2- 172.21.15.162:0/3832165266 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f75c00554e0 0x7f75c00579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:11.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.811+0000 7f75e4ff9700 1 -- 172.21.15.162:0/3832165266 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f75d806d650 con 0x7f75e80af480 2020-11-28T21:47:11.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.815+0000 7f75e77fe700 1 --2- 172.21.15.162:0/3832165266 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f75c00554e0 0x7f75c00579a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f75d001a000 tx=0x7f75d0015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:11.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.819+0000 7f75e4ff9700 1 -- 172.21.15.162:0/3832165266 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f75d80438b0 con 0x7f75e80af480 2020-11-28T21:47:11.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.963+0000 7f75ec8fc700 1 -- 172.21.15.162:0/3832165266 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f75e8134320 con 0x7f75e80af480 2020-11-28T21:47:11.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.963+0000 7f75e4ff9700 1 -- 172.21.15.162:0/3832165266 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16134 (secure 0 0 0) 0x7f75d8046ed0 con 0x7f75e80af480 2020-11-28T21:47:11.966 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:47:11.966 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":33,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:47:11.334213+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:47:08.251301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2020-11-28T21:47:09.333148+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:47:11.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.963+0000 7f75ec8fc700 1 -- 172.21.15.162:0/3832165266 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f75c00554e0 msgr2=0x7f75c00579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:11.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.963+0000 7f75ec8fc700 1 --2- 172.21.15.162:0/3832165266 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f75c00554e0 0x7f75c00579a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f75d001a000 tx=0x7f75d0015000).stop 2020-11-28T21:47:11.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.963+0000 7f75ec8fc700 1 -- 172.21.15.162:0/3832165266 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75e80af480 msgr2=0x7f75e80b3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:11.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.963+0000 7f75ec8fc700 1 --2- 172.21.15.162:0/3832165266 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75e80af480 0x7f75e80b3a90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f75e81368e0 tx=0x7f75d8001280).stop 2020-11-28T21:47:11.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.963+0000 7f75ec8fc700 1 -- 172.21.15.162:0/3832165266 shutdown_connections 2020-11-28T21:47:11.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.963+0000 7f75ec8fc700 1 --2- 172.21.15.162:0/3832165266 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f75c00554e0 0x7f75c00579a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:11.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.963+0000 7f75ec8fc700 1 --2- 172.21.15.162:0/3832165266 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f75e80a8550 0x7f75e80b4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:11.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.963+0000 7f75ec8fc700 1 --2- 172.21.15.162:0/3832165266 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f75e80b2290 0x7f75e80b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:11.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.963+0000 7f75ec8fc700 1 --2- 172.21.15.162:0/3832165266 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75e80af480 0x7f75e80b3a90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:11.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.967+0000 7f75ec8fc700 1 -- 172.21.15.162:0/3832165266 >> 172.21.15.162:0/3832165266 conn(0x7f75e8061320 msgr2=0x7f75e8141ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:11.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.967+0000 7f75ec8fc700 1 -- 172.21.15.162:0/3832165266 shutdown_connections 2020-11-28T21:47:11.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:11.967+0000 7f75ec8fc700 1 -- 172.21.15.162:0/3832165266 wait complete. 2020-11-28T21:47:11.979 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-11-28T21:47:12.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.091+0000 7f6164dd5700 1 Processor -- start 2020-11-28T21:47:12.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.095+0000 7f6164dd5700 1 -- start start 2020-11-28T21:47:12.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.095+0000 7f6164dd5700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f61600b2560 0x7f6160156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:12.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.095+0000 7f6164dd5700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f616005fa90 0x7f61601593c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:12.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.095+0000 7f6164dd5700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61600ab0b0 con 0x7f61600b2e80 2020-11-28T21:47:12.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.095+0000 7f6164dd5700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61600b3260 con 0x7f61600b2560 2020-11-28T21:47:12.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.095+0000 7f6164dd5700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61600ab2e0 con 0x7f616005fa90 2020-11-28T21:47:12.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.095+0000 7f6166037700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f616005fa90 0x7f61601593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:12.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.095+0000 7f6166037700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f616005fa90 0x7f61601593c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38690/0 (socket says 172.21.15.162:38690) 2020-11-28T21:47:12.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.095+0000 7f6166037700 1 -- 172.21.15.162:0/3951369251 learned_addr learned my addr 172.21.15.162:0/3951369251 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:12.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.095+0000 7f615f7fe700 1 --2- 172.21.15.162:0/3951369251 >> v2:172.21.15.118:3300/0 conn(0x7f61600b2560 0x7f6160156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:12.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.095+0000 7f6166037700 1 -- 172.21.15.162:0/3951369251 >> v2:172.21.15.118:3300/0 conn(0x7f61600b2560 msgr2=0x7f6160156ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:12.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.095+0000 7f6166037700 1 --2- 172.21.15.162:0/3951369251 >> v2:172.21.15.118:3300/0 conn(0x7f61600b2560 0x7f6160156ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:12.104 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.095+0000 7f6166037700 1 -- 172.21.15.162:0/3951369251 >> v1:172.21.15.118:6789/0 conn(0x7f61600b2e80 legacy=0x7f616005f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:12.104 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.095+0000 7f6166037700 1 -- 172.21.15.162:0/3951369251 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61600256d0 con 0x7f616005fa90 2020-11-28T21:47:12.104 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.095+0000 7f6166037700 1 --2- 172.21.15.162:0/3951369251 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f616005fa90 0x7f61601593c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f61580032b0 tx=0x7f6158009270).ready entity=mon.1 client_cookie=8480cf9d377badd7 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:12.104 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.095+0000 7f615effd700 1 -- 172.21.15.162:0/3951369251 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6158009e60 con 0x7f616005fa90 2020-11-28T21:47:12.104 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.095+0000 7f615effd700 1 -- 172.21.15.162:0/3951369251 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f615800bc00 con 0x7f616005fa90 2020-11-28T21:47:12.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.095+0000 7f615effd700 1 -- 172.21.15.162:0/3951369251 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f615800bd70 con 0x7f616005fa90 2020-11-28T21:47:12.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.095+0000 7f6164dd5700 1 -- 172.21.15.162:0/3951369251 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f616005fa90 msgr2=0x7f61601593c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:12.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.095+0000 7f6164dd5700 1 --2- 172.21.15.162:0/3951369251 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f616005fa90 0x7f61601593c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f61580032b0 tx=0x7f6158009270).stop 2020-11-28T21:47:12.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.099+0000 7f6164dd5700 1 -- 172.21.15.162:0/3951369251 shutdown_connections 2020-11-28T21:47:12.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.099+0000 7f6164dd5700 1 --2- 172.21.15.162:0/3951369251 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f616005fa90 0x7f61601593c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:12.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.099+0000 7f6164dd5700 1 --2- 172.21.15.162:0/3951369251 >> v2:172.21.15.118:3300/0 conn(0x7f61600b2560 0x7f6160156ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:12.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.099+0000 7f6164dd5700 1 -- 172.21.15.162:0/3951369251 >> 172.21.15.162:0/3951369251 conn(0x7f6160063b60 msgr2=0x7f6160063f40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:12.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.099+0000 7f6164dd5700 1 -- 172.21.15.162:0/3951369251 shutdown_connections 2020-11-28T21:47:12.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.099+0000 7f6164dd5700 1 -- 172.21.15.162:0/3951369251 wait complete. 2020-11-28T21:47:12.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.099+0000 7f6164dd5700 1 Processor -- start 2020-11-28T21:47:12.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.099+0000 7f6164dd5700 1 -- start start 2020-11-28T21:47:12.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.099+0000 7f6164dd5700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f616005fa90 0x7f616009b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:12.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.099+0000 7f6164dd5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61600b2560 0x7f6160141c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:12.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.099+0000 7f6164dd5700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f61600b2e80 0x7f6160142140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:12.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.099+0000 7f6164dd5700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61600adc30 con 0x7f616005fa90 2020-11-28T21:47:12.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.099+0000 7f6164dd5700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6160131410 con 0x7f61600b2560 2020-11-28T21:47:12.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.099+0000 7f6164dd5700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6160131b70 con 0x7f61600b2e80 2020-11-28T21:47:12.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.099+0000 7f615f7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61600b2560 0x7f6160141c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:12.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.099+0000 7f615f7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61600b2560 0x7f6160141c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38696/0 (socket says 172.21.15.162:38696) 2020-11-28T21:47:12.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.099+0000 7f615f7fe700 1 -- 172.21.15.162:0/1458833503 learned_addr learned my addr 172.21.15.162:0/1458833503 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:12.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.099+0000 7f6166037700 1 --2- 172.21.15.162:0/1458833503 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f61600b2e80 0x7f6160142140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:12.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.099+0000 7f615ffff700 1 --2- 172.21.15.162:0/1458833503 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f616005fa90 0x7f616009b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:12.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.103+0000 7f615f7fe700 1 -- 172.21.15.162:0/1458833503 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f61600b2e80 msgr2=0x7f6160142140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:12.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.103+0000 7f615f7fe700 1 --2- 172.21.15.162:0/1458833503 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f61600b2e80 0x7f6160142140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:12.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.103+0000 7f615f7fe700 1 -- 172.21.15.162:0/1458833503 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f616005fa90 msgr2=0x7f616009b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:12.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.103+0000 7f615f7fe700 1 --2- 172.21.15.162:0/1458833503 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f616005fa90 0x7f616009b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:12.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.103+0000 7f615f7fe700 1 -- 172.21.15.162:0/1458833503 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61600ae8d0 con 0x7f61600b2560 2020-11-28T21:47:12.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.103+0000 7f615ffff700 1 --2- 172.21.15.162:0/1458833503 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f616005fa90 0x7f616009b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:47:12.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.103+0000 7f6166037700 1 --2- 172.21.15.162:0/1458833503 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f61600b2e80 0x7f6160142140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:47:12.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.103+0000 7f615f7fe700 1 --2- 172.21.15.162:0/1458833503 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61600b2560 0x7f6160141c50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f61600ae6a0 tx=0x7f6154006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:12.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.103+0000 7f615cff9700 1 -- 172.21.15.162:0/1458833503 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f615400ca80 con 0x7f61600b2560 2020-11-28T21:47:12.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.103+0000 7f615cff9700 1 -- 172.21.15.162:0/1458833503 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f615400cbe0 con 0x7f61600b2560 2020-11-28T21:47:12.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.103+0000 7f615cff9700 1 -- 172.21.15.162:0/1458833503 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61540123f0 con 0x7f61600b2560 2020-11-28T21:47:12.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.103+0000 7f6164dd5700 1 -- 172.21.15.162:0/1458833503 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61601318d0 con 0x7f61600b2560 2020-11-28T21:47:12.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.103+0000 7f6164dd5700 1 -- 172.21.15.162:0/1458833503 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61600afe10 con 0x7f61600b2560 2020-11-28T21:47:12.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.103+0000 7f6164dd5700 1 -- 172.21.15.162:0/1458833503 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6160057050 con 0x7f61600b2560 2020-11-28T21:47:12.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.103+0000 7f615cff9700 1 -- 172.21.15.162:0/1458833503 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6154012550 con 0x7f61600b2560 2020-11-28T21:47:12.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.103+0000 7f615cff9700 1 --2- 172.21.15.162:0/1458833503 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f61400554d0 0x7f6140057990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:12.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.103+0000 7f615cff9700 1 -- 172.21.15.162:0/1458833503 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f615406e440 con 0x7f61600b2560 2020-11-28T21:47:12.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.103+0000 7f615ffff700 1 --2- 172.21.15.162:0/1458833503 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f61400554d0 0x7f6140057990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:12.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.107+0000 7f615ffff700 1 --2- 172.21.15.162:0/1458833503 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f61400554d0 0x7f6140057990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f615000b000 tx=0x7f6150009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:12.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.111+0000 7f615cff9700 1 -- 172.21.15.162:0/1458833503 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f61540446e0 con 0x7f61600b2560 2020-11-28T21:47:12.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.299+0000 7f6164dd5700 1 -- 172.21.15.162:0/1458833503 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f6160068350 con 0x7f61600b2560 2020-11-28T21:47:12.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.311+0000 7f615cff9700 1 -- 172.21.15.162:0/1458833503 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v11) v1 ==== 184+0+0 (secure 0 0 0) 0x7f6154047d00 con 0x7f61600b2560 2020-11-28T21:47:12.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.311+0000 7f6164dd5700 1 -- 172.21.15.162:0/1458833503 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f61400554d0 msgr2=0x7f6140057990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:12.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.311+0000 7f6164dd5700 1 --2- 172.21.15.162:0/1458833503 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f61400554d0 0x7f6140057990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f615000b000 tx=0x7f6150009040).stop 2020-11-28T21:47:12.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.311+0000 7f6164dd5700 1 -- 172.21.15.162:0/1458833503 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61600b2560 msgr2=0x7f6160141c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:12.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.311+0000 7f6164dd5700 1 --2- 172.21.15.162:0/1458833503 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61600b2560 0x7f6160141c50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f61600ae6a0 tx=0x7f6154006dc0).stop 2020-11-28T21:47:12.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.311+0000 7f6164dd5700 1 -- 172.21.15.162:0/1458833503 shutdown_connections 2020-11-28T21:47:12.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.311+0000 7f6164dd5700 1 --2- 172.21.15.162:0/1458833503 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f61400554d0 0x7f6140057990 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:12.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.311+0000 7f6164dd5700 1 --2- 172.21.15.162:0/1458833503 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f61600b2e80 0x7f6160142140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:12.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.311+0000 7f6164dd5700 1 --2- 172.21.15.162:0/1458833503 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f616005fa90 0x7f616009b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:12.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.311+0000 7f6164dd5700 1 --2- 172.21.15.162:0/1458833503 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61600b2560 0x7f6160141c50 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:12.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.311+0000 7f6164dd5700 1 -- 172.21.15.162:0/1458833503 >> 172.21.15.162:0/1458833503 conn(0x7f6160063b60 msgr2=0x7f616005f380 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:12.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.311+0000 7f6164dd5700 1 -- 172.21.15.162:0/1458833503 shutdown_connections 2020-11-28T21:47:12.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.311+0000 7f6164dd5700 1 -- 172.21.15.162:0/1458833503 wait complete. 2020-11-28T21:47:12.317 INFO:teuthology.orchestra.run.smithi162.stderr:updated caps for client.0 2020-11-28T21:47:12.329 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-11-28T21:47:12.457 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.451+0000 7f65d3fff700 1 Processor -- start 2020-11-28T21:47:12.457 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.451+0000 7f65d3fff700 1 -- start start 2020-11-28T21:47:12.457 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.451+0000 7f65d3fff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f65d40b2e80 0x7f65d405f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:12.457 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.451+0000 7f65d3fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f65d40b2560 0x7f65d405f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:12.458 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.451+0000 7f65d3fff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65d40ab0b0 con 0x7f65d40acb40 2020-11-28T21:47:12.458 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.451+0000 7f65d3fff700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65d405fd60 con 0x7f65d40b2e80 2020-11-28T21:47:12.458 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.451+0000 7f65d3fff700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65d40b3260 con 0x7f65d40b2560 2020-11-28T21:47:12.458 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.451+0000 7f65d2ffd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f65d40b2560 0x7f65d405f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:12.459 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.451+0000 7f65d2ffd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f65d40b2560 0x7f65d405f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38700/0 (socket says 172.21.15.162:38700) 2020-11-28T21:47:12.459 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.451+0000 7f65d2ffd700 1 -- 172.21.15.162:0/4213194149 learned_addr learned my addr 172.21.15.162:0/4213194149 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:12.459 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.451+0000 7f65d37fe700 1 --2- 172.21.15.162:0/4213194149 >> v2:172.21.15.118:3300/0 conn(0x7f65d40b2e80 0x7f65d405f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:12.459 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.451+0000 7f65d2ffd700 1 -- 172.21.15.162:0/4213194149 >> v2:172.21.15.118:3300/0 conn(0x7f65d40b2e80 msgr2=0x7f65d405f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:12.460 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.451+0000 7f65d2ffd700 1 --2- 172.21.15.162:0/4213194149 >> v2:172.21.15.118:3300/0 conn(0x7f65d40b2e80 0x7f65d405f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:12.460 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.451+0000 7f65d2ffd700 1 -- 172.21.15.162:0/4213194149 >> v1:172.21.15.118:6789/0 conn(0x7f65d40acb40 legacy=0x7f65d4060d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:12.460 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.451+0000 7f65d2ffd700 1 -- 172.21.15.162:0/4213194149 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d40ab2e0 con 0x7f65d40b2560 2020-11-28T21:47:12.460 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.451+0000 7f65d2ffd700 1 --2- 172.21.15.162:0/4213194149 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65d40b2560 0x7f65d405f870 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f65bc002f30 tx=0x7f65bc006e80).ready entity=mon.1 client_cookie=9e188cec43382fa server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:12.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.451+0000 7f65d27fc700 1 -- 172.21.15.162:0/4213194149 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65bc0097f0 con 0x7f65d40b2560 2020-11-28T21:47:12.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.451+0000 7f65d27fc700 1 -- 172.21.15.162:0/4213194149 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65bc009950 con 0x7f65d40b2560 2020-11-28T21:47:12.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.451+0000 7f65d27fc700 1 -- 172.21.15.162:0/4213194149 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65bc009cb0 con 0x7f65d40b2560 2020-11-28T21:47:12.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.451+0000 7f65d3fff700 1 -- 172.21.15.162:0/4213194149 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65d40b2560 msgr2=0x7f65d405f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:12.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.451+0000 7f65d3fff700 1 --2- 172.21.15.162:0/4213194149 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65d40b2560 0x7f65d405f870 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f65bc002f30 tx=0x7f65bc006e80).stop 2020-11-28T21:47:12.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.455+0000 7f65d3fff700 1 -- 172.21.15.162:0/4213194149 shutdown_connections 2020-11-28T21:47:12.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.455+0000 7f65d3fff700 1 --2- 172.21.15.162:0/4213194149 >> v2:172.21.15.118:3300/0 conn(0x7f65d40b2e80 0x7f65d405f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:12.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.455+0000 7f65d3fff700 1 --2- 172.21.15.162:0/4213194149 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65d40b2560 0x7f65d405f870 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:12.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.455+0000 7f65d3fff700 1 -- 172.21.15.162:0/4213194149 >> 172.21.15.162:0/4213194149 conn(0x7f65d4063b60 msgr2=0x7f65d4063f40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:12.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.455+0000 7f65d3fff700 1 -- 172.21.15.162:0/4213194149 shutdown_connections 2020-11-28T21:47:12.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.455+0000 7f65d3fff700 1 -- 172.21.15.162:0/4213194149 wait complete. 2020-11-28T21:47:12.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.455+0000 7f65d3fff700 1 Processor -- start 2020-11-28T21:47:12.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.455+0000 7f65d3fff700 1 -- start start 2020-11-28T21:47:12.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.455+0000 7f65d3fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65d40acb40 0x7f65d4130670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:12.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.455+0000 7f65d3fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f65d40b2560 0x7f65d4099380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:12.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.455+0000 7f65d3fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f65d40b2e80 0x7f65d4099870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:12.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.455+0000 7f65d3fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d40af8d0 con 0x7f65d40b2560 2020-11-28T21:47:12.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.455+0000 7f65d3fff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d4132a80 con 0x7f65d40acb40 2020-11-28T21:47:12.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.455+0000 7f65d3fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d4132ce0 con 0x7f65d40b2e80 2020-11-28T21:47:12.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.455+0000 7f65d37fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65d40acb40 0x7f65d4130670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:12.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.459+0000 7f65d37fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65d40acb40 0x7f65d4130670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38706/0 (socket says 172.21.15.162:38706) 2020-11-28T21:47:12.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.459+0000 7f65d37fe700 1 -- 172.21.15.162:0/244201725 learned_addr learned my addr 172.21.15.162:0/244201725 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:12.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.459+0000 7f65d2ffd700 1 --2- 172.21.15.162:0/244201725 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f65d40b2560 0x7f65d4099380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:12.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.459+0000 7f65d977e700 1 --2- 172.21.15.162:0/244201725 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f65d40b2e80 0x7f65d4099870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:12.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.459+0000 7f65d37fe700 1 -- 172.21.15.162:0/244201725 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f65d40b2e80 msgr2=0x7f65d4099870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:12.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.459+0000 7f65d37fe700 1 --2- 172.21.15.162:0/244201725 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f65d40b2e80 0x7f65d4099870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:12.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.459+0000 7f65d37fe700 1 -- 172.21.15.162:0/244201725 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f65d40b2560 msgr2=0x7f65d4099380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:12.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.459+0000 7f65d37fe700 1 --2- 172.21.15.162:0/244201725 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f65d40b2560 0x7f65d4099380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:12.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.459+0000 7f65d37fe700 1 -- 172.21.15.162:0/244201725 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d4144830 con 0x7f65d40acb40 2020-11-28T21:47:12.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.459+0000 7f65d37fe700 1 --2- 172.21.15.162:0/244201725 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65d40acb40 0x7f65d4130670 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f65cc00c000 tx=0x7f65cc0091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:12.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.459+0000 7f65cbfff700 1 -- 172.21.15.162:0/244201725 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65cc00cb50 con 0x7f65d40acb40 2020-11-28T21:47:12.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.459+0000 7f65cbfff700 1 -- 172.21.15.162:0/244201725 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65cc00ccb0 con 0x7f65d40acb40 2020-11-28T21:47:12.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.459+0000 7f65cbfff700 1 -- 172.21.15.162:0/244201725 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65cc0125e0 con 0x7f65d40acb40 2020-11-28T21:47:12.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.459+0000 7f65d3fff700 1 -- 172.21.15.162:0/244201725 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d4144ad0 con 0x7f65d40acb40 2020-11-28T21:47:12.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.459+0000 7f65d3fff700 1 -- 172.21.15.162:0/244201725 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d40ad480 con 0x7f65d40acb40 2020-11-28T21:47:12.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.459+0000 7f65cbfff700 1 -- 172.21.15.162:0/244201725 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f65cc022070 con 0x7f65d40acb40 2020-11-28T21:47:12.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.459+0000 7f65d3fff700 1 -- 172.21.15.162:0/244201725 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65d41446a0 con 0x7f65d40acb40 2020-11-28T21:47:12.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.459+0000 7f65cbfff700 1 --2- 172.21.15.162:0/244201725 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f65b80553b0 0x7f65b8057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:12.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.459+0000 7f65cbfff700 1 -- 172.21.15.162:0/244201725 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f65cc06e790 con 0x7f65d40acb40 2020-11-28T21:47:12.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.459+0000 7f65d2ffd700 1 --2- 172.21.15.162:0/244201725 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f65b80553b0 0x7f65b8057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:12.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.463+0000 7f65d2ffd700 1 --2- 172.21.15.162:0/244201725 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f65b80553b0 0x7f65b8057870 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f65bc01a000 tx=0x7f65bc015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:12.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.467+0000 7f65cbfff700 1 -- 172.21.15.162:0/244201725 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f65cc044980 con 0x7f65d40acb40 2020-11-28T21:47:12.656 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.655+0000 7f65d3fff700 1 -- 172.21.15.162:0/244201725 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f65d4057050 con 0x7f65d40acb40 2020-11-28T21:47:12.664 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.663+0000 7f65cbfff700 1 -- 172.21.15.162:0/244201725 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v12) v1 ==== 184+0+0 (secure 0 0 0) 0x7f65cc047fa0 con 0x7f65d40acb40 2020-11-28T21:47:12.665 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.663+0000 7f65d3fff700 1 -- 172.21.15.162:0/244201725 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f65b80553b0 msgr2=0x7f65b8057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:12.666 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.663+0000 7f65d3fff700 1 --2- 172.21.15.162:0/244201725 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f65b80553b0 0x7f65b8057870 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f65bc01a000 tx=0x7f65bc015000).stop 2020-11-28T21:47:12.666 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.663+0000 7f65d3fff700 1 -- 172.21.15.162:0/244201725 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65d40acb40 msgr2=0x7f65d4130670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:12.666 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.663+0000 7f65d3fff700 1 --2- 172.21.15.162:0/244201725 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65d40acb40 0x7f65d4130670 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f65cc00c000 tx=0x7f65cc0091a0).stop 2020-11-28T21:47:12.667 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.663+0000 7f65d3fff700 1 -- 172.21.15.162:0/244201725 shutdown_connections 2020-11-28T21:47:12.667 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.663+0000 7f65d3fff700 1 --2- 172.21.15.162:0/244201725 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f65b80553b0 0x7f65b8057870 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:12.667 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.663+0000 7f65d3fff700 1 --2- 172.21.15.162:0/244201725 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f65d40b2e80 0x7f65d4099870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:12.668 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.663+0000 7f65d3fff700 1 --2- 172.21.15.162:0/244201725 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65d40acb40 0x7f65d4130670 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:12.668 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.663+0000 7f65d3fff700 1 --2- 172.21.15.162:0/244201725 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f65d40b2560 0x7f65d4099380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:12.668 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.663+0000 7f65d3fff700 1 -- 172.21.15.162:0/244201725 >> 172.21.15.162:0/244201725 conn(0x7f65d4063b60 msgr2=0x7f65d4063f40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:12.668 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.663+0000 7f65d3fff700 1 -- 172.21.15.162:0/244201725 shutdown_connections 2020-11-28T21:47:12.669 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.663+0000 7f65d3fff700 1 -- 172.21.15.162:0/244201725 wait complete. 2020-11-28T21:47:12.669 INFO:teuthology.orchestra.run.smithi162.stderr:updated caps for client.1 2020-11-28T21:47:12.680 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:47:12.788 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.787+0000 7ff2b662d700 1 Processor -- start 2020-11-28T21:47:12.788 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.787+0000 7ff2b662d700 1 -- start start 2020-11-28T21:47:12.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.787+0000 7ff2b662d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff2b00603d0 0x7ff2b00b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:12.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.787+0000 7ff2b662d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff2b00b4c70 0x7ff2b0143020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:12.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.787+0000 7ff2b662d700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2b00b4110 con 0x7ff2b0142a00 2020-11-28T21:47:12.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.787+0000 7ff2b662d700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2b0060f60 con 0x7ff2b00b4c70 2020-11-28T21:47:12.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.787+0000 7ff2b662d700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2b00b43f0 con 0x7ff2b00603d0 2020-11-28T21:47:12.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.787+0000 7ff2b4e2a700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7ff2b0142a00 0x7ff2b009cfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:45558/0 (socket says 172.21.15.162:45558) 2020-11-28T21:47:12.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.787+0000 7ff2b4e2a700 1 -- 172.21.15.162:0/2512475525 learned_addr learned my addr 172.21.15.162:0/2512475525 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:47:12.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.787+0000 7ff2b562b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff2b00603d0 0x7ff2b00b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:12.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.787+0000 7ff2b5e2c700 1 --2- 172.21.15.162:0/2512475525 >> v2:172.21.15.118:3300/0 conn(0x7ff2b00b4c70 0x7ff2b0143020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:12.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.787+0000 7ff29ffff700 1 -- 172.21.15.162:0/2512475525 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 910506815 0 0) 0x7ff2b00b4110 con 0x7ff2b0142a00 2020-11-28T21:47:12.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.787+0000 7ff29ffff700 1 -- 172.21.15.162:0/2512475525 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff298003230 con 0x7ff2b0142a00 2020-11-28T21:47:12.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.787+0000 7ff2b562b700 1 -- 172.21.15.162:0/2512475525 >> v2:172.21.15.118:3300/0 conn(0x7ff2b00b4c70 msgr2=0x7ff2b0143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:12.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.787+0000 7ff2b562b700 1 --2- 172.21.15.162:0/2512475525 >> v2:172.21.15.118:3300/0 conn(0x7ff2b00b4c70 0x7ff2b0143020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:12.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.787+0000 7ff2b562b700 1 -- 172.21.15.162:0/2512475525 >> v1:172.21.15.162:6789/0 conn(0x7ff2b0142a00 legacy=0x7ff2b009cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:12.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.787+0000 7ff2b562b700 1 -- 172.21.15.162:0/2512475525 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b00b4690 con 0x7ff2b00603d0 2020-11-28T21:47:12.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.787+0000 7ff2b562b700 1 --2- 172.21.15.162:0/2512475525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2b00603d0 0x7ff2b00b3a70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7ff2ac002f30 tx=0x7ff2ac006dc0).ready entity=mon.1 client_cookie=5b1134f3a1138e8b server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:12.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.787+0000 7ff29ffff700 1 -- 172.21.15.162:0/2512475525 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2ac0096f0 con 0x7ff2b00603d0 2020-11-28T21:47:12.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.787+0000 7ff29ffff700 1 -- 172.21.15.162:0/2512475525 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2ac009850 con 0x7ff2b00603d0 2020-11-28T21:47:12.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.787+0000 7ff29ffff700 1 -- 172.21.15.162:0/2512475525 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2ac009bb0 con 0x7ff2b00603d0 2020-11-28T21:47:12.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.787+0000 7ff2b662d700 1 -- 172.21.15.162:0/2512475525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2b00603d0 msgr2=0x7ff2b00b3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:12.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.787+0000 7ff2b662d700 1 --2- 172.21.15.162:0/2512475525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2b00603d0 0x7ff2b00b3a70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7ff2ac002f30 tx=0x7ff2ac006dc0).stop 2020-11-28T21:47:12.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.787+0000 7ff2b662d700 1 -- 172.21.15.162:0/2512475525 shutdown_connections 2020-11-28T21:47:12.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.787+0000 7ff2b662d700 1 --2- 172.21.15.162:0/2512475525 >> v2:172.21.15.118:3300/0 conn(0x7ff2b00b4c70 0x7ff2b0143020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:12.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.787+0000 7ff2b662d700 1 --2- 172.21.15.162:0/2512475525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2b00603d0 0x7ff2b00b3a70 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:12.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.787+0000 7ff2b662d700 1 -- 172.21.15.162:0/2512475525 >> 172.21.15.162:0/2512475525 conn(0x7ff2b0066a40 msgr2=0x7ff2b0066e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:12.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.787+0000 7ff2b662d700 1 -- 172.21.15.162:0/2512475525 shutdown_connections 2020-11-28T21:47:12.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.787+0000 7ff2b662d700 1 -- 172.21.15.162:0/2512475525 wait complete. 2020-11-28T21:47:12.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.791+0000 7ff2b662d700 1 Processor -- start 2020-11-28T21:47:12.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.791+0000 7ff2b662d700 1 -- start start 2020-11-28T21:47:12.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.791+0000 7ff2b662d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff2b00603d0 0x7ff2b00b7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:12.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.791+0000 7ff2b662d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2b00b4c70 0x7ff2b01552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:12.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.791+0000 7ff2b662d700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff2b01557a0 0x7ff2b0151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:12.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.791+0000 7ff2b662d700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b00b2f10 con 0x7ff2b00603d0 2020-11-28T21:47:12.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.791+0000 7ff2b662d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b0155c50 con 0x7ff2b00b4c70 2020-11-28T21:47:12.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.791+0000 7ff2b662d700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b00b3100 con 0x7ff2b01557a0 2020-11-28T21:47:12.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.791+0000 7ff2b4e2a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2b00b4c70 0x7ff2b01552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:12.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.791+0000 7ff2b562b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff2b00603d0 0x7ff2b00b7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:12.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.791+0000 7ff2b4e2a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2b00b4c70 0x7ff2b01552b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38722/0 (socket says 172.21.15.162:38722) 2020-11-28T21:47:12.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.791+0000 7ff2b4e2a700 1 -- 172.21.15.162:0/1137989571 learned_addr learned my addr 172.21.15.162:0/1137989571 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:12.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.791+0000 7ff2b5e2c700 1 --2- 172.21.15.162:0/1137989571 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff2b01557a0 0x7ff2b0151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:12.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.791+0000 7ff2b4e2a700 1 -- 172.21.15.162:0/1137989571 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff2b01557a0 msgr2=0x7ff2b0151820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:12.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.791+0000 7ff2b4e2a700 1 --2- 172.21.15.162:0/1137989571 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff2b01557a0 0x7ff2b0151820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:12.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.791+0000 7ff2b4e2a700 1 -- 172.21.15.162:0/1137989571 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff2b00603d0 msgr2=0x7ff2b00b7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:12.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.791+0000 7ff2b4e2a700 1 --2- 172.21.15.162:0/1137989571 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff2b00603d0 0x7ff2b00b7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:12.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.791+0000 7ff2b4e2a700 1 -- 172.21.15.162:0/1137989571 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b00b33a0 con 0x7ff2b00b4c70 2020-11-28T21:47:12.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.791+0000 7ff2b4e2a700 1 --2- 172.21.15.162:0/1137989571 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2b00b4c70 0x7ff2b01552b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7ff2a00010d0 tx=0x7ff2a0007aa0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:12.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.791+0000 7ff29e7fc700 1 -- 172.21.15.162:0/1137989571 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2a00072e0 con 0x7ff2b00b4c70 2020-11-28T21:47:12.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.791+0000 7ff29e7fc700 1 -- 172.21.15.162:0/1137989571 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a0007440 con 0x7ff2b00b4c70 2020-11-28T21:47:12.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.791+0000 7ff2b662d700 1 -- 172.21.15.162:0/1137989571 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2b0154e10 con 0x7ff2b00b4c70 2020-11-28T21:47:12.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.791+0000 7ff29e7fc700 1 -- 172.21.15.162:0/1137989571 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2a00057d0 con 0x7ff2b00b4c70 2020-11-28T21:47:12.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.791+0000 7ff2b662d700 1 -- 172.21.15.162:0/1137989571 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b01459b0 con 0x7ff2b00b4c70 2020-11-28T21:47:12.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.795+0000 7ff2b662d700 1 -- 172.21.15.162:0/1137989571 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2b0154fa0 con 0x7ff2b00b4c70 2020-11-28T21:47:12.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.795+0000 7ff29e7fc700 1 -- 172.21.15.162:0/1137989571 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff2a0010070 con 0x7ff2b00b4c70 2020-11-28T21:47:12.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.795+0000 7ff29e7fc700 1 --2- 172.21.15.162:0/1137989571 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff2900554e0 0x7ff2900579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:12.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.795+0000 7ff29e7fc700 1 -- 172.21.15.162:0/1137989571 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff2a006d440 con 0x7ff2b00b4c70 2020-11-28T21:47:12.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.795+0000 7ff2b562b700 1 --2- 172.21.15.162:0/1137989571 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff2900554e0 0x7ff2900579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:12.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.795+0000 7ff2b562b700 1 --2- 172.21.15.162:0/1137989571 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff2900554e0 0x7ff2900579a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7ff2ac01b000 tx=0x7ff2ac015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:12.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.799+0000 7ff29e7fc700 1 -- 172.21.15.162:0/1137989571 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff2a0043740 con 0x7ff2b00b4c70 2020-11-28T21:47:12.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.983+0000 7ff2b662d700 1 -- 172.21.15.162:0/1137989571 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2b0057050 con 0x7ff2b00b4c70 2020-11-28T21:47:12.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.983+0000 7ff29e7fc700 1 -- 172.21.15.162:0/1137989571 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+2497 (secure 0 0 0) 0x7ff2a0046d60 con 0x7ff2b00b4c70 2020-11-28T21:47:12.986 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:47:12.986 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":10,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4839,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/1153453343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1153453343},{"type":"v1","addr":"172.21.15.162:6836","nonce":1153453343}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8},{"gid":4849,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2101329586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":2101329586},{"type":"v1","addr":"172.21.15.162:6837","nonce":2101329586}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:47:10.330722+0000","modified":"2020-11-28T21:47:11.333518+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4855},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4855":{"gid":4855,"name":"b","rank":0,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.118:6833/3038031199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3038031199},{"type":"v1","addr":"172.21.15.118:6833","nonce":3038031199}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-28T21:47:12.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.987+0000 7ff2b662d700 1 -- 172.21.15.162:0/1137989571 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff2900554e0 msgr2=0x7ff2900579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:12.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.987+0000 7ff2b662d700 1 --2- 172.21.15.162:0/1137989571 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff2900554e0 0x7ff2900579a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7ff2ac01b000 tx=0x7ff2ac015000).stop 2020-11-28T21:47:12.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.987+0000 7ff2b662d700 1 -- 172.21.15.162:0/1137989571 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2b00b4c70 msgr2=0x7ff2b01552b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:12.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.987+0000 7ff2b662d700 1 --2- 172.21.15.162:0/1137989571 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2b00b4c70 0x7ff2b01552b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7ff2a00010d0 tx=0x7ff2a0007aa0).stop 2020-11-28T21:47:12.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.987+0000 7ff2b662d700 1 -- 172.21.15.162:0/1137989571 shutdown_connections 2020-11-28T21:47:12.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.987+0000 7ff2b662d700 1 --2- 172.21.15.162:0/1137989571 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff2900554e0 0x7ff2900579a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:12.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.987+0000 7ff2b662d700 1 --2- 172.21.15.162:0/1137989571 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff2b01557a0 0x7ff2b0151820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:12.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.987+0000 7ff2b662d700 1 --2- 172.21.15.162:0/1137989571 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff2b00603d0 0x7ff2b00b7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:12.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.987+0000 7ff2b662d700 1 --2- 172.21.15.162:0/1137989571 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2b00b4c70 0x7ff2b01552b0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:12.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.987+0000 7ff2b662d700 1 -- 172.21.15.162:0/1137989571 >> 172.21.15.162:0/1137989571 conn(0x7ff2b0066a40 msgr2=0x7ff2b0140640 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:12.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.987+0000 7ff2b662d700 1 -- 172.21.15.162:0/1137989571 shutdown_connections 2020-11-28T21:47:12.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:12.987+0000 7ff2b662d700 1 -- 172.21.15.162:0/1137989571 wait complete. 2020-11-28T21:47:12.991 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 10 2020-11-28T21:47:13.308 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 10, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-28T21:47:10.330722+0000', 'modified': '2020-11-28T21:47:11.333518+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4855}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4855': {'gid': 4855, 'name': 'b', 'rank': 0, 'incarnation': 9, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.118:6833/3038031199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6832', 'nonce': 3038031199}, {'type': 'v1', 'addr': '172.21.15.118:6833', 'nonce': 3038031199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-28T21:47:13.308 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-28T21:47:13.309 INFO:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2020-11-28T21:47:13.441 INFO:teuthology.orchestra.run.smithi118.stdout:{"cluster_fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","whoami":0,"id":4855,"want_state":"up:active","state":"up:active","rank_uptime":3.0957342290000001,"mdsmap_epoch":10,"osdmap_epoch":33,"osdmap_epoch_barrier":32,"uptime":10.326471821} 2020-11-28T21:47:13.442 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", "id": 4855, "mdsmap_epoch": 10, "osdmap_epoch": 33, "osdmap_epoch_barrier": 32, "rank_uptime": 3.095734229, "state": "up:active", "uptime": 10.326471821, "want_state": "up:active", "whoami": 0 } 2020-11-28T21:47:13.443 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:47:13.562 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.559+0000 7fa43c30f700 1 Processor -- start 2020-11-28T21:47:13.562 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.559+0000 7fa43c30f700 1 -- start start 2020-11-28T21:47:13.563 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.559+0000 7fa43c30f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa4340b19f0 0x7fa434142a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:13.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.559+0000 7fa43c30f700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa4340b3230 0x7fa434142f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:13.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.563+0000 7fa43c30f700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa434140440 con 0x7fa434060e50 2020-11-28T21:47:13.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.563+0000 7fa43c30f700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4340b7160 con 0x7fa4340b3230 2020-11-28T21:47:13.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.563+0000 7fa43c30f700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4340b7400 con 0x7fa4340b19f0 2020-11-28T21:47:13.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.563+0000 7fa43a8ac700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa4340b3230 0x7fa434142f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:13.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.563+0000 7fa43a0ab700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fa434060e50 0x7fa4340b2880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:45572/0 (socket says 172.21.15.162:45572) 2020-11-28T21:47:13.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.563+0000 7fa43a0ab700 1 -- 172.21.15.162:0/2439436550 learned_addr learned my addr 172.21.15.162:0/2439436550 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:47:13.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.563+0000 7fa4398aa700 1 --2- 172.21.15.162:0/2439436550 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa4340b19f0 0x7fa434142a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:13.567 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.563+0000 7fa43a8ac700 1 -- 172.21.15.162:0/2439436550 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa4340b19f0 msgr2=0x7fa434142a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:13.567 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.563+0000 7fa43a8ac700 1 --2- 172.21.15.162:0/2439436550 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa4340b19f0 0x7fa434142a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:13.568 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.563+0000 7fa43a8ac700 1 -- 172.21.15.162:0/2439436550 >> v1:172.21.15.162:6789/0 conn(0x7fa434060e50 legacy=0x7fa4340b2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:13.568 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.563+0000 7fa43a8ac700 1 -- 172.21.15.162:0/2439436550 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4340b76a0 con 0x7fa4340b3230 2020-11-28T21:47:13.568 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.563+0000 7fa4390a9700 1 -- 172.21.15.162:0/2439436550 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1956949460 0 0) 0x7fa4300010d0 con 0x7fa434060e50 2020-11-28T21:47:13.568 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.563+0000 7fa43a8ac700 1 --2- 172.21.15.162:0/2439436550 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4340b3230 0x7fa434142f00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7fa42c0032b0 tx=0x7fa42c0091a0).ready entity=mon.1 client_cookie=e6c587624964072d server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:13.569 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.563+0000 7fa4390a9700 1 -- 172.21.15.162:0/2439436550 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa42c00b9a0 con 0x7fa4340b3230 2020-11-28T21:47:13.569 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.563+0000 7fa4390a9700 1 -- 172.21.15.162:0/2439436550 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa42c00bb00 con 0x7fa4340b3230 2020-11-28T21:47:13.569 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.563+0000 7fa4390a9700 1 -- 172.21.15.162:0/2439436550 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa42c00bc70 con 0x7fa4340b3230 2020-11-28T21:47:13.569 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.563+0000 7fa43c30f700 1 -- 172.21.15.162:0/2439436550 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4340b3230 msgr2=0x7fa434142f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:13.569 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.563+0000 7fa43c30f700 1 --2- 172.21.15.162:0/2439436550 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4340b3230 0x7fa434142f00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7fa42c0032b0 tx=0x7fa42c0091a0).stop 2020-11-28T21:47:13.570 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.563+0000 7fa43c30f700 1 -- 172.21.15.162:0/2439436550 shutdown_connections 2020-11-28T21:47:13.570 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.563+0000 7fa43c30f700 1 --2- 172.21.15.162:0/2439436550 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4340b3230 0x7fa434142f00 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:13.570 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.563+0000 7fa43c30f700 1 --2- 172.21.15.162:0/2439436550 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa4340b19f0 0x7fa434142a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:13.570 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.563+0000 7fa43c30f700 1 -- 172.21.15.162:0/2439436550 >> 172.21.15.162:0/2439436550 conn(0x7fa434062830 msgr2=0x7fa434062c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:13.571 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.563+0000 7fa43c30f700 1 -- 172.21.15.162:0/2439436550 shutdown_connections 2020-11-28T21:47:13.571 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.563+0000 7fa43c30f700 1 -- 172.21.15.162:0/2439436550 wait complete. 2020-11-28T21:47:13.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.567+0000 7fa43c30f700 1 Processor -- start 2020-11-28T21:47:13.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.567+0000 7fa43c30f700 1 -- start start 2020-11-28T21:47:13.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.567+0000 7fa43c30f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa434060e50 0x7fa4340af980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:13.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.567+0000 7fa43c30f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4340b19f0 0x7fa4340ac780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:13.574 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.567+0000 7fa43c30f700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa4340b3230 0x7fa43413a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:13.574 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.567+0000 7fa43c30f700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4340b0b00 con 0x7fa4340b3230 2020-11-28T21:47:13.574 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.567+0000 7fa43c30f700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4340af150 con 0x7fa4340b19f0 2020-11-28T21:47:13.574 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.567+0000 7fa43c30f700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4340ae7c0 con 0x7fa434060e50 2020-11-28T21:47:13.575 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.567+0000 7fa4398aa700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4340b19f0 0x7fa4340ac780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:13.575 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.567+0000 7fa43a0ab700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa434060e50 0x7fa4340af980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:13.575 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.567+0000 7fa4398aa700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4340b19f0 0x7fa4340ac780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38736/0 (socket says 172.21.15.162:38736) 2020-11-28T21:47:13.575 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.567+0000 7fa4398aa700 1 -- 172.21.15.162:0/1677264327 learned_addr learned my addr 172.21.15.162:0/1677264327 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:13.575 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.567+0000 7fa43a8ac700 1 --2- 172.21.15.162:0/1677264327 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa4340b3230 0x7fa43413a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:13.576 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.571+0000 7fa4398aa700 1 -- 172.21.15.162:0/1677264327 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa434060e50 msgr2=0x7fa4340af980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:13.576 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.571+0000 7fa4398aa700 1 --2- 172.21.15.162:0/1677264327 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa434060e50 0x7fa4340af980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:13.576 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.571+0000 7fa4398aa700 1 -- 172.21.15.162:0/1677264327 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa4340b3230 msgr2=0x7fa43413a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:13.576 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.571+0000 7fa4398aa700 1 --2- 172.21.15.162:0/1677264327 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa4340b3230 0x7fa43413a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:13.577 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.571+0000 7fa4398aa700 1 -- 172.21.15.162:0/1677264327 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4340aaec0 con 0x7fa4340b19f0 2020-11-28T21:47:13.577 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.571+0000 7fa43a0ab700 1 --2- 172.21.15.162:0/1677264327 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa434060e50 0x7fa4340af980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:47:13.577 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.571+0000 7fa4398aa700 1 --2- 172.21.15.162:0/1677264327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4340b19f0 0x7fa4340ac780 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7fa42800c000 tx=0x7fa428006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:13.577 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.571+0000 7fa4277fe700 1 -- 172.21.15.162:0/1677264327 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa42800ca10 con 0x7fa4340b19f0 2020-11-28T21:47:13.577 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.571+0000 7fa4277fe700 1 -- 172.21.15.162:0/1677264327 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa42800cb70 con 0x7fa4340b19f0 2020-11-28T21:47:13.578 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.571+0000 7fa43c30f700 1 -- 172.21.15.162:0/1677264327 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4340af380 con 0x7fa4340b19f0 2020-11-28T21:47:13.578 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.571+0000 7fa4277fe700 1 -- 172.21.15.162:0/1677264327 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa42800ee80 con 0x7fa4340b19f0 2020-11-28T21:47:13.578 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.571+0000 7fa43c30f700 1 -- 172.21.15.162:0/1677264327 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4340b0400 con 0x7fa4340b19f0 2020-11-28T21:47:13.579 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.571+0000 7fa43c30f700 1 -- 172.21.15.162:0/1677264327 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa434057050 con 0x7fa4340b19f0 2020-11-28T21:47:13.579 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.571+0000 7fa4277fe700 1 -- 172.21.15.162:0/1677264327 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa42801c460 con 0x7fa4340b19f0 2020-11-28T21:47:13.579 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.571+0000 7fa4277fe700 1 --2- 172.21.15.162:0/1677264327 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa414055440 0x7fa414057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:13.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.571+0000 7fa43a0ab700 1 --2- 172.21.15.162:0/1677264327 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa414055440 0x7fa414057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:13.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.571+0000 7fa4277fe700 1 -- 172.21.15.162:0/1677264327 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa42806e2c0 con 0x7fa4340b19f0 2020-11-28T21:47:13.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.575+0000 7fa43a0ab700 1 --2- 172.21.15.162:0/1677264327 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa414055440 0x7fa414057900 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fa43000d000 tx=0x7fa430007830).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:13.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.579+0000 7fa4277fe700 1 -- 172.21.15.162:0/1677264327 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa428044560 con 0x7fa4340b19f0 2020-11-28T21:47:13.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.723+0000 7fa43c30f700 1 -- 172.21.15.162:0/1677264327 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa4340b78a0 con 0x7fa4340b19f0 2020-11-28T21:47:13.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.727+0000 7fa4277fe700 1 -- 172.21.15.162:0/1677264327 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16134 (secure 0 0 0) 0x7fa428047b80 con 0x7fa4340b19f0 2020-11-28T21:47:13.728 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:47:13.728 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":33,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:47:11.334213+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:47:08.251301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2020-11-28T21:47:09.333148+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:47:13.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.727+0000 7fa43c30f700 1 -- 172.21.15.162:0/1677264327 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa414055440 msgr2=0x7fa414057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:13.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.727+0000 7fa43c30f700 1 --2- 172.21.15.162:0/1677264327 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa414055440 0x7fa414057900 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fa43000d000 tx=0x7fa430007830).stop 2020-11-28T21:47:13.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.727+0000 7fa43c30f700 1 -- 172.21.15.162:0/1677264327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4340b19f0 msgr2=0x7fa4340ac780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:13.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.727+0000 7fa43c30f700 1 --2- 172.21.15.162:0/1677264327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4340b19f0 0x7fa4340ac780 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7fa42800c000 tx=0x7fa428006dc0).stop 2020-11-28T21:47:13.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.727+0000 7fa43c30f700 1 -- 172.21.15.162:0/1677264327 shutdown_connections 2020-11-28T21:47:13.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.727+0000 7fa43c30f700 1 --2- 172.21.15.162:0/1677264327 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa414055440 0x7fa414057900 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:13.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.727+0000 7fa43c30f700 1 --2- 172.21.15.162:0/1677264327 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa434060e50 0x7fa4340af980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:13.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.727+0000 7fa43c30f700 1 --2- 172.21.15.162:0/1677264327 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa4340b3230 0x7fa43413a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:13.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.727+0000 7fa43c30f700 1 --2- 172.21.15.162:0/1677264327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4340b19f0 0x7fa4340ac780 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:13.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.727+0000 7fa43c30f700 1 -- 172.21.15.162:0/1677264327 >> 172.21.15.162:0/1677264327 conn(0x7fa434062830 msgr2=0x7fa43413d830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:13.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.727+0000 7fa43c30f700 1 -- 172.21.15.162:0/1677264327 shutdown_connections 2020-11-28T21:47:13.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.727+0000 7fa43c30f700 1 -- 172.21.15.162:0/1677264327 wait complete. 2020-11-28T21:47:13.742 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-28T21:47:08.251301+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2020-11-28T21:47:09.333148+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-28T21:47:13.742 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:47:13.859 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.859+0000 7fd6b2e25700 1 Processor -- start 2020-11-28T21:47:13.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.859+0000 7fd6b2e25700 1 -- start start 2020-11-28T21:47:13.862 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.859+0000 7fd6b2e25700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd6ac0b3ab0 0x7fd6ac05c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:13.862 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.859+0000 7fd6b2e25700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd6ac0b30e0 0x7fd6ac13fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:13.862 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.859+0000 7fd6b2e25700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6ac05baf0 con 0x7fd6ac0b2d00 2020-11-28T21:47:13.862 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.859+0000 7fd6b2e25700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6ac0b0e20 con 0x7fd6ac0b3ab0 2020-11-28T21:47:13.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.859+0000 7fd6b2e25700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac05bd90 con 0x7fd6ac0b30e0 2020-11-28T21:47:13.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.859+0000 7fd6b1e23700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fd6ac0b2d00 0x7fd6ac142ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:45586/0 (socket says 172.21.15.162:45586) 2020-11-28T21:47:13.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.859+0000 7fd6b1e23700 1 -- 172.21.15.162:0/2161499031 learned_addr learned my addr 172.21.15.162:0/2161499031 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:47:13.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.859+0000 7fd6b1622700 1 --2- 172.21.15.162:0/2161499031 >> v2:172.21.15.162:3300/0 conn(0x7fd6ac0b3ab0 0x7fd6ac05c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:13.864 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.859+0000 7fd6b2624700 1 --2- 172.21.15.162:0/2161499031 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd6ac0b30e0 0x7fd6ac13fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:13.864 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.859+0000 7fd6b0e21700 1 -- 172.21.15.162:0/2161499031 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3457234694 0 0) 0x7fd6ac05baf0 con 0x7fd6ac0b2d00 2020-11-28T21:47:13.864 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.859+0000 7fd6b0e21700 1 -- 172.21.15.162:0/2161499031 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd694003230 con 0x7fd6ac0b2d00 2020-11-28T21:47:13.864 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.859+0000 7fd6b1622700 1 -- 172.21.15.162:0/2161499031 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd6ac0b30e0 msgr2=0x7fd6ac13fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:13.864 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.859+0000 7fd6b1622700 1 --2- 172.21.15.162:0/2161499031 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd6ac0b30e0 0x7fd6ac13fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:13.865 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.859+0000 7fd6b1622700 1 -- 172.21.15.162:0/2161499031 >> v1:172.21.15.162:6789/0 conn(0x7fd6ac0b2d00 legacy=0x7fd6ac142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:13.865 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.859+0000 7fd6b1622700 1 -- 172.21.15.162:0/2161499031 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ac05c030 con 0x7fd6ac0b3ab0 2020-11-28T21:47:13.865 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.859+0000 7fd6b1622700 1 --2- 172.21.15.162:0/2161499031 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6ac0b3ab0 0x7fd6ac05c4d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7fd69c002f30 tx=0x7fd69c006dc0).ready entity=mon.1 client_cookie=4a30451b25521d80 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:13.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.859+0000 7fd6b0e21700 1 -- 172.21.15.162:0/2161499031 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd69c0096a0 con 0x7fd6ac0b3ab0 2020-11-28T21:47:13.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.859+0000 7fd6b0e21700 1 -- 172.21.15.162:0/2161499031 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd69c009800 con 0x7fd6ac0b3ab0 2020-11-28T21:47:13.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.859+0000 7fd6b0e21700 1 -- 172.21.15.162:0/2161499031 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd69c009b60 con 0x7fd6ac0b3ab0 2020-11-28T21:47:13.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.859+0000 7fd6b2e25700 1 -- 172.21.15.162:0/2161499031 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6ac0b3ab0 msgr2=0x7fd6ac05c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:13.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.859+0000 7fd6b2e25700 1 --2- 172.21.15.162:0/2161499031 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6ac0b3ab0 0x7fd6ac05c4d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7fd69c002f30 tx=0x7fd69c006dc0).stop 2020-11-28T21:47:13.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.863+0000 7fd6b2e25700 1 -- 172.21.15.162:0/2161499031 shutdown_connections 2020-11-28T21:47:13.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.863+0000 7fd6b2e25700 1 --2- 172.21.15.162:0/2161499031 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd6ac0b30e0 0x7fd6ac13fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:13.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.863+0000 7fd6b2e25700 1 --2- 172.21.15.162:0/2161499031 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6ac0b3ab0 0x7fd6ac05c4d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:13.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.863+0000 7fd6b2e25700 1 -- 172.21.15.162:0/2161499031 >> 172.21.15.162:0/2161499031 conn(0x7fd6ac0604c0 msgr2=0x7fd6ac0608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:13.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.863+0000 7fd6b2e25700 1 -- 172.21.15.162:0/2161499031 shutdown_connections 2020-11-28T21:47:13.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.863+0000 7fd6b2e25700 1 -- 172.21.15.162:0/2161499031 wait complete. 2020-11-28T21:47:13.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.863+0000 7fd6b2e25700 1 Processor -- start 2020-11-28T21:47:13.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.863+0000 7fd6b2e25700 1 -- start start 2020-11-28T21:47:13.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.863+0000 7fd6b2e25700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd6ac0b2d00 0x7fd6ac142700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:13.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.863+0000 7fd6b2e25700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd6ac0b30e0 0x7fd6ac145d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:13.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.863+0000 7fd6b2e25700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6ac0b3ab0 0x7fd6ac0b93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:13.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.863+0000 7fd6b2e25700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac0b8aa0 con 0x7fd6ac0b30e0 2020-11-28T21:47:13.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.863+0000 7fd6b2e25700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac0b86d0 con 0x7fd6ac0b3ab0 2020-11-28T21:47:13.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.863+0000 7fd6b2e25700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac0b8c90 con 0x7fd6ac0b2d00 2020-11-28T21:47:13.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.863+0000 7fd6b2624700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6ac0b3ab0 0x7fd6ac0b93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:13.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.863+0000 7fd6b2624700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6ac0b3ab0 0x7fd6ac0b93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38752/0 (socket says 172.21.15.162:38752) 2020-11-28T21:47:13.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.863+0000 7fd6b2624700 1 -- 172.21.15.162:0/401480439 learned_addr learned my addr 172.21.15.162:0/401480439 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:13.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.863+0000 7fd6b1e23700 1 --2- 172.21.15.162:0/401480439 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd6ac0b2d00 0x7fd6ac142700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:13.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.863+0000 7fd6b1622700 1 --2- 172.21.15.162:0/401480439 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd6ac0b30e0 0x7fd6ac145d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:13.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.867+0000 7fd6b2624700 1 -- 172.21.15.162:0/401480439 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd6ac0b2d00 msgr2=0x7fd6ac142700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:13.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.867+0000 7fd6b2624700 1 --2- 172.21.15.162:0/401480439 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd6ac0b2d00 0x7fd6ac142700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:13.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.867+0000 7fd6b2624700 1 -- 172.21.15.162:0/401480439 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd6ac0b30e0 msgr2=0x7fd6ac145d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:13.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.867+0000 7fd6b2624700 1 --2- 172.21.15.162:0/401480439 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd6ac0b30e0 0x7fd6ac145d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:13.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.867+0000 7fd6b2624700 1 -- 172.21.15.162:0/401480439 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ac0b8f30 con 0x7fd6ac0b3ab0 2020-11-28T21:47:13.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.867+0000 7fd6b1e23700 1 --2- 172.21.15.162:0/401480439 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd6ac0b2d00 0x7fd6ac142700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:47:13.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.867+0000 7fd6b1622700 1 --2- 172.21.15.162:0/401480439 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd6ac0b30e0 0x7fd6ac145d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:47:13.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.867+0000 7fd6b2624700 1 --2- 172.21.15.162:0/401480439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6ac0b3ab0 0x7fd6ac0b93d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7fd6ac141e70 tx=0x7fd6a4006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:13.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.867+0000 7fd69affd700 1 -- 172.21.15.162:0/401480439 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6a400cae0 con 0x7fd6ac0b3ab0 2020-11-28T21:47:13.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.867+0000 7fd69affd700 1 -- 172.21.15.162:0/401480439 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6a400cc40 con 0x7fd6ac0b3ab0 2020-11-28T21:47:13.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.867+0000 7fd69affd700 1 -- 172.21.15.162:0/401480439 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6a40123f0 con 0x7fd6ac0b3ab0 2020-11-28T21:47:13.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.867+0000 7fd6b2e25700 1 -- 172.21.15.162:0/401480439 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6ac146720 con 0x7fd6ac0b3ab0 2020-11-28T21:47:13.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.867+0000 7fd6b2e25700 1 -- 172.21.15.162:0/401480439 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6ac0b9ed0 con 0x7fd6ac0b3ab0 2020-11-28T21:47:13.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.867+0000 7fd6b2e25700 1 -- 172.21.15.162:0/401480439 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6ac0b90c0 con 0x7fd6ac0b3ab0 2020-11-28T21:47:13.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.867+0000 7fd69affd700 1 -- 172.21.15.162:0/401480439 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd6a4012550 con 0x7fd6ac0b3ab0 2020-11-28T21:47:13.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.867+0000 7fd69affd700 1 --2- 172.21.15.162:0/401480439 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd68c055440 0x7fd68c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:13.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.867+0000 7fd69affd700 1 -- 172.21.15.162:0/401480439 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd6a406e5c0 con 0x7fd6ac0b3ab0 2020-11-28T21:47:13.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.867+0000 7fd6b1e23700 1 --2- 172.21.15.162:0/401480439 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd68c055440 0x7fd68c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:13.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.867+0000 7fd6b1e23700 1 --2- 172.21.15.162:0/401480439 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd68c055440 0x7fd68c057900 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fd6a8008000 tx=0x7fd6a8009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:13.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:13.875+0000 7fd69affd700 1 -- 172.21.15.162:0/401480439 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd6a40446d0 con 0x7fd6ac0b3ab0 2020-11-28T21:47:14.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.015+0000 7fd6b2e25700 1 -- 172.21.15.162:0/401480439 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fd6ac057050 con 0x7fd6ac0b3ab0 2020-11-28T21:47:14.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.015+0000 7fd69affd700 1 -- 172.21.15.162:0/401480439 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v33) v1 ==== 110+0+10 (secure 0 0 0) 0x7fd6a4047cf0 con 0x7fd6ac0b3ab0 2020-11-28T21:47:14.019 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:47:14.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.019+0000 7fd6b2e25700 1 -- 172.21.15.162:0/401480439 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd68c055440 msgr2=0x7fd68c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:14.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.019+0000 7fd6b2e25700 1 --2- 172.21.15.162:0/401480439 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd68c055440 0x7fd68c057900 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fd6a8008000 tx=0x7fd6a8009040).stop 2020-11-28T21:47:14.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.019+0000 7fd6b2e25700 1 -- 172.21.15.162:0/401480439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6ac0b3ab0 msgr2=0x7fd6ac0b93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:14.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.019+0000 7fd6b2e25700 1 --2- 172.21.15.162:0/401480439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6ac0b3ab0 0x7fd6ac0b93d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7fd6ac141e70 tx=0x7fd6a4006dc0).stop 2020-11-28T21:47:14.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.019+0000 7fd6b2e25700 1 -- 172.21.15.162:0/401480439 shutdown_connections 2020-11-28T21:47:14.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.019+0000 7fd6b2e25700 1 --2- 172.21.15.162:0/401480439 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd68c055440 0x7fd68c057900 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:14.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.019+0000 7fd6b2e25700 1 --2- 172.21.15.162:0/401480439 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd6ac0b2d00 0x7fd6ac142700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:14.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.019+0000 7fd6b2e25700 1 --2- 172.21.15.162:0/401480439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6ac0b3ab0 0x7fd6ac0b93d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:14.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.019+0000 7fd6b2e25700 1 --2- 172.21.15.162:0/401480439 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd6ac0b30e0 0x7fd6ac145d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:14.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.019+0000 7fd6b2e25700 1 -- 172.21.15.162:0/401480439 >> 172.21.15.162:0/401480439 conn(0x7fd6ac0604c0 msgr2=0x7fd6ac13c650 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:14.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.019+0000 7fd6b2e25700 1 -- 172.21.15.162:0/401480439 shutdown_connections 2020-11-28T21:47:14.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.019+0000 7fd6b2e25700 1 -- 172.21.15.162:0/401480439 wait complete. 2020-11-28T21:47:14.035 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:47:14.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.139+0000 7f582f59e700 1 Processor -- start 2020-11-28T21:47:14.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.139+0000 7f582f59e700 1 -- start start 2020-11-28T21:47:14.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.139+0000 7f582f59e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f58300b3ab0 0x7f583005c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:14.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.139+0000 7f582f59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f58300b30e0 0x7f583013fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:14.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.139+0000 7f582f59e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f583005baf0 con 0x7f58300b2d00 2020-11-28T21:47:14.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.139+0000 7f582f59e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58300b0e20 con 0x7f58300b3ab0 2020-11-28T21:47:14.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.139+0000 7f582f59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f583005bd90 con 0x7f58300b30e0 2020-11-28T21:47:14.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.139+0000 7f582dd9b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f58300b3ab0 0x7f583005c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:14.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.139+0000 7f582dd9b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f58300b3ab0 0x7f583005c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38756/0 (socket says 172.21.15.162:38756) 2020-11-28T21:47:14.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.139+0000 7f582dd9b700 1 -- 172.21.15.162:0/2880422165 learned_addr learned my addr 172.21.15.162:0/2880422165 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:14.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.143+0000 7f582ed9d700 1 --2- 172.21.15.162:0/2880422165 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f58300b30e0 0x7f583013fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:14.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.143+0000 7f582dd9b700 1 -- 172.21.15.162:0/2880422165 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f58300b30e0 msgr2=0x7f583013fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:14.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.143+0000 7f582dd9b700 1 --2- 172.21.15.162:0/2880422165 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f58300b30e0 0x7f583013fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:14.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.143+0000 7f582dd9b700 1 -- 172.21.15.162:0/2880422165 >> v1:172.21.15.118:6789/0 conn(0x7f58300b2d00 legacy=0x7f5830142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:14.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.143+0000 7f582dd9b700 1 -- 172.21.15.162:0/2880422165 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f583005c030 con 0x7f58300b3ab0 2020-11-28T21:47:14.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.143+0000 7f582dd9b700 1 --2- 172.21.15.162:0/2880422165 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58300b3ab0 0x7f583005c4d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f58300acde0 tx=0x7f5818006e80).ready entity=mon.1 client_cookie=46f1a7772a8d9cc4 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:14.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.143+0000 7f582d59a700 1 -- 172.21.15.162:0/2880422165 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58180097b0 con 0x7f58300b3ab0 2020-11-28T21:47:14.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.143+0000 7f582d59a700 1 -- 172.21.15.162:0/2880422165 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5818009910 con 0x7f58300b3ab0 2020-11-28T21:47:14.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.143+0000 7f582f59e700 1 -- 172.21.15.162:0/2880422165 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58300b3ab0 msgr2=0x7f583005c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:14.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.143+0000 7f582d59a700 1 -- 172.21.15.162:0/2880422165 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5818009ac0 con 0x7f58300b3ab0 2020-11-28T21:47:14.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.143+0000 7f582f59e700 1 --2- 172.21.15.162:0/2880422165 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58300b3ab0 0x7f583005c4d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f58300acde0 tx=0x7f5818006e80).stop 2020-11-28T21:47:14.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.143+0000 7f582f59e700 1 -- 172.21.15.162:0/2880422165 shutdown_connections 2020-11-28T21:47:14.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.143+0000 7f582f59e700 1 --2- 172.21.15.162:0/2880422165 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f58300b30e0 0x7f583013fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:14.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.143+0000 7f582f59e700 1 --2- 172.21.15.162:0/2880422165 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58300b3ab0 0x7f583005c4d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:14.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.143+0000 7f582f59e700 1 -- 172.21.15.162:0/2880422165 >> 172.21.15.162:0/2880422165 conn(0x7f58300604c0 msgr2=0x7f58300608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:14.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.143+0000 7f582f59e700 1 -- 172.21.15.162:0/2880422165 shutdown_connections 2020-11-28T21:47:14.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.143+0000 7f582f59e700 1 -- 172.21.15.162:0/2880422165 wait complete. 2020-11-28T21:47:14.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.143+0000 7f582f59e700 1 Processor -- start 2020-11-28T21:47:14.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.143+0000 7f582f59e700 1 -- start start 2020-11-28T21:47:14.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.147+0000 7f582f59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f58300b2d00 0x7f58301412e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:14.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.147+0000 7f582f59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f58300b30e0 0x7f5830145510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:14.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.147+0000 7f582f59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58300b3ab0 0x7f58301429d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:14.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.147+0000 7f582f59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58300b8680 con 0x7f58300b2d00 2020-11-28T21:47:14.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.147+0000 7f582f59e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5830143850 con 0x7f58300b3ab0 2020-11-28T21:47:14.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.147+0000 7f582f59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58300b8d80 con 0x7f58300b30e0 2020-11-28T21:47:14.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.147+0000 7f582ed9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58300b3ab0 0x7f58301429d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:14.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.147+0000 7f582ed9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58300b3ab0 0x7f58301429d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38766/0 (socket says 172.21.15.162:38766) 2020-11-28T21:47:14.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.147+0000 7f582ed9d700 1 -- 172.21.15.162:0/3144057047 learned_addr learned my addr 172.21.15.162:0/3144057047 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:14.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.147+0000 7f582dd9b700 1 --2- 172.21.15.162:0/3144057047 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f58300b30e0 0x7f5830145510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:14.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.147+0000 7f582e59c700 1 --2- 172.21.15.162:0/3144057047 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f58300b2d00 0x7f58301412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:14.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.147+0000 7f582ed9d700 1 -- 172.21.15.162:0/3144057047 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f58300b30e0 msgr2=0x7f5830145510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:14.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.147+0000 7f582ed9d700 1 --2- 172.21.15.162:0/3144057047 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f58300b30e0 0x7f5830145510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:14.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.147+0000 7f582ed9d700 1 -- 172.21.15.162:0/3144057047 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f58300b2d00 msgr2=0x7f58301412e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:14.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.147+0000 7f582ed9d700 1 --2- 172.21.15.162:0/3144057047 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f58300b2d00 0x7f58301412e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:14.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.147+0000 7f582ed9d700 1 -- 172.21.15.162:0/3144057047 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58300b88b0 con 0x7f58300b3ab0 2020-11-28T21:47:14.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.147+0000 7f582dd9b700 1 --2- 172.21.15.162:0/3144057047 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f58300b30e0 0x7f5830145510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:47:14.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.147+0000 7f582ed9d700 1 --2- 172.21.15.162:0/3144057047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58300b3ab0 0x7f58301429d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f582800a000 tx=0x7f5828008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:14.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.147+0000 7f58277fe700 1 -- 172.21.15.162:0/3144057047 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58280127e0 con 0x7f58300b3ab0 2020-11-28T21:47:14.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.147+0000 7f58277fe700 1 -- 172.21.15.162:0/3144057047 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5828012940 con 0x7f58300b3ab0 2020-11-28T21:47:14.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.147+0000 7f582f59e700 1 -- 172.21.15.162:0/3144057047 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5830145c90 con 0x7f58300b3ab0 2020-11-28T21:47:14.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.147+0000 7f58277fe700 1 -- 172.21.15.162:0/3144057047 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5828012ca0 con 0x7f58300b3ab0 2020-11-28T21:47:14.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.147+0000 7f582f59e700 1 -- 172.21.15.162:0/3144057047 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58300b9e50 con 0x7f58300b3ab0 2020-11-28T21:47:14.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.147+0000 7f582f59e700 1 -- 172.21.15.162:0/3144057047 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58300b8ef0 con 0x7f58300b3ab0 2020-11-28T21:47:14.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.147+0000 7f58277fe700 1 -- 172.21.15.162:0/3144057047 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5828013480 con 0x7f58300b3ab0 2020-11-28T21:47:14.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.147+0000 7f58277fe700 1 --2- 172.21.15.162:0/3144057047 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f58080553b0 0x7f5808057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:14.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.147+0000 7f58277fe700 1 -- 172.21.15.162:0/3144057047 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f582806e680 con 0x7f58300b3ab0 2020-11-28T21:47:14.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.151+0000 7f582e59c700 1 --2- 172.21.15.162:0/3144057047 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f58080553b0 0x7f5808057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:14.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.151+0000 7f582e59c700 1 --2- 172.21.15.162:0/3144057047 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f58080553b0 0x7f5808057870 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f582000b000 tx=0x7f5820009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:14.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.155+0000 7f58277fe700 1 -- 172.21.15.162:0/3144057047 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5828044920 con 0x7f58300b3ab0 2020-11-28T21:47:14.295 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.295+0000 7f582f59e700 1 -- 172.21.15.162:0/3144057047 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5830143990 con 0x7f58300b3ab0 2020-11-28T21:47:14.296 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.295+0000 7f58277fe700 1 -- 172.21.15.162:0/3144057047 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v33) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5828047f40 con 0x7f58300b3ab0 2020-11-28T21:47:14.296 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:47:14.298 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.295+0000 7f582f59e700 1 -- 172.21.15.162:0/3144057047 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f58080553b0 msgr2=0x7f5808057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:14.298 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.295+0000 7f582f59e700 1 --2- 172.21.15.162:0/3144057047 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f58080553b0 0x7f5808057870 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f582000b000 tx=0x7f5820009040).stop 2020-11-28T21:47:14.298 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.295+0000 7f582f59e700 1 -- 172.21.15.162:0/3144057047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58300b3ab0 msgr2=0x7f58301429d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:14.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.295+0000 7f582f59e700 1 --2- 172.21.15.162:0/3144057047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58300b3ab0 0x7f58301429d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f582800a000 tx=0x7f5828008040).stop 2020-11-28T21:47:14.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.295+0000 7f582f59e700 1 -- 172.21.15.162:0/3144057047 shutdown_connections 2020-11-28T21:47:14.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.295+0000 7f582f59e700 1 --2- 172.21.15.162:0/3144057047 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f58080553b0 0x7f5808057870 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:14.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.295+0000 7f582f59e700 1 --2- 172.21.15.162:0/3144057047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58300b3ab0 0x7f58301429d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:14.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.295+0000 7f582f59e700 1 --2- 172.21.15.162:0/3144057047 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f58300b2d00 0x7f58301412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:14.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.295+0000 7f582f59e700 1 --2- 172.21.15.162:0/3144057047 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f58300b30e0 0x7f5830145510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:14.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.295+0000 7f582f59e700 1 -- 172.21.15.162:0/3144057047 >> 172.21.15.162:0/3144057047 conn(0x7f58300604c0 msgr2=0x7f58301468d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:14.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.295+0000 7f582f59e700 1 -- 172.21.15.162:0/3144057047 shutdown_connections 2020-11-28T21:47:14.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.295+0000 7f582f59e700 1 -- 172.21.15.162:0/3144057047 wait complete. 2020-11-28T21:47:14.316 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-28T21:47:14.428 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.427+0000 7f6cd9ba4700 1 Processor -- start 2020-11-28T21:47:14.428 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.427+0000 7f6cd9ba4700 1 -- start start 2020-11-28T21:47:14.432 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.427+0000 7f6cd9ba4700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f6cd40b9ce0 0x7f6cd41433c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:14.432 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.427+0000 7f6cd9ba4700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6cd40605c0 0x7f6cd4067610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:14.432 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.427+0000 7f6cd9ba4700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cd4143950 con 0x7f6cd40b3c30 2020-11-28T21:47:14.433 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.427+0000 7f6cd9ba4700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cd40ba0c0 con 0x7f6cd40b9ce0 2020-11-28T21:47:14.433 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.427+0000 7f6cd9ba4700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cd4066c30 con 0x7f6cd40605c0 2020-11-28T21:47:14.433 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.427+0000 7f6cd2ffd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6cd40605c0 0x7f6cd4067610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:14.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.427+0000 7f6cd2ffd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6cd40605c0 0x7f6cd4067610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38774/0 (socket says 172.21.15.162:38774) 2020-11-28T21:47:14.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.427+0000 7f6cd2ffd700 1 -- 172.21.15.162:0/4052999113 learned_addr learned my addr 172.21.15.162:0/4052999113 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:14.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.427+0000 7f6cd37fe700 1 --2- 172.21.15.162:0/4052999113 >> v2:172.21.15.118:3300/0 conn(0x7f6cd40b9ce0 0x7f6cd41433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:14.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.427+0000 7f6cd2ffd700 1 -- 172.21.15.162:0/4052999113 >> v2:172.21.15.118:3300/0 conn(0x7f6cd40b9ce0 msgr2=0x7f6cd41433c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:14.435 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.427+0000 7f6cd2ffd700 1 --2- 172.21.15.162:0/4052999113 >> v2:172.21.15.118:3300/0 conn(0x7f6cd40b9ce0 0x7f6cd41433c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:14.435 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.427+0000 7f6cd2ffd700 1 -- 172.21.15.162:0/4052999113 >> v1:172.21.15.118:6789/0 conn(0x7f6cd40b3c30 legacy=0x7f6cd413fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:14.435 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.427+0000 7f6cd2ffd700 1 -- 172.21.15.162:0/4052999113 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd4066ed0 con 0x7f6cd40605c0 2020-11-28T21:47:14.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.427+0000 7f6cd2ffd700 1 --2- 172.21.15.162:0/4052999113 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6cd40605c0 0x7f6cd4067610 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f6cc8002f30 tx=0x7f6cc8006e80).ready entity=mon.1 client_cookie=ff685a0c3a45591 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:14.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.427+0000 7f6cd27fc700 1 -- 172.21.15.162:0/4052999113 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6cc80097f0 con 0x7f6cd40605c0 2020-11-28T21:47:14.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.427+0000 7f6cd27fc700 1 -- 172.21.15.162:0/4052999113 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cc8009950 con 0x7f6cd40605c0 2020-11-28T21:47:14.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.427+0000 7f6cd27fc700 1 -- 172.21.15.162:0/4052999113 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6cc8009cb0 con 0x7f6cd40605c0 2020-11-28T21:47:14.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.427+0000 7f6cd9ba4700 1 -- 172.21.15.162:0/4052999113 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6cd40605c0 msgr2=0x7f6cd4067610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:14.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.427+0000 7f6cd9ba4700 1 --2- 172.21.15.162:0/4052999113 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6cd40605c0 0x7f6cd4067610 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f6cc8002f30 tx=0x7f6cc8006e80).stop 2020-11-28T21:47:14.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.427+0000 7f6cd9ba4700 1 -- 172.21.15.162:0/4052999113 shutdown_connections 2020-11-28T21:47:14.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.427+0000 7f6cd9ba4700 1 --2- 172.21.15.162:0/4052999113 >> v2:172.21.15.118:3300/0 conn(0x7f6cd40b9ce0 0x7f6cd41433c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:14.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.427+0000 7f6cd9ba4700 1 --2- 172.21.15.162:0/4052999113 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6cd40605c0 0x7f6cd4067610 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:14.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.427+0000 7f6cd9ba4700 1 -- 172.21.15.162:0/4052999113 >> 172.21.15.162:0/4052999113 conn(0x7f6cd405c430 msgr2=0x7f6cd40b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:14.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.427+0000 7f6cd9ba4700 1 -- 172.21.15.162:0/4052999113 shutdown_connections 2020-11-28T21:47:14.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.427+0000 7f6cd9ba4700 1 -- 172.21.15.162:0/4052999113 wait complete. 2020-11-28T21:47:14.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.431+0000 7f6cd9ba4700 1 Processor -- start 2020-11-28T21:47:14.444 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.431+0000 7f6cd9ba4700 1 -- start start 2020-11-28T21:47:14.444 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.431+0000 7f6cd9ba4700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6cd40605c0 0x7f6cd40b2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:14.444 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.431+0000 7f6cd9ba4700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6cd40b3c30 0x7f6cd408fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:14.444 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.431+0000 7f6cd9ba4700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6cd40b9ce0 0x7f6cd408ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:14.444 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.431+0000 7f6cd9ba4700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd41455d0 con 0x7f6cd40b9ce0 2020-11-28T21:47:14.445 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.431+0000 7f6cd9ba4700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd4146270 con 0x7f6cd40605c0 2020-11-28T21:47:14.445 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.431+0000 7f6cd9ba4700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd4145830 con 0x7f6cd40b3c30 2020-11-28T21:47:14.445 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.431+0000 7f6cd37fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6cd40605c0 0x7f6cd40b2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:14.445 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.431+0000 7f6cd37fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6cd40605c0 0x7f6cd40b2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38776/0 (socket says 172.21.15.162:38776) 2020-11-28T21:47:14.446 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.431+0000 7f6cd37fe700 1 -- 172.21.15.162:0/1170085955 learned_addr learned my addr 172.21.15.162:0/1170085955 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:14.446 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.431+0000 7f6cd2ffd700 1 --2- 172.21.15.162:0/1170085955 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6cd40b3c30 0x7f6cd408fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:14.446 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.431+0000 7f6cd3fff700 1 --2- 172.21.15.162:0/1170085955 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6cd40b9ce0 0x7f6cd408ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:14.446 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.431+0000 7f6cd37fe700 1 -- 172.21.15.162:0/1170085955 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6cd40b3c30 msgr2=0x7f6cd408fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:14.447 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.431+0000 7f6cd37fe700 1 --2- 172.21.15.162:0/1170085955 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6cd40b3c30 0x7f6cd408fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:14.447 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.431+0000 7f6cd37fe700 1 -- 172.21.15.162:0/1170085955 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6cd40b9ce0 msgr2=0x7f6cd408ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:14.447 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.431+0000 7f6cd37fe700 1 --2- 172.21.15.162:0/1170085955 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6cd40b9ce0 0x7f6cd408ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:14.448 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.431+0000 7f6cd37fe700 1 -- 172.21.15.162:0/1170085955 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd4145ad0 con 0x7f6cd40605c0 2020-11-28T21:47:14.448 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.431+0000 7f6cd3fff700 1 --2- 172.21.15.162:0/1170085955 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6cd40b9ce0 0x7f6cd408ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:47:14.448 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.431+0000 7f6cd37fe700 1 --2- 172.21.15.162:0/1170085955 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6cd40605c0 0x7f6cd40b2a50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f6cc400c000 tx=0x7f6cc4006d60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:14.448 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.431+0000 7f6cd0ff9700 1 -- 172.21.15.162:0/1170085955 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6cc400ca10 con 0x7f6cd40605c0 2020-11-28T21:47:14.448 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.431+0000 7f6cd0ff9700 1 -- 172.21.15.162:0/1170085955 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cc400cb70 con 0x7f6cd40605c0 2020-11-28T21:47:14.449 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.431+0000 7f6cd9ba4700 1 -- 172.21.15.162:0/1170085955 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cd40b31f0 con 0x7f6cd40605c0 2020-11-28T21:47:14.449 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.431+0000 7f6cd0ff9700 1 -- 172.21.15.162:0/1170085955 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6cc40123f0 con 0x7f6cd40605c0 2020-11-28T21:47:14.449 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.435+0000 7f6cd9ba4700 1 -- 172.21.15.162:0/1170085955 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cd4146730 con 0x7f6cd40605c0 2020-11-28T21:47:14.449 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.435+0000 7f6cd9ba4700 1 -- 172.21.15.162:0/1170085955 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cd4057050 con 0x7f6cd40605c0 2020-11-28T21:47:14.449 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.435+0000 7f6cd0ff9700 1 -- 172.21.15.162:0/1170085955 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6cc4012550 con 0x7f6cd40605c0 2020-11-28T21:47:14.450 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.435+0000 7f6cd0ff9700 1 --2- 172.21.15.162:0/1170085955 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6cb40553b0 0x7f6cb4057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:14.450 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.435+0000 7f6cd0ff9700 1 -- 172.21.15.162:0/1170085955 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6cc406e110 con 0x7f6cd40605c0 2020-11-28T21:47:14.450 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.435+0000 7f6cd2ffd700 1 --2- 172.21.15.162:0/1170085955 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6cb40553b0 0x7f6cb4057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:14.450 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.435+0000 7f6cd2ffd700 1 --2- 172.21.15.162:0/1170085955 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6cb40553b0 0x7f6cb4057870 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f6cc801a000 tx=0x7f6cc8015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:14.451 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.443+0000 7f6cd0ff9700 1 -- 172.21.15.162:0/1170085955 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6cc40443c0 con 0x7f6cd40605c0 2020-11-28T21:47:14.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.587+0000 7f6cd9ba4700 1 -- 172.21.15.162:0/1170085955 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6cd4145c60 con 0x7f6cd40605c0 2020-11-28T21:47:14.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.587+0000 7f6cd0ff9700 1 -- 172.21.15.162:0/1170085955 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) v1 ==== 104+0+10 (secure 0 0 0) 0x7f6cc40479e0 con 0x7f6cd40605c0 2020-11-28T21:47:14.589 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:47:14.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.587+0000 7f6cd9ba4700 1 -- 172.21.15.162:0/1170085955 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6cb40553b0 msgr2=0x7f6cb4057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:14.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.587+0000 7f6cd9ba4700 1 --2- 172.21.15.162:0/1170085955 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6cb40553b0 0x7f6cb4057870 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f6cc801a000 tx=0x7f6cc8015000).stop 2020-11-28T21:47:14.597 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.587+0000 7f6cd9ba4700 1 -- 172.21.15.162:0/1170085955 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6cd40605c0 msgr2=0x7f6cd40b2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:14.597 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.587+0000 7f6cd9ba4700 1 --2- 172.21.15.162:0/1170085955 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6cd40605c0 0x7f6cd40b2a50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f6cc400c000 tx=0x7f6cc4006d60).stop 2020-11-28T21:47:14.597 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.587+0000 7f6cd9ba4700 1 -- 172.21.15.162:0/1170085955 shutdown_connections 2020-11-28T21:47:14.597 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.587+0000 7f6cd9ba4700 1 --2- 172.21.15.162:0/1170085955 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6cb40553b0 0x7f6cb4057870 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:14.598 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.587+0000 7f6cd9ba4700 1 --2- 172.21.15.162:0/1170085955 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6cd40b9ce0 0x7f6cd408ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:14.598 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.587+0000 7f6cd9ba4700 1 --2- 172.21.15.162:0/1170085955 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6cd40605c0 0x7f6cd40b2a50 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:14.598 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.587+0000 7f6cd9ba4700 1 --2- 172.21.15.162:0/1170085955 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6cd40b3c30 0x7f6cd408fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:14.598 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.587+0000 7f6cd9ba4700 1 -- 172.21.15.162:0/1170085955 >> 172.21.15.162:0/1170085955 conn(0x7f6cd405c430 msgr2=0x7f6cd409e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:14.598 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.587+0000 7f6cd9ba4700 1 -- 172.21.15.162:0/1170085955 shutdown_connections 2020-11-28T21:47:14.599 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.587+0000 7f6cd9ba4700 1 -- 172.21.15.162:0/1170085955 wait complete. 2020-11-28T21:47:14.608 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-28T21:47:14.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.727+0000 7f6f7ada9700 1 Processor -- start 2020-11-28T21:47:14.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.727+0000 7f6f7ada9700 1 -- start start 2020-11-28T21:47:14.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.727+0000 7f6f7ada9700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f6f740a9960 0x7f6f740a9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:14.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.727+0000 7f6f7ada9700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6f74097f20 0x7f6f7414b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:14.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.727+0000 7f6f7ada9700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f740a8f50 con 0x7f6f74097ac0 2020-11-28T21:47:14.733 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.727+0000 7f6f7ada9700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f740b0b30 con 0x7f6f740a9960 2020-11-28T21:47:14.733 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.727+0000 7f6f7ada9700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f740a9180 con 0x7f6f74097f20 2020-11-28T21:47:14.742 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.731+0000 7f6f7a5a8700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6f74097f20 0x7f6f7414b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:14.743 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.731+0000 7f6f79da7700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f6f740a9960 0x7f6f740a9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:14.743 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.731+0000 7f6f795a6700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f6f74097ac0 0x7f6f74141330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.162:35776/0 (socket says 172.21.15.162:35776) 2020-11-28T21:47:14.743 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.731+0000 7f6f795a6700 1 -- 172.21.15.162:0/2904715317 learned_addr learned my addr 172.21.15.162:0/2904715317 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:47:14.744 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.731+0000 7f6f78da5700 1 -- 172.21.15.162:0/2904715317 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3077259878 0 0) 0x7f6f740a8f50 con 0x7f6f74097ac0 2020-11-28T21:47:14.744 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.731+0000 7f6f78da5700 1 -- 172.21.15.162:0/2904715317 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f5c003230 con 0x7f6f74097ac0 2020-11-28T21:47:14.744 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.731+0000 7f6f7a5a8700 1 -- 172.21.15.162:0/2904715317 >> v2:172.21.15.118:3300/0 conn(0x7f6f740a9960 msgr2=0x7f6f740a9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:14.745 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.731+0000 7f6f7a5a8700 1 --2- 172.21.15.162:0/2904715317 >> v2:172.21.15.118:3300/0 conn(0x7f6f740a9960 0x7f6f740a9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:14.745 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.731+0000 7f6f7a5a8700 1 -- 172.21.15.162:0/2904715317 >> v1:172.21.15.118:6789/0 conn(0x7f6f74097ac0 legacy=0x7f6f74141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:14.745 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.731+0000 7f6f7a5a8700 1 -- 172.21.15.162:0/2904715317 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f74060c60 con 0x7f6f74097f20 2020-11-28T21:47:14.745 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.731+0000 7f6f79da7700 1 --2- 172.21.15.162:0/2904715317 >> v2:172.21.15.118:3300/0 conn(0x7f6f740a9960 0x7f6f740a9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:47:14.745 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.731+0000 7f6f7a5a8700 1 --2- 172.21.15.162:0/2904715317 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6f74097f20 0x7f6f7414b1e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f6f70002f30 tx=0x7f6f70006dc0).ready entity=mon.2 client_cookie=d599b1a0e4562721 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:14.746 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.731+0000 7f6f78da5700 1 -- 172.21.15.162:0/2904715317 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f700096a0 con 0x7f6f74097f20 2020-11-28T21:47:14.746 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.731+0000 7f6f78da5700 1 -- 172.21.15.162:0/2904715317 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f70009800 con 0x7f6f74097f20 2020-11-28T21:47:14.746 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.731+0000 7f6f78da5700 1 -- 172.21.15.162:0/2904715317 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f70009b60 con 0x7f6f74097f20 2020-11-28T21:47:14.747 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.731+0000 7f6f7ada9700 1 -- 172.21.15.162:0/2904715317 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6f74097f20 msgr2=0x7f6f7414b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:14.747 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.731+0000 7f6f7ada9700 1 --2- 172.21.15.162:0/2904715317 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6f74097f20 0x7f6f7414b1e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f6f70002f30 tx=0x7f6f70006dc0).stop 2020-11-28T21:47:14.748 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.731+0000 7f6f7ada9700 1 -- 172.21.15.162:0/2904715317 shutdown_connections 2020-11-28T21:47:14.748 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.731+0000 7f6f7ada9700 1 --2- 172.21.15.162:0/2904715317 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6f74097f20 0x7f6f7414b1e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:14.750 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.731+0000 7f6f7ada9700 1 --2- 172.21.15.162:0/2904715317 >> v2:172.21.15.118:3300/0 conn(0x7f6f740a9960 0x7f6f740a9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:14.751 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.731+0000 7f6f7ada9700 1 -- 172.21.15.162:0/2904715317 >> 172.21.15.162:0/2904715317 conn(0x7f6f740b5520 msgr2=0x7f6f740b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:14.751 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.731+0000 7f6f7ada9700 1 -- 172.21.15.162:0/2904715317 shutdown_connections 2020-11-28T21:47:14.752 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.731+0000 7f6f7ada9700 1 -- 172.21.15.162:0/2904715317 wait complete. 2020-11-28T21:47:14.752 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.735+0000 7f6f7ada9700 1 Processor -- start 2020-11-28T21:47:14.752 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.735+0000 7f6f7ada9700 1 -- start start 2020-11-28T21:47:14.753 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.735+0000 7f6f7ada9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f74097ac0 0x7f6f74064ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:14.753 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.735+0000 7f6f7ada9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f74097f20 0x7f6f74151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:14.754 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.735+0000 7f6f7ada9700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6f740a9960 0x7f6f741523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:14.754 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.735+0000 7f6f7ada9700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f74067860 con 0x7f6f74097f20 2020-11-28T21:47:14.754 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.735+0000 7f6f7ada9700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f74067a90 con 0x7f6f74097ac0 2020-11-28T21:47:14.755 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.735+0000 7f6f7ada9700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f74099ed0 con 0x7f6f740a9960 2020-11-28T21:47:14.755 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.735+0000 7f6f79da7700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f74097ac0 0x7f6f74064ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:14.755 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.735+0000 7f6f79da7700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f74097ac0 0x7f6f74064ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38788/0 (socket says 172.21.15.162:38788) 2020-11-28T21:47:14.756 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.735+0000 7f6f79da7700 1 -- 172.21.15.162:0/4121814007 learned_addr learned my addr 172.21.15.162:0/4121814007 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:14.756 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.735+0000 7f6f795a6700 1 --2- 172.21.15.162:0/4121814007 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f74097f20 0x7f6f74151ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:14.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.735+0000 7f6f7a5a8700 1 --2- 172.21.15.162:0/4121814007 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6f740a9960 0x7f6f741523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:14.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.739+0000 7f6f79da7700 1 -- 172.21.15.162:0/4121814007 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6f740a9960 msgr2=0x7f6f741523b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:14.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.739+0000 7f6f79da7700 1 --2- 172.21.15.162:0/4121814007 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6f740a9960 0x7f6f741523b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:14.758 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.739+0000 7f6f79da7700 1 -- 172.21.15.162:0/4121814007 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f74097f20 msgr2=0x7f6f74151ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:14.758 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.739+0000 7f6f79da7700 1 --2- 172.21.15.162:0/4121814007 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f74097f20 0x7f6f74151ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:14.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.739+0000 7f6f79da7700 1 -- 172.21.15.162:0/4121814007 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f74151b40 con 0x7f6f74097ac0 2020-11-28T21:47:14.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.739+0000 7f6f79da7700 1 --2- 172.21.15.162:0/4121814007 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f74097ac0 0x7f6f74064ae0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7f6f6c00c000 tx=0x7f6f6c0091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:14.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.739+0000 7f6f62ffd700 1 -- 172.21.15.162:0/4121814007 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f6c00cbd0 con 0x7f6f74097ac0 2020-11-28T21:47:14.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.739+0000 7f6f62ffd700 1 -- 172.21.15.162:0/4121814007 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6c00cd30 con 0x7f6f74097ac0 2020-11-28T21:47:14.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.739+0000 7f6f7ada9700 1 -- 172.21.15.162:0/4121814007 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f740aff70 con 0x7f6f74097ac0 2020-11-28T21:47:14.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.739+0000 7f6f62ffd700 1 -- 172.21.15.162:0/4121814007 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f6c00cea0 con 0x7f6f74097ac0 2020-11-28T21:47:14.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.739+0000 7f6f7ada9700 1 -- 172.21.15.162:0/4121814007 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f74144730 con 0x7f6f74097ac0 2020-11-28T21:47:14.761 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.739+0000 7f6f7ada9700 1 -- 172.21.15.162:0/4121814007 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f74151cd0 con 0x7f6f74097ac0 2020-11-28T21:47:14.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.739+0000 7f6f62ffd700 1 -- 172.21.15.162:0/4121814007 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6f6c0125e0 con 0x7f6f74097ac0 2020-11-28T21:47:14.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.739+0000 7f6f62ffd700 1 --2- 172.21.15.162:0/4121814007 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6f540554e0 0x7f6f540579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:14.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.739+0000 7f6f62ffd700 1 -- 172.21.15.162:0/4121814007 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6f6c06e8e0 con 0x7f6f74097ac0 2020-11-28T21:47:14.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.739+0000 7f6f795a6700 1 --2- 172.21.15.162:0/4121814007 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6f540554e0 0x7f6f540579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:14.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.739+0000 7f6f795a6700 1 --2- 172.21.15.162:0/4121814007 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6f540554e0 0x7f6f540579a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f6f6400d000 tx=0x7f6f64009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:14.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.747+0000 7f6f62ffd700 1 -- 172.21.15.162:0/4121814007 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6f6c044b80 con 0x7f6f74097ac0 2020-11-28T21:47:14.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.887+0000 7f6f7ada9700 1 -- 172.21.15.162:0/4121814007 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6f7409a090 con 0x7f6f74097ac0 2020-11-28T21:47:14.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.887+0000 7f6f62ffd700 1 -- 172.21.15.162:0/4121814007 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) v1 ==== 100+0+10 (secure 0 0 0) 0x7f6f6c0481a0 con 0x7f6f74097ac0 2020-11-28T21:47:14.891 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:47:14.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.891+0000 7f6f7ada9700 1 -- 172.21.15.162:0/4121814007 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6f540554e0 msgr2=0x7f6f540579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:14.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.891+0000 7f6f7ada9700 1 --2- 172.21.15.162:0/4121814007 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6f540554e0 0x7f6f540579a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f6f6400d000 tx=0x7f6f64009040).stop 2020-11-28T21:47:14.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.891+0000 7f6f7ada9700 1 -- 172.21.15.162:0/4121814007 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f74097ac0 msgr2=0x7f6f74064ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:14.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.891+0000 7f6f7ada9700 1 --2- 172.21.15.162:0/4121814007 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f74097ac0 0x7f6f74064ae0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7f6f6c00c000 tx=0x7f6f6c0091a0).stop 2020-11-28T21:47:14.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.891+0000 7f6f7ada9700 1 -- 172.21.15.162:0/4121814007 shutdown_connections 2020-11-28T21:47:14.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.891+0000 7f6f7ada9700 1 --2- 172.21.15.162:0/4121814007 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6f540554e0 0x7f6f540579a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:14.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.891+0000 7f6f7ada9700 1 --2- 172.21.15.162:0/4121814007 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6f740a9960 0x7f6f741523b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:14.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.891+0000 7f6f7ada9700 1 --2- 172.21.15.162:0/4121814007 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f74097ac0 0x7f6f74064ae0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:14.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.891+0000 7f6f7ada9700 1 --2- 172.21.15.162:0/4121814007 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f74097f20 0x7f6f74151ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:14.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.891+0000 7f6f7ada9700 1 -- 172.21.15.162:0/4121814007 >> 172.21.15.162:0/4121814007 conn(0x7f6f740b5520 msgr2=0x7f6f7413fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:14.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.891+0000 7f6f7ada9700 1 -- 172.21.15.162:0/4121814007 shutdown_connections 2020-11-28T21:47:14.896 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:14.891+0000 7f6f7ada9700 1 -- 172.21.15.162:0/4121814007 wait complete. 2020-11-28T21:47:14.904 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-28T21:47:14.905 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:47:15.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.015+0000 7f6067fff700 1 Processor -- start 2020-11-28T21:47:15.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.015+0000 7f6067fff700 1 -- start start 2020-11-28T21:47:15.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.015+0000 7f6067fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f60680b4c70 0x7f60680b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:15.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.015+0000 7f6067fff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60680b4110 con 0x7f60680603d0 2020-11-28T21:47:15.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.015+0000 7f6067fff700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60680b43f0 con 0x7f6068142a00 2020-11-28T21:47:15.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.015+0000 7f6067fff700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6068060f60 con 0x7f60680b4c70 2020-11-28T21:47:15.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.015+0000 7f605e7fc700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f6068142a00 0x7f6068143020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:45642/0 (socket says 172.21.15.162:45642) 2020-11-28T21:47:15.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.015+0000 7f605e7fc700 1 -- 172.21.15.162:0/750928169 learned_addr learned my addr 172.21.15.162:0/750928169 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:47:15.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.015+0000 7f60677fe700 1 --2- 172.21.15.162:0/750928169 >> v2:172.21.15.162:3300/0 conn(0x7f60680b4c70 0x7f60680b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:15.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.015+0000 7f60667fc700 1 -- 172.21.15.162:0/750928169 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4209851649 0 0) 0x7f60680b43f0 con 0x7f6068142a00 2020-11-28T21:47:15.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.015+0000 7f60667fc700 1 -- 172.21.15.162:0/750928169 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6048003230 con 0x7f6068142a00 2020-11-28T21:47:15.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.015+0000 7f60667fc700 1 -- 172.21.15.162:0/750928169 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3672197150 0 0) 0x7f6048003230 con 0x7f6068142a00 2020-11-28T21:47:15.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.015+0000 7f60667fc700 1 -- 172.21.15.162:0/750928169 >> v2:172.21.15.162:3300/0 conn(0x7f60680b4c70 msgr2=0x7f60680b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:15.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.015+0000 7f60667fc700 1 --2- 172.21.15.162:0/750928169 >> v2:172.21.15.162:3300/0 conn(0x7f60680b4c70 0x7f60680b3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:15.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.015+0000 7f60667fc700 1 -- 172.21.15.162:0/750928169 >> v1:172.21.15.118:6789/0 conn(0x7f60680603d0 legacy=0x7f606809cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:15.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.015+0000 7f60667fc700 1 -- 172.21.15.162:0/750928169 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60680b4690 con 0x7f6068142a00 2020-11-28T21:47:15.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.015+0000 7f60667fc700 1 -- 172.21.15.162:0/750928169 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f6050001710 con 0x7f6068142a00 2020-11-28T21:47:15.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.015+0000 7f60667fc700 1 -- 172.21.15.162:0/750928169 >> v1:172.21.15.162:6789/0 conn(0x7f6068142a00 legacy=0x7f6068143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:15.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.015+0000 7f60667fc700 1 --2- 172.21.15.162:0/750928169 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6048003720 0x7f6048003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:15.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.015+0000 7f60667fc700 1 --2- 172.21.15.162:0/750928169 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60480044d0 0x7f6048004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:15.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.015+0000 7f60667fc700 1 --2- 172.21.15.162:0/750928169 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6048004e40 0x7f604800b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:15.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.015+0000 7f60667fc700 1 -- 172.21.15.162:0/750928169 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6068060f60 con 0x7f6048004e40 2020-11-28T21:47:15.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.015+0000 7f60667fc700 1 -- 172.21.15.162:0/750928169 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6048004250 con 0x7f60480044d0 2020-11-28T21:47:15.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.015+0000 7f60667fc700 1 -- 172.21.15.162:0/750928169 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f604800bc10 con 0x7f6048003720 2020-11-28T21:47:15.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.015+0000 7f60677fe700 1 --2- 172.21.15.162:0/750928169 >> v2:172.21.15.162:3300/0 conn(0x7f60680b4c70 0x7f60680b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:47:15.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.015+0000 7f605e7fc700 1 --2- 172.21.15.162:0/750928169 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60480044d0 0x7f6048004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:15.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.015+0000 7f6066ffd700 1 --2- 172.21.15.162:0/750928169 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6048003720 0x7f6048003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:15.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.015+0000 7f60677fe700 1 --2- 172.21.15.162:0/750928169 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6048004e40 0x7f604800b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:15.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.015+0000 7f605e7fc700 1 -- 172.21.15.162:0/750928169 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6048003720 msgr2=0x7f6048003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:15.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.015+0000 7f605e7fc700 1 --2- 172.21.15.162:0/750928169 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6048003720 0x7f6048003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:15.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.015+0000 7f605e7fc700 1 -- 172.21.15.162:0/750928169 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6048004e40 msgr2=0x7f604800b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:15.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.015+0000 7f605e7fc700 1 --2- 172.21.15.162:0/750928169 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6048004e40 0x7f604800b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:15.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.015+0000 7f60677fe700 1 -- 172.21.15.162:0/750928169 reap_dead start 2020-11-28T21:47:15.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.015+0000 7f605e7fc700 1 -- 172.21.15.162:0/750928169 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6048003230 con 0x7f60480044d0 2020-11-28T21:47:15.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.015+0000 7f6066ffd700 1 --2- 172.21.15.162:0/750928169 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6048003720 0x7f6048003b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:47:15.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.015+0000 7f605e7fc700 1 --2- 172.21.15.162:0/750928169 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60480044d0 0x7f6048004950 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f6050001420 tx=0x7f60680b4580).ready entity=mon.1 client_cookie=d26b399c317ce36b server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:15.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.015+0000 7f60667fc700 1 -- 172.21.15.162:0/750928169 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f60500070d0 con 0x7f60480044d0 2020-11-28T21:47:15.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.015+0000 7f60667fc700 1 -- 172.21.15.162:0/750928169 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6050007230 con 0x7f60480044d0 2020-11-28T21:47:15.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.015+0000 7f60667fc700 1 -- 172.21.15.162:0/750928169 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f60500055e0 con 0x7f60480044d0 2020-11-28T21:47:15.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.015+0000 7f6067fff700 1 -- 172.21.15.162:0/750928169 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60480044d0 msgr2=0x7f6048004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:15.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.015+0000 7f6067fff700 1 --2- 172.21.15.162:0/750928169 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60480044d0 0x7f6048004950 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f6050001420 tx=0x7f60680b4580).stop 2020-11-28T21:47:15.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.019+0000 7f6067fff700 1 -- 172.21.15.162:0/750928169 shutdown_connections 2020-11-28T21:47:15.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.019+0000 7f6067fff700 1 --2- 172.21.15.162:0/750928169 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60480044d0 0x7f6048004950 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:15.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.019+0000 7f6067fff700 1 -- 172.21.15.162:0/750928169 >> 172.21.15.162:0/750928169 conn(0x7f6068066a40 msgr2=0x7f6068066e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:15.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.019+0000 7f6067fff700 1 -- 172.21.15.162:0/750928169 shutdown_connections 2020-11-28T21:47:15.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.019+0000 7f6067fff700 1 -- 172.21.15.162:0/750928169 wait complete. 2020-11-28T21:47:15.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.019+0000 7f6067fff700 1 Processor -- start 2020-11-28T21:47:15.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.019+0000 7f6067fff700 1 -- start start 2020-11-28T21:47:15.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.019+0000 7f6067fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60480044d0 0x7f6068146d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:15.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.019+0000 7f6067fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f60681464b0 0x7f6068144d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:15.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.019+0000 7f6067fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6068145280 0x7f6068151510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:15.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.019+0000 7f6067fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60681483d0 con 0x7f60681464b0 2020-11-28T21:47:15.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.019+0000 7f6067fff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6068148650 con 0x7f60480044d0 2020-11-28T21:47:15.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.019+0000 7f6067fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6068148880 con 0x7f6068145280 2020-11-28T21:47:15.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.019+0000 7f6066ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60480044d0 0x7f6068146d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:15.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.019+0000 7f6066ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60480044d0 0x7f6068146d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38808/0 (socket says 172.21.15.162:38808) 2020-11-28T21:47:15.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.019+0000 7f6066ffd700 1 -- 172.21.15.162:0/3230023398 learned_addr learned my addr 172.21.15.162:0/3230023398 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:15.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.019+0000 7f60677fe700 1 --2- 172.21.15.162:0/3230023398 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6068145280 0x7f6068151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:15.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.019+0000 7f605e7fc700 1 --2- 172.21.15.162:0/3230023398 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f60681464b0 0x7f6068144d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:15.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.023+0000 7f6066ffd700 1 -- 172.21.15.162:0/3230023398 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6068145280 msgr2=0x7f6068151510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:15.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.023+0000 7f6066ffd700 1 --2- 172.21.15.162:0/3230023398 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6068145280 0x7f6068151510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:15.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.023+0000 7f6066ffd700 1 -- 172.21.15.162:0/3230023398 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f60681464b0 msgr2=0x7f6068144d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:15.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.023+0000 7f6066ffd700 1 --2- 172.21.15.162:0/3230023398 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f60681464b0 0x7f6068144d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:15.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.023+0000 7f6066ffd700 1 -- 172.21.15.162:0/3230023398 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60681479f0 con 0x7f60480044d0 2020-11-28T21:47:15.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.023+0000 7f6066ffd700 1 --2- 172.21.15.162:0/3230023398 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60480044d0 0x7f6068146d30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f6068145ea0 tx=0x7f6058009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:15.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.023+0000 7f6064ff9700 1 -- 172.21.15.162:0/3230023398 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f605800c9a0 con 0x7f60480044d0 2020-11-28T21:47:15.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.023+0000 7f6064ff9700 1 -- 172.21.15.162:0/3230023398 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f605800cb00 con 0x7f60480044d0 2020-11-28T21:47:15.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.023+0000 7f6067fff700 1 -- 172.21.15.162:0/3230023398 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6068147c90 con 0x7f60480044d0 2020-11-28T21:47:15.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.023+0000 7f6064ff9700 1 -- 172.21.15.162:0/3230023398 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f605800ce60 con 0x7f60480044d0 2020-11-28T21:47:15.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.023+0000 7f6067fff700 1 -- 172.21.15.162:0/3230023398 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6068147630 con 0x7f60480044d0 2020-11-28T21:47:15.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.023+0000 7f6064ff9700 1 -- 172.21.15.162:0/3230023398 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f605801b030 con 0x7f60480044d0 2020-11-28T21:47:15.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.023+0000 7f6067fff700 1 -- 172.21.15.162:0/3230023398 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6068146280 con 0x7f60480044d0 2020-11-28T21:47:15.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.023+0000 7f6064ff9700 1 --2- 172.21.15.162:0/3230023398 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f604c055440 0x7f604c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:15.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.023+0000 7f6064ff9700 1 -- 172.21.15.162:0/3230023398 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f605806ded0 con 0x7f60480044d0 2020-11-28T21:47:15.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.023+0000 7f605e7fc700 1 --2- 172.21.15.162:0/3230023398 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f604c055440 0x7f604c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:15.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.027+0000 7f605e7fc700 1 --2- 172.21.15.162:0/3230023398 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f604c055440 0x7f604c057900 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f6050019000 tx=0x7f6050013000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:15.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.031+0000 7f6064ff9700 1 -- 172.21.15.162:0/3230023398 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6058044280 con 0x7f60480044d0 2020-11-28T21:47:15.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.211+0000 7f6067fff700 1 -- 172.21.15.162:0/3230023398 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6068057050 con 0x7f60480044d0 2020-11-28T21:47:15.212 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.211+0000 7f6064ff9700 1 -- 172.21.15.162:0/3230023398 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+2497 (secure 0 0 0) 0x7f6058023090 con 0x7f60480044d0 2020-11-28T21:47:15.212 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:47:15.213 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":10,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4839,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/1153453343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1153453343},{"type":"v1","addr":"172.21.15.162:6836","nonce":1153453343}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8},{"gid":4849,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2101329586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":2101329586},{"type":"v1","addr":"172.21.15.162:6837","nonce":2101329586}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:47:10.330722+0000","modified":"2020-11-28T21:47:11.333518+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4855},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4855":{"gid":4855,"name":"b","rank":0,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.118:6833/3038031199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3038031199},{"type":"v1","addr":"172.21.15.118:6833","nonce":3038031199}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-28T21:47:15.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.211+0000 7f6067fff700 1 -- 172.21.15.162:0/3230023398 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f604c055440 msgr2=0x7f604c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:15.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.211+0000 7f6067fff700 1 --2- 172.21.15.162:0/3230023398 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f604c055440 0x7f604c057900 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f6050019000 tx=0x7f6050013000).stop 2020-11-28T21:47:15.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.211+0000 7f6067fff700 1 -- 172.21.15.162:0/3230023398 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60480044d0 msgr2=0x7f6068146d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:15.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.211+0000 7f6067fff700 1 --2- 172.21.15.162:0/3230023398 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60480044d0 0x7f6068146d30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f6068145ea0 tx=0x7f6058009040).stop 2020-11-28T21:47:15.216 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.211+0000 7f6067fff700 1 -- 172.21.15.162:0/3230023398 shutdown_connections 2020-11-28T21:47:15.216 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.211+0000 7f6067fff700 1 --2- 172.21.15.162:0/3230023398 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f604c055440 0x7f604c057900 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:15.216 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.211+0000 7f6067fff700 1 --2- 172.21.15.162:0/3230023398 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6068145280 0x7f6068151510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:15.216 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.211+0000 7f6067fff700 1 --2- 172.21.15.162:0/3230023398 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60480044d0 0x7f6068146d30 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:15.216 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.211+0000 7f6067fff700 1 --2- 172.21.15.162:0/3230023398 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f60681464b0 0x7f6068144d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:15.217 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.211+0000 7f6067fff700 1 -- 172.21.15.162:0/3230023398 >> 172.21.15.162:0/3230023398 conn(0x7f6068066a40 msgr2=0x7f606809cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:15.217 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.211+0000 7f6067fff700 1 -- 172.21.15.162:0/3230023398 shutdown_connections 2020-11-28T21:47:15.217 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:15.211+0000 7f6067fff700 1 -- 172.21.15.162:0/3230023398 wait complete. 2020-11-28T21:47:15.218 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 10 2020-11-28T21:47:15.528 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 10, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-28T21:47:10.330722+0000', 'modified': '2020-11-28T21:47:11.333518+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4855}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4855': {'gid': 4855, 'name': 'b', 'rank': 0, 'incarnation': 9, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.118:6833/3038031199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6832', 'nonce': 3038031199}, {'type': 'v1', 'addr': '172.21.15.118:6833', 'nonce': 3038031199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-28T21:47:15.529 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-28T21:47:15.530 INFO:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2020-11-28T21:47:15.658 INFO:teuthology.orchestra.run.smithi118.stdout:{"cluster_fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","whoami":0,"id":4855,"want_state":"up:active","state":"up:active","rank_uptime":5.3126055040000004,"mdsmap_epoch":10,"osdmap_epoch":33,"osdmap_epoch_barrier":32,"uptime":12.543342945999999} 2020-11-28T21:47:15.659 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", "id": 4855, "mdsmap_epoch": 10, "osdmap_epoch": 33, "osdmap_epoch_barrier": 32, "rank_uptime": 5.312605504, "state": "up:active", "uptime": 12.543342946, "want_state": "up:active", "whoami": 0 } 2020-11-28T21:47:15.659 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-11-28T21:47:15.660 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-11-28T21:47:15.660 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi192.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-11-28T21:47:15.661 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:47:15.661 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:47:15.675 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:47:15.733 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:47:15.734 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:47:15.791 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:47:15.792 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:47:15.793 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:47:15.794 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:47:15.838 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-11-28T21:47:15.839 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:47:15.888 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:47:15.906 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:47:15.907 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:47:15.964 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:2020-11-28T21:47:15.961+0000 7f8da98f80c0 -1 init, newargv = 0x55871e0add70 newargc=9 2020-11-28T21:47:15.964 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:ceph-fuse[11430]: starting ceph client 2020-11-28T21:47:15.965 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:47:15.966 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:47:15.966 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:47:15.967 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:47:15.974 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:ceph-fuse[11430]: starting fuse 2020-11-28T21:47:15.980 INFO:teuthology.orchestra.run.smithi192.stdout:49 2020-11-28T21:47:15.981 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2020-11-28T21:47:15.981 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:47:15.981 INFO:teuthology.orchestra.run.smithi192:> sudo python3 -c ' 2020-11-28T21:47:15.982 INFO:teuthology.orchestra.run.smithi192:> import glob 2020-11-28T21:47:15.982 INFO:teuthology.orchestra.run.smithi192:> import re 2020-11-28T21:47:15.983 INFO:teuthology.orchestra.run.smithi192:> import os 2020-11-28T21:47:15.983 INFO:teuthology.orchestra.run.smithi192:> import subprocess 2020-11-28T21:47:15.984 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:47:15.984 INFO:teuthology.orchestra.run.smithi192:> def find_socket(client_name): 2020-11-28T21:47:15.984 INFO:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-11-28T21:47:15.985 INFO:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2020-11-28T21:47:15.985 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:47:15.986 INFO:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2020-11-28T21:47:15.986 INFO:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2020-11-28T21:47:15.987 INFO:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2020-11-28T21:47:15.987 INFO:teuthology.orchestra.run.smithi192:> return files[0] 2020-11-28T21:47:15.987 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:47:15.988 INFO:teuthology.orchestra.run.smithi192:> for f in files: 2020-11-28T21:47:15.988 INFO:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-11-28T21:47:15.989 INFO:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2020-11-28T21:47:15.989 INFO:teuthology.orchestra.run.smithi192:> return f 2020-11-28T21:47:15.989 INFO:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-11-28T21:47:15.990 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:47:15.990 INFO:teuthology.orchestra.run.smithi192:> print(find_socket("client.0")) 2020-11-28T21:47:15.991 INFO:teuthology.orchestra.run.smithi192:> ' 2020-11-28T21:47:16.072 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.0.11430.asok 2020-11-28T21:47:16.077 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.11430.asok 2020-11-28T21:47:16.077 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:47:16.078 INFO:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.11430.asok status 2020-11-28T21:47:16.182 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2020-11-28T21:47:16.183 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2020-11-28T21:47:16.183 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "a83a3572a957356977342f461361c1cd3ec06dbd", 2020-11-28T21:47:16.183 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 15.2.6-595-ga83a3572 (a83a3572a957356977342f461361c1cd3ec06dbd) octopus (stable)", 2020-11-28T21:47:16.183 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "0", 2020-11-28T21:47:16.184 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2020-11-28T21:47:16.184 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-11-28T21:47:16.184 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "11430", 2020-11-28T21:47:16.184 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2020-11-28T21:47:16.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:47:16.185 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2020-11-28T21:47:16.185 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2020-11-28T21:47:16.185 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 5019, 2020-11-28T21:47:16.185 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2020-11-28T21:47:16.186 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2020-11-28T21:47:16.186 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2020-11-28T21:47:16.186 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 5019 2020-11-28T21:47:16.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:47:16.187 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2020-11-28T21:47:16.187 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2020-11-28T21:47:16.187 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2020-11-28T21:47:16.187 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 3415820922 2020-11-28T21:47:16.187 INFO:teuthology.orchestra.run.smithi192.stdout: } 2020-11-28T21:47:16.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:47:16.188 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2020-11-28T21:47:16.188 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2020-11-28T21:47:16.188 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2020-11-28T21:47:16.188 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 3415820922 2020-11-28T21:47:16.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:47:16.189 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.5019 172.21.15.192:0/3415820922", 2020-11-28T21:47:16.189 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "172.21.15.192:0/3415820922", 2020-11-28T21:47:16.189 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2020-11-28T21:47:16.190 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 10, 2020-11-28T21:47:16.190 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 33, 2020-11-28T21:47:16.190 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2020-11-28T21:47:16.190 INFO:teuthology.orchestra.run.smithi192.stdout: "blacklisted": false 2020-11-28T21:47:16.191 INFO:teuthology.orchestra.run.smithi192.stdout:} 2020-11-28T21:47:16.192 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:47:16.193 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:47:16.193 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:47:16.200 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2020-11-28T21:47:16.201 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-28T21:47:16.201 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:47:16.202 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:47:16.262 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_min_caps_per_client 2020-11-28T21:47:16.381 INFO:teuthology.orchestra.run.smithi162.stdout:{"mds_min_caps_per_client":"100"} 2020-11-28T21:47:16.382 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_min_caps_per_client": "100" } 2020-11-28T21:47:16.382 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:47:16.383 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:47:16.383 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:47:16.391 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2020-11-28T21:47:16.392 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-28T21:47:16.392 INFO:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-28T21:47:16.393 INFO:teuthology.orchestra.run.smithi192:> import sys 2020-11-28T21:47:16.393 INFO:teuthology.orchestra.run.smithi192:> import time 2020-11-28T21:47:16.394 INFO:teuthology.orchestra.run.smithi192:> import os 2020-11-28T21:47:16.394 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:47:16.394 INFO:teuthology.orchestra.run.smithi192:> n = 1000 2020-11-28T21:47:16.395 INFO:teuthology.orchestra.run.smithi192:> abs_path = "/home/ubuntu/cephtest/mnt.0/dc-dir/dc-file" 2020-11-28T21:47:16.395 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:47:16.395 INFO:teuthology.orchestra.run.smithi192:> if not os.path.exists(os.path.dirname(abs_path)): 2020-11-28T21:47:16.396 INFO:teuthology.orchestra.run.smithi192:> os.makedirs(os.path.dirname(abs_path)) 2020-11-28T21:47:16.396 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:47:16.397 INFO:teuthology.orchestra.run.smithi192:> for i in range(0, n): 2020-11-28T21:47:16.397 INFO:teuthology.orchestra.run.smithi192:> fname = "{0}_{1}".format(abs_path, i) 2020-11-28T21:47:16.397 INFO:teuthology.orchestra.run.smithi192:> with open(fname, '"'"'w'"'"') as f: 2020-11-28T21:47:16.398 INFO:teuthology.orchestra.run.smithi192:> f.write('"'"'content'"'"') 2020-11-28T21:47:16.398 INFO:teuthology.orchestra.run.smithi192:> if True: 2020-11-28T21:47:16.398 INFO:teuthology.orchestra.run.smithi192:> f.flush() 2020-11-28T21:47:16.399 INFO:teuthology.orchestra.run.smithi192:> os.fsync(f.fileno()) 2020-11-28T21:47:16.399 INFO:teuthology.orchestra.run.smithi192:> ' 2020-11-28T21:47:28.720 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:47:28.839 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.839+0000 7fe2c5be5700 1 Processor -- start 2020-11-28T21:47:28.839 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.839+0000 7fe2c5be5700 1 -- start start 2020-11-28T21:47:28.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.839+0000 7fe2c5be5700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe2c00b1990 0x7fe2c0140890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:28.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.839+0000 7fe2c5be5700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe2c00b11b0 0x7fe2c0140d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:28.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.839+0000 7fe2c5be5700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2c00b2870 con 0x7fe2c00b1d70 2020-11-28T21:47:28.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.839+0000 7fe2c5be5700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2c0146e20 con 0x7fe2c00b11b0 2020-11-28T21:47:28.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.839+0000 7fe2c5be5700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2c00b22e0 con 0x7fe2c00b1990 2020-11-28T21:47:28.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.839+0000 7fe2bf7fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe2c00b1990 0x7fe2c0140890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:28.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.839+0000 7fe2bf7fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe2c00b1990 0x7fe2c0140890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38822/0 (socket says 172.21.15.162:38822) 2020-11-28T21:47:28.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.839+0000 7fe2beffd700 1 --2- 172.21.15.162:0/2244156869 >> v2:172.21.15.118:3300/0 conn(0x7fe2c00b11b0 0x7fe2c0140d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:28.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.839+0000 7fe2bf7fe700 1 -- 172.21.15.162:0/2244156869 learned_addr learned my addr 172.21.15.162:0/2244156869 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:28.843 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.839+0000 7fe2bf7fe700 1 -- 172.21.15.162:0/2244156869 >> v2:172.21.15.118:3300/0 conn(0x7fe2c00b11b0 msgr2=0x7fe2c0140d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:28.843 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.839+0000 7fe2bf7fe700 1 --2- 172.21.15.162:0/2244156869 >> v2:172.21.15.118:3300/0 conn(0x7fe2c00b11b0 0x7fe2c0140d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:28.843 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.839+0000 7fe2bf7fe700 1 -- 172.21.15.162:0/2244156869 >> v1:172.21.15.118:6789/0 conn(0x7fe2c00b1d70 legacy=0x7fe2c014a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:28.843 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.839+0000 7fe2bf7fe700 1 -- 172.21.15.162:0/2244156869 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c00b0740 con 0x7fe2c00b1990 2020-11-28T21:47:28.844 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.839+0000 7fe2bf7fe700 1 --2- 172.21.15.162:0/2244156869 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2c00b1990 0x7fe2c0140890 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7fe2a8002f30 tx=0x7fe2a8006e80).ready entity=mon.1 client_cookie=7c24195bdb6b302d server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:28.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.839+0000 7fe2be7fc700 1 -- 172.21.15.162:0/2244156869 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2a80097f0 con 0x7fe2c00b1990 2020-11-28T21:47:28.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.839+0000 7fe2be7fc700 1 -- 172.21.15.162:0/2244156869 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a8009950 con 0x7fe2c00b1990 2020-11-28T21:47:28.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.839+0000 7fe2c5be5700 1 -- 172.21.15.162:0/2244156869 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2c00b1990 msgr2=0x7fe2c0140890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:28.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.839+0000 7fe2be7fc700 1 -- 172.21.15.162:0/2244156869 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2a8009cb0 con 0x7fe2c00b1990 2020-11-28T21:47:28.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.839+0000 7fe2c5be5700 1 --2- 172.21.15.162:0/2244156869 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2c00b1990 0x7fe2c0140890 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7fe2a8002f30 tx=0x7fe2a8006e80).stop 2020-11-28T21:47:28.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.839+0000 7fe2c5be5700 1 -- 172.21.15.162:0/2244156869 shutdown_connections 2020-11-28T21:47:28.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.839+0000 7fe2c5be5700 1 --2- 172.21.15.162:0/2244156869 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2c00b1990 0x7fe2c0140890 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:28.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.839+0000 7fe2c5be5700 1 --2- 172.21.15.162:0/2244156869 >> v2:172.21.15.118:3300/0 conn(0x7fe2c00b11b0 0x7fe2c0140d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:28.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.839+0000 7fe2c5be5700 1 -- 172.21.15.162:0/2244156869 >> 172.21.15.162:0/2244156869 conn(0x7fe2c005b970 msgr2=0x7fe2c005bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:28.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.839+0000 7fe2c5be5700 1 -- 172.21.15.162:0/2244156869 shutdown_connections 2020-11-28T21:47:28.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.839+0000 7fe2c5be5700 1 -- 172.21.15.162:0/2244156869 wait complete. 2020-11-28T21:47:28.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.843+0000 7fe2c5be5700 1 Processor -- start 2020-11-28T21:47:28.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.843+0000 7fe2c5be5700 1 -- start start 2020-11-28T21:47:28.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.843+0000 7fe2c5be5700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2c00b11b0 0x7fe2c0146180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:28.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.843+0000 7fe2c5be5700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe2c00b1990 0x7fe2c01564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:28.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.843+0000 7fe2c5be5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2c00b1d70 0x7fe2c01569d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:28.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.843+0000 7fe2c5be5700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c01425d0 con 0x7fe2c00b11b0 2020-11-28T21:47:28.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.843+0000 7fe2c5be5700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c0141970 con 0x7fe2c00b1d70 2020-11-28T21:47:28.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.843+0000 7fe2c5be5700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c00b38e0 con 0x7fe2c00b1990 2020-11-28T21:47:28.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.843+0000 7fe2bffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2c00b1d70 0x7fe2c01569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:28.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.843+0000 7fe2beffd700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe2c00b1990 0x7fe2c01564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:28.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.843+0000 7fe2beffd700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe2c00b1990 0x7fe2c01564e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:36052/0 (socket says 172.21.15.162:36052) 2020-11-28T21:47:28.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.843+0000 7fe2beffd700 1 -- 172.21.15.162:0/1696782696 learned_addr learned my addr 172.21.15.162:0/1696782696 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:28.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.843+0000 7fe2bffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2c00b1d70 0x7fe2c01569d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38830/0 (socket says 172.21.15.162:38830) 2020-11-28T21:47:28.852 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.843+0000 7fe2bf7fe700 1 --2- 172.21.15.162:0/1696782696 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2c00b11b0 0x7fe2c0146180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:28.852 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.843+0000 7fe2bffff700 1 -- 172.21.15.162:0/1696782696 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe2c00b1990 msgr2=0x7fe2c01564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:28.852 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.843+0000 7fe2bffff700 1 --2- 172.21.15.162:0/1696782696 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe2c00b1990 0x7fe2c01564e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:28.852 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.843+0000 7fe2bffff700 1 -- 172.21.15.162:0/1696782696 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2c00b11b0 msgr2=0x7fe2c0146180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:28.852 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.843+0000 7fe2bffff700 1 --2- 172.21.15.162:0/1696782696 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2c00b11b0 0x7fe2c0146180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:28.853 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.843+0000 7fe2bffff700 1 -- 172.21.15.162:0/1696782696 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c0155e00 con 0x7fe2c00b1d70 2020-11-28T21:47:28.853 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.843+0000 7fe2bffff700 1 --2- 172.21.15.162:0/1696782696 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2c00b1d70 0x7fe2c01569d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7fe2c01415f0 tx=0x7fe2b8007ee0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:28.853 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.847+0000 7fe2bcff9700 1 -- 172.21.15.162:0/1696782696 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2b800e4a0 con 0x7fe2c00b1d70 2020-11-28T21:47:28.853 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.847+0000 7fe2bcff9700 1 -- 172.21.15.162:0/1696782696 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b800e600 con 0x7fe2c00b1d70 2020-11-28T21:47:28.853 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.847+0000 7fe2c5be5700 1 -- 172.21.15.162:0/1696782696 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2c00b3640 con 0x7fe2c00b1d70 2020-11-28T21:47:28.854 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.847+0000 7fe2bcff9700 1 -- 172.21.15.162:0/1696782696 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2b800e960 con 0x7fe2c00b1d70 2020-11-28T21:47:28.854 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.847+0000 7fe2c5be5700 1 -- 172.21.15.162:0/1696782696 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2c00b3270 con 0x7fe2c00b1d70 2020-11-28T21:47:28.854 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.847+0000 7fe2b4ff9700 1 -- 172.21.15.162:0/1696782696 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe29c0033e0 con 0x7fe2c00b1d70 2020-11-28T21:47:28.855 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.847+0000 7fe2bcff9700 1 -- 172.21.15.162:0/1696782696 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe2b8019030 con 0x7fe2c00b1d70 2020-11-28T21:47:28.855 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.847+0000 7fe2bcff9700 1 --2- 172.21.15.162:0/1696782696 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe2a40554e0 0x7fe2a40579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:28.855 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.847+0000 7fe2bcff9700 1 -- 172.21.15.162:0/1696782696 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe2b806d400 con 0x7fe2c00b1d70 2020-11-28T21:47:28.856 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.847+0000 7fe2bf7fe700 1 --2- 172.21.15.162:0/1696782696 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe2a40554e0 0x7fe2a40579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:28.856 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.851+0000 7fe2bf7fe700 1 --2- 172.21.15.162:0/1696782696 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe2a40554e0 0x7fe2a40579a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fe2a801c000 tx=0x7fe2a8015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:28.858 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:28.855+0000 7fe2bcff9700 1 -- 172.21.15.162:0/1696782696 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe2b8043700 con 0x7fe2c00b1d70 2020-11-28T21:47:29.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.043+0000 7fe2b4ff9700 1 -- 172.21.15.162:0/1696782696 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe29c003850 con 0x7fe2c00b1d70 2020-11-28T21:47:29.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.043+0000 7fe2bcff9700 1 -- 172.21.15.162:0/1696782696 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+2497 (secure 0 0 0) 0x7fe2b8046d20 con 0x7fe2c00b1d70 2020-11-28T21:47:29.046 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:47:29.046 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":10,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4839,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/1153453343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1153453343},{"type":"v1","addr":"172.21.15.162:6836","nonce":1153453343}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8},{"gid":4849,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2101329586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":2101329586},{"type":"v1","addr":"172.21.15.162:6837","nonce":2101329586}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:47:10.330722+0000","modified":"2020-11-28T21:47:11.333518+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4855},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4855":{"gid":4855,"name":"b","rank":0,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.118:6833/3038031199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3038031199},{"type":"v1","addr":"172.21.15.118:6833","nonce":3038031199}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-28T21:47:29.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.047+0000 7fe2b4ff9700 1 -- 172.21.15.162:0/1696782696 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe2a40554e0 msgr2=0x7fe2a40579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:29.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.047+0000 7fe2b4ff9700 1 --2- 172.21.15.162:0/1696782696 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe2a40554e0 0x7fe2a40579a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fe2a801c000 tx=0x7fe2a8015000).stop 2020-11-28T21:47:29.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.047+0000 7fe2b4ff9700 1 -- 172.21.15.162:0/1696782696 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2c00b1d70 msgr2=0x7fe2c01569d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:29.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.047+0000 7fe2b4ff9700 1 --2- 172.21.15.162:0/1696782696 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2c00b1d70 0x7fe2c01569d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7fe2c01415f0 tx=0x7fe2b8007ee0).stop 2020-11-28T21:47:29.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.047+0000 7fe2b4ff9700 1 -- 172.21.15.162:0/1696782696 shutdown_connections 2020-11-28T21:47:29.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.047+0000 7fe2b4ff9700 1 --2- 172.21.15.162:0/1696782696 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe2a40554e0 0x7fe2a40579a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:29.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.047+0000 7fe2b4ff9700 1 --2- 172.21.15.162:0/1696782696 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2c00b1d70 0x7fe2c01569d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:29.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.047+0000 7fe2b4ff9700 1 --2- 172.21.15.162:0/1696782696 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2c00b11b0 0x7fe2c0146180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:29.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.047+0000 7fe2b4ff9700 1 --2- 172.21.15.162:0/1696782696 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe2c00b1990 0x7fe2c01564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:29.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.047+0000 7fe2b4ff9700 1 -- 172.21.15.162:0/1696782696 >> 172.21.15.162:0/1696782696 conn(0x7fe2c005b970 msgr2=0x7fe2c0148220 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:29.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.047+0000 7fe2b4ff9700 1 -- 172.21.15.162:0/1696782696 shutdown_connections 2020-11-28T21:47:29.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.047+0000 7fe2b4ff9700 1 -- 172.21.15.162:0/1696782696 wait complete. 2020-11-28T21:47:29.051 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 10 2020-11-28T21:47:29.059 INFO:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json config set mds_recall_max_caps 20 2020-11-28T21:47:29.179 INFO:teuthology.orchestra.run.smithi118.stdout:{"success":"mds_recall_max_caps = '20' (not observed, change may require restart) "} 2020-11-28T21:47:29.179 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_caps = '20' (not observed, change may require restart) " } 2020-11-28T21:47:29.180 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:47:29.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.295+0000 7fcca759e700 1 Processor -- start 2020-11-28T21:47:29.298 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.295+0000 7fcca759e700 1 -- start start 2020-11-28T21:47:29.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.295+0000 7fcca759e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcca80a9bc0 0x7fcca80a9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:29.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.295+0000 7fcca759e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcca80b1fd0 0x7fcca80b0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:29.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.295+0000 7fcca759e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcca80b0f30 con 0x7fcca80aa490 2020-11-28T21:47:29.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.295+0000 7fcca759e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcca8061030 con 0x7fcca80b1fd0 2020-11-28T21:47:29.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.295+0000 7fcca759e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcca80af980 con 0x7fcca80a9bc0 2020-11-28T21:47:29.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.295+0000 7fcca6d9d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcca80b1fd0 0x7fcca80b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:29.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.295+0000 7fcca659c700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcca80a9bc0 0x7fcca80a9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:29.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.295+0000 7fcca6d9d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcca80b1fd0 0x7fcca80b0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38838/0 (socket says 172.21.15.162:38838) 2020-11-28T21:47:29.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.295+0000 7fcca6d9d700 1 -- 172.21.15.162:0/3000926917 learned_addr learned my addr 172.21.15.162:0/3000926917 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:29.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.299+0000 7fcca559a700 1 -- 172.21.15.162:0/3000926917 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1263722129 0 0) 0x7fcca80b0f30 con 0x7fcca80aa490 2020-11-28T21:47:29.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.299+0000 7fcca559a700 1 -- 172.21.15.162:0/3000926917 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc8c003230 con 0x7fcca80aa490 2020-11-28T21:47:29.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.299+0000 7fcca659c700 1 -- 172.21.15.162:0/3000926917 >> v2:172.21.15.162:3300/0 conn(0x7fcca80b1fd0 msgr2=0x7fcca80b0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:29.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.299+0000 7fcca659c700 1 --2- 172.21.15.162:0/3000926917 >> v2:172.21.15.162:3300/0 conn(0x7fcca80b1fd0 0x7fcca80b0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:29.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.299+0000 7fcca659c700 1 -- 172.21.15.162:0/3000926917 >> v1:172.21.15.118:6789/0 conn(0x7fcca80aa490 legacy=0x7fcca80afb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:29.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.299+0000 7fcca659c700 1 -- 172.21.15.162:0/3000926917 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca80af6e0 con 0x7fcca80a9bc0 2020-11-28T21:47:29.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.299+0000 7fcca6d9d700 1 --2- 172.21.15.162:0/3000926917 >> v2:172.21.15.162:3300/0 conn(0x7fcca80b1fd0 0x7fcca80b0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:47:29.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.299+0000 7fcca659c700 1 --2- 172.21.15.162:0/3000926917 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcca80a9bc0 0x7fcca80a9fa0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fcc98002f30 tx=0x7fcc98006dc0).ready entity=mon.2 client_cookie=90ddab1c4ea56be1 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:29.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.299+0000 7fcca559a700 1 -- 172.21.15.162:0/3000926917 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc980096a0 con 0x7fcca80a9bc0 2020-11-28T21:47:29.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.299+0000 7fcca559a700 1 -- 172.21.15.162:0/3000926917 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc98009800 con 0x7fcca80a9bc0 2020-11-28T21:47:29.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.299+0000 7fcca559a700 1 -- 172.21.15.162:0/3000926917 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc98009b60 con 0x7fcca80a9bc0 2020-11-28T21:47:29.306 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.299+0000 7fcca759e700 1 -- 172.21.15.162:0/3000926917 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcca80a9bc0 msgr2=0x7fcca80a9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:29.306 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.299+0000 7fcca759e700 1 --2- 172.21.15.162:0/3000926917 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcca80a9bc0 0x7fcca80a9fa0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fcc98002f30 tx=0x7fcc98006dc0).stop 2020-11-28T21:47:29.306 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.299+0000 7fcca759e700 1 -- 172.21.15.162:0/3000926917 shutdown_connections 2020-11-28T21:47:29.306 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.299+0000 7fcca759e700 1 --2- 172.21.15.162:0/3000926917 >> v2:172.21.15.162:3300/0 conn(0x7fcca80b1fd0 0x7fcca80b0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:29.306 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.299+0000 7fcca759e700 1 --2- 172.21.15.162:0/3000926917 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcca80a9bc0 0x7fcca80a9fa0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:29.307 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.299+0000 7fcca759e700 1 -- 172.21.15.162:0/3000926917 >> 172.21.15.162:0/3000926917 conn(0x7fcca805b970 msgr2=0x7fcca805bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:29.307 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.299+0000 7fcca759e700 1 -- 172.21.15.162:0/3000926917 shutdown_connections 2020-11-28T21:47:29.307 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.299+0000 7fcca759e700 1 -- 172.21.15.162:0/3000926917 wait complete. 2020-11-28T21:47:29.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.299+0000 7fcca759e700 1 Processor -- start 2020-11-28T21:47:29.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.299+0000 7fcca759e700 1 -- start start 2020-11-28T21:47:29.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.303+0000 7fcca759e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcca80a9bc0 0x7fcca80ae2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:29.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.303+0000 7fcca759e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcca80aa490 0x7fcca8142f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:29.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.303+0000 7fcca759e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcca80b1fd0 0x7fcca8154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:29.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.303+0000 7fcca759e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca8144040 con 0x7fcca80aa490 2020-11-28T21:47:29.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.303+0000 7fcca759e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca8144270 con 0x7fcca80a9bc0 2020-11-28T21:47:29.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.303+0000 7fcca759e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcca8144510 con 0x7fcca80b1fd0 2020-11-28T21:47:29.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.303+0000 7fcca659c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcca80a9bc0 0x7fcca80ae2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:29.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.303+0000 7fcca6d9d700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcca80b1fd0 0x7fcca8154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:29.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.303+0000 7fcca659c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcca80a9bc0 0x7fcca80ae2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38840/0 (socket says 172.21.15.162:38840) 2020-11-28T21:47:29.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.303+0000 7fcca6d9d700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcca80b1fd0 0x7fcca8154bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:36068/0 (socket says 172.21.15.162:36068) 2020-11-28T21:47:29.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.303+0000 7fcca659c700 1 -- 172.21.15.162:0/1009104302 learned_addr learned my addr 172.21.15.162:0/1009104302 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:29.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.303+0000 7fcca5d9b700 1 --2- 172.21.15.162:0/1009104302 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcca80aa490 0x7fcca8142f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:29.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.303+0000 7fcca6d9d700 1 -- 172.21.15.162:0/1009104302 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcca80a9bc0 msgr2=0x7fcca80ae2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:29.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.303+0000 7fcca6d9d700 1 --2- 172.21.15.162:0/1009104302 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcca80a9bc0 0x7fcca80ae2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:29.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.303+0000 7fcca6d9d700 1 -- 172.21.15.162:0/1009104302 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcca80aa490 msgr2=0x7fcca8142f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:29.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.303+0000 7fcca6d9d700 1 --2- 172.21.15.162:0/1009104302 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcca80aa490 0x7fcca8142f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:29.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.303+0000 7fcca6d9d700 1 -- 172.21.15.162:0/1009104302 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca8142850 con 0x7fcca80b1fd0 2020-11-28T21:47:29.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.303+0000 7fcca6d9d700 1 --2- 172.21.15.162:0/1009104302 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcca80b1fd0 0x7fcca8154bb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fcca000c000 tx=0x7fcca0009270).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:29.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.303+0000 7fcc977fe700 1 -- 172.21.15.162:0/1009104302 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcca000cbd0 con 0x7fcca80b1fd0 2020-11-28T21:47:29.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.303+0000 7fcc977fe700 1 -- 172.21.15.162:0/1009104302 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcca000cd30 con 0x7fcca80b1fd0 2020-11-28T21:47:29.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.303+0000 7fcca759e700 1 -- 172.21.15.162:0/1009104302 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcca8142af0 con 0x7fcca80b1fd0 2020-11-28T21:47:29.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.303+0000 7fcc977fe700 1 -- 172.21.15.162:0/1009104302 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcca000cea0 con 0x7fcca80b1fd0 2020-11-28T21:47:29.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.303+0000 7fcca759e700 1 -- 172.21.15.162:0/1009104302 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcca8066e80 con 0x7fcca80b1fd0 2020-11-28T21:47:29.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.303+0000 7fcca759e700 1 -- 172.21.15.162:0/1009104302 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcca8057050 con 0x7fcca80b1fd0 2020-11-28T21:47:29.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.303+0000 7fcc977fe700 1 -- 172.21.15.162:0/1009104302 <== mon.2 v2:172.21.15.118:3301/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fcca00125e0 con 0x7fcca80b1fd0 2020-11-28T21:47:29.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.303+0000 7fcc977fe700 1 --2- 172.21.15.162:0/1009104302 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcc84055440 0x7fcc84057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:29.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.307+0000 7fcc977fe700 1 -- 172.21.15.162:0/1009104302 <== mon.2 v2:172.21.15.118:3301/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fcca006e300 con 0x7fcca80b1fd0 2020-11-28T21:47:29.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.307+0000 7fcca659c700 1 --2- 172.21.15.162:0/1009104302 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcc84055440 0x7fcc84057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:29.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.307+0000 7fcca659c700 1 --2- 172.21.15.162:0/1009104302 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcc84055440 0x7fcc84057900 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7fcc9801b000 tx=0x7fcc98015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:29.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.315+0000 7fcc977fe700 1 -- 172.21.15.162:0/1009104302 <== mon.2 v2:172.21.15.118:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fcca00445a0 con 0x7fcca80b1fd0 2020-11-28T21:47:29.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.499+0000 7fcca759e700 1 -- 172.21.15.162:0/1009104302 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcca8142c80 con 0x7fcca80b1fd0 2020-11-28T21:47:29.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.503+0000 7fcc977fe700 1 -- 172.21.15.162:0/1009104302 <== mon.2 v2:172.21.15.118:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+2497 (secure 0 0 0) 0x7fcca0016020 con 0x7fcca80b1fd0 2020-11-28T21:47:29.505 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:47:29.505 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":10,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4839,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/1153453343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1153453343},{"type":"v1","addr":"172.21.15.162:6836","nonce":1153453343}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8},{"gid":4849,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2101329586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":2101329586},{"type":"v1","addr":"172.21.15.162:6837","nonce":2101329586}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:47:10.330722+0000","modified":"2020-11-28T21:47:11.333518+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4855},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4855":{"gid":4855,"name":"b","rank":0,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.118:6833/3038031199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3038031199},{"type":"v1","addr":"172.21.15.118:6833","nonce":3038031199}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-28T21:47:29.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.503+0000 7fcca759e700 1 -- 172.21.15.162:0/1009104302 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcc84055440 msgr2=0x7fcc84057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:29.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.503+0000 7fcca759e700 1 --2- 172.21.15.162:0/1009104302 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcc84055440 0x7fcc84057900 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7fcc9801b000 tx=0x7fcc98015000).stop 2020-11-28T21:47:29.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.503+0000 7fcca759e700 1 -- 172.21.15.162:0/1009104302 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcca80b1fd0 msgr2=0x7fcca8154bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:29.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.503+0000 7fcca759e700 1 --2- 172.21.15.162:0/1009104302 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcca80b1fd0 0x7fcca8154bb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fcca000c000 tx=0x7fcca0009270).stop 2020-11-28T21:47:29.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.503+0000 7fcca759e700 1 -- 172.21.15.162:0/1009104302 shutdown_connections 2020-11-28T21:47:29.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.503+0000 7fcca759e700 1 --2- 172.21.15.162:0/1009104302 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcc84055440 0x7fcc84057900 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:29.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.503+0000 7fcca759e700 1 --2- 172.21.15.162:0/1009104302 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcca80b1fd0 0x7fcca8154bb0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:29.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.503+0000 7fcca759e700 1 --2- 172.21.15.162:0/1009104302 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcca80a9bc0 0x7fcca80ae2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:29.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.503+0000 7fcca759e700 1 --2- 172.21.15.162:0/1009104302 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcca80aa490 0x7fcca8142f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:29.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.503+0000 7fcca759e700 1 -- 172.21.15.162:0/1009104302 >> 172.21.15.162:0/1009104302 conn(0x7fcca805b970 msgr2=0x7fcca80b2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:29.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.503+0000 7fcca759e700 1 -- 172.21.15.162:0/1009104302 shutdown_connections 2020-11-28T21:47:29.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.503+0000 7fcca759e700 1 -- 172.21.15.162:0/1009104302 wait complete. 2020-11-28T21:47:29.509 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 10 2020-11-28T21:47:29.519 INFO:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json config set mds_recall_max_decay_threshold 400 2020-11-28T21:47:29.644 INFO:teuthology.orchestra.run.smithi118.stdout:{"success":"mds_recall_max_decay_threshold = '400' (not observed, change may require restart) "} 2020-11-28T21:47:29.645 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_decay_threshold = '400' (not observed, change may require restart) " } 2020-11-28T21:47:29.645 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:47:29.766 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.763+0000 7fd15ee6b700 1 Processor -- start 2020-11-28T21:47:29.766 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.763+0000 7fd15ee6b700 1 -- start start 2020-11-28T21:47:29.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.763+0000 7fd15ee6b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd158098b30 0x7fd1580994f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:29.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.763+0000 7fd15ee6b700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd1580b0000 0x7fd158060000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:29.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.763+0000 7fd15ee6b700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd1580604f0 0x7fd158060980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:29.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.763+0000 7fd15ee6b700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1580b1030 con 0x7fd1580b0000 2020-11-28T21:47:29.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.763+0000 7fd15ee6b700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1580b0480 con 0x7fd158098b30 2020-11-28T21:47:29.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.763+0000 7fd15ee6b700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd158098f10 con 0x7fd1580604f0 2020-11-28T21:47:29.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.767+0000 7fd15de69700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd158098b30 0x7fd1580994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:29.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.767+0000 7fd15de69700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd158098b30 0x7fd1580994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38850/0 (socket says 172.21.15.162:38850) 2020-11-28T21:47:29.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.767+0000 7fd15e66a700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd1580604f0 0x7fd158060980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:29.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.767+0000 7fd15de69700 1 -- 172.21.15.162:0/2591719277 learned_addr learned my addr 172.21.15.162:0/2591719277 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:29.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.767+0000 7fd15d668700 1 --2- 172.21.15.162:0/2591719277 >> v2:172.21.15.118:3300/0 conn(0x7fd1580b0000 0x7fd158060000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:29.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.767+0000 7fd15de69700 1 -- 172.21.15.162:0/2591719277 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd1580604f0 msgr2=0x7fd158060980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:29.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.767+0000 7fd15de69700 1 --2- 172.21.15.162:0/2591719277 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd1580604f0 0x7fd158060980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:29.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.767+0000 7fd15de69700 1 -- 172.21.15.162:0/2591719277 >> v2:172.21.15.118:3300/0 conn(0x7fd1580b0000 msgr2=0x7fd158060000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:29.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.767+0000 7fd15de69700 1 --2- 172.21.15.162:0/2591719277 >> v2:172.21.15.118:3300/0 conn(0x7fd1580b0000 0x7fd158060000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:29.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.767+0000 7fd15de69700 1 -- 172.21.15.162:0/2591719277 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1580b0680 con 0x7fd158098b30 2020-11-28T21:47:29.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.767+0000 7fd15de69700 1 --2- 172.21.15.162:0/2591719277 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd158098b30 0x7fd1580994f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7fd14c002f30 tx=0x7fd14c006e80).ready entity=mon.1 client_cookie=1a5af9342bd3cd5 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:29.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.767+0000 7fd15ce67700 1 -- 172.21.15.162:0/2591719277 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd14c009640 con 0x7fd158098b30 2020-11-28T21:47:29.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.767+0000 7fd15ce67700 1 -- 172.21.15.162:0/2591719277 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd14c0097a0 con 0x7fd158098b30 2020-11-28T21:47:29.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.767+0000 7fd15ce67700 1 -- 172.21.15.162:0/2591719277 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd14c009b00 con 0x7fd158098b30 2020-11-28T21:47:29.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.767+0000 7fd15ee6b700 1 -- 172.21.15.162:0/2591719277 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd158098b30 msgr2=0x7fd1580994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:29.776 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.767+0000 7fd15ee6b700 1 --2- 172.21.15.162:0/2591719277 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd158098b30 0x7fd1580994f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7fd14c002f30 tx=0x7fd14c006e80).stop 2020-11-28T21:47:29.776 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.767+0000 7fd15ee6b700 1 -- 172.21.15.162:0/2591719277 shutdown_connections 2020-11-28T21:47:29.776 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.767+0000 7fd15ee6b700 1 --2- 172.21.15.162:0/2591719277 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd1580604f0 0x7fd158060980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:29.776 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.767+0000 7fd15ee6b700 1 --2- 172.21.15.162:0/2591719277 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd158098b30 0x7fd1580994f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:29.776 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.767+0000 7fd15ee6b700 1 --2- 172.21.15.162:0/2591719277 >> v2:172.21.15.118:3300/0 conn(0x7fd1580b0000 0x7fd158060000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:29.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.767+0000 7fd15ee6b700 1 -- 172.21.15.162:0/2591719277 >> 172.21.15.162:0/2591719277 conn(0x7fd15805b970 msgr2=0x7fd15805bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:29.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.767+0000 7fd15ee6b700 1 -- 172.21.15.162:0/2591719277 shutdown_connections 2020-11-28T21:47:29.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.767+0000 7fd15ee6b700 1 -- 172.21.15.162:0/2591719277 wait complete. 2020-11-28T21:47:29.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.767+0000 7fd15ee6b700 1 Processor -- start 2020-11-28T21:47:29.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.771+0000 7fd15ee6b700 1 -- start start 2020-11-28T21:47:29.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.771+0000 7fd15ee6b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd1580604f0 0x7fd158065120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:29.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.771+0000 7fd15ee6b700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd158098b30 0x7fd158155ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:29.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.771+0000 7fd15ee6b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd1580b0000 0x7fd158155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:29.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.771+0000 7fd15ee6b700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1580657a0 con 0x7fd1580b0000 2020-11-28T21:47:29.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.771+0000 7fd15ee6b700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd158067240 con 0x7fd1580604f0 2020-11-28T21:47:29.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.771+0000 7fd15ee6b700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd158064bb0 con 0x7fd158098b30 2020-11-28T21:47:29.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.771+0000 7fd15de69700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd1580604f0 0x7fd158065120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:29.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.771+0000 7fd15de69700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd1580604f0 0x7fd158065120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38854/0 (socket says 172.21.15.162:38854) 2020-11-28T21:47:29.780 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.771+0000 7fd15de69700 1 -- 172.21.15.162:0/101847234 learned_addr learned my addr 172.21.15.162:0/101847234 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:29.780 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.771+0000 7fd15d668700 1 --2- 172.21.15.162:0/101847234 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd158098b30 0x7fd158155ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:29.780 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.771+0000 7fd15e66a700 1 --2- 172.21.15.162:0/101847234 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd1580b0000 0x7fd158155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:29.780 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.771+0000 7fd15de69700 1 -- 172.21.15.162:0/101847234 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd158098b30 msgr2=0x7fd158155ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:29.781 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.771+0000 7fd15de69700 1 --2- 172.21.15.162:0/101847234 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd158098b30 0x7fd158155ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:29.781 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.771+0000 7fd15de69700 1 -- 172.21.15.162:0/101847234 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd1580b0000 msgr2=0x7fd158155fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:29.781 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.771+0000 7fd15de69700 1 --2- 172.21.15.162:0/101847234 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd1580b0000 0x7fd158155fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:29.781 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.771+0000 7fd15de69700 1 -- 172.21.15.162:0/101847234 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd158065a00 con 0x7fd1580604f0 2020-11-28T21:47:29.782 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.771+0000 7fd15e66a700 1 --2- 172.21.15.162:0/101847234 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd1580b0000 0x7fd158155fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:47:29.782 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.771+0000 7fd15de69700 1 --2- 172.21.15.162:0/101847234 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd1580604f0 0x7fd158065120 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7fd14c002f30 tx=0x7fd14c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:29.782 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.771+0000 7fd156ffd700 1 -- 172.21.15.162:0/101847234 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd14c009b20 con 0x7fd1580604f0 2020-11-28T21:47:29.782 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.771+0000 7fd156ffd700 1 -- 172.21.15.162:0/101847234 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd14c011c60 con 0x7fd1580604f0 2020-11-28T21:47:29.782 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.771+0000 7fd15ee6b700 1 -- 172.21.15.162:0/101847234 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd158065ca0 con 0x7fd1580604f0 2020-11-28T21:47:29.783 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.771+0000 7fd156ffd700 1 -- 172.21.15.162:0/101847234 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd14c01d450 con 0x7fd1580604f0 2020-11-28T21:47:29.783 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.771+0000 7fd15ee6b700 1 -- 172.21.15.162:0/101847234 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1580676a0 con 0x7fd1580604f0 2020-11-28T21:47:29.783 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.771+0000 7fd15ee6b700 1 -- 172.21.15.162:0/101847234 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd158063ea0 con 0x7fd1580604f0 2020-11-28T21:47:29.784 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.775+0000 7fd156ffd700 1 -- 172.21.15.162:0/101847234 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd14c00f070 con 0x7fd1580604f0 2020-11-28T21:47:29.784 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.775+0000 7fd156ffd700 1 --2- 172.21.15.162:0/101847234 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd138055330 0x7fd1380577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:29.785 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.775+0000 7fd156ffd700 1 -- 172.21.15.162:0/101847234 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd14c06fab0 con 0x7fd1580604f0 2020-11-28T21:47:29.785 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.775+0000 7fd15d668700 1 --2- 172.21.15.162:0/101847234 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd138055330 0x7fd1380577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:29.785 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.775+0000 7fd15d668700 1 --2- 172.21.15.162:0/101847234 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd138055330 0x7fd1380577f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7fd14800b000 tx=0x7fd148008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:29.785 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.779+0000 7fd156ffd700 1 -- 172.21.15.162:0/101847234 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd14c045ce0 con 0x7fd1580604f0 2020-11-28T21:47:29.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.963+0000 7fd15ee6b700 1 -- 172.21.15.162:0/101847234 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd158057050 con 0x7fd1580604f0 2020-11-28T21:47:29.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.963+0000 7fd156ffd700 1 -- 172.21.15.162:0/101847234 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+2497 (secure 0 0 0) 0x7fd14c0492c0 con 0x7fd1580604f0 2020-11-28T21:47:29.967 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:47:29.967 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":10,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4839,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/1153453343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1153453343},{"type":"v1","addr":"172.21.15.162:6836","nonce":1153453343}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8},{"gid":4849,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2101329586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":2101329586},{"type":"v1","addr":"172.21.15.162:6837","nonce":2101329586}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:47:10.330722+0000","modified":"2020-11-28T21:47:11.333518+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4855},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4855":{"gid":4855,"name":"b","rank":0,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.118:6833/3038031199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3038031199},{"type":"v1","addr":"172.21.15.118:6833","nonce":3038031199}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-28T21:47:29.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.967+0000 7fd15ee6b700 1 -- 172.21.15.162:0/101847234 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd138055330 msgr2=0x7fd1380577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:29.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.967+0000 7fd15ee6b700 1 --2- 172.21.15.162:0/101847234 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd138055330 0x7fd1380577f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7fd14800b000 tx=0x7fd148008040).stop 2020-11-28T21:47:29.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.967+0000 7fd15ee6b700 1 -- 172.21.15.162:0/101847234 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd1580604f0 msgr2=0x7fd158065120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:29.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.967+0000 7fd15ee6b700 1 --2- 172.21.15.162:0/101847234 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd1580604f0 0x7fd158065120 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7fd14c002f30 tx=0x7fd14c015000).stop 2020-11-28T21:47:29.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.967+0000 7fd15ee6b700 1 -- 172.21.15.162:0/101847234 shutdown_connections 2020-11-28T21:47:29.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.967+0000 7fd15ee6b700 1 --2- 172.21.15.162:0/101847234 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd138055330 0x7fd1380577f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:29.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.967+0000 7fd15ee6b700 1 --2- 172.21.15.162:0/101847234 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd1580b0000 0x7fd158155fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:29.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.967+0000 7fd15ee6b700 1 --2- 172.21.15.162:0/101847234 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd1580604f0 0x7fd158065120 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:29.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.967+0000 7fd15ee6b700 1 --2- 172.21.15.162:0/101847234 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd158098b30 0x7fd158155ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:29.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.967+0000 7fd15ee6b700 1 -- 172.21.15.162:0/101847234 >> 172.21.15.162:0/101847234 conn(0x7fd15805b970 msgr2=0x7fd1580a8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:29.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.967+0000 7fd15ee6b700 1 -- 172.21.15.162:0/101847234 shutdown_connections 2020-11-28T21:47:29.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:29.967+0000 7fd15ee6b700 1 -- 172.21.15.162:0/101847234 wait complete. 2020-11-28T21:47:29.972 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 10 2020-11-28T21:47:29.985 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell mds.b cache drop 2020-11-28T21:47:30.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.091+0000 7f5c7f02a700 1 Processor -- start 2020-11-28T21:47:30.094 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.091+0000 7f5c7f02a700 1 -- start start 2020-11-28T21:47:30.095 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.095+0000 7f5c7f02a700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5c780b6130 0x7f5c78143020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:30.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.095+0000 7f5c7f02a700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5c780b54f0 0x7f5c78143510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:30.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.095+0000 7f5c7f02a700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c780ba0b0 con 0x7f5c780608f0 2020-11-28T21:47:30.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.095+0000 7f5c7f02a700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c78143a00 con 0x7f5c780b6130 2020-11-28T21:47:30.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.095+0000 7f5c7f02a700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c780b9e10 con 0x7f5c780b54f0 2020-11-28T21:47:30.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.095+0000 7f5c7cdc6700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f5c780608f0 0x7f5c7813c910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:45706/0 (socket says 172.21.15.162:45706) 2020-11-28T21:47:30.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.095+0000 7f5c7cdc6700 1 -- 172.21.15.162:0/821866766 learned_addr learned my addr 172.21.15.162:0/821866766 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:47:30.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.095+0000 7f5c7d5c7700 1 --2- 172.21.15.162:0/821866766 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5c780b54f0 0x7f5c78143510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:30.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.095+0000 7f5c77fff700 1 --2- 172.21.15.162:0/821866766 >> v2:172.21.15.118:3300/0 conn(0x7f5c780b6130 0x7f5c78143020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:30.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.095+0000 7f5c777fe700 1 -- 172.21.15.162:0/821866766 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2227096265 0 0) 0x7f5c780ba0b0 con 0x7f5c780608f0 2020-11-28T21:47:30.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.095+0000 7f5c777fe700 1 -- 172.21.15.162:0/821866766 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c58003230 con 0x7f5c780608f0 2020-11-28T21:47:30.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.095+0000 7f5c7d5c7700 1 -- 172.21.15.162:0/821866766 >> v2:172.21.15.118:3300/0 conn(0x7f5c780b6130 msgr2=0x7f5c78143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:30.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.095+0000 7f5c7d5c7700 1 --2- 172.21.15.162:0/821866766 >> v2:172.21.15.118:3300/0 conn(0x7f5c780b6130 0x7f5c78143020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:30.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.095+0000 7f5c7d5c7700 1 -- 172.21.15.162:0/821866766 >> v1:172.21.15.162:6789/0 conn(0x7f5c780608f0 legacy=0x7f5c7813c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:30.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.095+0000 7f5c7d5c7700 1 -- 172.21.15.162:0/821866766 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c78142b80 con 0x7f5c780b54f0 2020-11-28T21:47:30.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.095+0000 7f5c7d5c7700 1 --2- 172.21.15.162:0/821866766 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5c780b54f0 0x7f5c78143510 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f5c700032b0 tx=0x7f5c700091a0).ready entity=mon.2 client_cookie=51f3316c2297e7e0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:30.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.095+0000 7f5c777fe700 1 -- 172.21.15.162:0/821866766 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c7000b940 con 0x7f5c780b54f0 2020-11-28T21:47:30.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.095+0000 7f5c777fe700 1 -- 172.21.15.162:0/821866766 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7000baa0 con 0x7f5c780b54f0 2020-11-28T21:47:30.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.095+0000 7f5c777fe700 1 -- 172.21.15.162:0/821866766 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c7000bc10 con 0x7f5c780b54f0 2020-11-28T21:47:30.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.095+0000 7f5c7f02a700 1 -- 172.21.15.162:0/821866766 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5c780b54f0 msgr2=0x7f5c78143510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:30.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.095+0000 7f5c7f02a700 1 --2- 172.21.15.162:0/821866766 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5c780b54f0 0x7f5c78143510 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f5c700032b0 tx=0x7f5c700091a0).stop 2020-11-28T21:47:30.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.095+0000 7f5c7f02a700 1 -- 172.21.15.162:0/821866766 shutdown_connections 2020-11-28T21:47:30.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.095+0000 7f5c7f02a700 1 --2- 172.21.15.162:0/821866766 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5c780b54f0 0x7f5c78143510 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:30.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.095+0000 7f5c7f02a700 1 --2- 172.21.15.162:0/821866766 >> v2:172.21.15.118:3300/0 conn(0x7f5c780b6130 0x7f5c78143020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:30.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.095+0000 7f5c7f02a700 1 -- 172.21.15.162:0/821866766 >> 172.21.15.162:0/821866766 conn(0x7f5c78066c70 msgr2=0x7f5c78067050 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:30.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.095+0000 7f5c7f02a700 1 -- 172.21.15.162:0/821866766 shutdown_connections 2020-11-28T21:47:30.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.095+0000 7f5c7f02a700 1 -- 172.21.15.162:0/821866766 wait complete. 2020-11-28T21:47:30.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.099+0000 7f5c7f02a700 1 Processor -- start 2020-11-28T21:47:30.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.099+0000 7f5c7f02a700 1 -- start start 2020-11-28T21:47:30.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.099+0000 7f5c7f02a700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5c780608f0 0x7f5c780adba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:30.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.099+0000 7f5c7f02a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5c780b54f0 0x7f5c780b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:30.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.099+0000 7f5c7f02a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c780b6130 0x7f5c780aed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:30.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.099+0000 7f5c7f02a700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c780acf00 con 0x7f5c780b54f0 2020-11-28T21:47:30.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.099+0000 7f5c7f02a700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c78152520 con 0x7f5c780b6130 2020-11-28T21:47:30.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.099+0000 7f5c7f02a700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c780ad130 con 0x7f5c780608f0 2020-11-28T21:47:30.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.099+0000 7f5c7d5c7700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c780b6130 0x7f5c780aed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:30.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.099+0000 7f5c7d5c7700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c780b6130 0x7f5c780aed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38870/0 (socket says 172.21.15.162:38870) 2020-11-28T21:47:30.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.099+0000 7f5c7d5c7700 1 -- 172.21.15.162:0/2701543056 learned_addr learned my addr 172.21.15.162:0/2701543056 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:30.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.099+0000 7f5c77fff700 1 --2- 172.21.15.162:0/2701543056 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5c780b54f0 0x7f5c780b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:30.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.099+0000 7f5c7cdc6700 1 --2- 172.21.15.162:0/2701543056 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5c780608f0 0x7f5c780adba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:30.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.099+0000 7f5c7d5c7700 1 -- 172.21.15.162:0/2701543056 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5c780608f0 msgr2=0x7f5c780adba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:30.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.099+0000 7f5c7d5c7700 1 --2- 172.21.15.162:0/2701543056 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5c780608f0 0x7f5c780adba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:30.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.099+0000 7f5c7d5c7700 1 -- 172.21.15.162:0/2701543056 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5c780b54f0 msgr2=0x7f5c780b71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:30.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.099+0000 7f5c7d5c7700 1 --2- 172.21.15.162:0/2701543056 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5c780b54f0 0x7f5c780b71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:30.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.099+0000 7f5c7d5c7700 1 -- 172.21.15.162:0/2701543056 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c780ad3d0 con 0x7f5c780b6130 2020-11-28T21:47:30.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.099+0000 7f5c7cdc6700 1 --2- 172.21.15.162:0/2701543056 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5c780608f0 0x7f5c780adba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:47:30.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.099+0000 7f5c7d5c7700 1 --2- 172.21.15.162:0/2701543056 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c780b6130 0x7f5c780aed50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f5c700032b0 tx=0x7f5c70016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:30.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.099+0000 7f5c75ffb700 1 -- 172.21.15.162:0/2701543056 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c700203f0 con 0x7f5c780b6130 2020-11-28T21:47:30.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.099+0000 7f5c75ffb700 1 -- 172.21.15.162:0/2701543056 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c70020550 con 0x7f5c780b6130 2020-11-28T21:47:30.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.099+0000 7f5c7f02a700 1 -- 172.21.15.162:0/2701543056 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c781521a0 con 0x7f5c780b6130 2020-11-28T21:47:30.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.099+0000 7f5c75ffb700 1 -- 172.21.15.162:0/2701543056 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c700208b0 con 0x7f5c780b6130 2020-11-28T21:47:30.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.099+0000 7f5c7f02a700 1 -- 172.21.15.162:0/2701543056 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c780ae560 con 0x7f5c780b6130 2020-11-28T21:47:30.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.103+0000 7f5c75ffb700 1 -- 172.21.15.162:0/2701543056 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5c7001e070 con 0x7f5c780b6130 2020-11-28T21:47:30.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.103+0000 7f5c75ffb700 1 --2- 172.21.15.162:0/2701543056 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5c5c055440 0x7f5c5c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:30.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.103+0000 7f5c7cdc6700 1 --2- 172.21.15.162:0/2701543056 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5c5c055440 0x7f5c5c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:30.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.103+0000 7f5c75ffb700 1 -- 172.21.15.162:0/2701543056 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5c7006edf0 con 0x7f5c780b6130 2020-11-28T21:47:30.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.103+0000 7f5c7cdc6700 1 --2- 172.21.15.162:0/2701543056 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5c5c055440 0x7f5c5c057900 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f5c6800b000 tx=0x7f5c68009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:30.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.107+0000 7f5c8d9c5740 1 Processor -- start 2020-11-28T21:47:30.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.107+0000 7f5c8d9c5740 1 -- start start 2020-11-28T21:47:30.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.107+0000 7f5c8d9c5740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x171cf00 0x1765110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:30.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.107+0000 7f5c8d9c5740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x1585800 0x179f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:30.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.107+0000 7f5c8d9c5740 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x1517c00 0x179f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:30.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.107+0000 7f5c8d9c5740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x1691dd0 con 0x1585800 2020-11-28T21:47:30.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.107+0000 7f5c8d9c5740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x1622230 con 0x171cf00 2020-11-28T21:47:30.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.107+0000 7f5c8d9c5740 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x159e690 con 0x1517c00 2020-11-28T21:47:30.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.107+0000 7f5c7cdc6700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x171cf00 0x1765110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:30.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.107+0000 7f5c7cdc6700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x171cf00 0x1765110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38876/0 (socket says 172.21.15.162:38876) 2020-11-28T21:47:30.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.107+0000 7f5c7cdc6700 1 -- 172.21.15.162:0/3501814651 learned_addr learned my addr 172.21.15.162:0/3501814651 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:30.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.107+0000 7f5c7d5c7700 1 --2- 172.21.15.162:0/3501814651 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x1517c00 0x179f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:30.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.107+0000 7f5c77fff700 1 --2- 172.21.15.162:0/3501814651 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x1585800 0x179f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:30.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.107+0000 7f5c7cdc6700 1 -- 172.21.15.162:0/3501814651 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x1517c00 msgr2=0x179f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:30.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.107+0000 7f5c7cdc6700 1 --2- 172.21.15.162:0/3501814651 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x1517c00 0x179f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:30.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.107+0000 7f5c7cdc6700 1 -- 172.21.15.162:0/3501814651 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x1585800 msgr2=0x179f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:30.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.107+0000 7f5c7cdc6700 1 --2- 172.21.15.162:0/3501814651 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x1585800 0x179f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:30.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.107+0000 7f5c7cdc6700 1 -- 172.21.15.162:0/3501814651 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x16b2cf0 con 0x171cf00 2020-11-28T21:47:30.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.107+0000 7f5c7d5c7700 1 --2- 172.21.15.162:0/3501814651 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x1517c00 0x179f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:47:30.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.107+0000 7f5c77fff700 1 --2- 172.21.15.162:0/3501814651 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x1585800 0x179f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:47:30.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.107+0000 7f5c7cdc6700 1 --2- 172.21.15.162:0/3501814651 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x171cf00 0x1765110 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f5c68001d20 tx=0x7f5c68007000).ready entity=mon.1 client_cookie=7849bd3b8127e954 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:30.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.107+0000 7f5c7f02a700 1 -- 172.21.15.162:0/3501814651 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c68018aa0 con 0x171cf00 2020-11-28T21:47:30.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.111+0000 7f5c7f02a700 1 -- 172.21.15.162:0/3501814651 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c68018c00 con 0x171cf00 2020-11-28T21:47:30.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.111+0000 7f5c7f02a700 1 -- 172.21.15.162:0/3501814651 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c68016470 con 0x171cf00 2020-11-28T21:47:30.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.111+0000 7f5c8d9c5740 1 -- 172.21.15.162:0/3501814651 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x171cf00 msgr2=0x1765110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:30.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.111+0000 7f5c8d9c5740 1 --2- 172.21.15.162:0/3501814651 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x171cf00 0x1765110 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f5c68001d20 tx=0x7f5c68007000).stop 2020-11-28T21:47:30.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.111+0000 7f5c8d9c5740 1 -- 172.21.15.162:0/3501814651 shutdown_connections 2020-11-28T21:47:30.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.111+0000 7f5c8d9c5740 1 --2- 172.21.15.162:0/3501814651 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x1517c00 0x179f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:30.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.111+0000 7f5c8d9c5740 1 --2- 172.21.15.162:0/3501814651 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x171cf00 0x1765110 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:30.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.111+0000 7f5c8d9c5740 1 --2- 172.21.15.162:0/3501814651 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x1585800 0x179f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:30.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.111+0000 7f5c8d9c5740 1 -- 172.21.15.162:0/3501814651 >> 172.21.15.162:0/3501814651 conn(0x174c680 msgr2=0x174b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:30.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.111+0000 7f5c8d9c5740 1 -- 172.21.15.162:0/3501814651 shutdown_connections 2020-11-28T21:47:30.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.111+0000 7f5c8d9c5740 1 -- 172.21.15.162:0/3501814651 wait complete. 2020-11-28T21:47:30.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.111+0000 7f5c8d9c5740 1 -- start start 2020-11-28T21:47:30.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.111+0000 7f5c8d9c5740 1 Processor -- start 2020-11-28T21:47:30.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.115+0000 7f5c8d9c5740 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x1517c00 0x1792c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:30.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.115+0000 7f5c8d9c5740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x1585800 0x1793110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:30.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.115+0000 7f5c8d9c5740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x171cf00 0x17b15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:30.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.115+0000 7f5c8d9c5740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x16c1a50 con 0x171cf00 2020-11-28T21:47:30.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.115+0000 7f5c8d9c5740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x16c1b90 con 0x1585800 2020-11-28T21:47:30.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.115+0000 7f5c8d9c5740 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x16c3160 con 0x1517c00 2020-11-28T21:47:30.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.115+0000 7f5c77fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x1585800 0x1793110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:30.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.115+0000 7f5c77fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x1585800 0x1793110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38884/0 (socket says 172.21.15.162:38884) 2020-11-28T21:47:30.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.115+0000 7f5c8d9c5740 10 client.0 ll_register_callbacks cb 0x174c5e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-28T21:47:30.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.115+0000 7f5c77fff700 1 -- 172.21.15.162:0/1183737968 learned_addr learned my addr 172.21.15.162:0/1183737968 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:30.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.115+0000 7f5c7cdc6700 1 --2- 172.21.15.162:0/1183737968 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x1517c00 0x1792c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:30.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.115+0000 7f5c7d5c7700 1 --2- 172.21.15.162:0/1183737968 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x171cf00 0x17b15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:30.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.115+0000 7f5c77fff700 1 -- 172.21.15.162:0/1183737968 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x1517c00 msgr2=0x1792c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:30.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.115+0000 7f5c77fff700 1 --2- 172.21.15.162:0/1183737968 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x1517c00 0x1792c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:30.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.115+0000 7f5c77fff700 1 -- 172.21.15.162:0/1183737968 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x171cf00 msgr2=0x17b15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:30.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.115+0000 7f5c77fff700 1 --2- 172.21.15.162:0/1183737968 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x171cf00 0x17b15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:30.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.115+0000 7f5c77fff700 1 -- 172.21.15.162:0/1183737968 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1535770 con 0x1585800 2020-11-28T21:47:30.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.115+0000 7f5c7d5c7700 1 --2- 172.21.15.162:0/1183737968 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x171cf00 0x17b15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:47:30.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.115+0000 7f5c77fff700 1 --2- 172.21.15.162:0/1183737968 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x1585800 0x1793110 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7f5c6000b000 tx=0x7f5c60008040).ready entity=mon.1 client_cookie=a6e4ce7953ce525b server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:30.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.115+0000 7f5c4f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2020-11-28T21:47:30.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.115+0000 7f5c4f7fe700 1 -- 172.21.15.162:0/1183737968 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c60013ea0 con 0x1585800 2020-11-28T21:47:30.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.115+0000 7f5c8d9c5740 1 -- 172.21.15.162:0/1183737968 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x16727f0 con 0x1585800 2020-11-28T21:47:30.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.115+0000 7f5c4f7fe700 1 -- 172.21.15.162:0/1183737968 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5c60011990 con 0x1585800 2020-11-28T21:47:30.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.115+0000 7f5c4f7fe700 1 -- 172.21.15.162:0/1183737968 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=10) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c60011b50 con 0x1585800 2020-11-28T21:47:30.129 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.115+0000 7f5c8d9c5740 10 client.5098 fetch_fsmap learned FSMap version 10 2020-11-28T21:47:30.129 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.115+0000 7f5c8d9c5740 1 -- 172.21.15.162:0/1183737968 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=10,monmap=2+}) v3 -- 0x1540e50 con 0x1585800 2020-11-28T21:47:30.129 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.115+0000 7f5c4f7fe700 1 -- 172.21.15.162:0/1183737968 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x16727f0 con 0x1585800 2020-11-28T21:47:30.129 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.115+0000 7f5c4f7fe700 1 -- 172.21.15.162:0/1183737968 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 10) v1 ==== 1434+0+0 (secure 0 0 0) 0x7f5c60012040 con 0x1585800 2020-11-28T21:47:30.129 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.115+0000 7f5c8d9c5740 10 client.5098 fetch_fsmap finished waiting for FSMap version 10 2020-11-28T21:47:30.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.115+0000 7f5c8d9c5740 10 client.5098 resolve_mds: resolved ID 'b' to GID 4855 2020-11-28T21:47:30.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.115+0000 7f5c8d9c5740 20 client.5098 populate_metadata read hostname 'smithi162' 2020-11-28T21:47:30.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.115+0000 7f5c8d9c5740 1 --2- 172.21.15.162:0/1183737968 >> [v2:172.21.15.118:6832/3038031199,v1:172.21.15.118:6833/3038031199] conn(0x17b1f60 0x1795950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:30.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.115+0000 7f5c8d9c5740 4 client.5098 mds_command: new command op to 4855 tid=0[{"prefix": "get_command_descriptions"}] 2020-11-28T21:47:30.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.115+0000 7f5c8d9c5740 1 -- 172.21.15.162:0/1183737968 --> [v2:172.21.15.118:6832/3038031199,v1:172.21.15.118:6833/3038031199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x15aa8b0 con 0x17b1f60 2020-11-28T21:47:30.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.115+0000 7f5c7cdc6700 1 --2- 172.21.15.162:0/1183737968 >> [v2:172.21.15.118:6832/3038031199,v1:172.21.15.118:6833/3038031199] conn(0x17b1f60 0x1795950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:30.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.115+0000 7f5c4f7fe700 10 client.5098 ms_handle_connect on v2:172.21.15.118:6832/3038031199 2020-11-28T21:47:30.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.115+0000 7f5c7cdc6700 1 --2- 172.21.15.162:0/1183737968 >> [v2:172.21.15.118:6832/3038031199,v1:172.21.15.118:6833/3038031199] conn(0x17b1f60 0x1795950 crc :-1 s=READY pgs=10 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=63dcfb55727f15e5 server_cookie=930da6737e219a45 in_seq=0 out_seq=0 2020-11-28T21:47:30.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.119+0000 7f5c4f7fe700 1 -- 172.21.15.162:0/1183737968 <== mds.0 v2:172.21.15.118:6832/3038031199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16606 (crc 0 0 0) 0x15aa8b0 con 0x17b1f60 2020-11-28T21:47:30.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.119+0000 7f5c4f7fe700 10 client.5098 handle_command_reply: tid=0 2020-11-28T21:47:30.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.119+0000 7f5c8d9c5740 1 client.5098 shutdown 2020-11-28T21:47:30.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.119+0000 7f5c8d9c5740 1 -- 172.21.15.162:0/1183737968 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x1585800 msgr2=0x1793110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:30.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.119+0000 7f5c8d9c5740 1 --2- 172.21.15.162:0/1183737968 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x1585800 0x1793110 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7f5c6000b000 tx=0x7f5c60008040).stop 2020-11-28T21:47:30.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.119+0000 7f5c8d9c5740 1 -- 172.21.15.162:0/1183737968 shutdown_connections 2020-11-28T21:47:30.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.119+0000 7f5c8d9c5740 1 --2- 172.21.15.162:0/1183737968 >> [v2:172.21.15.118:6832/3038031199,v1:172.21.15.118:6833/3038031199] conn(0x17b1f60 0x1795950 crc :-1 s=READY pgs=10 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:30.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.119+0000 7f5c8d9c5740 1 --2- 172.21.15.162:0/1183737968 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x1517c00 0x1792c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:30.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.119+0000 7f5c8d9c5740 1 --2- 172.21.15.162:0/1183737968 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x171cf00 0x17b15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:30.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.119+0000 7f5c8d9c5740 1 --2- 172.21.15.162:0/1183737968 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x1585800 0x1793110 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:30.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.119+0000 7f5c8d9c5740 1 -- 172.21.15.162:0/1183737968 >> 172.21.15.162:0/1183737968 conn(0x174c680 msgr2=0x1798250 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:30.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.119+0000 7f5c4f7fe700 0 client.5098 ms_handle_reset on v2:172.21.15.118:6832/3038031199 2020-11-28T21:47:30.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.119+0000 7f5c8d9c5740 1 -- 172.21.15.162:0/1183737968 shutdown_connections 2020-11-28T21:47:30.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.119+0000 7f5c8d9c5740 1 -- 172.21.15.162:0/1183737968 wait complete. 2020-11-28T21:47:30.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.119+0000 7f5c8d9c5740 20 client.5098 trim_cache size 0 max 16384 2020-11-28T21:47:30.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.131+0000 7f5c8d9c5740 1 Processor -- start 2020-11-28T21:47:30.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.131+0000 7f5c8d9c5740 1 -- start start 2020-11-28T21:47:30.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.131+0000 7f5c8d9c5740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x1517c00 0x17664b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:30.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.131+0000 7f5c8d9c5740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x1585800 0x17a3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:30.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.131+0000 7f5c8d9c5740 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x171cf00 0x174b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:30.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.131+0000 7f5c8d9c5740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x16a44f0 con 0x1517c00 2020-11-28T21:47:30.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.131+0000 7f5c8d9c5740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x1763b50 con 0x1585800 2020-11-28T21:47:30.138 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.131+0000 7f5c8d9c5740 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x17b1900 con 0x171cf00 2020-11-28T21:47:30.138 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.131+0000 7f5c77fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x1585800 0x17a3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:30.138 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.131+0000 7f5c77fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x1585800 0x17a3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38892/0 (socket says 172.21.15.162:38892) 2020-11-28T21:47:30.138 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.131+0000 7f5c77fff700 1 -- 172.21.15.162:0/2992465611 learned_addr learned my addr 172.21.15.162:0/2992465611 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:30.139 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.131+0000 7f5c7cdc6700 1 --2- 172.21.15.162:0/2992465611 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x1517c00 0x17664b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:30.139 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.131+0000 7f5c7d5c7700 1 --2- 172.21.15.162:0/2992465611 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x171cf00 0x174b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:30.139 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.131+0000 7f5c77fff700 1 -- 172.21.15.162:0/2992465611 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x171cf00 msgr2=0x174b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:30.139 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.131+0000 7f5c77fff700 1 --2- 172.21.15.162:0/2992465611 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x171cf00 0x174b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:30.139 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.131+0000 7f5c77fff700 1 -- 172.21.15.162:0/2992465611 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x1517c00 msgr2=0x17664b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:30.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.131+0000 7f5c77fff700 1 --2- 172.21.15.162:0/2992465611 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x1517c00 0x17664b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:30.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.131+0000 7f5c77fff700 1 -- 172.21.15.162:0/2992465611 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c78145f60 con 0x1585800 2020-11-28T21:47:30.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.131+0000 7f5c7cdc6700 1 --2- 172.21.15.162:0/2992465611 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x1517c00 0x17664b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:47:30.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.131+0000 7f5c77fff700 1 --2- 172.21.15.162:0/2992465611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x1585800 0x17a3010 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x1797c00 tx=0x7f5c60021000).ready entity=mon.1 client_cookie=851ea25227ed35cb server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:30.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.131+0000 7f5c6dffb700 1 -- 172.21.15.162:0/2992465611 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c60013ea0 con 0x1585800 2020-11-28T21:47:30.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.131+0000 7f5c6dffb700 1 -- 172.21.15.162:0/2992465611 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c60013030 con 0x1585800 2020-11-28T21:47:30.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.131+0000 7f5c6dffb700 1 -- 172.21.15.162:0/2992465611 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c600131a0 con 0x1585800 2020-11-28T21:47:30.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.131+0000 7f5c8d9c5740 1 -- 172.21.15.162:0/2992465611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x1585800 msgr2=0x17a3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:47:30.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.131+0000 7f5c8d9c5740 1 --2- 172.21.15.162:0/2992465611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x1585800 0x17a3010 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x1797c00 tx=0x7f5c60021000).stop 2020-11-28T21:47:30.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c8d9c5740 1 -- 172.21.15.162:0/2992465611 shutdown_connections 2020-11-28T21:47:30.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c8d9c5740 1 --2- 172.21.15.162:0/2992465611 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x171cf00 0x174b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:30.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c8d9c5740 1 --2- 172.21.15.162:0/2992465611 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x1517c00 0x17664b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:30.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c8d9c5740 1 --2- 172.21.15.162:0/2992465611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x1585800 0x17a3010 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:30.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c8d9c5740 1 -- 172.21.15.162:0/2992465611 >> 172.21.15.162:0/2992465611 conn(0x174c680 msgr2=0x1763150 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:47:30.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c8d9c5740 1 -- 172.21.15.162:0/2992465611 shutdown_connections 2020-11-28T21:47:30.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c8d9c5740 1 -- 172.21.15.162:0/2992465611 wait complete. 2020-11-28T21:47:30.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c8d9c5740 1 -- start start 2020-11-28T21:47:30.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c8d9c5740 1 Processor -- start 2020-11-28T21:47:30.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c8d9c5740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x1517c00 0x17ffe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:30.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c8d9c5740 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x1585800 0x1800300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:30.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c8d9c5740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x171cf00 0x17fd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:30.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c8d9c5740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x17aa020 con 0x171cf00 2020-11-28T21:47:30.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c8d9c5740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x16910c0 con 0x1517c00 2020-11-28T21:47:30.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c8d9c5740 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x1798650 con 0x1585800 2020-11-28T21:47:30.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c8d9c5740 10 client.0 ll_register_callbacks cb 0x7f5c68007040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-28T21:47:30.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c7cdc6700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x1517c00 0x17ffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:30.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c7cdc6700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x1517c00 0x17ffe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38896/0 (socket says 172.21.15.162:38896) 2020-11-28T21:47:30.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c7cdc6700 1 -- 172.21.15.162:0/2201358244 learned_addr learned my addr 172.21.15.162:0/2201358244 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:47:30.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c7d5c7700 1 --2- 172.21.15.162:0/2201358244 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x171cf00 0x17fd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:30.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c77fff700 1 --2- 172.21.15.162:0/2201358244 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x1585800 0x1800300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:30.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c7cdc6700 1 -- 172.21.15.162:0/2201358244 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x1585800 msgr2=0x1800300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:30.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c7cdc6700 1 --2- 172.21.15.162:0/2201358244 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x1585800 0x1800300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:30.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c7cdc6700 1 -- 172.21.15.162:0/2201358244 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x171cf00 msgr2=0x17fd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:47:30.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c7cdc6700 1 --2- 172.21.15.162:0/2201358244 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x171cf00 0x17fd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:47:30.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c7cdc6700 1 -- 172.21.15.162:0/2201358244 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x17671b0 con 0x1517c00 2020-11-28T21:47:30.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c7d5c7700 1 --2- 172.21.15.162:0/2201358244 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x171cf00 0x17fd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:47:30.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c7cdc6700 1 --2- 172.21.15.162:0/2201358244 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x1517c00 0x17ffe10 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f5c68019000 tx=0x7f5c68024040).ready entity=mon.1 client_cookie=439aa352ac47b94a server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:47:30.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c6cff9700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2020-11-28T21:47:30.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c6cff9700 1 -- 172.21.15.162:0/2201358244 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c68018d00 con 0x1517c00 2020-11-28T21:47:30.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c8d9c5740 1 -- 172.21.15.162:0/2201358244 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x157b4b0 con 0x1517c00 2020-11-28T21:47:30.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c6cff9700 1 -- 172.21.15.162:0/2201358244 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5c68031c80 con 0x1517c00 2020-11-28T21:47:30.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c6cff9700 1 -- 172.21.15.162:0/2201358244 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=10) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c68018e60 con 0x1517c00 2020-11-28T21:47:30.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c8d9c5740 10 client.5104 fetch_fsmap learned FSMap version 10 2020-11-28T21:47:30.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c8d9c5740 1 -- 172.21.15.162:0/2201358244 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=10,monmap=2+}) v3 -- 0x1767810 con 0x1517c00 2020-11-28T21:47:30.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c6cff9700 1 -- 172.21.15.162:0/2201358244 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x157b4b0 con 0x1517c00 2020-11-28T21:47:30.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c6cff9700 1 -- 172.21.15.162:0/2201358244 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 10) v1 ==== 1434+0+0 (secure 0 0 0) 0x7f5c68021a40 con 0x1517c00 2020-11-28T21:47:30.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c8d9c5740 10 client.5104 fetch_fsmap finished waiting for FSMap version 10 2020-11-28T21:47:30.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c8d9c5740 10 client.5104 resolve_mds: resolved ID 'b' to GID 4855 2020-11-28T21:47:30.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c8d9c5740 20 client.5104 populate_metadata read hostname 'smithi162' 2020-11-28T21:47:30.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c8d9c5740 1 --2- 172.21.15.162:0/2201358244 >> [v2:172.21.15.118:6832/3038031199,v1:172.21.15.118:6833/3038031199] conn(0x17ae810 0x17fd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:47:30.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c8d9c5740 4 client.5104 mds_command: new command op to 4855 tid=0[{"prefix": "cache drop"}] 2020-11-28T21:47:30.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.135+0000 7f5c8d9c5740 1 -- 172.21.15.162:0/2201358244 --> [v2:172.21.15.118:6832/3038031199,v1:172.21.15.118:6833/3038031199] -- command(tid 0: {"prefix": "cache drop"}) v1 -- 0x15c9710 con 0x17ae810 2020-11-28T21:47:30.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.139+0000 7f5c77fff700 1 --2- 172.21.15.162:0/2201358244 >> [v2:172.21.15.118:6832/3038031199,v1:172.21.15.118:6833/3038031199] conn(0x17ae810 0x17fd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:47:30.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.139+0000 7f5c77fff700 1 --2- 172.21.15.162:0/2201358244 >> [v2:172.21.15.118:6832/3038031199,v1:172.21.15.118:6833/3038031199] conn(0x17ae810 0x17fd4f0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=abbd8bc423527002 server_cookie=e0984fc9dcdee542 in_seq=0 out_seq=0 2020-11-28T21:47:30.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:47:30.139+0000 7f5c6cff9700 10 client.5104 ms_handle_connect on v2:172.21.15.118:6832/3038031199 2020-11-28T21:47:35.244 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:47:35.247 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:47:35.251 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:47:35.254 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:48:05.365 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:48:05.369 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:48:05.371 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:48:05.374 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:48:14.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:14.587+0000 7f5c6cff9700 1 -- 172.21.15.162:0/2201358244 <== mds.0 v2:172.21.15.118:6832/3038031199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+386 (crc 0 0 0) 0x15c9710 con 0x17ae810 2020-11-28T21:48:14.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:14.587+0000 7f5c6cff9700 10 client.5104 handle_command_reply: tid=0 2020-11-28T21:48:14.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:14.587+0000 7f5c8d9c5740 1 client.5104 shutdown 2020-11-28T21:48:14.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:14.587+0000 7f5c8d9c5740 1 -- 172.21.15.162:0/2201358244 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x1517c00 msgr2=0x17ffe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:14.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:14.587+0000 7f5c8d9c5740 1 --2- 172.21.15.162:0/2201358244 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x1517c00 0x17ffe10 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f5c68019000 tx=0x7f5c68024040).stop 2020-11-28T21:48:14.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:14.587+0000 7f5c8d9c5740 1 -- 172.21.15.162:0/2201358244 shutdown_connections 2020-11-28T21:48:14.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:14.587+0000 7f5c8d9c5740 1 --2- 172.21.15.162:0/2201358244 >> [v2:172.21.15.118:6832/3038031199,v1:172.21.15.118:6833/3038031199] conn(0x17ae810 0x17fd4f0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:14.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:14.587+0000 7f5c8d9c5740 1 --2- 172.21.15.162:0/2201358244 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x171cf00 0x17fd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:14.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:14.587+0000 7f5c8d9c5740 1 --2- 172.21.15.162:0/2201358244 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x1517c00 0x17ffe10 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:14.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:14.591+0000 7f5c6cff9700 0 client.5104 ms_handle_reset on v2:172.21.15.118:6832/3038031199 2020-11-28T21:48:14.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:14.591+0000 7f5c8d9c5740 1 --2- 172.21.15.162:0/2201358244 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x1585800 0x1800300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:14.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:14.591+0000 7f5c8d9c5740 1 -- 172.21.15.162:0/2201358244 >> 172.21.15.162:0/2201358244 conn(0x174c680 msgr2=0x174b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:14.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:14.591+0000 7f5c8d9c5740 1 -- 172.21.15.162:0/2201358244 shutdown_connections 2020-11-28T21:48:14.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:14.591+0000 7f5c8d9c5740 1 -- 172.21.15.162:0/2201358244 wait complete. 2020-11-28T21:48:14.594 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:14.591+0000 7f5c8d9c5740 20 client.5104 trim_cache size 0 max 16384 2020-11-28T21:48:14.594 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2020-11-28T21:48:14.595 INFO:teuthology.orchestra.run.smithi162.stdout: "client_recall": { 2020-11-28T21:48:14.595 INFO:teuthology.orchestra.run.smithi162.stdout: "return_code": 0, 2020-11-28T21:48:14.595 INFO:teuthology.orchestra.run.smithi162.stdout: "message": "(0) Success", 2020-11-28T21:48:14.595 INFO:teuthology.orchestra.run.smithi162.stdout: "recalled": 920 2020-11-28T21:48:14.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:14.596 INFO:teuthology.orchestra.run.smithi162.stdout: "flush_journal": { 2020-11-28T21:48:14.596 INFO:teuthology.orchestra.run.smithi162.stdout: "return_code": 0, 2020-11-28T21:48:14.596 INFO:teuthology.orchestra.run.smithi162.stdout: "message": "" 2020-11-28T21:48:14.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:14.597 INFO:teuthology.orchestra.run.smithi162.stdout: "trim_cache": { 2020-11-28T21:48:14.597 INFO:teuthology.orchestra.run.smithi162.stdout: "trimmed": 900 2020-11-28T21:48:14.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:14.597 INFO:teuthology.orchestra.run.smithi162.stdout: "cache": { 2020-11-28T21:48:14.598 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": { 2020-11-28T21:48:14.598 INFO:teuthology.orchestra.run.smithi162.stdout: "items": 2491, 2020-11-28T21:48:14.598 INFO:teuthology.orchestra.run.smithi162.stdout: "bytes": 431648 2020-11-28T21:48:14.598 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:14.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:14.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 44.450269462999998 2020-11-28T21:48:14.599 INFO:teuthology.orchestra.run.smithi162.stdout:} 2020-11-28T21:48:14.600 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:14.591+0000 7f5c7f02a700 1 -- 172.21.15.162:0/2701543056 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5c5c055440 msgr2=0x7f5c5c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:14.601 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:14.591+0000 7f5c7f02a700 1 --2- 172.21.15.162:0/2701543056 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5c5c055440 0x7f5c5c057900 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f5c6800b000 tx=0x7f5c68009040).stop 2020-11-28T21:48:14.601 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:14.591+0000 7f5c7f02a700 1 -- 172.21.15.162:0/2701543056 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c780b6130 msgr2=0x7f5c780aed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:14.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:14.591+0000 7f5c7f02a700 1 --2- 172.21.15.162:0/2701543056 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c780b6130 0x7f5c780aed50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f5c700032b0 tx=0x7f5c70016000).stop 2020-11-28T21:48:14.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:14.591+0000 7f5c7f02a700 1 -- 172.21.15.162:0/2701543056 shutdown_connections 2020-11-28T21:48:14.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:14.591+0000 7f5c7f02a700 1 --2- 172.21.15.162:0/2701543056 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5c5c055440 0x7f5c5c057900 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:14.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:14.591+0000 7f5c7f02a700 1 --2- 172.21.15.162:0/2701543056 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5c780608f0 0x7f5c780adba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:14.603 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:14.591+0000 7f5c7f02a700 1 --2- 172.21.15.162:0/2701543056 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c780b6130 0x7f5c780aed50 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:14.603 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:14.591+0000 7f5c7f02a700 1 --2- 172.21.15.162:0/2701543056 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5c780b54f0 0x7f5c780b71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:14.603 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:14.591+0000 7f5c7f02a700 1 -- 172.21.15.162:0/2701543056 >> 172.21.15.162:0/2701543056 conn(0x7f5c78066c70 msgr2=0x7f5c78067050 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:14.603 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:14.591+0000 7f5c7f02a700 1 -- 172.21.15.162:0/2701543056 shutdown_connections 2020-11-28T21:48:14.603 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:14.591+0000 7f5c7f02a700 1 -- 172.21.15.162:0/2701543056 wait complete. 2020-11-28T21:48:14.605 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-28T21:48:14.606 INFO:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:48:14.624 INFO:teuthology.orchestra.run.smithi118:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:48:14.638 INFO:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:48:14.656 INFO:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:48:14.673 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-28T21:48:14.674 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:48:14.674 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:48:14.675 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:48:14.723 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2020-11-28T21:48:14.724 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-28T21:48:14.725 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2020-11-28T21:48:14.725 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-28T21:48:14.725 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:48:14.809 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:ceph-fuse[11430]: fuse finished with error 0 and tester_r 0 2020-11-28T21:48:14.810 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:48:14.811 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:48:14.811 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:48:14.857 INFO:teuthology.orchestra.run.smithi192.stdout:ext2/ext3 2020-11-28T21:48:14.859 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-11-28T21:48:14.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-28T21:48:14.976 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:48:15.026 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:48:15.027 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:48:15.027 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:48:15.035 INFO:teuthology.orchestra.run.smithi096.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-11-28T21:48:15.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:48:15.036 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2020-11-28T21:48:15.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-28T21:48:15.037 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:48:15.082 INFO:teuthology.orchestra.run.smithi096.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-11-28T21:48:15.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:48:15.084 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config rm mon mon_allow_pool_delete 2020-11-28T21:48:15.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.207+0000 7fd06f59e700 1 Processor -- start 2020-11-28T21:48:15.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.207+0000 7fd06f59e700 1 -- start start 2020-11-28T21:48:15.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.207+0000 7fd06f59e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd0700b5bf0 0x7fd070142980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:15.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.207+0000 7fd06f59e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd07005f380 con 0x7fd0700b3a60 2020-11-28T21:48:15.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.207+0000 7fd06f59e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd070143930 con 0x7fd070060cb0 2020-11-28T21:48:15.212 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.207+0000 7fd06f59e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd07005f5b0 con 0x7fd0700b5bf0 2020-11-28T21:48:15.212 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.207+0000 7fd06dd9b700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fd070060cb0 0x7fd070060230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:45750/0 (socket says 172.21.15.162:45750) 2020-11-28T21:48:15.212 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.207+0000 7fd06dd9b700 1 -- 172.21.15.162:0/3666219400 learned_addr learned my addr 172.21.15.162:0/3666219400 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:48:15.212 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.207+0000 7fd06e59c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd0700b5bf0 0x7fd070142980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:15.212 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.207+0000 7fd06d59a700 1 -- 172.21.15.162:0/3666219400 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3035159956 0 0) 0x7fd0640010d0 con 0x7fd070060cb0 2020-11-28T21:48:15.213 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.207+0000 7fd06d59a700 1 -- 172.21.15.162:0/3666219400 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd054003230 con 0x7fd070060cb0 2020-11-28T21:48:15.213 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.207+0000 7fd06e59c700 1 -- 172.21.15.162:0/3666219400 >> v1:172.21.15.162:6789/0 conn(0x7fd070060cb0 legacy=0x7fd070060230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:15.213 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.207+0000 7fd06e59c700 1 -- 172.21.15.162:0/3666219400 >> v1:172.21.15.118:6789/0 conn(0x7fd0700b3a60 legacy=0x7fd07013fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:15.213 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.207+0000 7fd06e59c700 1 -- 172.21.15.162:0/3666219400 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd07005f850 con 0x7fd0700b5bf0 2020-11-28T21:48:15.213 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.207+0000 7fd06e59c700 1 --2- 172.21.15.162:0/3666219400 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd0700b5bf0 0x7fd070142980 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7fd060002f30 tx=0x7fd060006dc0).ready entity=mon.1 client_cookie=4a92d1b6afc213c server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:15.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.207+0000 7fd06d59a700 1 -- 172.21.15.162:0/3666219400 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd060009800 con 0x7fd0700b5bf0 2020-11-28T21:48:15.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.207+0000 7fd06d59a700 1 -- 172.21.15.162:0/3666219400 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd060009960 con 0x7fd0700b5bf0 2020-11-28T21:48:15.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.207+0000 7fd06d59a700 1 -- 172.21.15.162:0/3666219400 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd060009cc0 con 0x7fd0700b5bf0 2020-11-28T21:48:15.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.207+0000 7fd06f59e700 1 -- 172.21.15.162:0/3666219400 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd0700b5bf0 msgr2=0x7fd070142980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:15.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.207+0000 7fd06f59e700 1 --2- 172.21.15.162:0/3666219400 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd0700b5bf0 0x7fd070142980 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7fd060002f30 tx=0x7fd060006dc0).stop 2020-11-28T21:48:15.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.211+0000 7fd06f59e700 1 -- 172.21.15.162:0/3666219400 shutdown_connections 2020-11-28T21:48:15.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.211+0000 7fd06f59e700 1 --2- 172.21.15.162:0/3666219400 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd0700b5bf0 0x7fd070142980 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:15.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.211+0000 7fd06f59e700 1 -- 172.21.15.162:0/3666219400 >> 172.21.15.162:0/3666219400 conn(0x7fd07005b970 msgr2=0x7fd07005bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:15.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.211+0000 7fd06f59e700 1 -- 172.21.15.162:0/3666219400 shutdown_connections 2020-11-28T21:48:15.216 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.211+0000 7fd06f59e700 1 -- 172.21.15.162:0/3666219400 wait complete. 2020-11-28T21:48:15.217 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.211+0000 7fd06f59e700 1 Processor -- start 2020-11-28T21:48:15.218 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.211+0000 7fd06f59e700 1 -- start start 2020-11-28T21:48:15.218 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.215+0000 7fd06f59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd070060cb0 0x7fd070144b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:15.218 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.215+0000 7fd06f59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd0700b3a60 0x7fd0700b7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:15.218 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.215+0000 7fd06f59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd0700b5bf0 0x7fd070152050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:15.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.215+0000 7fd06f59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd070151950 con 0x7fd0700b3a60 2020-11-28T21:48:15.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.215+0000 7fd06f59e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd070144260 con 0x7fd070060cb0 2020-11-28T21:48:15.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.215+0000 7fd06f59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd070151bb0 con 0x7fd0700b5bf0 2020-11-28T21:48:15.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.215+0000 7fd06e59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd070060cb0 0x7fd070144b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:15.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.215+0000 7fd06e59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd070060cb0 0x7fd070144b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38910/0 (socket says 172.21.15.162:38910) 2020-11-28T21:48:15.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.215+0000 7fd06e59c700 1 -- 172.21.15.162:0/1520129455 learned_addr learned my addr 172.21.15.162:0/1520129455 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:15.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.215+0000 7fd06dd9b700 1 --2- 172.21.15.162:0/1520129455 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd0700b3a60 0x7fd0700b7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:15.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.215+0000 7fd06ed9d700 1 --2- 172.21.15.162:0/1520129455 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd0700b5bf0 0x7fd070152050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:15.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.215+0000 7fd06e59c700 1 -- 172.21.15.162:0/1520129455 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd0700b5bf0 msgr2=0x7fd070152050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:15.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.215+0000 7fd06e59c700 1 --2- 172.21.15.162:0/1520129455 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd0700b5bf0 0x7fd070152050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:15.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.215+0000 7fd06e59c700 1 -- 172.21.15.162:0/1520129455 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd0700b3a60 msgr2=0x7fd0700b7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:15.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.215+0000 7fd06e59c700 1 --2- 172.21.15.162:0/1520129455 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd0700b3a60 0x7fd0700b7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:15.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.215+0000 7fd06e59c700 1 -- 172.21.15.162:0/1520129455 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd070144500 con 0x7fd070060cb0 2020-11-28T21:48:15.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.215+0000 7fd06e59c700 1 --2- 172.21.15.162:0/1520129455 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd070060cb0 0x7fd070144b50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7fd060002f30 tx=0x7fd060015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:15.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.215+0000 7fd05f7fe700 1 -- 172.21.15.162:0/1520129455 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd060011a20 con 0x7fd070060cb0 2020-11-28T21:48:15.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.215+0000 7fd06f59e700 1 -- 172.21.15.162:0/1520129455 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd070144760 con 0x7fd070060cb0 2020-11-28T21:48:15.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.215+0000 7fd05f7fe700 1 -- 172.21.15.162:0/1520129455 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd060011b80 con 0x7fd070060cb0 2020-11-28T21:48:15.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.215+0000 7fd05f7fe700 1 -- 172.21.15.162:0/1520129455 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd06001d440 con 0x7fd070060cb0 2020-11-28T21:48:15.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.215+0000 7fd06f59e700 1 -- 172.21.15.162:0/1520129455 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0700ada90 con 0x7fd070060cb0 2020-11-28T21:48:15.224 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.219+0000 7fd06f59e700 1 -- 172.21.15.162:0/1520129455 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd070144920 con 0x7fd070060cb0 2020-11-28T21:48:15.224 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.219+0000 7fd05f7fe700 1 -- 172.21.15.162:0/1520129455 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd060017070 con 0x7fd070060cb0 2020-11-28T21:48:15.225 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.219+0000 7fd05f7fe700 1 --2- 172.21.15.162:0/1520129455 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd04c0554e0 0x7fd04c0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:15.225 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.219+0000 7fd05f7fe700 1 -- 172.21.15.162:0/1520129455 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd06006eb50 con 0x7fd070060cb0 2020-11-28T21:48:15.225 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.219+0000 7fd06dd9b700 1 --2- 172.21.15.162:0/1520129455 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd04c0554e0 0x7fd04c0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:15.225 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.219+0000 7fd06dd9b700 1 --2- 172.21.15.162:0/1520129455 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd04c0554e0 0x7fd04c0579a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7fd06400d000 tx=0x7fd064009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:15.226 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.223+0000 7fd05f7fe700 1 -- 172.21.15.162:0/1520129455 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd060044cc0 con 0x7fd070060cb0 2020-11-28T21:48:15.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.367+0000 7fd06f59e700 1 -- 172.21.15.162:0/1520129455 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd070057050 con 0x7fd070060cb0 2020-11-28T21:48:15.377 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.375+0000 7fd05f7fe700 1 -- 172.21.15.162:0/1520129455 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v3) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd0600220a0 con 0x7fd070060cb0 2020-11-28T21:48:15.379 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.379+0000 7fd06f59e700 1 -- 172.21.15.162:0/1520129455 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd04c0554e0 msgr2=0x7fd04c0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:15.379 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.379+0000 7fd06f59e700 1 --2- 172.21.15.162:0/1520129455 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd04c0554e0 0x7fd04c0579a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7fd06400d000 tx=0x7fd064009040).stop 2020-11-28T21:48:15.380 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.379+0000 7fd06f59e700 1 -- 172.21.15.162:0/1520129455 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd070060cb0 msgr2=0x7fd070144b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:15.380 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.379+0000 7fd06f59e700 1 --2- 172.21.15.162:0/1520129455 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd070060cb0 0x7fd070144b50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7fd060002f30 tx=0x7fd060015000).stop 2020-11-28T21:48:15.380 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.379+0000 7fd06f59e700 1 -- 172.21.15.162:0/1520129455 shutdown_connections 2020-11-28T21:48:15.381 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.379+0000 7fd06f59e700 1 --2- 172.21.15.162:0/1520129455 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd04c0554e0 0x7fd04c0579a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:15.381 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.379+0000 7fd06f59e700 1 --2- 172.21.15.162:0/1520129455 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd0700b5bf0 0x7fd070152050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:15.381 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.379+0000 7fd06f59e700 1 --2- 172.21.15.162:0/1520129455 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd070060cb0 0x7fd070144b50 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:15.381 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.379+0000 7fd06f59e700 1 --2- 172.21.15.162:0/1520129455 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd0700b3a60 0x7fd0700b7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:15.381 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.379+0000 7fd06f59e700 1 -- 172.21.15.162:0/1520129455 >> 172.21.15.162:0/1520129455 conn(0x7fd07005b970 msgr2=0x7fd070140f10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:15.382 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.379+0000 7fd06f59e700 1 -- 172.21.15.162:0/1520129455 shutdown_connections 2020-11-28T21:48:15.382 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.379+0000 7fd06f59e700 1 -- 172.21.15.162:0/1520129455 wait complete. 2020-11-28T21:48:15.393 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command' 2020-11-28T21:48:15.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.499+0000 7fde4ae3e700 1 Processor -- start 2020-11-28T21:48:15.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.499+0000 7fde4ae3e700 1 -- start start 2020-11-28T21:48:15.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.503+0000 7fde4ae3e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fde440611e0 0x7fde440615c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:15.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.503+0000 7fde4ae3e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fde440b4b40 0x7fde4413ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:15.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.503+0000 7fde4ae3e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde440b1790 con 0x7fde440b10f0 2020-11-28T21:48:15.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.503+0000 7fde4ae3e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde440b1580 con 0x7fde440611e0 2020-11-28T21:48:15.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.503+0000 7fde4ae3e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde440b1a30 con 0x7fde440b4b40 2020-11-28T21:48:15.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.503+0000 7fde49e3c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fde440611e0 0x7fde440615c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:15.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.503+0000 7fde49e3c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fde440611e0 0x7fde440615c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38918/0 (socket says 172.21.15.162:38918) 2020-11-28T21:48:15.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.503+0000 7fde49e3c700 1 -- 172.21.15.162:0/4066282888 learned_addr learned my addr 172.21.15.162:0/4066282888 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:15.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.503+0000 7fde4a63d700 1 --2- 172.21.15.162:0/4066282888 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fde440b4b40 0x7fde4413ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:15.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.503+0000 7fde49e3c700 1 -- 172.21.15.162:0/4066282888 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fde440b4b40 msgr2=0x7fde4413ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:15.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.503+0000 7fde49e3c700 1 --2- 172.21.15.162:0/4066282888 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fde440b4b40 0x7fde4413ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:15.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.503+0000 7fde49e3c700 1 -- 172.21.15.162:0/4066282888 >> v1:172.21.15.118:6789/0 conn(0x7fde440b10f0 legacy=0x7fde440b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:15.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.503+0000 7fde49e3c700 1 -- 172.21.15.162:0/4066282888 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde440b3f90 con 0x7fde440611e0 2020-11-28T21:48:15.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.503+0000 7fde4a63d700 1 --2- 172.21.15.162:0/4066282888 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fde440b4b40 0x7fde4413ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:48:15.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.503+0000 7fde49e3c700 1 --2- 172.21.15.162:0/4066282888 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde440611e0 0x7fde440615c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7fde400032b0 tx=0x7fde4000a410).ready entity=mon.1 client_cookie=df584a7f3da83cf1 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:15.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.503+0000 7fde48e3a700 1 -- 172.21.15.162:0/4066282888 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde40010410 con 0x7fde440611e0 2020-11-28T21:48:15.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.503+0000 7fde48e3a700 1 -- 172.21.15.162:0/4066282888 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde40010570 con 0x7fde440611e0 2020-11-28T21:48:15.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.503+0000 7fde48e3a700 1 -- 172.21.15.162:0/4066282888 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde400108d0 con 0x7fde440611e0 2020-11-28T21:48:15.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.503+0000 7fde4ae3e700 1 -- 172.21.15.162:0/4066282888 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde440611e0 msgr2=0x7fde440615c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:15.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.503+0000 7fde4ae3e700 1 --2- 172.21.15.162:0/4066282888 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde440611e0 0x7fde440615c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7fde400032b0 tx=0x7fde4000a410).stop 2020-11-28T21:48:15.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.503+0000 7fde4ae3e700 1 -- 172.21.15.162:0/4066282888 shutdown_connections 2020-11-28T21:48:15.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.503+0000 7fde4ae3e700 1 --2- 172.21.15.162:0/4066282888 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fde440b4b40 0x7fde4413ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:15.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.503+0000 7fde4ae3e700 1 --2- 172.21.15.162:0/4066282888 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde440611e0 0x7fde440615c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:15.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.503+0000 7fde4ae3e700 1 -- 172.21.15.162:0/4066282888 >> 172.21.15.162:0/4066282888 conn(0x7fde440b54f0 msgr2=0x7fde440b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:15.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.503+0000 7fde4ae3e700 1 -- 172.21.15.162:0/4066282888 shutdown_connections 2020-11-28T21:48:15.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.503+0000 7fde4ae3e700 1 -- 172.21.15.162:0/4066282888 wait complete. 2020-11-28T21:48:15.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.507+0000 7fde4ae3e700 1 Processor -- start 2020-11-28T21:48:15.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.507+0000 7fde4ae3e700 1 -- start start 2020-11-28T21:48:15.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.507+0000 7fde4ae3e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde440611e0 0x7fde44140980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:15.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.507+0000 7fde4ae3e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fde440b10f0 0x7fde44144770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:15.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.507+0000 7fde4ae3e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fde440b4b40 0x7fde44140f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:15.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.507+0000 7fde4ae3e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde44141ea0 con 0x7fde440b4b40 2020-11-28T21:48:15.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.507+0000 7fde4ae3e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde441440d0 con 0x7fde440611e0 2020-11-28T21:48:15.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.507+0000 7fde4ae3e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde441401b0 con 0x7fde440b10f0 2020-11-28T21:48:15.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.507+0000 7fde49e3c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde440611e0 0x7fde44140980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:15.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.507+0000 7fde49e3c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde440611e0 0x7fde44140980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38928/0 (socket says 172.21.15.162:38928) 2020-11-28T21:48:15.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.507+0000 7fde49e3c700 1 -- 172.21.15.162:0/2529382250 learned_addr learned my addr 172.21.15.162:0/2529382250 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:15.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.507+0000 7fde4963b700 1 --2- 172.21.15.162:0/2529382250 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fde440b10f0 0x7fde44144770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:15.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.507+0000 7fde4a63d700 1 --2- 172.21.15.162:0/2529382250 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fde440b4b40 0x7fde44140f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:15.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.507+0000 7fde49e3c700 1 -- 172.21.15.162:0/2529382250 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fde440b10f0 msgr2=0x7fde44144770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:15.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.507+0000 7fde49e3c700 1 --2- 172.21.15.162:0/2529382250 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fde440b10f0 0x7fde44144770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:15.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.507+0000 7fde49e3c700 1 -- 172.21.15.162:0/2529382250 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fde440b4b40 msgr2=0x7fde44140f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:15.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.507+0000 7fde49e3c700 1 --2- 172.21.15.162:0/2529382250 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fde440b4b40 0x7fde44140f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:15.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.507+0000 7fde49e3c700 1 -- 172.21.15.162:0/2529382250 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde441420d0 con 0x7fde440611e0 2020-11-28T21:48:15.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.507+0000 7fde4a63d700 1 --2- 172.21.15.162:0/2529382250 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fde440b4b40 0x7fde44140f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:48:15.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.507+0000 7fde49e3c700 1 --2- 172.21.15.162:0/2529382250 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde440611e0 0x7fde44140980 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7fde400032b0 tx=0x7fde4000a410).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:15.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.507+0000 7fde32ffd700 1 -- 172.21.15.162:0/2529382250 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde4001a940 con 0x7fde440611e0 2020-11-28T21:48:15.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.507+0000 7fde32ffd700 1 -- 172.21.15.162:0/2529382250 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde4001aaa0 con 0x7fde440611e0 2020-11-28T21:48:15.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.507+0000 7fde4ae3e700 1 -- 172.21.15.162:0/2529382250 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde44097c40 con 0x7fde440611e0 2020-11-28T21:48:15.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.507+0000 7fde32ffd700 1 -- 172.21.15.162:0/2529382250 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde4001ae00 con 0x7fde440611e0 2020-11-28T21:48:15.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.507+0000 7fde4ae3e700 1 -- 172.21.15.162:0/2529382250 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde44140540 con 0x7fde440611e0 2020-11-28T21:48:15.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.507+0000 7fde4ae3e700 1 -- 172.21.15.162:0/2529382250 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde441402f0 con 0x7fde440611e0 2020-11-28T21:48:15.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.511+0000 7fde32ffd700 1 -- 172.21.15.162:0/2529382250 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fde4002a070 con 0x7fde440611e0 2020-11-28T21:48:15.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.511+0000 7fde32ffd700 1 --2- 172.21.15.162:0/2529382250 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fde240553b0 0x7fde24057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:15.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.511+0000 7fde32ffd700 1 -- 172.21.15.162:0/2529382250 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fde4006fa80 con 0x7fde440611e0 2020-11-28T21:48:15.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.511+0000 7fde4963b700 1 --2- 172.21.15.162:0/2529382250 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fde240553b0 0x7fde24057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:15.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.511+0000 7fde4963b700 1 --2- 172.21.15.162:0/2529382250 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fde240553b0 0x7fde24057870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7fde3400d000 tx=0x7fde34007b60).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:15.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.519+0000 7fde32ffd700 1 -- 172.21.15.162:0/2529382250 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fde40045ce0 con 0x7fde440611e0 2020-11-28T21:48:15.710 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.711+0000 7fde4ae3e700 1 -- 172.21.15.162:0/2529382250 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command"]} v 0) v1 -- 0x7fde44057050 con 0x7fde440611e0 2020-11-28T21:48:15.838 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.839+0000 7fde32ffd700 1 -- 172.21.15.162:0/2529382250 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command"]}]=0 v93) v1 ==== 139+0+0 (secure 0 0 0) 0x7fde40049300 con 0x7fde440611e0 2020-11-28T21:48:15.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.839+0000 7fde4ae3e700 1 -- 172.21.15.162:0/2529382250 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fde240553b0 msgr2=0x7fde24057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:15.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.839+0000 7fde4ae3e700 1 --2- 172.21.15.162:0/2529382250 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fde240553b0 0x7fde24057870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7fde3400d000 tx=0x7fde34007b60).stop 2020-11-28T21:48:15.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.839+0000 7fde4ae3e700 1 -- 172.21.15.162:0/2529382250 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde440611e0 msgr2=0x7fde44140980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:15.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.839+0000 7fde4ae3e700 1 --2- 172.21.15.162:0/2529382250 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde440611e0 0x7fde44140980 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7fde400032b0 tx=0x7fde4000a410).stop 2020-11-28T21:48:15.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.839+0000 7fde4ae3e700 1 -- 172.21.15.162:0/2529382250 shutdown_connections 2020-11-28T21:48:15.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.839+0000 7fde4ae3e700 1 --2- 172.21.15.162:0/2529382250 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fde240553b0 0x7fde24057870 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:15.843 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.839+0000 7fde4ae3e700 1 --2- 172.21.15.162:0/2529382250 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fde440b4b40 0x7fde44140f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:15.843 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.839+0000 7fde4ae3e700 1 --2- 172.21.15.162:0/2529382250 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde440611e0 0x7fde44140980 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:15.843 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.839+0000 7fde4ae3e700 1 --2- 172.21.15.162:0/2529382250 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fde440b10f0 0x7fde44144770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:15.843 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.839+0000 7fde4ae3e700 1 -- 172.21.15.162:0/2529382250 >> 172.21.15.162:0/2529382250 conn(0x7fde440b54f0 msgr2=0x7fde4413de40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:15.843 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.839+0000 7fde4ae3e700 1 -- 172.21.15.162:0/2529382250 shutdown_connections 2020-11-28T21:48:15.844 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.839+0000 7fde4ae3e700 1 -- 172.21.15.162:0/2529382250 wait complete. 2020-11-28T21:48:15.857 INFO:tasks.cephfs_test_runner:test_drop_cache_command (tasks.cephfs.test_misc.TestCacheDrop) ... ok 2020-11-28T21:48:15.858 INFO:tasks.cephfs_test_runner:Starting test: test_drop_cache_command_dead (tasks.cephfs.test_misc.TestCacheDrop) 2020-11-28T21:48:15.858 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead' 2020-11-28T21:48:15.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.975+0000 7f09e5f77700 1 Processor -- start 2020-11-28T21:48:15.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.975+0000 7f09e5f77700 1 -- start start 2020-11-28T21:48:15.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.979+0000 7f09e5f77700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f09e00aae10 0x7f09e005f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:15.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.979+0000 7f09e5f77700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f09e00ab990 0x7f09e00ab1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:15.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.979+0000 7f09e5f77700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09e00a85c0 con 0x7f09e0140800 2020-11-28T21:48:15.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.979+0000 7f09e5f77700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09e01448e0 con 0x7f09e00aae10 2020-11-28T21:48:15.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.979+0000 7f09e5f77700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e0144b80 con 0x7f09e00ab990 2020-11-28T21:48:15.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.979+0000 7f09e5776700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f09e0140800 0x7f09e0143ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:45780/0 (socket says 172.21.15.162:45780) 2020-11-28T21:48:15.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.979+0000 7f09e5776700 1 -- 172.21.15.162:0/3127050380 learned_addr learned my addr 172.21.15.162:0/3127050380 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:48:15.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.979+0000 7f09dffff700 1 --2- 172.21.15.162:0/3127050380 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f09e00ab990 0x7f09e00ab1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:15.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.979+0000 7f09e4f75700 1 --2- 172.21.15.162:0/3127050380 >> v2:172.21.15.118:3300/0 conn(0x7f09e00aae10 0x7f09e005f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:15.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.979+0000 7f09df7fe700 1 -- 172.21.15.162:0/3127050380 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1280064535 0 0) 0x7f09e00a85c0 con 0x7f09e0140800 2020-11-28T21:48:15.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.979+0000 7f09df7fe700 1 -- 172.21.15.162:0/3127050380 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09c0003230 con 0x7f09e0140800 2020-11-28T21:48:15.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.979+0000 7f09df7fe700 1 -- 172.21.15.162:0/3127050380 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3870204347 0 0) 0x7f09c0003230 con 0x7f09e0140800 2020-11-28T21:48:15.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.979+0000 7f09dffff700 1 -- 172.21.15.162:0/3127050380 >> v2:172.21.15.118:3300/0 conn(0x7f09e00aae10 msgr2=0x7f09e005f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:15.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.979+0000 7f09dffff700 1 --2- 172.21.15.162:0/3127050380 >> v2:172.21.15.118:3300/0 conn(0x7f09e00aae10 0x7f09e005f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:15.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.979+0000 7f09dffff700 1 -- 172.21.15.162:0/3127050380 >> v1:172.21.15.162:6789/0 conn(0x7f09e0140800 legacy=0x7f09e0143ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:15.992 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.979+0000 7f09dffff700 1 -- 172.21.15.162:0/3127050380 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e00af6b0 con 0x7f09e00ab990 2020-11-28T21:48:15.992 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.979+0000 7f09dffff700 1 --2- 172.21.15.162:0/3127050380 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f09e00ab990 0x7f09e00ab1f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f09d8001bf0 tx=0x7f09c8006dc0).ready entity=mon.2 client_cookie=53f38d1d659e3fbd server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:15.992 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.979+0000 7f09df7fe700 1 -- 172.21.15.162:0/3127050380 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09d8001a50 con 0x7f09e00ab990 2020-11-28T21:48:15.993 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.979+0000 7f09df7fe700 1 -- 172.21.15.162:0/3127050380 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c800d570 con 0x7f09e00ab990 2020-11-28T21:48:15.993 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.979+0000 7f09df7fe700 1 -- 172.21.15.162:0/3127050380 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09c800d6e0 con 0x7f09e00ab990 2020-11-28T21:48:15.994 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.979+0000 7f09e5f77700 1 -- 172.21.15.162:0/3127050380 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f09e00ab990 msgr2=0x7f09e00ab1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:15.994 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.979+0000 7f09e5f77700 1 --2- 172.21.15.162:0/3127050380 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f09e00ab990 0x7f09e00ab1f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f09d8001bf0 tx=0x7f09c8006dc0).stop 2020-11-28T21:48:15.994 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.979+0000 7f09e5f77700 1 -- 172.21.15.162:0/3127050380 shutdown_connections 2020-11-28T21:48:15.998 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.979+0000 7f09e5f77700 1 --2- 172.21.15.162:0/3127050380 >> v2:172.21.15.118:3300/0 conn(0x7f09e00aae10 0x7f09e005f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:15.998 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.979+0000 7f09e5f77700 1 --2- 172.21.15.162:0/3127050380 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f09e00ab990 0x7f09e00ab1f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:15.998 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.979+0000 7f09e5f77700 1 -- 172.21.15.162:0/3127050380 >> 172.21.15.162:0/3127050380 conn(0x7f09e005b970 msgr2=0x7f09e005bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:15.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.983+0000 7f09e5f77700 1 -- 172.21.15.162:0/3127050380 shutdown_connections 2020-11-28T21:48:15.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.983+0000 7f09e5f77700 1 -- 172.21.15.162:0/3127050380 wait complete. 2020-11-28T21:48:15.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.983+0000 7f09e5f77700 1 Processor -- start 2020-11-28T21:48:15.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.983+0000 7f09e5f77700 1 -- start start 2020-11-28T21:48:16.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.983+0000 7f09e5f77700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f09e00aae10 0x7f09e00b4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:16.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.983+0000 7f09e5f77700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f09e00ab990 0x7f09e00b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:16.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.983+0000 7f09e5f77700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f09e0140800 0x7f09e014f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:16.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.983+0000 7f09e5f77700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e00b3460 con 0x7f09e0140800 2020-11-28T21:48:16.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.983+0000 7f09e5f77700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e00b2b50 con 0x7f09e00ab990 2020-11-28T21:48:16.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.983+0000 7f09e5f77700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e00b3690 con 0x7f09e00aae10 2020-11-28T21:48:16.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.983+0000 7f09dffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f09e00ab990 0x7f09e00b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:16.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.983+0000 7f09dffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f09e00ab990 0x7f09e00b4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38938/0 (socket says 172.21.15.162:38938) 2020-11-28T21:48:16.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.983+0000 7f09dffff700 1 -- 172.21.15.162:0/3935773215 learned_addr learned my addr 172.21.15.162:0/3935773215 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:16.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.983+0000 7f09e4f75700 1 --2- 172.21.15.162:0/3935773215 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f09e00aae10 0x7f09e00b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:16.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.983+0000 7f09e5776700 1 --2- 172.21.15.162:0/3935773215 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f09e0140800 0x7f09e014f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:16.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.987+0000 7f09dffff700 1 -- 172.21.15.162:0/3935773215 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f09e00aae10 msgr2=0x7f09e00b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:16.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.987+0000 7f09dffff700 1 --2- 172.21.15.162:0/3935773215 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f09e00aae10 0x7f09e00b4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:16.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.987+0000 7f09dffff700 1 -- 172.21.15.162:0/3935773215 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f09e0140800 msgr2=0x7f09e014f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:16.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.987+0000 7f09dffff700 1 --2- 172.21.15.162:0/3935773215 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f09e0140800 0x7f09e014f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:16.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.987+0000 7f09dffff700 1 -- 172.21.15.162:0/3935773215 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e00b3930 con 0x7f09e00ab990 2020-11-28T21:48:16.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.987+0000 7f09dffff700 1 --2- 172.21.15.162:0/3935773215 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f09e00ab990 0x7f09e00b4d30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f09d8001bf0 tx=0x7f09c8015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:16.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.987+0000 7f09ddffb700 1 -- 172.21.15.162:0/3935773215 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09c8011a50 con 0x7f09e00ab990 2020-11-28T21:48:16.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.987+0000 7f09ddffb700 1 -- 172.21.15.162:0/3935773215 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c8011bb0 con 0x7f09e00ab990 2020-11-28T21:48:16.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.987+0000 7f09e5f77700 1 -- 172.21.15.162:0/3935773215 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09e00b2f80 con 0x7f09e00ab990 2020-11-28T21:48:16.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.987+0000 7f09ddffb700 1 -- 172.21.15.162:0/3935773215 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09c801d440 con 0x7f09e00ab990 2020-11-28T21:48:16.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.987+0000 7f09e5f77700 1 -- 172.21.15.162:0/3935773215 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09e0099d90 con 0x7f09e00ab990 2020-11-28T21:48:16.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.987+0000 7f09e5f77700 1 -- 172.21.15.162:0/3935773215 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09e0057050 con 0x7f09e00ab990 2020-11-28T21:48:16.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.987+0000 7f09ddffb700 1 -- 172.21.15.162:0/3935773215 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f09c8024030 con 0x7f09e00ab990 2020-11-28T21:48:16.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.987+0000 7f09ddffb700 1 --2- 172.21.15.162:0/3935773215 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f09c4055330 0x7f09c40577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:16.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.987+0000 7f09ddffb700 1 -- 172.21.15.162:0/3935773215 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f09c806eb90 con 0x7f09e00ab990 2020-11-28T21:48:16.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.987+0000 7f09e4f75700 1 --2- 172.21.15.162:0/3935773215 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f09c4055330 0x7f09c40577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:16.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.991+0000 7f09e4f75700 1 --2- 172.21.15.162:0/3935773215 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f09c4055330 0x7f09c40577f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f09d000b000 tx=0x7f09d0006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:16.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:15.995+0000 7f09ddffb700 1 -- 172.21.15.162:0/3935773215 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f09c8044e90 con 0x7f09e00ab990 2020-11-28T21:48:16.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.179+0000 7f09e5f77700 1 -- 172.21.15.162:0/3935773215 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead"]} v 0) v1 -- 0x7f09e00b2c90 con 0x7f09e00ab990 2020-11-28T21:48:16.840 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.839+0000 7f09ddffb700 1 -- 172.21.15.162:0/3935773215 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead"]}]=0 v94) v1 ==== 147+0+0 (secure 0 0 0) 0x7f09c8022020 con 0x7f09e00ab990 2020-11-28T21:48:16.844 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.843+0000 7f09e5f77700 1 -- 172.21.15.162:0/3935773215 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f09c4055330 msgr2=0x7f09c40577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:16.844 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.843+0000 7f09e5f77700 1 --2- 172.21.15.162:0/3935773215 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f09c4055330 0x7f09c40577f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f09d000b000 tx=0x7f09d0006dc0).stop 2020-11-28T21:48:16.844 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.843+0000 7f09e5f77700 1 -- 172.21.15.162:0/3935773215 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f09e00ab990 msgr2=0x7f09e00b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:16.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.843+0000 7f09e5f77700 1 --2- 172.21.15.162:0/3935773215 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f09e00ab990 0x7f09e00b4d30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f09d8001bf0 tx=0x7f09c8015000).stop 2020-11-28T21:48:16.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.843+0000 7f09e5f77700 1 -- 172.21.15.162:0/3935773215 shutdown_connections 2020-11-28T21:48:16.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.843+0000 7f09e5f77700 1 --2- 172.21.15.162:0/3935773215 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f09c4055330 0x7f09c40577f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:16.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.843+0000 7f09e5f77700 1 --2- 172.21.15.162:0/3935773215 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f09e00aae10 0x7f09e00b4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:16.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.843+0000 7f09e5f77700 1 --2- 172.21.15.162:0/3935773215 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f09e0140800 0x7f09e014f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:16.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.843+0000 7f09e5f77700 1 --2- 172.21.15.162:0/3935773215 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f09e00ab990 0x7f09e00b4d30 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:16.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.843+0000 7f09e5f77700 1 -- 172.21.15.162:0/3935773215 >> 172.21.15.162:0/3935773215 conn(0x7f09e005b970 msgr2=0x7f09e013fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:16.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.843+0000 7f09e5f77700 1 -- 172.21.15.162:0/3935773215 shutdown_connections 2020-11-28T21:48:16.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.843+0000 7f09e5f77700 1 -- 172.21.15.162:0/3935773215 wait complete. 2020-11-28T21:48:16.860 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config set mon mon_allow_pool_delete True 2020-11-28T21:48:16.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.983+0000 7f7141dcb700 1 Processor -- start 2020-11-28T21:48:16.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.983+0000 7f7141dcb700 1 -- start start 2020-11-28T21:48:16.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.983+0000 7f7141dcb700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f713c0b3ab0 0x7f713c05c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:16.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.983+0000 7f7141dcb700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f713c0b30e0 0x7f713c13fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:16.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.983+0000 7f7141dcb700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f713c05baf0 con 0x7f713c0b2d00 2020-11-28T21:48:16.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.983+0000 7f7141dcb700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f713c0b0e20 con 0x7f713c0b3ab0 2020-11-28T21:48:16.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.983+0000 7f7141dcb700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f713c05bd90 con 0x7f713c0b30e0 2020-11-28T21:48:16.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.983+0000 7f7140dc9700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f713c0b2d00 0x7f713c142ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:45792/0 (socket says 172.21.15.162:45792) 2020-11-28T21:48:16.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.983+0000 7f7140dc9700 1 -- 172.21.15.162:0/501509354 learned_addr learned my addr 172.21.15.162:0/501509354 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:48:16.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.983+0000 7f71415ca700 1 --2- 172.21.15.162:0/501509354 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f713c0b30e0 0x7f713c13fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:16.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.983+0000 7f713bfff700 1 --2- 172.21.15.162:0/501509354 >> v2:172.21.15.118:3300/0 conn(0x7f713c0b3ab0 0x7f713c05c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:16.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.983+0000 7f713b7fe700 1 -- 172.21.15.162:0/501509354 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1678166551 0 0) 0x7f713c05baf0 con 0x7f713c0b2d00 2020-11-28T21:48:16.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.983+0000 7f713b7fe700 1 -- 172.21.15.162:0/501509354 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f711c003230 con 0x7f713c0b2d00 2020-11-28T21:48:16.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.983+0000 7f713bfff700 1 -- 172.21.15.162:0/501509354 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f713c0b30e0 msgr2=0x7f713c13fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:16.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.983+0000 7f713bfff700 1 --2- 172.21.15.162:0/501509354 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f713c0b30e0 0x7f713c13fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:16.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.983+0000 7f713bfff700 1 -- 172.21.15.162:0/501509354 >> v1:172.21.15.162:6789/0 conn(0x7f713c0b2d00 legacy=0x7f713c142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:16.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.983+0000 7f713bfff700 1 -- 172.21.15.162:0/501509354 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f713c05c030 con 0x7f713c0b3ab0 2020-11-28T21:48:16.992 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.983+0000 7f71415ca700 1 --2- 172.21.15.162:0/501509354 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f713c0b30e0 0x7f713c13fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:48:16.992 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.987+0000 7f713bfff700 1 --2- 172.21.15.162:0/501509354 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f713c0b3ab0 0x7f713c05c4d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f712c002f30 tx=0x7f712c006dc0).ready entity=mon.0 client_cookie=132dcf05fc2e3f0f server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:16.992 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.987+0000 7f713b7fe700 1 -- 172.21.15.162:0/501509354 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f712c0096a0 con 0x7f713c0b3ab0 2020-11-28T21:48:16.992 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.987+0000 7f713b7fe700 1 -- 172.21.15.162:0/501509354 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712c009800 con 0x7f713c0b3ab0 2020-11-28T21:48:16.993 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.987+0000 7f713b7fe700 1 -- 172.21.15.162:0/501509354 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f712c009b60 con 0x7f713c0b3ab0 2020-11-28T21:48:16.993 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.987+0000 7f7141dcb700 1 -- 172.21.15.162:0/501509354 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f713c0b3ab0 msgr2=0x7f713c05c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:16.993 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.987+0000 7f7141dcb700 1 --2- 172.21.15.162:0/501509354 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f713c0b3ab0 0x7f713c05c4d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f712c002f30 tx=0x7f712c006dc0).stop 2020-11-28T21:48:16.997 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.987+0000 7f7141dcb700 1 -- 172.21.15.162:0/501509354 shutdown_connections 2020-11-28T21:48:16.998 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.987+0000 7f7141dcb700 1 --2- 172.21.15.162:0/501509354 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f713c0b30e0 0x7f713c13fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:16.998 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.987+0000 7f7141dcb700 1 --2- 172.21.15.162:0/501509354 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f713c0b3ab0 0x7f713c05c4d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:16.998 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.987+0000 7f7141dcb700 1 -- 172.21.15.162:0/501509354 >> 172.21.15.162:0/501509354 conn(0x7f713c0604c0 msgr2=0x7f713c0608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:16.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.987+0000 7f7141dcb700 1 -- 172.21.15.162:0/501509354 shutdown_connections 2020-11-28T21:48:16.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.987+0000 7f7141dcb700 1 -- 172.21.15.162:0/501509354 wait complete. 2020-11-28T21:48:17.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.987+0000 7f7141dcb700 1 Processor -- start 2020-11-28T21:48:17.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.987+0000 7f7141dcb700 1 -- start start 2020-11-28T21:48:17.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.991+0000 7f7141dcb700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f713c0b2d00 0x7f713c13d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:17.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.991+0000 7f7141dcb700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f713c0b30e0 0x7f713c145a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:17.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.991+0000 7f7141dcb700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f713c0b3ab0 0x7f713c145f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:17.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.991+0000 7f7141dcb700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713c0b6620 con 0x7f713c0b30e0 2020-11-28T21:48:17.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.991+0000 7f7141dcb700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713c0b9f70 con 0x7f713c0b3ab0 2020-11-28T21:48:17.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.991+0000 7f7141dcb700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f713c0b9cd0 con 0x7f713c0b2d00 2020-11-28T21:48:17.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.991+0000 7f71415ca700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f713c0b3ab0 0x7f713c145f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:17.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.991+0000 7f71415ca700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f713c0b3ab0 0x7f713c145f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38956/0 (socket says 172.21.15.162:38956) 2020-11-28T21:48:17.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.991+0000 7f71415ca700 1 -- 172.21.15.162:0/933121312 learned_addr learned my addr 172.21.15.162:0/933121312 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:17.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.991+0000 7f713bfff700 1 --2- 172.21.15.162:0/933121312 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f713c0b30e0 0x7f713c145a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:17.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.991+0000 7f7140dc9700 1 --2- 172.21.15.162:0/933121312 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f713c0b2d00 0x7f713c13d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:17.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.991+0000 7f71415ca700 1 -- 172.21.15.162:0/933121312 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f713c0b2d00 msgr2=0x7f713c13d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:17.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.991+0000 7f71415ca700 1 --2- 172.21.15.162:0/933121312 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f713c0b2d00 0x7f713c13d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:17.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.991+0000 7f71415ca700 1 -- 172.21.15.162:0/933121312 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f713c0b30e0 msgr2=0x7f713c145a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:17.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.991+0000 7f71415ca700 1 --2- 172.21.15.162:0/933121312 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f713c0b30e0 0x7f713c145a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:17.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.991+0000 7f71415ca700 1 -- 172.21.15.162:0/933121312 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f713c0b8b40 con 0x7f713c0b3ab0 2020-11-28T21:48:17.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.991+0000 7f71415ca700 1 --2- 172.21.15.162:0/933121312 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f713c0b3ab0 0x7f713c145f80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f713400c000 tx=0x7f71340091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:17.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.991+0000 7f7139ffb700 1 -- 172.21.15.162:0/933121312 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f713400cc80 con 0x7f713c0b3ab0 2020-11-28T21:48:17.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.991+0000 7f7141dcb700 1 -- 172.21.15.162:0/933121312 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f713c0b8de0 con 0x7f713c0b3ab0 2020-11-28T21:48:17.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.991+0000 7f7139ffb700 1 -- 172.21.15.162:0/933121312 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f713400cde0 con 0x7f713c0b3ab0 2020-11-28T21:48:17.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.991+0000 7f7139ffb700 1 -- 172.21.15.162:0/933121312 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71340125e0 con 0x7f713c0b3ab0 2020-11-28T21:48:17.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.991+0000 7f7141dcb700 1 -- 172.21.15.162:0/933121312 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f713c0b9930 con 0x7f713c0b3ab0 2020-11-28T21:48:17.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.991+0000 7f7141dcb700 1 -- 172.21.15.162:0/933121312 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f713c057050 con 0x7f713c0b3ab0 2020-11-28T21:48:17.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.991+0000 7f7139ffb700 1 -- 172.21.15.162:0/933121312 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7134022070 con 0x7f713c0b3ab0 2020-11-28T21:48:17.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.991+0000 7f7139ffb700 1 --2- 172.21.15.162:0/933121312 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7120055330 0x7f71200577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:17.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.991+0000 7f7139ffb700 1 -- 172.21.15.162:0/933121312 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f713406e9e0 con 0x7f713c0b3ab0 2020-11-28T21:48:17.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.991+0000 7f7140dc9700 1 --2- 172.21.15.162:0/933121312 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7120055330 0x7f71200577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:17.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.995+0000 7f7140dc9700 1 --2- 172.21.15.162:0/933121312 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7120055330 0x7f71200577f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f712400b000 tx=0x7f7124009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:17.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:16.999+0000 7f7139ffb700 1 -- 172.21.15.162:0/933121312 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7134044c40 con 0x7f713c0b3ab0 2020-11-28T21:48:17.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.143+0000 7f7141dcb700 1 -- 172.21.15.162:0/933121312 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7f713c0b8f70 con 0x7f713c0b3ab0 2020-11-28T21:48:17.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.151+0000 7f7139ffb700 1 -- 172.21.15.162:0/933121312 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v4) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7134016070 con 0x7f713c0b3ab0 2020-11-28T21:48:17.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.151+0000 7f7141dcb700 1 -- 172.21.15.162:0/933121312 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7120055330 msgr2=0x7f71200577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:17.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.151+0000 7f7141dcb700 1 --2- 172.21.15.162:0/933121312 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7120055330 0x7f71200577f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f712400b000 tx=0x7f7124009040).stop 2020-11-28T21:48:17.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.151+0000 7f7141dcb700 1 -- 172.21.15.162:0/933121312 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f713c0b3ab0 msgr2=0x7f713c145f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:17.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.151+0000 7f7141dcb700 1 --2- 172.21.15.162:0/933121312 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f713c0b3ab0 0x7f713c145f80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f713400c000 tx=0x7f71340091a0).stop 2020-11-28T21:48:17.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.151+0000 7f7141dcb700 1 -- 172.21.15.162:0/933121312 shutdown_connections 2020-11-28T21:48:17.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.151+0000 7f7141dcb700 1 --2- 172.21.15.162:0/933121312 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7120055330 0x7f71200577f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:17.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.151+0000 7f7141dcb700 1 --2- 172.21.15.162:0/933121312 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f713c0b2d00 0x7f713c13d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:17.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.151+0000 7f7141dcb700 1 --2- 172.21.15.162:0/933121312 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f713c0b3ab0 0x7f713c145f80 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:17.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.151+0000 7f7141dcb700 1 --2- 172.21.15.162:0/933121312 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f713c0b30e0 0x7f713c145a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:17.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.151+0000 7f7141dcb700 1 -- 172.21.15.162:0/933121312 >> 172.21.15.162:0/933121312 conn(0x7f713c0604c0 msgr2=0x7f713c0b58a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:17.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.151+0000 7f7141dcb700 1 -- 172.21.15.162:0/933121312 shutdown_connections 2020-11-28T21:48:17.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.151+0000 7f7141dcb700 1 -- 172.21.15.162:0/933121312 wait complete. 2020-11-28T21:48:17.165 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-28T21:48:17.166 INFO:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:48:17.184 INFO:teuthology.orchestra.run.smithi118:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:48:17.203 INFO:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:48:17.221 INFO:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:48:17.240 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-28T21:48:17.240 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:48:17.241 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:48:17.241 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:48:17.290 INFO:teuthology.orchestra.run.smithi192.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2020-11-28T21:48:17.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:48:17.291 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2020-11-28T21:48:17.292 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:48:17.292 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:48:17.293 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:48:17.299 INFO:teuthology.orchestra.run.smithi096.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-11-28T21:48:17.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:48:17.301 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2020-11-28T21:48:17.301 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2020-11-28T21:48:17.419 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.419+0000 7fd36d686700 1 Processor -- start 2020-11-28T21:48:17.420 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.419+0000 7fd36d686700 1 -- start start 2020-11-28T21:48:17.421 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.419+0000 7fd36d686700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd368060d10 0x7fd3680b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:17.421 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.419+0000 7fd36d686700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd3680b60e0 0x7fd368146180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:17.421 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.419+0000 7fd36d686700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3680b4930 con 0x7fd3680b4390 2020-11-28T21:48:17.421 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.419+0000 7fd36d686700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3680b2170 con 0x7fd3680b60e0 2020-11-28T21:48:17.422 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.419+0000 7fd36d686700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3680b6660 con 0x7fd368060d10 2020-11-28T21:48:17.422 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.419+0000 7fd3677fe700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fd3680b4390 0x7fd36813fc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:45806/0 (socket says 172.21.15.162:45806) 2020-11-28T21:48:17.422 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.419+0000 7fd3677fe700 1 -- 172.21.15.162:0/425857471 learned_addr learned my addr 172.21.15.162:0/425857471 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:48:17.424 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.419+0000 7fd366ffd700 1 --2- 172.21.15.162:0/425857471 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd368060d10 0x7fd3680b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:17.424 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.419+0000 7fd3667fc700 1 --2- 172.21.15.162:0/425857471 >> v2:172.21.15.118:3300/0 conn(0x7fd3680b60e0 0x7fd368146180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:17.424 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.419+0000 7fd365ffb700 1 -- 172.21.15.162:0/425857471 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 380298748 0 0) 0x7fd3680b4930 con 0x7fd3680b4390 2020-11-28T21:48:17.424 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.419+0000 7fd365ffb700 1 -- 172.21.15.162:0/425857471 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd348003230 con 0x7fd3680b4390 2020-11-28T21:48:17.425 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.423+0000 7fd366ffd700 1 -- 172.21.15.162:0/425857471 >> v2:172.21.15.118:3300/0 conn(0x7fd3680b60e0 msgr2=0x7fd368146180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:17.425 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.423+0000 7fd366ffd700 1 --2- 172.21.15.162:0/425857471 >> v2:172.21.15.118:3300/0 conn(0x7fd3680b60e0 0x7fd368146180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:17.425 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.423+0000 7fd366ffd700 1 -- 172.21.15.162:0/425857471 >> v1:172.21.15.162:6789/0 conn(0x7fd3680b4390 legacy=0x7fd36813fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:17.425 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.423+0000 7fd366ffd700 1 -- 172.21.15.162:0/425857471 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3680b4b60 con 0x7fd368060d10 2020-11-28T21:48:17.425 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.423+0000 7fd366ffd700 1 --2- 172.21.15.162:0/425857471 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd368060d10 0x7fd3680b52a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fd350002f30 tx=0x7fd350006dc0).ready entity=mon.2 client_cookie=65540a5b4b993f7c server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:17.426 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.423+0000 7fd365ffb700 1 -- 172.21.15.162:0/425857471 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3500096a0 con 0x7fd368060d10 2020-11-28T21:48:17.426 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.423+0000 7fd365ffb700 1 -- 172.21.15.162:0/425857471 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd350009800 con 0x7fd368060d10 2020-11-28T21:48:17.426 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.423+0000 7fd365ffb700 1 -- 172.21.15.162:0/425857471 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd350009b60 con 0x7fd368060d10 2020-11-28T21:48:17.426 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.423+0000 7fd36d686700 1 -- 172.21.15.162:0/425857471 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd368060d10 msgr2=0x7fd3680b52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:17.427 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.423+0000 7fd36d686700 1 --2- 172.21.15.162:0/425857471 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd368060d10 0x7fd3680b52a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fd350002f30 tx=0x7fd350006dc0).stop 2020-11-28T21:48:17.427 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.423+0000 7fd36d686700 1 -- 172.21.15.162:0/425857471 shutdown_connections 2020-11-28T21:48:17.427 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.423+0000 7fd36d686700 1 --2- 172.21.15.162:0/425857471 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd368060d10 0x7fd3680b52a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:17.427 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.423+0000 7fd36d686700 1 --2- 172.21.15.162:0/425857471 >> v2:172.21.15.118:3300/0 conn(0x7fd3680b60e0 0x7fd368146180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:17.427 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.423+0000 7fd36d686700 1 -- 172.21.15.162:0/425857471 >> 172.21.15.162:0/425857471 conn(0x7fd36805fe70 msgr2=0x7fd36806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:17.428 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.423+0000 7fd36d686700 1 -- 172.21.15.162:0/425857471 shutdown_connections 2020-11-28T21:48:17.428 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.423+0000 7fd36d686700 1 -- 172.21.15.162:0/425857471 wait complete. 2020-11-28T21:48:17.430 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.423+0000 7fd36d686700 1 Processor -- start 2020-11-28T21:48:17.430 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.423+0000 7fd36d686700 1 -- start start 2020-11-28T21:48:17.430 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.427+0000 7fd36d686700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd368060d10 0x7fd368149ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:17.430 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.427+0000 7fd36d686700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd3680b4390 0x7fd368155a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:17.431 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.427+0000 7fd36d686700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3680b60e0 0x7fd368151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:17.431 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.427+0000 7fd36d686700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3681550c0 con 0x7fd368060d10 2020-11-28T21:48:17.431 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.427+0000 7fd36d686700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3680b97f0 con 0x7fd3680b60e0 2020-11-28T21:48:17.431 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.427+0000 7fd36d686700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3681552f0 con 0x7fd3680b4390 2020-11-28T21:48:17.431 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.427+0000 7fd3677fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3680b60e0 0x7fd368151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:17.432 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.427+0000 7fd3677fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3680b60e0 0x7fd368151ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38966/0 (socket says 172.21.15.162:38966) 2020-11-28T21:48:17.432 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.427+0000 7fd3677fe700 1 -- 172.21.15.162:0/108415902 learned_addr learned my addr 172.21.15.162:0/108415902 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:17.432 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.427+0000 7fd366ffd700 1 --2- 172.21.15.162:0/108415902 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd368060d10 0x7fd368149ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:17.432 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.427+0000 7fd3667fc700 1 --2- 172.21.15.162:0/108415902 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd3680b4390 0x7fd368155a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:17.433 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.427+0000 7fd3677fe700 1 -- 172.21.15.162:0/108415902 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd3680b4390 msgr2=0x7fd368155a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:17.433 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.427+0000 7fd3677fe700 1 --2- 172.21.15.162:0/108415902 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd3680b4390 0x7fd368155a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:17.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.427+0000 7fd3677fe700 1 -- 172.21.15.162:0/108415902 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd368060d10 msgr2=0x7fd368149ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:17.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.427+0000 7fd3677fe700 1 --2- 172.21.15.162:0/108415902 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd368060d10 0x7fd368149ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:17.435 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.427+0000 7fd3677fe700 1 -- 172.21.15.162:0/108415902 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd368155590 con 0x7fd3680b60e0 2020-11-28T21:48:17.435 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.427+0000 7fd3677fe700 1 --2- 172.21.15.162:0/108415902 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3680b60e0 0x7fd368151ac0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7fd360001450 tx=0x7fd360007dd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:17.435 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.427+0000 7fd35ffff700 1 -- 172.21.15.162:0/108415902 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd360005b70 con 0x7fd3680b60e0 2020-11-28T21:48:17.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.427+0000 7fd35ffff700 1 -- 172.21.15.162:0/108415902 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd360005cd0 con 0x7fd3680b60e0 2020-11-28T21:48:17.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.427+0000 7fd36d686700 1 -- 172.21.15.162:0/108415902 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd368155f20 con 0x7fd3680b60e0 2020-11-28T21:48:17.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.427+0000 7fd35ffff700 1 -- 172.21.15.162:0/108415902 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd360007300 con 0x7fd3680b60e0 2020-11-28T21:48:17.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.427+0000 7fd36d686700 1 -- 172.21.15.162:0/108415902 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3680b86b0 con 0x7fd3680b60e0 2020-11-28T21:48:17.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.427+0000 7fd36d686700 1 -- 172.21.15.162:0/108415902 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd368057050 con 0x7fd3680b60e0 2020-11-28T21:48:17.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.427+0000 7fd35ffff700 1 -- 172.21.15.162:0/108415902 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd360010030 con 0x7fd3680b60e0 2020-11-28T21:48:17.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.431+0000 7fd35ffff700 1 --2- 172.21.15.162:0/108415902 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd34c0554e0 0x7fd34c0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:17.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.431+0000 7fd35ffff700 1 -- 172.21.15.162:0/108415902 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd36006e090 con 0x7fd3680b60e0 2020-11-28T21:48:17.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.431+0000 7fd366ffd700 1 --2- 172.21.15.162:0/108415902 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd34c0554e0 0x7fd34c0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:17.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.431+0000 7fd366ffd700 1 --2- 172.21.15.162:0/108415902 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd34c0554e0 0x7fd34c0579a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fd35001b000 tx=0x7fd350015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:17.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.439+0000 7fd35ffff700 1 -- 172.21.15.162:0/108415902 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd360044280 con 0x7fd3680b60e0 2020-11-28T21:48:17.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.579+0000 7fd36d686700 1 -- 172.21.15.162:0/108415902 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fd368044350 con 0x7fd3680b60e0 2020-11-28T21:48:17.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.583+0000 7fd35ffff700 1 -- 172.21.15.162:0/108415902 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v33) v1 ==== 81+0+33741 (secure 0 0 0) 0x7fd3600478a0 con 0x7fd3680b60e0 2020-11-28T21:48:17.584 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:48:17.584 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2020-11-28T21:48:17.584 INFO:teuthology.orchestra.run.smithi162.stdout: "epoch": 33, 2020-11-28T21:48:17.584 INFO:teuthology.orchestra.run.smithi162.stdout: "fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", 2020-11-28T21:48:17.585 INFO:teuthology.orchestra.run.smithi162.stdout: "created": "2020-11-28T21:46:11.724372+0000", 2020-11-28T21:48:17.585 INFO:teuthology.orchestra.run.smithi162.stdout: "modified": "2020-11-28T21:47:11.334213+0000", 2020-11-28T21:48:17.585 INFO:teuthology.orchestra.run.smithi162.stdout: "last_up_change": "2020-11-28T21:46:19.591650+0000", 2020-11-28T21:48:17.585 INFO:teuthology.orchestra.run.smithi162.stdout: "last_in_change": "2020-11-28T21:46:19.591650+0000", 2020-11-28T21:48:17.586 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-11-28T21:48:17.586 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_num": 5799936, 2020-11-28T21:48:17.586 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_set": [ 2020-11-28T21:48:17.586 INFO:teuthology.orchestra.run.smithi162.stdout: "pglog_hardlimit", 2020-11-28T21:48:17.586 INFO:teuthology.orchestra.run.smithi162.stdout: "purged_snapdirs", 2020-11-28T21:48:17.587 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_deletes", 2020-11-28T21:48:17.587 INFO:teuthology.orchestra.run.smithi162.stdout: "sortbitwise" 2020-11-28T21:48:17.587 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:48:17.587 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_version": 4, 2020-11-28T21:48:17.587 INFO:teuthology.orchestra.run.smithi162.stdout: "full_ratio": 0.94999998807907104, 2020-11-28T21:48:17.588 INFO:teuthology.orchestra.run.smithi162.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-11-28T21:48:17.588 INFO:teuthology.orchestra.run.smithi162.stdout: "nearfull_ratio": 0.85000002384185791, 2020-11-28T21:48:17.588 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_snapshot": "", 2020-11-28T21:48:17.588 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_max": 6, 2020-11-28T21:48:17.588 INFO:teuthology.orchestra.run.smithi162.stdout: "max_osd": 8, 2020-11-28T21:48:17.589 INFO:teuthology.orchestra.run.smithi162.stdout: "require_min_compat_client": "jewel", 2020-11-28T21:48:17.589 INFO:teuthology.orchestra.run.smithi162.stdout: "min_compat_client": "jewel", 2020-11-28T21:48:17.589 INFO:teuthology.orchestra.run.smithi162.stdout: "require_osd_release": "octopus", 2020-11-28T21:48:17.589 INFO:teuthology.orchestra.run.smithi162.stdout: "pools": [ 2020-11-28T21:48:17.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.590 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 1, 2020-11-28T21:48:17.590 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "device_health_metrics", 2020-11-28T21:48:17.590 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:46:15.639551+0000", 2020-11-28T21:48:17.590 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:48:17.591 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:48:17.591 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:48:17.591 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:48:17.591 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:48:17.592 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:48:17.592 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:48:17.592 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:48:17.592 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 1, 2020-11-28T21:48:17.593 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 1, 2020-11-28T21:48:17.593 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 1, 2020-11-28T21:48:17.596 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 1, 2020-11-28T21:48:17.596 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 1, 2020-11-28T21:48:17.596 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:48:17.597 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:48:17.597 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:48:17.597 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:48:17.597 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:48:17.597 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:48:17.598 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:48:17.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.598 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "16", 2020-11-28T21:48:17.598 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:48:17.599 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:48:17.599 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:48:17.599 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:48:17.599 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:48:17.599 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:48:17.600 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:48:17.600 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:48:17.600 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:48:17.600 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:48:17.600 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:48:17.601 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:48:17.601 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:48:17.601 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:48:17.601 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:48:17.601 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:48:17.602 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:48:17.602 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:48:17.602 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:48:17.602 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:48:17.602 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:48:17.603 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:48:17.603 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:48:17.603 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:48:17.603 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:48:17.604 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:48:17.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.604 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:48:17.604 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:48:17.604 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:48:17.605 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:48:17.605 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:48:17.605 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:48:17.605 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:48:17.605 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:48:17.606 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:48:17.606 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:48:17.606 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:48:17.606 INFO:teuthology.orchestra.run.smithi162.stdout: "options": { 2020-11-28T21:48:17.606 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_min": 1 2020-11-28T21:48:17.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.607 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:48:17.607 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr_devicehealth": {} 2020-11-28T21:48:17.607 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:17.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.608 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 2, 2020-11-28T21:48:17.608 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "rbd", 2020-11-28T21:48:17.608 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:46:23.901366+0000", 2020-11-28T21:48:17.608 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:48:17.609 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:48:17.609 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:48:17.609 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:48:17.609 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:48:17.609 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:48:17.610 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:48:17.610 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:48:17.610 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2020-11-28T21:48:17.610 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2020-11-28T21:48:17.611 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2020-11-28T21:48:17.611 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2020-11-28T21:48:17.611 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2020-11-28T21:48:17.611 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:48:17.611 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:48:17.612 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:48:17.612 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:48:17.612 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:48:17.612 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:48:17.612 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:48:17.613 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.613 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "19", 2020-11-28T21:48:17.613 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:48:17.613 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:48:17.613 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:48:17.614 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:48:17.614 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:48:17.614 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:48:17.614 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:48:17.614 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:48:17.615 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:48:17.615 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:48:17.615 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:48:17.615 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:48:17.616 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:48:17.616 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:48:17.616 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:48:17.616 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:48:17.616 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:48:17.617 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:48:17.617 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:48:17.617 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:48:17.617 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:48:17.617 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:48:17.618 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:48:17.618 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:48:17.618 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:48:17.618 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:48:17.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.619 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:48:17.619 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:48:17.619 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:48:17.619 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:48:17.620 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:48:17.620 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:48:17.620 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:48:17.620 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:48:17.620 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:48:17.621 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:48:17.621 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:48:17.621 INFO:teuthology.orchestra.run.smithi162.stdout: "options": {}, 2020-11-28T21:48:17.621 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:48:17.621 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd": {} 2020-11-28T21:48:17.622 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:17.622 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.622 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.622 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 5, 2020-11-28T21:48:17.622 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "cephfs_metadata", 2020-11-28T21:48:17.623 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:47:08.251301+0000", 2020-11-28T21:48:17.623 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:48:17.623 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:48:17.623 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:48:17.624 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:48:17.624 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:48:17.624 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:48:17.624 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:48:17.624 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:48:17.625 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2020-11-28T21:48:17.625 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2020-11-28T21:48:17.625 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2020-11-28T21:48:17.625 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2020-11-28T21:48:17.625 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2020-11-28T21:48:17.626 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:48:17.626 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:48:17.626 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:48:17.626 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:48:17.626 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:48:17.627 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:48:17.627 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:48:17.627 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.627 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "32", 2020-11-28T21:48:17.627 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:48:17.628 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:48:17.628 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:48:17.628 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:48:17.628 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:48:17.629 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:48:17.629 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:48:17.629 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:48:17.629 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:48:17.629 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:48:17.630 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:48:17.630 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:48:17.630 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:48:17.630 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:48:17.630 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:48:17.631 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:48:17.631 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:48:17.631 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:48:17.631 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:48:17.631 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:48:17.632 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:48:17.632 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:48:17.632 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:48:17.632 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:48:17.632 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:48:17.633 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:48:17.633 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.633 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:48:17.633 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:48:17.633 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:48:17.634 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:48:17.634 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:48:17.634 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:48:17.634 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:48:17.634 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:48:17.635 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:48:17.635 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:48:17.635 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:48:17.635 INFO:teuthology.orchestra.run.smithi162.stdout: "options": { 2020-11-28T21:48:17.635 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_bias": 4, 2020-11-28T21:48:17.636 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_min": 16, 2020-11-28T21:48:17.636 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_priority": 5 2020-11-28T21:48:17.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.636 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:48:17.637 INFO:teuthology.orchestra.run.smithi162.stdout: "cephfs": { 2020-11-28T21:48:17.637 INFO:teuthology.orchestra.run.smithi162.stdout: "metadata": "cephfs" 2020-11-28T21:48:17.637 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:17.637 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:17.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.638 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 6, 2020-11-28T21:48:17.638 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "cephfs_data", 2020-11-28T21:48:17.638 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:47:09.333148+0000", 2020-11-28T21:48:17.638 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:48:17.639 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:48:17.639 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:48:17.639 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:48:17.639 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:48:17.639 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:48:17.640 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:48:17.640 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:48:17.640 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2020-11-28T21:48:17.640 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2020-11-28T21:48:17.640 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2020-11-28T21:48:17.641 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2020-11-28T21:48:17.641 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2020-11-28T21:48:17.641 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:48:17.641 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:48:17.641 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:48:17.642 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:48:17.642 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:48:17.642 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:48:17.642 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:48:17.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.643 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "33", 2020-11-28T21:48:17.643 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:48:17.643 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:48:17.643 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:48:17.644 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:48:17.644 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:48:17.644 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:48:17.644 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:48:17.644 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:48:17.645 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:48:17.645 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:48:17.645 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:48:17.645 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:48:17.645 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:48:17.646 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:48:17.646 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:48:17.646 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:48:17.646 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:48:17.646 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:48:17.647 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:48:17.647 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:48:17.647 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:48:17.647 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:48:17.647 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:48:17.648 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:48:17.648 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:48:17.648 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:48:17.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.649 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:48:17.649 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:48:17.649 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:48:17.649 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:48:17.649 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:48:17.650 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:48:17.650 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:48:17.650 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:48:17.650 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:48:17.650 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:48:17.651 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:48:17.651 INFO:teuthology.orchestra.run.smithi162.stdout: "options": {}, 2020-11-28T21:48:17.651 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:48:17.651 INFO:teuthology.orchestra.run.smithi162.stdout: "cephfs": { 2020-11-28T21:48:17.651 INFO:teuthology.orchestra.run.smithi162.stdout: "data": "cephfs" 2020-11-28T21:48:17.652 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:17.652 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:17.652 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:17.652 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:48:17.652 INFO:teuthology.orchestra.run.smithi162.stdout: "osds": [ 2020-11-28T21:48:17.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.653 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 0, 2020-11-28T21:48:17.653 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "99c7d9ae-adfa-4fd7-880a-23e6baf3281b", 2020-11-28T21:48:17.653 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:48:17.653 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:48:17.654 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:48:17.654 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:48:17.654 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:48:17.654 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:48:17.655 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:48:17.655 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 31, 2020-11-28T21:48:17.655 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:48:17.655 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:48:17.655 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:48:17.656 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:48:17.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.656 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:48:17.656 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6810", 2020-11-28T21:48:17.656 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:48:17.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.657 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:48:17.657 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6811", 2020-11-28T21:48:17.657 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:48:17.658 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:17.658 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:48:17.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.658 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:48:17.658 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:48:17.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.659 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:48:17.659 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6812", 2020-11-28T21:48:17.659 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:48:17.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.660 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:48:17.660 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6813", 2020-11-28T21:48:17.660 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:48:17.660 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:17.661 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:48:17.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.661 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:48:17.661 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:48:17.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.662 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:48:17.662 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6816", 2020-11-28T21:48:17.662 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:48:17.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.663 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:48:17.663 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6817", 2020-11-28T21:48:17.663 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:48:17.663 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:17.663 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:48:17.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.664 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:48:17.664 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:48:17.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.665 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:48:17.665 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6814", 2020-11-28T21:48:17.665 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:48:17.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.666 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:48:17.666 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6815", 2020-11-28T21:48:17.666 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:48:17.666 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:17.666 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:48:17.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.667 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6811/12313", 2020-11-28T21:48:17.667 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6813/12313", 2020-11-28T21:48:17.667 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6817/12313", 2020-11-28T21:48:17.667 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6815/12313", 2020-11-28T21:48:17.668 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:48:17.668 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:48:17.668 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:48:17.668 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:48:17.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.669 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 1, 2020-11-28T21:48:17.669 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "6ce6220a-00ef-4d13-abba-56f87636fb5f", 2020-11-28T21:48:17.669 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:48:17.669 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:48:17.670 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:48:17.670 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:48:17.670 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:48:17.670 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:48:17.670 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:48:17.671 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 14, 2020-11-28T21:48:17.671 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:48:17.671 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:48:17.671 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:48:17.671 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:48:17.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.672 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:48:17.672 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6802", 2020-11-28T21:48:17.672 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:48:17.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.673 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:48:17.673 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6803", 2020-11-28T21:48:17.673 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:48:17.673 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:17.674 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:48:17.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.674 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:48:17.674 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:48:17.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.675 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:48:17.675 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6804", 2020-11-28T21:48:17.675 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:48:17.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.676 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:48:17.676 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6805", 2020-11-28T21:48:17.676 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:48:17.676 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:17.677 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:48:17.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.677 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:48:17.677 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:48:17.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.678 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:48:17.678 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6808", 2020-11-28T21:48:17.678 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:48:17.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.679 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:48:17.679 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6809", 2020-11-28T21:48:17.679 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:48:17.679 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:17.680 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:48:17.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.680 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:48:17.680 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:48:17.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.681 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:48:17.681 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6806", 2020-11-28T21:48:17.681 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:48:17.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.682 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:48:17.682 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6807", 2020-11-28T21:48:17.682 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:48:17.682 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:17.683 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:48:17.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.683 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6803/12311", 2020-11-28T21:48:17.683 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6805/12311", 2020-11-28T21:48:17.684 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6809/12311", 2020-11-28T21:48:17.684 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6807/12311", 2020-11-28T21:48:17.684 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:48:17.684 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:48:17.684 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:48:17.685 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:48:17.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.685 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 2, 2020-11-28T21:48:17.685 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "db4e510d-27c1-47fc-a902-8b32e7dfd73e", 2020-11-28T21:48:17.686 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:48:17.686 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:48:17.686 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:48:17.686 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:48:17.686 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:48:17.687 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:48:17.687 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:48:17.687 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 21, 2020-11-28T21:48:17.687 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:48:17.687 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:48:17.688 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:48:17.688 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:48:17.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.688 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:48:17.689 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6818", 2020-11-28T21:48:17.689 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:48:17.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.689 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:48:17.690 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6819", 2020-11-28T21:48:17.690 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:48:17.690 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:17.690 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:48:17.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.691 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:48:17.691 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:48:17.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.691 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:48:17.691 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6820", 2020-11-28T21:48:17.692 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:48:17.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.692 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:48:17.693 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6821", 2020-11-28T21:48:17.693 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:48:17.693 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:17.693 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:48:17.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.694 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:48:17.694 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:48:17.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.694 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:48:17.694 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6824", 2020-11-28T21:48:17.695 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:48:17.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.695 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:48:17.695 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6825", 2020-11-28T21:48:17.696 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:48:17.696 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:17.696 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:48:17.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.696 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:48:17.697 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:48:17.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.697 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:48:17.697 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6822", 2020-11-28T21:48:17.697 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:48:17.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.698 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:48:17.698 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6823", 2020-11-28T21:48:17.699 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:48:17.699 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:17.699 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:48:17.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.699 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6819/12312", 2020-11-28T21:48:17.700 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6821/12312", 2020-11-28T21:48:17.700 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6825/12312", 2020-11-28T21:48:17.700 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6823/12312", 2020-11-28T21:48:17.700 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:48:17.700 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:48:17.701 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:48:17.701 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:48:17.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.701 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 3, 2020-11-28T21:48:17.702 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "4094cb87-f04a-45e1-8710-fb4b651949f6", 2020-11-28T21:48:17.702 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:48:17.702 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:48:17.702 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:48:17.702 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:48:17.703 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:48:17.703 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:48:17.703 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:48:17.703 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 31, 2020-11-28T21:48:17.704 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:48:17.704 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:48:17.704 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:48:17.704 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:48:17.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.705 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:48:17.705 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6826", 2020-11-28T21:48:17.705 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:48:17.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.706 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:48:17.706 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6827", 2020-11-28T21:48:17.706 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:48:17.706 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:17.706 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:48:17.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.707 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:48:17.707 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:48:17.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.708 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:48:17.708 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6828", 2020-11-28T21:48:17.708 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:48:17.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.709 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:48:17.709 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6829", 2020-11-28T21:48:17.709 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:48:17.709 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:17.709 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:48:17.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.710 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:48:17.710 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:48:17.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.710 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:48:17.711 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6832", 2020-11-28T21:48:17.711 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:48:17.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.712 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:48:17.712 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6833", 2020-11-28T21:48:17.712 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:48:17.712 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:17.712 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:48:17.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.713 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:48:17.713 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:48:17.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.713 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:48:17.714 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6830", 2020-11-28T21:48:17.714 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:48:17.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.714 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:48:17.715 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6831", 2020-11-28T21:48:17.715 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:48:17.715 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:17.715 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:48:17.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.716 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6827/12323", 2020-11-28T21:48:17.716 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6829/12323", 2020-11-28T21:48:17.716 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6833/12323", 2020-11-28T21:48:17.716 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6831/12323", 2020-11-28T21:48:17.717 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:48:17.717 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:48:17.717 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:48:17.717 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:48:17.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.718 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 4, 2020-11-28T21:48:17.718 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "b907bf53-c297-4756-847b-3be96680eef5", 2020-11-28T21:48:17.718 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:48:17.719 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:48:17.719 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:48:17.719 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:48:17.719 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:48:17.719 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:48:17.720 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:48:17.720 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 30, 2020-11-28T21:48:17.720 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:48:17.720 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:48:17.721 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:48:17.721 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:48:17.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.721 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:48:17.721 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6800", 2020-11-28T21:48:17.722 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:48:17.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.722 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:48:17.722 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6801", 2020-11-28T21:48:17.723 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:48:17.723 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:17.723 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:48:17.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.723 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:48:17.724 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:48:17.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.724 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:48:17.724 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6802", 2020-11-28T21:48:17.725 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:48:17.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.726 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:48:17.727 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6803", 2020-11-28T21:48:17.727 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:48:17.727 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:17.727 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:48:17.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.728 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:48:17.728 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:48:17.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.728 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:48:17.729 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6806", 2020-11-28T21:48:17.729 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:48:17.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.729 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:48:17.730 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6807", 2020-11-28T21:48:17.730 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:48:17.730 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:17.730 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:48:17.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.731 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:48:17.731 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:48:17.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.731 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:48:17.732 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6804", 2020-11-28T21:48:17.732 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:48:17.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.732 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:48:17.733 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6805", 2020-11-28T21:48:17.733 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:48:17.733 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:17.733 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:48:17.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.734 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6801/12527", 2020-11-28T21:48:17.734 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6803/12527", 2020-11-28T21:48:17.734 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6807/12527", 2020-11-28T21:48:17.734 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6805/12527", 2020-11-28T21:48:17.734 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:48:17.735 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:48:17.735 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:48:17.735 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:48:17.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.736 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 5, 2020-11-28T21:48:17.736 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "55ba305d-0505-4212-aebb-631e8554b440", 2020-11-28T21:48:17.736 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:48:17.736 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:48:17.737 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:48:17.737 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:48:17.737 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:48:17.737 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:48:17.737 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:48:17.738 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 31, 2020-11-28T21:48:17.738 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:48:17.738 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:48:17.738 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:48:17.738 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:48:17.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.739 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:48:17.739 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6808", 2020-11-28T21:48:17.739 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:48:17.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.740 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:48:17.740 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6809", 2020-11-28T21:48:17.740 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:48:17.741 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:17.741 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:48:17.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.741 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:48:17.741 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:48:17.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.742 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:48:17.742 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6810", 2020-11-28T21:48:17.742 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:48:17.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.743 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:48:17.743 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6811", 2020-11-28T21:48:17.743 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:48:17.743 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:17.744 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:48:17.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.744 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:48:17.744 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:48:17.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.745 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:48:17.745 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6814", 2020-11-28T21:48:17.745 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:48:17.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.746 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:48:17.746 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6815", 2020-11-28T21:48:17.746 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:48:17.746 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:17.747 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:48:17.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.747 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:48:17.747 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:48:17.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.748 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:48:17.748 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6812", 2020-11-28T21:48:17.748 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:48:17.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.749 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:48:17.749 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6813", 2020-11-28T21:48:17.749 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:48:17.749 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:17.750 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:48:17.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.750 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6809/12529", 2020-11-28T21:48:17.750 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6811/12529", 2020-11-28T21:48:17.750 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6815/12529", 2020-11-28T21:48:17.751 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6813/12529", 2020-11-28T21:48:17.751 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:48:17.751 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:48:17.751 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:48:17.752 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:48:17.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.752 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 6, 2020-11-28T21:48:17.752 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "9ada6051-493b-46b5-ba22-1b401440490f", 2020-11-28T21:48:17.753 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:48:17.753 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:48:17.753 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:48:17.753 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:48:17.753 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:48:17.754 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:48:17.754 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:48:17.754 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 31, 2020-11-28T21:48:17.754 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:48:17.754 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:48:17.755 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:48:17.755 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:48:17.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.755 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:48:17.755 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6824", 2020-11-28T21:48:17.756 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:48:17.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.756 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:48:17.756 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6825", 2020-11-28T21:48:17.757 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:48:17.758 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:17.758 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:48:17.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.759 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:48:17.759 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:48:17.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.759 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:48:17.760 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6826", 2020-11-28T21:48:17.760 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:48:17.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.760 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:48:17.761 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6827", 2020-11-28T21:48:17.761 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:48:17.761 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:17.761 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:48:17.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.762 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:48:17.762 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:48:17.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.762 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:48:17.762 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6830", 2020-11-28T21:48:17.763 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:48:17.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.763 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:48:17.764 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6831", 2020-11-28T21:48:17.764 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:48:17.764 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:17.764 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:48:17.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.765 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:48:17.765 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:48:17.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.765 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:48:17.765 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6828", 2020-11-28T21:48:17.766 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:48:17.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.766 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:48:17.766 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6829", 2020-11-28T21:48:17.767 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:48:17.767 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:17.767 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:48:17.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.767 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6825/12530", 2020-11-28T21:48:17.768 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6827/12530", 2020-11-28T21:48:17.768 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6831/12530", 2020-11-28T21:48:17.768 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6829/12530", 2020-11-28T21:48:17.768 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:48:17.768 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:48:17.769 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:48:17.769 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:48:17.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.769 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 7, 2020-11-28T21:48:17.770 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "c328b06d-a216-4cba-aad7-9ad081340613", 2020-11-28T21:48:17.770 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:48:17.770 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:48:17.770 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:48:17.771 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:48:17.771 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:48:17.771 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:48:17.771 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:48:17.771 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 31, 2020-11-28T21:48:17.772 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:48:17.772 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:48:17.772 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:48:17.772 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:48:17.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.773 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:48:17.773 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6816", 2020-11-28T21:48:17.773 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:48:17.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.774 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:48:17.774 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6817", 2020-11-28T21:48:17.774 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:48:17.774 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:17.774 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:48:17.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.775 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:48:17.775 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:48:17.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.775 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:48:17.776 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6818", 2020-11-28T21:48:17.776 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:48:17.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.776 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:48:17.777 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6819", 2020-11-28T21:48:17.777 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:48:17.777 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:17.777 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:48:17.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.778 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:48:17.778 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:48:17.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.778 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:48:17.779 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6822", 2020-11-28T21:48:17.779 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:48:17.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.779 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:48:17.780 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6823", 2020-11-28T21:48:17.780 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:48:17.780 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:17.780 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:48:17.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.781 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:48:17.781 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:48:17.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.781 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:48:17.782 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6820", 2020-11-28T21:48:17.782 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:48:17.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.782 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:48:17.783 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6821", 2020-11-28T21:48:17.783 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:48:17.783 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:17.783 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:48:17.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.784 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6817/12528", 2020-11-28T21:48:17.784 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6819/12528", 2020-11-28T21:48:17.784 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6823/12528", 2020-11-28T21:48:17.784 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6821/12528", 2020-11-28T21:48:17.784 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:48:17.785 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:48:17.785 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:48:17.785 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:48:17.785 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:17.785 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:48:17.786 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_xinfo": [ 2020-11-28T21:48:17.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.786 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 0, 2020-11-28T21:48:17.786 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:48:17.787 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:48:17.787 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:48:17.787 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:48:17.787 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:48:17.787 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.038052+0000", 2020-11-28T21:48:17.788 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:48:17.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.788 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 1, 2020-11-28T21:48:17.788 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:48:17.789 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:48:17.789 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:48:17.789 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:48:17.789 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:48:17.789 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:17.947136+0000", 2020-11-28T21:48:17.790 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:48:17.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.790 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 2, 2020-11-28T21:48:17.790 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:48:17.791 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:48:17.791 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:48:17.791 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:48:17.791 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:48:17.791 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.001186+0000", 2020-11-28T21:48:17.792 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:48:17.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.792 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 3, 2020-11-28T21:48:17.792 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:48:17.793 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:48:17.793 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:48:17.793 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:48:17.793 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:48:17.794 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.199284+0000", 2020-11-28T21:48:17.794 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:48:17.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.794 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 4, 2020-11-28T21:48:17.795 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:48:17.795 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:48:17.795 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:48:17.795 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:48:17.795 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:48:17.796 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.038823+0000", 2020-11-28T21:48:17.796 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:48:17.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.796 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 5, 2020-11-28T21:48:17.797 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:48:17.797 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:48:17.797 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:48:17.797 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:48:17.797 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:48:17.798 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.009322+0000", 2020-11-28T21:48:17.798 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:48:17.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.798 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 6, 2020-11-28T21:48:17.799 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:48:17.799 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:48:17.799 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:48:17.799 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:48:17.799 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:48:17.800 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.238927+0000", 2020-11-28T21:48:17.800 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:48:17.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:17.801 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 7, 2020-11-28T21:48:17.801 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:48:17.801 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:48:17.801 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:48:17.801 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:48:17.802 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:48:17.802 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.029839+0000", 2020-11-28T21:48:17.802 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:48:17.802 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:17.802 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:48:17.803 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_upmap": [], 2020-11-28T21:48:17.803 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_upmap_items": [], 2020-11-28T21:48:17.803 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_temp": [], 2020-11-28T21:48:17.803 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_temp": [], 2020-11-28T21:48:17.803 INFO:teuthology.orchestra.run.smithi162.stdout: "blacklist": {}, 2020-11-28T21:48:17.804 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profiles": { 2020-11-28T21:48:17.804 INFO:teuthology.orchestra.run.smithi162.stdout: "default": { 2020-11-28T21:48:17.804 INFO:teuthology.orchestra.run.smithi162.stdout: "crush-failure-domain": "osd", 2020-11-28T21:48:17.804 INFO:teuthology.orchestra.run.smithi162.stdout: "k": "2", 2020-11-28T21:48:17.804 INFO:teuthology.orchestra.run.smithi162.stdout: "m": "1", 2020-11-28T21:48:17.805 INFO:teuthology.orchestra.run.smithi162.stdout: "plugin": "jerasure", 2020-11-28T21:48:17.805 INFO:teuthology.orchestra.run.smithi162.stdout: "ruleset-failure-domain": "osd", 2020-11-28T21:48:17.805 INFO:teuthology.orchestra.run.smithi162.stdout: "technique": "reed_sol_van" 2020-11-28T21:48:17.805 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:17.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:17.806 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps_queue": [], 2020-11-28T21:48:17.806 INFO:teuthology.orchestra.run.smithi162.stdout: "new_removed_snaps": [], 2020-11-28T21:48:17.806 INFO:teuthology.orchestra.run.smithi162.stdout: "new_purged_snaps": [], 2020-11-28T21:48:17.806 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_node_flags": {}, 2020-11-28T21:48:17.806 INFO:teuthology.orchestra.run.smithi162.stdout: "device_class_flags": {} 2020-11-28T21:48:17.807 INFO:teuthology.orchestra.run.smithi162.stdout:} 2020-11-28T21:48:17.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.583+0000 7fd36d686700 1 -- 172.21.15.162:0/108415902 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd34c0554e0 msgr2=0x7fd34c0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:17.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.583+0000 7fd36d686700 1 --2- 172.21.15.162:0/108415902 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd34c0554e0 0x7fd34c0579a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fd35001b000 tx=0x7fd350015000).stop 2020-11-28T21:48:17.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.583+0000 7fd36d686700 1 -- 172.21.15.162:0/108415902 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3680b60e0 msgr2=0x7fd368151ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:17.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.583+0000 7fd36d686700 1 --2- 172.21.15.162:0/108415902 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3680b60e0 0x7fd368151ac0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7fd360001450 tx=0x7fd360007dd0).stop 2020-11-28T21:48:17.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.583+0000 7fd36d686700 1 -- 172.21.15.162:0/108415902 shutdown_connections 2020-11-28T21:48:17.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.583+0000 7fd36d686700 1 --2- 172.21.15.162:0/108415902 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd34c0554e0 0x7fd34c0579a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:17.810 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.583+0000 7fd36d686700 1 --2- 172.21.15.162:0/108415902 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd3680b60e0 0x7fd368151ac0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:17.810 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.583+0000 7fd36d686700 1 --2- 172.21.15.162:0/108415902 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd368060d10 0x7fd368149ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:17.810 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.583+0000 7fd36d686700 1 --2- 172.21.15.162:0/108415902 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd3680b4390 0x7fd368155a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:17.810 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.583+0000 7fd36d686700 1 -- 172.21.15.162:0/108415902 >> 172.21.15.162:0/108415902 conn(0x7fd36805fe70 msgr2=0x7fd36805b970 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:17.810 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.583+0000 7fd36d686700 1 -- 172.21.15.162:0/108415902 shutdown_connections 2020-11-28T21:48:17.811 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.583+0000 7fd36d686700 1 -- 172.21.15.162:0/108415902 wait complete. 2020-11-28T21:48:17.812 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:48:17.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.931+0000 7f8bb9d58700 1 Processor -- start 2020-11-28T21:48:17.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.931+0000 7f8bb9d58700 1 -- start start 2020-11-28T21:48:17.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.931+0000 7f8bb9d58700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8bb4143530 0x7f8bb40b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:17.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.931+0000 7f8bb9d58700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bb40b0e50 con 0x7f8bb413c630 2020-11-28T21:48:17.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.931+0000 7f8bb9d58700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bb405c090 con 0x7f8bb405bcb0 2020-11-28T21:48:17.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.931+0000 7f8bb9d58700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb405fe40 con 0x7f8bb4143530 2020-11-28T21:48:17.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.931+0000 7f8bb8d56700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f8bb405bcb0 0x7f8bb40b0740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:45818/0 (socket says 172.21.15.162:45818) 2020-11-28T21:48:17.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.931+0000 7f8bb8d56700 1 -- 172.21.15.162:0/3529679220 learned_addr learned my addr 172.21.15.162:0/3529679220 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:48:17.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.931+0000 7f8bb3fff700 1 --2- 172.21.15.162:0/3529679220 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8bb4143530 0x7f8bb40b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:17.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.931+0000 7f8bb37fe700 1 -- 172.21.15.162:0/3529679220 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1664770290 0 0) 0x7f8bb405c090 con 0x7f8bb405bcb0 2020-11-28T21:48:17.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.931+0000 7f8bb37fe700 1 -- 172.21.15.162:0/3529679220 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ba0003230 con 0x7f8bb405bcb0 2020-11-28T21:48:17.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.931+0000 7f8bb37fe700 1 -- 172.21.15.162:0/3529679220 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3102433359 0 0) 0x7f8bb40b0e50 con 0x7f8bb413c630 2020-11-28T21:48:17.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.931+0000 7f8bb37fe700 1 -- 172.21.15.162:0/3529679220 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bb405c090 con 0x7f8bb413c630 2020-11-28T21:48:17.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.931+0000 7f8bb37fe700 1 -- 172.21.15.162:0/3529679220 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2631924330 0 0) 0x7f8ba0003230 con 0x7f8bb405bcb0 2020-11-28T21:48:17.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.931+0000 7f8bb37fe700 1 -- 172.21.15.162:0/3529679220 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8bb4143530 msgr2=0x7f8bb40b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:17.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.931+0000 7f8bb37fe700 1 --2- 172.21.15.162:0/3529679220 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8bb4143530 0x7f8bb40b1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:17.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.931+0000 7f8bb37fe700 1 -- 172.21.15.162:0/3529679220 >> v1:172.21.15.118:6789/0 conn(0x7f8bb413c630 legacy=0x7f8bb414d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:17.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.931+0000 7f8bb37fe700 1 -- 172.21.15.162:0/3529679220 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb40b22a0 con 0x7f8bb405bcb0 2020-11-28T21:48:17.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.931+0000 7f8bb37fe700 1 -- 172.21.15.162:0/3529679220 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f8bac001a50 con 0x7f8bb405bcb0 2020-11-28T21:48:17.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.931+0000 7f8bb3fff700 1 --2- 172.21.15.162:0/3529679220 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8bb4143530 0x7f8bb40b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:48:17.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.931+0000 7f8bb37fe700 1 -- 172.21.15.162:0/3529679220 >> v1:172.21.15.162:6789/0 conn(0x7f8bb405bcb0 legacy=0x7f8bb40b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:17.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.931+0000 7f8bb37fe700 1 --2- 172.21.15.162:0/3529679220 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8bb4143530 0x7f8ba00038b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:17.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.931+0000 7f8bb37fe700 1 --2- 172.21.15.162:0/3529679220 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8ba0003da0 0x7f8ba00044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:17.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.931+0000 7f8bb37fe700 1 --2- 172.21.15.162:0/3529679220 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ba00049c0 0x7f8ba0004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:17.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.931+0000 7f8bb37fe700 1 -- 172.21.15.162:0/3529679220 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb405fe40 con 0x7f8ba0003da0 2020-11-28T21:48:17.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.931+0000 7f8bb37fe700 1 -- 172.21.15.162:0/3529679220 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba0005330 con 0x7f8ba00049c0 2020-11-28T21:48:17.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.931+0000 7f8bb37fe700 1 -- 172.21.15.162:0/3529679220 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba000b750 con 0x7f8bb4143530 2020-11-28T21:48:17.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.931+0000 7f8bb9557700 1 --2- 172.21.15.162:0/3529679220 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ba00049c0 0x7f8ba0004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:17.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.931+0000 7f8bb8d56700 1 --2- 172.21.15.162:0/3529679220 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8bb4143530 0x7f8ba00038b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:17.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.931+0000 7f8bb3fff700 1 --2- 172.21.15.162:0/3529679220 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8ba0003da0 0x7f8ba00044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:17.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.935+0000 7f8bb9557700 1 -- 172.21.15.162:0/3529679220 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8bb4143530 msgr2=0x7f8ba00038b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:17.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.935+0000 7f8bb9557700 1 --2- 172.21.15.162:0/3529679220 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8bb4143530 0x7f8ba00038b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:17.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.935+0000 7f8bb9557700 1 -- 172.21.15.162:0/3529679220 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8ba0003da0 msgr2=0x7f8ba00044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:17.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.935+0000 7f8bb9557700 1 --2- 172.21.15.162:0/3529679220 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8ba0003da0 0x7f8ba00044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:17.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.935+0000 7f8bb9557700 1 -- 172.21.15.162:0/3529679220 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba0003230 con 0x7f8ba00049c0 2020-11-28T21:48:17.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.935+0000 7f8bb9557700 1 --2- 172.21.15.162:0/3529679220 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ba00049c0 0x7f8ba0004e40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f8ba40010d0 tx=0x7f8ba4007b60).ready entity=mon.1 client_cookie=68563a5ac8cae0ed server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:17.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.935+0000 7f8bb37fe700 1 -- 172.21.15.162:0/3529679220 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ba400c660 con 0x7f8ba00049c0 2020-11-28T21:48:17.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.935+0000 7f8bb37fe700 1 -- 172.21.15.162:0/3529679220 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba400c7c0 con 0x7f8ba00049c0 2020-11-28T21:48:17.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.935+0000 7f8bb37fe700 1 -- 172.21.15.162:0/3529679220 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ba400cb20 con 0x7f8ba00049c0 2020-11-28T21:48:17.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.935+0000 7f8bb9d58700 1 -- 172.21.15.162:0/3529679220 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ba00049c0 msgr2=0x7f8ba0004e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:17.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.935+0000 7f8bb9d58700 1 --2- 172.21.15.162:0/3529679220 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ba00049c0 0x7f8ba0004e40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f8ba40010d0 tx=0x7f8ba4007b60).stop 2020-11-28T21:48:17.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.935+0000 7f8bb9557700 1 -- 172.21.15.162:0/3529679220 reap_dead start 2020-11-28T21:48:17.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.935+0000 7f8bb9d58700 1 -- 172.21.15.162:0/3529679220 shutdown_connections 2020-11-28T21:48:17.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.935+0000 7f8bb9d58700 1 -- 172.21.15.162:0/3529679220 >> 172.21.15.162:0/3529679220 conn(0x7f8bb40b5da0 msgr2=0x7f8bb40b6180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:17.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.935+0000 7f8bb9d58700 1 -- 172.21.15.162:0/3529679220 shutdown_connections 2020-11-28T21:48:17.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.935+0000 7f8bb9d58700 1 -- 172.21.15.162:0/3529679220 wait complete. 2020-11-28T21:48:17.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.935+0000 7f8bb9d58700 1 Processor -- start 2020-11-28T21:48:17.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.935+0000 7f8bb9d58700 1 -- start start 2020-11-28T21:48:17.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.939+0000 7f8bb9d58700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8bb40afaa0 0x7f8bb40ad250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:17.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.939+0000 7f8bb9d58700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8bb4060470 0x7f8bb40608f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:17.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.939+0000 7f8bb9d58700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8bb4060de0 0x7f8bb4061260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:17.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.939+0000 7f8bb9d58700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb40b0120 con 0x7f8bb40afaa0 2020-11-28T21:48:17.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.939+0000 7f8bb9d58700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb41448f0 con 0x7f8bb4060de0 2020-11-28T21:48:17.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.939+0000 7f8bb9d58700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb40ad740 con 0x7f8bb4060470 2020-11-28T21:48:17.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.939+0000 7f8bb9557700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8bb4060de0 0x7f8bb4061260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:17.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.939+0000 7f8bb9557700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8bb4060de0 0x7f8bb4061260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38990/0 (socket says 172.21.15.162:38990) 2020-11-28T21:48:17.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.939+0000 7f8bb9557700 1 -- 172.21.15.162:0/4212407346 learned_addr learned my addr 172.21.15.162:0/4212407346 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:17.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.939+0000 7f8bb8d56700 1 --2- 172.21.15.162:0/4212407346 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8bb40afaa0 0x7f8bb40ad250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:17.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.939+0000 7f8bb3fff700 1 --2- 172.21.15.162:0/4212407346 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8bb4060470 0x7f8bb40608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:17.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.939+0000 7f8bb9557700 1 -- 172.21.15.162:0/4212407346 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8bb4060470 msgr2=0x7f8bb40608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:17.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.939+0000 7f8bb9557700 1 --2- 172.21.15.162:0/4212407346 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8bb4060470 0x7f8bb40608f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:17.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.939+0000 7f8bb9557700 1 -- 172.21.15.162:0/4212407346 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8bb40afaa0 msgr2=0x7f8bb40ad250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:17.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.939+0000 7f8bb9557700 1 --2- 172.21.15.162:0/4212407346 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8bb40afaa0 0x7f8bb40ad250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:17.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.939+0000 7f8bb9557700 1 -- 172.21.15.162:0/4212407346 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb40afe80 con 0x7f8bb4060de0 2020-11-28T21:48:17.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.939+0000 7f8bb8d56700 1 --2- 172.21.15.162:0/4212407346 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8bb40afaa0 0x7f8bb40ad250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:48:17.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.939+0000 7f8bb9557700 1 --2- 172.21.15.162:0/4212407346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8bb4060de0 0x7f8bb4061260 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f8bb4144600 tx=0x7f8ba4017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:17.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.939+0000 7f8bb1ffb700 1 -- 172.21.15.162:0/4212407346 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ba4001ea0 con 0x7f8bb4060de0 2020-11-28T21:48:17.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.939+0000 7f8bb1ffb700 1 -- 172.21.15.162:0/4212407346 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba4012630 con 0x7f8bb4060de0 2020-11-28T21:48:17.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.939+0000 7f8bb9d58700 1 -- 172.21.15.162:0/4212407346 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bb4145cb0 con 0x7f8bb4060de0 2020-11-28T21:48:17.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.939+0000 7f8bb1ffb700 1 -- 172.21.15.162:0/4212407346 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ba4012990 con 0x7f8bb4060de0 2020-11-28T21:48:17.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.939+0000 7f8bb9d58700 1 -- 172.21.15.162:0/4212407346 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bb41455c0 con 0x7f8bb4060de0 2020-11-28T21:48:17.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.939+0000 7f8bb9d58700 1 -- 172.21.15.162:0/4212407346 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bb4145e70 con 0x7f8bb4060de0 2020-11-28T21:48:17.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.939+0000 7f8bb1ffb700 1 -- 172.21.15.162:0/4212407346 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8ba4010070 con 0x7f8bb4060de0 2020-11-28T21:48:17.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.943+0000 7f8bb1ffb700 1 --2- 172.21.15.162:0/4212407346 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8b940553b0 0x7f8b94057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:17.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.943+0000 7f8bb8d56700 1 --2- 172.21.15.162:0/4212407346 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8b940553b0 0x7f8b94057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:17.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.943+0000 7f8bb1ffb700 1 -- 172.21.15.162:0/4212407346 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8ba406e850 con 0x7f8bb4060de0 2020-11-28T21:48:17.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.943+0000 7f8bb8d56700 1 --2- 172.21.15.162:0/4212407346 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8b940553b0 0x7f8b94057870 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f8bac005000 tx=0x7f8bac009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:17.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:17.947+0000 7f8bb1ffb700 1 -- 172.21.15.162:0/4212407346 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8ba4044b50 con 0x7f8bb4060de0 2020-11-28T21:48:18.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.131+0000 7f8bb9d58700 1 -- 172.21.15.162:0/4212407346 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8bb413c0c0 con 0x7f8bb4060de0 2020-11-28T21:48:18.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.131+0000 7f8bb1ffb700 1 -- 172.21.15.162:0/4212407346 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+2497 (secure 0 0 0) 0x7f8ba4048170 con 0x7f8bb4060de0 2020-11-28T21:48:18.131 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:48:18.131 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":10,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4839,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/1153453343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1153453343},{"type":"v1","addr":"172.21.15.162:6836","nonce":1153453343}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8},{"gid":4849,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2101329586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":2101329586},{"type":"v1","addr":"172.21.15.162:6837","nonce":2101329586}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:47:10.330722+0000","modified":"2020-11-28T21:47:11.333518+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4855},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4855":{"gid":4855,"name":"b","rank":0,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.118:6833/3038031199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3038031199},{"type":"v1","addr":"172.21.15.118:6833","nonce":3038031199}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-28T21:48:18.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.131+0000 7f8bb9d58700 1 -- 172.21.15.162:0/4212407346 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8b940553b0 msgr2=0x7f8b94057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:18.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.131+0000 7f8bb9d58700 1 --2- 172.21.15.162:0/4212407346 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8b940553b0 0x7f8b94057870 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f8bac005000 tx=0x7f8bac009040).stop 2020-11-28T21:48:18.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.131+0000 7f8bb9d58700 1 -- 172.21.15.162:0/4212407346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8bb4060de0 msgr2=0x7f8bb4061260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:18.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.131+0000 7f8bb9d58700 1 --2- 172.21.15.162:0/4212407346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8bb4060de0 0x7f8bb4061260 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f8bb4144600 tx=0x7f8ba4017000).stop 2020-11-28T21:48:18.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.131+0000 7f8bb9d58700 1 -- 172.21.15.162:0/4212407346 shutdown_connections 2020-11-28T21:48:18.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.131+0000 7f8bb9d58700 1 --2- 172.21.15.162:0/4212407346 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8b940553b0 0x7f8b94057870 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:18.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.131+0000 7f8bb9d58700 1 --2- 172.21.15.162:0/4212407346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8bb4060de0 0x7f8bb4061260 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:18.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.131+0000 7f8bb9d58700 1 --2- 172.21.15.162:0/4212407346 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8bb40afaa0 0x7f8bb40ad250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:18.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.131+0000 7f8bb9d58700 1 --2- 172.21.15.162:0/4212407346 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8bb4060470 0x7f8bb40608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:18.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.131+0000 7f8bb9d58700 1 -- 172.21.15.162:0/4212407346 >> 172.21.15.162:0/4212407346 conn(0x7f8bb40b5da0 msgr2=0x7f8bb41413a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:18.138 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.131+0000 7f8bb9d58700 1 -- 172.21.15.162:0/4212407346 shutdown_connections 2020-11-28T21:48:18.138 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.131+0000 7f8bb9d58700 1 -- 172.21.15.162:0/4212407346 wait complete. 2020-11-28T21:48:18.139 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 10 2020-11-28T21:48:18.145 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2020-11-28T21:48:18.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.255+0000 7f1716dba700 1 Processor -- start 2020-11-28T21:48:18.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.255+0000 7f1716dba700 1 -- start start 2020-11-28T21:48:18.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.255+0000 7f1716dba700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1710060cb0 0x7f1710142980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:18.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.255+0000 7f1716dba700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f17100b3a60 0x7f171013fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:18.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.255+0000 7f1716dba700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f171005f380 con 0x7f17100b5bf0 2020-11-28T21:48:18.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.255+0000 7f1716dba700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1710143930 con 0x7f1710060cb0 2020-11-28T21:48:18.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.255+0000 7f1716dba700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f171005f5b0 con 0x7f17100b3a60 2020-11-28T21:48:18.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.259+0000 7f17155b7700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1710060cb0 0x7f1710142980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:18.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.259+0000 7f1715db8700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f17100b5bf0 0x7f1710060230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:45838/0 (socket says 172.21.15.162:45838) 2020-11-28T21:48:18.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.259+0000 7f17155b7700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1710060cb0 0x7f1710142980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:38996/0 (socket says 172.21.15.162:38996) 2020-11-28T21:48:18.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.259+0000 7f1715db8700 1 -- 172.21.15.162:0/3461147959 learned_addr learned my addr 172.21.15.162:0/3461147959 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:48:18.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.259+0000 7f17165b9700 1 --2- 172.21.15.162:0/3461147959 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f17100b3a60 0x7f171013fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:18.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.259+0000 7f17155b7700 1 -- 172.21.15.162:0/3461147959 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f17100b3a60 msgr2=0x7f171013fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:18.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.259+0000 7f17155b7700 1 --2- 172.21.15.162:0/3461147959 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f17100b3a60 0x7f171013fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:18.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.259+0000 7f17155b7700 1 -- 172.21.15.162:0/3461147959 >> v1:172.21.15.162:6789/0 conn(0x7f17100b5bf0 legacy=0x7f1710060230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:18.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.259+0000 7f17155b7700 1 -- 172.21.15.162:0/3461147959 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f171005f850 con 0x7f1710060cb0 2020-11-28T21:48:18.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.259+0000 7f17155b7700 1 --2- 172.21.15.162:0/3461147959 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1710060cb0 0x7f1710142980 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f170c002f30 tx=0x7f170c006dc0).ready entity=mon.1 client_cookie=bffa0d3e7311a1ad server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:18.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.259+0000 7f1714db6700 1 -- 172.21.15.162:0/3461147959 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f170c0097f0 con 0x7f1710060cb0 2020-11-28T21:48:18.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.259+0000 7f1714db6700 1 -- 172.21.15.162:0/3461147959 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f170c009950 con 0x7f1710060cb0 2020-11-28T21:48:18.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.259+0000 7f1714db6700 1 -- 172.21.15.162:0/3461147959 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f170c009cb0 con 0x7f1710060cb0 2020-11-28T21:48:18.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.259+0000 7f1716dba700 1 -- 172.21.15.162:0/3461147959 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1710060cb0 msgr2=0x7f1710142980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:18.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.259+0000 7f1716dba700 1 --2- 172.21.15.162:0/3461147959 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1710060cb0 0x7f1710142980 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f170c002f30 tx=0x7f170c006dc0).stop 2020-11-28T21:48:18.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.259+0000 7f1716dba700 1 -- 172.21.15.162:0/3461147959 shutdown_connections 2020-11-28T21:48:18.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.259+0000 7f1716dba700 1 --2- 172.21.15.162:0/3461147959 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f17100b3a60 0x7f171013fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:18.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.259+0000 7f1716dba700 1 --2- 172.21.15.162:0/3461147959 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1710060cb0 0x7f1710142980 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:18.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.259+0000 7f1716dba700 1 -- 172.21.15.162:0/3461147959 >> 172.21.15.162:0/3461147959 conn(0x7f171005b970 msgr2=0x7f171005bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:18.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.259+0000 7f1716dba700 1 -- 172.21.15.162:0/3461147959 shutdown_connections 2020-11-28T21:48:18.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.259+0000 7f1716dba700 1 -- 172.21.15.162:0/3461147959 wait complete. 2020-11-28T21:48:18.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.259+0000 7f1716dba700 1 Processor -- start 2020-11-28T21:48:18.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.259+0000 7f1716dba700 1 -- start start 2020-11-28T21:48:18.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.263+0000 7f1716dba700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1710060cb0 0x7f17101465a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:18.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.263+0000 7f1716dba700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f17100b3a60 0x7f1710145ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:18.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.263+0000 7f1716dba700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f17100b5bf0 0x7f17100b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:18.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.263+0000 7f1716dba700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171005eb80 con 0x7f17100b5bf0 2020-11-28T21:48:18.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.263+0000 7f1716dba700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17101451c0 con 0x7f1710060cb0 2020-11-28T21:48:18.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.263+0000 7f1716dba700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17100b0780 con 0x7f17100b3a60 2020-11-28T21:48:18.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.263+0000 7f1715db8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1710060cb0 0x7f17101465a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:18.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.263+0000 7f1715db8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1710060cb0 0x7f17101465a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39000/0 (socket says 172.21.15.162:39000) 2020-11-28T21:48:18.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.263+0000 7f1715db8700 1 -- 172.21.15.162:0/2558178561 learned_addr learned my addr 172.21.15.162:0/2558178561 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:18.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.263+0000 7f17155b7700 1 --2- 172.21.15.162:0/2558178561 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f17100b3a60 0x7f1710145ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:18.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.263+0000 7f17165b9700 1 --2- 172.21.15.162:0/2558178561 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f17100b5bf0 0x7f17100b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:18.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.263+0000 7f1715db8700 1 -- 172.21.15.162:0/2558178561 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f17100b3a60 msgr2=0x7f1710145ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:18.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.263+0000 7f1715db8700 1 --2- 172.21.15.162:0/2558178561 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f17100b3a60 0x7f1710145ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:18.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.263+0000 7f1715db8700 1 -- 172.21.15.162:0/2558178561 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f17100b5bf0 msgr2=0x7f17100b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:18.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.263+0000 7f1715db8700 1 --2- 172.21.15.162:0/2558178561 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f17100b5bf0 0x7f17100b21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:18.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.263+0000 7f1715db8700 1 -- 172.21.15.162:0/2558178561 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1710145640 con 0x7f1710060cb0 2020-11-28T21:48:18.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.263+0000 7f1715db8700 1 --2- 172.21.15.162:0/2558178561 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1710060cb0 0x7f17101465a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f1710151890 tx=0x7f1704007c30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:18.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.263+0000 7f1702ffd700 1 -- 172.21.15.162:0/2558178561 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1704007320 con 0x7f1710060cb0 2020-11-28T21:48:18.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.263+0000 7f1702ffd700 1 -- 172.21.15.162:0/2558178561 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1704007480 con 0x7f1710060cb0 2020-11-28T21:48:18.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.263+0000 7f1702ffd700 1 -- 172.21.15.162:0/2558178561 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1704005220 con 0x7f1710060cb0 2020-11-28T21:48:18.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.263+0000 7f1716dba700 1 -- 172.21.15.162:0/2558178561 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1710146bb0 con 0x7f1710060cb0 2020-11-28T21:48:18.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.263+0000 7f1716dba700 1 -- 172.21.15.162:0/2558178561 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17100b02a0 con 0x7f1710060cb0 2020-11-28T21:48:18.273 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.263+0000 7f1716dba700 1 -- 172.21.15.162:0/2558178561 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1710057050 con 0x7f1710060cb0 2020-11-28T21:48:18.273 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.267+0000 7f1702ffd700 1 -- 172.21.15.162:0/2558178561 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f170401b070 con 0x7f1710060cb0 2020-11-28T21:48:18.273 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.267+0000 7f1702ffd700 1 --2- 172.21.15.162:0/2558178561 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f16f0055330 0x7f16f00577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:18.273 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.267+0000 7f1702ffd700 1 -- 172.21.15.162:0/2558178561 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f170406d2a0 con 0x7f1710060cb0 2020-11-28T21:48:18.273 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.267+0000 7f17155b7700 1 --2- 172.21.15.162:0/2558178561 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f16f0055330 0x7f16f00577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:18.274 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.267+0000 7f17155b7700 1 --2- 172.21.15.162:0/2558178561 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f16f0055330 0x7f16f00577f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f170c01b000 tx=0x7f170c015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:18.274 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.271+0000 7f1702ffd700 1 -- 172.21.15.162:0/2558178561 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f17040435a0 con 0x7f1710060cb0 2020-11-28T21:48:18.458 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:18.455+0000 7f1716dba700 1 -- 172.21.15.162:0/2558178561 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1710068350 con 0x7f1710060cb0 2020-11-28T21:48:19.148 INFO:tasks.ceph.mon.b.smithi118.stderr:2020-11-28T21:48:19.146+0000 7ff4e1520700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-11-28T21:48:19.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.155+0000 7f1702ffd700 1 -- 172.21.15.162:0/2558178561 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v11) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1704046bc0 con 0x7f1710060cb0 2020-11-28T21:48:19.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.159+0000 7f1716dba700 1 -- 172.21.15.162:0/2558178561 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f16f0055330 msgr2=0x7f16f00577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:19.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.159+0000 7f1716dba700 1 --2- 172.21.15.162:0/2558178561 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f16f0055330 0x7f16f00577f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f170c01b000 tx=0x7f170c015000).stop 2020-11-28T21:48:19.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.159+0000 7f1716dba700 1 -- 172.21.15.162:0/2558178561 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1710060cb0 msgr2=0x7f17101465a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:19.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.159+0000 7f1716dba700 1 --2- 172.21.15.162:0/2558178561 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1710060cb0 0x7f17101465a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f1710151890 tx=0x7f1704007c30).stop 2020-11-28T21:48:19.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.159+0000 7f1716dba700 1 -- 172.21.15.162:0/2558178561 shutdown_connections 2020-11-28T21:48:19.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.159+0000 7f1716dba700 1 --2- 172.21.15.162:0/2558178561 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f16f0055330 0x7f16f00577f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:19.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.159+0000 7f1716dba700 1 --2- 172.21.15.162:0/2558178561 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f17100b5bf0 0x7f17100b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:19.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.159+0000 7f1716dba700 1 --2- 172.21.15.162:0/2558178561 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1710060cb0 0x7f17101465a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:19.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.159+0000 7f1716dba700 1 --2- 172.21.15.162:0/2558178561 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f17100b3a60 0x7f1710145ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:19.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.159+0000 7f1716dba700 1 -- 172.21.15.162:0/2558178561 >> 172.21.15.162:0/2558178561 conn(0x7f171005b970 msgr2=0x7f17100b8670 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:19.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.159+0000 7f1716dba700 1 -- 172.21.15.162:0/2558178561 shutdown_connections 2020-11-28T21:48:19.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.159+0000 7f1716dba700 1 -- 172.21.15.162:0/2558178561 wait complete. 2020-11-28T21:48:19.163 INFO:teuthology.orchestra.run.smithi162.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-11-28T21:48:19.176 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:48:19.194 INFO:tasks.ceph.mds.b.smithi118.stdout:starting mds.b at 2020-11-28T21:48:19.291 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.291+0000 7fab58e2b700 1 Processor -- start 2020-11-28T21:48:19.292 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.291+0000 7fab58e2b700 1 -- start start 2020-11-28T21:48:19.296 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.291+0000 7fab58e2b700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab5405bcb0 0x7fab54143530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:19.296 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.291+0000 7fab58e2b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fab5413c630 0x7fab5413cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:19.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.291+0000 7fab58e2b700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab540b0e50 con 0x7fab540b1c00 2020-11-28T21:48:19.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.291+0000 7fab58e2b700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab5405fe40 con 0x7fab5413c630 2020-11-28T21:48:19.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.291+0000 7fab58e2b700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab540b2080 con 0x7fab5405bcb0 2020-11-28T21:48:19.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.291+0000 7fab53fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fab5413c630 0x7fab5413cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:19.298 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.291+0000 7fab53fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fab5413c630 0x7fab5413cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39012/0 (socket says 172.21.15.162:39012) 2020-11-28T21:48:19.298 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.291+0000 7fab53fff700 1 -- 172.21.15.162:0/4105889837 learned_addr learned my addr 172.21.15.162:0/4105889837 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:19.298 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.291+0000 7fab537fe700 1 --2- 172.21.15.162:0/4105889837 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab5405bcb0 0x7fab54143530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:19.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.291+0000 7fab53fff700 1 -- 172.21.15.162:0/4105889837 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab5405bcb0 msgr2=0x7fab54143530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:19.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.291+0000 7fab53fff700 1 --2- 172.21.15.162:0/4105889837 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab5405bcb0 0x7fab54143530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:19.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.291+0000 7fab52ffd700 1 -- 172.21.15.162:0/4105889837 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2616116759 0 0) 0x7fab540b0e50 con 0x7fab540b1c00 2020-11-28T21:48:19.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.291+0000 7fab53fff700 1 -- 172.21.15.162:0/4105889837 >> v1:172.21.15.118:6789/0 conn(0x7fab540b1c00 legacy=0x7fab540b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:19.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.291+0000 7fab53fff700 1 -- 172.21.15.162:0/4105889837 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab5405c090 con 0x7fab5413c630 2020-11-28T21:48:19.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.291+0000 7fab537fe700 1 --2- 172.21.15.162:0/4105889837 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab5405bcb0 0x7fab54143530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:48:19.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.295+0000 7fab53fff700 1 --2- 172.21.15.162:0/4105889837 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab5413c630 0x7fab5413cac0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7fab4c0032b0 tx=0x7fab4c009270).ready entity=mon.1 client_cookie=6e2ebb1c4343235b server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:19.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.295+0000 7fab52ffd700 1 -- 172.21.15.162:0/4105889837 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab4c009e60 con 0x7fab5413c630 2020-11-28T21:48:19.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.295+0000 7fab52ffd700 1 -- 172.21.15.162:0/4105889837 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab4c00bc00 con 0x7fab5413c630 2020-11-28T21:48:19.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.295+0000 7fab52ffd700 1 -- 172.21.15.162:0/4105889837 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab4c00bd70 con 0x7fab5413c630 2020-11-28T21:48:19.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.295+0000 7fab58e2b700 1 -- 172.21.15.162:0/4105889837 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab5413c630 msgr2=0x7fab5413cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:19.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.295+0000 7fab58e2b700 1 --2- 172.21.15.162:0/4105889837 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab5413c630 0x7fab5413cac0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7fab4c0032b0 tx=0x7fab4c009270).stop 2020-11-28T21:48:19.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.295+0000 7fab58e2b700 1 -- 172.21.15.162:0/4105889837 shutdown_connections 2020-11-28T21:48:19.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.295+0000 7fab58e2b700 1 --2- 172.21.15.162:0/4105889837 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab5413c630 0x7fab5413cac0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:19.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.295+0000 7fab58e2b700 1 --2- 172.21.15.162:0/4105889837 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab5405bcb0 0x7fab54143530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:19.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.295+0000 7fab58e2b700 1 -- 172.21.15.162:0/4105889837 >> 172.21.15.162:0/4105889837 conn(0x7fab540b5da0 msgr2=0x7fab540b6180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:19.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.295+0000 7fab58e2b700 1 -- 172.21.15.162:0/4105889837 shutdown_connections 2020-11-28T21:48:19.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.295+0000 7fab58e2b700 1 -- 172.21.15.162:0/4105889837 wait complete. 2020-11-28T21:48:19.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.295+0000 7fab58e2b700 1 Processor -- start 2020-11-28T21:48:19.306 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.295+0000 7fab58e2b700 1 -- start start 2020-11-28T21:48:19.306 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.295+0000 7fab58e2b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab5405bcb0 0x7fab54148670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:19.306 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.295+0000 7fab58e2b700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab540b1c00 0x7fab540ad170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:19.306 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.295+0000 7fab58e2b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab5413c630 0x7fab540aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:19.307 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.295+0000 7fab58e2b700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab540b0280 con 0x7fab5405bcb0 2020-11-28T21:48:19.307 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.295+0000 7fab58e2b700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab54146090 con 0x7fab5413c630 2020-11-28T21:48:19.307 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.295+0000 7fab58e2b700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab54146330 con 0x7fab540b1c00 2020-11-28T21:48:19.307 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.295+0000 7fab53fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab5413c630 0x7fab540aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:19.307 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.295+0000 7fab53fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab5413c630 0x7fab540aecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39016/0 (socket says 172.21.15.162:39016) 2020-11-28T21:48:19.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.295+0000 7fab53fff700 1 -- 172.21.15.162:0/2349373548 learned_addr learned my addr 172.21.15.162:0/2349373548 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:19.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.299+0000 7fab4bfff700 1 --2- 172.21.15.162:0/2349373548 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab540b1c00 0x7fab540ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:19.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.299+0000 7fab537fe700 1 --2- 172.21.15.162:0/2349373548 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab5405bcb0 0x7fab54148670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:19.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.299+0000 7fab53fff700 1 -- 172.21.15.162:0/2349373548 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab540b1c00 msgr2=0x7fab540ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:19.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.299+0000 7fab53fff700 1 --2- 172.21.15.162:0/2349373548 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab540b1c00 0x7fab540ad170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:19.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.299+0000 7fab53fff700 1 -- 172.21.15.162:0/2349373548 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab5405bcb0 msgr2=0x7fab54148670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:19.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.299+0000 7fab53fff700 1 --2- 172.21.15.162:0/2349373548 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab5405bcb0 0x7fab54148670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:19.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.299+0000 7fab53fff700 1 -- 172.21.15.162:0/2349373548 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab541465d0 con 0x7fab5413c630 2020-11-28T21:48:19.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.299+0000 7fab537fe700 1 --2- 172.21.15.162:0/2349373548 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab5405bcb0 0x7fab54148670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:48:19.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.299+0000 7fab53fff700 1 --2- 172.21.15.162:0/2349373548 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab5413c630 0x7fab540aecd0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7fab4c0032b0 tx=0x7fab4c016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:19.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.299+0000 7fab517fa700 1 -- 172.21.15.162:0/2349373548 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab4c01f5e0 con 0x7fab5413c630 2020-11-28T21:48:19.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.299+0000 7fab517fa700 1 -- 172.21.15.162:0/2349373548 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab4c01f740 con 0x7fab5413c630 2020-11-28T21:48:19.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.299+0000 7fab58e2b700 1 -- 172.21.15.162:0/2349373548 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab54154dc0 con 0x7fab5413c630 2020-11-28T21:48:19.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.299+0000 7fab517fa700 1 -- 172.21.15.162:0/2349373548 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab4c01faa0 con 0x7fab5413c630 2020-11-28T21:48:19.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.299+0000 7fab58e2b700 1 -- 172.21.15.162:0/2349373548 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab540ad800 con 0x7fab5413c630 2020-11-28T21:48:19.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.299+0000 7fab58e2b700 1 -- 172.21.15.162:0/2349373548 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab54057050 con 0x7fab5413c630 2020-11-28T21:48:19.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.299+0000 7fab517fa700 1 -- 172.21.15.162:0/2349373548 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fab4c026030 con 0x7fab5413c630 2020-11-28T21:48:19.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.303+0000 7fab517fa700 1 --2- 172.21.15.162:0/2349373548 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fab380553f0 0x7fab380578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:19.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.303+0000 7fab537fe700 1 --2- 172.21.15.162:0/2349373548 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fab380553f0 0x7fab380578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:19.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.303+0000 7fab517fa700 1 -- 172.21.15.162:0/2349373548 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fab4c06f580 con 0x7fab5413c630 2020-11-28T21:48:19.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.303+0000 7fab537fe700 1 --2- 172.21.15.162:0/2349373548 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fab380553f0 0x7fab380578b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7fab4400b000 tx=0x7fab44008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:19.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.307+0000 7fab517fa700 1 -- 172.21.15.162:0/2349373548 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fab4c045720 con 0x7fab5413c630 2020-11-28T21:48:19.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.491+0000 7fab58e2b700 1 -- 172.21.15.162:0/2349373548 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab54146e40 con 0x7fab5413c630 2020-11-28T21:48:19.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.491+0000 7fab517fa700 1 -- 172.21.15.162:0/2349373548 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+2134 (secure 0 0 0) 0x7fab4c024090 con 0x7fab5413c630 2020-11-28T21:48:19.492 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:48:19.492 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4839,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/1153453343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1153453343},{"type":"v1","addr":"172.21.15.162:6836","nonce":1153453343}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8},{"gid":4849,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2101329586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":2101329586},{"type":"v1","addr":"172.21.15.162:6837","nonce":2101329586}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8}],"filesystems":[{"mdsmap":{"epoch":11,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:47:10.330722+0000","modified":"2020-11-28T21:48:19.149314+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-28T21:48:19.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.495+0000 7fab58e2b700 1 -- 172.21.15.162:0/2349373548 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fab380553f0 msgr2=0x7fab380578b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:19.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.495+0000 7fab58e2b700 1 --2- 172.21.15.162:0/2349373548 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fab380553f0 0x7fab380578b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7fab4400b000 tx=0x7fab44008040).stop 2020-11-28T21:48:19.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.495+0000 7fab58e2b700 1 -- 172.21.15.162:0/2349373548 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab5413c630 msgr2=0x7fab540aecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:19.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.495+0000 7fab58e2b700 1 --2- 172.21.15.162:0/2349373548 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab5413c630 0x7fab540aecd0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7fab4c0032b0 tx=0x7fab4c016000).stop 2020-11-28T21:48:19.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.495+0000 7fab58e2b700 1 -- 172.21.15.162:0/2349373548 shutdown_connections 2020-11-28T21:48:19.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.495+0000 7fab58e2b700 1 --2- 172.21.15.162:0/2349373548 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fab380553f0 0x7fab380578b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:19.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.495+0000 7fab58e2b700 1 --2- 172.21.15.162:0/2349373548 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab5413c630 0x7fab540aecd0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:19.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.495+0000 7fab58e2b700 1 --2- 172.21.15.162:0/2349373548 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab5405bcb0 0x7fab54148670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:19.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.495+0000 7fab58e2b700 1 --2- 172.21.15.162:0/2349373548 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab540b1c00 0x7fab540ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:19.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.495+0000 7fab58e2b700 1 -- 172.21.15.162:0/2349373548 >> 172.21.15.162:0/2349373548 conn(0x7fab540b5da0 msgr2=0x7fab54142150 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:19.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.495+0000 7fab58e2b700 1 -- 172.21.15.162:0/2349373548 shutdown_connections 2020-11-28T21:48:19.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.495+0000 7fab58e2b700 1 -- 172.21.15.162:0/2349373548 wait complete. 2020-11-28T21:48:19.498 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 11 2020-11-28T21:48:19.512 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2020-11-28T21:48:19.624 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.623+0000 7fc4bdffa700 1 Processor -- start 2020-11-28T21:48:19.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.623+0000 7fc4bdffa700 1 -- start start 2020-11-28T21:48:19.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.623+0000 7fc4bdffa700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fc4b8060cb0 0x7fc4b8142980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:19.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.623+0000 7fc4bdffa700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4b805f380 con 0x7fc4b80b5bf0 2020-11-28T21:48:19.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.623+0000 7fc4bdffa700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4b8143930 con 0x7fc4b80b3a60 2020-11-28T21:48:19.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.623+0000 7fc4bdffa700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4b805f5b0 con 0x7fc4b8060cb0 2020-11-28T21:48:19.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.623+0000 7fc4bd7f9700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fc4b80b3a60 0x7fc4b813fbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:45870/0 (socket says 172.21.15.162:45870) 2020-11-28T21:48:19.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.623+0000 7fc4bd7f9700 1 -- 172.21.15.162:0/2770592632 learned_addr learned my addr 172.21.15.162:0/2770592632 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:48:19.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.623+0000 7fc4b7fff700 1 --2- 172.21.15.162:0/2770592632 >> v2:172.21.15.118:3300/0 conn(0x7fc4b8060cb0 0x7fc4b8142980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:19.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.627+0000 7fc4b77fe700 1 -- 172.21.15.162:0/2770592632 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2007860835 0 0) 0x7fc4b0001450 con 0x7fc4b80b3a60 2020-11-28T21:48:19.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.627+0000 7fc4b77fe700 1 -- 172.21.15.162:0/2770592632 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4a4003230 con 0x7fc4b80b3a60 2020-11-28T21:48:19.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.627+0000 7fc4b7fff700 1 -- 172.21.15.162:0/2770592632 >> v1:172.21.15.162:6789/0 conn(0x7fc4b80b3a60 legacy=0x7fc4b813fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:19.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.627+0000 7fc4b7fff700 1 -- 172.21.15.162:0/2770592632 >> v1:172.21.15.118:6789/0 conn(0x7fc4b80b5bf0 legacy=0x7fc4b8060230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:19.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.627+0000 7fc4b77fe700 1 -- 172.21.15.162:0/2770592632 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3760982319 0 0) 0x7fc4b805f380 con 0x7fc4b80b5bf0 2020-11-28T21:48:19.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.627+0000 7fc4b7fff700 1 -- 172.21.15.162:0/2770592632 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b805f850 con 0x7fc4b8060cb0 2020-11-28T21:48:19.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.627+0000 7fc4b7fff700 1 --2- 172.21.15.162:0/2770592632 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc4b8060cb0 0x7fc4b8142980 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7fc4a8002f30 tx=0x7fc4a8006dc0).ready entity=mon.0 client_cookie=c01f44a78930b01 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:19.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.627+0000 7fc4b77fe700 1 -- 172.21.15.162:0/2770592632 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4a8009800 con 0x7fc4b8060cb0 2020-11-28T21:48:19.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.627+0000 7fc4b77fe700 1 -- 172.21.15.162:0/2770592632 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4a8009960 con 0x7fc4b8060cb0 2020-11-28T21:48:19.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.627+0000 7fc4b77fe700 1 -- 172.21.15.162:0/2770592632 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4a8009cc0 con 0x7fc4b8060cb0 2020-11-28T21:48:19.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.627+0000 7fc4bdffa700 1 -- 172.21.15.162:0/2770592632 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc4b8060cb0 msgr2=0x7fc4b8142980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:19.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.627+0000 7fc4bdffa700 1 --2- 172.21.15.162:0/2770592632 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc4b8060cb0 0x7fc4b8142980 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7fc4a8002f30 tx=0x7fc4a8006dc0).stop 2020-11-28T21:48:19.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.627+0000 7fc4bdffa700 1 -- 172.21.15.162:0/2770592632 shutdown_connections 2020-11-28T21:48:19.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.627+0000 7fc4bdffa700 1 --2- 172.21.15.162:0/2770592632 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc4b8060cb0 0x7fc4b8142980 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:19.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.627+0000 7fc4bdffa700 1 -- 172.21.15.162:0/2770592632 >> 172.21.15.162:0/2770592632 conn(0x7fc4b805b970 msgr2=0x7fc4b805bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:19.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.627+0000 7fc4bdffa700 1 -- 172.21.15.162:0/2770592632 shutdown_connections 2020-11-28T21:48:19.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.627+0000 7fc4bdffa700 1 -- 172.21.15.162:0/2770592632 wait complete. 2020-11-28T21:48:19.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.627+0000 7fc4bdffa700 1 Processor -- start 2020-11-28T21:48:19.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.627+0000 7fc4bdffa700 1 -- start start 2020-11-28T21:48:19.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.631+0000 7fc4bdffa700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc4b8060cb0 0x7fc4b80aec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:19.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.631+0000 7fc4bdffa700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc4b80b3a60 0x7fc4b81458d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:19.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.631+0000 7fc4bdffa700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4b80b5bf0 0x7fc4b8145dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:19.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.631+0000 7fc4bdffa700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b80b23a0 con 0x7fc4b8060cb0 2020-11-28T21:48:19.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.631+0000 7fc4bdffa700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b80b20c0 con 0x7fc4b80b5bf0 2020-11-28T21:48:19.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.631+0000 7fc4bdffa700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b8143ed0 con 0x7fc4b80b3a60 2020-11-28T21:48:19.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.631+0000 7fc4bd7f9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4b80b5bf0 0x7fc4b8145dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:19.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.631+0000 7fc4bd7f9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4b80b5bf0 0x7fc4b8145dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39032/0 (socket says 172.21.15.162:39032) 2020-11-28T21:48:19.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.631+0000 7fc4bd7f9700 1 -- 172.21.15.162:0/1707401156 learned_addr learned my addr 172.21.15.162:0/1707401156 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:19.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.631+0000 7fc4b7fff700 1 --2- 172.21.15.162:0/1707401156 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc4b80b3a60 0x7fc4b81458d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:19.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.631+0000 7fc4bcff8700 1 --2- 172.21.15.162:0/1707401156 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc4b8060cb0 0x7fc4b80aec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:19.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.631+0000 7fc4bd7f9700 1 -- 172.21.15.162:0/1707401156 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc4b80b3a60 msgr2=0x7fc4b81458d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:19.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.631+0000 7fc4bd7f9700 1 --2- 172.21.15.162:0/1707401156 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc4b80b3a60 0x7fc4b81458d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:19.639 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.631+0000 7fc4bd7f9700 1 -- 172.21.15.162:0/1707401156 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc4b8060cb0 msgr2=0x7fc4b80aec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:19.639 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.631+0000 7fc4bd7f9700 1 --2- 172.21.15.162:0/1707401156 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc4b8060cb0 0x7fc4b80aec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:19.639 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.631+0000 7fc4bd7f9700 1 -- 172.21.15.162:0/1707401156 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b8144170 con 0x7fc4b80b5bf0 2020-11-28T21:48:19.639 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.631+0000 7fc4b7fff700 1 --2- 172.21.15.162:0/1707401156 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc4b80b3a60 0x7fc4b81458d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:48:19.639 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.631+0000 7fc4bcff8700 1 --2- 172.21.15.162:0/1707401156 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc4b8060cb0 0x7fc4b80aec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:48:19.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.631+0000 7fc4bd7f9700 1 --2- 172.21.15.162:0/1707401156 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4b80b5bf0 0x7fc4b8145dc0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7fc4b0001620 tx=0x7fc4b0007bf0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:19.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.631+0000 7fc4b5ffb700 1 -- 172.21.15.162:0/1707401156 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4b0005b70 con 0x7fc4b80b5bf0 2020-11-28T21:48:19.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.631+0000 7fc4b5ffb700 1 -- 172.21.15.162:0/1707401156 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b0005cd0 con 0x7fc4b80b5bf0 2020-11-28T21:48:19.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.631+0000 7fc4bdffa700 1 -- 172.21.15.162:0/1707401156 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4b8144410 con 0x7fc4b80b5bf0 2020-11-28T21:48:19.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.631+0000 7fc4b5ffb700 1 -- 172.21.15.162:0/1707401156 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4b0007300 con 0x7fc4b80b5bf0 2020-11-28T21:48:19.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.631+0000 7fc4bdffa700 1 -- 172.21.15.162:0/1707401156 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4b80af300 con 0x7fc4b80b5bf0 2020-11-28T21:48:19.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.631+0000 7fc4bdffa700 1 -- 172.21.15.162:0/1707401156 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4b8145740 con 0x7fc4b80b5bf0 2020-11-28T21:48:19.642 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.631+0000 7fc4b5ffb700 1 -- 172.21.15.162:0/1707401156 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc4b0010030 con 0x7fc4b80b5bf0 2020-11-28T21:48:19.642 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.635+0000 7fc4b5ffb700 1 --2- 172.21.15.162:0/1707401156 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc498055330 0x7fc4980577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:19.642 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.635+0000 7fc4b5ffb700 1 -- 172.21.15.162:0/1707401156 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fc4b006dea0 con 0x7fc4b80b5bf0 2020-11-28T21:48:19.642 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.635+0000 7fc4bcff8700 1 --2- 172.21.15.162:0/1707401156 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc498055330 0x7fc4980577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:19.642 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.635+0000 7fc4bcff8700 1 --2- 172.21.15.162:0/1707401156 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc498055330 0x7fc4980577f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7fc4ac00b000 tx=0x7fc4ac009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:19.643 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.639+0000 7fc4b5ffb700 1 -- 172.21.15.162:0/1707401156 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc4b0044160 con 0x7fc4b80b5bf0 2020-11-28T21:48:19.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:19.827+0000 7fc4bdffa700 1 -- 172.21.15.162:0/1707401156 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc4b8143930 con 0x7fc4b80b5bf0 2020-11-28T21:48:20.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.159+0000 7fc4b5ffb700 1 -- 172.21.15.162:0/1707401156 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v12) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc4b8143930 con 0x7fc4b80b5bf0 2020-11-28T21:48:20.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.163+0000 7fc4bdffa700 1 -- 172.21.15.162:0/1707401156 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc498055330 msgr2=0x7fc4980577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:20.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.163+0000 7fc4bdffa700 1 --2- 172.21.15.162:0/1707401156 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc498055330 0x7fc4980577f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7fc4ac00b000 tx=0x7fc4ac009040).stop 2020-11-28T21:48:20.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.163+0000 7fc4bdffa700 1 -- 172.21.15.162:0/1707401156 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4b80b5bf0 msgr2=0x7fc4b8145dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:20.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.163+0000 7fc4bdffa700 1 --2- 172.21.15.162:0/1707401156 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4b80b5bf0 0x7fc4b8145dc0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7fc4b0001620 tx=0x7fc4b0007bf0).stop 2020-11-28T21:48:20.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.163+0000 7fc4bdffa700 1 -- 172.21.15.162:0/1707401156 shutdown_connections 2020-11-28T21:48:20.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.163+0000 7fc4bdffa700 1 --2- 172.21.15.162:0/1707401156 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc498055330 0x7fc4980577f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:20.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.163+0000 7fc4bdffa700 1 --2- 172.21.15.162:0/1707401156 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4b80b5bf0 0x7fc4b8145dc0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:20.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.163+0000 7fc4bdffa700 1 --2- 172.21.15.162:0/1707401156 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc4b8060cb0 0x7fc4b80aec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:20.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.163+0000 7fc4bdffa700 1 --2- 172.21.15.162:0/1707401156 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc4b80b3a60 0x7fc4b81458d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:20.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.163+0000 7fc4bdffa700 1 -- 172.21.15.162:0/1707401156 >> 172.21.15.162:0/1707401156 conn(0x7fc4b805b970 msgr2=0x7fc4b80b8110 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:20.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.163+0000 7fc4bdffa700 1 -- 172.21.15.162:0/1707401156 shutdown_connections 2020-11-28T21:48:20.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.163+0000 7fc4bdffa700 1 -- 172.21.15.162:0/1707401156 wait complete. 2020-11-28T21:48:20.179 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-11-28T21:48:20.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.299+0000 7fb9eecda700 1 Processor -- start 2020-11-28T21:48:20.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.299+0000 7fb9eecda700 1 -- start start 2020-11-28T21:48:20.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9eecda700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fb9e80b3770 0x7fb9e80b2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:20.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9eecda700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb9e80aa8b0 0x7fb9e80b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:20.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9eecda700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9e8062e40 con 0x7fb9e8065750 2020-11-28T21:48:20.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9eecda700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9e80b2e30 con 0x7fb9e80b3770 2020-11-28T21:48:20.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9eecda700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e809cb50 con 0x7fb9e80aa8b0 2020-11-28T21:48:20.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9e7fff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fb9e8065750 0x7fb9e8065dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:45880/0 (socket says 172.21.15.162:45880) 2020-11-28T21:48:20.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9e7fff700 1 -- 172.21.15.162:0/2214900629 learned_addr learned my addr 172.21.15.162:0/2214900629 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:48:20.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9ed277700 1 --2- 172.21.15.162:0/2214900629 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb9e80aa8b0 0x7fb9e80b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:20.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9eca76700 1 --2- 172.21.15.162:0/2214900629 >> v2:172.21.15.118:3300/0 conn(0x7fb9e80b3770 0x7fb9e80b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:20.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9e77fe700 1 -- 172.21.15.162:0/2214900629 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2291361582 0 0) 0x7fb9e8062e40 con 0x7fb9e8065750 2020-11-28T21:48:20.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9e77fe700 1 -- 172.21.15.162:0/2214900629 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9c8003230 con 0x7fb9e8065750 2020-11-28T21:48:20.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9e77fe700 1 -- 172.21.15.162:0/2214900629 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2956804294 0 0) 0x7fb9c8003230 con 0x7fb9e8065750 2020-11-28T21:48:20.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9e77fe700 1 -- 172.21.15.162:0/2214900629 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb9e80aa8b0 msgr2=0x7fb9e80b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:20.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9e77fe700 1 --2- 172.21.15.162:0/2214900629 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb9e80aa8b0 0x7fb9e80b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:20.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9e77fe700 1 -- 172.21.15.162:0/2214900629 >> v2:172.21.15.118:3300/0 conn(0x7fb9e80b3770 msgr2=0x7fb9e80b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:20.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9e77fe700 1 --2- 172.21.15.162:0/2214900629 >> v2:172.21.15.118:3300/0 conn(0x7fb9e80b3770 0x7fb9e80b2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:20.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9e77fe700 1 -- 172.21.15.162:0/2214900629 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e809cdf0 con 0x7fb9e8065750 2020-11-28T21:48:20.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9e77fe700 1 -- 172.21.15.162:0/2214900629 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7fb9d00016d0 con 0x7fb9e8065750 2020-11-28T21:48:20.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9e77fe700 1 -- 172.21.15.162:0/2214900629 >> v1:172.21.15.162:6789/0 conn(0x7fb9e8065750 legacy=0x7fb9e8065dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:20.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9e77fe700 1 --2- 172.21.15.162:0/2214900629 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb9e80aa8b0 0x7fb9c80037a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:20.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9e77fe700 1 --2- 172.21.15.162:0/2214900629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9c8003d00 0x7fb9c80044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:20.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9e77fe700 1 --2- 172.21.15.162:0/2214900629 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb9c80049c0 0x7fb9c8004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:20.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9e77fe700 1 -- 172.21.15.162:0/2214900629 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e80b2e30 con 0x7fb9c80049c0 2020-11-28T21:48:20.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9e77fe700 1 -- 172.21.15.162:0/2214900629 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e809cb50 con 0x7fb9c8003d00 2020-11-28T21:48:20.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9e77fe700 1 -- 172.21.15.162:0/2214900629 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c80042c0 con 0x7fb9e80aa8b0 2020-11-28T21:48:20.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9e7fff700 1 --2- 172.21.15.162:0/2214900629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9c8003d00 0x7fb9c80044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:20.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9eca76700 1 --2- 172.21.15.162:0/2214900629 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb9e80aa8b0 0x7fb9c80037a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:20.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9ed277700 1 --2- 172.21.15.162:0/2214900629 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb9c80049c0 0x7fb9c8004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:20.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9e7fff700 1 -- 172.21.15.162:0/2214900629 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb9e80aa8b0 msgr2=0x7fb9c80037a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:20.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9e7fff700 1 --2- 172.21.15.162:0/2214900629 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb9e80aa8b0 0x7fb9c80037a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:20.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9e7fff700 1 -- 172.21.15.162:0/2214900629 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb9c80049c0 msgr2=0x7fb9c8004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:20.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9e7fff700 1 --2- 172.21.15.162:0/2214900629 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb9c80049c0 0x7fb9c8004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:20.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9e7fff700 1 -- 172.21.15.162:0/2214900629 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c8003230 con 0x7fb9c8003d00 2020-11-28T21:48:20.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9eca76700 1 --2- 172.21.15.162:0/2214900629 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb9e80aa8b0 0x7fb9c80037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:48:20.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9ed277700 1 --2- 172.21.15.162:0/2214900629 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb9c80049c0 0x7fb9c8004e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:48:20.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9e7fff700 1 --2- 172.21.15.162:0/2214900629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9c8003d00 0x7fb9c80044d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7fb9e809a740 tx=0x7fb9e809d090).ready entity=mon.1 client_cookie=2bef62f5b2d6ebd4 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:20.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9e77fe700 1 -- 172.21.15.162:0/2214900629 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9d000cea0 con 0x7fb9c8003d00 2020-11-28T21:48:20.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9e77fe700 1 -- 172.21.15.162:0/2214900629 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d00070d0 con 0x7fb9c8003d00 2020-11-28T21:48:20.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9e77fe700 1 -- 172.21.15.162:0/2214900629 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9d0007430 con 0x7fb9c8003d00 2020-11-28T21:48:20.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9eecda700 1 -- 172.21.15.162:0/2214900629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9c8003d00 msgr2=0x7fb9c80044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:20.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9eecda700 1 --2- 172.21.15.162:0/2214900629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9c8003d00 0x7fb9c80044d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7fb9e809a740 tx=0x7fb9e809d090).stop 2020-11-28T21:48:20.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9ed277700 1 -- 172.21.15.162:0/2214900629 reap_dead start 2020-11-28T21:48:20.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9eecda700 1 -- 172.21.15.162:0/2214900629 shutdown_connections 2020-11-28T21:48:20.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9eecda700 1 -- 172.21.15.162:0/2214900629 >> 172.21.15.162:0/2214900629 conn(0x7fb9e805b970 msgr2=0x7fb9e805bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:20.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9eecda700 1 -- 172.21.15.162:0/2214900629 shutdown_connections 2020-11-28T21:48:20.318 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.303+0000 7fb9eecda700 1 -- 172.21.15.162:0/2214900629 wait complete. 2020-11-28T21:48:20.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.307+0000 7fb9eecda700 1 Processor -- start 2020-11-28T21:48:20.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.307+0000 7fb9eecda700 1 -- start start 2020-11-28T21:48:20.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.307+0000 7fb9eecda700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb9e80b1f40 0x7fb9e81550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:20.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.307+0000 7fb9eecda700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9e8152200 0x7fb9e81555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:20.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.311+0000 7fb9eecda700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb9e80aaeb0 0x7fb9e8060310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:20.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.311+0000 7fb9eecda700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e8152710 con 0x7fb9e80aaeb0 2020-11-28T21:48:20.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.311+0000 7fb9eecda700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e80ac3e0 con 0x7fb9e8152200 2020-11-28T21:48:20.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.311+0000 7fb9eecda700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e80b2320 con 0x7fb9e80b1f40 2020-11-28T21:48:20.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.311+0000 7fb9e7fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9e8152200 0x7fb9e81555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:20.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.311+0000 7fb9eca76700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb9e80b1f40 0x7fb9e81550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:20.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.311+0000 7fb9e7fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9e8152200 0x7fb9e81555a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39050/0 (socket says 172.21.15.162:39050) 2020-11-28T21:48:20.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.311+0000 7fb9e7fff700 1 -- 172.21.15.162:0/4019863532 learned_addr learned my addr 172.21.15.162:0/4019863532 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:20.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.311+0000 7fb9ed277700 1 --2- 172.21.15.162:0/4019863532 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb9e80aaeb0 0x7fb9e8060310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:20.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.311+0000 7fb9e7fff700 1 -- 172.21.15.162:0/4019863532 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb9e80b1f40 msgr2=0x7fb9e81550b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:20.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.311+0000 7fb9e7fff700 1 --2- 172.21.15.162:0/4019863532 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb9e80b1f40 0x7fb9e81550b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:20.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.311+0000 7fb9e7fff700 1 -- 172.21.15.162:0/4019863532 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb9e80aaeb0 msgr2=0x7fb9e8060310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:20.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.311+0000 7fb9e7fff700 1 --2- 172.21.15.162:0/4019863532 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb9e80aaeb0 0x7fb9e8060310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:20.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.311+0000 7fb9e7fff700 1 -- 172.21.15.162:0/4019863532 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e80ab8e0 con 0x7fb9e8152200 2020-11-28T21:48:20.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.311+0000 7fb9e7fff700 1 --2- 172.21.15.162:0/4019863532 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9e8152200 0x7fb9e81555a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7fb9e809a740 tx=0x7fb9d0013000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:20.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.311+0000 7fb9e5ffb700 1 -- 172.21.15.162:0/4019863532 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9d001a070 con 0x7fb9e8152200 2020-11-28T21:48:20.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.311+0000 7fb9e5ffb700 1 -- 172.21.15.162:0/4019863532 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d0005c10 con 0x7fb9e8152200 2020-11-28T21:48:20.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.311+0000 7fb9e5ffb700 1 -- 172.21.15.162:0/4019863532 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9d0013450 con 0x7fb9e8152200 2020-11-28T21:48:20.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.311+0000 7fb9eecda700 1 -- 172.21.15.162:0/4019863532 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9e80abb40 con 0x7fb9e8152200 2020-11-28T21:48:20.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.311+0000 7fb9eecda700 1 -- 172.21.15.162:0/4019863532 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9e8155a90 con 0x7fb9e8152200 2020-11-28T21:48:20.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.311+0000 7fb9eecda700 1 -- 172.21.15.162:0/4019863532 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9e8057050 con 0x7fb9e8152200 2020-11-28T21:48:20.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.315+0000 7fb9e5ffb700 1 -- 172.21.15.162:0/4019863532 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb9d0010070 con 0x7fb9e8152200 2020-11-28T21:48:20.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.315+0000 7fb9e5ffb700 1 --2- 172.21.15.162:0/4019863532 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb9cc0553b0 0x7fb9cc057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:20.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.315+0000 7fb9e5ffb700 1 -- 172.21.15.162:0/4019863532 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fb9d006e9b0 con 0x7fb9e8152200 2020-11-28T21:48:20.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.315+0000 7fb9eca76700 1 --2- 172.21.15.162:0/4019863532 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb9cc0553b0 0x7fb9cc057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:20.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.315+0000 7fb9eca76700 1 --2- 172.21.15.162:0/4019863532 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb9cc0553b0 0x7fb9cc057870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7fb9d8008000 tx=0x7fb9d8008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:20.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.323+0000 7fb9e5ffb700 1 -- 172.21.15.162:0/4019863532 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb9d0044c60 con 0x7fb9e8152200 2020-11-28T21:48:20.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:20.463+0000 7fb9eecda700 1 -- 172.21.15.162:0/4019863532 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb9e80abcd0 con 0x7fb9e8152200 2020-11-28T21:48:21.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.215+0000 7fb9e5ffb700 1 -- 172.21.15.162:0/4019863532 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v35) v1 ==== 185+0+0 (secure 0 0 0) 0x7fb9d0048280 con 0x7fb9e8152200 2020-11-28T21:48:21.218 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.219+0000 7fb9eecda700 1 -- 172.21.15.162:0/4019863532 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb9cc0553b0 msgr2=0x7fb9cc057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:21.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.219+0000 7fb9eecda700 1 --2- 172.21.15.162:0/4019863532 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb9cc0553b0 0x7fb9cc057870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7fb9d8008000 tx=0x7fb9d8008040).stop 2020-11-28T21:48:21.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.219+0000 7fb9eecda700 1 -- 172.21.15.162:0/4019863532 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9e8152200 msgr2=0x7fb9e81555a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:21.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.219+0000 7fb9eecda700 1 --2- 172.21.15.162:0/4019863532 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9e8152200 0x7fb9e81555a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7fb9e809a740 tx=0x7fb9d0013000).stop 2020-11-28T21:48:21.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.219+0000 7fb9eecda700 1 -- 172.21.15.162:0/4019863532 shutdown_connections 2020-11-28T21:48:21.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.219+0000 7fb9eecda700 1 --2- 172.21.15.162:0/4019863532 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb9cc0553b0 0x7fb9cc057870 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:21.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.219+0000 7fb9eecda700 1 --2- 172.21.15.162:0/4019863532 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb9e80b1f40 0x7fb9e81550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:21.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.219+0000 7fb9eecda700 1 --2- 172.21.15.162:0/4019863532 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb9e80aaeb0 0x7fb9e8060310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:21.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.219+0000 7fb9eecda700 1 --2- 172.21.15.162:0/4019863532 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9e8152200 0x7fb9e81555a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:21.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.219+0000 7fb9eecda700 1 -- 172.21.15.162:0/4019863532 >> 172.21.15.162:0/4019863532 conn(0x7fb9e805b970 msgr2=0x7fb9e8066220 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:21.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.219+0000 7fb9eecda700 1 -- 172.21.15.162:0/4019863532 shutdown_connections 2020-11-28T21:48:21.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.219+0000 7fb9eecda700 1 -- 172.21.15.162:0/4019863532 wait complete. 2020-11-28T21:48:21.222 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_metadata' removed 2020-11-28T21:48:21.234 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-11-28T21:48:21.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.351+0000 7fc69c8cf700 1 Processor -- start 2020-11-28T21:48:21.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.351+0000 7fc69c8cf700 1 -- start start 2020-11-28T21:48:21.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.355+0000 7fc69c8cf700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc6980a9960 0x7fc6980a9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:21.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.355+0000 7fc69c8cf700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fc698097ac0 0x7fc698141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:21.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.355+0000 7fc69c8cf700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc698097f20 0x7fc698141820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:21.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.355+0000 7fc69c8cf700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6980a8f50 con 0x7fc698097ac0 2020-11-28T21:48:21.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.355+0000 7fc69c8cf700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6980b0b30 con 0x7fc6980a9960 2020-11-28T21:48:21.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.355+0000 7fc69c8cf700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6980a9180 con 0x7fc698097f20 2020-11-28T21:48:21.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.355+0000 7fc6977fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc6980a9960 0x7fc6980a9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:21.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.355+0000 7fc6977fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc6980a9960 0x7fc6980a9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39056/0 (socket says 172.21.15.162:39056) 2020-11-28T21:48:21.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.355+0000 7fc6977fe700 1 -- 172.21.15.162:0/2518454872 learned_addr learned my addr 172.21.15.162:0/2518454872 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:21.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.355+0000 7fc697fff700 1 --2- 172.21.15.162:0/2518454872 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc698097f20 0x7fc698141820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:21.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.355+0000 7fc68effd700 1 --2- 172.21.15.162:0/2518454872 >> v2:172.21.15.118:3300/0 conn(0x7fc698097ac0 0x7fc698141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:21.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.355+0000 7fc6977fe700 1 -- 172.21.15.162:0/2518454872 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc698097f20 msgr2=0x7fc698141820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:21.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.355+0000 7fc6977fe700 1 --2- 172.21.15.162:0/2518454872 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc698097f20 0x7fc698141820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:21.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.355+0000 7fc6977fe700 1 -- 172.21.15.162:0/2518454872 >> v2:172.21.15.118:3300/0 conn(0x7fc698097ac0 msgr2=0x7fc698141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:21.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.355+0000 7fc6977fe700 1 --2- 172.21.15.162:0/2518454872 >> v2:172.21.15.118:3300/0 conn(0x7fc698097ac0 0x7fc698141330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:21.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.355+0000 7fc6977fe700 1 -- 172.21.15.162:0/2518454872 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc698060c60 con 0x7fc6980a9960 2020-11-28T21:48:21.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.355+0000 7fc6977fe700 1 --2- 172.21.15.162:0/2518454872 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6980a9960 0x7fc6980a9d40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7fc688002f30 tx=0x7fc688006e80).ready entity=mon.1 client_cookie=9560846b91abae57 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:21.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.355+0000 7fc696ffd700 1 -- 172.21.15.162:0/2518454872 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc688009640 con 0x7fc6980a9960 2020-11-28T21:48:21.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.355+0000 7fc696ffd700 1 -- 172.21.15.162:0/2518454872 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6880097a0 con 0x7fc6980a9960 2020-11-28T21:48:21.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.355+0000 7fc696ffd700 1 -- 172.21.15.162:0/2518454872 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc688009b00 con 0x7fc6980a9960 2020-11-28T21:48:21.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.355+0000 7fc69c8cf700 1 -- 172.21.15.162:0/2518454872 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6980a9960 msgr2=0x7fc6980a9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:21.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.355+0000 7fc69c8cf700 1 --2- 172.21.15.162:0/2518454872 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6980a9960 0x7fc6980a9d40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7fc688002f30 tx=0x7fc688006e80).stop 2020-11-28T21:48:21.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.355+0000 7fc69c8cf700 1 -- 172.21.15.162:0/2518454872 shutdown_connections 2020-11-28T21:48:21.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.355+0000 7fc69c8cf700 1 --2- 172.21.15.162:0/2518454872 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc698097f20 0x7fc698141820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:21.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.355+0000 7fc69c8cf700 1 --2- 172.21.15.162:0/2518454872 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6980a9960 0x7fc6980a9d40 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:21.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.355+0000 7fc69c8cf700 1 --2- 172.21.15.162:0/2518454872 >> v2:172.21.15.118:3300/0 conn(0x7fc698097ac0 0x7fc698141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:21.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.355+0000 7fc69c8cf700 1 -- 172.21.15.162:0/2518454872 >> 172.21.15.162:0/2518454872 conn(0x7fc6980b5520 msgr2=0x7fc6980b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:21.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.355+0000 7fc69c8cf700 1 -- 172.21.15.162:0/2518454872 shutdown_connections 2020-11-28T21:48:21.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.355+0000 7fc69c8cf700 1 -- 172.21.15.162:0/2518454872 wait complete. 2020-11-28T21:48:21.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.359+0000 7fc69c8cf700 1 Processor -- start 2020-11-28T21:48:21.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.359+0000 7fc69c8cf700 1 -- start start 2020-11-28T21:48:21.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.359+0000 7fc69c8cf700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc698097ac0 0x7fc69805bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:21.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.359+0000 7fc69c8cf700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc698097f20 0x7fc69813b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:21.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.359+0000 7fc69c8cf700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6980a9960 0x7fc69813b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:21.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.359+0000 7fc69c8cf700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc698151950 con 0x7fc698097ac0 2020-11-28T21:48:21.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.359+0000 7fc69c8cf700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6980996e0 con 0x7fc6980a9960 2020-11-28T21:48:21.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.359+0000 7fc69c8cf700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc698099980 con 0x7fc698097f20 2020-11-28T21:48:21.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.359+0000 7fc697fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6980a9960 0x7fc69813b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:21.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.359+0000 7fc697fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6980a9960 0x7fc69813b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39066/0 (socket says 172.21.15.162:39066) 2020-11-28T21:48:21.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.359+0000 7fc697fff700 1 -- 172.21.15.162:0/3654961145 learned_addr learned my addr 172.21.15.162:0/3654961145 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:21.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.359+0000 7fc68effd700 1 --2- 172.21.15.162:0/3654961145 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc698097f20 0x7fc69813b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:21.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.359+0000 7fc6977fe700 1 --2- 172.21.15.162:0/3654961145 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc698097ac0 0x7fc69805bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:21.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.359+0000 7fc697fff700 1 -- 172.21.15.162:0/3654961145 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc698097f20 msgr2=0x7fc69813b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:21.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.359+0000 7fc697fff700 1 --2- 172.21.15.162:0/3654961145 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc698097f20 0x7fc69813b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:21.371 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.359+0000 7fc697fff700 1 -- 172.21.15.162:0/3654961145 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc698097ac0 msgr2=0x7fc69805bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:21.371 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.359+0000 7fc697fff700 1 --2- 172.21.15.162:0/3654961145 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc698097ac0 0x7fc69805bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:21.371 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.359+0000 7fc697fff700 1 -- 172.21.15.162:0/3654961145 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc698099c20 con 0x7fc6980a9960 2020-11-28T21:48:21.371 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.363+0000 7fc697fff700 1 --2- 172.21.15.162:0/3654961145 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6980a9960 0x7fc69813b520 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7fc698151c20 tx=0x7fc6900091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:21.371 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.363+0000 7fc6957fa700 1 -- 172.21.15.162:0/3654961145 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc69000eea0 con 0x7fc6980a9960 2020-11-28T21:48:21.372 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.363+0000 7fc69c8cf700 1 -- 172.21.15.162:0/3654961145 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc69813eb20 con 0x7fc6980a9960 2020-11-28T21:48:21.372 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.363+0000 7fc6957fa700 1 -- 172.21.15.162:0/3654961145 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc69000cbb0 con 0x7fc6980a9960 2020-11-28T21:48:21.372 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.363+0000 7fc6957fa700 1 -- 172.21.15.162:0/3654961145 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6900123f0 con 0x7fc6980a9960 2020-11-28T21:48:21.372 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.363+0000 7fc69c8cf700 1 -- 172.21.15.162:0/3654961145 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc69813bbb0 con 0x7fc6980a9960 2020-11-28T21:48:21.372 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.363+0000 7fc69c8cf700 1 -- 172.21.15.162:0/3654961145 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc698099db0 con 0x7fc6980a9960 2020-11-28T21:48:21.373 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.363+0000 7fc6957fa700 1 -- 172.21.15.162:0/3654961145 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc690022070 con 0x7fc6980a9960 2020-11-28T21:48:21.373 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.363+0000 7fc6957fa700 1 --2- 172.21.15.162:0/3654961145 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc67c0554e0 0x7fc67c0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:21.373 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.363+0000 7fc6957fa700 1 -- 172.21.15.162:0/3654961145 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5721+0+0 (secure 0 0 0) 0x7fc69006e360 con 0x7fc6980a9960 2020-11-28T21:48:21.373 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.363+0000 7fc6977fe700 1 --2- 172.21.15.162:0/3654961145 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc67c0554e0 0x7fc67c0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:21.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.367+0000 7fc6977fe700 1 --2- 172.21.15.162:0/3654961145 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc67c0554e0 0x7fc67c0579a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fc688019000 tx=0x7fc688015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:21.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.371+0000 7fc6957fa700 1 -- 172.21.15.162:0/3654961145 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc690044720 con 0x7fc6980a9960 2020-11-28T21:48:21.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:21.519+0000 7fc69c8cf700 1 -- 172.21.15.162:0/3654961145 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc69813bbb0 con 0x7fc6980a9960 2020-11-28T21:48:22.224 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.223+0000 7fc6957fa700 1 -- 172.21.15.162:0/3654961145 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v36) v1 ==== 173+0+0 (secure 0 0 0) 0x7fc69813bbb0 con 0x7fc6980a9960 2020-11-28T21:48:22.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.227+0000 7fc69c8cf700 1 -- 172.21.15.162:0/3654961145 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc67c0554e0 msgr2=0x7fc67c0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:22.229 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.227+0000 7fc69c8cf700 1 --2- 172.21.15.162:0/3654961145 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc67c0554e0 0x7fc67c0579a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fc688019000 tx=0x7fc688015000).stop 2020-11-28T21:48:22.229 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.227+0000 7fc69c8cf700 1 -- 172.21.15.162:0/3654961145 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6980a9960 msgr2=0x7fc69813b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:22.229 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.227+0000 7fc69c8cf700 1 --2- 172.21.15.162:0/3654961145 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6980a9960 0x7fc69813b520 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7fc698151c20 tx=0x7fc6900091a0).stop 2020-11-28T21:48:22.230 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.227+0000 7fc69c8cf700 1 -- 172.21.15.162:0/3654961145 shutdown_connections 2020-11-28T21:48:22.230 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.227+0000 7fc69c8cf700 1 --2- 172.21.15.162:0/3654961145 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc67c0554e0 0x7fc67c0579a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:22.231 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.227+0000 7fc69c8cf700 1 --2- 172.21.15.162:0/3654961145 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6980a9960 0x7fc69813b520 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:22.231 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.227+0000 7fc69c8cf700 1 --2- 172.21.15.162:0/3654961145 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc698097ac0 0x7fc69805bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:22.231 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.227+0000 7fc69c8cf700 1 --2- 172.21.15.162:0/3654961145 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc698097f20 0x7fc69813b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:22.231 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.227+0000 7fc69c8cf700 1 -- 172.21.15.162:0/3654961145 >> 172.21.15.162:0/3654961145 conn(0x7fc6980b5520 msgr2=0x7fc698142ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:22.232 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.227+0000 7fc69c8cf700 1 -- 172.21.15.162:0/3654961145 shutdown_connections 2020-11-28T21:48:22.232 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.227+0000 7fc69c8cf700 1 -- 172.21.15.162:0/3654961145 wait complete. 2020-11-28T21:48:22.232 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_data' removed 2020-11-28T21:48:22.246 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-28T21:48:22.246 INFO:tasks.ceph.mds.a:Stopping old one... 2020-11-28T21:48:22.246 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-11-28T21:48:22.247 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T21:48:22.247 INFO:tasks.ceph.mds.c:Restarting daemon 2020-11-28T21:48:22.247 INFO:tasks.ceph.mds.c:Stopping old one... 2020-11-28T21:48:22.248 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2020-11-28T21:48:22.248 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T21:48:22.249 INFO:tasks.ceph.mds.b:Restarting daemon 2020-11-28T21:48:22.249 INFO:tasks.ceph.mds.b:Stopping old one... 2020-11-28T21:48:22.249 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-11-28T21:48:22.250 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T21:48:22.258 INFO:tasks.ceph.mds.a:Stopped 2020-11-28T21:48:22.259 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-28T21:48:22.260 INFO:tasks.ceph.mds.b:Stopped 2020-11-28T21:48:22.260 INFO:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-11-28T21:48:22.261 INFO:tasks.ceph.mds.c:Stopped 2020-11-28T21:48:22.262 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2020-11-28T21:48:22.304 INFO:tasks.ceph.mds.b:Started 2020-11-28T21:48:22.304 DEBUG:teuthology.parallel:result is None 2020-11-28T21:48:22.308 INFO:tasks.ceph.mds.a:Started 2020-11-28T21:48:22.308 DEBUG:teuthology.parallel:result is None 2020-11-28T21:48:22.309 INFO:tasks.ceph.mds.c:Started 2020-11-28T21:48:22.309 DEBUG:teuthology.parallel:result is None 2020-11-28T21:48:22.310 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist clear 2020-11-28T21:48:22.383 INFO:tasks.ceph.mds.b.smithi118.stdout:starting mds.b at 2020-11-28T21:48:22.387 INFO:tasks.ceph.mds.c.smithi162.stdout:starting mds.c at 2020-11-28T21:48:22.391 INFO:tasks.ceph.mds.a.smithi162.stdout:starting mds.a at 2020-11-28T21:48:22.435 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.435+0000 7f4abefeb700 1 Processor -- start 2020-11-28T21:48:22.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.435+0000 7f4abefeb700 1 -- start start 2020-11-28T21:48:22.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.435+0000 7f4abefeb700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4ab80b5430 0x7f4ab8143360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:22.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.435+0000 7f4abefeb700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ab80b8040 con 0x7f4ab80b5a70 2020-11-28T21:48:22.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.435+0000 7f4abefeb700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ab80b8720 con 0x7f4ab80b5e50 2020-11-28T21:48:22.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.435+0000 7f4abefeb700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ab80b7600 con 0x7f4ab80b5430 2020-11-28T21:48:22.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.435+0000 7f4abe7ea700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4ab80b5430 0x7f4ab8143360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:22.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.435+0000 7f4abd7e8700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f4ab80b5e50 0x7f4ab809d220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:45940/0 (socket says 172.21.15.162:45940) 2020-11-28T21:48:22.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.435+0000 7f4abd7e8700 1 -- 172.21.15.162:0/2936828501 learned_addr learned my addr 172.21.15.162:0/2936828501 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:48:22.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.435+0000 7f4abcfe7700 1 -- 172.21.15.162:0/2936828501 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 477202246 0 0) 0x7f4ab80b8720 con 0x7f4ab80b5e50 2020-11-28T21:48:22.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.435+0000 7f4abcfe7700 1 -- 172.21.15.162:0/2936828501 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4aa0003230 con 0x7f4ab80b5e50 2020-11-28T21:48:22.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.435+0000 7f4abe7ea700 1 -- 172.21.15.162:0/2936828501 >> v1:172.21.15.162:6789/0 conn(0x7f4ab80b5e50 legacy=0x7f4ab809d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:22.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.435+0000 7f4abcfe7700 1 -- 172.21.15.162:0/2936828501 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3151798835 0 0) 0x7f4ab80b8040 con 0x7f4ab80b5a70 2020-11-28T21:48:22.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.435+0000 7f4abe7ea700 1 -- 172.21.15.162:0/2936828501 >> v1:172.21.15.118:6789/0 conn(0x7f4ab80b5a70 legacy=0x7f4ab80b6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:22.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.435+0000 7f4abe7ea700 1 -- 172.21.15.162:0/2936828501 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab8142980 con 0x7f4ab80b5430 2020-11-28T21:48:22.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.435+0000 7f4abe7ea700 1 --2- 172.21.15.162:0/2936828501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ab80b5430 0x7f4ab8143360 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f4ab80b6960 tx=0x7f4ab00091a0).ready entity=mon.1 client_cookie=703d90f7f2fc8b10 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:22.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.435+0000 7f4abcfe7700 1 -- 172.21.15.162:0/2936828501 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ab000b9b0 con 0x7f4ab80b5430 2020-11-28T21:48:22.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.435+0000 7f4abcfe7700 1 -- 172.21.15.162:0/2936828501 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab000bb10 con 0x7f4ab80b5430 2020-11-28T21:48:22.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.435+0000 7f4abcfe7700 1 -- 172.21.15.162:0/2936828501 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ab000bc80 con 0x7f4ab80b5430 2020-11-28T21:48:22.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.435+0000 7f4abefeb700 1 -- 172.21.15.162:0/2936828501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ab80b5430 msgr2=0x7f4ab8143360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:22.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.435+0000 7f4abefeb700 1 --2- 172.21.15.162:0/2936828501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ab80b5430 0x7f4ab8143360 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f4ab80b6960 tx=0x7f4ab00091a0).stop 2020-11-28T21:48:22.452 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.439+0000 7f4abefeb700 1 -- 172.21.15.162:0/2936828501 shutdown_connections 2020-11-28T21:48:22.453 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.439+0000 7f4abefeb700 1 --2- 172.21.15.162:0/2936828501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ab80b5430 0x7f4ab8143360 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:22.453 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.439+0000 7f4abefeb700 1 -- 172.21.15.162:0/2936828501 >> 172.21.15.162:0/2936828501 conn(0x7f4ab805b970 msgr2=0x7f4ab805bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:22.453 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.439+0000 7f4abefeb700 1 -- 172.21.15.162:0/2936828501 shutdown_connections 2020-11-28T21:48:22.453 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.439+0000 7f4abefeb700 1 -- 172.21.15.162:0/2936828501 wait complete. 2020-11-28T21:48:22.454 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.439+0000 7f4abefeb700 1 Processor -- start 2020-11-28T21:48:22.454 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.439+0000 7f4abefeb700 1 -- start start 2020-11-28T21:48:22.454 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.439+0000 7f4abefeb700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ab80b5430 0x7f4ab80b2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:22.454 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.439+0000 7f4abefeb700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4ab80b5a70 0x7f4ab80ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:22.454 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.439+0000 7f4abefeb700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ab80b5e50 0x7f4ab80aec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:22.455 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.439+0000 7f4abefeb700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab80ad180 con 0x7f4ab80b5e50 2020-11-28T21:48:22.455 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.439+0000 7f4abefeb700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab80b69a0 con 0x7f4ab80b5430 2020-11-28T21:48:22.455 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.439+0000 7f4abefeb700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab805f580 con 0x7f4ab80b5a70 2020-11-28T21:48:22.455 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.439+0000 7f4abdfe9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ab80b5430 0x7f4ab80b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:22.456 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.439+0000 7f4abdfe9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ab80b5430 0x7f4ab80b2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39100/0 (socket says 172.21.15.162:39100) 2020-11-28T21:48:22.458 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.439+0000 7f4abdfe9700 1 -- 172.21.15.162:0/849505094 learned_addr learned my addr 172.21.15.162:0/849505094 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:22.458 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.439+0000 7f4abd7e8700 1 --2- 172.21.15.162:0/849505094 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4ab80b5a70 0x7f4ab80ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:22.458 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.439+0000 7f4abe7ea700 1 --2- 172.21.15.162:0/849505094 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ab80b5e50 0x7f4ab80aec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:22.458 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.439+0000 7f4abdfe9700 1 -- 172.21.15.162:0/849505094 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4ab80b5a70 msgr2=0x7f4ab80ad8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:22.459 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.439+0000 7f4abdfe9700 1 --2- 172.21.15.162:0/849505094 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4ab80b5a70 0x7f4ab80ad8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:22.459 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.439+0000 7f4abdfe9700 1 -- 172.21.15.162:0/849505094 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ab80b5e50 msgr2=0x7f4ab80aec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:22.459 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.439+0000 7f4abdfe9700 1 --2- 172.21.15.162:0/849505094 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ab80b5e50 0x7f4ab80aec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:22.459 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.439+0000 7f4abdfe9700 1 -- 172.21.15.162:0/849505094 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab80b0710 con 0x7f4ab80b5430 2020-11-28T21:48:22.459 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.439+0000 7f4abe7ea700 1 --2- 172.21.15.162:0/849505094 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ab80b5e50 0x7f4ab80aec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:48:22.460 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.439+0000 7f4abdfe9700 1 --2- 172.21.15.162:0/849505094 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ab80b5430 0x7f4ab80b2850 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f4aa8000b80 tx=0x7f4aa8007a40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:22.460 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.439+0000 7f4aa6ffd700 1 -- 172.21.15.162:0/849505094 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4aa800e4a0 con 0x7f4ab80b5430 2020-11-28T21:48:22.460 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.439+0000 7f4aa6ffd700 1 -- 172.21.15.162:0/849505094 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa800e600 con 0x7f4ab80b5430 2020-11-28T21:48:22.460 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.439+0000 7f4abefeb700 1 -- 172.21.15.162:0/849505094 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ab80b0ed0 con 0x7f4ab80b5430 2020-11-28T21:48:22.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.439+0000 7f4aa6ffd700 1 -- 172.21.15.162:0/849505094 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4aa800e960 con 0x7f4ab80b5430 2020-11-28T21:48:22.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.439+0000 7f4abefeb700 1 -- 172.21.15.162:0/849505094 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ab80adde0 con 0x7f4ab80b5430 2020-11-28T21:48:22.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.443+0000 7f4aa6ffd700 1 -- 172.21.15.162:0/849505094 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4aa8019030 con 0x7f4ab80b5430 2020-11-28T21:48:22.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.443+0000 7f4abefeb700 1 -- 172.21.15.162:0/849505094 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ab80b08a0 con 0x7f4ab80b5430 2020-11-28T21:48:22.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.443+0000 7f4aa6ffd700 1 --2- 172.21.15.162:0/849505094 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4a980554e0 0x7f4a980579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:22.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.443+0000 7f4aa6ffd700 1 -- 172.21.15.162:0/849505094 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f4aa806d1b0 con 0x7f4ab80b5430 2020-11-28T21:48:22.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.443+0000 7f4abd7e8700 1 --2- 172.21.15.162:0/849505094 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4a980554e0 0x7f4a980579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:22.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.443+0000 7f4abd7e8700 1 --2- 172.21.15.162:0/849505094 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4a980554e0 0x7f4a980579a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f4ab400b000 tx=0x7f4ab4009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:22.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.447+0000 7f4aa6ffd700 1 -- 172.21.15.162:0/849505094 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4aa8041df0 con 0x7f4ab80b5430 2020-11-28T21:48:22.600 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:22.599+0000 7f4abefeb700 1 -- 172.21.15.162:0/849505094 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f4ab8057050 con 0x7f4ab80b5430 2020-11-28T21:48:23.428 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.427+0000 7f4aa6ffd700 1 -- 172.21.15.162:0/849505094 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v39) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4aa8043840 con 0x7f4ab80b5430 2020-11-28T21:48:23.432 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.431+0000 7f4abefeb700 1 -- 172.21.15.162:0/849505094 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4a980554e0 msgr2=0x7f4a980579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:23.432 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.431+0000 7f4abefeb700 1 --2- 172.21.15.162:0/849505094 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4a980554e0 0x7f4a980579a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f4ab400b000 tx=0x7f4ab4009040).stop 2020-11-28T21:48:23.433 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.431+0000 7f4abefeb700 1 -- 172.21.15.162:0/849505094 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ab80b5430 msgr2=0x7f4ab80b2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:23.433 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.431+0000 7f4abefeb700 1 --2- 172.21.15.162:0/849505094 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ab80b5430 0x7f4ab80b2850 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f4aa8000b80 tx=0x7f4aa8007a40).stop 2020-11-28T21:48:23.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.431+0000 7f4abefeb700 1 -- 172.21.15.162:0/849505094 shutdown_connections 2020-11-28T21:48:23.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.431+0000 7f4abefeb700 1 --2- 172.21.15.162:0/849505094 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4a980554e0 0x7f4a980579a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:23.435 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.431+0000 7f4abefeb700 1 --2- 172.21.15.162:0/849505094 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ab80b5e50 0x7f4ab80aec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:23.435 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.431+0000 7f4abefeb700 1 --2- 172.21.15.162:0/849505094 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ab80b5430 0x7f4ab80b2850 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:23.435 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.431+0000 7f4abefeb700 1 --2- 172.21.15.162:0/849505094 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4ab80b5a70 0x7f4ab80ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:23.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.431+0000 7f4abefeb700 1 -- 172.21.15.162:0/849505094 >> 172.21.15.162:0/849505094 conn(0x7f4ab805b970 msgr2=0x7f4ab8141fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:23.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.431+0000 7f4abefeb700 1 -- 172.21.15.162:0/849505094 shutdown_connections 2020-11-28T21:48:23.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.431+0000 7f4abefeb700 1 -- 172.21.15.162:0/849505094 wait complete. 2020-11-28T21:48:23.437 INFO:teuthology.orchestra.run.smithi162.stderr: removed all blacklist entries 2020-11-28T21:48:23.447 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2020-11-28T21:48:23.448 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth ls --format=json-pretty 2020-11-28T21:48:23.566 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.567+0000 7f0a3cfa9700 1 Processor -- start 2020-11-28T21:48:23.566 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.567+0000 7f0a3cfa9700 1 -- start start 2020-11-28T21:48:23.568 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.567+0000 7f0a3cfa9700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0a380b1990 0x7f0a38140890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:23.569 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.567+0000 7f0a3cfa9700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0a380b1d70 0x7f0a3814a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:23.569 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.567+0000 7f0a3cfa9700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a380b2870 con 0x7f0a380b11b0 2020-11-28T21:48:23.570 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.567+0000 7f0a3cfa9700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a38146e20 con 0x7f0a380b1990 2020-11-28T21:48:23.570 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.567+0000 7f0a3cfa9700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a380b22e0 con 0x7f0a380b1d70 2020-11-28T21:48:23.570 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.567+0000 7f0a3659c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0a380b1990 0x7f0a38140890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:23.570 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.567+0000 7f0a3659c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0a380b1990 0x7f0a38140890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39110/0 (socket says 172.21.15.162:39110) 2020-11-28T21:48:23.571 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.567+0000 7f0a3659c700 1 -- 172.21.15.162:0/1550043978 learned_addr learned my addr 172.21.15.162:0/1550043978 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:23.571 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.567+0000 7f0a36d9d700 1 --2- 172.21.15.162:0/1550043978 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0a380b1d70 0x7f0a3814a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:23.571 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.567+0000 7f0a3659c700 1 -- 172.21.15.162:0/1550043978 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0a380b1d70 msgr2=0x7f0a3814a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:23.571 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.567+0000 7f0a3659c700 1 --2- 172.21.15.162:0/1550043978 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0a380b1d70 0x7f0a3814a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:23.571 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.567+0000 7f0a3659c700 1 -- 172.21.15.162:0/1550043978 >> v1:172.21.15.118:6789/0 conn(0x7f0a380b11b0 legacy=0x7f0a38140d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:23.572 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.567+0000 7f0a3659c700 1 -- 172.21.15.162:0/1550043978 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a380b0740 con 0x7f0a380b1990 2020-11-28T21:48:23.572 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.567+0000 7f0a3659c700 1 --2- 172.21.15.162:0/1550043978 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a380b1990 0x7f0a38140890 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f0a28002f30 tx=0x7f0a28006e80).ready entity=mon.1 client_cookie=a8ff41bf7870ba65 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:23.572 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.567+0000 7f0a35d9b700 1 -- 172.21.15.162:0/1550043978 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a280097f0 con 0x7f0a380b1990 2020-11-28T21:48:23.572 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.567+0000 7f0a35d9b700 1 -- 172.21.15.162:0/1550043978 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a28009950 con 0x7f0a380b1990 2020-11-28T21:48:23.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.567+0000 7f0a35d9b700 1 -- 172.21.15.162:0/1550043978 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a28009cb0 con 0x7f0a380b1990 2020-11-28T21:48:23.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.567+0000 7f0a3cfa9700 1 -- 172.21.15.162:0/1550043978 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a380b1990 msgr2=0x7f0a38140890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:23.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.567+0000 7f0a3cfa9700 1 --2- 172.21.15.162:0/1550043978 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a380b1990 0x7f0a38140890 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f0a28002f30 tx=0x7f0a28006e80).stop 2020-11-28T21:48:23.576 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.567+0000 7f0a3cfa9700 1 -- 172.21.15.162:0/1550043978 shutdown_connections 2020-11-28T21:48:23.576 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.567+0000 7f0a3cfa9700 1 --2- 172.21.15.162:0/1550043978 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0a380b1d70 0x7f0a3814a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:23.576 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.567+0000 7f0a3cfa9700 1 --2- 172.21.15.162:0/1550043978 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a380b1990 0x7f0a38140890 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:23.576 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.567+0000 7f0a3cfa9700 1 -- 172.21.15.162:0/1550043978 >> 172.21.15.162:0/1550043978 conn(0x7f0a3805b970 msgr2=0x7f0a3805bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:23.577 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.571+0000 7f0a3cfa9700 1 -- 172.21.15.162:0/1550043978 shutdown_connections 2020-11-28T21:48:23.577 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.571+0000 7f0a3cfa9700 1 -- 172.21.15.162:0/1550043978 wait complete. 2020-11-28T21:48:23.577 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.571+0000 7f0a3cfa9700 1 Processor -- start 2020-11-28T21:48:23.577 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.571+0000 7f0a3cfa9700 1 -- start start 2020-11-28T21:48:23.578 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.575+0000 7f0a3cfa9700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0a380b11b0 0x7f0a3814db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:23.578 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.575+0000 7f0a3cfa9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a380b1990 0x7f0a38143fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:23.578 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.575+0000 7f0a3cfa9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a380b1d70 0x7f0a380b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:23.578 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.575+0000 7f0a3cfa9700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a38142870 con 0x7f0a380b1d70 2020-11-28T21:48:23.579 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.575+0000 7f0a3cfa9700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a38144490 con 0x7f0a380b1990 2020-11-28T21:48:23.579 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.575+0000 7f0a3cfa9700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a380b3640 con 0x7f0a380b11b0 2020-11-28T21:48:23.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.575+0000 7f0a2dd9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a380b1990 0x7f0a38143fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:23.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.575+0000 7f0a2dd9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a380b1990 0x7f0a38143fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39118/0 (socket says 172.21.15.162:39118) 2020-11-28T21:48:23.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.575+0000 7f0a2dd9b700 1 -- 172.21.15.162:0/74991523 learned_addr learned my addr 172.21.15.162:0/74991523 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:23.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.575+0000 7f0a3659c700 1 --2- 172.21.15.162:0/74991523 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0a380b11b0 0x7f0a3814db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:23.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.575+0000 7f0a36d9d700 1 --2- 172.21.15.162:0/74991523 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a380b1d70 0x7f0a380b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:23.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.575+0000 7f0a2dd9b700 1 -- 172.21.15.162:0/74991523 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0a380b11b0 msgr2=0x7f0a3814db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:23.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.575+0000 7f0a2dd9b700 1 --2- 172.21.15.162:0/74991523 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0a380b11b0 0x7f0a3814db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:23.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.575+0000 7f0a2dd9b700 1 -- 172.21.15.162:0/74991523 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a380b1d70 msgr2=0x7f0a380b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:23.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.575+0000 7f0a2dd9b700 1 --2- 172.21.15.162:0/74991523 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a380b1d70 0x7f0a380b3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:23.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.575+0000 7f0a2dd9b700 1 -- 172.21.15.162:0/74991523 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a38141640 con 0x7f0a380b1990 2020-11-28T21:48:23.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.575+0000 7f0a2dd9b700 1 --2- 172.21.15.162:0/74991523 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a380b1990 0x7f0a38143fa0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7f0a38141560 tx=0x7f0a20007bb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:23.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.575+0000 7f0a2ffff700 1 -- 172.21.15.162:0/74991523 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a20007030 con 0x7f0a380b1990 2020-11-28T21:48:23.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.575+0000 7f0a2ffff700 1 -- 172.21.15.162:0/74991523 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a20007190 con 0x7f0a380b1990 2020-11-28T21:48:23.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.575+0000 7f0a2ffff700 1 -- 172.21.15.162:0/74991523 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a200064e0 con 0x7f0a380b1990 2020-11-28T21:48:23.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.575+0000 7f0a3cfa9700 1 -- 172.21.15.162:0/74991523 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a381418e0 con 0x7f0a380b1990 2020-11-28T21:48:23.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.575+0000 7f0a3cfa9700 1 -- 172.21.15.162:0/74991523 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a3814e3c0 con 0x7f0a380b1990 2020-11-28T21:48:23.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.575+0000 7f0a3cfa9700 1 -- 172.21.15.162:0/74991523 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a38057050 con 0x7f0a380b1990 2020-11-28T21:48:23.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.575+0000 7f0a2ffff700 1 -- 172.21.15.162:0/74991523 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0a20010070 con 0x7f0a380b1990 2020-11-28T21:48:23.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.579+0000 7f0a2ffff700 1 --2- 172.21.15.162:0/74991523 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0a1c055440 0x7f0a1c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:23.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.579+0000 7f0a2ffff700 1 -- 172.21.15.162:0/74991523 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f0a2006d160 con 0x7f0a380b1990 2020-11-28T21:48:23.586 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.579+0000 7f0a3659c700 1 --2- 172.21.15.162:0/74991523 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0a1c055440 0x7f0a1c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:23.586 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.579+0000 7f0a3659c700 1 --2- 172.21.15.162:0/74991523 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0a1c055440 0x7f0a1c057900 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f0a2801c000 tx=0x7f0a28015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:23.586 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.583+0000 7f0a2ffff700 1 -- 172.21.15.162:0/74991523 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0a20043790 con 0x7f0a380b1990 2020-11-28T21:48:23.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.771+0000 7f0a3cfa9700 1 -- 172.21.15.162:0/74991523 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f0a380b50a0 con 0x7f0a380b1990 2020-11-28T21:48:23.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.771+0000 7f0a2ffff700 1 -- 172.21.15.162:0/74991523 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+4255 (secure 0 0 0) 0x7f0a20018030 con 0x7f0a380b1990 2020-11-28T21:48:23.772 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:48:23.772 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2020-11-28T21:48:23.772 INFO:teuthology.orchestra.run.smithi162.stdout: "auth_dump": [ 2020-11-28T21:48:23.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:23.773 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mds.a", 2020-11-28T21:48:23.773 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfWxUpKxAAIUDwQRKwz8H4Qk9qE356Bg==", 2020-11-28T21:48:23.773 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:48:23.773 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:48:23.774 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:48:23.774 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:48:23.774 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:48:23.774 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:23.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:23.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:23.775 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mds.b", 2020-11-28T21:48:23.775 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfKY2zORAAaO65etMZiYcaTDNX0nsEPw==", 2020-11-28T21:48:23.775 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:48:23.775 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:48:23.776 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:48:23.776 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:48:23.776 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:48:23.776 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:23.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:23.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:23.777 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mds.c", 2020-11-28T21:48:23.777 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfdQbRMxAAWlcjesxTZzZnvKCQneSjjQ==", 2020-11-28T21:48:23.777 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:48:23.778 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:48:23.778 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:48:23.778 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:48:23.778 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:48:23.778 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:23.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:23.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:23.779 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.0", 2020-11-28T21:48:23.779 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQADxcJfF3WLLxAAxsB5YPjH1yjv/vhlRbijqw==", 2020-11-28T21:48:23.779 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:48:23.780 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:48:23.780 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:48:23.780 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:48:23.780 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:23.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:23.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:23.781 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.1", 2020-11-28T21:48:23.781 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAGxcJfKYbBFhAA1x4DzFtkQQv6uWLIdOi1qQ==", 2020-11-28T21:48:23.781 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:48:23.781 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:48:23.782 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:48:23.782 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:48:23.782 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:23.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:23.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:23.783 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.2", 2020-11-28T21:48:23.783 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAIxcJfvcYUOhAASENjM0HfAxxyFe13mmE36Q==", 2020-11-28T21:48:23.783 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:48:23.783 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:48:23.783 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:48:23.784 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:48:23.784 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:23.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:23.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:23.784 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.3", 2020-11-28T21:48:23.785 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQALxcJfc/OHIRAAFYx3s8Ssttt15hjoH04tow==", 2020-11-28T21:48:23.785 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:48:23.785 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:48:23.785 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:48:23.785 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:48:23.786 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:23.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:23.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:23.786 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.4", 2020-11-28T21:48:23.786 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAPxcJffGWqCRAAb8CCN5oUPkdDh6BJH5HEnw==", 2020-11-28T21:48:23.787 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:48:23.787 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:48:23.787 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:48:23.787 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:48:23.787 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:23.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:23.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:23.788 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.5", 2020-11-28T21:48:23.788 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQARxcJfo8GKKRAAfbsCDb4L2wULiKmHyzu7/A==", 2020-11-28T21:48:23.788 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:48:23.789 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:48:23.789 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:48:23.789 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:48:23.789 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:23.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:23.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:23.790 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.6", 2020-11-28T21:48:23.790 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAUxcJf8Y2MEBAAqQX6rX3LK0zodlI2wK4JrA==", 2020-11-28T21:48:23.790 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:48:23.791 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:48:23.791 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:48:23.791 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:48:23.791 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:23.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:23.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:23.792 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.7", 2020-11-28T21:48:23.792 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAWxcJfCLBdMhAAf8QixQMQDrfZOGWN8rHe5g==", 2020-11-28T21:48:23.792 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:48:23.792 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:48:23.793 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:48:23.793 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:48:23.793 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:23.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:23.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:23.794 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "client.0", 2020-11-28T21:48:23.794 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQABxcJfS9vDAxAAUk4FU8FVt4if/CRH5ZisOA==", 2020-11-28T21:48:23.794 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:48:23.794 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:48:23.795 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow r", 2020-11-28T21:48:23.795 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow rw pool=cephfs_data" 2020-11-28T21:48:23.795 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:23.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:23.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:23.796 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "client.1", 2020-11-28T21:48:23.797 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQABxcJfuCU8BhAAXUSoSK3D0XRo58uHNzcjXg==", 2020-11-28T21:48:23.797 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:48:23.797 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:48:23.797 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow r", 2020-11-28T21:48:23.797 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow rw pool=cephfs_data" 2020-11-28T21:48:23.798 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:23.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:23.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:23.798 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "client.admin", 2020-11-28T21:48:23.799 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfdlTWBxAApzAxlVjzdnETQF//Vnsg5w==", 2020-11-28T21:48:23.799 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:48:23.799 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow *", 2020-11-28T21:48:23.799 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:48:23.799 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:48:23.800 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:48:23.800 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:23.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:23.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:23.800 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mgr.x", 2020-11-28T21:48:23.801 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfZhdvKBAAT7zCCsD2OVCHjvddGLxQ/A==", 2020-11-28T21:48:23.801 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:48:23.801 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow *", 2020-11-28T21:48:23.801 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow profile mgr", 2020-11-28T21:48:23.801 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:48:23.802 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:23.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:23.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:23.802 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mgr.y", 2020-11-28T21:48:23.802 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfs8fvJRAAP0Atp074EeMHiTtpCCVHuA==", 2020-11-28T21:48:23.803 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:48:23.803 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow *", 2020-11-28T21:48:23.803 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow profile mgr", 2020-11-28T21:48:23.803 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:48:23.803 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:23.804 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:23.804 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:48:23.804 INFO:teuthology.orchestra.run.smithi162.stdout:} 2020-11-28T21:48:23.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.771+0000 7f0a3cfa9700 1 -- 172.21.15.162:0/74991523 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0a1c055440 msgr2=0x7f0a1c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:23.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.771+0000 7f0a3cfa9700 1 --2- 172.21.15.162:0/74991523 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0a1c055440 0x7f0a1c057900 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f0a2801c000 tx=0x7f0a28015000).stop 2020-11-28T21:48:23.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.771+0000 7f0a3cfa9700 1 -- 172.21.15.162:0/74991523 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a380b1990 msgr2=0x7f0a38143fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:23.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.771+0000 7f0a3cfa9700 1 --2- 172.21.15.162:0/74991523 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a380b1990 0x7f0a38143fa0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7f0a38141560 tx=0x7f0a20007bb0).stop 2020-11-28T21:48:23.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.771+0000 7f0a3cfa9700 1 -- 172.21.15.162:0/74991523 shutdown_connections 2020-11-28T21:48:23.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.771+0000 7f0a3cfa9700 1 --2- 172.21.15.162:0/74991523 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0a1c055440 0x7f0a1c057900 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:23.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.771+0000 7f0a3cfa9700 1 --2- 172.21.15.162:0/74991523 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0a380b11b0 0x7f0a3814db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:23.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.771+0000 7f0a3cfa9700 1 --2- 172.21.15.162:0/74991523 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a380b1d70 0x7f0a380b3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:23.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.771+0000 7f0a3cfa9700 1 --2- 172.21.15.162:0/74991523 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a380b1990 0x7f0a38143fa0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:23.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.771+0000 7f0a3cfa9700 1 -- 172.21.15.162:0/74991523 >> 172.21.15.162:0/74991523 conn(0x7f0a3805b970 msgr2=0x7f0a381481e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:23.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.771+0000 7f0a3cfa9700 1 -- 172.21.15.162:0/74991523 shutdown_connections 2020-11-28T21:48:23.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.771+0000 7f0a3cfa9700 1 -- 172.21.15.162:0/74991523 wait complete. 2020-11-28T21:48:23.809 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:48:23.916 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.915+0000 7f03a5e82700 1 Processor -- start 2020-11-28T21:48:23.916 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.915+0000 7f03a5e82700 1 -- start start 2020-11-28T21:48:23.918 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.919+0000 7f03a5e82700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f03a00603d0 0x7f03a00b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:23.918 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.919+0000 7f03a5e82700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f03a00b4c70 0x7f03a0143020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:23.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.919+0000 7f03a5e82700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03a00b4110 con 0x7f03a0142a00 2020-11-28T21:48:23.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.919+0000 7f03a5e82700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03a0060f60 con 0x7f03a00b4c70 2020-11-28T21:48:23.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.919+0000 7f03a5e82700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03a00b43f0 con 0x7f03a00603d0 2020-11-28T21:48:23.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.919+0000 7f03a4e80700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f03a00603d0 0x7f03a00b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:23.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.919+0000 7f03a4e80700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f03a00603d0 0x7f03a00b3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39124/0 (socket says 172.21.15.162:39124) 2020-11-28T21:48:23.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.919+0000 7f03a4e80700 1 -- 172.21.15.162:0/2900793835 learned_addr learned my addr 172.21.15.162:0/2900793835 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:23.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.919+0000 7f03a5681700 1 --2- 172.21.15.162:0/2900793835 >> v2:172.21.15.118:3300/0 conn(0x7f03a00b4c70 0x7f03a0143020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:23.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.919+0000 7f03a4e80700 1 -- 172.21.15.162:0/2900793835 >> v2:172.21.15.118:3300/0 conn(0x7f03a00b4c70 msgr2=0x7f03a0143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:23.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.919+0000 7f03a4e80700 1 --2- 172.21.15.162:0/2900793835 >> v2:172.21.15.118:3300/0 conn(0x7f03a00b4c70 0x7f03a0143020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:23.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.919+0000 7f03a4e80700 1 -- 172.21.15.162:0/2900793835 >> v1:172.21.15.118:6789/0 conn(0x7f03a0142a00 legacy=0x7f03a009cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:23.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.919+0000 7f03a4e80700 1 -- 172.21.15.162:0/2900793835 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a00b4690 con 0x7f03a00603d0 2020-11-28T21:48:23.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.919+0000 7f03a4e80700 1 --2- 172.21.15.162:0/2900793835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03a00603d0 0x7f03a00b3a70 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7f0394002f30 tx=0x7f0394006e80).ready entity=mon.1 client_cookie=e5b3433da13da607 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:23.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.919+0000 7f039f7fe700 1 -- 172.21.15.162:0/2900793835 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f03940097f0 con 0x7f03a00603d0 2020-11-28T21:48:23.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.919+0000 7f039f7fe700 1 -- 172.21.15.162:0/2900793835 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0394009950 con 0x7f03a00603d0 2020-11-28T21:48:23.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.919+0000 7f039f7fe700 1 -- 172.21.15.162:0/2900793835 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0394009cb0 con 0x7f03a00603d0 2020-11-28T21:48:23.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.919+0000 7f03a5e82700 1 -- 172.21.15.162:0/2900793835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03a00603d0 msgr2=0x7f03a00b3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:23.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.919+0000 7f03a5e82700 1 --2- 172.21.15.162:0/2900793835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03a00603d0 0x7f03a00b3a70 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7f0394002f30 tx=0x7f0394006e80).stop 2020-11-28T21:48:23.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.919+0000 7f03a5e82700 1 -- 172.21.15.162:0/2900793835 shutdown_connections 2020-11-28T21:48:23.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.919+0000 7f03a5e82700 1 --2- 172.21.15.162:0/2900793835 >> v2:172.21.15.118:3300/0 conn(0x7f03a00b4c70 0x7f03a0143020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:23.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.919+0000 7f03a5e82700 1 --2- 172.21.15.162:0/2900793835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03a00603d0 0x7f03a00b3a70 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:23.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.919+0000 7f03a5e82700 1 -- 172.21.15.162:0/2900793835 >> 172.21.15.162:0/2900793835 conn(0x7f03a0066a40 msgr2=0x7f03a0066e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:23.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.919+0000 7f03a5e82700 1 -- 172.21.15.162:0/2900793835 shutdown_connections 2020-11-28T21:48:23.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.919+0000 7f03a5e82700 1 -- 172.21.15.162:0/2900793835 wait complete. 2020-11-28T21:48:23.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.923+0000 7f03a5e82700 1 Processor -- start 2020-11-28T21:48:23.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.923+0000 7f03a5e82700 1 -- start start 2020-11-28T21:48:23.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.923+0000 7f03a5e82700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f03a00603d0 0x7f03a00b7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:23.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.923+0000 7f03a5e82700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03a00b4c70 0x7f03a00b2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:23.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.923+0000 7f03a5e82700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f03a00b2000 0x7f03a00b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:23.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.923+0000 7f03a5e82700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a00b2510 con 0x7f03a00b2000 2020-11-28T21:48:23.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.923+0000 7f03a5e82700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a0143700 con 0x7f03a00b4c70 2020-11-28T21:48:23.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.923+0000 7f03a5e82700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a00b8900 con 0x7f03a00603d0 2020-11-28T21:48:23.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.923+0000 7f039ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03a00b4c70 0x7f03a00b2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:23.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.923+0000 7f039ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03a00b4c70 0x7f03a00b2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39134/0 (socket says 172.21.15.162:39134) 2020-11-28T21:48:23.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.923+0000 7f039ffff700 1 -- 172.21.15.162:0/106783815 learned_addr learned my addr 172.21.15.162:0/106783815 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:23.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.923+0000 7f03a4e80700 1 --2- 172.21.15.162:0/106783815 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f03a00603d0 0x7f03a00b7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:23.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.923+0000 7f03a5681700 1 --2- 172.21.15.162:0/106783815 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f03a00b2000 0x7f03a00b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:23.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.923+0000 7f039ffff700 1 -- 172.21.15.162:0/106783815 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f03a00603d0 msgr2=0x7f03a00b7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:23.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.923+0000 7f039ffff700 1 --2- 172.21.15.162:0/106783815 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f03a00603d0 0x7f03a00b7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:23.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.923+0000 7f039ffff700 1 -- 172.21.15.162:0/106783815 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f03a00b2000 msgr2=0x7f03a00b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:23.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.923+0000 7f039ffff700 1 --2- 172.21.15.162:0/106783815 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f03a00b2000 0x7f03a00b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:23.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.923+0000 7f039ffff700 1 -- 172.21.15.162:0/106783815 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a00b8ba0 con 0x7f03a00b4c70 2020-11-28T21:48:23.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.923+0000 7f039ffff700 1 --2- 172.21.15.162:0/106783815 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03a00b4c70 0x7f03a00b2a00 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7f03a00b9c10 tx=0x7f0390007c20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:23.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.923+0000 7f039dffb700 1 -- 172.21.15.162:0/106783815 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0390013410 con 0x7f03a00b4c70 2020-11-28T21:48:23.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.923+0000 7f039dffb700 1 -- 172.21.15.162:0/106783815 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0390013570 con 0x7f03a00b4c70 2020-11-28T21:48:23.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.923+0000 7f03a5e82700 1 -- 172.21.15.162:0/106783815 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03a00b8e40 con 0x7f03a00b4c70 2020-11-28T21:48:23.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.923+0000 7f039dffb700 1 -- 172.21.15.162:0/106783815 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f03900138d0 con 0x7f03a00b4c70 2020-11-28T21:48:23.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.923+0000 7f03a5e82700 1 -- 172.21.15.162:0/106783815 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03a00b33e0 con 0x7f03a00b4c70 2020-11-28T21:48:23.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.927+0000 7f03a5e82700 1 -- 172.21.15.162:0/106783815 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03a0057050 con 0x7f03a00b4c70 2020-11-28T21:48:23.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.927+0000 7f039dffb700 1 -- 172.21.15.162:0/106783815 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0390010030 con 0x7f03a00b4c70 2020-11-28T21:48:23.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.927+0000 7f039dffb700 1 --2- 172.21.15.162:0/106783815 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f03800554e0 0x7f03800579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:23.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.927+0000 7f039dffb700 1 -- 172.21.15.162:0/106783815 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f039006d920 con 0x7f03a00b4c70 2020-11-28T21:48:23.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.927+0000 7f03a4e80700 1 --2- 172.21.15.162:0/106783815 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f03800554e0 0x7f03800579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:23.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.927+0000 7f03a4e80700 1 --2- 172.21.15.162:0/106783815 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f03800554e0 0x7f03800579a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f039401c000 tx=0x7f0394015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:23.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:23.935+0000 7f039dffb700 1 -- 172.21.15.162:0/106783815 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0390043f50 con 0x7f03a00b4c70 2020-11-28T21:48:24.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.071+0000 7f03a5e82700 1 -- 172.21.15.162:0/106783815 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03a0069b60 con 0x7f03a00b4c70 2020-11-28T21:48:24.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.075+0000 7f039dffb700 1 -- 172.21.15.162:0/106783815 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+13277 (secure 0 0 0) 0x7f0390047570 con 0x7f03a00b4c70 2020-11-28T21:48:24.077 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:48:24.077 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":39,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:48:23.420533+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:48:24.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.075+0000 7f03a5e82700 1 -- 172.21.15.162:0/106783815 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f03800554e0 msgr2=0x7f03800579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:24.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.075+0000 7f03a5e82700 1 --2- 172.21.15.162:0/106783815 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f03800554e0 0x7f03800579a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f039401c000 tx=0x7f0394015000).stop 2020-11-28T21:48:24.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.075+0000 7f03a5e82700 1 -- 172.21.15.162:0/106783815 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03a00b4c70 msgr2=0x7f03a00b2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:24.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.075+0000 7f03a5e82700 1 --2- 172.21.15.162:0/106783815 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03a00b4c70 0x7f03a00b2a00 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7f03a00b9c10 tx=0x7f0390007c20).stop 2020-11-28T21:48:24.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.075+0000 7f03a5e82700 1 -- 172.21.15.162:0/106783815 shutdown_connections 2020-11-28T21:48:24.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.075+0000 7f03a5e82700 1 --2- 172.21.15.162:0/106783815 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f03800554e0 0x7f03800579a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:24.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.075+0000 7f03a5e82700 1 --2- 172.21.15.162:0/106783815 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f03a00603d0 0x7f03a00b7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:24.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.075+0000 7f03a5e82700 1 --2- 172.21.15.162:0/106783815 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f03a00b2000 0x7f03a00b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:24.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.075+0000 7f03a5e82700 1 --2- 172.21.15.162:0/106783815 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03a00b4c70 0x7f03a00b2a00 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:24.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.075+0000 7f03a5e82700 1 -- 172.21.15.162:0/106783815 >> 172.21.15.162:0/106783815 conn(0x7f03a0066a40 msgr2=0x7f03a009cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:24.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.075+0000 7f03a5e82700 1 -- 172.21.15.162:0/106783815 shutdown_connections 2020-11-28T21:48:24.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.075+0000 7f03a5e82700 1 -- 172.21.15.162:0/106783815 wait complete. 2020-11-28T21:48:24.088 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-28T21:48:24.089 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:48:24.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.203+0000 7fd40b59e700 1 Processor -- start 2020-11-28T21:48:24.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.203+0000 7fd40b59e700 1 -- start start 2020-11-28T21:48:24.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.203+0000 7fd40b59e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd40c13cb50 0x7fd40c140180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:24.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.203+0000 7fd40b59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd40c13cf30 0x7fd40c0ac350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:24.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.203+0000 7fd40b59e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd40c0b1360 con 0x7fd40c05cca0 2020-11-28T21:48:24.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.203+0000 7fd40b59e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd40c0add60 con 0x7fd40c13cb50 2020-11-28T21:48:24.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.203+0000 7fd40b59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c0b25c0 con 0x7fd40c13cf30 2020-11-28T21:48:24.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.203+0000 7fd40a59c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd40c13cb50 0x7fd40c140180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:24.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.203+0000 7fd40a59c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd40c13cb50 0x7fd40c140180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39138/0 (socket says 172.21.15.162:39138) 2020-11-28T21:48:24.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.203+0000 7fd40a59c700 1 -- 172.21.15.162:0/1372509786 learned_addr learned my addr 172.21.15.162:0/1372509786 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:24.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.203+0000 7fd40ad9d700 1 --2- 172.21.15.162:0/1372509786 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd40c13cf30 0x7fd40c0ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:24.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.203+0000 7fd409d9b700 1 -- 172.21.15.162:0/1372509786 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1776874316 0 0) 0x7fd40c0b1360 con 0x7fd40c05cca0 2020-11-28T21:48:24.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.203+0000 7fd409d9b700 1 -- 172.21.15.162:0/1372509786 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3ec003230 con 0x7fd40c05cca0 2020-11-28T21:48:24.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.203+0000 7fd40a59c700 1 -- 172.21.15.162:0/1372509786 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd40c13cf30 msgr2=0x7fd40c0ac350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:24.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.203+0000 7fd40a59c700 1 --2- 172.21.15.162:0/1372509786 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd40c13cf30 0x7fd40c0ac350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:24.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.203+0000 7fd40a59c700 1 -- 172.21.15.162:0/1372509786 >> v1:172.21.15.118:6789/0 conn(0x7fd40c05cca0 legacy=0x7fd40c0b1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:24.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.203+0000 7fd40a59c700 1 -- 172.21.15.162:0/1372509786 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40c0ae8c0 con 0x7fd40c13cb50 2020-11-28T21:48:24.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.203+0000 7fd40a59c700 1 --2- 172.21.15.162:0/1372509786 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd40c13cb50 0x7fd40c140180 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7fd3fc002f30 tx=0x7fd3fc006e80).ready entity=mon.1 client_cookie=fc5bf4a780ca43e9 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:24.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.203+0000 7fd409d9b700 1 -- 172.21.15.162:0/1372509786 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3fc009640 con 0x7fd40c13cb50 2020-11-28T21:48:24.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.203+0000 7fd409d9b700 1 -- 172.21.15.162:0/1372509786 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3fc0097a0 con 0x7fd40c13cb50 2020-11-28T21:48:24.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.203+0000 7fd409d9b700 1 -- 172.21.15.162:0/1372509786 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3fc009b00 con 0x7fd40c13cb50 2020-11-28T21:48:24.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.203+0000 7fd40b59e700 1 -- 172.21.15.162:0/1372509786 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd40c13cb50 msgr2=0x7fd40c140180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:24.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.203+0000 7fd40b59e700 1 --2- 172.21.15.162:0/1372509786 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd40c13cb50 0x7fd40c140180 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7fd3fc002f30 tx=0x7fd3fc006e80).stop 2020-11-28T21:48:24.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.203+0000 7fd40b59e700 1 -- 172.21.15.162:0/1372509786 shutdown_connections 2020-11-28T21:48:24.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.203+0000 7fd40b59e700 1 --2- 172.21.15.162:0/1372509786 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd40c13cf30 0x7fd40c0ac350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:24.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.203+0000 7fd40b59e700 1 --2- 172.21.15.162:0/1372509786 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd40c13cb50 0x7fd40c140180 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:24.212 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.203+0000 7fd40b59e700 1 -- 172.21.15.162:0/1372509786 >> 172.21.15.162:0/1372509786 conn(0x7fd40c0b57d0 msgr2=0x7fd40c0b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:24.212 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.203+0000 7fd40b59e700 1 -- 172.21.15.162:0/1372509786 shutdown_connections 2020-11-28T21:48:24.212 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.207+0000 7fd40b59e700 1 -- 172.21.15.162:0/1372509786 wait complete. 2020-11-28T21:48:24.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.207+0000 7fd40b59e700 1 Processor -- start 2020-11-28T21:48:24.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.207+0000 7fd40b59e700 1 -- start start 2020-11-28T21:48:24.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.207+0000 7fd40b59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd40c05cca0 0x7fd40c13ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:24.216 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.207+0000 7fd40b59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd40c13cb50 0x7fd40c13bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:24.216 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.207+0000 7fd40b59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd40c13cf30 0x7fd40c13c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:24.216 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.207+0000 7fd40b59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c064330 con 0x7fd40c13cf30 2020-11-28T21:48:24.216 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.207+0000 7fd40b59e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c0aad70 con 0x7fd40c05cca0 2020-11-28T21:48:24.217 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.207+0000 7fd40b59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c0aafa0 con 0x7fd40c13cb50 2020-11-28T21:48:24.217 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.207+0000 7fd40a59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd40c05cca0 0x7fd40c13ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:24.218 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.207+0000 7fd40a59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd40c05cca0 0x7fd40c13ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39148/0 (socket says 172.21.15.162:39148) 2020-11-28T21:48:24.218 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.207+0000 7fd40a59c700 1 -- 172.21.15.162:0/619204361 learned_addr learned my addr 172.21.15.162:0/619204361 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:24.218 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.207+0000 7fd40ad9d700 1 --2- 172.21.15.162:0/619204361 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd40c13cf30 0x7fd40c13c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:24.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.207+0000 7fd401d9b700 1 --2- 172.21.15.162:0/619204361 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd40c13cb50 0x7fd40c13bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:24.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.207+0000 7fd40a59c700 1 -- 172.21.15.162:0/619204361 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd40c13cb50 msgr2=0x7fd40c13bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:24.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.207+0000 7fd40a59c700 1 --2- 172.21.15.162:0/619204361 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd40c13cb50 0x7fd40c13bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:24.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.207+0000 7fd40a59c700 1 -- 172.21.15.162:0/619204361 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd40c13cf30 msgr2=0x7fd40c13c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:24.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.207+0000 7fd40a59c700 1 --2- 172.21.15.162:0/619204361 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd40c13cf30 0x7fd40c13c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:24.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.207+0000 7fd40a59c700 1 -- 172.21.15.162:0/619204361 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40c0ab240 con 0x7fd40c05cca0 2020-11-28T21:48:24.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.207+0000 7fd401d9b700 1 --2- 172.21.15.162:0/619204361 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd40c13cb50 0x7fd40c13bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:48:24.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.207+0000 7fd40ad9d700 1 --2- 172.21.15.162:0/619204361 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd40c13cf30 0x7fd40c13c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:48:24.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.207+0000 7fd40a59c700 1 --2- 172.21.15.162:0/619204361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd40c05cca0 0x7fd40c13ddb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7fd3fc002f30 tx=0x7fd3fc015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:24.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.207+0000 7fd403fff700 1 -- 172.21.15.162:0/619204361 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3fc00b720 con 0x7fd40c05cca0 2020-11-28T21:48:24.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.207+0000 7fd403fff700 1 -- 172.21.15.162:0/619204361 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3fc011c60 con 0x7fd40c05cca0 2020-11-28T21:48:24.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.207+0000 7fd40b59e700 1 -- 172.21.15.162:0/619204361 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd40c0ab4e0 con 0x7fd40c05cca0 2020-11-28T21:48:24.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.207+0000 7fd403fff700 1 -- 172.21.15.162:0/619204361 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3fc01d450 con 0x7fd40c05cca0 2020-11-28T21:48:24.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.207+0000 7fd40b59e700 1 -- 172.21.15.162:0/619204361 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd40c0aa360 con 0x7fd40c05cca0 2020-11-28T21:48:24.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.207+0000 7fd40b59e700 1 -- 172.21.15.162:0/619204361 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd40c057050 con 0x7fd40c05cca0 2020-11-28T21:48:24.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.211+0000 7fd403fff700 1 -- 172.21.15.162:0/619204361 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd3fc00f070 con 0x7fd40c05cca0 2020-11-28T21:48:24.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.211+0000 7fd403fff700 1 --2- 172.21.15.162:0/619204361 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd3f00554e0 0x7fd3f00579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:24.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.211+0000 7fd403fff700 1 -- 172.21.15.162:0/619204361 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fd3fc070100 con 0x7fd40c05cca0 2020-11-28T21:48:24.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.211+0000 7fd401d9b700 1 --2- 172.21.15.162:0/619204361 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd3f00554e0 0x7fd3f00579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:24.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.211+0000 7fd401d9b700 1 --2- 172.21.15.162:0/619204361 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd3f00554e0 0x7fd3f00579a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7fd3f400b000 tx=0x7fd3f4009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:24.224 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.215+0000 7fd403fff700 1 -- 172.21.15.162:0/619204361 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd3fc046730 con 0x7fd40c05cca0 2020-11-28T21:48:24.356 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.355+0000 7fd40b59e700 1 -- 172.21.15.162:0/619204361 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fd40c068350 con 0x7fd40c05cca0 2020-11-28T21:48:24.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.355+0000 7fd403fff700 1 -- 172.21.15.162:0/619204361 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v39) v1 ==== 110+0+10 (secure 0 0 0) 0x7fd3fc049d50 con 0x7fd40c05cca0 2020-11-28T21:48:24.357 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:48:24.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.359+0000 7fd40b59e700 1 -- 172.21.15.162:0/619204361 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd3f00554e0 msgr2=0x7fd3f00579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:24.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.359+0000 7fd40b59e700 1 --2- 172.21.15.162:0/619204361 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd3f00554e0 0x7fd3f00579a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7fd3f400b000 tx=0x7fd3f4009040).stop 2020-11-28T21:48:24.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.359+0000 7fd40b59e700 1 -- 172.21.15.162:0/619204361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd40c05cca0 msgr2=0x7fd40c13ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:24.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.359+0000 7fd40b59e700 1 --2- 172.21.15.162:0/619204361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd40c05cca0 0x7fd40c13ddb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7fd3fc002f30 tx=0x7fd3fc015000).stop 2020-11-28T21:48:24.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.359+0000 7fd40b59e700 1 -- 172.21.15.162:0/619204361 shutdown_connections 2020-11-28T21:48:24.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.359+0000 7fd40b59e700 1 --2- 172.21.15.162:0/619204361 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd3f00554e0 0x7fd3f00579a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:24.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.359+0000 7fd40b59e700 1 --2- 172.21.15.162:0/619204361 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd40c13cf30 0x7fd40c13c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:24.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.359+0000 7fd40b59e700 1 --2- 172.21.15.162:0/619204361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd40c05cca0 0x7fd40c13ddb0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:24.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.359+0000 7fd40b59e700 1 --2- 172.21.15.162:0/619204361 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd40c13cb50 0x7fd40c13bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:24.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.359+0000 7fd40b59e700 1 -- 172.21.15.162:0/619204361 >> 172.21.15.162:0/619204361 conn(0x7fd40c0b57d0 msgr2=0x7fd40c0b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:24.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.359+0000 7fd40b59e700 1 -- 172.21.15.162:0/619204361 shutdown_connections 2020-11-28T21:48:24.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.359+0000 7fd40b59e700 1 -- 172.21.15.162:0/619204361 wait complete. 2020-11-28T21:48:24.371 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:48:24.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.487+0000 7fa615ebb700 1 Processor -- start 2020-11-28T21:48:24.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.487+0000 7fa615ebb700 1 -- start start 2020-11-28T21:48:24.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.487+0000 7fa615ebb700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa6100b3c90 0x7fa610142a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:24.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.487+0000 7fa615ebb700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa61005f380 0x7fa610146050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:24.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.487+0000 7fa615ebb700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6100aac00 con 0x7fa6100b18c0 2020-11-28T21:48:24.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.487+0000 7fa615ebb700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa61013cf80 con 0x7fa6100b3c90 2020-11-28T21:48:24.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.487+0000 7fa615ebb700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6100b4070 con 0x7fa61005f380 2020-11-28T21:48:24.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.487+0000 7fa614eb9700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa6100b3c90 0x7fa610142a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:24.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.487+0000 7fa60ffff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fa6100b18c0 0x7fa610142f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:45998/0 (socket says 172.21.15.162:45998) 2020-11-28T21:48:24.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.487+0000 7fa60ffff700 1 -- 172.21.15.162:0/665232059 learned_addr learned my addr 172.21.15.162:0/665232059 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:48:24.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.491+0000 7fa6156ba700 1 --2- 172.21.15.162:0/665232059 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa61005f380 0x7fa610146050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:24.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.491+0000 7fa614eb9700 1 -- 172.21.15.162:0/665232059 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa61005f380 msgr2=0x7fa610146050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:24.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.491+0000 7fa614eb9700 1 --2- 172.21.15.162:0/665232059 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa61005f380 0x7fa610146050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:24.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.491+0000 7fa614eb9700 1 -- 172.21.15.162:0/665232059 >> v1:172.21.15.162:6789/0 conn(0x7fa6100b18c0 legacy=0x7fa610142f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:24.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.491+0000 7fa60f7fe700 1 -- 172.21.15.162:0/665232059 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3806346556 0 0) 0x7fa6100aac00 con 0x7fa6100b18c0 2020-11-28T21:48:24.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.491+0000 7fa614eb9700 1 -- 172.21.15.162:0/665232059 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa610060a10 con 0x7fa6100b3c90 2020-11-28T21:48:24.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.491+0000 7fa6156ba700 1 --2- 172.21.15.162:0/665232059 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa61005f380 0x7fa610146050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:48:24.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.491+0000 7fa614eb9700 1 --2- 172.21.15.162:0/665232059 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6100b3c90 0x7fa610142a80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7fa604002f30 tx=0x7fa604006dc0).ready entity=mon.1 client_cookie=ee03b1d5b1156e7e server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:24.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.491+0000 7fa60f7fe700 1 -- 172.21.15.162:0/665232059 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa6040097f0 con 0x7fa6100b3c90 2020-11-28T21:48:24.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.491+0000 7fa60f7fe700 1 -- 172.21.15.162:0/665232059 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa604009950 con 0x7fa6100b3c90 2020-11-28T21:48:24.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.491+0000 7fa60f7fe700 1 -- 172.21.15.162:0/665232059 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa604009cb0 con 0x7fa6100b3c90 2020-11-28T21:48:24.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.491+0000 7fa615ebb700 1 -- 172.21.15.162:0/665232059 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6100b3c90 msgr2=0x7fa610142a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:24.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.491+0000 7fa615ebb700 1 --2- 172.21.15.162:0/665232059 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6100b3c90 0x7fa610142a80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7fa604002f30 tx=0x7fa604006dc0).stop 2020-11-28T21:48:24.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.491+0000 7fa615ebb700 1 -- 172.21.15.162:0/665232059 shutdown_connections 2020-11-28T21:48:24.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.491+0000 7fa615ebb700 1 --2- 172.21.15.162:0/665232059 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6100b3c90 0x7fa610142a80 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:24.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.491+0000 7fa615ebb700 1 --2- 172.21.15.162:0/665232059 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa61005f380 0x7fa610146050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:24.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.491+0000 7fa615ebb700 1 -- 172.21.15.162:0/665232059 >> 172.21.15.162:0/665232059 conn(0x7fa610062630 msgr2=0x7fa610062a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:24.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.491+0000 7fa615ebb700 1 -- 172.21.15.162:0/665232059 shutdown_connections 2020-11-28T21:48:24.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.491+0000 7fa615ebb700 1 -- 172.21.15.162:0/665232059 wait complete. 2020-11-28T21:48:24.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.491+0000 7fa615ebb700 1 Processor -- start 2020-11-28T21:48:24.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.491+0000 7fa615ebb700 1 -- start start 2020-11-28T21:48:24.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.495+0000 7fa615ebb700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa61005f380 0x7fa61013f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:24.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.495+0000 7fa615ebb700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa6100b18c0 0x7fa610155330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:24.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.495+0000 7fa615ebb700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6100b3c90 0x7fa610155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:24.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.495+0000 7fa615ebb700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa610145300 con 0x7fa61005f380 2020-11-28T21:48:24.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.495+0000 7fa615ebb700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa610142ea0 con 0x7fa6100b3c90 2020-11-28T21:48:24.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.495+0000 7fa615ebb700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa610143100 con 0x7fa6100b18c0 2020-11-28T21:48:24.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.495+0000 7fa6156ba700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6100b3c90 0x7fa610155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:24.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.495+0000 7fa6156ba700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6100b3c90 0x7fa610155820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39158/0 (socket says 172.21.15.162:39158) 2020-11-28T21:48:24.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.495+0000 7fa6156ba700 1 -- 172.21.15.162:0/3215070239 learned_addr learned my addr 172.21.15.162:0/3215070239 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:24.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.495+0000 7fa614eb9700 1 --2- 172.21.15.162:0/3215070239 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa61005f380 0x7fa61013f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:24.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.495+0000 7fa60ffff700 1 --2- 172.21.15.162:0/3215070239 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa6100b18c0 0x7fa610155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:24.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.495+0000 7fa6156ba700 1 -- 172.21.15.162:0/3215070239 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa6100b18c0 msgr2=0x7fa610155330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:24.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.495+0000 7fa6156ba700 1 --2- 172.21.15.162:0/3215070239 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa6100b18c0 0x7fa610155330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:24.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.495+0000 7fa6156ba700 1 -- 172.21.15.162:0/3215070239 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa61005f380 msgr2=0x7fa61013f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:24.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.495+0000 7fa6156ba700 1 --2- 172.21.15.162:0/3215070239 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa61005f380 0x7fa61013f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:24.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.495+0000 7fa6156ba700 1 -- 172.21.15.162:0/3215070239 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6101433a0 con 0x7fa6100b3c90 2020-11-28T21:48:24.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.495+0000 7fa60ffff700 1 --2- 172.21.15.162:0/3215070239 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa6100b18c0 0x7fa610155330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:48:24.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.495+0000 7fa614eb9700 1 --2- 172.21.15.162:0/3215070239 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa61005f380 0x7fa61013f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:48:24.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.495+0000 7fa6156ba700 1 --2- 172.21.15.162:0/3215070239 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6100b3c90 0x7fa610155820 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7fa610142370 tx=0x7fa608008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:24.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.495+0000 7fa60dffb700 1 -- 172.21.15.162:0/3215070239 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa60800c980 con 0x7fa6100b3c90 2020-11-28T21:48:24.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.495+0000 7fa60dffb700 1 -- 172.21.15.162:0/3215070239 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa60800cae0 con 0x7fa6100b3c90 2020-11-28T21:48:24.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.495+0000 7fa615ebb700 1 -- 172.21.15.162:0/3215070239 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa610143640 con 0x7fa6100b3c90 2020-11-28T21:48:24.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.495+0000 7fa60dffb700 1 -- 172.21.15.162:0/3215070239 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa60800ce40 con 0x7fa6100b3c90 2020-11-28T21:48:24.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.495+0000 7fa615ebb700 1 -- 172.21.15.162:0/3215070239 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa61013ff60 con 0x7fa6100b3c90 2020-11-28T21:48:24.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.499+0000 7fa615ebb700 1 -- 172.21.15.162:0/3215070239 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa610067810 con 0x7fa6100b3c90 2020-11-28T21:48:24.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.499+0000 7fa60dffb700 1 -- 172.21.15.162:0/3215070239 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa60801d460 con 0x7fa6100b3c90 2020-11-28T21:48:24.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.499+0000 7fa60dffb700 1 --2- 172.21.15.162:0/3215070239 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa5f0055440 0x7fa5f0057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:24.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.499+0000 7fa60dffb700 1 -- 172.21.15.162:0/3215070239 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fa60806e180 con 0x7fa6100b3c90 2020-11-28T21:48:24.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.499+0000 7fa614eb9700 1 --2- 172.21.15.162:0/3215070239 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa5f0055440 0x7fa5f0057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:24.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.499+0000 7fa614eb9700 1 --2- 172.21.15.162:0/3215070239 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa5f0055440 0x7fa5f0057900 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7fa604019000 tx=0x7fa604015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:24.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.503+0000 7fa60dffb700 1 -- 172.21.15.162:0/3215070239 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa608044750 con 0x7fa6100b3c90 2020-11-28T21:48:24.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.639+0000 7fa615ebb700 1 -- 172.21.15.162:0/3215070239 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa610057050 con 0x7fa6100b3c90 2020-11-28T21:48:24.642 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.639+0000 7fa60dffb700 1 -- 172.21.15.162:0/3215070239 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v39) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa608047d70 con 0x7fa6100b3c90 2020-11-28T21:48:24.642 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:48:24.644 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.643+0000 7fa615ebb700 1 -- 172.21.15.162:0/3215070239 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa5f0055440 msgr2=0x7fa5f0057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:24.644 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.643+0000 7fa615ebb700 1 --2- 172.21.15.162:0/3215070239 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa5f0055440 0x7fa5f0057900 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7fa604019000 tx=0x7fa604015000).stop 2020-11-28T21:48:24.645 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.643+0000 7fa615ebb700 1 -- 172.21.15.162:0/3215070239 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6100b3c90 msgr2=0x7fa610155820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:24.645 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.643+0000 7fa615ebb700 1 --2- 172.21.15.162:0/3215070239 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6100b3c90 0x7fa610155820 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7fa610142370 tx=0x7fa608008040).stop 2020-11-28T21:48:24.645 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.643+0000 7fa615ebb700 1 -- 172.21.15.162:0/3215070239 shutdown_connections 2020-11-28T21:48:24.645 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.643+0000 7fa615ebb700 1 --2- 172.21.15.162:0/3215070239 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa5f0055440 0x7fa5f0057900 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:24.646 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.643+0000 7fa615ebb700 1 --2- 172.21.15.162:0/3215070239 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6100b3c90 0x7fa610155820 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:24.646 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.643+0000 7fa615ebb700 1 --2- 172.21.15.162:0/3215070239 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa61005f380 0x7fa61013f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:24.647 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.643+0000 7fa615ebb700 1 --2- 172.21.15.162:0/3215070239 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa6100b18c0 0x7fa610155330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:24.647 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.643+0000 7fa615ebb700 1 -- 172.21.15.162:0/3215070239 >> 172.21.15.162:0/3215070239 conn(0x7fa610062630 msgr2=0x7fa6100658b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:24.647 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.643+0000 7fa615ebb700 1 -- 172.21.15.162:0/3215070239 shutdown_connections 2020-11-28T21:48:24.647 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.643+0000 7fa615ebb700 1 -- 172.21.15.162:0/3215070239 wait complete. 2020-11-28T21:48:24.654 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2020-11-28T21:48:24.766 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6acffff700 1 Processor -- start 2020-11-28T21:48:24.767 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6acffff700 1 -- start start 2020-11-28T21:48:24.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6acffff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6ad00b5430 0x7f6ad0143360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:24.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6acffff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ad00b8040 con 0x7f6ad00b5e50 2020-11-28T21:48:24.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6acffff700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ad00b8720 con 0x7f6ad00b5a70 2020-11-28T21:48:24.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6acffff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad00b7600 con 0x7f6ad00b5430 2020-11-28T21:48:24.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6aceffd700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f6ad00b5a70 0x7f6ad00b6ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:46008/0 (socket says 172.21.15.162:46008) 2020-11-28T21:48:24.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6aceffd700 1 -- 172.21.15.162:0/3195716423 learned_addr learned my addr 172.21.15.162:0/3195716423 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:48:24.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6acf7fe700 1 --2- 172.21.15.162:0/3195716423 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6ad00b5430 0x7f6ad0143360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:24.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6acdffb700 1 -- 172.21.15.162:0/3195716423 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2871732789 0 0) 0x7f6ad00b8720 con 0x7f6ad00b5a70 2020-11-28T21:48:24.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6acdffb700 1 -- 172.21.15.162:0/3195716423 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ab4003230 con 0x7f6ad00b5a70 2020-11-28T21:48:24.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6acdffb700 1 -- 172.21.15.162:0/3195716423 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3929642140 0 0) 0x7f6ab4003230 con 0x7f6ad00b5a70 2020-11-28T21:48:24.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6acdffb700 1 -- 172.21.15.162:0/3195716423 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6ad00b5430 msgr2=0x7f6ad0143360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:24.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6acdffb700 1 --2- 172.21.15.162:0/3195716423 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6ad00b5430 0x7f6ad0143360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:24.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6acdffb700 1 -- 172.21.15.162:0/3195716423 >> v1:172.21.15.118:6789/0 conn(0x7f6ad00b5e50 legacy=0x7f6ad009d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:24.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6acdffb700 1 -- 172.21.15.162:0/3195716423 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad0142980 con 0x7f6ad00b5a70 2020-11-28T21:48:24.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6acdffb700 1 -- 172.21.15.162:0/3195716423 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f6ab80016d0 con 0x7f6ad00b5a70 2020-11-28T21:48:24.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6acdffb700 1 -- 172.21.15.162:0/3195716423 >> v1:172.21.15.162:6789/0 conn(0x7f6ad00b5a70 legacy=0x7f6ad00b6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:24.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6acdffb700 1 --2- 172.21.15.162:0/3195716423 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6ad00b5430 0x7f6ab4003720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:24.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6acdffb700 1 --2- 172.21.15.162:0/3195716423 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6ab4003c30 0x7f6ab40044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:24.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6acdffb700 1 --2- 172.21.15.162:0/3195716423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ab40049c0 0x7f6ab4004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:24.776 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6acdffb700 1 -- 172.21.15.162:0/3195716423 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad00b7600 con 0x7f6ab4003c30 2020-11-28T21:48:24.776 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6acdffb700 1 -- 172.21.15.162:0/3195716423 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab4004220 con 0x7f6ab40049c0 2020-11-28T21:48:24.776 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6acdffb700 1 -- 172.21.15.162:0/3195716423 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab400b720 con 0x7f6ad00b5430 2020-11-28T21:48:24.776 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6acf7fe700 1 --2- 172.21.15.162:0/3195716423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ab40049c0 0x7f6ab4004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:24.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6aceffd700 1 --2- 172.21.15.162:0/3195716423 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6ad00b5430 0x7f6ab4003720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:24.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6ace7fc700 1 --2- 172.21.15.162:0/3195716423 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6ab4003c30 0x7f6ab40044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:24.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6acf7fe700 1 -- 172.21.15.162:0/3195716423 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6ad00b5430 msgr2=0x7f6ab4003720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:24.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6acf7fe700 1 --2- 172.21.15.162:0/3195716423 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6ad00b5430 0x7f6ab4003720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:24.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6acf7fe700 1 -- 172.21.15.162:0/3195716423 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6ab4003c30 msgr2=0x7f6ab40044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:24.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6acf7fe700 1 --2- 172.21.15.162:0/3195716423 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6ab4003c30 0x7f6ab40044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:24.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6acf7fe700 1 -- 172.21.15.162:0/3195716423 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab4003230 con 0x7f6ab40049c0 2020-11-28T21:48:24.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6aceffd700 1 --2- 172.21.15.162:0/3195716423 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6ad00b5430 0x7f6ab4003720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:48:24.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6acf7fe700 1 --2- 172.21.15.162:0/3195716423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ab40049c0 0x7f6ab4004e40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7f6ac000c000 tx=0x7f6ac00091a0).ready entity=mon.1 client_cookie=b8ead4e51ed5c5b6 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:24.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6acdffb700 1 -- 172.21.15.162:0/3195716423 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ac000ed00 con 0x7f6ab40049c0 2020-11-28T21:48:24.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6acdffb700 1 -- 172.21.15.162:0/3195716423 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ac000ee60 con 0x7f6ab40049c0 2020-11-28T21:48:24.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6acdffb700 1 -- 172.21.15.162:0/3195716423 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ac00125e0 con 0x7f6ab40049c0 2020-11-28T21:48:24.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6acffff700 1 -- 172.21.15.162:0/3195716423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ab40049c0 msgr2=0x7f6ab4004e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:24.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6acffff700 1 --2- 172.21.15.162:0/3195716423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ab40049c0 0x7f6ab4004e40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7f6ac000c000 tx=0x7f6ac00091a0).stop 2020-11-28T21:48:24.780 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6acf7fe700 1 -- 172.21.15.162:0/3195716423 reap_dead start 2020-11-28T21:48:24.780 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6acffff700 1 -- 172.21.15.162:0/3195716423 shutdown_connections 2020-11-28T21:48:24.780 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6acffff700 1 -- 172.21.15.162:0/3195716423 >> 172.21.15.162:0/3195716423 conn(0x7f6ad005b970 msgr2=0x7f6ad005bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:24.781 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6acffff700 1 -- 172.21.15.162:0/3195716423 shutdown_connections 2020-11-28T21:48:24.781 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.767+0000 7f6acffff700 1 -- 172.21.15.162:0/3195716423 wait complete. 2020-11-28T21:48:24.784 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.771+0000 7f6acffff700 1 Processor -- start 2020-11-28T21:48:24.784 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.771+0000 7f6acffff700 1 -- start start 2020-11-28T21:48:24.785 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.771+0000 7f6acffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ad0144410 0x7f6ad01447f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:24.785 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.771+0000 7f6acffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6ad00acee0 0x7f6ad00ae7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:24.785 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.771+0000 7f6acffff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6ad00ad360 0x7f6ad00aeca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:24.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.771+0000 7f6acffff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad0146140 con 0x7f6ad00acee0 2020-11-28T21:48:24.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.771+0000 7f6acffff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad0146610 con 0x7f6ad0144410 2020-11-28T21:48:24.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.771+0000 7f6acffff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad0146370 con 0x7f6ad00ad360 2020-11-28T21:48:24.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.771+0000 7f6aceffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ad0144410 0x7f6ad01447f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:24.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.771+0000 7f6aceffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ad0144410 0x7f6ad01447f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39176/0 (socket says 172.21.15.162:39176) 2020-11-28T21:48:24.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.771+0000 7f6aceffd700 1 -- 172.21.15.162:0/810911801 learned_addr learned my addr 172.21.15.162:0/810911801 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:24.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.771+0000 7f6ace7fc700 1 --2- 172.21.15.162:0/810911801 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6ad00acee0 0x7f6ad00ae7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:24.788 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.771+0000 7f6acf7fe700 1 --2- 172.21.15.162:0/810911801 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6ad00ad360 0x7f6ad00aeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:24.788 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.771+0000 7f6aceffd700 1 -- 172.21.15.162:0/810911801 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6ad00ad360 msgr2=0x7f6ad00aeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:24.788 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.771+0000 7f6aceffd700 1 --2- 172.21.15.162:0/810911801 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6ad00ad360 0x7f6ad00aeca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:24.788 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.771+0000 7f6aceffd700 1 -- 172.21.15.162:0/810911801 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6ad00acee0 msgr2=0x7f6ad00ae7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:24.788 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.771+0000 7f6aceffd700 1 --2- 172.21.15.162:0/810911801 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6ad00acee0 0x7f6ad00ae7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:24.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.771+0000 7f6aceffd700 1 -- 172.21.15.162:0/810911801 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad00ad890 con 0x7f6ad0144410 2020-11-28T21:48:24.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.771+0000 7f6aceffd700 1 --2- 172.21.15.162:0/810911801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ad0144410 0x7f6ad01447f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f6ab800b000 tx=0x7f6ad00b6930).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:24.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.771+0000 7f6ac7fff700 1 -- 172.21.15.162:0/810911801 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ab800eaa0 con 0x7f6ad0144410 2020-11-28T21:48:24.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.771+0000 7f6ac7fff700 1 -- 172.21.15.162:0/810911801 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ab8010040 con 0x7f6ad0144410 2020-11-28T21:48:24.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.771+0000 7f6ac7fff700 1 -- 172.21.15.162:0/810911801 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ab800edf0 con 0x7f6ad0144410 2020-11-28T21:48:24.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.771+0000 7f6acffff700 1 -- 172.21.15.162:0/810911801 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ad00608b0 con 0x7f6ad0144410 2020-11-28T21:48:24.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.771+0000 7f6acffff700 1 -- 172.21.15.162:0/810911801 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ad01457e0 con 0x7f6ad0144410 2020-11-28T21:48:24.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.775+0000 7f6acffff700 1 -- 172.21.15.162:0/810911801 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ad00ae410 con 0x7f6ad0144410 2020-11-28T21:48:24.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.775+0000 7f6ac7fff700 1 -- 172.21.15.162:0/810911801 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6ab8015440 con 0x7f6ad0144410 2020-11-28T21:48:24.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.775+0000 7f6ac7fff700 1 --2- 172.21.15.162:0/810911801 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6aa80554e0 0x7f6aa80579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:24.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.775+0000 7f6ac7fff700 1 -- 172.21.15.162:0/810911801 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f6ab806d6a0 con 0x7f6ad0144410 2020-11-28T21:48:24.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.775+0000 7f6ace7fc700 1 --2- 172.21.15.162:0/810911801 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6aa80554e0 0x7f6aa80579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:24.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.775+0000 7f6ace7fc700 1 --2- 172.21.15.162:0/810911801 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6aa80554e0 0x7f6aa80579a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f6ac800d000 tx=0x7f6ac800c040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:24.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.779+0000 7f6ac7fff700 1 -- 172.21.15.162:0/810911801 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6ab8043d20 con 0x7f6ad0144410 2020-11-28T21:48:24.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.923+0000 7f6acffff700 1 -- 172.21.15.162:0/810911801 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f6ad0057050 con 0x7f6ad0144410 2020-11-28T21:48:24.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.923+0000 7f6ac7fff700 1 -- 172.21.15.162:0/810911801 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v39) v1 ==== 84+0+143 (secure 0 0 0) 0x7f6ab801a020 con 0x7f6ad0144410 2020-11-28T21:48:24.924 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:48:24.925 INFO:teuthology.orchestra.run.smithi162.stdout:[ 2020-11-28T21:48:24.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:24.925 INFO:teuthology.orchestra.run.smithi162.stdout: "poolnum": 1, 2020-11-28T21:48:24.925 INFO:teuthology.orchestra.run.smithi162.stdout: "poolname": "device_health_metrics" 2020-11-28T21:48:24.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:24.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:48:24.926 INFO:teuthology.orchestra.run.smithi162.stdout: "poolnum": 2, 2020-11-28T21:48:24.926 INFO:teuthology.orchestra.run.smithi162.stdout: "poolname": "rbd" 2020-11-28T21:48:24.926 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:24.927 INFO:teuthology.orchestra.run.smithi162.stdout:] 2020-11-28T21:48:24.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.927+0000 7f6acffff700 1 -- 172.21.15.162:0/810911801 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6aa80554e0 msgr2=0x7f6aa80579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:24.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.927+0000 7f6acffff700 1 --2- 172.21.15.162:0/810911801 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6aa80554e0 0x7f6aa80579a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f6ac800d000 tx=0x7f6ac800c040).stop 2020-11-28T21:48:24.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.927+0000 7f6acffff700 1 -- 172.21.15.162:0/810911801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ad0144410 msgr2=0x7f6ad01447f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:24.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.927+0000 7f6acffff700 1 --2- 172.21.15.162:0/810911801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ad0144410 0x7f6ad01447f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f6ab800b000 tx=0x7f6ad00b6930).stop 2020-11-28T21:48:24.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.927+0000 7f6acffff700 1 -- 172.21.15.162:0/810911801 shutdown_connections 2020-11-28T21:48:24.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.927+0000 7f6acffff700 1 --2- 172.21.15.162:0/810911801 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6aa80554e0 0x7f6aa80579a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:24.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.927+0000 7f6acffff700 1 --2- 172.21.15.162:0/810911801 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6ad00ad360 0x7f6ad00aeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:24.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.927+0000 7f6acffff700 1 --2- 172.21.15.162:0/810911801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ad0144410 0x7f6ad01447f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:24.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.927+0000 7f6acffff700 1 --2- 172.21.15.162:0/810911801 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6ad00acee0 0x7f6ad00ae7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:24.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.927+0000 7f6acffff700 1 -- 172.21.15.162:0/810911801 >> 172.21.15.162:0/810911801 conn(0x7f6ad005b970 msgr2=0x7f6ad0140830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:24.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.927+0000 7f6acffff700 1 -- 172.21.15.162:0/810911801 shutdown_connections 2020-11-28T21:48:24.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:24.927+0000 7f6acffff700 1 -- 172.21.15.162:0/810911801 wait complete. 2020-11-28T21:48:24.945 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-11-28T21:48:24.946 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2020-11-28T21:48:25.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.063+0000 7f315492d700 1 Processor -- start 2020-11-28T21:48:25.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.063+0000 7f315492d700 1 -- start start 2020-11-28T21:48:25.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.063+0000 7f315492d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f31500b3ab0 0x7f315005c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:25.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.063+0000 7f315492d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f31500b30e0 0x7f315013fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:25.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.067+0000 7f315492d700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f315005baf0 con 0x7f31500b2d00 2020-11-28T21:48:25.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.067+0000 7f315492d700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31500b0e20 con 0x7f31500b3ab0 2020-11-28T21:48:25.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.067+0000 7f315492d700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f315005bd90 con 0x7f31500b30e0 2020-11-28T21:48:25.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.067+0000 7f314f59e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f31500b30e0 0x7f315013fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:25.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.067+0000 7f314f59e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f31500b30e0 0x7f315013fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39188/0 (socket says 172.21.15.162:39188) 2020-11-28T21:48:25.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.067+0000 7f314f59e700 1 -- 172.21.15.162:0/2319050525 learned_addr learned my addr 172.21.15.162:0/2319050525 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:25.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.067+0000 7f314e59c700 1 --2- 172.21.15.162:0/2319050525 >> v2:172.21.15.118:3300/0 conn(0x7f31500b3ab0 0x7f315005c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:25.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.067+0000 7f314f59e700 1 -- 172.21.15.162:0/2319050525 >> v2:172.21.15.118:3300/0 conn(0x7f31500b3ab0 msgr2=0x7f315005c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:25.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.067+0000 7f314f59e700 1 --2- 172.21.15.162:0/2319050525 >> v2:172.21.15.118:3300/0 conn(0x7f31500b3ab0 0x7f315005c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:25.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.067+0000 7f314f59e700 1 -- 172.21.15.162:0/2319050525 >> v1:172.21.15.118:6789/0 conn(0x7f31500b2d00 legacy=0x7f3150142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:25.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.067+0000 7f314f59e700 1 -- 172.21.15.162:0/2319050525 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f315005c030 con 0x7f31500b30e0 2020-11-28T21:48:25.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.067+0000 7f314f59e700 1 --2- 172.21.15.162:0/2319050525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31500b30e0 0x7f315013fa70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f31480032b0 tx=0x7f3148009270).ready entity=mon.1 client_cookie=102546be3df9ff22 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:25.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.067+0000 7f314dd9b700 1 -- 172.21.15.162:0/2319050525 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f314800bc00 con 0x7f31500b30e0 2020-11-28T21:48:25.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.067+0000 7f314dd9b700 1 -- 172.21.15.162:0/2319050525 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f314800bd60 con 0x7f31500b30e0 2020-11-28T21:48:25.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.067+0000 7f314dd9b700 1 -- 172.21.15.162:0/2319050525 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3148011400 con 0x7f31500b30e0 2020-11-28T21:48:25.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.067+0000 7f315492d700 1 -- 172.21.15.162:0/2319050525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31500b30e0 msgr2=0x7f315013fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:25.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.067+0000 7f315492d700 1 --2- 172.21.15.162:0/2319050525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31500b30e0 0x7f315013fa70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f31480032b0 tx=0x7f3148009270).stop 2020-11-28T21:48:25.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.067+0000 7f315492d700 1 -- 172.21.15.162:0/2319050525 shutdown_connections 2020-11-28T21:48:25.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.067+0000 7f315492d700 1 --2- 172.21.15.162:0/2319050525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31500b30e0 0x7f315013fa70 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:25.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.067+0000 7f315492d700 1 --2- 172.21.15.162:0/2319050525 >> v2:172.21.15.118:3300/0 conn(0x7f31500b3ab0 0x7f315005c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:25.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.067+0000 7f315492d700 1 -- 172.21.15.162:0/2319050525 >> 172.21.15.162:0/2319050525 conn(0x7f31500604c0 msgr2=0x7f31500608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:25.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.067+0000 7f315492d700 1 -- 172.21.15.162:0/2319050525 shutdown_connections 2020-11-28T21:48:25.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.067+0000 7f315492d700 1 -- 172.21.15.162:0/2319050525 wait complete. 2020-11-28T21:48:25.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.071+0000 7f315492d700 1 Processor -- start 2020-11-28T21:48:25.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.071+0000 7f315492d700 1 -- start start 2020-11-28T21:48:25.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.071+0000 7f315492d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f31500b2d00 0x7f3150145820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:25.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.071+0000 7f315492d700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f31500b30e0 0x7f3150134a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:25.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.071+0000 7f315492d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31500b3ab0 0x7f3150134f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:25.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.071+0000 7f315492d700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31500b9940 con 0x7f31500b2d00 2020-11-28T21:48:25.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.071+0000 7f315492d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31500b9be0 con 0x7f31500b3ab0 2020-11-28T21:48:25.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.071+0000 7f315492d700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31500b9080 con 0x7f31500b30e0 2020-11-28T21:48:25.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.071+0000 7f314f59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31500b3ab0 0x7f3150134f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:25.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.071+0000 7f314f59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31500b3ab0 0x7f3150134f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39194/0 (socket says 172.21.15.162:39194) 2020-11-28T21:48:25.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.071+0000 7f314f59e700 1 -- 172.21.15.162:0/84827341 learned_addr learned my addr 172.21.15.162:0/84827341 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:25.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.071+0000 7f314e59c700 1 --2- 172.21.15.162:0/84827341 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f31500b30e0 0x7f3150134a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:25.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.071+0000 7f314ed9d700 1 --2- 172.21.15.162:0/84827341 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f31500b2d00 0x7f3150145820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:25.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.071+0000 7f314f59e700 1 -- 172.21.15.162:0/84827341 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f31500b30e0 msgr2=0x7f3150134a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:25.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.071+0000 7f314f59e700 1 --2- 172.21.15.162:0/84827341 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f31500b30e0 0x7f3150134a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:25.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.071+0000 7f314f59e700 1 -- 172.21.15.162:0/84827341 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f31500b2d00 msgr2=0x7f3150145820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:25.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.071+0000 7f314f59e700 1 --2- 172.21.15.162:0/84827341 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f31500b2d00 0x7f3150145820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:25.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.071+0000 7f314f59e700 1 -- 172.21.15.162:0/84827341 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31500b9520 con 0x7f31500b3ab0 2020-11-28T21:48:25.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.071+0000 7f314f59e700 1 --2- 172.21.15.162:0/84827341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31500b3ab0 0x7f3150134f40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f31480032b0 tx=0x7f3148016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:25.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.071+0000 7f313ffff700 1 -- 172.21.15.162:0/84827341 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f314801f670 con 0x7f31500b3ab0 2020-11-28T21:48:25.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.071+0000 7f313ffff700 1 -- 172.21.15.162:0/84827341 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f314801f7d0 con 0x7f31500b3ab0 2020-11-28T21:48:25.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.071+0000 7f315492d700 1 -- 172.21.15.162:0/84827341 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31500afb50 con 0x7f31500b3ab0 2020-11-28T21:48:25.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.071+0000 7f313ffff700 1 -- 172.21.15.162:0/84827341 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f314801fb30 con 0x7f31500b3ab0 2020-11-28T21:48:25.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.071+0000 7f315492d700 1 -- 172.21.15.162:0/84827341 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31500b0080 con 0x7f31500b3ab0 2020-11-28T21:48:25.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.075+0000 7f313ffff700 1 -- 172.21.15.162:0/84827341 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f314800f070 con 0x7f31500b3ab0 2020-11-28T21:48:25.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.075+0000 7f315492d700 1 -- 172.21.15.162:0/84827341 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31500b8ea0 con 0x7f31500b3ab0 2020-11-28T21:48:25.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.075+0000 7f313ffff700 1 --2- 172.21.15.162:0/84827341 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f312c055330 0x7f312c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:25.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.075+0000 7f313ffff700 1 -- 172.21.15.162:0/84827341 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f314806efd0 con 0x7f31500b3ab0 2020-11-28T21:48:25.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.075+0000 7f314ed9d700 1 --2- 172.21.15.162:0/84827341 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f312c055330 0x7f312c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:25.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.075+0000 7f314ed9d700 1 --2- 172.21.15.162:0/84827341 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f312c055330 0x7f312c0577f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f314400d000 tx=0x7f3144000b80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:25.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.079+0000 7f313ffff700 1 -- 172.21.15.162:0/84827341 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f3148045560 con 0x7f31500b3ab0 2020-11-28T21:48:25.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.219+0000 7f315492d700 1 -- 172.21.15.162:0/84827341 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f31500b9240 con 0x7f31500b3ab0 2020-11-28T21:48:25.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.431+0000 7f313ffff700 1 -- 172.21.15.162:0/84827341 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v40) v1 ==== 133+0+0 (secure 0 0 0) 0x7f31480240a0 con 0x7f31500b3ab0 2020-11-28T21:48:25.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.435+0000 7f315492d700 1 -- 172.21.15.162:0/84827341 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f312c055330 msgr2=0x7f312c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:25.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.435+0000 7f315492d700 1 --2- 172.21.15.162:0/84827341 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f312c055330 0x7f312c0577f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f314400d000 tx=0x7f3144000b80).stop 2020-11-28T21:48:25.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.435+0000 7f315492d700 1 -- 172.21.15.162:0/84827341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31500b3ab0 msgr2=0x7f3150134f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:25.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.435+0000 7f315492d700 1 --2- 172.21.15.162:0/84827341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31500b3ab0 0x7f3150134f40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f31480032b0 tx=0x7f3148016000).stop 2020-11-28T21:48:25.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.435+0000 7f315492d700 1 -- 172.21.15.162:0/84827341 shutdown_connections 2020-11-28T21:48:25.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.435+0000 7f315492d700 1 --2- 172.21.15.162:0/84827341 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f312c055330 0x7f312c0577f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:25.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.435+0000 7f315492d700 1 --2- 172.21.15.162:0/84827341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31500b3ab0 0x7f3150134f40 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:25.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.435+0000 7f315492d700 1 --2- 172.21.15.162:0/84827341 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f31500b2d00 0x7f3150145820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:25.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.435+0000 7f315492d700 1 --2- 172.21.15.162:0/84827341 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f31500b30e0 0x7f3150134a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:25.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.435+0000 7f315492d700 1 -- 172.21.15.162:0/84827341 >> 172.21.15.162:0/84827341 conn(0x7f31500604c0 msgr2=0x7f315013c050 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:25.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.435+0000 7f315492d700 1 -- 172.21.15.162:0/84827341 shutdown_connections 2020-11-28T21:48:25.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.435+0000 7f315492d700 1 -- 172.21.15.162:0/84827341 wait complete. 2020-11-28T21:48:25.440 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_metadata' created 2020-11-28T21:48:25.454 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2020-11-28T21:48:25.569 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.567+0000 7f6a26c93700 1 Processor -- start 2020-11-28T21:48:25.569 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.567+0000 7f6a26c93700 1 -- start start 2020-11-28T21:48:25.571 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.571+0000 7f6a26c93700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f6a200b2b60 0x7f6a20146a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:25.571 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.571+0000 7f6a26c93700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a20062e40 con 0x7f6a200b3770 2020-11-28T21:48:25.571 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.571+0000 7f6a26c93700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a2009cb50 con 0x7f6a20065750 2020-11-28T21:48:25.571 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.571+0000 7f6a26c93700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a2009cdf0 con 0x7f6a200b2b60 2020-11-28T21:48:25.572 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.571+0000 7f6a17fff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f6a20065750 0x7f6a20146300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:46042/0 (socket says 172.21.15.162:46042) 2020-11-28T21:48:25.572 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.571+0000 7f6a17fff700 1 -- 172.21.15.162:0/1466708141 learned_addr learned my addr 172.21.15.162:0/1466708141 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:48:25.572 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.571+0000 7f6a25230700 1 --2- 172.21.15.162:0/1466708141 >> v2:172.21.15.118:3300/0 conn(0x7f6a200b2b60 0x7f6a20146a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:25.572 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.571+0000 7f6a177fe700 1 -- 172.21.15.162:0/1466708141 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3131659345 0 0) 0x7f6a2009cb50 con 0x7f6a20065750 2020-11-28T21:48:25.572 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.571+0000 7f6a177fe700 1 -- 172.21.15.162:0/1466708141 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a0c003230 con 0x7f6a20065750 2020-11-28T21:48:25.575 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.571+0000 7f6a177fe700 1 -- 172.21.15.162:0/1466708141 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3840053480 0 0) 0x7f6a0c003230 con 0x7f6a20065750 2020-11-28T21:48:25.575 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.571+0000 7f6a177fe700 1 -- 172.21.15.162:0/1466708141 >> v2:172.21.15.118:3300/0 conn(0x7f6a200b2b60 msgr2=0x7f6a20146a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:25.575 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.571+0000 7f6a177fe700 1 --2- 172.21.15.162:0/1466708141 >> v2:172.21.15.118:3300/0 conn(0x7f6a200b2b60 0x7f6a20146a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:25.576 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.571+0000 7f6a177fe700 1 -- 172.21.15.162:0/1466708141 >> v1:172.21.15.118:6789/0 conn(0x7f6a200b3770 legacy=0x7f6a200b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:25.576 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.571+0000 7f6a177fe700 1 -- 172.21.15.162:0/1466708141 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a200b3f40 con 0x7f6a20065750 2020-11-28T21:48:25.576 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.571+0000 7f6a177fe700 1 -- 172.21.15.162:0/1466708141 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f6a100016d0 con 0x7f6a20065750 2020-11-28T21:48:25.576 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.571+0000 7f6a177fe700 1 -- 172.21.15.162:0/1466708141 >> v1:172.21.15.162:6789/0 conn(0x7f6a20065750 legacy=0x7f6a20146300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:25.576 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.571+0000 7f6a177fe700 1 --2- 172.21.15.162:0/1466708141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a0c003720 0x7f6a0c003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:25.577 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.571+0000 7f6a177fe700 1 --2- 172.21.15.162:0/1466708141 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6a0c0044d0 0x7f6a0c004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:25.577 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.571+0000 7f6a177fe700 1 --2- 172.21.15.162:0/1466708141 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6a0c004e40 0x7f6a0c00b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:25.577 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.571+0000 7f6a177fe700 1 -- 172.21.15.162:0/1466708141 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2009cdf0 con 0x7f6a0c0044d0 2020-11-28T21:48:25.577 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.571+0000 7f6a177fe700 1 -- 172.21.15.162:0/1466708141 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0c004250 con 0x7f6a0c003720 2020-11-28T21:48:25.578 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.571+0000 7f6a177fe700 1 -- 172.21.15.162:0/1466708141 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0c00bc10 con 0x7f6a0c004e40 2020-11-28T21:48:25.578 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.571+0000 7f6a24a2f700 1 --2- 172.21.15.162:0/1466708141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a0c003720 0x7f6a0c003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:25.578 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.571+0000 7f6a25230700 1 --2- 172.21.15.162:0/1466708141 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6a0c004e40 0x7f6a0c00b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:25.578 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.571+0000 7f6a17fff700 1 --2- 172.21.15.162:0/1466708141 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6a0c0044d0 0x7f6a0c004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:25.578 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.571+0000 7f6a24a2f700 1 -- 172.21.15.162:0/1466708141 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6a0c004e40 msgr2=0x7f6a0c00b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:25.579 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.571+0000 7f6a24a2f700 1 --2- 172.21.15.162:0/1466708141 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6a0c004e40 0x7f6a0c00b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:25.579 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.571+0000 7f6a24a2f700 1 -- 172.21.15.162:0/1466708141 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6a0c0044d0 msgr2=0x7f6a0c004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:25.579 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.571+0000 7f6a24a2f700 1 --2- 172.21.15.162:0/1466708141 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6a0c0044d0 0x7f6a0c004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:25.579 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.571+0000 7f6a24a2f700 1 -- 172.21.15.162:0/1466708141 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a0c003230 con 0x7f6a0c003720 2020-11-28T21:48:25.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.571+0000 7f6a25230700 1 -- 172.21.15.162:0/1466708141 reap_dead start 2020-11-28T21:48:25.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.571+0000 7f6a17fff700 1 --2- 172.21.15.162:0/1466708141 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6a0c0044d0 0x7f6a0c004950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:48:25.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.571+0000 7f6a24a2f700 1 --2- 172.21.15.162:0/1466708141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a0c003720 0x7f6a0c003b40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f6a20067080 tx=0x7f6a1c007b60).ready entity=mon.1 client_cookie=4244914bbe2cece5 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:25.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.571+0000 7f6a177fe700 1 -- 172.21.15.162:0/1466708141 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a1c00c660 con 0x7f6a0c003720 2020-11-28T21:48:25.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.571+0000 7f6a177fe700 1 -- 172.21.15.162:0/1466708141 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a1c00c7c0 con 0x7f6a0c003720 2020-11-28T21:48:25.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.571+0000 7f6a177fe700 1 -- 172.21.15.162:0/1466708141 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a1c00cb20 con 0x7f6a0c003720 2020-11-28T21:48:25.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.571+0000 7f6a26c93700 1 -- 172.21.15.162:0/1466708141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a0c003720 msgr2=0x7f6a0c003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:25.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.571+0000 7f6a26c93700 1 --2- 172.21.15.162:0/1466708141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a0c003720 0x7f6a0c003b40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f6a20067080 tx=0x7f6a1c007b60).stop 2020-11-28T21:48:25.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.575+0000 7f6a26c93700 1 -- 172.21.15.162:0/1466708141 shutdown_connections 2020-11-28T21:48:25.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.575+0000 7f6a26c93700 1 --2- 172.21.15.162:0/1466708141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a0c003720 0x7f6a0c003b40 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:25.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.575+0000 7f6a26c93700 1 -- 172.21.15.162:0/1466708141 >> 172.21.15.162:0/1466708141 conn(0x7f6a2005b970 msgr2=0x7f6a2005bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:25.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.575+0000 7f6a26c93700 1 -- 172.21.15.162:0/1466708141 shutdown_connections 2020-11-28T21:48:25.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.575+0000 7f6a26c93700 1 -- 172.21.15.162:0/1466708141 wait complete. 2020-11-28T21:48:25.586 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.575+0000 7f6a26c93700 1 Processor -- start 2020-11-28T21:48:25.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.575+0000 7f6a26c93700 1 -- start start 2020-11-28T21:48:25.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.575+0000 7f6a26c93700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6a0c003720 0x7f6a2009c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:25.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.575+0000 7f6a26c93700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a201553f0 0x7f6a20155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:25.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.575+0000 7f6a26c93700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6a20151ef0 0x7f6a201523a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:25.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.575+0000 7f6a26c93700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a20061820 con 0x7f6a20151ef0 2020-11-28T21:48:25.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.575+0000 7f6a26c93700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a20151b00 con 0x7f6a201553f0 2020-11-28T21:48:25.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.575+0000 7f6a26c93700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2005f490 con 0x7f6a0c003720 2020-11-28T21:48:25.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.575+0000 7f6a17fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a201553f0 0x7f6a20155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:25.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.575+0000 7f6a17fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a201553f0 0x7f6a20155870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39212/0 (socket says 172.21.15.162:39212) 2020-11-28T21:48:25.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.575+0000 7f6a24a2f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6a0c003720 0x7f6a2009c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:25.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.575+0000 7f6a17fff700 1 -- 172.21.15.162:0/2484952047 learned_addr learned my addr 172.21.15.162:0/2484952047 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:25.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.575+0000 7f6a25230700 1 --2- 172.21.15.162:0/2484952047 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6a20151ef0 0x7f6a201523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:25.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.579+0000 7f6a17fff700 1 -- 172.21.15.162:0/2484952047 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6a0c003720 msgr2=0x7f6a2009c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:25.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.579+0000 7f6a17fff700 1 --2- 172.21.15.162:0/2484952047 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6a0c003720 0x7f6a2009c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:25.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.579+0000 7f6a17fff700 1 -- 172.21.15.162:0/2484952047 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6a20151ef0 msgr2=0x7f6a201523a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:25.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.579+0000 7f6a17fff700 1 --2- 172.21.15.162:0/2484952047 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6a20151ef0 0x7f6a201523a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:25.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.579+0000 7f6a17fff700 1 -- 172.21.15.162:0/2484952047 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a2005f730 con 0x7f6a201553f0 2020-11-28T21:48:25.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.579+0000 7f6a17fff700 1 --2- 172.21.15.162:0/2484952047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a201553f0 0x7f6a20155870 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f6a10008000 tx=0x7f6a10009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:25.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.579+0000 7f6a15ffb700 1 -- 172.21.15.162:0/2484952047 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a10007170 con 0x7f6a201553f0 2020-11-28T21:48:25.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.579+0000 7f6a15ffb700 1 -- 172.21.15.162:0/2484952047 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a100072d0 con 0x7f6a201553f0 2020-11-28T21:48:25.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.579+0000 7f6a26c93700 1 -- 172.21.15.162:0/2484952047 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a20148760 con 0x7f6a201553f0 2020-11-28T21:48:25.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.579+0000 7f6a15ffb700 1 -- 172.21.15.162:0/2484952047 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a100048f0 con 0x7f6a201553f0 2020-11-28T21:48:25.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.579+0000 7f6a26c93700 1 -- 172.21.15.162:0/2484952047 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a200614a0 con 0x7f6a201553f0 2020-11-28T21:48:25.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.579+0000 7f6a15ffb700 1 -- 172.21.15.162:0/2484952047 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6a10004a50 con 0x7f6a201553f0 2020-11-28T21:48:25.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.579+0000 7f6a26c93700 1 -- 172.21.15.162:0/2484952047 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a2005f8c0 con 0x7f6a201553f0 2020-11-28T21:48:25.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.579+0000 7f6a15ffb700 1 --2- 172.21.15.162:0/2484952047 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6a00055330 0x7f6a000577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:25.594 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.579+0000 7f6a15ffb700 1 -- 172.21.15.162:0/2484952047 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f6a1006de50 con 0x7f6a201553f0 2020-11-28T21:48:25.594 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.579+0000 7f6a24a2f700 1 --2- 172.21.15.162:0/2484952047 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6a00055330 0x7f6a000577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:25.594 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.579+0000 7f6a24a2f700 1 --2- 172.21.15.162:0/2484952047 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6a00055330 0x7f6a000577f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f6a1c01b000 tx=0x7f6a1c017000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:25.594 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.583+0000 7f6a15ffb700 1 -- 172.21.15.162:0/2484952047 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6a10044210 con 0x7f6a201553f0 2020-11-28T21:48:25.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:25.727+0000 7f6a26c93700 1 -- 172.21.15.162:0/2484952047 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f6a20057050 con 0x7f6a201553f0 2020-11-28T21:48:26.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.435+0000 7f6a15ffb700 1 -- 172.21.15.162:0/2484952047 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v41) v1 ==== 125+0+0 (secure 0 0 0) 0x7f6a10047830 con 0x7f6a201553f0 2020-11-28T21:48:26.445 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.443+0000 7f6a26c93700 1 -- 172.21.15.162:0/2484952047 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6a00055330 msgr2=0x7f6a000577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:26.445 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.443+0000 7f6a26c93700 1 --2- 172.21.15.162:0/2484952047 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6a00055330 0x7f6a000577f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f6a1c01b000 tx=0x7f6a1c017000).stop 2020-11-28T21:48:26.446 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.443+0000 7f6a26c93700 1 -- 172.21.15.162:0/2484952047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a201553f0 msgr2=0x7f6a20155870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:26.446 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.443+0000 7f6a26c93700 1 --2- 172.21.15.162:0/2484952047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a201553f0 0x7f6a20155870 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f6a10008000 tx=0x7f6a10009040).stop 2020-11-28T21:48:26.446 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.447+0000 7f6a26c93700 1 -- 172.21.15.162:0/2484952047 shutdown_connections 2020-11-28T21:48:26.447 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.447+0000 7f6a26c93700 1 --2- 172.21.15.162:0/2484952047 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6a00055330 0x7f6a000577f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:26.447 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.447+0000 7f6a26c93700 1 --2- 172.21.15.162:0/2484952047 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6a0c003720 0x7f6a2009c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:26.447 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.447+0000 7f6a26c93700 1 --2- 172.21.15.162:0/2484952047 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6a20151ef0 0x7f6a201523a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:26.447 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.447+0000 7f6a26c93700 1 --2- 172.21.15.162:0/2484952047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a201553f0 0x7f6a20155870 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:26.448 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.447+0000 7f6a26c93700 1 -- 172.21.15.162:0/2484952047 >> 172.21.15.162:0/2484952047 conn(0x7f6a2005b970 msgr2=0x7f6a2005bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:26.448 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.447+0000 7f6a26c93700 1 -- 172.21.15.162:0/2484952047 shutdown_connections 2020-11-28T21:48:26.448 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.447+0000 7f6a26c93700 1 -- 172.21.15.162:0/2484952047 wait complete. 2020-11-28T21:48:26.449 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_data' created 2020-11-28T21:48:26.461 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2020-11-28T21:48:26.575 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.575+0000 7f8ac7fff700 1 Processor -- start 2020-11-28T21:48:26.575 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.575+0000 7f8ac7fff700 1 -- start start 2020-11-28T21:48:26.577 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.575+0000 7f8ac7fff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f8ac80b2d00 0x7f8ac805c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:26.578 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.575+0000 7f8ac7fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8ac80b3ab0 0x7f8ac80b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:26.578 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.575+0000 7f8ac7fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8ac8142ad0 0x7f8ac8142f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:26.578 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.575+0000 7f8ac7fff700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ac80b0e20 con 0x7f8ac80b2d00 2020-11-28T21:48:26.578 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.575+0000 7f8ac7fff700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ac805baf0 con 0x7f8ac80b3ab0 2020-11-28T21:48:26.578 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.575+0000 7f8ac7fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac805bd90 con 0x7f8ac8142ad0 2020-11-28T21:48:26.579 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.575+0000 7f8ac67fc700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8ac80b3ab0 0x7f8ac80b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:26.579 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.575+0000 7f8ac67fc700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8ac80b3ab0 0x7f8ac80b30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39220/0 (socket says 172.21.15.162:39220) 2020-11-28T21:48:26.579 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.575+0000 7f8ac67fc700 1 -- 172.21.15.162:0/3047118826 learned_addr learned my addr 172.21.15.162:0/3047118826 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:26.579 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.575+0000 7f8ac6ffd700 1 --2- 172.21.15.162:0/3047118826 >> v2:172.21.15.118:3300/0 conn(0x7f8ac80b2d00 0x7f8ac805c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:26.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.575+0000 7f8ac77fe700 1 --2- 172.21.15.162:0/3047118826 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8ac8142ad0 0x7f8ac8142f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:26.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.575+0000 7f8ac67fc700 1 -- 172.21.15.162:0/3047118826 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8ac8142ad0 msgr2=0x7f8ac8142f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:26.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.575+0000 7f8ac67fc700 1 --2- 172.21.15.162:0/3047118826 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8ac8142ad0 0x7f8ac8142f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:26.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.575+0000 7f8ac67fc700 1 -- 172.21.15.162:0/3047118826 >> v2:172.21.15.118:3300/0 conn(0x7f8ac80b2d00 msgr2=0x7f8ac805c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:26.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.575+0000 7f8ac67fc700 1 --2- 172.21.15.162:0/3047118826 >> v2:172.21.15.118:3300/0 conn(0x7f8ac80b2d00 0x7f8ac805c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:26.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.575+0000 7f8ac67fc700 1 -- 172.21.15.162:0/3047118826 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac805c030 con 0x7f8ac80b3ab0 2020-11-28T21:48:26.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.579+0000 7f8ac67fc700 1 --2- 172.21.15.162:0/3047118826 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ac80b3ab0 0x7f8ac80b30e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f8ab8002f30 tx=0x7f8ab8006e80).ready entity=mon.1 client_cookie=d3bc81b6aae39ad server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:26.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.579+0000 7f8ac5ffb700 1 -- 172.21.15.162:0/3047118826 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ab80097f0 con 0x7f8ac80b3ab0 2020-11-28T21:48:26.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.579+0000 7f8ac5ffb700 1 -- 172.21.15.162:0/3047118826 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab8009950 con 0x7f8ac80b3ab0 2020-11-28T21:48:26.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.579+0000 7f8ac5ffb700 1 -- 172.21.15.162:0/3047118826 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ab8009cb0 con 0x7f8ac80b3ab0 2020-11-28T21:48:26.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.579+0000 7f8ac7fff700 1 -- 172.21.15.162:0/3047118826 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ac80b3ab0 msgr2=0x7f8ac80b30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:26.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.579+0000 7f8ac7fff700 1 --2- 172.21.15.162:0/3047118826 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ac80b3ab0 0x7f8ac80b30e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f8ab8002f30 tx=0x7f8ab8006e80).stop 2020-11-28T21:48:26.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.579+0000 7f8ac7fff700 1 -- 172.21.15.162:0/3047118826 shutdown_connections 2020-11-28T21:48:26.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.579+0000 7f8ac7fff700 1 --2- 172.21.15.162:0/3047118826 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8ac8142ad0 0x7f8ac8142f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:26.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.579+0000 7f8ac7fff700 1 --2- 172.21.15.162:0/3047118826 >> v2:172.21.15.118:3300/0 conn(0x7f8ac80b2d00 0x7f8ac805c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:26.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.579+0000 7f8ac7fff700 1 --2- 172.21.15.162:0/3047118826 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ac80b3ab0 0x7f8ac80b30e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:26.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.579+0000 7f8ac7fff700 1 -- 172.21.15.162:0/3047118826 >> 172.21.15.162:0/3047118826 conn(0x7f8ac80604c0 msgr2=0x7f8ac80608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:26.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.579+0000 7f8ac7fff700 1 -- 172.21.15.162:0/3047118826 shutdown_connections 2020-11-28T21:48:26.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.579+0000 7f8ac7fff700 1 -- 172.21.15.162:0/3047118826 wait complete. 2020-11-28T21:48:26.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.579+0000 7f8ac7fff700 1 Processor -- start 2020-11-28T21:48:26.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.583+0000 7f8ac7fff700 1 -- start start 2020-11-28T21:48:26.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.583+0000 7f8ac7fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8ac80b2d00 0x7f8ac80b4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:26.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.583+0000 7f8ac7fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ac80b3ab0 0x7f8ac80b9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:26.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.583+0000 7f8ac7fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8ac8142ad0 0x7f8ac8154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:26.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.583+0000 7f8ac7fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac813dac0 con 0x7f8ac8142ad0 2020-11-28T21:48:26.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.583+0000 7f8ac7fff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac80b5d50 con 0x7f8ac80b3ab0 2020-11-28T21:48:26.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.583+0000 7f8ac7fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac813dcf0 con 0x7f8ac80b2d00 2020-11-28T21:48:26.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.583+0000 7f8ac67fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ac80b3ab0 0x7f8ac80b9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:26.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.583+0000 7f8ac67fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ac80b3ab0 0x7f8ac80b9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39226/0 (socket says 172.21.15.162:39226) 2020-11-28T21:48:26.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.583+0000 7f8ac67fc700 1 -- 172.21.15.162:0/569645147 learned_addr learned my addr 172.21.15.162:0/569645147 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:26.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.583+0000 7f8ac6ffd700 1 --2- 172.21.15.162:0/569645147 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8ac80b2d00 0x7f8ac80b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:26.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.583+0000 7f8ac77fe700 1 --2- 172.21.15.162:0/569645147 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8ac8142ad0 0x7f8ac8154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:26.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.583+0000 7f8ac67fc700 1 -- 172.21.15.162:0/569645147 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8ac80b2d00 msgr2=0x7f8ac80b4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:26.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.583+0000 7f8ac67fc700 1 --2- 172.21.15.162:0/569645147 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8ac80b2d00 0x7f8ac80b4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:26.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.583+0000 7f8ac67fc700 1 -- 172.21.15.162:0/569645147 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8ac8142ad0 msgr2=0x7f8ac8154eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:26.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.583+0000 7f8ac67fc700 1 --2- 172.21.15.162:0/569645147 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8ac8142ad0 0x7f8ac8154eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:26.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.583+0000 7f8ac67fc700 1 -- 172.21.15.162:0/569645147 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac813e040 con 0x7f8ac80b3ab0 2020-11-28T21:48:26.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.583+0000 7f8ac67fc700 1 --2- 172.21.15.162:0/569645147 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ac80b3ab0 0x7f8ac80b9900 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f8ab8002f30 tx=0x7f8ab8015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:26.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.583+0000 7f8ab7fff700 1 -- 172.21.15.162:0/569645147 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ab8011d70 con 0x7f8ac80b3ab0 2020-11-28T21:48:26.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.583+0000 7f8ab7fff700 1 -- 172.21.15.162:0/569645147 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab801e460 con 0x7f8ac80b3ab0 2020-11-28T21:48:26.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.583+0000 7f8ac7fff700 1 -- 172.21.15.162:0/569645147 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ac813f430 con 0x7f8ac80b3ab0 2020-11-28T21:48:26.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.583+0000 7f8ab7fff700 1 -- 172.21.15.162:0/569645147 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ab801e7c0 con 0x7f8ac80b3ab0 2020-11-28T21:48:26.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.583+0000 7f8ac7fff700 1 -- 172.21.15.162:0/569645147 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ac8137da0 con 0x7f8ac80b3ab0 2020-11-28T21:48:26.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.587+0000 7f8ac7fff700 1 -- 172.21.15.162:0/569645147 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ac80b5ec0 con 0x7f8ac80b3ab0 2020-11-28T21:48:26.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.587+0000 7f8ab7fff700 1 -- 172.21.15.162:0/569645147 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8ab8025030 con 0x7f8ac80b3ab0 2020-11-28T21:48:26.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.587+0000 7f8ab7fff700 1 --2- 172.21.15.162:0/569645147 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8aa4055440 0x7f8aa4057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:26.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.587+0000 7f8ab7fff700 1 -- 172.21.15.162:0/569645147 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5957+0+0 (secure 0 0 0) 0x7f8ab806f200 con 0x7f8ac80b3ab0 2020-11-28T21:48:26.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.587+0000 7f8ac6ffd700 1 --2- 172.21.15.162:0/569645147 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8aa4055440 0x7f8aa4057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:26.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.587+0000 7f8ac6ffd700 1 --2- 172.21.15.162:0/569645147 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8aa4055440 0x7f8aa4057900 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f8abc00c000 tx=0x7f8abc008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:26.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.591+0000 7f8ab7fff700 1 -- 172.21.15.162:0/569645147 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8ac80b5ec0 con 0x7f8ac80b3ab0 2020-11-28T21:48:26.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.779+0000 7f8ac7fff700 1 -- 172.21.15.162:0/569645147 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f8ac80b5e90 con 0x7f8ac80b3ab0 2020-11-28T21:48:26.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.791+0000 7f8ab7fff700 1 -- 172.21.15.162:0/569645147 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 7 and data pool 8 v14) v1 ==== 187+0+0 (secure 0 0 0) 0x7f8ab8023020 con 0x7f8ac80b3ab0 2020-11-28T21:48:26.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.795+0000 7f8ac7fff700 1 -- 172.21.15.162:0/569645147 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8aa4055440 msgr2=0x7f8aa4057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:26.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.795+0000 7f8ac7fff700 1 --2- 172.21.15.162:0/569645147 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8aa4055440 0x7f8aa4057900 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f8abc00c000 tx=0x7f8abc008040).stop 2020-11-28T21:48:26.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.799+0000 7f8ac7fff700 1 -- 172.21.15.162:0/569645147 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ac80b3ab0 msgr2=0x7f8ac80b9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:26.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.799+0000 7f8ac7fff700 1 --2- 172.21.15.162:0/569645147 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ac80b3ab0 0x7f8ac80b9900 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f8ab8002f30 tx=0x7f8ab8015000).stop 2020-11-28T21:48:26.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.799+0000 7f8ac7fff700 1 -- 172.21.15.162:0/569645147 shutdown_connections 2020-11-28T21:48:26.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.799+0000 7f8ac7fff700 1 --2- 172.21.15.162:0/569645147 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8aa4055440 0x7f8aa4057900 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:26.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.799+0000 7f8ac7fff700 1 --2- 172.21.15.162:0/569645147 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8ac80b2d00 0x7f8ac80b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:26.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.799+0000 7f8ac7fff700 1 --2- 172.21.15.162:0/569645147 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8ac8142ad0 0x7f8ac8154eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:26.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.799+0000 7f8ac7fff700 1 --2- 172.21.15.162:0/569645147 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ac80b3ab0 0x7f8ac80b9900 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:26.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.799+0000 7f8ac7fff700 1 -- 172.21.15.162:0/569645147 >> 172.21.15.162:0/569645147 conn(0x7f8ac80604c0 msgr2=0x7f8ac8149410 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:26.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.799+0000 7f8ac7fff700 1 -- 172.21.15.162:0/569645147 shutdown_connections 2020-11-28T21:48:26.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.799+0000 7f8ac7fff700 1 -- 172.21.15.162:0/569645147 wait complete. 2020-11-28T21:48:26.802 INFO:teuthology.orchestra.run.smithi162.stderr:new fs with metadata pool 7 and data pool 8 2020-11-28T21:48:26.815 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:48:26.915 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.915+0000 7f2337fff700 1 Processor -- start 2020-11-28T21:48:26.915 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.915+0000 7f2337fff700 1 -- start start 2020-11-28T21:48:26.917 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.915+0000 7f2337fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f23380994f0 0x7f23380b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:26.917 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.915+0000 7f2337fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2338060710 0x7f23380b1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:26.917 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.915+0000 7f2337fff700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2338098f10 con 0x7f2338098b30 2020-11-28T21:48:26.917 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.915+0000 7f2337fff700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23380b1030 con 0x7f23380994f0 2020-11-28T21:48:26.918 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.915+0000 7f2337fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23380b0680 con 0x7f2338060710 2020-11-28T21:48:26.918 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.915+0000 7f2336ffd700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f2338098b30 0x7f2338060000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:46086/0 (socket says 172.21.15.162:46086) 2020-11-28T21:48:26.918 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.915+0000 7f2336ffd700 1 -- 172.21.15.162:0/1503241091 learned_addr learned my addr 172.21.15.162:0/1503241091 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:48:26.918 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.915+0000 7f23367fc700 1 --2- 172.21.15.162:0/1503241091 >> v2:172.21.15.162:3300/0 conn(0x7f23380994f0 0x7f23380b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:26.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.915+0000 7f2335ffb700 1 -- 172.21.15.162:0/1503241091 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1536389821 0 0) 0x7f2338098f10 con 0x7f2338098b30 2020-11-28T21:48:26.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.915+0000 7f23377fe700 1 --2- 172.21.15.162:0/1503241091 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2338060710 0x7f23380b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:26.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.915+0000 7f2335ffb700 1 -- 172.21.15.162:0/1503241091 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2318003230 con 0x7f2338098b30 2020-11-28T21:48:26.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.915+0000 7f23367fc700 1 -- 172.21.15.162:0/1503241091 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2338060710 msgr2=0x7f23380b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:26.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.915+0000 7f23367fc700 1 --2- 172.21.15.162:0/1503241091 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2338060710 0x7f23380b1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:26.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.915+0000 7f23367fc700 1 -- 172.21.15.162:0/1503241091 >> v1:172.21.15.162:6789/0 conn(0x7f2338098b30 legacy=0x7f2338060000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:26.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.915+0000 7f23367fc700 1 -- 172.21.15.162:0/1503241091 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23380b0920 con 0x7f23380994f0 2020-11-28T21:48:26.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.915+0000 7f2335ffb700 1 -- 172.21.15.162:0/1503241091 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2081502637 0 0) 0x7f2318003230 con 0x7f2338098b30 2020-11-28T21:48:26.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.915+0000 7f23367fc700 1 --2- 172.21.15.162:0/1503241091 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23380994f0 0x7f23380b0000 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f2320002f30 tx=0x7f2320006dc0).ready entity=mon.1 client_cookie=6fb51a8e76f25385 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:26.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.915+0000 7f2335ffb700 1 -- 172.21.15.162:0/1503241091 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2320009760 con 0x7f23380994f0 2020-11-28T21:48:26.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.915+0000 7f2335ffb700 1 -- 172.21.15.162:0/1503241091 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23200098c0 con 0x7f23380994f0 2020-11-28T21:48:26.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.915+0000 7f2337fff700 1 -- 172.21.15.162:0/1503241091 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23380994f0 msgr2=0x7f23380b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:26.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.915+0000 7f2335ffb700 1 -- 172.21.15.162:0/1503241091 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2320009c20 con 0x7f23380994f0 2020-11-28T21:48:26.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.915+0000 7f2337fff700 1 --2- 172.21.15.162:0/1503241091 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23380994f0 0x7f23380b0000 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f2320002f30 tx=0x7f2320006dc0).stop 2020-11-28T21:48:26.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.915+0000 7f2337fff700 1 -- 172.21.15.162:0/1503241091 shutdown_connections 2020-11-28T21:48:26.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.915+0000 7f2337fff700 1 --2- 172.21.15.162:0/1503241091 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2338060710 0x7f23380b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:26.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.915+0000 7f2337fff700 1 --2- 172.21.15.162:0/1503241091 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23380994f0 0x7f23380b0000 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:26.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.915+0000 7f2337fff700 1 -- 172.21.15.162:0/1503241091 >> 172.21.15.162:0/1503241091 conn(0x7f233805b970 msgr2=0x7f233805bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:26.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.915+0000 7f2337fff700 1 -- 172.21.15.162:0/1503241091 shutdown_connections 2020-11-28T21:48:26.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.915+0000 7f2337fff700 1 -- 172.21.15.162:0/1503241091 wait complete. 2020-11-28T21:48:26.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.919+0000 7f2337fff700 1 Processor -- start 2020-11-28T21:48:26.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.919+0000 7f2337fff700 1 -- start start 2020-11-28T21:48:26.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.919+0000 7f2337fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2338060710 0x7f2338155f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:26.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.919+0000 7f2337fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2338098b30 0x7f2338156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:26.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.919+0000 7f2337fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23380994f0 0x7f2338064bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:26.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.919+0000 7f2337fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2338156af0 con 0x7f2338098b30 2020-11-28T21:48:26.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.919+0000 7f2337fff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233809b2f0 con 0x7f2338060710 2020-11-28T21:48:26.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.919+0000 7f2337fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2338066c70 con 0x7f23380994f0 2020-11-28T21:48:26.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.919+0000 7f2336ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2338060710 0x7f2338155f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:26.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.919+0000 7f2336ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2338060710 0x7f2338155f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39248/0 (socket says 172.21.15.162:39248) 2020-11-28T21:48:26.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.919+0000 7f2336ffd700 1 -- 172.21.15.162:0/790781499 learned_addr learned my addr 172.21.15.162:0/790781499 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:26.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.919+0000 7f23377fe700 1 --2- 172.21.15.162:0/790781499 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23380994f0 0x7f2338064bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:26.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.919+0000 7f23367fc700 1 --2- 172.21.15.162:0/790781499 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2338098b30 0x7f2338156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:26.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.919+0000 7f2336ffd700 1 -- 172.21.15.162:0/790781499 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23380994f0 msgr2=0x7f2338064bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:26.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.919+0000 7f2336ffd700 1 --2- 172.21.15.162:0/790781499 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23380994f0 0x7f2338064bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:26.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.919+0000 7f2336ffd700 1 -- 172.21.15.162:0/790781499 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2338098b30 msgr2=0x7f2338156600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:26.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.919+0000 7f2336ffd700 1 --2- 172.21.15.162:0/790781499 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2338098b30 0x7f2338156600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:26.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.919+0000 7f2336ffd700 1 -- 172.21.15.162:0/790781499 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2338066f10 con 0x7f2338060710 2020-11-28T21:48:26.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.919+0000 7f23367fc700 1 --2- 172.21.15.162:0/790781499 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2338098b30 0x7f2338156600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:48:26.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.919+0000 7f2336ffd700 1 --2- 172.21.15.162:0/790781499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2338060710 0x7f2338155f80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f2330000f00 tx=0x7f2330007af0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:26.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.919+0000 7f232ffff700 1 -- 172.21.15.162:0/790781499 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2330005c40 con 0x7f2338060710 2020-11-28T21:48:26.931 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.919+0000 7f232ffff700 1 -- 172.21.15.162:0/790781499 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2330005da0 con 0x7f2338060710 2020-11-28T21:48:26.931 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.919+0000 7f2337fff700 1 -- 172.21.15.162:0/790781499 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f233809b590 con 0x7f2338060710 2020-11-28T21:48:26.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.919+0000 7f232ffff700 1 -- 172.21.15.162:0/790781499 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2330007300 con 0x7f2338060710 2020-11-28T21:48:26.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.919+0000 7f2337fff700 1 -- 172.21.15.162:0/790781499 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23380650b0 con 0x7f2338060710 2020-11-28T21:48:26.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.919+0000 7f2337fff700 1 -- 172.21.15.162:0/790781499 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2338156470 con 0x7f2338060710 2020-11-28T21:48:26.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.923+0000 7f232ffff700 1 -- 172.21.15.162:0/790781499 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2330010030 con 0x7f2338060710 2020-11-28T21:48:26.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.923+0000 7f232ffff700 1 --2- 172.21.15.162:0/790781499 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f231c055440 0x7f231c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:26.933 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.923+0000 7f23367fc700 1 --2- 172.21.15.162:0/790781499 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f231c055440 0x7f231c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:26.933 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.923+0000 7f232ffff700 1 -- 172.21.15.162:0/790781499 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f233006e1f0 con 0x7f2338060710 2020-11-28T21:48:26.933 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.923+0000 7f23367fc700 1 --2- 172.21.15.162:0/790781499 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f231c055440 0x7f231c057900 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f2320007000 tx=0x7f2320015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:26.933 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:26.927+0000 7f232ffff700 1 -- 172.21.15.162:0/790781499 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f23300444a0 con 0x7f2338060710 2020-11-28T21:48:27.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.071+0000 7f2337fff700 1 -- 172.21.15.162:0/790781499 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23380670a0 con 0x7f2338060710 2020-11-28T21:48:27.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.071+0000 7f232ffff700 1 -- 172.21.15.162:0/790781499 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+16147 (secure 0 0 0) 0x7f2330047ac0 con 0x7f2338060710 2020-11-28T21:48:27.074 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:48:27.074 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":43,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:48:26.781944+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:48:25.281355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2020-11-28T21:48:25.782088+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:48:27.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.075+0000 7f2337fff700 1 -- 172.21.15.162:0/790781499 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f231c055440 msgr2=0x7f231c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:27.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.075+0000 7f2337fff700 1 --2- 172.21.15.162:0/790781499 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f231c055440 0x7f231c057900 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f2320007000 tx=0x7f2320015000).stop 2020-11-28T21:48:27.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.075+0000 7f2337fff700 1 -- 172.21.15.162:0/790781499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2338060710 msgr2=0x7f2338155f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:27.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.075+0000 7f2337fff700 1 --2- 172.21.15.162:0/790781499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2338060710 0x7f2338155f80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f2330000f00 tx=0x7f2330007af0).stop 2020-11-28T21:48:27.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.075+0000 7f2337fff700 1 -- 172.21.15.162:0/790781499 shutdown_connections 2020-11-28T21:48:27.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.075+0000 7f2337fff700 1 --2- 172.21.15.162:0/790781499 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f231c055440 0x7f231c057900 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:27.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.075+0000 7f2337fff700 1 --2- 172.21.15.162:0/790781499 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23380994f0 0x7f2338064bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:27.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.075+0000 7f2337fff700 1 --2- 172.21.15.162:0/790781499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2338060710 0x7f2338155f80 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:27.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.075+0000 7f2337fff700 1 --2- 172.21.15.162:0/790781499 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2338098b30 0x7f2338156600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:27.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.075+0000 7f2337fff700 1 -- 172.21.15.162:0/790781499 >> 172.21.15.162:0/790781499 conn(0x7f233805b970 msgr2=0x7f23380ab6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:27.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.075+0000 7f2337fff700 1 -- 172.21.15.162:0/790781499 shutdown_connections 2020-11-28T21:48:27.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.075+0000 7f2337fff700 1 -- 172.21.15.162:0/790781499 wait complete. 2020-11-28T21:48:27.088 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:48:27.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.199+0000 7f913359e700 1 Processor -- start 2020-11-28T21:48:27.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.199+0000 7f913359e700 1 -- start start 2020-11-28T21:48:27.200 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.199+0000 7f913359e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f91340994f0 0x7f91340b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:27.200 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.199+0000 7f913359e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f9134060710 0x7f91340b1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:27.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.199+0000 7f913359e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9134098f10 con 0x7f9134098b30 2020-11-28T21:48:27.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.199+0000 7f913359e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91340b1030 con 0x7f9134060710 2020-11-28T21:48:27.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.199+0000 7f913359e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91340b0680 con 0x7f91340994f0 2020-11-28T21:48:27.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.199+0000 7f9131d9b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f91340994f0 0x7f91340b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:27.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.199+0000 7f9131d9b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f91340994f0 0x7f91340b0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39258/0 (socket says 172.21.15.162:39258) 2020-11-28T21:48:27.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.199+0000 7f9131d9b700 1 -- 172.21.15.162:0/2522929006 learned_addr learned my addr 172.21.15.162:0/2522929006 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:27.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.199+0000 7f9132d9d700 1 --2- 172.21.15.162:0/2522929006 >> v2:172.21.15.118:3300/0 conn(0x7f9134060710 0x7f91340b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:27.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.199+0000 7f9131d9b700 1 -- 172.21.15.162:0/2522929006 >> v2:172.21.15.118:3300/0 conn(0x7f9134060710 msgr2=0x7f91340b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:27.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.199+0000 7f9131d9b700 1 --2- 172.21.15.162:0/2522929006 >> v2:172.21.15.118:3300/0 conn(0x7f9134060710 0x7f91340b1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:27.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.199+0000 7f9131d9b700 1 -- 172.21.15.162:0/2522929006 >> v1:172.21.15.118:6789/0 conn(0x7f9134098b30 legacy=0x7f9134060000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:27.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.199+0000 7f9131d9b700 1 -- 172.21.15.162:0/2522929006 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91340b0920 con 0x7f91340994f0 2020-11-28T21:48:27.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.199+0000 7f9131d9b700 1 --2- 172.21.15.162:0/2522929006 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91340994f0 0x7f91340b0000 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f911c0032b0 tx=0x7f911c009270).ready entity=mon.1 client_cookie=564abe62b10910bb server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:27.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.199+0000 7f913159a700 1 -- 172.21.15.162:0/2522929006 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f911c009e60 con 0x7f91340994f0 2020-11-28T21:48:27.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.199+0000 7f913159a700 1 -- 172.21.15.162:0/2522929006 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911c00bc00 con 0x7f91340994f0 2020-11-28T21:48:27.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.199+0000 7f913159a700 1 -- 172.21.15.162:0/2522929006 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f911c00bd70 con 0x7f91340994f0 2020-11-28T21:48:27.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.199+0000 7f913359e700 1 -- 172.21.15.162:0/2522929006 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91340994f0 msgr2=0x7f91340b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:27.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.199+0000 7f913359e700 1 --2- 172.21.15.162:0/2522929006 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91340994f0 0x7f91340b0000 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f911c0032b0 tx=0x7f911c009270).stop 2020-11-28T21:48:27.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.199+0000 7f913359e700 1 -- 172.21.15.162:0/2522929006 shutdown_connections 2020-11-28T21:48:27.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.199+0000 7f913359e700 1 --2- 172.21.15.162:0/2522929006 >> v2:172.21.15.118:3300/0 conn(0x7f9134060710 0x7f91340b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:27.213 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.199+0000 7f913359e700 1 --2- 172.21.15.162:0/2522929006 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91340994f0 0x7f91340b0000 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:27.213 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.199+0000 7f913359e700 1 -- 172.21.15.162:0/2522929006 >> 172.21.15.162:0/2522929006 conn(0x7f913405b970 msgr2=0x7f913405bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:27.213 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.199+0000 7f913359e700 1 -- 172.21.15.162:0/2522929006 shutdown_connections 2020-11-28T21:48:27.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.199+0000 7f913359e700 1 -- 172.21.15.162:0/2522929006 wait complete. 2020-11-28T21:48:27.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.203+0000 7f913359e700 1 Processor -- start 2020-11-28T21:48:27.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.203+0000 7f913359e700 1 -- start start 2020-11-28T21:48:27.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.203+0000 7f913359e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9134060710 0x7f91340ab1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:27.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.203+0000 7f913359e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9134098b30 0x7f91341598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:27.216 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.203+0000 7f913359e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91340994f0 0x7f9134159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:27.216 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.203+0000 7f913359e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91340ae230 con 0x7f9134098b30 2020-11-28T21:48:27.216 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.203+0000 7f913359e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91340adc10 con 0x7f91340994f0 2020-11-28T21:48:27.216 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.203+0000 7f913359e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9134159460 con 0x7f9134060710 2020-11-28T21:48:27.216 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.203+0000 7f9132d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91340994f0 0x7f9134159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:27.217 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.203+0000 7f9132d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91340994f0 0x7f9134159db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39266/0 (socket says 172.21.15.162:39266) 2020-11-28T21:48:27.217 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.203+0000 7f9132d9d700 1 -- 172.21.15.162:0/2694896091 learned_addr learned my addr 172.21.15.162:0/2694896091 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:27.217 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.203+0000 7f9131d9b700 1 --2- 172.21.15.162:0/2694896091 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9134098b30 0x7f91341598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:27.217 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.203+0000 7f913259c700 1 --2- 172.21.15.162:0/2694896091 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9134060710 0x7f91340ab1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:27.218 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.203+0000 7f9132d9d700 1 -- 172.21.15.162:0/2694896091 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9134060710 msgr2=0x7f91340ab1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:27.218 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.203+0000 7f9132d9d700 1 --2- 172.21.15.162:0/2694896091 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9134060710 0x7f91340ab1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:27.218 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.203+0000 7f9132d9d700 1 -- 172.21.15.162:0/2694896091 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9134098b30 msgr2=0x7f91341598c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:27.218 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.203+0000 7f9132d9d700 1 --2- 172.21.15.162:0/2694896091 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9134098b30 0x7f91341598c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:27.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.203+0000 7f9132d9d700 1 -- 172.21.15.162:0/2694896091 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91340ad970 con 0x7f91340994f0 2020-11-28T21:48:27.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.203+0000 7f913259c700 1 --2- 172.21.15.162:0/2694896091 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9134060710 0x7f91340ab1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:48:27.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.203+0000 7f9132d9d700 1 --2- 172.21.15.162:0/2694896091 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91340994f0 0x7f9134159db0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f912c006a50 tx=0x7f912c006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:27.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.203+0000 7f912b7fe700 1 -- 172.21.15.162:0/2694896091 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f912c00ca10 con 0x7f91340994f0 2020-11-28T21:48:27.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.203+0000 7f912b7fe700 1 -- 172.21.15.162:0/2694896091 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912c00be90 con 0x7f91340994f0 2020-11-28T21:48:27.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.203+0000 7f912b7fe700 1 -- 172.21.15.162:0/2694896091 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f912c00cd60 con 0x7f91340994f0 2020-11-28T21:48:27.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.203+0000 7f913359e700 1 -- 172.21.15.162:0/2694896091 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91340adf90 con 0x7f91340994f0 2020-11-28T21:48:27.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.203+0000 7f913359e700 1 -- 172.21.15.162:0/2694896091 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91340ad710 con 0x7f91340994f0 2020-11-28T21:48:27.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.207+0000 7f913359e700 1 -- 172.21.15.162:0/2694896091 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9134057050 con 0x7f91340994f0 2020-11-28T21:48:27.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.207+0000 7f912b7fe700 1 -- 172.21.15.162:0/2694896091 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f912c01c460 con 0x7f91340994f0 2020-11-28T21:48:27.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.207+0000 7f912b7fe700 1 --2- 172.21.15.162:0/2694896091 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9118055330 0x7f91180577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:27.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.207+0000 7f912b7fe700 1 -- 172.21.15.162:0/2694896091 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f912c06e320 con 0x7f91340994f0 2020-11-28T21:48:27.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.207+0000 7f913259c700 1 --2- 172.21.15.162:0/2694896091 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9118055330 0x7f91180577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:27.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.207+0000 7f913259c700 1 --2- 172.21.15.162:0/2694896091 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9118055330 0x7f91180577f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f912400d000 tx=0x7f9124009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:27.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.211+0000 7f912b7fe700 1 -- 172.21.15.162:0/2694896091 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f912c0445c0 con 0x7f91340994f0 2020-11-28T21:48:27.356 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.355+0000 7f913359e700 1 -- 172.21.15.162:0/2694896091 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91341595a0 con 0x7f91340994f0 2020-11-28T21:48:27.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.359+0000 7f912b7fe700 1 -- 172.21.15.162:0/2694896091 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+16147 (secure 0 0 0) 0x7f912c047be0 con 0x7f91340994f0 2020-11-28T21:48:27.359 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:48:27.359 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":43,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:48:26.781944+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:48:25.281355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2020-11-28T21:48:25.782088+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:48:27.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.359+0000 7f913359e700 1 -- 172.21.15.162:0/2694896091 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9118055330 msgr2=0x7f91180577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:27.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.359+0000 7f913359e700 1 --2- 172.21.15.162:0/2694896091 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9118055330 0x7f91180577f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f912400d000 tx=0x7f9124009040).stop 2020-11-28T21:48:27.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.359+0000 7f913359e700 1 -- 172.21.15.162:0/2694896091 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91340994f0 msgr2=0x7f9134159db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:27.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.359+0000 7f913359e700 1 --2- 172.21.15.162:0/2694896091 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91340994f0 0x7f9134159db0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f912c006a50 tx=0x7f912c006dc0).stop 2020-11-28T21:48:27.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.359+0000 7f913359e700 1 -- 172.21.15.162:0/2694896091 shutdown_connections 2020-11-28T21:48:27.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.359+0000 7f913359e700 1 --2- 172.21.15.162:0/2694896091 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9118055330 0x7f91180577f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:27.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.359+0000 7f913359e700 1 --2- 172.21.15.162:0/2694896091 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9134060710 0x7f91340ab1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:27.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.359+0000 7f913359e700 1 --2- 172.21.15.162:0/2694896091 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91340994f0 0x7f9134159db0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:27.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.359+0000 7f913359e700 1 --2- 172.21.15.162:0/2694896091 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9134098b30 0x7f91341598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:27.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.359+0000 7f913359e700 1 -- 172.21.15.162:0/2694896091 >> 172.21.15.162:0/2694896091 conn(0x7f913405b970 msgr2=0x7f9134149fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:27.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.359+0000 7f913359e700 1 -- 172.21.15.162:0/2694896091 shutdown_connections 2020-11-28T21:48:27.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.359+0000 7f913359e700 1 -- 172.21.15.162:0/2694896091 wait complete. 2020-11-28T21:48:27.372 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2020-11-28T21:48:27.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.491+0000 7f3e47fff700 1 Processor -- start 2020-11-28T21:48:27.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.491+0000 7f3e47fff700 1 -- start start 2020-11-28T21:48:27.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.491+0000 7f3e47fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3e480b3950 0x7f3e480b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:27.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.491+0000 7f3e47fff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e480b1f50 con 0x7f3e480b18c0 2020-11-28T21:48:27.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.491+0000 7f3e47fff700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e48143e90 con 0x7f3e480b44e0 2020-11-28T21:48:27.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.491+0000 7f3e47fff700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e480aab50 con 0x7f3e480b3950 2020-11-28T21:48:27.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.491+0000 7f3e46ffd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3e480b3950 0x7f3e480b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:27.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.491+0000 7f3e477fe700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f3e480b44e0 0x7f3e480b2c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:46118/0 (socket says 172.21.15.162:46118) 2020-11-28T21:48:27.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.491+0000 7f3e477fe700 1 -- 172.21.15.162:0/3885170190 learned_addr learned my addr 172.21.15.162:0/3885170190 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:48:27.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.491+0000 7f3e46ffd700 1 -- 172.21.15.162:0/3885170190 >> v1:172.21.15.162:6789/0 conn(0x7f3e480b44e0 legacy=0x7f3e480b2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:27.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.491+0000 7f3e45ffb700 1 -- 172.21.15.162:0/3885170190 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2969680165 0 0) 0x7f3e48143e90 con 0x7f3e480b44e0 2020-11-28T21:48:27.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.491+0000 7f3e46ffd700 1 -- 172.21.15.162:0/3885170190 >> v1:172.21.15.118:6789/0 conn(0x7f3e480b18c0 legacy=0x7f3e4813dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:27.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.491+0000 7f3e46ffd700 1 -- 172.21.15.162:0/3885170190 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e480aa8b0 con 0x7f3e480b3950 2020-11-28T21:48:27.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.491+0000 7f3e46ffd700 1 --2- 172.21.15.162:0/3885170190 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e480b3950 0x7f3e480b3d30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f3e300010d0 tx=0x7f3e38006dc0).ready entity=mon.1 client_cookie=8c0410658d930901 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:27.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.491+0000 7f3e45ffb700 1 -- 172.21.15.162:0/3885170190 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3e380096b0 con 0x7f3e480b3950 2020-11-28T21:48:27.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.491+0000 7f3e45ffb700 1 -- 172.21.15.162:0/3885170190 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e38009810 con 0x7f3e480b3950 2020-11-28T21:48:27.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.491+0000 7f3e45ffb700 1 -- 172.21.15.162:0/3885170190 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3e38009b70 con 0x7f3e480b3950 2020-11-28T21:48:27.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.491+0000 7f3e47fff700 1 -- 172.21.15.162:0/3885170190 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e480b3950 msgr2=0x7f3e480b3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:27.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.491+0000 7f3e47fff700 1 --2- 172.21.15.162:0/3885170190 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e480b3950 0x7f3e480b3d30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f3e300010d0 tx=0x7f3e38006dc0).stop 2020-11-28T21:48:27.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.491+0000 7f3e47fff700 1 -- 172.21.15.162:0/3885170190 shutdown_connections 2020-11-28T21:48:27.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.491+0000 7f3e47fff700 1 --2- 172.21.15.162:0/3885170190 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e480b3950 0x7f3e480b3d30 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:27.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.491+0000 7f3e47fff700 1 -- 172.21.15.162:0/3885170190 >> 172.21.15.162:0/3885170190 conn(0x7f3e48062630 msgr2=0x7f3e48062a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:27.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.491+0000 7f3e47fff700 1 -- 172.21.15.162:0/3885170190 shutdown_connections 2020-11-28T21:48:27.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.491+0000 7f3e47fff700 1 -- 172.21.15.162:0/3885170190 wait complete. 2020-11-28T21:48:27.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.495+0000 7f3e47fff700 1 Processor -- start 2020-11-28T21:48:27.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.495+0000 7f3e47fff700 1 -- start start 2020-11-28T21:48:27.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.495+0000 7f3e47fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3e480b18c0 0x7f3e4813e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:27.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.495+0000 7f3e47fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3e480b3950 0x7f3e48143060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:27.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.495+0000 7f3e47fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e480b44e0 0x7f3e481400c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:27.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.495+0000 7f3e47fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e480b0960 con 0x7f3e480b18c0 2020-11-28T21:48:27.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.495+0000 7f3e47fff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e480af2b0 con 0x7f3e480b44e0 2020-11-28T21:48:27.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.495+0000 7f3e47fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e480af550 con 0x7f3e480b3950 2020-11-28T21:48:27.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.495+0000 7f3e477fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e480b44e0 0x7f3e481400c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:27.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.495+0000 7f3e46ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3e480b18c0 0x7f3e4813e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:27.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.495+0000 7f3e46ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3e480b18c0 0x7f3e4813e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.162:46264/0 (socket says 172.21.15.162:46264) 2020-11-28T21:48:27.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.495+0000 7f3e46ffd700 1 -- 172.21.15.162:0/4001639559 learned_addr learned my addr 172.21.15.162:0/4001639559 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:27.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.495+0000 7f3e467fc700 1 --2- 172.21.15.162:0/4001639559 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3e480b3950 0x7f3e48143060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:27.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.495+0000 7f3e477fe700 1 -- 172.21.15.162:0/4001639559 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3e480b3950 msgr2=0x7f3e48143060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:27.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.495+0000 7f3e477fe700 1 --2- 172.21.15.162:0/4001639559 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3e480b3950 0x7f3e48143060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:27.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.495+0000 7f3e477fe700 1 -- 172.21.15.162:0/4001639559 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3e480b18c0 msgr2=0x7f3e4813e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:27.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.495+0000 7f3e477fe700 1 --2- 172.21.15.162:0/4001639559 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3e480b18c0 0x7f3e4813e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:27.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.495+0000 7f3e477fe700 1 -- 172.21.15.162:0/4001639559 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e480af7f0 con 0x7f3e480b44e0 2020-11-28T21:48:27.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.495+0000 7f3e46ffd700 1 --2- 172.21.15.162:0/4001639559 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3e480b18c0 0x7f3e4813e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:48:27.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.495+0000 7f3e467fc700 1 --2- 172.21.15.162:0/4001639559 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3e480b3950 0x7f3e48143060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:48:27.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.495+0000 7f3e477fe700 1 --2- 172.21.15.162:0/4001639559 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e480b44e0 0x7f3e481400c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f3e40001210 tx=0x7f3e40007ef0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:27.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.495+0000 7f3e3ffff700 1 -- 172.21.15.162:0/4001639559 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3e40005b70 con 0x7f3e480b44e0 2020-11-28T21:48:27.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.495+0000 7f3e47fff700 1 -- 172.21.15.162:0/4001639559 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e480b05e0 con 0x7f3e480b44e0 2020-11-28T21:48:27.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.495+0000 7f3e3ffff700 1 -- 172.21.15.162:0/4001639559 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e40005cd0 con 0x7f3e480b44e0 2020-11-28T21:48:27.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.495+0000 7f3e3ffff700 1 -- 172.21.15.162:0/4001639559 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3e40004a60 con 0x7f3e480b44e0 2020-11-28T21:48:27.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.495+0000 7f3e47fff700 1 -- 172.21.15.162:0/4001639559 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e48140d80 con 0x7f3e480b44e0 2020-11-28T21:48:27.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.495+0000 7f3e47fff700 1 -- 172.21.15.162:0/4001639559 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e48057050 con 0x7f3e480b44e0 2020-11-28T21:48:27.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.499+0000 7f3e3ffff700 1 -- 172.21.15.162:0/4001639559 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f3e40010070 con 0x7f3e480b44e0 2020-11-28T21:48:27.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.499+0000 7f3e3ffff700 1 --2- 172.21.15.162:0/4001639559 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3e2c0553f0 0x7f3e2c0578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:27.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.499+0000 7f3e3ffff700 1 -- 172.21.15.162:0/4001639559 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3e4006da10 con 0x7f3e480b44e0 2020-11-28T21:48:27.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.499+0000 7f3e46ffd700 1 --2- 172.21.15.162:0/4001639559 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3e2c0553f0 0x7f3e2c0578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:27.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.499+0000 7f3e46ffd700 1 --2- 172.21.15.162:0/4001639559 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3e2c0553f0 0x7f3e2c0578b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f3e3801b000 tx=0x7f3e38015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:27.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.503+0000 7f3e3ffff700 1 -- 172.21.15.162:0/4001639559 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f3e40043b80 con 0x7f3e480b44e0 2020-11-28T21:48:27.688 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.687+0000 7f3e47fff700 1 -- 172.21.15.162:0/4001639559 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f3e4805d250 con 0x7f3e480b44e0 2020-11-28T21:48:27.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.791+0000 7f3e3ffff700 1 -- 172.21.15.162:0/4001639559 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3e40019030 con 0x7f3e480b44e0 2020-11-28T21:48:27.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.795+0000 7f3e47fff700 1 -- 172.21.15.162:0/4001639559 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3e2c0553f0 msgr2=0x7f3e2c0578b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:27.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.795+0000 7f3e47fff700 1 --2- 172.21.15.162:0/4001639559 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3e2c0553f0 0x7f3e2c0578b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f3e3801b000 tx=0x7f3e38015000).stop 2020-11-28T21:48:27.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.795+0000 7f3e47fff700 1 -- 172.21.15.162:0/4001639559 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e480b44e0 msgr2=0x7f3e481400c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:27.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.795+0000 7f3e47fff700 1 --2- 172.21.15.162:0/4001639559 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e480b44e0 0x7f3e481400c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f3e40001210 tx=0x7f3e40007ef0).stop 2020-11-28T21:48:27.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.795+0000 7f3e47fff700 1 -- 172.21.15.162:0/4001639559 shutdown_connections 2020-11-28T21:48:27.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.795+0000 7f3e47fff700 1 --2- 172.21.15.162:0/4001639559 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3e2c0553f0 0x7f3e2c0578b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:27.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.795+0000 7f3e47fff700 1 --2- 172.21.15.162:0/4001639559 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e480b44e0 0x7f3e481400c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:27.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.795+0000 7f3e47fff700 1 --2- 172.21.15.162:0/4001639559 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3e480b18c0 0x7f3e4813e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:27.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.795+0000 7f3e47fff700 1 --2- 172.21.15.162:0/4001639559 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3e480b3950 0x7f3e48143060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:27.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.795+0000 7f3e47fff700 1 -- 172.21.15.162:0/4001639559 >> 172.21.15.162:0/4001639559 conn(0x7f3e48062630 msgr2=0x7f3e48141cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:27.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.795+0000 7f3e47fff700 1 -- 172.21.15.162:0/4001639559 shutdown_connections 2020-11-28T21:48:27.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.795+0000 7f3e47fff700 1 -- 172.21.15.162:0/4001639559 wait complete. 2020-11-28T21:48:27.812 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:48:27.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.923+0000 7f6e5692d700 1 Processor -- start 2020-11-28T21:48:27.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.923+0000 7f6e5692d700 1 -- start start 2020-11-28T21:48:27.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.927+0000 7f6e5692d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6e50098b30 0x7f6e500994f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:27.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.927+0000 7f6e5692d700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6e500b0000 0x7f6e50060000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:27.933 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.927+0000 7f6e5692d700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e50098f10 con 0x7f6e500604f0 2020-11-28T21:48:27.933 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.927+0000 7f6e5692d700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e500b1030 con 0x7f6e50098b30 2020-11-28T21:48:27.933 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.927+0000 7f6e5692d700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e500b0480 con 0x7f6e500b0000 2020-11-28T21:48:27.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.927+0000 7f6e5592b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6e50098b30 0x7f6e500994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:27.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.927+0000 7f6e5612c700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f6e500604f0 0x7f6e5014a550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:46130/0 (socket says 172.21.15.162:46130) 2020-11-28T21:48:27.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.927+0000 7f6e5612c700 1 -- 172.21.15.162:0/666734672 learned_addr learned my addr 172.21.15.162:0/666734672 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:48:27.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.927+0000 7f6e5512a700 1 --2- 172.21.15.162:0/666734672 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6e500b0000 0x7f6e50060000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:27.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.927+0000 7f6e5592b700 1 -- 172.21.15.162:0/666734672 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6e500b0000 msgr2=0x7f6e50060000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:27.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.927+0000 7f6e5592b700 1 --2- 172.21.15.162:0/666734672 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6e500b0000 0x7f6e50060000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:27.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.927+0000 7f6e5592b700 1 -- 172.21.15.162:0/666734672 >> v1:172.21.15.162:6789/0 conn(0x7f6e500604f0 legacy=0x7f6e5014a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:27.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.927+0000 7f6e54929700 1 -- 172.21.15.162:0/666734672 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3373112746 0 0) 0x7f6e50098f10 con 0x7f6e500604f0 2020-11-28T21:48:27.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.927+0000 7f6e5592b700 1 -- 172.21.15.162:0/666734672 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e500b0680 con 0x7f6e50098b30 2020-11-28T21:48:27.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.927+0000 7f6e5592b700 1 --2- 172.21.15.162:0/666734672 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e50098b30 0x7f6e500994f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f6e50099be0 tx=0x7f6e40006dc0).ready entity=mon.1 client_cookie=d73980dd239405c9 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:27.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.927+0000 7f6e54929700 1 -- 172.21.15.162:0/666734672 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e40009720 con 0x7f6e50098b30 2020-11-28T21:48:27.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.927+0000 7f6e54929700 1 -- 172.21.15.162:0/666734672 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e40009880 con 0x7f6e50098b30 2020-11-28T21:48:27.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.927+0000 7f6e54929700 1 -- 172.21.15.162:0/666734672 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e40009be0 con 0x7f6e50098b30 2020-11-28T21:48:27.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.927+0000 7f6e5692d700 1 -- 172.21.15.162:0/666734672 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e50098b30 msgr2=0x7f6e500994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:27.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.927+0000 7f6e5692d700 1 --2- 172.21.15.162:0/666734672 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e50098b30 0x7f6e500994f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f6e50099be0 tx=0x7f6e40006dc0).stop 2020-11-28T21:48:27.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.927+0000 7f6e5692d700 1 -- 172.21.15.162:0/666734672 shutdown_connections 2020-11-28T21:48:27.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.927+0000 7f6e5692d700 1 --2- 172.21.15.162:0/666734672 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e50098b30 0x7f6e500994f0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:27.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.927+0000 7f6e5692d700 1 --2- 172.21.15.162:0/666734672 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6e500b0000 0x7f6e50060000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:27.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.927+0000 7f6e5692d700 1 -- 172.21.15.162:0/666734672 >> 172.21.15.162:0/666734672 conn(0x7f6e5005b970 msgr2=0x7f6e5005bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:27.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.927+0000 7f6e5692d700 1 -- 172.21.15.162:0/666734672 shutdown_connections 2020-11-28T21:48:27.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.927+0000 7f6e5692d700 1 -- 172.21.15.162:0/666734672 wait complete. 2020-11-28T21:48:27.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.931+0000 7f6e5692d700 1 Processor -- start 2020-11-28T21:48:27.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.931+0000 7f6e5692d700 1 -- start start 2020-11-28T21:48:27.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.931+0000 7f6e5692d700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6e500604f0 0x7f6e50063340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:27.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.931+0000 7f6e5692d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6e50098b30 0x7f6e50065170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:27.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.931+0000 7f6e5692d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e500b0000 0x7f6e50065660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:27.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.931+0000 7f6e5692d700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e500672c0 con 0x7f6e50098b30 2020-11-28T21:48:27.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.931+0000 7f6e5692d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e501567c0 con 0x7f6e500b0000 2020-11-28T21:48:27.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.931+0000 7f6e5692d700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e50156a60 con 0x7f6e500604f0 2020-11-28T21:48:27.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.931+0000 7f6e5612c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e500b0000 0x7f6e50065660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:27.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.931+0000 7f6e5512a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6e50098b30 0x7f6e50065170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:27.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.931+0000 7f6e5612c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e500b0000 0x7f6e50065660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39294/0 (socket says 172.21.15.162:39294) 2020-11-28T21:48:27.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.931+0000 7f6e5612c700 1 -- 172.21.15.162:0/2408462758 learned_addr learned my addr 172.21.15.162:0/2408462758 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:27.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.931+0000 7f6e5592b700 1 --2- 172.21.15.162:0/2408462758 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6e500604f0 0x7f6e50063340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:27.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.931+0000 7f6e5612c700 1 -- 172.21.15.162:0/2408462758 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6e500604f0 msgr2=0x7f6e50063340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:27.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.931+0000 7f6e5612c700 1 --2- 172.21.15.162:0/2408462758 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6e500604f0 0x7f6e50063340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:27.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.931+0000 7f6e5612c700 1 -- 172.21.15.162:0/2408462758 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6e50098b30 msgr2=0x7f6e50065170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:27.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.931+0000 7f6e5612c700 1 --2- 172.21.15.162:0/2408462758 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6e50098b30 0x7f6e50065170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:27.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.931+0000 7f6e5612c700 1 -- 172.21.15.162:0/2408462758 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e50063830 con 0x7f6e500b0000 2020-11-28T21:48:27.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.931+0000 7f6e5612c700 1 --2- 172.21.15.162:0/2408462758 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e500b0000 0x7f6e50065660 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f6e48001210 tx=0x7f6e48007d90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:27.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.931+0000 7f6e4effd700 1 -- 172.21.15.162:0/2408462758 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e48005b70 con 0x7f6e500b0000 2020-11-28T21:48:27.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.931+0000 7f6e4effd700 1 -- 172.21.15.162:0/2408462758 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e48005cd0 con 0x7f6e500b0000 2020-11-28T21:48:27.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.931+0000 7f6e5692d700 1 -- 172.21.15.162:0/2408462758 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e50066f40 con 0x7f6e500b0000 2020-11-28T21:48:27.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.931+0000 7f6e4effd700 1 -- 172.21.15.162:0/2408462758 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e48004a60 con 0x7f6e500b0000 2020-11-28T21:48:27.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.931+0000 7f6e5692d700 1 -- 172.21.15.162:0/2408462758 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e500acfa0 con 0x7f6e500b0000 2020-11-28T21:48:27.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.935+0000 7f6e5692d700 1 -- 172.21.15.162:0/2408462758 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e50156c20 con 0x7f6e500b0000 2020-11-28T21:48:27.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.935+0000 7f6e4effd700 1 -- 172.21.15.162:0/2408462758 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6e48010070 con 0x7f6e500b0000 2020-11-28T21:48:27.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.935+0000 7f6e4effd700 1 --2- 172.21.15.162:0/2408462758 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6e300553b0 0x7f6e30057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:27.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.935+0000 7f6e4effd700 1 -- 172.21.15.162:0/2408462758 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6e4806d8f0 con 0x7f6e500b0000 2020-11-28T21:48:27.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.935+0000 7f6e5592b700 1 --2- 172.21.15.162:0/2408462758 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6e300553b0 0x7f6e30057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:27.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.935+0000 7f6e5592b700 1 --2- 172.21.15.162:0/2408462758 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6e300553b0 0x7f6e30057870 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f6e4001b000 tx=0x7f6e40015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:27.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:27.939+0000 7f6e4effd700 1 -- 172.21.15.162:0/2408462758 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6e48043bf0 con 0x7f6e500b0000 2020-11-28T21:48:28.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.119+0000 7f6e5692d700 1 -- 172.21.15.162:0/2408462758 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e50057050 con 0x7f6e500b0000 2020-11-28T21:48:28.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.119+0000 7f6e4effd700 1 -- 172.21.15.162:0/2408462758 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+2497 (secure 0 0 0) 0x7f6e48047210 con 0x7f6e500b0000 2020-11-28T21:48:28.122 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:48:28.122 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":15,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4856,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/913296056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":913296056},{"type":"v1","addr":"172.21.15.118:6833","nonce":913296056}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":13},{"gid":5209,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6835/1318110356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1318110356},{"type":"v1","addr":"172.21.15.162:6835","nonce":1318110356}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:48:26.782354+0000","modified":"2020-11-28T21:48:27.785277+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5212},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5212":{"gid":5212,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/2706248574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2706248574},{"type":"v1","addr":"172.21.15.162:6837","nonce":2706248574}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-28T21:48:28.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.123+0000 7f6e5692d700 1 -- 172.21.15.162:0/2408462758 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6e300553b0 msgr2=0x7f6e30057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:28.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.123+0000 7f6e5692d700 1 --2- 172.21.15.162:0/2408462758 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6e300553b0 0x7f6e30057870 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f6e4001b000 tx=0x7f6e40015000).stop 2020-11-28T21:48:28.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.123+0000 7f6e5692d700 1 -- 172.21.15.162:0/2408462758 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e500b0000 msgr2=0x7f6e50065660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:28.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.123+0000 7f6e5692d700 1 --2- 172.21.15.162:0/2408462758 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e500b0000 0x7f6e50065660 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f6e48001210 tx=0x7f6e48007d90).stop 2020-11-28T21:48:28.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.123+0000 7f6e5692d700 1 -- 172.21.15.162:0/2408462758 shutdown_connections 2020-11-28T21:48:28.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.123+0000 7f6e5692d700 1 --2- 172.21.15.162:0/2408462758 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6e300553b0 0x7f6e30057870 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:28.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.123+0000 7f6e5692d700 1 --2- 172.21.15.162:0/2408462758 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6e500604f0 0x7f6e50063340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:28.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.123+0000 7f6e5692d700 1 --2- 172.21.15.162:0/2408462758 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e500b0000 0x7f6e50065660 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:28.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.123+0000 7f6e5692d700 1 --2- 172.21.15.162:0/2408462758 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6e50098b30 0x7f6e50065170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:28.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.123+0000 7f6e5692d700 1 -- 172.21.15.162:0/2408462758 >> 172.21.15.162:0/2408462758 conn(0x7f6e5005b970 msgr2=0x7f6e500b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:28.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.123+0000 7f6e5692d700 1 -- 172.21.15.162:0/2408462758 shutdown_connections 2020-11-28T21:48:28.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.123+0000 7f6e5692d700 1 -- 172.21.15.162:0/2408462758 wait complete. 2020-11-28T21:48:28.128 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 15 2020-11-28T21:48:28.134 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:48:28.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.251+0000 7fd5008cb700 1 Processor -- start 2020-11-28T21:48:28.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.251+0000 7fd5008cb700 1 -- start start 2020-11-28T21:48:28.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.251+0000 7fd5008cb700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd4fc0b1fd0 0x7fd4fc0afb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:28.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.251+0000 7fd5008cb700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd4fc0b0060 0x7fd4fc0aab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:28.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.251+0000 7fd5008cb700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4fc0b0f30 con 0x7fd4fc0a9bc0 2020-11-28T21:48:28.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.251+0000 7fd5008cb700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4fc061030 con 0x7fd4fc0b1fd0 2020-11-28T21:48:28.253 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.251+0000 7fd5008cb700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4fc0af980 con 0x7fd4fc0b0060 2020-11-28T21:48:28.253 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.251+0000 7fd4fb59e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd4fc0b0060 0x7fd4fc0aab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:28.253 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.251+0000 7fd4fb59e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd4fc0b0060 0x7fd4fc0aab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39298/0 (socket says 172.21.15.162:39298) 2020-11-28T21:48:28.253 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.251+0000 7fd4fb59e700 1 -- 172.21.15.162:0/3880773825 learned_addr learned my addr 172.21.15.162:0/3880773825 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:28.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.251+0000 7fd4fa59c700 1 --2- 172.21.15.162:0/3880773825 >> v2:172.21.15.118:3300/0 conn(0x7fd4fc0b1fd0 0x7fd4fc0afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:28.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.251+0000 7fd4fb59e700 1 -- 172.21.15.162:0/3880773825 >> v2:172.21.15.118:3300/0 conn(0x7fd4fc0b1fd0 msgr2=0x7fd4fc0afb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:28.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.251+0000 7fd4fb59e700 1 --2- 172.21.15.162:0/3880773825 >> v2:172.21.15.118:3300/0 conn(0x7fd4fc0b1fd0 0x7fd4fc0afb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:28.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.251+0000 7fd4fb59e700 1 -- 172.21.15.162:0/3880773825 >> v1:172.21.15.118:6789/0 conn(0x7fd4fc0a9bc0 legacy=0x7fd4fc0a9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:28.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.251+0000 7fd4fb59e700 1 -- 172.21.15.162:0/3880773825 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4fc0aa6b0 con 0x7fd4fc0b0060 2020-11-28T21:48:28.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.251+0000 7fd4fa59c700 1 --2- 172.21.15.162:0/3880773825 >> v2:172.21.15.118:3300/0 conn(0x7fd4fc0b1fd0 0x7fd4fc0afb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:48:28.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.251+0000 7fd4fb59e700 1 --2- 172.21.15.162:0/3880773825 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4fc0b0060 0x7fd4fc0aab20 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7fd4f40032b0 tx=0x7fd4f400a410).ready entity=mon.1 client_cookie=b2de9c321e49811e server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:28.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.251+0000 7fd4f9d9b700 1 -- 172.21.15.162:0/3880773825 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4f4010410 con 0x7fd4fc0b0060 2020-11-28T21:48:28.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.251+0000 7fd4f9d9b700 1 -- 172.21.15.162:0/3880773825 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f4010570 con 0x7fd4fc0b0060 2020-11-28T21:48:28.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.251+0000 7fd4f9d9b700 1 -- 172.21.15.162:0/3880773825 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4f40108d0 con 0x7fd4fc0b0060 2020-11-28T21:48:28.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.251+0000 7fd5008cb700 1 -- 172.21.15.162:0/3880773825 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4fc0b0060 msgr2=0x7fd4fc0aab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:28.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.251+0000 7fd5008cb700 1 --2- 172.21.15.162:0/3880773825 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4fc0b0060 0x7fd4fc0aab20 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7fd4f40032b0 tx=0x7fd4f400a410).stop 2020-11-28T21:48:28.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.251+0000 7fd5008cb700 1 -- 172.21.15.162:0/3880773825 shutdown_connections 2020-11-28T21:48:28.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.251+0000 7fd5008cb700 1 --2- 172.21.15.162:0/3880773825 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4fc0b0060 0x7fd4fc0aab20 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:28.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.251+0000 7fd5008cb700 1 --2- 172.21.15.162:0/3880773825 >> v2:172.21.15.118:3300/0 conn(0x7fd4fc0b1fd0 0x7fd4fc0afb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:28.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.251+0000 7fd5008cb700 1 -- 172.21.15.162:0/3880773825 >> 172.21.15.162:0/3880773825 conn(0x7fd4fc05b970 msgr2=0x7fd4fc05bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:28.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.251+0000 7fd5008cb700 1 -- 172.21.15.162:0/3880773825 shutdown_connections 2020-11-28T21:48:28.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.251+0000 7fd5008cb700 1 -- 172.21.15.162:0/3880773825 wait complete. 2020-11-28T21:48:28.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.255+0000 7fd5008cb700 1 Processor -- start 2020-11-28T21:48:28.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.255+0000 7fd5008cb700 1 -- start start 2020-11-28T21:48:28.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.255+0000 7fd5008cb700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd4fc0a9bc0 0x7fd4fc140480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:28.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.255+0000 7fd5008cb700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd4fc0b0060 0x7fd4fc0aac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:28.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.255+0000 7fd5008cb700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4fc0b1fd0 0x7fd4fc141900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:28.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.255+0000 7fd5008cb700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc141df0 con 0x7fd4fc0a9bc0 2020-11-28T21:48:28.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.255+0000 7fd5008cb700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc064680 con 0x7fd4fc0b1fd0 2020-11-28T21:48:28.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.255+0000 7fd5008cb700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc149100 con 0x7fd4fc0b0060 2020-11-28T21:48:28.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.255+0000 7fd4fb59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4fc0b1fd0 0x7fd4fc141900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:28.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.255+0000 7fd4fad9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd4fc0a9bc0 0x7fd4fc140480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:28.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.255+0000 7fd4fad9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd4fc0a9bc0 0x7fd4fc140480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.162:46292/0 (socket says 172.21.15.162:46292) 2020-11-28T21:48:28.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.255+0000 7fd4fad9d700 1 -- 172.21.15.162:0/2921294848 learned_addr learned my addr 172.21.15.162:0/2921294848 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:28.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.255+0000 7fd4fa59c700 1 --2- 172.21.15.162:0/2921294848 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd4fc0b0060 0x7fd4fc0aac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:28.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.255+0000 7fd4fb59e700 1 -- 172.21.15.162:0/2921294848 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd4fc0b0060 msgr2=0x7fd4fc0aac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:28.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.255+0000 7fd4fb59e700 1 --2- 172.21.15.162:0/2921294848 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd4fc0b0060 0x7fd4fc0aac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:28.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.255+0000 7fd4fb59e700 1 -- 172.21.15.162:0/2921294848 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd4fc0a9bc0 msgr2=0x7fd4fc140480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:28.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.255+0000 7fd4fb59e700 1 --2- 172.21.15.162:0/2921294848 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd4fc0a9bc0 0x7fd4fc140480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:28.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.255+0000 7fd4fb59e700 1 -- 172.21.15.162:0/2921294848 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4fc149360 con 0x7fd4fc0b1fd0 2020-11-28T21:48:28.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.255+0000 7fd4fad9d700 1 --2- 172.21.15.162:0/2921294848 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd4fc0a9bc0 0x7fd4fc140480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:48:28.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.255+0000 7fd4fa59c700 1 --2- 172.21.15.162:0/2921294848 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd4fc0b0060 0x7fd4fc0aac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:48:28.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.255+0000 7fd4fb59e700 1 --2- 172.21.15.162:0/2921294848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4fc0b1fd0 0x7fd4fc141900 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7fd4f40032b0 tx=0x7fd4f400a410).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:28.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.259+0000 7fd4ebfff700 1 -- 172.21.15.162:0/2921294848 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4f401a880 con 0x7fd4fc0b1fd0 2020-11-28T21:48:28.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.259+0000 7fd4ebfff700 1 -- 172.21.15.162:0/2921294848 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f401a9e0 con 0x7fd4fc0b1fd0 2020-11-28T21:48:28.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.259+0000 7fd4ebfff700 1 -- 172.21.15.162:0/2921294848 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4f401ad40 con 0x7fd4fc0b1fd0 2020-11-28T21:48:28.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.259+0000 7fd5008cb700 1 -- 172.21.15.162:0/2921294848 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4fc154de0 con 0x7fd4fc0b1fd0 2020-11-28T21:48:28.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.259+0000 7fd5008cb700 1 -- 172.21.15.162:0/2921294848 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4fc13f9e0 con 0x7fd4fc0b1fd0 2020-11-28T21:48:28.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.259+0000 7fd5008cb700 1 -- 172.21.15.162:0/2921294848 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4fc057050 con 0x7fd4fc0b1fd0 2020-11-28T21:48:28.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.259+0000 7fd4ebfff700 1 -- 172.21.15.162:0/2921294848 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd4f402a070 con 0x7fd4fc0b1fd0 2020-11-28T21:48:28.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.259+0000 7fd4ebfff700 1 --2- 172.21.15.162:0/2921294848 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd4d8055350 0x7fd4d8057810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:28.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.263+0000 7fd4ebfff700 1 -- 172.21.15.162:0/2921294848 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd4f406fad0 con 0x7fd4fc0b1fd0 2020-11-28T21:48:28.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.263+0000 7fd4fad9d700 1 --2- 172.21.15.162:0/2921294848 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd4d8055350 0x7fd4d8057810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:28.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.263+0000 7fd4fad9d700 1 --2- 172.21.15.162:0/2921294848 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd4d8055350 0x7fd4d8057810 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7fd4ec00d000 tx=0x7fd4ec009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:28.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.267+0000 7fd4ebfff700 1 -- 172.21.15.162:0/2921294848 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd4f4045dd0 con 0x7fd4fc0b1fd0 2020-11-28T21:48:28.410 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.407+0000 7fd5008cb700 1 -- 172.21.15.162:0/2921294848 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4fc149660 con 0x7fd4fc0b1fd0 2020-11-28T21:48:28.411 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.411+0000 7fd4ebfff700 1 -- 172.21.15.162:0/2921294848 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16134 (secure 0 0 0) 0x7fd4f4014070 con 0x7fd4fc0b1fd0 2020-11-28T21:48:28.412 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:48:28.412 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":44,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:48:27.785910+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:48:25.281355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2020-11-28T21:48:25.782088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:48:28.414 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.411+0000 7fd5008cb700 1 -- 172.21.15.162:0/2921294848 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd4d8055350 msgr2=0x7fd4d8057810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:28.418 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.411+0000 7fd5008cb700 1 --2- 172.21.15.162:0/2921294848 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd4d8055350 0x7fd4d8057810 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7fd4ec00d000 tx=0x7fd4ec009040).stop 2020-11-28T21:48:28.419 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.411+0000 7fd5008cb700 1 -- 172.21.15.162:0/2921294848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4fc0b1fd0 msgr2=0x7fd4fc141900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:28.419 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.411+0000 7fd5008cb700 1 --2- 172.21.15.162:0/2921294848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4fc0b1fd0 0x7fd4fc141900 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7fd4f40032b0 tx=0x7fd4f400a410).stop 2020-11-28T21:48:28.419 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.411+0000 7fd5008cb700 1 -- 172.21.15.162:0/2921294848 shutdown_connections 2020-11-28T21:48:28.419 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.411+0000 7fd5008cb700 1 --2- 172.21.15.162:0/2921294848 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd4d8055350 0x7fd4d8057810 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:28.420 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.411+0000 7fd5008cb700 1 --2- 172.21.15.162:0/2921294848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4fc0b1fd0 0x7fd4fc141900 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:28.420 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.411+0000 7fd5008cb700 1 --2- 172.21.15.162:0/2921294848 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd4fc0a9bc0 0x7fd4fc140480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:28.420 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.411+0000 7fd5008cb700 1 --2- 172.21.15.162:0/2921294848 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd4fc0b0060 0x7fd4fc0aac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:28.420 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.411+0000 7fd5008cb700 1 -- 172.21.15.162:0/2921294848 >> 172.21.15.162:0/2921294848 conn(0x7fd4fc05b970 msgr2=0x7fd4fc142d40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:28.421 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.411+0000 7fd5008cb700 1 -- 172.21.15.162:0/2921294848 shutdown_connections 2020-11-28T21:48:28.421 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.411+0000 7fd5008cb700 1 -- 172.21.15.162:0/2921294848 wait complete. 2020-11-28T21:48:28.425 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-11-28T21:48:28.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.547+0000 7fe16e195700 1 Processor -- start 2020-11-28T21:48:28.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.547+0000 7fe16e195700 1 -- start start 2020-11-28T21:48:28.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.547+0000 7fe16e195700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe1680b2e80 0x7fe16805f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:28.555 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.547+0000 7fe16e195700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe1680b2560 0x7fe16805f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:28.555 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.547+0000 7fe16e195700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1680ab0b0 con 0x7fe1680acb40 2020-11-28T21:48:28.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.547+0000 7fe16e195700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe16805fd60 con 0x7fe1680b2560 2020-11-28T21:48:28.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.547+0000 7fe16e195700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1680b3260 con 0x7fe1680b2e80 2020-11-28T21:48:28.557 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.547+0000 7fe16f3f7700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fe1680acb40 0x7fe168060d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:46160/0 (socket says 172.21.15.162:46160) 2020-11-28T21:48:28.557 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.547+0000 7fe16f3f7700 1 -- 172.21.15.162:0/981247540 learned_addr learned my addr 172.21.15.162:0/981247540 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:48:28.557 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.547+0000 7fe16d193700 1 --2- 172.21.15.162:0/981247540 >> v2:172.21.15.118:3300/0 conn(0x7fe1680b2560 0x7fe16805f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:28.558 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.547+0000 7fe16d994700 1 --2- 172.21.15.162:0/981247540 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe1680b2e80 0x7fe16805f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:28.558 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.547+0000 7fe16c992700 1 -- 172.21.15.162:0/981247540 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1652923402 0 0) 0x7fe1680ab0b0 con 0x7fe1680acb40 2020-11-28T21:48:28.559 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.547+0000 7fe16c992700 1 -- 172.21.15.162:0/981247540 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe154003230 con 0x7fe1680acb40 2020-11-28T21:48:28.559 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.547+0000 7fe16d193700 1 -- 172.21.15.162:0/981247540 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe1680b2e80 msgr2=0x7fe16805f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:28.559 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.547+0000 7fe16d193700 1 --2- 172.21.15.162:0/981247540 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe1680b2e80 0x7fe16805f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:28.560 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.547+0000 7fe16d193700 1 -- 172.21.15.162:0/981247540 >> v1:172.21.15.162:6789/0 conn(0x7fe1680acb40 legacy=0x7fe168060d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:28.560 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.547+0000 7fe16d193700 1 -- 172.21.15.162:0/981247540 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1680ab2e0 con 0x7fe1680b2560 2020-11-28T21:48:28.560 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.547+0000 7fe16d994700 1 --2- 172.21.15.162:0/981247540 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe1680b2e80 0x7fe16805f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:48:28.561 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.547+0000 7fe16d193700 1 --2- 172.21.15.162:0/981247540 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe1680b2560 0x7fe16805f870 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fe158002f30 tx=0x7fe158006dc0).ready entity=mon.0 client_cookie=862ca766a2942c4d server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:28.561 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.551+0000 7fe16c992700 1 -- 172.21.15.162:0/981247540 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1580096a0 con 0x7fe1680b2560 2020-11-28T21:48:28.562 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.551+0000 7fe16c992700 1 -- 172.21.15.162:0/981247540 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe158009800 con 0x7fe1680b2560 2020-11-28T21:48:28.562 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.551+0000 7fe16c992700 1 -- 172.21.15.162:0/981247540 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe158009b60 con 0x7fe1680b2560 2020-11-28T21:48:28.562 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.551+0000 7fe16e195700 1 -- 172.21.15.162:0/981247540 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe1680b2560 msgr2=0x7fe16805f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:28.563 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.551+0000 7fe16e195700 1 --2- 172.21.15.162:0/981247540 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe1680b2560 0x7fe16805f870 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fe158002f30 tx=0x7fe158006dc0).stop 2020-11-28T21:48:28.563 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.551+0000 7fe16e195700 1 -- 172.21.15.162:0/981247540 shutdown_connections 2020-11-28T21:48:28.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.551+0000 7fe16e195700 1 --2- 172.21.15.162:0/981247540 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe1680b2e80 0x7fe16805f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:28.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.551+0000 7fe16e195700 1 --2- 172.21.15.162:0/981247540 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe1680b2560 0x7fe16805f870 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:28.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.551+0000 7fe16e195700 1 -- 172.21.15.162:0/981247540 >> 172.21.15.162:0/981247540 conn(0x7fe168063b60 msgr2=0x7fe168063f40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:28.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.551+0000 7fe16e195700 1 -- 172.21.15.162:0/981247540 shutdown_connections 2020-11-28T21:48:28.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.551+0000 7fe16e195700 1 -- 172.21.15.162:0/981247540 wait complete. 2020-11-28T21:48:28.566 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.551+0000 7fe16e195700 1 Processor -- start 2020-11-28T21:48:28.566 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.551+0000 7fe16e195700 1 -- start start 2020-11-28T21:48:28.571 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.555+0000 7fe16e195700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1680acb40 0x7fe168133790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:28.571 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.555+0000 7fe16e195700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe1680b2560 0x7fe16813d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:28.572 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.555+0000 7fe16e195700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe1680b2e80 0x7fe16813d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:28.572 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.555+0000 7fe16e195700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16809a530 con 0x7fe1680b2560 2020-11-28T21:48:28.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.555+0000 7fe16e195700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16809a7d0 con 0x7fe1680acb40 2020-11-28T21:48:28.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.555+0000 7fe16e195700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe16809aa70 con 0x7fe1680b2e80 2020-11-28T21:48:28.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.555+0000 7fe16d994700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1680acb40 0x7fe168133790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:28.575 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.555+0000 7fe16d994700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1680acb40 0x7fe168133790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39318/0 (socket says 172.21.15.162:39318) 2020-11-28T21:48:28.575 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.555+0000 7fe16d994700 1 -- 172.21.15.162:0/2846589437 learned_addr learned my addr 172.21.15.162:0/2846589437 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:28.575 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.555+0000 7fe16f3f7700 1 --2- 172.21.15.162:0/2846589437 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe1680b2e80 0x7fe16813d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:28.576 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.555+0000 7fe16d193700 1 --2- 172.21.15.162:0/2846589437 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe1680b2560 0x7fe16813d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:28.576 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.555+0000 7fe16d994700 1 -- 172.21.15.162:0/2846589437 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe1680b2e80 msgr2=0x7fe16813d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:28.576 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.555+0000 7fe16d994700 1 --2- 172.21.15.162:0/2846589437 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe1680b2e80 0x7fe16813d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:28.576 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.555+0000 7fe16d994700 1 -- 172.21.15.162:0/2846589437 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe1680b2560 msgr2=0x7fe16813d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:28.576 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.555+0000 7fe16d994700 1 --2- 172.21.15.162:0/2846589437 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe1680b2560 0x7fe16813d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:28.577 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.555+0000 7fe16d994700 1 -- 172.21.15.162:0/2846589437 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe16809ad10 con 0x7fe1680acb40 2020-11-28T21:48:28.577 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.555+0000 7fe16d994700 1 --2- 172.21.15.162:0/2846589437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1680acb40 0x7fe168133790 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7fe16400c000 tx=0x7fe1640091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:28.577 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.555+0000 7fe15e7fc700 1 -- 172.21.15.162:0/2846589437 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe16400cbd0 con 0x7fe1680acb40 2020-11-28T21:48:28.577 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.555+0000 7fe15e7fc700 1 -- 172.21.15.162:0/2846589437 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe16400cd30 con 0x7fe1680acb40 2020-11-28T21:48:28.577 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.555+0000 7fe16e195700 1 -- 172.21.15.162:0/2846589437 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe16813dbd0 con 0x7fe1680acb40 2020-11-28T21:48:28.578 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.555+0000 7fe15e7fc700 1 -- 172.21.15.162:0/2846589437 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe16400cea0 con 0x7fe1680acb40 2020-11-28T21:48:28.578 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.555+0000 7fe16e195700 1 -- 172.21.15.162:0/2846589437 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1680b02f0 con 0x7fe1680acb40 2020-11-28T21:48:28.578 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.559+0000 7fe16e195700 1 -- 172.21.15.162:0/2846589437 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe16809aea0 con 0x7fe1680acb40 2020-11-28T21:48:28.578 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.559+0000 7fe15e7fc700 1 -- 172.21.15.162:0/2846589437 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe164022070 con 0x7fe1680acb40 2020-11-28T21:48:28.579 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.559+0000 7fe15e7fc700 1 --2- 172.21.15.162:0/2846589437 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe1500554d0 0x7fe150057990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:28.579 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.559+0000 7fe15e7fc700 1 -- 172.21.15.162:0/2846589437 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe16406e6d0 con 0x7fe1680acb40 2020-11-28T21:48:28.579 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.559+0000 7fe16d193700 1 --2- 172.21.15.162:0/2846589437 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe1500554d0 0x7fe150057990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:28.579 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.559+0000 7fe16d193700 1 --2- 172.21.15.162:0/2846589437 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe1500554d0 0x7fe150057990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7fe15801b000 tx=0x7fe158015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:28.579 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.567+0000 7fe15e7fc700 1 -- 172.21.15.162:0/2846589437 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe1640449d0 con 0x7fe1680acb40 2020-11-28T21:48:28.756 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.755+0000 7fe16e195700 1 -- 172.21.15.162:0/2846589437 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fe168057050 con 0x7fe1680acb40 2020-11-28T21:48:28.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.763+0000 7fe15e7fc700 1 -- 172.21.15.162:0/2846589437 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v13) v1 ==== 184+0+0 (secure 0 0 0) 0x7fe164047ff0 con 0x7fe1680acb40 2020-11-28T21:48:28.766 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.767+0000 7fe16e195700 1 -- 172.21.15.162:0/2846589437 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe1500554d0 msgr2=0x7fe150057990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:28.767 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.767+0000 7fe16e195700 1 --2- 172.21.15.162:0/2846589437 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe1500554d0 0x7fe150057990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7fe15801b000 tx=0x7fe158015000).stop 2020-11-28T21:48:28.767 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.767+0000 7fe16e195700 1 -- 172.21.15.162:0/2846589437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1680acb40 msgr2=0x7fe168133790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:28.767 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.767+0000 7fe16e195700 1 --2- 172.21.15.162:0/2846589437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1680acb40 0x7fe168133790 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7fe16400c000 tx=0x7fe1640091a0).stop 2020-11-28T21:48:28.767 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.767+0000 7fe16e195700 1 -- 172.21.15.162:0/2846589437 shutdown_connections 2020-11-28T21:48:28.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.767+0000 7fe16e195700 1 --2- 172.21.15.162:0/2846589437 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe1500554d0 0x7fe150057990 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:28.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.767+0000 7fe16e195700 1 --2- 172.21.15.162:0/2846589437 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe1680b2e80 0x7fe16813d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:28.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.767+0000 7fe16e195700 1 --2- 172.21.15.162:0/2846589437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1680acb40 0x7fe168133790 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:28.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.767+0000 7fe16e195700 1 --2- 172.21.15.162:0/2846589437 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe1680b2560 0x7fe16813d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:28.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.767+0000 7fe16e195700 1 -- 172.21.15.162:0/2846589437 >> 172.21.15.162:0/2846589437 conn(0x7fe168063b60 msgr2=0x7fe168063f40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:28.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.767+0000 7fe16e195700 1 -- 172.21.15.162:0/2846589437 shutdown_connections 2020-11-28T21:48:28.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.767+0000 7fe16e195700 1 -- 172.21.15.162:0/2846589437 wait complete. 2020-11-28T21:48:28.769 INFO:teuthology.orchestra.run.smithi162.stderr:updated caps for client.0 2020-11-28T21:48:28.779 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-11-28T21:48:28.907 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.903+0000 7f04230c9700 1 Processor -- start 2020-11-28T21:48:28.907 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.903+0000 7f04230c9700 1 -- start start 2020-11-28T21:48:28.907 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.903+0000 7f04230c9700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f041c0a86d0 0x7f041c0addd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:28.908 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.903+0000 7f04230c9700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f041c0b5980 con 0x7f041c080500 2020-11-28T21:48:28.908 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.903+0000 7f04230c9700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f041c0809a0 con 0x7f041c0b55a0 2020-11-28T21:48:28.908 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.903+0000 7f04230c9700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f041c0afe80 con 0x7f041c0a86d0 2020-11-28T21:48:28.908 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.903+0000 7f0421666700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f041c0b55a0 0x7f041c0ad6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:46174/0 (socket says 172.21.15.162:46174) 2020-11-28T21:48:28.908 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.903+0000 7f0421666700 1 -- 172.21.15.162:0/2605568398 learned_addr learned my addr 172.21.15.162:0/2605568398 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:48:28.909 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.903+0000 7f0421e67700 1 --2- 172.21.15.162:0/2605568398 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f041c0a86d0 0x7f041c0addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:28.909 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.903+0000 7f0413fff700 1 -- 172.21.15.162:0/2605568398 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2768089123 0 0) 0x7f041c0809a0 con 0x7f041c0b55a0 2020-11-28T21:48:28.909 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.903+0000 7f0413fff700 1 -- 172.21.15.162:0/2605568398 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0408003230 con 0x7f041c0b55a0 2020-11-28T21:48:28.909 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.903+0000 7f0413fff700 1 -- 172.21.15.162:0/2605568398 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 283801250 0 0) 0x7f041c0b5980 con 0x7f041c080500 2020-11-28T21:48:28.909 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.903+0000 7f0413fff700 1 -- 172.21.15.162:0/2605568398 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f041c0809a0 con 0x7f041c080500 2020-11-28T21:48:28.910 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.903+0000 7f0421e67700 1 -- 172.21.15.162:0/2605568398 >> v1:172.21.15.162:6789/0 conn(0x7f041c0b55a0 legacy=0x7f041c0ad6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:28.910 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.903+0000 7f0421e67700 1 -- 172.21.15.162:0/2605568398 >> v1:172.21.15.118:6789/0 conn(0x7f041c080500 legacy=0x7f041c153080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:28.910 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.903+0000 7f0421e67700 1 -- 172.21.15.162:0/2605568398 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f041c0af740 con 0x7f041c0a86d0 2020-11-28T21:48:28.910 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.903+0000 7f0421e67700 1 --2- 172.21.15.162:0/2605568398 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f041c0a86d0 0x7f041c0addd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f04140032b0 tx=0x7f04140091a0).ready entity=mon.2 client_cookie=1ab286551258b09b server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:28.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.903+0000 7f0413fff700 1 -- 172.21.15.162:0/2605568398 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f041400b940 con 0x7f041c0a86d0 2020-11-28T21:48:28.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.903+0000 7f0413fff700 1 -- 172.21.15.162:0/2605568398 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f041400baa0 con 0x7f041c0a86d0 2020-11-28T21:48:28.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.903+0000 7f04230c9700 1 -- 172.21.15.162:0/2605568398 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f041c0a86d0 msgr2=0x7f041c0addd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:28.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.903+0000 7f0413fff700 1 -- 172.21.15.162:0/2605568398 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f041400bc10 con 0x7f041c0a86d0 2020-11-28T21:48:28.912 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.903+0000 7f04230c9700 1 --2- 172.21.15.162:0/2605568398 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f041c0a86d0 0x7f041c0addd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f04140032b0 tx=0x7f04140091a0).stop 2020-11-28T21:48:28.912 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.907+0000 7f04230c9700 1 -- 172.21.15.162:0/2605568398 shutdown_connections 2020-11-28T21:48:28.912 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.907+0000 7f04230c9700 1 --2- 172.21.15.162:0/2605568398 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f041c0a86d0 0x7f041c0addd0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:28.912 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.907+0000 7f04230c9700 1 -- 172.21.15.162:0/2605568398 >> 172.21.15.162:0/2605568398 conn(0x7f041c0626d0 msgr2=0x7f041c062ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:28.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.907+0000 7f04230c9700 1 -- 172.21.15.162:0/2605568398 shutdown_connections 2020-11-28T21:48:28.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.907+0000 7f04230c9700 1 -- 172.21.15.162:0/2605568398 wait complete. 2020-11-28T21:48:28.918 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.907+0000 7f04230c9700 1 Processor -- start 2020-11-28T21:48:28.918 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.907+0000 7f04230c9700 1 -- start start 2020-11-28T21:48:28.918 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.907+0000 7f04230c9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f041c080500 0x7f041c0b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:28.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.907+0000 7f04230c9700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f041c0a86d0 0x7f041c15ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:28.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.907+0000 7f04230c9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f041c0b55a0 0x7f041c137970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:28.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.907+0000 7f04230c9700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c15b040 con 0x7f041c0b55a0 2020-11-28T21:48:28.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.907+0000 7f04230c9700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c1586d0 con 0x7f041c080500 2020-11-28T21:48:28.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.907+0000 7f04230c9700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f041c157cf0 con 0x7f041c0a86d0 2020-11-28T21:48:28.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.907+0000 7f0421666700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f041c080500 0x7f041c0b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:28.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.907+0000 7f0421666700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f041c080500 0x7f041c0b3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39332/0 (socket says 172.21.15.162:39332) 2020-11-28T21:48:28.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.907+0000 7f0421666700 1 -- 172.21.15.162:0/3875612503 learned_addr learned my addr 172.21.15.162:0/3875612503 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:28.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.907+0000 7f0420e65700 1 --2- 172.21.15.162:0/3875612503 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f041c0a86d0 0x7f041c15ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:28.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.911+0000 7f0421e67700 1 --2- 172.21.15.162:0/3875612503 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f041c0b55a0 0x7f041c137970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:28.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.911+0000 7f0421666700 1 -- 172.21.15.162:0/3875612503 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f041c0a86d0 msgr2=0x7f041c15ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:28.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.911+0000 7f0421666700 1 --2- 172.21.15.162:0/3875612503 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f041c0a86d0 0x7f041c15ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:28.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.911+0000 7f0421666700 1 -- 172.21.15.162:0/3875612503 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f041c0b55a0 msgr2=0x7f041c137970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:28.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.911+0000 7f0421666700 1 --2- 172.21.15.162:0/3875612503 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f041c0b55a0 0x7f041c137970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:28.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.911+0000 7f0421666700 1 -- 172.21.15.162:0/3875612503 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f041c157f90 con 0x7f041c080500 2020-11-28T21:48:28.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.911+0000 7f0421666700 1 --2- 172.21.15.162:0/3875612503 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f041c080500 0x7f041c0b3dc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f041c15b180 tx=0x7f0418007b20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:28.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.911+0000 7f0411ffb700 1 -- 172.21.15.162:0/3875612503 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04180072e0 con 0x7f041c080500 2020-11-28T21:48:28.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.911+0000 7f0411ffb700 1 -- 172.21.15.162:0/3875612503 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0418007440 con 0x7f041c080500 2020-11-28T21:48:28.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.911+0000 7f04230c9700 1 -- 172.21.15.162:0/3875612503 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f041c158230 con 0x7f041c080500 2020-11-28T21:48:28.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.911+0000 7f0411ffb700 1 -- 172.21.15.162:0/3875612503 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04180057d0 con 0x7f041c080500 2020-11-28T21:48:28.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.911+0000 7f04230c9700 1 -- 172.21.15.162:0/3875612503 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f041c1554d0 con 0x7f041c080500 2020-11-28T21:48:28.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.911+0000 7f04230c9700 1 -- 172.21.15.162:0/3875612503 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f041c057050 con 0x7f041c080500 2020-11-28T21:48:28.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.911+0000 7f0411ffb700 1 -- 172.21.15.162:0/3875612503 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f041801a060 con 0x7f041c080500 2020-11-28T21:48:28.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.911+0000 7f0411ffb700 1 --2- 172.21.15.162:0/3875612503 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f03fc0553b0 0x7f03fc057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:28.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.911+0000 7f0411ffb700 1 -- 172.21.15.162:0/3875612503 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f041806d630 con 0x7f041c080500 2020-11-28T21:48:28.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.911+0000 7f0420e65700 1 --2- 172.21.15.162:0/3875612503 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f03fc0553b0 0x7f03fc057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:28.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.915+0000 7f0420e65700 1 --2- 172.21.15.162:0/3875612503 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f03fc0553b0 0x7f03fc057870 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f040c00d000 tx=0x7f040c0079d0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:28.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:28.919+0000 7f0411ffb700 1 -- 172.21.15.162:0/3875612503 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0418043930 con 0x7f041c080500 2020-11-28T21:48:29.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.107+0000 7f04230c9700 1 -- 172.21.15.162:0/3875612503 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f041c1583c0 con 0x7f041c080500 2020-11-28T21:48:29.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.115+0000 7f0411ffb700 1 -- 172.21.15.162:0/3875612503 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v14) v1 ==== 184+0+0 (secure 0 0 0) 0x7f0418018070 con 0x7f041c080500 2020-11-28T21:48:29.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.119+0000 7f04230c9700 1 -- 172.21.15.162:0/3875612503 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f03fc0553b0 msgr2=0x7f03fc057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:29.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.119+0000 7f04230c9700 1 --2- 172.21.15.162:0/3875612503 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f03fc0553b0 0x7f03fc057870 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f040c00d000 tx=0x7f040c0079d0).stop 2020-11-28T21:48:29.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.119+0000 7f04230c9700 1 -- 172.21.15.162:0/3875612503 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f041c080500 msgr2=0x7f041c0b3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:29.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.119+0000 7f04230c9700 1 --2- 172.21.15.162:0/3875612503 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f041c080500 0x7f041c0b3dc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f041c15b180 tx=0x7f0418007b20).stop 2020-11-28T21:48:29.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.119+0000 7f04230c9700 1 -- 172.21.15.162:0/3875612503 shutdown_connections 2020-11-28T21:48:29.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.119+0000 7f04230c9700 1 --2- 172.21.15.162:0/3875612503 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f03fc0553b0 0x7f03fc057870 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:29.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.119+0000 7f04230c9700 1 --2- 172.21.15.162:0/3875612503 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f041c0b55a0 0x7f041c137970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:29.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.119+0000 7f04230c9700 1 --2- 172.21.15.162:0/3875612503 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f041c080500 0x7f041c0b3dc0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:29.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.119+0000 7f04230c9700 1 --2- 172.21.15.162:0/3875612503 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f041c0a86d0 0x7f041c15ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:29.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.119+0000 7f04230c9700 1 -- 172.21.15.162:0/3875612503 >> 172.21.15.162:0/3875612503 conn(0x7f041c0626d0 msgr2=0x7f041c0ad680 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:29.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.119+0000 7f04230c9700 1 -- 172.21.15.162:0/3875612503 shutdown_connections 2020-11-28T21:48:29.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.119+0000 7f04230c9700 1 -- 172.21.15.162:0/3875612503 wait complete. 2020-11-28T21:48:29.123 INFO:teuthology.orchestra.run.smithi162.stderr:updated caps for client.1 2020-11-28T21:48:29.134 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:48:29.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.243+0000 7f40c7fff700 1 Processor -- start 2020-11-28T21:48:29.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.243+0000 7f40c7fff700 1 -- start start 2020-11-28T21:48:29.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.243+0000 7f40c7fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f40c80b3a50 0x7f40c8062cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:29.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.243+0000 7f40c7fff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f40c80664a0 0x7f40c8066910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:29.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.243+0000 7f40c7fff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40c80b3230 con 0x7f40c81406c0 2020-11-28T21:48:29.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.243+0000 7f40c7fff700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40c8144bf0 con 0x7f40c80664a0 2020-11-28T21:48:29.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.243+0000 7f40c7fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c8143da0 con 0x7f40c80b3a50 2020-11-28T21:48:29.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.243+0000 7f40c6ffd700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f40c80b3a50 0x7f40c8062cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:29.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.243+0000 7f40c77fe700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f40c80664a0 0x7f40c8066910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:29.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.243+0000 7f40c6ffd700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f40c80b3a50 0x7f40c8062cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:36566/0 (socket says 172.21.15.162:36566) 2020-11-28T21:48:29.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.243+0000 7f40bffff700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f40c81406c0 0x7f40c81444e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.162:36332/0 (socket says 172.21.15.162:36332) 2020-11-28T21:48:29.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.243+0000 7f40c6ffd700 1 -- 172.21.15.162:0/3479961376 learned_addr learned my addr 172.21.15.162:0/3479961376 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:29.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.247+0000 7f40c77fe700 1 -- 172.21.15.162:0/3479961376 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f40c80b3a50 msgr2=0x7f40c8062cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:29.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.247+0000 7f40c77fe700 1 --2- 172.21.15.162:0/3479961376 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f40c80b3a50 0x7f40c8062cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:29.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.247+0000 7f40c77fe700 1 -- 172.21.15.162:0/3479961376 >> v1:172.21.15.118:6789/0 conn(0x7f40c81406c0 legacy=0x7f40c81444e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:29.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.247+0000 7f40c67fc700 1 -- 172.21.15.162:0/3479961376 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2244304315 0 0) 0x7f40c80b3230 con 0x7f40c81406c0 2020-11-28T21:48:29.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.247+0000 7f40c77fe700 1 -- 172.21.15.162:0/3479961376 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c8144040 con 0x7f40c80664a0 2020-11-28T21:48:29.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.247+0000 7f40c77fe700 1 --2- 172.21.15.162:0/3479961376 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f40c80664a0 0x7f40c8066910 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f40c00032b0 tx=0x7f40c00091a0).ready entity=mon.0 client_cookie=2ca758708aa1d31f server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:29.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.247+0000 7f40c67fc700 1 -- 172.21.15.162:0/3479961376 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40c000b9a0 con 0x7f40c80664a0 2020-11-28T21:48:29.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.247+0000 7f40c67fc700 1 -- 172.21.15.162:0/3479961376 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c000bb00 con 0x7f40c80664a0 2020-11-28T21:48:29.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.247+0000 7f40c67fc700 1 -- 172.21.15.162:0/3479961376 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40c000bc70 con 0x7f40c80664a0 2020-11-28T21:48:29.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.247+0000 7f40c7fff700 1 -- 172.21.15.162:0/3479961376 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f40c80664a0 msgr2=0x7f40c8066910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:29.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.247+0000 7f40c7fff700 1 --2- 172.21.15.162:0/3479961376 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f40c80664a0 0x7f40c8066910 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f40c00032b0 tx=0x7f40c00091a0).stop 2020-11-28T21:48:29.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.247+0000 7f40c7fff700 1 -- 172.21.15.162:0/3479961376 shutdown_connections 2020-11-28T21:48:29.253 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.247+0000 7f40c7fff700 1 --2- 172.21.15.162:0/3479961376 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f40c80664a0 0x7f40c8066910 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:29.253 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.247+0000 7f40c7fff700 1 --2- 172.21.15.162:0/3479961376 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f40c80b3a50 0x7f40c8062cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:29.253 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.247+0000 7f40c7fff700 1 -- 172.21.15.162:0/3479961376 >> 172.21.15.162:0/3479961376 conn(0x7f40c805b970 msgr2=0x7f40c805bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:29.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.247+0000 7f40c7fff700 1 -- 172.21.15.162:0/3479961376 shutdown_connections 2020-11-28T21:48:29.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.247+0000 7f40c7fff700 1 -- 172.21.15.162:0/3479961376 wait complete. 2020-11-28T21:48:29.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.247+0000 7f40c7fff700 1 Processor -- start 2020-11-28T21:48:29.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.247+0000 7f40c7fff700 1 -- start start 2020-11-28T21:48:29.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.251+0000 7f40c7fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40c80664a0 0x7f40c805c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:29.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.251+0000 7f40c7fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f40c80b3a50 0x7f40c80ac080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:29.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.251+0000 7f40c7fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f40c81406c0 0x7f40c814f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:29.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.251+0000 7f40c7fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c8153a90 con 0x7f40c81406c0 2020-11-28T21:48:29.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.251+0000 7f40c7fff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c8153560 con 0x7f40c80664a0 2020-11-28T21:48:29.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.251+0000 7f40c7fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c81537f0 con 0x7f40c80b3a50 2020-11-28T21:48:29.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.251+0000 7f40c6ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40c80664a0 0x7f40c805c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:29.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.251+0000 7f40c6ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40c80664a0 0x7f40c805c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39348/0 (socket says 172.21.15.162:39348) 2020-11-28T21:48:29.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.251+0000 7f40c77fe700 1 --2- 172.21.15.162:0/3250143063 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f40c81406c0 0x7f40c814f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:29.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.251+0000 7f40c6ffd700 1 -- 172.21.15.162:0/3250143063 learned_addr learned my addr 172.21.15.162:0/3250143063 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:29.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.251+0000 7f40bffff700 1 --2- 172.21.15.162:0/3250143063 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f40c80b3a50 0x7f40c80ac080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:29.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.251+0000 7f40c6ffd700 1 -- 172.21.15.162:0/3250143063 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f40c80b3a50 msgr2=0x7f40c80ac080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:29.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.251+0000 7f40c6ffd700 1 --2- 172.21.15.162:0/3250143063 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f40c80b3a50 0x7f40c80ac080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:29.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.251+0000 7f40c6ffd700 1 -- 172.21.15.162:0/3250143063 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f40c81406c0 msgr2=0x7f40c814f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:29.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.251+0000 7f40c6ffd700 1 --2- 172.21.15.162:0/3250143063 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f40c81406c0 0x7f40c814f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:29.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.251+0000 7f40c6ffd700 1 -- 172.21.15.162:0/3250143063 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c80abbe0 con 0x7f40c80664a0 2020-11-28T21:48:29.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.251+0000 7f40c6ffd700 1 --2- 172.21.15.162:0/3250143063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40c80664a0 0x7f40c805c090 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f40b000c000 tx=0x7f40b0008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:29.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.251+0000 7f40c4ff9700 1 -- 172.21.15.162:0/3250143063 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40b000c820 con 0x7f40c80664a0 2020-11-28T21:48:29.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.251+0000 7f40c4ff9700 1 -- 172.21.15.162:0/3250143063 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b000c980 con 0x7f40c80664a0 2020-11-28T21:48:29.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.251+0000 7f40c7fff700 1 -- 172.21.15.162:0/3250143063 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40c80ac570 con 0x7f40c80664a0 2020-11-28T21:48:29.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.251+0000 7f40c4ff9700 1 -- 172.21.15.162:0/3250143063 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40b000cce0 con 0x7f40c80664a0 2020-11-28T21:48:29.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.251+0000 7f40c7fff700 1 -- 172.21.15.162:0/3250143063 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c80aa6b0 con 0x7f40c80664a0 2020-11-28T21:48:29.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.251+0000 7f40c7fff700 1 -- 172.21.15.162:0/3250143063 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40c8057050 con 0x7f40c80664a0 2020-11-28T21:48:29.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.255+0000 7f40c4ff9700 1 -- 172.21.15.162:0/3250143063 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f40b0022070 con 0x7f40c80664a0 2020-11-28T21:48:29.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.255+0000 7f40c4ff9700 1 --2- 172.21.15.162:0/3250143063 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f40ac0554e0 0x7f40ac0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:29.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.255+0000 7f40c4ff9700 1 -- 172.21.15.162:0/3250143063 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f40b006e320 con 0x7f40c80664a0 2020-11-28T21:48:29.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.255+0000 7f40bffff700 1 --2- 172.21.15.162:0/3250143063 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f40ac0554e0 0x7f40ac0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:29.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.255+0000 7f40bffff700 1 --2- 172.21.15.162:0/3250143063 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f40ac0554e0 0x7f40ac0579a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f40b800b000 tx=0x7f40b8007160).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:29.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.263+0000 7f40c4ff9700 1 -- 172.21.15.162:0/3250143063 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f40b0044620 con 0x7f40c80664a0 2020-11-28T21:48:29.448 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.447+0000 7f40c7fff700 1 -- 172.21.15.162:0/3250143063 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40c8067280 con 0x7f40c80664a0 2020-11-28T21:48:29.451 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.451+0000 7f40c4ff9700 1 -- 172.21.15.162:0/3250143063 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+2497 (secure 0 0 0) 0x7f40b0047c40 con 0x7f40c80664a0 2020-11-28T21:48:29.451 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:48:29.451 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":15,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4856,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/913296056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":913296056},{"type":"v1","addr":"172.21.15.118:6833","nonce":913296056}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":13},{"gid":5209,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6835/1318110356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1318110356},{"type":"v1","addr":"172.21.15.162:6835","nonce":1318110356}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:48:26.782354+0000","modified":"2020-11-28T21:48:27.785277+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5212},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5212":{"gid":5212,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/2706248574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2706248574},{"type":"v1","addr":"172.21.15.162:6837","nonce":2706248574}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-28T21:48:29.454 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.451+0000 7f40c7fff700 1 -- 172.21.15.162:0/3250143063 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f40ac0554e0 msgr2=0x7f40ac0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:29.454 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.451+0000 7f40c7fff700 1 --2- 172.21.15.162:0/3250143063 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f40ac0554e0 0x7f40ac0579a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f40b800b000 tx=0x7f40b8007160).stop 2020-11-28T21:48:29.454 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.451+0000 7f40c7fff700 1 -- 172.21.15.162:0/3250143063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40c80664a0 msgr2=0x7f40c805c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:29.454 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.451+0000 7f40c7fff700 1 --2- 172.21.15.162:0/3250143063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40c80664a0 0x7f40c805c090 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f40b000c000 tx=0x7f40b0008040).stop 2020-11-28T21:48:29.454 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.451+0000 7f40c7fff700 1 -- 172.21.15.162:0/3250143063 shutdown_connections 2020-11-28T21:48:29.455 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.451+0000 7f40c7fff700 1 --2- 172.21.15.162:0/3250143063 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f40ac0554e0 0x7f40ac0579a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:29.455 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.451+0000 7f40c7fff700 1 --2- 172.21.15.162:0/3250143063 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f40c81406c0 0x7f40c814f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:29.455 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.451+0000 7f40c7fff700 1 --2- 172.21.15.162:0/3250143063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40c80664a0 0x7f40c805c090 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:29.455 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.451+0000 7f40c7fff700 1 --2- 172.21.15.162:0/3250143063 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f40c80b3a50 0x7f40c80ac080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:29.456 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.451+0000 7f40c7fff700 1 -- 172.21.15.162:0/3250143063 >> 172.21.15.162:0/3250143063 conn(0x7f40c805b970 msgr2=0x7f40c813c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:29.456 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.451+0000 7f40c7fff700 1 -- 172.21.15.162:0/3250143063 shutdown_connections 2020-11-28T21:48:29.456 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:29.451+0000 7f40c7fff700 1 -- 172.21.15.162:0/3250143063 wait complete. 2020-11-28T21:48:29.457 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 15 2020-11-28T21:48:29.769 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-28T21:48:26.782354+0000', 'modified': '2020-11-28T21:48:27.785277+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5212}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5212': {'gid': 5212, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6837/2706248574', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6836', 'nonce': 2706248574}, {'type': 'v1', 'addr': '172.21.15.162:6837', 'nonce': 2706248574}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-28T21:48:29.770 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-28T21:48:29.770 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-28T21:48:29.900 INFO:teuthology.orchestra.run.smithi162.stdout:{"cluster_fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","whoami":0,"id":5212,"want_state":"up:active","state":"up:active","rank_uptime":3.1044412339999998,"mdsmap_epoch":15,"osdmap_epoch":44,"osdmap_epoch_barrier":43,"uptime":7.4962792970000001} 2020-11-28T21:48:29.901 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", "id": 5212, "mdsmap_epoch": 15, "osdmap_epoch": 44, "osdmap_epoch_barrier": 43, "rank_uptime": 3.104441234, "state": "up:active", "uptime": 7.496279297, "want_state": "up:active", "whoami": 0 } 2020-11-28T21:48:29.901 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:48:30.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.015+0000 7f5446d00700 1 Processor -- start 2020-11-28T21:48:30.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.015+0000 7f5446d00700 1 -- start start 2020-11-28T21:48:30.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.015+0000 7f5446d00700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5440143530 0x7f54400b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:30.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.015+0000 7f5446d00700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54400b0e50 con 0x7f544013c630 2020-11-28T21:48:30.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.015+0000 7f5446d00700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f544005c090 con 0x7f544005bcb0 2020-11-28T21:48:30.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.015+0000 7f5446d00700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f544005fe40 con 0x7f5440143530 2020-11-28T21:48:30.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.015+0000 7f5445cfe700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f544005bcb0 0x7f54400b0740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:46198/0 (socket says 172.21.15.162:46198) 2020-11-28T21:48:30.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.015+0000 7f5445cfe700 1 -- 172.21.15.162:0/554320322 learned_addr learned my addr 172.21.15.162:0/554320322 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:48:30.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.015+0000 7f54454fd700 1 --2- 172.21.15.162:0/554320322 >> v2:172.21.15.162:3300/0 conn(0x7f5440143530 0x7f54400b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:30.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.015+0000 7f5444cfc700 1 -- 172.21.15.162:0/554320322 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1402153617 0 0) 0x7f544005c090 con 0x7f544005bcb0 2020-11-28T21:48:30.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.015+0000 7f5444cfc700 1 -- 172.21.15.162:0/554320322 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5428003230 con 0x7f544005bcb0 2020-11-28T21:48:30.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.015+0000 7f54454fd700 1 -- 172.21.15.162:0/554320322 >> v1:172.21.15.162:6789/0 conn(0x7f544005bcb0 legacy=0x7f54400b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:30.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.015+0000 7f54454fd700 1 -- 172.21.15.162:0/554320322 >> v1:172.21.15.118:6789/0 conn(0x7f544013c630 legacy=0x7f544014d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:30.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.015+0000 7f54454fd700 1 -- 172.21.15.162:0/554320322 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54400b22a0 con 0x7f5440143530 2020-11-28T21:48:30.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.015+0000 7f54454fd700 1 --2- 172.21.15.162:0/554320322 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5440143530 0x7f54400b1c00 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f543c002f30 tx=0x7f543c006dc0).ready entity=mon.1 client_cookie=bd8a05086a7b9d91 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:30.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.015+0000 7f5444cfc700 1 -- 172.21.15.162:0/554320322 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f543c009800 con 0x7f5440143530 2020-11-28T21:48:30.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.019+0000 7f5444cfc700 1 -- 172.21.15.162:0/554320322 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f543c009960 con 0x7f5440143530 2020-11-28T21:48:30.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.019+0000 7f5444cfc700 1 -- 172.21.15.162:0/554320322 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f543c009cc0 con 0x7f5440143530 2020-11-28T21:48:30.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.019+0000 7f5446d00700 1 -- 172.21.15.162:0/554320322 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5440143530 msgr2=0x7f54400b1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:30.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.019+0000 7f5446d00700 1 --2- 172.21.15.162:0/554320322 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5440143530 0x7f54400b1c00 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f543c002f30 tx=0x7f543c006dc0).stop 2020-11-28T21:48:30.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.019+0000 7f5446d00700 1 -- 172.21.15.162:0/554320322 shutdown_connections 2020-11-28T21:48:30.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.019+0000 7f5446d00700 1 --2- 172.21.15.162:0/554320322 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5440143530 0x7f54400b1c00 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:30.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.019+0000 7f5446d00700 1 -- 172.21.15.162:0/554320322 >> 172.21.15.162:0/554320322 conn(0x7f54400b5da0 msgr2=0x7f54400b6180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:30.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.019+0000 7f5446d00700 1 -- 172.21.15.162:0/554320322 shutdown_connections 2020-11-28T21:48:30.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.019+0000 7f5446d00700 1 -- 172.21.15.162:0/554320322 wait complete. 2020-11-28T21:48:30.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.019+0000 7f5446d00700 1 Processor -- start 2020-11-28T21:48:30.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.019+0000 7f5446d00700 1 -- start start 2020-11-28T21:48:30.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.019+0000 7f5446d00700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f544005bcb0 0x7f54401457a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:30.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.019+0000 7f5446d00700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f544013c630 0x7f544013f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:30.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.019+0000 7f5446d00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5440143530 0x7f544013fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:30.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.019+0000 7f5446d00700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5440146c10 con 0x7f544013c630 2020-11-28T21:48:30.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.019+0000 7f5446d00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f544013e2f0 con 0x7f5440143530 2020-11-28T21:48:30.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.019+0000 7f5446d00700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f544013e590 con 0x7f544005bcb0 2020-11-28T21:48:30.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.019+0000 7f54464ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5440143530 0x7f544013fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:30.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.019+0000 7f5445cfe700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f544005bcb0 0x7f54401457a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:30.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.019+0000 7f54464ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5440143530 0x7f544013fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39364/0 (socket says 172.21.15.162:39364) 2020-11-28T21:48:30.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.019+0000 7f54454fd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f544013c630 0x7f544013f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:30.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.019+0000 7f54464ff700 1 -- 172.21.15.162:0/3610339590 learned_addr learned my addr 172.21.15.162:0/3610339590 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:30.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.023+0000 7f54464ff700 1 -- 172.21.15.162:0/3610339590 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f544005bcb0 msgr2=0x7f54401457a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:30.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.023+0000 7f54464ff700 1 --2- 172.21.15.162:0/3610339590 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f544005bcb0 0x7f54401457a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:30.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.023+0000 7f54464ff700 1 -- 172.21.15.162:0/3610339590 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f544013c630 msgr2=0x7f544013f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:30.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.023+0000 7f54464ff700 1 --2- 172.21.15.162:0/3610339590 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f544013c630 0x7f544013f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:30.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.023+0000 7f54464ff700 1 -- 172.21.15.162:0/3610339590 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f544013dd90 con 0x7f5440143530 2020-11-28T21:48:30.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.023+0000 7f54464ff700 1 --2- 172.21.15.162:0/3610339590 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5440143530 0x7f544013fae0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f5440063660 tx=0x7f5438007f10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:30.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.023+0000 7f542effd700 1 -- 172.21.15.162:0/3610339590 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f543800a5a0 con 0x7f5440143530 2020-11-28T21:48:30.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.023+0000 7f542effd700 1 -- 172.21.15.162:0/3610339590 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f543800a700 con 0x7f5440143530 2020-11-28T21:48:30.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.023+0000 7f5446d00700 1 -- 172.21.15.162:0/3610339590 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5440143dd0 con 0x7f5440143530 2020-11-28T21:48:30.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.023+0000 7f542effd700 1 -- 172.21.15.162:0/3610339590 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f543800aa60 con 0x7f5440143530 2020-11-28T21:48:30.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.023+0000 7f5446d00700 1 -- 172.21.15.162:0/3610339590 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f544013ec50 con 0x7f5440143530 2020-11-28T21:48:30.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.023+0000 7f5446d00700 1 -- 172.21.15.162:0/3610339590 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5440145fd0 con 0x7f5440143530 2020-11-28T21:48:30.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.023+0000 7f542effd700 1 -- 172.21.15.162:0/3610339590 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5438019030 con 0x7f5440143530 2020-11-28T21:48:30.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.023+0000 7f542effd700 1 --2- 172.21.15.162:0/3610339590 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5420055440 0x7f5420057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:30.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.027+0000 7f542effd700 1 -- 172.21.15.162:0/3610339590 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f543806dc80 con 0x7f5440143530 2020-11-28T21:48:30.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.027+0000 7f5445cfe700 1 --2- 172.21.15.162:0/3610339590 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5420055440 0x7f5420057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:30.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.027+0000 7f5445cfe700 1 --2- 172.21.15.162:0/3610339590 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5420055440 0x7f5420057900 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f543000d000 tx=0x7f5430009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:30.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.031+0000 7f542effd700 1 -- 172.21.15.162:0/3610339590 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5438043f80 con 0x7f5440143530 2020-11-28T21:48:30.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.167+0000 7f5446d00700 1 -- 172.21.15.162:0/3610339590 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f544013e750 con 0x7f5440143530 2020-11-28T21:48:30.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.171+0000 7f542effd700 1 -- 172.21.15.162:0/3610339590 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16134 (secure 0 0 0) 0x7f54380475a0 con 0x7f5440143530 2020-11-28T21:48:30.172 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:48:30.173 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":44,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:48:27.785910+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:48:25.281355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2020-11-28T21:48:25.782088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:48:30.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.171+0000 7f5446d00700 1 -- 172.21.15.162:0/3610339590 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5420055440 msgr2=0x7f5420057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:30.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.171+0000 7f5446d00700 1 --2- 172.21.15.162:0/3610339590 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5420055440 0x7f5420057900 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f543000d000 tx=0x7f5430009040).stop 2020-11-28T21:48:30.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.171+0000 7f5446d00700 1 -- 172.21.15.162:0/3610339590 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5440143530 msgr2=0x7f544013fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:30.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.171+0000 7f5446d00700 1 --2- 172.21.15.162:0/3610339590 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5440143530 0x7f544013fae0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f5440063660 tx=0x7f5438007f10).stop 2020-11-28T21:48:30.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.171+0000 7f5446d00700 1 -- 172.21.15.162:0/3610339590 shutdown_connections 2020-11-28T21:48:30.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.171+0000 7f5446d00700 1 --2- 172.21.15.162:0/3610339590 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5420055440 0x7f5420057900 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:30.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.171+0000 7f5446d00700 1 --2- 172.21.15.162:0/3610339590 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f544005bcb0 0x7f54401457a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:30.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.171+0000 7f5446d00700 1 --2- 172.21.15.162:0/3610339590 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5440143530 0x7f544013fae0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:30.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.171+0000 7f5446d00700 1 --2- 172.21.15.162:0/3610339590 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f544013c630 0x7f544013f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:30.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.171+0000 7f5446d00700 1 -- 172.21.15.162:0/3610339590 >> 172.21.15.162:0/3610339590 conn(0x7f54400b5da0 msgr2=0x7f54401423b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:30.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.175+0000 7f5446d00700 1 -- 172.21.15.162:0/3610339590 shutdown_connections 2020-11-28T21:48:30.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.175+0000 7f5446d00700 1 -- 172.21.15.162:0/3610339590 wait complete. 2020-11-28T21:48:30.186 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-28T21:48:25.281355+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2020-11-28T21:48:25.782088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-28T21:48:30.188 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:48:30.307 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.307+0000 7f2ddc86f700 1 Processor -- start 2020-11-28T21:48:30.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.307+0000 7f2ddc86f700 1 -- start start 2020-11-28T21:48:30.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.307+0000 7f2ddc86f700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2dd8060d10 0x7f2dd8098ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:30.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.307+0000 7f2ddc86f700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dd80afec0 con 0x7f2dd8099d10 2020-11-28T21:48:30.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.307+0000 7f2ddc86f700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dd80b0160 con 0x7f2dd80b09f0 2020-11-28T21:48:30.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.307+0000 7f2ddc86f700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dd80608b0 con 0x7f2dd8060d10 2020-11-28T21:48:30.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.307+0000 7f2dd6ffd700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f2dd80b09f0 0x7f2dd8099600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:46212/0 (socket says 172.21.15.162:46212) 2020-11-28T21:48:30.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.307+0000 7f2dd6ffd700 1 -- 172.21.15.162:0/188689628 learned_addr learned my addr 172.21.15.162:0/188689628 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:48:30.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.307+0000 7f2dd77fe700 1 --2- 172.21.15.162:0/188689628 >> v2:172.21.15.162:3300/0 conn(0x7f2dd8060d10 0x7f2dd8098ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:30.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.307+0000 7f2dd67fc700 1 -- 172.21.15.162:0/188689628 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3476179656 0 0) 0x7f2dd80b0160 con 0x7f2dd80b09f0 2020-11-28T21:48:30.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.307+0000 7f2dd67fc700 1 -- 172.21.15.162:0/188689628 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2db8003230 con 0x7f2dd80b09f0 2020-11-28T21:48:30.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.311+0000 7f2dd67fc700 1 -- 172.21.15.162:0/188689628 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 854078676 0 0) 0x7f2dd80afec0 con 0x7f2dd8099d10 2020-11-28T21:48:30.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.311+0000 7f2dd77fe700 1 -- 172.21.15.162:0/188689628 >> v1:172.21.15.162:6789/0 conn(0x7f2dd80b09f0 legacy=0x7f2dd8099600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:30.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.311+0000 7f2dd77fe700 1 -- 172.21.15.162:0/188689628 >> v1:172.21.15.118:6789/0 conn(0x7f2dd8099d10 legacy=0x7f2dd814d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:30.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.311+0000 7f2dd77fe700 1 -- 172.21.15.162:0/188689628 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dd80b0400 con 0x7f2dd8060d10 2020-11-28T21:48:30.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.311+0000 7f2dd77fe700 1 --2- 172.21.15.162:0/188689628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2dd8060d10 0x7f2dd8098ac0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f2dc8002f30 tx=0x7f2dc8006dc0).ready entity=mon.1 client_cookie=8fcd5a85a6b0da23 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:30.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.311+0000 7f2dd67fc700 1 -- 172.21.15.162:0/188689628 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2dc8009800 con 0x7f2dd8060d10 2020-11-28T21:48:30.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.311+0000 7f2dd67fc700 1 -- 172.21.15.162:0/188689628 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dc8009960 con 0x7f2dd8060d10 2020-11-28T21:48:30.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.311+0000 7f2dd67fc700 1 -- 172.21.15.162:0/188689628 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2dc8009cc0 con 0x7f2dd8060d10 2020-11-28T21:48:30.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.311+0000 7f2ddc86f700 1 -- 172.21.15.162:0/188689628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2dd8060d10 msgr2=0x7f2dd8098ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:30.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.311+0000 7f2ddc86f700 1 --2- 172.21.15.162:0/188689628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2dd8060d10 0x7f2dd8098ac0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f2dc8002f30 tx=0x7f2dc8006dc0).stop 2020-11-28T21:48:30.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.311+0000 7f2ddc86f700 1 -- 172.21.15.162:0/188689628 shutdown_connections 2020-11-28T21:48:30.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.311+0000 7f2ddc86f700 1 --2- 172.21.15.162:0/188689628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2dd8060d10 0x7f2dd8098ac0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:30.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.311+0000 7f2ddc86f700 1 -- 172.21.15.162:0/188689628 >> 172.21.15.162:0/188689628 conn(0x7f2dd80b5520 msgr2=0x7f2dd80b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:30.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.311+0000 7f2ddc86f700 1 -- 172.21.15.162:0/188689628 shutdown_connections 2020-11-28T21:48:30.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.311+0000 7f2ddc86f700 1 -- 172.21.15.162:0/188689628 wait complete. 2020-11-28T21:48:30.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.311+0000 7f2ddc86f700 1 Processor -- start 2020-11-28T21:48:30.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.311+0000 7f2ddc86f700 1 -- start start 2020-11-28T21:48:30.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.315+0000 7f2ddc86f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2dd8060d10 0x7f2dd80a9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:30.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.315+0000 7f2ddc86f700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2dd8099d10 0x7f2dd80aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:30.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.315+0000 7f2ddc86f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2dd80b09f0 0x7f2dd808fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:30.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.315+0000 7f2ddc86f700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd80ab120 con 0x7f2dd8099d10 2020-11-28T21:48:30.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.315+0000 7f2ddc86f700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd80aad70 con 0x7f2dd80b09f0 2020-11-28T21:48:30.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.315+0000 7f2ddc86f700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd80ab350 con 0x7f2dd8060d10 2020-11-28T21:48:30.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.315+0000 7f2dd7fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2dd80b09f0 0x7f2dd808fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:30.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.315+0000 7f2dd7fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2dd80b09f0 0x7f2dd808fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39378/0 (socket says 172.21.15.162:39378) 2020-11-28T21:48:30.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.315+0000 7f2dd7fff700 1 -- 172.21.15.162:0/3444022058 learned_addr learned my addr 172.21.15.162:0/3444022058 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:30.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.315+0000 7f2dd77fe700 1 --2- 172.21.15.162:0/3444022058 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2dd8060d10 0x7f2dd80a9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:30.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.315+0000 7f2dd6ffd700 1 --2- 172.21.15.162:0/3444022058 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2dd8099d10 0x7f2dd80aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:30.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.315+0000 7f2dd7fff700 1 -- 172.21.15.162:0/3444022058 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2dd8060d10 msgr2=0x7f2dd80a9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:30.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.315+0000 7f2dd7fff700 1 --2- 172.21.15.162:0/3444022058 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2dd8060d10 0x7f2dd80a9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:30.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.315+0000 7f2dd7fff700 1 -- 172.21.15.162:0/3444022058 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2dd8099d10 msgr2=0x7f2dd80aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:30.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.315+0000 7f2dd7fff700 1 --2- 172.21.15.162:0/3444022058 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2dd8099d10 0x7f2dd80aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:30.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.315+0000 7f2dd7fff700 1 -- 172.21.15.162:0/3444022058 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dd808f910 con 0x7f2dd80b09f0 2020-11-28T21:48:30.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.315+0000 7f2dd7fff700 1 --2- 172.21.15.162:0/3444022058 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2dd80b09f0 0x7f2dd808fdb0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f2dd0000f00 tx=0x7f2dd0007d30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:30.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.315+0000 7f2dd4ff9700 1 -- 172.21.15.162:0/3444022058 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2dd000a5a0 con 0x7f2dd80b09f0 2020-11-28T21:48:30.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.315+0000 7f2dd4ff9700 1 -- 172.21.15.162:0/3444022058 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dd000a700 con 0x7f2dd80b09f0 2020-11-28T21:48:30.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.315+0000 7f2ddc86f700 1 -- 172.21.15.162:0/3444022058 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dd8149c70 con 0x7f2dd80b09f0 2020-11-28T21:48:30.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.315+0000 7f2dd4ff9700 1 -- 172.21.15.162:0/3444022058 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2dd000aa60 con 0x7f2dd80b09f0 2020-11-28T21:48:30.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.315+0000 7f2ddc86f700 1 -- 172.21.15.162:0/3444022058 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dd80aa810 con 0x7f2dd80b09f0 2020-11-28T21:48:30.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.315+0000 7f2ddc86f700 1 -- 172.21.15.162:0/3444022058 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dd80905e0 con 0x7f2dd80b09f0 2020-11-28T21:48:30.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.319+0000 7f2dd4ff9700 1 -- 172.21.15.162:0/3444022058 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2dd0010070 con 0x7f2dd80b09f0 2020-11-28T21:48:30.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.319+0000 7f2dd4ff9700 1 --2- 172.21.15.162:0/3444022058 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2dbc0554e0 0x7f2dbc0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:30.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.319+0000 7f2dd4ff9700 1 -- 172.21.15.162:0/3444022058 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2dd006d4a0 con 0x7f2dd80b09f0 2020-11-28T21:48:30.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.319+0000 7f2dd77fe700 1 --2- 172.21.15.162:0/3444022058 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2dbc0554e0 0x7f2dbc0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:30.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.319+0000 7f2dd77fe700 1 --2- 172.21.15.162:0/3444022058 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2dbc0554e0 0x7f2dbc0579a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f2dc8019000 tx=0x7f2dc8015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:30.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.323+0000 7f2dd4ff9700 1 -- 172.21.15.162:0/3444022058 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f2dd80905e0 con 0x7f2dd80b09f0 2020-11-28T21:48:30.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.463+0000 7f2ddc86f700 1 -- 172.21.15.162:0/3444022058 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f2dd80905e0 con 0x7f2dd80b09f0 2020-11-28T21:48:30.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.463+0000 7f2dd4ff9700 1 -- 172.21.15.162:0/3444022058 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v44) v1 ==== 110+0+10 (secure 0 0 0) 0x7f2dd80905e0 con 0x7f2dd80b09f0 2020-11-28T21:48:30.463 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:48:30.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.463+0000 7f2ddc86f700 1 -- 172.21.15.162:0/3444022058 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2dbc0554e0 msgr2=0x7f2dbc0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:30.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.463+0000 7f2ddc86f700 1 --2- 172.21.15.162:0/3444022058 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2dbc0554e0 0x7f2dbc0579a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f2dc8019000 tx=0x7f2dc8015000).stop 2020-11-28T21:48:30.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.463+0000 7f2ddc86f700 1 -- 172.21.15.162:0/3444022058 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2dd80b09f0 msgr2=0x7f2dd808fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:30.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.463+0000 7f2ddc86f700 1 --2- 172.21.15.162:0/3444022058 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2dd80b09f0 0x7f2dd808fdb0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f2dd0000f00 tx=0x7f2dd0007d30).stop 2020-11-28T21:48:30.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.463+0000 7f2ddc86f700 1 -- 172.21.15.162:0/3444022058 shutdown_connections 2020-11-28T21:48:30.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.463+0000 7f2ddc86f700 1 --2- 172.21.15.162:0/3444022058 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2dbc0554e0 0x7f2dbc0579a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:30.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.463+0000 7f2ddc86f700 1 --2- 172.21.15.162:0/3444022058 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2dd8060d10 0x7f2dd80a9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:30.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.463+0000 7f2ddc86f700 1 --2- 172.21.15.162:0/3444022058 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2dd80b09f0 0x7f2dd808fdb0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:30.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.463+0000 7f2ddc86f700 1 --2- 172.21.15.162:0/3444022058 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2dd8099d10 0x7f2dd80aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:30.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.463+0000 7f2ddc86f700 1 -- 172.21.15.162:0/3444022058 >> 172.21.15.162:0/3444022058 conn(0x7f2dd80b5520 msgr2=0x7f2dd8098820 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:30.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.463+0000 7f2ddc86f700 1 -- 172.21.15.162:0/3444022058 shutdown_connections 2020-11-28T21:48:30.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.463+0000 7f2ddc86f700 1 -- 172.21.15.162:0/3444022058 wait complete. 2020-11-28T21:48:30.477 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:48:30.594 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.595+0000 7fde17666700 1 Processor -- start 2020-11-28T21:48:30.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.595+0000 7fde17666700 1 -- start start 2020-11-28T21:48:30.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.595+0000 7fde17666700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fde101433c0 0x7fde100605c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:30.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.595+0000 7fde17666700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fde10067610 0x7fde1013fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:30.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.595+0000 7fde17666700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde10143950 con 0x7fde100b9ce0 2020-11-28T21:48:30.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.595+0000 7fde17666700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde100b4340 con 0x7fde10067610 2020-11-28T21:48:30.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.595+0000 7fde17666700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde10067ac0 con 0x7fde101433c0 2020-11-28T21:48:30.598 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.595+0000 7fde15c03700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fde10067610 0x7fde1013fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:30.598 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.595+0000 7fde15c03700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fde10067610 0x7fde1013fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.162:46374/0 (socket says 172.21.15.162:46374) 2020-11-28T21:48:30.598 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.595+0000 7fde15c03700 1 -- 172.21.15.162:0/3599414982 learned_addr learned my addr 172.21.15.162:0/3599414982 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:30.599 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.595+0000 7fde14c01700 1 --2- 172.21.15.162:0/3599414982 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fde101433c0 0x7fde100605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:30.599 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.595+0000 7fde15c03700 1 -- 172.21.15.162:0/3599414982 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fde101433c0 msgr2=0x7fde100605c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:30.599 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.595+0000 7fde15c03700 1 --2- 172.21.15.162:0/3599414982 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fde101433c0 0x7fde100605c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:30.599 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.595+0000 7fde15c03700 1 -- 172.21.15.162:0/3599414982 >> v1:172.21.15.118:6789/0 conn(0x7fde100b9ce0 legacy=0x7fde100b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:30.599 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.595+0000 7fde15c03700 1 -- 172.21.15.162:0/3599414982 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde100ba0c0 con 0x7fde10067610 2020-11-28T21:48:30.600 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.595+0000 7fde15c03700 1 --2- 172.21.15.162:0/3599414982 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fde10067610 0x7fde1013fc80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7fde0c001120 tx=0x7fde08009270).ready entity=mon.0 client_cookie=424b8f37d4221724 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:30.600 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.595+0000 7fde07fff700 1 -- 172.21.15.162:0/3599414982 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde08009e90 con 0x7fde10067610 2020-11-28T21:48:30.600 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.595+0000 7fde07fff700 1 -- 172.21.15.162:0/3599414982 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde0800bac0 con 0x7fde10067610 2020-11-28T21:48:30.600 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.595+0000 7fde07fff700 1 -- 172.21.15.162:0/3599414982 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde0800bc30 con 0x7fde10067610 2020-11-28T21:48:30.601 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.595+0000 7fde17666700 1 -- 172.21.15.162:0/3599414982 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fde10067610 msgr2=0x7fde1013fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:30.601 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.595+0000 7fde17666700 1 --2- 172.21.15.162:0/3599414982 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fde10067610 0x7fde1013fc80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7fde0c001120 tx=0x7fde08009270).stop 2020-11-28T21:48:30.601 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.599+0000 7fde17666700 1 -- 172.21.15.162:0/3599414982 shutdown_connections 2020-11-28T21:48:30.601 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.599+0000 7fde17666700 1 --2- 172.21.15.162:0/3599414982 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fde10067610 0x7fde1013fc80 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:30.601 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.599+0000 7fde17666700 1 --2- 172.21.15.162:0/3599414982 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fde101433c0 0x7fde100605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:30.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.599+0000 7fde17666700 1 -- 172.21.15.162:0/3599414982 >> 172.21.15.162:0/3599414982 conn(0x7fde1005c430 msgr2=0x7fde100b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:30.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.599+0000 7fde17666700 1 -- 172.21.15.162:0/3599414982 shutdown_connections 2020-11-28T21:48:30.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.599+0000 7fde17666700 1 -- 172.21.15.162:0/3599414982 wait complete. 2020-11-28T21:48:30.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.599+0000 7fde17666700 1 Processor -- start 2020-11-28T21:48:30.605 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.599+0000 7fde17666700 1 -- start start 2020-11-28T21:48:30.605 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.599+0000 7fde17666700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde10067610 0x7fde1009d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:30.605 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.599+0000 7fde17666700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fde100b9ce0 0x7fde100b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:30.605 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.599+0000 7fde17666700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fde101433c0 0x7fde10144dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:30.605 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.599+0000 7fde17666700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde10154e50 con 0x7fde101433c0 2020-11-28T21:48:30.606 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.599+0000 7fde17666700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde100b1090 con 0x7fde10067610 2020-11-28T21:48:30.606 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.599+0000 7fde17666700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde101550b0 con 0x7fde100b9ce0 2020-11-28T21:48:30.606 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.599+0000 7fde15402700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde10067610 0x7fde1009d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:30.606 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.599+0000 7fde15402700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde10067610 0x7fde1009d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39388/0 (socket says 172.21.15.162:39388) 2020-11-28T21:48:30.606 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.599+0000 7fde15402700 1 -- 172.21.15.162:0/1833220671 learned_addr learned my addr 172.21.15.162:0/1833220671 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:30.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.599+0000 7fde15c03700 1 --2- 172.21.15.162:0/1833220671 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fde101433c0 0x7fde10144dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:30.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.599+0000 7fde14c01700 1 --2- 172.21.15.162:0/1833220671 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fde100b9ce0 0x7fde100b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:30.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.599+0000 7fde15402700 1 -- 172.21.15.162:0/1833220671 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fde100b9ce0 msgr2=0x7fde100b0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:30.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.599+0000 7fde15402700 1 --2- 172.21.15.162:0/1833220671 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fde100b9ce0 0x7fde100b0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:30.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.599+0000 7fde15402700 1 -- 172.21.15.162:0/1833220671 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fde101433c0 msgr2=0x7fde10144dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:30.608 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.599+0000 7fde15402700 1 --2- 172.21.15.162:0/1833220671 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fde101433c0 0x7fde10144dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:30.608 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.599+0000 7fde15402700 1 -- 172.21.15.162:0/1833220671 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde10155350 con 0x7fde10067610 2020-11-28T21:48:30.608 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.599+0000 7fde15402700 1 --2- 172.21.15.162:0/1833220671 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde10067610 0x7fde1009d680 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7fde0c0011e0 tx=0x7fde0c007bd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:30.608 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.599+0000 7fde067fc700 1 -- 172.21.15.162:0/1833220671 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde0c010510 con 0x7fde10067610 2020-11-28T21:48:30.608 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.599+0000 7fde067fc700 1 -- 172.21.15.162:0/1833220671 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde0c010670 con 0x7fde10067610 2020-11-28T21:48:30.609 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.599+0000 7fde17666700 1 -- 172.21.15.162:0/1833220671 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde101555f0 con 0x7fde10067610 2020-11-28T21:48:30.609 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.599+0000 7fde067fc700 1 -- 172.21.15.162:0/1833220671 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde0c0107e0 con 0x7fde10067610 2020-11-28T21:48:30.609 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.599+0000 7fde17666700 1 -- 172.21.15.162:0/1833220671 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde100acaa0 con 0x7fde10067610 2020-11-28T21:48:30.609 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.603+0000 7fde17666700 1 -- 172.21.15.162:0/1833220671 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde10155cb0 con 0x7fde10067610 2020-11-28T21:48:30.609 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.603+0000 7fde067fc700 1 -- 172.21.15.162:0/1833220671 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fde0c010070 con 0x7fde10067610 2020-11-28T21:48:30.610 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.603+0000 7fde067fc700 1 --2- 172.21.15.162:0/1833220671 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fddf8055500 0x7fddf80579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:30.610 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.603+0000 7fde067fc700 1 -- 172.21.15.162:0/1833220671 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fde0c06da30 con 0x7fde10067610 2020-11-28T21:48:30.610 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.603+0000 7fde14c01700 1 --2- 172.21.15.162:0/1833220671 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fddf8055500 0x7fddf80579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:30.610 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.603+0000 7fde14c01700 1 --2- 172.21.15.162:0/1833220671 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fddf8055500 0x7fddf80579c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7fde0000b000 tx=0x7fde00008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:30.611 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.611+0000 7fde067fc700 1 -- 172.21.15.162:0/1833220671 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fde0c043d30 con 0x7fde10067610 2020-11-28T21:48:30.750 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.751+0000 7fde17666700 1 -- 172.21.15.162:0/1833220671 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fde10057050 con 0x7fde10067610 2020-11-28T21:48:30.756 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.751+0000 7fde067fc700 1 -- 172.21.15.162:0/1833220671 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v44) v1 ==== 92+0+10 (secure 0 0 0) 0x7fde0c047350 con 0x7fde10067610 2020-11-28T21:48:30.756 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.751+0000 7fde17666700 1 -- 172.21.15.162:0/1833220671 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fddf8055500 msgr2=0x7fddf80579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:30.756 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.751+0000 7fde17666700 1 --2- 172.21.15.162:0/1833220671 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fddf8055500 0x7fddf80579c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7fde0000b000 tx=0x7fde00008040).stop 2020-11-28T21:48:30.756 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.751+0000 7fde17666700 1 -- 172.21.15.162:0/1833220671 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde10067610 msgr2=0x7fde1009d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:30.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.751+0000 7fde17666700 1 --2- 172.21.15.162:0/1833220671 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde10067610 0x7fde1009d680 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7fde0c0011e0 tx=0x7fde0c007bd0).stop 2020-11-28T21:48:30.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.751+0000 7fde17666700 1 -- 172.21.15.162:0/1833220671 shutdown_connections 2020-11-28T21:48:30.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.751+0000 7fde17666700 1 --2- 172.21.15.162:0/1833220671 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fddf8055500 0x7fddf80579c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:30.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.751+0000 7fde17666700 1 --2- 172.21.15.162:0/1833220671 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fde101433c0 0x7fde10144dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:30.758 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.751+0000 7fde17666700 1 --2- 172.21.15.162:0/1833220671 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde10067610 0x7fde1009d680 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:30.758 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.751+0000 7fde17666700 1 --2- 172.21.15.162:0/1833220671 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fde100b9ce0 0x7fde100b0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:30.758 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.751+0000 7fde17666700 1 -- 172.21.15.162:0/1833220671 >> 172.21.15.162:0/1833220671 conn(0x7fde1005c430 msgr2=0x7fde1005f750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:30.758 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.751+0000 7fde17666700 1 -- 172.21.15.162:0/1833220671 shutdown_connections 2020-11-28T21:48:30.758 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.751+0000 7fde17666700 1 -- 172.21.15.162:0/1833220671 wait complete. 2020-11-28T21:48:30.759 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:48:30.765 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-28T21:48:30.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.875+0000 7f86a8eb0700 1 Processor -- start 2020-11-28T21:48:30.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.875+0000 7f86a8eb0700 1 -- start start 2020-11-28T21:48:30.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.879+0000 7f86a8eb0700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f86a40b3950 0x7f86a40b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:30.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.879+0000 7f86a8eb0700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f86a40b18c0 0x7f86a413dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:30.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.879+0000 7f86a8eb0700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f86a40b44e0 0x7f86a413e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:30.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.879+0000 7f86a8eb0700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86a40aab50 con 0x7f86a40b44e0 2020-11-28T21:48:30.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.879+0000 7f86a8eb0700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86a40b1f50 con 0x7f86a40b3950 2020-11-28T21:48:30.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.879+0000 7f86a8eb0700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86a4143e90 con 0x7f86a40b18c0 2020-11-28T21:48:30.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.879+0000 7f86a37fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f86a40b3950 0x7f86a40b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:30.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.879+0000 7f86a37fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f86a40b3950 0x7f86a40b3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39398/0 (socket says 172.21.15.162:39398) 2020-11-28T21:48:30.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.879+0000 7f86a3fff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f86a40b44e0 0x7f86a413e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:30.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.879+0000 7f86a37fe700 1 -- 172.21.15.162:0/4073719893 learned_addr learned my addr 172.21.15.162:0/4073719893 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:30.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.879+0000 7f86a2ffd700 1 --2- 172.21.15.162:0/4073719893 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f86a40b18c0 0x7f86a413dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:30.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.879+0000 7f86a37fe700 1 -- 172.21.15.162:0/4073719893 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f86a40b18c0 msgr2=0x7f86a413dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:30.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.879+0000 7f86a37fe700 1 --2- 172.21.15.162:0/4073719893 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f86a40b18c0 0x7f86a413dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:30.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.879+0000 7f86a37fe700 1 -- 172.21.15.162:0/4073719893 >> v2:172.21.15.118:3300/0 conn(0x7f86a40b44e0 msgr2=0x7f86a413e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:30.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.879+0000 7f86a37fe700 1 --2- 172.21.15.162:0/4073719893 >> v2:172.21.15.118:3300/0 conn(0x7f86a40b44e0 0x7f86a413e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:30.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.879+0000 7f86a37fe700 1 -- 172.21.15.162:0/4073719893 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a40aa8b0 con 0x7f86a40b3950 2020-11-28T21:48:30.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.879+0000 7f86a37fe700 1 --2- 172.21.15.162:0/4073719893 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86a40b3950 0x7f86a40b3d30 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f8694002f30 tx=0x7f8694006e80).ready entity=mon.1 client_cookie=a72f18756d23eb25 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:30.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.879+0000 7f86a27fc700 1 -- 172.21.15.162:0/4073719893 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8694009640 con 0x7f86a40b3950 2020-11-28T21:48:30.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.879+0000 7f86a27fc700 1 -- 172.21.15.162:0/4073719893 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86940097a0 con 0x7f86a40b3950 2020-11-28T21:48:30.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.879+0000 7f86a27fc700 1 -- 172.21.15.162:0/4073719893 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8694009b00 con 0x7f86a40b3950 2020-11-28T21:48:30.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.879+0000 7f86a8eb0700 1 -- 172.21.15.162:0/4073719893 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86a40b3950 msgr2=0x7f86a40b3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:30.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.879+0000 7f86a8eb0700 1 --2- 172.21.15.162:0/4073719893 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86a40b3950 0x7f86a40b3d30 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f8694002f30 tx=0x7f8694006e80).stop 2020-11-28T21:48:30.889 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.879+0000 7f86a8eb0700 1 -- 172.21.15.162:0/4073719893 shutdown_connections 2020-11-28T21:48:30.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.879+0000 7f86a8eb0700 1 --2- 172.21.15.162:0/4073719893 >> v2:172.21.15.118:3300/0 conn(0x7f86a40b44e0 0x7f86a413e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:30.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.879+0000 7f86a8eb0700 1 --2- 172.21.15.162:0/4073719893 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86a40b3950 0x7f86a40b3d30 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:30.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.879+0000 7f86a8eb0700 1 --2- 172.21.15.162:0/4073719893 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f86a40b18c0 0x7f86a413dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:30.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.879+0000 7f86a8eb0700 1 -- 172.21.15.162:0/4073719893 >> 172.21.15.162:0/4073719893 conn(0x7f86a4062630 msgr2=0x7f86a4062a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:30.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.879+0000 7f86a8eb0700 1 -- 172.21.15.162:0/4073719893 shutdown_connections 2020-11-28T21:48:30.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.879+0000 7f86a8eb0700 1 -- 172.21.15.162:0/4073719893 wait complete. 2020-11-28T21:48:30.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.883+0000 7f86a8eb0700 1 Processor -- start 2020-11-28T21:48:30.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.883+0000 7f86a8eb0700 1 -- start start 2020-11-28T21:48:30.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.883+0000 7f86a8eb0700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f86a40b18c0 0x7f86a405f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:30.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.883+0000 7f86a8eb0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86a40b3950 0x7f86a4067290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:30.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.883+0000 7f86a8eb0700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f86a40b44e0 0x7f86a4141250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:30.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.883+0000 7f86a8eb0700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a4140550 con 0x7f86a40b44e0 2020-11-28T21:48:30.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.883+0000 7f86a8eb0700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a41407f0 con 0x7f86a40b3950 2020-11-28T21:48:30.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.883+0000 7f86a8eb0700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86a4140a90 con 0x7f86a40b18c0 2020-11-28T21:48:30.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.883+0000 7f86a2ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86a40b3950 0x7f86a4067290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:30.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.883+0000 7f86a2ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86a40b3950 0x7f86a4067290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39402/0 (socket says 172.21.15.162:39402) 2020-11-28T21:48:30.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.883+0000 7f86a2ffd700 1 -- 172.21.15.162:0/510539985 learned_addr learned my addr 172.21.15.162:0/510539985 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:30.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.883+0000 7f86a3fff700 1 --2- 172.21.15.162:0/510539985 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f86a40b44e0 0x7f86a4141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:30.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.883+0000 7f86a37fe700 1 --2- 172.21.15.162:0/510539985 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f86a40b18c0 0x7f86a405f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:30.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.883+0000 7f86a2ffd700 1 -- 172.21.15.162:0/510539985 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f86a40b18c0 msgr2=0x7f86a405f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:30.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.883+0000 7f86a2ffd700 1 --2- 172.21.15.162:0/510539985 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f86a40b18c0 0x7f86a405f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:30.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.883+0000 7f86a2ffd700 1 -- 172.21.15.162:0/510539985 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f86a40b44e0 msgr2=0x7f86a4141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:30.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.883+0000 7f86a2ffd700 1 --2- 172.21.15.162:0/510539985 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f86a40b44e0 0x7f86a4141250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:30.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.883+0000 7f86a2ffd700 1 -- 172.21.15.162:0/510539985 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a41401d0 con 0x7f86a40b3950 2020-11-28T21:48:30.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.883+0000 7f86a37fe700 1 --2- 172.21.15.162:0/510539985 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f86a40b18c0 0x7f86a405f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:48:30.896 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.883+0000 7f86a2ffd700 1 --2- 172.21.15.162:0/510539985 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86a40b3950 0x7f86a4067290 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f868c00c000 tx=0x7f868c0091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:30.896 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.883+0000 7f86a0ff9700 1 -- 172.21.15.162:0/510539985 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f868c0095b0 con 0x7f86a40b3950 2020-11-28T21:48:30.896 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.883+0000 7f86a0ff9700 1 -- 172.21.15.162:0/510539985 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868c00cbb0 con 0x7f86a40b3950 2020-11-28T21:48:30.896 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.883+0000 7f86a8eb0700 1 -- 172.21.15.162:0/510539985 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86a4060fa0 con 0x7f86a40b3950 2020-11-28T21:48:30.897 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.883+0000 7f86a0ff9700 1 -- 172.21.15.162:0/510539985 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f868c0123f0 con 0x7f86a40b3950 2020-11-28T21:48:30.897 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.883+0000 7f86a8eb0700 1 -- 172.21.15.162:0/510539985 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86a4141960 con 0x7f86a40b3950 2020-11-28T21:48:30.897 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.887+0000 7f86a8eb0700 1 -- 172.21.15.162:0/510539985 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86a413ee40 con 0x7f86a40b3950 2020-11-28T21:48:30.897 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.887+0000 7f86a0ff9700 1 -- 172.21.15.162:0/510539985 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f868c022070 con 0x7f86a40b3950 2020-11-28T21:48:30.898 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.887+0000 7f86a0ff9700 1 --2- 172.21.15.162:0/510539985 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f867c055330 0x7f867c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:30.898 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.887+0000 7f86a0ff9700 1 -- 172.21.15.162:0/510539985 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f868c06e6c0 con 0x7f86a40b3950 2020-11-28T21:48:30.898 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.887+0000 7f86a37fe700 1 --2- 172.21.15.162:0/510539985 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f867c055330 0x7f867c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:30.898 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.887+0000 7f86a37fe700 1 --2- 172.21.15.162:0/510539985 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f867c055330 0x7f867c0577f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f8694019000 tx=0x7f8694015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:30.899 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:30.895+0000 7f86a0ff9700 1 -- 172.21.15.162:0/510539985 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f86a413ee40 con 0x7f86a40b3950 2020-11-28T21:48:31.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.027+0000 7f86a8eb0700 1 -- 172.21.15.162:0/510539985 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f86a413ee40 con 0x7f86a40b3950 2020-11-28T21:48:31.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.027+0000 7f86a0ff9700 1 -- 172.21.15.162:0/510539985 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v44) v1 ==== 104+0+10 (secure 0 0 0) 0x7f86a413ee40 con 0x7f86a40b3950 2020-11-28T21:48:31.028 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:48:31.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.027+0000 7f86a8eb0700 1 -- 172.21.15.162:0/510539985 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f867c055330 msgr2=0x7f867c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:31.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.027+0000 7f86a8eb0700 1 --2- 172.21.15.162:0/510539985 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f867c055330 0x7f867c0577f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f8694019000 tx=0x7f8694015000).stop 2020-11-28T21:48:31.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.027+0000 7f86a8eb0700 1 -- 172.21.15.162:0/510539985 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86a40b3950 msgr2=0x7f86a4067290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:31.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.027+0000 7f86a8eb0700 1 --2- 172.21.15.162:0/510539985 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86a40b3950 0x7f86a4067290 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f868c00c000 tx=0x7f868c0091a0).stop 2020-11-28T21:48:31.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.027+0000 7f86a8eb0700 1 -- 172.21.15.162:0/510539985 shutdown_connections 2020-11-28T21:48:31.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.027+0000 7f86a8eb0700 1 --2- 172.21.15.162:0/510539985 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f867c055330 0x7f867c0577f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:31.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.031+0000 7f86a8eb0700 1 --2- 172.21.15.162:0/510539985 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f86a40b18c0 0x7f86a405f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:31.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.031+0000 7f86a8eb0700 1 --2- 172.21.15.162:0/510539985 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f86a40b44e0 0x7f86a4141250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:31.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.031+0000 7f86a8eb0700 1 --2- 172.21.15.162:0/510539985 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86a40b3950 0x7f86a4067290 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:31.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.031+0000 7f86a8eb0700 1 -- 172.21.15.162:0/510539985 >> 172.21.15.162:0/510539985 conn(0x7f86a4062630 msgr2=0x7f86a40b4d30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:31.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.031+0000 7f86a8eb0700 1 -- 172.21.15.162:0/510539985 shutdown_connections 2020-11-28T21:48:31.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.031+0000 7f86a8eb0700 1 -- 172.21.15.162:0/510539985 wait complete. 2020-11-28T21:48:31.047 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-28T21:48:31.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.167+0000 7f7a4ec69700 1 Processor -- start 2020-11-28T21:48:31.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.167+0000 7f7a4ec69700 1 -- start start 2020-11-28T21:48:31.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.167+0000 7f7a4ec69700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f7a480af5f0 0x7f7a481445a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:31.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.167+0000 7f7a4ec69700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7a48061100 0x7f7a48140660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:31.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.167+0000 7f7a4ec69700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a48143c30 con 0x7f7a480b1b20 2020-11-28T21:48:31.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.167+0000 7f7a4ec69700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a480b2680 con 0x7f7a480af5f0 2020-11-28T21:48:31.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.167+0000 7f7a4ec69700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a48061d90 con 0x7f7a48061100 2020-11-28T21:48:31.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.171+0000 7f7a4d466700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7a48061100 0x7f7a48140660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:31.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.171+0000 7f7a4d466700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7a48061100 0x7f7a48140660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39410/0 (socket says 172.21.15.162:39410) 2020-11-28T21:48:31.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.171+0000 7f7a4d466700 1 -- 172.21.15.162:0/4002449719 learned_addr learned my addr 172.21.15.162:0/4002449719 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:31.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.171+0000 7f7a4dc67700 1 --2- 172.21.15.162:0/4002449719 >> v2:172.21.15.118:3300/0 conn(0x7f7a480af5f0 0x7f7a481445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:31.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.171+0000 7f7a4d466700 1 -- 172.21.15.162:0/4002449719 >> v2:172.21.15.118:3300/0 conn(0x7f7a480af5f0 msgr2=0x7f7a481445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:31.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.171+0000 7f7a4d466700 1 --2- 172.21.15.162:0/4002449719 >> v2:172.21.15.118:3300/0 conn(0x7f7a480af5f0 0x7f7a481445a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:31.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.171+0000 7f7a4d466700 1 -- 172.21.15.162:0/4002449719 >> v1:172.21.15.118:6789/0 conn(0x7f7a480b1b20 legacy=0x7f7a480b1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:31.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.171+0000 7f7a4d466700 1 -- 172.21.15.162:0/4002449719 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a48062030 con 0x7f7a48061100 2020-11-28T21:48:31.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.171+0000 7f7a4d466700 1 --2- 172.21.15.162:0/4002449719 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a48061100 0x7f7a48140660 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7f7a3c002f30 tx=0x7f7a3c006e80).ready entity=mon.1 client_cookie=52a70b04deee19d3 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:31.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.171+0000 7f7a4cc65700 1 -- 172.21.15.162:0/4002449719 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a3c0097f0 con 0x7f7a48061100 2020-11-28T21:48:31.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.171+0000 7f7a4cc65700 1 -- 172.21.15.162:0/4002449719 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a3c009950 con 0x7f7a48061100 2020-11-28T21:48:31.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.171+0000 7f7a4cc65700 1 -- 172.21.15.162:0/4002449719 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a3c009cb0 con 0x7f7a48061100 2020-11-28T21:48:31.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.171+0000 7f7a4ec69700 1 -- 172.21.15.162:0/4002449719 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a48061100 msgr2=0x7f7a48140660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:31.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.171+0000 7f7a4ec69700 1 --2- 172.21.15.162:0/4002449719 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a48061100 0x7f7a48140660 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7f7a3c002f30 tx=0x7f7a3c006e80).stop 2020-11-28T21:48:31.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.171+0000 7f7a4ec69700 1 -- 172.21.15.162:0/4002449719 shutdown_connections 2020-11-28T21:48:31.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.171+0000 7f7a4ec69700 1 --2- 172.21.15.162:0/4002449719 >> v2:172.21.15.118:3300/0 conn(0x7f7a480af5f0 0x7f7a481445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:31.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.171+0000 7f7a4ec69700 1 --2- 172.21.15.162:0/4002449719 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a48061100 0x7f7a48140660 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:31.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.171+0000 7f7a4ec69700 1 -- 172.21.15.162:0/4002449719 >> 172.21.15.162:0/4002449719 conn(0x7f7a480b5de0 msgr2=0x7f7a480b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:31.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.171+0000 7f7a4ec69700 1 -- 172.21.15.162:0/4002449719 shutdown_connections 2020-11-28T21:48:31.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.171+0000 7f7a4ec69700 1 -- 172.21.15.162:0/4002449719 wait complete. 2020-11-28T21:48:31.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.175+0000 7f7a4ec69700 1 Processor -- start 2020-11-28T21:48:31.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.175+0000 7f7a4ec69700 1 -- start start 2020-11-28T21:48:31.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.175+0000 7f7a4ec69700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a48061100 0x7f7a48152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:31.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.175+0000 7f7a4ec69700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7a480af5f0 0x7f7a480aac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:31.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.175+0000 7f7a4ec69700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a480b1b20 0x7f7a4813d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:31.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.175+0000 7f7a4ec69700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a48153750 con 0x7f7a48061100 2020-11-28T21:48:31.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.175+0000 7f7a4ec69700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a480665f0 con 0x7f7a480b1b20 2020-11-28T21:48:31.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.175+0000 7f7a4ec69700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a48066890 con 0x7f7a480af5f0 2020-11-28T21:48:31.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.175+0000 7f7a4e468700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a480b1b20 0x7f7a4813d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:31.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.175+0000 7f7a4e468700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a480b1b20 0x7f7a4813d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39420/0 (socket says 172.21.15.162:39420) 2020-11-28T21:48:31.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.175+0000 7f7a4e468700 1 -- 172.21.15.162:0/2539950590 learned_addr learned my addr 172.21.15.162:0/2539950590 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:31.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.175+0000 7f7a4dc67700 1 --2- 172.21.15.162:0/2539950590 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a48061100 0x7f7a48152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:31.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.175+0000 7f7a4d466700 1 --2- 172.21.15.162:0/2539950590 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7a480af5f0 0x7f7a480aac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:31.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.175+0000 7f7a4e468700 1 -- 172.21.15.162:0/2539950590 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7a480af5f0 msgr2=0x7f7a480aac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:31.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.175+0000 7f7a4e468700 1 --2- 172.21.15.162:0/2539950590 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7a480af5f0 0x7f7a480aac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:31.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.175+0000 7f7a4e468700 1 -- 172.21.15.162:0/2539950590 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a48061100 msgr2=0x7f7a48152ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:31.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.175+0000 7f7a4e468700 1 --2- 172.21.15.162:0/2539950590 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a48061100 0x7f7a48152ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:31.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.175+0000 7f7a4e468700 1 -- 172.21.15.162:0/2539950590 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a48153390 con 0x7f7a480b1b20 2020-11-28T21:48:31.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.175+0000 7f7a4dc67700 1 --2- 172.21.15.162:0/2539950590 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a48061100 0x7f7a48152ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:48:31.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.175+0000 7f7a4e468700 1 --2- 172.21.15.162:0/2539950590 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a480b1b20 0x7f7a4813d080 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f7a48067c60 tx=0x7f7a40007f70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:31.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.175+0000 7f7a3affd700 1 -- 172.21.15.162:0/2539950590 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a40003080 con 0x7f7a480b1b20 2020-11-28T21:48:31.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.175+0000 7f7a3affd700 1 -- 172.21.15.162:0/2539950590 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4000a7b0 con 0x7f7a480b1b20 2020-11-28T21:48:31.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.175+0000 7f7a4ec69700 1 -- 172.21.15.162:0/2539950590 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a48065010 con 0x7f7a480b1b20 2020-11-28T21:48:31.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.175+0000 7f7a3affd700 1 -- 172.21.15.162:0/2539950590 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a4000ab10 con 0x7f7a480b1b20 2020-11-28T21:48:31.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.175+0000 7f7a4ec69700 1 -- 172.21.15.162:0/2539950590 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a480ab2b0 con 0x7f7a480b1b20 2020-11-28T21:48:31.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.179+0000 7f7a4ec69700 1 -- 172.21.15.162:0/2539950590 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a48066a50 con 0x7f7a480b1b20 2020-11-28T21:48:31.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.179+0000 7f7a3affd700 1 -- 172.21.15.162:0/2539950590 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7a40019030 con 0x7f7a480b1b20 2020-11-28T21:48:31.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.179+0000 7f7a3affd700 1 --2- 172.21.15.162:0/2539950590 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7a280554e0 0x7f7a280579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:31.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.179+0000 7f7a3affd700 1 -- 172.21.15.162:0/2539950590 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7a4006d730 con 0x7f7a480b1b20 2020-11-28T21:48:31.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.179+0000 7f7a4dc67700 1 --2- 172.21.15.162:0/2539950590 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7a280554e0 0x7f7a280579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:31.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.179+0000 7f7a4dc67700 1 --2- 172.21.15.162:0/2539950590 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7a280554e0 0x7f7a280579a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f7a4400b000 tx=0x7f7a44008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:31.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.191+0000 7f7a3affd700 1 -- 172.21.15.162:0/2539950590 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7a40043990 con 0x7f7a480b1b20 2020-11-28T21:48:31.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.331+0000 7f7a4ec69700 1 -- 172.21.15.162:0/2539950590 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7a48153520 con 0x7f7a480b1b20 2020-11-28T21:48:31.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.331+0000 7f7a3affd700 1 -- 172.21.15.162:0/2539950590 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v44) v1 ==== 100+0+10 (secure 0 0 0) 0x7f7a40046fb0 con 0x7f7a480b1b20 2020-11-28T21:48:31.332 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:48:31.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.335+0000 7f7a4ec69700 1 -- 172.21.15.162:0/2539950590 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7a280554e0 msgr2=0x7f7a280579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:31.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.335+0000 7f7a4ec69700 1 --2- 172.21.15.162:0/2539950590 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7a280554e0 0x7f7a280579a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f7a4400b000 tx=0x7f7a44008040).stop 2020-11-28T21:48:31.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.335+0000 7f7a4ec69700 1 -- 172.21.15.162:0/2539950590 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a480b1b20 msgr2=0x7f7a4813d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:31.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.335+0000 7f7a4ec69700 1 --2- 172.21.15.162:0/2539950590 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a480b1b20 0x7f7a4813d080 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f7a48067c60 tx=0x7f7a40007f70).stop 2020-11-28T21:48:31.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.335+0000 7f7a4ec69700 1 -- 172.21.15.162:0/2539950590 shutdown_connections 2020-11-28T21:48:31.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.335+0000 7f7a4ec69700 1 --2- 172.21.15.162:0/2539950590 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7a280554e0 0x7f7a280579a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:31.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.335+0000 7f7a4ec69700 1 --2- 172.21.15.162:0/2539950590 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a480b1b20 0x7f7a4813d080 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:31.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.335+0000 7f7a4ec69700 1 --2- 172.21.15.162:0/2539950590 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a48061100 0x7f7a48152ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:31.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.335+0000 7f7a4ec69700 1 --2- 172.21.15.162:0/2539950590 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7a480af5f0 0x7f7a480aac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:31.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.335+0000 7f7a4ec69700 1 -- 172.21.15.162:0/2539950590 >> 172.21.15.162:0/2539950590 conn(0x7f7a480b5de0 msgr2=0x7f7a480b1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:31.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.335+0000 7f7a4ec69700 1 -- 172.21.15.162:0/2539950590 shutdown_connections 2020-11-28T21:48:31.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.335+0000 7f7a4ec69700 1 -- 172.21.15.162:0/2539950590 wait complete. 2020-11-28T21:48:31.349 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-28T21:48:31.350 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:48:31.460 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.459+0000 7ff42b59e700 1 Processor -- start 2020-11-28T21:48:31.460 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.459+0000 7ff42b59e700 1 -- start start 2020-11-28T21:48:31.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.459+0000 7ff42b59e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff4240477e0 0x7ff4240411c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:31.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.459+0000 7ff42b59e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff424006b30 0x7ff424047bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:31.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.459+0000 7ff42b59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff42402c8b0 0x7ff42402cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:31.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.459+0000 7ff42b59e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff42402d210 con 0x7ff4240477e0 2020-11-28T21:48:31.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.459+0000 7ff42b59e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff424046e70 con 0x7ff424006b30 2020-11-28T21:48:31.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.459+0000 7ff42b59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4240d73f0 con 0x7ff42402c8b0 2020-11-28T21:48:31.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.459+0000 7ff429d9b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff424006b30 0x7ff424047bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:31.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.459+0000 7ff42a59c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff4240477e0 0x7ff4240411c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:31.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.459+0000 7ff429d9b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff424006b30 0x7ff424047bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39426/0 (socket says 172.21.15.162:39426) 2020-11-28T21:48:31.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.459+0000 7ff429d9b700 1 -- 172.21.15.162:0/3946723565 learned_addr learned my addr 172.21.15.162:0/3946723565 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:31.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.459+0000 7ff42ad9d700 1 --2- 172.21.15.162:0/3946723565 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff42402c8b0 0x7ff42402cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:31.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.463+0000 7ff429d9b700 1 -- 172.21.15.162:0/3946723565 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff42402c8b0 msgr2=0x7ff42402cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:31.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.463+0000 7ff429d9b700 1 --2- 172.21.15.162:0/3946723565 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff42402c8b0 0x7ff42402cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:31.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.463+0000 7ff429d9b700 1 -- 172.21.15.162:0/3946723565 >> v2:172.21.15.118:3300/0 conn(0x7ff4240477e0 msgr2=0x7ff4240411c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:31.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.463+0000 7ff429d9b700 1 --2- 172.21.15.162:0/3946723565 >> v2:172.21.15.118:3300/0 conn(0x7ff4240477e0 0x7ff4240411c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:31.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.463+0000 7ff429d9b700 1 -- 172.21.15.162:0/3946723565 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4240d7690 con 0x7ff424006b30 2020-11-28T21:48:31.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.463+0000 7ff429d9b700 1 --2- 172.21.15.162:0/3946723565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff424006b30 0x7ff424047bc0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7ff414002f30 tx=0x7ff414006e80).ready entity=mon.1 client_cookie=e5999a49785df365 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:31.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.463+0000 7ff42959a700 1 -- 172.21.15.162:0/3946723565 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff414009640 con 0x7ff424006b30 2020-11-28T21:48:31.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.463+0000 7ff42959a700 1 -- 172.21.15.162:0/3946723565 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4140097a0 con 0x7ff424006b30 2020-11-28T21:48:31.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.463+0000 7ff42959a700 1 -- 172.21.15.162:0/3946723565 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff414009b00 con 0x7ff424006b30 2020-11-28T21:48:31.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.463+0000 7ff42b59e700 1 -- 172.21.15.162:0/3946723565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff424006b30 msgr2=0x7ff424047bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:31.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.463+0000 7ff42b59e700 1 --2- 172.21.15.162:0/3946723565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff424006b30 0x7ff424047bc0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7ff414002f30 tx=0x7ff414006e80).stop 2020-11-28T21:48:31.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.463+0000 7ff42b59e700 1 -- 172.21.15.162:0/3946723565 shutdown_connections 2020-11-28T21:48:31.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.463+0000 7ff42b59e700 1 --2- 172.21.15.162:0/3946723565 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff42402c8b0 0x7ff42402cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:31.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.463+0000 7ff42b59e700 1 --2- 172.21.15.162:0/3946723565 >> v2:172.21.15.118:3300/0 conn(0x7ff4240477e0 0x7ff4240411c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:31.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.463+0000 7ff42b59e700 1 --2- 172.21.15.162:0/3946723565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff424006b30 0x7ff424047bc0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:31.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.463+0000 7ff42b59e700 1 -- 172.21.15.162:0/3946723565 >> 172.21.15.162:0/3946723565 conn(0x7ff42404b100 msgr2=0x7ff4240470d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:31.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.463+0000 7ff42b59e700 1 -- 172.21.15.162:0/3946723565 shutdown_connections 2020-11-28T21:48:31.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.463+0000 7ff42b59e700 1 -- 172.21.15.162:0/3946723565 wait complete. 2020-11-28T21:48:31.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.463+0000 7ff42b59e700 1 Processor -- start 2020-11-28T21:48:31.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.463+0000 7ff42b59e700 1 -- start start 2020-11-28T21:48:31.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.467+0000 7ff42b59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff424006b30 0x7ff4240d9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:31.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.467+0000 7ff42b59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff42402c8b0 0x7ff4240e34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:31.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.467+0000 7ff42b59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff4240477e0 0x7ff42402ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:31.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.467+0000 7ff42b59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff424049a90 con 0x7ff42402c8b0 2020-11-28T21:48:31.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.467+0000 7ff42b59e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff424049d30 con 0x7ff424006b30 2020-11-28T21:48:31.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.467+0000 7ff42b59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4240e3050 con 0x7ff4240477e0 2020-11-28T21:48:31.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.467+0000 7ff42a59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff424006b30 0x7ff4240d9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:31.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.467+0000 7ff42a59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff424006b30 0x7ff4240d9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39430/0 (socket says 172.21.15.162:39430) 2020-11-28T21:48:31.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.467+0000 7ff42a59c700 1 -- 172.21.15.162:0/1411579466 learned_addr learned my addr 172.21.15.162:0/1411579466 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:31.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.467+0000 7ff42ad9d700 1 --2- 172.21.15.162:0/1411579466 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff4240477e0 0x7ff42402ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:31.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.467+0000 7ff429d9b700 1 --2- 172.21.15.162:0/1411579466 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff42402c8b0 0x7ff4240e34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:31.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.467+0000 7ff42a59c700 1 -- 172.21.15.162:0/1411579466 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff4240477e0 msgr2=0x7ff42402ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:31.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.467+0000 7ff42a59c700 1 --2- 172.21.15.162:0/1411579466 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff4240477e0 0x7ff42402ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:31.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.467+0000 7ff42a59c700 1 -- 172.21.15.162:0/1411579466 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff42402c8b0 msgr2=0x7ff4240e34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:31.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.467+0000 7ff42a59c700 1 --2- 172.21.15.162:0/1411579466 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff42402c8b0 0x7ff4240e34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:31.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.467+0000 7ff42a59c700 1 -- 172.21.15.162:0/1411579466 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4240e2d40 con 0x7ff424006b30 2020-11-28T21:48:31.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.467+0000 7ff429d9b700 1 --2- 172.21.15.162:0/1411579466 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff42402c8b0 0x7ff4240e34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:48:31.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.467+0000 7ff42a59c700 1 --2- 172.21.15.162:0/1411579466 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff424006b30 0x7ff4240d9c90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7ff42c064000 tx=0x7ff42c05a760).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:31.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.467+0000 7ff4237fe700 1 -- 172.21.15.162:0/1411579466 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff42c05cef0 con 0x7ff424006b30 2020-11-28T21:48:31.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.467+0000 7ff4237fe700 1 -- 172.21.15.162:0/1411579466 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff42c05d050 con 0x7ff424006b30 2020-11-28T21:48:31.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.467+0000 7ff4237fe700 1 -- 172.21.15.162:0/1411579466 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff42c060420 con 0x7ff424006b30 2020-11-28T21:48:31.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.467+0000 7ff42b59e700 1 -- 172.21.15.162:0/1411579466 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff42402a730 con 0x7ff424006b30 2020-11-28T21:48:31.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.467+0000 7ff42b59e700 1 -- 172.21.15.162:0/1411579466 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff424049800 con 0x7ff424006b30 2020-11-28T21:48:31.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.467+0000 7ff42b59e700 1 -- 172.21.15.162:0/1411579466 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4240e3d20 con 0x7ff424006b30 2020-11-28T21:48:31.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.467+0000 7ff4237fe700 1 -- 172.21.15.162:0/1411579466 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff42c060580 con 0x7ff424006b30 2020-11-28T21:48:31.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.467+0000 7ff4237fe700 1 --2- 172.21.15.162:0/1411579466 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff4100553b0 0x7ff410057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:31.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.467+0000 7ff4237fe700 1 -- 172.21.15.162:0/1411579466 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff42c0dcd70 con 0x7ff424006b30 2020-11-28T21:48:31.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.467+0000 7ff429d9b700 1 --2- 172.21.15.162:0/1411579466 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff4100553b0 0x7ff410057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:31.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.467+0000 7ff429d9b700 1 --2- 172.21.15.162:0/1411579466 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff4100553b0 0x7ff410057870 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7ff414019000 tx=0x7ff414015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:31.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.475+0000 7ff4237fe700 1 -- 172.21.15.162:0/1411579466 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff42c0b3010 con 0x7ff424006b30 2020-11-28T21:48:31.656 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.655+0000 7ff42b59e700 1 -- 172.21.15.162:0/1411579466 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff424004cc0 con 0x7ff424006b30 2020-11-28T21:48:31.656 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.655+0000 7ff4237fe700 1 -- 172.21.15.162:0/1411579466 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+2497 (secure 0 0 0) 0x7ff42c0b6630 con 0x7ff424006b30 2020-11-28T21:48:31.657 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:48:31.657 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":15,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4856,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/913296056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":913296056},{"type":"v1","addr":"172.21.15.118:6833","nonce":913296056}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":13},{"gid":5209,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6835/1318110356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1318110356},{"type":"v1","addr":"172.21.15.162:6835","nonce":1318110356}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:48:26.782354+0000","modified":"2020-11-28T21:48:27.785277+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5212},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5212":{"gid":5212,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/2706248574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2706248574},{"type":"v1","addr":"172.21.15.162:6837","nonce":2706248574}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-28T21:48:31.659 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.659+0000 7ff42b59e700 1 -- 172.21.15.162:0/1411579466 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff4100553b0 msgr2=0x7ff410057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:31.659 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.659+0000 7ff42b59e700 1 --2- 172.21.15.162:0/1411579466 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff4100553b0 0x7ff410057870 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7ff414019000 tx=0x7ff414015000).stop 2020-11-28T21:48:31.659 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.659+0000 7ff42b59e700 1 -- 172.21.15.162:0/1411579466 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff424006b30 msgr2=0x7ff4240d9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:31.659 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.659+0000 7ff42b59e700 1 --2- 172.21.15.162:0/1411579466 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff424006b30 0x7ff4240d9c90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7ff42c064000 tx=0x7ff42c05a760).stop 2020-11-28T21:48:31.660 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.659+0000 7ff42b59e700 1 -- 172.21.15.162:0/1411579466 shutdown_connections 2020-11-28T21:48:31.660 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.659+0000 7ff42b59e700 1 --2- 172.21.15.162:0/1411579466 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff4100553b0 0x7ff410057870 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:31.660 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.659+0000 7ff42b59e700 1 --2- 172.21.15.162:0/1411579466 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff4240477e0 0x7ff42402ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:31.660 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.659+0000 7ff42b59e700 1 --2- 172.21.15.162:0/1411579466 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff424006b30 0x7ff4240d9c90 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:31.660 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.659+0000 7ff42b59e700 1 --2- 172.21.15.162:0/1411579466 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff42402c8b0 0x7ff4240e34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:31.661 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.659+0000 7ff42b59e700 1 -- 172.21.15.162:0/1411579466 >> 172.21.15.162:0/1411579466 conn(0x7ff42404b100 msgr2=0x7ff4240ce9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:31.661 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.659+0000 7ff42b59e700 1 -- 172.21.15.162:0/1411579466 shutdown_connections 2020-11-28T21:48:31.661 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:31.659+0000 7ff42b59e700 1 -- 172.21.15.162:0/1411579466 wait complete. 2020-11-28T21:48:31.662 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 15 2020-11-28T21:48:31.972 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-28T21:48:26.782354+0000', 'modified': '2020-11-28T21:48:27.785277+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5212}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5212': {'gid': 5212, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6837/2706248574', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6836', 'nonce': 2706248574}, {'type': 'v1', 'addr': '172.21.15.162:6837', 'nonce': 2706248574}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-28T21:48:31.973 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-28T21:48:31.973 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-28T21:48:32.101 INFO:teuthology.orchestra.run.smithi162.stdout:{"cluster_fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","whoami":0,"id":5212,"want_state":"up:active","state":"up:active","rank_uptime":5.3030532859999999,"mdsmap_epoch":15,"osdmap_epoch":44,"osdmap_epoch_barrier":43,"uptime":9.6948830479999994} 2020-11-28T21:48:32.102 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", "id": 5212, "mdsmap_epoch": 15, "osdmap_epoch": 44, "osdmap_epoch_barrier": 43, "rank_uptime": 5.303053286, "state": "up:active", "uptime": 9.694883048, "want_state": "up:active", "whoami": 0 } 2020-11-28T21:48:32.102 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-11-28T21:48:32.103 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-11-28T21:48:32.103 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi192.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-11-28T21:48:32.104 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:48:32.104 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:48:32.113 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:48:32.170 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:48:32.171 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:48:32.226 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:48:32.227 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:48:32.228 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:48:32.228 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:48:32.273 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-11-28T21:48:32.274 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:48:32.317 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:48:32.333 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:48:32.334 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:48:32.389 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:48:32.390 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:48:32.391 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:48:32.391 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:48:32.395 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:ceph-fuse[11525]: starting ceph client2020-11-28T21:48:32.393+0000 7f5f6b2560c0 -1 init, newargv = 0x55ee26e80be0 newargc=9 2020-11-28T21:48:32.395 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr: 2020-11-28T21:48:32.409 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:ceph-fuse[11525]: starting fuse 2020-11-28T21:48:33.503 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:48:33.518 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:48:33.519 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:48:33.573 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:48:33.575 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:48:33.577 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:48:33.577 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:48:33.626 INFO:teuthology.orchestra.run.smithi192.stdout:49 2020-11-28T21:48:33.626 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2020-11-28T21:48:33.627 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:48:33.627 INFO:teuthology.orchestra.run.smithi192:> sudo python3 -c ' 2020-11-28T21:48:33.628 INFO:teuthology.orchestra.run.smithi192:> import glob 2020-11-28T21:48:33.628 INFO:teuthology.orchestra.run.smithi192:> import re 2020-11-28T21:48:33.629 INFO:teuthology.orchestra.run.smithi192:> import os 2020-11-28T21:48:33.629 INFO:teuthology.orchestra.run.smithi192:> import subprocess 2020-11-28T21:48:33.629 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:48:33.630 INFO:teuthology.orchestra.run.smithi192:> def find_socket(client_name): 2020-11-28T21:48:33.630 INFO:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-11-28T21:48:33.630 INFO:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2020-11-28T21:48:33.631 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:48:33.631 INFO:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2020-11-28T21:48:33.632 INFO:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2020-11-28T21:48:33.632 INFO:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2020-11-28T21:48:33.632 INFO:teuthology.orchestra.run.smithi192:> return files[0] 2020-11-28T21:48:33.633 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:48:33.633 INFO:teuthology.orchestra.run.smithi192:> for f in files: 2020-11-28T21:48:33.634 INFO:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-11-28T21:48:33.634 INFO:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2020-11-28T21:48:33.634 INFO:teuthology.orchestra.run.smithi192:> return f 2020-11-28T21:48:33.635 INFO:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-11-28T21:48:33.635 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:48:33.635 INFO:teuthology.orchestra.run.smithi192:> print(find_socket("client.0")) 2020-11-28T21:48:33.636 INFO:teuthology.orchestra.run.smithi192:> ' 2020-11-28T21:48:33.716 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.0.11525.asok 2020-11-28T21:48:33.721 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.11525.asok 2020-11-28T21:48:33.721 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:48:33.722 INFO:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.11525.asok status 2020-11-28T21:48:33.820 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2020-11-28T21:48:33.821 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2020-11-28T21:48:33.821 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "a83a3572a957356977342f461361c1cd3ec06dbd", 2020-11-28T21:48:33.821 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 15.2.6-595-ga83a3572 (a83a3572a957356977342f461361c1cd3ec06dbd) octopus (stable)", 2020-11-28T21:48:33.821 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "0", 2020-11-28T21:48:33.822 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2020-11-28T21:48:33.822 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-11-28T21:48:33.822 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "11525", 2020-11-28T21:48:33.822 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2020-11-28T21:48:33.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:48:33.823 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2020-11-28T21:48:33.823 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2020-11-28T21:48:33.823 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 5371, 2020-11-28T21:48:33.823 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2020-11-28T21:48:33.824 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2020-11-28T21:48:33.824 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2020-11-28T21:48:33.824 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 5371 2020-11-28T21:48:33.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:48:33.824 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2020-11-28T21:48:33.825 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2020-11-28T21:48:33.825 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2020-11-28T21:48:33.825 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 606397186 2020-11-28T21:48:33.825 INFO:teuthology.orchestra.run.smithi192.stdout: } 2020-11-28T21:48:33.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:48:33.826 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2020-11-28T21:48:33.826 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2020-11-28T21:48:33.826 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2020-11-28T21:48:33.826 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 606397186 2020-11-28T21:48:33.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:48:33.827 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.5371 172.21.15.192:0/606397186", 2020-11-28T21:48:33.827 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "172.21.15.192:0/606397186", 2020-11-28T21:48:33.827 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2020-11-28T21:48:33.827 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 15, 2020-11-28T21:48:33.827 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 44, 2020-11-28T21:48:33.828 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2020-11-28T21:48:33.828 INFO:teuthology.orchestra.run.smithi192.stdout: "blacklisted": false 2020-11-28T21:48:33.828 INFO:teuthology.orchestra.run.smithi192.stdout:} 2020-11-28T21:48:33.830 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:48:33.831 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:48:33.831 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:48:33.838 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2020-11-28T21:48:33.839 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-28T21:48:33.839 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:48:33.840 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:48:33.896 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:48:33.897 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:48:33.897 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:48:33.950 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2020-11-28T21:48:33.951 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-28T21:48:33.952 INFO:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-28T21:48:33.952 INFO:teuthology.orchestra.run.smithi192:> import sys 2020-11-28T21:48:33.952 INFO:teuthology.orchestra.run.smithi192:> import time 2020-11-28T21:48:33.953 INFO:teuthology.orchestra.run.smithi192:> import os 2020-11-28T21:48:33.953 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:48:33.953 INFO:teuthology.orchestra.run.smithi192:> n = 1000 2020-11-28T21:48:33.954 INFO:teuthology.orchestra.run.smithi192:> abs_path = "/home/ubuntu/cephtest/mnt.0/dc-dir/dc-file" 2020-11-28T21:48:33.954 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:48:33.955 INFO:teuthology.orchestra.run.smithi192:> if not os.path.exists(os.path.dirname(abs_path)): 2020-11-28T21:48:33.955 INFO:teuthology.orchestra.run.smithi192:> os.makedirs(os.path.dirname(abs_path)) 2020-11-28T21:48:33.956 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:48:33.956 INFO:teuthology.orchestra.run.smithi192:> for i in range(0, n): 2020-11-28T21:48:33.956 INFO:teuthology.orchestra.run.smithi192:> fname = "{0}_{1}".format(abs_path, i) 2020-11-28T21:48:33.957 INFO:teuthology.orchestra.run.smithi192:> with open(fname, '"'"'w'"'"') as f: 2020-11-28T21:48:33.957 INFO:teuthology.orchestra.run.smithi192:> f.write('"'"'content'"'"') 2020-11-28T21:48:33.958 INFO:teuthology.orchestra.run.smithi192:> if True: 2020-11-28T21:48:33.958 INFO:teuthology.orchestra.run.smithi192:> f.flush() 2020-11-28T21:48:33.959 INFO:teuthology.orchestra.run.smithi192:> os.fsync(f.fileno()) 2020-11-28T21:48:33.959 INFO:teuthology.orchestra.run.smithi192:> ' 2020-11-28T21:48:35.411 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:48:35.420 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:48:35.435 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:48:35.457 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:48:46.677 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:48:46.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.799+0000 7fec7d940700 1 Processor -- start 2020-11-28T21:48:46.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.799+0000 7fec7d940700 1 -- start start 2020-11-28T21:48:46.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.799+0000 7fec7d940700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fec780611e0 0x7fec780615c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:46.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.803+0000 7fec7d940700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fec780b4b40 0x7fec7813ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:46.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.803+0000 7fec7d940700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec780b1790 con 0x7fec780b10f0 2020-11-28T21:48:46.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.803+0000 7fec7d940700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec780b1580 con 0x7fec780611e0 2020-11-28T21:48:46.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.803+0000 7fec7d940700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec780b1a30 con 0x7fec780b4b40 2020-11-28T21:48:46.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.803+0000 7fec7d13f700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fec780b4b40 0x7fec7813ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:46.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.803+0000 7fec7d13f700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fec780b4b40 0x7fec7813ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39444/0 (socket says 172.21.15.162:39444) 2020-11-28T21:48:46.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.803+0000 7fec7d13f700 1 -- 172.21.15.162:0/1906612923 learned_addr learned my addr 172.21.15.162:0/1906612923 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:46.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.803+0000 7fec77fff700 1 --1- 172.21.15.162:0/1906612923 >> v1:172.21.15.162:6789/0 conn(0x7fec780b10f0 0x7fec780b4430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:46286/0 (socket says 172.21.15.162:46286) 2020-11-28T21:48:46.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.803+0000 7fec7c93e700 1 --2- 172.21.15.162:0/1906612923 >> v2:172.21.15.118:3300/0 conn(0x7fec780611e0 0x7fec780615c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:46.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.803+0000 7fec7d13f700 1 -- 172.21.15.162:0/1906612923 >> v2:172.21.15.118:3300/0 conn(0x7fec780611e0 msgr2=0x7fec780615c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:46.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.803+0000 7fec7d13f700 1 --2- 172.21.15.162:0/1906612923 >> v2:172.21.15.118:3300/0 conn(0x7fec780611e0 0x7fec780615c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:46.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.803+0000 7fec7d13f700 1 -- 172.21.15.162:0/1906612923 >> v1:172.21.15.162:6789/0 conn(0x7fec780b10f0 legacy=0x7fec780b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:46.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.803+0000 7fec7d13f700 1 -- 172.21.15.162:0/1906612923 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec780b3f90 con 0x7fec780b4b40 2020-11-28T21:48:46.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.803+0000 7fec7d13f700 1 --2- 172.21.15.162:0/1906612923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec780b4b40 0x7fec7813ee60 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7fec68002f30 tx=0x7fec68006e80).ready entity=mon.1 client_cookie=5e5ad310c0441d4c server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:46.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.803+0000 7fec777fe700 1 -- 172.21.15.162:0/1906612923 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec680099e0 con 0x7fec780b4b40 2020-11-28T21:48:46.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.803+0000 7fec777fe700 1 -- 172.21.15.162:0/1906612923 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec68009b40 con 0x7fec780b4b40 2020-11-28T21:48:46.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.803+0000 7fec777fe700 1 -- 172.21.15.162:0/1906612923 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec68009ea0 con 0x7fec780b4b40 2020-11-28T21:48:46.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.803+0000 7fec7d940700 1 -- 172.21.15.162:0/1906612923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec780b4b40 msgr2=0x7fec7813ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:46.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.803+0000 7fec7d940700 1 --2- 172.21.15.162:0/1906612923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec780b4b40 0x7fec7813ee60 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7fec68002f30 tx=0x7fec68006e80).stop 2020-11-28T21:48:46.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.803+0000 7fec7d940700 1 -- 172.21.15.162:0/1906612923 shutdown_connections 2020-11-28T21:48:46.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.803+0000 7fec7d940700 1 --2- 172.21.15.162:0/1906612923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec780b4b40 0x7fec7813ee60 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:46.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.803+0000 7fec7d940700 1 --2- 172.21.15.162:0/1906612923 >> v2:172.21.15.118:3300/0 conn(0x7fec780611e0 0x7fec780615c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:46.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.803+0000 7fec7d940700 1 -- 172.21.15.162:0/1906612923 >> 172.21.15.162:0/1906612923 conn(0x7fec780b54f0 msgr2=0x7fec780b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:46.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.803+0000 7fec7d940700 1 -- 172.21.15.162:0/1906612923 shutdown_connections 2020-11-28T21:48:46.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.803+0000 7fec7d940700 1 -- 172.21.15.162:0/1906612923 wait complete. 2020-11-28T21:48:46.811 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.807+0000 7fec7d940700 1 Processor -- start 2020-11-28T21:48:46.811 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.807+0000 7fec7d940700 1 -- start start 2020-11-28T21:48:46.812 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.807+0000 7fec7d940700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fec780611e0 0x7fec780640b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:46.812 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.807+0000 7fec7d940700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec780b10f0 0x7fec7809a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:46.812 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.807+0000 7fec7d940700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fec780b4b40 0x7fec780652a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:46.812 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.807+0000 7fec7d940700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7813b2d0 con 0x7fec780611e0 2020-11-28T21:48:46.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.807+0000 7fec7d940700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7813a760 con 0x7fec780b10f0 2020-11-28T21:48:46.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.807+0000 7fec7d940700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec7813aa00 con 0x7fec780b4b40 2020-11-28T21:48:46.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.807+0000 7fec77fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec780b10f0 0x7fec7809a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:46.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.807+0000 7fec77fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec780b10f0 0x7fec7809a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39448/0 (socket says 172.21.15.162:39448) 2020-11-28T21:48:46.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.807+0000 7fec77fff700 1 -- 172.21.15.162:0/3164141993 learned_addr learned my addr 172.21.15.162:0/3164141993 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:46.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.807+0000 7fec7c93e700 1 --2- 172.21.15.162:0/3164141993 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fec780611e0 0x7fec780640b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:46.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.807+0000 7fec7d13f700 1 --2- 172.21.15.162:0/3164141993 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fec780b4b40 0x7fec780652a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:46.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.811+0000 7fec77fff700 1 -- 172.21.15.162:0/3164141993 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fec780b4b40 msgr2=0x7fec780652a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:46.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.811+0000 7fec77fff700 1 --2- 172.21.15.162:0/3164141993 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fec780b4b40 0x7fec780652a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:46.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.811+0000 7fec77fff700 1 -- 172.21.15.162:0/3164141993 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fec780611e0 msgr2=0x7fec780640b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:46.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.811+0000 7fec77fff700 1 --2- 172.21.15.162:0/3164141993 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fec780611e0 0x7fec780640b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:46.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.811+0000 7fec77fff700 1 -- 172.21.15.162:0/3164141993 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec78063970 con 0x7fec780b10f0 2020-11-28T21:48:46.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.811+0000 7fec77fff700 1 --2- 172.21.15.162:0/3164141993 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec780b10f0 0x7fec7809a0a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7fec78060310 tx=0x7fec60007bd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:46.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.811+0000 7fec75ffb700 1 -- 172.21.15.162:0/3164141993 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec600072e0 con 0x7fec780b10f0 2020-11-28T21:48:46.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.811+0000 7fec75ffb700 1 -- 172.21.15.162:0/3164141993 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec60005520 con 0x7fec780b10f0 2020-11-28T21:48:46.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.811+0000 7fec7d940700 1 -- 172.21.15.162:0/3164141993 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec78063c10 con 0x7fec780b10f0 2020-11-28T21:48:46.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.811+0000 7fec75ffb700 1 -- 172.21.15.162:0/3164141993 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec60005880 con 0x7fec780b10f0 2020-11-28T21:48:46.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.811+0000 7fec7d940700 1 -- 172.21.15.162:0/3164141993 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec7813a300 con 0x7fec780b10f0 2020-11-28T21:48:46.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.811+0000 7fec75ffb700 1 -- 172.21.15.162:0/3164141993 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fec6001a060 con 0x7fec780b10f0 2020-11-28T21:48:46.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.811+0000 7fec7d940700 1 -- 172.21.15.162:0/3164141993 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec78063da0 con 0x7fec780b10f0 2020-11-28T21:48:46.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.811+0000 7fec75ffb700 1 --2- 172.21.15.162:0/3164141993 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fec5c055440 0x7fec5c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:46.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.811+0000 7fec75ffb700 1 -- 172.21.15.162:0/3164141993 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fec6006d250 con 0x7fec780b10f0 2020-11-28T21:48:46.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.811+0000 7fec7c93e700 1 --2- 172.21.15.162:0/3164141993 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fec5c055440 0x7fec5c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:46.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.815+0000 7fec7c93e700 1 --2- 172.21.15.162:0/3164141993 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fec5c055440 0x7fec5c057900 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7fec7000b000 tx=0x7fec700091a0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:46.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.819+0000 7fec75ffb700 1 -- 172.21.15.162:0/3164141993 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fec60043550 con 0x7fec780b10f0 2020-11-28T21:48:47.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:46.999+0000 7fec7d940700 1 -- 172.21.15.162:0/3164141993 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec78057050 con 0x7fec780b10f0 2020-11-28T21:48:47.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.003+0000 7fec75ffb700 1 -- 172.21.15.162:0/3164141993 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+2497 (secure 0 0 0) 0x7fec60018020 con 0x7fec780b10f0 2020-11-28T21:48:47.002 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:48:47.002 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":15,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4856,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/913296056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":913296056},{"type":"v1","addr":"172.21.15.118:6833","nonce":913296056}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":13},{"gid":5209,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6835/1318110356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1318110356},{"type":"v1","addr":"172.21.15.162:6835","nonce":1318110356}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:48:26.782354+0000","modified":"2020-11-28T21:48:27.785277+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5212},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5212":{"gid":5212,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/2706248574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2706248574},{"type":"v1","addr":"172.21.15.162:6837","nonce":2706248574}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-28T21:48:47.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.003+0000 7fec7d940700 1 -- 172.21.15.162:0/3164141993 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fec5c055440 msgr2=0x7fec5c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:47.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.003+0000 7fec7d940700 1 --2- 172.21.15.162:0/3164141993 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fec5c055440 0x7fec5c057900 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7fec7000b000 tx=0x7fec700091a0).stop 2020-11-28T21:48:47.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.003+0000 7fec7d940700 1 -- 172.21.15.162:0/3164141993 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec780b10f0 msgr2=0x7fec7809a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:47.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.003+0000 7fec7d940700 1 --2- 172.21.15.162:0/3164141993 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec780b10f0 0x7fec7809a0a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7fec78060310 tx=0x7fec60007bd0).stop 2020-11-28T21:48:47.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.003+0000 7fec7d940700 1 -- 172.21.15.162:0/3164141993 shutdown_connections 2020-11-28T21:48:47.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.003+0000 7fec7d940700 1 --2- 172.21.15.162:0/3164141993 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fec5c055440 0x7fec5c057900 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:47.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.003+0000 7fec7d940700 1 --2- 172.21.15.162:0/3164141993 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fec780b4b40 0x7fec780652a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:47.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.003+0000 7fec7d940700 1 --2- 172.21.15.162:0/3164141993 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fec780611e0 0x7fec780640b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:47.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.003+0000 7fec7d940700 1 --2- 172.21.15.162:0/3164141993 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec780b10f0 0x7fec7809a0a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:47.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.003+0000 7fec7d940700 1 -- 172.21.15.162:0/3164141993 >> 172.21.15.162:0/3164141993 conn(0x7fec780b54f0 msgr2=0x7fec7813cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:47.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.003+0000 7fec7d940700 1 -- 172.21.15.162:0/3164141993 shutdown_connections 2020-11-28T21:48:47.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.003+0000 7fec7d940700 1 -- 172.21.15.162:0/3164141993 wait complete. 2020-11-28T21:48:47.007 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 15 2020-11-28T21:48:47.015 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_recall_max_caps 20 2020-11-28T21:48:47.134 INFO:teuthology.orchestra.run.smithi162.stdout:{"success":"mds_recall_max_caps = '20' (not observed, change may require restart) "} 2020-11-28T21:48:47.135 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_caps = '20' (not observed, change may require restart) " } 2020-11-28T21:48:47.135 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:48:47.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.247+0000 7f427220c700 1 Processor -- start 2020-11-28T21:48:47.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.247+0000 7f427220c700 1 -- start start 2020-11-28T21:48:47.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.247+0000 7f427220c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f426c098b30 0x7f426c0994f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:47.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.247+0000 7f427220c700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f426c0b0000 0x7f426c060000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:47.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.247+0000 7f427220c700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f426c098f10 con 0x7f426c0604f0 2020-11-28T21:48:47.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.247+0000 7f427220c700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f426c0b1030 con 0x7f426c098b30 2020-11-28T21:48:47.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.247+0000 7f427220c700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f426c0b0480 con 0x7f426c0b0000 2020-11-28T21:48:47.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.247+0000 7f427120a700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f426c098b30 0x7f426c0994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:47.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.247+0000 7f427120a700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f426c098b30 0x7f426c0994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.162:46442/0 (socket says 172.21.15.162:46442) 2020-11-28T21:48:47.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.247+0000 7f4271a0b700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f426c0604f0 0x7f426c14a550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.162:36450/0 (socket says 172.21.15.162:36450) 2020-11-28T21:48:47.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.247+0000 7f427120a700 1 -- 172.21.15.162:0/2568329395 learned_addr learned my addr 172.21.15.162:0/2568329395 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:47.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.247+0000 7f4270a09700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f426c0b0000 0x7f426c060000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:47.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.247+0000 7f427120a700 1 -- 172.21.15.162:0/2568329395 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f426c0b0000 msgr2=0x7f426c060000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:47.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.247+0000 7f427120a700 1 --2- 172.21.15.162:0/2568329395 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f426c0b0000 0x7f426c060000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:47.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.247+0000 7f427120a700 1 -- 172.21.15.162:0/2568329395 >> v1:172.21.15.118:6789/0 conn(0x7f426c0604f0 legacy=0x7f426c14a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:47.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.247+0000 7f427120a700 1 -- 172.21.15.162:0/2568329395 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f426c0b0680 con 0x7f426c098b30 2020-11-28T21:48:47.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.247+0000 7f427120a700 1 --2- 172.21.15.162:0/2568329395 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f426c098b30 0x7f426c0994f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f426c099be0 tx=0x7f4264009270).ready entity=mon.0 client_cookie=e683e8eaa25511df server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:47.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.247+0000 7f4263fff700 1 -- 172.21.15.162:0/2568329395 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4264009e90 con 0x7f426c098b30 2020-11-28T21:48:47.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.247+0000 7f4263fff700 1 -- 172.21.15.162:0/2568329395 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f426400bbc0 con 0x7f426c098b30 2020-11-28T21:48:47.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.247+0000 7f4263fff700 1 -- 172.21.15.162:0/2568329395 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f426400bd30 con 0x7f426c098b30 2020-11-28T21:48:47.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.247+0000 7f427220c700 1 -- 172.21.15.162:0/2568329395 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f426c098b30 msgr2=0x7f426c0994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:47.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.247+0000 7f427220c700 1 --2- 172.21.15.162:0/2568329395 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f426c098b30 0x7f426c0994f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f426c099be0 tx=0x7f4264009270).stop 2020-11-28T21:48:47.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.251+0000 7f427220c700 1 -- 172.21.15.162:0/2568329395 shutdown_connections 2020-11-28T21:48:47.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.251+0000 7f427220c700 1 --2- 172.21.15.162:0/2568329395 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f426c098b30 0x7f426c0994f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:47.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.251+0000 7f427220c700 1 --2- 172.21.15.162:0/2568329395 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f426c0b0000 0x7f426c060000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:47.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.251+0000 7f427220c700 1 -- 172.21.15.162:0/2568329395 >> 172.21.15.162:0/2568329395 conn(0x7f426c05b970 msgr2=0x7f426c05bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:47.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.251+0000 7f427220c700 1 -- 172.21.15.162:0/2568329395 shutdown_connections 2020-11-28T21:48:47.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.251+0000 7f427220c700 1 -- 172.21.15.162:0/2568329395 wait complete. 2020-11-28T21:48:47.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.251+0000 7f427220c700 1 Processor -- start 2020-11-28T21:48:47.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.251+0000 7f427220c700 1 -- start start 2020-11-28T21:48:47.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.251+0000 7f427220c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f426c0604f0 0x7f426c1481b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:47.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.251+0000 7f427220c700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f426c098b30 0x7f426c155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:47.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.251+0000 7f427220c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f426c0b0000 0x7f426c067170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:47.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.251+0000 7f427220c700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426c156150 con 0x7f426c0b0000 2020-11-28T21:48:47.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.251+0000 7f427220c700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426c0aed30 con 0x7f426c0604f0 2020-11-28T21:48:47.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.251+0000 7f427220c700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f426c0aefd0 con 0x7f426c098b30 2020-11-28T21:48:47.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.251+0000 7f427120a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f426c0604f0 0x7f426c1481b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:47.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.251+0000 7f427120a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f426c0604f0 0x7f426c1481b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39464/0 (socket says 172.21.15.162:39464) 2020-11-28T21:48:47.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.251+0000 7f4271a0b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f426c0b0000 0x7f426c067170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:47.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.251+0000 7f427120a700 1 -- 172.21.15.162:0/1552013287 learned_addr learned my addr 172.21.15.162:0/1552013287 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:47.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.251+0000 7f4270a09700 1 --2- 172.21.15.162:0/1552013287 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f426c098b30 0x7f426c155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:47.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.251+0000 7f4271a0b700 1 -- 172.21.15.162:0/1552013287 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f426c098b30 msgr2=0x7f426c155c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:47.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.251+0000 7f4271a0b700 1 --2- 172.21.15.162:0/1552013287 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f426c098b30 0x7f426c155c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:47.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.251+0000 7f4271a0b700 1 -- 172.21.15.162:0/1552013287 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f426c0604f0 msgr2=0x7f426c1481b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:47.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.251+0000 7f4271a0b700 1 --2- 172.21.15.162:0/1552013287 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f426c0604f0 0x7f426c1481b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:47.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.251+0000 7f4271a0b700 1 -- 172.21.15.162:0/1552013287 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f426c066a30 con 0x7f426c0b0000 2020-11-28T21:48:47.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.251+0000 7f427120a700 1 --2- 172.21.15.162:0/1552013287 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f426c0604f0 0x7f426c1481b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:48:47.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.251+0000 7f4271a0b700 1 --2- 172.21.15.162:0/1552013287 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f426c0b0000 0x7f426c067170 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f426c05cc70 tx=0x7f4268007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:47.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.255+0000 7f42627fc700 1 -- 172.21.15.162:0/1552013287 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f42680072e0 con 0x7f426c0b0000 2020-11-28T21:48:47.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.255+0000 7f427220c700 1 -- 172.21.15.162:0/1552013287 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f426c066cd0 con 0x7f426c0b0000 2020-11-28T21:48:47.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.255+0000 7f42627fc700 1 -- 172.21.15.162:0/1552013287 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4268005520 con 0x7f426c0b0000 2020-11-28T21:48:47.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.255+0000 7f427220c700 1 -- 172.21.15.162:0/1552013287 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f426c0ad2d0 con 0x7f426c0b0000 2020-11-28T21:48:47.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.255+0000 7f42627fc700 1 -- 172.21.15.162:0/1552013287 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4268005880 con 0x7f426c0b0000 2020-11-28T21:48:47.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.255+0000 7f427220c700 1 -- 172.21.15.162:0/1552013287 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f426c057050 con 0x7f426c0b0000 2020-11-28T21:48:47.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.255+0000 7f42627fc700 1 -- 172.21.15.162:0/1552013287 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4268010070 con 0x7f426c0b0000 2020-11-28T21:48:47.273 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.255+0000 7f42627fc700 1 --2- 172.21.15.162:0/1552013287 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f42540554e0 0x7f42540579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:47.273 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.255+0000 7f42627fc700 1 -- 172.21.15.162:0/1552013287 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f426806db20 con 0x7f426c0b0000 2020-11-28T21:48:47.273 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.255+0000 7f427120a700 1 --2- 172.21.15.162:0/1552013287 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f42540554e0 0x7f42540579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:47.274 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.255+0000 7f427120a700 1 --2- 172.21.15.162:0/1552013287 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f42540554e0 0x7f42540579a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f426401a000 tx=0x7f4264016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:47.275 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.267+0000 7f42627fc700 1 -- 172.21.15.162:0/1552013287 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4268043e20 con 0x7f426c0b0000 2020-11-28T21:48:47.454 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.455+0000 7f427220c700 1 -- 172.21.15.162:0/1552013287 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f426c068350 con 0x7f426c0b0000 2020-11-28T21:48:47.460 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.455+0000 7f42627fc700 1 -- 172.21.15.162:0/1552013287 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+2497 (secure 0 0 0) 0x7f4268047440 con 0x7f426c0b0000 2020-11-28T21:48:47.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.455+0000 7f427220c700 1 -- 172.21.15.162:0/1552013287 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f42540554e0 msgr2=0x7f42540579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:47.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.455+0000 7f427220c700 1 --2- 172.21.15.162:0/1552013287 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f42540554e0 0x7f42540579a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f426401a000 tx=0x7f4264016000).stop 2020-11-28T21:48:47.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.455+0000 7f427220c700 1 -- 172.21.15.162:0/1552013287 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f426c0b0000 msgr2=0x7f426c067170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:47.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.455+0000 7f427220c700 1 --2- 172.21.15.162:0/1552013287 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f426c0b0000 0x7f426c067170 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f426c05cc70 tx=0x7f4268007ad0).stop 2020-11-28T21:48:47.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.455+0000 7f427220c700 1 -- 172.21.15.162:0/1552013287 shutdown_connections 2020-11-28T21:48:47.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.455+0000 7f427220c700 1 --2- 172.21.15.162:0/1552013287 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f42540554e0 0x7f42540579a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:47.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.455+0000 7f427220c700 1 --2- 172.21.15.162:0/1552013287 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f426c0b0000 0x7f426c067170 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:47.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.455+0000 7f427220c700 1 --2- 172.21.15.162:0/1552013287 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f426c0604f0 0x7f426c1481b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:47.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.455+0000 7f427220c700 1 --2- 172.21.15.162:0/1552013287 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f426c098b30 0x7f426c155c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:47.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.455+0000 7f427220c700 1 -- 172.21.15.162:0/1552013287 >> 172.21.15.162:0/1552013287 conn(0x7f426c05b970 msgr2=0x7f426c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:47.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.455+0000 7f427220c700 1 -- 172.21.15.162:0/1552013287 shutdown_connections 2020-11-28T21:48:47.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.455+0000 7f427220c700 1 -- 172.21.15.162:0/1552013287 wait complete. 2020-11-28T21:48:47.463 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 15 2020-11-28T21:48:47.464 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:48:47.464 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":15,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4856,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/913296056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":913296056},{"type":"v1","addr":"172.21.15.118:6833","nonce":913296056}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":13},{"gid":5209,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6835/1318110356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1318110356},{"type":"v1","addr":"172.21.15.162:6835","nonce":1318110356}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:48:26.782354+0000","modified":"2020-11-28T21:48:27.785277+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5212},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5212":{"gid":5212,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/2706248574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2706248574},{"type":"v1","addr":"172.21.15.162:6837","nonce":2706248574}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-28T21:48:47.469 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_recall_max_decay_threshold 400 2020-11-28T21:48:47.584 INFO:teuthology.orchestra.run.smithi162.stdout:{"success":"mds_recall_max_decay_threshold = '400' (not observed, change may require restart) "} 2020-11-28T21:48:47.585 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_decay_threshold = '400' (not observed, change may require restart) " } 2020-11-28T21:48:47.585 INFO:tasks.cephfs.fuse_mount:Killing ceph-fuse connection on ubuntu@smithi192.front.sepia.ceph.com... 2020-11-28T21:48:47.603 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:48:47.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.719+0000 7f69812ee700 1 Processor -- start 2020-11-28T21:48:47.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.719+0000 7f69812ee700 1 -- start start 2020-11-28T21:48:47.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.723+0000 7f69812ee700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f697c0603d0 0x7f697c0b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:47.723 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.723+0000 7f69812ee700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f697c0b4110 con 0x7f697c142a00 2020-11-28T21:48:47.723 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.723+0000 7f69812ee700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f697c0b43f0 con 0x7f697c0b4c70 2020-11-28T21:48:47.723 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.723+0000 7f69812ee700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f697c060f60 con 0x7f697c0603d0 2020-11-28T21:48:47.723 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.723+0000 7f697bfff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f697c0603d0 0x7f697c0b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:47.723 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.723+0000 7f697bfff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f697c0603d0 0x7f697c0b3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39468/0 (socket says 172.21.15.162:39468) 2020-11-28T21:48:47.724 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.723+0000 7f697bfff700 1 -- 172.21.15.162:0/2807488871 learned_addr learned my addr 172.21.15.162:0/2807488871 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:47.724 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.723+0000 7f697bfff700 1 -- 172.21.15.162:0/2807488871 >> v1:172.21.15.162:6789/0 conn(0x7f697c0b4c70 legacy=0x7f697c143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:47.724 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.723+0000 7f697bfff700 1 -- 172.21.15.162:0/2807488871 >> v1:172.21.15.118:6789/0 conn(0x7f697c142a00 legacy=0x7f697c09cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:47.724 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.723+0000 7f697bfff700 1 -- 172.21.15.162:0/2807488871 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f697c0b4690 con 0x7f697c0603d0 2020-11-28T21:48:47.725 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.723+0000 7f697bfff700 1 --2- 172.21.15.162:0/2807488871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f697c0603d0 0x7f697c0b3a70 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f696c002f30 tx=0x7f696c006e80).ready entity=mon.1 client_cookie=590e23857a44f709 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:47.725 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.727+0000 7f697b7fe700 1 -- 172.21.15.162:0/2807488871 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f696c0097f0 con 0x7f697c0603d0 2020-11-28T21:48:47.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.727+0000 7f697b7fe700 1 -- 172.21.15.162:0/2807488871 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f696c009950 con 0x7f697c0603d0 2020-11-28T21:48:47.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.727+0000 7f697b7fe700 1 -- 172.21.15.162:0/2807488871 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f696c009ac0 con 0x7f697c0603d0 2020-11-28T21:48:47.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.727+0000 7f69812ee700 1 -- 172.21.15.162:0/2807488871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f697c0603d0 msgr2=0x7f697c0b3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:47.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.727+0000 7f69812ee700 1 --2- 172.21.15.162:0/2807488871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f697c0603d0 0x7f697c0b3a70 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f696c002f30 tx=0x7f696c006e80).stop 2020-11-28T21:48:47.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.727+0000 7f69812ee700 1 -- 172.21.15.162:0/2807488871 shutdown_connections 2020-11-28T21:48:47.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.727+0000 7f69812ee700 1 --2- 172.21.15.162:0/2807488871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f697c0603d0 0x7f697c0b3a70 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:47.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.727+0000 7f69812ee700 1 -- 172.21.15.162:0/2807488871 >> 172.21.15.162:0/2807488871 conn(0x7f697c066a40 msgr2=0x7f697c066e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:47.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.727+0000 7f69812ee700 1 -- 172.21.15.162:0/2807488871 shutdown_connections 2020-11-28T21:48:47.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.727+0000 7f69812ee700 1 -- 172.21.15.162:0/2807488871 wait complete. 2020-11-28T21:48:47.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.731+0000 7f69812ee700 1 Processor -- start 2020-11-28T21:48:47.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.731+0000 7f69812ee700 1 -- start start 2020-11-28T21:48:47.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.731+0000 7f69812ee700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f697c0603d0 0x7f697c152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:47.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.731+0000 7f69812ee700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f697c0b4c70 0x7f697c0b7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:47.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.731+0000 7f69812ee700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f697c154db0 0x7f697c155220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:47.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.731+0000 7f69812ee700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c0b85f0 con 0x7f697c0603d0 2020-11-28T21:48:47.733 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.731+0000 7f69812ee700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c1527e0 con 0x7f697c0b4c70 2020-11-28T21:48:47.733 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.731+0000 7f69812ee700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f697c0b9190 con 0x7f697c154db0 2020-11-28T21:48:47.733 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.731+0000 7f69737fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f697c0b4c70 0x7f697c0b7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:47.734 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.731+0000 7f69737fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f697c0b4c70 0x7f697c0b7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39478/0 (socket says 172.21.15.162:39478) 2020-11-28T21:48:47.734 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.731+0000 7f69737fe700 1 -- 172.21.15.162:0/3525497753 learned_addr learned my addr 172.21.15.162:0/3525497753 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:47.735 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.731+0000 7f697bfff700 1 --2- 172.21.15.162:0/3525497753 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f697c0603d0 0x7f697c152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:47.735 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.731+0000 7f6980aed700 1 --2- 172.21.15.162:0/3525497753 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f697c154db0 0x7f697c155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:47.735 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.731+0000 7f69737fe700 1 -- 172.21.15.162:0/3525497753 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f697c154db0 msgr2=0x7f697c155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:47.735 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.731+0000 7f69737fe700 1 --2- 172.21.15.162:0/3525497753 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f697c154db0 0x7f697c155220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:47.736 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.731+0000 7f69737fe700 1 -- 172.21.15.162:0/3525497753 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f697c0603d0 msgr2=0x7f697c152220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:47.736 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.731+0000 7f69737fe700 1 --2- 172.21.15.162:0/3525497753 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f697c0603d0 0x7f697c152220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:47.736 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.731+0000 7f69737fe700 1 -- 172.21.15.162:0/3525497753 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f697c0b8c50 con 0x7f697c0b4c70 2020-11-28T21:48:47.736 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.731+0000 7f69737fe700 1 --2- 172.21.15.162:0/3525497753 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f697c0b4c70 0x7f697c0b7fa0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7f697c0ae550 tx=0x7f6964007c20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:47.738 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.735+0000 7f6979ffb700 1 -- 172.21.15.162:0/3525497753 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6964013410 con 0x7f697c0b4c70 2020-11-28T21:48:47.738 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.735+0000 7f69812ee700 1 -- 172.21.15.162:0/3525497753 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f697c0b8ef0 con 0x7f697c0b4c70 2020-11-28T21:48:47.738 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.735+0000 7f69812ee700 1 -- 172.21.15.162:0/3525497753 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f697c0b31f0 con 0x7f697c0b4c70 2020-11-28T21:48:47.740 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.735+0000 7f6979ffb700 1 -- 172.21.15.162:0/3525497753 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6964013570 con 0x7f697c0b4c70 2020-11-28T21:48:47.740 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.735+0000 7f6979ffb700 1 -- 172.21.15.162:0/3525497753 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69640136e0 con 0x7f697c0b4c70 2020-11-28T21:48:47.741 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.735+0000 7f69812ee700 1 -- 172.21.15.162:0/3525497753 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f697c0ae360 con 0x7f697c0b4c70 2020-11-28T21:48:47.741 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.735+0000 7f6979ffb700 1 -- 172.21.15.162:0/3525497753 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6964019020 con 0x7f697c0b4c70 2020-11-28T21:48:47.742 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.735+0000 7f6979ffb700 1 --2- 172.21.15.162:0/3525497753 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f69600554e0 0x7f69600579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:47.742 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.735+0000 7f6979ffb700 1 -- 172.21.15.162:0/3525497753 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f696406d7d0 con 0x7f697c0b4c70 2020-11-28T21:48:47.742 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.735+0000 7f697bfff700 1 --2- 172.21.15.162:0/3525497753 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f69600554e0 0x7f69600579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:47.742 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.735+0000 7f697bfff700 1 --2- 172.21.15.162:0/3525497753 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f69600554e0 0x7f69600579a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f696c01b000 tx=0x7f696c016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:47.743 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.743+0000 7f6979ffb700 1 -- 172.21.15.162:0/3525497753 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6964043a30 con 0x7f697c0b4c70 2020-11-28T21:48:47.931 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.931+0000 7f69812ee700 1 -- 172.21.15.162:0/3525497753 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f697c057050 con 0x7f697c0b4c70 2020-11-28T21:48:47.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.931+0000 7f6979ffb700 1 -- 172.21.15.162:0/3525497753 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+2497 (secure 0 0 0) 0x7f6964047050 con 0x7f697c0b4c70 2020-11-28T21:48:47.932 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:48:47.932 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":15,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4856,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/913296056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":913296056},{"type":"v1","addr":"172.21.15.118:6833","nonce":913296056}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":13},{"gid":5209,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6835/1318110356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1318110356},{"type":"v1","addr":"172.21.15.162:6835","nonce":1318110356}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:48:26.782354+0000","modified":"2020-11-28T21:48:27.785277+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5212},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5212":{"gid":5212,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/2706248574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2706248574},{"type":"v1","addr":"172.21.15.162:6837","nonce":2706248574}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-28T21:48:47.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.935+0000 7f69812ee700 1 -- 172.21.15.162:0/3525497753 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f69600554e0 msgr2=0x7f69600579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:47.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.935+0000 7f69812ee700 1 --2- 172.21.15.162:0/3525497753 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f69600554e0 0x7f69600579a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f696c01b000 tx=0x7f696c016000).stop 2020-11-28T21:48:47.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.935+0000 7f69812ee700 1 -- 172.21.15.162:0/3525497753 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f697c0b4c70 msgr2=0x7f697c0b7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:47.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.935+0000 7f69812ee700 1 --2- 172.21.15.162:0/3525497753 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f697c0b4c70 0x7f697c0b7fa0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7f697c0ae550 tx=0x7f6964007c20).stop 2020-11-28T21:48:47.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.935+0000 7f69812ee700 1 -- 172.21.15.162:0/3525497753 shutdown_connections 2020-11-28T21:48:47.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.935+0000 7f69812ee700 1 --2- 172.21.15.162:0/3525497753 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f69600554e0 0x7f69600579a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:47.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.935+0000 7f69812ee700 1 --2- 172.21.15.162:0/3525497753 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f697c154db0 0x7f697c155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:47.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.935+0000 7f69812ee700 1 --2- 172.21.15.162:0/3525497753 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f697c0603d0 0x7f697c152220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:47.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.935+0000 7f69812ee700 1 --2- 172.21.15.162:0/3525497753 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f697c0b4c70 0x7f697c0b7fa0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:47.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.935+0000 7f69812ee700 1 -- 172.21.15.162:0/3525497753 >> 172.21.15.162:0/3525497753 conn(0x7f697c066a40 msgr2=0x7f697c09cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:47.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.935+0000 7f69812ee700 1 -- 172.21.15.162:0/3525497753 shutdown_connections 2020-11-28T21:48:47.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:47.935+0000 7f69812ee700 1 -- 172.21.15.162:0/3525497753 wait complete. 2020-11-28T21:48:47.937 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 15 2020-11-28T21:48:47.946 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell mds.a cache drop 2020-11-28T21:48:48.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.063+0000 7f01f359e700 1 Processor -- start 2020-11-28T21:48:48.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.063+0000 7f01f359e700 1 -- start start 2020-11-28T21:48:48.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.067+0000 7f01f359e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f01f405c1e0 0x7f01f40b1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:48.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.067+0000 7f01f359e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01f40b22f0 con 0x7f01f4060900 2020-11-28T21:48:48.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.067+0000 7f01f359e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01f4060ce0 con 0x7f01f405c840 2020-11-28T21:48:48.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.067+0000 7f01f359e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01f4146360 con 0x7f01f405c1e0 2020-11-28T21:48:48.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.067+0000 7f01f2d9d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f01f405c1e0 0x7f01f40b1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:48.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.067+0000 7f01f2d9d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f01f405c1e0 0x7f01f40b1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39486/0 (socket says 172.21.15.162:39486) 2020-11-28T21:48:48.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.067+0000 7f01f2d9d700 1 -- 172.21.15.162:0/290983843 learned_addr learned my addr 172.21.15.162:0/290983843 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:48.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.067+0000 7f01ebfff700 1 --1- 172.21.15.162:0/290983843 >> v1:172.21.15.162:6789/0 conn(0x7f01f405c840 0x7f01f4142a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:46326/0 (socket says 172.21.15.162:46326) 2020-11-28T21:48:48.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.067+0000 7f01f2d9d700 1 -- 172.21.15.162:0/290983843 >> v1:172.21.15.162:6789/0 conn(0x7f01f405c840 legacy=0x7f01f4142a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:48.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.067+0000 7f01f2d9d700 1 -- 172.21.15.162:0/290983843 >> v1:172.21.15.118:6789/0 conn(0x7f01f4060900 legacy=0x7f01f40b16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:48.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.067+0000 7f01f2d9d700 1 -- 172.21.15.162:0/290983843 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f4146600 con 0x7f01f405c1e0 2020-11-28T21:48:48.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.067+0000 7f01f2d9d700 1 --2- 172.21.15.162:0/290983843 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01f405c1e0 0x7f01f40b1e00 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f01ec0032b0 tx=0x7f01ec009270).ready entity=mon.1 client_cookie=41c657b1823677e0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:48.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.067+0000 7f01f1d9b700 1 -- 172.21.15.162:0/290983843 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01ec00bc00 con 0x7f01f405c1e0 2020-11-28T21:48:48.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.067+0000 7f01f1d9b700 1 -- 172.21.15.162:0/290983843 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ec00bd60 con 0x7f01f405c1e0 2020-11-28T21:48:48.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.067+0000 7f01f359e700 1 -- 172.21.15.162:0/290983843 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01f405c1e0 msgr2=0x7f01f40b1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:48.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.067+0000 7f01f359e700 1 --2- 172.21.15.162:0/290983843 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01f405c1e0 0x7f01f40b1e00 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f01ec0032b0 tx=0x7f01ec009270).stop 2020-11-28T21:48:48.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.067+0000 7f01f359e700 1 -- 172.21.15.162:0/290983843 shutdown_connections 2020-11-28T21:48:48.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.067+0000 7f01f359e700 1 --2- 172.21.15.162:0/290983843 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01f405c1e0 0x7f01f40b1e00 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:48.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.067+0000 7f01f359e700 1 -- 172.21.15.162:0/290983843 >> 172.21.15.162:0/290983843 conn(0x7f01f406aa70 msgr2=0x7f01f4066af0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:48.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.067+0000 7f01f359e700 1 -- 172.21.15.162:0/290983843 shutdown_connections 2020-11-28T21:48:48.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.067+0000 7f01f359e700 1 -- 172.21.15.162:0/290983843 wait complete. 2020-11-28T21:48:48.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.067+0000 7f01f359e700 1 Processor -- start 2020-11-28T21:48:48.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.067+0000 7f01f359e700 1 -- start start 2020-11-28T21:48:48.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.071+0000 7f01f359e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01f405c1e0 0x7f01f412cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:48.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.071+0000 7f01f359e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f01f405c840 0x7f01f41551d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:48.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.071+0000 7f01f359e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01f4060900 0x7f01f41556c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:48.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.071+0000 7f01f359e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f4155bb0 con 0x7f01f405c840 2020-11-28T21:48:48.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.071+0000 7f01f359e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f41422f0 con 0x7f01f405c1e0 2020-11-28T21:48:48.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.071+0000 7f01f359e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f4142d60 con 0x7f01f4060900 2020-11-28T21:48:48.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.071+0000 7f01f259c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01f405c1e0 0x7f01f412cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:48.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.071+0000 7f01f259c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01f405c1e0 0x7f01f412cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39488/0 (socket says 172.21.15.162:39488) 2020-11-28T21:48:48.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.071+0000 7f01f259c700 1 -- 172.21.15.162:0/1429230584 learned_addr learned my addr 172.21.15.162:0/1429230584 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:48.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.071+0000 7f01ebfff700 1 --2- 172.21.15.162:0/1429230584 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f01f405c840 0x7f01f41551d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:48.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.071+0000 7f01f2d9d700 1 --2- 172.21.15.162:0/1429230584 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01f4060900 0x7f01f41556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:48.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.071+0000 7f01f259c700 1 -- 172.21.15.162:0/1429230584 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01f4060900 msgr2=0x7f01f41556c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:48.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.071+0000 7f01f259c700 1 --2- 172.21.15.162:0/1429230584 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01f4060900 0x7f01f41556c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:48.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.071+0000 7f01f259c700 1 -- 172.21.15.162:0/1429230584 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f01f405c840 msgr2=0x7f01f41551d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:48.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.071+0000 7f01f259c700 1 --2- 172.21.15.162:0/1429230584 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f01f405c840 0x7f01f41551d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:48.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.071+0000 7f01f259c700 1 -- 172.21.15.162:0/1429230584 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f40adab0 con 0x7f01f405c1e0 2020-11-28T21:48:48.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.071+0000 7f01ebfff700 1 --2- 172.21.15.162:0/1429230584 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f01f405c840 0x7f01f41551d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:48:48.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.071+0000 7f01f2d9d700 1 --2- 172.21.15.162:0/1429230584 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01f4060900 0x7f01f41556c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:48:48.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.071+0000 7f01f259c700 1 --2- 172.21.15.162:0/1429230584 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01f405c1e0 0x7f01f412cbf0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f01f4155cf0 tx=0x7f01e4007c20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:48.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.071+0000 7f01eb7fe700 1 -- 172.21.15.162:0/1429230584 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01e400e4a0 con 0x7f01f405c1e0 2020-11-28T21:48:48.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.071+0000 7f01f359e700 1 -- 172.21.15.162:0/1429230584 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f414a0f0 con 0x7f01f405c1e0 2020-11-28T21:48:48.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.071+0000 7f01f359e700 1 -- 172.21.15.162:0/1429230584 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f4142850 con 0x7f01f405c1e0 2020-11-28T21:48:48.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.071+0000 7f01eb7fe700 1 -- 172.21.15.162:0/1429230584 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e400e600 con 0x7f01f405c1e0 2020-11-28T21:48:48.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.071+0000 7f01eb7fe700 1 -- 172.21.15.162:0/1429230584 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01e400e7b0 con 0x7f01f405c1e0 2020-11-28T21:48:48.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.075+0000 7f01eb7fe700 1 -- 172.21.15.162:0/1429230584 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f01e4019020 con 0x7f01f405c1e0 2020-11-28T21:48:48.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.075+0000 7f01eb7fe700 1 --2- 172.21.15.162:0/1429230584 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f01d8055350 0x7f01d8057810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:48.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.075+0000 7f01eb7fe700 1 -- 172.21.15.162:0/1429230584 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f01e406d9a0 con 0x7f01f405c1e0 2020-11-28T21:48:48.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.075+0000 7f01ebfff700 1 --2- 172.21.15.162:0/1429230584 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f01d8055350 0x7f01d8057810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:48.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.075+0000 7f01ebfff700 1 --2- 172.21.15.162:0/1429230584 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f01d8055350 0x7f01d8057810 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f01dc00b000 tx=0x7f01dc000b80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:48.083 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.075+0000 7f02074dc740 1 Processor -- start 2020-11-28T21:48:48.083 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.075+0000 7f02074dc740 1 -- start start 2020-11-28T21:48:48.083 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.079+0000 7f02074dc740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x1a6ff00 0x1a9e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:48.083 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.079+0000 7f02074dc740 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x18d8800 0x1af2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:48.083 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.079+0000 7f02074dc740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x186ac00 0x1af6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:48.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.079+0000 7f02074dc740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x19e4dd0 con 0x1a6ff00 2020-11-28T21:48:48.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.079+0000 7f02074dc740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x1975230 con 0x186ac00 2020-11-28T21:48:48.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.079+0000 7f02074dc740 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x18f1690 con 0x18d8800 2020-11-28T21:48:48.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.079+0000 7f01f2d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x186ac00 0x1af6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:48.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.079+0000 7f01f2d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x186ac00 0x1af6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39500/0 (socket says 172.21.15.162:39500) 2020-11-28T21:48:48.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.079+0000 7f01f2d9d700 1 -- 172.21.15.162:0/304165349 learned_addr learned my addr 172.21.15.162:0/304165349 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:48.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.079+0000 7f01ebfff700 1 --2- 172.21.15.162:0/304165349 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x18d8800 0x1af2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:48.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.079+0000 7f01f259c700 1 --2- 172.21.15.162:0/304165349 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x1a6ff00 0x1a9e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:48.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.079+0000 7f01f2d9d700 1 -- 172.21.15.162:0/304165349 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x18d8800 msgr2=0x1af2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:48.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.079+0000 7f01f2d9d700 1 --2- 172.21.15.162:0/304165349 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x18d8800 0x1af2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:48.086 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.079+0000 7f01f2d9d700 1 -- 172.21.15.162:0/304165349 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x1a6ff00 msgr2=0x1a9e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:48.086 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.079+0000 7f01f2d9d700 1 --2- 172.21.15.162:0/304165349 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x1a6ff00 0x1a9e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:48.086 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.079+0000 7f01f2d9d700 1 -- 172.21.15.162:0/304165349 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1a05cf0 con 0x186ac00 2020-11-28T21:48:48.086 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.079+0000 7f01f259c700 1 --2- 172.21.15.162:0/304165349 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x1a6ff00 0x1a9e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:48:48.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.079+0000 7f01f2d9d700 1 --2- 172.21.15.162:0/304165349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x186ac00 0x1af6eb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x1aed320 tx=0x7f01ec016000).ready entity=mon.1 client_cookie=83f4823c4354edf server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:48.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.079+0000 7f01f359e700 1 -- 172.21.15.162:0/304165349 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01ec00bcd0 con 0x186ac00 2020-11-28T21:48:48.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.079+0000 7f01f359e700 1 -- 172.21.15.162:0/304165349 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ec01f670 con 0x186ac00 2020-11-28T21:48:48.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.079+0000 7f01f359e700 1 -- 172.21.15.162:0/304165349 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01ec01f7e0 con 0x186ac00 2020-11-28T21:48:48.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.079+0000 7f02074dc740 1 -- 172.21.15.162:0/304165349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x186ac00 msgr2=0x1af6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:48.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.079+0000 7f02074dc740 1 --2- 172.21.15.162:0/304165349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x186ac00 0x1af6eb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x1aed320 tx=0x7f01ec016000).stop 2020-11-28T21:48:48.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.079+0000 7f02074dc740 1 -- 172.21.15.162:0/304165349 shutdown_connections 2020-11-28T21:48:48.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.079+0000 7f02074dc740 1 --2- 172.21.15.162:0/304165349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x186ac00 0x1af6eb0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:48.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.079+0000 7f02074dc740 1 --2- 172.21.15.162:0/304165349 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x1a6ff00 0x1a9e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:48.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.079+0000 7f02074dc740 1 --2- 172.21.15.162:0/304165349 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x18d8800 0x1af2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:48.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.079+0000 7f02074dc740 1 -- 172.21.15.162:0/304165349 >> 172.21.15.162:0/304165349 conn(0x1a9f820 msgr2=0x1abb2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:48.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.079+0000 7f02074dc740 1 -- 172.21.15.162:0/304165349 shutdown_connections 2020-11-28T21:48:48.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.079+0000 7f02074dc740 1 -- 172.21.15.162:0/304165349 wait complete. 2020-11-28T21:48:48.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.079+0000 7f02074dc740 1 -- start start 2020-11-28T21:48:48.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.079+0000 7f02074dc740 1 Processor -- start 2020-11-28T21:48:48.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.083+0000 7f02074dc740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x186ac00 0x1abe4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:48.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.083+0000 7f02074dc740 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x18d8800 0x1af4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:48.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.083+0000 7f02074dc740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x1a6ff00 0x1af2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:48.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.083+0000 7f02074dc740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a14a50 con 0x186ac00 2020-11-28T21:48:48.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.083+0000 7f02074dc740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a14b90 con 0x1a6ff00 2020-11-28T21:48:48.094 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.083+0000 7f02074dc740 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x1a16160 con 0x18d8800 2020-11-28T21:48:48.094 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.083+0000 7f02074dc740 10 client.0 ll_register_callbacks cb 0x1a9f780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-28T21:48:48.094 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.083+0000 7f01f2d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x1a6ff00 0x1af2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:48.094 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.083+0000 7f01f2d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x1a6ff00 0x1af2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39506/0 (socket says 172.21.15.162:39506) 2020-11-28T21:48:48.094 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.083+0000 7f01f2d9d700 1 -- 172.21.15.162:0/1522223476 learned_addr learned my addr 172.21.15.162:0/1522223476 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:48.095 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.083+0000 7f01ebfff700 1 --2- 172.21.15.162:0/1522223476 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x18d8800 0x1af4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:48.095 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.083+0000 7f01f259c700 1 --2- 172.21.15.162:0/1522223476 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x186ac00 0x1abe4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:48.095 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.083+0000 7f01f2d9d700 1 -- 172.21.15.162:0/1522223476 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x18d8800 msgr2=0x1af4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:48.095 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.083+0000 7f01f2d9d700 1 --2- 172.21.15.162:0/1522223476 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x18d8800 0x1af4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:48.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.083+0000 7f01f2d9d700 1 -- 172.21.15.162:0/1522223476 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x186ac00 msgr2=0x1abe4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:48.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.083+0000 7f01f2d9d700 1 --2- 172.21.15.162:0/1522223476 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x186ac00 0x1abe4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:48.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.083+0000 7f01f2d9d700 1 -- 172.21.15.162:0/1522223476 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1888770 con 0x1a6ff00 2020-11-28T21:48:48.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.083+0000 7f01f2d9d700 1 --2- 172.21.15.162:0/1522223476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x1a6ff00 0x1af2600 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x1aed320 tx=0x7f01ec01e000).ready entity=mon.1 client_cookie=77854cf442eb1f05 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:48.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.083+0000 7f01d17fa700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2020-11-28T21:48:48.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.083+0000 7f01d17fa700 1 -- 172.21.15.162:0/1522223476 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01ec01fc70 con 0x1a6ff00 2020-11-28T21:48:48.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.083+0000 7f01d17fa700 1 -- 172.21.15.162:0/1522223476 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f01ec01fdd0 con 0x1a6ff00 2020-11-28T21:48:48.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.083+0000 7f02074dc740 1 -- 172.21.15.162:0/1522223476 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x19c57f0 con 0x1a6ff00 2020-11-28T21:48:48.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.083+0000 7f01d17fa700 1 -- 172.21.15.162:0/1522223476 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01ec006040 con 0x1a6ff00 2020-11-28T21:48:48.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.083+0000 7f02074dc740 10 client.5404 fetch_fsmap learned FSMap version 15 2020-11-28T21:48:48.098 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.083+0000 7f02074dc740 1 -- 172.21.15.162:0/1522223476 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x1893e50 con 0x1a6ff00 2020-11-28T21:48:48.098 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.083+0000 7f01d17fa700 1 -- 172.21.15.162:0/1522223476 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x19c57f0 con 0x1a6ff00 2020-11-28T21:48:48.098 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.083+0000 7f01d17fa700 1 -- 172.21.15.162:0/1522223476 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 15) v1 ==== 1434+0+0 (secure 0 0 0) 0x7f01ec0067b0 con 0x1a6ff00 2020-11-28T21:48:48.098 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.083+0000 7f02074dc740 10 client.5404 fetch_fsmap finished waiting for FSMap version 15 2020-11-28T21:48:48.098 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.083+0000 7f02074dc740 10 client.5404 resolve_mds: resolved ID 'a' to GID 5212 2020-11-28T21:48:48.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.083+0000 7f02074dc740 20 client.5404 populate_metadata read hostname 'smithi162' 2020-11-28T21:48:48.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.083+0000 7f02074dc740 1 --2- 172.21.15.162:0/1522223476 >> [v2:172.21.15.162:6836/2706248574,v1:172.21.15.162:6837/2706248574] conn(0x1af1d70 0x1b01850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:48.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.083+0000 7f02074dc740 4 client.5404 mds_command: new command op to 5212 tid=0[{"prefix": "get_command_descriptions"}] 2020-11-28T21:48:48.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.083+0000 7f02074dc740 1 -- 172.21.15.162:0/1522223476 --> [v2:172.21.15.162:6836/2706248574,v1:172.21.15.162:6837/2706248574] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x18fd8b0 con 0x1af1d70 2020-11-28T21:48:48.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.083+0000 7f01f259c700 1 --2- 172.21.15.162:0/1522223476 >> [v2:172.21.15.162:6836/2706248574,v1:172.21.15.162:6837/2706248574] conn(0x1af1d70 0x1b01850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:48.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.083+0000 7f01f259c700 1 --2- 172.21.15.162:0/1522223476 >> [v2:172.21.15.162:6836/2706248574,v1:172.21.15.162:6837/2706248574] conn(0x1af1d70 0x1b01850 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=9a42dd39ce8b0a8b server_cookie=6f466af9331c604 in_seq=0 out_seq=0 2020-11-28T21:48:48.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.083+0000 7f01d17fa700 10 client.5404 ms_handle_connect on v2:172.21.15.162:6836/2706248574 2020-11-28T21:48:48.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.087+0000 7f01d17fa700 1 -- 172.21.15.162:0/1522223476 <== mds.0 v2:172.21.15.162:6836/2706248574 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16606 (crc 0 0 0) 0x18fd8b0 con 0x1af1d70 2020-11-28T21:48:48.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.087+0000 7f01d17fa700 10 client.5404 handle_command_reply: tid=0 2020-11-28T21:48:48.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.087+0000 7f02074dc740 1 client.5404 shutdown 2020-11-28T21:48:48.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.087+0000 7f02074dc740 1 -- 172.21.15.162:0/1522223476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x1a6ff00 msgr2=0x1af2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:48.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.087+0000 7f02074dc740 1 --2- 172.21.15.162:0/1522223476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x1a6ff00 0x1af2600 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x1aed320 tx=0x7f01ec01e000).stop 2020-11-28T21:48:48.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.087+0000 7f02074dc740 1 -- 172.21.15.162:0/1522223476 shutdown_connections 2020-11-28T21:48:48.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.087+0000 7f02074dc740 1 --2- 172.21.15.162:0/1522223476 >> [v2:172.21.15.162:6836/2706248574,v1:172.21.15.162:6837/2706248574] conn(0x1af1d70 0x1b01850 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:48.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.087+0000 7f02074dc740 1 --2- 172.21.15.162:0/1522223476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x1a6ff00 0x1af2600 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:48.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.087+0000 7f02074dc740 1 --2- 172.21.15.162:0/1522223476 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x186ac00 0x1abe4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:48.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.087+0000 7f01d17fa700 0 client.5404 ms_handle_reset on v2:172.21.15.162:6836/2706248574 2020-11-28T21:48:48.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.087+0000 7f02074dc740 1 --2- 172.21.15.162:0/1522223476 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x18d8800 0x1af4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:48.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.087+0000 7f02074dc740 1 -- 172.21.15.162:0/1522223476 >> 172.21.15.162:0/1522223476 conn(0x1a9f820 msgr2=0x1abb2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:48.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.087+0000 7f02074dc740 1 -- 172.21.15.162:0/1522223476 shutdown_connections 2020-11-28T21:48:48.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.087+0000 7f02074dc740 1 -- 172.21.15.162:0/1522223476 wait complete. 2020-11-28T21:48:48.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.087+0000 7f02074dc740 20 client.5404 trim_cache size 0 max 16384 2020-11-28T21:48:48.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.103+0000 7f02074dc740 1 Processor -- start 2020-11-28T21:48:48.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.103+0000 7f02074dc740 1 -- start start 2020-11-28T21:48:48.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.103+0000 7f02074dc740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x186ac00 0x1aeba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:48.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.103+0000 7f02074dc740 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x18d8800 0x1aecc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:48.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.103+0000 7f02074dc740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x1a6ff00 0x1aec1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:48.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.103+0000 7f02074dc740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x19f74f0 con 0x1a6ff00 2020-11-28T21:48:48.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.103+0000 7f02074dc740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x1aeca50 con 0x186ac00 2020-11-28T21:48:48.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.103+0000 7f02074dc740 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x1aee060 con 0x18d8800 2020-11-28T21:48:48.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.103+0000 7f01f259c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x186ac00 0x1aeba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:48.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.103+0000 7f01f259c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x186ac00 0x1aeba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39514/0 (socket says 172.21.15.162:39514) 2020-11-28T21:48:48.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.103+0000 7f01f259c700 1 -- 172.21.15.162:0/3177782323 learned_addr learned my addr 172.21.15.162:0/3177782323 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:48.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.103+0000 7f01f2d9d700 1 --2- 172.21.15.162:0/3177782323 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x1a6ff00 0x1aec1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:48.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.103+0000 7f01ebfff700 1 --2- 172.21.15.162:0/3177782323 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x18d8800 0x1aecc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:48.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.103+0000 7f01f259c700 1 -- 172.21.15.162:0/3177782323 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x18d8800 msgr2=0x1aecc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:48.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.103+0000 7f01f259c700 1 --2- 172.21.15.162:0/3177782323 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x18d8800 0x1aecc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:48.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.103+0000 7f01f259c700 1 -- 172.21.15.162:0/3177782323 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x1a6ff00 msgr2=0x1aec1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:48.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.103+0000 7f01f259c700 1 --2- 172.21.15.162:0/3177782323 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x1a6ff00 0x1aec1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:48.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.103+0000 7f01f259c700 1 -- 172.21.15.162:0/3177782323 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f4144550 con 0x186ac00 2020-11-28T21:48:48.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.103+0000 7f01f259c700 1 --2- 172.21.15.162:0/3177782323 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x186ac00 0x1aeba90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7f01e4055000 tx=0x7f01e4049040).ready entity=mon.1 client_cookie=21db20d4eb24e0f3 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:48.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.103+0000 7f01d37fe700 1 -- 172.21.15.162:0/3177782323 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01e4043640 con 0x186ac00 2020-11-28T21:48:48.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.103+0000 7f01d37fe700 1 -- 172.21.15.162:0/3177782323 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e404f040 con 0x186ac00 2020-11-28T21:48:48.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.103+0000 7f01d37fe700 1 -- 172.21.15.162:0/3177782323 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01e40437a0 con 0x186ac00 2020-11-28T21:48:48.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.103+0000 7f02074dc740 1 -- 172.21.15.162:0/3177782323 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x186ac00 msgr2=0x1aeba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:48.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.103+0000 7f02074dc740 1 --2- 172.21.15.162:0/3177782323 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x186ac00 0x1aeba90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7f01e4055000 tx=0x7f01e4049040).stop 2020-11-28T21:48:48.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.103+0000 7f02074dc740 1 -- 172.21.15.162:0/3177782323 shutdown_connections 2020-11-28T21:48:48.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.103+0000 7f02074dc740 1 --2- 172.21.15.162:0/3177782323 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x1a6ff00 0x1aec1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:48.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.103+0000 7f02074dc740 1 --2- 172.21.15.162:0/3177782323 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x186ac00 0x1aeba90 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:48.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.103+0000 7f02074dc740 1 --2- 172.21.15.162:0/3177782323 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x18d8800 0x1aecc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:48.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.103+0000 7f02074dc740 1 -- 172.21.15.162:0/3177782323 >> 172.21.15.162:0/3177782323 conn(0x1a9f820 msgr2=0x1abe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:48.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.103+0000 7f02074dc740 1 -- 172.21.15.162:0/3177782323 shutdown_connections 2020-11-28T21:48:48.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.103+0000 7f02074dc740 1 -- 172.21.15.162:0/3177782323 wait complete. 2020-11-28T21:48:48.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.103+0000 7f02074dc740 1 -- start start 2020-11-28T21:48:48.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.103+0000 7f02074dc740 1 Processor -- start 2020-11-28T21:48:48.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.107+0000 7f02074dc740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x186ac00 0x1b4cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:48.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.107+0000 7f02074dc740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x18d8800 0x1b53300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:48.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.107+0000 7f02074dc740 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x1a6ff00 0x1b537f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:48.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.107+0000 7f02074dc740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x1af0530 con 0x186ac00 2020-11-28T21:48:48.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.107+0000 7f02074dc740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ab69e0 con 0x18d8800 2020-11-28T21:48:48.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.107+0000 7f02074dc740 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x1b50870 con 0x1a6ff00 2020-11-28T21:48:48.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.107+0000 7f02074dc740 10 client.0 ll_register_callbacks cb 0x1abea60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-28T21:48:48.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.107+0000 7f01ebfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x18d8800 0x1b53300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:48.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.107+0000 7f01ebfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x18d8800 0x1b53300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39518/0 (socket says 172.21.15.162:39518) 2020-11-28T21:48:48.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.107+0000 7f01ebfff700 1 -- 172.21.15.162:0/1513168431 learned_addr learned my addr 172.21.15.162:0/1513168431 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:48.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.107+0000 7f01f2d9d700 1 --2- 172.21.15.162:0/1513168431 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x1a6ff00 0x1b537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:48.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.107+0000 7f01f259c700 1 --2- 172.21.15.162:0/1513168431 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x186ac00 0x1b4cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:48.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.107+0000 7f01ebfff700 1 -- 172.21.15.162:0/1513168431 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x1a6ff00 msgr2=0x1b537f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:48.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.107+0000 7f01ebfff700 1 --2- 172.21.15.162:0/1513168431 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x1a6ff00 0x1b537f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:48.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.107+0000 7f01ebfff700 1 -- 172.21.15.162:0/1513168431 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x186ac00 msgr2=0x1b4cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:48.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.107+0000 7f01ebfff700 1 --2- 172.21.15.162:0/1513168431 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x186ac00 0x1b4cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:48.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.107+0000 7f01ebfff700 1 -- 172.21.15.162:0/1513168431 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1b53ce0 con 0x18d8800 2020-11-28T21:48:48.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.107+0000 7f01f2d9d700 1 --2- 172.21.15.162:0/1513168431 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x1a6ff00 0x1b537f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:48:48.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.107+0000 7f01ebfff700 1 --2- 172.21.15.162:0/1513168431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x18d8800 0x1b53300 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f01dc015000 tx=0x7f01dc00a000).ready entity=mon.1 client_cookie=a3c8d25d9bb55cb8 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:48.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.107+0000 7f01d27fc700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2020-11-28T21:48:48.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.107+0000 7f01d27fc700 1 -- 172.21.15.162:0/1513168431 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01dc01c070 con 0x18d8800 2020-11-28T21:48:48.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.107+0000 7f02074dc740 1 -- 172.21.15.162:0/1513168431 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x18ce4b0 con 0x18d8800 2020-11-28T21:48:48.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.107+0000 7f01d27fc700 1 -- 172.21.15.162:0/1513168431 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f01dc023e30 con 0x18d8800 2020-11-28T21:48:48.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.107+0000 7f01d27fc700 1 -- 172.21.15.162:0/1513168431 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01dc0213b0 con 0x18d8800 2020-11-28T21:48:48.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.107+0000 7f02074dc740 10 client.5410 fetch_fsmap learned FSMap version 15 2020-11-28T21:48:48.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.107+0000 7f02074dc740 1 -- 172.21.15.162:0/1513168431 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x1ab6660 con 0x18d8800 2020-11-28T21:48:48.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.107+0000 7f01d27fc700 1 -- 172.21.15.162:0/1513168431 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x18ce4b0 con 0x18d8800 2020-11-28T21:48:48.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.107+0000 7f01d27fc700 1 -- 172.21.15.162:0/1513168431 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 15) v1 ==== 1434+0+0 (secure 0 0 0) 0x7f01dc020070 con 0x18d8800 2020-11-28T21:48:48.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.107+0000 7f02074dc740 10 client.5410 fetch_fsmap finished waiting for FSMap version 15 2020-11-28T21:48:48.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.107+0000 7f02074dc740 10 client.5410 resolve_mds: resolved ID 'a' to GID 5212 2020-11-28T21:48:48.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.107+0000 7f02074dc740 20 client.5410 populate_metadata read hostname 'smithi162' 2020-11-28T21:48:48.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.107+0000 7f02074dc740 1 --2- 172.21.15.162:0/1513168431 >> [v2:172.21.15.162:6836/2706248574,v1:172.21.15.162:6837/2706248574] conn(0x1992e20 0x1aee300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:48.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.107+0000 7f02074dc740 4 client.5410 mds_command: new command op to 5212 tid=0[{"prefix": "cache drop"}] 2020-11-28T21:48:48.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.107+0000 7f02074dc740 1 -- 172.21.15.162:0/1513168431 --> [v2:172.21.15.162:6836/2706248574,v1:172.21.15.162:6837/2706248574] -- command(tid 0: {"prefix": "cache drop"}) v1 -- 0x191c710 con 0x1992e20 2020-11-28T21:48:48.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.107+0000 7f01f259c700 1 --2- 172.21.15.162:0/1513168431 >> [v2:172.21.15.162:6836/2706248574,v1:172.21.15.162:6837/2706248574] conn(0x1992e20 0x1aee300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:48.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.107+0000 7f01d27fc700 10 client.5410 ms_handle_connect on v2:172.21.15.162:6836/2706248574 2020-11-28T21:48:48.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:48.107+0000 7f01f259c700 1 --2- 172.21.15.162:0/1513168431 >> [v2:172.21.15.162:6836/2706248574,v1:172.21.15.162:6837/2706248574] conn(0x1992e20 0x1aee300 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=4f01b49835e95393 server_cookie=73a930290290f4a in_seq=0 out_seq=0 2020-11-28T21:48:59.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.367+0000 7f01d27fc700 1 -- 172.21.15.162:0/1513168431 <== mds.0 v2:172.21.15.162:6836/2706248574 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+378 (crc 0 0 0) 0x191c710 con 0x1992e20 2020-11-28T21:48:59.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.367+0000 7f01d27fc700 10 client.5410 handle_command_reply: tid=0 2020-11-28T21:48:59.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.367+0000 7f02074dc740 1 client.5410 shutdown 2020-11-28T21:48:59.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.367+0000 7f02074dc740 1 -- 172.21.15.162:0/1513168431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x18d8800 msgr2=0x1b53300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:59.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.367+0000 7f02074dc740 1 --2- 172.21.15.162:0/1513168431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x18d8800 0x1b53300 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f01dc015000 tx=0x7f01dc00a000).stop 2020-11-28T21:48:59.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.367+0000 7f02074dc740 1 -- 172.21.15.162:0/1513168431 shutdown_connections 2020-11-28T21:48:59.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.367+0000 7f02074dc740 1 --2- 172.21.15.162:0/1513168431 >> [v2:172.21.15.162:6836/2706248574,v1:172.21.15.162:6837/2706248574] conn(0x1992e20 0x1aee300 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:59.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.367+0000 7f02074dc740 1 --2- 172.21.15.162:0/1513168431 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x1a6ff00 0x1b537f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:59.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.367+0000 7f02074dc740 1 --2- 172.21.15.162:0/1513168431 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x186ac00 0x1b4cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:59.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.367+0000 7f01d27fc700 0 client.5410 ms_handle_reset on v2:172.21.15.162:6836/2706248574 2020-11-28T21:48:59.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.367+0000 7f02074dc740 1 --2- 172.21.15.162:0/1513168431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x18d8800 0x1b53300 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:59.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.367+0000 7f02074dc740 1 -- 172.21.15.162:0/1513168431 >> 172.21.15.162:0/1513168431 conn(0x1a9f820 msgr2=0x1a9e600 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:59.371 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.367+0000 7f02074dc740 1 -- 172.21.15.162:0/1513168431 shutdown_connections 2020-11-28T21:48:59.372 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.367+0000 7f02074dc740 1 -- 172.21.15.162:0/1513168431 wait complete. 2020-11-28T21:48:59.372 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.367+0000 7f02074dc740 20 client.5410 trim_cache size 0 max 16384 2020-11-28T21:48:59.372 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.367+0000 7f01f359e700 1 -- 172.21.15.162:0/1429230584 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f01d8055350 msgr2=0x7f01d8057810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:59.372 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.367+0000 7f01f359e700 1 --2- 172.21.15.162:0/1429230584 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f01d8055350 0x7f01d8057810 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f01dc00b000 tx=0x7f01dc000b80).stop 2020-11-28T21:48:59.373 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.367+0000 7f01f359e700 1 -- 172.21.15.162:0/1429230584 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01f405c1e0 msgr2=0x7f01f412cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:59.373 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.367+0000 7f01f359e700 1 --2- 172.21.15.162:0/1429230584 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01f405c1e0 0x7f01f412cbf0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f01f4155cf0 tx=0x7f01e4007c20).stop 2020-11-28T21:48:59.373 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.367+0000 7f01f359e700 1 -- 172.21.15.162:0/1429230584 shutdown_connections 2020-11-28T21:48:59.373 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.367+0000 7f01f359e700 1 --2- 172.21.15.162:0/1429230584 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f01d8055350 0x7f01d8057810 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:59.373 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.367+0000 7f01f359e700 1 --2- 172.21.15.162:0/1429230584 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01f4060900 0x7f01f41556c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:59.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.367+0000 7f01f359e700 1 --2- 172.21.15.162:0/1429230584 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01f405c1e0 0x7f01f412cbf0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:59.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.367+0000 7f01f359e700 1 --2- 172.21.15.162:0/1429230584 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f01f405c840 0x7f01f41551d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:59.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.367+0000 7f01f359e700 1 -- 172.21.15.162:0/1429230584 >> 172.21.15.162:0/1429230584 conn(0x7f01f406aa70 msgr2=0x7f01f40b9880 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:59.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.367+0000 7f01f359e700 1 -- 172.21.15.162:0/1429230584 shutdown_connections 2020-11-28T21:48:59.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.367+0000 7f01f359e700 1 -- 172.21.15.162:0/1429230584 wait complete. 2020-11-28T21:48:59.375 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2020-11-28T21:48:59.375 INFO:teuthology.orchestra.run.smithi162.stdout: "client_recall": { 2020-11-28T21:48:59.375 INFO:teuthology.orchestra.run.smithi162.stdout: "return_code": 0, 2020-11-28T21:48:59.376 INFO:teuthology.orchestra.run.smithi162.stdout: "message": "(0) Success", 2020-11-28T21:48:59.376 INFO:teuthology.orchestra.run.smithi162.stdout: "recalled": 220 2020-11-28T21:48:59.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:59.377 INFO:teuthology.orchestra.run.smithi162.stdout: "flush_journal": { 2020-11-28T21:48:59.377 INFO:teuthology.orchestra.run.smithi162.stdout: "return_code": 0, 2020-11-28T21:48:59.377 INFO:teuthology.orchestra.run.smithi162.stdout: "message": "" 2020-11-28T21:48:59.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:59.381 INFO:teuthology.orchestra.run.smithi162.stdout: "trim_cache": { 2020-11-28T21:48:59.381 INFO:teuthology.orchestra.run.smithi162.stdout: "trimmed": 0 2020-11-28T21:48:59.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:59.382 INFO:teuthology.orchestra.run.smithi162.stdout: "cache": { 2020-11-28T21:48:59.382 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": { 2020-11-28T21:48:59.382 INFO:teuthology.orchestra.run.smithi162.stdout: "items": 19916, 2020-11-28T21:48:59.383 INFO:teuthology.orchestra.run.smithi162.stdout: "bytes": 3094944 2020-11-28T21:48:59.383 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:48:59.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:48:59.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 11.2568077 2020-11-28T21:48:59.383 INFO:teuthology.orchestra.run.smithi162.stdout:} 2020-11-28T21:48:59.385 INFO:tasks.cephfs.fuse_mount:Cleaning up killed ceph-fuse connection 2020-11-28T21:48:59.385 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:48:59.386 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:48:59.386 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:48:59.393 INFO:teuthology.orchestra.run.smithi192.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Transport endpoint is not connected 2020-11-28T21:48:59.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:48:59.395 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2020-11-28T21:48:59.395 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2020-11-28T21:48:59.395 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-28T21:48:59.396 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:48:59.484 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:48:59.485 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:48:59.485 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:48:59.492 INFO:teuthology.orchestra.run.smithi192.stdout:ext2/ext3 2020-11-28T21:48:59.493 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-11-28T21:48:59.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-28T21:48:59.494 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:48:59.541 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:48:59.659 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.659+0000 7f8c8d384700 1 Processor -- start 2020-11-28T21:48:59.659 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.659+0000 7f8c8d384700 1 -- start start 2020-11-28T21:48:59.660 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.659+0000 7f8c8d384700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8c880b5a70 0x7f8c88143360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:59.660 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.659+0000 7f8c8d384700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f8c880b5e50 0x7f8c880b6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:59.661 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.659+0000 7f8c8d384700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c880b8040 con 0x7f8c880b5430 2020-11-28T21:48:59.661 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.659+0000 7f8c8d384700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c880b8720 con 0x7f8c880b5e50 2020-11-28T21:48:59.661 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.659+0000 7f8c8d384700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c88142980 con 0x7f8c880b5a70 2020-11-28T21:48:59.661 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.659+0000 7f8c877fe700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f8c880b5e50 0x7f8c880b6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:59.662 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.659+0000 7f8c87fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8c880b5a70 0x7f8c88143360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:59.662 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.659+0000 7f8c877fe700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f8c880b5e50 0x7f8c880b6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.162:46518/0 (socket says 172.21.15.162:46518) 2020-11-28T21:48:59.662 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.659+0000 7f8c877fe700 1 -- 172.21.15.162:0/2931595505 learned_addr learned my addr 172.21.15.162:0/2931595505 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:59.662 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.659+0000 7f8c8cb83700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f8c880b5430 0x7f8c8809d220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.162:36524/0 (socket says 172.21.15.162:36524) 2020-11-28T21:48:59.664 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.663+0000 7f8c877fe700 1 -- 172.21.15.162:0/2931595505 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8c880b5a70 msgr2=0x7f8c88143360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:59.664 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.663+0000 7f8c877fe700 1 --2- 172.21.15.162:0/2931595505 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8c880b5a70 0x7f8c88143360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:59.664 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.663+0000 7f8c877fe700 1 -- 172.21.15.162:0/2931595505 >> v1:172.21.15.118:6789/0 conn(0x7f8c880b5430 legacy=0x7f8c8809d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:59.665 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.663+0000 7f8c877fe700 1 -- 172.21.15.162:0/2931595505 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c88142c20 con 0x7f8c880b5e50 2020-11-28T21:48:59.665 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.663+0000 7f8c87fff700 1 --2- 172.21.15.162:0/2931595505 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8c880b5a70 0x7f8c88143360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:48:59.665 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.663+0000 7f8c86ffd700 1 -- 172.21.15.162:0/2931595505 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 701045357 0 0) 0x7f8c880b8040 con 0x7f8c880b5430 2020-11-28T21:48:59.665 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.663+0000 7f8c877fe700 1 --2- 172.21.15.162:0/2931595505 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8c880b5e50 0x7f8c880b6ef0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f8c78002f30 tx=0x7f8c78006e80).ready entity=mon.0 client_cookie=4360bafff9a859b6 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:59.666 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.663+0000 7f8c86ffd700 1 -- 172.21.15.162:0/2931595505 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c780097f0 con 0x7f8c880b5e50 2020-11-28T21:48:59.666 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.663+0000 7f8c86ffd700 1 -- 172.21.15.162:0/2931595505 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c78009950 con 0x7f8c880b5e50 2020-11-28T21:48:59.666 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.663+0000 7f8c8d384700 1 -- 172.21.15.162:0/2931595505 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8c880b5e50 msgr2=0x7f8c880b6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:59.666 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.663+0000 7f8c8d384700 1 --2- 172.21.15.162:0/2931595505 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8c880b5e50 0x7f8c880b6ef0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f8c78002f30 tx=0x7f8c78006e80).stop 2020-11-28T21:48:59.666 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.663+0000 7f8c86ffd700 1 -- 172.21.15.162:0/2931595505 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c78009cb0 con 0x7f8c880b5e50 2020-11-28T21:48:59.667 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.663+0000 7f8c8d384700 1 -- 172.21.15.162:0/2931595505 shutdown_connections 2020-11-28T21:48:59.667 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.663+0000 7f8c8d384700 1 --2- 172.21.15.162:0/2931595505 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8c880b5a70 0x7f8c88143360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:59.667 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.663+0000 7f8c8d384700 1 --2- 172.21.15.162:0/2931595505 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8c880b5e50 0x7f8c880b6ef0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:59.667 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.663+0000 7f8c8d384700 1 -- 172.21.15.162:0/2931595505 >> 172.21.15.162:0/2931595505 conn(0x7f8c8805b970 msgr2=0x7f8c8805bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:59.668 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.663+0000 7f8c8d384700 1 -- 172.21.15.162:0/2931595505 shutdown_connections 2020-11-28T21:48:59.668 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.663+0000 7f8c8d384700 1 -- 172.21.15.162:0/2931595505 wait complete. 2020-11-28T21:48:59.668 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.663+0000 7f8c8d384700 1 Processor -- start 2020-11-28T21:48:59.668 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.663+0000 7f8c8d384700 1 -- start start 2020-11-28T21:48:59.670 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.663+0000 7f8c8d384700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8c880b5430 0x7f8c880606c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:59.671 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.663+0000 7f8c8d384700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c880b5a70 0x7f8c881455c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:59.671 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.663+0000 7f8c8d384700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8c880b5e50 0x7f8c88145ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:59.671 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.663+0000 7f8c8d384700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c88149280 con 0x7f8c880b5430 2020-11-28T21:48:59.671 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.663+0000 7f8c8d384700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c88149840 con 0x7f8c880b5a70 2020-11-28T21:48:59.672 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.663+0000 7f8c8d384700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c88149ae0 con 0x7f8c880b5e50 2020-11-28T21:48:59.672 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.663+0000 7f8c877fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c880b5a70 0x7f8c881455c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:59.672 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.663+0000 7f8c877fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c880b5a70 0x7f8c881455c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39536/0 (socket says 172.21.15.162:39536) 2020-11-28T21:48:59.672 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.667+0000 7f8c877fe700 1 -- 172.21.15.162:0/1743324840 learned_addr learned my addr 172.21.15.162:0/1743324840 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:59.673 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.667+0000 7f8c87fff700 1 --2- 172.21.15.162:0/1743324840 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8c880b5430 0x7f8c880606c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:59.673 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.667+0000 7f8c8cb83700 1 --2- 172.21.15.162:0/1743324840 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8c880b5e50 0x7f8c88145ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:59.673 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.667+0000 7f8c877fe700 1 -- 172.21.15.162:0/1743324840 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8c880b5e50 msgr2=0x7f8c88145ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:59.673 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.667+0000 7f8c877fe700 1 --2- 172.21.15.162:0/1743324840 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8c880b5e50 0x7f8c88145ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:59.673 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.667+0000 7f8c877fe700 1 -- 172.21.15.162:0/1743324840 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8c880b5430 msgr2=0x7f8c880606c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:59.674 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.667+0000 7f8c877fe700 1 --2- 172.21.15.162:0/1743324840 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8c880b5430 0x7f8c880606c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:59.674 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.667+0000 7f8c877fe700 1 -- 172.21.15.162:0/1743324840 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c880ae880 con 0x7f8c880b5a70 2020-11-28T21:48:59.674 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.667+0000 7f8c877fe700 1 --2- 172.21.15.162:0/1743324840 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c880b5a70 0x7f8c881455c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f8c78002f30 tx=0x7f8c78015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:59.674 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.667+0000 7f8c857fa700 1 -- 172.21.15.162:0/1743324840 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c7800bee0 con 0x7f8c880b5a70 2020-11-28T21:48:59.674 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.667+0000 7f8c857fa700 1 -- 172.21.15.162:0/1743324840 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c78011b60 con 0x7f8c880b5a70 2020-11-28T21:48:59.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.667+0000 7f8c857fa700 1 -- 172.21.15.162:0/1743324840 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c78006bb0 con 0x7f8c880b5a70 2020-11-28T21:48:59.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.667+0000 7f8c8d384700 1 -- 172.21.15.162:0/1743324840 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c88149d80 con 0x7f8c880b5a70 2020-11-28T21:48:59.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.667+0000 7f8c8d384700 1 -- 172.21.15.162:0/1743324840 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c8813f5e0 con 0x7f8c880b5a70 2020-11-28T21:48:59.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.667+0000 7f8c8d384700 1 -- 172.21.15.162:0/1743324840 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c88057050 con 0x7f8c880b5a70 2020-11-28T21:48:59.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.671+0000 7f8c857fa700 1 -- 172.21.15.162:0/1743324840 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8c78025030 con 0x7f8c880b5a70 2020-11-28T21:48:59.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.671+0000 7f8c857fa700 1 --2- 172.21.15.162:0/1743324840 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8c68055500 0x7f8c680579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:59.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.671+0000 7f8c87fff700 1 --2- 172.21.15.162:0/1743324840 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8c68055500 0x7f8c680579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:59.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.671+0000 7f8c857fa700 1 -- 172.21.15.162:0/1743324840 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8c7806eec0 con 0x7f8c880b5a70 2020-11-28T21:48:59.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.671+0000 7f8c87fff700 1 --2- 172.21.15.162:0/1743324840 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8c68055500 0x7f8c680579c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f8c7c00b000 tx=0x7f8c7c006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:59.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.675+0000 7f8c857fa700 1 -- 172.21.15.162:0/1743324840 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8c780451c0 con 0x7f8c880b5a70 2020-11-28T21:48:59.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.819+0000 7f8c8d384700 1 -- 172.21.15.162:0/1743324840 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c8813e910 con 0x7f8c880b5a70 2020-11-28T21:48:59.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.819+0000 7f8c857fa700 1 -- 172.21.15.162:0/1743324840 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16134 (secure 0 0 0) 0x7f8c780487e0 con 0x7f8c880b5a70 2020-11-28T21:48:59.820 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:48:59.821 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":44,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:48:27.785910+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:48:25.281355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2020-11-28T21:48:25.782088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:48:59.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.823+0000 7f8c8d384700 1 -- 172.21.15.162:0/1743324840 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8c68055500 msgr2=0x7f8c680579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:59.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.823+0000 7f8c8d384700 1 --2- 172.21.15.162:0/1743324840 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8c68055500 0x7f8c680579c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f8c7c00b000 tx=0x7f8c7c006dc0).stop 2020-11-28T21:48:59.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.823+0000 7f8c8d384700 1 -- 172.21.15.162:0/1743324840 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c880b5a70 msgr2=0x7f8c881455c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:59.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.823+0000 7f8c8d384700 1 --2- 172.21.15.162:0/1743324840 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c880b5a70 0x7f8c881455c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f8c78002f30 tx=0x7f8c78015000).stop 2020-11-28T21:48:59.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.823+0000 7f8c8d384700 1 -- 172.21.15.162:0/1743324840 shutdown_connections 2020-11-28T21:48:59.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.823+0000 7f8c8d384700 1 --2- 172.21.15.162:0/1743324840 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8c68055500 0x7f8c680579c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:59.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.823+0000 7f8c8d384700 1 --2- 172.21.15.162:0/1743324840 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8c880b5e50 0x7f8c88145ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:59.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.823+0000 7f8c8d384700 1 --2- 172.21.15.162:0/1743324840 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8c880b5430 0x7f8c880606c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:59.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.823+0000 7f8c8d384700 1 --2- 172.21.15.162:0/1743324840 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c880b5a70 0x7f8c881455c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:59.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.823+0000 7f8c8d384700 1 -- 172.21.15.162:0/1743324840 >> 172.21.15.162:0/1743324840 conn(0x7f8c8805b970 msgr2=0x7f8c8813eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:59.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.823+0000 7f8c8d384700 1 -- 172.21.15.162:0/1743324840 shutdown_connections 2020-11-28T21:48:59.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.823+0000 7f8c8d384700 1 -- 172.21.15.162:0/1743324840 wait complete. 2020-11-28T21:48:59.834 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-28T21:48:25.281355+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2020-11-28T21:48:25.782088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-28T21:48:59.836 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:48:59.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.955+0000 7f18f7fff700 1 Processor -- start 2020-11-28T21:48:59.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.955+0000 7f18f7fff700 1 -- start start 2020-11-28T21:48:59.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.955+0000 7f18f7fff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f18f80b20a0 0x7f18f8140590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:59.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.955+0000 7f18f7fff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18f80b1420 con 0x7f18f813dbe0 2020-11-28T21:48:59.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.955+0000 7f18f7fff700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18f80b16c0 con 0x7f18f813dfc0 2020-11-28T21:48:59.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.955+0000 7f18f7fff700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18f80b0040 con 0x7f18f80b20a0 2020-11-28T21:48:59.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.955+0000 7f18f67fc700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f18f813dfc0 0x7f18f805b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:46386/0 (socket says 172.21.15.162:46386) 2020-11-28T21:48:59.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.955+0000 7f18f67fc700 1 -- 172.21.15.162:0/3876484462 learned_addr learned my addr 172.21.15.162:0/3876484462 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:48:59.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.959+0000 7f18f77fe700 1 --2- 172.21.15.162:0/3876484462 >> v2:172.21.15.118:3300/0 conn(0x7f18f80b20a0 0x7f18f8140590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:59.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.959+0000 7f18f5ffb700 1 -- 172.21.15.162:0/3876484462 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1798364421 0 0) 0x7f18f80b16c0 con 0x7f18f813dfc0 2020-11-28T21:48:59.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.959+0000 7f18f5ffb700 1 -- 172.21.15.162:0/3876484462 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18dc003230 con 0x7f18f813dfc0 2020-11-28T21:48:59.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.959+0000 7f18f5ffb700 1 -- 172.21.15.162:0/3876484462 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2756091194 0 0) 0x7f18dc003230 con 0x7f18f813dfc0 2020-11-28T21:48:59.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.959+0000 7f18f5ffb700 1 -- 172.21.15.162:0/3876484462 >> v2:172.21.15.118:3300/0 conn(0x7f18f80b20a0 msgr2=0x7f18f8140590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:59.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.959+0000 7f18f5ffb700 1 --2- 172.21.15.162:0/3876484462 >> v2:172.21.15.118:3300/0 conn(0x7f18f80b20a0 0x7f18f8140590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:59.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.959+0000 7f18f5ffb700 1 -- 172.21.15.162:0/3876484462 >> v1:172.21.15.118:6789/0 conn(0x7f18f813dbe0 legacy=0x7f18f8060920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:59.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.959+0000 7f18f5ffb700 1 -- 172.21.15.162:0/3876484462 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f80b1960 con 0x7f18f813dfc0 2020-11-28T21:48:59.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.959+0000 7f18f5ffb700 1 -- 172.21.15.162:0/3876484462 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f18ec0016d0 con 0x7f18f813dfc0 2020-11-28T21:48:59.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.959+0000 7f18f5ffb700 1 -- 172.21.15.162:0/3876484462 >> v1:172.21.15.162:6789/0 conn(0x7f18f813dfc0 legacy=0x7f18f805b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:59.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.959+0000 7f18f5ffb700 1 --2- 172.21.15.162:0/3876484462 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18dc003720 0x7f18dc003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:59.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.959+0000 7f18f5ffb700 1 --2- 172.21.15.162:0/3876484462 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f18dc0044d0 0x7f18dc004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:59.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.959+0000 7f18f5ffb700 1 --2- 172.21.15.162:0/3876484462 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f18dc004e40 0x7f18dc00b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:59.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.959+0000 7f18f5ffb700 1 -- 172.21.15.162:0/3876484462 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f80b0040 con 0x7f18dc0044d0 2020-11-28T21:48:59.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.959+0000 7f18f5ffb700 1 -- 172.21.15.162:0/3876484462 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc004250 con 0x7f18dc003720 2020-11-28T21:48:59.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.959+0000 7f18f5ffb700 1 -- 172.21.15.162:0/3876484462 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc00bc10 con 0x7f18dc004e40 2020-11-28T21:48:59.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.959+0000 7f18f6ffd700 1 --2- 172.21.15.162:0/3876484462 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18dc003720 0x7f18dc003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:59.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.959+0000 7f18f67fc700 1 --2- 172.21.15.162:0/3876484462 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f18dc0044d0 0x7f18dc004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:59.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.959+0000 7f18f77fe700 1 --2- 172.21.15.162:0/3876484462 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f18dc004e40 0x7f18dc00b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:59.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.959+0000 7f18f6ffd700 1 -- 172.21.15.162:0/3876484462 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f18dc004e40 msgr2=0x7f18dc00b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:59.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.959+0000 7f18f6ffd700 1 --2- 172.21.15.162:0/3876484462 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f18dc004e40 0x7f18dc00b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:59.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.959+0000 7f18f6ffd700 1 -- 172.21.15.162:0/3876484462 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f18dc0044d0 msgr2=0x7f18dc004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:48:59.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.959+0000 7f18f6ffd700 1 --2- 172.21.15.162:0/3876484462 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f18dc0044d0 0x7f18dc004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:59.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.959+0000 7f18f77fe700 1 -- 172.21.15.162:0/3876484462 reap_dead start 2020-11-28T21:48:59.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.959+0000 7f18f6ffd700 1 -- 172.21.15.162:0/3876484462 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18dc003230 con 0x7f18dc003720 2020-11-28T21:48:59.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.959+0000 7f18f6ffd700 1 --2- 172.21.15.162:0/3876484462 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18dc003720 0x7f18dc003b40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f18f8141720 tx=0x7f18e8007b60).ready entity=mon.1 client_cookie=150d7601f94779d3 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:59.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.959+0000 7f18f5ffb700 1 -- 172.21.15.162:0/3876484462 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18e800c660 con 0x7f18dc003720 2020-11-28T21:48:59.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.959+0000 7f18f5ffb700 1 -- 172.21.15.162:0/3876484462 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e800c7c0 con 0x7f18dc003720 2020-11-28T21:48:59.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.959+0000 7f18f5ffb700 1 -- 172.21.15.162:0/3876484462 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18e800cb20 con 0x7f18dc003720 2020-11-28T21:48:59.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.959+0000 7f18f7fff700 1 -- 172.21.15.162:0/3876484462 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18dc003720 msgr2=0x7f18dc003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:59.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.959+0000 7f18f7fff700 1 --2- 172.21.15.162:0/3876484462 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18dc003720 0x7f18dc003b40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f18f8141720 tx=0x7f18e8007b60).stop 2020-11-28T21:48:59.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.959+0000 7f18f7fff700 1 -- 172.21.15.162:0/3876484462 shutdown_connections 2020-11-28T21:48:59.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.959+0000 7f18f7fff700 1 --2- 172.21.15.162:0/3876484462 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18dc003720 0x7f18dc003b40 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:59.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.959+0000 7f18f7fff700 1 -- 172.21.15.162:0/3876484462 >> 172.21.15.162:0/3876484462 conn(0x7f18f80b2830 msgr2=0x7f18f80b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:48:59.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.959+0000 7f18f7fff700 1 -- 172.21.15.162:0/3876484462 shutdown_connections 2020-11-28T21:48:59.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.959+0000 7f18f7fff700 1 -- 172.21.15.162:0/3876484462 wait complete. 2020-11-28T21:48:59.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.963+0000 7f18f7fff700 1 Processor -- start 2020-11-28T21:48:59.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.963+0000 7f18f7fff700 1 -- start start 2020-11-28T21:48:59.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.963+0000 7f18f7fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f18dc003720 0x7f18f813f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:59.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.963+0000 7f18f7fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f18f8138110 0x7f18f8138590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:59.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.963+0000 7f18f7fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18f8144c20 0x7f18f81450b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:59.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.963+0000 7f18f7fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f80b58a0 con 0x7f18dc003720 2020-11-28T21:48:59.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.963+0000 7f18f7fff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f81468e0 con 0x7f18f8144c20 2020-11-28T21:48:59.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.963+0000 7f18f7fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f813fdc0 con 0x7f18f8138110 2020-11-28T21:48:59.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.963+0000 7f18f77fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18f8144c20 0x7f18f81450b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:59.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.963+0000 7f18f67fc700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f18f8138110 0x7f18f8138590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:59.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.963+0000 7f18f6ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f18dc003720 0x7f18f813f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:59.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.963+0000 7f18f77fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18f8144c20 0x7f18f81450b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39558/0 (socket says 172.21.15.162:39558) 2020-11-28T21:48:59.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.963+0000 7f18f77fe700 1 -- 172.21.15.162:0/3349795594 learned_addr learned my addr 172.21.15.162:0/3349795594 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:48:59.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.963+0000 7f18f77fe700 1 -- 172.21.15.162:0/3349795594 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f18f8138110 msgr2=0x7f18f8138590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:59.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.963+0000 7f18f77fe700 1 --2- 172.21.15.162:0/3349795594 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f18f8138110 0x7f18f8138590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:59.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.963+0000 7f18f77fe700 1 -- 172.21.15.162:0/3349795594 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f18dc003720 msgr2=0x7f18f813f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:48:59.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.963+0000 7f18f77fe700 1 --2- 172.21.15.162:0/3349795594 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f18dc003720 0x7f18f813f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:48:59.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.963+0000 7f18f77fe700 1 -- 172.21.15.162:0/3349795594 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f8138a80 con 0x7f18f8144c20 2020-11-28T21:48:59.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.963+0000 7f18f67fc700 1 --2- 172.21.15.162:0/3349795594 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f18f8138110 0x7f18f8138590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:48:59.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.963+0000 7f18f77fe700 1 --2- 172.21.15.162:0/3349795594 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18f8144c20 0x7f18f81450b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7f18f80b59e0 tx=0x7f18f00091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:59.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.963+0000 7f18e7fff700 1 -- 172.21.15.162:0/3349795594 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18f0019070 con 0x7f18f8144c20 2020-11-28T21:48:59.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.963+0000 7f18f7fff700 1 -- 172.21.15.162:0/3349795594 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18f8143ca0 con 0x7f18f8144c20 2020-11-28T21:48:59.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.963+0000 7f18e7fff700 1 -- 172.21.15.162:0/3349795594 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f000fab0 con 0x7f18f8144c20 2020-11-28T21:48:59.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.963+0000 7f18e7fff700 1 -- 172.21.15.162:0/3349795594 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18f000fe10 con 0x7f18f8144c20 2020-11-28T21:48:59.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.963+0000 7f18f7fff700 1 -- 172.21.15.162:0/3349795594 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18f8141990 con 0x7f18f8144c20 2020-11-28T21:48:59.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.967+0000 7f18f7fff700 1 -- 172.21.15.162:0/3349795594 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18f8057050 con 0x7f18f8144c20 2020-11-28T21:48:59.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.967+0000 7f18e7fff700 1 -- 172.21.15.162:0/3349795594 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f18f0014440 con 0x7f18f8144c20 2020-11-28T21:48:59.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.967+0000 7f18e7fff700 1 --2- 172.21.15.162:0/3349795594 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f18d40553b0 0x7f18d4057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:48:59.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.967+0000 7f18e7fff700 1 -- 172.21.15.162:0/3349795594 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f18f006f390 con 0x7f18f8144c20 2020-11-28T21:48:59.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.967+0000 7f18f6ffd700 1 --2- 172.21.15.162:0/3349795594 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f18d40553b0 0x7f18d4057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:48:59.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.971+0000 7f18f6ffd700 1 --2- 172.21.15.162:0/3349795594 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f18d40553b0 0x7f18d4057870 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f18e801a000 tx=0x7f18e8016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:48:59.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:48:59.975+0000 7f18e7fff700 1 -- 172.21.15.162:0/3349795594 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f18f0045630 con 0x7f18f8144c20 2020-11-28T21:49:00.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.115+0000 7f18f7fff700 1 -- 172.21.15.162:0/3349795594 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f18f813ff00 con 0x7f18f8144c20 2020-11-28T21:49:00.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.115+0000 7f18e7fff700 1 -- 172.21.15.162:0/3349795594 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v44) v1 ==== 110+0+10 (secure 0 0 0) 0x7f18f0048c50 con 0x7f18f8144c20 2020-11-28T21:49:00.114 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:49:00.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.115+0000 7f18f7fff700 1 -- 172.21.15.162:0/3349795594 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f18d40553b0 msgr2=0x7f18d4057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:00.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.115+0000 7f18f7fff700 1 --2- 172.21.15.162:0/3349795594 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f18d40553b0 0x7f18d4057870 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f18e801a000 tx=0x7f18e8016000).stop 2020-11-28T21:49:00.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.115+0000 7f18f7fff700 1 -- 172.21.15.162:0/3349795594 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18f8144c20 msgr2=0x7f18f81450b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:00.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.115+0000 7f18f7fff700 1 --2- 172.21.15.162:0/3349795594 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18f8144c20 0x7f18f81450b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7f18f80b59e0 tx=0x7f18f00091a0).stop 2020-11-28T21:49:00.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.115+0000 7f18f7fff700 1 -- 172.21.15.162:0/3349795594 shutdown_connections 2020-11-28T21:49:00.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.115+0000 7f18f7fff700 1 --2- 172.21.15.162:0/3349795594 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f18d40553b0 0x7f18d4057870 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:00.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.115+0000 7f18f7fff700 1 --2- 172.21.15.162:0/3349795594 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18f8144c20 0x7f18f81450b0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:00.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.115+0000 7f18f7fff700 1 --2- 172.21.15.162:0/3349795594 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f18dc003720 0x7f18f813f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:00.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.115+0000 7f18f7fff700 1 --2- 172.21.15.162:0/3349795594 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f18f8138110 0x7f18f8138590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:00.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.115+0000 7f18f7fff700 1 -- 172.21.15.162:0/3349795594 >> 172.21.15.162:0/3349795594 conn(0x7f18f80b2830 msgr2=0x7f18f80ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:49:00.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.115+0000 7f18f7fff700 1 -- 172.21.15.162:0/3349795594 shutdown_connections 2020-11-28T21:49:00.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.115+0000 7f18f7fff700 1 -- 172.21.15.162:0/3349795594 wait complete. 2020-11-28T21:49:00.132 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:49:00.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.251+0000 7fc362a46700 1 Processor -- start 2020-11-28T21:49:00.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.251+0000 7fc362a46700 1 -- start start 2020-11-28T21:49:00.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.255+0000 7fc362a46700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fc35c05c4d0 0x7fc35c0b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:00.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.255+0000 7fc362a46700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc35c05baf0 con 0x7fc35c0b3ab0 2020-11-28T21:49:00.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.255+0000 7fc362a46700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc35c05bd90 con 0x7fc35c0b2d00 2020-11-28T21:49:00.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.255+0000 7fc362a46700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc35c0b0e20 con 0x7fc35c05c4d0 2020-11-28T21:49:00.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.255+0000 7fc361a44700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fc35c0b2d00 0x7fc35c142ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:46406/0 (socket says 172.21.15.162:46406) 2020-11-28T21:49:00.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.255+0000 7fc361a44700 1 -- 172.21.15.162:0/1584117205 learned_addr learned my addr 172.21.15.162:0/1584117205 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:49:00.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.255+0000 7fc362245700 1 --2- 172.21.15.162:0/1584117205 >> v2:172.21.15.118:3300/0 conn(0x7fc35c05c4d0 0x7fc35c0b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:00.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.255+0000 7fc360a42700 1 -- 172.21.15.162:0/1584117205 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 583453118 0 0) 0x7fc35c05bd90 con 0x7fc35c0b2d00 2020-11-28T21:49:00.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.255+0000 7fc360a42700 1 -- 172.21.15.162:0/1584117205 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc344003230 con 0x7fc35c0b2d00 2020-11-28T21:49:00.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.255+0000 7fc360a42700 1 -- 172.21.15.162:0/1584117205 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3601087471 0 0) 0x7fc344003230 con 0x7fc35c0b2d00 2020-11-28T21:49:00.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.255+0000 7fc362245700 1 -- 172.21.15.162:0/1584117205 >> v1:172.21.15.162:6789/0 conn(0x7fc35c0b2d00 legacy=0x7fc35c142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:00.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.255+0000 7fc362245700 1 -- 172.21.15.162:0/1584117205 >> v1:172.21.15.118:6789/0 conn(0x7fc35c0b3ab0 legacy=0x7fc35c13fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:00.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.255+0000 7fc362245700 1 -- 172.21.15.162:0/1584117205 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc35c05c030 con 0x7fc35c05c4d0 2020-11-28T21:49:00.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.255+0000 7fc362245700 1 --2- 172.21.15.162:0/1584117205 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc35c05c4d0 0x7fc35c0b30e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7fc358001870 tx=0x7fc3540091a0).ready entity=mon.0 client_cookie=96f3f727fa014ff2 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:00.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.255+0000 7fc360a42700 1 -- 172.21.15.162:0/1584117205 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc3580016d0 con 0x7fc35c05c4d0 2020-11-28T21:49:00.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.255+0000 7fc360a42700 1 -- 172.21.15.162:0/1584117205 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc35400b880 con 0x7fc35c05c4d0 2020-11-28T21:49:00.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.255+0000 7fc360a42700 1 -- 172.21.15.162:0/1584117205 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc35400b9f0 con 0x7fc35c05c4d0 2020-11-28T21:49:00.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.255+0000 7fc362a46700 1 -- 172.21.15.162:0/1584117205 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc35c05c4d0 msgr2=0x7fc35c0b30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:00.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.255+0000 7fc362a46700 1 --2- 172.21.15.162:0/1584117205 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc35c05c4d0 0x7fc35c0b30e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7fc358001870 tx=0x7fc3540091a0).stop 2020-11-28T21:49:00.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.255+0000 7fc362a46700 1 -- 172.21.15.162:0/1584117205 shutdown_connections 2020-11-28T21:49:00.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.255+0000 7fc362a46700 1 --2- 172.21.15.162:0/1584117205 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc35c05c4d0 0x7fc35c0b30e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:00.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.255+0000 7fc362a46700 1 -- 172.21.15.162:0/1584117205 >> 172.21.15.162:0/1584117205 conn(0x7fc35c0604c0 msgr2=0x7fc35c0608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:49:00.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.255+0000 7fc362a46700 1 -- 172.21.15.162:0/1584117205 shutdown_connections 2020-11-28T21:49:00.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.255+0000 7fc362a46700 1 -- 172.21.15.162:0/1584117205 wait complete. 2020-11-28T21:49:00.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.259+0000 7fc362a46700 1 Processor -- start 2020-11-28T21:49:00.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.259+0000 7fc362a46700 1 -- start start 2020-11-28T21:49:00.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.259+0000 7fc362a46700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc35c05c4d0 0x7fc35c13c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:00.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.259+0000 7fc362a46700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc35c0b2d00 0x7fc35c145800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:00.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.259+0000 7fc362a46700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc35c0b3ab0 0x7fc35c151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:00.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.259+0000 7fc362a46700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c0add60 con 0x7fc35c05c4d0 2020-11-28T21:49:00.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.259+0000 7fc362a46700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c0ad380 con 0x7fc35c0b3ab0 2020-11-28T21:49:00.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.259+0000 7fc362a46700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c0ad620 con 0x7fc35c0b2d00 2020-11-28T21:49:00.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.259+0000 7fc362245700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc35c0b3ab0 0x7fc35c151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:00.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.259+0000 7fc362245700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc35c0b3ab0 0x7fc35c151aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39572/0 (socket says 172.21.15.162:39572) 2020-11-28T21:49:00.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.259+0000 7fc362245700 1 -- 172.21.15.162:0/2793218872 learned_addr learned my addr 172.21.15.162:0/2793218872 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:49:00.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.259+0000 7fc361a44700 1 --2- 172.21.15.162:0/2793218872 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc35c05c4d0 0x7fc35c13c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:00.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.259+0000 7fc361243700 1 --2- 172.21.15.162:0/2793218872 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc35c0b2d00 0x7fc35c145800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:00.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.259+0000 7fc362245700 1 -- 172.21.15.162:0/2793218872 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc35c0b2d00 msgr2=0x7fc35c145800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:00.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.259+0000 7fc362245700 1 --2- 172.21.15.162:0/2793218872 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc35c0b2d00 0x7fc35c145800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:00.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.259+0000 7fc362245700 1 -- 172.21.15.162:0/2793218872 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc35c05c4d0 msgr2=0x7fc35c13c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:00.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.259+0000 7fc362245700 1 --2- 172.21.15.162:0/2793218872 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc35c05c4d0 0x7fc35c13c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:00.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.259+0000 7fc362245700 1 -- 172.21.15.162:0/2793218872 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc35c0ad8c0 con 0x7fc35c0b3ab0 2020-11-28T21:49:00.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.259+0000 7fc362245700 1 --2- 172.21.15.162:0/2793218872 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc35c0b3ab0 0x7fc35c151aa0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7fc358001870 tx=0x7fc354016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:00.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.263+0000 7fc34effd700 1 -- 172.21.15.162:0/2793218872 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc35400b9d0 con 0x7fc35c0b3ab0 2020-11-28T21:49:00.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.263+0000 7fc34effd700 1 -- 172.21.15.162:0/2793218872 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc354021480 con 0x7fc35c0b3ab0 2020-11-28T21:49:00.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.263+0000 7fc362a46700 1 -- 172.21.15.162:0/2793218872 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc35c148fd0 con 0x7fc35c0b3ab0 2020-11-28T21:49:00.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.263+0000 7fc34effd700 1 -- 172.21.15.162:0/2793218872 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc3540217e0 con 0x7fc35c0b3ab0 2020-11-28T21:49:00.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.263+0000 7fc362a46700 1 -- 172.21.15.162:0/2793218872 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc35c13cd50 con 0x7fc35c0b3ab0 2020-11-28T21:49:00.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.263+0000 7fc362a46700 1 -- 172.21.15.162:0/2793218872 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc35c057050 con 0x7fc35c0b3ab0 2020-11-28T21:49:00.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.263+0000 7fc34effd700 1 -- 172.21.15.162:0/2793218872 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc35400f070 con 0x7fc35c0b3ab0 2020-11-28T21:49:00.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.263+0000 7fc34effd700 1 --2- 172.21.15.162:0/2793218872 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc33c055330 0x7fc33c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:00.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.263+0000 7fc34effd700 1 -- 172.21.15.162:0/2793218872 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc35406ef20 con 0x7fc35c0b3ab0 2020-11-28T21:49:00.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.263+0000 7fc361a44700 1 --2- 172.21.15.162:0/2793218872 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc33c055330 0x7fc33c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:00.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.267+0000 7fc361a44700 1 --2- 172.21.15.162:0/2793218872 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc33c055330 0x7fc33c0577f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7fc35800d000 tx=0x7fc358000b80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:00.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.271+0000 7fc34effd700 1 -- 172.21.15.162:0/2793218872 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc3540452d0 con 0x7fc35c0b3ab0 2020-11-28T21:49:00.417 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.415+0000 7fc362a46700 1 -- 172.21.15.162:0/2793218872 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc35c0b6320 con 0x7fc35c0b3ab0 2020-11-28T21:49:00.418 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.419+0000 7fc34effd700 1 -- 172.21.15.162:0/2793218872 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v44) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc354015070 con 0x7fc35c0b3ab0 2020-11-28T21:49:00.419 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:49:00.420 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.419+0000 7fc362a46700 1 -- 172.21.15.162:0/2793218872 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc33c055330 msgr2=0x7fc33c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:00.420 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.419+0000 7fc362a46700 1 --2- 172.21.15.162:0/2793218872 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc33c055330 0x7fc33c0577f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7fc35800d000 tx=0x7fc358000b80).stop 2020-11-28T21:49:00.421 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.419+0000 7fc362a46700 1 -- 172.21.15.162:0/2793218872 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc35c0b3ab0 msgr2=0x7fc35c151aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:00.421 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.419+0000 7fc362a46700 1 --2- 172.21.15.162:0/2793218872 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc35c0b3ab0 0x7fc35c151aa0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7fc358001870 tx=0x7fc354016000).stop 2020-11-28T21:49:00.421 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.419+0000 7fc362a46700 1 -- 172.21.15.162:0/2793218872 shutdown_connections 2020-11-28T21:49:00.421 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.419+0000 7fc362a46700 1 --2- 172.21.15.162:0/2793218872 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc33c055330 0x7fc33c0577f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:00.421 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.419+0000 7fc362a46700 1 --2- 172.21.15.162:0/2793218872 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc35c0b3ab0 0x7fc35c151aa0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:00.422 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.419+0000 7fc362a46700 1 --2- 172.21.15.162:0/2793218872 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc35c05c4d0 0x7fc35c13c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:00.422 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.419+0000 7fc362a46700 1 --2- 172.21.15.162:0/2793218872 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc35c0b2d00 0x7fc35c145800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:00.422 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.419+0000 7fc362a46700 1 -- 172.21.15.162:0/2793218872 >> 172.21.15.162:0/2793218872 conn(0x7fc35c0604c0 msgr2=0x7fc35c0b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:49:00.422 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.419+0000 7fc362a46700 1 -- 172.21.15.162:0/2793218872 shutdown_connections 2020-11-28T21:49:00.423 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.419+0000 7fc362a46700 1 -- 172.21.15.162:0/2793218872 wait complete. 2020-11-28T21:49:00.436 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-28T21:49:00.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.551+0000 7fd71ee75700 1 Processor -- start 2020-11-28T21:49:00.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.551+0000 7fd71ee75700 1 -- start start 2020-11-28T21:49:00.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71ee75700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd7180610e0 0x7fd718061530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:00.555 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71ee75700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7180603f0 con 0x7fd718146a40 2020-11-28T21:49:00.555 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71ee75700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd71805c080 con 0x7fd718060010 2020-11-28T21:49:00.555 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71ee75700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd718145e70 con 0x7fd7180610e0 2020-11-28T21:49:00.555 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71de73700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fd718060010 0x7fd71805b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:46420/0 (socket says 172.21.15.162:46420) 2020-11-28T21:49:00.555 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71de73700 1 -- 172.21.15.162:0/2231238540 learned_addr learned my addr 172.21.15.162:0/2231238540 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:49:00.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71e674700 1 --2- 172.21.15.162:0/2231238540 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd7180610e0 0x7fd718061530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:00.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71ce71700 1 -- 172.21.15.162:0/2231238540 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 758149811 0 0) 0x7fd71805c080 con 0x7fd718060010 2020-11-28T21:49:00.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71ce71700 1 -- 172.21.15.162:0/2231238540 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd700003230 con 0x7fd718060010 2020-11-28T21:49:00.559 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71ce71700 1 -- 172.21.15.162:0/2231238540 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4008246604 0 0) 0x7fd7180603f0 con 0x7fd718146a40 2020-11-28T21:49:00.559 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71ce71700 1 -- 172.21.15.162:0/2231238540 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd71805c080 con 0x7fd718146a40 2020-11-28T21:49:00.559 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71ce71700 1 -- 172.21.15.162:0/2231238540 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2233404627 0 0) 0x7fd700003230 con 0x7fd718060010 2020-11-28T21:49:00.560 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71ce71700 1 -- 172.21.15.162:0/2231238540 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd7180610e0 msgr2=0x7fd718061530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:49:00.560 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71ce71700 1 --2- 172.21.15.162:0/2231238540 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd7180610e0 0x7fd718061530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:00.560 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71ce71700 1 -- 172.21.15.162:0/2231238540 >> v1:172.21.15.118:6789/0 conn(0x7fd718146a40 legacy=0x7fd71813fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:00.560 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71ce71700 1 -- 172.21.15.162:0/2231238540 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd718146060 con 0x7fd718060010 2020-11-28T21:49:00.561 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71ce71700 1 -- 172.21.15.162:0/2231238540 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7fd710001a50 con 0x7fd718060010 2020-11-28T21:49:00.561 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71ce71700 1 -- 172.21.15.162:0/2231238540 >> v1:172.21.15.162:6789/0 conn(0x7fd718060010 legacy=0x7fd71805b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:00.561 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71ce71700 1 --2- 172.21.15.162:0/2231238540 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd7180610e0 0x7fd7000038b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:00.561 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71ce71700 1 --2- 172.21.15.162:0/2231238540 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd700003df0 0x7fd7000044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:00.562 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71ce71700 1 --2- 172.21.15.162:0/2231238540 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd7000049c0 0x7fd700004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:00.562 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71ce71700 1 -- 172.21.15.162:0/2231238540 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd700005360 con 0x7fd7000049c0 2020-11-28T21:49:00.562 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71ce71700 1 -- 172.21.15.162:0/2231238540 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70000b720 con 0x7fd700003df0 2020-11-28T21:49:00.563 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71ce71700 1 -- 172.21.15.162:0/2231238540 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd70000b8e0 con 0x7fd7180610e0 2020-11-28T21:49:00.563 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71d672700 1 --2- 172.21.15.162:0/2231238540 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd700003df0 0x7fd7000044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:00.563 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71de73700 1 --2- 172.21.15.162:0/2231238540 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd7180610e0 0x7fd7000038b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:00.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71e674700 1 --2- 172.21.15.162:0/2231238540 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd7000049c0 0x7fd700004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:00.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71d672700 1 -- 172.21.15.162:0/2231238540 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd7180610e0 msgr2=0x7fd7000038b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:49:00.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71d672700 1 --2- 172.21.15.162:0/2231238540 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd7180610e0 0x7fd7000038b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:00.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71d672700 1 -- 172.21.15.162:0/2231238540 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd7000049c0 msgr2=0x7fd700004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:49:00.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71d672700 1 --2- 172.21.15.162:0/2231238540 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd7000049c0 0x7fd700004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:00.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71d672700 1 -- 172.21.15.162:0/2231238540 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd700003230 con 0x7fd700003df0 2020-11-28T21:49:00.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71d672700 1 --2- 172.21.15.162:0/2231238540 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd700003df0 0x7fd7000044d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7fd7000054a0 tx=0x7fd70c007b60).ready entity=mon.1 client_cookie=d0a85fabf9a4123d server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:00.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71ce71700 1 -- 172.21.15.162:0/2231238540 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd70c00c7d0 con 0x7fd700003df0 2020-11-28T21:49:00.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71ce71700 1 -- 172.21.15.162:0/2231238540 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd70c00c930 con 0x7fd700003df0 2020-11-28T21:49:00.566 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71ce71700 1 -- 172.21.15.162:0/2231238540 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd70c00cc90 con 0x7fd700003df0 2020-11-28T21:49:00.566 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71ee75700 1 -- 172.21.15.162:0/2231238540 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd700003df0 msgr2=0x7fd7000044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:00.567 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71ee75700 1 --2- 172.21.15.162:0/2231238540 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd700003df0 0x7fd7000044d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7fd7000054a0 tx=0x7fd70c007b60).stop 2020-11-28T21:49:00.567 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71e674700 1 -- 172.21.15.162:0/2231238540 reap_dead start 2020-11-28T21:49:00.567 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71ee75700 1 -- 172.21.15.162:0/2231238540 shutdown_connections 2020-11-28T21:49:00.567 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71ee75700 1 -- 172.21.15.162:0/2231238540 >> 172.21.15.162:0/2231238540 conn(0x7fd7180b5de0 msgr2=0x7fd7180b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:49:00.568 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71ee75700 1 -- 172.21.15.162:0/2231238540 shutdown_connections 2020-11-28T21:49:00.568 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.555+0000 7fd71ee75700 1 -- 172.21.15.162:0/2231238540 wait complete. 2020-11-28T21:49:00.568 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.559+0000 7fd71ee75700 1 Processor -- start 2020-11-28T21:49:00.568 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.559+0000 7fd71ee75700 1 -- start start 2020-11-28T21:49:00.571 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.559+0000 7fd71ee75700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd718064cd0 0x7fd718154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:00.571 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.559+0000 7fd71ee75700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd718066d90 0x7fd718154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:00.571 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.559+0000 7fd71ee75700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd718065910 0x7fd718142d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:00.571 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.559+0000 7fd71ee75700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7180650b0 con 0x7fd718065910 2020-11-28T21:49:00.572 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.559+0000 7fd71ee75700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd718067310 con 0x7fd718064cd0 2020-11-28T21:49:00.572 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.559+0000 7fd71ee75700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd718066060 con 0x7fd718066d90 2020-11-28T21:49:00.572 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.559+0000 7fd71de73700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd718064cd0 0x7fd718154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:00.572 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.559+0000 7fd71de73700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd718064cd0 0x7fd718154990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39588/0 (socket says 172.21.15.162:39588) 2020-11-28T21:49:00.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.559+0000 7fd71de73700 1 -- 172.21.15.162:0/3433862850 learned_addr learned my addr 172.21.15.162:0/3433862850 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:49:00.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.559+0000 7fd71e674700 1 --2- 172.21.15.162:0/3433862850 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd718065910 0x7fd718142d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:00.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.559+0000 7fd71d672700 1 --2- 172.21.15.162:0/3433862850 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd718066d90 0x7fd718154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:00.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.559+0000 7fd71de73700 1 -- 172.21.15.162:0/3433862850 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd718066d90 msgr2=0x7fd718154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:00.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.559+0000 7fd71de73700 1 --2- 172.21.15.162:0/3433862850 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd718066d90 0x7fd718154e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:00.574 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.559+0000 7fd71de73700 1 -- 172.21.15.162:0/3433862850 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd718065910 msgr2=0x7fd718142d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:00.574 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.559+0000 7fd71de73700 1 --2- 172.21.15.162:0/3433862850 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd718065910 0x7fd718142d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:00.574 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.559+0000 7fd71de73700 1 -- 172.21.15.162:0/3433862850 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd718066300 con 0x7fd718064cd0 2020-11-28T21:49:00.574 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.559+0000 7fd71de73700 1 --2- 172.21.15.162:0/3433862850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd718064cd0 0x7fd718154990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7fd71000d000 tx=0x7fd7100021d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:00.575 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.559+0000 7fd70affd700 1 -- 172.21.15.162:0/3433862850 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd71000fc90 con 0x7fd718064cd0 2020-11-28T21:49:00.575 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.559+0000 7fd71ee75700 1 -- 172.21.15.162:0/3433862850 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7180665a0 con 0x7fd718064cd0 2020-11-28T21:49:00.575 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.559+0000 7fd70affd700 1 -- 172.21.15.162:0/3433862850 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd71000fdf0 con 0x7fd718064cd0 2020-11-28T21:49:00.575 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.559+0000 7fd70affd700 1 -- 172.21.15.162:0/3433862850 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd710012540 con 0x7fd718064cd0 2020-11-28T21:49:00.575 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.563+0000 7fd71ee75700 1 -- 172.21.15.162:0/3433862850 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd718155370 con 0x7fd718064cd0 2020-11-28T21:49:00.576 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.563+0000 7fd71ee75700 1 -- 172.21.15.162:0/3433862850 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd718141b70 con 0x7fd718064cd0 2020-11-28T21:49:00.576 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.563+0000 7fd70affd700 1 -- 172.21.15.162:0/3433862850 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd71001b030 con 0x7fd718064cd0 2020-11-28T21:49:00.576 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.563+0000 7fd70affd700 1 --2- 172.21.15.162:0/3433862850 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd6f8055440 0x7fd6f8057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:00.577 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.563+0000 7fd70affd700 1 -- 172.21.15.162:0/3433862850 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd71006dd10 con 0x7fd718064cd0 2020-11-28T21:49:00.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.563+0000 7fd71d672700 1 --2- 172.21.15.162:0/3433862850 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd6f8055440 0x7fd6f8057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:00.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.563+0000 7fd71d672700 1 --2- 172.21.15.162:0/3433862850 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd6f8055440 0x7fd6f8057900 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7fd70c01b000 tx=0x7fd70c017000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:00.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.567+0000 7fd70affd700 1 -- 172.21.15.162:0/3433862850 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd710043f10 con 0x7fd718064cd0 2020-11-28T21:49:00.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.711+0000 7fd71ee75700 1 -- 172.21.15.162:0/3433862850 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd718066730 con 0x7fd718064cd0 2020-11-28T21:49:00.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.711+0000 7fd70affd700 1 -- 172.21.15.162:0/3433862850 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v44) v1 ==== 104+0+10 (secure 0 0 0) 0x7fd710019070 con 0x7fd718064cd0 2020-11-28T21:49:00.713 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:49:00.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.715+0000 7fd71ee75700 1 -- 172.21.15.162:0/3433862850 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd6f8055440 msgr2=0x7fd6f8057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:00.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.715+0000 7fd71ee75700 1 --2- 172.21.15.162:0/3433862850 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd6f8055440 0x7fd6f8057900 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7fd70c01b000 tx=0x7fd70c017000).stop 2020-11-28T21:49:00.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.715+0000 7fd71ee75700 1 -- 172.21.15.162:0/3433862850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd718064cd0 msgr2=0x7fd718154990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:00.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.715+0000 7fd71ee75700 1 --2- 172.21.15.162:0/3433862850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd718064cd0 0x7fd718154990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7fd71000d000 tx=0x7fd7100021d0).stop 2020-11-28T21:49:00.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.715+0000 7fd71ee75700 1 -- 172.21.15.162:0/3433862850 shutdown_connections 2020-11-28T21:49:00.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.715+0000 7fd71ee75700 1 --2- 172.21.15.162:0/3433862850 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd6f8055440 0x7fd6f8057900 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:00.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.715+0000 7fd71ee75700 1 --2- 172.21.15.162:0/3433862850 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd718065910 0x7fd718142d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:00.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.715+0000 7fd71ee75700 1 --2- 172.21.15.162:0/3433862850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd718064cd0 0x7fd718154990 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:00.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.715+0000 7fd71ee75700 1 --2- 172.21.15.162:0/3433862850 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd718066d90 0x7fd718154e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:00.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.715+0000 7fd71ee75700 1 -- 172.21.15.162:0/3433862850 >> 172.21.15.162:0/3433862850 conn(0x7fd7180b5de0 msgr2=0x7fd71805b970 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:49:00.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.715+0000 7fd71ee75700 1 -- 172.21.15.162:0/3433862850 shutdown_connections 2020-11-28T21:49:00.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.715+0000 7fd71ee75700 1 -- 172.21.15.162:0/3433862850 wait complete. 2020-11-28T21:49:00.726 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-28T21:49:00.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.843+0000 7fa53dc87700 1 Processor -- start 2020-11-28T21:49:00.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.843+0000 7fa53dc87700 1 -- start start 2020-11-28T21:49:00.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.843+0000 7fa53dc87700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa53805c980 0x7fa53805cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:00.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.843+0000 7fa53dc87700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa538095070 0x7fa538095c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:00.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.843+0000 7fa53dc87700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa538140860 con 0x7fa538097d90 2020-11-28T21:49:00.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.843+0000 7fa53dc87700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa538140b00 con 0x7fa53805c980 2020-11-28T21:49:00.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.843+0000 7fa53dc87700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa538140da0 con 0x7fa538095070 2020-11-28T21:49:00.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.843+0000 7fa5377fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa53805c980 0x7fa53805cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:00.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.843+0000 7fa52effd700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fa538097d90 0x7fa538094960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:46442/0 (socket says 172.21.15.162:46442) 2020-11-28T21:49:00.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.843+0000 7fa52effd700 1 -- 172.21.15.162:0/2848135205 learned_addr learned my addr 172.21.15.162:0/2848135205 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:49:00.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.843+0000 7fa537fff700 1 --2- 172.21.15.162:0/2848135205 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa538095070 0x7fa538095c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:00.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.843+0000 7fa536ffd700 1 -- 172.21.15.162:0/2848135205 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1924459655 0 0) 0x7fa538140860 con 0x7fa538097d90 2020-11-28T21:49:00.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.843+0000 7fa536ffd700 1 -- 172.21.15.162:0/2848135205 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa518003230 con 0x7fa538097d90 2020-11-28T21:49:00.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.847+0000 7fa5377fe700 1 -- 172.21.15.162:0/2848135205 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa538095070 msgr2=0x7fa538095c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:49:00.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.847+0000 7fa5377fe700 1 --2- 172.21.15.162:0/2848135205 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa538095070 0x7fa538095c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:00.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.847+0000 7fa536ffd700 1 -- 172.21.15.162:0/2848135205 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3739834352 0 0) 0x7fa518003230 con 0x7fa538097d90 2020-11-28T21:49:00.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.847+0000 7fa5377fe700 1 -- 172.21.15.162:0/2848135205 >> v1:172.21.15.162:6789/0 conn(0x7fa538097d90 legacy=0x7fa538094960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:00.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.847+0000 7fa5377fe700 1 -- 172.21.15.162:0/2848135205 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa538141040 con 0x7fa53805c980 2020-11-28T21:49:00.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.847+0000 7fa537fff700 1 --2- 172.21.15.162:0/2848135205 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa538095070 0x7fa538095c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:49:00.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.847+0000 7fa5377fe700 1 --2- 172.21.15.162:0/2848135205 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa53805c980 0x7fa53805cd60 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7fa520001870 tx=0x7fa528006dc0).ready entity=mon.1 client_cookie=21f2fe1c28466e37 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:00.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.847+0000 7fa536ffd700 1 -- 172.21.15.162:0/2848135205 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5200016d0 con 0x7fa53805c980 2020-11-28T21:49:00.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.847+0000 7fa536ffd700 1 -- 172.21.15.162:0/2848135205 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5280095d0 con 0x7fa53805c980 2020-11-28T21:49:00.852 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.847+0000 7fa536ffd700 1 -- 172.21.15.162:0/2848135205 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa528009740 con 0x7fa53805c980 2020-11-28T21:49:00.852 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.847+0000 7fa53dc87700 1 -- 172.21.15.162:0/2848135205 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa53805c980 msgr2=0x7fa53805cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:00.852 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.847+0000 7fa53dc87700 1 --2- 172.21.15.162:0/2848135205 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa53805c980 0x7fa53805cd60 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7fa520001870 tx=0x7fa528006dc0).stop 2020-11-28T21:49:00.856 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.847+0000 7fa53dc87700 1 -- 172.21.15.162:0/2848135205 shutdown_connections 2020-11-28T21:49:00.856 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.847+0000 7fa53dc87700 1 --2- 172.21.15.162:0/2848135205 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa53805c980 0x7fa53805cd60 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:00.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.847+0000 7fa53dc87700 1 --2- 172.21.15.162:0/2848135205 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa538095070 0x7fa538095c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:00.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.847+0000 7fa53dc87700 1 -- 172.21.15.162:0/2848135205 >> 172.21.15.162:0/2848135205 conn(0x7fa5380b6220 msgr2=0x7fa5380b21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:49:00.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.847+0000 7fa53dc87700 1 -- 172.21.15.162:0/2848135205 shutdown_connections 2020-11-28T21:49:00.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.847+0000 7fa53dc87700 1 -- 172.21.15.162:0/2848135205 wait complete. 2020-11-28T21:49:00.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.847+0000 7fa53dc87700 1 Processor -- start 2020-11-28T21:49:00.858 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.847+0000 7fa53dc87700 1 -- start start 2020-11-28T21:49:00.858 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.851+0000 7fa53dc87700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa53805c980 0x7fa53813fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:00.858 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.851+0000 7fa53dc87700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa538095070 0x7fa5380990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:00.858 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.851+0000 7fa53dc87700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa538097d90 0x7fa5380accd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:00.859 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.851+0000 7fa53dc87700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53809a500 con 0x7fa538097d90 2020-11-28T21:49:00.859 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.851+0000 7fa53dc87700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa538099750 con 0x7fa53805c980 2020-11-28T21:49:00.859 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.851+0000 7fa53dc87700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5380af470 con 0x7fa538095070 2020-11-28T21:49:00.859 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.851+0000 7fa5377fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa53805c980 0x7fa53813fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:00.859 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.851+0000 7fa5377fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa53805c980 0x7fa53813fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39602/0 (socket says 172.21.15.162:39602) 2020-11-28T21:49:00.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.851+0000 7fa5377fe700 1 -- 172.21.15.162:0/4165699001 learned_addr learned my addr 172.21.15.162:0/4165699001 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:49:00.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.851+0000 7fa52effd700 1 --2- 172.21.15.162:0/4165699001 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa538095070 0x7fa5380990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:00.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.851+0000 7fa537fff700 1 --2- 172.21.15.162:0/4165699001 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa538097d90 0x7fa5380accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:00.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.851+0000 7fa5377fe700 1 -- 172.21.15.162:0/4165699001 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa538095070 msgr2=0x7fa5380990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:00.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.851+0000 7fa5377fe700 1 --2- 172.21.15.162:0/4165699001 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa538095070 0x7fa5380990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:00.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.851+0000 7fa5377fe700 1 -- 172.21.15.162:0/4165699001 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa538097d90 msgr2=0x7fa5380accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:00.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.851+0000 7fa5377fe700 1 --2- 172.21.15.162:0/4165699001 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa538097d90 0x7fa5380accd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:00.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.851+0000 7fa5377fe700 1 -- 172.21.15.162:0/4165699001 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5380af710 con 0x7fa53805c980 2020-11-28T21:49:00.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.851+0000 7fa537fff700 1 --2- 172.21.15.162:0/4165699001 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa538097d90 0x7fa5380accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:49:00.862 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.851+0000 7fa5377fe700 1 --2- 172.21.15.162:0/4165699001 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa53805c980 0x7fa53813fdf0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7fa520001870 tx=0x7fa528015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:00.862 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.851+0000 7fa5357fa700 1 -- 172.21.15.162:0/4165699001 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa528020070 con 0x7fa53805c980 2020-11-28T21:49:00.862 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.851+0000 7fa53dc87700 1 -- 172.21.15.162:0/4165699001 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa53813dec0 con 0x7fa53805c980 2020-11-28T21:49:00.862 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.851+0000 7fa5357fa700 1 -- 172.21.15.162:0/4165699001 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa528011db0 con 0x7fa53805c980 2020-11-28T21:49:00.862 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.851+0000 7fa5357fa700 1 -- 172.21.15.162:0/4165699001 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa52801c660 con 0x7fa53805c980 2020-11-28T21:49:00.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.851+0000 7fa53dc87700 1 -- 172.21.15.162:0/4165699001 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5380ad1c0 con 0x7fa53805c980 2020-11-28T21:49:00.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.855+0000 7fa53dc87700 1 -- 172.21.15.162:0/4165699001 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5380998c0 con 0x7fa53805c980 2020-11-28T21:49:00.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.855+0000 7fa5357fa700 1 -- 172.21.15.162:0/4165699001 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa52800f070 con 0x7fa53805c980 2020-11-28T21:49:00.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.855+0000 7fa5357fa700 1 --2- 172.21.15.162:0/4165699001 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa51c0553b0 0x7fa51c057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:00.864 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.855+0000 7fa5357fa700 1 -- 172.21.15.162:0/4165699001 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa52806f560 con 0x7fa53805c980 2020-11-28T21:49:00.864 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.855+0000 7fa52effd700 1 --2- 172.21.15.162:0/4165699001 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa51c0553b0 0x7fa51c057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:00.864 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.855+0000 7fa52effd700 1 --2- 172.21.15.162:0/4165699001 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa51c0553b0 0x7fa51c057870 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7fa52000b000 tx=0x7fa520001c30).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:00.865 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:00.859+0000 7fa5357fa700 1 -- 172.21.15.162:0/4165699001 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa5380998c0 con 0x7fa53805c980 2020-11-28T21:49:01.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.003+0000 7fa53dc87700 1 -- 172.21.15.162:0/4165699001 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa538099890 con 0x7fa53805c980 2020-11-28T21:49:01.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.003+0000 7fa5357fa700 1 -- 172.21.15.162:0/4165699001 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v44) v1 ==== 100+0+10 (secure 0 0 0) 0x7fa5280457c0 con 0x7fa53805c980 2020-11-28T21:49:01.003 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:49:01.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.003+0000 7fa53dc87700 1 -- 172.21.15.162:0/4165699001 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa51c0553b0 msgr2=0x7fa51c057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:01.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.003+0000 7fa53dc87700 1 --2- 172.21.15.162:0/4165699001 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa51c0553b0 0x7fa51c057870 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7fa52000b000 tx=0x7fa520001c30).stop 2020-11-28T21:49:01.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.003+0000 7fa53dc87700 1 -- 172.21.15.162:0/4165699001 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa53805c980 msgr2=0x7fa53813fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:01.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.003+0000 7fa53dc87700 1 --2- 172.21.15.162:0/4165699001 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa53805c980 0x7fa53813fdf0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7fa520001870 tx=0x7fa528015000).stop 2020-11-28T21:49:01.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.003+0000 7fa53dc87700 1 -- 172.21.15.162:0/4165699001 shutdown_connections 2020-11-28T21:49:01.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.003+0000 7fa53dc87700 1 --2- 172.21.15.162:0/4165699001 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa51c0553b0 0x7fa51c057870 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:01.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.003+0000 7fa53dc87700 1 --2- 172.21.15.162:0/4165699001 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa538097d90 0x7fa5380accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:01.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.003+0000 7fa53dc87700 1 --2- 172.21.15.162:0/4165699001 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa53805c980 0x7fa53813fdf0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:01.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.003+0000 7fa53dc87700 1 --2- 172.21.15.162:0/4165699001 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa538095070 0x7fa5380990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:01.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.003+0000 7fa53dc87700 1 -- 172.21.15.162:0/4165699001 >> 172.21.15.162:0/4165699001 conn(0x7fa5380b6220 msgr2=0x7fa5380aa710 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:49:01.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.003+0000 7fa53dc87700 1 -- 172.21.15.162:0/4165699001 shutdown_connections 2020-11-28T21:49:01.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.003+0000 7fa53dc87700 1 -- 172.21.15.162:0/4165699001 wait complete. 2020-11-28T21:49:01.017 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-28T21:49:01.017 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:49:01.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.123+0000 7f1eba038700 1 Processor -- start 2020-11-28T21:49:01.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.123+0000 7f1eba038700 1 -- start start 2020-11-28T21:49:01.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.123+0000 7f1eba038700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1eb4098b30 0x7f1eb40994f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:01.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.123+0000 7f1eba038700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1eb4060710 0x7f1eb40b1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:01.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.123+0000 7f1eba038700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eb4098f10 con 0x7f1eb40b0000 2020-11-28T21:49:01.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.123+0000 7f1eba038700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eb40b1030 con 0x7f1eb4060710 2020-11-28T21:49:01.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.123+0000 7f1eba038700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb40b0480 con 0x7f1eb4098b30 2020-11-28T21:49:01.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.123+0000 7f1eb9837700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1eb4060710 0x7f1eb40b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:01.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.123+0000 7f1eb8835700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f1eb40b0000 0x7f1eb4060000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.162:36602/0 (socket says 172.21.15.162:36602) 2020-11-28T21:49:01.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.123+0000 7f1eb9036700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1eb4098b30 0x7f1eb40994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:01.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.123+0000 7f1eb8835700 1 -- 172.21.15.162:0/765143730 learned_addr learned my addr 172.21.15.162:0/765143730 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:49:01.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.127+0000 7f1eb9837700 1 -- 172.21.15.162:0/765143730 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1eb4098b30 msgr2=0x7f1eb40994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:49:01.129 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.127+0000 7f1eb9837700 1 --2- 172.21.15.162:0/765143730 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1eb4098b30 0x7f1eb40994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:01.129 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.127+0000 7f1eb9837700 1 -- 172.21.15.162:0/765143730 >> v1:172.21.15.118:6789/0 conn(0x7f1eb40b0000 legacy=0x7f1eb4060000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:01.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.127+0000 7f1eb9837700 1 -- 172.21.15.162:0/765143730 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb40b0680 con 0x7f1eb4060710 2020-11-28T21:49:01.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.127+0000 7f1eb9837700 1 --2- 172.21.15.162:0/765143730 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1eb4060710 0x7f1eb40b1580 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f1eb4099be0 tx=0x7f1eac0091a0).ready entity=mon.0 client_cookie=5d92acfa1aeb967e server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:01.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.127+0000 7f1ea7fff700 1 -- 172.21.15.162:0/765143730 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1eac00b9b0 con 0x7f1eb4060710 2020-11-28T21:49:01.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.127+0000 7f1ea7fff700 1 -- 172.21.15.162:0/765143730 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eac00bb10 con 0x7f1eb4060710 2020-11-28T21:49:01.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.127+0000 7f1ea7fff700 1 -- 172.21.15.162:0/765143730 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1eac00bc80 con 0x7f1eb4060710 2020-11-28T21:49:01.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.127+0000 7f1eba038700 1 -- 172.21.15.162:0/765143730 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1eb4060710 msgr2=0x7f1eb40b1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:01.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.127+0000 7f1eba038700 1 --2- 172.21.15.162:0/765143730 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1eb4060710 0x7f1eb40b1580 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f1eb4099be0 tx=0x7f1eac0091a0).stop 2020-11-28T21:49:01.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.127+0000 7f1eba038700 1 -- 172.21.15.162:0/765143730 shutdown_connections 2020-11-28T21:49:01.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.127+0000 7f1eba038700 1 --2- 172.21.15.162:0/765143730 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1eb4060710 0x7f1eb40b1580 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:01.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.127+0000 7f1eba038700 1 --2- 172.21.15.162:0/765143730 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1eb4098b30 0x7f1eb40994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:01.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.127+0000 7f1eba038700 1 -- 172.21.15.162:0/765143730 >> 172.21.15.162:0/765143730 conn(0x7f1eb405b970 msgr2=0x7f1eb405bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:49:01.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.127+0000 7f1eba038700 1 -- 172.21.15.162:0/765143730 shutdown_connections 2020-11-28T21:49:01.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.127+0000 7f1eba038700 1 -- 172.21.15.162:0/765143730 wait complete. 2020-11-28T21:49:01.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.127+0000 7f1eba038700 1 Processor -- start 2020-11-28T21:49:01.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.127+0000 7f1eba038700 1 -- start start 2020-11-28T21:49:01.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.131+0000 7f1eba038700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1eb4060710 0x7f1eb40ab620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:01.138 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.131+0000 7f1eba038700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1eb4098b30 0x7f1eb4066730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:01.138 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.131+0000 7f1eba038700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1eb40b0000 0x7f1eb4066c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:01.138 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.131+0000 7f1eba038700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb4064bd0 con 0x7f1eb40b0000 2020-11-28T21:49:01.138 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.131+0000 7f1eba038700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb40add40 con 0x7f1eb4060710 2020-11-28T21:49:01.139 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.131+0000 7f1eba038700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb40adf70 con 0x7f1eb4098b30 2020-11-28T21:49:01.139 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.131+0000 7f1eb9036700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1eb4060710 0x7f1eb40ab620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:01.139 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.131+0000 7f1eb9036700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1eb4060710 0x7f1eb40ab620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39616/0 (socket says 172.21.15.162:39616) 2020-11-28T21:49:01.139 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.131+0000 7f1eb9036700 1 -- 172.21.15.162:0/115287666 learned_addr learned my addr 172.21.15.162:0/115287666 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:49:01.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.131+0000 7f1eb8835700 1 --2- 172.21.15.162:0/115287666 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1eb4098b30 0x7f1eb4066730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:01.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.131+0000 7f1eb9837700 1 --2- 172.21.15.162:0/115287666 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1eb40b0000 0x7f1eb4066c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:01.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.131+0000 7f1eb9036700 1 -- 172.21.15.162:0/115287666 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1eb4098b30 msgr2=0x7f1eb4066730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:01.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.131+0000 7f1eb9036700 1 --2- 172.21.15.162:0/115287666 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1eb4098b30 0x7f1eb4066730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:01.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.131+0000 7f1eb9036700 1 -- 172.21.15.162:0/115287666 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1eb40b0000 msgr2=0x7f1eb4066c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:01.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.131+0000 7f1eb9036700 1 --2- 172.21.15.162:0/115287666 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1eb40b0000 0x7f1eb4066c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:01.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.131+0000 7f1eb9036700 1 -- 172.21.15.162:0/115287666 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb4067110 con 0x7f1eb4060710 2020-11-28T21:49:01.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.131+0000 7f1eb9036700 1 --2- 172.21.15.162:0/115287666 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1eb4060710 0x7f1eb40ab620 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f1eb000c000 tx=0x7f1eb0006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:01.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.131+0000 7f1ea67fc700 1 -- 172.21.15.162:0/115287666 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1eb000ca10 con 0x7f1eb4060710 2020-11-28T21:49:01.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.131+0000 7f1ea67fc700 1 -- 172.21.15.162:0/115287666 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb000cb70 con 0x7f1eb4060710 2020-11-28T21:49:01.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.131+0000 7f1eba038700 1 -- 172.21.15.162:0/115287666 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eb40ae230 con 0x7f1eb4060710 2020-11-28T21:49:01.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.131+0000 7f1ea67fc700 1 -- 172.21.15.162:0/115287666 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1eb00123f0 con 0x7f1eb4060710 2020-11-28T21:49:01.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.131+0000 7f1eba038700 1 -- 172.21.15.162:0/115287666 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb40ad740 con 0x7f1eb4060710 2020-11-28T21:49:01.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.131+0000 7f1eba038700 1 -- 172.21.15.162:0/115287666 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eb40ae3c0 con 0x7f1eb4060710 2020-11-28T21:49:01.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.131+0000 7f1ea67fc700 1 -- 172.21.15.162:0/115287666 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1eb0012550 con 0x7f1eb4060710 2020-11-28T21:49:01.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.135+0000 7f1ea67fc700 1 --2- 172.21.15.162:0/115287666 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1e9c055330 0x7f1e9c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:01.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.135+0000 7f1eb8835700 1 --2- 172.21.15.162:0/115287666 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1e9c055330 0x7f1e9c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:01.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.135+0000 7f1ea67fc700 1 -- 172.21.15.162:0/115287666 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1eb006e420 con 0x7f1eb4060710 2020-11-28T21:49:01.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.135+0000 7f1eb8835700 1 --2- 172.21.15.162:0/115287666 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1e9c055330 0x7f1e9c0577f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f1ea800d000 tx=0x7f1ea8007bb0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:01.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.139+0000 7f1ea67fc700 1 -- 172.21.15.162:0/115287666 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1eb00445b0 con 0x7f1eb4060710 2020-11-28T21:49:01.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.323+0000 7f1eba038700 1 -- 172.21.15.162:0/115287666 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1eb4057050 con 0x7f1eb4060710 2020-11-28T21:49:01.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.323+0000 7f1ea67fc700 1 -- 172.21.15.162:0/115287666 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+2497 (secure 0 0 0) 0x7f1eb0047bd0 con 0x7f1eb4060710 2020-11-28T21:49:01.324 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:49:01.325 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":15,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4856,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/913296056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":913296056},{"type":"v1","addr":"172.21.15.118:6833","nonce":913296056}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":13},{"gid":5209,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6835/1318110356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1318110356},{"type":"v1","addr":"172.21.15.162:6835","nonce":1318110356}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:48:26.782354+0000","modified":"2020-11-28T21:48:27.785277+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5212},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5212":{"gid":5212,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/2706248574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2706248574},{"type":"v1","addr":"172.21.15.162:6837","nonce":2706248574}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-28T21:49:01.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.327+0000 7f1eba038700 1 -- 172.21.15.162:0/115287666 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1e9c055330 msgr2=0x7f1e9c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:01.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.327+0000 7f1eba038700 1 --2- 172.21.15.162:0/115287666 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1e9c055330 0x7f1e9c0577f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f1ea800d000 tx=0x7f1ea8007bb0).stop 2020-11-28T21:49:01.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.327+0000 7f1eba038700 1 -- 172.21.15.162:0/115287666 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1eb4060710 msgr2=0x7f1eb40ab620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:01.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.327+0000 7f1eba038700 1 --2- 172.21.15.162:0/115287666 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1eb4060710 0x7f1eb40ab620 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f1eb000c000 tx=0x7f1eb0006dc0).stop 2020-11-28T21:49:01.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.327+0000 7f1eba038700 1 -- 172.21.15.162:0/115287666 shutdown_connections 2020-11-28T21:49:01.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.327+0000 7f1eba038700 1 --2- 172.21.15.162:0/115287666 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1e9c055330 0x7f1e9c0577f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:01.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.327+0000 7f1eba038700 1 --2- 172.21.15.162:0/115287666 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1eb40b0000 0x7f1eb4066c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:01.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.327+0000 7f1eba038700 1 --2- 172.21.15.162:0/115287666 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1eb4060710 0x7f1eb40ab620 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:01.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.327+0000 7f1eba038700 1 --2- 172.21.15.162:0/115287666 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1eb4098b30 0x7f1eb4066730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:01.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.327+0000 7f1eba038700 1 -- 172.21.15.162:0/115287666 >> 172.21.15.162:0/115287666 conn(0x7f1eb405b970 msgr2=0x7f1eb40b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:49:01.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.327+0000 7f1eba038700 1 -- 172.21.15.162:0/115287666 shutdown_connections 2020-11-28T21:49:01.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:01.327+0000 7f1eba038700 1 -- 172.21.15.162:0/115287666 wait complete. 2020-11-28T21:49:01.330 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 15 2020-11-28T21:49:01.639 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-28T21:48:26.782354+0000', 'modified': '2020-11-28T21:48:27.785277+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5212}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5212': {'gid': 5212, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6837/2706248574', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6836', 'nonce': 2706248574}, {'type': 'v1', 'addr': '172.21.15.162:6837', 'nonce': 2706248574}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-28T21:49:01.640 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-28T21:49:01.640 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-28T21:49:01.771 INFO:teuthology.orchestra.run.smithi162.stdout:{"cluster_fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","whoami":0,"id":5212,"want_state":"up:active","state":"up:active","rank_uptime":34.976302609000001,"mdsmap_epoch":15,"osdmap_epoch":44,"osdmap_epoch_barrier":43,"uptime":39.368143011000001} 2020-11-28T21:49:01.772 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", "id": 5212, "mdsmap_epoch": 15, "osdmap_epoch": 44, "osdmap_epoch_barrier": 43, "rank_uptime": 34.976302609, "state": "up:active", "uptime": 39.368143011, "want_state": "up:active", "whoami": 0 } 2020-11-28T21:49:01.772 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-11-28T21:49:01.772 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-11-28T21:49:01.773 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi192.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-11-28T21:49:01.773 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:49:01.774 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:49:01.783 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:49:01.838 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:49:01.838 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:49:01.893 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:49:01.894 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:49:01.895 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:49:01.896 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:49:01.942 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-11-28T21:49:01.942 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:49:01.984 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:49:01.999 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:49:02.000 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:49:02.058 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:49:02.059 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:49:02.060 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:49:02.061 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:49:02.062 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:2020-11-28T21:49:02.061+0000 7facc99270c0 -1 init, newargv = 0x5571f7392550 newargc=9 2020-11-28T21:49:02.062 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:ceph-fuse[11606]: starting ceph client 2020-11-28T21:49:02.071 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:ceph-fuse[11606]: starting fuse 2020-11-28T21:49:03.168 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:49:03.184 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:49:03.184 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:49:03.241 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:49:03.242 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:49:03.243 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:49:03.243 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:49:03.289 INFO:teuthology.orchestra.run.smithi192.stdout:49 2020-11-28T21:49:03.289 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2020-11-28T21:49:03.289 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:49:03.290 INFO:teuthology.orchestra.run.smithi192:> sudo python3 -c ' 2020-11-28T21:49:03.290 INFO:teuthology.orchestra.run.smithi192:> import glob 2020-11-28T21:49:03.291 INFO:teuthology.orchestra.run.smithi192:> import re 2020-11-28T21:49:03.291 INFO:teuthology.orchestra.run.smithi192:> import os 2020-11-28T21:49:03.291 INFO:teuthology.orchestra.run.smithi192:> import subprocess 2020-11-28T21:49:03.292 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:49:03.292 INFO:teuthology.orchestra.run.smithi192:> def find_socket(client_name): 2020-11-28T21:49:03.293 INFO:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-11-28T21:49:03.293 INFO:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2020-11-28T21:49:03.293 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:49:03.294 INFO:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2020-11-28T21:49:03.294 INFO:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2020-11-28T21:49:03.294 INFO:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2020-11-28T21:49:03.295 INFO:teuthology.orchestra.run.smithi192:> return files[0] 2020-11-28T21:49:03.295 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:49:03.296 INFO:teuthology.orchestra.run.smithi192:> for f in files: 2020-11-28T21:49:03.296 INFO:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-11-28T21:49:03.296 INFO:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2020-11-28T21:49:03.297 INFO:teuthology.orchestra.run.smithi192:> return f 2020-11-28T21:49:03.297 INFO:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-11-28T21:49:03.297 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:49:03.298 INFO:teuthology.orchestra.run.smithi192:> print(find_socket("client.0")) 2020-11-28T21:49:03.298 INFO:teuthology.orchestra.run.smithi192:> ' 2020-11-28T21:49:03.381 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.0.11606.asok 2020-11-28T21:49:03.386 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.11606.asok 2020-11-28T21:49:03.387 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:49:03.388 INFO:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.11606.asok status 2020-11-28T21:49:03.486 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2020-11-28T21:49:03.487 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2020-11-28T21:49:03.487 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "a83a3572a957356977342f461361c1cd3ec06dbd", 2020-11-28T21:49:03.488 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 15.2.6-595-ga83a3572 (a83a3572a957356977342f461361c1cd3ec06dbd) octopus (stable)", 2020-11-28T21:49:03.488 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "0", 2020-11-28T21:49:03.488 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2020-11-28T21:49:03.488 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-11-28T21:49:03.489 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "11606", 2020-11-28T21:49:03.489 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2020-11-28T21:49:03.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:49:03.490 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2020-11-28T21:49:03.490 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2020-11-28T21:49:03.490 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 5030, 2020-11-28T21:49:03.490 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2020-11-28T21:49:03.491 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2020-11-28T21:49:03.491 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2020-11-28T21:49:03.491 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 5030 2020-11-28T21:49:03.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:49:03.492 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2020-11-28T21:49:03.492 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2020-11-28T21:49:03.492 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2020-11-28T21:49:03.493 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 796185475 2020-11-28T21:49:03.493 INFO:teuthology.orchestra.run.smithi192.stdout: } 2020-11-28T21:49:03.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:49:03.493 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2020-11-28T21:49:03.494 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2020-11-28T21:49:03.494 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2020-11-28T21:49:03.494 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 796185475 2020-11-28T21:49:03.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:49:03.497 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.5030 172.21.15.192:0/796185475", 2020-11-28T21:49:03.497 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "172.21.15.192:0/796185475", 2020-11-28T21:49:03.497 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2020-11-28T21:49:03.498 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 15, 2020-11-28T21:49:03.498 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 44, 2020-11-28T21:49:03.498 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2020-11-28T21:49:03.498 INFO:teuthology.orchestra.run.smithi192.stdout: "blacklisted": false 2020-11-28T21:49:03.498 INFO:teuthology.orchestra.run.smithi192.stdout:} 2020-11-28T21:49:03.500 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:49:03.500 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:49:03.501 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:49:03.508 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2020-11-28T21:49:03.509 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-28T21:49:03.509 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:49:03.510 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:49:06.925 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:49:06.927 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:49:06.930 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:49:06.933 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:49:37.057 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:49:37.062 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:49:37.067 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:49:37.075 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:49:52.400 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-28T21:49:52.401 INFO:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:49:52.420 INFO:teuthology.orchestra.run.smithi118:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:49:52.439 INFO:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:49:52.459 INFO:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:49:52.479 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-28T21:49:52.480 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:49:52.481 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:49:52.483 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:49:52.534 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2020-11-28T21:49:52.535 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-28T21:49:52.535 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2020-11-28T21:49:52.536 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-28T21:49:52.536 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:49:52.626 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:ceph-fuse[11606]: fuse finished with error 0 and tester_r 0 2020-11-28T21:49:52.628 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:49:52.628 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:49:52.629 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:49:52.636 INFO:teuthology.orchestra.run.smithi192.stdout:ext2/ext3 2020-11-28T21:49:52.638 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-11-28T21:49:52.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-28T21:49:52.729 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:49:52.736 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:49:52.736 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:49:52.737 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:49:52.748 INFO:teuthology.orchestra.run.smithi096.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-11-28T21:49:52.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:49:52.749 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2020-11-28T21:49:52.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-28T21:49:52.750 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:49:52.796 INFO:teuthology.orchestra.run.smithi096.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-11-28T21:49:52.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:49:52.799 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config rm mon mon_allow_pool_delete 2020-11-28T21:49:52.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.927+0000 7ff3ebfff700 1 Processor -- start 2020-11-28T21:49:52.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.927+0000 7ff3ebfff700 1 -- start start 2020-11-28T21:49:52.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.927+0000 7ff3ebfff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff3ec064c70 0x7ff3ec066290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:52.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.927+0000 7ff3ebfff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff3ec13f6c0 0x7ff3ec066780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:52.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.927+0000 7ff3ebfff700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3ec065050 con 0x7ff3ec066c70 2020-11-28T21:49:52.931 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.927+0000 7ff3ebfff700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3ec13fc40 con 0x7ff3ec064c70 2020-11-28T21:49:52.931 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.927+0000 7ff3ebfff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec13fee0 con 0x7ff3ec13f6c0 2020-11-28T21:49:52.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.927+0000 7ff3eaffd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff3ec064c70 0x7ff3ec066290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:52.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.927+0000 7ff3eaffd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff3ec064c70 0x7ff3ec066290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39624/0 (socket says 172.21.15.162:39624) 2020-11-28T21:49:52.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.927+0000 7ff3eaffd700 1 -- 172.21.15.162:0/4069978216 learned_addr learned my addr 172.21.15.162:0/4069978216 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:49:52.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.927+0000 7ff3e27fc700 1 --2- 172.21.15.162:0/4069978216 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff3ec13f6c0 0x7ff3ec066780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:52.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.927+0000 7ff3eaffd700 1 -- 172.21.15.162:0/4069978216 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff3ec13f6c0 msgr2=0x7ff3ec066780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:49:52.933 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.927+0000 7ff3eaffd700 1 --2- 172.21.15.162:0/4069978216 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff3ec13f6c0 0x7ff3ec066780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:52.933 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.927+0000 7ff3eaffd700 1 -- 172.21.15.162:0/4069978216 >> v1:172.21.15.162:6789/0 conn(0x7ff3ec066c70 legacy=0x7ff3ec142c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:52.933 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.927+0000 7ff3eaffd700 1 -- 172.21.15.162:0/4069978216 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ec140180 con 0x7ff3ec064c70 2020-11-28T21:49:52.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.927+0000 7ff3eaffd700 1 --2- 172.21.15.162:0/4069978216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3ec064c70 0x7ff3ec066290 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7ff3dc002f30 tx=0x7ff3dc006e80).ready entity=mon.1 client_cookie=f2427e128dbfb37b server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:52.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.927+0000 7ff3ea7fc700 1 -- 172.21.15.162:0/4069978216 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3dc0099e0 con 0x7ff3ec064c70 2020-11-28T21:49:52.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.931+0000 7ff3ea7fc700 1 -- 172.21.15.162:0/4069978216 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3dc009b40 con 0x7ff3ec064c70 2020-11-28T21:49:52.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.931+0000 7ff3ea7fc700 1 -- 172.21.15.162:0/4069978216 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3dc009ea0 con 0x7ff3ec064c70 2020-11-28T21:49:52.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.931+0000 7ff3ebfff700 1 -- 172.21.15.162:0/4069978216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3ec064c70 msgr2=0x7ff3ec066290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:52.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.931+0000 7ff3ebfff700 1 --2- 172.21.15.162:0/4069978216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3ec064c70 0x7ff3ec066290 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7ff3dc002f30 tx=0x7ff3dc006e80).stop 2020-11-28T21:49:52.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.931+0000 7ff3ebfff700 1 -- 172.21.15.162:0/4069978216 shutdown_connections 2020-11-28T21:49:52.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.931+0000 7ff3ebfff700 1 --2- 172.21.15.162:0/4069978216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3ec064c70 0x7ff3ec066290 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:52.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.931+0000 7ff3ebfff700 1 --2- 172.21.15.162:0/4069978216 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff3ec13f6c0 0x7ff3ec066780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:52.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.931+0000 7ff3ebfff700 1 -- 172.21.15.162:0/4069978216 >> 172.21.15.162:0/4069978216 conn(0x7ff3ec0b5da0 msgr2=0x7ff3ec0b6180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:49:52.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.931+0000 7ff3ebfff700 1 -- 172.21.15.162:0/4069978216 shutdown_connections 2020-11-28T21:49:52.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.931+0000 7ff3ebfff700 1 -- 172.21.15.162:0/4069978216 wait complete. 2020-11-28T21:49:52.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.931+0000 7ff3ebfff700 1 Processor -- start 2020-11-28T21:49:52.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.931+0000 7ff3ebfff700 1 -- start start 2020-11-28T21:49:52.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.931+0000 7ff3ebfff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff3ec064c70 0x7ff3ec145350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:52.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.931+0000 7ff3ebfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff3ec066c70 0x7ff3ec1511f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:52.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.931+0000 7ff3ebfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3ec13f6c0 0x7ff3ec1516e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:52.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.931+0000 7ff3ebfff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec13b3b0 con 0x7ff3ec066c70 2020-11-28T21:49:52.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.931+0000 7ff3ebfff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec0b3bb0 con 0x7ff3ec13f6c0 2020-11-28T21:49:52.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.931+0000 7ff3ebfff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec151bd0 con 0x7ff3ec064c70 2020-11-28T21:49:52.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.931+0000 7ff3eb7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3ec13f6c0 0x7ff3ec1516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:52.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.931+0000 7ff3eb7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3ec13f6c0 0x7ff3ec1516e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39634/0 (socket says 172.21.15.162:39634) 2020-11-28T21:49:52.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.931+0000 7ff3eb7fe700 1 -- 172.21.15.162:0/2539622676 learned_addr learned my addr 172.21.15.162:0/2539622676 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:49:52.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.931+0000 7ff3e27fc700 1 --2- 172.21.15.162:0/2539622676 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff3ec066c70 0x7ff3ec1511f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:52.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.931+0000 7ff3eaffd700 1 --2- 172.21.15.162:0/2539622676 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff3ec064c70 0x7ff3ec145350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:52.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.935+0000 7ff3eb7fe700 1 -- 172.21.15.162:0/2539622676 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff3ec064c70 msgr2=0x7ff3ec145350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:52.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.935+0000 7ff3eb7fe700 1 --2- 172.21.15.162:0/2539622676 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff3ec064c70 0x7ff3ec145350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:52.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.935+0000 7ff3eb7fe700 1 -- 172.21.15.162:0/2539622676 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff3ec066c70 msgr2=0x7ff3ec1511f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:52.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.935+0000 7ff3eb7fe700 1 --2- 172.21.15.162:0/2539622676 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff3ec066c70 0x7ff3ec1511f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:52.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.935+0000 7ff3eb7fe700 1 -- 172.21.15.162:0/2539622676 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ec13b5e0 con 0x7ff3ec13f6c0 2020-11-28T21:49:52.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.935+0000 7ff3eb7fe700 1 --2- 172.21.15.162:0/2539622676 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3ec13f6c0 0x7ff3ec1516e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7ff3ec0646f0 tx=0x7ff3e4007ed0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:52.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.935+0000 7ff3e8ff9700 1 -- 172.21.15.162:0/2539622676 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3e400ce40 con 0x7ff3ec13f6c0 2020-11-28T21:49:52.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.935+0000 7ff3ebfff700 1 -- 172.21.15.162:0/2539622676 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3ec13b880 con 0x7ff3ec13f6c0 2020-11-28T21:49:52.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.935+0000 7ff3e8ff9700 1 -- 172.21.15.162:0/2539622676 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e400a7b0 con 0x7ff3ec13f6c0 2020-11-28T21:49:52.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.935+0000 7ff3e8ff9700 1 -- 172.21.15.162:0/2539622676 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3e400ab10 con 0x7ff3ec13f6c0 2020-11-28T21:49:52.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.935+0000 7ff3ebfff700 1 -- 172.21.15.162:0/2539622676 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3ec0b4280 con 0x7ff3ec13f6c0 2020-11-28T21:49:52.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.935+0000 7ff3ebfff700 1 -- 172.21.15.162:0/2539622676 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3ec057050 con 0x7ff3ec13f6c0 2020-11-28T21:49:52.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.935+0000 7ff3e8ff9700 1 -- 172.21.15.162:0/2539622676 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff3e4019030 con 0x7ff3ec13f6c0 2020-11-28T21:49:52.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.935+0000 7ff3e8ff9700 1 --2- 172.21.15.162:0/2539622676 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff3d0055440 0x7ff3d0057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:52.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.935+0000 7ff3e8ff9700 1 -- 172.21.15.162:0/2539622676 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff3e406d850 con 0x7ff3ec13f6c0 2020-11-28T21:49:52.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.935+0000 7ff3eaffd700 1 --2- 172.21.15.162:0/2539622676 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff3d0055440 0x7ff3d0057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:52.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.939+0000 7ff3eaffd700 1 --2- 172.21.15.162:0/2539622676 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff3d0055440 0x7ff3d0057900 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7ff3dc01d000 tx=0x7ff3dc018000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:52.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:52.943+0000 7ff3e8ff9700 1 -- 172.21.15.162:0/2539622676 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff3e4043a50 con 0x7ff3ec13f6c0 2020-11-28T21:49:53.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.079+0000 7ff3ebfff700 1 -- 172.21.15.162:0/2539622676 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ff3ec06a760 con 0x7ff3ec13f6c0 2020-11-28T21:49:53.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.091+0000 7ff3e8ff9700 1 -- 172.21.15.162:0/2539622676 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v5) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff3e4047070 con 0x7ff3ec13f6c0 2020-11-28T21:49:53.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.091+0000 7ff3ebfff700 1 -- 172.21.15.162:0/2539622676 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff3d0055440 msgr2=0x7ff3d0057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:53.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.091+0000 7ff3ebfff700 1 --2- 172.21.15.162:0/2539622676 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff3d0055440 0x7ff3d0057900 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7ff3dc01d000 tx=0x7ff3dc018000).stop 2020-11-28T21:49:53.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.091+0000 7ff3ebfff700 1 -- 172.21.15.162:0/2539622676 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3ec13f6c0 msgr2=0x7ff3ec1516e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:53.094 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.091+0000 7ff3ebfff700 1 --2- 172.21.15.162:0/2539622676 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3ec13f6c0 0x7ff3ec1516e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7ff3ec0646f0 tx=0x7ff3e4007ed0).stop 2020-11-28T21:49:53.094 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.091+0000 7ff3ebfff700 1 -- 172.21.15.162:0/2539622676 shutdown_connections 2020-11-28T21:49:53.094 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.091+0000 7ff3ebfff700 1 --2- 172.21.15.162:0/2539622676 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff3d0055440 0x7ff3d0057900 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:53.095 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.091+0000 7ff3ebfff700 1 --2- 172.21.15.162:0/2539622676 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff3ec064c70 0x7ff3ec145350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:53.095 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.091+0000 7ff3ebfff700 1 --2- 172.21.15.162:0/2539622676 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3ec13f6c0 0x7ff3ec1516e0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:53.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.091+0000 7ff3ebfff700 1 --2- 172.21.15.162:0/2539622676 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff3ec066c70 0x7ff3ec1511f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:53.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.091+0000 7ff3ebfff700 1 -- 172.21.15.162:0/2539622676 >> 172.21.15.162:0/2539622676 conn(0x7ff3ec0b5da0 msgr2=0x7ff3ec1397b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:49:53.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.091+0000 7ff3ebfff700 1 -- 172.21.15.162:0/2539622676 shutdown_connections 2020-11-28T21:49:53.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.091+0000 7ff3ebfff700 1 -- 172.21.15.162:0/2539622676 wait complete. 2020-11-28T21:49:53.106 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead' 2020-11-28T21:49:53.216 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.219+0000 7f01f2e65700 1 Processor -- start 2020-11-28T21:49:53.217 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.219+0000 7f01f2e65700 1 -- start start 2020-11-28T21:49:53.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.219+0000 7f01f2e65700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01ec0b2290 0x7f01ec0ae4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:53.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.219+0000 7f01f2e65700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01ec0b2670 con 0x7f01ec0a8550 2020-11-28T21:49:53.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.219+0000 7f01f2e65700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01ec0afa00 con 0x7f01ec0af480 2020-11-28T21:49:53.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.219+0000 7f01f2e65700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec0afca0 con 0x7f01ec0b2290 2020-11-28T21:49:53.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.219+0000 7f01f2664700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f01ec0af480 0x7f01ec1417a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:46484/0 (socket says 172.21.15.162:46484) 2020-11-28T21:49:53.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.219+0000 7f01f2664700 1 -- 172.21.15.162:0/3591970935 learned_addr learned my addr 172.21.15.162:0/3591970935 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:49:53.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.219+0000 7f01f1e63700 1 --2- 172.21.15.162:0/3591970935 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01ec0b2290 0x7f01ec0ae4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:53.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.219+0000 7f01f0e61700 1 -- 172.21.15.162:0/3591970935 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1448522303 0 0) 0x7f01ec0afa00 con 0x7f01ec0af480 2020-11-28T21:49:53.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.219+0000 7f01f0e61700 1 -- 172.21.15.162:0/3591970935 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01d8003230 con 0x7f01ec0af480 2020-11-28T21:49:53.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.219+0000 7f01f0e61700 1 -- 172.21.15.162:0/3591970935 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2884531887 0 0) 0x7f01ec0b2670 con 0x7f01ec0a8550 2020-11-28T21:49:53.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.219+0000 7f01f0e61700 1 -- 172.21.15.162:0/3591970935 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01ec0afa00 con 0x7f01ec0a8550 2020-11-28T21:49:53.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.219+0000 7f01f1e63700 1 -- 172.21.15.162:0/3591970935 >> v1:172.21.15.162:6789/0 conn(0x7f01ec0af480 legacy=0x7f01ec1417a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:53.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.219+0000 7f01f1e63700 1 -- 172.21.15.162:0/3591970935 >> v1:172.21.15.118:6789/0 conn(0x7f01ec0a8550 legacy=0x7f01ec0ae9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:53.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.219+0000 7f01f1e63700 1 -- 172.21.15.162:0/3591970935 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ec0acb80 con 0x7f01ec0b2290 2020-11-28T21:49:53.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.219+0000 7f01f1e63700 1 --2- 172.21.15.162:0/3591970935 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01ec0b2290 0x7f01ec0ae4c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f01e4002f30 tx=0x7f01e4006dc0).ready entity=mon.2 client_cookie=a8710924085cca60 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:53.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.219+0000 7f01f0e61700 1 -- 172.21.15.162:0/3591970935 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01e4009730 con 0x7f01ec0b2290 2020-11-28T21:49:53.224 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.219+0000 7f01f0e61700 1 -- 172.21.15.162:0/3591970935 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e4009890 con 0x7f01ec0b2290 2020-11-28T21:49:53.224 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.219+0000 7f01f0e61700 1 -- 172.21.15.162:0/3591970935 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01e4009bf0 con 0x7f01ec0b2290 2020-11-28T21:49:53.224 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.219+0000 7f01f2e65700 1 -- 172.21.15.162:0/3591970935 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01ec0b2290 msgr2=0x7f01ec0ae4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:53.224 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.219+0000 7f01f2e65700 1 --2- 172.21.15.162:0/3591970935 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01ec0b2290 0x7f01ec0ae4c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f01e4002f30 tx=0x7f01e4006dc0).stop 2020-11-28T21:49:53.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.219+0000 7f01f2e65700 1 -- 172.21.15.162:0/3591970935 shutdown_connections 2020-11-28T21:49:53.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.219+0000 7f01f2e65700 1 --2- 172.21.15.162:0/3591970935 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01ec0b2290 0x7f01ec0ae4c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:53.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.219+0000 7f01f2e65700 1 -- 172.21.15.162:0/3591970935 >> 172.21.15.162:0/3591970935 conn(0x7f01ec061320 msgr2=0x7f01ec062e50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:49:53.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.219+0000 7f01f2e65700 1 -- 172.21.15.162:0/3591970935 shutdown_connections 2020-11-28T21:49:53.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.219+0000 7f01f2e65700 1 -- 172.21.15.162:0/3591970935 wait complete. 2020-11-28T21:49:53.229 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.223+0000 7f01f2e65700 1 Processor -- start 2020-11-28T21:49:53.229 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.223+0000 7f01f2e65700 1 -- start start 2020-11-28T21:49:53.229 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.223+0000 7f01f2e65700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01ec0a8550 0x7f01ec13fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:53.229 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.223+0000 7f01f2e65700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01ec0af480 0x7f01ec065370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:53.229 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.223+0000 7f01f2e65700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f01ec0b2290 0x7f01ec065860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:53.230 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.223+0000 7f01f2e65700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec064ae0 con 0x7f01ec0b2290 2020-11-28T21:49:53.230 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.223+0000 7f01f2e65700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec0aa4b0 con 0x7f01ec0af480 2020-11-28T21:49:53.230 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.223+0000 7f01f2e65700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec0b34b0 con 0x7f01ec0a8550 2020-11-28T21:49:53.230 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.223+0000 7f01f1662700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01ec0af480 0x7f01ec065370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:53.231 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.223+0000 7f01f1662700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01ec0af480 0x7f01ec065370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39646/0 (socket says 172.21.15.162:39646) 2020-11-28T21:49:53.231 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.223+0000 7f01f2664700 1 --2- 172.21.15.162:0/3459654530 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f01ec0b2290 0x7f01ec065860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:53.231 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.223+0000 7f01f1662700 1 -- 172.21.15.162:0/3459654530 learned_addr learned my addr 172.21.15.162:0/3459654530 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:49:53.231 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.223+0000 7f01f1e63700 1 --2- 172.21.15.162:0/3459654530 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01ec0a8550 0x7f01ec13fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:53.232 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.223+0000 7f01f1662700 1 -- 172.21.15.162:0/3459654530 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01ec0a8550 msgr2=0x7f01ec13fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:53.232 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.223+0000 7f01f1662700 1 --2- 172.21.15.162:0/3459654530 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01ec0a8550 0x7f01ec13fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:53.232 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.223+0000 7f01f1662700 1 -- 172.21.15.162:0/3459654530 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f01ec0b2290 msgr2=0x7f01ec065860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:53.232 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.223+0000 7f01f1662700 1 --2- 172.21.15.162:0/3459654530 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f01ec0b2290 0x7f01ec065860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:53.233 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.223+0000 7f01f1662700 1 -- 172.21.15.162:0/3459654530 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ec140520 con 0x7f01ec0af480 2020-11-28T21:49:53.233 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.223+0000 7f01f1662700 1 --2- 172.21.15.162:0/3459654530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01ec0af480 0x7f01ec065370 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f01e00010d0 tx=0x7f01e0007ab0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:53.233 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.227+0000 7f01eaffd700 1 -- 172.21.15.162:0/3459654530 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01e000f470 con 0x7f01ec0af480 2020-11-28T21:49:53.233 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.227+0000 7f01eaffd700 1 -- 172.21.15.162:0/3459654530 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e000f5d0 con 0x7f01ec0af480 2020-11-28T21:49:53.234 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.227+0000 7f01f2e65700 1 -- 172.21.15.162:0/3459654530 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01ec065d50 con 0x7f01ec0af480 2020-11-28T21:49:53.234 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.227+0000 7f01eaffd700 1 -- 172.21.15.162:0/3459654530 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01e000f930 con 0x7f01ec0af480 2020-11-28T21:49:53.234 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.227+0000 7f01f2e65700 1 -- 172.21.15.162:0/3459654530 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01ec0b43e0 con 0x7f01ec0af480 2020-11-28T21:49:53.234 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.227+0000 7f01f2e65700 1 -- 172.21.15.162:0/3459654530 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01ec0b36b0 con 0x7f01ec0af480 2020-11-28T21:49:53.235 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.227+0000 7f01eaffd700 1 -- 172.21.15.162:0/3459654530 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f01e0019030 con 0x7f01ec0af480 2020-11-28T21:49:53.235 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.227+0000 7f01eaffd700 1 --2- 172.21.15.162:0/3459654530 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f01cc0554e0 0x7f01cc0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:53.235 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.227+0000 7f01eaffd700 1 -- 172.21.15.162:0/3459654530 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f01e006d490 con 0x7f01ec0af480 2020-11-28T21:49:53.236 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.227+0000 7f01f1e63700 1 --2- 172.21.15.162:0/3459654530 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f01cc0554e0 0x7f01cc0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:53.236 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.231+0000 7f01f1e63700 1 --2- 172.21.15.162:0/3459654530 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f01cc0554e0 0x7f01cc0579a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f01e4019000 tx=0x7f01e4015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:53.237 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.235+0000 7f01eaffd700 1 -- 172.21.15.162:0/3459654530 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f01e0043680 con 0x7f01ec0af480 2020-11-28T21:49:53.425 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.427+0000 7f01f2e65700 1 -- 172.21.15.162:0/3459654530 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead"]} v 0) v1 -- 0x7f01ec057050 con 0x7f01ec0af480 2020-11-28T21:49:53.739 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.739+0000 7f01eaffd700 1 -- 172.21.15.162:0/3459654530 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead"]}]=0 v152) v1 ==== 144+0+0 (secure 0 0 0) 0x7f01e0046ca0 con 0x7f01ec0af480 2020-11-28T21:49:53.743 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.743+0000 7f01f2e65700 1 -- 172.21.15.162:0/3459654530 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f01cc0554e0 msgr2=0x7f01cc0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:53.744 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.743+0000 7f01f2e65700 1 --2- 172.21.15.162:0/3459654530 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f01cc0554e0 0x7f01cc0579a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f01e4019000 tx=0x7f01e4015000).stop 2020-11-28T21:49:53.744 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.743+0000 7f01f2e65700 1 -- 172.21.15.162:0/3459654530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01ec0af480 msgr2=0x7f01ec065370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:53.745 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.743+0000 7f01f2e65700 1 --2- 172.21.15.162:0/3459654530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01ec0af480 0x7f01ec065370 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f01e00010d0 tx=0x7f01e0007ab0).stop 2020-11-28T21:49:53.745 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.743+0000 7f01f2e65700 1 -- 172.21.15.162:0/3459654530 shutdown_connections 2020-11-28T21:49:53.745 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.743+0000 7f01f2e65700 1 --2- 172.21.15.162:0/3459654530 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f01cc0554e0 0x7f01cc0579a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:53.746 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.743+0000 7f01f2e65700 1 --2- 172.21.15.162:0/3459654530 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01ec0a8550 0x7f01ec13fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:53.746 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.743+0000 7f01f2e65700 1 --2- 172.21.15.162:0/3459654530 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f01ec0b2290 0x7f01ec065860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:53.747 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.743+0000 7f01f2e65700 1 --2- 172.21.15.162:0/3459654530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01ec0af480 0x7f01ec065370 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:53.747 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.743+0000 7f01f2e65700 1 -- 172.21.15.162:0/3459654530 >> 172.21.15.162:0/3459654530 conn(0x7f01ec061320 msgr2=0x7f01ec142d40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:49:53.748 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.743+0000 7f01f2e65700 1 -- 172.21.15.162:0/3459654530 shutdown_connections 2020-11-28T21:49:53.748 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.743+0000 7f01f2e65700 1 -- 172.21.15.162:0/3459654530 wait complete. 2020-11-28T21:49:53.759 INFO:tasks.cephfs_test_runner:test_drop_cache_command_dead (tasks.cephfs.test_misc.TestCacheDrop) ... ok 2020-11-28T21:49:53.759 INFO:tasks.cephfs_test_runner:Starting test: test_drop_cache_command_dead_timeout (tasks.cephfs.test_misc.TestCacheDrop) 2020-11-28T21:49:53.760 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout' 2020-11-28T21:49:53.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.875+0000 7f4dbc991700 1 Processor -- start 2020-11-28T21:49:53.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.875+0000 7f4dbc991700 1 -- start start 2020-11-28T21:49:53.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.875+0000 7f4dbc991700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4db80994f0 0x7f4db80b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:53.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.875+0000 7f4dbc991700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4db8060710 0x7f4db80b1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:53.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.875+0000 7f4dbc991700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4db8098f10 con 0x7f4db8098b30 2020-11-28T21:49:53.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.875+0000 7f4dbc991700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4db80b1030 con 0x7f4db80994f0 2020-11-28T21:49:53.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.875+0000 7f4dbc991700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4db80b0680 con 0x7f4db8060710 2020-11-28T21:49:53.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.875+0000 7f4db659c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4db80994f0 0x7f4db80b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:53.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.875+0000 7f4db659c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4db80994f0 0x7f4db80b0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39652/0 (socket says 172.21.15.162:39652) 2020-11-28T21:49:53.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.875+0000 7f4db659c700 1 -- 172.21.15.162:0/741695286 learned_addr learned my addr 172.21.15.162:0/741695286 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:49:53.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.875+0000 7f4db759e700 1 --2- 172.21.15.162:0/741695286 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4db8060710 0x7f4db80b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:53.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.879+0000 7f4db659c700 1 -- 172.21.15.162:0/741695286 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4db8060710 msgr2=0x7f4db80b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:49:53.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.879+0000 7f4db659c700 1 --2- 172.21.15.162:0/741695286 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4db8060710 0x7f4db80b1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:53.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.879+0000 7f4db659c700 1 -- 172.21.15.162:0/741695286 >> v1:172.21.15.162:6789/0 conn(0x7f4db8098b30 legacy=0x7f4db8060000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:53.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.879+0000 7f4db659c700 1 -- 172.21.15.162:0/741695286 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db80b0920 con 0x7f4db80994f0 2020-11-28T21:49:53.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.879+0000 7f4db759e700 1 --2- 172.21.15.162:0/741695286 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4db8060710 0x7f4db80b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:49:53.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.879+0000 7f4db659c700 1 --2- 172.21.15.162:0/741695286 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4db80994f0 0x7f4db80b0000 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f4db00032b0 tx=0x7f4db0009270).ready entity=mon.1 client_cookie=f14520401d7fdc1e server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:53.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.879+0000 7f4db5d9b700 1 -- 172.21.15.162:0/741695286 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4db000bc00 con 0x7f4db80994f0 2020-11-28T21:49:53.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.879+0000 7f4db5d9b700 1 -- 172.21.15.162:0/741695286 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4db000bd60 con 0x7f4db80994f0 2020-11-28T21:49:53.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.879+0000 7f4db5d9b700 1 -- 172.21.15.162:0/741695286 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4db0011400 con 0x7f4db80994f0 2020-11-28T21:49:53.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.879+0000 7f4dbc991700 1 -- 172.21.15.162:0/741695286 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4db80994f0 msgr2=0x7f4db80b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:53.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.879+0000 7f4dbc991700 1 --2- 172.21.15.162:0/741695286 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4db80994f0 0x7f4db80b0000 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f4db00032b0 tx=0x7f4db0009270).stop 2020-11-28T21:49:53.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.879+0000 7f4dbc991700 1 -- 172.21.15.162:0/741695286 shutdown_connections 2020-11-28T21:49:53.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.879+0000 7f4dbc991700 1 --2- 172.21.15.162:0/741695286 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4db8060710 0x7f4db80b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:53.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.879+0000 7f4dbc991700 1 --2- 172.21.15.162:0/741695286 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4db80994f0 0x7f4db80b0000 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:53.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.879+0000 7f4dbc991700 1 -- 172.21.15.162:0/741695286 >> 172.21.15.162:0/741695286 conn(0x7f4db805b970 msgr2=0x7f4db805bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:49:53.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.879+0000 7f4dbc991700 1 -- 172.21.15.162:0/741695286 shutdown_connections 2020-11-28T21:49:53.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.879+0000 7f4dbc991700 1 -- 172.21.15.162:0/741695286 wait complete. 2020-11-28T21:49:53.887 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.879+0000 7f4dbc991700 1 Processor -- start 2020-11-28T21:49:53.888 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.879+0000 7f4dbc991700 1 -- start start 2020-11-28T21:49:53.888 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.879+0000 7f4dbc991700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4db8060710 0x7f4db80aaa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:53.888 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.879+0000 7f4dbc991700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4db8098b30 0x7f4db8063340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:53.888 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.879+0000 7f4dbc991700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4db80994f0 0x7f4db80ae960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:53.889 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.879+0000 7f4dbc991700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db8156700 con 0x7f4db80994f0 2020-11-28T21:49:53.889 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.879+0000 7f4dbc991700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db80adc70 con 0x7f4db8060710 2020-11-28T21:49:53.889 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.879+0000 7f4dbc991700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4db80ad9d0 con 0x7f4db8098b30 2020-11-28T21:49:53.889 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.883+0000 7f4db6d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4db8060710 0x7f4db80aaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:53.889 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.883+0000 7f4db6d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4db8060710 0x7f4db80aaa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39658/0 (socket says 172.21.15.162:39658) 2020-11-28T21:49:53.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.883+0000 7f4db6d9d700 1 -- 172.21.15.162:0/4135080791 learned_addr learned my addr 172.21.15.162:0/4135080791 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:49:53.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.883+0000 7f4db659c700 1 --2- 172.21.15.162:0/4135080791 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4db8098b30 0x7f4db8063340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:53.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.883+0000 7f4db759e700 1 --2- 172.21.15.162:0/4135080791 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4db80994f0 0x7f4db80ae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:53.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.883+0000 7f4db6d9d700 1 -- 172.21.15.162:0/4135080791 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4db8098b30 msgr2=0x7f4db8063340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:53.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.883+0000 7f4db6d9d700 1 --2- 172.21.15.162:0/4135080791 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4db8098b30 0x7f4db8063340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:53.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.883+0000 7f4db6d9d700 1 -- 172.21.15.162:0/4135080791 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4db80994f0 msgr2=0x7f4db80ae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:53.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.883+0000 7f4db6d9d700 1 --2- 172.21.15.162:0/4135080791 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4db80994f0 0x7f4db80ae960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:53.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.883+0000 7f4db6d9d700 1 -- 172.21.15.162:0/4135080791 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db8156960 con 0x7f4db8060710 2020-11-28T21:49:53.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.883+0000 7f4db6d9d700 1 --2- 172.21.15.162:0/4135080791 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4db8060710 0x7f4db80aaa30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7f4db8061f00 tx=0x7f4da0007be0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:53.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.883+0000 7f4daffff700 1 -- 172.21.15.162:0/4135080791 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4da0007240 con 0x7f4db8060710 2020-11-28T21:49:53.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.883+0000 7f4daffff700 1 -- 172.21.15.162:0/4135080791 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4da00073a0 con 0x7f4db8060710 2020-11-28T21:49:53.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.883+0000 7f4dbc991700 1 -- 172.21.15.162:0/4135080791 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4db80ab360 con 0x7f4db8060710 2020-11-28T21:49:53.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.883+0000 7f4daffff700 1 -- 172.21.15.162:0/4135080791 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4da00066d0 con 0x7f4db8060710 2020-11-28T21:49:53.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.883+0000 7f4dbc991700 1 -- 172.21.15.162:0/4135080791 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4db80aeff0 con 0x7f4db8060710 2020-11-28T21:49:53.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.883+0000 7f4daffff700 1 -- 172.21.15.162:0/4135080791 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4da0010070 con 0x7f4db8060710 2020-11-28T21:49:53.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.883+0000 7f4dbc991700 1 -- 172.21.15.162:0/4135080791 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4db8057050 con 0x7f4db8060710 2020-11-28T21:49:53.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.883+0000 7f4daffff700 1 --2- 172.21.15.162:0/4135080791 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4d9c0553b0 0x7f4d9c057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:53.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.883+0000 7f4daffff700 1 -- 172.21.15.162:0/4135080791 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4da006d5b0 con 0x7f4db8060710 2020-11-28T21:49:53.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.887+0000 7f4db659c700 1 --2- 172.21.15.162:0/4135080791 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4d9c0553b0 0x7f4d9c057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:53.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.887+0000 7f4db659c700 1 --2- 172.21.15.162:0/4135080791 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4d9c0553b0 0x7f4d9c057870 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f4db001b000 tx=0x7f4db0016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:53.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:53.891+0000 7f4daffff700 1 -- 172.21.15.162:0/4135080791 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4da00438b0 con 0x7f4db8060710 2020-11-28T21:49:54.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.075+0000 7f4dbc991700 1 -- 172.21.15.162:0/4135080791 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout"]} v 0) v1 -- 0x7f4db8156af0 con 0x7f4db8060710 2020-11-28T21:49:54.742 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.743+0000 7f4daffff700 1 -- 172.21.15.162:0/4135080791 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout"]}]=0 v153) v1 ==== 155+0+0 (secure 0 0 0) 0x7f4da0046ed0 con 0x7f4db8060710 2020-11-28T21:49:54.745 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.747+0000 7f4dbc991700 1 -- 172.21.15.162:0/4135080791 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4d9c0553b0 msgr2=0x7f4d9c057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:54.746 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.747+0000 7f4dbc991700 1 --2- 172.21.15.162:0/4135080791 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4d9c0553b0 0x7f4d9c057870 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f4db001b000 tx=0x7f4db0016000).stop 2020-11-28T21:49:54.746 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.747+0000 7f4dbc991700 1 -- 172.21.15.162:0/4135080791 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4db8060710 msgr2=0x7f4db80aaa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:54.746 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.747+0000 7f4dbc991700 1 --2- 172.21.15.162:0/4135080791 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4db8060710 0x7f4db80aaa30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7f4db8061f00 tx=0x7f4da0007be0).stop 2020-11-28T21:49:54.746 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.747+0000 7f4dbc991700 1 -- 172.21.15.162:0/4135080791 shutdown_connections 2020-11-28T21:49:54.747 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.747+0000 7f4dbc991700 1 --2- 172.21.15.162:0/4135080791 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4d9c0553b0 0x7f4d9c057870 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:54.747 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.747+0000 7f4dbc991700 1 --2- 172.21.15.162:0/4135080791 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4db80994f0 0x7f4db80ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:54.747 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.747+0000 7f4dbc991700 1 --2- 172.21.15.162:0/4135080791 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4db8060710 0x7f4db80aaa30 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:54.747 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.747+0000 7f4dbc991700 1 --2- 172.21.15.162:0/4135080791 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4db8098b30 0x7f4db8063340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:54.747 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.747+0000 7f4dbc991700 1 -- 172.21.15.162:0/4135080791 >> 172.21.15.162:0/4135080791 conn(0x7f4db805b970 msgr2=0x7f4db8148990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:49:54.748 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.747+0000 7f4dbc991700 1 -- 172.21.15.162:0/4135080791 shutdown_connections 2020-11-28T21:49:54.748 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.747+0000 7f4dbc991700 1 -- 172.21.15.162:0/4135080791 wait complete. 2020-11-28T21:49:54.762 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config set mon mon_allow_pool_delete True 2020-11-28T21:49:54.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.887+0000 7f7074cc6700 1 Processor -- start 2020-11-28T21:49:54.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.887+0000 7f7074cc6700 1 -- start start 2020-11-28T21:49:54.886 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.887+0000 7f7074cc6700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f70700ac500 0x7f707005fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:54.887 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.887+0000 7f7074cc6700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70700ac8e0 con 0x7f707013f560 2020-11-28T21:49:54.887 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.887+0000 7f7074cc6700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7070060690 con 0x7f70700ae9d0 2020-11-28T21:49:54.887 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.887+0000 7f7074cc6700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f707013ccf0 con 0x7f70700ac500 2020-11-28T21:49:54.887 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.887+0000 7f706effd700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f70700ae9d0 0x7f7070141b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:46512/0 (socket says 172.21.15.162:46512) 2020-11-28T21:49:54.887 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.887+0000 7f706effd700 1 -- 172.21.15.162:0/466747821 learned_addr learned my addr 172.21.15.162:0/466747821 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:49:54.888 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.887+0000 7f706f7fe700 1 --2- 172.21.15.162:0/466747821 >> v2:172.21.15.118:3300/0 conn(0x7f70700ac500 0x7f707005fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:54.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.887+0000 7f706e7fc700 1 -- 172.21.15.162:0/466747821 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 410723845 0 0) 0x7f7070060690 con 0x7f70700ae9d0 2020-11-28T21:49:54.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.887+0000 7f706e7fc700 1 -- 172.21.15.162:0/466747821 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7050003230 con 0x7f70700ae9d0 2020-11-28T21:49:54.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.887+0000 7f706e7fc700 1 -- 172.21.15.162:0/466747821 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3536942367 0 0) 0x7f7050003230 con 0x7f70700ae9d0 2020-11-28T21:49:54.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.887+0000 7f706e7fc700 1 -- 172.21.15.162:0/466747821 >> v2:172.21.15.118:3300/0 conn(0x7f70700ac500 msgr2=0x7f707005fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:49:54.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.887+0000 7f706e7fc700 1 --2- 172.21.15.162:0/466747821 >> v2:172.21.15.118:3300/0 conn(0x7f70700ac500 0x7f707005fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:54.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.887+0000 7f706e7fc700 1 -- 172.21.15.162:0/466747821 >> v1:172.21.15.118:6789/0 conn(0x7f707013f560 legacy=0x7f707013f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:54.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.887+0000 7f706e7fc700 1 -- 172.21.15.162:0/466747821 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7070060930 con 0x7f70700ae9d0 2020-11-28T21:49:54.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.887+0000 7f706e7fc700 1 -- 172.21.15.162:0/466747821 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f70580016d0 con 0x7f70700ae9d0 2020-11-28T21:49:54.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.887+0000 7f706e7fc700 1 -- 172.21.15.162:0/466747821 >> v1:172.21.15.162:6789/0 conn(0x7f70700ae9d0 legacy=0x7f7070141b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:54.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.887+0000 7f706e7fc700 1 --2- 172.21.15.162:0/466747821 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7050003720 0x7f7050003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:54.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.887+0000 7f706e7fc700 1 --2- 172.21.15.162:0/466747821 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f70500044d0 0x7f7050004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:54.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.887+0000 7f706e7fc700 1 --2- 172.21.15.162:0/466747821 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7050004e40 0x7f705000b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:54.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.887+0000 7f706e7fc700 1 -- 172.21.15.162:0/466747821 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707013ccf0 con 0x7f7050003720 2020-11-28T21:49:54.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.887+0000 7f706e7fc700 1 -- 172.21.15.162:0/466747821 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7050004250 con 0x7f7050004e40 2020-11-28T21:49:54.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.887+0000 7f706e7fc700 1 -- 172.21.15.162:0/466747821 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f705000bc10 con 0x7f70500044d0 2020-11-28T21:49:54.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.887+0000 7f706ffff700 1 --2- 172.21.15.162:0/466747821 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7050004e40 0x7f705000b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:54.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.887+0000 7f706f7fe700 1 --2- 172.21.15.162:0/466747821 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7050003720 0x7f7050003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:54.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.887+0000 7f706effd700 1 --2- 172.21.15.162:0/466747821 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f70500044d0 0x7f7050004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:54.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.891+0000 7f706ffff700 1 -- 172.21.15.162:0/466747821 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f70500044d0 msgr2=0x7f7050004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:49:54.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.891+0000 7f706ffff700 1 --2- 172.21.15.162:0/466747821 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f70500044d0 0x7f7050004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:54.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.891+0000 7f706ffff700 1 -- 172.21.15.162:0/466747821 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7050003720 msgr2=0x7f7050003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:49:54.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.891+0000 7f706ffff700 1 --2- 172.21.15.162:0/466747821 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7050003720 0x7f7050003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:54.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.891+0000 7f706ffff700 1 -- 172.21.15.162:0/466747821 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7050003230 con 0x7f7050004e40 2020-11-28T21:49:54.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.891+0000 7f706effd700 1 --2- 172.21.15.162:0/466747821 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f70500044d0 0x7f7050004950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:49:54.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.891+0000 7f706ffff700 1 -- 172.21.15.162:0/466747821 reap_dead start 2020-11-28T21:49:54.896 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.891+0000 7f706ffff700 1 --2- 172.21.15.162:0/466747821 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7050004e40 0x7f705000b720 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f70700ad840 tx=0x7f7068007ee0).ready entity=mon.1 client_cookie=e0a7ae64beb98f20 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:54.896 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.891+0000 7f706e7fc700 1 -- 172.21.15.162:0/466747821 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f706800cb20 con 0x7f7050004e40 2020-11-28T21:49:54.896 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.891+0000 7f706e7fc700 1 -- 172.21.15.162:0/466747821 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f706800cc80 con 0x7f7050004e40 2020-11-28T21:49:54.896 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.891+0000 7f706e7fc700 1 -- 172.21.15.162:0/466747821 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7068013410 con 0x7f7050004e40 2020-11-28T21:49:54.897 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.891+0000 7f7074cc6700 1 -- 172.21.15.162:0/466747821 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7050004e40 msgr2=0x7f705000b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:54.897 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.891+0000 7f7074cc6700 1 --2- 172.21.15.162:0/466747821 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7050004e40 0x7f705000b720 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f70700ad840 tx=0x7f7068007ee0).stop 2020-11-28T21:49:54.897 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.891+0000 7f7074cc6700 1 -- 172.21.15.162:0/466747821 shutdown_connections 2020-11-28T21:49:54.897 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.891+0000 7f7074cc6700 1 --2- 172.21.15.162:0/466747821 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7050004e40 0x7f705000b720 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:54.898 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.891+0000 7f7074cc6700 1 -- 172.21.15.162:0/466747821 >> 172.21.15.162:0/466747821 conn(0x7f70700b57d0 msgr2=0x7f70700b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:49:54.898 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.891+0000 7f7074cc6700 1 -- 172.21.15.162:0/466747821 shutdown_connections 2020-11-28T21:49:54.898 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.891+0000 7f7074cc6700 1 -- 172.21.15.162:0/466747821 wait complete. 2020-11-28T21:49:54.901 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.891+0000 7f7074cc6700 1 Processor -- start 2020-11-28T21:49:54.901 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.891+0000 7f7074cc6700 1 -- start start 2020-11-28T21:49:54.901 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.891+0000 7f7074cc6700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7050004e40 0x7f7070143810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:54.902 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.891+0000 7f7074cc6700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7070064f60 0x7f7070152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:54.902 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.891+0000 7f7074cc6700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70701531c0 0x7f7070063340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:54.902 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.891+0000 7f7074cc6700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7070067480 con 0x7f7070064f60 2020-11-28T21:49:54.902 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.891+0000 7f7074cc6700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70700653e0 con 0x7f70701531c0 2020-11-28T21:49:54.903 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.891+0000 7f7074cc6700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7070066aa0 con 0x7f7050004e40 2020-11-28T21:49:54.903 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.891+0000 7f706ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70701531c0 0x7f7070063340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:54.903 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.891+0000 7f706ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70701531c0 0x7f7070063340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39680/0 (socket says 172.21.15.162:39680) 2020-11-28T21:49:54.903 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.891+0000 7f706ffff700 1 -- 172.21.15.162:0/2393330746 learned_addr learned my addr 172.21.15.162:0/2393330746 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:49:54.903 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.891+0000 7f706effd700 1 --2- 172.21.15.162:0/2393330746 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7070064f60 0x7f7070152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:54.904 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.891+0000 7f706f7fe700 1 --2- 172.21.15.162:0/2393330746 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7050004e40 0x7f7070143810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:54.904 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.895+0000 7f706ffff700 1 -- 172.21.15.162:0/2393330746 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7050004e40 msgr2=0x7f7070143810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:54.904 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.895+0000 7f706ffff700 1 --2- 172.21.15.162:0/2393330746 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7050004e40 0x7f7070143810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:54.904 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.895+0000 7f706ffff700 1 -- 172.21.15.162:0/2393330746 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7070064f60 msgr2=0x7f7070152cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:54.905 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.895+0000 7f706ffff700 1 --2- 172.21.15.162:0/2393330746 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7070064f60 0x7f7070152cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:54.905 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.895+0000 7f706ffff700 1 -- 172.21.15.162:0/2393330746 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7070066d40 con 0x7f70701531c0 2020-11-28T21:49:54.905 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.895+0000 7f706effd700 1 --2- 172.21.15.162:0/2393330746 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7070064f60 0x7f7070152cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:49:54.905 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.895+0000 7f706ffff700 1 --2- 172.21.15.162:0/2393330746 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70701531c0 0x7f7070063340 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7f70700ad840 tx=0x7f7068016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:54.905 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.895+0000 7f706cff9700 1 -- 172.21.15.162:0/2393330746 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7068006860 con 0x7f70701531c0 2020-11-28T21:49:54.906 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.895+0000 7f706cff9700 1 -- 172.21.15.162:0/2393330746 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70680069c0 con 0x7f70701531c0 2020-11-28T21:49:54.906 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.895+0000 7f7074cc6700 1 -- 172.21.15.162:0/2393330746 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7070066fe0 con 0x7f70701531c0 2020-11-28T21:49:54.906 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.895+0000 7f706cff9700 1 -- 172.21.15.162:0/2393330746 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7068006d20 con 0x7f70701531c0 2020-11-28T21:49:54.906 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.895+0000 7f7074cc6700 1 -- 172.21.15.162:0/2393330746 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7070153620 con 0x7f70701531c0 2020-11-28T21:49:54.906 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.895+0000 7f706cff9700 1 -- 172.21.15.162:0/2393330746 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7068024030 con 0x7f70701531c0 2020-11-28T21:49:54.907 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.895+0000 7f7074cc6700 1 -- 172.21.15.162:0/2393330746 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7070066900 con 0x7f70701531c0 2020-11-28T21:49:54.907 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.895+0000 7f706cff9700 1 --2- 172.21.15.162:0/2393330746 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f70540553f0 0x7f70540578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:54.907 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.895+0000 7f706cff9700 1 -- 172.21.15.162:0/2393330746 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f706806eea0 con 0x7f70701531c0 2020-11-28T21:49:54.907 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.895+0000 7f706f7fe700 1 --2- 172.21.15.162:0/2393330746 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f70540553f0 0x7f70540578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:54.908 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.895+0000 7f706f7fe700 1 --2- 172.21.15.162:0/2393330746 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f70540553f0 0x7f70540578b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f7060009000 tx=0x7f7060008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:54.908 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:54.899+0000 7f706cff9700 1 -- 172.21.15.162:0/2393330746 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7070066900 con 0x7f70701531c0 2020-11-28T21:49:55.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.035+0000 7f7074cc6700 1 -- 172.21.15.162:0/2393330746 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7f7070066900 con 0x7f70701531c0 2020-11-28T21:49:55.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.043+0000 7f706cff9700 1 -- 172.21.15.162:0/2393330746 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7070066900 con 0x7f70701531c0 2020-11-28T21:49:55.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.047+0000 7f7074cc6700 1 -- 172.21.15.162:0/2393330746 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f70540553f0 msgr2=0x7f70540578b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:55.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.047+0000 7f7074cc6700 1 --2- 172.21.15.162:0/2393330746 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f70540553f0 0x7f70540578b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f7060009000 tx=0x7f7060008040).stop 2020-11-28T21:49:55.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.047+0000 7f7074cc6700 1 -- 172.21.15.162:0/2393330746 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70701531c0 msgr2=0x7f7070063340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:55.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.047+0000 7f7074cc6700 1 --2- 172.21.15.162:0/2393330746 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70701531c0 0x7f7070063340 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7f70700ad840 tx=0x7f7068016000).stop 2020-11-28T21:49:55.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.047+0000 7f7074cc6700 1 -- 172.21.15.162:0/2393330746 shutdown_connections 2020-11-28T21:49:55.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.047+0000 7f7074cc6700 1 --2- 172.21.15.162:0/2393330746 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f70540553f0 0x7f70540578b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:55.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.047+0000 7f7074cc6700 1 --2- 172.21.15.162:0/2393330746 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7050004e40 0x7f7070143810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:55.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.047+0000 7f7074cc6700 1 --2- 172.21.15.162:0/2393330746 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70701531c0 0x7f7070063340 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:55.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.047+0000 7f7074cc6700 1 --2- 172.21.15.162:0/2393330746 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7070064f60 0x7f7070152cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:55.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.047+0000 7f7074cc6700 1 -- 172.21.15.162:0/2393330746 >> 172.21.15.162:0/2393330746 conn(0x7f70700b57d0 msgr2=0x7f707013c210 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:49:55.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.047+0000 7f7074cc6700 1 -- 172.21.15.162:0/2393330746 shutdown_connections 2020-11-28T21:49:55.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.047+0000 7f7074cc6700 1 -- 172.21.15.162:0/2393330746 wait complete. 2020-11-28T21:49:55.058 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-28T21:49:55.058 INFO:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:49:55.076 INFO:teuthology.orchestra.run.smithi118:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:49:55.096 INFO:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:49:55.115 INFO:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:49:55.134 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-28T21:49:55.134 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:49:55.135 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:49:55.135 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:49:55.184 INFO:teuthology.orchestra.run.smithi192.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2020-11-28T21:49:55.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:49:55.185 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2020-11-28T21:49:55.186 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:49:55.186 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:49:55.187 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:49:55.194 INFO:teuthology.orchestra.run.smithi096.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-11-28T21:49:55.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:49:55.195 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2020-11-28T21:49:55.196 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2020-11-28T21:49:55.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.315+0000 7f6fc359e700 1 Processor -- start 2020-11-28T21:49:55.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.315+0000 7f6fc359e700 1 -- start start 2020-11-28T21:49:55.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.315+0000 7f6fc359e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f6fc40b5e50 0x7f6fc4143360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:55.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.315+0000 7f6fc359e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fc40b8040 con 0x7f6fc40b5a70 2020-11-28T21:49:55.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.315+0000 7f6fc359e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fc40b8720 con 0x7f6fc40b5430 2020-11-28T21:49:55.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.315+0000 7f6fc359e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fc40b7600 con 0x7f6fc40b5e50 2020-11-28T21:49:55.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.315+0000 7f6fc2d9d700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f6fc40b5430 0x7f6fc409d220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:46534/0 (socket says 172.21.15.162:46534) 2020-11-28T21:49:55.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.315+0000 7f6fc2d9d700 1 -- 172.21.15.162:0/1492016496 learned_addr learned my addr 172.21.15.162:0/1492016496 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:49:55.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.315+0000 7f6fc1d9b700 1 --2- 172.21.15.162:0/1492016496 >> v2:172.21.15.118:3300/0 conn(0x7f6fc40b5e50 0x7f6fc4143360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:55.318 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.315+0000 7f6fc159a700 1 -- 172.21.15.162:0/1492016496 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 889285693 0 0) 0x7f6fc40b8720 con 0x7f6fc40b5430 2020-11-28T21:49:55.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.315+0000 7f6fc159a700 1 -- 172.21.15.162:0/1492016496 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fa4003230 con 0x7f6fc40b5430 2020-11-28T21:49:55.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.319+0000 7f6fc159a700 1 -- 172.21.15.162:0/1492016496 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1354138612 0 0) 0x7f6fc40b8040 con 0x7f6fc40b5a70 2020-11-28T21:49:55.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.319+0000 7f6fc159a700 1 -- 172.21.15.162:0/1492016496 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fc40b8720 con 0x7f6fc40b5a70 2020-11-28T21:49:55.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.319+0000 7f6fc1d9b700 1 -- 172.21.15.162:0/1492016496 >> v1:172.21.15.162:6789/0 conn(0x7f6fc40b5430 legacy=0x7f6fc409d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:55.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.319+0000 7f6fc1d9b700 1 -- 172.21.15.162:0/1492016496 >> v1:172.21.15.118:6789/0 conn(0x7f6fc40b5a70 legacy=0x7f6fc40b6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:55.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.319+0000 7f6fc1d9b700 1 -- 172.21.15.162:0/1492016496 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc4142980 con 0x7f6fc40b5e50 2020-11-28T21:49:55.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.319+0000 7f6fc1d9b700 1 --2- 172.21.15.162:0/1492016496 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6fc40b5e50 0x7f6fc4143360 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f6fc40b6960 tx=0x7f6fac006dc0).ready entity=mon.0 client_cookie=e8d6515879978e0d server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:55.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.319+0000 7f6fc159a700 1 -- 172.21.15.162:0/1492016496 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fac009740 con 0x7f6fc40b5e50 2020-11-28T21:49:55.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.319+0000 7f6fc159a700 1 -- 172.21.15.162:0/1492016496 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fac0098a0 con 0x7f6fc40b5e50 2020-11-28T21:49:55.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.319+0000 7f6fc159a700 1 -- 172.21.15.162:0/1492016496 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fac009c00 con 0x7f6fc40b5e50 2020-11-28T21:49:55.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.319+0000 7f6fc359e700 1 -- 172.21.15.162:0/1492016496 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6fc40b5e50 msgr2=0x7f6fc4143360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:55.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.319+0000 7f6fc359e700 1 --2- 172.21.15.162:0/1492016496 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6fc40b5e50 0x7f6fc4143360 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f6fc40b6960 tx=0x7f6fac006dc0).stop 2020-11-28T21:49:55.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.319+0000 7f6fc359e700 1 -- 172.21.15.162:0/1492016496 shutdown_connections 2020-11-28T21:49:55.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.319+0000 7f6fc359e700 1 --2- 172.21.15.162:0/1492016496 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6fc40b5e50 0x7f6fc4143360 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:55.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.319+0000 7f6fc359e700 1 -- 172.21.15.162:0/1492016496 >> 172.21.15.162:0/1492016496 conn(0x7f6fc405b970 msgr2=0x7f6fc405bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:49:55.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.319+0000 7f6fc359e700 1 -- 172.21.15.162:0/1492016496 shutdown_connections 2020-11-28T21:49:55.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.319+0000 7f6fc359e700 1 -- 172.21.15.162:0/1492016496 wait complete. 2020-11-28T21:49:55.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.319+0000 7f6fc359e700 1 Processor -- start 2020-11-28T21:49:55.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.319+0000 7f6fc359e700 1 -- start start 2020-11-28T21:49:55.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.323+0000 7f6fc359e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6fc40b5430 0x7f6fc4145f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:55.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.323+0000 7f6fc359e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6fc40b5a70 0x7f6fc40b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:55.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.323+0000 7f6fc359e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6fc40b5e50 0x7f6fc40b0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:55.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.323+0000 7f6fc359e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc40b3810 con 0x7f6fc40b5e50 2020-11-28T21:49:55.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.323+0000 7f6fc359e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc4152780 con 0x7f6fc40b5430 2020-11-28T21:49:55.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.323+0000 7f6fc359e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc40ae260 con 0x7f6fc40b5a70 2020-11-28T21:49:55.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.323+0000 7f6fc259c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6fc40b5430 0x7f6fc4145f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:55.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.323+0000 7f6fc259c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6fc40b5430 0x7f6fc4145f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39692/0 (socket says 172.21.15.162:39692) 2020-11-28T21:49:55.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.323+0000 7f6fc259c700 1 -- 172.21.15.162:0/3536374227 learned_addr learned my addr 172.21.15.162:0/3536374227 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:49:55.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.323+0000 7f6fc2d9d700 1 --2- 172.21.15.162:0/3536374227 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6fc40b5e50 0x7f6fc40b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:55.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.323+0000 7f6fc1d9b700 1 --2- 172.21.15.162:0/3536374227 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6fc40b5a70 0x7f6fc40b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:55.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.323+0000 7f6fc259c700 1 -- 172.21.15.162:0/3536374227 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6fc40b5a70 msgr2=0x7f6fc40b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:55.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.323+0000 7f6fc259c700 1 --2- 172.21.15.162:0/3536374227 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6fc40b5a70 0x7f6fc40b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:55.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.323+0000 7f6fc259c700 1 -- 172.21.15.162:0/3536374227 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6fc40b5e50 msgr2=0x7f6fc40b0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:55.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.323+0000 7f6fc259c700 1 --2- 172.21.15.162:0/3536374227 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6fc40b5e50 0x7f6fc40b0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:55.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.323+0000 7f6fc259c700 1 -- 172.21.15.162:0/3536374227 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc40ae500 con 0x7f6fc40b5430 2020-11-28T21:49:55.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.323+0000 7f6fc2d9d700 1 --2- 172.21.15.162:0/3536374227 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6fc40b5e50 0x7f6fc40b0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:49:55.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.323+0000 7f6fc259c700 1 --2- 172.21.15.162:0/3536374227 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6fc40b5430 0x7f6fc4145f20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f6fb4001450 tx=0x7f6fb4007e30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:55.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.323+0000 7f6fbb7fe700 1 -- 172.21.15.162:0/3536374227 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fb4002310 con 0x7f6fc40b5430 2020-11-28T21:49:55.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.323+0000 7f6fbb7fe700 1 -- 172.21.15.162:0/3536374227 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb4002470 con 0x7f6fc40b5430 2020-11-28T21:49:55.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.323+0000 7f6fbb7fe700 1 -- 172.21.15.162:0/3536374227 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fb4001800 con 0x7f6fc40b5430 2020-11-28T21:49:55.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.323+0000 7f6fc359e700 1 -- 172.21.15.162:0/3536374227 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fc40ae7a0 con 0x7f6fc40b5430 2020-11-28T21:49:55.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.323+0000 7f6fc359e700 1 -- 172.21.15.162:0/3536374227 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fc4146410 con 0x7f6fc40b5430 2020-11-28T21:49:55.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.323+0000 7f6fc359e700 1 -- 172.21.15.162:0/3536374227 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fc4146a00 con 0x7f6fc40b5430 2020-11-28T21:49:55.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.323+0000 7f6fbb7fe700 1 -- 172.21.15.162:0/3536374227 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6fb4019030 con 0x7f6fc40b5430 2020-11-28T21:49:55.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.327+0000 7f6fbb7fe700 1 --2- 172.21.15.162:0/3536374227 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6fa80554e0 0x7f6fa80579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:55.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.327+0000 7f6fbb7fe700 1 -- 172.21.15.162:0/3536374227 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6fb406d650 con 0x7f6fc40b5430 2020-11-28T21:49:55.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.327+0000 7f6fc1d9b700 1 --2- 172.21.15.162:0/3536374227 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6fa80554e0 0x7f6fa80579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:55.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.327+0000 7f6fc1d9b700 1 --2- 172.21.15.162:0/3536374227 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6fa80554e0 0x7f6fa80579a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f6fac01b000 tx=0x7f6fac015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:55.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.331+0000 7f6fbb7fe700 1 -- 172.21.15.162:0/3536374227 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6fb4043950 con 0x7f6fc40b5430 2020-11-28T21:49:55.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.475+0000 7f6fc359e700 1 -- 172.21.15.162:0/3536374227 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f6fc40ae930 con 0x7f6fc40b5430 2020-11-28T21:49:55.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.475+0000 7f6fbb7fe700 1 -- 172.21.15.162:0/3536374227 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v44) v1 ==== 81+0+33741 (secure 0 0 0) 0x7f6fb4046f70 con 0x7f6fc40b5430 2020-11-28T21:49:55.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.475+0000 7f6fc359e700 1 -- 172.21.15.162:0/3536374227 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6fa80554e0 msgr2=0x7f6fa80579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:55.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.479+0000 7f6fc359e700 1 --2- 172.21.15.162:0/3536374227 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6fa80554e0 0x7f6fa80579a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f6fac01b000 tx=0x7f6fac015000).stop 2020-11-28T21:49:55.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.479+0000 7f6fc359e700 1 -- 172.21.15.162:0/3536374227 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6fc40b5430 msgr2=0x7f6fc4145f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:55.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.479+0000 7f6fc359e700 1 --2- 172.21.15.162:0/3536374227 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6fc40b5430 0x7f6fc4145f20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f6fb4001450 tx=0x7f6fb4007e30).stop 2020-11-28T21:49:55.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.479+0000 7f6fc359e700 1 -- 172.21.15.162:0/3536374227 shutdown_connections 2020-11-28T21:49:55.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.479+0000 7f6fc359e700 1 --2- 172.21.15.162:0/3536374227 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6fa80554e0 0x7f6fa80579a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:55.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.479+0000 7f6fc359e700 1 --2- 172.21.15.162:0/3536374227 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6fc40b5e50 0x7f6fc40b0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:55.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.479+0000 7f6fc359e700 1 --2- 172.21.15.162:0/3536374227 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6fc40b5430 0x7f6fc4145f20 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:55.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.479+0000 7f6fc359e700 1 --2- 172.21.15.162:0/3536374227 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6fc40b5a70 0x7f6fc40b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:55.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.479+0000 7f6fc359e700 1 -- 172.21.15.162:0/3536374227 >> 172.21.15.162:0/3536374227 conn(0x7f6fc405b970 msgr2=0x7f6fc405f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:49:55.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.479+0000 7f6fc359e700 1 -- 172.21.15.162:0/3536374227 shutdown_connections 2020-11-28T21:49:55.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.479+0000 7f6fc359e700 1 -- 172.21.15.162:0/3536374227 wait complete. 2020-11-28T21:49:55.480 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:49:55.480 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2020-11-28T21:49:55.481 INFO:teuthology.orchestra.run.smithi162.stdout: "epoch": 44, 2020-11-28T21:49:55.481 INFO:teuthology.orchestra.run.smithi162.stdout: "fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", 2020-11-28T21:49:55.481 INFO:teuthology.orchestra.run.smithi162.stdout: "created": "2020-11-28T21:46:11.724372+0000", 2020-11-28T21:49:55.481 INFO:teuthology.orchestra.run.smithi162.stdout: "modified": "2020-11-28T21:48:27.785910+0000", 2020-11-28T21:49:55.482 INFO:teuthology.orchestra.run.smithi162.stdout: "last_up_change": "2020-11-28T21:46:19.591650+0000", 2020-11-28T21:49:55.482 INFO:teuthology.orchestra.run.smithi162.stdout: "last_in_change": "2020-11-28T21:46:19.591650+0000", 2020-11-28T21:49:55.482 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-11-28T21:49:55.482 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_num": 5799936, 2020-11-28T21:49:55.482 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_set": [ 2020-11-28T21:49:55.483 INFO:teuthology.orchestra.run.smithi162.stdout: "pglog_hardlimit", 2020-11-28T21:49:55.483 INFO:teuthology.orchestra.run.smithi162.stdout: "purged_snapdirs", 2020-11-28T21:49:55.483 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_deletes", 2020-11-28T21:49:55.483 INFO:teuthology.orchestra.run.smithi162.stdout: "sortbitwise" 2020-11-28T21:49:55.484 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:49:55.484 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_version": 4, 2020-11-28T21:49:55.484 INFO:teuthology.orchestra.run.smithi162.stdout: "full_ratio": 0.94999998807907104, 2020-11-28T21:49:55.484 INFO:teuthology.orchestra.run.smithi162.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-11-28T21:49:55.485 INFO:teuthology.orchestra.run.smithi162.stdout: "nearfull_ratio": 0.85000002384185791, 2020-11-28T21:49:55.485 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_snapshot": "", 2020-11-28T21:49:55.485 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_max": 8, 2020-11-28T21:49:55.485 INFO:teuthology.orchestra.run.smithi162.stdout: "max_osd": 8, 2020-11-28T21:49:55.485 INFO:teuthology.orchestra.run.smithi162.stdout: "require_min_compat_client": "jewel", 2020-11-28T21:49:55.486 INFO:teuthology.orchestra.run.smithi162.stdout: "min_compat_client": "jewel", 2020-11-28T21:49:55.486 INFO:teuthology.orchestra.run.smithi162.stdout: "require_osd_release": "octopus", 2020-11-28T21:49:55.486 INFO:teuthology.orchestra.run.smithi162.stdout: "pools": [ 2020-11-28T21:49:55.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.487 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 1, 2020-11-28T21:49:55.487 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "device_health_metrics", 2020-11-28T21:49:55.487 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:46:15.639551+0000", 2020-11-28T21:49:55.487 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:49:55.487 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:49:55.488 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:49:55.488 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:49:55.488 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:49:55.488 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:49:55.488 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:49:55.489 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:49:55.489 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 1, 2020-11-28T21:49:55.489 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 1, 2020-11-28T21:49:55.489 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 1, 2020-11-28T21:49:55.489 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 1, 2020-11-28T21:49:55.490 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 1, 2020-11-28T21:49:55.490 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:49:55.490 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:49:55.490 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:49:55.491 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:49:55.491 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:49:55.491 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:49:55.491 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:49:55.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.492 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "16", 2020-11-28T21:49:55.492 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:49:55.492 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:49:55.492 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:49:55.492 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:49:55.493 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:49:55.493 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:49:55.493 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:49:55.493 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:49:55.493 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:49:55.494 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:49:55.494 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:49:55.494 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:49:55.494 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:49:55.495 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:49:55.495 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:49:55.495 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:49:55.495 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:49:55.495 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:49:55.496 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:49:55.496 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:49:55.496 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:49:55.496 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:49:55.497 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:49:55.497 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:49:55.497 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:49:55.497 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:49:55.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.498 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:49:55.498 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:49:55.498 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:49:55.498 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:49:55.499 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:49:55.499 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:49:55.499 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:49:55.499 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:49:55.500 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:49:55.500 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:49:55.500 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:49:55.500 INFO:teuthology.orchestra.run.smithi162.stdout: "options": { 2020-11-28T21:49:55.500 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_min": 1 2020-11-28T21:49:55.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.501 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:49:55.501 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr_devicehealth": {} 2020-11-28T21:49:55.501 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:49:55.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.502 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 2, 2020-11-28T21:49:55.502 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "rbd", 2020-11-28T21:49:55.502 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:46:23.901366+0000", 2020-11-28T21:49:55.503 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:49:55.503 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:49:55.503 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:49:55.503 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:49:55.504 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:49:55.504 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:49:55.504 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:49:55.504 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:49:55.504 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2020-11-28T21:49:55.505 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2020-11-28T21:49:55.505 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2020-11-28T21:49:55.505 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2020-11-28T21:49:55.505 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2020-11-28T21:49:55.506 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:49:55.506 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:49:55.506 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:49:55.506 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:49:55.506 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:49:55.507 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:49:55.507 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:49:55.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.507 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "19", 2020-11-28T21:49:55.507 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:49:55.508 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:49:55.508 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:49:55.508 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:49:55.508 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:49:55.508 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:49:55.509 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:49:55.509 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:49:55.509 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:49:55.509 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:49:55.509 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:49:55.510 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:49:55.510 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:49:55.510 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:49:55.510 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:49:55.511 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:49:55.511 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:49:55.511 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:49:55.511 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:49:55.511 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:49:55.512 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:49:55.512 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:49:55.512 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:49:55.512 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:49:55.513 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:49:55.513 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:49:55.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.513 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:49:55.513 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:49:55.514 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:49:55.514 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:49:55.514 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:49:55.514 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:49:55.515 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:49:55.515 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:49:55.515 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:49:55.515 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:49:55.515 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:49:55.516 INFO:teuthology.orchestra.run.smithi162.stdout: "options": {}, 2020-11-28T21:49:55.516 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:49:55.516 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd": {} 2020-11-28T21:49:55.516 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:49:55.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.517 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 7, 2020-11-28T21:49:55.517 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "cephfs_metadata", 2020-11-28T21:49:55.517 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:48:25.281355+0000", 2020-11-28T21:49:55.518 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:49:55.518 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:49:55.518 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:49:55.518 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:49:55.519 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:49:55.519 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:49:55.519 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:49:55.519 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:49:55.519 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2020-11-28T21:49:55.520 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2020-11-28T21:49:55.520 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2020-11-28T21:49:55.520 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2020-11-28T21:49:55.520 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2020-11-28T21:49:55.520 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:49:55.521 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:49:55.521 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:49:55.521 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:49:55.521 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:49:55.521 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:49:55.522 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:49:55.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.522 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "43", 2020-11-28T21:49:55.522 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:49:55.522 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:49:55.523 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:49:55.523 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:49:55.523 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:49:55.523 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:49:55.524 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:49:55.524 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:49:55.524 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:49:55.524 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:49:55.524 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:49:55.525 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:49:55.525 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:49:55.525 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:49:55.525 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:49:55.525 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:49:55.526 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:49:55.526 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:49:55.526 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:49:55.526 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:49:55.526 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:49:55.527 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:49:55.527 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:49:55.527 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:49:55.527 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:49:55.527 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:49:55.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.528 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:49:55.528 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:49:55.528 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:49:55.529 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:49:55.529 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:49:55.529 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:49:55.529 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:49:55.529 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:49:55.530 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:49:55.530 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:49:55.530 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:49:55.530 INFO:teuthology.orchestra.run.smithi162.stdout: "options": { 2020-11-28T21:49:55.531 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_bias": 4, 2020-11-28T21:49:55.531 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_min": 16, 2020-11-28T21:49:55.531 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_priority": 5 2020-11-28T21:49:55.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.532 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:49:55.532 INFO:teuthology.orchestra.run.smithi162.stdout: "cephfs": { 2020-11-28T21:49:55.532 INFO:teuthology.orchestra.run.smithi162.stdout: "metadata": "cephfs" 2020-11-28T21:49:55.532 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:49:55.532 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:49:55.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.534 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 8, 2020-11-28T21:49:55.534 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "cephfs_data", 2020-11-28T21:49:55.534 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:48:25.782088+0000", 2020-11-28T21:49:55.535 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:49:55.535 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:49:55.535 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:49:55.535 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:49:55.536 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:49:55.536 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:49:55.536 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:49:55.536 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:49:55.536 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2020-11-28T21:49:55.537 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2020-11-28T21:49:55.537 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2020-11-28T21:49:55.537 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2020-11-28T21:49:55.537 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2020-11-28T21:49:55.537 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:49:55.538 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:49:55.538 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:49:55.538 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:49:55.538 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:49:55.539 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:49:55.539 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:49:55.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.539 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "44", 2020-11-28T21:49:55.539 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:49:55.540 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:49:55.540 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:49:55.540 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:49:55.540 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:49:55.540 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:49:55.541 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:49:55.541 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:49:55.541 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:49:55.541 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:49:55.541 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:49:55.542 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:49:55.542 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:49:55.542 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:49:55.542 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:49:55.542 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:49:55.543 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:49:55.543 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:49:55.543 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:49:55.543 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:49:55.544 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:49:55.544 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:49:55.544 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:49:55.544 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:49:55.544 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:49:55.545 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:49:55.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.545 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:49:55.545 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:49:55.545 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:49:55.546 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:49:55.546 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:49:55.546 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:49:55.546 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:49:55.546 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:49:55.547 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:49:55.547 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:49:55.547 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:49:55.547 INFO:teuthology.orchestra.run.smithi162.stdout: "options": {}, 2020-11-28T21:49:55.547 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:49:55.548 INFO:teuthology.orchestra.run.smithi162.stdout: "cephfs": { 2020-11-28T21:49:55.548 INFO:teuthology.orchestra.run.smithi162.stdout: "data": "cephfs" 2020-11-28T21:49:55.548 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:49:55.548 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:49:55.548 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:49:55.549 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:49:55.549 INFO:teuthology.orchestra.run.smithi162.stdout: "osds": [ 2020-11-28T21:49:55.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.549 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 0, 2020-11-28T21:49:55.549 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "99c7d9ae-adfa-4fd7-880a-23e6baf3281b", 2020-11-28T21:49:55.550 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:49:55.550 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:49:55.550 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:49:55.550 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:49:55.550 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:49:55.551 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:49:55.551 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:49:55.551 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 31, 2020-11-28T21:49:55.551 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:49:55.551 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:49:55.552 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:49:55.552 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:49:55.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.552 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:49:55.552 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6810", 2020-11-28T21:49:55.553 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:49:55.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.553 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:49:55.553 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6811", 2020-11-28T21:49:55.554 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:49:55.554 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:49:55.554 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:49:55.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.554 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:49:55.555 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:49:55.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.555 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:49:55.555 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6812", 2020-11-28T21:49:55.555 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:49:55.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.556 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:49:55.556 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6813", 2020-11-28T21:49:55.556 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:49:55.556 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:49:55.557 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:49:55.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.557 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:49:55.557 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:49:55.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.558 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:49:55.558 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6816", 2020-11-28T21:49:55.558 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:49:55.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.559 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:49:55.559 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6817", 2020-11-28T21:49:55.559 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:49:55.559 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:49:55.559 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:49:55.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.560 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:49:55.560 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:49:55.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.560 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:49:55.561 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6814", 2020-11-28T21:49:55.561 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:49:55.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.561 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:49:55.562 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6815", 2020-11-28T21:49:55.562 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:49:55.562 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:49:55.562 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:49:55.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.563 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6811/12313", 2020-11-28T21:49:55.563 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6813/12313", 2020-11-28T21:49:55.563 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6817/12313", 2020-11-28T21:49:55.563 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6815/12313", 2020-11-28T21:49:55.563 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:49:55.564 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:49:55.564 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:49:55.564 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:49:55.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.565 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 1, 2020-11-28T21:49:55.565 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "6ce6220a-00ef-4d13-abba-56f87636fb5f", 2020-11-28T21:49:55.565 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:49:55.565 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:49:55.565 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:49:55.566 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:49:55.566 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:49:55.566 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:49:55.566 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:49:55.566 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 41, 2020-11-28T21:49:55.567 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:49:55.567 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:49:55.567 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:49:55.567 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:49:55.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.568 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:49:55.568 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6802", 2020-11-28T21:49:55.568 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:49:55.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.569 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:49:55.569 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6803", 2020-11-28T21:49:55.569 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:49:55.569 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:49:55.569 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:49:55.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.570 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:49:55.570 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:49:55.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.570 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:49:55.571 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6804", 2020-11-28T21:49:55.571 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:49:55.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.571 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:49:55.572 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6805", 2020-11-28T21:49:55.572 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:49:55.572 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:49:55.572 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:49:55.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.573 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:49:55.573 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:49:55.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.573 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:49:55.573 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6808", 2020-11-28T21:49:55.574 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:49:55.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.574 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:49:55.574 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6809", 2020-11-28T21:49:55.575 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:49:55.575 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:49:55.575 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:49:55.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.575 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:49:55.576 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:49:55.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.576 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:49:55.576 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6806", 2020-11-28T21:49:55.576 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:49:55.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.577 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:49:55.577 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6807", 2020-11-28T21:49:55.577 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:49:55.578 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:49:55.578 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:49:55.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.578 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6803/12311", 2020-11-28T21:49:55.578 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6805/12311", 2020-11-28T21:49:55.579 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6809/12311", 2020-11-28T21:49:55.579 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6807/12311", 2020-11-28T21:49:55.579 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:49:55.579 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:49:55.579 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:49:55.579 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:49:55.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.580 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 2, 2020-11-28T21:49:55.580 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "db4e510d-27c1-47fc-a902-8b32e7dfd73e", 2020-11-28T21:49:55.580 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:49:55.581 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:49:55.581 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:49:55.581 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:49:55.581 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:49:55.581 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:49:55.582 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:49:55.582 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 41, 2020-11-28T21:49:55.582 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:49:55.582 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:49:55.582 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:49:55.583 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:49:55.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.583 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:49:55.583 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6818", 2020-11-28T21:49:55.583 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:49:55.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.584 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:49:55.584 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6819", 2020-11-28T21:49:55.584 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:49:55.585 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:49:55.585 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:49:55.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.585 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:49:55.585 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:49:55.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.586 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:49:55.586 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6820", 2020-11-28T21:49:55.586 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:49:55.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.587 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:49:55.587 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6821", 2020-11-28T21:49:55.587 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:49:55.587 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:49:55.588 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:49:55.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.588 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:49:55.588 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:49:55.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.589 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:49:55.589 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6824", 2020-11-28T21:49:55.589 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:49:55.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.590 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:49:55.590 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6825", 2020-11-28T21:49:55.590 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:49:55.590 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:49:55.590 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:49:55.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.591 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:49:55.591 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:49:55.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.591 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:49:55.592 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6822", 2020-11-28T21:49:55.592 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:49:55.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.592 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:49:55.593 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6823", 2020-11-28T21:49:55.593 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:49:55.593 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:49:55.593 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:49:55.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.594 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6819/12312", 2020-11-28T21:49:55.594 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6821/12312", 2020-11-28T21:49:55.594 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6825/12312", 2020-11-28T21:49:55.594 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6823/12312", 2020-11-28T21:49:55.594 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:49:55.595 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:49:55.595 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:49:55.595 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:49:55.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.596 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 3, 2020-11-28T21:49:55.596 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "4094cb87-f04a-45e1-8710-fb4b651949f6", 2020-11-28T21:49:55.596 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:49:55.596 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:49:55.596 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:49:55.596 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:49:55.597 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:49:55.597 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:49:55.597 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:49:55.597 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 40, 2020-11-28T21:49:55.597 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:49:55.598 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:49:55.598 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:49:55.598 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:49:55.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.598 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:49:55.599 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6826", 2020-11-28T21:49:55.599 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:49:55.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.599 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:49:55.600 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6827", 2020-11-28T21:49:55.600 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:49:55.600 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:49:55.600 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:49:55.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.601 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:49:55.601 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:49:55.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.601 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:49:55.601 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6828", 2020-11-28T21:49:55.602 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:49:55.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.602 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.602 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:49:55.602 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6829", 2020-11-28T21:49:55.603 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:49:55.603 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:49:55.603 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:49:55.603 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.603 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:49:55.604 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:49:55.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.604 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:49:55.604 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6832", 2020-11-28T21:49:55.604 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:49:55.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.605 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:49:55.605 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6833", 2020-11-28T21:49:55.605 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:49:55.605 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:49:55.606 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:49:55.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.606 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:49:55.606 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:49:55.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.607 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:49:55.607 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6830", 2020-11-28T21:49:55.607 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:49:55.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.608 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:49:55.608 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6831", 2020-11-28T21:49:55.608 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:49:55.608 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:49:55.608 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:49:55.609 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.609 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6827/12323", 2020-11-28T21:49:55.609 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6829/12323", 2020-11-28T21:49:55.609 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6833/12323", 2020-11-28T21:49:55.609 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6831/12323", 2020-11-28T21:49:55.610 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:49:55.610 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:49:55.610 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:49:55.610 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:49:55.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.611 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.611 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 4, 2020-11-28T21:49:55.611 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "b907bf53-c297-4756-847b-3be96680eef5", 2020-11-28T21:49:55.611 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:49:55.611 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:49:55.612 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:49:55.612 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:49:55.612 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:49:55.612 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:49:55.612 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:49:55.613 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 40, 2020-11-28T21:49:55.613 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:49:55.613 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:49:55.613 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:49:55.613 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:49:55.614 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.614 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:49:55.614 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6800", 2020-11-28T21:49:55.614 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:49:55.614 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.615 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.615 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:49:55.615 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6801", 2020-11-28T21:49:55.615 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:49:55.615 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:49:55.616 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:49:55.616 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.616 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:49:55.616 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:49:55.616 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.617 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:49:55.617 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6802", 2020-11-28T21:49:55.617 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:49:55.617 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.617 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.618 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:49:55.618 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6803", 2020-11-28T21:49:55.618 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:49:55.618 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:49:55.618 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:49:55.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.619 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:49:55.619 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:49:55.619 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.619 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:49:55.620 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6806", 2020-11-28T21:49:55.620 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:49:55.620 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.620 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.620 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:49:55.621 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6807", 2020-11-28T21:49:55.621 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:49:55.621 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:49:55.621 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:49:55.621 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.622 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:49:55.622 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:49:55.622 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.622 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:49:55.622 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6804", 2020-11-28T21:49:55.623 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:49:55.623 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.623 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.623 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:49:55.623 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6805", 2020-11-28T21:49:55.623 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:49:55.624 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:49:55.624 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:49:55.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.624 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6801/12527", 2020-11-28T21:49:55.624 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6803/12527", 2020-11-28T21:49:55.625 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6807/12527", 2020-11-28T21:49:55.625 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6805/12527", 2020-11-28T21:49:55.625 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:49:55.625 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:49:55.625 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:49:55.626 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:49:55.626 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.626 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.626 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 5, 2020-11-28T21:49:55.626 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "55ba305d-0505-4212-aebb-631e8554b440", 2020-11-28T21:49:55.627 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:49:55.627 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:49:55.627 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:49:55.627 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:49:55.627 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:49:55.628 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:49:55.628 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:49:55.628 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 41, 2020-11-28T21:49:55.628 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:49:55.628 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:49:55.629 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:49:55.629 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:49:55.629 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.629 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:49:55.629 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6808", 2020-11-28T21:49:55.630 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:49:55.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.630 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:49:55.630 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6809", 2020-11-28T21:49:55.630 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:49:55.631 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:49:55.631 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:49:55.631 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.631 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:49:55.631 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:49:55.632 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.632 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:49:55.632 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6810", 2020-11-28T21:49:55.632 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:49:55.632 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.633 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.633 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:49:55.633 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6811", 2020-11-28T21:49:55.633 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:49:55.633 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:49:55.634 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:49:55.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.634 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:49:55.634 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:49:55.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.635 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:49:55.635 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6814", 2020-11-28T21:49:55.635 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:49:55.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.636 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:49:55.636 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6815", 2020-11-28T21:49:55.636 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:49:55.636 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:49:55.636 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:49:55.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.637 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:49:55.637 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:49:55.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.637 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:49:55.637 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6812", 2020-11-28T21:49:55.638 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:49:55.638 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.638 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:49:55.638 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6813", 2020-11-28T21:49:55.639 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:49:55.639 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:49:55.639 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:49:55.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.639 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6809/12529", 2020-11-28T21:49:55.640 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6811/12529", 2020-11-28T21:49:55.640 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6815/12529", 2020-11-28T21:49:55.640 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6813/12529", 2020-11-28T21:49:55.640 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:49:55.640 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:49:55.641 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:49:55.641 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:49:55.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.641 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 6, 2020-11-28T21:49:55.642 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "9ada6051-493b-46b5-ba22-1b401440490f", 2020-11-28T21:49:55.642 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:49:55.642 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:49:55.642 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:49:55.642 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:49:55.643 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:49:55.643 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:49:55.643 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:49:55.643 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 41, 2020-11-28T21:49:55.643 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:49:55.644 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:49:55.644 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:49:55.644 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:49:55.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.644 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:49:55.645 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6824", 2020-11-28T21:49:55.645 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:49:55.645 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.645 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:49:55.646 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6825", 2020-11-28T21:49:55.646 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:49:55.646 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:49:55.646 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:49:55.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.647 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:49:55.647 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:49:55.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.647 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:49:55.647 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6826", 2020-11-28T21:49:55.647 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:49:55.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.648 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:49:55.648 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6827", 2020-11-28T21:49:55.648 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:49:55.649 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:49:55.649 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:49:55.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.649 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:49:55.649 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:49:55.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.650 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:49:55.650 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6830", 2020-11-28T21:49:55.650 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:49:55.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.651 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:49:55.651 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6831", 2020-11-28T21:49:55.651 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:49:55.651 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:49:55.652 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:49:55.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.652 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:49:55.652 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:49:55.652 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.653 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:49:55.653 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6828", 2020-11-28T21:49:55.653 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:49:55.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.654 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:49:55.654 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6829", 2020-11-28T21:49:55.654 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:49:55.654 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:49:55.654 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:49:55.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.655 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6825/12530", 2020-11-28T21:49:55.655 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6827/12530", 2020-11-28T21:49:55.655 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6831/12530", 2020-11-28T21:49:55.655 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6829/12530", 2020-11-28T21:49:55.656 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:49:55.656 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:49:55.656 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:49:55.656 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:49:55.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.657 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 7, 2020-11-28T21:49:55.657 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "c328b06d-a216-4cba-aad7-9ad081340613", 2020-11-28T21:49:55.657 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:49:55.657 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:49:55.658 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:49:55.658 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:49:55.658 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:49:55.658 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:49:55.658 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:49:55.659 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 41, 2020-11-28T21:49:55.659 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:49:55.659 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:49:55.659 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:49:55.659 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:49:55.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.660 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:49:55.660 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6816", 2020-11-28T21:49:55.660 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:49:55.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.661 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:49:55.661 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6817", 2020-11-28T21:49:55.661 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:49:55.661 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:49:55.662 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:49:55.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.662 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:49:55.662 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:49:55.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.663 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:49:55.663 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6818", 2020-11-28T21:49:55.663 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:49:55.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.663 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:49:55.664 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6819", 2020-11-28T21:49:55.664 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:49:55.664 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:49:55.664 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:49:55.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.665 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:49:55.665 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:49:55.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.665 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:49:55.665 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6822", 2020-11-28T21:49:55.666 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:49:55.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.666 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:49:55.666 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6823", 2020-11-28T21:49:55.667 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:49:55.667 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:49:55.667 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:49:55.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.667 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:49:55.668 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:49:55.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.668 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:49:55.668 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6820", 2020-11-28T21:49:55.669 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:49:55.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.669 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:49:55.669 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6821", 2020-11-28T21:49:55.669 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:49:55.670 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:49:55.670 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:49:55.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.670 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6817/12528", 2020-11-28T21:49:55.670 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6819/12528", 2020-11-28T21:49:55.671 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6823/12528", 2020-11-28T21:49:55.671 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6821/12528", 2020-11-28T21:49:55.671 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:49:55.671 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:49:55.671 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:49:55.672 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:49:55.672 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:49:55.672 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:49:55.672 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_xinfo": [ 2020-11-28T21:49:55.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.673 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 0, 2020-11-28T21:49:55.673 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:49:55.673 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:49:55.674 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:49:55.674 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:49:55.674 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:49:55.674 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.038052+0000", 2020-11-28T21:49:55.674 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:49:55.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.675 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 1, 2020-11-28T21:49:55.675 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:49:55.675 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:49:55.676 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:49:55.676 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:49:55.676 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:49:55.676 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:17.947136+0000", 2020-11-28T21:49:55.676 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:49:55.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.677 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 2, 2020-11-28T21:49:55.677 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:49:55.677 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:49:55.678 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:49:55.678 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:49:55.678 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:49:55.678 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.001186+0000", 2020-11-28T21:49:55.678 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:49:55.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.679 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 3, 2020-11-28T21:49:55.679 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:49:55.679 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:49:55.680 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:49:55.680 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:49:55.680 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:49:55.680 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.199284+0000", 2020-11-28T21:49:55.680 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:49:55.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.681 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 4, 2020-11-28T21:49:55.681 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:49:55.681 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:49:55.681 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:49:55.682 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:49:55.682 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:49:55.682 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.038823+0000", 2020-11-28T21:49:55.682 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:49:55.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.683 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 5, 2020-11-28T21:49:55.683 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:49:55.683 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:49:55.683 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:49:55.684 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:49:55.684 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:49:55.684 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.009322+0000", 2020-11-28T21:49:55.684 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:49:55.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.685 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 6, 2020-11-28T21:49:55.685 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:49:55.685 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:49:55.685 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:49:55.686 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:49:55.686 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:49:55.686 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.238927+0000", 2020-11-28T21:49:55.686 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:49:55.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:49:55.687 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 7, 2020-11-28T21:49:55.687 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:49:55.687 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:49:55.687 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:49:55.688 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:49:55.688 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:49:55.688 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.029839+0000", 2020-11-28T21:49:55.688 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:49:55.688 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:49:55.689 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:49:55.689 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_upmap": [], 2020-11-28T21:49:55.689 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_upmap_items": [], 2020-11-28T21:49:55.689 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_temp": [], 2020-11-28T21:49:55.689 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_temp": [], 2020-11-28T21:49:55.689 INFO:teuthology.orchestra.run.smithi162.stdout: "blacklist": {}, 2020-11-28T21:49:55.690 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profiles": { 2020-11-28T21:49:55.690 INFO:teuthology.orchestra.run.smithi162.stdout: "default": { 2020-11-28T21:49:55.690 INFO:teuthology.orchestra.run.smithi162.stdout: "crush-failure-domain": "osd", 2020-11-28T21:49:55.690 INFO:teuthology.orchestra.run.smithi162.stdout: "k": "2", 2020-11-28T21:49:55.690 INFO:teuthology.orchestra.run.smithi162.stdout: "m": "1", 2020-11-28T21:49:55.691 INFO:teuthology.orchestra.run.smithi162.stdout: "plugin": "jerasure", 2020-11-28T21:49:55.691 INFO:teuthology.orchestra.run.smithi162.stdout: "ruleset-failure-domain": "osd", 2020-11-28T21:49:55.691 INFO:teuthology.orchestra.run.smithi162.stdout: "technique": "reed_sol_van" 2020-11-28T21:49:55.691 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:49:55.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:49:55.692 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps_queue": [], 2020-11-28T21:49:55.692 INFO:teuthology.orchestra.run.smithi162.stdout: "new_removed_snaps": [], 2020-11-28T21:49:55.692 INFO:teuthology.orchestra.run.smithi162.stdout: "new_purged_snaps": [], 2020-11-28T21:49:55.692 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_node_flags": {}, 2020-11-28T21:49:55.692 INFO:teuthology.orchestra.run.smithi162.stdout: "device_class_flags": {} 2020-11-28T21:49:55.693 INFO:teuthology.orchestra.run.smithi162.stdout:} 2020-11-28T21:49:55.694 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:49:55.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.815+0000 7fcfe1128700 1 Processor -- start 2020-11-28T21:49:55.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.815+0000 7fcfe1128700 1 -- start start 2020-11-28T21:49:55.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.815+0000 7fcfe1128700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcfdc0603d0 0x7fcfdc0b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:55.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.815+0000 7fcfe1128700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfdc0b4110 con 0x7fcfdc142a00 2020-11-28T21:49:55.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.815+0000 7fcfe1128700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfdc0b43f0 con 0x7fcfdc0b4c70 2020-11-28T21:49:55.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.815+0000 7fcfe1128700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc060f60 con 0x7fcfdc0603d0 2020-11-28T21:49:55.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.815+0000 7fcfe0927700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fcfdc0b4c70 0x7fcfdc143020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:46548/0 (socket says 172.21.15.162:46548) 2020-11-28T21:49:55.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.815+0000 7fcfe0927700 1 -- 172.21.15.162:0/3586242382 learned_addr learned my addr 172.21.15.162:0/3586242382 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:49:55.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.815+0000 7fcfdbfff700 1 --2- 172.21.15.162:0/3586242382 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcfdc0603d0 0x7fcfdc0b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:55.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.815+0000 7fcfdaffd700 1 -- 172.21.15.162:0/3586242382 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2969226822 0 0) 0x7fcfdc0b43f0 con 0x7fcfdc0b4c70 2020-11-28T21:49:55.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.815+0000 7fcfdaffd700 1 -- 172.21.15.162:0/3586242382 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfc8003230 con 0x7fcfdc0b4c70 2020-11-28T21:49:55.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.819+0000 7fcfdaffd700 1 -- 172.21.15.162:0/3586242382 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3853290196 0 0) 0x7fcfdc0b4110 con 0x7fcfdc142a00 2020-11-28T21:49:55.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.819+0000 7fcfdaffd700 1 -- 172.21.15.162:0/3586242382 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfdc0b43f0 con 0x7fcfdc142a00 2020-11-28T21:49:55.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.819+0000 7fcfdbfff700 1 -- 172.21.15.162:0/3586242382 >> v1:172.21.15.162:6789/0 conn(0x7fcfdc0b4c70 legacy=0x7fcfdc143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:55.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.819+0000 7fcfdbfff700 1 -- 172.21.15.162:0/3586242382 >> v1:172.21.15.118:6789/0 conn(0x7fcfdc142a00 legacy=0x7fcfdc09cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:55.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.819+0000 7fcfdbfff700 1 -- 172.21.15.162:0/3586242382 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfdc0b4690 con 0x7fcfdc0603d0 2020-11-28T21:49:55.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.819+0000 7fcfdbfff700 1 --2- 172.21.15.162:0/3586242382 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcfdc0603d0 0x7fcfdc0b3a70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fcfcc002f30 tx=0x7fcfcc006dc0).ready entity=mon.2 client_cookie=aa7744904e923dab server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:55.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.819+0000 7fcfdaffd700 1 -- 172.21.15.162:0/3586242382 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcfcc009730 con 0x7fcfdc0603d0 2020-11-28T21:49:55.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.819+0000 7fcfdaffd700 1 -- 172.21.15.162:0/3586242382 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfcc009890 con 0x7fcfdc0603d0 2020-11-28T21:49:55.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.819+0000 7fcfe1128700 1 -- 172.21.15.162:0/3586242382 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcfdc0603d0 msgr2=0x7fcfdc0b3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:55.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.819+0000 7fcfdaffd700 1 -- 172.21.15.162:0/3586242382 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcfcc009bf0 con 0x7fcfdc0603d0 2020-11-28T21:49:55.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.819+0000 7fcfe1128700 1 --2- 172.21.15.162:0/3586242382 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcfdc0603d0 0x7fcfdc0b3a70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fcfcc002f30 tx=0x7fcfcc006dc0).stop 2020-11-28T21:49:55.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.819+0000 7fcfe1128700 1 -- 172.21.15.162:0/3586242382 shutdown_connections 2020-11-28T21:49:55.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.819+0000 7fcfe1128700 1 --2- 172.21.15.162:0/3586242382 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcfdc0603d0 0x7fcfdc0b3a70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:55.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.819+0000 7fcfe1128700 1 -- 172.21.15.162:0/3586242382 >> 172.21.15.162:0/3586242382 conn(0x7fcfdc066a40 msgr2=0x7fcfdc066e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:49:55.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.819+0000 7fcfe1128700 1 -- 172.21.15.162:0/3586242382 shutdown_connections 2020-11-28T21:49:55.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.819+0000 7fcfe1128700 1 -- 172.21.15.162:0/3586242382 wait complete. 2020-11-28T21:49:55.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.819+0000 7fcfe1128700 1 Processor -- start 2020-11-28T21:49:55.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.819+0000 7fcfe1128700 1 -- start start 2020-11-28T21:49:55.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.819+0000 7fcfe1128700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfdc0603d0 0x7fcfdc0afbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:55.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.819+0000 7fcfe1128700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcfdc0b4c70 0x7fcfdc146c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:55.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.819+0000 7fcfe1128700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcfdc1452b0 0x7fcfdc0adf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:55.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.819+0000 7fcfe1128700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc0b00b0 con 0x7fcfdc0b4c70 2020-11-28T21:49:55.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.819+0000 7fcfe1128700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc155ad0 con 0x7fcfdc0603d0 2020-11-28T21:49:55.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.819+0000 7fcfe1128700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc154bf0 con 0x7fcfdc1452b0 2020-11-28T21:49:55.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.819+0000 7fcfdbfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfdc0603d0 0x7fcfdc0afbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:55.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.819+0000 7fcfdbfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfdc0603d0 0x7fcfdc0afbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39706/0 (socket says 172.21.15.162:39706) 2020-11-28T21:49:55.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.819+0000 7fcfdbfff700 1 -- 172.21.15.162:0/2692466601 learned_addr learned my addr 172.21.15.162:0/2692466601 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:49:55.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.823+0000 7fcfdb7fe700 1 --2- 172.21.15.162:0/2692466601 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcfdc0b4c70 0x7fcfdc146c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:55.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.823+0000 7fcfe0927700 1 --2- 172.21.15.162:0/2692466601 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcfdc1452b0 0x7fcfdc0adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:55.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.823+0000 7fcfdbfff700 1 -- 172.21.15.162:0/2692466601 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcfdc1452b0 msgr2=0x7fcfdc0adf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:55.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.823+0000 7fcfdbfff700 1 --2- 172.21.15.162:0/2692466601 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcfdc1452b0 0x7fcfdc0adf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:55.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.823+0000 7fcfdbfff700 1 -- 172.21.15.162:0/2692466601 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcfdc0b4c70 msgr2=0x7fcfdc146c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:55.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.823+0000 7fcfdbfff700 1 --2- 172.21.15.162:0/2692466601 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcfdc0b4c70 0x7fcfdc146c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:55.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.823+0000 7fcfdbfff700 1 -- 172.21.15.162:0/2692466601 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfdc154e50 con 0x7fcfdc0603d0 2020-11-28T21:49:55.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.823+0000 7fcfdbfff700 1 --2- 172.21.15.162:0/2692466601 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfdc0603d0 0x7fcfdc0afbc0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7fcfdc1559b0 tx=0x7fcfcc015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:55.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.823+0000 7fcfd97fa700 1 -- 172.21.15.162:0/2692466601 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcfcc011ac0 con 0x7fcfdc0603d0 2020-11-28T21:49:55.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.823+0000 7fcfd97fa700 1 -- 172.21.15.162:0/2692466601 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfcc011c20 con 0x7fcfdc0603d0 2020-11-28T21:49:55.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.823+0000 7fcfe1128700 1 -- 172.21.15.162:0/2692466601 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfdc1550f0 con 0x7fcfdc0603d0 2020-11-28T21:49:55.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.823+0000 7fcfd97fa700 1 -- 172.21.15.162:0/2692466601 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcfcc01d450 con 0x7fcfdc0603d0 2020-11-28T21:49:55.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.823+0000 7fcfe1128700 1 -- 172.21.15.162:0/2692466601 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfdc147180 con 0x7fcfdc0603d0 2020-11-28T21:49:55.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.823+0000 7fcfd97fa700 1 -- 172.21.15.162:0/2692466601 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fcfcc00f070 con 0x7fcfdc0603d0 2020-11-28T21:49:55.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.823+0000 7fcfe1128700 1 -- 172.21.15.162:0/2692466601 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfdc057050 con 0x7fcfdc0603d0 2020-11-28T21:49:55.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.823+0000 7fcfd97fa700 1 --2- 172.21.15.162:0/2692466601 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcfbc055470 0x7fcfbc057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:55.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.827+0000 7fcfd97fa700 1 -- 172.21.15.162:0/2692466601 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fcfcc06f780 con 0x7fcfdc0603d0 2020-11-28T21:49:55.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.827+0000 7fcfdb7fe700 1 --2- 172.21.15.162:0/2692466601 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcfbc055470 0x7fcfbc057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:55.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.827+0000 7fcfdb7fe700 1 --2- 172.21.15.162:0/2692466601 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcfbc055470 0x7fcfbc057930 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7fcfd000d000 tx=0x7fcfd0009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:55.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:55.831+0000 7fcfd97fa700 1 -- 172.21.15.162:0/2692466601 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fcfcc045a80 con 0x7fcfdc0603d0 2020-11-28T21:49:56.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.019+0000 7fcfe1128700 1 -- 172.21.15.162:0/2692466601 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcfdc069b60 con 0x7fcfdc0603d0 2020-11-28T21:49:56.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.019+0000 7fcfd97fa700 1 -- 172.21.15.162:0/2692466601 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+2497 (secure 0 0 0) 0x7fcfcc0490a0 con 0x7fcfdc0603d0 2020-11-28T21:49:56.018 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:49:56.018 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":15,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4856,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/913296056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":913296056},{"type":"v1","addr":"172.21.15.118:6833","nonce":913296056}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":13},{"gid":5209,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6835/1318110356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1318110356},{"type":"v1","addr":"172.21.15.162:6835","nonce":1318110356}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:48:26.782354+0000","modified":"2020-11-28T21:48:27.785277+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5212},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5212":{"gid":5212,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/2706248574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2706248574},{"type":"v1","addr":"172.21.15.162:6837","nonce":2706248574}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-28T21:49:56.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.019+0000 7fcfe1128700 1 -- 172.21.15.162:0/2692466601 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcfbc055470 msgr2=0x7fcfbc057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:56.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.019+0000 7fcfe1128700 1 --2- 172.21.15.162:0/2692466601 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcfbc055470 0x7fcfbc057930 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7fcfd000d000 tx=0x7fcfd0009040).stop 2020-11-28T21:49:56.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.019+0000 7fcfe1128700 1 -- 172.21.15.162:0/2692466601 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfdc0603d0 msgr2=0x7fcfdc0afbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:56.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.019+0000 7fcfe1128700 1 --2- 172.21.15.162:0/2692466601 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfdc0603d0 0x7fcfdc0afbc0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7fcfdc1559b0 tx=0x7fcfcc015000).stop 2020-11-28T21:49:56.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.019+0000 7fcfe1128700 1 -- 172.21.15.162:0/2692466601 shutdown_connections 2020-11-28T21:49:56.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.019+0000 7fcfe1128700 1 --2- 172.21.15.162:0/2692466601 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcfbc055470 0x7fcfbc057930 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:56.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.019+0000 7fcfe1128700 1 --2- 172.21.15.162:0/2692466601 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcfdc1452b0 0x7fcfdc0adf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:56.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.019+0000 7fcfe1128700 1 --2- 172.21.15.162:0/2692466601 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfdc0603d0 0x7fcfdc0afbc0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:56.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.019+0000 7fcfe1128700 1 --2- 172.21.15.162:0/2692466601 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcfdc0b4c70 0x7fcfdc146c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:56.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.019+0000 7fcfe1128700 1 -- 172.21.15.162:0/2692466601 >> 172.21.15.162:0/2692466601 conn(0x7fcfdc066a40 msgr2=0x7fcfdc1421f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:49:56.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.019+0000 7fcfe1128700 1 -- 172.21.15.162:0/2692466601 shutdown_connections 2020-11-28T21:49:56.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.023+0000 7fcfe1128700 1 -- 172.21.15.162:0/2692466601 wait complete. 2020-11-28T21:49:56.022 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 15 2020-11-28T21:49:56.031 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2020-11-28T21:49:56.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.143+0000 7fb3d3fff700 1 Processor -- start 2020-11-28T21:49:56.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.143+0000 7fb3d3fff700 1 -- start start 2020-11-28T21:49:56.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.143+0000 7fb3d3fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb3d40acca0 0x7fb3d4142bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:56.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.143+0000 7fb3d3fff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3d40b3660 con 0x7fb3d40ac3b0 2020-11-28T21:49:56.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.143+0000 7fb3d3fff700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3d40b07e0 con 0x7fb3d40ac790 2020-11-28T21:49:56.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.143+0000 7fb3d3fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d40abe90 con 0x7fb3d40acca0 2020-11-28T21:49:56.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.143+0000 7fb3d27fc700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fb3d40ac790 0x7fb3d40b2f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:46558/0 (socket says 172.21.15.162:46558) 2020-11-28T21:49:56.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.143+0000 7fb3d27fc700 1 -- 172.21.15.162:0/2058778165 learned_addr learned my addr 172.21.15.162:0/2058778165 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:49:56.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.143+0000 7fb3d37fe700 1 --2- 172.21.15.162:0/2058778165 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb3d40acca0 0x7fb3d4142bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:56.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.143+0000 7fb3d1ffb700 1 -- 172.21.15.162:0/2058778165 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4290334776 0 0) 0x7fb3d40b07e0 con 0x7fb3d40ac790 2020-11-28T21:49:56.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.143+0000 7fb3d1ffb700 1 -- 172.21.15.162:0/2058778165 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3b8003230 con 0x7fb3d40ac790 2020-11-28T21:49:56.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.143+0000 7fb3d1ffb700 1 -- 172.21.15.162:0/2058778165 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2042992989 0 0) 0x7fb3d40b3660 con 0x7fb3d40ac3b0 2020-11-28T21:49:56.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.143+0000 7fb3d1ffb700 1 -- 172.21.15.162:0/2058778165 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3d40b07e0 con 0x7fb3d40ac3b0 2020-11-28T21:49:56.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.143+0000 7fb3d37fe700 1 -- 172.21.15.162:0/2058778165 >> v1:172.21.15.162:6789/0 conn(0x7fb3d40ac790 legacy=0x7fb3d40b2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:56.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.143+0000 7fb3d37fe700 1 -- 172.21.15.162:0/2058778165 >> v1:172.21.15.118:6789/0 conn(0x7fb3d40ac3b0 legacy=0x7fb3d40ab780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:56.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.143+0000 7fb3d37fe700 1 -- 172.21.15.162:0/2058778165 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d40b0a80 con 0x7fb3d40acca0 2020-11-28T21:49:56.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.143+0000 7fb3d37fe700 1 --2- 172.21.15.162:0/2058778165 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb3d40acca0 0x7fb3d4142bb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fb3cc0032b0 tx=0x7fb3cc0091a0).ready entity=mon.2 client_cookie=a0922a2bb5d724f8 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:56.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.147+0000 7fb3d1ffb700 1 -- 172.21.15.162:0/2058778165 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3cc00b940 con 0x7fb3d40acca0 2020-11-28T21:49:56.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.147+0000 7fb3d1ffb700 1 -- 172.21.15.162:0/2058778165 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3cc00baa0 con 0x7fb3d40acca0 2020-11-28T21:49:56.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.147+0000 7fb3d3fff700 1 -- 172.21.15.162:0/2058778165 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb3d40acca0 msgr2=0x7fb3d4142bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:56.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.147+0000 7fb3d1ffb700 1 -- 172.21.15.162:0/2058778165 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3cc00bc10 con 0x7fb3d40acca0 2020-11-28T21:49:56.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.147+0000 7fb3d3fff700 1 --2- 172.21.15.162:0/2058778165 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb3d40acca0 0x7fb3d4142bb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fb3cc0032b0 tx=0x7fb3cc0091a0).stop 2020-11-28T21:49:56.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.147+0000 7fb3d3fff700 1 -- 172.21.15.162:0/2058778165 shutdown_connections 2020-11-28T21:49:56.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.147+0000 7fb3d3fff700 1 --2- 172.21.15.162:0/2058778165 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb3d40acca0 0x7fb3d4142bb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:56.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.147+0000 7fb3d3fff700 1 -- 172.21.15.162:0/2058778165 >> 172.21.15.162:0/2058778165 conn(0x7fb3d40b6870 msgr2=0x7fb3d40b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:49:56.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.147+0000 7fb3d3fff700 1 -- 172.21.15.162:0/2058778165 shutdown_connections 2020-11-28T21:49:56.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.147+0000 7fb3d3fff700 1 -- 172.21.15.162:0/2058778165 wait complete. 2020-11-28T21:49:56.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.147+0000 7fb3d3fff700 1 Processor -- start 2020-11-28T21:49:56.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.147+0000 7fb3d3fff700 1 -- start start 2020-11-28T21:49:56.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.147+0000 7fb3d3fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3d40ac3b0 0x7fb3d40afa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:56.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.147+0000 7fb3d3fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb3d40ac790 0x7fb3d40af3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:56.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.151+0000 7fb3d3fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb3d40acca0 0x7fb3d40b5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:56.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.151+0000 7fb3d3fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d40647e0 con 0x7fb3d40ac790 2020-11-28T21:49:56.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.151+0000 7fb3d3fff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d4064a80 con 0x7fb3d40ac3b0 2020-11-28T21:49:56.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.151+0000 7fb3d3fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d40b4fa0 con 0x7fb3d40acca0 2020-11-28T21:49:56.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.151+0000 7fb3d2ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3d40ac3b0 0x7fb3d40afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:56.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.151+0000 7fb3d2ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3d40ac3b0 0x7fb3d40afa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39720/0 (socket says 172.21.15.162:39720) 2020-11-28T21:49:56.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.151+0000 7fb3d2ffd700 1 -- 172.21.15.162:0/245779021 learned_addr learned my addr 172.21.15.162:0/245779021 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:49:56.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.151+0000 7fb3d27fc700 1 --2- 172.21.15.162:0/245779021 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb3d40ac790 0x7fb3d40af3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:56.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.151+0000 7fb3d37fe700 1 --2- 172.21.15.162:0/245779021 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb3d40acca0 0x7fb3d40b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:56.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.151+0000 7fb3d2ffd700 1 -- 172.21.15.162:0/245779021 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb3d40acca0 msgr2=0x7fb3d40b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:56.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.151+0000 7fb3d2ffd700 1 --2- 172.21.15.162:0/245779021 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb3d40acca0 0x7fb3d40b5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:56.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.151+0000 7fb3d2ffd700 1 -- 172.21.15.162:0/245779021 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb3d40ac790 msgr2=0x7fb3d40af3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:56.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.151+0000 7fb3d2ffd700 1 --2- 172.21.15.162:0/245779021 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb3d40ac790 0x7fb3d40af3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:56.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.151+0000 7fb3d2ffd700 1 -- 172.21.15.162:0/245779021 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d4064460 con 0x7fb3d40ac3b0 2020-11-28T21:49:56.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.151+0000 7fb3d2ffd700 1 --2- 172.21.15.162:0/245779021 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3d40ac3b0 0x7fb3d40afa40 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7fb3c80010d0 tx=0x7fb3c8007a60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:56.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.151+0000 7fb3c3fff700 1 -- 172.21.15.162:0/245779021 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3c800f470 con 0x7fb3d40ac3b0 2020-11-28T21:49:56.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.151+0000 7fb3c3fff700 1 -- 172.21.15.162:0/245779021 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3c800f5d0 con 0x7fb3d40ac3b0 2020-11-28T21:49:56.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.151+0000 7fb3d3fff700 1 -- 172.21.15.162:0/245779021 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3d40aecf0 con 0x7fb3d40ac3b0 2020-11-28T21:49:56.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.151+0000 7fb3c3fff700 1 -- 172.21.15.162:0/245779021 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3c800f930 con 0x7fb3d40ac3b0 2020-11-28T21:49:56.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.151+0000 7fb3d3fff700 1 -- 172.21.15.162:0/245779021 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3d4065f60 con 0x7fb3d40ac3b0 2020-11-28T21:49:56.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.151+0000 7fb3d3fff700 1 -- 172.21.15.162:0/245779021 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3d4064c40 con 0x7fb3d40ac3b0 2020-11-28T21:49:56.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.151+0000 7fb3c3fff700 1 -- 172.21.15.162:0/245779021 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb3c8019030 con 0x7fb3d40ac3b0 2020-11-28T21:49:56.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.155+0000 7fb3c3fff700 1 --2- 172.21.15.162:0/245779021 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb3b0055330 0x7fb3b00577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:56.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.155+0000 7fb3c3fff700 1 -- 172.21.15.162:0/245779021 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb3c806d390 con 0x7fb3d40ac3b0 2020-11-28T21:49:56.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.155+0000 7fb3d27fc700 1 --2- 172.21.15.162:0/245779021 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb3b0055330 0x7fb3b00577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:56.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.155+0000 7fb3d27fc700 1 --2- 172.21.15.162:0/245779021 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb3b0055330 0x7fb3b00577f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7fb3c400d000 tx=0x7fb3c4009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:56.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.163+0000 7fb3c3fff700 1 -- 172.21.15.162:0/245779021 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb3c80435f0 con 0x7fb3d40ac3b0 2020-11-28T21:49:56.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:56.343+0000 7fb3d3fff700 1 -- 172.21.15.162:0/245779021 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb3d4057050 con 0x7fb3d40ac3b0 2020-11-28T21:49:57.041 INFO:tasks.ceph.mon.b.smithi118.stderr:2020-11-28T21:49:57.042+0000 7ff4e1520700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-11-28T21:49:57.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.051+0000 7fb3c3fff700 1 -- 172.21.15.162:0/245779021 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v16) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb3c8002e60 con 0x7fb3d40ac3b0 2020-11-28T21:49:57.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.055+0000 7fb3d3fff700 1 -- 172.21.15.162:0/245779021 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb3b0055330 msgr2=0x7fb3b00577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:57.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.055+0000 7fb3d3fff700 1 --2- 172.21.15.162:0/245779021 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb3b0055330 0x7fb3b00577f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7fb3c400d000 tx=0x7fb3c4009040).stop 2020-11-28T21:49:57.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.055+0000 7fb3d3fff700 1 -- 172.21.15.162:0/245779021 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3d40ac3b0 msgr2=0x7fb3d40afa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:57.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.055+0000 7fb3d3fff700 1 --2- 172.21.15.162:0/245779021 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3d40ac3b0 0x7fb3d40afa40 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7fb3c80010d0 tx=0x7fb3c8007a60).stop 2020-11-28T21:49:57.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.055+0000 7fb3d3fff700 1 -- 172.21.15.162:0/245779021 shutdown_connections 2020-11-28T21:49:57.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.055+0000 7fb3d3fff700 1 --2- 172.21.15.162:0/245779021 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb3b0055330 0x7fb3b00577f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:57.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.055+0000 7fb3d3fff700 1 --2- 172.21.15.162:0/245779021 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb3d40acca0 0x7fb3d40b5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:57.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.055+0000 7fb3d3fff700 1 --2- 172.21.15.162:0/245779021 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3d40ac3b0 0x7fb3d40afa40 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:57.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.055+0000 7fb3d3fff700 1 --2- 172.21.15.162:0/245779021 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb3d40ac790 0x7fb3d40af3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:57.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.055+0000 7fb3d3fff700 1 -- 172.21.15.162:0/245779021 >> 172.21.15.162:0/245779021 conn(0x7fb3d40b6870 msgr2=0x7fb3d40611c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:49:57.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.055+0000 7fb3d3fff700 1 -- 172.21.15.162:0/245779021 shutdown_connections 2020-11-28T21:49:57.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.055+0000 7fb3d3fff700 1 -- 172.21.15.162:0/245779021 wait complete. 2020-11-28T21:49:57.060 INFO:teuthology.orchestra.run.smithi162.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-11-28T21:49:57.071 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:49:57.086 INFO:tasks.ceph.mds.a.smithi162.stdout:starting mds.a at 2020-11-28T21:49:57.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.183+0000 7f8eac316700 1 Processor -- start 2020-11-28T21:49:57.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.183+0000 7f8eac316700 1 -- start start 2020-11-28T21:49:57.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.183+0000 7f8eac316700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8ea40aabb0 0x7f8ea40b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:57.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.183+0000 7f8eac316700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ea4065b00 con 0x7f8ea40b1c20 2020-11-28T21:49:57.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.183+0000 7f8eac316700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ea40b37c0 con 0x7f8ea40aa7d0 2020-11-28T21:49:57.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.183+0000 7f8eac316700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ea413e7f0 con 0x7f8ea40aabb0 2020-11-28T21:49:57.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.183+0000 7f8eaa8b3700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8ea40aabb0 0x7f8ea40b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:57.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.183+0000 7f8eaa0b2700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f8ea40aa7d0 0x7f8ea4140ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:46584/0 (socket says 172.21.15.162:46584) 2020-11-28T21:49:57.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.183+0000 7f8eaa0b2700 1 -- 172.21.15.162:0/2329144004 learned_addr learned my addr 172.21.15.162:0/2329144004 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:49:57.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.183+0000 7f8ea90b0700 1 -- 172.21.15.162:0/2329144004 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 444278933 0 0) 0x7f8ea40b37c0 con 0x7f8ea40aa7d0 2020-11-28T21:49:57.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.183+0000 7f8ea90b0700 1 -- 172.21.15.162:0/2329144004 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e8c003230 con 0x7f8ea40aa7d0 2020-11-28T21:49:57.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.183+0000 7f8eaa8b3700 1 -- 172.21.15.162:0/2329144004 >> v1:172.21.15.162:6789/0 conn(0x7f8ea40aa7d0 legacy=0x7f8ea4140ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:57.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.183+0000 7f8eaa8b3700 1 -- 172.21.15.162:0/2329144004 >> v1:172.21.15.118:6789/0 conn(0x7f8ea40b1c20 legacy=0x7f8ea40b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:57.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.183+0000 7f8eaa8b3700 1 -- 172.21.15.162:0/2329144004 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea4065d30 con 0x7f8ea40aabb0 2020-11-28T21:49:57.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.183+0000 7f8eaa8b3700 1 --2- 172.21.15.162:0/2329144004 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ea40aabb0 0x7f8ea40b2370 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7f8e9c0032b0 tx=0x7f8e9c0091a0).ready entity=mon.1 client_cookie=db0ab357cae1509c server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:57.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.183+0000 7f8ea90b0700 1 -- 172.21.15.162:0/2329144004 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e9c00b9a0 con 0x7f8ea40aabb0 2020-11-28T21:49:57.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.183+0000 7f8ea90b0700 1 -- 172.21.15.162:0/2329144004 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e9c00bb00 con 0x7f8ea40aabb0 2020-11-28T21:49:57.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.183+0000 7f8ea90b0700 1 -- 172.21.15.162:0/2329144004 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e9c00bc70 con 0x7f8ea40aabb0 2020-11-28T21:49:57.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.183+0000 7f8eac316700 1 -- 172.21.15.162:0/2329144004 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ea40aabb0 msgr2=0x7f8ea40b2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:57.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.183+0000 7f8eac316700 1 --2- 172.21.15.162:0/2329144004 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ea40aabb0 0x7f8ea40b2370 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7f8e9c0032b0 tx=0x7f8e9c0091a0).stop 2020-11-28T21:49:57.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.183+0000 7f8eac316700 1 -- 172.21.15.162:0/2329144004 shutdown_connections 2020-11-28T21:49:57.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.183+0000 7f8eac316700 1 --2- 172.21.15.162:0/2329144004 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ea40aabb0 0x7f8ea40b2370 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:57.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.183+0000 7f8eac316700 1 -- 172.21.15.162:0/2329144004 >> 172.21.15.162:0/2329144004 conn(0x7f8ea4062b50 msgr2=0x7f8ea4062f30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:49:57.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.187+0000 7f8eac316700 1 -- 172.21.15.162:0/2329144004 shutdown_connections 2020-11-28T21:49:57.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.187+0000 7f8eac316700 1 -- 172.21.15.162:0/2329144004 wait complete. 2020-11-28T21:49:57.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.187+0000 7f8eac316700 1 Processor -- start 2020-11-28T21:49:57.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.187+0000 7f8eac316700 1 -- start start 2020-11-28T21:49:57.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.187+0000 7f8eac316700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8ea40aa7d0 0x7f8ea40abe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:57.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.187+0000 7f8eac316700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8ea40aabb0 0x7f8ea41447a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:57.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.187+0000 7f8eac316700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ea40b1c20 0x7f8ea4060310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:57.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.187+0000 7f8eac316700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea405fd80 con 0x7f8ea40aa7d0 2020-11-28T21:49:57.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.187+0000 7f8eac316700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea40ae6d0 con 0x7f8ea40b1c20 2020-11-28T21:49:57.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.187+0000 7f8eac316700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea40adc90 con 0x7f8ea40aabb0 2020-11-28T21:49:57.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.187+0000 7f8eaa8b3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ea40b1c20 0x7f8ea4060310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:57.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.187+0000 7f8eaa8b3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ea40b1c20 0x7f8ea4060310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39750/0 (socket says 172.21.15.162:39750) 2020-11-28T21:49:57.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.187+0000 7f8eaa8b3700 1 -- 172.21.15.162:0/530363889 learned_addr learned my addr 172.21.15.162:0/530363889 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:49:57.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.187+0000 7f8eaa0b2700 1 --2- 172.21.15.162:0/530363889 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8ea40aa7d0 0x7f8ea40abe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:57.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.187+0000 7f8ea98b1700 1 --2- 172.21.15.162:0/530363889 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8ea40aabb0 0x7f8ea41447a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:57.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.187+0000 7f8eaa8b3700 1 -- 172.21.15.162:0/530363889 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8ea40aabb0 msgr2=0x7f8ea41447a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:57.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.187+0000 7f8eaa8b3700 1 --2- 172.21.15.162:0/530363889 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8ea40aabb0 0x7f8ea41447a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:57.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.187+0000 7f8eaa8b3700 1 -- 172.21.15.162:0/530363889 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8ea40aa7d0 msgr2=0x7f8ea40abe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:57.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.187+0000 7f8eaa8b3700 1 --2- 172.21.15.162:0/530363889 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8ea40aa7d0 0x7f8ea40abe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:57.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.187+0000 7f8eaa8b3700 1 -- 172.21.15.162:0/530363889 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea4144c90 con 0x7f8ea40b1c20 2020-11-28T21:49:57.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.187+0000 7f8eaa8b3700 1 --2- 172.21.15.162:0/530363889 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ea40b1c20 0x7f8ea4060310 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7f8e9c0032b0 tx=0x7f8e9c016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:57.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.191+0000 7f8e937fe700 1 -- 172.21.15.162:0/530363889 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e9c011e60 con 0x7f8ea40b1c20 2020-11-28T21:49:57.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.191+0000 7f8e937fe700 1 -- 172.21.15.162:0/530363889 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e9c01f3f0 con 0x7f8ea40b1c20 2020-11-28T21:49:57.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.191+0000 7f8eac316700 1 -- 172.21.15.162:0/530363889 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ea405f750 con 0x7f8ea40b1c20 2020-11-28T21:49:57.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.191+0000 7f8e937fe700 1 -- 172.21.15.162:0/530363889 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e9c01f750 con 0x7f8ea40b1c20 2020-11-28T21:49:57.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.191+0000 7f8eac316700 1 -- 172.21.15.162:0/530363889 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea40ab090 con 0x7f8ea40b1c20 2020-11-28T21:49:57.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.191+0000 7f8eac316700 1 -- 172.21.15.162:0/530363889 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ea40ade50 con 0x7f8ea40b1c20 2020-11-28T21:49:57.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.191+0000 7f8e937fe700 1 -- 172.21.15.162:0/530363889 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8e9c026030 con 0x7f8ea40b1c20 2020-11-28T21:49:57.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.191+0000 7f8e937fe700 1 --2- 172.21.15.162:0/530363889 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8e84055440 0x7f8e84057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:57.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.191+0000 7f8e937fe700 1 -- 172.21.15.162:0/530363889 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f8e9c06eb00 con 0x7f8ea40b1c20 2020-11-28T21:49:57.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.191+0000 7f8eaa0b2700 1 --2- 172.21.15.162:0/530363889 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8e84055440 0x7f8e84057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:57.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.191+0000 7f8eaa0b2700 1 --2- 172.21.15.162:0/530363889 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8e84055440 0x7f8e84057900 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f8ea000d000 tx=0x7f8ea0009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:57.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.199+0000 7f8e937fe700 1 -- 172.21.15.162:0/530363889 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8e9c044db0 con 0x7f8ea40b1c20 2020-11-28T21:49:57.378 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.379+0000 7f8eac316700 1 -- 172.21.15.162:0/530363889 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ea4057050 con 0x7f8ea40b1c20 2020-11-28T21:49:57.379 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.379+0000 7f8e937fe700 1 -- 172.21.15.162:0/530363889 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+2133 (secure 0 0 0) 0x7f8e9c024090 con 0x7f8ea40b1c20 2020-11-28T21:49:57.379 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:49:57.380 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":16,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4856,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/913296056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":913296056},{"type":"v1","addr":"172.21.15.118:6833","nonce":913296056}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":13},{"gid":5209,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6835/1318110356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1318110356},{"type":"v1","addr":"172.21.15.162:6835","nonce":1318110356}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14}],"filesystems":[{"mdsmap":{"epoch":16,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:48:26.782354+0000","modified":"2020-11-28T21:49:57.044220+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-28T21:49:57.381 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.383+0000 7f8eac316700 1 -- 172.21.15.162:0/530363889 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8e84055440 msgr2=0x7f8e84057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:57.381 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.383+0000 7f8eac316700 1 --2- 172.21.15.162:0/530363889 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8e84055440 0x7f8e84057900 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f8ea000d000 tx=0x7f8ea0009040).stop 2020-11-28T21:49:57.381 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.383+0000 7f8eac316700 1 -- 172.21.15.162:0/530363889 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ea40b1c20 msgr2=0x7f8ea4060310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:57.382 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.383+0000 7f8eac316700 1 --2- 172.21.15.162:0/530363889 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ea40b1c20 0x7f8ea4060310 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7f8e9c0032b0 tx=0x7f8e9c016000).stop 2020-11-28T21:49:57.382 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.383+0000 7f8eac316700 1 -- 172.21.15.162:0/530363889 shutdown_connections 2020-11-28T21:49:57.382 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.383+0000 7f8eac316700 1 --2- 172.21.15.162:0/530363889 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8e84055440 0x7f8e84057900 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:57.383 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.383+0000 7f8eac316700 1 --2- 172.21.15.162:0/530363889 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ea40b1c20 0x7f8ea4060310 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:57.383 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.383+0000 7f8eac316700 1 --2- 172.21.15.162:0/530363889 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8ea40aa7d0 0x7f8ea40abe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:57.383 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.383+0000 7f8eac316700 1 --2- 172.21.15.162:0/530363889 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8ea40aabb0 0x7f8ea41447a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:57.383 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.383+0000 7f8eac316700 1 -- 172.21.15.162:0/530363889 >> 172.21.15.162:0/530363889 conn(0x7f8ea4062b50 msgr2=0x7f8ea40b6660 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:49:57.383 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.383+0000 7f8eac316700 1 -- 172.21.15.162:0/530363889 shutdown_connections 2020-11-28T21:49:57.384 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.383+0000 7f8eac316700 1 -- 172.21.15.162:0/530363889 wait complete. 2020-11-28T21:49:57.384 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 16 2020-11-28T21:49:57.393 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2020-11-28T21:49:57.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.555+0000 7f8ff6914700 1 Processor -- start 2020-11-28T21:49:57.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.555+0000 7f8ff6914700 1 -- start start 2020-11-28T21:49:57.558 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.555+0000 7f8ff6914700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f8ff00b7920 0x7f8ff013f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:57.558 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.555+0000 7f8ff6914700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ff00b7d00 con 0x7f8ff00b2da0 2020-11-28T21:49:57.559 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.555+0000 7f8ff6914700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ff0143300 con 0x7f8ff00b24c0 2020-11-28T21:49:57.559 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.555+0000 7f8ff6914700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ff0143560 con 0x7f8ff00b7920 2020-11-28T21:49:57.559 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.555+0000 7f8ff6113700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f8ff00b24c0 0x7f8ff014d220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:46602/0 (socket says 172.21.15.162:46602) 2020-11-28T21:49:57.560 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.555+0000 7f8ff6113700 1 -- 172.21.15.162:0/1164272426 learned_addr learned my addr 172.21.15.162:0/1164272426 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:49:57.560 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.555+0000 7f8ff5912700 1 --2- 172.21.15.162:0/1164272426 >> v2:172.21.15.118:3300/0 conn(0x7f8ff00b7920 0x7f8ff013f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:57.560 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.555+0000 7f8ff4910700 1 -- 172.21.15.162:0/1164272426 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3985287185 0 0) 0x7f8ff0143300 con 0x7f8ff00b24c0 2020-11-28T21:49:57.560 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.555+0000 7f8ff4910700 1 -- 172.21.15.162:0/1164272426 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fd8003230 con 0x7f8ff00b24c0 2020-11-28T21:49:57.561 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.555+0000 7f8ff4910700 1 -- 172.21.15.162:0/1164272426 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3668012212 0 0) 0x7f8fd8003230 con 0x7f8ff00b24c0 2020-11-28T21:49:57.561 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.555+0000 7f8ff4910700 1 -- 172.21.15.162:0/1164272426 >> v2:172.21.15.118:3300/0 conn(0x7f8ff00b7920 msgr2=0x7f8ff013f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:49:57.561 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.555+0000 7f8ff4910700 1 --2- 172.21.15.162:0/1164272426 >> v2:172.21.15.118:3300/0 conn(0x7f8ff00b7920 0x7f8ff013f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:57.561 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.555+0000 7f8ff4910700 1 -- 172.21.15.162:0/1164272426 >> v1:172.21.15.118:6789/0 conn(0x7f8ff00b2da0 legacy=0x7f8ff00b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:57.561 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.555+0000 7f8ff4910700 1 -- 172.21.15.162:0/1164272426 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff00b1910 con 0x7f8ff00b24c0 2020-11-28T21:49:57.562 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.555+0000 7f8ff4910700 1 -- 172.21.15.162:0/1164272426 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f8fe8001a70 con 0x7f8ff00b24c0 2020-11-28T21:49:57.562 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.555+0000 7f8ff4910700 1 -- 172.21.15.162:0/1164272426 >> v1:172.21.15.162:6789/0 conn(0x7f8ff00b24c0 legacy=0x7f8ff014d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:57.562 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.555+0000 7f8ff4910700 1 --2- 172.21.15.162:0/1164272426 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fd8003720 0x7f8fd8003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:57.562 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.555+0000 7f8ff4910700 1 --2- 172.21.15.162:0/1164272426 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8fd80044d0 0x7f8fd8004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:57.563 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.555+0000 7f8ff4910700 1 --2- 172.21.15.162:0/1164272426 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fd8004e40 0x7f8fd800b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:57.563 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.555+0000 7f8ff4910700 1 -- 172.21.15.162:0/1164272426 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff0143560 con 0x7f8fd8004e40 2020-11-28T21:49:57.563 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.555+0000 7f8ff4910700 1 -- 172.21.15.162:0/1164272426 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd8004250 con 0x7f8fd8003720 2020-11-28T21:49:57.563 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.555+0000 7f8ff4910700 1 -- 172.21.15.162:0/1164272426 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd800bc10 con 0x7f8fd80044d0 2020-11-28T21:49:57.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.555+0000 7f8ff5912700 1 --2- 172.21.15.162:0/1164272426 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fd8003720 0x7f8fd8003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:57.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.555+0000 7f8ff6113700 1 --2- 172.21.15.162:0/1164272426 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fd8004e40 0x7f8fd800b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:57.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.555+0000 7f8ff5111700 1 --2- 172.21.15.162:0/1164272426 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8fd80044d0 0x7f8fd8004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:57.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.555+0000 7f8ff5912700 1 -- 172.21.15.162:0/1164272426 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8fd80044d0 msgr2=0x7f8fd8004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:49:57.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.555+0000 7f8ff5912700 1 --2- 172.21.15.162:0/1164272426 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8fd80044d0 0x7f8fd8004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:57.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.559+0000 7f8ff5912700 1 -- 172.21.15.162:0/1164272426 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fd8004e40 msgr2=0x7f8fd800b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:49:57.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.559+0000 7f8ff5912700 1 --2- 172.21.15.162:0/1164272426 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fd8004e40 0x7f8fd800b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:57.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.559+0000 7f8ff6113700 1 -- 172.21.15.162:0/1164272426 reap_dead start 2020-11-28T21:49:57.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.559+0000 7f8ff5912700 1 -- 172.21.15.162:0/1164272426 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd8003230 con 0x7f8fd8003720 2020-11-28T21:49:57.566 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.559+0000 7f8ff5111700 1 --2- 172.21.15.162:0/1164272426 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8fd80044d0 0x7f8fd8004950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:49:57.566 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.559+0000 7f8ff5912700 1 --2- 172.21.15.162:0/1164272426 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fd8003720 0x7f8fd8003b40 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7f8fe400c000 tx=0x7f8fe4008040).ready entity=mon.1 client_cookie=1d858323dae2a3 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:57.566 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.559+0000 7f8ff4910700 1 -- 172.21.15.162:0/1164272426 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fe400e970 con 0x7f8fd8003720 2020-11-28T21:49:57.566 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.559+0000 7f8ff4910700 1 -- 172.21.15.162:0/1164272426 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe400ead0 con 0x7f8fd8003720 2020-11-28T21:49:57.567 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.559+0000 7f8ff4910700 1 -- 172.21.15.162:0/1164272426 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fe400ee30 con 0x7f8fd8003720 2020-11-28T21:49:57.567 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.559+0000 7f8ff6914700 1 -- 172.21.15.162:0/1164272426 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fd8003720 msgr2=0x7f8fd8003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:57.568 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.559+0000 7f8ff6914700 1 --2- 172.21.15.162:0/1164272426 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fd8003720 0x7f8fd8003b40 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7f8fe400c000 tx=0x7f8fe4008040).stop 2020-11-28T21:49:57.568 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.559+0000 7f8ff6914700 1 -- 172.21.15.162:0/1164272426 shutdown_connections 2020-11-28T21:49:57.568 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.559+0000 7f8ff6914700 1 --2- 172.21.15.162:0/1164272426 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fd8003720 0x7f8fd8003b40 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:57.568 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.559+0000 7f8ff6914700 1 -- 172.21.15.162:0/1164272426 >> 172.21.15.162:0/1164272426 conn(0x7f8ff005c440 msgr2=0x7f8ff00b6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:49:57.568 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.559+0000 7f8ff6914700 1 -- 172.21.15.162:0/1164272426 shutdown_connections 2020-11-28T21:49:57.569 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.559+0000 7f8ff6914700 1 -- 172.21.15.162:0/1164272426 wait complete. 2020-11-28T21:49:57.572 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.559+0000 7f8ff6914700 1 Processor -- start 2020-11-28T21:49:57.572 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.559+0000 7f8ff6914700 1 -- start start 2020-11-28T21:49:57.572 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.559+0000 7f8ff6914700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fd8003720 0x7f8ff0146620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:57.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.559+0000 7f8ff6914700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ff0145010 0x7f8ff00ae790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:57.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.559+0000 7f8ff6914700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8ff0151eb0 0x7f8ff0152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:57.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.563+0000 7f8ff6914700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff0151a80 con 0x7f8fd8003720 2020-11-28T21:49:57.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.563+0000 7f8ff6914700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff0146d60 con 0x7f8ff0145010 2020-11-28T21:49:57.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.563+0000 7f8ff6914700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff0149820 con 0x7f8ff0151eb0 2020-11-28T21:49:57.574 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.563+0000 7f8ff5111700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ff0145010 0x7f8ff00ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:57.574 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.563+0000 7f8ff5111700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ff0145010 0x7f8ff00ae790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39768/0 (socket says 172.21.15.162:39768) 2020-11-28T21:49:57.574 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.563+0000 7f8ff5111700 1 -- 172.21.15.162:0/2751540154 learned_addr learned my addr 172.21.15.162:0/2751540154 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:49:57.574 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.563+0000 7f8ff5912700 1 --2- 172.21.15.162:0/2751540154 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fd8003720 0x7f8ff0146620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:57.575 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.563+0000 7f8ff6113700 1 --2- 172.21.15.162:0/2751540154 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8ff0151eb0 0x7f8ff0152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:57.575 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.563+0000 7f8ff5111700 1 -- 172.21.15.162:0/2751540154 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8ff0151eb0 msgr2=0x7f8ff0152310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:57.575 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.563+0000 7f8ff5111700 1 --2- 172.21.15.162:0/2751540154 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8ff0151eb0 0x7f8ff0152310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:57.575 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.563+0000 7f8ff5111700 1 -- 172.21.15.162:0/2751540154 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fd8003720 msgr2=0x7f8ff0146620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:57.576 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.563+0000 7f8ff5111700 1 --2- 172.21.15.162:0/2751540154 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fd8003720 0x7f8ff0146620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:57.576 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.563+0000 7f8ff5111700 1 -- 172.21.15.162:0/2751540154 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff0149ac0 con 0x7f8ff0145010 2020-11-28T21:49:57.576 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.563+0000 7f8ff5912700 1 --2- 172.21.15.162:0/2751540154 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fd8003720 0x7f8ff0146620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:49:57.576 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.563+0000 7f8ff5111700 1 --2- 172.21.15.162:0/2751540154 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ff0145010 0x7f8ff00ae790 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f8fec00b000 tx=0x7f8fec009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:57.577 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.563+0000 7f8fe2ffd700 1 -- 172.21.15.162:0/2751540154 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fec010710 con 0x7f8ff0145010 2020-11-28T21:49:57.577 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.563+0000 7f8fe2ffd700 1 -- 172.21.15.162:0/2751540154 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fec010870 con 0x7f8ff0145010 2020-11-28T21:49:57.578 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.563+0000 7f8ff6914700 1 -- 172.21.15.162:0/2751540154 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ff0149d60 con 0x7f8ff0145010 2020-11-28T21:49:57.578 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.563+0000 7f8fe2ffd700 1 -- 172.21.15.162:0/2751540154 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fec010bd0 con 0x7f8ff0145010 2020-11-28T21:49:57.578 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.563+0000 7f8ff6914700 1 -- 172.21.15.162:0/2751540154 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff0143cb0 con 0x7f8ff0145010 2020-11-28T21:49:57.578 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.563+0000 7f8ff6914700 1 -- 172.21.15.162:0/2751540154 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ff0151bc0 con 0x7f8ff0145010 2020-11-28T21:49:57.579 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.563+0000 7f8fe2ffd700 1 -- 172.21.15.162:0/2751540154 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8fec010d30 con 0x7f8ff0145010 2020-11-28T21:49:57.579 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.563+0000 7f8fe2ffd700 1 --2- 172.21.15.162:0/2751540154 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8fd0055440 0x7f8fd0057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:57.579 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.563+0000 7f8fe2ffd700 1 -- 172.21.15.162:0/2751540154 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f8fec06e440 con 0x7f8ff0145010 2020-11-28T21:49:57.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.563+0000 7f8ff5912700 1 --2- 172.21.15.162:0/2751540154 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8fd0055440 0x7f8fd0057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:57.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.563+0000 7f8ff5912700 1 --2- 172.21.15.162:0/2751540154 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8fd0055440 0x7f8fd0057900 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f8fe4008000 tx=0x7f8fe401a000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:57.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.571+0000 7f8fe2ffd700 1 -- 172.21.15.162:0/2751540154 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8fec044650 con 0x7f8ff0145010 2020-11-28T21:49:57.754 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:57.755+0000 7f8ff6914700 1 -- 172.21.15.162:0/2751540154 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8ff0057050 con 0x7f8ff0145010 2020-11-28T21:49:58.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.051+0000 7f8fe2ffd700 1 -- 172.21.15.162:0/2751540154 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8fec047c70 con 0x7f8ff0145010 2020-11-28T21:49:58.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.055+0000 7f8ff6914700 1 -- 172.21.15.162:0/2751540154 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8fd0055440 msgr2=0x7f8fd0057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:58.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.055+0000 7f8ff6914700 1 --2- 172.21.15.162:0/2751540154 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8fd0055440 0x7f8fd0057900 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f8fe4008000 tx=0x7f8fe401a000).stop 2020-11-28T21:49:58.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.055+0000 7f8ff6914700 1 -- 172.21.15.162:0/2751540154 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ff0145010 msgr2=0x7f8ff00ae790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:58.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.055+0000 7f8ff6914700 1 --2- 172.21.15.162:0/2751540154 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ff0145010 0x7f8ff00ae790 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f8fec00b000 tx=0x7f8fec009040).stop 2020-11-28T21:49:58.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.055+0000 7f8ff6914700 1 -- 172.21.15.162:0/2751540154 shutdown_connections 2020-11-28T21:49:58.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.055+0000 7f8ff6914700 1 --2- 172.21.15.162:0/2751540154 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8fd0055440 0x7f8fd0057900 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:58.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.055+0000 7f8ff6914700 1 --2- 172.21.15.162:0/2751540154 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8ff0151eb0 0x7f8ff0152310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:58.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.055+0000 7f8ff6914700 1 --2- 172.21.15.162:0/2751540154 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fd8003720 0x7f8ff0146620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:58.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.055+0000 7f8ff6914700 1 --2- 172.21.15.162:0/2751540154 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ff0145010 0x7f8ff00ae790 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:58.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.055+0000 7f8ff6914700 1 -- 172.21.15.162:0/2751540154 >> 172.21.15.162:0/2751540154 conn(0x7f8ff005c440 msgr2=0x7f8ff01411c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:49:58.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.055+0000 7f8ff6914700 1 -- 172.21.15.162:0/2751540154 shutdown_connections 2020-11-28T21:49:58.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.055+0000 7f8ff6914700 1 -- 172.21.15.162:0/2751540154 wait complete. 2020-11-28T21:49:58.071 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-11-28T21:49:58.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.195+0000 7f7d77fff700 1 Processor -- start 2020-11-28T21:49:58.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.199+0000 7f7d77fff700 1 -- start start 2020-11-28T21:49:58.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.199+0000 7f7d77fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7d781404c0 0x7f7d78060450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:58.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.199+0000 7f7d77fff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f7d78060940 0x7f7d7814d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:58.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.199+0000 7f7d77fff700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d780aff20 con 0x7f7d780b21f0 2020-11-28T21:49:58.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.199+0000 7f7d77fff700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d780af240 con 0x7f7d78060940 2020-11-28T21:49:58.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.199+0000 7f7d77fff700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d780b25d0 con 0x7f7d781404c0 2020-11-28T21:49:58.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.199+0000 7f7d767fc700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7d781404c0 0x7f7d78060450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:58.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.199+0000 7f7d767fc700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7d781404c0 0x7f7d78060450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39776/0 (socket says 172.21.15.162:39776) 2020-11-28T21:49:58.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.199+0000 7f7d76ffd700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f7d780b21f0 0x7f7d7813fdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:46620/0 (socket says 172.21.15.162:46620) 2020-11-28T21:49:58.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.199+0000 7f7d767fc700 1 -- 172.21.15.162:0/265452504 learned_addr learned my addr 172.21.15.162:0/265452504 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:49:58.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.199+0000 7f7d777fe700 1 --2- 172.21.15.162:0/265452504 >> v2:172.21.15.118:3300/0 conn(0x7f7d78060940 0x7f7d7814d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:58.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.199+0000 7f7d767fc700 1 -- 172.21.15.162:0/265452504 >> v2:172.21.15.118:3300/0 conn(0x7f7d78060940 msgr2=0x7f7d7814d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:49:58.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.199+0000 7f7d767fc700 1 --2- 172.21.15.162:0/265452504 >> v2:172.21.15.118:3300/0 conn(0x7f7d78060940 0x7f7d7814d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:58.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.199+0000 7f7d767fc700 1 -- 172.21.15.162:0/265452504 >> v1:172.21.15.162:6789/0 conn(0x7f7d780b21f0 legacy=0x7f7d7813fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:58.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.199+0000 7f7d767fc700 1 -- 172.21.15.162:0/265452504 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d7805fff0 con 0x7f7d781404c0 2020-11-28T21:49:58.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.199+0000 7f7d767fc700 1 --2- 172.21.15.162:0/265452504 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d781404c0 0x7f7d78060450 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f7d68001450 tx=0x7f7d60006e80).ready entity=mon.1 client_cookie=b4f6aaaa08a519f4 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:58.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.199+0000 7f7d75ffb700 1 -- 172.21.15.162:0/265452504 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d600096b0 con 0x7f7d781404c0 2020-11-28T21:49:58.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.199+0000 7f7d75ffb700 1 -- 172.21.15.162:0/265452504 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d60009810 con 0x7f7d781404c0 2020-11-28T21:49:58.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.199+0000 7f7d75ffb700 1 -- 172.21.15.162:0/265452504 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d60009980 con 0x7f7d781404c0 2020-11-28T21:49:58.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.199+0000 7f7d77fff700 1 -- 172.21.15.162:0/265452504 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d781404c0 msgr2=0x7f7d78060450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:58.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.199+0000 7f7d77fff700 1 --2- 172.21.15.162:0/265452504 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d781404c0 0x7f7d78060450 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f7d68001450 tx=0x7f7d60006e80).stop 2020-11-28T21:49:58.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.199+0000 7f7d77fff700 1 -- 172.21.15.162:0/265452504 shutdown_connections 2020-11-28T21:49:58.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.199+0000 7f7d77fff700 1 --2- 172.21.15.162:0/265452504 >> v2:172.21.15.118:3300/0 conn(0x7f7d78060940 0x7f7d7814d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:58.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.199+0000 7f7d77fff700 1 --2- 172.21.15.162:0/265452504 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d781404c0 0x7f7d78060450 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:58.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.199+0000 7f7d77fff700 1 -- 172.21.15.162:0/265452504 >> 172.21.15.162:0/265452504 conn(0x7f7d780b57d0 msgr2=0x7f7d780b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:49:58.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.199+0000 7f7d77fff700 1 -- 172.21.15.162:0/265452504 shutdown_connections 2020-11-28T21:49:58.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.199+0000 7f7d77fff700 1 -- 172.21.15.162:0/265452504 wait complete. 2020-11-28T21:49:58.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.203+0000 7f7d77fff700 1 Processor -- start 2020-11-28T21:49:58.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.203+0000 7f7d77fff700 1 -- start start 2020-11-28T21:49:58.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.203+0000 7f7d77fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7d78060940 0x7f7d7813a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:58.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.203+0000 7f7d77fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d780b21f0 0x7f7d78138e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:58.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.203+0000 7f7d77fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7d781404c0 0x7f7d78139360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:58.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.203+0000 7f7d77fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d781440d0 con 0x7f7d78060940 2020-11-28T21:49:58.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.203+0000 7f7d77fff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d781456b0 con 0x7f7d780b21f0 2020-11-28T21:49:58.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.203+0000 7f7d77fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d78140d40 con 0x7f7d781404c0 2020-11-28T21:49:58.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.203+0000 7f7d767fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d780b21f0 0x7f7d78138e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:58.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.203+0000 7f7d767fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d780b21f0 0x7f7d78138e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39784/0 (socket says 172.21.15.162:39784) 2020-11-28T21:49:58.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.203+0000 7f7d767fc700 1 -- 172.21.15.162:0/25663390 learned_addr learned my addr 172.21.15.162:0/25663390 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:49:58.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.203+0000 7f7d76ffd700 1 --2- 172.21.15.162:0/25663390 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7d78060940 0x7f7d7813a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:58.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.203+0000 7f7d777fe700 1 --2- 172.21.15.162:0/25663390 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7d781404c0 0x7f7d78139360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:58.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.203+0000 7f7d767fc700 1 -- 172.21.15.162:0/25663390 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7d781404c0 msgr2=0x7f7d78139360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:58.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.203+0000 7f7d767fc700 1 --2- 172.21.15.162:0/25663390 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7d781404c0 0x7f7d78139360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:58.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.203+0000 7f7d767fc700 1 -- 172.21.15.162:0/25663390 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7d78060940 msgr2=0x7f7d7813a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:58.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.203+0000 7f7d767fc700 1 --2- 172.21.15.162:0/25663390 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7d78060940 0x7f7d7813a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:58.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.203+0000 7f7d767fc700 1 -- 172.21.15.162:0/25663390 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d78140fe0 con 0x7f7d780b21f0 2020-11-28T21:49:58.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.203+0000 7f7d767fc700 1 --2- 172.21.15.162:0/25663390 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d780b21f0 0x7f7d78138e70 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f7d68001450 tx=0x7f7d60016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:58.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.207+0000 7f7d6ffff700 1 -- 172.21.15.162:0/25663390 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d60009ea0 con 0x7f7d780b21f0 2020-11-28T21:49:58.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.207+0000 7f7d6ffff700 1 -- 172.21.15.162:0/25663390 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d60011d70 con 0x7f7d780b21f0 2020-11-28T21:49:58.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.207+0000 7f7d77fff700 1 -- 172.21.15.162:0/25663390 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d78141280 con 0x7f7d780b21f0 2020-11-28T21:49:58.212 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.207+0000 7f7d6ffff700 1 -- 172.21.15.162:0/25663390 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d600205f0 con 0x7f7d780b21f0 2020-11-28T21:49:58.212 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.207+0000 7f7d77fff700 1 -- 172.21.15.162:0/25663390 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d78139850 con 0x7f7d780b21f0 2020-11-28T21:49:58.212 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.207+0000 7f7d77fff700 1 -- 172.21.15.162:0/25663390 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d78057050 con 0x7f7d780b21f0 2020-11-28T21:49:58.212 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.207+0000 7f7d6ffff700 1 -- 172.21.15.162:0/25663390 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7d60027030 con 0x7f7d780b21f0 2020-11-28T21:49:58.213 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.207+0000 7f7d6ffff700 1 --2- 172.21.15.162:0/25663390 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7d5c055330 0x7f7d5c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:58.213 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.207+0000 7f7d6ffff700 1 -- 172.21.15.162:0/25663390 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f7d6006e980 con 0x7f7d780b21f0 2020-11-28T21:49:58.213 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.207+0000 7f7d76ffd700 1 --2- 172.21.15.162:0/25663390 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7d5c055330 0x7f7d5c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:58.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.211+0000 7f7d76ffd700 1 --2- 172.21.15.162:0/25663390 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7d5c055330 0x7f7d5c0577f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f7d6800d000 tx=0x7f7d68007e90).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:58.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.215+0000 7f7d6ffff700 1 -- 172.21.15.162:0/25663390 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7d60044c30 con 0x7f7d780b21f0 2020-11-28T21:49:58.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:58.351+0000 7f7d77fff700 1 -- 172.21.15.162:0/25663390 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7d7805d250 con 0x7f7d780b21f0 2020-11-28T21:49:59.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.131+0000 7f7d6ffff700 1 -- 172.21.15.162:0/25663390 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v46) v1 ==== 185+0+0 (secure 0 0 0) 0x7f7d600150e0 con 0x7f7d780b21f0 2020-11-28T21:49:59.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.135+0000 7f7d77fff700 1 -- 172.21.15.162:0/25663390 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7d5c055330 msgr2=0x7f7d5c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:59.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.135+0000 7f7d77fff700 1 --2- 172.21.15.162:0/25663390 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7d5c055330 0x7f7d5c0577f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f7d6800d000 tx=0x7f7d68007e90).stop 2020-11-28T21:49:59.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.135+0000 7f7d77fff700 1 -- 172.21.15.162:0/25663390 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d780b21f0 msgr2=0x7f7d78138e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:59.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.135+0000 7f7d77fff700 1 --2- 172.21.15.162:0/25663390 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d780b21f0 0x7f7d78138e70 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f7d68001450 tx=0x7f7d60016000).stop 2020-11-28T21:49:59.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.135+0000 7f7d77fff700 1 -- 172.21.15.162:0/25663390 shutdown_connections 2020-11-28T21:49:59.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.135+0000 7f7d77fff700 1 --2- 172.21.15.162:0/25663390 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7d5c055330 0x7f7d5c0577f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:59.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.135+0000 7f7d77fff700 1 --2- 172.21.15.162:0/25663390 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7d781404c0 0x7f7d78139360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:59.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.135+0000 7f7d77fff700 1 --2- 172.21.15.162:0/25663390 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7d78060940 0x7f7d7813a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:59.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.135+0000 7f7d77fff700 1 --2- 172.21.15.162:0/25663390 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d780b21f0 0x7f7d78138e70 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:59.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.135+0000 7f7d77fff700 1 -- 172.21.15.162:0/25663390 >> 172.21.15.162:0/25663390 conn(0x7f7d780b57d0 msgr2=0x7f7d780abd70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:49:59.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.135+0000 7f7d77fff700 1 -- 172.21.15.162:0/25663390 shutdown_connections 2020-11-28T21:49:59.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.135+0000 7f7d77fff700 1 -- 172.21.15.162:0/25663390 wait complete. 2020-11-28T21:49:59.136 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_metadata' removed 2020-11-28T21:49:59.149 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-11-28T21:49:59.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.271+0000 7ff6ee726700 1 Processor -- start 2020-11-28T21:49:59.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.271+0000 7ff6ee726700 1 -- start start 2020-11-28T21:49:59.274 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.275+0000 7ff6ee726700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6e80b2d00 0x7ff6e805c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:59.274 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.275+0000 7ff6ee726700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff6e80b3ab0 0x7ff6e80b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:59.274 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.275+0000 7ff6ee726700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff6e8142ad0 0x7ff6e8142f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:59.275 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.275+0000 7ff6ee726700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6e80b0e20 con 0x7ff6e8142ad0 2020-11-28T21:49:59.275 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.275+0000 7ff6ee726700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6e805baf0 con 0x7ff6e80b3ab0 2020-11-28T21:49:59.275 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.275+0000 7ff6ee726700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e805bd90 con 0x7ff6e80b2d00 2020-11-28T21:49:59.275 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.275+0000 7ff6ecf23700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff6e80b3ab0 0x7ff6e80b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:59.275 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.275+0000 7ff6ecf23700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff6e80b3ab0 0x7ff6e80b30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39792/0 (socket says 172.21.15.162:39792) 2020-11-28T21:49:59.276 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.275+0000 7ff6ecf23700 1 -- 172.21.15.162:0/2352979725 learned_addr learned my addr 172.21.15.162:0/2352979725 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:49:59.276 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.275+0000 7ff6edf25700 1 --2- 172.21.15.162:0/2352979725 >> v2:172.21.15.118:3300/0 conn(0x7ff6e8142ad0 0x7ff6e8142f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:59.276 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.275+0000 7ff6ed724700 1 --2- 172.21.15.162:0/2352979725 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6e80b2d00 0x7ff6e805c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:59.276 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.275+0000 7ff6ecf23700 1 -- 172.21.15.162:0/2352979725 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6e80b2d00 msgr2=0x7ff6e805c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:49:59.276 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.275+0000 7ff6ecf23700 1 --2- 172.21.15.162:0/2352979725 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6e80b2d00 0x7ff6e805c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:59.277 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.275+0000 7ff6ecf23700 1 -- 172.21.15.162:0/2352979725 >> v2:172.21.15.118:3300/0 conn(0x7ff6e8142ad0 msgr2=0x7ff6e8142f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:49:59.277 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.275+0000 7ff6ecf23700 1 --2- 172.21.15.162:0/2352979725 >> v2:172.21.15.118:3300/0 conn(0x7ff6e8142ad0 0x7ff6e8142f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:59.277 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.275+0000 7ff6ecf23700 1 -- 172.21.15.162:0/2352979725 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e805c030 con 0x7ff6e80b3ab0 2020-11-28T21:49:59.277 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.275+0000 7ff6ed724700 1 --2- 172.21.15.162:0/2352979725 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6e80b2d00 0x7ff6e805c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:49:59.277 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.275+0000 7ff6ecf23700 1 --2- 172.21.15.162:0/2352979725 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6e80b3ab0 0x7ff6e80b30e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7ff6e4002f30 tx=0x7ff6e4006e80).ready entity=mon.1 client_cookie=c69ac37f9aca1d90 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:59.278 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.275+0000 7ff6d3fff700 1 -- 172.21.15.162:0/2352979725 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6e40097f0 con 0x7ff6e80b3ab0 2020-11-28T21:49:59.278 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.275+0000 7ff6d3fff700 1 -- 172.21.15.162:0/2352979725 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e4009950 con 0x7ff6e80b3ab0 2020-11-28T21:49:59.278 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.275+0000 7ff6d3fff700 1 -- 172.21.15.162:0/2352979725 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6e4009cb0 con 0x7ff6e80b3ab0 2020-11-28T21:49:59.279 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.275+0000 7ff6ee726700 1 -- 172.21.15.162:0/2352979725 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6e80b3ab0 msgr2=0x7ff6e80b30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:59.279 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.275+0000 7ff6ee726700 1 --2- 172.21.15.162:0/2352979725 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6e80b3ab0 0x7ff6e80b30e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7ff6e4002f30 tx=0x7ff6e4006e80).stop 2020-11-28T21:49:59.281 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.275+0000 7ff6ee726700 1 -- 172.21.15.162:0/2352979725 shutdown_connections 2020-11-28T21:49:59.281 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.275+0000 7ff6ee726700 1 --2- 172.21.15.162:0/2352979725 >> v2:172.21.15.118:3300/0 conn(0x7ff6e8142ad0 0x7ff6e8142f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:59.282 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.275+0000 7ff6ee726700 1 --2- 172.21.15.162:0/2352979725 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6e80b2d00 0x7ff6e805c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:59.282 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.275+0000 7ff6ee726700 1 --2- 172.21.15.162:0/2352979725 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6e80b3ab0 0x7ff6e80b30e0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:59.282 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.275+0000 7ff6ee726700 1 -- 172.21.15.162:0/2352979725 >> 172.21.15.162:0/2352979725 conn(0x7ff6e80604c0 msgr2=0x7ff6e80608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:49:59.282 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.275+0000 7ff6ee726700 1 -- 172.21.15.162:0/2352979725 shutdown_connections 2020-11-28T21:49:59.282 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.275+0000 7ff6ee726700 1 -- 172.21.15.162:0/2352979725 wait complete. 2020-11-28T21:49:59.283 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.279+0000 7ff6ee726700 1 Processor -- start 2020-11-28T21:49:59.283 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.279+0000 7ff6ee726700 1 -- start start 2020-11-28T21:49:59.283 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.279+0000 7ff6ee726700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff6e80b2d00 0x7ff6e8151840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:59.283 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.279+0000 7ff6ee726700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6e80b3ab0 0x7ff6e80b99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:59.284 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.279+0000 7ff6ee726700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6e8142ad0 0x7ff6e8149530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:59.284 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.279+0000 7ff6ee726700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e80b8d00 con 0x7ff6e80b2d00 2020-11-28T21:49:59.284 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.279+0000 7ff6ee726700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e80b8fa0 con 0x7ff6e80b3ab0 2020-11-28T21:49:59.284 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.279+0000 7ff6ee726700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e80b9240 con 0x7ff6e8142ad0 2020-11-28T21:49:59.284 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.279+0000 7ff6ecf23700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6e80b3ab0 0x7ff6e80b99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:59.285 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.279+0000 7ff6ecf23700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6e80b3ab0 0x7ff6e80b99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39796/0 (socket says 172.21.15.162:39796) 2020-11-28T21:49:59.285 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.279+0000 7ff6ecf23700 1 -- 172.21.15.162:0/3636492442 learned_addr learned my addr 172.21.15.162:0/3636492442 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:49:59.285 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.279+0000 7ff6edf25700 1 --2- 172.21.15.162:0/3636492442 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6e8142ad0 0x7ff6e8149530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:59.285 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.279+0000 7ff6ed724700 1 --2- 172.21.15.162:0/3636492442 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff6e80b2d00 0x7ff6e8151840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:59.285 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.279+0000 7ff6edf25700 1 -- 172.21.15.162:0/3636492442 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6e80b3ab0 msgr2=0x7ff6e80b99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:59.286 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.279+0000 7ff6edf25700 1 --2- 172.21.15.162:0/3636492442 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6e80b3ab0 0x7ff6e80b99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:59.286 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.279+0000 7ff6edf25700 1 -- 172.21.15.162:0/3636492442 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff6e80b2d00 msgr2=0x7ff6e8151840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:49:59.286 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.279+0000 7ff6edf25700 1 --2- 172.21.15.162:0/3636492442 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff6e80b2d00 0x7ff6e8151840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:49:59.286 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.279+0000 7ff6edf25700 1 -- 172.21.15.162:0/3636492442 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e80b8980 con 0x7ff6e8142ad0 2020-11-28T21:49:59.286 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.279+0000 7ff6ed724700 1 --2- 172.21.15.162:0/3636492442 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff6e80b2d00 0x7ff6e8151840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:49:59.287 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.279+0000 7ff6ecf23700 1 --2- 172.21.15.162:0/3636492442 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6e80b3ab0 0x7ff6e80b99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:49:59.287 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.279+0000 7ff6edf25700 1 --2- 172.21.15.162:0/3636492442 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6e8142ad0 0x7ff6e8149530 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7ff6e000c000 tx=0x7ff6e00091a0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:59.287 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.283+0000 7ff6daffd700 1 -- 172.21.15.162:0/3636492442 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6e000c8f0 con 0x7ff6e8142ad0 2020-11-28T21:49:59.287 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.283+0000 7ff6daffd700 1 -- 172.21.15.162:0/3636492442 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e000ca50 con 0x7ff6e8142ad0 2020-11-28T21:49:59.287 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.283+0000 7ff6ee726700 1 -- 172.21.15.162:0/3636492442 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6e80b7570 con 0x7ff6e8142ad0 2020-11-28T21:49:59.288 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.283+0000 7ff6daffd700 1 -- 172.21.15.162:0/3636492442 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6e000cdb0 con 0x7ff6e8142ad0 2020-11-28T21:49:59.288 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.283+0000 7ff6ee726700 1 -- 172.21.15.162:0/3636492442 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6e80b70e0 con 0x7ff6e8142ad0 2020-11-28T21:49:59.289 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.283+0000 7ff6ee726700 1 -- 172.21.15.162:0/3636492442 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6e8149d60 con 0x7ff6e8142ad0 2020-11-28T21:49:59.289 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.283+0000 7ff6daffd700 1 -- 172.21.15.162:0/3636492442 <== mon.2 v2:172.21.15.118:3301/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff6e0022070 con 0x7ff6e8142ad0 2020-11-28T21:49:59.289 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.283+0000 7ff6daffd700 1 --2- 172.21.15.162:0/3636492442 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff6c40553f0 0x7ff6c40578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:49:59.289 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.283+0000 7ff6ed724700 1 --2- 172.21.15.162:0/3636492442 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff6c40553f0 0x7ff6c40578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:49:59.290 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.283+0000 7ff6daffd700 1 -- 172.21.15.162:0/3636492442 <== mon.2 v2:172.21.15.118:3301/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5721+0+0 (secure 0 0 0) 0x7ff6e006e2d0 con 0x7ff6e8142ad0 2020-11-28T21:49:59.290 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.287+0000 7ff6ed724700 1 --2- 172.21.15.162:0/3636492442 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff6c40553f0 0x7ff6c40578b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7ff6dc00c000 tx=0x7ff6dc008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:49:59.291 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.291+0000 7ff6daffd700 1 -- 172.21.15.162:0/3636492442 <== mon.2 v2:172.21.15.118:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff6e8149d60 con 0x7ff6e8142ad0 2020-11-28T21:49:59.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:49:59.443+0000 7ff6ee726700 1 -- 172.21.15.162:0/3636492442 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff6e8149d60 con 0x7ff6e8142ad0 2020-11-28T21:50:00.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.139+0000 7ff6daffd700 1 -- 172.21.15.162:0/3636492442 <== mon.2 v2:172.21.15.118:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v47) v1 ==== 173+0+0 (secure 0 0 0) 0x7ff6e8149d60 con 0x7ff6e8142ad0 2020-11-28T21:50:00.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.139+0000 7ff6ee726700 1 -- 172.21.15.162:0/3636492442 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff6c40553f0 msgr2=0x7ff6c40578b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:00.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.139+0000 7ff6ee726700 1 --2- 172.21.15.162:0/3636492442 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff6c40553f0 0x7ff6c40578b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7ff6dc00c000 tx=0x7ff6dc008040).stop 2020-11-28T21:50:00.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.139+0000 7ff6ee726700 1 -- 172.21.15.162:0/3636492442 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6e8142ad0 msgr2=0x7ff6e8149530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:00.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.139+0000 7ff6ee726700 1 --2- 172.21.15.162:0/3636492442 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6e8142ad0 0x7ff6e8149530 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7ff6e000c000 tx=0x7ff6e00091a0).stop 2020-11-28T21:50:00.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.143+0000 7ff6ee726700 1 -- 172.21.15.162:0/3636492442 shutdown_connections 2020-11-28T21:50:00.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.143+0000 7ff6ee726700 1 --2- 172.21.15.162:0/3636492442 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff6c40553f0 0x7ff6c40578b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:00.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.143+0000 7ff6ee726700 1 --2- 172.21.15.162:0/3636492442 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6e8142ad0 0x7ff6e8149530 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:00.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.143+0000 7ff6ee726700 1 --2- 172.21.15.162:0/3636492442 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff6e80b2d00 0x7ff6e8151840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:00.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.143+0000 7ff6ee726700 1 --2- 172.21.15.162:0/3636492442 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6e80b3ab0 0x7ff6e80b99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:00.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.143+0000 7ff6ee726700 1 -- 172.21.15.162:0/3636492442 >> 172.21.15.162:0/3636492442 conn(0x7ff6e80604c0 msgr2=0x7ff6e80608a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:00.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.143+0000 7ff6ee726700 1 -- 172.21.15.162:0/3636492442 shutdown_connections 2020-11-28T21:50:00.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.143+0000 7ff6ee726700 1 -- 172.21.15.162:0/3636492442 wait complete. 2020-11-28T21:50:00.143 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_data' removed 2020-11-28T21:50:00.158 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-28T21:50:00.159 INFO:tasks.ceph.mds.a:Stopping old one... 2020-11-28T21:50:00.159 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-11-28T21:50:00.159 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T21:50:00.160 INFO:tasks.ceph.mds.c:Restarting daemon 2020-11-28T21:50:00.160 INFO:tasks.ceph.mds.c:Stopping old one... 2020-11-28T21:50:00.160 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2020-11-28T21:50:00.160 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T21:50:00.161 INFO:tasks.ceph.mds.b:Restarting daemon 2020-11-28T21:50:00.161 INFO:tasks.ceph.mds.b:Stopping old one... 2020-11-28T21:50:00.161 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-11-28T21:50:00.162 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T21:50:00.170 INFO:tasks.ceph.mds.c:Stopped 2020-11-28T21:50:00.171 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2020-11-28T21:50:00.171 INFO:tasks.ceph.mds.a:Stopped 2020-11-28T21:50:00.172 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-28T21:50:00.173 INFO:tasks.ceph.mds.b:Stopped 2020-11-28T21:50:00.173 INFO:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-11-28T21:50:00.213 INFO:tasks.ceph.mds.c:Started 2020-11-28T21:50:00.214 DEBUG:teuthology.parallel:result is None 2020-11-28T21:50:00.215 INFO:tasks.ceph.mds.a:Started 2020-11-28T21:50:00.215 DEBUG:teuthology.parallel:result is None 2020-11-28T21:50:00.217 INFO:tasks.ceph.mds.b:Started 2020-11-28T21:50:00.218 DEBUG:teuthology.parallel:result is None 2020-11-28T21:50:00.218 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist clear 2020-11-28T21:50:00.282 INFO:tasks.ceph.mds.a.smithi162.stdout:starting mds.a at 2020-11-28T21:50:00.295 INFO:tasks.ceph.mds.b.smithi118.stdout:starting mds.b at 2020-11-28T21:50:00.298 INFO:tasks.ceph.mds.c.smithi162.stdout:starting mds.c at 2020-11-28T21:50:00.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.331+0000 7fce9a52f700 1 Processor -- start 2020-11-28T21:50:00.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.335+0000 7fce9a52f700 1 -- start start 2020-11-28T21:50:00.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.335+0000 7fce9a52f700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fce940b33f0 0x7fce941435d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:00.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.335+0000 7fce9a52f700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce940b4fb0 con 0x7fce9409b470 2020-11-28T21:50:00.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.335+0000 7fce9a52f700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce94142a60 con 0x7fce940b4bd0 2020-11-28T21:50:00.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.335+0000 7fce9a52f700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce940b3db0 con 0x7fce940b33f0 2020-11-28T21:50:00.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.335+0000 7fce99d2e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fce940b33f0 0x7fce941435d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:00.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.335+0000 7fce9952d700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fce940b4bd0 0x7fce9409ad60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:46672/0 (socket says 172.21.15.162:46672) 2020-11-28T21:50:00.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.335+0000 7fce9952d700 1 -- 172.21.15.162:0/604574865 learned_addr learned my addr 172.21.15.162:0/604574865 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:50:00.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.335+0000 7fce99d2e700 1 -- 172.21.15.162:0/604574865 >> v1:172.21.15.162:6789/0 conn(0x7fce940b4bd0 legacy=0x7fce9409ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:00.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.335+0000 7fce99d2e700 1 -- 172.21.15.162:0/604574865 >> v1:172.21.15.118:6789/0 conn(0x7fce9409b470 legacy=0x7fce94142ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:00.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.335+0000 7fce99d2e700 1 -- 172.21.15.162:0/604574865 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce940b4050 con 0x7fce940b33f0 2020-11-28T21:50:00.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.335+0000 7fce99d2e700 1 --2- 172.21.15.162:0/604574865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce940b33f0 0x7fce941435d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7fce840010d0 tx=0x7fce8c0091a0).ready entity=mon.1 client_cookie=a54de683257c619e server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:00.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.335+0000 7fce83fff700 1 -- 172.21.15.162:0/604574865 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce8c00b910 con 0x7fce940b33f0 2020-11-28T21:50:00.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.335+0000 7fce83fff700 1 -- 172.21.15.162:0/604574865 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce8c00ba70 con 0x7fce940b33f0 2020-11-28T21:50:00.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.335+0000 7fce83fff700 1 -- 172.21.15.162:0/604574865 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce8c00bdd0 con 0x7fce940b33f0 2020-11-28T21:50:00.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.335+0000 7fce9a52f700 1 -- 172.21.15.162:0/604574865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce940b33f0 msgr2=0x7fce941435d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:00.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.335+0000 7fce9a52f700 1 --2- 172.21.15.162:0/604574865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce940b33f0 0x7fce941435d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7fce840010d0 tx=0x7fce8c0091a0).stop 2020-11-28T21:50:00.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.335+0000 7fce9a52f700 1 -- 172.21.15.162:0/604574865 shutdown_connections 2020-11-28T21:50:00.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.335+0000 7fce9a52f700 1 --2- 172.21.15.162:0/604574865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce940b33f0 0x7fce941435d0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:00.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.335+0000 7fce9a52f700 1 -- 172.21.15.162:0/604574865 >> 172.21.15.162:0/604574865 conn(0x7fce940b54f0 msgr2=0x7fce940b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:00.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.335+0000 7fce9a52f700 1 -- 172.21.15.162:0/604574865 shutdown_connections 2020-11-28T21:50:00.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.335+0000 7fce9a52f700 1 -- 172.21.15.162:0/604574865 wait complete. 2020-11-28T21:50:00.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.339+0000 7fce9a52f700 1 Processor -- start 2020-11-28T21:50:00.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.339+0000 7fce9a52f700 1 -- start start 2020-11-28T21:50:00.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.339+0000 7fce9a52f700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce940b33f0 0x7fce9409a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:00.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.339+0000 7fce9a52f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce940b4bd0 0x7fce940aacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:00.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.339+0000 7fce9a52f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce940ab1b0 0x7fce94153130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:00.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.339+0000 7fce9a52f700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce94063970 con 0x7fce940b33f0 2020-11-28T21:50:00.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.339+0000 7fce9a52f700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce94065260 con 0x7fce940b4bd0 2020-11-28T21:50:00.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.339+0000 7fce9a52f700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce940ab660 con 0x7fce940ab1b0 2020-11-28T21:50:00.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.339+0000 7fce98d2c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce940b4bd0 0x7fce940aacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:00.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.339+0000 7fce9952d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce940b33f0 0x7fce9409a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:00.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.339+0000 7fce98d2c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce940b4bd0 0x7fce940aacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39836/0 (socket says 172.21.15.162:39836) 2020-11-28T21:50:00.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.339+0000 7fce99d2e700 1 --2- 172.21.15.162:0/4088247078 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce940ab1b0 0x7fce94153130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:00.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.339+0000 7fce98d2c700 1 -- 172.21.15.162:0/4088247078 learned_addr learned my addr 172.21.15.162:0/4088247078 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:00.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.339+0000 7fce98d2c700 1 -- 172.21.15.162:0/4088247078 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce940ab1b0 msgr2=0x7fce94153130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:00.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.339+0000 7fce98d2c700 1 --2- 172.21.15.162:0/4088247078 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce940ab1b0 0x7fce94153130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:00.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.339+0000 7fce98d2c700 1 -- 172.21.15.162:0/4088247078 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce940b33f0 msgr2=0x7fce9409a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:00.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.339+0000 7fce98d2c700 1 --2- 172.21.15.162:0/4088247078 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce940b33f0 0x7fce9409a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:00.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.339+0000 7fce98d2c700 1 -- 172.21.15.162:0/4088247078 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce94066ea0 con 0x7fce940b4bd0 2020-11-28T21:50:00.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.339+0000 7fce98d2c700 1 --2- 172.21.15.162:0/4088247078 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce940b4bd0 0x7fce940aacc0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7fce84001190 tx=0x7fce84007bd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:00.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.339+0000 7fce827fc700 1 -- 172.21.15.162:0/4088247078 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce84007240 con 0x7fce940b4bd0 2020-11-28T21:50:00.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.339+0000 7fce827fc700 1 -- 172.21.15.162:0/4088247078 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce840073a0 con 0x7fce940b4bd0 2020-11-28T21:50:00.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.339+0000 7fce9a52f700 1 -- 172.21.15.162:0/4088247078 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce94067140 con 0x7fce940b4bd0 2020-11-28T21:50:00.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.339+0000 7fce827fc700 1 -- 172.21.15.162:0/4088247078 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce840066d0 con 0x7fce940b4bd0 2020-11-28T21:50:00.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.339+0000 7fce9a52f700 1 -- 172.21.15.162:0/4088247078 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce9409aa30 con 0x7fce940b4bd0 2020-11-28T21:50:00.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.343+0000 7fce9a52f700 1 -- 172.21.15.162:0/4088247078 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce94057050 con 0x7fce940b4bd0 2020-11-28T21:50:00.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.343+0000 7fce827fc700 1 -- 172.21.15.162:0/4088247078 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fce84019030 con 0x7fce940b4bd0 2020-11-28T21:50:00.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.343+0000 7fce827fc700 1 --2- 172.21.15.162:0/4088247078 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fce74055440 0x7fce74057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:00.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.343+0000 7fce827fc700 1 -- 172.21.15.162:0/4088247078 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5339+0+0 (secure 0 0 0) 0x7fce8406d150 con 0x7fce940b4bd0 2020-11-28T21:50:00.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.343+0000 7fce9952d700 1 --2- 172.21.15.162:0/4088247078 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fce74055440 0x7fce74057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:00.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.343+0000 7fce9952d700 1 --2- 172.21.15.162:0/4088247078 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fce74055440 0x7fce74057900 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7fce9000d000 tx=0x7fce90009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:00.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.347+0000 7fce827fc700 1 -- 172.21.15.162:0/4088247078 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fce84043730 con 0x7fce940b4bd0 2020-11-28T21:50:00.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:00.503+0000 7fce9a52f700 1 -- 172.21.15.162:0/4088247078 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fce94065e40 con 0x7fce940b4bd0 2020-11-28T21:50:01.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.327+0000 7fce827fc700 1 -- 172.21.15.162:0/4088247078 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v50) v1 ==== 97+0+0 (secure 0 0 0) 0x7fce84046d50 con 0x7fce940b4bd0 2020-11-28T21:50:01.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.331+0000 7fce9a52f700 1 -- 172.21.15.162:0/4088247078 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fce74055440 msgr2=0x7fce74057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:01.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.331+0000 7fce9a52f700 1 --2- 172.21.15.162:0/4088247078 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fce74055440 0x7fce74057900 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7fce9000d000 tx=0x7fce90009040).stop 2020-11-28T21:50:01.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.331+0000 7fce9a52f700 1 -- 172.21.15.162:0/4088247078 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce940b4bd0 msgr2=0x7fce940aacc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:01.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.331+0000 7fce9a52f700 1 --2- 172.21.15.162:0/4088247078 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce940b4bd0 0x7fce940aacc0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7fce84001190 tx=0x7fce84007bd0).stop 2020-11-28T21:50:01.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.331+0000 7fce9a52f700 1 -- 172.21.15.162:0/4088247078 shutdown_connections 2020-11-28T21:50:01.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.331+0000 7fce9a52f700 1 --2- 172.21.15.162:0/4088247078 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fce74055440 0x7fce74057900 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:01.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.331+0000 7fce9a52f700 1 --2- 172.21.15.162:0/4088247078 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce940ab1b0 0x7fce94153130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:01.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.331+0000 7fce9a52f700 1 --2- 172.21.15.162:0/4088247078 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce940b33f0 0x7fce9409a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:01.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.331+0000 7fce9a52f700 1 --2- 172.21.15.162:0/4088247078 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce940b4bd0 0x7fce940aacc0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:01.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.331+0000 7fce9a52f700 1 -- 172.21.15.162:0/4088247078 >> 172.21.15.162:0/4088247078 conn(0x7fce940b54f0 msgr2=0x7fce940b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:01.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.331+0000 7fce9a52f700 1 -- 172.21.15.162:0/4088247078 shutdown_connections 2020-11-28T21:50:01.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.331+0000 7fce9a52f700 1 -- 172.21.15.162:0/4088247078 wait complete. 2020-11-28T21:50:01.333 INFO:teuthology.orchestra.run.smithi162.stderr: removed all blacklist entries 2020-11-28T21:50:01.347 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2020-11-28T21:50:01.347 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth ls --format=json-pretty 2020-11-28T21:50:01.457 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.459+0000 7fd675bdc700 1 Processor -- start 2020-11-28T21:50:01.458 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.459+0000 7fd675bdc700 1 -- start start 2020-11-28T21:50:01.460 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.459+0000 7fd675bdc700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd6700611e0 0x7fd6700615c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:01.460 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.459+0000 7fd675bdc700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd6700b10f0 0x7fd6700b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:01.460 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.459+0000 7fd675bdc700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6700b1550 con 0x7fd6700b4920 2020-11-28T21:50:01.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.459+0000 7fd675bdc700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6700b1790 con 0x7fd6700b10f0 2020-11-28T21:50:01.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.459+0000 7fd675bdc700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6700b1a30 con 0x7fd6700611e0 2020-11-28T21:50:01.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.459+0000 7fd674bda700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd6700611e0 0x7fd6700615c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:01.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.459+0000 7fd66ffff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd6700b10f0 0x7fd6700b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:01.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.459+0000 7fd674bda700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd6700611e0 0x7fd6700615c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:37068/0 (socket says 172.21.15.162:37068) 2020-11-28T21:50:01.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.459+0000 7fd674bda700 1 -- 172.21.15.162:0/1367728701 learned_addr learned my addr 172.21.15.162:0/1367728701 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:01.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.463+0000 7fd674bda700 1 -- 172.21.15.162:0/1367728701 >> v2:172.21.15.118:3300/0 conn(0x7fd6700b10f0 msgr2=0x7fd6700b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:01.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.463+0000 7fd674bda700 1 --2- 172.21.15.162:0/1367728701 >> v2:172.21.15.118:3300/0 conn(0x7fd6700b10f0 0x7fd6700b4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:01.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.463+0000 7fd674bda700 1 -- 172.21.15.162:0/1367728701 >> v1:172.21.15.118:6789/0 conn(0x7fd6700b4920 legacy=0x7fd67013ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:01.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.463+0000 7fd674bda700 1 -- 172.21.15.162:0/1367728701 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6700b3f90 con 0x7fd6700611e0 2020-11-28T21:50:01.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.463+0000 7fd66ffff700 1 --2- 172.21.15.162:0/1367728701 >> v2:172.21.15.118:3300/0 conn(0x7fd6700b10f0 0x7fd6700b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:50:01.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.463+0000 7fd674bda700 1 --2- 172.21.15.162:0/1367728701 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd6700611e0 0x7fd6700615c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fd6600032b0 tx=0x7fd660009270).ready entity=mon.2 client_cookie=b3e3bdf50a0c0d88 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:01.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.463+0000 7fd66f7fe700 1 -- 172.21.15.162:0/1367728701 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd660009e60 con 0x7fd6700611e0 2020-11-28T21:50:01.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.463+0000 7fd66f7fe700 1 -- 172.21.15.162:0/1367728701 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd66000bc00 con 0x7fd6700611e0 2020-11-28T21:50:01.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.463+0000 7fd66f7fe700 1 -- 172.21.15.162:0/1367728701 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd66000bd70 con 0x7fd6700611e0 2020-11-28T21:50:01.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.463+0000 7fd675bdc700 1 -- 172.21.15.162:0/1367728701 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd6700611e0 msgr2=0x7fd6700615c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:01.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.463+0000 7fd675bdc700 1 --2- 172.21.15.162:0/1367728701 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd6700611e0 0x7fd6700615c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fd6600032b0 tx=0x7fd660009270).stop 2020-11-28T21:50:01.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.463+0000 7fd675bdc700 1 -- 172.21.15.162:0/1367728701 shutdown_connections 2020-11-28T21:50:01.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.463+0000 7fd675bdc700 1 --2- 172.21.15.162:0/1367728701 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd6700611e0 0x7fd6700615c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:01.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.463+0000 7fd675bdc700 1 --2- 172.21.15.162:0/1367728701 >> v2:172.21.15.118:3300/0 conn(0x7fd6700b10f0 0x7fd6700b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:01.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.463+0000 7fd675bdc700 1 -- 172.21.15.162:0/1367728701 >> 172.21.15.162:0/1367728701 conn(0x7fd6700b54f0 msgr2=0x7fd6700b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:01.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.463+0000 7fd675bdc700 1 -- 172.21.15.162:0/1367728701 shutdown_connections 2020-11-28T21:50:01.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.463+0000 7fd675bdc700 1 -- 172.21.15.162:0/1367728701 wait complete. 2020-11-28T21:50:01.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.467+0000 7fd675bdc700 1 Processor -- start 2020-11-28T21:50:01.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.467+0000 7fd675bdc700 1 -- start start 2020-11-28T21:50:01.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.467+0000 7fd675bdc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd6700611e0 0x7fd670151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:01.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.467+0000 7fd675bdc700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd6700b10f0 0x7fd670067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:01.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.467+0000 7fd675bdc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6700b4920 0x7fd670063970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:01.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.467+0000 7fd675bdc700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd670067a10 con 0x7fd6700611e0 2020-11-28T21:50:01.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.467+0000 7fd675bdc700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6701512b0 con 0x7fd6700b4920 2020-11-28T21:50:01.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.467+0000 7fd675bdc700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd670151550 con 0x7fd6700b10f0 2020-11-28T21:50:01.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.467+0000 7fd674bda700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd6700611e0 0x7fd670151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:01.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.467+0000 7fd674bda700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd6700611e0 0x7fd670151c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.162:46838/0 (socket says 172.21.15.162:46838) 2020-11-28T21:50:01.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.467+0000 7fd674bda700 1 -- 172.21.15.162:0/1133191263 learned_addr learned my addr 172.21.15.162:0/1133191263 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:01.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.467+0000 7fd6753db700 1 --2- 172.21.15.162:0/1133191263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6700b4920 0x7fd670063970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:01.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.467+0000 7fd66ffff700 1 --2- 172.21.15.162:0/1133191263 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd6700b10f0 0x7fd670067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:01.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.467+0000 7fd6753db700 1 -- 172.21.15.162:0/1133191263 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd6700b10f0 msgr2=0x7fd670067390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:01.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.467+0000 7fd6753db700 1 --2- 172.21.15.162:0/1133191263 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd6700b10f0 0x7fd670067390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:01.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.467+0000 7fd6753db700 1 -- 172.21.15.162:0/1133191263 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd6700611e0 msgr2=0x7fd670151c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:01.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.467+0000 7fd6753db700 1 --2- 172.21.15.162:0/1133191263 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd6700611e0 0x7fd670151c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:01.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.467+0000 7fd6753db700 1 -- 172.21.15.162:0/1133191263 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6701517f0 con 0x7fd6700b4920 2020-11-28T21:50:01.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.467+0000 7fd674bda700 1 --2- 172.21.15.162:0/1133191263 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd6700611e0 0x7fd670151c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:50:01.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.467+0000 7fd6753db700 1 --2- 172.21.15.162:0/1133191263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6700b4920 0x7fd670063970 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7fd6700b4310 tx=0x7fd668007b60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:01.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.467+0000 7fd66dffb700 1 -- 172.21.15.162:0/1133191263 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd668010660 con 0x7fd6700b4920 2020-11-28T21:50:01.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.467+0000 7fd66dffb700 1 -- 172.21.15.162:0/1133191263 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6680107c0 con 0x7fd6700b4920 2020-11-28T21:50:01.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.467+0000 7fd675bdc700 1 -- 172.21.15.162:0/1133191263 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd670067130 con 0x7fd6700b4920 2020-11-28T21:50:01.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.467+0000 7fd66dffb700 1 -- 172.21.15.162:0/1133191263 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd668010b20 con 0x7fd6700b4920 2020-11-28T21:50:01.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.471+0000 7fd675bdc700 1 -- 172.21.15.162:0/1133191263 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd670064080 con 0x7fd6700b4920 2020-11-28T21:50:01.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.471+0000 7fd675bdc700 1 -- 172.21.15.162:0/1133191263 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd670057050 con 0x7fd6700b4920 2020-11-28T21:50:01.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.471+0000 7fd66dffb700 1 -- 172.21.15.162:0/1133191263 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd668020070 con 0x7fd6700b4920 2020-11-28T21:50:01.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.471+0000 7fd66dffb700 1 --2- 172.21.15.162:0/1133191263 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd650055330 0x7fd6500577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:01.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.471+0000 7fd66dffb700 1 -- 172.21.15.162:0/1133191263 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fd66806f010 con 0x7fd6700b4920 2020-11-28T21:50:01.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.471+0000 7fd674bda700 1 --2- 172.21.15.162:0/1133191263 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd650055330 0x7fd6500577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:01.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.471+0000 7fd674bda700 1 --2- 172.21.15.162:0/1133191263 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd650055330 0x7fd6500577f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7fd66001a000 tx=0x7fd660016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:01.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.479+0000 7fd66dffb700 1 -- 172.21.15.162:0/1133191263 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd6680455a0 con 0x7fd6700b4920 2020-11-28T21:50:01.666 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.667+0000 7fd675bdc700 1 -- 172.21.15.162:0/1133191263 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd6700b1550 con 0x7fd6700b4920 2020-11-28T21:50:01.667 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.667+0000 7fd66dffb700 1 -- 172.21.15.162:0/1133191263 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+4255 (secure 0 0 0) 0x7fd6700b1550 con 0x7fd6700b4920 2020-11-28T21:50:01.668 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:50:01.668 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2020-11-28T21:50:01.668 INFO:teuthology.orchestra.run.smithi162.stdout: "auth_dump": [ 2020-11-28T21:50:01.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:50:01.669 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mds.a", 2020-11-28T21:50:01.669 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfWxUpKxAAIUDwQRKwz8H4Qk9qE356Bg==", 2020-11-28T21:50:01.669 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:50:01.669 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:50:01.669 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:50:01.670 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:50:01.670 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:50:01.670 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:50:01.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:50:01.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:50:01.671 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mds.b", 2020-11-28T21:50:01.671 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfKY2zORAAaO65etMZiYcaTDNX0nsEPw==", 2020-11-28T21:50:01.671 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:50:01.671 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:50:01.671 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:50:01.672 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:50:01.672 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:50:01.672 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:50:01.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:50:01.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:50:01.673 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mds.c", 2020-11-28T21:50:01.673 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfdQbRMxAAWlcjesxTZzZnvKCQneSjjQ==", 2020-11-28T21:50:01.673 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:50:01.674 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:50:01.674 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:50:01.674 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:50:01.674 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:50:01.674 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:50:01.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:50:01.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:50:01.677 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.0", 2020-11-28T21:50:01.677 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQADxcJfF3WLLxAAxsB5YPjH1yjv/vhlRbijqw==", 2020-11-28T21:50:01.678 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:50:01.678 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:50:01.679 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:50:01.679 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:50:01.680 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:50:01.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:50:01.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:50:01.681 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.1", 2020-11-28T21:50:01.681 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAGxcJfKYbBFhAA1x4DzFtkQQv6uWLIdOi1qQ==", 2020-11-28T21:50:01.682 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:50:01.682 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:50:01.683 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:50:01.683 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:50:01.683 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:50:01.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:50:01.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:50:01.685 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.2", 2020-11-28T21:50:01.685 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAIxcJfvcYUOhAASENjM0HfAxxyFe13mmE36Q==", 2020-11-28T21:50:01.686 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:50:01.686 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:50:01.686 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:50:01.687 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:50:01.687 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:50:01.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:50:01.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:50:01.689 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.3", 2020-11-28T21:50:01.689 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQALxcJfc/OHIRAAFYx3s8Ssttt15hjoH04tow==", 2020-11-28T21:50:01.689 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:50:01.690 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:50:01.690 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:50:01.691 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:50:01.691 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:50:01.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:50:01.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:50:01.692 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.4", 2020-11-28T21:50:01.693 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAPxcJffGWqCRAAb8CCN5oUPkdDh6BJH5HEnw==", 2020-11-28T21:50:01.693 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:50:01.693 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:50:01.693 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:50:01.694 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:50:01.694 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:50:01.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:50:01.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:50:01.695 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.5", 2020-11-28T21:50:01.695 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQARxcJfo8GKKRAAfbsCDb4L2wULiKmHyzu7/A==", 2020-11-28T21:50:01.695 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:50:01.695 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:50:01.695 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:50:01.696 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:50:01.696 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:50:01.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:50:01.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:50:01.697 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.6", 2020-11-28T21:50:01.697 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAUxcJf8Y2MEBAAqQX6rX3LK0zodlI2wK4JrA==", 2020-11-28T21:50:01.697 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:50:01.698 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:50:01.698 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:50:01.699 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:50:01.699 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:50:01.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:50:01.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:50:01.700 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.7", 2020-11-28T21:50:01.701 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAWxcJfCLBdMhAAf8QixQMQDrfZOGWN8rHe5g==", 2020-11-28T21:50:01.701 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:50:01.701 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:50:01.702 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:50:01.702 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:50:01.703 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:50:01.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:50:01.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:50:01.704 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "client.0", 2020-11-28T21:50:01.704 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQABxcJfS9vDAxAAUk4FU8FVt4if/CRH5ZisOA==", 2020-11-28T21:50:01.705 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:50:01.705 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:50:01.706 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow r", 2020-11-28T21:50:01.706 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow rw pool=cephfs_data" 2020-11-28T21:50:01.706 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:50:01.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:50:01.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:50:01.708 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "client.1", 2020-11-28T21:50:01.708 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQABxcJfuCU8BhAAXUSoSK3D0XRo58uHNzcjXg==", 2020-11-28T21:50:01.709 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:50:01.709 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:50:01.709 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow r", 2020-11-28T21:50:01.710 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow rw pool=cephfs_data" 2020-11-28T21:50:01.710 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:50:01.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:50:01.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:50:01.711 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "client.admin", 2020-11-28T21:50:01.712 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfdlTWBxAApzAxlVjzdnETQF//Vnsg5w==", 2020-11-28T21:50:01.712 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:50:01.713 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow *", 2020-11-28T21:50:01.713 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:50:01.714 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:50:01.714 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:50:01.714 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:50:01.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:50:01.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:50:01.716 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mgr.x", 2020-11-28T21:50:01.718 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfZhdvKBAAT7zCCsD2OVCHjvddGLxQ/A==", 2020-11-28T21:50:01.718 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:50:01.718 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow *", 2020-11-28T21:50:01.718 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow profile mgr", 2020-11-28T21:50:01.719 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:50:01.719 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:50:01.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:50:01.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:50:01.720 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mgr.y", 2020-11-28T21:50:01.720 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfs8fvJRAAP0Atp074EeMHiTtpCCVHuA==", 2020-11-28T21:50:01.720 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:50:01.720 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow *", 2020-11-28T21:50:01.720 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow profile mgr", 2020-11-28T21:50:01.721 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:50:01.721 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:50:01.721 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:50:01.721 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:50:01.722 INFO:teuthology.orchestra.run.smithi162.stdout:} 2020-11-28T21:50:01.724 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.671+0000 7fd675bdc700 1 -- 172.21.15.162:0/1133191263 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd650055330 msgr2=0x7fd6500577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:01.725 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.671+0000 7fd675bdc700 1 --2- 172.21.15.162:0/1133191263 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd650055330 0x7fd6500577f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7fd66001a000 tx=0x7fd660016000).stop 2020-11-28T21:50:01.725 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.671+0000 7fd675bdc700 1 -- 172.21.15.162:0/1133191263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6700b4920 msgr2=0x7fd670063970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:01.725 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.671+0000 7fd675bdc700 1 --2- 172.21.15.162:0/1133191263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6700b4920 0x7fd670063970 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7fd6700b4310 tx=0x7fd668007b60).stop 2020-11-28T21:50:01.725 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.671+0000 7fd675bdc700 1 -- 172.21.15.162:0/1133191263 shutdown_connections 2020-11-28T21:50:01.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.671+0000 7fd675bdc700 1 --2- 172.21.15.162:0/1133191263 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd650055330 0x7fd6500577f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:01.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.671+0000 7fd675bdc700 1 --2- 172.21.15.162:0/1133191263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6700b4920 0x7fd670063970 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:01.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.671+0000 7fd675bdc700 1 --2- 172.21.15.162:0/1133191263 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd6700611e0 0x7fd670151c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:01.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.671+0000 7fd675bdc700 1 --2- 172.21.15.162:0/1133191263 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd6700b10f0 0x7fd670067390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:01.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.671+0000 7fd675bdc700 1 -- 172.21.15.162:0/1133191263 >> 172.21.15.162:0/1133191263 conn(0x7fd6700b54f0 msgr2=0x7fd67005ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:01.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.671+0000 7fd675bdc700 1 -- 172.21.15.162:0/1133191263 shutdown_connections 2020-11-28T21:50:01.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.671+0000 7fd675bdc700 1 -- 172.21.15.162:0/1133191263 wait complete. 2020-11-28T21:50:01.727 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:50:01.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.843+0000 7f4aff59e700 1 Processor -- start 2020-11-28T21:50:01.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.843+0000 7f4aff59e700 1 -- start start 2020-11-28T21:50:01.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.843+0000 7f4aff59e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4b000b5a70 0x7f4b00143360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:01.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.843+0000 7f4aff59e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f4b000b5e50 0x7f4b000b6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:01.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.843+0000 7f4aff59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4b000b5430 0x7f4b0009d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:01.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.843+0000 7f4aff59e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b000b8040 con 0x7f4b000b5e50 2020-11-28T21:50:01.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.843+0000 7f4aff59e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b000b8720 con 0x7f4b000b5a70 2020-11-28T21:50:01.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.843+0000 7f4aff59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b00142980 con 0x7f4b000b5430 2020-11-28T21:50:01.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.843+0000 7f4afe59c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4b000b5a70 0x7f4b00143360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:01.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.843+0000 7f4afe59c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4b000b5a70 0x7f4b00143360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39858/0 (socket says 172.21.15.162:39858) 2020-11-28T21:50:01.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.843+0000 7f4afe59c700 1 -- 172.21.15.162:0/4250866933 learned_addr learned my addr 172.21.15.162:0/4250866933 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:01.852 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.843+0000 7f4af7fff700 1 --2- 172.21.15.162:0/4250866933 >> v2:172.21.15.118:3300/0 conn(0x7f4b000b5e50 0x7f4b000b6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:01.852 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.843+0000 7f4afed9d700 1 --2- 172.21.15.162:0/4250866933 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4b000b5430 0x7f4b0009d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:01.852 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.843+0000 7f4afe59c700 1 -- 172.21.15.162:0/4250866933 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4b000b5430 msgr2=0x7f4b0009d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:01.853 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.843+0000 7f4afe59c700 1 --2- 172.21.15.162:0/4250866933 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4b000b5430 0x7f4b0009d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:01.853 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.843+0000 7f4afe59c700 1 -- 172.21.15.162:0/4250866933 >> v2:172.21.15.118:3300/0 conn(0x7f4b000b5e50 msgr2=0x7f4b000b6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:01.854 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.843+0000 7f4afe59c700 1 --2- 172.21.15.162:0/4250866933 >> v2:172.21.15.118:3300/0 conn(0x7f4b000b5e50 0x7f4b000b6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:01.854 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.843+0000 7f4afe59c700 1 -- 172.21.15.162:0/4250866933 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b00142c20 con 0x7f4b000b5a70 2020-11-28T21:50:01.855 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.843+0000 7f4afe59c700 1 --2- 172.21.15.162:0/4250866933 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b000b5a70 0x7f4b00143360 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7f4ae8002f30 tx=0x7f4ae8006e80).ready entity=mon.1 client_cookie=eb312eaba71d0375 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:01.855 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.847+0000 7f4afdd9b700 1 -- 172.21.15.162:0/4250866933 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ae8009640 con 0x7f4b000b5a70 2020-11-28T21:50:01.855 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.847+0000 7f4afdd9b700 1 -- 172.21.15.162:0/4250866933 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae80097a0 con 0x7f4b000b5a70 2020-11-28T21:50:01.856 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.847+0000 7f4afdd9b700 1 -- 172.21.15.162:0/4250866933 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ae8009b00 con 0x7f4b000b5a70 2020-11-28T21:50:01.856 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.847+0000 7f4aff59e700 1 -- 172.21.15.162:0/4250866933 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b000b5a70 msgr2=0x7f4b00143360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:01.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.847+0000 7f4aff59e700 1 --2- 172.21.15.162:0/4250866933 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b000b5a70 0x7f4b00143360 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7f4ae8002f30 tx=0x7f4ae8006e80).stop 2020-11-28T21:50:01.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.847+0000 7f4aff59e700 1 -- 172.21.15.162:0/4250866933 shutdown_connections 2020-11-28T21:50:01.858 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.847+0000 7f4aff59e700 1 --2- 172.21.15.162:0/4250866933 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4b000b5430 0x7f4b0009d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:01.858 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.847+0000 7f4aff59e700 1 --2- 172.21.15.162:0/4250866933 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b000b5a70 0x7f4b00143360 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:01.858 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.847+0000 7f4aff59e700 1 --2- 172.21.15.162:0/4250866933 >> v2:172.21.15.118:3300/0 conn(0x7f4b000b5e50 0x7f4b000b6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:01.859 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.847+0000 7f4aff59e700 1 -- 172.21.15.162:0/4250866933 >> 172.21.15.162:0/4250866933 conn(0x7f4b0005b970 msgr2=0x7f4b0005bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:01.859 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.847+0000 7f4aff59e700 1 -- 172.21.15.162:0/4250866933 shutdown_connections 2020-11-28T21:50:01.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.847+0000 7f4aff59e700 1 -- 172.21.15.162:0/4250866933 wait complete. 2020-11-28T21:50:01.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.847+0000 7f4aff59e700 1 Processor -- start 2020-11-28T21:50:01.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.847+0000 7f4aff59e700 1 -- start start 2020-11-28T21:50:01.865 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.847+0000 7f4aff59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4b000b5430 0x7f4b0009ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:01.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.847+0000 7f4aff59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b000b5a70 0x7f4b001444f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:01.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.847+0000 7f4aff59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4b000b5e50 0x7f4b001449e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:01.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.847+0000 7f4aff59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b001527b0 con 0x7f4b000b5e50 2020-11-28T21:50:01.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.847+0000 7f4aff59e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b00151bc0 con 0x7f4b000b5a70 2020-11-28T21:50:01.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.847+0000 7f4aff59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b00143df0 con 0x7f4b000b5430 2020-11-28T21:50:01.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.847+0000 7f4af7fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b000b5a70 0x7f4b001444f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:01.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.847+0000 7f4af7fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b000b5a70 0x7f4b001444f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39866/0 (socket says 172.21.15.162:39866) 2020-11-28T21:50:01.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.847+0000 7f4af7fff700 1 -- 172.21.15.162:0/2686205181 learned_addr learned my addr 172.21.15.162:0/2686205181 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:01.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.847+0000 7f4afe59c700 1 --2- 172.21.15.162:0/2686205181 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4b000b5430 0x7f4b0009ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:01.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.847+0000 7f4afed9d700 1 --2- 172.21.15.162:0/2686205181 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4b000b5e50 0x7f4b001449e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:01.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.851+0000 7f4af7fff700 1 -- 172.21.15.162:0/2686205181 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4b000b5430 msgr2=0x7f4b0009ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:01.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.851+0000 7f4af7fff700 1 --2- 172.21.15.162:0/2686205181 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4b000b5430 0x7f4b0009ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:01.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.851+0000 7f4af7fff700 1 -- 172.21.15.162:0/2686205181 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4b000b5e50 msgr2=0x7f4b001449e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:01.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.851+0000 7f4af7fff700 1 --2- 172.21.15.162:0/2686205181 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4b000b5e50 0x7f4b001449e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:01.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.851+0000 7f4af7fff700 1 -- 172.21.15.162:0/2686205181 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b00144ed0 con 0x7f4b000b5a70 2020-11-28T21:50:01.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.851+0000 7f4af7fff700 1 --2- 172.21.15.162:0/2686205181 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b000b5a70 0x7f4b001444f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7f4af000c000 tx=0x7f4af0006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:01.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.851+0000 7f4af77fe700 1 -- 172.21.15.162:0/2686205181 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4af000ca10 con 0x7f4b000b5a70 2020-11-28T21:50:01.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.851+0000 7f4af77fe700 1 -- 172.21.15.162:0/2686205181 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af000cb70 con 0x7f4b000b5a70 2020-11-28T21:50:01.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.851+0000 7f4aff59e700 1 -- 172.21.15.162:0/2686205181 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b00151920 con 0x7f4b000b5a70 2020-11-28T21:50:01.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.851+0000 7f4af77fe700 1 -- 172.21.15.162:0/2686205181 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4af000be50 con 0x7f4b000b5a70 2020-11-28T21:50:01.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.851+0000 7f4aff59e700 1 -- 172.21.15.162:0/2686205181 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b00145190 con 0x7f4b000b5a70 2020-11-28T21:50:01.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.851+0000 7f4aff59e700 1 -- 172.21.15.162:0/2686205181 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b00057050 con 0x7f4b000b5a70 2020-11-28T21:50:01.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.851+0000 7f4af77fe700 1 -- 172.21.15.162:0/2686205181 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4af001b400 con 0x7f4b000b5a70 2020-11-28T21:50:01.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.851+0000 7f4af77fe700 1 --2- 172.21.15.162:0/2686205181 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4ad80554e0 0x7f4ad80579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:01.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.851+0000 7f4af77fe700 1 -- 172.21.15.162:0/2686205181 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f4af006dda0 con 0x7f4b000b5a70 2020-11-28T21:50:01.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.851+0000 7f4afe59c700 1 --2- 172.21.15.162:0/2686205181 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4ad80554e0 0x7f4ad80579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:01.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.855+0000 7f4afe59c700 1 --2- 172.21.15.162:0/2686205181 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4ad80554e0 0x7f4ad80579a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f4ae801b000 tx=0x7f4ae8015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:01.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.859+0000 7f4af77fe700 1 -- 172.21.15.162:0/2686205181 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4af0044260 con 0x7f4b000b5a70 2020-11-28T21:50:01.998 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:01.999+0000 7f4aff59e700 1 -- 172.21.15.162:0/2686205181 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4b0009f240 con 0x7f4b000b5a70 2020-11-28T21:50:02.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.003+0000 7f4af77fe700 1 -- 172.21.15.162:0/2686205181 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+13277 (secure 0 0 0) 0x7f4af0047880 con 0x7f4b000b5a70 2020-11-28T21:50:02.000 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:50:02.001 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":50,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:50:01.320930+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:50:02.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.003+0000 7f4aff59e700 1 -- 172.21.15.162:0/2686205181 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4ad80554e0 msgr2=0x7f4ad80579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:02.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.003+0000 7f4aff59e700 1 --2- 172.21.15.162:0/2686205181 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4ad80554e0 0x7f4ad80579a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f4ae801b000 tx=0x7f4ae8015000).stop 2020-11-28T21:50:02.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.003+0000 7f4aff59e700 1 -- 172.21.15.162:0/2686205181 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b000b5a70 msgr2=0x7f4b001444f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:02.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.003+0000 7f4aff59e700 1 --2- 172.21.15.162:0/2686205181 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b000b5a70 0x7f4b001444f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7f4af000c000 tx=0x7f4af0006dc0).stop 2020-11-28T21:50:02.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.003+0000 7f4aff59e700 1 -- 172.21.15.162:0/2686205181 shutdown_connections 2020-11-28T21:50:02.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.003+0000 7f4aff59e700 1 --2- 172.21.15.162:0/2686205181 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4ad80554e0 0x7f4ad80579a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:02.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.003+0000 7f4aff59e700 1 --2- 172.21.15.162:0/2686205181 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4b000b5430 0x7f4b0009ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:02.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.003+0000 7f4aff59e700 1 --2- 172.21.15.162:0/2686205181 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4b000b5e50 0x7f4b001449e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:02.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.003+0000 7f4aff59e700 1 --2- 172.21.15.162:0/2686205181 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b000b5a70 0x7f4b001444f0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:02.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.003+0000 7f4aff59e700 1 -- 172.21.15.162:0/2686205181 >> 172.21.15.162:0/2686205181 conn(0x7f4b0005b970 msgr2=0x7f4b00143260 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:02.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.003+0000 7f4aff59e700 1 -- 172.21.15.162:0/2686205181 shutdown_connections 2020-11-28T21:50:02.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.003+0000 7f4aff59e700 1 -- 172.21.15.162:0/2686205181 wait complete. 2020-11-28T21:50:02.014 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-28T21:50:02.015 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:50:02.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.135+0000 7f1a37564700 1 Processor -- start 2020-11-28T21:50:02.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.135+0000 7f1a37564700 1 -- start start 2020-11-28T21:50:02.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.135+0000 7f1a37564700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1a3005cd60 0x7f1a30094960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:02.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.135+0000 7f1a37564700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a30097320 con 0x7f1a30094e50 2020-11-28T21:50:02.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.135+0000 7f1a37564700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a30140860 con 0x7f1a3005c980 2020-11-28T21:50:02.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.135+0000 7f1a37564700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a30140b00 con 0x7f1a3005cd60 2020-11-28T21:50:02.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.135+0000 7f1a35300700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f1a3005c980 0x7f1a30097d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:46716/0 (socket says 172.21.15.162:46716) 2020-11-28T21:50:02.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.135+0000 7f1a35300700 1 -- 172.21.15.162:0/1963216346 learned_addr learned my addr 172.21.15.162:0/1963216346 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:50:02.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.135+0000 7f1a34aff700 1 --2- 172.21.15.162:0/1963216346 >> v2:172.21.15.118:3300/0 conn(0x7f1a3005cd60 0x7f1a30094960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:02.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.135+0000 7f1a27fff700 1 -- 172.21.15.162:0/1963216346 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 698989033 0 0) 0x7f1a30140860 con 0x7f1a3005c980 2020-11-28T21:50:02.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.135+0000 7f1a27fff700 1 -- 172.21.15.162:0/1963216346 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a14003230 con 0x7f1a3005c980 2020-11-28T21:50:02.139 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.139+0000 7f1a27fff700 1 -- 172.21.15.162:0/1963216346 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1437368603 0 0) 0x7f1a14003230 con 0x7f1a3005c980 2020-11-28T21:50:02.139 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.139+0000 7f1a27fff700 1 -- 172.21.15.162:0/1963216346 >> v2:172.21.15.118:3300/0 conn(0x7f1a3005cd60 msgr2=0x7f1a30094960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:02.139 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.139+0000 7f1a27fff700 1 --2- 172.21.15.162:0/1963216346 >> v2:172.21.15.118:3300/0 conn(0x7f1a3005cd60 0x7f1a30094960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:02.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.139+0000 7f1a27fff700 1 -- 172.21.15.162:0/1963216346 >> v1:172.21.15.118:6789/0 conn(0x7f1a30094e50 legacy=0x7f1a30095c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:02.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.139+0000 7f1a27fff700 1 -- 172.21.15.162:0/1963216346 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a30140da0 con 0x7f1a3005c980 2020-11-28T21:50:02.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.139+0000 7f1a27fff700 1 -- 172.21.15.162:0/1963216346 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f1a28001a50 con 0x7f1a3005c980 2020-11-28T21:50:02.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.139+0000 7f1a27fff700 1 -- 172.21.15.162:0/1963216346 >> v1:172.21.15.162:6789/0 conn(0x7f1a3005c980 legacy=0x7f1a30097d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:02.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.139+0000 7f1a27fff700 1 --2- 172.21.15.162:0/1963216346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a14003720 0x7f1a14003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:02.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.139+0000 7f1a27fff700 1 --2- 172.21.15.162:0/1963216346 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1a140044d0 0x7f1a14004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:02.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.139+0000 7f1a27fff700 1 --2- 172.21.15.162:0/1963216346 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1a14004e40 0x7f1a1400b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:02.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.139+0000 7f1a27fff700 1 -- 172.21.15.162:0/1963216346 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a30140b00 con 0x7f1a140044d0 2020-11-28T21:50:02.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.139+0000 7f1a27fff700 1 -- 172.21.15.162:0/1963216346 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a14004250 con 0x7f1a14003720 2020-11-28T21:50:02.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.139+0000 7f1a27fff700 1 -- 172.21.15.162:0/1963216346 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1400bc10 con 0x7f1a14004e40 2020-11-28T21:50:02.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.139+0000 7f1a35300700 1 --2- 172.21.15.162:0/1963216346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a14003720 0x7f1a14003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:02.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.139+0000 7f1a35b01700 1 --2- 172.21.15.162:0/1963216346 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1a14004e40 0x7f1a1400b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:02.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.139+0000 7f1a35300700 1 -- 172.21.15.162:0/1963216346 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1a14004e40 msgr2=0x7f1a1400b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:02.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.139+0000 7f1a34aff700 1 --2- 172.21.15.162:0/1963216346 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1a140044d0 0x7f1a14004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:02.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.139+0000 7f1a35300700 1 --2- 172.21.15.162:0/1963216346 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1a14004e40 0x7f1a1400b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:02.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.139+0000 7f1a35300700 1 -- 172.21.15.162:0/1963216346 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1a140044d0 msgr2=0x7f1a14004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:02.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.139+0000 7f1a35300700 1 --2- 172.21.15.162:0/1963216346 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1a140044d0 0x7f1a14004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:02.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.139+0000 7f1a35b01700 1 -- 172.21.15.162:0/1963216346 reap_dead start 2020-11-28T21:50:02.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.139+0000 7f1a35300700 1 -- 172.21.15.162:0/1963216346 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a14003230 con 0x7f1a14003720 2020-11-28T21:50:02.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.139+0000 7f1a34aff700 1 --2- 172.21.15.162:0/1963216346 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1a140044d0 0x7f1a14004950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:50:02.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.139+0000 7f1a35300700 1 --2- 172.21.15.162:0/1963216346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a14003720 0x7f1a14003b40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f1a28001760 tx=0x7f1a30140750).ready entity=mon.1 client_cookie=e265071123d57846 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:02.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.139+0000 7f1a27fff700 1 -- 172.21.15.162:0/1963216346 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a28005b70 con 0x7f1a14003720 2020-11-28T21:50:02.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.139+0000 7f1a27fff700 1 -- 172.21.15.162:0/1963216346 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a28005cd0 con 0x7f1a14003720 2020-11-28T21:50:02.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.139+0000 7f1a27fff700 1 -- 172.21.15.162:0/1963216346 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a280072c0 con 0x7f1a14003720 2020-11-28T21:50:02.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.139+0000 7f1a37564700 1 -- 172.21.15.162:0/1963216346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a14003720 msgr2=0x7f1a14003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:02.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.139+0000 7f1a37564700 1 --2- 172.21.15.162:0/1963216346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a14003720 0x7f1a14003b40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f1a28001760 tx=0x7f1a30140750).stop 2020-11-28T21:50:02.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.139+0000 7f1a37564700 1 -- 172.21.15.162:0/1963216346 shutdown_connections 2020-11-28T21:50:02.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.139+0000 7f1a37564700 1 --2- 172.21.15.162:0/1963216346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a14003720 0x7f1a14003b40 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:02.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.139+0000 7f1a37564700 1 -- 172.21.15.162:0/1963216346 >> 172.21.15.162:0/1963216346 conn(0x7f1a300b6220 msgr2=0x7f1a300b21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:02.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.139+0000 7f1a37564700 1 -- 172.21.15.162:0/1963216346 shutdown_connections 2020-11-28T21:50:02.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.139+0000 7f1a37564700 1 -- 172.21.15.162:0/1963216346 wait complete. 2020-11-28T21:50:02.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.139+0000 7f1a37564700 1 Processor -- start 2020-11-28T21:50:02.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.143+0000 7f1a37564700 1 -- start start 2020-11-28T21:50:02.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.143+0000 7f1a37564700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1a14003720 0x7f1a3013fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:02.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.143+0000 7f1a37564700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1a300b4ff0 0x7f1a300987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:02.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.143+0000 7f1a37564700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a300ad000 0x7f1a300ad480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:02.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.143+0000 7f1a37564700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a300ad970 con 0x7f1a300b4ff0 2020-11-28T21:50:02.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.143+0000 7f1a37564700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a30098cb0 con 0x7f1a300ad000 2020-11-28T21:50:02.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.143+0000 7f1a37564700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a30099da0 con 0x7f1a14003720 2020-11-28T21:50:02.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.143+0000 7f1a35b01700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a300ad000 0x7f1a300ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:02.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.143+0000 7f1a35b01700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a300ad000 0x7f1a300ad480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39888/0 (socket says 172.21.15.162:39888) 2020-11-28T21:50:02.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.143+0000 7f1a35b01700 1 -- 172.21.15.162:0/234773217 learned_addr learned my addr 172.21.15.162:0/234773217 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:02.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.143+0000 7f1a35300700 1 --2- 172.21.15.162:0/234773217 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1a14003720 0x7f1a3013fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:02.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.143+0000 7f1a34aff700 1 --2- 172.21.15.162:0/234773217 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1a300b4ff0 0x7f1a300987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:02.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.143+0000 7f1a35b01700 1 -- 172.21.15.162:0/234773217 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1a14003720 msgr2=0x7f1a3013fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:02.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.143+0000 7f1a35b01700 1 --2- 172.21.15.162:0/234773217 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1a14003720 0x7f1a3013fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:02.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.143+0000 7f1a35b01700 1 -- 172.21.15.162:0/234773217 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1a300b4ff0 msgr2=0x7f1a300987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:02.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.143+0000 7f1a35b01700 1 --2- 172.21.15.162:0/234773217 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1a300b4ff0 0x7f1a300987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:02.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.143+0000 7f1a35b01700 1 -- 172.21.15.162:0/234773217 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a3009a040 con 0x7f1a300ad000 2020-11-28T21:50:02.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.143+0000 7f1a35300700 1 --2- 172.21.15.162:0/234773217 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1a14003720 0x7f1a3013fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:50:02.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.143+0000 7f1a35b01700 1 --2- 172.21.15.162:0/234773217 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a300ad000 0x7f1a300ad480 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7f1a3009a660 tx=0x7f1a2c009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:02.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.143+0000 7f1a267fc700 1 -- 172.21.15.162:0/234773217 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a2c00f440 con 0x7f1a300ad000 2020-11-28T21:50:02.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.143+0000 7f1a267fc700 1 -- 172.21.15.162:0/234773217 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a2c00f5a0 con 0x7f1a300ad000 2020-11-28T21:50:02.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.143+0000 7f1a267fc700 1 -- 172.21.15.162:0/234773217 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a2c00f900 con 0x7f1a300ad000 2020-11-28T21:50:02.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.143+0000 7f1a37564700 1 -- 172.21.15.162:0/234773217 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a3009a2e0 con 0x7f1a300ad000 2020-11-28T21:50:02.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.143+0000 7f1a37564700 1 -- 172.21.15.162:0/234773217 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a3009a780 con 0x7f1a300ad000 2020-11-28T21:50:02.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.147+0000 7f1a37564700 1 -- 172.21.15.162:0/234773217 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a3009a4a0 con 0x7f1a300ad000 2020-11-28T21:50:02.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.151+0000 7f1a267fc700 1 -- 172.21.15.162:0/234773217 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1a2c01d070 con 0x7f1a300ad000 2020-11-28T21:50:02.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.151+0000 7f1a267fc700 1 --2- 172.21.15.162:0/234773217 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1a18055440 0x7f1a18057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:02.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.151+0000 7f1a35300700 1 --2- 172.21.15.162:0/234773217 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1a18055440 0x7f1a18057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:02.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.151+0000 7f1a267fc700 1 -- 172.21.15.162:0/234773217 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f1a2c06d490 con 0x7f1a300ad000 2020-11-28T21:50:02.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.151+0000 7f1a35300700 1 --2- 172.21.15.162:0/234773217 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1a18055440 0x7f1a18057900 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f1a28017000 tx=0x7f1a28015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:02.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.155+0000 7f1a267fc700 1 -- 172.21.15.162:0/234773217 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1a2c0439b0 con 0x7f1a300ad000 2020-11-28T21:50:02.294 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.295+0000 7f1a37564700 1 -- 172.21.15.162:0/234773217 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f1a30057050 con 0x7f1a300ad000 2020-11-28T21:50:02.295 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.295+0000 7f1a267fc700 1 -- 172.21.15.162:0/234773217 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v50) v1 ==== 110+0+10 (secure 0 0 0) 0x7f1a2c01b020 con 0x7f1a300ad000 2020-11-28T21:50:02.295 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:50:02.298 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.299+0000 7f1a37564700 1 -- 172.21.15.162:0/234773217 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1a18055440 msgr2=0x7f1a18057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:02.298 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.299+0000 7f1a37564700 1 --2- 172.21.15.162:0/234773217 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1a18055440 0x7f1a18057900 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f1a28017000 tx=0x7f1a28015000).stop 2020-11-28T21:50:02.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.299+0000 7f1a37564700 1 -- 172.21.15.162:0/234773217 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a300ad000 msgr2=0x7f1a300ad480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:02.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.299+0000 7f1a37564700 1 --2- 172.21.15.162:0/234773217 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a300ad000 0x7f1a300ad480 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7f1a3009a660 tx=0x7f1a2c009040).stop 2020-11-28T21:50:02.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.299+0000 7f1a37564700 1 -- 172.21.15.162:0/234773217 shutdown_connections 2020-11-28T21:50:02.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.299+0000 7f1a37564700 1 --2- 172.21.15.162:0/234773217 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1a18055440 0x7f1a18057900 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:02.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.299+0000 7f1a37564700 1 --2- 172.21.15.162:0/234773217 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1a14003720 0x7f1a3013fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:02.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.299+0000 7f1a37564700 1 --2- 172.21.15.162:0/234773217 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a300ad000 0x7f1a300ad480 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:02.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.299+0000 7f1a37564700 1 --2- 172.21.15.162:0/234773217 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1a300b4ff0 0x7f1a300987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:02.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.299+0000 7f1a37564700 1 -- 172.21.15.162:0/234773217 >> 172.21.15.162:0/234773217 conn(0x7f1a300b6220 msgr2=0x7f1a30096010 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:02.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.299+0000 7f1a37564700 1 -- 172.21.15.162:0/234773217 shutdown_connections 2020-11-28T21:50:02.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.299+0000 7f1a37564700 1 -- 172.21.15.162:0/234773217 wait complete. 2020-11-28T21:50:02.308 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:50:02.418 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.419+0000 7ffb8e008700 1 Processor -- start 2020-11-28T21:50:02.419 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.419+0000 7ffb8e008700 1 -- start start 2020-11-28T21:50:02.420 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.419+0000 7ffb8e008700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffb880991d0 0x7ffb88099600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:02.420 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.419+0000 7ffb8e008700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb880afec0 con 0x7ffb88060d10 2020-11-28T21:50:02.420 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.419+0000 7ffb8e008700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb880b0160 con 0x7ffb880b09f0 2020-11-28T21:50:02.420 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.419+0000 7ffb8e008700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb880608b0 con 0x7ffb880991d0 2020-11-28T21:50:02.421 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.423+0000 7ffb8d807700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7ffb880b09f0 0x7ffb88099af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:46740/0 (socket says 172.21.15.162:46740) 2020-11-28T21:50:02.421 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.423+0000 7ffb8d807700 1 -- 172.21.15.162:0/4044520605 learned_addr learned my addr 172.21.15.162:0/4044520605 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:50:02.421 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.423+0000 7ffb8c805700 1 --2- 172.21.15.162:0/4044520605 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffb880991d0 0x7ffb88099600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:02.423 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.423+0000 7ffb73fff700 1 -- 172.21.15.162:0/4044520605 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4152151514 0 0) 0x7ffb880b0160 con 0x7ffb880b09f0 2020-11-28T21:50:02.423 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.423+0000 7ffb73fff700 1 -- 172.21.15.162:0/4044520605 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb6c003230 con 0x7ffb880b09f0 2020-11-28T21:50:02.423 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.423+0000 7ffb73fff700 1 -- 172.21.15.162:0/4044520605 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3066902962 0 0) 0x7ffb880afec0 con 0x7ffb88060d10 2020-11-28T21:50:02.423 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.423+0000 7ffb73fff700 1 -- 172.21.15.162:0/4044520605 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb880b0160 con 0x7ffb88060d10 2020-11-28T21:50:02.424 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.423+0000 7ffb8c805700 1 -- 172.21.15.162:0/4044520605 >> v1:172.21.15.162:6789/0 conn(0x7ffb880b09f0 legacy=0x7ffb88099af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:02.424 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.423+0000 7ffb73fff700 1 -- 172.21.15.162:0/4044520605 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 674994254 0 0) 0x7ffb6c003230 con 0x7ffb880b09f0 2020-11-28T21:50:02.424 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.423+0000 7ffb8c805700 1 -- 172.21.15.162:0/4044520605 >> v1:172.21.15.118:6789/0 conn(0x7ffb88060d10 legacy=0x7ffb88098ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:02.425 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.423+0000 7ffb8c805700 1 -- 172.21.15.162:0/4044520605 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb880b0400 con 0x7ffb880991d0 2020-11-28T21:50:02.425 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.423+0000 7ffb8c805700 1 --2- 172.21.15.162:0/4044520605 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffb880991d0 0x7ffb88099600 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7ffb80001490 tx=0x7ffb7c006dc0).ready entity=mon.2 client_cookie=d9d0da92563b95c3 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:02.425 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.423+0000 7ffb73fff700 1 -- 172.21.15.162:0/4044520605 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb80001a90 con 0x7ffb880991d0 2020-11-28T21:50:02.425 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.423+0000 7ffb73fff700 1 -- 172.21.15.162:0/4044520605 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb7c00d5c0 con 0x7ffb880991d0 2020-11-28T21:50:02.425 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.423+0000 7ffb73fff700 1 -- 172.21.15.162:0/4044520605 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb7c00d730 con 0x7ffb880991d0 2020-11-28T21:50:02.426 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.423+0000 7ffb8e008700 1 -- 172.21.15.162:0/4044520605 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffb880991d0 msgr2=0x7ffb88099600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:02.426 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.423+0000 7ffb8e008700 1 --2- 172.21.15.162:0/4044520605 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffb880991d0 0x7ffb88099600 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7ffb80001490 tx=0x7ffb7c006dc0).stop 2020-11-28T21:50:02.426 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.423+0000 7ffb8e008700 1 -- 172.21.15.162:0/4044520605 shutdown_connections 2020-11-28T21:50:02.427 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.423+0000 7ffb8e008700 1 --2- 172.21.15.162:0/4044520605 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffb880991d0 0x7ffb88099600 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:02.427 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.423+0000 7ffb8e008700 1 -- 172.21.15.162:0/4044520605 >> 172.21.15.162:0/4044520605 conn(0x7ffb880b5520 msgr2=0x7ffb880b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:02.427 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.423+0000 7ffb8e008700 1 -- 172.21.15.162:0/4044520605 shutdown_connections 2020-11-28T21:50:02.427 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.423+0000 7ffb8e008700 1 -- 172.21.15.162:0/4044520605 wait complete. 2020-11-28T21:50:02.430 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.427+0000 7ffb8e008700 1 Processor -- start 2020-11-28T21:50:02.430 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.427+0000 7ffb8e008700 1 -- start start 2020-11-28T21:50:02.430 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.427+0000 7ffb8e008700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffb88060d10 0x7ffb881519c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:02.430 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.427+0000 7ffb8e008700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb880991d0 0x7ffb880919f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:02.430 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.427+0000 7ffb8e008700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffb880b09f0 0x7ffb880925f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:02.431 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.427+0000 7ffb8e008700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb88092ae0 con 0x7ffb880b09f0 2020-11-28T21:50:02.431 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.427+0000 7ffb8e008700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8805c1f0 con 0x7ffb880991d0 2020-11-28T21:50:02.431 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.427+0000 7ffb8e008700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8808fe90 con 0x7ffb88060d10 2020-11-28T21:50:02.431 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.427+0000 7ffb8c805700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb880991d0 0x7ffb880919f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:02.431 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.427+0000 7ffb8c805700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb880991d0 0x7ffb880919f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39898/0 (socket says 172.21.15.162:39898) 2020-11-28T21:50:02.432 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.427+0000 7ffb8c805700 1 -- 172.21.15.162:0/1665997546 learned_addr learned my addr 172.21.15.162:0/1665997546 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:02.432 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.427+0000 7ffb8d807700 1 --2- 172.21.15.162:0/1665997546 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffb880b09f0 0x7ffb880925f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:02.432 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.427+0000 7ffb8d006700 1 --2- 172.21.15.162:0/1665997546 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffb88060d10 0x7ffb881519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:02.432 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.427+0000 7ffb8c805700 1 -- 172.21.15.162:0/1665997546 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffb88060d10 msgr2=0x7ffb881519c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:02.433 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.427+0000 7ffb8c805700 1 --2- 172.21.15.162:0/1665997546 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffb88060d10 0x7ffb881519c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:02.433 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.427+0000 7ffb8c805700 1 -- 172.21.15.162:0/1665997546 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffb880b09f0 msgr2=0x7ffb880925f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:02.433 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.427+0000 7ffb8c805700 1 --2- 172.21.15.162:0/1665997546 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffb880b09f0 0x7ffb880925f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:02.433 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.427+0000 7ffb8c805700 1 -- 172.21.15.162:0/1665997546 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb88090130 con 0x7ffb880991d0 2020-11-28T21:50:02.433 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.427+0000 7ffb8c805700 1 --2- 172.21.15.162:0/1665997546 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb880991d0 0x7ffb880919f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7ffb80001490 tx=0x7ffb7c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:02.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.427+0000 7ffb7affd700 1 -- 172.21.15.162:0/1665997546 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb7c00de10 con 0x7ffb880991d0 2020-11-28T21:50:02.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.427+0000 7ffb7affd700 1 -- 172.21.15.162:0/1665997546 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb7c011ce0 con 0x7ffb880991d0 2020-11-28T21:50:02.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.427+0000 7ffb8e008700 1 -- 172.21.15.162:0/1665997546 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb880903d0 con 0x7ffb880991d0 2020-11-28T21:50:02.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.427+0000 7ffb7affd700 1 -- 172.21.15.162:0/1665997546 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb7c01d640 con 0x7ffb880991d0 2020-11-28T21:50:02.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.427+0000 7ffb8e008700 1 -- 172.21.15.162:0/1665997546 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb8808f910 con 0x7ffb880991d0 2020-11-28T21:50:02.435 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.431+0000 7ffb8e008700 1 -- 172.21.15.162:0/1665997546 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb88057050 con 0x7ffb880991d0 2020-11-28T21:50:02.435 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.431+0000 7ffb7affd700 1 -- 172.21.15.162:0/1665997546 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ffb7c024030 con 0x7ffb880991d0 2020-11-28T21:50:02.435 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.431+0000 7ffb7affd700 1 --2- 172.21.15.162:0/1665997546 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ffb640553b0 0x7ffb64057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:02.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.431+0000 7ffb7affd700 1 -- 172.21.15.162:0/1665997546 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5253+0+0 (secure 0 0 0) 0x7ffb7c06ede0 con 0x7ffb880991d0 2020-11-28T21:50:02.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.431+0000 7ffb8d006700 1 --2- 172.21.15.162:0/1665997546 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ffb640553b0 0x7ffb64057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:02.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.431+0000 7ffb8d006700 1 --2- 172.21.15.162:0/1665997546 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ffb640553b0 0x7ffb64057870 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7ffb8400b000 tx=0x7ffb84009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:02.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.439+0000 7ffb7affd700 1 -- 172.21.15.162:0/1665997546 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ffb7c045410 con 0x7ffb880991d0 2020-11-28T21:50:02.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.583+0000 7ffb8e008700 1 -- 172.21.15.162:0/1665997546 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ffb8805d040 con 0x7ffb880991d0 2020-11-28T21:50:02.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.583+0000 7ffb7affd700 1 -- 172.21.15.162:0/1665997546 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v50) v1 ==== 92+0+10 (secure 0 0 0) 0x7ffb7c022070 con 0x7ffb880991d0 2020-11-28T21:50:02.582 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:50:02.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.587+0000 7ffb8e008700 1 -- 172.21.15.162:0/1665997546 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ffb640553b0 msgr2=0x7ffb64057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:02.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.587+0000 7ffb8e008700 1 --2- 172.21.15.162:0/1665997546 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ffb640553b0 0x7ffb64057870 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7ffb8400b000 tx=0x7ffb84009040).stop 2020-11-28T21:50:02.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.587+0000 7ffb8e008700 1 -- 172.21.15.162:0/1665997546 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb880991d0 msgr2=0x7ffb880919f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:02.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.587+0000 7ffb8e008700 1 --2- 172.21.15.162:0/1665997546 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb880991d0 0x7ffb880919f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7ffb80001490 tx=0x7ffb7c015000).stop 2020-11-28T21:50:02.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.587+0000 7ffb8e008700 1 -- 172.21.15.162:0/1665997546 shutdown_connections 2020-11-28T21:50:02.586 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.587+0000 7ffb8e008700 1 --2- 172.21.15.162:0/1665997546 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ffb640553b0 0x7ffb64057870 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:02.586 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.587+0000 7ffb8e008700 1 --2- 172.21.15.162:0/1665997546 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffb88060d10 0x7ffb881519c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:02.586 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.587+0000 7ffb8e008700 1 --2- 172.21.15.162:0/1665997546 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffb880b09f0 0x7ffb880925f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:02.586 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.587+0000 7ffb8e008700 1 --2- 172.21.15.162:0/1665997546 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb880991d0 0x7ffb880919f0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:02.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.587+0000 7ffb8e008700 1 -- 172.21.15.162:0/1665997546 >> 172.21.15.162:0/1665997546 conn(0x7ffb880b5520 msgr2=0x7ffb880b3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:02.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.587+0000 7ffb8e008700 1 -- 172.21.15.162:0/1665997546 shutdown_connections 2020-11-28T21:50:02.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.587+0000 7ffb8e008700 1 -- 172.21.15.162:0/1665997546 wait complete. 2020-11-28T21:50:02.599 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2020-11-28T21:50:02.707 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.707+0000 7f3dd2540700 1 Processor -- start 2020-11-28T21:50:02.708 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.707+0000 7f3dd2540700 1 -- start start 2020-11-28T21:50:02.708 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.711+0000 7f3dd2540700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3dcc060d10 0x7f3dcc0b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:02.709 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.711+0000 7f3dd2540700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f3dcc0b4390 0x7f3dcc13fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:02.709 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.711+0000 7f3dd2540700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dcc0b4930 con 0x7f3dcc0b60e0 2020-11-28T21:50:02.709 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.711+0000 7f3dd2540700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dcc0b2170 con 0x7f3dcc0b4390 2020-11-28T21:50:02.709 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.711+0000 7f3dd2540700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dcc0b6660 con 0x7f3dcc060d10 2020-11-28T21:50:02.709 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.711+0000 7f3dcb7fe700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f3dcc0b60e0 0x7f3dcc146180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:46752/0 (socket says 172.21.15.162:46752) 2020-11-28T21:50:02.710 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.711+0000 7f3dcb7fe700 1 -- 172.21.15.162:0/3561462838 learned_addr learned my addr 172.21.15.162:0/3561462838 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:50:02.710 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.711+0000 7f3dcbfff700 1 --2- 172.21.15.162:0/3561462838 >> v2:172.21.15.162:3300/0 conn(0x7f3dcc060d10 0x7f3dcc0b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:02.711 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.711+0000 7f3dd0add700 1 --2- 172.21.15.162:0/3561462838 >> v2:172.21.15.118:3300/0 conn(0x7f3dcc0b4390 0x7f3dcc13fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:02.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.711+0000 7f3dcaffd700 1 -- 172.21.15.162:0/3561462838 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1293700999 0 0) 0x7f3dcc0b4930 con 0x7f3dcc0b60e0 2020-11-28T21:50:02.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.711+0000 7f3dcaffd700 1 -- 172.21.15.162:0/3561462838 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dac003230 con 0x7f3dcc0b60e0 2020-11-28T21:50:02.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.711+0000 7f3dcbfff700 1 -- 172.21.15.162:0/3561462838 >> v2:172.21.15.118:3300/0 conn(0x7f3dcc0b4390 msgr2=0x7f3dcc13fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:02.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.711+0000 7f3dcbfff700 1 --2- 172.21.15.162:0/3561462838 >> v2:172.21.15.118:3300/0 conn(0x7f3dcc0b4390 0x7f3dcc13fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:02.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.711+0000 7f3dcbfff700 1 -- 172.21.15.162:0/3561462838 >> v1:172.21.15.162:6789/0 conn(0x7f3dcc0b60e0 legacy=0x7f3dcc146180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:02.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.711+0000 7f3dcbfff700 1 -- 172.21.15.162:0/3561462838 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dcc0b4b60 con 0x7f3dcc060d10 2020-11-28T21:50:02.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.711+0000 7f3dd0add700 1 --2- 172.21.15.162:0/3561462838 >> v2:172.21.15.118:3300/0 conn(0x7f3dcc0b4390 0x7f3dcc13fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:50:02.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.711+0000 7f3dcbfff700 1 --2- 172.21.15.162:0/3561462838 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3dcc060d10 0x7f3dcc0b52a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f3db4002f30 tx=0x7f3db4006dc0).ready entity=mon.1 client_cookie=8b35a223af7b6319 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:02.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.711+0000 7f3dcaffd700 1 -- 172.21.15.162:0/3561462838 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3db40097b0 con 0x7f3dcc060d10 2020-11-28T21:50:02.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.711+0000 7f3dcaffd700 1 -- 172.21.15.162:0/3561462838 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db4009910 con 0x7f3dcc060d10 2020-11-28T21:50:02.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.711+0000 7f3dcaffd700 1 -- 172.21.15.162:0/3561462838 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3db4009c70 con 0x7f3dcc060d10 2020-11-28T21:50:02.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.711+0000 7f3dd2540700 1 -- 172.21.15.162:0/3561462838 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3dcc060d10 msgr2=0x7f3dcc0b52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:02.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.711+0000 7f3dd2540700 1 --2- 172.21.15.162:0/3561462838 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3dcc060d10 0x7f3dcc0b52a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f3db4002f30 tx=0x7f3db4006dc0).stop 2020-11-28T21:50:02.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.711+0000 7f3dd2540700 1 -- 172.21.15.162:0/3561462838 shutdown_connections 2020-11-28T21:50:02.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.711+0000 7f3dd2540700 1 --2- 172.21.15.162:0/3561462838 >> v2:172.21.15.118:3300/0 conn(0x7f3dcc0b4390 0x7f3dcc13fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:02.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.711+0000 7f3dd2540700 1 --2- 172.21.15.162:0/3561462838 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3dcc060d10 0x7f3dcc0b52a0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:02.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.711+0000 7f3dd2540700 1 -- 172.21.15.162:0/3561462838 >> 172.21.15.162:0/3561462838 conn(0x7f3dcc05fe70 msgr2=0x7f3dcc06aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:02.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.711+0000 7f3dd2540700 1 -- 172.21.15.162:0/3561462838 shutdown_connections 2020-11-28T21:50:02.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.711+0000 7f3dd2540700 1 -- 172.21.15.162:0/3561462838 wait complete. 2020-11-28T21:50:02.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.715+0000 7f3dd2540700 1 Processor -- start 2020-11-28T21:50:02.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.715+0000 7f3dd2540700 1 -- start start 2020-11-28T21:50:02.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.715+0000 7f3dd2540700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3dcc060d10 0x7f3dcc1440c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:02.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.715+0000 7f3dd2540700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3dcc0b4390 0x7f3dcc140a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:02.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.715+0000 7f3dd2540700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3dcc0b60e0 0x7f3dcc13b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:02.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.715+0000 7f3dd2540700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc13af20 con 0x7f3dcc060d10 2020-11-28T21:50:02.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.715+0000 7f3dd2540700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc152b80 con 0x7f3dcc0b60e0 2020-11-28T21:50:02.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.715+0000 7f3dd2540700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc13b150 con 0x7f3dcc0b4390 2020-11-28T21:50:02.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.715+0000 7f3dd0add700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3dcc0b60e0 0x7f3dcc13b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:02.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.715+0000 7f3dcbfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3dcc060d10 0x7f3dcc1440c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:02.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.715+0000 7f3dcbfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3dcc060d10 0x7f3dcc1440c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.162:46900/0 (socket says 172.21.15.162:46900) 2020-11-28T21:50:02.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.715+0000 7f3dd0add700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3dcc0b60e0 0x7f3dcc13b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39916/0 (socket says 172.21.15.162:39916) 2020-11-28T21:50:02.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.715+0000 7f3dcbfff700 1 -- 172.21.15.162:0/3610815046 learned_addr learned my addr 172.21.15.162:0/3610815046 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:02.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.715+0000 7f3dcb7fe700 1 --2- 172.21.15.162:0/3610815046 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3dcc0b4390 0x7f3dcc140a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:02.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.715+0000 7f3dd0add700 1 -- 172.21.15.162:0/3610815046 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3dcc0b4390 msgr2=0x7f3dcc140a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:02.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.715+0000 7f3dd0add700 1 --2- 172.21.15.162:0/3610815046 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3dcc0b4390 0x7f3dcc140a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:02.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.715+0000 7f3dd0add700 1 -- 172.21.15.162:0/3610815046 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3dcc060d10 msgr2=0x7f3dcc1440c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:02.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.715+0000 7f3dd0add700 1 --2- 172.21.15.162:0/3610815046 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3dcc060d10 0x7f3dcc1440c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:02.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.715+0000 7f3dd0add700 1 -- 172.21.15.162:0/3610815046 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dcc13b3f0 con 0x7f3dcc0b60e0 2020-11-28T21:50:02.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.715+0000 7f3dcbfff700 1 --2- 172.21.15.162:0/3610815046 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3dcc060d10 0x7f3dcc1440c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:50:02.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.715+0000 7f3dd0add700 1 --2- 172.21.15.162:0/3610815046 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3dcc0b60e0 0x7f3dcc13b890 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7f3dcc13db30 tx=0x7f3dc4008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:02.723 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.715+0000 7f3dc97fa700 1 -- 172.21.15.162:0/3610815046 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dc400c7a0 con 0x7f3dcc0b60e0 2020-11-28T21:50:02.723 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.715+0000 7f3dc97fa700 1 -- 172.21.15.162:0/3610815046 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc400c900 con 0x7f3dcc0b60e0 2020-11-28T21:50:02.723 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.715+0000 7f3dc97fa700 1 -- 172.21.15.162:0/3610815046 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dc400cc60 con 0x7f3dcc0b60e0 2020-11-28T21:50:02.723 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.715+0000 7f3dd2540700 1 -- 172.21.15.162:0/3610815046 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dcc13bd80 con 0x7f3dcc0b60e0 2020-11-28T21:50:02.723 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.715+0000 7f3dd2540700 1 -- 172.21.15.162:0/3610815046 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dcc143a70 con 0x7f3dcc0b60e0 2020-11-28T21:50:02.723 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.719+0000 7f3dc97fa700 1 -- 172.21.15.162:0/3610815046 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f3dc401d460 con 0x7f3dcc0b60e0 2020-11-28T21:50:02.724 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.719+0000 7f3dd2540700 1 -- 172.21.15.162:0/3610815046 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dcc13b580 con 0x7f3dcc0b60e0 2020-11-28T21:50:02.724 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.719+0000 7f3dc97fa700 1 --2- 172.21.15.162:0/3610815046 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3db0055440 0x7f3db0057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:02.724 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.719+0000 7f3dc97fa700 1 -- 172.21.15.162:0/3610815046 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f3dc406df80 con 0x7f3dcc0b60e0 2020-11-28T21:50:02.724 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.719+0000 7f3dcbfff700 1 --2- 172.21.15.162:0/3610815046 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3db0055440 0x7f3db0057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:02.725 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.719+0000 7f3dcbfff700 1 --2- 172.21.15.162:0/3610815046 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3db0055440 0x7f3db0057900 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f3db401a000 tx=0x7f3db4015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:02.725 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.723+0000 7f3dc97fa700 1 -- 172.21.15.162:0/3610815046 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f3dc4044550 con 0x7f3dcc0b60e0 2020-11-28T21:50:02.862 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.863+0000 7f3dd2540700 1 -- 172.21.15.162:0/3610815046 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3dcc13bf10 con 0x7f3dcc0b60e0 2020-11-28T21:50:02.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.863+0000 7f3dc97fa700 1 -- 172.21.15.162:0/3610815046 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v50) v1 ==== 84+0+143 (secure 0 0 0) 0x7f3dc4047b70 con 0x7f3dcc0b60e0 2020-11-28T21:50:02.863 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:50:02.864 INFO:teuthology.orchestra.run.smithi162.stdout:[ 2020-11-28T21:50:02.864 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:50:02.864 INFO:teuthology.orchestra.run.smithi162.stdout: "poolnum": 1, 2020-11-28T21:50:02.864 INFO:teuthology.orchestra.run.smithi162.stdout: "poolname": "device_health_metrics" 2020-11-28T21:50:02.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:50:02.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:50:02.865 INFO:teuthology.orchestra.run.smithi162.stdout: "poolnum": 2, 2020-11-28T21:50:02.865 INFO:teuthology.orchestra.run.smithi162.stdout: "poolname": "rbd" 2020-11-28T21:50:02.866 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:50:02.866 INFO:teuthology.orchestra.run.smithi162.stdout:] 2020-11-28T21:50:02.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.867+0000 7f3dd2540700 1 -- 172.21.15.162:0/3610815046 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3db0055440 msgr2=0x7f3db0057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:02.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.867+0000 7f3dd2540700 1 --2- 172.21.15.162:0/3610815046 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3db0055440 0x7f3db0057900 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f3db401a000 tx=0x7f3db4015000).stop 2020-11-28T21:50:02.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.867+0000 7f3dd2540700 1 -- 172.21.15.162:0/3610815046 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3dcc0b60e0 msgr2=0x7f3dcc13b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:02.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.867+0000 7f3dd2540700 1 --2- 172.21.15.162:0/3610815046 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3dcc0b60e0 0x7f3dcc13b890 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7f3dcc13db30 tx=0x7f3dc4008040).stop 2020-11-28T21:50:02.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.867+0000 7f3dd2540700 1 -- 172.21.15.162:0/3610815046 shutdown_connections 2020-11-28T21:50:02.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.867+0000 7f3dd2540700 1 --2- 172.21.15.162:0/3610815046 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3db0055440 0x7f3db0057900 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:02.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.867+0000 7f3dd2540700 1 --2- 172.21.15.162:0/3610815046 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3dcc0b60e0 0x7f3dcc13b890 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:02.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.867+0000 7f3dd2540700 1 --2- 172.21.15.162:0/3610815046 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3dcc060d10 0x7f3dcc1440c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:02.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.867+0000 7f3dd2540700 1 --2- 172.21.15.162:0/3610815046 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3dcc0b4390 0x7f3dcc140a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:02.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.867+0000 7f3dd2540700 1 -- 172.21.15.162:0/3610815046 >> 172.21.15.162:0/3610815046 conn(0x7f3dcc05fe70 msgr2=0x7f3dcc0afdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:02.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.867+0000 7f3dd2540700 1 -- 172.21.15.162:0/3610815046 shutdown_connections 2020-11-28T21:50:02.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.867+0000 7f3dd2540700 1 -- 172.21.15.162:0/3610815046 wait complete. 2020-11-28T21:50:02.882 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-11-28T21:50:02.883 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2020-11-28T21:50:02.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.999+0000 7ff7d9ee9700 1 Processor -- start 2020-11-28T21:50:03.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:02.999+0000 7ff7d9ee9700 1 -- start start 2020-11-28T21:50:03.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.003+0000 7ff7d9ee9700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff7d40b3ab0 0x7ff7d405c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:03.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.003+0000 7ff7d9ee9700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7d405baf0 con 0x7ff7d40b30e0 2020-11-28T21:50:03.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.003+0000 7ff7d9ee9700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7d405bd90 con 0x7ff7d40b2d00 2020-11-28T21:50:03.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.003+0000 7ff7d9ee9700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d40b0e20 con 0x7ff7d40b3ab0 2020-11-28T21:50:03.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.003+0000 7ff7d8ee7700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7ff7d40b2d00 0x7ff7d4142ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:46764/0 (socket says 172.21.15.162:46764) 2020-11-28T21:50:03.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.003+0000 7ff7d8ee7700 1 -- 172.21.15.162:0/3703574401 learned_addr learned my addr 172.21.15.162:0/3703574401 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:50:03.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.003+0000 7ff7d3fff700 1 --2- 172.21.15.162:0/3703574401 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff7d40b3ab0 0x7ff7d405c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:03.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.003+0000 7ff7d37fe700 1 -- 172.21.15.162:0/3703574401 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3892992665 0 0) 0x7ff7d405bd90 con 0x7ff7d40b2d00 2020-11-28T21:50:03.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.003+0000 7ff7d37fe700 1 -- 172.21.15.162:0/3703574401 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7b8003230 con 0x7ff7d40b2d00 2020-11-28T21:50:03.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.003+0000 7ff7d3fff700 1 -- 172.21.15.162:0/3703574401 >> v1:172.21.15.162:6789/0 conn(0x7ff7d40b2d00 legacy=0x7ff7d4142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:03.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.003+0000 7ff7d3fff700 1 -- 172.21.15.162:0/3703574401 >> v1:172.21.15.118:6789/0 conn(0x7ff7d40b30e0 legacy=0x7ff7d413fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:03.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.003+0000 7ff7d3fff700 1 -- 172.21.15.162:0/3703574401 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d405c030 con 0x7ff7d40b3ab0 2020-11-28T21:50:03.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.003+0000 7ff7d3fff700 1 --2- 172.21.15.162:0/3703574401 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff7d40b3ab0 0x7ff7d405c4d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7ff7c8002f30 tx=0x7ff7c8006dc0).ready entity=mon.2 client_cookie=3c0d7cedc2d4a7b2 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:03.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.003+0000 7ff7d37fe700 1 -- 172.21.15.162:0/3703574401 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7c80097b0 con 0x7ff7d40b3ab0 2020-11-28T21:50:03.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.003+0000 7ff7d37fe700 1 -- 172.21.15.162:0/3703574401 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7c8009910 con 0x7ff7d40b3ab0 2020-11-28T21:50:03.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.003+0000 7ff7d37fe700 1 -- 172.21.15.162:0/3703574401 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7c8009c70 con 0x7ff7d40b3ab0 2020-11-28T21:50:03.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.003+0000 7ff7d9ee9700 1 -- 172.21.15.162:0/3703574401 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff7d40b3ab0 msgr2=0x7ff7d405c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:03.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.003+0000 7ff7d9ee9700 1 --2- 172.21.15.162:0/3703574401 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff7d40b3ab0 0x7ff7d405c4d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7ff7c8002f30 tx=0x7ff7c8006dc0).stop 2020-11-28T21:50:03.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.003+0000 7ff7d9ee9700 1 -- 172.21.15.162:0/3703574401 shutdown_connections 2020-11-28T21:50:03.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.003+0000 7ff7d9ee9700 1 --2- 172.21.15.162:0/3703574401 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff7d40b3ab0 0x7ff7d405c4d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:03.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.003+0000 7ff7d9ee9700 1 -- 172.21.15.162:0/3703574401 >> 172.21.15.162:0/3703574401 conn(0x7ff7d40604c0 msgr2=0x7ff7d40608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:03.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.003+0000 7ff7d9ee9700 1 -- 172.21.15.162:0/3703574401 shutdown_connections 2020-11-28T21:50:03.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.003+0000 7ff7d9ee9700 1 -- 172.21.15.162:0/3703574401 wait complete. 2020-11-28T21:50:03.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.007+0000 7ff7d9ee9700 1 Processor -- start 2020-11-28T21:50:03.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.007+0000 7ff7d9ee9700 1 -- start start 2020-11-28T21:50:03.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.007+0000 7ff7d9ee9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff7d40b2d00 0x7ff7d40b51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:03.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.007+0000 7ff7d9ee9700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff7d40b30e0 0x7ff7d4137940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:03.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.007+0000 7ff7d9ee9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7d40b3ab0 0x7ff7d4137e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:03.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.007+0000 7ff7d9ee9700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d4144250 con 0x7ff7d40b2d00 2020-11-28T21:50:03.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.007+0000 7ff7d9ee9700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d405eb80 con 0x7ff7d40b3ab0 2020-11-28T21:50:03.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.007+0000 7ff7d9ee9700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d4146280 con 0x7ff7d40b30e0 2020-11-28T21:50:03.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.007+0000 7ff7d96e8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7d40b3ab0 0x7ff7d4137e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:03.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.007+0000 7ff7d3fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff7d40b30e0 0x7ff7d4137940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:03.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.007+0000 7ff7d96e8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7d40b3ab0 0x7ff7d4137e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39928/0 (socket says 172.21.15.162:39928) 2020-11-28T21:50:03.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.007+0000 7ff7d96e8700 1 -- 172.21.15.162:0/2533366820 learned_addr learned my addr 172.21.15.162:0/2533366820 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:03.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.007+0000 7ff7d8ee7700 1 --2- 172.21.15.162:0/2533366820 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff7d40b2d00 0x7ff7d40b51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:03.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.007+0000 7ff7d96e8700 1 -- 172.21.15.162:0/2533366820 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff7d40b30e0 msgr2=0x7ff7d4137940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:03.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.007+0000 7ff7d96e8700 1 --2- 172.21.15.162:0/2533366820 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff7d40b30e0 0x7ff7d4137940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:03.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.007+0000 7ff7d96e8700 1 -- 172.21.15.162:0/2533366820 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff7d40b2d00 msgr2=0x7ff7d40b51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:03.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.007+0000 7ff7d96e8700 1 --2- 172.21.15.162:0/2533366820 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff7d40b2d00 0x7ff7d40b51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:03.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.007+0000 7ff7d96e8700 1 -- 172.21.15.162:0/2533366820 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d41458a0 con 0x7ff7d40b3ab0 2020-11-28T21:50:03.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.007+0000 7ff7d96e8700 1 --2- 172.21.15.162:0/2533366820 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7d40b3ab0 0x7ff7d4137e30 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7ff7d40611e0 tx=0x7ff7cc007f10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:03.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.007+0000 7ff7d1ffb700 1 -- 172.21.15.162:0/2533366820 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7cc00a5a0 con 0x7ff7d40b3ab0 2020-11-28T21:50:03.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.007+0000 7ff7d1ffb700 1 -- 172.21.15.162:0/2533366820 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7cc00a700 con 0x7ff7d40b3ab0 2020-11-28T21:50:03.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.007+0000 7ff7d9ee9700 1 -- 172.21.15.162:0/2533366820 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7d4145b40 con 0x7ff7d40b3ab0 2020-11-28T21:50:03.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.007+0000 7ff7d1ffb700 1 -- 172.21.15.162:0/2533366820 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7cc00aa60 con 0x7ff7d40b3ab0 2020-11-28T21:50:03.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.007+0000 7ff7d9ee9700 1 -- 172.21.15.162:0/2533366820 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7d4138320 con 0x7ff7d40b3ab0 2020-11-28T21:50:03.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.011+0000 7ff7d9ee9700 1 -- 172.21.15.162:0/2533366820 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7d4057050 con 0x7ff7d40b3ab0 2020-11-28T21:50:03.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.011+0000 7ff7d1ffb700 1 -- 172.21.15.162:0/2533366820 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff7cc019030 con 0x7ff7d40b3ab0 2020-11-28T21:50:03.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.011+0000 7ff7d1ffb700 1 --2- 172.21.15.162:0/2533366820 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff7bc055330 0x7ff7bc0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:03.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.011+0000 7ff7d1ffb700 1 -- 172.21.15.162:0/2533366820 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5253+0+0 (secure 0 0 0) 0x7ff7cc06d450 con 0x7ff7d40b3ab0 2020-11-28T21:50:03.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.011+0000 7ff7d8ee7700 1 --2- 172.21.15.162:0/2533366820 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff7bc055330 0x7ff7bc0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:03.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.011+0000 7ff7d8ee7700 1 --2- 172.21.15.162:0/2533366820 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff7bc055330 0x7ff7bc0577f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7ff7c000d000 tx=0x7ff7c0009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:03.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.015+0000 7ff7d1ffb700 1 -- 172.21.15.162:0/2533366820 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff7cc043a80 con 0x7ff7d40b3ab0 2020-11-28T21:50:03.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:03.151+0000 7ff7d9ee9700 1 -- 172.21.15.162:0/2533366820 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7ff7d405d250 con 0x7ff7d40b3ab0 2020-11-28T21:50:04.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.163+0000 7ff7d1ffb700 1 -- 172.21.15.162:0/2533366820 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v51) v1 ==== 133+0+0 (secure 0 0 0) 0x7ff7cc0470a0 con 0x7ff7d40b3ab0 2020-11-28T21:50:04.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.167+0000 7ff7d9ee9700 1 -- 172.21.15.162:0/2533366820 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff7bc055330 msgr2=0x7ff7bc0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:04.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.167+0000 7ff7d9ee9700 1 --2- 172.21.15.162:0/2533366820 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff7bc055330 0x7ff7bc0577f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7ff7c000d000 tx=0x7ff7c0009040).stop 2020-11-28T21:50:04.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.167+0000 7ff7d9ee9700 1 -- 172.21.15.162:0/2533366820 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7d40b3ab0 msgr2=0x7ff7d4137e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:04.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.167+0000 7ff7d9ee9700 1 --2- 172.21.15.162:0/2533366820 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7d40b3ab0 0x7ff7d4137e30 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7ff7d40611e0 tx=0x7ff7cc007f10).stop 2020-11-28T21:50:04.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.167+0000 7ff7d9ee9700 1 -- 172.21.15.162:0/2533366820 shutdown_connections 2020-11-28T21:50:04.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.167+0000 7ff7d9ee9700 1 --2- 172.21.15.162:0/2533366820 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff7bc055330 0x7ff7bc0577f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:04.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.167+0000 7ff7d9ee9700 1 --2- 172.21.15.162:0/2533366820 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7d40b3ab0 0x7ff7d4137e30 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:04.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.167+0000 7ff7d9ee9700 1 --2- 172.21.15.162:0/2533366820 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff7d40b2d00 0x7ff7d40b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:04.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.167+0000 7ff7d9ee9700 1 --2- 172.21.15.162:0/2533366820 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff7d40b30e0 0x7ff7d4137940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:04.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.167+0000 7ff7d9ee9700 1 -- 172.21.15.162:0/2533366820 >> 172.21.15.162:0/2533366820 conn(0x7ff7d40604c0 msgr2=0x7ff7d40b78d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:04.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.167+0000 7ff7d9ee9700 1 -- 172.21.15.162:0/2533366820 shutdown_connections 2020-11-28T21:50:04.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.167+0000 7ff7d9ee9700 1 -- 172.21.15.162:0/2533366820 wait complete. 2020-11-28T21:50:04.169 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_metadata' created 2020-11-28T21:50:04.181 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2020-11-28T21:50:04.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.303+0000 7f7180986700 1 Processor -- start 2020-11-28T21:50:04.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.303+0000 7f7180986700 1 -- start start 2020-11-28T21:50:04.306 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.303+0000 7f7180986700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f717c0991d0 0x7f717c099600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:04.306 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.303+0000 7f7180986700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f717c0b09f0 0x7f717c099af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:04.307 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.303+0000 7f7180986700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f717c0afec0 con 0x7f717c060d10 2020-11-28T21:50:04.307 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.303+0000 7f7180986700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f717c0608b0 con 0x7f717c0991d0 2020-11-28T21:50:04.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.303+0000 7f7180986700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f717c0b0160 con 0x7f717c0b09f0 2020-11-28T21:50:04.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.303+0000 7f717b7fe700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f717c060d10 0x7f717c098ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:46776/0 (socket says 172.21.15.162:46776) 2020-11-28T21:50:04.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.303+0000 7f717b7fe700 1 -- 172.21.15.162:0/3706532661 learned_addr learned my addr 172.21.15.162:0/3706532661 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:50:04.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.303+0000 7f717affd700 1 --2- 172.21.15.162:0/3706532661 >> v2:172.21.15.118:3300/0 conn(0x7f717c0991d0 0x7f717c099600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:04.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.307+0000 7f717bfff700 1 --2- 172.21.15.162:0/3706532661 >> v2:172.21.15.162:3300/0 conn(0x7f717c0b09f0 0x7f717c099af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:04.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.307+0000 7f717affd700 1 -- 172.21.15.162:0/3706532661 >> v2:172.21.15.162:3300/0 conn(0x7f717c0b09f0 msgr2=0x7f717c099af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:04.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.307+0000 7f717affd700 1 --2- 172.21.15.162:0/3706532661 >> v2:172.21.15.162:3300/0 conn(0x7f717c0b09f0 0x7f717c099af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:04.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.307+0000 7f717affd700 1 -- 172.21.15.162:0/3706532661 >> v1:172.21.15.162:6789/0 conn(0x7f717c060d10 legacy=0x7f717c098ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:04.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.307+0000 7f717affd700 1 -- 172.21.15.162:0/3706532661 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f717c0b0400 con 0x7f717c0991d0 2020-11-28T21:50:04.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.307+0000 7f717bfff700 1 --2- 172.21.15.162:0/3706532661 >> v2:172.21.15.162:3300/0 conn(0x7f717c0b09f0 0x7f717c099af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:50:04.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.307+0000 7f717affd700 1 --2- 172.21.15.162:0/3706532661 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f717c0991d0 0x7f717c099600 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f716c002f30 tx=0x7f716c006dc0).ready entity=mon.0 client_cookie=bcbe62cc7b9a0368 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:04.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.307+0000 7f717a7fc700 1 -- 172.21.15.162:0/3706532661 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f716c0097f0 con 0x7f717c0991d0 2020-11-28T21:50:04.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.307+0000 7f717a7fc700 1 -- 172.21.15.162:0/3706532661 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716c009950 con 0x7f717c0991d0 2020-11-28T21:50:04.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.307+0000 7f717a7fc700 1 -- 172.21.15.162:0/3706532661 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f716c009cb0 con 0x7f717c0991d0 2020-11-28T21:50:04.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.307+0000 7f7180986700 1 -- 172.21.15.162:0/3706532661 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f717c0991d0 msgr2=0x7f717c099600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:04.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.307+0000 7f7180986700 1 --2- 172.21.15.162:0/3706532661 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f717c0991d0 0x7f717c099600 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f716c002f30 tx=0x7f716c006dc0).stop 2020-11-28T21:50:04.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.307+0000 7f7180986700 1 -- 172.21.15.162:0/3706532661 shutdown_connections 2020-11-28T21:50:04.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.307+0000 7f7180986700 1 --2- 172.21.15.162:0/3706532661 >> v2:172.21.15.162:3300/0 conn(0x7f717c0b09f0 0x7f717c099af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:04.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.307+0000 7f7180986700 1 --2- 172.21.15.162:0/3706532661 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f717c0991d0 0x7f717c099600 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:04.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.307+0000 7f7180986700 1 -- 172.21.15.162:0/3706532661 >> 172.21.15.162:0/3706532661 conn(0x7f717c0b5520 msgr2=0x7f717c0b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:04.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.307+0000 7f7180986700 1 -- 172.21.15.162:0/3706532661 shutdown_connections 2020-11-28T21:50:04.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.307+0000 7f7180986700 1 -- 172.21.15.162:0/3706532661 wait complete. 2020-11-28T21:50:04.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.307+0000 7f7180986700 1 Processor -- start 2020-11-28T21:50:04.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.307+0000 7f7180986700 1 -- start start 2020-11-28T21:50:04.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.311+0000 7f7180986700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f717c060d10 0x7f717c092290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:04.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.311+0000 7f7180986700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f717c0991d0 0x7f717c08fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:04.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.311+0000 7f7180986700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f717c0b09f0 0x7f717c0900d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:04.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.311+0000 7f7180986700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717c0929f0 con 0x7f717c0991d0 2020-11-28T21:50:04.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.311+0000 7f7180986700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717c0ab570 con 0x7f717c0b09f0 2020-11-28T21:50:04.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.311+0000 7f7180986700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f717c0ab030 con 0x7f717c060d10 2020-11-28T21:50:04.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.311+0000 7f717bfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f717c0b09f0 0x7f717c0900d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:04.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.311+0000 7f717bfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f717c0b09f0 0x7f717c0900d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39942/0 (socket says 172.21.15.162:39942) 2020-11-28T21:50:04.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.311+0000 7f717bfff700 1 -- 172.21.15.162:0/1114324069 learned_addr learned my addr 172.21.15.162:0/1114324069 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:04.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.311+0000 7f717b7fe700 1 --2- 172.21.15.162:0/1114324069 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f717c060d10 0x7f717c092290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:04.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.311+0000 7f717affd700 1 --2- 172.21.15.162:0/1114324069 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f717c0991d0 0x7f717c08fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:04.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.311+0000 7f717bfff700 1 -- 172.21.15.162:0/1114324069 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f717c060d10 msgr2=0x7f717c092290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:04.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.311+0000 7f717bfff700 1 --2- 172.21.15.162:0/1114324069 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f717c060d10 0x7f717c092290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:04.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.311+0000 7f717bfff700 1 -- 172.21.15.162:0/1114324069 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f717c0991d0 msgr2=0x7f717c08fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:04.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.311+0000 7f717bfff700 1 --2- 172.21.15.162:0/1114324069 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f717c0991d0 0x7f717c08fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:04.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.311+0000 7f717bfff700 1 -- 172.21.15.162:0/1114324069 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f717c0ab2d0 con 0x7f717c0b09f0 2020-11-28T21:50:04.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.311+0000 7f717affd700 1 --2- 172.21.15.162:0/1114324069 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f717c0991d0 0x7f717c08fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:50:04.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.311+0000 7f717bfff700 1 --2- 172.21.15.162:0/1114324069 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f717c0b09f0 0x7f717c0900d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f7174007000 tx=0x7f7174008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:04.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.311+0000 7f7178ff9700 1 -- 172.21.15.162:0/1114324069 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7174007790 con 0x7f717c0b09f0 2020-11-28T21:50:04.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.311+0000 7f7178ff9700 1 -- 172.21.15.162:0/1114324069 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71740078f0 con 0x7f717c0b09f0 2020-11-28T21:50:04.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.311+0000 7f7180986700 1 -- 172.21.15.162:0/1114324069 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f717c0905c0 con 0x7f717c0b09f0 2020-11-28T21:50:04.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.311+0000 7f7178ff9700 1 -- 172.21.15.162:0/1114324069 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7174007c50 con 0x7f717c0b09f0 2020-11-28T21:50:04.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.311+0000 7f7180986700 1 -- 172.21.15.162:0/1114324069 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f717c155890 con 0x7f717c0b09f0 2020-11-28T21:50:04.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.311+0000 7f7178ff9700 1 -- 172.21.15.162:0/1114324069 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7174019030 con 0x7f717c0b09f0 2020-11-28T21:50:04.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.311+0000 7f7180986700 1 -- 172.21.15.162:0/1114324069 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71580033e0 con 0x7f717c0b09f0 2020-11-28T21:50:04.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.315+0000 7f7178ff9700 1 --2- 172.21.15.162:0/1114324069 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f715c0554e0 0x7f715c0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:04.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.315+0000 7f7178ff9700 1 -- 172.21.15.162:0/1114324069 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f717406dc20 con 0x7f717c0b09f0 2020-11-28T21:50:04.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.315+0000 7f717b7fe700 1 --2- 172.21.15.162:0/1114324069 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f715c0554e0 0x7f715c0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:04.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.315+0000 7f717b7fe700 1 --2- 172.21.15.162:0/1114324069 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f715c0554e0 0x7f715c0579a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f717000d000 tx=0x7f7170009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:04.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.319+0000 7f7178ff9700 1 -- 172.21.15.162:0/1114324069 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f71740440f0 con 0x7f717c0b09f0 2020-11-28T21:50:04.454 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:04.455+0000 7f7180986700 1 -- 172.21.15.162:0/1114324069 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f71580022a0 con 0x7f717c0b09f0 2020-11-28T21:50:05.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.311+0000 7f7178ff9700 1 -- 172.21.15.162:0/1114324069 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v53) v1 ==== 125+0+0 (secure 0 0 0) 0x7f7174047710 con 0x7f717c0b09f0 2020-11-28T21:50:05.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.315+0000 7f7180986700 1 -- 172.21.15.162:0/1114324069 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f715c0554e0 msgr2=0x7f715c0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:05.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.315+0000 7f7180986700 1 --2- 172.21.15.162:0/1114324069 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f715c0554e0 0x7f715c0579a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f717000d000 tx=0x7f7170009040).stop 2020-11-28T21:50:05.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.315+0000 7f7180986700 1 -- 172.21.15.162:0/1114324069 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f717c0b09f0 msgr2=0x7f717c0900d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:05.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.315+0000 7f7180986700 1 --2- 172.21.15.162:0/1114324069 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f717c0b09f0 0x7f717c0900d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f7174007000 tx=0x7f7174008040).stop 2020-11-28T21:50:05.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.315+0000 7f7180986700 1 -- 172.21.15.162:0/1114324069 shutdown_connections 2020-11-28T21:50:05.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.315+0000 7f7180986700 1 --2- 172.21.15.162:0/1114324069 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f715c0554e0 0x7f715c0579a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:05.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.315+0000 7f7180986700 1 --2- 172.21.15.162:0/1114324069 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f717c060d10 0x7f717c092290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:05.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.315+0000 7f7180986700 1 --2- 172.21.15.162:0/1114324069 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f717c0b09f0 0x7f717c0900d0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:05.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.315+0000 7f7180986700 1 --2- 172.21.15.162:0/1114324069 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f717c0991d0 0x7f717c08fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:05.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.315+0000 7f7180986700 1 -- 172.21.15.162:0/1114324069 >> 172.21.15.162:0/1114324069 conn(0x7f717c0b5520 msgr2=0x7f717c0b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:05.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.315+0000 7f7180986700 1 -- 172.21.15.162:0/1114324069 shutdown_connections 2020-11-28T21:50:05.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.315+0000 7f7180986700 1 -- 172.21.15.162:0/1114324069 wait complete. 2020-11-28T21:50:05.318 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_data' created 2020-11-28T21:50:05.330 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2020-11-28T21:50:05.451 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.451+0000 7f6751742700 1 Processor -- start 2020-11-28T21:50:05.451 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.451+0000 7f6751742700 1 -- start start 2020-11-28T21:50:05.452 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.455+0000 7f6751742700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f674c0af6a0 0x7f674c0b2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:05.452 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.455+0000 7f6751742700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f674c060f00 con 0x7f674c0b1eb0 2020-11-28T21:50:05.453 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.455+0000 7f6751742700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f674c1471f0 con 0x7f674c060a70 2020-11-28T21:50:05.453 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.455+0000 7f6751742700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f674c060090 con 0x7f674c0af6a0 2020-11-28T21:50:05.453 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.455+0000 7f6750f41700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f674c060a70 0x7f674c0afdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:46796/0 (socket says 172.21.15.162:46796) 2020-11-28T21:50:05.453 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.455+0000 7f6750f41700 1 -- 172.21.15.162:0/2697474709 learned_addr learned my addr 172.21.15.162:0/2697474709 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:50:05.454 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.455+0000 7f674b7fe700 1 --2- 172.21.15.162:0/2697474709 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f674c0af6a0 0x7f674c0b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:05.454 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.455+0000 7f674affd700 1 -- 172.21.15.162:0/2697474709 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 459681559 0 0) 0x7f674c1471f0 con 0x7f674c060a70 2020-11-28T21:50:05.454 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.455+0000 7f674affd700 1 -- 172.21.15.162:0/2697474709 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6738003230 con 0x7f674c060a70 2020-11-28T21:50:05.456 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.455+0000 7f674b7fe700 1 -- 172.21.15.162:0/2697474709 >> v1:172.21.15.162:6789/0 conn(0x7f674c060a70 legacy=0x7f674c0afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:05.456 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.455+0000 7f674b7fe700 1 -- 172.21.15.162:0/2697474709 >> v1:172.21.15.118:6789/0 conn(0x7f674c0b1eb0 legacy=0x7f674c144350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:05.456 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.455+0000 7f674affd700 1 -- 172.21.15.162:0/2697474709 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2795725393 0 0) 0x7f674c060f00 con 0x7f674c0b1eb0 2020-11-28T21:50:05.457 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.455+0000 7f674b7fe700 1 -- 172.21.15.162:0/2697474709 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f674c060330 con 0x7f674c0af6a0 2020-11-28T21:50:05.457 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.455+0000 7f674b7fe700 1 --2- 172.21.15.162:0/2697474709 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f674c0af6a0 0x7f674c0b2290 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f673c0032b0 tx=0x7f673c0091a0).ready entity=mon.2 client_cookie=f48ba89327a5c078 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:05.457 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.455+0000 7f674affd700 1 -- 172.21.15.162:0/2697474709 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f673c00b9f0 con 0x7f674c0af6a0 2020-11-28T21:50:05.457 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.455+0000 7f674affd700 1 -- 172.21.15.162:0/2697474709 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f673c00bb50 con 0x7f674c0af6a0 2020-11-28T21:50:05.458 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.455+0000 7f6751742700 1 -- 172.21.15.162:0/2697474709 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f674c0af6a0 msgr2=0x7f674c0b2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:05.458 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.455+0000 7f674affd700 1 -- 172.21.15.162:0/2697474709 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f673c00bcc0 con 0x7f674c0af6a0 2020-11-28T21:50:05.458 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.455+0000 7f6751742700 1 --2- 172.21.15.162:0/2697474709 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f674c0af6a0 0x7f674c0b2290 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f673c0032b0 tx=0x7f673c0091a0).stop 2020-11-28T21:50:05.458 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.455+0000 7f6751742700 1 -- 172.21.15.162:0/2697474709 shutdown_connections 2020-11-28T21:50:05.459 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.455+0000 7f6751742700 1 --2- 172.21.15.162:0/2697474709 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f674c0af6a0 0x7f674c0b2290 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:05.459 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.455+0000 7f6751742700 1 -- 172.21.15.162:0/2697474709 >> 172.21.15.162:0/2697474709 conn(0x7f674c0b57d0 msgr2=0x7f674c0b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:05.459 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.455+0000 7f6751742700 1 -- 172.21.15.162:0/2697474709 shutdown_connections 2020-11-28T21:50:05.459 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.455+0000 7f6751742700 1 -- 172.21.15.162:0/2697474709 wait complete. 2020-11-28T21:50:05.459 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.459+0000 7f6751742700 1 Processor -- start 2020-11-28T21:50:05.460 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.459+0000 7f6751742700 1 -- start start 2020-11-28T21:50:05.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.459+0000 7f6751742700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f674c060a70 0x7f674c0aa510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:05.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.459+0000 7f6751742700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f674c0af6a0 0x7f674c066b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:05.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.459+0000 7f6751742700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f674c0b1eb0 0x7f674c0abc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:05.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.459+0000 7f6751742700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674c0a8f20 con 0x7f674c0af6a0 2020-11-28T21:50:05.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.459+0000 7f6751742700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674c156b50 con 0x7f674c060a70 2020-11-28T21:50:05.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.459+0000 7f6751742700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f674c156df0 con 0x7f674c0b1eb0 2020-11-28T21:50:05.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.459+0000 7f674bfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f674c060a70 0x7f674c0aa510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:05.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.459+0000 7f674bfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f674c060a70 0x7f674c0aa510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39954/0 (socket says 172.21.15.162:39954) 2020-11-28T21:50:05.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.459+0000 7f674bfff700 1 -- 172.21.15.162:0/730321672 learned_addr learned my addr 172.21.15.162:0/730321672 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:05.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.459+0000 7f6750f41700 1 --2- 172.21.15.162:0/730321672 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f674c0b1eb0 0x7f674c0abc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:05.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.459+0000 7f674b7fe700 1 --2- 172.21.15.162:0/730321672 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f674c0af6a0 0x7f674c066b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:05.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.459+0000 7f674bfff700 1 -- 172.21.15.162:0/730321672 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f674c0b1eb0 msgr2=0x7f674c0abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:05.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.459+0000 7f674bfff700 1 --2- 172.21.15.162:0/730321672 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f674c0b1eb0 0x7f674c0abc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:05.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.459+0000 7f674bfff700 1 -- 172.21.15.162:0/730321672 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f674c0af6a0 msgr2=0x7f674c066b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:05.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.459+0000 7f674bfff700 1 --2- 172.21.15.162:0/730321672 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f674c0af6a0 0x7f674c066b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:05.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.459+0000 7f674bfff700 1 -- 172.21.15.162:0/730321672 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f674c0a8c10 con 0x7f674c060a70 2020-11-28T21:50:05.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.459+0000 7f674bfff700 1 --2- 172.21.15.162:0/730321672 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f674c060a70 0x7f674c0aa510 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f6740000e90 tx=0x7f6740007c20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:05.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.459+0000 7f67497fa700 1 -- 172.21.15.162:0/730321672 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6740013410 con 0x7f674c060a70 2020-11-28T21:50:05.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.459+0000 7f6751742700 1 -- 172.21.15.162:0/730321672 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f674c0ac120 con 0x7f674c060a70 2020-11-28T21:50:05.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.459+0000 7f67497fa700 1 -- 172.21.15.162:0/730321672 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6740013570 con 0x7f674c060a70 2020-11-28T21:50:05.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.459+0000 7f67497fa700 1 -- 172.21.15.162:0/730321672 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f67400136e0 con 0x7f674c060a70 2020-11-28T21:50:05.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.459+0000 7f6751742700 1 -- 172.21.15.162:0/730321672 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f674c0a87d0 con 0x7f674c060a70 2020-11-28T21:50:05.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.463+0000 7f6751742700 1 -- 172.21.15.162:0/730321672 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f674c156fb0 con 0x7f674c060a70 2020-11-28T21:50:05.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.463+0000 7f67497fa700 1 -- 172.21.15.162:0/730321672 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6740010030 con 0x7f674c060a70 2020-11-28T21:50:05.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.463+0000 7f67497fa700 1 --2- 172.21.15.162:0/730321672 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f672c055330 0x7f672c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:05.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.463+0000 7f67497fa700 1 -- 172.21.15.162:0/730321672 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5957+0+0 (secure 0 0 0) 0x7f674006d880 con 0x7f674c060a70 2020-11-28T21:50:05.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.463+0000 7f674b7fe700 1 --2- 172.21.15.162:0/730321672 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f672c055330 0x7f672c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:05.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.463+0000 7f674b7fe700 1 --2- 172.21.15.162:0/730321672 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f672c055330 0x7f672c0577f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f673c019000 tx=0x7f673c016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:05.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.467+0000 7f67497fa700 1 -- 172.21.15.162:0/730321672 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6740043a60 con 0x7f674c060a70 2020-11-28T21:50:05.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.651+0000 7f6751742700 1 -- 172.21.15.162:0/730321672 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f674c057050 con 0x7f674c060a70 2020-11-28T21:50:05.664 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.663+0000 7f67497fa700 1 -- 172.21.15.162:0/730321672 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 9 and data pool 10 v20) v1 ==== 188+0+0 (secure 0 0 0) 0x7f6740047080 con 0x7f674c060a70 2020-11-28T21:50:05.667 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.667+0000 7f6751742700 1 -- 172.21.15.162:0/730321672 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f672c055330 msgr2=0x7f672c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:05.668 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.667+0000 7f6751742700 1 --2- 172.21.15.162:0/730321672 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f672c055330 0x7f672c0577f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f673c019000 tx=0x7f673c016000).stop 2020-11-28T21:50:05.668 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.667+0000 7f6751742700 1 -- 172.21.15.162:0/730321672 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f674c060a70 msgr2=0x7f674c0aa510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:05.668 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.667+0000 7f6751742700 1 --2- 172.21.15.162:0/730321672 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f674c060a70 0x7f674c0aa510 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f6740000e90 tx=0x7f6740007c20).stop 2020-11-28T21:50:05.669 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.671+0000 7f6751742700 1 -- 172.21.15.162:0/730321672 shutdown_connections 2020-11-28T21:50:05.669 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.671+0000 7f6751742700 1 --2- 172.21.15.162:0/730321672 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f672c055330 0x7f672c0577f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:05.669 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.671+0000 7f6751742700 1 --2- 172.21.15.162:0/730321672 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f674c0b1eb0 0x7f674c0abc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:05.669 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.671+0000 7f6751742700 1 --2- 172.21.15.162:0/730321672 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f674c060a70 0x7f674c0aa510 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:05.670 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.671+0000 7f6751742700 1 --2- 172.21.15.162:0/730321672 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f674c0af6a0 0x7f674c066b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:05.670 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.671+0000 7f6751742700 1 -- 172.21.15.162:0/730321672 >> 172.21.15.162:0/730321672 conn(0x7f674c0b57d0 msgr2=0x7f674c144ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:05.670 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.671+0000 7f6751742700 1 -- 172.21.15.162:0/730321672 shutdown_connections 2020-11-28T21:50:05.670 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.671+0000 7f6751742700 1 -- 172.21.15.162:0/730321672 wait complete. 2020-11-28T21:50:05.671 INFO:teuthology.orchestra.run.smithi162.stderr:new fs with metadata pool 9 and data pool 10 2020-11-28T21:50:05.684 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:50:05.782 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.783+0000 7f96f759e700 1 Processor -- start 2020-11-28T21:50:05.783 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.783+0000 7f96f759e700 1 -- start start 2020-11-28T21:50:05.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.783+0000 7f96f759e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f96f8098b30 0x7f96f80994f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:05.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.783+0000 7f96f759e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96f8098f10 con 0x7f96f80b0000 2020-11-28T21:50:05.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.783+0000 7f96f759e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96f80b0680 con 0x7f96f8060710 2020-11-28T21:50:05.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.783+0000 7f96f759e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f80b1030 con 0x7f96f8098b30 2020-11-28T21:50:05.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.783+0000 7f96f6d9d700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f96f8060710 0x7f96f80b1580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:46818/0 (socket says 172.21.15.162:46818) 2020-11-28T21:50:05.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.783+0000 7f96f6d9d700 1 -- 172.21.15.162:0/2799867616 learned_addr learned my addr 172.21.15.162:0/2799867616 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:50:05.788 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.783+0000 7f96f659c700 1 --2- 172.21.15.162:0/2799867616 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f96f8098b30 0x7f96f80994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:05.788 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.783+0000 7f96f5d9b700 1 -- 172.21.15.162:0/2799867616 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2674663129 0 0) 0x7f96f80b0680 con 0x7f96f8060710 2020-11-28T21:50:05.788 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.783+0000 7f96f5d9b700 1 -- 172.21.15.162:0/2799867616 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96d8003230 con 0x7f96f8060710 2020-11-28T21:50:05.788 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.787+0000 7f96f5d9b700 1 -- 172.21.15.162:0/2799867616 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1055553856 0 0) 0x7f96f8098f10 con 0x7f96f80b0000 2020-11-28T21:50:05.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.787+0000 7f96f5d9b700 1 -- 172.21.15.162:0/2799867616 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96f80b0680 con 0x7f96f80b0000 2020-11-28T21:50:05.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.787+0000 7f96f5d9b700 1 -- 172.21.15.162:0/2799867616 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4120874651 0 0) 0x7f96d8003230 con 0x7f96f8060710 2020-11-28T21:50:05.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.787+0000 7f96f5d9b700 1 -- 172.21.15.162:0/2799867616 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f96f8098b30 msgr2=0x7f96f80994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:05.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.787+0000 7f96f5d9b700 1 --2- 172.21.15.162:0/2799867616 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f96f8098b30 0x7f96f80994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:05.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.787+0000 7f96f5d9b700 1 -- 172.21.15.162:0/2799867616 >> v1:172.21.15.118:6789/0 conn(0x7f96f80b0000 legacy=0x7f96f8060000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:05.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.787+0000 7f96f5d9b700 1 -- 172.21.15.162:0/2799867616 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f80b0920 con 0x7f96f8060710 2020-11-28T21:50:05.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.787+0000 7f96f5d9b700 1 -- 172.21.15.162:0/2799867616 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f96f0001a50 con 0x7f96f8060710 2020-11-28T21:50:05.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.787+0000 7f96f5d9b700 1 -- 172.21.15.162:0/2799867616 >> v1:172.21.15.162:6789/0 conn(0x7f96f8060710 legacy=0x7f96f80b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:05.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.787+0000 7f96f5d9b700 1 --2- 172.21.15.162:0/2799867616 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96d80038e0 0x7f96d8003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:05.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.787+0000 7f96f5d9b700 1 --2- 172.21.15.162:0/2799867616 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f96f8098b30 0x7f96d80044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:05.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.787+0000 7f96f5d9b700 1 --2- 172.21.15.162:0/2799867616 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96d80049c0 0x7f96d8004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:05.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.787+0000 7f96f5d9b700 1 -- 172.21.15.162:0/2799867616 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f80b1030 con 0x7f96d80049c0 2020-11-28T21:50:05.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.787+0000 7f96f5d9b700 1 -- 172.21.15.162:0/2799867616 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d8005330 con 0x7f96d80038e0 2020-11-28T21:50:05.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.787+0000 7f96f5d9b700 1 -- 172.21.15.162:0/2799867616 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d800b720 con 0x7f96f8098b30 2020-11-28T21:50:05.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.787+0000 7f96f659c700 1 --2- 172.21.15.162:0/2799867616 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96d80038e0 0x7f96d8003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:05.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.787+0000 7f96effff700 1 --2- 172.21.15.162:0/2799867616 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f96f8098b30 0x7f96d80044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:05.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.787+0000 7f96f6d9d700 1 --2- 172.21.15.162:0/2799867616 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96d80049c0 0x7f96d8004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:05.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.787+0000 7f96f659c700 1 -- 172.21.15.162:0/2799867616 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f96f8098b30 msgr2=0x7f96d80044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:05.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.787+0000 7f96f659c700 1 --2- 172.21.15.162:0/2799867616 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f96f8098b30 0x7f96d80044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:05.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.787+0000 7f96f659c700 1 -- 172.21.15.162:0/2799867616 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96d80049c0 msgr2=0x7f96d8004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:05.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.787+0000 7f96f659c700 1 --2- 172.21.15.162:0/2799867616 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96d80049c0 0x7f96d8004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:05.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.787+0000 7f96f659c700 1 -- 172.21.15.162:0/2799867616 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d8003230 con 0x7f96d80038e0 2020-11-28T21:50:05.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.787+0000 7f96f6d9d700 1 --2- 172.21.15.162:0/2799867616 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96d80049c0 0x7f96d8004e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:50:05.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.787+0000 7f96f659c700 1 --2- 172.21.15.162:0/2799867616 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96d80038e0 0x7f96d8003ce0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7f96e000c000 tx=0x7f96e0006dc0).ready entity=mon.1 client_cookie=8244fa667414e4a6 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:05.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.787+0000 7f96f5d9b700 1 -- 172.21.15.162:0/2799867616 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96e000ebc0 con 0x7f96d80038e0 2020-11-28T21:50:05.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.787+0000 7f96f5d9b700 1 -- 172.21.15.162:0/2799867616 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e000ed20 con 0x7f96d80038e0 2020-11-28T21:50:05.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.787+0000 7f96f5d9b700 1 -- 172.21.15.162:0/2799867616 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96e00125e0 con 0x7f96d80038e0 2020-11-28T21:50:05.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.787+0000 7f96f759e700 1 -- 172.21.15.162:0/2799867616 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96d80038e0 msgr2=0x7f96d8003ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:05.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.787+0000 7f96f759e700 1 --2- 172.21.15.162:0/2799867616 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96d80038e0 0x7f96d8003ce0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7f96e000c000 tx=0x7f96e0006dc0).stop 2020-11-28T21:50:05.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.787+0000 7f96f6d9d700 1 -- 172.21.15.162:0/2799867616 reap_dead start 2020-11-28T21:50:05.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.787+0000 7f96f759e700 1 -- 172.21.15.162:0/2799867616 shutdown_connections 2020-11-28T21:50:05.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.787+0000 7f96f759e700 1 -- 172.21.15.162:0/2799867616 >> 172.21.15.162:0/2799867616 conn(0x7f96f805b970 msgr2=0x7f96f805bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:05.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.787+0000 7f96f759e700 1 -- 172.21.15.162:0/2799867616 shutdown_connections 2020-11-28T21:50:05.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.787+0000 7f96f759e700 1 -- 172.21.15.162:0/2799867616 wait complete. 2020-11-28T21:50:05.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.787+0000 7f96f759e700 1 Processor -- start 2020-11-28T21:50:05.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.787+0000 7f96f759e700 1 -- start start 2020-11-28T21:50:05.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.791+0000 7f96f759e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96f8063680 0x7f96f81567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:05.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.791+0000 7f96f759e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96f80a7750 0x7f96f80a7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:05.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.791+0000 7f96f759e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f96f80647d0 0x7f96f8064c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:05.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.791+0000 7f96f759e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f80b29f0 con 0x7f96f80a7750 2020-11-28T21:50:05.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.791+0000 7f96f759e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f80676d0 con 0x7f96f8063680 2020-11-28T21:50:05.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.791+0000 7f96f759e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f80aeca0 con 0x7f96f80647d0 2020-11-28T21:50:05.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.791+0000 7f96f659c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96f8063680 0x7f96f81567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:05.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.791+0000 7f96f659c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96f8063680 0x7f96f81567f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39984/0 (socket says 172.21.15.162:39984) 2020-11-28T21:50:05.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.791+0000 7f96f659c700 1 -- 172.21.15.162:0/3248380529 learned_addr learned my addr 172.21.15.162:0/3248380529 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:05.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.791+0000 7f96effff700 1 --2- 172.21.15.162:0/3248380529 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96f80a7750 0x7f96f80a7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:05.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.791+0000 7f96f6d9d700 1 --2- 172.21.15.162:0/3248380529 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f96f80647d0 0x7f96f8064c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:05.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.791+0000 7f96f659c700 1 -- 172.21.15.162:0/3248380529 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f96f80647d0 msgr2=0x7f96f8064c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:05.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.791+0000 7f96f659c700 1 --2- 172.21.15.162:0/3248380529 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f96f80647d0 0x7f96f8064c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:05.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.791+0000 7f96f659c700 1 -- 172.21.15.162:0/3248380529 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96f80a7750 msgr2=0x7f96f80a7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:05.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.791+0000 7f96f659c700 1 --2- 172.21.15.162:0/3248380529 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96f80a7750 0x7f96f80a7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:05.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.791+0000 7f96f659c700 1 -- 172.21.15.162:0/3248380529 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f8063a60 con 0x7f96f8063680 2020-11-28T21:50:05.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.791+0000 7f96f6d9d700 1 --2- 172.21.15.162:0/3248380529 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f96f80647d0 0x7f96f8064c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:50:05.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.791+0000 7f96f659c700 1 --2- 172.21.15.162:0/3248380529 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96f8063680 0x7f96f81567f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7f96f8065e10 tx=0x7f96e0019000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:05.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.791+0000 7f96ef7fe700 1 -- 172.21.15.162:0/3248380529 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96e001d410 con 0x7f96f8063680 2020-11-28T21:50:05.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.791+0000 7f96ef7fe700 1 -- 172.21.15.162:0/3248380529 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e0021070 con 0x7f96f8063680 2020-11-28T21:50:05.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.791+0000 7f96f759e700 1 -- 172.21.15.162:0/3248380529 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96f8156ce0 con 0x7f96f8063680 2020-11-28T21:50:05.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.791+0000 7f96ef7fe700 1 -- 172.21.15.162:0/3248380529 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96e001d760 con 0x7f96f8063680 2020-11-28T21:50:05.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.791+0000 7f96f759e700 1 -- 172.21.15.162:0/3248380529 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96f8067190 con 0x7f96f8063680 2020-11-28T21:50:05.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.791+0000 7f96f759e700 1 -- 172.21.15.162:0/3248380529 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96f80644c0 con 0x7f96f8063680 2020-11-28T21:50:05.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.791+0000 7f96ef7fe700 1 -- 172.21.15.162:0/3248380529 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f96e002d070 con 0x7f96f8063680 2020-11-28T21:50:05.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.791+0000 7f96ef7fe700 1 --2- 172.21.15.162:0/3248380529 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f96dc0553b0 0x7f96dc057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:05.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.791+0000 7f96ef7fe700 1 -- 172.21.15.162:0/3248380529 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f96e006ffe0 con 0x7f96f8063680 2020-11-28T21:50:05.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.791+0000 7f96effff700 1 --2- 172.21.15.162:0/3248380529 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f96dc0553b0 0x7f96dc057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:05.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.795+0000 7f96effff700 1 --2- 172.21.15.162:0/3248380529 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f96dc0553b0 0x7f96dc057870 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f96e800d000 tx=0x7f96e8009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:05.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.795+0000 7f96ef7fe700 1 -- 172.21.15.162:0/3248380529 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f96e00462e0 con 0x7f96f8063680 2020-11-28T21:50:05.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.935+0000 7f96f759e700 1 -- 172.21.15.162:0/3248380529 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f96f8057050 con 0x7f96f8063680 2020-11-28T21:50:05.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.939+0000 7f96ef7fe700 1 -- 172.21.15.162:0/3248380529 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16149 (secure 0 0 0) 0x7f96e0049900 con 0x7f96f8063680 2020-11-28T21:50:05.938 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:50:05.939 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":54,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:50:05.654810+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:50:03.211406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2020-11-28T21:50:04.511075+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:50:05.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.943+0000 7f96f759e700 1 -- 172.21.15.162:0/3248380529 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f96dc0553b0 msgr2=0x7f96dc057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:05.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.943+0000 7f96f759e700 1 --2- 172.21.15.162:0/3248380529 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f96dc0553b0 0x7f96dc057870 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f96e800d000 tx=0x7f96e8009040).stop 2020-11-28T21:50:05.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.943+0000 7f96f759e700 1 -- 172.21.15.162:0/3248380529 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96f8063680 msgr2=0x7f96f81567f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:05.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.943+0000 7f96f759e700 1 --2- 172.21.15.162:0/3248380529 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96f8063680 0x7f96f81567f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7f96f8065e10 tx=0x7f96e0019000).stop 2020-11-28T21:50:05.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.943+0000 7f96f759e700 1 -- 172.21.15.162:0/3248380529 shutdown_connections 2020-11-28T21:50:05.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.943+0000 7f96f759e700 1 --2- 172.21.15.162:0/3248380529 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f96dc0553b0 0x7f96dc057870 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:05.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.943+0000 7f96f759e700 1 --2- 172.21.15.162:0/3248380529 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f96f80647d0 0x7f96f8064c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:05.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.943+0000 7f96f759e700 1 --2- 172.21.15.162:0/3248380529 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96f8063680 0x7f96f81567f0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:05.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.943+0000 7f96f759e700 1 --2- 172.21.15.162:0/3248380529 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96f80a7750 0x7f96f80a7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:05.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.943+0000 7f96f759e700 1 -- 172.21.15.162:0/3248380529 >> 172.21.15.162:0/3248380529 conn(0x7f96f805b970 msgr2=0x7f96f805bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:05.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.943+0000 7f96f759e700 1 -- 172.21.15.162:0/3248380529 shutdown_connections 2020-11-28T21:50:05.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:05.943+0000 7f96f759e700 1 -- 172.21.15.162:0/3248380529 wait complete. 2020-11-28T21:50:05.960 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:50:06.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.075+0000 7f8df759e700 1 Processor -- start 2020-11-28T21:50:06.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.075+0000 7f8df759e700 1 -- start start 2020-11-28T21:50:06.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.079+0000 7f8df759e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f8df8060cf0 0x7f8df80aa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:06.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.079+0000 7f8df759e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8df805cbd0 0x7f8df80aa9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:06.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.079+0000 7f8df759e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8df80b0230 con 0x7f8df80af640 2020-11-28T21:50:06.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.079+0000 7f8df759e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8df80b12a0 con 0x7f8df8060cf0 2020-11-28T21:50:06.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.079+0000 7f8df759e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df80610d0 con 0x7f8df805cbd0 2020-11-28T21:50:06.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.079+0000 7f8df5d9b700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8df805cbd0 0x7f8df80aa9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:06.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.079+0000 7f8df5d9b700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8df805cbd0 0x7f8df80aa9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:37216/0 (socket says 172.21.15.162:37216) 2020-11-28T21:50:06.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.079+0000 7f8df5d9b700 1 -- 172.21.15.162:0/1271102530 learned_addr learned my addr 172.21.15.162:0/1271102530 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:06.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.079+0000 7f8df659c700 1 --2- 172.21.15.162:0/1271102530 >> v2:172.21.15.118:3300/0 conn(0x7f8df8060cf0 0x7f8df80aa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:06.083 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.079+0000 7f8df5d9b700 1 -- 172.21.15.162:0/1271102530 >> v2:172.21.15.118:3300/0 conn(0x7f8df8060cf0 msgr2=0x7f8df80aa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:06.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.079+0000 7f8df5d9b700 1 --2- 172.21.15.162:0/1271102530 >> v2:172.21.15.118:3300/0 conn(0x7f8df8060cf0 0x7f8df80aa4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:06.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.079+0000 7f8df5d9b700 1 -- 172.21.15.162:0/1271102530 >> v1:172.21.15.118:6789/0 conn(0x7f8df80af640 legacy=0x7f8df80ae1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:06.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.079+0000 7f8df5d9b700 1 -- 172.21.15.162:0/1271102530 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df80a8c00 con 0x7f8df805cbd0 2020-11-28T21:50:06.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.079+0000 7f8df5d9b700 1 --2- 172.21.15.162:0/1271102530 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8df805cbd0 0x7f8df80aa9e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f8de0002f30 tx=0x7f8de0006e80).ready entity=mon.2 client_cookie=45573fa830ccac49 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:06.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.079+0000 7f8df559a700 1 -- 172.21.15.162:0/1271102530 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8de00099e0 con 0x7f8df805cbd0 2020-11-28T21:50:06.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.079+0000 7f8df559a700 1 -- 172.21.15.162:0/1271102530 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de0009b40 con 0x7f8df805cbd0 2020-11-28T21:50:06.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.079+0000 7f8df559a700 1 -- 172.21.15.162:0/1271102530 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8de0009ea0 con 0x7f8df805cbd0 2020-11-28T21:50:06.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.079+0000 7f8df759e700 1 -- 172.21.15.162:0/1271102530 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8df805cbd0 msgr2=0x7f8df80aa9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:06.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.079+0000 7f8df759e700 1 --2- 172.21.15.162:0/1271102530 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8df805cbd0 0x7f8df80aa9e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f8de0002f30 tx=0x7f8de0006e80).stop 2020-11-28T21:50:06.086 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.079+0000 7f8df759e700 1 -- 172.21.15.162:0/1271102530 shutdown_connections 2020-11-28T21:50:06.086 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.079+0000 7f8df759e700 1 --2- 172.21.15.162:0/1271102530 >> v2:172.21.15.118:3300/0 conn(0x7f8df8060cf0 0x7f8df80aa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:06.086 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.079+0000 7f8df759e700 1 --2- 172.21.15.162:0/1271102530 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8df805cbd0 0x7f8df80aa9e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:06.086 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.079+0000 7f8df759e700 1 -- 172.21.15.162:0/1271102530 >> 172.21.15.162:0/1271102530 conn(0x7f8df805b970 msgr2=0x7f8df805bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:06.086 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.079+0000 7f8df759e700 1 -- 172.21.15.162:0/1271102530 shutdown_connections 2020-11-28T21:50:06.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.079+0000 7f8df759e700 1 -- 172.21.15.162:0/1271102530 wait complete. 2020-11-28T21:50:06.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.079+0000 7f8df759e700 1 Processor -- start 2020-11-28T21:50:06.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.079+0000 7f8df759e700 1 -- start start 2020-11-28T21:50:06.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.083+0000 7f8df759e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8df805cbd0 0x7f8df80ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:06.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.083+0000 7f8df759e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8df8060cf0 0x7f8df814deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:06.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.083+0000 7f8df759e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8df80af640 0x7f8df8063340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:06.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.083+0000 7f8df759e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df814e3a0 con 0x7f8df805cbd0 2020-11-28T21:50:06.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.083+0000 7f8df759e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df814e590 con 0x7f8df80af640 2020-11-28T21:50:06.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.083+0000 7f8df759e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df808fde0 con 0x7f8df8060cf0 2020-11-28T21:50:06.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.083+0000 7f8df6d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8df80af640 0x7f8df8063340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:06.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.083+0000 7f8df5d9b700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8df8060cf0 0x7f8df814deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:06.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.083+0000 7f8df6d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8df80af640 0x7f8df8063340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40002/0 (socket says 172.21.15.162:40002) 2020-11-28T21:50:06.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.083+0000 7f8df6d9d700 1 -- 172.21.15.162:0/1982393684 learned_addr learned my addr 172.21.15.162:0/1982393684 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:06.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.083+0000 7f8df659c700 1 --2- 172.21.15.162:0/1982393684 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8df805cbd0 0x7f8df80ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:06.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.083+0000 7f8df6d9d700 1 -- 172.21.15.162:0/1982393684 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8df8060cf0 msgr2=0x7f8df814deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:06.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.083+0000 7f8df6d9d700 1 --2- 172.21.15.162:0/1982393684 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8df8060cf0 0x7f8df814deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:06.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.083+0000 7f8df6d9d700 1 -- 172.21.15.162:0/1982393684 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8df805cbd0 msgr2=0x7f8df80ad2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:06.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.083+0000 7f8df6d9d700 1 --2- 172.21.15.162:0/1982393684 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8df805cbd0 0x7f8df80ad2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:06.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.083+0000 7f8df6d9d700 1 -- 172.21.15.162:0/1982393684 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df8090080 con 0x7f8df80af640 2020-11-28T21:50:06.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.083+0000 7f8df5d9b700 1 --2- 172.21.15.162:0/1982393684 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8df8060cf0 0x7f8df814deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:50:06.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.083+0000 7f8df6d9d700 1 --2- 172.21.15.162:0/1982393684 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8df80af640 0x7f8df8063340 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7f8df8067980 tx=0x7f8df0007f20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:06.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.083+0000 7f8def7fe700 1 -- 172.21.15.162:0/1982393684 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8df0010510 con 0x7f8df80af640 2020-11-28T21:50:06.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.083+0000 7f8def7fe700 1 -- 172.21.15.162:0/1982393684 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df0010670 con 0x7f8df80af640 2020-11-28T21:50:06.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.083+0000 7f8df759e700 1 -- 172.21.15.162:0/1982393684 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df8090320 con 0x7f8df80af640 2020-11-28T21:50:06.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.083+0000 7f8def7fe700 1 -- 172.21.15.162:0/1982393684 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8df00107e0 con 0x7f8df80af640 2020-11-28T21:50:06.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.083+0000 7f8df759e700 1 -- 172.21.15.162:0/1982393684 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df8064910 con 0x7f8df80af640 2020-11-28T21:50:06.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.083+0000 7f8df759e700 1 -- 172.21.15.162:0/1982393684 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8df80904b0 con 0x7f8df80af640 2020-11-28T21:50:06.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.087+0000 7f8def7fe700 1 -- 172.21.15.162:0/1982393684 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8df0010070 con 0x7f8df80af640 2020-11-28T21:50:06.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.087+0000 7f8def7fe700 1 --2- 172.21.15.162:0/1982393684 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8ddc055440 0x7f8ddc057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:06.094 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.087+0000 7f8def7fe700 1 -- 172.21.15.162:0/1982393684 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8df006d9e0 con 0x7f8df80af640 2020-11-28T21:50:06.094 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.087+0000 7f8df659c700 1 --2- 172.21.15.162:0/1982393684 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8ddc055440 0x7f8ddc057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:06.094 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.087+0000 7f8df659c700 1 --2- 172.21.15.162:0/1982393684 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8ddc055440 0x7f8ddc057900 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f8de800b000 tx=0x7f8de8006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:06.094 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.091+0000 7f8def7fe700 1 -- 172.21.15.162:0/1982393684 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8df0043ce0 con 0x7f8df80af640 2020-11-28T21:50:06.234 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.235+0000 7f8df759e700 1 -- 172.21.15.162:0/1982393684 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8df8057050 con 0x7f8df80af640 2020-11-28T21:50:06.235 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.239+0000 7f8def7fe700 1 -- 172.21.15.162:0/1982393684 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16149 (secure 0 0 0) 0x7f8df0047300 con 0x7f8df80af640 2020-11-28T21:50:06.236 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:50:06.236 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":54,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:50:05.654810+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:50:03.211406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2020-11-28T21:50:04.511075+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:50:06.238 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.239+0000 7f8df759e700 1 -- 172.21.15.162:0/1982393684 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8ddc055440 msgr2=0x7f8ddc057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:06.238 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.239+0000 7f8df759e700 1 --2- 172.21.15.162:0/1982393684 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8ddc055440 0x7f8ddc057900 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f8de800b000 tx=0x7f8de8006dc0).stop 2020-11-28T21:50:06.238 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.239+0000 7f8df759e700 1 -- 172.21.15.162:0/1982393684 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8df80af640 msgr2=0x7f8df8063340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:06.239 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.239+0000 7f8df759e700 1 --2- 172.21.15.162:0/1982393684 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8df80af640 0x7f8df8063340 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7f8df8067980 tx=0x7f8df0007f20).stop 2020-11-28T21:50:06.239 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.239+0000 7f8df759e700 1 -- 172.21.15.162:0/1982393684 shutdown_connections 2020-11-28T21:50:06.239 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.239+0000 7f8df759e700 1 --2- 172.21.15.162:0/1982393684 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8ddc055440 0x7f8ddc057900 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:06.239 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.239+0000 7f8df759e700 1 --2- 172.21.15.162:0/1982393684 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8df80af640 0x7f8df8063340 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:06.239 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.239+0000 7f8df759e700 1 --2- 172.21.15.162:0/1982393684 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8df805cbd0 0x7f8df80ad2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:06.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.239+0000 7f8df759e700 1 --2- 172.21.15.162:0/1982393684 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8df8060cf0 0x7f8df814deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:06.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.239+0000 7f8df759e700 1 -- 172.21.15.162:0/1982393684 >> 172.21.15.162:0/1982393684 conn(0x7f8df805b970 msgr2=0x7f8df8143a00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:06.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.239+0000 7f8df759e700 1 -- 172.21.15.162:0/1982393684 shutdown_connections 2020-11-28T21:50:06.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.239+0000 7f8df759e700 1 -- 172.21.15.162:0/1982393684 wait complete. 2020-11-28T21:50:06.250 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2020-11-28T21:50:06.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.363+0000 7fb0bdd20700 1 Processor -- start 2020-11-28T21:50:06.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.363+0000 7fb0bdd20700 1 -- start start 2020-11-28T21:50:06.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.363+0000 7fb0bdd20700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fb0b80b5730 0x7fb0b814ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:06.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.363+0000 7fb0bdd20700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb0b814b140 0x7fb0b8147660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:06.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.363+0000 7fb0bdd20700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb0b80b1120 0x7fb0b80af820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:06.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.363+0000 7fb0bdd20700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0b80a8690 con 0x7fb0b80b5730 2020-11-28T21:50:06.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.363+0000 7fb0bdd20700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0b80a88c0 con 0x7fb0b80b1120 2020-11-28T21:50:06.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.363+0000 7fb0bdd20700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b81471c0 con 0x7fb0b814b140 2020-11-28T21:50:06.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.363+0000 7fb0b7fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb0b80b1120 0x7fb0b80af820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:06.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.363+0000 7fb0b7fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb0b80b1120 0x7fb0b80af820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40006/0 (socket says 172.21.15.162:40006) 2020-11-28T21:50:06.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.363+0000 7fb0b7fff700 1 -- 172.21.15.162:0/1016251305 learned_addr learned my addr 172.21.15.162:0/1016251305 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:06.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.367+0000 7fb0b6ffd700 1 --2- 172.21.15.162:0/1016251305 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb0b814b140 0x7fb0b8147660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:06.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.367+0000 7fb0b77fe700 1 --2- 172.21.15.162:0/1016251305 >> v2:172.21.15.118:3300/0 conn(0x7fb0b80b5730 0x7fb0b814ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:06.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.367+0000 7fb0b7fff700 1 -- 172.21.15.162:0/1016251305 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb0b814b140 msgr2=0x7fb0b8147660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:06.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.367+0000 7fb0b7fff700 1 --2- 172.21.15.162:0/1016251305 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb0b814b140 0x7fb0b8147660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:06.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.367+0000 7fb0b7fff700 1 -- 172.21.15.162:0/1016251305 >> v2:172.21.15.118:3300/0 conn(0x7fb0b80b5730 msgr2=0x7fb0b814ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:06.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.367+0000 7fb0b7fff700 1 --2- 172.21.15.162:0/1016251305 >> v2:172.21.15.118:3300/0 conn(0x7fb0b80b5730 0x7fb0b814ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:06.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.367+0000 7fb0b7fff700 1 -- 172.21.15.162:0/1016251305 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b8146ef0 con 0x7fb0b80b1120 2020-11-28T21:50:06.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.367+0000 7fb0b7fff700 1 --2- 172.21.15.162:0/1016251305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0b80b1120 0x7fb0b80af820 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7fb0b0002f30 tx=0x7fb0b0006e80).ready entity=mon.1 client_cookie=f5b9695ae885d37e server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:06.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.367+0000 7fb0b67fc700 1 -- 172.21.15.162:0/1016251305 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb0b0009640 con 0x7fb0b80b1120 2020-11-28T21:50:06.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.367+0000 7fb0b67fc700 1 -- 172.21.15.162:0/1016251305 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b00097a0 con 0x7fb0b80b1120 2020-11-28T21:50:06.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.367+0000 7fb0b67fc700 1 -- 172.21.15.162:0/1016251305 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb0b0009b00 con 0x7fb0b80b1120 2020-11-28T21:50:06.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.367+0000 7fb0bdd20700 1 -- 172.21.15.162:0/1016251305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0b80b1120 msgr2=0x7fb0b80af820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:06.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.367+0000 7fb0bdd20700 1 --2- 172.21.15.162:0/1016251305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0b80b1120 0x7fb0b80af820 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7fb0b0002f30 tx=0x7fb0b0006e80).stop 2020-11-28T21:50:06.373 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.367+0000 7fb0bdd20700 1 -- 172.21.15.162:0/1016251305 shutdown_connections 2020-11-28T21:50:06.373 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.367+0000 7fb0bdd20700 1 --2- 172.21.15.162:0/1016251305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0b80b1120 0x7fb0b80af820 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:06.373 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.367+0000 7fb0bdd20700 1 --2- 172.21.15.162:0/1016251305 >> v2:172.21.15.118:3300/0 conn(0x7fb0b80b5730 0x7fb0b814ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:06.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.367+0000 7fb0bdd20700 1 --2- 172.21.15.162:0/1016251305 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb0b814b140 0x7fb0b8147660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:06.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.367+0000 7fb0bdd20700 1 -- 172.21.15.162:0/1016251305 >> 172.21.15.162:0/1016251305 conn(0x7fb0b805c3f0 msgr2=0x7fb0b80b4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:06.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.367+0000 7fb0bdd20700 1 -- 172.21.15.162:0/1016251305 shutdown_connections 2020-11-28T21:50:06.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.367+0000 7fb0bdd20700 1 -- 172.21.15.162:0/1016251305 wait complete. 2020-11-28T21:50:06.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.371+0000 7fb0bdd20700 1 Processor -- start 2020-11-28T21:50:06.375 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.371+0000 7fb0bdd20700 1 -- start start 2020-11-28T21:50:06.375 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.371+0000 7fb0bdd20700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0b80b1120 0x7fb0b809a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:06.375 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.371+0000 7fb0bdd20700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb0b80b5730 0x7fb0b80ac650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:06.375 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.371+0000 7fb0bdd20700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb0b814b140 0x7fb0b80acb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:06.375 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.371+0000 7fb0bdd20700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b814dcc0 con 0x7fb0b814b140 2020-11-28T21:50:06.376 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.371+0000 7fb0bdd20700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b814d700 con 0x7fb0b80b1120 2020-11-28T21:50:06.376 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.371+0000 7fb0bdd20700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b814def0 con 0x7fb0b80b5730 2020-11-28T21:50:06.376 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.371+0000 7fb0b77fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0b80b1120 0x7fb0b809a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:06.376 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.371+0000 7fb0b77fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0b80b1120 0x7fb0b809a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40012/0 (socket says 172.21.15.162:40012) 2020-11-28T21:50:06.376 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.371+0000 7fb0b77fe700 1 -- 172.21.15.162:0/455301008 learned_addr learned my addr 172.21.15.162:0/455301008 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:06.377 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.371+0000 7fb0b6ffd700 1 --2- 172.21.15.162:0/455301008 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb0b80b5730 0x7fb0b80ac650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:06.377 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.371+0000 7fb0b77fe700 1 -- 172.21.15.162:0/455301008 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb0b80b5730 msgr2=0x7fb0b80ac650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:06.377 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.371+0000 7fb0b77fe700 1 --2- 172.21.15.162:0/455301008 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb0b80b5730 0x7fb0b80ac650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:06.377 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.371+0000 7fb0b77fe700 1 -- 172.21.15.162:0/455301008 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb0b814b140 msgr2=0x7fb0b80acb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:06.378 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.371+0000 7fb0b7fff700 1 --2- 172.21.15.162:0/455301008 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb0b814b140 0x7fb0b80acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:06.378 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.371+0000 7fb0b77fe700 1 --2- 172.21.15.162:0/455301008 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb0b814b140 0x7fb0b80acb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:06.378 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.371+0000 7fb0b77fe700 1 -- 172.21.15.162:0/455301008 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b80ac1b0 con 0x7fb0b80b1120 2020-11-28T21:50:06.378 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.371+0000 7fb0b6ffd700 1 --2- 172.21.15.162:0/455301008 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb0b80b5730 0x7fb0b80ac650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:50:06.378 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.371+0000 7fb0b7fff700 1 --2- 172.21.15.162:0/455301008 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb0b814b140 0x7fb0b80acb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:50:06.379 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.371+0000 7fb0b77fe700 1 --2- 172.21.15.162:0/455301008 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0b80b1120 0x7fb0b809a6c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7fb0b814ca40 tx=0x7fb0a80091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:06.379 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.371+0000 7fb0b4ff9700 1 -- 172.21.15.162:0/455301008 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb0a800cc00 con 0x7fb0b80b1120 2020-11-28T21:50:06.379 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.371+0000 7fb0b4ff9700 1 -- 172.21.15.162:0/455301008 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a800cd60 con 0x7fb0b80b1120 2020-11-28T21:50:06.380 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.371+0000 7fb0bdd20700 1 -- 172.21.15.162:0/455301008 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0b814cba0 con 0x7fb0b80b1120 2020-11-28T21:50:06.380 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.371+0000 7fb0b4ff9700 1 -- 172.21.15.162:0/455301008 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb0a80125e0 con 0x7fb0b80b1120 2020-11-28T21:50:06.380 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.371+0000 7fb0bdd20700 1 -- 172.21.15.162:0/455301008 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0b8155fb0 con 0x7fb0b80b1120 2020-11-28T21:50:06.380 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.375+0000 7fb0bdd20700 1 -- 172.21.15.162:0/455301008 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0b80ac340 con 0x7fb0b80b1120 2020-11-28T21:50:06.380 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.375+0000 7fb0b4ff9700 1 -- 172.21.15.162:0/455301008 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb0a8022070 con 0x7fb0b80b1120 2020-11-28T21:50:06.381 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.375+0000 7fb0b4ff9700 1 --2- 172.21.15.162:0/455301008 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb09c055330 0x7fb09c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:06.381 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.375+0000 7fb0b4ff9700 1 -- 172.21.15.162:0/455301008 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb0a806e8a0 con 0x7fb0b80b1120 2020-11-28T21:50:06.382 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.375+0000 7fb0b6ffd700 1 --2- 172.21.15.162:0/455301008 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb09c055330 0x7fb09c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:06.382 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.375+0000 7fb0b6ffd700 1 --2- 172.21.15.162:0/455301008 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb09c055330 0x7fb09c0577f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7fb0a000b000 tx=0x7fb0a0008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:06.383 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.383+0000 7fb0b4ff9700 1 -- 172.21.15.162:0/455301008 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb0a8044b00 con 0x7fb0b80b1120 2020-11-28T21:50:06.568 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.571+0000 7fb0bdd20700 1 -- 172.21.15.162:0/455301008 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb0b8057050 con 0x7fb0b80b1120 2020-11-28T21:50:06.662 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.663+0000 7fb0b4ff9700 1 -- 172.21.15.162:0/455301008 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v21) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb0a8048120 con 0x7fb0b80b1120 2020-11-28T21:50:06.665 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.667+0000 7fb0bdd20700 1 -- 172.21.15.162:0/455301008 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb09c055330 msgr2=0x7fb09c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:06.665 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.667+0000 7fb0bdd20700 1 --2- 172.21.15.162:0/455301008 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb09c055330 0x7fb09c0577f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7fb0a000b000 tx=0x7fb0a0008040).stop 2020-11-28T21:50:06.665 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.667+0000 7fb0bdd20700 1 -- 172.21.15.162:0/455301008 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0b80b1120 msgr2=0x7fb0b809a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:06.666 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.667+0000 7fb0bdd20700 1 --2- 172.21.15.162:0/455301008 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0b80b1120 0x7fb0b809a6c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7fb0b814ca40 tx=0x7fb0a80091a0).stop 2020-11-28T21:50:06.667 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.667+0000 7fb0bdd20700 1 -- 172.21.15.162:0/455301008 shutdown_connections 2020-11-28T21:50:06.668 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.667+0000 7fb0bdd20700 1 --2- 172.21.15.162:0/455301008 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb09c055330 0x7fb09c0577f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:06.668 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.667+0000 7fb0bdd20700 1 --2- 172.21.15.162:0/455301008 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb0b814b140 0x7fb0b80acb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:06.669 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.667+0000 7fb0bdd20700 1 --2- 172.21.15.162:0/455301008 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0b80b1120 0x7fb0b809a6c0 secure :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 rx=0x7fb0b814ca40 tx=0x7fb0a80091a0).stop 2020-11-28T21:50:06.669 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.667+0000 7fb0bdd20700 1 --2- 172.21.15.162:0/455301008 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb0b80b5730 0x7fb0b80ac650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:06.669 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.667+0000 7fb0bdd20700 1 -- 172.21.15.162:0/455301008 >> 172.21.15.162:0/455301008 conn(0x7fb0b805c3f0 msgr2=0x7fb0b80b0950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:06.670 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.667+0000 7fb0bdd20700 1 -- 172.21.15.162:0/455301008 shutdown_connections 2020-11-28T21:50:06.670 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.667+0000 7fb0bdd20700 1 -- 172.21.15.162:0/455301008 wait complete. 2020-11-28T21:50:06.681 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:50:06.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.795+0000 7f3dcafa5700 1 Processor -- start 2020-11-28T21:50:06.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.795+0000 7f3dcafa5700 1 -- start start 2020-11-28T21:50:06.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.795+0000 7f3dcafa5700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f3dc40b1870 0x7f3dc4142f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:06.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.795+0000 7f3dcafa5700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3dc4143440 0x7f3dc40b0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:06.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.795+0000 7f3dcafa5700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3dc40b4720 0x7f3dc40b0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:06.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.799+0000 7f3dcafa5700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dc40615a0 con 0x7f3dc40b1870 2020-11-28T21:50:06.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.799+0000 7f3dcafa5700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dc40afff0 con 0x7f3dc40b4720 2020-11-28T21:50:06.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.799+0000 7f3dcafa5700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc40b1c50 con 0x7f3dc4143440 2020-11-28T21:50:06.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.799+0000 7f3dca7a4700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3dc40b4720 0x7f3dc40b0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:06.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.799+0000 7f3dc97a2700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3dc4143440 0x7f3dc40b0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:06.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.799+0000 7f3dc97a2700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3dc4143440 0x7f3dc40b0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:37246/0 (socket says 172.21.15.162:37246) 2020-11-28T21:50:06.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.799+0000 7f3dc9fa3700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f3dc40b1870 0x7f3dc4142f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:06.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.799+0000 7f3dca7a4700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3dc40b4720 0x7f3dc40b0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40024/0 (socket says 172.21.15.162:40024) 2020-11-28T21:50:06.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.799+0000 7f3dc97a2700 1 -- 172.21.15.162:0/4288911129 learned_addr learned my addr 172.21.15.162:0/4288911129 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:06.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.799+0000 7f3dc9fa3700 1 -- 172.21.15.162:0/4288911129 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3dc4143440 msgr2=0x7f3dc40b0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:06.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.799+0000 7f3dc9fa3700 1 --2- 172.21.15.162:0/4288911129 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3dc4143440 0x7f3dc40b0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:06.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.799+0000 7f3dc9fa3700 1 -- 172.21.15.162:0/4288911129 >> v2:172.21.15.162:3300/0 conn(0x7f3dc40b4720 msgr2=0x7f3dc40b0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:06.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.799+0000 7f3dc9fa3700 1 --2- 172.21.15.162:0/4288911129 >> v2:172.21.15.162:3300/0 conn(0x7f3dc40b4720 0x7f3dc40b0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:06.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.799+0000 7f3dc9fa3700 1 -- 172.21.15.162:0/4288911129 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc4060a60 con 0x7f3dc40b1870 2020-11-28T21:50:06.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.799+0000 7f3dc97a2700 1 --2- 172.21.15.162:0/4288911129 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3dc4143440 0x7f3dc40b0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:50:06.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.799+0000 7f3dca7a4700 1 --2- 172.21.15.162:0/4288911129 >> v2:172.21.15.162:3300/0 conn(0x7f3dc40b4720 0x7f3dc40b0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:50:06.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.799+0000 7f3dc9fa3700 1 --2- 172.21.15.162:0/4288911129 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3dc40b1870 0x7f3dc4142f50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f3db4002f30 tx=0x7f3db4006dc0).ready entity=mon.0 client_cookie=dedf7a44482dda67 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:06.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.799+0000 7f3dc8fa1700 1 -- 172.21.15.162:0/4288911129 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3db4009650 con 0x7f3dc40b1870 2020-11-28T21:50:06.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.799+0000 7f3dc8fa1700 1 -- 172.21.15.162:0/4288911129 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db40097b0 con 0x7f3dc40b1870 2020-11-28T21:50:06.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.799+0000 7f3dcafa5700 1 -- 172.21.15.162:0/4288911129 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3dc40b1870 msgr2=0x7f3dc4142f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:06.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.799+0000 7f3dcafa5700 1 --2- 172.21.15.162:0/4288911129 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3dc40b1870 0x7f3dc4142f50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f3db4002f30 tx=0x7f3db4006dc0).stop 2020-11-28T21:50:06.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.799+0000 7f3dcafa5700 1 -- 172.21.15.162:0/4288911129 shutdown_connections 2020-11-28T21:50:06.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.799+0000 7f3dcafa5700 1 --2- 172.21.15.162:0/4288911129 >> v2:172.21.15.162:3300/0 conn(0x7f3dc40b4720 0x7f3dc40b0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:06.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.799+0000 7f3dcafa5700 1 --2- 172.21.15.162:0/4288911129 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3dc40b1870 0x7f3dc4142f50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:06.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.799+0000 7f3dcafa5700 1 --2- 172.21.15.162:0/4288911129 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3dc4143440 0x7f3dc40b0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:06.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.799+0000 7f3dcafa5700 1 -- 172.21.15.162:0/4288911129 >> 172.21.15.162:0/4288911129 conn(0x7f3dc40b54f0 msgr2=0x7f3dc40b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:06.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.799+0000 7f3dcafa5700 1 -- 172.21.15.162:0/4288911129 shutdown_connections 2020-11-28T21:50:06.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.799+0000 7f3dcafa5700 1 -- 172.21.15.162:0/4288911129 wait complete. 2020-11-28T21:50:06.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.803+0000 7f3dcafa5700 1 Processor -- start 2020-11-28T21:50:06.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.803+0000 7f3dcafa5700 1 -- start start 2020-11-28T21:50:06.812 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.803+0000 7f3dcafa5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3dc40b1870 0x7f3dc413c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:06.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.803+0000 7f3dcafa5700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3dc40b4720 0x7f3dc4151810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:06.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.803+0000 7f3dcafa5700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3dc4143440 0x7f3dc4151d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:06.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.803+0000 7f3dcafa5700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc4149f90 con 0x7f3dc40b4720 2020-11-28T21:50:06.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.803+0000 7f3dcafa5700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc4149d20 con 0x7f3dc40b1870 2020-11-28T21:50:06.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.803+0000 7f3dcafa5700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc4099740 con 0x7f3dc4143440 2020-11-28T21:50:06.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.803+0000 7f3dc9fa3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3dc40b1870 0x7f3dc413c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:06.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.803+0000 7f3dc9fa3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3dc40b1870 0x7f3dc413c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40026/0 (socket says 172.21.15.162:40026) 2020-11-28T21:50:06.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.803+0000 7f3dc9fa3700 1 -- 172.21.15.162:0/964362200 learned_addr learned my addr 172.21.15.162:0/964362200 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:06.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.803+0000 7f3dc97a2700 1 --2- 172.21.15.162:0/964362200 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3dc40b4720 0x7f3dc4151810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:06.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.803+0000 7f3dca7a4700 1 --2- 172.21.15.162:0/964362200 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3dc4143440 0x7f3dc4151d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:06.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.803+0000 7f3dc9fa3700 1 -- 172.21.15.162:0/964362200 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3dc4143440 msgr2=0x7f3dc4151d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:06.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.803+0000 7f3dc9fa3700 1 --2- 172.21.15.162:0/964362200 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3dc4143440 0x7f3dc4151d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:06.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.803+0000 7f3dc9fa3700 1 -- 172.21.15.162:0/964362200 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3dc40b4720 msgr2=0x7f3dc4151810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:06.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.803+0000 7f3dc9fa3700 1 --2- 172.21.15.162:0/964362200 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3dc40b4720 0x7f3dc4151810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:06.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.803+0000 7f3dc9fa3700 1 -- 172.21.15.162:0/964362200 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc40999e0 con 0x7f3dc40b1870 2020-11-28T21:50:06.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.803+0000 7f3dc9fa3700 1 --2- 172.21.15.162:0/964362200 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3dc40b1870 0x7f3dc413c330 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7f3db4002f30 tx=0x7f3db4015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:06.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.803+0000 7f3db2ffd700 1 -- 172.21.15.162:0/964362200 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3db4011ad0 con 0x7f3dc40b1870 2020-11-28T21:50:06.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.803+0000 7f3db2ffd700 1 -- 172.21.15.162:0/964362200 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db4011c30 con 0x7f3dc40b1870 2020-11-28T21:50:06.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.803+0000 7f3db2ffd700 1 -- 172.21.15.162:0/964362200 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3db401d450 con 0x7f3dc40b1870 2020-11-28T21:50:06.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.803+0000 7f3dcafa5700 1 -- 172.21.15.162:0/964362200 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dc4099c80 con 0x7f3dc40b1870 2020-11-28T21:50:06.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.803+0000 7f3dcafa5700 1 -- 172.21.15.162:0/964362200 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc4148a50 con 0x7f3dc40b1870 2020-11-28T21:50:06.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.807+0000 7f3dcafa5700 1 -- 172.21.15.162:0/964362200 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dc4057050 con 0x7f3dc40b1870 2020-11-28T21:50:06.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.807+0000 7f3db2ffd700 1 -- 172.21.15.162:0/964362200 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f3db4024030 con 0x7f3dc40b1870 2020-11-28T21:50:06.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.807+0000 7f3db2ffd700 1 --2- 172.21.15.162:0/964362200 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3da40553b0 0x7f3da4057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:06.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.807+0000 7f3db2ffd700 1 -- 172.21.15.162:0/964362200 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3db406ee30 con 0x7f3dc40b1870 2020-11-28T21:50:06.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.807+0000 7f3dc97a2700 1 --2- 172.21.15.162:0/964362200 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3da40553b0 0x7f3da4057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:06.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.807+0000 7f3dc97a2700 1 --2- 172.21.15.162:0/964362200 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3da40553b0 0x7f3da4057870 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f3dc0008000 tx=0x7f3dc0008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:06.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.811+0000 7f3db2ffd700 1 -- 172.21.15.162:0/964362200 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f3db4045130 con 0x7f3dc40b1870 2020-11-28T21:50:06.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.995+0000 7f3dcafa5700 1 -- 172.21.15.162:0/964362200 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3dc4061a60 con 0x7f3dc40b1870 2020-11-28T21:50:06.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.995+0000 7f3db2ffd700 1 -- 172.21.15.162:0/964362200 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+2501 (secure 0 0 0) 0x7f3db4022070 con 0x7f3dc40b1870 2020-11-28T21:50:06.996 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:50:06.996 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":21,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/1245801281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1245801281},{"type":"v1","addr":"172.21.15.118:6833","nonce":1245801281}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":18},{"gid":5536,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6835/3629429916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3629429916},{"type":"v1","addr":"172.21.15.162:6835","nonce":3629429916}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":19}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:50:05.655172+0000","modified":"2020-11-28T21:50:06.657968+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5551},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5551":{"gid":5551,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/1902721947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1902721947},{"type":"v1","addr":"172.21.15.162:6837","nonce":1902721947}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-28T21:50:06.997 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.999+0000 7f3dcafa5700 1 -- 172.21.15.162:0/964362200 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3da40553b0 msgr2=0x7f3da4057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:06.998 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.999+0000 7f3dcafa5700 1 --2- 172.21.15.162:0/964362200 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3da40553b0 0x7f3da4057870 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f3dc0008000 tx=0x7f3dc0008040).stop 2020-11-28T21:50:06.998 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.999+0000 7f3dcafa5700 1 -- 172.21.15.162:0/964362200 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3dc40b1870 msgr2=0x7f3dc413c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:06.998 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.999+0000 7f3dcafa5700 1 --2- 172.21.15.162:0/964362200 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3dc40b1870 0x7f3dc413c330 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7f3db4002f30 tx=0x7f3db4015000).stop 2020-11-28T21:50:06.998 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.999+0000 7f3dcafa5700 1 -- 172.21.15.162:0/964362200 shutdown_connections 2020-11-28T21:50:06.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.999+0000 7f3dcafa5700 1 --2- 172.21.15.162:0/964362200 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3da40553b0 0x7f3da4057870 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:06.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.999+0000 7f3dcafa5700 1 --2- 172.21.15.162:0/964362200 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3dc4143440 0x7f3dc4151d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:06.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.999+0000 7f3dcafa5700 1 --2- 172.21.15.162:0/964362200 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3dc40b1870 0x7f3dc413c330 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:06.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.999+0000 7f3dcafa5700 1 --2- 172.21.15.162:0/964362200 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3dc40b4720 0x7f3dc4151810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:06.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.999+0000 7f3dcafa5700 1 -- 172.21.15.162:0/964362200 >> 172.21.15.162:0/964362200 conn(0x7f3dc40b54f0 msgr2=0x7f3dc405b970 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:07.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.999+0000 7f3dcafa5700 1 -- 172.21.15.162:0/964362200 shutdown_connections 2020-11-28T21:50:07.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:06.999+0000 7f3dcafa5700 1 -- 172.21.15.162:0/964362200 wait complete. 2020-11-28T21:50:07.000 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 21 2020-11-28T21:50:07.010 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:50:07.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.139+0000 7fee16c28700 1 Processor -- start 2020-11-28T21:50:07.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.139+0000 7fee16c28700 1 -- start start 2020-11-28T21:50:07.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.139+0000 7fee16c28700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fee100603d0 0x7fee100b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:07.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.139+0000 7fee16c28700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fee100b4c70 0x7fee10143020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:07.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.139+0000 7fee16c28700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee100b4110 con 0x7fee10142a00 2020-11-28T21:50:07.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.139+0000 7fee16c28700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee10060f60 con 0x7fee100603d0 2020-11-28T21:50:07.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.139+0000 7fee16c28700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee100b43f0 con 0x7fee100b4c70 2020-11-28T21:50:07.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.139+0000 7fee16427700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fee100b4c70 0x7fee10143020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:07.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.139+0000 7fee15425700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fee10142a00 0x7fee1009cfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:46880/0 (socket says 172.21.15.162:46880) 2020-11-28T21:50:07.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.139+0000 7fee16427700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fee100b4c70 0x7fee10143020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40038/0 (socket says 172.21.15.162:40038) 2020-11-28T21:50:07.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.139+0000 7fee16427700 1 -- 172.21.15.162:0/1624332217 learned_addr learned my addr 172.21.15.162:0/1624332217 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:07.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.139+0000 7fee15c26700 1 --2- 172.21.15.162:0/1624332217 >> v2:172.21.15.118:3300/0 conn(0x7fee100603d0 0x7fee100b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:07.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.139+0000 7fee16427700 1 -- 172.21.15.162:0/1624332217 >> v2:172.21.15.118:3300/0 conn(0x7fee100603d0 msgr2=0x7fee100b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:07.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.139+0000 7fee16427700 1 --2- 172.21.15.162:0/1624332217 >> v2:172.21.15.118:3300/0 conn(0x7fee100603d0 0x7fee100b3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:07.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.139+0000 7fee16427700 1 -- 172.21.15.162:0/1624332217 >> v1:172.21.15.162:6789/0 conn(0x7fee10142a00 legacy=0x7fee1009cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:07.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.139+0000 7fee16427700 1 -- 172.21.15.162:0/1624332217 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee100b4690 con 0x7fee100b4c70 2020-11-28T21:50:07.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.139+0000 7fee16427700 1 --2- 172.21.15.162:0/1624332217 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee100b4c70 0x7fee10143020 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7fee080032b0 tx=0x7fee08009270).ready entity=mon.1 client_cookie=2286ea0d84c5371e server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:07.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.139+0000 7fee14c24700 1 -- 172.21.15.162:0/1624332217 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee0800bc00 con 0x7fee100b4c70 2020-11-28T21:50:07.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.139+0000 7fee14c24700 1 -- 172.21.15.162:0/1624332217 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee0800bd60 con 0x7fee100b4c70 2020-11-28T21:50:07.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.139+0000 7fee14c24700 1 -- 172.21.15.162:0/1624332217 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee08011400 con 0x7fee100b4c70 2020-11-28T21:50:07.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.139+0000 7fee16c28700 1 -- 172.21.15.162:0/1624332217 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee100b4c70 msgr2=0x7fee10143020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:07.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.139+0000 7fee16c28700 1 --2- 172.21.15.162:0/1624332217 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee100b4c70 0x7fee10143020 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7fee080032b0 tx=0x7fee08009270).stop 2020-11-28T21:50:07.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.139+0000 7fee16c28700 1 -- 172.21.15.162:0/1624332217 shutdown_connections 2020-11-28T21:50:07.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.139+0000 7fee16c28700 1 --2- 172.21.15.162:0/1624332217 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee100b4c70 0x7fee10143020 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:07.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.139+0000 7fee16c28700 1 --2- 172.21.15.162:0/1624332217 >> v2:172.21.15.118:3300/0 conn(0x7fee100603d0 0x7fee100b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:07.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.139+0000 7fee16c28700 1 -- 172.21.15.162:0/1624332217 >> 172.21.15.162:0/1624332217 conn(0x7fee10066a40 msgr2=0x7fee10066e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:07.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.139+0000 7fee16c28700 1 -- 172.21.15.162:0/1624332217 shutdown_connections 2020-11-28T21:50:07.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.139+0000 7fee16c28700 1 -- 172.21.15.162:0/1624332217 wait complete. 2020-11-28T21:50:07.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.143+0000 7fee16c28700 1 Processor -- start 2020-11-28T21:50:07.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.143+0000 7fee16c28700 1 -- start start 2020-11-28T21:50:07.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.147+0000 7fee16c28700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee100603d0 0x7fee10140840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:07.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.147+0000 7fee16c28700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee100b4c70 0x7fee10154f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:07.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.147+0000 7fee16c28700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee100b0940 0x7fee10155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:07.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.147+0000 7fee16c28700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1005d1b0 con 0x7fee100b0940 2020-11-28T21:50:07.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.147+0000 7fee16c28700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee100b0de0 con 0x7fee100b4c70 2020-11-28T21:50:07.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.147+0000 7fee16c28700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee100ae2d0 con 0x7fee100603d0 2020-11-28T21:50:07.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.147+0000 7fee15425700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee100b4c70 0x7fee10154f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:07.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.147+0000 7fee15425700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee100b4c70 0x7fee10154f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40042/0 (socket says 172.21.15.162:40042) 2020-11-28T21:50:07.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.147+0000 7fee15425700 1 -- 172.21.15.162:0/3483227942 learned_addr learned my addr 172.21.15.162:0/3483227942 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:07.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.147+0000 7fee15c26700 1 --2- 172.21.15.162:0/3483227942 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee100603d0 0x7fee10140840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:07.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.147+0000 7fee16427700 1 --2- 172.21.15.162:0/3483227942 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee100b0940 0x7fee10155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:07.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.147+0000 7fee15425700 1 -- 172.21.15.162:0/3483227942 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee100603d0 msgr2=0x7fee10140840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:07.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.147+0000 7fee15425700 1 --2- 172.21.15.162:0/3483227942 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee100603d0 0x7fee10140840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:07.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.147+0000 7fee15425700 1 -- 172.21.15.162:0/3483227942 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee100b0940 msgr2=0x7fee10155460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:07.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.147+0000 7fee15425700 1 --2- 172.21.15.162:0/3483227942 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee100b0940 0x7fee10155460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:07.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.147+0000 7fee15425700 1 -- 172.21.15.162:0/3483227942 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee100ae570 con 0x7fee100b4c70 2020-11-28T21:50:07.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.147+0000 7fee15425700 1 --2- 172.21.15.162:0/3483227942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee100b4c70 0x7fee10154f70 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7fee100b9c10 tx=0x7fee00007c30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:07.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.147+0000 7fedfeffd700 1 -- 172.21.15.162:0/3483227942 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee000072e0 con 0x7fee100b4c70 2020-11-28T21:50:07.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.147+0000 7fedfeffd700 1 -- 172.21.15.162:0/3483227942 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee00005520 con 0x7fee100b4c70 2020-11-28T21:50:07.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.147+0000 7fee16c28700 1 -- 172.21.15.162:0/3483227942 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee100aea10 con 0x7fee100b4c70 2020-11-28T21:50:07.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.147+0000 7fedfeffd700 1 -- 172.21.15.162:0/3483227942 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee00007440 con 0x7fee100b4c70 2020-11-28T21:50:07.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.147+0000 7fee16c28700 1 -- 172.21.15.162:0/3483227942 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee100b1900 con 0x7fee100b4c70 2020-11-28T21:50:07.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.147+0000 7fee16c28700 1 -- 172.21.15.162:0/3483227942 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee100ae700 con 0x7fee100b4c70 2020-11-28T21:50:07.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.147+0000 7fedfeffd700 1 -- 172.21.15.162:0/3483227942 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fee00010070 con 0x7fee100b4c70 2020-11-28T21:50:07.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.151+0000 7fedfeffd700 1 --2- 172.21.15.162:0/3483227942 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fedf00554e0 0x7fedf00579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:07.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.151+0000 7fedfeffd700 1 -- 172.21.15.162:0/3483227942 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fee0006d3f0 con 0x7fee100b4c70 2020-11-28T21:50:07.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.151+0000 7fee15c26700 1 --2- 172.21.15.162:0/3483227942 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fedf00554e0 0x7fedf00579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:07.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.151+0000 7fee15c26700 1 --2- 172.21.15.162:0/3483227942 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fedf00554e0 0x7fedf00579a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7fee0c00c000 tx=0x7fee0c008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:07.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.155+0000 7fedfeffd700 1 -- 172.21.15.162:0/3483227942 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fee00043690 con 0x7fee100b4c70 2020-11-28T21:50:07.295 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.295+0000 7fee16c28700 1 -- 172.21.15.162:0/3483227942 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee10057050 con 0x7fee100b4c70 2020-11-28T21:50:07.561 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:50:07.562 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.299+0000 7fedfeffd700 1 -- 172.21.15.162:0/3483227942 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16149 (secure 0 0 0) 0x7fee00046cb0 con 0x7fee100b4c70 2020-11-28T21:50:07.562 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.299+0000 7fee16c28700 1 -- 172.21.15.162:0/3483227942 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fedf00554e0 msgr2=0x7fedf00579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:07.562 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.299+0000 7fee16c28700 1 --2- 172.21.15.162:0/3483227942 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fedf00554e0 0x7fedf00579a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7fee0c00c000 tx=0x7fee0c008040).stop 2020-11-28T21:50:07.562 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.299+0000 7fee16c28700 1 -- 172.21.15.162:0/3483227942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee100b4c70 msgr2=0x7fee10154f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:07.563 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.299+0000 7fee16c28700 1 --2- 172.21.15.162:0/3483227942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee100b4c70 0x7fee10154f70 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7fee100b9c10 tx=0x7fee00007c30).stop 2020-11-28T21:50:07.563 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.299+0000 7fee16c28700 1 -- 172.21.15.162:0/3483227942 shutdown_connections 2020-11-28T21:50:07.563 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.299+0000 7fee16c28700 1 --2- 172.21.15.162:0/3483227942 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fedf00554e0 0x7fedf00579a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:07.563 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.299+0000 7fee16c28700 1 --2- 172.21.15.162:0/3483227942 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee100603d0 0x7fee10140840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:07.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.299+0000 7fee16c28700 1 --2- 172.21.15.162:0/3483227942 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee100b0940 0x7fee10155460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:07.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.299+0000 7fee16c28700 1 --2- 172.21.15.162:0/3483227942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee100b4c70 0x7fee10154f70 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:07.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.299+0000 7fee16c28700 1 -- 172.21.15.162:0/3483227942 >> 172.21.15.162:0/3483227942 conn(0x7fee10066a40 msgr2=0x7fee10141770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:07.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.299+0000 7fee16c28700 1 -- 172.21.15.162:0/3483227942 shutdown_connections 2020-11-28T21:50:07.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.299+0000 7fee16c28700 1 -- 172.21.15.162:0/3483227942 wait complete. 2020-11-28T21:50:07.565 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:50:07.565 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":55,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:50:06.658620+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:50:03.211406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2020-11-28T21:50:04.511075+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:50:07.566 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-11-28T21:50:07.574 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:50:07.584 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:50:07.645 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:50:07.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.707+0000 7faf01d40700 1 Processor -- start 2020-11-28T21:50:07.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.707+0000 7faf01d40700 1 -- start start 2020-11-28T21:50:07.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.711+0000 7faf01d40700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7faefc05fa90 0x7faefc15a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:07.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.711+0000 7faf01d40700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faefc0ab0b0 con 0x7faefc0b2e80 2020-11-28T21:50:07.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.711+0000 7faf01d40700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faefc0b3260 con 0x7faefc0b2560 2020-11-28T21:50:07.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.711+0000 7faf01d40700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faefc0ab2e0 con 0x7faefc05fa90 2020-11-28T21:50:07.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.711+0000 7faf00d3e700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7faefc0b2560 0x7faefc156ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:46894/0 (socket says 172.21.15.162:46894) 2020-11-28T21:50:07.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.711+0000 7faf00d3e700 1 -- 172.21.15.162:0/3403680072 learned_addr learned my addr 172.21.15.162:0/3403680072 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:50:07.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.711+0000 7faf02fa2700 1 --2- 172.21.15.162:0/3403680072 >> v2:172.21.15.118:3300/0 conn(0x7faefc05fa90 0x7faefc15a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:07.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.711+0000 7faef3fff700 1 -- 172.21.15.162:0/3403680072 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1236165150 0 0) 0x7faefc0b3260 con 0x7faefc0b2560 2020-11-28T21:50:07.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.711+0000 7faef3fff700 1 -- 172.21.15.162:0/3403680072 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faee8003230 con 0x7faefc0b2560 2020-11-28T21:50:07.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.711+0000 7faf02fa2700 1 -- 172.21.15.162:0/3403680072 >> v1:172.21.15.162:6789/0 conn(0x7faefc0b2560 legacy=0x7faefc156ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:07.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.711+0000 7faf02fa2700 1 -- 172.21.15.162:0/3403680072 >> v1:172.21.15.118:6789/0 conn(0x7faefc0b2e80 legacy=0x7faefc05f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:07.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.711+0000 7faf02fa2700 1 -- 172.21.15.162:0/3403680072 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faefc0256d0 con 0x7faefc05fa90 2020-11-28T21:50:07.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.711+0000 7faf02fa2700 1 --2- 172.21.15.162:0/3403680072 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faefc05fa90 0x7faefc15a5f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7faef40032b0 tx=0x7faef40091a0).ready entity=mon.0 client_cookie=b684f8eacdd2fd57 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:07.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.711+0000 7faef3fff700 1 -- 172.21.15.162:0/3403680072 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faef400b9a0 con 0x7faefc05fa90 2020-11-28T21:50:07.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.711+0000 7faef3fff700 1 -- 172.21.15.162:0/3403680072 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef400bb00 con 0x7faefc05fa90 2020-11-28T21:50:07.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.711+0000 7faef3fff700 1 -- 172.21.15.162:0/3403680072 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faef400bc70 con 0x7faefc05fa90 2020-11-28T21:50:07.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.711+0000 7faf01d40700 1 -- 172.21.15.162:0/3403680072 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faefc05fa90 msgr2=0x7faefc15a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:07.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.711+0000 7faf01d40700 1 --2- 172.21.15.162:0/3403680072 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faefc05fa90 0x7faefc15a5f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7faef40032b0 tx=0x7faef40091a0).stop 2020-11-28T21:50:07.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.711+0000 7faf01d40700 1 -- 172.21.15.162:0/3403680072 shutdown_connections 2020-11-28T21:50:07.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.711+0000 7faf01d40700 1 --2- 172.21.15.162:0/3403680072 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faefc05fa90 0x7faefc15a5f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:07.723 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.715+0000 7faf01d40700 1 -- 172.21.15.162:0/3403680072 >> 172.21.15.162:0/3403680072 conn(0x7faefc063b60 msgr2=0x7faefc063f40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:07.723 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.715+0000 7faf01d40700 1 -- 172.21.15.162:0/3403680072 shutdown_connections 2020-11-28T21:50:07.724 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.715+0000 7faf01d40700 1 -- 172.21.15.162:0/3403680072 wait complete. 2020-11-28T21:50:07.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.715+0000 7faf01d40700 1 Processor -- start 2020-11-28T21:50:07.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.715+0000 7faf01d40700 1 -- start start 2020-11-28T21:50:07.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.715+0000 7faf01d40700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faefc05fa90 0x7faefc1302a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:07.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.715+0000 7faf01d40700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faefc0b2560 0x7faefc0657d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:07.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.715+0000 7faf01d40700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faefc0b2e80 0x7faefc141f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:07.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.715+0000 7faf01d40700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faefc141bd0 con 0x7faefc0b2e80 2020-11-28T21:50:07.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.715+0000 7faf01d40700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faefc09bf90 con 0x7faefc05fa90 2020-11-28T21:50:07.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.715+0000 7faf01d40700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faefc066250 con 0x7faefc0b2560 2020-11-28T21:50:07.733 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.715+0000 7faf0153f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faefc05fa90 0x7faefc1302a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:07.733 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.715+0000 7faf0153f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faefc05fa90 0x7faefc1302a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40054/0 (socket says 172.21.15.162:40054) 2020-11-28T21:50:07.734 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.719+0000 7faf0153f700 1 -- 172.21.15.162:0/778868135 learned_addr learned my addr 172.21.15.162:0/778868135 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:07.734 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.719+0000 7faf00d3e700 1 --2- 172.21.15.162:0/778868135 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faefc0b2560 0x7faefc0657d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:07.735 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.719+0000 7faf02fa2700 1 --2- 172.21.15.162:0/778868135 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faefc0b2e80 0x7faefc141f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:07.735 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.719+0000 7faf0153f700 1 -- 172.21.15.162:0/778868135 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faefc0b2560 msgr2=0x7faefc0657d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:07.735 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.719+0000 7faf0153f700 1 --2- 172.21.15.162:0/778868135 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faefc0b2560 0x7faefc0657d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:07.736 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.719+0000 7faf0153f700 1 -- 172.21.15.162:0/778868135 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faefc0b2e80 msgr2=0x7faefc141f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:07.736 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.719+0000 7faf0153f700 1 --2- 172.21.15.162:0/778868135 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faefc0b2e80 0x7faefc141f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:07.737 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.719+0000 7faf0153f700 1 -- 172.21.15.162:0/778868135 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faefc132880 con 0x7faefc05fa90 2020-11-28T21:50:07.737 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.719+0000 7faf00d3e700 1 --2- 172.21.15.162:0/778868135 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faefc0b2560 0x7faefc0657d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:50:07.737 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.719+0000 7faf0153f700 1 --2- 172.21.15.162:0/778868135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faefc05fa90 0x7faefc1302a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7faefc063710 tx=0x7faef8007c20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:07.738 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.719+0000 7faef1ffb700 1 -- 172.21.15.162:0/778868135 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faef8013410 con 0x7faefc05fa90 2020-11-28T21:50:07.738 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.719+0000 7faef1ffb700 1 -- 172.21.15.162:0/778868135 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef8013570 con 0x7faefc05fa90 2020-11-28T21:50:07.739 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.719+0000 7faf01d40700 1 -- 172.21.15.162:0/778868135 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faefc132b20 con 0x7faefc05fa90 2020-11-28T21:50:07.739 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.719+0000 7faef1ffb700 1 -- 172.21.15.162:0/778868135 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faef80138d0 con 0x7faefc05fa90 2020-11-28T21:50:07.739 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.719+0000 7faf01d40700 1 -- 172.21.15.162:0/778868135 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faefc09b440 con 0x7faefc05fa90 2020-11-28T21:50:07.740 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.719+0000 7faf01d40700 1 -- 172.21.15.162:0/778868135 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faefc057050 con 0x7faefc05fa90 2020-11-28T21:50:07.740 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.719+0000 7faef1ffb700 1 -- 172.21.15.162:0/778868135 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7faef8010030 con 0x7faefc05fa90 2020-11-28T21:50:07.740 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.723+0000 7faef1ffb700 1 --2- 172.21.15.162:0/778868135 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7faedc055320 0x7faedc0577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:07.740 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.723+0000 7faf00d3e700 1 --2- 172.21.15.162:0/778868135 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7faedc055320 0x7faedc0577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:07.741 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.723+0000 7faef1ffb700 1 -- 172.21.15.162:0/778868135 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7faef806dcb0 con 0x7faefc05fa90 2020-11-28T21:50:07.741 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.723+0000 7faf00d3e700 1 --2- 172.21.15.162:0/778868135 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7faedc055320 0x7faedc0577e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7faeec00d000 tx=0x7faeec009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:07.741 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.727+0000 7faef1ffb700 1 -- 172.21.15.162:0/778868135 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7faef8043fb0 con 0x7faefc05fa90 2020-11-28T21:50:07.909 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.911+0000 7faf01d40700 1 -- 172.21.15.162:0/778868135 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7faefc068350 con 0x7faefc05fa90 2020-11-28T21:50:07.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.919+0000 7faef1ffb700 1 -- 172.21.15.162:0/778868135 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v15) v1 ==== 184+0+0 (secure 0 0 0) 0x7faef80475d0 con 0x7faefc05fa90 2020-11-28T21:50:07.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.923+0000 7faf01d40700 1 -- 172.21.15.162:0/778868135 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7faedc055320 msgr2=0x7faedc0577e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:07.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.923+0000 7faf01d40700 1 --2- 172.21.15.162:0/778868135 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7faedc055320 0x7faedc0577e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7faeec00d000 tx=0x7faeec009040).stop 2020-11-28T21:50:07.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.923+0000 7faf01d40700 1 -- 172.21.15.162:0/778868135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faefc05fa90 msgr2=0x7faefc1302a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:07.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.923+0000 7faf01d40700 1 --2- 172.21.15.162:0/778868135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faefc05fa90 0x7faefc1302a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7faefc063710 tx=0x7faef8007c20).stop 2020-11-28T21:50:07.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.923+0000 7faf01d40700 1 -- 172.21.15.162:0/778868135 shutdown_connections 2020-11-28T21:50:07.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.923+0000 7faf01d40700 1 --2- 172.21.15.162:0/778868135 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7faedc055320 0x7faedc0577e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:07.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.923+0000 7faf01d40700 1 --2- 172.21.15.162:0/778868135 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faefc0b2e80 0x7faefc141f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:07.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.923+0000 7faf01d40700 1 --2- 172.21.15.162:0/778868135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faefc05fa90 0x7faefc1302a0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:07.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.923+0000 7faf01d40700 1 --2- 172.21.15.162:0/778868135 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faefc0b2560 0x7faefc0657d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:07.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.923+0000 7faf01d40700 1 -- 172.21.15.162:0/778868135 >> 172.21.15.162:0/778868135 conn(0x7faefc063b60 msgr2=0x7faefc063f40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:07.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.923+0000 7faf01d40700 1 -- 172.21.15.162:0/778868135 shutdown_connections 2020-11-28T21:50:07.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:07.923+0000 7faf01d40700 1 -- 172.21.15.162:0/778868135 wait complete. 2020-11-28T21:50:07.925 INFO:teuthology.orchestra.run.smithi162.stderr:updated caps for client.0 2020-11-28T21:50:07.937 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-11-28T21:50:08.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.063+0000 7fbece576700 1 Processor -- start 2020-11-28T21:50:08.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.063+0000 7fbece576700 1 -- start start 2020-11-28T21:50:08.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.063+0000 7fbece576700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbec805fa90 0x7fbec815a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:08.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.063+0000 7fbece576700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbec80ab0b0 con 0x7fbec80b2560 2020-11-28T21:50:08.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.063+0000 7fbece576700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbec80b3260 con 0x7fbec80b2e80 2020-11-28T21:50:08.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.063+0000 7fbece576700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbec80ab2e0 con 0x7fbec805fa90 2020-11-28T21:50:08.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.063+0000 7fbecf7d8700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbec805fa90 0x7fbec815a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:08.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.063+0000 7fbecf7d8700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbec805fa90 0x7fbec815a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40066/0 (socket says 172.21.15.162:40066) 2020-11-28T21:50:08.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.063+0000 7fbecf7d8700 1 -- 172.21.15.162:0/1953741567 learned_addr learned my addr 172.21.15.162:0/1953741567 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:08.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.063+0000 7fbecdd75700 1 --1- 172.21.15.162:0/1953741567 >> v1:172.21.15.162:6789/0 conn(0x7fbec80b2e80 0x7fbec805f380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:46908/0 (socket says 172.21.15.162:46908) 2020-11-28T21:50:08.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.063+0000 7fbecf7d8700 1 -- 172.21.15.162:0/1953741567 >> v1:172.21.15.162:6789/0 conn(0x7fbec80b2e80 legacy=0x7fbec805f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:08.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.063+0000 7fbecf7d8700 1 -- 172.21.15.162:0/1953741567 >> v1:172.21.15.118:6789/0 conn(0x7fbec80b2560 legacy=0x7fbec8156ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:08.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.063+0000 7fbecf7d8700 1 -- 172.21.15.162:0/1953741567 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec80256d0 con 0x7fbec805fa90 2020-11-28T21:50:08.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.063+0000 7fbecf7d8700 1 --2- 172.21.15.162:0/1953741567 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbec805fa90 0x7fbec815a5f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7fbec00032b0 tx=0x7fbec0009270).ready entity=mon.1 client_cookie=d8ec702931c1177f server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:08.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.063+0000 7fbeccd73700 1 -- 172.21.15.162:0/1953741567 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbec000bc00 con 0x7fbec805fa90 2020-11-28T21:50:08.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.063+0000 7fbeccd73700 1 -- 172.21.15.162:0/1953741567 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec000bd60 con 0x7fbec805fa90 2020-11-28T21:50:08.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.063+0000 7fbeccd73700 1 -- 172.21.15.162:0/1953741567 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbec0011400 con 0x7fbec805fa90 2020-11-28T21:50:08.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.063+0000 7fbece576700 1 -- 172.21.15.162:0/1953741567 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbec805fa90 msgr2=0x7fbec815a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:08.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.063+0000 7fbece576700 1 --2- 172.21.15.162:0/1953741567 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbec805fa90 0x7fbec815a5f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7fbec00032b0 tx=0x7fbec0009270).stop 2020-11-28T21:50:08.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.067+0000 7fbece576700 1 -- 172.21.15.162:0/1953741567 shutdown_connections 2020-11-28T21:50:08.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.067+0000 7fbece576700 1 --2- 172.21.15.162:0/1953741567 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbec805fa90 0x7fbec815a5f0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:08.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.067+0000 7fbece576700 1 -- 172.21.15.162:0/1953741567 >> 172.21.15.162:0/1953741567 conn(0x7fbec8063b60 msgr2=0x7fbec8063f40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:08.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.067+0000 7fbece576700 1 -- 172.21.15.162:0/1953741567 shutdown_connections 2020-11-28T21:50:08.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.067+0000 7fbece576700 1 -- 172.21.15.162:0/1953741567 wait complete. 2020-11-28T21:50:08.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.067+0000 7fbece576700 1 Processor -- start 2020-11-28T21:50:08.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.067+0000 7fbece576700 1 -- start start 2020-11-28T21:50:08.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.071+0000 7fbece576700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbec805fa90 0x7fbec80b5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:08.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.071+0000 7fbece576700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbec80b2560 0x7fbec8066810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:08.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.071+0000 7fbece576700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbec80b2e80 0x7fbec8066d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:08.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.071+0000 7fbece576700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec809bb00 con 0x7fbec80b2e80 2020-11-28T21:50:08.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.071+0000 7fbece576700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec813b330 con 0x7fbec805fa90 2020-11-28T21:50:08.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.071+0000 7fbece576700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbec8141a00 con 0x7fbec80b2560 2020-11-28T21:50:08.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.071+0000 7fbecdd75700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbec805fa90 0x7fbec80b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:08.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.071+0000 7fbecdd75700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbec805fa90 0x7fbec80b5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40068/0 (socket says 172.21.15.162:40068) 2020-11-28T21:50:08.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.071+0000 7fbecdd75700 1 -- 172.21.15.162:0/3570859170 learned_addr learned my addr 172.21.15.162:0/3570859170 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:08.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.071+0000 7fbecd574700 1 --2- 172.21.15.162:0/3570859170 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbec80b2560 0x7fbec8066810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:08.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.071+0000 7fbecf7d8700 1 --2- 172.21.15.162:0/3570859170 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbec80b2e80 0x7fbec8066d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:08.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.071+0000 7fbecdd75700 1 -- 172.21.15.162:0/3570859170 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbec80b2560 msgr2=0x7fbec8066810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:08.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.071+0000 7fbecdd75700 1 --2- 172.21.15.162:0/3570859170 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbec80b2560 0x7fbec8066810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:08.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.071+0000 7fbecdd75700 1 -- 172.21.15.162:0/3570859170 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbec80b2e80 msgr2=0x7fbec8066d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:08.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.071+0000 7fbecdd75700 1 --2- 172.21.15.162:0/3570859170 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbec80b2e80 0x7fbec8066d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:08.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.071+0000 7fbecdd75700 1 -- 172.21.15.162:0/3570859170 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec812f3d0 con 0x7fbec805fa90 2020-11-28T21:50:08.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.071+0000 7fbecdd75700 1 --2- 172.21.15.162:0/3570859170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbec805fa90 0x7fbec80b5460 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7fbec80637b0 tx=0x7fbec4007c30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:08.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.071+0000 7fbebe7fc700 1 -- 172.21.15.162:0/3570859170 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbec40072e0 con 0x7fbec805fa90 2020-11-28T21:50:08.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.071+0000 7fbebe7fc700 1 -- 172.21.15.162:0/3570859170 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec4005520 con 0x7fbec805fa90 2020-11-28T21:50:08.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.071+0000 7fbece576700 1 -- 172.21.15.162:0/3570859170 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbec812f670 con 0x7fbec805fa90 2020-11-28T21:50:08.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.071+0000 7fbebe7fc700 1 -- 172.21.15.162:0/3570859170 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbec4007440 con 0x7fbec805fa90 2020-11-28T21:50:08.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.071+0000 7fbece576700 1 -- 172.21.15.162:0/3570859170 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbec80af950 con 0x7fbec805fa90 2020-11-28T21:50:08.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.071+0000 7fbece576700 1 -- 172.21.15.162:0/3570859170 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbec8057050 con 0x7fbec805fa90 2020-11-28T21:50:08.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.071+0000 7fbebe7fc700 1 -- 172.21.15.162:0/3570859170 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fbec4010070 con 0x7fbec805fa90 2020-11-28T21:50:08.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.075+0000 7fbebe7fc700 1 --2- 172.21.15.162:0/3570859170 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbeb0055440 0x7fbeb0057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:08.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.075+0000 7fbebe7fc700 1 -- 172.21.15.162:0/3570859170 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbec406d3d0 con 0x7fbec805fa90 2020-11-28T21:50:08.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.075+0000 7fbecd574700 1 --2- 172.21.15.162:0/3570859170 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbeb0055440 0x7fbeb0057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:08.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.075+0000 7fbecd574700 1 --2- 172.21.15.162:0/3570859170 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbeb0055440 0x7fbeb0057900 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7fbeb800d000 tx=0x7fbeb8007b60).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:08.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.079+0000 7fbebe7fc700 1 -- 172.21.15.162:0/3570859170 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fbec4043670 con 0x7fbec805fa90 2020-11-28T21:50:08.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.267+0000 7fbece576700 1 -- 172.21.15.162:0/3570859170 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fbec8068350 con 0x7fbec805fa90 2020-11-28T21:50:08.276 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.275+0000 7fbebe7fc700 1 -- 172.21.15.162:0/3570859170 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v16) v1 ==== 184+0+0 (secure 0 0 0) 0x7fbec4046c90 con 0x7fbec805fa90 2020-11-28T21:50:08.278 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.279+0000 7fbece576700 1 -- 172.21.15.162:0/3570859170 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbeb0055440 msgr2=0x7fbeb0057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:08.279 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.279+0000 7fbece576700 1 --2- 172.21.15.162:0/3570859170 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbeb0055440 0x7fbeb0057900 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7fbeb800d000 tx=0x7fbeb8007b60).stop 2020-11-28T21:50:08.279 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.279+0000 7fbece576700 1 -- 172.21.15.162:0/3570859170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbec805fa90 msgr2=0x7fbec80b5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:08.279 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.279+0000 7fbece576700 1 --2- 172.21.15.162:0/3570859170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbec805fa90 0x7fbec80b5460 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7fbec80637b0 tx=0x7fbec4007c30).stop 2020-11-28T21:50:08.280 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.279+0000 7fbece576700 1 -- 172.21.15.162:0/3570859170 shutdown_connections 2020-11-28T21:50:08.280 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.279+0000 7fbece576700 1 --2- 172.21.15.162:0/3570859170 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbeb0055440 0x7fbeb0057900 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:08.280 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.279+0000 7fbece576700 1 --2- 172.21.15.162:0/3570859170 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbec80b2e80 0x7fbec8066d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:08.280 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.279+0000 7fbece576700 1 --2- 172.21.15.162:0/3570859170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbec805fa90 0x7fbec80b5460 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:08.281 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.279+0000 7fbece576700 1 --2- 172.21.15.162:0/3570859170 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbec80b2560 0x7fbec8066810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:08.281 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.279+0000 7fbece576700 1 -- 172.21.15.162:0/3570859170 >> 172.21.15.162:0/3570859170 conn(0x7fbec8063b60 msgr2=0x7fbec805f380 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:08.281 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.279+0000 7fbece576700 1 -- 172.21.15.162:0/3570859170 shutdown_connections 2020-11-28T21:50:08.281 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.279+0000 7fbece576700 1 -- 172.21.15.162:0/3570859170 wait complete. 2020-11-28T21:50:08.283 INFO:teuthology.orchestra.run.smithi162.stderr:updated caps for client.1 2020-11-28T21:50:08.292 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:50:08.409 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.411+0000 7f7b71ff9700 1 Processor -- start 2020-11-28T21:50:08.409 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.411+0000 7f7b71ff9700 1 -- start start 2020-11-28T21:50:08.411 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.411+0000 7f7b71ff9700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7b6c0b7780 0x7f7b6c066ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:08.411 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.411+0000 7f7b71ff9700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7b6c0b33d0 0x7f7b6c061590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:08.412 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.411+0000 7f7b71ff9700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b6c143c90 con 0x7f7b6c062ac0 2020-11-28T21:50:08.412 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.411+0000 7f7b71ff9700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b6c0674e0 con 0x7f7b6c0b33d0 2020-11-28T21:50:08.412 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.411+0000 7f7b71ff9700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6c144b40 con 0x7f7b6c0b7780 2020-11-28T21:50:08.412 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.411+0000 7f7b717f8700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7b6c0b33d0 0x7f7b6c061590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:08.412 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.411+0000 7f7b717f8700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7b6c0b33d0 0x7f7b6c061590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40078/0 (socket says 172.21.15.162:40078) 2020-11-28T21:50:08.413 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.411+0000 7f7b717f8700 1 -- 172.21.15.162:0/1403852011 learned_addr learned my addr 172.21.15.162:0/1403852011 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:08.413 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.411+0000 7f7b70ff7700 1 --2- 172.21.15.162:0/1403852011 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7b6c0b7780 0x7f7b6c066ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:08.413 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.411+0000 7f7b717f8700 1 -- 172.21.15.162:0/1403852011 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7b6c0b7780 msgr2=0x7f7b6c066ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:08.413 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.411+0000 7f7b717f8700 1 --2- 172.21.15.162:0/1403852011 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7b6c0b7780 0x7f7b6c066ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:08.413 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.411+0000 7f7b717f8700 1 -- 172.21.15.162:0/1403852011 >> v1:172.21.15.118:6789/0 conn(0x7f7b6c062ac0 legacy=0x7f7b6c0b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:08.414 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.411+0000 7f7b717f8700 1 -- 172.21.15.162:0/1403852011 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b6c143ec0 con 0x7f7b6c0b33d0 2020-11-28T21:50:08.414 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.411+0000 7f7b70ff7700 1 --2- 172.21.15.162:0/1403852011 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7b6c0b7780 0x7f7b6c066ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:50:08.414 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.411+0000 7f7b717f8700 1 --2- 172.21.15.162:0/1403852011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b6c0b33d0 0x7f7b6c061590 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f7b6c0640e0 tx=0x7f7b64006e80).ready entity=mon.1 client_cookie=d59eea5711ddadc2 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:08.414 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.411+0000 7f7b6b7fe700 1 -- 172.21.15.162:0/1403852011 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b640097b0 con 0x7f7b6c0b33d0 2020-11-28T21:50:08.414 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.411+0000 7f7b6b7fe700 1 -- 172.21.15.162:0/1403852011 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b64009910 con 0x7f7b6c0b33d0 2020-11-28T21:50:08.415 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.411+0000 7f7b6b7fe700 1 -- 172.21.15.162:0/1403852011 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b64009ac0 con 0x7f7b6c0b33d0 2020-11-28T21:50:08.415 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.411+0000 7f7b71ff9700 1 -- 172.21.15.162:0/1403852011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b6c0b33d0 msgr2=0x7f7b6c061590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:08.415 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.411+0000 7f7b71ff9700 1 --2- 172.21.15.162:0/1403852011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b6c0b33d0 0x7f7b6c061590 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f7b6c0640e0 tx=0x7f7b64006e80).stop 2020-11-28T21:50:08.416 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.415+0000 7f7b71ff9700 1 -- 172.21.15.162:0/1403852011 shutdown_connections 2020-11-28T21:50:08.416 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.415+0000 7f7b71ff9700 1 --2- 172.21.15.162:0/1403852011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b6c0b33d0 0x7f7b6c061590 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:08.417 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.415+0000 7f7b71ff9700 1 --2- 172.21.15.162:0/1403852011 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7b6c0b7780 0x7f7b6c066ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:08.417 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.415+0000 7f7b71ff9700 1 -- 172.21.15.162:0/1403852011 >> 172.21.15.162:0/1403852011 conn(0x7f7b6c05c440 msgr2=0x7f7b6c0b6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:08.417 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.415+0000 7f7b71ff9700 1 -- 172.21.15.162:0/1403852011 shutdown_connections 2020-11-28T21:50:08.417 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.415+0000 7f7b71ff9700 1 -- 172.21.15.162:0/1403852011 wait complete. 2020-11-28T21:50:08.417 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.415+0000 7f7b71ff9700 1 Processor -- start 2020-11-28T21:50:08.418 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.415+0000 7f7b71ff9700 1 -- start start 2020-11-28T21:50:08.422 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.419+0000 7f7b71ff9700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7b6c062ac0 0x7f7b6c140870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:08.422 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.419+0000 7f7b71ff9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7b6c0b33d0 0x7f7b6c08fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:08.423 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.419+0000 7f7b71ff9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b6c0b7780 0x7f7b6c090290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:08.423 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.419+0000 7f7b71ff9700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6c091c40 con 0x7f7b6c0b33d0 2020-11-28T21:50:08.424 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.419+0000 7f7b71ff9700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6c14f6f0 con 0x7f7b6c0b7780 2020-11-28T21:50:08.424 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.419+0000 7f7b71ff9700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6c090780 con 0x7f7b6c062ac0 2020-11-28T21:50:08.424 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.419+0000 7f7b717f8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b6c0b7780 0x7f7b6c090290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:08.424 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.419+0000 7f7b717f8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b6c0b7780 0x7f7b6c090290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40086/0 (socket says 172.21.15.162:40086) 2020-11-28T21:50:08.425 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.419+0000 7f7b717f8700 1 -- 172.21.15.162:0/1946946343 learned_addr learned my addr 172.21.15.162:0/1946946343 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:08.425 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.419+0000 7f7b70ff7700 1 --2- 172.21.15.162:0/1946946343 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7b6c062ac0 0x7f7b6c140870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:08.426 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.419+0000 7f7b6bfff700 1 --2- 172.21.15.162:0/1946946343 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7b6c0b33d0 0x7f7b6c08fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:08.426 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.419+0000 7f7b717f8700 1 -- 172.21.15.162:0/1946946343 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7b6c062ac0 msgr2=0x7f7b6c140870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:08.426 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.419+0000 7f7b717f8700 1 --2- 172.21.15.162:0/1946946343 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7b6c062ac0 0x7f7b6c140870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:08.427 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.419+0000 7f7b717f8700 1 -- 172.21.15.162:0/1946946343 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7b6c0b33d0 msgr2=0x7f7b6c08fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:08.427 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.419+0000 7f7b717f8700 1 --2- 172.21.15.162:0/1946946343 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7b6c0b33d0 0x7f7b6c08fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:08.428 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.419+0000 7f7b717f8700 1 -- 172.21.15.162:0/1946946343 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b6c14f8e0 con 0x7f7b6c0b7780 2020-11-28T21:50:08.428 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.419+0000 7f7b6bfff700 1 --2- 172.21.15.162:0/1946946343 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7b6c0b33d0 0x7f7b6c08fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:50:08.428 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.419+0000 7f7b70ff7700 1 --2- 172.21.15.162:0/1946946343 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7b6c062ac0 0x7f7b6c140870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:50:08.429 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.419+0000 7f7b717f8700 1 --2- 172.21.15.162:0/1946946343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b6c0b7780 0x7f7b6c090290 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f7b6c0640e0 tx=0x7f7b64016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:08.429 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.419+0000 7f7b69ffb700 1 -- 172.21.15.162:0/1946946343 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b64021440 con 0x7f7b6c0b7780 2020-11-28T21:50:08.429 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.419+0000 7f7b69ffb700 1 -- 172.21.15.162:0/1946946343 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b640215a0 con 0x7f7b6c0b7780 2020-11-28T21:50:08.430 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.419+0000 7f7b71ff9700 1 -- 172.21.15.162:0/1946946343 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b6c14fb80 con 0x7f7b6c0b7780 2020-11-28T21:50:08.431 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.419+0000 7f7b69ffb700 1 -- 172.21.15.162:0/1946946343 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b64021900 con 0x7f7b6c0b7780 2020-11-28T21:50:08.431 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.419+0000 7f7b71ff9700 1 -- 172.21.15.162:0/1946946343 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b6c1500c0 con 0x7f7b6c0b7780 2020-11-28T21:50:08.432 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.423+0000 7f7b69ffb700 1 -- 172.21.15.162:0/1946946343 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7b64027020 con 0x7f7b6c0b7780 2020-11-28T21:50:08.432 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.423+0000 7f7b71ff9700 1 -- 172.21.15.162:0/1946946343 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b6c057050 con 0x7f7b6c0b7780 2020-11-28T21:50:08.432 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.423+0000 7f7b69ffb700 1 --2- 172.21.15.162:0/1946946343 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7b4c055330 0x7f7b4c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:08.432 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.423+0000 7f7b69ffb700 1 -- 172.21.15.162:0/1946946343 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7b6406f0f0 con 0x7f7b6c0b7780 2020-11-28T21:50:08.433 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.423+0000 7f7b70ff7700 1 --2- 172.21.15.162:0/1946946343 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7b4c055330 0x7f7b4c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:08.433 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.423+0000 7f7b70ff7700 1 --2- 172.21.15.162:0/1946946343 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7b4c055330 0x7f7b4c0577f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f7b5c00b000 tx=0x7f7b5c008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:08.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.427+0000 7f7b69ffb700 1 -- 172.21.15.162:0/1946946343 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7b64045620 con 0x7f7b6c0b7780 2020-11-28T21:50:08.610 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.611+0000 7f7b71ff9700 1 -- 172.21.15.162:0/1946946343 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b6c0676d0 con 0x7f7b6c0b7780 2020-11-28T21:50:08.611 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.611+0000 7f7b69ffb700 1 -- 172.21.15.162:0/1946946343 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+2501 (secure 0 0 0) 0x7f7b64015070 con 0x7f7b6c0b7780 2020-11-28T21:50:08.611 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:50:08.611 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":21,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/1245801281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1245801281},{"type":"v1","addr":"172.21.15.118:6833","nonce":1245801281}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":18},{"gid":5536,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6835/3629429916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3629429916},{"type":"v1","addr":"172.21.15.162:6835","nonce":3629429916}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":19}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:50:05.655172+0000","modified":"2020-11-28T21:50:06.657968+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5551},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5551":{"gid":5551,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/1902721947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1902721947},{"type":"v1","addr":"172.21.15.162:6837","nonce":1902721947}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-28T21:50:08.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.615+0000 7f7b71ff9700 1 -- 172.21.15.162:0/1946946343 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7b4c055330 msgr2=0x7f7b4c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:08.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.615+0000 7f7b71ff9700 1 --2- 172.21.15.162:0/1946946343 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7b4c055330 0x7f7b4c0577f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f7b5c00b000 tx=0x7f7b5c008040).stop 2020-11-28T21:50:08.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.615+0000 7f7b71ff9700 1 -- 172.21.15.162:0/1946946343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b6c0b7780 msgr2=0x7f7b6c090290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:08.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.615+0000 7f7b71ff9700 1 --2- 172.21.15.162:0/1946946343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b6c0b7780 0x7f7b6c090290 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f7b6c0640e0 tx=0x7f7b64016000).stop 2020-11-28T21:50:08.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.615+0000 7f7b71ff9700 1 -- 172.21.15.162:0/1946946343 shutdown_connections 2020-11-28T21:50:08.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.615+0000 7f7b71ff9700 1 --2- 172.21.15.162:0/1946946343 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7b4c055330 0x7f7b4c0577f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:08.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.615+0000 7f7b71ff9700 1 --2- 172.21.15.162:0/1946946343 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7b6c062ac0 0x7f7b6c140870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:08.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.615+0000 7f7b71ff9700 1 --2- 172.21.15.162:0/1946946343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b6c0b7780 0x7f7b6c090290 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:08.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.615+0000 7f7b71ff9700 1 --2- 172.21.15.162:0/1946946343 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7b6c0b33d0 0x7f7b6c08fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:08.615 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.615+0000 7f7b71ff9700 1 -- 172.21.15.162:0/1946946343 >> 172.21.15.162:0/1946946343 conn(0x7f7b6c05c440 msgr2=0x7f7b6c0b5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:08.615 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.615+0000 7f7b71ff9700 1 -- 172.21.15.162:0/1946946343 shutdown_connections 2020-11-28T21:50:08.615 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:08.615+0000 7f7b71ff9700 1 -- 172.21.15.162:0/1946946343 wait complete. 2020-11-28T21:50:08.616 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 21 2020-11-28T21:50:08.927 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-28T21:50:05.655172+0000', 'modified': '2020-11-28T21:50:06.657968+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5551}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5551': {'gid': 5551, 'name': 'c', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6837/1902721947', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6836', 'nonce': 1902721947}, {'type': 'v1', 'addr': '172.21.15.162:6837', 'nonce': 1902721947}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-28T21:50:08.927 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-28T21:50:08.928 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-28T21:50:09.058 INFO:teuthology.orchestra.run.smithi162.stdout:{"cluster_fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","whoami":0,"id":5551,"want_state":"up:active","state":"up:active","rank_uptime":3.3920801310000002,"mdsmap_epoch":21,"osdmap_epoch":56,"osdmap_epoch_barrier":54,"uptime":8.7485028029999992} 2020-11-28T21:50:09.058 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", "id": 5551, "mdsmap_epoch": 21, "osdmap_epoch": 56, "osdmap_epoch_barrier": 54, "rank_uptime": 3.392080131, "state": "up:active", "uptime": 8.748502803, "want_state": "up:active", "whoami": 0 } 2020-11-28T21:50:09.059 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:50:09.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.171+0000 7f14b4193700 1 Processor -- start 2020-11-28T21:50:09.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.171+0000 7f14b4193700 1 -- start start 2020-11-28T21:50:09.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.171+0000 7f14b4193700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14ac060ce0 0x7f14ac1400f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:09.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.171+0000 7f14b4193700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f14ac1405e0 0x7f14ac0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:09.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.171+0000 7f14b4193700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14ac0b0c40 con 0x7f14ac0b07a0 2020-11-28T21:50:09.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.171+0000 7f14b4193700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14ac0ad800 con 0x7f14ac1405e0 2020-11-28T21:50:09.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.171+0000 7f14b4193700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac0adaa0 con 0x7f14ac060ce0 2020-11-28T21:50:09.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.171+0000 7f14b172e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f14ac1405e0 0x7f14ac0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:09.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.171+0000 7f14b172e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f14ac1405e0 0x7f14ac0b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40090/0 (socket says 172.21.15.162:40090) 2020-11-28T21:50:09.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.171+0000 7f14b172e700 1 -- 172.21.15.162:0/3159432710 learned_addr learned my addr 172.21.15.162:0/3159432710 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:09.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.171+0000 7f14b1f2f700 1 --2- 172.21.15.162:0/3159432710 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14ac060ce0 0x7f14ac1400f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:09.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.175+0000 7f14b172e700 1 -- 172.21.15.162:0/3159432710 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14ac060ce0 msgr2=0x7f14ac1400f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:09.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.175+0000 7f14b172e700 1 --2- 172.21.15.162:0/3159432710 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14ac060ce0 0x7f14ac1400f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:09.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.175+0000 7f14b0f2d700 1 -- 172.21.15.162:0/3159432710 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2038611051 0 0) 0x7f14ac0b0c40 con 0x7f14ac0b07a0 2020-11-28T21:50:09.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.175+0000 7f14b172e700 1 -- 172.21.15.162:0/3159432710 >> v1:172.21.15.162:6789/0 conn(0x7f14ac0b07a0 legacy=0x7f14ac146090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:09.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.175+0000 7f14b172e700 1 -- 172.21.15.162:0/3159432710 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ac0add40 con 0x7f14ac1405e0 2020-11-28T21:50:09.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.175+0000 7f14b172e700 1 --2- 172.21.15.162:0/3159432710 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14ac1405e0 0x7f14ac0b51b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f149c002f30 tx=0x7f149c006e80).ready entity=mon.1 client_cookie=7bd71872ddb1b184 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:09.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.175+0000 7f14b0f2d700 1 -- 172.21.15.162:0/3159432710 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f149c0097f0 con 0x7f14ac1405e0 2020-11-28T21:50:09.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.175+0000 7f14b0f2d700 1 -- 172.21.15.162:0/3159432710 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f149c009950 con 0x7f14ac1405e0 2020-11-28T21:50:09.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.175+0000 7f14b0f2d700 1 -- 172.21.15.162:0/3159432710 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f149c009cb0 con 0x7f14ac1405e0 2020-11-28T21:50:09.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.175+0000 7f14b4193700 1 -- 172.21.15.162:0/3159432710 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14ac1405e0 msgr2=0x7f14ac0b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:09.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.175+0000 7f14b4193700 1 --2- 172.21.15.162:0/3159432710 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14ac1405e0 0x7f14ac0b51b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f149c002f30 tx=0x7f149c006e80).stop 2020-11-28T21:50:09.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.175+0000 7f14b4193700 1 -- 172.21.15.162:0/3159432710 shutdown_connections 2020-11-28T21:50:09.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.175+0000 7f14b4193700 1 --2- 172.21.15.162:0/3159432710 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14ac060ce0 0x7f14ac1400f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:09.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.175+0000 7f14b4193700 1 --2- 172.21.15.162:0/3159432710 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14ac1405e0 0x7f14ac0b51b0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:09.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.175+0000 7f14b4193700 1 -- 172.21.15.162:0/3159432710 >> 172.21.15.162:0/3159432710 conn(0x7f14ac06ac10 msgr2=0x7f14ac062c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:09.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.175+0000 7f14b4193700 1 -- 172.21.15.162:0/3159432710 shutdown_connections 2020-11-28T21:50:09.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.175+0000 7f14b4193700 1 -- 172.21.15.162:0/3159432710 wait complete. 2020-11-28T21:50:09.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.175+0000 7f14b4193700 1 Processor -- start 2020-11-28T21:50:09.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.175+0000 7f14b4193700 1 -- start start 2020-11-28T21:50:09.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.179+0000 7f14b4193700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14ac060ce0 0x7f14ac1432b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:09.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.179+0000 7f14b4193700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14ac0b07a0 0x7f14ac0ab7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:09.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.179+0000 7f14b4193700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14ac1405e0 0x7f14ac151e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:09.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.179+0000 7f14b4193700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac13c270 con 0x7f14ac060ce0 2020-11-28T21:50:09.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.179+0000 7f14b4193700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac151820 con 0x7f14ac0b07a0 2020-11-28T21:50:09.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.179+0000 7f14b4193700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac0aaac0 con 0x7f14ac1405e0 2020-11-28T21:50:09.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.179+0000 7f14b172e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14ac0b07a0 0x7f14ac0ab7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:09.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.179+0000 7f14b172e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14ac0b07a0 0x7f14ac0ab7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40098/0 (socket says 172.21.15.162:40098) 2020-11-28T21:50:09.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.179+0000 7f14b172e700 1 -- 172.21.15.162:0/210542478 learned_addr learned my addr 172.21.15.162:0/210542478 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:09.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.179+0000 7f14b2730700 1 --2- 172.21.15.162:0/210542478 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14ac1405e0 0x7f14ac151e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:09.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.179+0000 7f14b1f2f700 1 --2- 172.21.15.162:0/210542478 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14ac060ce0 0x7f14ac1432b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:09.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.179+0000 7f14b172e700 1 -- 172.21.15.162:0/210542478 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14ac1405e0 msgr2=0x7f14ac151e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:09.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.179+0000 7f14b172e700 1 --2- 172.21.15.162:0/210542478 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14ac1405e0 0x7f14ac151e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:09.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.179+0000 7f14b172e700 1 -- 172.21.15.162:0/210542478 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14ac060ce0 msgr2=0x7f14ac1432b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:09.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.179+0000 7f14b172e700 1 --2- 172.21.15.162:0/210542478 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14ac060ce0 0x7f14ac1432b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:09.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.179+0000 7f14b172e700 1 -- 172.21.15.162:0/210542478 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ac0aad60 con 0x7f14ac0b07a0 2020-11-28T21:50:09.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.179+0000 7f14b1f2f700 1 --2- 172.21.15.162:0/210542478 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14ac060ce0 0x7f14ac1432b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:50:09.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.179+0000 7f14b172e700 1 --2- 172.21.15.162:0/210542478 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14ac0b07a0 0x7f14ac0ab7c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f14ac13c3b0 tx=0x7f149c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:09.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.179+0000 7f149affd700 1 -- 172.21.15.162:0/210542478 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f149c011b90 con 0x7f14ac0b07a0 2020-11-28T21:50:09.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.179+0000 7f149affd700 1 -- 172.21.15.162:0/210542478 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f149c011cf0 con 0x7f14ac0b07a0 2020-11-28T21:50:09.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.179+0000 7f149affd700 1 -- 172.21.15.162:0/210542478 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f149c01e650 con 0x7f14ac0b07a0 2020-11-28T21:50:09.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.179+0000 7f14b4193700 1 -- 172.21.15.162:0/210542478 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14ac0ab000 con 0x7f14ac0b07a0 2020-11-28T21:50:09.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.179+0000 7f14b4193700 1 -- 172.21.15.162:0/210542478 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14ac1418d0 con 0x7f14ac0b07a0 2020-11-28T21:50:09.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.179+0000 7f14b4193700 1 -- 172.21.15.162:0/210542478 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14ac057050 con 0x7f14ac0b07a0 2020-11-28T21:50:09.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.183+0000 7f149affd700 1 -- 172.21.15.162:0/210542478 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f149c025030 con 0x7f14ac0b07a0 2020-11-28T21:50:09.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.183+0000 7f149affd700 1 --2- 172.21.15.162:0/210542478 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f148c055470 0x7f148c057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:09.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.183+0000 7f149affd700 1 -- 172.21.15.162:0/210542478 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f149c06f2a0 con 0x7f14ac0b07a0 2020-11-28T21:50:09.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.183+0000 7f14b1f2f700 1 --2- 172.21.15.162:0/210542478 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f148c055470 0x7f148c057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:09.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.183+0000 7f14b1f2f700 1 --2- 172.21.15.162:0/210542478 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f148c055470 0x7f148c057930 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f14a800b000 tx=0x7f14a8008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:09.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.191+0000 7f149affd700 1 -- 172.21.15.162:0/210542478 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f149c0455a0 con 0x7f14ac0b07a0 2020-11-28T21:50:09.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.331+0000 7f14b4193700 1 -- 172.21.15.162:0/210542478 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14ac0b1910 con 0x7f14ac0b07a0 2020-11-28T21:50:09.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.331+0000 7f149affd700 1 -- 172.21.15.162:0/210542478 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+16136 (secure 0 0 0) 0x7f149c048bc0 con 0x7f14ac0b07a0 2020-11-28T21:50:09.331 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:50:09.331 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":56,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:50:07.723574+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:50:03.211406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2020-11-28T21:50:04.511075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:50:09.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.335+0000 7f14b4193700 1 -- 172.21.15.162:0/210542478 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f148c055470 msgr2=0x7f148c057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:09.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.335+0000 7f14b4193700 1 --2- 172.21.15.162:0/210542478 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f148c055470 0x7f148c057930 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f14a800b000 tx=0x7f14a8008040).stop 2020-11-28T21:50:09.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.335+0000 7f14b4193700 1 -- 172.21.15.162:0/210542478 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14ac0b07a0 msgr2=0x7f14ac0ab7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:09.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.335+0000 7f14b4193700 1 --2- 172.21.15.162:0/210542478 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14ac0b07a0 0x7f14ac0ab7c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f14ac13c3b0 tx=0x7f149c015000).stop 2020-11-28T21:50:09.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.335+0000 7f14b4193700 1 -- 172.21.15.162:0/210542478 shutdown_connections 2020-11-28T21:50:09.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.335+0000 7f14b4193700 1 --2- 172.21.15.162:0/210542478 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f148c055470 0x7f148c057930 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:09.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.335+0000 7f14b4193700 1 --2- 172.21.15.162:0/210542478 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14ac1405e0 0x7f14ac151e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:09.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.335+0000 7f14b4193700 1 --2- 172.21.15.162:0/210542478 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14ac060ce0 0x7f14ac1432b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:09.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.335+0000 7f14b4193700 1 --2- 172.21.15.162:0/210542478 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14ac0b07a0 0x7f14ac0ab7c0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:09.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.335+0000 7f14b4193700 1 -- 172.21.15.162:0/210542478 >> 172.21.15.162:0/210542478 conn(0x7f14ac06ac10 msgr2=0x7f14ac0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:09.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.335+0000 7f14b4193700 1 -- 172.21.15.162:0/210542478 shutdown_connections 2020-11-28T21:50:09.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.335+0000 7f14b4193700 1 -- 172.21.15.162:0/210542478 wait complete. 2020-11-28T21:50:09.344 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-28T21:50:03.211406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2020-11-28T21:50:04.511075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-28T21:50:09.345 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:50:09.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.467+0000 7fa859e5e700 1 Processor -- start 2020-11-28T21:50:09.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.467+0000 7fa859e5e700 1 -- start start 2020-11-28T21:50:09.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.467+0000 7fa859e5e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa8540b3d60 0x7fa8540b3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:09.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.467+0000 7fa859e5e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fa85409a7f0 0x7fa85409ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:09.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.467+0000 7fa859e5e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa85409b160 0x7fa85414a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:09.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.467+0000 7fa859e5e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8540b4140 con 0x7fa85409a7f0 2020-11-28T21:50:09.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.467+0000 7fa859e5e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa854066160 con 0x7fa85409b160 2020-11-28T21:50:09.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.467+0000 7fa859e5e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa854066400 con 0x7fa8540b3d60 2020-11-28T21:50:09.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.467+0000 7fa853fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa85409b160 0x7fa85414a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:09.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.467+0000 7fa853fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa85409b160 0x7fa85414a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40106/0 (socket says 172.21.15.162:40106) 2020-11-28T21:50:09.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.467+0000 7fa853fff700 1 -- 172.21.15.162:0/861545669 learned_addr learned my addr 172.21.15.162:0/861545669 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:09.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.467+0000 7fa8537fe700 1 --2- 172.21.15.162:0/861545669 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa8540b3d60 0x7fa8540b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:09.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.467+0000 7fa852ffd700 1 --2- 172.21.15.162:0/861545669 >> v2:172.21.15.118:3300/0 conn(0x7fa85409a7f0 0x7fa85409ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:09.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.467+0000 7fa853fff700 1 -- 172.21.15.162:0/861545669 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa8540b3d60 msgr2=0x7fa8540b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:09.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.467+0000 7fa853fff700 1 --2- 172.21.15.162:0/861545669 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa8540b3d60 0x7fa8540b3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:09.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.467+0000 7fa853fff700 1 -- 172.21.15.162:0/861545669 >> v2:172.21.15.118:3300/0 conn(0x7fa85409a7f0 msgr2=0x7fa85409ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:09.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.467+0000 7fa853fff700 1 --2- 172.21.15.162:0/861545669 >> v2:172.21.15.118:3300/0 conn(0x7fa85409a7f0 0x7fa85409ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:09.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.467+0000 7fa853fff700 1 -- 172.21.15.162:0/861545669 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8540666a0 con 0x7fa85409b160 2020-11-28T21:50:09.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.467+0000 7fa853fff700 1 --2- 172.21.15.162:0/861545669 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa85409b160 0x7fa85414a930 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7fa84c0032b0 tx=0x7fa84c009270).ready entity=mon.1 client_cookie=e99c01f9dc1f4207 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:09.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.467+0000 7fa8527fc700 1 -- 172.21.15.162:0/861545669 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa84c00bc00 con 0x7fa85409b160 2020-11-28T21:50:09.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.467+0000 7fa8527fc700 1 -- 172.21.15.162:0/861545669 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa84c00bd60 con 0x7fa85409b160 2020-11-28T21:50:09.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.467+0000 7fa8527fc700 1 -- 172.21.15.162:0/861545669 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa84c011400 con 0x7fa85409b160 2020-11-28T21:50:09.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.467+0000 7fa859e5e700 1 -- 172.21.15.162:0/861545669 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa85409b160 msgr2=0x7fa85414a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:09.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.467+0000 7fa859e5e700 1 --2- 172.21.15.162:0/861545669 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa85409b160 0x7fa85414a930 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7fa84c0032b0 tx=0x7fa84c009270).stop 2020-11-28T21:50:09.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.471+0000 7fa859e5e700 1 -- 172.21.15.162:0/861545669 shutdown_connections 2020-11-28T21:50:09.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.471+0000 7fa859e5e700 1 --2- 172.21.15.162:0/861545669 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa85409b160 0x7fa85414a930 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:09.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.471+0000 7fa859e5e700 1 --2- 172.21.15.162:0/861545669 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa8540b3d60 0x7fa8540b3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:09.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.471+0000 7fa859e5e700 1 --2- 172.21.15.162:0/861545669 >> v2:172.21.15.118:3300/0 conn(0x7fa85409a7f0 0x7fa85409ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:09.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.471+0000 7fa859e5e700 1 -- 172.21.15.162:0/861545669 >> 172.21.15.162:0/861545669 conn(0x7fa85405b970 msgr2=0x7fa85405bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:09.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.471+0000 7fa859e5e700 1 -- 172.21.15.162:0/861545669 shutdown_connections 2020-11-28T21:50:09.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.471+0000 7fa859e5e700 1 -- 172.21.15.162:0/861545669 wait complete. 2020-11-28T21:50:09.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.471+0000 7fa859e5e700 1 Processor -- start 2020-11-28T21:50:09.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.471+0000 7fa859e5e700 1 -- start start 2020-11-28T21:50:09.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.471+0000 7fa859e5e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa85409a7f0 0x7fa8540b5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:09.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.471+0000 7fa859e5e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8540b3d60 0x7fa8541468f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:09.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.471+0000 7fa859e5e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa8540b5e40 0x7fa854146de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:09.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.471+0000 7fa859e5e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8540ae480 con 0x7fa8540b5e40 2020-11-28T21:50:09.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.471+0000 7fa859e5e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8540ae6d0 con 0x7fa8540b3d60 2020-11-28T21:50:09.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.471+0000 7fa859e5e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8540ae970 con 0x7fa85409a7f0 2020-11-28T21:50:09.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.471+0000 7fa852ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8540b3d60 0x7fa8541468f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:09.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.471+0000 7fa852ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8540b3d60 0x7fa8541468f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40112/0 (socket says 172.21.15.162:40112) 2020-11-28T21:50:09.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.471+0000 7fa852ffd700 1 -- 172.21.15.162:0/2790160676 learned_addr learned my addr 172.21.15.162:0/2790160676 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:09.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.471+0000 7fa8537fe700 1 --2- 172.21.15.162:0/2790160676 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa85409a7f0 0x7fa8540b5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:09.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.471+0000 7fa853fff700 1 --2- 172.21.15.162:0/2790160676 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa8540b5e40 0x7fa854146de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:09.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.471+0000 7fa852ffd700 1 -- 172.21.15.162:0/2790160676 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa85409a7f0 msgr2=0x7fa8540b5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:09.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.471+0000 7fa852ffd700 1 --2- 172.21.15.162:0/2790160676 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa85409a7f0 0x7fa8540b5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:09.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.471+0000 7fa852ffd700 1 -- 172.21.15.162:0/2790160676 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa8540b5e40 msgr2=0x7fa854146de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:09.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.471+0000 7fa852ffd700 1 --2- 172.21.15.162:0/2790160676 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa8540b5e40 0x7fa854146de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:09.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.471+0000 7fa852ffd700 1 -- 172.21.15.162:0/2790160676 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8540aec10 con 0x7fa8540b3d60 2020-11-28T21:50:09.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.471+0000 7fa852ffd700 1 --2- 172.21.15.162:0/2790160676 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8540b3d60 0x7fa8541468f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7fa844003070 tx=0x7fa844008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:09.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.471+0000 7fa850ff9700 1 -- 172.21.15.162:0/2790160676 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa844009790 con 0x7fa8540b3d60 2020-11-28T21:50:09.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.471+0000 7fa859e5e700 1 -- 172.21.15.162:0/2790160676 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8541472d0 con 0x7fa8540b3d60 2020-11-28T21:50:09.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.471+0000 7fa850ff9700 1 -- 172.21.15.162:0/2790160676 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8440098f0 con 0x7fa8540b3d60 2020-11-28T21:50:09.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.471+0000 7fa850ff9700 1 -- 172.21.15.162:0/2790160676 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa844009c50 con 0x7fa8540b3d60 2020-11-28T21:50:09.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.471+0000 7fa859e5e700 1 -- 172.21.15.162:0/2790160676 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8540b04b0 con 0x7fa8540b3d60 2020-11-28T21:50:09.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.475+0000 7fa859e5e700 1 -- 172.21.15.162:0/2790160676 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8540aeda0 con 0x7fa8540b3d60 2020-11-28T21:50:09.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.475+0000 7fa850ff9700 1 -- 172.21.15.162:0/2790160676 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa844013480 con 0x7fa8540b3d60 2020-11-28T21:50:09.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.475+0000 7fa850ff9700 1 --2- 172.21.15.162:0/2790160676 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa834055330 0x7fa8340577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:09.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.475+0000 7fa850ff9700 1 -- 172.21.15.162:0/2790160676 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa84406e3d0 con 0x7fa8540b3d60 2020-11-28T21:50:09.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.475+0000 7fa8537fe700 1 --2- 172.21.15.162:0/2790160676 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa834055330 0x7fa8340577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:09.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.475+0000 7fa8537fe700 1 --2- 172.21.15.162:0/2790160676 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa834055330 0x7fa8340577f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7fa84800c000 tx=0x7fa848008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:09.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.483+0000 7fa850ff9700 1 -- 172.21.15.162:0/2790160676 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa8440445d0 con 0x7fa8540b3d60 2020-11-28T21:50:09.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.623+0000 7fa859e5e700 1 -- 172.21.15.162:0/2790160676 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fa854057050 con 0x7fa8540b3d60 2020-11-28T21:50:09.624 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.623+0000 7fa850ff9700 1 -- 172.21.15.162:0/2790160676 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v56) v1 ==== 110+0+10 (secure 0 0 0) 0x7fa844047bf0 con 0x7fa8540b3d60 2020-11-28T21:50:09.624 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:50:09.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.627+0000 7fa859e5e700 1 -- 172.21.15.162:0/2790160676 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa834055330 msgr2=0x7fa8340577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:09.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.627+0000 7fa859e5e700 1 --2- 172.21.15.162:0/2790160676 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa834055330 0x7fa8340577f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7fa84800c000 tx=0x7fa848008040).stop 2020-11-28T21:50:09.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.627+0000 7fa859e5e700 1 -- 172.21.15.162:0/2790160676 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8540b3d60 msgr2=0x7fa8541468f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:09.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.627+0000 7fa859e5e700 1 --2- 172.21.15.162:0/2790160676 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8540b3d60 0x7fa8541468f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7fa844003070 tx=0x7fa844008040).stop 2020-11-28T21:50:09.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.627+0000 7fa859e5e700 1 -- 172.21.15.162:0/2790160676 shutdown_connections 2020-11-28T21:50:09.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.627+0000 7fa859e5e700 1 --2- 172.21.15.162:0/2790160676 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa834055330 0x7fa8340577f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:09.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.627+0000 7fa859e5e700 1 --2- 172.21.15.162:0/2790160676 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa85409a7f0 0x7fa8540b5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:09.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.627+0000 7fa859e5e700 1 --2- 172.21.15.162:0/2790160676 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa8540b5e40 0x7fa854146de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:09.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.627+0000 7fa859e5e700 1 --2- 172.21.15.162:0/2790160676 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8540b3d60 0x7fa8541468f0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:09.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.627+0000 7fa859e5e700 1 -- 172.21.15.162:0/2790160676 >> 172.21.15.162:0/2790160676 conn(0x7fa85405b970 msgr2=0x7fa8540abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:09.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.627+0000 7fa859e5e700 1 -- 172.21.15.162:0/2790160676 shutdown_connections 2020-11-28T21:50:09.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.627+0000 7fa859e5e700 1 -- 172.21.15.162:0/2790160676 wait complete. 2020-11-28T21:50:09.637 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:50:09.753 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.755+0000 7f8544874700 1 Processor -- start 2020-11-28T21:50:09.754 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.755+0000 7f8544874700 1 -- start start 2020-11-28T21:50:09.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.755+0000 7f8544874700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f85400b2d00 0x7f854005c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:09.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.755+0000 7f8544874700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f854005baf0 con 0x7f85400b30e0 2020-11-28T21:50:09.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.755+0000 7f8544874700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f854005bd90 con 0x7f85400b3ab0 2020-11-28T21:50:09.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.755+0000 7f8544874700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85400b0e20 con 0x7f85400b2d00 2020-11-28T21:50:09.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.755+0000 7f853e59c700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f85400b3ab0 0x7f8540142ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:46966/0 (socket says 172.21.15.162:46966) 2020-11-28T21:50:09.761 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.755+0000 7f853e59c700 1 -- 172.21.15.162:0/2789986903 learned_addr learned my addr 172.21.15.162:0/2789986903 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:50:09.761 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.755+0000 7f853ed9d700 1 --2- 172.21.15.162:0/2789986903 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f85400b2d00 0x7f854005c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:09.761 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.755+0000 7f853dd9b700 1 -- 172.21.15.162:0/2789986903 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 43163337 0 0) 0x7f854005bd90 con 0x7f85400b3ab0 2020-11-28T21:50:09.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.755+0000 7f853dd9b700 1 -- 172.21.15.162:0/2789986903 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8524003230 con 0x7f85400b3ab0 2020-11-28T21:50:09.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.755+0000 7f853dd9b700 1 -- 172.21.15.162:0/2789986903 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2427281955 0 0) 0x7f854005baf0 con 0x7f85400b30e0 2020-11-28T21:50:09.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.755+0000 7f853dd9b700 1 -- 172.21.15.162:0/2789986903 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f854005bd90 con 0x7f85400b30e0 2020-11-28T21:50:09.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.755+0000 7f853ed9d700 1 -- 172.21.15.162:0/2789986903 >> v1:172.21.15.162:6789/0 conn(0x7f85400b3ab0 legacy=0x7f8540142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:09.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.755+0000 7f853ed9d700 1 -- 172.21.15.162:0/2789986903 >> v1:172.21.15.118:6789/0 conn(0x7f85400b30e0 legacy=0x7f854013fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:09.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.755+0000 7f853ed9d700 1 -- 172.21.15.162:0/2789986903 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f854005c030 con 0x7f85400b2d00 2020-11-28T21:50:09.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.755+0000 7f853ed9d700 1 --2- 172.21.15.162:0/2789986903 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f85400b2d00 0x7f854005c4d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f8530002f30 tx=0x7f8530006dc0).ready entity=mon.2 client_cookie=1eb346501ccde5ea server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:09.765 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.759+0000 7f853dd9b700 1 -- 172.21.15.162:0/2789986903 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8530009730 con 0x7f85400b2d00 2020-11-28T21:50:09.765 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.759+0000 7f853dd9b700 1 -- 172.21.15.162:0/2789986903 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8530009890 con 0x7f85400b2d00 2020-11-28T21:50:09.765 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.759+0000 7f853dd9b700 1 -- 172.21.15.162:0/2789986903 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8530009bf0 con 0x7f85400b2d00 2020-11-28T21:50:09.766 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.759+0000 7f8544874700 1 -- 172.21.15.162:0/2789986903 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f85400b2d00 msgr2=0x7f854005c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:09.767 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.759+0000 7f8544874700 1 --2- 172.21.15.162:0/2789986903 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f85400b2d00 0x7f854005c4d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f8530002f30 tx=0x7f8530006dc0).stop 2020-11-28T21:50:09.767 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.759+0000 7f8544874700 1 -- 172.21.15.162:0/2789986903 shutdown_connections 2020-11-28T21:50:09.767 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.759+0000 7f8544874700 1 --2- 172.21.15.162:0/2789986903 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f85400b2d00 0x7f854005c4d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:09.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.759+0000 7f8544874700 1 -- 172.21.15.162:0/2789986903 >> 172.21.15.162:0/2789986903 conn(0x7f85400604c0 msgr2=0x7f85400608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:09.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.759+0000 7f8544874700 1 -- 172.21.15.162:0/2789986903 shutdown_connections 2020-11-28T21:50:09.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.759+0000 7f8544874700 1 -- 172.21.15.162:0/2789986903 wait complete. 2020-11-28T21:50:09.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.759+0000 7f8544874700 1 Processor -- start 2020-11-28T21:50:09.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.759+0000 7f8544874700 1 -- start start 2020-11-28T21:50:09.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.759+0000 7f8544874700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f85400b2d00 0x7f8540149f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:09.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.759+0000 7f8544874700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f85400b30e0 0x7f8540148bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:09.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.759+0000 7f8544874700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85400b3ab0 0x7f8540151ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:09.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.759+0000 7f8544874700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85400add00 con 0x7f85400b30e0 2020-11-28T21:50:09.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.759+0000 7f8544874700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85400adfa0 con 0x7f85400b3ab0 2020-11-28T21:50:09.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.759+0000 7f8544874700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85400ae240 con 0x7f85400b2d00 2020-11-28T21:50:09.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.763+0000 7f853f59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85400b3ab0 0x7f8540151ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:09.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.763+0000 7f853f59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85400b3ab0 0x7f8540151ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40128/0 (socket says 172.21.15.162:40128) 2020-11-28T21:50:09.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.763+0000 7f853f59e700 1 -- 172.21.15.162:0/3112948002 learned_addr learned my addr 172.21.15.162:0/3112948002 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:09.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.763+0000 7f853ed9d700 1 --2- 172.21.15.162:0/3112948002 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f85400b2d00 0x7f8540149f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:09.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.763+0000 7f853e59c700 1 --2- 172.21.15.162:0/3112948002 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f85400b30e0 0x7f8540148bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:09.776 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.763+0000 7f853f59e700 1 -- 172.21.15.162:0/3112948002 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f85400b2d00 msgr2=0x7f8540149f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:09.776 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.763+0000 7f853f59e700 1 --2- 172.21.15.162:0/3112948002 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f85400b2d00 0x7f8540149f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:09.776 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.763+0000 7f853f59e700 1 -- 172.21.15.162:0/3112948002 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f85400b30e0 msgr2=0x7f8540148bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:09.776 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.763+0000 7f853f59e700 1 --2- 172.21.15.162:0/3112948002 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f85400b30e0 0x7f8540148bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:09.776 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.763+0000 7f853f59e700 1 -- 172.21.15.162:0/3112948002 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85400ad9f0 con 0x7f85400b3ab0 2020-11-28T21:50:09.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.763+0000 7f853e59c700 1 --2- 172.21.15.162:0/3112948002 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f85400b30e0 0x7f8540148bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:50:09.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.763+0000 7f853f59e700 1 --2- 172.21.15.162:0/3112948002 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85400b3ab0 0x7f8540151ad0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7f8538001450 tx=0x7f8538007da0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:09.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.763+0000 7f852ffff700 1 -- 172.21.15.162:0/3112948002 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8538002240 con 0x7f85400b3ab0 2020-11-28T21:50:09.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.763+0000 7f852ffff700 1 -- 172.21.15.162:0/3112948002 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85380023a0 con 0x7f85400b3ab0 2020-11-28T21:50:09.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.763+0000 7f852ffff700 1 -- 172.21.15.162:0/3112948002 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8538001800 con 0x7f85400b3ab0 2020-11-28T21:50:09.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.763+0000 7f8544874700 1 -- 172.21.15.162:0/3112948002 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85401490b0 con 0x7f85400b3ab0 2020-11-28T21:50:09.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.763+0000 7f8544874700 1 -- 172.21.15.162:0/3112948002 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8540151fc0 con 0x7f85400b3ab0 2020-11-28T21:50:09.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.763+0000 7f8544874700 1 -- 172.21.15.162:0/3112948002 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8540057050 con 0x7f85400b3ab0 2020-11-28T21:50:09.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.763+0000 7f852ffff700 1 -- 172.21.15.162:0/3112948002 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8538019030 con 0x7f85400b3ab0 2020-11-28T21:50:09.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.763+0000 7f852ffff700 1 --2- 172.21.15.162:0/3112948002 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f851c055330 0x7f851c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:09.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.767+0000 7f853ed9d700 1 --2- 172.21.15.162:0/3112948002 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f851c055330 0x7f851c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:09.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.767+0000 7f852ffff700 1 -- 172.21.15.162:0/3112948002 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f853806d920 con 0x7f85400b3ab0 2020-11-28T21:50:09.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.767+0000 7f853ed9d700 1 --2- 172.21.15.162:0/3112948002 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f851c055330 0x7f851c0577f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f853001b000 tx=0x7f8530015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:09.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.767+0000 7f852ffff700 1 -- 172.21.15.162:0/3112948002 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8538043a90 con 0x7f85400b3ab0 2020-11-28T21:50:09.907 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.907+0000 7f8544874700 1 -- 172.21.15.162:0/3112948002 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f854005d250 con 0x7f85400b3ab0 2020-11-28T21:50:09.907 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.907+0000 7f852ffff700 1 -- 172.21.15.162:0/3112948002 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v56) v1 ==== 92+0+10 (secure 0 0 0) 0x7f85380470b0 con 0x7f85400b3ab0 2020-11-28T21:50:09.908 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:50:09.909 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.911+0000 7f8544874700 1 -- 172.21.15.162:0/3112948002 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f851c055330 msgr2=0x7f851c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:09.909 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.911+0000 7f8544874700 1 --2- 172.21.15.162:0/3112948002 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f851c055330 0x7f851c0577f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f853001b000 tx=0x7f8530015000).stop 2020-11-28T21:50:09.910 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.911+0000 7f8544874700 1 -- 172.21.15.162:0/3112948002 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85400b3ab0 msgr2=0x7f8540151ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:09.910 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.911+0000 7f8544874700 1 --2- 172.21.15.162:0/3112948002 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85400b3ab0 0x7f8540151ad0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7f8538001450 tx=0x7f8538007da0).stop 2020-11-28T21:50:09.910 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.911+0000 7f8544874700 1 -- 172.21.15.162:0/3112948002 shutdown_connections 2020-11-28T21:50:09.910 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.911+0000 7f8544874700 1 --2- 172.21.15.162:0/3112948002 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f851c055330 0x7f851c0577f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:09.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.911+0000 7f8544874700 1 --2- 172.21.15.162:0/3112948002 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f85400b2d00 0x7f8540149f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:09.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.911+0000 7f8544874700 1 --2- 172.21.15.162:0/3112948002 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85400b3ab0 0x7f8540151ad0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:09.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.911+0000 7f8544874700 1 --2- 172.21.15.162:0/3112948002 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f85400b30e0 0x7f8540148bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:09.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.911+0000 7f8544874700 1 -- 172.21.15.162:0/3112948002 >> 172.21.15.162:0/3112948002 conn(0x7f85400604c0 msgr2=0x7f8540140020 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:09.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.911+0000 7f8544874700 1 -- 172.21.15.162:0/3112948002 shutdown_connections 2020-11-28T21:50:09.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:09.911+0000 7f8544874700 1 -- 172.21.15.162:0/3112948002 wait complete. 2020-11-28T21:50:09.921 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-28T21:50:10.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.031+0000 7fd04093f700 1 Processor -- start 2020-11-28T21:50:10.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.031+0000 7fd04093f700 1 -- start start 2020-11-28T21:50:10.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.031+0000 7fd04093f700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd03c0b9ce0 0x7fd03c1433c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:10.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.031+0000 7fd04093f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd03c067610 0x7fd03c13fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:10.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.031+0000 7fd04093f700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd03c143950 con 0x7fd03c0605c0 2020-11-28T21:50:10.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.031+0000 7fd04093f700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd03c0b4340 con 0x7fd03c0b9ce0 2020-11-28T21:50:10.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.031+0000 7fd04093f700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c0ba0c0 con 0x7fd03c067610 2020-11-28T21:50:10.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.031+0000 7fd039d9b700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fd03c0605c0 0x7fd03c0b3c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:46976/0 (socket says 172.21.15.162:46976) 2020-11-28T21:50:10.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.031+0000 7fd039d9b700 1 -- 172.21.15.162:0/1943809467 learned_addr learned my addr 172.21.15.162:0/1943809467 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:50:10.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.031+0000 7fd03ad9d700 1 --2- 172.21.15.162:0/1943809467 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd03c067610 0x7fd03c13fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:10.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.031+0000 7fd03a59c700 1 --2- 172.21.15.162:0/1943809467 >> v2:172.21.15.118:3300/0 conn(0x7fd03c0b9ce0 0x7fd03c1433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:10.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.035+0000 7fd03959a700 1 -- 172.21.15.162:0/1943809467 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1334291723 0 0) 0x7fd03c143950 con 0x7fd03c0605c0 2020-11-28T21:50:10.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.035+0000 7fd03959a700 1 -- 172.21.15.162:0/1943809467 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd01c003230 con 0x7fd03c0605c0 2020-11-28T21:50:10.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.035+0000 7fd03ad9d700 1 -- 172.21.15.162:0/1943809467 >> v2:172.21.15.118:3300/0 conn(0x7fd03c0b9ce0 msgr2=0x7fd03c1433c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:10.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.035+0000 7fd03ad9d700 1 --2- 172.21.15.162:0/1943809467 >> v2:172.21.15.118:3300/0 conn(0x7fd03c0b9ce0 0x7fd03c1433c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:10.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.035+0000 7fd03ad9d700 1 -- 172.21.15.162:0/1943809467 >> v1:172.21.15.162:6789/0 conn(0x7fd03c0605c0 legacy=0x7fd03c0b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:10.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.035+0000 7fd03ad9d700 1 -- 172.21.15.162:0/1943809467 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd03c067a70 con 0x7fd03c067610 2020-11-28T21:50:10.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.035+0000 7fd03ad9d700 1 --2- 172.21.15.162:0/1943809467 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd03c067610 0x7fd03c13fc80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fd02c0032b0 tx=0x7fd02c0091a0).ready entity=mon.2 client_cookie=397e35ceb405391c server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:10.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.035+0000 7fd03959a700 1 -- 172.21.15.162:0/1943809467 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd02c00b940 con 0x7fd03c067610 2020-11-28T21:50:10.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.035+0000 7fd03959a700 1 -- 172.21.15.162:0/1943809467 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd02c00baa0 con 0x7fd03c067610 2020-11-28T21:50:10.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.035+0000 7fd03959a700 1 -- 172.21.15.162:0/1943809467 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd02c00bc10 con 0x7fd03c067610 2020-11-28T21:50:10.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.035+0000 7fd04093f700 1 -- 172.21.15.162:0/1943809467 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd03c067610 msgr2=0x7fd03c13fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:10.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.035+0000 7fd04093f700 1 --2- 172.21.15.162:0/1943809467 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd03c067610 0x7fd03c13fc80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fd02c0032b0 tx=0x7fd02c0091a0).stop 2020-11-28T21:50:10.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.035+0000 7fd04093f700 1 -- 172.21.15.162:0/1943809467 shutdown_connections 2020-11-28T21:50:10.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.035+0000 7fd04093f700 1 --2- 172.21.15.162:0/1943809467 >> v2:172.21.15.118:3300/0 conn(0x7fd03c0b9ce0 0x7fd03c1433c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:10.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.035+0000 7fd04093f700 1 --2- 172.21.15.162:0/1943809467 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd03c067610 0x7fd03c13fc80 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:10.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.035+0000 7fd04093f700 1 -- 172.21.15.162:0/1943809467 >> 172.21.15.162:0/1943809467 conn(0x7fd03c05c430 msgr2=0x7fd03c0b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:10.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.035+0000 7fd04093f700 1 -- 172.21.15.162:0/1943809467 shutdown_connections 2020-11-28T21:50:10.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.035+0000 7fd04093f700 1 -- 172.21.15.162:0/1943809467 wait complete. 2020-11-28T21:50:10.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.035+0000 7fd04093f700 1 Processor -- start 2020-11-28T21:50:10.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.035+0000 7fd04093f700 1 -- start start 2020-11-28T21:50:10.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.039+0000 7fd04093f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd03c0605c0 0x7fd03c0b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:10.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.039+0000 7fd04093f700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd03c067610 0x7fd03c0af280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:10.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.039+0000 7fd04093f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd03c0b9ce0 0x7fd03c0af770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:10.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.039+0000 7fd04093f700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c09f2c0 con 0x7fd03c067610 2020-11-28T21:50:10.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.039+0000 7fd04093f700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c0afc60 con 0x7fd03c0605c0 2020-11-28T21:50:10.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.039+0000 7fd04093f700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c145a60 con 0x7fd03c0b9ce0 2020-11-28T21:50:10.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.039+0000 7fd03a59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd03c0605c0 0x7fd03c0b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:10.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.039+0000 7fd03a59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd03c0605c0 0x7fd03c0b06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40138/0 (socket says 172.21.15.162:40138) 2020-11-28T21:50:10.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.039+0000 7fd03a59c700 1 -- 172.21.15.162:0/2811171103 learned_addr learned my addr 172.21.15.162:0/2811171103 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:10.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.039+0000 7fd03ad9d700 1 --2- 172.21.15.162:0/2811171103 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd03c0b9ce0 0x7fd03c0af770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:10.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.039+0000 7fd039d9b700 1 --2- 172.21.15.162:0/2811171103 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd03c067610 0x7fd03c0af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:10.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.039+0000 7fd03a59c700 1 -- 172.21.15.162:0/2811171103 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd03c0b9ce0 msgr2=0x7fd03c0af770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:10.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.039+0000 7fd03a59c700 1 --2- 172.21.15.162:0/2811171103 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd03c0b9ce0 0x7fd03c0af770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:10.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.039+0000 7fd03a59c700 1 -- 172.21.15.162:0/2811171103 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd03c067610 msgr2=0x7fd03c0af280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:10.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.039+0000 7fd03a59c700 1 --2- 172.21.15.162:0/2811171103 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd03c067610 0x7fd03c0af280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:10.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.039+0000 7fd03a59c700 1 -- 172.21.15.162:0/2811171103 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd03c1457c0 con 0x7fd03c0605c0 2020-11-28T21:50:10.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.039+0000 7fd03a59c700 1 --2- 172.21.15.162:0/2811171103 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd03c0605c0 0x7fd03c0b06c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7fd03400c000 tx=0x7fd034006d60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:10.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.039+0000 7fd0337fe700 1 -- 172.21.15.162:0/2811171103 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd034005ea0 con 0x7fd03c0605c0 2020-11-28T21:50:10.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.039+0000 7fd04093f700 1 -- 172.21.15.162:0/2811171103 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd03c152280 con 0x7fd03c0605c0 2020-11-28T21:50:10.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.039+0000 7fd0337fe700 1 -- 172.21.15.162:0/2811171103 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd03400ca10 con 0x7fd03c0605c0 2020-11-28T21:50:10.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.039+0000 7fd0337fe700 1 -- 172.21.15.162:0/2811171103 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd03400cd70 con 0x7fd03c0605c0 2020-11-28T21:50:10.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.039+0000 7fd04093f700 1 -- 172.21.15.162:0/2811171103 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd03c0b0bb0 con 0x7fd03c0605c0 2020-11-28T21:50:10.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.039+0000 7fd04093f700 1 -- 172.21.15.162:0/2811171103 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd03c057050 con 0x7fd03c0605c0 2020-11-28T21:50:10.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.043+0000 7fd0337fe700 1 -- 172.21.15.162:0/2811171103 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd03401b410 con 0x7fd03c0605c0 2020-11-28T21:50:10.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.043+0000 7fd0337fe700 1 --2- 172.21.15.162:0/2811171103 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd0200554e0 0x7fd0200579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:10.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.043+0000 7fd0337fe700 1 -- 172.21.15.162:0/2811171103 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd03406df50 con 0x7fd03c0605c0 2020-11-28T21:50:10.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.043+0000 7fd039d9b700 1 --2- 172.21.15.162:0/2811171103 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd0200554e0 0x7fd0200579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:10.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.043+0000 7fd039d9b700 1 --2- 172.21.15.162:0/2811171103 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd0200554e0 0x7fd0200579a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7fd02400d000 tx=0x7fd024009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:10.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.051+0000 7fd0337fe700 1 -- 172.21.15.162:0/2811171103 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd034044150 con 0x7fd03c0605c0 2020-11-28T21:50:10.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.191+0000 7fd04093f700 1 -- 172.21.15.162:0/2811171103 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd03c068350 con 0x7fd03c0605c0 2020-11-28T21:50:10.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.199+0000 7fd0337fe700 1 -- 172.21.15.162:0/2811171103 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v56) v1 ==== 104+0+10 (secure 0 0 0) 0x7fd034047770 con 0x7fd03c0605c0 2020-11-28T21:50:10.197 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:50:10.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.199+0000 7fd04093f700 1 -- 172.21.15.162:0/2811171103 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd0200554e0 msgr2=0x7fd0200579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:10.200 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.199+0000 7fd04093f700 1 --2- 172.21.15.162:0/2811171103 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd0200554e0 0x7fd0200579a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7fd02400d000 tx=0x7fd024009040).stop 2020-11-28T21:50:10.200 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.199+0000 7fd04093f700 1 -- 172.21.15.162:0/2811171103 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd03c0605c0 msgr2=0x7fd03c0b06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:10.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.199+0000 7fd04093f700 1 --2- 172.21.15.162:0/2811171103 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd03c0605c0 0x7fd03c0b06c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7fd03400c000 tx=0x7fd034006d60).stop 2020-11-28T21:50:10.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.199+0000 7fd04093f700 1 -- 172.21.15.162:0/2811171103 shutdown_connections 2020-11-28T21:50:10.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.199+0000 7fd04093f700 1 --2- 172.21.15.162:0/2811171103 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd0200554e0 0x7fd0200579a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:10.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.199+0000 7fd04093f700 1 --2- 172.21.15.162:0/2811171103 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd03c0b9ce0 0x7fd03c0af770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:10.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.199+0000 7fd04093f700 1 --2- 172.21.15.162:0/2811171103 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd03c0605c0 0x7fd03c0b06c0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:10.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.199+0000 7fd04093f700 1 --2- 172.21.15.162:0/2811171103 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd03c067610 0x7fd03c0af280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:10.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.199+0000 7fd04093f700 1 -- 172.21.15.162:0/2811171103 >> 172.21.15.162:0/2811171103 conn(0x7fd03c05c430 msgr2=0x7fd03c0b6490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:10.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.199+0000 7fd04093f700 1 -- 172.21.15.162:0/2811171103 shutdown_connections 2020-11-28T21:50:10.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.199+0000 7fd04093f700 1 -- 172.21.15.162:0/2811171103 wait complete. 2020-11-28T21:50:10.217 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-28T21:50:10.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.339+0000 7f96d1f59700 1 Processor -- start 2020-11-28T21:50:10.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.339+0000 7f96d1f59700 1 -- start start 2020-11-28T21:50:10.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.339+0000 7f96d1f59700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f96cc0b55a0 0x7f96cc145f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:10.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.339+0000 7f96d1f59700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f96cc0ae920 0x7f96cc146480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:10.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.339+0000 7f96d1f59700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96cc0adfb0 con 0x7f96cc0afc90 2020-11-28T21:50:10.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.339+0000 7f96d1f59700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96cc0b5980 con 0x7f96cc0ae920 2020-11-28T21:50:10.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.339+0000 7f96d1f59700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96cc0ae1e0 con 0x7f96cc0b55a0 2020-11-28T21:50:10.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.339+0000 7f96cb7fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f96cc0b55a0 0x7f96cc145f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:10.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.339+0000 7f96cb7fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f96cc0b55a0 0x7f96cc145f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40146/0 (socket says 172.21.15.162:40146) 2020-11-28T21:50:10.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.339+0000 7f96cb7fe700 1 -- 172.21.15.162:0/4172787128 learned_addr learned my addr 172.21.15.162:0/4172787128 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:10.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.339+0000 7f96cb7fe700 1 -- 172.21.15.162:0/4172787128 >> v2:172.21.15.118:3300/0 conn(0x7f96cc0ae920 msgr2=0x7f96cc146480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:10.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.339+0000 7f96cbfff700 1 --2- 172.21.15.162:0/4172787128 >> v2:172.21.15.118:3300/0 conn(0x7f96cc0ae920 0x7f96cc146480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:10.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.339+0000 7f96cb7fe700 1 --2- 172.21.15.162:0/4172787128 >> v2:172.21.15.118:3300/0 conn(0x7f96cc0ae920 0x7f96cc146480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:10.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.339+0000 7f96cb7fe700 1 -- 172.21.15.162:0/4172787128 >> v1:172.21.15.118:6789/0 conn(0x7f96cc0afc90 legacy=0x7f96cc13c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:10.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.339+0000 7f96cb7fe700 1 -- 172.21.15.162:0/4172787128 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96cc0ae480 con 0x7f96cc0b55a0 2020-11-28T21:50:10.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.339+0000 7f96cbfff700 1 --2- 172.21.15.162:0/4172787128 >> v2:172.21.15.118:3300/0 conn(0x7f96cc0ae920 0x7f96cc146480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:50:10.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.339+0000 7f96cb7fe700 1 --2- 172.21.15.162:0/4172787128 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96cc0b55a0 0x7f96cc145f90 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f96cc060170 tx=0x7f96bc006e80).ready entity=mon.1 client_cookie=28e3330f14ce1c93 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:10.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.339+0000 7f96caffd700 1 -- 172.21.15.162:0/4172787128 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96bc0097b0 con 0x7f96cc0b55a0 2020-11-28T21:50:10.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.339+0000 7f96caffd700 1 -- 172.21.15.162:0/4172787128 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96bc009910 con 0x7f96cc0b55a0 2020-11-28T21:50:10.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.339+0000 7f96caffd700 1 -- 172.21.15.162:0/4172787128 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96bc009a80 con 0x7f96cc0b55a0 2020-11-28T21:50:10.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.339+0000 7f96d1f59700 1 -- 172.21.15.162:0/4172787128 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96cc0b55a0 msgr2=0x7f96cc145f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:10.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.339+0000 7f96d1f59700 1 --2- 172.21.15.162:0/4172787128 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96cc0b55a0 0x7f96cc145f90 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f96cc060170 tx=0x7f96bc006e80).stop 2020-11-28T21:50:10.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.339+0000 7f96d1f59700 1 -- 172.21.15.162:0/4172787128 shutdown_connections 2020-11-28T21:50:10.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.339+0000 7f96d1f59700 1 --2- 172.21.15.162:0/4172787128 >> v2:172.21.15.118:3300/0 conn(0x7f96cc0ae920 0x7f96cc146480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:10.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.339+0000 7f96d1f59700 1 --2- 172.21.15.162:0/4172787128 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96cc0b55a0 0x7f96cc145f90 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:10.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.339+0000 7f96d1f59700 1 -- 172.21.15.162:0/4172787128 >> 172.21.15.162:0/4172787128 conn(0x7f96cc060c80 msgr2=0x7f96cc061090 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:10.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.339+0000 7f96d1f59700 1 -- 172.21.15.162:0/4172787128 shutdown_connections 2020-11-28T21:50:10.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.339+0000 7f96d1f59700 1 -- 172.21.15.162:0/4172787128 wait complete. 2020-11-28T21:50:10.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.343+0000 7f96d1f59700 1 Processor -- start 2020-11-28T21:50:10.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.343+0000 7f96d1f59700 1 -- start start 2020-11-28T21:50:10.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.343+0000 7f96d1f59700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96cc0ae920 0x7f96cc13b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:10.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.343+0000 7f96d1f59700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f96cc0afc90 0x7f96cc142ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:10.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.343+0000 7f96d1f59700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96cc0b55a0 0x7f96cc151960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:10.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.343+0000 7f96d1f59700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96cc13a900 con 0x7f96cc0ae920 2020-11-28T21:50:10.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.343+0000 7f96d1f59700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96cc145970 con 0x7f96cc0b55a0 2020-11-28T21:50:10.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.343+0000 7f96d1f59700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96cc139ef0 con 0x7f96cc0afc90 2020-11-28T21:50:10.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.343+0000 7f96cbfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96cc0b55a0 0x7f96cc151960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:10.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.343+0000 7f96cbfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96cc0b55a0 0x7f96cc151960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40154/0 (socket says 172.21.15.162:40154) 2020-11-28T21:50:10.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.343+0000 7f96cbfff700 1 -- 172.21.15.162:0/507801490 learned_addr learned my addr 172.21.15.162:0/507801490 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:10.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.343+0000 7f96cb7fe700 1 --2- 172.21.15.162:0/507801490 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96cc0ae920 0x7f96cc13b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:10.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.343+0000 7f96c2ffd700 1 --2- 172.21.15.162:0/507801490 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f96cc0afc90 0x7f96cc142ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:10.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.343+0000 7f96cbfff700 1 -- 172.21.15.162:0/507801490 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f96cc0afc90 msgr2=0x7f96cc142ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:10.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.343+0000 7f96cbfff700 1 --2- 172.21.15.162:0/507801490 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f96cc0afc90 0x7f96cc142ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:10.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.343+0000 7f96cbfff700 1 -- 172.21.15.162:0/507801490 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96cc0ae920 msgr2=0x7f96cc13b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:10.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.343+0000 7f96cbfff700 1 --2- 172.21.15.162:0/507801490 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96cc0ae920 0x7f96cc13b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:10.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.343+0000 7f96cbfff700 1 -- 172.21.15.162:0/507801490 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96cc13ab30 con 0x7f96cc0b55a0 2020-11-28T21:50:10.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.343+0000 7f96cbfff700 1 --2- 172.21.15.162:0/507801490 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96cc0b55a0 0x7f96cc151960 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7f96cc13ef90 tx=0x7f96c4008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:10.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.343+0000 7f96c97fa700 1 -- 172.21.15.162:0/507801490 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96c400a8b0 con 0x7f96cc0b55a0 2020-11-28T21:50:10.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.343+0000 7f96c97fa700 1 -- 172.21.15.162:0/507801490 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c400aa10 con 0x7f96cc0b55a0 2020-11-28T21:50:10.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.343+0000 7f96d1f59700 1 -- 172.21.15.162:0/507801490 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96cc13add0 con 0x7f96cc0b55a0 2020-11-28T21:50:10.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.343+0000 7f96c97fa700 1 -- 172.21.15.162:0/507801490 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96c400ad70 con 0x7f96cc0b55a0 2020-11-28T21:50:10.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.343+0000 7f96d1f59700 1 -- 172.21.15.162:0/507801490 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96cc146d30 con 0x7f96cc0b55a0 2020-11-28T21:50:10.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.347+0000 7f96d1f59700 1 -- 172.21.15.162:0/507801490 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96cc057050 con 0x7f96cc0b55a0 2020-11-28T21:50:10.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.347+0000 7f96c97fa700 1 -- 172.21.15.162:0/507801490 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f96c4011410 con 0x7f96cc0b55a0 2020-11-28T21:50:10.355 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.347+0000 7f96c97fa700 1 --2- 172.21.15.162:0/507801490 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f96b0055330 0x7f96b00577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:10.356 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.347+0000 7f96c97fa700 1 -- 172.21.15.162:0/507801490 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f96c406e5c0 con 0x7f96cc0b55a0 2020-11-28T21:50:10.356 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.347+0000 7f96cb7fe700 1 --2- 172.21.15.162:0/507801490 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f96b0055330 0x7f96b00577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:10.356 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.347+0000 7f96cb7fe700 1 --2- 172.21.15.162:0/507801490 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f96b0055330 0x7f96b00577f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f96bc01a000 tx=0x7f96bc016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:10.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.351+0000 7f96c97fa700 1 -- 172.21.15.162:0/507801490 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f96c4044750 con 0x7f96cc0b55a0 2020-11-28T21:50:10.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.491+0000 7f96d1f59700 1 -- 172.21.15.162:0/507801490 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f96cc05c980 con 0x7f96cc0b55a0 2020-11-28T21:50:10.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.491+0000 7f96c97fa700 1 -- 172.21.15.162:0/507801490 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v56) v1 ==== 100+0+10 (secure 0 0 0) 0x7f96c4047d70 con 0x7f96cc0b55a0 2020-11-28T21:50:10.490 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:50:10.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.491+0000 7f96d1f59700 1 -- 172.21.15.162:0/507801490 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f96b0055330 msgr2=0x7f96b00577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:10.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.491+0000 7f96d1f59700 1 --2- 172.21.15.162:0/507801490 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f96b0055330 0x7f96b00577f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f96bc01a000 tx=0x7f96bc016000).stop 2020-11-28T21:50:10.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.491+0000 7f96d1f59700 1 -- 172.21.15.162:0/507801490 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96cc0b55a0 msgr2=0x7f96cc151960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:10.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.491+0000 7f96d1f59700 1 --2- 172.21.15.162:0/507801490 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96cc0b55a0 0x7f96cc151960 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7f96cc13ef90 tx=0x7f96c4008040).stop 2020-11-28T21:50:10.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.491+0000 7f96d1f59700 1 -- 172.21.15.162:0/507801490 shutdown_connections 2020-11-28T21:50:10.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.491+0000 7f96d1f59700 1 --2- 172.21.15.162:0/507801490 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f96b0055330 0x7f96b00577f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:10.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.491+0000 7f96d1f59700 1 --2- 172.21.15.162:0/507801490 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96cc0b55a0 0x7f96cc151960 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:10.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.491+0000 7f96d1f59700 1 --2- 172.21.15.162:0/507801490 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96cc0ae920 0x7f96cc13b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:10.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.491+0000 7f96d1f59700 1 --2- 172.21.15.162:0/507801490 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f96cc0afc90 0x7f96cc142ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:10.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.491+0000 7f96d1f59700 1 -- 172.21.15.162:0/507801490 >> 172.21.15.162:0/507801490 conn(0x7f96cc060c80 msgr2=0x7f96cc144600 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:10.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.491+0000 7f96d1f59700 1 -- 172.21.15.162:0/507801490 shutdown_connections 2020-11-28T21:50:10.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.491+0000 7f96d1f59700 1 -- 172.21.15.162:0/507801490 wait complete. 2020-11-28T21:50:10.503 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-28T21:50:10.504 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:50:10.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.619+0000 7f66e7fff700 1 Processor -- start 2020-11-28T21:50:10.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.619+0000 7f66e7fff700 1 -- start start 2020-11-28T21:50:10.622 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.623+0000 7f66e7fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f66e8064c70 0x7f66e8066290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:10.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.623+0000 7f66e7fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f66e8065750 0x7f66e813c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:10.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.623+0000 7f66e7fff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66e8065050 con 0x7f66e813f6c0 2020-11-28T21:50:10.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.623+0000 7f66e7fff700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66e813fc40 con 0x7f66e8064c70 2020-11-28T21:50:10.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.623+0000 7f66e7fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66e813fee0 con 0x7f66e8065750 2020-11-28T21:50:10.624 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.623+0000 7f66e6ffd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f66e8064c70 0x7f66e8066290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:10.624 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.623+0000 7f66e6ffd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f66e8064c70 0x7f66e8066290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40162/0 (socket says 172.21.15.162:40162) 2020-11-28T21:50:10.624 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.623+0000 7f66e6ffd700 1 -- 172.21.15.162:0/3726247566 learned_addr learned my addr 172.21.15.162:0/3726247566 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:10.624 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.623+0000 7f66e77fe700 1 --2- 172.21.15.162:0/3726247566 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f66e8065750 0x7f66e813c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:10.624 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.623+0000 7f66e6ffd700 1 -- 172.21.15.162:0/3726247566 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f66e8065750 msgr2=0x7f66e813c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:10.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.623+0000 7f66e6ffd700 1 --2- 172.21.15.162:0/3726247566 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f66e8065750 0x7f66e813c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:10.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.623+0000 7f66e6ffd700 1 -- 172.21.15.162:0/3726247566 >> v1:172.21.15.118:6789/0 conn(0x7f66e813f6c0 legacy=0x7f66e8066780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:10.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.623+0000 7f66e6ffd700 1 -- 172.21.15.162:0/3726247566 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e8140180 con 0x7f66e8064c70 2020-11-28T21:50:10.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.623+0000 7f66e6ffd700 1 --2- 172.21.15.162:0/3726247566 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66e8064c70 0x7f66e8066290 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7f66d8002f30 tx=0x7f66d8006e80).ready entity=mon.1 client_cookie=3e0e1b57281270bf server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:10.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.623+0000 7f66e5ffb700 1 -- 172.21.15.162:0/3726247566 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66d80097f0 con 0x7f66e8064c70 2020-11-28T21:50:10.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.623+0000 7f66e5ffb700 1 -- 172.21.15.162:0/3726247566 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66d8009950 con 0x7f66e8064c70 2020-11-28T21:50:10.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.623+0000 7f66e5ffb700 1 -- 172.21.15.162:0/3726247566 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66d8009cb0 con 0x7f66e8064c70 2020-11-28T21:50:10.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.623+0000 7f66e7fff700 1 -- 172.21.15.162:0/3726247566 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66e8064c70 msgr2=0x7f66e8066290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:10.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.623+0000 7f66e7fff700 1 --2- 172.21.15.162:0/3726247566 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66e8064c70 0x7f66e8066290 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7f66d8002f30 tx=0x7f66d8006e80).stop 2020-11-28T21:50:10.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.623+0000 7f66e7fff700 1 -- 172.21.15.162:0/3726247566 shutdown_connections 2020-11-28T21:50:10.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.623+0000 7f66e7fff700 1 --2- 172.21.15.162:0/3726247566 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f66e8065750 0x7f66e813c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:10.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.623+0000 7f66e7fff700 1 --2- 172.21.15.162:0/3726247566 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66e8064c70 0x7f66e8066290 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:10.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.623+0000 7f66e7fff700 1 -- 172.21.15.162:0/3726247566 >> 172.21.15.162:0/3726247566 conn(0x7f66e80b5da0 msgr2=0x7f66e80b6180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:10.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.623+0000 7f66e7fff700 1 -- 172.21.15.162:0/3726247566 shutdown_connections 2020-11-28T21:50:10.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.623+0000 7f66e7fff700 1 -- 172.21.15.162:0/3726247566 wait complete. 2020-11-28T21:50:10.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.623+0000 7f66e7fff700 1 Processor -- start 2020-11-28T21:50:10.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.627+0000 7f66e7fff700 1 -- start start 2020-11-28T21:50:10.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.627+0000 7f66e7fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f66e8064c70 0x7f66e80616a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:10.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.627+0000 7f66e7fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66e8065750 0x7f66e81392c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:10.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.627+0000 7f66e7fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66e813f6c0 0x7f66e8152f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:10.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.627+0000 7f66e7fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e81381a0 con 0x7f66e813f6c0 2020-11-28T21:50:10.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.627+0000 7f66e7fff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e8137f00 con 0x7f66e8065750 2020-11-28T21:50:10.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.627+0000 7f66e7fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66e8138b80 con 0x7f66e8064c70 2020-11-28T21:50:10.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.627+0000 7f66e67fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66e8065750 0x7f66e81392c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:10.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.627+0000 7f66e67fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66e8065750 0x7f66e81392c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40166/0 (socket says 172.21.15.162:40166) 2020-11-28T21:50:10.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.627+0000 7f66e67fc700 1 -- 172.21.15.162:0/899951079 learned_addr learned my addr 172.21.15.162:0/899951079 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:10.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.627+0000 7f66e77fe700 1 --2- 172.21.15.162:0/899951079 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66e813f6c0 0x7f66e8152f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:10.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.627+0000 7f66e6ffd700 1 --2- 172.21.15.162:0/899951079 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f66e8064c70 0x7f66e80616a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:10.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.627+0000 7f66e67fc700 1 -- 172.21.15.162:0/899951079 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f66e8064c70 msgr2=0x7f66e80616a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:10.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.627+0000 7f66e67fc700 1 --2- 172.21.15.162:0/899951079 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f66e8064c70 0x7f66e80616a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:10.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.627+0000 7f66e67fc700 1 -- 172.21.15.162:0/899951079 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66e813f6c0 msgr2=0x7f66e8152f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:10.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.627+0000 7f66e67fc700 1 --2- 172.21.15.162:0/899951079 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66e813f6c0 0x7f66e8152f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:10.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.627+0000 7f66e67fc700 1 -- 172.21.15.162:0/899951079 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e8138e20 con 0x7f66e8065750 2020-11-28T21:50:10.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.627+0000 7f66e67fc700 1 --2- 172.21.15.162:0/899951079 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66e8065750 0x7f66e81392c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7f66e805c490 tx=0x7f66d4007bd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:10.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.627+0000 7f66dffff700 1 -- 172.21.15.162:0/899951079 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66d4007240 con 0x7f66e8065750 2020-11-28T21:50:10.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.627+0000 7f66dffff700 1 -- 172.21.15.162:0/899951079 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66d40073a0 con 0x7f66e8065750 2020-11-28T21:50:10.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.627+0000 7f66e7fff700 1 -- 172.21.15.162:0/899951079 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66e8136550 con 0x7f66e8065750 2020-11-28T21:50:10.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.627+0000 7f66dffff700 1 -- 172.21.15.162:0/899951079 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66d40066d0 con 0x7f66e8065750 2020-11-28T21:50:10.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.627+0000 7f66e7fff700 1 -- 172.21.15.162:0/899951079 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66e8153600 con 0x7f66e8065750 2020-11-28T21:50:10.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.627+0000 7f66e7fff700 1 -- 172.21.15.162:0/899951079 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66e8138fb0 con 0x7f66e8065750 2020-11-28T21:50:10.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.631+0000 7f66dffff700 1 -- 172.21.15.162:0/899951079 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f66d4010030 con 0x7f66e8065750 2020-11-28T21:50:10.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.631+0000 7f66dffff700 1 --2- 172.21.15.162:0/899951079 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f66c4055440 0x7f66c4057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:10.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.631+0000 7f66dffff700 1 -- 172.21.15.162:0/899951079 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f66d406db70 con 0x7f66e8065750 2020-11-28T21:50:10.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.631+0000 7f66e6ffd700 1 --2- 172.21.15.162:0/899951079 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f66c4055440 0x7f66c4057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:10.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.631+0000 7f66e6ffd700 1 --2- 172.21.15.162:0/899951079 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f66c4055440 0x7f66c4057900 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f66d801a000 tx=0x7f66d8015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:10.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.639+0000 7f66dffff700 1 -- 172.21.15.162:0/899951079 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f66d4043e70 con 0x7f66e8065750 2020-11-28T21:50:10.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.827+0000 7f66e7fff700 1 -- 172.21.15.162:0/899951079 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66e8057050 con 0x7f66e8065750 2020-11-28T21:50:10.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.831+0000 7f66dffff700 1 -- 172.21.15.162:0/899951079 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+2501 (secure 0 0 0) 0x7f66d4047490 con 0x7f66e8065750 2020-11-28T21:50:10.828 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:50:10.829 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":21,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/1245801281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1245801281},{"type":"v1","addr":"172.21.15.118:6833","nonce":1245801281}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":18},{"gid":5536,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6835/3629429916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3629429916},{"type":"v1","addr":"172.21.15.162:6835","nonce":3629429916}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":19}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:50:05.655172+0000","modified":"2020-11-28T21:50:06.657968+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5551},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5551":{"gid":5551,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/1902721947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1902721947},{"type":"v1","addr":"172.21.15.162:6837","nonce":1902721947}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-28T21:50:10.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.831+0000 7f66e7fff700 1 -- 172.21.15.162:0/899951079 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f66c4055440 msgr2=0x7f66c4057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:10.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.831+0000 7f66e7fff700 1 --2- 172.21.15.162:0/899951079 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f66c4055440 0x7f66c4057900 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f66d801a000 tx=0x7f66d8015000).stop 2020-11-28T21:50:10.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.831+0000 7f66e7fff700 1 -- 172.21.15.162:0/899951079 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66e8065750 msgr2=0x7f66e81392c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:10.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.831+0000 7f66e7fff700 1 --2- 172.21.15.162:0/899951079 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66e8065750 0x7f66e81392c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7f66e805c490 tx=0x7f66d4007bd0).stop 2020-11-28T21:50:10.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.831+0000 7f66e7fff700 1 -- 172.21.15.162:0/899951079 shutdown_connections 2020-11-28T21:50:10.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.831+0000 7f66e7fff700 1 --2- 172.21.15.162:0/899951079 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f66c4055440 0x7f66c4057900 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:10.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.831+0000 7f66e7fff700 1 --2- 172.21.15.162:0/899951079 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f66e8064c70 0x7f66e80616a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:10.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.831+0000 7f66e7fff700 1 --2- 172.21.15.162:0/899951079 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66e813f6c0 0x7f66e8152f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:10.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.831+0000 7f66e7fff700 1 --2- 172.21.15.162:0/899951079 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66e8065750 0x7f66e81392c0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:10.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.831+0000 7f66e7fff700 1 -- 172.21.15.162:0/899951079 >> 172.21.15.162:0/899951079 conn(0x7f66e80b5da0 msgr2=0x7f66e813aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:10.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.831+0000 7f66e7fff700 1 -- 172.21.15.162:0/899951079 shutdown_connections 2020-11-28T21:50:10.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:10.831+0000 7f66e7fff700 1 -- 172.21.15.162:0/899951079 wait complete. 2020-11-28T21:50:10.834 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 21 2020-11-28T21:50:11.143 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-28T21:50:05.655172+0000', 'modified': '2020-11-28T21:50:06.657968+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5551}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5551': {'gid': 5551, 'name': 'c', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6837/1902721947', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6836', 'nonce': 1902721947}, {'type': 'v1', 'addr': '172.21.15.162:6837', 'nonce': 1902721947}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-28T21:50:11.144 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-28T21:50:11.144 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-28T21:50:11.277 INFO:teuthology.orchestra.run.smithi162.stdout:{"cluster_fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","whoami":0,"id":5551,"want_state":"up:active","state":"up:active","rank_uptime":5.6102810649999997,"mdsmap_epoch":21,"osdmap_epoch":56,"osdmap_epoch_barrier":54,"uptime":10.966701306999999} 2020-11-28T21:50:11.277 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", "id": 5551, "mdsmap_epoch": 21, "osdmap_epoch": 56, "osdmap_epoch_barrier": 54, "rank_uptime": 5.610281065, "state": "up:active", "uptime": 10.966701307, "want_state": "up:active", "whoami": 0 } 2020-11-28T21:50:11.278 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-11-28T21:50:11.278 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-11-28T21:50:11.279 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi192.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-11-28T21:50:11.279 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:50:11.280 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:50:11.287 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:50:11.344 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:50:11.345 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:50:11.401 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:50:11.402 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:50:11.403 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:50:11.404 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:50:11.451 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-11-28T21:50:11.451 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:50:11.495 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:50:11.511 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:50:11.511 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:50:11.567 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:50:11.568 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:50:11.569 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:50:11.569 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:50:11.572 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:2020-11-28T21:50:11.573+0000 7f86292da0c0 -1 init, newargv = 0x55a0acef7a10 newargc=9 2020-11-28T21:50:11.572 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:ceph-fuse[11698]: starting ceph client 2020-11-28T21:50:11.582 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:ceph-fuse[11698]: starting fuse 2020-11-28T21:50:12.677 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:50:12.694 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:50:12.695 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:50:12.754 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:50:12.756 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:50:12.756 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:50:12.757 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:50:12.804 INFO:teuthology.orchestra.run.smithi192.stdout:49 2020-11-28T21:50:12.805 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2020-11-28T21:50:12.805 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:50:12.806 INFO:teuthology.orchestra.run.smithi192:> sudo python3 -c ' 2020-11-28T21:50:12.807 INFO:teuthology.orchestra.run.smithi192:> import glob 2020-11-28T21:50:12.808 INFO:teuthology.orchestra.run.smithi192:> import re 2020-11-28T21:50:12.809 INFO:teuthology.orchestra.run.smithi192:> import os 2020-11-28T21:50:12.809 INFO:teuthology.orchestra.run.smithi192:> import subprocess 2020-11-28T21:50:12.810 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:50:12.811 INFO:teuthology.orchestra.run.smithi192:> def find_socket(client_name): 2020-11-28T21:50:12.811 INFO:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-11-28T21:50:12.813 INFO:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2020-11-28T21:50:12.813 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:50:12.814 INFO:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2020-11-28T21:50:12.815 INFO:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2020-11-28T21:50:12.816 INFO:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2020-11-28T21:50:12.816 INFO:teuthology.orchestra.run.smithi192:> return files[0] 2020-11-28T21:50:12.817 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:50:12.818 INFO:teuthology.orchestra.run.smithi192:> for f in files: 2020-11-28T21:50:12.818 INFO:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-11-28T21:50:12.819 INFO:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2020-11-28T21:50:12.820 INFO:teuthology.orchestra.run.smithi192:> return f 2020-11-28T21:50:12.821 INFO:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-11-28T21:50:12.821 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:50:12.822 INFO:teuthology.orchestra.run.smithi192:> print(find_socket("client.0")) 2020-11-28T21:50:12.823 INFO:teuthology.orchestra.run.smithi192:> ' 2020-11-28T21:50:12.896 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.0.11698.asok 2020-11-28T21:50:12.901 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.11698.asok 2020-11-28T21:50:12.902 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:50:12.902 INFO:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.11698.asok status 2020-11-28T21:50:13.008 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2020-11-28T21:50:13.008 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2020-11-28T21:50:13.009 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "a83a3572a957356977342f461361c1cd3ec06dbd", 2020-11-28T21:50:13.009 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 15.2.6-595-ga83a3572 (a83a3572a957356977342f461361c1cd3ec06dbd) octopus (stable)", 2020-11-28T21:50:13.009 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "0", 2020-11-28T21:50:13.009 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2020-11-28T21:50:13.010 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-11-28T21:50:13.010 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "11698", 2020-11-28T21:50:13.010 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2020-11-28T21:50:13.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:50:13.011 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2020-11-28T21:50:13.011 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2020-11-28T21:50:13.011 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 5210, 2020-11-28T21:50:13.011 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2020-11-28T21:50:13.013 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2020-11-28T21:50:13.013 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2020-11-28T21:50:13.013 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 5210 2020-11-28T21:50:13.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:50:13.014 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2020-11-28T21:50:13.014 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2020-11-28T21:50:13.014 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2020-11-28T21:50:13.015 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 2679763850 2020-11-28T21:50:13.015 INFO:teuthology.orchestra.run.smithi192.stdout: } 2020-11-28T21:50:13.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:50:13.016 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2020-11-28T21:50:13.016 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2020-11-28T21:50:13.017 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2020-11-28T21:50:13.017 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 2679763850 2020-11-28T21:50:13.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:50:13.018 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.5210 172.21.15.192:0/2679763850", 2020-11-28T21:50:13.018 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "172.21.15.192:0/2679763850", 2020-11-28T21:50:13.019 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2020-11-28T21:50:13.019 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 21, 2020-11-28T21:50:13.020 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 56, 2020-11-28T21:50:13.020 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2020-11-28T21:50:13.020 INFO:teuthology.orchestra.run.smithi192.stdout: "blacklisted": false 2020-11-28T21:50:13.021 INFO:teuthology.orchestra.run.smithi192.stdout:} 2020-11-28T21:50:13.023 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:50:13.023 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:50:13.024 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:50:13.030 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2020-11-28T21:50:13.032 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-28T21:50:13.033 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:50:13.034 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:50:13.091 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:50:13.091 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:50:13.092 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:50:13.140 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2020-11-28T21:50:13.142 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-28T21:50:13.143 INFO:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-28T21:50:13.143 INFO:teuthology.orchestra.run.smithi192:> import sys 2020-11-28T21:50:13.143 INFO:teuthology.orchestra.run.smithi192:> import time 2020-11-28T21:50:13.144 INFO:teuthology.orchestra.run.smithi192:> import os 2020-11-28T21:50:13.144 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:50:13.145 INFO:teuthology.orchestra.run.smithi192:> n = 1000 2020-11-28T21:50:13.145 INFO:teuthology.orchestra.run.smithi192:> abs_path = "/home/ubuntu/cephtest/mnt.0/dc-dir/dc-file" 2020-11-28T21:50:13.145 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:50:13.146 INFO:teuthology.orchestra.run.smithi192:> if not os.path.exists(os.path.dirname(abs_path)): 2020-11-28T21:50:13.146 INFO:teuthology.orchestra.run.smithi192:> os.makedirs(os.path.dirname(abs_path)) 2020-11-28T21:50:13.147 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:50:13.147 INFO:teuthology.orchestra.run.smithi192:> for i in range(0, n): 2020-11-28T21:50:13.147 INFO:teuthology.orchestra.run.smithi192:> fname = "{0}_{1}".format(abs_path, i) 2020-11-28T21:50:13.148 INFO:teuthology.orchestra.run.smithi192:> with open(fname, '"'"'w'"'"') as f: 2020-11-28T21:50:13.148 INFO:teuthology.orchestra.run.smithi192:> f.write('"'"'content'"'"') 2020-11-28T21:50:13.149 INFO:teuthology.orchestra.run.smithi192:> if True: 2020-11-28T21:50:13.149 INFO:teuthology.orchestra.run.smithi192:> f.flush() 2020-11-28T21:50:13.149 INFO:teuthology.orchestra.run.smithi192:> os.fsync(f.fileno()) 2020-11-28T21:50:13.150 INFO:teuthology.orchestra.run.smithi192:> ' 2020-11-28T21:50:25.673 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:50:25.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.791+0000 7f2d5f7c1700 1 Processor -- start 2020-11-28T21:50:25.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.791+0000 7f2d5f7c1700 1 -- start start 2020-11-28T21:50:25.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.791+0000 7f2d5f7c1700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f2d58143440 0x7f2d58060dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:25.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.791+0000 7f2d5f7c1700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2d580b10b0 0x7f2d5813bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:25.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.791+0000 7f2d5f7c1700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d58143920 con 0x7f2d580b1990 2020-11-28T21:50:25.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.791+0000 7f2d5f7c1700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d580af7e0 con 0x7f2d58143440 2020-11-28T21:50:25.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.791+0000 7f2d5f7c1700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d580b20e0 con 0x7f2d580b10b0 2020-11-28T21:50:25.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.791+0000 7f2d5dd5e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2d580b10b0 0x7f2d5813bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:25.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.791+0000 7f2d5d55d700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f2d580b1990 0x7f2d580606c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:47018/0 (socket says 172.21.15.162:47018) 2020-11-28T21:50:25.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.791+0000 7f2d5d55d700 1 -- 172.21.15.162:0/2126632251 learned_addr learned my addr 172.21.15.162:0/2126632251 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:50:25.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.791+0000 7f2d5dd5e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2d580b10b0 0x7f2d5813bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40178/0 (socket says 172.21.15.162:40178) 2020-11-28T21:50:25.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.791+0000 7f2d5cd5c700 1 --2- 172.21.15.162:0/2126632251 >> v2:172.21.15.118:3300/0 conn(0x7f2d58143440 0x7f2d58060dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:25.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.791+0000 7f2d5dd5e700 1 -- 172.21.15.162:0/2126632251 >> v2:172.21.15.118:3300/0 conn(0x7f2d58143440 msgr2=0x7f2d58060dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:25.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.791+0000 7f2d5dd5e700 1 --2- 172.21.15.162:0/2126632251 >> v2:172.21.15.118:3300/0 conn(0x7f2d58143440 0x7f2d58060dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:25.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.791+0000 7f2d5dd5e700 1 -- 172.21.15.162:0/2126632251 >> v1:172.21.15.162:6789/0 conn(0x7f2d580b1990 legacy=0x7f2d580606c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:25.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.791+0000 7f2d4ffff700 1 -- 172.21.15.162:0/2126632251 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3991185867 0 0) 0x7f2d58143920 con 0x7f2d580b1990 2020-11-28T21:50:25.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.791+0000 7f2d5dd5e700 1 -- 172.21.15.162:0/2126632251 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d58060290 con 0x7f2d580b10b0 2020-11-28T21:50:25.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.795+0000 7f2d5dd5e700 1 --2- 172.21.15.162:0/2126632251 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d580b10b0 0x7f2d5813bfd0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7f2d500032b0 tx=0x7f2d500091a0).ready entity=mon.1 client_cookie=35d309480cc14152 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:25.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.795+0000 7f2d4ffff700 1 -- 172.21.15.162:0/2126632251 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d5000b9a0 con 0x7f2d580b10b0 2020-11-28T21:50:25.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.795+0000 7f2d4ffff700 1 -- 172.21.15.162:0/2126632251 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d5000bb00 con 0x7f2d580b10b0 2020-11-28T21:50:25.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.795+0000 7f2d4ffff700 1 -- 172.21.15.162:0/2126632251 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d5000bc70 con 0x7f2d580b10b0 2020-11-28T21:50:25.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.795+0000 7f2d5f7c1700 1 -- 172.21.15.162:0/2126632251 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d580b10b0 msgr2=0x7f2d5813bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:25.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.795+0000 7f2d5f7c1700 1 --2- 172.21.15.162:0/2126632251 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d580b10b0 0x7f2d5813bfd0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7f2d500032b0 tx=0x7f2d500091a0).stop 2020-11-28T21:50:25.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.795+0000 7f2d5f7c1700 1 -- 172.21.15.162:0/2126632251 shutdown_connections 2020-11-28T21:50:25.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.795+0000 7f2d5f7c1700 1 --2- 172.21.15.162:0/2126632251 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d580b10b0 0x7f2d5813bfd0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:25.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.795+0000 7f2d5f7c1700 1 --2- 172.21.15.162:0/2126632251 >> v2:172.21.15.118:3300/0 conn(0x7f2d58143440 0x7f2d58060dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:25.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.795+0000 7f2d5f7c1700 1 -- 172.21.15.162:0/2126632251 >> 172.21.15.162:0/2126632251 conn(0x7f2d5805b970 msgr2=0x7f2d5805bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:25.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.795+0000 7f2d5f7c1700 1 -- 172.21.15.162:0/2126632251 shutdown_connections 2020-11-28T21:50:25.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.795+0000 7f2d5f7c1700 1 -- 172.21.15.162:0/2126632251 wait complete. 2020-11-28T21:50:25.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.795+0000 7f2d5f7c1700 1 Processor -- start 2020-11-28T21:50:25.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.795+0000 7f2d5f7c1700 1 -- start start 2020-11-28T21:50:25.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.799+0000 7f2d5f7c1700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2d580b10b0 0x7f2d5813c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:25.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.799+0000 7f2d5f7c1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d580b1990 0x7f2d58147320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:25.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.799+0000 7f2d5f7c1700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d58143440 0x7f2d58147810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:25.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.799+0000 7f2d5f7c1700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d58149940 con 0x7f2d58143440 2020-11-28T21:50:25.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.799+0000 7f2d5f7c1700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d58152640 con 0x7f2d580b1990 2020-11-28T21:50:25.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.799+0000 7f2d5f7c1700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d581528e0 con 0x7f2d580b10b0 2020-11-28T21:50:25.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.799+0000 7f2d5cd5c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d580b1990 0x7f2d58147320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:25.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.799+0000 7f2d5cd5c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d580b1990 0x7f2d58147320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40182/0 (socket says 172.21.15.162:40182) 2020-11-28T21:50:25.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.799+0000 7f2d5cd5c700 1 -- 172.21.15.162:0/1494633869 learned_addr learned my addr 172.21.15.162:0/1494633869 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:25.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.799+0000 7f2d5d55d700 1 --2- 172.21.15.162:0/1494633869 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2d580b10b0 0x7f2d5813c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:25.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.799+0000 7f2d5dd5e700 1 --2- 172.21.15.162:0/1494633869 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d58143440 0x7f2d58147810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:25.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.799+0000 7f2d5cd5c700 1 -- 172.21.15.162:0/1494633869 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2d580b10b0 msgr2=0x7f2d5813c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:25.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.799+0000 7f2d5cd5c700 1 --2- 172.21.15.162:0/1494633869 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2d580b10b0 0x7f2d5813c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:25.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.799+0000 7f2d5cd5c700 1 -- 172.21.15.162:0/1494633869 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d58143440 msgr2=0x7f2d58147810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:25.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.799+0000 7f2d5cd5c700 1 --2- 172.21.15.162:0/1494633869 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d58143440 0x7f2d58147810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:25.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.799+0000 7f2d5cd5c700 1 -- 172.21.15.162:0/1494633869 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d58149630 con 0x7f2d580b1990 2020-11-28T21:50:25.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.799+0000 7f2d5cd5c700 1 --2- 172.21.15.162:0/1494633869 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d580b1990 0x7f2d58147320 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f2d5400c000 tx=0x7f2d54006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:25.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.799+0000 7f2d4e7fc700 1 -- 172.21.15.162:0/1494633869 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d5400c7e0 con 0x7f2d580b1990 2020-11-28T21:50:25.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.799+0000 7f2d4e7fc700 1 -- 172.21.15.162:0/1494633869 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d5400c940 con 0x7f2d580b1990 2020-11-28T21:50:25.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.799+0000 7f2d5f7c1700 1 -- 172.21.15.162:0/1494633869 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d58149fc0 con 0x7f2d580b1990 2020-11-28T21:50:25.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.799+0000 7f2d4e7fc700 1 -- 172.21.15.162:0/1494633869 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d5400cca0 con 0x7f2d580b1990 2020-11-28T21:50:25.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.799+0000 7f2d5f7c1700 1 -- 172.21.15.162:0/1494633869 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d5813fe40 con 0x7f2d580b1990 2020-11-28T21:50:25.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.799+0000 7f2d5f7c1700 1 -- 172.21.15.162:0/1494633869 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d58152aa0 con 0x7f2d580b1990 2020-11-28T21:50:25.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.803+0000 7f2d4e7fc700 1 -- 172.21.15.162:0/1494633869 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2d5401c460 con 0x7f2d580b1990 2020-11-28T21:50:25.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.803+0000 7f2d4e7fc700 1 --2- 172.21.15.162:0/1494633869 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2d40055440 0x7f2d40057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:25.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.803+0000 7f2d4e7fc700 1 -- 172.21.15.162:0/1494633869 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2d5406e210 con 0x7f2d580b1990 2020-11-28T21:50:25.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.803+0000 7f2d5d55d700 1 --2- 172.21.15.162:0/1494633869 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2d40055440 0x7f2d40057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:25.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.803+0000 7f2d5d55d700 1 --2- 172.21.15.162:0/1494633869 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2d40055440 0x7f2d40057900 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f2d4800d000 tx=0x7f2d48001160).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:25.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.807+0000 7f2d4e7fc700 1 -- 172.21.15.162:0/1494633869 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f2d540444b0 con 0x7f2d580b1990 2020-11-28T21:50:25.996 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.999+0000 7f2d5f7c1700 1 -- 172.21.15.162:0/1494633869 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d58057050 con 0x7f2d580b1990 2020-11-28T21:50:25.997 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.999+0000 7f2d4e7fc700 1 -- 172.21.15.162:0/1494633869 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+2501 (secure 0 0 0) 0x7f2d54047ad0 con 0x7f2d580b1990 2020-11-28T21:50:25.998 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:50:25.998 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":21,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/1245801281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1245801281},{"type":"v1","addr":"172.21.15.118:6833","nonce":1245801281}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":18},{"gid":5536,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6835/3629429916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3629429916},{"type":"v1","addr":"172.21.15.162:6835","nonce":3629429916}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":19}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:50:05.655172+0000","modified":"2020-11-28T21:50:06.657968+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5551},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5551":{"gid":5551,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/1902721947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1902721947},{"type":"v1","addr":"172.21.15.162:6837","nonce":1902721947}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-28T21:50:25.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.999+0000 7f2d5f7c1700 1 -- 172.21.15.162:0/1494633869 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2d40055440 msgr2=0x7f2d40057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:25.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.999+0000 7f2d5f7c1700 1 --2- 172.21.15.162:0/1494633869 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2d40055440 0x7f2d40057900 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f2d4800d000 tx=0x7f2d48001160).stop 2020-11-28T21:50:25.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.999+0000 7f2d5f7c1700 1 -- 172.21.15.162:0/1494633869 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d580b1990 msgr2=0x7f2d58147320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:26.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:25.999+0000 7f2d5f7c1700 1 --2- 172.21.15.162:0/1494633869 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d580b1990 0x7f2d58147320 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f2d5400c000 tx=0x7f2d54006dc0).stop 2020-11-28T21:50:26.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.003+0000 7f2d5f7c1700 1 -- 172.21.15.162:0/1494633869 shutdown_connections 2020-11-28T21:50:26.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.003+0000 7f2d5f7c1700 1 --2- 172.21.15.162:0/1494633869 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2d40055440 0x7f2d40057900 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:26.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.003+0000 7f2d5f7c1700 1 --2- 172.21.15.162:0/1494633869 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2d580b10b0 0x7f2d5813c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:26.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.003+0000 7f2d5f7c1700 1 --2- 172.21.15.162:0/1494633869 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d58143440 0x7f2d58147810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:26.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.003+0000 7f2d5f7c1700 1 --2- 172.21.15.162:0/1494633869 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d580b1990 0x7f2d58147320 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:26.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.003+0000 7f2d5f7c1700 1 -- 172.21.15.162:0/1494633869 >> 172.21.15.162:0/1494633869 conn(0x7f2d5805b970 msgr2=0x7f2d58146260 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:26.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.003+0000 7f2d5f7c1700 1 -- 172.21.15.162:0/1494633869 shutdown_connections 2020-11-28T21:50:26.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.003+0000 7f2d5f7c1700 1 -- 172.21.15.162:0/1494633869 wait complete. 2020-11-28T21:50:26.002 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 21 2020-11-28T21:50:26.011 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json config set mds_recall_max_caps 20 2020-11-28T21:50:26.122 INFO:teuthology.orchestra.run.smithi162.stdout:{"success":"mds_recall_max_caps = '20' (not observed, change may require restart) "} 2020-11-28T21:50:26.122 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_caps = '20' (not observed, change may require restart) " } 2020-11-28T21:50:26.123 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:50:26.232 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.235+0000 7f2d6400d700 1 Processor -- start 2020-11-28T21:50:26.232 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.235+0000 7f2d6400d700 1 -- start start 2020-11-28T21:50:26.234 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.235+0000 7f2d6400d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f2d5c0b1990 0x7f2d5c0606c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:26.235 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.235+0000 7f2d6400d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2d5c143440 0x7f2d5c060bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:26.235 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.235+0000 7f2d6400d700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d5c143920 con 0x7f2d5c0b10b0 2020-11-28T21:50:26.235 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.235+0000 7f2d6400d700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d5c0af7e0 con 0x7f2d5c0b1990 2020-11-28T21:50:26.235 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.235+0000 7f2d6400d700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d5c0b20e0 con 0x7f2d5c143440 2020-11-28T21:50:26.236 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.235+0000 7f2d615a8700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2d5c143440 0x7f2d5c060bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:26.236 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.235+0000 7f2d615a8700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2d5c143440 0x7f2d5c060bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40190/0 (socket says 172.21.15.162:40190) 2020-11-28T21:50:26.236 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.235+0000 7f2d615a8700 1 -- 172.21.15.162:0/3504242428 learned_addr learned my addr 172.21.15.162:0/3504242428 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:26.236 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.235+0000 7f2d61da9700 1 --2- 172.21.15.162:0/3504242428 >> v2:172.21.15.118:3300/0 conn(0x7f2d5c0b1990 0x7f2d5c0606c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:26.237 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.235+0000 7f2d615a8700 1 -- 172.21.15.162:0/3504242428 >> v2:172.21.15.118:3300/0 conn(0x7f2d5c0b1990 msgr2=0x7f2d5c0606c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:26.237 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.235+0000 7f2d615a8700 1 --2- 172.21.15.162:0/3504242428 >> v2:172.21.15.118:3300/0 conn(0x7f2d5c0b1990 0x7f2d5c0606c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:26.237 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.235+0000 7f2d615a8700 1 -- 172.21.15.162:0/3504242428 >> v1:172.21.15.118:6789/0 conn(0x7f2d5c0b10b0 legacy=0x7f2d5c145fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:26.237 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.235+0000 7f2d615a8700 1 -- 172.21.15.162:0/3504242428 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d5c060290 con 0x7f2d5c143440 2020-11-28T21:50:26.237 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.235+0000 7f2d615a8700 1 --2- 172.21.15.162:0/3504242428 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d5c143440 0x7f2d5c060bb0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7f2d4c002f30 tx=0x7f2d4c006e80).ready entity=mon.1 client_cookie=81dd79a85f487350 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:26.238 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.235+0000 7f2d60da7700 1 -- 172.21.15.162:0/3504242428 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d4c0097f0 con 0x7f2d5c143440 2020-11-28T21:50:26.238 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.235+0000 7f2d60da7700 1 -- 172.21.15.162:0/3504242428 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d4c009950 con 0x7f2d5c143440 2020-11-28T21:50:26.238 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.235+0000 7f2d60da7700 1 -- 172.21.15.162:0/3504242428 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d4c009cb0 con 0x7f2d5c143440 2020-11-28T21:50:26.238 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.235+0000 7f2d6400d700 1 -- 172.21.15.162:0/3504242428 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d5c143440 msgr2=0x7f2d5c060bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:26.238 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.235+0000 7f2d6400d700 1 --2- 172.21.15.162:0/3504242428 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d5c143440 0x7f2d5c060bb0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7f2d4c002f30 tx=0x7f2d4c006e80).stop 2020-11-28T21:50:26.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.239+0000 7f2d6400d700 1 -- 172.21.15.162:0/3504242428 shutdown_connections 2020-11-28T21:50:26.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.239+0000 7f2d6400d700 1 --2- 172.21.15.162:0/3504242428 >> v2:172.21.15.118:3300/0 conn(0x7f2d5c0b1990 0x7f2d5c0606c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:26.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.239+0000 7f2d6400d700 1 --2- 172.21.15.162:0/3504242428 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d5c143440 0x7f2d5c060bb0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:26.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.239+0000 7f2d6400d700 1 -- 172.21.15.162:0/3504242428 >> 172.21.15.162:0/3504242428 conn(0x7f2d5c05b970 msgr2=0x7f2d5c05bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:26.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.239+0000 7f2d6400d700 1 -- 172.21.15.162:0/3504242428 shutdown_connections 2020-11-28T21:50:26.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.239+0000 7f2d6400d700 1 -- 172.21.15.162:0/3504242428 wait complete. 2020-11-28T21:50:26.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.239+0000 7f2d6400d700 1 Processor -- start 2020-11-28T21:50:26.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.239+0000 7f2d6400d700 1 -- start start 2020-11-28T21:50:26.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.239+0000 7f2d6400d700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2d5c0b10b0 0x7f2d5c063670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:26.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.239+0000 7f2d6400d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d5c0b1990 0x7f2d5c1407c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:26.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.239+0000 7f2d6400d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d5c143440 0x7f2d5c140cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:26.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.239+0000 7f2d6400d700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5c1554e0 con 0x7f2d5c0b1990 2020-11-28T21:50:26.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.239+0000 7f2d6400d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5c155780 con 0x7f2d5c143440 2020-11-28T21:50:26.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.239+0000 7f2d6400d700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5c155a20 con 0x7f2d5c0b10b0 2020-11-28T21:50:26.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.239+0000 7f2d625aa700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d5c143440 0x7f2d5c140cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:26.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.239+0000 7f2d61da9700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2d5c0b10b0 0x7f2d5c063670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:26.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.239+0000 7f2d61da9700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2d5c0b10b0 0x7f2d5c063670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:37418/0 (socket says 172.21.15.162:37418) 2020-11-28T21:50:26.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.239+0000 7f2d61da9700 1 -- 172.21.15.162:0/4272999823 learned_addr learned my addr 172.21.15.162:0/4272999823 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:26.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.239+0000 7f2d615a8700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d5c0b1990 0x7f2d5c1407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:26.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.239+0000 7f2d61da9700 1 -- 172.21.15.162:0/4272999823 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d5c143440 msgr2=0x7f2d5c140cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:26.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.239+0000 7f2d61da9700 1 --2- 172.21.15.162:0/4272999823 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d5c143440 0x7f2d5c140cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:26.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.239+0000 7f2d61da9700 1 -- 172.21.15.162:0/4272999823 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d5c0b1990 msgr2=0x7f2d5c1407c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:26.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.239+0000 7f2d61da9700 1 --2- 172.21.15.162:0/4272999823 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d5c0b1990 0x7f2d5c1407c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:26.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.239+0000 7f2d61da9700 1 -- 172.21.15.162:0/4272999823 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d5c155cc0 con 0x7f2d5c0b10b0 2020-11-28T21:50:26.253 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.239+0000 7f2d615a8700 1 --2- 172.21.15.162:0/4272999823 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d5c0b1990 0x7f2d5c1407c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:50:26.253 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.239+0000 7f2d625aa700 1 --2- 172.21.15.162:0/4272999823 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d5c143440 0x7f2d5c140cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:50:26.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.243+0000 7f2d61da9700 1 --2- 172.21.15.162:0/4272999823 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2d5c0b10b0 0x7f2d5c063670 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f2d58006dd0 tx=0x7f2d580091a0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:26.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.243+0000 7f2d4affd700 1 -- 172.21.15.162:0/4272999823 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d5800cb00 con 0x7f2d5c0b10b0 2020-11-28T21:50:26.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.243+0000 7f2d6400d700 1 -- 172.21.15.162:0/4272999823 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d5c1402e0 con 0x7f2d5c0b10b0 2020-11-28T21:50:26.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.243+0000 7f2d4affd700 1 -- 172.21.15.162:0/4272999823 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d5800cc60 con 0x7f2d5c0b10b0 2020-11-28T21:50:26.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.243+0000 7f2d4affd700 1 -- 172.21.15.162:0/4272999823 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d580123f0 con 0x7f2d5c0b10b0 2020-11-28T21:50:26.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.243+0000 7f2d6400d700 1 -- 172.21.15.162:0/4272999823 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d5c156160 con 0x7f2d5c0b10b0 2020-11-28T21:50:26.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.243+0000 7f2d6400d700 1 -- 172.21.15.162:0/4272999823 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d5c142130 con 0x7f2d5c0b10b0 2020-11-28T21:50:26.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.243+0000 7f2d4affd700 1 -- 172.21.15.162:0/4272999823 <== mon.2 v2:172.21.15.118:3301/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2d58022070 con 0x7f2d5c0b10b0 2020-11-28T21:50:26.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.247+0000 7f2d4affd700 1 --2- 172.21.15.162:0/4272999823 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2d3c055440 0x7f2d3c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:26.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.247+0000 7f2d4affd700 1 -- 172.21.15.162:0/4272999823 <== mon.2 v2:172.21.15.118:3301/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2d5806e710 con 0x7f2d5c0b10b0 2020-11-28T21:50:26.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.247+0000 7f2d615a8700 1 --2- 172.21.15.162:0/4272999823 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2d3c055440 0x7f2d3c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:26.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.247+0000 7f2d615a8700 1 --2- 172.21.15.162:0/4272999823 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2d3c055440 0x7f2d3c057900 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f2d4c01c000 tx=0x7f2d4c015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:26.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.255+0000 7f2d4affd700 1 -- 172.21.15.162:0/4272999823 <== mon.2 v2:172.21.15.118:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f2d5c142130 con 0x7f2d5c0b10b0 2020-11-28T21:50:26.435 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.439+0000 7f2d6400d700 1 -- 172.21.15.162:0/4272999823 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d5c142130 con 0x7f2d5c0b10b0 2020-11-28T21:50:26.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.439+0000 7f2d4affd700 1 -- 172.21.15.162:0/4272999823 <== mon.2 v2:172.21.15.118:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+2501 (secure 0 0 0) 0x7f2d5c142130 con 0x7f2d5c0b10b0 2020-11-28T21:50:26.438 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:50:26.438 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":21,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/1245801281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1245801281},{"type":"v1","addr":"172.21.15.118:6833","nonce":1245801281}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":18},{"gid":5536,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6835/3629429916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3629429916},{"type":"v1","addr":"172.21.15.162:6835","nonce":3629429916}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":19}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:50:05.655172+0000","modified":"2020-11-28T21:50:06.657968+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5551},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5551":{"gid":5551,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/1902721947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1902721947},{"type":"v1","addr":"172.21.15.162:6837","nonce":1902721947}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-28T21:50:26.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.439+0000 7f2d6400d700 1 -- 172.21.15.162:0/4272999823 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2d3c055440 msgr2=0x7f2d3c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:26.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.439+0000 7f2d6400d700 1 --2- 172.21.15.162:0/4272999823 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2d3c055440 0x7f2d3c057900 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f2d4c01c000 tx=0x7f2d4c015000).stop 2020-11-28T21:50:26.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.439+0000 7f2d6400d700 1 -- 172.21.15.162:0/4272999823 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2d5c0b10b0 msgr2=0x7f2d5c063670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:26.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.439+0000 7f2d6400d700 1 --2- 172.21.15.162:0/4272999823 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2d5c0b10b0 0x7f2d5c063670 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f2d58006dd0 tx=0x7f2d580091a0).stop 2020-11-28T21:50:26.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.439+0000 7f2d6400d700 1 -- 172.21.15.162:0/4272999823 shutdown_connections 2020-11-28T21:50:26.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.439+0000 7f2d6400d700 1 --2- 172.21.15.162:0/4272999823 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2d3c055440 0x7f2d3c057900 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:26.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.439+0000 7f2d6400d700 1 --2- 172.21.15.162:0/4272999823 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2d5c0b10b0 0x7f2d5c063670 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:26.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.439+0000 7f2d6400d700 1 --2- 172.21.15.162:0/4272999823 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d5c143440 0x7f2d5c140cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:26.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.439+0000 7f2d6400d700 1 --2- 172.21.15.162:0/4272999823 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d5c0b1990 0x7f2d5c1407c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:26.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.439+0000 7f2d6400d700 1 -- 172.21.15.162:0/4272999823 >> 172.21.15.162:0/4272999823 conn(0x7f2d5c05b970 msgr2=0x7f2d5c145f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:26.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.439+0000 7f2d6400d700 1 -- 172.21.15.162:0/4272999823 shutdown_connections 2020-11-28T21:50:26.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.439+0000 7f2d6400d700 1 -- 172.21.15.162:0/4272999823 wait complete. 2020-11-28T21:50:26.442 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 21 2020-11-28T21:50:26.451 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json config set mds_recall_max_decay_threshold 400 2020-11-28T21:50:26.566 INFO:teuthology.orchestra.run.smithi162.stdout:{"success":"mds_recall_max_decay_threshold = '400' (not observed, change may require restart) "} 2020-11-28T21:50:26.566 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_decay_threshold = '400' (not observed, change may require restart) " } 2020-11-28T21:50:26.567 INFO:tasks.cephfs.fuse_mount:Killing ceph-fuse connection on ubuntu@smithi192.front.sepia.ceph.com... 2020-11-28T21:50:26.585 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:50:26.701 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.703+0000 7fb0a9aac700 1 Processor -- start 2020-11-28T21:50:26.701 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.703+0000 7fb0a9aac700 1 -- start start 2020-11-28T21:50:26.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.703+0000 7fb0a9aac700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb0a4098b30 0x7fb0a40994f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:26.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.703+0000 7fb0a9aac700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fb0a4060710 0x7fb0a40b1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:26.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.703+0000 7fb0a9aac700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0a4098f10 con 0x7fb0a40b0000 2020-11-28T21:50:26.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.703+0000 7fb0a9aac700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0a40b1030 con 0x7fb0a4060710 2020-11-28T21:50:26.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.703+0000 7fb0a9aac700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0a40b0480 con 0x7fb0a4098b30 2020-11-28T21:50:26.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.703+0000 7fb0a8aaa700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb0a4098b30 0x7fb0a40994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:26.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.703+0000 7fb0a8aaa700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb0a4098b30 0x7fb0a40994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40202/0 (socket says 172.21.15.162:40202) 2020-11-28T21:50:26.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.703+0000 7fb0a8aaa700 1 -- 172.21.15.162:0/2318087600 learned_addr learned my addr 172.21.15.162:0/2318087600 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:26.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.703+0000 7fb0a92ab700 1 --2- 172.21.15.162:0/2318087600 >> v2:172.21.15.118:3300/0 conn(0x7fb0a4060710 0x7fb0a40b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:26.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.703+0000 7fb0a8aaa700 1 -- 172.21.15.162:0/2318087600 >> v2:172.21.15.118:3300/0 conn(0x7fb0a4060710 msgr2=0x7fb0a40b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:26.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.703+0000 7fb0a8aaa700 1 --2- 172.21.15.162:0/2318087600 >> v2:172.21.15.118:3300/0 conn(0x7fb0a4060710 0x7fb0a40b1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:26.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.703+0000 7fb0a8aaa700 1 -- 172.21.15.162:0/2318087600 >> v1:172.21.15.118:6789/0 conn(0x7fb0a40b0000 legacy=0x7fb0a4060000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:26.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.703+0000 7fb0a8aaa700 1 -- 172.21.15.162:0/2318087600 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a40b0680 con 0x7fb0a4098b30 2020-11-28T21:50:26.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.703+0000 7fb0a8aaa700 1 --2- 172.21.15.162:0/2318087600 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0a4098b30 0x7fb0a40994f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7fb08c002f30 tx=0x7fb08c006e80).ready entity=mon.1 client_cookie=706676f26c7e3523 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:26.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.703+0000 7fb0a37fe700 1 -- 172.21.15.162:0/2318087600 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb08c0097f0 con 0x7fb0a4098b30 2020-11-28T21:50:26.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.703+0000 7fb0a37fe700 1 -- 172.21.15.162:0/2318087600 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb08c009950 con 0x7fb0a4098b30 2020-11-28T21:50:26.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.703+0000 7fb0a37fe700 1 -- 172.21.15.162:0/2318087600 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb08c009cb0 con 0x7fb0a4098b30 2020-11-28T21:50:26.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.703+0000 7fb0a9aac700 1 -- 172.21.15.162:0/2318087600 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0a4098b30 msgr2=0x7fb0a40994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:26.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.703+0000 7fb0a9aac700 1 --2- 172.21.15.162:0/2318087600 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0a4098b30 0x7fb0a40994f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7fb08c002f30 tx=0x7fb08c006e80).stop 2020-11-28T21:50:26.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.707+0000 7fb0a9aac700 1 -- 172.21.15.162:0/2318087600 shutdown_connections 2020-11-28T21:50:26.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.707+0000 7fb0a9aac700 1 --2- 172.21.15.162:0/2318087600 >> v2:172.21.15.118:3300/0 conn(0x7fb0a4060710 0x7fb0a40b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:26.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.707+0000 7fb0a9aac700 1 --2- 172.21.15.162:0/2318087600 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0a4098b30 0x7fb0a40994f0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:26.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.707+0000 7fb0a9aac700 1 -- 172.21.15.162:0/2318087600 >> 172.21.15.162:0/2318087600 conn(0x7fb0a405b970 msgr2=0x7fb0a405bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:26.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.707+0000 7fb0a9aac700 1 -- 172.21.15.162:0/2318087600 shutdown_connections 2020-11-28T21:50:26.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.707+0000 7fb0a9aac700 1 -- 172.21.15.162:0/2318087600 wait complete. 2020-11-28T21:50:26.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.707+0000 7fb0a9aac700 1 Processor -- start 2020-11-28T21:50:26.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.707+0000 7fb0a9aac700 1 -- start start 2020-11-28T21:50:26.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.707+0000 7fb0a9aac700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb0a4060710 0x7fb0a40ada10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:26.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.707+0000 7fb0a9aac700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0a4098b30 0x7fb0a4159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:26.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.707+0000 7fb0a9aac700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb0a40b0000 0x7fb0a4159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:26.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.707+0000 7fb0a9aac700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a40ae330 con 0x7fb0a40b0000 2020-11-28T21:50:26.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.707+0000 7fb0a9aac700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a40ae5d0 con 0x7fb0a4098b30 2020-11-28T21:50:26.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.707+0000 7fb0a9aac700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a40ae870 con 0x7fb0a4060710 2020-11-28T21:50:26.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.707+0000 7fb0a3fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0a4098b30 0x7fb0a4159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:26.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.707+0000 7fb0a3fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0a4098b30 0x7fb0a4159520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40208/0 (socket says 172.21.15.162:40208) 2020-11-28T21:50:26.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.707+0000 7fb0a3fff700 1 -- 172.21.15.162:0/1256436965 learned_addr learned my addr 172.21.15.162:0/1256436965 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:26.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.707+0000 7fb0a8aaa700 1 --2- 172.21.15.162:0/1256436965 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb0a4060710 0x7fb0a40ada10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:26.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.707+0000 7fb0a92ab700 1 --2- 172.21.15.162:0/1256436965 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb0a40b0000 0x7fb0a4159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:26.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.707+0000 7fb0a3fff700 1 -- 172.21.15.162:0/1256436965 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb0a4060710 msgr2=0x7fb0a40ada10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:26.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.711+0000 7fb0a3fff700 1 --2- 172.21.15.162:0/1256436965 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb0a4060710 0x7fb0a40ada10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:26.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.711+0000 7fb0a3fff700 1 -- 172.21.15.162:0/1256436965 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb0a40b0000 msgr2=0x7fb0a4159a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:26.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.711+0000 7fb0a3fff700 1 --2- 172.21.15.162:0/1256436965 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb0a40b0000 0x7fb0a4159a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:26.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.711+0000 7fb0a3fff700 1 -- 172.21.15.162:0/1256436965 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a40aeb10 con 0x7fb0a4098b30 2020-11-28T21:50:26.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.711+0000 7fb0a92ab700 1 --2- 172.21.15.162:0/1256436965 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb0a40b0000 0x7fb0a4159a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:50:26.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.711+0000 7fb0a3fff700 1 --2- 172.21.15.162:0/1256436965 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0a4098b30 0x7fb0a4159520 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7fb0a4061f00 tx=0x7fb094007c20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:26.723 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.711+0000 7fb0a1ffb700 1 -- 172.21.15.162:0/1256436965 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb09400e4a0 con 0x7fb0a4098b30 2020-11-28T21:50:26.723 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.711+0000 7fb0a1ffb700 1 -- 172.21.15.162:0/1256436965 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb09400e600 con 0x7fb0a4098b30 2020-11-28T21:50:26.723 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.711+0000 7fb0a1ffb700 1 -- 172.21.15.162:0/1256436965 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb09400e960 con 0x7fb0a4098b30 2020-11-28T21:50:26.723 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.711+0000 7fb0a9aac700 1 -- 172.21.15.162:0/1256436965 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0a40aefb0 con 0x7fb0a4098b30 2020-11-28T21:50:26.723 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.711+0000 7fb0a9aac700 1 -- 172.21.15.162:0/1256436965 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0a40ab150 con 0x7fb0a4098b30 2020-11-28T21:50:26.724 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.711+0000 7fb0a9aac700 1 -- 172.21.15.162:0/1256436965 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0a40aeca0 con 0x7fb0a4098b30 2020-11-28T21:50:26.724 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.711+0000 7fb0a1ffb700 1 -- 172.21.15.162:0/1256436965 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb094010030 con 0x7fb0a4098b30 2020-11-28T21:50:26.724 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.711+0000 7fb0a1ffb700 1 --2- 172.21.15.162:0/1256436965 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb07c055440 0x7fb07c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:26.724 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.711+0000 7fb0a8aaa700 1 --2- 172.21.15.162:0/1256436965 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb07c055440 0x7fb07c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:26.725 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.711+0000 7fb0a1ffb700 1 -- 172.21.15.162:0/1256436965 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb09406dea0 con 0x7fb0a4098b30 2020-11-28T21:50:26.725 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.711+0000 7fb0a8aaa700 1 --2- 172.21.15.162:0/1256436965 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb07c055440 0x7fb07c057900 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7fb08c01a000 tx=0x7fb08c015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:26.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.719+0000 7fb0a1ffb700 1 -- 172.21.15.162:0/1256436965 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb094044090 con 0x7fb0a4098b30 2020-11-28T21:50:26.904 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.903+0000 7fb0a9aac700 1 -- 172.21.15.162:0/1256436965 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0a4057050 con 0x7fb0a4098b30 2020-11-28T21:50:26.904 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.907+0000 7fb0a1ffb700 1 -- 172.21.15.162:0/1256436965 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+2501 (secure 0 0 0) 0x7fb0940476b0 con 0x7fb0a4098b30 2020-11-28T21:50:26.905 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:50:26.905 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":21,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/1245801281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1245801281},{"type":"v1","addr":"172.21.15.118:6833","nonce":1245801281}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":18},{"gid":5536,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6835/3629429916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3629429916},{"type":"v1","addr":"172.21.15.162:6835","nonce":3629429916}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":19}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:50:05.655172+0000","modified":"2020-11-28T21:50:06.657968+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5551},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5551":{"gid":5551,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/1902721947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1902721947},{"type":"v1","addr":"172.21.15.162:6837","nonce":1902721947}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-28T21:50:26.907 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.907+0000 7fb0a9aac700 1 -- 172.21.15.162:0/1256436965 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb07c055440 msgr2=0x7fb07c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:26.907 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.907+0000 7fb0a9aac700 1 --2- 172.21.15.162:0/1256436965 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb07c055440 0x7fb07c057900 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7fb08c01a000 tx=0x7fb08c015000).stop 2020-11-28T21:50:26.908 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.907+0000 7fb0a9aac700 1 -- 172.21.15.162:0/1256436965 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0a4098b30 msgr2=0x7fb0a4159520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:26.908 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.907+0000 7fb0a9aac700 1 --2- 172.21.15.162:0/1256436965 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0a4098b30 0x7fb0a4159520 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7fb0a4061f00 tx=0x7fb094007c20).stop 2020-11-28T21:50:26.909 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.907+0000 7fb0a9aac700 1 -- 172.21.15.162:0/1256436965 shutdown_connections 2020-11-28T21:50:26.909 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.907+0000 7fb0a9aac700 1 --2- 172.21.15.162:0/1256436965 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb07c055440 0x7fb07c057900 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:26.910 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.907+0000 7fb0a9aac700 1 --2- 172.21.15.162:0/1256436965 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb0a4060710 0x7fb0a40ada10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:26.910 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.907+0000 7fb0a9aac700 1 --2- 172.21.15.162:0/1256436965 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb0a40b0000 0x7fb0a4159a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:26.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.907+0000 7fb0a9aac700 1 --2- 172.21.15.162:0/1256436965 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0a4098b30 0x7fb0a4159520 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:26.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.907+0000 7fb0a9aac700 1 -- 172.21.15.162:0/1256436965 >> 172.21.15.162:0/1256436965 conn(0x7fb0a405b970 msgr2=0x7fb0a40b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:26.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.907+0000 7fb0a9aac700 1 -- 172.21.15.162:0/1256436965 shutdown_connections 2020-11-28T21:50:26.912 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:26.907+0000 7fb0a9aac700 1 -- 172.21.15.162:0/1256436965 wait complete. 2020-11-28T21:50:26.912 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 21 2020-11-28T21:50:26.918 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell mds.c cache drop 5 2020-11-28T21:50:27.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.031+0000 7f63e8e6c700 1 Processor -- start 2020-11-28T21:50:27.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.031+0000 7f63e8e6c700 1 -- start start 2020-11-28T21:50:27.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.031+0000 7f63e8e6c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f63e41433c0 0x7f63e40605c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:27.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.031+0000 7f63e8e6c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f63e4067610 0x7f63e413fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:27.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.031+0000 7f63e8e6c700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63e4143950 con 0x7f63e40b9ce0 2020-11-28T21:50:27.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.031+0000 7f63e8e6c700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63e40b4340 con 0x7f63e41433c0 2020-11-28T21:50:27.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.031+0000 7f63e8e6c700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63e4067ac0 con 0x7f63e4067610 2020-11-28T21:50:27.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.035+0000 7f63e2d9d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f63e4067610 0x7f63e413fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:27.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.035+0000 7f63e2d9d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f63e4067610 0x7f63e413fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40218/0 (socket says 172.21.15.162:40218) 2020-11-28T21:50:27.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.035+0000 7f63e2d9d700 1 -- 172.21.15.162:0/586471796 learned_addr learned my addr 172.21.15.162:0/586471796 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:27.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.035+0000 7f63e1d9b700 1 --2- 172.21.15.162:0/586471796 >> v2:172.21.15.118:3300/0 conn(0x7f63e41433c0 0x7f63e40605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:27.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.035+0000 7f63e2d9d700 1 -- 172.21.15.162:0/586471796 >> v2:172.21.15.118:3300/0 conn(0x7f63e41433c0 msgr2=0x7f63e40605c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:27.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.035+0000 7f63e2d9d700 1 --2- 172.21.15.162:0/586471796 >> v2:172.21.15.118:3300/0 conn(0x7f63e41433c0 0x7f63e40605c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:27.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.035+0000 7f63e2d9d700 1 -- 172.21.15.162:0/586471796 >> v1:172.21.15.118:6789/0 conn(0x7f63e40b9ce0 legacy=0x7f63e40b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:27.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.035+0000 7f63e2d9d700 1 -- 172.21.15.162:0/586471796 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e40ba0c0 con 0x7f63e4067610 2020-11-28T21:50:27.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.035+0000 7f63e2d9d700 1 --2- 172.21.15.162:0/586471796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63e4067610 0x7f63e413fc80 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7f63dc0032b0 tx=0x7f63dc009270).ready entity=mon.1 client_cookie=614085c4a11be44d server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:27.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.035+0000 7f63e159a700 1 -- 172.21.15.162:0/586471796 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63dc00bc00 con 0x7f63e4067610 2020-11-28T21:50:27.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.035+0000 7f63e159a700 1 -- 172.21.15.162:0/586471796 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63dc00bd60 con 0x7f63e4067610 2020-11-28T21:50:27.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.035+0000 7f63e159a700 1 -- 172.21.15.162:0/586471796 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63dc011400 con 0x7f63e4067610 2020-11-28T21:50:27.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.035+0000 7f63e8e6c700 1 -- 172.21.15.162:0/586471796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63e4067610 msgr2=0x7f63e413fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:27.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.035+0000 7f63e8e6c700 1 --2- 172.21.15.162:0/586471796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63e4067610 0x7f63e413fc80 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7f63dc0032b0 tx=0x7f63dc009270).stop 2020-11-28T21:50:27.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.035+0000 7f63e8e6c700 1 -- 172.21.15.162:0/586471796 shutdown_connections 2020-11-28T21:50:27.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.035+0000 7f63e8e6c700 1 --2- 172.21.15.162:0/586471796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63e4067610 0x7f63e413fc80 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:27.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.035+0000 7f63e8e6c700 1 --2- 172.21.15.162:0/586471796 >> v2:172.21.15.118:3300/0 conn(0x7f63e41433c0 0x7f63e40605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:27.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.035+0000 7f63e8e6c700 1 -- 172.21.15.162:0/586471796 >> 172.21.15.162:0/586471796 conn(0x7f63e405c430 msgr2=0x7f63e40b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:27.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.035+0000 7f63e8e6c700 1 -- 172.21.15.162:0/586471796 shutdown_connections 2020-11-28T21:50:27.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.035+0000 7f63e8e6c700 1 -- 172.21.15.162:0/586471796 wait complete. 2020-11-28T21:50:27.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.035+0000 7f63e8e6c700 1 Processor -- start 2020-11-28T21:50:27.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.035+0000 7f63e8e6c700 1 -- start start 2020-11-28T21:50:27.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.039+0000 7f63e8e6c700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f63e4067610 0x7f63e41461a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:27.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.039+0000 7f63e8e6c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63e40b9ce0 0x7f63e4146690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:27.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.039+0000 7f63e8e6c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f63e41433c0 0x7f63e4155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:27.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.039+0000 7f63e8e6c700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e40b0800 con 0x7f63e41433c0 2020-11-28T21:50:27.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.039+0000 7f63e8e6c700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e40b2a30 con 0x7f63e40b9ce0 2020-11-28T21:50:27.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.039+0000 7f63e8e6c700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63e40b2cd0 con 0x7f63e4067610 2020-11-28T21:50:27.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.039+0000 7f63e1d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63e40b9ce0 0x7f63e4146690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:27.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.039+0000 7f63e1d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63e40b9ce0 0x7f63e4146690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40224/0 (socket says 172.21.15.162:40224) 2020-11-28T21:50:27.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.039+0000 7f63e1d9b700 1 -- 172.21.15.162:0/3429216124 learned_addr learned my addr 172.21.15.162:0/3429216124 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:27.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.039+0000 7f63e259c700 1 --2- 172.21.15.162:0/3429216124 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f63e4067610 0x7f63e41461a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:27.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.039+0000 7f63e2d9d700 1 --2- 172.21.15.162:0/3429216124 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f63e41433c0 0x7f63e4155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:27.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.039+0000 7f63e1d9b700 1 -- 172.21.15.162:0/3429216124 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f63e4067610 msgr2=0x7f63e41461a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:27.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.039+0000 7f63e1d9b700 1 --2- 172.21.15.162:0/3429216124 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f63e4067610 0x7f63e41461a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:27.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.039+0000 7f63e1d9b700 1 -- 172.21.15.162:0/3429216124 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f63e41433c0 msgr2=0x7f63e4155190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:27.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.039+0000 7f63e1d9b700 1 --2- 172.21.15.162:0/3429216124 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f63e41433c0 0x7f63e4155190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:27.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.039+0000 7f63e1d9b700 1 -- 172.21.15.162:0/3429216124 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e40b2f70 con 0x7f63e40b9ce0 2020-11-28T21:50:27.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.039+0000 7f63e1d9b700 1 --2- 172.21.15.162:0/3429216124 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63e40b9ce0 0x7f63e4146690 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f63cc003070 tx=0x7f63cc008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:27.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.039+0000 7f63db7fe700 1 -- 172.21.15.162:0/3429216124 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63cc011780 con 0x7f63e40b9ce0 2020-11-28T21:50:27.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.039+0000 7f63db7fe700 1 -- 172.21.15.162:0/3429216124 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63cc0118e0 con 0x7f63e40b9ce0 2020-11-28T21:50:27.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.039+0000 7f63db7fe700 1 -- 172.21.15.162:0/3429216124 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63cc011c40 con 0x7f63e40b9ce0 2020-11-28T21:50:27.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.039+0000 7f63e8e6c700 1 -- 172.21.15.162:0/3429216124 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63e40b0b50 con 0x7f63e40b9ce0 2020-11-28T21:50:27.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.039+0000 7f63e8e6c700 1 -- 172.21.15.162:0/3429216124 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63e4155680 con 0x7f63e40b9ce0 2020-11-28T21:50:27.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.043+0000 7f63db7fe700 1 -- 172.21.15.162:0/3429216124 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f63cc01c460 con 0x7f63e40b9ce0 2020-11-28T21:50:27.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.043+0000 7f63db7fe700 1 --2- 172.21.15.162:0/3429216124 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f63c80554e0 0x7f63c80579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:27.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.043+0000 7f63db7fe700 1 -- 172.21.15.162:0/3429216124 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f63cc06e270 con 0x7f63e40b9ce0 2020-11-28T21:50:27.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.043+0000 7f63e259c700 1 --2- 172.21.15.162:0/3429216124 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f63c80554e0 0x7f63c80579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:27.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.043+0000 7f63e259c700 1 --2- 172.21.15.162:0/3429216124 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f63c80554e0 0x7f63c80579a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f63d400d000 tx=0x7f63d4009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:27.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.043+0000 7f63f7807740 1 Processor -- start 2020-11-28T21:50:27.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.043+0000 7f63f7807740 1 -- start start 2020-11-28T21:50:27.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.047+0000 7f63f7807740 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0xfddf20 0x1026130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:27.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.047+0000 7f63f7807740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0xe46820 0x1060430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:27.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.047+0000 7f63f7807740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xdd8c20 0x1060920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:27.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.047+0000 7f63f7807740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0xf52df0 con 0xe46820 2020-11-28T21:50:27.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.047+0000 7f63f7807740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0xee3250 con 0xdd8c20 2020-11-28T21:50:27.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.047+0000 7f63f7807740 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0xe5f6b0 con 0xfddf20 2020-11-28T21:50:27.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.047+0000 7f63e2d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xdd8c20 0x1060920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:27.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.047+0000 7f63e259c700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0xfddf20 0x1026130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:27.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.047+0000 7f63e2d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xdd8c20 0x1060920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40232/0 (socket says 172.21.15.162:40232) 2020-11-28T21:50:27.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.047+0000 7f63e2d9d700 1 -- 172.21.15.162:0/2191736161 learned_addr learned my addr 172.21.15.162:0/2191736161 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:27.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.047+0000 7f63e1d9b700 1 --2- 172.21.15.162:0/2191736161 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0xe46820 0x1060430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:27.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.047+0000 7f63e2d9d700 1 -- 172.21.15.162:0/2191736161 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0xfddf20 msgr2=0x1026130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:27.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.047+0000 7f63e2d9d700 1 --2- 172.21.15.162:0/2191736161 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0xfddf20 0x1026130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:27.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.047+0000 7f63e2d9d700 1 -- 172.21.15.162:0/2191736161 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0xe46820 msgr2=0x1060430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:27.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.047+0000 7f63e2d9d700 1 --2- 172.21.15.162:0/2191736161 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0xe46820 0x1060430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:27.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.047+0000 7f63e2d9d700 1 -- 172.21.15.162:0/2191736161 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xf73d10 con 0xdd8c20 2020-11-28T21:50:27.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.047+0000 7f63e1d9b700 1 --2- 172.21.15.162:0/2191736161 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0xe46820 0x1060430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:50:27.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.047+0000 7f63e2d9d700 1 --2- 172.21.15.162:0/2191736161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xdd8c20 0x1060920 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7f63dc01a000 tx=0x7f63dc016000).ready entity=mon.1 client_cookie=4923746348cf9f3c server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:27.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.047+0000 7f63e8e6c700 1 -- 172.21.15.162:0/2191736161 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63dc011670 con 0xdd8c20 2020-11-28T21:50:27.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.047+0000 7f63e8e6c700 1 -- 172.21.15.162:0/2191736161 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63dc0117d0 con 0xdd8c20 2020-11-28T21:50:27.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.047+0000 7f63e8e6c700 1 -- 172.21.15.162:0/2191736161 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63dc011b30 con 0xdd8c20 2020-11-28T21:50:27.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.047+0000 7f63f7807740 1 -- 172.21.15.162:0/2191736161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xdd8c20 msgr2=0x1060920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:27.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.047+0000 7f63f7807740 1 --2- 172.21.15.162:0/2191736161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xdd8c20 0x1060920 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7f63dc01a000 tx=0x7f63dc016000).stop 2020-11-28T21:50:27.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.047+0000 7f63f7807740 1 -- 172.21.15.162:0/2191736161 shutdown_connections 2020-11-28T21:50:27.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.047+0000 7f63f7807740 1 --2- 172.21.15.162:0/2191736161 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0xfddf20 0x1026130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:27.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.047+0000 7f63f7807740 1 --2- 172.21.15.162:0/2191736161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xdd8c20 0x1060920 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:27.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.047+0000 7f63f7807740 1 --2- 172.21.15.162:0/2191736161 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0xe46820 0x1060430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:27.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.047+0000 7f63f7807740 1 -- 172.21.15.162:0/2191736161 >> 172.21.15.162:0/2191736161 conn(0x100d6a0 msgr2=0x100c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:27.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.047+0000 7f63f7807740 1 -- 172.21.15.162:0/2191736161 shutdown_connections 2020-11-28T21:50:27.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.047+0000 7f63f7807740 1 -- 172.21.15.162:0/2191736161 wait complete. 2020-11-28T21:50:27.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.051+0000 7f63f7807740 1 -- start start 2020-11-28T21:50:27.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.051+0000 7f63f7807740 1 Processor -- start 2020-11-28T21:50:27.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.051+0000 7f63f7807740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0xdd8c20 0x1063210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:27.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.051+0000 7f63f7807740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xe46820 0x1053a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:27.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.051+0000 7f63f7807740 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0xfddf20 0x1053f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:27.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.051+0000 7f63f7807740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0xf82a70 con 0xdd8c20 2020-11-28T21:50:27.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.051+0000 7f63f7807740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0xf82bb0 con 0xe46820 2020-11-28T21:50:27.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.051+0000 7f63f7807740 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0xf84180 con 0xfddf20 2020-11-28T21:50:27.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.051+0000 7f63e1d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xe46820 0x1053a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:27.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.051+0000 7f63f7807740 10 client.0 ll_register_callbacks cb 0x100d600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-28T21:50:27.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.051+0000 7f63e1d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xe46820 0x1053a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40236/0 (socket says 172.21.15.162:40236) 2020-11-28T21:50:27.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.051+0000 7f63e1d9b700 1 -- 172.21.15.162:0/1262075238 learned_addr learned my addr 172.21.15.162:0/1262075238 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:27.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.051+0000 7f63e259c700 1 --2- 172.21.15.162:0/1262075238 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0xdd8c20 0x1063210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:27.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.051+0000 7f63e2d9d700 1 --2- 172.21.15.162:0/1262075238 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0xfddf20 0x1053f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:27.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.051+0000 7f63e1d9b700 1 -- 172.21.15.162:0/1262075238 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0xfddf20 msgr2=0x1053f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:27.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.051+0000 7f63e1d9b700 1 --2- 172.21.15.162:0/1262075238 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0xfddf20 0x1053f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:27.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.051+0000 7f63e1d9b700 1 -- 172.21.15.162:0/1262075238 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0xdd8c20 msgr2=0x1063210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:27.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.051+0000 7f63e1d9b700 1 --2- 172.21.15.162:0/1262075238 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0xdd8c20 0x1063210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:27.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.051+0000 7f63e1d9b700 1 -- 172.21.15.162:0/1262075238 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0xdf6790 con 0xe46820 2020-11-28T21:50:27.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.051+0000 7f63e1d9b700 1 --2- 172.21.15.162:0/1262075238 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xe46820 0x1053a10 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x10623a0 tx=0x7f63cc000b80).ready entity=mon.1 client_cookie=1e65cc40e0f2ae9f server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:27.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.051+0000 7f63c17fa700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2020-11-28T21:50:27.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.055+0000 7f63c17fa700 1 -- 172.21.15.162:0/1262075238 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63cc047280 con 0xe46820 2020-11-28T21:50:27.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.055+0000 7f63f7807740 1 -- 172.21.15.162:0/1262075238 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xf33810 con 0xe46820 2020-11-28T21:50:27.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.055+0000 7f63c17fa700 1 -- 172.21.15.162:0/1262075238 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f63cc0473e0 con 0xe46820 2020-11-28T21:50:27.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.055+0000 7f63c17fa700 1 -- 172.21.15.162:0/1262075238 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63cc06ca40 con 0xe46820 2020-11-28T21:50:27.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.055+0000 7f63f7807740 10 client.5734 fetch_fsmap learned FSMap version 21 2020-11-28T21:50:27.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.055+0000 7f63f7807740 1 -- 172.21.15.162:0/1262075238 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0xe01e70 con 0xe46820 2020-11-28T21:50:27.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.055+0000 7f63c17fa700 1 -- 172.21.15.162:0/1262075238 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xf33810 con 0xe46820 2020-11-28T21:50:27.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.055+0000 7f63c17fa700 1 -- 172.21.15.162:0/1262075238 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 21) v1 ==== 1434+0+0 (secure 0 0 0) 0x7f63cc06cba0 con 0xe46820 2020-11-28T21:50:27.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.055+0000 7f63f7807740 10 client.5734 fetch_fsmap finished waiting for FSMap version 21 2020-11-28T21:50:27.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.055+0000 7f63f7807740 10 client.5734 resolve_mds: resolved ID 'c' to GID 5551 2020-11-28T21:50:27.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.055+0000 7f63f7807740 20 client.5734 populate_metadata read hostname 'smithi162' 2020-11-28T21:50:27.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.055+0000 7f63f7807740 1 --2- 172.21.15.162:0/1262075238 >> [v2:172.21.15.162:6836/1902721947,v1:172.21.15.162:6837/1902721947] conn(0x1063700 0x106f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:27.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.055+0000 7f63f7807740 4 client.5734 mds_command: new command op to 5551 tid=0[{"prefix": "get_command_descriptions"}] 2020-11-28T21:50:27.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.055+0000 7f63f7807740 1 -- 172.21.15.162:0/1262075238 --> [v2:172.21.15.162:6836/1902721947,v1:172.21.15.162:6837/1902721947] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0xe6b8d0 con 0x1063700 2020-11-28T21:50:27.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.055+0000 7f63e259c700 1 --2- 172.21.15.162:0/1262075238 >> [v2:172.21.15.162:6836/1902721947,v1:172.21.15.162:6837/1902721947] conn(0x1063700 0x106f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:27.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.055+0000 7f63c17fa700 10 client.5734 ms_handle_connect on v2:172.21.15.162:6836/1902721947 2020-11-28T21:50:27.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.055+0000 7f63e259c700 1 --2- 172.21.15.162:0/1262075238 >> [v2:172.21.15.162:6836/1902721947,v1:172.21.15.162:6837/1902721947] conn(0x1063700 0x106f3a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=70d04dd1340a2d60 server_cookie=92f94e668f317f6e in_seq=0 out_seq=0 2020-11-28T21:50:27.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.055+0000 7f63c17fa700 1 -- 172.21.15.162:0/1262075238 <== mds.0 v2:172.21.15.162:6836/1902721947 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16606 (crc 0 0 0) 0xe6b8d0 con 0x1063700 2020-11-28T21:50:27.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.055+0000 7f63c17fa700 10 client.5734 handle_command_reply: tid=0 2020-11-28T21:50:27.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.055+0000 7f63f7807740 1 client.5734 shutdown 2020-11-28T21:50:27.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.055+0000 7f63f7807740 1 -- 172.21.15.162:0/1262075238 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xe46820 msgr2=0x1053a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:27.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.055+0000 7f63f7807740 1 --2- 172.21.15.162:0/1262075238 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xe46820 0x1053a10 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x10623a0 tx=0x7f63cc000b80).stop 2020-11-28T21:50:27.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.055+0000 7f63f7807740 1 -- 172.21.15.162:0/1262075238 shutdown_connections 2020-11-28T21:50:27.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.055+0000 7f63f7807740 1 --2- 172.21.15.162:0/1262075238 >> [v2:172.21.15.162:6836/1902721947,v1:172.21.15.162:6837/1902721947] conn(0x1063700 0x106f3a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:27.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.055+0000 7f63c17fa700 0 client.5734 ms_handle_reset on v2:172.21.15.162:6836/1902721947 2020-11-28T21:50:27.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.055+0000 7f63f7807740 1 --2- 172.21.15.162:0/1262075238 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0xfddf20 0x1053f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:27.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.055+0000 7f63f7807740 1 --2- 172.21.15.162:0/1262075238 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0xdd8c20 0x1063210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:27.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.055+0000 7f63f7807740 1 --2- 172.21.15.162:0/1262075238 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xe46820 0x1053a10 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:27.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.055+0000 7f63f7807740 1 -- 172.21.15.162:0/1262075238 >> 172.21.15.162:0/1262075238 conn(0x100d6a0 msgr2=0x10591a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:27.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.055+0000 7f63f7807740 1 -- 172.21.15.162:0/1262075238 shutdown_connections 2020-11-28T21:50:27.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.055+0000 7f63f7807740 1 -- 172.21.15.162:0/1262075238 wait complete. 2020-11-28T21:50:27.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.055+0000 7f63f7807740 20 client.5734 trim_cache size 0 max 16384 2020-11-28T21:50:27.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.067+0000 7f63f7807740 1 Processor -- start 2020-11-28T21:50:27.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.067+0000 7f63f7807740 1 -- start start 2020-11-28T21:50:27.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.067+0000 7f63f7807740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0xdd8c20 0x10273f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:27.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.067+0000 7f63f7807740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xe46820 0x106bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:27.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.067+0000 7f63f7807740 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0xfddf20 0x106c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:27.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.067+0000 7f63f7807740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0xf65510 con 0xdd8c20 2020-11-28T21:50:27.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.067+0000 7f63f7807740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x102a1f0 con 0xe46820 2020-11-28T21:50:27.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.067+0000 7f63f7807740 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x1061da0 con 0xfddf20 2020-11-28T21:50:27.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.067+0000 7f63e1d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xe46820 0x106bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:27.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.067+0000 7f63e1d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xe46820 0x106bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40246/0 (socket says 172.21.15.162:40246) 2020-11-28T21:50:27.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.067+0000 7f63e1d9b700 1 -- 172.21.15.162:0/2191494308 learned_addr learned my addr 172.21.15.162:0/2191494308 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:27.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.067+0000 7f63e2d9d700 1 --2- 172.21.15.162:0/2191494308 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0xfddf20 0x106c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:27.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.071+0000 7f63e259c700 1 --2- 172.21.15.162:0/2191494308 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0xdd8c20 0x10273f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:27.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.071+0000 7f63e1d9b700 1 -- 172.21.15.162:0/2191494308 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0xfddf20 msgr2=0x106c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:27.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.071+0000 7f63e1d9b700 1 --2- 172.21.15.162:0/2191494308 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0xfddf20 0x106c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:27.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.071+0000 7f63e1d9b700 1 -- 172.21.15.162:0/2191494308 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0xdd8c20 msgr2=0x10273f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:27.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.071+0000 7f63e1d9b700 1 --2- 172.21.15.162:0/2191494308 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0xdd8c20 0x10273f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:27.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.071+0000 7f63e1d9b700 1 -- 172.21.15.162:0/2191494308 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e40b7650 con 0xe46820 2020-11-28T21:50:27.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.071+0000 7f63e1d9b700 1 --2- 172.21.15.162:0/2191494308 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xe46820 0x106bde0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x10623a0 tx=0x7f63cc026000).ready entity=mon.1 client_cookie=29a83f93a32b892a server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:27.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.071+0000 7f63c37fe700 1 -- 172.21.15.162:0/2191494308 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63cc06e0b0 con 0xe46820 2020-11-28T21:50:27.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.071+0000 7f63c37fe700 1 -- 172.21.15.162:0/2191494308 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63cc047ea0 con 0xe46820 2020-11-28T21:50:27.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.071+0000 7f63c37fe700 1 -- 172.21.15.162:0/2191494308 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63cc046df0 con 0xe46820 2020-11-28T21:50:27.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.071+0000 7f63f7807740 1 -- 172.21.15.162:0/2191494308 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xe46820 msgr2=0x106bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:27.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.071+0000 7f63f7807740 1 --2- 172.21.15.162:0/2191494308 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xe46820 0x106bde0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x10623a0 tx=0x7f63cc026000).stop 2020-11-28T21:50:27.083 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.071+0000 7f63f7807740 1 -- 172.21.15.162:0/2191494308 shutdown_connections 2020-11-28T21:50:27.083 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.071+0000 7f63f7807740 1 --2- 172.21.15.162:0/2191494308 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0xfddf20 0x106c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:27.083 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.071+0000 7f63f7807740 1 --2- 172.21.15.162:0/2191494308 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0xdd8c20 0x10273f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:27.083 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.071+0000 7f63f7807740 1 --2- 172.21.15.162:0/2191494308 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xe46820 0x106bde0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:27.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.071+0000 7f63f7807740 1 -- 172.21.15.162:0/2191494308 >> 172.21.15.162:0/2191494308 conn(0x100d6a0 msgr2=0x10266b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:27.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.071+0000 7f63f7807740 1 -- 172.21.15.162:0/2191494308 shutdown_connections 2020-11-28T21:50:27.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.071+0000 7f63f7807740 1 -- 172.21.15.162:0/2191494308 wait complete. 2020-11-28T21:50:27.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.071+0000 7f63f7807740 1 -- start start 2020-11-28T21:50:27.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.071+0000 7f63f7807740 1 Processor -- start 2020-11-28T21:50:27.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.071+0000 7f63f7807740 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0xdd8c20 0x1028200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:27.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.071+0000 7f63f7807740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0xe46820 0x10c3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:27.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.071+0000 7f63f7807740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xfddf20 0x10c6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:27.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.071+0000 7f63f7807740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x10262c0 con 0xe46820 2020-11-28T21:50:27.086 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.071+0000 7f63f7807740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x102a540 con 0xfddf20 2020-11-28T21:50:27.086 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.071+0000 7f63f7807740 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0xf6baf0 con 0xdd8c20 2020-11-28T21:50:27.086 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.071+0000 7f63f7807740 10 client.0 ll_register_callbacks cb 0x10726a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-28T21:50:27.086 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.071+0000 7f63e2d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xfddf20 0x10c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:27.086 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.071+0000 7f63e2d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xfddf20 0x10c6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40252/0 (socket says 172.21.15.162:40252) 2020-11-28T21:50:27.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.071+0000 7f63e2d9d700 1 -- 172.21.15.162:0/2900910146 learned_addr learned my addr 172.21.15.162:0/2900910146 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:27.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.071+0000 7f63e259c700 1 --2- 172.21.15.162:0/2900910146 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0xdd8c20 0x1028200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:27.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.071+0000 7f63e1d9b700 1 --2- 172.21.15.162:0/2900910146 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0xe46820 0x10c3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:27.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.071+0000 7f63e2d9d700 1 -- 172.21.15.162:0/2900910146 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0xdd8c20 msgr2=0x1028200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:27.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.071+0000 7f63e2d9d700 1 --2- 172.21.15.162:0/2900910146 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0xdd8c20 0x1028200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:27.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.071+0000 7f63e2d9d700 1 -- 172.21.15.162:0/2900910146 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0xe46820 msgr2=0x10c3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:27.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.071+0000 7f63e2d9d700 1 --2- 172.21.15.162:0/2900910146 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0xe46820 0x10c3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:27.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.071+0000 7f63e2d9d700 1 -- 172.21.15.162:0/2900910146 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0xf6b850 con 0xfddf20 2020-11-28T21:50:27.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.075+0000 7f63e2d9d700 1 --2- 172.21.15.162:0/2900910146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xfddf20 0x10c6190 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x10c6070 tx=0x7f63dc01f000).ready entity=mon.1 client_cookie=6be38fcca25eed86 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:27.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.075+0000 7f63c27fc700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2020-11-28T21:50:27.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.075+0000 7f63c27fc700 1 -- 172.21.15.162:0/2900910146 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63dc030070 con 0xfddf20 2020-11-28T21:50:27.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.075+0000 7f63f7807740 1 -- 172.21.15.162:0/2900910146 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xe3c4d0 con 0xfddf20 2020-11-28T21:50:27.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.075+0000 7f63c27fc700 1 -- 172.21.15.162:0/2900910146 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f63dc020680 con 0xfddf20 2020-11-28T21:50:27.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.075+0000 7f63c27fc700 1 -- 172.21.15.162:0/2900910146 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63dc020840 con 0xfddf20 2020-11-28T21:50:27.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.075+0000 7f63f7807740 10 client.5740 fetch_fsmap learned FSMap version 21 2020-11-28T21:50:27.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.075+0000 7f63f7807740 1 -- 172.21.15.162:0/2900910146 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x106fdb0 con 0xfddf20 2020-11-28T21:50:27.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.075+0000 7f63c27fc700 1 -- 172.21.15.162:0/2900910146 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xe3c4d0 con 0xfddf20 2020-11-28T21:50:27.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.075+0000 7f63c27fc700 1 -- 172.21.15.162:0/2900910146 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 21) v1 ==== 1434+0+0 (secure 0 0 0) 0x7f63dc01e040 con 0xfddf20 2020-11-28T21:50:27.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.075+0000 7f63f7807740 10 client.5740 fetch_fsmap finished waiting for FSMap version 21 2020-11-28T21:50:27.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.075+0000 7f63f7807740 10 client.5740 resolve_mds: resolved ID 'c' to GID 5551 2020-11-28T21:50:27.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.075+0000 7f63f7807740 20 client.5740 populate_metadata read hostname 'smithi162' 2020-11-28T21:50:27.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.075+0000 7f63f7807740 1 --2- 172.21.15.162:0/2900910146 >> [v2:172.21.15.162:6836/1902721947,v1:172.21.15.162:6837/1902721947] conn(0x10c3a70 0x10c6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:27.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.075+0000 7f63f7807740 4 client.5740 mds_command: new command op to 5551 tid=0[{"prefix": "cache drop", "timeout": 5}] 2020-11-28T21:50:27.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.075+0000 7f63f7807740 1 -- 172.21.15.162:0/2900910146 --> [v2:172.21.15.162:6836/1902721947,v1:172.21.15.162:6837/1902721947] -- command(tid 0: {"prefix": "cache drop", "timeout": 5}) v1 -- 0xe8a730 con 0x10c3a70 2020-11-28T21:50:27.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.075+0000 7f63e259c700 1 --2- 172.21.15.162:0/2900910146 >> [v2:172.21.15.162:6836/1902721947,v1:172.21.15.162:6837/1902721947] conn(0x10c3a70 0x10c6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:27.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.075+0000 7f63c27fc700 10 client.5740 ms_handle_connect on v2:172.21.15.162:6836/1902721947 2020-11-28T21:50:27.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:27.075+0000 7f63e259c700 1 --2- 172.21.15.162:0/2900910146 >> [v2:172.21.15.162:6836/1902721947,v1:172.21.15.162:6837/1902721947] conn(0x10c3a70 0x10c6680 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=f9c20dda7f32a2fc server_cookie=52d6918dbe83189e in_seq=0 out_seq=0 2020-11-28T21:50:32.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.315+0000 7f63c27fc700 1 -- 172.21.15.162:0/2900910146 <== mds.0 v2:172.21.15.162:6836/1902721947 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+404 (crc 0 0 0) 0xe8a730 con 0x10c3a70 2020-11-28T21:50:32.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.315+0000 7f63c27fc700 10 client.5740 handle_command_reply: tid=0 2020-11-28T21:50:32.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.315+0000 7f63f7807740 1 client.5740 shutdown 2020-11-28T21:50:32.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.315+0000 7f63f7807740 1 -- 172.21.15.162:0/2900910146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xfddf20 msgr2=0x10c6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:32.318 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.315+0000 7f63f7807740 1 --2- 172.21.15.162:0/2900910146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xfddf20 0x10c6190 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x10c6070 tx=0x7f63dc01f000).stop 2020-11-28T21:50:32.318 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.315+0000 7f63f7807740 1 -- 172.21.15.162:0/2900910146 shutdown_connections 2020-11-28T21:50:32.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.315+0000 7f63f7807740 1 --2- 172.21.15.162:0/2900910146 >> [v2:172.21.15.162:6836/1902721947,v1:172.21.15.162:6837/1902721947] conn(0x10c3a70 0x10c6680 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:32.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.315+0000 7f63f7807740 1 --2- 172.21.15.162:0/2900910146 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0xdd8c20 0x1028200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:32.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.315+0000 7f63f7807740 1 --2- 172.21.15.162:0/2900910146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xfddf20 0x10c6190 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:32.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.315+0000 7f63f7807740 1 --2- 172.21.15.162:0/2900910146 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0xe46820 0x10c3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:32.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.315+0000 7f63f7807740 1 -- 172.21.15.162:0/2900910146 >> 172.21.15.162:0/2900910146 conn(0x100d6a0 msgr2=0x10273f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:32.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.315+0000 7f63c27fc700 0 client.5740 ms_handle_reset on v2:172.21.15.162:6836/1902721947 2020-11-28T21:50:32.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.315+0000 7f63f7807740 1 -- 172.21.15.162:0/2900910146 shutdown_connections 2020-11-28T21:50:32.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.315+0000 7f63f7807740 1 -- 172.21.15.162:0/2900910146 wait complete. 2020-11-28T21:50:32.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.315+0000 7f63f7807740 20 client.5740 trim_cache size 0 max 16384 2020-11-28T21:50:32.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.319+0000 7f63e8e6c700 1 -- 172.21.15.162:0/3429216124 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f63c80554e0 msgr2=0x7f63c80579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:32.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.319+0000 7f63e8e6c700 1 --2- 172.21.15.162:0/3429216124 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f63c80554e0 0x7f63c80579a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f63d400d000 tx=0x7f63d4009040).stop 2020-11-28T21:50:32.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.319+0000 7f63e8e6c700 1 -- 172.21.15.162:0/3429216124 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63e40b9ce0 msgr2=0x7f63e4146690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:32.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.319+0000 7f63e8e6c700 1 --2- 172.21.15.162:0/3429216124 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63e40b9ce0 0x7f63e4146690 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f63cc003070 tx=0x7f63cc008040).stop 2020-11-28T21:50:32.325 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2020-11-28T21:50:32.325 INFO:teuthology.orchestra.run.smithi162.stdout: "client_recall": { 2020-11-28T21:50:32.325 INFO:teuthology.orchestra.run.smithi162.stdout: "return_code": -110, 2020-11-28T21:50:32.325 INFO:teuthology.orchestra.run.smithi162.stdout: "message": "(110) Connection timed out", 2020-11-28T21:50:32.326 INFO:teuthology.orchestra.run.smithi162.stdout: "recalled": 120 2020-11-28T21:50:32.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:50:32.326 INFO:teuthology.orchestra.run.smithi162.stdout: "flush_journal": { 2020-11-28T21:50:32.326 INFO:teuthology.orchestra.run.smithi162.stdout: "return_code": 0, 2020-11-28T21:50:32.327 INFO:teuthology.orchestra.run.smithi162.stdout: "message": "" 2020-11-28T21:50:32.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:50:32.327 INFO:teuthology.orchestra.run.smithi162.stdout: "trim_cache": { 2020-11-28T21:50:32.327 INFO:teuthology.orchestra.run.smithi162.stdout: "trimmed": 0 2020-11-28T21:50:32.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:50:32.328 INFO:teuthology.orchestra.run.smithi162.stdout: "cache": { 2020-11-28T21:50:32.328 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": { 2020-11-28T21:50:32.328 INFO:teuthology.orchestra.run.smithi162.stdout: "items": 19983, 2020-11-28T21:50:32.329 INFO:teuthology.orchestra.run.smithi162.stdout: "bytes": 3099232 2020-11-28T21:50:32.329 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:50:32.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:50:32.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2412850640000004 2020-11-28T21:50:32.329 INFO:teuthology.orchestra.run.smithi162.stdout:} 2020-11-28T21:50:32.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.319+0000 7f63e8e6c700 1 -- 172.21.15.162:0/3429216124 shutdown_connections 2020-11-28T21:50:32.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.319+0000 7f63e8e6c700 1 --2- 172.21.15.162:0/3429216124 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f63c80554e0 0x7f63c80579a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:32.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.319+0000 7f63e8e6c700 1 --2- 172.21.15.162:0/3429216124 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f63e4067610 0x7f63e41461a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:32.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.319+0000 7f63e8e6c700 1 --2- 172.21.15.162:0/3429216124 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f63e41433c0 0x7f63e4155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:32.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.319+0000 7f63e8e6c700 1 --2- 172.21.15.162:0/3429216124 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63e40b9ce0 0x7f63e4146690 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:32.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.319+0000 7f63e8e6c700 1 -- 172.21.15.162:0/3429216124 >> 172.21.15.162:0/3429216124 conn(0x7f63e405c430 msgr2=0x7f63e413f290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:32.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.319+0000 7f63e8e6c700 1 -- 172.21.15.162:0/3429216124 shutdown_connections 2020-11-28T21:50:32.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.319+0000 7f63e8e6c700 1 -- 172.21.15.162:0/3429216124 wait complete. 2020-11-28T21:50:32.333 INFO:tasks.cephfs.fuse_mount:Cleaning up killed ceph-fuse connection 2020-11-28T21:50:32.333 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:50:32.334 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:50:32.335 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:50:32.342 INFO:teuthology.orchestra.run.smithi192.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Transport endpoint is not connected 2020-11-28T21:50:32.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:50:32.345 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2020-11-28T21:50:32.346 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2020-11-28T21:50:32.346 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-28T21:50:32.346 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:50:32.427 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:50:32.428 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:50:32.428 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:50:32.475 INFO:teuthology.orchestra.run.smithi192.stdout:ext2/ext3 2020-11-28T21:50:32.477 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-11-28T21:50:32.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-28T21:50:32.479 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:50:32.529 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:50:32.645 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.647+0000 7f148d49d700 1 Processor -- start 2020-11-28T21:50:32.646 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.647+0000 7f148d49d700 1 -- start start 2020-11-28T21:50:32.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.647+0000 7f148d49d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1488098b30 0x7f14880994f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:32.651 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.647+0000 7f148d49d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f14880b0000 0x7f1488060000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:32.651 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.647+0000 7f148d49d700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14880604f0 0x7f1488060980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:32.652 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.647+0000 7f148d49d700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14880b1030 con 0x7f14880b0000 2020-11-28T21:50:32.652 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.647+0000 7f148d49d700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14880b0480 con 0x7f1488098b30 2020-11-28T21:50:32.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.647+0000 7f148d49d700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1488098f10 con 0x7f14880604f0 2020-11-28T21:50:32.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.647+0000 7f1487fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1488098b30 0x7f14880994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:32.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.647+0000 7f148cc9c700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14880604f0 0x7f1488060980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:32.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.647+0000 7f1487fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1488098b30 0x7f14880994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40262/0 (socket says 172.21.15.162:40262) 2020-11-28T21:50:32.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.647+0000 7f1487fff700 1 -- 172.21.15.162:0/1303227475 learned_addr learned my addr 172.21.15.162:0/1303227475 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:32.655 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.647+0000 7f14877fe700 1 --2- 172.21.15.162:0/1303227475 >> v2:172.21.15.118:3300/0 conn(0x7f14880b0000 0x7f1488060000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:32.655 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.647+0000 7f1487fff700 1 -- 172.21.15.162:0/1303227475 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14880604f0 msgr2=0x7f1488060980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:32.656 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.647+0000 7f1487fff700 1 --2- 172.21.15.162:0/1303227475 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14880604f0 0x7f1488060980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:32.656 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.647+0000 7f1487fff700 1 -- 172.21.15.162:0/1303227475 >> v2:172.21.15.118:3300/0 conn(0x7f14880b0000 msgr2=0x7f1488060000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:32.657 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.647+0000 7f1487fff700 1 --2- 172.21.15.162:0/1303227475 >> v2:172.21.15.118:3300/0 conn(0x7f14880b0000 0x7f1488060000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:32.657 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.647+0000 7f1487fff700 1 -- 172.21.15.162:0/1303227475 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14880b0680 con 0x7f1488098b30 2020-11-28T21:50:32.657 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.647+0000 7f14877fe700 1 --2- 172.21.15.162:0/1303227475 >> v2:172.21.15.118:3300/0 conn(0x7f14880b0000 0x7f1488060000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:50:32.658 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.651+0000 7f1487fff700 1 --2- 172.21.15.162:0/1303227475 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1488098b30 0x7f14880994f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7f147c002f30 tx=0x7f147c006e80).ready entity=mon.1 client_cookie=e23d71e3d70e1076 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:32.658 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.651+0000 7f1486ffd700 1 -- 172.21.15.162:0/1303227475 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f147c009640 con 0x7f1488098b30 2020-11-28T21:50:32.659 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.651+0000 7f1486ffd700 1 -- 172.21.15.162:0/1303227475 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f147c0097a0 con 0x7f1488098b30 2020-11-28T21:50:32.659 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.651+0000 7f1486ffd700 1 -- 172.21.15.162:0/1303227475 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f147c009b00 con 0x7f1488098b30 2020-11-28T21:50:32.660 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.651+0000 7f148d49d700 1 -- 172.21.15.162:0/1303227475 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1488098b30 msgr2=0x7f14880994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:32.660 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.651+0000 7f148d49d700 1 --2- 172.21.15.162:0/1303227475 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1488098b30 0x7f14880994f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7f147c002f30 tx=0x7f147c006e80).stop 2020-11-28T21:50:32.660 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.651+0000 7f148d49d700 1 -- 172.21.15.162:0/1303227475 shutdown_connections 2020-11-28T21:50:32.661 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.651+0000 7f148d49d700 1 --2- 172.21.15.162:0/1303227475 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14880604f0 0x7f1488060980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:32.661 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.651+0000 7f148d49d700 1 --2- 172.21.15.162:0/1303227475 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1488098b30 0x7f14880994f0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:32.662 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.651+0000 7f148d49d700 1 --2- 172.21.15.162:0/1303227475 >> v2:172.21.15.118:3300/0 conn(0x7f14880b0000 0x7f1488060000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:32.662 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.651+0000 7f148d49d700 1 -- 172.21.15.162:0/1303227475 >> 172.21.15.162:0/1303227475 conn(0x7f148805b970 msgr2=0x7f148805bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:32.663 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.651+0000 7f148d49d700 1 -- 172.21.15.162:0/1303227475 shutdown_connections 2020-11-28T21:50:32.663 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.651+0000 7f148d49d700 1 -- 172.21.15.162:0/1303227475 wait complete. 2020-11-28T21:50:32.663 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.651+0000 7f148d49d700 1 Processor -- start 2020-11-28T21:50:32.664 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.651+0000 7f148d49d700 1 -- start start 2020-11-28T21:50:32.668 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.651+0000 7f148d49d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14880604f0 0x7f1488066cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:32.669 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.651+0000 7f148d49d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1488098b30 0x7f14880b4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:32.669 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.651+0000 7f148d49d700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14880b0000 0x7f1488064180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:32.669 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.651+0000 7f148d49d700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14880aef00 con 0x7f1488098b30 2020-11-28T21:50:32.669 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.651+0000 7f148d49d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14880aec60 con 0x7f14880604f0 2020-11-28T21:50:32.670 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.651+0000 7f148d49d700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1488159a50 con 0x7f14880b0000 2020-11-28T21:50:32.670 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.651+0000 7f1487fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14880604f0 0x7f1488066cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:32.670 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.655+0000 7f1487fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14880604f0 0x7f1488066cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40266/0 (socket says 172.21.15.162:40266) 2020-11-28T21:50:32.670 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.655+0000 7f1487fff700 1 -- 172.21.15.162:0/2974671354 learned_addr learned my addr 172.21.15.162:0/2974671354 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:32.671 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.655+0000 7f14877fe700 1 --2- 172.21.15.162:0/2974671354 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1488098b30 0x7f14880b4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:32.671 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.655+0000 7f148cc9c700 1 --2- 172.21.15.162:0/2974671354 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14880b0000 0x7f1488064180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:32.671 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.655+0000 7f1487fff700 1 -- 172.21.15.162:0/2974671354 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14880b0000 msgr2=0x7f1488064180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:32.671 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.655+0000 7f1487fff700 1 --2- 172.21.15.162:0/2974671354 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14880b0000 0x7f1488064180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:32.672 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.655+0000 7f1487fff700 1 -- 172.21.15.162:0/2974671354 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1488098b30 msgr2=0x7f14880b4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:32.672 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.655+0000 7f1487fff700 1 --2- 172.21.15.162:0/2974671354 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1488098b30 0x7f14880b4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:32.672 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.655+0000 7f1487fff700 1 -- 172.21.15.162:0/2974671354 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1488159cf0 con 0x7f14880604f0 2020-11-28T21:50:32.672 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.655+0000 7f1487fff700 1 --2- 172.21.15.162:0/2974671354 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14880604f0 0x7f1488066cb0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7f147c002f30 tx=0x7f147c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:32.672 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.655+0000 7f14857fa700 1 -- 172.21.15.162:0/2974671354 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f147c009e80 con 0x7f14880604f0 2020-11-28T21:50:32.673 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.655+0000 7f14857fa700 1 -- 172.21.15.162:0/2974671354 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f147c011c60 con 0x7f14880604f0 2020-11-28T21:50:32.673 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.655+0000 7f14857fa700 1 -- 172.21.15.162:0/2974671354 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f147c01d450 con 0x7f14880604f0 2020-11-28T21:50:32.673 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.655+0000 7f148d49d700 1 -- 172.21.15.162:0/2974671354 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14880b4870 con 0x7f14880604f0 2020-11-28T21:50:32.673 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.655+0000 7f148d49d700 1 -- 172.21.15.162:0/2974671354 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1488064670 con 0x7f14880604f0 2020-11-28T21:50:32.674 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.655+0000 7f148d49d700 1 -- 172.21.15.162:0/2974671354 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1488159e80 con 0x7f14880604f0 2020-11-28T21:50:32.674 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.655+0000 7f14857fa700 1 -- 172.21.15.162:0/2974671354 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f147c00f070 con 0x7f14880604f0 2020-11-28T21:50:32.674 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.655+0000 7f14857fa700 1 --2- 172.21.15.162:0/2974671354 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1468055440 0x7f1468057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:32.674 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.655+0000 7f14857fa700 1 -- 172.21.15.162:0/2974671354 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f147c06f7a0 con 0x7f14880604f0 2020-11-28T21:50:32.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.655+0000 7f14877fe700 1 --2- 172.21.15.162:0/2974671354 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1468055440 0x7f1468057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:32.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.659+0000 7f14877fe700 1 --2- 172.21.15.162:0/2974671354 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1468055440 0x7f1468057900 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f147800b000 tx=0x7f1478008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:32.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.663+0000 7f14857fa700 1 -- 172.21.15.162:0/2974671354 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f147c045ae0 con 0x7f14880604f0 2020-11-28T21:50:32.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.795+0000 7f148d49d700 1 -- 172.21.15.162:0/2974671354 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1488057050 con 0x7f14880604f0 2020-11-28T21:50:32.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.795+0000 7f14857fa700 1 -- 172.21.15.162:0/2974671354 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+16136 (secure 0 0 0) 0x7f147c0490c0 con 0x7f14880604f0 2020-11-28T21:50:32.795 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:50:32.796 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":56,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:50:07.723574+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:50:03.211406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2020-11-28T21:50:04.511075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:50:32.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.799+0000 7f148d49d700 1 -- 172.21.15.162:0/2974671354 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1468055440 msgr2=0x7f1468057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:32.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.799+0000 7f148d49d700 1 --2- 172.21.15.162:0/2974671354 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1468055440 0x7f1468057900 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f147800b000 tx=0x7f1478008040).stop 2020-11-28T21:50:32.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.799+0000 7f148d49d700 1 -- 172.21.15.162:0/2974671354 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14880604f0 msgr2=0x7f1488066cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:32.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.799+0000 7f148d49d700 1 --2- 172.21.15.162:0/2974671354 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14880604f0 0x7f1488066cb0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7f147c002f30 tx=0x7f147c015000).stop 2020-11-28T21:50:32.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.799+0000 7f148d49d700 1 -- 172.21.15.162:0/2974671354 shutdown_connections 2020-11-28T21:50:32.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.799+0000 7f148d49d700 1 --2- 172.21.15.162:0/2974671354 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1468055440 0x7f1468057900 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:32.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.799+0000 7f148d49d700 1 --2- 172.21.15.162:0/2974671354 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14880b0000 0x7f1488064180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:32.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.799+0000 7f148d49d700 1 --2- 172.21.15.162:0/2974671354 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14880604f0 0x7f1488066cb0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:32.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.799+0000 7f148d49d700 1 --2- 172.21.15.162:0/2974671354 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1488098b30 0x7f14880b4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:32.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.799+0000 7f148d49d700 1 -- 172.21.15.162:0/2974671354 >> 172.21.15.162:0/2974671354 conn(0x7f148805b970 msgr2=0x7f14880b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:32.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.799+0000 7f148d49d700 1 -- 172.21.15.162:0/2974671354 shutdown_connections 2020-11-28T21:50:32.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.799+0000 7f148d49d700 1 -- 172.21.15.162:0/2974671354 wait complete. 2020-11-28T21:50:32.823 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-28T21:50:03.211406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2020-11-28T21:50:04.511075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-28T21:50:32.824 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:50:32.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.951+0000 7f87139ee700 1 Processor -- start 2020-11-28T21:50:32.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.951+0000 7f87139ee700 1 -- start start 2020-11-28T21:50:32.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.951+0000 7f87139ee700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f870c0b9ce0 0x7f870c1433c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:32.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.951+0000 7f87139ee700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f870c067610 0x7f870c13fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:32.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.951+0000 7f87139ee700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f870c143950 con 0x7f870c0605c0 2020-11-28T21:50:32.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.951+0000 7f87139ee700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f870c0b4340 con 0x7f870c067610 2020-11-28T21:50:32.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.951+0000 7f87139ee700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f870c0ba0c0 con 0x7f870c0b9ce0 2020-11-28T21:50:32.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.951+0000 7f871178a700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f870c0b9ce0 0x7f870c1433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:32.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.951+0000 7f8711f8b700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f870c067610 0x7f870c13fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:32.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.951+0000 7f871178a700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f870c0b9ce0 0x7f870c1433c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:37498/0 (socket says 172.21.15.162:37498) 2020-11-28T21:50:32.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.951+0000 7f871178a700 1 -- 172.21.15.162:0/2557052501 learned_addr learned my addr 172.21.15.162:0/2557052501 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:32.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.951+0000 7f8710f89700 1 --1- 172.21.15.162:0/2557052501 >> v1:172.21.15.118:6789/0 conn(0x7f870c0605c0 0x7f870c0b3c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.162:37270/0 (socket says 172.21.15.162:37270) 2020-11-28T21:50:32.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.951+0000 7f8703fff700 1 -- 172.21.15.162:0/2557052501 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 30115915 0 0) 0x7f870c143950 con 0x7f870c0605c0 2020-11-28T21:50:32.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.951+0000 7f8703fff700 1 -- 172.21.15.162:0/2557052501 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86f0003230 con 0x7f870c0605c0 2020-11-28T21:50:32.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.951+0000 7f871178a700 1 -- 172.21.15.162:0/2557052501 >> v2:172.21.15.118:3300/0 conn(0x7f870c067610 msgr2=0x7f870c13fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:32.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.951+0000 7f871178a700 1 --2- 172.21.15.162:0/2557052501 >> v2:172.21.15.118:3300/0 conn(0x7f870c067610 0x7f870c13fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:32.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.951+0000 7f871178a700 1 -- 172.21.15.162:0/2557052501 >> v1:172.21.15.118:6789/0 conn(0x7f870c0605c0 legacy=0x7f870c0b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:32.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.951+0000 7f871178a700 1 -- 172.21.15.162:0/2557052501 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f870c067aa0 con 0x7f870c0b9ce0 2020-11-28T21:50:32.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.951+0000 7f871178a700 1 --2- 172.21.15.162:0/2557052501 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f870c0b9ce0 0x7f870c1433c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f8708002f30 tx=0x7f8708006e20).ready entity=mon.2 client_cookie=9cc78f2c6ff594b9 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:32.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.951+0000 7f8703fff700 1 -- 172.21.15.162:0/2557052501 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8708009760 con 0x7f870c0b9ce0 2020-11-28T21:50:32.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.951+0000 7f8703fff700 1 -- 172.21.15.162:0/2557052501 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87080098c0 con 0x7f870c0b9ce0 2020-11-28T21:50:32.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.951+0000 7f8703fff700 1 -- 172.21.15.162:0/2557052501 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8708009c20 con 0x7f870c0b9ce0 2020-11-28T21:50:32.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.951+0000 7f87139ee700 1 -- 172.21.15.162:0/2557052501 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f870c0b9ce0 msgr2=0x7f870c1433c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:32.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.951+0000 7f87139ee700 1 --2- 172.21.15.162:0/2557052501 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f870c0b9ce0 0x7f870c1433c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f8708002f30 tx=0x7f8708006e20).stop 2020-11-28T21:50:32.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.955+0000 7f87139ee700 1 -- 172.21.15.162:0/2557052501 shutdown_connections 2020-11-28T21:50:32.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.955+0000 7f87139ee700 1 --2- 172.21.15.162:0/2557052501 >> v2:172.21.15.118:3300/0 conn(0x7f870c067610 0x7f870c13fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:32.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.955+0000 7f87139ee700 1 --2- 172.21.15.162:0/2557052501 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f870c0b9ce0 0x7f870c1433c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:32.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.955+0000 7f87139ee700 1 -- 172.21.15.162:0/2557052501 >> 172.21.15.162:0/2557052501 conn(0x7f870c05c430 msgr2=0x7f870c0b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:32.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.955+0000 7f87139ee700 1 -- 172.21.15.162:0/2557052501 shutdown_connections 2020-11-28T21:50:32.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.955+0000 7f87139ee700 1 -- 172.21.15.162:0/2557052501 wait complete. 2020-11-28T21:50:32.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.955+0000 7f87139ee700 1 Processor -- start 2020-11-28T21:50:32.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.955+0000 7f87139ee700 1 -- start start 2020-11-28T21:50:32.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.955+0000 7f87139ee700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f870c0605c0 0x7f870c0b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:32.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.955+0000 7f87139ee700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f870c067610 0x7f870c0b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:32.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.955+0000 7f87139ee700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f870c0b9ce0 0x7f870c154c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:32.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.959+0000 7f87139ee700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870c0b36b0 con 0x7f870c0605c0 2020-11-28T21:50:32.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.959+0000 7f87139ee700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870c152530 con 0x7f870c067610 2020-11-28T21:50:32.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.959+0000 7f87139ee700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f870c09dca0 con 0x7f870c0b9ce0 2020-11-28T21:50:32.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.959+0000 7f8710f89700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f870c067610 0x7f870c0b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:32.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.959+0000 7f8711f8b700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f870c0b9ce0 0x7f870c154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:32.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.959+0000 7f8710f89700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f870c067610 0x7f870c0b31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40282/0 (socket says 172.21.15.162:40282) 2020-11-28T21:50:32.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.959+0000 7f8710f89700 1 -- 172.21.15.162:0/3336417991 learned_addr learned my addr 172.21.15.162:0/3336417991 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:32.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.959+0000 7f871178a700 1 --2- 172.21.15.162:0/3336417991 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f870c0605c0 0x7f870c0b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:32.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.959+0000 7f8710f89700 1 -- 172.21.15.162:0/3336417991 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f870c0b9ce0 msgr2=0x7f870c154c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:32.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.959+0000 7f8710f89700 1 --2- 172.21.15.162:0/3336417991 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f870c0b9ce0 0x7f870c154c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:32.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.959+0000 7f8710f89700 1 -- 172.21.15.162:0/3336417991 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f870c0605c0 msgr2=0x7f870c0b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:32.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.959+0000 7f8710f89700 1 --2- 172.21.15.162:0/3336417991 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f870c0605c0 0x7f870c0b2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:32.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.959+0000 7f8710f89700 1 -- 172.21.15.162:0/3336417991 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f870c09df40 con 0x7f870c067610 2020-11-28T21:50:32.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.959+0000 7f8711f8b700 1 --2- 172.21.15.162:0/3336417991 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f870c0b9ce0 0x7f870c154c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:50:32.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.959+0000 7f8710f89700 1 --2- 172.21.15.162:0/3336417991 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f870c067610 0x7f870c0b31c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7f86fc0010d0 tx=0x7f86fc007aa0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:32.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.959+0000 7f87027fc700 1 -- 172.21.15.162:0/3336417991 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f86fc0072e0 con 0x7f870c067610 2020-11-28T21:50:32.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.959+0000 7f87139ee700 1 -- 172.21.15.162:0/3336417991 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f870c09e1e0 con 0x7f870c067610 2020-11-28T21:50:32.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.959+0000 7f87027fc700 1 -- 172.21.15.162:0/3336417991 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86fc005520 con 0x7f870c067610 2020-11-28T21:50:32.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.959+0000 7f87027fc700 1 -- 172.21.15.162:0/3336417991 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f86fc005690 con 0x7f870c067610 2020-11-28T21:50:32.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.959+0000 7f87139ee700 1 -- 172.21.15.162:0/3336417991 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f870c1427f0 con 0x7f870c067610 2020-11-28T21:50:32.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.959+0000 7f87027fc700 1 -- 172.21.15.162:0/3336417991 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f86fc010070 con 0x7f870c067610 2020-11-28T21:50:32.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.959+0000 7f87139ee700 1 -- 172.21.15.162:0/3336417991 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f870c057050 con 0x7f870c067610 2020-11-28T21:50:32.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.959+0000 7f87027fc700 1 --2- 172.21.15.162:0/3336417991 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f86f40554e0 0x7f86f40579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:32.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.959+0000 7f87027fc700 1 -- 172.21.15.162:0/3336417991 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f86fc06d350 con 0x7f870c067610 2020-11-28T21:50:32.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.959+0000 7f871178a700 1 --2- 172.21.15.162:0/3336417991 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f86f40554e0 0x7f86f40579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:32.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.963+0000 7f871178a700 1 --2- 172.21.15.162:0/3336417991 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f86f40554e0 0x7f86f40579a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f870801a000 tx=0x7f8708015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:32.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:32.971+0000 7f87027fc700 1 -- 172.21.15.162:0/3336417991 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f86fc0435b0 con 0x7f870c067610 2020-11-28T21:50:33.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.111+0000 7f87139ee700 1 -- 172.21.15.162:0/3336417991 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f870c1427f0 con 0x7f870c067610 2020-11-28T21:50:33.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.111+0000 7f87027fc700 1 -- 172.21.15.162:0/3336417991 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v56) v1 ==== 110+0+10 (secure 0 0 0) 0x7f870c1427f0 con 0x7f870c067610 2020-11-28T21:50:33.111 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:50:33.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.115+0000 7f87139ee700 1 -- 172.21.15.162:0/3336417991 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f86f40554e0 msgr2=0x7f86f40579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:33.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.115+0000 7f87139ee700 1 --2- 172.21.15.162:0/3336417991 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f86f40554e0 0x7f86f40579a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f870801a000 tx=0x7f8708015000).stop 2020-11-28T21:50:33.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.115+0000 7f87139ee700 1 -- 172.21.15.162:0/3336417991 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f870c067610 msgr2=0x7f870c0b31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:33.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.115+0000 7f87139ee700 1 --2- 172.21.15.162:0/3336417991 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f870c067610 0x7f870c0b31c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7f86fc0010d0 tx=0x7f86fc007aa0).stop 2020-11-28T21:50:33.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.115+0000 7f87139ee700 1 -- 172.21.15.162:0/3336417991 shutdown_connections 2020-11-28T21:50:33.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.115+0000 7f87139ee700 1 --2- 172.21.15.162:0/3336417991 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f86f40554e0 0x7f86f40579a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:33.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.115+0000 7f87139ee700 1 --2- 172.21.15.162:0/3336417991 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f870c0b9ce0 0x7f870c154c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:33.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.115+0000 7f87139ee700 1 --2- 172.21.15.162:0/3336417991 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f870c0605c0 0x7f870c0b2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:33.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.115+0000 7f87139ee700 1 --2- 172.21.15.162:0/3336417991 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f870c067610 0x7f870c0b31c0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:33.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.115+0000 7f87139ee700 1 -- 172.21.15.162:0/3336417991 >> 172.21.15.162:0/3336417991 conn(0x7f870c05c430 msgr2=0x7f870c0b7dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:33.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.115+0000 7f87139ee700 1 -- 172.21.15.162:0/3336417991 shutdown_connections 2020-11-28T21:50:33.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.115+0000 7f87139ee700 1 -- 172.21.15.162:0/3336417991 wait complete. 2020-11-28T21:50:33.124 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:50:33.242 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.243+0000 7fe715dad700 1 Processor -- start 2020-11-28T21:50:33.242 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.243+0000 7fe715dad700 1 -- start start 2020-11-28T21:50:33.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.243+0000 7fe715dad700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe7100b3770 0x7fe7100b2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:33.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.247+0000 7fe715dad700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe710065750 0x7fe7100b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:33.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.247+0000 7fe715dad700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe710062e40 con 0x7fe7100aa8b0 2020-11-28T21:50:33.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.247+0000 7fe715dad700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7100b2e30 con 0x7fe710065750 2020-11-28T21:50:33.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.247+0000 7fe715dad700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe71009cb50 con 0x7fe7100b3770 2020-11-28T21:50:33.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.247+0000 7fe70effd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe710065750 0x7fe7100b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:33.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.247+0000 7fe70ffff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fe7100aa8b0 0x7fe710065dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:47134/0 (socket says 172.21.15.162:47134) 2020-11-28T21:50:33.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.247+0000 7fe70ffff700 1 -- 172.21.15.162:0/2661484959 learned_addr learned my addr 172.21.15.162:0/2661484959 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:50:33.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.247+0000 7fe70f7fe700 1 --2- 172.21.15.162:0/2661484959 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe7100b3770 0x7fe7100b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:33.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.247+0000 7fe70effd700 1 -- 172.21.15.162:0/2661484959 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe7100b3770 msgr2=0x7fe7100b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:33.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.247+0000 7fe70effd700 1 --2- 172.21.15.162:0/2661484959 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe7100b3770 0x7fe7100b2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:33.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.247+0000 7fe70effd700 1 -- 172.21.15.162:0/2661484959 >> v1:172.21.15.162:6789/0 conn(0x7fe7100aa8b0 legacy=0x7fe710065dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:33.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.247+0000 7fe70e7fc700 1 -- 172.21.15.162:0/2661484959 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3846434666 0 0) 0x7fe710062e40 con 0x7fe7100aa8b0 2020-11-28T21:50:33.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.247+0000 7fe70effd700 1 -- 172.21.15.162:0/2661484959 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe71009cdf0 con 0x7fe710065750 2020-11-28T21:50:33.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.247+0000 7fe70effd700 1 --2- 172.21.15.162:0/2661484959 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe710065750 0x7fe7100b2940 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7fe700002f30 tx=0x7fe700006dc0).ready entity=mon.1 client_cookie=52dffe0613bddb57 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:33.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.247+0000 7fe70e7fc700 1 -- 172.21.15.162:0/2661484959 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe7000097f0 con 0x7fe710065750 2020-11-28T21:50:33.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.247+0000 7fe70e7fc700 1 -- 172.21.15.162:0/2661484959 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe700009950 con 0x7fe710065750 2020-11-28T21:50:33.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.247+0000 7fe70e7fc700 1 -- 172.21.15.162:0/2661484959 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe700009cb0 con 0x7fe710065750 2020-11-28T21:50:33.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.247+0000 7fe715dad700 1 -- 172.21.15.162:0/2661484959 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe710065750 msgr2=0x7fe7100b2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:33.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.247+0000 7fe715dad700 1 --2- 172.21.15.162:0/2661484959 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe710065750 0x7fe7100b2940 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7fe700002f30 tx=0x7fe700006dc0).stop 2020-11-28T21:50:33.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.247+0000 7fe715dad700 1 -- 172.21.15.162:0/2661484959 shutdown_connections 2020-11-28T21:50:33.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.247+0000 7fe715dad700 1 --2- 172.21.15.162:0/2661484959 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe7100b3770 0x7fe7100b2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:33.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.247+0000 7fe715dad700 1 --2- 172.21.15.162:0/2661484959 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe710065750 0x7fe7100b2940 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:33.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.247+0000 7fe715dad700 1 -- 172.21.15.162:0/2661484959 >> 172.21.15.162:0/2661484959 conn(0x7fe71005b970 msgr2=0x7fe71005bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:33.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.247+0000 7fe715dad700 1 -- 172.21.15.162:0/2661484959 shutdown_connections 2020-11-28T21:50:33.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.247+0000 7fe715dad700 1 -- 172.21.15.162:0/2661484959 wait complete. 2020-11-28T21:50:33.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.251+0000 7fe715dad700 1 Processor -- start 2020-11-28T21:50:33.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.251+0000 7fe715dad700 1 -- start start 2020-11-28T21:50:33.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.251+0000 7fe715dad700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe710065750 0x7fe710143500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:33.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.251+0000 7fe715dad700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7100aa8b0 0x7fe7100ad240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:33.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.251+0000 7fe715dad700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe7100b3770 0x7fe710061b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:33.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.251+0000 7fe715dad700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe710060c70 con 0x7fe710065750 2020-11-28T21:50:33.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.251+0000 7fe715dad700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe710060730 con 0x7fe7100aa8b0 2020-11-28T21:50:33.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.251+0000 7fe715dad700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7100609d0 con 0x7fe7100b3770 2020-11-28T21:50:33.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.251+0000 7fe70effd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7100aa8b0 0x7fe7100ad240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:33.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.251+0000 7fe70effd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7100aa8b0 0x7fe7100ad240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40296/0 (socket says 172.21.15.162:40296) 2020-11-28T21:50:33.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.251+0000 7fe70effd700 1 -- 172.21.15.162:0/1970503566 learned_addr learned my addr 172.21.15.162:0/1970503566 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:33.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.251+0000 7fe70ffff700 1 --2- 172.21.15.162:0/1970503566 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe7100b3770 0x7fe710061b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:33.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.251+0000 7fe70f7fe700 1 --2- 172.21.15.162:0/1970503566 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe710065750 0x7fe710143500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:33.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.251+0000 7fe70effd700 1 -- 172.21.15.162:0/1970503566 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe7100b3770 msgr2=0x7fe710061b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:33.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.251+0000 7fe70effd700 1 --2- 172.21.15.162:0/1970503566 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe7100b3770 0x7fe710061b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:33.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.251+0000 7fe70effd700 1 -- 172.21.15.162:0/1970503566 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe710065750 msgr2=0x7fe710143500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:33.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.251+0000 7fe70effd700 1 --2- 172.21.15.162:0/1970503566 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe710065750 0x7fe710143500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:33.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.251+0000 7fe70effd700 1 -- 172.21.15.162:0/1970503566 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe710061680 con 0x7fe7100aa8b0 2020-11-28T21:50:33.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.251+0000 7fe70effd700 1 --2- 172.21.15.162:0/1970503566 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7100aa8b0 0x7fe7100ad240 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7fe710151820 tx=0x7fe700015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:33.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.251+0000 7fe70cff9700 1 -- 172.21.15.162:0/1970503566 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe700009c50 con 0x7fe7100aa8b0 2020-11-28T21:50:33.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.251+0000 7fe70cff9700 1 -- 172.21.15.162:0/1970503566 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe700011da0 con 0x7fe7100aa8b0 2020-11-28T21:50:33.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.251+0000 7fe715dad700 1 -- 172.21.15.162:0/1970503566 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7100acfe0 con 0x7fe7100aa8b0 2020-11-28T21:50:33.274 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.251+0000 7fe70cff9700 1 -- 172.21.15.162:0/1970503566 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe70001d640 con 0x7fe7100aa8b0 2020-11-28T21:50:33.275 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.251+0000 7fe715dad700 1 -- 172.21.15.162:0/1970503566 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7100aaeb0 con 0x7fe7100aa8b0 2020-11-28T21:50:33.275 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.255+0000 7fe715dad700 1 -- 172.21.15.162:0/1970503566 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe710060390 con 0x7fe7100aa8b0 2020-11-28T21:50:33.275 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.255+0000 7fe70cff9700 1 -- 172.21.15.162:0/1970503566 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe700024030 con 0x7fe7100aa8b0 2020-11-28T21:50:33.276 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.255+0000 7fe70cff9700 1 --2- 172.21.15.162:0/1970503566 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe6f40554e0 0x7fe6f40579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:33.276 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.255+0000 7fe70cff9700 1 -- 172.21.15.162:0/1970503566 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe70006f160 con 0x7fe7100aa8b0 2020-11-28T21:50:33.276 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.255+0000 7fe70f7fe700 1 --2- 172.21.15.162:0/1970503566 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe6f40554e0 0x7fe6f40579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:33.276 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.255+0000 7fe70f7fe700 1 --2- 172.21.15.162:0/1970503566 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe6f40554e0 0x7fe6f40579a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7fe6f800c000 tx=0x7fe6f8008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:33.277 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.259+0000 7fe70cff9700 1 -- 172.21.15.162:0/1970503566 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe710060390 con 0x7fe7100aa8b0 2020-11-28T21:50:33.387 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.387+0000 7fe715dad700 1 -- 172.21.15.162:0/1970503566 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe710060390 con 0x7fe7100aa8b0 2020-11-28T21:50:33.387 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.391+0000 7fe70cff9700 1 -- 172.21.15.162:0/1970503566 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v56) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe710060390 con 0x7fe7100aa8b0 2020-11-28T21:50:33.388 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:50:33.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.391+0000 7fe715dad700 1 -- 172.21.15.162:0/1970503566 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe6f40554e0 msgr2=0x7fe6f40579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:33.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.391+0000 7fe715dad700 1 --2- 172.21.15.162:0/1970503566 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe6f40554e0 0x7fe6f40579a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7fe6f800c000 tx=0x7fe6f8008040).stop 2020-11-28T21:50:33.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.391+0000 7fe715dad700 1 -- 172.21.15.162:0/1970503566 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7100aa8b0 msgr2=0x7fe7100ad240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:33.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.391+0000 7fe715dad700 1 --2- 172.21.15.162:0/1970503566 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7100aa8b0 0x7fe7100ad240 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7fe710151820 tx=0x7fe700015000).stop 2020-11-28T21:50:33.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.391+0000 7fe715dad700 1 -- 172.21.15.162:0/1970503566 shutdown_connections 2020-11-28T21:50:33.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.391+0000 7fe715dad700 1 --2- 172.21.15.162:0/1970503566 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe6f40554e0 0x7fe6f40579a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:33.394 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.391+0000 7fe715dad700 1 --2- 172.21.15.162:0/1970503566 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe7100b3770 0x7fe710061b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:33.394 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.391+0000 7fe715dad700 1 --2- 172.21.15.162:0/1970503566 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe710065750 0x7fe710143500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:33.394 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.391+0000 7fe715dad700 1 --2- 172.21.15.162:0/1970503566 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7100aa8b0 0x7fe7100ad240 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:33.394 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.391+0000 7fe715dad700 1 -- 172.21.15.162:0/1970503566 >> 172.21.15.162:0/1970503566 conn(0x7fe71005b970 msgr2=0x7fe710144860 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:33.395 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.391+0000 7fe715dad700 1 -- 172.21.15.162:0/1970503566 shutdown_connections 2020-11-28T21:50:33.395 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.391+0000 7fe715dad700 1 -- 172.21.15.162:0/1970503566 wait complete. 2020-11-28T21:50:33.405 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-28T21:50:33.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.523+0000 7f412d9ce700 1 Processor -- start 2020-11-28T21:50:33.521 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.523+0000 7f412d9ce700 1 -- start start 2020-11-28T21:50:33.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.523+0000 7f412d9ce700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f41280af6a0 0x7f41280b2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:33.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.523+0000 7f412d9ce700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4128060a70 0x7f41280afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:33.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.523+0000 7f412d9ce700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4128060f00 con 0x7f41280b1eb0 2020-11-28T21:50:33.526 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.523+0000 7f412d9ce700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41281471f0 con 0x7f41280af6a0 2020-11-28T21:50:33.526 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.523+0000 7f412d9ce700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4128060090 con 0x7f4128060a70 2020-11-28T21:50:33.527 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.523+0000 7f412d1cd700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4128060a70 0x7f41280afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:33.527 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.523+0000 7f412d1cd700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4128060a70 0x7f41280afdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:37528/0 (socket says 172.21.15.162:37528) 2020-11-28T21:50:33.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.523+0000 7f412d1cd700 1 -- 172.21.15.162:0/1876182279 learned_addr learned my addr 172.21.15.162:0/1876182279 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:33.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.523+0000 7f4127fff700 1 --2- 172.21.15.162:0/1876182279 >> v2:172.21.15.118:3300/0 conn(0x7f41280af6a0 0x7f41280b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:33.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.523+0000 7f412d1cd700 1 -- 172.21.15.162:0/1876182279 >> v2:172.21.15.118:3300/0 conn(0x7f41280af6a0 msgr2=0x7f41280b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:33.530 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.523+0000 7f412d1cd700 1 --2- 172.21.15.162:0/1876182279 >> v2:172.21.15.118:3300/0 conn(0x7f41280af6a0 0x7f41280b2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:33.530 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.523+0000 7f412d1cd700 1 -- 172.21.15.162:0/1876182279 >> v1:172.21.15.118:6789/0 conn(0x7f41280b1eb0 legacy=0x7f4128144350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:33.530 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.523+0000 7f41277fe700 1 -- 172.21.15.162:0/1876182279 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 516961545 0 0) 0x7f4128060f00 con 0x7f41280b1eb0 2020-11-28T21:50:33.531 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.523+0000 7f412d1cd700 1 -- 172.21.15.162:0/1876182279 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4128060330 con 0x7f4128060a70 2020-11-28T21:50:33.531 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.523+0000 7f412d1cd700 1 --2- 172.21.15.162:0/1876182279 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4128060a70 0x7f41280afdc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f41200032b0 tx=0x7f4120009270).ready entity=mon.2 client_cookie=7b8d0156de73af20 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:33.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.523+0000 7f41277fe700 1 -- 172.21.15.162:0/1876182279 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4120009e60 con 0x7f4128060a70 2020-11-28T21:50:33.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.523+0000 7f41277fe700 1 -- 172.21.15.162:0/1876182279 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f412000bc00 con 0x7f4128060a70 2020-11-28T21:50:33.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.523+0000 7f41277fe700 1 -- 172.21.15.162:0/1876182279 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f412000bd70 con 0x7f4128060a70 2020-11-28T21:50:33.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.523+0000 7f412d9ce700 1 -- 172.21.15.162:0/1876182279 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4128060a70 msgr2=0x7f41280afdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:33.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.523+0000 7f412d9ce700 1 --2- 172.21.15.162:0/1876182279 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4128060a70 0x7f41280afdc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f41200032b0 tx=0x7f4120009270).stop 2020-11-28T21:50:33.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.527+0000 7f412d9ce700 1 -- 172.21.15.162:0/1876182279 shutdown_connections 2020-11-28T21:50:33.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.527+0000 7f412d9ce700 1 --2- 172.21.15.162:0/1876182279 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4128060a70 0x7f41280afdc0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:33.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.527+0000 7f412d9ce700 1 --2- 172.21.15.162:0/1876182279 >> v2:172.21.15.118:3300/0 conn(0x7f41280af6a0 0x7f41280b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:33.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.527+0000 7f412d9ce700 1 -- 172.21.15.162:0/1876182279 >> 172.21.15.162:0/1876182279 conn(0x7f41280b57d0 msgr2=0x7f41280b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:33.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.527+0000 7f412d9ce700 1 -- 172.21.15.162:0/1876182279 shutdown_connections 2020-11-28T21:50:33.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.527+0000 7f412d9ce700 1 -- 172.21.15.162:0/1876182279 wait complete. 2020-11-28T21:50:33.543 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.527+0000 7f412d9ce700 1 Processor -- start 2020-11-28T21:50:33.544 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.527+0000 7f412d9ce700 1 -- start start 2020-11-28T21:50:33.544 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.531+0000 7f412d9ce700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4128060a70 0x7f41280b3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:33.545 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.531+0000 7f412d9ce700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41280af6a0 0x7f41281596b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:33.545 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.531+0000 7f412d9ce700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f41280b1eb0 0x7f4128159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:33.545 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.531+0000 7f412d9ce700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41280671d0 con 0x7f41280b1eb0 2020-11-28T21:50:33.546 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.531+0000 7f412d9ce700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41280abea0 con 0x7f41280af6a0 2020-11-28T21:50:33.546 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.531+0000 7f412d9ce700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f412815a090 con 0x7f4128060a70 2020-11-28T21:50:33.547 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.531+0000 7f4127fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41280af6a0 0x7f41281596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:33.547 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.531+0000 7f4127fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41280af6a0 0x7f41281596b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40310/0 (socket says 172.21.15.162:40310) 2020-11-28T21:50:33.547 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.531+0000 7f4127fff700 1 -- 172.21.15.162:0/1004795329 learned_addr learned my addr 172.21.15.162:0/1004795329 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:33.547 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.531+0000 7f412d1cd700 1 --2- 172.21.15.162:0/1004795329 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f41280b1eb0 0x7f4128159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:33.548 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.531+0000 7f412c9cc700 1 --2- 172.21.15.162:0/1004795329 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4128060a70 0x7f41280b3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:33.548 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.531+0000 7f4127fff700 1 -- 172.21.15.162:0/1004795329 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4128060a70 msgr2=0x7f41280b3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:33.548 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.531+0000 7f4127fff700 1 --2- 172.21.15.162:0/1004795329 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4128060a70 0x7f41280b3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:33.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.531+0000 7f4127fff700 1 -- 172.21.15.162:0/1004795329 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f41280b1eb0 msgr2=0x7f4128159ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:33.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.531+0000 7f4127fff700 1 --2- 172.21.15.162:0/1004795329 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f41280b1eb0 0x7f4128159ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:33.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.531+0000 7f4127fff700 1 -- 172.21.15.162:0/1004795329 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41280abc00 con 0x7f41280af6a0 2020-11-28T21:50:33.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.531+0000 7f4127fff700 1 --2- 172.21.15.162:0/1004795329 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41280af6a0 0x7f41281596b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7f4128066e50 tx=0x7f411c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:33.550 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.531+0000 7f4125ffb700 1 -- 172.21.15.162:0/1004795329 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f411c00c820 con 0x7f41280af6a0 2020-11-28T21:50:33.550 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.531+0000 7f4125ffb700 1 -- 172.21.15.162:0/1004795329 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411c00c980 con 0x7f41280af6a0 2020-11-28T21:50:33.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.531+0000 7f412d9ce700 1 -- 172.21.15.162:0/1004795329 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4128066f30 con 0x7f41280af6a0 2020-11-28T21:50:33.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.531+0000 7f4125ffb700 1 -- 172.21.15.162:0/1004795329 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f411c00cce0 con 0x7f41280af6a0 2020-11-28T21:50:33.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.531+0000 7f412d9ce700 1 -- 172.21.15.162:0/1004795329 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4128063a40 con 0x7f41280af6a0 2020-11-28T21:50:33.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.531+0000 7f412d9ce700 1 -- 172.21.15.162:0/1004795329 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4128057050 con 0x7f41280af6a0 2020-11-28T21:50:33.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.531+0000 7f4125ffb700 1 -- 172.21.15.162:0/1004795329 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f411c022070 con 0x7f41280af6a0 2020-11-28T21:50:33.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.535+0000 7f4125ffb700 1 --2- 172.21.15.162:0/1004795329 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f41080553b0 0x7f4108057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:33.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.535+0000 7f412c9cc700 1 --2- 172.21.15.162:0/1004795329 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f41080553b0 0x7f4108057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:33.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.535+0000 7f4125ffb700 1 -- 172.21.15.162:0/1004795329 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f411c06e2f0 con 0x7f41280af6a0 2020-11-28T21:50:33.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.535+0000 7f412c9cc700 1 --2- 172.21.15.162:0/1004795329 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f41080553b0 0x7f4108057870 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f411800d000 tx=0x7f4118007b60).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:33.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.539+0000 7f4125ffb700 1 -- 172.21.15.162:0/1004795329 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f411c0445f0 con 0x7f41280af6a0 2020-11-28T21:50:33.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.679+0000 7f412d9ce700 1 -- 172.21.15.162:0/1004795329 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f41280a89f0 con 0x7f41280af6a0 2020-11-28T21:50:33.680 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.683+0000 7f4125ffb700 1 -- 172.21.15.162:0/1004795329 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v56) v1 ==== 104+0+10 (secure 0 0 0) 0x7f411c047c10 con 0x7f41280af6a0 2020-11-28T21:50:33.681 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:50:33.683 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.683+0000 7f412d9ce700 1 -- 172.21.15.162:0/1004795329 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f41080553b0 msgr2=0x7f4108057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:33.684 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.683+0000 7f412d9ce700 1 --2- 172.21.15.162:0/1004795329 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f41080553b0 0x7f4108057870 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f411800d000 tx=0x7f4118007b60).stop 2020-11-28T21:50:33.684 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.683+0000 7f412d9ce700 1 -- 172.21.15.162:0/1004795329 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41280af6a0 msgr2=0x7f41281596b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:33.684 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.683+0000 7f412d9ce700 1 --2- 172.21.15.162:0/1004795329 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41280af6a0 0x7f41281596b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7f4128066e50 tx=0x7f411c008040).stop 2020-11-28T21:50:33.685 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.683+0000 7f412d9ce700 1 -- 172.21.15.162:0/1004795329 shutdown_connections 2020-11-28T21:50:33.685 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.683+0000 7f412d9ce700 1 --2- 172.21.15.162:0/1004795329 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f41080553b0 0x7f4108057870 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:33.686 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.683+0000 7f412d9ce700 1 --2- 172.21.15.162:0/1004795329 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4128060a70 0x7f41280b3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:33.686 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.683+0000 7f412d9ce700 1 --2- 172.21.15.162:0/1004795329 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f41280b1eb0 0x7f4128159ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:33.687 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.683+0000 7f412d9ce700 1 --2- 172.21.15.162:0/1004795329 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41280af6a0 0x7f41281596b0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:33.687 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.683+0000 7f412d9ce700 1 -- 172.21.15.162:0/1004795329 >> 172.21.15.162:0/1004795329 conn(0x7f41280b57d0 msgr2=0x7f4128146ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:33.687 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.683+0000 7f412d9ce700 1 -- 172.21.15.162:0/1004795329 shutdown_connections 2020-11-28T21:50:33.689 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.683+0000 7f412d9ce700 1 -- 172.21.15.162:0/1004795329 wait complete. 2020-11-28T21:50:33.696 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-28T21:50:33.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.819+0000 7f90189bc700 1 Processor -- start 2020-11-28T21:50:33.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.819+0000 7f90189bc700 1 -- start start 2020-11-28T21:50:33.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.819+0000 7f90189bc700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f90140b9ce0 0x7f90141433c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:33.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.819+0000 7f90189bc700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f9014067610 0x7f901413fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:33.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.819+0000 7f90189bc700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9014143950 con 0x7f90140605c0 2020-11-28T21:50:33.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.819+0000 7f90189bc700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90140b4340 con 0x7f9014067610 2020-11-28T21:50:33.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.819+0000 7f90189bc700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90140ba0c0 con 0x7f90140b9ce0 2020-11-28T21:50:33.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.819+0000 7f9011d9b700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f90140605c0 0x7f90140b3c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:47160/0 (socket says 172.21.15.162:47160) 2020-11-28T21:50:33.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.819+0000 7f9011d9b700 1 -- 172.21.15.162:0/3451807883 learned_addr learned my addr 172.21.15.162:0/3451807883 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:50:33.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.819+0000 7f901259c700 1 --2- 172.21.15.162:0/3451807883 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f90140b9ce0 0x7f90141433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:33.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.819+0000 7f9012d9d700 1 --2- 172.21.15.162:0/3451807883 >> v2:172.21.15.118:3300/0 conn(0x7f9014067610 0x7f901413fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:33.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.819+0000 7f901159a700 1 -- 172.21.15.162:0/3451807883 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2820637351 0 0) 0x7f9014143950 con 0x7f90140605c0 2020-11-28T21:50:33.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.819+0000 7f901159a700 1 -- 172.21.15.162:0/3451807883 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ff8003230 con 0x7f90140605c0 2020-11-28T21:50:33.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.819+0000 7f901159a700 1 -- 172.21.15.162:0/3451807883 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3323774542 0 0) 0x7f8ff8003230 con 0x7f90140605c0 2020-11-28T21:50:33.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.819+0000 7f901259c700 1 -- 172.21.15.162:0/3451807883 >> v2:172.21.15.118:3300/0 conn(0x7f9014067610 msgr2=0x7f901413fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:50:33.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.819+0000 7f901259c700 1 --2- 172.21.15.162:0/3451807883 >> v2:172.21.15.118:3300/0 conn(0x7f9014067610 0x7f901413fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:33.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.819+0000 7f901259c700 1 -- 172.21.15.162:0/3451807883 >> v1:172.21.15.162:6789/0 conn(0x7f90140605c0 legacy=0x7f90140b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:33.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.819+0000 7f901259c700 1 -- 172.21.15.162:0/3451807883 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9014067aa0 con 0x7f90140b9ce0 2020-11-28T21:50:33.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.819+0000 7f9012d9d700 1 --2- 172.21.15.162:0/3451807883 >> v2:172.21.15.118:3300/0 conn(0x7f9014067610 0x7f901413fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:50:33.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.819+0000 7f901259c700 1 --2- 172.21.15.162:0/3451807883 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f90140b9ce0 0x7f90141433c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f9008001870 tx=0x7f9004006d60).ready entity=mon.2 client_cookie=846aa572d97fbdf7 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:33.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.819+0000 7f901159a700 1 -- 172.21.15.162:0/3451807883 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f90080016d0 con 0x7f90140b9ce0 2020-11-28T21:50:33.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.819+0000 7f901159a700 1 -- 172.21.15.162:0/3451807883 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f900400f510 con 0x7f90140b9ce0 2020-11-28T21:50:33.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.819+0000 7f901159a700 1 -- 172.21.15.162:0/3451807883 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f900400f680 con 0x7f90140b9ce0 2020-11-28T21:50:33.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.819+0000 7f90189bc700 1 -- 172.21.15.162:0/3451807883 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f90140b9ce0 msgr2=0x7f90141433c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:33.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.819+0000 7f90189bc700 1 --2- 172.21.15.162:0/3451807883 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f90140b9ce0 0x7f90141433c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f9008001870 tx=0x7f9004006d60).stop 2020-11-28T21:50:33.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.819+0000 7f90189bc700 1 -- 172.21.15.162:0/3451807883 shutdown_connections 2020-11-28T21:50:33.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.819+0000 7f90189bc700 1 --2- 172.21.15.162:0/3451807883 >> v2:172.21.15.118:3300/0 conn(0x7f9014067610 0x7f901413fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:33.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.819+0000 7f90189bc700 1 --2- 172.21.15.162:0/3451807883 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f90140b9ce0 0x7f90141433c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:33.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.819+0000 7f90189bc700 1 -- 172.21.15.162:0/3451807883 >> 172.21.15.162:0/3451807883 conn(0x7f901405c430 msgr2=0x7f90140b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:33.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.819+0000 7f90189bc700 1 -- 172.21.15.162:0/3451807883 shutdown_connections 2020-11-28T21:50:33.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.819+0000 7f90189bc700 1 -- 172.21.15.162:0/3451807883 wait complete. 2020-11-28T21:50:33.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.823+0000 7f90189bc700 1 Processor -- start 2020-11-28T21:50:33.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.823+0000 7f90189bc700 1 -- start start 2020-11-28T21:50:33.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.823+0000 7f90189bc700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f90140605c0 0x7f9014142260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:33.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.823+0000 7f90189bc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9014067610 0x7f90140abce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:33.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.823+0000 7f90189bc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f90140b9ce0 0x7f90140ac1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:33.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.823+0000 7f90189bc700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9014146e60 con 0x7f90140b9ce0 2020-11-28T21:50:33.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.823+0000 7f90189bc700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9014146bc0 con 0x7f9014067610 2020-11-28T21:50:33.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.823+0000 7f90189bc700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90140b0710 con 0x7f90140605c0 2020-11-28T21:50:33.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.823+0000 7f9011d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9014067610 0x7f90140abce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:33.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.823+0000 7f9011d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9014067610 0x7f90140abce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40324/0 (socket says 172.21.15.162:40324) 2020-11-28T21:50:33.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.823+0000 7f9011d9b700 1 -- 172.21.15.162:0/2840593858 learned_addr learned my addr 172.21.15.162:0/2840593858 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:33.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.823+0000 7f901259c700 1 --2- 172.21.15.162:0/2840593858 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f90140605c0 0x7f9014142260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:33.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.823+0000 7f9012d9d700 1 --2- 172.21.15.162:0/2840593858 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f90140b9ce0 0x7f90140ac1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:33.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.823+0000 7f9011d9b700 1 -- 172.21.15.162:0/2840593858 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f90140605c0 msgr2=0x7f9014142260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:33.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.823+0000 7f9011d9b700 1 --2- 172.21.15.162:0/2840593858 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f90140605c0 0x7f9014142260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:33.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.823+0000 7f9011d9b700 1 -- 172.21.15.162:0/2840593858 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f90140b9ce0 msgr2=0x7f90140ac1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:33.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.823+0000 7f9011d9b700 1 --2- 172.21.15.162:0/2840593858 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f90140b9ce0 0x7f90140ac1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:33.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.823+0000 7f9011d9b700 1 -- 172.21.15.162:0/2840593858 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90140ac6c0 con 0x7f9014067610 2020-11-28T21:50:33.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.823+0000 7f9012d9d700 1 --2- 172.21.15.162:0/2840593858 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f90140b9ce0 0x7f90140ac1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:50:33.838 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.823+0000 7f9011d9b700 1 --2- 172.21.15.162:0/2840593858 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9014067610 0x7f90140abce0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f9008001930 tx=0x7f9008001af0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:33.838 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.823+0000 7f90037fe700 1 -- 172.21.15.162:0/2840593858 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9008007170 con 0x7f9014067610 2020-11-28T21:50:33.838 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.823+0000 7f90037fe700 1 -- 172.21.15.162:0/2840593858 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90080072d0 con 0x7f9014067610 2020-11-28T21:50:33.838 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.823+0000 7f90189bc700 1 -- 172.21.15.162:0/2840593858 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9014145290 con 0x7f9014067610 2020-11-28T21:50:33.839 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.823+0000 7f90037fe700 1 -- 172.21.15.162:0/2840593858 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f900800a9a0 con 0x7f9014067610 2020-11-28T21:50:33.839 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.823+0000 7f90189bc700 1 -- 172.21.15.162:0/2840593858 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90140b8120 con 0x7f9014067610 2020-11-28T21:50:33.839 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.827+0000 7f90037fe700 1 -- 172.21.15.162:0/2840593858 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9008010070 con 0x7f9014067610 2020-11-28T21:50:33.839 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.827+0000 7f90189bc700 1 -- 172.21.15.162:0/2840593858 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9014057050 con 0x7f9014067610 2020-11-28T21:50:33.839 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.827+0000 7f90037fe700 1 --2- 172.21.15.162:0/2840593858 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8ff00554e0 0x7f8ff00579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:33.840 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.827+0000 7f901259c700 1 --2- 172.21.15.162:0/2840593858 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8ff00554e0 0x7f8ff00579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:33.840 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.827+0000 7f90037fe700 1 -- 172.21.15.162:0/2840593858 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f900806de40 con 0x7f9014067610 2020-11-28T21:50:33.840 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.827+0000 7f901259c700 1 --2- 172.21.15.162:0/2840593858 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8ff00554e0 0x7f8ff00579a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f9004017000 tx=0x7f9004014000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:33.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.831+0000 7f90037fe700 1 -- 172.21.15.162:0/2840593858 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9008044030 con 0x7f9014067610 2020-11-28T21:50:33.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.971+0000 7f90189bc700 1 -- 172.21.15.162:0/2840593858 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9014068350 con 0x7f9014067610 2020-11-28T21:50:33.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.971+0000 7f90037fe700 1 -- 172.21.15.162:0/2840593858 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v56) v1 ==== 100+0+10 (secure 0 0 0) 0x7f9008047650 con 0x7f9014067610 2020-11-28T21:50:33.970 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:50:33.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.971+0000 7f90189bc700 1 -- 172.21.15.162:0/2840593858 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8ff00554e0 msgr2=0x7f8ff00579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:33.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.971+0000 7f90189bc700 1 --2- 172.21.15.162:0/2840593858 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8ff00554e0 0x7f8ff00579a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f9004017000 tx=0x7f9004014000).stop 2020-11-28T21:50:33.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.971+0000 7f90189bc700 1 -- 172.21.15.162:0/2840593858 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9014067610 msgr2=0x7f90140abce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:33.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.971+0000 7f90189bc700 1 --2- 172.21.15.162:0/2840593858 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9014067610 0x7f90140abce0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f9008001930 tx=0x7f9008001af0).stop 2020-11-28T21:50:33.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.975+0000 7f90189bc700 1 -- 172.21.15.162:0/2840593858 shutdown_connections 2020-11-28T21:50:33.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.975+0000 7f90189bc700 1 --2- 172.21.15.162:0/2840593858 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8ff00554e0 0x7f8ff00579a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:33.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.975+0000 7f90189bc700 1 --2- 172.21.15.162:0/2840593858 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f90140605c0 0x7f9014142260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:33.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.975+0000 7f90189bc700 1 --2- 172.21.15.162:0/2840593858 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f90140b9ce0 0x7f90140ac1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:33.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.975+0000 7f90189bc700 1 --2- 172.21.15.162:0/2840593858 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9014067610 0x7f90140abce0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:33.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.975+0000 7f90189bc700 1 -- 172.21.15.162:0/2840593858 >> 172.21.15.162:0/2840593858 conn(0x7f901405c430 msgr2=0x7f901405f750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:33.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.975+0000 7f90189bc700 1 -- 172.21.15.162:0/2840593858 shutdown_connections 2020-11-28T21:50:33.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:33.975+0000 7f90189bc700 1 -- 172.21.15.162:0/2840593858 wait complete. 2020-11-28T21:50:33.988 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-28T21:50:33.989 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:50:34.098 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.099+0000 7f6f95e98700 1 Processor -- start 2020-11-28T21:50:34.098 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.099+0000 7f6f95e98700 1 -- start start 2020-11-28T21:50:34.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.103+0000 7f6f95e98700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f6f90140cb0 0x7f6f90141120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:34.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.103+0000 7f6f95e98700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f90060a10 con 0x7f6f9009abb0 2020-11-28T21:50:34.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.103+0000 7f6f95e98700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f900b3b00 con 0x7f6f90060010 2020-11-28T21:50:34.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.103+0000 7f6f95e98700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f90060770 con 0x7f6f90140cb0 2020-11-28T21:50:34.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.103+0000 7f6f87fff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f6f90060010 0x7f6f901405a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:47176/0 (socket says 172.21.15.162:47176) 2020-11-28T21:50:34.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.103+0000 7f6f87fff700 1 -- 172.21.15.162:0/2665870257 learned_addr learned my addr 172.21.15.162:0/2665870257 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:50:34.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.103+0000 7f6f95697700 1 --2- 172.21.15.162:0/2665870257 >> v2:172.21.15.118:3300/0 conn(0x7f6f90140cb0 0x7f6f90141120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:34.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.103+0000 7f6f877fe700 1 -- 172.21.15.162:0/2665870257 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1629799881 0 0) 0x7f6f900b3b00 con 0x7f6f90060010 2020-11-28T21:50:34.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.103+0000 7f6f877fe700 1 -- 172.21.15.162:0/2665870257 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f7c003230 con 0x7f6f90060010 2020-11-28T21:50:34.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.103+0000 7f6f877fe700 1 -- 172.21.15.162:0/2665870257 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2406933730 0 0) 0x7f6f90060a10 con 0x7f6f9009abb0 2020-11-28T21:50:34.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.103+0000 7f6f877fe700 1 -- 172.21.15.162:0/2665870257 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f900b3b00 con 0x7f6f9009abb0 2020-11-28T21:50:34.104 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.103+0000 7f6f95697700 1 -- 172.21.15.162:0/2665870257 >> v1:172.21.15.162:6789/0 conn(0x7f6f90060010 legacy=0x7f6f901405a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:34.104 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.103+0000 7f6f95697700 1 -- 172.21.15.162:0/2665870257 >> v1:172.21.15.118:6789/0 conn(0x7f6f9009abb0 legacy=0x7f6f900b33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:34.104 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.103+0000 7f6f95697700 1 -- 172.21.15.162:0/2665870257 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f900aff30 con 0x7f6f90140cb0 2020-11-28T21:50:34.104 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.103+0000 7f6f95697700 1 --2- 172.21.15.162:0/2665870257 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f90140cb0 0x7f6f90141120 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f6f880032b0 tx=0x7f6f880091a0).ready entity=mon.0 client_cookie=9d237f88168823b0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:34.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.103+0000 7f6f877fe700 1 -- 172.21.15.162:0/2665870257 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f8800b940 con 0x7f6f90140cb0 2020-11-28T21:50:34.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.103+0000 7f6f877fe700 1 -- 172.21.15.162:0/2665870257 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f8800baa0 con 0x7f6f90140cb0 2020-11-28T21:50:34.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.103+0000 7f6f877fe700 1 -- 172.21.15.162:0/2665870257 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f8800bc10 con 0x7f6f90140cb0 2020-11-28T21:50:34.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.103+0000 7f6f95e98700 1 -- 172.21.15.162:0/2665870257 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f90140cb0 msgr2=0x7f6f90141120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:34.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.103+0000 7f6f95e98700 1 --2- 172.21.15.162:0/2665870257 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f90140cb0 0x7f6f90141120 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f6f880032b0 tx=0x7f6f880091a0).stop 2020-11-28T21:50:34.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.103+0000 7f6f95e98700 1 -- 172.21.15.162:0/2665870257 shutdown_connections 2020-11-28T21:50:34.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.103+0000 7f6f95e98700 1 --2- 172.21.15.162:0/2665870257 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f90140cb0 0x7f6f90141120 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:34.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.103+0000 7f6f95e98700 1 -- 172.21.15.162:0/2665870257 >> 172.21.15.162:0/2665870257 conn(0x7f6f900b54f0 msgr2=0x7f6f900b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:34.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.103+0000 7f6f95e98700 1 -- 172.21.15.162:0/2665870257 shutdown_connections 2020-11-28T21:50:34.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.103+0000 7f6f95e98700 1 -- 172.21.15.162:0/2665870257 wait complete. 2020-11-28T21:50:34.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.107+0000 7f6f95e98700 1 Processor -- start 2020-11-28T21:50:34.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.107+0000 7f6f95e98700 1 -- start start 2020-11-28T21:50:34.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.111+0000 7f6f95e98700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6f90060010 0x7f6f90144100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:34.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.111+0000 7f6f95e98700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f9009abb0 0x7f6f9013e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:34.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.111+0000 7f6f95e98700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f90140cb0 0x7f6f9013eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:34.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.111+0000 7f6f95e98700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f90067a70 con 0x7f6f9009abb0 2020-11-28T21:50:34.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.111+0000 7f6f95e98700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f90064210 con 0x7f6f90140cb0 2020-11-28T21:50:34.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.111+0000 7f6f95e98700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f90063cd0 con 0x7f6f90060010 2020-11-28T21:50:34.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.111+0000 7f6f95697700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f90140cb0 0x7f6f9013eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:34.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.111+0000 7f6f95697700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f90140cb0 0x7f6f9013eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40340/0 (socket says 172.21.15.162:40340) 2020-11-28T21:50:34.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.111+0000 7f6f95697700 1 -- 172.21.15.162:0/341412306 learned_addr learned my addr 172.21.15.162:0/341412306 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:50:34.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.111+0000 7f6f87fff700 1 --2- 172.21.15.162:0/341412306 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f9009abb0 0x7f6f9013e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:34.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.111+0000 7f6f94e96700 1 --2- 172.21.15.162:0/341412306 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6f90060010 0x7f6f90144100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:34.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.111+0000 7f6f95697700 1 -- 172.21.15.162:0/341412306 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6f90060010 msgr2=0x7f6f90144100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:34.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.111+0000 7f6f95697700 1 --2- 172.21.15.162:0/341412306 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6f90060010 0x7f6f90144100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:34.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.111+0000 7f6f95697700 1 -- 172.21.15.162:0/341412306 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f9009abb0 msgr2=0x7f6f9013e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:34.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.111+0000 7f6f95697700 1 --2- 172.21.15.162:0/341412306 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f9009abb0 0x7f6f9013e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:34.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.111+0000 7f6f95697700 1 -- 172.21.15.162:0/341412306 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f90063f70 con 0x7f6f90140cb0 2020-11-28T21:50:34.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.111+0000 7f6f95697700 1 --2- 172.21.15.162:0/341412306 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f90140cb0 0x7f6f9013eba0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7f6f880032b0 tx=0x7f6f88016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:34.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.111+0000 7f6f85ffb700 1 -- 172.21.15.162:0/341412306 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f880203f0 con 0x7f6f90140cb0 2020-11-28T21:50:34.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.111+0000 7f6f85ffb700 1 -- 172.21.15.162:0/341412306 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f88020550 con 0x7f6f90140cb0 2020-11-28T21:50:34.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.111+0000 7f6f95e98700 1 -- 172.21.15.162:0/341412306 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f900672e0 con 0x7f6f90140cb0 2020-11-28T21:50:34.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.111+0000 7f6f85ffb700 1 -- 172.21.15.162:0/341412306 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f880208b0 con 0x7f6f90140cb0 2020-11-28T21:50:34.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.111+0000 7f6f95e98700 1 -- 172.21.15.162:0/341412306 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f900675d0 con 0x7f6f90140cb0 2020-11-28T21:50:34.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.111+0000 7f6f95e98700 1 -- 172.21.15.162:0/341412306 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f90099ec0 con 0x7f6f90140cb0 2020-11-28T21:50:34.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.115+0000 7f6f85ffb700 1 -- 172.21.15.162:0/341412306 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6f8801e070 con 0x7f6f90140cb0 2020-11-28T21:50:34.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.115+0000 7f6f85ffb700 1 --2- 172.21.15.162:0/341412306 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6f700553b0 0x7f6f70057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:50:34.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.115+0000 7f6f94e96700 1 --2- 172.21.15.162:0/341412306 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6f700553b0 0x7f6f70057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:50:34.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.115+0000 7f6f85ffb700 1 -- 172.21.15.162:0/341412306 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6f8806eb70 con 0x7f6f90140cb0 2020-11-28T21:50:34.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.115+0000 7f6f94e96700 1 --2- 172.21.15.162:0/341412306 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6f700553b0 0x7f6f70057870 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f6f8c00d000 tx=0x7f6f8c0079f0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:50:34.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.119+0000 7f6f85ffb700 1 -- 172.21.15.162:0/341412306 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6f90099ec0 con 0x7f6f90140cb0 2020-11-28T21:50:34.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.303+0000 7f6f95e98700 1 -- 172.21.15.162:0/341412306 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f90099ec0 con 0x7f6f90140cb0 2020-11-28T21:50:34.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.307+0000 7f6f85ffb700 1 -- 172.21.15.162:0/341412306 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+2501 (secure 0 0 0) 0x7f6f90099ec0 con 0x7f6f90140cb0 2020-11-28T21:50:34.304 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:50:34.304 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":21,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/1245801281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1245801281},{"type":"v1","addr":"172.21.15.118:6833","nonce":1245801281}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":18},{"gid":5536,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6835/3629429916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3629429916},{"type":"v1","addr":"172.21.15.162:6835","nonce":3629429916}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":19}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:50:05.655172+0000","modified":"2020-11-28T21:50:06.657968+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5551},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5551":{"gid":5551,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/1902721947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1902721947},{"type":"v1","addr":"172.21.15.162:6837","nonce":1902721947}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-28T21:50:34.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.307+0000 7f6f95e98700 1 -- 172.21.15.162:0/341412306 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6f700553b0 msgr2=0x7f6f70057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:34.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.307+0000 7f6f95e98700 1 --2- 172.21.15.162:0/341412306 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6f700553b0 0x7f6f70057870 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f6f8c00d000 tx=0x7f6f8c0079f0).stop 2020-11-28T21:50:34.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.307+0000 7f6f95e98700 1 -- 172.21.15.162:0/341412306 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f90140cb0 msgr2=0x7f6f9013eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:50:34.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.307+0000 7f6f95e98700 1 --2- 172.21.15.162:0/341412306 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f90140cb0 0x7f6f9013eba0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7f6f880032b0 tx=0x7f6f88016000).stop 2020-11-28T21:50:34.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.307+0000 7f6f95e98700 1 -- 172.21.15.162:0/341412306 shutdown_connections 2020-11-28T21:50:34.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.307+0000 7f6f95e98700 1 --2- 172.21.15.162:0/341412306 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6f700553b0 0x7f6f70057870 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:34.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.307+0000 7f6f95e98700 1 --2- 172.21.15.162:0/341412306 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6f90060010 0x7f6f90144100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:34.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.307+0000 7f6f95e98700 1 --2- 172.21.15.162:0/341412306 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f90140cb0 0x7f6f9013eba0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:34.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.307+0000 7f6f95e98700 1 --2- 172.21.15.162:0/341412306 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f9009abb0 0x7f6f9013e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:50:34.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.307+0000 7f6f95e98700 1 -- 172.21.15.162:0/341412306 >> 172.21.15.162:0/341412306 conn(0x7f6f900b54f0 msgr2=0x7f6f90141da0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:50:34.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.307+0000 7f6f95e98700 1 -- 172.21.15.162:0/341412306 shutdown_connections 2020-11-28T21:50:34.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:50:34.307+0000 7f6f95e98700 1 -- 172.21.15.162:0/341412306 wait complete. 2020-11-28T21:50:34.317 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 21 2020-11-28T21:50:34.623 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-28T21:50:05.655172+0000', 'modified': '2020-11-28T21:50:06.657968+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5551}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5551': {'gid': 5551, 'name': 'c', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6837/1902721947', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6836', 'nonce': 1902721947}, {'type': 'v1', 'addr': '172.21.15.162:6837', 'nonce': 1902721947}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-28T21:50:34.624 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-28T21:50:34.624 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-28T21:50:34.752 INFO:teuthology.orchestra.run.smithi162.stdout:{"cluster_fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","whoami":0,"id":5551,"want_state":"up:active","state":"up:active","rank_uptime":29.087545455000001,"mdsmap_epoch":21,"osdmap_epoch":56,"osdmap_epoch_barrier":54,"uptime":34.443970174} 2020-11-28T21:50:34.753 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", "id": 5551, "mdsmap_epoch": 21, "osdmap_epoch": 56, "osdmap_epoch_barrier": 54, "rank_uptime": 29.087545455, "state": "up:active", "uptime": 34.443970174, "want_state": "up:active", "whoami": 0 } 2020-11-28T21:50:34.754 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-11-28T21:50:34.754 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-11-28T21:50:34.754 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi192.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-11-28T21:50:34.755 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:50:34.755 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:50:34.763 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:50:34.819 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:50:34.820 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:50:34.881 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:50:34.881 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:50:34.882 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:50:34.882 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:50:34.931 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-11-28T21:50:34.931 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:50:34.979 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:50:34.995 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:50:34.996 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:50:35.050 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:50:35.052 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:50:35.053 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:50:35.054 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:50:35.055 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:2020-11-28T21:50:35.053+0000 7f14fa7c00c0 -1 init, newargv = 0x55671786d7a0 newargc=9 2020-11-28T21:50:35.056 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:ceph-fuse[11777]: starting ceph client 2020-11-28T21:50:35.064 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:ceph-fuse[11777]: starting fuse 2020-11-28T21:50:35.071 INFO:teuthology.orchestra.run.smithi192.stdout:49 2020-11-28T21:50:35.073 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2020-11-28T21:50:35.073 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:50:35.074 INFO:teuthology.orchestra.run.smithi192:> sudo python3 -c ' 2020-11-28T21:50:35.075 INFO:teuthology.orchestra.run.smithi192:> import glob 2020-11-28T21:50:35.076 INFO:teuthology.orchestra.run.smithi192:> import re 2020-11-28T21:50:35.077 INFO:teuthology.orchestra.run.smithi192:> import os 2020-11-28T21:50:35.077 INFO:teuthology.orchestra.run.smithi192:> import subprocess 2020-11-28T21:50:35.078 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:50:35.079 INFO:teuthology.orchestra.run.smithi192:> def find_socket(client_name): 2020-11-28T21:50:35.080 INFO:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-11-28T21:50:35.080 INFO:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2020-11-28T21:50:35.081 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:50:35.082 INFO:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2020-11-28T21:50:35.083 INFO:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2020-11-28T21:50:35.083 INFO:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2020-11-28T21:50:35.084 INFO:teuthology.orchestra.run.smithi192:> return files[0] 2020-11-28T21:50:35.085 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:50:35.086 INFO:teuthology.orchestra.run.smithi192:> for f in files: 2020-11-28T21:50:35.087 INFO:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-11-28T21:50:35.088 INFO:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2020-11-28T21:50:35.088 INFO:teuthology.orchestra.run.smithi192:> return f 2020-11-28T21:50:35.089 INFO:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-11-28T21:50:35.090 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:50:35.091 INFO:teuthology.orchestra.run.smithi192:> print(find_socket("client.0")) 2020-11-28T21:50:35.092 INFO:teuthology.orchestra.run.smithi192:> ' 2020-11-28T21:50:35.168 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.0.11777.asok 2020-11-28T21:50:35.172 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.11777.asok 2020-11-28T21:50:35.173 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:50:35.173 INFO:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.11777.asok status 2020-11-28T21:50:35.275 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2020-11-28T21:50:35.276 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2020-11-28T21:50:35.276 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "a83a3572a957356977342f461361c1cd3ec06dbd", 2020-11-28T21:50:35.276 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 15.2.6-595-ga83a3572 (a83a3572a957356977342f461361c1cd3ec06dbd) octopus (stable)", 2020-11-28T21:50:35.276 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "0", 2020-11-28T21:50:35.277 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2020-11-28T21:50:35.277 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-11-28T21:50:35.277 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "11777", 2020-11-28T21:50:35.277 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2020-11-28T21:50:35.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:50:35.278 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2020-11-28T21:50:35.278 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2020-11-28T21:50:35.278 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 5691, 2020-11-28T21:50:35.279 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2020-11-28T21:50:35.279 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2020-11-28T21:50:35.279 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2020-11-28T21:50:35.279 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 5691 2020-11-28T21:50:35.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:50:35.280 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2020-11-28T21:50:35.280 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2020-11-28T21:50:35.280 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2020-11-28T21:50:35.281 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 1605041043 2020-11-28T21:50:35.281 INFO:teuthology.orchestra.run.smithi192.stdout: } 2020-11-28T21:50:35.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:50:35.282 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2020-11-28T21:50:35.282 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2020-11-28T21:50:35.282 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2020-11-28T21:50:35.282 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 1605041043 2020-11-28T21:50:35.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:50:35.283 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.5691 172.21.15.192:0/1605041043", 2020-11-28T21:50:35.283 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "172.21.15.192:0/1605041043", 2020-11-28T21:50:35.283 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2020-11-28T21:50:35.283 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 21, 2020-11-28T21:50:35.284 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 56, 2020-11-28T21:50:35.284 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2020-11-28T21:50:35.284 INFO:teuthology.orchestra.run.smithi192.stdout: "blacklisted": false 2020-11-28T21:50:35.284 INFO:teuthology.orchestra.run.smithi192.stdout:} 2020-11-28T21:50:35.286 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:50:35.286 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:50:35.287 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:50:35.295 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2020-11-28T21:50:35.296 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-28T21:50:35.297 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:50:35.297 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:50:37.744 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:50:37.763 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:50:37.784 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:50:37.806 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:51:07.866 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:51:07.868 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:51:07.871 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:51:07.873 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:51:30.306 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-28T21:51:30.307 INFO:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:51:30.330 INFO:teuthology.orchestra.run.smithi118:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:51:30.355 INFO:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:51:30.378 INFO:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:51:30.400 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-28T21:51:30.401 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:51:30.401 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:51:30.402 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:51:30.454 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2020-11-28T21:51:30.455 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-28T21:51:30.455 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2020-11-28T21:51:30.456 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-28T21:51:30.456 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:51:30.545 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:ceph-fuse[11777]: fuse finished with error 0 and tester_r 0 2020-11-28T21:51:30.546 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:51:30.547 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:51:30.547 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:51:30.599 INFO:teuthology.orchestra.run.smithi192.stdout:ext2/ext3 2020-11-28T21:51:30.600 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-11-28T21:51:30.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-28T21:51:30.715 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:51:30.765 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:51:30.766 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:51:30.767 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:51:30.774 INFO:teuthology.orchestra.run.smithi096.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-11-28T21:51:30.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:51:30.776 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2020-11-28T21:51:30.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-28T21:51:30.778 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:51:30.821 INFO:teuthology.orchestra.run.smithi096.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-11-28T21:51:30.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:51:30.823 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config rm mon mon_allow_pool_delete 2020-11-28T21:51:30.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.947+0000 7f9ca3cdd700 1 Processor -- start 2020-11-28T21:51:30.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.951+0000 7f9ca3cdd700 1 -- start start 2020-11-28T21:51:30.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.951+0000 7f9ca3cdd700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9c9c0b60e0 0x7f9c9c0b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:30.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.951+0000 7f9ca3cdd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9c9c0b4390 0x7f9c9c13fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:30.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.951+0000 7f9ca3cdd700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c9c0b4930 con 0x7f9c9c060d10 2020-11-28T21:51:30.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.951+0000 7f9ca3cdd700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c9c0b2170 con 0x7f9c9c0b4390 2020-11-28T21:51:30.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.951+0000 7f9ca3cdd700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9c0b6660 con 0x7f9c9c0b60e0 2020-11-28T21:51:30.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.951+0000 7f9ca227a700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9c9c0b4390 0x7f9c9c13fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:30.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.951+0000 7f9ca227a700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9c9c0b4390 0x7f9c9c13fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40348/0 (socket says 172.21.15.162:40348) 2020-11-28T21:51:30.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.951+0000 7f9ca227a700 1 -- 172.21.15.162:0/2577029803 learned_addr learned my addr 172.21.15.162:0/2577029803 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:30.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.951+0000 7f9ca1278700 1 --2- 172.21.15.162:0/2577029803 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9c9c0b60e0 0x7f9c9c0b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:30.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.951+0000 7f9ca227a700 1 -- 172.21.15.162:0/2577029803 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9c9c0b60e0 msgr2=0x7f9c9c0b52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:51:30.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.951+0000 7f9ca227a700 1 --2- 172.21.15.162:0/2577029803 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9c9c0b60e0 0x7f9c9c0b52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:30.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.951+0000 7f9ca227a700 1 -- 172.21.15.162:0/2577029803 >> v1:172.21.15.118:6789/0 conn(0x7f9c9c060d10 legacy=0x7f9c9c146180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:51:30.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.951+0000 7f9ca227a700 1 -- 172.21.15.162:0/2577029803 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c9c0b4b60 con 0x7f9c9c0b4390 2020-11-28T21:51:30.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.951+0000 7f9ca227a700 1 --2- 172.21.15.162:0/2577029803 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c9c0b4390 0x7f9c9c13fc40 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7f9c940032b0 tx=0x7f9c94009270).ready entity=mon.1 client_cookie=58faca6b8e14e771 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:30.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.951+0000 7f9ca0a77700 1 -- 172.21.15.162:0/2577029803 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c9400bc00 con 0x7f9c9c0b4390 2020-11-28T21:51:30.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.951+0000 7f9ca0a77700 1 -- 172.21.15.162:0/2577029803 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c9400bd60 con 0x7f9c9c0b4390 2020-11-28T21:51:30.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.951+0000 7f9ca0a77700 1 -- 172.21.15.162:0/2577029803 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c94011400 con 0x7f9c9c0b4390 2020-11-28T21:51:30.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.951+0000 7f9ca3cdd700 1 -- 172.21.15.162:0/2577029803 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c9c0b4390 msgr2=0x7f9c9c13fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:30.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.951+0000 7f9ca3cdd700 1 --2- 172.21.15.162:0/2577029803 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c9c0b4390 0x7f9c9c13fc40 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7f9c940032b0 tx=0x7f9c94009270).stop 2020-11-28T21:51:30.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.951+0000 7f9ca3cdd700 1 -- 172.21.15.162:0/2577029803 shutdown_connections 2020-11-28T21:51:30.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.951+0000 7f9ca3cdd700 1 --2- 172.21.15.162:0/2577029803 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c9c0b4390 0x7f9c9c13fc40 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:30.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.951+0000 7f9ca3cdd700 1 --2- 172.21.15.162:0/2577029803 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9c9c0b60e0 0x7f9c9c0b52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:30.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.951+0000 7f9ca3cdd700 1 -- 172.21.15.162:0/2577029803 >> 172.21.15.162:0/2577029803 conn(0x7f9c9c05fe70 msgr2=0x7f9c9c06aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:30.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.951+0000 7f9ca3cdd700 1 -- 172.21.15.162:0/2577029803 shutdown_connections 2020-11-28T21:51:30.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.951+0000 7f9ca3cdd700 1 -- 172.21.15.162:0/2577029803 wait complete. 2020-11-28T21:51:30.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.955+0000 7f9ca3cdd700 1 Processor -- start 2020-11-28T21:51:30.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.955+0000 7f9ca3cdd700 1 -- start start 2020-11-28T21:51:30.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.955+0000 7f9ca3cdd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9c9c060d10 0x7f9c9c0b8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:30.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.955+0000 7f9ca3cdd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c9c0b4390 0x7f9c9c149a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:30.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.955+0000 7f9ca3cdd700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9c9c0b60e0 0x7f9c9c151c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:30.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.955+0000 7f9ca3cdd700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9c155c90 con 0x7f9c9c060d10 2020-11-28T21:51:30.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.955+0000 7f9ca3cdd700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9c155f30 con 0x7f9c9c0b4390 2020-11-28T21:51:30.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.955+0000 7f9ca3cdd700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9c0b7ac0 con 0x7f9c9c0b60e0 2020-11-28T21:51:30.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.955+0000 7f9ca1278700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c9c0b4390 0x7f9c9c149a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:30.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.955+0000 7f9ca1278700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c9c0b4390 0x7f9c9c149a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40350/0 (socket says 172.21.15.162:40350) 2020-11-28T21:51:30.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.955+0000 7f9ca1278700 1 -- 172.21.15.162:0/195007323 learned_addr learned my addr 172.21.15.162:0/195007323 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:30.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.955+0000 7f9ca227a700 1 --2- 172.21.15.162:0/195007323 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9c9c0b60e0 0x7f9c9c151c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:30.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.955+0000 7f9ca1a79700 1 --2- 172.21.15.162:0/195007323 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9c9c060d10 0x7f9c9c0b8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:30.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.955+0000 7f9ca1278700 1 -- 172.21.15.162:0/195007323 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9c9c0b60e0 msgr2=0x7f9c9c151c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:30.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.955+0000 7f9ca1278700 1 --2- 172.21.15.162:0/195007323 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9c9c0b60e0 0x7f9c9c151c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:30.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.955+0000 7f9ca1278700 1 -- 172.21.15.162:0/195007323 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9c9c060d10 msgr2=0x7f9c9c0b8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:30.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.955+0000 7f9ca1278700 1 --2- 172.21.15.162:0/195007323 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9c9c060d10 0x7f9c9c0b8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:30.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.955+0000 7f9ca1278700 1 -- 172.21.15.162:0/195007323 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c9c0b7d60 con 0x7f9c9c0b4390 2020-11-28T21:51:30.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.955+0000 7f9ca1a79700 1 --2- 172.21.15.162:0/195007323 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9c9c060d10 0x7f9c9c0b8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:51:30.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.955+0000 7f9ca1278700 1 --2- 172.21.15.162:0/195007323 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c9c0b4390 0x7f9c9c149a80 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7f9c9800a000 tx=0x7f9c98008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:30.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.955+0000 7f9c8effd700 1 -- 172.21.15.162:0/195007323 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c98009980 con 0x7f9c9c0b4390 2020-11-28T21:51:30.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.955+0000 7f9c8effd700 1 -- 172.21.15.162:0/195007323 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c98009ae0 con 0x7f9c9c0b4390 2020-11-28T21:51:30.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.955+0000 7f9ca3cdd700 1 -- 172.21.15.162:0/195007323 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c9c143b10 con 0x7f9c9c0b4390 2020-11-28T21:51:30.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.955+0000 7f9c8effd700 1 -- 172.21.15.162:0/195007323 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c98009e40 con 0x7f9c9c0b4390 2020-11-28T21:51:30.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.955+0000 7f9ca3cdd700 1 -- 172.21.15.162:0/195007323 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c9c13dd50 con 0x7f9c9c0b4390 2020-11-28T21:51:30.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.959+0000 7f9c8effd700 1 -- 172.21.15.162:0/195007323 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9c98013480 con 0x7f9c9c0b4390 2020-11-28T21:51:30.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.959+0000 7f9ca3cdd700 1 -- 172.21.15.162:0/195007323 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c9c0b7ef0 con 0x7f9c9c0b4390 2020-11-28T21:51:30.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.959+0000 7f9c8effd700 1 --2- 172.21.15.162:0/195007323 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9c7c055440 0x7f9c7c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:30.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.959+0000 7f9ca1a79700 1 --2- 172.21.15.162:0/195007323 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9c7c055440 0x7f9c7c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:30.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.959+0000 7f9c8effd700 1 -- 172.21.15.162:0/195007323 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9c9806e6d0 con 0x7f9c9c0b4390 2020-11-28T21:51:30.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.959+0000 7f9ca1a79700 1 --2- 172.21.15.162:0/195007323 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9c7c055440 0x7f9c7c057900 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f9c90008000 tx=0x7f9c90009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:30.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:30.967+0000 7f9c8effd700 1 -- 172.21.15.162:0/195007323 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9c98044970 con 0x7f9c9c0b4390 2020-11-28T21:51:31.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.103+0000 7f9ca3cdd700 1 -- 172.21.15.162:0/195007323 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f9c9c057050 con 0x7f9c9c0b4390 2020-11-28T21:51:31.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.115+0000 7f9c8effd700 1 -- 172.21.15.162:0/195007323 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v7) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9c98047f90 con 0x7f9c9c0b4390 2020-11-28T21:51:31.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.115+0000 7f9ca3cdd700 1 -- 172.21.15.162:0/195007323 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9c7c055440 msgr2=0x7f9c7c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:31.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.115+0000 7f9ca3cdd700 1 --2- 172.21.15.162:0/195007323 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9c7c055440 0x7f9c7c057900 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f9c90008000 tx=0x7f9c90009040).stop 2020-11-28T21:51:31.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.115+0000 7f9ca3cdd700 1 -- 172.21.15.162:0/195007323 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c9c0b4390 msgr2=0x7f9c9c149a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:31.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.115+0000 7f9ca3cdd700 1 --2- 172.21.15.162:0/195007323 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c9c0b4390 0x7f9c9c149a80 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7f9c9800a000 tx=0x7f9c98008040).stop 2020-11-28T21:51:31.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.115+0000 7f9ca3cdd700 1 -- 172.21.15.162:0/195007323 shutdown_connections 2020-11-28T21:51:31.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.115+0000 7f9ca3cdd700 1 --2- 172.21.15.162:0/195007323 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9c7c055440 0x7f9c7c057900 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:31.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.115+0000 7f9ca3cdd700 1 --2- 172.21.15.162:0/195007323 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9c9c0b60e0 0x7f9c9c151c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:31.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.115+0000 7f9ca3cdd700 1 --2- 172.21.15.162:0/195007323 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9c9c060d10 0x7f9c9c0b8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:31.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.115+0000 7f9ca3cdd700 1 --2- 172.21.15.162:0/195007323 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c9c0b4390 0x7f9c9c149a80 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:31.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.115+0000 7f9ca3cdd700 1 -- 172.21.15.162:0/195007323 >> 172.21.15.162:0/195007323 conn(0x7f9c9c05fe70 msgr2=0x7f9c9c0af9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:31.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.115+0000 7f9ca3cdd700 1 -- 172.21.15.162:0/195007323 shutdown_connections 2020-11-28T21:51:31.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.115+0000 7f9ca3cdd700 1 -- 172.21.15.162:0/195007323 wait complete. 2020-11-28T21:51:31.128 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout' 2020-11-28T21:51:31.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.247+0000 7fd0b759e700 1 Processor -- start 2020-11-28T21:51:31.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.247+0000 7fd0b759e700 1 -- start start 2020-11-28T21:51:31.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.247+0000 7fd0b759e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd0b805cbd0 0x7fd0b80aac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:31.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.247+0000 7fd0b759e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd0b80af640 0x7fd0b80ae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:31.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.247+0000 7fd0b759e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0b80b0230 con 0x7fd0b8060cf0 2020-11-28T21:51:31.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.247+0000 7fd0b759e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0b80b12a0 con 0x7fd0b805cbd0 2020-11-28T21:51:31.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.247+0000 7fd0b759e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b80610d0 con 0x7fd0b80af640 2020-11-28T21:51:31.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.247+0000 7fd0b5d9b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd0b805cbd0 0x7fd0b80aac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:31.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.247+0000 7fd0b5d9b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd0b805cbd0 0x7fd0b80aac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40360/0 (socket says 172.21.15.162:40360) 2020-11-28T21:51:31.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.247+0000 7fd0b5d9b700 1 -- 172.21.15.162:0/2355639033 learned_addr learned my addr 172.21.15.162:0/2355639033 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:31.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.247+0000 7fd0b6d9d700 1 --2- 172.21.15.162:0/2355639033 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd0b80af640 0x7fd0b80ae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:31.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.251+0000 7fd0b5d9b700 1 -- 172.21.15.162:0/2355639033 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd0b80af640 msgr2=0x7fd0b80ae1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:51:31.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.251+0000 7fd0b5d9b700 1 --2- 172.21.15.162:0/2355639033 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd0b80af640 0x7fd0b80ae1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:31.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.251+0000 7fd0b5d9b700 1 -- 172.21.15.162:0/2355639033 >> v1:172.21.15.118:6789/0 conn(0x7fd0b8060cf0 legacy=0x7fd0b80aa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:51:31.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.251+0000 7fd0b5d9b700 1 -- 172.21.15.162:0/2355639033 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b80ab0f0 con 0x7fd0b805cbd0 2020-11-28T21:51:31.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.251+0000 7fd0b5d9b700 1 --2- 172.21.15.162:0/2355639033 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd0b805cbd0 0x7fd0b80aac00 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7fd0b8067980 tx=0x7fd0a8006e80).ready entity=mon.1 client_cookie=6eb64d8fca64d56f server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:31.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.251+0000 7fd0b559a700 1 -- 172.21.15.162:0/2355639033 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0a80097b0 con 0x7fd0b805cbd0 2020-11-28T21:51:31.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.251+0000 7fd0b559a700 1 -- 172.21.15.162:0/2355639033 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a8009910 con 0x7fd0b805cbd0 2020-11-28T21:51:31.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.251+0000 7fd0b559a700 1 -- 172.21.15.162:0/2355639033 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0a8009ac0 con 0x7fd0b805cbd0 2020-11-28T21:51:31.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.251+0000 7fd0b759e700 1 -- 172.21.15.162:0/2355639033 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd0b805cbd0 msgr2=0x7fd0b80aac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:31.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.251+0000 7fd0b759e700 1 --2- 172.21.15.162:0/2355639033 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd0b805cbd0 0x7fd0b80aac00 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7fd0b8067980 tx=0x7fd0a8006e80).stop 2020-11-28T21:51:31.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.251+0000 7fd0b759e700 1 -- 172.21.15.162:0/2355639033 shutdown_connections 2020-11-28T21:51:31.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.251+0000 7fd0b759e700 1 --2- 172.21.15.162:0/2355639033 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd0b80af640 0x7fd0b80ae1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:31.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.251+0000 7fd0b759e700 1 --2- 172.21.15.162:0/2355639033 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd0b805cbd0 0x7fd0b80aac00 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:31.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.251+0000 7fd0b759e700 1 -- 172.21.15.162:0/2355639033 >> 172.21.15.162:0/2355639033 conn(0x7fd0b805b970 msgr2=0x7fd0b805bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:31.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.251+0000 7fd0b759e700 1 -- 172.21.15.162:0/2355639033 shutdown_connections 2020-11-28T21:51:31.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.251+0000 7fd0b759e700 1 -- 172.21.15.162:0/2355639033 wait complete. 2020-11-28T21:51:31.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.251+0000 7fd0b759e700 1 Processor -- start 2020-11-28T21:51:31.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.251+0000 7fd0b759e700 1 -- start start 2020-11-28T21:51:31.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.251+0000 7fd0b759e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd0b805cbd0 0x7fd0b8066440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:31.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.251+0000 7fd0b759e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd0b8060cf0 0x7fd0b8148370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:31.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.251+0000 7fd0b759e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd0b80af640 0x7fd0b814b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:31.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.251+0000 7fd0b759e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b81478c0 con 0x7fd0b805cbd0 2020-11-28T21:51:31.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.251+0000 7fd0b759e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b8147af0 con 0x7fd0b80af640 2020-11-28T21:51:31.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.251+0000 7fd0b759e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b8147d90 con 0x7fd0b8060cf0 2020-11-28T21:51:31.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.251+0000 7fd0b6d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd0b80af640 0x7fd0b814b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:31.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.251+0000 7fd0b6d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd0b80af640 0x7fd0b814b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40368/0 (socket says 172.21.15.162:40368) 2020-11-28T21:51:31.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.251+0000 7fd0b6d9d700 1 -- 172.21.15.162:0/940378029 learned_addr learned my addr 172.21.15.162:0/940378029 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:31.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.251+0000 7fd0b659c700 1 --2- 172.21.15.162:0/940378029 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd0b805cbd0 0x7fd0b8066440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:31.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.251+0000 7fd0b5d9b700 1 --2- 172.21.15.162:0/940378029 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd0b8060cf0 0x7fd0b8148370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:31.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.255+0000 7fd0b659c700 1 -- 172.21.15.162:0/940378029 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd0b8060cf0 msgr2=0x7fd0b8148370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:31.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.255+0000 7fd0b659c700 1 --2- 172.21.15.162:0/940378029 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd0b8060cf0 0x7fd0b8148370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:31.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.255+0000 7fd0b659c700 1 -- 172.21.15.162:0/940378029 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd0b80af640 msgr2=0x7fd0b814b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:31.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.255+0000 7fd0b659c700 1 --2- 172.21.15.162:0/940378029 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd0b80af640 0x7fd0b814b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:31.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.255+0000 7fd0b659c700 1 -- 172.21.15.162:0/940378029 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b80a8c00 con 0x7fd0b805cbd0 2020-11-28T21:51:31.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.255+0000 7fd0b6d9d700 1 --2- 172.21.15.162:0/940378029 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd0b80af640 0x7fd0b814b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:51:31.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.255+0000 7fd0b659c700 1 --2- 172.21.15.162:0/940378029 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd0b805cbd0 0x7fd0b8066440 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7fd0a0002ef0 tx=0x7fd0a0006c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:31.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.255+0000 7fd0af7fe700 1 -- 172.21.15.162:0/940378029 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0a0002320 con 0x7fd0b805cbd0 2020-11-28T21:51:31.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.255+0000 7fd0b759e700 1 -- 172.21.15.162:0/940378029 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0b8148860 con 0x7fd0b805cbd0 2020-11-28T21:51:31.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.255+0000 7fd0af7fe700 1 -- 172.21.15.162:0/940378029 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a0000e90 con 0x7fd0b805cbd0 2020-11-28T21:51:31.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.255+0000 7fd0af7fe700 1 -- 172.21.15.162:0/940378029 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0a0002670 con 0x7fd0b805cbd0 2020-11-28T21:51:31.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.255+0000 7fd0b759e700 1 -- 172.21.15.162:0/940378029 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0b814bb20 con 0x7fd0b805cbd0 2020-11-28T21:51:31.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.255+0000 7fd0b759e700 1 -- 172.21.15.162:0/940378029 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0b8147f50 con 0x7fd0b805cbd0 2020-11-28T21:51:31.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.255+0000 7fd0af7fe700 1 -- 172.21.15.162:0/940378029 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd0a000f030 con 0x7fd0b805cbd0 2020-11-28T21:51:31.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.255+0000 7fd0af7fe700 1 --2- 172.21.15.162:0/940378029 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd09c055330 0x7fd09c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:31.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.259+0000 7fd0af7fe700 1 -- 172.21.15.162:0/940378029 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd0a006dbc0 con 0x7fd0b805cbd0 2020-11-28T21:51:31.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.259+0000 7fd0b5d9b700 1 --2- 172.21.15.162:0/940378029 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd09c055330 0x7fd09c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:31.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.259+0000 7fd0b5d9b700 1 --2- 172.21.15.162:0/940378029 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd09c055330 0x7fd09c0577f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7fd0a801b000 tx=0x7fd0a8016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:31.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.267+0000 7fd0af7fe700 1 -- 172.21.15.162:0/940378029 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd0a0043ec0 con 0x7fd0b805cbd0 2020-11-28T21:51:31.455 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.459+0000 7fd0b759e700 1 -- 172.21.15.162:0/940378029 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout"]} v 0) v1 -- 0x7fd0b8057050 con 0x7fd0b805cbd0 2020-11-28T21:51:31.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.791+0000 7fd0af7fe700 1 -- 172.21.15.162:0/940378029 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout"]}]=0 v212) v1 ==== 152+0+0 (secure 0 0 0) 0x7fd0a0033020 con 0x7fd0b805cbd0 2020-11-28T21:51:31.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.791+0000 7fd0b759e700 1 -- 172.21.15.162:0/940378029 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd09c055330 msgr2=0x7fd09c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:31.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.791+0000 7fd0b759e700 1 --2- 172.21.15.162:0/940378029 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd09c055330 0x7fd09c0577f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7fd0a801b000 tx=0x7fd0a8016000).stop 2020-11-28T21:51:31.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.791+0000 7fd0b759e700 1 -- 172.21.15.162:0/940378029 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd0b805cbd0 msgr2=0x7fd0b8066440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:31.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.791+0000 7fd0b759e700 1 --2- 172.21.15.162:0/940378029 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd0b805cbd0 0x7fd0b8066440 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7fd0a0002ef0 tx=0x7fd0a0006c80).stop 2020-11-28T21:51:31.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.791+0000 7fd0b759e700 1 -- 172.21.15.162:0/940378029 shutdown_connections 2020-11-28T21:51:31.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.791+0000 7fd0b759e700 1 --2- 172.21.15.162:0/940378029 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd09c055330 0x7fd09c0577f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:31.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.791+0000 7fd0b759e700 1 --2- 172.21.15.162:0/940378029 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd0b80af640 0x7fd0b814b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:31.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.791+0000 7fd0b759e700 1 --2- 172.21.15.162:0/940378029 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd0b805cbd0 0x7fd0b8066440 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:31.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.791+0000 7fd0b759e700 1 --2- 172.21.15.162:0/940378029 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd0b8060cf0 0x7fd0b8148370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:31.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.791+0000 7fd0b759e700 1 -- 172.21.15.162:0/940378029 >> 172.21.15.162:0/940378029 conn(0x7fd0b805b970 msgr2=0x7fd0b805f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:31.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.795+0000 7fd0b759e700 1 -- 172.21.15.162:0/940378029 shutdown_connections 2020-11-28T21:51:31.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.795+0000 7fd0b759e700 1 -- 172.21.15.162:0/940378029 wait complete. 2020-11-28T21:51:31.808 INFO:tasks.cephfs_test_runner:test_drop_cache_command_dead_timeout (tasks.cephfs.test_misc.TestCacheDrop) ... ok 2020-11-28T21:51:31.809 INFO:tasks.cephfs_test_runner:Starting test: test_drop_cache_command_timeout (tasks.cephfs.test_misc.TestCacheDrop) 2020-11-28T21:51:31.809 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout' 2020-11-28T21:51:31.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.931+0000 7f2304e21700 1 Processor -- start 2020-11-28T21:51:31.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.931+0000 7f2304e21700 1 -- start start 2020-11-28T21:51:31.933 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.935+0000 7f2304e21700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23000b4bd0 0x7f2300098660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:31.933 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.935+0000 7f2304e21700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f23000b33f0 0x7f230009ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:31.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.935+0000 7f2304e21700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f230009b250 0x7f2300142ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:31.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.935+0000 7f2304e21700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2300098b50 con 0x7f230009b250 2020-11-28T21:51:31.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.935+0000 7f2304e21700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f230009b6e0 con 0x7f23000b33f0 2020-11-28T21:51:31.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.935+0000 7f2304e21700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23000b4fb0 con 0x7f23000b4bd0 2020-11-28T21:51:31.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.935+0000 7f22feffd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f23000b33f0 0x7f230009ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:31.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.935+0000 7f22feffd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f23000b33f0 0x7f230009ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40374/0 (socket says 172.21.15.162:40374) 2020-11-28T21:51:31.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.935+0000 7f22feffd700 1 -- 172.21.15.162:0/275476109 learned_addr learned my addr 172.21.15.162:0/275476109 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:31.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.935+0000 7f22fffff700 1 --2- 172.21.15.162:0/275476109 >> v2:172.21.15.118:3300/0 conn(0x7f230009b250 0x7f2300142ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:31.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.935+0000 7f22ff7fe700 1 --2- 172.21.15.162:0/275476109 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23000b4bd0 0x7f2300098660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:31.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.935+0000 7f22feffd700 1 -- 172.21.15.162:0/275476109 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23000b4bd0 msgr2=0x7f2300098660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:51:31.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.935+0000 7f22feffd700 1 --2- 172.21.15.162:0/275476109 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23000b4bd0 0x7f2300098660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:31.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.935+0000 7f22feffd700 1 -- 172.21.15.162:0/275476109 >> v2:172.21.15.118:3300/0 conn(0x7f230009b250 msgr2=0x7f2300142ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:51:31.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.935+0000 7f22feffd700 1 --2- 172.21.15.162:0/275476109 >> v2:172.21.15.118:3300/0 conn(0x7f230009b250 0x7f2300142ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:31.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.935+0000 7f22feffd700 1 -- 172.21.15.162:0/275476109 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2300142a60 con 0x7f23000b33f0 2020-11-28T21:51:31.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.935+0000 7f22fffff700 1 --2- 172.21.15.162:0/275476109 >> v2:172.21.15.118:3300/0 conn(0x7f230009b250 0x7f2300142ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:51:31.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.935+0000 7f22feffd700 1 --2- 172.21.15.162:0/275476109 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23000b33f0 0x7f230009ad60 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f22f4002f30 tx=0x7f22f4006e80).ready entity=mon.1 client_cookie=4bba77c44f313eee server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:31.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.935+0000 7f22fe7fc700 1 -- 172.21.15.162:0/275476109 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f22f40097f0 con 0x7f23000b33f0 2020-11-28T21:51:31.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.935+0000 7f22fe7fc700 1 -- 172.21.15.162:0/275476109 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f4009950 con 0x7f23000b33f0 2020-11-28T21:51:31.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.935+0000 7f22fe7fc700 1 -- 172.21.15.162:0/275476109 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f22f4009cb0 con 0x7f23000b33f0 2020-11-28T21:51:31.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.935+0000 7f2304e21700 1 -- 172.21.15.162:0/275476109 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23000b33f0 msgr2=0x7f230009ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:31.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.935+0000 7f2304e21700 1 --2- 172.21.15.162:0/275476109 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23000b33f0 0x7f230009ad60 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f22f4002f30 tx=0x7f22f4006e80).stop 2020-11-28T21:51:31.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.935+0000 7f2304e21700 1 -- 172.21.15.162:0/275476109 shutdown_connections 2020-11-28T21:51:31.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.935+0000 7f2304e21700 1 --2- 172.21.15.162:0/275476109 >> v2:172.21.15.118:3300/0 conn(0x7f230009b250 0x7f2300142ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:31.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.935+0000 7f2304e21700 1 --2- 172.21.15.162:0/275476109 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23000b4bd0 0x7f2300098660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:31.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.935+0000 7f2304e21700 1 --2- 172.21.15.162:0/275476109 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23000b33f0 0x7f230009ad60 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:31.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.935+0000 7f2304e21700 1 -- 172.21.15.162:0/275476109 >> 172.21.15.162:0/275476109 conn(0x7f23000b54f0 msgr2=0x7f23000b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:31.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.935+0000 7f2304e21700 1 -- 172.21.15.162:0/275476109 shutdown_connections 2020-11-28T21:51:31.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.935+0000 7f2304e21700 1 -- 172.21.15.162:0/275476109 wait complete. 2020-11-28T21:51:31.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.939+0000 7f2304e21700 1 Processor -- start 2020-11-28T21:51:31.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.939+0000 7f2304e21700 1 -- start start 2020-11-28T21:51:31.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.939+0000 7f2304e21700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f230009b250 0x7f230014ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:31.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.939+0000 7f2304e21700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23000b33f0 0x7f230013cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:31.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.939+0000 7f2304e21700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23000b4bd0 0x7f2300146240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:31.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.939+0000 7f2304e21700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2300143520 con 0x7f230009b250 2020-11-28T21:51:31.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.939+0000 7f2304e21700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230014f330 con 0x7f23000b4bd0 2020-11-28T21:51:31.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.939+0000 7f2304e21700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2300143750 con 0x7f23000b33f0 2020-11-28T21:51:31.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.939+0000 7f22fffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23000b4bd0 0x7f2300146240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:31.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.939+0000 7f22fffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23000b4bd0 0x7f2300146240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40382/0 (socket says 172.21.15.162:40382) 2020-11-28T21:51:31.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.939+0000 7f22fffff700 1 -- 172.21.15.162:0/1784909227 learned_addr learned my addr 172.21.15.162:0/1784909227 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:31.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.939+0000 7f22ff7fe700 1 --2- 172.21.15.162:0/1784909227 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f230009b250 0x7f230014ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:31.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.939+0000 7f22feffd700 1 --2- 172.21.15.162:0/1784909227 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23000b33f0 0x7f230013cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:31.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.939+0000 7f22fffff700 1 -- 172.21.15.162:0/1784909227 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23000b33f0 msgr2=0x7f230013cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:31.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.939+0000 7f22fffff700 1 --2- 172.21.15.162:0/1784909227 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23000b33f0 0x7f230013cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:31.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.939+0000 7f22fffff700 1 -- 172.21.15.162:0/1784909227 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f230009b250 msgr2=0x7f230014ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:31.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.939+0000 7f22fffff700 1 --2- 172.21.15.162:0/1784909227 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f230009b250 0x7f230014ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:31.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.939+0000 7f22fffff700 1 -- 172.21.15.162:0/1784909227 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23001439f0 con 0x7f23000b4bd0 2020-11-28T21:51:31.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.939+0000 7f22feffd700 1 --2- 172.21.15.162:0/1784909227 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23000b33f0 0x7f230013cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:51:31.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.939+0000 7f22fffff700 1 --2- 172.21.15.162:0/1784909227 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23000b4bd0 0x7f2300146240 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7f2300143470 tx=0x7f22f8008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:31.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.939+0000 7f22fcff9700 1 -- 172.21.15.162:0/1784909227 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f22f8009980 con 0x7f23000b4bd0 2020-11-28T21:51:31.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.939+0000 7f22fcff9700 1 -- 172.21.15.162:0/1784909227 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f8009ae0 con 0x7f23000b4bd0 2020-11-28T21:51:31.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.939+0000 7f2304e21700 1 -- 172.21.15.162:0/1784909227 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2300143c90 con 0x7f23000b4bd0 2020-11-28T21:51:31.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.939+0000 7f22fcff9700 1 -- 172.21.15.162:0/1784909227 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f22f8009e40 con 0x7f23000b4bd0 2020-11-28T21:51:31.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.939+0000 7f2304e21700 1 -- 172.21.15.162:0/1784909227 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f230013e030 con 0x7f23000b4bd0 2020-11-28T21:51:31.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.943+0000 7f22fcff9700 1 -- 172.21.15.162:0/1784909227 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f22f8021060 con 0x7f23000b4bd0 2020-11-28T21:51:31.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.943+0000 7f2304e21700 1 -- 172.21.15.162:0/1784909227 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2300057050 con 0x7f23000b4bd0 2020-11-28T21:51:31.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.943+0000 7f22fcff9700 1 --2- 172.21.15.162:0/1784909227 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f22e0055330 0x7f22e00577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:31.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.943+0000 7f22fcff9700 1 -- 172.21.15.162:0/1784909227 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f22f80701f0 con 0x7f23000b4bd0 2020-11-28T21:51:31.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.943+0000 7f22ff7fe700 1 --2- 172.21.15.162:0/1784909227 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f22e0055330 0x7f22e00577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:31.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.943+0000 7f22ff7fe700 1 --2- 172.21.15.162:0/1784909227 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f22e0055330 0x7f22e00577f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f22f000b000 tx=0x7f22f0008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:31.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:31.947+0000 7f22fcff9700 1 -- 172.21.15.162:0/1784909227 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f22f80464f0 con 0x7f23000b4bd0 2020-11-28T21:51:32.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.151+0000 7f2304e21700 1 -- 172.21.15.162:0/1784909227 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout"]} v 0) v1 -- 0x7f230005d5e0 con 0x7f23000b4bd0 2020-11-28T21:51:32.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.795+0000 7f22fcff9700 1 -- 172.21.15.162:0/1784909227 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout"]}]=0 v213) v1 ==== 150+0+0 (secure 0 0 0) 0x7f22f8049b10 con 0x7f23000b4bd0 2020-11-28T21:51:32.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.795+0000 7f2304e21700 1 -- 172.21.15.162:0/1784909227 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f22e0055330 msgr2=0x7f22e00577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:32.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.795+0000 7f2304e21700 1 --2- 172.21.15.162:0/1784909227 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f22e0055330 0x7f22e00577f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f22f000b000 tx=0x7f22f0008040).stop 2020-11-28T21:51:32.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.795+0000 7f2304e21700 1 -- 172.21.15.162:0/1784909227 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23000b4bd0 msgr2=0x7f2300146240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:32.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.795+0000 7f2304e21700 1 --2- 172.21.15.162:0/1784909227 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23000b4bd0 0x7f2300146240 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7f2300143470 tx=0x7f22f8008040).stop 2020-11-28T21:51:32.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.795+0000 7f2304e21700 1 -- 172.21.15.162:0/1784909227 shutdown_connections 2020-11-28T21:51:32.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.795+0000 7f2304e21700 1 --2- 172.21.15.162:0/1784909227 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f22e0055330 0x7f22e00577f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:32.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.795+0000 7f2304e21700 1 --2- 172.21.15.162:0/1784909227 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23000b4bd0 0x7f2300146240 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:32.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.795+0000 7f2304e21700 1 --2- 172.21.15.162:0/1784909227 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f230009b250 0x7f230014ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:32.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.795+0000 7f2304e21700 1 --2- 172.21.15.162:0/1784909227 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23000b33f0 0x7f230013cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:32.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.795+0000 7f2304e21700 1 -- 172.21.15.162:0/1784909227 >> 172.21.15.162:0/1784909227 conn(0x7f23000b54f0 msgr2=0x7f230013be20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:32.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.795+0000 7f2304e21700 1 -- 172.21.15.162:0/1784909227 shutdown_connections 2020-11-28T21:51:32.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.795+0000 7f2304e21700 1 -- 172.21.15.162:0/1784909227 wait complete. 2020-11-28T21:51:32.812 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config set mon mon_allow_pool_delete True 2020-11-28T21:51:32.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.939+0000 7f24bbbb5700 1 Processor -- start 2020-11-28T21:51:32.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.939+0000 7f24bbbb5700 1 -- start start 2020-11-28T21:51:32.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.939+0000 7f24bbbb5700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f24b40b9ce0 0x7f24b41433c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:32.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.939+0000 7f24bbbb5700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f24b4067610 0x7f24b413fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:32.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.939+0000 7f24bbbb5700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24b4143950 con 0x7f24b40605c0 2020-11-28T21:51:32.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.939+0000 7f24bbbb5700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24b40b4340 con 0x7f24b40b9ce0 2020-11-28T21:51:32.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.939+0000 7f24bbbb5700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24b40ba0c0 con 0x7f24b4067610 2020-11-28T21:51:32.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.939+0000 7f24ba152700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f24b4067610 0x7f24b413fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:32.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.939+0000 7f24b9150700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f24b40605c0 0x7f24b40b3c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:47230/0 (socket says 172.21.15.162:47230) 2020-11-28T21:51:32.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.939+0000 7f24ba152700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f24b4067610 0x7f24b413fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40390/0 (socket says 172.21.15.162:40390) 2020-11-28T21:51:32.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.939+0000 7f24b9150700 1 -- 172.21.15.162:0/2280290122 learned_addr learned my addr 172.21.15.162:0/2280290122 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:51:32.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.943+0000 7f24b9951700 1 --2- 172.21.15.162:0/2280290122 >> v2:172.21.15.118:3300/0 conn(0x7f24b40b9ce0 0x7f24b41433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:32.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.943+0000 7f24ba152700 1 -- 172.21.15.162:0/2280290122 >> v2:172.21.15.118:3300/0 conn(0x7f24b40b9ce0 msgr2=0x7f24b41433c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:51:32.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.943+0000 7f24ba152700 1 --2- 172.21.15.162:0/2280290122 >> v2:172.21.15.118:3300/0 conn(0x7f24b40b9ce0 0x7f24b41433c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:32.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.943+0000 7f24ba152700 1 -- 172.21.15.162:0/2280290122 >> v1:172.21.15.162:6789/0 conn(0x7f24b40605c0 legacy=0x7f24b40b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:32.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.943+0000 7f24ba152700 1 -- 172.21.15.162:0/2280290122 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b4067aa0 con 0x7f24b4067610 2020-11-28T21:51:32.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.943+0000 7f24b9951700 1 --2- 172.21.15.162:0/2280290122 >> v2:172.21.15.118:3300/0 conn(0x7f24b40b9ce0 0x7f24b41433c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:51:32.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.943+0000 7f24ba152700 1 --2- 172.21.15.162:0/2280290122 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24b4067610 0x7f24b413fc80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7f24ac0032b0 tx=0x7f24ac0091a0).ready entity=mon.1 client_cookie=f0c11a666857de38 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:32.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.943+0000 7f24b894f700 1 -- 172.21.15.162:0/2280290122 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24ac00ba50 con 0x7f24b4067610 2020-11-28T21:51:32.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.943+0000 7f24b894f700 1 -- 172.21.15.162:0/2280290122 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24ac00bbb0 con 0x7f24b4067610 2020-11-28T21:51:32.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.943+0000 7f24b894f700 1 -- 172.21.15.162:0/2280290122 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24ac00bd20 con 0x7f24b4067610 2020-11-28T21:51:32.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.943+0000 7f24bbbb5700 1 -- 172.21.15.162:0/2280290122 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24b4067610 msgr2=0x7f24b413fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:32.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.943+0000 7f24bbbb5700 1 --2- 172.21.15.162:0/2280290122 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24b4067610 0x7f24b413fc80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7f24ac0032b0 tx=0x7f24ac0091a0).stop 2020-11-28T21:51:32.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.943+0000 7f24bbbb5700 1 -- 172.21.15.162:0/2280290122 shutdown_connections 2020-11-28T21:51:32.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.943+0000 7f24bbbb5700 1 --2- 172.21.15.162:0/2280290122 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24b4067610 0x7f24b413fc80 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:32.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.943+0000 7f24bbbb5700 1 --2- 172.21.15.162:0/2280290122 >> v2:172.21.15.118:3300/0 conn(0x7f24b40b9ce0 0x7f24b41433c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:32.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.943+0000 7f24bbbb5700 1 -- 172.21.15.162:0/2280290122 >> 172.21.15.162:0/2280290122 conn(0x7f24b405c430 msgr2=0x7f24b40b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:32.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.943+0000 7f24bbbb5700 1 -- 172.21.15.162:0/2280290122 shutdown_connections 2020-11-28T21:51:32.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.943+0000 7f24bbbb5700 1 -- 172.21.15.162:0/2280290122 wait complete. 2020-11-28T21:51:32.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.943+0000 7f24bbbb5700 1 Processor -- start 2020-11-28T21:51:32.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.943+0000 7f24bbbb5700 1 -- start start 2020-11-28T21:51:32.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.947+0000 7f24bbbb5700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24b40605c0 0x7f24b409e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:32.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.947+0000 7f24bbbb5700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f24b4067610 0x7f24b4152390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:32.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.947+0000 7f24bbbb5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24b40b9ce0 0x7f24b40b0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:32.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.947+0000 7f24bbbb5700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b4152880 con 0x7f24b40605c0 2020-11-28T21:51:32.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.947+0000 7f24bbbb5700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b409eb40 con 0x7f24b40b9ce0 2020-11-28T21:51:32.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.947+0000 7f24bbbb5700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24b409ede0 con 0x7f24b4067610 2020-11-28T21:51:32.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.947+0000 7f24ba152700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24b40b9ce0 0x7f24b40b0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:32.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.947+0000 7f24ba152700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24b40b9ce0 0x7f24b40b0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40392/0 (socket says 172.21.15.162:40392) 2020-11-28T21:51:32.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.947+0000 7f24ba152700 1 -- 172.21.15.162:0/1086712419 learned_addr learned my addr 172.21.15.162:0/1086712419 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:32.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.947+0000 7f24b9951700 1 --2- 172.21.15.162:0/1086712419 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24b40605c0 0x7f24b409e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:32.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.947+0000 7f24b9150700 1 --2- 172.21.15.162:0/1086712419 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f24b4067610 0x7f24b4152390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:32.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.947+0000 7f24ba152700 1 -- 172.21.15.162:0/1086712419 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f24b4067610 msgr2=0x7f24b4152390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:32.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.947+0000 7f24ba152700 1 --2- 172.21.15.162:0/1086712419 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f24b4067610 0x7f24b4152390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:32.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.947+0000 7f24ba152700 1 -- 172.21.15.162:0/1086712419 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24b40605c0 msgr2=0x7f24b409e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:32.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.947+0000 7f24ba152700 1 --2- 172.21.15.162:0/1086712419 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24b40605c0 0x7f24b409e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:32.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.947+0000 7f24ba152700 1 -- 172.21.15.162:0/1086712419 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b40b06a0 con 0x7f24b40b9ce0 2020-11-28T21:51:32.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.947+0000 7f24ba152700 1 --2- 172.21.15.162:0/1086712419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24b40b9ce0 0x7f24b40b0de0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7f24ac0032b0 tx=0x7f24ac016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:32.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.947+0000 7f24a2ffd700 1 -- 172.21.15.162:0/1086712419 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24ac010660 con 0x7f24b40b9ce0 2020-11-28T21:51:32.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.947+0000 7f24a2ffd700 1 -- 172.21.15.162:0/1086712419 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24ac0107c0 con 0x7f24b40b9ce0 2020-11-28T21:51:32.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.947+0000 7f24bbbb5700 1 -- 172.21.15.162:0/1086712419 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24b40b0940 con 0x7f24b40b9ce0 2020-11-28T21:51:32.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.947+0000 7f24a2ffd700 1 -- 172.21.15.162:0/1086712419 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24ac010b20 con 0x7f24b40b9ce0 2020-11-28T21:51:32.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.947+0000 7f24bbbb5700 1 -- 172.21.15.162:0/1086712419 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24b409e830 con 0x7f24b40b9ce0 2020-11-28T21:51:32.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.947+0000 7f24bbbb5700 1 -- 172.21.15.162:0/1086712419 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24b40b1610 con 0x7f24b40b9ce0 2020-11-28T21:51:32.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.951+0000 7f24a2ffd700 1 -- 172.21.15.162:0/1086712419 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f24ac026030 con 0x7f24b40b9ce0 2020-11-28T21:51:32.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.951+0000 7f24a2ffd700 1 --2- 172.21.15.162:0/1086712419 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f24940554e0 0x7f24940579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:32.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.951+0000 7f24b9951700 1 --2- 172.21.15.162:0/1086712419 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f24940554e0 0x7f24940579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:32.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.951+0000 7f24a2ffd700 1 -- 172.21.15.162:0/1086712419 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f24ac06efd0 con 0x7f24b40b9ce0 2020-11-28T21:51:32.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.951+0000 7f24b9951700 1 --2- 172.21.15.162:0/1086712419 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f24940554e0 0x7f24940579a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f24b000b000 tx=0x7f24b0008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:32.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:32.955+0000 7f24a2ffd700 1 -- 172.21.15.162:0/1086712419 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f24ac0452d0 con 0x7f24b40b9ce0 2020-11-28T21:51:33.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.091+0000 7f24bbbb5700 1 -- 172.21.15.162:0/1086712419 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7f24b40b0ad0 con 0x7f24b40b9ce0 2020-11-28T21:51:33.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.099+0000 7f24a2ffd700 1 -- 172.21.15.162:0/1086712419 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f24ac0240c0 con 0x7f24b40b9ce0 2020-11-28T21:51:33.098 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.103+0000 7f24bbbb5700 1 -- 172.21.15.162:0/1086712419 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f24940554e0 msgr2=0x7f24940579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:33.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.103+0000 7f24bbbb5700 1 --2- 172.21.15.162:0/1086712419 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f24940554e0 0x7f24940579a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f24b000b000 tx=0x7f24b0008040).stop 2020-11-28T21:51:33.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.103+0000 7f24bbbb5700 1 -- 172.21.15.162:0/1086712419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24b40b9ce0 msgr2=0x7f24b40b0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:33.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.103+0000 7f24bbbb5700 1 --2- 172.21.15.162:0/1086712419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24b40b9ce0 0x7f24b40b0de0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7f24ac0032b0 tx=0x7f24ac016000).stop 2020-11-28T21:51:33.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.103+0000 7f24bbbb5700 1 -- 172.21.15.162:0/1086712419 shutdown_connections 2020-11-28T21:51:33.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.103+0000 7f24bbbb5700 1 --2- 172.21.15.162:0/1086712419 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f24940554e0 0x7f24940579a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:33.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.103+0000 7f24bbbb5700 1 --2- 172.21.15.162:0/1086712419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24b40b9ce0 0x7f24b40b0de0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:33.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.103+0000 7f24bbbb5700 1 --2- 172.21.15.162:0/1086712419 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24b40605c0 0x7f24b409e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:33.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.103+0000 7f24bbbb5700 1 --2- 172.21.15.162:0/1086712419 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f24b4067610 0x7f24b4152390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:33.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.103+0000 7f24bbbb5700 1 -- 172.21.15.162:0/1086712419 >> 172.21.15.162:0/1086712419 conn(0x7f24b405c430 msgr2=0x7f24b40b7170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:33.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.103+0000 7f24bbbb5700 1 -- 172.21.15.162:0/1086712419 shutdown_connections 2020-11-28T21:51:33.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.103+0000 7f24bbbb5700 1 -- 172.21.15.162:0/1086712419 wait complete. 2020-11-28T21:51:33.112 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-28T21:51:33.113 INFO:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:51:33.131 INFO:teuthology.orchestra.run.smithi118:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:51:33.149 INFO:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:51:33.163 INFO:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:51:33.181 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-28T21:51:33.182 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:51:33.182 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:51:33.183 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:51:33.230 INFO:teuthology.orchestra.run.smithi192.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2020-11-28T21:51:33.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:51:33.231 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2020-11-28T21:51:33.232 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:51:33.233 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:51:33.233 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:51:33.240 INFO:teuthology.orchestra.run.smithi096.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-11-28T21:51:33.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:51:33.241 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2020-11-28T21:51:33.242 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2020-11-28T21:51:33.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.367+0000 7f697693d700 1 Processor -- start 2020-11-28T21:51:33.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.367+0000 7f697693d700 1 -- start start 2020-11-28T21:51:33.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.367+0000 7f697693d700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f69700b11b0 0x7f6970140fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:33.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.367+0000 7f697693d700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69700b2870 con 0x7f69700b1d70 2020-11-28T21:51:33.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.367+0000 7f697693d700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69700b22e0 con 0x7f69700b1990 2020-11-28T21:51:33.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.367+0000 7f697693d700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6970146e20 con 0x7f69700b11b0 2020-11-28T21:51:33.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.367+0000 7f696ffff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f69700b1990 0x7f6970140890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:47244/0 (socket says 172.21.15.162:47244) 2020-11-28T21:51:33.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.367+0000 7f696ffff700 1 -- 172.21.15.162:0/3254188029 learned_addr learned my addr 172.21.15.162:0/3254188029 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:51:33.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.367+0000 7f696f7fe700 1 --2- 172.21.15.162:0/3254188029 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f69700b11b0 0x7f6970140fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:33.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.367+0000 7f696effd700 1 -- 172.21.15.162:0/3254188029 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2038716531 0 0) 0x7f69700b22e0 con 0x7f69700b1990 2020-11-28T21:51:33.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.367+0000 7f696effd700 1 -- 172.21.15.162:0/3254188029 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6950003230 con 0x7f69700b1990 2020-11-28T21:51:33.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.367+0000 7f696effd700 1 -- 172.21.15.162:0/3254188029 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3107615308 0 0) 0x7f6950003230 con 0x7f69700b1990 2020-11-28T21:51:33.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.367+0000 7f696f7fe700 1 -- 172.21.15.162:0/3254188029 >> v1:172.21.15.162:6789/0 conn(0x7f69700b1990 legacy=0x7f6970140890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:33.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.367+0000 7f696f7fe700 1 -- 172.21.15.162:0/3254188029 >> v1:172.21.15.118:6789/0 conn(0x7f69700b1d70 legacy=0x7f697014a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:33.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.367+0000 7f696f7fe700 1 -- 172.21.15.162:0/3254188029 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69700b0740 con 0x7f69700b11b0 2020-11-28T21:51:33.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.367+0000 7f696f7fe700 1 --2- 172.21.15.162:0/3254188029 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f69700b11b0 0x7f6970140fa0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f69580010d0 tx=0x7f69680091a0).ready entity=mon.2 client_cookie=43783671049cdb78 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:33.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.367+0000 7f696effd700 1 -- 172.21.15.162:0/3254188029 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69600016d0 con 0x7f69700b11b0 2020-11-28T21:51:33.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.367+0000 7f696effd700 1 -- 172.21.15.162:0/3254188029 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f696800b8b0 con 0x7f69700b11b0 2020-11-28T21:51:33.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.367+0000 7f696effd700 1 -- 172.21.15.162:0/3254188029 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f696800ba20 con 0x7f69700b11b0 2020-11-28T21:51:33.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.367+0000 7f697693d700 1 -- 172.21.15.162:0/3254188029 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f69700b11b0 msgr2=0x7f6970140fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:33.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.367+0000 7f697693d700 1 --2- 172.21.15.162:0/3254188029 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f69700b11b0 0x7f6970140fa0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f69580010d0 tx=0x7f69680091a0).stop 2020-11-28T21:51:33.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.367+0000 7f697693d700 1 -- 172.21.15.162:0/3254188029 shutdown_connections 2020-11-28T21:51:33.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.367+0000 7f697693d700 1 --2- 172.21.15.162:0/3254188029 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f69700b11b0 0x7f6970140fa0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:33.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.367+0000 7f697693d700 1 -- 172.21.15.162:0/3254188029 >> 172.21.15.162:0/3254188029 conn(0x7f697005b970 msgr2=0x7f697005bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:33.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.371+0000 7f697693d700 1 -- 172.21.15.162:0/3254188029 shutdown_connections 2020-11-28T21:51:33.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.371+0000 7f697693d700 1 -- 172.21.15.162:0/3254188029 wait complete. 2020-11-28T21:51:33.372 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.371+0000 7f697693d700 1 Processor -- start 2020-11-28T21:51:33.372 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.371+0000 7f697693d700 1 -- start start 2020-11-28T21:51:33.372 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.375+0000 7f697693d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69700b11b0 0x7f6970143520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:33.372 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.375+0000 7f697693d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f69700b1990 0x7f69700ae540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:33.373 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.375+0000 7f697693d700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f69700b1d70 0x7f69701415a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:33.373 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.375+0000 7f697693d700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69700b5640 con 0x7f69700b1990 2020-11-28T21:51:33.373 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.375+0000 7f697693d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69700b4270 con 0x7f69700b11b0 2020-11-28T21:51:33.373 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.375+0000 7f697693d700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69700b5830 con 0x7f69700b1d70 2020-11-28T21:51:33.373 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.375+0000 7f696ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69700b11b0 0x7f6970143520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:33.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.375+0000 7f696ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69700b11b0 0x7f6970143520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40406/0 (socket says 172.21.15.162:40406) 2020-11-28T21:51:33.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.375+0000 7f696ffff700 1 -- 172.21.15.162:0/1244921855 learned_addr learned my addr 172.21.15.162:0/1244921855 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:33.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.375+0000 7f696f7fe700 1 --2- 172.21.15.162:0/1244921855 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f69700b1990 0x7f69700ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:33.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.375+0000 7f6974eda700 1 --2- 172.21.15.162:0/1244921855 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f69700b1d70 0x7f69701415a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:33.375 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.375+0000 7f696ffff700 1 -- 172.21.15.162:0/1244921855 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f69700b1d70 msgr2=0x7f69701415a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:33.375 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.375+0000 7f696ffff700 1 --2- 172.21.15.162:0/1244921855 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f69700b1d70 0x7f69701415a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:33.375 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.375+0000 7f696ffff700 1 -- 172.21.15.162:0/1244921855 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f69700b1990 msgr2=0x7f69700ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:33.375 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.375+0000 7f696ffff700 1 --2- 172.21.15.162:0/1244921855 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f69700b1990 0x7f69700ae540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:33.375 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.375+0000 7f696ffff700 1 -- 172.21.15.162:0/1244921855 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69700b44a0 con 0x7f69700b11b0 2020-11-28T21:51:33.376 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.375+0000 7f696ffff700 1 --2- 172.21.15.162:0/1244921855 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69700b11b0 0x7f6970143520 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f6960001930 tx=0x7f6960001280).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:33.376 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.375+0000 7f696d7fa700 1 -- 172.21.15.162:0/1244921855 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6960007170 con 0x7f69700b11b0 2020-11-28T21:51:33.376 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.375+0000 7f696d7fa700 1 -- 172.21.15.162:0/1244921855 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69600072d0 con 0x7f69700b11b0 2020-11-28T21:51:33.376 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.375+0000 7f697693d700 1 -- 172.21.15.162:0/1244921855 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69700b4740 con 0x7f69700b11b0 2020-11-28T21:51:33.376 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.375+0000 7f696d7fa700 1 -- 172.21.15.162:0/1244921855 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69600048f0 con 0x7f69700b11b0 2020-11-28T21:51:33.377 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.375+0000 7f697693d700 1 -- 172.21.15.162:0/1244921855 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6970141f20 con 0x7f69700b11b0 2020-11-28T21:51:33.377 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.375+0000 7f697693d700 1 -- 172.21.15.162:0/1244921855 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69700b48d0 con 0x7f69700b11b0 2020-11-28T21:51:33.378 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.379+0000 7f696d7fa700 1 -- 172.21.15.162:0/1244921855 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f696001b070 con 0x7f69700b11b0 2020-11-28T21:51:33.378 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.379+0000 7f696d7fa700 1 --2- 172.21.15.162:0/1244921855 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6954055440 0x7f6954057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:33.378 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.379+0000 7f696d7fa700 1 -- 172.21.15.162:0/1244921855 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f696006d580 con 0x7f69700b11b0 2020-11-28T21:51:33.378 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.379+0000 7f696f7fe700 1 --2- 172.21.15.162:0/1244921855 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6954055440 0x7f6954057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:33.379 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.379+0000 7f696f7fe700 1 --2- 172.21.15.162:0/1244921855 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6954055440 0x7f6954057900 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f696801b000 tx=0x7f69700ac7c0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:33.380 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.383+0000 7f696d7fa700 1 -- 172.21.15.162:0/1244921855 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6960043880 con 0x7f69700b11b0 2020-11-28T21:51:33.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.527+0000 7f697693d700 1 -- 172.21.15.162:0/1244921855 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f6970057050 con 0x7f69700b11b0 2020-11-28T21:51:33.526 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.531+0000 7f696d7fa700 1 -- 172.21.15.162:0/1244921855 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v56) v1 ==== 81+0+33743 (secure 0 0 0) 0x7f6960046ea0 con 0x7f69700b11b0 2020-11-28T21:51:33.527 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:51:33.527 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2020-11-28T21:51:33.527 INFO:teuthology.orchestra.run.smithi162.stdout: "epoch": 56, 2020-11-28T21:51:33.528 INFO:teuthology.orchestra.run.smithi162.stdout: "fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", 2020-11-28T21:51:33.528 INFO:teuthology.orchestra.run.smithi162.stdout: "created": "2020-11-28T21:46:11.724372+0000", 2020-11-28T21:51:33.528 INFO:teuthology.orchestra.run.smithi162.stdout: "modified": "2020-11-28T21:50:07.723574+0000", 2020-11-28T21:51:33.528 INFO:teuthology.orchestra.run.smithi162.stdout: "last_up_change": "2020-11-28T21:46:19.591650+0000", 2020-11-28T21:51:33.528 INFO:teuthology.orchestra.run.smithi162.stdout: "last_in_change": "2020-11-28T21:46:19.591650+0000", 2020-11-28T21:51:33.529 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-11-28T21:51:33.529 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_num": 5799936, 2020-11-28T21:51:33.529 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_set": [ 2020-11-28T21:51:33.529 INFO:teuthology.orchestra.run.smithi162.stdout: "pglog_hardlimit", 2020-11-28T21:51:33.529 INFO:teuthology.orchestra.run.smithi162.stdout: "purged_snapdirs", 2020-11-28T21:51:33.530 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_deletes", 2020-11-28T21:51:33.530 INFO:teuthology.orchestra.run.smithi162.stdout: "sortbitwise" 2020-11-28T21:51:33.530 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:51:33.530 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_version": 4, 2020-11-28T21:51:33.530 INFO:teuthology.orchestra.run.smithi162.stdout: "full_ratio": 0.94999998807907104, 2020-11-28T21:51:33.531 INFO:teuthology.orchestra.run.smithi162.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-11-28T21:51:33.531 INFO:teuthology.orchestra.run.smithi162.stdout: "nearfull_ratio": 0.85000002384185791, 2020-11-28T21:51:33.531 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_snapshot": "", 2020-11-28T21:51:33.531 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_max": 10, 2020-11-28T21:51:33.531 INFO:teuthology.orchestra.run.smithi162.stdout: "max_osd": 8, 2020-11-28T21:51:33.532 INFO:teuthology.orchestra.run.smithi162.stdout: "require_min_compat_client": "jewel", 2020-11-28T21:51:33.532 INFO:teuthology.orchestra.run.smithi162.stdout: "min_compat_client": "jewel", 2020-11-28T21:51:33.532 INFO:teuthology.orchestra.run.smithi162.stdout: "require_osd_release": "octopus", 2020-11-28T21:51:33.532 INFO:teuthology.orchestra.run.smithi162.stdout: "pools": [ 2020-11-28T21:51:33.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.533 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 1, 2020-11-28T21:51:33.533 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "device_health_metrics", 2020-11-28T21:51:33.533 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:46:15.639551+0000", 2020-11-28T21:51:33.533 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:51:33.533 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:51:33.534 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:51:33.534 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:51:33.534 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:51:33.534 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:51:33.535 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:51:33.535 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:51:33.535 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 1, 2020-11-28T21:51:33.535 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 1, 2020-11-28T21:51:33.535 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 1, 2020-11-28T21:51:33.536 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 1, 2020-11-28T21:51:33.536 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 1, 2020-11-28T21:51:33.536 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:51:33.536 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:51:33.536 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:51:33.537 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:51:33.537 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:51:33.537 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:51:33.537 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:51:33.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.538 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "16", 2020-11-28T21:51:33.538 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:51:33.538 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:51:33.538 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:51:33.538 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:51:33.539 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:51:33.539 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:51:33.539 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:51:33.539 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:51:33.539 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:51:33.540 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:51:33.540 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:51:33.540 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:51:33.540 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:51:33.540 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:51:33.541 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:51:33.541 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:51:33.541 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:51:33.541 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:51:33.541 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:51:33.542 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:51:33.542 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:51:33.542 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:51:33.542 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:51:33.542 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:51:33.543 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:51:33.543 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:51:33.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.543 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:51:33.543 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:51:33.544 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:51:33.544 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:51:33.544 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:51:33.544 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:51:33.544 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:51:33.545 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:51:33.545 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:51:33.545 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:51:33.545 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:51:33.545 INFO:teuthology.orchestra.run.smithi162.stdout: "options": { 2020-11-28T21:51:33.546 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_min": 1 2020-11-28T21:51:33.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.546 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:51:33.546 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr_devicehealth": {} 2020-11-28T21:51:33.547 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:33.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.547 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 2, 2020-11-28T21:51:33.547 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "rbd", 2020-11-28T21:51:33.547 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:46:23.901366+0000", 2020-11-28T21:51:33.548 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:51:33.548 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:51:33.548 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:51:33.548 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:51:33.549 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:51:33.549 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:51:33.549 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:51:33.549 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:51:33.549 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2020-11-28T21:51:33.550 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2020-11-28T21:51:33.550 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2020-11-28T21:51:33.550 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2020-11-28T21:51:33.550 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2020-11-28T21:51:33.550 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:51:33.551 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:51:33.551 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:51:33.551 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:51:33.551 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:51:33.551 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:51:33.552 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:51:33.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.552 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "19", 2020-11-28T21:51:33.552 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:51:33.552 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:51:33.553 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:51:33.553 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:51:33.553 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:51:33.553 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:51:33.553 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:51:33.554 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:51:33.554 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:51:33.554 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:51:33.554 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:51:33.554 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:51:33.555 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:51:33.555 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:51:33.555 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:51:33.555 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:51:33.555 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:51:33.556 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:51:33.556 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:51:33.556 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:51:33.556 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:51:33.556 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:51:33.557 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:51:33.557 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:51:33.557 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:51:33.557 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:51:33.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.558 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:51:33.558 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:51:33.558 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:51:33.558 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:51:33.558 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:51:33.559 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:51:33.559 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:51:33.559 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:51:33.559 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:51:33.560 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:51:33.560 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:51:33.560 INFO:teuthology.orchestra.run.smithi162.stdout: "options": {}, 2020-11-28T21:51:33.560 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:51:33.560 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd": {} 2020-11-28T21:51:33.561 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:33.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.561 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 9, 2020-11-28T21:51:33.561 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "cephfs_metadata", 2020-11-28T21:51:33.562 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:50:03.211406+0000", 2020-11-28T21:51:33.562 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:51:33.562 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:51:33.562 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:51:33.562 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:51:33.563 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:51:33.563 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:51:33.563 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:51:33.563 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:51:33.563 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2020-11-28T21:51:33.564 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2020-11-28T21:51:33.564 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2020-11-28T21:51:33.564 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2020-11-28T21:51:33.564 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2020-11-28T21:51:33.564 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:51:33.565 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:51:33.565 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:51:33.565 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:51:33.565 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:51:33.565 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:51:33.566 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:51:33.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.566 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "54", 2020-11-28T21:51:33.566 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:51:33.566 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:51:33.567 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:51:33.567 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:51:33.567 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:51:33.567 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:51:33.567 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:51:33.568 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:51:33.568 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:51:33.568 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:51:33.568 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:51:33.568 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:51:33.569 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:51:33.569 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:51:33.569 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:51:33.569 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:51:33.569 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:51:33.570 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:51:33.570 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:51:33.570 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:51:33.570 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:51:33.570 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:51:33.571 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:51:33.571 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:51:33.571 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:51:33.571 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:51:33.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.572 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:51:33.572 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:51:33.572 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:51:33.572 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:51:33.572 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:51:33.573 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:51:33.573 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:51:33.573 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:51:33.573 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:51:33.573 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:51:33.574 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:51:33.574 INFO:teuthology.orchestra.run.smithi162.stdout: "options": { 2020-11-28T21:51:33.574 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_bias": 4, 2020-11-28T21:51:33.574 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_min": 16, 2020-11-28T21:51:33.574 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_priority": 5 2020-11-28T21:51:33.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.575 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:51:33.575 INFO:teuthology.orchestra.run.smithi162.stdout: "cephfs": { 2020-11-28T21:51:33.575 INFO:teuthology.orchestra.run.smithi162.stdout: "metadata": "cephfs" 2020-11-28T21:51:33.575 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:33.576 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:33.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.576 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 10, 2020-11-28T21:51:33.576 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "cephfs_data", 2020-11-28T21:51:33.577 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:50:04.511075+0000", 2020-11-28T21:51:33.577 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:51:33.577 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:51:33.577 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:51:33.577 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:51:33.578 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:51:33.578 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:51:33.578 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:51:33.578 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:51:33.579 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2020-11-28T21:51:33.579 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2020-11-28T21:51:33.579 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2020-11-28T21:51:33.579 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2020-11-28T21:51:33.579 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2020-11-28T21:51:33.580 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:51:33.580 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:51:33.580 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:51:33.580 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:51:33.580 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:51:33.581 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:51:33.581 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:51:33.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.581 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "56", 2020-11-28T21:51:33.581 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:51:33.581 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:51:33.582 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:51:33.582 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:51:33.582 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:51:33.582 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:51:33.583 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:51:33.583 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:51:33.583 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:51:33.583 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:51:33.583 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:51:33.584 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:51:33.584 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:51:33.584 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:51:33.584 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:51:33.584 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:51:33.584 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:51:33.585 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:51:33.585 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:51:33.585 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:51:33.585 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:51:33.585 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:51:33.586 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:51:33.586 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:51:33.586 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:51:33.586 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:51:33.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.587 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:51:33.587 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:51:33.587 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:51:33.587 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:51:33.588 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:51:33.588 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:51:33.588 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:51:33.588 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:51:33.588 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:51:33.589 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:51:33.589 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:51:33.589 INFO:teuthology.orchestra.run.smithi162.stdout: "options": {}, 2020-11-28T21:51:33.589 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:51:33.589 INFO:teuthology.orchestra.run.smithi162.stdout: "cephfs": { 2020-11-28T21:51:33.590 INFO:teuthology.orchestra.run.smithi162.stdout: "data": "cephfs" 2020-11-28T21:51:33.590 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:33.590 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:33.590 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:33.590 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:51:33.591 INFO:teuthology.orchestra.run.smithi162.stdout: "osds": [ 2020-11-28T21:51:33.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.591 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 0, 2020-11-28T21:51:33.591 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "99c7d9ae-adfa-4fd7-880a-23e6baf3281b", 2020-11-28T21:51:33.591 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:51:33.592 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:51:33.592 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:51:33.593 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:51:33.593 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:51:33.593 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:51:33.593 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:51:33.593 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 54, 2020-11-28T21:51:33.594 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:51:33.594 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:51:33.594 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:51:33.594 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:51:33.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.595 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:51:33.595 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6810", 2020-11-28T21:51:33.595 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:51:33.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.596 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:51:33.596 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6811", 2020-11-28T21:51:33.596 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:51:33.596 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:33.597 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:51:33.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.597 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:51:33.597 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:51:33.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.598 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:51:33.598 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6812", 2020-11-28T21:51:33.598 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:51:33.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.599 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:51:33.599 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6813", 2020-11-28T21:51:33.599 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:51:33.599 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:33.599 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:51:33.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.600 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:51:33.600 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:51:33.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.601 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:51:33.601 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6816", 2020-11-28T21:51:33.601 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:51:33.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.602 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:51:33.602 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6817", 2020-11-28T21:51:33.602 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:51:33.602 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:33.602 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:51:33.603 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.603 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:51:33.603 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:51:33.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.603 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:51:33.604 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6814", 2020-11-28T21:51:33.604 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:51:33.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.604 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:51:33.605 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6815", 2020-11-28T21:51:33.605 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:51:33.605 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:33.605 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:51:33.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.606 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6811/12313", 2020-11-28T21:51:33.606 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6813/12313", 2020-11-28T21:51:33.606 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6817/12313", 2020-11-28T21:51:33.606 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6815/12313", 2020-11-28T21:51:33.606 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:51:33.607 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:51:33.607 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:51:33.607 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:51:33.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.608 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 1, 2020-11-28T21:51:33.608 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "6ce6220a-00ef-4d13-abba-56f87636fb5f", 2020-11-28T21:51:33.608 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:51:33.608 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:51:33.608 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:51:33.609 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:51:33.609 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:51:33.609 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:51:33.609 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:51:33.609 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 41, 2020-11-28T21:51:33.610 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:51:33.610 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:51:33.610 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:51:33.610 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:51:33.611 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.611 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:51:33.611 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6802", 2020-11-28T21:51:33.611 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:51:33.611 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.612 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.612 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:51:33.612 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6803", 2020-11-28T21:51:33.612 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:51:33.612 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:33.613 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:51:33.613 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.613 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:51:33.613 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:51:33.613 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.614 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:51:33.614 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6804", 2020-11-28T21:51:33.614 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:51:33.614 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.614 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.615 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:51:33.615 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6805", 2020-11-28T21:51:33.615 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:51:33.615 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:33.615 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:51:33.616 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.616 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:51:33.616 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:51:33.616 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.616 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:51:33.617 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6808", 2020-11-28T21:51:33.617 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:51:33.617 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.617 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.617 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:51:33.618 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6809", 2020-11-28T21:51:33.618 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:51:33.618 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:33.618 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:51:33.618 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.619 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:51:33.619 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:51:33.619 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.619 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:51:33.619 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6806", 2020-11-28T21:51:33.620 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:51:33.620 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.620 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.620 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:51:33.620 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6807", 2020-11-28T21:51:33.621 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:51:33.621 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:33.621 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:51:33.621 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.622 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6803/12311", 2020-11-28T21:51:33.622 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6805/12311", 2020-11-28T21:51:33.622 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6809/12311", 2020-11-28T21:51:33.622 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6807/12311", 2020-11-28T21:51:33.622 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:51:33.623 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:51:33.623 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:51:33.623 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:51:33.623 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.623 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.624 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 2, 2020-11-28T21:51:33.624 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "db4e510d-27c1-47fc-a902-8b32e7dfd73e", 2020-11-28T21:51:33.624 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:51:33.624 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:51:33.624 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:51:33.625 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:51:33.625 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:51:33.625 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:51:33.625 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:51:33.625 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 53, 2020-11-28T21:51:33.626 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:51:33.626 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:51:33.626 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:51:33.626 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:51:33.626 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.627 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:51:33.627 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6818", 2020-11-28T21:51:33.627 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:51:33.627 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.627 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.628 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:51:33.628 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6819", 2020-11-28T21:51:33.628 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:51:33.628 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:33.628 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:51:33.629 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.629 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:51:33.629 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:51:33.629 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.629 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:51:33.630 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6820", 2020-11-28T21:51:33.630 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:51:33.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.630 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:51:33.631 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6821", 2020-11-28T21:51:33.631 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:51:33.631 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:33.631 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:51:33.631 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.632 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:51:33.632 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:51:33.632 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.632 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:51:33.632 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6824", 2020-11-28T21:51:33.633 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:51:33.633 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.633 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.633 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:51:33.633 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6825", 2020-11-28T21:51:33.634 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:51:33.634 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:33.634 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:51:33.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.635 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:51:33.635 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:51:33.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.635 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:51:33.635 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6822", 2020-11-28T21:51:33.636 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:51:33.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.636 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:51:33.636 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6823", 2020-11-28T21:51:33.637 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:51:33.637 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:33.637 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:51:33.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.637 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6819/12312", 2020-11-28T21:51:33.638 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6821/12312", 2020-11-28T21:51:33.638 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6825/12312", 2020-11-28T21:51:33.638 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6823/12312", 2020-11-28T21:51:33.638 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:51:33.638 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:51:33.639 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:51:33.639 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:51:33.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.639 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 3, 2020-11-28T21:51:33.640 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "4094cb87-f04a-45e1-8710-fb4b651949f6", 2020-11-28T21:51:33.640 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:51:33.640 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:51:33.640 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:51:33.640 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:51:33.641 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:51:33.641 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:51:33.641 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:51:33.641 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 53, 2020-11-28T21:51:33.641 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:51:33.642 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:51:33.642 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:51:33.642 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:51:33.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.642 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:51:33.643 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6826", 2020-11-28T21:51:33.643 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:51:33.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.643 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:51:33.644 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6827", 2020-11-28T21:51:33.644 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:51:33.644 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:33.644 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:51:33.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.645 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:51:33.645 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:51:33.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.645 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:51:33.645 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6828", 2020-11-28T21:51:33.646 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:51:33.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.646 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:51:33.646 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6829", 2020-11-28T21:51:33.647 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:51:33.647 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:33.647 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:51:33.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.648 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:51:33.648 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:51:33.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.648 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:51:33.648 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6832", 2020-11-28T21:51:33.649 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:51:33.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.649 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:51:33.649 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6833", 2020-11-28T21:51:33.650 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:51:33.650 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:33.650 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:51:33.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.650 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:51:33.651 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:51:33.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.651 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:51:33.651 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6830", 2020-11-28T21:51:33.651 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:51:33.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.652 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.652 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:51:33.652 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6831", 2020-11-28T21:51:33.652 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:51:33.653 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:33.653 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:51:33.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.653 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6827/12323", 2020-11-28T21:51:33.653 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6829/12323", 2020-11-28T21:51:33.654 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6833/12323", 2020-11-28T21:51:33.654 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6831/12323", 2020-11-28T21:51:33.654 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:51:33.654 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:51:33.654 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:51:33.655 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:51:33.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.655 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 4, 2020-11-28T21:51:33.655 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "b907bf53-c297-4756-847b-3be96680eef5", 2020-11-28T21:51:33.656 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:51:33.656 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:51:33.656 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:51:33.656 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:51:33.656 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:51:33.657 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:51:33.657 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:51:33.657 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 53, 2020-11-28T21:51:33.657 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:51:33.657 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:51:33.658 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:51:33.658 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:51:33.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.658 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:51:33.658 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6800", 2020-11-28T21:51:33.659 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:51:33.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.659 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:51:33.660 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6801", 2020-11-28T21:51:33.660 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:51:33.660 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:33.660 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:51:33.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.661 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:51:33.661 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:51:33.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.661 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:51:33.661 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6802", 2020-11-28T21:51:33.662 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:51:33.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.662 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:51:33.662 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6803", 2020-11-28T21:51:33.663 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:51:33.663 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:33.663 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:51:33.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.663 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:51:33.664 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:51:33.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.664 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:51:33.664 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6806", 2020-11-28T21:51:33.664 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:51:33.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.665 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:51:33.665 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6807", 2020-11-28T21:51:33.665 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:51:33.666 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:33.666 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:51:33.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.666 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:51:33.666 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:51:33.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.667 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:51:33.667 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6804", 2020-11-28T21:51:33.667 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:51:33.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.668 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:51:33.668 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6805", 2020-11-28T21:51:33.668 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:51:33.668 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:33.669 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:51:33.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.669 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6801/12527", 2020-11-28T21:51:33.669 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6803/12527", 2020-11-28T21:51:33.669 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6807/12527", 2020-11-28T21:51:33.670 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6805/12527", 2020-11-28T21:51:33.670 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:51:33.670 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:51:33.670 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:51:33.670 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:51:33.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.671 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 5, 2020-11-28T21:51:33.671 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "55ba305d-0505-4212-aebb-631e8554b440", 2020-11-28T21:51:33.671 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:51:33.672 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:51:33.672 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:51:33.672 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:51:33.672 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:51:33.672 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:51:33.673 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:51:33.673 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 53, 2020-11-28T21:51:33.673 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:51:33.673 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:51:33.673 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:51:33.674 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:51:33.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.674 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:51:33.674 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6808", 2020-11-28T21:51:33.674 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:51:33.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.675 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:51:33.675 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6809", 2020-11-28T21:51:33.675 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:51:33.676 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:33.676 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:51:33.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.676 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:51:33.676 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:51:33.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.677 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:51:33.677 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6810", 2020-11-28T21:51:33.677 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:51:33.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.678 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:51:33.678 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6811", 2020-11-28T21:51:33.678 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:51:33.678 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:33.679 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:51:33.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.679 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:51:33.679 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:51:33.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.680 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:51:33.680 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6814", 2020-11-28T21:51:33.680 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:51:33.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.681 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:51:33.681 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6815", 2020-11-28T21:51:33.681 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:51:33.681 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:33.681 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:51:33.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.682 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:51:33.682 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:51:33.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.682 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:51:33.683 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6812", 2020-11-28T21:51:33.683 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:51:33.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.683 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:51:33.684 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6813", 2020-11-28T21:51:33.684 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:51:33.684 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:33.684 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:51:33.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.685 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6809/12529", 2020-11-28T21:51:33.685 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6811/12529", 2020-11-28T21:51:33.685 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6815/12529", 2020-11-28T21:51:33.685 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6813/12529", 2020-11-28T21:51:33.685 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:51:33.686 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:51:33.686 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:51:33.686 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:51:33.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.687 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 6, 2020-11-28T21:51:33.687 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "9ada6051-493b-46b5-ba22-1b401440490f", 2020-11-28T21:51:33.687 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:51:33.687 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:51:33.687 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:51:33.688 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:51:33.688 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:51:33.688 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:51:33.688 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:51:33.688 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 51, 2020-11-28T21:51:33.689 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:51:33.689 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:51:33.689 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:51:33.689 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:51:33.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.690 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:51:33.690 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6824", 2020-11-28T21:51:33.690 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:51:33.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.691 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:51:33.691 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6825", 2020-11-28T21:51:33.691 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:51:33.691 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:33.691 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:51:33.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.692 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:51:33.692 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:51:33.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.692 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:51:33.693 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6826", 2020-11-28T21:51:33.693 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:51:33.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.693 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:51:33.694 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6827", 2020-11-28T21:51:33.694 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:51:33.694 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:33.694 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:51:33.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.695 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:51:33.695 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:51:33.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.695 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:51:33.695 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6830", 2020-11-28T21:51:33.696 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:51:33.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.696 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:51:33.696 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6831", 2020-11-28T21:51:33.697 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:51:33.697 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:33.697 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:51:33.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.697 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:51:33.698 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:51:33.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.698 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:51:33.698 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6828", 2020-11-28T21:51:33.698 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:51:33.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.699 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:51:33.699 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6829", 2020-11-28T21:51:33.700 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:51:33.700 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:33.700 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:51:33.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.700 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6825/12530", 2020-11-28T21:51:33.701 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6827/12530", 2020-11-28T21:51:33.701 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6831/12530", 2020-11-28T21:51:33.701 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6829/12530", 2020-11-28T21:51:33.701 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:51:33.701 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:51:33.702 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:51:33.702 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:51:33.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.702 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 7, 2020-11-28T21:51:33.703 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "c328b06d-a216-4cba-aad7-9ad081340613", 2020-11-28T21:51:33.703 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:51:33.703 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:51:33.703 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:51:33.703 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:51:33.704 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:51:33.704 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:51:33.704 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:51:33.704 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 51, 2020-11-28T21:51:33.704 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:51:33.705 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:51:33.705 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:51:33.705 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:51:33.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.705 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:51:33.706 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6816", 2020-11-28T21:51:33.706 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:51:33.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.706 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:51:33.707 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6817", 2020-11-28T21:51:33.707 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:51:33.707 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:33.707 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:51:33.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.708 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:51:33.708 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:51:33.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.708 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:51:33.708 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6818", 2020-11-28T21:51:33.709 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:51:33.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.709 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:51:33.709 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6819", 2020-11-28T21:51:33.710 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:51:33.710 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:33.710 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:51:33.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.710 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:51:33.711 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:51:33.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.711 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:51:33.711 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6822", 2020-11-28T21:51:33.711 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:51:33.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.712 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:51:33.712 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6823", 2020-11-28T21:51:33.712 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:51:33.713 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:33.713 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:51:33.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.713 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:51:33.713 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:51:33.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.714 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:51:33.714 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6820", 2020-11-28T21:51:33.714 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:51:33.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.715 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:51:33.715 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6821", 2020-11-28T21:51:33.715 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:51:33.715 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:33.716 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:51:33.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.716 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6817/12528", 2020-11-28T21:51:33.716 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6819/12528", 2020-11-28T21:51:33.716 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6823/12528", 2020-11-28T21:51:33.717 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6821/12528", 2020-11-28T21:51:33.717 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:51:33.717 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:51:33.717 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:51:33.717 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:51:33.718 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:33.718 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:51:33.718 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_xinfo": [ 2020-11-28T21:51:33.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.718 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 0, 2020-11-28T21:51:33.719 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:51:33.719 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:51:33.719 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:51:33.719 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:51:33.719 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:51:33.720 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.038052+0000", 2020-11-28T21:51:33.720 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:51:33.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.721 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 1, 2020-11-28T21:51:33.721 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:51:33.721 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:51:33.722 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:51:33.722 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:51:33.722 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:51:33.722 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:17.947136+0000", 2020-11-28T21:51:33.722 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:51:33.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.723 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 2, 2020-11-28T21:51:33.723 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:51:33.723 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:51:33.724 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:51:33.724 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:51:33.724 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:51:33.724 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.001186+0000", 2020-11-28T21:51:33.724 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:51:33.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.725 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 3, 2020-11-28T21:51:33.725 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:51:33.725 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:51:33.726 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:51:33.726 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:51:33.726 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:51:33.726 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.199284+0000", 2020-11-28T21:51:33.727 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:51:33.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.727 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 4, 2020-11-28T21:51:33.727 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:51:33.728 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:51:33.728 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:51:33.728 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:51:33.728 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:51:33.728 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.038823+0000", 2020-11-28T21:51:33.729 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:51:33.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.729 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 5, 2020-11-28T21:51:33.729 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:51:33.730 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:51:33.730 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:51:33.730 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:51:33.730 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:51:33.730 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.009322+0000", 2020-11-28T21:51:33.731 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:51:33.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.731 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 6, 2020-11-28T21:51:33.731 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:51:33.732 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:51:33.732 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:51:33.732 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:51:33.732 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:51:33.732 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.238927+0000", 2020-11-28T21:51:33.733 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:51:33.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:33.733 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 7, 2020-11-28T21:51:33.733 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:51:33.734 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:51:33.734 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:51:33.734 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:51:33.734 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:51:33.734 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.029839+0000", 2020-11-28T21:51:33.735 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:51:33.735 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:33.735 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:51:33.735 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_upmap": [], 2020-11-28T21:51:33.735 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_upmap_items": [], 2020-11-28T21:51:33.736 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_temp": [], 2020-11-28T21:51:33.736 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_temp": [], 2020-11-28T21:51:33.736 INFO:teuthology.orchestra.run.smithi162.stdout: "blacklist": {}, 2020-11-28T21:51:33.736 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profiles": { 2020-11-28T21:51:33.736 INFO:teuthology.orchestra.run.smithi162.stdout: "default": { 2020-11-28T21:51:33.737 INFO:teuthology.orchestra.run.smithi162.stdout: "crush-failure-domain": "osd", 2020-11-28T21:51:33.737 INFO:teuthology.orchestra.run.smithi162.stdout: "k": "2", 2020-11-28T21:51:33.737 INFO:teuthology.orchestra.run.smithi162.stdout: "m": "1", 2020-11-28T21:51:33.737 INFO:teuthology.orchestra.run.smithi162.stdout: "plugin": "jerasure", 2020-11-28T21:51:33.737 INFO:teuthology.orchestra.run.smithi162.stdout: "ruleset-failure-domain": "osd", 2020-11-28T21:51:33.738 INFO:teuthology.orchestra.run.smithi162.stdout: "technique": "reed_sol_van" 2020-11-28T21:51:33.738 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:33.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:33.738 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps_queue": [], 2020-11-28T21:51:33.738 INFO:teuthology.orchestra.run.smithi162.stdout: "new_removed_snaps": [], 2020-11-28T21:51:33.739 INFO:teuthology.orchestra.run.smithi162.stdout: "new_purged_snaps": [], 2020-11-28T21:51:33.739 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_node_flags": {}, 2020-11-28T21:51:33.739 INFO:teuthology.orchestra.run.smithi162.stdout: "device_class_flags": {} 2020-11-28T21:51:33.739 INFO:teuthology.orchestra.run.smithi162.stdout:} 2020-11-28T21:51:33.741 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.531+0000 7f697693d700 1 -- 172.21.15.162:0/1244921855 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6954055440 msgr2=0x7f6954057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:33.741 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.531+0000 7f697693d700 1 --2- 172.21.15.162:0/1244921855 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6954055440 0x7f6954057900 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f696801b000 tx=0x7f69700ac7c0).stop 2020-11-28T21:51:33.741 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.531+0000 7f697693d700 1 -- 172.21.15.162:0/1244921855 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69700b11b0 msgr2=0x7f6970143520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:33.741 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.531+0000 7f697693d700 1 --2- 172.21.15.162:0/1244921855 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69700b11b0 0x7f6970143520 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f6960001930 tx=0x7f6960001280).stop 2020-11-28T21:51:33.742 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.531+0000 7f697693d700 1 -- 172.21.15.162:0/1244921855 shutdown_connections 2020-11-28T21:51:33.742 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.531+0000 7f697693d700 1 --2- 172.21.15.162:0/1244921855 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6954055440 0x7f6954057900 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:33.742 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.531+0000 7f697693d700 1 --2- 172.21.15.162:0/1244921855 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f69700b1d70 0x7f69701415a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:33.742 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.531+0000 7f697693d700 1 --2- 172.21.15.162:0/1244921855 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69700b11b0 0x7f6970143520 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:33.742 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.531+0000 7f697693d700 1 --2- 172.21.15.162:0/1244921855 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f69700b1990 0x7f69700ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:33.743 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.531+0000 7f697693d700 1 -- 172.21.15.162:0/1244921855 >> 172.21.15.162:0/1244921855 conn(0x7f697005b970 msgr2=0x7f6970060290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:33.743 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.531+0000 7f697693d700 1 -- 172.21.15.162:0/1244921855 shutdown_connections 2020-11-28T21:51:33.743 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.531+0000 7f697693d700 1 -- 172.21.15.162:0/1244921855 wait complete. 2020-11-28T21:51:33.744 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:51:33.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.867+0000 7f4dd51f5700 1 Processor -- start 2020-11-28T21:51:33.864 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.867+0000 7f4dd51f5700 1 -- start start 2020-11-28T21:51:33.865 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.867+0000 7f4dd51f5700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4dd00608f0 0x7f4dd014b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:33.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.867+0000 7f4dd51f5700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dd00b6210 con 0x7f4dd009d890 2020-11-28T21:51:33.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.867+0000 7f4dd51f5700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dd00b4a70 con 0x7f4dd00b5e30 2020-11-28T21:51:33.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.867+0000 7f4dd51f5700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd00606e0 con 0x7f4dd00608f0 2020-11-28T21:51:33.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.867+0000 7f4dcffff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f4dd00b5e30 0x7f4dd009d180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:47260/0 (socket says 172.21.15.162:47260) 2020-11-28T21:51:33.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.867+0000 7f4dcffff700 1 -- 172.21.15.162:0/1247827563 learned_addr learned my addr 172.21.15.162:0/1247827563 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:51:33.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.867+0000 7f4dd49f4700 1 --2- 172.21.15.162:0/1247827563 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4dd00608f0 0x7f4dd014b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:33.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.867+0000 7f4dceffd700 1 -- 172.21.15.162:0/1247827563 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3557377633 0 0) 0x7f4dd00b4a70 con 0x7f4dd00b5e30 2020-11-28T21:51:33.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.867+0000 7f4dceffd700 1 -- 172.21.15.162:0/1247827563 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4dbc003230 con 0x7f4dd00b5e30 2020-11-28T21:51:33.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.867+0000 7f4dceffd700 1 -- 172.21.15.162:0/1247827563 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2968398533 0 0) 0x7f4dd00b6210 con 0x7f4dd009d890 2020-11-28T21:51:33.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.867+0000 7f4dceffd700 1 -- 172.21.15.162:0/1247827563 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4dd00b4a70 con 0x7f4dd009d890 2020-11-28T21:51:33.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.867+0000 7f4dd49f4700 1 -- 172.21.15.162:0/1247827563 >> v1:172.21.15.162:6789/0 conn(0x7f4dd00b5e30 legacy=0x7f4dd009d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:33.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.867+0000 7f4dceffd700 1 -- 172.21.15.162:0/1247827563 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2294439548 0 0) 0x7f4dbc003230 con 0x7f4dd00b5e30 2020-11-28T21:51:33.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.867+0000 7f4dd49f4700 1 -- 172.21.15.162:0/1247827563 >> v1:172.21.15.118:6789/0 conn(0x7f4dd009d890 legacy=0x7f4dd014a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:33.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.867+0000 7f4dd49f4700 1 -- 172.21.15.162:0/1247827563 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd00b3d70 con 0x7f4dd00608f0 2020-11-28T21:51:33.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.871+0000 7f4dd49f4700 1 --2- 172.21.15.162:0/1247827563 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4dd00608f0 0x7f4dd014b040 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f4dc80032b0 tx=0x7f4dc80091a0).ready entity=mon.2 client_cookie=6a05f21db09eca06 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:33.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.871+0000 7f4dceffd700 1 -- 172.21.15.162:0/1247827563 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4dc800b940 con 0x7f4dd00608f0 2020-11-28T21:51:33.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.871+0000 7f4dceffd700 1 -- 172.21.15.162:0/1247827563 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dc800baa0 con 0x7f4dd00608f0 2020-11-28T21:51:33.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.871+0000 7f4dd51f5700 1 -- 172.21.15.162:0/1247827563 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4dd00608f0 msgr2=0x7f4dd014b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:33.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.871+0000 7f4dd51f5700 1 --2- 172.21.15.162:0/1247827563 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4dd00608f0 0x7f4dd014b040 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f4dc80032b0 tx=0x7f4dc80091a0).stop 2020-11-28T21:51:33.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.871+0000 7f4dd51f5700 1 -- 172.21.15.162:0/1247827563 shutdown_connections 2020-11-28T21:51:33.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.871+0000 7f4dd51f5700 1 --2- 172.21.15.162:0/1247827563 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4dd00608f0 0x7f4dd014b040 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:33.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.871+0000 7f4dd51f5700 1 -- 172.21.15.162:0/1247827563 >> 172.21.15.162:0/1247827563 conn(0x7f4dd005b970 msgr2=0x7f4dd005bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:33.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.871+0000 7f4dd51f5700 1 -- 172.21.15.162:0/1247827563 shutdown_connections 2020-11-28T21:51:33.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.871+0000 7f4dd51f5700 1 -- 172.21.15.162:0/1247827563 wait complete. 2020-11-28T21:51:33.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.871+0000 7f4dd51f5700 1 Processor -- start 2020-11-28T21:51:33.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.871+0000 7f4dd51f5700 1 -- start start 2020-11-28T21:51:33.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.875+0000 7f4dd51f5700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4dd00608f0 0x7f4dd00b1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:33.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.875+0000 7f4dd51f5700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4dd009d890 0x7f4dd0063c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:33.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.875+0000 7f4dd51f5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4dd00b5e30 0x7f4dd0064100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:33.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.875+0000 7f4dd51f5700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd00648a0 con 0x7f4dd00608f0 2020-11-28T21:51:33.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.875+0000 7f4dd51f5700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd0159490 con 0x7f4dd00b5e30 2020-11-28T21:51:33.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.875+0000 7f4dd51f5700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd0064ad0 con 0x7f4dd009d890 2020-11-28T21:51:33.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.875+0000 7f4dd49f4700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4dd00b5e30 0x7f4dd0064100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:33.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.875+0000 7f4dd49f4700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4dd00b5e30 0x7f4dd0064100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40422/0 (socket says 172.21.15.162:40422) 2020-11-28T21:51:33.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.875+0000 7f4dd49f4700 1 -- 172.21.15.162:0/2209444567 learned_addr learned my addr 172.21.15.162:0/2209444567 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:33.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.875+0000 7f4dcffff700 1 --2- 172.21.15.162:0/2209444567 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4dd00608f0 0x7f4dd00b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:33.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.875+0000 7f4dcf7fe700 1 --2- 172.21.15.162:0/2209444567 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4dd009d890 0x7f4dd0063c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:33.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.875+0000 7f4dd49f4700 1 -- 172.21.15.162:0/2209444567 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4dd009d890 msgr2=0x7f4dd0063c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:33.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.875+0000 7f4dd49f4700 1 --2- 172.21.15.162:0/2209444567 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4dd009d890 0x7f4dd0063c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:33.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.875+0000 7f4dd49f4700 1 -- 172.21.15.162:0/2209444567 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4dd00608f0 msgr2=0x7f4dd00b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:33.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.875+0000 7f4dd49f4700 1 --2- 172.21.15.162:0/2209444567 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4dd00608f0 0x7f4dd00b1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:33.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.875+0000 7f4dd49f4700 1 -- 172.21.15.162:0/2209444567 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd0064d70 con 0x7f4dd00b5e30 2020-11-28T21:51:33.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.875+0000 7f4dcffff700 1 --2- 172.21.15.162:0/2209444567 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4dd00608f0 0x7f4dd00b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:51:33.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.875+0000 7f4dd49f4700 1 --2- 172.21.15.162:0/2209444567 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4dd00b5e30 0x7f4dd0064100 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f4dd0065040 tx=0x7f4dc8016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:33.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.875+0000 7f4dcd7fa700 1 -- 172.21.15.162:0/2209444567 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4dc8011e20 con 0x7f4dd00b5e30 2020-11-28T21:51:33.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.875+0000 7f4dcd7fa700 1 -- 172.21.15.162:0/2209444567 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dc8020400 con 0x7f4dd00b5e30 2020-11-28T21:51:33.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.875+0000 7f4dd51f5700 1 -- 172.21.15.162:0/2209444567 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dd00645f0 con 0x7f4dd00b5e30 2020-11-28T21:51:33.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.875+0000 7f4dcd7fa700 1 -- 172.21.15.162:0/2209444567 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4dc8020760 con 0x7f4dd00b5e30 2020-11-28T21:51:33.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.875+0000 7f4dd51f5700 1 -- 172.21.15.162:0/2209444567 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd00acc70 con 0x7f4dd00b5e30 2020-11-28T21:51:33.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.875+0000 7f4dd51f5700 1 -- 172.21.15.162:0/2209444567 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dd0057050 con 0x7f4dd00b5e30 2020-11-28T21:51:33.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.875+0000 7f4dcd7fa700 1 -- 172.21.15.162:0/2209444567 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4dc8027030 con 0x7f4dd00b5e30 2020-11-28T21:51:33.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.875+0000 7f4dcd7fa700 1 --2- 172.21.15.162:0/2209444567 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4db0055330 0x7f4db00577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:33.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.875+0000 7f4dcd7fa700 1 -- 172.21.15.162:0/2209444567 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4dc806eee0 con 0x7f4dd00b5e30 2020-11-28T21:51:33.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.875+0000 7f4dcffff700 1 --2- 172.21.15.162:0/2209444567 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4db0055330 0x7f4db00577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:33.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.879+0000 7f4dcffff700 1 --2- 172.21.15.162:0/2209444567 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4db0055330 0x7f4db00577f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f4dc400b000 tx=0x7f4dc40070e0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:33.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:33.883+0000 7f4dcd7fa700 1 -- 172.21.15.162:0/2209444567 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4dc80451e0 con 0x7f4dd00b5e30 2020-11-28T21:51:34.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.067+0000 7f4dd51f5700 1 -- 172.21.15.162:0/2209444567 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4dd005d140 con 0x7f4dd00b5e30 2020-11-28T21:51:34.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.071+0000 7f4dcd7fa700 1 -- 172.21.15.162:0/2209444567 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+2501 (secure 0 0 0) 0x7f4dc80259e0 con 0x7f4dd00b5e30 2020-11-28T21:51:34.066 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:51:34.067 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":21,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/1245801281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1245801281},{"type":"v1","addr":"172.21.15.118:6833","nonce":1245801281}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":18},{"gid":5536,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6835/3629429916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3629429916},{"type":"v1","addr":"172.21.15.162:6835","nonce":3629429916}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":19}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:50:05.655172+0000","modified":"2020-11-28T21:50:06.657968+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5551},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5551":{"gid":5551,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/1902721947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1902721947},{"type":"v1","addr":"172.21.15.162:6837","nonce":1902721947}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-28T21:51:34.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.071+0000 7f4dd51f5700 1 -- 172.21.15.162:0/2209444567 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4db0055330 msgr2=0x7f4db00577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:34.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.071+0000 7f4dd51f5700 1 --2- 172.21.15.162:0/2209444567 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4db0055330 0x7f4db00577f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f4dc400b000 tx=0x7f4dc40070e0).stop 2020-11-28T21:51:34.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.071+0000 7f4dd51f5700 1 -- 172.21.15.162:0/2209444567 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4dd00b5e30 msgr2=0x7f4dd0064100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:34.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.071+0000 7f4dd51f5700 1 --2- 172.21.15.162:0/2209444567 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4dd00b5e30 0x7f4dd0064100 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f4dd0065040 tx=0x7f4dc8016000).stop 2020-11-28T21:51:34.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.071+0000 7f4dd51f5700 1 -- 172.21.15.162:0/2209444567 shutdown_connections 2020-11-28T21:51:34.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.071+0000 7f4dd51f5700 1 --2- 172.21.15.162:0/2209444567 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4db0055330 0x7f4db00577f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:34.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.071+0000 7f4dd51f5700 1 --2- 172.21.15.162:0/2209444567 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4dd00b5e30 0x7f4dd0064100 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:34.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.071+0000 7f4dd51f5700 1 --2- 172.21.15.162:0/2209444567 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4dd00608f0 0x7f4dd00b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:34.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.071+0000 7f4dd51f5700 1 --2- 172.21.15.162:0/2209444567 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4dd009d890 0x7f4dd0063c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:34.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.071+0000 7f4dd51f5700 1 -- 172.21.15.162:0/2209444567 >> 172.21.15.162:0/2209444567 conn(0x7f4dd005b970 msgr2=0x7f4dd00b64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:34.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.071+0000 7f4dd51f5700 1 -- 172.21.15.162:0/2209444567 shutdown_connections 2020-11-28T21:51:34.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.071+0000 7f4dd51f5700 1 -- 172.21.15.162:0/2209444567 wait complete. 2020-11-28T21:51:34.071 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 21 2020-11-28T21:51:34.080 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2020-11-28T21:51:34.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.199+0000 7f6b471ad700 1 Processor -- start 2020-11-28T21:51:34.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.199+0000 7f6b471ad700 1 -- start start 2020-11-28T21:51:34.200 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.199+0000 7f6b471ad700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6b4005cbd0 0x7f6b400aac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:34.200 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.199+0000 7f6b471ad700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b400b0230 con 0x7f6b400af640 2020-11-28T21:51:34.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.199+0000 7f6b471ad700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b400ab0f0 con 0x7f6b40060cf0 2020-11-28T21:51:34.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.199+0000 7f6b471ad700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b400b12a0 con 0x7f6b4005cbd0 2020-11-28T21:51:34.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.199+0000 7f6b461ab700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f6b40060cf0 0x7f6b400aa4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:47274/0 (socket says 172.21.15.162:47274) 2020-11-28T21:51:34.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.199+0000 7f6b461ab700 1 -- 172.21.15.162:0/3424836977 learned_addr learned my addr 172.21.15.162:0/3424836977 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:51:34.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.199+0000 7f6b459aa700 1 --2- 172.21.15.162:0/3424836977 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6b4005cbd0 0x7f6b400aac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:34.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.199+0000 7f6b451a9700 1 -- 172.21.15.162:0/3424836977 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1672016560 0 0) 0x7f6b400ab0f0 con 0x7f6b40060cf0 2020-11-28T21:51:34.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.199+0000 7f6b451a9700 1 -- 172.21.15.162:0/3424836977 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b28003230 con 0x7f6b40060cf0 2020-11-28T21:51:34.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.199+0000 7f6b459aa700 1 -- 172.21.15.162:0/3424836977 >> v1:172.21.15.162:6789/0 conn(0x7f6b40060cf0 legacy=0x7f6b400aa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:34.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.199+0000 7f6b459aa700 1 -- 172.21.15.162:0/3424836977 >> v1:172.21.15.118:6789/0 conn(0x7f6b400af640 legacy=0x7f6b400ae1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:34.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.199+0000 7f6b451a9700 1 -- 172.21.15.162:0/3424836977 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4046905169 0 0) 0x7f6b400b0230 con 0x7f6b400af640 2020-11-28T21:51:34.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.199+0000 7f6b459aa700 1 -- 172.21.15.162:0/3424836977 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b400a8c00 con 0x7f6b4005cbd0 2020-11-28T21:51:34.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.199+0000 7f6b459aa700 1 --2- 172.21.15.162:0/3424836977 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6b4005cbd0 0x7f6b400aac00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f6b30002f30 tx=0x7f6b30006dc0).ready entity=mon.2 client_cookie=8baf12977721411e server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:34.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.203+0000 7f6b451a9700 1 -- 172.21.15.162:0/3424836977 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b300097b0 con 0x7f6b4005cbd0 2020-11-28T21:51:34.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.203+0000 7f6b451a9700 1 -- 172.21.15.162:0/3424836977 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b30009910 con 0x7f6b4005cbd0 2020-11-28T21:51:34.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.203+0000 7f6b451a9700 1 -- 172.21.15.162:0/3424836977 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b30009c70 con 0x7f6b4005cbd0 2020-11-28T21:51:34.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.203+0000 7f6b471ad700 1 -- 172.21.15.162:0/3424836977 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6b4005cbd0 msgr2=0x7f6b400aac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:34.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.203+0000 7f6b471ad700 1 --2- 172.21.15.162:0/3424836977 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6b4005cbd0 0x7f6b400aac00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f6b30002f30 tx=0x7f6b30006dc0).stop 2020-11-28T21:51:34.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.203+0000 7f6b471ad700 1 -- 172.21.15.162:0/3424836977 shutdown_connections 2020-11-28T21:51:34.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.203+0000 7f6b471ad700 1 --2- 172.21.15.162:0/3424836977 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6b4005cbd0 0x7f6b400aac00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:34.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.203+0000 7f6b471ad700 1 -- 172.21.15.162:0/3424836977 >> 172.21.15.162:0/3424836977 conn(0x7f6b4005b970 msgr2=0x7f6b4005bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:34.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.203+0000 7f6b471ad700 1 -- 172.21.15.162:0/3424836977 shutdown_connections 2020-11-28T21:51:34.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.203+0000 7f6b471ad700 1 -- 172.21.15.162:0/3424836977 wait complete. 2020-11-28T21:51:34.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.203+0000 7f6b471ad700 1 Processor -- start 2020-11-28T21:51:34.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.203+0000 7f6b471ad700 1 -- start start 2020-11-28T21:51:34.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.207+0000 7f6b471ad700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6b4005cbd0 0x7f6b40066620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:34.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.207+0000 7f6b471ad700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6b40060cf0 0x7f6b40152450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:34.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.207+0000 7f6b471ad700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b400af640 0x7f6b4014ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:34.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.207+0000 7f6b471ad700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b40152940 con 0x7f6b4005cbd0 2020-11-28T21:51:34.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.207+0000 7f6b471ad700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b400985f0 con 0x7f6b400af640 2020-11-28T21:51:34.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.207+0000 7f6b471ad700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b40098890 con 0x7f6b40060cf0 2020-11-28T21:51:34.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.207+0000 7f6b469ac700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b400af640 0x7f6b4014ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:34.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.207+0000 7f6b469ac700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b400af640 0x7f6b4014ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40438/0 (socket says 172.21.15.162:40438) 2020-11-28T21:51:34.212 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.207+0000 7f6b469ac700 1 -- 172.21.15.162:0/2789708257 learned_addr learned my addr 172.21.15.162:0/2789708257 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:34.212 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.207+0000 7f6b461ab700 1 --2- 172.21.15.162:0/2789708257 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6b4005cbd0 0x7f6b40066620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:34.212 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.207+0000 7f6b459aa700 1 --2- 172.21.15.162:0/2789708257 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6b40060cf0 0x7f6b40152450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:34.213 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.207+0000 7f6b469ac700 1 -- 172.21.15.162:0/2789708257 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6b40060cf0 msgr2=0x7f6b40152450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:34.213 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.207+0000 7f6b469ac700 1 --2- 172.21.15.162:0/2789708257 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6b40060cf0 0x7f6b40152450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:34.213 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.207+0000 7f6b469ac700 1 -- 172.21.15.162:0/2789708257 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6b4005cbd0 msgr2=0x7f6b40066620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:34.213 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.207+0000 7f6b469ac700 1 --2- 172.21.15.162:0/2789708257 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6b4005cbd0 0x7f6b40066620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:34.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.207+0000 7f6b469ac700 1 -- 172.21.15.162:0/2789708257 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b40098b30 con 0x7f6b400af640 2020-11-28T21:51:34.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.207+0000 7f6b459aa700 1 --2- 172.21.15.162:0/2789708257 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6b40060cf0 0x7f6b40152450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:51:34.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.207+0000 7f6b469ac700 1 --2- 172.21.15.162:0/2789708257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b400af640 0x7f6b4014ec80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f6b38000f00 tx=0x7f6b38007f60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:34.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.207+0000 7f6b2f7fe700 1 -- 172.21.15.162:0/2789708257 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b38006a70 con 0x7f6b400af640 2020-11-28T21:51:34.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.207+0000 7f6b2f7fe700 1 -- 172.21.15.162:0/2789708257 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b38006bd0 con 0x7f6b400af640 2020-11-28T21:51:34.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.207+0000 7f6b2f7fe700 1 -- 172.21.15.162:0/2789708257 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b38007030 con 0x7f6b400af640 2020-11-28T21:51:34.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.207+0000 7f6b471ad700 1 -- 172.21.15.162:0/2789708257 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b40098dd0 con 0x7f6b400af640 2020-11-28T21:51:34.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.207+0000 7f6b471ad700 1 -- 172.21.15.162:0/2789708257 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b40067620 con 0x7f6b400af640 2020-11-28T21:51:34.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.207+0000 7f6b471ad700 1 -- 172.21.15.162:0/2789708257 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b40099270 con 0x7f6b400af640 2020-11-28T21:51:34.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.207+0000 7f6b2f7fe700 1 -- 172.21.15.162:0/2789708257 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6b38019030 con 0x7f6b400af640 2020-11-28T21:51:34.216 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.211+0000 7f6b2f7fe700 1 --2- 172.21.15.162:0/2789708257 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6b20055440 0x7f6b20057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:34.216 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.211+0000 7f6b2f7fe700 1 -- 172.21.15.162:0/2789708257 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6b3806d4d0 con 0x7f6b400af640 2020-11-28T21:51:34.216 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.211+0000 7f6b461ab700 1 --2- 172.21.15.162:0/2789708257 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6b20055440 0x7f6b20057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:34.216 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.211+0000 7f6b461ab700 1 --2- 172.21.15.162:0/2789708257 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6b20055440 0x7f6b20057900 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f6b3c00d000 tx=0x7f6b3c009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:34.217 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.215+0000 7f6b2f7fe700 1 -- 172.21.15.162:0/2789708257 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6b40099270 con 0x7f6b400af640 2020-11-28T21:51:34.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:34.399+0000 7f6b471ad700 1 -- 172.21.15.162:0/2789708257 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6b40099270 con 0x7f6b400af640 2020-11-28T21:51:35.093 INFO:tasks.ceph.mon.b.smithi118.stderr:2020-11-28T21:51:35.094+0000 7ff4e1520700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-11-28T21:51:35.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.103+0000 7f6b2f7fe700 1 -- 172.21.15.162:0/2789708257 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v22) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6b40099270 con 0x7f6b400af640 2020-11-28T21:51:35.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.107+0000 7f6b471ad700 1 -- 172.21.15.162:0/2789708257 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6b20055440 msgr2=0x7f6b20057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:35.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.107+0000 7f6b471ad700 1 --2- 172.21.15.162:0/2789708257 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6b20055440 0x7f6b20057900 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f6b3c00d000 tx=0x7f6b3c009040).stop 2020-11-28T21:51:35.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.107+0000 7f6b471ad700 1 -- 172.21.15.162:0/2789708257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b400af640 msgr2=0x7f6b4014ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:35.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.107+0000 7f6b471ad700 1 --2- 172.21.15.162:0/2789708257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b400af640 0x7f6b4014ec80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f6b38000f00 tx=0x7f6b38007f60).stop 2020-11-28T21:51:35.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.107+0000 7f6b471ad700 1 -- 172.21.15.162:0/2789708257 shutdown_connections 2020-11-28T21:51:35.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.107+0000 7f6b471ad700 1 --2- 172.21.15.162:0/2789708257 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6b20055440 0x7f6b20057900 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:35.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.107+0000 7f6b471ad700 1 --2- 172.21.15.162:0/2789708257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b400af640 0x7f6b4014ec80 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:35.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.107+0000 7f6b471ad700 1 --2- 172.21.15.162:0/2789708257 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6b4005cbd0 0x7f6b40066620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:35.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.107+0000 7f6b471ad700 1 --2- 172.21.15.162:0/2789708257 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6b40060cf0 0x7f6b40152450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:35.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.107+0000 7f6b471ad700 1 -- 172.21.15.162:0/2789708257 >> 172.21.15.162:0/2789708257 conn(0x7f6b4005b970 msgr2=0x7f6b400b4070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:35.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.107+0000 7f6b471ad700 1 -- 172.21.15.162:0/2789708257 shutdown_connections 2020-11-28T21:51:35.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.107+0000 7f6b471ad700 1 -- 172.21.15.162:0/2789708257 wait complete. 2020-11-28T21:51:35.108 INFO:teuthology.orchestra.run.smithi162.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-11-28T21:51:35.122 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:51:35.139 INFO:tasks.ceph.mds.c.smithi162.stdout:starting mds.c at 2020-11-28T21:51:35.233 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.235+0000 7f849759e700 1 Processor -- start 2020-11-28T21:51:35.233 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.235+0000 7f849759e700 1 -- start start 2020-11-28T21:51:35.235 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.239+0000 7f849759e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84980b3290 0x7f84980b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:35.235 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.239+0000 7f849759e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f84981467f0 0x7f84981429d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:35.236 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.239+0000 7f849759e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8498142ec0 0x7f8498143350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:35.236 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.239+0000 7f849759e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84980b3670 con 0x7f84981467f0 2020-11-28T21:51:35.236 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.239+0000 7f849759e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84980b0850 con 0x7f8498142ec0 2020-11-28T21:51:35.236 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.239+0000 7f849759e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84980b0af0 con 0x7f84980b3290 2020-11-28T21:51:35.237 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.239+0000 7f8496d9d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8498142ec0 0x7f8498143350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:35.237 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.239+0000 7f8496d9d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8498142ec0 0x7f8498143350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40456/0 (socket says 172.21.15.162:40456) 2020-11-28T21:51:35.237 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.239+0000 7f8496d9d700 1 -- 172.21.15.162:0/2576930267 learned_addr learned my addr 172.21.15.162:0/2576930267 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:35.237 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.239+0000 7f849659c700 1 --2- 172.21.15.162:0/2576930267 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84980b3290 0x7f84980b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:35.238 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.239+0000 7f8495d9b700 1 --2- 172.21.15.162:0/2576930267 >> v2:172.21.15.118:3300/0 conn(0x7f84981467f0 0x7f84981429d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:35.238 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.239+0000 7f8496d9d700 1 -- 172.21.15.162:0/2576930267 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84980b3290 msgr2=0x7f84980b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:51:35.238 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.239+0000 7f8496d9d700 1 --2- 172.21.15.162:0/2576930267 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84980b3290 0x7f84980b14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:35.238 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.239+0000 7f8496d9d700 1 -- 172.21.15.162:0/2576930267 >> v2:172.21.15.118:3300/0 conn(0x7f84981467f0 msgr2=0x7f84981429d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:51:35.239 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.239+0000 7f8496d9d700 1 --2- 172.21.15.162:0/2576930267 >> v2:172.21.15.118:3300/0 conn(0x7f84981467f0 0x7f84981429d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:35.239 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.239+0000 7f8496d9d700 1 -- 172.21.15.162:0/2576930267 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84980b0d90 con 0x7f8498142ec0 2020-11-28T21:51:35.239 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.239+0000 7f8496d9d700 1 --2- 172.21.15.162:0/2576930267 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8498142ec0 0x7f8498143350 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f84900032b0 tx=0x7f8490009270).ready entity=mon.1 client_cookie=ddd26cee4c94fc77 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:35.239 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.239+0000 7f849559a700 1 -- 172.21.15.162:0/2576930267 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f849000bc00 con 0x7f8498142ec0 2020-11-28T21:51:35.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.239+0000 7f849559a700 1 -- 172.21.15.162:0/2576930267 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f849000bd60 con 0x7f8498142ec0 2020-11-28T21:51:35.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.239+0000 7f849559a700 1 -- 172.21.15.162:0/2576930267 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8490011400 con 0x7f8498142ec0 2020-11-28T21:51:35.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.239+0000 7f849759e700 1 -- 172.21.15.162:0/2576930267 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8498142ec0 msgr2=0x7f8498143350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:35.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.239+0000 7f849759e700 1 --2- 172.21.15.162:0/2576930267 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8498142ec0 0x7f8498143350 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f84900032b0 tx=0x7f8490009270).stop 2020-11-28T21:51:35.243 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.239+0000 7f849759e700 1 -- 172.21.15.162:0/2576930267 shutdown_connections 2020-11-28T21:51:35.243 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.239+0000 7f849759e700 1 --2- 172.21.15.162:0/2576930267 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8498142ec0 0x7f8498143350 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:35.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.239+0000 7f849759e700 1 --2- 172.21.15.162:0/2576930267 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84980b3290 0x7f84980b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:35.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.239+0000 7f849759e700 1 --2- 172.21.15.162:0/2576930267 >> v2:172.21.15.118:3300/0 conn(0x7f84981467f0 0x7f84981429d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:35.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.239+0000 7f849759e700 1 -- 172.21.15.162:0/2576930267 >> 172.21.15.162:0/2576930267 conn(0x7f84980b6870 msgr2=0x7f84980b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:35.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.239+0000 7f849759e700 1 -- 172.21.15.162:0/2576930267 shutdown_connections 2020-11-28T21:51:35.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.239+0000 7f849759e700 1 -- 172.21.15.162:0/2576930267 wait complete. 2020-11-28T21:51:35.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.239+0000 7f849759e700 1 Processor -- start 2020-11-28T21:51:35.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.239+0000 7f849759e700 1 -- start start 2020-11-28T21:51:35.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.243+0000 7f849759e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84980b3290 0x7f849813ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:35.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.243+0000 7f849759e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8498142ec0 0x7f84980a95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:35.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.243+0000 7f849759e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84981467f0 0x7f8498144b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:35.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.243+0000 7f849759e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8498149980 con 0x7f8498142ec0 2020-11-28T21:51:35.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.243+0000 7f849759e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84980667f0 con 0x7f84981467f0 2020-11-28T21:51:35.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.243+0000 7f849759e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8498148a40 con 0x7f84980b3290 2020-11-28T21:51:35.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.243+0000 7f8496d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84981467f0 0x7f8498144b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:35.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.243+0000 7f8496d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84981467f0 0x7f8498144b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40464/0 (socket says 172.21.15.162:40464) 2020-11-28T21:51:35.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.243+0000 7f8496d9d700 1 -- 172.21.15.162:0/2765656453 learned_addr learned my addr 172.21.15.162:0/2765656453 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:35.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.243+0000 7f849659c700 1 --2- 172.21.15.162:0/2765656453 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84980b3290 0x7f849813ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:35.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.243+0000 7f8495d9b700 1 --2- 172.21.15.162:0/2765656453 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8498142ec0 0x7f84980a95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:35.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.243+0000 7f8496d9d700 1 -- 172.21.15.162:0/2765656453 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84980b3290 msgr2=0x7f849813ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:35.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.243+0000 7f8496d9d700 1 --2- 172.21.15.162:0/2765656453 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84980b3290 0x7f849813ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:35.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.243+0000 7f8496d9d700 1 -- 172.21.15.162:0/2765656453 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8498142ec0 msgr2=0x7f84980a95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:35.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.243+0000 7f8496d9d700 1 --2- 172.21.15.162:0/2765656453 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8498142ec0 0x7f84980a95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:35.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.243+0000 7f8496d9d700 1 -- 172.21.15.162:0/2765656453 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8498147b20 con 0x7f84981467f0 2020-11-28T21:51:35.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.243+0000 7f849659c700 1 --2- 172.21.15.162:0/2765656453 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84980b3290 0x7f849813ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:51:35.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.243+0000 7f8495d9b700 1 --2- 172.21.15.162:0/2765656453 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8498142ec0 0x7f84980a95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:51:35.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.243+0000 7f8496d9d700 1 --2- 172.21.15.162:0/2765656453 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84981467f0 0x7f8498144b90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f84900032b0 tx=0x7f8490016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:35.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.243+0000 7f84877fe700 1 -- 172.21.15.162:0/2765656453 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f849001e610 con 0x7f84981467f0 2020-11-28T21:51:35.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.243+0000 7f84877fe700 1 -- 172.21.15.162:0/2765656453 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f849001e770 con 0x7f84981467f0 2020-11-28T21:51:35.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.243+0000 7f849759e700 1 -- 172.21.15.162:0/2765656453 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8498147dc0 con 0x7f84981467f0 2020-11-28T21:51:35.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.243+0000 7f84877fe700 1 -- 172.21.15.162:0/2765656453 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f849001eb10 con 0x7f84981467f0 2020-11-28T21:51:35.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.243+0000 7f849759e700 1 -- 172.21.15.162:0/2765656453 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8498064ae0 con 0x7f84981467f0 2020-11-28T21:51:35.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.243+0000 7f849759e700 1 -- 172.21.15.162:0/2765656453 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8498147300 con 0x7f84981467f0 2020-11-28T21:51:35.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.243+0000 7f84877fe700 1 -- 172.21.15.162:0/2765656453 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f849000f070 con 0x7f84981467f0 2020-11-28T21:51:35.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.243+0000 7f84877fe700 1 --2- 172.21.15.162:0/2765656453 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8474055330 0x7f84740577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:35.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.243+0000 7f84877fe700 1 -- 172.21.15.162:0/2765656453 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f849006f730 con 0x7f84981467f0 2020-11-28T21:51:35.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.247+0000 7f849659c700 1 --2- 172.21.15.162:0/2765656453 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8474055330 0x7f84740577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:35.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.247+0000 7f849659c700 1 --2- 172.21.15.162:0/2765656453 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8474055330 0x7f84740577f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f848800c000 tx=0x7f8488008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:35.253 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.251+0000 7f84877fe700 1 -- 172.21.15.162:0/2765656453 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f84900459e0 con 0x7f84981467f0 2020-11-28T21:51:35.433 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.435+0000 7f849759e700 1 -- 172.21.15.162:0/2765656453 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8498148c00 con 0x7f84981467f0 2020-11-28T21:51:35.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.435+0000 7f84877fe700 1 -- 172.21.15.162:0/2765656453 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+2137 (secure 0 0 0) 0x7f8490015090 con 0x7f84981467f0 2020-11-28T21:51:35.434 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:51:35.434 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":22,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/1245801281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1245801281},{"type":"v1","addr":"172.21.15.118:6833","nonce":1245801281}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":18},{"gid":5536,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6835/3629429916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3629429916},{"type":"v1","addr":"172.21.15.162:6835","nonce":3629429916}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:50:05.655172+0000","modified":"2020-11-28T21:51:35.098469+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-28T21:51:35.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.439+0000 7f849759e700 1 -- 172.21.15.162:0/2765656453 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8474055330 msgr2=0x7f84740577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:35.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.439+0000 7f849759e700 1 --2- 172.21.15.162:0/2765656453 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8474055330 0x7f84740577f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f848800c000 tx=0x7f8488008040).stop 2020-11-28T21:51:35.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.439+0000 7f849759e700 1 -- 172.21.15.162:0/2765656453 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84981467f0 msgr2=0x7f8498144b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:35.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.439+0000 7f849759e700 1 --2- 172.21.15.162:0/2765656453 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84981467f0 0x7f8498144b90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f84900032b0 tx=0x7f8490016000).stop 2020-11-28T21:51:35.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.439+0000 7f849759e700 1 -- 172.21.15.162:0/2765656453 shutdown_connections 2020-11-28T21:51:35.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.439+0000 7f849759e700 1 --2- 172.21.15.162:0/2765656453 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8474055330 0x7f84740577f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:35.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.439+0000 7f849759e700 1 --2- 172.21.15.162:0/2765656453 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84980b3290 0x7f849813ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:35.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.439+0000 7f849759e700 1 --2- 172.21.15.162:0/2765656453 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84981467f0 0x7f8498144b90 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:35.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.439+0000 7f849759e700 1 --2- 172.21.15.162:0/2765656453 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8498142ec0 0x7f84980a95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:35.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.439+0000 7f849759e700 1 -- 172.21.15.162:0/2765656453 >> 172.21.15.162:0/2765656453 conn(0x7f84980b6870 msgr2=0x7f8498141710 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:35.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.439+0000 7f849759e700 1 -- 172.21.15.162:0/2765656453 shutdown_connections 2020-11-28T21:51:35.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.439+0000 7f849759e700 1 -- 172.21.15.162:0/2765656453 wait complete. 2020-11-28T21:51:35.439 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 22 2020-11-28T21:51:35.448 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2020-11-28T21:51:35.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.611+0000 7fe2c1074700 1 Processor -- start 2020-11-28T21:51:35.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.611+0000 7fe2c1074700 1 -- start start 2020-11-28T21:51:35.609 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.611+0000 7fe2c1074700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe2bc060cb0 0x7fe2bc142980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:35.609 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.611+0000 7fe2c1074700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe2bc0b3a60 0x7fe2bc13fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:35.609 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.611+0000 7fe2c1074700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2bc05f380 con 0x7fe2bc0b5bf0 2020-11-28T21:51:35.610 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.611+0000 7fe2c1074700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2bc143930 con 0x7fe2bc0b3a60 2020-11-28T21:51:35.610 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.611+0000 7fe2c1074700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc05f5b0 con 0x7fe2bc060cb0 2020-11-28T21:51:35.610 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.611+0000 7fe2bbfff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fe2bc0b5bf0 0x7fe2bc060230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:47312/0 (socket says 172.21.15.162:47312) 2020-11-28T21:51:35.610 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.611+0000 7fe2bbfff700 1 -- 172.21.15.162:0/3350386684 learned_addr learned my addr 172.21.15.162:0/3350386684 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:51:35.611 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.611+0000 7fe2c0873700 1 --2- 172.21.15.162:0/3350386684 >> v2:172.21.15.118:3300/0 conn(0x7fe2bc0b3a60 0x7fe2bc13fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:35.611 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.611+0000 7fe2bb7fe700 1 --2- 172.21.15.162:0/3350386684 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe2bc060cb0 0x7fe2bc142980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:35.611 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.611+0000 7fe2baffd700 1 -- 172.21.15.162:0/3350386684 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1199287994 0 0) 0x7fe2bc05f380 con 0x7fe2bc0b5bf0 2020-11-28T21:51:35.611 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.611+0000 7fe2baffd700 1 -- 172.21.15.162:0/3350386684 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2a8003230 con 0x7fe2bc0b5bf0 2020-11-28T21:51:35.615 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.611+0000 7fe2baffd700 1 -- 172.21.15.162:0/3350386684 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3494505505 0 0) 0x7fe2a8003230 con 0x7fe2bc0b5bf0 2020-11-28T21:51:35.615 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.611+0000 7fe2baffd700 1 -- 172.21.15.162:0/3350386684 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe2bc060cb0 msgr2=0x7fe2bc142980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:51:35.615 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.611+0000 7fe2baffd700 1 --2- 172.21.15.162:0/3350386684 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe2bc060cb0 0x7fe2bc142980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:35.616 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.611+0000 7fe2baffd700 1 -- 172.21.15.162:0/3350386684 >> v2:172.21.15.118:3300/0 conn(0x7fe2bc0b3a60 msgr2=0x7fe2bc13fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:51:35.616 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.611+0000 7fe2baffd700 1 --2- 172.21.15.162:0/3350386684 >> v2:172.21.15.118:3300/0 conn(0x7fe2bc0b3a60 0x7fe2bc13fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:35.616 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.611+0000 7fe2baffd700 1 -- 172.21.15.162:0/3350386684 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2bc05f850 con 0x7fe2bc0b5bf0 2020-11-28T21:51:35.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.611+0000 7fe2baffd700 1 -- 172.21.15.162:0/3350386684 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7fe2ac0016d0 con 0x7fe2bc0b5bf0 2020-11-28T21:51:35.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.611+0000 7fe2baffd700 1 -- 172.21.15.162:0/3350386684 >> v1:172.21.15.162:6789/0 conn(0x7fe2bc0b5bf0 legacy=0x7fe2bc060230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:35.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.611+0000 7fe2baffd700 1 --2- 172.21.15.162:0/3350386684 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe2bc060cb0 0x7fe2a8003720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:35.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.611+0000 7fe2baffd700 1 --2- 172.21.15.162:0/3350386684 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2a8003c30 0x7fe2a80044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:35.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.611+0000 7fe2baffd700 1 --2- 172.21.15.162:0/3350386684 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2a80049c0 0x7fe2a8004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:35.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.611+0000 7fe2baffd700 1 -- 172.21.15.162:0/3350386684 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc143930 con 0x7fe2a80049c0 2020-11-28T21:51:35.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.611+0000 7fe2baffd700 1 -- 172.21.15.162:0/3350386684 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc05f5b0 con 0x7fe2a8003c30 2020-11-28T21:51:35.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.611+0000 7fe2baffd700 1 -- 172.21.15.162:0/3350386684 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a80041f0 con 0x7fe2bc060cb0 2020-11-28T21:51:35.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.615+0000 7fe2bb7fe700 1 --2- 172.21.15.162:0/3350386684 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2a8003c30 0x7fe2a80044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:35.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.615+0000 7fe2bbfff700 1 --2- 172.21.15.162:0/3350386684 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe2bc060cb0 0x7fe2a8003720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:35.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.615+0000 7fe2c0873700 1 --2- 172.21.15.162:0/3350386684 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2a80049c0 0x7fe2a8004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:35.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.615+0000 7fe2bb7fe700 1 -- 172.21.15.162:0/3350386684 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe2bc060cb0 msgr2=0x7fe2a8003720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:51:35.620 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.615+0000 7fe2bb7fe700 1 --2- 172.21.15.162:0/3350386684 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe2bc060cb0 0x7fe2a8003720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:35.620 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.615+0000 7fe2bb7fe700 1 -- 172.21.15.162:0/3350386684 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2a80049c0 msgr2=0x7fe2a8004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:51:35.620 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.615+0000 7fe2bb7fe700 1 --2- 172.21.15.162:0/3350386684 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2a80049c0 0x7fe2a8004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:35.620 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.615+0000 7fe2bb7fe700 1 -- 172.21.15.162:0/3350386684 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a8003230 con 0x7fe2a8003c30 2020-11-28T21:51:35.621 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.615+0000 7fe2c0873700 1 --2- 172.21.15.162:0/3350386684 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2a80049c0 0x7fe2a8004e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:51:35.621 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.615+0000 7fe2bb7fe700 1 --2- 172.21.15.162:0/3350386684 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2a8003c30 0x7fe2a80044d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7fe2b000c000 tx=0x7fe2b0008040).ready entity=mon.1 client_cookie=35b7939af799b634 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:35.621 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.615+0000 7fe2baffd700 1 -- 172.21.15.162:0/3350386684 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2b000e9d0 con 0x7fe2a8003c30 2020-11-28T21:51:35.621 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.615+0000 7fe2baffd700 1 -- 172.21.15.162:0/3350386684 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b000eb30 con 0x7fe2a8003c30 2020-11-28T21:51:35.622 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.615+0000 7fe2baffd700 1 -- 172.21.15.162:0/3350386684 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2b000ee90 con 0x7fe2a8003c30 2020-11-28T21:51:35.622 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.615+0000 7fe2c1074700 1 -- 172.21.15.162:0/3350386684 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2a8003c30 msgr2=0x7fe2a80044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:35.622 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.615+0000 7fe2c1074700 1 --2- 172.21.15.162:0/3350386684 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2a8003c30 0x7fe2a80044d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7fe2b000c000 tx=0x7fe2b0008040).stop 2020-11-28T21:51:35.622 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.615+0000 7fe2c0873700 1 -- 172.21.15.162:0/3350386684 reap_dead start 2020-11-28T21:51:35.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.615+0000 7fe2c1074700 1 -- 172.21.15.162:0/3350386684 shutdown_connections 2020-11-28T21:51:35.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.615+0000 7fe2c1074700 1 -- 172.21.15.162:0/3350386684 >> 172.21.15.162:0/3350386684 conn(0x7fe2bc05b970 msgr2=0x7fe2bc05bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:35.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.615+0000 7fe2c1074700 1 -- 172.21.15.162:0/3350386684 shutdown_connections 2020-11-28T21:51:35.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.615+0000 7fe2c1074700 1 -- 172.21.15.162:0/3350386684 wait complete. 2020-11-28T21:51:35.624 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.615+0000 7fe2c1074700 1 Processor -- start 2020-11-28T21:51:35.624 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.615+0000 7fe2c1074700 1 -- start start 2020-11-28T21:51:35.624 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.619+0000 7fe2c1074700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe2bc145df0 0x7fe2bc13ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:35.624 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.619+0000 7fe2c1074700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2bc146a30 0x7fe2bc13f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:35.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.619+0000 7fe2c1074700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2bc0acee0 0x7fe2bc0ad360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:35.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.619+0000 7fe2c1074700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc145530 con 0x7fe2bc146a30 2020-11-28T21:51:35.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.619+0000 7fe2c1074700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc1461d0 con 0x7fe2bc0acee0 2020-11-28T21:51:35.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.619+0000 7fe2c1074700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc145760 con 0x7fe2bc145df0 2020-11-28T21:51:35.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.619+0000 7fe2c0873700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2bc0acee0 0x7fe2bc0ad360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:35.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.619+0000 7fe2c0873700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2bc0acee0 0x7fe2bc0ad360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40484/0 (socket says 172.21.15.162:40484) 2020-11-28T21:51:35.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.619+0000 7fe2c0873700 1 -- 172.21.15.162:0/800845151 learned_addr learned my addr 172.21.15.162:0/800845151 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:35.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.619+0000 7fe2bbfff700 1 --2- 172.21.15.162:0/800845151 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe2bc145df0 0x7fe2bc13ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:35.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.619+0000 7fe2bb7fe700 1 --2- 172.21.15.162:0/800845151 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2bc146a30 0x7fe2bc13f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:35.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.619+0000 7fe2c0873700 1 -- 172.21.15.162:0/800845151 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe2bc145df0 msgr2=0x7fe2bc13ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:35.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.619+0000 7fe2c0873700 1 --2- 172.21.15.162:0/800845151 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe2bc145df0 0x7fe2bc13ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:35.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.619+0000 7fe2c0873700 1 -- 172.21.15.162:0/800845151 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2bc146a30 msgr2=0x7fe2bc13f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:35.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.619+0000 7fe2c0873700 1 --2- 172.21.15.162:0/800845151 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2bc146a30 0x7fe2bc13f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:35.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.619+0000 7fe2c0873700 1 -- 172.21.15.162:0/800845151 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2bc1490e0 con 0x7fe2bc0acee0 2020-11-28T21:51:35.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.619+0000 7fe2c0873700 1 --2- 172.21.15.162:0/800845151 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2bc0acee0 0x7fe2bc0ad360 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7fe2bc144f60 tx=0x7fe2b4008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:35.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.619+0000 7fe2b97fa700 1 -- 172.21.15.162:0/800845151 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2b4017070 con 0x7fe2bc0acee0 2020-11-28T21:51:35.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.619+0000 7fe2b97fa700 1 -- 172.21.15.162:0/800845151 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b4011790 con 0x7fe2bc0acee0 2020-11-28T21:51:35.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.619+0000 7fe2c1074700 1 -- 172.21.15.162:0/800845151 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2bc149380 con 0x7fe2bc0acee0 2020-11-28T21:51:35.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.619+0000 7fe2b97fa700 1 -- 172.21.15.162:0/800845151 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2b4011b30 con 0x7fe2bc0acee0 2020-11-28T21:51:35.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.619+0000 7fe2c1074700 1 -- 172.21.15.162:0/800845151 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2bc0afb80 con 0x7fe2bc0acee0 2020-11-28T21:51:35.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.619+0000 7fe2c1074700 1 -- 172.21.15.162:0/800845151 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2bc146860 con 0x7fe2bc0acee0 2020-11-28T21:51:35.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.623+0000 7fe2b97fa700 1 -- 172.21.15.162:0/800845151 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe2b4011c90 con 0x7fe2bc0acee0 2020-11-28T21:51:35.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.623+0000 7fe2b97fa700 1 --2- 172.21.15.162:0/800845151 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe29c0553b0 0x7fe29c057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:35.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.623+0000 7fe2b97fa700 1 -- 172.21.15.162:0/800845151 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fe2b406e2a0 con 0x7fe2bc0acee0 2020-11-28T21:51:35.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.623+0000 7fe2bbfff700 1 --2- 172.21.15.162:0/800845151 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe29c0553b0 0x7fe29c057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:35.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.623+0000 7fe2bbfff700 1 --2- 172.21.15.162:0/800845151 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe29c0553b0 0x7fe29c057870 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7fe2ac00d000 tx=0x7fe2ac007040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:35.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.627+0000 7fe2b97fa700 1 -- 172.21.15.162:0/800845151 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe2b4044550 con 0x7fe2bc0acee0 2020-11-28T21:51:35.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:35.811+0000 7fe2c1074700 1 -- 172.21.15.162:0/800845151 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe2bc057050 con 0x7fe2bc0acee0 2020-11-28T21:51:36.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.811+0000 7fe2b97fa700 1 -- 172.21.15.162:0/800845151 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe2b4047b70 con 0x7fe2bc0acee0 2020-11-28T21:51:36.812 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.815+0000 7fe2c1074700 1 -- 172.21.15.162:0/800845151 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe29c0553b0 msgr2=0x7fe29c057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:36.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.815+0000 7fe2c1074700 1 --2- 172.21.15.162:0/800845151 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe29c0553b0 0x7fe29c057870 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7fe2ac00d000 tx=0x7fe2ac007040).stop 2020-11-28T21:51:36.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.815+0000 7fe2c1074700 1 -- 172.21.15.162:0/800845151 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2bc0acee0 msgr2=0x7fe2bc0ad360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:36.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.815+0000 7fe2c1074700 1 --2- 172.21.15.162:0/800845151 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2bc0acee0 0x7fe2bc0ad360 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7fe2bc144f60 tx=0x7fe2b4008040).stop 2020-11-28T21:51:36.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.815+0000 7fe2c1074700 1 -- 172.21.15.162:0/800845151 shutdown_connections 2020-11-28T21:51:36.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.815+0000 7fe2c1074700 1 --2- 172.21.15.162:0/800845151 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe29c0553b0 0x7fe29c057870 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:36.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.815+0000 7fe2c1074700 1 --2- 172.21.15.162:0/800845151 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe2bc145df0 0x7fe2bc13ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:36.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.815+0000 7fe2c1074700 1 --2- 172.21.15.162:0/800845151 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2bc0acee0 0x7fe2bc0ad360 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:36.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.815+0000 7fe2c1074700 1 --2- 172.21.15.162:0/800845151 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2bc146a30 0x7fe2bc13f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:36.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.815+0000 7fe2c1074700 1 -- 172.21.15.162:0/800845151 >> 172.21.15.162:0/800845151 conn(0x7fe2bc05b970 msgr2=0x7fe2bc0b90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:36.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.815+0000 7fe2c1074700 1 -- 172.21.15.162:0/800845151 shutdown_connections 2020-11-28T21:51:36.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.815+0000 7fe2c1074700 1 -- 172.21.15.162:0/800845151 wait complete. 2020-11-28T21:51:36.831 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-11-28T21:51:36.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.955+0000 7fc6c2e8e700 1 Processor -- start 2020-11-28T21:51:36.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.955+0000 7fc6c2e8e700 1 -- start start 2020-11-28T21:51:36.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.959+0000 7fc6c2e8e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fc6bc0b9ce0 0x7fc6bc1433c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:36.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.959+0000 7fc6c2e8e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc6bc0605c0 0x7fc6bc067610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:36.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.959+0000 7fc6c2e8e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6bc143950 con 0x7fc6bc0b3c30 2020-11-28T21:51:36.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.959+0000 7fc6c2e8e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6bc0ba0c0 con 0x7fc6bc0b9ce0 2020-11-28T21:51:36.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.959+0000 7fc6c2e8e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6bc066c30 con 0x7fc6bc0605c0 2020-11-28T21:51:36.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.959+0000 7fc6c142b700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fc6bc0b3c30 0x7fc6bc13fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:47332/0 (socket says 172.21.15.162:47332) 2020-11-28T21:51:36.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.959+0000 7fc6c142b700 1 -- 172.21.15.162:0/862557150 learned_addr learned my addr 172.21.15.162:0/862557150 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:51:36.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.959+0000 7fc6bbfff700 1 --2- 172.21.15.162:0/862557150 >> v2:172.21.15.162:3300/0 conn(0x7fc6bc0605c0 0x7fc6bc067610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:36.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.959+0000 7fc6c0c2a700 1 --2- 172.21.15.162:0/862557150 >> v2:172.21.15.118:3300/0 conn(0x7fc6bc0b9ce0 0x7fc6bc1433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:36.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.959+0000 7fc6bb7fe700 1 -- 172.21.15.162:0/862557150 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2060139787 0 0) 0x7fc6bc143950 con 0x7fc6bc0b3c30 2020-11-28T21:51:36.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.959+0000 7fc6bb7fe700 1 -- 172.21.15.162:0/862557150 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc69c003230 con 0x7fc6bc0b3c30 2020-11-28T21:51:36.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.959+0000 7fc6bbfff700 1 -- 172.21.15.162:0/862557150 >> v2:172.21.15.118:3300/0 conn(0x7fc6bc0b9ce0 msgr2=0x7fc6bc1433c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:51:36.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.959+0000 7fc6bbfff700 1 --2- 172.21.15.162:0/862557150 >> v2:172.21.15.118:3300/0 conn(0x7fc6bc0b9ce0 0x7fc6bc1433c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:36.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.959+0000 7fc6bbfff700 1 -- 172.21.15.162:0/862557150 >> v1:172.21.15.162:6789/0 conn(0x7fc6bc0b3c30 legacy=0x7fc6bc13fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:36.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.959+0000 7fc6bbfff700 1 -- 172.21.15.162:0/862557150 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6bc066ed0 con 0x7fc6bc0605c0 2020-11-28T21:51:36.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.959+0000 7fc6bbfff700 1 --2- 172.21.15.162:0/862557150 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6bc0605c0 0x7fc6bc067610 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7fc6a4002f30 tx=0x7fc6a4006dc0).ready entity=mon.1 client_cookie=20e2e4f91cc24002 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:36.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.959+0000 7fc6bb7fe700 1 -- 172.21.15.162:0/862557150 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6a40096f0 con 0x7fc6bc0605c0 2020-11-28T21:51:36.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.959+0000 7fc6bb7fe700 1 -- 172.21.15.162:0/862557150 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a4009850 con 0x7fc6bc0605c0 2020-11-28T21:51:36.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.959+0000 7fc6bb7fe700 1 -- 172.21.15.162:0/862557150 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6a4009bb0 con 0x7fc6bc0605c0 2020-11-28T21:51:36.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.959+0000 7fc6c2e8e700 1 -- 172.21.15.162:0/862557150 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6bc0605c0 msgr2=0x7fc6bc067610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:36.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.959+0000 7fc6c2e8e700 1 --2- 172.21.15.162:0/862557150 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6bc0605c0 0x7fc6bc067610 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7fc6a4002f30 tx=0x7fc6a4006dc0).stop 2020-11-28T21:51:36.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.959+0000 7fc6c2e8e700 1 -- 172.21.15.162:0/862557150 shutdown_connections 2020-11-28T21:51:36.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.959+0000 7fc6c2e8e700 1 --2- 172.21.15.162:0/862557150 >> v2:172.21.15.118:3300/0 conn(0x7fc6bc0b9ce0 0x7fc6bc1433c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:36.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.959+0000 7fc6c2e8e700 1 --2- 172.21.15.162:0/862557150 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6bc0605c0 0x7fc6bc067610 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:36.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.959+0000 7fc6c2e8e700 1 -- 172.21.15.162:0/862557150 >> 172.21.15.162:0/862557150 conn(0x7fc6bc05c430 msgr2=0x7fc6bc0b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:36.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.959+0000 7fc6c2e8e700 1 -- 172.21.15.162:0/862557150 shutdown_connections 2020-11-28T21:51:36.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.959+0000 7fc6c2e8e700 1 -- 172.21.15.162:0/862557150 wait complete. 2020-11-28T21:51:36.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.963+0000 7fc6c2e8e700 1 Processor -- start 2020-11-28T21:51:36.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.963+0000 7fc6c2e8e700 1 -- start start 2020-11-28T21:51:36.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.963+0000 7fc6c2e8e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6bc0605c0 0x7fc6bc0af500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:36.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.963+0000 7fc6c2e8e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc6bc0b3c30 0x7fc6bc13ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:36.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.963+0000 7fc6c2e8e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc6bc0b9ce0 0x7fc6bc13f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:36.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.963+0000 7fc6c2e8e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc13f8e0 con 0x7fc6bc0b9ce0 2020-11-28T21:51:36.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.963+0000 7fc6c2e8e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc0b0590 con 0x7fc6bc0605c0 2020-11-28T21:51:36.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.963+0000 7fc6c2e8e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc0b1050 con 0x7fc6bc0b3c30 2020-11-28T21:51:36.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.963+0000 7fc6c0c2a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6bc0605c0 0x7fc6bc0af500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:36.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.963+0000 7fc6c0c2a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6bc0605c0 0x7fc6bc0af500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40494/0 (socket says 172.21.15.162:40494) 2020-11-28T21:51:36.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.963+0000 7fc6c0c2a700 1 -- 172.21.15.162:0/938303701 learned_addr learned my addr 172.21.15.162:0/938303701 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:36.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.963+0000 7fc6c142b700 1 --2- 172.21.15.162:0/938303701 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc6bc0b9ce0 0x7fc6bc13f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:36.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.963+0000 7fc6bbfff700 1 --2- 172.21.15.162:0/938303701 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc6bc0b3c30 0x7fc6bc13ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:36.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.963+0000 7fc6c0c2a700 1 -- 172.21.15.162:0/938303701 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc6bc0b3c30 msgr2=0x7fc6bc13ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:36.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.963+0000 7fc6c0c2a700 1 --2- 172.21.15.162:0/938303701 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc6bc0b3c30 0x7fc6bc13ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:36.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.963+0000 7fc6c0c2a700 1 -- 172.21.15.162:0/938303701 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc6bc0b9ce0 msgr2=0x7fc6bc13f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:36.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.963+0000 7fc6c0c2a700 1 --2- 172.21.15.162:0/938303701 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc6bc0b9ce0 0x7fc6bc13f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:36.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.963+0000 7fc6c0c2a700 1 -- 172.21.15.162:0/938303701 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6bc0b07c0 con 0x7fc6bc0605c0 2020-11-28T21:51:36.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.963+0000 7fc6c142b700 1 --2- 172.21.15.162:0/938303701 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc6bc0b9ce0 0x7fc6bc13f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:51:36.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.963+0000 7fc6c0c2a700 1 --2- 172.21.15.162:0/938303701 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6bc0605c0 0x7fc6bc0af500 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7fc6ac00c000 tx=0x7fc6ac006d60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:36.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.963+0000 7fc6b9ffb700 1 -- 172.21.15.162:0/938303701 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6ac00ca10 con 0x7fc6bc0605c0 2020-11-28T21:51:36.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.963+0000 7fc6b9ffb700 1 -- 172.21.15.162:0/938303701 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6ac00cb70 con 0x7fc6bc0605c0 2020-11-28T21:51:36.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.963+0000 7fc6c2e8e700 1 -- 172.21.15.162:0/938303701 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6bc0b0c60 con 0x7fc6bc0605c0 2020-11-28T21:51:36.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.963+0000 7fc6b9ffb700 1 -- 172.21.15.162:0/938303701 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6ac0123f0 con 0x7fc6bc0605c0 2020-11-28T21:51:36.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.963+0000 7fc6c2e8e700 1 -- 172.21.15.162:0/938303701 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6bc145ff0 con 0x7fc6bc0605c0 2020-11-28T21:51:36.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.963+0000 7fc6c2e8e700 1 -- 172.21.15.162:0/938303701 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6bc1403b0 con 0x7fc6bc0605c0 2020-11-28T21:51:36.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.967+0000 7fc6b9ffb700 1 -- 172.21.15.162:0/938303701 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc6ac012550 con 0x7fc6bc0605c0 2020-11-28T21:51:36.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.967+0000 7fc6b9ffb700 1 --2- 172.21.15.162:0/938303701 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc6a0055330 0x7fc6a00577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:36.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.967+0000 7fc6bbfff700 1 --2- 172.21.15.162:0/938303701 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc6a0055330 0x7fc6a00577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:36.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.967+0000 7fc6b9ffb700 1 -- 172.21.15.162:0/938303701 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fc6ac06e1b0 con 0x7fc6bc0605c0 2020-11-28T21:51:36.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.967+0000 7fc6bbfff700 1 --2- 172.21.15.162:0/938303701 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc6a0055330 0x7fc6a00577f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7fc6a4019000 tx=0x7fc6a4015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:36.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:36.971+0000 7fc6b9ffb700 1 -- 172.21.15.162:0/938303701 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc6ac0443a0 con 0x7fc6bc0605c0 2020-11-28T21:51:37.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.111+0000 7fc6c2e8e700 1 -- 172.21.15.162:0/938303701 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc6bc057050 con 0x7fc6bc0605c0 2020-11-28T21:51:37.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.823+0000 7fc6b9ffb700 1 -- 172.21.15.162:0/938303701 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v58) v1 ==== 185+0+0 (secure 0 0 0) 0x7fc6ac0479c0 con 0x7fc6bc0605c0 2020-11-28T21:51:37.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.823+0000 7fc6c2e8e700 1 -- 172.21.15.162:0/938303701 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc6a0055330 msgr2=0x7fc6a00577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:37.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.823+0000 7fc6c2e8e700 1 --2- 172.21.15.162:0/938303701 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc6a0055330 0x7fc6a00577f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7fc6a4019000 tx=0x7fc6a4015000).stop 2020-11-28T21:51:37.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.823+0000 7fc6c2e8e700 1 -- 172.21.15.162:0/938303701 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6bc0605c0 msgr2=0x7fc6bc0af500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:37.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.823+0000 7fc6c2e8e700 1 --2- 172.21.15.162:0/938303701 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6bc0605c0 0x7fc6bc0af500 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7fc6ac00c000 tx=0x7fc6ac006d60).stop 2020-11-28T21:51:37.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.823+0000 7fc6c2e8e700 1 -- 172.21.15.162:0/938303701 shutdown_connections 2020-11-28T21:51:37.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.823+0000 7fc6c2e8e700 1 --2- 172.21.15.162:0/938303701 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc6a0055330 0x7fc6a00577f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:37.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.827+0000 7fc6c2e8e700 1 --2- 172.21.15.162:0/938303701 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc6bc0b9ce0 0x7fc6bc13f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:37.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.827+0000 7fc6c2e8e700 1 --2- 172.21.15.162:0/938303701 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6bc0605c0 0x7fc6bc0af500 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:37.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.827+0000 7fc6c2e8e700 1 --2- 172.21.15.162:0/938303701 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc6bc0b3c30 0x7fc6bc13ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:37.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.827+0000 7fc6c2e8e700 1 -- 172.21.15.162:0/938303701 >> 172.21.15.162:0/938303701 conn(0x7fc6bc05c430 msgr2=0x7fc6bc141830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:37.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.827+0000 7fc6c2e8e700 1 -- 172.21.15.162:0/938303701 shutdown_connections 2020-11-28T21:51:37.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.827+0000 7fc6c2e8e700 1 -- 172.21.15.162:0/938303701 wait complete. 2020-11-28T21:51:37.825 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_metadata' removed 2020-11-28T21:51:37.841 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-11-28T21:51:37.920 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:51:37.923 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:51:37.925 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:51:37.927 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:51:37.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.959+0000 7fab78de3700 1 Processor -- start 2020-11-28T21:51:37.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.959+0000 7fab78de3700 1 -- start start 2020-11-28T21:51:37.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.959+0000 7fab78de3700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fab740b3950 0x7fab740b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:37.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.959+0000 7fab78de3700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab740b18c0 0x7fab7413dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:37.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.959+0000 7fab78de3700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fab740b44e0 0x7fab7413e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:37.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.959+0000 7fab78de3700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab740aab50 con 0x7fab740b44e0 2020-11-28T21:51:37.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.963+0000 7fab78de3700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab740b1f50 con 0x7fab740b3950 2020-11-28T21:51:37.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.963+0000 7fab78de3700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab74143e90 con 0x7fab740b18c0 2020-11-28T21:51:37.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.963+0000 7fab737fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fab740b3950 0x7fab740b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:37.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.963+0000 7fab737fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fab740b3950 0x7fab740b3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40502/0 (socket says 172.21.15.162:40502) 2020-11-28T21:51:37.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.963+0000 7fab737fe700 1 -- 172.21.15.162:0/1626401152 learned_addr learned my addr 172.21.15.162:0/1626401152 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:37.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.963+0000 7fab72ffd700 1 --2- 172.21.15.162:0/1626401152 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab740b18c0 0x7fab7413dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:37.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.963+0000 7fab73fff700 1 --2- 172.21.15.162:0/1626401152 >> v2:172.21.15.118:3300/0 conn(0x7fab740b44e0 0x7fab7413e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:37.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.963+0000 7fab737fe700 1 -- 172.21.15.162:0/1626401152 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab740b18c0 msgr2=0x7fab7413dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:51:37.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.963+0000 7fab737fe700 1 --2- 172.21.15.162:0/1626401152 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab740b18c0 0x7fab7413dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:37.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.963+0000 7fab737fe700 1 -- 172.21.15.162:0/1626401152 >> v2:172.21.15.118:3300/0 conn(0x7fab740b44e0 msgr2=0x7fab7413e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:51:37.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.963+0000 7fab737fe700 1 --2- 172.21.15.162:0/1626401152 >> v2:172.21.15.118:3300/0 conn(0x7fab740b44e0 0x7fab7413e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:37.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.963+0000 7fab737fe700 1 -- 172.21.15.162:0/1626401152 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab740aa8b0 con 0x7fab740b3950 2020-11-28T21:51:37.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.963+0000 7fab72ffd700 1 --2- 172.21.15.162:0/1626401152 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab740b18c0 0x7fab7413dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:51:37.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.963+0000 7fab737fe700 1 --2- 172.21.15.162:0/1626401152 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab740b3950 0x7fab740b3d30 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7fab6c002f30 tx=0x7fab6c006e80).ready entity=mon.1 client_cookie=93e92e32140d53b7 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:37.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.963+0000 7fab727fc700 1 -- 172.21.15.162:0/1626401152 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab6c0097f0 con 0x7fab740b3950 2020-11-28T21:51:37.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.963+0000 7fab727fc700 1 -- 172.21.15.162:0/1626401152 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab6c009950 con 0x7fab740b3950 2020-11-28T21:51:37.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.963+0000 7fab727fc700 1 -- 172.21.15.162:0/1626401152 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab6c009cb0 con 0x7fab740b3950 2020-11-28T21:51:37.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.963+0000 7fab78de3700 1 -- 172.21.15.162:0/1626401152 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab740b3950 msgr2=0x7fab740b3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:37.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.963+0000 7fab78de3700 1 --2- 172.21.15.162:0/1626401152 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab740b3950 0x7fab740b3d30 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7fab6c002f30 tx=0x7fab6c006e80).stop 2020-11-28T21:51:37.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.963+0000 7fab78de3700 1 -- 172.21.15.162:0/1626401152 shutdown_connections 2020-11-28T21:51:37.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.963+0000 7fab78de3700 1 --2- 172.21.15.162:0/1626401152 >> v2:172.21.15.118:3300/0 conn(0x7fab740b44e0 0x7fab7413e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:37.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.963+0000 7fab78de3700 1 --2- 172.21.15.162:0/1626401152 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab740b3950 0x7fab740b3d30 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:37.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.963+0000 7fab78de3700 1 --2- 172.21.15.162:0/1626401152 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab740b18c0 0x7fab7413dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:37.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.963+0000 7fab78de3700 1 -- 172.21.15.162:0/1626401152 >> 172.21.15.162:0/1626401152 conn(0x7fab74062630 msgr2=0x7fab74062a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:37.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.963+0000 7fab78de3700 1 -- 172.21.15.162:0/1626401152 shutdown_connections 2020-11-28T21:51:37.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.963+0000 7fab78de3700 1 -- 172.21.15.162:0/1626401152 wait complete. 2020-11-28T21:51:37.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.963+0000 7fab78de3700 1 Processor -- start 2020-11-28T21:51:37.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.963+0000 7fab78de3700 1 -- start start 2020-11-28T21:51:37.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.963+0000 7fab78de3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab740b18c0 0x7fab740ae430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:37.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.963+0000 7fab78de3700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab740b3950 0x7fab7405f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:37.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.967+0000 7fab78de3700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab740b44e0 0x7fab7405fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:37.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.967+0000 7fab78de3700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab740aed50 con 0x7fab740b3950 2020-11-28T21:51:37.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.967+0000 7fab78de3700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab740aeff0 con 0x7fab740b18c0 2020-11-28T21:51:37.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.967+0000 7fab78de3700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab740af290 con 0x7fab740b44e0 2020-11-28T21:51:37.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.967+0000 7fab737fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab740b18c0 0x7fab740ae430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:37.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.967+0000 7fab737fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab740b18c0 0x7fab740ae430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40508/0 (socket says 172.21.15.162:40508) 2020-11-28T21:51:37.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.967+0000 7fab737fe700 1 -- 172.21.15.162:0/1974138751 learned_addr learned my addr 172.21.15.162:0/1974138751 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:37.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.967+0000 7fab73fff700 1 --2- 172.21.15.162:0/1974138751 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab740b44e0 0x7fab7405fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:37.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.967+0000 7fab72ffd700 1 --2- 172.21.15.162:0/1974138751 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab740b3950 0x7fab7405f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:37.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.967+0000 7fab737fe700 1 -- 172.21.15.162:0/1974138751 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab740b44e0 msgr2=0x7fab7405fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:37.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.967+0000 7fab737fe700 1 --2- 172.21.15.162:0/1974138751 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab740b44e0 0x7fab7405fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:37.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.967+0000 7fab737fe700 1 -- 172.21.15.162:0/1974138751 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab740b3950 msgr2=0x7fab7405f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:37.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.967+0000 7fab737fe700 1 --2- 172.21.15.162:0/1974138751 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab740b3950 0x7fab7405f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:37.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.967+0000 7fab737fe700 1 -- 172.21.15.162:0/1974138751 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab74060f90 con 0x7fab740b18c0 2020-11-28T21:51:37.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.967+0000 7fab737fe700 1 --2- 172.21.15.162:0/1974138751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab740b18c0 0x7fab740ae430 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7fab6c002f30 tx=0x7fab6c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:37.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.967+0000 7fab70ff9700 1 -- 172.21.15.162:0/1974138751 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab6c011db0 con 0x7fab740b18c0 2020-11-28T21:51:37.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.967+0000 7fab70ff9700 1 -- 172.21.15.162:0/1974138751 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab6c01e460 con 0x7fab740b18c0 2020-11-28T21:51:37.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.967+0000 7fab78de3700 1 -- 172.21.15.162:0/1974138751 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab740ae920 con 0x7fab740b18c0 2020-11-28T21:51:37.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.967+0000 7fab70ff9700 1 -- 172.21.15.162:0/1974138751 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab6c01e7c0 con 0x7fab740b18c0 2020-11-28T21:51:37.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.967+0000 7fab78de3700 1 -- 172.21.15.162:0/1974138751 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab74141720 con 0x7fab740b18c0 2020-11-28T21:51:37.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.967+0000 7fab78de3700 1 -- 172.21.15.162:0/1974138751 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab74057050 con 0x7fab740b18c0 2020-11-28T21:51:37.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.967+0000 7fab70ff9700 1 -- 172.21.15.162:0/1974138751 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fab6c01a040 con 0x7fab740b18c0 2020-11-28T21:51:37.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.967+0000 7fab70ff9700 1 --2- 172.21.15.162:0/1974138751 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fab54055350 0x7fab54057810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:37.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.967+0000 7fab70ff9700 1 -- 172.21.15.162:0/1974138751 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5721+0+0 (secure 0 0 0) 0x7fab6c06ef30 con 0x7fab740b18c0 2020-11-28T21:51:37.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.967+0000 7fab72ffd700 1 --2- 172.21.15.162:0/1974138751 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fab54055350 0x7fab54057810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:37.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.971+0000 7fab72ffd700 1 --2- 172.21.15.162:0/1974138751 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fab54055350 0x7fab54057810 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7fab6800c000 tx=0x7fab68008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:37.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:37.975+0000 7fab70ff9700 1 -- 172.21.15.162:0/1974138751 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fab6c0452f0 con 0x7fab740b18c0 2020-11-28T21:51:38.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:38.119+0000 7fab78de3700 1 -- 172.21.15.162:0/1974138751 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fab7405d250 con 0x7fab740b18c0 2020-11-28T21:51:38.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:38.831+0000 7fab70ff9700 1 -- 172.21.15.162:0/1974138751 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v59) v1 ==== 173+0+0 (secure 0 0 0) 0x7fab6c023020 con 0x7fab740b18c0 2020-11-28T21:51:38.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:38.835+0000 7fab78de3700 1 -- 172.21.15.162:0/1974138751 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fab54055350 msgr2=0x7fab54057810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:38.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:38.835+0000 7fab78de3700 1 --2- 172.21.15.162:0/1974138751 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fab54055350 0x7fab54057810 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7fab6800c000 tx=0x7fab68008040).stop 2020-11-28T21:51:38.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:38.835+0000 7fab78de3700 1 -- 172.21.15.162:0/1974138751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab740b18c0 msgr2=0x7fab740ae430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:38.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:38.835+0000 7fab78de3700 1 --2- 172.21.15.162:0/1974138751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab740b18c0 0x7fab740ae430 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7fab6c002f30 tx=0x7fab6c015000).stop 2020-11-28T21:51:38.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:38.835+0000 7fab78de3700 1 -- 172.21.15.162:0/1974138751 shutdown_connections 2020-11-28T21:51:38.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:38.835+0000 7fab78de3700 1 --2- 172.21.15.162:0/1974138751 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fab54055350 0x7fab54057810 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:38.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:38.835+0000 7fab78de3700 1 --2- 172.21.15.162:0/1974138751 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab740b44e0 0x7fab7405fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:38.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:38.835+0000 7fab78de3700 1 --2- 172.21.15.162:0/1974138751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab740b18c0 0x7fab740ae430 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:38.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:38.835+0000 7fab78de3700 1 --2- 172.21.15.162:0/1974138751 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab740b3950 0x7fab7405f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:38.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:38.835+0000 7fab78de3700 1 -- 172.21.15.162:0/1974138751 >> 172.21.15.162:0/1974138751 conn(0x7fab74062630 msgr2=0x7fab74142f80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:38.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:38.835+0000 7fab78de3700 1 -- 172.21.15.162:0/1974138751 shutdown_connections 2020-11-28T21:51:38.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:38.835+0000 7fab78de3700 1 -- 172.21.15.162:0/1974138751 wait complete. 2020-11-28T21:51:38.835 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_data' removed 2020-11-28T21:51:38.850 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-28T21:51:38.850 INFO:tasks.ceph.mds.a:Stopping old one... 2020-11-28T21:51:38.850 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-11-28T21:51:38.851 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T21:51:38.851 INFO:tasks.ceph.mds.c:Restarting daemon 2020-11-28T21:51:38.851 INFO:tasks.ceph.mds.c:Stopping old one... 2020-11-28T21:51:38.852 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2020-11-28T21:51:38.852 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T21:51:38.852 INFO:tasks.ceph.mds.b:Restarting daemon 2020-11-28T21:51:38.853 INFO:tasks.ceph.mds.b:Stopping old one... 2020-11-28T21:51:38.853 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-11-28T21:51:38.853 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T21:51:38.860 INFO:tasks.ceph.mds.a:Stopped 2020-11-28T21:51:38.860 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-28T21:51:38.866 INFO:tasks.ceph.mds.b:Stopped 2020-11-28T21:51:38.867 INFO:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-11-28T21:51:38.868 INFO:tasks.ceph.mds.c:Stopped 2020-11-28T21:51:38.869 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2020-11-28T21:51:38.912 INFO:tasks.ceph.mds.b:Started 2020-11-28T21:51:38.912 DEBUG:teuthology.parallel:result is None 2020-11-28T21:51:38.914 INFO:tasks.ceph.mds.a:Started 2020-11-28T21:51:38.915 DEBUG:teuthology.parallel:result is None 2020-11-28T21:51:38.919 INFO:tasks.ceph.mds.c:Started 2020-11-28T21:51:38.919 DEBUG:teuthology.parallel:result is None 2020-11-28T21:51:38.920 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist clear 2020-11-28T21:51:38.987 INFO:tasks.ceph.mds.b.smithi118.stdout:starting mds.b at 2020-11-28T21:51:38.988 INFO:tasks.ceph.mds.a.smithi162.stdout:starting mds.a at 2020-11-28T21:51:39.003 INFO:tasks.ceph.mds.c.smithi162.stdout:starting mds.c at 2020-11-28T21:51:39.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.043+0000 7feab8f07700 1 Processor -- start 2020-11-28T21:51:39.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.043+0000 7feab8f07700 1 -- start start 2020-11-28T21:51:39.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.047+0000 7feab8f07700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7feab406aef0 0x7feab4140080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:39.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.047+0000 7feab8f07700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feab40af910 con 0x7feab413c530 2020-11-28T21:51:39.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.047+0000 7feab8f07700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feab41438d0 con 0x7feab40af1f0 2020-11-28T21:51:39.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.047+0000 7feab8f07700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab40b66f0 con 0x7feab406aef0 2020-11-28T21:51:39.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.047+0000 7feab259c700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7feab40af1f0 0x7feab413f970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:47390/0 (socket says 172.21.15.162:47390) 2020-11-28T21:51:39.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.047+0000 7feab259c700 1 -- 172.21.15.162:0/4099206213 learned_addr learned my addr 172.21.15.162:0/4099206213 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:51:39.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.047+0000 7feab1d9b700 1 --2- 172.21.15.162:0/4099206213 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7feab406aef0 0x7feab4140080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:39.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.047+0000 7feab159a700 1 -- 172.21.15.162:0/4099206213 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 861779817 0 0) 0x7feab41438d0 con 0x7feab40af1f0 2020-11-28T21:51:39.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.047+0000 7feab159a700 1 -- 172.21.15.162:0/4099206213 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea94003230 con 0x7feab40af1f0 2020-11-28T21:51:39.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.047+0000 7feab159a700 1 -- 172.21.15.162:0/4099206213 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1975383311 0 0) 0x7feab40af910 con 0x7feab413c530 2020-11-28T21:51:39.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.047+0000 7feab159a700 1 -- 172.21.15.162:0/4099206213 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feab41438d0 con 0x7feab413c530 2020-11-28T21:51:39.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.047+0000 7feab1d9b700 1 -- 172.21.15.162:0/4099206213 >> v1:172.21.15.162:6789/0 conn(0x7feab40af1f0 legacy=0x7feab413f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:39.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.047+0000 7feab159a700 1 -- 172.21.15.162:0/4099206213 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 238400239 0 0) 0x7fea94003230 con 0x7feab40af1f0 2020-11-28T21:51:39.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.047+0000 7feab1d9b700 1 -- 172.21.15.162:0/4099206213 >> v1:172.21.15.118:6789/0 conn(0x7feab413c530 legacy=0x7feab413c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:39.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.047+0000 7feab1d9b700 1 -- 172.21.15.162:0/4099206213 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab4143b70 con 0x7feab406aef0 2020-11-28T21:51:39.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.047+0000 7feab1d9b700 1 --2- 172.21.15.162:0/4099206213 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7feab406aef0 0x7feab4140080 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7feaac001490 tx=0x7fea9c006dc0).ready entity=mon.2 client_cookie=6bf0b7aae1934f65 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:39.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.047+0000 7feab159a700 1 -- 172.21.15.162:0/4099206213 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feaac001a90 con 0x7feab406aef0 2020-11-28T21:51:39.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.047+0000 7feab159a700 1 -- 172.21.15.162:0/4099206213 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea9c00d5c0 con 0x7feab406aef0 2020-11-28T21:51:39.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.047+0000 7feab159a700 1 -- 172.21.15.162:0/4099206213 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea9c00d730 con 0x7feab406aef0 2020-11-28T21:51:39.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.047+0000 7feab8f07700 1 -- 172.21.15.162:0/4099206213 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7feab406aef0 msgr2=0x7feab4140080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:39.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.047+0000 7feab8f07700 1 --2- 172.21.15.162:0/4099206213 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7feab406aef0 0x7feab4140080 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7feaac001490 tx=0x7fea9c006dc0).stop 2020-11-28T21:51:39.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.047+0000 7feab8f07700 1 -- 172.21.15.162:0/4099206213 shutdown_connections 2020-11-28T21:51:39.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.047+0000 7feab8f07700 1 --2- 172.21.15.162:0/4099206213 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7feab406aef0 0x7feab4140080 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:39.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.047+0000 7feab8f07700 1 -- 172.21.15.162:0/4099206213 >> 172.21.15.162:0/4099206213 conn(0x7feab40b56f0 msgr2=0x7feab40b5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:39.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.047+0000 7feab8f07700 1 -- 172.21.15.162:0/4099206213 shutdown_connections 2020-11-28T21:51:39.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.047+0000 7feab8f07700 1 -- 172.21.15.162:0/4099206213 wait complete. 2020-11-28T21:51:39.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.047+0000 7feab8f07700 1 Processor -- start 2020-11-28T21:51:39.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.047+0000 7feab8f07700 1 -- start start 2020-11-28T21:51:39.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.051+0000 7feab8f07700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7feab406aef0 0x7feab40b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:39.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.051+0000 7feab8f07700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feab40af1f0 0x7feab41530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:39.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.051+0000 7feab8f07700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feab413c530 0x7feab41535b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:39.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.051+0000 7feab8f07700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab40643f0 con 0x7feab413c530 2020-11-28T21:51:39.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.051+0000 7feab8f07700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab414f7d0 con 0x7feab40af1f0 2020-11-28T21:51:39.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.051+0000 7feab8f07700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab40b47d0 con 0x7feab406aef0 2020-11-28T21:51:39.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.051+0000 7feab1d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feab40af1f0 0x7feab41530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:39.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.051+0000 7feab1d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feab40af1f0 0x7feab41530c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40552/0 (socket says 172.21.15.162:40552) 2020-11-28T21:51:39.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.051+0000 7feab1d9b700 1 -- 172.21.15.162:0/2256857229 learned_addr learned my addr 172.21.15.162:0/2256857229 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:39.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.051+0000 7feab259c700 1 --2- 172.21.15.162:0/2256857229 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7feab406aef0 0x7feab40b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:39.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.051+0000 7feab2d9d700 1 --2- 172.21.15.162:0/2256857229 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feab413c530 0x7feab41535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:39.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.051+0000 7feab1d9b700 1 -- 172.21.15.162:0/2256857229 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7feab406aef0 msgr2=0x7feab40b0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:39.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.051+0000 7feab1d9b700 1 --2- 172.21.15.162:0/2256857229 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7feab406aef0 0x7feab40b0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:39.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.051+0000 7feab1d9b700 1 -- 172.21.15.162:0/2256857229 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feab413c530 msgr2=0x7feab41535b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:39.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.051+0000 7feab1d9b700 1 --2- 172.21.15.162:0/2256857229 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feab413c530 0x7feab41535b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:39.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.051+0000 7feab1d9b700 1 -- 172.21.15.162:0/2256857229 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab40b4a70 con 0x7feab40af1f0 2020-11-28T21:51:39.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.051+0000 7feab1d9b700 1 --2- 172.21.15.162:0/2256857229 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feab40af1f0 0x7feab41530c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7feaac001490 tx=0x7fea9c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:39.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.051+0000 7feaab7fe700 1 -- 172.21.15.162:0/2256857229 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea9c011b80 con 0x7feab40af1f0 2020-11-28T21:51:39.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.051+0000 7feaab7fe700 1 -- 172.21.15.162:0/2256857229 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea9c011ce0 con 0x7feab40af1f0 2020-11-28T21:51:39.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.051+0000 7feaab7fe700 1 -- 172.21.15.162:0/2256857229 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea9c01d640 con 0x7feab40af1f0 2020-11-28T21:51:39.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.051+0000 7feab8f07700 1 -- 172.21.15.162:0/2256857229 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feab40b4d10 con 0x7feab40af1f0 2020-11-28T21:51:39.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.051+0000 7feab8f07700 1 -- 172.21.15.162:0/2256857229 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feab41431c0 con 0x7feab40af1f0 2020-11-28T21:51:39.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.051+0000 7feab8f07700 1 -- 172.21.15.162:0/2256857229 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feab4057050 con 0x7feab40af1f0 2020-11-28T21:51:39.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.051+0000 7feaab7fe700 1 -- 172.21.15.162:0/2256857229 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fea9c00f070 con 0x7feab40af1f0 2020-11-28T21:51:39.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.055+0000 7feaab7fe700 1 --2- 172.21.15.162:0/2256857229 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fea98055440 0x7fea98057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:39.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.055+0000 7feaab7fe700 1 -- 172.21.15.162:0/2256857229 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5339+0+0 (secure 0 0 0) 0x7fea9c06ea10 con 0x7feab40af1f0 2020-11-28T21:51:39.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.055+0000 7feab259c700 1 --2- 172.21.15.162:0/2256857229 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fea98055440 0x7fea98057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:39.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.055+0000 7feab259c700 1 --2- 172.21.15.162:0/2256857229 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fea98055440 0x7fea98057900 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7feaac00b000 tx=0x7feaac000f00).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:39.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.059+0000 7feaab7fe700 1 -- 172.21.15.162:0/2256857229 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fea9c044ff0 con 0x7feab40af1f0 2020-11-28T21:51:39.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:39.219+0000 7feab8f07700 1 -- 172.21.15.162:0/2256857229 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7feab40b4ea0 con 0x7feab40af1f0 2020-11-28T21:51:40.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.023+0000 7feaab7fe700 1 -- 172.21.15.162:0/2256857229 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v62) v1 ==== 97+0+0 (secure 0 0 0) 0x7fea9c022070 con 0x7feab40af1f0 2020-11-28T21:51:40.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.027+0000 7feab8f07700 1 -- 172.21.15.162:0/2256857229 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fea98055440 msgr2=0x7fea98057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:40.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.027+0000 7feab8f07700 1 --2- 172.21.15.162:0/2256857229 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fea98055440 0x7fea98057900 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7feaac00b000 tx=0x7feaac000f00).stop 2020-11-28T21:51:40.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.027+0000 7feab8f07700 1 -- 172.21.15.162:0/2256857229 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feab40af1f0 msgr2=0x7feab41530c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:40.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.027+0000 7feab8f07700 1 --2- 172.21.15.162:0/2256857229 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feab40af1f0 0x7feab41530c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7feaac001490 tx=0x7fea9c015000).stop 2020-11-28T21:51:40.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.027+0000 7feab8f07700 1 -- 172.21.15.162:0/2256857229 shutdown_connections 2020-11-28T21:51:40.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.027+0000 7feab8f07700 1 --2- 172.21.15.162:0/2256857229 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fea98055440 0x7fea98057900 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:40.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.027+0000 7feab8f07700 1 --2- 172.21.15.162:0/2256857229 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7feab406aef0 0x7feab40b0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:40.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.027+0000 7feab8f07700 1 --2- 172.21.15.162:0/2256857229 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feab413c530 0x7feab41535b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:40.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.027+0000 7feab8f07700 1 --2- 172.21.15.162:0/2256857229 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feab40af1f0 0x7feab41530c0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:40.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.027+0000 7feab8f07700 1 -- 172.21.15.162:0/2256857229 >> 172.21.15.162:0/2256857229 conn(0x7feab40b56f0 msgr2=0x7feab40aca40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:40.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.027+0000 7feab8f07700 1 -- 172.21.15.162:0/2256857229 shutdown_connections 2020-11-28T21:51:40.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.027+0000 7feab8f07700 1 -- 172.21.15.162:0/2256857229 wait complete. 2020-11-28T21:51:40.027 INFO:teuthology.orchestra.run.smithi162.stderr: removed all blacklist entries 2020-11-28T21:51:40.040 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2020-11-28T21:51:40.041 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth ls --format=json-pretty 2020-11-28T21:51:40.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.159+0000 7f7212351700 1 Processor -- start 2020-11-28T21:51:40.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.159+0000 7f7212351700 1 -- start start 2020-11-28T21:51:40.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.159+0000 7f7212351700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f720c0b60e0 0x7f720c0b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:40.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.159+0000 7f7212351700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f720c0b4390 0x7f720c13fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:40.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.159+0000 7f7212351700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f720c0b4930 con 0x7f720c060d10 2020-11-28T21:51:40.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.159+0000 7f7212351700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f720c0b2170 con 0x7f720c0b60e0 2020-11-28T21:51:40.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.159+0000 7f7212351700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f720c0b6660 con 0x7f720c0b4390 2020-11-28T21:51:40.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.159+0000 7f72108ee700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f720c0b4390 0x7f720c13fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:40.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.159+0000 7f72108ee700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f720c0b4390 0x7f720c13fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40566/0 (socket says 172.21.15.162:40566) 2020-11-28T21:51:40.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.159+0000 7f72108ee700 1 -- 172.21.15.162:0/3804188211 learned_addr learned my addr 172.21.15.162:0/3804188211 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:40.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.159+0000 7f720b7fe700 1 --2- 172.21.15.162:0/3804188211 >> v2:172.21.15.118:3300/0 conn(0x7f720c0b60e0 0x7f720c0b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:40.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.163+0000 7f72108ee700 1 -- 172.21.15.162:0/3804188211 >> v2:172.21.15.118:3300/0 conn(0x7f720c0b60e0 msgr2=0x7f720c0b52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:51:40.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.163+0000 7f72108ee700 1 --2- 172.21.15.162:0/3804188211 >> v2:172.21.15.118:3300/0 conn(0x7f720c0b60e0 0x7f720c0b52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:40.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.163+0000 7f72108ee700 1 -- 172.21.15.162:0/3804188211 >> v1:172.21.15.118:6789/0 conn(0x7f720c060d10 legacy=0x7f720c146180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:40.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.163+0000 7f72108ee700 1 -- 172.21.15.162:0/3804188211 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f720c0b4b60 con 0x7f720c0b4390 2020-11-28T21:51:40.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.163+0000 7f72108ee700 1 --2- 172.21.15.162:0/3804188211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f720c0b4390 0x7f720c13fc40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7f72040032b0 tx=0x7f7204009270).ready entity=mon.1 client_cookie=98dad58e2a8c0f07 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:40.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.163+0000 7f720affd700 1 -- 172.21.15.162:0/3804188211 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7204009e60 con 0x7f720c0b4390 2020-11-28T21:51:40.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.163+0000 7f720affd700 1 -- 172.21.15.162:0/3804188211 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f720400bc00 con 0x7f720c0b4390 2020-11-28T21:51:40.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.163+0000 7f720affd700 1 -- 172.21.15.162:0/3804188211 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f720400bd70 con 0x7f720c0b4390 2020-11-28T21:51:40.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.163+0000 7f7212351700 1 -- 172.21.15.162:0/3804188211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f720c0b4390 msgr2=0x7f720c13fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:40.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.163+0000 7f7212351700 1 --2- 172.21.15.162:0/3804188211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f720c0b4390 0x7f720c13fc40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7f72040032b0 tx=0x7f7204009270).stop 2020-11-28T21:51:40.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.163+0000 7f7212351700 1 -- 172.21.15.162:0/3804188211 shutdown_connections 2020-11-28T21:51:40.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.163+0000 7f7212351700 1 --2- 172.21.15.162:0/3804188211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f720c0b4390 0x7f720c13fc40 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:40.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.163+0000 7f7212351700 1 --2- 172.21.15.162:0/3804188211 >> v2:172.21.15.118:3300/0 conn(0x7f720c0b60e0 0x7f720c0b52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:40.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.163+0000 7f7212351700 1 -- 172.21.15.162:0/3804188211 >> 172.21.15.162:0/3804188211 conn(0x7f720c05fe70 msgr2=0x7f720c06aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:40.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.163+0000 7f7212351700 1 -- 172.21.15.162:0/3804188211 shutdown_connections 2020-11-28T21:51:40.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.163+0000 7f7212351700 1 -- 172.21.15.162:0/3804188211 wait complete. 2020-11-28T21:51:40.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.163+0000 7f7212351700 1 Processor -- start 2020-11-28T21:51:40.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.163+0000 7f7212351700 1 -- start start 2020-11-28T21:51:40.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.167+0000 7f7212351700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f720c060d10 0x7f720c0ae830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:40.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.167+0000 7f7212351700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f720c0b4390 0x7f720c136910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:40.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.167+0000 7f7212351700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f720c0b60e0 0x7f720c136e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:40.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.167+0000 7f7212351700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720c135db0 con 0x7f720c060d10 2020-11-28T21:51:40.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.167+0000 7f7212351700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720c137930 con 0x7f720c0b4390 2020-11-28T21:51:40.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.167+0000 7f7212351700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f720c1364b0 con 0x7f720c0b60e0 2020-11-28T21:51:40.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.167+0000 7f720b7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f720c0b4390 0x7f720c136910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:40.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.167+0000 7f720b7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f720c0b4390 0x7f720c136910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40570/0 (socket says 172.21.15.162:40570) 2020-11-28T21:51:40.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.167+0000 7f720b7fe700 1 -- 172.21.15.162:0/1280729354 learned_addr learned my addr 172.21.15.162:0/1280729354 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:40.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.167+0000 7f72108ee700 1 --2- 172.21.15.162:0/1280729354 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f720c0b60e0 0x7f720c136e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:40.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.167+0000 7f720bfff700 1 --2- 172.21.15.162:0/1280729354 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f720c060d10 0x7f720c0ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:40.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.167+0000 7f720b7fe700 1 -- 172.21.15.162:0/1280729354 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f720c0b60e0 msgr2=0x7f720c136e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:40.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.167+0000 7f720b7fe700 1 --2- 172.21.15.162:0/1280729354 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f720c0b60e0 0x7f720c136e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:40.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.167+0000 7f720b7fe700 1 -- 172.21.15.162:0/1280729354 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f720c060d10 msgr2=0x7f720c0ae830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:40.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.167+0000 7f720b7fe700 1 --2- 172.21.15.162:0/1280729354 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f720c060d10 0x7f720c0ae830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:40.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.167+0000 7f720b7fe700 1 -- 172.21.15.162:0/1280729354 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f720c13c1a0 con 0x7f720c0b4390 2020-11-28T21:51:40.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.167+0000 7f72108ee700 1 --2- 172.21.15.162:0/1280729354 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f720c0b60e0 0x7f720c136e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:51:40.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.167+0000 7f720b7fe700 1 --2- 172.21.15.162:0/1280729354 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f720c0b4390 0x7f720c136910 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7f720000c000 tx=0x7f7200006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:40.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.167+0000 7f72097fa700 1 -- 172.21.15.162:0/1280729354 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f720000ca10 con 0x7f720c0b4390 2020-11-28T21:51:40.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.167+0000 7f72097fa700 1 -- 172.21.15.162:0/1280729354 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f720000cb70 con 0x7f720c0b4390 2020-11-28T21:51:40.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.167+0000 7f7212351700 1 -- 172.21.15.162:0/1280729354 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f720c13bc60 con 0x7f720c0b4390 2020-11-28T21:51:40.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.167+0000 7f72097fa700 1 -- 172.21.15.162:0/1280729354 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72000123f0 con 0x7f720c0b4390 2020-11-28T21:51:40.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.167+0000 7f7212351700 1 -- 172.21.15.162:0/1280729354 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f720c0aed20 con 0x7f720c0b4390 2020-11-28T21:51:40.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.167+0000 7f7212351700 1 -- 172.21.15.162:0/1280729354 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f720c057050 con 0x7f720c0b4390 2020-11-28T21:51:40.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.171+0000 7f72097fa700 1 -- 172.21.15.162:0/1280729354 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7200012550 con 0x7f720c0b4390 2020-11-28T21:51:40.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.171+0000 7f72097fa700 1 --2- 172.21.15.162:0/1280729354 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f71ec055470 0x7f71ec057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:40.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.171+0000 7f72097fa700 1 -- 172.21.15.162:0/1280729354 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f720006dfd0 con 0x7f720c0b4390 2020-11-28T21:51:40.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.171+0000 7f720bfff700 1 --2- 172.21.15.162:0/1280729354 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f71ec055470 0x7f71ec057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:40.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.171+0000 7f720bfff700 1 --2- 172.21.15.162:0/1280729354 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f71ec055470 0x7f71ec057930 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f71fc00d000 tx=0x7f71fc000b80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:40.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.175+0000 7f72097fa700 1 -- 172.21.15.162:0/1280729354 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7200044490 con 0x7f720c0b4390 2020-11-28T21:51:40.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.359+0000 7f7212351700 1 -- 172.21.15.162:0/1280729354 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f720c05c760 con 0x7f720c0b4390 2020-11-28T21:51:40.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.359+0000 7f72097fa700 1 -- 172.21.15.162:0/1280729354 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+4255 (secure 0 0 0) 0x7f7200047ab0 con 0x7f720c0b4390 2020-11-28T21:51:40.358 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:51:40.358 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2020-11-28T21:51:40.358 INFO:teuthology.orchestra.run.smithi162.stdout: "auth_dump": [ 2020-11-28T21:51:40.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:40.359 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mds.a", 2020-11-28T21:51:40.359 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfWxUpKxAAIUDwQRKwz8H4Qk9qE356Bg==", 2020-11-28T21:51:40.359 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:51:40.359 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:51:40.360 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:51:40.360 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:51:40.360 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:51:40.360 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:40.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:40.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:40.361 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mds.b", 2020-11-28T21:51:40.361 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfKY2zORAAaO65etMZiYcaTDNX0nsEPw==", 2020-11-28T21:51:40.361 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:51:40.361 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:51:40.362 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:51:40.362 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:51:40.362 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:51:40.362 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:40.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:40.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:40.363 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mds.c", 2020-11-28T21:51:40.363 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfdQbRMxAAWlcjesxTZzZnvKCQneSjjQ==", 2020-11-28T21:51:40.363 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:51:40.363 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:51:40.364 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:51:40.364 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:51:40.364 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:51:40.364 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:40.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:40.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:40.365 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.0", 2020-11-28T21:51:40.365 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQADxcJfF3WLLxAAxsB5YPjH1yjv/vhlRbijqw==", 2020-11-28T21:51:40.365 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:51:40.365 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:51:40.366 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:51:40.366 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:51:40.366 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:40.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:40.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:40.367 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.1", 2020-11-28T21:51:40.367 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAGxcJfKYbBFhAA1x4DzFtkQQv6uWLIdOi1qQ==", 2020-11-28T21:51:40.367 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:51:40.367 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:51:40.367 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:51:40.368 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:51:40.368 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:40.368 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:40.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:40.368 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.2", 2020-11-28T21:51:40.369 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAIxcJfvcYUOhAASENjM0HfAxxyFe13mmE36Q==", 2020-11-28T21:51:40.369 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:51:40.369 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:51:40.369 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:51:40.370 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:51:40.370 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:40.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:40.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:40.371 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.3", 2020-11-28T21:51:40.371 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQALxcJfc/OHIRAAFYx3s8Ssttt15hjoH04tow==", 2020-11-28T21:51:40.371 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:51:40.371 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:51:40.371 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:51:40.372 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:51:40.372 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:40.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:40.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:40.372 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.4", 2020-11-28T21:51:40.373 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAPxcJffGWqCRAAb8CCN5oUPkdDh6BJH5HEnw==", 2020-11-28T21:51:40.373 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:51:40.373 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:51:40.373 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:51:40.373 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:51:40.374 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:40.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:40.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:40.375 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.5", 2020-11-28T21:51:40.375 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQARxcJfo8GKKRAAfbsCDb4L2wULiKmHyzu7/A==", 2020-11-28T21:51:40.375 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:51:40.376 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:51:40.376 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:51:40.376 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:51:40.377 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:40.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:40.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:40.378 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.6", 2020-11-28T21:51:40.379 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAUxcJf8Y2MEBAAqQX6rX3LK0zodlI2wK4JrA==", 2020-11-28T21:51:40.379 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:51:40.379 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:51:40.380 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:51:40.380 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:51:40.380 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:40.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:40.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:40.382 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.7", 2020-11-28T21:51:40.382 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAWxcJfCLBdMhAAf8QixQMQDrfZOGWN8rHe5g==", 2020-11-28T21:51:40.382 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:51:40.383 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:51:40.383 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:51:40.383 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:51:40.384 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:40.384 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:40.384 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:40.385 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "client.0", 2020-11-28T21:51:40.385 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQABxcJfS9vDAxAAUk4FU8FVt4if/CRH5ZisOA==", 2020-11-28T21:51:40.385 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:51:40.387 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:51:40.387 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow r", 2020-11-28T21:51:40.388 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow rw pool=cephfs_data" 2020-11-28T21:51:40.388 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:40.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:40.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:40.389 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "client.1", 2020-11-28T21:51:40.390 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQABxcJfuCU8BhAAXUSoSK3D0XRo58uHNzcjXg==", 2020-11-28T21:51:40.390 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:51:40.390 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:51:40.391 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow r", 2020-11-28T21:51:40.391 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow rw pool=cephfs_data" 2020-11-28T21:51:40.392 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:40.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:40.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:40.393 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "client.admin", 2020-11-28T21:51:40.393 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfdlTWBxAApzAxlVjzdnETQF//Vnsg5w==", 2020-11-28T21:51:40.393 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:51:40.394 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow *", 2020-11-28T21:51:40.394 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:51:40.394 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:51:40.395 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:51:40.395 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:40.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:40.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:40.396 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mgr.x", 2020-11-28T21:51:40.396 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfZhdvKBAAT7zCCsD2OVCHjvddGLxQ/A==", 2020-11-28T21:51:40.396 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:51:40.396 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow *", 2020-11-28T21:51:40.397 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow profile mgr", 2020-11-28T21:51:40.397 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:51:40.397 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:40.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:40.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:40.398 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mgr.y", 2020-11-28T21:51:40.398 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfs8fvJRAAP0Atp074EeMHiTtpCCVHuA==", 2020-11-28T21:51:40.399 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:51:40.399 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow *", 2020-11-28T21:51:40.399 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow profile mgr", 2020-11-28T21:51:40.399 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:51:40.400 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:40.400 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:40.400 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:51:40.400 INFO:teuthology.orchestra.run.smithi162.stdout:} 2020-11-28T21:51:40.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.363+0000 7f7212351700 1 -- 172.21.15.162:0/1280729354 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f71ec055470 msgr2=0x7f71ec057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:40.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.363+0000 7f7212351700 1 --2- 172.21.15.162:0/1280729354 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f71ec055470 0x7f71ec057930 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f71fc00d000 tx=0x7f71fc000b80).stop 2020-11-28T21:51:40.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.363+0000 7f7212351700 1 -- 172.21.15.162:0/1280729354 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f720c0b4390 msgr2=0x7f720c136910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:40.404 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.363+0000 7f7212351700 1 --2- 172.21.15.162:0/1280729354 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f720c0b4390 0x7f720c136910 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7f720000c000 tx=0x7f7200006dc0).stop 2020-11-28T21:51:40.404 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.363+0000 7f7212351700 1 -- 172.21.15.162:0/1280729354 shutdown_connections 2020-11-28T21:51:40.404 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.363+0000 7f7212351700 1 --2- 172.21.15.162:0/1280729354 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f71ec055470 0x7f71ec057930 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:40.404 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.363+0000 7f7212351700 1 --2- 172.21.15.162:0/1280729354 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f720c0b60e0 0x7f720c136e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:40.405 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.363+0000 7f7212351700 1 --2- 172.21.15.162:0/1280729354 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f720c060d10 0x7f720c0ae830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:40.405 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.363+0000 7f7212351700 1 --2- 172.21.15.162:0/1280729354 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f720c0b4390 0x7f720c136910 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:40.405 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.363+0000 7f7212351700 1 -- 172.21.15.162:0/1280729354 >> 172.21.15.162:0/1280729354 conn(0x7f720c05fe70 msgr2=0x7f720c13e640 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:40.405 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.363+0000 7f7212351700 1 -- 172.21.15.162:0/1280729354 shutdown_connections 2020-11-28T21:51:40.406 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.363+0000 7f7212351700 1 -- 172.21.15.162:0/1280729354 wait complete. 2020-11-28T21:51:40.406 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:51:40.527 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.523+0000 7f57df59e700 1 Processor -- start 2020-11-28T21:51:40.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.523+0000 7f57df59e700 1 -- start start 2020-11-28T21:51:40.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.523+0000 7f57df59e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f57e00b7920 0x7f57e013f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:40.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.523+0000 7f57df59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f57e00b2da0 0x7f57e013fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:40.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.523+0000 7f57df59e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f57e0066fa0 0x7f57e00b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:40.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.523+0000 7f57df59e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57e0140360 con 0x7f57e0066fa0 2020-11-28T21:51:40.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.523+0000 7f57df59e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57e00b7d00 con 0x7f57e00b7920 2020-11-28T21:51:40.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.523+0000 7f57df59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e0143300 con 0x7f57e00b2da0 2020-11-28T21:51:40.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.523+0000 7f57de59c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f57e00b7920 0x7f57e013f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:40.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.523+0000 7f57de59c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f57e00b7920 0x7f57e013f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40576/0 (socket says 172.21.15.162:40576) 2020-11-28T21:51:40.530 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.523+0000 7f57de59c700 1 -- 172.21.15.162:0/1617160969 learned_addr learned my addr 172.21.15.162:0/1617160969 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:40.530 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.523+0000 7f57ddd9b700 1 --2- 172.21.15.162:0/1617160969 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f57e00b2da0 0x7f57e013fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:40.530 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.523+0000 7f57ded9d700 1 --2- 172.21.15.162:0/1617160969 >> v2:172.21.15.118:3300/0 conn(0x7f57e0066fa0 0x7f57e00b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:40.530 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.523+0000 7f57de59c700 1 -- 172.21.15.162:0/1617160969 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f57e00b2da0 msgr2=0x7f57e013fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:51:40.530 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.523+0000 7f57de59c700 1 --2- 172.21.15.162:0/1617160969 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f57e00b2da0 0x7f57e013fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:40.531 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.523+0000 7f57de59c700 1 -- 172.21.15.162:0/1617160969 >> v2:172.21.15.118:3300/0 conn(0x7f57e0066fa0 msgr2=0x7f57e00b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:51:40.531 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.523+0000 7f57de59c700 1 --2- 172.21.15.162:0/1617160969 >> v2:172.21.15.118:3300/0 conn(0x7f57e0066fa0 0x7f57e00b1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:40.531 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.523+0000 7f57de59c700 1 -- 172.21.15.162:0/1617160969 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e0143560 con 0x7f57e00b7920 2020-11-28T21:51:40.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.523+0000 7f57ded9d700 1 --2- 172.21.15.162:0/1617160969 >> v2:172.21.15.118:3300/0 conn(0x7f57e0066fa0 0x7f57e00b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:51:40.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.523+0000 7f57de59c700 1 --2- 172.21.15.162:0/1617160969 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57e00b7920 0x7f57e013f980 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7f57d00032b0 tx=0x7f57d0009270).ready entity=mon.1 client_cookie=9f40bd7ba9afa8be server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:40.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.523+0000 7f57dd59a700 1 -- 172.21.15.162:0/1617160969 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57d000bc00 con 0x7f57e00b7920 2020-11-28T21:51:40.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.523+0000 7f57dd59a700 1 -- 172.21.15.162:0/1617160969 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d000bd60 con 0x7f57e00b7920 2020-11-28T21:51:40.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.523+0000 7f57dd59a700 1 -- 172.21.15.162:0/1617160969 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57d0011400 con 0x7f57e00b7920 2020-11-28T21:51:40.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.523+0000 7f57df59e700 1 -- 172.21.15.162:0/1617160969 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57e00b7920 msgr2=0x7f57e013f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:40.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.523+0000 7f57df59e700 1 --2- 172.21.15.162:0/1617160969 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57e00b7920 0x7f57e013f980 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7f57d00032b0 tx=0x7f57d0009270).stop 2020-11-28T21:51:40.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.523+0000 7f57df59e700 1 -- 172.21.15.162:0/1617160969 shutdown_connections 2020-11-28T21:51:40.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.527+0000 7f57df59e700 1 --2- 172.21.15.162:0/1617160969 >> v2:172.21.15.118:3300/0 conn(0x7f57e0066fa0 0x7f57e00b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:40.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.527+0000 7f57df59e700 1 --2- 172.21.15.162:0/1617160969 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57e00b7920 0x7f57e013f980 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:40.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.527+0000 7f57df59e700 1 --2- 172.21.15.162:0/1617160969 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f57e00b2da0 0x7f57e013fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:40.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.527+0000 7f57df59e700 1 -- 172.21.15.162:0/1617160969 >> 172.21.15.162:0/1617160969 conn(0x7f57e005c440 msgr2=0x7f57e00b6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:40.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.527+0000 7f57df59e700 1 -- 172.21.15.162:0/1617160969 shutdown_connections 2020-11-28T21:51:40.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.527+0000 7f57df59e700 1 -- 172.21.15.162:0/1617160969 wait complete. 2020-11-28T21:51:40.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.527+0000 7f57df59e700 1 Processor -- start 2020-11-28T21:51:40.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.527+0000 7f57df59e700 1 -- start start 2020-11-28T21:51:40.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.527+0000 7f57df59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f57e0066fa0 0x7f57e0154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:40.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.527+0000 7f57df59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57e00b2da0 0x7f57e00ad140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:40.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.527+0000 7f57df59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f57e00b7920 0x7f57e00ae990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:40.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.527+0000 7f57df59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e00b1060 con 0x7f57e00b7920 2020-11-28T21:51:40.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.527+0000 7f57df59e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e00ae2c0 con 0x7f57e00b2da0 2020-11-28T21:51:40.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.527+0000 7f57df59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e01457d0 con 0x7f57e0066fa0 2020-11-28T21:51:40.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.527+0000 7f57ddd9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57e00b2da0 0x7f57e00ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:40.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.527+0000 7f57ddd9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57e00b2da0 0x7f57e00ad140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40584/0 (socket says 172.21.15.162:40584) 2020-11-28T21:51:40.537 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.527+0000 7f57ddd9b700 1 -- 172.21.15.162:0/2672437170 learned_addr learned my addr 172.21.15.162:0/2672437170 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:40.538 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.527+0000 7f57de59c700 1 --2- 172.21.15.162:0/2672437170 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f57e0066fa0 0x7f57e0154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:40.538 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.527+0000 7f57ded9d700 1 --2- 172.21.15.162:0/2672437170 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f57e00b7920 0x7f57e00ae990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:40.538 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.527+0000 7f57ddd9b700 1 -- 172.21.15.162:0/2672437170 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f57e0066fa0 msgr2=0x7f57e0154b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:40.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.527+0000 7f57ddd9b700 1 --2- 172.21.15.162:0/2672437170 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f57e0066fa0 0x7f57e0154b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:40.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.527+0000 7f57ddd9b700 1 -- 172.21.15.162:0/2672437170 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f57e00b7920 msgr2=0x7f57e00ae990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:40.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.527+0000 7f57ddd9b700 1 --2- 172.21.15.162:0/2672437170 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f57e00b7920 0x7f57e00ae990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:40.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.527+0000 7f57ddd9b700 1 -- 172.21.15.162:0/2672437170 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e00ae4f0 con 0x7f57e00b2da0 2020-11-28T21:51:40.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.527+0000 7f57ddd9b700 1 --2- 172.21.15.162:0/2672437170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57e00b2da0 0x7f57e00ad140 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7f57d4003070 tx=0x7f57d4008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:40.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.531+0000 7f57cf7fe700 1 -- 172.21.15.162:0/2672437170 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57d4009980 con 0x7f57e00b2da0 2020-11-28T21:51:40.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.531+0000 7f57cf7fe700 1 -- 172.21.15.162:0/2672437170 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d4009ae0 con 0x7f57e00b2da0 2020-11-28T21:51:40.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.531+0000 7f57df59e700 1 -- 172.21.15.162:0/2672437170 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57e0145a70 con 0x7f57e00b2da0 2020-11-28T21:51:40.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.531+0000 7f57cf7fe700 1 -- 172.21.15.162:0/2672437170 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57d4009e40 con 0x7f57e00b2da0 2020-11-28T21:51:40.543 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.531+0000 7f57df59e700 1 -- 172.21.15.162:0/2672437170 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e0145450 con 0x7f57e00b2da0 2020-11-28T21:51:40.545 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.531+0000 7f57cf7fe700 1 -- 172.21.15.162:0/2672437170 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f57d4013480 con 0x7f57e00b2da0 2020-11-28T21:51:40.545 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.531+0000 7f57df59e700 1 -- 172.21.15.162:0/2672437170 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57e00ae680 con 0x7f57e00b2da0 2020-11-28T21:51:40.546 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.531+0000 7f57cf7fe700 1 --2- 172.21.15.162:0/2672437170 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f57bc055330 0x7f57bc0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:40.546 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.531+0000 7f57cf7fe700 1 -- 172.21.15.162:0/2672437170 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f57d406e630 con 0x7f57e00b2da0 2020-11-28T21:51:40.546 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.531+0000 7f57de59c700 1 --2- 172.21.15.162:0/2672437170 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f57bc055330 0x7f57bc0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:40.547 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.531+0000 7f57de59c700 1 --2- 172.21.15.162:0/2672437170 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f57bc055330 0x7f57bc0577f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f57d001b000 tx=0x7f57d0016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:40.547 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.535+0000 7f57cf7fe700 1 -- 172.21.15.162:0/2672437170 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f57d4044af0 con 0x7f57e00b2da0 2020-11-28T21:51:40.671 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.675+0000 7f57df59e700 1 -- 172.21.15.162:0/2672437170 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57e0057050 con 0x7f57e00b2da0 2020-11-28T21:51:40.674 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.675+0000 7f57cf7fe700 1 -- 172.21.15.162:0/2672437170 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+13278 (secure 0 0 0) 0x7f57d4048110 con 0x7f57e00b2da0 2020-11-28T21:51:40.674 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:51:40.675 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":62,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:51:40.014323+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:51:40.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.679+0000 7f57df59e700 1 -- 172.21.15.162:0/2672437170 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f57bc055330 msgr2=0x7f57bc0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:40.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.679+0000 7f57df59e700 1 --2- 172.21.15.162:0/2672437170 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f57bc055330 0x7f57bc0577f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f57d001b000 tx=0x7f57d0016000).stop 2020-11-28T21:51:40.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.679+0000 7f57df59e700 1 -- 172.21.15.162:0/2672437170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57e00b2da0 msgr2=0x7f57e00ad140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:40.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.679+0000 7f57df59e700 1 --2- 172.21.15.162:0/2672437170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57e00b2da0 0x7f57e00ad140 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7f57d4003070 tx=0x7f57d4008040).stop 2020-11-28T21:51:40.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.679+0000 7f57df59e700 1 -- 172.21.15.162:0/2672437170 shutdown_connections 2020-11-28T21:51:40.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.679+0000 7f57df59e700 1 --2- 172.21.15.162:0/2672437170 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f57bc055330 0x7f57bc0577f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:40.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.679+0000 7f57df59e700 1 --2- 172.21.15.162:0/2672437170 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f57e0066fa0 0x7f57e0154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:40.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.679+0000 7f57df59e700 1 --2- 172.21.15.162:0/2672437170 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f57e00b7920 0x7f57e00ae990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:40.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.679+0000 7f57df59e700 1 --2- 172.21.15.162:0/2672437170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57e00b2da0 0x7f57e00ad140 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:40.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.679+0000 7f57df59e700 1 -- 172.21.15.162:0/2672437170 >> 172.21.15.162:0/2672437170 conn(0x7f57e005c440 msgr2=0x7f57e00b60b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:40.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.679+0000 7f57df59e700 1 -- 172.21.15.162:0/2672437170 shutdown_connections 2020-11-28T21:51:40.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.679+0000 7f57df59e700 1 -- 172.21.15.162:0/2672437170 wait complete. 2020-11-28T21:51:40.687 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-28T21:51:40.688 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:51:40.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.815+0000 7f7f40a3f700 1 Processor -- start 2020-11-28T21:51:40.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.815+0000 7f7f40a3f700 1 -- start start 2020-11-28T21:51:40.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.815+0000 7f7f40a3f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7f3c0af6a0 0x7f7f3c0b2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:40.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.815+0000 7f7f40a3f700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7f3c060a70 0x7f7f3c0afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:40.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.815+0000 7f7f40a3f700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f3c060f00 con 0x7f7f3c0b1eb0 2020-11-28T21:51:40.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.815+0000 7f7f40a3f700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f3c1471f0 con 0x7f7f3c060a70 2020-11-28T21:51:40.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.815+0000 7f7f40a3f700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3c060090 con 0x7f7f3c0af6a0 2020-11-28T21:51:40.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.815+0000 7f7f3bfff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7f3c060a70 0x7f7f3c0afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:40.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.815+0000 7f7f3bfff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7f3c060a70 0x7f7f3c0afdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40594/0 (socket says 172.21.15.162:40594) 2020-11-28T21:51:40.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.815+0000 7f7f3bfff700 1 -- 172.21.15.162:0/3964281256 learned_addr learned my addr 172.21.15.162:0/3964281256 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:40.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.815+0000 7f7f3affd700 1 --2- 172.21.15.162:0/3964281256 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7f3c0af6a0 0x7f7f3c0b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:40.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.815+0000 7f7f3bfff700 1 -- 172.21.15.162:0/3964281256 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7f3c0af6a0 msgr2=0x7f7f3c0b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:51:40.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.815+0000 7f7f3bfff700 1 --2- 172.21.15.162:0/3964281256 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7f3c0af6a0 0x7f7f3c0b2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:40.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.815+0000 7f7f3bfff700 1 -- 172.21.15.162:0/3964281256 >> v1:172.21.15.118:6789/0 conn(0x7f7f3c0b1eb0 legacy=0x7f7f3c144350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:40.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.815+0000 7f7f3bfff700 1 -- 172.21.15.162:0/3964281256 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f3c060330 con 0x7f7f3c060a70 2020-11-28T21:51:40.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.815+0000 7f7f3affd700 1 --2- 172.21.15.162:0/3964281256 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7f3c0af6a0 0x7f7f3c0b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:51:40.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.815+0000 7f7f3bfff700 1 --2- 172.21.15.162:0/3964281256 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f3c060a70 0x7f7f3c0afdc0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7f7f34002f30 tx=0x7f7f34006e80).ready entity=mon.1 client_cookie=aaf9c8eabe06cfda server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:40.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.815+0000 7f7f3a7fc700 1 -- 172.21.15.162:0/3964281256 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f340097f0 con 0x7f7f3c060a70 2020-11-28T21:51:40.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.819+0000 7f7f3a7fc700 1 -- 172.21.15.162:0/3964281256 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f34009950 con 0x7f7f3c060a70 2020-11-28T21:51:40.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.819+0000 7f7f3a7fc700 1 -- 172.21.15.162:0/3964281256 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f34009cb0 con 0x7f7f3c060a70 2020-11-28T21:51:40.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.819+0000 7f7f40a3f700 1 -- 172.21.15.162:0/3964281256 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f3c060a70 msgr2=0x7f7f3c0afdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:40.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.819+0000 7f7f40a3f700 1 --2- 172.21.15.162:0/3964281256 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f3c060a70 0x7f7f3c0afdc0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7f7f34002f30 tx=0x7f7f34006e80).stop 2020-11-28T21:51:40.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.819+0000 7f7f40a3f700 1 -- 172.21.15.162:0/3964281256 shutdown_connections 2020-11-28T21:51:40.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.819+0000 7f7f40a3f700 1 --2- 172.21.15.162:0/3964281256 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f3c060a70 0x7f7f3c0afdc0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:40.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.819+0000 7f7f40a3f700 1 --2- 172.21.15.162:0/3964281256 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7f3c0af6a0 0x7f7f3c0b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:40.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.819+0000 7f7f40a3f700 1 -- 172.21.15.162:0/3964281256 >> 172.21.15.162:0/3964281256 conn(0x7f7f3c0b57d0 msgr2=0x7f7f3c0b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:40.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.819+0000 7f7f40a3f700 1 -- 172.21.15.162:0/3964281256 shutdown_connections 2020-11-28T21:51:40.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.819+0000 7f7f40a3f700 1 -- 172.21.15.162:0/3964281256 wait complete. 2020-11-28T21:51:40.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.819+0000 7f7f40a3f700 1 Processor -- start 2020-11-28T21:51:40.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.819+0000 7f7f40a3f700 1 -- start start 2020-11-28T21:51:40.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.819+0000 7f7f40a3f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7f3c060a70 0x7f7f3c145a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:40.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.819+0000 7f7f40a3f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f3c0af6a0 0x7f7f3c156a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:40.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.819+0000 7f7f40a3f700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7f3c0b1eb0 0x7f7f3c156f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:40.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.819+0000 7f7f40a3f700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3c156360 con 0x7f7f3c0b1eb0 2020-11-28T21:51:40.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.819+0000 7f7f40a3f700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3c065ad0 con 0x7f7f3c0af6a0 2020-11-28T21:51:40.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.819+0000 7f7f40a3f700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3c065d70 con 0x7f7f3c060a70 2020-11-28T21:51:40.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.819+0000 7f7f3affd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f3c0af6a0 0x7f7f3c156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:40.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.819+0000 7f7f3affd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f3c0af6a0 0x7f7f3c156a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40598/0 (socket says 172.21.15.162:40598) 2020-11-28T21:51:40.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.819+0000 7f7f3affd700 1 -- 172.21.15.162:0/810742006 learned_addr learned my addr 172.21.15.162:0/810742006 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:40.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.819+0000 7f7f3bfff700 1 --2- 172.21.15.162:0/810742006 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7f3c0b1eb0 0x7f7f3c156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:40.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.819+0000 7f7f3b7fe700 1 --2- 172.21.15.162:0/810742006 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7f3c060a70 0x7f7f3c145a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:40.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.823+0000 7f7f3affd700 1 -- 172.21.15.162:0/810742006 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7f3c060a70 msgr2=0x7f7f3c145a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:40.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.823+0000 7f7f3affd700 1 --2- 172.21.15.162:0/810742006 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7f3c060a70 0x7f7f3c145a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:40.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.823+0000 7f7f3affd700 1 -- 172.21.15.162:0/810742006 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7f3c0b1eb0 msgr2=0x7f7f3c156f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:40.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.823+0000 7f7f3affd700 1 --2- 172.21.15.162:0/810742006 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7f3c0b1eb0 0x7f7f3c156f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:40.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.823+0000 7f7f3affd700 1 -- 172.21.15.162:0/810742006 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f3c066010 con 0x7f7f3c0af6a0 2020-11-28T21:51:40.838 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.823+0000 7f7f3affd700 1 --2- 172.21.15.162:0/810742006 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f3c0af6a0 0x7f7f3c156a60 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f7f2c007000 tx=0x7f7f2c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:40.844 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.823+0000 7f7f38ff9700 1 -- 172.21.15.162:0/810742006 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f2c00a820 con 0x7f7f3c0af6a0 2020-11-28T21:51:40.844 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.823+0000 7f7f38ff9700 1 -- 172.21.15.162:0/810742006 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f2c00a980 con 0x7f7f3c0af6a0 2020-11-28T21:51:40.844 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.823+0000 7f7f40a3f700 1 -- 172.21.15.162:0/810742006 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f3c1565c0 con 0x7f7f3c0af6a0 2020-11-28T21:51:40.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.823+0000 7f7f38ff9700 1 -- 172.21.15.162:0/810742006 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f2c00ace0 con 0x7f7f3c0af6a0 2020-11-28T21:51:40.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.823+0000 7f7f40a3f700 1 -- 172.21.15.162:0/810742006 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f3c067170 con 0x7f7f3c0af6a0 2020-11-28T21:51:40.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.823+0000 7f7f40a3f700 1 -- 172.21.15.162:0/810742006 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f3c066930 con 0x7f7f3c0af6a0 2020-11-28T21:51:40.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.823+0000 7f7f38ff9700 1 -- 172.21.15.162:0/810742006 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7f2c01b460 con 0x7f7f3c0af6a0 2020-11-28T21:51:40.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.823+0000 7f7f38ff9700 1 --2- 172.21.15.162:0/810742006 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7f200553b0 0x7f7f20057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:40.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.823+0000 7f7f38ff9700 1 -- 172.21.15.162:0/810742006 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f7f2c06dfe0 con 0x7f7f3c0af6a0 2020-11-28T21:51:40.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.827+0000 7f7f3b7fe700 1 --2- 172.21.15.162:0/810742006 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7f200553b0 0x7f7f20057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:40.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.827+0000 7f7f3b7fe700 1 --2- 172.21.15.162:0/810742006 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7f200553b0 0x7f7f20057870 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f7f2400d000 tx=0x7f7f24007b40).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:40.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.831+0000 7f7f38ff9700 1 -- 172.21.15.162:0/810742006 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7f3c066930 con 0x7f7f3c0af6a0 2020-11-28T21:51:40.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.971+0000 7f7f40a3f700 1 -- 172.21.15.162:0/810742006 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f7f3c066930 con 0x7f7f3c0af6a0 2020-11-28T21:51:40.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.975+0000 7f7f38ff9700 1 -- 172.21.15.162:0/810742006 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v62) v1 ==== 110+0+10 (secure 0 0 0) 0x7f7f3c066930 con 0x7f7f3c0af6a0 2020-11-28T21:51:40.972 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:51:40.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.975+0000 7f7f40a3f700 1 -- 172.21.15.162:0/810742006 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7f200553b0 msgr2=0x7f7f20057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:40.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.975+0000 7f7f40a3f700 1 --2- 172.21.15.162:0/810742006 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7f200553b0 0x7f7f20057870 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f7f2400d000 tx=0x7f7f24007b40).stop 2020-11-28T21:51:40.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.975+0000 7f7f40a3f700 1 -- 172.21.15.162:0/810742006 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f3c0af6a0 msgr2=0x7f7f3c156a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:40.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.975+0000 7f7f40a3f700 1 --2- 172.21.15.162:0/810742006 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f3c0af6a0 0x7f7f3c156a60 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f7f2c007000 tx=0x7f7f2c008040).stop 2020-11-28T21:51:40.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.975+0000 7f7f40a3f700 1 -- 172.21.15.162:0/810742006 shutdown_connections 2020-11-28T21:51:40.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.975+0000 7f7f40a3f700 1 --2- 172.21.15.162:0/810742006 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7f200553b0 0x7f7f20057870 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:40.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.975+0000 7f7f40a3f700 1 --2- 172.21.15.162:0/810742006 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7f3c060a70 0x7f7f3c145a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:40.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.975+0000 7f7f40a3f700 1 --2- 172.21.15.162:0/810742006 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7f3c0b1eb0 0x7f7f3c156f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:40.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.975+0000 7f7f40a3f700 1 --2- 172.21.15.162:0/810742006 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f3c0af6a0 0x7f7f3c156a60 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:40.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.975+0000 7f7f40a3f700 1 -- 172.21.15.162:0/810742006 >> 172.21.15.162:0/810742006 conn(0x7f7f3c0b57d0 msgr2=0x7f7f3c05c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:40.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.975+0000 7f7f40a3f700 1 -- 172.21.15.162:0/810742006 shutdown_connections 2020-11-28T21:51:40.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:40.975+0000 7f7f40a3f700 1 -- 172.21.15.162:0/810742006 wait complete. 2020-11-28T21:51:40.989 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:51:41.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.111+0000 7fe041d60700 1 Processor -- start 2020-11-28T21:51:41.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.111+0000 7fe041d60700 1 -- start start 2020-11-28T21:51:41.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.111+0000 7fe041d60700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe03c0b2d00 0x7fe03c05c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:41.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.111+0000 7fe041d60700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe03c0b3ab0 0x7fe03c0b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:41.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.111+0000 7fe041d60700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe03c142ad0 0x7fe03c142f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:41.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.111+0000 7fe041d60700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe03c0b0e20 con 0x7fe03c0b2d00 2020-11-28T21:51:41.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.111+0000 7fe041d60700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe03c05baf0 con 0x7fe03c142ad0 2020-11-28T21:51:41.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.111+0000 7fe041d60700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c05bd90 con 0x7fe03c0b3ab0 2020-11-28T21:51:41.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.111+0000 7fe04155f700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe03c142ad0 0x7fe03c142f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:41.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.111+0000 7fe04155f700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe03c142ad0 0x7fe03c142f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40608/0 (socket says 172.21.15.162:40608) 2020-11-28T21:51:41.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.111+0000 7fe04155f700 1 -- 172.21.15.162:0/777348199 learned_addr learned my addr 172.21.15.162:0/777348199 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:41.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.111+0000 7fe03bfff700 1 --2- 172.21.15.162:0/777348199 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe03c0b3ab0 0x7fe03c0b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:41.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.111+0000 7fe040d5e700 1 --2- 172.21.15.162:0/777348199 >> v2:172.21.15.118:3300/0 conn(0x7fe03c0b2d00 0x7fe03c05c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:41.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.111+0000 7fe04155f700 1 -- 172.21.15.162:0/777348199 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe03c0b3ab0 msgr2=0x7fe03c0b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:51:41.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.111+0000 7fe04155f700 1 --2- 172.21.15.162:0/777348199 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe03c0b3ab0 0x7fe03c0b30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:41.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.111+0000 7fe04155f700 1 -- 172.21.15.162:0/777348199 >> v2:172.21.15.118:3300/0 conn(0x7fe03c0b2d00 msgr2=0x7fe03c05c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:51:41.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.111+0000 7fe04155f700 1 --2- 172.21.15.162:0/777348199 >> v2:172.21.15.118:3300/0 conn(0x7fe03c0b2d00 0x7fe03c05c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:41.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.111+0000 7fe04155f700 1 -- 172.21.15.162:0/777348199 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe03c05c030 con 0x7fe03c142ad0 2020-11-28T21:51:41.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.111+0000 7fe03bfff700 1 --2- 172.21.15.162:0/777348199 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe03c0b3ab0 0x7fe03c0b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:51:41.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.111+0000 7fe04155f700 1 --2- 172.21.15.162:0/777348199 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe03c142ad0 0x7fe03c142f60 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7fe0340032b0 tx=0x7fe034009270).ready entity=mon.1 client_cookie=88910829311bae7a server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:41.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.111+0000 7fe03b7fe700 1 -- 172.21.15.162:0/777348199 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe03400bc00 con 0x7fe03c142ad0 2020-11-28T21:51:41.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.111+0000 7fe03b7fe700 1 -- 172.21.15.162:0/777348199 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe03400bd60 con 0x7fe03c142ad0 2020-11-28T21:51:41.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.111+0000 7fe041d60700 1 -- 172.21.15.162:0/777348199 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe03c142ad0 msgr2=0x7fe03c142f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:41.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.111+0000 7fe03b7fe700 1 -- 172.21.15.162:0/777348199 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe034011400 con 0x7fe03c142ad0 2020-11-28T21:51:41.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.115+0000 7fe041d60700 1 --2- 172.21.15.162:0/777348199 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe03c142ad0 0x7fe03c142f60 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7fe0340032b0 tx=0x7fe034009270).stop 2020-11-28T21:51:41.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.115+0000 7fe041d60700 1 -- 172.21.15.162:0/777348199 shutdown_connections 2020-11-28T21:51:41.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.115+0000 7fe041d60700 1 --2- 172.21.15.162:0/777348199 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe03c142ad0 0x7fe03c142f60 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:41.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.115+0000 7fe041d60700 1 --2- 172.21.15.162:0/777348199 >> v2:172.21.15.118:3300/0 conn(0x7fe03c0b2d00 0x7fe03c05c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:41.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.115+0000 7fe041d60700 1 --2- 172.21.15.162:0/777348199 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe03c0b3ab0 0x7fe03c0b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:41.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.115+0000 7fe041d60700 1 -- 172.21.15.162:0/777348199 >> 172.21.15.162:0/777348199 conn(0x7fe03c0604c0 msgr2=0x7fe03c0608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:41.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.115+0000 7fe041d60700 1 -- 172.21.15.162:0/777348199 shutdown_connections 2020-11-28T21:51:41.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.115+0000 7fe041d60700 1 -- 172.21.15.162:0/777348199 wait complete. 2020-11-28T21:51:41.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.115+0000 7fe041d60700 1 Processor -- start 2020-11-28T21:51:41.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.115+0000 7fe041d60700 1 -- start start 2020-11-28T21:51:41.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.115+0000 7fe041d60700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe03c0b2d00 0x7fe03c13eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:41.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.115+0000 7fe041d60700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe03c0b3ab0 0x7fe03c0af910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:41.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.115+0000 7fe041d60700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe03c142ad0 0x7fe03c154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:41.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.115+0000 7fe041d60700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c147d40 con 0x7fe03c142ad0 2020-11-28T21:51:41.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.115+0000 7fe041d60700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c147fa0 con 0x7fe03c0b3ab0 2020-11-28T21:51:41.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.115+0000 7fe041d60700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c0af1d0 con 0x7fe03c0b2d00 2020-11-28T21:51:41.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.115+0000 7fe03bfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe03c0b3ab0 0x7fe03c0af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:41.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.115+0000 7fe03bfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe03c0b3ab0 0x7fe03c0af910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40612/0 (socket says 172.21.15.162:40612) 2020-11-28T21:51:41.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.115+0000 7fe03bfff700 1 -- 172.21.15.162:0/1781320246 learned_addr learned my addr 172.21.15.162:0/1781320246 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:41.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.115+0000 7fe04155f700 1 --2- 172.21.15.162:0/1781320246 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe03c142ad0 0x7fe03c154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:41.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.115+0000 7fe040d5e700 1 --2- 172.21.15.162:0/1781320246 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe03c0b2d00 0x7fe03c13eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:41.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.115+0000 7fe03bfff700 1 -- 172.21.15.162:0/1781320246 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe03c0b2d00 msgr2=0x7fe03c13eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:41.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.115+0000 7fe03bfff700 1 --2- 172.21.15.162:0/1781320246 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe03c0b2d00 0x7fe03c13eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:41.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.115+0000 7fe03bfff700 1 -- 172.21.15.162:0/1781320246 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe03c142ad0 msgr2=0x7fe03c154d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:41.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.115+0000 7fe03bfff700 1 --2- 172.21.15.162:0/1781320246 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe03c142ad0 0x7fe03c154d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:41.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.119+0000 7fe03bfff700 1 -- 172.21.15.162:0/1781320246 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe03c0af470 con 0x7fe03c0b3ab0 2020-11-28T21:51:41.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.119+0000 7fe04155f700 1 --2- 172.21.15.162:0/1781320246 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe03c142ad0 0x7fe03c154d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:51:41.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.119+0000 7fe03bfff700 1 --2- 172.21.15.162:0/1781320246 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe03c0b3ab0 0x7fe03c0af910 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7fe03000a000 tx=0x7fe030008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:41.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.119+0000 7fe039ffb700 1 -- 172.21.15.162:0/1781320246 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe030009920 con 0x7fe03c0b3ab0 2020-11-28T21:51:41.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.119+0000 7fe039ffb700 1 -- 172.21.15.162:0/1781320246 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe030009a80 con 0x7fe03c0b3ab0 2020-11-28T21:51:41.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.119+0000 7fe041d60700 1 -- 172.21.15.162:0/1781320246 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe03c0afe00 con 0x7fe03c0b3ab0 2020-11-28T21:51:41.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.119+0000 7fe039ffb700 1 -- 172.21.15.162:0/1781320246 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe030009de0 con 0x7fe03c0b3ab0 2020-11-28T21:51:41.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.119+0000 7fe041d60700 1 -- 172.21.15.162:0/1781320246 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe03c0aeea0 con 0x7fe03c0b3ab0 2020-11-28T21:51:41.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.119+0000 7fe041d60700 1 -- 172.21.15.162:0/1781320246 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe03c057050 con 0x7fe03c0b3ab0 2020-11-28T21:51:41.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.119+0000 7fe039ffb700 1 -- 172.21.15.162:0/1781320246 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe030013480 con 0x7fe03c0b3ab0 2020-11-28T21:51:41.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.119+0000 7fe039ffb700 1 --2- 172.21.15.162:0/1781320246 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe01c055330 0x7fe01c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:41.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.119+0000 7fe039ffb700 1 -- 172.21.15.162:0/1781320246 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fe03006e330 con 0x7fe03c0b3ab0 2020-11-28T21:51:41.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.119+0000 7fe040d5e700 1 --2- 172.21.15.162:0/1781320246 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe01c055330 0x7fe01c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:41.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.119+0000 7fe040d5e700 1 --2- 172.21.15.162:0/1781320246 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe01c055330 0x7fe01c0577f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7fe02c00c000 tx=0x7fe02c008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:41.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.131+0000 7fe039ffb700 1 -- 172.21.15.162:0/1781320246 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe030044900 con 0x7fe03c0b3ab0 2020-11-28T21:51:41.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.275+0000 7fe041d60700 1 -- 172.21.15.162:0/1781320246 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe03c05d250 con 0x7fe03c0b3ab0 2020-11-28T21:51:41.273 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.275+0000 7fe039ffb700 1 -- 172.21.15.162:0/1781320246 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v62) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe030047f20 con 0x7fe03c0b3ab0 2020-11-28T21:51:41.274 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:51:41.277 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.279+0000 7fe041d60700 1 -- 172.21.15.162:0/1781320246 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe01c055330 msgr2=0x7fe01c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:41.278 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.279+0000 7fe041d60700 1 --2- 172.21.15.162:0/1781320246 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe01c055330 0x7fe01c0577f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7fe02c00c000 tx=0x7fe02c008040).stop 2020-11-28T21:51:41.278 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.279+0000 7fe041d60700 1 -- 172.21.15.162:0/1781320246 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe03c0b3ab0 msgr2=0x7fe03c0af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:41.279 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.279+0000 7fe041d60700 1 --2- 172.21.15.162:0/1781320246 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe03c0b3ab0 0x7fe03c0af910 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7fe03000a000 tx=0x7fe030008040).stop 2020-11-28T21:51:41.279 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.279+0000 7fe041d60700 1 -- 172.21.15.162:0/1781320246 shutdown_connections 2020-11-28T21:51:41.280 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.279+0000 7fe041d60700 1 --2- 172.21.15.162:0/1781320246 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe01c055330 0x7fe01c0577f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:41.280 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.279+0000 7fe041d60700 1 --2- 172.21.15.162:0/1781320246 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe03c0b2d00 0x7fe03c13eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:41.280 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.279+0000 7fe041d60700 1 --2- 172.21.15.162:0/1781320246 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe03c142ad0 0x7fe03c154d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:41.281 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.279+0000 7fe041d60700 1 --2- 172.21.15.162:0/1781320246 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe03c0b3ab0 0x7fe03c0af910 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:41.281 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.279+0000 7fe041d60700 1 -- 172.21.15.162:0/1781320246 >> 172.21.15.162:0/1781320246 conn(0x7fe03c0604c0 msgr2=0x7fe03c141f70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:41.281 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.279+0000 7fe041d60700 1 -- 172.21.15.162:0/1781320246 shutdown_connections 2020-11-28T21:51:41.282 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.279+0000 7fe041d60700 1 -- 172.21.15.162:0/1781320246 wait complete. 2020-11-28T21:51:41.304 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2020-11-28T21:51:41.412 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.415+0000 7fccab59e700 1 Processor -- start 2020-11-28T21:51:41.413 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.415+0000 7fccab59e700 1 -- start start 2020-11-28T21:51:41.419 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.415+0000 7fccab59e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fccac13f400 0x7fccac09b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:41.419 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.419+0000 7fccab59e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fccac0b1940 0x7fccac13fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:41.420 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.419+0000 7fccab59e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccac097a90 con 0x7fccac0b45d0 2020-11-28T21:51:41.420 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.419+0000 7fccab59e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccac098180 con 0x7fccac13f400 2020-11-28T21:51:41.421 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.419+0000 7fccab59e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccac0b49b0 con 0x7fccac0b1940 2020-11-28T21:51:41.421 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.419+0000 7fccaad9d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fccac0b1940 0x7fccac13fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:41.421 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.419+0000 7fccaad9d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fccac0b1940 0x7fccac13fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40622/0 (socket says 172.21.15.162:40622) 2020-11-28T21:51:41.422 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.419+0000 7fccaad9d700 1 -- 172.21.15.162:0/1991039703 learned_addr learned my addr 172.21.15.162:0/1991039703 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:41.422 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.419+0000 7fcca1d9b700 1 --2- 172.21.15.162:0/1991039703 >> v2:172.21.15.118:3300/0 conn(0x7fccac13f400 0x7fccac09b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:41.422 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.419+0000 7fcca9d9b700 1 -- 172.21.15.162:0/1991039703 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2127625097 0 0) 0x7fccac097a90 con 0x7fccac0b45d0 2020-11-28T21:51:41.423 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.419+0000 7fccaad9d700 1 -- 172.21.15.162:0/1991039703 >> v2:172.21.15.118:3300/0 conn(0x7fccac13f400 msgr2=0x7fccac09b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:51:41.423 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.419+0000 7fccaad9d700 1 --2- 172.21.15.162:0/1991039703 >> v2:172.21.15.118:3300/0 conn(0x7fccac13f400 0x7fccac09b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:41.423 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.419+0000 7fccaad9d700 1 -- 172.21.15.162:0/1991039703 >> v1:172.21.15.118:6789/0 conn(0x7fccac0b45d0 legacy=0x7fccac0afe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:41.424 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.419+0000 7fccaad9d700 1 -- 172.21.15.162:0/1991039703 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccac0987e0 con 0x7fccac0b1940 2020-11-28T21:51:41.424 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.419+0000 7fccaad9d700 1 --2- 172.21.15.162:0/1991039703 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccac0b1940 0x7fccac13fab0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7fcca40032b0 tx=0x7fcca4009270).ready entity=mon.1 client_cookie=58ad53fb54155ac2 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:41.425 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.419+0000 7fcca9d9b700 1 -- 172.21.15.162:0/1991039703 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcca4009e60 con 0x7fccac0b1940 2020-11-28T21:51:41.425 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.419+0000 7fcca9d9b700 1 -- 172.21.15.162:0/1991039703 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcca400bc00 con 0x7fccac0b1940 2020-11-28T21:51:41.425 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.419+0000 7fcca9d9b700 1 -- 172.21.15.162:0/1991039703 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcca400bd70 con 0x7fccac0b1940 2020-11-28T21:51:41.426 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.419+0000 7fccab59e700 1 -- 172.21.15.162:0/1991039703 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccac0b1940 msgr2=0x7fccac13fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:41.426 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.419+0000 7fccab59e700 1 --2- 172.21.15.162:0/1991039703 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccac0b1940 0x7fccac13fab0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7fcca40032b0 tx=0x7fcca4009270).stop 2020-11-28T21:51:41.426 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.419+0000 7fccab59e700 1 -- 172.21.15.162:0/1991039703 shutdown_connections 2020-11-28T21:51:41.427 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.419+0000 7fccab59e700 1 --2- 172.21.15.162:0/1991039703 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccac0b1940 0x7fccac13fab0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:41.427 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.419+0000 7fccab59e700 1 --2- 172.21.15.162:0/1991039703 >> v2:172.21.15.118:3300/0 conn(0x7fccac13f400 0x7fccac09b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:41.427 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.419+0000 7fccab59e700 1 -- 172.21.15.162:0/1991039703 >> 172.21.15.162:0/1991039703 conn(0x7fccac0b54f0 msgr2=0x7fccac0b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:41.428 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.419+0000 7fccab59e700 1 -- 172.21.15.162:0/1991039703 shutdown_connections 2020-11-28T21:51:41.428 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.419+0000 7fccab59e700 1 -- 172.21.15.162:0/1991039703 wait complete. 2020-11-28T21:51:41.428 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.423+0000 7fccab59e700 1 Processor -- start 2020-11-28T21:51:41.429 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.423+0000 7fccab59e700 1 -- start start 2020-11-28T21:51:41.433 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.423+0000 7fccab59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fccac0b1940 0x7fccac13e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:41.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.423+0000 7fccab59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fccac0b45d0 0x7fccac143b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:41.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.423+0000 7fccab59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccac13f400 0x7fccac144030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:41.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.423+0000 7fccab59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccac144520 con 0x7fccac0b1940 2020-11-28T21:51:41.435 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.423+0000 7fccab59e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccac13ecb0 con 0x7fccac13f400 2020-11-28T21:51:41.435 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.423+0000 7fccab59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccac145de0 con 0x7fccac0b45d0 2020-11-28T21:51:41.435 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.423+0000 7fccaad9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccac13f400 0x7fccac144030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:41.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.423+0000 7fccaa59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fccac0b1940 0x7fccac13e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:41.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.423+0000 7fccaad9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccac13f400 0x7fccac144030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40628/0 (socket says 172.21.15.162:40628) 2020-11-28T21:51:41.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.423+0000 7fccaad9d700 1 -- 172.21.15.162:0/3291690739 learned_addr learned my addr 172.21.15.162:0/3291690739 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:41.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.423+0000 7fcca1d9b700 1 --2- 172.21.15.162:0/3291690739 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fccac0b45d0 0x7fccac143b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:41.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.423+0000 7fccaa59c700 1 -- 172.21.15.162:0/3291690739 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fccac0b45d0 msgr2=0x7fccac143b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:41.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.423+0000 7fccaa59c700 1 --2- 172.21.15.162:0/3291690739 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fccac0b45d0 0x7fccac143b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:41.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.423+0000 7fccaa59c700 1 -- 172.21.15.162:0/3291690739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccac13f400 msgr2=0x7fccac144030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:41.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.423+0000 7fccaa59c700 1 --2- 172.21.15.162:0/3291690739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccac13f400 0x7fccac144030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:41.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.423+0000 7fccaa59c700 1 -- 172.21.15.162:0/3291690739 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccac145b40 con 0x7fccac0b1940 2020-11-28T21:51:41.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.423+0000 7fccaad9d700 1 --2- 172.21.15.162:0/3291690739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccac13f400 0x7fccac144030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:51:41.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.427+0000 7fccaa59c700 1 --2- 172.21.15.162:0/3291690739 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fccac0b1940 0x7fccac13e060 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7fcc9c000b80 tx=0x7fcc9c007980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:41.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.427+0000 7fcca3fff700 1 -- 172.21.15.162:0/3291690739 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc9c013410 con 0x7fccac0b1940 2020-11-28T21:51:41.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.427+0000 7fcca3fff700 1 -- 172.21.15.162:0/3291690739 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc9c013570 con 0x7fccac0b1940 2020-11-28T21:51:41.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.427+0000 7fcca3fff700 1 -- 172.21.15.162:0/3291690739 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc9c0138d0 con 0x7fccac0b1940 2020-11-28T21:51:41.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.427+0000 7fccab59e700 1 -- 172.21.15.162:0/3291690739 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccac138530 con 0x7fccac0b1940 2020-11-28T21:51:41.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.427+0000 7fccab59e700 1 -- 172.21.15.162:0/3291690739 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccac138c70 con 0x7fccac0b1940 2020-11-28T21:51:41.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.427+0000 7fccab59e700 1 -- 172.21.15.162:0/3291690739 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccac1389d0 con 0x7fccac0b1940 2020-11-28T21:51:41.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.427+0000 7fcca3fff700 1 -- 172.21.15.162:0/3291690739 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fcc9c010030 con 0x7fccac0b1940 2020-11-28T21:51:41.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.427+0000 7fcca3fff700 1 --2- 172.21.15.162:0/3291690739 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcc90055330 0x7fcc900577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:41.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.427+0000 7fcca3fff700 1 -- 172.21.15.162:0/3291690739 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fcc9c06db80 con 0x7fccac0b1940 2020-11-28T21:51:41.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.427+0000 7fcca1d9b700 1 --2- 172.21.15.162:0/3291690739 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcc90055330 0x7fcc900577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:41.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.431+0000 7fcca1d9b700 1 --2- 172.21.15.162:0/3291690739 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcc90055330 0x7fcc900577f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7fcc9400b000 tx=0x7fcc94006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:41.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.439+0000 7fcca3fff700 1 -- 172.21.15.162:0/3291690739 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fcc9c0440a0 con 0x7fccac0b1940 2020-11-28T21:51:41.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.583+0000 7fccab59e700 1 -- 172.21.15.162:0/3291690739 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fccac1389d0 con 0x7fccac0b1940 2020-11-28T21:51:41.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.583+0000 7fcca3fff700 1 -- 172.21.15.162:0/3291690739 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v62) v1 ==== 84+0+143 (secure 0 0 0) 0x7fccac1389d0 con 0x7fccac0b1940 2020-11-28T21:51:41.581 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:51:41.582 INFO:teuthology.orchestra.run.smithi162.stdout:[ 2020-11-28T21:51:41.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:41.582 INFO:teuthology.orchestra.run.smithi162.stdout: "poolnum": 1, 2020-11-28T21:51:41.583 INFO:teuthology.orchestra.run.smithi162.stdout: "poolname": "device_health_metrics" 2020-11-28T21:51:41.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:51:41.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:51:41.584 INFO:teuthology.orchestra.run.smithi162.stdout: "poolnum": 2, 2020-11-28T21:51:41.584 INFO:teuthology.orchestra.run.smithi162.stdout: "poolname": "rbd" 2020-11-28T21:51:41.585 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:51:41.585 INFO:teuthology.orchestra.run.smithi162.stdout:] 2020-11-28T21:51:41.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.583+0000 7fccab59e700 1 -- 172.21.15.162:0/3291690739 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcc90055330 msgr2=0x7fcc900577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:41.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.583+0000 7fccab59e700 1 --2- 172.21.15.162:0/3291690739 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcc90055330 0x7fcc900577f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7fcc9400b000 tx=0x7fcc94006dc0).stop 2020-11-28T21:51:41.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.583+0000 7fccab59e700 1 -- 172.21.15.162:0/3291690739 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fccac0b1940 msgr2=0x7fccac13e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:41.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.583+0000 7fccab59e700 1 --2- 172.21.15.162:0/3291690739 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fccac0b1940 0x7fccac13e060 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7fcc9c000b80 tx=0x7fcc9c007980).stop 2020-11-28T21:51:41.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.583+0000 7fccab59e700 1 -- 172.21.15.162:0/3291690739 shutdown_connections 2020-11-28T21:51:41.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.583+0000 7fccab59e700 1 --2- 172.21.15.162:0/3291690739 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcc90055330 0x7fcc900577f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:41.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.583+0000 7fccab59e700 1 --2- 172.21.15.162:0/3291690739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccac13f400 0x7fccac144030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:41.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.583+0000 7fccab59e700 1 --2- 172.21.15.162:0/3291690739 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fccac0b1940 0x7fccac13e060 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:41.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.583+0000 7fccab59e700 1 --2- 172.21.15.162:0/3291690739 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fccac0b45d0 0x7fccac143b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:41.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.583+0000 7fccab59e700 1 -- 172.21.15.162:0/3291690739 >> 172.21.15.162:0/3291690739 conn(0x7fccac0b54f0 msgr2=0x7fccac0afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:41.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.583+0000 7fccab59e700 1 -- 172.21.15.162:0/3291690739 shutdown_connections 2020-11-28T21:51:41.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.583+0000 7fccab59e700 1 -- 172.21.15.162:0/3291690739 wait complete. 2020-11-28T21:51:41.595 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-11-28T21:51:41.596 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2020-11-28T21:51:41.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.715+0000 7f82dffff700 1 Processor -- start 2020-11-28T21:51:41.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.715+0000 7f82dffff700 1 -- start start 2020-11-28T21:51:41.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.715+0000 7f82dffff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f82e00b3950 0x7f82e00b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:41.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.715+0000 7f82dffff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f82e00b44e0 0x7f82e00b2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:41.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.715+0000 7f82dffff700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82e00b1f50 con 0x7f82e00b18c0 2020-11-28T21:51:41.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.715+0000 7f82dffff700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82e00aab50 con 0x7f82e00b44e0 2020-11-28T21:51:41.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.715+0000 7f82dffff700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82e0143e90 con 0x7f82e00b3950 2020-11-28T21:51:41.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.715+0000 7f82deffd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f82e00b3950 0x7f82e00b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:41.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.715+0000 7f82de7fc700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f82e00b18c0 0x7f82e013dd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:47476/0 (socket says 172.21.15.162:47476) 2020-11-28T21:51:41.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.715+0000 7f82de7fc700 1 -- 172.21.15.162:0/1765683926 learned_addr learned my addr 172.21.15.162:0/1765683926 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:51:41.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.719+0000 7f82df7fe700 1 --2- 172.21.15.162:0/1765683926 >> v2:172.21.15.118:3300/0 conn(0x7f82e00b44e0 0x7f82e00b2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:41.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.719+0000 7f82ddffb700 1 -- 172.21.15.162:0/1765683926 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2115865026 0 0) 0x7f82e00b1f50 con 0x7f82e00b18c0 2020-11-28T21:51:41.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.719+0000 7f82ddffb700 1 -- 172.21.15.162:0/1765683926 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82c0003230 con 0x7f82e00b18c0 2020-11-28T21:51:41.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.719+0000 7f82deffd700 1 -- 172.21.15.162:0/1765683926 >> v2:172.21.15.118:3300/0 conn(0x7f82e00b44e0 msgr2=0x7f82e00b2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:51:41.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.719+0000 7f82deffd700 1 --2- 172.21.15.162:0/1765683926 >> v2:172.21.15.118:3300/0 conn(0x7f82e00b44e0 0x7f82e00b2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:41.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.719+0000 7f82deffd700 1 -- 172.21.15.162:0/1765683926 >> v1:172.21.15.162:6789/0 conn(0x7f82e00b18c0 legacy=0x7f82e013dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:41.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.719+0000 7f82deffd700 1 -- 172.21.15.162:0/1765683926 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e00aa8b0 con 0x7f82e00b3950 2020-11-28T21:51:41.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.719+0000 7f82deffd700 1 --2- 172.21.15.162:0/1765683926 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82e00b3950 0x7f82e00b3d30 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7f82d0002f30 tx=0x7f82d0006dc0).ready entity=mon.1 client_cookie=97667013aef908a0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:41.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.719+0000 7f82ddffb700 1 -- 172.21.15.162:0/1765683926 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82d00096f0 con 0x7f82e00b3950 2020-11-28T21:51:41.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.719+0000 7f82ddffb700 1 -- 172.21.15.162:0/1765683926 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d0009850 con 0x7f82e00b3950 2020-11-28T21:51:41.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.719+0000 7f82ddffb700 1 -- 172.21.15.162:0/1765683926 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82d0009bb0 con 0x7f82e00b3950 2020-11-28T21:51:41.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.719+0000 7f82dffff700 1 -- 172.21.15.162:0/1765683926 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82e00b3950 msgr2=0x7f82e00b3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:41.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.719+0000 7f82dffff700 1 --2- 172.21.15.162:0/1765683926 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82e00b3950 0x7f82e00b3d30 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7f82d0002f30 tx=0x7f82d0006dc0).stop 2020-11-28T21:51:41.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.719+0000 7f82dffff700 1 -- 172.21.15.162:0/1765683926 shutdown_connections 2020-11-28T21:51:41.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.719+0000 7f82dffff700 1 --2- 172.21.15.162:0/1765683926 >> v2:172.21.15.118:3300/0 conn(0x7f82e00b44e0 0x7f82e00b2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:41.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.719+0000 7f82dffff700 1 --2- 172.21.15.162:0/1765683926 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82e00b3950 0x7f82e00b3d30 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:41.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.719+0000 7f82dffff700 1 -- 172.21.15.162:0/1765683926 >> 172.21.15.162:0/1765683926 conn(0x7f82e0062630 msgr2=0x7f82e0062a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:41.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.719+0000 7f82dffff700 1 -- 172.21.15.162:0/1765683926 shutdown_connections 2020-11-28T21:51:41.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.719+0000 7f82dffff700 1 -- 172.21.15.162:0/1765683926 wait complete. 2020-11-28T21:51:41.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.719+0000 7f82dffff700 1 Processor -- start 2020-11-28T21:51:41.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.719+0000 7f82dffff700 1 -- start start 2020-11-28T21:51:41.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.719+0000 7f82dffff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f82e00b18c0 0x7f82e00ada70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:41.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.719+0000 7f82dffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82e00b3950 0x7f82e014f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:41.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.719+0000 7f82dffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82e00b44e0 0x7f82e014f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:41.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.719+0000 7f82dffff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e00b6e50 con 0x7f82e00b44e0 2020-11-28T21:51:41.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.719+0000 7f82dffff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e01412b0 con 0x7f82e00b3950 2020-11-28T21:51:41.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.719+0000 7f82dffff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e01407b0 con 0x7f82e00b18c0 2020-11-28T21:51:41.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.719+0000 7f82de7fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82e00b3950 0x7f82e014f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:41.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.719+0000 7f82deffd700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f82e00b18c0 0x7f82e00ada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:41.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.719+0000 7f82de7fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82e00b3950 0x7f82e014f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40640/0 (socket says 172.21.15.162:40640) 2020-11-28T21:51:41.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.719+0000 7f82de7fc700 1 -- 172.21.15.162:0/2719120761 learned_addr learned my addr 172.21.15.162:0/2719120761 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:41.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.719+0000 7f82df7fe700 1 --2- 172.21.15.162:0/2719120761 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82e00b44e0 0x7f82e014f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:41.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.723+0000 7f82de7fc700 1 -- 172.21.15.162:0/2719120761 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f82e00b18c0 msgr2=0x7f82e00ada70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:41.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.723+0000 7f82de7fc700 1 --2- 172.21.15.162:0/2719120761 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f82e00b18c0 0x7f82e00ada70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:41.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.723+0000 7f82de7fc700 1 -- 172.21.15.162:0/2719120761 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82e00b44e0 msgr2=0x7f82e014f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:41.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.723+0000 7f82de7fc700 1 --2- 172.21.15.162:0/2719120761 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82e00b44e0 0x7f82e014f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:41.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.723+0000 7f82de7fc700 1 -- 172.21.15.162:0/2719120761 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e0141010 con 0x7f82e00b3950 2020-11-28T21:51:41.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.723+0000 7f82de7fc700 1 --2- 172.21.15.162:0/2719120761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82e00b3950 0x7f82e014f490 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7f82c80010d0 tx=0x7f82c8007a70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:41.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.723+0000 7f82d7fff700 1 -- 172.21.15.162:0/2719120761 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82c80072e0 con 0x7f82e00b3950 2020-11-28T21:51:41.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.723+0000 7f82d7fff700 1 -- 172.21.15.162:0/2719120761 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c8007440 con 0x7f82e00b3950 2020-11-28T21:51:41.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.723+0000 7f82d7fff700 1 -- 172.21.15.162:0/2719120761 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82c80057d0 con 0x7f82e00b3950 2020-11-28T21:51:41.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.723+0000 7f82dffff700 1 -- 172.21.15.162:0/2719120761 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82e0140a50 con 0x7f82e00b3950 2020-11-28T21:51:41.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.723+0000 7f82dffff700 1 -- 172.21.15.162:0/2719120761 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82e01414c0 con 0x7f82e00b3950 2020-11-28T21:51:41.733 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.723+0000 7f82dffff700 1 -- 172.21.15.162:0/2719120761 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82e0057050 con 0x7f82e00b3950 2020-11-28T21:51:41.733 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.723+0000 7f82d7fff700 1 -- 172.21.15.162:0/2719120761 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f82c8010070 con 0x7f82e00b3950 2020-11-28T21:51:41.733 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.723+0000 7f82d7fff700 1 --2- 172.21.15.162:0/2719120761 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f82c40553b0 0x7f82c4057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:41.733 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.723+0000 7f82d7fff700 1 -- 172.21.15.162:0/2719120761 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f82c806d290 con 0x7f82e00b3950 2020-11-28T21:51:41.733 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.723+0000 7f82deffd700 1 --2- 172.21.15.162:0/2719120761 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f82c40553b0 0x7f82c4057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:41.734 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.727+0000 7f82deffd700 1 --2- 172.21.15.162:0/2719120761 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f82c40553b0 0x7f82c4057870 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f82d001b000 tx=0x7f82d0015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:41.734 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.731+0000 7f82d7fff700 1 -- 172.21.15.162:0/2719120761 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f82c80438c0 con 0x7f82e00b3950 2020-11-28T21:51:41.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:41.879+0000 7f82dffff700 1 -- 172.21.15.162:0/2719120761 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f82e005d250 con 0x7f82e00b3950 2020-11-28T21:51:42.864 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:42.866+0000 7f82d7fff700 1 -- 172.21.15.162:0/2719120761 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v63) v1 ==== 133+0+0 (secure 0 0 0) 0x7f82c8046ee0 con 0x7f82e00b3950 2020-11-28T21:51:42.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:42.870+0000 7f82dffff700 1 -- 172.21.15.162:0/2719120761 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f82c40553b0 msgr2=0x7f82c4057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:42.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:42.870+0000 7f82dffff700 1 --2- 172.21.15.162:0/2719120761 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f82c40553b0 0x7f82c4057870 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f82d001b000 tx=0x7f82d0015000).stop 2020-11-28T21:51:42.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:42.870+0000 7f82dffff700 1 -- 172.21.15.162:0/2719120761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82e00b3950 msgr2=0x7f82e014f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:42.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:42.870+0000 7f82dffff700 1 --2- 172.21.15.162:0/2719120761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82e00b3950 0x7f82e014f490 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7f82c80010d0 tx=0x7f82c8007a70).stop 2020-11-28T21:51:42.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:42.870+0000 7f82dffff700 1 -- 172.21.15.162:0/2719120761 shutdown_connections 2020-11-28T21:51:42.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:42.870+0000 7f82dffff700 1 --2- 172.21.15.162:0/2719120761 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f82c40553b0 0x7f82c4057870 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:42.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:42.870+0000 7f82dffff700 1 --2- 172.21.15.162:0/2719120761 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f82e00b18c0 0x7f82e00ada70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:42.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:42.870+0000 7f82dffff700 1 --2- 172.21.15.162:0/2719120761 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82e00b44e0 0x7f82e014f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:42.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:42.870+0000 7f82dffff700 1 --2- 172.21.15.162:0/2719120761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82e00b3950 0x7f82e014f490 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:42.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:42.870+0000 7f82dffff700 1 -- 172.21.15.162:0/2719120761 >> 172.21.15.162:0/2719120761 conn(0x7f82e0062630 msgr2=0x7f82e00b4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:42.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:42.870+0000 7f82dffff700 1 -- 172.21.15.162:0/2719120761 shutdown_connections 2020-11-28T21:51:42.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:42.870+0000 7f82dffff700 1 -- 172.21.15.162:0/2719120761 wait complete. 2020-11-28T21:51:42.878 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_metadata' created 2020-11-28T21:51:42.882 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2020-11-28T21:51:43.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.002+0000 7f9a89d33700 1 Processor -- start 2020-11-28T21:51:43.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.002+0000 7f9a89d33700 1 -- start start 2020-11-28T21:51:43.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.002+0000 7f9a89d33700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9a840b9ce0 0x7f9a841433c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:43.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.002+0000 7f9a89d33700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f9a840605c0 0x7f9a84067610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:43.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.002+0000 7f9a89d33700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a84143950 con 0x7f9a840b3c30 2020-11-28T21:51:43.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.002+0000 7f9a89d33700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a840ba0c0 con 0x7f9a840605c0 2020-11-28T21:51:43.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.002+0000 7f9a89d33700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a84066c30 con 0x7f9a840b9ce0 2020-11-28T21:51:43.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.002+0000 7f9a837fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9a840b9ce0 0x7f9a841433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:43.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.002+0000 7f9a83fff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f9a840b3c30 0x7f9a8413fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:47494/0 (socket says 172.21.15.162:47494) 2020-11-28T21:51:43.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.002+0000 7f9a83fff700 1 -- 172.21.15.162:0/3821730054 learned_addr learned my addr 172.21.15.162:0/3821730054 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:51:43.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.002+0000 7f9a82ffd700 1 --2- 172.21.15.162:0/3821730054 >> v2:172.21.15.118:3300/0 conn(0x7f9a840605c0 0x7f9a84067610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:43.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.002+0000 7f9a837fe700 1 -- 172.21.15.162:0/3821730054 >> v2:172.21.15.118:3300/0 conn(0x7f9a840605c0 msgr2=0x7f9a84067610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:51:43.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.002+0000 7f9a837fe700 1 --2- 172.21.15.162:0/3821730054 >> v2:172.21.15.118:3300/0 conn(0x7f9a840605c0 0x7f9a84067610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:43.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.002+0000 7f9a837fe700 1 -- 172.21.15.162:0/3821730054 >> v1:172.21.15.162:6789/0 conn(0x7f9a840b3c30 legacy=0x7f9a8413fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:43.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.002+0000 7f9a837fe700 1 -- 172.21.15.162:0/3821730054 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a84066ed0 con 0x7f9a840b9ce0 2020-11-28T21:51:43.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.002+0000 7f9a827fc700 1 -- 172.21.15.162:0/3821730054 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 314075891 0 0) 0x7f9a84143950 con 0x7f9a840b3c30 2020-11-28T21:51:43.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.006+0000 7f9a837fe700 1 --2- 172.21.15.162:0/3821730054 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a840b9ce0 0x7f9a841433c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f9a74002f30 tx=0x7f9a74006dc0).ready entity=mon.1 client_cookie=fbd0fae4bad25bd server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:43.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.006+0000 7f9a827fc700 1 -- 172.21.15.162:0/3821730054 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a74009600 con 0x7f9a840b9ce0 2020-11-28T21:51:43.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.006+0000 7f9a827fc700 1 -- 172.21.15.162:0/3821730054 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a74009760 con 0x7f9a840b9ce0 2020-11-28T21:51:43.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.006+0000 7f9a827fc700 1 -- 172.21.15.162:0/3821730054 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a740098d0 con 0x7f9a840b9ce0 2020-11-28T21:51:43.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.006+0000 7f9a89d33700 1 -- 172.21.15.162:0/3821730054 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a840b9ce0 msgr2=0x7f9a841433c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:43.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.006+0000 7f9a89d33700 1 --2- 172.21.15.162:0/3821730054 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a840b9ce0 0x7f9a841433c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f9a74002f30 tx=0x7f9a74006dc0).stop 2020-11-28T21:51:43.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.006+0000 7f9a89d33700 1 -- 172.21.15.162:0/3821730054 shutdown_connections 2020-11-28T21:51:43.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.006+0000 7f9a89d33700 1 --2- 172.21.15.162:0/3821730054 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a840b9ce0 0x7f9a841433c0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:43.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.006+0000 7f9a89d33700 1 --2- 172.21.15.162:0/3821730054 >> v2:172.21.15.118:3300/0 conn(0x7f9a840605c0 0x7f9a84067610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:43.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.006+0000 7f9a89d33700 1 -- 172.21.15.162:0/3821730054 >> 172.21.15.162:0/3821730054 conn(0x7f9a8405c430 msgr2=0x7f9a840b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:43.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.006+0000 7f9a89d33700 1 -- 172.21.15.162:0/3821730054 shutdown_connections 2020-11-28T21:51:43.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.006+0000 7f9a89d33700 1 -- 172.21.15.162:0/3821730054 wait complete. 2020-11-28T21:51:43.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.006+0000 7f9a89d33700 1 Processor -- start 2020-11-28T21:51:43.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.006+0000 7f9a89d33700 1 -- start start 2020-11-28T21:51:43.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.006+0000 7f9a89d33700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9a840605c0 0x7f9a840b5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:43.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.006+0000 7f9a89d33700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a840b3c30 0x7f9a840b0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:43.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.006+0000 7f9a89d33700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9a840b9ce0 0x7f9a840b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:43.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.006+0000 7f9a89d33700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a840b1c00 con 0x7f9a840b9ce0 2020-11-28T21:51:43.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.006+0000 7f9a89d33700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a841455f0 con 0x7f9a840b3c30 2020-11-28T21:51:43.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.006+0000 7f9a89d33700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a84145990 con 0x7f9a840605c0 2020-11-28T21:51:43.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.006+0000 7f9a82ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a840b3c30 0x7f9a840b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:43.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.006+0000 7f9a82ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a840b3c30 0x7f9a840b0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40654/0 (socket says 172.21.15.162:40654) 2020-11-28T21:51:43.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.006+0000 7f9a82ffd700 1 -- 172.21.15.162:0/2709007250 learned_addr learned my addr 172.21.15.162:0/2709007250 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:43.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.006+0000 7f9a837fe700 1 --2- 172.21.15.162:0/2709007250 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9a840605c0 0x7f9a840b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:43.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.010+0000 7f9a83fff700 1 --2- 172.21.15.162:0/2709007250 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9a840b9ce0 0x7f9a840b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:43.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.010+0000 7f9a82ffd700 1 -- 172.21.15.162:0/2709007250 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9a840605c0 msgr2=0x7f9a840b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:43.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.010+0000 7f9a82ffd700 1 --2- 172.21.15.162:0/2709007250 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9a840605c0 0x7f9a840b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:43.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.010+0000 7f9a82ffd700 1 -- 172.21.15.162:0/2709007250 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9a840b9ce0 msgr2=0x7f9a840b0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:43.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.010+0000 7f9a82ffd700 1 --2- 172.21.15.162:0/2709007250 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9a840b9ce0 0x7f9a840b0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:43.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.010+0000 7f9a82ffd700 1 -- 172.21.15.162:0/2709007250 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a84145f80 con 0x7f9a840b3c30 2020-11-28T21:51:43.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.010+0000 7f9a82ffd700 1 --2- 172.21.15.162:0/2709007250 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a840b3c30 0x7f9a840b0560 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7f9a6c00c000 tx=0x7f9a6c0091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:43.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.010+0000 7f9a80ff9700 1 -- 172.21.15.162:0/2709007250 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a6c00cbb0 con 0x7f9a840b3c30 2020-11-28T21:51:43.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.010+0000 7f9a80ff9700 1 -- 172.21.15.162:0/2709007250 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a6c00cd10 con 0x7f9a840b3c30 2020-11-28T21:51:43.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.010+0000 7f9a89d33700 1 -- 172.21.15.162:0/2709007250 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a84146220 con 0x7f9a840b3c30 2020-11-28T21:51:43.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.010+0000 7f9a80ff9700 1 -- 172.21.15.162:0/2709007250 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a6c0125e0 con 0x7f9a840b3c30 2020-11-28T21:51:43.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.010+0000 7f9a89d33700 1 -- 172.21.15.162:0/2709007250 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a840b33e0 con 0x7f9a840b3c30 2020-11-28T21:51:43.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.010+0000 7f9a89d33700 1 -- 172.21.15.162:0/2709007250 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a84145b90 con 0x7f9a840b3c30 2020-11-28T21:51:43.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.010+0000 7f9a80ff9700 1 -- 172.21.15.162:0/2709007250 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9a6c022070 con 0x7f9a840b3c30 2020-11-28T21:51:43.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.010+0000 7f9a80ff9700 1 --2- 172.21.15.162:0/2709007250 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9a68055350 0x7f9a68057810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:43.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.010+0000 7f9a837fe700 1 --2- 172.21.15.162:0/2709007250 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9a68055350 0x7f9a68057810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:43.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.010+0000 7f9a80ff9700 1 -- 172.21.15.162:0/2709007250 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f9a6c06e500 con 0x7f9a840b3c30 2020-11-28T21:51:43.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.014+0000 7f9a837fe700 1 --2- 172.21.15.162:0/2709007250 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9a68055350 0x7f9a68057810 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f9a74018000 tx=0x7f9a74016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:43.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.018+0000 7f9a80ff9700 1 -- 172.21.15.162:0/2709007250 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9a6c0449d0 con 0x7f9a840b3c30 2020-11-28T21:51:43.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.166+0000 7f9a89d33700 1 -- 172.21.15.162:0/2709007250 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f9a841463b0 con 0x7f9a840b3c30 2020-11-28T21:51:43.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.870+0000 7f9a80ff9700 1 -- 172.21.15.162:0/2709007250 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v64) v1 ==== 125+0+0 (secure 0 0 0) 0x7f9a6c047ff0 con 0x7f9a840b3c30 2020-11-28T21:51:43.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.878+0000 7f9a89d33700 1 -- 172.21.15.162:0/2709007250 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9a68055350 msgr2=0x7f9a68057810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:43.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.878+0000 7f9a89d33700 1 --2- 172.21.15.162:0/2709007250 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9a68055350 0x7f9a68057810 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f9a74018000 tx=0x7f9a74016000).stop 2020-11-28T21:51:43.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.878+0000 7f9a89d33700 1 -- 172.21.15.162:0/2709007250 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a840b3c30 msgr2=0x7f9a840b0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:43.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.878+0000 7f9a89d33700 1 --2- 172.21.15.162:0/2709007250 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a840b3c30 0x7f9a840b0560 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7f9a6c00c000 tx=0x7f9a6c0091a0).stop 2020-11-28T21:51:43.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.878+0000 7f9a89d33700 1 -- 172.21.15.162:0/2709007250 shutdown_connections 2020-11-28T21:51:43.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.878+0000 7f9a89d33700 1 --2- 172.21.15.162:0/2709007250 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9a68055350 0x7f9a68057810 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:43.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.878+0000 7f9a89d33700 1 --2- 172.21.15.162:0/2709007250 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9a840605c0 0x7f9a840b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:43.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.878+0000 7f9a89d33700 1 --2- 172.21.15.162:0/2709007250 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9a840b9ce0 0x7f9a840b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:43.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.878+0000 7f9a89d33700 1 --2- 172.21.15.162:0/2709007250 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a840b3c30 0x7f9a840b0560 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:43.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.878+0000 7f9a89d33700 1 -- 172.21.15.162:0/2709007250 >> 172.21.15.162:0/2709007250 conn(0x7f9a8405c430 msgr2=0x7f9a84141830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:43.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.878+0000 7f9a89d33700 1 -- 172.21.15.162:0/2709007250 shutdown_connections 2020-11-28T21:51:43.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:43.878+0000 7f9a89d33700 1 -- 172.21.15.162:0/2709007250 wait complete. 2020-11-28T21:51:43.882 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_data' created 2020-11-28T21:51:43.893 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2020-11-28T21:51:44.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.010+0000 7f8080f81700 1 Processor -- start 2020-11-28T21:51:44.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.010+0000 7f8080f81700 1 -- start start 2020-11-28T21:51:44.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.014+0000 7f8080f81700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f807c099270 0x7f807c05c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:44.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.014+0000 7f8080f81700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f807c05ce70 0x7f807c149db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:44.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.014+0000 7f8080f81700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f807c0b16b0 con 0x7f807c09b3f0 2020-11-28T21:51:44.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.014+0000 7f8080f81700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f807c0b18e0 con 0x7f807c099270 2020-11-28T21:51:44.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.014+0000 7f8080f81700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f807c0acb00 con 0x7f807c05ce70 2020-11-28T21:51:44.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.014+0000 7f807b7fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f807c099270 0x7f807c05c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:44.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.014+0000 7f8072ffd700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f807c09b3f0 0x7f807c066fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:47506/0 (socket says 172.21.15.162:47506) 2020-11-28T21:51:44.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.014+0000 7f8072ffd700 1 -- 172.21.15.162:0/2168307628 learned_addr learned my addr 172.21.15.162:0/2168307628 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:51:44.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.014+0000 7f807bfff700 1 --2- 172.21.15.162:0/2168307628 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f807c05ce70 0x7f807c149db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:44.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.014+0000 7f807b7fe700 1 -- 172.21.15.162:0/2168307628 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f807c05ce70 msgr2=0x7f807c149db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:51:44.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.014+0000 7f807b7fe700 1 --2- 172.21.15.162:0/2168307628 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f807c05ce70 0x7f807c149db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:44.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.014+0000 7f807b7fe700 1 -- 172.21.15.162:0/2168307628 >> v1:172.21.15.162:6789/0 conn(0x7f807c09b3f0 legacy=0x7f807c066fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:44.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.014+0000 7f807b7fe700 1 -- 172.21.15.162:0/2168307628 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f807c0acda0 con 0x7f807c099270 2020-11-28T21:51:44.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.014+0000 7f807bfff700 1 --2- 172.21.15.162:0/2168307628 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f807c05ce70 0x7f807c149db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:51:44.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.014+0000 7f807b7fe700 1 --2- 172.21.15.162:0/2168307628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f807c099270 0x7f807c05c980 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7f806c002f30 tx=0x7f806c006dc0).ready entity=mon.1 client_cookie=a231507a9e72a678 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:44.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.014+0000 7f807affd700 1 -- 172.21.15.162:0/2168307628 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f806c0097f0 con 0x7f807c099270 2020-11-28T21:51:44.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.014+0000 7f807affd700 1 -- 172.21.15.162:0/2168307628 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f806c009950 con 0x7f807c099270 2020-11-28T21:51:44.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.014+0000 7f807affd700 1 -- 172.21.15.162:0/2168307628 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f806c009cb0 con 0x7f807c099270 2020-11-28T21:51:44.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.014+0000 7f8080f81700 1 -- 172.21.15.162:0/2168307628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f807c099270 msgr2=0x7f807c05c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:44.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.014+0000 7f8080f81700 1 --2- 172.21.15.162:0/2168307628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f807c099270 0x7f807c05c980 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7f806c002f30 tx=0x7f806c006dc0).stop 2020-11-28T21:51:44.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.014+0000 7f8080f81700 1 -- 172.21.15.162:0/2168307628 shutdown_connections 2020-11-28T21:51:44.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.014+0000 7f8080f81700 1 --2- 172.21.15.162:0/2168307628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f807c099270 0x7f807c05c980 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:44.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.014+0000 7f8080f81700 1 --2- 172.21.15.162:0/2168307628 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f807c05ce70 0x7f807c149db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:44.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.014+0000 7f8080f81700 1 -- 172.21.15.162:0/2168307628 >> 172.21.15.162:0/2168307628 conn(0x7f807c0b1fe0 msgr2=0x7f807c0b23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:44.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.014+0000 7f8080f81700 1 -- 172.21.15.162:0/2168307628 shutdown_connections 2020-11-28T21:51:44.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.014+0000 7f8080f81700 1 -- 172.21.15.162:0/2168307628 wait complete. 2020-11-28T21:51:44.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.018+0000 7f8080f81700 1 Processor -- start 2020-11-28T21:51:44.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.018+0000 7f8080f81700 1 -- start start 2020-11-28T21:51:44.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.018+0000 7f8080f81700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f807c099270 0x7f807c0ae750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:44.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.018+0000 7f8080f81700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f807c060cb0 0x7f807c1550c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:44.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.018+0000 7f8080f81700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f807c097280 0x7f807c145320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:44.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.018+0000 7f8080f81700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807c14f770 con 0x7f807c097280 2020-11-28T21:51:44.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.018+0000 7f8080f81700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807c091890 con 0x7f807c060cb0 2020-11-28T21:51:44.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.018+0000 7f8080f81700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f807c095fd0 con 0x7f807c099270 2020-11-28T21:51:44.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.018+0000 7f8072ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f807c060cb0 0x7f807c1550c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:44.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.018+0000 7f8072ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f807c060cb0 0x7f807c1550c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40670/0 (socket says 172.21.15.162:40670) 2020-11-28T21:51:44.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.018+0000 7f8072ffd700 1 -- 172.21.15.162:0/214582665 learned_addr learned my addr 172.21.15.162:0/214582665 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:44.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.018+0000 7f807bfff700 1 --2- 172.21.15.162:0/214582665 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f807c097280 0x7f807c145320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:44.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.018+0000 7f807b7fe700 1 --2- 172.21.15.162:0/214582665 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f807c099270 0x7f807c0ae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:44.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.018+0000 7f8072ffd700 1 -- 172.21.15.162:0/214582665 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f807c099270 msgr2=0x7f807c0ae750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:44.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.018+0000 7f8072ffd700 1 --2- 172.21.15.162:0/214582665 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f807c099270 0x7f807c0ae750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:44.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.018+0000 7f8072ffd700 1 -- 172.21.15.162:0/214582665 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f807c097280 msgr2=0x7f807c145320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:44.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.018+0000 7f8072ffd700 1 --2- 172.21.15.162:0/214582665 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f807c097280 0x7f807c145320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:44.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.018+0000 7f8072ffd700 1 -- 172.21.15.162:0/214582665 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f807c096270 con 0x7f807c060cb0 2020-11-28T21:51:44.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.018+0000 7f807b7fe700 1 --2- 172.21.15.162:0/214582665 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f807c099270 0x7f807c0ae750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:51:44.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.018+0000 7f8072ffd700 1 --2- 172.21.15.162:0/214582665 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f807c060cb0 0x7f807c1550c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f807c06ac90 tx=0x7f8064007c30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:44.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.018+0000 7f80797fa700 1 -- 172.21.15.162:0/214582665 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8064007380 con 0x7f807c060cb0 2020-11-28T21:51:44.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.018+0000 7f80797fa700 1 -- 172.21.15.162:0/214582665 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8064005030 con 0x7f807c060cb0 2020-11-28T21:51:44.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.018+0000 7f80797fa700 1 -- 172.21.15.162:0/214582665 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8064005390 con 0x7f807c060cb0 2020-11-28T21:51:44.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.018+0000 7f8080f81700 1 -- 172.21.15.162:0/214582665 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f807c096de0 con 0x7f807c060cb0 2020-11-28T21:51:44.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.018+0000 7f8080f81700 1 -- 172.21.15.162:0/214582665 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f807c0600f0 con 0x7f807c060cb0 2020-11-28T21:51:44.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.018+0000 7f8080f81700 1 -- 172.21.15.162:0/214582665 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f807c096f70 con 0x7f807c060cb0 2020-11-28T21:51:44.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.022+0000 7f80797fa700 1 -- 172.21.15.162:0/214582665 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f806401b070 con 0x7f807c060cb0 2020-11-28T21:51:44.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.022+0000 7f80797fa700 1 --2- 172.21.15.162:0/214582665 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f80600553b0 0x7f8060057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:44.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.022+0000 7f80797fa700 1 -- 172.21.15.162:0/214582665 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5957+0+0 (secure 0 0 0) 0x7f806406d3f0 con 0x7f807c060cb0 2020-11-28T21:51:44.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.022+0000 7f807b7fe700 1 --2- 172.21.15.162:0/214582665 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f80600553b0 0x7f8060057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:44.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.022+0000 7f807b7fe700 1 --2- 172.21.15.162:0/214582665 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f80600553b0 0x7f8060057870 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f806c016000 tx=0x7f806c015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:44.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.026+0000 7f80797fa700 1 -- 172.21.15.162:0/214582665 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f80640436e0 con 0x7f807c060cb0 2020-11-28T21:51:44.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.222+0000 7f8080f81700 1 -- 172.21.15.162:0/214582665 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f807c057050 con 0x7f807c060cb0 2020-11-28T21:51:44.237 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.238+0000 7f80797fa700 1 -- 172.21.15.162:0/214582665 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 11 and data pool 12 v26) v1 ==== 189+0+0 (secure 0 0 0) 0x7f8064046d00 con 0x7f807c060cb0 2020-11-28T21:51:44.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.250+0000 7f8080f81700 1 -- 172.21.15.162:0/214582665 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f80600553b0 msgr2=0x7f8060057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:44.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.250+0000 7f8080f81700 1 --2- 172.21.15.162:0/214582665 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f80600553b0 0x7f8060057870 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f806c016000 tx=0x7f806c015000).stop 2020-11-28T21:51:44.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.250+0000 7f8080f81700 1 -- 172.21.15.162:0/214582665 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f807c060cb0 msgr2=0x7f807c1550c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:44.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.250+0000 7f8080f81700 1 --2- 172.21.15.162:0/214582665 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f807c060cb0 0x7f807c1550c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f807c06ac90 tx=0x7f8064007c30).stop 2020-11-28T21:51:44.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.250+0000 7f8080f81700 1 -- 172.21.15.162:0/214582665 shutdown_connections 2020-11-28T21:51:44.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.250+0000 7f8080f81700 1 --2- 172.21.15.162:0/214582665 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f80600553b0 0x7f8060057870 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:44.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.250+0000 7f8080f81700 1 --2- 172.21.15.162:0/214582665 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f807c099270 0x7f807c0ae750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:44.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.250+0000 7f8080f81700 1 --2- 172.21.15.162:0/214582665 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f807c097280 0x7f807c145320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:44.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.250+0000 7f8080f81700 1 --2- 172.21.15.162:0/214582665 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f807c060cb0 0x7f807c1550c0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:44.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.250+0000 7f8080f81700 1 -- 172.21.15.162:0/214582665 >> 172.21.15.162:0/214582665 conn(0x7f807c0b1fe0 msgr2=0x7f807c066dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:44.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.250+0000 7f8080f81700 1 -- 172.21.15.162:0/214582665 shutdown_connections 2020-11-28T21:51:44.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.250+0000 7f8080f81700 1 -- 172.21.15.162:0/214582665 wait complete. 2020-11-28T21:51:44.253 INFO:teuthology.orchestra.run.smithi162.stderr:new fs with metadata pool 11 and data pool 12 2020-11-28T21:51:44.262 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:51:44.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.362+0000 7fa964b85700 1 Processor -- start 2020-11-28T21:51:44.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.362+0000 7fa964b85700 1 -- start start 2020-11-28T21:51:44.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.366+0000 7fa964b85700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa96005bcb0 0x7fa960143530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:44.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.366+0000 7fa964b85700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa96013c630 0x7fa96013cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:44.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.366+0000 7fa964b85700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9600b0e50 con 0x7fa9600b1c00 2020-11-28T21:51:44.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.366+0000 7fa964b85700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa96005fe40 con 0x7fa96005bcb0 2020-11-28T21:51:44.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.366+0000 7fa964b85700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9600b2080 con 0x7fa96013c630 2020-11-28T21:51:44.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.366+0000 7fa95f7fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa96005bcb0 0x7fa960143530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:44.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.366+0000 7fa95f7fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa96005bcb0 0x7fa960143530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40688/0 (socket says 172.21.15.162:40688) 2020-11-28T21:51:44.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.366+0000 7fa95f7fe700 1 -- 172.21.15.162:0/2152930802 learned_addr learned my addr 172.21.15.162:0/2152930802 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:44.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.366+0000 7fa95ffff700 1 --2- 172.21.15.162:0/2152930802 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa96013c630 0x7fa96013cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:44.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.366+0000 7fa95f7fe700 1 -- 172.21.15.162:0/2152930802 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa96013c630 msgr2=0x7fa96013cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:51:44.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.366+0000 7fa95f7fe700 1 --2- 172.21.15.162:0/2152930802 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa96013c630 0x7fa96013cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:44.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.366+0000 7fa95f7fe700 1 -- 172.21.15.162:0/2152930802 >> v1:172.21.15.118:6789/0 conn(0x7fa9600b1c00 legacy=0x7fa9600b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:44.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.366+0000 7fa95f7fe700 1 -- 172.21.15.162:0/2152930802 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa96005c090 con 0x7fa96005bcb0 2020-11-28T21:51:44.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.366+0000 7fa95f7fe700 1 --2- 172.21.15.162:0/2152930802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa96005bcb0 0x7fa960143530 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7fa96005ff80 tx=0x7fa950006e80).ready entity=mon.1 client_cookie=35b89e008038a80c server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:44.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.366+0000 7fa95effd700 1 -- 172.21.15.162:0/2152930802 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9500097b0 con 0x7fa96005bcb0 2020-11-28T21:51:44.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.366+0000 7fa95effd700 1 -- 172.21.15.162:0/2152930802 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa950009910 con 0x7fa96005bcb0 2020-11-28T21:51:44.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.366+0000 7fa95effd700 1 -- 172.21.15.162:0/2152930802 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa950009c70 con 0x7fa96005bcb0 2020-11-28T21:51:44.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.366+0000 7fa964b85700 1 -- 172.21.15.162:0/2152930802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa96005bcb0 msgr2=0x7fa960143530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:44.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.366+0000 7fa964b85700 1 --2- 172.21.15.162:0/2152930802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa96005bcb0 0x7fa960143530 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7fa96005ff80 tx=0x7fa950006e80).stop 2020-11-28T21:51:44.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.366+0000 7fa964b85700 1 -- 172.21.15.162:0/2152930802 shutdown_connections 2020-11-28T21:51:44.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.366+0000 7fa964b85700 1 --2- 172.21.15.162:0/2152930802 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa96013c630 0x7fa96013cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:44.371 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.366+0000 7fa964b85700 1 --2- 172.21.15.162:0/2152930802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa96005bcb0 0x7fa960143530 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:44.371 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.366+0000 7fa964b85700 1 -- 172.21.15.162:0/2152930802 >> 172.21.15.162:0/2152930802 conn(0x7fa9600b5da0 msgr2=0x7fa9600b6180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:44.371 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.366+0000 7fa964b85700 1 -- 172.21.15.162:0/2152930802 shutdown_connections 2020-11-28T21:51:44.371 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.366+0000 7fa964b85700 1 -- 172.21.15.162:0/2152930802 wait complete. 2020-11-28T21:51:44.371 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.366+0000 7fa964b85700 1 Processor -- start 2020-11-28T21:51:44.372 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.366+0000 7fa964b85700 1 -- start start 2020-11-28T21:51:44.372 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.370+0000 7fa964b85700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa96005bcb0 0x7fa960061310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:44.372 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.370+0000 7fa964b85700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa9600b1c00 0x7fa960146a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:44.372 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.370+0000 7fa964b85700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa96013c630 0x7fa960145d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:44.372 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.370+0000 7fa964b85700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa960061b50 con 0x7fa9600b1c00 2020-11-28T21:51:44.373 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.370+0000 7fa964b85700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9600b0050 con 0x7fa96013c630 2020-11-28T21:51:44.373 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.370+0000 7fa964b85700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9600ae6d0 con 0x7fa96005bcb0 2020-11-28T21:51:44.373 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.370+0000 7fa95ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa96013c630 0x7fa960145d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:44.373 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.370+0000 7fa957fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa9600b1c00 0x7fa960146a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:44.373 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.370+0000 7fa957fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa9600b1c00 0x7fa960146a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.162:47680/0 (socket says 172.21.15.162:47680) 2020-11-28T21:51:44.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.370+0000 7fa957fff700 1 -- 172.21.15.162:0/3812190116 learned_addr learned my addr 172.21.15.162:0/3812190116 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:44.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.370+0000 7fa95f7fe700 1 --2- 172.21.15.162:0/3812190116 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa96005bcb0 0x7fa960061310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:44.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.370+0000 7fa95ffff700 1 -- 172.21.15.162:0/3812190116 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa96005bcb0 msgr2=0x7fa960061310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:44.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.370+0000 7fa95ffff700 1 --2- 172.21.15.162:0/3812190116 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa96005bcb0 0x7fa960061310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:44.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.370+0000 7fa95ffff700 1 -- 172.21.15.162:0/3812190116 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa9600b1c00 msgr2=0x7fa960146a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:44.375 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.370+0000 7fa95ffff700 1 --2- 172.21.15.162:0/3812190116 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa9600b1c00 0x7fa960146a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:44.375 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.370+0000 7fa95ffff700 1 -- 172.21.15.162:0/3812190116 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9600ae970 con 0x7fa96013c630 2020-11-28T21:51:44.375 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.370+0000 7fa95ffff700 1 --2- 172.21.15.162:0/3812190116 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa96013c630 0x7fa960145d90 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7fa95800c000 tx=0x7fa9580091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:44.375 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.370+0000 7fa95d7fa700 1 -- 172.21.15.162:0/3812190116 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa95800c8f0 con 0x7fa96013c630 2020-11-28T21:51:44.375 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.370+0000 7fa95d7fa700 1 -- 172.21.15.162:0/3812190116 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa95800ca50 con 0x7fa96013c630 2020-11-28T21:51:44.376 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.370+0000 7fa964b85700 1 -- 172.21.15.162:0/3812190116 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9600aec10 con 0x7fa96013c630 2020-11-28T21:51:44.376 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.370+0000 7fa95d7fa700 1 -- 172.21.15.162:0/3812190116 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa95800cdb0 con 0x7fa96013c630 2020-11-28T21:51:44.376 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.370+0000 7fa964b85700 1 -- 172.21.15.162:0/3812190116 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9601446f0 con 0x7fa96013c630 2020-11-28T21:51:44.377 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.370+0000 7fa964b85700 1 -- 172.21.15.162:0/3812190116 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa960143fc0 con 0x7fa96013c630 2020-11-28T21:51:44.377 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.370+0000 7fa95d7fa700 1 -- 172.21.15.162:0/3812190116 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa958022070 con 0x7fa96013c630 2020-11-28T21:51:44.377 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.370+0000 7fa95d7fa700 1 --2- 172.21.15.162:0/3812190116 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa9440553b0 0x7fa944057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:44.378 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.370+0000 7fa95d7fa700 1 -- 172.21.15.162:0/3812190116 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa95806e4a0 con 0x7fa96013c630 2020-11-28T21:51:44.378 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.370+0000 7fa95f7fe700 1 --2- 172.21.15.162:0/3812190116 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa9440553b0 0x7fa944057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:44.378 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.374+0000 7fa95f7fe700 1 --2- 172.21.15.162:0/3812190116 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa9440553b0 0x7fa944057870 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7fa95001c000 tx=0x7fa950015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:44.378 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.374+0000 7fa95d7fa700 1 -- 172.21.15.162:0/3812190116 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa960143fc0 con 0x7fa96013c630 2020-11-28T21:51:44.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.514+0000 7fa964b85700 1 -- 172.21.15.162:0/3812190116 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa960143fc0 con 0x7fa96013c630 2020-11-28T21:51:44.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.514+0000 7fa95d7fa700 1 -- 172.21.15.162:0/3812190116 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+16150 (secure 0 0 0) 0x7fa960143fc0 con 0x7fa96013c630 2020-11-28T21:51:44.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.514+0000 7fa964b85700 1 -- 172.21.15.162:0/3812190116 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa9440553b0 msgr2=0x7fa944057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:44.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.514+0000 7fa964b85700 1 --2- 172.21.15.162:0/3812190116 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa9440553b0 0x7fa944057870 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7fa95001c000 tx=0x7fa950015000).stop 2020-11-28T21:51:44.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.518+0000 7fa964b85700 1 -- 172.21.15.162:0/3812190116 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa96013c630 msgr2=0x7fa960145d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:44.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.518+0000 7fa964b85700 1 --2- 172.21.15.162:0/3812190116 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa96013c630 0x7fa960145d90 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7fa95800c000 tx=0x7fa9580091a0).stop 2020-11-28T21:51:44.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.518+0000 7fa964b85700 1 -- 172.21.15.162:0/3812190116 shutdown_connections 2020-11-28T21:51:44.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.518+0000 7fa964b85700 1 --2- 172.21.15.162:0/3812190116 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa9440553b0 0x7fa944057870 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:44.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.518+0000 7fa964b85700 1 --2- 172.21.15.162:0/3812190116 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa96005bcb0 0x7fa960061310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:44.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.518+0000 7fa964b85700 1 --2- 172.21.15.162:0/3812190116 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa96013c630 0x7fa960145d90 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:44.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.518+0000 7fa964b85700 1 --2- 172.21.15.162:0/3812190116 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa9600b1c00 0x7fa960146a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:44.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.518+0000 7fa964b85700 1 -- 172.21.15.162:0/3812190116 >> 172.21.15.162:0/3812190116 conn(0x7fa9600b5da0 msgr2=0x7fa9601421c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:44.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.518+0000 7fa964b85700 1 -- 172.21.15.162:0/3812190116 shutdown_connections 2020-11-28T21:51:44.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.518+0000 7fa964b85700 1 -- 172.21.15.162:0/3812190116 wait complete. 2020-11-28T21:51:44.518 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:51:44.518 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":65,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:51:44.225990+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:51:41.931840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2020-11-28T21:51:43.225560+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:51:44.527 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:51:44.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.630+0000 7f8db895d700 1 Processor -- start 2020-11-28T21:51:44.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.630+0000 7f8db895d700 1 -- start start 2020-11-28T21:51:44.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.634+0000 7f8db895d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8db40603d0 0x7f8db40b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:44.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.634+0000 7f8db895d700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8db40b4110 con 0x7f8db40b4c70 2020-11-28T21:51:44.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.634+0000 7f8db895d700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8db40b43f0 con 0x7f8db4142a00 2020-11-28T21:51:44.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.634+0000 7f8db895d700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8db4060f60 con 0x7f8db40603d0 2020-11-28T21:51:44.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.634+0000 7f8db259c700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f8db4142a00 0x7f8db409cfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:47544/0 (socket says 172.21.15.162:47544) 2020-11-28T21:51:44.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.634+0000 7f8db259c700 1 -- 172.21.15.162:0/4223618910 learned_addr learned my addr 172.21.15.162:0/4223618910 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:51:44.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.634+0000 7f8db2d9d700 1 --2- 172.21.15.162:0/4223618910 >> v2:172.21.15.162:3300/0 conn(0x7f8db40603d0 0x7f8db40b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:44.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.634+0000 7f8db1d9b700 1 -- 172.21.15.162:0/4223618910 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3613693464 0 0) 0x7f8db40b43f0 con 0x7f8db4142a00 2020-11-28T21:51:44.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.634+0000 7f8db1d9b700 1 -- 172.21.15.162:0/4223618910 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d94003230 con 0x7f8db4142a00 2020-11-28T21:51:44.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.634+0000 7f8db2d9d700 1 -- 172.21.15.162:0/4223618910 >> v1:172.21.15.162:6789/0 conn(0x7f8db4142a00 legacy=0x7f8db409cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:44.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.634+0000 7f8db2d9d700 1 -- 172.21.15.162:0/4223618910 >> v1:172.21.15.118:6789/0 conn(0x7f8db40b4c70 legacy=0x7f8db4143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:44.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.634+0000 7f8db2d9d700 1 -- 172.21.15.162:0/4223618910 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db40b4690 con 0x7f8db40603d0 2020-11-28T21:51:44.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.634+0000 7f8db2d9d700 1 --2- 172.21.15.162:0/4223618910 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8db40603d0 0x7f8db40b3a70 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f8da4002f30 tx=0x7f8da4006dc0).ready entity=mon.1 client_cookie=d4d107247b1728d6 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:44.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.634+0000 7f8db1d9b700 1 -- 172.21.15.162:0/4223618910 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8da4009800 con 0x7f8db40603d0 2020-11-28T21:51:44.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.634+0000 7f8db1d9b700 1 -- 172.21.15.162:0/4223618910 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da4009960 con 0x7f8db40603d0 2020-11-28T21:51:44.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.634+0000 7f8db1d9b700 1 -- 172.21.15.162:0/4223618910 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8da4009cc0 con 0x7f8db40603d0 2020-11-28T21:51:44.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.634+0000 7f8db895d700 1 -- 172.21.15.162:0/4223618910 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8db40603d0 msgr2=0x7f8db40b3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:44.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.634+0000 7f8db895d700 1 --2- 172.21.15.162:0/4223618910 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8db40603d0 0x7f8db40b3a70 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f8da4002f30 tx=0x7f8da4006dc0).stop 2020-11-28T21:51:44.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.634+0000 7f8db895d700 1 -- 172.21.15.162:0/4223618910 shutdown_connections 2020-11-28T21:51:44.639 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.634+0000 7f8db895d700 1 --2- 172.21.15.162:0/4223618910 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8db40603d0 0x7f8db40b3a70 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:44.639 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.634+0000 7f8db895d700 1 -- 172.21.15.162:0/4223618910 >> 172.21.15.162:0/4223618910 conn(0x7f8db4066a40 msgr2=0x7f8db4066e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:44.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.634+0000 7f8db895d700 1 -- 172.21.15.162:0/4223618910 shutdown_connections 2020-11-28T21:51:44.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.634+0000 7f8db895d700 1 -- 172.21.15.162:0/4223618910 wait complete. 2020-11-28T21:51:44.642 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.638+0000 7f8db895d700 1 Processor -- start 2020-11-28T21:51:44.642 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.638+0000 7f8db895d700 1 -- start start 2020-11-28T21:51:44.643 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.638+0000 7f8db895d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8db40603d0 0x7f8db41553b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:44.643 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.638+0000 7f8db895d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8db40b4c70 0x7f8db41480d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:44.643 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.642+0000 7f8db895d700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8db41485c0 0x7f8db409daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:44.643 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.642+0000 7f8db895d700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db4154cb0 con 0x7f8db40b4c70 2020-11-28T21:51:44.643 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.642+0000 7f8db895d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db4149ea0 con 0x7f8db40603d0 2020-11-28T21:51:44.644 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.642+0000 7f8db895d700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db4148f70 con 0x7f8db41485c0 2020-11-28T21:51:44.644 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.642+0000 7f8db2d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8db40603d0 0x7f8db41553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:44.644 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.642+0000 7f8db2d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8db40603d0 0x7f8db41553b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40706/0 (socket says 172.21.15.162:40706) 2020-11-28T21:51:44.644 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.642+0000 7f8db2d9d700 1 -- 172.21.15.162:0/2785637964 learned_addr learned my addr 172.21.15.162:0/2785637964 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:44.645 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.642+0000 7f8db259c700 1 --2- 172.21.15.162:0/2785637964 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8db40b4c70 0x7f8db41480d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:44.645 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.642+0000 7f8db359e700 1 --2- 172.21.15.162:0/2785637964 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8db41485c0 0x7f8db409daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:44.645 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.642+0000 7f8db2d9d700 1 -- 172.21.15.162:0/2785637964 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8db41485c0 msgr2=0x7f8db409daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:44.645 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.642+0000 7f8db2d9d700 1 --2- 172.21.15.162:0/2785637964 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8db41485c0 0x7f8db409daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:44.645 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.642+0000 7f8db2d9d700 1 -- 172.21.15.162:0/2785637964 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8db40b4c70 msgr2=0x7f8db41480d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:44.646 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.642+0000 7f8db2d9d700 1 --2- 172.21.15.162:0/2785637964 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8db40b4c70 0x7f8db41480d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:44.646 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.642+0000 7f8db2d9d700 1 -- 172.21.15.162:0/2785637964 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db4149c00 con 0x7f8db40603d0 2020-11-28T21:51:44.646 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.642+0000 7f8db2d9d700 1 --2- 172.21.15.162:0/2785637964 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8db40603d0 0x7f8db41553b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7f8da4002f30 tx=0x7f8da4015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:44.646 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.642+0000 7f8dabfff700 1 -- 172.21.15.162:0/2785637964 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8da4011b80 con 0x7f8db40603d0 2020-11-28T21:51:44.646 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.642+0000 7f8dabfff700 1 -- 172.21.15.162:0/2785637964 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da4011ce0 con 0x7f8db40603d0 2020-11-28T21:51:44.647 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.642+0000 7f8db895d700 1 -- 172.21.15.162:0/2785637964 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8db4154f10 con 0x7f8db40603d0 2020-11-28T21:51:44.647 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.642+0000 7f8dabfff700 1 -- 172.21.15.162:0/2785637964 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8da401d630 con 0x7f8db40603d0 2020-11-28T21:51:44.647 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.642+0000 7f8db895d700 1 -- 172.21.15.162:0/2785637964 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8db40ae520 con 0x7f8db40603d0 2020-11-28T21:51:44.647 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.642+0000 7f8db895d700 1 -- 172.21.15.162:0/2785637964 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8db41550a0 con 0x7f8db40603d0 2020-11-28T21:51:44.647 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.642+0000 7f8dabfff700 1 -- 172.21.15.162:0/2785637964 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8da4024030 con 0x7f8db40603d0 2020-11-28T21:51:44.648 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.642+0000 7f8dabfff700 1 --2- 172.21.15.162:0/2785637964 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8d980554e0 0x7f8d980579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:44.648 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.646+0000 7f8dabfff700 1 -- 172.21.15.162:0/2785637964 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8da406ec90 con 0x7f8db40603d0 2020-11-28T21:51:44.649 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.646+0000 7f8db259c700 1 --2- 172.21.15.162:0/2785637964 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8d980554e0 0x7f8d980579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:44.649 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.646+0000 7f8db259c700 1 --2- 172.21.15.162:0/2785637964 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8d980554e0 0x7f8d980579a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f8d9c00d000 tx=0x7f8d9c009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:44.649 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.650+0000 7f8dabfff700 1 -- 172.21.15.162:0/2785637964 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8da4044f90 con 0x7f8db40603d0 2020-11-28T21:51:44.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.790+0000 7f8db895d700 1 -- 172.21.15.162:0/2785637964 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8db4057050 con 0x7f8db40603d0 2020-11-28T21:51:44.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.794+0000 7f8dabfff700 1 -- 172.21.15.162:0/2785637964 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+16150 (secure 0 0 0) 0x7f8da4022070 con 0x7f8db40603d0 2020-11-28T21:51:44.793 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:51:44.793 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":65,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:51:44.225990+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:51:41.931840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2020-11-28T21:51:43.225560+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:51:44.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.794+0000 7f8db895d700 1 -- 172.21.15.162:0/2785637964 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8d980554e0 msgr2=0x7f8d980579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:44.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.794+0000 7f8db895d700 1 --2- 172.21.15.162:0/2785637964 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8d980554e0 0x7f8d980579a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f8d9c00d000 tx=0x7f8d9c009040).stop 2020-11-28T21:51:44.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.794+0000 7f8db895d700 1 -- 172.21.15.162:0/2785637964 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8db40603d0 msgr2=0x7f8db41553b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:44.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.794+0000 7f8db895d700 1 --2- 172.21.15.162:0/2785637964 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8db40603d0 0x7f8db41553b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7f8da4002f30 tx=0x7f8da4015000).stop 2020-11-28T21:51:44.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.794+0000 7f8db895d700 1 -- 172.21.15.162:0/2785637964 shutdown_connections 2020-11-28T21:51:44.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.794+0000 7f8db895d700 1 --2- 172.21.15.162:0/2785637964 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8d980554e0 0x7f8d980579a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:44.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.794+0000 7f8db895d700 1 --2- 172.21.15.162:0/2785637964 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8db41485c0 0x7f8db409daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:44.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.794+0000 7f8db895d700 1 --2- 172.21.15.162:0/2785637964 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8db40603d0 0x7f8db41553b0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:44.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.794+0000 7f8db895d700 1 --2- 172.21.15.162:0/2785637964 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8db40b4c70 0x7f8db41480d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:44.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.794+0000 7f8db895d700 1 -- 172.21.15.162:0/2785637964 >> 172.21.15.162:0/2785637964 conn(0x7f8db4066a40 msgr2=0x7f8db4140630 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:44.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.798+0000 7f8db895d700 1 -- 172.21.15.162:0/2785637964 shutdown_connections 2020-11-28T21:51:44.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.798+0000 7f8db895d700 1 -- 172.21.15.162:0/2785637964 wait complete. 2020-11-28T21:51:44.807 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2020-11-28T21:51:44.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.922+0000 7f0e85901700 1 Processor -- start 2020-11-28T21:51:44.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.926+0000 7f0e85901700 1 -- start start 2020-11-28T21:51:44.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.926+0000 7f0e85901700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f0e800649c0 0x7f0e80143580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:44.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.926+0000 7f0e85901700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0e800b0bf0 0x7f0e8013fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:44.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.926+0000 7f0e85901700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e80064da0 con 0x7f0e80060200 2020-11-28T21:51:44.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.926+0000 7f0e85901700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e801429b0 con 0x7f0e800649c0 2020-11-28T21:51:44.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.926+0000 7f0e85901700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e80142ba0 con 0x7f0e800b0bf0 2020-11-28T21:51:44.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.926+0000 7f0e7e7fc700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f0e80060200 0x7f0e8005b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:47558/0 (socket says 172.21.15.162:47558) 2020-11-28T21:51:44.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.926+0000 7f0e7e7fc700 1 -- 172.21.15.162:0/1939924919 learned_addr learned my addr 172.21.15.162:0/1939924919 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:51:44.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.926+0000 7f0e7f7fe700 1 --2- 172.21.15.162:0/1939924919 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0e800b0bf0 0x7f0e8013fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:44.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.926+0000 7f0e7effd700 1 --2- 172.21.15.162:0/1939924919 >> v2:172.21.15.118:3300/0 conn(0x7f0e800649c0 0x7f0e80143580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:44.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.926+0000 7f0e7dffb700 1 -- 172.21.15.162:0/1939924919 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 273763725 0 0) 0x7f0e80064da0 con 0x7f0e80060200 2020-11-28T21:51:44.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.926+0000 7f0e7dffb700 1 -- 172.21.15.162:0/1939924919 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e64003230 con 0x7f0e80060200 2020-11-28T21:51:44.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.926+0000 7f0e7effd700 1 -- 172.21.15.162:0/1939924919 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0e800b0bf0 msgr2=0x7f0e8013fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:51:44.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.926+0000 7f0e7effd700 1 --2- 172.21.15.162:0/1939924919 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0e800b0bf0 0x7f0e8013fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:44.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.926+0000 7f0e7effd700 1 -- 172.21.15.162:0/1939924919 >> v1:172.21.15.162:6789/0 conn(0x7f0e80060200 legacy=0x7f0e8005b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:44.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.926+0000 7f0e7effd700 1 -- 172.21.15.162:0/1939924919 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e80142e40 con 0x7f0e800649c0 2020-11-28T21:51:44.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.926+0000 7f0e7effd700 1 --2- 172.21.15.162:0/1939924919 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0e800649c0 0x7f0e80143580 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f0e70002f30 tx=0x7f0e70006dc0).ready entity=mon.0 client_cookie=163397fa51af7656 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:44.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.926+0000 7f0e7dffb700 1 -- 172.21.15.162:0/1939924919 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e700096a0 con 0x7f0e800649c0 2020-11-28T21:51:44.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.926+0000 7f0e7dffb700 1 -- 172.21.15.162:0/1939924919 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e70009800 con 0x7f0e800649c0 2020-11-28T21:51:44.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.926+0000 7f0e85901700 1 -- 172.21.15.162:0/1939924919 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0e800649c0 msgr2=0x7f0e80143580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:44.931 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.926+0000 7f0e7dffb700 1 -- 172.21.15.162:0/1939924919 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e70009b60 con 0x7f0e800649c0 2020-11-28T21:51:44.931 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.926+0000 7f0e85901700 1 --2- 172.21.15.162:0/1939924919 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0e800649c0 0x7f0e80143580 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f0e70002f30 tx=0x7f0e70006dc0).stop 2020-11-28T21:51:44.931 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.930+0000 7f0e85901700 1 -- 172.21.15.162:0/1939924919 shutdown_connections 2020-11-28T21:51:44.931 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.930+0000 7f0e85901700 1 --2- 172.21.15.162:0/1939924919 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0e800649c0 0x7f0e80143580 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:44.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.930+0000 7f0e85901700 1 --2- 172.21.15.162:0/1939924919 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0e800b0bf0 0x7f0e8013fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:44.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.930+0000 7f0e85901700 1 -- 172.21.15.162:0/1939924919 >> 172.21.15.162:0/1939924919 conn(0x7f0e80061740 msgr2=0x7f0e80061b50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:44.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.930+0000 7f0e85901700 1 -- 172.21.15.162:0/1939924919 shutdown_connections 2020-11-28T21:51:44.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.930+0000 7f0e85901700 1 -- 172.21.15.162:0/1939924919 wait complete. 2020-11-28T21:51:44.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.930+0000 7f0e85901700 1 Processor -- start 2020-11-28T21:51:44.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.930+0000 7f0e85901700 1 -- start start 2020-11-28T21:51:44.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.930+0000 7f0e85901700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0e80060200 0x7f0e80066610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:44.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.930+0000 7f0e85901700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e800649c0 0x7f0e800aee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:44.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.930+0000 7f0e85901700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0e800b0bf0 0x7f0e800af340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:44.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.930+0000 7f0e85901700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e800ad210 con 0x7f0e800b0bf0 2020-11-28T21:51:44.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.930+0000 7f0e85901700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e800ad4b0 con 0x7f0e800649c0 2020-11-28T21:51:44.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.930+0000 7f0e85901700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e800ad750 con 0x7f0e80060200 2020-11-28T21:51:44.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.930+0000 7f0e7e7fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e800649c0 0x7f0e800aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:44.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.930+0000 7f0e7e7fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e800649c0 0x7f0e800aee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40720/0 (socket says 172.21.15.162:40720) 2020-11-28T21:51:44.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.930+0000 7f0e7e7fc700 1 -- 172.21.15.162:0/1930835876 learned_addr learned my addr 172.21.15.162:0/1930835876 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:44.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.930+0000 7f0e7f7fe700 1 --2- 172.21.15.162:0/1930835876 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0e800b0bf0 0x7f0e800af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:44.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.930+0000 7f0e7effd700 1 --2- 172.21.15.162:0/1930835876 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0e80060200 0x7f0e80066610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:44.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.930+0000 7f0e7e7fc700 1 -- 172.21.15.162:0/1930835876 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0e80060200 msgr2=0x7f0e80066610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:44.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.930+0000 7f0e7e7fc700 1 --2- 172.21.15.162:0/1930835876 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0e80060200 0x7f0e80066610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:44.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.930+0000 7f0e7e7fc700 1 -- 172.21.15.162:0/1930835876 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0e800b0bf0 msgr2=0x7f0e800af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:44.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.930+0000 7f0e7e7fc700 1 --2- 172.21.15.162:0/1930835876 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0e800b0bf0 0x7f0e800af340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:44.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.930+0000 7f0e7e7fc700 1 -- 172.21.15.162:0/1930835876 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e80066e00 con 0x7f0e800649c0 2020-11-28T21:51:44.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.930+0000 7f0e7f7fe700 1 --2- 172.21.15.162:0/1930835876 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0e800b0bf0 0x7f0e800af340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:51:44.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.930+0000 7f0e7e7fc700 1 --2- 172.21.15.162:0/1930835876 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e800649c0 0x7f0e800aee50 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7f0e740010d0 tx=0x7f0e74007a70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:44.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.934+0000 7f0e6ffff700 1 -- 172.21.15.162:0/1930835876 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e74007320 con 0x7f0e800649c0 2020-11-28T21:51:44.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.934+0000 7f0e6ffff700 1 -- 172.21.15.162:0/1930835876 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e74007480 con 0x7f0e800649c0 2020-11-28T21:51:44.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.934+0000 7f0e85901700 1 -- 172.21.15.162:0/1930835876 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e800ad9f0 con 0x7f0e800649c0 2020-11-28T21:51:44.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.934+0000 7f0e6ffff700 1 -- 172.21.15.162:0/1930835876 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e740057d0 con 0x7f0e800649c0 2020-11-28T21:51:44.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.934+0000 7f0e85901700 1 -- 172.21.15.162:0/1930835876 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e80146220 con 0x7f0e800649c0 2020-11-28T21:51:44.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.934+0000 7f0e85901700 1 -- 172.21.15.162:0/1930835876 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e801455b0 con 0x7f0e800649c0 2020-11-28T21:51:44.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.934+0000 7f0e6ffff700 1 -- 172.21.15.162:0/1930835876 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0e74010030 con 0x7f0e800649c0 2020-11-28T21:51:44.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.934+0000 7f0e6ffff700 1 --2- 172.21.15.162:0/1930835876 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0e5c055440 0x7f0e5c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:44.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.934+0000 7f0e7effd700 1 --2- 172.21.15.162:0/1930835876 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0e5c055440 0x7f0e5c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:44.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.934+0000 7f0e6ffff700 1 -- 172.21.15.162:0/1930835876 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0e7406ddc0 con 0x7f0e800649c0 2020-11-28T21:51:44.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.934+0000 7f0e7effd700 1 --2- 172.21.15.162:0/1930835876 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0e5c055440 0x7f0e5c057900 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f0e70019000 tx=0x7f0e70015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:44.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:44.942+0000 7f0e6ffff700 1 -- 172.21.15.162:0/1930835876 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0e801455b0 con 0x7f0e800649c0 2020-11-28T21:51:45.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.130+0000 7f0e85901700 1 -- 172.21.15.162:0/1930835876 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f0e801455b0 con 0x7f0e800649c0 2020-11-28T21:51:45.239 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.242+0000 7f0e6ffff700 1 -- 172.21.15.162:0/1930835876 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v27) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0e801455b0 con 0x7f0e800649c0 2020-11-28T21:51:45.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.242+0000 7f0e85901700 1 -- 172.21.15.162:0/1930835876 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0e5c055440 msgr2=0x7f0e5c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:45.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.242+0000 7f0e85901700 1 --2- 172.21.15.162:0/1930835876 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0e5c055440 0x7f0e5c057900 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f0e70019000 tx=0x7f0e70015000).stop 2020-11-28T21:51:45.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.242+0000 7f0e85901700 1 -- 172.21.15.162:0/1930835876 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e800649c0 msgr2=0x7f0e800aee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:45.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.242+0000 7f0e85901700 1 --2- 172.21.15.162:0/1930835876 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e800649c0 0x7f0e800aee50 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7f0e740010d0 tx=0x7f0e74007a70).stop 2020-11-28T21:51:45.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.242+0000 7f0e85901700 1 -- 172.21.15.162:0/1930835876 shutdown_connections 2020-11-28T21:51:45.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.242+0000 7f0e85901700 1 --2- 172.21.15.162:0/1930835876 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0e5c055440 0x7f0e5c057900 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:45.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.242+0000 7f0e85901700 1 --2- 172.21.15.162:0/1930835876 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0e80060200 0x7f0e80066610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:45.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.242+0000 7f0e85901700 1 --2- 172.21.15.162:0/1930835876 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0e800b0bf0 0x7f0e800af340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:45.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.242+0000 7f0e85901700 1 --2- 172.21.15.162:0/1930835876 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e800649c0 0x7f0e800aee50 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:45.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.242+0000 7f0e85901700 1 -- 172.21.15.162:0/1930835876 >> 172.21.15.162:0/1930835876 conn(0x7f0e80061740 msgr2=0x7f0e800b5420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:45.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.242+0000 7f0e85901700 1 -- 172.21.15.162:0/1930835876 shutdown_connections 2020-11-28T21:51:45.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.242+0000 7f0e85901700 1 -- 172.21.15.162:0/1930835876 wait complete. 2020-11-28T21:51:45.260 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:51:45.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.390+0000 7f9e5759e700 1 Processor -- start 2020-11-28T21:51:45.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.390+0000 7f9e5759e700 1 -- start start 2020-11-28T21:51:45.392 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.394+0000 7f9e5759e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9e58143da0 0x7f9e5813d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:45.393 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.394+0000 7f9e5759e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f9e580b1500 0x7f9e5813da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:45.393 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.394+0000 7f9e5759e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e58066df0 con 0x7f9e58066a10 2020-11-28T21:51:45.393 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.394+0000 7f9e5759e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e58060100 con 0x7f9e580b1500 2020-11-28T21:51:45.393 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.394+0000 7f9e5759e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e580b12b0 con 0x7f9e58143da0 2020-11-28T21:51:45.394 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.394+0000 7f9e55d9b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9e58143da0 0x7f9e5813d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:45.394 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.394+0000 7f9e56d9d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f9e580b1500 0x7f9e5813da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:45.394 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.394+0000 7f9e55d9b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9e58143da0 0x7f9e5813d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40728/0 (socket says 172.21.15.162:40728) 2020-11-28T21:51:45.394 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.394+0000 7f9e55d9b700 1 -- 172.21.15.162:0/2652143227 learned_addr learned my addr 172.21.15.162:0/2652143227 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:45.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.394+0000 7f9e55d9b700 1 -- 172.21.15.162:0/2652143227 >> v2:172.21.15.118:3300/0 conn(0x7f9e580b1500 msgr2=0x7f9e5813da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:51:45.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.394+0000 7f9e55d9b700 1 --2- 172.21.15.162:0/2652143227 >> v2:172.21.15.118:3300/0 conn(0x7f9e580b1500 0x7f9e5813da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:45.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.394+0000 7f9e55d9b700 1 -- 172.21.15.162:0/2652143227 >> v1:172.21.15.118:6789/0 conn(0x7f9e58066a10 legacy=0x7f9e58061760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:45.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.394+0000 7f9e5559a700 1 -- 172.21.15.162:0/2652143227 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3090325167 0 0) 0x7f9e58066df0 con 0x7f9e58066a10 2020-11-28T21:51:45.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.394+0000 7f9e55d9b700 1 -- 172.21.15.162:0/2652143227 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e580b0d70 con 0x7f9e58143da0 2020-11-28T21:51:45.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.394+0000 7f9e56d9d700 1 --2- 172.21.15.162:0/2652143227 >> v2:172.21.15.118:3300/0 conn(0x7f9e580b1500 0x7f9e5813da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:51:45.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.394+0000 7f9e55d9b700 1 --2- 172.21.15.162:0/2652143227 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e58143da0 0x7f9e5813d530 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7f9e4c002f30 tx=0x7f9e4c006e80).ready entity=mon.1 client_cookie=d0b78e1a42ca09be server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:45.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.394+0000 7f9e5559a700 1 -- 172.21.15.162:0/2652143227 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e4c0097f0 con 0x7f9e58143da0 2020-11-28T21:51:45.398 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.394+0000 7f9e5559a700 1 -- 172.21.15.162:0/2652143227 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e4c009950 con 0x7f9e58143da0 2020-11-28T21:51:45.398 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.394+0000 7f9e5559a700 1 -- 172.21.15.162:0/2652143227 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e4c009cb0 con 0x7f9e58143da0 2020-11-28T21:51:45.400 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.394+0000 7f9e5759e700 1 -- 172.21.15.162:0/2652143227 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e58143da0 msgr2=0x7f9e5813d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:45.400 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.394+0000 7f9e5759e700 1 --2- 172.21.15.162:0/2652143227 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e58143da0 0x7f9e5813d530 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7f9e4c002f30 tx=0x7f9e4c006e80).stop 2020-11-28T21:51:45.400 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.398+0000 7f9e5759e700 1 -- 172.21.15.162:0/2652143227 shutdown_connections 2020-11-28T21:51:45.400 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.398+0000 7f9e5759e700 1 --2- 172.21.15.162:0/2652143227 >> v2:172.21.15.118:3300/0 conn(0x7f9e580b1500 0x7f9e5813da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:45.401 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.398+0000 7f9e5759e700 1 --2- 172.21.15.162:0/2652143227 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e58143da0 0x7f9e5813d530 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:45.401 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.398+0000 7f9e5759e700 1 -- 172.21.15.162:0/2652143227 >> 172.21.15.162:0/2652143227 conn(0x7f9e580b5da0 msgr2=0x7f9e580b6180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:45.401 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.398+0000 7f9e5759e700 1 -- 172.21.15.162:0/2652143227 shutdown_connections 2020-11-28T21:51:45.401 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.398+0000 7f9e5759e700 1 -- 172.21.15.162:0/2652143227 wait complete. 2020-11-28T21:51:45.404 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.398+0000 7f9e5759e700 1 Processor -- start 2020-11-28T21:51:45.404 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.398+0000 7f9e5759e700 1 -- start start 2020-11-28T21:51:45.405 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.402+0000 7f9e5759e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9e58066a10 0x7f9e580af9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:45.405 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.402+0000 7f9e5759e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e580b1500 0x7f9e58151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:45.406 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.402+0000 7f9e5759e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e58143da0 0x7f9e58151f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:45.406 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.402+0000 7f9e5759e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e58149db0 con 0x7f9e58143da0 2020-11-28T21:51:45.407 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.402+0000 7f9e5759e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e58151660 con 0x7f9e580b1500 2020-11-28T21:51:45.407 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.402+0000 7f9e5759e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e580ae3d0 con 0x7f9e58066a10 2020-11-28T21:51:45.407 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.402+0000 7f9e55d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e580b1500 0x7f9e58151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:45.407 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.402+0000 7f9e55d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e580b1500 0x7f9e58151a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40734/0 (socket says 172.21.15.162:40734) 2020-11-28T21:51:45.407 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.402+0000 7f9e55d9b700 1 -- 172.21.15.162:0/452325167 learned_addr learned my addr 172.21.15.162:0/452325167 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:45.408 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.402+0000 7f9e56d9d700 1 --2- 172.21.15.162:0/452325167 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e58143da0 0x7f9e58151f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:45.408 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.402+0000 7f9e5659c700 1 --2- 172.21.15.162:0/452325167 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9e58066a10 0x7f9e580af9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:45.408 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.402+0000 7f9e55d9b700 1 -- 172.21.15.162:0/452325167 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9e58066a10 msgr2=0x7f9e580af9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:45.408 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.402+0000 7f9e55d9b700 1 --2- 172.21.15.162:0/452325167 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9e58066a10 0x7f9e580af9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:45.409 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.402+0000 7f9e55d9b700 1 -- 172.21.15.162:0/452325167 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e58143da0 msgr2=0x7f9e58151f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:45.409 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.402+0000 7f9e55d9b700 1 --2- 172.21.15.162:0/452325167 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e58143da0 0x7f9e58151f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:45.409 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.402+0000 7f9e55d9b700 1 -- 172.21.15.162:0/452325167 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e580b57c0 con 0x7f9e580b1500 2020-11-28T21:51:45.409 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.402+0000 7f9e55d9b700 1 --2- 172.21.15.162:0/452325167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e580b1500 0x7f9e58151a50 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7f9e4c002f30 tx=0x7f9e4c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:45.410 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.402+0000 7f9e477fe700 1 -- 172.21.15.162:0/452325167 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e4c009e60 con 0x7f9e580b1500 2020-11-28T21:51:45.410 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.402+0000 7f9e477fe700 1 -- 172.21.15.162:0/452325167 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e4c009c50 con 0x7f9e580b1500 2020-11-28T21:51:45.410 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.402+0000 7f9e5759e700 1 -- 172.21.15.162:0/452325167 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e580b4660 con 0x7f9e580b1500 2020-11-28T21:51:45.410 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.402+0000 7f9e477fe700 1 -- 172.21.15.162:0/452325167 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e4c01e650 con 0x7f9e580b1500 2020-11-28T21:51:45.411 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.402+0000 7f9e5759e700 1 -- 172.21.15.162:0/452325167 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e580b5a60 con 0x7f9e580b1500 2020-11-28T21:51:45.411 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.402+0000 7f9e477fe700 1 -- 172.21.15.162:0/452325167 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9e4c025030 con 0x7f9e580b1500 2020-11-28T21:51:45.411 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.402+0000 7f9e5759e700 1 -- 172.21.15.162:0/452325167 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e580b5560 con 0x7f9e580b1500 2020-11-28T21:51:45.411 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.406+0000 7f9e477fe700 1 --2- 172.21.15.162:0/452325167 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9e34055440 0x7f9e34057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:45.411 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.406+0000 7f9e5659c700 1 --2- 172.21.15.162:0/452325167 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9e34055440 0x7f9e34057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:45.412 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.406+0000 7f9e477fe700 1 -- 172.21.15.162:0/452325167 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9e4c06f060 con 0x7f9e580b1500 2020-11-28T21:51:45.412 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.406+0000 7f9e5659c700 1 --2- 172.21.15.162:0/452325167 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9e34055440 0x7f9e34057900 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f9e4800b000 tx=0x7f9e48009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:45.415 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.414+0000 7f9e477fe700 1 -- 172.21.15.162:0/452325167 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9e4c045250 con 0x7f9e580b1500 2020-11-28T21:51:45.598 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.598+0000 7f9e5759e700 1 -- 172.21.15.162:0/452325167 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e58057050 con 0x7f9e580b1500 2020-11-28T21:51:45.599 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.598+0000 7f9e477fe700 1 -- 172.21.15.162:0/452325167 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+2502 (secure 0 0 0) 0x7f9e4c023070 con 0x7f9e580b1500 2020-11-28T21:51:45.599 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:51:45.599 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":27,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5342,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6837/3666321033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3666321033},{"type":"v1","addr":"172.21.15.162:6837","nonce":3666321033}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":25},{"gid":5769,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/4179306206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":4179306206},{"type":"v1","addr":"172.21.15.118:6833","nonce":4179306206}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":24}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:51:44.226383+0000","modified":"2020-11-28T21:51:45.230212+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5869},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5869":{"gid":5869,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6835/1433205970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1433205970},{"type":"v1","addr":"172.21.15.162:6835","nonce":1433205970}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-11-28T21:51:45.601 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.602+0000 7f9e5759e700 1 -- 172.21.15.162:0/452325167 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9e34055440 msgr2=0x7f9e34057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:45.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.602+0000 7f9e5759e700 1 --2- 172.21.15.162:0/452325167 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9e34055440 0x7f9e34057900 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f9e4800b000 tx=0x7f9e48009040).stop 2020-11-28T21:51:45.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.602+0000 7f9e5759e700 1 -- 172.21.15.162:0/452325167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e580b1500 msgr2=0x7f9e58151a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:45.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.602+0000 7f9e5759e700 1 --2- 172.21.15.162:0/452325167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e580b1500 0x7f9e58151a50 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7f9e4c002f30 tx=0x7f9e4c015000).stop 2020-11-28T21:51:45.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.602+0000 7f9e5759e700 1 -- 172.21.15.162:0/452325167 shutdown_connections 2020-11-28T21:51:45.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.602+0000 7f9e5759e700 1 --2- 172.21.15.162:0/452325167 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9e34055440 0x7f9e34057900 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:45.603 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.602+0000 7f9e5759e700 1 --2- 172.21.15.162:0/452325167 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9e58066a10 0x7f9e580af9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:45.603 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.602+0000 7f9e5759e700 1 --2- 172.21.15.162:0/452325167 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e58143da0 0x7f9e58151f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:45.603 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.602+0000 7f9e5759e700 1 --2- 172.21.15.162:0/452325167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e580b1500 0x7f9e58151a50 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:45.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.602+0000 7f9e5759e700 1 -- 172.21.15.162:0/452325167 >> 172.21.15.162:0/452325167 conn(0x7f9e580b5da0 msgr2=0x7f9e5813e670 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:45.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.602+0000 7f9e5759e700 1 -- 172.21.15.162:0/452325167 shutdown_connections 2020-11-28T21:51:45.605 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.602+0000 7f9e5759e700 1 -- 172.21.15.162:0/452325167 wait complete. 2020-11-28T21:51:45.605 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 27 2020-11-28T21:51:45.615 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:51:45.724 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.726+0000 7f72f8959700 1 Processor -- start 2020-11-28T21:51:45.725 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.726+0000 7f72f8959700 1 -- start start 2020-11-28T21:51:45.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.726+0000 7f72f8959700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f72f40b5bf0 0x7f72f4142980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:45.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.726+0000 7f72f8959700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f72f40b3a60 0x7f72f413fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:45.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.726+0000 7f72f8959700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72f405f380 con 0x7f72f4060cb0 2020-11-28T21:51:45.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.726+0000 7f72f8959700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72f4143930 con 0x7f72f40b5bf0 2020-11-28T21:51:45.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.726+0000 7f72f8959700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72f405f5b0 con 0x7f72f40b3a60 2020-11-28T21:51:45.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.726+0000 7f72f37fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f72f40b5bf0 0x7f72f4142980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:45.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.726+0000 7f72f2ffd700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f72f4060cb0 0x7f72f4060230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:47588/0 (socket says 172.21.15.162:47588) 2020-11-28T21:51:45.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.726+0000 7f72f2ffd700 1 -- 172.21.15.162:0/2219982762 learned_addr learned my addr 172.21.15.162:0/2219982762 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:51:45.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.726+0000 7f72f3fff700 1 --2- 172.21.15.162:0/2219982762 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f72f40b3a60 0x7f72f413fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:45.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.726+0000 7f72f37fe700 1 -- 172.21.15.162:0/2219982762 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f72f40b3a60 msgr2=0x7f72f413fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:51:45.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.726+0000 7f72f37fe700 1 --2- 172.21.15.162:0/2219982762 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f72f40b3a60 0x7f72f413fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:45.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.726+0000 7f72f27fc700 1 -- 172.21.15.162:0/2219982762 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1502089347 0 0) 0x7f72f405f380 con 0x7f72f4060cb0 2020-11-28T21:51:45.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.726+0000 7f72f37fe700 1 -- 172.21.15.162:0/2219982762 >> v1:172.21.15.162:6789/0 conn(0x7f72f4060cb0 legacy=0x7f72f4060230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:45.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.726+0000 7f72f37fe700 1 -- 172.21.15.162:0/2219982762 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f405f850 con 0x7f72f40b5bf0 2020-11-28T21:51:45.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.726+0000 7f72f37fe700 1 --2- 172.21.15.162:0/2219982762 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f72f40b5bf0 0x7f72f4142980 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7f72ec0032b0 tx=0x7f72ec0091a0).ready entity=mon.1 client_cookie=2e550bff69b9e92e server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:45.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.726+0000 7f72f27fc700 1 -- 172.21.15.162:0/2219982762 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72ec00b9a0 con 0x7f72f40b5bf0 2020-11-28T21:51:45.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.726+0000 7f72f27fc700 1 -- 172.21.15.162:0/2219982762 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72ec00bb00 con 0x7f72f40b5bf0 2020-11-28T21:51:45.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.726+0000 7f72f27fc700 1 -- 172.21.15.162:0/2219982762 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72ec00bc70 con 0x7f72f40b5bf0 2020-11-28T21:51:45.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.726+0000 7f72f8959700 1 -- 172.21.15.162:0/2219982762 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f72f40b5bf0 msgr2=0x7f72f4142980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:45.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.726+0000 7f72f8959700 1 --2- 172.21.15.162:0/2219982762 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f72f40b5bf0 0x7f72f4142980 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7f72ec0032b0 tx=0x7f72ec0091a0).stop 2020-11-28T21:51:45.734 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.730+0000 7f72f8959700 1 -- 172.21.15.162:0/2219982762 shutdown_connections 2020-11-28T21:51:45.734 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.730+0000 7f72f8959700 1 --2- 172.21.15.162:0/2219982762 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f72f40b5bf0 0x7f72f4142980 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:45.734 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.730+0000 7f72f8959700 1 --2- 172.21.15.162:0/2219982762 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f72f40b3a60 0x7f72f413fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:45.735 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.730+0000 7f72f8959700 1 -- 172.21.15.162:0/2219982762 >> 172.21.15.162:0/2219982762 conn(0x7f72f405b970 msgr2=0x7f72f405bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:45.735 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.730+0000 7f72f8959700 1 -- 172.21.15.162:0/2219982762 shutdown_connections 2020-11-28T21:51:45.735 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.730+0000 7f72f8959700 1 -- 172.21.15.162:0/2219982762 wait complete. 2020-11-28T21:51:45.735 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.730+0000 7f72f8959700 1 Processor -- start 2020-11-28T21:51:45.736 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.730+0000 7f72f8959700 1 -- start start 2020-11-28T21:51:45.736 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.730+0000 7f72f8959700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f72f4060cb0 0x7f72f4154ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:45.736 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.730+0000 7f72f8959700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f72f40b3a60 0x7f72f40ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:45.736 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.730+0000 7f72f8959700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f72f40b5bf0 0x7f72f4145ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:45.736 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.730+0000 7f72f8959700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f409e7c0 con 0x7f72f40b3a60 2020-11-28T21:51:45.737 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.730+0000 7f72f8959700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f409ea60 con 0x7f72f4060cb0 2020-11-28T21:51:45.737 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.730+0000 7f72f8959700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72f4155690 con 0x7f72f40b5bf0 2020-11-28T21:51:45.737 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.730+0000 7f72f37fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f72f4060cb0 0x7f72f4154ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:45.738 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.730+0000 7f72f37fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f72f4060cb0 0x7f72f4154ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40748/0 (socket says 172.21.15.162:40748) 2020-11-28T21:51:45.738 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.730+0000 7f72f37fe700 1 -- 172.21.15.162:0/4040734875 learned_addr learned my addr 172.21.15.162:0/4040734875 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:45.738 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.730+0000 7f72f2ffd700 1 --2- 172.21.15.162:0/4040734875 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f72f40b3a60 0x7f72f40ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:45.739 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.730+0000 7f72f3fff700 1 --2- 172.21.15.162:0/4040734875 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f72f40b5bf0 0x7f72f4145ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:45.739 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.730+0000 7f72f37fe700 1 -- 172.21.15.162:0/4040734875 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f72f40b5bf0 msgr2=0x7f72f4145ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:45.739 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.730+0000 7f72f37fe700 1 --2- 172.21.15.162:0/4040734875 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f72f40b5bf0 0x7f72f4145ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:45.739 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.730+0000 7f72f37fe700 1 -- 172.21.15.162:0/4040734875 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f72f40b3a60 msgr2=0x7f72f40ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:45.740 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.730+0000 7f72f37fe700 1 --2- 172.21.15.162:0/4040734875 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f72f40b3a60 0x7f72f40ad8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:45.740 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.730+0000 7f72f37fe700 1 -- 172.21.15.162:0/4040734875 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f41553b0 con 0x7f72f4060cb0 2020-11-28T21:51:45.740 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.734+0000 7f72f37fe700 1 --2- 172.21.15.162:0/4040734875 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f72f4060cb0 0x7f72f4154ec0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7f72ec0032b0 tx=0x7f72ec016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:45.740 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.734+0000 7f72f0ff9700 1 -- 172.21.15.162:0/4040734875 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72ec0205f0 con 0x7f72f4060cb0 2020-11-28T21:51:45.740 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.734+0000 7f72f0ff9700 1 -- 172.21.15.162:0/4040734875 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72ec020750 con 0x7f72f4060cb0 2020-11-28T21:51:45.741 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.734+0000 7f72f8959700 1 -- 172.21.15.162:0/4040734875 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72f40ae490 con 0x7f72f4060cb0 2020-11-28T21:51:45.741 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.734+0000 7f72f0ff9700 1 -- 172.21.15.162:0/4040734875 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72ec020ab0 con 0x7f72f4060cb0 2020-11-28T21:51:45.741 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.734+0000 7f72f8959700 1 -- 172.21.15.162:0/4040734875 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72f409efa0 con 0x7f72f4060cb0 2020-11-28T21:51:45.741 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.734+0000 7f72f8959700 1 -- 172.21.15.162:0/4040734875 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72f4057050 con 0x7f72f4060cb0 2020-11-28T21:51:45.742 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.734+0000 7f72f0ff9700 1 -- 172.21.15.162:0/4040734875 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f72ec027030 con 0x7f72f4060cb0 2020-11-28T21:51:45.742 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.734+0000 7f72f0ff9700 1 --2- 172.21.15.162:0/4040734875 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f72d80554e0 0x7f72d80579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:45.742 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.734+0000 7f72f0ff9700 1 -- 172.21.15.162:0/4040734875 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f72ec06f340 con 0x7f72f4060cb0 2020-11-28T21:51:45.742 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.734+0000 7f72f2ffd700 1 --2- 172.21.15.162:0/4040734875 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f72d80554e0 0x7f72d80579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:45.743 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.738+0000 7f72f2ffd700 1 --2- 172.21.15.162:0/4040734875 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f72d80554e0 0x7f72d80579a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f72e400d000 tx=0x7f72e4001160).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:45.743 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.742+0000 7f72f0ff9700 1 -- 172.21.15.162:0/4040734875 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f72ec049070 con 0x7f72f4060cb0 2020-11-28T21:51:45.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.882+0000 7f72f8959700 1 -- 172.21.15.162:0/4040734875 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f72f4068350 con 0x7f72f4060cb0 2020-11-28T21:51:45.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.886+0000 7f72f0ff9700 1 -- 172.21.15.162:0/4040734875 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16137 (secure 0 0 0) 0x7f72ec015090 con 0x7f72f4060cb0 2020-11-28T21:51:45.884 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:51:45.885 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":66,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:51:45.229278+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:51:41.931840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2020-11-28T21:51:43.225560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:51:45.889 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.886+0000 7f72f8959700 1 -- 172.21.15.162:0/4040734875 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f72d80554e0 msgr2=0x7f72d80579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:45.889 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.886+0000 7f72f8959700 1 --2- 172.21.15.162:0/4040734875 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f72d80554e0 0x7f72d80579a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f72e400d000 tx=0x7f72e4001160).stop 2020-11-28T21:51:45.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.886+0000 7f72f8959700 1 -- 172.21.15.162:0/4040734875 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f72f4060cb0 msgr2=0x7f72f4154ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:45.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.886+0000 7f72f8959700 1 --2- 172.21.15.162:0/4040734875 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f72f4060cb0 0x7f72f4154ec0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7f72ec0032b0 tx=0x7f72ec016000).stop 2020-11-28T21:51:45.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.886+0000 7f72f8959700 1 -- 172.21.15.162:0/4040734875 shutdown_connections 2020-11-28T21:51:45.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.886+0000 7f72f8959700 1 --2- 172.21.15.162:0/4040734875 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f72d80554e0 0x7f72d80579a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:45.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.886+0000 7f72f8959700 1 --2- 172.21.15.162:0/4040734875 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f72f40b5bf0 0x7f72f4145ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:45.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.886+0000 7f72f8959700 1 --2- 172.21.15.162:0/4040734875 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f72f4060cb0 0x7f72f4154ec0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:45.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.886+0000 7f72f8959700 1 --2- 172.21.15.162:0/4040734875 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f72f40b3a60 0x7f72f40ad8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:45.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.886+0000 7f72f8959700 1 -- 172.21.15.162:0/4040734875 >> 172.21.15.162:0/4040734875 conn(0x7f72f405b970 msgr2=0x7f72f40b7fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:45.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.886+0000 7f72f8959700 1 -- 172.21.15.162:0/4040734875 shutdown_connections 2020-11-28T21:51:45.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:45.886+0000 7f72f8959700 1 -- 172.21.15.162:0/4040734875 wait complete. 2020-11-28T21:51:45.897 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-11-28T21:51:46.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.010+0000 7f542a39e700 1 Processor -- start 2020-11-28T21:51:46.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.010+0000 7f542a39e700 1 -- start start 2020-11-28T21:51:46.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.010+0000 7f542a39e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f54240b2560 0x7f5424156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:46.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.010+0000 7f542a39e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f542405fa90 0x7f54241593c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:46.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.010+0000 7f542a39e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54240ab0b0 con 0x7f54240b2e80 2020-11-28T21:51:46.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.010+0000 7f542a39e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54240b3260 con 0x7f542405fa90 2020-11-28T21:51:46.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.010+0000 7f542a39e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54240ab2e0 con 0x7f54240b2560 2020-11-28T21:51:46.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.010+0000 7f542b600700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f542405fa90 0x7f54241593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:46.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.010+0000 7f5429b9d700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f54240b2e80 0x7f542405f380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:47598/0 (socket says 172.21.15.162:47598) 2020-11-28T21:51:46.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.010+0000 7f542b600700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f542405fa90 0x7f54241593c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40758/0 (socket says 172.21.15.162:40758) 2020-11-28T21:51:46.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.010+0000 7f5429b9d700 1 -- 172.21.15.162:0/861115254 learned_addr learned my addr 172.21.15.162:0/861115254 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:51:46.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.010+0000 7f542939c700 1 --2- 172.21.15.162:0/861115254 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f54240b2560 0x7f5424156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:46.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.010+0000 7f542b600700 1 -- 172.21.15.162:0/861115254 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f54240b2560 msgr2=0x7f5424156ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:51:46.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.010+0000 7f542b600700 1 --2- 172.21.15.162:0/861115254 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f54240b2560 0x7f5424156ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:46.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.010+0000 7f542b600700 1 -- 172.21.15.162:0/861115254 >> v1:172.21.15.162:6789/0 conn(0x7f54240b2e80 legacy=0x7f542405f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:46.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.010+0000 7f542b600700 1 -- 172.21.15.162:0/861115254 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54240256d0 con 0x7f542405fa90 2020-11-28T21:51:46.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.014+0000 7f542b600700 1 --2- 172.21.15.162:0/861115254 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f542405fa90 0x7f54241593c0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f541c0032b0 tx=0x7f541c0091a0).ready entity=mon.1 client_cookie=e4f0c155e08a602f server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:46.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.014+0000 7f5428b9b700 1 -- 172.21.15.162:0/861115254 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f541c00ba50 con 0x7f542405fa90 2020-11-28T21:51:46.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.014+0000 7f5428b9b700 1 -- 172.21.15.162:0/861115254 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541c00bbb0 con 0x7f542405fa90 2020-11-28T21:51:46.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.014+0000 7f5428b9b700 1 -- 172.21.15.162:0/861115254 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f541c00bd20 con 0x7f542405fa90 2020-11-28T21:51:46.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.014+0000 7f542a39e700 1 -- 172.21.15.162:0/861115254 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f542405fa90 msgr2=0x7f54241593c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:46.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.014+0000 7f542a39e700 1 --2- 172.21.15.162:0/861115254 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f542405fa90 0x7f54241593c0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f541c0032b0 tx=0x7f541c0091a0).stop 2020-11-28T21:51:46.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.014+0000 7f542a39e700 1 -- 172.21.15.162:0/861115254 shutdown_connections 2020-11-28T21:51:46.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.014+0000 7f542a39e700 1 --2- 172.21.15.162:0/861115254 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f542405fa90 0x7f54241593c0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:46.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.014+0000 7f542a39e700 1 --2- 172.21.15.162:0/861115254 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f54240b2560 0x7f5424156ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:46.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.014+0000 7f542a39e700 1 -- 172.21.15.162:0/861115254 >> 172.21.15.162:0/861115254 conn(0x7f5424063b60 msgr2=0x7f5424063f40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:46.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.014+0000 7f542a39e700 1 -- 172.21.15.162:0/861115254 shutdown_connections 2020-11-28T21:51:46.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.014+0000 7f542a39e700 1 -- 172.21.15.162:0/861115254 wait complete. 2020-11-28T21:51:46.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.014+0000 7f542a39e700 1 Processor -- start 2020-11-28T21:51:46.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.014+0000 7f542a39e700 1 -- start start 2020-11-28T21:51:46.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.018+0000 7f542a39e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f542405fa90 0x7f542409b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:46.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.018+0000 7f542a39e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f54240b2560 0x7f5424132860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:46.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.018+0000 7f542a39e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54240b2e80 0x7f5424064840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:46.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.018+0000 7f542a39e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54240677d0 con 0x7f54240b2560 2020-11-28T21:51:46.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.018+0000 7f542a39e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542409aa00 con 0x7f54240b2e80 2020-11-28T21:51:46.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.018+0000 7f542a39e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f542409aca0 con 0x7f542405fa90 2020-11-28T21:51:46.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.018+0000 7f542b600700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54240b2e80 0x7f5424064840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:46.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.018+0000 7f542b600700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54240b2e80 0x7f5424064840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40764/0 (socket says 172.21.15.162:40764) 2020-11-28T21:51:46.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.018+0000 7f542b600700 1 -- 172.21.15.162:0/2936154243 learned_addr learned my addr 172.21.15.162:0/2936154243 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:46.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.018+0000 7f5429b9d700 1 --2- 172.21.15.162:0/2936154243 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f542405fa90 0x7f542409b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:46.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.018+0000 7f542939c700 1 --2- 172.21.15.162:0/2936154243 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f54240b2560 0x7f5424132860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:46.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.018+0000 7f542b600700 1 -- 172.21.15.162:0/2936154243 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f542405fa90 msgr2=0x7f542409b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:46.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.018+0000 7f542b600700 1 --2- 172.21.15.162:0/2936154243 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f542405fa90 0x7f542409b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:46.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.018+0000 7f542b600700 1 -- 172.21.15.162:0/2936154243 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f54240b2560 msgr2=0x7f5424132860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:46.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.018+0000 7f542b600700 1 --2- 172.21.15.162:0/2936154243 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f54240b2560 0x7f5424132860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:46.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.018+0000 7f542b600700 1 -- 172.21.15.162:0/2936154243 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54240673a0 con 0x7f54240b2e80 2020-11-28T21:51:46.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.018+0000 7f542b600700 1 --2- 172.21.15.162:0/2936154243 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54240b2e80 0x7f5424064840 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7f541c0032b0 tx=0x7f541c016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:46.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.018+0000 7f541a7fc700 1 -- 172.21.15.162:0/2936154243 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f541c01d7f0 con 0x7f54240b2e80 2020-11-28T21:51:46.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.018+0000 7f542a39e700 1 -- 172.21.15.162:0/2936154243 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54241303a0 con 0x7f54240b2e80 2020-11-28T21:51:46.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.018+0000 7f541a7fc700 1 -- 172.21.15.162:0/2936154243 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541c021070 con 0x7f54240b2e80 2020-11-28T21:51:46.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.018+0000 7f541a7fc700 1 -- 172.21.15.162:0/2936154243 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f541c01db40 con 0x7f54240b2e80 2020-11-28T21:51:46.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.018+0000 7f542a39e700 1 -- 172.21.15.162:0/2936154243 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5424064f50 con 0x7f54240b2e80 2020-11-28T21:51:46.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.018+0000 7f542a39e700 1 -- 172.21.15.162:0/2936154243 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f542409a860 con 0x7f54240b2e80 2020-11-28T21:51:46.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.018+0000 7f541a7fc700 1 -- 172.21.15.162:0/2936154243 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f541c02d070 con 0x7f54240b2e80 2020-11-28T21:51:46.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.022+0000 7f541a7fc700 1 --2- 172.21.15.162:0/2936154243 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f540c0554d0 0x7f540c057990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:46.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.022+0000 7f541a7fc700 1 -- 172.21.15.162:0/2936154243 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f541c070fd0 con 0x7f54240b2e80 2020-11-28T21:51:46.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.022+0000 7f5429b9d700 1 --2- 172.21.15.162:0/2936154243 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f540c0554d0 0x7f540c057990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:46.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.022+0000 7f5429b9d700 1 --2- 172.21.15.162:0/2936154243 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f540c0554d0 0x7f540c057990 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f542000d000 tx=0x7f5420007bb0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:46.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.022+0000 7f541a7fc700 1 -- 172.21.15.162:0/2936154243 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f542409a860 con 0x7f54240b2e80 2020-11-28T21:51:46.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.210+0000 7f542a39e700 1 -- 172.21.15.162:0/2936154243 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f542409a860 con 0x7f54240b2e80 2020-11-28T21:51:46.217 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.218+0000 7f541a7fc700 1 -- 172.21.15.162:0/2936154243 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v17) v1 ==== 184+0+0 (secure 0 0 0) 0x7f542409a860 con 0x7f54240b2e80 2020-11-28T21:51:46.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.222+0000 7f542a39e700 1 -- 172.21.15.162:0/2936154243 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f540c0554d0 msgr2=0x7f540c057990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:46.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.222+0000 7f542a39e700 1 --2- 172.21.15.162:0/2936154243 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f540c0554d0 0x7f540c057990 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f542000d000 tx=0x7f5420007bb0).stop 2020-11-28T21:51:46.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.222+0000 7f542a39e700 1 -- 172.21.15.162:0/2936154243 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54240b2e80 msgr2=0x7f5424064840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:46.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.222+0000 7f542a39e700 1 --2- 172.21.15.162:0/2936154243 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54240b2e80 0x7f5424064840 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7f541c0032b0 tx=0x7f541c016000).stop 2020-11-28T21:51:46.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.222+0000 7f542a39e700 1 -- 172.21.15.162:0/2936154243 shutdown_connections 2020-11-28T21:51:46.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.222+0000 7f542a39e700 1 --2- 172.21.15.162:0/2936154243 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f540c0554d0 0x7f540c057990 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:46.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.222+0000 7f542a39e700 1 --2- 172.21.15.162:0/2936154243 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f542405fa90 0x7f542409b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:46.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.222+0000 7f542a39e700 1 --2- 172.21.15.162:0/2936154243 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54240b2e80 0x7f5424064840 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:46.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.222+0000 7f542a39e700 1 --2- 172.21.15.162:0/2936154243 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f54240b2560 0x7f5424132860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:46.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.222+0000 7f542a39e700 1 -- 172.21.15.162:0/2936154243 >> 172.21.15.162:0/2936154243 conn(0x7f5424063b60 msgr2=0x7f542405f380 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:46.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.222+0000 7f542a39e700 1 -- 172.21.15.162:0/2936154243 shutdown_connections 2020-11-28T21:51:46.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.222+0000 7f542a39e700 1 -- 172.21.15.162:0/2936154243 wait complete. 2020-11-28T21:51:46.224 INFO:teuthology.orchestra.run.smithi162.stderr:updated caps for client.0 2020-11-28T21:51:46.238 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-11-28T21:51:46.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.366+0000 7f19016dd700 1 Processor -- start 2020-11-28T21:51:46.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.366+0000 7f19016dd700 1 -- start start 2020-11-28T21:51:46.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.366+0000 7f19016dd700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f18fc0b2e80 0x7f18fc05f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:46.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.366+0000 7f19016dd700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18fc0ab0b0 con 0x7f18fc0acb40 2020-11-28T21:51:46.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.366+0000 7f19016dd700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18fc0b3260 con 0x7f18fc0b2560 2020-11-28T21:51:46.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.366+0000 7f19016dd700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc061b50 con 0x7f18fc0b2e80 2020-11-28T21:51:46.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.366+0000 7f18fbfff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f18fc0b2560 0x7f18fc060d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:47612/0 (socket says 172.21.15.162:47612) 2020-11-28T21:51:46.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.366+0000 7f18fbfff700 1 -- 172.21.15.162:0/1630744311 learned_addr learned my addr 172.21.15.162:0/1630744311 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:51:46.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.366+0000 7f1900edc700 1 --2- 172.21.15.162:0/1630744311 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f18fc0b2e80 0x7f18fc05f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:46.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.366+0000 7f18fb7fe700 1 -- 172.21.15.162:0/1630744311 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1854276714 0 0) 0x7f18fc0b3260 con 0x7f18fc0b2560 2020-11-28T21:51:46.371 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.366+0000 7f18fb7fe700 1 -- 172.21.15.162:0/1630744311 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18e8003230 con 0x7f18fc0b2560 2020-11-28T21:51:46.372 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.366+0000 7f18fb7fe700 1 -- 172.21.15.162:0/1630744311 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1658635670 0 0) 0x7f18e8003230 con 0x7f18fc0b2560 2020-11-28T21:51:46.372 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.366+0000 7f1900edc700 1 -- 172.21.15.162:0/1630744311 >> v1:172.21.15.162:6789/0 conn(0x7f18fc0b2560 legacy=0x7f18fc060d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:46.372 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.366+0000 7f1900edc700 1 -- 172.21.15.162:0/1630744311 >> v1:172.21.15.118:6789/0 conn(0x7f18fc0acb40 legacy=0x7f18fc061440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:46.372 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.366+0000 7f1900edc700 1 -- 172.21.15.162:0/1630744311 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18fc0ab2e0 con 0x7f18fc0b2e80 2020-11-28T21:51:46.372 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.366+0000 7f1900edc700 1 --2- 172.21.15.162:0/1630744311 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f18fc0b2e80 0x7f18fc05f380 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f18f40010d0 tx=0x7f18ec0091a0).ready entity=mon.2 client_cookie=5a38d8c02b5a8c4a server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:46.373 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.366+0000 7f18fb7fe700 1 -- 172.21.15.162:0/1630744311 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18e40016d0 con 0x7f18fc0b2e80 2020-11-28T21:51:46.373 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.366+0000 7f18fb7fe700 1 -- 172.21.15.162:0/1630744311 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18ec00b8b0 con 0x7f18fc0b2e80 2020-11-28T21:51:46.373 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.366+0000 7f18fb7fe700 1 -- 172.21.15.162:0/1630744311 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18ec00ba20 con 0x7f18fc0b2e80 2020-11-28T21:51:46.373 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.366+0000 7f19016dd700 1 -- 172.21.15.162:0/1630744311 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f18fc0b2e80 msgr2=0x7f18fc05f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:46.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.366+0000 7f19016dd700 1 --2- 172.21.15.162:0/1630744311 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f18fc0b2e80 0x7f18fc05f380 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f18f40010d0 tx=0x7f18ec0091a0).stop 2020-11-28T21:51:46.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.370+0000 7f19016dd700 1 -- 172.21.15.162:0/1630744311 shutdown_connections 2020-11-28T21:51:46.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.370+0000 7f19016dd700 1 --2- 172.21.15.162:0/1630744311 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f18fc0b2e80 0x7f18fc05f380 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:46.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.370+0000 7f19016dd700 1 -- 172.21.15.162:0/1630744311 >> 172.21.15.162:0/1630744311 conn(0x7f18fc063b60 msgr2=0x7f18fc063f40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:46.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.370+0000 7f19016dd700 1 -- 172.21.15.162:0/1630744311 shutdown_connections 2020-11-28T21:51:46.375 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.370+0000 7f19016dd700 1 -- 172.21.15.162:0/1630744311 wait complete. 2020-11-28T21:51:46.377 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.370+0000 7f19016dd700 1 Processor -- start 2020-11-28T21:51:46.377 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.370+0000 7f19016dd700 1 -- start start 2020-11-28T21:51:46.378 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.370+0000 7f19016dd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18fc0acb40 0x7f18fc066bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:46.378 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.370+0000 7f19016dd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f18fc0b2560 0x7f18fc0b5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:46.379 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.370+0000 7f19016dd700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f18fc0b2e80 0x7f18fc0b5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:46.379 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.370+0000 7f19016dd700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc0b6530 con 0x7f18fc0b2560 2020-11-28T21:51:46.379 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.370+0000 7f19016dd700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc141db0 con 0x7f18fc0acb40 2020-11-28T21:51:46.380 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.370+0000 7f19016dd700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc142050 con 0x7f18fc0b2e80 2020-11-28T21:51:46.380 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.370+0000 7f1900edc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18fc0acb40 0x7f18fc066bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:46.380 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.370+0000 7f1900edc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18fc0acb40 0x7f18fc066bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40774/0 (socket says 172.21.15.162:40774) 2020-11-28T21:51:46.380 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.370+0000 7f1900edc700 1 -- 172.21.15.162:0/518907678 learned_addr learned my addr 172.21.15.162:0/518907678 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:46.381 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.370+0000 7f190293f700 1 --2- 172.21.15.162:0/518907678 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f18fc0b2e80 0x7f18fc0b5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:46.381 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.370+0000 7f18fbfff700 1 --2- 172.21.15.162:0/518907678 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f18fc0b2560 0x7f18fc0b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:46.381 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.374+0000 7f1900edc700 1 -- 172.21.15.162:0/518907678 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f18fc0b2e80 msgr2=0x7f18fc0b5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:46.382 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.374+0000 7f1900edc700 1 --2- 172.21.15.162:0/518907678 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f18fc0b2e80 0x7f18fc0b5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:46.382 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.374+0000 7f1900edc700 1 -- 172.21.15.162:0/518907678 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f18fc0b2560 msgr2=0x7f18fc0b5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:46.382 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.374+0000 7f1900edc700 1 --2- 172.21.15.162:0/518907678 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f18fc0b2560 0x7f18fc0b5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:46.382 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.374+0000 7f1900edc700 1 -- 172.21.15.162:0/518907678 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18fc1422f0 con 0x7f18fc0acb40 2020-11-28T21:51:46.383 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.374+0000 7f1900edc700 1 --2- 172.21.15.162:0/518907678 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18fc0acb40 0x7f18fc066bf0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f18f40010d0 tx=0x7f18fc0b1c30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:46.383 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.374+0000 7f18f97fa700 1 -- 172.21.15.162:0/518907678 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18ec011c30 con 0x7f18fc0acb40 2020-11-28T21:51:46.383 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.374+0000 7f18f97fa700 1 -- 172.21.15.162:0/518907678 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18ec011d90 con 0x7f18fc0acb40 2020-11-28T21:51:46.383 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.374+0000 7f19016dd700 1 -- 172.21.15.162:0/518907678 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18fc142590 con 0x7f18fc0acb40 2020-11-28T21:51:46.383 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.374+0000 7f18f97fa700 1 -- 172.21.15.162:0/518907678 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18ec01d640 con 0x7f18fc0acb40 2020-11-28T21:51:46.384 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.374+0000 7f19016dd700 1 -- 172.21.15.162:0/518907678 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18fc067630 con 0x7f18fc0acb40 2020-11-28T21:51:46.384 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.374+0000 7f19016dd700 1 -- 172.21.15.162:0/518907678 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18fc141b20 con 0x7f18fc0acb40 2020-11-28T21:51:46.384 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.374+0000 7f18f97fa700 1 -- 172.21.15.162:0/518907678 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f18ec024030 con 0x7f18fc0acb40 2020-11-28T21:51:46.384 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.374+0000 7f18f97fa700 1 --2- 172.21.15.162:0/518907678 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f18e00554d0 0x7f18e0057990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:46.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.374+0000 7f18f97fa700 1 -- 172.21.15.162:0/518907678 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f18ec06eff0 con 0x7f18fc0acb40 2020-11-28T21:51:46.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.374+0000 7f18fbfff700 1 --2- 172.21.15.162:0/518907678 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f18e00554d0 0x7f18e0057990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:46.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.378+0000 7f18fbfff700 1 --2- 172.21.15.162:0/518907678 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f18e00554d0 0x7f18e0057990 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f18e400d000 tx=0x7f18e4000b80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:46.386 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.382+0000 7f18f97fa700 1 -- 172.21.15.162:0/518907678 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f18ec0452f0 con 0x7f18fc0acb40 2020-11-28T21:51:46.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.574+0000 7f19016dd700 1 -- 172.21.15.162:0/518907678 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f18fc057050 con 0x7f18fc0acb40 2020-11-28T21:51:46.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.582+0000 7f18f97fa700 1 -- 172.21.15.162:0/518907678 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v18) v1 ==== 184+0+0 (secure 0 0 0) 0x7f18ec022070 con 0x7f18fc0acb40 2020-11-28T21:51:46.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.586+0000 7f19016dd700 1 -- 172.21.15.162:0/518907678 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f18e00554d0 msgr2=0x7f18e0057990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:46.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.586+0000 7f19016dd700 1 --2- 172.21.15.162:0/518907678 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f18e00554d0 0x7f18e0057990 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f18e400d000 tx=0x7f18e4000b80).stop 2020-11-28T21:51:46.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.586+0000 7f19016dd700 1 -- 172.21.15.162:0/518907678 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18fc0acb40 msgr2=0x7f18fc066bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:46.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.586+0000 7f19016dd700 1 --2- 172.21.15.162:0/518907678 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18fc0acb40 0x7f18fc066bf0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f18f40010d0 tx=0x7f18fc0b1c30).stop 2020-11-28T21:51:46.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.586+0000 7f19016dd700 1 -- 172.21.15.162:0/518907678 shutdown_connections 2020-11-28T21:51:46.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.586+0000 7f19016dd700 1 --2- 172.21.15.162:0/518907678 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f18e00554d0 0x7f18e0057990 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:46.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.586+0000 7f19016dd700 1 --2- 172.21.15.162:0/518907678 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f18fc0b2e80 0x7f18fc0b5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:46.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.586+0000 7f19016dd700 1 --2- 172.21.15.162:0/518907678 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18fc0acb40 0x7f18fc066bf0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:46.586 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.586+0000 7f19016dd700 1 --2- 172.21.15.162:0/518907678 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f18fc0b2560 0x7f18fc0b5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:46.586 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.586+0000 7f19016dd700 1 -- 172.21.15.162:0/518907678 >> 172.21.15.162:0/518907678 conn(0x7f18fc063b60 msgr2=0x7f18fc0b0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:46.586 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.586+0000 7f19016dd700 1 -- 172.21.15.162:0/518907678 shutdown_connections 2020-11-28T21:51:46.586 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.586+0000 7f19016dd700 1 -- 172.21.15.162:0/518907678 wait complete. 2020-11-28T21:51:46.587 INFO:teuthology.orchestra.run.smithi162.stderr:updated caps for client.1 2020-11-28T21:51:46.599 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:51:46.706 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.706+0000 7f4d9cde0700 1 Processor -- start 2020-11-28T21:51:46.706 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.706+0000 7f4d9cde0700 1 -- start start 2020-11-28T21:51:46.706 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.706+0000 7f4d9cde0700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f4d98098b30 0x7f4d980994f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:46.706 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.706+0000 7f4d9cde0700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4d98060710 0x7f4d980b1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:46.707 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.706+0000 7f4d9cde0700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d98098f10 con 0x7f4d980b0000 2020-11-28T21:51:46.707 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.706+0000 7f4d9cde0700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d980b1030 con 0x7f4d98098b30 2020-11-28T21:51:46.707 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.706+0000 7f4d9cde0700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d980b0480 con 0x7f4d98060710 2020-11-28T21:51:46.707 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.706+0000 7f4d977fe700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f4d98098b30 0x7f4d980994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:46.708 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.706+0000 7f4d97fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4d98060710 0x7f4d980b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:46.708 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.706+0000 7f4d96ffd700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f4d980b0000 0x7f4d98060000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.162:37774/0 (socket says 172.21.15.162:37774) 2020-11-28T21:51:46.708 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.706+0000 7f4d96ffd700 1 -- 172.21.15.162:0/551612867 learned_addr learned my addr 172.21.15.162:0/551612867 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:51:46.708 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.706+0000 7f4d977fe700 1 -- 172.21.15.162:0/551612867 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4d98060710 msgr2=0x7f4d980b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:51:46.708 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.706+0000 7f4d977fe700 1 --2- 172.21.15.162:0/551612867 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4d98060710 0x7f4d980b1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:46.709 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.706+0000 7f4d977fe700 1 -- 172.21.15.162:0/551612867 >> v1:172.21.15.118:6789/0 conn(0x7f4d980b0000 legacy=0x7f4d98060000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:46.709 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.706+0000 7f4d977fe700 1 -- 172.21.15.162:0/551612867 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d980b0680 con 0x7f4d98098b30 2020-11-28T21:51:46.709 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.706+0000 7f4d977fe700 1 --2- 172.21.15.162:0/551612867 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4d98098b30 0x7f4d980994f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f4d80002f30 tx=0x7f4d80006dc0).ready entity=mon.0 client_cookie=f6b0a7fab509cebc server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:46.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.706+0000 7f4d967fc700 1 -- 172.21.15.162:0/551612867 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d80009760 con 0x7f4d98098b30 2020-11-28T21:51:46.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.706+0000 7f4d967fc700 1 -- 172.21.15.162:0/551612867 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d800098c0 con 0x7f4d98098b30 2020-11-28T21:51:46.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.706+0000 7f4d967fc700 1 -- 172.21.15.162:0/551612867 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d80009c20 con 0x7f4d98098b30 2020-11-28T21:51:46.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.706+0000 7f4d9cde0700 1 -- 172.21.15.162:0/551612867 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4d98098b30 msgr2=0x7f4d980994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:46.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.706+0000 7f4d9cde0700 1 --2- 172.21.15.162:0/551612867 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4d98098b30 0x7f4d980994f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f4d80002f30 tx=0x7f4d80006dc0).stop 2020-11-28T21:51:46.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.710+0000 7f4d9cde0700 1 -- 172.21.15.162:0/551612867 shutdown_connections 2020-11-28T21:51:46.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.710+0000 7f4d9cde0700 1 --2- 172.21.15.162:0/551612867 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4d98060710 0x7f4d980b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:46.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.710+0000 7f4d9cde0700 1 --2- 172.21.15.162:0/551612867 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4d98098b30 0x7f4d980994f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:46.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.710+0000 7f4d9cde0700 1 -- 172.21.15.162:0/551612867 >> 172.21.15.162:0/551612867 conn(0x7f4d9805b970 msgr2=0x7f4d9805bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:46.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.710+0000 7f4d9cde0700 1 -- 172.21.15.162:0/551612867 shutdown_connections 2020-11-28T21:51:46.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.710+0000 7f4d9cde0700 1 -- 172.21.15.162:0/551612867 wait complete. 2020-11-28T21:51:46.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.710+0000 7f4d9cde0700 1 Processor -- start 2020-11-28T21:51:46.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.710+0000 7f4d9cde0700 1 -- start start 2020-11-28T21:51:46.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.710+0000 7f4d9cde0700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4d98060710 0x7f4d980aadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:46.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.710+0000 7f4d9cde0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d98098b30 0x7f4d980ad620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:46.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.710+0000 7f4d9cde0700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4d980b0000 0x7f4d98063650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:46.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.710+0000 7f4d9cde0700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d98155af0 con 0x7f4d98060710 2020-11-28T21:51:46.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.710+0000 7f4d9cde0700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d980ae710 con 0x7f4d98098b30 2020-11-28T21:51:46.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.710+0000 7f4d9cde0700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d980adf30 con 0x7f4d980b0000 2020-11-28T21:51:46.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.710+0000 7f4d96ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d98098b30 0x7f4d980ad620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:46.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.710+0000 7f4d96ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d98098b30 0x7f4d980ad620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40786/0 (socket says 172.21.15.162:40786) 2020-11-28T21:51:46.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.710+0000 7f4d96ffd700 1 -- 172.21.15.162:0/3929132825 learned_addr learned my addr 172.21.15.162:0/3929132825 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:46.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.710+0000 7f4d97fff700 1 --2- 172.21.15.162:0/3929132825 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4d980b0000 0x7f4d98063650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:46.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.710+0000 7f4d977fe700 1 --2- 172.21.15.162:0/3929132825 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4d98060710 0x7f4d980aadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:46.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.710+0000 7f4d97fff700 1 -- 172.21.15.162:0/3929132825 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d98098b30 msgr2=0x7f4d980ad620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:46.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.710+0000 7f4d97fff700 1 --2- 172.21.15.162:0/3929132825 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d98098b30 0x7f4d980ad620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:46.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.710+0000 7f4d97fff700 1 -- 172.21.15.162:0/3929132825 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4d98060710 msgr2=0x7f4d980aadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:46.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.710+0000 7f4d97fff700 1 --2- 172.21.15.162:0/3929132825 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4d98060710 0x7f4d980aadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:46.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.710+0000 7f4d97fff700 1 -- 172.21.15.162:0/3929132825 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d980ae1d0 con 0x7f4d980b0000 2020-11-28T21:51:46.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.710+0000 7f4d96ffd700 1 --2- 172.21.15.162:0/3929132825 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d98098b30 0x7f4d980ad620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:51:46.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.714+0000 7f4d97fff700 1 --2- 172.21.15.162:0/3929132825 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4d980b0000 0x7f4d98063650 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f4d9000c000 tx=0x7f4d900091a0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:46.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.714+0000 7f4d94ff9700 1 -- 172.21.15.162:0/3929132825 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d9000c8d0 con 0x7f4d980b0000 2020-11-28T21:51:46.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.714+0000 7f4d94ff9700 1 -- 172.21.15.162:0/3929132825 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d9000ca30 con 0x7f4d980b0000 2020-11-28T21:51:46.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.714+0000 7f4d9cde0700 1 -- 172.21.15.162:0/3929132825 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d980ae470 con 0x7f4d980b0000 2020-11-28T21:51:46.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.714+0000 7f4d94ff9700 1 -- 172.21.15.162:0/3929132825 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d9000cd90 con 0x7f4d980b0000 2020-11-28T21:51:46.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.714+0000 7f4d9cde0700 1 -- 172.21.15.162:0/3929132825 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d980aead0 con 0x7f4d980b0000 2020-11-28T21:51:46.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.714+0000 7f4d9cde0700 1 -- 172.21.15.162:0/3929132825 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d98057050 con 0x7f4d980b0000 2020-11-28T21:51:46.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.714+0000 7f4d94ff9700 1 -- 172.21.15.162:0/3929132825 <== mon.2 v2:172.21.15.118:3301/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4d90022070 con 0x7f4d980b0000 2020-11-28T21:51:46.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.714+0000 7f4d94ff9700 1 --2- 172.21.15.162:0/3929132825 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4d7c055440 0x7f4d7c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:46.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.714+0000 7f4d94ff9700 1 -- 172.21.15.162:0/3929132825 <== mon.2 v2:172.21.15.118:3301/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4d9006e600 con 0x7f4d980b0000 2020-11-28T21:51:46.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.714+0000 7f4d977fe700 1 --2- 172.21.15.162:0/3929132825 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4d7c055440 0x7f4d7c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:46.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.714+0000 7f4d977fe700 1 --2- 172.21.15.162:0/3929132825 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4d7c055440 0x7f4d7c057900 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f4d8001a000 tx=0x7f4d80015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:46.723 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.726+0000 7f4d94ff9700 1 -- 172.21.15.162:0/3929132825 <== mon.2 v2:172.21.15.118:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4d90044860 con 0x7f4d980b0000 2020-11-28T21:51:46.909 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.910+0000 7f4d9cde0700 1 -- 172.21.15.162:0/3929132825 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d98068350 con 0x7f4d980b0000 2020-11-28T21:51:46.910 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.910+0000 7f4d94ff9700 1 -- 172.21.15.162:0/3929132825 <== mon.2 v2:172.21.15.118:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+2502 (secure 0 0 0) 0x7f4d90016070 con 0x7f4d980b0000 2020-11-28T21:51:46.910 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:51:46.910 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":27,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5342,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6837/3666321033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3666321033},{"type":"v1","addr":"172.21.15.162:6837","nonce":3666321033}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":25},{"gid":5769,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/4179306206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":4179306206},{"type":"v1","addr":"172.21.15.118:6833","nonce":4179306206}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":24}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:51:44.226383+0000","modified":"2020-11-28T21:51:45.230212+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5869},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5869":{"gid":5869,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6835/1433205970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1433205970},{"type":"v1","addr":"172.21.15.162:6835","nonce":1433205970}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-11-28T21:51:46.912 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.914+0000 7f4d9cde0700 1 -- 172.21.15.162:0/3929132825 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4d7c055440 msgr2=0x7f4d7c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:46.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.914+0000 7f4d9cde0700 1 --2- 172.21.15.162:0/3929132825 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4d7c055440 0x7f4d7c057900 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f4d8001a000 tx=0x7f4d80015000).stop 2020-11-28T21:51:46.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.914+0000 7f4d9cde0700 1 -- 172.21.15.162:0/3929132825 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4d980b0000 msgr2=0x7f4d98063650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:46.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.914+0000 7f4d9cde0700 1 --2- 172.21.15.162:0/3929132825 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4d980b0000 0x7f4d98063650 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f4d9000c000 tx=0x7f4d900091a0).stop 2020-11-28T21:51:46.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.914+0000 7f4d9cde0700 1 -- 172.21.15.162:0/3929132825 shutdown_connections 2020-11-28T21:51:46.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.914+0000 7f4d9cde0700 1 --2- 172.21.15.162:0/3929132825 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4d7c055440 0x7f4d7c057900 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:46.914 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.914+0000 7f4d9cde0700 1 --2- 172.21.15.162:0/3929132825 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4d980b0000 0x7f4d98063650 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:46.914 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.914+0000 7f4d9cde0700 1 --2- 172.21.15.162:0/3929132825 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4d98060710 0x7f4d980aadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:46.914 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.914+0000 7f4d9cde0700 1 --2- 172.21.15.162:0/3929132825 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d98098b30 0x7f4d980ad620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:46.914 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.914+0000 7f4d9cde0700 1 -- 172.21.15.162:0/3929132825 >> 172.21.15.162:0/3929132825 conn(0x7f4d9805b970 msgr2=0x7f4d980a8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:46.915 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.914+0000 7f4d9cde0700 1 -- 172.21.15.162:0/3929132825 shutdown_connections 2020-11-28T21:51:46.915 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:46.914+0000 7f4d9cde0700 1 -- 172.21.15.162:0/3929132825 wait complete. 2020-11-28T21:51:46.915 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 27 2020-11-28T21:51:47.227 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-28T21:51:44.226383+0000', 'modified': '2020-11-28T21:51:45.230212+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5869}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5869': {'gid': 5869, 'name': 'a', 'rank': 0, 'incarnation': 26, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6835/1433205970', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6834', 'nonce': 1433205970}, {'type': 'v1', 'addr': '172.21.15.162:6835', 'nonce': 1433205970}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-28T21:51:47.227 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-28T21:51:47.228 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-28T21:51:47.357 INFO:teuthology.orchestra.run.smithi162.stdout:{"cluster_fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","whoami":0,"id":5869,"want_state":"up:active","state":"up:active","rank_uptime":3.1214837129999999,"mdsmap_epoch":27,"osdmap_epoch":66,"osdmap_epoch_barrier":66,"uptime":8.3553041419999996} 2020-11-28T21:51:47.357 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", "id": 5869, "mdsmap_epoch": 27, "osdmap_epoch": 66, "osdmap_epoch_barrier": 66, "rank_uptime": 3.121483713, "state": "up:active", "uptime": 8.355304142, "want_state": "up:active", "whoami": 0 } 2020-11-28T21:51:47.358 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:51:47.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.474+0000 7f23e3476700 1 Processor -- start 2020-11-28T21:51:47.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.474+0000 7f23e3476700 1 -- start start 2020-11-28T21:51:47.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.474+0000 7f23e3476700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f23dc06aeb0 0x7f23dc0a8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:47.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.474+0000 7f23e3476700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f23dc0af5b0 0x7f23dc13fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:47.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.474+0000 7f23e3476700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23dc0b1d40 con 0x7f23dc0acef0 2020-11-28T21:51:47.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.474+0000 7f23e3476700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23dc0b1fe0 con 0x7f23dc06aeb0 2020-11-28T21:51:47.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.474+0000 7f23e3476700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23dc0b2280 con 0x7f23dc0af5b0 2020-11-28T21:51:47.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.474+0000 7f23e1212700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f23dc0acef0 0x7f23dc0a8650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:47638/0 (socket says 172.21.15.162:47638) 2020-11-28T21:51:47.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.474+0000 7f23e1212700 1 -- 172.21.15.162:0/3589049208 learned_addr learned my addr 172.21.15.162:0/3589049208 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:51:47.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.474+0000 7f23e1a13700 1 --2- 172.21.15.162:0/3589049208 >> v2:172.21.15.162:3300/0 conn(0x7f23dc0af5b0 0x7f23dc13fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:47.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.474+0000 7f23e0a11700 1 --2- 172.21.15.162:0/3589049208 >> v2:172.21.15.118:3300/0 conn(0x7f23dc06aeb0 0x7f23dc0a8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:47.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.474+0000 7f23d3fff700 1 -- 172.21.15.162:0/3589049208 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2383491579 0 0) 0x7f23dc0b1d40 con 0x7f23dc0acef0 2020-11-28T21:51:47.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.474+0000 7f23d3fff700 1 -- 172.21.15.162:0/3589049208 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23c0003230 con 0x7f23dc0acef0 2020-11-28T21:51:47.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.474+0000 7f23e1a13700 1 -- 172.21.15.162:0/3589049208 >> v2:172.21.15.118:3300/0 conn(0x7f23dc06aeb0 msgr2=0x7f23dc0a8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:51:47.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.474+0000 7f23e1a13700 1 --2- 172.21.15.162:0/3589049208 >> v2:172.21.15.118:3300/0 conn(0x7f23dc06aeb0 0x7f23dc0a8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:47.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.474+0000 7f23e1a13700 1 -- 172.21.15.162:0/3589049208 >> v1:172.21.15.162:6789/0 conn(0x7f23dc0acef0 legacy=0x7f23dc0a8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:47.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.474+0000 7f23e1a13700 1 -- 172.21.15.162:0/3589049208 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23dc0b2520 con 0x7f23dc0af5b0 2020-11-28T21:51:47.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.474+0000 7f23e0a11700 1 --2- 172.21.15.162:0/3589049208 >> v2:172.21.15.118:3300/0 conn(0x7f23dc06aeb0 0x7f23dc0a8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:51:47.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.474+0000 7f23e1a13700 1 --2- 172.21.15.162:0/3589049208 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23dc0af5b0 0x7f23dc13fab0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7f23d8002f30 tx=0x7f23d8006dc0).ready entity=mon.1 client_cookie=c3a8a1add97dddd6 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:47.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.474+0000 7f23d3fff700 1 -- 172.21.15.162:0/3589049208 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23d80097b0 con 0x7f23dc0af5b0 2020-11-28T21:51:47.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.474+0000 7f23d3fff700 1 -- 172.21.15.162:0/3589049208 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d8009910 con 0x7f23dc0af5b0 2020-11-28T21:51:47.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.474+0000 7f23d3fff700 1 -- 172.21.15.162:0/3589049208 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23d8009c70 con 0x7f23dc0af5b0 2020-11-28T21:51:47.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.474+0000 7f23e3476700 1 -- 172.21.15.162:0/3589049208 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23dc0af5b0 msgr2=0x7f23dc13fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:47.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.474+0000 7f23e3476700 1 --2- 172.21.15.162:0/3589049208 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23dc0af5b0 0x7f23dc13fab0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7f23d8002f30 tx=0x7f23d8006dc0).stop 2020-11-28T21:51:47.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.474+0000 7f23e3476700 1 -- 172.21.15.162:0/3589049208 shutdown_connections 2020-11-28T21:51:47.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.474+0000 7f23e3476700 1 --2- 172.21.15.162:0/3589049208 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23dc0af5b0 0x7f23dc13fab0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:47.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.474+0000 7f23e3476700 1 --2- 172.21.15.162:0/3589049208 >> v2:172.21.15.118:3300/0 conn(0x7f23dc06aeb0 0x7f23dc0a8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:47.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.474+0000 7f23e3476700 1 -- 172.21.15.162:0/3589049208 >> 172.21.15.162:0/3589049208 conn(0x7f23dc062ec0 msgr2=0x7f23dc0632a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:47.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.478+0000 7f23e3476700 1 -- 172.21.15.162:0/3589049208 shutdown_connections 2020-11-28T21:51:47.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.478+0000 7f23e3476700 1 -- 172.21.15.162:0/3589049208 wait complete. 2020-11-28T21:51:47.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.478+0000 7f23e3476700 1 Processor -- start 2020-11-28T21:51:47.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.478+0000 7f23e3476700 1 -- start start 2020-11-28T21:51:47.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.478+0000 7f23e3476700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f23dc0acef0 0x7f23dc0b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:47.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.478+0000 7f23e3476700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23dc0af5b0 0x7f23dc1516d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:47.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.478+0000 7f23e3476700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23dc066de0 0x7f23dc1423c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:47.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.478+0000 7f23e3476700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23dc066840 con 0x7f23dc0acef0 2020-11-28T21:51:47.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.478+0000 7f23e3476700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23dc067310 con 0x7f23dc0af5b0 2020-11-28T21:51:47.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.478+0000 7f23e3476700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23dc066a70 con 0x7f23dc066de0 2020-11-28T21:51:47.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.478+0000 7f23e0a11700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23dc0af5b0 0x7f23dc1516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:47.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.478+0000 7f23e0a11700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23dc0af5b0 0x7f23dc1516d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40802/0 (socket says 172.21.15.162:40802) 2020-11-28T21:51:47.485 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.478+0000 7f23e0a11700 1 -- 172.21.15.162:0/203975885 learned_addr learned my addr 172.21.15.162:0/203975885 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:47.485 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.478+0000 7f23e1a13700 1 --2- 172.21.15.162:0/203975885 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23dc066de0 0x7f23dc1423c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:47.485 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.478+0000 7f23e1212700 1 --2- 172.21.15.162:0/203975885 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f23dc0acef0 0x7f23dc0b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:47.485 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.478+0000 7f23e0a11700 1 -- 172.21.15.162:0/203975885 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23dc066de0 msgr2=0x7f23dc1423c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:47.486 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.478+0000 7f23e0a11700 1 --2- 172.21.15.162:0/203975885 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23dc066de0 0x7f23dc1423c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:47.486 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.478+0000 7f23e0a11700 1 -- 172.21.15.162:0/203975885 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f23dc0acef0 msgr2=0x7f23dc0b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:47.486 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.478+0000 7f23e0a11700 1 --2- 172.21.15.162:0/203975885 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f23dc0acef0 0x7f23dc0b0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:47.486 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.478+0000 7f23e0a11700 1 -- 172.21.15.162:0/203975885 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23dc141f20 con 0x7f23dc0af5b0 2020-11-28T21:51:47.487 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.478+0000 7f23e0a11700 1 --2- 172.21.15.162:0/203975885 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23dc0af5b0 0x7f23dc1516d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7f23cc006910 tx=0x7f23cc008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:47.487 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.482+0000 7f23d27fc700 1 -- 172.21.15.162:0/203975885 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23cc009980 con 0x7f23dc0af5b0 2020-11-28T21:51:47.487 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.482+0000 7f23d27fc700 1 -- 172.21.15.162:0/203975885 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23cc009ae0 con 0x7f23dc0af5b0 2020-11-28T21:51:47.487 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.482+0000 7f23e3476700 1 -- 172.21.15.162:0/203975885 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23dc0664d0 con 0x7f23dc0af5b0 2020-11-28T21:51:47.487 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.482+0000 7f23d27fc700 1 -- 172.21.15.162:0/203975885 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23cc009e40 con 0x7f23dc0af5b0 2020-11-28T21:51:47.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.482+0000 7f23e3476700 1 -- 172.21.15.162:0/203975885 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23dc142c00 con 0x7f23dc0af5b0 2020-11-28T21:51:47.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.482+0000 7f23e3476700 1 -- 172.21.15.162:0/203975885 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23dc1420b0 con 0x7f23dc0af5b0 2020-11-28T21:51:47.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.482+0000 7f23d27fc700 1 -- 172.21.15.162:0/203975885 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f23cc013480 con 0x7f23dc0af5b0 2020-11-28T21:51:47.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.482+0000 7f23d27fc700 1 --2- 172.21.15.162:0/203975885 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f23c4055440 0x7f23c4057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:47.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.482+0000 7f23d27fc700 1 -- 172.21.15.162:0/203975885 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f23cc06e620 con 0x7f23dc0af5b0 2020-11-28T21:51:47.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.482+0000 7f23e1212700 1 --2- 172.21.15.162:0/203975885 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f23c4055440 0x7f23c4057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:47.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.482+0000 7f23e1212700 1 --2- 172.21.15.162:0/203975885 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f23c4055440 0x7f23c4057900 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f23d400d000 tx=0x7f23d4009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:47.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.490+0000 7f23d27fc700 1 -- 172.21.15.162:0/203975885 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f23cc0448c0 con 0x7f23dc0af5b0 2020-11-28T21:51:47.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.630+0000 7f23e3476700 1 -- 172.21.15.162:0/203975885 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23dc057050 con 0x7f23dc0af5b0 2020-11-28T21:51:47.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.630+0000 7f23d27fc700 1 -- 172.21.15.162:0/203975885 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16137 (secure 0 0 0) 0x7f23cc047ee0 con 0x7f23dc0af5b0 2020-11-28T21:51:47.631 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:51:47.632 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":66,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:51:45.229278+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:51:41.931840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2020-11-28T21:51:43.225560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:51:47.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.634+0000 7f23e3476700 1 -- 172.21.15.162:0/203975885 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f23c4055440 msgr2=0x7f23c4057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:47.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.634+0000 7f23e3476700 1 --2- 172.21.15.162:0/203975885 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f23c4055440 0x7f23c4057900 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f23d400d000 tx=0x7f23d4009040).stop 2020-11-28T21:51:47.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.634+0000 7f23e3476700 1 -- 172.21.15.162:0/203975885 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23dc0af5b0 msgr2=0x7f23dc1516d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:47.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.634+0000 7f23e3476700 1 --2- 172.21.15.162:0/203975885 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23dc0af5b0 0x7f23dc1516d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7f23cc006910 tx=0x7f23cc008040).stop 2020-11-28T21:51:47.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.634+0000 7f23e3476700 1 -- 172.21.15.162:0/203975885 shutdown_connections 2020-11-28T21:51:47.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.634+0000 7f23e3476700 1 --2- 172.21.15.162:0/203975885 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f23c4055440 0x7f23c4057900 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:47.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.634+0000 7f23e3476700 1 --2- 172.21.15.162:0/203975885 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23dc066de0 0x7f23dc1423c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:47.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.634+0000 7f23e3476700 1 --2- 172.21.15.162:0/203975885 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f23dc0acef0 0x7f23dc0b0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:47.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.634+0000 7f23e3476700 1 --2- 172.21.15.162:0/203975885 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23dc0af5b0 0x7f23dc1516d0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:47.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.634+0000 7f23e3476700 1 -- 172.21.15.162:0/203975885 >> 172.21.15.162:0/203975885 conn(0x7f23dc062ec0 msgr2=0x7f23dc05b970 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:47.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.634+0000 7f23e3476700 1 -- 172.21.15.162:0/203975885 shutdown_connections 2020-11-28T21:51:47.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.634+0000 7f23e3476700 1 -- 172.21.15.162:0/203975885 wait complete. 2020-11-28T21:51:47.647 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-28T21:51:41.931840+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2020-11-28T21:51:43.225560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-28T21:51:47.648 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:51:47.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.769+0000 7f566359e700 1 Processor -- start 2020-11-28T21:51:47.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.769+0000 7f566359e700 1 -- start start 2020-11-28T21:51:47.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.769+0000 7f566359e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f56640b21f0 0x7f56641434f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:47.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.769+0000 7f566359e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5664141e80 0x7f56640606f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:47.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.769+0000 7f566359e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56640af6a0 con 0x7f56640ae3d0 2020-11-28T21:51:47.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.769+0000 7f566359e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56640b25d0 con 0x7f56640b21f0 2020-11-28T21:51:47.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.769+0000 7f566359e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f566405fff0 con 0x7f5664141e80 2020-11-28T21:51:47.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.769+0000 7f5662d9d700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f56640ae3d0 0x7f5664143f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:47656/0 (socket says 172.21.15.162:47656) 2020-11-28T21:51:47.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.769+0000 7f5662d9d700 1 -- 172.21.15.162:0/779614576 learned_addr learned my addr 172.21.15.162:0/779614576 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:51:47.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.769+0000 7f5661d9b700 1 --2- 172.21.15.162:0/779614576 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5664141e80 0x7f56640606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:47.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.769+0000 7f566259c700 1 --2- 172.21.15.162:0/779614576 >> v2:172.21.15.118:3300/0 conn(0x7f56640b21f0 0x7f56641434f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:47.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.769+0000 7f566159a700 1 -- 172.21.15.162:0/779614576 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 51820379 0 0) 0x7f56640af6a0 con 0x7f56640ae3d0 2020-11-28T21:51:47.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.769+0000 7f566159a700 1 -- 172.21.15.162:0/779614576 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5644003230 con 0x7f56640ae3d0 2020-11-28T21:51:47.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.769+0000 7f5661d9b700 1 -- 172.21.15.162:0/779614576 >> v2:172.21.15.118:3300/0 conn(0x7f56640b21f0 msgr2=0x7f56641434f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:51:47.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.769+0000 7f5661d9b700 1 --2- 172.21.15.162:0/779614576 >> v2:172.21.15.118:3300/0 conn(0x7f56640b21f0 0x7f56641434f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:47.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.769+0000 7f566159a700 1 -- 172.21.15.162:0/779614576 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3942940896 0 0) 0x7f5644003230 con 0x7f56640ae3d0 2020-11-28T21:51:47.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.769+0000 7f5661d9b700 1 -- 172.21.15.162:0/779614576 >> v1:172.21.15.162:6789/0 conn(0x7f56640ae3d0 legacy=0x7f5664143f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:47.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.769+0000 7f5661d9b700 1 -- 172.21.15.162:0/779614576 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56640af8d0 con 0x7f5664141e80 2020-11-28T21:51:47.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.769+0000 7f5661d9b700 1 --2- 172.21.15.162:0/779614576 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5664141e80 0x7f56640606f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f565c001bf0 tx=0x7f564c006dc0).ready entity=mon.2 client_cookie=cd8f90e062a625bd server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:47.776 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.769+0000 7f566159a700 1 -- 172.21.15.162:0/779614576 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f565c001a50 con 0x7f5664141e80 2020-11-28T21:51:47.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.769+0000 7f566159a700 1 -- 172.21.15.162:0/779614576 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f564c00d570 con 0x7f5664141e80 2020-11-28T21:51:47.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.769+0000 7f566159a700 1 -- 172.21.15.162:0/779614576 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f564c00d6e0 con 0x7f5664141e80 2020-11-28T21:51:47.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.769+0000 7f566359e700 1 -- 172.21.15.162:0/779614576 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5664141e80 msgr2=0x7f56640606f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:47.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.769+0000 7f566359e700 1 --2- 172.21.15.162:0/779614576 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5664141e80 0x7f56640606f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f565c001bf0 tx=0x7f564c006dc0).stop 2020-11-28T21:51:47.784 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.769+0000 7f566359e700 1 -- 172.21.15.162:0/779614576 shutdown_connections 2020-11-28T21:51:47.784 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.769+0000 7f566359e700 1 --2- 172.21.15.162:0/779614576 >> v2:172.21.15.118:3300/0 conn(0x7f56640b21f0 0x7f56641434f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:47.784 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.769+0000 7f566359e700 1 --2- 172.21.15.162:0/779614576 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5664141e80 0x7f56640606f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:47.785 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.769+0000 7f566359e700 1 -- 172.21.15.162:0/779614576 >> 172.21.15.162:0/779614576 conn(0x7f56640b57d0 msgr2=0x7f56640b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:47.785 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.773+0000 7f566359e700 1 -- 172.21.15.162:0/779614576 shutdown_connections 2020-11-28T21:51:47.785 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.773+0000 7f566359e700 1 -- 172.21.15.162:0/779614576 wait complete. 2020-11-28T21:51:47.785 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.773+0000 7f566359e700 1 Processor -- start 2020-11-28T21:51:47.785 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.773+0000 7f566359e700 1 -- start start 2020-11-28T21:51:47.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.773+0000 7f566359e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f56640ae3d0 0x7f5664066410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:47.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.773+0000 7f566359e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56640b21f0 0x7f5664154fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:47.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.773+0000 7f566359e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5664141e80 0x7f5664155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:47.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.773+0000 7f566359e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56640ad7c0 con 0x7f5664141e80 2020-11-28T21:51:47.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.773+0000 7f566359e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56640ad930 con 0x7f56640b21f0 2020-11-28T21:51:47.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.773+0000 7f566359e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56640adbd0 con 0x7f56640ae3d0 2020-11-28T21:51:47.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.773+0000 7f5661d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56640b21f0 0x7f5664154fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:47.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.773+0000 7f5661d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56640b21f0 0x7f5664154fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40816/0 (socket says 172.21.15.162:40816) 2020-11-28T21:51:47.788 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.773+0000 7f5661d9b700 1 -- 172.21.15.162:0/2236260698 learned_addr learned my addr 172.21.15.162:0/2236260698 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:47.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.773+0000 7f5662d9d700 1 --2- 172.21.15.162:0/2236260698 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5664141e80 0x7f5664155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:47.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.773+0000 7f566259c700 1 --2- 172.21.15.162:0/2236260698 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f56640ae3d0 0x7f5664066410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:47.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.773+0000 7f5661d9b700 1 -- 172.21.15.162:0/2236260698 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f56640ae3d0 msgr2=0x7f5664066410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:47.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.773+0000 7f5661d9b700 1 --2- 172.21.15.162:0/2236260698 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f56640ae3d0 0x7f5664066410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:47.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.773+0000 7f5661d9b700 1 -- 172.21.15.162:0/2236260698 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5664141e80 msgr2=0x7f5664155490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:47.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.773+0000 7f5661d9b700 1 --2- 172.21.15.162:0/2236260698 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5664141e80 0x7f5664155490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:47.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.773+0000 7f5661d9b700 1 -- 172.21.15.162:0/2236260698 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56640ade70 con 0x7f56640b21f0 2020-11-28T21:51:47.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.773+0000 7f5661d9b700 1 --2- 172.21.15.162:0/2236260698 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56640b21f0 0x7f5664154fa0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7f565c001bf0 tx=0x7f564c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:47.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.773+0000 7f565b7fe700 1 -- 172.21.15.162:0/2236260698 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f564c011a10 con 0x7f56640b21f0 2020-11-28T21:51:47.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.773+0000 7f565b7fe700 1 -- 172.21.15.162:0/2236260698 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f564c011b70 con 0x7f56640b21f0 2020-11-28T21:51:47.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.773+0000 7f566359e700 1 -- 172.21.15.162:0/2236260698 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56640ae110 con 0x7f56640b21f0 2020-11-28T21:51:47.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.773+0000 7f565b7fe700 1 -- 172.21.15.162:0/2236260698 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f564c01d480 con 0x7f56640b21f0 2020-11-28T21:51:47.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.773+0000 7f566359e700 1 -- 172.21.15.162:0/2236260698 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56640a7ff0 con 0x7f56640b21f0 2020-11-28T21:51:47.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.777+0000 7f566359e700 1 -- 172.21.15.162:0/2236260698 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5664057050 con 0x7f56640b21f0 2020-11-28T21:51:47.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.777+0000 7f565b7fe700 1 -- 172.21.15.162:0/2236260698 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f564c024030 con 0x7f56640b21f0 2020-11-28T21:51:47.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.777+0000 7f565b7fe700 1 --2- 172.21.15.162:0/2236260698 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5648055440 0x7f5648057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:47.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.777+0000 7f565b7fe700 1 -- 172.21.15.162:0/2236260698 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f564c06ece0 con 0x7f56640b21f0 2020-11-28T21:51:47.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.777+0000 7f566259c700 1 --2- 172.21.15.162:0/2236260698 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5648055440 0x7f5648057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:47.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.777+0000 7f566259c700 1 --2- 172.21.15.162:0/2236260698 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5648055440 0x7f5648057900 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f565400b000 tx=0x7f5654006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:47.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.785+0000 7f565b7fe700 1 -- 172.21.15.162:0/2236260698 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f564c044ed0 con 0x7f56640b21f0 2020-11-28T21:51:47.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.925+0000 7f566359e700 1 -- 172.21.15.162:0/2236260698 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f566405d250 con 0x7f56640b21f0 2020-11-28T21:51:47.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.925+0000 7f565b7fe700 1 -- 172.21.15.162:0/2236260698 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v66) v1 ==== 110+0+10 (secure 0 0 0) 0x7f564c022070 con 0x7f56640b21f0 2020-11-28T21:51:47.928 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:51:47.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.929+0000 7f566359e700 1 -- 172.21.15.162:0/2236260698 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5648055440 msgr2=0x7f5648057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:47.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.929+0000 7f566359e700 1 --2- 172.21.15.162:0/2236260698 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5648055440 0x7f5648057900 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f565400b000 tx=0x7f5654006dc0).stop 2020-11-28T21:51:47.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.929+0000 7f566359e700 1 -- 172.21.15.162:0/2236260698 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56640b21f0 msgr2=0x7f5664154fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:47.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.929+0000 7f566359e700 1 --2- 172.21.15.162:0/2236260698 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56640b21f0 0x7f5664154fa0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7f565c001bf0 tx=0x7f564c015000).stop 2020-11-28T21:51:47.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.929+0000 7f566359e700 1 -- 172.21.15.162:0/2236260698 shutdown_connections 2020-11-28T21:51:47.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.929+0000 7f566359e700 1 --2- 172.21.15.162:0/2236260698 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5648055440 0x7f5648057900 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:47.931 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.929+0000 7f566359e700 1 --2- 172.21.15.162:0/2236260698 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f56640ae3d0 0x7f5664066410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:47.931 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.929+0000 7f566359e700 1 --2- 172.21.15.162:0/2236260698 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5664141e80 0x7f5664155490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:47.931 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.929+0000 7f566359e700 1 --2- 172.21.15.162:0/2236260698 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56640b21f0 0x7f5664154fa0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:47.931 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.929+0000 7f566359e700 1 -- 172.21.15.162:0/2236260698 >> 172.21.15.162:0/2236260698 conn(0x7f56640b57d0 msgr2=0x7f56640a9220 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:47.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.929+0000 7f566359e700 1 -- 172.21.15.162:0/2236260698 shutdown_connections 2020-11-28T21:51:47.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:47.929+0000 7f566359e700 1 -- 172.21.15.162:0/2236260698 wait complete. 2020-11-28T21:51:47.941 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:51:48.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.049+0000 7fea563b3700 1 Processor -- start 2020-11-28T21:51:48.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.049+0000 7fea563b3700 1 -- start start 2020-11-28T21:51:48.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.049+0000 7fea563b3700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fea500b1eb0 0x7fea500b2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:48.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.049+0000 7fea563b3700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fea50060a70 0x7fea500afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:48.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.049+0000 7fea563b3700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea50060f00 con 0x7fea500af6a0 2020-11-28T21:51:48.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.049+0000 7fea563b3700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea501471f0 con 0x7fea500b1eb0 2020-11-28T21:51:48.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.049+0000 7fea563b3700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea50060090 con 0x7fea50060a70 2020-11-28T21:51:48.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.049+0000 7fea54bb0700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fea500af6a0 0x7fea50144350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:47666/0 (socket says 172.21.15.162:47666) 2020-11-28T21:51:48.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.049+0000 7fea54bb0700 1 -- 172.21.15.162:0/1728271087 learned_addr learned my addr 172.21.15.162:0/1728271087 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:51:48.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.049+0000 7fea55bb2700 1 --2- 172.21.15.162:0/1728271087 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fea50060a70 0x7fea500afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:48.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.049+0000 7fea553b1700 1 --2- 172.21.15.162:0/1728271087 >> v2:172.21.15.118:3300/0 conn(0x7fea500b1eb0 0x7fea500b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:48.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.049+0000 7fea47fff700 1 -- 172.21.15.162:0/1728271087 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 65529868 0 0) 0x7fea50060f00 con 0x7fea500af6a0 2020-11-28T21:51:48.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.049+0000 7fea47fff700 1 -- 172.21.15.162:0/1728271087 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea34003230 con 0x7fea500af6a0 2020-11-28T21:51:48.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.049+0000 7fea55bb2700 1 -- 172.21.15.162:0/1728271087 >> v2:172.21.15.118:3300/0 conn(0x7fea500b1eb0 msgr2=0x7fea500b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:51:48.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.049+0000 7fea55bb2700 1 --2- 172.21.15.162:0/1728271087 >> v2:172.21.15.118:3300/0 conn(0x7fea500b1eb0 0x7fea500b2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:48.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.049+0000 7fea55bb2700 1 -- 172.21.15.162:0/1728271087 >> v1:172.21.15.162:6789/0 conn(0x7fea500af6a0 legacy=0x7fea50144350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:48.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.049+0000 7fea55bb2700 1 -- 172.21.15.162:0/1728271087 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea50060330 con 0x7fea50060a70 2020-11-28T21:51:48.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.049+0000 7fea55bb2700 1 --2- 172.21.15.162:0/1728271087 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fea50060a70 0x7fea500afdc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fea480032b0 tx=0x7fea480091a0).ready entity=mon.2 client_cookie=9390ccaac0990d57 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:48.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.049+0000 7fea47fff700 1 -- 172.21.15.162:0/1728271087 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea4800b940 con 0x7fea50060a70 2020-11-28T21:51:48.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.053+0000 7fea47fff700 1 -- 172.21.15.162:0/1728271087 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea4800baa0 con 0x7fea50060a70 2020-11-28T21:51:48.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.053+0000 7fea563b3700 1 -- 172.21.15.162:0/1728271087 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fea50060a70 msgr2=0x7fea500afdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:48.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.053+0000 7fea47fff700 1 -- 172.21.15.162:0/1728271087 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea4800bc10 con 0x7fea50060a70 2020-11-28T21:51:48.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.053+0000 7fea563b3700 1 --2- 172.21.15.162:0/1728271087 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fea50060a70 0x7fea500afdc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fea480032b0 tx=0x7fea480091a0).stop 2020-11-28T21:51:48.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.053+0000 7fea563b3700 1 -- 172.21.15.162:0/1728271087 shutdown_connections 2020-11-28T21:51:48.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.053+0000 7fea563b3700 1 --2- 172.21.15.162:0/1728271087 >> v2:172.21.15.118:3300/0 conn(0x7fea500b1eb0 0x7fea500b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:48.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.053+0000 7fea563b3700 1 --2- 172.21.15.162:0/1728271087 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fea50060a70 0x7fea500afdc0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:48.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.053+0000 7fea563b3700 1 -- 172.21.15.162:0/1728271087 >> 172.21.15.162:0/1728271087 conn(0x7fea500b57d0 msgr2=0x7fea500b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:48.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.053+0000 7fea563b3700 1 -- 172.21.15.162:0/1728271087 shutdown_connections 2020-11-28T21:51:48.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.053+0000 7fea563b3700 1 -- 172.21.15.162:0/1728271087 wait complete. 2020-11-28T21:51:48.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.053+0000 7fea563b3700 1 Processor -- start 2020-11-28T21:51:48.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.053+0000 7fea563b3700 1 -- start start 2020-11-28T21:51:48.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.053+0000 7fea563b3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea50060a70 0x7fea50063470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:48.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.053+0000 7fea563b3700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fea500af6a0 0x7fea50148890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:48.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.053+0000 7fea563b3700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fea500b1eb0 0x7fea50148d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:48.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.053+0000 7fea563b3700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea50149270 con 0x7fea500af6a0 2020-11-28T21:51:48.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.053+0000 7fea563b3700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea5015a420 con 0x7fea50060a70 2020-11-28T21:51:48.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.053+0000 7fea563b3700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea501483f0 con 0x7fea500b1eb0 2020-11-28T21:51:48.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.053+0000 7fea553b1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea50060a70 0x7fea50063470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:48.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.053+0000 7fea553b1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea50060a70 0x7fea50063470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40828/0 (socket says 172.21.15.162:40828) 2020-11-28T21:51:48.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.053+0000 7fea553b1700 1 -- 172.21.15.162:0/2097967146 learned_addr learned my addr 172.21.15.162:0/2097967146 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:48.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.053+0000 7fea55bb2700 1 --2- 172.21.15.162:0/2097967146 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fea500b1eb0 0x7fea50148d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:48.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.053+0000 7fea54bb0700 1 --2- 172.21.15.162:0/2097967146 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fea500af6a0 0x7fea50148890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:48.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.057+0000 7fea553b1700 1 -- 172.21.15.162:0/2097967146 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fea500b1eb0 msgr2=0x7fea50148d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:48.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.057+0000 7fea553b1700 1 --2- 172.21.15.162:0/2097967146 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fea500b1eb0 0x7fea50148d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:48.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.057+0000 7fea553b1700 1 -- 172.21.15.162:0/2097967146 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fea500af6a0 msgr2=0x7fea50148890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:48.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.057+0000 7fea553b1700 1 --2- 172.21.15.162:0/2097967146 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fea500af6a0 0x7fea50148890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:48.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.057+0000 7fea553b1700 1 -- 172.21.15.162:0/2097967146 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea5015a150 con 0x7fea50060a70 2020-11-28T21:51:48.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.057+0000 7fea553b1700 1 --2- 172.21.15.162:0/2097967146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea50060a70 0x7fea50063470 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7fea501493b0 tx=0x7fea40008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:48.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.057+0000 7fea467fc700 1 -- 172.21.15.162:0/2097967146 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea4000c820 con 0x7fea50060a70 2020-11-28T21:51:48.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.057+0000 7fea467fc700 1 -- 172.21.15.162:0/2097967146 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea4000c980 con 0x7fea50060a70 2020-11-28T21:51:48.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.057+0000 7fea467fc700 1 -- 172.21.15.162:0/2097967146 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea4000cce0 con 0x7fea50060a70 2020-11-28T21:51:48.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.057+0000 7fea563b3700 1 -- 172.21.15.162:0/2097967146 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea50063b60 con 0x7fea50060a70 2020-11-28T21:51:48.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.057+0000 7fea563b3700 1 -- 172.21.15.162:0/2097967146 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea50067120 con 0x7fea50060a70 2020-11-28T21:51:48.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.057+0000 7fea563b3700 1 -- 172.21.15.162:0/2097967146 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea501485b0 con 0x7fea50060a70 2020-11-28T21:51:48.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.057+0000 7fea467fc700 1 -- 172.21.15.162:0/2097967146 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fea40022070 con 0x7fea50060a70 2020-11-28T21:51:48.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.057+0000 7fea467fc700 1 --2- 172.21.15.162:0/2097967146 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fea38055500 0x7fea380579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:48.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.057+0000 7fea54bb0700 1 --2- 172.21.15.162:0/2097967146 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fea38055500 0x7fea380579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:48.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.057+0000 7fea467fc700 1 -- 172.21.15.162:0/2097967146 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fea4006e180 con 0x7fea50060a70 2020-11-28T21:51:48.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.061+0000 7fea54bb0700 1 --2- 172.21.15.162:0/2097967146 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fea38055500 0x7fea380579c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7fea4c00d000 tx=0x7fea4c009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:48.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.065+0000 7fea467fc700 1 -- 172.21.15.162:0/2097967146 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fea40044480 con 0x7fea50060a70 2020-11-28T21:51:48.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.205+0000 7fea563b3700 1 -- 172.21.15.162:0/2097967146 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fea50057050 con 0x7fea50060a70 2020-11-28T21:51:48.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.205+0000 7fea467fc700 1 -- 172.21.15.162:0/2097967146 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v66) v1 ==== 92+0+10 (secure 0 0 0) 0x7fea40047aa0 con 0x7fea50060a70 2020-11-28T21:51:48.206 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:51:48.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.209+0000 7fea563b3700 1 -- 172.21.15.162:0/2097967146 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fea38055500 msgr2=0x7fea380579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:48.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.209+0000 7fea563b3700 1 --2- 172.21.15.162:0/2097967146 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fea38055500 0x7fea380579c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7fea4c00d000 tx=0x7fea4c009040).stop 2020-11-28T21:51:48.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.209+0000 7fea563b3700 1 -- 172.21.15.162:0/2097967146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea50060a70 msgr2=0x7fea50063470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:48.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.209+0000 7fea563b3700 1 --2- 172.21.15.162:0/2097967146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea50060a70 0x7fea50063470 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7fea501493b0 tx=0x7fea40008040).stop 2020-11-28T21:51:48.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.209+0000 7fea563b3700 1 -- 172.21.15.162:0/2097967146 shutdown_connections 2020-11-28T21:51:48.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.209+0000 7fea563b3700 1 --2- 172.21.15.162:0/2097967146 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fea38055500 0x7fea380579c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:48.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.209+0000 7fea563b3700 1 --2- 172.21.15.162:0/2097967146 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fea500b1eb0 0x7fea50148d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:48.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.209+0000 7fea563b3700 1 --2- 172.21.15.162:0/2097967146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea50060a70 0x7fea50063470 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:48.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.209+0000 7fea563b3700 1 --2- 172.21.15.162:0/2097967146 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fea500af6a0 0x7fea50148890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:48.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.209+0000 7fea563b3700 1 -- 172.21.15.162:0/2097967146 >> 172.21.15.162:0/2097967146 conn(0x7fea500b57d0 msgr2=0x7fea500b2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:48.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.209+0000 7fea563b3700 1 -- 172.21.15.162:0/2097967146 shutdown_connections 2020-11-28T21:51:48.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.209+0000 7fea563b3700 1 -- 172.21.15.162:0/2097967146 wait complete. 2020-11-28T21:51:48.220 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-28T21:51:48.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.341+0000 7fc955ad9700 1 Processor -- start 2020-11-28T21:51:48.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.341+0000 7fc955ad9700 1 -- start start 2020-11-28T21:51:48.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.341+0000 7fc955ad9700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc95013cf30 0x7fc9500ac350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:48.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.341+0000 7fc955ad9700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9500b1360 con 0x7fc95005cca0 2020-11-28T21:51:48.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.341+0000 7fc955ad9700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9500b25c0 con 0x7fc95013cb50 2020-11-28T21:51:48.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.341+0000 7fc955ad9700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9500add60 con 0x7fc95013cf30 2020-11-28T21:51:48.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.341+0000 7fc9552d8700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc95013cf30 0x7fc9500ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:48.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.341+0000 7fc9552d8700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc95013cf30 0x7fc9500ac350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40840/0 (socket says 172.21.15.162:40840) 2020-11-28T21:51:48.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.341+0000 7fc954ad7700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fc95013cb50 0x7fc950140180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:47682/0 (socket says 172.21.15.162:47682) 2020-11-28T21:51:48.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.341+0000 7fc9552d8700 1 -- 172.21.15.162:0/3732363310 learned_addr learned my addr 172.21.15.162:0/3732363310 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:48.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.341+0000 7fc9552d8700 1 -- 172.21.15.162:0/3732363310 >> v1:172.21.15.162:6789/0 conn(0x7fc95013cb50 legacy=0x7fc950140180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:48.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.341+0000 7fc9552d8700 1 -- 172.21.15.162:0/3732363310 >> v1:172.21.15.118:6789/0 conn(0x7fc95005cca0 legacy=0x7fc9500b1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:51:48.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.341+0000 7fc9552d8700 1 -- 172.21.15.162:0/3732363310 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9500ae8c0 con 0x7fc95013cf30 2020-11-28T21:51:48.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.341+0000 7fc9552d8700 1 --2- 172.21.15.162:0/3732363310 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc95013cf30 0x7fc9500ac350 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7fc95005ad70 tx=0x7fc948009270).ready entity=mon.1 client_cookie=673ec52e770d7d17 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:48.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.341+0000 7fc94f7fe700 1 -- 172.21.15.162:0/3732363310 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc94800bbc0 con 0x7fc95013cf30 2020-11-28T21:51:48.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.341+0000 7fc94f7fe700 1 -- 172.21.15.162:0/3732363310 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94800bd20 con 0x7fc95013cf30 2020-11-28T21:51:48.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.341+0000 7fc94f7fe700 1 -- 172.21.15.162:0/3732363310 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc94800be90 con 0x7fc95013cf30 2020-11-28T21:51:48.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.341+0000 7fc955ad9700 1 -- 172.21.15.162:0/3732363310 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc95013cf30 msgr2=0x7fc9500ac350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:48.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.341+0000 7fc955ad9700 1 --2- 172.21.15.162:0/3732363310 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc95013cf30 0x7fc9500ac350 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7fc95005ad70 tx=0x7fc948009270).stop 2020-11-28T21:51:48.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.345+0000 7fc955ad9700 1 -- 172.21.15.162:0/3732363310 shutdown_connections 2020-11-28T21:51:48.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.345+0000 7fc955ad9700 1 --2- 172.21.15.162:0/3732363310 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc95013cf30 0x7fc9500ac350 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:48.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.345+0000 7fc955ad9700 1 -- 172.21.15.162:0/3732363310 >> 172.21.15.162:0/3732363310 conn(0x7fc9500b57d0 msgr2=0x7fc9500b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:48.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.345+0000 7fc955ad9700 1 -- 172.21.15.162:0/3732363310 shutdown_connections 2020-11-28T21:51:48.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.345+0000 7fc955ad9700 1 -- 172.21.15.162:0/3732363310 wait complete. 2020-11-28T21:51:48.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.345+0000 7fc955ad9700 1 Processor -- start 2020-11-28T21:51:48.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.345+0000 7fc955ad9700 1 -- start start 2020-11-28T21:51:48.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.345+0000 7fc955ad9700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc95005cca0 0x7fc9500ab9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:48.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.345+0000 7fc955ad9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc95013cb50 0x7fc950063620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:48.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.345+0000 7fc955ad9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc95013cf30 0x7fc950063b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:48.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.345+0000 7fc955ad9700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc950143a80 con 0x7fc95013cf30 2020-11-28T21:51:48.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.345+0000 7fc955ad9700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc950066a20 con 0x7fc95013cb50 2020-11-28T21:51:48.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.345+0000 7fc955ad9700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9500667c0 con 0x7fc95005cca0 2020-11-28T21:51:48.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.345+0000 7fc94ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc95013cb50 0x7fc950063620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:48.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.345+0000 7fc94ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc95013cb50 0x7fc950063620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40844/0 (socket says 172.21.15.162:40844) 2020-11-28T21:51:48.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.345+0000 7fc94ffff700 1 -- 172.21.15.162:0/1863669530 learned_addr learned my addr 172.21.15.162:0/1863669530 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:48.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.345+0000 7fc954ad7700 1 --2- 172.21.15.162:0/1863669530 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc95005cca0 0x7fc9500ab9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:48.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.345+0000 7fc94ffff700 1 -- 172.21.15.162:0/1863669530 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc95005cca0 msgr2=0x7fc9500ab9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:48.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.345+0000 7fc94ffff700 1 --2- 172.21.15.162:0/1863669530 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc95005cca0 0x7fc9500ab9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:48.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.345+0000 7fc94ffff700 1 -- 172.21.15.162:0/1863669530 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc95013cf30 msgr2=0x7fc950063b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:48.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.345+0000 7fc9552d8700 1 --2- 172.21.15.162:0/1863669530 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc95013cf30 0x7fc950063b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:48.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.345+0000 7fc94ffff700 1 --2- 172.21.15.162:0/1863669530 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc95013cf30 0x7fc950063b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:48.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.345+0000 7fc94ffff700 1 -- 172.21.15.162:0/1863669530 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc950143340 con 0x7fc95013cb50 2020-11-28T21:51:48.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.345+0000 7fc9552d8700 1 --2- 172.21.15.162:0/1863669530 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc95013cf30 0x7fc950063b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:51:48.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.345+0000 7fc94ffff700 1 --2- 172.21.15.162:0/1863669530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc95013cb50 0x7fc950063620 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7fc950143260 tx=0x7fc944007c00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:48.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.345+0000 7fc94dffb700 1 -- 172.21.15.162:0/1863669530 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc94400e4a0 con 0x7fc95013cb50 2020-11-28T21:51:48.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.345+0000 7fc94dffb700 1 -- 172.21.15.162:0/1863669530 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94400e600 con 0x7fc95013cb50 2020-11-28T21:51:48.355 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.345+0000 7fc955ad9700 1 -- 172.21.15.162:0/1863669530 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9501435e0 con 0x7fc95013cb50 2020-11-28T21:51:48.355 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.345+0000 7fc94dffb700 1 -- 172.21.15.162:0/1863669530 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc94400e960 con 0x7fc95013cb50 2020-11-28T21:51:48.355 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.345+0000 7fc955ad9700 1 -- 172.21.15.162:0/1863669530 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc950064000 con 0x7fc95013cb50 2020-11-28T21:51:48.356 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.349+0000 7fc955ad9700 1 -- 172.21.15.162:0/1863669530 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc950143770 con 0x7fc95013cb50 2020-11-28T21:51:48.356 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.349+0000 7fc94dffb700 1 -- 172.21.15.162:0/1863669530 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc944019030 con 0x7fc95013cb50 2020-11-28T21:51:48.356 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.349+0000 7fc94dffb700 1 --2- 172.21.15.162:0/1863669530 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc9300554e0 0x7fc9300579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:48.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.349+0000 7fc94dffb700 1 -- 172.21.15.162:0/1863669530 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc94406d460 con 0x7fc95013cb50 2020-11-28T21:51:48.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.349+0000 7fc954ad7700 1 --2- 172.21.15.162:0/1863669530 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc9300554e0 0x7fc9300579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:48.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.349+0000 7fc954ad7700 1 --2- 172.21.15.162:0/1863669530 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc9300554e0 0x7fc9300579a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7fc940008000 tx=0x7fc940009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:48.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.353+0000 7fc94dffb700 1 -- 172.21.15.162:0/1863669530 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc944043760 con 0x7fc95013cb50 2020-11-28T21:51:48.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.493+0000 7fc955ad9700 1 -- 172.21.15.162:0/1863669530 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc950057050 con 0x7fc95013cb50 2020-11-28T21:51:48.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.493+0000 7fc94dffb700 1 -- 172.21.15.162:0/1863669530 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v66) v1 ==== 104+0+10 (secure 0 0 0) 0x7fc944046d80 con 0x7fc95013cb50 2020-11-28T21:51:48.494 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:51:48.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.497+0000 7fc955ad9700 1 -- 172.21.15.162:0/1863669530 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc9300554e0 msgr2=0x7fc9300579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:48.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.497+0000 7fc955ad9700 1 --2- 172.21.15.162:0/1863669530 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc9300554e0 0x7fc9300579a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7fc940008000 tx=0x7fc940009040).stop 2020-11-28T21:51:48.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.497+0000 7fc955ad9700 1 -- 172.21.15.162:0/1863669530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc95013cb50 msgr2=0x7fc950063620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:48.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.497+0000 7fc955ad9700 1 --2- 172.21.15.162:0/1863669530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc95013cb50 0x7fc950063620 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7fc950143260 tx=0x7fc944007c00).stop 2020-11-28T21:51:48.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.497+0000 7fc955ad9700 1 -- 172.21.15.162:0/1863669530 shutdown_connections 2020-11-28T21:51:48.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.497+0000 7fc955ad9700 1 --2- 172.21.15.162:0/1863669530 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc9300554e0 0x7fc9300579a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:48.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.497+0000 7fc955ad9700 1 --2- 172.21.15.162:0/1863669530 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc95005cca0 0x7fc9500ab9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:48.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.497+0000 7fc955ad9700 1 --2- 172.21.15.162:0/1863669530 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc95013cf30 0x7fc950063b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:48.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.497+0000 7fc955ad9700 1 --2- 172.21.15.162:0/1863669530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc95013cb50 0x7fc950063620 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:48.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.497+0000 7fc955ad9700 1 -- 172.21.15.162:0/1863669530 >> 172.21.15.162:0/1863669530 conn(0x7fc9500b57d0 msgr2=0x7fc9500b4390 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:48.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.497+0000 7fc955ad9700 1 -- 172.21.15.162:0/1863669530 shutdown_connections 2020-11-28T21:51:48.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.497+0000 7fc955ad9700 1 -- 172.21.15.162:0/1863669530 wait complete. 2020-11-28T21:51:48.509 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-28T21:51:48.622 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.621+0000 7f7dfd082700 1 Processor -- start 2020-11-28T21:51:48.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.621+0000 7f7dfd082700 1 -- start start 2020-11-28T21:51:48.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.621+0000 7f7dfd082700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7df80b1eb0 0x7f7df80b2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:48.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.621+0000 7f7dfd082700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f7df8060a70 0x7f7df80afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:48.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.621+0000 7f7dfd082700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7df8060f00 con 0x7f7df80af6a0 2020-11-28T21:51:48.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.621+0000 7f7dfd082700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7df81471f0 con 0x7f7df8060a70 2020-11-28T21:51:48.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.621+0000 7f7dfd082700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7df8060090 con 0x7f7df80b1eb0 2020-11-28T21:51:48.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.621+0000 7f7df77fe700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f7df80af6a0 0x7f7df8144350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:47694/0 (socket says 172.21.15.162:47694) 2020-11-28T21:51:48.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.621+0000 7f7df77fe700 1 -- 172.21.15.162:0/2072568180 learned_addr learned my addr 172.21.15.162:0/2072568180 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:51:48.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.625+0000 7f7df7fff700 1 --2- 172.21.15.162:0/2072568180 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7df80b1eb0 0x7f7df80b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:48.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.625+0000 7f7dfc881700 1 --2- 172.21.15.162:0/2072568180 >> v2:172.21.15.118:3300/0 conn(0x7f7df8060a70 0x7f7df80afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:48.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.625+0000 7f7df6ffd700 1 -- 172.21.15.162:0/2072568180 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 209289013 0 0) 0x7f7df8060f00 con 0x7f7df80af6a0 2020-11-28T21:51:48.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.625+0000 7f7df6ffd700 1 -- 172.21.15.162:0/2072568180 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7dd8003230 con 0x7f7df80af6a0 2020-11-28T21:51:48.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.625+0000 7f7df7fff700 1 -- 172.21.15.162:0/2072568180 >> v2:172.21.15.118:3300/0 conn(0x7f7df8060a70 msgr2=0x7f7df80afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:51:48.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.625+0000 7f7df7fff700 1 --2- 172.21.15.162:0/2072568180 >> v2:172.21.15.118:3300/0 conn(0x7f7df8060a70 0x7f7df80afdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:48.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.625+0000 7f7df7fff700 1 -- 172.21.15.162:0/2072568180 >> v1:172.21.15.162:6789/0 conn(0x7f7df80af6a0 legacy=0x7f7df8144350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:48.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.625+0000 7f7df7fff700 1 -- 172.21.15.162:0/2072568180 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df8060330 con 0x7f7df80b1eb0 2020-11-28T21:51:48.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.625+0000 7f7df7fff700 1 --2- 172.21.15.162:0/2072568180 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7df80b1eb0 0x7f7df80b2290 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f7de0002f30 tx=0x7f7de0006dc0).ready entity=mon.2 client_cookie=89521d073699516c server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:48.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.625+0000 7f7df6ffd700 1 -- 172.21.15.162:0/2072568180 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7de00096a0 con 0x7f7df80b1eb0 2020-11-28T21:51:48.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.625+0000 7f7df6ffd700 1 -- 172.21.15.162:0/2072568180 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7de0009800 con 0x7f7df80b1eb0 2020-11-28T21:51:48.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.625+0000 7f7df6ffd700 1 -- 172.21.15.162:0/2072568180 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7de0009b60 con 0x7f7df80b1eb0 2020-11-28T21:51:48.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.625+0000 7f7dfd082700 1 -- 172.21.15.162:0/2072568180 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7df80b1eb0 msgr2=0x7f7df80b2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:48.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.625+0000 7f7dfd082700 1 --2- 172.21.15.162:0/2072568180 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7df80b1eb0 0x7f7df80b2290 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f7de0002f30 tx=0x7f7de0006dc0).stop 2020-11-28T21:51:48.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.625+0000 7f7dfd082700 1 -- 172.21.15.162:0/2072568180 shutdown_connections 2020-11-28T21:51:48.639 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.625+0000 7f7dfd082700 1 --2- 172.21.15.162:0/2072568180 >> v2:172.21.15.118:3300/0 conn(0x7f7df8060a70 0x7f7df80afdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:48.639 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.625+0000 7f7dfd082700 1 --2- 172.21.15.162:0/2072568180 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7df80b1eb0 0x7f7df80b2290 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:48.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.625+0000 7f7dfd082700 1 -- 172.21.15.162:0/2072568180 >> 172.21.15.162:0/2072568180 conn(0x7f7df80b57d0 msgr2=0x7f7df80b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:48.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.625+0000 7f7dfd082700 1 -- 172.21.15.162:0/2072568180 shutdown_connections 2020-11-28T21:51:48.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.625+0000 7f7dfd082700 1 -- 172.21.15.162:0/2072568180 wait complete. 2020-11-28T21:51:48.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.625+0000 7f7dfd082700 1 Processor -- start 2020-11-28T21:51:48.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.625+0000 7f7dfd082700 1 -- start start 2020-11-28T21:51:48.642 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.629+0000 7f7dfd082700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7df8060a70 0x7f7df81598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:48.642 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.629+0000 7f7dfd082700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7df80af6a0 0x7f7df8063be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:48.643 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.629+0000 7f7dfd082700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7df80b1eb0 0x7f7df80abcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:48.643 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.629+0000 7f7dfd082700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df8067510 con 0x7f7df80af6a0 2020-11-28T21:51:48.643 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.629+0000 7f7dfd082700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df8066fd0 con 0x7f7df80b1eb0 2020-11-28T21:51:48.644 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.629+0000 7f7dfd082700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7df8067270 con 0x7f7df8060a70 2020-11-28T21:51:48.644 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.629+0000 7f7dfc881700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7df80b1eb0 0x7f7df80abcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:48.645 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.629+0000 7f7dfc881700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7df80b1eb0 0x7f7df80abcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40858/0 (socket says 172.21.15.162:40858) 2020-11-28T21:51:48.645 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.629+0000 7f7dfc881700 1 -- 172.21.15.162:0/2380303253 learned_addr learned my addr 172.21.15.162:0/2380303253 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:48.645 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.629+0000 7f7df7fff700 1 --2- 172.21.15.162:0/2380303253 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7df8060a70 0x7f7df81598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:48.646 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.629+0000 7f7df77fe700 1 --2- 172.21.15.162:0/2380303253 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7df80af6a0 0x7f7df8063be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:48.646 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.629+0000 7f7dfc881700 1 -- 172.21.15.162:0/2380303253 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7df8060a70 msgr2=0x7f7df81598a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:48.647 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.629+0000 7f7dfc881700 1 --2- 172.21.15.162:0/2380303253 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7df8060a70 0x7f7df81598a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:48.647 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.629+0000 7f7dfc881700 1 -- 172.21.15.162:0/2380303253 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7df80af6a0 msgr2=0x7f7df8063be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:48.647 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.629+0000 7f7dfc881700 1 --2- 172.21.15.162:0/2380303253 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7df80af6a0 0x7f7df8063be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:48.647 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.629+0000 7f7dfc881700 1 -- 172.21.15.162:0/2380303253 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df8159ee0 con 0x7f7df80b1eb0 2020-11-28T21:51:48.648 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.629+0000 7f7dfc881700 1 --2- 172.21.15.162:0/2380303253 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7df80b1eb0 0x7f7df80abcb0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7f7df8066510 tx=0x7f7df00091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:48.648 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.629+0000 7f7df57fa700 1 -- 172.21.15.162:0/2380303253 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7df000ee60 con 0x7f7df80b1eb0 2020-11-28T21:51:48.648 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.629+0000 7f7df57fa700 1 -- 172.21.15.162:0/2380303253 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7df000cbb0 con 0x7f7df80b1eb0 2020-11-28T21:51:48.648 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.629+0000 7f7df57fa700 1 -- 172.21.15.162:0/2380303253 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7df00123f0 con 0x7f7df80b1eb0 2020-11-28T21:51:48.649 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.629+0000 7f7dfd082700 1 -- 172.21.15.162:0/2380303253 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7df815a180 con 0x7f7df80b1eb0 2020-11-28T21:51:48.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.629+0000 7f7dfd082700 1 -- 172.21.15.162:0/2380303253 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7df80ab080 con 0x7f7df80b1eb0 2020-11-28T21:51:48.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.629+0000 7f7dfd082700 1 -- 172.21.15.162:0/2380303253 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7df8057050 con 0x7f7df80b1eb0 2020-11-28T21:51:48.652 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.629+0000 7f7df57fa700 1 -- 172.21.15.162:0/2380303253 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7df0022070 con 0x7f7df80b1eb0 2020-11-28T21:51:48.652 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.629+0000 7f7df57fa700 1 --2- 172.21.15.162:0/2380303253 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7ddc0554e0 0x7f7ddc0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:48.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.629+0000 7f7df57fa700 1 -- 172.21.15.162:0/2380303253 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7df006e620 con 0x7f7df80b1eb0 2020-11-28T21:51:48.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.629+0000 7f7df7fff700 1 --2- 172.21.15.162:0/2380303253 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7ddc0554e0 0x7f7ddc0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:48.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.633+0000 7f7df7fff700 1 --2- 172.21.15.162:0/2380303253 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7ddc0554e0 0x7f7ddc0579a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f7de001b000 tx=0x7f7de0015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:48.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.637+0000 7f7df57fa700 1 -- 172.21.15.162:0/2380303253 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7df0044920 con 0x7f7df80b1eb0 2020-11-28T21:51:48.782 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.781+0000 7f7dfd082700 1 -- 172.21.15.162:0/2380303253 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7df80a89f0 con 0x7f7df80b1eb0 2020-11-28T21:51:48.782 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.781+0000 7f7df57fa700 1 -- 172.21.15.162:0/2380303253 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v66) v1 ==== 100+0+10 (secure 0 0 0) 0x7f7df0016070 con 0x7f7df80b1eb0 2020-11-28T21:51:48.783 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:51:48.784 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.785+0000 7f7dfd082700 1 -- 172.21.15.162:0/2380303253 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7ddc0554e0 msgr2=0x7f7ddc0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:48.785 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.785+0000 7f7dfd082700 1 --2- 172.21.15.162:0/2380303253 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7ddc0554e0 0x7f7ddc0579a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f7de001b000 tx=0x7f7de0015000).stop 2020-11-28T21:51:48.785 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.785+0000 7f7dfd082700 1 -- 172.21.15.162:0/2380303253 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7df80b1eb0 msgr2=0x7f7df80abcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:48.785 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.785+0000 7f7dfd082700 1 --2- 172.21.15.162:0/2380303253 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7df80b1eb0 0x7f7df80abcb0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7f7df8066510 tx=0x7f7df00091a0).stop 2020-11-28T21:51:48.785 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.785+0000 7f7dfd082700 1 -- 172.21.15.162:0/2380303253 shutdown_connections 2020-11-28T21:51:48.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.785+0000 7f7dfd082700 1 --2- 172.21.15.162:0/2380303253 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7ddc0554e0 0x7f7ddc0579a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:48.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.785+0000 7f7dfd082700 1 --2- 172.21.15.162:0/2380303253 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7df8060a70 0x7f7df81598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:48.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.785+0000 7f7dfd082700 1 --2- 172.21.15.162:0/2380303253 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7df80b1eb0 0x7f7df80abcb0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:48.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.785+0000 7f7dfd082700 1 --2- 172.21.15.162:0/2380303253 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7df80af6a0 0x7f7df8063be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:48.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.785+0000 7f7dfd082700 1 -- 172.21.15.162:0/2380303253 >> 172.21.15.162:0/2380303253 conn(0x7f7df80b57d0 msgr2=0x7f7df8146870 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:48.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.785+0000 7f7dfd082700 1 -- 172.21.15.162:0/2380303253 shutdown_connections 2020-11-28T21:51:48.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.785+0000 7f7dfd082700 1 -- 172.21.15.162:0/2380303253 wait complete. 2020-11-28T21:51:48.797 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-28T21:51:48.797 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:51:48.910 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.909+0000 7fd968853700 1 Processor -- start 2020-11-28T21:51:48.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.909+0000 7fd968853700 1 -- start start 2020-11-28T21:51:48.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.909+0000 7fd968853700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd964066a10 0x7fd964061760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:48.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.909+0000 7fd968853700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd9640b1500 0x7fd964061c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:48.914 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.909+0000 7fd968853700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd964066df0 con 0x7fd964143da0 2020-11-28T21:51:48.914 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.909+0000 7fd968853700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd964060720 con 0x7fd9640b1500 2020-11-28T21:51:48.914 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.909+0000 7fd968853700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9640b12b0 con 0x7fd964066a10 2020-11-28T21:51:48.914 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.909+0000 7fd963fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd9640b1500 0x7fd964061c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:48.915 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.909+0000 7fd963fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd9640b1500 0x7fd964061c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40866/0 (socket says 172.21.15.162:40866) 2020-11-28T21:51:48.915 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.913+0000 7fd963fff700 1 -- 172.21.15.162:0/359126384 learned_addr learned my addr 172.21.15.162:0/359126384 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:48.915 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.913+0000 7fd9637fe700 1 --2- 172.21.15.162:0/359126384 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd964066a10 0x7fd964061760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:48.916 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.913+0000 7fd963fff700 1 -- 172.21.15.162:0/359126384 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd964066a10 msgr2=0x7fd964061760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:51:48.916 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.913+0000 7fd963fff700 1 --2- 172.21.15.162:0/359126384 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd964066a10 0x7fd964061760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:48.916 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.913+0000 7fd963fff700 1 -- 172.21.15.162:0/359126384 >> v1:172.21.15.118:6789/0 conn(0x7fd964143da0 legacy=0x7fd96405b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:48.916 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.913+0000 7fd963fff700 1 -- 172.21.15.162:0/359126384 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9640b0d70 con 0x7fd9640b1500 2020-11-28T21:51:48.916 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.913+0000 7fd9637fe700 1 --2- 172.21.15.162:0/359126384 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd964066a10 0x7fd964061760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:51:48.917 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.913+0000 7fd963fff700 1 --2- 172.21.15.162:0/359126384 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9640b1500 0x7fd964061c50 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7fd95c0032b0 tx=0x7fd95c009270).ready entity=mon.1 client_cookie=ffa5032056506142 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:48.917 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.913+0000 7fd9627fc700 1 -- 172.21.15.162:0/359126384 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd95c00bc00 con 0x7fd9640b1500 2020-11-28T21:51:48.917 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.913+0000 7fd9627fc700 1 -- 172.21.15.162:0/359126384 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd95c00bd60 con 0x7fd9640b1500 2020-11-28T21:51:48.917 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.913+0000 7fd968853700 1 -- 172.21.15.162:0/359126384 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9640b1500 msgr2=0x7fd964061c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:48.918 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.913+0000 7fd968853700 1 --2- 172.21.15.162:0/359126384 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9640b1500 0x7fd964061c50 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7fd95c0032b0 tx=0x7fd95c009270).stop 2020-11-28T21:51:48.918 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.913+0000 7fd9627fc700 1 -- 172.21.15.162:0/359126384 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd95c011400 con 0x7fd9640b1500 2020-11-28T21:51:48.918 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.913+0000 7fd968853700 1 -- 172.21.15.162:0/359126384 shutdown_connections 2020-11-28T21:51:48.918 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.913+0000 7fd968853700 1 --2- 172.21.15.162:0/359126384 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9640b1500 0x7fd964061c50 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:48.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.913+0000 7fd968853700 1 --2- 172.21.15.162:0/359126384 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd964066a10 0x7fd964061760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:48.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.913+0000 7fd968853700 1 -- 172.21.15.162:0/359126384 >> 172.21.15.162:0/359126384 conn(0x7fd9640b5da0 msgr2=0x7fd9640b6180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:48.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.913+0000 7fd968853700 1 -- 172.21.15.162:0/359126384 shutdown_connections 2020-11-28T21:51:48.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.913+0000 7fd968853700 1 -- 172.21.15.162:0/359126384 wait complete. 2020-11-28T21:51:48.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.913+0000 7fd968853700 1 Processor -- start 2020-11-28T21:51:48.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.913+0000 7fd968853700 1 -- start start 2020-11-28T21:51:48.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.913+0000 7fd968853700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd964066a10 0x7fd964133e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:48.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.913+0000 7fd968853700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9640b1500 0x7fd964141da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:48.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.913+0000 7fd968853700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd964143da0 0x7fd964142290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:48.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.913+0000 7fd968853700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd964065360 con 0x7fd9640b1500 2020-11-28T21:51:48.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.913+0000 7fd968853700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9640b5480 con 0x7fd964066a10 2020-11-28T21:51:48.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.913+0000 7fd968853700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd964066050 con 0x7fd964143da0 2020-11-28T21:51:48.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.913+0000 7fd9637fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd964066a10 0x7fd964133e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:48.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.913+0000 7fd9637fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd964066a10 0x7fd964133e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40870/0 (socket says 172.21.15.162:40870) 2020-11-28T21:51:48.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.913+0000 7fd9637fe700 1 -- 172.21.15.162:0/3296396003 learned_addr learned my addr 172.21.15.162:0/3296396003 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:51:48.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.913+0000 7fd962ffd700 1 --2- 172.21.15.162:0/3296396003 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9640b1500 0x7fd964141da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:48.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.913+0000 7fd963fff700 1 --2- 172.21.15.162:0/3296396003 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd964143da0 0x7fd964142290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:48.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.917+0000 7fd9637fe700 1 -- 172.21.15.162:0/3296396003 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd964143da0 msgr2=0x7fd964142290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:48.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.917+0000 7fd9637fe700 1 --2- 172.21.15.162:0/3296396003 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd964143da0 0x7fd964142290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:48.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.917+0000 7fd9637fe700 1 -- 172.21.15.162:0/3296396003 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9640b1500 msgr2=0x7fd964141da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:48.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.917+0000 7fd9637fe700 1 --2- 172.21.15.162:0/3296396003 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9640b1500 0x7fd964141da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:48.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.917+0000 7fd9637fe700 1 -- 172.21.15.162:0/3296396003 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9640650c0 con 0x7fd964066a10 2020-11-28T21:51:48.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.917+0000 7fd9637fe700 1 --2- 172.21.15.162:0/3296396003 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd964066a10 0x7fd964133e30 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7fd954007000 tx=0x7fd954008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:48.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.917+0000 7fd960ff9700 1 -- 172.21.15.162:0/3296396003 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd954010580 con 0x7fd964066a10 2020-11-28T21:51:48.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.917+0000 7fd960ff9700 1 -- 172.21.15.162:0/3296396003 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9540106e0 con 0x7fd964066a10 2020-11-28T21:51:48.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.917+0000 7fd968853700 1 -- 172.21.15.162:0/3296396003 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9640b4de0 con 0x7fd964066a10 2020-11-28T21:51:48.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.917+0000 7fd960ff9700 1 -- 172.21.15.162:0/3296396003 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd954010a40 con 0x7fd964066a10 2020-11-28T21:51:48.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.917+0000 7fd968853700 1 -- 172.21.15.162:0/3296396003 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd964143a70 con 0x7fd964066a10 2020-11-28T21:51:48.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.917+0000 7fd968853700 1 -- 172.21.15.162:0/3296396003 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd964065e90 con 0x7fd964066a10 2020-11-28T21:51:48.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.917+0000 7fd960ff9700 1 -- 172.21.15.162:0/3296396003 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd954010ba0 con 0x7fd964066a10 2020-11-28T21:51:48.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.917+0000 7fd960ff9700 1 --2- 172.21.15.162:0/3296396003 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd944055470 0x7fd944057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:51:48.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.917+0000 7fd960ff9700 1 -- 172.21.15.162:0/3296396003 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd95406df80 con 0x7fd964066a10 2020-11-28T21:51:48.931 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.917+0000 7fd962ffd700 1 --2- 172.21.15.162:0/3296396003 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd944055470 0x7fd944057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:51:48.931 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.921+0000 7fd962ffd700 1 --2- 172.21.15.162:0/3296396003 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd944055470 0x7fd944057930 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7fd95800d000 tx=0x7fd958007b40).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:51:48.931 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:48.925+0000 7fd960ff9700 1 -- 172.21.15.162:0/3296396003 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd954044220 con 0x7fd964066a10 2020-11-28T21:51:49.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:49.117+0000 7fd968853700 1 -- 172.21.15.162:0/3296396003 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd964057050 con 0x7fd964066a10 2020-11-28T21:51:49.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:49.117+0000 7fd960ff9700 1 -- 172.21.15.162:0/3296396003 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+2502 (secure 0 0 0) 0x7fd954047840 con 0x7fd964066a10 2020-11-28T21:51:49.124 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:51:49.125 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":27,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5342,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6837/3666321033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3666321033},{"type":"v1","addr":"172.21.15.162:6837","nonce":3666321033}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":25},{"gid":5769,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/4179306206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":4179306206},{"type":"v1","addr":"172.21.15.118:6833","nonce":4179306206}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":24}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:51:44.226383+0000","modified":"2020-11-28T21:51:45.230212+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5869},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5869":{"gid":5869,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6835/1433205970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1433205970},{"type":"v1","addr":"172.21.15.162:6835","nonce":1433205970}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-11-28T21:51:49.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:49.121+0000 7fd968853700 1 -- 172.21.15.162:0/3296396003 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd944055470 msgr2=0x7fd944057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:49.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:49.121+0000 7fd968853700 1 --2- 172.21.15.162:0/3296396003 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd944055470 0x7fd944057930 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7fd95800d000 tx=0x7fd958007b40).stop 2020-11-28T21:51:49.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:49.121+0000 7fd968853700 1 -- 172.21.15.162:0/3296396003 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd964066a10 msgr2=0x7fd964133e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:51:49.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:49.121+0000 7fd968853700 1 --2- 172.21.15.162:0/3296396003 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd964066a10 0x7fd964133e30 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7fd954007000 tx=0x7fd954008040).stop 2020-11-28T21:51:49.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:49.121+0000 7fd968853700 1 -- 172.21.15.162:0/3296396003 shutdown_connections 2020-11-28T21:51:49.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:49.121+0000 7fd968853700 1 --2- 172.21.15.162:0/3296396003 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd944055470 0x7fd944057930 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:49.129 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:49.121+0000 7fd968853700 1 --2- 172.21.15.162:0/3296396003 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd964143da0 0x7fd964142290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:49.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:49.121+0000 7fd968853700 1 --2- 172.21.15.162:0/3296396003 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd964066a10 0x7fd964133e30 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:49.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:49.121+0000 7fd968853700 1 --2- 172.21.15.162:0/3296396003 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9640b1500 0x7fd964141da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:51:49.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:49.121+0000 7fd968853700 1 -- 172.21.15.162:0/3296396003 >> 172.21.15.162:0/3296396003 conn(0x7fd9640b5da0 msgr2=0x7fd96405fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:51:49.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:49.121+0000 7fd968853700 1 -- 172.21.15.162:0/3296396003 shutdown_connections 2020-11-28T21:51:49.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:51:49.121+0000 7fd968853700 1 -- 172.21.15.162:0/3296396003 wait complete. 2020-11-28T21:51:49.135 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 27 2020-11-28T21:51:49.445 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-28T21:51:44.226383+0000', 'modified': '2020-11-28T21:51:45.230212+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5869}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5869': {'gid': 5869, 'name': 'a', 'rank': 0, 'incarnation': 26, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6835/1433205970', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6834', 'nonce': 1433205970}, {'type': 'v1', 'addr': '172.21.15.162:6835', 'nonce': 1433205970}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-28T21:51:49.446 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-28T21:51:49.447 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-28T21:51:49.575 INFO:teuthology.orchestra.run.smithi162.stdout:{"cluster_fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","whoami":0,"id":5869,"want_state":"up:active","state":"up:active","rank_uptime":5.3394888869999999,"mdsmap_epoch":27,"osdmap_epoch":66,"osdmap_epoch_barrier":66,"uptime":10.573309264000001} 2020-11-28T21:51:49.576 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", "id": 5869, "mdsmap_epoch": 27, "osdmap_epoch": 66, "osdmap_epoch_barrier": 66, "rank_uptime": 5.339488887, "state": "up:active", "uptime": 10.573309264, "want_state": "up:active", "whoami": 0 } 2020-11-28T21:51:49.577 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-11-28T21:51:49.578 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-11-28T21:51:49.579 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi192.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-11-28T21:51:49.579 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:51:49.580 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:51:49.590 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:51:49.646 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:51:49.647 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:51:49.702 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:51:49.703 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:51:49.705 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:51:49.705 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:51:49.754 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-11-28T21:51:49.755 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:51:49.797 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:51:49.813 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:51:49.814 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:51:49.870 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:51:49.871 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:51:49.872 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:51:49.872 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:51:49.873 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:2020-11-28T21:51:49.877+0000 7f4b091490c0 -1 init, newargv = 0x563773e4fab0 newargc=9ceph-fuse[ 2020-11-28T21:51:49.873 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:11950]: starting ceph client 2020-11-28T21:51:49.883 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:ceph-fuse[11950]: starting fuse 2020-11-28T21:51:50.978 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:51:50.994 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:51:50.995 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:51:51.054 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:51:51.055 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:51:51.055 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:51:51.056 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:51:51.102 INFO:teuthology.orchestra.run.smithi192.stdout:49 2020-11-28T21:51:51.102 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2020-11-28T21:51:51.103 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:51:51.103 INFO:teuthology.orchestra.run.smithi192:> sudo python3 -c ' 2020-11-28T21:51:51.104 INFO:teuthology.orchestra.run.smithi192:> import glob 2020-11-28T21:51:51.104 INFO:teuthology.orchestra.run.smithi192:> import re 2020-11-28T21:51:51.105 INFO:teuthology.orchestra.run.smithi192:> import os 2020-11-28T21:51:51.105 INFO:teuthology.orchestra.run.smithi192:> import subprocess 2020-11-28T21:51:51.105 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:51:51.106 INFO:teuthology.orchestra.run.smithi192:> def find_socket(client_name): 2020-11-28T21:51:51.107 INFO:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-11-28T21:51:51.107 INFO:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2020-11-28T21:51:51.107 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:51:51.108 INFO:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2020-11-28T21:51:51.108 INFO:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2020-11-28T21:51:51.109 INFO:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2020-11-28T21:51:51.109 INFO:teuthology.orchestra.run.smithi192:> return files[0] 2020-11-28T21:51:51.110 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:51:51.110 INFO:teuthology.orchestra.run.smithi192:> for f in files: 2020-11-28T21:51:51.110 INFO:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-11-28T21:51:51.111 INFO:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2020-11-28T21:51:51.111 INFO:teuthology.orchestra.run.smithi192:> return f 2020-11-28T21:51:51.112 INFO:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-11-28T21:51:51.112 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:51:51.113 INFO:teuthology.orchestra.run.smithi192:> print(find_socket("client.0")) 2020-11-28T21:51:51.113 INFO:teuthology.orchestra.run.smithi192:> ' 2020-11-28T21:51:51.194 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.0.11950.asok 2020-11-28T21:51:51.199 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.11950.asok 2020-11-28T21:51:51.200 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:51:51.200 INFO:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.11950.asok status 2020-11-28T21:51:51.299 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2020-11-28T21:51:51.299 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2020-11-28T21:51:51.300 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "a83a3572a957356977342f461361c1cd3ec06dbd", 2020-11-28T21:51:51.300 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 15.2.6-595-ga83a3572 (a83a3572a957356977342f461361c1cd3ec06dbd) octopus (stable)", 2020-11-28T21:51:51.300 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "0", 2020-11-28T21:51:51.300 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2020-11-28T21:51:51.300 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-11-28T21:51:51.301 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "11950", 2020-11-28T21:51:51.301 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2020-11-28T21:51:51.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:51:51.301 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2020-11-28T21:51:51.301 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2020-11-28T21:51:51.302 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 5916, 2020-11-28T21:51:51.302 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2020-11-28T21:51:51.302 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2020-11-28T21:51:51.302 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2020-11-28T21:51:51.302 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 5916 2020-11-28T21:51:51.303 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:51:51.303 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2020-11-28T21:51:51.303 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2020-11-28T21:51:51.303 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2020-11-28T21:51:51.303 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 3377332923 2020-11-28T21:51:51.304 INFO:teuthology.orchestra.run.smithi192.stdout: } 2020-11-28T21:51:51.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:51:51.304 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2020-11-28T21:51:51.304 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2020-11-28T21:51:51.305 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2020-11-28T21:51:51.305 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 3377332923 2020-11-28T21:51:51.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:51:51.305 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.5916 172.21.15.192:0/3377332923", 2020-11-28T21:51:51.305 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "172.21.15.192:0/3377332923", 2020-11-28T21:51:51.306 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2020-11-28T21:51:51.306 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 27, 2020-11-28T21:51:51.306 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 66, 2020-11-28T21:51:51.306 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2020-11-28T21:51:51.306 INFO:teuthology.orchestra.run.smithi192.stdout: "blacklisted": false 2020-11-28T21:51:51.307 INFO:teuthology.orchestra.run.smithi192.stdout:} 2020-11-28T21:51:51.308 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:51:51.309 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:51:51.309 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:51:51.316 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2020-11-28T21:51:51.318 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-28T21:51:51.319 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:51:51.320 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:51:51.377 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:51:51.378 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:51:51.378 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:51:51.426 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2020-11-28T21:51:51.427 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-28T21:51:51.428 INFO:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-28T21:51:51.429 INFO:teuthology.orchestra.run.smithi192:> import sys 2020-11-28T21:51:51.429 INFO:teuthology.orchestra.run.smithi192:> import time 2020-11-28T21:51:51.429 INFO:teuthology.orchestra.run.smithi192:> import os 2020-11-28T21:51:51.430 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:51:51.430 INFO:teuthology.orchestra.run.smithi192:> n = 1000 2020-11-28T21:51:51.431 INFO:teuthology.orchestra.run.smithi192:> abs_path = "/home/ubuntu/cephtest/mnt.0/dc-dir/dc-file" 2020-11-28T21:51:51.431 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:51:51.432 INFO:teuthology.orchestra.run.smithi192:> if not os.path.exists(os.path.dirname(abs_path)): 2020-11-28T21:51:51.432 INFO:teuthology.orchestra.run.smithi192:> os.makedirs(os.path.dirname(abs_path)) 2020-11-28T21:51:51.432 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:51:51.433 INFO:teuthology.orchestra.run.smithi192:> for i in range(0, n): 2020-11-28T21:51:51.433 INFO:teuthology.orchestra.run.smithi192:> fname = "{0}_{1}".format(abs_path, i) 2020-11-28T21:51:51.433 INFO:teuthology.orchestra.run.smithi192:> with open(fname, '"'"'w'"'"') as f: 2020-11-28T21:51:51.434 INFO:teuthology.orchestra.run.smithi192:> f.write('"'"'content'"'"') 2020-11-28T21:51:51.434 INFO:teuthology.orchestra.run.smithi192:> if True: 2020-11-28T21:51:51.435 INFO:teuthology.orchestra.run.smithi192:> f.flush() 2020-11-28T21:51:51.435 INFO:teuthology.orchestra.run.smithi192:> os.fsync(f.fileno()) 2020-11-28T21:51:51.436 INFO:teuthology.orchestra.run.smithi192:> ' 2020-11-28T21:52:02.958 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:52:03.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.071+0000 7f28c1c0f700 1 Processor -- start 2020-11-28T21:52:03.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.071+0000 7f28c1c0f700 1 -- start start 2020-11-28T21:52:03.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.075+0000 7f28c1c0f700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f28bc0611e0 0x7f28bc0615c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:03.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.075+0000 7f28c1c0f700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f28bc0b10f0 0x7f28bc0b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:03.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.075+0000 7f28c1c0f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28bc0b4920 0x7f28bc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:03.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.075+0000 7f28c1c0f700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28bc0b1580 con 0x7f28bc0611e0 2020-11-28T21:52:03.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.075+0000 7f28c1c0f700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28bc0b1790 con 0x7f28bc0b10f0 2020-11-28T21:52:03.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.075+0000 7f28c1c0f700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28bc0b1a30 con 0x7f28bc0b4920 2020-11-28T21:52:03.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.075+0000 7f28b3fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f28bc0b10f0 0x7f28bc0b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:03.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.075+0000 7f28b3fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f28bc0b10f0 0x7f28bc0b4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40880/0 (socket says 172.21.15.162:40880) 2020-11-28T21:52:03.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.075+0000 7f28b3fff700 1 -- 172.21.15.162:0/3532701071 learned_addr learned my addr 172.21.15.162:0/3532701071 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:03.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.075+0000 7f28c0c0d700 1 --2- 172.21.15.162:0/3532701071 >> v2:172.21.15.118:3300/0 conn(0x7f28bc0611e0 0x7f28bc0615c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:03.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.075+0000 7f28c140e700 1 --2- 172.21.15.162:0/3532701071 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28bc0b4920 0x7f28bc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:03.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.075+0000 7f28b3fff700 1 -- 172.21.15.162:0/3532701071 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28bc0b4920 msgr2=0x7f28bc0b4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:03.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.075+0000 7f28b3fff700 1 --2- 172.21.15.162:0/3532701071 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28bc0b4920 0x7f28bc0b4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:03.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.075+0000 7f28b3fff700 1 -- 172.21.15.162:0/3532701071 >> v2:172.21.15.118:3300/0 conn(0x7f28bc0611e0 msgr2=0x7f28bc0615c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:03.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.075+0000 7f28b3fff700 1 --2- 172.21.15.162:0/3532701071 >> v2:172.21.15.118:3300/0 conn(0x7f28bc0611e0 0x7f28bc0615c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:03.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.075+0000 7f28b3fff700 1 -- 172.21.15.162:0/3532701071 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28bc0b3f90 con 0x7f28bc0b10f0 2020-11-28T21:52:03.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.075+0000 7f28b3fff700 1 --2- 172.21.15.162:0/3532701071 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28bc0b10f0 0x7f28bc0b4430 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7f28ac002f30 tx=0x7f28ac006e80).ready entity=mon.1 client_cookie=519702345bc8daf7 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:03.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.075+0000 7f28b37fe700 1 -- 172.21.15.162:0/3532701071 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28ac009640 con 0x7f28bc0b10f0 2020-11-28T21:52:03.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.075+0000 7f28b37fe700 1 -- 172.21.15.162:0/3532701071 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28ac0097a0 con 0x7f28bc0b10f0 2020-11-28T21:52:03.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.075+0000 7f28b37fe700 1 -- 172.21.15.162:0/3532701071 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28ac009b00 con 0x7f28bc0b10f0 2020-11-28T21:52:03.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.075+0000 7f28c1c0f700 1 -- 172.21.15.162:0/3532701071 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28bc0b10f0 msgr2=0x7f28bc0b4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:03.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.075+0000 7f28c1c0f700 1 --2- 172.21.15.162:0/3532701071 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28bc0b10f0 0x7f28bc0b4430 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7f28ac002f30 tx=0x7f28ac006e80).stop 2020-11-28T21:52:03.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.079+0000 7f28c1c0f700 1 -- 172.21.15.162:0/3532701071 shutdown_connections 2020-11-28T21:52:03.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.079+0000 7f28c1c0f700 1 --2- 172.21.15.162:0/3532701071 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28bc0b4920 0x7f28bc0b4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:03.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.079+0000 7f28c1c0f700 1 --2- 172.21.15.162:0/3532701071 >> v2:172.21.15.118:3300/0 conn(0x7f28bc0611e0 0x7f28bc0615c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:03.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.079+0000 7f28c1c0f700 1 --2- 172.21.15.162:0/3532701071 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28bc0b10f0 0x7f28bc0b4430 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:03.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.079+0000 7f28c1c0f700 1 -- 172.21.15.162:0/3532701071 >> 172.21.15.162:0/3532701071 conn(0x7f28bc0b54f0 msgr2=0x7f28bc0b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:03.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.079+0000 7f28c1c0f700 1 -- 172.21.15.162:0/3532701071 shutdown_connections 2020-11-28T21:52:03.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.079+0000 7f28c1c0f700 1 -- 172.21.15.162:0/3532701071 wait complete. 2020-11-28T21:52:03.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.079+0000 7f28c1c0f700 1 Processor -- start 2020-11-28T21:52:03.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.079+0000 7f28c1c0f700 1 -- start start 2020-11-28T21:52:03.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.083+0000 7f28c1c0f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28bc0611e0 0x7f28bc147da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:03.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.083+0000 7f28c1c0f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28bc0b10f0 0x7f28bc09a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:03.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.083+0000 7f28c1c0f700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28bc0b4920 0x7f28bc09ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:03.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.083+0000 7f28c1c0f700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28bc09b070 con 0x7f28bc0b4920 2020-11-28T21:52:03.086 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.083+0000 7f28c1c0f700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28bc141500 con 0x7f28bc0611e0 2020-11-28T21:52:03.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.083+0000 7f28c1c0f700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28bc1554a0 con 0x7f28bc0b10f0 2020-11-28T21:52:03.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.083+0000 7f28c0c0d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28bc0611e0 0x7f28bc147da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:03.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.083+0000 7f28c0c0d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28bc0611e0 0x7f28bc147da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40886/0 (socket says 172.21.15.162:40886) 2020-11-28T21:52:03.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.083+0000 7f28b3fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28bc0b10f0 0x7f28bc09a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:03.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.083+0000 7f28c0c0d700 1 -- 172.21.15.162:0/3613048252 learned_addr learned my addr 172.21.15.162:0/3613048252 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:03.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.083+0000 7f28c140e700 1 --2- 172.21.15.162:0/3613048252 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28bc0b4920 0x7f28bc09ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:03.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.083+0000 7f28c0c0d700 1 -- 172.21.15.162:0/3613048252 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28bc0b10f0 msgr2=0x7f28bc09a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:03.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.083+0000 7f28c0c0d700 1 --2- 172.21.15.162:0/3613048252 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28bc0b10f0 0x7f28bc09a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:03.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.083+0000 7f28c0c0d700 1 -- 172.21.15.162:0/3613048252 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28bc0b4920 msgr2=0x7f28bc09ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:03.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.083+0000 7f28c0c0d700 1 --2- 172.21.15.162:0/3613048252 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28bc0b4920 0x7f28bc09ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:03.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.083+0000 7f28c0c0d700 1 -- 172.21.15.162:0/3613048252 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28bc155740 con 0x7f28bc0611e0 2020-11-28T21:52:03.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.083+0000 7f28c0c0d700 1 --2- 172.21.15.162:0/3613048252 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28bc0611e0 0x7f28bc147da0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7f28b800c000 tx=0x7f28b80091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:03.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.083+0000 7f28b1ffb700 1 -- 172.21.15.162:0/3613048252 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28b800cb00 con 0x7f28bc0611e0 2020-11-28T21:52:03.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.083+0000 7f28b1ffb700 1 -- 172.21.15.162:0/3613048252 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b800cc60 con 0x7f28bc0611e0 2020-11-28T21:52:03.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.083+0000 7f28c1c0f700 1 -- 172.21.15.162:0/3613048252 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28bc141760 con 0x7f28bc0611e0 2020-11-28T21:52:03.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.083+0000 7f28b1ffb700 1 -- 172.21.15.162:0/3613048252 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28b80123f0 con 0x7f28bc0611e0 2020-11-28T21:52:03.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.083+0000 7f28c1c0f700 1 -- 172.21.15.162:0/3613048252 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28bc141ed0 con 0x7f28bc0611e0 2020-11-28T21:52:03.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.083+0000 7f28c1c0f700 1 -- 172.21.15.162:0/3613048252 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28bc1471b0 con 0x7f28bc0611e0 2020-11-28T21:52:03.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.083+0000 7f28b1ffb700 1 -- 172.21.15.162:0/3613048252 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f28b8022070 con 0x7f28bc0611e0 2020-11-28T21:52:03.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.087+0000 7f28b1ffb700 1 --2- 172.21.15.162:0/3613048252 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f289c055440 0x7f289c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:03.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.087+0000 7f28b1ffb700 1 -- 172.21.15.162:0/3613048252 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f28b806e660 con 0x7f28bc0611e0 2020-11-28T21:52:03.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.087+0000 7f28b3fff700 1 --2- 172.21.15.162:0/3613048252 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f289c055440 0x7f289c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:03.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.087+0000 7f28b3fff700 1 --2- 172.21.15.162:0/3613048252 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f289c055440 0x7f289c057900 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7f28ac01b000 tx=0x7f28ac015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:03.094 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.095+0000 7f28b1ffb700 1 -- 172.21.15.162:0/3613048252 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f28bc1471b0 con 0x7f28bc0611e0 2020-11-28T21:52:03.279 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.279+0000 7f28c1c0f700 1 -- 172.21.15.162:0/3613048252 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28bc1471b0 con 0x7f28bc0611e0 2020-11-28T21:52:03.280 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.279+0000 7f28b1ffb700 1 -- 172.21.15.162:0/3613048252 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+2502 (secure 0 0 0) 0x7f28bc1471b0 con 0x7f28bc0611e0 2020-11-28T21:52:03.281 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:52:03.281 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":27,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5342,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6837/3666321033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3666321033},{"type":"v1","addr":"172.21.15.162:6837","nonce":3666321033}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":25},{"gid":5769,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/4179306206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":4179306206},{"type":"v1","addr":"172.21.15.118:6833","nonce":4179306206}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":24}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:51:44.226383+0000","modified":"2020-11-28T21:51:45.230212+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5869},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5869":{"gid":5869,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6835/1433205970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1433205970},{"type":"v1","addr":"172.21.15.162:6835","nonce":1433205970}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-11-28T21:52:03.284 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.279+0000 7f28c1c0f700 1 -- 172.21.15.162:0/3613048252 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f289c055440 msgr2=0x7f289c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:03.284 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.279+0000 7f28c1c0f700 1 --2- 172.21.15.162:0/3613048252 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f289c055440 0x7f289c057900 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7f28ac01b000 tx=0x7f28ac015000).stop 2020-11-28T21:52:03.285 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.283+0000 7f28c1c0f700 1 -- 172.21.15.162:0/3613048252 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28bc0611e0 msgr2=0x7f28bc147da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:03.285 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.283+0000 7f28c1c0f700 1 --2- 172.21.15.162:0/3613048252 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28bc0611e0 0x7f28bc147da0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7f28b800c000 tx=0x7f28b80091a0).stop 2020-11-28T21:52:03.286 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.283+0000 7f28c1c0f700 1 -- 172.21.15.162:0/3613048252 shutdown_connections 2020-11-28T21:52:03.286 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.283+0000 7f28c1c0f700 1 --2- 172.21.15.162:0/3613048252 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f289c055440 0x7f289c057900 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:03.286 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.283+0000 7f28c1c0f700 1 --2- 172.21.15.162:0/3613048252 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28bc0b4920 0x7f28bc09ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:03.287 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.283+0000 7f28c1c0f700 1 --2- 172.21.15.162:0/3613048252 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28bc0611e0 0x7f28bc147da0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:03.287 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.283+0000 7f28c1c0f700 1 --2- 172.21.15.162:0/3613048252 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28bc0b10f0 0x7f28bc09a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:03.287 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.283+0000 7f28c1c0f700 1 -- 172.21.15.162:0/3613048252 >> 172.21.15.162:0/3613048252 conn(0x7f28bc0b54f0 msgr2=0x7f28bc05c540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:03.288 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.283+0000 7f28c1c0f700 1 -- 172.21.15.162:0/3613048252 shutdown_connections 2020-11-28T21:52:03.288 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.283+0000 7f28c1c0f700 1 -- 172.21.15.162:0/3613048252 wait complete. 2020-11-28T21:52:03.288 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 27 2020-11-28T21:52:03.294 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_recall_max_caps 20 2020-11-28T21:52:03.407 INFO:teuthology.orchestra.run.smithi162.stdout:{"success":"mds_recall_max_caps = '20' (not observed, change may require restart) "} 2020-11-28T21:52:03.407 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_caps = '20' (not observed, change may require restart) " } 2020-11-28T21:52:03.408 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:52:03.521 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.519+0000 7fc87d8c3700 1 Processor -- start 2020-11-28T21:52:03.521 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.519+0000 7fc87d8c3700 1 -- start start 2020-11-28T21:52:03.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.523+0000 7fc87d8c3700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc8780603d0 0x7fc8780b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:03.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.523+0000 7fc87d8c3700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8780b4110 con 0x7fc878142a00 2020-11-28T21:52:03.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.523+0000 7fc87d8c3700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8780b43f0 con 0x7fc8780b4c70 2020-11-28T21:52:03.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.523+0000 7fc87d8c3700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc878060f60 con 0x7fc8780603d0 2020-11-28T21:52:03.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.523+0000 7fc87d0c2700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fc8780b4c70 0x7fc878143020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:47740/0 (socket says 172.21.15.162:47740) 2020-11-28T21:52:03.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.523+0000 7fc87d0c2700 1 -- 172.21.15.162:0/1080084133 learned_addr learned my addr 172.21.15.162:0/1080084133 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:52:03.526 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.523+0000 7fc87c8c1700 1 --2- 172.21.15.162:0/1080084133 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc8780603d0 0x7fc8780b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:03.526 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.523+0000 7fc8777fe700 1 -- 172.21.15.162:0/1080084133 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 819491071 0 0) 0x7fc8780b43f0 con 0x7fc8780b4c70 2020-11-28T21:52:03.526 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.523+0000 7fc8777fe700 1 -- 172.21.15.162:0/1080084133 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc85c003230 con 0x7fc8780b4c70 2020-11-28T21:52:03.526 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.523+0000 7fc8777fe700 1 -- 172.21.15.162:0/1080084133 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 893700984 0 0) 0x7fc85c003230 con 0x7fc8780b4c70 2020-11-28T21:52:03.527 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.523+0000 7fc8777fe700 1 -- 172.21.15.162:0/1080084133 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc8780603d0 msgr2=0x7fc8780b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:03.527 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.523+0000 7fc8777fe700 1 --2- 172.21.15.162:0/1080084133 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc8780603d0 0x7fc8780b3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:03.527 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.523+0000 7fc8777fe700 1 -- 172.21.15.162:0/1080084133 >> v1:172.21.15.118:6789/0 conn(0x7fc878142a00 legacy=0x7fc87809cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:03.527 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.523+0000 7fc8777fe700 1 -- 172.21.15.162:0/1080084133 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8780b4690 con 0x7fc8780b4c70 2020-11-28T21:52:03.527 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.523+0000 7fc8777fe700 1 -- 172.21.15.162:0/1080084133 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7fc8700016d0 con 0x7fc8780b4c70 2020-11-28T21:52:03.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.523+0000 7fc8777fe700 1 -- 172.21.15.162:0/1080084133 >> v1:172.21.15.162:6789/0 conn(0x7fc8780b4c70 legacy=0x7fc878143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:03.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.523+0000 7fc8777fe700 1 --2- 172.21.15.162:0/1080084133 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc85c003720 0x7fc85c003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:03.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.523+0000 7fc8777fe700 1 --2- 172.21.15.162:0/1080084133 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc8780603d0 0x7fc85c0044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:03.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.523+0000 7fc8777fe700 1 --2- 172.21.15.162:0/1080084133 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc85c0049c0 0x7fc85c004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:03.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.523+0000 7fc8777fe700 1 -- 172.21.15.162:0/1080084133 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc878060f60 con 0x7fc85c0049c0 2020-11-28T21:52:03.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.523+0000 7fc8777fe700 1 -- 172.21.15.162:0/1080084133 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c0041c0 con 0x7fc85c003720 2020-11-28T21:52:03.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.523+0000 7fc8777fe700 1 -- 172.21.15.162:0/1080084133 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c00b720 con 0x7fc8780603d0 2020-11-28T21:52:03.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.523+0000 7fc87c8c1700 1 --2- 172.21.15.162:0/1080084133 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc85c003720 0x7fc85c003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:03.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.523+0000 7fc877fff700 1 --2- 172.21.15.162:0/1080084133 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc8780603d0 0x7fc85c0044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:03.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.523+0000 7fc87d0c2700 1 --2- 172.21.15.162:0/1080084133 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc85c0049c0 0x7fc85c004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:03.530 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.523+0000 7fc87c8c1700 1 -- 172.21.15.162:0/1080084133 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc8780603d0 msgr2=0x7fc85c0044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:03.530 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.523+0000 7fc87c8c1700 1 --2- 172.21.15.162:0/1080084133 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc8780603d0 0x7fc85c0044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:03.530 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.523+0000 7fc87c8c1700 1 -- 172.21.15.162:0/1080084133 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc85c0049c0 msgr2=0x7fc85c004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:03.530 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.523+0000 7fc87c8c1700 1 --2- 172.21.15.162:0/1080084133 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc85c0049c0 0x7fc85c004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:03.530 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.523+0000 7fc87c8c1700 1 -- 172.21.15.162:0/1080084133 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85c003230 con 0x7fc85c003720 2020-11-28T21:52:03.531 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.523+0000 7fc877fff700 1 --2- 172.21.15.162:0/1080084133 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc8780603d0 0x7fc85c0044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:52:03.531 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.523+0000 7fc87c8c1700 1 --2- 172.21.15.162:0/1080084133 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc85c003720 0x7fc85c003b40 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7fc86000c000 tx=0x7fc8600091a0).ready entity=mon.1 client_cookie=7e011383794db801 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:03.531 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.523+0000 7fc8777fe700 1 -- 172.21.15.162:0/1080084133 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc86000eb80 con 0x7fc85c003720 2020-11-28T21:52:03.531 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.523+0000 7fc8777fe700 1 -- 172.21.15.162:0/1080084133 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86000ece0 con 0x7fc85c003720 2020-11-28T21:52:03.531 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.523+0000 7fc8777fe700 1 -- 172.21.15.162:0/1080084133 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc8600125e0 con 0x7fc85c003720 2020-11-28T21:52:03.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.523+0000 7fc87d8c3700 1 -- 172.21.15.162:0/1080084133 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc85c003720 msgr2=0x7fc85c003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:03.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.523+0000 7fc87d8c3700 1 --2- 172.21.15.162:0/1080084133 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc85c003720 0x7fc85c003b40 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7fc86000c000 tx=0x7fc8600091a0).stop 2020-11-28T21:52:03.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.523+0000 7fc87d0c2700 1 -- 172.21.15.162:0/1080084133 reap_dead start 2020-11-28T21:52:03.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.527+0000 7fc87d8c3700 1 -- 172.21.15.162:0/1080084133 shutdown_connections 2020-11-28T21:52:03.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.527+0000 7fc87d8c3700 1 -- 172.21.15.162:0/1080084133 >> 172.21.15.162:0/1080084133 conn(0x7fc878066a40 msgr2=0x7fc878066e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:03.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.527+0000 7fc87d8c3700 1 -- 172.21.15.162:0/1080084133 shutdown_connections 2020-11-28T21:52:03.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.527+0000 7fc87d8c3700 1 -- 172.21.15.162:0/1080084133 wait complete. 2020-11-28T21:52:03.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.527+0000 7fc87d8c3700 1 Processor -- start 2020-11-28T21:52:03.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.527+0000 7fc87d8c3700 1 -- start start 2020-11-28T21:52:03.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.527+0000 7fc87d8c3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc87808fc50 0x7fc878151d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:03.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.527+0000 7fc87d8c3700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc8781521f0 0x7fc8780b0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:03.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.527+0000 7fc87d8c3700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc8780b0e00 0x7fc878154d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:03.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.527+0000 7fc87d8c3700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8780b27b0 con 0x7fc8781521f0 2020-11-28T21:52:03.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.527+0000 7fc87d8c3700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc878149860 con 0x7fc87808fc50 2020-11-28T21:52:03.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.527+0000 7fc87d8c3700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc878090030 con 0x7fc8780b0e00 2020-11-28T21:52:03.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.527+0000 7fc87c8c1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc87808fc50 0x7fc878151d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:03.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.531+0000 7fc87c8c1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc87808fc50 0x7fc878151d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40904/0 (socket says 172.21.15.162:40904) 2020-11-28T21:52:03.543 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.531+0000 7fc87c8c1700 1 -- 172.21.15.162:0/823464602 learned_addr learned my addr 172.21.15.162:0/823464602 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:03.543 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.531+0000 7fc87d0c2700 1 --2- 172.21.15.162:0/823464602 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc8780b0e00 0x7fc878154d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:03.543 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.531+0000 7fc877fff700 1 --2- 172.21.15.162:0/823464602 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc8781521f0 0x7fc8780b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:03.543 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.531+0000 7fc87c8c1700 1 -- 172.21.15.162:0/823464602 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc8780b0e00 msgr2=0x7fc878154d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:03.544 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.531+0000 7fc87c8c1700 1 --2- 172.21.15.162:0/823464602 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc8780b0e00 0x7fc878154d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:03.544 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.531+0000 7fc87c8c1700 1 -- 172.21.15.162:0/823464602 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc8781521f0 msgr2=0x7fc8780b0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:03.544 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.531+0000 7fc87c8c1700 1 --2- 172.21.15.162:0/823464602 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc8781521f0 0x7fc8780b0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:03.544 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.531+0000 7fc87c8c1700 1 -- 172.21.15.162:0/823464602 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc878148900 con 0x7fc87808fc50 2020-11-28T21:52:03.545 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.531+0000 7fc87c8c1700 1 --2- 172.21.15.162:0/823464602 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc87808fc50 0x7fc878151d00 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7fc878151c50 tx=0x7fc86001a000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:03.545 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.531+0000 7fc875ffb700 1 -- 172.21.15.162:0/823464602 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc8600243f0 con 0x7fc87808fc50 2020-11-28T21:52:03.545 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.531+0000 7fc875ffb700 1 -- 172.21.15.162:0/823464602 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc860024550 con 0x7fc87808fc50 2020-11-28T21:52:03.545 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.531+0000 7fc87d8c3700 1 -- 172.21.15.162:0/823464602 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8780b3280 con 0x7fc87808fc50 2020-11-28T21:52:03.546 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.531+0000 7fc875ffb700 1 -- 172.21.15.162:0/823464602 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc8600248b0 con 0x7fc87808fc50 2020-11-28T21:52:03.546 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.531+0000 7fc87d8c3700 1 -- 172.21.15.162:0/823464602 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8781518d0 con 0x7fc87808fc50 2020-11-28T21:52:03.546 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.531+0000 7fc87d8c3700 1 -- 172.21.15.162:0/823464602 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc878151a90 con 0x7fc87808fc50 2020-11-28T21:52:03.546 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.531+0000 7fc875ffb700 1 -- 172.21.15.162:0/823464602 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc86002a020 con 0x7fc87808fc50 2020-11-28T21:52:03.547 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.531+0000 7fc875ffb700 1 --2- 172.21.15.162:0/823464602 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc850055330 0x7fc8500577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:03.547 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.531+0000 7fc875ffb700 1 -- 172.21.15.162:0/823464602 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc86006fa40 con 0x7fc87808fc50 2020-11-28T21:52:03.547 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.531+0000 7fc877fff700 1 --2- 172.21.15.162:0/823464602 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc850055330 0x7fc8500577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:03.547 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.535+0000 7fc877fff700 1 --2- 172.21.15.162:0/823464602 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc850055330 0x7fc8500577f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7fc868009000 tx=0x7fc868009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:03.548 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.535+0000 7fc875ffb700 1 -- 172.21.15.162:0/823464602 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc860045ca0 con 0x7fc87808fc50 2020-11-28T21:52:03.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.715+0000 7fc87d8c3700 1 -- 172.21.15.162:0/823464602 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc878057050 con 0x7fc87808fc50 2020-11-28T21:52:03.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.715+0000 7fc875ffb700 1 -- 172.21.15.162:0/823464602 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+2502 (secure 0 0 0) 0x7fc860016070 con 0x7fc87808fc50 2020-11-28T21:52:03.717 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:52:03.717 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":27,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5342,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6837/3666321033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3666321033},{"type":"v1","addr":"172.21.15.162:6837","nonce":3666321033}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":25},{"gid":5769,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/4179306206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":4179306206},{"type":"v1","addr":"172.21.15.118:6833","nonce":4179306206}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":24}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:51:44.226383+0000","modified":"2020-11-28T21:51:45.230212+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5869},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5869":{"gid":5869,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6835/1433205970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1433205970},{"type":"v1","addr":"172.21.15.162:6835","nonce":1433205970}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-11-28T21:52:03.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.715+0000 7fc87d8c3700 1 -- 172.21.15.162:0/823464602 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc850055330 msgr2=0x7fc8500577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:03.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.715+0000 7fc87d8c3700 1 --2- 172.21.15.162:0/823464602 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc850055330 0x7fc8500577f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7fc868009000 tx=0x7fc868009040).stop 2020-11-28T21:52:03.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.715+0000 7fc87d8c3700 1 -- 172.21.15.162:0/823464602 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc87808fc50 msgr2=0x7fc878151d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:03.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.715+0000 7fc87d8c3700 1 --2- 172.21.15.162:0/823464602 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc87808fc50 0x7fc878151d00 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7fc878151c50 tx=0x7fc86001a000).stop 2020-11-28T21:52:03.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.719+0000 7fc87d8c3700 1 -- 172.21.15.162:0/823464602 shutdown_connections 2020-11-28T21:52:03.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.719+0000 7fc87d8c3700 1 --2- 172.21.15.162:0/823464602 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc850055330 0x7fc8500577f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:03.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.719+0000 7fc87d8c3700 1 --2- 172.21.15.162:0/823464602 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc8780b0e00 0x7fc878154d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:03.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.719+0000 7fc87d8c3700 1 --2- 172.21.15.162:0/823464602 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc87808fc50 0x7fc878151d00 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:03.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.719+0000 7fc87d8c3700 1 --2- 172.21.15.162:0/823464602 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc8781521f0 0x7fc8780b0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:03.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.719+0000 7fc87d8c3700 1 -- 172.21.15.162:0/823464602 >> 172.21.15.162:0/823464602 conn(0x7fc878066a40 msgr2=0x7fc878141540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:03.723 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.719+0000 7fc87d8c3700 1 -- 172.21.15.162:0/823464602 shutdown_connections 2020-11-28T21:52:03.723 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.719+0000 7fc87d8c3700 1 -- 172.21.15.162:0/823464602 wait complete. 2020-11-28T21:52:03.724 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 27 2020-11-28T21:52:03.737 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_recall_max_decay_threshold 400 2020-11-28T21:52:03.852 INFO:teuthology.orchestra.run.smithi162.stdout:{"success":"mds_recall_max_decay_threshold = '400' (not observed, change may require restart) "} 2020-11-28T21:52:03.853 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_decay_threshold = '400' (not observed, change may require restart) " } 2020-11-28T21:52:03.853 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:52:03.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.979+0000 7fce7a2b4700 1 Processor -- start 2020-11-28T21:52:03.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.979+0000 7fce7a2b4700 1 -- start start 2020-11-28T21:52:03.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.979+0000 7fce7a2b4700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce74060cd0 0x7fce740b1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:03.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.979+0000 7fce7a2b4700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce74061ab0 con 0x7fce74143840 2020-11-28T21:52:03.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.979+0000 7fce7a2b4700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce74061570 con 0x7fce740abc80 2020-11-28T21:52:03.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.979+0000 7fce7a2b4700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce74061810 con 0x7fce74060cd0 2020-11-28T21:52:03.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.979+0000 7fce792b2700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fce740abc80 0x7fce74143130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:47756/0 (socket says 172.21.15.162:47756) 2020-11-28T21:52:03.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.979+0000 7fce792b2700 1 -- 172.21.15.162:0/4139689649 learned_addr learned my addr 172.21.15.162:0/4139689649 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:52:03.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.979+0000 7fce79ab3700 1 --2- 172.21.15.162:0/4139689649 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce74060cd0 0x7fce740b1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:03.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.979+0000 7fce63fff700 1 -- 172.21.15.162:0/4139689649 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2106235994 0 0) 0x7fce74061570 con 0x7fce740abc80 2020-11-28T21:52:03.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.979+0000 7fce63fff700 1 -- 172.21.15.162:0/4139689649 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce5c003230 con 0x7fce740abc80 2020-11-28T21:52:03.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.979+0000 7fce63fff700 1 -- 172.21.15.162:0/4139689649 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 156624795 0 0) 0x7fce5c003230 con 0x7fce740abc80 2020-11-28T21:52:03.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.979+0000 7fce63fff700 1 -- 172.21.15.162:0/4139689649 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce74060cd0 msgr2=0x7fce740b1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:03.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.979+0000 7fce63fff700 1 --2- 172.21.15.162:0/4139689649 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce74060cd0 0x7fce740b1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:03.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.979+0000 7fce63fff700 1 -- 172.21.15.162:0/4139689649 >> v1:172.21.15.118:6789/0 conn(0x7fce74143840 legacy=0x7fce740b0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:03.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.979+0000 7fce63fff700 1 -- 172.21.15.162:0/4139689649 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce74142c90 con 0x7fce740abc80 2020-11-28T21:52:03.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.979+0000 7fce63fff700 1 -- 172.21.15.162:0/4139689649 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7fce6c001a50 con 0x7fce740abc80 2020-11-28T21:52:03.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.979+0000 7fce63fff700 1 -- 172.21.15.162:0/4139689649 >> v1:172.21.15.162:6789/0 conn(0x7fce740abc80 legacy=0x7fce74143130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:03.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.979+0000 7fce63fff700 1 --2- 172.21.15.162:0/4139689649 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce5c003720 0x7fce5c003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:03.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.979+0000 7fce63fff700 1 --2- 172.21.15.162:0/4139689649 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce74060cd0 0x7fce5c0044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:03.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.979+0000 7fce63fff700 1 --2- 172.21.15.162:0/4139689649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce5c0049c0 0x7fce5c004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:03.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.979+0000 7fce63fff700 1 -- 172.21.15.162:0/4139689649 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce74061810 con 0x7fce5c003720 2020-11-28T21:52:03.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.979+0000 7fce63fff700 1 -- 172.21.15.162:0/4139689649 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5c0041c0 con 0x7fce5c0049c0 2020-11-28T21:52:03.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.979+0000 7fce63fff700 1 -- 172.21.15.162:0/4139689649 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce5c00b720 con 0x7fce74060cd0 2020-11-28T21:52:03.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.979+0000 7fce79ab3700 1 --2- 172.21.15.162:0/4139689649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce5c0049c0 0x7fce5c004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:03.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.979+0000 7fce792b2700 1 --2- 172.21.15.162:0/4139689649 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce5c003720 0x7fce5c003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:03.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.979+0000 7fce78ab1700 1 --2- 172.21.15.162:0/4139689649 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce74060cd0 0x7fce5c0044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:03.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.979+0000 7fce79ab3700 1 -- 172.21.15.162:0/4139689649 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce74060cd0 msgr2=0x7fce5c0044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:03.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.979+0000 7fce79ab3700 1 --2- 172.21.15.162:0/4139689649 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce74060cd0 0x7fce5c0044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:03.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.979+0000 7fce79ab3700 1 -- 172.21.15.162:0/4139689649 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce5c003720 msgr2=0x7fce5c003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:03.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.979+0000 7fce79ab3700 1 --2- 172.21.15.162:0/4139689649 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce5c003720 0x7fce5c003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:03.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.979+0000 7fce79ab3700 1 -- 172.21.15.162:0/4139689649 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce5c003230 con 0x7fce5c0049c0 2020-11-28T21:52:03.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.983+0000 7fce79ab3700 1 --2- 172.21.15.162:0/4139689649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce5c0049c0 0x7fce5c004e40 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7fce5c0054a0 tx=0x7fce70008040).ready entity=mon.1 client_cookie=e18ccb8a3ede2d62 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:03.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.983+0000 7fce63fff700 1 -- 172.21.15.162:0/4139689649 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce7000e970 con 0x7fce5c0049c0 2020-11-28T21:52:03.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.983+0000 7fce63fff700 1 -- 172.21.15.162:0/4139689649 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce7000ead0 con 0x7fce5c0049c0 2020-11-28T21:52:03.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.983+0000 7fce63fff700 1 -- 172.21.15.162:0/4139689649 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce7000ee30 con 0x7fce5c0049c0 2020-11-28T21:52:03.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.983+0000 7fce7a2b4700 1 -- 172.21.15.162:0/4139689649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce5c0049c0 msgr2=0x7fce5c004e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:03.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.983+0000 7fce7a2b4700 1 --2- 172.21.15.162:0/4139689649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce5c0049c0 0x7fce5c004e40 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7fce5c0054a0 tx=0x7fce70008040).stop 2020-11-28T21:52:03.992 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.983+0000 7fce79ab3700 1 -- 172.21.15.162:0/4139689649 reap_dead start 2020-11-28T21:52:03.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.983+0000 7fce7a2b4700 1 -- 172.21.15.162:0/4139689649 shutdown_connections 2020-11-28T21:52:03.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.983+0000 7fce7a2b4700 1 -- 172.21.15.162:0/4139689649 >> 172.21.15.162:0/4139689649 conn(0x7fce740b5540 msgr2=0x7fce740b5920 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:03.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.983+0000 7fce7a2b4700 1 -- 172.21.15.162:0/4139689649 shutdown_connections 2020-11-28T21:52:03.996 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.983+0000 7fce7a2b4700 1 -- 172.21.15.162:0/4139689649 wait complete. 2020-11-28T21:52:03.996 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.983+0000 7fce7a2b4700 1 Processor -- start 2020-11-28T21:52:03.996 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.983+0000 7fce7a2b4700 1 -- start start 2020-11-28T21:52:03.996 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.983+0000 7fce7a2b4700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce74140dd0 0x7fce740adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:03.996 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.983+0000 7fce7a2b4700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce7409a630 0x7fce740aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:03.997 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.983+0000 7fce7a2b4700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce74098590 0x7fce740af020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:03.997 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.983+0000 7fce7a2b4700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce740ae4b0 con 0x7fce7409a630 2020-11-28T21:52:03.997 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.983+0000 7fce7a2b4700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce741411b0 con 0x7fce74140dd0 2020-11-28T21:52:03.997 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.983+0000 7fce7a2b4700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce740ae6d0 con 0x7fce74098590 2020-11-28T21:52:03.998 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.987+0000 7fce792b2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce74140dd0 0x7fce740adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:03.998 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.987+0000 7fce792b2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce74140dd0 0x7fce740adfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40924/0 (socket says 172.21.15.162:40924) 2020-11-28T21:52:03.998 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.987+0000 7fce792b2700 1 -- 172.21.15.162:0/3716975845 learned_addr learned my addr 172.21.15.162:0/3716975845 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:03.998 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.987+0000 7fce79ab3700 1 --2- 172.21.15.162:0/3716975845 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce74098590 0x7fce740af020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:03.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.987+0000 7fce78ab1700 1 --2- 172.21.15.162:0/3716975845 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce7409a630 0x7fce740aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:03.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.987+0000 7fce792b2700 1 -- 172.21.15.162:0/3716975845 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce74098590 msgr2=0x7fce740af020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:03.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.987+0000 7fce792b2700 1 --2- 172.21.15.162:0/3716975845 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce74098590 0x7fce740af020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:04.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.987+0000 7fce792b2700 1 -- 172.21.15.162:0/3716975845 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce7409a630 msgr2=0x7fce740aeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:04.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.987+0000 7fce792b2700 1 --2- 172.21.15.162:0/3716975845 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce7409a630 0x7fce740aeb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:04.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.987+0000 7fce792b2700 1 -- 172.21.15.162:0/3716975845 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce74099ef0 con 0x7fce74140dd0 2020-11-28T21:52:04.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.987+0000 7fce79ab3700 1 --2- 172.21.15.162:0/3716975845 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce74098590 0x7fce740af020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:52:04.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.987+0000 7fce792b2700 1 --2- 172.21.15.162:0/3716975845 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce74140dd0 0x7fce740adfc0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7fce6c00d000 tx=0x7fce74061700).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:04.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.987+0000 7fce627fc700 1 -- 172.21.15.162:0/3716975845 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce6c012350 con 0x7fce74140dd0 2020-11-28T21:52:04.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.987+0000 7fce627fc700 1 -- 172.21.15.162:0/3716975845 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6c0124b0 con 0x7fce74140dd0 2020-11-28T21:52:04.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.987+0000 7fce627fc700 1 -- 172.21.15.162:0/3716975845 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce6c012810 con 0x7fce74140dd0 2020-11-28T21:52:04.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.987+0000 7fce7a2b4700 1 -- 172.21.15.162:0/3716975845 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce7409a190 con 0x7fce74140dd0 2020-11-28T21:52:04.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.987+0000 7fce7a2b4700 1 -- 172.21.15.162:0/3716975845 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce7413a860 con 0x7fce74140dd0 2020-11-28T21:52:04.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.987+0000 7fce7a2b4700 1 -- 172.21.15.162:0/3716975845 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce7409a320 con 0x7fce74140dd0 2020-11-28T21:52:04.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.987+0000 7fce627fc700 1 -- 172.21.15.162:0/3716975845 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fce6c012970 con 0x7fce74140dd0 2020-11-28T21:52:04.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.987+0000 7fce627fc700 1 --2- 172.21.15.162:0/3716975845 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fce540554e0 0x7fce540579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:04.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.987+0000 7fce627fc700 1 -- 172.21.15.162:0/3716975845 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fce6c06dbe0 con 0x7fce74140dd0 2020-11-28T21:52:04.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.987+0000 7fce78ab1700 1 --2- 172.21.15.162:0/3716975845 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fce540554e0 0x7fce540579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:04.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.991+0000 7fce78ab1700 1 --2- 172.21.15.162:0/3716975845 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fce540554e0 0x7fce540579a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7fce6400d000 tx=0x7fce64009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:04.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:03.995+0000 7fce627fc700 1 -- 172.21.15.162:0/3716975845 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fce6c043ee0 con 0x7fce74140dd0 2020-11-28T21:52:04.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.191+0000 7fce7a2b4700 1 -- 172.21.15.162:0/3716975845 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce74057050 con 0x7fce74140dd0 2020-11-28T21:52:04.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.191+0000 7fce627fc700 1 -- 172.21.15.162:0/3716975845 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+2502 (secure 0 0 0) 0x7fce6c019020 con 0x7fce74140dd0 2020-11-28T21:52:04.192 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:52:04.192 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":27,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5342,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6837/3666321033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3666321033},{"type":"v1","addr":"172.21.15.162:6837","nonce":3666321033}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":25},{"gid":5769,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/4179306206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":4179306206},{"type":"v1","addr":"172.21.15.118:6833","nonce":4179306206}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":24}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:51:44.226383+0000","modified":"2020-11-28T21:51:45.230212+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5869},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5869":{"gid":5869,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6835/1433205970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1433205970},{"type":"v1","addr":"172.21.15.162:6835","nonce":1433205970}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-11-28T21:52:04.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.191+0000 7fce7a2b4700 1 -- 172.21.15.162:0/3716975845 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fce540554e0 msgr2=0x7fce540579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:04.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.191+0000 7fce7a2b4700 1 --2- 172.21.15.162:0/3716975845 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fce540554e0 0x7fce540579a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7fce6400d000 tx=0x7fce64009040).stop 2020-11-28T21:52:04.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.191+0000 7fce7a2b4700 1 -- 172.21.15.162:0/3716975845 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce74140dd0 msgr2=0x7fce740adfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:04.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.191+0000 7fce7a2b4700 1 --2- 172.21.15.162:0/3716975845 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce74140dd0 0x7fce740adfc0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7fce6c00d000 tx=0x7fce74061700).stop 2020-11-28T21:52:04.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.191+0000 7fce7a2b4700 1 -- 172.21.15.162:0/3716975845 shutdown_connections 2020-11-28T21:52:04.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.191+0000 7fce7a2b4700 1 --2- 172.21.15.162:0/3716975845 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fce540554e0 0x7fce540579a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:04.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.191+0000 7fce7a2b4700 1 --2- 172.21.15.162:0/3716975845 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce74098590 0x7fce740af020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:04.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.191+0000 7fce7a2b4700 1 --2- 172.21.15.162:0/3716975845 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce74140dd0 0x7fce740adfc0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:04.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.191+0000 7fce7a2b4700 1 --2- 172.21.15.162:0/3716975845 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce7409a630 0x7fce740aeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:04.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.191+0000 7fce7a2b4700 1 -- 172.21.15.162:0/3716975845 >> 172.21.15.162:0/3716975845 conn(0x7fce740b5540 msgr2=0x7fce740b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:04.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.195+0000 7fce7a2b4700 1 -- 172.21.15.162:0/3716975845 shutdown_connections 2020-11-28T21:52:04.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.195+0000 7fce7a2b4700 1 -- 172.21.15.162:0/3716975845 wait complete. 2020-11-28T21:52:04.196 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 27 2020-11-28T21:52:04.211 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell mds.a cache drop 10 2020-11-28T21:52:04.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.319+0000 7fe14f59e700 1 Processor -- start 2020-11-28T21:52:04.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.319+0000 7fe14f59e700 1 -- start start 2020-11-28T21:52:04.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.319+0000 7fe14f59e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe1500b2d00 0x7fe15005c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:04.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.319+0000 7fe14f59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe1500b3ab0 0x7fe1500b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:04.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.319+0000 7fe14f59e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe15005baf0 con 0x7fe150142ad0 2020-11-28T21:52:04.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.319+0000 7fe14f59e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1500b0e20 con 0x7fe1500b2d00 2020-11-28T21:52:04.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.319+0000 7fe14f59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe15005bd90 con 0x7fe1500b3ab0 2020-11-28T21:52:04.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.323+0000 7fe14e59c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe1500b2d00 0x7fe15005c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:04.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.323+0000 7fe14dd9b700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe1500b3ab0 0x7fe1500b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:04.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.323+0000 7fe14e59c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe1500b2d00 0x7fe15005c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40936/0 (socket says 172.21.15.162:40936) 2020-11-28T21:52:04.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.323+0000 7fe14ed9d700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fe150142ad0 0x7fe15013fa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.162:37926/0 (socket says 172.21.15.162:37926) 2020-11-28T21:52:04.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.323+0000 7fe14dd9b700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe1500b3ab0 0x7fe1500b30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:38156/0 (socket says 172.21.15.162:38156) 2020-11-28T21:52:04.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.323+0000 7fe14e59c700 1 -- 172.21.15.162:0/3163723315 learned_addr learned my addr 172.21.15.162:0/3163723315 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:04.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.323+0000 7fe14e59c700 1 -- 172.21.15.162:0/3163723315 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe1500b3ab0 msgr2=0x7fe1500b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:04.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.323+0000 7fe14e59c700 1 --2- 172.21.15.162:0/3163723315 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe1500b3ab0 0x7fe1500b30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:04.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.323+0000 7fe14e59c700 1 -- 172.21.15.162:0/3163723315 >> v1:172.21.15.118:6789/0 conn(0x7fe150142ad0 legacy=0x7fe15013fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:04.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.323+0000 7fe14e59c700 1 -- 172.21.15.162:0/3163723315 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe15005c030 con 0x7fe1500b2d00 2020-11-28T21:52:04.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.323+0000 7fe14e59c700 1 --2- 172.21.15.162:0/3163723315 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1500b2d00 0x7fe15005c4d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7fe138002f30 tx=0x7fe138006e80).ready entity=mon.1 client_cookie=97e55c79c6a1909e server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:04.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.323+0000 7fe14d59a700 1 -- 172.21.15.162:0/3163723315 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1380097f0 con 0x7fe1500b2d00 2020-11-28T21:52:04.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.323+0000 7fe14d59a700 1 -- 172.21.15.162:0/3163723315 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe138009950 con 0x7fe1500b2d00 2020-11-28T21:52:04.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.323+0000 7fe14d59a700 1 -- 172.21.15.162:0/3163723315 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe138009cb0 con 0x7fe1500b2d00 2020-11-28T21:52:04.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.323+0000 7fe14f59e700 1 -- 172.21.15.162:0/3163723315 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1500b2d00 msgr2=0x7fe15005c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:04.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.323+0000 7fe14f59e700 1 --2- 172.21.15.162:0/3163723315 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1500b2d00 0x7fe15005c4d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7fe138002f30 tx=0x7fe138006e80).stop 2020-11-28T21:52:04.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.323+0000 7fe14f59e700 1 -- 172.21.15.162:0/3163723315 shutdown_connections 2020-11-28T21:52:04.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.323+0000 7fe14f59e700 1 --2- 172.21.15.162:0/3163723315 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1500b2d00 0x7fe15005c4d0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:04.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.323+0000 7fe14f59e700 1 --2- 172.21.15.162:0/3163723315 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe1500b3ab0 0x7fe1500b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:04.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.323+0000 7fe14f59e700 1 -- 172.21.15.162:0/3163723315 >> 172.21.15.162:0/3163723315 conn(0x7fe1500604c0 msgr2=0x7fe1500608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:04.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.323+0000 7fe14f59e700 1 -- 172.21.15.162:0/3163723315 shutdown_connections 2020-11-28T21:52:04.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.323+0000 7fe14f59e700 1 -- 172.21.15.162:0/3163723315 wait complete. 2020-11-28T21:52:04.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.323+0000 7fe14f59e700 1 Processor -- start 2020-11-28T21:52:04.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.323+0000 7fe14f59e700 1 -- start start 2020-11-28T21:52:04.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.327+0000 7fe14f59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1500b2d00 0x7fe150141ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:04.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.327+0000 7fe14f59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe1500b3ab0 0x7fe1500b79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:04.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.327+0000 7fe14f59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe150142ad0 0x7fe1500b7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:04.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.327+0000 7fe14f59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1500b7020 con 0x7fe150142ad0 2020-11-28T21:52:04.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.327+0000 7fe14f59e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1500b8b30 con 0x7fe1500b2d00 2020-11-28T21:52:04.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.327+0000 7fe14f59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe150143fe0 con 0x7fe1500b3ab0 2020-11-28T21:52:04.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.327+0000 7fe14e59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1500b2d00 0x7fe150141ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:04.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.327+0000 7fe14e59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1500b2d00 0x7fe150141ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40938/0 (socket says 172.21.15.162:40938) 2020-11-28T21:52:04.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.327+0000 7fe14e59c700 1 -- 172.21.15.162:0/2551294527 learned_addr learned my addr 172.21.15.162:0/2551294527 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:04.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.327+0000 7fe14ed9d700 1 --2- 172.21.15.162:0/2551294527 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe150142ad0 0x7fe1500b7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:04.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.327+0000 7fe14dd9b700 1 --2- 172.21.15.162:0/2551294527 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe1500b3ab0 0x7fe1500b79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:04.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.327+0000 7fe14e59c700 1 -- 172.21.15.162:0/2551294527 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe1500b3ab0 msgr2=0x7fe1500b79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:04.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.327+0000 7fe14e59c700 1 --2- 172.21.15.162:0/2551294527 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe1500b3ab0 0x7fe1500b79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:04.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.327+0000 7fe14e59c700 1 -- 172.21.15.162:0/2551294527 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe150142ad0 msgr2=0x7fe1500b7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:04.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.327+0000 7fe14e59c700 1 --2- 172.21.15.162:0/2551294527 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe150142ad0 0x7fe1500b7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:04.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.327+0000 7fe14e59c700 1 -- 172.21.15.162:0/2551294527 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe150144280 con 0x7fe1500b2d00 2020-11-28T21:52:04.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.327+0000 7fe14e59c700 1 --2- 172.21.15.162:0/2551294527 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1500b2d00 0x7fe150141ef0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7fe138002f30 tx=0x7fe138015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:04.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.327+0000 7fe1477fe700 1 -- 172.21.15.162:0/2551294527 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe138011ca0 con 0x7fe1500b2d00 2020-11-28T21:52:04.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.327+0000 7fe1477fe700 1 -- 172.21.15.162:0/2551294527 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe138011e00 con 0x7fe1500b2d00 2020-11-28T21:52:04.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.327+0000 7fe1477fe700 1 -- 172.21.15.162:0/2551294527 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe13801e650 con 0x7fe1500b2d00 2020-11-28T21:52:04.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.327+0000 7fe14f59e700 1 -- 172.21.15.162:0/2551294527 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1501464b0 con 0x7fe1500b2d00 2020-11-28T21:52:04.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.327+0000 7fe14f59e700 1 -- 172.21.15.162:0/2551294527 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe150146780 con 0x7fe1500b2d00 2020-11-28T21:52:04.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.331+0000 7fe1477fe700 1 -- 172.21.15.162:0/2551294527 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe138025030 con 0x7fe1500b2d00 2020-11-28T21:52:04.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.331+0000 7fe1477fe700 1 --2- 172.21.15.162:0/2551294527 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe134055440 0x7fe134057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:04.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.331+0000 7fe1477fe700 1 -- 172.21.15.162:0/2551294527 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe13806f460 con 0x7fe1500b2d00 2020-11-28T21:52:04.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.331+0000 7fe14dd9b700 1 --2- 172.21.15.162:0/2551294527 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe134055440 0x7fe134057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:04.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.331+0000 7fe14dd9b700 1 --2- 172.21.15.162:0/2551294527 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe134055440 0x7fe134057900 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7fe14000b000 tx=0x7fe140008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:04.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.331+0000 7fe163439740 1 Processor -- start 2020-11-28T21:52:04.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.331+0000 7fe163439740 1 -- start start 2020-11-28T21:52:04.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.331+0000 7fe163439740 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x1053f20 0x1082850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:04.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.331+0000 7fe163439740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xebc820 0x10d6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:04.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.331+0000 7fe163439740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0xe4ec20 0x10daed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:04.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.331+0000 7fe163439740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0xfc8df0 con 0xe4ec20 2020-11-28T21:52:04.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.331+0000 7fe163439740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0xf59250 con 0xebc820 2020-11-28T21:52:04.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.331+0000 7fe163439740 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0xed56b0 con 0x1053f20 2020-11-28T21:52:04.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.331+0000 7fe14dd9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xebc820 0x10d6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:04.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.331+0000 7fe14dd9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xebc820 0x10d6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40950/0 (socket says 172.21.15.162:40950) 2020-11-28T21:52:04.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.331+0000 7fe14dd9b700 1 -- 172.21.15.162:0/3235559390 learned_addr learned my addr 172.21.15.162:0/3235559390 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:04.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.331+0000 7fe14ed9d700 1 --2- 172.21.15.162:0/3235559390 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0xe4ec20 0x10daed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:04.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.331+0000 7fe14e59c700 1 --2- 172.21.15.162:0/3235559390 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x1053f20 0x1082850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:04.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.331+0000 7fe14dd9b700 1 -- 172.21.15.162:0/3235559390 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x1053f20 msgr2=0x1082850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:04.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.331+0000 7fe14dd9b700 1 --2- 172.21.15.162:0/3235559390 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x1053f20 0x1082850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:04.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.331+0000 7fe14dd9b700 1 -- 172.21.15.162:0/3235559390 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0xe4ec20 msgr2=0x10daed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:04.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.331+0000 7fe14dd9b700 1 --2- 172.21.15.162:0/3235559390 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0xe4ec20 0x10daed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:04.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.331+0000 7fe14dd9b700 1 -- 172.21.15.162:0/3235559390 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xfe9d10 con 0xebc820 2020-11-28T21:52:04.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.335+0000 7fe14dd9b700 1 --2- 172.21.15.162:0/3235559390 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xebc820 0x10d6b00 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x10d1340 tx=0x7fe140015000).ready entity=mon.1 client_cookie=3a136a9fb92d711f server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:04.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.335+0000 7fe14f59e700 1 -- 172.21.15.162:0/3235559390 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe14000b340 con 0xebc820 2020-11-28T21:52:04.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.335+0000 7fe14f59e700 1 -- 172.21.15.162:0/3235559390 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe140014dc0 con 0xebc820 2020-11-28T21:52:04.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.335+0000 7fe14f59e700 1 -- 172.21.15.162:0/3235559390 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe14001f5f0 con 0xebc820 2020-11-28T21:52:04.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.335+0000 7fe163439740 1 -- 172.21.15.162:0/3235559390 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xebc820 msgr2=0x10d6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:04.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.335+0000 7fe163439740 1 --2- 172.21.15.162:0/3235559390 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xebc820 0x10d6b00 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x10d1340 tx=0x7fe140015000).stop 2020-11-28T21:52:04.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.335+0000 7fe163439740 1 -- 172.21.15.162:0/3235559390 shutdown_connections 2020-11-28T21:52:04.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.335+0000 7fe163439740 1 --2- 172.21.15.162:0/3235559390 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x1053f20 0x1082850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:04.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.335+0000 7fe163439740 1 --2- 172.21.15.162:0/3235559390 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0xe4ec20 0x10daed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:04.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.335+0000 7fe163439740 1 --2- 172.21.15.162:0/3235559390 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xebc820 0x10d6b00 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:04.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.335+0000 7fe163439740 1 -- 172.21.15.162:0/3235559390 >> 172.21.15.162:0/3235559390 conn(0x1083840 msgr2=0x109f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:04.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.335+0000 7fe163439740 1 -- 172.21.15.162:0/3235559390 shutdown_connections 2020-11-28T21:52:04.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.335+0000 7fe163439740 1 -- 172.21.15.162:0/3235559390 wait complete. 2020-11-28T21:52:04.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.335+0000 7fe163439740 1 -- start start 2020-11-28T21:52:04.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.335+0000 7fe163439740 1 Processor -- start 2020-11-28T21:52:04.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.335+0000 7fe163439740 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0xe4ec20 0x1082a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:04.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.335+0000 7fe163439740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xebc820 0x10e5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:04.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.335+0000 7fe163439740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x1053f20 0x10e88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:04.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.335+0000 7fe163439740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0xff8a70 con 0x1053f20 2020-11-28T21:52:04.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.335+0000 7fe163439740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0xff8bb0 con 0xebc820 2020-11-28T21:52:04.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.335+0000 7fe163439740 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0xffa180 con 0xe4ec20 2020-11-28T21:52:04.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.335+0000 7fe163439740 10 client.0 ll_register_callbacks cb 0x10837a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-28T21:52:04.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.335+0000 7fe14dd9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xebc820 0x10e5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:04.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.339+0000 7fe14dd9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xebc820 0x10e5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40954/0 (socket says 172.21.15.162:40954) 2020-11-28T21:52:04.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.339+0000 7fe14dd9b700 1 -- 172.21.15.162:0/2255838432 learned_addr learned my addr 172.21.15.162:0/2255838432 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:04.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.339+0000 7fe14ed9d700 1 --2- 172.21.15.162:0/2255838432 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x1053f20 0x10e88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:04.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.339+0000 7fe14e59c700 1 --2- 172.21.15.162:0/2255838432 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0xe4ec20 0x1082a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:04.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.339+0000 7fe14dd9b700 1 -- 172.21.15.162:0/2255838432 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0xe4ec20 msgr2=0x1082a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:04.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.339+0000 7fe14dd9b700 1 --2- 172.21.15.162:0/2255838432 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0xe4ec20 0x1082a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:04.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.339+0000 7fe14dd9b700 1 -- 172.21.15.162:0/2255838432 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x1053f20 msgr2=0x10e88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:04.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.339+0000 7fe14dd9b700 1 --2- 172.21.15.162:0/2255838432 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x1053f20 0x10e88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:04.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.339+0000 7fe14dd9b700 1 -- 172.21.15.162:0/2255838432 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0xe6c790 con 0xebc820 2020-11-28T21:52:04.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.339+0000 7fe14dd9b700 1 --2- 172.21.15.162:0/2255838432 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xebc820 0x10e5460 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x10a1720 tx=0x7fe14001f000).ready entity=mon.1 client_cookie=d075c34184f85bc0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:04.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.339+0000 7fe12d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2020-11-28T21:52:04.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.339+0000 7fe12d7fa700 1 -- 172.21.15.162:0/2255838432 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe140029630 con 0xebc820 2020-11-28T21:52:04.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.339+0000 7fe163439740 1 -- 172.21.15.162:0/2255838432 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xfa9810 con 0xebc820 2020-11-28T21:52:04.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.339+0000 7fe12d7fa700 1 -- 172.21.15.162:0/2255838432 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe140029790 con 0xebc820 2020-11-28T21:52:04.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.339+0000 7fe12d7fa700 1 -- 172.21.15.162:0/2255838432 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1400106e0 con 0xebc820 2020-11-28T21:52:04.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.339+0000 7fe163439740 10 client.6067 fetch_fsmap learned FSMap version 27 2020-11-28T21:52:04.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.339+0000 7fe163439740 1 -- 172.21.15.162:0/2255838432 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0xe77e70 con 0xebc820 2020-11-28T21:52:04.355 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.339+0000 7fe12d7fa700 1 -- 172.21.15.162:0/2255838432 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xfa9810 con 0xebc820 2020-11-28T21:52:04.355 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.339+0000 7fe12d7fa700 1 -- 172.21.15.162:0/2255838432 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 27) v1 ==== 1434+0+0 (secure 0 0 0) 0x7fe140012040 con 0xebc820 2020-11-28T21:52:04.355 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.339+0000 7fe163439740 10 client.6067 fetch_fsmap finished waiting for FSMap version 27 2020-11-28T21:52:04.355 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.339+0000 7fe163439740 10 client.6067 resolve_mds: resolved ID 'a' to GID 5869 2020-11-28T21:52:04.355 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.339+0000 7fe163439740 20 client.6067 populate_metadata read hostname 'smithi162' 2020-11-28T21:52:04.356 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.339+0000 7fe163439740 1 --2- 172.21.15.162:0/2255838432 >> [v2:172.21.15.162:6834/1433205970,v1:172.21.15.162:6835/1433205970] conn(0x10e4d20 0x10e8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:04.356 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.339+0000 7fe163439740 4 client.6067 mds_command: new command op to 5869 tid=0[{"prefix": "get_command_descriptions"}] 2020-11-28T21:52:04.356 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.339+0000 7fe163439740 1 -- 172.21.15.162:0/2255838432 --> [v2:172.21.15.162:6834/1433205970,v1:172.21.15.162:6835/1433205970] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0xee18d0 con 0x10e4d20 2020-11-28T21:52:04.356 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.339+0000 7fe14e59c700 1 --2- 172.21.15.162:0/2255838432 >> [v2:172.21.15.162:6834/1433205970,v1:172.21.15.162:6835/1433205970] conn(0x10e4d20 0x10e8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:04.356 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.339+0000 7fe12d7fa700 10 client.6067 ms_handle_connect on v2:172.21.15.162:6834/1433205970 2020-11-28T21:52:04.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.339+0000 7fe14e59c700 1 --2- 172.21.15.162:0/2255838432 >> [v2:172.21.15.162:6834/1433205970,v1:172.21.15.162:6835/1433205970] conn(0x10e4d20 0x10e8d90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=6784d123636b5131 server_cookie=1ef7e4ce4dbe309c in_seq=0 out_seq=0 2020-11-28T21:52:04.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.339+0000 7fe12d7fa700 1 -- 172.21.15.162:0/2255838432 <== mds.0 v2:172.21.15.162:6834/1433205970 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16606 (crc 0 0 0) 0xee18d0 con 0x10e4d20 2020-11-28T21:52:04.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.339+0000 7fe12d7fa700 10 client.6067 handle_command_reply: tid=0 2020-11-28T21:52:04.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.339+0000 7fe163439740 1 client.6067 shutdown 2020-11-28T21:52:04.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.339+0000 7fe163439740 1 -- 172.21.15.162:0/2255838432 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xebc820 msgr2=0x10e5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:04.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.339+0000 7fe163439740 1 --2- 172.21.15.162:0/2255838432 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xebc820 0x10e5460 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x10a1720 tx=0x7fe14001f000).stop 2020-11-28T21:52:04.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.339+0000 7fe163439740 1 -- 172.21.15.162:0/2255838432 shutdown_connections 2020-11-28T21:52:04.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.339+0000 7fe163439740 1 --2- 172.21.15.162:0/2255838432 >> [v2:172.21.15.162:6834/1433205970,v1:172.21.15.162:6835/1433205970] conn(0x10e4d20 0x10e8d90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:04.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.343+0000 7fe163439740 1 --2- 172.21.15.162:0/2255838432 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0xe4ec20 0x1082a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:04.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.343+0000 7fe163439740 1 --2- 172.21.15.162:0/2255838432 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x1053f20 0x10e88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:04.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.343+0000 7fe12d7fa700 0 client.6067 ms_handle_reset on v2:172.21.15.162:6834/1433205970 2020-11-28T21:52:04.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.343+0000 7fe163439740 1 --2- 172.21.15.162:0/2255838432 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xebc820 0x10e5460 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:04.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.343+0000 7fe163439740 1 -- 172.21.15.162:0/2255838432 >> 172.21.15.162:0/2255838432 conn(0x1083840 msgr2=0x109f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:04.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.343+0000 7fe163439740 1 -- 172.21.15.162:0/2255838432 shutdown_connections 2020-11-28T21:52:04.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.343+0000 7fe163439740 1 -- 172.21.15.162:0/2255838432 wait complete. 2020-11-28T21:52:04.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.343+0000 7fe163439740 20 client.6067 trim_cache size 0 max 16384 2020-11-28T21:52:04.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.359+0000 7fe163439740 1 Processor -- start 2020-11-28T21:52:04.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.359+0000 7fe163439740 1 -- start start 2020-11-28T21:52:04.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.359+0000 7fe163439740 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0xe4ec20 0x109d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:04.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.359+0000 7fe163439740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xebc820 0x10d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:04.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.359+0000 7fe163439740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x1053f20 0x109e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:04.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.359+0000 7fe163439740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0xfdb510 con 0x1053f20 2020-11-28T21:52:04.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.359+0000 7fe163439740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x10a2480 con 0xebc820 2020-11-28T21:52:04.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.359+0000 7fe163439740 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x10e6080 con 0xe4ec20 2020-11-28T21:52:04.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.359+0000 7fe14dd9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xebc820 0x10d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:04.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.359+0000 7fe14dd9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xebc820 0x10d0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40962/0 (socket says 172.21.15.162:40962) 2020-11-28T21:52:04.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.359+0000 7fe14dd9b700 1 -- 172.21.15.162:0/4170544887 learned_addr learned my addr 172.21.15.162:0/4170544887 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:04.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.359+0000 7fe14e59c700 1 --2- 172.21.15.162:0/4170544887 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0xe4ec20 0x109d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:04.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.359+0000 7fe14ed9d700 1 --2- 172.21.15.162:0/4170544887 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x1053f20 0x109e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:04.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.359+0000 7fe14dd9b700 1 -- 172.21.15.162:0/4170544887 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0xe4ec20 msgr2=0x109d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:04.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.359+0000 7fe14dd9b700 1 --2- 172.21.15.162:0/4170544887 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0xe4ec20 0x109d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:04.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.359+0000 7fe14dd9b700 1 -- 172.21.15.162:0/4170544887 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x1053f20 msgr2=0x109e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:04.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.359+0000 7fe14dd9b700 1 --2- 172.21.15.162:0/4170544887 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x1053f20 0x109e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:04.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.359+0000 7fe14dd9b700 1 -- 172.21.15.162:0/4170544887 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe15013d920 con 0xebc820 2020-11-28T21:52:04.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.359+0000 7fe14dd9b700 1 --2- 172.21.15.162:0/4170544887 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xebc820 0x10d0340 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x10f55a0 tx=0x7fe140028000).ready entity=mon.1 client_cookie=5b96849c7f5fcf48 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:04.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.359+0000 7fe12f7fe700 1 -- 172.21.15.162:0/4170544887 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe14003e070 con 0xebc820 2020-11-28T21:52:04.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.363+0000 7fe12f7fe700 1 -- 172.21.15.162:0/4170544887 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe140013a00 con 0xebc820 2020-11-28T21:52:04.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.363+0000 7fe12f7fe700 1 -- 172.21.15.162:0/4170544887 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe140013d60 con 0xebc820 2020-11-28T21:52:04.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.363+0000 7fe163439740 1 -- 172.21.15.162:0/4170544887 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xebc820 msgr2=0x10d0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:04.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.363+0000 7fe163439740 1 --2- 172.21.15.162:0/4170544887 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xebc820 0x10d0340 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x10f55a0 tx=0x7fe140028000).stop 2020-11-28T21:52:04.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.363+0000 7fe163439740 1 -- 172.21.15.162:0/4170544887 shutdown_connections 2020-11-28T21:52:04.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.363+0000 7fe163439740 1 --2- 172.21.15.162:0/4170544887 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0xe4ec20 0x109d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:04.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.363+0000 7fe163439740 1 --2- 172.21.15.162:0/4170544887 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x1053f20 0x109e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:04.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.363+0000 7fe163439740 1 --2- 172.21.15.162:0/4170544887 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xebc820 0x10d0340 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:04.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.363+0000 7fe163439740 1 -- 172.21.15.162:0/4170544887 >> 172.21.15.162:0/4170544887 conn(0x1083840 msgr2=0x10d0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:04.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.363+0000 7fe163439740 1 -- 172.21.15.162:0/4170544887 shutdown_connections 2020-11-28T21:52:04.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.363+0000 7fe163439740 1 -- 172.21.15.162:0/4170544887 wait complete. 2020-11-28T21:52:04.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.363+0000 7fe163439740 1 -- start start 2020-11-28T21:52:04.371 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.363+0000 7fe163439740 1 Processor -- start 2020-11-28T21:52:04.371 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.363+0000 7fe163439740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0xe4ec20 0x10d6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:04.371 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.363+0000 7fe163439740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xebc820 0x113e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:04.371 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.367+0000 7fe163439740 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x1053f20 0x113edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:04.371 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.367+0000 7fe163439740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x10d52e0 con 0xe4ec20 2020-11-28T21:52:04.372 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.367+0000 7fe163439740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0xfe1a60 con 0xebc820 2020-11-28T21:52:04.372 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.367+0000 7fe163439740 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x10d04a0 con 0x1053f20 2020-11-28T21:52:04.372 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.367+0000 7fe163439740 10 client.0 ll_register_callbacks cb 0x10d58e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-28T21:52:04.372 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.367+0000 7fe14dd9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xebc820 0x113e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:04.372 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.367+0000 7fe14e59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0xe4ec20 0x10d6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:04.373 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.367+0000 7fe14dd9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xebc820 0x113e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40968/0 (socket says 172.21.15.162:40968) 2020-11-28T21:52:04.373 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.367+0000 7fe14dd9b700 1 -- 172.21.15.162:0/2202546991 learned_addr learned my addr 172.21.15.162:0/2202546991 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:04.373 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.367+0000 7fe14ed9d700 1 --2- 172.21.15.162:0/2202546991 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x1053f20 0x113edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:04.373 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.367+0000 7fe14dd9b700 1 -- 172.21.15.162:0/2202546991 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x1053f20 msgr2=0x113edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:04.373 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.367+0000 7fe14dd9b700 1 --2- 172.21.15.162:0/2202546991 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x1053f20 0x113edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:04.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.367+0000 7fe14dd9b700 1 -- 172.21.15.162:0/2202546991 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0xe4ec20 msgr2=0x10d6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:04.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.367+0000 7fe14dd9b700 1 --2- 172.21.15.162:0/2202546991 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0xe4ec20 0x10d6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:04.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.367+0000 7fe14dd9b700 1 -- 172.21.15.162:0/2202546991 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x10cf970 con 0xebc820 2020-11-28T21:52:04.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.367+0000 7fe14e59c700 1 --2- 172.21.15.162:0/2202546991 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0xe4ec20 0x10d6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:52:04.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.367+0000 7fe14dd9b700 1 --2- 172.21.15.162:0/2202546991 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xebc820 0x113e8e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x10f55a0 tx=0x7fe140021040).ready entity=mon.1 client_cookie=4b69702cc10c54 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:04.375 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.367+0000 7fe12e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2020-11-28T21:52:04.375 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.367+0000 7fe12e7fc700 1 -- 172.21.15.162:0/2202546991 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe14003e040 con 0xebc820 2020-11-28T21:52:04.375 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.367+0000 7fe12e7fc700 1 -- 172.21.15.162:0/2202546991 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe140045c80 con 0xebc820 2020-11-28T21:52:04.375 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.367+0000 7fe163439740 1 -- 172.21.15.162:0/2202546991 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xeb24d0 con 0xebc820 2020-11-28T21:52:04.375 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.367+0000 7fe12e7fc700 1 -- 172.21.15.162:0/2202546991 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe14000b340 con 0xebc820 2020-11-28T21:52:04.376 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.367+0000 7fe163439740 10 client.6073 fetch_fsmap learned FSMap version 27 2020-11-28T21:52:04.376 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.367+0000 7fe163439740 1 -- 172.21.15.162:0/2202546991 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x113f320 con 0xebc820 2020-11-28T21:52:04.376 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.367+0000 7fe12e7fc700 1 -- 172.21.15.162:0/2202546991 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xeb24d0 con 0xebc820 2020-11-28T21:52:04.376 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.367+0000 7fe12e7fc700 1 -- 172.21.15.162:0/2202546991 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 27) v1 ==== 1434+0+0 (secure 0 0 0) 0x7fe140041030 con 0xebc820 2020-11-28T21:52:04.377 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.367+0000 7fe163439740 10 client.6073 fetch_fsmap finished waiting for FSMap version 27 2020-11-28T21:52:04.377 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.367+0000 7fe163439740 10 client.6073 resolve_mds: resolved ID 'a' to GID 5869 2020-11-28T21:52:04.377 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.367+0000 7fe163439740 20 client.6073 populate_metadata read hostname 'smithi162' 2020-11-28T21:52:04.377 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.367+0000 7fe163439740 1 --2- 172.21.15.162:0/2202546991 >> [v2:172.21.15.162:6834/1433205970,v1:172.21.15.162:6835/1433205970] conn(0x109bdd0 0x109c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:04.377 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.367+0000 7fe163439740 4 client.6073 mds_command: new command op to 5869 tid=0[{"prefix": "cache drop", "timeout": 10}] 2020-11-28T21:52:04.378 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.367+0000 7fe163439740 1 -- 172.21.15.162:0/2202546991 --> [v2:172.21.15.162:6834/1433205970,v1:172.21.15.162:6835/1433205970] -- command(tid 0: {"prefix": "cache drop", "timeout": 10}) v1 -- 0xf00730 con 0x109bdd0 2020-11-28T21:52:04.378 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.367+0000 7fe14e59c700 1 --2- 172.21.15.162:0/2202546991 >> [v2:172.21.15.162:6834/1433205970,v1:172.21.15.162:6835/1433205970] conn(0x109bdd0 0x109c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:04.378 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.367+0000 7fe12e7fc700 10 client.6073 ms_handle_connect on v2:172.21.15.162:6834/1433205970 2020-11-28T21:52:04.379 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:04.367+0000 7fe14e59c700 1 --2- 172.21.15.162:0/2202546991 >> [v2:172.21.15.162:6834/1433205970,v1:172.21.15.162:6835/1433205970] conn(0x109bdd0 0x109c1b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=b8c9fd5cc70c1b6 server_cookie=153b634c86fb310b in_seq=0 out_seq=0 2020-11-28T21:52:08.034 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:52:08.037 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:52:08.039 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:52:08.042 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:52:14.684 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:14.682+0000 7fe12e7fc700 1 -- 172.21.15.162:0/2202546991 <== mds.0 v2:172.21.15.162:6834/1433205970 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+406 (crc 0 0 0) 0xf00730 con 0x109bdd0 2020-11-28T21:52:14.685 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:14.682+0000 7fe12e7fc700 10 client.6073 handle_command_reply: tid=0 2020-11-28T21:52:14.685 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:14.682+0000 7fe163439740 1 client.6073 shutdown 2020-11-28T21:52:14.685 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:14.682+0000 7fe163439740 1 -- 172.21.15.162:0/2202546991 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xebc820 msgr2=0x113e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:14.685 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:14.682+0000 7fe163439740 1 --2- 172.21.15.162:0/2202546991 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xebc820 0x113e8e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x10f55a0 tx=0x7fe140021040).stop 2020-11-28T21:52:14.686 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:14.682+0000 7fe163439740 1 -- 172.21.15.162:0/2202546991 shutdown_connections 2020-11-28T21:52:14.686 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:14.682+0000 7fe163439740 1 --2- 172.21.15.162:0/2202546991 >> [v2:172.21.15.162:6834/1433205970,v1:172.21.15.162:6835/1433205970] conn(0x109bdd0 0x109c1b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:14.686 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:14.682+0000 7fe163439740 1 --2- 172.21.15.162:0/2202546991 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x1053f20 0x113edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:14.686 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:14.682+0000 7fe163439740 1 --2- 172.21.15.162:0/2202546991 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0xe4ec20 0x10d6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:14.686 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:14.682+0000 7fe163439740 1 --2- 172.21.15.162:0/2202546991 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0xebc820 0x113e8e0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:14.687 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:14.682+0000 7fe12e7fc700 0 client.6073 ms_handle_reset on v2:172.21.15.162:6834/1433205970 2020-11-28T21:52:14.687 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:14.682+0000 7fe163439740 1 -- 172.21.15.162:0/2202546991 >> 172.21.15.162:0/2202546991 conn(0x1083840 msgr2=0x109e230 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:14.687 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:14.682+0000 7fe163439740 1 -- 172.21.15.162:0/2202546991 shutdown_connections 2020-11-28T21:52:14.687 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:14.682+0000 7fe163439740 1 -- 172.21.15.162:0/2202546991 wait complete. 2020-11-28T21:52:14.688 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:14.682+0000 7fe163439740 20 client.6073 trim_cache size 0 max 16384 2020-11-28T21:52:14.688 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:14.682+0000 7fe14f59e700 1 -- 172.21.15.162:0/2551294527 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe134055440 msgr2=0x7fe134057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:14.688 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:14.682+0000 7fe14f59e700 1 --2- 172.21.15.162:0/2551294527 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe134055440 0x7fe134057900 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7fe14000b000 tx=0x7fe140008040).stop 2020-11-28T21:52:14.688 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:14.682+0000 7fe14f59e700 1 -- 172.21.15.162:0/2551294527 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1500b2d00 msgr2=0x7fe150141ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:14.688 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:14.682+0000 7fe14f59e700 1 --2- 172.21.15.162:0/2551294527 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1500b2d00 0x7fe150141ef0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7fe138002f30 tx=0x7fe138015000).stop 2020-11-28T21:52:14.689 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:14.682+0000 7fe14f59e700 1 -- 172.21.15.162:0/2551294527 shutdown_connections 2020-11-28T21:52:14.689 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:14.682+0000 7fe14f59e700 1 --2- 172.21.15.162:0/2551294527 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe134055440 0x7fe134057900 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:14.689 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:14.682+0000 7fe14f59e700 1 --2- 172.21.15.162:0/2551294527 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe150142ad0 0x7fe1500b7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:14.689 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:14.682+0000 7fe14f59e700 1 --2- 172.21.15.162:0/2551294527 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1500b2d00 0x7fe150141ef0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:14.689 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:14.682+0000 7fe14f59e700 1 --2- 172.21.15.162:0/2551294527 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe1500b3ab0 0x7fe1500b79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:14.690 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:14.682+0000 7fe14f59e700 1 -- 172.21.15.162:0/2551294527 >> 172.21.15.162:0/2551294527 conn(0x7fe1500604c0 msgr2=0x7fe15013f840 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:14.690 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:14.682+0000 7fe14f59e700 1 -- 172.21.15.162:0/2551294527 shutdown_connections 2020-11-28T21:52:14.690 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:14.682+0000 7fe14f59e700 1 -- 172.21.15.162:0/2551294527 wait complete. 2020-11-28T21:52:14.690 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2020-11-28T21:52:14.690 INFO:teuthology.orchestra.run.smithi162.stdout: "client_recall": { 2020-11-28T21:52:14.691 INFO:teuthology.orchestra.run.smithi162.stdout: "return_code": -110, 2020-11-28T21:52:14.691 INFO:teuthology.orchestra.run.smithi162.stdout: "message": "(110) Connection timed out", 2020-11-28T21:52:14.691 INFO:teuthology.orchestra.run.smithi162.stdout: "recalled": 240 2020-11-28T21:52:14.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:14.691 INFO:teuthology.orchestra.run.smithi162.stdout: "flush_journal": { 2020-11-28T21:52:14.692 INFO:teuthology.orchestra.run.smithi162.stdout: "return_code": 0, 2020-11-28T21:52:14.692 INFO:teuthology.orchestra.run.smithi162.stdout: "message": "" 2020-11-28T21:52:14.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:14.692 INFO:teuthology.orchestra.run.smithi162.stdout: "trim_cache": { 2020-11-28T21:52:14.693 INFO:teuthology.orchestra.run.smithi162.stdout: "trimmed": 220 2020-11-28T21:52:14.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:14.693 INFO:teuthology.orchestra.run.smithi162.stdout: "cache": { 2020-11-28T21:52:14.693 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": { 2020-11-28T21:52:14.693 INFO:teuthology.orchestra.run.smithi162.stdout: "items": 15448, 2020-11-28T21:52:14.694 INFO:teuthology.orchestra.run.smithi162.stdout: "bytes": 2428464 2020-11-28T21:52:14.694 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:14.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:14.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 10.313089142000001 2020-11-28T21:52:14.694 INFO:teuthology.orchestra.run.smithi162.stdout:} 2020-11-28T21:52:14.695 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-28T21:52:14.696 INFO:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:52:14.714 INFO:teuthology.orchestra.run.smithi118:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:52:14.731 INFO:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:52:14.747 INFO:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:52:14.766 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-28T21:52:14.767 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:52:14.767 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:52:14.768 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:52:14.818 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2020-11-28T21:52:14.819 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-28T21:52:14.819 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2020-11-28T21:52:14.820 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-28T21:52:14.820 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:52:14.903 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:ceph-fuse[11950]: fuse finished with error 0 and tester_r 0 2020-11-28T21:52:14.905 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:52:14.905 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:52:14.906 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:52:14.953 INFO:teuthology.orchestra.run.smithi192.stdout:ext2/ext3 2020-11-28T21:52:14.954 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-11-28T21:52:15.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-28T21:52:15.070 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:52:15.117 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:52:15.118 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:52:15.118 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:52:15.125 INFO:teuthology.orchestra.run.smithi096.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-11-28T21:52:15.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:52:15.126 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2020-11-28T21:52:15.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-28T21:52:15.127 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:52:15.172 INFO:teuthology.orchestra.run.smithi096.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-11-28T21:52:15.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:52:15.173 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config rm mon mon_allow_pool_delete 2020-11-28T21:52:15.289 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.286+0000 7f899ca4c700 1 Processor -- start 2020-11-28T21:52:15.290 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.286+0000 7f899ca4c700 1 -- start start 2020-11-28T21:52:15.291 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.290+0000 7f899ca4c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f89980b7320 0x7f899813d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:15.292 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.290+0000 7f899ca4c700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f89980b19f0 0x7f899813dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:15.292 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.290+0000 7f899ca4c700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89980b7700 con 0x7f89980b1f00 2020-11-28T21:52:15.292 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.290+0000 7f899ca4c700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8998067b50 con 0x7f89980b7320 2020-11-28T21:52:15.292 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.290+0000 7f899ca4c700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8998067df0 con 0x7f89980b19f0 2020-11-28T21:52:15.293 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.290+0000 7f899659c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f89980b7320 0x7f899813d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:15.293 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.290+0000 7f899659c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f89980b7320 0x7f899813d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40982/0 (socket says 172.21.15.162:40982) 2020-11-28T21:52:15.293 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.290+0000 7f899659c700 1 -- 172.21.15.162:0/2921013517 learned_addr learned my addr 172.21.15.162:0/2921013517 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:15.293 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.290+0000 7f8995d9b700 1 --2- 172.21.15.162:0/2921013517 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f89980b19f0 0x7f899813dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:15.295 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.290+0000 7f899659c700 1 -- 172.21.15.162:0/2921013517 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f89980b19f0 msgr2=0x7f899813dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:15.295 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.290+0000 7f899659c700 1 --2- 172.21.15.162:0/2921013517 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f89980b19f0 0x7f899813dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:15.296 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.290+0000 7f899659c700 1 -- 172.21.15.162:0/2921013517 >> v1:172.21.15.118:6789/0 conn(0x7f89980b1f00 legacy=0x7f89980b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:15.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.290+0000 7f899559a700 1 -- 172.21.15.162:0/2921013517 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3607812764 0 0) 0x7f89980b7700 con 0x7f89980b1f00 2020-11-28T21:52:15.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.290+0000 7f899659c700 1 -- 172.21.15.162:0/2921013517 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8998065c80 con 0x7f89980b7320 2020-11-28T21:52:15.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.290+0000 7f899659c700 1 --2- 172.21.15.162:0/2921013517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89980b7320 0x7f899813d9d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f898c002f30 tx=0x7f898c006e80).ready entity=mon.1 client_cookie=60321622efcfb863 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:15.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.290+0000 7f899559a700 1 -- 172.21.15.162:0/2921013517 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f898c0097f0 con 0x7f89980b7320 2020-11-28T21:52:15.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.290+0000 7f899559a700 1 -- 172.21.15.162:0/2921013517 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f898c009950 con 0x7f89980b7320 2020-11-28T21:52:15.298 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.290+0000 7f899559a700 1 -- 172.21.15.162:0/2921013517 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f898c009cb0 con 0x7f89980b7320 2020-11-28T21:52:15.298 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.290+0000 7f899ca4c700 1 -- 172.21.15.162:0/2921013517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89980b7320 msgr2=0x7f899813d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:15.298 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.290+0000 7f899ca4c700 1 --2- 172.21.15.162:0/2921013517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89980b7320 0x7f899813d9d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f898c002f30 tx=0x7f898c006e80).stop 2020-11-28T21:52:15.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.290+0000 7f899ca4c700 1 -- 172.21.15.162:0/2921013517 shutdown_connections 2020-11-28T21:52:15.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.290+0000 7f899ca4c700 1 --2- 172.21.15.162:0/2921013517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89980b7320 0x7f899813d9d0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:15.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.290+0000 7f899ca4c700 1 --2- 172.21.15.162:0/2921013517 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f89980b19f0 0x7f899813dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:15.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.290+0000 7f899ca4c700 1 -- 172.21.15.162:0/2921013517 >> 172.21.15.162:0/2921013517 conn(0x7f8998062830 msgr2=0x7f8998062c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:15.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.290+0000 7f899ca4c700 1 -- 172.21.15.162:0/2921013517 shutdown_connections 2020-11-28T21:52:15.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.290+0000 7f899ca4c700 1 -- 172.21.15.162:0/2921013517 wait complete. 2020-11-28T21:52:15.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.294+0000 7f899ca4c700 1 Processor -- start 2020-11-28T21:52:15.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.294+0000 7f899ca4c700 1 -- start start 2020-11-28T21:52:15.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.294+0000 7f899ca4c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f89980b19f0 0x7f899813eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:15.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.294+0000 7f899ca4c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89980b1f00 0x7f89980aae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:15.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.294+0000 7f899ca4c700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f89980b7320 0x7f89980ab310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:15.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.294+0000 7f899ca4c700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89980ab800 con 0x7f89980b19f0 2020-11-28T21:52:15.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.294+0000 7f899ca4c700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89980addf0 con 0x7f89980b1f00 2020-11-28T21:52:15.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.294+0000 7f899ca4c700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89980ad4e0 con 0x7f89980b7320 2020-11-28T21:52:15.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.294+0000 7f8995d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89980b1f00 0x7f89980aae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:15.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.294+0000 7f8995d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89980b1f00 0x7f89980aae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40988/0 (socket says 172.21.15.162:40988) 2020-11-28T21:52:15.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.294+0000 7f8995d9b700 1 -- 172.21.15.162:0/820373548 learned_addr learned my addr 172.21.15.162:0/820373548 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:15.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.294+0000 7f899659c700 1 --2- 172.21.15.162:0/820373548 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f89980b19f0 0x7f899813eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:15.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.294+0000 7f8996d9d700 1 --2- 172.21.15.162:0/820373548 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f89980b7320 0x7f89980ab310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:15.306 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.294+0000 7f8995d9b700 1 -- 172.21.15.162:0/820373548 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f89980b7320 msgr2=0x7f89980ab310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:15.306 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.294+0000 7f8995d9b700 1 --2- 172.21.15.162:0/820373548 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f89980b7320 0x7f89980ab310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:15.306 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.294+0000 7f8995d9b700 1 -- 172.21.15.162:0/820373548 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f89980b19f0 msgr2=0x7f899813eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:15.307 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.294+0000 7f8995d9b700 1 --2- 172.21.15.162:0/820373548 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f89980b19f0 0x7f899813eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:15.307 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.294+0000 7f8995d9b700 1 -- 172.21.15.162:0/820373548 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89981500f0 con 0x7f89980b1f00 2020-11-28T21:52:15.307 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.294+0000 7f8995d9b700 1 --2- 172.21.15.162:0/820373548 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89980b1f00 0x7f89980aae20 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7f89980ab940 tx=0x7f8988006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:15.307 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.294+0000 7f89877fe700 1 -- 172.21.15.162:0/820373548 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f898800ca60 con 0x7f89980b1f00 2020-11-28T21:52:15.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.294+0000 7f89877fe700 1 -- 172.21.15.162:0/820373548 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f898800cbc0 con 0x7f89980b1f00 2020-11-28T21:52:15.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.294+0000 7f899ca4c700 1 -- 172.21.15.162:0/820373548 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89980ad8e0 con 0x7f89980b1f00 2020-11-28T21:52:15.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.294+0000 7f89877fe700 1 -- 172.21.15.162:0/820373548 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8988013450 con 0x7f89980b1f00 2020-11-28T21:52:15.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.294+0000 7f899ca4c700 1 -- 172.21.15.162:0/820373548 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89980aa650 con 0x7f89980b1f00 2020-11-28T21:52:15.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.298+0000 7f89877fe700 1 -- 172.21.15.162:0/820373548 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f898801a030 con 0x7f89980b1f00 2020-11-28T21:52:15.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.298+0000 7f899ca4c700 1 -- 172.21.15.162:0/820373548 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89980adf60 con 0x7f89980b1f00 2020-11-28T21:52:15.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.298+0000 7f89877fe700 1 --2- 172.21.15.162:0/820373548 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8974055330 0x7f89740577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:15.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.298+0000 7f89877fe700 1 -- 172.21.15.162:0/820373548 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f898806df50 con 0x7f89980b1f00 2020-11-28T21:52:15.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.298+0000 7f899659c700 1 --2- 172.21.15.162:0/820373548 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8974055330 0x7f89740577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:15.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.298+0000 7f899659c700 1 --2- 172.21.15.162:0/820373548 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8974055330 0x7f89740577f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f898c01a000 tx=0x7f898c015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:15.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.302+0000 7f89877fe700 1 -- 172.21.15.162:0/820373548 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8988047c80 con 0x7f89980b1f00 2020-11-28T21:52:15.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.430+0000 7f899ca4c700 1 -- 172.21.15.162:0/820373548 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f8998057050 con 0x7f89980b1f00 2020-11-28T21:52:15.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.442+0000 7f89877fe700 1 -- 172.21.15.162:0/820373548 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8988047870 con 0x7f89980b1f00 2020-11-28T21:52:15.445 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.442+0000 7f899ca4c700 1 -- 172.21.15.162:0/820373548 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8974055330 msgr2=0x7f89740577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:15.446 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.442+0000 7f899ca4c700 1 --2- 172.21.15.162:0/820373548 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8974055330 0x7f89740577f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f898c01a000 tx=0x7f898c015000).stop 2020-11-28T21:52:15.446 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.442+0000 7f899ca4c700 1 -- 172.21.15.162:0/820373548 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89980b1f00 msgr2=0x7f89980aae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:15.446 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.442+0000 7f899ca4c700 1 --2- 172.21.15.162:0/820373548 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89980b1f00 0x7f89980aae20 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7f89980ab940 tx=0x7f8988006dc0).stop 2020-11-28T21:52:15.446 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.442+0000 7f899ca4c700 1 -- 172.21.15.162:0/820373548 shutdown_connections 2020-11-28T21:52:15.447 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.442+0000 7f899ca4c700 1 --2- 172.21.15.162:0/820373548 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8974055330 0x7f89740577f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:15.447 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.442+0000 7f899ca4c700 1 --2- 172.21.15.162:0/820373548 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f89980b7320 0x7f89980ab310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:15.447 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.442+0000 7f899ca4c700 1 --2- 172.21.15.162:0/820373548 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f89980b19f0 0x7f899813eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:15.447 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.442+0000 7f899ca4c700 1 --2- 172.21.15.162:0/820373548 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89980b1f00 0x7f89980aae20 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:15.448 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.442+0000 7f899ca4c700 1 -- 172.21.15.162:0/820373548 >> 172.21.15.162:0/820373548 conn(0x7f8998062830 msgr2=0x7f899813ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:15.448 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.442+0000 7f899ca4c700 1 -- 172.21.15.162:0/820373548 shutdown_connections 2020-11-28T21:52:15.448 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.442+0000 7f899ca4c700 1 -- 172.21.15.162:0/820373548 wait complete. 2020-11-28T21:52:15.458 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout' 2020-11-28T21:52:15.567 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.566+0000 7f8370f45700 1 Processor -- start 2020-11-28T21:52:15.567 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.566+0000 7f8370f45700 1 -- start start 2020-11-28T21:52:15.569 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.566+0000 7f8370f45700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f836c0a9560 0x7f836c0a9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:15.570 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.566+0000 7f8370f45700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f836c0b16a0 0x7f836c0b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:15.570 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.566+0000 7f8370f45700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f836c0a8890 con 0x7f836c0b2020 2020-11-28T21:52:15.570 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.566+0000 7f8370f45700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f836c0a86a0 con 0x7f836c0a9560 2020-11-28T21:52:15.570 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.566+0000 7f8370f45700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f836c0a9e30 con 0x7f836c0b16a0 2020-11-28T21:52:15.571 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.566+0000 7f836affd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f836c0b16a0 0x7f836c0b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:15.571 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.566+0000 7f836affd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f836c0b16a0 0x7f836c0b1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40996/0 (socket says 172.21.15.162:40996) 2020-11-28T21:52:15.571 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.566+0000 7f836b7fe700 1 --2- 172.21.15.162:0/2837723078 >> v2:172.21.15.118:3300/0 conn(0x7f836c0a9560 0x7f836c0a9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:15.571 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.566+0000 7f836affd700 1 -- 172.21.15.162:0/2837723078 learned_addr learned my addr 172.21.15.162:0/2837723078 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:15.571 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.566+0000 7f836affd700 1 -- 172.21.15.162:0/2837723078 >> v2:172.21.15.118:3300/0 conn(0x7f836c0a9560 msgr2=0x7f836c0a9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:15.572 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.566+0000 7f836affd700 1 --2- 172.21.15.162:0/2837723078 >> v2:172.21.15.118:3300/0 conn(0x7f836c0a9560 0x7f836c0a9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:15.572 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.566+0000 7f836affd700 1 -- 172.21.15.162:0/2837723078 >> v1:172.21.15.118:6789/0 conn(0x7f836c0b2020 legacy=0x7f836c05f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:15.572 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.566+0000 7f836affd700 1 -- 172.21.15.162:0/2837723078 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f836c0b02f0 con 0x7f836c0b16a0 2020-11-28T21:52:15.572 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.566+0000 7f836affd700 1 --2- 172.21.15.162:0/2837723078 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f836c0b16a0 0x7f836c0b1b30 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f835c002f30 tx=0x7f835c006e80).ready entity=mon.1 client_cookie=7db599cfc435af51 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:15.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.566+0000 7f836a7fc700 1 -- 172.21.15.162:0/2837723078 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f835c0097f0 con 0x7f836c0b16a0 2020-11-28T21:52:15.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.566+0000 7f836a7fc700 1 -- 172.21.15.162:0/2837723078 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f835c009950 con 0x7f836c0b16a0 2020-11-28T21:52:15.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.566+0000 7f836a7fc700 1 -- 172.21.15.162:0/2837723078 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f835c009cb0 con 0x7f836c0b16a0 2020-11-28T21:52:15.574 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.566+0000 7f8370f45700 1 -- 172.21.15.162:0/2837723078 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f836c0b16a0 msgr2=0x7f836c0b1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:15.574 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.566+0000 7f8370f45700 1 --2- 172.21.15.162:0/2837723078 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f836c0b16a0 0x7f836c0b1b30 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f835c002f30 tx=0x7f835c006e80).stop 2020-11-28T21:52:15.576 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.566+0000 7f8370f45700 1 -- 172.21.15.162:0/2837723078 shutdown_connections 2020-11-28T21:52:15.577 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.566+0000 7f8370f45700 1 --2- 172.21.15.162:0/2837723078 >> v2:172.21.15.118:3300/0 conn(0x7f836c0a9560 0x7f836c0a9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:15.577 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.566+0000 7f8370f45700 1 --2- 172.21.15.162:0/2837723078 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f836c0b16a0 0x7f836c0b1b30 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:15.577 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.566+0000 7f8370f45700 1 -- 172.21.15.162:0/2837723078 >> 172.21.15.162:0/2837723078 conn(0x7f836c05b970 msgr2=0x7f836c05bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:15.577 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.566+0000 7f8370f45700 1 -- 172.21.15.162:0/2837723078 shutdown_connections 2020-11-28T21:52:15.578 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.566+0000 7f8370f45700 1 -- 172.21.15.162:0/2837723078 wait complete. 2020-11-28T21:52:15.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.570+0000 7f8370f45700 1 Processor -- start 2020-11-28T21:52:15.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.570+0000 7f8370f45700 1 -- start start 2020-11-28T21:52:15.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.570+0000 7f8370f45700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f836c0a9560 0x7f836c0aa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:15.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.570+0000 7f8370f45700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f836c0b16a0 0x7f836c145ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:15.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.570+0000 7f8370f45700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f836c0b2020 0x7f836c1463c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:15.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.570+0000 7f8370f45700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836c0acad0 con 0x7f836c0b16a0 2020-11-28T21:52:15.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.570+0000 7f8370f45700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836c152360 con 0x7f836c0b2020 2020-11-28T21:52:15.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.570+0000 7f8370f45700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f836c152600 con 0x7f836c0a9560 2020-11-28T21:52:15.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.570+0000 7f836bfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f836c0b2020 0x7f836c1463c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:15.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.570+0000 7f836bfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f836c0b2020 0x7f836c1463c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41002/0 (socket says 172.21.15.162:41002) 2020-11-28T21:52:15.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.570+0000 7f836bfff700 1 -- 172.21.15.162:0/1304872974 learned_addr learned my addr 172.21.15.162:0/1304872974 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:15.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.570+0000 7f836affd700 1 --2- 172.21.15.162:0/1304872974 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f836c0b16a0 0x7f836c145ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:15.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.570+0000 7f836b7fe700 1 --2- 172.21.15.162:0/1304872974 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f836c0a9560 0x7f836c0aa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:15.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.570+0000 7f836bfff700 1 -- 172.21.15.162:0/1304872974 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f836c0a9560 msgr2=0x7f836c0aa2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:15.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.570+0000 7f836bfff700 1 --2- 172.21.15.162:0/1304872974 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f836c0a9560 0x7f836c0aa2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:15.586 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.570+0000 7f836bfff700 1 -- 172.21.15.162:0/1304872974 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f836c0b16a0 msgr2=0x7f836c145ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:15.586 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.570+0000 7f836bfff700 1 --2- 172.21.15.162:0/1304872974 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f836c0b16a0 0x7f836c145ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:15.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.570+0000 7f836bfff700 1 -- 172.21.15.162:0/1304872974 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f836c145a30 con 0x7f836c0b2020 2020-11-28T21:52:15.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.570+0000 7f836bfff700 1 --2- 172.21.15.162:0/1304872974 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f836c0b2020 0x7f836c1463c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f836c067df0 tx=0x7f8364007f70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:15.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.570+0000 7f8368ff9700 1 -- 172.21.15.162:0/1304872974 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8364003080 con 0x7f836c0b2020 2020-11-28T21:52:15.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.570+0000 7f8368ff9700 1 -- 172.21.15.162:0/1304872974 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f836400a7b0 con 0x7f836c0b2020 2020-11-28T21:52:15.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.570+0000 7f8370f45700 1 -- 172.21.15.162:0/1304872974 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f836c1468b0 con 0x7f836c0b2020 2020-11-28T21:52:15.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.570+0000 7f8368ff9700 1 -- 172.21.15.162:0/1304872974 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f836400ab10 con 0x7f836c0b2020 2020-11-28T21:52:15.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.570+0000 7f8370f45700 1 -- 172.21.15.162:0/1304872974 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f836c0ae170 con 0x7f836c0b2020 2020-11-28T21:52:15.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.570+0000 7f8370f45700 1 -- 172.21.15.162:0/1304872974 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f836c1472d0 con 0x7f836c0b2020 2020-11-28T21:52:15.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.574+0000 7f8368ff9700 1 -- 172.21.15.162:0/1304872974 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8364019030 con 0x7f836c0b2020 2020-11-28T21:52:15.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.574+0000 7f8368ff9700 1 --2- 172.21.15.162:0/1304872974 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f83440554e0 0x7f83440579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:15.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.574+0000 7f8368ff9700 1 -- 172.21.15.162:0/1304872974 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f836406d8b0 con 0x7f836c0b2020 2020-11-28T21:52:15.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.574+0000 7f836b7fe700 1 --2- 172.21.15.162:0/1304872974 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f83440554e0 0x7f83440579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:15.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.574+0000 7f836b7fe700 1 --2- 172.21.15.162:0/1304872974 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f83440554e0 0x7f83440579a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f835400b000 tx=0x7f8354006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:15.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.578+0000 7f8368ff9700 1 -- 172.21.15.162:0/1304872974 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8364043bb0 con 0x7f836c0b2020 2020-11-28T21:52:15.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.770+0000 7f8370f45700 1 -- 172.21.15.162:0/1304872974 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout"]} v 0) v1 -- 0x7f836c1472d0 con 0x7f836c0b2020 2020-11-28T21:52:15.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.826+0000 7f8368ff9700 1 -- 172.21.15.162:0/1304872974 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout"]}]=0 v245) v1 ==== 147+0+0 (secure 0 0 0) 0x7f836c1472d0 con 0x7f836c0b2020 2020-11-28T21:52:15.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.830+0000 7f8370f45700 1 -- 172.21.15.162:0/1304872974 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f83440554e0 msgr2=0x7f83440579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:15.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.830+0000 7f8370f45700 1 --2- 172.21.15.162:0/1304872974 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f83440554e0 0x7f83440579a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f835400b000 tx=0x7f8354006dc0).stop 2020-11-28T21:52:15.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.830+0000 7f8370f45700 1 -- 172.21.15.162:0/1304872974 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f836c0b2020 msgr2=0x7f836c1463c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:15.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.830+0000 7f8370f45700 1 --2- 172.21.15.162:0/1304872974 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f836c0b2020 0x7f836c1463c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f836c067df0 tx=0x7f8364007f70).stop 2020-11-28T21:52:15.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.830+0000 7f8370f45700 1 -- 172.21.15.162:0/1304872974 shutdown_connections 2020-11-28T21:52:15.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.830+0000 7f8370f45700 1 --2- 172.21.15.162:0/1304872974 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f83440554e0 0x7f83440579a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:15.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.830+0000 7f8370f45700 1 --2- 172.21.15.162:0/1304872974 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f836c0a9560 0x7f836c0aa2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:15.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.830+0000 7f8370f45700 1 --2- 172.21.15.162:0/1304872974 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f836c0b2020 0x7f836c1463c0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:15.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.830+0000 7f8370f45700 1 --2- 172.21.15.162:0/1304872974 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f836c0b16a0 0x7f836c145ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:15.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.830+0000 7f8370f45700 1 -- 172.21.15.162:0/1304872974 >> 172.21.15.162:0/1304872974 conn(0x7f836c05b970 msgr2=0x7f836c0b4770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:15.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.830+0000 7f8370f45700 1 -- 172.21.15.162:0/1304872974 shutdown_connections 2020-11-28T21:52:15.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.830+0000 7f8370f45700 1 -- 172.21.15.162:0/1304872974 wait complete. 2020-11-28T21:52:15.849 INFO:tasks.cephfs_test_runner:test_drop_cache_command_timeout (tasks.cephfs.test_misc.TestCacheDrop) ... ok 2020-11-28T21:52:15.850 INFO:tasks.cephfs_test_runner:Starting test: test_cap_revoke_nonresponder (tasks.cephfs.test_misc.TestMisc) 2020-11-28T21:52:15.851 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder' 2020-11-28T21:52:15.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.958+0000 7f5ea84b9700 1 Processor -- start 2020-11-28T21:52:15.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.958+0000 7f5ea84b9700 1 -- start start 2020-11-28T21:52:15.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.958+0000 7f5ea84b9700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5ea00b1990 0x7f5ea0140890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:15.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.958+0000 7f5ea84b9700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5ea00b11b0 0x7f5ea0140d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:15.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.958+0000 7f5ea84b9700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5ea00b1d70 0x7f5ea014a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:15.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.958+0000 7f5ea84b9700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ea0146e20 con 0x7f5ea00b1d70 2020-11-28T21:52:15.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.958+0000 7f5ea84b9700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ea00b2870 con 0x7f5ea00b1990 2020-11-28T21:52:15.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.958+0000 7f5ea84b9700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea00b22e0 con 0x7f5ea00b11b0 2020-11-28T21:52:15.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.958+0000 7f5ea6255700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5ea00b1990 0x7f5ea0140890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:15.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.958+0000 7f5ea6255700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5ea00b1990 0x7f5ea0140890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41012/0 (socket says 172.21.15.162:41012) 2020-11-28T21:52:15.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.958+0000 7f5ea6255700 1 -- 172.21.15.162:0/709356285 learned_addr learned my addr 172.21.15.162:0/709356285 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:15.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.958+0000 7f5ea6a56700 1 --2- 172.21.15.162:0/709356285 >> v2:172.21.15.118:3300/0 conn(0x7f5ea00b1d70 0x7f5ea014a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:15.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.958+0000 7f5ea5a54700 1 --2- 172.21.15.162:0/709356285 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5ea00b11b0 0x7f5ea0140d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:15.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.958+0000 7f5ea6255700 1 -- 172.21.15.162:0/709356285 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5ea00b11b0 msgr2=0x7f5ea0140d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:15.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.958+0000 7f5ea6255700 1 --2- 172.21.15.162:0/709356285 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5ea00b11b0 0x7f5ea0140d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:15.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.958+0000 7f5ea6255700 1 -- 172.21.15.162:0/709356285 >> v2:172.21.15.118:3300/0 conn(0x7f5ea00b1d70 msgr2=0x7f5ea014a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:15.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.958+0000 7f5ea6255700 1 --2- 172.21.15.162:0/709356285 >> v2:172.21.15.118:3300/0 conn(0x7f5ea00b1d70 0x7f5ea014a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:15.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.958+0000 7f5ea6255700 1 -- 172.21.15.162:0/709356285 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea00b0740 con 0x7f5ea00b1990 2020-11-28T21:52:15.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.958+0000 7f5ea6255700 1 --2- 172.21.15.162:0/709356285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ea00b1990 0x7f5ea0140890 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7f5e980032b0 tx=0x7f5e98009270).ready entity=mon.1 client_cookie=2a4c887a63f6d199 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:15.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.958+0000 7f5ea5253700 1 -- 172.21.15.162:0/709356285 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e98009e60 con 0x7f5ea00b1990 2020-11-28T21:52:15.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.958+0000 7f5ea5253700 1 -- 172.21.15.162:0/709356285 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e9800bb50 con 0x7f5ea00b1990 2020-11-28T21:52:15.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.958+0000 7f5ea5253700 1 -- 172.21.15.162:0/709356285 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e9800bcc0 con 0x7f5ea00b1990 2020-11-28T21:52:15.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.958+0000 7f5ea84b9700 1 -- 172.21.15.162:0/709356285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ea00b1990 msgr2=0x7f5ea0140890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:15.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.958+0000 7f5ea84b9700 1 --2- 172.21.15.162:0/709356285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ea00b1990 0x7f5ea0140890 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7f5e980032b0 tx=0x7f5e98009270).stop 2020-11-28T21:52:15.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.962+0000 7f5ea84b9700 1 -- 172.21.15.162:0/709356285 shutdown_connections 2020-11-28T21:52:15.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.962+0000 7f5ea84b9700 1 --2- 172.21.15.162:0/709356285 >> v2:172.21.15.118:3300/0 conn(0x7f5ea00b1d70 0x7f5ea014a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:15.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.962+0000 7f5ea84b9700 1 --2- 172.21.15.162:0/709356285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ea00b1990 0x7f5ea0140890 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:15.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.962+0000 7f5ea84b9700 1 --2- 172.21.15.162:0/709356285 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5ea00b11b0 0x7f5ea0140d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:15.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.962+0000 7f5ea84b9700 1 -- 172.21.15.162:0/709356285 >> 172.21.15.162:0/709356285 conn(0x7f5ea005b970 msgr2=0x7f5ea005bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:15.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.962+0000 7f5ea84b9700 1 -- 172.21.15.162:0/709356285 shutdown_connections 2020-11-28T21:52:15.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.962+0000 7f5ea84b9700 1 -- 172.21.15.162:0/709356285 wait complete. 2020-11-28T21:52:15.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.962+0000 7f5ea84b9700 1 Processor -- start 2020-11-28T21:52:15.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.962+0000 7f5ea84b9700 1 -- start start 2020-11-28T21:52:15.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.962+0000 7f5ea84b9700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5ea00b11b0 0x7f5ea00ae540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:15.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.962+0000 7f5ea84b9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ea00b1990 0x7f5ea0159080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:15.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.962+0000 7f5ea84b9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5ea00b1d70 0x7f5ea0159570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:15.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.962+0000 7f5ea84b9700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea0144500 con 0x7f5ea00b1d70 2020-11-28T21:52:15.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.962+0000 7f5ea84b9700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea00b4e00 con 0x7f5ea00b1990 2020-11-28T21:52:15.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.962+0000 7f5ea84b9700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea00b5ab0 con 0x7f5ea00b11b0 2020-11-28T21:52:15.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.962+0000 7f5ea5a54700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ea00b1990 0x7f5ea0159080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:15.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.962+0000 7f5ea6255700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5ea00b11b0 0x7f5ea00ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:15.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.962+0000 7f5ea5a54700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ea00b1990 0x7f5ea0159080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41016/0 (socket says 172.21.15.162:41016) 2020-11-28T21:52:15.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.962+0000 7f5ea5a54700 1 -- 172.21.15.162:0/1551435818 learned_addr learned my addr 172.21.15.162:0/1551435818 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:15.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.962+0000 7f5ea6a56700 1 --2- 172.21.15.162:0/1551435818 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5ea00b1d70 0x7f5ea0159570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:15.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.962+0000 7f5ea5a54700 1 -- 172.21.15.162:0/1551435818 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5ea00b11b0 msgr2=0x7f5ea00ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:15.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.962+0000 7f5ea5a54700 1 --2- 172.21.15.162:0/1551435818 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5ea00b11b0 0x7f5ea00ae540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:15.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.962+0000 7f5ea5a54700 1 -- 172.21.15.162:0/1551435818 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5ea00b1d70 msgr2=0x7f5ea0159570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:15.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.962+0000 7f5ea5a54700 1 --2- 172.21.15.162:0/1551435818 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5ea00b1d70 0x7f5ea0159570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:15.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.962+0000 7f5ea5a54700 1 -- 172.21.15.162:0/1551435818 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea0159a60 con 0x7f5ea00b1990 2020-11-28T21:52:15.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.962+0000 7f5ea5a54700 1 --2- 172.21.15.162:0/1551435818 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ea00b1990 0x7f5ea0159080 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7f5e9000c000 tx=0x7f5e90006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:15.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.966+0000 7f5e9f7fe700 1 -- 172.21.15.162:0/1551435818 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e9000ca10 con 0x7f5ea00b1990 2020-11-28T21:52:15.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.966+0000 7f5e9f7fe700 1 -- 172.21.15.162:0/1551435818 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e9000cb70 con 0x7f5ea00b1990 2020-11-28T21:52:15.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.966+0000 7f5ea84b9700 1 -- 172.21.15.162:0/1551435818 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ea00b5810 con 0x7f5ea00b1990 2020-11-28T21:52:15.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.966+0000 7f5e9f7fe700 1 -- 172.21.15.162:0/1551435818 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e900123f0 con 0x7f5ea00b1990 2020-11-28T21:52:15.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.966+0000 7f5ea84b9700 1 -- 172.21.15.162:0/1551435818 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea00b5260 con 0x7f5ea00b1990 2020-11-28T21:52:15.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.966+0000 7f5ea84b9700 1 -- 172.21.15.162:0/1551435818 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ea00b4f70 con 0x7f5ea00b1990 2020-11-28T21:52:15.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.966+0000 7f5e9f7fe700 1 -- 172.21.15.162:0/1551435818 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5e90012550 con 0x7f5ea00b1990 2020-11-28T21:52:15.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.966+0000 7f5e9f7fe700 1 --2- 172.21.15.162:0/1551435818 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5e84055330 0x7f5e840577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:15.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.966+0000 7f5e9f7fe700 1 -- 172.21.15.162:0/1551435818 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5e9006e3d0 con 0x7f5ea00b1990 2020-11-28T21:52:15.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.966+0000 7f5ea6255700 1 --2- 172.21.15.162:0/1551435818 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5e84055330 0x7f5e840577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:15.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.966+0000 7f5ea6255700 1 --2- 172.21.15.162:0/1551435818 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5e84055330 0x7f5e840577f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f5e9801a000 tx=0x7f5e98016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:15.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:15.974+0000 7f5e9f7fe700 1 -- 172.21.15.162:0/1551435818 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5e90044670 con 0x7f5ea00b1990 2020-11-28T21:52:16.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.166+0000 7f5ea84b9700 1 -- 172.21.15.162:0/1551435818 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder"]} v 0) v1 -- 0x7f5ea0057050 con 0x7f5ea00b1990 2020-11-28T21:52:16.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.830+0000 7f5e9f7fe700 1 -- 172.21.15.162:0/1551435818 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder"]}]=0 v246) v1 ==== 142+0+0 (secure 0 0 0) 0x7f5e90047c90 con 0x7f5ea00b1990 2020-11-28T21:52:16.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.834+0000 7f5ea84b9700 1 -- 172.21.15.162:0/1551435818 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5e84055330 msgr2=0x7f5e840577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:16.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.834+0000 7f5ea84b9700 1 --2- 172.21.15.162:0/1551435818 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5e84055330 0x7f5e840577f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f5e9801a000 tx=0x7f5e98016000).stop 2020-11-28T21:52:16.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.834+0000 7f5ea84b9700 1 -- 172.21.15.162:0/1551435818 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ea00b1990 msgr2=0x7f5ea0159080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:16.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.834+0000 7f5ea84b9700 1 --2- 172.21.15.162:0/1551435818 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ea00b1990 0x7f5ea0159080 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7f5e9000c000 tx=0x7f5e90006dc0).stop 2020-11-28T21:52:16.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.834+0000 7f5ea84b9700 1 -- 172.21.15.162:0/1551435818 shutdown_connections 2020-11-28T21:52:16.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.834+0000 7f5ea84b9700 1 --2- 172.21.15.162:0/1551435818 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5e84055330 0x7f5e840577f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:16.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.834+0000 7f5ea84b9700 1 --2- 172.21.15.162:0/1551435818 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5ea00b11b0 0x7f5ea00ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:16.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.834+0000 7f5ea84b9700 1 --2- 172.21.15.162:0/1551435818 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5ea00b1d70 0x7f5ea0159570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:16.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.834+0000 7f5ea84b9700 1 --2- 172.21.15.162:0/1551435818 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ea00b1990 0x7f5ea0159080 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:16.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.834+0000 7f5ea84b9700 1 -- 172.21.15.162:0/1551435818 >> 172.21.15.162:0/1551435818 conn(0x7f5ea005b970 msgr2=0x7f5ea0144cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:16.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.834+0000 7f5ea84b9700 1 -- 172.21.15.162:0/1551435818 shutdown_connections 2020-11-28T21:52:16.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.834+0000 7f5ea84b9700 1 -- 172.21.15.162:0/1551435818 wait complete. 2020-11-28T21:52:16.854 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config set mon mon_allow_pool_delete True 2020-11-28T21:52:16.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.974+0000 7fb7109a7700 1 Processor -- start 2020-11-28T21:52:16.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.974+0000 7fb7109a7700 1 -- start start 2020-11-28T21:52:16.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.978+0000 7fb7109a7700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb70c0afec0 0x7fb70c0b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:16.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.978+0000 7fb7109a7700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb70c0a8df0 0x7fb70c099620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:16.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.978+0000 7fb7109a7700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb70c060500 con 0x7fb70c099b10 2020-11-28T21:52:16.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.978+0000 7fb7109a7700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb70c060ce0 con 0x7fb70c0a8df0 2020-11-28T21:52:16.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.978+0000 7fb7109a7700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c0607a0 con 0x7fb70c0afec0 2020-11-28T21:52:16.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.978+0000 7fb70259c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb70c0a8df0 0x7fb70c099620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:16.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.978+0000 7fb70259c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb70c0a8df0 0x7fb70c099620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41024/0 (socket says 172.21.15.162:41024) 2020-11-28T21:52:16.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.978+0000 7fb70259c700 1 -- 172.21.15.162:0/92856715 learned_addr learned my addr 172.21.15.162:0/92856715 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:16.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.978+0000 7fb70ad9d700 1 --2- 172.21.15.162:0/92856715 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb70c0afec0 0x7fb70c0b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:16.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.978+0000 7fb70a59c700 1 -- 172.21.15.162:0/92856715 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3382748755 0 0) 0x7fb70c060500 con 0x7fb70c099b10 2020-11-28T21:52:16.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.978+0000 7fb70a59c700 1 -- 172.21.15.162:0/92856715 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6ec003230 con 0x7fb70c099b10 2020-11-28T21:52:16.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.978+0000 7fb70259c700 1 -- 172.21.15.162:0/92856715 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb70c0afec0 msgr2=0x7fb70c0b02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:16.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.978+0000 7fb70259c700 1 --2- 172.21.15.162:0/92856715 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb70c0afec0 0x7fb70c0b02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:16.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.978+0000 7fb70259c700 1 -- 172.21.15.162:0/92856715 >> v1:172.21.15.118:6789/0 conn(0x7fb70c099b10 legacy=0x7fb70c05b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:16.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.978+0000 7fb70259c700 1 -- 172.21.15.162:0/92856715 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb70c060a40 con 0x7fb70c0a8df0 2020-11-28T21:52:16.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.978+0000 7fb70ad9d700 1 --2- 172.21.15.162:0/92856715 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb70c0afec0 0x7fb70c0b02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:52:16.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.978+0000 7fb70259c700 1 --2- 172.21.15.162:0/92856715 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb70c0a8df0 0x7fb70c099620 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7fb6f4002f30 tx=0x7fb6f4006e80).ready entity=mon.1 client_cookie=b7c3ddb0d91271da server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:16.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.978+0000 7fb70a59c700 1 -- 172.21.15.162:0/92856715 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6f4009640 con 0x7fb70c0a8df0 2020-11-28T21:52:16.992 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.978+0000 7fb70a59c700 1 -- 172.21.15.162:0/92856715 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6f40097a0 con 0x7fb70c0a8df0 2020-11-28T21:52:16.992 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.978+0000 7fb70a59c700 1 -- 172.21.15.162:0/92856715 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6f4009b00 con 0x7fb70c0a8df0 2020-11-28T21:52:16.992 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.978+0000 7fb7109a7700 1 -- 172.21.15.162:0/92856715 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb70c0a8df0 msgr2=0x7fb70c099620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:16.993 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.978+0000 7fb7109a7700 1 --2- 172.21.15.162:0/92856715 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb70c0a8df0 0x7fb70c099620 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7fb6f4002f30 tx=0x7fb6f4006e80).stop 2020-11-28T21:52:16.993 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.982+0000 7fb7109a7700 1 -- 172.21.15.162:0/92856715 shutdown_connections 2020-11-28T21:52:16.993 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.982+0000 7fb7109a7700 1 --2- 172.21.15.162:0/92856715 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb70c0afec0 0x7fb70c0b02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:16.994 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.982+0000 7fb7109a7700 1 --2- 172.21.15.162:0/92856715 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb70c0a8df0 0x7fb70c099620 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:16.994 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.982+0000 7fb7109a7700 1 -- 172.21.15.162:0/92856715 >> 172.21.15.162:0/92856715 conn(0x7fb70c0b5520 msgr2=0x7fb70c0b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:16.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.982+0000 7fb7109a7700 1 -- 172.21.15.162:0/92856715 shutdown_connections 2020-11-28T21:52:16.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.982+0000 7fb7109a7700 1 -- 172.21.15.162:0/92856715 wait complete. 2020-11-28T21:52:17.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.982+0000 7fb7109a7700 1 Processor -- start 2020-11-28T21:52:17.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.982+0000 7fb7109a7700 1 -- start start 2020-11-28T21:52:17.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.986+0000 7fb7109a7700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb70c099b10 0x7fb70c0933c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:17.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.986+0000 7fb7109a7700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb70c0a8df0 0x7fb70c0938b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:17.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.986+0000 7fb7109a7700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb70c0afec0 0x7fb70c0972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:17.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.986+0000 7fb7109a7700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c093da0 con 0x7fb70c0a8df0 2020-11-28T21:52:17.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.986+0000 7fb7109a7700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c153300 con 0x7fb70c099b10 2020-11-28T21:52:17.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.986+0000 7fb7109a7700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c08ff00 con 0x7fb70c0afec0 2020-11-28T21:52:17.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.986+0000 7fb70ad9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb70c099b10 0x7fb70c0933c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:17.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.986+0000 7fb70ad9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb70c099b10 0x7fb70c0933c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41028/0 (socket says 172.21.15.162:41028) 2020-11-28T21:52:17.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.986+0000 7fb70ad9d700 1 -- 172.21.15.162:0/1120804877 learned_addr learned my addr 172.21.15.162:0/1120804877 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:17.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.986+0000 7fb70b59e700 1 --2- 172.21.15.162:0/1120804877 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb70c0afec0 0x7fb70c0972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:17.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.986+0000 7fb70259c700 1 --2- 172.21.15.162:0/1120804877 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb70c0a8df0 0x7fb70c0938b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:17.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.986+0000 7fb70ad9d700 1 -- 172.21.15.162:0/1120804877 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb70c0afec0 msgr2=0x7fb70c0972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:17.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.986+0000 7fb70ad9d700 1 --2- 172.21.15.162:0/1120804877 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb70c0afec0 0x7fb70c0972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:17.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.986+0000 7fb70ad9d700 1 -- 172.21.15.162:0/1120804877 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb70c0a8df0 msgr2=0x7fb70c0938b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:17.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.986+0000 7fb70ad9d700 1 --2- 172.21.15.162:0/1120804877 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb70c0a8df0 0x7fb70c0938b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:17.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.986+0000 7fb70ad9d700 1 -- 172.21.15.162:0/1120804877 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb70c08f910 con 0x7fb70c099b10 2020-11-28T21:52:17.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.986+0000 7fb70b59e700 1 --2- 172.21.15.162:0/1120804877 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb70c0afec0 0x7fb70c0972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:52:17.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.986+0000 7fb70ad9d700 1 --2- 172.21.15.162:0/1120804877 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb70c099b10 0x7fb70c0933c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7fb6fc00c000 tx=0x7fb6fc006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:17.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.986+0000 7fb708d99700 1 -- 172.21.15.162:0/1120804877 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6fc00cb90 con 0x7fb70c099b10 2020-11-28T21:52:17.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.986+0000 7fb708d99700 1 -- 172.21.15.162:0/1120804877 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6fc00ccf0 con 0x7fb70c099b10 2020-11-28T21:52:17.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.986+0000 7fb7109a7700 1 -- 172.21.15.162:0/1120804877 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb70c153530 con 0x7fb70c099b10 2020-11-28T21:52:17.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.986+0000 7fb708d99700 1 -- 172.21.15.162:0/1120804877 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6fc0125e0 con 0x7fb70c099b10 2020-11-28T21:52:17.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.986+0000 7fb7109a7700 1 -- 172.21.15.162:0/1120804877 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb70c0af8e0 con 0x7fb70c099b10 2020-11-28T21:52:17.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.990+0000 7fb7109a7700 1 -- 172.21.15.162:0/1120804877 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb70c1536c0 con 0x7fb70c099b10 2020-11-28T21:52:17.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.990+0000 7fb708d99700 1 -- 172.21.15.162:0/1120804877 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb6fc012740 con 0x7fb70c099b10 2020-11-28T21:52:17.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.990+0000 7fb708d99700 1 --2- 172.21.15.162:0/1120804877 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb6f0055330 0x7fb6f00577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:17.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.990+0000 7fb708d99700 1 -- 172.21.15.162:0/1120804877 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb6fc06e4d0 con 0x7fb70c099b10 2020-11-28T21:52:17.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.990+0000 7fb70259c700 1 --2- 172.21.15.162:0/1120804877 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb6f0055330 0x7fb6f00577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:17.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.990+0000 7fb70259c700 1 --2- 172.21.15.162:0/1120804877 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb6f0055330 0x7fb6f00577f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7fb6f4019000 tx=0x7fb6f4015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:17.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:16.994+0000 7fb708d99700 1 -- 172.21.15.162:0/1120804877 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb6fc0447d0 con 0x7fb70c099b10 2020-11-28T21:52:17.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.134+0000 7fb7109a7700 1 -- 172.21.15.162:0/1120804877 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7fb70c057050 con 0x7fb70c099b10 2020-11-28T21:52:17.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.142+0000 7fb708d99700 1 -- 172.21.15.162:0/1120804877 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb6fc00ce60 con 0x7fb70c099b10 2020-11-28T21:52:17.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.146+0000 7fb7109a7700 1 -- 172.21.15.162:0/1120804877 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb6f0055330 msgr2=0x7fb6f00577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:17.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.146+0000 7fb7109a7700 1 --2- 172.21.15.162:0/1120804877 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb6f0055330 0x7fb6f00577f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7fb6f4019000 tx=0x7fb6f4015000).stop 2020-11-28T21:52:17.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.146+0000 7fb7109a7700 1 -- 172.21.15.162:0/1120804877 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb70c099b10 msgr2=0x7fb70c0933c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:17.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.146+0000 7fb7109a7700 1 --2- 172.21.15.162:0/1120804877 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb70c099b10 0x7fb70c0933c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7fb6fc00c000 tx=0x7fb6fc006dc0).stop 2020-11-28T21:52:17.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.146+0000 7fb7109a7700 1 -- 172.21.15.162:0/1120804877 shutdown_connections 2020-11-28T21:52:17.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.146+0000 7fb7109a7700 1 --2- 172.21.15.162:0/1120804877 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb6f0055330 0x7fb6f00577f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:17.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.146+0000 7fb7109a7700 1 --2- 172.21.15.162:0/1120804877 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb70c0afec0 0x7fb70c0972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:17.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.146+0000 7fb7109a7700 1 --2- 172.21.15.162:0/1120804877 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb70c099b10 0x7fb70c0933c0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:17.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.146+0000 7fb7109a7700 1 --2- 172.21.15.162:0/1120804877 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb70c0a8df0 0x7fb70c0938b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:17.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.146+0000 7fb7109a7700 1 -- 172.21.15.162:0/1120804877 >> 172.21.15.162:0/1120804877 conn(0x7fb70c0b5520 msgr2=0x7fb70c06a280 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:17.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.146+0000 7fb7109a7700 1 -- 172.21.15.162:0/1120804877 shutdown_connections 2020-11-28T21:52:17.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.146+0000 7fb7109a7700 1 -- 172.21.15.162:0/1120804877 wait complete. 2020-11-28T21:52:17.160 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-28T21:52:17.161 INFO:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:52:17.181 INFO:teuthology.orchestra.run.smithi118:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:52:17.201 INFO:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:52:17.218 INFO:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:52:17.237 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-28T21:52:17.238 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:52:17.239 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:52:17.239 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:52:17.288 INFO:teuthology.orchestra.run.smithi192.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2020-11-28T21:52:17.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:52:17.290 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2020-11-28T21:52:17.291 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:52:17.293 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:52:17.293 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:52:17.302 INFO:teuthology.orchestra.run.smithi096.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-11-28T21:52:17.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:52:17.303 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2020-11-28T21:52:17.304 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2020-11-28T21:52:17.422 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.418+0000 7fb00bfff700 1 Processor -- start 2020-11-28T21:52:17.422 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.418+0000 7fb00bfff700 1 -- start start 2020-11-28T21:52:17.427 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.422+0000 7fb00bfff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fb00c0b01a0 0x7fb00c143f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:17.428 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.422+0000 7fb00bfff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb00c060010 0x7fb00c144450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:17.428 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.422+0000 7fb00bfff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb00c143ba0 con 0x7fb00c09abb0 2020-11-28T21:52:17.428 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.422+0000 7fb00bfff700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb00c0618a0 con 0x7fb00c0b01a0 2020-11-28T21:52:17.428 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.422+0000 7fb00bfff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c09af90 con 0x7fb00c060010 2020-11-28T21:52:17.429 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.422+0000 7fb00b7fe700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb00c060010 0x7fb00c144450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:17.429 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.422+0000 7fb0027fc700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fb00c0b01a0 0x7fb00c143f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:17.429 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.422+0000 7fb00b7fe700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb00c060010 0x7fb00c144450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:38262/0 (socket says 172.21.15.162:38262) 2020-11-28T21:52:17.429 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.422+0000 7fb00b7fe700 1 -- 172.21.15.162:0/1264114745 learned_addr learned my addr 172.21.15.162:0/1264114745 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:17.429 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.422+0000 7fb0027fc700 1 -- 172.21.15.162:0/1264114745 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb00c060010 msgr2=0x7fb00c144450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:17.430 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.422+0000 7fb0027fc700 1 --2- 172.21.15.162:0/1264114745 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb00c060010 0x7fb00c144450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:17.430 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.422+0000 7fb0027fc700 1 -- 172.21.15.162:0/1264114745 >> v1:172.21.15.118:6789/0 conn(0x7fb00c09abb0 legacy=0x7fb00c061190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:17.430 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.422+0000 7fb0027fc700 1 -- 172.21.15.162:0/1264114745 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb00c0b1d10 con 0x7fb00c0b01a0 2020-11-28T21:52:17.430 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.422+0000 7fb0027fc700 1 --2- 172.21.15.162:0/1264114745 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb00c0b01a0 0x7fb00c143f60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7faff4002f30 tx=0x7faff4006dc0).ready entity=mon.0 client_cookie=489e01a9d8878894 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:17.431 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.422+0000 7fb00a7fc700 1 -- 172.21.15.162:0/1264114745 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faff4009760 con 0x7fb00c0b01a0 2020-11-28T21:52:17.431 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.422+0000 7fb00a7fc700 1 -- 172.21.15.162:0/1264114745 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff40098c0 con 0x7fb00c0b01a0 2020-11-28T21:52:17.431 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.422+0000 7fb00a7fc700 1 -- 172.21.15.162:0/1264114745 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faff4009c20 con 0x7fb00c0b01a0 2020-11-28T21:52:17.431 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.422+0000 7fb00bfff700 1 -- 172.21.15.162:0/1264114745 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb00c0b01a0 msgr2=0x7fb00c143f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:17.431 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.422+0000 7fb00bfff700 1 --2- 172.21.15.162:0/1264114745 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb00c0b01a0 0x7fb00c143f60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7faff4002f30 tx=0x7faff4006dc0).stop 2020-11-28T21:52:17.432 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.422+0000 7fb00bfff700 1 -- 172.21.15.162:0/1264114745 shutdown_connections 2020-11-28T21:52:17.432 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.422+0000 7fb00bfff700 1 --2- 172.21.15.162:0/1264114745 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb00c060010 0x7fb00c144450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:17.433 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.422+0000 7fb00bfff700 1 --2- 172.21.15.162:0/1264114745 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb00c0b01a0 0x7fb00c143f60 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:17.433 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.422+0000 7fb00bfff700 1 -- 172.21.15.162:0/1264114745 >> 172.21.15.162:0/1264114745 conn(0x7fb00c0b54f0 msgr2=0x7fb00c0b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:17.433 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.422+0000 7fb00bfff700 1 -- 172.21.15.162:0/1264114745 shutdown_connections 2020-11-28T21:52:17.433 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.422+0000 7fb00bfff700 1 -- 172.21.15.162:0/1264114745 wait complete. 2020-11-28T21:52:17.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.426+0000 7fb00bfff700 1 Processor -- start 2020-11-28T21:52:17.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.426+0000 7fb00bfff700 1 -- start start 2020-11-28T21:52:17.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.426+0000 7fb00bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb00c060010 0x7fb00c099c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:17.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.426+0000 7fb00bfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb00c09abb0 0x7fb00c0989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:17.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.426+0000 7fb00bfff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb00c0b01a0 0x7fb00c065790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:17.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.426+0000 7fb00bfff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c064730 con 0x7fb00c060010 2020-11-28T21:52:17.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.426+0000 7fb00bfff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c0ae8f0 con 0x7fb00c09abb0 2020-11-28T21:52:17.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.426+0000 7fb00bfff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c098fe0 con 0x7fb00c0b01a0 2020-11-28T21:52:17.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.426+0000 7fb0027fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb00c09abb0 0x7fb00c0989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:17.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.426+0000 7fb0027fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb00c09abb0 0x7fb00c0989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41042/0 (socket says 172.21.15.162:41042) 2020-11-28T21:52:17.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.426+0000 7fb0027fc700 1 -- 172.21.15.162:0/1832098694 learned_addr learned my addr 172.21.15.162:0/1832098694 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:17.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.426+0000 7fb00b7fe700 1 --2- 172.21.15.162:0/1832098694 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb00c0b01a0 0x7fb00c065790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:17.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.426+0000 7fb00affd700 1 --2- 172.21.15.162:0/1832098694 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb00c060010 0x7fb00c099c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:17.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.426+0000 7fb0027fc700 1 -- 172.21.15.162:0/1832098694 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb00c0b01a0 msgr2=0x7fb00c065790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:17.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.426+0000 7fb0027fc700 1 --2- 172.21.15.162:0/1832098694 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb00c0b01a0 0x7fb00c065790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:17.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.426+0000 7fb0027fc700 1 -- 172.21.15.162:0/1832098694 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb00c060010 msgr2=0x7fb00c099c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:17.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.426+0000 7fb0027fc700 1 --2- 172.21.15.162:0/1832098694 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb00c060010 0x7fb00c099c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:17.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.426+0000 7fb0027fc700 1 -- 172.21.15.162:0/1832098694 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb00c099280 con 0x7fb00c09abb0 2020-11-28T21:52:17.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.426+0000 7fb0027fc700 1 --2- 172.21.15.162:0/1832098694 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb00c09abb0 0x7fb00c0989a0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7faff4002f30 tx=0x7faff4015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:17.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.426+0000 7fb008ff9700 1 -- 172.21.15.162:0/1832098694 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faff4011a20 con 0x7fb00c09abb0 2020-11-28T21:52:17.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.426+0000 7fb008ff9700 1 -- 172.21.15.162:0/1832098694 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff4011b80 con 0x7fb00c09abb0 2020-11-28T21:52:17.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.426+0000 7fb00bfff700 1 -- 172.21.15.162:0/1832098694 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb00c099520 con 0x7fb00c09abb0 2020-11-28T21:52:17.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.426+0000 7fb008ff9700 1 -- 172.21.15.162:0/1832098694 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faff401d440 con 0x7fb00c09abb0 2020-11-28T21:52:17.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.426+0000 7fb00bfff700 1 -- 172.21.15.162:0/1832098694 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb00c0af040 con 0x7fb00c09abb0 2020-11-28T21:52:17.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.430+0000 7fb00bfff700 1 -- 172.21.15.162:0/1832098694 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb00c057050 con 0x7fb00c09abb0 2020-11-28T21:52:17.444 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.430+0000 7fb008ff9700 1 -- 172.21.15.162:0/1832098694 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7faff4024030 con 0x7fb00c09abb0 2020-11-28T21:52:17.444 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.430+0000 7fb008ff9700 1 --2- 172.21.15.162:0/1832098694 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7faff0055440 0x7faff0057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:17.444 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.430+0000 7fb008ff9700 1 -- 172.21.15.162:0/1832098694 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6073+0+0 (secure 0 0 0) 0x7faff406ed50 con 0x7fb00c09abb0 2020-11-28T21:52:17.445 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.430+0000 7fb00affd700 1 --2- 172.21.15.162:0/1832098694 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7faff0055440 0x7faff0057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:17.446 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.430+0000 7fb00affd700 1 --2- 172.21.15.162:0/1832098694 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7faff0055440 0x7faff0057900 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7faffc00d000 tx=0x7faffc007b60).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:17.447 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.434+0000 7fb008ff9700 1 -- 172.21.15.162:0/1832098694 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7faff4045050 con 0x7fb00c09abb0 2020-11-28T21:52:17.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.578+0000 7fb00bfff700 1 -- 172.21.15.162:0/1832098694 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fb00c05d5e0 con 0x7fb00c09abb0 2020-11-28T21:52:17.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.582+0000 7fb008ff9700 1 -- 172.21.15.162:0/1832098694 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v66) v1 ==== 81+0+33744 (secure 0 0 0) 0x7faff4022020 con 0x7fb00c09abb0 2020-11-28T21:52:17.584 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:52:17.584 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2020-11-28T21:52:17.585 INFO:teuthology.orchestra.run.smithi162.stdout: "epoch": 66, 2020-11-28T21:52:17.585 INFO:teuthology.orchestra.run.smithi162.stdout: "fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", 2020-11-28T21:52:17.585 INFO:teuthology.orchestra.run.smithi162.stdout: "created": "2020-11-28T21:46:11.724372+0000", 2020-11-28T21:52:17.586 INFO:teuthology.orchestra.run.smithi162.stdout: "modified": "2020-11-28T21:51:45.229278+0000", 2020-11-28T21:52:17.586 INFO:teuthology.orchestra.run.smithi162.stdout: "last_up_change": "2020-11-28T21:46:19.591650+0000", 2020-11-28T21:52:17.586 INFO:teuthology.orchestra.run.smithi162.stdout: "last_in_change": "2020-11-28T21:46:19.591650+0000", 2020-11-28T21:52:17.587 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-11-28T21:52:17.587 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_num": 5799936, 2020-11-28T21:52:17.587 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_set": [ 2020-11-28T21:52:17.588 INFO:teuthology.orchestra.run.smithi162.stdout: "pglog_hardlimit", 2020-11-28T21:52:17.588 INFO:teuthology.orchestra.run.smithi162.stdout: "purged_snapdirs", 2020-11-28T21:52:17.588 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_deletes", 2020-11-28T21:52:17.589 INFO:teuthology.orchestra.run.smithi162.stdout: "sortbitwise" 2020-11-28T21:52:17.589 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:52:17.589 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_version": 4, 2020-11-28T21:52:17.590 INFO:teuthology.orchestra.run.smithi162.stdout: "full_ratio": 0.94999998807907104, 2020-11-28T21:52:17.590 INFO:teuthology.orchestra.run.smithi162.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-11-28T21:52:17.590 INFO:teuthology.orchestra.run.smithi162.stdout: "nearfull_ratio": 0.85000002384185791, 2020-11-28T21:52:17.591 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_snapshot": "", 2020-11-28T21:52:17.591 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_max": 12, 2020-11-28T21:52:17.591 INFO:teuthology.orchestra.run.smithi162.stdout: "max_osd": 8, 2020-11-28T21:52:17.592 INFO:teuthology.orchestra.run.smithi162.stdout: "require_min_compat_client": "jewel", 2020-11-28T21:52:17.592 INFO:teuthology.orchestra.run.smithi162.stdout: "min_compat_client": "jewel", 2020-11-28T21:52:17.592 INFO:teuthology.orchestra.run.smithi162.stdout: "require_osd_release": "octopus", 2020-11-28T21:52:17.593 INFO:teuthology.orchestra.run.smithi162.stdout: "pools": [ 2020-11-28T21:52:17.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.593 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 1, 2020-11-28T21:52:17.594 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "device_health_metrics", 2020-11-28T21:52:17.594 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:46:15.639551+0000", 2020-11-28T21:52:17.594 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:52:17.595 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:52:17.595 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:52:17.595 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:52:17.596 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:52:17.596 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:52:17.596 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:52:17.597 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:52:17.598 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 1, 2020-11-28T21:52:17.598 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 1, 2020-11-28T21:52:17.599 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 1, 2020-11-28T21:52:17.599 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 1, 2020-11-28T21:52:17.599 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 1, 2020-11-28T21:52:17.600 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:52:17.600 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:52:17.601 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:52:17.601 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:52:17.601 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:52:17.602 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:52:17.602 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:52:17.603 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.603 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "16", 2020-11-28T21:52:17.603 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:52:17.604 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:52:17.604 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:52:17.605 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:52:17.605 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:52:17.605 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:52:17.606 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:52:17.606 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:52:17.607 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:52:17.607 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:52:17.608 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:52:17.608 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:52:17.609 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:52:17.609 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:52:17.609 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:52:17.610 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:52:17.610 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:52:17.610 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:52:17.610 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:52:17.611 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:52:17.611 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:52:17.611 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:52:17.612 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:52:17.612 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:52:17.612 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:52:17.613 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:52:17.613 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.613 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:52:17.614 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:52:17.614 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:52:17.614 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:52:17.614 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:52:17.615 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:52:17.615 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:52:17.615 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:52:17.615 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:52:17.615 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:52:17.616 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:52:17.616 INFO:teuthology.orchestra.run.smithi162.stdout: "options": { 2020-11-28T21:52:17.616 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_min": 1 2020-11-28T21:52:17.616 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.617 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:52:17.617 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr_devicehealth": {} 2020-11-28T21:52:17.617 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:17.618 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.618 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.619 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 2, 2020-11-28T21:52:17.619 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "rbd", 2020-11-28T21:52:17.619 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:46:23.901366+0000", 2020-11-28T21:52:17.620 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:52:17.620 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:52:17.620 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:52:17.621 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:52:17.621 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:52:17.621 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:52:17.622 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:52:17.622 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:52:17.622 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2020-11-28T21:52:17.622 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2020-11-28T21:52:17.623 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2020-11-28T21:52:17.623 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2020-11-28T21:52:17.623 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2020-11-28T21:52:17.623 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:52:17.623 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:52:17.624 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:52:17.624 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:52:17.624 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:52:17.624 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:52:17.625 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:52:17.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.625 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "19", 2020-11-28T21:52:17.625 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:52:17.626 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:52:17.626 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:52:17.626 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:52:17.626 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:52:17.626 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:52:17.627 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:52:17.627 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:52:17.627 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:52:17.628 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:52:17.628 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:52:17.628 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:52:17.628 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:52:17.629 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:52:17.629 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:52:17.629 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:52:17.629 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:52:17.630 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:52:17.630 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:52:17.630 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:52:17.630 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:52:17.630 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:52:17.631 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:52:17.631 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:52:17.631 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:52:17.631 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:52:17.632 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.632 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:52:17.632 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:52:17.632 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:52:17.632 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:52:17.633 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:52:17.633 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:52:17.633 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:52:17.633 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:52:17.634 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:52:17.634 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:52:17.634 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:52:17.634 INFO:teuthology.orchestra.run.smithi162.stdout: "options": {}, 2020-11-28T21:52:17.634 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:52:17.635 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd": {} 2020-11-28T21:52:17.635 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:17.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.635 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 11, 2020-11-28T21:52:17.636 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "cephfs_metadata", 2020-11-28T21:52:17.636 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:51:41.931840+0000", 2020-11-28T21:52:17.636 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:52:17.637 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:52:17.637 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:52:17.637 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:52:17.638 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:52:17.638 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:52:17.638 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:52:17.639 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:52:17.639 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2020-11-28T21:52:17.639 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2020-11-28T21:52:17.639 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2020-11-28T21:52:17.639 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2020-11-28T21:52:17.640 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2020-11-28T21:52:17.640 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:52:17.640 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:52:17.640 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:52:17.640 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:52:17.641 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:52:17.641 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:52:17.641 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:52:17.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.641 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "65", 2020-11-28T21:52:17.642 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:52:17.642 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:52:17.642 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:52:17.642 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:52:17.643 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:52:17.643 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:52:17.643 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:52:17.644 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:52:17.644 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:52:17.644 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:52:17.644 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:52:17.645 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:52:17.645 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:52:17.645 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:52:17.645 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:52:17.646 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:52:17.646 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:52:17.646 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:52:17.646 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:52:17.646 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:52:17.647 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:52:17.647 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:52:17.647 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:52:17.647 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:52:17.647 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:52:17.648 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:52:17.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.648 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:52:17.648 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:52:17.649 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:52:17.649 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:52:17.649 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:52:17.649 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:52:17.650 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:52:17.650 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:52:17.650 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:52:17.650 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:52:17.651 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:52:17.651 INFO:teuthology.orchestra.run.smithi162.stdout: "options": { 2020-11-28T21:52:17.651 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_bias": 4, 2020-11-28T21:52:17.651 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_min": 16, 2020-11-28T21:52:17.652 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_priority": 5 2020-11-28T21:52:17.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.652 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:52:17.653 INFO:teuthology.orchestra.run.smithi162.stdout: "cephfs": { 2020-11-28T21:52:17.653 INFO:teuthology.orchestra.run.smithi162.stdout: "metadata": "cephfs" 2020-11-28T21:52:17.653 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:17.654 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:17.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.655 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 12, 2020-11-28T21:52:17.655 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "cephfs_data", 2020-11-28T21:52:17.656 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:51:43.225560+0000", 2020-11-28T21:52:17.656 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:52:17.656 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:52:17.657 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:52:17.657 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:52:17.658 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:52:17.658 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:52:17.658 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:52:17.659 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:52:17.659 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2020-11-28T21:52:17.659 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2020-11-28T21:52:17.660 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2020-11-28T21:52:17.660 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2020-11-28T21:52:17.660 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2020-11-28T21:52:17.661 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:52:17.661 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:52:17.661 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:52:17.662 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:52:17.662 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:52:17.662 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:52:17.663 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:52:17.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.663 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "66", 2020-11-28T21:52:17.664 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:52:17.664 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:52:17.665 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:52:17.665 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:52:17.665 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:52:17.666 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:52:17.666 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:52:17.667 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:52:17.667 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:52:17.667 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:52:17.668 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:52:17.668 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:52:17.669 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:52:17.669 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:52:17.669 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:52:17.670 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:52:17.670 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:52:17.670 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:52:17.671 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:52:17.671 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:52:17.671 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:52:17.672 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:52:17.672 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:52:17.672 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:52:17.672 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:52:17.673 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:52:17.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.673 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:52:17.674 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:52:17.674 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:52:17.674 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:52:17.675 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:52:17.675 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:52:17.675 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:52:17.676 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:52:17.676 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:52:17.676 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:52:17.677 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:52:17.677 INFO:teuthology.orchestra.run.smithi162.stdout: "options": {}, 2020-11-28T21:52:17.677 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:52:17.677 INFO:teuthology.orchestra.run.smithi162.stdout: "cephfs": { 2020-11-28T21:52:17.677 INFO:teuthology.orchestra.run.smithi162.stdout: "data": "cephfs" 2020-11-28T21:52:17.678 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:17.678 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:17.678 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:17.679 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:52:17.679 INFO:teuthology.orchestra.run.smithi162.stdout: "osds": [ 2020-11-28T21:52:17.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.680 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 0, 2020-11-28T21:52:17.680 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "99c7d9ae-adfa-4fd7-880a-23e6baf3281b", 2020-11-28T21:52:17.680 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:52:17.681 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:52:17.681 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:52:17.682 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:52:17.682 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:52:17.683 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:52:17.683 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:52:17.684 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 64, 2020-11-28T21:52:17.684 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:52:17.684 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:52:17.685 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:52:17.685 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:52:17.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.686 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:52:17.686 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6810", 2020-11-28T21:52:17.687 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:52:17.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.688 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:52:17.688 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6811", 2020-11-28T21:52:17.689 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:52:17.689 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:17.690 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:52:17.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.691 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:52:17.691 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:52:17.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.692 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:52:17.692 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6812", 2020-11-28T21:52:17.693 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:52:17.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.694 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:52:17.694 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6813", 2020-11-28T21:52:17.695 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:52:17.695 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:17.695 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:52:17.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.696 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:52:17.697 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:52:17.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.697 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:52:17.698 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6816", 2020-11-28T21:52:17.698 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:52:17.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.699 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:52:17.700 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6817", 2020-11-28T21:52:17.700 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:52:17.700 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:17.701 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:52:17.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.701 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:52:17.702 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:52:17.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.703 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:52:17.703 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6814", 2020-11-28T21:52:17.703 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:52:17.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.704 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:52:17.705 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6815", 2020-11-28T21:52:17.705 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:52:17.706 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:17.706 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:52:17.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.707 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6811/12313", 2020-11-28T21:52:17.707 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6813/12313", 2020-11-28T21:52:17.707 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6817/12313", 2020-11-28T21:52:17.708 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6815/12313", 2020-11-28T21:52:17.708 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:52:17.708 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:52:17.709 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:52:17.709 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:52:17.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.710 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 1, 2020-11-28T21:52:17.711 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "6ce6220a-00ef-4d13-abba-56f87636fb5f", 2020-11-28T21:52:17.711 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:52:17.711 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:52:17.712 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:52:17.712 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:52:17.712 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:52:17.713 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:52:17.713 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:52:17.713 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 64, 2020-11-28T21:52:17.714 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:52:17.714 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:52:17.715 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:52:17.715 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:52:17.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.716 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:52:17.716 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6802", 2020-11-28T21:52:17.716 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:52:17.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.717 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:52:17.718 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6803", 2020-11-28T21:52:17.718 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:52:17.718 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:17.719 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:52:17.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.719 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:52:17.720 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:52:17.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.720 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:52:17.721 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6804", 2020-11-28T21:52:17.721 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:52:17.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.722 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:52:17.722 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6805", 2020-11-28T21:52:17.723 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:52:17.723 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:17.724 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:52:17.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.724 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:52:17.724 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:52:17.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.725 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:52:17.725 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6808", 2020-11-28T21:52:17.725 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:52:17.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.726 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:52:17.726 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6809", 2020-11-28T21:52:17.727 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:52:17.727 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:17.727 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:52:17.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.727 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:52:17.728 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:52:17.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.728 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:52:17.728 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6806", 2020-11-28T21:52:17.728 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:52:17.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.731 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:52:17.731 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6807", 2020-11-28T21:52:17.732 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:52:17.732 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:17.732 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:52:17.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.733 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6803/12311", 2020-11-28T21:52:17.733 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6805/12311", 2020-11-28T21:52:17.733 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6809/12311", 2020-11-28T21:52:17.734 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6807/12311", 2020-11-28T21:52:17.734 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:52:17.734 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:52:17.734 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:52:17.735 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:52:17.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.736 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 2, 2020-11-28T21:52:17.736 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "db4e510d-27c1-47fc-a902-8b32e7dfd73e", 2020-11-28T21:52:17.737 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:52:17.737 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:52:17.737 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:52:17.738 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:52:17.738 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:52:17.739 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:52:17.739 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:52:17.739 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 64, 2020-11-28T21:52:17.740 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:52:17.740 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:52:17.741 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:52:17.741 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:52:17.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.742 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:52:17.742 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6818", 2020-11-28T21:52:17.743 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:52:17.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.744 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:52:17.744 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6819", 2020-11-28T21:52:17.745 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:52:17.745 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:17.745 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:52:17.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.746 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:52:17.747 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:52:17.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.747 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:52:17.748 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6820", 2020-11-28T21:52:17.748 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:52:17.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.749 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:52:17.750 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6821", 2020-11-28T21:52:17.750 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:52:17.751 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:17.751 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:52:17.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.752 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:52:17.752 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:52:17.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.753 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:52:17.753 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6824", 2020-11-28T21:52:17.754 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:52:17.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.755 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:52:17.755 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6825", 2020-11-28T21:52:17.756 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:52:17.756 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:17.757 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:52:17.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.757 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:52:17.758 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:52:17.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.758 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:52:17.759 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6822", 2020-11-28T21:52:17.759 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:52:17.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.760 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:52:17.761 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6823", 2020-11-28T21:52:17.761 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:52:17.761 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:17.762 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:52:17.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.762 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6819/12312", 2020-11-28T21:52:17.762 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6821/12312", 2020-11-28T21:52:17.762 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6825/12312", 2020-11-28T21:52:17.763 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6823/12312", 2020-11-28T21:52:17.763 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:52:17.763 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:52:17.763 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:52:17.763 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:52:17.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.764 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 3, 2020-11-28T21:52:17.764 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "4094cb87-f04a-45e1-8710-fb4b651949f6", 2020-11-28T21:52:17.764 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:52:17.765 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:52:17.765 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:52:17.765 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:52:17.765 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:52:17.765 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:52:17.766 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:52:17.766 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 63, 2020-11-28T21:52:17.766 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:52:17.766 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:52:17.766 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:52:17.767 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:52:17.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.767 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:52:17.767 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6826", 2020-11-28T21:52:17.767 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:52:17.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.768 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:52:17.768 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6827", 2020-11-28T21:52:17.768 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:52:17.769 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:17.769 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:52:17.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.769 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:52:17.769 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:52:17.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.770 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:52:17.770 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6828", 2020-11-28T21:52:17.770 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:52:17.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.771 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:52:17.771 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6829", 2020-11-28T21:52:17.771 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:52:17.771 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:17.772 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:52:17.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.772 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:52:17.772 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:52:17.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.773 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:52:17.773 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6832", 2020-11-28T21:52:17.773 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:52:17.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.774 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:52:17.774 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6833", 2020-11-28T21:52:17.774 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:52:17.774 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:17.774 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:52:17.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.775 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:52:17.775 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:52:17.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.775 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:52:17.776 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6830", 2020-11-28T21:52:17.776 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:52:17.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.776 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:52:17.777 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6831", 2020-11-28T21:52:17.777 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:52:17.777 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:17.777 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:52:17.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.778 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6827/12323", 2020-11-28T21:52:17.778 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6829/12323", 2020-11-28T21:52:17.778 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6833/12323", 2020-11-28T21:52:17.778 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6831/12323", 2020-11-28T21:52:17.778 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:52:17.779 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:52:17.779 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:52:17.779 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:52:17.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.780 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 4, 2020-11-28T21:52:17.780 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "b907bf53-c297-4756-847b-3be96680eef5", 2020-11-28T21:52:17.780 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:52:17.781 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:52:17.781 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:52:17.781 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:52:17.781 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:52:17.781 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:52:17.782 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:52:17.782 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 64, 2020-11-28T21:52:17.782 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:52:17.782 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:52:17.782 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:52:17.783 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:52:17.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.783 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:52:17.783 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6800", 2020-11-28T21:52:17.783 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:52:17.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.784 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:52:17.784 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6801", 2020-11-28T21:52:17.784 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:52:17.785 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:17.785 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:52:17.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.785 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:52:17.785 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:52:17.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.786 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:52:17.786 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6802", 2020-11-28T21:52:17.786 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:52:17.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.787 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:52:17.787 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6803", 2020-11-28T21:52:17.787 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:52:17.787 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:17.788 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:52:17.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.788 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:52:17.788 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:52:17.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.789 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:52:17.789 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6806", 2020-11-28T21:52:17.789 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:52:17.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.790 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:52:17.790 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6807", 2020-11-28T21:52:17.790 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:52:17.790 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:17.790 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:52:17.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.791 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:52:17.791 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:52:17.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.791 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:52:17.792 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6804", 2020-11-28T21:52:17.792 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:52:17.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.792 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:52:17.793 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6805", 2020-11-28T21:52:17.793 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:52:17.793 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:17.793 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:52:17.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.794 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6801/12527", 2020-11-28T21:52:17.794 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6803/12527", 2020-11-28T21:52:17.794 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6807/12527", 2020-11-28T21:52:17.794 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6805/12527", 2020-11-28T21:52:17.794 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:52:17.795 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:52:17.795 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:52:17.795 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:52:17.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.796 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 5, 2020-11-28T21:52:17.796 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "55ba305d-0505-4212-aebb-631e8554b440", 2020-11-28T21:52:17.796 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:52:17.796 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:52:17.796 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:52:17.797 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:52:17.797 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:52:17.797 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:52:17.797 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:52:17.797 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 63, 2020-11-28T21:52:17.798 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:52:17.798 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:52:17.798 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:52:17.798 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:52:17.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.799 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:52:17.799 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6808", 2020-11-28T21:52:17.799 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:52:17.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.800 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:52:17.800 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6809", 2020-11-28T21:52:17.800 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:52:17.800 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:17.800 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:52:17.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.801 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:52:17.801 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:52:17.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.801 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:52:17.802 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6810", 2020-11-28T21:52:17.802 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:52:17.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.802 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:52:17.803 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6811", 2020-11-28T21:52:17.803 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:52:17.803 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:17.803 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:52:17.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.804 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:52:17.804 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:52:17.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.804 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:52:17.804 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6814", 2020-11-28T21:52:17.805 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:52:17.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.805 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:52:17.805 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6815", 2020-11-28T21:52:17.806 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:52:17.806 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:17.806 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:52:17.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.806 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:52:17.807 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:52:17.807 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.807 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:52:17.807 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6812", 2020-11-28T21:52:17.807 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:52:17.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.808 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:52:17.808 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6813", 2020-11-28T21:52:17.808 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:52:17.809 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:17.809 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:52:17.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.809 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6809/12529", 2020-11-28T21:52:17.809 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6811/12529", 2020-11-28T21:52:17.810 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6815/12529", 2020-11-28T21:52:17.810 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6813/12529", 2020-11-28T21:52:17.810 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:52:17.810 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:52:17.810 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:52:17.811 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:52:17.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.811 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 6, 2020-11-28T21:52:17.811 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "9ada6051-493b-46b5-ba22-1b401440490f", 2020-11-28T21:52:17.812 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:52:17.812 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:52:17.812 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:52:17.812 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:52:17.812 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:52:17.813 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:52:17.813 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:52:17.813 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 64, 2020-11-28T21:52:17.813 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:52:17.813 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:52:17.814 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:52:17.814 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:52:17.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.814 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:52:17.815 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6824", 2020-11-28T21:52:17.815 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:52:17.815 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.815 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.815 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:52:17.816 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6825", 2020-11-28T21:52:17.816 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:52:17.816 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:17.816 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:52:17.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.816 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:52:17.817 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:52:17.817 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.817 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:52:17.817 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6826", 2020-11-28T21:52:17.817 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:52:17.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.818 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.818 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:52:17.818 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6827", 2020-11-28T21:52:17.818 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:52:17.819 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:17.819 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:52:17.819 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.819 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:52:17.819 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:52:17.820 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.820 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:52:17.820 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6830", 2020-11-28T21:52:17.820 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:52:17.820 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.821 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.821 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:52:17.821 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6831", 2020-11-28T21:52:17.821 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:52:17.821 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:17.822 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:52:17.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.822 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:52:17.822 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:52:17.822 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.823 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:52:17.823 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6828", 2020-11-28T21:52:17.823 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:52:17.823 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.823 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.824 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:52:17.824 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6829", 2020-11-28T21:52:17.824 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:52:17.824 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:17.825 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:52:17.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.825 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6825/12530", 2020-11-28T21:52:17.825 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6827/12530", 2020-11-28T21:52:17.825 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6831/12530", 2020-11-28T21:52:17.826 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6829/12530", 2020-11-28T21:52:17.826 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:52:17.826 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:52:17.826 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:52:17.826 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:52:17.827 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.827 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.827 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 7, 2020-11-28T21:52:17.827 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "c328b06d-a216-4cba-aad7-9ad081340613", 2020-11-28T21:52:17.827 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:52:17.828 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:52:17.828 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:52:17.828 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:52:17.828 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:52:17.829 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:52:17.829 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:52:17.829 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 64, 2020-11-28T21:52:17.829 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:52:17.830 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:52:17.830 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:52:17.830 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:52:17.830 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.831 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:52:17.831 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6816", 2020-11-28T21:52:17.831 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:52:17.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.832 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.832 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:52:17.832 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6817", 2020-11-28T21:52:17.832 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:52:17.833 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:17.833 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:52:17.833 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.833 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:52:17.833 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:52:17.834 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.834 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:52:17.834 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6818", 2020-11-28T21:52:17.835 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:52:17.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.835 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.835 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:52:17.836 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6819", 2020-11-28T21:52:17.836 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:52:17.836 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:17.836 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:52:17.836 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.837 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:52:17.837 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:52:17.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.837 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:52:17.838 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6822", 2020-11-28T21:52:17.838 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:52:17.838 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.838 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.839 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:52:17.839 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6823", 2020-11-28T21:52:17.840 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:52:17.840 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:17.840 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:52:17.841 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.841 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:52:17.842 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:52:17.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.842 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:52:17.843 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6820", 2020-11-28T21:52:17.843 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:52:17.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.844 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:52:17.844 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6821", 2020-11-28T21:52:17.845 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:52:17.845 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:17.845 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:52:17.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.846 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6817/12528", 2020-11-28T21:52:17.846 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6819/12528", 2020-11-28T21:52:17.846 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6823/12528", 2020-11-28T21:52:17.846 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6821/12528", 2020-11-28T21:52:17.847 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:52:17.847 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:52:17.847 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:52:17.847 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:52:17.848 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:17.848 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:52:17.848 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_xinfo": [ 2020-11-28T21:52:17.848 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.849 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 0, 2020-11-28T21:52:17.849 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:52:17.849 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:52:17.849 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:52:17.850 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:52:17.850 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:52:17.850 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.038052+0000", 2020-11-28T21:52:17.850 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:52:17.851 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.851 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 1, 2020-11-28T21:52:17.851 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:52:17.851 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:52:17.852 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:52:17.852 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:52:17.852 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:52:17.853 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:17.947136+0000", 2020-11-28T21:52:17.853 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:52:17.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.853 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 2, 2020-11-28T21:52:17.854 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:52:17.854 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:52:17.854 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:52:17.855 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:52:17.855 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:52:17.855 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.001186+0000", 2020-11-28T21:52:17.856 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:52:17.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.856 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 3, 2020-11-28T21:52:17.857 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:52:17.857 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:52:17.857 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:52:17.857 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:52:17.858 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:52:17.858 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.199284+0000", 2020-11-28T21:52:17.858 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:52:17.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.859 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 4, 2020-11-28T21:52:17.859 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:52:17.859 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:52:17.859 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:52:17.860 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:52:17.860 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:52:17.860 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.038823+0000", 2020-11-28T21:52:17.860 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:52:17.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.861 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 5, 2020-11-28T21:52:17.861 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:52:17.861 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:52:17.861 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:52:17.862 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:52:17.862 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:52:17.862 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.009322+0000", 2020-11-28T21:52:17.863 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:52:17.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.864 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 6, 2020-11-28T21:52:17.864 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:52:17.864 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:52:17.865 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:52:17.865 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:52:17.866 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:52:17.866 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.238927+0000", 2020-11-28T21:52:17.866 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:52:17.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:17.867 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 7, 2020-11-28T21:52:17.868 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:52:17.868 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:52:17.869 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:52:17.869 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:52:17.869 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:52:17.870 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.029839+0000", 2020-11-28T21:52:17.870 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:52:17.871 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:17.871 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:52:17.871 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_upmap": [], 2020-11-28T21:52:17.872 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_upmap_items": [], 2020-11-28T21:52:17.872 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_temp": [], 2020-11-28T21:52:17.872 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_temp": [], 2020-11-28T21:52:17.873 INFO:teuthology.orchestra.run.smithi162.stdout: "blacklist": {}, 2020-11-28T21:52:17.873 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profiles": { 2020-11-28T21:52:17.873 INFO:teuthology.orchestra.run.smithi162.stdout: "default": { 2020-11-28T21:52:17.874 INFO:teuthology.orchestra.run.smithi162.stdout: "crush-failure-domain": "osd", 2020-11-28T21:52:17.874 INFO:teuthology.orchestra.run.smithi162.stdout: "k": "2", 2020-11-28T21:52:17.874 INFO:teuthology.orchestra.run.smithi162.stdout: "m": "1", 2020-11-28T21:52:17.875 INFO:teuthology.orchestra.run.smithi162.stdout: "plugin": "jerasure", 2020-11-28T21:52:17.875 INFO:teuthology.orchestra.run.smithi162.stdout: "ruleset-failure-domain": "osd", 2020-11-28T21:52:17.875 INFO:teuthology.orchestra.run.smithi162.stdout: "technique": "reed_sol_van" 2020-11-28T21:52:17.876 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:17.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:17.876 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps_queue": [], 2020-11-28T21:52:17.877 INFO:teuthology.orchestra.run.smithi162.stdout: "new_removed_snaps": [], 2020-11-28T21:52:17.877 INFO:teuthology.orchestra.run.smithi162.stdout: "new_purged_snaps": [], 2020-11-28T21:52:17.878 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_node_flags": {}, 2020-11-28T21:52:17.878 INFO:teuthology.orchestra.run.smithi162.stdout: "device_class_flags": {} 2020-11-28T21:52:17.878 INFO:teuthology.orchestra.run.smithi162.stdout:} 2020-11-28T21:52:17.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.582+0000 7fb00bfff700 1 -- 172.21.15.162:0/1832098694 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7faff0055440 msgr2=0x7faff0057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:17.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.582+0000 7fb00bfff700 1 --2- 172.21.15.162:0/1832098694 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7faff0055440 0x7faff0057900 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7faffc00d000 tx=0x7faffc007b60).stop 2020-11-28T21:52:17.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.582+0000 7fb00bfff700 1 -- 172.21.15.162:0/1832098694 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb00c09abb0 msgr2=0x7fb00c0989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:17.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.582+0000 7fb00bfff700 1 --2- 172.21.15.162:0/1832098694 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb00c09abb0 0x7fb00c0989a0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7faff4002f30 tx=0x7faff4015000).stop 2020-11-28T21:52:17.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.582+0000 7fb00bfff700 1 -- 172.21.15.162:0/1832098694 shutdown_connections 2020-11-28T21:52:17.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.582+0000 7fb00bfff700 1 --2- 172.21.15.162:0/1832098694 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7faff0055440 0x7faff0057900 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:17.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.582+0000 7fb00bfff700 1 --2- 172.21.15.162:0/1832098694 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb00c0b01a0 0x7fb00c065790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:17.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.582+0000 7fb00bfff700 1 --2- 172.21.15.162:0/1832098694 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb00c060010 0x7fb00c099c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:17.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.582+0000 7fb00bfff700 1 --2- 172.21.15.162:0/1832098694 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb00c09abb0 0x7fb00c0989a0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:17.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.582+0000 7fb00bfff700 1 -- 172.21.15.162:0/1832098694 >> 172.21.15.162:0/1832098694 conn(0x7fb00c0b54f0 msgr2=0x7fb00c0b3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:17.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.582+0000 7fb00bfff700 1 -- 172.21.15.162:0/1832098694 shutdown_connections 2020-11-28T21:52:17.886 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:17.582+0000 7fb00bfff700 1 -- 172.21.15.162:0/1832098694 wait complete. 2020-11-28T21:52:17.887 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:52:18.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.006+0000 7fa8ff59e700 1 Processor -- start 2020-11-28T21:52:18.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.006+0000 7fa8ff59e700 1 -- start start 2020-11-28T21:52:18.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.006+0000 7fa8ff59e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa900098b30 0x7fa9000994f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:18.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.006+0000 7fa8ff59e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fa9000b0000 0x7fa900060000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:18.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.006+0000 7fa8ff59e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa900098f10 con 0x7fa9000604f0 2020-11-28T21:52:18.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.006+0000 7fa8ff59e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9000b1030 con 0x7fa9000b0000 2020-11-28T21:52:18.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.006+0000 7fa8ff59e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9000b0480 con 0x7fa900098b30 2020-11-28T21:52:18.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.006+0000 7fa8fed9d700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fa9000604f0 0x7fa90014a550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:47894/0 (socket says 172.21.15.162:47894) 2020-11-28T21:52:18.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.006+0000 7fa8fed9d700 1 -- 172.21.15.162:0/2134053052 learned_addr learned my addr 172.21.15.162:0/2134053052 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:52:18.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.006+0000 7fa8fe59c700 1 --2- 172.21.15.162:0/2134053052 >> v2:172.21.15.162:3300/0 conn(0x7fa900098b30 0x7fa9000994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:18.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.006+0000 7fa8fd59a700 1 -- 172.21.15.162:0/2134053052 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3341017393 0 0) 0x7fa900098f10 con 0x7fa9000604f0 2020-11-28T21:52:18.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.006+0000 7fa8fd59a700 1 -- 172.21.15.162:0/2134053052 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8e4003230 con 0x7fa9000604f0 2020-11-28T21:52:18.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.006+0000 7fa8fdd9b700 1 --2- 172.21.15.162:0/2134053052 >> v2:172.21.15.118:3300/0 conn(0x7fa9000b0000 0x7fa900060000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:18.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.006+0000 7fa8fe59c700 1 -- 172.21.15.162:0/2134053052 >> v2:172.21.15.118:3300/0 conn(0x7fa9000b0000 msgr2=0x7fa900060000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:18.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.006+0000 7fa8fe59c700 1 --2- 172.21.15.162:0/2134053052 >> v2:172.21.15.118:3300/0 conn(0x7fa9000b0000 0x7fa900060000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:18.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.006+0000 7fa8fe59c700 1 -- 172.21.15.162:0/2134053052 >> v1:172.21.15.162:6789/0 conn(0x7fa9000604f0 legacy=0x7fa90014a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:18.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.006+0000 7fa8fd59a700 1 -- 172.21.15.162:0/2134053052 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2108936732 0 0) 0x7fa8e4003230 con 0x7fa9000604f0 2020-11-28T21:52:18.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.006+0000 7fa8fe59c700 1 -- 172.21.15.162:0/2134053052 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9000b0680 con 0x7fa900098b30 2020-11-28T21:52:18.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.006+0000 7fa8fdd9b700 1 --2- 172.21.15.162:0/2134053052 >> v2:172.21.15.118:3300/0 conn(0x7fa9000b0000 0x7fa900060000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:52:18.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.006+0000 7fa8fe59c700 1 --2- 172.21.15.162:0/2134053052 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa900098b30 0x7fa9000994f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7fa8f0001bf0 tx=0x7fa8e8006dc0).ready entity=mon.1 client_cookie=5dbec2c7b6d20c92 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:18.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.006+0000 7fa8fd59a700 1 -- 172.21.15.162:0/2134053052 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8f0001a50 con 0x7fa900098b30 2020-11-28T21:52:18.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.010+0000 7fa8fd59a700 1 -- 172.21.15.162:0/2134053052 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8e8009620 con 0x7fa900098b30 2020-11-28T21:52:18.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.010+0000 7fa8fd59a700 1 -- 172.21.15.162:0/2134053052 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8e8009790 con 0x7fa900098b30 2020-11-28T21:52:18.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.010+0000 7fa8ff59e700 1 -- 172.21.15.162:0/2134053052 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa900098b30 msgr2=0x7fa9000994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:18.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.010+0000 7fa8ff59e700 1 --2- 172.21.15.162:0/2134053052 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa900098b30 0x7fa9000994f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7fa8f0001bf0 tx=0x7fa8e8006dc0).stop 2020-11-28T21:52:18.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.010+0000 7fa8ff59e700 1 -- 172.21.15.162:0/2134053052 shutdown_connections 2020-11-28T21:52:18.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.010+0000 7fa8ff59e700 1 --2- 172.21.15.162:0/2134053052 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa900098b30 0x7fa9000994f0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:18.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.010+0000 7fa8ff59e700 1 --2- 172.21.15.162:0/2134053052 >> v2:172.21.15.118:3300/0 conn(0x7fa9000b0000 0x7fa900060000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:18.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.010+0000 7fa8ff59e700 1 -- 172.21.15.162:0/2134053052 >> 172.21.15.162:0/2134053052 conn(0x7fa90005b970 msgr2=0x7fa90005bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:18.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.010+0000 7fa8ff59e700 1 -- 172.21.15.162:0/2134053052 shutdown_connections 2020-11-28T21:52:18.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.010+0000 7fa8ff59e700 1 -- 172.21.15.162:0/2134053052 wait complete. 2020-11-28T21:52:18.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.010+0000 7fa8ff59e700 1 Processor -- start 2020-11-28T21:52:18.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.010+0000 7fa8ff59e700 1 -- start start 2020-11-28T21:52:18.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.010+0000 7fa8ff59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa9000604f0 0x7fa9000634e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:18.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.010+0000 7fa8ff59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa900098b30 0x7fa900158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:18.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.010+0000 7fa8ff59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa9000b0000 0x7fa9001594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:18.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.010+0000 7fa8ff59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9001588d0 con 0x7fa9000604f0 2020-11-28T21:52:18.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.010+0000 7fa8ff59e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa900158b40 con 0x7fa900098b30 2020-11-28T21:52:18.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.010+0000 7fa8ff59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa900066a30 con 0x7fa9000b0000 2020-11-28T21:52:18.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.010+0000 7fa8fdd9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa900098b30 0x7fa900158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:18.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.010+0000 7fa8fdd9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa900098b30 0x7fa900158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41054/0 (socket says 172.21.15.162:41054) 2020-11-28T21:52:18.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.010+0000 7fa8fdd9b700 1 -- 172.21.15.162:0/2967179828 learned_addr learned my addr 172.21.15.162:0/2967179828 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:18.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.010+0000 7fa8fe59c700 1 --2- 172.21.15.162:0/2967179828 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa9000604f0 0x7fa9000634e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:18.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.010+0000 7fa8fed9d700 1 --2- 172.21.15.162:0/2967179828 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa9000b0000 0x7fa9001594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:18.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.010+0000 7fa8fdd9b700 1 -- 172.21.15.162:0/2967179828 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa9000b0000 msgr2=0x7fa9001594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:18.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.010+0000 7fa8fdd9b700 1 --2- 172.21.15.162:0/2967179828 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa9000b0000 0x7fa9001594e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:18.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.010+0000 7fa8fdd9b700 1 -- 172.21.15.162:0/2967179828 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa9000604f0 msgr2=0x7fa9000634e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:18.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.010+0000 7fa8fdd9b700 1 --2- 172.21.15.162:0/2967179828 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa9000604f0 0x7fa9000634e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:18.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.010+0000 7fa8fdd9b700 1 -- 172.21.15.162:0/2967179828 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa900066cd0 con 0x7fa900098b30 2020-11-28T21:52:18.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.010+0000 7fa8fe59c700 1 --2- 172.21.15.162:0/2967179828 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa9000604f0 0x7fa9000634e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:52:18.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.014+0000 7fa8fdd9b700 1 --2- 172.21.15.162:0/2967179828 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa900098b30 0x7fa900158ff0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7fa8f8006910 tx=0x7fa8f8008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:18.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.014+0000 7fa8f77fe700 1 -- 172.21.15.162:0/2967179828 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8f8009980 con 0x7fa900098b30 2020-11-28T21:52:18.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.014+0000 7fa8f77fe700 1 -- 172.21.15.162:0/2967179828 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f8009ae0 con 0x7fa900098b30 2020-11-28T21:52:18.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.014+0000 7fa8ff59e700 1 -- 172.21.15.162:0/2967179828 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9000639d0 con 0x7fa900098b30 2020-11-28T21:52:18.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.014+0000 7fa8f77fe700 1 -- 172.21.15.162:0/2967179828 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8f8009e40 con 0x7fa900098b30 2020-11-28T21:52:18.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.014+0000 7fa8ff59e700 1 -- 172.21.15.162:0/2967179828 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa900067620 con 0x7fa900098b30 2020-11-28T21:52:18.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.014+0000 7fa8ff59e700 1 -- 172.21.15.162:0/2967179828 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa900158d40 con 0x7fa900098b30 2020-11-28T21:52:18.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.014+0000 7fa8f77fe700 1 -- 172.21.15.162:0/2967179828 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa8f8013480 con 0x7fa900098b30 2020-11-28T21:52:18.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.014+0000 7fa8f77fe700 1 --2- 172.21.15.162:0/2967179828 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa8d80553b0 0x7fa8d8057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:18.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.014+0000 7fa8f77fe700 1 -- 172.21.15.162:0/2967179828 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa8f806e6b0 con 0x7fa900098b30 2020-11-28T21:52:18.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.014+0000 7fa8fe59c700 1 --2- 172.21.15.162:0/2967179828 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa8d80553b0 0x7fa8d8057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:18.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.014+0000 7fa8fe59c700 1 --2- 172.21.15.162:0/2967179828 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa8d80553b0 0x7fa8d8057870 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7fa8e8017000 tx=0x7fa8e8015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:18.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.026+0000 7fa8f77fe700 1 -- 172.21.15.162:0/2967179828 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa8f8044950 con 0x7fa900098b30 2020-11-28T21:52:18.218 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.214+0000 7fa8ff59e700 1 -- 172.21.15.162:0/2967179828 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa900057050 con 0x7fa900098b30 2020-11-28T21:52:18.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.214+0000 7fa8f77fe700 1 -- 172.21.15.162:0/2967179828 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+2502 (secure 0 0 0) 0x7fa8f8047f70 con 0x7fa900098b30 2020-11-28T21:52:18.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.214+0000 7fa8ff59e700 1 -- 172.21.15.162:0/2967179828 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa8d80553b0 msgr2=0x7fa8d8057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:18.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.214+0000 7fa8ff59e700 1 --2- 172.21.15.162:0/2967179828 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa8d80553b0 0x7fa8d8057870 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7fa8e8017000 tx=0x7fa8e8015000).stop 2020-11-28T21:52:18.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.214+0000 7fa8ff59e700 1 -- 172.21.15.162:0/2967179828 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa900098b30 msgr2=0x7fa900158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:18.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.214+0000 7fa8ff59e700 1 --2- 172.21.15.162:0/2967179828 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa900098b30 0x7fa900158ff0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7fa8f8006910 tx=0x7fa8f8008040).stop 2020-11-28T21:52:18.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.214+0000 7fa8ff59e700 1 -- 172.21.15.162:0/2967179828 shutdown_connections 2020-11-28T21:52:18.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.214+0000 7fa8ff59e700 1 --2- 172.21.15.162:0/2967179828 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa8d80553b0 0x7fa8d8057870 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:18.220 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:52:18.221 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":27,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5342,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6837/3666321033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3666321033},{"type":"v1","addr":"172.21.15.162:6837","nonce":3666321033}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":25},{"gid":5769,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/4179306206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":4179306206},{"type":"v1","addr":"172.21.15.118:6833","nonce":4179306206}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":24}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:51:44.226383+0000","modified":"2020-11-28T21:51:45.230212+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5869},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5869":{"gid":5869,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6835/1433205970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1433205970},{"type":"v1","addr":"172.21.15.162:6835","nonce":1433205970}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-11-28T21:52:18.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.214+0000 7fa8ff59e700 1 --2- 172.21.15.162:0/2967179828 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa9000b0000 0x7fa9001594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:18.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.214+0000 7fa8ff59e700 1 --2- 172.21.15.162:0/2967179828 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa9000604f0 0x7fa9000634e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:18.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.214+0000 7fa8ff59e700 1 --2- 172.21.15.162:0/2967179828 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa900098b30 0x7fa900158ff0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:18.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.214+0000 7fa8ff59e700 1 -- 172.21.15.162:0/2967179828 >> 172.21.15.162:0/2967179828 conn(0x7fa90005b970 msgr2=0x7fa9000a8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:18.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.214+0000 7fa8ff59e700 1 -- 172.21.15.162:0/2967179828 shutdown_connections 2020-11-28T21:52:18.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.214+0000 7fa8ff59e700 1 -- 172.21.15.162:0/2967179828 wait complete. 2020-11-28T21:52:18.223 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 27 2020-11-28T21:52:18.230 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2020-11-28T21:52:18.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.342+0000 7fcd36156700 1 Processor -- start 2020-11-28T21:52:18.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.342+0000 7fcd36156700 1 -- start start 2020-11-28T21:52:18.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.342+0000 7fcd36156700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fcd300603d0 0x7fcd300b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:18.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.342+0000 7fcd36156700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcd30142a00 0x7fcd3009cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:18.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.342+0000 7fcd36156700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd300b4110 con 0x7fcd300b4c70 2020-11-28T21:52:18.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.342+0000 7fcd36156700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd30060f60 con 0x7fcd300603d0 2020-11-28T21:52:18.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.342+0000 7fcd36156700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd300b43f0 con 0x7fcd30142a00 2020-11-28T21:52:18.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.342+0000 7fcd34953700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcd30142a00 0x7fcd3009cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:18.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.342+0000 7fcd35955700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fcd300b4c70 0x7fcd30143020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:47908/0 (socket says 172.21.15.162:47908) 2020-11-28T21:52:18.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.342+0000 7fcd35955700 1 -- 172.21.15.162:0/2017774994 learned_addr learned my addr 172.21.15.162:0/2017774994 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:52:18.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.346+0000 7fcd35154700 1 --2- 172.21.15.162:0/2017774994 >> v2:172.21.15.118:3300/0 conn(0x7fcd300603d0 0x7fcd300b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:18.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.346+0000 7fcd34953700 1 -- 172.21.15.162:0/2017774994 >> v2:172.21.15.118:3300/0 conn(0x7fcd300603d0 msgr2=0x7fcd300b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:18.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.346+0000 7fcd34953700 1 --2- 172.21.15.162:0/2017774994 >> v2:172.21.15.118:3300/0 conn(0x7fcd300603d0 0x7fcd300b3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:18.355 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.346+0000 7fcd27fff700 1 -- 172.21.15.162:0/2017774994 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1270516018 0 0) 0x7fcd300b4110 con 0x7fcd300b4c70 2020-11-28T21:52:18.355 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.346+0000 7fcd34953700 1 -- 172.21.15.162:0/2017774994 >> v1:172.21.15.162:6789/0 conn(0x7fcd300b4c70 legacy=0x7fcd30143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:18.355 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.346+0000 7fcd34953700 1 -- 172.21.15.162:0/2017774994 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd300b4690 con 0x7fcd30142a00 2020-11-28T21:52:18.356 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.346+0000 7fcd34953700 1 --2- 172.21.15.162:0/2017774994 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd30142a00 0x7fcd3009cfc0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7fcd2c002f30 tx=0x7fcd2c006dc0).ready entity=mon.1 client_cookie=ec863ebb10eb97ce server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:18.356 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.346+0000 7fcd27fff700 1 -- 172.21.15.162:0/2017774994 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd2c0097b0 con 0x7fcd30142a00 2020-11-28T21:52:18.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.346+0000 7fcd27fff700 1 -- 172.21.15.162:0/2017774994 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd2c009910 con 0x7fcd30142a00 2020-11-28T21:52:18.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.346+0000 7fcd27fff700 1 -- 172.21.15.162:0/2017774994 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd2c009c70 con 0x7fcd30142a00 2020-11-28T21:52:18.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.346+0000 7fcd36156700 1 -- 172.21.15.162:0/2017774994 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd30142a00 msgr2=0x7fcd3009cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:18.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.346+0000 7fcd36156700 1 --2- 172.21.15.162:0/2017774994 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd30142a00 0x7fcd3009cfc0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7fcd2c002f30 tx=0x7fcd2c006dc0).stop 2020-11-28T21:52:18.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.346+0000 7fcd36156700 1 -- 172.21.15.162:0/2017774994 shutdown_connections 2020-11-28T21:52:18.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.346+0000 7fcd36156700 1 --2- 172.21.15.162:0/2017774994 >> v2:172.21.15.118:3300/0 conn(0x7fcd300603d0 0x7fcd300b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:18.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.346+0000 7fcd36156700 1 --2- 172.21.15.162:0/2017774994 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd30142a00 0x7fcd3009cfc0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:18.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.346+0000 7fcd36156700 1 -- 172.21.15.162:0/2017774994 >> 172.21.15.162:0/2017774994 conn(0x7fcd30066a40 msgr2=0x7fcd30066e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:18.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.346+0000 7fcd36156700 1 -- 172.21.15.162:0/2017774994 shutdown_connections 2020-11-28T21:52:18.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.346+0000 7fcd36156700 1 -- 172.21.15.162:0/2017774994 wait complete. 2020-11-28T21:52:18.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.346+0000 7fcd36156700 1 Processor -- start 2020-11-28T21:52:18.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.346+0000 7fcd36156700 1 -- start start 2020-11-28T21:52:18.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.350+0000 7fcd36156700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcd300603d0 0x7fcd301514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:18.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.350+0000 7fcd36156700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcd300b4c70 0x7fcd30151990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:18.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.350+0000 7fcd36156700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd30146f70 0x7fcd30147420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:18.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.350+0000 7fcd36156700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd301449b0 con 0x7fcd300b4c70 2020-11-28T21:52:18.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.350+0000 7fcd36156700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd30144710 con 0x7fcd30146f70 2020-11-28T21:52:18.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.350+0000 7fcd36156700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd30151f50 con 0x7fcd300603d0 2020-11-28T21:52:18.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.350+0000 7fcd35955700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd30146f70 0x7fcd30147420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:18.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.350+0000 7fcd35955700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd30146f70 0x7fcd30147420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41070/0 (socket says 172.21.15.162:41070) 2020-11-28T21:52:18.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.350+0000 7fcd35955700 1 -- 172.21.15.162:0/4181667341 learned_addr learned my addr 172.21.15.162:0/4181667341 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:18.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.350+0000 7fcd35154700 1 --2- 172.21.15.162:0/4181667341 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcd300603d0 0x7fcd301514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:18.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.350+0000 7fcd34953700 1 --2- 172.21.15.162:0/4181667341 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcd300b4c70 0x7fcd30151990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:18.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.350+0000 7fcd35955700 1 -- 172.21.15.162:0/4181667341 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcd300603d0 msgr2=0x7fcd301514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:18.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.350+0000 7fcd35955700 1 --2- 172.21.15.162:0/4181667341 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcd300603d0 0x7fcd301514a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:18.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.350+0000 7fcd35955700 1 -- 172.21.15.162:0/4181667341 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcd300b4c70 msgr2=0x7fcd30151990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:18.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.350+0000 7fcd35955700 1 --2- 172.21.15.162:0/4181667341 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcd300b4c70 0x7fcd30151990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:18.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.350+0000 7fcd35955700 1 -- 172.21.15.162:0/4181667341 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd30147910 con 0x7fcd30146f70 2020-11-28T21:52:18.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.350+0000 7fcd35955700 1 --2- 172.21.15.162:0/4181667341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd30146f70 0x7fcd30147420 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7fcd28001210 tx=0x7fcd28007d90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:18.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.350+0000 7fcd267fc700 1 -- 172.21.15.162:0/4181667341 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd28005c40 con 0x7fcd30146f70 2020-11-28T21:52:18.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.350+0000 7fcd267fc700 1 -- 172.21.15.162:0/4181667341 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd28005da0 con 0x7fcd30146f70 2020-11-28T21:52:18.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.350+0000 7fcd36156700 1 -- 172.21.15.162:0/4181667341 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd300af4f0 con 0x7fcd30146f70 2020-11-28T21:52:18.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.350+0000 7fcd267fc700 1 -- 172.21.15.162:0/4181667341 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd28004a60 con 0x7fcd30146f70 2020-11-28T21:52:18.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.350+0000 7fcd36156700 1 -- 172.21.15.162:0/4181667341 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd300afa00 con 0x7fcd30146f70 2020-11-28T21:52:18.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.350+0000 7fcd36156700 1 -- 172.21.15.162:0/4181667341 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd30057050 con 0x7fcd30146f70 2020-11-28T21:52:18.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.350+0000 7fcd267fc700 1 -- 172.21.15.162:0/4181667341 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fcd2801b060 con 0x7fcd30146f70 2020-11-28T21:52:18.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.354+0000 7fcd267fc700 1 --2- 172.21.15.162:0/4181667341 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcd180553b0 0x7fcd18057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:18.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.354+0000 7fcd267fc700 1 -- 172.21.15.162:0/4181667341 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fcd2806da90 con 0x7fcd30146f70 2020-11-28T21:52:18.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.354+0000 7fcd35154700 1 --2- 172.21.15.162:0/4181667341 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcd180553b0 0x7fcd18057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:18.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.354+0000 7fcd35154700 1 --2- 172.21.15.162:0/4181667341 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcd180553b0 0x7fcd18057870 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7fcd2000b000 tx=0x7fcd20008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:18.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.358+0000 7fcd267fc700 1 -- 172.21.15.162:0/4181667341 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fcd28043d90 con 0x7fcd30146f70 2020-11-28T21:52:18.547 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:18.545+0000 7fcd36156700 1 -- 172.21.15.162:0/4181667341 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fcd301522f0 con 0x7fcd30146f70 2020-11-28T21:52:19.000 INFO:tasks.ceph.mds.a.smithi162.stderr:2020-11-28T21:52:18.997+0000 7fa0f9eb9700 -1 MDSIOContextBase: blacklisted! Restarting... 2020-11-28T21:52:19.025 INFO:tasks.ceph.mds.a.smithi162.stderr: -1> 2020-11-28T21:52:18.997+0000 7fa0f9eb9700 -1 MDSIOContextBase: blacklisted! Restarting... 2020-11-28T21:52:19.044 INFO:tasks.ceph.mds.a.smithi162.stdout:starting mds.a at 2020-11-28T21:52:19.147 INFO:tasks.ceph.mon.b.smithi118.stderr:2020-11-28T21:52:19.143+0000 7ff4e1520700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-11-28T21:52:19.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.145+0000 7fcd267fc700 1 -- 172.21.15.162:0/4181667341 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) v1 ==== 161+0+0 (secure 0 0 0) 0x7fcd280473b0 con 0x7fcd30146f70 2020-11-28T21:52:19.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.149+0000 7fcd36156700 1 -- 172.21.15.162:0/4181667341 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcd180553b0 msgr2=0x7fcd18057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:19.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.149+0000 7fcd36156700 1 --2- 172.21.15.162:0/4181667341 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcd180553b0 0x7fcd18057870 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7fcd2000b000 tx=0x7fcd20008040).stop 2020-11-28T21:52:19.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.149+0000 7fcd36156700 1 -- 172.21.15.162:0/4181667341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd30146f70 msgr2=0x7fcd30147420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:19.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.149+0000 7fcd36156700 1 --2- 172.21.15.162:0/4181667341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd30146f70 0x7fcd30147420 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7fcd28001210 tx=0x7fcd28007d90).stop 2020-11-28T21:52:19.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.149+0000 7fcd36156700 1 -- 172.21.15.162:0/4181667341 shutdown_connections 2020-11-28T21:52:19.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.149+0000 7fcd36156700 1 --2- 172.21.15.162:0/4181667341 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcd180553b0 0x7fcd18057870 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:19.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.149+0000 7fcd36156700 1 --2- 172.21.15.162:0/4181667341 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcd300603d0 0x7fcd301514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:19.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.149+0000 7fcd36156700 1 --2- 172.21.15.162:0/4181667341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd30146f70 0x7fcd30147420 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:19.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.149+0000 7fcd36156700 1 --2- 172.21.15.162:0/4181667341 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcd300b4c70 0x7fcd30151990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:19.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.149+0000 7fcd36156700 1 -- 172.21.15.162:0/4181667341 >> 172.21.15.162:0/4181667341 conn(0x7fcd30066a40 msgr2=0x7fcd30142df0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:19.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.149+0000 7fcd36156700 1 -- 172.21.15.162:0/4181667341 shutdown_connections 2020-11-28T21:52:19.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.149+0000 7fcd36156700 1 -- 172.21.15.162:0/4181667341 wait complete. 2020-11-28T21:52:19.156 INFO:teuthology.orchestra.run.smithi162.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-11-28T21:52:19.173 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:52:19.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.577+0000 7f28a673b700 1 Processor -- start 2020-11-28T21:52:19.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.577+0000 7f28a673b700 1 -- start start 2020-11-28T21:52:19.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.581+0000 7f28a673b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f28a00994f0 0x7f28a00b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:19.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.581+0000 7f28a673b700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28a0060710 0x7f28a00b1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:19.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.581+0000 7f28a673b700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28a0098f10 con 0x7f28a0098b30 2020-11-28T21:52:19.586 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.581+0000 7f28a673b700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28a00b1030 con 0x7f28a00994f0 2020-11-28T21:52:19.586 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.581+0000 7f28a673b700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28a00b0680 con 0x7f28a0060710 2020-11-28T21:52:19.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.581+0000 7f28a4f38700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f28a00994f0 0x7f28a00b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:19.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.581+0000 7f28a5739700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f28a0098b30 0x7f28a0060000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:47928/0 (socket says 172.21.15.162:47928) 2020-11-28T21:52:19.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.581+0000 7f28a5739700 1 -- 172.21.15.162:0/349095020 learned_addr learned my addr 172.21.15.162:0/349095020 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:52:19.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.581+0000 7f28a5f3a700 1 --2- 172.21.15.162:0/349095020 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28a0060710 0x7f28a00b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:19.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.581+0000 7f28a4f38700 1 -- 172.21.15.162:0/349095020 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28a0060710 msgr2=0x7f28a00b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:19.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.581+0000 7f28a4f38700 1 --2- 172.21.15.162:0/349095020 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28a0060710 0x7f28a00b1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:19.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.581+0000 7f28a4f38700 1 -- 172.21.15.162:0/349095020 >> v1:172.21.15.162:6789/0 conn(0x7f28a0098b30 legacy=0x7f28a0060000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:19.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.581+0000 7f2893fff700 1 -- 172.21.15.162:0/349095020 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3095707195 0 0) 0x7f28a0098f10 con 0x7f28a0098b30 2020-11-28T21:52:19.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.581+0000 7f28a4f38700 1 -- 172.21.15.162:0/349095020 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a00b0920 con 0x7f28a00994f0 2020-11-28T21:52:19.594 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.581+0000 7f28a4f38700 1 --2- 172.21.15.162:0/349095020 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28a00994f0 0x7f28a00b0000 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7f2894002f30 tx=0x7f2894006dc0).ready entity=mon.1 client_cookie=6e98a6cc157c54a8 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:19.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.581+0000 7f2893fff700 1 -- 172.21.15.162:0/349095020 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2894009760 con 0x7f28a00994f0 2020-11-28T21:52:19.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.581+0000 7f2893fff700 1 -- 172.21.15.162:0/349095020 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28940098c0 con 0x7f28a00994f0 2020-11-28T21:52:19.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.581+0000 7f2893fff700 1 -- 172.21.15.162:0/349095020 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2894009c20 con 0x7f28a00994f0 2020-11-28T21:52:19.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.581+0000 7f28a673b700 1 -- 172.21.15.162:0/349095020 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28a00994f0 msgr2=0x7f28a00b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:19.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.581+0000 7f28a673b700 1 --2- 172.21.15.162:0/349095020 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28a00994f0 0x7f28a00b0000 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7f2894002f30 tx=0x7f2894006dc0).stop 2020-11-28T21:52:19.597 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.581+0000 7f28a673b700 1 -- 172.21.15.162:0/349095020 shutdown_connections 2020-11-28T21:52:19.597 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.581+0000 7f28a673b700 1 --2- 172.21.15.162:0/349095020 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28a0060710 0x7f28a00b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:19.597 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.581+0000 7f28a673b700 1 --2- 172.21.15.162:0/349095020 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28a00994f0 0x7f28a00b0000 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:19.598 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.581+0000 7f28a673b700 1 -- 172.21.15.162:0/349095020 >> 172.21.15.162:0/349095020 conn(0x7f28a005b970 msgr2=0x7f28a005bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:19.598 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.581+0000 7f28a673b700 1 -- 172.21.15.162:0/349095020 shutdown_connections 2020-11-28T21:52:19.598 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.581+0000 7f28a673b700 1 -- 172.21.15.162:0/349095020 wait complete. 2020-11-28T21:52:19.599 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.581+0000 7f28a673b700 1 Processor -- start 2020-11-28T21:52:19.599 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.581+0000 7f28a673b700 1 -- start start 2020-11-28T21:52:19.599 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.585+0000 7f28a673b700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28a0060710 0x7f28a0063680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:19.600 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.585+0000 7f28a673b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28a0098b30 0x7f28a0155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:19.600 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.585+0000 7f28a673b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28a00994f0 0x7f28a00adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:19.600 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.585+0000 7f28a673b700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a0065840 con 0x7f28a00994f0 2020-11-28T21:52:19.601 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.585+0000 7f28a673b700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a0065ae0 con 0x7f28a0098b30 2020-11-28T21:52:19.601 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.585+0000 7f28a673b700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28a0065d80 con 0x7f28a0060710 2020-11-28T21:52:19.601 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.585+0000 7f28a4f38700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28a0098b30 0x7f28a0155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:19.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.585+0000 7f28a4f38700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28a0098b30 0x7f28a0155bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41092/0 (socket says 172.21.15.162:41092) 2020-11-28T21:52:19.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.585+0000 7f28a4f38700 1 -- 172.21.15.162:0/1090338331 learned_addr learned my addr 172.21.15.162:0/1090338331 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:19.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.585+0000 7f28a5739700 1 --2- 172.21.15.162:0/1090338331 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28a0060710 0x7f28a0063680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:19.603 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.585+0000 7f28a5f3a700 1 --2- 172.21.15.162:0/1090338331 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28a00994f0 0x7f28a00adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:19.603 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.585+0000 7f28a4f38700 1 -- 172.21.15.162:0/1090338331 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28a0060710 msgr2=0x7f28a0063680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:19.603 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.585+0000 7f28a4f38700 1 --2- 172.21.15.162:0/1090338331 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28a0060710 0x7f28a0063680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:19.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.585+0000 7f28a4f38700 1 -- 172.21.15.162:0/1090338331 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28a00994f0 msgr2=0x7f28a00adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:19.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.585+0000 7f28a4f38700 1 --2- 172.21.15.162:0/1090338331 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28a00994f0 0x7f28a00adfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:19.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.585+0000 7f28a4f38700 1 -- 172.21.15.162:0/1090338331 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a00adb10 con 0x7f28a0098b30 2020-11-28T21:52:19.605 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.585+0000 7f28a4f38700 1 --2- 172.21.15.162:0/1090338331 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28a0098b30 0x7f28a0155bc0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f2894002f30 tx=0x7f2894015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:19.605 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.585+0000 7f28927fc700 1 -- 172.21.15.162:0/1090338331 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2894011910 con 0x7f28a0098b30 2020-11-28T21:52:19.605 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.585+0000 7f28927fc700 1 -- 172.21.15.162:0/1090338331 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2894011a70 con 0x7f28a0098b30 2020-11-28T21:52:19.606 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.585+0000 7f28927fc700 1 -- 172.21.15.162:0/1090338331 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2894011dd0 con 0x7f28a0098b30 2020-11-28T21:52:19.606 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.585+0000 7f28a673b700 1 -- 172.21.15.162:0/1090338331 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28a0063e70 con 0x7f28a0098b30 2020-11-28T21:52:19.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.585+0000 7f28a673b700 1 -- 172.21.15.162:0/1090338331 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28a00ae6c0 con 0x7f28a0098b30 2020-11-28T21:52:19.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.589+0000 7f28a673b700 1 -- 172.21.15.162:0/1090338331 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28a00adca0 con 0x7f28a0098b30 2020-11-28T21:52:19.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.589+0000 7f28927fc700 1 -- 172.21.15.162:0/1090338331 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2894024030 con 0x7f28a0098b30 2020-11-28T21:52:19.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.589+0000 7f28927fc700 1 --2- 172.21.15.162:0/1090338331 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2888055350 0x7f2888057810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:19.608 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.589+0000 7f28927fc700 1 -- 172.21.15.162:0/1090338331 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f289406edf0 con 0x7f28a0098b30 2020-11-28T21:52:19.608 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.589+0000 7f28a5739700 1 --2- 172.21.15.162:0/1090338331 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2888055350 0x7f2888057810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:19.608 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.589+0000 7f28a5739700 1 --2- 172.21.15.162:0/1090338331 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2888055350 0x7f2888057810 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f289c00d000 tx=0x7f289c001160).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:19.609 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.597+0000 7f28927fc700 1 -- 172.21.15.162:0/1090338331 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f28940450a0 con 0x7f28a0098b30 2020-11-28T21:52:19.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.781+0000 7f28a673b700 1 -- 172.21.15.162:0/1090338331 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28a0057050 con 0x7f28a0098b30 2020-11-28T21:52:19.788 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.785+0000 7f28927fc700 1 -- 172.21.15.162:0/1090338331 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+2138 (secure 0 0 0) 0x7f2894022020 con 0x7f28a0098b30 2020-11-28T21:52:19.788 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:52:19.788 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":28,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5342,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6837/3666321033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3666321033},{"type":"v1","addr":"172.21.15.162:6837","nonce":3666321033}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":25},{"gid":5769,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/4179306206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":4179306206},{"type":"v1","addr":"172.21.15.118:6833","nonce":4179306206}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":24}],"filesystems":[{"mdsmap":{"epoch":28,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:51:44.226383+0000","modified":"2020-11-28T21:52:19.144515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-11-28T21:52:19.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.785+0000 7f28a673b700 1 -- 172.21.15.162:0/1090338331 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2888055350 msgr2=0x7f2888057810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:19.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.785+0000 7f28a673b700 1 --2- 172.21.15.162:0/1090338331 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2888055350 0x7f2888057810 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f289c00d000 tx=0x7f289c001160).stop 2020-11-28T21:52:19.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.785+0000 7f28a673b700 1 -- 172.21.15.162:0/1090338331 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28a0098b30 msgr2=0x7f28a0155bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:19.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.785+0000 7f28a673b700 1 --2- 172.21.15.162:0/1090338331 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28a0098b30 0x7f28a0155bc0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f2894002f30 tx=0x7f2894015000).stop 2020-11-28T21:52:19.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.785+0000 7f28a673b700 1 -- 172.21.15.162:0/1090338331 shutdown_connections 2020-11-28T21:52:19.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.785+0000 7f28a673b700 1 --2- 172.21.15.162:0/1090338331 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2888055350 0x7f2888057810 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:19.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.785+0000 7f28a673b700 1 --2- 172.21.15.162:0/1090338331 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28a0060710 0x7f28a0063680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:19.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.785+0000 7f28a673b700 1 --2- 172.21.15.162:0/1090338331 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28a00994f0 0x7f28a00adfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:19.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.785+0000 7f28a673b700 1 --2- 172.21.15.162:0/1090338331 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28a0098b30 0x7f28a0155bc0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:19.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.785+0000 7f28a673b700 1 -- 172.21.15.162:0/1090338331 >> 172.21.15.162:0/1090338331 conn(0x7f28a005b970 msgr2=0x7f28a0149030 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:19.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.785+0000 7f28a673b700 1 -- 172.21.15.162:0/1090338331 shutdown_connections 2020-11-28T21:52:19.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.785+0000 7f28a673b700 1 -- 172.21.15.162:0/1090338331 wait complete. 2020-11-28T21:52:19.794 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 28 2020-11-28T21:52:19.802 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2020-11-28T21:52:19.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.925+0000 7fbd9532c700 1 Processor -- start 2020-11-28T21:52:19.931 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.925+0000 7fbd9532c700 1 -- start start 2020-11-28T21:52:19.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.929+0000 7fbd9532c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbd90060d10 0x7fbd900b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:19.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.929+0000 7fbd9532c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fbd900b4390 0x7fbd9013fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:19.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.929+0000 7fbd9532c700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd900b4930 con 0x7fbd900b60e0 2020-11-28T21:52:19.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.929+0000 7fbd9532c700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd900b2170 con 0x7fbd900b4390 2020-11-28T21:52:19.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.929+0000 7fbd9532c700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd900b6660 con 0x7fbd90060d10 2020-11-28T21:52:19.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.929+0000 7fbd8effd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbd90060d10 0x7fbd900b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:19.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.929+0000 7fbd8effd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbd90060d10 0x7fbd900b52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41100/0 (socket says 172.21.15.162:41100) 2020-11-28T21:52:19.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.929+0000 7fbd8effd700 1 -- 172.21.15.162:0/2963476442 learned_addr learned my addr 172.21.15.162:0/2963476442 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:19.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.929+0000 7fbd8f7fe700 1 --2- 172.21.15.162:0/2963476442 >> v2:172.21.15.118:3300/0 conn(0x7fbd900b4390 0x7fbd9013fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:19.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.929+0000 7fbd8effd700 1 -- 172.21.15.162:0/2963476442 >> v2:172.21.15.118:3300/0 conn(0x7fbd900b4390 msgr2=0x7fbd9013fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:19.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.929+0000 7fbd8effd700 1 --2- 172.21.15.162:0/2963476442 >> v2:172.21.15.118:3300/0 conn(0x7fbd900b4390 0x7fbd9013fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:19.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.929+0000 7fbd8effd700 1 -- 172.21.15.162:0/2963476442 >> v1:172.21.15.118:6789/0 conn(0x7fbd900b60e0 legacy=0x7fbd90146180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:19.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.929+0000 7fbd8effd700 1 -- 172.21.15.162:0/2963476442 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd900b4b60 con 0x7fbd90060d10 2020-11-28T21:52:19.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.929+0000 7fbd8effd700 1 --2- 172.21.15.162:0/2963476442 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd90060d10 0x7fbd900b52a0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7fbd80002f30 tx=0x7fbd80006e80).ready entity=mon.1 client_cookie=a6b9dda84e29c2c5 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:19.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.929+0000 7fbd8dffb700 1 -- 172.21.15.162:0/2963476442 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd800097f0 con 0x7fbd90060d10 2020-11-28T21:52:19.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.929+0000 7fbd8dffb700 1 -- 172.21.15.162:0/2963476442 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd80009950 con 0x7fbd90060d10 2020-11-28T21:52:19.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.929+0000 7fbd8dffb700 1 -- 172.21.15.162:0/2963476442 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd80009cb0 con 0x7fbd90060d10 2020-11-28T21:52:19.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.929+0000 7fbd9532c700 1 -- 172.21.15.162:0/2963476442 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd90060d10 msgr2=0x7fbd900b52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:19.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.929+0000 7fbd9532c700 1 --2- 172.21.15.162:0/2963476442 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd90060d10 0x7fbd900b52a0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7fbd80002f30 tx=0x7fbd80006e80).stop 2020-11-28T21:52:19.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.929+0000 7fbd9532c700 1 -- 172.21.15.162:0/2963476442 shutdown_connections 2020-11-28T21:52:19.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.929+0000 7fbd9532c700 1 --2- 172.21.15.162:0/2963476442 >> v2:172.21.15.118:3300/0 conn(0x7fbd900b4390 0x7fbd9013fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:19.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.929+0000 7fbd9532c700 1 --2- 172.21.15.162:0/2963476442 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd90060d10 0x7fbd900b52a0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:19.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.929+0000 7fbd9532c700 1 -- 172.21.15.162:0/2963476442 >> 172.21.15.162:0/2963476442 conn(0x7fbd9005fe70 msgr2=0x7fbd9006aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:19.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.933+0000 7fbd9532c700 1 -- 172.21.15.162:0/2963476442 shutdown_connections 2020-11-28T21:52:19.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.933+0000 7fbd9532c700 1 -- 172.21.15.162:0/2963476442 wait complete. 2020-11-28T21:52:19.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.933+0000 7fbd9532c700 1 Processor -- start 2020-11-28T21:52:19.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.933+0000 7fbd9532c700 1 -- start start 2020-11-28T21:52:19.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.933+0000 7fbd9532c700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbd90060d10 0x7fbd90147a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:19.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.933+0000 7fbd9532c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbd900b4390 0x7fbd901521d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:19.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.933+0000 7fbd9532c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd900b60e0 0x7fbd901526c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:19.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.933+0000 7fbd9532c700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd90151de0 con 0x7fbd900b4390 2020-11-28T21:52:19.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.933+0000 7fbd9532c700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd901454b0 con 0x7fbd900b60e0 2020-11-28T21:52:19.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.933+0000 7fbd9532c700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd901456e0 con 0x7fbd90060d10 2020-11-28T21:52:19.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.933+0000 7fbd8f7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd900b60e0 0x7fbd901526c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:19.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.933+0000 7fbd8effd700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbd90060d10 0x7fbd90147a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:19.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.933+0000 7fbd8f7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd900b60e0 0x7fbd901526c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41108/0 (socket says 172.21.15.162:41108) 2020-11-28T21:52:19.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.933+0000 7fbd8f7fe700 1 -- 172.21.15.162:0/1090888915 learned_addr learned my addr 172.21.15.162:0/1090888915 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:19.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.933+0000 7fbd8e7fc700 1 --2- 172.21.15.162:0/1090888915 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbd900b4390 0x7fbd901521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:19.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.937+0000 7fbd8f7fe700 1 -- 172.21.15.162:0/1090888915 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbd90060d10 msgr2=0x7fbd90147a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:19.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.937+0000 7fbd8f7fe700 1 --2- 172.21.15.162:0/1090888915 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbd90060d10 0x7fbd90147a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:19.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.937+0000 7fbd8f7fe700 1 -- 172.21.15.162:0/1090888915 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbd900b4390 msgr2=0x7fbd901521d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:19.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.937+0000 7fbd8f7fe700 1 --2- 172.21.15.162:0/1090888915 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbd900b4390 0x7fbd901521d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:19.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.937+0000 7fbd8f7fe700 1 -- 172.21.15.162:0/1090888915 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd90145980 con 0x7fbd900b60e0 2020-11-28T21:52:19.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.937+0000 7fbd8f7fe700 1 --2- 172.21.15.162:0/1090888915 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd900b60e0 0x7fbd901526c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7fbd88006dd0 tx=0x7fbd880091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:19.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.937+0000 7fbd7ffff700 1 -- 172.21.15.162:0/1090888915 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd8800c950 con 0x7fbd900b60e0 2020-11-28T21:52:19.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.937+0000 7fbd9532c700 1 -- 172.21.15.162:0/1090888915 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd90140ee0 con 0x7fbd900b60e0 2020-11-28T21:52:19.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.937+0000 7fbd7ffff700 1 -- 172.21.15.162:0/1090888915 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd8800cab0 con 0x7fbd900b60e0 2020-11-28T21:52:19.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.937+0000 7fbd7ffff700 1 -- 172.21.15.162:0/1090888915 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd8800ce10 con 0x7fbd900b60e0 2020-11-28T21:52:19.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.937+0000 7fbd9532c700 1 -- 172.21.15.162:0/1090888915 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd90147050 con 0x7fbd900b60e0 2020-11-28T21:52:19.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.937+0000 7fbd9532c700 1 -- 172.21.15.162:0/1090888915 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd90057050 con 0x7fbd900b60e0 2020-11-28T21:52:19.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.937+0000 7fbd7ffff700 1 -- 172.21.15.162:0/1090888915 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fbd88022070 con 0x7fbd900b60e0 2020-11-28T21:52:19.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.941+0000 7fbd7ffff700 1 --2- 172.21.15.162:0/1090888915 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbd6c055440 0x7fbd6c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:19.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.941+0000 7fbd7ffff700 1 -- 172.21.15.162:0/1090888915 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fbd8806e760 con 0x7fbd900b60e0 2020-11-28T21:52:19.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.941+0000 7fbd8effd700 1 --2- 172.21.15.162:0/1090888915 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbd6c055440 0x7fbd6c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:19.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.941+0000 7fbd8effd700 1 --2- 172.21.15.162:0/1090888915 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbd6c055440 0x7fbd6c057900 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7fbd8001c000 tx=0x7fbd80015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:19.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:19.945+0000 7fbd7ffff700 1 -- 172.21.15.162:0/1090888915 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fbd88044880 con 0x7fbd900b60e0 2020-11-28T21:52:20.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.129+0000 7fbd9532c700 1 -- 172.21.15.162:0/1090888915 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbd900676b0 con 0x7fbd900b60e0 2020-11-28T21:52:20.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.845+0000 7fbd7ffff700 1 -- 172.21.15.162:0/1090888915 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbd88016020 con 0x7fbd900b60e0 2020-11-28T21:52:20.853 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.849+0000 7fbd9532c700 1 -- 172.21.15.162:0/1090888915 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbd6c055440 msgr2=0x7fbd6c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:20.854 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.849+0000 7fbd9532c700 1 --2- 172.21.15.162:0/1090888915 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbd6c055440 0x7fbd6c057900 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7fbd8001c000 tx=0x7fbd80015000).stop 2020-11-28T21:52:20.854 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.849+0000 7fbd9532c700 1 -- 172.21.15.162:0/1090888915 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd900b60e0 msgr2=0x7fbd901526c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:20.855 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.849+0000 7fbd9532c700 1 --2- 172.21.15.162:0/1090888915 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd900b60e0 0x7fbd901526c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7fbd88006dd0 tx=0x7fbd880091a0).stop 2020-11-28T21:52:20.855 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.849+0000 7fbd9532c700 1 -- 172.21.15.162:0/1090888915 shutdown_connections 2020-11-28T21:52:20.855 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.849+0000 7fbd9532c700 1 --2- 172.21.15.162:0/1090888915 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbd6c055440 0x7fbd6c057900 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:20.856 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.849+0000 7fbd9532c700 1 --2- 172.21.15.162:0/1090888915 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbd90060d10 0x7fbd90147a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:20.856 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.849+0000 7fbd9532c700 1 --2- 172.21.15.162:0/1090888915 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd900b60e0 0x7fbd901526c0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:20.856 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.849+0000 7fbd9532c700 1 --2- 172.21.15.162:0/1090888915 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbd900b4390 0x7fbd901521d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:20.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.849+0000 7fbd9532c700 1 -- 172.21.15.162:0/1090888915 >> 172.21.15.162:0/1090888915 conn(0x7fbd9005fe70 msgr2=0x7fbd9013fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:20.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.849+0000 7fbd9532c700 1 -- 172.21.15.162:0/1090888915 shutdown_connections 2020-11-28T21:52:20.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.849+0000 7fbd9532c700 1 -- 172.21.15.162:0/1090888915 wait complete. 2020-11-28T21:52:20.870 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-11-28T21:52:20.996 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.993+0000 7fa95165f700 1 Processor -- start 2020-11-28T21:52:20.996 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.993+0000 7fa95165f700 1 -- start start 2020-11-28T21:52:20.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.993+0000 7fa95165f700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa94c0b3c90 0x7fa94c142a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:20.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.993+0000 7fa95165f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa94c0b18c0 0x7fa94c142f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:20.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.993+0000 7fa95165f700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa94c0aac00 con 0x7fa94c05f380 2020-11-28T21:52:21.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.993+0000 7fa95165f700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa94c13cf80 con 0x7fa94c0b3c90 2020-11-28T21:52:21.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.993+0000 7fa95165f700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c0b4070 con 0x7fa94c0b18c0 2020-11-28T21:52:21.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.993+0000 7fa94bfff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa94c0b3c90 0x7fa94c142a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:21.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.993+0000 7fa950e5e700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fa94c05f380 0x7fa94c146050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:47956/0 (socket says 172.21.15.162:47956) 2020-11-28T21:52:21.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.993+0000 7fa950e5e700 1 -- 172.21.15.162:0/444199961 learned_addr learned my addr 172.21.15.162:0/444199961 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:52:21.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.993+0000 7fa94b7fe700 1 --2- 172.21.15.162:0/444199961 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa94c0b18c0 0x7fa94c142f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:21.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.993+0000 7fa94bfff700 1 -- 172.21.15.162:0/444199961 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa94c0b18c0 msgr2=0x7fa94c142f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:21.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.993+0000 7fa94bfff700 1 --2- 172.21.15.162:0/444199961 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa94c0b18c0 0x7fa94c142f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:21.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.993+0000 7fa94bfff700 1 -- 172.21.15.162:0/444199961 >> v1:172.21.15.162:6789/0 conn(0x7fa94c05f380 legacy=0x7fa94c146050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:21.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.993+0000 7fa94bfff700 1 -- 172.21.15.162:0/444199961 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa94c060a10 con 0x7fa94c0b3c90 2020-11-28T21:52:21.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.993+0000 7fa94affd700 1 -- 172.21.15.162:0/444199961 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4040119494 0 0) 0x7fa94c0aac00 con 0x7fa94c05f380 2020-11-28T21:52:21.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.993+0000 7fa94b7fe700 1 --2- 172.21.15.162:0/444199961 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa94c0b18c0 0x7fa94c142f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:52:21.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.993+0000 7fa94bfff700 1 --2- 172.21.15.162:0/444199961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa94c0b3c90 0x7fa94c142a80 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7fa93c002f30 tx=0x7fa93c006dc0).ready entity=mon.1 client_cookie=657e24d9e353a3b8 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:21.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.993+0000 7fa94affd700 1 -- 172.21.15.162:0/444199961 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa93c0097f0 con 0x7fa94c0b3c90 2020-11-28T21:52:21.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.993+0000 7fa94affd700 1 -- 172.21.15.162:0/444199961 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa93c009950 con 0x7fa94c0b3c90 2020-11-28T21:52:21.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.993+0000 7fa94affd700 1 -- 172.21.15.162:0/444199961 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa93c009cb0 con 0x7fa94c0b3c90 2020-11-28T21:52:21.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.993+0000 7fa95165f700 1 -- 172.21.15.162:0/444199961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa94c0b3c90 msgr2=0x7fa94c142a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:21.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.993+0000 7fa95165f700 1 --2- 172.21.15.162:0/444199961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa94c0b3c90 0x7fa94c142a80 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7fa93c002f30 tx=0x7fa93c006dc0).stop 2020-11-28T21:52:21.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.993+0000 7fa95165f700 1 -- 172.21.15.162:0/444199961 shutdown_connections 2020-11-28T21:52:21.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.993+0000 7fa95165f700 1 --2- 172.21.15.162:0/444199961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa94c0b3c90 0x7fa94c142a80 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:21.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.993+0000 7fa95165f700 1 --2- 172.21.15.162:0/444199961 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa94c0b18c0 0x7fa94c142f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:21.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.993+0000 7fa95165f700 1 -- 172.21.15.162:0/444199961 >> 172.21.15.162:0/444199961 conn(0x7fa94c062630 msgr2=0x7fa94c062a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:21.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.993+0000 7fa95165f700 1 -- 172.21.15.162:0/444199961 shutdown_connections 2020-11-28T21:52:21.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.993+0000 7fa95165f700 1 -- 172.21.15.162:0/444199961 wait complete. 2020-11-28T21:52:21.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.997+0000 7fa95165f700 1 Processor -- start 2020-11-28T21:52:21.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.997+0000 7fa95165f700 1 -- start start 2020-11-28T21:52:21.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.997+0000 7fa95165f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa94c05f380 0x7fa94c05c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:21.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.997+0000 7fa95165f700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa94c0b18c0 0x7fa94c152020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:21.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.997+0000 7fa95165f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa94c0b3c90 0x7fa94c152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:21.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.997+0000 7fa95165f700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c151bc0 con 0x7fa94c0b18c0 2020-11-28T21:52:21.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.997+0000 7fa95165f700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c152a00 con 0x7fa94c05f380 2020-11-28T21:52:21.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.997+0000 7fa95165f700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c140a30 con 0x7fa94c0b3c90 2020-11-28T21:52:21.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.997+0000 7fa94bfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa94c05f380 0x7fa94c05c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:21.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.997+0000 7fa94bfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa94c05f380 0x7fa94c05c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41118/0 (socket says 172.21.15.162:41118) 2020-11-28T21:52:21.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.997+0000 7fa94bfff700 1 -- 172.21.15.162:0/1825314494 learned_addr learned my addr 172.21.15.162:0/1825314494 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:21.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.997+0000 7fa950e5e700 1 --2- 172.21.15.162:0/1825314494 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa94c0b3c90 0x7fa94c152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:21.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.997+0000 7fa94b7fe700 1 --2- 172.21.15.162:0/1825314494 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa94c0b18c0 0x7fa94c152020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:21.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.997+0000 7fa94bfff700 1 -- 172.21.15.162:0/1825314494 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa94c0b3c90 msgr2=0x7fa94c152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:21.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.997+0000 7fa94bfff700 1 --2- 172.21.15.162:0/1825314494 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa94c0b3c90 0x7fa94c152510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:21.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.997+0000 7fa94bfff700 1 -- 172.21.15.162:0/1825314494 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa94c0b18c0 msgr2=0x7fa94c152020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:21.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.997+0000 7fa94bfff700 1 --2- 172.21.15.162:0/1825314494 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa94c0b18c0 0x7fa94c152020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:21.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.997+0000 7fa94bfff700 1 -- 172.21.15.162:0/1825314494 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa94c140cd0 con 0x7fa94c05f380 2020-11-28T21:52:21.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.997+0000 7fa950e5e700 1 --2- 172.21.15.162:0/1825314494 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa94c0b3c90 0x7fa94c152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:52:21.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.997+0000 7fa94b7fe700 1 --2- 172.21.15.162:0/1825314494 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa94c0b18c0 0x7fa94c152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:52:21.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.997+0000 7fa94bfff700 1 --2- 172.21.15.162:0/1825314494 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa94c05f380 0x7fa94c05c510 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7fa93c002f30 tx=0x7fa93c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:21.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:20.997+0000 7fa9497fa700 1 -- 172.21.15.162:0/1825314494 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa93c011ca0 con 0x7fa94c05f380 2020-11-28T21:52:21.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:21.001+0000 7fa9497fa700 1 -- 172.21.15.162:0/1825314494 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa93c011e00 con 0x7fa94c05f380 2020-11-28T21:52:21.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:21.001+0000 7fa95165f700 1 -- 172.21.15.162:0/1825314494 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa94c140f70 con 0x7fa94c05f380 2020-11-28T21:52:21.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:21.001+0000 7fa9497fa700 1 -- 172.21.15.162:0/1825314494 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa93c01d640 con 0x7fa94c05f380 2020-11-28T21:52:21.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:21.001+0000 7fa95165f700 1 -- 172.21.15.162:0/1825314494 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa94c13d0d0 con 0x7fa94c05f380 2020-11-28T21:52:21.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:21.001+0000 7fa95165f700 1 -- 172.21.15.162:0/1825314494 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa94c151d30 con 0x7fa94c05f380 2020-11-28T21:52:21.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:21.001+0000 7fa9497fa700 1 -- 172.21.15.162:0/1825314494 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa93c024030 con 0x7fa94c05f380 2020-11-28T21:52:21.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:21.001+0000 7fa9497fa700 1 --2- 172.21.15.162:0/1825314494 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa92c055440 0x7fa92c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:21.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:21.005+0000 7fa94b7fe700 1 --2- 172.21.15.162:0/1825314494 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa92c055440 0x7fa92c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:21.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:21.005+0000 7fa9497fa700 1 -- 172.21.15.162:0/1825314494 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fa93c06f350 con 0x7fa94c05f380 2020-11-28T21:52:21.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:21.005+0000 7fa94b7fe700 1 --2- 172.21.15.162:0/1825314494 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa92c055440 0x7fa92c057900 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7fa940008000 tx=0x7fa940008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:21.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:21.009+0000 7fa9497fa700 1 -- 172.21.15.162:0/1825314494 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa93c045600 con 0x7fa94c05f380 2020-11-28T21:52:21.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:21.149+0000 7fa95165f700 1 -- 172.21.15.162:0/1825314494 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa94c057050 con 0x7fa94c05f380 2020-11-28T21:52:21.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:21.857+0000 7fa9497fa700 1 -- 172.21.15.162:0/1825314494 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v68) v1 ==== 185+0+0 (secure 0 0 0) 0x7fa93c022020 con 0x7fa94c05f380 2020-11-28T21:52:21.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:21.861+0000 7fa95165f700 1 -- 172.21.15.162:0/1825314494 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa92c055440 msgr2=0x7fa92c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:21.864 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:21.861+0000 7fa95165f700 1 --2- 172.21.15.162:0/1825314494 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa92c055440 0x7fa92c057900 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7fa940008000 tx=0x7fa940008040).stop 2020-11-28T21:52:21.864 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:21.861+0000 7fa95165f700 1 -- 172.21.15.162:0/1825314494 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa94c05f380 msgr2=0x7fa94c05c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:21.864 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:21.861+0000 7fa95165f700 1 --2- 172.21.15.162:0/1825314494 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa94c05f380 0x7fa94c05c510 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7fa93c002f30 tx=0x7fa93c015000).stop 2020-11-28T21:52:21.865 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:21.861+0000 7fa95165f700 1 -- 172.21.15.162:0/1825314494 shutdown_connections 2020-11-28T21:52:21.865 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:21.861+0000 7fa95165f700 1 --2- 172.21.15.162:0/1825314494 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa92c055440 0x7fa92c057900 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:21.865 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:21.861+0000 7fa95165f700 1 --2- 172.21.15.162:0/1825314494 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa94c0b3c90 0x7fa94c152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:21.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:21.861+0000 7fa95165f700 1 --2- 172.21.15.162:0/1825314494 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa94c05f380 0x7fa94c05c510 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:21.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:21.861+0000 7fa95165f700 1 --2- 172.21.15.162:0/1825314494 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa94c0b18c0 0x7fa94c152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:21.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:21.861+0000 7fa95165f700 1 -- 172.21.15.162:0/1825314494 >> 172.21.15.162:0/1825314494 conn(0x7fa94c062630 msgr2=0x7fa94c145d50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:21.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:21.861+0000 7fa95165f700 1 -- 172.21.15.162:0/1825314494 shutdown_connections 2020-11-28T21:52:21.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:21.861+0000 7fa95165f700 1 -- 172.21.15.162:0/1825314494 wait complete. 2020-11-28T21:52:21.867 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_metadata' removed 2020-11-28T21:52:21.882 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-11-28T21:52:22.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.001+0000 7f5039522700 1 Processor -- start 2020-11-28T21:52:22.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.001+0000 7f5039522700 1 -- start start 2020-11-28T21:52:22.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.001+0000 7f5039522700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f50340b9ce0 0x7f50341433c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:22.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.001+0000 7f5039522700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f50340605c0 0x7f5034067610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:22.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.001+0000 7f5039522700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5034143950 con 0x7f50340b3c30 2020-11-28T21:52:22.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.001+0000 7f5039522700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50340ba0c0 con 0x7f50340605c0 2020-11-28T21:52:22.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.001+0000 7f5039522700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5034066c30 con 0x7f50340b9ce0 2020-11-28T21:52:22.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.001+0000 7f50337fe700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f50340b3c30 0x7f503413fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:47974/0 (socket says 172.21.15.162:47974) 2020-11-28T21:52:22.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.001+0000 7f50337fe700 1 -- 172.21.15.162:0/1535211596 learned_addr learned my addr 172.21.15.162:0/1535211596 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:52:22.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.005+0000 7f50327fc700 1 --2- 172.21.15.162:0/1535211596 >> v2:172.21.15.118:3300/0 conn(0x7f50340605c0 0x7f5034067610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:22.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.005+0000 7f5032ffd700 1 --2- 172.21.15.162:0/1535211596 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f50340b9ce0 0x7f50341433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:22.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.005+0000 7f5031ffb700 1 -- 172.21.15.162:0/1535211596 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1593637061 0 0) 0x7f5034143950 con 0x7f50340b3c30 2020-11-28T21:52:22.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.005+0000 7f5031ffb700 1 -- 172.21.15.162:0/1535211596 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5014003230 con 0x7f50340b3c30 2020-11-28T21:52:22.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.005+0000 7f5031ffb700 1 -- 172.21.15.162:0/1535211596 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2783173717 0 0) 0x7f5014003230 con 0x7f50340b3c30 2020-11-28T21:52:22.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.005+0000 7f5031ffb700 1 -- 172.21.15.162:0/1535211596 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f50340b9ce0 msgr2=0x7f50341433c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:22.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.005+0000 7f5031ffb700 1 --2- 172.21.15.162:0/1535211596 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f50340b9ce0 0x7f50341433c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:22.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.005+0000 7f5031ffb700 1 -- 172.21.15.162:0/1535211596 >> v2:172.21.15.118:3300/0 conn(0x7f50340605c0 msgr2=0x7f5034067610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:22.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.005+0000 7f5031ffb700 1 --2- 172.21.15.162:0/1535211596 >> v2:172.21.15.118:3300/0 conn(0x7f50340605c0 0x7f5034067610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:22.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.005+0000 7f5031ffb700 1 -- 172.21.15.162:0/1535211596 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5034066ed0 con 0x7f50340b3c30 2020-11-28T21:52:22.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.005+0000 7f5031ffb700 1 -- 172.21.15.162:0/1535211596 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f502c001a50 con 0x7f50340b3c30 2020-11-28T21:52:22.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.005+0000 7f5031ffb700 1 -- 172.21.15.162:0/1535211596 >> v1:172.21.15.162:6789/0 conn(0x7f50340b3c30 legacy=0x7f503413fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:22.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.005+0000 7f5031ffb700 1 --2- 172.21.15.162:0/1535211596 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5014003720 0x7f5014003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:22.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.005+0000 7f5031ffb700 1 --2- 172.21.15.162:0/1535211596 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f50140044d0 0x7f5014004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:22.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.005+0000 7f5031ffb700 1 --2- 172.21.15.162:0/1535211596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5014004e40 0x7f501400b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:22.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.005+0000 7f5031ffb700 1 -- 172.21.15.162:0/1535211596 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50340ba0c0 con 0x7f5014003720 2020-11-28T21:52:22.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.005+0000 7f5031ffb700 1 -- 172.21.15.162:0/1535211596 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5034066c30 con 0x7f5014004e40 2020-11-28T21:52:22.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.005+0000 7f5031ffb700 1 -- 172.21.15.162:0/1535211596 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50140041b0 con 0x7f50140044d0 2020-11-28T21:52:22.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.005+0000 7f50337fe700 1 --2- 172.21.15.162:0/1535211596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5014004e40 0x7f501400b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:22.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.005+0000 7f50327fc700 1 --2- 172.21.15.162:0/1535211596 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f50140044d0 0x7f5014004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:22.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.005+0000 7f5032ffd700 1 --2- 172.21.15.162:0/1535211596 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5014003720 0x7f5014003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:22.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.005+0000 7f50337fe700 1 -- 172.21.15.162:0/1535211596 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f50140044d0 msgr2=0x7f5014004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:22.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.005+0000 7f50337fe700 1 --2- 172.21.15.162:0/1535211596 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f50140044d0 0x7f5014004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:22.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.005+0000 7f50337fe700 1 -- 172.21.15.162:0/1535211596 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5014003720 msgr2=0x7f5014003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:22.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.005+0000 7f50337fe700 1 --2- 172.21.15.162:0/1535211596 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5014003720 0x7f5014003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:22.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.005+0000 7f50337fe700 1 -- 172.21.15.162:0/1535211596 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5014003230 con 0x7f5014004e40 2020-11-28T21:52:22.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.005+0000 7f50327fc700 1 --2- 172.21.15.162:0/1535211596 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f50140044d0 0x7f5014004950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:52:22.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.005+0000 7f50337fe700 1 --2- 172.21.15.162:0/1535211596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5014004e40 0x7f501400b720 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7f502c001760 tx=0x7f5034067060).ready entity=mon.1 client_cookie=61d62a305f806e79 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:22.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.005+0000 7f5031ffb700 1 -- 172.21.15.162:0/1535211596 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f502c005b70 con 0x7f5014004e40 2020-11-28T21:52:22.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.005+0000 7f5031ffb700 1 -- 172.21.15.162:0/1535211596 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f502c005cd0 con 0x7f5014004e40 2020-11-28T21:52:22.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.005+0000 7f5031ffb700 1 -- 172.21.15.162:0/1535211596 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f502c007300 con 0x7f5014004e40 2020-11-28T21:52:22.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.005+0000 7f5039522700 1 -- 172.21.15.162:0/1535211596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5014004e40 msgr2=0x7f501400b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:22.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.005+0000 7f5039522700 1 --2- 172.21.15.162:0/1535211596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5014004e40 0x7f501400b720 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7f502c001760 tx=0x7f5034067060).stop 2020-11-28T21:52:22.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.005+0000 7f50337fe700 1 -- 172.21.15.162:0/1535211596 reap_dead start 2020-11-28T21:52:22.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.005+0000 7f5039522700 1 -- 172.21.15.162:0/1535211596 shutdown_connections 2020-11-28T21:52:22.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.005+0000 7f5039522700 1 -- 172.21.15.162:0/1535211596 >> 172.21.15.162:0/1535211596 conn(0x7f503405c430 msgr2=0x7f50340b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:22.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.005+0000 7f5039522700 1 -- 172.21.15.162:0/1535211596 shutdown_connections 2020-11-28T21:52:22.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.005+0000 7f5039522700 1 -- 172.21.15.162:0/1535211596 wait complete. 2020-11-28T21:52:22.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.009+0000 7f5039522700 1 Processor -- start 2020-11-28T21:52:22.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.009+0000 7f5039522700 1 -- start start 2020-11-28T21:52:22.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.009+0000 7f5039522700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f503409e8a0 0x7f503413ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:22.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.009+0000 7f5039522700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50340b32b0 0x7f503413f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:22.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.009+0000 7f5039522700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f50341550d0 0x7f5034155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:22.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.009+0000 7f5039522700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503409ec80 con 0x7f503409e8a0 2020-11-28T21:52:22.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.009+0000 7f5039522700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503413f860 con 0x7f50340b32b0 2020-11-28T21:52:22.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.009+0000 7f5039522700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50340ad230 con 0x7f50341550d0 2020-11-28T21:52:22.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.009+0000 7f50327fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50340b32b0 0x7f503413f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:22.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.009+0000 7f50327fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50340b32b0 0x7f503413f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41140/0 (socket says 172.21.15.162:41140) 2020-11-28T21:52:22.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.009+0000 7f50327fc700 1 -- 172.21.15.162:0/228976826 learned_addr learned my addr 172.21.15.162:0/228976826 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:22.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.013+0000 7f5032ffd700 1 --2- 172.21.15.162:0/228976826 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f503409e8a0 0x7f503413ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:22.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.013+0000 7f50337fe700 1 --2- 172.21.15.162:0/228976826 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f50341550d0 0x7f5034155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:22.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.013+0000 7f50327fc700 1 -- 172.21.15.162:0/228976826 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f50341550d0 msgr2=0x7f5034155580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:22.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.013+0000 7f50327fc700 1 --2- 172.21.15.162:0/228976826 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f50341550d0 0x7f5034155580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:22.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.013+0000 7f50327fc700 1 -- 172.21.15.162:0/228976826 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f503409e8a0 msgr2=0x7f503413ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:22.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.013+0000 7f50327fc700 1 --2- 172.21.15.162:0/228976826 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f503409e8a0 0x7f503413ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:22.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.013+0000 7f50327fc700 1 -- 172.21.15.162:0/228976826 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50340ad4d0 con 0x7f50340b32b0 2020-11-28T21:52:22.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.013+0000 7f50327fc700 1 --2- 172.21.15.162:0/228976826 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50340b32b0 0x7f503413f370 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7f501c008000 tx=0x7f501c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:22.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.013+0000 7f502bfff700 1 -- 172.21.15.162:0/228976826 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f501c00e690 con 0x7f50340b32b0 2020-11-28T21:52:22.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.013+0000 7f502bfff700 1 -- 172.21.15.162:0/228976826 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f501c00e7f0 con 0x7f50340b32b0 2020-11-28T21:52:22.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.013+0000 7f502bfff700 1 -- 172.21.15.162:0/228976826 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f501c00eb50 con 0x7f50340b32b0 2020-11-28T21:52:22.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.013+0000 7f5039522700 1 -- 172.21.15.162:0/228976826 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50340ad770 con 0x7f50340b32b0 2020-11-28T21:52:22.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.013+0000 7f5039522700 1 -- 172.21.15.162:0/228976826 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50340af810 con 0x7f50340b32b0 2020-11-28T21:52:22.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.013+0000 7f5039522700 1 -- 172.21.15.162:0/228976826 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50340ad900 con 0x7f50340b32b0 2020-11-28T21:52:22.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.013+0000 7f502bfff700 1 -- 172.21.15.162:0/228976826 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f501c024070 con 0x7f50340b32b0 2020-11-28T21:52:22.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.017+0000 7f502bfff700 1 --2- 172.21.15.162:0/228976826 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5018055330 0x7f50180577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:22.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.017+0000 7f502bfff700 1 -- 172.21.15.162:0/228976826 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5721+0+0 (secure 0 0 0) 0x7f501c06e5e0 con 0x7f50340b32b0 2020-11-28T21:52:22.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.017+0000 7f5032ffd700 1 --2- 172.21.15.162:0/228976826 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5018055330 0x7f50180577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:22.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.017+0000 7f5032ffd700 1 --2- 172.21.15.162:0/228976826 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5018055330 0x7f50180577f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f5024006000 tx=0x7f5024008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:22.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.025+0000 7f502bfff700 1 -- 172.21.15.162:0/228976826 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f501c044930 con 0x7f50340b32b0 2020-11-28T21:52:22.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.165+0000 7f5039522700 1 -- 172.21.15.162:0/228976826 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5034057050 con 0x7f50340b32b0 2020-11-28T21:52:22.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.865+0000 7f502bfff700 1 -- 172.21.15.162:0/228976826 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v69) v1 ==== 173+0+0 (secure 0 0 0) 0x7f501c00ecb0 con 0x7f50340b32b0 2020-11-28T21:52:22.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.869+0000 7f5039522700 1 -- 172.21.15.162:0/228976826 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5018055330 msgr2=0x7f50180577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:22.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.869+0000 7f5039522700 1 --2- 172.21.15.162:0/228976826 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5018055330 0x7f50180577f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f5024006000 tx=0x7f5024008040).stop 2020-11-28T21:52:22.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.869+0000 7f5039522700 1 -- 172.21.15.162:0/228976826 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50340b32b0 msgr2=0x7f503413f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:22.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.869+0000 7f5039522700 1 --2- 172.21.15.162:0/228976826 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50340b32b0 0x7f503413f370 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7f501c008000 tx=0x7f501c008040).stop 2020-11-28T21:52:22.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.869+0000 7f5039522700 1 -- 172.21.15.162:0/228976826 shutdown_connections 2020-11-28T21:52:22.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.869+0000 7f5039522700 1 --2- 172.21.15.162:0/228976826 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5018055330 0x7f50180577f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:22.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.869+0000 7f5039522700 1 --2- 172.21.15.162:0/228976826 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f50341550d0 0x7f5034155580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:22.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.869+0000 7f5039522700 1 --2- 172.21.15.162:0/228976826 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f503409e8a0 0x7f503413ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:22.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.869+0000 7f5039522700 1 --2- 172.21.15.162:0/228976826 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50340b32b0 0x7f503413f370 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:22.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.869+0000 7f5039522700 1 -- 172.21.15.162:0/228976826 >> 172.21.15.162:0/228976826 conn(0x7f503405c430 msgr2=0x7f50340b7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:22.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.869+0000 7f5039522700 1 -- 172.21.15.162:0/228976826 shutdown_connections 2020-11-28T21:52:22.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:22.869+0000 7f5039522700 1 -- 172.21.15.162:0/228976826 wait complete. 2020-11-28T21:52:22.879 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_data' removed 2020-11-28T21:52:22.892 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-28T21:52:22.892 INFO:tasks.ceph.mds.a:Stopping old one... 2020-11-28T21:52:22.892 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-11-28T21:52:22.893 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T21:52:22.893 INFO:tasks.ceph.mds.c:Restarting daemon 2020-11-28T21:52:22.894 INFO:tasks.ceph.mds.c:Stopping old one... 2020-11-28T21:52:22.894 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2020-11-28T21:52:22.895 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T21:52:22.895 INFO:tasks.ceph.mds.b:Restarting daemon 2020-11-28T21:52:22.895 INFO:tasks.ceph.mds.b:Stopping old one... 2020-11-28T21:52:22.896 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-11-28T21:52:22.897 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T21:52:22.905 INFO:tasks.ceph.mds.a:Stopped 2020-11-28T21:52:22.906 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-28T21:52:22.909 INFO:tasks.ceph.mds.b:Stopped 2020-11-28T21:52:22.909 INFO:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-11-28T21:52:22.910 INFO:tasks.ceph.mds.c:Stopped 2020-11-28T21:52:22.911 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2020-11-28T21:52:22.951 INFO:tasks.ceph.mds.b:Started 2020-11-28T21:52:22.952 DEBUG:teuthology.parallel:result is None 2020-11-28T21:52:22.954 INFO:tasks.ceph.mds.a:Started 2020-11-28T21:52:22.955 DEBUG:teuthology.parallel:result is None 2020-11-28T21:52:22.955 INFO:tasks.ceph.mds.c:Started 2020-11-28T21:52:22.956 DEBUG:teuthology.parallel:result is None 2020-11-28T21:52:22.956 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist clear 2020-11-28T21:52:23.024 INFO:tasks.ceph.mds.c.smithi162.stdout:starting mds.c at 2020-11-28T21:52:23.028 INFO:tasks.ceph.mds.a.smithi162.stdout:starting mds.a at 2020-11-28T21:52:23.029 INFO:tasks.ceph.mds.b.smithi118.stdout:starting mds.b at 2020-11-28T21:52:23.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.077+0000 7f6260b06700 1 Processor -- start 2020-11-28T21:52:23.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.077+0000 7f6260b06700 1 -- start start 2020-11-28T21:52:23.083 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.081+0000 7f6260b06700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f625c09b470 0x7f625c0b33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:23.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.081+0000 7f6260b06700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f625c098660 0x7f625c13f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:23.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.081+0000 7f6260b06700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f625c0b4fb0 con 0x7f625c0b4bd0 2020-11-28T21:52:23.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.081+0000 7f6260b06700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f625c13f430 con 0x7f625c09b470 2020-11-28T21:52:23.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.081+0000 7f6260b06700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f625c098be0 con 0x7f625c098660 2020-11-28T21:52:23.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.081+0000 7f6253fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f625c09b470 0x7f625c0b33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:23.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.081+0000 7f625b7fe700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f625c0b4bd0 0x7f625c09ad60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:48014/0 (socket says 172.21.15.162:48014) 2020-11-28T21:52:23.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.081+0000 7f625b7fe700 1 -- 172.21.15.162:0/139853791 learned_addr learned my addr 172.21.15.162:0/139853791 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:52:23.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.081+0000 7f625bfff700 1 --2- 172.21.15.162:0/139853791 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f625c098660 0x7f625c13f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:23.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.081+0000 7f6253fff700 1 -- 172.21.15.162:0/139853791 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f625c098660 msgr2=0x7f625c13f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:23.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.081+0000 7f6253fff700 1 --2- 172.21.15.162:0/139853791 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f625c098660 0x7f625c13f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:23.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.081+0000 7f6253fff700 1 -- 172.21.15.162:0/139853791 >> v1:172.21.15.162:6789/0 conn(0x7f625c0b4bd0 legacy=0x7f625c09ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:23.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.081+0000 7f625affd700 1 -- 172.21.15.162:0/139853791 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2594161511 0 0) 0x7f625c0b4fb0 con 0x7f625c0b4bd0 2020-11-28T21:52:23.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.081+0000 7f6253fff700 1 -- 172.21.15.162:0/139853791 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f625c0b3e90 con 0x7f625c09b470 2020-11-28T21:52:23.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.081+0000 7f625bfff700 1 --2- 172.21.15.162:0/139853791 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f625c098660 0x7f625c13f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:52:23.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.081+0000 7f6253fff700 1 --2- 172.21.15.162:0/139853791 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f625c09b470 0x7f625c0b33f0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7f6244002f30 tx=0x7f6244006dc0).ready entity=mon.1 client_cookie=7fb6c9b874118414 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:23.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.081+0000 7f625affd700 1 -- 172.21.15.162:0/139853791 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6244009760 con 0x7f625c09b470 2020-11-28T21:52:23.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.081+0000 7f625affd700 1 -- 172.21.15.162:0/139853791 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62440098c0 con 0x7f625c09b470 2020-11-28T21:52:23.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.081+0000 7f625affd700 1 -- 172.21.15.162:0/139853791 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6244009c20 con 0x7f625c09b470 2020-11-28T21:52:23.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.081+0000 7f6260b06700 1 -- 172.21.15.162:0/139853791 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f625c09b470 msgr2=0x7f625c0b33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:23.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.081+0000 7f6260b06700 1 --2- 172.21.15.162:0/139853791 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f625c09b470 0x7f625c0b33f0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7f6244002f30 tx=0x7f6244006dc0).stop 2020-11-28T21:52:23.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.081+0000 7f6260b06700 1 -- 172.21.15.162:0/139853791 shutdown_connections 2020-11-28T21:52:23.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.081+0000 7f6260b06700 1 --2- 172.21.15.162:0/139853791 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f625c098660 0x7f625c13f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:23.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.081+0000 7f6260b06700 1 --2- 172.21.15.162:0/139853791 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f625c09b470 0x7f625c0b33f0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:23.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.081+0000 7f6260b06700 1 -- 172.21.15.162:0/139853791 >> 172.21.15.162:0/139853791 conn(0x7f625c0b54f0 msgr2=0x7f625c0b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:23.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.081+0000 7f6260b06700 1 -- 172.21.15.162:0/139853791 shutdown_connections 2020-11-28T21:52:23.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.081+0000 7f6260b06700 1 -- 172.21.15.162:0/139853791 wait complete. 2020-11-28T21:52:23.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.085+0000 7f6260b06700 1 Processor -- start 2020-11-28T21:52:23.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.085+0000 7f6260b06700 1 -- start start 2020-11-28T21:52:23.094 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.085+0000 7f6260b06700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f625c098660 0x7f625c13c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:23.094 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.085+0000 7f6260b06700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f625c0b4bd0 0x7f625c13e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:23.095 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.085+0000 7f6260b06700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f625c098f30 0x7f625c0ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:23.095 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.085+0000 7f6260b06700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f625c13c3b0 con 0x7f625c098660 2020-11-28T21:52:23.095 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.085+0000 7f6260b06700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f625c099f00 con 0x7f625c098f30 2020-11-28T21:52:23.095 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.085+0000 7f6260b06700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f625c13e790 con 0x7f625c0b4bd0 2020-11-28T21:52:23.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.085+0000 7f625bfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f625c098f30 0x7f625c0ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:23.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.085+0000 7f625bfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f625c098f30 0x7f625c0ad6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41180/0 (socket says 172.21.15.162:41180) 2020-11-28T21:52:23.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.085+0000 7f625bfff700 1 -- 172.21.15.162:0/2738299159 learned_addr learned my addr 172.21.15.162:0/2738299159 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:23.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.085+0000 7f625b7fe700 1 --2- 172.21.15.162:0/2738299159 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f625c098660 0x7f625c13c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:23.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.085+0000 7f6253fff700 1 --2- 172.21.15.162:0/2738299159 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f625c0b4bd0 0x7f625c13e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:23.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.085+0000 7f625bfff700 1 -- 172.21.15.162:0/2738299159 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f625c0b4bd0 msgr2=0x7f625c13e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:23.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.085+0000 7f625bfff700 1 --2- 172.21.15.162:0/2738299159 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f625c0b4bd0 0x7f625c13e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:23.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.085+0000 7f625bfff700 1 -- 172.21.15.162:0/2738299159 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f625c098660 msgr2=0x7f625c13c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:23.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.085+0000 7f625bfff700 1 --2- 172.21.15.162:0/2738299159 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f625c098660 0x7f625c13c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:23.098 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.085+0000 7f625bfff700 1 -- 172.21.15.162:0/2738299159 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f625c09a160 con 0x7f625c098f30 2020-11-28T21:52:23.098 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.085+0000 7f625bfff700 1 --2- 172.21.15.162:0/2738299159 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f625c098f30 0x7f625c0ad6d0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7f625c13e920 tx=0x7f62540091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:23.098 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.085+0000 7f62597fa700 1 -- 172.21.15.162:0/2738299159 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f625400cb50 con 0x7f625c098f30 2020-11-28T21:52:23.098 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.085+0000 7f62597fa700 1 -- 172.21.15.162:0/2738299159 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f625400ccb0 con 0x7f625c098f30 2020-11-28T21:52:23.098 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.085+0000 7f6260b06700 1 -- 172.21.15.162:0/2738299159 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f625c13be70 con 0x7f625c098f30 2020-11-28T21:52:23.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.085+0000 7f62597fa700 1 -- 172.21.15.162:0/2738299159 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62540125e0 con 0x7f625c098f30 2020-11-28T21:52:23.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.085+0000 7f6260b06700 1 -- 172.21.15.162:0/2738299159 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f625c0993c0 con 0x7f625c098f30 2020-11-28T21:52:23.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.089+0000 7f6260b06700 1 -- 172.21.15.162:0/2738299159 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f625c0add60 con 0x7f625c098f30 2020-11-28T21:52:23.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.089+0000 7f62597fa700 1 -- 172.21.15.162:0/2738299159 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6254022070 con 0x7f625c098f30 2020-11-28T21:52:23.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.089+0000 7f62597fa700 1 --2- 172.21.15.162:0/2738299159 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f62340553b0 0x7f6234057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:23.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.089+0000 7f62597fa700 1 -- 172.21.15.162:0/2738299159 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f625406e550 con 0x7f625c098f30 2020-11-28T21:52:23.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.089+0000 7f625b7fe700 1 --2- 172.21.15.162:0/2738299159 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f62340553b0 0x7f6234057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:23.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.089+0000 7f625b7fe700 1 --2- 172.21.15.162:0/2738299159 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f62340553b0 0x7f6234057870 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f624c00d000 tx=0x7f624c007bd0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:23.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.093+0000 7f62597fa700 1 -- 172.21.15.162:0/2738299159 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6254044a20 con 0x7f625c098f30 2020-11-28T21:52:23.113 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2020-11-28T21:52:23.114 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2020-11-28T21:52:23.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:23.249+0000 7f6260b06700 1 -- 172.21.15.162:0/2738299159 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f625c13df90 con 0x7f625c098f30 2020-11-28T21:52:24.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.073+0000 7f62597fa700 1 -- 172.21.15.162:0/2738299159 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v71) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6254016020 con 0x7f625c098f30 2020-11-28T21:52:24.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.077+0000 7f6260b06700 1 -- 172.21.15.162:0/2738299159 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f62340553b0 msgr2=0x7f6234057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:24.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.077+0000 7f6260b06700 1 --2- 172.21.15.162:0/2738299159 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f62340553b0 0x7f6234057870 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f624c00d000 tx=0x7f624c007bd0).stop 2020-11-28T21:52:24.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.077+0000 7f6260b06700 1 -- 172.21.15.162:0/2738299159 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f625c098f30 msgr2=0x7f625c0ad6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:24.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.077+0000 7f6260b06700 1 --2- 172.21.15.162:0/2738299159 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f625c098f30 0x7f625c0ad6d0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7f625c13e920 tx=0x7f62540091a0).stop 2020-11-28T21:52:24.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.077+0000 7f6260b06700 1 -- 172.21.15.162:0/2738299159 shutdown_connections 2020-11-28T21:52:24.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.077+0000 7f6260b06700 1 --2- 172.21.15.162:0/2738299159 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f62340553b0 0x7f6234057870 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:24.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.077+0000 7f6260b06700 1 --2- 172.21.15.162:0/2738299159 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f625c098f30 0x7f625c0ad6d0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:24.083 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.077+0000 7f6260b06700 1 --2- 172.21.15.162:0/2738299159 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f625c098660 0x7f625c13c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:24.083 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.077+0000 7f6260b06700 1 --2- 172.21.15.162:0/2738299159 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f625c0b4bd0 0x7f625c13e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:24.083 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.077+0000 7f6260b06700 1 -- 172.21.15.162:0/2738299159 >> 172.21.15.162:0/2738299159 conn(0x7f625c0b54f0 msgr2=0x7f625c0b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:24.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.077+0000 7f6260b06700 1 -- 172.21.15.162:0/2738299159 shutdown_connections 2020-11-28T21:52:24.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.077+0000 7f6260b06700 1 -- 172.21.15.162:0/2738299159 wait complete. 2020-11-28T21:52:24.085 INFO:teuthology.orchestra.run.smithi162.stderr: removed all blacklist entries 2020-11-28T21:52:24.094 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2020-11-28T21:52:24.095 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth ls --format=json-pretty 2020-11-28T21:52:24.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.205+0000 7f5dc962f700 1 Processor -- start 2020-11-28T21:52:24.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.205+0000 7f5dc962f700 1 -- start start 2020-11-28T21:52:24.213 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.209+0000 7f5dc962f700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5dc40b11b0 0x7f5dc4140fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:24.213 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.209+0000 7f5dc962f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5dc40b1d70 0x7f5dc414a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:24.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.209+0000 7f5dc962f700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dc40b2870 con 0x7f5dc40b1990 2020-11-28T21:52:24.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.209+0000 7f5dc962f700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dc4146e20 con 0x7f5dc40b11b0 2020-11-28T21:52:24.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.209+0000 7f5dc962f700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc40b22e0 con 0x7f5dc40b1d70 2020-11-28T21:52:24.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.209+0000 7f5dc27fc700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5dc40b11b0 0x7f5dc4140fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:24.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.209+0000 7f5dc37fe700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5dc40b1d70 0x7f5dc414a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:24.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.209+0000 7f5dc2ffd700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f5dc40b1990 0x7f5dc4140890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.162:38182/0 (socket says 172.21.15.162:38182) 2020-11-28T21:52:24.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.209+0000 7f5dc2ffd700 1 -- 172.21.15.162:0/2637368898 learned_addr learned my addr 172.21.15.162:0/2637368898 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:52:24.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.209+0000 7f5dc37fe700 1 -- 172.21.15.162:0/2637368898 >> v2:172.21.15.118:3300/0 conn(0x7f5dc40b11b0 msgr2=0x7f5dc4140fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:24.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.209+0000 7f5dc37fe700 1 --2- 172.21.15.162:0/2637368898 >> v2:172.21.15.118:3300/0 conn(0x7f5dc40b11b0 0x7f5dc4140fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:24.216 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.209+0000 7f5dc37fe700 1 -- 172.21.15.162:0/2637368898 >> v1:172.21.15.118:6789/0 conn(0x7f5dc40b1990 legacy=0x7f5dc4140890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:24.216 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.209+0000 7f5dc37fe700 1 -- 172.21.15.162:0/2637368898 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc40b0740 con 0x7f5dc40b1d70 2020-11-28T21:52:24.216 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.209+0000 7f5dc27fc700 1 --2- 172.21.15.162:0/2637368898 >> v2:172.21.15.118:3300/0 conn(0x7f5dc40b11b0 0x7f5dc4140fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:52:24.217 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.209+0000 7f5dc37fe700 1 --2- 172.21.15.162:0/2637368898 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5dc40b1d70 0x7f5dc414a770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f5dbc002f30 tx=0x7f5dbc006dc0).ready entity=mon.2 client_cookie=eb4332056409098c server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:24.217 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.209+0000 7f5dc1ffb700 1 -- 172.21.15.162:0/2637368898 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5dbc009760 con 0x7f5dc40b1d70 2020-11-28T21:52:24.217 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.209+0000 7f5dc1ffb700 1 -- 172.21.15.162:0/2637368898 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dbc0098c0 con 0x7f5dc40b1d70 2020-11-28T21:52:24.218 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.209+0000 7f5dc1ffb700 1 -- 172.21.15.162:0/2637368898 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5dbc009c20 con 0x7f5dc40b1d70 2020-11-28T21:52:24.218 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.209+0000 7f5dc962f700 1 -- 172.21.15.162:0/2637368898 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5dc40b1d70 msgr2=0x7f5dc414a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:24.218 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.209+0000 7f5dc962f700 1 --2- 172.21.15.162:0/2637368898 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5dc40b1d70 0x7f5dc414a770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f5dbc002f30 tx=0x7f5dbc006dc0).stop 2020-11-28T21:52:24.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.209+0000 7f5dc962f700 1 -- 172.21.15.162:0/2637368898 shutdown_connections 2020-11-28T21:52:24.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.209+0000 7f5dc962f700 1 --2- 172.21.15.162:0/2637368898 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5dc40b1d70 0x7f5dc414a770 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:24.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.209+0000 7f5dc962f700 1 --2- 172.21.15.162:0/2637368898 >> v2:172.21.15.118:3300/0 conn(0x7f5dc40b11b0 0x7f5dc4140fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:24.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.209+0000 7f5dc962f700 1 -- 172.21.15.162:0/2637368898 >> 172.21.15.162:0/2637368898 conn(0x7f5dc405b970 msgr2=0x7f5dc405bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:24.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.209+0000 7f5dc962f700 1 -- 172.21.15.162:0/2637368898 shutdown_connections 2020-11-28T21:52:24.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.209+0000 7f5dc962f700 1 -- 172.21.15.162:0/2637368898 wait complete. 2020-11-28T21:52:24.224 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.213+0000 7f5dc962f700 1 Processor -- start 2020-11-28T21:52:24.224 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.213+0000 7f5dc962f700 1 -- start start 2020-11-28T21:52:24.224 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.213+0000 7f5dc962f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5dc40b11b0 0x7f5dc41566c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:24.224 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.213+0000 7f5dc962f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5dc40b1990 0x7f5dc414d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:24.225 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.213+0000 7f5dc962f700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5dc40b1d70 0x7f5dc41590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:24.225 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.213+0000 7f5dc962f700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc414e500 con 0x7f5dc40b1d70 2020-11-28T21:52:24.225 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.213+0000 7f5dc962f700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc41430f0 con 0x7f5dc40b1990 2020-11-28T21:52:24.225 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.213+0000 7f5dc962f700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc414bec0 con 0x7f5dc40b11b0 2020-11-28T21:52:24.226 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.213+0000 7f5dc27fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5dc40b1990 0x7f5dc414d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:24.226 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.213+0000 7f5dc2ffd700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5dc40b11b0 0x7f5dc41566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:24.226 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.213+0000 7f5dc37fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5dc40b1d70 0x7f5dc41590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:24.226 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.213+0000 7f5dc27fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5dc40b1990 0x7f5dc414d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41196/0 (socket says 172.21.15.162:41196) 2020-11-28T21:52:24.227 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.213+0000 7f5dc27fc700 1 -- 172.21.15.162:0/878834241 learned_addr learned my addr 172.21.15.162:0/878834241 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:24.227 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.213+0000 7f5dc27fc700 1 -- 172.21.15.162:0/878834241 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5dc40b11b0 msgr2=0x7f5dc41566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:24.227 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.213+0000 7f5dc27fc700 1 --2- 172.21.15.162:0/878834241 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5dc40b11b0 0x7f5dc41566c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:24.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.213+0000 7f5dc27fc700 1 -- 172.21.15.162:0/878834241 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5dc40b1d70 msgr2=0x7f5dc41590b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:24.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.213+0000 7f5dc27fc700 1 --2- 172.21.15.162:0/878834241 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5dc40b1d70 0x7f5dc41590b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:24.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.213+0000 7f5dc27fc700 1 -- 172.21.15.162:0/878834241 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc4143fe0 con 0x7f5dc40b1990 2020-11-28T21:52:24.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.213+0000 7f5dc27fc700 1 --2- 172.21.15.162:0/878834241 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5dc40b1990 0x7f5dc414d870 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7f5db400c000 tx=0x7f5db40091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:24.229 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.213+0000 7f5dbbfff700 1 -- 172.21.15.162:0/878834241 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5db400cbd0 con 0x7f5dc40b1990 2020-11-28T21:52:24.229 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.213+0000 7f5dbbfff700 1 -- 172.21.15.162:0/878834241 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db400cd30 con 0x7f5dc40b1990 2020-11-28T21:52:24.229 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.213+0000 7f5dc962f700 1 -- 172.21.15.162:0/878834241 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dc414e230 con 0x7f5dc40b1990 2020-11-28T21:52:24.229 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.213+0000 7f5dbbfff700 1 -- 172.21.15.162:0/878834241 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5db400cea0 con 0x7f5dc40b1990 2020-11-28T21:52:24.230 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.213+0000 7f5dc962f700 1 -- 172.21.15.162:0/878834241 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc41448a0 con 0x7f5dc40b1990 2020-11-28T21:52:24.230 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.217+0000 7f5dc962f700 1 -- 172.21.15.162:0/878834241 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dc4156510 con 0x7f5dc40b1990 2020-11-28T21:52:24.230 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.217+0000 7f5dbbfff700 1 -- 172.21.15.162:0/878834241 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5db4022070 con 0x7f5dc40b1990 2020-11-28T21:52:24.231 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.217+0000 7f5dbbfff700 1 --2- 172.21.15.162:0/878834241 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5da80553b0 0x7f5da8057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:24.231 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.217+0000 7f5dc2ffd700 1 --2- 172.21.15.162:0/878834241 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5da80553b0 0x7f5da8057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:24.231 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.217+0000 7f5dbbfff700 1 -- 172.21.15.162:0/878834241 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f5db406e620 con 0x7f5dc40b1990 2020-11-28T21:52:24.232 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.217+0000 7f5dc2ffd700 1 --2- 172.21.15.162:0/878834241 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5da80553b0 0x7f5da8057870 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f5dac00d000 tx=0x7f5dac009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:24.232 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.221+0000 7f5dbbfff700 1 -- 172.21.15.162:0/878834241 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5db4044ac0 con 0x7f5dc40b1990 2020-11-28T21:52:24.417 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.413+0000 7f5dc962f700 1 -- 172.21.15.162:0/878834241 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f5dc414c0c0 con 0x7f5dc40b1990 2020-11-28T21:52:24.417 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.413+0000 7f5dbbfff700 1 -- 172.21.15.162:0/878834241 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+4255 (secure 0 0 0) 0x7f5db40480e0 con 0x7f5dc40b1990 2020-11-28T21:52:24.418 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:52:24.418 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2020-11-28T21:52:24.418 INFO:teuthology.orchestra.run.smithi162.stdout: "auth_dump": [ 2020-11-28T21:52:24.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:24.419 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mds.a", 2020-11-28T21:52:24.419 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfWxUpKxAAIUDwQRKwz8H4Qk9qE356Bg==", 2020-11-28T21:52:24.419 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:52:24.419 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:52:24.419 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:52:24.420 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:52:24.420 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:52:24.420 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:24.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:24.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:24.422 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mds.b", 2020-11-28T21:52:24.422 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfKY2zORAAaO65etMZiYcaTDNX0nsEPw==", 2020-11-28T21:52:24.423 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:52:24.423 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:52:24.424 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:52:24.424 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:52:24.424 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:52:24.424 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:24.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:24.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:24.425 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mds.c", 2020-11-28T21:52:24.426 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfdQbRMxAAWlcjesxTZzZnvKCQneSjjQ==", 2020-11-28T21:52:24.426 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:52:24.426 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:52:24.427 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:52:24.427 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:52:24.427 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:52:24.428 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:24.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:24.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:24.429 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.0", 2020-11-28T21:52:24.429 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQADxcJfF3WLLxAAxsB5YPjH1yjv/vhlRbijqw==", 2020-11-28T21:52:24.429 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:52:24.429 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:52:24.430 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:52:24.430 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:52:24.430 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:24.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:24.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:24.431 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.1", 2020-11-28T21:52:24.431 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAGxcJfKYbBFhAA1x4DzFtkQQv6uWLIdOi1qQ==", 2020-11-28T21:52:24.431 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:52:24.432 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:52:24.432 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:52:24.433 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:52:24.433 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:24.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:24.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:24.433 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.2", 2020-11-28T21:52:24.434 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAIxcJfvcYUOhAASENjM0HfAxxyFe13mmE36Q==", 2020-11-28T21:52:24.434 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:52:24.434 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:52:24.434 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:52:24.435 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:52:24.435 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:24.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:24.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:24.435 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.3", 2020-11-28T21:52:24.436 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQALxcJfc/OHIRAAFYx3s8Ssttt15hjoH04tow==", 2020-11-28T21:52:24.436 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:52:24.436 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:52:24.436 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:52:24.436 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:52:24.437 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:24.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:24.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:24.437 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.4", 2020-11-28T21:52:24.437 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAPxcJffGWqCRAAb8CCN5oUPkdDh6BJH5HEnw==", 2020-11-28T21:52:24.438 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:52:24.438 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:52:24.438 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:52:24.438 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:52:24.438 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:24.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:24.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:24.439 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.5", 2020-11-28T21:52:24.439 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQARxcJfo8GKKRAAfbsCDb4L2wULiKmHyzu7/A==", 2020-11-28T21:52:24.439 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:52:24.440 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:52:24.440 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:52:24.440 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:52:24.440 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:24.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:24.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:24.441 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.6", 2020-11-28T21:52:24.441 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAUxcJf8Y2MEBAAqQX6rX3LK0zodlI2wK4JrA==", 2020-11-28T21:52:24.442 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:52:24.442 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:52:24.442 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:52:24.443 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:52:24.443 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:24.443 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:24.443 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:24.444 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.7", 2020-11-28T21:52:24.444 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAWxcJfCLBdMhAAf8QixQMQDrfZOGWN8rHe5g==", 2020-11-28T21:52:24.444 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:52:24.444 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:52:24.444 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:52:24.445 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:52:24.445 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:24.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:24.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:24.446 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "client.0", 2020-11-28T21:52:24.448 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQABxcJfS9vDAxAAUk4FU8FVt4if/CRH5ZisOA==", 2020-11-28T21:52:24.448 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:52:24.448 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:52:24.449 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow r", 2020-11-28T21:52:24.449 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow rw pool=cephfs_data" 2020-11-28T21:52:24.449 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:24.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:24.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:24.451 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "client.1", 2020-11-28T21:52:24.451 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQABxcJfuCU8BhAAXUSoSK3D0XRo58uHNzcjXg==", 2020-11-28T21:52:24.451 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:52:24.452 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:52:24.452 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow r", 2020-11-28T21:52:24.453 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow rw pool=cephfs_data" 2020-11-28T21:52:24.453 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:24.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:24.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:24.454 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "client.admin", 2020-11-28T21:52:24.455 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfdlTWBxAApzAxlVjzdnETQF//Vnsg5w==", 2020-11-28T21:52:24.455 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:52:24.455 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow *", 2020-11-28T21:52:24.456 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:52:24.456 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:52:24.457 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:52:24.457 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:24.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:24.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:24.458 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mgr.x", 2020-11-28T21:52:24.459 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfZhdvKBAAT7zCCsD2OVCHjvddGLxQ/A==", 2020-11-28T21:52:24.459 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:52:24.460 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow *", 2020-11-28T21:52:24.460 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow profile mgr", 2020-11-28T21:52:24.460 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:52:24.461 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:24.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:24.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:24.462 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mgr.y", 2020-11-28T21:52:24.462 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfs8fvJRAAP0Atp074EeMHiTtpCCVHuA==", 2020-11-28T21:52:24.463 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:52:24.463 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow *", 2020-11-28T21:52:24.463 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow profile mgr", 2020-11-28T21:52:24.464 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:52:24.464 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:24.465 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:24.465 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:52:24.465 INFO:teuthology.orchestra.run.smithi162.stdout:} 2020-11-28T21:52:24.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.417+0000 7f5dc962f700 1 -- 172.21.15.162:0/878834241 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5da80553b0 msgr2=0x7f5da8057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:24.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.417+0000 7f5dc962f700 1 --2- 172.21.15.162:0/878834241 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5da80553b0 0x7f5da8057870 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f5dac00d000 tx=0x7f5dac009040).stop 2020-11-28T21:52:24.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.417+0000 7f5dc962f700 1 -- 172.21.15.162:0/878834241 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5dc40b1990 msgr2=0x7f5dc414d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:24.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.417+0000 7f5dc962f700 1 --2- 172.21.15.162:0/878834241 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5dc40b1990 0x7f5dc414d870 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7f5db400c000 tx=0x7f5db40091a0).stop 2020-11-28T21:52:24.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.417+0000 7f5dc962f700 1 -- 172.21.15.162:0/878834241 shutdown_connections 2020-11-28T21:52:24.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.417+0000 7f5dc962f700 1 --2- 172.21.15.162:0/878834241 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5da80553b0 0x7f5da8057870 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:24.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.417+0000 7f5dc962f700 1 --2- 172.21.15.162:0/878834241 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5dc40b11b0 0x7f5dc41566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:24.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.417+0000 7f5dc962f700 1 --2- 172.21.15.162:0/878834241 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5dc40b1d70 0x7f5dc41590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:24.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.417+0000 7f5dc962f700 1 --2- 172.21.15.162:0/878834241 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5dc40b1990 0x7f5dc414d870 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:24.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.417+0000 7f5dc962f700 1 -- 172.21.15.162:0/878834241 >> 172.21.15.162:0/878834241 conn(0x7f5dc405b970 msgr2=0x7f5dc40ac1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:24.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.417+0000 7f5dc962f700 1 -- 172.21.15.162:0/878834241 shutdown_connections 2020-11-28T21:52:24.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.417+0000 7f5dc962f700 1 -- 172.21.15.162:0/878834241 wait complete. 2020-11-28T21:52:24.474 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:52:24.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.585+0000 7f730af96700 1 Processor -- start 2020-11-28T21:52:24.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.585+0000 7f730af96700 1 -- start start 2020-11-28T21:52:24.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.585+0000 7f730af96700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f73040b60e0 0x7f73040b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:24.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.585+0000 7f730af96700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73040b4930 con 0x7f73040b4390 2020-11-28T21:52:24.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.585+0000 7f730af96700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73040b2170 con 0x7f7304060d10 2020-11-28T21:52:24.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.585+0000 7f730af96700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73040b6660 con 0x7f73040b60e0 2020-11-28T21:52:24.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.585+0000 7f7308d32700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f7304060d10 0x7f7304146180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:48046/0 (socket says 172.21.15.162:48046) 2020-11-28T21:52:24.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.589+0000 7f7308d32700 1 -- 172.21.15.162:0/2250646413 learned_addr learned my addr 172.21.15.162:0/2250646413 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:52:24.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.589+0000 7f7303fff700 1 --2- 172.21.15.162:0/2250646413 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f73040b60e0 0x7f73040b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:24.594 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.589+0000 7f73037fe700 1 -- 172.21.15.162:0/2250646413 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2309866761 0 0) 0x7f73040b2170 con 0x7f7304060d10 2020-11-28T21:52:24.594 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.589+0000 7f73037fe700 1 -- 172.21.15.162:0/2250646413 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72e4003230 con 0x7f7304060d10 2020-11-28T21:52:24.594 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.589+0000 7f73037fe700 1 -- 172.21.15.162:0/2250646413 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1182671093 0 0) 0x7f73040b4930 con 0x7f73040b4390 2020-11-28T21:52:24.594 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.589+0000 7f73037fe700 1 -- 172.21.15.162:0/2250646413 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73040b2170 con 0x7f73040b4390 2020-11-28T21:52:24.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.589+0000 7f7303fff700 1 -- 172.21.15.162:0/2250646413 >> v1:172.21.15.162:6789/0 conn(0x7f7304060d10 legacy=0x7f7304146180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:24.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.589+0000 7f7303fff700 1 -- 172.21.15.162:0/2250646413 >> v1:172.21.15.118:6789/0 conn(0x7f73040b4390 legacy=0x7f730413fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:24.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.589+0000 7f7303fff700 1 -- 172.21.15.162:0/2250646413 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73040b4b60 con 0x7f73040b60e0 2020-11-28T21:52:24.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.589+0000 7f7303fff700 1 --2- 172.21.15.162:0/2250646413 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f73040b60e0 0x7f73040b52a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f72ec002f30 tx=0x7f72ec006dc0).ready entity=mon.2 client_cookie=a8903b558810a14f server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:24.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.589+0000 7f73037fe700 1 -- 172.21.15.162:0/2250646413 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72ec009730 con 0x7f73040b60e0 2020-11-28T21:52:24.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.589+0000 7f73037fe700 1 -- 172.21.15.162:0/2250646413 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72ec009890 con 0x7f73040b60e0 2020-11-28T21:52:24.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.589+0000 7f73037fe700 1 -- 172.21.15.162:0/2250646413 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72ec009bf0 con 0x7f73040b60e0 2020-11-28T21:52:24.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.589+0000 7f730af96700 1 -- 172.21.15.162:0/2250646413 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f73040b60e0 msgr2=0x7f73040b52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:24.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.589+0000 7f730af96700 1 --2- 172.21.15.162:0/2250646413 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f73040b60e0 0x7f73040b52a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f72ec002f30 tx=0x7f72ec006dc0).stop 2020-11-28T21:52:24.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.589+0000 7f730af96700 1 -- 172.21.15.162:0/2250646413 shutdown_connections 2020-11-28T21:52:24.597 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.589+0000 7f730af96700 1 --2- 172.21.15.162:0/2250646413 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f73040b60e0 0x7f73040b52a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:24.597 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.589+0000 7f730af96700 1 -- 172.21.15.162:0/2250646413 >> 172.21.15.162:0/2250646413 conn(0x7f730405fe70 msgr2=0x7f730406aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:24.597 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.589+0000 7f730af96700 1 -- 172.21.15.162:0/2250646413 shutdown_connections 2020-11-28T21:52:24.597 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.589+0000 7f730af96700 1 -- 172.21.15.162:0/2250646413 wait complete. 2020-11-28T21:52:24.599 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.589+0000 7f730af96700 1 Processor -- start 2020-11-28T21:52:24.599 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.593+0000 7f730af96700 1 -- start start 2020-11-28T21:52:24.600 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.593+0000 7f730af96700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7304060d10 0x7f73040b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:24.600 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.593+0000 7f730af96700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f73040b4390 0x7f73041521e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:24.600 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.593+0000 7f730af96700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73040b60e0 0x7f73041526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:24.600 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.593+0000 7f730af96700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7304152bc0 con 0x7f73040b4390 2020-11-28T21:52:24.600 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.593+0000 7f730af96700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73040b7b10 con 0x7f73040b60e0 2020-11-28T21:52:24.601 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.593+0000 7f730af96700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7304151bc0 con 0x7f7304060d10 2020-11-28T21:52:24.601 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.593+0000 7f7309533700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73040b60e0 0x7f73041526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:24.601 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.593+0000 7f7303fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f73040b4390 0x7f73041521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:24.601 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.593+0000 7f7309533700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73040b60e0 0x7f73041526d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41212/0 (socket says 172.21.15.162:41212) 2020-11-28T21:52:24.601 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.593+0000 7f7309533700 1 -- 172.21.15.162:0/1616461559 learned_addr learned my addr 172.21.15.162:0/1616461559 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:24.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.593+0000 7f7308d32700 1 --2- 172.21.15.162:0/1616461559 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7304060d10 0x7f73040b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:24.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.593+0000 7f7309533700 1 -- 172.21.15.162:0/1616461559 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7304060d10 msgr2=0x7f73040b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:24.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.593+0000 7f7309533700 1 --2- 172.21.15.162:0/1616461559 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7304060d10 0x7f73040b2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:24.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.593+0000 7f7309533700 1 -- 172.21.15.162:0/1616461559 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f73040b4390 msgr2=0x7f73041521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:24.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.593+0000 7f7309533700 1 --2- 172.21.15.162:0/1616461559 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f73040b4390 0x7f73041521e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:24.603 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.593+0000 7f7309533700 1 -- 172.21.15.162:0/1616461559 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73040b7d40 con 0x7f73040b60e0 2020-11-28T21:52:24.603 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.593+0000 7f7309533700 1 --2- 172.21.15.162:0/1616461559 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73040b60e0 0x7f73041526d0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7f72fc001450 tx=0x7f72fc007d60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:24.603 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.593+0000 7f7301ffb700 1 -- 172.21.15.162:0/1616461559 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72fc00a5a0 con 0x7f73040b60e0 2020-11-28T21:52:24.603 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.593+0000 7f730af96700 1 -- 172.21.15.162:0/1616461559 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73040b7fe0 con 0x7f73040b60e0 2020-11-28T21:52:24.603 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.593+0000 7f7301ffb700 1 -- 172.21.15.162:0/1616461559 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72fc00a700 con 0x7f73040b60e0 2020-11-28T21:52:24.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.593+0000 7f7301ffb700 1 -- 172.21.15.162:0/1616461559 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72fc00a870 con 0x7f73040b60e0 2020-11-28T21:52:24.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.593+0000 7f730af96700 1 -- 172.21.15.162:0/1616461559 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7304141fa0 con 0x7f73040b60e0 2020-11-28T21:52:24.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.597+0000 7f730af96700 1 -- 172.21.15.162:0/1616461559 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73040b8170 con 0x7f73040b60e0 2020-11-28T21:52:24.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.597+0000 7f7301ffb700 1 -- 172.21.15.162:0/1616461559 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f72fc010070 con 0x7f73040b60e0 2020-11-28T21:52:24.605 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.597+0000 7f7301ffb700 1 --2- 172.21.15.162:0/1616461559 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f72e8055470 0x7f72e8057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:24.605 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.597+0000 7f7301ffb700 1 -- 172.21.15.162:0/1616461559 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f72fc06d950 con 0x7f73040b60e0 2020-11-28T21:52:24.606 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.597+0000 7f7308d32700 1 --2- 172.21.15.162:0/1616461559 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f72e8055470 0x7f72e8057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:24.606 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.597+0000 7f7308d32700 1 --2- 172.21.15.162:0/1616461559 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f72e8055470 0x7f72e8057930 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7f72f400d000 tx=0x7f72f4009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:24.606 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.601+0000 7f7301ffb700 1 -- 172.21.15.162:0/1616461559 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f72fc043e20 con 0x7f73040b60e0 2020-11-28T21:52:24.747 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.745+0000 7f730af96700 1 -- 172.21.15.162:0/1616461559 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7304057050 con 0x7f73040b60e0 2020-11-28T21:52:24.748 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.745+0000 7f7301ffb700 1 -- 172.21.15.162:0/1616461559 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+13278 (secure 0 0 0) 0x7f72fc047440 con 0x7f73040b60e0 2020-11-28T21:52:24.749 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:52:24.749 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":71,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:52:24.066704+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:52:24.751 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.745+0000 7f730af96700 1 -- 172.21.15.162:0/1616461559 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f72e8055470 msgr2=0x7f72e8057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:24.751 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.745+0000 7f730af96700 1 --2- 172.21.15.162:0/1616461559 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f72e8055470 0x7f72e8057930 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7f72f400d000 tx=0x7f72f4009040).stop 2020-11-28T21:52:24.751 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.745+0000 7f730af96700 1 -- 172.21.15.162:0/1616461559 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73040b60e0 msgr2=0x7f73041526d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:24.751 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.745+0000 7f730af96700 1 --2- 172.21.15.162:0/1616461559 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73040b60e0 0x7f73041526d0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7f72fc001450 tx=0x7f72fc007d60).stop 2020-11-28T21:52:24.752 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.745+0000 7f730af96700 1 -- 172.21.15.162:0/1616461559 shutdown_connections 2020-11-28T21:52:24.752 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.745+0000 7f730af96700 1 --2- 172.21.15.162:0/1616461559 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f72e8055470 0x7f72e8057930 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:24.752 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.745+0000 7f730af96700 1 --2- 172.21.15.162:0/1616461559 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7304060d10 0x7f73040b2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:24.752 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.745+0000 7f730af96700 1 --2- 172.21.15.162:0/1616461559 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73040b60e0 0x7f73041526d0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:24.752 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.745+0000 7f730af96700 1 --2- 172.21.15.162:0/1616461559 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f73040b4390 0x7f73041521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:24.753 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.745+0000 7f730af96700 1 -- 172.21.15.162:0/1616461559 >> 172.21.15.162:0/1616461559 conn(0x7f730405fe70 msgr2=0x7f730413d020 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:24.753 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.745+0000 7f730af96700 1 -- 172.21.15.162:0/1616461559 shutdown_connections 2020-11-28T21:52:24.753 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.745+0000 7f730af96700 1 -- 172.21.15.162:0/1616461559 wait complete. 2020-11-28T21:52:24.763 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-28T21:52:24.764 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:52:24.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.873+0000 7f068f59e700 1 Processor -- start 2020-11-28T21:52:24.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.873+0000 7f068f59e700 1 -- start start 2020-11-28T21:52:24.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.877+0000 7f068f59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f069009a270 0x7f069005bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:24.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.877+0000 7f068f59e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f069009a670 con 0x7f069013d0c0 2020-11-28T21:52:24.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.877+0000 7f068f59e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f069006aca0 con 0x7f06900b15e0 2020-11-28T21:52:24.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.877+0000 7f068f59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f069013d4a0 con 0x7f069009a270 2020-11-28T21:52:24.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.877+0000 7f068dd9b700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f06900b15e0 0x7f0690099b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:48062/0 (socket says 172.21.15.162:48062) 2020-11-28T21:52:24.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.877+0000 7f068dd9b700 1 -- 172.21.15.162:0/3263871445 learned_addr learned my addr 172.21.15.162:0/3263871445 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:52:24.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.877+0000 7f068ed9d700 1 --2- 172.21.15.162:0/3263871445 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f069009a270 0x7f069005bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:24.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.877+0000 7f068ed9d700 1 -- 172.21.15.162:0/3263871445 >> v1:172.21.15.162:6789/0 conn(0x7f06900b15e0 legacy=0x7f0690099b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:24.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.877+0000 7f068ed9d700 1 -- 172.21.15.162:0/3263871445 >> v1:172.21.15.118:6789/0 conn(0x7f069013d0c0 legacy=0x7f06900b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:24.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.877+0000 7f068ed9d700 1 -- 172.21.15.162:0/3263871445 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f069006af40 con 0x7f069009a270 2020-11-28T21:52:24.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.877+0000 7f068ed9d700 1 --2- 172.21.15.162:0/3263871445 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f069009a270 0x7f069005bf90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f06880032b0 tx=0x7f06880091a0).ready entity=mon.2 client_cookie=9d4dcd6cc89e9568 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:24.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.877+0000 7f068d59a700 1 -- 172.21.15.162:0/3263871445 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0688009e70 con 0x7f069009a270 2020-11-28T21:52:24.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.877+0000 7f068d59a700 1 -- 172.21.15.162:0/3263871445 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f068800bc00 con 0x7f069009a270 2020-11-28T21:52:24.886 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.877+0000 7f068d59a700 1 -- 172.21.15.162:0/3263871445 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f068800bd70 con 0x7f069009a270 2020-11-28T21:52:24.886 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.877+0000 7f068f59e700 1 -- 172.21.15.162:0/3263871445 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f069009a270 msgr2=0x7f069005bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:24.886 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.877+0000 7f068f59e700 1 --2- 172.21.15.162:0/3263871445 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f069009a270 0x7f069005bf90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f06880032b0 tx=0x7f06880091a0).stop 2020-11-28T21:52:24.886 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.881+0000 7f068f59e700 1 -- 172.21.15.162:0/3263871445 shutdown_connections 2020-11-28T21:52:24.886 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.881+0000 7f068f59e700 1 --2- 172.21.15.162:0/3263871445 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f069009a270 0x7f069005bf90 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:24.887 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.881+0000 7f068f59e700 1 -- 172.21.15.162:0/3263871445 >> 172.21.15.162:0/3263871445 conn(0x7f06900b5520 msgr2=0x7f06900b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:24.887 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.881+0000 7f068f59e700 1 -- 172.21.15.162:0/3263871445 shutdown_connections 2020-11-28T21:52:24.887 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.881+0000 7f068f59e700 1 -- 172.21.15.162:0/3263871445 wait complete. 2020-11-28T21:52:24.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.881+0000 7f068f59e700 1 Processor -- start 2020-11-28T21:52:24.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.881+0000 7f068f59e700 1 -- start start 2020-11-28T21:52:24.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.881+0000 7f068f59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f069009a270 0x7f06900604f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:24.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.881+0000 7f068f59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f06900b15e0 0x7f0690066b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:24.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.885+0000 7f068f59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f069013d0c0 0x7f0690065ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:24.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.885+0000 7f068f59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0690098da0 con 0x7f06900b15e0 2020-11-28T21:52:24.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.885+0000 7f068f59e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0690066780 con 0x7f069009a270 2020-11-28T21:52:24.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.885+0000 7f068f59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06900675a0 con 0x7f069013d0c0 2020-11-28T21:52:24.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.885+0000 7f068e59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f069009a270 0x7f06900604f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:24.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.885+0000 7f068e59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f069009a270 0x7f06900604f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41222/0 (socket says 172.21.15.162:41222) 2020-11-28T21:52:24.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.885+0000 7f068e59c700 1 -- 172.21.15.162:0/1150144364 learned_addr learned my addr 172.21.15.162:0/1150144364 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:24.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.885+0000 7f068ed9d700 1 --2- 172.21.15.162:0/1150144364 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f069013d0c0 0x7f0690065ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:24.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.885+0000 7f068dd9b700 1 --2- 172.21.15.162:0/1150144364 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f06900b15e0 0x7f0690066b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:24.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.885+0000 7f068e59c700 1 -- 172.21.15.162:0/1150144364 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f069013d0c0 msgr2=0x7f0690065ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:24.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.885+0000 7f068e59c700 1 --2- 172.21.15.162:0/1150144364 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f069013d0c0 0x7f0690065ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:24.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.885+0000 7f068e59c700 1 -- 172.21.15.162:0/1150144364 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f06900b15e0 msgr2=0x7f0690066b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:24.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.885+0000 7f068e59c700 1 --2- 172.21.15.162:0/1150144364 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f06900b15e0 0x7f0690066b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:24.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.885+0000 7f068e59c700 1 -- 172.21.15.162:0/1150144364 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06900677e0 con 0x7f069009a270 2020-11-28T21:52:24.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.885+0000 7f068e59c700 1 --2- 172.21.15.162:0/1150144364 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f069009a270 0x7f06900604f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f0690062580 tx=0x7f0680007c20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:24.896 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.885+0000 7f067f7fe700 1 -- 172.21.15.162:0/1150144364 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0680013410 con 0x7f069009a270 2020-11-28T21:52:24.896 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.885+0000 7f067f7fe700 1 -- 172.21.15.162:0/1150144364 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0680013570 con 0x7f069009a270 2020-11-28T21:52:24.896 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.885+0000 7f068f59e700 1 -- 172.21.15.162:0/1150144364 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06900a9120 con 0x7f069009a270 2020-11-28T21:52:24.896 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.885+0000 7f067f7fe700 1 -- 172.21.15.162:0/1150144364 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06800138d0 con 0x7f069009a270 2020-11-28T21:52:24.896 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.885+0000 7f068f59e700 1 -- 172.21.15.162:0/1150144364 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06900a8ef0 con 0x7f069009a270 2020-11-28T21:52:24.897 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.885+0000 7f068f59e700 1 -- 172.21.15.162:0/1150144364 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0690067390 con 0x7f069009a270 2020-11-28T21:52:24.897 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.885+0000 7f067f7fe700 1 -- 172.21.15.162:0/1150144364 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0680010030 con 0x7f069009a270 2020-11-28T21:52:24.898 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.889+0000 7f067f7fe700 1 --2- 172.21.15.162:0/1150144364 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f066c055330 0x7f066c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:24.898 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.889+0000 7f067f7fe700 1 -- 172.21.15.162:0/1150144364 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f068006d900 con 0x7f069009a270 2020-11-28T21:52:24.898 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.889+0000 7f068dd9b700 1 --2- 172.21.15.162:0/1150144364 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f066c055330 0x7f066c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:24.898 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.889+0000 7f068dd9b700 1 --2- 172.21.15.162:0/1150144364 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f066c055330 0x7f066c0577f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7f068400d000 tx=0x7f0684009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:24.899 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:24.893+0000 7f067f7fe700 1 -- 172.21.15.162:0/1150144364 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0680043f30 con 0x7f069009a270 2020-11-28T21:52:25.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.037+0000 7f068f59e700 1 -- 172.21.15.162:0/1150144364 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f0690057050 con 0x7f069009a270 2020-11-28T21:52:25.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.037+0000 7f067f7fe700 1 -- 172.21.15.162:0/1150144364 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v71) v1 ==== 110+0+10 (secure 0 0 0) 0x7f0680047550 con 0x7f069009a270 2020-11-28T21:52:25.042 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:52:25.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.041+0000 7f068f59e700 1 -- 172.21.15.162:0/1150144364 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f066c055330 msgr2=0x7f066c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:25.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.041+0000 7f068f59e700 1 --2- 172.21.15.162:0/1150144364 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f066c055330 0x7f066c0577f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7f068400d000 tx=0x7f0684009040).stop 2020-11-28T21:52:25.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.041+0000 7f068f59e700 1 -- 172.21.15.162:0/1150144364 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f069009a270 msgr2=0x7f06900604f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:25.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.041+0000 7f068f59e700 1 --2- 172.21.15.162:0/1150144364 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f069009a270 0x7f06900604f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f0690062580 tx=0x7f0680007c20).stop 2020-11-28T21:52:25.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.041+0000 7f068f59e700 1 -- 172.21.15.162:0/1150144364 shutdown_connections 2020-11-28T21:52:25.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.041+0000 7f068f59e700 1 --2- 172.21.15.162:0/1150144364 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f066c055330 0x7f066c0577f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:25.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.041+0000 7f068f59e700 1 --2- 172.21.15.162:0/1150144364 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f069013d0c0 0x7f0690065ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:25.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.041+0000 7f068f59e700 1 --2- 172.21.15.162:0/1150144364 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f069009a270 0x7f06900604f0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:25.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.041+0000 7f068f59e700 1 --2- 172.21.15.162:0/1150144364 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f06900b15e0 0x7f0690066b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:25.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.041+0000 7f068f59e700 1 -- 172.21.15.162:0/1150144364 >> 172.21.15.162:0/1150144364 conn(0x7f06900b5520 msgr2=0x7f069005c500 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:25.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.041+0000 7f068f59e700 1 -- 172.21.15.162:0/1150144364 shutdown_connections 2020-11-28T21:52:25.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.041+0000 7f068f59e700 1 -- 172.21.15.162:0/1150144364 wait complete. 2020-11-28T21:52:25.055 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:52:25.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.169+0000 7f6ee1751700 1 Processor -- start 2020-11-28T21:52:25.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.169+0000 7f6ee1751700 1 -- start start 2020-11-28T21:52:25.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6ee1751700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6edc060d10 0x7f6edc098ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:25.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6ee1751700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6edc0b09f0 0x7f6edc098fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:25.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6ee1751700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6edc0afec0 con 0x7f6edc099600 2020-11-28T21:52:25.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6ee1751700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6edc0608b0 con 0x7f6edc060d10 2020-11-28T21:52:25.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6edbfff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6edc060d10 0x7f6edc098ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:25.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6ee1751700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6edc0b0160 con 0x7f6edc0b09f0 2020-11-28T21:52:25.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6edbfff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6edc060d10 0x7f6edc098ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41230/0 (socket says 172.21.15.162:41230) 2020-11-28T21:52:25.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6edbfff700 1 -- 172.21.15.162:0/3981949784 learned_addr learned my addr 172.21.15.162:0/3981949784 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:25.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6edbfff700 1 -- 172.21.15.162:0/3981949784 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6edc0b09f0 msgr2=0x7f6edc098fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:25.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6ed37fe700 1 --2- 172.21.15.162:0/3981949784 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6edc0b09f0 0x7f6edc098fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:25.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6edbfff700 1 --2- 172.21.15.162:0/3981949784 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6edc0b09f0 0x7f6edc098fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:25.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6edbfff700 1 -- 172.21.15.162:0/3981949784 >> v1:172.21.15.162:6789/0 conn(0x7f6edc099600 legacy=0x7f6edc099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:25.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6edbfff700 1 -- 172.21.15.162:0/3981949784 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6edc0b0400 con 0x7f6edc060d10 2020-11-28T21:52:25.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6ed37fe700 1 --2- 172.21.15.162:0/3981949784 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6edc0b09f0 0x7f6edc098fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:52:25.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6edb7fe700 1 -- 172.21.15.162:0/3981949784 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 186488352 0 0) 0x7f6edc0afec0 con 0x7f6edc099600 2020-11-28T21:52:25.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6edbfff700 1 --2- 172.21.15.162:0/3981949784 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6edc060d10 0x7f6edc098ac0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f6ecc0032b0 tx=0x7f6ecc009270).ready entity=mon.1 client_cookie=1c86646304380c87 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:25.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6edb7fe700 1 -- 172.21.15.162:0/3981949784 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ecc00bc00 con 0x7f6edc060d10 2020-11-28T21:52:25.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6edb7fe700 1 -- 172.21.15.162:0/3981949784 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ecc00bd60 con 0x7f6edc060d10 2020-11-28T21:52:25.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6edb7fe700 1 -- 172.21.15.162:0/3981949784 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ecc011400 con 0x7f6edc060d10 2020-11-28T21:52:25.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6ee1751700 1 -- 172.21.15.162:0/3981949784 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6edc060d10 msgr2=0x7f6edc098ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:25.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6ee1751700 1 --2- 172.21.15.162:0/3981949784 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6edc060d10 0x7f6edc098ac0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f6ecc0032b0 tx=0x7f6ecc009270).stop 2020-11-28T21:52:25.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6ee1751700 1 -- 172.21.15.162:0/3981949784 shutdown_connections 2020-11-28T21:52:25.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6ee1751700 1 --2- 172.21.15.162:0/3981949784 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6edc060d10 0x7f6edc098ac0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:25.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6ee1751700 1 --2- 172.21.15.162:0/3981949784 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6edc0b09f0 0x7f6edc098fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:25.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6ee1751700 1 -- 172.21.15.162:0/3981949784 >> 172.21.15.162:0/3981949784 conn(0x7f6edc0b5520 msgr2=0x7f6edc0b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:25.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6ee1751700 1 -- 172.21.15.162:0/3981949784 shutdown_connections 2020-11-28T21:52:25.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6ee1751700 1 -- 172.21.15.162:0/3981949784 wait complete. 2020-11-28T21:52:25.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6ee1751700 1 Processor -- start 2020-11-28T21:52:25.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6ee1751700 1 -- start start 2020-11-28T21:52:25.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6ee1751700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6edc060d10 0x7f6edc1499e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:25.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6ee1751700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6edc099600 0x7f6edc0abbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:25.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6ee1751700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6edc0b09f0 0x7f6edc090050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:25.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6ee1751700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6edc0aab80 con 0x7f6edc0b09f0 2020-11-28T21:52:25.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6ee1751700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6edc0aae20 con 0x7f6edc060d10 2020-11-28T21:52:25.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6ee1751700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6edc08f910 con 0x7f6edc099600 2020-11-28T21:52:25.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6edbfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6edc060d10 0x7f6edc1499e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:25.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6edbfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6edc060d10 0x7f6edc1499e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41236/0 (socket says 172.21.15.162:41236) 2020-11-28T21:52:25.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6edbfff700 1 -- 172.21.15.162:0/3036708982 learned_addr learned my addr 172.21.15.162:0/3036708982 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:25.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6ed37fe700 1 --2- 172.21.15.162:0/3036708982 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6edc099600 0x7f6edc0abbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:25.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6ee0f50700 1 --2- 172.21.15.162:0/3036708982 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6edc0b09f0 0x7f6edc090050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:25.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6edbfff700 1 -- 172.21.15.162:0/3036708982 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6edc099600 msgr2=0x7f6edc0abbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:25.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6edbfff700 1 --2- 172.21.15.162:0/3036708982 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6edc099600 0x7f6edc0abbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:25.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6edbfff700 1 -- 172.21.15.162:0/3036708982 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6edc0b09f0 msgr2=0x7f6edc090050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:25.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6edbfff700 1 --2- 172.21.15.162:0/3036708982 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6edc0b09f0 0x7f6edc090050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:25.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6edbfff700 1 -- 172.21.15.162:0/3036708982 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6edc08fbb0 con 0x7f6edc060d10 2020-11-28T21:52:25.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6ee0f50700 1 --2- 172.21.15.162:0/3036708982 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6edc0b09f0 0x7f6edc090050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:52:25.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6edbfff700 1 --2- 172.21.15.162:0/3036708982 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6edc060d10 0x7f6edc1499e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7f6ecc0032b0 tx=0x7f6ecc016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:25.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6ed9ffb700 1 -- 172.21.15.162:0/3036708982 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ecc0107a0 con 0x7f6edc060d10 2020-11-28T21:52:25.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6ed9ffb700 1 -- 172.21.15.162:0/3036708982 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ecc010900 con 0x7f6edc060d10 2020-11-28T21:52:25.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6ee1751700 1 -- 172.21.15.162:0/3036708982 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6edc0aa840 con 0x7f6edc060d10 2020-11-28T21:52:25.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6ed9ffb700 1 -- 172.21.15.162:0/3036708982 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ecc010c60 con 0x7f6edc060d10 2020-11-28T21:52:25.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6ee1751700 1 -- 172.21.15.162:0/3036708982 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6edc0ac0c0 con 0x7f6edc060d10 2020-11-28T21:52:25.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6ee1751700 1 -- 172.21.15.162:0/3036708982 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6edc0aa9d0 con 0x7f6edc060d10 2020-11-28T21:52:25.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6ed9ffb700 1 -- 172.21.15.162:0/3036708982 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6ecc00f070 con 0x7f6edc060d10 2020-11-28T21:52:25.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6ed9ffb700 1 --2- 172.21.15.162:0/3036708982 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6ec0055330 0x7f6ec00577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:25.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6ed9ffb700 1 -- 172.21.15.162:0/3036708982 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f6ecc06f550 con 0x7f6edc060d10 2020-11-28T21:52:25.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.173+0000 7f6ed37fe700 1 --2- 172.21.15.162:0/3036708982 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6ec0055330 0x7f6ec00577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:25.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.181+0000 7f6ed37fe700 1 --2- 172.21.15.162:0/3036708982 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6ec0055330 0x7f6ec00577f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f6ec400b000 tx=0x7f6ec4008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:25.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.185+0000 7f6ed9ffb700 1 -- 172.21.15.162:0/3036708982 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6ecc045cb0 con 0x7f6edc060d10 2020-11-28T21:52:25.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.325+0000 7f6ee1751700 1 -- 172.21.15.162:0/3036708982 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6edc08fd40 con 0x7f6edc060d10 2020-11-28T21:52:25.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.329+0000 7f6ed9ffb700 1 -- 172.21.15.162:0/3036708982 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v71) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6ecc049250 con 0x7f6edc060d10 2020-11-28T21:52:25.328 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:52:25.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.329+0000 7f6ee1751700 1 -- 172.21.15.162:0/3036708982 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6ec0055330 msgr2=0x7f6ec00577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:25.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.329+0000 7f6ee1751700 1 --2- 172.21.15.162:0/3036708982 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6ec0055330 0x7f6ec00577f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f6ec400b000 tx=0x7f6ec4008040).stop 2020-11-28T21:52:25.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.329+0000 7f6ee1751700 1 -- 172.21.15.162:0/3036708982 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6edc060d10 msgr2=0x7f6edc1499e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:25.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.329+0000 7f6ee1751700 1 --2- 172.21.15.162:0/3036708982 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6edc060d10 0x7f6edc1499e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7f6ecc0032b0 tx=0x7f6ecc016000).stop 2020-11-28T21:52:25.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.329+0000 7f6ee1751700 1 -- 172.21.15.162:0/3036708982 shutdown_connections 2020-11-28T21:52:25.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.329+0000 7f6ee1751700 1 --2- 172.21.15.162:0/3036708982 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6ec0055330 0x7f6ec00577f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:25.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.329+0000 7f6ee1751700 1 --2- 172.21.15.162:0/3036708982 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6edc0b09f0 0x7f6edc090050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:25.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.329+0000 7f6ee1751700 1 --2- 172.21.15.162:0/3036708982 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6edc060d10 0x7f6edc1499e0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:25.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.329+0000 7f6ee1751700 1 --2- 172.21.15.162:0/3036708982 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6edc099600 0x7f6edc0abbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:25.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.329+0000 7f6ee1751700 1 -- 172.21.15.162:0/3036708982 >> 172.21.15.162:0/3036708982 conn(0x7f6edc0b5520 msgr2=0x7f6edc0b3d40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:25.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.329+0000 7f6ee1751700 1 -- 172.21.15.162:0/3036708982 shutdown_connections 2020-11-28T21:52:25.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.329+0000 7f6ee1751700 1 -- 172.21.15.162:0/3036708982 wait complete. 2020-11-28T21:52:25.340 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2020-11-28T21:52:25.450 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.449+0000 7fb3e7ffe700 1 Processor -- start 2020-11-28T21:52:25.451 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.449+0000 7fb3e7ffe700 1 -- start start 2020-11-28T21:52:25.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e7ffe700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb3e00b4160 0x7fb3e0061090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:25.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e7ffe700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fb3e005fbe0 0x7fb3e00b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:25.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e7ffe700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3e00b6fe0 con 0x7fb3e0060cb0 2020-11-28T21:52:25.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e7ffe700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3e00b7f70 con 0x7fb3e005fbe0 2020-11-28T21:52:25.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e7ffe700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e0144bd0 con 0x7fb3e00b4160 2020-11-28T21:52:25.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e5d9a700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fb3e0060cb0 0x7fb3e00b3a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:48088/0 (socket says 172.21.15.162:48088) 2020-11-28T21:52:25.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e5d9a700 1 -- 172.21.15.162:0/1001345154 learned_addr learned my addr 172.21.15.162:0/1001345154 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:52:25.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e5599700 1 --2- 172.21.15.162:0/1001345154 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb3e00b4160 0x7fb3e0061090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:25.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e659b700 1 --2- 172.21.15.162:0/1001345154 >> v2:172.21.15.118:3300/0 conn(0x7fb3e005fbe0 0x7fb3e00b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:25.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e4d98700 1 -- 172.21.15.162:0/1001345154 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2363123586 0 0) 0x7fb3e00b6fe0 con 0x7fb3e0060cb0 2020-11-28T21:52:25.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e4d98700 1 -- 172.21.15.162:0/1001345154 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3c4003230 con 0x7fb3e0060cb0 2020-11-28T21:52:25.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e4d98700 1 -- 172.21.15.162:0/1001345154 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2847327762 0 0) 0x7fb3c4003230 con 0x7fb3e0060cb0 2020-11-28T21:52:25.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e4d98700 1 -- 172.21.15.162:0/1001345154 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb3e00b4160 msgr2=0x7fb3e0061090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:25.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e4d98700 1 --2- 172.21.15.162:0/1001345154 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb3e00b4160 0x7fb3e0061090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:25.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e4d98700 1 -- 172.21.15.162:0/1001345154 >> v2:172.21.15.118:3300/0 conn(0x7fb3e005fbe0 msgr2=0x7fb3e00b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:25.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e4d98700 1 --2- 172.21.15.162:0/1001345154 >> v2:172.21.15.118:3300/0 conn(0x7fb3e005fbe0 0x7fb3e00b2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:25.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e4d98700 1 -- 172.21.15.162:0/1001345154 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e00b7750 con 0x7fb3e0060cb0 2020-11-28T21:52:25.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e4d98700 1 -- 172.21.15.162:0/1001345154 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7fb3d8001a50 con 0x7fb3e0060cb0 2020-11-28T21:52:25.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e4d98700 1 -- 172.21.15.162:0/1001345154 >> v1:172.21.15.162:6789/0 conn(0x7fb3e0060cb0 legacy=0x7fb3e00b3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:25.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e4d98700 1 --2- 172.21.15.162:0/1001345154 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3c40037a0 0x7fb3c4003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:25.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e4d98700 1 --2- 172.21.15.162:0/1001345154 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb3c40044d0 0x7fb3c4004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:25.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e4d98700 1 --2- 172.21.15.162:0/1001345154 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb3e00b4160 0x7fb3c4004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:25.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e4d98700 1 -- 172.21.15.162:0/1001345154 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e0144bd0 con 0x7fb3c40044d0 2020-11-28T21:52:25.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e4d98700 1 -- 172.21.15.162:0/1001345154 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c4005330 con 0x7fb3c40037a0 2020-11-28T21:52:25.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e4d98700 1 -- 172.21.15.162:0/1001345154 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c400b720 con 0x7fb3e00b4160 2020-11-28T21:52:25.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e5d9a700 1 --2- 172.21.15.162:0/1001345154 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3c40037a0 0x7fb3c4003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:25.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e5599700 1 --2- 172.21.15.162:0/1001345154 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb3c40044d0 0x7fb3c4004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:25.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e659b700 1 --2- 172.21.15.162:0/1001345154 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb3e00b4160 0x7fb3c4004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:25.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e5d9a700 1 -- 172.21.15.162:0/1001345154 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb3e00b4160 msgr2=0x7fb3c4004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:25.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e5d9a700 1 --2- 172.21.15.162:0/1001345154 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb3e00b4160 0x7fb3c4004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:25.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e5d9a700 1 -- 172.21.15.162:0/1001345154 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb3c40044d0 msgr2=0x7fb3c4004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:25.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e5d9a700 1 --2- 172.21.15.162:0/1001345154 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb3c40044d0 0x7fb3c4004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:25.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e5d9a700 1 -- 172.21.15.162:0/1001345154 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c4003230 con 0x7fb3c40037a0 2020-11-28T21:52:25.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e5599700 1 --2- 172.21.15.162:0/1001345154 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb3c40044d0 0x7fb3c4004950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:52:25.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e5d9a700 1 --2- 172.21.15.162:0/1001345154 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3c40037a0 0x7fb3c4003bc0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7fb3d8001760 tx=0x7fb3d8007a40).ready entity=mon.1 client_cookie=6dc26b0a36fc935 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:25.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e4d98700 1 -- 172.21.15.162:0/1001345154 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3d8005b70 con 0x7fb3c40037a0 2020-11-28T21:52:25.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e4d98700 1 -- 172.21.15.162:0/1001345154 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d8005cd0 con 0x7fb3c40037a0 2020-11-28T21:52:25.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e4d98700 1 -- 172.21.15.162:0/1001345154 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3d80072c0 con 0x7fb3c40037a0 2020-11-28T21:52:25.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e7ffe700 1 -- 172.21.15.162:0/1001345154 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3c40037a0 msgr2=0x7fb3c4003bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:25.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e7ffe700 1 --2- 172.21.15.162:0/1001345154 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3c40037a0 0x7fb3c4003bc0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7fb3d8001760 tx=0x7fb3d8007a40).stop 2020-11-28T21:52:25.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e659b700 1 -- 172.21.15.162:0/1001345154 reap_dead start 2020-11-28T21:52:25.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e7ffe700 1 -- 172.21.15.162:0/1001345154 shutdown_connections 2020-11-28T21:52:25.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e7ffe700 1 -- 172.21.15.162:0/1001345154 >> 172.21.15.162:0/1001345154 conn(0x7fb3e0062830 msgr2=0x7fb3e0062c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:25.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e7ffe700 1 -- 172.21.15.162:0/1001345154 shutdown_connections 2020-11-28T21:52:25.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.453+0000 7fb3e7ffe700 1 -- 172.21.15.162:0/1001345154 wait complete. 2020-11-28T21:52:25.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.457+0000 7fb3e7ffe700 1 Processor -- start 2020-11-28T21:52:25.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.457+0000 7fb3e7ffe700 1 -- start start 2020-11-28T21:52:25.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.457+0000 7fb3e7ffe700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb3e00ad2b0 0x7fb3e01564a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:25.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.457+0000 7fb3e7ffe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3e0156990 0x7fb3e00ae740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:25.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.457+0000 7fb3e7ffe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb3e00aec30 0x7fb3e0159660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:25.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.457+0000 7fb3e7ffe700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e00ae3c0 con 0x7fb3e00aec30 2020-11-28T21:52:25.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.457+0000 7fb3e7ffe700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e01560b0 con 0x7fb3e0156990 2020-11-28T21:52:25.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.457+0000 7fb3e7ffe700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e00b11e0 con 0x7fb3e00ad2b0 2020-11-28T21:52:25.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.457+0000 7fb3e5599700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3e0156990 0x7fb3e00ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:25.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.457+0000 7fb3e5599700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3e0156990 0x7fb3e00ae740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41258/0 (socket says 172.21.15.162:41258) 2020-11-28T21:52:25.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.457+0000 7fb3e5599700 1 -- 172.21.15.162:0/392932033 learned_addr learned my addr 172.21.15.162:0/392932033 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:25.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.457+0000 7fb3e659b700 1 --2- 172.21.15.162:0/392932033 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb3e00aec30 0x7fb3e0159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:25.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.457+0000 7fb3e5d9a700 1 --2- 172.21.15.162:0/392932033 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb3e00ad2b0 0x7fb3e01564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:25.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.457+0000 7fb3e5599700 1 -- 172.21.15.162:0/392932033 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb3e00ad2b0 msgr2=0x7fb3e01564a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:25.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.457+0000 7fb3e5599700 1 --2- 172.21.15.162:0/392932033 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb3e00ad2b0 0x7fb3e01564a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:25.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.457+0000 7fb3e5599700 1 -- 172.21.15.162:0/392932033 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb3e00aec30 msgr2=0x7fb3e0159660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:25.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.457+0000 7fb3e5599700 1 --2- 172.21.15.162:0/392932033 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb3e00aec30 0x7fb3e0159660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:25.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.457+0000 7fb3e5599700 1 -- 172.21.15.162:0/392932033 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e00ad690 con 0x7fb3e0156990 2020-11-28T21:52:25.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.457+0000 7fb3e5599700 1 --2- 172.21.15.162:0/392932033 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3e0156990 0x7fb3e00ae740 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7fb3e014ca70 tx=0x7fb3d4008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:25.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.457+0000 7fb3d2ffd700 1 -- 172.21.15.162:0/392932033 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3d400e820 con 0x7fb3e0156990 2020-11-28T21:52:25.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.457+0000 7fb3d2ffd700 1 -- 172.21.15.162:0/392932033 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d400e980 con 0x7fb3e0156990 2020-11-28T21:52:25.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.457+0000 7fb3e7ffe700 1 -- 172.21.15.162:0/392932033 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3e014ce30 con 0x7fb3e0156990 2020-11-28T21:52:25.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.457+0000 7fb3d2ffd700 1 -- 172.21.15.162:0/392932033 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3d400ece0 con 0x7fb3e0156990 2020-11-28T21:52:25.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.457+0000 7fb3e7ffe700 1 -- 172.21.15.162:0/392932033 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3e014d370 con 0x7fb3e0156990 2020-11-28T21:52:25.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.461+0000 7fb3e7ffe700 1 -- 172.21.15.162:0/392932033 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3e0057050 con 0x7fb3e0156990 2020-11-28T21:52:25.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.461+0000 7fb3d2ffd700 1 -- 172.21.15.162:0/392932033 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb3d4024070 con 0x7fb3e0156990 2020-11-28T21:52:25.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.461+0000 7fb3d2ffd700 1 --2- 172.21.15.162:0/392932033 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb3c8055440 0x7fb3c8057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:25.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.461+0000 7fb3d2ffd700 1 -- 172.21.15.162:0/392932033 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fb3d406e520 con 0x7fb3e0156990 2020-11-28T21:52:25.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.461+0000 7fb3e5d9a700 1 --2- 172.21.15.162:0/392932033 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb3c8055440 0x7fb3c8057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:25.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.461+0000 7fb3e5d9a700 1 --2- 172.21.15.162:0/392932033 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb3c8055440 0x7fb3c8057900 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7fb3d8018000 tx=0x7fb3d8015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:25.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.469+0000 7fb3d2ffd700 1 -- 172.21.15.162:0/392932033 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb3d40449e0 con 0x7fb3e0156990 2020-11-28T21:52:25.610 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.609+0000 7fb3e7ffe700 1 -- 172.21.15.162:0/392932033 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb3e00618a0 con 0x7fb3e0156990 2020-11-28T21:52:25.612 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.609+0000 7fb3d2ffd700 1 -- 172.21.15.162:0/392932033 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v71) v1 ==== 84+0+143 (secure 0 0 0) 0x7fb3d4048000 con 0x7fb3e0156990 2020-11-28T21:52:25.612 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:52:25.612 INFO:teuthology.orchestra.run.smithi162.stdout:[ 2020-11-28T21:52:25.613 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:25.613 INFO:teuthology.orchestra.run.smithi162.stdout: "poolnum": 1, 2020-11-28T21:52:25.614 INFO:teuthology.orchestra.run.smithi162.stdout: "poolname": "device_health_metrics" 2020-11-28T21:52:25.614 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:52:25.614 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:52:25.615 INFO:teuthology.orchestra.run.smithi162.stdout: "poolnum": 2, 2020-11-28T21:52:25.615 INFO:teuthology.orchestra.run.smithi162.stdout: "poolname": "rbd" 2020-11-28T21:52:25.616 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:52:25.616 INFO:teuthology.orchestra.run.smithi162.stdout:] 2020-11-28T21:52:25.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.613+0000 7fb3e7ffe700 1 -- 172.21.15.162:0/392932033 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb3c8055440 msgr2=0x7fb3c8057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:25.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.613+0000 7fb3e7ffe700 1 --2- 172.21.15.162:0/392932033 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb3c8055440 0x7fb3c8057900 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7fb3d8018000 tx=0x7fb3d8015000).stop 2020-11-28T21:52:25.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.613+0000 7fb3e7ffe700 1 -- 172.21.15.162:0/392932033 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3e0156990 msgr2=0x7fb3e00ae740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:25.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.613+0000 7fb3e7ffe700 1 --2- 172.21.15.162:0/392932033 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3e0156990 0x7fb3e00ae740 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7fb3e014ca70 tx=0x7fb3d4008040).stop 2020-11-28T21:52:25.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.613+0000 7fb3e7ffe700 1 -- 172.21.15.162:0/392932033 shutdown_connections 2020-11-28T21:52:25.620 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.613+0000 7fb3e7ffe700 1 --2- 172.21.15.162:0/392932033 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb3c8055440 0x7fb3c8057900 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:25.620 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.613+0000 7fb3e7ffe700 1 --2- 172.21.15.162:0/392932033 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb3e00ad2b0 0x7fb3e01564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:25.622 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.613+0000 7fb3e7ffe700 1 --2- 172.21.15.162:0/392932033 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb3e00aec30 0x7fb3e0159660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:25.622 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.613+0000 7fb3e7ffe700 1 --2- 172.21.15.162:0/392932033 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3e0156990 0x7fb3e00ae740 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:25.622 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.613+0000 7fb3e7ffe700 1 -- 172.21.15.162:0/392932033 >> 172.21.15.162:0/392932033 conn(0x7fb3e0062830 msgr2=0x7fb3e00b4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:25.622 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.613+0000 7fb3e7ffe700 1 -- 172.21.15.162:0/392932033 shutdown_connections 2020-11-28T21:52:25.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.613+0000 7fb3e7ffe700 1 -- 172.21.15.162:0/392932033 wait complete. 2020-11-28T21:52:25.625 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-11-28T21:52:25.625 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2020-11-28T21:52:25.742 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.741+0000 7f1b5667d700 1 Processor -- start 2020-11-28T21:52:25.742 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.741+0000 7f1b5667d700 1 -- start start 2020-11-28T21:52:25.744 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.745+0000 7f1b5667d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1b50060d10 0x7f1b50098ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:25.745 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.745+0000 7f1b5667d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1b50099d10 0x7f1b50098fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:25.745 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.745+0000 7f1b5667d700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b500afec0 con 0x7f1b500b09f0 2020-11-28T21:52:25.745 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.745+0000 7f1b5667d700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b500608b0 con 0x7f1b50099d10 2020-11-28T21:52:25.746 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.745+0000 7f1b5667d700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b500b0160 con 0x7f1b50060d10 2020-11-28T21:52:25.746 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.745+0000 7f1b5567b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1b50060d10 0x7f1b50098ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:25.746 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.745+0000 7f1b54e7a700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f1b500b09f0 0x7f1b50099600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:48110/0 (socket says 172.21.15.162:48110) 2020-11-28T21:52:25.747 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.745+0000 7f1b5567b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1b50060d10 0x7f1b50098ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41264/0 (socket says 172.21.15.162:41264) 2020-11-28T21:52:25.747 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.745+0000 7f1b54e7a700 1 -- 172.21.15.162:0/1104937172 learned_addr learned my addr 172.21.15.162:0/1104937172 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:52:25.747 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.745+0000 7f1b55e7c700 1 --2- 172.21.15.162:0/1104937172 >> v2:172.21.15.118:3300/0 conn(0x7f1b50099d10 0x7f1b50098fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:25.749 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.745+0000 7f1b5567b700 1 -- 172.21.15.162:0/1104937172 >> v2:172.21.15.118:3300/0 conn(0x7f1b50099d10 msgr2=0x7f1b50098fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:25.749 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.745+0000 7f1b5567b700 1 --2- 172.21.15.162:0/1104937172 >> v2:172.21.15.118:3300/0 conn(0x7f1b50099d10 0x7f1b50098fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:25.749 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.745+0000 7f1b5567b700 1 -- 172.21.15.162:0/1104937172 >> v1:172.21.15.162:6789/0 conn(0x7f1b500b09f0 legacy=0x7f1b50099600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:25.750 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.745+0000 7f1b5567b700 1 -- 172.21.15.162:0/1104937172 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b500b0400 con 0x7f1b50060d10 2020-11-28T21:52:25.750 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.745+0000 7f1b5567b700 1 --2- 172.21.15.162:0/1104937172 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b50060d10 0x7f1b50098ac0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7f1b4c002f30 tx=0x7f1b4c006dc0).ready entity=mon.1 client_cookie=a92c348485a9b685 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:25.750 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.745+0000 7f1b3ffff700 1 -- 172.21.15.162:0/1104937172 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b4c0097b0 con 0x7f1b50060d10 2020-11-28T21:52:25.751 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.745+0000 7f1b3ffff700 1 -- 172.21.15.162:0/1104937172 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b4c009910 con 0x7f1b50060d10 2020-11-28T21:52:25.751 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.745+0000 7f1b3ffff700 1 -- 172.21.15.162:0/1104937172 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b4c009c70 con 0x7f1b50060d10 2020-11-28T21:52:25.751 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.745+0000 7f1b5667d700 1 -- 172.21.15.162:0/1104937172 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b50060d10 msgr2=0x7f1b50098ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:25.752 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.745+0000 7f1b5667d700 1 --2- 172.21.15.162:0/1104937172 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b50060d10 0x7f1b50098ac0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7f1b4c002f30 tx=0x7f1b4c006dc0).stop 2020-11-28T21:52:25.752 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.745+0000 7f1b5667d700 1 -- 172.21.15.162:0/1104937172 shutdown_connections 2020-11-28T21:52:25.752 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.745+0000 7f1b5667d700 1 --2- 172.21.15.162:0/1104937172 >> v2:172.21.15.118:3300/0 conn(0x7f1b50099d10 0x7f1b50098fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:25.752 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.745+0000 7f1b5667d700 1 --2- 172.21.15.162:0/1104937172 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b50060d10 0x7f1b50098ac0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:25.753 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.745+0000 7f1b5667d700 1 -- 172.21.15.162:0/1104937172 >> 172.21.15.162:0/1104937172 conn(0x7f1b500b5520 msgr2=0x7f1b500b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:25.753 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.745+0000 7f1b5667d700 1 -- 172.21.15.162:0/1104937172 shutdown_connections 2020-11-28T21:52:25.753 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.745+0000 7f1b5667d700 1 -- 172.21.15.162:0/1104937172 wait complete. 2020-11-28T21:52:25.753 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.749+0000 7f1b5667d700 1 Processor -- start 2020-11-28T21:52:25.753 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.749+0000 7f1b5667d700 1 -- start start 2020-11-28T21:52:25.756 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.749+0000 7f1b5667d700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1b50060d10 0x7f1b500aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:25.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.749+0000 7f1b5667d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1b50099d10 0x7f1b50155500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:25.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.749+0000 7f1b5667d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b500b09f0 0x7f1b501559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:25.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.749+0000 7f1b5667d700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5008f910 con 0x7f1b50099d10 2020-11-28T21:52:25.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.749+0000 7f1b5667d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5008fb40 con 0x7f1b500b09f0 2020-11-28T21:52:25.758 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.749+0000 7f1b5667d700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5008fde0 con 0x7f1b50060d10 2020-11-28T21:52:25.758 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.749+0000 7f1b55e7c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b500b09f0 0x7f1b501559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:25.758 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.753+0000 7f1b55e7c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b500b09f0 0x7f1b501559f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41274/0 (socket says 172.21.15.162:41274) 2020-11-28T21:52:25.758 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.753+0000 7f1b54e7a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1b50099d10 0x7f1b50155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:25.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.753+0000 7f1b55e7c700 1 -- 172.21.15.162:0/2739876325 learned_addr learned my addr 172.21.15.162:0/2739876325 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:25.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.753+0000 7f1b5567b700 1 --2- 172.21.15.162:0/2739876325 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1b50060d10 0x7f1b500aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:25.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.753+0000 7f1b55e7c700 1 -- 172.21.15.162:0/2739876325 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1b50060d10 msgr2=0x7f1b500aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:25.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.753+0000 7f1b55e7c700 1 --2- 172.21.15.162:0/2739876325 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1b50060d10 0x7f1b500aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:25.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.753+0000 7f1b55e7c700 1 -- 172.21.15.162:0/2739876325 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1b50099d10 msgr2=0x7f1b50155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:25.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.753+0000 7f1b55e7c700 1 --2- 172.21.15.162:0/2739876325 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1b50099d10 0x7f1b50155500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:25.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.753+0000 7f1b55e7c700 1 -- 172.21.15.162:0/2739876325 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b50090080 con 0x7f1b500b09f0 2020-11-28T21:52:25.761 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.753+0000 7f1b5567b700 1 --2- 172.21.15.162:0/2739876325 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1b50060d10 0x7f1b500aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:52:25.761 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.753+0000 7f1b55e7c700 1 --2- 172.21.15.162:0/2739876325 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b500b09f0 0x7f1b501559f0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7f1b48006dd0 tx=0x7f1b480091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:25.761 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.753+0000 7f1b3e7fc700 1 -- 172.21.15.162:0/2739876325 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b4800ee70 con 0x7f1b500b09f0 2020-11-28T21:52:25.761 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.753+0000 7f1b3e7fc700 1 -- 172.21.15.162:0/2739876325 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b4800cbb0 con 0x7f1b500b09f0 2020-11-28T21:52:25.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.753+0000 7f1b5667d700 1 -- 172.21.15.162:0/2739876325 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b500926f0 con 0x7f1b500b09f0 2020-11-28T21:52:25.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.753+0000 7f1b3e7fc700 1 -- 172.21.15.162:0/2739876325 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b480123f0 con 0x7f1b500b09f0 2020-11-28T21:52:25.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.753+0000 7f1b5667d700 1 -- 172.21.15.162:0/2739876325 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b500aaa10 con 0x7f1b500b09f0 2020-11-28T21:52:25.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.753+0000 7f1b5667d700 1 -- 172.21.15.162:0/2739876325 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b50156080 con 0x7f1b500b09f0 2020-11-28T21:52:25.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.753+0000 7f1b3e7fc700 1 -- 172.21.15.162:0/2739876325 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1b48022070 con 0x7f1b500b09f0 2020-11-28T21:52:25.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.753+0000 7f1b3e7fc700 1 --2- 172.21.15.162:0/2739876325 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1b300554e0 0x7f1b300579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:25.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.753+0000 7f1b3e7fc700 1 -- 172.21.15.162:0/2739876325 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f1b4806e330 con 0x7f1b500b09f0 2020-11-28T21:52:25.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.753+0000 7f1b5567b700 1 --2- 172.21.15.162:0/2739876325 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1b300554e0 0x7f1b300579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:25.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.757+0000 7f1b5567b700 1 --2- 172.21.15.162:0/2739876325 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1b300554e0 0x7f1b300579a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f1b4c018000 tx=0x7f1b4c015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:25.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.761+0000 7f1b3e7fc700 1 -- 172.21.15.162:0/2739876325 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1b48044960 con 0x7f1b500b09f0 2020-11-28T21:52:25.900 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:25.901+0000 7f1b5667d700 1 -- 172.21.15.162:0/2739876325 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f1b50090210 con 0x7f1b500b09f0 2020-11-28T21:52:26.903 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:26.901+0000 7f1b3e7fc700 1 -- 172.21.15.162:0/2739876325 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v72) v1 ==== 133+0+0 (secure 0 0 0) 0x7f1b48016070 con 0x7f1b500b09f0 2020-11-28T21:52:26.908 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:26.909+0000 7f1b5667d700 1 -- 172.21.15.162:0/2739876325 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1b300554e0 msgr2=0x7f1b300579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:26.908 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:26.909+0000 7f1b5667d700 1 --2- 172.21.15.162:0/2739876325 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1b300554e0 0x7f1b300579a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f1b4c018000 tx=0x7f1b4c015000).stop 2020-11-28T21:52:26.909 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:26.909+0000 7f1b5667d700 1 -- 172.21.15.162:0/2739876325 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b500b09f0 msgr2=0x7f1b501559f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:26.909 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:26.909+0000 7f1b5667d700 1 --2- 172.21.15.162:0/2739876325 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b500b09f0 0x7f1b501559f0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7f1b48006dd0 tx=0x7f1b480091a0).stop 2020-11-28T21:52:26.910 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:26.909+0000 7f1b5667d700 1 -- 172.21.15.162:0/2739876325 shutdown_connections 2020-11-28T21:52:26.910 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:26.909+0000 7f1b5667d700 1 --2- 172.21.15.162:0/2739876325 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1b300554e0 0x7f1b300579a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:26.910 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:26.909+0000 7f1b5667d700 1 --2- 172.21.15.162:0/2739876325 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1b50060d10 0x7f1b500aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:26.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:26.909+0000 7f1b5667d700 1 --2- 172.21.15.162:0/2739876325 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b500b09f0 0x7f1b501559f0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:26.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:26.909+0000 7f1b5667d700 1 --2- 172.21.15.162:0/2739876325 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1b50099d10 0x7f1b50155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:26.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:26.909+0000 7f1b5667d700 1 -- 172.21.15.162:0/2739876325 >> 172.21.15.162:0/2739876325 conn(0x7f1b500b5520 msgr2=0x7f1b50146290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:26.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:26.909+0000 7f1b5667d700 1 -- 172.21.15.162:0/2739876325 shutdown_connections 2020-11-28T21:52:26.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:26.909+0000 7f1b5667d700 1 -- 172.21.15.162:0/2739876325 wait complete. 2020-11-28T21:52:26.912 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_metadata' created 2020-11-28T21:52:26.923 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2020-11-28T21:52:27.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.041+0000 7f0a5359e700 1 Processor -- start 2020-11-28T21:52:27.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.041+0000 7f0a5359e700 1 -- start start 2020-11-28T21:52:27.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.041+0000 7f0a5359e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0a540b2d00 0x7f0a5405c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:27.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.041+0000 7f0a5359e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0a540b3ab0 0x7f0a540b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:27.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.041+0000 7f0a5359e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a5405baf0 con 0x7f0a54142ad0 2020-11-28T21:52:27.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.041+0000 7f0a5359e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a540b0e20 con 0x7f0a540b2d00 2020-11-28T21:52:27.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.041+0000 7f0a5359e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5405bd90 con 0x7f0a540b3ab0 2020-11-28T21:52:27.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.041+0000 7f0a5259c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0a540b2d00 0x7f0a5405c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:27.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.041+0000 7f0a52d9d700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f0a54142ad0 0x7f0a5413fa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:48126/0 (socket says 172.21.15.162:48126) 2020-11-28T21:52:27.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.041+0000 7f0a52d9d700 1 -- 172.21.15.162:0/2659122126 learned_addr learned my addr 172.21.15.162:0/2659122126 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:52:27.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.041+0000 7f0a51d9b700 1 --2- 172.21.15.162:0/2659122126 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0a540b3ab0 0x7f0a540b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:27.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.045+0000 7f0a5259c700 1 -- 172.21.15.162:0/2659122126 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0a540b3ab0 msgr2=0x7f0a540b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:27.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.045+0000 7f0a5259c700 1 --2- 172.21.15.162:0/2659122126 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0a540b3ab0 0x7f0a540b30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:27.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.045+0000 7f0a5159a700 1 -- 172.21.15.162:0/2659122126 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1125179364 0 0) 0x7f0a5405baf0 con 0x7f0a54142ad0 2020-11-28T21:52:27.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.045+0000 7f0a5259c700 1 -- 172.21.15.162:0/2659122126 >> v1:172.21.15.162:6789/0 conn(0x7f0a54142ad0 legacy=0x7f0a5413fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:27.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.045+0000 7f0a5259c700 1 -- 172.21.15.162:0/2659122126 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a5405c030 con 0x7f0a540b2d00 2020-11-28T21:52:27.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.045+0000 7f0a5259c700 1 --2- 172.21.15.162:0/2659122126 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a540b2d00 0x7f0a5405c4d0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7f0a3c002f30 tx=0x7f0a3c006dc0).ready entity=mon.1 client_cookie=6b37631c55bbf1f6 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:27.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.045+0000 7f0a5159a700 1 -- 172.21.15.162:0/2659122126 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a3c009760 con 0x7f0a540b2d00 2020-11-28T21:52:27.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.045+0000 7f0a5159a700 1 -- 172.21.15.162:0/2659122126 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a3c0098c0 con 0x7f0a540b2d00 2020-11-28T21:52:27.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.045+0000 7f0a5159a700 1 -- 172.21.15.162:0/2659122126 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a3c009c20 con 0x7f0a540b2d00 2020-11-28T21:52:27.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.045+0000 7f0a5359e700 1 -- 172.21.15.162:0/2659122126 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a540b2d00 msgr2=0x7f0a5405c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:27.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.045+0000 7f0a5359e700 1 --2- 172.21.15.162:0/2659122126 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a540b2d00 0x7f0a5405c4d0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7f0a3c002f30 tx=0x7f0a3c006dc0).stop 2020-11-28T21:52:27.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.045+0000 7f0a5359e700 1 -- 172.21.15.162:0/2659122126 shutdown_connections 2020-11-28T21:52:27.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.045+0000 7f0a5359e700 1 --2- 172.21.15.162:0/2659122126 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a540b2d00 0x7f0a5405c4d0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:27.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.045+0000 7f0a5359e700 1 --2- 172.21.15.162:0/2659122126 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0a540b3ab0 0x7f0a540b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:27.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.045+0000 7f0a5359e700 1 -- 172.21.15.162:0/2659122126 >> 172.21.15.162:0/2659122126 conn(0x7f0a540604c0 msgr2=0x7f0a540608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:27.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.045+0000 7f0a5359e700 1 -- 172.21.15.162:0/2659122126 shutdown_connections 2020-11-28T21:52:27.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.045+0000 7f0a5359e700 1 -- 172.21.15.162:0/2659122126 wait complete. 2020-11-28T21:52:27.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.045+0000 7f0a5359e700 1 Processor -- start 2020-11-28T21:52:27.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.045+0000 7f0a5359e700 1 -- start start 2020-11-28T21:52:27.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.049+0000 7f0a5359e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a540b2d00 0x7f0a540b8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:27.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.049+0000 7f0a5359e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0a540b3ab0 0x7f0a54151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:27.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.049+0000 7f0a5359e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a54142ad0 0x7f0a541522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:27.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.049+0000 7f0a5359e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a54151730 con 0x7f0a540b2d00 2020-11-28T21:52:27.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.049+0000 7f0a5359e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a540ad8f0 con 0x7f0a54142ad0 2020-11-28T21:52:27.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.049+0000 7f0a5359e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5413b410 con 0x7f0a540b3ab0 2020-11-28T21:52:27.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.049+0000 7f0a52d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a54142ad0 0x7f0a541522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:27.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.049+0000 7f0a52d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a54142ad0 0x7f0a541522b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41288/0 (socket says 172.21.15.162:41288) 2020-11-28T21:52:27.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.049+0000 7f0a52d9d700 1 -- 172.21.15.162:0/2707893350 learned_addr learned my addr 172.21.15.162:0/2707893350 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:27.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.049+0000 7f0a5259c700 1 --2- 172.21.15.162:0/2707893350 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a540b2d00 0x7f0a540b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:27.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.049+0000 7f0a51d9b700 1 --2- 172.21.15.162:0/2707893350 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0a540b3ab0 0x7f0a54151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:27.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.049+0000 7f0a52d9d700 1 -- 172.21.15.162:0/2707893350 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0a540b3ab0 msgr2=0x7f0a54151dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:27.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.049+0000 7f0a52d9d700 1 --2- 172.21.15.162:0/2707893350 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0a540b3ab0 0x7f0a54151dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:27.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.049+0000 7f0a52d9d700 1 -- 172.21.15.162:0/2707893350 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a540b2d00 msgr2=0x7f0a540b8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:27.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.049+0000 7f0a52d9d700 1 --2- 172.21.15.162:0/2707893350 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a540b2d00 0x7f0a540b8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:27.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.049+0000 7f0a52d9d700 1 -- 172.21.15.162:0/2707893350 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a54151920 con 0x7f0a54142ad0 2020-11-28T21:52:27.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.049+0000 7f0a52d9d700 1 --2- 172.21.15.162:0/2707893350 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a54142ad0 0x7f0a541522b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7f0a4c000f00 tx=0x7f0a4c007f70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:27.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.049+0000 7f0a4b7fe700 1 -- 172.21.15.162:0/2707893350 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a4c005b70 con 0x7f0a54142ad0 2020-11-28T21:52:27.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.049+0000 7f0a5359e700 1 -- 172.21.15.162:0/2707893350 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a540b8900 con 0x7f0a54142ad0 2020-11-28T21:52:27.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.049+0000 7f0a4b7fe700 1 -- 172.21.15.162:0/2707893350 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4c005cd0 con 0x7f0a54142ad0 2020-11-28T21:52:27.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.049+0000 7f0a4b7fe700 1 -- 172.21.15.162:0/2707893350 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a4c007300 con 0x7f0a54142ad0 2020-11-28T21:52:27.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.049+0000 7f0a5359e700 1 -- 172.21.15.162:0/2707893350 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a540ad570 con 0x7f0a54142ad0 2020-11-28T21:52:27.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.049+0000 7f0a5359e700 1 -- 172.21.15.162:0/2707893350 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a540ad100 con 0x7f0a54142ad0 2020-11-28T21:52:27.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.053+0000 7f0a4b7fe700 1 -- 172.21.15.162:0/2707893350 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0a4c010030 con 0x7f0a54142ad0 2020-11-28T21:52:27.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.053+0000 7f0a4b7fe700 1 --2- 172.21.15.162:0/2707893350 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0a380553b0 0x7f0a38057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:27.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.053+0000 7f0a4b7fe700 1 -- 172.21.15.162:0/2707893350 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f0a4c06ddc0 con 0x7f0a54142ad0 2020-11-28T21:52:27.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.053+0000 7f0a5259c700 1 --2- 172.21.15.162:0/2707893350 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0a380553b0 0x7f0a38057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:27.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.053+0000 7f0a5259c700 1 --2- 172.21.15.162:0/2707893350 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0a380553b0 0x7f0a38057870 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f0a3c01a000 tx=0x7f0a3c015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:27.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.057+0000 7f0a4b7fe700 1 -- 172.21.15.162:0/2707893350 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0a540ad100 con 0x7f0a54142ad0 2020-11-28T21:52:27.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:27.201+0000 7f0a5359e700 1 -- 172.21.15.162:0/2707893350 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f0a540ad100 con 0x7f0a54142ad0 2020-11-28T21:52:28.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.101+0000 7f0a4b7fe700 1 -- 172.21.15.162:0/2707893350 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v74) v1 ==== 125+0+0 (secure 0 0 0) 0x7f0a540ad100 con 0x7f0a54142ad0 2020-11-28T21:52:28.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.109+0000 7f0a5359e700 1 -- 172.21.15.162:0/2707893350 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0a380553b0 msgr2=0x7f0a38057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:28.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.109+0000 7f0a5359e700 1 --2- 172.21.15.162:0/2707893350 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0a380553b0 0x7f0a38057870 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f0a3c01a000 tx=0x7f0a3c015000).stop 2020-11-28T21:52:28.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.109+0000 7f0a5359e700 1 -- 172.21.15.162:0/2707893350 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a54142ad0 msgr2=0x7f0a541522b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:28.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.109+0000 7f0a5359e700 1 --2- 172.21.15.162:0/2707893350 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a54142ad0 0x7f0a541522b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7f0a4c000f00 tx=0x7f0a4c007f70).stop 2020-11-28T21:52:28.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.109+0000 7f0a5359e700 1 -- 172.21.15.162:0/2707893350 shutdown_connections 2020-11-28T21:52:28.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.109+0000 7f0a5359e700 1 --2- 172.21.15.162:0/2707893350 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0a380553b0 0x7f0a38057870 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:28.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.109+0000 7f0a5359e700 1 --2- 172.21.15.162:0/2707893350 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a54142ad0 0x7f0a541522b0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:28.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.109+0000 7f0a5359e700 1 --2- 172.21.15.162:0/2707893350 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a540b2d00 0x7f0a540b8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:28.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.109+0000 7f0a5359e700 1 --2- 172.21.15.162:0/2707893350 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0a540b3ab0 0x7f0a54151dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:28.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.109+0000 7f0a5359e700 1 -- 172.21.15.162:0/2707893350 >> 172.21.15.162:0/2707893350 conn(0x7f0a540604c0 msgr2=0x7f0a5413f840 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:28.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.109+0000 7f0a5359e700 1 -- 172.21.15.162:0/2707893350 shutdown_connections 2020-11-28T21:52:28.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.109+0000 7f0a5359e700 1 -- 172.21.15.162:0/2707893350 wait complete. 2020-11-28T21:52:28.117 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_data' created 2020-11-28T21:52:28.125 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2020-11-28T21:52:28.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.245+0000 7fc7b67ed700 1 Processor -- start 2020-11-28T21:52:28.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.245+0000 7fc7b67ed700 1 -- start start 2020-11-28T21:52:28.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.249+0000 7fc7b67ed700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7b0060010 0x7fc7b0146a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:28.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.249+0000 7fc7b67ed700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7b00603f0 con 0x7fc7b005bdd0 2020-11-28T21:52:28.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.249+0000 7fc7b67ed700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7b0146060 con 0x7fc7b005b970 2020-11-28T21:52:28.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.249+0000 7fc7b67ed700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b0145e70 con 0x7fc7b0060010 2020-11-28T21:52:28.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.249+0000 7fc7b4fea700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fc7b005b970 0x7fc7b013fe80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:48138/0 (socket says 172.21.15.162:48138) 2020-11-28T21:52:28.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.249+0000 7fc7b4fea700 1 -- 172.21.15.162:0/2655175657 learned_addr learned my addr 172.21.15.162:0/2655175657 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:52:28.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.249+0000 7fc7b57eb700 1 --2- 172.21.15.162:0/2655175657 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7b0060010 0x7fc7b0146a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:28.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.249+0000 7fc7affff700 1 -- 172.21.15.162:0/2655175657 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4048901937 0 0) 0x7fc7b0146060 con 0x7fc7b005b970 2020-11-28T21:52:28.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.249+0000 7fc7affff700 1 -- 172.21.15.162:0/2655175657 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc79c003230 con 0x7fc7b005b970 2020-11-28T21:52:28.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.249+0000 7fc7affff700 1 -- 172.21.15.162:0/2655175657 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2724497296 0 0) 0x7fc7b00603f0 con 0x7fc7b005bdd0 2020-11-28T21:52:28.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.249+0000 7fc7affff700 1 -- 172.21.15.162:0/2655175657 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7b0146060 con 0x7fc7b005bdd0 2020-11-28T21:52:28.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.249+0000 7fc7affff700 1 -- 172.21.15.162:0/2655175657 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2798980493 0 0) 0x7fc79c003230 con 0x7fc7b005b970 2020-11-28T21:52:28.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.249+0000 7fc7b57eb700 1 -- 172.21.15.162:0/2655175657 >> v1:172.21.15.162:6789/0 conn(0x7fc7b005b970 legacy=0x7fc7b013fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:28.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.249+0000 7fc7b57eb700 1 -- 172.21.15.162:0/2655175657 >> v1:172.21.15.118:6789/0 conn(0x7fc7b005bdd0 legacy=0x7fc7b00610e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:28.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.249+0000 7fc7b57eb700 1 -- 172.21.15.162:0/2655175657 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b0146300 con 0x7fc7b0060010 2020-11-28T21:52:28.253 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.249+0000 7fc7b57eb700 1 --2- 172.21.15.162:0/2655175657 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7b0060010 0x7fc7b0146a40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fc7a40018b0 tx=0x7fc7a00091a0).ready entity=mon.2 client_cookie=b213eabe2ae3d8c6 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:28.253 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.249+0000 7fc7affff700 1 -- 172.21.15.162:0/2655175657 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7a4001710 con 0x7fc7b0060010 2020-11-28T21:52:28.253 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.249+0000 7fc7affff700 1 -- 172.21.15.162:0/2655175657 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a000b880 con 0x7fc7b0060010 2020-11-28T21:52:28.253 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.249+0000 7fc7affff700 1 -- 172.21.15.162:0/2655175657 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7a000b9f0 con 0x7fc7b0060010 2020-11-28T21:52:28.253 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.249+0000 7fc7b67ed700 1 -- 172.21.15.162:0/2655175657 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7b0060010 msgr2=0x7fc7b0146a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:28.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.249+0000 7fc7b67ed700 1 --2- 172.21.15.162:0/2655175657 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7b0060010 0x7fc7b0146a40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fc7a40018b0 tx=0x7fc7a00091a0).stop 2020-11-28T21:52:28.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.249+0000 7fc7b67ed700 1 -- 172.21.15.162:0/2655175657 shutdown_connections 2020-11-28T21:52:28.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.249+0000 7fc7b67ed700 1 --2- 172.21.15.162:0/2655175657 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7b0060010 0x7fc7b0146a40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:28.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.249+0000 7fc7b67ed700 1 -- 172.21.15.162:0/2655175657 >> 172.21.15.162:0/2655175657 conn(0x7fc7b00b5de0 msgr2=0x7fc7b00b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:28.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.249+0000 7fc7b67ed700 1 -- 172.21.15.162:0/2655175657 shutdown_connections 2020-11-28T21:52:28.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.249+0000 7fc7b67ed700 1 -- 172.21.15.162:0/2655175657 wait complete. 2020-11-28T21:52:28.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.253+0000 7fc7b67ed700 1 Processor -- start 2020-11-28T21:52:28.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.253+0000 7fc7b67ed700 1 -- start start 2020-11-28T21:52:28.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.253+0000 7fc7b67ed700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7b005b970 0x7fc7b00ab730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:28.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.253+0000 7fc7b67ed700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7b005bdd0 0x7fc7b00abc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:28.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.253+0000 7fc7b67ed700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7b0060010 0x7fc7b0065090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:28.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.253+0000 7fc7b67ed700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b0151850 con 0x7fc7b0060010 2020-11-28T21:52:28.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.253+0000 7fc7b67ed700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b00aa2a0 con 0x7fc7b005b970 2020-11-28T21:52:28.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.253+0000 7fc7b67ed700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b0152800 con 0x7fc7b005bdd0 2020-11-28T21:52:28.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.253+0000 7fc7b57eb700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7b005b970 0x7fc7b00ab730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:28.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.253+0000 7fc7b57eb700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7b005b970 0x7fc7b00ab730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41298/0 (socket says 172.21.15.162:41298) 2020-11-28T21:52:28.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.253+0000 7fc7b57eb700 1 -- 172.21.15.162:0/2377235261 learned_addr learned my addr 172.21.15.162:0/2377235261 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:28.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.257+0000 7fc7b4fea700 1 --2- 172.21.15.162:0/2377235261 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7b005bdd0 0x7fc7b00abc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:28.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.257+0000 7fc7b5fec700 1 --2- 172.21.15.162:0/2377235261 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7b0060010 0x7fc7b0065090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:28.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.257+0000 7fc7b57eb700 1 -- 172.21.15.162:0/2377235261 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7b005bdd0 msgr2=0x7fc7b00abc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:28.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.257+0000 7fc7b57eb700 1 --2- 172.21.15.162:0/2377235261 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7b005bdd0 0x7fc7b00abc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:28.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.257+0000 7fc7b57eb700 1 -- 172.21.15.162:0/2377235261 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7b0060010 msgr2=0x7fc7b0065090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:28.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.257+0000 7fc7b57eb700 1 --2- 172.21.15.162:0/2377235261 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7b0060010 0x7fc7b0065090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:28.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.257+0000 7fc7b57eb700 1 -- 172.21.15.162:0/2377235261 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b00aa000 con 0x7fc7b005b970 2020-11-28T21:52:28.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.257+0000 7fc7b5fec700 1 --2- 172.21.15.162:0/2377235261 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7b0060010 0x7fc7b0065090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:52:28.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.257+0000 7fc7b57eb700 1 --2- 172.21.15.162:0/2377235261 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7b005b970 0x7fc7b00ab730 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7fc7a40018b0 tx=0x7fc7a0016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:28.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.257+0000 7fc7ae7fc700 1 -- 172.21.15.162:0/2377235261 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7a00203f0 con 0x7fc7b005b970 2020-11-28T21:52:28.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.257+0000 7fc7ae7fc700 1 -- 172.21.15.162:0/2377235261 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a0020550 con 0x7fc7b005b970 2020-11-28T21:52:28.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.257+0000 7fc7ae7fc700 1 -- 172.21.15.162:0/2377235261 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7a00208b0 con 0x7fc7b005b970 2020-11-28T21:52:28.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.257+0000 7fc7b67ed700 1 -- 172.21.15.162:0/2377235261 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7b00ac360 con 0x7fc7b005b970 2020-11-28T21:52:28.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.257+0000 7fc7b67ed700 1 -- 172.21.15.162:0/2377235261 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7b00acb00 con 0x7fc7b005b970 2020-11-28T21:52:28.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.257+0000 7fc7b67ed700 1 -- 172.21.15.162:0/2377235261 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7b0057050 con 0x7fc7b005b970 2020-11-28T21:52:28.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.257+0000 7fc7ae7fc700 1 -- 172.21.15.162:0/2377235261 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc7a001e070 con 0x7fc7b005b970 2020-11-28T21:52:28.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.257+0000 7fc7ae7fc700 1 --2- 172.21.15.162:0/2377235261 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc790055330 0x7fc7900577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:28.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.257+0000 7fc7ae7fc700 1 -- 172.21.15.162:0/2377235261 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5957+0+0 (secure 0 0 0) 0x7fc7a006ea70 con 0x7fc7b005b970 2020-11-28T21:52:28.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.257+0000 7fc7b4fea700 1 --2- 172.21.15.162:0/2377235261 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc790055330 0x7fc7900577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:28.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.261+0000 7fc7b4fea700 1 --2- 172.21.15.162:0/2377235261 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc790055330 0x7fc7900577f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7fc7a400b000 tx=0x7fc7a4000b80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:28.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.265+0000 7fc7ae7fc700 1 -- 172.21.15.162:0/2377235261 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc7a0044de0 con 0x7fc7b005b970 2020-11-28T21:52:28.450 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.449+0000 7fc7b67ed700 1 -- 172.21.15.162:0/2377235261 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fc7b0069c00 con 0x7fc7b005b970 2020-11-28T21:52:28.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.465+0000 7fc7ae7fc700 1 -- 172.21.15.162:0/2377235261 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 13 and data pool 14 v32) v1 ==== 189+0+0 (secure 0 0 0) 0x7fc7a006ee60 con 0x7fc7b005b970 2020-11-28T21:52:28.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.473+0000 7fc7b67ed700 1 -- 172.21.15.162:0/2377235261 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc790055330 msgr2=0x7fc7900577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:28.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.473+0000 7fc7b67ed700 1 --2- 172.21.15.162:0/2377235261 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc790055330 0x7fc7900577f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7fc7a400b000 tx=0x7fc7a4000b80).stop 2020-11-28T21:52:28.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.473+0000 7fc7b67ed700 1 -- 172.21.15.162:0/2377235261 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7b005b970 msgr2=0x7fc7b00ab730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:28.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.473+0000 7fc7b67ed700 1 --2- 172.21.15.162:0/2377235261 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7b005b970 0x7fc7b00ab730 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7fc7a40018b0 tx=0x7fc7a0016000).stop 2020-11-28T21:52:28.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.473+0000 7fc7b67ed700 1 -- 172.21.15.162:0/2377235261 shutdown_connections 2020-11-28T21:52:28.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.473+0000 7fc7b67ed700 1 --2- 172.21.15.162:0/2377235261 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc790055330 0x7fc7900577f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:28.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.473+0000 7fc7b67ed700 1 --2- 172.21.15.162:0/2377235261 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7b0060010 0x7fc7b0065090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:28.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.473+0000 7fc7b67ed700 1 --2- 172.21.15.162:0/2377235261 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7b005b970 0x7fc7b00ab730 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:28.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.473+0000 7fc7b67ed700 1 --2- 172.21.15.162:0/2377235261 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7b005bdd0 0x7fc7b00abc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:28.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.473+0000 7fc7b67ed700 1 -- 172.21.15.162:0/2377235261 >> 172.21.15.162:0/2377235261 conn(0x7fc7b00b5de0 msgr2=0x7fc7b013dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:28.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.473+0000 7fc7b67ed700 1 -- 172.21.15.162:0/2377235261 shutdown_connections 2020-11-28T21:52:28.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.473+0000 7fc7b67ed700 1 -- 172.21.15.162:0/2377235261 wait complete. 2020-11-28T21:52:28.476 INFO:teuthology.orchestra.run.smithi162.stderr:new fs with metadata pool 13 and data pool 14 2020-11-28T21:52:28.491 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:52:28.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.589+0000 7f9061bf7700 1 Processor -- start 2020-11-28T21:52:28.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.589+0000 7f9061bf7700 1 -- start start 2020-11-28T21:52:28.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.589+0000 7f9061bf7700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f905c147a90 0x7f905c0b2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:28.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.589+0000 7f9061bf7700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f905c09ca50 0x7f905c0b2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:28.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.589+0000 7f9061bf7700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f905c0654e0 con 0x7f905c0b7780 2020-11-28T21:52:28.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.589+0000 7f9061bf7700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f905c065780 con 0x7f905c09ca50 2020-11-28T21:52:28.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.589+0000 7f9061bf7700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f905c065a20 con 0x7f905c147a90 2020-11-28T21:52:28.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.589+0000 7f905bfff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f905c147a90 0x7f905c0b2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:28.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.589+0000 7f905bfff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f905c147a90 0x7f905c0b2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41316/0 (socket says 172.21.15.162:41316) 2020-11-28T21:52:28.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.589+0000 7f905bfff700 1 -- 172.21.15.162:0/2822327616 learned_addr learned my addr 172.21.15.162:0/2822327616 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:28.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.589+0000 7f90613f6700 1 --2- 172.21.15.162:0/2822327616 >> v2:172.21.15.118:3300/0 conn(0x7f905c09ca50 0x7f905c0b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:28.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.589+0000 7f905bfff700 1 -- 172.21.15.162:0/2822327616 >> v2:172.21.15.118:3300/0 conn(0x7f905c09ca50 msgr2=0x7f905c0b2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:28.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.589+0000 7f905bfff700 1 --2- 172.21.15.162:0/2822327616 >> v2:172.21.15.118:3300/0 conn(0x7f905c09ca50 0x7f905c0b2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:28.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.589+0000 7f905bfff700 1 -- 172.21.15.162:0/2822327616 >> v1:172.21.15.118:6789/0 conn(0x7f905c0b7780 legacy=0x7f905c0b1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:28.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.589+0000 7f905bfff700 1 -- 172.21.15.162:0/2822327616 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f905c065cc0 con 0x7f905c147a90 2020-11-28T21:52:28.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.589+0000 7f90613f6700 1 --2- 172.21.15.162:0/2822327616 >> v2:172.21.15.118:3300/0 conn(0x7f905c09ca50 0x7f905c0b2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:52:28.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.589+0000 7f905bfff700 1 --2- 172.21.15.162:0/2822327616 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f905c147a90 0x7f905c0b2390 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7f905c0679e0 tx=0x7f9048006e80).ready entity=mon.1 client_cookie=38d167891483c62e server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:28.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.589+0000 7f905b7fe700 1 -- 172.21.15.162:0/2822327616 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f90480097b0 con 0x7f905c147a90 2020-11-28T21:52:28.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.589+0000 7f905b7fe700 1 -- 172.21.15.162:0/2822327616 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9048009910 con 0x7f905c147a90 2020-11-28T21:52:28.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.589+0000 7f905b7fe700 1 -- 172.21.15.162:0/2822327616 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9048009a80 con 0x7f905c147a90 2020-11-28T21:52:28.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.589+0000 7f9061bf7700 1 -- 172.21.15.162:0/2822327616 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f905c147a90 msgr2=0x7f905c0b2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:28.594 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.589+0000 7f9061bf7700 1 --2- 172.21.15.162:0/2822327616 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f905c147a90 0x7f905c0b2390 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7f905c0679e0 tx=0x7f9048006e80).stop 2020-11-28T21:52:28.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.589+0000 7f9061bf7700 1 -- 172.21.15.162:0/2822327616 shutdown_connections 2020-11-28T21:52:28.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.589+0000 7f9061bf7700 1 --2- 172.21.15.162:0/2822327616 >> v2:172.21.15.118:3300/0 conn(0x7f905c09ca50 0x7f905c0b2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:28.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.589+0000 7f9061bf7700 1 --2- 172.21.15.162:0/2822327616 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f905c147a90 0x7f905c0b2390 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:28.597 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.589+0000 7f9061bf7700 1 -- 172.21.15.162:0/2822327616 >> 172.21.15.162:0/2822327616 conn(0x7f905c05c440 msgr2=0x7f905c0b6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:28.597 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.589+0000 7f9061bf7700 1 -- 172.21.15.162:0/2822327616 shutdown_connections 2020-11-28T21:52:28.597 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.589+0000 7f9061bf7700 1 -- 172.21.15.162:0/2822327616 wait complete. 2020-11-28T21:52:28.597 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.593+0000 7f9061bf7700 1 Processor -- start 2020-11-28T21:52:28.598 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.593+0000 7f9061bf7700 1 -- start start 2020-11-28T21:52:28.598 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.593+0000 7f9061bf7700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f905c09ca50 0x7f905c0aacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:28.598 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.593+0000 7f9061bf7700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f905c0b7780 0x7f905c142e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:28.598 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.593+0000 7f9061bf7700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f905c147a90 0x7f905c143bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:28.598 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.593+0000 7f9061bf7700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905c14acd0 con 0x7f905c0b7780 2020-11-28T21:52:28.599 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.593+0000 7f9061bf7700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905c0adb80 con 0x7f905c147a90 2020-11-28T21:52:28.599 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.593+0000 7f9061bf7700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f905c14af00 con 0x7f905c09ca50 2020-11-28T21:52:28.599 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.593+0000 7f90613f6700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f905c147a90 0x7f905c143bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:28.599 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.593+0000 7f905bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f905c0b7780 0x7f905c142e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:28.599 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.593+0000 7f90613f6700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f905c147a90 0x7f905c143bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41324/0 (socket says 172.21.15.162:41324) 2020-11-28T21:52:28.600 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.593+0000 7f90613f6700 1 -- 172.21.15.162:0/2349321526 learned_addr learned my addr 172.21.15.162:0/2349321526 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:28.600 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.593+0000 7f9060bf5700 1 --2- 172.21.15.162:0/2349321526 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f905c09ca50 0x7f905c0aacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:28.600 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.593+0000 7f90613f6700 1 -- 172.21.15.162:0/2349321526 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f905c09ca50 msgr2=0x7f905c0aacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:28.600 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.593+0000 7f90613f6700 1 --2- 172.21.15.162:0/2349321526 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f905c09ca50 0x7f905c0aacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:28.600 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.593+0000 7f90613f6700 1 -- 172.21.15.162:0/2349321526 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f905c0b7780 msgr2=0x7f905c142e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:28.601 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.593+0000 7f90613f6700 1 --2- 172.21.15.162:0/2349321526 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f905c0b7780 0x7f905c142e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:28.601 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.593+0000 7f90613f6700 1 -- 172.21.15.162:0/2349321526 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f905c14b1a0 con 0x7f905c147a90 2020-11-28T21:52:28.601 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.593+0000 7f90613f6700 1 --2- 172.21.15.162:0/2349321526 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f905c147a90 0x7f905c143bb0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7f9054007000 tx=0x7f9054008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:28.601 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.593+0000 7f9059ffb700 1 -- 172.21.15.162:0/2349321526 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f905400a820 con 0x7f905c147a90 2020-11-28T21:52:28.601 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.593+0000 7f9059ffb700 1 -- 172.21.15.162:0/2349321526 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f905400a980 con 0x7f905c147a90 2020-11-28T21:52:28.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.593+0000 7f9061bf7700 1 -- 172.21.15.162:0/2349321526 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f905c155d70 con 0x7f905c147a90 2020-11-28T21:52:28.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.593+0000 7f9059ffb700 1 -- 172.21.15.162:0/2349321526 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f905400ace0 con 0x7f905c147a90 2020-11-28T21:52:28.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.593+0000 7f9061bf7700 1 -- 172.21.15.162:0/2349321526 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f905c14a960 con 0x7f905c147a90 2020-11-28T21:52:28.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.597+0000 7f9059ffb700 1 -- 172.21.15.162:0/2349321526 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9054011410 con 0x7f905c147a90 2020-11-28T21:52:28.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.597+0000 7f9061bf7700 1 -- 172.21.15.162:0/2349321526 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f905c155f00 con 0x7f905c147a90 2020-11-28T21:52:28.603 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.597+0000 7f9059ffb700 1 --2- 172.21.15.162:0/2349321526 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9044055470 0x7f9044057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:28.603 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.597+0000 7f9059ffb700 1 -- 172.21.15.162:0/2349321526 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f905406e440 con 0x7f905c147a90 2020-11-28T21:52:28.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.597+0000 7f9060bf5700 1 --2- 172.21.15.162:0/2349321526 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9044055470 0x7f9044057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:28.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.597+0000 7f9060bf5700 1 --2- 172.21.15.162:0/2349321526 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9044055470 0x7f9044057930 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f905000d000 tx=0x7f9050007b40).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:28.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.601+0000 7f9059ffb700 1 -- 172.21.15.162:0/2349321526 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f90540445d0 con 0x7f905c147a90 2020-11-28T21:52:28.743 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.741+0000 7f9061bf7700 1 -- 172.21.15.162:0/2349321526 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f905c057050 con 0x7f905c147a90 2020-11-28T21:52:28.745 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.745+0000 7f9059ffb700 1 -- 172.21.15.162:0/2349321526 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+16163 (secure 0 0 0) 0x7f9054047bf0 con 0x7f905c147a90 2020-11-28T21:52:28.745 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:52:28.746 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":75,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:52:28.454375+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:52:25.955831+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2020-11-28T21:52:27.265702+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:52:28.749 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.749+0000 7f9061bf7700 1 -- 172.21.15.162:0/2349321526 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9044055470 msgr2=0x7f9044057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:28.749 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.749+0000 7f9061bf7700 1 --2- 172.21.15.162:0/2349321526 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9044055470 0x7f9044057930 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f905000d000 tx=0x7f9050007b40).stop 2020-11-28T21:52:28.749 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.749+0000 7f9061bf7700 1 -- 172.21.15.162:0/2349321526 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f905c147a90 msgr2=0x7f905c143bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:28.749 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.749+0000 7f9061bf7700 1 --2- 172.21.15.162:0/2349321526 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f905c147a90 0x7f905c143bb0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7f9054007000 tx=0x7f9054008040).stop 2020-11-28T21:52:28.750 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.749+0000 7f9061bf7700 1 -- 172.21.15.162:0/2349321526 shutdown_connections 2020-11-28T21:52:28.750 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.749+0000 7f9061bf7700 1 --2- 172.21.15.162:0/2349321526 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9044055470 0x7f9044057930 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:28.750 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.749+0000 7f9061bf7700 1 --2- 172.21.15.162:0/2349321526 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f905c09ca50 0x7f905c0aacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:28.750 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.749+0000 7f9061bf7700 1 --2- 172.21.15.162:0/2349321526 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f905c147a90 0x7f905c143bb0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:28.751 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.749+0000 7f9061bf7700 1 --2- 172.21.15.162:0/2349321526 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f905c0b7780 0x7f905c142e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:28.751 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.749+0000 7f9061bf7700 1 -- 172.21.15.162:0/2349321526 >> 172.21.15.162:0/2349321526 conn(0x7f905c05c440 msgr2=0x7f905c144cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:28.751 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.749+0000 7f9061bf7700 1 -- 172.21.15.162:0/2349321526 shutdown_connections 2020-11-28T21:52:28.751 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.749+0000 7f9061bf7700 1 -- 172.21.15.162:0/2349321526 wait complete. 2020-11-28T21:52:28.766 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:52:28.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.873+0000 7f52cffff700 1 Processor -- start 2020-11-28T21:52:28.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.873+0000 7f52cffff700 1 -- start start 2020-11-28T21:52:28.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.873+0000 7f52cffff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f52d00aae10 0x7f52d005f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:28.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.873+0000 7f52cffff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d00a85c0 con 0x7f52d00ab990 2020-11-28T21:52:28.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.873+0000 7f52cffff700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d01448e0 con 0x7f52d00ab1f0 2020-11-28T21:52:28.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.873+0000 7f52cffff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d0144b80 con 0x7f52d00aae10 2020-11-28T21:52:28.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.873+0000 7f52cf7fe700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f52d00ab1f0 0x7f52d0143ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:48176/0 (socket says 172.21.15.162:48176) 2020-11-28T21:52:28.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.873+0000 7f52cf7fe700 1 -- 172.21.15.162:0/2886967179 learned_addr learned my addr 172.21.15.162:0/2886967179 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:52:28.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.873+0000 7f52ceffd700 1 --2- 172.21.15.162:0/2886967179 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f52d00aae10 0x7f52d005f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:28.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.873+0000 7f52cdffb700 1 -- 172.21.15.162:0/2886967179 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2078835970 0 0) 0x7f52d01448e0 con 0x7f52d00ab1f0 2020-11-28T21:52:28.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.873+0000 7f52cdffb700 1 -- 172.21.15.162:0/2886967179 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52b4003230 con 0x7f52d00ab1f0 2020-11-28T21:52:28.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.873+0000 7f52cdffb700 1 -- 172.21.15.162:0/2886967179 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1720149729 0 0) 0x7f52b4003230 con 0x7f52d00ab1f0 2020-11-28T21:52:28.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.873+0000 7f52cdffb700 1 -- 172.21.15.162:0/2886967179 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f52d00aae10 msgr2=0x7f52d005f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:28.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.873+0000 7f52cdffb700 1 --2- 172.21.15.162:0/2886967179 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f52d00aae10 0x7f52d005f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:28.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.873+0000 7f52cdffb700 1 -- 172.21.15.162:0/2886967179 >> v1:172.21.15.118:6789/0 conn(0x7f52d00ab990 legacy=0x7f52d0140800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:28.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.873+0000 7f52cdffb700 1 -- 172.21.15.162:0/2886967179 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d00af6b0 con 0x7f52d00ab1f0 2020-11-28T21:52:28.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.873+0000 7f52cdffb700 1 -- 172.21.15.162:0/2886967179 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f52c8001a50 con 0x7f52d00ab1f0 2020-11-28T21:52:28.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.873+0000 7f52cdffb700 1 -- 172.21.15.162:0/2886967179 >> v1:172.21.15.162:6789/0 conn(0x7f52d00ab1f0 legacy=0x7f52d0143ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:28.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.873+0000 7f52cdffb700 1 --2- 172.21.15.162:0/2886967179 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f52b4003720 0x7f52b4003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:28.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.873+0000 7f52cdffb700 1 --2- 172.21.15.162:0/2886967179 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f52d00aae10 0x7f52b40044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:28.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.873+0000 7f52cdffb700 1 --2- 172.21.15.162:0/2886967179 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52b40049c0 0x7f52b4004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:28.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.873+0000 7f52cdffb700 1 -- 172.21.15.162:0/2886967179 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d0144b80 con 0x7f52b4003720 2020-11-28T21:52:28.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.873+0000 7f52cdffb700 1 -- 172.21.15.162:0/2886967179 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b40041c0 con 0x7f52b40049c0 2020-11-28T21:52:28.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.873+0000 7f52cdffb700 1 -- 172.21.15.162:0/2886967179 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52b400b720 con 0x7f52d00aae10 2020-11-28T21:52:28.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.873+0000 7f52cf7fe700 1 --2- 172.21.15.162:0/2886967179 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52b40049c0 0x7f52b4004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:28.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.873+0000 7f52ceffd700 1 --2- 172.21.15.162:0/2886967179 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f52b4003720 0x7f52b4003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:28.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.873+0000 7f52cf7fe700 1 -- 172.21.15.162:0/2886967179 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f52d00aae10 msgr2=0x7f52b40044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:28.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.873+0000 7f52ce7fc700 1 --2- 172.21.15.162:0/2886967179 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f52d00aae10 0x7f52b40044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:28.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.877+0000 7f52cf7fe700 1 --2- 172.21.15.162:0/2886967179 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f52d00aae10 0x7f52b40044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:28.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.877+0000 7f52cf7fe700 1 -- 172.21.15.162:0/2886967179 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f52b4003720 msgr2=0x7f52b4003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:28.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.877+0000 7f52cf7fe700 1 --2- 172.21.15.162:0/2886967179 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f52b4003720 0x7f52b4003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:28.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.877+0000 7f52cf7fe700 1 -- 172.21.15.162:0/2886967179 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b4003230 con 0x7f52b40049c0 2020-11-28T21:52:28.886 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.877+0000 7f52ceffd700 1 --2- 172.21.15.162:0/2886967179 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f52b4003720 0x7f52b4003b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:52:28.886 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.877+0000 7f52ce7fc700 1 --2- 172.21.15.162:0/2886967179 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f52d00aae10 0x7f52b40044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:52:28.886 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.877+0000 7f52cf7fe700 1 --2- 172.21.15.162:0/2886967179 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52b40049c0 0x7f52b4004e40 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7f52c8001760 tx=0x7f52c8007af0).ready entity=mon.1 client_cookie=4b6df3dc9193a735 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:28.886 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.877+0000 7f52cdffb700 1 -- 172.21.15.162:0/2886967179 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52c8005b70 con 0x7f52b40049c0 2020-11-28T21:52:28.886 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.877+0000 7f52cdffb700 1 -- 172.21.15.162:0/2886967179 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c8005cd0 con 0x7f52b40049c0 2020-11-28T21:52:28.887 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.877+0000 7f52cdffb700 1 -- 172.21.15.162:0/2886967179 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52c80072c0 con 0x7f52b40049c0 2020-11-28T21:52:28.887 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.877+0000 7f52cffff700 1 -- 172.21.15.162:0/2886967179 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52b40049c0 msgr2=0x7f52b4004e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:28.888 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.877+0000 7f52cffff700 1 --2- 172.21.15.162:0/2886967179 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52b40049c0 0x7f52b4004e40 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7f52c8001760 tx=0x7f52c8007af0).stop 2020-11-28T21:52:28.888 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.877+0000 7f52cf7fe700 1 -- 172.21.15.162:0/2886967179 reap_dead start 2020-11-28T21:52:28.888 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.877+0000 7f52cffff700 1 -- 172.21.15.162:0/2886967179 shutdown_connections 2020-11-28T21:52:28.888 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.877+0000 7f52cffff700 1 -- 172.21.15.162:0/2886967179 >> 172.21.15.162:0/2886967179 conn(0x7f52d005b970 msgr2=0x7f52d005bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:28.888 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.877+0000 7f52cffff700 1 -- 172.21.15.162:0/2886967179 shutdown_connections 2020-11-28T21:52:28.889 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.877+0000 7f52cffff700 1 -- 172.21.15.162:0/2886967179 wait complete. 2020-11-28T21:52:28.889 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.877+0000 7f52cffff700 1 Processor -- start 2020-11-28T21:52:28.889 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.877+0000 7f52cffff700 1 -- start start 2020-11-28T21:52:28.889 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.877+0000 7f52cffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f52d013ce20 0x7f52d013a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:28.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.881+0000 7f52cffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52d00a9ab0 0x7f52d013ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:28.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.881+0000 7f52cffff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f52d00a9220 0x7f52d0151840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:28.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.881+0000 7f52cffff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d00aca00 con 0x7f52d013ce20 2020-11-28T21:52:28.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.881+0000 7f52cffff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d013d200 con 0x7f52d00a9ab0 2020-11-28T21:52:28.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.881+0000 7f52cffff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d00aa000 con 0x7f52d00a9220 2020-11-28T21:52:28.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.881+0000 7f52ce7fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52d00a9ab0 0x7f52d013ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:28.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.881+0000 7f52ce7fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52d00a9ab0 0x7f52d013ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41342/0 (socket says 172.21.15.162:41342) 2020-11-28T21:52:28.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.881+0000 7f52ce7fc700 1 -- 172.21.15.162:0/696567431 learned_addr learned my addr 172.21.15.162:0/696567431 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:28.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.881+0000 7f52ceffd700 1 --2- 172.21.15.162:0/696567431 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f52d013ce20 0x7f52d013a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:28.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.881+0000 7f52cf7fe700 1 --2- 172.21.15.162:0/696567431 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f52d00a9220 0x7f52d0151840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:28.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.881+0000 7f52ce7fc700 1 -- 172.21.15.162:0/696567431 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f52d00a9220 msgr2=0x7f52d0151840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:28.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.881+0000 7f52ce7fc700 1 --2- 172.21.15.162:0/696567431 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f52d00a9220 0x7f52d0151840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:28.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.881+0000 7f52ce7fc700 1 -- 172.21.15.162:0/696567431 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f52d013ce20 msgr2=0x7f52d013a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:28.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.881+0000 7f52ce7fc700 1 --2- 172.21.15.162:0/696567431 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f52d013ce20 0x7f52d013a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:28.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.881+0000 7f52ce7fc700 1 -- 172.21.15.162:0/696567431 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d00aa2a0 con 0x7f52d00a9ab0 2020-11-28T21:52:28.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.881+0000 7f52ceffd700 1 --2- 172.21.15.162:0/696567431 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f52d013ce20 0x7f52d013a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:52:28.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.881+0000 7f52cf7fe700 1 --2- 172.21.15.162:0/696567431 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f52d00a9220 0x7f52d0151840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:52:28.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.881+0000 7f52ce7fc700 1 --2- 172.21.15.162:0/696567431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52d00a9ab0 0x7f52d013ad30 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7f52c4008000 tx=0x7f52c4009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:28.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.881+0000 7f52bffff700 1 -- 172.21.15.162:0/696567431 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52c400f510 con 0x7f52d00a9ab0 2020-11-28T21:52:28.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.881+0000 7f52bffff700 1 -- 172.21.15.162:0/696567431 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c400f670 con 0x7f52d00a9ab0 2020-11-28T21:52:28.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.881+0000 7f52cffff700 1 -- 172.21.15.162:0/696567431 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52d00aa540 con 0x7f52d00a9ab0 2020-11-28T21:52:28.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.881+0000 7f52bffff700 1 -- 172.21.15.162:0/696567431 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52c400f9d0 con 0x7f52d00a9ab0 2020-11-28T21:52:28.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.881+0000 7f52cffff700 1 -- 172.21.15.162:0/696567431 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52d013b220 con 0x7f52d00a9ab0 2020-11-28T21:52:28.896 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.881+0000 7f52cffff700 1 -- 172.21.15.162:0/696567431 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52d0057050 con 0x7f52d00a9ab0 2020-11-28T21:52:28.896 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.881+0000 7f52bffff700 1 -- 172.21.15.162:0/696567431 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f52c400fb30 con 0x7f52d00a9ab0 2020-11-28T21:52:28.896 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.881+0000 7f52bffff700 1 --2- 172.21.15.162:0/696567431 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f52ac0554e0 0x7f52ac0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:28.897 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.881+0000 7f52bffff700 1 -- 172.21.15.162:0/696567431 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f52c406dcd0 con 0x7f52d00a9ab0 2020-11-28T21:52:28.897 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.881+0000 7f52ceffd700 1 --2- 172.21.15.162:0/696567431 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f52ac0554e0 0x7f52ac0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:28.897 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.885+0000 7f52ceffd700 1 --2- 172.21.15.162:0/696567431 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f52ac0554e0 0x7f52ac0579a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f52c0008000 tx=0x7f52c000b040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:28.897 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:28.889+0000 7f52bffff700 1 -- 172.21.15.162:0/696567431 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f52c4043fd0 con 0x7f52d00a9ab0 2020-11-28T21:52:29.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.029+0000 7f52cffff700 1 -- 172.21.15.162:0/696567431 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52d0068350 con 0x7f52d00a9ab0 2020-11-28T21:52:29.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.033+0000 7f52bffff700 1 -- 172.21.15.162:0/696567431 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+16163 (secure 0 0 0) 0x7f52c40475f0 con 0x7f52d00a9ab0 2020-11-28T21:52:29.034 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:52:29.034 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":75,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:52:28.454375+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:52:25.955831+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2020-11-28T21:52:27.265702+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:52:29.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.033+0000 7f52cffff700 1 -- 172.21.15.162:0/696567431 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f52ac0554e0 msgr2=0x7f52ac0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:29.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.033+0000 7f52cffff700 1 --2- 172.21.15.162:0/696567431 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f52ac0554e0 0x7f52ac0579a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f52c0008000 tx=0x7f52c000b040).stop 2020-11-28T21:52:29.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.033+0000 7f52cffff700 1 -- 172.21.15.162:0/696567431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52d00a9ab0 msgr2=0x7f52d013ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:29.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.033+0000 7f52cffff700 1 --2- 172.21.15.162:0/696567431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52d00a9ab0 0x7f52d013ad30 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7f52c4008000 tx=0x7f52c4009040).stop 2020-11-28T21:52:29.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.033+0000 7f52cffff700 1 -- 172.21.15.162:0/696567431 shutdown_connections 2020-11-28T21:52:29.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.033+0000 7f52cffff700 1 --2- 172.21.15.162:0/696567431 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f52ac0554e0 0x7f52ac0579a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:29.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.033+0000 7f52cffff700 1 --2- 172.21.15.162:0/696567431 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f52d00a9220 0x7f52d0151840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:29.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.033+0000 7f52cffff700 1 --2- 172.21.15.162:0/696567431 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f52d013ce20 0x7f52d013a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:29.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.033+0000 7f52cffff700 1 --2- 172.21.15.162:0/696567431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52d00a9ab0 0x7f52d013ad30 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:29.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.033+0000 7f52cffff700 1 -- 172.21.15.162:0/696567431 >> 172.21.15.162:0/696567431 conn(0x7f52d005b970 msgr2=0x7f52d013e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:29.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.033+0000 7f52cffff700 1 -- 172.21.15.162:0/696567431 shutdown_connections 2020-11-28T21:52:29.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.033+0000 7f52cffff700 1 -- 172.21.15.162:0/696567431 wait complete. 2020-11-28T21:52:29.049 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2020-11-28T21:52:29.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.161+0000 7f624ab8d700 1 Processor -- start 2020-11-28T21:52:29.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.161+0000 7f624ab8d700 1 -- start start 2020-11-28T21:52:29.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.161+0000 7f624ab8d700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f62441433c0 0x7f62440605c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:29.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.161+0000 7f624ab8d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6244067610 0x7f624413fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:29.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.161+0000 7f624ab8d700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6244143950 con 0x7f62440b9ce0 2020-11-28T21:52:29.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.161+0000 7f624ab8d700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62440b4340 con 0x7f6244067610 2020-11-28T21:52:29.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.161+0000 7f624ab8d700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6244067ac0 con 0x7f62441433c0 2020-11-28T21:52:29.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.161+0000 7f624912a700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6244067610 0x7f624413fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:29.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.161+0000 7f624912a700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6244067610 0x7f624413fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41352/0 (socket says 172.21.15.162:41352) 2020-11-28T21:52:29.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.165+0000 7f624912a700 1 -- 172.21.15.162:0/412296430 learned_addr learned my addr 172.21.15.162:0/412296430 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:29.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.165+0000 7f6243fff700 1 --2- 172.21.15.162:0/412296430 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f62441433c0 0x7f62440605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:29.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.165+0000 7f624912a700 1 -- 172.21.15.162:0/412296430 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f62441433c0 msgr2=0x7f62440605c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:29.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.165+0000 7f624912a700 1 --2- 172.21.15.162:0/412296430 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f62441433c0 0x7f62440605c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:29.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.165+0000 7f624912a700 1 -- 172.21.15.162:0/412296430 >> v1:172.21.15.118:6789/0 conn(0x7f62440b9ce0 legacy=0x7f62440b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:29.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.165+0000 7f624912a700 1 -- 172.21.15.162:0/412296430 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62440ba0c0 con 0x7f6244067610 2020-11-28T21:52:29.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.165+0000 7f624912a700 1 --2- 172.21.15.162:0/412296430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6244067610 0x7f624413fc80 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7f6244142a60 tx=0x7f623c009270).ready entity=mon.1 client_cookie=a192caab5226c95a server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:29.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.165+0000 7f62437fe700 1 -- 172.21.15.162:0/412296430 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f623c00bbc0 con 0x7f6244067610 2020-11-28T21:52:29.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.165+0000 7f62437fe700 1 -- 172.21.15.162:0/412296430 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f623c00bd20 con 0x7f6244067610 2020-11-28T21:52:29.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.165+0000 7f62437fe700 1 -- 172.21.15.162:0/412296430 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f623c00be90 con 0x7f6244067610 2020-11-28T21:52:29.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.165+0000 7f624ab8d700 1 -- 172.21.15.162:0/412296430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6244067610 msgr2=0x7f624413fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:29.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.165+0000 7f624ab8d700 1 --2- 172.21.15.162:0/412296430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6244067610 0x7f624413fc80 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7f6244142a60 tx=0x7f623c009270).stop 2020-11-28T21:52:29.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.165+0000 7f624ab8d700 1 -- 172.21.15.162:0/412296430 shutdown_connections 2020-11-28T21:52:29.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.165+0000 7f624ab8d700 1 --2- 172.21.15.162:0/412296430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6244067610 0x7f624413fc80 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:29.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.165+0000 7f624ab8d700 1 --2- 172.21.15.162:0/412296430 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f62441433c0 0x7f62440605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:29.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.165+0000 7f624ab8d700 1 -- 172.21.15.162:0/412296430 >> 172.21.15.162:0/412296430 conn(0x7f624405c430 msgr2=0x7f62440b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:29.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.165+0000 7f624ab8d700 1 -- 172.21.15.162:0/412296430 shutdown_connections 2020-11-28T21:52:29.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.165+0000 7f624ab8d700 1 -- 172.21.15.162:0/412296430 wait complete. 2020-11-28T21:52:29.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.169+0000 7f624ab8d700 1 Processor -- start 2020-11-28T21:52:29.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.169+0000 7f624ab8d700 1 -- start start 2020-11-28T21:52:29.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.169+0000 7f624ab8d700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6244067610 0x7f6244151880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:29.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.169+0000 7f624ab8d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62440b9ce0 0x7f6244151d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:29.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.169+0000 7f624ab8d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f62441433c0 0x7f62440ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:29.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.169+0000 7f624ab8d700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62440b6c20 con 0x7f62441433c0 2020-11-28T21:52:29.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.169+0000 7f624ab8d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62440ad850 con 0x7f62440b9ce0 2020-11-28T21:52:29.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.169+0000 7f624ab8d700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6244152410 con 0x7f6244067610 2020-11-28T21:52:29.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.169+0000 7f6243fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62440b9ce0 0x7f6244151d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:29.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.169+0000 7f6243fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62440b9ce0 0x7f6244151d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41356/0 (socket says 172.21.15.162:41356) 2020-11-28T21:52:29.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.169+0000 7f6243fff700 1 -- 172.21.15.162:0/588321918 learned_addr learned my addr 172.21.15.162:0/588321918 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:29.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.169+0000 7f624912a700 1 --2- 172.21.15.162:0/588321918 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f62441433c0 0x7f62440ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:29.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.169+0000 7f6248929700 1 --2- 172.21.15.162:0/588321918 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6244067610 0x7f6244151880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:29.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.169+0000 7f6243fff700 1 -- 172.21.15.162:0/588321918 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6244067610 msgr2=0x7f6244151880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:29.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.169+0000 7f6243fff700 1 --2- 172.21.15.162:0/588321918 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6244067610 0x7f6244151880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:29.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.169+0000 7f6243fff700 1 -- 172.21.15.162:0/588321918 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f62441433c0 msgr2=0x7f62440ae210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:29.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.169+0000 7f6243fff700 1 --2- 172.21.15.162:0/588321918 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f62441433c0 0x7f62440ae210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:29.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.169+0000 7f6243fff700 1 -- 172.21.15.162:0/588321918 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62441526b0 con 0x7f62440b9ce0 2020-11-28T21:52:29.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.169+0000 7f6243fff700 1 --2- 172.21.15.162:0/588321918 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62440b9ce0 0x7f6244151d70 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7f62440adfc0 tx=0x7f622c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:29.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.169+0000 7f6241ffb700 1 -- 172.21.15.162:0/588321918 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f622c0127e0 con 0x7f62440b9ce0 2020-11-28T21:52:29.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.169+0000 7f6241ffb700 1 -- 172.21.15.162:0/588321918 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f622c012940 con 0x7f62440b9ce0 2020-11-28T21:52:29.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.169+0000 7f624ab8d700 1 -- 172.21.15.162:0/588321918 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62440b0aa0 con 0x7f62440b9ce0 2020-11-28T21:52:29.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.169+0000 7f6241ffb700 1 -- 172.21.15.162:0/588321918 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f622c012ca0 con 0x7f62440b9ce0 2020-11-28T21:52:29.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.169+0000 7f624ab8d700 1 -- 172.21.15.162:0/588321918 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62440b0f40 con 0x7f62440b9ce0 2020-11-28T21:52:29.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.173+0000 7f624ab8d700 1 -- 172.21.15.162:0/588321918 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62440b2f90 con 0x7f62440b9ce0 2020-11-28T21:52:29.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.173+0000 7f6241ffb700 1 -- 172.21.15.162:0/588321918 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f622c013480 con 0x7f62440b9ce0 2020-11-28T21:52:29.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.173+0000 7f6241ffb700 1 --2- 172.21.15.162:0/588321918 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f62280554e0 0x7f62280579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:29.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.173+0000 7f6241ffb700 1 -- 172.21.15.162:0/588321918 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f622c06e530 con 0x7f62440b9ce0 2020-11-28T21:52:29.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.173+0000 7f6248929700 1 --2- 172.21.15.162:0/588321918 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f62280554e0 0x7f62280579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:29.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.173+0000 7f6248929700 1 --2- 172.21.15.162:0/588321918 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f62280554e0 0x7f62280579a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f623400d000 tx=0x7f6234007b50).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:29.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.181+0000 7f6241ffb700 1 -- 172.21.15.162:0/588321918 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f62440b2f90 con 0x7f62440b9ce0 2020-11-28T21:52:29.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.365+0000 7f624ab8d700 1 -- 172.21.15.162:0/588321918 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f62440b2f90 con 0x7f62440b9ce0 2020-11-28T21:52:29.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.461+0000 7f6241ffb700 1 -- 172.21.15.162:0/588321918 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v33) v1 ==== 118+0+0 (secure 0 0 0) 0x7f62440b2f90 con 0x7f62440b9ce0 2020-11-28T21:52:29.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.465+0000 7f624ab8d700 1 -- 172.21.15.162:0/588321918 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f62280554e0 msgr2=0x7f62280579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:29.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.465+0000 7f624ab8d700 1 --2- 172.21.15.162:0/588321918 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f62280554e0 0x7f62280579a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f623400d000 tx=0x7f6234007b50).stop 2020-11-28T21:52:29.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.465+0000 7f624ab8d700 1 -- 172.21.15.162:0/588321918 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62440b9ce0 msgr2=0x7f6244151d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:29.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.465+0000 7f624ab8d700 1 --2- 172.21.15.162:0/588321918 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62440b9ce0 0x7f6244151d70 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7f62440adfc0 tx=0x7f622c008040).stop 2020-11-28T21:52:29.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.465+0000 7f624ab8d700 1 -- 172.21.15.162:0/588321918 shutdown_connections 2020-11-28T21:52:29.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.465+0000 7f624ab8d700 1 --2- 172.21.15.162:0/588321918 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f62280554e0 0x7f62280579a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:29.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.465+0000 7f624ab8d700 1 --2- 172.21.15.162:0/588321918 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6244067610 0x7f6244151880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:29.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.465+0000 7f624ab8d700 1 --2- 172.21.15.162:0/588321918 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f62441433c0 0x7f62440ae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:29.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.465+0000 7f624ab8d700 1 --2- 172.21.15.162:0/588321918 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62440b9ce0 0x7f6244151d70 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:29.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.465+0000 7f624ab8d700 1 -- 172.21.15.162:0/588321918 >> 172.21.15.162:0/588321918 conn(0x7f624405c430 msgr2=0x7f624405f750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:29.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.469+0000 7f624ab8d700 1 -- 172.21.15.162:0/588321918 shutdown_connections 2020-11-28T21:52:29.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.469+0000 7f624ab8d700 1 -- 172.21.15.162:0/588321918 wait complete. 2020-11-28T21:52:29.485 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:52:29.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.601+0000 7fc704fc4700 1 Processor -- start 2020-11-28T21:52:29.603 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.601+0000 7fc704fc4700 1 -- start start 2020-11-28T21:52:29.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.601+0000 7fc704fc4700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7000615c0 0x7fc7000b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:29.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.601+0000 7fc704fc4700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc7000b4b40 0x7fc70013ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:29.605 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.601+0000 7fc704fc4700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7000b1790 con 0x7fc7000611e0 2020-11-28T21:52:29.605 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.601+0000 7fc704fc4700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7000b1a30 con 0x7fc7000b4b40 2020-11-28T21:52:29.605 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.601+0000 7fc704fc4700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7000b3f90 con 0x7fc7000615c0 2020-11-28T21:52:29.605 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.605+0000 7fc6fffff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc7000b4b40 0x7fc70013ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:29.606 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.605+0000 7fc6fffff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc7000b4b40 0x7fc70013ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41362/0 (socket says 172.21.15.162:41362) 2020-11-28T21:52:29.606 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.605+0000 7fc6fffff700 1 -- 172.21.15.162:0/1492925560 learned_addr learned my addr 172.21.15.162:0/1492925560 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:29.606 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.605+0000 7fc6feffd700 1 --2- 172.21.15.162:0/1492925560 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7000615c0 0x7fc7000b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:29.606 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.605+0000 7fc6fffff700 1 -- 172.21.15.162:0/1492925560 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7000615c0 msgr2=0x7fc7000b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:29.606 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.605+0000 7fc6fffff700 1 --2- 172.21.15.162:0/1492925560 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7000615c0 0x7fc7000b10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:29.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.605+0000 7fc6fffff700 1 -- 172.21.15.162:0/1492925560 >> v1:172.21.15.118:6789/0 conn(0x7fc7000611e0 legacy=0x7fc7000b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:29.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.605+0000 7fc6fffff700 1 -- 172.21.15.162:0/1492925560 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7000b4fb0 con 0x7fc7000b4b40 2020-11-28T21:52:29.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.605+0000 7fc6fffff700 1 --2- 172.21.15.162:0/1492925560 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7000b4b40 0x7fc70013ee60 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7fc6f8002f30 tx=0x7fc6f8006e80).ready entity=mon.1 client_cookie=6af6a7f3233f9b34 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:29.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.605+0000 7fc6fe7fc700 1 -- 172.21.15.162:0/1492925560 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6f80097f0 con 0x7fc7000b4b40 2020-11-28T21:52:29.608 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.605+0000 7fc6fe7fc700 1 -- 172.21.15.162:0/1492925560 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f8009950 con 0x7fc7000b4b40 2020-11-28T21:52:29.608 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.605+0000 7fc6fe7fc700 1 -- 172.21.15.162:0/1492925560 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6f8009ac0 con 0x7fc7000b4b40 2020-11-28T21:52:29.608 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.605+0000 7fc704fc4700 1 -- 172.21.15.162:0/1492925560 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7000b4b40 msgr2=0x7fc70013ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:29.608 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.605+0000 7fc704fc4700 1 --2- 172.21.15.162:0/1492925560 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7000b4b40 0x7fc70013ee60 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7fc6f8002f30 tx=0x7fc6f8006e80).stop 2020-11-28T21:52:29.611 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.605+0000 7fc704fc4700 1 -- 172.21.15.162:0/1492925560 shutdown_connections 2020-11-28T21:52:29.612 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.605+0000 7fc704fc4700 1 --2- 172.21.15.162:0/1492925560 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7000b4b40 0x7fc70013ee60 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:29.612 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.605+0000 7fc704fc4700 1 --2- 172.21.15.162:0/1492925560 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7000615c0 0x7fc7000b10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:29.612 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.605+0000 7fc704fc4700 1 -- 172.21.15.162:0/1492925560 >> 172.21.15.162:0/1492925560 conn(0x7fc7000b54f0 msgr2=0x7fc7000b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:29.612 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.605+0000 7fc704fc4700 1 -- 172.21.15.162:0/1492925560 shutdown_connections 2020-11-28T21:52:29.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.605+0000 7fc704fc4700 1 -- 172.21.15.162:0/1492925560 wait complete. 2020-11-28T21:52:29.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.605+0000 7fc704fc4700 1 Processor -- start 2020-11-28T21:52:29.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.605+0000 7fc704fc4700 1 -- start start 2020-11-28T21:52:29.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.609+0000 7fc704fc4700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7000611e0 0x7fc7000656c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:29.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.609+0000 7fc704fc4700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7000615c0 0x7fc70012e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:29.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.609+0000 7fc704fc4700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7000b4b40 0x7fc70013ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:29.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.609+0000 7fc704fc4700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70009b4d0 con 0x7fc7000b4b40 2020-11-28T21:52:29.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.609+0000 7fc704fc4700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc700067690 con 0x7fc7000611e0 2020-11-28T21:52:29.615 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.609+0000 7fc704fc4700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc700067930 con 0x7fc7000615c0 2020-11-28T21:52:29.615 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.609+0000 7fc6ff7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7000611e0 0x7fc7000656c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:29.615 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.609+0000 7fc6ff7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7000611e0 0x7fc7000656c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41368/0 (socket says 172.21.15.162:41368) 2020-11-28T21:52:29.615 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.609+0000 7fc6ff7fe700 1 -- 172.21.15.162:0/426331131 learned_addr learned my addr 172.21.15.162:0/426331131 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:29.615 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.609+0000 7fc6fffff700 1 --2- 172.21.15.162:0/426331131 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7000b4b40 0x7fc70013ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:29.616 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.609+0000 7fc6feffd700 1 --2- 172.21.15.162:0/426331131 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7000615c0 0x7fc70012e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:29.616 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.609+0000 7fc6ff7fe700 1 -- 172.21.15.162:0/426331131 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7000615c0 msgr2=0x7fc70012e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:29.616 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.609+0000 7fc6ff7fe700 1 --2- 172.21.15.162:0/426331131 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7000615c0 0x7fc70012e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:29.616 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.609+0000 7fc6ff7fe700 1 -- 172.21.15.162:0/426331131 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7000b4b40 msgr2=0x7fc70013ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:29.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.609+0000 7fc6ff7fe700 1 --2- 172.21.15.162:0/426331131 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7000b4b40 0x7fc70013ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:29.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.609+0000 7fc6ff7fe700 1 -- 172.21.15.162:0/426331131 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc70009b150 con 0x7fc7000611e0 2020-11-28T21:52:29.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.609+0000 7fc6ff7fe700 1 --2- 172.21.15.162:0/426331131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7000611e0 0x7fc7000656c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7fc7000b4310 tx=0x7fc6f0007be0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:29.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.609+0000 7fc6fcff9700 1 -- 172.21.15.162:0/426331131 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6f0007240 con 0x7fc7000611e0 2020-11-28T21:52:29.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.609+0000 7fc6fcff9700 1 -- 172.21.15.162:0/426331131 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f00073a0 con 0x7fc7000611e0 2020-11-28T21:52:29.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.609+0000 7fc704fc4700 1 -- 172.21.15.162:0/426331131 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc700099650 con 0x7fc7000611e0 2020-11-28T21:52:29.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.609+0000 7fc6fcff9700 1 -- 172.21.15.162:0/426331131 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6f00066d0 con 0x7fc7000611e0 2020-11-28T21:52:29.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.609+0000 7fc704fc4700 1 -- 172.21.15.162:0/426331131 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc70013c0f0 con 0x7fc7000611e0 2020-11-28T21:52:29.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.609+0000 7fc6fcff9700 1 -- 172.21.15.162:0/426331131 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc6f0010070 con 0x7fc7000611e0 2020-11-28T21:52:29.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.609+0000 7fc6fcff9700 1 --2- 172.21.15.162:0/426331131 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc6d8055500 0x7fc6d80579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:29.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.609+0000 7fc704fc4700 1 -- 172.21.15.162:0/426331131 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc700067b30 con 0x7fc7000611e0 2020-11-28T21:52:29.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.609+0000 7fc6fcff9700 1 -- 172.21.15.162:0/426331131 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc6f006d590 con 0x7fc7000611e0 2020-11-28T21:52:29.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.609+0000 7fc6feffd700 1 --2- 172.21.15.162:0/426331131 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc6d8055500 0x7fc6d80579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:29.620 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.613+0000 7fc6feffd700 1 --2- 172.21.15.162:0/426331131 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc6d8055500 0x7fc6d80579c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7fc6e800b000 tx=0x7fc6e80091a0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:29.620 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.617+0000 7fc6fcff9700 1 -- 172.21.15.162:0/426331131 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc6f00472c0 con 0x7fc7000611e0 2020-11-28T21:52:29.726 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2020-11-28T21:52:29.726 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2020-11-28T21:52:29.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.801+0000 7fc704fc4700 1 -- 172.21.15.162:0/426331131 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc700057050 con 0x7fc7000611e0 2020-11-28T21:52:29.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.805+0000 7fc6fcff9700 1 -- 172.21.15.162:0/426331131 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+2499 (secure 0 0 0) 0x7fc6f0046eb0 con 0x7fc7000611e0 2020-11-28T21:52:29.805 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:52:29.805 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":33,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5540,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2116325247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2116325247},{"type":"v1","addr":"172.21.15.162:6835","nonce":2116325247}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":30},{"gid":5543,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.118:6833/3491069258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3491069258},{"type":"v1","addr":"172.21.15.118:6833","nonce":3491069258}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:52:28.454745+0000","modified":"2020-11-28T21:52:29.457610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6172},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6172":{"gid":6172,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/163256983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":163256983},{"type":"v1","addr":"172.21.15.162:6837","nonce":163256983}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-28T21:52:29.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.805+0000 7fc704fc4700 1 -- 172.21.15.162:0/426331131 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc6d8055500 msgr2=0x7fc6d80579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:29.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.805+0000 7fc704fc4700 1 --2- 172.21.15.162:0/426331131 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc6d8055500 0x7fc6d80579c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7fc6e800b000 tx=0x7fc6e80091a0).stop 2020-11-28T21:52:29.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.805+0000 7fc704fc4700 1 -- 172.21.15.162:0/426331131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7000611e0 msgr2=0x7fc7000656c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:29.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.805+0000 7fc704fc4700 1 --2- 172.21.15.162:0/426331131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7000611e0 0x7fc7000656c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7fc7000b4310 tx=0x7fc6f0007be0).stop 2020-11-28T21:52:29.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.805+0000 7fc704fc4700 1 -- 172.21.15.162:0/426331131 shutdown_connections 2020-11-28T21:52:29.810 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.805+0000 7fc704fc4700 1 --2- 172.21.15.162:0/426331131 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc6d8055500 0x7fc6d80579c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:29.810 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.805+0000 7fc704fc4700 1 --2- 172.21.15.162:0/426331131 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7000b4b40 0x7fc70013ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:29.811 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.805+0000 7fc704fc4700 1 --2- 172.21.15.162:0/426331131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7000611e0 0x7fc7000656c0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:29.811 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.805+0000 7fc704fc4700 1 --2- 172.21.15.162:0/426331131 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7000615c0 0x7fc70012e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:29.811 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.805+0000 7fc704fc4700 1 -- 172.21.15.162:0/426331131 >> 172.21.15.162:0/426331131 conn(0x7fc7000b54f0 msgr2=0x7fc70013cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:29.812 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.805+0000 7fc704fc4700 1 -- 172.21.15.162:0/426331131 shutdown_connections 2020-11-28T21:52:29.812 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.805+0000 7fc704fc4700 1 -- 172.21.15.162:0/426331131 wait complete. 2020-11-28T21:52:29.813 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 33 2020-11-28T21:52:29.818 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:52:29.933 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.933+0000 7f74627e2700 1 Processor -- start 2020-11-28T21:52:29.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.933+0000 7f74627e2700 1 -- start start 2020-11-28T21:52:29.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.933+0000 7f74627e2700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f745c066a10 0x7f745c061760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:29.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.933+0000 7f74627e2700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f745c143da0 0x7f745c061c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:29.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.933+0000 7f74627e2700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f745c0b1500 0x7f745c05b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:29.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.933+0000 7f74627e2700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f745c066df0 con 0x7f745c0b1500 2020-11-28T21:52:29.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.933+0000 7f74627e2700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f745c060100 con 0x7f745c066a10 2020-11-28T21:52:29.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.933+0000 7f74627e2700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f745c0b12b0 con 0x7f745c143da0 2020-11-28T21:52:29.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.933+0000 7f74617e0700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f745c066a10 0x7f745c061760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:29.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.933+0000 7f74617e0700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f745c066a10 0x7f745c061760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41378/0 (socket says 172.21.15.162:41378) 2020-11-28T21:52:29.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.933+0000 7f74617e0700 1 -- 172.21.15.162:0/153288789 learned_addr learned my addr 172.21.15.162:0/153288789 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:29.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.933+0000 7f7460fdf700 1 --2- 172.21.15.162:0/153288789 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f745c143da0 0x7f745c061c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:29.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.933+0000 7f7461fe1700 1 --2- 172.21.15.162:0/153288789 >> v2:172.21.15.118:3300/0 conn(0x7f745c0b1500 0x7f745c05b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:29.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.937+0000 7f74617e0700 1 -- 172.21.15.162:0/153288789 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f745c143da0 msgr2=0x7f745c061c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:29.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.937+0000 7f74617e0700 1 --2- 172.21.15.162:0/153288789 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f745c143da0 0x7f745c061c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:29.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.937+0000 7f74617e0700 1 -- 172.21.15.162:0/153288789 >> v2:172.21.15.118:3300/0 conn(0x7f745c0b1500 msgr2=0x7f745c05b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:29.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.937+0000 7f74617e0700 1 --2- 172.21.15.162:0/153288789 >> v2:172.21.15.118:3300/0 conn(0x7f745c0b1500 0x7f745c05b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:29.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.937+0000 7f74617e0700 1 -- 172.21.15.162:0/153288789 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f745c0b0d70 con 0x7f745c066a10 2020-11-28T21:52:29.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.937+0000 7f74617e0700 1 --2- 172.21.15.162:0/153288789 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f745c066a10 0x7f745c061760 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7f744c002f30 tx=0x7f744c006e80).ready entity=mon.1 client_cookie=a8d6068653372f1b server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:29.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.937+0000 7f745bfff700 1 -- 172.21.15.162:0/153288789 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f744c009640 con 0x7f745c066a10 2020-11-28T21:52:29.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.937+0000 7f745bfff700 1 -- 172.21.15.162:0/153288789 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f744c0097a0 con 0x7f745c066a10 2020-11-28T21:52:29.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.937+0000 7f745bfff700 1 -- 172.21.15.162:0/153288789 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f744c009b00 con 0x7f745c066a10 2020-11-28T21:52:29.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.937+0000 7f74627e2700 1 -- 172.21.15.162:0/153288789 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f745c066a10 msgr2=0x7f745c061760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:29.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.937+0000 7f74627e2700 1 --2- 172.21.15.162:0/153288789 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f745c066a10 0x7f745c061760 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7f744c002f30 tx=0x7f744c006e80).stop 2020-11-28T21:52:29.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.937+0000 7f74627e2700 1 -- 172.21.15.162:0/153288789 shutdown_connections 2020-11-28T21:52:29.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.937+0000 7f74627e2700 1 --2- 172.21.15.162:0/153288789 >> v2:172.21.15.118:3300/0 conn(0x7f745c0b1500 0x7f745c05b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:29.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.937+0000 7f74627e2700 1 --2- 172.21.15.162:0/153288789 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f745c066a10 0x7f745c061760 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:29.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.937+0000 7f74627e2700 1 --2- 172.21.15.162:0/153288789 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f745c143da0 0x7f745c061c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:29.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.937+0000 7f74627e2700 1 -- 172.21.15.162:0/153288789 >> 172.21.15.162:0/153288789 conn(0x7f745c0b5da0 msgr2=0x7f745c0b6180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:29.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.937+0000 7f74627e2700 1 -- 172.21.15.162:0/153288789 shutdown_connections 2020-11-28T21:52:29.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.937+0000 7f74627e2700 1 -- 172.21.15.162:0/153288789 wait complete. 2020-11-28T21:52:29.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.937+0000 7f74627e2700 1 Processor -- start 2020-11-28T21:52:29.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.937+0000 7f74627e2700 1 -- start start 2020-11-28T21:52:29.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.937+0000 7f74627e2700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f745c066a10 0x7f745c1463a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:29.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.937+0000 7f74627e2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f745c0b1500 0x7f745c0b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:29.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.937+0000 7f74627e2700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f745c143da0 0x7f745c139920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:29.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.937+0000 7f74627e2700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745c138870 con 0x7f745c066a10 2020-11-28T21:52:29.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.937+0000 7f74627e2700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745c0b41d0 con 0x7f745c0b1500 2020-11-28T21:52:29.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.937+0000 7f74627e2700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f745c138aa0 con 0x7f745c143da0 2020-11-28T21:52:29.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.941+0000 7f7460fdf700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f745c0b1500 0x7f745c0b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:29.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.941+0000 7f7460fdf700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f745c0b1500 0x7f745c0b48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41384/0 (socket says 172.21.15.162:41384) 2020-11-28T21:52:29.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.941+0000 7f7460fdf700 1 -- 172.21.15.162:0/2770298627 learned_addr learned my addr 172.21.15.162:0/2770298627 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:29.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.941+0000 7f74617e0700 1 --2- 172.21.15.162:0/2770298627 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f745c066a10 0x7f745c1463a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:29.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.941+0000 7f7461fe1700 1 --2- 172.21.15.162:0/2770298627 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f745c143da0 0x7f745c139920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:29.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.941+0000 7f7460fdf700 1 -- 172.21.15.162:0/2770298627 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f745c143da0 msgr2=0x7f745c139920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:29.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.941+0000 7f7460fdf700 1 --2- 172.21.15.162:0/2770298627 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f745c143da0 0x7f745c139920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:29.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.941+0000 7f7460fdf700 1 -- 172.21.15.162:0/2770298627 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f745c066a10 msgr2=0x7f745c1463a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:29.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.941+0000 7f7460fdf700 1 --2- 172.21.15.162:0/2770298627 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f745c066a10 0x7f745c1463a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:29.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.941+0000 7f7460fdf700 1 -- 172.21.15.162:0/2770298627 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f745c0b4430 con 0x7f745c0b1500 2020-11-28T21:52:29.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.941+0000 7f7460fdf700 1 --2- 172.21.15.162:0/2770298627 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f745c0b1500 0x7f745c0b48d0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7f745000c000 tx=0x7f74500091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:29.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.941+0000 7f745a7fc700 1 -- 172.21.15.162:0/2770298627 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f745000cb90 con 0x7f745c0b1500 2020-11-28T21:52:29.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.941+0000 7f745a7fc700 1 -- 172.21.15.162:0/2770298627 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f745000ccf0 con 0x7f745c0b1500 2020-11-28T21:52:29.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.941+0000 7f74627e2700 1 -- 172.21.15.162:0/2770298627 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f745c0b58e0 con 0x7f745c0b1500 2020-11-28T21:52:29.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.941+0000 7f745a7fc700 1 -- 172.21.15.162:0/2770298627 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74500125e0 con 0x7f745c0b1500 2020-11-28T21:52:29.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.941+0000 7f74627e2700 1 -- 172.21.15.162:0/2770298627 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f745c139130 con 0x7f745c0b1500 2020-11-28T21:52:29.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.941+0000 7f74627e2700 1 -- 172.21.15.162:0/2770298627 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f745c057050 con 0x7f745c0b1500 2020-11-28T21:52:29.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.941+0000 7f745a7fc700 1 -- 172.21.15.162:0/2770298627 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7450012740 con 0x7f745c0b1500 2020-11-28T21:52:29.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.945+0000 7f745a7fc700 1 --2- 172.21.15.162:0/2770298627 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f743c0553b0 0x7f743c057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:29.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.945+0000 7f745a7fc700 1 -- 172.21.15.162:0/2770298627 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f745006e980 con 0x7f745c0b1500 2020-11-28T21:52:29.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.945+0000 7f74617e0700 1 --2- 172.21.15.162:0/2770298627 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f743c0553b0 0x7f743c057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:29.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.945+0000 7f74617e0700 1 --2- 172.21.15.162:0/2770298627 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f743c0553b0 0x7f743c057870 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f744c01b000 tx=0x7f744c015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:29.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:29.949+0000 7f745a7fc700 1 -- 172.21.15.162:0/2770298627 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7450044c20 con 0x7f745c0b1500 2020-11-28T21:52:30.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.093+0000 7f74627e2700 1 -- 172.21.15.162:0/2770298627 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f745c061f20 con 0x7f745c0b1500 2020-11-28T21:52:30.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.093+0000 7f745a7fc700 1 -- 172.21.15.162:0/2770298627 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+16137 (secure 0 0 0) 0x7f745000ce60 con 0x7f745c0b1500 2020-11-28T21:52:30.094 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:52:30.094 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":76,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:52:29.458247+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:52:25.955831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2020-11-28T21:52:27.265702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:52:30.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.097+0000 7f74627e2700 1 -- 172.21.15.162:0/2770298627 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f743c0553b0 msgr2=0x7f743c057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:30.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.097+0000 7f74627e2700 1 --2- 172.21.15.162:0/2770298627 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f743c0553b0 0x7f743c057870 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f744c01b000 tx=0x7f744c015000).stop 2020-11-28T21:52:30.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.097+0000 7f74627e2700 1 -- 172.21.15.162:0/2770298627 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f745c0b1500 msgr2=0x7f745c0b48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:30.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.097+0000 7f74627e2700 1 --2- 172.21.15.162:0/2770298627 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f745c0b1500 0x7f745c0b48d0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7f745000c000 tx=0x7f74500091a0).stop 2020-11-28T21:52:30.098 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.097+0000 7f74627e2700 1 -- 172.21.15.162:0/2770298627 shutdown_connections 2020-11-28T21:52:30.098 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.097+0000 7f74627e2700 1 --2- 172.21.15.162:0/2770298627 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f743c0553b0 0x7f743c057870 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:30.098 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.097+0000 7f74627e2700 1 --2- 172.21.15.162:0/2770298627 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f745c143da0 0x7f745c139920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:30.098 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.097+0000 7f74627e2700 1 --2- 172.21.15.162:0/2770298627 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f745c066a10 0x7f745c1463a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:30.098 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.097+0000 7f74627e2700 1 --2- 172.21.15.162:0/2770298627 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f745c0b1500 0x7f745c0b48d0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:30.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.097+0000 7f74627e2700 1 -- 172.21.15.162:0/2770298627 >> 172.21.15.162:0/2770298627 conn(0x7f745c0b5da0 msgr2=0x7f745c140840 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:30.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.097+0000 7f74627e2700 1 -- 172.21.15.162:0/2770298627 shutdown_connections 2020-11-28T21:52:30.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.097+0000 7f74627e2700 1 -- 172.21.15.162:0/2770298627 wait complete. 2020-11-28T21:52:30.114 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-11-28T21:52:30.241 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.237+0000 7f42c2ae3700 1 Processor -- start 2020-11-28T21:52:30.242 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.237+0000 7f42c2ae3700 1 -- start start 2020-11-28T21:52:30.242 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.237+0000 7f42c2ae3700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f42bc0b2e80 0x7f42bc05f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:30.242 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.237+0000 7f42c2ae3700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f42bc0acb40 0x7f42bc05f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:30.243 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.237+0000 7f42c2ae3700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42bc0ab0b0 con 0x7f42bc0b2560 2020-11-28T21:52:30.243 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.237+0000 7f42c2ae3700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42bc05fd60 con 0x7f42bc0b2e80 2020-11-28T21:52:30.243 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.237+0000 7f42c2ae3700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42bc0b3260 con 0x7f42bc0acb40 2020-11-28T21:52:30.243 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.237+0000 7f42c3d45700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f42bc0acb40 0x7f42bc05f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:30.243 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.237+0000 7f42c3d45700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f42bc0acb40 0x7f42bc05f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41394/0 (socket says 172.21.15.162:41394) 2020-11-28T21:52:30.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.237+0000 7f42c3d45700 1 -- 172.21.15.162:0/1189991352 learned_addr learned my addr 172.21.15.162:0/1189991352 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:30.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.237+0000 7f42c22e2700 1 --2- 172.21.15.162:0/1189991352 >> v2:172.21.15.118:3300/0 conn(0x7f42bc0b2e80 0x7f42bc05f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:30.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.237+0000 7f42c3d45700 1 -- 172.21.15.162:0/1189991352 >> v2:172.21.15.118:3300/0 conn(0x7f42bc0b2e80 msgr2=0x7f42bc05f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:30.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.237+0000 7f42c3d45700 1 --2- 172.21.15.162:0/1189991352 >> v2:172.21.15.118:3300/0 conn(0x7f42bc0b2e80 0x7f42bc05f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:30.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.237+0000 7f42c3d45700 1 -- 172.21.15.162:0/1189991352 >> v1:172.21.15.118:6789/0 conn(0x7f42bc0b2560 legacy=0x7f42bc060d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:30.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.237+0000 7f42c3d45700 1 -- 172.21.15.162:0/1189991352 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42bc0ab2e0 con 0x7f42bc0acb40 2020-11-28T21:52:30.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.237+0000 7f42c3d45700 1 --2- 172.21.15.162:0/1189991352 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42bc0acb40 0x7f42bc05f870 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7f42b40032b0 tx=0x7f42b4009270).ready entity=mon.1 client_cookie=6e254e0cedbb31b server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:30.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.237+0000 7f42c12e0700 1 -- 172.21.15.162:0/1189991352 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f42b4009e60 con 0x7f42bc0acb40 2020-11-28T21:52:30.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.241+0000 7f42c12e0700 1 -- 172.21.15.162:0/1189991352 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42b400bc00 con 0x7f42bc0acb40 2020-11-28T21:52:30.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.241+0000 7f42c12e0700 1 -- 172.21.15.162:0/1189991352 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f42b400bd70 con 0x7f42bc0acb40 2020-11-28T21:52:30.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.241+0000 7f42c2ae3700 1 -- 172.21.15.162:0/1189991352 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42bc0acb40 msgr2=0x7f42bc05f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:30.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.241+0000 7f42c2ae3700 1 --2- 172.21.15.162:0/1189991352 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42bc0acb40 0x7f42bc05f870 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7f42b40032b0 tx=0x7f42b4009270).stop 2020-11-28T21:52:30.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.241+0000 7f42c2ae3700 1 -- 172.21.15.162:0/1189991352 shutdown_connections 2020-11-28T21:52:30.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.241+0000 7f42c2ae3700 1 --2- 172.21.15.162:0/1189991352 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42bc0acb40 0x7f42bc05f870 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:30.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.241+0000 7f42c2ae3700 1 --2- 172.21.15.162:0/1189991352 >> v2:172.21.15.118:3300/0 conn(0x7f42bc0b2e80 0x7f42bc05f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:30.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.241+0000 7f42c2ae3700 1 -- 172.21.15.162:0/1189991352 >> 172.21.15.162:0/1189991352 conn(0x7f42bc063b60 msgr2=0x7f42bc063f40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:30.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.241+0000 7f42c2ae3700 1 -- 172.21.15.162:0/1189991352 shutdown_connections 2020-11-28T21:52:30.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.241+0000 7f42c2ae3700 1 -- 172.21.15.162:0/1189991352 wait complete. 2020-11-28T21:52:30.253 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.241+0000 7f42c2ae3700 1 Processor -- start 2020-11-28T21:52:30.253 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.241+0000 7f42c2ae3700 1 -- start start 2020-11-28T21:52:30.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.245+0000 7f42c2ae3700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f42bc0acb40 0x7f42bc130980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:30.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.245+0000 7f42c2ae3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42bc0b2560 0x7f42bc099230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:30.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.245+0000 7f42c2ae3700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f42bc0b2e80 0x7f42bc141a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:30.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.245+0000 7f42c2ae3700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc0678c0 con 0x7f42bc0b2e80 2020-11-28T21:52:30.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.245+0000 7f42c2ae3700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc0668d0 con 0x7f42bc0b2560 2020-11-28T21:52:30.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.245+0000 7f42c2ae3700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc065ef0 con 0x7f42bc0acb40 2020-11-28T21:52:30.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.245+0000 7f42c1ae1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42bc0b2560 0x7f42bc099230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:30.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.245+0000 7f42c1ae1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42bc0b2560 0x7f42bc099230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41398/0 (socket says 172.21.15.162:41398) 2020-11-28T21:52:30.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.245+0000 7f42c1ae1700 1 -- 172.21.15.162:0/3062424550 learned_addr learned my addr 172.21.15.162:0/3062424550 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:30.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.245+0000 7f42c22e2700 1 --2- 172.21.15.162:0/3062424550 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f42bc0acb40 0x7f42bc130980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:30.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.245+0000 7f42c3d45700 1 --2- 172.21.15.162:0/3062424550 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f42bc0b2e80 0x7f42bc141a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:30.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.245+0000 7f42c1ae1700 1 -- 172.21.15.162:0/3062424550 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f42bc0acb40 msgr2=0x7f42bc130980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:30.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.245+0000 7f42c1ae1700 1 --2- 172.21.15.162:0/3062424550 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f42bc0acb40 0x7f42bc130980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:30.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.245+0000 7f42c1ae1700 1 -- 172.21.15.162:0/3062424550 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f42bc0b2e80 msgr2=0x7f42bc141a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:30.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.245+0000 7f42c1ae1700 1 --2- 172.21.15.162:0/3062424550 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f42bc0b2e80 0x7f42bc141a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:30.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.245+0000 7f42c1ae1700 1 -- 172.21.15.162:0/3062424550 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42bc066190 con 0x7f42bc0b2560 2020-11-28T21:52:30.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.245+0000 7f42c1ae1700 1 --2- 172.21.15.162:0/3062424550 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42bc0b2560 0x7f42bc099230 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7f42bc063710 tx=0x7f42ac007c20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:30.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.245+0000 7f42b2ffd700 1 -- 172.21.15.162:0/3062424550 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f42ac00e4a0 con 0x7f42bc0b2560 2020-11-28T21:52:30.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.245+0000 7f42b2ffd700 1 -- 172.21.15.162:0/3062424550 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42ac00e600 con 0x7f42bc0b2560 2020-11-28T21:52:30.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.245+0000 7f42c2ae3700 1 -- 172.21.15.162:0/3062424550 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42bc066430 con 0x7f42bc0b2560 2020-11-28T21:52:30.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.245+0000 7f42b2ffd700 1 -- 172.21.15.162:0/3062424550 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f42ac00e960 con 0x7f42bc0b2560 2020-11-28T21:52:30.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.245+0000 7f42c2ae3700 1 -- 172.21.15.162:0/3062424550 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42bc099720 con 0x7f42bc0b2560 2020-11-28T21:52:30.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.249+0000 7f42c2ae3700 1 -- 172.21.15.162:0/3062424550 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42bc0665c0 con 0x7f42bc0b2560 2020-11-28T21:52:30.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.249+0000 7f42b2ffd700 1 -- 172.21.15.162:0/3062424550 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f42ac010030 con 0x7f42bc0b2560 2020-11-28T21:52:30.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.249+0000 7f42b2ffd700 1 --2- 172.21.15.162:0/3062424550 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f42a40554d0 0x7f42a4057990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:30.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.249+0000 7f42b2ffd700 1 -- 172.21.15.162:0/3062424550 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f42ac06dd70 con 0x7f42bc0b2560 2020-11-28T21:52:30.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.249+0000 7f42c22e2700 1 --2- 172.21.15.162:0/3062424550 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f42a40554d0 0x7f42a4057990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:30.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.249+0000 7f42c22e2700 1 --2- 172.21.15.162:0/3062424550 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f42a40554d0 0x7f42a4057990 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f42b800b000 tx=0x7f42b8006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:30.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.253+0000 7f42b2ffd700 1 -- 172.21.15.162:0/3062424550 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f42ac044070 con 0x7f42bc0b2560 2020-11-28T21:52:30.449 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.449+0000 7f42c2ae3700 1 -- 172.21.15.162:0/3062424550 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f42bc057050 con 0x7f42bc0b2560 2020-11-28T21:52:30.458 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.457+0000 7f42b2ffd700 1 -- 172.21.15.162:0/3062424550 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v19) v1 ==== 184+0+0 (secure 0 0 0) 0x7f42ac047690 con 0x7f42bc0b2560 2020-11-28T21:52:30.460 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.461+0000 7f42c2ae3700 1 -- 172.21.15.162:0/3062424550 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f42a40554d0 msgr2=0x7f42a4057990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:30.460 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.461+0000 7f42c2ae3700 1 --2- 172.21.15.162:0/3062424550 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f42a40554d0 0x7f42a4057990 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f42b800b000 tx=0x7f42b8006dc0).stop 2020-11-28T21:52:30.460 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.461+0000 7f42c2ae3700 1 -- 172.21.15.162:0/3062424550 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42bc0b2560 msgr2=0x7f42bc099230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:30.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.461+0000 7f42c2ae3700 1 --2- 172.21.15.162:0/3062424550 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42bc0b2560 0x7f42bc099230 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7f42bc063710 tx=0x7f42ac007c20).stop 2020-11-28T21:52:30.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.461+0000 7f42c2ae3700 1 -- 172.21.15.162:0/3062424550 shutdown_connections 2020-11-28T21:52:30.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.461+0000 7f42c2ae3700 1 --2- 172.21.15.162:0/3062424550 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f42a40554d0 0x7f42a4057990 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:30.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.461+0000 7f42c2ae3700 1 --2- 172.21.15.162:0/3062424550 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f42bc0acb40 0x7f42bc130980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:30.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.461+0000 7f42c2ae3700 1 --2- 172.21.15.162:0/3062424550 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f42bc0b2e80 0x7f42bc141a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:30.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.461+0000 7f42c2ae3700 1 --2- 172.21.15.162:0/3062424550 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42bc0b2560 0x7f42bc099230 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:30.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.461+0000 7f42c2ae3700 1 -- 172.21.15.162:0/3062424550 >> 172.21.15.162:0/3062424550 conn(0x7f42bc063b60 msgr2=0x7f42bc133570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:30.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.461+0000 7f42c2ae3700 1 -- 172.21.15.162:0/3062424550 shutdown_connections 2020-11-28T21:52:30.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.461+0000 7f42c2ae3700 1 -- 172.21.15.162:0/3062424550 wait complete. 2020-11-28T21:52:30.463 INFO:teuthology.orchestra.run.smithi162.stderr:updated caps for client.0 2020-11-28T21:52:30.473 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-11-28T21:52:30.601 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.597+0000 7f84f74b0700 1 Processor -- start 2020-11-28T21:52:30.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.597+0000 7f84f74b0700 1 -- start start 2020-11-28T21:52:30.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.597+0000 7f84f74b0700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84f005fa90 0x7f84f015a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:30.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.597+0000 7f84f74b0700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84f00ab0b0 con 0x7f84f00b2e80 2020-11-28T21:52:30.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.597+0000 7f84f74b0700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84f00b3260 con 0x7f84f00b2560 2020-11-28T21:52:30.603 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.597+0000 7f84f74b0700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84f00ab2e0 con 0x7f84f005fa90 2020-11-28T21:52:30.603 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.597+0000 7f84f64ae700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f84f00b2560 0x7f84f0156ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:48248/0 (socket says 172.21.15.162:48248) 2020-11-28T21:52:30.603 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.597+0000 7f84f64ae700 1 -- 172.21.15.162:0/3671200893 learned_addr learned my addr 172.21.15.162:0/3671200893 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:52:30.603 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.597+0000 7f84f8712700 1 --2- 172.21.15.162:0/3671200893 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84f005fa90 0x7f84f015a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:30.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.597+0000 7f84f5cad700 1 -- 172.21.15.162:0/3671200893 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1146849334 0 0) 0x7f84f00b3260 con 0x7f84f00b2560 2020-11-28T21:52:30.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.597+0000 7f84f5cad700 1 -- 172.21.15.162:0/3671200893 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84dc003230 con 0x7f84f00b2560 2020-11-28T21:52:30.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.597+0000 7f84f5cad700 1 -- 172.21.15.162:0/3671200893 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3279488443 0 0) 0x7f84f00ab0b0 con 0x7f84f00b2e80 2020-11-28T21:52:30.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.597+0000 7f84f5cad700 1 -- 172.21.15.162:0/3671200893 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84f00b3260 con 0x7f84f00b2e80 2020-11-28T21:52:30.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.597+0000 7f84f5cad700 1 -- 172.21.15.162:0/3671200893 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3096334576 0 0) 0x7f84dc003230 con 0x7f84f00b2560 2020-11-28T21:52:30.605 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.597+0000 7f84f8712700 1 -- 172.21.15.162:0/3671200893 >> v1:172.21.15.162:6789/0 conn(0x7f84f00b2560 legacy=0x7f84f0156ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:30.605 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.597+0000 7f84f8712700 1 -- 172.21.15.162:0/3671200893 >> v1:172.21.15.118:6789/0 conn(0x7f84f00b2e80 legacy=0x7f84f005f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:30.605 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.597+0000 7f84f8712700 1 -- 172.21.15.162:0/3671200893 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f00256d0 con 0x7f84f005fa90 2020-11-28T21:52:30.605 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.601+0000 7f84f8712700 1 --2- 172.21.15.162:0/3671200893 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84f005fa90 0x7f84f015a5f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f84e00010d0 tx=0x7f84e80091a0).ready entity=mon.2 client_cookie=e126865c4be7a3ad server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:30.606 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.601+0000 7f84f5cad700 1 -- 172.21.15.162:0/3671200893 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84e00016d0 con 0x7f84f005fa90 2020-11-28T21:52:30.606 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.601+0000 7f84f5cad700 1 -- 172.21.15.162:0/3671200893 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e800b880 con 0x7f84f005fa90 2020-11-28T21:52:30.606 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.601+0000 7f84f5cad700 1 -- 172.21.15.162:0/3671200893 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84e800b9f0 con 0x7f84f005fa90 2020-11-28T21:52:30.606 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.601+0000 7f84f74b0700 1 -- 172.21.15.162:0/3671200893 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84f005fa90 msgr2=0x7f84f015a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:30.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.601+0000 7f84f74b0700 1 --2- 172.21.15.162:0/3671200893 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84f005fa90 0x7f84f015a5f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f84e00010d0 tx=0x7f84e80091a0).stop 2020-11-28T21:52:30.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.601+0000 7f84f74b0700 1 -- 172.21.15.162:0/3671200893 shutdown_connections 2020-11-28T21:52:30.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.601+0000 7f84f74b0700 1 --2- 172.21.15.162:0/3671200893 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84f005fa90 0x7f84f015a5f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:30.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.601+0000 7f84f74b0700 1 -- 172.21.15.162:0/3671200893 >> 172.21.15.162:0/3671200893 conn(0x7f84f0063b60 msgr2=0x7f84f0063f40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:30.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.601+0000 7f84f74b0700 1 -- 172.21.15.162:0/3671200893 shutdown_connections 2020-11-28T21:52:30.608 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.601+0000 7f84f74b0700 1 -- 172.21.15.162:0/3671200893 wait complete. 2020-11-28T21:52:30.611 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.601+0000 7f84f74b0700 1 Processor -- start 2020-11-28T21:52:30.611 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.601+0000 7f84f74b0700 1 -- start start 2020-11-28T21:52:30.612 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.605+0000 7f84f74b0700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84f005fa90 0x7f84f00ac150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:30.612 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.605+0000 7f84f74b0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84f00b2560 0x7f84f00b5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:30.612 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.605+0000 7f84f74b0700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f84f00b2e80 0x7f84f013ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:30.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.605+0000 7f84f74b0700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f013e4f0 con 0x7f84f00b2e80 2020-11-28T21:52:30.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.605+0000 7f84f74b0700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f00b50d0 con 0x7f84f00b2560 2020-11-28T21:52:30.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.605+0000 7f84f74b0700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84f00b5370 con 0x7f84f005fa90 2020-11-28T21:52:30.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.605+0000 7f84f64ae700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84f00b2560 0x7f84f00b5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:30.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.605+0000 7f84f64ae700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84f00b2560 0x7f84f00b5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41412/0 (socket says 172.21.15.162:41412) 2020-11-28T21:52:30.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.605+0000 7f84f64ae700 1 -- 172.21.15.162:0/662974267 learned_addr learned my addr 172.21.15.162:0/662974267 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:30.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.605+0000 7f84f8712700 1 --2- 172.21.15.162:0/662974267 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f84f00b2e80 0x7f84f013ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:30.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.605+0000 7f84f6caf700 1 --2- 172.21.15.162:0/662974267 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84f005fa90 0x7f84f00ac150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:30.615 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.605+0000 7f84f64ae700 1 -- 172.21.15.162:0/662974267 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84f005fa90 msgr2=0x7f84f00ac150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:30.615 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.605+0000 7f84f64ae700 1 --2- 172.21.15.162:0/662974267 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84f005fa90 0x7f84f00ac150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:30.615 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.605+0000 7f84f64ae700 1 -- 172.21.15.162:0/662974267 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f84f00b2e80 msgr2=0x7f84f013ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:30.615 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.605+0000 7f84f64ae700 1 --2- 172.21.15.162:0/662974267 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f84f00b2e80 0x7f84f013ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:30.615 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.605+0000 7f84f64ae700 1 -- 172.21.15.162:0/662974267 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f013e750 con 0x7f84f00b2560 2020-11-28T21:52:30.616 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.605+0000 7f84f6caf700 1 --2- 172.21.15.162:0/662974267 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84f005fa90 0x7f84f00ac150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:52:30.616 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.605+0000 7f84f64ae700 1 --2- 172.21.15.162:0/662974267 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84f00b2560 0x7f84f00b5610 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7f84e00018b0 tx=0x7f84e0001aa0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:30.616 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.605+0000 7f84e77fe700 1 -- 172.21.15.162:0/662974267 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84e000ce40 con 0x7f84f00b2560 2020-11-28T21:52:30.616 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.605+0000 7f84e77fe700 1 -- 172.21.15.162:0/662974267 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e0007170 con 0x7f84f00b2560 2020-11-28T21:52:30.616 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.605+0000 7f84f74b0700 1 -- 172.21.15.162:0/662974267 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84f00662b0 con 0x7f84f00b2560 2020-11-28T21:52:30.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.605+0000 7f84e77fe700 1 -- 172.21.15.162:0/662974267 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84e000a7b0 con 0x7f84f00b2560 2020-11-28T21:52:30.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.605+0000 7f84f74b0700 1 -- 172.21.15.162:0/662974267 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84f00acbc0 con 0x7f84f00b2560 2020-11-28T21:52:30.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.605+0000 7f84f74b0700 1 -- 172.21.15.162:0/662974267 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84f013e8e0 con 0x7f84f00b2560 2020-11-28T21:52:30.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.605+0000 7f84e77fe700 1 -- 172.21.15.162:0/662974267 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f84e0010070 con 0x7f84f00b2560 2020-11-28T21:52:30.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.609+0000 7f84e77fe700 1 --2- 172.21.15.162:0/662974267 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f84d8055320 0x7f84d80577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:30.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.609+0000 7f84e77fe700 1 -- 172.21.15.162:0/662974267 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f84e006da80 con 0x7f84f00b2560 2020-11-28T21:52:30.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.609+0000 7f84f6caf700 1 --2- 172.21.15.162:0/662974267 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f84d8055320 0x7f84d80577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:30.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.609+0000 7f84f6caf700 1 --2- 172.21.15.162:0/662974267 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f84d8055320 0x7f84d80577e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f84ec008000 tx=0x7f84ec009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:30.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.613+0000 7f84e77fe700 1 -- 172.21.15.162:0/662974267 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f84e0043ce0 con 0x7f84f00b2560 2020-11-28T21:52:30.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.805+0000 7f84f74b0700 1 -- 172.21.15.162:0/662974267 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f84f0057050 con 0x7f84f00b2560 2020-11-28T21:52:30.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.813+0000 7f84e77fe700 1 -- 172.21.15.162:0/662974267 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v20) v1 ==== 184+0+0 (secure 0 0 0) 0x7f84e0019030 con 0x7f84f00b2560 2020-11-28T21:52:30.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.817+0000 7f84f74b0700 1 -- 172.21.15.162:0/662974267 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f84d8055320 msgr2=0x7f84d80577e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:30.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.817+0000 7f84f74b0700 1 --2- 172.21.15.162:0/662974267 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f84d8055320 0x7f84d80577e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f84ec008000 tx=0x7f84ec009040).stop 2020-11-28T21:52:30.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.817+0000 7f84f74b0700 1 -- 172.21.15.162:0/662974267 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84f00b2560 msgr2=0x7f84f00b5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:30.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.817+0000 7f84f74b0700 1 --2- 172.21.15.162:0/662974267 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84f00b2560 0x7f84f00b5610 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7f84e00018b0 tx=0x7f84e0001aa0).stop 2020-11-28T21:52:30.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.817+0000 7f84f74b0700 1 -- 172.21.15.162:0/662974267 shutdown_connections 2020-11-28T21:52:30.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.817+0000 7f84f74b0700 1 --2- 172.21.15.162:0/662974267 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f84d8055320 0x7f84d80577e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:30.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.817+0000 7f84f74b0700 1 --2- 172.21.15.162:0/662974267 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84f005fa90 0x7f84f00ac150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:30.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.817+0000 7f84f74b0700 1 --2- 172.21.15.162:0/662974267 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f84f00b2e80 0x7f84f013ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:30.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.817+0000 7f84f74b0700 1 --2- 172.21.15.162:0/662974267 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84f00b2560 0x7f84f00b5610 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:30.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.817+0000 7f84f74b0700 1 -- 172.21.15.162:0/662974267 >> 172.21.15.162:0/662974267 conn(0x7f84f0063b60 msgr2=0x7f84f0133c90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:30.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.817+0000 7f84f74b0700 1 -- 172.21.15.162:0/662974267 shutdown_connections 2020-11-28T21:52:30.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.817+0000 7f84f74b0700 1 -- 172.21.15.162:0/662974267 wait complete. 2020-11-28T21:52:30.819 INFO:teuthology.orchestra.run.smithi162.stderr:updated caps for client.1 2020-11-28T21:52:30.835 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:52:30.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.949+0000 7f0011656700 1 Processor -- start 2020-11-28T21:52:30.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.949+0000 7f0011656700 1 -- start start 2020-11-28T21:52:30.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.949+0000 7f0011656700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f000c0b61b0 0x7f000c13fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:30.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.949+0000 7f0011656700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f000c0b5520 0x7f000c13ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:30.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.949+0000 7f0011656700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f000c145e70 con 0x7f000c060900 2020-11-28T21:52:30.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.949+0000 7f0011656700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f000c060ce0 con 0x7f000c0b5520 2020-11-28T21:52:30.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.949+0000 7f0011656700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f000c1460a0 con 0x7f000c0b61b0 2020-11-28T21:52:30.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.949+0000 7f000b7fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f000c0b5520 0x7f000c13ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:30.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.949+0000 7f000b7fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f000c0b5520 0x7f000c13ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41422/0 (socket says 172.21.15.162:41422) 2020-11-28T21:52:30.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.949+0000 7f000b7fe700 1 -- 172.21.15.162:0/1317578033 learned_addr learned my addr 172.21.15.162:0/1317578033 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:30.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.949+0000 7f0003fff700 1 --2- 172.21.15.162:0/1317578033 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f000c0b61b0 0x7f000c13fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:30.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.949+0000 7f000b7fe700 1 -- 172.21.15.162:0/1317578033 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f000c0b61b0 msgr2=0x7f000c13fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:30.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.949+0000 7f000b7fe700 1 --2- 172.21.15.162:0/1317578033 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f000c0b61b0 0x7f000c13fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:30.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.949+0000 7f000b7fe700 1 -- 172.21.15.162:0/1317578033 >> v1:172.21.15.118:6789/0 conn(0x7f000c060900 legacy=0x7f000c1428a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:30.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.949+0000 7f000b7fe700 1 -- 172.21.15.162:0/1317578033 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000c146340 con 0x7f000c0b5520 2020-11-28T21:52:30.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.949+0000 7f000b7fe700 1 --2- 172.21.15.162:0/1317578033 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f000c0b5520 0x7f000c13ffb0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7f00040032b0 tx=0x7f0004009270).ready entity=mon.1 client_cookie=1e7ec0bf2d4616ca server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:30.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.953+0000 7f000a7fc700 1 -- 172.21.15.162:0/1317578033 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0004009e60 con 0x7f000c0b5520 2020-11-28T21:52:30.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.953+0000 7f000a7fc700 1 -- 172.21.15.162:0/1317578033 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f000400bc00 con 0x7f000c0b5520 2020-11-28T21:52:30.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.953+0000 7f000a7fc700 1 -- 172.21.15.162:0/1317578033 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f000400bd70 con 0x7f000c0b5520 2020-11-28T21:52:30.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.953+0000 7f0011656700 1 -- 172.21.15.162:0/1317578033 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f000c0b5520 msgr2=0x7f000c13ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:30.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.953+0000 7f0011656700 1 --2- 172.21.15.162:0/1317578033 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f000c0b5520 0x7f000c13ffb0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7f00040032b0 tx=0x7f0004009270).stop 2020-11-28T21:52:30.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.953+0000 7f0011656700 1 -- 172.21.15.162:0/1317578033 shutdown_connections 2020-11-28T21:52:30.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.953+0000 7f0011656700 1 --2- 172.21.15.162:0/1317578033 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f000c0b5520 0x7f000c13ffb0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:30.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.953+0000 7f0011656700 1 --2- 172.21.15.162:0/1317578033 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f000c0b61b0 0x7f000c13fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:30.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.953+0000 7f0011656700 1 -- 172.21.15.162:0/1317578033 >> 172.21.15.162:0/1317578033 conn(0x7f000c066af0 msgr2=0x7f000c066ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:30.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.953+0000 7f0011656700 1 -- 172.21.15.162:0/1317578033 shutdown_connections 2020-11-28T21:52:30.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.953+0000 7f0011656700 1 -- 172.21.15.162:0/1317578033 wait complete. 2020-11-28T21:52:30.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.953+0000 7f0011656700 1 Processor -- start 2020-11-28T21:52:30.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.953+0000 7f0011656700 1 -- start start 2020-11-28T21:52:30.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.953+0000 7f0011656700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f000c060900 0x7f000c143040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:30.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.953+0000 7f0011656700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f000c0b5520 0x7f000c144e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:30.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.953+0000 7f0011656700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f000c0b61b0 0x7f000c0adab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:30.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.953+0000 7f0011656700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000c1524b0 con 0x7f000c0b61b0 2020-11-28T21:52:30.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.953+0000 7f0011656700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000c0aedc0 con 0x7f000c0b5520 2020-11-28T21:52:30.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.953+0000 7f0011656700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f000c0ad0d0 con 0x7f000c060900 2020-11-28T21:52:30.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.953+0000 7f0003fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f000c0b5520 0x7f000c144e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:30.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.953+0000 7f0003fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f000c0b5520 0x7f000c144e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41426/0 (socket says 172.21.15.162:41426) 2020-11-28T21:52:30.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.953+0000 7f0003fff700 1 -- 172.21.15.162:0/1258446679 learned_addr learned my addr 172.21.15.162:0/1258446679 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:30.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.953+0000 7f000affd700 1 --2- 172.21.15.162:0/1258446679 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f000c060900 0x7f000c143040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:30.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.953+0000 7f000b7fe700 1 --2- 172.21.15.162:0/1258446679 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f000c0b61b0 0x7f000c0adab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:30.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.957+0000 7f0003fff700 1 -- 172.21.15.162:0/1258446679 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f000c060900 msgr2=0x7f000c143040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:30.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.957+0000 7f0003fff700 1 --2- 172.21.15.162:0/1258446679 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f000c060900 0x7f000c143040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:30.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.957+0000 7f0003fff700 1 -- 172.21.15.162:0/1258446679 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f000c0b61b0 msgr2=0x7f000c0adab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:30.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.957+0000 7f0003fff700 1 --2- 172.21.15.162:0/1258446679 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f000c0b61b0 0x7f000c0adab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:30.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.957+0000 7f0003fff700 1 -- 172.21.15.162:0/1258446679 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000c0ad370 con 0x7f000c0b5520 2020-11-28T21:52:30.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.957+0000 7f0003fff700 1 --2- 172.21.15.162:0/1258446679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f000c0b5520 0x7f000c144e10 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7efff400c000 tx=0x7efff4006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:30.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.957+0000 7f0008ff9700 1 -- 172.21.15.162:0/1258446679 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efff400ca10 con 0x7f000c0b5520 2020-11-28T21:52:30.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.957+0000 7f0011656700 1 -- 172.21.15.162:0/1258446679 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f000c0ad610 con 0x7f000c0b5520 2020-11-28T21:52:30.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.957+0000 7f0008ff9700 1 -- 172.21.15.162:0/1258446679 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff400cb70 con 0x7f000c0b5520 2020-11-28T21:52:30.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.957+0000 7f0008ff9700 1 -- 172.21.15.162:0/1258446679 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efff400be50 con 0x7f000c0b5520 2020-11-28T21:52:30.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.957+0000 7f0011656700 1 -- 172.21.15.162:0/1258446679 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f000c151fe0 con 0x7f000c0b5520 2020-11-28T21:52:30.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.957+0000 7f0011656700 1 -- 172.21.15.162:0/1258446679 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f000c057050 con 0x7f000c0b5520 2020-11-28T21:52:30.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.957+0000 7f0008ff9700 1 -- 172.21.15.162:0/1258446679 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7efff401b400 con 0x7f000c0b5520 2020-11-28T21:52:30.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.957+0000 7f0008ff9700 1 --2- 172.21.15.162:0/1258446679 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7effe4055440 0x7effe4057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:30.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.957+0000 7f0008ff9700 1 -- 172.21.15.162:0/1258446679 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6073+0+0 (secure 0 0 0) 0x7efff406e1b0 con 0x7f000c0b5520 2020-11-28T21:52:30.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.957+0000 7f000affd700 1 --2- 172.21.15.162:0/1258446679 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7effe4055440 0x7effe4057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:30.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.961+0000 7f000affd700 1 --2- 172.21.15.162:0/1258446679 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7effe4055440 0x7effe4057900 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7efffc00d000 tx=0x7efffc007b60).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:30.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:30.969+0000 7f0008ff9700 1 -- 172.21.15.162:0/1258446679 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7efff40442c0 con 0x7f000c0b5520 2020-11-28T21:52:31.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.157+0000 7f0011656700 1 -- 172.21.15.162:0/1258446679 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f000c069bd0 con 0x7f000c0b5520 2020-11-28T21:52:31.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.157+0000 7f0008ff9700 1 -- 172.21.15.162:0/1258446679 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+2499 (secure 0 0 0) 0x7efff40478e0 con 0x7f000c0b5520 2020-11-28T21:52:31.158 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:52:31.159 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":33,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5540,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2116325247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2116325247},{"type":"v1","addr":"172.21.15.162:6835","nonce":2116325247}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":30},{"gid":5543,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.118:6833/3491069258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3491069258},{"type":"v1","addr":"172.21.15.118:6833","nonce":3491069258}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:52:28.454745+0000","modified":"2020-11-28T21:52:29.457610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6172},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6172":{"gid":6172,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/163256983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":163256983},{"type":"v1","addr":"172.21.15.162:6837","nonce":163256983}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-28T21:52:31.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.161+0000 7f0011656700 1 -- 172.21.15.162:0/1258446679 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7effe4055440 msgr2=0x7effe4057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:31.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.161+0000 7f0011656700 1 --2- 172.21.15.162:0/1258446679 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7effe4055440 0x7effe4057900 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7efffc00d000 tx=0x7efffc007b60).stop 2020-11-28T21:52:31.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.161+0000 7f0011656700 1 -- 172.21.15.162:0/1258446679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f000c0b5520 msgr2=0x7f000c144e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:31.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.161+0000 7f0011656700 1 --2- 172.21.15.162:0/1258446679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f000c0b5520 0x7f000c144e10 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7efff400c000 tx=0x7efff4006dc0).stop 2020-11-28T21:52:31.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.161+0000 7f0011656700 1 -- 172.21.15.162:0/1258446679 shutdown_connections 2020-11-28T21:52:31.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.161+0000 7f0011656700 1 --2- 172.21.15.162:0/1258446679 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7effe4055440 0x7effe4057900 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:31.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.161+0000 7f0011656700 1 --2- 172.21.15.162:0/1258446679 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f000c060900 0x7f000c143040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:31.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.161+0000 7f0011656700 1 --2- 172.21.15.162:0/1258446679 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f000c0b61b0 0x7f000c0adab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:31.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.161+0000 7f0011656700 1 --2- 172.21.15.162:0/1258446679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f000c0b5520 0x7f000c144e10 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:31.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.161+0000 7f0011656700 1 -- 172.21.15.162:0/1258446679 >> 172.21.15.162:0/1258446679 conn(0x7f000c066af0 msgr2=0x7f000c0b8260 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:31.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.161+0000 7f0011656700 1 -- 172.21.15.162:0/1258446679 shutdown_connections 2020-11-28T21:52:31.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.161+0000 7f0011656700 1 -- 172.21.15.162:0/1258446679 wait complete. 2020-11-28T21:52:31.163 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 33 2020-11-28T21:52:31.475 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-28T21:52:28.454745+0000', 'modified': '2020-11-28T21:52:29.457610+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6172}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6172': {'gid': 6172, 'name': 'a', 'rank': 0, 'incarnation': 32, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6837/163256983', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6836', 'nonce': 163256983}, {'type': 'v1', 'addr': '172.21.15.162:6837', 'nonce': 163256983}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-28T21:52:31.476 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-28T21:52:31.476 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-28T21:52:31.606 INFO:teuthology.orchestra.run.smithi162.stdout:{"cluster_fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","whoami":0,"id":6172,"want_state":"up:active","state":"up:active","rank_uptime":3.1385290349999999,"mdsmap_epoch":33,"osdmap_epoch":76,"osdmap_epoch_barrier":75,"uptime":8.5660212520000005} 2020-11-28T21:52:31.606 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", "id": 6172, "mdsmap_epoch": 33, "osdmap_epoch": 76, "osdmap_epoch_barrier": 75, "rank_uptime": 3.138529035, "state": "up:active", "uptime": 8.566021252, "want_state": "up:active", "whoami": 0 } 2020-11-28T21:52:31.607 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:52:31.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.717+0000 7f1f13fff700 1 Processor -- start 2020-11-28T21:52:31.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.717+0000 7f1f13fff700 1 -- start start 2020-11-28T21:52:31.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.717+0000 7f1f13fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1f140603d0 0x7f1f140b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:31.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.717+0000 7f1f13fff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1f140b4c70 0x7f1f14143020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:31.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.717+0000 7f1f13fff700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f140b4110 con 0x7f1f14142a00 2020-11-28T21:52:31.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.717+0000 7f1f13fff700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f14060f60 con 0x7f1f140b4c70 2020-11-28T21:52:31.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.717+0000 7f1f13fff700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f140b43f0 con 0x7f1f140603d0 2020-11-28T21:52:31.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.717+0000 7f1f12ffd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1f140603d0 0x7f1f140b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:31.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.717+0000 7f1f127fc700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f1f14142a00 0x7f1f1409cfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:48278/0 (socket says 172.21.15.162:48278) 2020-11-28T21:52:31.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.717+0000 7f1f127fc700 1 -- 172.21.15.162:0/4021179423 learned_addr learned my addr 172.21.15.162:0/4021179423 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:52:31.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.717+0000 7f1f12ffd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1f140603d0 0x7f1f140b3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41432/0 (socket says 172.21.15.162:41432) 2020-11-28T21:52:31.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.717+0000 7f1f137fe700 1 --2- 172.21.15.162:0/4021179423 >> v2:172.21.15.118:3300/0 conn(0x7f1f140b4c70 0x7f1f14143020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:31.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.717+0000 7f1f12ffd700 1 -- 172.21.15.162:0/4021179423 >> v2:172.21.15.118:3300/0 conn(0x7f1f140b4c70 msgr2=0x7f1f14143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:31.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.717+0000 7f1f12ffd700 1 --2- 172.21.15.162:0/4021179423 >> v2:172.21.15.118:3300/0 conn(0x7f1f140b4c70 0x7f1f14143020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:31.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.717+0000 7f1f12ffd700 1 -- 172.21.15.162:0/4021179423 >> v1:172.21.15.162:6789/0 conn(0x7f1f14142a00 legacy=0x7f1f1409cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:31.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.717+0000 7f1f12ffd700 1 -- 172.21.15.162:0/4021179423 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f140b4690 con 0x7f1f140603d0 2020-11-28T21:52:31.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.717+0000 7f1f12ffd700 1 --2- 172.21.15.162:0/4021179423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f140603d0 0x7f1f140b3a70 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7f1f0c002f30 tx=0x7f1f0c006dc0).ready entity=mon.1 client_cookie=347618a818c1d1ac server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:31.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.717+0000 7f1f11ffb700 1 -- 172.21.15.162:0/4021179423 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f0c0097f0 con 0x7f1f140603d0 2020-11-28T21:52:31.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.717+0000 7f1f11ffb700 1 -- 172.21.15.162:0/4021179423 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f0c009950 con 0x7f1f140603d0 2020-11-28T21:52:31.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.717+0000 7f1f11ffb700 1 -- 172.21.15.162:0/4021179423 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f0c009cb0 con 0x7f1f140603d0 2020-11-28T21:52:31.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.717+0000 7f1f13fff700 1 -- 172.21.15.162:0/4021179423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f140603d0 msgr2=0x7f1f140b3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:31.723 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.717+0000 7f1f13fff700 1 --2- 172.21.15.162:0/4021179423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f140603d0 0x7f1f140b3a70 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7f1f0c002f30 tx=0x7f1f0c006dc0).stop 2020-11-28T21:52:31.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.717+0000 7f1f13fff700 1 -- 172.21.15.162:0/4021179423 shutdown_connections 2020-11-28T21:52:31.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.717+0000 7f1f13fff700 1 --2- 172.21.15.162:0/4021179423 >> v2:172.21.15.118:3300/0 conn(0x7f1f140b4c70 0x7f1f14143020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:31.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.717+0000 7f1f13fff700 1 --2- 172.21.15.162:0/4021179423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f140603d0 0x7f1f140b3a70 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:31.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.717+0000 7f1f13fff700 1 -- 172.21.15.162:0/4021179423 >> 172.21.15.162:0/4021179423 conn(0x7f1f14066a40 msgr2=0x7f1f14066e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:31.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.721+0000 7f1f13fff700 1 -- 172.21.15.162:0/4021179423 shutdown_connections 2020-11-28T21:52:31.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.721+0000 7f1f13fff700 1 -- 172.21.15.162:0/4021179423 wait complete. 2020-11-28T21:52:31.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.721+0000 7f1f13fff700 1 Processor -- start 2020-11-28T21:52:31.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.721+0000 7f1f13fff700 1 -- start start 2020-11-28T21:52:31.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.721+0000 7f1f13fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1f140603d0 0x7f1f141521a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:31.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.721+0000 7f1f13fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1f140b4c70 0x7f1f14146250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:31.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.721+0000 7f1f13fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f14146740 0x7f1f140b74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:31.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.721+0000 7f1f13fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f14152690 con 0x7f1f140603d0 2020-11-28T21:52:31.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.721+0000 7f1f13fff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f140b6d70 con 0x7f1f14146740 2020-11-28T21:52:31.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.721+0000 7f1f13fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f140b8910 con 0x7f1f140b4c70 2020-11-28T21:52:31.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.721+0000 7f1f137fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f14146740 0x7f1f140b74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:31.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.721+0000 7f1f137fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f14146740 0x7f1f140b74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41442/0 (socket says 172.21.15.162:41442) 2020-11-28T21:52:31.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.721+0000 7f1f137fe700 1 -- 172.21.15.162:0/495457881 learned_addr learned my addr 172.21.15.162:0/495457881 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:31.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.721+0000 7f1f12ffd700 1 --2- 172.21.15.162:0/495457881 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1f140603d0 0x7f1f141521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:31.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.721+0000 7f1f127fc700 1 --2- 172.21.15.162:0/495457881 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1f140b4c70 0x7f1f14146250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:31.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.721+0000 7f1f137fe700 1 -- 172.21.15.162:0/495457881 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1f140b4c70 msgr2=0x7f1f14146250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:31.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.721+0000 7f1f137fe700 1 --2- 172.21.15.162:0/495457881 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1f140b4c70 0x7f1f14146250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:31.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.721+0000 7f1f137fe700 1 -- 172.21.15.162:0/495457881 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1f140603d0 msgr2=0x7f1f141521a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:31.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.721+0000 7f1f137fe700 1 --2- 172.21.15.162:0/495457881 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1f140603d0 0x7f1f141521a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:31.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.721+0000 7f1f137fe700 1 -- 172.21.15.162:0/495457881 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f140b7240 con 0x7f1f14146740 2020-11-28T21:52:31.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.721+0000 7f1f127fc700 1 --2- 172.21.15.162:0/495457881 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1f140b4c70 0x7f1f14146250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:52:31.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.721+0000 7f1f137fe700 1 --2- 172.21.15.162:0/495457881 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f14146740 0x7f1f140b74e0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f1f14144e90 tx=0x7f1f04008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:31.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.721+0000 7f1f0bfff700 1 -- 172.21.15.162:0/495457881 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f0400bad0 con 0x7f1f14146740 2020-11-28T21:52:31.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.721+0000 7f1f0bfff700 1 -- 172.21.15.162:0/495457881 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f0400bc30 con 0x7f1f14146740 2020-11-28T21:52:31.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.721+0000 7f1f0bfff700 1 -- 172.21.15.162:0/495457881 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f04012420 con 0x7f1f14146740 2020-11-28T21:52:31.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.721+0000 7f1f13fff700 1 -- 172.21.15.162:0/495457881 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f140b6fa0 con 0x7f1f14146740 2020-11-28T21:52:31.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.721+0000 7f1f13fff700 1 -- 172.21.15.162:0/495457881 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f14146bf0 con 0x7f1f14146740 2020-11-28T21:52:31.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.725+0000 7f1f13fff700 1 -- 172.21.15.162:0/495457881 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f14057050 con 0x7f1f14146740 2020-11-28T21:52:31.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.725+0000 7f1f0bfff700 1 -- 172.21.15.162:0/495457881 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1f04012580 con 0x7f1f14146740 2020-11-28T21:52:31.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.725+0000 7f1f0bfff700 1 --2- 172.21.15.162:0/495457881 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1ef80554e0 0x7f1ef80579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:31.733 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.725+0000 7f1f0bfff700 1 -- 172.21.15.162:0/495457881 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1f0406e6d0 con 0x7f1f14146740 2020-11-28T21:52:31.733 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.725+0000 7f1f12ffd700 1 --2- 172.21.15.162:0/495457881 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1ef80554e0 0x7f1ef80579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:31.733 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.725+0000 7f1f12ffd700 1 --2- 172.21.15.162:0/495457881 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1ef80554e0 0x7f1ef80579a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f1f0c019000 tx=0x7f1f0c015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:31.734 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.733+0000 7f1f0bfff700 1 -- 172.21.15.162:0/495457881 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1f04044970 con 0x7f1f14146740 2020-11-28T21:52:31.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.873+0000 7f1f13fff700 1 -- 172.21.15.162:0/495457881 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f14069b60 con 0x7f1f14146740 2020-11-28T21:52:31.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.873+0000 7f1f0bfff700 1 -- 172.21.15.162:0/495457881 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+16137 (secure 0 0 0) 0x7f1f04047f90 con 0x7f1f14146740 2020-11-28T21:52:31.874 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:52:31.875 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":76,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:52:29.458247+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:52:25.955831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2020-11-28T21:52:27.265702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:52:31.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.877+0000 7f1f13fff700 1 -- 172.21.15.162:0/495457881 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1ef80554e0 msgr2=0x7f1ef80579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:31.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.877+0000 7f1f13fff700 1 --2- 172.21.15.162:0/495457881 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1ef80554e0 0x7f1ef80579a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f1f0c019000 tx=0x7f1f0c015000).stop 2020-11-28T21:52:31.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.877+0000 7f1f13fff700 1 -- 172.21.15.162:0/495457881 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f14146740 msgr2=0x7f1f140b74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:31.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.877+0000 7f1f13fff700 1 --2- 172.21.15.162:0/495457881 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f14146740 0x7f1f140b74e0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f1f14144e90 tx=0x7f1f04008040).stop 2020-11-28T21:52:31.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.877+0000 7f1f13fff700 1 -- 172.21.15.162:0/495457881 shutdown_connections 2020-11-28T21:52:31.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.877+0000 7f1f13fff700 1 --2- 172.21.15.162:0/495457881 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1ef80554e0 0x7f1ef80579a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:31.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.877+0000 7f1f13fff700 1 --2- 172.21.15.162:0/495457881 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f14146740 0x7f1f140b74e0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:31.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.877+0000 7f1f13fff700 1 --2- 172.21.15.162:0/495457881 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1f140603d0 0x7f1f141521a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:31.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.877+0000 7f1f13fff700 1 --2- 172.21.15.162:0/495457881 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1f140b4c70 0x7f1f14146250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:31.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.877+0000 7f1f13fff700 1 -- 172.21.15.162:0/495457881 >> 172.21.15.162:0/495457881 conn(0x7f1f14066a40 msgr2=0x7f1f1409cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:31.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.877+0000 7f1f13fff700 1 -- 172.21.15.162:0/495457881 shutdown_connections 2020-11-28T21:52:31.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:31.877+0000 7f1f13fff700 1 -- 172.21.15.162:0/495457881 wait complete. 2020-11-28T21:52:31.892 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-28T21:52:25.955831+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2020-11-28T21:52:27.265702+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-28T21:52:31.893 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:52:32.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.009+0000 7f5810af3700 1 Processor -- start 2020-11-28T21:52:32.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.009+0000 7f5810af3700 1 -- start start 2020-11-28T21:52:32.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.009+0000 7f5810af3700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f580c0b5730 0x7f580c14ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:32.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.009+0000 7f5810af3700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f580c14b140 0x7f580c147660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:32.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.009+0000 7f5810af3700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f580c0b1120 0x7f580c0af820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:32.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.009+0000 7f5810af3700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f580c0a8690 con 0x7f580c0b1120 2020-11-28T21:52:32.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.009+0000 7f5810af3700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f580c0a88c0 con 0x7f580c0b5730 2020-11-28T21:52:32.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.009+0000 7f5810af3700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f580c1471c0 con 0x7f580c14b140 2020-11-28T21:52:32.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.009+0000 7f580a59c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f580c0b5730 0x7f580c14ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:32.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.009+0000 7f580ad9d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f580c0b1120 0x7f580c0af820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:32.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.009+0000 7f580a59c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f580c0b5730 0x7f580c14ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41448/0 (socket says 172.21.15.162:41448) 2020-11-28T21:52:32.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.009+0000 7f580a59c700 1 -- 172.21.15.162:0/3910875747 learned_addr learned my addr 172.21.15.162:0/3910875747 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:32.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.009+0000 7f5801d9b700 1 --2- 172.21.15.162:0/3910875747 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f580c14b140 0x7f580c147660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:32.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.009+0000 7f580a59c700 1 -- 172.21.15.162:0/3910875747 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f580c14b140 msgr2=0x7f580c147660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:32.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.009+0000 7f580a59c700 1 --2- 172.21.15.162:0/3910875747 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f580c14b140 0x7f580c147660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:32.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.009+0000 7f580a59c700 1 -- 172.21.15.162:0/3910875747 >> v2:172.21.15.118:3300/0 conn(0x7f580c0b1120 msgr2=0x7f580c0af820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:32.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.009+0000 7f580a59c700 1 --2- 172.21.15.162:0/3910875747 >> v2:172.21.15.118:3300/0 conn(0x7f580c0b1120 0x7f580c0af820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:32.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.009+0000 7f580a59c700 1 -- 172.21.15.162:0/3910875747 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f580c146ef0 con 0x7f580c0b5730 2020-11-28T21:52:32.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.009+0000 7f580a59c700 1 --2- 172.21.15.162:0/3910875747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f580c0b5730 0x7f580c14ac50 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7f57fc002f30 tx=0x7f57fc006e80).ready entity=mon.1 client_cookie=c3af9de66e93afb2 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:32.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.009+0000 7f5809d9b700 1 -- 172.21.15.162:0/3910875747 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57fc009640 con 0x7f580c0b5730 2020-11-28T21:52:32.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.009+0000 7f5809d9b700 1 -- 172.21.15.162:0/3910875747 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57fc0097a0 con 0x7f580c0b5730 2020-11-28T21:52:32.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.009+0000 7f5810af3700 1 -- 172.21.15.162:0/3910875747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f580c0b5730 msgr2=0x7f580c14ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:32.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.009+0000 7f5810af3700 1 --2- 172.21.15.162:0/3910875747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f580c0b5730 0x7f580c14ac50 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7f57fc002f30 tx=0x7f57fc006e80).stop 2020-11-28T21:52:32.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.009+0000 7f5809d9b700 1 -- 172.21.15.162:0/3910875747 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57fc009b00 con 0x7f580c0b5730 2020-11-28T21:52:32.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.013+0000 7f5810af3700 1 -- 172.21.15.162:0/3910875747 shutdown_connections 2020-11-28T21:52:32.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.013+0000 7f5810af3700 1 --2- 172.21.15.162:0/3910875747 >> v2:172.21.15.118:3300/0 conn(0x7f580c0b1120 0x7f580c0af820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:32.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.013+0000 7f5810af3700 1 --2- 172.21.15.162:0/3910875747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f580c0b5730 0x7f580c14ac50 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:32.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.013+0000 7f5810af3700 1 --2- 172.21.15.162:0/3910875747 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f580c14b140 0x7f580c147660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:32.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.013+0000 7f5810af3700 1 -- 172.21.15.162:0/3910875747 >> 172.21.15.162:0/3910875747 conn(0x7f580c05c3f0 msgr2=0x7f580c0b4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:32.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.013+0000 7f5810af3700 1 -- 172.21.15.162:0/3910875747 shutdown_connections 2020-11-28T21:52:32.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.013+0000 7f5810af3700 1 -- 172.21.15.162:0/3910875747 wait complete. 2020-11-28T21:52:32.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.013+0000 7f5810af3700 1 Processor -- start 2020-11-28T21:52:32.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.013+0000 7f5810af3700 1 -- start start 2020-11-28T21:52:32.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.013+0000 7f5810af3700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f580c0b1120 0x7f580c0b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:32.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.013+0000 7f5810af3700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f580c0b5730 0x7f580c0ab380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:32.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.013+0000 7f5810af3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f580c14b140 0x7f580c0ab870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:32.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.013+0000 7f5810af3700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580c0abd60 con 0x7f580c0b1120 2020-11-28T21:52:32.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.013+0000 7f5810af3700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580c0adc40 con 0x7f580c14b140 2020-11-28T21:52:32.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.013+0000 7f5810af3700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f580c0adee0 con 0x7f580c0b5730 2020-11-28T21:52:32.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.013+0000 7f580ad9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f580c14b140 0x7f580c0ab870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:32.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.013+0000 7f580ad9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f580c14b140 0x7f580c0ab870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41456/0 (socket says 172.21.15.162:41456) 2020-11-28T21:52:32.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.013+0000 7f580ad9d700 1 -- 172.21.15.162:0/3837534166 learned_addr learned my addr 172.21.15.162:0/3837534166 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:32.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.013+0000 7f580a59c700 1 --2- 172.21.15.162:0/3837534166 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f580c0b1120 0x7f580c0b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:32.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.013+0000 7f5801d9b700 1 --2- 172.21.15.162:0/3837534166 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f580c0b5730 0x7f580c0ab380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:32.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.013+0000 7f580ad9d700 1 -- 172.21.15.162:0/3837534166 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f580c0b5730 msgr2=0x7f580c0ab380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:32.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.013+0000 7f580ad9d700 1 --2- 172.21.15.162:0/3837534166 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f580c0b5730 0x7f580c0ab380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:32.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.013+0000 7f580ad9d700 1 -- 172.21.15.162:0/3837534166 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f580c0b1120 msgr2=0x7f580c0b4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:32.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.013+0000 7f580ad9d700 1 --2- 172.21.15.162:0/3837534166 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f580c0b1120 0x7f580c0b4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:32.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.013+0000 7f580ad9d700 1 -- 172.21.15.162:0/3837534166 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f580c0ae180 con 0x7f580c14b140 2020-11-28T21:52:32.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.013+0000 7f580ad9d700 1 --2- 172.21.15.162:0/3837534166 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f580c14b140 0x7f580c0ab870 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 rx=0x7f580c156000 tx=0x7f58040091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:32.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.013+0000 7f5803fff700 1 -- 172.21.15.162:0/3837534166 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f580400cbb0 con 0x7f580c14b140 2020-11-28T21:52:32.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.013+0000 7f5803fff700 1 -- 172.21.15.162:0/3837534166 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580400cd10 con 0x7f580c14b140 2020-11-28T21:52:32.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.013+0000 7f5810af3700 1 -- 172.21.15.162:0/3837534166 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f580c0ad900 con 0x7f580c14b140 2020-11-28T21:52:32.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.013+0000 7f5803fff700 1 -- 172.21.15.162:0/3837534166 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58040125e0 con 0x7f580c14b140 2020-11-28T21:52:32.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.013+0000 7f5810af3700 1 -- 172.21.15.162:0/3837534166 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f580c0a98b0 con 0x7f580c14b140 2020-11-28T21:52:32.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.017+0000 7f5810af3700 1 -- 172.21.15.162:0/3837534166 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f580c0ae310 con 0x7f580c14b140 2020-11-28T21:52:32.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.017+0000 7f5803fff700 1 -- 172.21.15.162:0/3837534166 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5804022070 con 0x7f580c14b140 2020-11-28T21:52:32.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.017+0000 7f5803fff700 1 --2- 172.21.15.162:0/3837534166 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f57f0055440 0x7f57f0057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:32.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.017+0000 7f5803fff700 1 -- 172.21.15.162:0/3837534166 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f580406e730 con 0x7f580c14b140 2020-11-28T21:52:32.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.017+0000 7f580a59c700 1 --2- 172.21.15.162:0/3837534166 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f57f0055440 0x7f57f0057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:32.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.017+0000 7f580a59c700 1 --2- 172.21.15.162:0/3837534166 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f57f0055440 0x7f57f0057900 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f57fc019000 tx=0x7f57fc015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:32.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.021+0000 7f5803fff700 1 -- 172.21.15.162:0/3837534166 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5804044a30 con 0x7f580c14b140 2020-11-28T21:52:32.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.160+0000 7f5810af3700 1 -- 172.21.15.162:0/3837534166 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f580c057050 con 0x7f580c14b140 2020-11-28T21:52:32.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.160+0000 7f5803fff700 1 -- 172.21.15.162:0/3837534166 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v76) v1 ==== 110+0+10 (secure 0 0 0) 0x7f5804016020 con 0x7f580c14b140 2020-11-28T21:52:32.161 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:52:32.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.160+0000 7f5810af3700 1 -- 172.21.15.162:0/3837534166 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f57f0055440 msgr2=0x7f57f0057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:32.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.160+0000 7f5810af3700 1 --2- 172.21.15.162:0/3837534166 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f57f0055440 0x7f57f0057900 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f57fc019000 tx=0x7f57fc015000).stop 2020-11-28T21:52:32.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.160+0000 7f5810af3700 1 -- 172.21.15.162:0/3837534166 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f580c14b140 msgr2=0x7f580c0ab870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:32.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.160+0000 7f5810af3700 1 --2- 172.21.15.162:0/3837534166 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f580c14b140 0x7f580c0ab870 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 rx=0x7f580c156000 tx=0x7f58040091a0).stop 2020-11-28T21:52:32.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.160+0000 7f5810af3700 1 -- 172.21.15.162:0/3837534166 shutdown_connections 2020-11-28T21:52:32.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.160+0000 7f5810af3700 1 --2- 172.21.15.162:0/3837534166 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f57f0055440 0x7f57f0057900 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:32.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.160+0000 7f5810af3700 1 --2- 172.21.15.162:0/3837534166 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f580c14b140 0x7f580c0ab870 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:32.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.160+0000 7f5810af3700 1 --2- 172.21.15.162:0/3837534166 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f580c0b1120 0x7f580c0b4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:32.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.160+0000 7f5810af3700 1 --2- 172.21.15.162:0/3837534166 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f580c0b5730 0x7f580c0ab380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:32.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.160+0000 7f5810af3700 1 -- 172.21.15.162:0/3837534166 >> 172.21.15.162:0/3837534166 conn(0x7f580c05c3f0 msgr2=0x7f580c0608a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:32.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.160+0000 7f5810af3700 1 -- 172.21.15.162:0/3837534166 shutdown_connections 2020-11-28T21:52:32.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.160+0000 7f5810af3700 1 -- 172.21.15.162:0/3837534166 wait complete. 2020-11-28T21:52:32.174 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:52:32.293 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.292+0000 7faaa7437700 1 Processor -- start 2020-11-28T21:52:32.294 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.292+0000 7faaa7437700 1 -- start start 2020-11-28T21:52:32.295 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.292+0000 7faaa7437700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7faaa00b9ce0 0x7faaa01433c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:32.295 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.292+0000 7faaa7437700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faaa00605c0 0x7faaa0067610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:32.295 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.292+0000 7faaa7437700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaa0143950 con 0x7faaa00b3c30 2020-11-28T21:52:32.296 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.292+0000 7faaa7437700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaa00ba0c0 con 0x7faaa00b9ce0 2020-11-28T21:52:32.296 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.292+0000 7faaa7437700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaa0066c30 con 0x7faaa00605c0 2020-11-28T21:52:32.296 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.292+0000 7faaa51d3700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7faaa00b9ce0 0x7faaa01433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:32.296 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.292+0000 7faaa51d3700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7faaa00b9ce0 0x7faaa01433c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41460/0 (socket says 172.21.15.162:41460) 2020-11-28T21:52:32.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.292+0000 7faaa51d3700 1 -- 172.21.15.162:0/4042795074 learned_addr learned my addr 172.21.15.162:0/4042795074 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:32.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.292+0000 7faaa49d2700 1 --2- 172.21.15.162:0/4042795074 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faaa00605c0 0x7faaa0067610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:32.298 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.296+0000 7faaa51d3700 1 -- 172.21.15.162:0/4042795074 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faaa00605c0 msgr2=0x7faaa0067610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:32.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.296+0000 7faaa51d3700 1 --2- 172.21.15.162:0/4042795074 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faaa00605c0 0x7faaa0067610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:32.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.296+0000 7faaa51d3700 1 -- 172.21.15.162:0/4042795074 >> v1:172.21.15.118:6789/0 conn(0x7faaa00b3c30 legacy=0x7faaa013fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:32.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.296+0000 7faaa51d3700 1 -- 172.21.15.162:0/4042795074 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa0066ed0 con 0x7faaa00b9ce0 2020-11-28T21:52:32.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.296+0000 7faaa51d3700 1 --2- 172.21.15.162:0/4042795074 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faaa00b9ce0 0x7faaa01433c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7faa94002f30 tx=0x7faa94006e20).ready entity=mon.1 client_cookie=3359e8d55a234630 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:32.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.296+0000 7faa8bfff700 1 -- 172.21.15.162:0/4042795074 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa940097f0 con 0x7faaa00b9ce0 2020-11-28T21:52:32.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.296+0000 7faa8bfff700 1 -- 172.21.15.162:0/4042795074 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa94009950 con 0x7faaa00b9ce0 2020-11-28T21:52:32.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.296+0000 7faa8bfff700 1 -- 172.21.15.162:0/4042795074 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa94009cb0 con 0x7faaa00b9ce0 2020-11-28T21:52:32.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.296+0000 7faaa7437700 1 -- 172.21.15.162:0/4042795074 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faaa00b9ce0 msgr2=0x7faaa01433c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:32.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.296+0000 7faaa7437700 1 --2- 172.21.15.162:0/4042795074 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faaa00b9ce0 0x7faaa01433c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7faa94002f30 tx=0x7faa94006e20).stop 2020-11-28T21:52:32.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.296+0000 7faaa7437700 1 -- 172.21.15.162:0/4042795074 shutdown_connections 2020-11-28T21:52:32.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.296+0000 7faaa7437700 1 --2- 172.21.15.162:0/4042795074 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faaa00b9ce0 0x7faaa01433c0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:32.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.296+0000 7faaa7437700 1 --2- 172.21.15.162:0/4042795074 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faaa00605c0 0x7faaa0067610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:32.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.296+0000 7faaa7437700 1 -- 172.21.15.162:0/4042795074 >> 172.21.15.162:0/4042795074 conn(0x7faaa005c430 msgr2=0x7faaa00b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:32.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.296+0000 7faaa7437700 1 -- 172.21.15.162:0/4042795074 shutdown_connections 2020-11-28T21:52:32.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.296+0000 7faaa7437700 1 -- 172.21.15.162:0/4042795074 wait complete. 2020-11-28T21:52:32.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.296+0000 7faaa7437700 1 Processor -- start 2020-11-28T21:52:32.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.296+0000 7faaa7437700 1 -- start start 2020-11-28T21:52:32.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.296+0000 7faaa7437700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faaa00605c0 0x7faaa0146810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:32.306 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.296+0000 7faaa7437700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faaa00b3c30 0x7faaa00af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:32.306 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.296+0000 7faaa7437700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faaa00b9ce0 0x7faaa013fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:32.306 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.296+0000 7faaa7437700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa0146d00 con 0x7faaa00b3c30 2020-11-28T21:52:32.306 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.296+0000 7faaa7437700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa01456e0 con 0x7faaa00b9ce0 2020-11-28T21:52:32.307 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.296+0000 7faaa7437700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaa0145980 con 0x7faaa00605c0 2020-11-28T21:52:32.307 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.296+0000 7faaa59d4700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faaa00b9ce0 0x7faaa013fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:32.307 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.300+0000 7faaa59d4700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faaa00b9ce0 0x7faaa013fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41470/0 (socket says 172.21.15.162:41470) 2020-11-28T21:52:32.307 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.300+0000 7faaa59d4700 1 -- 172.21.15.162:0/935963383 learned_addr learned my addr 172.21.15.162:0/935963383 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:32.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.300+0000 7faaa49d2700 1 --2- 172.21.15.162:0/935963383 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faaa00b3c30 0x7faaa00af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:32.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.300+0000 7faaa51d3700 1 --2- 172.21.15.162:0/935963383 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faaa00605c0 0x7faaa0146810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:32.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.300+0000 7faaa59d4700 1 -- 172.21.15.162:0/935963383 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faaa00605c0 msgr2=0x7faaa0146810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:32.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.300+0000 7faaa59d4700 1 --2- 172.21.15.162:0/935963383 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faaa00605c0 0x7faaa0146810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:32.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.300+0000 7faaa59d4700 1 -- 172.21.15.162:0/935963383 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faaa00b3c30 msgr2=0x7faaa00af8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:32.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.300+0000 7faaa59d4700 1 --2- 172.21.15.162:0/935963383 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faaa00b3c30 0x7faaa00af8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:32.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.300+0000 7faaa59d4700 1 -- 172.21.15.162:0/935963383 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa0145c20 con 0x7faaa00b9ce0 2020-11-28T21:52:32.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.300+0000 7faaa49d2700 1 --2- 172.21.15.162:0/935963383 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faaa00b3c30 0x7faaa00af8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:52:32.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.300+0000 7faaa59d4700 1 --2- 172.21.15.162:0/935963383 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faaa00b9ce0 0x7faaa013fc60 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7faaa00b0350 tx=0x7faa98007f10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:32.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.300+0000 7faa92ffd700 1 -- 172.21.15.162:0/935963383 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa9800ce20 con 0x7faaa00b9ce0 2020-11-28T21:52:32.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.300+0000 7faaa7437700 1 -- 172.21.15.162:0/935963383 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaa00b5050 con 0x7faaa00b9ce0 2020-11-28T21:52:32.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.300+0000 7faa92ffd700 1 -- 172.21.15.162:0/935963383 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa9800a7b0 con 0x7faaa00b9ce0 2020-11-28T21:52:32.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.300+0000 7faaa7437700 1 -- 172.21.15.162:0/935963383 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaa01460c0 con 0x7faaa00b9ce0 2020-11-28T21:52:32.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.300+0000 7faa92ffd700 1 -- 172.21.15.162:0/935963383 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa9800ab10 con 0x7faaa00b9ce0 2020-11-28T21:52:32.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.300+0000 7faaa7437700 1 -- 172.21.15.162:0/935963383 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaa00aff50 con 0x7faaa00b9ce0 2020-11-28T21:52:32.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.300+0000 7faa92ffd700 1 -- 172.21.15.162:0/935963383 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7faa98019030 con 0x7faaa00b9ce0 2020-11-28T21:52:32.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.300+0000 7faa92ffd700 1 --2- 172.21.15.162:0/935963383 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7faa7c055330 0x7faa7c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:32.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.300+0000 7faa92ffd700 1 -- 172.21.15.162:0/935963383 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6073+0+0 (secure 0 0 0) 0x7faa9806d8e0 con 0x7faaa00b9ce0 2020-11-28T21:52:32.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.304+0000 7faaa51d3700 1 --2- 172.21.15.162:0/935963383 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7faa7c055330 0x7faa7c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:32.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.304+0000 7faaa51d3700 1 --2- 172.21.15.162:0/935963383 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7faa7c055330 0x7faa7c0577f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7faa9401b000 tx=0x7faa94015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:32.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.308+0000 7faa92ffd700 1 -- 172.21.15.162:0/935963383 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7faa98043b40 con 0x7faaa00b9ce0 2020-11-28T21:52:32.452 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.452+0000 7faaa7437700 1 -- 172.21.15.162:0/935963383 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7faaa0145db0 con 0x7faaa00b9ce0 2020-11-28T21:52:32.453 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.452+0000 7faa92ffd700 1 -- 172.21.15.162:0/935963383 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v76) v1 ==== 92+0+10 (secure 0 0 0) 0x7faa98047160 con 0x7faaa00b9ce0 2020-11-28T21:52:32.453 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:52:32.455 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.452+0000 7faaa7437700 1 -- 172.21.15.162:0/935963383 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7faa7c055330 msgr2=0x7faa7c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:32.455 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.452+0000 7faaa7437700 1 --2- 172.21.15.162:0/935963383 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7faa7c055330 0x7faa7c0577f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7faa9401b000 tx=0x7faa94015000).stop 2020-11-28T21:52:32.455 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.452+0000 7faaa7437700 1 -- 172.21.15.162:0/935963383 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faaa00b9ce0 msgr2=0x7faaa013fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:32.456 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.452+0000 7faaa7437700 1 --2- 172.21.15.162:0/935963383 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faaa00b9ce0 0x7faaa013fc60 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7faaa00b0350 tx=0x7faa98007f10).stop 2020-11-28T21:52:32.456 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.452+0000 7faaa7437700 1 -- 172.21.15.162:0/935963383 shutdown_connections 2020-11-28T21:52:32.456 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.452+0000 7faaa7437700 1 --2- 172.21.15.162:0/935963383 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7faa7c055330 0x7faa7c0577f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:32.456 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.452+0000 7faaa7437700 1 --2- 172.21.15.162:0/935963383 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faaa00605c0 0x7faaa0146810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:32.456 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.452+0000 7faaa7437700 1 --2- 172.21.15.162:0/935963383 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faaa00b9ce0 0x7faaa013fc60 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:32.457 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.452+0000 7faaa7437700 1 --2- 172.21.15.162:0/935963383 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faaa00b3c30 0x7faaa00af8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:32.457 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.452+0000 7faaa7437700 1 -- 172.21.15.162:0/935963383 >> 172.21.15.162:0/935963383 conn(0x7faaa005c430 msgr2=0x7faaa00b81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:32.457 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.452+0000 7faaa7437700 1 -- 172.21.15.162:0/935963383 shutdown_connections 2020-11-28T21:52:32.457 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.452+0000 7faaa7437700 1 -- 172.21.15.162:0/935963383 wait complete. 2020-11-28T21:52:32.466 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-28T21:52:32.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.580+0000 7f6a04dff700 1 Processor -- start 2020-11-28T21:52:32.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.580+0000 7f6a04dff700 1 -- start start 2020-11-28T21:52:32.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.580+0000 7f6a04dff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6a001433c0 0x7f6a000605c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:32.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.580+0000 7f6a04dff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f6a00067610 0x7f6a0013fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:32.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.580+0000 7f6a04dff700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a00143950 con 0x7f6a000b9ce0 2020-11-28T21:52:32.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.580+0000 7f6a04dff700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a000b4340 con 0x7f6a00067610 2020-11-28T21:52:32.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.580+0000 7f6a04dff700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a00067ac0 con 0x7f6a001433c0 2020-11-28T21:52:32.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.580+0000 7f69fe59c700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f6a000b9ce0 0x7f6a000b3c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:48318/0 (socket says 172.21.15.162:48318) 2020-11-28T21:52:32.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.580+0000 7f69fe59c700 1 -- 172.21.15.162:0/990970557 learned_addr learned my addr 172.21.15.162:0/990970557 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:52:32.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.580+0000 7f69fdd9b700 1 --2- 172.21.15.162:0/990970557 >> v2:172.21.15.162:3300/0 conn(0x7f6a001433c0 0x7f6a000605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:32.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.580+0000 7f69fed9d700 1 --2- 172.21.15.162:0/990970557 >> v2:172.21.15.118:3300/0 conn(0x7f6a00067610 0x7f6a0013fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:32.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.580+0000 7f69fd59a700 1 -- 172.21.15.162:0/990970557 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1655219334 0 0) 0x7f6a00143950 con 0x7f6a000b9ce0 2020-11-28T21:52:32.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.580+0000 7f69fd59a700 1 -- 172.21.15.162:0/990970557 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69e4003230 con 0x7f6a000b9ce0 2020-11-28T21:52:32.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.580+0000 7f69fdd9b700 1 -- 172.21.15.162:0/990970557 >> v2:172.21.15.118:3300/0 conn(0x7f6a00067610 msgr2=0x7f6a0013fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:32.586 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.580+0000 7f69fdd9b700 1 --2- 172.21.15.162:0/990970557 >> v2:172.21.15.118:3300/0 conn(0x7f6a00067610 0x7f6a0013fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:32.586 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.580+0000 7f69fdd9b700 1 -- 172.21.15.162:0/990970557 >> v1:172.21.15.162:6789/0 conn(0x7f6a000b9ce0 legacy=0x7f6a000b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:32.586 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.580+0000 7f69fd59a700 1 -- 172.21.15.162:0/990970557 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2365061879 0 0) 0x7f69e4003230 con 0x7f6a000b9ce0 2020-11-28T21:52:32.586 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.580+0000 7f69fdd9b700 1 -- 172.21.15.162:0/990970557 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a000ba0c0 con 0x7f6a001433c0 2020-11-28T21:52:32.586 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.580+0000 7f69fed9d700 1 --2- 172.21.15.162:0/990970557 >> v2:172.21.15.118:3300/0 conn(0x7f6a00067610 0x7f6a0013fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:52:32.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.580+0000 7f69fdd9b700 1 --2- 172.21.15.162:0/990970557 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a001433c0 0x7f6a000605c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7f69f00019f0 tx=0x7f69f4006dc0).ready entity=mon.1 client_cookie=af736971c654921d server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:32.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.580+0000 7f69fd59a700 1 -- 172.21.15.162:0/990970557 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69f0001850 con 0x7f6a001433c0 2020-11-28T21:52:32.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.580+0000 7f69fd59a700 1 -- 172.21.15.162:0/990970557 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f4009620 con 0x7f6a001433c0 2020-11-28T21:52:32.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.580+0000 7f69fd59a700 1 -- 172.21.15.162:0/990970557 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69f4009790 con 0x7f6a001433c0 2020-11-28T21:52:32.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.580+0000 7f6a04dff700 1 -- 172.21.15.162:0/990970557 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a001433c0 msgr2=0x7f6a000605c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:32.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.580+0000 7f6a04dff700 1 --2- 172.21.15.162:0/990970557 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a001433c0 0x7f6a000605c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7f69f00019f0 tx=0x7f69f4006dc0).stop 2020-11-28T21:52:32.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.584+0000 7f6a04dff700 1 -- 172.21.15.162:0/990970557 shutdown_connections 2020-11-28T21:52:32.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.584+0000 7f6a04dff700 1 --2- 172.21.15.162:0/990970557 >> v2:172.21.15.118:3300/0 conn(0x7f6a00067610 0x7f6a0013fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:32.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.584+0000 7f6a04dff700 1 --2- 172.21.15.162:0/990970557 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a001433c0 0x7f6a000605c0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:32.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.584+0000 7f6a04dff700 1 -- 172.21.15.162:0/990970557 >> 172.21.15.162:0/990970557 conn(0x7f6a0005c430 msgr2=0x7f6a000b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:32.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.584+0000 7f6a04dff700 1 -- 172.21.15.162:0/990970557 shutdown_connections 2020-11-28T21:52:32.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.584+0000 7f6a04dff700 1 -- 172.21.15.162:0/990970557 wait complete. 2020-11-28T21:52:32.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.584+0000 7f6a04dff700 1 Processor -- start 2020-11-28T21:52:32.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.584+0000 7f6a04dff700 1 -- start start 2020-11-28T21:52:32.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.584+0000 7f6a04dff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a00067610 0x7f6a00147af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:32.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.584+0000 7f6a04dff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6a000b9ce0 0x7f6a00147400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:32.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.584+0000 7f6a04dff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6a001433c0 0x7f6a000adc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:32.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.584+0000 7f6a04dff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0013ec00 con 0x7f6a000b9ce0 2020-11-28T21:52:32.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.584+0000 7f6a04dff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a00151c20 con 0x7f6a00067610 2020-11-28T21:52:32.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.584+0000 7f6a04dff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a00151ec0 con 0x7f6a001433c0 2020-11-28T21:52:32.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.584+0000 7f69fe59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a00067610 0x7f6a00147af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:32.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.584+0000 7f69fe59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a00067610 0x7f6a00147af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41480/0 (socket says 172.21.15.162:41480) 2020-11-28T21:52:32.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.584+0000 7f69fe59c700 1 -- 172.21.15.162:0/3117552099 learned_addr learned my addr 172.21.15.162:0/3117552099 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:32.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.584+0000 7f69fe59c700 1 -- 172.21.15.162:0/3117552099 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6a001433c0 msgr2=0x7f6a000adc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:32.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.584+0000 7f69fe59c700 1 --2- 172.21.15.162:0/3117552099 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6a001433c0 0x7f6a000adc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-28T21:52:32.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.584+0000 7f69fe59c700 1 -- 172.21.15.162:0/3117552099 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6a000b9ce0 msgr2=0x7f6a00147400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:32.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.584+0000 7f69fe59c700 1 --2- 172.21.15.162:0/3117552099 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6a000b9ce0 0x7f6a00147400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-28T21:52:32.594 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.584+0000 7f69fe59c700 1 -- 172.21.15.162:0/3117552099 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a00152160 con 0x7f6a00067610 2020-11-28T21:52:32.594 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.588+0000 7f69fe59c700 1 --2- 172.21.15.162:0/3117552099 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a00067610 0x7f6a00147af0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7f69f0001a30 tx=0x7f69f0002ce0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:32.594 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.588+0000 7f69ef7fe700 1 -- 172.21.15.162:0/3117552099 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69f00054b0 con 0x7f6a00067610 2020-11-28T21:52:32.594 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.588+0000 7f69ef7fe700 1 -- 172.21.15.162:0/3117552099 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f0005610 con 0x7f6a00067610 2020-11-28T21:52:32.594 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.588+0000 7f69ef7fe700 1 -- 172.21.15.162:0/3117552099 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69f0005970 con 0x7f6a00067610 2020-11-28T21:52:32.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.588+0000 7f6a04dff700 1 -- 172.21.15.162:0/3117552099 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a000b18a0 con 0x7f6a00067610 2020-11-28T21:52:32.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.588+0000 7f6a04dff700 1 -- 172.21.15.162:0/3117552099 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a00151960 con 0x7f6a00067610 2020-11-28T21:52:32.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.588+0000 7f6a04dff700 1 -- 172.21.15.162:0/3117552099 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a00057050 con 0x7f6a00067610 2020-11-28T21:52:32.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.592+0000 7f69ef7fe700 1 -- 172.21.15.162:0/3117552099 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f69f0010030 con 0x7f6a00067610 2020-11-28T21:52:32.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.592+0000 7f69ef7fe700 1 --2- 172.21.15.162:0/3117552099 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f69dc0554e0 0x7f69dc0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:32.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.592+0000 7f69ef7fe700 1 -- 172.21.15.162:0/3117552099 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f69f006e4d0 con 0x7f6a00067610 2020-11-28T21:52:32.597 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.592+0000 7f69fdd9b700 1 --2- 172.21.15.162:0/3117552099 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f69dc0554e0 0x7f69dc0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:32.597 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.592+0000 7f69fdd9b700 1 --2- 172.21.15.162:0/3117552099 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f69dc0554e0 0x7f69dc0579a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f69f00019f0 tx=0x7f69f4015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:32.597 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.596+0000 7f69ef7fe700 1 -- 172.21.15.162:0/3117552099 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f69f00447d0 con 0x7f6a00067610 2020-11-28T21:52:32.736 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.736+0000 7f6a04dff700 1 -- 172.21.15.162:0/3117552099 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6a00068350 con 0x7f6a00067610 2020-11-28T21:52:32.736 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.736+0000 7f69ef7fe700 1 -- 172.21.15.162:0/3117552099 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v76) v1 ==== 104+0+10 (secure 0 0 0) 0x7f69f0047df0 con 0x7f6a00067610 2020-11-28T21:52:32.737 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:52:32.738 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.736+0000 7f6a04dff700 1 -- 172.21.15.162:0/3117552099 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f69dc0554e0 msgr2=0x7f69dc0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:32.738 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.736+0000 7f6a04dff700 1 --2- 172.21.15.162:0/3117552099 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f69dc0554e0 0x7f69dc0579a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f69f00019f0 tx=0x7f69f4015000).stop 2020-11-28T21:52:32.739 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.736+0000 7f6a04dff700 1 -- 172.21.15.162:0/3117552099 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a00067610 msgr2=0x7f6a00147af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:32.739 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.736+0000 7f6a04dff700 1 --2- 172.21.15.162:0/3117552099 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a00067610 0x7f6a00147af0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7f69f0001a30 tx=0x7f69f0002ce0).stop 2020-11-28T21:52:32.739 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.736+0000 7f6a04dff700 1 -- 172.21.15.162:0/3117552099 shutdown_connections 2020-11-28T21:52:32.739 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.736+0000 7f6a04dff700 1 --2- 172.21.15.162:0/3117552099 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f69dc0554e0 0x7f69dc0579a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:32.739 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.736+0000 7f6a04dff700 1 --2- 172.21.15.162:0/3117552099 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6a001433c0 0x7f6a000adc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-28T21:52:32.740 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.736+0000 7f6a04dff700 1 --2- 172.21.15.162:0/3117552099 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a00067610 0x7f6a00147af0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:32.740 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.736+0000 7f6a04dff700 1 --2- 172.21.15.162:0/3117552099 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6a000b9ce0 0x7f6a00147400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-28T21:52:32.740 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.736+0000 7f6a04dff700 1 -- 172.21.15.162:0/3117552099 >> 172.21.15.162:0/3117552099 conn(0x7f6a0005c430 msgr2=0x7f6a0009e370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:32.740 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.736+0000 7f6a04dff700 1 -- 172.21.15.162:0/3117552099 shutdown_connections 2020-11-28T21:52:32.740 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.736+0000 7f6a04dff700 1 -- 172.21.15.162:0/3117552099 wait complete. 2020-11-28T21:52:32.749 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-28T21:52:32.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.860+0000 7f1ee94ba700 1 Processor -- start 2020-11-28T21:52:32.862 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.860+0000 7f1ee94ba700 1 -- start start 2020-11-28T21:52:32.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.860+0000 7f1ee94ba700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1ee40b2d00 0x7f1ee405c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:32.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.860+0000 7f1ee94ba700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1ee40b3ab0 0x7f1ee40b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:32.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.860+0000 7f1ee94ba700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ee4142ad0 0x7f1ee4142f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:32.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.860+0000 7f1ee94ba700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ee40b0e20 con 0x7f1ee40b3ab0 2020-11-28T21:52:32.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.860+0000 7f1ee94ba700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ee405baf0 con 0x7f1ee40b2d00 2020-11-28T21:52:32.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.860+0000 7f1ee94ba700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee405bd90 con 0x7f1ee4142ad0 2020-11-28T21:52:32.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.860+0000 7f1ee3fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1ee40b2d00 0x7f1ee405c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:32.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.860+0000 7f1ee3fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1ee40b2d00 0x7f1ee405c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41492/0 (socket says 172.21.15.162:41492) 2020-11-28T21:52:32.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.860+0000 7f1ee37fe700 1 --2- 172.21.15.162:0/4234326942 >> v2:172.21.15.118:3300/0 conn(0x7f1ee40b3ab0 0x7f1ee40b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:32.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.860+0000 7f1ee3fff700 1 -- 172.21.15.162:0/4234326942 learned_addr learned my addr 172.21.15.162:0/4234326942 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:32.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.860+0000 7f1ee8cb9700 1 --2- 172.21.15.162:0/4234326942 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ee4142ad0 0x7f1ee4142f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:32.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.860+0000 7f1ee3fff700 1 -- 172.21.15.162:0/4234326942 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ee4142ad0 msgr2=0x7f1ee4142f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:32.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.860+0000 7f1ee3fff700 1 --2- 172.21.15.162:0/4234326942 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ee4142ad0 0x7f1ee4142f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:32.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.860+0000 7f1ee3fff700 1 -- 172.21.15.162:0/4234326942 >> v2:172.21.15.118:3300/0 conn(0x7f1ee40b3ab0 msgr2=0x7f1ee40b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:32.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.860+0000 7f1ee3fff700 1 --2- 172.21.15.162:0/4234326942 >> v2:172.21.15.118:3300/0 conn(0x7f1ee40b3ab0 0x7f1ee40b30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:32.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.860+0000 7f1ee3fff700 1 -- 172.21.15.162:0/4234326942 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee405c030 con 0x7f1ee40b2d00 2020-11-28T21:52:32.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.860+0000 7f1ee37fe700 1 --2- 172.21.15.162:0/4234326942 >> v2:172.21.15.118:3300/0 conn(0x7f1ee40b3ab0 0x7f1ee40b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:52:32.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.864+0000 7f1ee3fff700 1 --2- 172.21.15.162:0/4234326942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ee40b2d00 0x7f1ee405c4d0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7f1ed40032b0 tx=0x7f1ed4009270).ready entity=mon.1 client_cookie=16832b0ba0f1ac91 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:32.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.864+0000 7f1ee2ffd700 1 -- 172.21.15.162:0/4234326942 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ed4009e60 con 0x7f1ee40b2d00 2020-11-28T21:52:32.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.864+0000 7f1ee2ffd700 1 -- 172.21.15.162:0/4234326942 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed400bb50 con 0x7f1ee40b2d00 2020-11-28T21:52:32.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.864+0000 7f1ee2ffd700 1 -- 172.21.15.162:0/4234326942 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ed400bcc0 con 0x7f1ee40b2d00 2020-11-28T21:52:32.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.864+0000 7f1ee94ba700 1 -- 172.21.15.162:0/4234326942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ee40b2d00 msgr2=0x7f1ee405c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:32.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.864+0000 7f1ee94ba700 1 --2- 172.21.15.162:0/4234326942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ee40b2d00 0x7f1ee405c4d0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7f1ed40032b0 tx=0x7f1ed4009270).stop 2020-11-28T21:52:32.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.864+0000 7f1ee94ba700 1 -- 172.21.15.162:0/4234326942 shutdown_connections 2020-11-28T21:52:32.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.864+0000 7f1ee94ba700 1 --2- 172.21.15.162:0/4234326942 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ee4142ad0 0x7f1ee4142f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:32.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.864+0000 7f1ee94ba700 1 --2- 172.21.15.162:0/4234326942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ee40b2d00 0x7f1ee405c4d0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:32.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.864+0000 7f1ee94ba700 1 --2- 172.21.15.162:0/4234326942 >> v2:172.21.15.118:3300/0 conn(0x7f1ee40b3ab0 0x7f1ee40b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:32.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.864+0000 7f1ee94ba700 1 -- 172.21.15.162:0/4234326942 >> 172.21.15.162:0/4234326942 conn(0x7f1ee40604c0 msgr2=0x7f1ee40608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:32.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.864+0000 7f1ee94ba700 1 -- 172.21.15.162:0/4234326942 shutdown_connections 2020-11-28T21:52:32.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.864+0000 7f1ee94ba700 1 -- 172.21.15.162:0/4234326942 wait complete. 2020-11-28T21:52:32.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.864+0000 7f1ee94ba700 1 Processor -- start 2020-11-28T21:52:32.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.864+0000 7f1ee94ba700 1 -- start start 2020-11-28T21:52:32.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.864+0000 7f1ee94ba700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ee40b2d00 0x7f1ee4137960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:32.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.864+0000 7f1ee94ba700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ee40b3ab0 0x7f1ee41551a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:32.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.864+0000 7f1ee94ba700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1ee4142ad0 0x7f1ee4155690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:32.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.864+0000 7f1ee94ba700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee4147660 con 0x7f1ee4142ad0 2020-11-28T21:52:32.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.864+0000 7f1ee94ba700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee40b4e30 con 0x7f1ee40b2d00 2020-11-28T21:52:32.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.864+0000 7f1ee94ba700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee41473c0 con 0x7f1ee40b3ab0 2020-11-28T21:52:32.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.864+0000 7f1ee3fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ee40b2d00 0x7f1ee4137960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:32.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.864+0000 7f1ee3fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ee40b2d00 0x7f1ee4137960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41498/0 (socket says 172.21.15.162:41498) 2020-11-28T21:52:32.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.864+0000 7f1ee3fff700 1 -- 172.21.15.162:0/2575551480 learned_addr learned my addr 172.21.15.162:0/2575551480 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:32.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.864+0000 7f1ee8cb9700 1 --2- 172.21.15.162:0/2575551480 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1ee4142ad0 0x7f1ee4155690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:32.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.864+0000 7f1ee37fe700 1 --2- 172.21.15.162:0/2575551480 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ee40b3ab0 0x7f1ee41551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:32.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.868+0000 7f1ee3fff700 1 -- 172.21.15.162:0/2575551480 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ee40b3ab0 msgr2=0x7f1ee41551a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:32.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.868+0000 7f1ee3fff700 1 --2- 172.21.15.162:0/2575551480 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ee40b3ab0 0x7f1ee41551a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:32.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.868+0000 7f1ee3fff700 1 -- 172.21.15.162:0/2575551480 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1ee4142ad0 msgr2=0x7f1ee4155690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:32.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.868+0000 7f1ee3fff700 1 --2- 172.21.15.162:0/2575551480 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1ee4142ad0 0x7f1ee4155690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:32.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.868+0000 7f1ee3fff700 1 -- 172.21.15.162:0/2575551480 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee40b5090 con 0x7f1ee40b2d00 2020-11-28T21:52:32.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.868+0000 7f1ee3fff700 1 --2- 172.21.15.162:0/2575551480 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ee40b2d00 0x7f1ee4137960 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7f1ed40032b0 tx=0x7f1ed4016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:32.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.868+0000 7f1ee17fa700 1 -- 172.21.15.162:0/2575551480 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ed401e610 con 0x7f1ee40b2d00 2020-11-28T21:52:32.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.868+0000 7f1ee17fa700 1 -- 172.21.15.162:0/2575551480 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed401e770 con 0x7f1ee40b2d00 2020-11-28T21:52:32.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.868+0000 7f1ee94ba700 1 -- 172.21.15.162:0/2575551480 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ee413f470 con 0x7f1ee40b2d00 2020-11-28T21:52:32.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.868+0000 7f1ee17fa700 1 -- 172.21.15.162:0/2575551480 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ed401ead0 con 0x7f1ee40b2d00 2020-11-28T21:52:32.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.868+0000 7f1ee94ba700 1 -- 172.21.15.162:0/2575551480 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ee4147a20 con 0x7f1ee40b2d00 2020-11-28T21:52:32.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.868+0000 7f1ee94ba700 1 -- 172.21.15.162:0/2575551480 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ee4057050 con 0x7f1ee40b2d00 2020-11-28T21:52:32.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.868+0000 7f1ee17fa700 1 -- 172.21.15.162:0/2575551480 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1ed400f070 con 0x7f1ee40b2d00 2020-11-28T21:52:32.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.868+0000 7f1ee17fa700 1 --2- 172.21.15.162:0/2575551480 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1ebc0554e0 0x7f1ebc0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:32.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.868+0000 7f1ee17fa700 1 -- 172.21.15.162:0/2575551480 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1ed406fe30 con 0x7f1ee40b2d00 2020-11-28T21:52:32.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.868+0000 7f1ee37fe700 1 --2- 172.21.15.162:0/2575551480 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1ebc0554e0 0x7f1ebc0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:32.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.872+0000 7f1ee37fe700 1 --2- 172.21.15.162:0/2575551480 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1ebc0554e0 0x7f1ebc0579a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f1ecc00b000 tx=0x7f1ecc006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:32.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:32.876+0000 7f1ee17fa700 1 -- 172.21.15.162:0/2575551480 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1ed4049a50 con 0x7f1ee40b2d00 2020-11-28T21:52:33.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.016+0000 7f1ee94ba700 1 -- 172.21.15.162:0/2575551480 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1ee4147a20 con 0x7f1ee40b2d00 2020-11-28T21:52:33.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.016+0000 7f1ee17fa700 1 -- 172.21.15.162:0/2575551480 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v76) v1 ==== 100+0+10 (secure 0 0 0) 0x7f1ed4015020 con 0x7f1ee40b2d00 2020-11-28T21:52:33.019 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:52:33.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.020+0000 7f1ee94ba700 1 -- 172.21.15.162:0/2575551480 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1ebc0554e0 msgr2=0x7f1ebc0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:33.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.020+0000 7f1ee94ba700 1 --2- 172.21.15.162:0/2575551480 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1ebc0554e0 0x7f1ebc0579a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f1ecc00b000 tx=0x7f1ecc006dc0).stop 2020-11-28T21:52:33.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.020+0000 7f1ee94ba700 1 -- 172.21.15.162:0/2575551480 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ee40b2d00 msgr2=0x7f1ee4137960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:33.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.020+0000 7f1ee94ba700 1 --2- 172.21.15.162:0/2575551480 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ee40b2d00 0x7f1ee4137960 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7f1ed40032b0 tx=0x7f1ed4016000).stop 2020-11-28T21:52:33.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.020+0000 7f1ee94ba700 1 -- 172.21.15.162:0/2575551480 shutdown_connections 2020-11-28T21:52:33.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.020+0000 7f1ee94ba700 1 --2- 172.21.15.162:0/2575551480 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1ebc0554e0 0x7f1ebc0579a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:33.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.020+0000 7f1ee94ba700 1 --2- 172.21.15.162:0/2575551480 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1ee4142ad0 0x7f1ee4155690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:33.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.020+0000 7f1ee94ba700 1 --2- 172.21.15.162:0/2575551480 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ee40b2d00 0x7f1ee4137960 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:33.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.020+0000 7f1ee94ba700 1 --2- 172.21.15.162:0/2575551480 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ee40b3ab0 0x7f1ee41551a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:33.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.020+0000 7f1ee94ba700 1 -- 172.21.15.162:0/2575551480 >> 172.21.15.162:0/2575551480 conn(0x7f1ee40604c0 msgr2=0x7f1ee413abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:33.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.020+0000 7f1ee94ba700 1 -- 172.21.15.162:0/2575551480 shutdown_connections 2020-11-28T21:52:33.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.020+0000 7f1ee94ba700 1 -- 172.21.15.162:0/2575551480 wait complete. 2020-11-28T21:52:33.039 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-28T21:52:33.040 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:52:33.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.144+0000 7f6ec51de700 1 Processor -- start 2020-11-28T21:52:33.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.144+0000 7f6ec51de700 1 -- start start 2020-11-28T21:52:33.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.144+0000 7f6ec51de700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6ec00b7920 0x7f6ec013f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:33.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.144+0000 7f6ec51de700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6ec00b2da0 0x7f6ec013fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:33.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.144+0000 7f6ec51de700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ec0140360 con 0x7f6ec0066fa0 2020-11-28T21:52:33.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.144+0000 7f6ec51de700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ec00b7d00 con 0x7f6ec00b7920 2020-11-28T21:52:33.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.144+0000 7f6ec51de700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec0143300 con 0x7f6ec00b2da0 2020-11-28T21:52:33.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.144+0000 7f6ebffff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6ec00b7920 0x7f6ec013f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:33.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.144+0000 7f6ec49dd700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f6ec0066fa0 0x7f6ec00b1db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:48350/0 (socket says 172.21.15.162:48350) 2020-11-28T21:52:33.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.144+0000 7f6ebffff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6ec00b7920 0x7f6ec013f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41502/0 (socket says 172.21.15.162:41502) 2020-11-28T21:52:33.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.144+0000 7f6ec49dd700 1 -- 172.21.15.162:0/2247822896 learned_addr learned my addr 172.21.15.162:0/2247822896 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:52:33.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.144+0000 7f6ebf7fe700 1 --2- 172.21.15.162:0/2247822896 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6ec00b2da0 0x7f6ec013fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:33.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.144+0000 7f6ebffff700 1 -- 172.21.15.162:0/2247822896 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6ec00b2da0 msgr2=0x7f6ec013fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:33.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.144+0000 7f6ebffff700 1 --2- 172.21.15.162:0/2247822896 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6ec00b2da0 0x7f6ec013fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:33.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.144+0000 7f6ebffff700 1 -- 172.21.15.162:0/2247822896 >> v1:172.21.15.162:6789/0 conn(0x7f6ec0066fa0 legacy=0x7f6ec00b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:33.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.144+0000 7f6ebffff700 1 -- 172.21.15.162:0/2247822896 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec0143560 con 0x7f6ec00b7920 2020-11-28T21:52:33.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.144+0000 7f6ebeffd700 1 -- 172.21.15.162:0/2247822896 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2631124684 0 0) 0x7f6eb0001450 con 0x7f6ec0066fa0 2020-11-28T21:52:33.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.144+0000 7f6ebf7fe700 1 --2- 172.21.15.162:0/2247822896 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6ec00b2da0 0x7f6ec013fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:52:33.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.144+0000 7f6ebffff700 1 --2- 172.21.15.162:0/2247822896 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ec00b7920 0x7f6ec013f980 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7f6eb8002f30 tx=0x7f6eb8006dc0).ready entity=mon.1 client_cookie=94ea5092f2f8e7c9 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:33.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.144+0000 7f6ebeffd700 1 -- 172.21.15.162:0/2247822896 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6eb80097f0 con 0x7f6ec00b7920 2020-11-28T21:52:33.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.144+0000 7f6ebeffd700 1 -- 172.21.15.162:0/2247822896 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb8009950 con 0x7f6ec00b7920 2020-11-28T21:52:33.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.144+0000 7f6ebeffd700 1 -- 172.21.15.162:0/2247822896 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6eb8009cb0 con 0x7f6ec00b7920 2020-11-28T21:52:33.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.144+0000 7f6ec51de700 1 -- 172.21.15.162:0/2247822896 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ec00b7920 msgr2=0x7f6ec013f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:33.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.144+0000 7f6ec51de700 1 --2- 172.21.15.162:0/2247822896 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ec00b7920 0x7f6ec013f980 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7f6eb8002f30 tx=0x7f6eb8006dc0).stop 2020-11-28T21:52:33.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.144+0000 7f6ec51de700 1 -- 172.21.15.162:0/2247822896 shutdown_connections 2020-11-28T21:52:33.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.144+0000 7f6ec51de700 1 --2- 172.21.15.162:0/2247822896 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ec00b7920 0x7f6ec013f980 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:33.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.144+0000 7f6ec51de700 1 --2- 172.21.15.162:0/2247822896 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6ec00b2da0 0x7f6ec013fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:33.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.144+0000 7f6ec51de700 1 -- 172.21.15.162:0/2247822896 >> 172.21.15.162:0/2247822896 conn(0x7f6ec005c440 msgr2=0x7f6ec00b6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:33.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.144+0000 7f6ec51de700 1 -- 172.21.15.162:0/2247822896 shutdown_connections 2020-11-28T21:52:33.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.144+0000 7f6ec51de700 1 -- 172.21.15.162:0/2247822896 wait complete. 2020-11-28T21:52:33.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.148+0000 7f6ec51de700 1 Processor -- start 2020-11-28T21:52:33.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.148+0000 7f6ec51de700 1 -- start start 2020-11-28T21:52:33.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.148+0000 7f6ec51de700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ec0066fa0 0x7f6ec0154c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:33.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.148+0000 7f6ec51de700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6ec00b2da0 0x7f6ec0149390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:33.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.148+0000 7f6ec51de700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6ec00b7920 0x7f6ec00ae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:33.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.148+0000 7f6ec51de700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec00aa7b0 con 0x7f6ec00b7920 2020-11-28T21:52:33.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.148+0000 7f6ec51de700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec0146a90 con 0x7f6ec0066fa0 2020-11-28T21:52:33.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.148+0000 7f6ec51de700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec01456c0 con 0x7f6ec00b2da0 2020-11-28T21:52:33.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.148+0000 7f6ebffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ec0066fa0 0x7f6ec0154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:33.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.148+0000 7f6ebffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ec0066fa0 0x7f6ec0154c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41508/0 (socket says 172.21.15.162:41508) 2020-11-28T21:52:33.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.148+0000 7f6ebffff700 1 -- 172.21.15.162:0/505459310 learned_addr learned my addr 172.21.15.162:0/505459310 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:33.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.148+0000 7f6ebf7fe700 1 --2- 172.21.15.162:0/505459310 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6ec00b2da0 0x7f6ec0149390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:33.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.148+0000 7f6ec49dd700 1 --2- 172.21.15.162:0/505459310 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6ec00b7920 0x7f6ec00ae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:33.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.148+0000 7f6ebffff700 1 -- 172.21.15.162:0/505459310 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6ec00b2da0 msgr2=0x7f6ec0149390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:33.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.148+0000 7f6ebffff700 1 --2- 172.21.15.162:0/505459310 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6ec00b2da0 0x7f6ec0149390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:33.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.148+0000 7f6ebffff700 1 -- 172.21.15.162:0/505459310 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6ec00b7920 msgr2=0x7f6ec00ae2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:33.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.148+0000 7f6ebffff700 1 --2- 172.21.15.162:0/505459310 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6ec00b7920 0x7f6ec00ae2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:33.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.148+0000 7f6ebffff700 1 -- 172.21.15.162:0/505459310 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec01452c0 con 0x7f6ec0066fa0 2020-11-28T21:52:33.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.148+0000 7f6ebf7fe700 1 --2- 172.21.15.162:0/505459310 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6ec00b2da0 0x7f6ec0149390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:52:33.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.148+0000 7f6ebffff700 1 --2- 172.21.15.162:0/505459310 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ec0066fa0 0x7f6ec0154c00 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7f6eb8002f30 tx=0x7f6eb8015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:33.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.148+0000 7f6ebd7fa700 1 -- 172.21.15.162:0/505459310 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6eb8011ce0 con 0x7f6ec0066fa0 2020-11-28T21:52:33.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.148+0000 7f6ebd7fa700 1 -- 172.21.15.162:0/505459310 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb8011e40 con 0x7f6ec0066fa0 2020-11-28T21:52:33.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.148+0000 7f6ec51de700 1 -- 172.21.15.162:0/505459310 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ec00ae7a0 con 0x7f6ec0066fa0 2020-11-28T21:52:33.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.148+0000 7f6ebd7fa700 1 -- 172.21.15.162:0/505459310 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6eb801d640 con 0x7f6ec0066fa0 2020-11-28T21:52:33.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.148+0000 7f6ec51de700 1 -- 172.21.15.162:0/505459310 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ec00aedc0 con 0x7f6ec0066fa0 2020-11-28T21:52:33.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.152+0000 7f6ec51de700 1 -- 172.21.15.162:0/505459310 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ec01468e0 con 0x7f6ec0066fa0 2020-11-28T21:52:33.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.152+0000 7f6ebd7fa700 1 -- 172.21.15.162:0/505459310 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6eb8024030 con 0x7f6ec0066fa0 2020-11-28T21:52:33.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.152+0000 7f6ebd7fa700 1 --2- 172.21.15.162:0/505459310 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6e98055350 0x7f6e98057810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:33.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.152+0000 7f6ebd7fa700 1 -- 172.21.15.162:0/505459310 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6eb806f260 con 0x7f6ec0066fa0 2020-11-28T21:52:33.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.152+0000 7f6ebf7fe700 1 --2- 172.21.15.162:0/505459310 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6e98055350 0x7f6e98057810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:33.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.152+0000 7f6ebf7fe700 1 --2- 172.21.15.162:0/505459310 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6e98055350 0x7f6e98057810 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f6ea800c000 tx=0x7f6ea8008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:33.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.156+0000 7f6ebd7fa700 1 -- 172.21.15.162:0/505459310 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6eb8045560 con 0x7f6ec0066fa0 2020-11-28T21:52:33.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.340+0000 7f6ec51de700 1 -- 172.21.15.162:0/505459310 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ec0057050 con 0x7f6ec0066fa0 2020-11-28T21:52:33.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.340+0000 7f6ebd7fa700 1 -- 172.21.15.162:0/505459310 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+2499 (secure 0 0 0) 0x7f6eb8022020 con 0x7f6ec0066fa0 2020-11-28T21:52:33.344 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:52:33.344 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":33,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5540,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2116325247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2116325247},{"type":"v1","addr":"172.21.15.162:6835","nonce":2116325247}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":30},{"gid":5543,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.118:6833/3491069258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3491069258},{"type":"v1","addr":"172.21.15.118:6833","nonce":3491069258}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:52:28.454745+0000","modified":"2020-11-28T21:52:29.457610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6172},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6172":{"gid":6172,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/163256983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":163256983},{"type":"v1","addr":"172.21.15.162:6837","nonce":163256983}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-28T21:52:33.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.344+0000 7f6ec51de700 1 -- 172.21.15.162:0/505459310 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6e98055350 msgr2=0x7f6e98057810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:33.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.344+0000 7f6ec51de700 1 --2- 172.21.15.162:0/505459310 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6e98055350 0x7f6e98057810 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f6ea800c000 tx=0x7f6ea8008040).stop 2020-11-28T21:52:33.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.344+0000 7f6ec51de700 1 -- 172.21.15.162:0/505459310 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ec0066fa0 msgr2=0x7f6ec0154c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:33.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.344+0000 7f6ec51de700 1 --2- 172.21.15.162:0/505459310 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ec0066fa0 0x7f6ec0154c00 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7f6eb8002f30 tx=0x7f6eb8015000).stop 2020-11-28T21:52:33.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.344+0000 7f6ec51de700 1 -- 172.21.15.162:0/505459310 shutdown_connections 2020-11-28T21:52:33.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.344+0000 7f6ec51de700 1 --2- 172.21.15.162:0/505459310 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6e98055350 0x7f6e98057810 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:33.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.344+0000 7f6ec51de700 1 --2- 172.21.15.162:0/505459310 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6ec00b7920 0x7f6ec00ae2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:33.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.344+0000 7f6ec51de700 1 --2- 172.21.15.162:0/505459310 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ec0066fa0 0x7f6ec0154c00 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:33.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.344+0000 7f6ec51de700 1 --2- 172.21.15.162:0/505459310 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6ec00b2da0 0x7f6ec0149390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:33.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.344+0000 7f6ec51de700 1 -- 172.21.15.162:0/505459310 >> 172.21.15.162:0/505459310 conn(0x7f6ec005c440 msgr2=0x7f6ec0141a20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:33.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.344+0000 7f6ec51de700 1 -- 172.21.15.162:0/505459310 shutdown_connections 2020-11-28T21:52:33.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:33.344+0000 7f6ec51de700 1 -- 172.21.15.162:0/505459310 wait complete. 2020-11-28T21:52:33.351 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 33 2020-11-28T21:52:33.663 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-28T21:52:28.454745+0000', 'modified': '2020-11-28T21:52:29.457610+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6172}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6172': {'gid': 6172, 'name': 'a', 'rank': 0, 'incarnation': 32, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6837/163256983', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6836', 'nonce': 163256983}, {'type': 'v1', 'addr': '172.21.15.162:6837', 'nonce': 163256983}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-28T21:52:33.664 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-28T21:52:33.664 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-28T21:52:33.793 INFO:teuthology.orchestra.run.smithi162.stdout:{"cluster_fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","whoami":0,"id":6172,"want_state":"up:active","state":"up:active","rank_uptime":5.3256764560000001,"mdsmap_epoch":33,"osdmap_epoch":76,"osdmap_epoch_barrier":75,"uptime":10.753168666000001} 2020-11-28T21:52:33.793 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", "id": 6172, "mdsmap_epoch": 33, "osdmap_epoch": 76, "osdmap_epoch_barrier": 75, "rank_uptime": 5.325676456, "state": "up:active", "uptime": 10.753168666, "want_state": "up:active", "whoami": 0 } 2020-11-28T21:52:33.794 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-11-28T21:52:33.794 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-11-28T21:52:33.795 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi192.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-11-28T21:52:33.795 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:52:33.795 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:52:33.803 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:52:33.860 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:52:33.861 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:52:33.915 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:52:33.916 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:52:33.917 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:52:33.918 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:52:33.967 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-11-28T21:52:33.967 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:52:34.010 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:52:34.027 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:52:34.028 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:52:34.083 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:52:34.084 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:52:34.085 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:52:34.085 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:52:34.086 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:2020-11-28T21:52:34.084+0000 7fe39f0320c0 -1 init, newargv = 0x55ba419df750 newargc=9ceph-fuse[12048]: starting ceph client 2020-11-28T21:52:34.087 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr: 2020-11-28T21:52:34.097 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:ceph-fuse[12048]: starting fuse 2020-11-28T21:52:35.191 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:52:35.211 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:52:35.211 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:52:35.268 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:52:35.269 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:52:35.269 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:52:35.270 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:52:35.322 INFO:teuthology.orchestra.run.smithi192.stdout:49 2020-11-28T21:52:35.323 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2020-11-28T21:52:35.324 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:52:35.324 INFO:teuthology.orchestra.run.smithi192:> sudo python3 -c ' 2020-11-28T21:52:35.325 INFO:teuthology.orchestra.run.smithi192:> import glob 2020-11-28T21:52:35.325 INFO:teuthology.orchestra.run.smithi192:> import re 2020-11-28T21:52:35.325 INFO:teuthology.orchestra.run.smithi192:> import os 2020-11-28T21:52:35.326 INFO:teuthology.orchestra.run.smithi192:> import subprocess 2020-11-28T21:52:35.326 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:52:35.327 INFO:teuthology.orchestra.run.smithi192:> def find_socket(client_name): 2020-11-28T21:52:35.327 INFO:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-11-28T21:52:35.328 INFO:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2020-11-28T21:52:35.328 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:52:35.328 INFO:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2020-11-28T21:52:35.329 INFO:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2020-11-28T21:52:35.330 INFO:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2020-11-28T21:52:35.331 INFO:teuthology.orchestra.run.smithi192:> return files[0] 2020-11-28T21:52:35.332 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:52:35.332 INFO:teuthology.orchestra.run.smithi192:> for f in files: 2020-11-28T21:52:35.333 INFO:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-11-28T21:52:35.334 INFO:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2020-11-28T21:52:35.334 INFO:teuthology.orchestra.run.smithi192:> return f 2020-11-28T21:52:35.334 INFO:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-11-28T21:52:35.335 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:52:35.335 INFO:teuthology.orchestra.run.smithi192:> print(find_socket("client.0")) 2020-11-28T21:52:35.336 INFO:teuthology.orchestra.run.smithi192:> ' 2020-11-28T21:52:35.414 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.0.12048.asok 2020-11-28T21:52:35.419 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.12048.asok 2020-11-28T21:52:35.420 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:52:35.420 INFO:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.12048.asok status 2020-11-28T21:52:35.517 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2020-11-28T21:52:35.518 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2020-11-28T21:52:35.518 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "a83a3572a957356977342f461361c1cd3ec06dbd", 2020-11-28T21:52:35.518 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 15.2.6-595-ga83a3572 (a83a3572a957356977342f461361c1cd3ec06dbd) octopus (stable)", 2020-11-28T21:52:35.519 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "0", 2020-11-28T21:52:35.519 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2020-11-28T21:52:35.519 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-11-28T21:52:35.519 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "12048", 2020-11-28T21:52:35.520 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2020-11-28T21:52:35.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:52:35.520 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2020-11-28T21:52:35.521 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2020-11-28T21:52:35.521 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 5654, 2020-11-28T21:52:35.521 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2020-11-28T21:52:35.521 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2020-11-28T21:52:35.521 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2020-11-28T21:52:35.522 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 5654 2020-11-28T21:52:35.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:52:35.522 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2020-11-28T21:52:35.523 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2020-11-28T21:52:35.523 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2020-11-28T21:52:35.523 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 2498670596 2020-11-28T21:52:35.523 INFO:teuthology.orchestra.run.smithi192.stdout: } 2020-11-28T21:52:35.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:52:35.524 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2020-11-28T21:52:35.524 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2020-11-28T21:52:35.524 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2020-11-28T21:52:35.525 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 2498670596 2020-11-28T21:52:35.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:52:35.525 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.5654 172.21.15.192:0/2498670596", 2020-11-28T21:52:35.525 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "172.21.15.192:0/2498670596", 2020-11-28T21:52:35.526 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2020-11-28T21:52:35.526 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 33, 2020-11-28T21:52:35.526 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 76, 2020-11-28T21:52:35.527 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2020-11-28T21:52:35.527 INFO:teuthology.orchestra.run.smithi192.stdout: "blacklisted": false 2020-11-28T21:52:35.528 INFO:teuthology.orchestra.run.smithi192.stdout:} 2020-11-28T21:52:35.529 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:52:35.530 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:52:35.530 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:52:35.539 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2020-11-28T21:52:35.539 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-28T21:52:35.540 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:52:35.540 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:52:35.597 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:52:35.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.712+0000 7fbad4ed6700 1 Processor -- start 2020-11-28T21:52:35.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.712+0000 7fbad4ed6700 1 -- start start 2020-11-28T21:52:35.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.716+0000 7fbad4ed6700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fbad005cbd0 0x7fbad00aac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:35.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.716+0000 7fbad4ed6700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad00b0230 con 0x7fbad0060cf0 2020-11-28T21:52:35.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.716+0000 7fbad4ed6700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad00ab0f0 con 0x7fbad00af640 2020-11-28T21:52:35.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.716+0000 7fbad4ed6700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbad00b12a0 con 0x7fbad005cbd0 2020-11-28T21:52:35.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.716+0000 7fbacffff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fbad00af640 0x7fbad00ae1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:48364/0 (socket says 172.21.15.162:48364) 2020-11-28T21:52:35.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.716+0000 7fbacffff700 1 -- 172.21.15.162:0/2317102033 learned_addr learned my addr 172.21.15.162:0/2317102033 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:52:35.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.716+0000 7fbaceffd700 1 --2- 172.21.15.162:0/2317102033 >> v2:172.21.15.118:3300/0 conn(0x7fbad005cbd0 0x7fbad00aac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:35.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.716+0000 7fbace7fc700 1 -- 172.21.15.162:0/2317102033 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2170336145 0 0) 0x7fbad00ab0f0 con 0x7fbad00af640 2020-11-28T21:52:35.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.716+0000 7fbace7fc700 1 -- 172.21.15.162:0/2317102033 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbabc003230 con 0x7fbad00af640 2020-11-28T21:52:35.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.716+0000 7fbaceffd700 1 -- 172.21.15.162:0/2317102033 >> v1:172.21.15.162:6789/0 conn(0x7fbad00af640 legacy=0x7fbad00ae1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:35.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.716+0000 7fbace7fc700 1 -- 172.21.15.162:0/2317102033 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 344284022 0 0) 0x7fbabc003230 con 0x7fbad00af640 2020-11-28T21:52:35.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.716+0000 7fbaceffd700 1 -- 172.21.15.162:0/2317102033 >> v1:172.21.15.118:6789/0 conn(0x7fbad0060cf0 legacy=0x7fbad00aa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:35.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.716+0000 7fbaceffd700 1 -- 172.21.15.162:0/2317102033 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad00a8c00 con 0x7fbad005cbd0 2020-11-28T21:52:35.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.716+0000 7fbaceffd700 1 --2- 172.21.15.162:0/2317102033 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbad005cbd0 0x7fbad00aac00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7fbac0001450 tx=0x7fbac4006dc0).ready entity=mon.0 client_cookie=b71845165864173 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:35.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.716+0000 7fbace7fc700 1 -- 172.21.15.162:0/2317102033 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbac80016d0 con 0x7fbad005cbd0 2020-11-28T21:52:35.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.716+0000 7fbace7fc700 1 -- 172.21.15.162:0/2317102033 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbac400d5c0 con 0x7fbad005cbd0 2020-11-28T21:52:35.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.716+0000 7fbace7fc700 1 -- 172.21.15.162:0/2317102033 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbac400d730 con 0x7fbad005cbd0 2020-11-28T21:52:35.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.716+0000 7fbad4ed6700 1 -- 172.21.15.162:0/2317102033 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbad005cbd0 msgr2=0x7fbad00aac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:35.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.716+0000 7fbad4ed6700 1 --2- 172.21.15.162:0/2317102033 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbad005cbd0 0x7fbad00aac00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7fbac0001450 tx=0x7fbac4006dc0).stop 2020-11-28T21:52:35.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.716+0000 7fbad4ed6700 1 -- 172.21.15.162:0/2317102033 shutdown_connections 2020-11-28T21:52:35.724 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.716+0000 7fbad4ed6700 1 --2- 172.21.15.162:0/2317102033 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbad005cbd0 0x7fbad00aac00 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:35.724 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.716+0000 7fbad4ed6700 1 -- 172.21.15.162:0/2317102033 >> 172.21.15.162:0/2317102033 conn(0x7fbad005b970 msgr2=0x7fbad005bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:35.724 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.716+0000 7fbad4ed6700 1 -- 172.21.15.162:0/2317102033 shutdown_connections 2020-11-28T21:52:35.724 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.716+0000 7fbad4ed6700 1 -- 172.21.15.162:0/2317102033 wait complete. 2020-11-28T21:52:35.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.720+0000 7fbad4ed6700 1 Processor -- start 2020-11-28T21:52:35.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.720+0000 7fbad4ed6700 1 -- start start 2020-11-28T21:52:35.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.720+0000 7fbad4ed6700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbad005cbd0 0x7fbad005f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:35.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.720+0000 7fbad4ed6700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbad0060cf0 0x7fbad0066910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:35.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.720+0000 7fbad4ed6700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbad00af640 0x7fbad0066e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:35.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.720+0000 7fbad4ed6700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad0151ef0 con 0x7fbad0060cf0 2020-11-28T21:52:35.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.720+0000 7fbad4ed6700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad0152190 con 0x7fbad005cbd0 2020-11-28T21:52:35.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.720+0000 7fbad4ed6700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbad0152430 con 0x7fbad00af640 2020-11-28T21:52:35.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.720+0000 7fbacf7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbad005cbd0 0x7fbad005f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:35.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.720+0000 7fbacf7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbad005cbd0 0x7fbad005f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41524/0 (socket says 172.21.15.162:41524) 2020-11-28T21:52:35.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.724+0000 7fbacf7fe700 1 -- 172.21.15.162:0/1342729198 learned_addr learned my addr 172.21.15.162:0/1342729198 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:35.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.724+0000 7fbacffff700 1 --2- 172.21.15.162:0/1342729198 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbad00af640 0x7fbad0066e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:35.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.724+0000 7fbaceffd700 1 --2- 172.21.15.162:0/1342729198 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbad0060cf0 0x7fbad0066910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:35.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.724+0000 7fbacf7fe700 1 -- 172.21.15.162:0/1342729198 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbad00af640 msgr2=0x7fbad0066e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:35.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.724+0000 7fbacf7fe700 1 --2- 172.21.15.162:0/1342729198 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbad00af640 0x7fbad0066e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:35.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.724+0000 7fbacf7fe700 1 -- 172.21.15.162:0/1342729198 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbad0060cf0 msgr2=0x7fbad0066910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:35.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.724+0000 7fbacf7fe700 1 --2- 172.21.15.162:0/1342729198 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbad0060cf0 0x7fbad0066910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:35.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.724+0000 7fbacf7fe700 1 -- 172.21.15.162:0/1342729198 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad01526d0 con 0x7fbad005cbd0 2020-11-28T21:52:35.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.724+0000 7fbacf7fe700 1 --2- 172.21.15.162:0/1342729198 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbad005cbd0 0x7fbad005f950 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7fbac0001510 tx=0x7fbac0007f50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:35.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.724+0000 7fbaccff9700 1 -- 172.21.15.162:0/1342729198 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbac0006a70 con 0x7fbad005cbd0 2020-11-28T21:52:35.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.724+0000 7fbaccff9700 1 -- 172.21.15.162:0/1342729198 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbac0006bd0 con 0x7fbad005cbd0 2020-11-28T21:52:35.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.724+0000 7fbaccff9700 1 -- 172.21.15.162:0/1342729198 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbac0007030 con 0x7fbad005cbd0 2020-11-28T21:52:35.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.724+0000 7fbad4ed6700 1 -- 172.21.15.162:0/1342729198 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbad005fe40 con 0x7fbad005cbd0 2020-11-28T21:52:35.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.724+0000 7fbad4ed6700 1 -- 172.21.15.162:0/1342729198 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbad00672f0 con 0x7fbad005cbd0 2020-11-28T21:52:35.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.724+0000 7fbad4ed6700 1 -- 172.21.15.162:0/1342729198 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbad0152860 con 0x7fbad005cbd0 2020-11-28T21:52:35.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.724+0000 7fbaccff9700 1 -- 172.21.15.162:0/1342729198 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fbac0019030 con 0x7fbad005cbd0 2020-11-28T21:52:35.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.724+0000 7fbaccff9700 1 --2- 172.21.15.162:0/1342729198 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbab0055440 0x7fbab0057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:35.733 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.724+0000 7fbaccff9700 1 -- 172.21.15.162:0/1342729198 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbac006d710 con 0x7fbad005cbd0 2020-11-28T21:52:35.733 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.724+0000 7fbaceffd700 1 --2- 172.21.15.162:0/1342729198 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbab0055440 0x7fbab0057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:35.734 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.728+0000 7fbaceffd700 1 --2- 172.21.15.162:0/1342729198 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbab0055440 0x7fbab0057900 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7fbac401a000 tx=0x7fbac4006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:35.734 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.732+0000 7fbaccff9700 1 -- 172.21.15.162:0/1342729198 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fbac0041df0 con 0x7fbad005cbd0 2020-11-28T21:52:35.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.876+0000 7fbad4ed6700 1 -- 172.21.15.162:0/1342729198 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbad0057050 con 0x7fbad005cbd0 2020-11-28T21:52:35.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.880+0000 7fbaccff9700 1 -- 172.21.15.162:0/1342729198 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+16137 (secure 0 0 0) 0x7fbac0024f20 con 0x7fbad005cbd0 2020-11-28T21:52:35.880 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:52:35.881 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":76,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:52:29.458247+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:52:25.955831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2020-11-28T21:52:27.265702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:52:35.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.880+0000 7fbad4ed6700 1 -- 172.21.15.162:0/1342729198 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbab0055440 msgr2=0x7fbab0057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:35.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.880+0000 7fbad4ed6700 1 --2- 172.21.15.162:0/1342729198 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbab0055440 0x7fbab0057900 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7fbac401a000 tx=0x7fbac4006dc0).stop 2020-11-28T21:52:35.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.880+0000 7fbad4ed6700 1 -- 172.21.15.162:0/1342729198 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbad005cbd0 msgr2=0x7fbad005f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:35.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.880+0000 7fbad4ed6700 1 --2- 172.21.15.162:0/1342729198 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbad005cbd0 0x7fbad005f950 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7fbac0001510 tx=0x7fbac0007f50).stop 2020-11-28T21:52:35.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.880+0000 7fbad4ed6700 1 -- 172.21.15.162:0/1342729198 shutdown_connections 2020-11-28T21:52:35.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.880+0000 7fbad4ed6700 1 --2- 172.21.15.162:0/1342729198 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbab0055440 0x7fbab0057900 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:35.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.880+0000 7fbad4ed6700 1 --2- 172.21.15.162:0/1342729198 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbad00af640 0x7fbad0066e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:35.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.880+0000 7fbad4ed6700 1 --2- 172.21.15.162:0/1342729198 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbad005cbd0 0x7fbad005f950 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:35.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.880+0000 7fbad4ed6700 1 --2- 172.21.15.162:0/1342729198 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbad0060cf0 0x7fbad0066910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:35.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.880+0000 7fbad4ed6700 1 -- 172.21.15.162:0/1342729198 >> 172.21.15.162:0/1342729198 conn(0x7fbad005b970 msgr2=0x7fbad0142f40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:35.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.880+0000 7fbad4ed6700 1 -- 172.21.15.162:0/1342729198 shutdown_connections 2020-11-28T21:52:35.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:35.880+0000 7fbad4ed6700 1 -- 172.21.15.162:0/1342729198 wait complete. 2020-11-28T21:52:35.895 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-28T21:52:25.955831+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2020-11-28T21:52:27.265702+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-28T21:52:35.896 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:52:36.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.024+0000 7fd8b7805700 1 Processor -- start 2020-11-28T21:52:36.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.024+0000 7fd8b7805700 1 -- start start 2020-11-28T21:52:36.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.024+0000 7fd8b7805700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd8b005ccc0 0x7fd8b013fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:36.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.024+0000 7fd8b7805700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd8b00ac240 0x7fd8b013e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:36.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.024+0000 7fd8b7805700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b0061630 con 0x7fd8b00b1200 2020-11-28T21:52:36.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.024+0000 7fd8b7805700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8b005d0a0 con 0x7fd8b00ac240 2020-11-28T21:52:36.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.024+0000 7fd8b7805700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b00aa710 con 0x7fd8b005ccc0 2020-11-28T21:52:36.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.024+0000 7fd8b5da2700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd8b00ac240 0x7fd8b013e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:36.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.024+0000 7fd8b5da2700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd8b00ac240 0x7fd8b013e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41534/0 (socket says 172.21.15.162:41534) 2020-11-28T21:52:36.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.024+0000 7fd8b4da0700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fd8b00b1200 0x7fd8b01401d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:48376/0 (socket says 172.21.15.162:48376) 2020-11-28T21:52:36.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.024+0000 7fd8b5da2700 1 -- 172.21.15.162:0/3244985295 learned_addr learned my addr 172.21.15.162:0/3244985295 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:36.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.024+0000 7fd8b55a1700 1 --2- 172.21.15.162:0/3244985295 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd8b005ccc0 0x7fd8b013fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:36.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.024+0000 7fd8b5da2700 1 -- 172.21.15.162:0/3244985295 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd8b005ccc0 msgr2=0x7fd8b013fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:36.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.024+0000 7fd8b5da2700 1 --2- 172.21.15.162:0/3244985295 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd8b005ccc0 0x7fd8b013fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:36.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.024+0000 7fd8b5da2700 1 -- 172.21.15.162:0/3244985295 >> v1:172.21.15.162:6789/0 conn(0x7fd8b00b1200 legacy=0x7fd8b01401d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:36.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.024+0000 7fd8a7fff700 1 -- 172.21.15.162:0/3244985295 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2288459615 0 0) 0x7fd8b0061630 con 0x7fd8b00b1200 2020-11-28T21:52:36.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.024+0000 7fd8b5da2700 1 -- 172.21.15.162:0/3244985295 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b0061860 con 0x7fd8b00ac240 2020-11-28T21:52:36.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.028+0000 7fd8b5da2700 1 --2- 172.21.15.162:0/3244985295 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8b00ac240 0x7fd8b013e640 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7fd8a80032b0 tx=0x7fd8a8009270).ready entity=mon.1 client_cookie=c11aaa8376f9a96e server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:36.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.028+0000 7fd8a7fff700 1 -- 172.21.15.162:0/3244985295 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8a800bdf0 con 0x7fd8b00ac240 2020-11-28T21:52:36.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.028+0000 7fd8a7fff700 1 -- 172.21.15.162:0/3244985295 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a8009e60 con 0x7fd8b00ac240 2020-11-28T21:52:36.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.028+0000 7fd8a7fff700 1 -- 172.21.15.162:0/3244985295 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8a8010660 con 0x7fd8b00ac240 2020-11-28T21:52:36.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.028+0000 7fd8b7805700 1 -- 172.21.15.162:0/3244985295 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8b00ac240 msgr2=0x7fd8b013e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:36.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.028+0000 7fd8b7805700 1 --2- 172.21.15.162:0/3244985295 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8b00ac240 0x7fd8b013e640 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7fd8a80032b0 tx=0x7fd8a8009270).stop 2020-11-28T21:52:36.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.028+0000 7fd8b7805700 1 -- 172.21.15.162:0/3244985295 shutdown_connections 2020-11-28T21:52:36.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.028+0000 7fd8b7805700 1 --2- 172.21.15.162:0/3244985295 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8b00ac240 0x7fd8b013e640 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:36.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.028+0000 7fd8b7805700 1 --2- 172.21.15.162:0/3244985295 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd8b005ccc0 0x7fd8b013fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:36.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.028+0000 7fd8b7805700 1 -- 172.21.15.162:0/3244985295 >> 172.21.15.162:0/3244985295 conn(0x7fd8b00b6220 msgr2=0x7fd8b00b21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:36.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.028+0000 7fd8b7805700 1 -- 172.21.15.162:0/3244985295 shutdown_connections 2020-11-28T21:52:36.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.028+0000 7fd8b7805700 1 -- 172.21.15.162:0/3244985295 wait complete. 2020-11-28T21:52:36.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.028+0000 7fd8b7805700 1 Processor -- start 2020-11-28T21:52:36.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.032+0000 7fd8b7805700 1 -- start start 2020-11-28T21:52:36.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.032+0000 7fd8b7805700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd8b005ccc0 0x7fd8b0066d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:36.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.032+0000 7fd8b7805700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8b00ac240 0x7fd8b0067280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:36.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.032+0000 7fd8b7805700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd8b00b1200 0x7fd8b014d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:36.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.032+0000 7fd8b7805700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b01519f0 con 0x7fd8b005ccc0 2020-11-28T21:52:36.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.032+0000 7fd8b7805700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b0067770 con 0x7fd8b00ac240 2020-11-28T21:52:36.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.032+0000 7fd8b7805700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b0067c30 con 0x7fd8b00b1200 2020-11-28T21:52:36.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.032+0000 7fd8b4da0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8b00ac240 0x7fd8b0067280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:36.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.032+0000 7fd8b4da0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8b00ac240 0x7fd8b0067280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41538/0 (socket says 172.21.15.162:41538) 2020-11-28T21:52:36.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.032+0000 7fd8b4da0700 1 -- 172.21.15.162:0/171377743 learned_addr learned my addr 172.21.15.162:0/171377743 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:36.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.032+0000 7fd8b5da2700 1 --2- 172.21.15.162:0/171377743 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd8b00b1200 0x7fd8b014d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:36.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.032+0000 7fd8b55a1700 1 --2- 172.21.15.162:0/171377743 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd8b005ccc0 0x7fd8b0066d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:36.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.032+0000 7fd8b4da0700 1 -- 172.21.15.162:0/171377743 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd8b00b1200 msgr2=0x7fd8b014d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:36.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.032+0000 7fd8b4da0700 1 --2- 172.21.15.162:0/171377743 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd8b00b1200 0x7fd8b014d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:36.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.032+0000 7fd8b4da0700 1 -- 172.21.15.162:0/171377743 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd8b005ccc0 msgr2=0x7fd8b0066d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:36.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.032+0000 7fd8b4da0700 1 --2- 172.21.15.162:0/171377743 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd8b005ccc0 0x7fd8b0066d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:36.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.032+0000 7fd8b4da0700 1 -- 172.21.15.162:0/171377743 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b00aefe0 con 0x7fd8b00ac240 2020-11-28T21:52:36.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.032+0000 7fd8b4da0700 1 --2- 172.21.15.162:0/171377743 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8b00ac240 0x7fd8b0067280 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7fd8b00659f0 tx=0x7fd8a0007c00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:36.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.032+0000 7fd8a67fc700 1 -- 172.21.15.162:0/171377743 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8a00072e0 con 0x7fd8b00ac240 2020-11-28T21:52:36.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.032+0000 7fd8a67fc700 1 -- 172.21.15.162:0/171377743 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a0005520 con 0x7fd8b00ac240 2020-11-28T21:52:36.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.032+0000 7fd8a67fc700 1 -- 172.21.15.162:0/171377743 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8a0005880 con 0x7fd8b00ac240 2020-11-28T21:52:36.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.032+0000 7fd8b7805700 1 -- 172.21.15.162:0/171377743 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b00af230 con 0x7fd8b00ac240 2020-11-28T21:52:36.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.032+0000 7fd8b7805700 1 -- 172.21.15.162:0/171377743 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b0151550 con 0x7fd8b00ac240 2020-11-28T21:52:36.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.036+0000 7fd8a67fc700 1 -- 172.21.15.162:0/171377743 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd8a0010070 con 0x7fd8b00ac240 2020-11-28T21:52:36.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.036+0000 7fd8b7805700 1 -- 172.21.15.162:0/171377743 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b0151710 con 0x7fd8b00ac240 2020-11-28T21:52:36.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.036+0000 7fd8a67fc700 1 --2- 172.21.15.162:0/171377743 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd898055500 0x7fd8980579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:36.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.036+0000 7fd8a67fc700 1 -- 172.21.15.162:0/171377743 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd8a006d590 con 0x7fd8b00ac240 2020-11-28T21:52:36.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.036+0000 7fd8b55a1700 1 --2- 172.21.15.162:0/171377743 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd898055500 0x7fd8980579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:36.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.036+0000 7fd8b55a1700 1 --2- 172.21.15.162:0/171377743 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd898055500 0x7fd8980579c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7fd8ac00b000 tx=0x7fd8ac008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:36.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.040+0000 7fd8a67fc700 1 -- 172.21.15.162:0/171377743 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd8a00437f0 con 0x7fd8b00ac240 2020-11-28T21:52:36.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.184+0000 7fd8b7805700 1 -- 172.21.15.162:0/171377743 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fd8b00afaf0 con 0x7fd8b00ac240 2020-11-28T21:52:36.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.184+0000 7fd8a67fc700 1 -- 172.21.15.162:0/171377743 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v76) v1 ==== 110+0+10 (secure 0 0 0) 0x7fd8a0046e10 con 0x7fd8b00ac240 2020-11-28T21:52:36.185 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:52:36.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.184+0000 7fd8b7805700 1 -- 172.21.15.162:0/171377743 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd898055500 msgr2=0x7fd8980579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:36.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.184+0000 7fd8b7805700 1 --2- 172.21.15.162:0/171377743 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd898055500 0x7fd8980579c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7fd8ac00b000 tx=0x7fd8ac008040).stop 2020-11-28T21:52:36.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.184+0000 7fd8b7805700 1 -- 172.21.15.162:0/171377743 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8b00ac240 msgr2=0x7fd8b0067280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:36.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.184+0000 7fd8b7805700 1 --2- 172.21.15.162:0/171377743 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8b00ac240 0x7fd8b0067280 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7fd8b00659f0 tx=0x7fd8a0007c00).stop 2020-11-28T21:52:36.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.184+0000 7fd8b7805700 1 -- 172.21.15.162:0/171377743 shutdown_connections 2020-11-28T21:52:36.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.184+0000 7fd8b7805700 1 --2- 172.21.15.162:0/171377743 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd898055500 0x7fd8980579c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:36.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.184+0000 7fd8b7805700 1 --2- 172.21.15.162:0/171377743 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd8b00b1200 0x7fd8b014d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:36.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.184+0000 7fd8b7805700 1 --2- 172.21.15.162:0/171377743 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd8b005ccc0 0x7fd8b0066d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:36.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.184+0000 7fd8b7805700 1 --2- 172.21.15.162:0/171377743 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8b00ac240 0x7fd8b0067280 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:36.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.184+0000 7fd8b7805700 1 -- 172.21.15.162:0/171377743 >> 172.21.15.162:0/171377743 conn(0x7fd8b00b6220 msgr2=0x7fd8b00b3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:36.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.184+0000 7fd8b7805700 1 -- 172.21.15.162:0/171377743 shutdown_connections 2020-11-28T21:52:36.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.184+0000 7fd8b7805700 1 -- 172.21.15.162:0/171377743 wait complete. 2020-11-28T21:52:36.205 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:52:36.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.324+0000 7f4612e9e700 1 Processor -- start 2020-11-28T21:52:36.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.324+0000 7f4612e9e700 1 -- start start 2020-11-28T21:52:36.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.328+0000 7f4612e9e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f460c140040 0x7f460c0ad620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:36.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.328+0000 7f4612e9e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f460c0adb10 0x7f460c0b1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:36.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.328+0000 7f4612e9e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f460c1404c0 con 0x7f460c0ad240 2020-11-28T21:52:36.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.328+0000 7f4612e9e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f460c0b1490 con 0x7f460c140040 2020-11-28T21:52:36.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.328+0000 7f4612e9e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f460c1431b0 con 0x7f460c0adb10 2020-11-28T21:52:36.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.328+0000 7f4611e9c700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f460c0ad240 0x7f460c142aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.162:38536/0 (socket says 172.21.15.162:38536) 2020-11-28T21:52:36.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.328+0000 7f4611e9c700 1 -- 172.21.15.162:0/10235983 learned_addr learned my addr 172.21.15.162:0/10235983 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:52:36.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.328+0000 7f461269d700 1 --2- 172.21.15.162:0/10235983 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f460c0adb10 0x7f460c0b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:36.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.328+0000 7f461169b700 1 --2- 172.21.15.162:0/10235983 >> v2:172.21.15.118:3300/0 conn(0x7f460c140040 0x7f460c0ad620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:36.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.328+0000 7f4610e9a700 1 -- 172.21.15.162:0/10235983 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2617609712 0 0) 0x7f46080010d0 con 0x7f460c0ad240 2020-11-28T21:52:36.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.328+0000 7f4610e9a700 1 -- 172.21.15.162:0/10235983 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45f4003230 con 0x7f460c0ad240 2020-11-28T21:52:36.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.328+0000 7f461269d700 1 -- 172.21.15.162:0/10235983 >> v2:172.21.15.118:3300/0 conn(0x7f460c140040 msgr2=0x7f460c0ad620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:36.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.328+0000 7f461269d700 1 --2- 172.21.15.162:0/10235983 >> v2:172.21.15.118:3300/0 conn(0x7f460c140040 0x7f460c0ad620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:36.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.328+0000 7f461269d700 1 -- 172.21.15.162:0/10235983 >> v1:172.21.15.118:6789/0 conn(0x7f460c0ad240 legacy=0x7f460c142aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:36.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.328+0000 7f461269d700 1 -- 172.21.15.162:0/10235983 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f460c0b1080 con 0x7f460c0adb10 2020-11-28T21:52:36.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.328+0000 7f461269d700 1 --2- 172.21.15.162:0/10235983 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f460c0adb10 0x7f460c0b1900 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f46040032b0 tx=0x7f46040091a0).ready entity=mon.2 client_cookie=b365c2bdee144a21 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:36.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.328+0000 7f4610e9a700 1 -- 172.21.15.162:0/10235983 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f460400b940 con 0x7f460c0adb10 2020-11-28T21:52:36.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.328+0000 7f4610e9a700 1 -- 172.21.15.162:0/10235983 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f460400baa0 con 0x7f460c0adb10 2020-11-28T21:52:36.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.328+0000 7f4610e9a700 1 -- 172.21.15.162:0/10235983 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f460400bc10 con 0x7f460c0adb10 2020-11-28T21:52:36.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.328+0000 7f4612e9e700 1 -- 172.21.15.162:0/10235983 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f460c0adb10 msgr2=0x7f460c0b1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:36.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.328+0000 7f4612e9e700 1 --2- 172.21.15.162:0/10235983 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f460c0adb10 0x7f460c0b1900 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f46040032b0 tx=0x7f46040091a0).stop 2020-11-28T21:52:36.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.332+0000 7f4612e9e700 1 -- 172.21.15.162:0/10235983 shutdown_connections 2020-11-28T21:52:36.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.332+0000 7f4612e9e700 1 --2- 172.21.15.162:0/10235983 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f460c0adb10 0x7f460c0b1900 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:36.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.332+0000 7f4612e9e700 1 --2- 172.21.15.162:0/10235983 >> v2:172.21.15.118:3300/0 conn(0x7f460c140040 0x7f460c0ad620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:36.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.332+0000 7f4612e9e700 1 -- 172.21.15.162:0/10235983 >> 172.21.15.162:0/10235983 conn(0x7f460c0b2830 msgr2=0x7f460c0b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:36.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.332+0000 7f4612e9e700 1 -- 172.21.15.162:0/10235983 shutdown_connections 2020-11-28T21:52:36.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.332+0000 7f4612e9e700 1 -- 172.21.15.162:0/10235983 wait complete. 2020-11-28T21:52:36.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.332+0000 7f4612e9e700 1 Processor -- start 2020-11-28T21:52:36.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.332+0000 7f4612e9e700 1 -- start start 2020-11-28T21:52:36.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.332+0000 7f4612e9e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f460c0ad240 0x7f460c066e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:36.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.332+0000 7f4612e9e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f460c0adb10 0x7f460c064840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:36.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.332+0000 7f4612e9e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f460c140040 0x7f460c064d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:36.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.332+0000 7f4612e9e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460c0643f0 con 0x7f460c140040 2020-11-28T21:52:36.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.332+0000 7f4612e9e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460c0b05e0 con 0x7f460c0adb10 2020-11-28T21:52:36.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.332+0000 7f4612e9e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f460c063af0 con 0x7f460c0ad240 2020-11-28T21:52:36.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.332+0000 7f461169b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f460c0adb10 0x7f460c064840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:36.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.332+0000 7f461169b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f460c0adb10 0x7f460c064840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41550/0 (socket says 172.21.15.162:41550) 2020-11-28T21:52:36.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.332+0000 7f461169b700 1 -- 172.21.15.162:0/1501232706 learned_addr learned my addr 172.21.15.162:0/1501232706 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:36.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.332+0000 7f4611e9c700 1 --2- 172.21.15.162:0/1501232706 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f460c0ad240 0x7f460c066e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:36.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.332+0000 7f461269d700 1 --2- 172.21.15.162:0/1501232706 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f460c140040 0x7f460c064d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:36.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.336+0000 7f461169b700 1 -- 172.21.15.162:0/1501232706 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f460c0ad240 msgr2=0x7f460c066e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:36.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.336+0000 7f461169b700 1 --2- 172.21.15.162:0/1501232706 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f460c0ad240 0x7f460c066e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:36.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.336+0000 7f461169b700 1 -- 172.21.15.162:0/1501232706 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f460c140040 msgr2=0x7f460c064d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:36.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.336+0000 7f461169b700 1 --2- 172.21.15.162:0/1501232706 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f460c140040 0x7f460c064d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:36.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.336+0000 7f461169b700 1 -- 172.21.15.162:0/1501232706 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f460c142510 con 0x7f460c0adb10 2020-11-28T21:52:36.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.336+0000 7f461169b700 1 --2- 172.21.15.162:0/1501232706 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f460c0adb10 0x7f460c064840 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7f45fc00c000 tx=0x7f45fc006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:36.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.336+0000 7f45faffd700 1 -- 172.21.15.162:0/1501232706 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45fc00ca10 con 0x7f460c0adb10 2020-11-28T21:52:36.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.336+0000 7f45faffd700 1 -- 172.21.15.162:0/1501232706 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45fc00cb70 con 0x7f460c0adb10 2020-11-28T21:52:36.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.336+0000 7f4612e9e700 1 -- 172.21.15.162:0/1501232706 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f460c142f50 con 0x7f460c0adb10 2020-11-28T21:52:36.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.336+0000 7f45faffd700 1 -- 172.21.15.162:0/1501232706 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45fc0123f0 con 0x7f460c0adb10 2020-11-28T21:52:36.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.336+0000 7f4612e9e700 1 -- 172.21.15.162:0/1501232706 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f460c064150 con 0x7f460c0adb10 2020-11-28T21:52:36.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.336+0000 7f4612e9e700 1 -- 172.21.15.162:0/1501232706 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f460c1503b0 con 0x7f460c0adb10 2020-11-28T21:52:36.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.336+0000 7f45faffd700 1 -- 172.21.15.162:0/1501232706 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f45fc012550 con 0x7f460c0adb10 2020-11-28T21:52:36.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.336+0000 7f45faffd700 1 --2- 172.21.15.162:0/1501232706 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f45ec055440 0x7f45ec057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:36.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.336+0000 7f45faffd700 1 -- 172.21.15.162:0/1501232706 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f45fc06e4c0 con 0x7f460c0adb10 2020-11-28T21:52:36.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.336+0000 7f4611e9c700 1 --2- 172.21.15.162:0/1501232706 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f45ec055440 0x7f45ec057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:36.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.336+0000 7f4611e9c700 1 --2- 172.21.15.162:0/1501232706 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f45ec055440 0x7f45ec057900 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7f460800b000 tx=0x7f4608009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:36.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.348+0000 7f45faffd700 1 -- 172.21.15.162:0/1501232706 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f45fc0446c0 con 0x7f460c0adb10 2020-11-28T21:52:36.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.488+0000 7f4612e9e700 1 -- 172.21.15.162:0/1501232706 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f460c057050 con 0x7f460c0adb10 2020-11-28T21:52:36.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.488+0000 7f45faffd700 1 -- 172.21.15.162:0/1501232706 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v76) v1 ==== 92+0+10 (secure 0 0 0) 0x7f45fc047ce0 con 0x7f460c0adb10 2020-11-28T21:52:36.489 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:52:36.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.488+0000 7f4612e9e700 1 -- 172.21.15.162:0/1501232706 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f45ec055440 msgr2=0x7f45ec057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:36.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.488+0000 7f4612e9e700 1 --2- 172.21.15.162:0/1501232706 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f45ec055440 0x7f45ec057900 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7f460800b000 tx=0x7f4608009040).stop 2020-11-28T21:52:36.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.488+0000 7f4612e9e700 1 -- 172.21.15.162:0/1501232706 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f460c0adb10 msgr2=0x7f460c064840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:36.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.488+0000 7f4612e9e700 1 --2- 172.21.15.162:0/1501232706 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f460c0adb10 0x7f460c064840 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7f45fc00c000 tx=0x7f45fc006dc0).stop 2020-11-28T21:52:36.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.488+0000 7f4612e9e700 1 -- 172.21.15.162:0/1501232706 shutdown_connections 2020-11-28T21:52:36.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.488+0000 7f4612e9e700 1 --2- 172.21.15.162:0/1501232706 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f45ec055440 0x7f45ec057900 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:36.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.488+0000 7f4612e9e700 1 --2- 172.21.15.162:0/1501232706 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f460c0ad240 0x7f460c066e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:36.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.488+0000 7f4612e9e700 1 --2- 172.21.15.162:0/1501232706 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f460c140040 0x7f460c064d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:36.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.488+0000 7f4612e9e700 1 --2- 172.21.15.162:0/1501232706 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f460c0adb10 0x7f460c064840 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:36.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.488+0000 7f4612e9e700 1 -- 172.21.15.162:0/1501232706 >> 172.21.15.162:0/1501232706 conn(0x7f460c0b2830 msgr2=0x7f460c0ab5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:36.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.488+0000 7f4612e9e700 1 -- 172.21.15.162:0/1501232706 shutdown_connections 2020-11-28T21:52:36.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.488+0000 7f4612e9e700 1 -- 172.21.15.162:0/1501232706 wait complete. 2020-11-28T21:52:36.506 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-28T21:52:36.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.620+0000 7fa46af4a700 1 Processor -- start 2020-11-28T21:52:36.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.620+0000 7fa46af4a700 1 -- start start 2020-11-28T21:52:36.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.624+0000 7fa46af4a700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa4640605c0 0x7fa464140390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:36.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.624+0000 7fa46af4a700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa464143950 con 0x7fa4640b9ce0 2020-11-28T21:52:36.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.624+0000 7fa46af4a700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4640ba0c0 con 0x7fa4641433c0 2020-11-28T21:52:36.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.624+0000 7fa46af4a700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4640b4340 con 0x7fa4640605c0 2020-11-28T21:52:36.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.624+0000 7fa463fff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fa4641433c0 0x7fa46413fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:48402/0 (socket says 172.21.15.162:48402) 2020-11-28T21:52:36.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.624+0000 7fa463fff700 1 -- 172.21.15.162:0/1010629316 learned_addr learned my addr 172.21.15.162:0/1010629316 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:52:36.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.624+0000 7fa4694e7700 1 --2- 172.21.15.162:0/1010629316 >> v2:172.21.15.162:3300/0 conn(0x7fa4640605c0 0x7fa464140390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:36.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.624+0000 7fa4637fe700 1 -- 172.21.15.162:0/1010629316 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1157612589 0 0) 0x7fa4640ba0c0 con 0x7fa4641433c0 2020-11-28T21:52:36.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.624+0000 7fa4637fe700 1 -- 172.21.15.162:0/1010629316 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa450003230 con 0x7fa4641433c0 2020-11-28T21:52:36.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.624+0000 7fa4637fe700 1 -- 172.21.15.162:0/1010629316 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2721519599 0 0) 0x7fa450003230 con 0x7fa4641433c0 2020-11-28T21:52:36.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.624+0000 7fa4694e7700 1 -- 172.21.15.162:0/1010629316 >> v1:172.21.15.162:6789/0 conn(0x7fa4641433c0 legacy=0x7fa46413fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:36.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.624+0000 7fa4694e7700 1 -- 172.21.15.162:0/1010629316 >> v1:172.21.15.118:6789/0 conn(0x7fa4640b9ce0 legacy=0x7fa4640b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:36.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.624+0000 7fa4694e7700 1 -- 172.21.15.162:0/1010629316 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa464066c30 con 0x7fa4640605c0 2020-11-28T21:52:36.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.624+0000 7fa4694e7700 1 --2- 172.21.15.162:0/1010629316 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4640605c0 0x7fa464140390 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0x7fa458001870 tx=0x7fa45c0091a0).ready entity=mon.1 client_cookie=90c464fb5874433d server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:36.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.624+0000 7fa4637fe700 1 -- 172.21.15.162:0/1010629316 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4580016d0 con 0x7fa4640605c0 2020-11-28T21:52:36.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.624+0000 7fa4637fe700 1 -- 172.21.15.162:0/1010629316 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa45c00b880 con 0x7fa4640605c0 2020-11-28T21:52:36.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.624+0000 7fa4637fe700 1 -- 172.21.15.162:0/1010629316 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa45c00b9f0 con 0x7fa4640605c0 2020-11-28T21:52:36.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.624+0000 7fa46af4a700 1 -- 172.21.15.162:0/1010629316 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4640605c0 msgr2=0x7fa464140390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:36.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.624+0000 7fa46af4a700 1 --2- 172.21.15.162:0/1010629316 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4640605c0 0x7fa464140390 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0x7fa458001870 tx=0x7fa45c0091a0).stop 2020-11-28T21:52:36.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.624+0000 7fa46af4a700 1 -- 172.21.15.162:0/1010629316 shutdown_connections 2020-11-28T21:52:36.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.624+0000 7fa46af4a700 1 --2- 172.21.15.162:0/1010629316 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4640605c0 0x7fa464140390 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:36.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.624+0000 7fa46af4a700 1 -- 172.21.15.162:0/1010629316 >> 172.21.15.162:0/1010629316 conn(0x7fa46405c430 msgr2=0x7fa4640b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:36.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.624+0000 7fa46af4a700 1 -- 172.21.15.162:0/1010629316 shutdown_connections 2020-11-28T21:52:36.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.624+0000 7fa46af4a700 1 -- 172.21.15.162:0/1010629316 wait complete. 2020-11-28T21:52:36.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.628+0000 7fa46af4a700 1 Processor -- start 2020-11-28T21:52:36.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.628+0000 7fa46af4a700 1 -- start start 2020-11-28T21:52:36.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.628+0000 7fa46af4a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4640605c0 0x7fa464144f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:36.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.628+0000 7fa46af4a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa4640b9ce0 0x7fa4640b5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:36.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.628+0000 7fa46af4a700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa4641433c0 0x7fa4640b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:36.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.628+0000 7fa46af4a700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa464144380 con 0x7fa4640b9ce0 2020-11-28T21:52:36.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.628+0000 7fa46af4a700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa464145620 con 0x7fa4640605c0 2020-11-28T21:52:36.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.628+0000 7fa46af4a700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4640b5f40 con 0x7fa4641433c0 2020-11-28T21:52:36.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.628+0000 7fa468ce6700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4640605c0 0x7fa464144f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:36.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.628+0000 7fa463fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa4640b9ce0 0x7fa4640b5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:36.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.628+0000 7fa4694e7700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa4641433c0 0x7fa4640b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:36.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.628+0000 7fa468ce6700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4640605c0 0x7fa464144f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41564/0 (socket says 172.21.15.162:41564) 2020-11-28T21:52:36.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.628+0000 7fa468ce6700 1 -- 172.21.15.162:0/2390271408 learned_addr learned my addr 172.21.15.162:0/2390271408 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:36.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.628+0000 7fa4694e7700 1 -- 172.21.15.162:0/2390271408 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4640605c0 msgr2=0x7fa464144f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:36.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.628+0000 7fa4694e7700 1 --2- 172.21.15.162:0/2390271408 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4640605c0 0x7fa464144f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:36.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.628+0000 7fa4694e7700 1 -- 172.21.15.162:0/2390271408 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa4640b9ce0 msgr2=0x7fa4640b5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:36.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.628+0000 7fa4694e7700 1 --2- 172.21.15.162:0/2390271408 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa4640b9ce0 0x7fa4640b5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:36.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.628+0000 7fa4694e7700 1 -- 172.21.15.162:0/2390271408 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa464152420 con 0x7fa4641433c0 2020-11-28T21:52:36.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.628+0000 7fa468ce6700 1 --2- 172.21.15.162:0/2390271408 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4640605c0 0x7fa464144f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:52:36.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.628+0000 7fa463fff700 1 --2- 172.21.15.162:0/2390271408 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa4640b9ce0 0x7fa4640b5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:52:36.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.628+0000 7fa4694e7700 1 --2- 172.21.15.162:0/2390271408 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa4641433c0 0x7fa4640b4d20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fa458001870 tx=0x7fa45c016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:36.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.628+0000 7fa461ffb700 1 -- 172.21.15.162:0/2390271408 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa45c006b30 con 0x7fa4641433c0 2020-11-28T21:52:36.639 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.632+0000 7fa461ffb700 1 -- 172.21.15.162:0/2390271408 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa45c011b80 con 0x7fa4641433c0 2020-11-28T21:52:36.639 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.632+0000 7fa46af4a700 1 -- 172.21.15.162:0/2390271408 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4641526c0 con 0x7fa4641433c0 2020-11-28T21:52:36.639 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.632+0000 7fa461ffb700 1 -- 172.21.15.162:0/2390271408 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa45c021440 con 0x7fa4641433c0 2020-11-28T21:52:36.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.632+0000 7fa46af4a700 1 -- 172.21.15.162:0/2390271408 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4640b53b0 con 0x7fa4641433c0 2020-11-28T21:52:36.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.632+0000 7fa46af4a700 1 -- 172.21.15.162:0/2390271408 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa464152280 con 0x7fa4641433c0 2020-11-28T21:52:36.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.632+0000 7fa461ffb700 1 -- 172.21.15.162:0/2390271408 <== mon.2 v2:172.21.15.118:3301/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa45c027020 con 0x7fa4641433c0 2020-11-28T21:52:36.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.632+0000 7fa461ffb700 1 --2- 172.21.15.162:0/2390271408 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa4440554e0 0x7fa4440579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:36.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.632+0000 7fa461ffb700 1 -- 172.21.15.162:0/2390271408 <== mon.2 v2:172.21.15.118:3301/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa45c06f040 con 0x7fa4641433c0 2020-11-28T21:52:36.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.632+0000 7fa468ce6700 1 --2- 172.21.15.162:0/2390271408 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa4440554e0 0x7fa4440579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:36.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.636+0000 7fa468ce6700 1 --2- 172.21.15.162:0/2390271408 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa4440554e0 0x7fa4440579a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7fa45400d000 tx=0x7fa454009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:36.643 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.644+0000 7fa461ffb700 1 -- 172.21.15.162:0/2390271408 <== mon.2 v2:172.21.15.118:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa45c06adf0 con 0x7fa4641433c0 2020-11-28T21:52:36.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.784+0000 7fa46af4a700 1 -- 172.21.15.162:0/2390271408 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa4640b6140 con 0x7fa4641433c0 2020-11-28T21:52:36.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.784+0000 7fa461ffb700 1 -- 172.21.15.162:0/2390271408 <== mon.2 v2:172.21.15.118:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v76) v1 ==== 104+0+10 (secure 0 0 0) 0x7fa45c0150a0 con 0x7fa4641433c0 2020-11-28T21:52:36.787 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:52:36.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.788+0000 7fa46af4a700 1 -- 172.21.15.162:0/2390271408 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa4440554e0 msgr2=0x7fa4440579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:36.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.788+0000 7fa46af4a700 1 --2- 172.21.15.162:0/2390271408 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa4440554e0 0x7fa4440579a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7fa45400d000 tx=0x7fa454009040).stop 2020-11-28T21:52:36.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.788+0000 7fa46af4a700 1 -- 172.21.15.162:0/2390271408 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa4641433c0 msgr2=0x7fa4640b4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:36.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.788+0000 7fa46af4a700 1 --2- 172.21.15.162:0/2390271408 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa4641433c0 0x7fa4640b4d20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fa458001870 tx=0x7fa45c016000).stop 2020-11-28T21:52:36.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.788+0000 7fa46af4a700 1 -- 172.21.15.162:0/2390271408 shutdown_connections 2020-11-28T21:52:36.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.788+0000 7fa46af4a700 1 --2- 172.21.15.162:0/2390271408 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa4440554e0 0x7fa4440579a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:36.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.788+0000 7fa46af4a700 1 --2- 172.21.15.162:0/2390271408 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa4641433c0 0x7fa4640b4d20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:36.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.788+0000 7fa46af4a700 1 --2- 172.21.15.162:0/2390271408 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4640605c0 0x7fa464144f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:36.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.788+0000 7fa46af4a700 1 --2- 172.21.15.162:0/2390271408 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa4640b9ce0 0x7fa4640b5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:36.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.788+0000 7fa46af4a700 1 -- 172.21.15.162:0/2390271408 >> 172.21.15.162:0/2390271408 conn(0x7fa46405c430 msgr2=0x7fa46405f750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:36.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.788+0000 7fa46af4a700 1 -- 172.21.15.162:0/2390271408 shutdown_connections 2020-11-28T21:52:36.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.788+0000 7fa46af4a700 1 -- 172.21.15.162:0/2390271408 wait complete. 2020-11-28T21:52:36.801 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-28T21:52:36.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.916+0000 7ff61bfff700 1 Processor -- start 2020-11-28T21:52:36.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.916+0000 7ff61bfff700 1 -- start start 2020-11-28T21:52:36.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.920+0000 7ff61bfff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff61c0b2d00 0x7ff61c05c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:36.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.920+0000 7ff61bfff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff61c0b3ab0 0x7ff61c0b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:36.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.920+0000 7ff61bfff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff61c142ad0 0x7ff61c142f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:36.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.920+0000 7ff61bfff700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff61c0b0e20 con 0x7ff61c0b2d00 2020-11-28T21:52:36.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.920+0000 7ff61bfff700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff61c05baf0 con 0x7ff61c142ad0 2020-11-28T21:52:36.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.920+0000 7ff61bfff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c05bd90 con 0x7ff61c0b3ab0 2020-11-28T21:52:36.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.920+0000 7ff61b7fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff61c142ad0 0x7ff61c142f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:36.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.920+0000 7ff61b7fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff61c142ad0 0x7ff61c142f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41576/0 (socket says 172.21.15.162:41576) 2020-11-28T21:52:36.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.920+0000 7ff61b7fe700 1 -- 172.21.15.162:0/1538564814 learned_addr learned my addr 172.21.15.162:0/1538564814 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:36.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.920+0000 7ff61a7fc700 1 --2- 172.21.15.162:0/1538564814 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff61c0b3ab0 0x7ff61c0b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:36.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.920+0000 7ff61affd700 1 --2- 172.21.15.162:0/1538564814 >> v2:172.21.15.118:3300/0 conn(0x7ff61c0b2d00 0x7ff61c05c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:36.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.920+0000 7ff61b7fe700 1 -- 172.21.15.162:0/1538564814 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff61c0b3ab0 msgr2=0x7ff61c0b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:36.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.920+0000 7ff61b7fe700 1 --2- 172.21.15.162:0/1538564814 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff61c0b3ab0 0x7ff61c0b30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:36.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.920+0000 7ff61b7fe700 1 -- 172.21.15.162:0/1538564814 >> v2:172.21.15.118:3300/0 conn(0x7ff61c0b2d00 msgr2=0x7ff61c05c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:36.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.920+0000 7ff61b7fe700 1 --2- 172.21.15.162:0/1538564814 >> v2:172.21.15.118:3300/0 conn(0x7ff61c0b2d00 0x7ff61c05c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:36.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.920+0000 7ff61b7fe700 1 -- 172.21.15.162:0/1538564814 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff61c05c030 con 0x7ff61c142ad0 2020-11-28T21:52:36.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.920+0000 7ff61b7fe700 1 --2- 172.21.15.162:0/1538564814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff61c142ad0 0x7ff61c142f60 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7ff614002f30 tx=0x7ff614006e80).ready entity=mon.1 client_cookie=1c76a847f269dba9 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:36.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.920+0000 7ff619ffb700 1 -- 172.21.15.162:0/1538564814 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6140097f0 con 0x7ff61c142ad0 2020-11-28T21:52:36.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.920+0000 7ff619ffb700 1 -- 172.21.15.162:0/1538564814 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff614009950 con 0x7ff61c142ad0 2020-11-28T21:52:36.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.920+0000 7ff619ffb700 1 -- 172.21.15.162:0/1538564814 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff614009cb0 con 0x7ff61c142ad0 2020-11-28T21:52:36.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.920+0000 7ff61bfff700 1 -- 172.21.15.162:0/1538564814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff61c142ad0 msgr2=0x7ff61c142f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:36.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.920+0000 7ff61bfff700 1 --2- 172.21.15.162:0/1538564814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff61c142ad0 0x7ff61c142f60 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7ff614002f30 tx=0x7ff614006e80).stop 2020-11-28T21:52:36.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.920+0000 7ff61bfff700 1 -- 172.21.15.162:0/1538564814 shutdown_connections 2020-11-28T21:52:36.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.920+0000 7ff61bfff700 1 --2- 172.21.15.162:0/1538564814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff61c142ad0 0x7ff61c142f60 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:36.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.920+0000 7ff61bfff700 1 --2- 172.21.15.162:0/1538564814 >> v2:172.21.15.118:3300/0 conn(0x7ff61c0b2d00 0x7ff61c05c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:36.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.920+0000 7ff61bfff700 1 --2- 172.21.15.162:0/1538564814 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff61c0b3ab0 0x7ff61c0b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:36.931 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.920+0000 7ff61bfff700 1 -- 172.21.15.162:0/1538564814 >> 172.21.15.162:0/1538564814 conn(0x7ff61c0604c0 msgr2=0x7ff61c0608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:36.931 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.920+0000 7ff61bfff700 1 -- 172.21.15.162:0/1538564814 shutdown_connections 2020-11-28T21:52:36.931 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.920+0000 7ff61bfff700 1 -- 172.21.15.162:0/1538564814 wait complete. 2020-11-28T21:52:36.931 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.924+0000 7ff61bfff700 1 Processor -- start 2020-11-28T21:52:36.931 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.924+0000 7ff61bfff700 1 -- start start 2020-11-28T21:52:36.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.924+0000 7ff61bfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff61c0b2d00 0x7ff61c0b9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:36.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.924+0000 7ff61bfff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff61c0b3ab0 0x7ff61c149c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:36.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.924+0000 7ff61bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff61c142ad0 0x7ff61c148d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:36.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.924+0000 7ff61bfff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c1493e0 con 0x7ff61c142ad0 2020-11-28T21:52:36.933 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.924+0000 7ff61bfff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c149610 con 0x7ff61c0b2d00 2020-11-28T21:52:36.933 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.924+0000 7ff61bfff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c1498b0 con 0x7ff61c0b3ab0 2020-11-28T21:52:36.933 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.924+0000 7ff61affd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff61c0b2d00 0x7ff61c0b9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:36.933 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.924+0000 7ff61affd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff61c0b2d00 0x7ff61c0b9bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41578/0 (socket says 172.21.15.162:41578) 2020-11-28T21:52:36.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.924+0000 7ff61affd700 1 -- 172.21.15.162:0/3053586157 learned_addr learned my addr 172.21.15.162:0/3053586157 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:36.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.924+0000 7ff61a7fc700 1 --2- 172.21.15.162:0/3053586157 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff61c0b3ab0 0x7ff61c149c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:36.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.924+0000 7ff61b7fe700 1 --2- 172.21.15.162:0/3053586157 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff61c142ad0 0x7ff61c148d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:36.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.924+0000 7ff61affd700 1 -- 172.21.15.162:0/3053586157 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff61c0b3ab0 msgr2=0x7ff61c149c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:36.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.924+0000 7ff61affd700 1 --2- 172.21.15.162:0/3053586157 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff61c0b3ab0 0x7ff61c149c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:36.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.924+0000 7ff61affd700 1 -- 172.21.15.162:0/3053586157 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff61c142ad0 msgr2=0x7ff61c148d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:36.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.924+0000 7ff61affd700 1 --2- 172.21.15.162:0/3053586157 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff61c142ad0 0x7ff61c148d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:36.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.924+0000 7ff61affd700 1 -- 172.21.15.162:0/3053586157 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff61c0b4eb0 con 0x7ff61c0b2d00 2020-11-28T21:52:36.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.924+0000 7ff61affd700 1 --2- 172.21.15.162:0/3053586157 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff61c0b2d00 0x7ff61c0b9bc0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0x7ff61c0b8a70 tx=0x7ff610008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:36.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.924+0000 7ff60bfff700 1 -- 172.21.15.162:0/3053586157 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff610009790 con 0x7ff61c0b2d00 2020-11-28T21:52:36.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.924+0000 7ff61bfff700 1 -- 172.21.15.162:0/3053586157 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff61c0b5150 con 0x7ff61c0b2d00 2020-11-28T21:52:36.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.924+0000 7ff60bfff700 1 -- 172.21.15.162:0/3053586157 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6100098f0 con 0x7ff61c0b2d00 2020-11-28T21:52:36.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.924+0000 7ff60bfff700 1 -- 172.21.15.162:0/3053586157 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff610009c50 con 0x7ff61c0b2d00 2020-11-28T21:52:36.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.924+0000 7ff61bfff700 1 -- 172.21.15.162:0/3053586157 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff61c0b5810 con 0x7ff61c0b2d00 2020-11-28T21:52:36.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.928+0000 7ff61bfff700 1 -- 172.21.15.162:0/3053586157 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff61c057050 con 0x7ff61c0b2d00 2020-11-28T21:52:36.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.928+0000 7ff60bfff700 1 -- 172.21.15.162:0/3053586157 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff610013480 con 0x7ff61c0b2d00 2020-11-28T21:52:36.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.928+0000 7ff60bfff700 1 --2- 172.21.15.162:0/3053586157 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff5f8055440 0x7ff5f8057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:36.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.928+0000 7ff60bfff700 1 -- 172.21.15.162:0/3053586157 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff61006e3b0 con 0x7ff61c0b2d00 2020-11-28T21:52:36.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.928+0000 7ff61a7fc700 1 --2- 172.21.15.162:0/3053586157 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff5f8055440 0x7ff5f8057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:36.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.928+0000 7ff61a7fc700 1 --2- 172.21.15.162:0/3053586157 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff5f8055440 0x7ff5f8057900 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7ff60c00c000 tx=0x7ff60c008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:36.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:36.932+0000 7ff60bfff700 1 -- 172.21.15.162:0/3053586157 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff6100445b0 con 0x7ff61c0b2d00 2020-11-28T21:52:37.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.084+0000 7ff61bfff700 1 -- 172.21.15.162:0/3053586157 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff61c143cb0 con 0x7ff61c0b2d00 2020-11-28T21:52:37.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.084+0000 7ff60bfff700 1 -- 172.21.15.162:0/3053586157 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v76) v1 ==== 100+0+10 (secure 0 0 0) 0x7ff610047bd0 con 0x7ff61c0b2d00 2020-11-28T21:52:37.085 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:52:37.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.084+0000 7ff61bfff700 1 -- 172.21.15.162:0/3053586157 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff5f8055440 msgr2=0x7ff5f8057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:37.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.084+0000 7ff61bfff700 1 --2- 172.21.15.162:0/3053586157 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff5f8055440 0x7ff5f8057900 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7ff60c00c000 tx=0x7ff60c008040).stop 2020-11-28T21:52:37.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.084+0000 7ff61bfff700 1 -- 172.21.15.162:0/3053586157 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff61c0b2d00 msgr2=0x7ff61c0b9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:37.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.084+0000 7ff61bfff700 1 --2- 172.21.15.162:0/3053586157 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff61c0b2d00 0x7ff61c0b9bc0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0x7ff61c0b8a70 tx=0x7ff610008040).stop 2020-11-28T21:52:37.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.084+0000 7ff61bfff700 1 -- 172.21.15.162:0/3053586157 shutdown_connections 2020-11-28T21:52:37.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.084+0000 7ff61bfff700 1 --2- 172.21.15.162:0/3053586157 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff5f8055440 0x7ff5f8057900 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:37.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.084+0000 7ff61bfff700 1 --2- 172.21.15.162:0/3053586157 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff61c142ad0 0x7ff61c148d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:37.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.084+0000 7ff61bfff700 1 --2- 172.21.15.162:0/3053586157 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff61c0b2d00 0x7ff61c0b9bc0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:37.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.084+0000 7ff61bfff700 1 --2- 172.21.15.162:0/3053586157 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff61c0b3ab0 0x7ff61c149c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:37.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.084+0000 7ff61bfff700 1 -- 172.21.15.162:0/3053586157 >> 172.21.15.162:0/3053586157 conn(0x7ff61c0604c0 msgr2=0x7ff61c05c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:37.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.084+0000 7ff61bfff700 1 -- 172.21.15.162:0/3053586157 shutdown_connections 2020-11-28T21:52:37.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.084+0000 7ff61bfff700 1 -- 172.21.15.162:0/3053586157 wait complete. 2020-11-28T21:52:37.105 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-28T21:52:37.106 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:52:37.217 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.216+0000 7f1edeb66700 1 Processor -- start 2020-11-28T21:52:37.218 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.216+0000 7f1edeb66700 1 -- start start 2020-11-28T21:52:37.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.216+0000 7f1edeb66700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1ed80ac790 0x7f1ed8142bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:37.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.216+0000 7f1edeb66700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1ed80acca0 0x7f1ed80b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:37.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.216+0000 7f1edeb66700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ed80b07e0 con 0x7f1ed80ac3b0 2020-11-28T21:52:37.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.216+0000 7f1edeb66700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ed80abe90 con 0x7f1ed80acca0 2020-11-28T21:52:37.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.216+0000 7f1edeb66700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ed80b0a80 con 0x7f1ed80ac790 2020-11-28T21:52:37.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.216+0000 7f1edd363700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1ed80ac790 0x7f1ed8142bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:37.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.216+0000 7f1eddb64700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f1ed80ac3b0 0x7f1ed80ab780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.162:38578/0 (socket says 172.21.15.162:38578) 2020-11-28T21:52:37.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.216+0000 7f1eddb64700 1 -- 172.21.15.162:0/505654197 learned_addr learned my addr 172.21.15.162:0/505654197 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:52:37.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.216+0000 7f1ede365700 1 --2- 172.21.15.162:0/505654197 >> v2:172.21.15.118:3300/0 conn(0x7f1ed80acca0 0x7f1ed80b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:37.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.216+0000 7f1edd363700 1 -- 172.21.15.162:0/505654197 >> v2:172.21.15.118:3300/0 conn(0x7f1ed80acca0 msgr2=0x7f1ed80b2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:37.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.216+0000 7f1edd363700 1 --2- 172.21.15.162:0/505654197 >> v2:172.21.15.118:3300/0 conn(0x7f1ed80acca0 0x7f1ed80b2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:37.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.216+0000 7f1edcb62700 1 -- 172.21.15.162:0/505654197 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4271990068 0 0) 0x7f1ed80b07e0 con 0x7f1ed80ac3b0 2020-11-28T21:52:37.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.216+0000 7f1edd363700 1 -- 172.21.15.162:0/505654197 >> v1:172.21.15.118:6789/0 conn(0x7f1ed80ac3b0 legacy=0x7f1ed80ab780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:37.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.216+0000 7f1edd363700 1 -- 172.21.15.162:0/505654197 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed80b0d20 con 0x7f1ed80ac790 2020-11-28T21:52:37.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.216+0000 7f1edd363700 1 --2- 172.21.15.162:0/505654197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ed80ac790 0x7f1ed8142bb0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7f1ec8002f30 tx=0x7f1ec8006dc0).ready entity=mon.1 client_cookie=967a9a043c4338b3 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:37.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.220+0000 7f1edcb62700 1 -- 172.21.15.162:0/505654197 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ec80097b0 con 0x7f1ed80ac790 2020-11-28T21:52:37.224 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.220+0000 7f1edcb62700 1 -- 172.21.15.162:0/505654197 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ec8009910 con 0x7f1ed80ac790 2020-11-28T21:52:37.224 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.220+0000 7f1edcb62700 1 -- 172.21.15.162:0/505654197 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ec8009c70 con 0x7f1ed80ac790 2020-11-28T21:52:37.224 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.220+0000 7f1edeb66700 1 -- 172.21.15.162:0/505654197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ed80ac790 msgr2=0x7f1ed8142bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:37.224 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.220+0000 7f1edeb66700 1 --2- 172.21.15.162:0/505654197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ed80ac790 0x7f1ed8142bb0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7f1ec8002f30 tx=0x7f1ec8006dc0).stop 2020-11-28T21:52:37.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.220+0000 7f1edeb66700 1 -- 172.21.15.162:0/505654197 shutdown_connections 2020-11-28T21:52:37.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.220+0000 7f1edeb66700 1 --2- 172.21.15.162:0/505654197 >> v2:172.21.15.118:3300/0 conn(0x7f1ed80acca0 0x7f1ed80b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:37.229 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.220+0000 7f1edeb66700 1 --2- 172.21.15.162:0/505654197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ed80ac790 0x7f1ed8142bb0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:37.229 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.220+0000 7f1edeb66700 1 -- 172.21.15.162:0/505654197 >> 172.21.15.162:0/505654197 conn(0x7f1ed80b6870 msgr2=0x7f1ed80b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:37.229 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.220+0000 7f1edeb66700 1 -- 172.21.15.162:0/505654197 shutdown_connections 2020-11-28T21:52:37.229 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.220+0000 7f1edeb66700 1 -- 172.21.15.162:0/505654197 wait complete. 2020-11-28T21:52:37.229 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.220+0000 7f1edeb66700 1 Processor -- start 2020-11-28T21:52:37.230 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.220+0000 7f1edeb66700 1 -- start start 2020-11-28T21:52:37.230 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.220+0000 7f1edeb66700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ed80ac3b0 0x7f1ed80af260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:37.230 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.220+0000 7f1edeb66700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1ed80ac790 0x7f1ed8065ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:37.230 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.220+0000 7f1edeb66700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ed80acca0 0x7f1ed80661d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:37.231 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.220+0000 7f1edeb66700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed81400c0 con 0x7f1ed80ac790 2020-11-28T21:52:37.231 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.220+0000 7f1edeb66700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed8143c30 con 0x7f1ed80acca0 2020-11-28T21:52:37.231 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.220+0000 7f1edeb66700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed8067750 con 0x7f1ed80ac3b0 2020-11-28T21:52:37.231 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.220+0000 7f1ede365700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ed80acca0 0x7f1ed80661d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:37.231 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.220+0000 7f1ede365700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ed80acca0 0x7f1ed80661d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41596/0 (socket says 172.21.15.162:41596) 2020-11-28T21:52:37.232 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.220+0000 7f1ede365700 1 -- 172.21.15.162:0/1380848962 learned_addr learned my addr 172.21.15.162:0/1380848962 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:37.232 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.220+0000 7f1eddb64700 1 --2- 172.21.15.162:0/1380848962 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ed80ac3b0 0x7f1ed80af260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:37.232 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.220+0000 7f1edd363700 1 --2- 172.21.15.162:0/1380848962 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1ed80ac790 0x7f1ed8065ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:37.232 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.224+0000 7f1ede365700 1 -- 172.21.15.162:0/1380848962 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ed80ac3b0 msgr2=0x7f1ed80af260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:37.233 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.224+0000 7f1ede365700 1 --2- 172.21.15.162:0/1380848962 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ed80ac3b0 0x7f1ed80af260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:37.233 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.224+0000 7f1ede365700 1 -- 172.21.15.162:0/1380848962 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1ed80ac790 msgr2=0x7f1ed8065ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:37.233 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.224+0000 7f1ede365700 1 --2- 172.21.15.162:0/1380848962 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1ed80ac790 0x7f1ed8065ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:37.233 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.224+0000 7f1ede365700 1 -- 172.21.15.162:0/1380848962 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed81402f0 con 0x7f1ed80acca0 2020-11-28T21:52:37.233 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.224+0000 7f1ede365700 1 --2- 172.21.15.162:0/1380848962 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ed80acca0 0x7f1ed80661d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 rx=0x7f1ed4006dd0 tx=0x7f1ed40091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:37.234 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.224+0000 7f1ec6ffd700 1 -- 172.21.15.162:0/1380848962 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ed400c8f0 con 0x7f1ed80acca0 2020-11-28T21:52:37.234 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.224+0000 7f1ec6ffd700 1 -- 172.21.15.162:0/1380848962 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed400ca50 con 0x7f1ed80acca0 2020-11-28T21:52:37.234 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.224+0000 7f1ec6ffd700 1 -- 172.21.15.162:0/1380848962 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ed400cdb0 con 0x7f1ed80acca0 2020-11-28T21:52:37.234 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.224+0000 7f1edeb66700 1 -- 172.21.15.162:0/1380848962 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ed8144560 con 0x7f1ed80acca0 2020-11-28T21:52:37.235 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.224+0000 7f1edeb66700 1 -- 172.21.15.162:0/1380848962 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ed80b0000 con 0x7f1ed80acca0 2020-11-28T21:52:37.235 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.224+0000 7f1edeb66700 1 -- 172.21.15.162:0/1380848962 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ed8144230 con 0x7f1ed80acca0 2020-11-28T21:52:37.235 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.224+0000 7f1ec6ffd700 1 -- 172.21.15.162:0/1380848962 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1ed4022070 con 0x7f1ed80acca0 2020-11-28T21:52:37.235 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.224+0000 7f1ec6ffd700 1 --2- 172.21.15.162:0/1380848962 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1eb8055350 0x7f1eb8057810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:37.235 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.224+0000 7f1ec6ffd700 1 -- 172.21.15.162:0/1380848962 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1ed406e4d0 con 0x7f1ed80acca0 2020-11-28T21:52:37.236 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.224+0000 7f1eddb64700 1 --2- 172.21.15.162:0/1380848962 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1eb8055350 0x7f1eb8057810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:37.236 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.224+0000 7f1eddb64700 1 --2- 172.21.15.162:0/1380848962 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1eb8055350 0x7f1eb8057810 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f1ed000d000 tx=0x7f1ed0001160).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:37.236 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.228+0000 7f1ec6ffd700 1 -- 172.21.15.162:0/1380848962 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1ed40447d0 con 0x7f1ed80acca0 2020-11-28T21:52:37.416 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.416+0000 7f1edeb66700 1 -- 172.21.15.162:0/1380848962 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ed8143dc0 con 0x7f1ed80acca0 2020-11-28T21:52:37.417 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.416+0000 7f1ec6ffd700 1 -- 172.21.15.162:0/1380848962 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+2499 (secure 0 0 0) 0x7f1ed4016070 con 0x7f1ed80acca0 2020-11-28T21:52:37.417 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:52:37.417 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":33,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5540,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2116325247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2116325247},{"type":"v1","addr":"172.21.15.162:6835","nonce":2116325247}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":30},{"gid":5543,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.118:6833/3491069258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3491069258},{"type":"v1","addr":"172.21.15.118:6833","nonce":3491069258}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:52:28.454745+0000","modified":"2020-11-28T21:52:29.457610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6172},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6172":{"gid":6172,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/163256983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":163256983},{"type":"v1","addr":"172.21.15.162:6837","nonce":163256983}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-28T21:52:37.419 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.416+0000 7f1edeb66700 1 -- 172.21.15.162:0/1380848962 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1eb8055350 msgr2=0x7f1eb8057810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:37.419 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.416+0000 7f1edeb66700 1 --2- 172.21.15.162:0/1380848962 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1eb8055350 0x7f1eb8057810 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f1ed000d000 tx=0x7f1ed0001160).stop 2020-11-28T21:52:37.419 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.416+0000 7f1edeb66700 1 -- 172.21.15.162:0/1380848962 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ed80acca0 msgr2=0x7f1ed80661d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:37.420 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.416+0000 7f1edeb66700 1 --2- 172.21.15.162:0/1380848962 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ed80acca0 0x7f1ed80661d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 rx=0x7f1ed4006dd0 tx=0x7f1ed40091a0).stop 2020-11-28T21:52:37.420 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.416+0000 7f1edeb66700 1 -- 172.21.15.162:0/1380848962 shutdown_connections 2020-11-28T21:52:37.420 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.416+0000 7f1edeb66700 1 --2- 172.21.15.162:0/1380848962 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1eb8055350 0x7f1eb8057810 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:37.420 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.416+0000 7f1edeb66700 1 --2- 172.21.15.162:0/1380848962 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ed80ac3b0 0x7f1ed80af260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:37.420 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.416+0000 7f1edeb66700 1 --2- 172.21.15.162:0/1380848962 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ed80acca0 0x7f1ed80661d0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:37.421 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.416+0000 7f1edeb66700 1 --2- 172.21.15.162:0/1380848962 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1ed80ac790 0x7f1ed8065ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:37.421 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.416+0000 7f1edeb66700 1 -- 172.21.15.162:0/1380848962 >> 172.21.15.162:0/1380848962 conn(0x7f1ed80b6870 msgr2=0x7f1ed80b3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:37.421 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.416+0000 7f1edeb66700 1 -- 172.21.15.162:0/1380848962 shutdown_connections 2020-11-28T21:52:37.421 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:37.416+0000 7f1edeb66700 1 -- 172.21.15.162:0/1380848962 wait complete. 2020-11-28T21:52:37.422 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 33 2020-11-28T21:52:37.732 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-28T21:52:28.454745+0000', 'modified': '2020-11-28T21:52:29.457610+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6172}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6172': {'gid': 6172, 'name': 'a', 'rank': 0, 'incarnation': 32, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6837/163256983', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6836', 'nonce': 163256983}, {'type': 'v1', 'addr': '172.21.15.162:6837', 'nonce': 163256983}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-28T21:52:37.733 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-28T21:52:37.734 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-28T21:52:37.861 INFO:teuthology.orchestra.run.smithi162.stdout:{"cluster_fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","whoami":0,"id":6172,"want_state":"up:active","state":"up:active","rank_uptime":9.3939162950000004,"mdsmap_epoch":33,"osdmap_epoch":76,"osdmap_epoch_barrier":75,"uptime":14.821399940999999} 2020-11-28T21:52:37.862 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", "id": 6172, "mdsmap_epoch": 33, "osdmap_epoch": 76, "osdmap_epoch_barrier": 75, "rank_uptime": 9.393916295, "state": "up:active", "uptime": 14.821399941, "want_state": "up:active", "whoami": 0 } 2020-11-28T21:52:37.862 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-11-28T21:52:37.863 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2020-11-28T21:52:37.863 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi096.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2020-11-28T21:52:37.863 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:52:37.864 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:52:37.871 INFO:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2020-11-28T21:52:37.925 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:52:37.926 INFO:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:52:37.983 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:52:37.984 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:52:37.984 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:52:37.985 INFO:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2020-11-28T21:52:38.030 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-11-28T21:52:38.031 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:52:38.074 INFO:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2020-11-28T21:52:38.091 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:52:38.091 INFO:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:52:38.143 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:52:38.147 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:52:38.149 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:52:38.150 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:52:38.150 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:52:38.151 INFO:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2020-11-28T21:52:38.152 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:2020-11-28T21:52:38.148+0000 7fbbacf160c0 -1 init, newargv = 0x55aef6f012d0 newargc=9 2020-11-28T21:52:38.153 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[11536]: starting ceph client 2020-11-28T21:52:38.153 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:52:38.156 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:52:38.162 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[11536]: starting fuse 2020-11-28T21:52:39.259 INFO:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2020-11-28T21:52:39.276 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:52:39.277 INFO:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:52:39.336 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:52:39.337 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:52:39.337 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:52:39.338 INFO:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2020-11-28T21:52:39.387 INFO:teuthology.orchestra.run.smithi096.stdout:49 2020-11-28T21:52:39.387 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2020-11-28T21:52:39.388 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:52:39.388 INFO:teuthology.orchestra.run.smithi096:> sudo python3 -c ' 2020-11-28T21:52:39.389 INFO:teuthology.orchestra.run.smithi096:> import glob 2020-11-28T21:52:39.389 INFO:teuthology.orchestra.run.smithi096:> import re 2020-11-28T21:52:39.390 INFO:teuthology.orchestra.run.smithi096:> import os 2020-11-28T21:52:39.391 INFO:teuthology.orchestra.run.smithi096:> import subprocess 2020-11-28T21:52:39.391 INFO:teuthology.orchestra.run.smithi096:> 2020-11-28T21:52:39.392 INFO:teuthology.orchestra.run.smithi096:> def find_socket(client_name): 2020-11-28T21:52:39.392 INFO:teuthology.orchestra.run.smithi096:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2020-11-28T21:52:39.393 INFO:teuthology.orchestra.run.smithi096:> files = glob.glob(asok_path) 2020-11-28T21:52:39.394 INFO:teuthology.orchestra.run.smithi096:> 2020-11-28T21:52:39.394 INFO:teuthology.orchestra.run.smithi096:> # Given a non-glob path, it better be there 2020-11-28T21:52:39.394 INFO:teuthology.orchestra.run.smithi096:> if "*" not in asok_path: 2020-11-28T21:52:39.395 INFO:teuthology.orchestra.run.smithi096:> assert(len(files) == 1) 2020-11-28T21:52:39.395 INFO:teuthology.orchestra.run.smithi096:> return files[0] 2020-11-28T21:52:39.395 INFO:teuthology.orchestra.run.smithi096:> 2020-11-28T21:52:39.396 INFO:teuthology.orchestra.run.smithi096:> for f in files: 2020-11-28T21:52:39.396 INFO:teuthology.orchestra.run.smithi096:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-11-28T21:52:39.397 INFO:teuthology.orchestra.run.smithi096:> if os.path.exists("/proc/{0}".format(pid)): 2020-11-28T21:52:39.397 INFO:teuthology.orchestra.run.smithi096:> return f 2020-11-28T21:52:39.398 INFO:teuthology.orchestra.run.smithi096:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-11-28T21:52:39.399 INFO:teuthology.orchestra.run.smithi096:> 2020-11-28T21:52:39.400 INFO:teuthology.orchestra.run.smithi096:> print(find_socket("client.1")) 2020-11-28T21:52:39.401 INFO:teuthology.orchestra.run.smithi096:> ' 2020-11-28T21:52:39.479 INFO:teuthology.orchestra.run.smithi096.stdout:/var/run/ceph/ceph-client.1.11536.asok 2020-11-28T21:52:39.484 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.11536.asok 2020-11-28T21:52:39.484 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:52:39.485 INFO:teuthology.orchestra.run.smithi096:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.11536.asok status 2020-11-28T21:52:39.585 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-11-28T21:52:39.586 INFO:teuthology.orchestra.run.smithi096.stdout: "metadata": { 2020-11-28T21:52:39.586 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_sha1": "a83a3572a957356977342f461361c1cd3ec06dbd", 2020-11-28T21:52:39.586 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_version": "ceph version 15.2.6-595-ga83a3572 (a83a3572a957356977342f461361c1cd3ec06dbd) octopus (stable)", 2020-11-28T21:52:39.586 INFO:teuthology.orchestra.run.smithi096.stdout: "entity_id": "1", 2020-11-28T21:52:39.587 INFO:teuthology.orchestra.run.smithi096.stdout: "hostname": "smithi096", 2020-11-28T21:52:39.587 INFO:teuthology.orchestra.run.smithi096.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2020-11-28T21:52:39.587 INFO:teuthology.orchestra.run.smithi096.stdout: "pid": "11536", 2020-11-28T21:52:39.588 INFO:teuthology.orchestra.run.smithi096.stdout: "root": "/" 2020-11-28T21:52:39.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-11-28T21:52:39.588 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_count": 0, 2020-11-28T21:52:39.589 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_pinned_count": 0, 2020-11-28T21:52:39.589 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 6216, 2020-11-28T21:52:39.589 INFO:teuthology.orchestra.run.smithi096.stdout: "inst": { 2020-11-28T21:52:39.589 INFO:teuthology.orchestra.run.smithi096.stdout: "name": { 2020-11-28T21:52:39.590 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "client", 2020-11-28T21:52:39.590 INFO:teuthology.orchestra.run.smithi096.stdout: "num": 6216 2020-11-28T21:52:39.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-11-28T21:52:39.591 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2020-11-28T21:52:39.591 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2020-11-28T21:52:39.591 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2020-11-28T21:52:39.592 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2029121045 2020-11-28T21:52:39.592 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-11-28T21:52:39.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-11-28T21:52:39.592 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2020-11-28T21:52:39.592 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2020-11-28T21:52:39.593 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2020-11-28T21:52:39.593 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2029121045 2020-11-28T21:52:39.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-11-28T21:52:39.593 INFO:teuthology.orchestra.run.smithi096.stdout: "inst_str": "client.6216 172.21.15.96:0/2029121045", 2020-11-28T21:52:39.593 INFO:teuthology.orchestra.run.smithi096.stdout: "addr_str": "172.21.15.96:0/2029121045", 2020-11-28T21:52:39.594 INFO:teuthology.orchestra.run.smithi096.stdout: "inode_count": 1, 2020-11-28T21:52:39.594 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_epoch": 33, 2020-11-28T21:52:39.594 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch": 76, 2020-11-28T21:52:39.594 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch_barrier": 0, 2020-11-28T21:52:39.595 INFO:teuthology.orchestra.run.smithi096.stdout: "blacklisted": false 2020-11-28T21:52:39.595 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-11-28T21:52:39.596 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:52:39.596 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:52:39.597 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:52:39.604 INFO:teuthology.orchestra.run.smithi096.stdout:fuseblk 2020-11-28T21:52:39.605 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-11-28T21:52:39.606 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:52:39.606 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:52:39.667 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:52:39.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.784+0000 7fdeacc1d700 1 Processor -- start 2020-11-28T21:52:39.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.784+0000 7fdeacc1d700 1 -- start start 2020-11-28T21:52:39.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.784+0000 7fdeacc1d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fdea8098b30 0x7fdea80994f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:39.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.784+0000 7fdeacc1d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fdea80b0000 0x7fdea8060000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:39.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.784+0000 7fdeacc1d700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdea8098f10 con 0x7fdea80604f0 2020-11-28T21:52:39.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.784+0000 7fdeacc1d700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdea80b1030 con 0x7fdea80b0000 2020-11-28T21:52:39.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.784+0000 7fdeacc1d700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdea80b0480 con 0x7fdea8098b30 2020-11-28T21:52:39.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.784+0000 7fdea77fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fdea8098b30 0x7fdea80994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:39.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.784+0000 7fdea77fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fdea8098b30 0x7fdea80994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41600/0 (socket says 172.21.15.162:41600) 2020-11-28T21:52:39.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.784+0000 7fdea77fe700 1 -- 172.21.15.162:0/2053286628 learned_addr learned my addr 172.21.15.162:0/2053286628 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:39.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.784+0000 7fdea6ffd700 1 --2- 172.21.15.162:0/2053286628 >> v2:172.21.15.118:3300/0 conn(0x7fdea80b0000 0x7fdea8060000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:39.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.784+0000 7fdea77fe700 1 -- 172.21.15.162:0/2053286628 >> v2:172.21.15.118:3300/0 conn(0x7fdea80b0000 msgr2=0x7fdea8060000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:39.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.784+0000 7fdea77fe700 1 --2- 172.21.15.162:0/2053286628 >> v2:172.21.15.118:3300/0 conn(0x7fdea80b0000 0x7fdea8060000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:39.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.784+0000 7fdea77fe700 1 -- 172.21.15.162:0/2053286628 >> v1:172.21.15.118:6789/0 conn(0x7fdea80604f0 legacy=0x7fdea814a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:39.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.788+0000 7fdea77fe700 1 -- 172.21.15.162:0/2053286628 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea80b0680 con 0x7fdea8098b30 2020-11-28T21:52:39.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.788+0000 7fdea6ffd700 1 --2- 172.21.15.162:0/2053286628 >> v2:172.21.15.118:3300/0 conn(0x7fdea80b0000 0x7fdea8060000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:52:39.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.788+0000 7fdea77fe700 1 --2- 172.21.15.162:0/2053286628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdea8098b30 0x7fdea80994f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7fdea8061f00 tx=0x7fde98006e80).ready entity=mon.1 client_cookie=7a80d251d4de879b server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:39.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.788+0000 7fdea67fc700 1 -- 172.21.15.162:0/2053286628 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde980097b0 con 0x7fdea8098b30 2020-11-28T21:52:39.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.788+0000 7fdea67fc700 1 -- 172.21.15.162:0/2053286628 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde98009910 con 0x7fdea8098b30 2020-11-28T21:52:39.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.788+0000 7fdea67fc700 1 -- 172.21.15.162:0/2053286628 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde98009a80 con 0x7fdea8098b30 2020-11-28T21:52:39.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.788+0000 7fdeacc1d700 1 -- 172.21.15.162:0/2053286628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdea8098b30 msgr2=0x7fdea80994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:39.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.788+0000 7fdeacc1d700 1 --2- 172.21.15.162:0/2053286628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdea8098b30 0x7fdea80994f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7fdea8061f00 tx=0x7fde98006e80).stop 2020-11-28T21:52:39.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.788+0000 7fdeacc1d700 1 -- 172.21.15.162:0/2053286628 shutdown_connections 2020-11-28T21:52:39.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.788+0000 7fdeacc1d700 1 --2- 172.21.15.162:0/2053286628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdea8098b30 0x7fdea80994f0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:39.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.788+0000 7fdeacc1d700 1 --2- 172.21.15.162:0/2053286628 >> v2:172.21.15.118:3300/0 conn(0x7fdea80b0000 0x7fdea8060000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:39.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.788+0000 7fdeacc1d700 1 -- 172.21.15.162:0/2053286628 >> 172.21.15.162:0/2053286628 conn(0x7fdea805b970 msgr2=0x7fdea805bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:39.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.788+0000 7fdeacc1d700 1 -- 172.21.15.162:0/2053286628 shutdown_connections 2020-11-28T21:52:39.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.788+0000 7fdeacc1d700 1 -- 172.21.15.162:0/2053286628 wait complete. 2020-11-28T21:52:39.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.788+0000 7fdeacc1d700 1 Processor -- start 2020-11-28T21:52:39.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.788+0000 7fdeacc1d700 1 -- start start 2020-11-28T21:52:39.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.788+0000 7fdeacc1d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdea80604f0 0x7fdea80b3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:39.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.788+0000 7fdeacc1d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdea8098b30 0x7fdea81592c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:39.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.788+0000 7fdeacc1d700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdea80b0000 0x7fdea8064e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:39.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.788+0000 7fdeacc1d700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea80ad0b0 con 0x7fdea80604f0 2020-11-28T21:52:39.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.788+0000 7fdeacc1d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea80649d0 con 0x7fdea8098b30 2020-11-28T21:52:39.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.788+0000 7fdeacc1d700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea80ace10 con 0x7fdea80b0000 2020-11-28T21:52:39.810 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.792+0000 7fdea6ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdea8098b30 0x7fdea81592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:39.810 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.792+0000 7fdea6ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdea8098b30 0x7fdea81592c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41608/0 (socket says 172.21.15.162:41608) 2020-11-28T21:52:39.811 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.792+0000 7fdea6ffd700 1 -- 172.21.15.162:0/1283084311 learned_addr learned my addr 172.21.15.162:0/1283084311 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:39.811 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.792+0000 7fdea77fe700 1 --2- 172.21.15.162:0/1283084311 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdea80604f0 0x7fdea80b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:39.811 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.792+0000 7fdea7fff700 1 --2- 172.21.15.162:0/1283084311 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdea80b0000 0x7fdea8064e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:39.812 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.792+0000 7fdea6ffd700 1 -- 172.21.15.162:0/1283084311 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdea80b0000 msgr2=0x7fdea8064e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:39.812 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.792+0000 7fdea6ffd700 1 --2- 172.21.15.162:0/1283084311 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdea80b0000 0x7fdea8064e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:39.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.792+0000 7fdea6ffd700 1 -- 172.21.15.162:0/1283084311 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdea80604f0 msgr2=0x7fdea80b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:39.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.792+0000 7fdea6ffd700 1 --2- 172.21.15.162:0/1283084311 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdea80604f0 0x7fdea80b3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:39.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.792+0000 7fdea6ffd700 1 -- 172.21.15.162:0/1283084311 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea80ae8c0 con 0x7fdea8098b30 2020-11-28T21:52:39.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.792+0000 7fdea77fe700 1 --2- 172.21.15.162:0/1283084311 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdea80604f0 0x7fdea80b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:52:39.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.792+0000 7fdea6ffd700 1 --2- 172.21.15.162:0/1283084311 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdea8098b30 0x7fdea81592c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 rx=0x7fde90007000 tx=0x7fde90008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:39.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.792+0000 7fdea4ff9700 1 -- 172.21.15.162:0/1283084311 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde9000ce30 con 0x7fdea8098b30 2020-11-28T21:52:39.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.792+0000 7fdeacc1d700 1 -- 172.21.15.162:0/1283084311 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdea80aeb60 con 0x7fdea8098b30 2020-11-28T21:52:39.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.792+0000 7fdeacc1d700 1 -- 172.21.15.162:0/1283084311 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdea80aca10 con 0x7fdea8098b30 2020-11-28T21:52:39.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.792+0000 7fdea4ff9700 1 -- 172.21.15.162:0/1283084311 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde90007750 con 0x7fdea8098b30 2020-11-28T21:52:39.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.792+0000 7fdea4ff9700 1 -- 172.21.15.162:0/1283084311 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde90007ab0 con 0x7fdea8098b30 2020-11-28T21:52:39.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.792+0000 7fdeacc1d700 1 -- 172.21.15.162:0/1283084311 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdea8057050 con 0x7fdea8098b30 2020-11-28T21:52:39.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.792+0000 7fdea4ff9700 1 -- 172.21.15.162:0/1283084311 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fde90007c10 con 0x7fdea8098b30 2020-11-28T21:52:39.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.796+0000 7fdea4ff9700 1 --2- 172.21.15.162:0/1283084311 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fde8c055330 0x7fde8c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:39.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.796+0000 7fdea4ff9700 1 -- 172.21.15.162:0/1283084311 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fde9006e250 con 0x7fdea8098b30 2020-11-28T21:52:39.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.796+0000 7fdea77fe700 1 --2- 172.21.15.162:0/1283084311 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fde8c055330 0x7fde8c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:39.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.796+0000 7fdea77fe700 1 --2- 172.21.15.162:0/1283084311 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fde8c055330 0x7fde8c0577f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7fde98019000 tx=0x7fde98016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:39.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.800+0000 7fdea4ff9700 1 -- 172.21.15.162:0/1283084311 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fde900443c0 con 0x7fdea8098b30 2020-11-28T21:52:39.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.984+0000 7fdeacc1d700 1 -- 172.21.15.162:0/1283084311 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdea80b0840 con 0x7fdea8098b30 2020-11-28T21:52:39.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.984+0000 7fdea4ff9700 1 -- 172.21.15.162:0/1283084311 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+2499 (secure 0 0 0) 0x7fde900479e0 con 0x7fdea8098b30 2020-11-28T21:52:39.986 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:52:39.987 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":33,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5540,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2116325247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2116325247},{"type":"v1","addr":"172.21.15.162:6835","nonce":2116325247}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":30},{"gid":5543,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.118:6833/3491069258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3491069258},{"type":"v1","addr":"172.21.15.118:6833","nonce":3491069258}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:52:28.454745+0000","modified":"2020-11-28T21:52:29.457610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6172},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6172":{"gid":6172,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/163256983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":163256983},{"type":"v1","addr":"172.21.15.162:6837","nonce":163256983}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-28T21:52:39.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.988+0000 7fdeacc1d700 1 -- 172.21.15.162:0/1283084311 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fde8c055330 msgr2=0x7fde8c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:39.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.988+0000 7fdeacc1d700 1 --2- 172.21.15.162:0/1283084311 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fde8c055330 0x7fde8c0577f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7fde98019000 tx=0x7fde98016000).stop 2020-11-28T21:52:39.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.988+0000 7fdeacc1d700 1 -- 172.21.15.162:0/1283084311 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdea8098b30 msgr2=0x7fdea81592c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:39.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.988+0000 7fdeacc1d700 1 --2- 172.21.15.162:0/1283084311 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdea8098b30 0x7fdea81592c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 rx=0x7fde90007000 tx=0x7fde90008040).stop 2020-11-28T21:52:39.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.988+0000 7fdeacc1d700 1 -- 172.21.15.162:0/1283084311 shutdown_connections 2020-11-28T21:52:39.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.988+0000 7fdeacc1d700 1 --2- 172.21.15.162:0/1283084311 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fde8c055330 0x7fde8c0577f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:39.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.988+0000 7fdeacc1d700 1 --2- 172.21.15.162:0/1283084311 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdea80b0000 0x7fdea8064e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:39.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.988+0000 7fdeacc1d700 1 --2- 172.21.15.162:0/1283084311 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdea80604f0 0x7fdea80b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:39.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.988+0000 7fdeacc1d700 1 --2- 172.21.15.162:0/1283084311 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdea8098b30 0x7fdea81592c0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:39.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.988+0000 7fdeacc1d700 1 -- 172.21.15.162:0/1283084311 >> 172.21.15.162:0/1283084311 conn(0x7fdea805b970 msgr2=0x7fdea80a8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:39.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.988+0000 7fdeacc1d700 1 -- 172.21.15.162:0/1283084311 shutdown_connections 2020-11-28T21:52:39.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:39.988+0000 7fdeacc1d700 1 -- 172.21.15.162:0/1283084311 wait complete. 2020-11-28T21:52:39.992 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 33 2020-11-28T21:52:40.001 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:52:40.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.112+0000 7f3399fb3700 1 Processor -- start 2020-11-28T21:52:40.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.112+0000 7f3399fb3700 1 -- start start 2020-11-28T21:52:40.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.112+0000 7f3399fb3700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f339409abb0 0x7f3394061190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:40.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.112+0000 7f3399fb3700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f3394060010 0x7f3394144670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:40.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.112+0000 7f3399fb3700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3394143ba0 con 0x7f33940b01a0 2020-11-28T21:52:40.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.112+0000 7f3399fb3700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3394144b60 con 0x7f3394060010 2020-11-28T21:52:40.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.112+0000 7f3399fb3700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f339409af90 con 0x7f339409abb0 2020-11-28T21:52:40.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.112+0000 7f3393fff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f33940b01a0 0x7f3394143f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:48462/0 (socket says 172.21.15.162:48462) 2020-11-28T21:52:40.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.112+0000 7f3393fff700 1 -- 172.21.15.162:0/2112873168 learned_addr learned my addr 172.21.15.162:0/2112873168 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:52:40.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.112+0000 7f3398fb1700 1 --2- 172.21.15.162:0/2112873168 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f339409abb0 0x7f3394061190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:40.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.112+0000 7f33997b2700 1 --2- 172.21.15.162:0/2112873168 >> v2:172.21.15.118:3300/0 conn(0x7f3394060010 0x7f3394144670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:40.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.112+0000 7f33937fe700 1 -- 172.21.15.162:0/2112873168 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 406559722 0 0) 0x7f3394143ba0 con 0x7f33940b01a0 2020-11-28T21:52:40.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.112+0000 7f33937fe700 1 -- 172.21.15.162:0/2112873168 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3380003230 con 0x7f33940b01a0 2020-11-28T21:52:40.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.116+0000 7f3398fb1700 1 -- 172.21.15.162:0/2112873168 >> v2:172.21.15.118:3300/0 conn(0x7f3394060010 msgr2=0x7f3394144670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:52:40.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.116+0000 7f3398fb1700 1 --2- 172.21.15.162:0/2112873168 >> v2:172.21.15.118:3300/0 conn(0x7f3394060010 0x7f3394144670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:40.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.116+0000 7f3398fb1700 1 -- 172.21.15.162:0/2112873168 >> v1:172.21.15.162:6789/0 conn(0x7f33940b01a0 legacy=0x7f3394143f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:40.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.116+0000 7f3398fb1700 1 -- 172.21.15.162:0/2112873168 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33940b1d10 con 0x7f339409abb0 2020-11-28T21:52:40.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.116+0000 7f3398fb1700 1 --2- 172.21.15.162:0/2112873168 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f339409abb0 0x7f3394061190 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f33840032b0 tx=0x7f33840091a0).ready entity=mon.2 client_cookie=f85a452dfb84a91 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:40.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.116+0000 7f33937fe700 1 -- 172.21.15.162:0/2112873168 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f338400b940 con 0x7f339409abb0 2020-11-28T21:52:40.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.116+0000 7f33937fe700 1 -- 172.21.15.162:0/2112873168 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f338400baa0 con 0x7f339409abb0 2020-11-28T21:52:40.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.116+0000 7f3399fb3700 1 -- 172.21.15.162:0/2112873168 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f339409abb0 msgr2=0x7f3394061190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:40.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.116+0000 7f33937fe700 1 -- 172.21.15.162:0/2112873168 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f338400bc10 con 0x7f339409abb0 2020-11-28T21:52:40.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.116+0000 7f3399fb3700 1 --2- 172.21.15.162:0/2112873168 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f339409abb0 0x7f3394061190 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f33840032b0 tx=0x7f33840091a0).stop 2020-11-28T21:52:40.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.116+0000 7f3399fb3700 1 -- 172.21.15.162:0/2112873168 shutdown_connections 2020-11-28T21:52:40.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.116+0000 7f3399fb3700 1 --2- 172.21.15.162:0/2112873168 >> v2:172.21.15.118:3300/0 conn(0x7f3394060010 0x7f3394144670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:40.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.116+0000 7f3399fb3700 1 --2- 172.21.15.162:0/2112873168 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f339409abb0 0x7f3394061190 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:40.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.116+0000 7f3399fb3700 1 -- 172.21.15.162:0/2112873168 >> 172.21.15.162:0/2112873168 conn(0x7f33940b54f0 msgr2=0x7f33940b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:40.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.116+0000 7f3399fb3700 1 -- 172.21.15.162:0/2112873168 shutdown_connections 2020-11-28T21:52:40.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.116+0000 7f3399fb3700 1 -- 172.21.15.162:0/2112873168 wait complete. 2020-11-28T21:52:40.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.116+0000 7f3399fb3700 1 Processor -- start 2020-11-28T21:52:40.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.116+0000 7f3399fb3700 1 -- start start 2020-11-28T21:52:40.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.116+0000 7f3399fb3700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3394060010 0x7f33941470f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:40.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.116+0000 7f3399fb3700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f339409abb0 0x7f3394096de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:40.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.116+0000 7f3399fb3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33940b01a0 0x7f33941519d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:40.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.116+0000 7f3399fb3700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3394151340 con 0x7f339409abb0 2020-11-28T21:52:40.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.116+0000 7f3399fb3700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33940972d0 con 0x7f33940b01a0 2020-11-28T21:52:40.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.116+0000 7f3399fb3700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33940974f0 con 0x7f3394060010 2020-11-28T21:52:40.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.116+0000 7f33997b2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33940b01a0 0x7f33941519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:40.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.116+0000 7f33997b2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33940b01a0 0x7f33941519d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41624/0 (socket says 172.21.15.162:41624) 2020-11-28T21:52:40.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.116+0000 7f33997b2700 1 -- 172.21.15.162:0/3429670128 learned_addr learned my addr 172.21.15.162:0/3429670128 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:52:40.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.116+0000 7f3398fb1700 1 --2- 172.21.15.162:0/3429670128 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3394060010 0x7f33941470f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:40.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.116+0000 7f3393fff700 1 --2- 172.21.15.162:0/3429670128 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f339409abb0 0x7f3394096de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:40.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.120+0000 7f33997b2700 1 -- 172.21.15.162:0/3429670128 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3394060010 msgr2=0x7f33941470f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:40.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.120+0000 7f33997b2700 1 --2- 172.21.15.162:0/3429670128 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3394060010 0x7f33941470f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:40.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.120+0000 7f33997b2700 1 -- 172.21.15.162:0/3429670128 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f339409abb0 msgr2=0x7f3394096de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:40.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.120+0000 7f33997b2700 1 --2- 172.21.15.162:0/3429670128 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f339409abb0 0x7f3394096de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:40.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.120+0000 7f33997b2700 1 -- 172.21.15.162:0/3429670128 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3394097790 con 0x7f33940b01a0 2020-11-28T21:52:40.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.120+0000 7f3398fb1700 1 --2- 172.21.15.162:0/3429670128 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3394060010 0x7f33941470f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:52:40.129 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.120+0000 7f3393fff700 1 --2- 172.21.15.162:0/3429670128 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f339409abb0 0x7f3394096de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:52:40.129 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.120+0000 7f33997b2700 1 --2- 172.21.15.162:0/3429670128 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33940b01a0 0x7f33941519d0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7f338c006a50 tx=0x7f338c006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:40.129 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.120+0000 7f3391ffb700 1 -- 172.21.15.162:0/3429670128 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f338c00cae0 con 0x7f33940b01a0 2020-11-28T21:52:40.129 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.120+0000 7f3391ffb700 1 -- 172.21.15.162:0/3429670128 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f338c00cc40 con 0x7f33940b01a0 2020-11-28T21:52:40.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.120+0000 7f3399fb3700 1 -- 172.21.15.162:0/3429670128 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3394151530 con 0x7f33940b01a0 2020-11-28T21:52:40.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.120+0000 7f3391ffb700 1 -- 172.21.15.162:0/3429670128 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f338c0123f0 con 0x7f33940b01a0 2020-11-28T21:52:40.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.120+0000 7f3399fb3700 1 -- 172.21.15.162:0/3429670128 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3394098880 con 0x7f33940b01a0 2020-11-28T21:52:40.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.120+0000 7f3399fb3700 1 -- 172.21.15.162:0/3429670128 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3394140630 con 0x7f33940b01a0 2020-11-28T21:52:40.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.120+0000 7f3391ffb700 1 -- 172.21.15.162:0/3429670128 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f338c012550 con 0x7f33940b01a0 2020-11-28T21:52:40.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.120+0000 7f3391ffb700 1 --2- 172.21.15.162:0/3429670128 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3374055440 0x7f3374057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:52:40.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.120+0000 7f3391ffb700 1 -- 172.21.15.162:0/3429670128 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f338c06e550 con 0x7f33940b01a0 2020-11-28T21:52:40.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.120+0000 7f3398fb1700 1 --2- 172.21.15.162:0/3429670128 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3374055440 0x7f3374057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:52:40.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.120+0000 7f3398fb1700 1 --2- 172.21.15.162:0/3429670128 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3374055440 0x7f3374057900 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f3384019000 tx=0x7f3384016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:52:40.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.124+0000 7f3391ffb700 1 -- 172.21.15.162:0/3429670128 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f3394140630 con 0x7f33940b01a0 2020-11-28T21:52:40.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.308+0000 7f3399fb3700 1 -- 172.21.15.162:0/3429670128 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3394140630 con 0x7f33940b01a0 2020-11-28T21:52:40.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.308+0000 7f3391ffb700 1 -- 172.21.15.162:0/3429670128 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+2499 (secure 0 0 0) 0x7f3394140630 con 0x7f33940b01a0 2020-11-28T21:52:40.310 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:52:40.310 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":33,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5540,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2116325247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2116325247},{"type":"v1","addr":"172.21.15.162:6835","nonce":2116325247}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":30},{"gid":5543,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.118:6833/3491069258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3491069258},{"type":"v1","addr":"172.21.15.118:6833","nonce":3491069258}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:52:28.454745+0000","modified":"2020-11-28T21:52:29.457610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6172},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6172":{"gid":6172,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/163256983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":163256983},{"type":"v1","addr":"172.21.15.162:6837","nonce":163256983}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-28T21:52:40.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.308+0000 7f3399fb3700 1 -- 172.21.15.162:0/3429670128 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3374055440 msgr2=0x7f3374057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:40.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.308+0000 7f3399fb3700 1 --2- 172.21.15.162:0/3429670128 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3374055440 0x7f3374057900 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f3384019000 tx=0x7f3384016000).stop 2020-11-28T21:52:40.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.308+0000 7f3399fb3700 1 -- 172.21.15.162:0/3429670128 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33940b01a0 msgr2=0x7f33941519d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:52:40.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.308+0000 7f3399fb3700 1 --2- 172.21.15.162:0/3429670128 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33940b01a0 0x7f33941519d0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7f338c006a50 tx=0x7f338c006dc0).stop 2020-11-28T21:52:40.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.308+0000 7f3399fb3700 1 -- 172.21.15.162:0/3429670128 shutdown_connections 2020-11-28T21:52:40.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.308+0000 7f3399fb3700 1 --2- 172.21.15.162:0/3429670128 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3374055440 0x7f3374057900 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:40.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.308+0000 7f3399fb3700 1 --2- 172.21.15.162:0/3429670128 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3394060010 0x7f33941470f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:40.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.308+0000 7f3399fb3700 1 --2- 172.21.15.162:0/3429670128 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33940b01a0 0x7f33941519d0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:40.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.308+0000 7f3399fb3700 1 --2- 172.21.15.162:0/3429670128 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f339409abb0 0x7f3394096de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:52:40.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.312+0000 7f3399fb3700 1 -- 172.21.15.162:0/3429670128 >> 172.21.15.162:0/3429670128 conn(0x7f33940b54f0 msgr2=0x7f33940b3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:52:40.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.312+0000 7f3399fb3700 1 -- 172.21.15.162:0/3429670128 shutdown_connections 2020-11-28T21:52:40.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:52:40.312+0000 7f3399fb3700 1 -- 172.21.15.162:0/3429670128 wait complete. 2020-11-28T21:52:40.316 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 33 2020-11-28T21:52:40.324 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_cap_revoke_eviction_timeout 30.0 2020-11-28T21:52:40.442 INFO:teuthology.orchestra.run.smithi162.stdout:{"success":"mds_cap_revoke_eviction_timeout = '30.000000' "} 2020-11-28T21:52:40.443 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_cap_revoke_eviction_timeout = '30.000000' " } 2020-11-28T21:52:40.443 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:52:40.444 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:52:40.444 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:52:40.452 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2020-11-28T21:52:40.453 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-28T21:52:40.453 INFO:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-28T21:52:40.454 INFO:teuthology.orchestra.run.smithi192:> import time 2020-11-28T21:52:40.454 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:52:40.455 INFO:teuthology.orchestra.run.smithi192:> with open("/home/ubuntu/cephtest/mnt.0/background_file", '"'"'w'"'"') as f: 2020-11-28T21:52:40.455 INFO:teuthology.orchestra.run.smithi192:> f.write('"'"'content'"'"') 2020-11-28T21:52:40.455 INFO:teuthology.orchestra.run.smithi192:> f.flush() 2020-11-28T21:52:40.456 INFO:teuthology.orchestra.run.smithi192:> f.write('"'"'content2'"'"') 2020-11-28T21:52:40.456 INFO:teuthology.orchestra.run.smithi192:> while True: 2020-11-28T21:52:40.456 INFO:teuthology.orchestra.run.smithi192:> time.sleep(1) 2020-11-28T21:52:40.457 INFO:teuthology.orchestra.run.smithi192:> ' 2020-11-28T21:52:40.498 INFO:teuthology.orchestra.run.smithi192:> sudo ls /home/ubuntu/cephtest/mnt.0/background_file 2020-11-28T21:52:40.520 INFO:teuthology.orchestra.run.smithi192.stderr:ls: cannot access '/home/ubuntu/cephtest/mnt.0/background_file': No such file or directory 2020-11-28T21:52:40.521 DEBUG:teuthology.orchestra.run:got remote process result: 2 2020-11-28T21:52:41.523 INFO:teuthology.orchestra.run.smithi192:> sudo ls /home/ubuntu/cephtest/mnt.0/background_file 2020-11-28T21:52:41.538 INFO:teuthology.orchestra.run.smithi192.stdout:/home/ubuntu/cephtest/mnt.0/background_file 2020-11-28T21:52:41.540 DEBUG:tasks.cephfs.mount:File background_file became visible from 0 after 1s 2020-11-28T21:52:41.540 INFO:teuthology.orchestra.run.smithi096:> sudo ls /home/ubuntu/cephtest/mnt.1/background_file 2020-11-28T21:52:41.580 INFO:teuthology.orchestra.run.smithi096.stdout:/home/ubuntu/cephtest/mnt.1/background_file 2020-11-28T21:52:41.582 DEBUG:tasks.cephfs.mount:File background_file became visible from 1 after 0s 2020-11-28T21:52:41.582 INFO:tasks.cephfs.fuse_mount:Killing ceph-fuse connection on ubuntu@smithi192.front.sepia.ceph.com... 2020-11-28T21:52:41.598 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:52:41.599 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:52:41.599 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:52:41.631 INFO:teuthology.orchestra.run.smithi096.stdout:fuseblk 2020-11-28T21:52:41.632 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-11-28T21:52:41.632 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-28T21:52:41.633 INFO:teuthology.orchestra.run.smithi096:> import os 2020-11-28T21:52:41.633 INFO:teuthology.orchestra.run.smithi096:> import time 2020-11-28T21:52:41.633 INFO:teuthology.orchestra.run.smithi096:> 2020-11-28T21:52:41.634 INFO:teuthology.orchestra.run.smithi096:> fd = os.open("/home/ubuntu/cephtest/mnt.1/background_file", os.O_RDWR | os.O_CREAT, 0o644) 2020-11-28T21:52:41.634 INFO:teuthology.orchestra.run.smithi096:> try: 2020-11-28T21:52:41.634 INFO:teuthology.orchestra.run.smithi096:> while True: 2020-11-28T21:52:41.635 INFO:teuthology.orchestra.run.smithi096:> os.write(fd, b'"'"'content'"'"') 2020-11-28T21:52:41.635 INFO:teuthology.orchestra.run.smithi096:> time.sleep(1) 2020-11-28T21:52:41.635 INFO:teuthology.orchestra.run.smithi096:> if not False: 2020-11-28T21:52:41.636 INFO:teuthology.orchestra.run.smithi096:> break 2020-11-28T21:52:41.636 INFO:teuthology.orchestra.run.smithi096:> except IOError as e: 2020-11-28T21:52:41.636 INFO:teuthology.orchestra.run.smithi096:> pass 2020-11-28T21:52:41.637 INFO:teuthology.orchestra.run.smithi096:> os.close(fd) 2020-11-28T21:52:41.637 INFO:teuthology.orchestra.run.smithi096:> ' 2020-11-28T21:53:08.448 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:53:08.451 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:53:08.454 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:53:08.456 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:53:19.197 INFO:tasks.cephfs.test_misc:cap_waiter waited 37.52207064628601s 2020-11-28T21:53:19.198 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist ls --format=json 2020-11-28T21:53:19.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.317+0000 7f93f0dc1700 1 Processor -- start 2020-11-28T21:53:19.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.317+0000 7f93f0dc1700 1 -- start start 2020-11-28T21:53:19.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.317+0000 7f93f0dc1700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f93ec0603d0 0x7f93ec0b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:19.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.317+0000 7f93f0dc1700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f93ec142a00 0x7f93ec09cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:19.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.317+0000 7f93f0dc1700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93ec0b4110 con 0x7f93ec0b4c70 2020-11-28T21:53:19.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.317+0000 7f93f0dc1700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93ec060f60 con 0x7f93ec0603d0 2020-11-28T21:53:19.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.317+0000 7f93f0dc1700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93ec0b43f0 con 0x7f93ec142a00 2020-11-28T21:53:19.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.317+0000 7f93eaffd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f93ec142a00 0x7f93ec09cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:19.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.317+0000 7f93ebfff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f93ec0b4c70 0x7f93ec143020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:48478/0 (socket says 172.21.15.162:48478) 2020-11-28T21:53:19.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.317+0000 7f93ebfff700 1 -- 172.21.15.162:0/1914211642 learned_addr learned my addr 172.21.15.162:0/1914211642 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:19.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.317+0000 7f93eb7fe700 1 --2- 172.21.15.162:0/1914211642 >> v2:172.21.15.118:3300/0 conn(0x7f93ec0603d0 0x7f93ec0b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:19.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.317+0000 7f93eaffd700 1 -- 172.21.15.162:0/1914211642 >> v2:172.21.15.118:3300/0 conn(0x7f93ec0603d0 msgr2=0x7f93ec0b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:19.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.317+0000 7f93eaffd700 1 --2- 172.21.15.162:0/1914211642 >> v2:172.21.15.118:3300/0 conn(0x7f93ec0603d0 0x7f93ec0b3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:19.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.317+0000 7f93eaffd700 1 -- 172.21.15.162:0/1914211642 >> v1:172.21.15.162:6789/0 conn(0x7f93ec0b4c70 legacy=0x7f93ec143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:19.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.317+0000 7f93eaffd700 1 -- 172.21.15.162:0/1914211642 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93ec0b4690 con 0x7f93ec142a00 2020-11-28T21:53:19.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.317+0000 7f93eaffd700 1 --2- 172.21.15.162:0/1914211642 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93ec142a00 0x7f93ec09cfc0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0x7f93d4002f30 tx=0x7f93d4006dc0).ready entity=mon.1 client_cookie=c267479c8b8320a3 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:19.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.317+0000 7f93ea7fc700 1 -- 172.21.15.162:0/1914211642 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93d40097f0 con 0x7f93ec142a00 2020-11-28T21:53:19.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.317+0000 7f93ea7fc700 1 -- 172.21.15.162:0/1914211642 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93d4009950 con 0x7f93ec142a00 2020-11-28T21:53:19.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.317+0000 7f93ea7fc700 1 -- 172.21.15.162:0/1914211642 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93d4009cb0 con 0x7f93ec142a00 2020-11-28T21:53:19.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.317+0000 7f93f0dc1700 1 -- 172.21.15.162:0/1914211642 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93ec142a00 msgr2=0x7f93ec09cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:19.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.317+0000 7f93f0dc1700 1 --2- 172.21.15.162:0/1914211642 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93ec142a00 0x7f93ec09cfc0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0x7f93d4002f30 tx=0x7f93d4006dc0).stop 2020-11-28T21:53:19.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.317+0000 7f93f0dc1700 1 -- 172.21.15.162:0/1914211642 shutdown_connections 2020-11-28T21:53:19.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.317+0000 7f93f0dc1700 1 --2- 172.21.15.162:0/1914211642 >> v2:172.21.15.118:3300/0 conn(0x7f93ec0603d0 0x7f93ec0b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:19.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.317+0000 7f93f0dc1700 1 --2- 172.21.15.162:0/1914211642 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93ec142a00 0x7f93ec09cfc0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:19.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.317+0000 7f93f0dc1700 1 -- 172.21.15.162:0/1914211642 >> 172.21.15.162:0/1914211642 conn(0x7f93ec066a40 msgr2=0x7f93ec066e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:19.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.317+0000 7f93f0dc1700 1 -- 172.21.15.162:0/1914211642 shutdown_connections 2020-11-28T21:53:19.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.317+0000 7f93f0dc1700 1 -- 172.21.15.162:0/1914211642 wait complete. 2020-11-28T21:53:19.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.321+0000 7f93f0dc1700 1 Processor -- start 2020-11-28T21:53:19.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.321+0000 7f93f0dc1700 1 -- start start 2020-11-28T21:53:19.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.321+0000 7f93f0dc1700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93ec0603d0 0x7f93ec0b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:19.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.321+0000 7f93f0dc1700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f93ec0b4c70 0x7f93ec1498b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:19.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.321+0000 7f93f0dc1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93ec13c7f0 0x7f93ec154c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:19.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.321+0000 7f93f0dc1700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec0b8cf0 con 0x7f93ec0603d0 2020-11-28T21:53:19.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.321+0000 7f93f0dc1700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec13bf80 con 0x7f93ec13c7f0 2020-11-28T21:53:19.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.321+0000 7f93f0dc1700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec0b8fa0 con 0x7f93ec0b4c70 2020-11-28T21:53:19.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.321+0000 7f93ebfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93ec13c7f0 0x7f93ec154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:19.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.321+0000 7f93ebfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93ec13c7f0 0x7f93ec154c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41640/0 (socket says 172.21.15.162:41640) 2020-11-28T21:53:19.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.321+0000 7f93ebfff700 1 -- 172.21.15.162:0/4277106589 learned_addr learned my addr 172.21.15.162:0/4277106589 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:19.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.321+0000 7f93eb7fe700 1 --2- 172.21.15.162:0/4277106589 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93ec0603d0 0x7f93ec0b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:19.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.321+0000 7f93eaffd700 1 --2- 172.21.15.162:0/4277106589 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f93ec0b4c70 0x7f93ec1498b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:19.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.321+0000 7f93ebfff700 1 -- 172.21.15.162:0/4277106589 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f93ec0b4c70 msgr2=0x7f93ec1498b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:19.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.321+0000 7f93ebfff700 1 --2- 172.21.15.162:0/4277106589 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f93ec0b4c70 0x7f93ec1498b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:19.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.321+0000 7f93ebfff700 1 -- 172.21.15.162:0/4277106589 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93ec0603d0 msgr2=0x7f93ec0b7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:19.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.321+0000 7f93ebfff700 1 --2- 172.21.15.162:0/4277106589 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93ec0603d0 0x7f93ec0b7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:19.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.321+0000 7f93ebfff700 1 -- 172.21.15.162:0/4277106589 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93ec13c1b0 con 0x7f93ec13c7f0 2020-11-28T21:53:19.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.321+0000 7f93eb7fe700 1 --2- 172.21.15.162:0/4277106589 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93ec0603d0 0x7f93ec0b7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:53:19.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.321+0000 7f93ebfff700 1 --2- 172.21.15.162:0/4277106589 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93ec13c7f0 0x7f93ec154c10 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 rx=0x7f93ec0b9c10 tx=0x7f93e4007f10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:19.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.321+0000 7f93e8ff9700 1 -- 172.21.15.162:0/4277106589 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93e40020e0 con 0x7f93ec13c7f0 2020-11-28T21:53:19.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.321+0000 7f93e8ff9700 1 -- 172.21.15.162:0/4277106589 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93e4005b70 con 0x7f93ec13c7f0 2020-11-28T21:53:19.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.321+0000 7f93e8ff9700 1 -- 172.21.15.162:0/4277106589 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93e4007110 con 0x7f93ec13c7f0 2020-11-28T21:53:19.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.321+0000 7f93f0dc1700 1 -- 172.21.15.162:0/4277106589 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93ec0b8a50 con 0x7f93ec13c7f0 2020-11-28T21:53:19.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.321+0000 7f93f0dc1700 1 -- 172.21.15.162:0/4277106589 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93ec13d810 con 0x7f93ec13c7f0 2020-11-28T21:53:19.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.325+0000 7f93f0dc1700 1 -- 172.21.15.162:0/4277106589 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93ec057050 con 0x7f93ec13c7f0 2020-11-28T21:53:19.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.325+0000 7f93e8ff9700 1 -- 172.21.15.162:0/4277106589 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f93e4010030 con 0x7f93ec13c7f0 2020-11-28T21:53:19.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.325+0000 7f93e8ff9700 1 --2- 172.21.15.162:0/4277106589 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f93d00554e0 0x7f93d00579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:19.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.325+0000 7f93e8ff9700 1 -- 172.21.15.162:0/4277106589 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f93e406e0a0 con 0x7f93ec13c7f0 2020-11-28T21:53:19.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.325+0000 7f93eb7fe700 1 --2- 172.21.15.162:0/4277106589 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f93d00554e0 0x7f93d00579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:19.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.325+0000 7f93eb7fe700 1 --2- 172.21.15.162:0/4277106589 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f93d00554e0 0x7f93d00579a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f93dc00c000 tx=0x7f93dc008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:19.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.329+0000 7f93e8ff9700 1 -- 172.21.15.162:0/4277106589 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f93e40441e0 con 0x7f93ec13c7f0 2020-11-28T21:53:19.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.481+0000 7f93f0dc1700 1 -- 172.21.15.162:0/4277106589 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd blacklist ls", "format": "json"} v 0) v1 -- 0x7f93ec069b60 con 0x7f93ec13c7f0 2020-11-28T21:53:19.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.485+0000 7f93e8ff9700 1 -- 172.21.15.162:0/4277106589 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist ls", "format": "json"}]=0 listed 1 entries v77) v1 ==== 98+0+81 (secure 0 0 0) 0x7f93e4047800 con 0x7f93ec13c7f0 2020-11-28T21:53:19.491 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:53:19.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.485+0000 7f93f0dc1700 1 -- 172.21.15.162:0/4277106589 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f93d00554e0 msgr2=0x7f93d00579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:19.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.485+0000 7f93f0dc1700 1 --2- 172.21.15.162:0/4277106589 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f93d00554e0 0x7f93d00579a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f93dc00c000 tx=0x7f93dc008040).stop 2020-11-28T21:53:19.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.485+0000 7f93f0dc1700 1 -- 172.21.15.162:0/4277106589 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93ec13c7f0 msgr2=0x7f93ec154c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:19.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.485+0000 7f93f0dc1700 1 --2- 172.21.15.162:0/4277106589 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93ec13c7f0 0x7f93ec154c10 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 rx=0x7f93ec0b9c10 tx=0x7f93e4007f10).stop 2020-11-28T21:53:19.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.485+0000 7f93f0dc1700 1 -- 172.21.15.162:0/4277106589 shutdown_connections 2020-11-28T21:53:19.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.485+0000 7f93f0dc1700 1 --2- 172.21.15.162:0/4277106589 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f93d00554e0 0x7f93d00579a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:19.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.485+0000 7f93f0dc1700 1 --2- 172.21.15.162:0/4277106589 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93ec13c7f0 0x7f93ec154c10 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:19.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.485+0000 7f93f0dc1700 1 --2- 172.21.15.162:0/4277106589 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93ec0603d0 0x7f93ec0b7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:19.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.485+0000 7f93f0dc1700 1 --2- 172.21.15.162:0/4277106589 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f93ec0b4c70 0x7f93ec1498b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:19.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.485+0000 7f93f0dc1700 1 -- 172.21.15.162:0/4277106589 >> 172.21.15.162:0/4277106589 conn(0x7f93ec066a40 msgr2=0x7f93ec13dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:19.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.485+0000 7f93f0dc1700 1 -- 172.21.15.162:0/4277106589 shutdown_connections 2020-11-28T21:53:19.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.485+0000 7f93f0dc1700 1 -- 172.21.15.162:0/4277106589 wait complete. 2020-11-28T21:53:19.500 INFO:teuthology.orchestra.run.smithi162.stderr:listed 1 entries 2020-11-28T21:53:19.504 INFO:teuthology.orchestra.run.smithi162.stdout:[{"addr":"172.21.15.192:0/2498670596","until":"2020-11-28T22:53:13.037657+0000"}] 2020-11-28T21:53:19.521 INFO:tasks.cephfs.fuse_mount:Cleaning up killed ceph-fuse connection 2020-11-28T21:53:19.522 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:53:19.523 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:53:19.524 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:53:19.569 INFO:teuthology.orchestra.run.smithi192.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Transport endpoint is not connected 2020-11-28T21:53:19.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:53:19.571 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2020-11-28T21:53:19.572 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2020-11-28T21:53:19.572 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-28T21:53:19.573 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:53:19.645 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:53:19.646 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:53:19.646 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:53:19.693 INFO:teuthology.orchestra.run.smithi192.stdout:ext2/ext3 2020-11-28T21:53:19.694 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-11-28T21:53:19.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-28T21:53:19.695 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:53:19.745 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:53:19.864 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.857+0000 7ff006f17700 1 Processor -- start 2020-11-28T21:53:19.864 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.857+0000 7ff006f17700 1 -- start start 2020-11-28T21:53:19.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.857+0000 7ff006f17700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff000142a00 0x7ff0000b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:19.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.857+0000 7ff006f17700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff0000b4c70 0x7ff000143020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:19.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.857+0000 7ff006f17700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0000b4110 con 0x7ff0000603d0 2020-11-28T21:53:19.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.857+0000 7ff006f17700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff000060f60 con 0x7ff0000b4c70 2020-11-28T21:53:19.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.857+0000 7ff006f17700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0000b43f0 con 0x7ff000142a00 2020-11-28T21:53:19.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.857+0000 7ff005f15700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7ff0000603d0 0x7ff00009cfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.162:38636/0 (socket says 172.21.15.162:38636) 2020-11-28T21:53:19.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.857+0000 7ff005f15700 1 -- 172.21.15.162:0/3986444210 learned_addr learned my addr 172.21.15.162:0/3986444210 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:19.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.857+0000 7ff006716700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff0000b4c70 0x7ff000143020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:19.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.857+0000 7ff005714700 1 --2- 172.21.15.162:0/3986444210 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff000142a00 0x7ff0000b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:19.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.857+0000 7ff004f13700 1 -- 172.21.15.162:0/3986444210 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1360174361 0 0) 0x7ff0000b4110 con 0x7ff0000603d0 2020-11-28T21:53:19.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.857+0000 7ff004f13700 1 -- 172.21.15.162:0/3986444210 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefe8003230 con 0x7ff0000603d0 2020-11-28T21:53:19.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.857+0000 7ff006716700 1 -- 172.21.15.162:0/3986444210 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff000142a00 msgr2=0x7ff0000b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:19.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.857+0000 7ff006716700 1 --2- 172.21.15.162:0/3986444210 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff000142a00 0x7ff0000b3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:19.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.857+0000 7ff006716700 1 -- 172.21.15.162:0/3986444210 >> v1:172.21.15.118:6789/0 conn(0x7ff0000603d0 legacy=0x7ff00009cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:19.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.857+0000 7ff006716700 1 -- 172.21.15.162:0/3986444210 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0000b4690 con 0x7ff0000b4c70 2020-11-28T21:53:19.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.861+0000 7ff006716700 1 --2- 172.21.15.162:0/3986444210 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff0000b4c70 0x7ff000143020 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7feff8002f30 tx=0x7feff8006dc0).ready entity=mon.0 client_cookie=748b58ff7ed43825 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:19.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.861+0000 7ff004f13700 1 -- 172.21.15.162:0/3986444210 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feff80096a0 con 0x7ff0000b4c70 2020-11-28T21:53:19.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.861+0000 7ff004f13700 1 -- 172.21.15.162:0/3986444210 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff8009800 con 0x7ff0000b4c70 2020-11-28T21:53:19.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.861+0000 7ff004f13700 1 -- 172.21.15.162:0/3986444210 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feff8009b60 con 0x7ff0000b4c70 2020-11-28T21:53:19.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.861+0000 7ff006f17700 1 -- 172.21.15.162:0/3986444210 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff0000b4c70 msgr2=0x7ff000143020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:19.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.861+0000 7ff006f17700 1 --2- 172.21.15.162:0/3986444210 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff0000b4c70 0x7ff000143020 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7feff8002f30 tx=0x7feff8006dc0).stop 2020-11-28T21:53:19.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.861+0000 7ff006f17700 1 -- 172.21.15.162:0/3986444210 shutdown_connections 2020-11-28T21:53:19.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.861+0000 7ff006f17700 1 --2- 172.21.15.162:0/3986444210 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff0000b4c70 0x7ff000143020 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:19.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.861+0000 7ff006f17700 1 --2- 172.21.15.162:0/3986444210 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff000142a00 0x7ff0000b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:19.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.861+0000 7ff006f17700 1 -- 172.21.15.162:0/3986444210 >> 172.21.15.162:0/3986444210 conn(0x7ff000066a40 msgr2=0x7ff000066e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:19.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.861+0000 7ff006f17700 1 -- 172.21.15.162:0/3986444210 shutdown_connections 2020-11-28T21:53:19.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.861+0000 7ff006f17700 1 -- 172.21.15.162:0/3986444210 wait complete. 2020-11-28T21:53:19.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.861+0000 7ff006f17700 1 Processor -- start 2020-11-28T21:53:19.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.861+0000 7ff006f17700 1 -- start start 2020-11-28T21:53:19.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.865+0000 7ff006f17700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff0000603d0 0x7ff000154d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:19.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.865+0000 7ff006f17700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0000b4c70 0x7ff0001528e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:19.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.865+0000 7ff006f17700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff0000b18d0 0x7ff000152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:19.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.865+0000 7ff006f17700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff000155840 con 0x7ff0000b18d0 2020-11-28T21:53:19.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.865+0000 7ff006f17700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0001555d0 con 0x7ff0000b4c70 2020-11-28T21:53:19.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.865+0000 7ff006f17700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0000b0ef0 con 0x7ff0000603d0 2020-11-28T21:53:19.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.865+0000 7ff005714700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0000b4c70 0x7ff0001528e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:19.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.865+0000 7ff005f15700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff0000603d0 0x7ff000154d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:19.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.865+0000 7ff005714700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0000b4c70 0x7ff0001528e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41654/0 (socket says 172.21.15.162:41654) 2020-11-28T21:53:19.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.865+0000 7ff005714700 1 -- 172.21.15.162:0/1427501372 learned_addr learned my addr 172.21.15.162:0/1427501372 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:19.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.865+0000 7ff006716700 1 --2- 172.21.15.162:0/1427501372 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff0000b18d0 0x7ff000152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:19.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.865+0000 7ff005714700 1 -- 172.21.15.162:0/1427501372 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff0000603d0 msgr2=0x7ff000154d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:19.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.865+0000 7ff005714700 1 --2- 172.21.15.162:0/1427501372 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff0000603d0 0x7ff000154d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:19.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.865+0000 7ff005714700 1 -- 172.21.15.162:0/1427501372 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff0000b18d0 msgr2=0x7ff000152dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:19.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.865+0000 7ff005714700 1 --2- 172.21.15.162:0/1427501372 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff0000b18d0 0x7ff000152dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:19.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.865+0000 7ff005714700 1 -- 172.21.15.162:0/1427501372 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0000b1190 con 0x7ff0000b4c70 2020-11-28T21:53:19.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.865+0000 7ff005714700 1 --2- 172.21.15.162:0/1427501372 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0000b4c70 0x7ff0001528e0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 rx=0x7feff000c000 tx=0x7feff0008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:19.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.865+0000 7fefeeffd700 1 -- 172.21.15.162:0/1427501372 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feff000be30 con 0x7ff0000b4c70 2020-11-28T21:53:19.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.865+0000 7fefeeffd700 1 -- 172.21.15.162:0/1427501372 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff000c820 con 0x7ff0000b4c70 2020-11-28T21:53:19.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.865+0000 7ff006f17700 1 -- 172.21.15.162:0/1427501372 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0000b1430 con 0x7ff0000b4c70 2020-11-28T21:53:19.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.865+0000 7fefeeffd700 1 -- 172.21.15.162:0/1427501372 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feff000cb80 con 0x7ff0000b4c70 2020-11-28T21:53:19.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.865+0000 7ff006f17700 1 -- 172.21.15.162:0/1427501372 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff000155210 con 0x7ff0000b4c70 2020-11-28T21:53:19.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.869+0000 7fefeeffd700 1 -- 172.21.15.162:0/1427501372 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7feff000cce0 con 0x7ff0000b4c70 2020-11-28T21:53:19.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.869+0000 7ff006f17700 1 -- 172.21.15.162:0/1427501372 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff000057050 con 0x7ff0000b4c70 2020-11-28T21:53:19.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.869+0000 7fefeeffd700 1 --2- 172.21.15.162:0/1427501372 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fefe00553b0 0x7fefe0057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:19.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.869+0000 7fefeeffd700 1 -- 172.21.15.162:0/1427501372 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6116+0+0 (secure 0 0 0) 0x7feff006e370 con 0x7ff0000b4c70 2020-11-28T21:53:19.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.869+0000 7ff005f15700 1 --2- 172.21.15.162:0/1427501372 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fefe00553b0 0x7fefe0057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:19.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.869+0000 7ff005f15700 1 --2- 172.21.15.162:0/1427501372 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fefe00553b0 0x7fefe0057870 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7feffc00d000 tx=0x7feffc009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:19.886 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:19.877+0000 7fefeeffd700 1 -- 172.21.15.162:0/1427501372 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7feff0044530 con 0x7ff0000b4c70 2020-11-28T21:53:20.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.017+0000 7ff006f17700 1 -- 172.21.15.162:0/1427501372 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff000069b60 con 0x7ff0000b4c70 2020-11-28T21:53:20.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.021+0000 7fefeeffd700 1 -- 172.21.15.162:0/1427501372 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+16199 (secure 0 0 0) 0x7feff0047b50 con 0x7ff0000b4c70 2020-11-28T21:53:20.027 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:53:20.028 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":77,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:53:13.878105+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:52:25.955831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2020-11-28T21:52:27.265702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"172.21.15.192:0/2498670596":"2020-11-28T22:53:13.037657+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:53:20.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.021+0000 7ff006f17700 1 -- 172.21.15.162:0/1427501372 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fefe00553b0 msgr2=0x7fefe0057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:20.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.021+0000 7ff006f17700 1 --2- 172.21.15.162:0/1427501372 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fefe00553b0 0x7fefe0057870 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7feffc00d000 tx=0x7feffc009040).stop 2020-11-28T21:53:20.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.021+0000 7ff006f17700 1 -- 172.21.15.162:0/1427501372 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0000b4c70 msgr2=0x7ff0001528e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:20.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.021+0000 7ff006f17700 1 --2- 172.21.15.162:0/1427501372 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0000b4c70 0x7ff0001528e0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 rx=0x7feff000c000 tx=0x7feff0008040).stop 2020-11-28T21:53:20.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.025+0000 7ff006f17700 1 -- 172.21.15.162:0/1427501372 shutdown_connections 2020-11-28T21:53:20.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.025+0000 7ff006f17700 1 --2- 172.21.15.162:0/1427501372 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fefe00553b0 0x7fefe0057870 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:20.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.025+0000 7ff006f17700 1 --2- 172.21.15.162:0/1427501372 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff0000603d0 0x7ff000154d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:20.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.025+0000 7ff006f17700 1 --2- 172.21.15.162:0/1427501372 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff0000b18d0 0x7ff000152dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:20.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.025+0000 7ff006f17700 1 --2- 172.21.15.162:0/1427501372 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0000b4c70 0x7ff0001528e0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:20.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.025+0000 7ff006f17700 1 -- 172.21.15.162:0/1427501372 >> 172.21.15.162:0/1427501372 conn(0x7ff000066a40 msgr2=0x7ff00009cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:20.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.025+0000 7ff006f17700 1 -- 172.21.15.162:0/1427501372 shutdown_connections 2020-11-28T21:53:20.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.025+0000 7ff006f17700 1 -- 172.21.15.162:0/1427501372 wait complete. 2020-11-28T21:53:20.047 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-28T21:52:25.955831+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2020-11-28T21:52:27.265702+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-28T21:53:20.048 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:53:20.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.161+0000 7f4d412e1700 1 Processor -- start 2020-11-28T21:53:20.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.161+0000 7f4d412e1700 1 -- start start 2020-11-28T21:53:20.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.161+0000 7f4d412e1700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4d3c0b3ab0 0x7f4d3c05c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:20.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.161+0000 7f4d412e1700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d3c05baf0 con 0x7f4d3c0b30e0 2020-11-28T21:53:20.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.161+0000 7f4d412e1700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d3c05bd90 con 0x7f4d3c0b2d00 2020-11-28T21:53:20.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.161+0000 7f4d412e1700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d3c0b0e20 con 0x7f4d3c0b3ab0 2020-11-28T21:53:20.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.161+0000 7f4d3b7fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4d3c0b3ab0 0x7f4d3c05c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:20.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.161+0000 7f4d3bfff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f4d3c0b2d00 0x7f4d3c142ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:48502/0 (socket says 172.21.15.162:48502) 2020-11-28T21:53:20.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.161+0000 7f4d3bfff700 1 -- 172.21.15.162:0/2263612892 learned_addr learned my addr 172.21.15.162:0/2263612892 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:20.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.165+0000 7f4d3b7fe700 1 -- 172.21.15.162:0/2263612892 >> v1:172.21.15.162:6789/0 conn(0x7f4d3c0b2d00 legacy=0x7f4d3c142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:20.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.165+0000 7f4d3b7fe700 1 -- 172.21.15.162:0/2263612892 >> v1:172.21.15.118:6789/0 conn(0x7f4d3c0b30e0 legacy=0x7f4d3c13fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:20.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.165+0000 7f4d3b7fe700 1 -- 172.21.15.162:0/2263612892 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d3c05c030 con 0x7f4d3c0b3ab0 2020-11-28T21:53:20.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.165+0000 7f4d3b7fe700 1 --2- 172.21.15.162:0/2263612892 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d3c0b3ab0 0x7f4d3c05c4d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7f4d2c002f30 tx=0x7f4d2c006dc0).ready entity=mon.1 client_cookie=fcbd8eee8ef37379 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:20.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.165+0000 7f4d3affd700 1 -- 172.21.15.162:0/2263612892 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d2c0097f0 con 0x7f4d3c0b3ab0 2020-11-28T21:53:20.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.165+0000 7f4d3affd700 1 -- 172.21.15.162:0/2263612892 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d2c009950 con 0x7f4d3c0b3ab0 2020-11-28T21:53:20.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.165+0000 7f4d3affd700 1 -- 172.21.15.162:0/2263612892 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d2c009cb0 con 0x7f4d3c0b3ab0 2020-11-28T21:53:20.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.165+0000 7f4d412e1700 1 -- 172.21.15.162:0/2263612892 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d3c0b3ab0 msgr2=0x7f4d3c05c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:20.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.165+0000 7f4d412e1700 1 --2- 172.21.15.162:0/2263612892 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d3c0b3ab0 0x7f4d3c05c4d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7f4d2c002f30 tx=0x7f4d2c006dc0).stop 2020-11-28T21:53:20.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.165+0000 7f4d412e1700 1 -- 172.21.15.162:0/2263612892 shutdown_connections 2020-11-28T21:53:20.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.165+0000 7f4d412e1700 1 --2- 172.21.15.162:0/2263612892 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d3c0b3ab0 0x7f4d3c05c4d0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:20.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.165+0000 7f4d412e1700 1 -- 172.21.15.162:0/2263612892 >> 172.21.15.162:0/2263612892 conn(0x7f4d3c0604c0 msgr2=0x7f4d3c0608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:20.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.165+0000 7f4d412e1700 1 -- 172.21.15.162:0/2263612892 shutdown_connections 2020-11-28T21:53:20.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.165+0000 7f4d412e1700 1 -- 172.21.15.162:0/2263612892 wait complete. 2020-11-28T21:53:20.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.169+0000 7f4d412e1700 1 Processor -- start 2020-11-28T21:53:20.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.169+0000 7f4d412e1700 1 -- start start 2020-11-28T21:53:20.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.169+0000 7f4d412e1700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4d3c0b2d00 0x7f4d3c0b5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:20.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.169+0000 7f4d412e1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d3c0b30e0 0x7f4d3c0b92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:20.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.169+0000 7f4d412e1700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4d3c0b3ab0 0x7f4d3c0b97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:20.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.169+0000 7f4d412e1700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c0af6e0 con 0x7f4d3c0b3ab0 2020-11-28T21:53:20.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.169+0000 7f4d412e1700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c0afae0 con 0x7f4d3c0b30e0 2020-11-28T21:53:20.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.169+0000 7f4d412e1700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c0afd10 con 0x7f4d3c0b2d00 2020-11-28T21:53:20.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.169+0000 7f4d3b7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d3c0b30e0 0x7f4d3c0b92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:20.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.169+0000 7f4d3bfff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4d3c0b2d00 0x7f4d3c0b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:20.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.169+0000 7f4d3b7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d3c0b30e0 0x7f4d3c0b92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41666/0 (socket says 172.21.15.162:41666) 2020-11-28T21:53:20.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.169+0000 7f4d3b7fe700 1 -- 172.21.15.162:0/388633025 learned_addr learned my addr 172.21.15.162:0/388633025 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:20.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.169+0000 7f4d40ae0700 1 --2- 172.21.15.162:0/388633025 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4d3c0b3ab0 0x7f4d3c0b97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:20.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.169+0000 7f4d3b7fe700 1 -- 172.21.15.162:0/388633025 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4d3c0b2d00 msgr2=0x7f4d3c0b5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:20.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.169+0000 7f4d3b7fe700 1 --2- 172.21.15.162:0/388633025 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4d3c0b2d00 0x7f4d3c0b5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:20.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.169+0000 7f4d3b7fe700 1 -- 172.21.15.162:0/388633025 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4d3c0b3ab0 msgr2=0x7f4d3c0b97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:20.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.169+0000 7f4d3b7fe700 1 --2- 172.21.15.162:0/388633025 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4d3c0b3ab0 0x7f4d3c0b97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:20.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.169+0000 7f4d3b7fe700 1 -- 172.21.15.162:0/388633025 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d3c0aefa0 con 0x7f4d3c0b30e0 2020-11-28T21:53:20.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.169+0000 7f4d3b7fe700 1 --2- 172.21.15.162:0/388633025 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d3c0b30e0 0x7f4d3c0b92c0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 rx=0x7f4d3c0b9210 tx=0x7f4d2c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:20.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.173+0000 7f4d397fa700 1 -- 172.21.15.162:0/388633025 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d2c009c50 con 0x7f4d3c0b30e0 2020-11-28T21:53:20.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.173+0000 7f4d397fa700 1 -- 172.21.15.162:0/388633025 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d2c011e10 con 0x7f4d3c0b30e0 2020-11-28T21:53:20.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.173+0000 7f4d397fa700 1 -- 172.21.15.162:0/388633025 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d2c01d640 con 0x7f4d3c0b30e0 2020-11-28T21:53:20.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.173+0000 7f4d412e1700 1 -- 172.21.15.162:0/388633025 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d3c0af240 con 0x7f4d3c0b30e0 2020-11-28T21:53:20.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.173+0000 7f4d412e1700 1 -- 172.21.15.162:0/388633025 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d3c0b7b90 con 0x7f4d3c0b30e0 2020-11-28T21:53:20.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.173+0000 7f4d412e1700 1 -- 172.21.15.162:0/388633025 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d3c0af3d0 con 0x7f4d3c0b30e0 2020-11-28T21:53:20.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.173+0000 7f4d397fa700 1 -- 172.21.15.162:0/388633025 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4d2c024030 con 0x7f4d3c0b30e0 2020-11-28T21:53:20.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.173+0000 7f4d397fa700 1 --2- 172.21.15.162:0/388633025 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4d200553b0 0x7f4d20057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:20.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.173+0000 7f4d397fa700 1 -- 172.21.15.162:0/388633025 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f4d2c06f1d0 con 0x7f4d3c0b30e0 2020-11-28T21:53:20.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.173+0000 7f4d3bfff700 1 --2- 172.21.15.162:0/388633025 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4d200553b0 0x7f4d20057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:20.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.173+0000 7f4d3bfff700 1 --2- 172.21.15.162:0/388633025 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4d200553b0 0x7f4d20057870 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f4d2400d000 tx=0x7f4d24009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:20.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.177+0000 7f4d397fa700 1 -- 172.21.15.162:0/388633025 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4d2c0454a0 con 0x7f4d3c0b30e0 2020-11-28T21:53:20.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.317+0000 7f4d412e1700 1 -- 172.21.15.162:0/388633025 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f4d3c057050 con 0x7f4d3c0b30e0 2020-11-28T21:53:20.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.317+0000 7f4d397fa700 1 -- 172.21.15.162:0/388633025 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v77) v1 ==== 110+0+10 (secure 0 0 0) 0x7f4d2c022020 con 0x7f4d3c0b30e0 2020-11-28T21:53:20.326 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:53:20.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.321+0000 7f4d412e1700 1 -- 172.21.15.162:0/388633025 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4d200553b0 msgr2=0x7f4d20057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:20.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.321+0000 7f4d412e1700 1 --2- 172.21.15.162:0/388633025 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4d200553b0 0x7f4d20057870 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f4d2400d000 tx=0x7f4d24009040).stop 2020-11-28T21:53:20.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.321+0000 7f4d412e1700 1 -- 172.21.15.162:0/388633025 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d3c0b30e0 msgr2=0x7f4d3c0b92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:20.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.321+0000 7f4d412e1700 1 --2- 172.21.15.162:0/388633025 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d3c0b30e0 0x7f4d3c0b92c0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 rx=0x7f4d3c0b9210 tx=0x7f4d2c015000).stop 2020-11-28T21:53:20.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.321+0000 7f4d412e1700 1 -- 172.21.15.162:0/388633025 shutdown_connections 2020-11-28T21:53:20.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.321+0000 7f4d412e1700 1 --2- 172.21.15.162:0/388633025 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4d200553b0 0x7f4d20057870 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:20.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.321+0000 7f4d412e1700 1 --2- 172.21.15.162:0/388633025 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4d3c0b2d00 0x7f4d3c0b5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:20.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.321+0000 7f4d412e1700 1 --2- 172.21.15.162:0/388633025 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4d3c0b3ab0 0x7f4d3c0b97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:20.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.321+0000 7f4d412e1700 1 --2- 172.21.15.162:0/388633025 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d3c0b30e0 0x7f4d3c0b92c0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:20.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.321+0000 7f4d412e1700 1 -- 172.21.15.162:0/388633025 >> 172.21.15.162:0/388633025 conn(0x7f4d3c0604c0 msgr2=0x7f4d3c0608a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:20.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.321+0000 7f4d412e1700 1 -- 172.21.15.162:0/388633025 shutdown_connections 2020-11-28T21:53:20.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.321+0000 7f4d412e1700 1 -- 172.21.15.162:0/388633025 wait complete. 2020-11-28T21:53:20.339 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:53:20.458 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.453+0000 7f9ec8860700 1 Processor -- start 2020-11-28T21:53:20.459 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.453+0000 7f9ec8860700 1 -- start start 2020-11-28T21:53:20.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.453+0000 7f9ec8860700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9ec4140040 0x7f9ec40ad620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:20.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.453+0000 7f9ec8860700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ec41404c0 con 0x7f9ec40adb10 2020-11-28T21:53:20.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.453+0000 7f9ec8860700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ec40b1080 con 0x7f9ec40ad240 2020-11-28T21:53:20.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.453+0000 7f9ec8860700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec40b1490 con 0x7f9ec4140040 2020-11-28T21:53:20.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.453+0000 7f9ec2d9d700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f9ec40ad240 0x7f9ec4142aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:48516/0 (socket says 172.21.15.162:48516) 2020-11-28T21:53:20.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.453+0000 7f9ec2d9d700 1 -- 172.21.15.162:0/361316327 learned_addr learned my addr 172.21.15.162:0/361316327 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:20.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.453+0000 7f9ec259c700 1 --2- 172.21.15.162:0/361316327 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9ec4140040 0x7f9ec40ad620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:20.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.453+0000 7f9ec1d9b700 1 -- 172.21.15.162:0/361316327 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1198237946 0 0) 0x7f9ec40b1080 con 0x7f9ec40ad240 2020-11-28T21:53:20.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.453+0000 7f9ec1d9b700 1 -- 172.21.15.162:0/361316327 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ea4003230 con 0x7f9ec40ad240 2020-11-28T21:53:20.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.453+0000 7f9ec1d9b700 1 -- 172.21.15.162:0/361316327 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3692930821 0 0) 0x7f9ea4003230 con 0x7f9ec40ad240 2020-11-28T21:53:20.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.453+0000 7f9ec1d9b700 1 -- 172.21.15.162:0/361316327 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9ec4140040 msgr2=0x7f9ec40ad620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:20.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.453+0000 7f9ec1d9b700 1 --2- 172.21.15.162:0/361316327 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9ec4140040 0x7f9ec40ad620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:20.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.453+0000 7f9ec1d9b700 1 -- 172.21.15.162:0/361316327 >> v1:172.21.15.118:6789/0 conn(0x7f9ec40adb10 legacy=0x7f9ec40b1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:20.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.453+0000 7f9ec1d9b700 1 -- 172.21.15.162:0/361316327 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec413f660 con 0x7f9ec40ad240 2020-11-28T21:53:20.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.453+0000 7f9ec1d9b700 1 -- 172.21.15.162:0/361316327 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f9eac001710 con 0x7f9ec40ad240 2020-11-28T21:53:20.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.453+0000 7f9ec1d9b700 1 -- 172.21.15.162:0/361316327 >> v1:172.21.15.162:6789/0 conn(0x7f9ec40ad240 legacy=0x7f9ec4142aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:20.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.453+0000 7f9ec1d9b700 1 --2- 172.21.15.162:0/361316327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ea4003720 0x7f9ea4003b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:20.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.453+0000 7f9ec1d9b700 1 --2- 172.21.15.162:0/361316327 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9ec4140040 0x7f9ea40044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:20.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.453+0000 7f9ec1d9b700 1 --2- 172.21.15.162:0/361316327 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9ea40049c0 0x7f9ea4004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:20.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.453+0000 7f9ec1d9b700 1 -- 172.21.15.162:0/361316327 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec40b1490 con 0x7f9ea40049c0 2020-11-28T21:53:20.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.453+0000 7f9ec1d9b700 1 -- 172.21.15.162:0/361316327 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea4004130 con 0x7f9ea4003720 2020-11-28T21:53:20.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.453+0000 7f9ec1d9b700 1 -- 172.21.15.162:0/361316327 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea40042f0 con 0x7f9ec4140040 2020-11-28T21:53:20.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.453+0000 7f9ec2d9d700 1 --2- 172.21.15.162:0/361316327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ea4003720 0x7f9ea4003b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:20.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.453+0000 7f9ec259c700 1 --2- 172.21.15.162:0/361316327 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9ec4140040 0x7f9ea40044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:20.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.453+0000 7f9ec359e700 1 --2- 172.21.15.162:0/361316327 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9ea40049c0 0x7f9ea4004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:20.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.453+0000 7f9ec2d9d700 1 -- 172.21.15.162:0/361316327 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9ec4140040 msgr2=0x7f9ea40044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:20.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.453+0000 7f9ec2d9d700 1 --2- 172.21.15.162:0/361316327 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9ec4140040 0x7f9ea40044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:20.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.453+0000 7f9ec2d9d700 1 -- 172.21.15.162:0/361316327 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9ea40049c0 msgr2=0x7f9ea4004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:20.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.453+0000 7f9ec2d9d700 1 --2- 172.21.15.162:0/361316327 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9ea40049c0 0x7f9ea4004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:20.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.453+0000 7f9ec2d9d700 1 -- 172.21.15.162:0/361316327 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea4003230 con 0x7f9ea4003720 2020-11-28T21:53:20.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.453+0000 7f9ec359e700 1 --2- 172.21.15.162:0/361316327 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9ea40049c0 0x7f9ea4004e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:53:20.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.453+0000 7f9ec259c700 1 --2- 172.21.15.162:0/361316327 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9ec4140040 0x7f9ea40044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:53:20.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.453+0000 7f9ec2d9d700 1 --2- 172.21.15.162:0/361316327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ea4003720 0x7f9ea4003b00 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0x7f9eac001420 tx=0x7f9ec40b37d0).ready entity=mon.1 client_cookie=9e757a37eb826a45 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:20.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.453+0000 7f9ec1d9b700 1 -- 172.21.15.162:0/361316327 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9eac00ce00 con 0x7f9ea4003720 2020-11-28T21:53:20.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.453+0000 7f9ec1d9b700 1 -- 172.21.15.162:0/361316327 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eac0072e0 con 0x7f9ea4003720 2020-11-28T21:53:20.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.453+0000 7f9ec1d9b700 1 -- 172.21.15.162:0/361316327 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9eac0057d0 con 0x7f9ea4003720 2020-11-28T21:53:20.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.453+0000 7f9ec8860700 1 -- 172.21.15.162:0/361316327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ea4003720 msgr2=0x7f9ea4003b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:20.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.453+0000 7f9ec8860700 1 --2- 172.21.15.162:0/361316327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ea4003720 0x7f9ea4003b00 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0x7f9eac001420 tx=0x7f9ec40b37d0).stop 2020-11-28T21:53:20.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.453+0000 7f9ec359e700 1 -- 172.21.15.162:0/361316327 reap_dead start 2020-11-28T21:53:20.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.457+0000 7f9ec8860700 1 -- 172.21.15.162:0/361316327 shutdown_connections 2020-11-28T21:53:20.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.457+0000 7f9ec8860700 1 -- 172.21.15.162:0/361316327 >> 172.21.15.162:0/361316327 conn(0x7f9ec40b2830 msgr2=0x7f9ec40b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:20.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.457+0000 7f9ec8860700 1 -- 172.21.15.162:0/361316327 shutdown_connections 2020-11-28T21:53:20.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.457+0000 7f9ec8860700 1 -- 172.21.15.162:0/361316327 wait complete. 2020-11-28T21:53:20.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.457+0000 7f9ec8860700 1 Processor -- start 2020-11-28T21:53:20.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.457+0000 7f9ec8860700 1 -- start start 2020-11-28T21:53:20.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.457+0000 7f9ec8860700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ec413b350 0x7f9ec4067070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:20.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.461+0000 7f9ec8860700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9ec414c960 0x7f9ec41530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:20.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.461+0000 7f9ec8860700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9ec4143af0 0x7f9ec41535b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:20.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.461+0000 7f9ec8860700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec41431b0 con 0x7f9ec414c960 2020-11-28T21:53:20.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.461+0000 7f9ec8860700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec40affa0 con 0x7f9ec413b350 2020-11-28T21:53:20.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.461+0000 7f9ec8860700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec4063c10 con 0x7f9ec4143af0 2020-11-28T21:53:20.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.461+0000 7f9ec2d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ec413b350 0x7f9ec4067070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:20.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.461+0000 7f9ec2d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ec413b350 0x7f9ec4067070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41684/0 (socket says 172.21.15.162:41684) 2020-11-28T21:53:20.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.461+0000 7f9ec2d9d700 1 -- 172.21.15.162:0/1652274473 learned_addr learned my addr 172.21.15.162:0/1652274473 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:20.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.461+0000 7f9ec259c700 1 --2- 172.21.15.162:0/1652274473 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9ec414c960 0x7f9ec41530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:20.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.461+0000 7f9ec359e700 1 --2- 172.21.15.162:0/1652274473 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9ec4143af0 0x7f9ec41535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:20.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.461+0000 7f9ec2d9d700 1 -- 172.21.15.162:0/1652274473 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9ec4143af0 msgr2=0x7f9ec41535b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:20.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.461+0000 7f9ec2d9d700 1 --2- 172.21.15.162:0/1652274473 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9ec4143af0 0x7f9ec41535b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:20.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.461+0000 7f9ec2d9d700 1 -- 172.21.15.162:0/1652274473 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9ec414c960 msgr2=0x7f9ec41530c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:20.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.461+0000 7f9ec2d9d700 1 --2- 172.21.15.162:0/1652274473 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9ec414c960 0x7f9ec41530c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:20.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.461+0000 7f9ec2d9d700 1 -- 172.21.15.162:0/1652274473 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec4063920 con 0x7f9ec413b350 2020-11-28T21:53:20.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.461+0000 7f9ec359e700 1 --2- 172.21.15.162:0/1652274473 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9ec4143af0 0x7f9ec41535b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:53:20.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.461+0000 7f9ec2d9d700 1 --2- 172.21.15.162:0/1652274473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ec413b350 0x7f9ec4067070 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 rx=0x7f9ec4066110 tx=0x7f9eac015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:20.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.461+0000 7f9ebbfff700 1 -- 172.21.15.162:0/1652274473 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9eac0048a0 con 0x7f9ec413b350 2020-11-28T21:53:20.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.461+0000 7f9ebbfff700 1 -- 172.21.15.162:0/1652274473 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eac004a00 con 0x7f9ec413b350 2020-11-28T21:53:20.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.461+0000 7f9ec8860700 1 -- 172.21.15.162:0/1652274473 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ec4063e40 con 0x7f9ec413b350 2020-11-28T21:53:20.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.461+0000 7f9ebbfff700 1 -- 172.21.15.162:0/1652274473 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9eac004d60 con 0x7f9ec413b350 2020-11-28T21:53:20.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.461+0000 7f9ec8860700 1 -- 172.21.15.162:0/1652274473 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ec4064210 con 0x7f9ec413b350 2020-11-28T21:53:20.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.461+0000 7f9ec8860700 1 -- 172.21.15.162:0/1652274473 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ec4057050 con 0x7f9ec413b350 2020-11-28T21:53:20.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.461+0000 7f9ebbfff700 1 -- 172.21.15.162:0/1652274473 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9eac010070 con 0x7f9ec413b350 2020-11-28T21:53:20.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.461+0000 7f9ebbfff700 1 --2- 172.21.15.162:0/1652274473 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9ea80553b0 0x7f9ea8057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:20.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.465+0000 7f9ebbfff700 1 -- 172.21.15.162:0/1652274473 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f9eac06ea50 con 0x7f9ec413b350 2020-11-28T21:53:20.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.465+0000 7f9ec259c700 1 --2- 172.21.15.162:0/1652274473 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9ea80553b0 0x7f9ea8057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:20.485 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.465+0000 7f9ec259c700 1 --2- 172.21.15.162:0/1652274473 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9ea80553b0 0x7f9ea8057870 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f9eb4008000 tx=0x7f9eb4008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:20.485 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.469+0000 7f9ebbfff700 1 -- 172.21.15.162:0/1652274473 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9eac044c10 con 0x7f9ec413b350 2020-11-28T21:53:20.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.609+0000 7f9ec8860700 1 -- 172.21.15.162:0/1652274473 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9ec414c700 con 0x7f9ec413b350 2020-11-28T21:53:20.615 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.609+0000 7f9ebbfff700 1 -- 172.21.15.162:0/1652274473 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v77) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9eac024030 con 0x7f9ec413b350 2020-11-28T21:53:20.615 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:53:20.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.609+0000 7f9ec8860700 1 -- 172.21.15.162:0/1652274473 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9ea80553b0 msgr2=0x7f9ea8057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:20.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.609+0000 7f9ec8860700 1 --2- 172.21.15.162:0/1652274473 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9ea80553b0 0x7f9ea8057870 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f9eb4008000 tx=0x7f9eb4008040).stop 2020-11-28T21:53:20.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.609+0000 7f9ec8860700 1 -- 172.21.15.162:0/1652274473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ec413b350 msgr2=0x7f9ec4067070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:20.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.609+0000 7f9ec8860700 1 --2- 172.21.15.162:0/1652274473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ec413b350 0x7f9ec4067070 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 rx=0x7f9ec4066110 tx=0x7f9eac015000).stop 2020-11-28T21:53:20.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.609+0000 7f9ec8860700 1 -- 172.21.15.162:0/1652274473 shutdown_connections 2020-11-28T21:53:20.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.609+0000 7f9ec8860700 1 --2- 172.21.15.162:0/1652274473 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9ea80553b0 0x7f9ea8057870 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:20.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.609+0000 7f9ec8860700 1 --2- 172.21.15.162:0/1652274473 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9ec4143af0 0x7f9ec41535b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:20.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.609+0000 7f9ec8860700 1 --2- 172.21.15.162:0/1652274473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ec413b350 0x7f9ec4067070 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:20.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.609+0000 7f9ec8860700 1 --2- 172.21.15.162:0/1652274473 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9ec414c960 0x7f9ec41530c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:20.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.609+0000 7f9ec8860700 1 -- 172.21.15.162:0/1652274473 >> 172.21.15.162:0/1652274473 conn(0x7f9ec40b2830 msgr2=0x7f9ec405f380 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:20.620 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.609+0000 7f9ec8860700 1 -- 172.21.15.162:0/1652274473 shutdown_connections 2020-11-28T21:53:20.620 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.609+0000 7f9ec8860700 1 -- 172.21.15.162:0/1652274473 wait complete. 2020-11-28T21:53:20.632 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-28T21:53:20.748 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.741+0000 7f9e9f1b2700 1 Processor -- start 2020-11-28T21:53:20.749 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.741+0000 7f9e9f1b2700 1 -- start start 2020-11-28T21:53:20.750 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.741+0000 7f9e9f1b2700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f9e980b21f0 0x7f9e9813fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:20.750 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.741+0000 7f9e9f1b2700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e980aff20 con 0x7f9e981402a0 2020-11-28T21:53:20.750 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.741+0000 7f9e9f1b2700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e98140750 con 0x7f9e98060b60 2020-11-28T21:53:20.750 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.741+0000 7f9e9f1b2700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e980af240 con 0x7f9e980b21f0 2020-11-28T21:53:20.751 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.741+0000 7f9e9e9b1700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f9e98060b60 0x7f9e9814d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:48540/0 (socket says 172.21.15.162:48540) 2020-11-28T21:53:20.751 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.741+0000 7f9e9e9b1700 1 -- 172.21.15.162:0/881992778 learned_addr learned my addr 172.21.15.162:0/881992778 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:20.751 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.745+0000 7f9e9e1b0700 1 --2- 172.21.15.162:0/881992778 >> v2:172.21.15.118:3300/0 conn(0x7f9e980b21f0 0x7f9e9813fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:20.751 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.745+0000 7f9e9d1ae700 1 -- 172.21.15.162:0/881992778 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1654444409 0 0) 0x7f9e98140750 con 0x7f9e98060b60 2020-11-28T21:53:20.752 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.745+0000 7f9e9d1ae700 1 -- 172.21.15.162:0/881992778 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e84003230 con 0x7f9e98060b60 2020-11-28T21:53:20.753 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.745+0000 7f9e9d1ae700 1 -- 172.21.15.162:0/881992778 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2978324765 0 0) 0x7f9e980aff20 con 0x7f9e981402a0 2020-11-28T21:53:20.754 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.745+0000 7f9e9d1ae700 1 -- 172.21.15.162:0/881992778 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e98140750 con 0x7f9e981402a0 2020-11-28T21:53:20.754 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.745+0000 7f9e9e1b0700 1 -- 172.21.15.162:0/881992778 >> v1:172.21.15.162:6789/0 conn(0x7f9e98060b60 legacy=0x7f9e9814d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:20.754 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.745+0000 7f9e9e1b0700 1 -- 172.21.15.162:0/881992778 >> v1:172.21.15.118:6789/0 conn(0x7f9e981402a0 legacy=0x7f9e98060450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:20.754 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.745+0000 7f9e9e1b0700 1 -- 172.21.15.162:0/881992778 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e980b25d0 con 0x7f9e980b21f0 2020-11-28T21:53:20.756 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.745+0000 7f9e9e1b0700 1 --2- 172.21.15.162:0/881992778 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e980b21f0 0x7f9e9813fdb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f9e90002f30 tx=0x7f9e90006dc0).ready entity=mon.0 client_cookie=64f7d8c5039327c2 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:20.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.745+0000 7f9e9d1ae700 1 -- 172.21.15.162:0/881992778 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e90009740 con 0x7f9e980b21f0 2020-11-28T21:53:20.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.745+0000 7f9e9d1ae700 1 -- 172.21.15.162:0/881992778 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e900098a0 con 0x7f9e980b21f0 2020-11-28T21:53:20.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.745+0000 7f9e9d1ae700 1 -- 172.21.15.162:0/881992778 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e90009c00 con 0x7f9e980b21f0 2020-11-28T21:53:20.758 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.745+0000 7f9e9f1b2700 1 -- 172.21.15.162:0/881992778 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e980b21f0 msgr2=0x7f9e9813fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:20.758 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.745+0000 7f9e9f1b2700 1 --2- 172.21.15.162:0/881992778 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e980b21f0 0x7f9e9813fdb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f9e90002f30 tx=0x7f9e90006dc0).stop 2020-11-28T21:53:20.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.745+0000 7f9e9f1b2700 1 -- 172.21.15.162:0/881992778 shutdown_connections 2020-11-28T21:53:20.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.745+0000 7f9e9f1b2700 1 --2- 172.21.15.162:0/881992778 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e980b21f0 0x7f9e9813fdb0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:20.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.745+0000 7f9e9f1b2700 1 -- 172.21.15.162:0/881992778 >> 172.21.15.162:0/881992778 conn(0x7f9e980b57d0 msgr2=0x7f9e980b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:20.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.745+0000 7f9e9f1b2700 1 -- 172.21.15.162:0/881992778 shutdown_connections 2020-11-28T21:53:20.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.745+0000 7f9e9f1b2700 1 -- 172.21.15.162:0/881992778 wait complete. 2020-11-28T21:53:20.766 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.749+0000 7f9e9f1b2700 1 Processor -- start 2020-11-28T21:53:20.766 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.749+0000 7f9e9f1b2700 1 -- start start 2020-11-28T21:53:20.767 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.749+0000 7f9e9f1b2700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9e98060b60 0x7f9e980b4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:20.767 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.749+0000 7f9e9f1b2700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e980b21f0 0x7f9e981448a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:20.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.749+0000 7f9e9f1b2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e981402a0 0x7f9e98144d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:20.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.749+0000 7f9e9f1b2700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e98146f10 con 0x7f9e980b21f0 2020-11-28T21:53:20.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.749+0000 7f9e9f1b2700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e98146120 con 0x7f9e981402a0 2020-11-28T21:53:20.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.749+0000 7f9e9f1b2700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e981442a0 con 0x7f9e98060b60 2020-11-28T21:53:20.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.749+0000 7f9e9e9b1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e981402a0 0x7f9e98144d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:20.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.749+0000 7f9e9e9b1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e981402a0 0x7f9e98144d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41702/0 (socket says 172.21.15.162:41702) 2020-11-28T21:53:20.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.749+0000 7f9e9e9b1700 1 -- 172.21.15.162:0/3341358379 learned_addr learned my addr 172.21.15.162:0/3341358379 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:20.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.749+0000 7f9e9e1b0700 1 --2- 172.21.15.162:0/3341358379 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9e98060b60 0x7f9e980b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:20.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.749+0000 7f9e9d9af700 1 --2- 172.21.15.162:0/3341358379 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e980b21f0 0x7f9e981448a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:20.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.749+0000 7f9e9e9b1700 1 -- 172.21.15.162:0/3341358379 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9e98060b60 msgr2=0x7f9e980b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:20.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.749+0000 7f9e9e9b1700 1 --2- 172.21.15.162:0/3341358379 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9e98060b60 0x7f9e980b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:20.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.749+0000 7f9e9e9b1700 1 -- 172.21.15.162:0/3341358379 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e980b21f0 msgr2=0x7f9e981448a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:20.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.749+0000 7f9e9e9b1700 1 --2- 172.21.15.162:0/3341358379 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e980b21f0 0x7f9e981448a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:20.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.749+0000 7f9e9e9b1700 1 -- 172.21.15.162:0/3341358379 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e98064770 con 0x7f9e981402a0 2020-11-28T21:53:20.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.749+0000 7f9e9e9b1700 1 --2- 172.21.15.162:0/3341358379 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e981402a0 0x7f9e98144d90 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7f9e98066d00 tx=0x7f9e88007a50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:20.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.749+0000 7f9e977fe700 1 -- 172.21.15.162:0/3341358379 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e880072e0 con 0x7f9e981402a0 2020-11-28T21:53:20.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.749+0000 7f9e9f1b2700 1 -- 172.21.15.162:0/3341358379 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e98064a10 con 0x7f9e981402a0 2020-11-28T21:53:20.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.749+0000 7f9e977fe700 1 -- 172.21.15.162:0/3341358379 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e88007440 con 0x7f9e981402a0 2020-11-28T21:53:20.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.749+0000 7f9e977fe700 1 -- 172.21.15.162:0/3341358379 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e880057d0 con 0x7f9e981402a0 2020-11-28T21:53:20.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.749+0000 7f9e9f1b2700 1 -- 172.21.15.162:0/3341358379 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e98065150 con 0x7f9e981402a0 2020-11-28T21:53:20.776 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.753+0000 7f9e977fe700 1 -- 172.21.15.162:0/3341358379 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9e88010070 con 0x7f9e981402a0 2020-11-28T21:53:20.776 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.753+0000 7f9e9f1b2700 1 -- 172.21.15.162:0/3341358379 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e98057050 con 0x7f9e981402a0 2020-11-28T21:53:20.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.753+0000 7f9e977fe700 1 --2- 172.21.15.162:0/3341358379 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9e78055330 0x7f9e780577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:20.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.753+0000 7f9e977fe700 1 -- 172.21.15.162:0/3341358379 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f9e8806d610 con 0x7f9e981402a0 2020-11-28T21:53:20.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.753+0000 7f9e9e1b0700 1 --2- 172.21.15.162:0/3341358379 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9e78055330 0x7f9e780577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:20.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.753+0000 7f9e9e1b0700 1 --2- 172.21.15.162:0/3341358379 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9e78055330 0x7f9e780577f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f9e90019000 tx=0x7f9e90015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:20.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.761+0000 7f9e977fe700 1 -- 172.21.15.162:0/3341358379 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9e88043840 con 0x7f9e981402a0 2020-11-28T21:53:20.901 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.893+0000 7f9e9f1b2700 1 -- 172.21.15.162:0/3341358379 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9e980ac1f0 con 0x7f9e981402a0 2020-11-28T21:53:20.902 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.897+0000 7f9e977fe700 1 -- 172.21.15.162:0/3341358379 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) v1 ==== 104+0+10 (secure 0 0 0) 0x7f9e88046e60 con 0x7f9e981402a0 2020-11-28T21:53:20.902 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:53:20.904 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.897+0000 7f9e9f1b2700 1 -- 172.21.15.162:0/3341358379 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9e78055330 msgr2=0x7f9e780577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:20.904 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.897+0000 7f9e9f1b2700 1 --2- 172.21.15.162:0/3341358379 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9e78055330 0x7f9e780577f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f9e90019000 tx=0x7f9e90015000).stop 2020-11-28T21:53:20.904 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.897+0000 7f9e9f1b2700 1 -- 172.21.15.162:0/3341358379 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e981402a0 msgr2=0x7f9e98144d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:20.905 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.897+0000 7f9e9f1b2700 1 --2- 172.21.15.162:0/3341358379 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e981402a0 0x7f9e98144d90 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7f9e98066d00 tx=0x7f9e88007a50).stop 2020-11-28T21:53:20.905 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.897+0000 7f9e9f1b2700 1 -- 172.21.15.162:0/3341358379 shutdown_connections 2020-11-28T21:53:20.905 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.897+0000 7f9e9f1b2700 1 --2- 172.21.15.162:0/3341358379 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9e78055330 0x7f9e780577f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:20.905 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.897+0000 7f9e9f1b2700 1 --2- 172.21.15.162:0/3341358379 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9e98060b60 0x7f9e980b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:20.905 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.897+0000 7f9e9f1b2700 1 --2- 172.21.15.162:0/3341358379 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e981402a0 0x7f9e98144d90 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:20.906 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.897+0000 7f9e9f1b2700 1 --2- 172.21.15.162:0/3341358379 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e980b21f0 0x7f9e981448a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:20.906 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.897+0000 7f9e9f1b2700 1 -- 172.21.15.162:0/3341358379 >> 172.21.15.162:0/3341358379 conn(0x7f9e980b57d0 msgr2=0x7f9e9813e350 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:20.906 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.897+0000 7f9e9f1b2700 1 -- 172.21.15.162:0/3341358379 shutdown_connections 2020-11-28T21:53:20.906 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:20.897+0000 7f9e9f1b2700 1 -- 172.21.15.162:0/3341358379 wait complete. 2020-11-28T21:53:20.916 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-28T21:53:21.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.025+0000 7f7891896700 1 Processor -- start 2020-11-28T21:53:21.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.025+0000 7f7891896700 1 -- start start 2020-11-28T21:53:21.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.025+0000 7f7891896700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f788c0b1900 0x7f788c0b1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:21.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.025+0000 7f7891896700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f788c1404c0 con 0x7f788c0ad240 2020-11-28T21:53:21.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.025+0000 7f7891896700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f788c0b1080 con 0x7f788c140040 2020-11-28T21:53:21.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.025+0000 7f7891896700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f788c0b1490 con 0x7f788c0b1900 2020-11-28T21:53:21.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.025+0000 7f788bfff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f788c140040 0x7f788c0ad620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:48552/0 (socket says 172.21.15.162:48552) 2020-11-28T21:53:21.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.025+0000 7f788bfff700 1 -- 172.21.15.162:0/1085506302 learned_addr learned my addr 172.21.15.162:0/1085506302 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:21.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.025+0000 7f7891095700 1 --2- 172.21.15.162:0/1085506302 >> v2:172.21.15.118:3300/0 conn(0x7f788c0b1900 0x7f788c0b1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:21.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.025+0000 7f788b7fe700 1 -- 172.21.15.162:0/1085506302 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1563723458 0 0) 0x7f788c0b1080 con 0x7f788c140040 2020-11-28T21:53:21.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.025+0000 7f788b7fe700 1 -- 172.21.15.162:0/1085506302 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f786c003230 con 0x7f788c140040 2020-11-28T21:53:21.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.029+0000 7f788b7fe700 1 -- 172.21.15.162:0/1085506302 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3748172682 0 0) 0x7f786c003230 con 0x7f788c140040 2020-11-28T21:53:21.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.029+0000 7f7891095700 1 -- 172.21.15.162:0/1085506302 >> v1:172.21.15.162:6789/0 conn(0x7f788c140040 legacy=0x7f788c0ad620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:21.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.029+0000 7f7891095700 1 -- 172.21.15.162:0/1085506302 >> v1:172.21.15.118:6789/0 conn(0x7f788c0ad240 legacy=0x7f788c142aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:21.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.029+0000 7f7891095700 1 -- 172.21.15.162:0/1085506302 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f788c13f660 con 0x7f788c0b1900 2020-11-28T21:53:21.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.029+0000 7f7891095700 1 --2- 172.21.15.162:0/1085506302 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f788c0b1900 0x7f788c0b1d90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f7874001870 tx=0x7f787c006dc0).ready entity=mon.0 client_cookie=da5867d0dab165df server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:21.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.029+0000 7f788b7fe700 1 -- 172.21.15.162:0/1085506302 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78740016d0 con 0x7f788c0b1900 2020-11-28T21:53:21.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.029+0000 7f788b7fe700 1 -- 172.21.15.162:0/1085506302 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f787c00f820 con 0x7f788c0b1900 2020-11-28T21:53:21.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.029+0000 7f788b7fe700 1 -- 172.21.15.162:0/1085506302 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f787c00fb80 con 0x7f788c0b1900 2020-11-28T21:53:21.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.029+0000 7f7891896700 1 -- 172.21.15.162:0/1085506302 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f788c0b1900 msgr2=0x7f788c0b1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:21.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.029+0000 7f7891896700 1 --2- 172.21.15.162:0/1085506302 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f788c0b1900 0x7f788c0b1d90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f7874001870 tx=0x7f787c006dc0).stop 2020-11-28T21:53:21.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.029+0000 7f7891896700 1 -- 172.21.15.162:0/1085506302 shutdown_connections 2020-11-28T21:53:21.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.029+0000 7f7891896700 1 --2- 172.21.15.162:0/1085506302 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f788c0b1900 0x7f788c0b1d90 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:21.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.029+0000 7f7891896700 1 -- 172.21.15.162:0/1085506302 >> 172.21.15.162:0/1085506302 conn(0x7f788c0b2830 msgr2=0x7f788c0b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:21.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.029+0000 7f7891896700 1 -- 172.21.15.162:0/1085506302 shutdown_connections 2020-11-28T21:53:21.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.029+0000 7f7891896700 1 -- 172.21.15.162:0/1085506302 wait complete. 2020-11-28T21:53:21.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.029+0000 7f7891896700 1 Processor -- start 2020-11-28T21:53:21.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.029+0000 7f7891896700 1 -- start start 2020-11-28T21:53:21.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.029+0000 7f7891896700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f788c0ad240 0x7f788c0afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:21.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.029+0000 7f7891896700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f788c0b1900 0x7f788c064ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:21.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.029+0000 7f7891896700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f788c140040 0x7f788c064fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:21.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.029+0000 7f7891896700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788c1431b0 con 0x7f788c0b1900 2020-11-28T21:53:21.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.029+0000 7f7891896700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788c142d40 con 0x7f788c140040 2020-11-28T21:53:21.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.029+0000 7f7891896700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f788c05eb10 con 0x7f788c0ad240 2020-11-28T21:53:21.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.029+0000 7f7891095700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f788c140040 0x7f788c064fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:21.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.029+0000 7f7891095700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f788c140040 0x7f788c064fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41716/0 (socket says 172.21.15.162:41716) 2020-11-28T21:53:21.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.029+0000 7f7891095700 1 -- 172.21.15.162:0/3245697820 learned_addr learned my addr 172.21.15.162:0/3245697820 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:21.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.029+0000 7f788bfff700 1 --2- 172.21.15.162:0/3245697820 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f788c0b1900 0x7f788c064ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:21.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.029+0000 7f7890894700 1 --2- 172.21.15.162:0/3245697820 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f788c0ad240 0x7f788c0afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:21.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.029+0000 7f7891095700 1 -- 172.21.15.162:0/3245697820 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f788c0ad240 msgr2=0x7f788c0afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:21.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.029+0000 7f7891095700 1 --2- 172.21.15.162:0/3245697820 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f788c0ad240 0x7f788c0afdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:21.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.029+0000 7f7891095700 1 -- 172.21.15.162:0/3245697820 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f788c0b1900 msgr2=0x7f788c064ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:21.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.029+0000 7f7891095700 1 --2- 172.21.15.162:0/3245697820 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f788c0b1900 0x7f788c064ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:21.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.029+0000 7f7891095700 1 -- 172.21.15.162:0/3245697820 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f788c0af3b0 con 0x7f788c140040 2020-11-28T21:53:21.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.029+0000 7f788bfff700 1 --2- 172.21.15.162:0/3245697820 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f788c0b1900 0x7f788c064ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:53:21.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.029+0000 7f7890894700 1 --2- 172.21.15.162:0/3245697820 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f788c0ad240 0x7f788c0afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:53:21.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.029+0000 7f7891095700 1 --2- 172.21.15.162:0/3245697820 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f788c140040 0x7f788c064fd0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 rx=0x7f7874001870 tx=0x7f787c017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:21.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.033+0000 7f7889ffb700 1 -- 172.21.15.162:0/3245697820 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f787c01a480 con 0x7f788c140040 2020-11-28T21:53:21.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.033+0000 7f7889ffb700 1 -- 172.21.15.162:0/3245697820 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f787c01a5e0 con 0x7f788c140040 2020-11-28T21:53:21.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.033+0000 7f7891896700 1 -- 172.21.15.162:0/3245697820 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f788c0af610 con 0x7f788c140040 2020-11-28T21:53:21.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.033+0000 7f7889ffb700 1 -- 172.21.15.162:0/3245697820 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f787c01a940 con 0x7f788c140040 2020-11-28T21:53:21.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.033+0000 7f7891896700 1 -- 172.21.15.162:0/3245697820 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f788c0aefe0 con 0x7f788c140040 2020-11-28T21:53:21.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.033+0000 7f7891896700 1 -- 172.21.15.162:0/3245697820 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f788c057050 con 0x7f788c140040 2020-11-28T21:53:21.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.033+0000 7f7889ffb700 1 -- 172.21.15.162:0/3245697820 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f787c02a070 con 0x7f788c140040 2020-11-28T21:53:21.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.033+0000 7f7889ffb700 1 --2- 172.21.15.162:0/3245697820 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7870055330 0x7f78700577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:21.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.033+0000 7f7890894700 1 --2- 172.21.15.162:0/3245697820 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7870055330 0x7f78700577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:21.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.033+0000 7f7889ffb700 1 -- 172.21.15.162:0/3245697820 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f787c06f7b0 con 0x7f788c140040 2020-11-28T21:53:21.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.033+0000 7f7890894700 1 --2- 172.21.15.162:0/3245697820 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7870055330 0x7f78700577f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f7884008000 tx=0x7f7884009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:21.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.045+0000 7f7889ffb700 1 -- 172.21.15.162:0/3245697820 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f787c0459e0 con 0x7f788c140040 2020-11-28T21:53:21.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.189+0000 7f7891896700 1 -- 172.21.15.162:0/3245697820 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f788c068350 con 0x7f788c140040 2020-11-28T21:53:21.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.189+0000 7f7889ffb700 1 -- 172.21.15.162:0/3245697820 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) v1 ==== 100+0+10 (secure 0 0 0) 0x7f787c049000 con 0x7f788c140040 2020-11-28T21:53:21.195 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:53:21.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.189+0000 7f7891896700 1 -- 172.21.15.162:0/3245697820 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7870055330 msgr2=0x7f78700577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:21.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.189+0000 7f7891896700 1 --2- 172.21.15.162:0/3245697820 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7870055330 0x7f78700577f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f7884008000 tx=0x7f7884009040).stop 2020-11-28T21:53:21.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.189+0000 7f7891896700 1 -- 172.21.15.162:0/3245697820 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f788c140040 msgr2=0x7f788c064fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:21.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.189+0000 7f7891896700 1 --2- 172.21.15.162:0/3245697820 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f788c140040 0x7f788c064fd0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 rx=0x7f7874001870 tx=0x7f787c017000).stop 2020-11-28T21:53:21.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.189+0000 7f7891896700 1 -- 172.21.15.162:0/3245697820 shutdown_connections 2020-11-28T21:53:21.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.189+0000 7f7891896700 1 --2- 172.21.15.162:0/3245697820 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7870055330 0x7f78700577f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:21.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.189+0000 7f7891896700 1 --2- 172.21.15.162:0/3245697820 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f788c0ad240 0x7f788c0afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:21.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.189+0000 7f7891896700 1 --2- 172.21.15.162:0/3245697820 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f788c140040 0x7f788c064fd0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:21.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.189+0000 7f7891896700 1 --2- 172.21.15.162:0/3245697820 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f788c0b1900 0x7f788c064ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:21.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.189+0000 7f7891896700 1 -- 172.21.15.162:0/3245697820 >> 172.21.15.162:0/3245697820 conn(0x7f788c0b2830 msgr2=0x7f788c0618b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:21.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.189+0000 7f7891896700 1 -- 172.21.15.162:0/3245697820 shutdown_connections 2020-11-28T21:53:21.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.189+0000 7f7891896700 1 -- 172.21.15.162:0/3245697820 wait complete. 2020-11-28T21:53:21.208 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-28T21:53:21.209 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:53:21.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.309+0000 7ff02ecd5700 1 Processor -- start 2020-11-28T21:53:21.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.309+0000 7ff02ecd5700 1 -- start start 2020-11-28T21:53:21.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.309+0000 7ff02ecd5700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff0280b11b0 0x7ff028140fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:21.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.309+0000 7ff02ecd5700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff0280b1d70 0x7ff02814a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:21.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.309+0000 7ff02ecd5700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0280b2870 con 0x7ff0280b1990 2020-11-28T21:53:21.318 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.309+0000 7ff02ecd5700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff028146e20 con 0x7ff0280b11b0 2020-11-28T21:53:21.318 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.309+0000 7ff02ecd5700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0280b22e0 con 0x7ff0280b1d70 2020-11-28T21:53:21.318 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.309+0000 7ff02ca71700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7ff0280b1990 0x7ff028140890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:48564/0 (socket says 172.21.15.162:48564) 2020-11-28T21:53:21.318 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.309+0000 7ff02ca71700 1 -- 172.21.15.162:0/764941923 learned_addr learned my addr 172.21.15.162:0/764941923 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:21.318 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.309+0000 7ff027fff700 1 --2- 172.21.15.162:0/764941923 >> v2:172.21.15.162:3300/0 conn(0x7ff0280b11b0 0x7ff028140fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:21.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.309+0000 7ff02d272700 1 --2- 172.21.15.162:0/764941923 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff0280b1d70 0x7ff02814a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:21.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.309+0000 7ff0277fe700 1 -- 172.21.15.162:0/764941923 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3762854429 0 0) 0x7ff0280b2870 con 0x7ff0280b1990 2020-11-28T21:53:21.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.309+0000 7ff0277fe700 1 -- 172.21.15.162:0/764941923 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff014003230 con 0x7ff0280b1990 2020-11-28T21:53:21.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.309+0000 7ff027fff700 1 -- 172.21.15.162:0/764941923 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff0280b1d70 msgr2=0x7ff02814a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:21.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.309+0000 7ff027fff700 1 --2- 172.21.15.162:0/764941923 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff0280b1d70 0x7ff02814a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:21.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.309+0000 7ff027fff700 1 -- 172.21.15.162:0/764941923 >> v1:172.21.15.162:6789/0 conn(0x7ff0280b1990 legacy=0x7ff028140890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:21.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.309+0000 7ff027fff700 1 -- 172.21.15.162:0/764941923 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0280b0740 con 0x7ff0280b11b0 2020-11-28T21:53:21.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.309+0000 7ff027fff700 1 --2- 172.21.15.162:0/764941923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0280b11b0 0x7ff028140fa0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 rx=0x7ff01c002f30 tx=0x7ff01c006dc0).ready entity=mon.1 client_cookie=372b1e0b02d9195 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:21.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.309+0000 7ff0277fe700 1 -- 172.21.15.162:0/764941923 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff01c0096a0 con 0x7ff0280b11b0 2020-11-28T21:53:21.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.309+0000 7ff0277fe700 1 -- 172.21.15.162:0/764941923 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff01c009800 con 0x7ff0280b11b0 2020-11-28T21:53:21.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.309+0000 7ff0277fe700 1 -- 172.21.15.162:0/764941923 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff01c009b60 con 0x7ff0280b11b0 2020-11-28T21:53:21.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.309+0000 7ff02ecd5700 1 -- 172.21.15.162:0/764941923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0280b11b0 msgr2=0x7ff028140fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:21.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.309+0000 7ff02ecd5700 1 --2- 172.21.15.162:0/764941923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0280b11b0 0x7ff028140fa0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 rx=0x7ff01c002f30 tx=0x7ff01c006dc0).stop 2020-11-28T21:53:21.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.313+0000 7ff02ecd5700 1 -- 172.21.15.162:0/764941923 shutdown_connections 2020-11-28T21:53:21.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.313+0000 7ff02ecd5700 1 --2- 172.21.15.162:0/764941923 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff0280b1d70 0x7ff02814a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:21.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.313+0000 7ff02ecd5700 1 --2- 172.21.15.162:0/764941923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0280b11b0 0x7ff028140fa0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:21.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.313+0000 7ff02ecd5700 1 -- 172.21.15.162:0/764941923 >> 172.21.15.162:0/764941923 conn(0x7ff02805b970 msgr2=0x7ff02805bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:21.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.313+0000 7ff02ecd5700 1 -- 172.21.15.162:0/764941923 shutdown_connections 2020-11-28T21:53:21.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.313+0000 7ff02ecd5700 1 -- 172.21.15.162:0/764941923 wait complete. 2020-11-28T21:53:21.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.313+0000 7ff02ecd5700 1 Processor -- start 2020-11-28T21:53:21.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.313+0000 7ff02ecd5700 1 -- start start 2020-11-28T21:53:21.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.313+0000 7ff02ecd5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0280b11b0 0x7ff0280662d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:21.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.313+0000 7ff02ecd5700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff0280b1990 0x7ff028066950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:21.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.313+0000 7ff02ecd5700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff0280b1d70 0x7ff028155c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:21.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.313+0000 7ff02ecd5700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff028064260 con 0x7ff0280b1990 2020-11-28T21:53:21.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.313+0000 7ff02ecd5700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff028144800 con 0x7ff0280b11b0 2020-11-28T21:53:21.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.313+0000 7ff02ecd5700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff028066e40 con 0x7ff0280b1d70 2020-11-28T21:53:21.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.313+0000 7ff02ca71700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0280b11b0 0x7ff0280662d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:21.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.313+0000 7ff02ca71700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0280b11b0 0x7ff0280662d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41726/0 (socket says 172.21.15.162:41726) 2020-11-28T21:53:21.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.313+0000 7ff02ca71700 1 -- 172.21.15.162:0/3029834615 learned_addr learned my addr 172.21.15.162:0/3029834615 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:21.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.313+0000 7ff02d272700 1 --2- 172.21.15.162:0/3029834615 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff0280b1d70 0x7ff028155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:21.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.313+0000 7ff027fff700 1 --2- 172.21.15.162:0/3029834615 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff0280b1990 0x7ff028066950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:21.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.313+0000 7ff02ca71700 1 -- 172.21.15.162:0/3029834615 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff0280b1d70 msgr2=0x7ff028155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:21.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.313+0000 7ff02ca71700 1 --2- 172.21.15.162:0/3029834615 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff0280b1d70 0x7ff028155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:21.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.313+0000 7ff02ca71700 1 -- 172.21.15.162:0/3029834615 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff0280b1990 msgr2=0x7ff028066950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:21.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.313+0000 7ff02ca71700 1 --2- 172.21.15.162:0/3029834615 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff0280b1990 0x7ff028066950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:21.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.313+0000 7ff02ca71700 1 -- 172.21.15.162:0/3029834615 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff028063fd0 con 0x7ff0280b11b0 2020-11-28T21:53:21.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.313+0000 7ff027fff700 1 --2- 172.21.15.162:0/3029834615 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff0280b1990 0x7ff028066950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:53:21.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.317+0000 7ff02ca71700 1 --2- 172.21.15.162:0/3029834615 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0280b11b0 0x7ff0280662d0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 rx=0x7ff018001450 tx=0x7ff018007e20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:21.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.317+0000 7ff025ffb700 1 -- 172.21.15.162:0/3029834615 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff018005b70 con 0x7ff0280b11b0 2020-11-28T21:53:21.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.317+0000 7ff025ffb700 1 -- 172.21.15.162:0/3029834615 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff018005cd0 con 0x7ff0280b11b0 2020-11-28T21:53:21.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.317+0000 7ff02ecd5700 1 -- 172.21.15.162:0/3029834615 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff028144560 con 0x7ff0280b11b0 2020-11-28T21:53:21.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.317+0000 7ff025ffb700 1 -- 172.21.15.162:0/3029834615 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff018007300 con 0x7ff0280b11b0 2020-11-28T21:53:21.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.317+0000 7ff02ecd5700 1 -- 172.21.15.162:0/3029834615 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff028143a30 con 0x7ff0280b11b0 2020-11-28T21:53:21.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.317+0000 7ff02ecd5700 1 -- 172.21.15.162:0/3029834615 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0281562c0 con 0x7ff0280b11b0 2020-11-28T21:53:21.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.317+0000 7ff025ffb700 1 -- 172.21.15.162:0/3029834615 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff018010030 con 0x7ff0280b11b0 2020-11-28T21:53:21.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.317+0000 7ff025ffb700 1 --2- 172.21.15.162:0/3029834615 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff0080554e0 0x7ff0080579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:21.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.317+0000 7ff025ffb700 1 -- 172.21.15.162:0/3029834615 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6116+0+0 (secure 0 0 0) 0x7ff01806dff0 con 0x7ff0280b11b0 2020-11-28T21:53:21.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.317+0000 7ff027fff700 1 --2- 172.21.15.162:0/3029834615 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff0080554e0 0x7ff0080579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:21.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.317+0000 7ff027fff700 1 --2- 172.21.15.162:0/3029834615 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff0080554e0 0x7ff0080579a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7ff01c016000 tx=0x7ff01c015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:21.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.321+0000 7ff025ffb700 1 -- 172.21.15.162:0/3029834615 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff0180442c0 con 0x7ff0280b11b0 2020-11-28T21:53:21.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.505+0000 7ff02ecd5700 1 -- 172.21.15.162:0/3029834615 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0280667c0 con 0x7ff0280b11b0 2020-11-28T21:53:21.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.505+0000 7ff025ffb700 1 -- 172.21.15.162:0/3029834615 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+2499 (secure 0 0 0) 0x7ff0180478e0 con 0x7ff0280b11b0 2020-11-28T21:53:21.512 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:53:21.512 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":33,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5540,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2116325247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2116325247},{"type":"v1","addr":"172.21.15.162:6835","nonce":2116325247}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":30},{"gid":5543,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.118:6833/3491069258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3491069258},{"type":"v1","addr":"172.21.15.118:6833","nonce":3491069258}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:52:28.454745+0000","modified":"2020-11-28T21:52:29.457610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6172},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6172":{"gid":6172,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/163256983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":163256983},{"type":"v1","addr":"172.21.15.162:6837","nonce":163256983}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-28T21:53:21.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.505+0000 7ff02ecd5700 1 -- 172.21.15.162:0/3029834615 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff0080554e0 msgr2=0x7ff0080579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:21.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.505+0000 7ff02ecd5700 1 --2- 172.21.15.162:0/3029834615 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff0080554e0 0x7ff0080579a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7ff01c016000 tx=0x7ff01c015000).stop 2020-11-28T21:53:21.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.505+0000 7ff02ecd5700 1 -- 172.21.15.162:0/3029834615 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0280b11b0 msgr2=0x7ff0280662d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:21.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.505+0000 7ff02ecd5700 1 --2- 172.21.15.162:0/3029834615 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0280b11b0 0x7ff0280662d0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 rx=0x7ff018001450 tx=0x7ff018007e20).stop 2020-11-28T21:53:21.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.509+0000 7ff02ecd5700 1 -- 172.21.15.162:0/3029834615 shutdown_connections 2020-11-28T21:53:21.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.509+0000 7ff02ecd5700 1 --2- 172.21.15.162:0/3029834615 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff0080554e0 0x7ff0080579a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:21.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.509+0000 7ff02ecd5700 1 --2- 172.21.15.162:0/3029834615 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff0280b1d70 0x7ff028155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:21.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.509+0000 7ff02ecd5700 1 --2- 172.21.15.162:0/3029834615 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0280b11b0 0x7ff0280662d0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:21.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.509+0000 7ff02ecd5700 1 --2- 172.21.15.162:0/3029834615 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff0280b1990 0x7ff028066950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:21.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.509+0000 7ff02ecd5700 1 -- 172.21.15.162:0/3029834615 >> 172.21.15.162:0/3029834615 conn(0x7ff02805b970 msgr2=0x7ff028149130 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:21.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.509+0000 7ff02ecd5700 1 -- 172.21.15.162:0/3029834615 shutdown_connections 2020-11-28T21:53:21.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:21.509+0000 7ff02ecd5700 1 -- 172.21.15.162:0/3029834615 wait complete. 2020-11-28T21:53:21.517 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 33 2020-11-28T21:53:21.828 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-28T21:52:28.454745+0000', 'modified': '2020-11-28T21:52:29.457610+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6172}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6172': {'gid': 6172, 'name': 'a', 'rank': 0, 'incarnation': 32, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6837/163256983', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6836', 'nonce': 163256983}, {'type': 'v1', 'addr': '172.21.15.162:6837', 'nonce': 163256983}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-28T21:53:21.829 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-28T21:53:21.829 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-28T21:53:21.960 INFO:teuthology.orchestra.run.smithi162.stdout:{"cluster_fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","whoami":0,"id":6172,"want_state":"up:active","state":"up:active","rank_uptime":53.490719599999998,"mdsmap_epoch":33,"osdmap_epoch":77,"osdmap_epoch_barrier":77,"uptime":58.918204205999999} 2020-11-28T21:53:21.961 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", "id": 6172, "mdsmap_epoch": 33, "osdmap_epoch": 77, "osdmap_epoch_barrier": 77, "rank_uptime": 53.4907196, "state": "up:active", "uptime": 58.918204206, "want_state": "up:active", "whoami": 0 } 2020-11-28T21:53:21.961 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-11-28T21:53:21.961 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-11-28T21:53:21.962 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi192.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-11-28T21:53:21.962 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:53:21.963 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:53:21.979 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:53:22.048 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:53:22.049 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:53:22.112 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:53:22.114 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:53:22.115 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:53:22.115 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:53:22.174 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-11-28T21:53:22.175 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:53:22.227 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:53:22.248 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:53:22.249 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:53:22.303 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:ceph-fuse[12134]: starting ceph client2020-11-28T21:53:22.296+0000 7fd1db1f80c0 -1 init, newargv = 0x5594c0cc9340 newargc=9 2020-11-28T21:53:22.303 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr: 2020-11-28T21:53:22.307 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:53:22.307 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:53:22.308 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:53:22.308 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:53:22.313 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:ceph-fuse[12134]: starting fuse 2020-11-28T21:53:22.320 INFO:teuthology.orchestra.run.smithi192.stdout:49 2020-11-28T21:53:22.321 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2020-11-28T21:53:22.321 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:53:22.321 INFO:teuthology.orchestra.run.smithi192:> sudo python3 -c ' 2020-11-28T21:53:22.322 INFO:teuthology.orchestra.run.smithi192:> import glob 2020-11-28T21:53:22.322 INFO:teuthology.orchestra.run.smithi192:> import re 2020-11-28T21:53:22.323 INFO:teuthology.orchestra.run.smithi192:> import os 2020-11-28T21:53:22.323 INFO:teuthology.orchestra.run.smithi192:> import subprocess 2020-11-28T21:53:22.323 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:53:22.324 INFO:teuthology.orchestra.run.smithi192:> def find_socket(client_name): 2020-11-28T21:53:22.324 INFO:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-11-28T21:53:22.325 INFO:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2020-11-28T21:53:22.325 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:53:22.325 INFO:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2020-11-28T21:53:22.326 INFO:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2020-11-28T21:53:22.326 INFO:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2020-11-28T21:53:22.326 INFO:teuthology.orchestra.run.smithi192:> return files[0] 2020-11-28T21:53:22.327 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:53:22.327 INFO:teuthology.orchestra.run.smithi192:> for f in files: 2020-11-28T21:53:22.328 INFO:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-11-28T21:53:22.328 INFO:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2020-11-28T21:53:22.328 INFO:teuthology.orchestra.run.smithi192:> return f 2020-11-28T21:53:22.329 INFO:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-11-28T21:53:22.329 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:53:22.330 INFO:teuthology.orchestra.run.smithi192:> print(find_socket("client.0")) 2020-11-28T21:53:22.330 INFO:teuthology.orchestra.run.smithi192:> ' 2020-11-28T21:53:22.417 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.0.12134.asok 2020-11-28T21:53:22.423 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.12134.asok 2020-11-28T21:53:22.424 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:53:22.424 INFO:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.12134.asok status 2020-11-28T21:53:22.528 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2020-11-28T21:53:22.529 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2020-11-28T21:53:22.529 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "a83a3572a957356977342f461361c1cd3ec06dbd", 2020-11-28T21:53:22.529 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 15.2.6-595-ga83a3572 (a83a3572a957356977342f461361c1cd3ec06dbd) octopus (stable)", 2020-11-28T21:53:22.530 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "0", 2020-11-28T21:53:22.530 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2020-11-28T21:53:22.532 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-11-28T21:53:22.533 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "12134", 2020-11-28T21:53:22.533 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2020-11-28T21:53:22.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:53:22.535 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2020-11-28T21:53:22.535 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2020-11-28T21:53:22.535 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 6276, 2020-11-28T21:53:22.536 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2020-11-28T21:53:22.536 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2020-11-28T21:53:22.536 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2020-11-28T21:53:22.537 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 6276 2020-11-28T21:53:22.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:53:22.537 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2020-11-28T21:53:22.538 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2020-11-28T21:53:22.538 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2020-11-28T21:53:22.538 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 2876361034 2020-11-28T21:53:22.539 INFO:teuthology.orchestra.run.smithi192.stdout: } 2020-11-28T21:53:22.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:53:22.540 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2020-11-28T21:53:22.541 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2020-11-28T21:53:22.541 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2020-11-28T21:53:22.542 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 2876361034 2020-11-28T21:53:22.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:53:22.544 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.6276 172.21.15.192:0/2876361034", 2020-11-28T21:53:22.545 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "172.21.15.192:0/2876361034", 2020-11-28T21:53:22.546 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2020-11-28T21:53:22.547 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 33, 2020-11-28T21:53:22.548 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 77, 2020-11-28T21:53:22.549 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2020-11-28T21:53:22.550 INFO:teuthology.orchestra.run.smithi192.stdout: "blacklisted": false 2020-11-28T21:53:22.551 INFO:teuthology.orchestra.run.smithi192.stdout:} 2020-11-28T21:53:22.552 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:53:22.552 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:53:22.553 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:53:22.564 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2020-11-28T21:53:22.565 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-28T21:53:22.565 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:53:22.566 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:53:22.634 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-28T21:53:22.635 INFO:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:53:22.659 INFO:teuthology.orchestra.run.smithi118:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:53:22.688 INFO:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:53:22.716 INFO:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:53:22.740 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-28T21:53:22.741 INFO:tasks.cephfs.mount:Terminating background process 2020-11-28T21:53:22.742 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:53:22.743 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:53:22.743 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:53:22.799 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2020-11-28T21:53:22.801 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-28T21:53:22.801 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2020-11-28T21:53:22.802 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-28T21:53:22.803 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:53:22.876 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:ceph-fuse[12134]: fuse finished with error 0 and tester_r 0 2020-11-28T21:53:22.878 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:53:22.878 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:53:22.879 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:53:22.930 INFO:teuthology.orchestra.run.smithi192.stdout:ext2/ext3 2020-11-28T21:53:22.931 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-11-28T21:53:23.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-28T21:53:23.051 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:53:23.098 INFO:tasks.cephfs.mount:Terminating background process 2020-11-28T21:53:23.098 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:53:23.099 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:53:23.100 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:53:23.108 INFO:teuthology.orchestra.run.smithi096.stdout:fuseblk 2020-11-28T21:53:23.109 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-11-28T21:53:23.110 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi096.front.sepia.ceph.com... 2020-11-28T21:53:23.111 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-28T21:53:23.111 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:53:23.205 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[11536]: fuse finished with error 0 and tester_r 0 2020-11-28T21:53:23.207 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:53:23.208 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:53:23.208 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:53:23.216 INFO:teuthology.orchestra.run.smithi096.stdout:ext2/ext3 2020-11-28T21:53:23.217 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-11-28T21:53:23.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-28T21:53:23.336 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:53:23.386 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config rm mon mon_allow_pool_delete 2020-11-28T21:53:23.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.505+0000 7fb5fb59e700 1 Processor -- start 2020-11-28T21:53:23.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.505+0000 7fb5fb59e700 1 -- start start 2020-11-28T21:53:23.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.505+0000 7fb5fb59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb5fc0611e0 0x7fb5fc0615c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:23.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.505+0000 7fb5fb59e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb5fc0b4b40 0x7fb5fc13ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:23.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.505+0000 7fb5fb59e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5fc0b1550 con 0x7fb5fc0b10f0 2020-11-28T21:53:23.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.505+0000 7fb5fb59e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5fc0b1790 con 0x7fb5fc0b4b40 2020-11-28T21:53:23.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.505+0000 7fb5fb59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc0b1a30 con 0x7fb5fc0611e0 2020-11-28T21:53:23.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.505+0000 7fb5fad9d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb5fc0b4b40 0x7fb5fc13ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:23.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.505+0000 7fb5fad9d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb5fc0b4b40 0x7fb5fc13ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41738/0 (socket says 172.21.15.162:41738) 2020-11-28T21:53:23.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.505+0000 7fb5f9d9b700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fb5fc0b10f0 0x7fb5fc0b4430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:48578/0 (socket says 172.21.15.162:48578) 2020-11-28T21:53:23.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.505+0000 7fb5fad9d700 1 -- 172.21.15.162:0/165326000 learned_addr learned my addr 172.21.15.162:0/165326000 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:23.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.505+0000 7fb5fa59c700 1 --2- 172.21.15.162:0/165326000 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb5fc0611e0 0x7fb5fc0615c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:23.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.505+0000 7fb5fad9d700 1 -- 172.21.15.162:0/165326000 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb5fc0611e0 msgr2=0x7fb5fc0615c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:23.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.505+0000 7fb5fad9d700 1 --2- 172.21.15.162:0/165326000 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb5fc0611e0 0x7fb5fc0615c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:23.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.505+0000 7fb5fad9d700 1 -- 172.21.15.162:0/165326000 >> v1:172.21.15.162:6789/0 conn(0x7fb5fc0b10f0 legacy=0x7fb5fc0b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:23.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.505+0000 7fb5fad9d700 1 -- 172.21.15.162:0/165326000 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5fc0b3f90 con 0x7fb5fc0b4b40 2020-11-28T21:53:23.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.505+0000 7fb5fa59c700 1 --2- 172.21.15.162:0/165326000 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb5fc0611e0 0x7fb5fc0615c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:53:23.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.505+0000 7fb5fad9d700 1 --2- 172.21.15.162:0/165326000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5fc0b4b40 0x7fb5fc13ee60 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 rx=0x7fb5f40032b0 tx=0x7fb5f4009270).ready entity=mon.1 client_cookie=eed711d5984f2f57 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:23.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.505+0000 7fb5f959a700 1 -- 172.21.15.162:0/165326000 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5f400bdf0 con 0x7fb5fc0b4b40 2020-11-28T21:53:23.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.505+0000 7fb5f959a700 1 -- 172.21.15.162:0/165326000 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f4010470 con 0x7fb5fc0b4b40 2020-11-28T21:53:23.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.505+0000 7fb5f959a700 1 -- 172.21.15.162:0/165326000 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5f40107d0 con 0x7fb5fc0b4b40 2020-11-28T21:53:23.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.509+0000 7fb5fb59e700 1 -- 172.21.15.162:0/165326000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5fc0b4b40 msgr2=0x7fb5fc13ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:23.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.509+0000 7fb5fb59e700 1 --2- 172.21.15.162:0/165326000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5fc0b4b40 0x7fb5fc13ee60 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 rx=0x7fb5f40032b0 tx=0x7fb5f4009270).stop 2020-11-28T21:53:23.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.509+0000 7fb5fb59e700 1 -- 172.21.15.162:0/165326000 shutdown_connections 2020-11-28T21:53:23.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.509+0000 7fb5fb59e700 1 --2- 172.21.15.162:0/165326000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5fc0b4b40 0x7fb5fc13ee60 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:23.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.509+0000 7fb5fb59e700 1 --2- 172.21.15.162:0/165326000 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb5fc0611e0 0x7fb5fc0615c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:23.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.509+0000 7fb5fb59e700 1 -- 172.21.15.162:0/165326000 >> 172.21.15.162:0/165326000 conn(0x7fb5fc0b54f0 msgr2=0x7fb5fc0b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:23.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.509+0000 7fb5fb59e700 1 -- 172.21.15.162:0/165326000 shutdown_connections 2020-11-28T21:53:23.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.509+0000 7fb5fb59e700 1 -- 172.21.15.162:0/165326000 wait complete. 2020-11-28T21:53:23.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.509+0000 7fb5fb59e700 1 Processor -- start 2020-11-28T21:53:23.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.509+0000 7fb5fb59e700 1 -- start start 2020-11-28T21:53:23.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.509+0000 7fb5fb59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb5fc0611e0 0x7fb5fc0988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:23.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.509+0000 7fb5fb59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb5fc0b10f0 0x7fb5fc14d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:23.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.509+0000 7fb5fb59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5fc0b4b40 0x7fb5fc14dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:23.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.509+0000 7fb5fb59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc13e860 con 0x7fb5fc0b10f0 2020-11-28T21:53:23.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.509+0000 7fb5fb59e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc14e1f0 con 0x7fb5fc0b4b40 2020-11-28T21:53:23.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.509+0000 7fb5fb59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc14d2b0 con 0x7fb5fc0611e0 2020-11-28T21:53:23.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.513+0000 7fb5fad9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5fc0b4b40 0x7fb5fc14dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:23.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.513+0000 7fb5fad9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5fc0b4b40 0x7fb5fc14dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41742/0 (socket says 172.21.15.162:41742) 2020-11-28T21:53:23.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.513+0000 7fb5fad9d700 1 -- 172.21.15.162:0/2634682873 learned_addr learned my addr 172.21.15.162:0/2634682873 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:23.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.513+0000 7fb5f9d9b700 1 --2- 172.21.15.162:0/2634682873 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb5fc0b10f0 0x7fb5fc14d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:23.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.513+0000 7fb5fa59c700 1 --2- 172.21.15.162:0/2634682873 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb5fc0611e0 0x7fb5fc0988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:23.521 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.513+0000 7fb5fad9d700 1 -- 172.21.15.162:0/2634682873 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb5fc0611e0 msgr2=0x7fb5fc0988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:23.521 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.513+0000 7fb5fad9d700 1 --2- 172.21.15.162:0/2634682873 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb5fc0611e0 0x7fb5fc0988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:23.521 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.513+0000 7fb5fad9d700 1 -- 172.21.15.162:0/2634682873 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb5fc0b10f0 msgr2=0x7fb5fc14d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:23.521 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.513+0000 7fb5fad9d700 1 --2- 172.21.15.162:0/2634682873 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb5fc0b10f0 0x7fb5fc14d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:23.521 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.513+0000 7fb5fad9d700 1 -- 172.21.15.162:0/2634682873 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5fc13bd90 con 0x7fb5fc0b4b40 2020-11-28T21:53:23.522 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.513+0000 7fb5fa59c700 1 --2- 172.21.15.162:0/2634682873 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb5fc0611e0 0x7fb5fc0988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:53:23.522 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.513+0000 7fb5fad9d700 1 --2- 172.21.15.162:0/2634682873 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5fc0b4b40 0x7fb5fc14dd00 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7fb5f40032b0 tx=0x7fb5f4018000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:23.522 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.513+0000 7fb5eb7fe700 1 -- 172.21.15.162:0/2634682873 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5f4018800 con 0x7fb5fc0b4b40 2020-11-28T21:53:23.522 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.513+0000 7fb5eb7fe700 1 -- 172.21.15.162:0/2634682873 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f401e040 con 0x7fb5fc0b4b40 2020-11-28T21:53:23.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.513+0000 7fb5fb59e700 1 -- 172.21.15.162:0/2634682873 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5fc13e5c0 con 0x7fb5fc0b4b40 2020-11-28T21:53:23.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.513+0000 7fb5eb7fe700 1 -- 172.21.15.162:0/2634682873 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5f4018b50 con 0x7fb5fc0b4b40 2020-11-28T21:53:23.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.513+0000 7fb5fb59e700 1 -- 172.21.15.162:0/2634682873 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5fc097ba0 con 0x7fb5fc0b4b40 2020-11-28T21:53:23.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.513+0000 7fb5fb59e700 1 -- 172.21.15.162:0/2634682873 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5fc057050 con 0x7fb5fc0b4b40 2020-11-28T21:53:23.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.513+0000 7fb5eb7fe700 1 -- 172.21.15.162:0/2634682873 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb5f4018cb0 con 0x7fb5fc0b4b40 2020-11-28T21:53:23.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.513+0000 7fb5eb7fe700 1 --2- 172.21.15.162:0/2634682873 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb5d8055440 0x7fb5d8057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:23.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.513+0000 7fb5eb7fe700 1 -- 172.21.15.162:0/2634682873 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fb5f4071870 con 0x7fb5fc0b4b40 2020-11-28T21:53:23.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.513+0000 7fb5fa59c700 1 --2- 172.21.15.162:0/2634682873 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb5d8055440 0x7fb5d8057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:23.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.517+0000 7fb5fa59c700 1 --2- 172.21.15.162:0/2634682873 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb5d8055440 0x7fb5d8057900 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7fb5ec008000 tx=0x7fb5ec008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:23.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.521+0000 7fb5eb7fe700 1 -- 172.21.15.162:0/2634682873 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb5f4047bf0 con 0x7fb5fc0b4b40 2020-11-28T21:53:23.663 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.661+0000 7fb5fb59e700 1 -- 172.21.15.162:0/2634682873 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb5fc05d5e0 con 0x7fb5fc0b4b40 2020-11-28T21:53:23.673 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.669+0000 7fb5eb7fe700 1 -- 172.21.15.162:0/2634682873 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb5f404b210 con 0x7fb5fc0b4b40 2020-11-28T21:53:23.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.673+0000 7fb5fb59e700 1 -- 172.21.15.162:0/2634682873 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb5d8055440 msgr2=0x7fb5d8057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:23.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.673+0000 7fb5fb59e700 1 --2- 172.21.15.162:0/2634682873 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb5d8055440 0x7fb5d8057900 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7fb5ec008000 tx=0x7fb5ec008040).stop 2020-11-28T21:53:23.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.673+0000 7fb5fb59e700 1 -- 172.21.15.162:0/2634682873 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5fc0b4b40 msgr2=0x7fb5fc14dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:23.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.673+0000 7fb5fb59e700 1 --2- 172.21.15.162:0/2634682873 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5fc0b4b40 0x7fb5fc14dd00 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7fb5f40032b0 tx=0x7fb5f4018000).stop 2020-11-28T21:53:23.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.673+0000 7fb5fb59e700 1 -- 172.21.15.162:0/2634682873 shutdown_connections 2020-11-28T21:53:23.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.673+0000 7fb5fb59e700 1 --2- 172.21.15.162:0/2634682873 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb5d8055440 0x7fb5d8057900 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:23.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.673+0000 7fb5fb59e700 1 --2- 172.21.15.162:0/2634682873 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb5fc0611e0 0x7fb5fc0988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:23.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.673+0000 7fb5fb59e700 1 --2- 172.21.15.162:0/2634682873 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5fc0b4b40 0x7fb5fc14dd00 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:23.680 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.673+0000 7fb5fb59e700 1 --2- 172.21.15.162:0/2634682873 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb5fc0b10f0 0x7fb5fc14d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:23.680 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.673+0000 7fb5fb59e700 1 -- 172.21.15.162:0/2634682873 >> 172.21.15.162:0/2634682873 conn(0x7fb5fc0b54f0 msgr2=0x7fb5fc0b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:23.680 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.673+0000 7fb5fb59e700 1 -- 172.21.15.162:0/2634682873 shutdown_connections 2020-11-28T21:53:23.681 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.673+0000 7fb5fb59e700 1 -- 172.21.15.162:0/2634682873 wait complete. 2020-11-28T21:53:23.689 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Ended test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder' 2020-11-28T21:53:23.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.801+0000 7f18317e7700 1 Processor -- start 2020-11-28T21:53:23.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.801+0000 7f18317e7700 1 -- start start 2020-11-28T21:53:23.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.801+0000 7f18317e7700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f182c0b1990 0x7f182c140890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:23.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.801+0000 7f18317e7700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f182c0b1d70 0x7f182c14a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:23.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.801+0000 7f18317e7700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f182c0b2870 con 0x7f182c0b11b0 2020-11-28T21:53:23.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.801+0000 7f18317e7700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f182c146e20 con 0x7f182c0b1d70 2020-11-28T21:53:23.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.801+0000 7f18317e7700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182c0b22e0 con 0x7f182c0b1990 2020-11-28T21:53:23.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.801+0000 7f182b7fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f182c0b1d70 0x7f182c14a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:23.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.801+0000 7f182b7fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f182c0b1d70 0x7f182c14a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41750/0 (socket says 172.21.15.162:41750) 2020-11-28T21:53:23.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.801+0000 7f182b7fe700 1 -- 172.21.15.162:0/1510374994 learned_addr learned my addr 172.21.15.162:0/1510374994 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:23.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.801+0000 7f182affd700 1 --2- 172.21.15.162:0/1510374994 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f182c0b1990 0x7f182c140890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:23.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.801+0000 7f182b7fe700 1 -- 172.21.15.162:0/1510374994 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f182c0b1990 msgr2=0x7f182c140890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:23.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.801+0000 7f182b7fe700 1 --2- 172.21.15.162:0/1510374994 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f182c0b1990 0x7f182c140890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:23.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.801+0000 7f182b7fe700 1 -- 172.21.15.162:0/1510374994 >> v1:172.21.15.118:6789/0 conn(0x7f182c0b11b0 legacy=0x7f182c140d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:23.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.801+0000 7f182b7fe700 1 -- 172.21.15.162:0/1510374994 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f182c0b0740 con 0x7f182c0b1d70 2020-11-28T21:53:23.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.801+0000 7f182affd700 1 --2- 172.21.15.162:0/1510374994 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f182c0b1990 0x7f182c140890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:53:23.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.801+0000 7f182b7fe700 1 --2- 172.21.15.162:0/1510374994 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f182c0b1d70 0x7f182c14a770 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 rx=0x7f18240032b0 tx=0x7f1824009270).ready entity=mon.1 client_cookie=c9abd5c53a12f94c server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:23.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.801+0000 7f1829ffb700 1 -- 172.21.15.162:0/1510374994 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f182400bc00 con 0x7f182c0b1d70 2020-11-28T21:53:23.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.801+0000 7f1829ffb700 1 -- 172.21.15.162:0/1510374994 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182400bd60 con 0x7f182c0b1d70 2020-11-28T21:53:23.810 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.801+0000 7f18317e7700 1 -- 172.21.15.162:0/1510374994 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f182c0b1d70 msgr2=0x7f182c14a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:23.810 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.801+0000 7f1829ffb700 1 -- 172.21.15.162:0/1510374994 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1824011400 con 0x7f182c0b1d70 2020-11-28T21:53:23.810 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.801+0000 7f18317e7700 1 --2- 172.21.15.162:0/1510374994 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f182c0b1d70 0x7f182c14a770 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 rx=0x7f18240032b0 tx=0x7f1824009270).stop 2020-11-28T21:53:23.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.801+0000 7f18317e7700 1 -- 172.21.15.162:0/1510374994 shutdown_connections 2020-11-28T21:53:23.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.801+0000 7f18317e7700 1 --2- 172.21.15.162:0/1510374994 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f182c0b1d70 0x7f182c14a770 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:23.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.801+0000 7f18317e7700 1 --2- 172.21.15.162:0/1510374994 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f182c0b1990 0x7f182c140890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:23.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.801+0000 7f18317e7700 1 -- 172.21.15.162:0/1510374994 >> 172.21.15.162:0/1510374994 conn(0x7f182c05b970 msgr2=0x7f182c05bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:23.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.801+0000 7f18317e7700 1 -- 172.21.15.162:0/1510374994 shutdown_connections 2020-11-28T21:53:23.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.801+0000 7f18317e7700 1 -- 172.21.15.162:0/1510374994 wait complete. 2020-11-28T21:53:23.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.805+0000 7f18317e7700 1 Processor -- start 2020-11-28T21:53:23.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.805+0000 7f18317e7700 1 -- start start 2020-11-28T21:53:23.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.805+0000 7f18317e7700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f182c0b11b0 0x7f182c1467d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:23.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.805+0000 7f18317e7700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f182c0b1990 0x7f182c0663d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:23.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.805+0000 7f18317e7700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f182c0b1d70 0x7f182c158fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:23.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.805+0000 7f18317e7700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182c1444a0 con 0x7f182c0b1990 2020-11-28T21:53:23.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.805+0000 7f18317e7700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182c143a40 con 0x7f182c0b11b0 2020-11-28T21:53:23.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.805+0000 7f18317e7700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182c143ce0 con 0x7f182c0b1d70 2020-11-28T21:53:23.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.805+0000 7f182affd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f182c0b11b0 0x7f182c1467d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:23.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.805+0000 7f182affd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f182c0b11b0 0x7f182c1467d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41754/0 (socket says 172.21.15.162:41754) 2020-11-28T21:53:23.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.805+0000 7f182affd700 1 -- 172.21.15.162:0/330330642 learned_addr learned my addr 172.21.15.162:0/330330642 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:23.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.805+0000 7f182b7fe700 1 --2- 172.21.15.162:0/330330642 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f182c0b1d70 0x7f182c158fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:23.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.805+0000 7f182a7fc700 1 --2- 172.21.15.162:0/330330642 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f182c0b1990 0x7f182c0663d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:23.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.805+0000 7f182affd700 1 -- 172.21.15.162:0/330330642 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f182c0b1d70 msgr2=0x7f182c158fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:23.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.805+0000 7f182affd700 1 --2- 172.21.15.162:0/330330642 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f182c0b1d70 0x7f182c158fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:23.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.805+0000 7f182affd700 1 -- 172.21.15.162:0/330330642 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f182c0b1990 msgr2=0x7f182c0663d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:23.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.805+0000 7f182affd700 1 --2- 172.21.15.162:0/330330642 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f182c0b1990 0x7f182c0663d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:23.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.805+0000 7f182affd700 1 -- 172.21.15.162:0/330330642 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f182c065c90 con 0x7f182c0b11b0 2020-11-28T21:53:23.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.805+0000 7f182a7fc700 1 --2- 172.21.15.162:0/330330642 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f182c0b1990 0x7f182c0663d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:53:23.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.805+0000 7f182affd700 1 --2- 172.21.15.162:0/330330642 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f182c0b11b0 0x7f182c1467d0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 rx=0x7f181c007000 tx=0x7f181c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:23.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.805+0000 7f1823fff700 1 -- 172.21.15.162:0/330330642 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f181c00ce90 con 0x7f182c0b11b0 2020-11-28T21:53:23.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.805+0000 7f18317e7700 1 -- 172.21.15.162:0/330330642 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f182c065f30 con 0x7f182c0b11b0 2020-11-28T21:53:23.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.805+0000 7f1823fff700 1 -- 172.21.15.162:0/330330642 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181c007750 con 0x7f182c0b11b0 2020-11-28T21:53:23.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.805+0000 7f1823fff700 1 -- 172.21.15.162:0/330330642 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f181c007ab0 con 0x7f182c0b11b0 2020-11-28T21:53:23.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.805+0000 7f18317e7700 1 -- 172.21.15.162:0/330330642 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f182c0668c0 con 0x7f182c0b11b0 2020-11-28T21:53:23.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.809+0000 7f18317e7700 1 -- 172.21.15.162:0/330330642 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f182c057050 con 0x7f182c0b11b0 2020-11-28T21:53:23.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.809+0000 7f1823fff700 1 -- 172.21.15.162:0/330330642 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f181c007c10 con 0x7f182c0b11b0 2020-11-28T21:53:23.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.809+0000 7f1823fff700 1 --2- 172.21.15.162:0/330330642 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1810055440 0x7f1810057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:23.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.809+0000 7f1823fff700 1 -- 172.21.15.162:0/330330642 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f181c06e2f0 con 0x7f182c0b11b0 2020-11-28T21:53:23.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.809+0000 7f182a7fc700 1 --2- 172.21.15.162:0/330330642 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1810055440 0x7f1810057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:23.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.809+0000 7f182a7fc700 1 --2- 172.21.15.162:0/330330642 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1810055440 0x7f1810057900 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f181400b000 tx=0x7f1814009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:23.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:23.813+0000 7f1823fff700 1 -- 172.21.15.162:0/330330642 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f181c0443d0 con 0x7f182c0b11b0 2020-11-28T21:53:24.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:24.005+0000 7f18317e7700 1 -- 172.21.15.162:0/330330642 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder"]} v 0) v1 -- 0x7f182c0660c0 con 0x7f182c0b11b0 2020-11-28T21:53:24.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:24.937+0000 7f1823fff700 1 -- 172.21.15.162:0/330330642 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder"]}]=0 v293) v1 ==== 139+0+0 (secure 0 0 0) 0x7f181c0479f0 con 0x7f182c0b11b0 2020-11-28T21:53:24.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:24.941+0000 7f18317e7700 1 -- 172.21.15.162:0/330330642 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1810055440 msgr2=0x7f1810057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:24.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:24.941+0000 7f18317e7700 1 --2- 172.21.15.162:0/330330642 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1810055440 0x7f1810057900 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f181400b000 tx=0x7f1814009040).stop 2020-11-28T21:53:24.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:24.941+0000 7f18317e7700 1 -- 172.21.15.162:0/330330642 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f182c0b11b0 msgr2=0x7f182c1467d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:24.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:24.941+0000 7f18317e7700 1 --2- 172.21.15.162:0/330330642 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f182c0b11b0 0x7f182c1467d0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 rx=0x7f181c007000 tx=0x7f181c008040).stop 2020-11-28T21:53:24.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:24.941+0000 7f18317e7700 1 -- 172.21.15.162:0/330330642 shutdown_connections 2020-11-28T21:53:24.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:24.941+0000 7f18317e7700 1 --2- 172.21.15.162:0/330330642 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1810055440 0x7f1810057900 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:24.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:24.941+0000 7f18317e7700 1 --2- 172.21.15.162:0/330330642 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f182c0b1d70 0x7f182c158fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:24.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:24.941+0000 7f18317e7700 1 --2- 172.21.15.162:0/330330642 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f182c0b11b0 0x7f182c1467d0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:24.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:24.941+0000 7f18317e7700 1 --2- 172.21.15.162:0/330330642 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f182c0b1990 0x7f182c0663d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:24.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:24.941+0000 7f18317e7700 1 -- 172.21.15.162:0/330330642 >> 172.21.15.162:0/330330642 conn(0x7f182c05b970 msgr2=0x7f182c144cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:24.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:24.941+0000 7f18317e7700 1 -- 172.21.15.162:0/330330642 shutdown_connections 2020-11-28T21:53:24.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:24.941+0000 7f18317e7700 1 -- 172.21.15.162:0/330330642 wait complete. 2020-11-28T21:53:24.961 INFO:tasks.cephfs_test_runner:test_cap_revoke_nonresponder (tasks.cephfs.test_misc.TestMisc) ... ok 2020-11-28T21:53:24.962 INFO:tasks.cephfs_test_runner:Starting test: test_dump_inode (tasks.cephfs.test_misc.TestMisc) 2020-11-28T21:53:24.962 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode' 2020-11-28T21:53:25.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.073+0000 7f5bb5731700 1 Processor -- start 2020-11-28T21:53:25.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.073+0000 7f5bb5731700 1 -- start start 2020-11-28T21:53:25.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.073+0000 7f5bb5731700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5bb00b1990 0x7f5bb0140890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:25.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.073+0000 7f5bb5731700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5bb00b11b0 0x7f5bb0140d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:25.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.073+0000 7f5bb5731700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bb00b2870 con 0x7f5bb00b1d70 2020-11-28T21:53:25.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.073+0000 7f5bb5731700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bb0146e20 con 0x7f5bb00b1990 2020-11-28T21:53:25.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.073+0000 7f5bb5731700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb00b22e0 con 0x7f5bb00b11b0 2020-11-28T21:53:25.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.073+0000 7f5baeffd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5bb00b1990 0x7f5bb0140890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:25.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.073+0000 7f5baf7fe700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f5bb00b1d70 0x7f5bb014a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:48610/0 (socket says 172.21.15.162:48610) 2020-11-28T21:53:25.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.073+0000 7f5baf7fe700 1 -- 172.21.15.162:0/15657815 learned_addr learned my addr 172.21.15.162:0/15657815 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:25.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.073+0000 7f5bae7fc700 1 --2- 172.21.15.162:0/15657815 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5bb00b11b0 0x7f5bb0140d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:25.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.073+0000 7f5badffb700 1 -- 172.21.15.162:0/15657815 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4087333509 0 0) 0x7f5bb00b2870 con 0x7f5bb00b1d70 2020-11-28T21:53:25.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.073+0000 7f5badffb700 1 -- 172.21.15.162:0/15657815 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b94003230 con 0x7f5bb00b1d70 2020-11-28T21:53:25.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.073+0000 7f5baeffd700 1 -- 172.21.15.162:0/15657815 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5bb00b11b0 msgr2=0x7f5bb0140d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:25.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.073+0000 7f5baeffd700 1 --2- 172.21.15.162:0/15657815 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5bb00b11b0 0x7f5bb0140d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:25.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.073+0000 7f5baeffd700 1 -- 172.21.15.162:0/15657815 >> v1:172.21.15.162:6789/0 conn(0x7f5bb00b1d70 legacy=0x7f5bb014a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:25.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.073+0000 7f5baeffd700 1 -- 172.21.15.162:0/15657815 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb00b0740 con 0x7f5bb00b1990 2020-11-28T21:53:25.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.073+0000 7f5baeffd700 1 --2- 172.21.15.162:0/15657815 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bb00b1990 0x7f5bb0140890 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 rx=0x7f5ba0002f30 tx=0x7f5ba0006dc0).ready entity=mon.1 client_cookie=7606dc16209ca1ab server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:25.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.073+0000 7f5badffb700 1 -- 172.21.15.162:0/15657815 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ba00096a0 con 0x7f5bb00b1990 2020-11-28T21:53:25.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.073+0000 7f5badffb700 1 -- 172.21.15.162:0/15657815 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba0009800 con 0x7f5bb00b1990 2020-11-28T21:53:25.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.073+0000 7f5badffb700 1 -- 172.21.15.162:0/15657815 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ba0009b60 con 0x7f5bb00b1990 2020-11-28T21:53:25.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.077+0000 7f5bb5731700 1 -- 172.21.15.162:0/15657815 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bb00b1990 msgr2=0x7f5bb0140890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:25.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.077+0000 7f5bb5731700 1 --2- 172.21.15.162:0/15657815 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bb00b1990 0x7f5bb0140890 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 rx=0x7f5ba0002f30 tx=0x7f5ba0006dc0).stop 2020-11-28T21:53:25.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.077+0000 7f5bb5731700 1 -- 172.21.15.162:0/15657815 shutdown_connections 2020-11-28T21:53:25.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.077+0000 7f5bb5731700 1 --2- 172.21.15.162:0/15657815 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bb00b1990 0x7f5bb0140890 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:25.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.077+0000 7f5bb5731700 1 --2- 172.21.15.162:0/15657815 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5bb00b11b0 0x7f5bb0140d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:25.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.077+0000 7f5bb5731700 1 -- 172.21.15.162:0/15657815 >> 172.21.15.162:0/15657815 conn(0x7f5bb005b970 msgr2=0x7f5bb005bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:25.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.077+0000 7f5bb5731700 1 -- 172.21.15.162:0/15657815 shutdown_connections 2020-11-28T21:53:25.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.077+0000 7f5bb5731700 1 -- 172.21.15.162:0/15657815 wait complete. 2020-11-28T21:53:25.086 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.077+0000 7f5bb5731700 1 Processor -- start 2020-11-28T21:53:25.086 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.077+0000 7f5bb5731700 1 -- start start 2020-11-28T21:53:25.086 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.077+0000 7f5bb5731700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bb00b11b0 0x7f5bb00665b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:25.086 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.077+0000 7f5bb5731700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5bb00b1990 0x7f5bb0064340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:25.086 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.077+0000 7f5bb5731700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5bb00b1d70 0x7f5bb0064830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:25.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.077+0000 7f5bb5731700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb0143270 con 0x7f5bb00b1d70 2020-11-28T21:53:25.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.077+0000 7f5bb5731700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb0144050 con 0x7f5bb00b11b0 2020-11-28T21:53:25.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.077+0000 7f5bb5731700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb0144280 con 0x7f5bb00b1990 2020-11-28T21:53:25.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.077+0000 7f5baeffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bb00b11b0 0x7f5bb00665b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:25.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.077+0000 7f5baeffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bb00b11b0 0x7f5bb00665b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41768/0 (socket says 172.21.15.162:41768) 2020-11-28T21:53:25.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.077+0000 7f5baeffd700 1 -- 172.21.15.162:0/1700575127 learned_addr learned my addr 172.21.15.162:0/1700575127 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:25.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.077+0000 7f5bae7fc700 1 --2- 172.21.15.162:0/1700575127 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5bb00b1990 0x7f5bb0064340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:25.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.077+0000 7f5baf7fe700 1 --2- 172.21.15.162:0/1700575127 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5bb00b1d70 0x7f5bb0064830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:25.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.081+0000 7f5baeffd700 1 -- 172.21.15.162:0/1700575127 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5bb00b1990 msgr2=0x7f5bb0064340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:25.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.081+0000 7f5baeffd700 1 --2- 172.21.15.162:0/1700575127 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5bb00b1990 0x7f5bb0064340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:25.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.081+0000 7f5baeffd700 1 -- 172.21.15.162:0/1700575127 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5bb00b1d70 msgr2=0x7f5bb0064830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:25.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.081+0000 7f5baeffd700 1 --2- 172.21.15.162:0/1700575127 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5bb00b1d70 0x7f5bb0064830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:25.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.081+0000 7f5baeffd700 1 -- 172.21.15.162:0/1700575127 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb0159ce0 con 0x7f5bb00b11b0 2020-11-28T21:53:25.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.081+0000 7f5bae7fc700 1 --2- 172.21.15.162:0/1700575127 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5bb00b1990 0x7f5bb0064340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:53:25.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.081+0000 7f5baeffd700 1 --2- 172.21.15.162:0/1700575127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bb00b11b0 0x7f5bb00665b0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 rx=0x7f5ba0002f30 tx=0x7f5ba0015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:25.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.081+0000 7f5b9ffff700 1 -- 172.21.15.162:0/1700575127 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ba0009ea0 con 0x7f5bb00b11b0 2020-11-28T21:53:25.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.081+0000 7f5b9ffff700 1 -- 172.21.15.162:0/1700575127 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba0011ca0 con 0x7f5bb00b11b0 2020-11-28T21:53:25.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.081+0000 7f5b9ffff700 1 -- 172.21.15.162:0/1700575127 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ba001d450 con 0x7f5bb00b11b0 2020-11-28T21:53:25.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.081+0000 7f5bb5731700 1 -- 172.21.15.162:0/1700575127 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bb0143c70 con 0x7f5bb00b11b0 2020-11-28T21:53:25.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.081+0000 7f5bb5731700 1 -- 172.21.15.162:0/1700575127 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bb0067310 con 0x7f5bb00b11b0 2020-11-28T21:53:25.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.081+0000 7f5bb5731700 1 -- 172.21.15.162:0/1700575127 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bb0057050 con 0x7f5bb00b11b0 2020-11-28T21:53:25.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.081+0000 7f5b9ffff700 1 -- 172.21.15.162:0/1700575127 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5ba000f070 con 0x7f5bb00b11b0 2020-11-28T21:53:25.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.081+0000 7f5b9ffff700 1 --2- 172.21.15.162:0/1700575127 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5b8c055440 0x7f5b8c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:25.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.081+0000 7f5bae7fc700 1 --2- 172.21.15.162:0/1700575127 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5b8c055440 0x7f5b8c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:25.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.081+0000 7f5b9ffff700 1 -- 172.21.15.162:0/1700575127 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f5ba006fab0 con 0x7f5bb00b11b0 2020-11-28T21:53:25.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.081+0000 7f5bae7fc700 1 --2- 172.21.15.162:0/1700575127 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5b8c055440 0x7f5b8c057900 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f5ba400b000 tx=0x7f5ba4008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:25.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.089+0000 7f5b9ffff700 1 -- 172.21.15.162:0/1700575127 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5ba0045c70 con 0x7f5bb00b11b0 2020-11-28T21:53:25.279 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.277+0000 7f5bb5731700 1 -- 172.21.15.162:0/1700575127 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode"]} v 0) v1 -- 0x7f5bb0159e70 con 0x7f5bb00b11b0 2020-11-28T21:53:25.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.945+0000 7f5b9ffff700 1 -- 172.21.15.162:0/1700575127 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode"]}]=0 v294) v1 ==== 129+0+0 (secure 0 0 0) 0x7f5ba0049290 con 0x7f5bb00b11b0 2020-11-28T21:53:25.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.949+0000 7f5bb5731700 1 -- 172.21.15.162:0/1700575127 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5b8c055440 msgr2=0x7f5b8c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:25.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.949+0000 7f5bb5731700 1 --2- 172.21.15.162:0/1700575127 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5b8c055440 0x7f5b8c057900 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f5ba400b000 tx=0x7f5ba4008040).stop 2020-11-28T21:53:25.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.949+0000 7f5bb5731700 1 -- 172.21.15.162:0/1700575127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bb00b11b0 msgr2=0x7f5bb00665b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:25.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.949+0000 7f5bb5731700 1 --2- 172.21.15.162:0/1700575127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bb00b11b0 0x7f5bb00665b0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 rx=0x7f5ba0002f30 tx=0x7f5ba0015000).stop 2020-11-28T21:53:25.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.949+0000 7f5bb5731700 1 -- 172.21.15.162:0/1700575127 shutdown_connections 2020-11-28T21:53:25.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.949+0000 7f5bb5731700 1 --2- 172.21.15.162:0/1700575127 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5b8c055440 0x7f5b8c057900 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:25.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.949+0000 7f5bb5731700 1 --2- 172.21.15.162:0/1700575127 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5bb00b1d70 0x7f5bb0064830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:25.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.949+0000 7f5bb5731700 1 --2- 172.21.15.162:0/1700575127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bb00b11b0 0x7f5bb00665b0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:25.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.949+0000 7f5bb5731700 1 --2- 172.21.15.162:0/1700575127 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5bb00b1990 0x7f5bb0064340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:25.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.949+0000 7f5bb5731700 1 -- 172.21.15.162:0/1700575127 >> 172.21.15.162:0/1700575127 conn(0x7f5bb005b970 msgr2=0x7f5bb0148db0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:25.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.949+0000 7f5bb5731700 1 -- 172.21.15.162:0/1700575127 shutdown_connections 2020-11-28T21:53:25.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:25.949+0000 7f5bb5731700 1 -- 172.21.15.162:0/1700575127 wait complete. 2020-11-28T21:53:25.968 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config set mon mon_allow_pool_delete True 2020-11-28T21:53:26.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.089+0000 7f058f128700 1 Processor -- start 2020-11-28T21:53:26.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.089+0000 7f058f128700 1 -- start start 2020-11-28T21:53:26.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.093+0000 7f058f128700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f05880b3ab0 0x7f058805c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:26.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.093+0000 7f058f128700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f058805baf0 con 0x7f05880b30e0 2020-11-28T21:53:26.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.093+0000 7f058f128700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f058805bd90 con 0x7f05880b2d00 2020-11-28T21:53:26.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.093+0000 7f058f128700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05880b0e20 con 0x7f05880b3ab0 2020-11-28T21:53:26.098 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.093+0000 7f058d925700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f05880b3ab0 0x7f058805c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:26.098 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.093+0000 7f058e126700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f05880b2d00 0x7f0588142ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:48620/0 (socket says 172.21.15.162:48620) 2020-11-28T21:53:26.098 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.093+0000 7f058e126700 1 -- 172.21.15.162:0/833627257 learned_addr learned my addr 172.21.15.162:0/833627257 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:26.098 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.093+0000 7f058d925700 1 -- 172.21.15.162:0/833627257 >> v1:172.21.15.162:6789/0 conn(0x7f05880b2d00 legacy=0x7f0588142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:26.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.093+0000 7f058d925700 1 -- 172.21.15.162:0/833627257 >> v1:172.21.15.118:6789/0 conn(0x7f05880b30e0 legacy=0x7f058813fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:26.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.093+0000 7f058d925700 1 -- 172.21.15.162:0/833627257 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f058805c030 con 0x7f05880b3ab0 2020-11-28T21:53:26.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.093+0000 7f058d925700 1 --2- 172.21.15.162:0/833627257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05880b3ab0 0x7f058805c4d0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0x7f0578002f30 tx=0x7f0578006dc0).ready entity=mon.1 client_cookie=ff420f5f96fa69b8 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:26.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.093+0000 7f058d124700 1 -- 172.21.15.162:0/833627257 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05780097f0 con 0x7f05880b3ab0 2020-11-28T21:53:26.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.093+0000 7f058d124700 1 -- 172.21.15.162:0/833627257 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0578009950 con 0x7f05880b3ab0 2020-11-28T21:53:26.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.093+0000 7f058d124700 1 -- 172.21.15.162:0/833627257 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0578009cb0 con 0x7f05880b3ab0 2020-11-28T21:53:26.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.093+0000 7f058f128700 1 -- 172.21.15.162:0/833627257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05880b3ab0 msgr2=0x7f058805c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:26.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.093+0000 7f058f128700 1 --2- 172.21.15.162:0/833627257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05880b3ab0 0x7f058805c4d0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0x7f0578002f30 tx=0x7f0578006dc0).stop 2020-11-28T21:53:26.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.093+0000 7f058f128700 1 -- 172.21.15.162:0/833627257 shutdown_connections 2020-11-28T21:53:26.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.093+0000 7f058f128700 1 --2- 172.21.15.162:0/833627257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05880b3ab0 0x7f058805c4d0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:26.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.093+0000 7f058f128700 1 -- 172.21.15.162:0/833627257 >> 172.21.15.162:0/833627257 conn(0x7f05880604c0 msgr2=0x7f05880608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:26.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.093+0000 7f058f128700 1 -- 172.21.15.162:0/833627257 shutdown_connections 2020-11-28T21:53:26.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.093+0000 7f058f128700 1 -- 172.21.15.162:0/833627257 wait complete. 2020-11-28T21:53:26.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.093+0000 7f058f128700 1 Processor -- start 2020-11-28T21:53:26.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.093+0000 7f058f128700 1 -- start start 2020-11-28T21:53:26.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.097+0000 7f058f128700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f05880b2d00 0x7f058813e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:26.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.097+0000 7f058f128700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05880b30e0 0x7f0588152080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:26.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.097+0000 7f058f128700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f05880b3ab0 0x7f0588152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:26.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.097+0000 7f058f128700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0588147f60 con 0x7f05880b2d00 2020-11-28T21:53:26.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.097+0000 7f058f128700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05880b7fd0 con 0x7f05880b30e0 2020-11-28T21:53:26.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.097+0000 7f058f128700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05880b72c0 con 0x7f05880b3ab0 2020-11-28T21:53:26.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.097+0000 7f058d925700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05880b30e0 0x7f0588152080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:26.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.097+0000 7f058d925700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05880b30e0 0x7f0588152080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41784/0 (socket says 172.21.15.162:41784) 2020-11-28T21:53:26.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.097+0000 7f058d925700 1 -- 172.21.15.162:0/774042118 learned_addr learned my addr 172.21.15.162:0/774042118 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:26.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.097+0000 7f058e126700 1 --2- 172.21.15.162:0/774042118 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f05880b2d00 0x7f058813e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:26.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.097+0000 7f058e927700 1 --2- 172.21.15.162:0/774042118 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f05880b3ab0 0x7f0588152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:26.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.097+0000 7f058d925700 1 -- 172.21.15.162:0/774042118 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f05880b3ab0 msgr2=0x7f0588152570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:26.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.097+0000 7f058d925700 1 --2- 172.21.15.162:0/774042118 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f05880b3ab0 0x7f0588152570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:26.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.097+0000 7f058d925700 1 -- 172.21.15.162:0/774042118 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f05880b2d00 msgr2=0x7f058813e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:26.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.097+0000 7f058d925700 1 --2- 172.21.15.162:0/774042118 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f05880b2d00 0x7f058813e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:26.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.097+0000 7f058d925700 1 -- 172.21.15.162:0/774042118 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0588152a60 con 0x7f05880b30e0 2020-11-28T21:53:26.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.097+0000 7f058d925700 1 --2- 172.21.15.162:0/774042118 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05880b30e0 0x7f0588152080 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 rx=0x7f05881476a0 tx=0x7f0578015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:26.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.097+0000 7f05777fe700 1 -- 172.21.15.162:0/774042118 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0578011e10 con 0x7f05880b30e0 2020-11-28T21:53:26.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.097+0000 7f05777fe700 1 -- 172.21.15.162:0/774042118 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f057801d450 con 0x7f05880b30e0 2020-11-28T21:53:26.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.097+0000 7f058f128700 1 -- 172.21.15.162:0/774042118 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0588155c20 con 0x7f05880b30e0 2020-11-28T21:53:26.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.097+0000 7f05777fe700 1 -- 172.21.15.162:0/774042118 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f057801d7b0 con 0x7f05880b30e0 2020-11-28T21:53:26.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.097+0000 7f058f128700 1 -- 172.21.15.162:0/774042118 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0588151b70 con 0x7f05880b30e0 2020-11-28T21:53:26.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.097+0000 7f058f128700 1 -- 172.21.15.162:0/774042118 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0588151d90 con 0x7f05880b30e0 2020-11-28T21:53:26.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.097+0000 7f05777fe700 1 -- 172.21.15.162:0/774042118 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0578024030 con 0x7f05880b30e0 2020-11-28T21:53:26.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.101+0000 7f05777fe700 1 --2- 172.21.15.162:0/774042118 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0568055330 0x7f05680577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:26.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.101+0000 7f058e126700 1 --2- 172.21.15.162:0/774042118 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0568055330 0x7f05680577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:26.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.101+0000 7f05777fe700 1 -- 172.21.15.162:0/774042118 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f057806f240 con 0x7f05880b30e0 2020-11-28T21:53:26.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.101+0000 7f058e126700 1 --2- 172.21.15.162:0/774042118 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0568055330 0x7f05680577f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f058400d000 tx=0x7f0584009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:26.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.105+0000 7f05777fe700 1 -- 172.21.15.162:0/774042118 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0578045400 con 0x7f05880b30e0 2020-11-28T21:53:26.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.245+0000 7f058f128700 1 -- 172.21.15.162:0/774042118 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7f0588057050 con 0x7f05880b30e0 2020-11-28T21:53:26.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.253+0000 7f05777fe700 1 -- 172.21.15.162:0/774042118 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0578022020 con 0x7f05880b30e0 2020-11-28T21:53:26.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.253+0000 7f058f128700 1 -- 172.21.15.162:0/774042118 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0568055330 msgr2=0x7f05680577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:26.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.253+0000 7f058f128700 1 --2- 172.21.15.162:0/774042118 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0568055330 0x7f05680577f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f058400d000 tx=0x7f0584009040).stop 2020-11-28T21:53:26.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.253+0000 7f058f128700 1 -- 172.21.15.162:0/774042118 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05880b30e0 msgr2=0x7f0588152080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:26.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.253+0000 7f058f128700 1 --2- 172.21.15.162:0/774042118 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05880b30e0 0x7f0588152080 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 rx=0x7f05881476a0 tx=0x7f0578015000).stop 2020-11-28T21:53:26.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.253+0000 7f058f128700 1 -- 172.21.15.162:0/774042118 shutdown_connections 2020-11-28T21:53:26.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.253+0000 7f058f128700 1 --2- 172.21.15.162:0/774042118 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0568055330 0x7f05680577f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:26.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.253+0000 7f058f128700 1 --2- 172.21.15.162:0/774042118 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f05880b3ab0 0x7f0588152570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:26.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.253+0000 7f058f128700 1 --2- 172.21.15.162:0/774042118 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f05880b2d00 0x7f058813e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:26.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.253+0000 7f058f128700 1 --2- 172.21.15.162:0/774042118 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05880b30e0 0x7f0588152080 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:26.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.253+0000 7f058f128700 1 -- 172.21.15.162:0/774042118 >> 172.21.15.162:0/774042118 conn(0x7f05880604c0 msgr2=0x7f05880b18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:26.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.253+0000 7f058f128700 1 -- 172.21.15.162:0/774042118 shutdown_connections 2020-11-28T21:53:26.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.253+0000 7f058f128700 1 -- 172.21.15.162:0/774042118 wait complete. 2020-11-28T21:53:26.271 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-28T21:53:26.272 INFO:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:53:26.290 INFO:teuthology.orchestra.run.smithi118:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:53:26.307 INFO:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:53:26.326 INFO:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:53:26.344 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-28T21:53:26.345 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:53:26.345 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:53:26.345 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:53:26.393 INFO:teuthology.orchestra.run.smithi192.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2020-11-28T21:53:26.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:53:26.395 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2020-11-28T21:53:26.395 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:53:26.396 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:53:26.396 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:53:26.403 INFO:teuthology.orchestra.run.smithi096.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-11-28T21:53:26.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:53:26.405 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2020-11-28T21:53:26.405 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2020-11-28T21:53:26.522 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.521+0000 7f77ed34d700 1 Processor -- start 2020-11-28T21:53:26.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.521+0000 7f77ed34d700 1 -- start start 2020-11-28T21:53:26.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.521+0000 7f77ed34d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f77e809abb0 0x7f77e8061190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:26.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.521+0000 7f77ed34d700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f77e80b01a0 0x7f77e8143f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:26.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.521+0000 7f77ed34d700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77e8143ba0 con 0x7f77e8060010 2020-11-28T21:53:26.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.521+0000 7f77ed34d700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77e8144b60 con 0x7f77e809abb0 2020-11-28T21:53:26.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.521+0000 7f77ed34d700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77e809af90 con 0x7f77e80b01a0 2020-11-28T21:53:26.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.521+0000 7f77e7fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f77e809abb0 0x7f77e8061190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:26.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.521+0000 7f77ecb4c700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f77e8060010 0x7f77e8144450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:48638/0 (socket says 172.21.15.162:48638) 2020-11-28T21:53:26.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.521+0000 7f77ecb4c700 1 -- 172.21.15.162:0/729365448 learned_addr learned my addr 172.21.15.162:0/729365448 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:26.527 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.521+0000 7f77e77fe700 1 --2- 172.21.15.162:0/729365448 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f77e80b01a0 0x7f77e8143f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:26.527 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.521+0000 7f77e7fff700 1 -- 172.21.15.162:0/729365448 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f77e80b01a0 msgr2=0x7f77e8143f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:26.527 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.521+0000 7f77e7fff700 1 --2- 172.21.15.162:0/729365448 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f77e80b01a0 0x7f77e8143f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:26.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.521+0000 7f77e7fff700 1 -- 172.21.15.162:0/729365448 >> v1:172.21.15.162:6789/0 conn(0x7f77e8060010 legacy=0x7f77e8144450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:26.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.521+0000 7f77e6ffd700 1 -- 172.21.15.162:0/729365448 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4113678474 0 0) 0x7f77e8143ba0 con 0x7f77e8060010 2020-11-28T21:53:26.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.521+0000 7f77e7fff700 1 -- 172.21.15.162:0/729365448 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e80b1d10 con 0x7f77e809abb0 2020-11-28T21:53:26.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.521+0000 7f77e7fff700 1 --2- 172.21.15.162:0/729365448 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f77e809abb0 0x7f77e8061190 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 rx=0x7f77e00032b0 tx=0x7f77e00091a0).ready entity=mon.1 client_cookie=721dbbd633b88b28 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:26.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.521+0000 7f77e6ffd700 1 -- 172.21.15.162:0/729365448 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77e000ba50 con 0x7f77e809abb0 2020-11-28T21:53:26.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.521+0000 7f77e6ffd700 1 -- 172.21.15.162:0/729365448 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e000bbb0 con 0x7f77e809abb0 2020-11-28T21:53:26.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.521+0000 7f77e6ffd700 1 -- 172.21.15.162:0/729365448 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77e000bd20 con 0x7f77e809abb0 2020-11-28T21:53:26.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.521+0000 7f77ed34d700 1 -- 172.21.15.162:0/729365448 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f77e809abb0 msgr2=0x7f77e8061190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:26.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.521+0000 7f77ed34d700 1 --2- 172.21.15.162:0/729365448 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f77e809abb0 0x7f77e8061190 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 rx=0x7f77e00032b0 tx=0x7f77e00091a0).stop 2020-11-28T21:53:26.530 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.525+0000 7f77ed34d700 1 -- 172.21.15.162:0/729365448 shutdown_connections 2020-11-28T21:53:26.530 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.525+0000 7f77ed34d700 1 --2- 172.21.15.162:0/729365448 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f77e809abb0 0x7f77e8061190 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:26.530 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.525+0000 7f77ed34d700 1 --2- 172.21.15.162:0/729365448 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f77e80b01a0 0x7f77e8143f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:26.530 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.525+0000 7f77ed34d700 1 -- 172.21.15.162:0/729365448 >> 172.21.15.162:0/729365448 conn(0x7f77e80b54f0 msgr2=0x7f77e80b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:26.530 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.525+0000 7f77ed34d700 1 -- 172.21.15.162:0/729365448 shutdown_connections 2020-11-28T21:53:26.531 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.525+0000 7f77ed34d700 1 -- 172.21.15.162:0/729365448 wait complete. 2020-11-28T21:53:26.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.525+0000 7f77ed34d700 1 Processor -- start 2020-11-28T21:53:26.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.525+0000 7f77ed34d700 1 -- start start 2020-11-28T21:53:26.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.525+0000 7f77ed34d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f77e8060010 0x7f77e8064cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:26.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.525+0000 7f77ed34d700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f77e809abb0 0x7f77e8066610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:26.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.525+0000 7f77ed34d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f77e80b01a0 0x7f77e813d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:26.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.525+0000 7f77ed34d700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e80677e0 con 0x7f77e80b01a0 2020-11-28T21:53:26.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.525+0000 7f77ed34d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e80aaa50 con 0x7f77e8060010 2020-11-28T21:53:26.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.525+0000 7f77ed34d700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77e80aacf0 con 0x7f77e809abb0 2020-11-28T21:53:26.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.525+0000 7f77e7fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f77e8060010 0x7f77e8064cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:26.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.525+0000 7f77e7fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f77e8060010 0x7f77e8064cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41796/0 (socket says 172.21.15.162:41796) 2020-11-28T21:53:26.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.525+0000 7f77e7fff700 1 -- 172.21.15.162:0/3769886603 learned_addr learned my addr 172.21.15.162:0/3769886603 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:26.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.525+0000 7f77e77fe700 1 --2- 172.21.15.162:0/3769886603 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f77e809abb0 0x7f77e8066610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:26.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.525+0000 7f77ecb4c700 1 --2- 172.21.15.162:0/3769886603 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f77e80b01a0 0x7f77e813d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:26.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.529+0000 7f77e7fff700 1 -- 172.21.15.162:0/3769886603 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f77e809abb0 msgr2=0x7f77e8066610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:26.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.529+0000 7f77e7fff700 1 --2- 172.21.15.162:0/3769886603 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f77e809abb0 0x7f77e8066610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:26.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.529+0000 7f77e7fff700 1 -- 172.21.15.162:0/3769886603 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f77e80b01a0 msgr2=0x7f77e813d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:26.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.529+0000 7f77e7fff700 1 --2- 172.21.15.162:0/3769886603 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f77e80b01a0 0x7f77e813d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:26.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.529+0000 7f77e7fff700 1 -- 172.21.15.162:0/3769886603 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e80aaf90 con 0x7f77e8060010 2020-11-28T21:53:26.537 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.529+0000 7f77e7fff700 1 --2- 172.21.15.162:0/3769886603 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f77e8060010 0x7f77e8064cc0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 rx=0x7f77e00032b0 tx=0x7f77e0016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:26.537 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.529+0000 7f77e57fa700 1 -- 172.21.15.162:0/3769886603 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77e001f5e0 con 0x7f77e8060010 2020-11-28T21:53:26.537 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.529+0000 7f77e57fa700 1 -- 172.21.15.162:0/3769886603 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e001f740 con 0x7f77e8060010 2020-11-28T21:53:26.537 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.529+0000 7f77ed34d700 1 -- 172.21.15.162:0/3769886603 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77e80ab230 con 0x7f77e8060010 2020-11-28T21:53:26.537 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.529+0000 7f77e57fa700 1 -- 172.21.15.162:0/3769886603 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77e001faa0 con 0x7f77e8060010 2020-11-28T21:53:26.538 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.529+0000 7f77ed34d700 1 -- 172.21.15.162:0/3769886603 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77e8066b00 con 0x7f77e8060010 2020-11-28T21:53:26.538 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.529+0000 7f77ed34d700 1 -- 172.21.15.162:0/3769886603 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77e8057050 con 0x7f77e8060010 2020-11-28T21:53:26.538 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.529+0000 7f77e57fa700 1 -- 172.21.15.162:0/3769886603 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f77e0026030 con 0x7f77e8060010 2020-11-28T21:53:26.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.529+0000 7f77e57fa700 1 --2- 172.21.15.162:0/3769886603 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f77cc055330 0x7f77cc0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:26.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.533+0000 7f77e57fa700 1 -- 172.21.15.162:0/3769886603 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f77e006f210 con 0x7f77e8060010 2020-11-28T21:53:26.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.533+0000 7f77e77fe700 1 --2- 172.21.15.162:0/3769886603 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f77cc055330 0x7f77cc0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:26.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.533+0000 7f77e77fe700 1 --2- 172.21.15.162:0/3769886603 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f77cc055330 0x7f77cc0577f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f77d000c000 tx=0x7f77d0008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:26.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.537+0000 7f77e57fa700 1 -- 172.21.15.162:0/3769886603 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f77e0045560 con 0x7f77e8060010 2020-11-28T21:53:26.682 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.681+0000 7f77ed34d700 1 -- 172.21.15.162:0/3769886603 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f77e805d5e0 con 0x7f77e8060010 2020-11-28T21:53:26.684 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.681+0000 7f77e57fa700 1 -- 172.21.15.162:0/3769886603 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v77) v1 ==== 81+0+33821 (secure 0 0 0) 0x7f77e0015090 con 0x7f77e8060010 2020-11-28T21:53:26.685 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:53:26.686 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2020-11-28T21:53:26.686 INFO:teuthology.orchestra.run.smithi162.stdout: "epoch": 77, 2020-11-28T21:53:26.686 INFO:teuthology.orchestra.run.smithi162.stdout: "fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", 2020-11-28T21:53:26.686 INFO:teuthology.orchestra.run.smithi162.stdout: "created": "2020-11-28T21:46:11.724372+0000", 2020-11-28T21:53:26.686 INFO:teuthology.orchestra.run.smithi162.stdout: "modified": "2020-11-28T21:53:13.878105+0000", 2020-11-28T21:53:26.687 INFO:teuthology.orchestra.run.smithi162.stdout: "last_up_change": "2020-11-28T21:46:19.591650+0000", 2020-11-28T21:53:26.687 INFO:teuthology.orchestra.run.smithi162.stdout: "last_in_change": "2020-11-28T21:46:19.591650+0000", 2020-11-28T21:53:26.687 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-11-28T21:53:26.687 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_num": 5799936, 2020-11-28T21:53:26.688 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_set": [ 2020-11-28T21:53:26.688 INFO:teuthology.orchestra.run.smithi162.stdout: "pglog_hardlimit", 2020-11-28T21:53:26.688 INFO:teuthology.orchestra.run.smithi162.stdout: "purged_snapdirs", 2020-11-28T21:53:26.688 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_deletes", 2020-11-28T21:53:26.689 INFO:teuthology.orchestra.run.smithi162.stdout: "sortbitwise" 2020-11-28T21:53:26.689 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:53:26.689 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_version": 4, 2020-11-28T21:53:26.689 INFO:teuthology.orchestra.run.smithi162.stdout: "full_ratio": 0.94999998807907104, 2020-11-28T21:53:26.689 INFO:teuthology.orchestra.run.smithi162.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-11-28T21:53:26.690 INFO:teuthology.orchestra.run.smithi162.stdout: "nearfull_ratio": 0.85000002384185791, 2020-11-28T21:53:26.690 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_snapshot": "", 2020-11-28T21:53:26.690 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_max": 14, 2020-11-28T21:53:26.690 INFO:teuthology.orchestra.run.smithi162.stdout: "max_osd": 8, 2020-11-28T21:53:26.690 INFO:teuthology.orchestra.run.smithi162.stdout: "require_min_compat_client": "jewel", 2020-11-28T21:53:26.691 INFO:teuthology.orchestra.run.smithi162.stdout: "min_compat_client": "jewel", 2020-11-28T21:53:26.691 INFO:teuthology.orchestra.run.smithi162.stdout: "require_osd_release": "octopus", 2020-11-28T21:53:26.691 INFO:teuthology.orchestra.run.smithi162.stdout: "pools": [ 2020-11-28T21:53:26.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.691 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 1, 2020-11-28T21:53:26.692 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "device_health_metrics", 2020-11-28T21:53:26.692 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:46:15.639551+0000", 2020-11-28T21:53:26.692 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:53:26.692 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:53:26.692 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:53:26.693 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:53:26.693 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:53:26.693 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:53:26.693 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:53:26.693 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:53:26.694 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 1, 2020-11-28T21:53:26.694 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 1, 2020-11-28T21:53:26.694 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 1, 2020-11-28T21:53:26.694 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 1, 2020-11-28T21:53:26.694 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 1, 2020-11-28T21:53:26.695 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:53:26.695 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:53:26.695 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:53:26.695 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:53:26.695 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:53:26.696 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:53:26.696 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:53:26.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.696 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "16", 2020-11-28T21:53:26.696 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:53:26.697 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:53:26.697 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:53:26.697 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:53:26.697 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:53:26.697 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:53:26.698 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:53:26.698 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:53:26.698 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:53:26.698 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:53:26.698 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:53:26.699 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:53:26.699 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:53:26.699 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:53:26.699 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:53:26.699 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:53:26.700 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:53:26.700 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:53:26.700 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:53:26.700 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:53:26.700 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:53:26.701 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:53:26.701 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:53:26.701 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:53:26.701 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:53:26.701 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:53:26.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.702 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:53:26.702 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:53:26.702 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:53:26.702 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:53:26.703 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:53:26.703 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:53:26.703 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:53:26.703 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:53:26.703 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:53:26.704 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:53:26.704 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:53:26.704 INFO:teuthology.orchestra.run.smithi162.stdout: "options": { 2020-11-28T21:53:26.704 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_min": 1 2020-11-28T21:53:26.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.705 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:53:26.705 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr_devicehealth": {} 2020-11-28T21:53:26.705 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:26.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.706 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 2, 2020-11-28T21:53:26.706 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "rbd", 2020-11-28T21:53:26.706 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:46:23.901366+0000", 2020-11-28T21:53:26.706 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:53:26.707 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:53:26.707 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:53:26.707 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:53:26.707 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:53:26.707 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:53:26.708 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:53:26.708 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:53:26.708 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2020-11-28T21:53:26.708 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2020-11-28T21:53:26.708 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2020-11-28T21:53:26.709 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2020-11-28T21:53:26.709 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2020-11-28T21:53:26.709 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:53:26.709 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:53:26.709 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:53:26.710 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:53:26.710 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:53:26.710 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:53:26.710 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:53:26.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.711 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "19", 2020-11-28T21:53:26.711 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:53:26.711 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:53:26.711 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:53:26.711 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:53:26.712 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:53:26.712 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:53:26.712 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:53:26.712 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:53:26.712 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:53:26.713 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:53:26.713 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:53:26.713 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:53:26.713 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:53:26.713 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:53:26.714 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:53:26.714 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:53:26.714 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:53:26.714 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:53:26.714 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:53:26.715 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:53:26.715 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:53:26.715 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:53:26.715 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:53:26.715 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:53:26.716 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:53:26.716 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:53:26.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.716 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:53:26.716 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:53:26.717 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:53:26.717 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:53:26.717 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:53:26.717 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:53:26.717 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:53:26.718 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:53:26.718 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:53:26.718 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:53:26.718 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:53:26.718 INFO:teuthology.orchestra.run.smithi162.stdout: "options": {}, 2020-11-28T21:53:26.719 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:53:26.719 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd": {} 2020-11-28T21:53:26.719 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:26.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.720 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 13, 2020-11-28T21:53:26.720 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "cephfs_metadata", 2020-11-28T21:53:26.720 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:52:25.955831+0000", 2020-11-28T21:53:26.720 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:53:26.720 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:53:26.721 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:53:26.721 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:53:26.721 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:53:26.721 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:53:26.721 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:53:26.721 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:53:26.722 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2020-11-28T21:53:26.722 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2020-11-28T21:53:26.722 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2020-11-28T21:53:26.722 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2020-11-28T21:53:26.722 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2020-11-28T21:53:26.723 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:53:26.723 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:53:26.723 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:53:26.723 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:53:26.723 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:53:26.724 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:53:26.724 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:53:26.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.724 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "76", 2020-11-28T21:53:26.724 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:53:26.725 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:53:26.725 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:53:26.725 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:53:26.725 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:53:26.725 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:53:26.726 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:53:26.726 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:53:26.726 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:53:26.726 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:53:26.726 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:53:26.727 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:53:26.727 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:53:26.727 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:53:26.727 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:53:26.727 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:53:26.728 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:53:26.728 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:53:26.728 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:53:26.728 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:53:26.728 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:53:26.729 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:53:26.729 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:53:26.729 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:53:26.729 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:53:26.729 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:53:26.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.730 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:53:26.730 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:53:26.730 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:53:26.730 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:53:26.731 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:53:26.731 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:53:26.731 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:53:26.731 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:53:26.731 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:53:26.732 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:53:26.732 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:53:26.732 INFO:teuthology.orchestra.run.smithi162.stdout: "options": { 2020-11-28T21:53:26.732 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_bias": 4, 2020-11-28T21:53:26.732 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_min": 16, 2020-11-28T21:53:26.733 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_priority": 5 2020-11-28T21:53:26.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.733 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:53:26.733 INFO:teuthology.orchestra.run.smithi162.stdout: "cephfs": { 2020-11-28T21:53:26.733 INFO:teuthology.orchestra.run.smithi162.stdout: "metadata": "cephfs" 2020-11-28T21:53:26.734 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:26.734 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:26.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.734 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 14, 2020-11-28T21:53:26.735 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "cephfs_data", 2020-11-28T21:53:26.735 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:52:27.265702+0000", 2020-11-28T21:53:26.735 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:53:26.735 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:53:26.735 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:53:26.736 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:53:26.736 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:53:26.736 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:53:26.736 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:53:26.736 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:53:26.737 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2020-11-28T21:53:26.737 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2020-11-28T21:53:26.737 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2020-11-28T21:53:26.737 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2020-11-28T21:53:26.737 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2020-11-28T21:53:26.738 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:53:26.738 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:53:26.738 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:53:26.738 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:53:26.739 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:53:26.739 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:53:26.739 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:53:26.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.739 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "76", 2020-11-28T21:53:26.740 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:53:26.740 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:53:26.740 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:53:26.740 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:53:26.740 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:53:26.741 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:53:26.741 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:53:26.741 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:53:26.741 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:53:26.741 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:53:26.742 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:53:26.742 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:53:26.742 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:53:26.742 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:53:26.742 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:53:26.743 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:53:26.743 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:53:26.743 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:53:26.743 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:53:26.743 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:53:26.744 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:53:26.744 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:53:26.744 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:53:26.744 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:53:26.744 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:53:26.745 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:53:26.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.745 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:53:26.745 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:53:26.745 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:53:26.746 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:53:26.746 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:53:26.746 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:53:26.746 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:53:26.746 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:53:26.747 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:53:26.747 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:53:26.747 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:53:26.747 INFO:teuthology.orchestra.run.smithi162.stdout: "options": {}, 2020-11-28T21:53:26.748 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:53:26.748 INFO:teuthology.orchestra.run.smithi162.stdout: "cephfs": { 2020-11-28T21:53:26.748 INFO:teuthology.orchestra.run.smithi162.stdout: "data": "cephfs" 2020-11-28T21:53:26.748 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:26.748 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:26.749 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:26.749 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:53:26.749 INFO:teuthology.orchestra.run.smithi162.stdout: "osds": [ 2020-11-28T21:53:26.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.749 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 0, 2020-11-28T21:53:26.750 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "99c7d9ae-adfa-4fd7-880a-23e6baf3281b", 2020-11-28T21:53:26.750 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:53:26.750 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:53:26.750 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:53:26.750 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:53:26.751 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:53:26.751 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:53:26.751 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:53:26.751 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 64, 2020-11-28T21:53:26.751 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:53:26.752 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:53:26.752 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:53:26.752 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:26.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.752 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:26.753 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6810", 2020-11-28T21:53:26.753 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:53:26.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.753 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:26.754 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6811", 2020-11-28T21:53:26.754 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:53:26.754 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:26.754 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:26.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.755 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:53:26.755 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:26.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.755 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:26.755 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6812", 2020-11-28T21:53:26.756 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:53:26.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.756 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:26.757 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6813", 2020-11-28T21:53:26.757 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:53:26.757 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:26.757 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:26.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.758 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:53:26.758 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:26.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.758 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:26.758 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6816", 2020-11-28T21:53:26.759 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:53:26.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.759 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:26.759 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6817", 2020-11-28T21:53:26.760 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:53:26.760 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:26.760 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:26.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.760 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:53:26.761 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:26.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.761 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:26.761 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6814", 2020-11-28T21:53:26.761 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:53:26.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.762 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:26.762 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6815", 2020-11-28T21:53:26.762 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:53:26.763 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:26.763 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:26.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.763 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6811/12313", 2020-11-28T21:53:26.763 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6813/12313", 2020-11-28T21:53:26.764 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6817/12313", 2020-11-28T21:53:26.764 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6815/12313", 2020-11-28T21:53:26.764 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:53:26.764 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:53:26.764 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:53:26.765 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:26.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.765 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 1, 2020-11-28T21:53:26.765 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "6ce6220a-00ef-4d13-abba-56f87636fb5f", 2020-11-28T21:53:26.766 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:53:26.766 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:53:26.766 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:53:26.766 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:53:26.766 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:53:26.767 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:53:26.767 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:53:26.767 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 74, 2020-11-28T21:53:26.767 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:53:26.767 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:53:26.768 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:53:26.768 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:26.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.768 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:26.768 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6802", 2020-11-28T21:53:26.768 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:53:26.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.769 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:26.769 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6803", 2020-11-28T21:53:26.769 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:53:26.770 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:26.770 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:26.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.770 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:53:26.770 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:26.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.771 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:26.771 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6804", 2020-11-28T21:53:26.771 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:53:26.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.772 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:26.772 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6805", 2020-11-28T21:53:26.772 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:53:26.772 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:26.773 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:26.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.773 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:53:26.773 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:26.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.774 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:26.774 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6808", 2020-11-28T21:53:26.774 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:53:26.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.775 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:26.775 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6809", 2020-11-28T21:53:26.775 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:53:26.775 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:26.775 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:26.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.776 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:53:26.776 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:26.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.776 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:26.777 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6806", 2020-11-28T21:53:26.777 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:53:26.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.777 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:26.777 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6807", 2020-11-28T21:53:26.778 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:53:26.778 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:26.778 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:26.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.778 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6803/12311", 2020-11-28T21:53:26.779 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6805/12311", 2020-11-28T21:53:26.779 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6809/12311", 2020-11-28T21:53:26.779 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6807/12311", 2020-11-28T21:53:26.779 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:53:26.779 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:53:26.780 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:53:26.780 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:26.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.780 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 2, 2020-11-28T21:53:26.781 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "db4e510d-27c1-47fc-a902-8b32e7dfd73e", 2020-11-28T21:53:26.781 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:53:26.781 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:53:26.781 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:53:26.781 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:53:26.782 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:53:26.782 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:53:26.782 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:53:26.782 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 74, 2020-11-28T21:53:26.782 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:53:26.783 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:53:26.783 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:53:26.783 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:26.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.783 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:26.784 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6818", 2020-11-28T21:53:26.784 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:53:26.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.785 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:26.786 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6819", 2020-11-28T21:53:26.786 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:53:26.786 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:26.786 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:26.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.787 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:53:26.787 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:26.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.787 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:26.787 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6820", 2020-11-28T21:53:26.788 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:53:26.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.788 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:26.788 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6821", 2020-11-28T21:53:26.789 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:53:26.789 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:26.789 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:26.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.790 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:53:26.790 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:26.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.790 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:26.790 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6824", 2020-11-28T21:53:26.791 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:53:26.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.791 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:26.791 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6825", 2020-11-28T21:53:26.792 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:53:26.792 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:26.792 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:26.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.792 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:53:26.793 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:26.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.793 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:26.793 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6822", 2020-11-28T21:53:26.793 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:53:26.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.794 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:26.794 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6823", 2020-11-28T21:53:26.794 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:53:26.795 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:26.795 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:26.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.795 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6819/12312", 2020-11-28T21:53:26.795 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6821/12312", 2020-11-28T21:53:26.796 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6825/12312", 2020-11-28T21:53:26.796 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6823/12312", 2020-11-28T21:53:26.796 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:53:26.796 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:53:26.796 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:53:26.797 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:26.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.797 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 3, 2020-11-28T21:53:26.797 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "4094cb87-f04a-45e1-8710-fb4b651949f6", 2020-11-28T21:53:26.797 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:53:26.798 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:53:26.798 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:53:26.798 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:53:26.798 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:53:26.799 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:53:26.799 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:53:26.799 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 73, 2020-11-28T21:53:26.799 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:53:26.799 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:53:26.799 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:53:26.800 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:26.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.800 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:26.800 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6826", 2020-11-28T21:53:26.800 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:53:26.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.801 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:26.801 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6827", 2020-11-28T21:53:26.801 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:53:26.802 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:26.802 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:26.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.802 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:53:26.802 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:26.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.803 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:26.803 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6828", 2020-11-28T21:53:26.803 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:53:26.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.804 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:26.804 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6829", 2020-11-28T21:53:26.804 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:53:26.804 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:26.805 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:26.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.805 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:53:26.805 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:26.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.806 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:26.806 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6832", 2020-11-28T21:53:26.806 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:53:26.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.806 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.807 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:26.807 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6833", 2020-11-28T21:53:26.807 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:53:26.808 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:26.808 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:26.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.808 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:53:26.808 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:26.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.809 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:26.809 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6830", 2020-11-28T21:53:26.809 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:53:26.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.810 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:26.810 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6831", 2020-11-28T21:53:26.810 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:53:26.810 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:26.811 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:26.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.811 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6827/12323", 2020-11-28T21:53:26.811 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6829/12323", 2020-11-28T21:53:26.811 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6833/12323", 2020-11-28T21:53:26.811 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6831/12323", 2020-11-28T21:53:26.812 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:53:26.812 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:53:26.812 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:53:26.812 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:26.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.813 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 4, 2020-11-28T21:53:26.813 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "b907bf53-c297-4756-847b-3be96680eef5", 2020-11-28T21:53:26.813 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:53:26.813 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:53:26.814 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:53:26.814 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:53:26.814 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:53:26.814 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:53:26.814 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:53:26.815 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 74, 2020-11-28T21:53:26.815 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:53:26.815 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:53:26.815 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:53:26.815 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:26.816 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.816 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:26.816 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6800", 2020-11-28T21:53:26.816 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:53:26.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.817 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.817 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:26.817 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6801", 2020-11-28T21:53:26.817 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:53:26.817 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:26.818 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:26.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.818 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:53:26.818 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:26.818 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.819 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:26.819 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6802", 2020-11-28T21:53:26.819 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:53:26.819 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.819 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.820 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:26.820 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6803", 2020-11-28T21:53:26.820 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:53:26.820 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:26.820 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:26.821 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.821 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:53:26.821 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:26.821 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.821 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:26.821 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6806", 2020-11-28T21:53:26.822 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:53:26.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.822 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.822 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:26.822 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6807", 2020-11-28T21:53:26.823 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:53:26.823 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:26.823 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:26.823 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.823 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:53:26.824 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:26.824 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.824 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:26.824 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6804", 2020-11-28T21:53:26.824 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:53:26.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.825 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.825 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:26.825 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6805", 2020-11-28T21:53:26.825 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:53:26.826 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:26.826 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:26.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.826 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6801/12527", 2020-11-28T21:53:26.826 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6803/12527", 2020-11-28T21:53:26.827 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6807/12527", 2020-11-28T21:53:26.827 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6805/12527", 2020-11-28T21:53:26.827 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:53:26.827 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:53:26.827 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:53:26.828 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:26.828 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.828 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.828 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 5, 2020-11-28T21:53:26.828 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "55ba305d-0505-4212-aebb-631e8554b440", 2020-11-28T21:53:26.829 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:53:26.829 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:53:26.829 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:53:26.829 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:53:26.829 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:53:26.830 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:53:26.830 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:53:26.830 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 74, 2020-11-28T21:53:26.830 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:53:26.830 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:53:26.831 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:53:26.831 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:26.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.831 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:26.831 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6808", 2020-11-28T21:53:26.831 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:53:26.832 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.832 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.832 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:26.832 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6809", 2020-11-28T21:53:26.832 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:53:26.833 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:26.833 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:26.833 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.833 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:53:26.833 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:26.834 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.834 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:26.834 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6810", 2020-11-28T21:53:26.834 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:53:26.834 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.835 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.835 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:26.835 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6811", 2020-11-28T21:53:26.835 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:53:26.835 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:26.836 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:26.836 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.836 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:53:26.836 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:26.836 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.837 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:26.837 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6814", 2020-11-28T21:53:26.837 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:53:26.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.838 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:26.838 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6815", 2020-11-28T21:53:26.838 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:53:26.838 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:26.839 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:26.839 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.839 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:53:26.839 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:26.840 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.840 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:26.840 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6812", 2020-11-28T21:53:26.840 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:53:26.840 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.840 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.841 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:26.841 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6813", 2020-11-28T21:53:26.841 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:53:26.841 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:26.841 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:26.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.842 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6809/12529", 2020-11-28T21:53:26.842 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6811/12529", 2020-11-28T21:53:26.842 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6815/12529", 2020-11-28T21:53:26.842 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6813/12529", 2020-11-28T21:53:26.843 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:53:26.843 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:53:26.843 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:53:26.843 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:26.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.844 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 6, 2020-11-28T21:53:26.844 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "9ada6051-493b-46b5-ba22-1b401440490f", 2020-11-28T21:53:26.844 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:53:26.844 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:53:26.845 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:53:26.845 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:53:26.845 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:53:26.845 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:53:26.845 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:53:26.846 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 72, 2020-11-28T21:53:26.846 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:53:26.846 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:53:26.846 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:53:26.846 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:26.847 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.847 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:26.847 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6824", 2020-11-28T21:53:26.847 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:53:26.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.848 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.848 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:26.848 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6825", 2020-11-28T21:53:26.848 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:53:26.849 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:26.849 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:26.849 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.849 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:53:26.849 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:26.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.850 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:26.850 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6826", 2020-11-28T21:53:26.850 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:53:26.851 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.851 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:26.851 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6827", 2020-11-28T21:53:26.851 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:53:26.851 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:26.852 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:26.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.852 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:53:26.852 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:26.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.853 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:26.853 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6830", 2020-11-28T21:53:26.853 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:53:26.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.854 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:26.854 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6831", 2020-11-28T21:53:26.854 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:53:26.854 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:26.854 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:26.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.855 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:53:26.855 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:26.855 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.856 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:26.856 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6828", 2020-11-28T21:53:26.856 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:53:26.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.857 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:26.857 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6829", 2020-11-28T21:53:26.857 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:53:26.857 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:26.858 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:26.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.858 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6825/12530", 2020-11-28T21:53:26.858 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6827/12530", 2020-11-28T21:53:26.858 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6831/12530", 2020-11-28T21:53:26.859 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6829/12530", 2020-11-28T21:53:26.859 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:53:26.859 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:53:26.859 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:53:26.859 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:26.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.860 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.860 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 7, 2020-11-28T21:53:26.860 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "c328b06d-a216-4cba-aad7-9ad081340613", 2020-11-28T21:53:26.860 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:53:26.861 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:53:26.861 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:53:26.861 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:53:26.861 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:53:26.861 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:53:26.861 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:53:26.862 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 74, 2020-11-28T21:53:26.862 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:53:26.862 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:53:26.862 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:53:26.863 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:26.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.863 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:26.863 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6816", 2020-11-28T21:53:26.863 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:53:26.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.864 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.864 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:26.864 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6817", 2020-11-28T21:53:26.864 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:53:26.864 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:26.865 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:26.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.865 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:53:26.865 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:26.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.866 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:26.866 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6818", 2020-11-28T21:53:26.866 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:53:26.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.867 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:26.867 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6819", 2020-11-28T21:53:26.867 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:53:26.867 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:26.867 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:26.868 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.868 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:53:26.868 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:26.868 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.868 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:26.869 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6822", 2020-11-28T21:53:26.869 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:53:26.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.869 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:26.870 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6823", 2020-11-28T21:53:26.870 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:53:26.870 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:26.870 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:26.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.871 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:53:26.871 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:26.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.871 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:26.871 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6820", 2020-11-28T21:53:26.872 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:53:26.872 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.872 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.872 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:26.872 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6821", 2020-11-28T21:53:26.873 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:53:26.873 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:26.873 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:26.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.873 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6817/12528", 2020-11-28T21:53:26.874 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6819/12528", 2020-11-28T21:53:26.874 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6823/12528", 2020-11-28T21:53:26.874 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6821/12528", 2020-11-28T21:53:26.874 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:53:26.874 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:53:26.875 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:53:26.875 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:26.875 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:26.875 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:53:26.875 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_xinfo": [ 2020-11-28T21:53:26.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.876 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 0, 2020-11-28T21:53:26.876 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:53:26.876 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:53:26.876 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:53:26.876 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:53:26.877 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:53:26.877 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.038052+0000", 2020-11-28T21:53:26.877 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:53:26.877 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.877 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.878 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 1, 2020-11-28T21:53:26.878 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:53:26.878 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:53:26.878 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:53:26.878 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:53:26.879 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:53:26.879 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:17.947136+0000", 2020-11-28T21:53:26.879 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:53:26.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.879 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.880 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 2, 2020-11-28T21:53:26.880 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:53:26.880 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:53:26.880 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:53:26.880 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:53:26.881 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:53:26.881 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.001186+0000", 2020-11-28T21:53:26.881 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:53:26.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.882 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 3, 2020-11-28T21:53:26.882 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:53:26.882 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:53:26.882 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:53:26.882 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:53:26.883 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:53:26.883 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.199284+0000", 2020-11-28T21:53:26.883 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:53:26.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.884 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 4, 2020-11-28T21:53:26.884 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:53:26.884 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:53:26.884 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:53:26.884 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:53:26.885 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:53:26.885 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.038823+0000", 2020-11-28T21:53:26.885 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:53:26.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.885 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 5, 2020-11-28T21:53:26.886 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:53:26.886 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:53:26.886 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:53:26.886 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:53:26.886 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:53:26.887 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.009322+0000", 2020-11-28T21:53:26.887 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:53:26.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.887 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 6, 2020-11-28T21:53:26.888 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:53:26.888 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:53:26.888 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:53:26.888 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:53:26.888 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:53:26.889 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.238927+0000", 2020-11-28T21:53:26.889 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:53:26.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:26.890 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 7, 2020-11-28T21:53:26.890 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:53:26.890 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:53:26.890 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:53:26.890 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:53:26.891 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:53:26.891 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.029839+0000", 2020-11-28T21:53:26.891 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:53:26.891 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:26.891 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:53:26.892 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_upmap": [], 2020-11-28T21:53:26.892 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_upmap_items": [], 2020-11-28T21:53:26.892 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_temp": [], 2020-11-28T21:53:26.892 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_temp": [], 2020-11-28T21:53:26.892 INFO:teuthology.orchestra.run.smithi162.stdout: "blacklist": { 2020-11-28T21:53:26.893 INFO:teuthology.orchestra.run.smithi162.stdout: "172.21.15.192:0/2498670596": "2020-11-28T22:53:13.037657+0000" 2020-11-28T21:53:26.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.893 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profiles": { 2020-11-28T21:53:26.893 INFO:teuthology.orchestra.run.smithi162.stdout: "default": { 2020-11-28T21:53:26.893 INFO:teuthology.orchestra.run.smithi162.stdout: "crush-failure-domain": "osd", 2020-11-28T21:53:26.894 INFO:teuthology.orchestra.run.smithi162.stdout: "k": "2", 2020-11-28T21:53:26.894 INFO:teuthology.orchestra.run.smithi162.stdout: "m": "1", 2020-11-28T21:53:26.894 INFO:teuthology.orchestra.run.smithi162.stdout: "plugin": "jerasure", 2020-11-28T21:53:26.894 INFO:teuthology.orchestra.run.smithi162.stdout: "ruleset-failure-domain": "osd", 2020-11-28T21:53:26.894 INFO:teuthology.orchestra.run.smithi162.stdout: "technique": "reed_sol_van" 2020-11-28T21:53:26.895 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:26.895 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:26.895 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps_queue": [], 2020-11-28T21:53:26.895 INFO:teuthology.orchestra.run.smithi162.stdout: "new_removed_snaps": [], 2020-11-28T21:53:26.895 INFO:teuthology.orchestra.run.smithi162.stdout: "new_purged_snaps": [], 2020-11-28T21:53:26.896 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_node_flags": {}, 2020-11-28T21:53:26.896 INFO:teuthology.orchestra.run.smithi162.stdout: "device_class_flags": {} 2020-11-28T21:53:26.896 INFO:teuthology.orchestra.run.smithi162.stdout:} 2020-11-28T21:53:26.897 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.685+0000 7f77ed34d700 1 -- 172.21.15.162:0/3769886603 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f77cc055330 msgr2=0x7f77cc0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:26.898 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.685+0000 7f77ed34d700 1 --2- 172.21.15.162:0/3769886603 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f77cc055330 0x7f77cc0577f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f77d000c000 tx=0x7f77d0008040).stop 2020-11-28T21:53:26.898 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.685+0000 7f77ed34d700 1 -- 172.21.15.162:0/3769886603 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f77e8060010 msgr2=0x7f77e8064cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:26.898 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.685+0000 7f77ed34d700 1 --2- 172.21.15.162:0/3769886603 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f77e8060010 0x7f77e8064cc0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 rx=0x7f77e00032b0 tx=0x7f77e0016000).stop 2020-11-28T21:53:26.898 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.685+0000 7f77ed34d700 1 -- 172.21.15.162:0/3769886603 shutdown_connections 2020-11-28T21:53:26.899 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.685+0000 7f77ed34d700 1 --2- 172.21.15.162:0/3769886603 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f77cc055330 0x7f77cc0577f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:26.899 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.685+0000 7f77ed34d700 1 --2- 172.21.15.162:0/3769886603 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f77e80b01a0 0x7f77e813d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:26.899 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.685+0000 7f77ed34d700 1 --2- 172.21.15.162:0/3769886603 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f77e8060010 0x7f77e8064cc0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:26.899 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.685+0000 7f77ed34d700 1 --2- 172.21.15.162:0/3769886603 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f77e809abb0 0x7f77e8066610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:26.899 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.685+0000 7f77ed34d700 1 -- 172.21.15.162:0/3769886603 >> 172.21.15.162:0/3769886603 conn(0x7f77e80b54f0 msgr2=0x7f77e8140550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:26.900 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.685+0000 7f77ed34d700 1 -- 172.21.15.162:0/3769886603 shutdown_connections 2020-11-28T21:53:26.900 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:26.685+0000 7f77ed34d700 1 -- 172.21.15.162:0/3769886603 wait complete. 2020-11-28T21:53:26.901 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:53:27.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.013+0000 7f820ffff700 1 Processor -- start 2020-11-28T21:53:27.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.013+0000 7f820ffff700 1 -- start start 2020-11-28T21:53:27.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.017+0000 7f820ffff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f82100a8530 0x7f821013db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:27.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.017+0000 7f820ffff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82100b2520 con 0x7f82100ab6e0 2020-11-28T21:53:27.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.017+0000 7f820ffff700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82100abac0 con 0x7f82101448e0 2020-11-28T21:53:27.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.017+0000 7f820ffff700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82100b4db0 con 0x7f82100a8530 2020-11-28T21:53:27.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.017+0000 7f82067fc700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f82101448e0 0x7f821013d480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:48650/0 (socket says 172.21.15.162:48650) 2020-11-28T21:53:27.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.017+0000 7f82067fc700 1 -- 172.21.15.162:0/4192190198 learned_addr learned my addr 172.21.15.162:0/4192190198 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:27.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.017+0000 7f820f7fe700 1 --2- 172.21.15.162:0/4192190198 >> v2:172.21.15.118:3300/0 conn(0x7f82100a8530 0x7f821013db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:27.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.017+0000 7f820e7fc700 1 -- 172.21.15.162:0/4192190198 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3052767068 0 0) 0x7f82100abac0 con 0x7f82101448e0 2020-11-28T21:53:27.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.017+0000 7f820e7fc700 1 -- 172.21.15.162:0/4192190198 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81f0003230 con 0x7f82101448e0 2020-11-28T21:53:27.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.017+0000 7f820e7fc700 1 -- 172.21.15.162:0/4192190198 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2247911761 0 0) 0x7f82100b2520 con 0x7f82100ab6e0 2020-11-28T21:53:27.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.017+0000 7f820e7fc700 1 -- 172.21.15.162:0/4192190198 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82100abac0 con 0x7f82100ab6e0 2020-11-28T21:53:27.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.017+0000 7f820e7fc700 1 -- 172.21.15.162:0/4192190198 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3537918597 0 0) 0x7f81f0003230 con 0x7f82101448e0 2020-11-28T21:53:27.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.017+0000 7f820f7fe700 1 -- 172.21.15.162:0/4192190198 >> v1:172.21.15.162:6789/0 conn(0x7f82101448e0 legacy=0x7f821013d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:27.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.017+0000 7f820f7fe700 1 -- 172.21.15.162:0/4192190198 >> v1:172.21.15.118:6789/0 conn(0x7f82100ab6e0 legacy=0x7f82101441d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:27.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.017+0000 7f820f7fe700 1 -- 172.21.15.162:0/4192190198 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82100b5050 con 0x7f82100a8530 2020-11-28T21:53:27.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.017+0000 7f820f7fe700 1 --2- 172.21.15.162:0/4192190198 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82100a8530 0x7f821013db90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f81f8001870 tx=0x7f82080091a0).ready entity=mon.0 client_cookie=84f1c944a7e48f36 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:27.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.017+0000 7f820e7fc700 1 -- 172.21.15.162:0/4192190198 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81f80016d0 con 0x7f82100a8530 2020-11-28T21:53:27.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.017+0000 7f820e7fc700 1 -- 172.21.15.162:0/4192190198 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f820800b920 con 0x7f82100a8530 2020-11-28T21:53:27.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.017+0000 7f820e7fc700 1 -- 172.21.15.162:0/4192190198 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f820800ba90 con 0x7f82100a8530 2020-11-28T21:53:27.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.017+0000 7f820ffff700 1 -- 172.21.15.162:0/4192190198 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82100a8530 msgr2=0x7f821013db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:27.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.021+0000 7f820ffff700 1 --2- 172.21.15.162:0/4192190198 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82100a8530 0x7f821013db90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f81f8001870 tx=0x7f82080091a0).stop 2020-11-28T21:53:27.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.021+0000 7f820ffff700 1 -- 172.21.15.162:0/4192190198 shutdown_connections 2020-11-28T21:53:27.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.021+0000 7f820ffff700 1 --2- 172.21.15.162:0/4192190198 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82100a8530 0x7f821013db90 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:27.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.021+0000 7f820ffff700 1 -- 172.21.15.162:0/4192190198 >> 172.21.15.162:0/4192190198 conn(0x7f821009a2f0 msgr2=0x7f821009a700 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:27.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.021+0000 7f820ffff700 1 -- 172.21.15.162:0/4192190198 shutdown_connections 2020-11-28T21:53:27.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.021+0000 7f820ffff700 1 -- 172.21.15.162:0/4192190198 wait complete. 2020-11-28T21:53:27.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.025+0000 7f820ffff700 1 Processor -- start 2020-11-28T21:53:27.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.025+0000 7f820ffff700 1 -- start start 2020-11-28T21:53:27.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.025+0000 7f820ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82100a8530 0x7f82100b41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:27.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.025+0000 7f820ffff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f82100ab6e0 0x7f8210099240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:27.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.025+0000 7f820ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82101448e0 0x7f8210099730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:27.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.025+0000 7f820ffff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8210064fa0 con 0x7f82101448e0 2020-11-28T21:53:27.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.025+0000 7f820ffff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821008f910 con 0x7f82100a8530 2020-11-28T21:53:27.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.025+0000 7f820ffff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82101507b0 con 0x7f82100ab6e0 2020-11-28T21:53:27.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.025+0000 7f820effd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82100a8530 0x7f82100b41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:27.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.025+0000 7f820effd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82100a8530 0x7f82100b41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41810/0 (socket says 172.21.15.162:41810) 2020-11-28T21:53:27.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.025+0000 7f820effd700 1 -- 172.21.15.162:0/3509836720 learned_addr learned my addr 172.21.15.162:0/3509836720 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:27.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.025+0000 7f820f7fe700 1 --2- 172.21.15.162:0/3509836720 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82101448e0 0x7f8210099730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:27.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.025+0000 7f82067fc700 1 --2- 172.21.15.162:0/3509836720 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f82100ab6e0 0x7f8210099240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:27.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.025+0000 7f820effd700 1 -- 172.21.15.162:0/3509836720 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f82100ab6e0 msgr2=0x7f8210099240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:27.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.025+0000 7f820effd700 1 --2- 172.21.15.162:0/3509836720 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f82100ab6e0 0x7f8210099240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:27.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.025+0000 7f820effd700 1 -- 172.21.15.162:0/3509836720 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82101448e0 msgr2=0x7f8210099730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:27.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.025+0000 7f820effd700 1 --2- 172.21.15.162:0/3509836720 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82101448e0 0x7f8210099730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:27.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.025+0000 7f820effd700 1 -- 172.21.15.162:0/3509836720 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82100651d0 con 0x7f82100a8530 2020-11-28T21:53:27.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.025+0000 7f820f7fe700 1 --2- 172.21.15.162:0/3509836720 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82101448e0 0x7f8210099730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:53:27.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.025+0000 7f820effd700 1 --2- 172.21.15.162:0/3509836720 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82100a8530 0x7f82100b41b0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 rx=0x7f82000010d0 tx=0x7f8200007ab0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:27.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.025+0000 7f820cff9700 1 -- 172.21.15.162:0/3509836720 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8200010510 con 0x7f82100a8530 2020-11-28T21:53:27.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.025+0000 7f820cff9700 1 -- 172.21.15.162:0/3509836720 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8200010670 con 0x7f82100a8530 2020-11-28T21:53:27.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.025+0000 7f820ffff700 1 -- 172.21.15.162:0/3509836720 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8210065470 con 0x7f82100a8530 2020-11-28T21:53:27.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.025+0000 7f820cff9700 1 -- 172.21.15.162:0/3509836720 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82000107e0 con 0x7f82100a8530 2020-11-28T21:53:27.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.025+0000 7f820ffff700 1 -- 172.21.15.162:0/3509836720 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8210065910 con 0x7f82100a8530 2020-11-28T21:53:27.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.029+0000 7f820ffff700 1 -- 172.21.15.162:0/3509836720 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8210141680 con 0x7f82100a8530 2020-11-28T21:53:27.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.029+0000 7f820cff9700 1 -- 172.21.15.162:0/3509836720 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8200010070 con 0x7f82100a8530 2020-11-28T21:53:27.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.029+0000 7f820cff9700 1 --2- 172.21.15.162:0/3509836720 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f81f4055440 0x7f81f4057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:27.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.029+0000 7f820cff9700 1 -- 172.21.15.162:0/3509836720 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f820006d9a0 con 0x7f82100a8530 2020-11-28T21:53:27.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.029+0000 7f82067fc700 1 --2- 172.21.15.162:0/3509836720 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f81f4055440 0x7f81f4057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:27.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.029+0000 7f82067fc700 1 --2- 172.21.15.162:0/3509836720 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f81f4055440 0x7f81f4057900 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f81f800b000 tx=0x7f81f8009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:27.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.033+0000 7f820cff9700 1 -- 172.21.15.162:0/3509836720 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8200043b60 con 0x7f82100a8530 2020-11-28T21:53:27.224 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.221+0000 7f820ffff700 1 -- 172.21.15.162:0/3509836720 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8210065600 con 0x7f82100a8530 2020-11-28T21:53:27.225 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.221+0000 7f820cff9700 1 -- 172.21.15.162:0/3509836720 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+2499 (secure 0 0 0) 0x7f8200047180 con 0x7f82100a8530 2020-11-28T21:53:27.225 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:53:27.225 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":33,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5540,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2116325247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2116325247},{"type":"v1","addr":"172.21.15.162:6835","nonce":2116325247}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":30},{"gid":5543,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.118:6833/3491069258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3491069258},{"type":"v1","addr":"172.21.15.118:6833","nonce":3491069258}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:52:28.454745+0000","modified":"2020-11-28T21:52:29.457610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6172},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6172":{"gid":6172,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/163256983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":163256983},{"type":"v1","addr":"172.21.15.162:6837","nonce":163256983}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-28T21:53:27.226 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.225+0000 7f820ffff700 1 -- 172.21.15.162:0/3509836720 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f81f4055440 msgr2=0x7f81f4057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:27.227 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.225+0000 7f820ffff700 1 --2- 172.21.15.162:0/3509836720 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f81f4055440 0x7f81f4057900 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f81f800b000 tx=0x7f81f8009040).stop 2020-11-28T21:53:27.227 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.225+0000 7f820ffff700 1 -- 172.21.15.162:0/3509836720 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82100a8530 msgr2=0x7f82100b41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:27.227 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.225+0000 7f820ffff700 1 --2- 172.21.15.162:0/3509836720 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82100a8530 0x7f82100b41b0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 rx=0x7f82000010d0 tx=0x7f8200007ab0).stop 2020-11-28T21:53:27.227 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.225+0000 7f820ffff700 1 -- 172.21.15.162:0/3509836720 shutdown_connections 2020-11-28T21:53:27.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.225+0000 7f820ffff700 1 --2- 172.21.15.162:0/3509836720 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f81f4055440 0x7f81f4057900 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:27.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.225+0000 7f820ffff700 1 --2- 172.21.15.162:0/3509836720 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82101448e0 0x7f8210099730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:27.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.225+0000 7f820ffff700 1 --2- 172.21.15.162:0/3509836720 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82100a8530 0x7f82100b41b0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:27.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.225+0000 7f820ffff700 1 --2- 172.21.15.162:0/3509836720 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f82100ab6e0 0x7f8210099240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:27.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.225+0000 7f820ffff700 1 -- 172.21.15.162:0/3509836720 >> 172.21.15.162:0/3509836720 conn(0x7f821009a2f0 msgr2=0x7f82101429e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:27.229 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.225+0000 7f820ffff700 1 -- 172.21.15.162:0/3509836720 shutdown_connections 2020-11-28T21:53:27.229 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.225+0000 7f820ffff700 1 -- 172.21.15.162:0/3509836720 wait complete. 2020-11-28T21:53:27.229 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 33 2020-11-28T21:53:27.238 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2020-11-28T21:53:27.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.349+0000 7f58d3fff700 1 Processor -- start 2020-11-28T21:53:27.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.349+0000 7f58d3fff700 1 -- start start 2020-11-28T21:53:27.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.349+0000 7f58d3fff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f58d4062cd0 0x7f58d4142980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:27.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.349+0000 7f58d3fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f58d40610b0 0x7f58d4142e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:27.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.349+0000 7f58d3fff700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58d40b2fd0 con 0x7f58d40b39a0 2020-11-28T21:53:27.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.349+0000 7f58d3fff700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58d40b3270 con 0x7f58d4062cd0 2020-11-28T21:53:27.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.349+0000 7f58d3fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58d4061bb0 con 0x7f58d40610b0 2020-11-28T21:53:27.355 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.349+0000 7f58d2ffd700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f58d40b39a0 0x7f58d4062050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:48662/0 (socket says 172.21.15.162:48662) 2020-11-28T21:53:27.355 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.349+0000 7f58d2ffd700 1 -- 172.21.15.162:0/3609966443 learned_addr learned my addr 172.21.15.162:0/3609966443 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:27.355 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.349+0000 7f58d27fc700 1 --2- 172.21.15.162:0/3609966443 >> v2:172.21.15.118:3300/0 conn(0x7f58d4062cd0 0x7f58d4142980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:27.355 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.349+0000 7f58d37fe700 1 --2- 172.21.15.162:0/3609966443 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f58d40610b0 0x7f58d4142e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:27.356 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.349+0000 7f58d1ffb700 1 -- 172.21.15.162:0/3609966443 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2965523140 0 0) 0x7f58d40b2fd0 con 0x7f58d40b39a0 2020-11-28T21:53:27.356 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.349+0000 7f58d1ffb700 1 -- 172.21.15.162:0/3609966443 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58b8003230 con 0x7f58d40b39a0 2020-11-28T21:53:27.356 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.349+0000 7f58d37fe700 1 -- 172.21.15.162:0/3609966443 >> v2:172.21.15.118:3300/0 conn(0x7f58d4062cd0 msgr2=0x7f58d4142980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:27.356 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.349+0000 7f58d37fe700 1 --2- 172.21.15.162:0/3609966443 >> v2:172.21.15.118:3300/0 conn(0x7f58d4062cd0 0x7f58d4142980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:27.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.349+0000 7f58d37fe700 1 -- 172.21.15.162:0/3609966443 >> v1:172.21.15.162:6789/0 conn(0x7f58d40b39a0 legacy=0x7f58d4062050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:27.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.349+0000 7f58d37fe700 1 -- 172.21.15.162:0/3609966443 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d4060e40 con 0x7f58d40610b0 2020-11-28T21:53:27.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.349+0000 7f58d37fe700 1 --2- 172.21.15.162:0/3609966443 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f58d40610b0 0x7f58d4142e70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f58cc0032b0 tx=0x7f58cc0091a0).ready entity=mon.2 client_cookie=d47d3cc682508f54 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:27.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.349+0000 7f58d1ffb700 1 -- 172.21.15.162:0/3609966443 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58cc00b940 con 0x7f58d40610b0 2020-11-28T21:53:27.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.349+0000 7f58d1ffb700 1 -- 172.21.15.162:0/3609966443 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58cc00baa0 con 0x7f58d40610b0 2020-11-28T21:53:27.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.349+0000 7f58d3fff700 1 -- 172.21.15.162:0/3609966443 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f58d40610b0 msgr2=0x7f58d4142e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:27.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.349+0000 7f58d1ffb700 1 -- 172.21.15.162:0/3609966443 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58cc00bc10 con 0x7f58d40610b0 2020-11-28T21:53:27.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.349+0000 7f58d3fff700 1 --2- 172.21.15.162:0/3609966443 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f58d40610b0 0x7f58d4142e70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f58cc0032b0 tx=0x7f58cc0091a0).stop 2020-11-28T21:53:27.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.353+0000 7f58d3fff700 1 -- 172.21.15.162:0/3609966443 shutdown_connections 2020-11-28T21:53:27.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.353+0000 7f58d3fff700 1 --2- 172.21.15.162:0/3609966443 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f58d40610b0 0x7f58d4142e70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:27.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.353+0000 7f58d3fff700 1 --2- 172.21.15.162:0/3609966443 >> v2:172.21.15.118:3300/0 conn(0x7f58d4062cd0 0x7f58d4142980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:27.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.353+0000 7f58d3fff700 1 -- 172.21.15.162:0/3609966443 >> 172.21.15.162:0/3609966443 conn(0x7f58d405b970 msgr2=0x7f58d405bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:27.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.353+0000 7f58d3fff700 1 -- 172.21.15.162:0/3609966443 shutdown_connections 2020-11-28T21:53:27.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.353+0000 7f58d3fff700 1 -- 172.21.15.162:0/3609966443 wait complete. 2020-11-28T21:53:27.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.353+0000 7f58d3fff700 1 Processor -- start 2020-11-28T21:53:27.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.353+0000 7f58d3fff700 1 -- start start 2020-11-28T21:53:27.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.353+0000 7f58d3fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58d40610b0 0x7f58d41424d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:27.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.353+0000 7f58d3fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f58d4062cd0 0x7f58d4141ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:27.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.353+0000 7f58d3fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f58d40b39a0 0x7f58d413acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:27.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.353+0000 7f58d3fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d4142f10 con 0x7f58d40b39a0 2020-11-28T21:53:27.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.353+0000 7f58d3fff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d405f900 con 0x7f58d40610b0 2020-11-28T21:53:27.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.353+0000 7f58d3fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58d4141600 con 0x7f58d4062cd0 2020-11-28T21:53:27.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.353+0000 7f58d2ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58d40610b0 0x7f58d41424d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:27.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.353+0000 7f58d2ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58d40610b0 0x7f58d41424d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41824/0 (socket says 172.21.15.162:41824) 2020-11-28T21:53:27.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.353+0000 7f58d2ffd700 1 -- 172.21.15.162:0/1594809667 learned_addr learned my addr 172.21.15.162:0/1594809667 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:27.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.357+0000 7f58d37fe700 1 --2- 172.21.15.162:0/1594809667 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f58d40b39a0 0x7f58d413acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:27.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.357+0000 7f58d27fc700 1 --2- 172.21.15.162:0/1594809667 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f58d4062cd0 0x7f58d4141ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:27.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.357+0000 7f58d2ffd700 1 -- 172.21.15.162:0/1594809667 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f58d4062cd0 msgr2=0x7f58d4141ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:27.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.357+0000 7f58d2ffd700 1 --2- 172.21.15.162:0/1594809667 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f58d4062cd0 0x7f58d4141ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:27.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.357+0000 7f58d2ffd700 1 -- 172.21.15.162:0/1594809667 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f58d40b39a0 msgr2=0x7f58d413acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:27.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.357+0000 7f58d2ffd700 1 --2- 172.21.15.162:0/1594809667 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f58d40b39a0 0x7f58d413acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:27.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.357+0000 7f58d2ffd700 1 -- 172.21.15.162:0/1594809667 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d41408c0 con 0x7f58d40610b0 2020-11-28T21:53:27.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.357+0000 7f58d27fc700 1 --2- 172.21.15.162:0/1594809667 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f58d4062cd0 0x7f58d4141ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:53:27.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.357+0000 7f58d2ffd700 1 --2- 172.21.15.162:0/1594809667 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58d40610b0 0x7f58d41424d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 rx=0x7f58c40010d0 tx=0x7f58c4007aa0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:27.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.357+0000 7f58c3fff700 1 -- 172.21.15.162:0/1594809667 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58c40072e0 con 0x7f58d40610b0 2020-11-28T21:53:27.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.357+0000 7f58d3fff700 1 -- 172.21.15.162:0/1594809667 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58d4140b60 con 0x7f58d40610b0 2020-11-28T21:53:27.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.357+0000 7f58c3fff700 1 -- 172.21.15.162:0/1594809667 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c4007440 con 0x7f58d40610b0 2020-11-28T21:53:27.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.357+0000 7f58d3fff700 1 -- 172.21.15.162:0/1594809667 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58d40aed70 con 0x7f58d40610b0 2020-11-28T21:53:27.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.357+0000 7f58c3fff700 1 -- 172.21.15.162:0/1594809667 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58c40057d0 con 0x7f58d40610b0 2020-11-28T21:53:27.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.357+0000 7f58d3fff700 1 -- 172.21.15.162:0/1594809667 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58d4057050 con 0x7f58d40610b0 2020-11-28T21:53:27.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.357+0000 7f58c3fff700 1 -- 172.21.15.162:0/1594809667 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f58c4010070 con 0x7f58d40610b0 2020-11-28T21:53:27.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.357+0000 7f58c3fff700 1 --2- 172.21.15.162:0/1594809667 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f58b00554e0 0x7f58b00579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:27.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.357+0000 7f58c3fff700 1 -- 172.21.15.162:0/1594809667 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f58c406da20 con 0x7f58d40610b0 2020-11-28T21:53:27.371 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.357+0000 7f58d27fc700 1 --2- 172.21.15.162:0/1594809667 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f58b00554e0 0x7f58b00579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:27.371 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.357+0000 7f58d27fc700 1 --2- 172.21.15.162:0/1594809667 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f58b00554e0 0x7f58b00579a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f58c8008000 tx=0x7f58c8008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:27.371 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.365+0000 7f58c3fff700 1 -- 172.21.15.162:0/1594809667 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f58c4043cf0 con 0x7f58d40610b0 2020-11-28T21:53:27.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.549+0000 7f58d3fff700 1 -- 172.21.15.162:0/1594809667 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f58d4068350 con 0x7f58d40610b0 2020-11-28T21:53:27.947 INFO:tasks.ceph.mon.b.smithi118.stderr:2020-11-28T21:53:27.942+0000 7ff4e1520700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-11-28T21:53:27.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.957+0000 7f58c3fff700 1 -- 172.21.15.162:0/1594809667 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v34) v1 ==== 161+0+0 (secure 0 0 0) 0x7f58c4047310 con 0x7f58d40610b0 2020-11-28T21:53:27.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.961+0000 7f58d3fff700 1 -- 172.21.15.162:0/1594809667 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f58b00554e0 msgr2=0x7f58b00579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:27.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.961+0000 7f58d3fff700 1 --2- 172.21.15.162:0/1594809667 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f58b00554e0 0x7f58b00579a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f58c8008000 tx=0x7f58c8008040).stop 2020-11-28T21:53:27.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.961+0000 7f58d3fff700 1 -- 172.21.15.162:0/1594809667 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58d40610b0 msgr2=0x7f58d41424d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:27.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.961+0000 7f58d3fff700 1 --2- 172.21.15.162:0/1594809667 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58d40610b0 0x7f58d41424d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 rx=0x7f58c40010d0 tx=0x7f58c4007aa0).stop 2020-11-28T21:53:27.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.961+0000 7f58d3fff700 1 -- 172.21.15.162:0/1594809667 shutdown_connections 2020-11-28T21:53:27.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.961+0000 7f58d3fff700 1 --2- 172.21.15.162:0/1594809667 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f58b00554e0 0x7f58b00579a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:27.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.961+0000 7f58d3fff700 1 --2- 172.21.15.162:0/1594809667 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f58d40b39a0 0x7f58d413acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:27.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.961+0000 7f58d3fff700 1 --2- 172.21.15.162:0/1594809667 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58d40610b0 0x7f58d41424d0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:27.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.961+0000 7f58d3fff700 1 --2- 172.21.15.162:0/1594809667 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f58d4062cd0 0x7f58d4141ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:27.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.961+0000 7f58d3fff700 1 -- 172.21.15.162:0/1594809667 >> 172.21.15.162:0/1594809667 conn(0x7f58d405b970 msgr2=0x7f58d40656b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:27.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.961+0000 7f58d3fff700 1 -- 172.21.15.162:0/1594809667 shutdown_connections 2020-11-28T21:53:27.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:27.961+0000 7f58d3fff700 1 -- 172.21.15.162:0/1594809667 wait complete. 2020-11-28T21:53:27.968 INFO:teuthology.orchestra.run.smithi162.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-11-28T21:53:27.986 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:53:27.992 INFO:tasks.ceph.mds.a.smithi162.stdout:starting mds.a at 2020-11-28T21:53:28.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.093+0000 7f43b59d1700 1 Processor -- start 2020-11-28T21:53:28.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.093+0000 7f43b59d1700 1 -- start start 2020-11-28T21:53:28.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.097+0000 7f43b59d1700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f43b0062ac0 0x7f43b00b33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:28.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.097+0000 7f43b59d1700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f43b0066ff0 0x7f43b0061590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:28.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.097+0000 7f43b59d1700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43b0143c90 con 0x7f43b00b7780 2020-11-28T21:53:28.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.097+0000 7f43b59d1700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43b0144b40 con 0x7f43b0066ff0 2020-11-28T21:53:28.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.097+0000 7f43b59d1700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b0143ec0 con 0x7f43b0062ac0 2020-11-28T21:53:28.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.097+0000 7f43affff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f43b0062ac0 0x7f43b00b33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:28.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.097+0000 7f43b51d0700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f43b0066ff0 0x7f43b0061590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:28.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.097+0000 7f43affff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f43b0062ac0 0x7f43b00b33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:39070/0 (socket says 172.21.15.162:39070) 2020-11-28T21:53:28.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.097+0000 7f43b51d0700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f43b0066ff0 0x7f43b0061590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.162:48836/0 (socket says 172.21.15.162:48836) 2020-11-28T21:53:28.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.097+0000 7f43affff700 1 -- 172.21.15.162:0/2340449798 learned_addr learned my addr 172.21.15.162:0/2340449798 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:28.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.097+0000 7f43b49cf700 1 --1- 172.21.15.162:0/2340449798 >> v1:172.21.15.118:6789/0 conn(0x7f43b00b7780 0x7f43b00b2cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.162:38836/0 (socket says 172.21.15.162:38836) 2020-11-28T21:53:28.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.097+0000 7f43b51d0700 1 -- 172.21.15.162:0/2340449798 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f43b0062ac0 msgr2=0x7f43b00b33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:28.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.097+0000 7f43b51d0700 1 --2- 172.21.15.162:0/2340449798 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f43b0062ac0 0x7f43b00b33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:28.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.097+0000 7f43b51d0700 1 -- 172.21.15.162:0/2340449798 >> v1:172.21.15.118:6789/0 conn(0x7f43b00b7780 legacy=0x7f43b00b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:28.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.097+0000 7f43b51d0700 1 -- 172.21.15.162:0/2340449798 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b0144160 con 0x7f43b0066ff0 2020-11-28T21:53:28.104 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.097+0000 7f43af7fe700 1 -- 172.21.15.162:0/2340449798 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3573196353 0 0) 0x7f43b0143c90 con 0x7f43b00b7780 2020-11-28T21:53:28.104 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.097+0000 7f43b51d0700 1 --2- 172.21.15.162:0/2340449798 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f43b0066ff0 0x7f43b0061590 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7f43a8002f30 tx=0x7f43a8006dc0).ready entity=mon.0 client_cookie=f8f980b024fd560b server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:28.104 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.097+0000 7f43af7fe700 1 -- 172.21.15.162:0/2340449798 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43a8009760 con 0x7f43b0066ff0 2020-11-28T21:53:28.104 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.097+0000 7f43af7fe700 1 -- 172.21.15.162:0/2340449798 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a80098c0 con 0x7f43b0066ff0 2020-11-28T21:53:28.104 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.097+0000 7f43af7fe700 1 -- 172.21.15.162:0/2340449798 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43a8009c20 con 0x7f43b0066ff0 2020-11-28T21:53:28.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.097+0000 7f43b59d1700 1 -- 172.21.15.162:0/2340449798 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f43b0066ff0 msgr2=0x7f43b0061590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:28.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.097+0000 7f43b59d1700 1 --2- 172.21.15.162:0/2340449798 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f43b0066ff0 0x7f43b0061590 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7f43a8002f30 tx=0x7f43a8006dc0).stop 2020-11-28T21:53:28.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.097+0000 7f43b59d1700 1 -- 172.21.15.162:0/2340449798 shutdown_connections 2020-11-28T21:53:28.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.097+0000 7f43b59d1700 1 --2- 172.21.15.162:0/2340449798 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f43b0066ff0 0x7f43b0061590 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:28.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.097+0000 7f43b59d1700 1 --2- 172.21.15.162:0/2340449798 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f43b0062ac0 0x7f43b00b33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:28.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.097+0000 7f43b59d1700 1 -- 172.21.15.162:0/2340449798 >> 172.21.15.162:0/2340449798 conn(0x7f43b005c440 msgr2=0x7f43b00b6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:28.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.097+0000 7f43b59d1700 1 -- 172.21.15.162:0/2340449798 shutdown_connections 2020-11-28T21:53:28.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.097+0000 7f43b59d1700 1 -- 172.21.15.162:0/2340449798 wait complete. 2020-11-28T21:53:28.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.101+0000 7f43b59d1700 1 Processor -- start 2020-11-28T21:53:28.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.101+0000 7f43b59d1700 1 -- start start 2020-11-28T21:53:28.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.101+0000 7f43b59d1700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f43b0062ac0 0x7f43b0140f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:28.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.101+0000 7f43b59d1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43b0066ff0 0x7f43b0092d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:28.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.101+0000 7f43b59d1700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f43b00b7780 0x7f43b014f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:28.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.101+0000 7f43b59d1700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b0153670 con 0x7f43b00b7780 2020-11-28T21:53:28.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.101+0000 7f43b59d1700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b00ad7f0 con 0x7f43b0066ff0 2020-11-28T21:53:28.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.101+0000 7f43b59d1700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b00ada90 con 0x7f43b0062ac0 2020-11-28T21:53:28.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.105+0000 7f43affff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43b0066ff0 0x7f43b0092d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:28.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.105+0000 7f43affff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43b0066ff0 0x7f43b0092d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41852/0 (socket says 172.21.15.162:41852) 2020-11-28T21:53:28.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.105+0000 7f43affff700 1 -- 172.21.15.162:0/1356837226 learned_addr learned my addr 172.21.15.162:0/1356837226 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:28.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.105+0000 7f43b49cf700 1 --2- 172.21.15.162:0/1356837226 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f43b0062ac0 0x7f43b0140f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:28.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.105+0000 7f43b51d0700 1 --2- 172.21.15.162:0/1356837226 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f43b00b7780 0x7f43b014f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:28.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.105+0000 7f43affff700 1 -- 172.21.15.162:0/1356837226 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f43b0062ac0 msgr2=0x7f43b0140f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:28.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.105+0000 7f43affff700 1 --2- 172.21.15.162:0/1356837226 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f43b0062ac0 0x7f43b0140f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:28.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.105+0000 7f43affff700 1 -- 172.21.15.162:0/1356837226 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f43b00b7780 msgr2=0x7f43b014f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:28.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.105+0000 7f43affff700 1 --2- 172.21.15.162:0/1356837226 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f43b00b7780 0x7f43b014f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:28.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.105+0000 7f43affff700 1 -- 172.21.15.162:0/1356837226 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b01532b0 con 0x7f43b0066ff0 2020-11-28T21:53:28.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.105+0000 7f43affff700 1 --2- 172.21.15.162:0/1356837226 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43b0066ff0 0x7f43b0092d60 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 rx=0x7f43a400c000 tx=0x7f43a4008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:28.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.105+0000 7f43adffb700 1 -- 172.21.15.162:0/1356837226 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43a400ee80 con 0x7f43b0066ff0 2020-11-28T21:53:28.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.105+0000 7f43adffb700 1 -- 172.21.15.162:0/1356837226 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a400c820 con 0x7f43b0066ff0 2020-11-28T21:53:28.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.105+0000 7f43b59d1700 1 -- 172.21.15.162:0/1356837226 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43b0093370 con 0x7f43b0066ff0 2020-11-28T21:53:28.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.105+0000 7f43adffb700 1 -- 172.21.15.162:0/1356837226 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43a400cb80 con 0x7f43b0066ff0 2020-11-28T21:53:28.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.105+0000 7f43b59d1700 1 -- 172.21.15.162:0/1356837226 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b014fa60 con 0x7f43b0066ff0 2020-11-28T21:53:28.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.105+0000 7f43adffb700 1 -- 172.21.15.162:0/1356837226 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f43a4022070 con 0x7f43b0066ff0 2020-11-28T21:53:28.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.105+0000 7f43b59d1700 1 -- 172.21.15.162:0/1356837226 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43b00adc50 con 0x7f43b0066ff0 2020-11-28T21:53:28.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.105+0000 7f43adffb700 1 --2- 172.21.15.162:0/1356837226 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4390055470 0x7f4390057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:28.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.105+0000 7f43adffb700 1 -- 172.21.15.162:0/1356837226 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6202+0+0 (secure 0 0 0) 0x7f43a406e230 con 0x7f43b0066ff0 2020-11-28T21:53:28.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.105+0000 7f43b49cf700 1 --2- 172.21.15.162:0/1356837226 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4390055470 0x7f4390057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:28.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.105+0000 7f43b49cf700 1 --2- 172.21.15.162:0/1356837226 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4390055470 0x7f4390057930 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7f43a000d000 tx=0x7f43a0007ef0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:28.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.109+0000 7f43adffb700 1 -- 172.21.15.162:0/1356837226 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f43a40444b0 con 0x7f43b0066ff0 2020-11-28T21:53:28.296 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.293+0000 7f43b59d1700 1 -- 172.21.15.162:0/1356837226 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43b0093500 con 0x7f43b0066ff0 2020-11-28T21:53:28.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.293+0000 7f43adffb700 1 -- 172.21.15.162:0/1356837226 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+2138 (secure 0 0 0) 0x7f43b0093500 con 0x7f43b0066ff0 2020-11-28T21:53:28.297 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:53:28.298 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":34,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5540,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2116325247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2116325247},{"type":"v1","addr":"172.21.15.162:6835","nonce":2116325247}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":30},{"gid":5543,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.118:6833/3491069258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3491069258},{"type":"v1","addr":"172.21.15.118:6833","nonce":3491069258}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":31}],"filesystems":[{"mdsmap":{"epoch":34,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:52:28.454745+0000","modified":"2020-11-28T21:53:27.946945+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-28T21:53:28.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.297+0000 7f43b59d1700 1 -- 172.21.15.162:0/1356837226 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4390055470 msgr2=0x7f4390057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:28.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.297+0000 7f43b59d1700 1 --2- 172.21.15.162:0/1356837226 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4390055470 0x7f4390057930 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7f43a000d000 tx=0x7f43a0007ef0).stop 2020-11-28T21:53:28.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.297+0000 7f43b59d1700 1 -- 172.21.15.162:0/1356837226 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43b0066ff0 msgr2=0x7f43b0092d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:28.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.297+0000 7f43b59d1700 1 --2- 172.21.15.162:0/1356837226 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43b0066ff0 0x7f43b0092d60 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 rx=0x7f43a400c000 tx=0x7f43a4008040).stop 2020-11-28T21:53:28.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.297+0000 7f43b59d1700 1 -- 172.21.15.162:0/1356837226 shutdown_connections 2020-11-28T21:53:28.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.297+0000 7f43b59d1700 1 --2- 172.21.15.162:0/1356837226 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4390055470 0x7f4390057930 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:28.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.297+0000 7f43b59d1700 1 --2- 172.21.15.162:0/1356837226 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f43b0062ac0 0x7f43b0140f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:28.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.297+0000 7f43b59d1700 1 --2- 172.21.15.162:0/1356837226 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f43b00b7780 0x7f43b014f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:28.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.297+0000 7f43b59d1700 1 --2- 172.21.15.162:0/1356837226 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43b0066ff0 0x7f43b0092d60 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:28.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.297+0000 7f43b59d1700 1 -- 172.21.15.162:0/1356837226 >> 172.21.15.162:0/1356837226 conn(0x7f43b005c440 msgr2=0x7f43b00b5d40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:28.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.297+0000 7f43b59d1700 1 -- 172.21.15.162:0/1356837226 shutdown_connections 2020-11-28T21:53:28.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.297+0000 7f43b59d1700 1 -- 172.21.15.162:0/1356837226 wait complete. 2020-11-28T21:53:28.302 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 34 2020-11-28T21:53:28.310 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2020-11-28T21:53:28.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.473+0000 7f4334d28700 1 Processor -- start 2020-11-28T21:53:28.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.473+0000 7f4334d28700 1 -- start start 2020-11-28T21:53:28.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.473+0000 7f4334d28700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f43300ae710 0x7f4330060070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:28.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.473+0000 7f4334d28700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f43300aeef0 0x7f4330140870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:28.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.473+0000 7f4334d28700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43300b0330 con 0x7f4330144620 2020-11-28T21:53:28.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.473+0000 7f4334d28700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43300aeaf0 con 0x7f43300aeef0 2020-11-28T21:53:28.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.473+0000 7f4334d28700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4330060c10 con 0x7f43300ae710 2020-11-28T21:53:28.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.473+0000 7f432ffff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f4330144620 0x7f433013db40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:48706/0 (socket says 172.21.15.162:48706) 2020-11-28T21:53:28.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.477+0000 7f432ffff700 1 -- 172.21.15.162:0/1181458831 learned_addr learned my addr 172.21.15.162:0/1181458831 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:28.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.477+0000 7f432effd700 1 --2- 172.21.15.162:0/1181458831 >> v2:172.21.15.118:3300/0 conn(0x7f43300aeef0 0x7f4330140870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:28.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.477+0000 7f432f7fe700 1 --2- 172.21.15.162:0/1181458831 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f43300ae710 0x7f4330060070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:28.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.477+0000 7f432e7fc700 1 -- 172.21.15.162:0/1181458831 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3653691482 0 0) 0x7f43300b0330 con 0x7f4330144620 2020-11-28T21:53:28.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.477+0000 7f432e7fc700 1 -- 172.21.15.162:0/1181458831 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4310003230 con 0x7f4330144620 2020-11-28T21:53:28.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.477+0000 7f432f7fe700 1 -- 172.21.15.162:0/1181458831 >> v2:172.21.15.118:3300/0 conn(0x7f43300aeef0 msgr2=0x7f4330140870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:28.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.477+0000 7f432f7fe700 1 --2- 172.21.15.162:0/1181458831 >> v2:172.21.15.118:3300/0 conn(0x7f43300aeef0 0x7f4330140870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:28.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.477+0000 7f432f7fe700 1 -- 172.21.15.162:0/1181458831 >> v1:172.21.15.162:6789/0 conn(0x7f4330144620 legacy=0x7f433013db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:28.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.477+0000 7f432e7fc700 1 -- 172.21.15.162:0/1181458831 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1767266241 0 0) 0x7f4310003230 con 0x7f4330144620 2020-11-28T21:53:28.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.477+0000 7f432f7fe700 1 -- 172.21.15.162:0/1181458831 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4330060eb0 con 0x7f43300ae710 2020-11-28T21:53:28.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.477+0000 7f432effd700 1 --2- 172.21.15.162:0/1181458831 >> v2:172.21.15.118:3300/0 conn(0x7f43300aeef0 0x7f4330140870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:53:28.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.477+0000 7f432f7fe700 1 --2- 172.21.15.162:0/1181458831 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f43300ae710 0x7f4330060070 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f4318002f30 tx=0x7f4318006dc0).ready entity=mon.2 client_cookie=6ce5d7c417db629a server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:28.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.477+0000 7f432e7fc700 1 -- 172.21.15.162:0/1181458831 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43180096a0 con 0x7f43300ae710 2020-11-28T21:53:28.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.477+0000 7f432e7fc700 1 -- 172.21.15.162:0/1181458831 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4318009800 con 0x7f43300ae710 2020-11-28T21:53:28.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.477+0000 7f432e7fc700 1 -- 172.21.15.162:0/1181458831 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4318009b60 con 0x7f43300ae710 2020-11-28T21:53:28.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.477+0000 7f4334d28700 1 -- 172.21.15.162:0/1181458831 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f43300ae710 msgr2=0x7f4330060070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:28.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.477+0000 7f4334d28700 1 --2- 172.21.15.162:0/1181458831 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f43300ae710 0x7f4330060070 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f4318002f30 tx=0x7f4318006dc0).stop 2020-11-28T21:53:28.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.477+0000 7f4334d28700 1 -- 172.21.15.162:0/1181458831 shutdown_connections 2020-11-28T21:53:28.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.477+0000 7f4334d28700 1 --2- 172.21.15.162:0/1181458831 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f43300ae710 0x7f4330060070 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:28.485 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.477+0000 7f4334d28700 1 --2- 172.21.15.162:0/1181458831 >> v2:172.21.15.118:3300/0 conn(0x7f43300aeef0 0x7f4330140870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:28.485 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.477+0000 7f4334d28700 1 -- 172.21.15.162:0/1181458831 >> 172.21.15.162:0/1181458831 conn(0x7f43300b17a0 msgr2=0x7f43300b1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:28.485 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.477+0000 7f4334d28700 1 -- 172.21.15.162:0/1181458831 shutdown_connections 2020-11-28T21:53:28.485 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.477+0000 7f4334d28700 1 -- 172.21.15.162:0/1181458831 wait complete. 2020-11-28T21:53:28.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.481+0000 7f4334d28700 1 Processor -- start 2020-11-28T21:53:28.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.481+0000 7f4334d28700 1 -- start start 2020-11-28T21:53:28.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.481+0000 7f4334d28700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43300ae710 0x7f43300b4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:28.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.481+0000 7f4334d28700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f43300aeef0 0x7f4330140d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:28.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.481+0000 7f4334d28700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4330144620 0x7f43301532c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:28.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.481+0000 7f4334d28700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433014f970 con 0x7f4330144620 2020-11-28T21:53:28.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.481+0000 7f4334d28700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4330064010 con 0x7f43300ae710 2020-11-28T21:53:28.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.481+0000 7f4334d28700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43300642b0 con 0x7f43300aeef0 2020-11-28T21:53:28.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.481+0000 7f432f7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43300ae710 0x7f43300b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:28.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.481+0000 7f432f7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43300ae710 0x7f43300b4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41864/0 (socket says 172.21.15.162:41864) 2020-11-28T21:53:28.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.481+0000 7f432f7fe700 1 -- 172.21.15.162:0/2366245856 learned_addr learned my addr 172.21.15.162:0/2366245856 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:28.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.481+0000 7f432effd700 1 --2- 172.21.15.162:0/2366245856 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f43300aeef0 0x7f4330140d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:28.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.481+0000 7f432ffff700 1 --2- 172.21.15.162:0/2366245856 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4330144620 0x7f43301532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:28.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.481+0000 7f432f7fe700 1 -- 172.21.15.162:0/2366245856 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f43300aeef0 msgr2=0x7f4330140d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:28.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.481+0000 7f432f7fe700 1 --2- 172.21.15.162:0/2366245856 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f43300aeef0 0x7f4330140d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:28.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.481+0000 7f432f7fe700 1 -- 172.21.15.162:0/2366245856 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4330144620 msgr2=0x7f43301532c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:28.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.481+0000 7f432f7fe700 1 --2- 172.21.15.162:0/2366245856 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4330144620 0x7f43301532c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:28.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.481+0000 7f432f7fe700 1 -- 172.21.15.162:0/2366245856 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4330064550 con 0x7f43300ae710 2020-11-28T21:53:28.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.481+0000 7f432ffff700 1 --2- 172.21.15.162:0/2366245856 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4330144620 0x7f43301532c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:53:28.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.481+0000 7f432f7fe700 1 --2- 172.21.15.162:0/2366245856 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43300ae710 0x7f43300b4170 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7f4318002f30 tx=0x7f4318015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:28.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.481+0000 7f432cff9700 1 -- 172.21.15.162:0/2366245856 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4318011b00 con 0x7f43300ae710 2020-11-28T21:53:28.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.481+0000 7f432cff9700 1 -- 172.21.15.162:0/2366245856 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4318011c60 con 0x7f43300ae710 2020-11-28T21:53:28.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.481+0000 7f4334d28700 1 -- 172.21.15.162:0/2366245856 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f433013f200 con 0x7f43300ae710 2020-11-28T21:53:28.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.481+0000 7f432cff9700 1 -- 172.21.15.162:0/2366245856 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f431801d450 con 0x7f43300ae710 2020-11-28T21:53:28.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.481+0000 7f4334d28700 1 -- 172.21.15.162:0/2366245856 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4330063c90 con 0x7f43300ae710 2020-11-28T21:53:28.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.485+0000 7f4334d28700 1 -- 172.21.15.162:0/2366245856 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4330057050 con 0x7f43300ae710 2020-11-28T21:53:28.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.485+0000 7f432cff9700 1 -- 172.21.15.162:0/2366245856 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4318024030 con 0x7f43300ae710 2020-11-28T21:53:28.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.485+0000 7f432cff9700 1 --2- 172.21.15.162:0/2366245856 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f43140554e0 0x7f43140579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:28.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.485+0000 7f432cff9700 1 -- 172.21.15.162:0/2366245856 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6202+0+0 (secure 0 0 0) 0x7f431806ef60 con 0x7f43300ae710 2020-11-28T21:53:28.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.485+0000 7f432effd700 1 --2- 172.21.15.162:0/2366245856 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f43140554e0 0x7f43140579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:28.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.485+0000 7f432effd700 1 --2- 172.21.15.162:0/2366245856 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f43140554e0 0x7f43140579a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f432000b000 tx=0x7f4320008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:28.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.489+0000 7f432cff9700 1 -- 172.21.15.162:0/2366245856 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f43180451e0 con 0x7f43300ae710 2020-11-28T21:53:28.672 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.669+0000 7f4334d28700 1 -- 172.21.15.162:0/2366245856 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f43300a89f0 con 0x7f43300ae710 2020-11-28T21:53:28.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.961+0000 7f432cff9700 1 -- 172.21.15.162:0/2366245856 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v35) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4318022020 con 0x7f43300ae710 2020-11-28T21:53:28.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.965+0000 7f4334d28700 1 -- 172.21.15.162:0/2366245856 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f43140554e0 msgr2=0x7f43140579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:28.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.965+0000 7f4334d28700 1 --2- 172.21.15.162:0/2366245856 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f43140554e0 0x7f43140579a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f432000b000 tx=0x7f4320008040).stop 2020-11-28T21:53:28.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.965+0000 7f4334d28700 1 -- 172.21.15.162:0/2366245856 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43300ae710 msgr2=0x7f43300b4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:28.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.965+0000 7f4334d28700 1 --2- 172.21.15.162:0/2366245856 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43300ae710 0x7f43300b4170 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7f4318002f30 tx=0x7f4318015000).stop 2020-11-28T21:53:28.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.965+0000 7f4334d28700 1 -- 172.21.15.162:0/2366245856 shutdown_connections 2020-11-28T21:53:28.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.965+0000 7f4334d28700 1 --2- 172.21.15.162:0/2366245856 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f43140554e0 0x7f43140579a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:28.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.965+0000 7f4334d28700 1 --2- 172.21.15.162:0/2366245856 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4330144620 0x7f43301532c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:28.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.965+0000 7f4334d28700 1 --2- 172.21.15.162:0/2366245856 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43300ae710 0x7f43300b4170 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:28.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.965+0000 7f4334d28700 1 --2- 172.21.15.162:0/2366245856 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f43300aeef0 0x7f4330140d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:28.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.965+0000 7f4334d28700 1 -- 172.21.15.162:0/2366245856 >> 172.21.15.162:0/2366245856 conn(0x7f43300b17a0 msgr2=0x7f43300ab760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:28.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.965+0000 7f4334d28700 1 -- 172.21.15.162:0/2366245856 shutdown_connections 2020-11-28T21:53:28.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:28.965+0000 7f4334d28700 1 -- 172.21.15.162:0/2366245856 wait complete. 2020-11-28T21:53:28.986 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-11-28T21:53:29.104 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.101+0000 7fb07c227700 1 Processor -- start 2020-11-28T21:53:29.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.101+0000 7fb07c227700 1 -- start start 2020-11-28T21:53:29.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.101+0000 7fb07c227700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb0740649c0 0x7fb074143580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:29.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.101+0000 7fb07c227700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb074064da0 con 0x7fb0740b0bf0 2020-11-28T21:53:29.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.101+0000 7fb07c227700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb074142ba0 con 0x7fb074060200 2020-11-28T21:53:29.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.101+0000 7fb07c227700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0741429b0 con 0x7fb0740649c0 2020-11-28T21:53:29.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.101+0000 7fb079fc3700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb0740649c0 0x7fb074143580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:29.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.101+0000 7fb0797c2700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fb074060200 0x7fb07405b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:48720/0 (socket says 172.21.15.162:48720) 2020-11-28T21:53:29.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.101+0000 7fb0797c2700 1 -- 172.21.15.162:0/2973778155 learned_addr learned my addr 172.21.15.162:0/2973778155 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:29.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.105+0000 7fb079fc3700 1 -- 172.21.15.162:0/2973778155 >> v1:172.21.15.162:6789/0 conn(0x7fb074060200 legacy=0x7fb07405b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:29.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.105+0000 7fb078fc1700 1 -- 172.21.15.162:0/2973778155 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1854728308 0 0) 0x7fb074142ba0 con 0x7fb074060200 2020-11-28T21:53:29.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.105+0000 7fb079fc3700 1 -- 172.21.15.162:0/2973778155 >> v1:172.21.15.118:6789/0 conn(0x7fb0740b0bf0 legacy=0x7fb07413fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:29.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.105+0000 7fb079fc3700 1 -- 172.21.15.162:0/2973778155 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb074142e40 con 0x7fb0740649c0 2020-11-28T21:53:29.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.105+0000 7fb079fc3700 1 --2- 172.21.15.162:0/2973778155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0740649c0 0x7fb074143580 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7fb06c0010d0 tx=0x7fb05c0091a0).ready entity=mon.1 client_cookie=491d76d7365947e server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:29.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.105+0000 7fb078fc1700 1 -- 172.21.15.162:0/2973778155 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb05c00b910 con 0x7fb0740649c0 2020-11-28T21:53:29.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.105+0000 7fb078fc1700 1 -- 172.21.15.162:0/2973778155 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb05c00ba70 con 0x7fb0740649c0 2020-11-28T21:53:29.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.105+0000 7fb078fc1700 1 -- 172.21.15.162:0/2973778155 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb05c00bbe0 con 0x7fb0740649c0 2020-11-28T21:53:29.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.105+0000 7fb07c227700 1 -- 172.21.15.162:0/2973778155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0740649c0 msgr2=0x7fb074143580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:29.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.105+0000 7fb07c227700 1 --2- 172.21.15.162:0/2973778155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0740649c0 0x7fb074143580 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7fb06c0010d0 tx=0x7fb05c0091a0).stop 2020-11-28T21:53:29.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.105+0000 7fb07c227700 1 -- 172.21.15.162:0/2973778155 shutdown_connections 2020-11-28T21:53:29.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.105+0000 7fb07c227700 1 --2- 172.21.15.162:0/2973778155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0740649c0 0x7fb074143580 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:29.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.105+0000 7fb07c227700 1 -- 172.21.15.162:0/2973778155 >> 172.21.15.162:0/2973778155 conn(0x7fb074061740 msgr2=0x7fb074061b50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:29.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.105+0000 7fb07c227700 1 -- 172.21.15.162:0/2973778155 shutdown_connections 2020-11-28T21:53:29.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.105+0000 7fb07c227700 1 -- 172.21.15.162:0/2973778155 wait complete. 2020-11-28T21:53:29.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.109+0000 7fb07c227700 1 Processor -- start 2020-11-28T21:53:29.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.109+0000 7fb07c227700 1 -- start start 2020-11-28T21:53:29.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.109+0000 7fb07c227700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb074060200 0x7fb074140b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:29.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.109+0000 7fb07c227700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb0740649c0 0x7fb074155280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:29.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.109+0000 7fb07c227700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0740b0bf0 0x7fb074155770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:29.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.109+0000 7fb07c227700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0740b0370 con 0x7fb074060200 2020-11-28T21:53:29.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.109+0000 7fb07c227700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb074154e20 con 0x7fb0740b0bf0 2020-11-28T21:53:29.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.109+0000 7fb07c227700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb074155c60 con 0x7fb0740649c0 2020-11-28T21:53:29.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.109+0000 7fb079fc3700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb074060200 0x7fb074140b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:29.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.109+0000 7fb07a7c4700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0740b0bf0 0x7fb074155770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:29.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.109+0000 7fb079fc3700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb074060200 0x7fb074140b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.162:48868/0 (socket says 172.21.15.162:48868) 2020-11-28T21:53:29.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.109+0000 7fb079fc3700 1 -- 172.21.15.162:0/1408940848 learned_addr learned my addr 172.21.15.162:0/1408940848 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:29.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.109+0000 7fb0797c2700 1 --2- 172.21.15.162:0/1408940848 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb0740649c0 0x7fb074155280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:29.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.109+0000 7fb07a7c4700 1 -- 172.21.15.162:0/1408940848 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb0740649c0 msgr2=0x7fb074155280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:29.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.109+0000 7fb07a7c4700 1 --2- 172.21.15.162:0/1408940848 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb0740649c0 0x7fb074155280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:29.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.109+0000 7fb07a7c4700 1 -- 172.21.15.162:0/1408940848 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb074060200 msgr2=0x7fb074140b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:29.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.109+0000 7fb07a7c4700 1 --2- 172.21.15.162:0/1408940848 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb074060200 0x7fb074140b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:29.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.109+0000 7fb07a7c4700 1 -- 172.21.15.162:0/1408940848 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb074143fe0 con 0x7fb0740b0bf0 2020-11-28T21:53:29.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.109+0000 7fb07a7c4700 1 --2- 172.21.15.162:0/1408940848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0740b0bf0 0x7fb074155770 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7fb0740b04b0 tx=0x7fb06c007b00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:29.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.109+0000 7fb072ffd700 1 -- 172.21.15.162:0/1408940848 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb06c013410 con 0x7fb0740b0bf0 2020-11-28T21:53:29.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.109+0000 7fb072ffd700 1 -- 172.21.15.162:0/1408940848 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb06c013570 con 0x7fb0740b0bf0 2020-11-28T21:53:29.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.109+0000 7fb07c227700 1 -- 172.21.15.162:0/1408940848 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb074144280 con 0x7fb0740b0bf0 2020-11-28T21:53:29.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.109+0000 7fb072ffd700 1 -- 172.21.15.162:0/1408940848 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb06c0138d0 con 0x7fb0740b0bf0 2020-11-28T21:53:29.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.109+0000 7fb07c227700 1 -- 172.21.15.162:0/1408940848 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb07413e360 con 0x7fb0740b0bf0 2020-11-28T21:53:29.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.113+0000 7fb07c227700 1 -- 172.21.15.162:0/1408940848 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb074151950 con 0x7fb0740b0bf0 2020-11-28T21:53:29.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.113+0000 7fb072ffd700 1 -- 172.21.15.162:0/1408940848 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb06c010030 con 0x7fb0740b0bf0 2020-11-28T21:53:29.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.113+0000 7fb072ffd700 1 --2- 172.21.15.162:0/1408940848 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb0580553b0 0x7fb058057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:29.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.113+0000 7fb079fc3700 1 --2- 172.21.15.162:0/1408940848 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb0580553b0 0x7fb058057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:29.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.113+0000 7fb072ffd700 1 -- 172.21.15.162:0/1408940848 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6202+0+0 (secure 0 0 0) 0x7fb06c06dcc0 con 0x7fb0740b0bf0 2020-11-28T21:53:29.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.113+0000 7fb079fc3700 1 --2- 172.21.15.162:0/1408940848 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb0580553b0 0x7fb058057870 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7fb05c01b000 tx=0x7fb05c016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:29.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.121+0000 7fb072ffd700 1 -- 172.21.15.162:0/1408940848 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb074151950 con 0x7fb0740b0bf0 2020-11-28T21:53:29.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.261+0000 7fb07c227700 1 -- 172.21.15.162:0/1408940848 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb074151950 con 0x7fb0740b0bf0 2020-11-28T21:53:29.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.981+0000 7fb072ffd700 1 -- 172.21.15.162:0/1408940848 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v79) v1 ==== 185+0+0 (secure 0 0 0) 0x7fb074151950 con 0x7fb0740b0bf0 2020-11-28T21:53:29.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.985+0000 7fb07c227700 1 -- 172.21.15.162:0/1408940848 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb0580553b0 msgr2=0x7fb058057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:29.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.985+0000 7fb07c227700 1 --2- 172.21.15.162:0/1408940848 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb0580553b0 0x7fb058057870 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7fb05c01b000 tx=0x7fb05c016000).stop 2020-11-28T21:53:29.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.985+0000 7fb07c227700 1 -- 172.21.15.162:0/1408940848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0740b0bf0 msgr2=0x7fb074155770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:29.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.985+0000 7fb07c227700 1 --2- 172.21.15.162:0/1408940848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0740b0bf0 0x7fb074155770 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7fb0740b04b0 tx=0x7fb06c007b00).stop 2020-11-28T21:53:29.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.985+0000 7fb07c227700 1 -- 172.21.15.162:0/1408940848 shutdown_connections 2020-11-28T21:53:29.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.985+0000 7fb07c227700 1 --2- 172.21.15.162:0/1408940848 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb0580553b0 0x7fb058057870 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:29.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.985+0000 7fb07c227700 1 --2- 172.21.15.162:0/1408940848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0740b0bf0 0x7fb074155770 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:29.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.985+0000 7fb07c227700 1 --2- 172.21.15.162:0/1408940848 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb074060200 0x7fb074140b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:29.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.985+0000 7fb07c227700 1 --2- 172.21.15.162:0/1408940848 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb0740649c0 0x7fb074155280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:29.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.985+0000 7fb07c227700 1 -- 172.21.15.162:0/1408940848 >> 172.21.15.162:0/1408940848 conn(0x7fb074061740 msgr2=0x7fb0740b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:29.992 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.985+0000 7fb07c227700 1 -- 172.21.15.162:0/1408940848 shutdown_connections 2020-11-28T21:53:29.992 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:29.985+0000 7fb07c227700 1 -- 172.21.15.162:0/1408940848 wait complete. 2020-11-28T21:53:29.992 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_metadata' removed 2020-11-28T21:53:30.003 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-11-28T21:53:30.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.117+0000 7f11b759e700 1 Processor -- start 2020-11-28T21:53:30.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.117+0000 7f11b759e700 1 -- start start 2020-11-28T21:53:30.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.117+0000 7f11b759e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f11b805bb10 0x7f11b8060920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:30.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.117+0000 7f11b759e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f11b80b2180 0x7f11b80611d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:30.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.117+0000 7f11b759e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11b8146d00 con 0x7f11b809bcd0 2020-11-28T21:53:30.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.117+0000 7f11b759e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11b80ac2c0 con 0x7f11b805bb10 2020-11-28T21:53:30.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.117+0000 7f11b759e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11b80ac020 con 0x7f11b80b2180 2020-11-28T21:53:30.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.117+0000 7f11b6d9d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f11b80b2180 0x7f11b80611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:30.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.117+0000 7f11b6d9d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f11b80b2180 0x7f11b80611d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41890/0 (socket says 172.21.15.162:41890) 2020-11-28T21:53:30.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.117+0000 7f11b6d9d700 1 -- 172.21.15.162:0/2420525165 learned_addr learned my addr 172.21.15.162:0/2420525165 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:30.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.117+0000 7f11b5d9b700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f11b809bcd0 0x7f11b8147400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:48736/0 (socket says 172.21.15.162:48736) 2020-11-28T21:53:30.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.117+0000 7f11b659c700 1 --2- 172.21.15.162:0/2420525165 >> v2:172.21.15.118:3300/0 conn(0x7f11b805bb10 0x7f11b8060920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:30.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.117+0000 7f11b6d9d700 1 -- 172.21.15.162:0/2420525165 >> v2:172.21.15.118:3300/0 conn(0x7f11b805bb10 msgr2=0x7f11b8060920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:30.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.117+0000 7f11b6d9d700 1 --2- 172.21.15.162:0/2420525165 >> v2:172.21.15.118:3300/0 conn(0x7f11b805bb10 0x7f11b8060920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:30.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.117+0000 7f11b6d9d700 1 -- 172.21.15.162:0/2420525165 >> v1:172.21.15.162:6789/0 conn(0x7f11b809bcd0 legacy=0x7f11b8147400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:30.129 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.117+0000 7f11b6d9d700 1 -- 172.21.15.162:0/2420525165 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b8146f60 con 0x7f11b80b2180 2020-11-28T21:53:30.129 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.117+0000 7f11b6d9d700 1 --2- 172.21.15.162:0/2420525165 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11b80b2180 0x7f11b80611d0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7f11b00032b0 tx=0x7f11b0009270).ready entity=mon.1 client_cookie=18bf9af333adbfed server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:30.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.117+0000 7f11b559a700 1 -- 172.21.15.162:0/2420525165 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11b000bc00 con 0x7f11b80b2180 2020-11-28T21:53:30.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.117+0000 7f11b559a700 1 -- 172.21.15.162:0/2420525165 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b000bd60 con 0x7f11b80b2180 2020-11-28T21:53:30.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.117+0000 7f11b559a700 1 -- 172.21.15.162:0/2420525165 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11b0011400 con 0x7f11b80b2180 2020-11-28T21:53:30.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.117+0000 7f11b759e700 1 -- 172.21.15.162:0/2420525165 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11b80b2180 msgr2=0x7f11b80611d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:30.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.117+0000 7f11b759e700 1 --2- 172.21.15.162:0/2420525165 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11b80b2180 0x7f11b80611d0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7f11b00032b0 tx=0x7f11b0009270).stop 2020-11-28T21:53:30.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.117+0000 7f11b759e700 1 -- 172.21.15.162:0/2420525165 shutdown_connections 2020-11-28T21:53:30.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.117+0000 7f11b759e700 1 --2- 172.21.15.162:0/2420525165 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11b80b2180 0x7f11b80611d0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:30.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.117+0000 7f11b759e700 1 --2- 172.21.15.162:0/2420525165 >> v2:172.21.15.118:3300/0 conn(0x7f11b805bb10 0x7f11b8060920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:30.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.117+0000 7f11b759e700 1 -- 172.21.15.162:0/2420525165 >> 172.21.15.162:0/2420525165 conn(0x7f11b80b2830 msgr2=0x7f11b80b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:30.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.117+0000 7f11b759e700 1 -- 172.21.15.162:0/2420525165 shutdown_connections 2020-11-28T21:53:30.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.117+0000 7f11b759e700 1 -- 172.21.15.162:0/2420525165 wait complete. 2020-11-28T21:53:30.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.121+0000 7f11b759e700 1 Processor -- start 2020-11-28T21:53:30.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.121+0000 7f11b759e700 1 -- start start 2020-11-28T21:53:30.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.125+0000 7f11b759e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f11b805bb10 0x7f11b8159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:30.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.125+0000 7f11b759e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11b809bcd0 0x7f11b8143230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:30.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.125+0000 7f11b759e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f11b80b2180 0x7f11b8155a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:30.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.125+0000 7f11b759e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b80b5760 con 0x7f11b805bb10 2020-11-28T21:53:30.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.125+0000 7f11b759e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b8159040 con 0x7f11b809bcd0 2020-11-28T21:53:30.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.125+0000 7f11b759e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11b81592e0 con 0x7f11b80b2180 2020-11-28T21:53:30.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.125+0000 7f11b5d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11b809bcd0 0x7f11b8143230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:30.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.125+0000 7f11b659c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f11b805bb10 0x7f11b8159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:30.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.125+0000 7f11b659c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f11b805bb10 0x7f11b8159a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.162:48882/0 (socket says 172.21.15.162:48882) 2020-11-28T21:53:30.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.125+0000 7f11b5d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11b809bcd0 0x7f11b8143230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41896/0 (socket says 172.21.15.162:41896) 2020-11-28T21:53:30.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.125+0000 7f11b659c700 1 -- 172.21.15.162:0/2248590726 learned_addr learned my addr 172.21.15.162:0/2248590726 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:30.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.125+0000 7f11b6d9d700 1 --2- 172.21.15.162:0/2248590726 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f11b80b2180 0x7f11b8155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:30.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.125+0000 7f11b5d9b700 1 -- 172.21.15.162:0/2248590726 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f11b80b2180 msgr2=0x7f11b8155a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:30.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.125+0000 7f11b5d9b700 1 --2- 172.21.15.162:0/2248590726 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f11b80b2180 0x7f11b8155a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:30.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.125+0000 7f11b5d9b700 1 -- 172.21.15.162:0/2248590726 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f11b805bb10 msgr2=0x7f11b8159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:30.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.125+0000 7f11b5d9b700 1 --2- 172.21.15.162:0/2248590726 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f11b805bb10 0x7f11b8159a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:30.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.125+0000 7f11b5d9b700 1 -- 172.21.15.162:0/2248590726 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b8159580 con 0x7f11b809bcd0 2020-11-28T21:53:30.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.125+0000 7f11b5d9b700 1 --2- 172.21.15.162:0/2248590726 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11b809bcd0 0x7f11b8143230 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0x7f11b80ad0c0 tx=0x7f11a8007b50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:30.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.125+0000 7f11af7fe700 1 -- 172.21.15.162:0/2248590726 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11a80072e0 con 0x7f11b809bcd0 2020-11-28T21:53:30.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.125+0000 7f11af7fe700 1 -- 172.21.15.162:0/2248590726 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a8005520 con 0x7f11b809bcd0 2020-11-28T21:53:30.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.125+0000 7f11b759e700 1 -- 172.21.15.162:0/2248590726 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11b8143720 con 0x7f11b809bcd0 2020-11-28T21:53:30.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.125+0000 7f11af7fe700 1 -- 172.21.15.162:0/2248590726 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11a8005880 con 0x7f11b809bcd0 2020-11-28T21:53:30.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.125+0000 7f11b759e700 1 -- 172.21.15.162:0/2248590726 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11b8143d20 con 0x7f11b809bcd0 2020-11-28T21:53:30.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.125+0000 7f11b759e700 1 -- 172.21.15.162:0/2248590726 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11b8057050 con 0x7f11b809bcd0 2020-11-28T21:53:30.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.125+0000 7f11af7fe700 1 -- 172.21.15.162:0/2248590726 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f11a8010070 con 0x7f11b809bcd0 2020-11-28T21:53:30.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.129+0000 7f11af7fe700 1 --2- 172.21.15.162:0/2248590726 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f119c055440 0x7f119c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:30.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.129+0000 7f11af7fe700 1 -- 172.21.15.162:0/2248590726 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5764+0+0 (secure 0 0 0) 0x7f11a806d370 con 0x7f11b809bcd0 2020-11-28T21:53:30.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.129+0000 7f11b659c700 1 --2- 172.21.15.162:0/2248590726 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f119c055440 0x7f119c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:30.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.129+0000 7f11b659c700 1 --2- 172.21.15.162:0/2248590726 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f119c055440 0x7f119c057900 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f11a0008000 tx=0x7f11a0008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:30.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.133+0000 7f11af7fe700 1 -- 172.21.15.162:0/2248590726 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f11a8043700 con 0x7f11b809bcd0 2020-11-28T21:53:30.279 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.277+0000 7f11b759e700 1 -- 172.21.15.162:0/2248590726 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f11b8068350 con 0x7f11b809bcd0 2020-11-28T21:53:30.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.985+0000 7f11af7fe700 1 -- 172.21.15.162:0/2248590726 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v80) v1 ==== 173+0+0 (secure 0 0 0) 0x7f11a8046d20 con 0x7f11b809bcd0 2020-11-28T21:53:30.993 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.989+0000 7f11b759e700 1 -- 172.21.15.162:0/2248590726 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f119c055440 msgr2=0x7f119c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:30.993 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.989+0000 7f11b759e700 1 --2- 172.21.15.162:0/2248590726 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f119c055440 0x7f119c057900 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f11a0008000 tx=0x7f11a0008040).stop 2020-11-28T21:53:30.994 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.989+0000 7f11b759e700 1 -- 172.21.15.162:0/2248590726 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11b809bcd0 msgr2=0x7f11b8143230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:30.994 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.989+0000 7f11b759e700 1 --2- 172.21.15.162:0/2248590726 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11b809bcd0 0x7f11b8143230 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0x7f11b80ad0c0 tx=0x7f11a8007b50).stop 2020-11-28T21:53:30.994 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.989+0000 7f11b759e700 1 -- 172.21.15.162:0/2248590726 shutdown_connections 2020-11-28T21:53:30.994 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.989+0000 7f11b759e700 1 --2- 172.21.15.162:0/2248590726 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f119c055440 0x7f119c057900 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:30.994 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.989+0000 7f11b759e700 1 --2- 172.21.15.162:0/2248590726 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f11b80b2180 0x7f11b8155a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:30.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.989+0000 7f11b759e700 1 --2- 172.21.15.162:0/2248590726 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f11b805bb10 0x7f11b8159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:30.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.989+0000 7f11b759e700 1 --2- 172.21.15.162:0/2248590726 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11b809bcd0 0x7f11b8143230 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:30.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.989+0000 7f11b759e700 1 -- 172.21.15.162:0/2248590726 >> 172.21.15.162:0/2248590726 conn(0x7f11b80b2830 msgr2=0x7f11b80b1080 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:30.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.989+0000 7f11b759e700 1 -- 172.21.15.162:0/2248590726 shutdown_connections 2020-11-28T21:53:30.996 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:30.989+0000 7f11b759e700 1 -- 172.21.15.162:0/2248590726 wait complete. 2020-11-28T21:53:30.996 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_data' removed 2020-11-28T21:53:31.009 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-28T21:53:31.009 INFO:tasks.ceph.mds.a:Stopping old one... 2020-11-28T21:53:31.009 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-11-28T21:53:31.010 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T21:53:31.010 INFO:tasks.ceph.mds.c:Restarting daemon 2020-11-28T21:53:31.010 INFO:tasks.ceph.mds.c:Stopping old one... 2020-11-28T21:53:31.011 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2020-11-28T21:53:31.011 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T21:53:31.011 INFO:tasks.ceph.mds.b:Restarting daemon 2020-11-28T21:53:31.012 INFO:tasks.ceph.mds.b:Stopping old one... 2020-11-28T21:53:31.012 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-11-28T21:53:31.012 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T21:53:31.022 INFO:tasks.ceph.mds.a:Stopped 2020-11-28T21:53:31.023 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-28T21:53:31.025 INFO:tasks.ceph.mds.b:Stopped 2020-11-28T21:53:31.025 INFO:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-11-28T21:53:31.026 INFO:tasks.ceph.mds.c:Stopped 2020-11-28T21:53:31.026 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2020-11-28T21:53:31.067 INFO:tasks.ceph.mds.a:Started 2020-11-28T21:53:31.068 DEBUG:teuthology.parallel:result is None 2020-11-28T21:53:31.069 INFO:tasks.ceph.mds.c:Started 2020-11-28T21:53:31.069 DEBUG:teuthology.parallel:result is None 2020-11-28T21:53:31.070 INFO:tasks.ceph.mds.b:Started 2020-11-28T21:53:31.070 DEBUG:teuthology.parallel:result is None 2020-11-28T21:53:31.071 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist clear 2020-11-28T21:53:31.139 INFO:tasks.ceph.mds.a.smithi162.stdout:starting mds.a at 2020-11-28T21:53:31.145 INFO:tasks.ceph.mds.c.smithi162.stdout:starting mds.c at 2020-11-28T21:53:31.150 INFO:tasks.ceph.mds.b.smithi118.stdout:starting mds.b at 2020-11-28T21:53:31.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.185+0000 7f9e6359e700 1 Processor -- start 2020-11-28T21:53:31.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.185+0000 7f9e6359e700 1 -- start start 2020-11-28T21:53:31.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.185+0000 7f9e6359e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9e64080960 0x7f9e640b2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:31.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.185+0000 7f9e6359e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9e640a99e0 0x7f9e640ab0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:31.200 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.185+0000 7f9e6359e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e64061210 con 0x7f9e64060e30 2020-11-28T21:53:31.200 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.185+0000 7f9e6359e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e64060160 con 0x7f9e64080960 2020-11-28T21:53:31.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.185+0000 7f9e6359e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e64060400 con 0x7f9e640a99e0 2020-11-28T21:53:31.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.185+0000 7f9e5bfff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9e64080960 0x7f9e640b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:31.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.185+0000 7f9e6259c700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f9e64060e30 0x7f9e640a92d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.162:38918/0 (socket says 172.21.15.162:38918) 2020-11-28T21:53:31.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.185+0000 7f9e6259c700 1 -- 172.21.15.162:0/3318553964 learned_addr learned my addr 172.21.15.162:0/3318553964 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:31.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.185+0000 7f9e62d9d700 1 --2- 172.21.15.162:0/3318553964 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9e640a99e0 0x7f9e640ab0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:31.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.185+0000 7f9e61d9b700 1 -- 172.21.15.162:0/3318553964 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2576452703 0 0) 0x7f9e64061210 con 0x7f9e64060e30 2020-11-28T21:53:31.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.185+0000 7f9e61d9b700 1 -- 172.21.15.162:0/3318553964 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e44003230 con 0x7f9e64060e30 2020-11-28T21:53:31.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.185+0000 7f9e62d9d700 1 -- 172.21.15.162:0/3318553964 >> v2:172.21.15.162:3300/0 conn(0x7f9e64080960 msgr2=0x7f9e640b2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:31.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.189+0000 7f9e62d9d700 1 --2- 172.21.15.162:0/3318553964 >> v2:172.21.15.162:3300/0 conn(0x7f9e64080960 0x7f9e640b2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:31.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.189+0000 7f9e62d9d700 1 -- 172.21.15.162:0/3318553964 >> v1:172.21.15.118:6789/0 conn(0x7f9e64060e30 legacy=0x7f9e640a92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:31.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.189+0000 7f9e62d9d700 1 -- 172.21.15.162:0/3318553964 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e640b4d40 con 0x7f9e640a99e0 2020-11-28T21:53:31.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.189+0000 7f9e5bfff700 1 --2- 172.21.15.162:0/3318553964 >> v2:172.21.15.162:3300/0 conn(0x7f9e64080960 0x7f9e640b2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:53:31.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.189+0000 7f9e62d9d700 1 --2- 172.21.15.162:0/3318553964 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9e640a99e0 0x7f9e640ab0b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f9e5c0032b0 tx=0x7f9e5c0091a0).ready entity=mon.2 client_cookie=2658fea2a009bc0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:31.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.189+0000 7f9e61d9b700 1 -- 172.21.15.162:0/3318553964 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e5c00b940 con 0x7f9e640a99e0 2020-11-28T21:53:31.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.189+0000 7f9e61d9b700 1 -- 172.21.15.162:0/3318553964 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e5c00baa0 con 0x7f9e640a99e0 2020-11-28T21:53:31.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.189+0000 7f9e61d9b700 1 -- 172.21.15.162:0/3318553964 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e5c00bc10 con 0x7f9e640a99e0 2020-11-28T21:53:31.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.189+0000 7f9e6359e700 1 -- 172.21.15.162:0/3318553964 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9e640a99e0 msgr2=0x7f9e640ab0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:31.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.189+0000 7f9e6359e700 1 --2- 172.21.15.162:0/3318553964 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9e640a99e0 0x7f9e640ab0b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f9e5c0032b0 tx=0x7f9e5c0091a0).stop 2020-11-28T21:53:31.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.189+0000 7f9e6359e700 1 -- 172.21.15.162:0/3318553964 shutdown_connections 2020-11-28T21:53:31.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.189+0000 7f9e6359e700 1 --2- 172.21.15.162:0/3318553964 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9e640a99e0 0x7f9e640ab0b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:31.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.189+0000 7f9e6359e700 1 --2- 172.21.15.162:0/3318553964 >> v2:172.21.15.162:3300/0 conn(0x7f9e64080960 0x7f9e640b2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:31.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.189+0000 7f9e6359e700 1 -- 172.21.15.162:0/3318553964 >> 172.21.15.162:0/3318553964 conn(0x7f9e6409a2f0 msgr2=0x7f9e6409a700 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:31.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.189+0000 7f9e6359e700 1 -- 172.21.15.162:0/3318553964 shutdown_connections 2020-11-28T21:53:31.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.189+0000 7f9e6359e700 1 -- 172.21.15.162:0/3318553964 wait complete. 2020-11-28T21:53:31.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.189+0000 7f9e6359e700 1 Processor -- start 2020-11-28T21:53:31.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.189+0000 7f9e6359e700 1 -- start start 2020-11-28T21:53:31.212 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.193+0000 7f9e6359e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9e64060e30 0x7f9e64095360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:31.212 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.193+0000 7f9e6359e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e64080960 0x7f9e64094240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:31.212 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.193+0000 7f9e6359e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e640a99e0 0x7f9e64097ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:31.213 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.193+0000 7f9e6359e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e64096a30 con 0x7f9e640a99e0 2020-11-28T21:53:31.213 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.193+0000 7f9e6359e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e64094730 con 0x7f9e64080960 2020-11-28T21:53:31.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.193+0000 7f9e6359e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e64163740 con 0x7f9e64060e30 2020-11-28T21:53:31.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.193+0000 7f9e5bfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e64080960 0x7f9e64094240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:31.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.193+0000 7f9e5bfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e64080960 0x7f9e64094240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41934/0 (socket says 172.21.15.162:41934) 2020-11-28T21:53:31.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.193+0000 7f9e5bfff700 1 -- 172.21.15.162:0/1837698109 learned_addr learned my addr 172.21.15.162:0/1837698109 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:31.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.193+0000 7f9e6259c700 1 --2- 172.21.15.162:0/1837698109 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9e64060e30 0x7f9e64095360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:31.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.193+0000 7f9e62d9d700 1 --2- 172.21.15.162:0/1837698109 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e640a99e0 0x7f9e64097ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:31.216 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.193+0000 7f9e5bfff700 1 -- 172.21.15.162:0/1837698109 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9e64060e30 msgr2=0x7f9e64095360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:31.216 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.193+0000 7f9e5bfff700 1 --2- 172.21.15.162:0/1837698109 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9e64060e30 0x7f9e64095360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:31.217 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.193+0000 7f9e5bfff700 1 -- 172.21.15.162:0/1837698109 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e640a99e0 msgr2=0x7f9e64097ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:31.218 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.193+0000 7f9e5bfff700 1 --2- 172.21.15.162:0/1837698109 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e640a99e0 0x7f9e64097ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:31.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.193+0000 7f9e5bfff700 1 -- 172.21.15.162:0/1837698109 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e64094950 con 0x7f9e64080960 2020-11-28T21:53:31.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.193+0000 7f9e5bfff700 1 --2- 172.21.15.162:0/1837698109 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e64080960 0x7f9e64094240 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7f9e4c00c000 tx=0x7f9e4c006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:31.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.193+0000 7f9e5b7fe700 1 -- 172.21.15.162:0/1837698109 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e4c00cb40 con 0x7f9e64080960 2020-11-28T21:53:31.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.193+0000 7f9e5b7fe700 1 -- 172.21.15.162:0/1837698109 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e4c00cca0 con 0x7f9e64080960 2020-11-28T21:53:31.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.193+0000 7f9e5b7fe700 1 -- 172.21.15.162:0/1837698109 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e4c0123f0 con 0x7f9e64080960 2020-11-28T21:53:31.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.193+0000 7f9e6359e700 1 -- 172.21.15.162:0/1837698109 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e64094bf0 con 0x7f9e64080960 2020-11-28T21:53:31.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.193+0000 7f9e6359e700 1 -- 172.21.15.162:0/1837698109 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e640983d0 con 0x7f9e64080960 2020-11-28T21:53:31.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.193+0000 7f9e6359e700 1 -- 172.21.15.162:0/1837698109 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e64094040 con 0x7f9e64080960 2020-11-28T21:53:31.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.193+0000 7f9e5b7fe700 1 -- 172.21.15.162:0/1837698109 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9e4c012550 con 0x7f9e64080960 2020-11-28T21:53:31.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.193+0000 7f9e5b7fe700 1 --2- 172.21.15.162:0/1837698109 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9e480553b0 0x7f9e48057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:31.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.193+0000 7f9e5b7fe700 1 -- 172.21.15.162:0/1837698109 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f9e4c06e360 con 0x7f9e64080960 2020-11-28T21:53:31.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.193+0000 7f9e6259c700 1 --2- 172.21.15.162:0/1837698109 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9e480553b0 0x7f9e48057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:31.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.197+0000 7f9e6259c700 1 --2- 172.21.15.162:0/1837698109 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9e480553b0 0x7f9e48057870 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7f9e5400d000 tx=0x7f9e54009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:31.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.201+0000 7f9e5b7fe700 1 -- 172.21.15.162:0/1837698109 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9e4c044810 con 0x7f9e64080960 2020-11-28T21:53:31.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:31.357+0000 7f9e6359e700 1 -- 172.21.15.162:0/1837698109 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f9e64057050 con 0x7f9e64080960 2020-11-28T21:53:32.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.193+0000 7f9e5b7fe700 1 -- 172.21.15.162:0/1837698109 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v83) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9e4c047e30 con 0x7f9e64080960 2020-11-28T21:53:32.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.197+0000 7f9e6359e700 1 -- 172.21.15.162:0/1837698109 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9e480553b0 msgr2=0x7f9e48057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:32.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.197+0000 7f9e6359e700 1 --2- 172.21.15.162:0/1837698109 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9e480553b0 0x7f9e48057870 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7f9e5400d000 tx=0x7f9e54009040).stop 2020-11-28T21:53:32.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.197+0000 7f9e6359e700 1 -- 172.21.15.162:0/1837698109 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e64080960 msgr2=0x7f9e64094240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:32.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.197+0000 7f9e6359e700 1 --2- 172.21.15.162:0/1837698109 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e64080960 0x7f9e64094240 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7f9e4c00c000 tx=0x7f9e4c006dc0).stop 2020-11-28T21:53:32.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.197+0000 7f9e6359e700 1 -- 172.21.15.162:0/1837698109 shutdown_connections 2020-11-28T21:53:32.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.197+0000 7f9e6359e700 1 --2- 172.21.15.162:0/1837698109 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9e480553b0 0x7f9e48057870 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:32.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.197+0000 7f9e6359e700 1 --2- 172.21.15.162:0/1837698109 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9e64060e30 0x7f9e64095360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:32.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.197+0000 7f9e6359e700 1 --2- 172.21.15.162:0/1837698109 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e640a99e0 0x7f9e64097ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:32.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.197+0000 7f9e6359e700 1 --2- 172.21.15.162:0/1837698109 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e64080960 0x7f9e64094240 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:32.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.197+0000 7f9e6359e700 1 -- 172.21.15.162:0/1837698109 >> 172.21.15.162:0/1837698109 conn(0x7f9e6409a2f0 msgr2=0x7f9e64064260 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:32.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.197+0000 7f9e6359e700 1 -- 172.21.15.162:0/1837698109 shutdown_connections 2020-11-28T21:53:32.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.197+0000 7f9e6359e700 1 -- 172.21.15.162:0/1837698109 wait complete. 2020-11-28T21:53:32.207 INFO:teuthology.orchestra.run.smithi162.stderr: removed all blacklist entries 2020-11-28T21:53:32.217 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2020-11-28T21:53:32.218 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth ls --format=json-pretty 2020-11-28T21:53:32.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.329+0000 7efe00f6a700 1 Processor -- start 2020-11-28T21:53:32.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.329+0000 7efe00f6a700 1 -- start start 2020-11-28T21:53:32.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.329+0000 7efe00f6a700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efdfc0603d0 0x7efdfc0b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:32.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.329+0000 7efe00f6a700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7efdfc142a00 0x7efdfc09cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:32.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.329+0000 7efe00f6a700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7efdfc0b4c70 0x7efdfc09d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:32.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.329+0000 7efe00f6a700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdfc060f60 con 0x7efdfc142a00 2020-11-28T21:53:32.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.329+0000 7efe00f6a700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdfc0b4110 con 0x7efdfc0b4c70 2020-11-28T21:53:32.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.329+0000 7efe00f6a700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc0b43f0 con 0x7efdfc0603d0 2020-11-28T21:53:32.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.329+0000 7efdfbfff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7efdfc0b4c70 0x7efdfc09d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:32.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.329+0000 7efdfbfff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7efdfc0b4c70 0x7efdfc09d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41946/0 (socket says 172.21.15.162:41946) 2020-11-28T21:53:32.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.329+0000 7efdfbfff700 1 -- 172.21.15.162:0/3100522539 learned_addr learned my addr 172.21.15.162:0/3100522539 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:32.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.329+0000 7efdf2ffd700 1 --2- 172.21.15.162:0/3100522539 >> v2:172.21.15.118:3300/0 conn(0x7efdfc142a00 0x7efdfc09cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:32.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.329+0000 7efdfb7fe700 1 --2- 172.21.15.162:0/3100522539 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efdfc0603d0 0x7efdfc0b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:32.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.329+0000 7efdfbfff700 1 -- 172.21.15.162:0/3100522539 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efdfc0603d0 msgr2=0x7efdfc0b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:32.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.333+0000 7efdfbfff700 1 --2- 172.21.15.162:0/3100522539 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efdfc0603d0 0x7efdfc0b3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:32.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.333+0000 7efdfbfff700 1 -- 172.21.15.162:0/3100522539 >> v2:172.21.15.118:3300/0 conn(0x7efdfc142a00 msgr2=0x7efdfc09cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:32.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.333+0000 7efdfbfff700 1 --2- 172.21.15.162:0/3100522539 >> v2:172.21.15.118:3300/0 conn(0x7efdfc142a00 0x7efdfc09cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:32.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.333+0000 7efdfbfff700 1 -- 172.21.15.162:0/3100522539 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdfc0b4690 con 0x7efdfc0b4c70 2020-11-28T21:53:32.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.333+0000 7efdf2ffd700 1 --2- 172.21.15.162:0/3100522539 >> v2:172.21.15.118:3300/0 conn(0x7efdfc142a00 0x7efdfc09cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:53:32.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.333+0000 7efdfbfff700 1 --2- 172.21.15.162:0/3100522539 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdfc0b4c70 0x7efdfc09d4b0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 rx=0x7efdf40032b0 tx=0x7efdf4009270).ready entity=mon.1 client_cookie=3fdf130d9d3c70dd server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:32.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.333+0000 7efdfaffd700 1 -- 172.21.15.162:0/3100522539 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efdf400bc00 con 0x7efdfc0b4c70 2020-11-28T21:53:32.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.333+0000 7efdfaffd700 1 -- 172.21.15.162:0/3100522539 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf400bd60 con 0x7efdfc0b4c70 2020-11-28T21:53:32.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.333+0000 7efe00f6a700 1 -- 172.21.15.162:0/3100522539 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdfc0b4c70 msgr2=0x7efdfc09d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:32.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.333+0000 7efdfaffd700 1 -- 172.21.15.162:0/3100522539 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efdf4011400 con 0x7efdfc0b4c70 2020-11-28T21:53:32.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.333+0000 7efe00f6a700 1 --2- 172.21.15.162:0/3100522539 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdfc0b4c70 0x7efdfc09d4b0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 rx=0x7efdf40032b0 tx=0x7efdf4009270).stop 2020-11-28T21:53:32.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.333+0000 7efe00f6a700 1 -- 172.21.15.162:0/3100522539 shutdown_connections 2020-11-28T21:53:32.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.333+0000 7efe00f6a700 1 --2- 172.21.15.162:0/3100522539 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdfc0b4c70 0x7efdfc09d4b0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:32.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.333+0000 7efe00f6a700 1 --2- 172.21.15.162:0/3100522539 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efdfc0603d0 0x7efdfc0b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:32.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.333+0000 7efe00f6a700 1 --2- 172.21.15.162:0/3100522539 >> v2:172.21.15.118:3300/0 conn(0x7efdfc142a00 0x7efdfc09cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:32.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.333+0000 7efe00f6a700 1 -- 172.21.15.162:0/3100522539 >> 172.21.15.162:0/3100522539 conn(0x7efdfc066a40 msgr2=0x7efdfc066e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:32.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.333+0000 7efe00f6a700 1 -- 172.21.15.162:0/3100522539 shutdown_connections 2020-11-28T21:53:32.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.333+0000 7efe00f6a700 1 -- 172.21.15.162:0/3100522539 wait complete. 2020-11-28T21:53:32.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.333+0000 7efe00f6a700 1 Processor -- start 2020-11-28T21:53:32.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.333+0000 7efe00f6a700 1 -- start start 2020-11-28T21:53:32.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.337+0000 7efe00f6a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efdfc0603d0 0x7efdfc0ac740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:32.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.337+0000 7efe00f6a700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efdfc0b4c70 0x7efdfc146fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:32.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.337+0000 7efe00f6a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdfc143d80 0x7efdfc144940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:32.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.337+0000 7efe00f6a700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc147860 con 0x7efdfc0603d0 2020-11-28T21:53:32.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.337+0000 7efe00f6a700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc0b3080 con 0x7efdfc143d80 2020-11-28T21:53:32.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.337+0000 7efe00f6a700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc0b2bb0 con 0x7efdfc0b4c70 2020-11-28T21:53:32.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.337+0000 7efdfbfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdfc143d80 0x7efdfc144940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:32.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.337+0000 7efdfbfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdfc143d80 0x7efdfc144940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41952/0 (socket says 172.21.15.162:41952) 2020-11-28T21:53:32.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.337+0000 7efdfbfff700 1 -- 172.21.15.162:0/3693623869 learned_addr learned my addr 172.21.15.162:0/3693623869 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:32.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.337+0000 7efdfb7fe700 1 --2- 172.21.15.162:0/3693623869 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efdfc0603d0 0x7efdfc0ac740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:32.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.337+0000 7efdf2ffd700 1 --2- 172.21.15.162:0/3693623869 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efdfc0b4c70 0x7efdfc146fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:32.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.337+0000 7efdfbfff700 1 -- 172.21.15.162:0/3693623869 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efdfc0b4c70 msgr2=0x7efdfc146fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:32.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.337+0000 7efdfbfff700 1 --2- 172.21.15.162:0/3693623869 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efdfc0b4c70 0x7efdfc146fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:32.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.337+0000 7efdfbfff700 1 -- 172.21.15.162:0/3693623869 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efdfc0603d0 msgr2=0x7efdfc0ac740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:32.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.337+0000 7efdfbfff700 1 --2- 172.21.15.162:0/3693623869 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efdfc0603d0 0x7efdfc0ac740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:32.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.337+0000 7efdfbfff700 1 -- 172.21.15.162:0/3693623869 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdfc1522d0 con 0x7efdfc143d80 2020-11-28T21:53:32.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.337+0000 7efdfbfff700 1 --2- 172.21.15.162:0/3693623869 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdfc143d80 0x7efdfc144940 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 rx=0x7efdf40032b0 tx=0x7efdf4016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:32.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.337+0000 7efdf97fa700 1 -- 172.21.15.162:0/3693623869 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efdf401f670 con 0x7efdfc143d80 2020-11-28T21:53:32.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.337+0000 7efdf97fa700 1 -- 172.21.15.162:0/3693623869 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf401f7d0 con 0x7efdfc143d80 2020-11-28T21:53:32.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.337+0000 7efdf97fa700 1 -- 172.21.15.162:0/3693623869 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efdf401fb30 con 0x7efdfc143d80 2020-11-28T21:53:32.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.337+0000 7efe00f6a700 1 -- 172.21.15.162:0/3693623869 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdfc0b2de0 con 0x7efdfc143d80 2020-11-28T21:53:32.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.337+0000 7efe00f6a700 1 -- 172.21.15.162:0/3693623869 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdfc144e30 con 0x7efdfc143d80 2020-11-28T21:53:32.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.337+0000 7efe00f6a700 1 -- 172.21.15.162:0/3693623869 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdfc152120 con 0x7efdfc143d80 2020-11-28T21:53:32.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.341+0000 7efdf97fa700 1 -- 172.21.15.162:0/3693623869 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7efdf4026030 con 0x7efdfc143d80 2020-11-28T21:53:32.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.341+0000 7efdf97fa700 1 --2- 172.21.15.162:0/3693623869 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7efde0055330 0x7efde00577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:32.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.341+0000 7efdf97fa700 1 -- 172.21.15.162:0/3693623869 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5253+0+0 (secure 0 0 0) 0x7efdf406f110 con 0x7efdfc143d80 2020-11-28T21:53:32.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.341+0000 7efdfb7fe700 1 --2- 172.21.15.162:0/3693623869 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7efde0055330 0x7efde00577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:32.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.341+0000 7efdfb7fe700 1 --2- 172.21.15.162:0/3693623869 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7efde0055330 0x7efde00577f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7efdec00c000 tx=0x7efdec008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:32.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.345+0000 7efdf97fa700 1 -- 172.21.15.162:0/3693623869 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7efdf40456a0 con 0x7efdfc143d80 2020-11-28T21:53:32.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.541+0000 7efe00f6a700 1 -- 172.21.15.162:0/3693623869 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7efdfc057050 con 0x7efdfc143d80 2020-11-28T21:53:32.544 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.541+0000 7efdf97fa700 1 -- 172.21.15.162:0/3693623869 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) v1 ==== 80+0+4255 (secure 0 0 0) 0x7efdf4015090 con 0x7efdfc143d80 2020-11-28T21:53:32.544 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:53:32.545 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2020-11-28T21:53:32.545 INFO:teuthology.orchestra.run.smithi162.stdout: "auth_dump": [ 2020-11-28T21:53:32.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:32.546 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mds.a", 2020-11-28T21:53:32.546 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfWxUpKxAAIUDwQRKwz8H4Qk9qE356Bg==", 2020-11-28T21:53:32.547 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:53:32.547 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:53:32.548 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:53:32.548 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:53:32.548 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:53:32.549 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:32.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:32.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:32.550 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mds.b", 2020-11-28T21:53:32.550 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfKY2zORAAaO65etMZiYcaTDNX0nsEPw==", 2020-11-28T21:53:32.551 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:53:32.551 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:53:32.552 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:53:32.552 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:53:32.553 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:53:32.553 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:32.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:32.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:32.554 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mds.c", 2020-11-28T21:53:32.554 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfdQbRMxAAWlcjesxTZzZnvKCQneSjjQ==", 2020-11-28T21:53:32.555 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:53:32.555 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:53:32.555 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:53:32.555 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:53:32.555 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:53:32.556 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:32.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:32.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:32.556 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.0", 2020-11-28T21:53:32.557 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQADxcJfF3WLLxAAxsB5YPjH1yjv/vhlRbijqw==", 2020-11-28T21:53:32.557 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:53:32.557 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:53:32.557 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:53:32.558 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:53:32.558 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:32.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:32.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:32.559 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.1", 2020-11-28T21:53:32.559 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAGxcJfKYbBFhAA1x4DzFtkQQv6uWLIdOi1qQ==", 2020-11-28T21:53:32.559 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:53:32.560 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:53:32.560 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:53:32.560 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:53:32.560 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:32.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:32.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:32.561 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.2", 2020-11-28T21:53:32.561 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAIxcJfvcYUOhAASENjM0HfAxxyFe13mmE36Q==", 2020-11-28T21:53:32.561 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:53:32.562 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:53:32.562 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:53:32.562 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:53:32.562 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:32.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:32.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:32.563 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.3", 2020-11-28T21:53:32.563 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQALxcJfc/OHIRAAFYx3s8Ssttt15hjoH04tow==", 2020-11-28T21:53:32.564 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:53:32.564 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:53:32.564 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:53:32.564 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:53:32.564 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:32.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:32.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:32.565 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.4", 2020-11-28T21:53:32.566 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAPxcJffGWqCRAAb8CCN5oUPkdDh6BJH5HEnw==", 2020-11-28T21:53:32.566 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:53:32.566 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:53:32.566 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:53:32.567 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:53:32.568 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:32.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:32.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:32.569 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.5", 2020-11-28T21:53:32.569 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQARxcJfo8GKKRAAfbsCDb4L2wULiKmHyzu7/A==", 2020-11-28T21:53:32.569 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:53:32.569 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:53:32.569 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:53:32.570 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:53:32.570 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:32.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:32.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:32.571 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.6", 2020-11-28T21:53:32.571 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAUxcJf8Y2MEBAAqQX6rX3LK0zodlI2wK4JrA==", 2020-11-28T21:53:32.571 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:53:32.571 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:53:32.571 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:53:32.572 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:53:32.572 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:32.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:32.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:32.572 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.7", 2020-11-28T21:53:32.573 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAWxcJfCLBdMhAAf8QixQMQDrfZOGWN8rHe5g==", 2020-11-28T21:53:32.573 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:53:32.573 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:53:32.574 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:53:32.574 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:53:32.574 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:32.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:32.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:32.575 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "client.0", 2020-11-28T21:53:32.575 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQABxcJfS9vDAxAAUk4FU8FVt4if/CRH5ZisOA==", 2020-11-28T21:53:32.575 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:53:32.575 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:53:32.576 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow r", 2020-11-28T21:53:32.576 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow rw pool=cephfs_data" 2020-11-28T21:53:32.576 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:32.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:32.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:32.578 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "client.1", 2020-11-28T21:53:32.578 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQABxcJfuCU8BhAAXUSoSK3D0XRo58uHNzcjXg==", 2020-11-28T21:53:32.578 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:53:32.579 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:53:32.579 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow r", 2020-11-28T21:53:32.579 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow rw pool=cephfs_data" 2020-11-28T21:53:32.580 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:32.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:32.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:32.581 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "client.admin", 2020-11-28T21:53:32.581 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfdlTWBxAApzAxlVjzdnETQF//Vnsg5w==", 2020-11-28T21:53:32.581 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:53:32.582 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow *", 2020-11-28T21:53:32.582 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:53:32.582 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:53:32.583 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:53:32.584 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:32.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:32.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:32.585 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mgr.x", 2020-11-28T21:53:32.585 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfZhdvKBAAT7zCCsD2OVCHjvddGLxQ/A==", 2020-11-28T21:53:32.585 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:53:32.586 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow *", 2020-11-28T21:53:32.586 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow profile mgr", 2020-11-28T21:53:32.586 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:53:32.587 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:32.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:32.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:32.588 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mgr.y", 2020-11-28T21:53:32.588 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfs8fvJRAAP0Atp074EeMHiTtpCCVHuA==", 2020-11-28T21:53:32.588 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:53:32.589 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow *", 2020-11-28T21:53:32.589 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow profile mgr", 2020-11-28T21:53:32.589 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:53:32.589 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:32.589 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:32.590 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:32.590 INFO:teuthology.orchestra.run.smithi162.stdout:} 2020-11-28T21:53:32.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.541+0000 7efe00f6a700 1 -- 172.21.15.162:0/3693623869 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7efde0055330 msgr2=0x7efde00577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:32.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.541+0000 7efe00f6a700 1 --2- 172.21.15.162:0/3693623869 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7efde0055330 0x7efde00577f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7efdec00c000 tx=0x7efdec008040).stop 2020-11-28T21:53:32.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.541+0000 7efe00f6a700 1 -- 172.21.15.162:0/3693623869 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdfc143d80 msgr2=0x7efdfc144940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:32.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.541+0000 7efe00f6a700 1 --2- 172.21.15.162:0/3693623869 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdfc143d80 0x7efdfc144940 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 rx=0x7efdf40032b0 tx=0x7efdf4016000).stop 2020-11-28T21:53:32.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.541+0000 7efe00f6a700 1 -- 172.21.15.162:0/3693623869 shutdown_connections 2020-11-28T21:53:32.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.541+0000 7efe00f6a700 1 --2- 172.21.15.162:0/3693623869 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7efde0055330 0x7efde00577f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:32.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.541+0000 7efe00f6a700 1 --2- 172.21.15.162:0/3693623869 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdfc143d80 0x7efdfc144940 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:32.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.541+0000 7efe00f6a700 1 --2- 172.21.15.162:0/3693623869 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efdfc0603d0 0x7efdfc0ac740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:32.594 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.541+0000 7efe00f6a700 1 --2- 172.21.15.162:0/3693623869 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efdfc0b4c70 0x7efdfc146fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:32.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.541+0000 7efe00f6a700 1 -- 172.21.15.162:0/3693623869 >> 172.21.15.162:0/3693623869 conn(0x7efdfc066a40 msgr2=0x7efdfc0b5d30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:32.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.541+0000 7efe00f6a700 1 -- 172.21.15.162:0/3693623869 shutdown_connections 2020-11-28T21:53:32.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.541+0000 7efe00f6a700 1 -- 172.21.15.162:0/3693623869 wait complete. 2020-11-28T21:53:32.596 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:53:32.709 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.705+0000 7ff712b52700 1 Processor -- start 2020-11-28T21:53:32.710 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.705+0000 7ff712b52700 1 -- start start 2020-11-28T21:53:32.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.709+0000 7ff712b52700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff70c0608f0 0x7ff70c13c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:32.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.709+0000 7ff712b52700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff70c143730 0x7ff70c13ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:32.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.709+0000 7ff712b52700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff70c0ba0b0 con 0x7ff70c0b6130 2020-11-28T21:53:32.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.709+0000 7ff712b52700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff70c0b9e10 con 0x7ff70c143730 2020-11-28T21:53:32.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.709+0000 7ff712b52700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff70c142b80 con 0x7ff70c0608f0 2020-11-28T21:53:32.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.709+0000 7ff70bfff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7ff70c0b6130 0x7ff70c143020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:48800/0 (socket says 172.21.15.162:48800) 2020-11-28T21:53:32.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.709+0000 7ff70bfff700 1 -- 172.21.15.162:0/3838944792 learned_addr learned my addr 172.21.15.162:0/3838944792 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:32.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.709+0000 7ff7110ef700 1 --2- 172.21.15.162:0/3838944792 >> v2:172.21.15.118:3300/0 conn(0x7ff70c143730 0x7ff70c13ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:32.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.709+0000 7ff7108ee700 1 --2- 172.21.15.162:0/3838944792 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff70c0608f0 0x7ff70c13c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:32.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.709+0000 7ff70b7fe700 1 -- 172.21.15.162:0/3838944792 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3738203458 0 0) 0x7ff70c0ba0b0 con 0x7ff70c0b6130 2020-11-28T21:53:32.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.709+0000 7ff70b7fe700 1 -- 172.21.15.162:0/3838944792 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6f8003230 con 0x7ff70c0b6130 2020-11-28T21:53:32.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.709+0000 7ff70b7fe700 1 -- 172.21.15.162:0/3838944792 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 778759247 0 0) 0x7ff6f8003230 con 0x7ff70c0b6130 2020-11-28T21:53:32.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.709+0000 7ff7110ef700 1 -- 172.21.15.162:0/3838944792 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff70c0608f0 msgr2=0x7ff70c13c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:32.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.709+0000 7ff7110ef700 1 --2- 172.21.15.162:0/3838944792 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff70c0608f0 0x7ff70c13c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:32.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.709+0000 7ff7110ef700 1 -- 172.21.15.162:0/3838944792 >> v1:172.21.15.162:6789/0 conn(0x7ff70c0b6130 legacy=0x7ff70c143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:32.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.709+0000 7ff7110ef700 1 -- 172.21.15.162:0/3838944792 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff70c0b9b30 con 0x7ff70c143730 2020-11-28T21:53:32.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.709+0000 7ff7110ef700 1 --2- 172.21.15.162:0/3838944792 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff70c143730 0x7ff70c13ce00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7ff700001870 tx=0x7ff7040091a0).ready entity=mon.0 client_cookie=5c2400b88ec803f1 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:32.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.709+0000 7ff70b7fe700 1 -- 172.21.15.162:0/3838944792 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7000016d0 con 0x7ff70c143730 2020-11-28T21:53:32.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.709+0000 7ff70b7fe700 1 -- 172.21.15.162:0/3838944792 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff70400b800 con 0x7ff70c143730 2020-11-28T21:53:32.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.709+0000 7ff70b7fe700 1 -- 172.21.15.162:0/3838944792 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff70400b970 con 0x7ff70c143730 2020-11-28T21:53:32.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.709+0000 7ff712b52700 1 -- 172.21.15.162:0/3838944792 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff70c143730 msgr2=0x7ff70c13ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:32.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.709+0000 7ff712b52700 1 --2- 172.21.15.162:0/3838944792 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff70c143730 0x7ff70c13ce00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7ff700001870 tx=0x7ff7040091a0).stop 2020-11-28T21:53:32.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.709+0000 7ff712b52700 1 -- 172.21.15.162:0/3838944792 shutdown_connections 2020-11-28T21:53:32.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.709+0000 7ff712b52700 1 --2- 172.21.15.162:0/3838944792 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff70c143730 0x7ff70c13ce00 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:32.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.709+0000 7ff712b52700 1 --2- 172.21.15.162:0/3838944792 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff70c0608f0 0x7ff70c13c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:32.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.709+0000 7ff712b52700 1 -- 172.21.15.162:0/3838944792 >> 172.21.15.162:0/3838944792 conn(0x7ff70c066c70 msgr2=0x7ff70c067050 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:32.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.709+0000 7ff712b52700 1 -- 172.21.15.162:0/3838944792 shutdown_connections 2020-11-28T21:53:32.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.709+0000 7ff712b52700 1 -- 172.21.15.162:0/3838944792 wait complete. 2020-11-28T21:53:32.723 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.713+0000 7ff712b52700 1 Processor -- start 2020-11-28T21:53:32.723 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.713+0000 7ff712b52700 1 -- start start 2020-11-28T21:53:32.723 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.713+0000 7ff712b52700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff70c0608f0 0x7ff70c0b2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:32.723 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.713+0000 7ff712b52700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff70c0b6130 0x7ff70c151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:32.723 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.713+0000 7ff712b52700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff70c143730 0x7ff70c154d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:32.724 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.713+0000 7ff712b52700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70c0b8cd0 con 0x7ff70c0b6130 2020-11-28T21:53:32.724 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.713+0000 7ff712b52700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70c0b3710 con 0x7ff70c143730 2020-11-28T21:53:32.724 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.713+0000 7ff712b52700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff70c0b3470 con 0x7ff70c0608f0 2020-11-28T21:53:32.725 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.713+0000 7ff7110ef700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff70c143730 0x7ff70c154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:32.725 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.713+0000 7ff7110ef700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff70c143730 0x7ff70c154d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41966/0 (socket says 172.21.15.162:41966) 2020-11-28T21:53:32.725 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.713+0000 7ff7110ef700 1 -- 172.21.15.162:0/2965333368 learned_addr learned my addr 172.21.15.162:0/2965333368 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:32.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.713+0000 7ff70bfff700 1 --2- 172.21.15.162:0/2965333368 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff70c0b6130 0x7ff70c151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:32.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.713+0000 7ff7108ee700 1 --2- 172.21.15.162:0/2965333368 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff70c0608f0 0x7ff70c0b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:32.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.713+0000 7ff7110ef700 1 -- 172.21.15.162:0/2965333368 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff70c0608f0 msgr2=0x7ff70c0b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:32.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.713+0000 7ff7110ef700 1 --2- 172.21.15.162:0/2965333368 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff70c0608f0 0x7ff70c0b2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:32.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.713+0000 7ff7110ef700 1 -- 172.21.15.162:0/2965333368 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff70c0b6130 msgr2=0x7ff70c151730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:32.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.713+0000 7ff7110ef700 1 --2- 172.21.15.162:0/2965333368 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff70c0b6130 0x7ff70c151730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:32.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.713+0000 7ff7110ef700 1 -- 172.21.15.162:0/2965333368 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff70c155820 con 0x7ff70c143730 2020-11-28T21:53:32.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.713+0000 7ff70bfff700 1 --2- 172.21.15.162:0/2965333368 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff70c0b6130 0x7ff70c151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:53:32.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.713+0000 7ff7110ef700 1 --2- 172.21.15.162:0/2965333368 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff70c143730 0x7ff70c154d00 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 rx=0x7ff70c0b8f70 tx=0x7ff704016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:32.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.717+0000 7ff709ffb700 1 -- 172.21.15.162:0/2965333368 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff70400b950 con 0x7ff70c143730 2020-11-28T21:53:32.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.717+0000 7ff712b52700 1 -- 172.21.15.162:0/2965333368 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff70c155ac0 con 0x7ff70c143730 2020-11-28T21:53:32.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.717+0000 7ff709ffb700 1 -- 172.21.15.162:0/2965333368 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7040203f0 con 0x7ff70c143730 2020-11-28T21:53:32.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.717+0000 7ff709ffb700 1 -- 172.21.15.162:0/2965333368 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff704020750 con 0x7ff70c143730 2020-11-28T21:53:32.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.717+0000 7ff712b52700 1 -- 172.21.15.162:0/2965333368 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff70c151c20 con 0x7ff70c143730 2020-11-28T21:53:32.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.717+0000 7ff709ffb700 1 -- 172.21.15.162:0/2965333368 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff70401e070 con 0x7ff70c143730 2020-11-28T21:53:32.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.717+0000 7ff712b52700 1 -- 172.21.15.162:0/2965333368 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff70c0b22b0 con 0x7ff70c143730 2020-11-28T21:53:32.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.717+0000 7ff709ffb700 1 --2- 172.21.15.162:0/2965333368 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff6ec055440 0x7ff6ec057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:32.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.717+0000 7ff7108ee700 1 --2- 172.21.15.162:0/2965333368 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff6ec055440 0x7ff6ec057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:32.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.717+0000 7ff709ffb700 1 -- 172.21.15.162:0/2965333368 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5253+0+0 (secure 0 0 0) 0x7ff70406e7f0 con 0x7ff70c143730 2020-11-28T21:53:32.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.717+0000 7ff7108ee700 1 --2- 172.21.15.162:0/2965333368 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff6ec055440 0x7ff6ec057900 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7ff6fc00b000 tx=0x7ff6fc008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:32.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.725+0000 7ff709ffb700 1 -- 172.21.15.162:0/2965333368 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff70c0b22b0 con 0x7ff70c143730 2020-11-28T21:53:32.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.869+0000 7ff712b52700 1 -- 172.21.15.162:0/2965333368 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff70c0b22b0 con 0x7ff70c143730 2020-11-28T21:53:32.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.869+0000 7ff709ffb700 1 -- 172.21.15.162:0/2965333368 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+13278 (secure 0 0 0) 0x7ff7040313f0 con 0x7ff70c143730 2020-11-28T21:53:32.873 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:53:32.873 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":83,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:53:32.186685+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:53:32.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.873+0000 7ff712b52700 1 -- 172.21.15.162:0/2965333368 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff6ec055440 msgr2=0x7ff6ec057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:32.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.873+0000 7ff712b52700 1 --2- 172.21.15.162:0/2965333368 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff6ec055440 0x7ff6ec057900 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7ff6fc00b000 tx=0x7ff6fc008040).stop 2020-11-28T21:53:32.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.873+0000 7ff712b52700 1 -- 172.21.15.162:0/2965333368 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff70c143730 msgr2=0x7ff70c154d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:32.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.873+0000 7ff712b52700 1 --2- 172.21.15.162:0/2965333368 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff70c143730 0x7ff70c154d00 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 rx=0x7ff70c0b8f70 tx=0x7ff704016000).stop 2020-11-28T21:53:32.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.873+0000 7ff712b52700 1 -- 172.21.15.162:0/2965333368 shutdown_connections 2020-11-28T21:53:32.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.873+0000 7ff712b52700 1 --2- 172.21.15.162:0/2965333368 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff6ec055440 0x7ff6ec057900 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:32.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.873+0000 7ff712b52700 1 --2- 172.21.15.162:0/2965333368 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff70c0608f0 0x7ff70c0b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:32.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.873+0000 7ff712b52700 1 --2- 172.21.15.162:0/2965333368 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff70c143730 0x7ff70c154d00 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:32.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.873+0000 7ff712b52700 1 --2- 172.21.15.162:0/2965333368 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff70c0b6130 0x7ff70c151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:32.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.873+0000 7ff712b52700 1 -- 172.21.15.162:0/2965333368 >> 172.21.15.162:0/2965333368 conn(0x7ff70c066c70 msgr2=0x7ff70c067050 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:32.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.873+0000 7ff712b52700 1 -- 172.21.15.162:0/2965333368 shutdown_connections 2020-11-28T21:53:32.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:32.873+0000 7ff712b52700 1 -- 172.21.15.162:0/2965333368 wait complete. 2020-11-28T21:53:32.888 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-28T21:53:32.889 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:53:33.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.001+0000 7fee4b3f7700 1 Processor -- start 2020-11-28T21:53:33.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.001+0000 7fee4b3f7700 1 -- start start 2020-11-28T21:53:33.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.001+0000 7fee4b3f7700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fee440b18c0 0x7fee440b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:33.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.001+0000 7fee4b3f7700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee440b44e0 0x7fee440b2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:33.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.001+0000 7fee4b3f7700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee440b1f50 con 0x7fee440b3950 2020-11-28T21:53:33.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.001+0000 7fee4b3f7700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee440aab50 con 0x7fee440b18c0 2020-11-28T21:53:33.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.001+0000 7fee4b3f7700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee44143e90 con 0x7fee440b44e0 2020-11-28T21:53:33.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.001+0000 7fee4a3f5700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fee440b3950 0x7fee4413dd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.162:38962/0 (socket says 172.21.15.162:38962) 2020-11-28T21:53:33.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.001+0000 7fee4a3f5700 1 -- 172.21.15.162:0/574915119 learned_addr learned my addr 172.21.15.162:0/574915119 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:33.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.001+0000 7fee49bf4700 1 --2- 172.21.15.162:0/574915119 >> v2:172.21.15.118:3300/0 conn(0x7fee440b18c0 0x7fee440b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:33.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.001+0000 7fee4abf6700 1 --2- 172.21.15.162:0/574915119 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee440b44e0 0x7fee440b2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:33.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.001+0000 7fee4abf6700 1 -- 172.21.15.162:0/574915119 >> v2:172.21.15.118:3300/0 conn(0x7fee440b18c0 msgr2=0x7fee440b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:33.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.001+0000 7fee4abf6700 1 --2- 172.21.15.162:0/574915119 >> v2:172.21.15.118:3300/0 conn(0x7fee440b18c0 0x7fee440b3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:33.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.001+0000 7fee4abf6700 1 -- 172.21.15.162:0/574915119 >> v1:172.21.15.118:6789/0 conn(0x7fee440b3950 legacy=0x7fee4413dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:33.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.001+0000 7fee4abf6700 1 -- 172.21.15.162:0/574915119 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee440aa8b0 con 0x7fee440b44e0 2020-11-28T21:53:33.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.005+0000 7fee4abf6700 1 --2- 172.21.15.162:0/574915119 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee440b44e0 0x7fee440b2c10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fee3c0032b0 tx=0x7fee3c0091a0).ready entity=mon.2 client_cookie=6a8a08126e098260 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:33.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.005+0000 7fee493f3700 1 -- 172.21.15.162:0/574915119 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee3c00b9a0 con 0x7fee440b44e0 2020-11-28T21:53:33.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.005+0000 7fee493f3700 1 -- 172.21.15.162:0/574915119 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee3c00bb00 con 0x7fee440b44e0 2020-11-28T21:53:33.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.005+0000 7fee493f3700 1 -- 172.21.15.162:0/574915119 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee3c00bc70 con 0x7fee440b44e0 2020-11-28T21:53:33.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.005+0000 7fee4b3f7700 1 -- 172.21.15.162:0/574915119 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee440b44e0 msgr2=0x7fee440b2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:33.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.005+0000 7fee4b3f7700 1 --2- 172.21.15.162:0/574915119 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee440b44e0 0x7fee440b2c10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fee3c0032b0 tx=0x7fee3c0091a0).stop 2020-11-28T21:53:33.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.005+0000 7fee4b3f7700 1 -- 172.21.15.162:0/574915119 shutdown_connections 2020-11-28T21:53:33.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.005+0000 7fee4b3f7700 1 --2- 172.21.15.162:0/574915119 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee440b44e0 0x7fee440b2c10 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:33.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.005+0000 7fee4b3f7700 1 --2- 172.21.15.162:0/574915119 >> v2:172.21.15.118:3300/0 conn(0x7fee440b18c0 0x7fee440b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:33.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.005+0000 7fee4b3f7700 1 -- 172.21.15.162:0/574915119 >> 172.21.15.162:0/574915119 conn(0x7fee44062630 msgr2=0x7fee44062a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:33.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.005+0000 7fee4b3f7700 1 -- 172.21.15.162:0/574915119 shutdown_connections 2020-11-28T21:53:33.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.005+0000 7fee4b3f7700 1 -- 172.21.15.162:0/574915119 wait complete. 2020-11-28T21:53:33.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.009+0000 7fee4b3f7700 1 Processor -- start 2020-11-28T21:53:33.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.009+0000 7fee4b3f7700 1 -- start start 2020-11-28T21:53:33.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.009+0000 7fee4b3f7700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee440b18c0 0x7fee440afa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:33.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.009+0000 7fee4b3f7700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee440b3950 0x7fee44152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:33.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.009+0000 7fee4b3f7700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee440b44e0 0x7fee44152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:33.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.009+0000 7fee4b3f7700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee44152b00 con 0x7fee440b3950 2020-11-28T21:53:33.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.009+0000 7fee4b3f7700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee440b0360 con 0x7fee440b18c0 2020-11-28T21:53:33.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.009+0000 7fee4b3f7700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee440b0600 con 0x7fee440b44e0 2020-11-28T21:53:33.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.009+0000 7fee4a3f5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee440b18c0 0x7fee440afa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:33.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.009+0000 7fee49bf4700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee440b3950 0x7fee44152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:33.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.009+0000 7fee4a3f5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee440b18c0 0x7fee440afa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41978/0 (socket says 172.21.15.162:41978) 2020-11-28T21:53:33.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.009+0000 7fee4a3f5700 1 -- 172.21.15.162:0/366656306 learned_addr learned my addr 172.21.15.162:0/366656306 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:33.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.009+0000 7fee4abf6700 1 --2- 172.21.15.162:0/366656306 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee440b44e0 0x7fee44152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:33.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.009+0000 7fee49bf4700 1 -- 172.21.15.162:0/366656306 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee440b44e0 msgr2=0x7fee44152610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:33.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.009+0000 7fee49bf4700 1 --2- 172.21.15.162:0/366656306 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee440b44e0 0x7fee44152610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:33.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.009+0000 7fee49bf4700 1 -- 172.21.15.162:0/366656306 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee440b18c0 msgr2=0x7fee440afa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:33.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.009+0000 7fee49bf4700 1 --2- 172.21.15.162:0/366656306 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee440b18c0 0x7fee440afa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:33.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.009+0000 7fee49bf4700 1 -- 172.21.15.162:0/366656306 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee44151c80 con 0x7fee440b3950 2020-11-28T21:53:33.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.009+0000 7fee49bf4700 1 --2- 172.21.15.162:0/366656306 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee440b3950 0x7fee44152120 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7fee3400c000 tx=0x7fee34006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:33.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.013+0000 7fee337fe700 1 -- 172.21.15.162:0/366656306 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee3400ca10 con 0x7fee440b3950 2020-11-28T21:53:33.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.013+0000 7fee4b3f7700 1 -- 172.21.15.162:0/366656306 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee44060f00 con 0x7fee440b3950 2020-11-28T21:53:33.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.013+0000 7fee337fe700 1 -- 172.21.15.162:0/366656306 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee3400cb70 con 0x7fee440b3950 2020-11-28T21:53:33.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.013+0000 7fee337fe700 1 -- 172.21.15.162:0/366656306 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee3400be40 con 0x7fee440b3950 2020-11-28T21:53:33.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.013+0000 7fee4b3f7700 1 -- 172.21.15.162:0/366656306 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee44141940 con 0x7fee440b3950 2020-11-28T21:53:33.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.013+0000 7fee337fe700 1 -- 172.21.15.162:0/366656306 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fee3401c460 con 0x7fee440b3950 2020-11-28T21:53:33.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.013+0000 7fee4b3f7700 1 -- 172.21.15.162:0/366656306 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee44151e10 con 0x7fee440b3950 2020-11-28T21:53:33.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.013+0000 7fee337fe700 1 --2- 172.21.15.162:0/366656306 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fee24055350 0x7fee24057810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:33.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.013+0000 7fee337fe700 1 -- 172.21.15.162:0/366656306 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fee3406df60 con 0x7fee440b3950 2020-11-28T21:53:33.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.013+0000 7fee4a3f5700 1 --2- 172.21.15.162:0/366656306 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fee24055350 0x7fee24057810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:33.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.013+0000 7fee4a3f5700 1 --2- 172.21.15.162:0/366656306 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fee24055350 0x7fee24057810 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7fee4000d000 tx=0x7fee40009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:33.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.025+0000 7fee337fe700 1 -- 172.21.15.162:0/366656306 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fee34044490 con 0x7fee440b3950 2020-11-28T21:53:33.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.165+0000 7fee4b3f7700 1 -- 172.21.15.162:0/366656306 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fee44057050 con 0x7fee440b3950 2020-11-28T21:53:33.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.165+0000 7fee337fe700 1 -- 172.21.15.162:0/366656306 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v83) v1 ==== 110+0+10 (secure 0 0 0) 0x7fee34047ab0 con 0x7fee440b3950 2020-11-28T21:53:33.167 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:53:33.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.165+0000 7fee4b3f7700 1 -- 172.21.15.162:0/366656306 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fee24055350 msgr2=0x7fee24057810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:33.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.165+0000 7fee4b3f7700 1 --2- 172.21.15.162:0/366656306 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fee24055350 0x7fee24057810 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7fee4000d000 tx=0x7fee40009040).stop 2020-11-28T21:53:33.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.165+0000 7fee4b3f7700 1 -- 172.21.15.162:0/366656306 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee440b3950 msgr2=0x7fee44152120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:33.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.165+0000 7fee4b3f7700 1 --2- 172.21.15.162:0/366656306 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee440b3950 0x7fee44152120 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7fee3400c000 tx=0x7fee34006dc0).stop 2020-11-28T21:53:33.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.165+0000 7fee4b3f7700 1 -- 172.21.15.162:0/366656306 shutdown_connections 2020-11-28T21:53:33.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.165+0000 7fee4b3f7700 1 --2- 172.21.15.162:0/366656306 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fee24055350 0x7fee24057810 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:33.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.165+0000 7fee4b3f7700 1 --2- 172.21.15.162:0/366656306 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee440b44e0 0x7fee44152610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:33.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.165+0000 7fee4b3f7700 1 --2- 172.21.15.162:0/366656306 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee440b18c0 0x7fee440afa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:33.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.165+0000 7fee4b3f7700 1 --2- 172.21.15.162:0/366656306 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee440b3950 0x7fee44152120 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:33.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.165+0000 7fee4b3f7700 1 -- 172.21.15.162:0/366656306 >> 172.21.15.162:0/366656306 conn(0x7fee44062630 msgr2=0x7fee44142d50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:33.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.165+0000 7fee4b3f7700 1 -- 172.21.15.162:0/366656306 shutdown_connections 2020-11-28T21:53:33.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.165+0000 7fee4b3f7700 1 -- 172.21.15.162:0/366656306 wait complete. 2020-11-28T21:53:33.181 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:53:33.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.293+0000 7f9580c04700 1 Processor -- start 2020-11-28T21:53:33.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.293+0000 7f9580c04700 1 -- start start 2020-11-28T21:53:33.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.293+0000 7f9580c04700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f957c0b3c90 0x7f957c142a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:33.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.293+0000 7f9580c04700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f957c0aac00 con 0x7f957c0b18c0 2020-11-28T21:53:33.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.293+0000 7f9580c04700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f957c060a10 con 0x7f957c05f380 2020-11-28T21:53:33.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.293+0000 7f9580c04700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f957c13cf80 con 0x7f957c0b3c90 2020-11-28T21:53:33.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.293+0000 7f957bfff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f957c05f380 0x7f957c146050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:48830/0 (socket says 172.21.15.162:48830) 2020-11-28T21:53:33.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.293+0000 7f957bfff700 1 -- 172.21.15.162:0/3298242443 learned_addr learned my addr 172.21.15.162:0/3298242443 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:33.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.293+0000 7f957b7fe700 1 --2- 172.21.15.162:0/3298242443 >> v2:172.21.15.118:3300/0 conn(0x7f957c0b3c90 0x7f957c142a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:33.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.297+0000 7f957a7fc700 1 -- 172.21.15.162:0/3298242443 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1635025751 0 0) 0x7f957c060a10 con 0x7f957c05f380 2020-11-28T21:53:33.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.297+0000 7f957a7fc700 1 -- 172.21.15.162:0/3298242443 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9568003230 con 0x7f957c05f380 2020-11-28T21:53:33.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.297+0000 7f957b7fe700 1 -- 172.21.15.162:0/3298242443 >> v1:172.21.15.162:6789/0 conn(0x7f957c05f380 legacy=0x7f957c146050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:33.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.297+0000 7f957b7fe700 1 -- 172.21.15.162:0/3298242443 >> v1:172.21.15.118:6789/0 conn(0x7f957c0b18c0 legacy=0x7f957c142f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:33.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.297+0000 7f957b7fe700 1 -- 172.21.15.162:0/3298242443 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f957c0b4070 con 0x7f957c0b3c90 2020-11-28T21:53:33.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.297+0000 7f957b7fe700 1 --2- 172.21.15.162:0/3298242443 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f957c0b3c90 0x7f957c142a80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f956c002f30 tx=0x7f956c006dc0).ready entity=mon.0 client_cookie=213ef75554c4a2f8 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:33.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.297+0000 7f957a7fc700 1 -- 172.21.15.162:0/3298242443 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f956c009800 con 0x7f957c0b3c90 2020-11-28T21:53:33.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.297+0000 7f957a7fc700 1 -- 172.21.15.162:0/3298242443 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f956c009960 con 0x7f957c0b3c90 2020-11-28T21:53:33.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.297+0000 7f957a7fc700 1 -- 172.21.15.162:0/3298242443 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f956c009cc0 con 0x7f957c0b3c90 2020-11-28T21:53:33.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.297+0000 7f9580c04700 1 -- 172.21.15.162:0/3298242443 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f957c0b3c90 msgr2=0x7f957c142a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:33.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.297+0000 7f9580c04700 1 --2- 172.21.15.162:0/3298242443 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f957c0b3c90 0x7f957c142a80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f956c002f30 tx=0x7f956c006dc0).stop 2020-11-28T21:53:33.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.297+0000 7f9580c04700 1 -- 172.21.15.162:0/3298242443 shutdown_connections 2020-11-28T21:53:33.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.297+0000 7f9580c04700 1 --2- 172.21.15.162:0/3298242443 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f957c0b3c90 0x7f957c142a80 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:33.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.297+0000 7f9580c04700 1 -- 172.21.15.162:0/3298242443 >> 172.21.15.162:0/3298242443 conn(0x7f957c062630 msgr2=0x7f957c062a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:33.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.297+0000 7f9580c04700 1 -- 172.21.15.162:0/3298242443 shutdown_connections 2020-11-28T21:53:33.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.297+0000 7f9580c04700 1 -- 172.21.15.162:0/3298242443 wait complete. 2020-11-28T21:53:33.307 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.297+0000 7f9580c04700 1 Processor -- start 2020-11-28T21:53:33.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.297+0000 7f9580c04700 1 -- start start 2020-11-28T21:53:33.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.297+0000 7f9580c04700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f957c05f380 0x7f957c13f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:33.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.297+0000 7f9580c04700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f957c0b18c0 0x7f957c066600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:33.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.297+0000 7f9580c04700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f957c0b3c90 0x7f957c144480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:33.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.297+0000 7f9580c04700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957c15a8a0 con 0x7f957c0b3c90 2020-11-28T21:53:33.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.297+0000 7f9580c04700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957c146ca0 con 0x7f957c05f380 2020-11-28T21:53:33.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.297+0000 7f9580c04700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f957c15a360 con 0x7f957c0b18c0 2020-11-28T21:53:33.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.297+0000 7f957b7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f957c05f380 0x7f957c13f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:33.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.297+0000 7f957b7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f957c05f380 0x7f957c13f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:41992/0 (socket says 172.21.15.162:41992) 2020-11-28T21:53:33.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.297+0000 7f957b7fe700 1 -- 172.21.15.162:0/2466166386 learned_addr learned my addr 172.21.15.162:0/2466166386 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:33.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.297+0000 7f957bfff700 1 --2- 172.21.15.162:0/2466166386 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f957c0b3c90 0x7f957c144480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:33.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.297+0000 7f957affd700 1 --2- 172.21.15.162:0/2466166386 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f957c0b18c0 0x7f957c066600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:33.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.297+0000 7f957b7fe700 1 -- 172.21.15.162:0/2466166386 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f957c0b18c0 msgr2=0x7f957c066600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:33.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.297+0000 7f957b7fe700 1 --2- 172.21.15.162:0/2466166386 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f957c0b18c0 0x7f957c066600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:33.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.297+0000 7f957b7fe700 1 -- 172.21.15.162:0/2466166386 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f957c0b3c90 msgr2=0x7f957c144480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:33.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.297+0000 7f957b7fe700 1 --2- 172.21.15.162:0/2466166386 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f957c0b3c90 0x7f957c144480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:33.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.297+0000 7f957b7fe700 1 -- 172.21.15.162:0/2466166386 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f957c15a600 con 0x7f957c05f380 2020-11-28T21:53:33.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.297+0000 7f957bfff700 1 --2- 172.21.15.162:0/2466166386 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f957c0b3c90 0x7f957c144480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:53:33.318 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.297+0000 7f957affd700 1 --2- 172.21.15.162:0/2466166386 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f957c0b18c0 0x7f957c066600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:53:33.318 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.301+0000 7f957b7fe700 1 --2- 172.21.15.162:0/2466166386 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f957c05f380 0x7f957c13f150 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 rx=0x7f956c002f30 tx=0x7f956c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:33.318 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.301+0000 7f9578ff9700 1 -- 172.21.15.162:0/2466166386 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f956c011ce0 con 0x7f957c05f380 2020-11-28T21:53:33.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.301+0000 7f9580c04700 1 -- 172.21.15.162:0/2466166386 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f957c146920 con 0x7f957c05f380 2020-11-28T21:53:33.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.301+0000 7f9578ff9700 1 -- 172.21.15.162:0/2466166386 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f956c011e40 con 0x7f957c05f380 2020-11-28T21:53:33.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.301+0000 7f9580c04700 1 -- 172.21.15.162:0/2466166386 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f957c0b4830 con 0x7f957c05f380 2020-11-28T21:53:33.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.301+0000 7f9578ff9700 1 -- 172.21.15.162:0/2466166386 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f956c027630 con 0x7f957c05f380 2020-11-28T21:53:33.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.301+0000 7f9580c04700 1 -- 172.21.15.162:0/2466166386 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f957c057050 con 0x7f957c05f380 2020-11-28T21:53:33.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.301+0000 7f9578ff9700 1 -- 172.21.15.162:0/2466166386 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f956c024030 con 0x7f957c05f380 2020-11-28T21:53:33.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.301+0000 7f9578ff9700 1 --2- 172.21.15.162:0/2466166386 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f955c055440 0x7f955c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:33.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.301+0000 7f9578ff9700 1 -- 172.21.15.162:0/2466166386 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f956c06ed40 con 0x7f957c05f380 2020-11-28T21:53:33.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.301+0000 7f957affd700 1 --2- 172.21.15.162:0/2466166386 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f955c055440 0x7f955c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:33.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.301+0000 7f957affd700 1 --2- 172.21.15.162:0/2466166386 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f955c055440 0x7f955c057900 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7f9570008000 tx=0x7f9570009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:33.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.313+0000 7f9578ff9700 1 -- 172.21.15.162:0/2466166386 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f956c0452d0 con 0x7f957c05f380 2020-11-28T21:53:33.455 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.453+0000 7f9580c04700 1 -- 172.21.15.162:0/2466166386 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f957c0b1d30 con 0x7f957c05f380 2020-11-28T21:53:33.456 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.453+0000 7f9578ff9700 1 -- 172.21.15.162:0/2466166386 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v83) v1 ==== 92+0+10 (secure 0 0 0) 0x7f956c022070 con 0x7f957c05f380 2020-11-28T21:53:33.456 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:53:33.457 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.453+0000 7f9580c04700 1 -- 172.21.15.162:0/2466166386 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f955c055440 msgr2=0x7f955c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:33.458 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.453+0000 7f9580c04700 1 --2- 172.21.15.162:0/2466166386 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f955c055440 0x7f955c057900 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7f9570008000 tx=0x7f9570009040).stop 2020-11-28T21:53:33.458 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.453+0000 7f9580c04700 1 -- 172.21.15.162:0/2466166386 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f957c05f380 msgr2=0x7f957c13f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:33.458 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.453+0000 7f9580c04700 1 --2- 172.21.15.162:0/2466166386 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f957c05f380 0x7f957c13f150 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 rx=0x7f956c002f30 tx=0x7f956c015000).stop 2020-11-28T21:53:33.459 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.453+0000 7f9580c04700 1 -- 172.21.15.162:0/2466166386 shutdown_connections 2020-11-28T21:53:33.459 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.453+0000 7f9580c04700 1 --2- 172.21.15.162:0/2466166386 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f955c055440 0x7f955c057900 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:33.459 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.453+0000 7f9580c04700 1 --2- 172.21.15.162:0/2466166386 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f957c0b3c90 0x7f957c144480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:33.460 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.453+0000 7f9580c04700 1 --2- 172.21.15.162:0/2466166386 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f957c05f380 0x7f957c13f150 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:33.460 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.453+0000 7f9580c04700 1 --2- 172.21.15.162:0/2466166386 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f957c0b18c0 0x7f957c066600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:33.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.453+0000 7f9580c04700 1 -- 172.21.15.162:0/2466166386 >> 172.21.15.162:0/2466166386 conn(0x7f957c062630 msgr2=0x7f957c0b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:33.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.453+0000 7f9580c04700 1 -- 172.21.15.162:0/2466166386 shutdown_connections 2020-11-28T21:53:33.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.453+0000 7f9580c04700 1 -- 172.21.15.162:0/2466166386 wait complete. 2020-11-28T21:53:33.469 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2020-11-28T21:53:33.578 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.577+0000 7fe22359e700 1 Processor -- start 2020-11-28T21:53:33.579 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.577+0000 7fe22359e700 1 -- start start 2020-11-28T21:53:33.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.577+0000 7fe22359e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe22413f6c0 0x7fe2240669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:33.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.577+0000 7fe22359e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe224065750 0x7fe22413c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:33.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.577+0000 7fe22359e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe224065050 con 0x7fe224064c70 2020-11-28T21:53:33.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.577+0000 7fe22359e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe22413fc40 con 0x7fe22413f6c0 2020-11-28T21:53:33.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.577+0000 7fe22359e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe22413fee0 con 0x7fe224065750 2020-11-28T21:53:33.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.577+0000 7fe22259c700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fe224064c70 0x7fe224066290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:48842/0 (socket says 172.21.15.162:48842) 2020-11-28T21:53:33.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.577+0000 7fe22259c700 1 -- 172.21.15.162:0/2535422592 learned_addr learned my addr 172.21.15.162:0/2535422592 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:33.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.577+0000 7fe221d9b700 1 --2- 172.21.15.162:0/2535422592 >> v2:172.21.15.118:3300/0 conn(0x7fe22413f6c0 0x7fe2240669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:33.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.577+0000 7fe222d9d700 1 --2- 172.21.15.162:0/2535422592 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe224065750 0x7fe22413c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:33.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.577+0000 7fe22159a700 1 -- 172.21.15.162:0/2535422592 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3367134992 0 0) 0x7fe224065050 con 0x7fe224064c70 2020-11-28T21:53:33.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.577+0000 7fe22159a700 1 -- 172.21.15.162:0/2535422592 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe208003230 con 0x7fe224064c70 2020-11-28T21:53:33.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.577+0000 7fe22159a700 1 -- 172.21.15.162:0/2535422592 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2531675447 0 0) 0x7fe208003230 con 0x7fe224064c70 2020-11-28T21:53:33.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.577+0000 7fe221d9b700 1 -- 172.21.15.162:0/2535422592 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe224065750 msgr2=0x7fe22413c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:33.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.577+0000 7fe221d9b700 1 --2- 172.21.15.162:0/2535422592 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe224065750 0x7fe22413c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:33.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.577+0000 7fe221d9b700 1 -- 172.21.15.162:0/2535422592 >> v1:172.21.15.162:6789/0 conn(0x7fe224064c70 legacy=0x7fe224066290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:33.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.577+0000 7fe221d9b700 1 -- 172.21.15.162:0/2535422592 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe224140180 con 0x7fe22413f6c0 2020-11-28T21:53:33.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.577+0000 7fe221d9b700 1 --2- 172.21.15.162:0/2535422592 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe22413f6c0 0x7fe2240669a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7fe214001870 tx=0x7fe218006dc0).ready entity=mon.0 client_cookie=d67d4ab21d7419f4 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:33.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.577+0000 7fe22159a700 1 -- 172.21.15.162:0/2535422592 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2140016d0 con 0x7fe22413f6c0 2020-11-28T21:53:33.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.577+0000 7fe22159a700 1 -- 172.21.15.162:0/2535422592 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe21800d570 con 0x7fe22413f6c0 2020-11-28T21:53:33.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.577+0000 7fe22359e700 1 -- 172.21.15.162:0/2535422592 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe22413f6c0 msgr2=0x7fe2240669a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:33.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.577+0000 7fe22159a700 1 -- 172.21.15.162:0/2535422592 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe21800d6e0 con 0x7fe22413f6c0 2020-11-28T21:53:33.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.577+0000 7fe22359e700 1 --2- 172.21.15.162:0/2535422592 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe22413f6c0 0x7fe2240669a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7fe214001870 tx=0x7fe218006dc0).stop 2020-11-28T21:53:33.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.577+0000 7fe22359e700 1 -- 172.21.15.162:0/2535422592 shutdown_connections 2020-11-28T21:53:33.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.577+0000 7fe22359e700 1 --2- 172.21.15.162:0/2535422592 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe224065750 0x7fe22413c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:33.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.577+0000 7fe22359e700 1 --2- 172.21.15.162:0/2535422592 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe22413f6c0 0x7fe2240669a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:33.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.577+0000 7fe22359e700 1 -- 172.21.15.162:0/2535422592 >> 172.21.15.162:0/2535422592 conn(0x7fe2240b5da0 msgr2=0x7fe2240b6180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:33.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.577+0000 7fe22359e700 1 -- 172.21.15.162:0/2535422592 shutdown_connections 2020-11-28T21:53:33.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.577+0000 7fe22359e700 1 -- 172.21.15.162:0/2535422592 wait complete. 2020-11-28T21:53:33.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.581+0000 7fe22359e700 1 Processor -- start 2020-11-28T21:53:33.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.581+0000 7fe22359e700 1 -- start start 2020-11-28T21:53:33.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.581+0000 7fe22359e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe224064c70 0x7fe224132ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:33.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.581+0000 7fe22359e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe224065750 0x7fe2240ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:33.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.581+0000 7fe22359e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe22413f6c0 0x7fe2240aed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:33.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.581+0000 7fe22359e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2240af280 con 0x7fe224065750 2020-11-28T21:53:33.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.581+0000 7fe22359e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2240ad170 con 0x7fe22413f6c0 2020-11-28T21:53:33.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.581+0000 7fe22359e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe22412d540 con 0x7fe224064c70 2020-11-28T21:53:33.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.581+0000 7fe222d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe22413f6c0 0x7fe2240aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:33.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.581+0000 7fe222d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe22413f6c0 0x7fe2240aed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42008/0 (socket says 172.21.15.162:42008) 2020-11-28T21:53:33.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.581+0000 7fe222d9d700 1 -- 172.21.15.162:0/2088799489 learned_addr learned my addr 172.21.15.162:0/2088799489 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:33.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.581+0000 7fe22259c700 1 --2- 172.21.15.162:0/2088799489 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe224064c70 0x7fe224132ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:33.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.581+0000 7fe221d9b700 1 --2- 172.21.15.162:0/2088799489 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe224065750 0x7fe2240ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:33.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.581+0000 7fe222d9d700 1 -- 172.21.15.162:0/2088799489 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe224064c70 msgr2=0x7fe224132ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:33.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.581+0000 7fe222d9d700 1 --2- 172.21.15.162:0/2088799489 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe224064c70 0x7fe224132ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:33.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.581+0000 7fe222d9d700 1 -- 172.21.15.162:0/2088799489 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe224065750 msgr2=0x7fe2240ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:33.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.581+0000 7fe222d9d700 1 --2- 172.21.15.162:0/2088799489 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe224065750 0x7fe2240ae8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:33.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.581+0000 7fe222d9d700 1 -- 172.21.15.162:0/2088799489 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe22413ec20 con 0x7fe22413f6c0 2020-11-28T21:53:33.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.581+0000 7fe222d9d700 1 --2- 172.21.15.162:0/2088799489 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe22413f6c0 0x7fe2240aed90 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 rx=0x7fe2240af3c0 tx=0x7fe21c0091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:33.594 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.581+0000 7fe2137fe700 1 -- 172.21.15.162:0/2088799489 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe21c00ee60 con 0x7fe22413f6c0 2020-11-28T21:53:33.594 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.581+0000 7fe2137fe700 1 -- 172.21.15.162:0/2088799489 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe21c00cbb0 con 0x7fe22413f6c0 2020-11-28T21:53:33.594 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.581+0000 7fe22359e700 1 -- 172.21.15.162:0/2088799489 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2240b45d0 con 0x7fe22413f6c0 2020-11-28T21:53:33.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.581+0000 7fe2137fe700 1 -- 172.21.15.162:0/2088799489 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe21c0123f0 con 0x7fe22413f6c0 2020-11-28T21:53:33.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.581+0000 7fe22359e700 1 -- 172.21.15.162:0/2088799489 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2240ad3a0 con 0x7fe22413f6c0 2020-11-28T21:53:33.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.585+0000 7fe22359e700 1 -- 172.21.15.162:0/2088799489 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe224057050 con 0x7fe22413f6c0 2020-11-28T21:53:33.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.585+0000 7fe2137fe700 1 -- 172.21.15.162:0/2088799489 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe21c022070 con 0x7fe22413f6c0 2020-11-28T21:53:33.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.585+0000 7fe2137fe700 1 --2- 172.21.15.162:0/2088799489 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe200055500 0x7fe2000579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:33.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.585+0000 7fe2137fe700 1 -- 172.21.15.162:0/2088799489 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fe21c06e320 con 0x7fe22413f6c0 2020-11-28T21:53:33.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.585+0000 7fe22259c700 1 --2- 172.21.15.162:0/2088799489 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe200055500 0x7fe2000579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:33.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.585+0000 7fe22259c700 1 --2- 172.21.15.162:0/2088799489 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe200055500 0x7fe2000579c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7fe21400d000 tx=0x7fe214000b80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:33.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.593+0000 7fe2137fe700 1 -- 172.21.15.162:0/2088799489 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe21c044950 con 0x7fe22413f6c0 2020-11-28T21:53:33.738 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.733+0000 7fe22359e700 1 -- 172.21.15.162:0/2088799489 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe224063480 con 0x7fe22413f6c0 2020-11-28T21:53:33.738 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.737+0000 7fe2137fe700 1 -- 172.21.15.162:0/2088799489 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v83) v1 ==== 84+0+143 (secure 0 0 0) 0x7fe21c016020 con 0x7fe22413f6c0 2020-11-28T21:53:33.739 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:53:33.739 INFO:teuthology.orchestra.run.smithi162.stdout:[ 2020-11-28T21:53:33.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:33.739 INFO:teuthology.orchestra.run.smithi162.stdout: "poolnum": 1, 2020-11-28T21:53:33.740 INFO:teuthology.orchestra.run.smithi162.stdout: "poolname": "device_health_metrics" 2020-11-28T21:53:33.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:33.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:33.740 INFO:teuthology.orchestra.run.smithi162.stdout: "poolnum": 2, 2020-11-28T21:53:33.741 INFO:teuthology.orchestra.run.smithi162.stdout: "poolname": "rbd" 2020-11-28T21:53:33.741 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:33.741 INFO:teuthology.orchestra.run.smithi162.stdout:] 2020-11-28T21:53:33.742 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.737+0000 7fe22359e700 1 -- 172.21.15.162:0/2088799489 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe200055500 msgr2=0x7fe2000579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:33.743 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.737+0000 7fe22359e700 1 --2- 172.21.15.162:0/2088799489 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe200055500 0x7fe2000579c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7fe21400d000 tx=0x7fe214000b80).stop 2020-11-28T21:53:33.743 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.737+0000 7fe22359e700 1 -- 172.21.15.162:0/2088799489 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe22413f6c0 msgr2=0x7fe2240aed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:33.743 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.737+0000 7fe22359e700 1 --2- 172.21.15.162:0/2088799489 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe22413f6c0 0x7fe2240aed90 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 rx=0x7fe2240af3c0 tx=0x7fe21c0091a0).stop 2020-11-28T21:53:33.743 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.737+0000 7fe22359e700 1 -- 172.21.15.162:0/2088799489 shutdown_connections 2020-11-28T21:53:33.744 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.737+0000 7fe22359e700 1 --2- 172.21.15.162:0/2088799489 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe200055500 0x7fe2000579c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:33.744 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.737+0000 7fe22359e700 1 --2- 172.21.15.162:0/2088799489 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe224064c70 0x7fe224132ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:33.744 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.737+0000 7fe22359e700 1 --2- 172.21.15.162:0/2088799489 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe22413f6c0 0x7fe2240aed90 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:33.745 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.737+0000 7fe22359e700 1 --2- 172.21.15.162:0/2088799489 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe224065750 0x7fe2240ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:33.745 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.737+0000 7fe22359e700 1 -- 172.21.15.162:0/2088799489 >> 172.21.15.162:0/2088799489 conn(0x7fe2240b5da0 msgr2=0x7fe224061d70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:33.745 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.737+0000 7fe22359e700 1 -- 172.21.15.162:0/2088799489 shutdown_connections 2020-11-28T21:53:33.745 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.737+0000 7fe22359e700 1 -- 172.21.15.162:0/2088799489 wait complete. 2020-11-28T21:53:33.752 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-11-28T21:53:33.753 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2020-11-28T21:53:33.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.869+0000 7f912b3d7700 1 Processor -- start 2020-11-28T21:53:33.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.869+0000 7f912b3d7700 1 -- start start 2020-11-28T21:53:33.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.869+0000 7f912b3d7700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f91240b2d00 0x7f912405c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:33.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f912b3d7700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f912405baf0 con 0x7f91240b30e0 2020-11-28T21:53:33.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f912b3d7700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f912405bd90 con 0x7f91240b3ab0 2020-11-28T21:53:33.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f912b3d7700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91240b0e20 con 0x7f91240b2d00 2020-11-28T21:53:33.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f9129bd4700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f91240b3ab0 0x7f9124142ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:48858/0 (socket says 172.21.15.162:48858) 2020-11-28T21:53:33.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f9129bd4700 1 -- 172.21.15.162:0/435976103 learned_addr learned my addr 172.21.15.162:0/435976103 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:33.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f912a3d5700 1 --2- 172.21.15.162:0/435976103 >> v2:172.21.15.118:3300/0 conn(0x7f91240b2d00 0x7f912405c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:33.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f91293d3700 1 -- 172.21.15.162:0/435976103 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3225889308 0 0) 0x7f912405bd90 con 0x7f91240b3ab0 2020-11-28T21:53:33.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f91293d3700 1 -- 172.21.15.162:0/435976103 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f910c003230 con 0x7f91240b3ab0 2020-11-28T21:53:33.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f91293d3700 1 -- 172.21.15.162:0/435976103 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 917639032 0 0) 0x7f912405baf0 con 0x7f91240b30e0 2020-11-28T21:53:33.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f91293d3700 1 -- 172.21.15.162:0/435976103 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f912405bd90 con 0x7f91240b30e0 2020-11-28T21:53:33.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f91293d3700 1 -- 172.21.15.162:0/435976103 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3149147255 0 0) 0x7f910c003230 con 0x7f91240b3ab0 2020-11-28T21:53:33.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f91293d3700 1 -- 172.21.15.162:0/435976103 >> v2:172.21.15.118:3300/0 conn(0x7f91240b2d00 msgr2=0x7f912405c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:33.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f91293d3700 1 --2- 172.21.15.162:0/435976103 >> v2:172.21.15.118:3300/0 conn(0x7f91240b2d00 0x7f912405c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:33.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f91293d3700 1 -- 172.21.15.162:0/435976103 >> v1:172.21.15.118:6789/0 conn(0x7f91240b30e0 legacy=0x7f912413fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:33.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f91293d3700 1 -- 172.21.15.162:0/435976103 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f912405c030 con 0x7f91240b3ab0 2020-11-28T21:53:33.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f91293d3700 1 -- 172.21.15.162:0/435976103 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f91200016d0 con 0x7f91240b3ab0 2020-11-28T21:53:33.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f91293d3700 1 -- 172.21.15.162:0/435976103 >> v1:172.21.15.162:6789/0 conn(0x7f91240b3ab0 legacy=0x7f9124142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:33.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f91293d3700 1 --2- 172.21.15.162:0/435976103 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f910c0038b0 0x7f910c003cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:33.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f91293d3700 1 --2- 172.21.15.162:0/435976103 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f910c0044d0 0x7f910c004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:33.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f91293d3700 1 --2- 172.21.15.162:0/435976103 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f910c004e40 0x7f910c00b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:33.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f91293d3700 1 -- 172.21.15.162:0/435976103 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91240b0e20 con 0x7f910c004e40 2020-11-28T21:53:33.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f91293d3700 1 -- 172.21.15.162:0/435976103 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910c0053a0 con 0x7f910c0044d0 2020-11-28T21:53:33.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f91293d3700 1 -- 172.21.15.162:0/435976103 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f910c00bc90 con 0x7f910c0038b0 2020-11-28T21:53:33.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f9129bd4700 1 --2- 172.21.15.162:0/435976103 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f910c0044d0 0x7f910c004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:33.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f912abd6700 1 --2- 172.21.15.162:0/435976103 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f910c004e40 0x7f910c00b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:33.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f912a3d5700 1 --2- 172.21.15.162:0/435976103 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f910c0038b0 0x7f910c003cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:33.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f9129bd4700 1 -- 172.21.15.162:0/435976103 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f910c0038b0 msgr2=0x7f910c003cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:33.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f9129bd4700 1 --2- 172.21.15.162:0/435976103 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f910c0038b0 0x7f910c003cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:33.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f9129bd4700 1 -- 172.21.15.162:0/435976103 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f910c004e40 msgr2=0x7f910c00b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:33.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f9129bd4700 1 --2- 172.21.15.162:0/435976103 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f910c004e40 0x7f910c00b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:33.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f9129bd4700 1 -- 172.21.15.162:0/435976103 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f910c003230 con 0x7f910c0044d0 2020-11-28T21:53:33.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f912abd6700 1 -- 172.21.15.162:0/435976103 reap_dead start 2020-11-28T21:53:33.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f9129bd4700 1 --2- 172.21.15.162:0/435976103 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f910c0044d0 0x7f910c004950 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7f91200014a0 tx=0x7f9120007740).ready entity=mon.1 client_cookie=b45963f3dfff86b7 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:33.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f91293d3700 1 -- 172.21.15.162:0/435976103 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91200070d0 con 0x7f910c0044d0 2020-11-28T21:53:33.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f91293d3700 1 -- 172.21.15.162:0/435976103 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9120007230 con 0x7f910c0044d0 2020-11-28T21:53:33.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f91293d3700 1 -- 172.21.15.162:0/435976103 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91200055e0 con 0x7f910c0044d0 2020-11-28T21:53:33.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f912b3d7700 1 -- 172.21.15.162:0/435976103 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f910c0044d0 msgr2=0x7f910c004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:33.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f912b3d7700 1 --2- 172.21.15.162:0/435976103 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f910c0044d0 0x7f910c004950 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7f91200014a0 tx=0x7f9120007740).stop 2020-11-28T21:53:33.886 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f912b3d7700 1 -- 172.21.15.162:0/435976103 shutdown_connections 2020-11-28T21:53:33.886 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f912b3d7700 1 --2- 172.21.15.162:0/435976103 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f910c0044d0 0x7f910c004950 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:33.886 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f912b3d7700 1 -- 172.21.15.162:0/435976103 >> 172.21.15.162:0/435976103 conn(0x7f91240604c0 msgr2=0x7f91240608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:33.886 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f912b3d7700 1 -- 172.21.15.162:0/435976103 shutdown_connections 2020-11-28T21:53:33.886 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.873+0000 7f912b3d7700 1 -- 172.21.15.162:0/435976103 wait complete. 2020-11-28T21:53:33.887 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.877+0000 7f912b3d7700 1 Processor -- start 2020-11-28T21:53:33.887 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.877+0000 7f912b3d7700 1 -- start start 2020-11-28T21:53:33.889 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.877+0000 7f912b3d7700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f910c0044d0 0x7f9124151780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:33.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.877+0000 7f912b3d7700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91240afa10 0x7f91241456a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:33.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.877+0000 7f912b3d7700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f91240b8cc0 0x7f91240b9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:33.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.877+0000 7f912b3d7700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9124143ea0 con 0x7f910c0044d0 2020-11-28T21:53:33.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.877+0000 7f912b3d7700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9124151d70 con 0x7f91240afa10 2020-11-28T21:53:33.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.877+0000 7f912b3d7700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9124152010 con 0x7f91240b8cc0 2020-11-28T21:53:33.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.877+0000 7f9129bd4700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91240afa10 0x7f91241456a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:33.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.877+0000 7f9129bd4700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91240afa10 0x7f91241456a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42028/0 (socket says 172.21.15.162:42028) 2020-11-28T21:53:33.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.877+0000 7f9129bd4700 1 -- 172.21.15.162:0/2530011065 learned_addr learned my addr 172.21.15.162:0/2530011065 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:33.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.877+0000 7f912a3d5700 1 --2- 172.21.15.162:0/2530011065 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f910c0044d0 0x7f9124151780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:33.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.877+0000 7f912abd6700 1 --2- 172.21.15.162:0/2530011065 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f91240b8cc0 0x7f91240b9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:33.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.877+0000 7f9129bd4700 1 -- 172.21.15.162:0/2530011065 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f91240b8cc0 msgr2=0x7f91240b9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:33.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.877+0000 7f9129bd4700 1 --2- 172.21.15.162:0/2530011065 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f91240b8cc0 0x7f91240b9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:33.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.877+0000 7f9129bd4700 1 -- 172.21.15.162:0/2530011065 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f910c0044d0 msgr2=0x7f9124151780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:33.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.877+0000 7f9129bd4700 1 --2- 172.21.15.162:0/2530011065 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f910c0044d0 0x7f9124151780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:33.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.877+0000 7f9129bd4700 1 -- 172.21.15.162:0/2530011065 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91241522b0 con 0x7f91240afa10 2020-11-28T21:53:33.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.877+0000 7f912abd6700 1 --2- 172.21.15.162:0/2530011065 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f91240b8cc0 0x7f91240b9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:53:33.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.877+0000 7f9129bd4700 1 --2- 172.21.15.162:0/2530011065 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91240afa10 0x7f91241456a0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7f91200014a0 tx=0x7f9120015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:33.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.877+0000 7f91137fe700 1 -- 172.21.15.162:0/2530011065 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9120005e20 con 0x7f91240afa10 2020-11-28T21:53:33.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.877+0000 7f91137fe700 1 -- 172.21.15.162:0/2530011065 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9120013660 con 0x7f91240afa10 2020-11-28T21:53:33.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.877+0000 7f912b3d7700 1 -- 172.21.15.162:0/2530011065 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9124152550 con 0x7f91240afa10 2020-11-28T21:53:33.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.877+0000 7f91137fe700 1 -- 172.21.15.162:0/2530011065 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91200139c0 con 0x7f91240afa10 2020-11-28T21:53:33.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.877+0000 7f912b3d7700 1 -- 172.21.15.162:0/2530011065 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91240b9880 con 0x7f91240afa10 2020-11-28T21:53:33.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.881+0000 7f912b3d7700 1 -- 172.21.15.162:0/2530011065 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91241526e0 con 0x7f91240afa10 2020-11-28T21:53:33.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.881+0000 7f91137fe700 1 -- 172.21.15.162:0/2530011065 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9120010070 con 0x7f91240afa10 2020-11-28T21:53:33.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.881+0000 7f91137fe700 1 --2- 172.21.15.162:0/2530011065 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f91040553b0 0x7f9104057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:33.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.881+0000 7f91137fe700 1 -- 172.21.15.162:0/2530011065 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f9120019030 con 0x7f91240afa10 2020-11-28T21:53:33.896 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.881+0000 7f912a3d5700 1 --2- 172.21.15.162:0/2530011065 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f91040553b0 0x7f9104057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:33.896 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.881+0000 7f912a3d5700 1 --2- 172.21.15.162:0/2530011065 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f91040553b0 0x7f9104057870 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7f9114008000 tx=0x7f9114008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:33.896 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:33.889+0000 7f91137fe700 1 -- 172.21.15.162:0/2530011065 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9120045250 con 0x7f91240afa10 2020-11-28T21:53:34.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:34.029+0000 7f912b3d7700 1 -- 172.21.15.162:0/2530011065 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f9124057050 con 0x7f91240afa10 2020-11-28T21:53:35.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.028+0000 7f91137fe700 1 -- 172.21.15.162:0/2530011065 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v84) v1 ==== 133+0+0 (secure 0 0 0) 0x7f9120048870 con 0x7f91240afa10 2020-11-28T21:53:35.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.032+0000 7f912b3d7700 1 -- 172.21.15.162:0/2530011065 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f91040553b0 msgr2=0x7f9104057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:35.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.032+0000 7f912b3d7700 1 --2- 172.21.15.162:0/2530011065 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f91040553b0 0x7f9104057870 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7f9114008000 tx=0x7f9114008040).stop 2020-11-28T21:53:35.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.032+0000 7f912b3d7700 1 -- 172.21.15.162:0/2530011065 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91240afa10 msgr2=0x7f91241456a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:35.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.032+0000 7f912b3d7700 1 --2- 172.21.15.162:0/2530011065 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91240afa10 0x7f91241456a0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7f91200014a0 tx=0x7f9120015000).stop 2020-11-28T21:53:35.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.032+0000 7f912b3d7700 1 -- 172.21.15.162:0/2530011065 shutdown_connections 2020-11-28T21:53:35.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.032+0000 7f912b3d7700 1 --2- 172.21.15.162:0/2530011065 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f91040553b0 0x7f9104057870 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:35.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.032+0000 7f912b3d7700 1 --2- 172.21.15.162:0/2530011065 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f91240b8cc0 0x7f91240b9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:35.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.032+0000 7f912b3d7700 1 --2- 172.21.15.162:0/2530011065 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f910c0044d0 0x7f9124151780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:35.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.032+0000 7f912b3d7700 1 --2- 172.21.15.162:0/2530011065 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91240afa10 0x7f91241456a0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:35.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.032+0000 7f912b3d7700 1 -- 172.21.15.162:0/2530011065 >> 172.21.15.162:0/2530011065 conn(0x7f91240604c0 msgr2=0x7f912413be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:35.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.032+0000 7f912b3d7700 1 -- 172.21.15.162:0/2530011065 shutdown_connections 2020-11-28T21:53:35.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.032+0000 7f912b3d7700 1 -- 172.21.15.162:0/2530011065 wait complete. 2020-11-28T21:53:35.038 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_metadata' created 2020-11-28T21:53:35.050 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2020-11-28T21:53:35.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.164+0000 7f40d597b700 1 Processor -- start 2020-11-28T21:53:35.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.164+0000 7f40d597b700 1 -- start start 2020-11-28T21:53:35.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.164+0000 7f40d597b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f40d00b44e0 0x7f40d00b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:35.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.164+0000 7f40d597b700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40d00b1f50 con 0x7f40d00b3950 2020-11-28T21:53:35.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.164+0000 7f40d597b700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40d0143e90 con 0x7f40d00b18c0 2020-11-28T21:53:35.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.164+0000 7f40d597b700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40d00aab50 con 0x7f40d00b44e0 2020-11-28T21:53:35.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.168+0000 7f40d517a700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f40d00b44e0 0x7f40d00b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:35.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.168+0000 7f40cffff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f40d00b18c0 0x7f40d00b2c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:48878/0 (socket says 172.21.15.162:48878) 2020-11-28T21:53:35.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.168+0000 7f40cffff700 1 -- 172.21.15.162:0/2734527733 learned_addr learned my addr 172.21.15.162:0/2734527733 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:35.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.168+0000 7f40d517a700 1 -- 172.21.15.162:0/2734527733 >> v1:172.21.15.162:6789/0 conn(0x7f40d00b18c0 legacy=0x7f40d00b2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:35.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.168+0000 7f40d517a700 1 -- 172.21.15.162:0/2734527733 >> v1:172.21.15.118:6789/0 conn(0x7f40d00b3950 legacy=0x7f40d013dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:35.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.168+0000 7f40d517a700 1 -- 172.21.15.162:0/2734527733 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d00aa8b0 con 0x7f40d00b44e0 2020-11-28T21:53:35.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.168+0000 7f40d517a700 1 --2- 172.21.15.162:0/2734527733 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40d00b44e0 0x7f40d00b3d30 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 rx=0x7f40c80032b0 tx=0x7f40c80091a0).ready entity=mon.1 client_cookie=2f2f2cbd47889afa server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:35.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.172+0000 7f40cf7fe700 1 -- 172.21.15.162:0/2734527733 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40c800ba50 con 0x7f40d00b44e0 2020-11-28T21:53:35.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.172+0000 7f40cf7fe700 1 -- 172.21.15.162:0/2734527733 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c800bbb0 con 0x7f40d00b44e0 2020-11-28T21:53:35.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.172+0000 7f40cf7fe700 1 -- 172.21.15.162:0/2734527733 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40c800bd20 con 0x7f40d00b44e0 2020-11-28T21:53:35.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.172+0000 7f40d597b700 1 -- 172.21.15.162:0/2734527733 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40d00b44e0 msgr2=0x7f40d00b3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:35.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.172+0000 7f40d597b700 1 --2- 172.21.15.162:0/2734527733 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40d00b44e0 0x7f40d00b3d30 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 rx=0x7f40c80032b0 tx=0x7f40c80091a0).stop 2020-11-28T21:53:35.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.172+0000 7f40d597b700 1 -- 172.21.15.162:0/2734527733 shutdown_connections 2020-11-28T21:53:35.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.172+0000 7f40d597b700 1 --2- 172.21.15.162:0/2734527733 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40d00b44e0 0x7f40d00b3d30 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:35.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.172+0000 7f40d597b700 1 -- 172.21.15.162:0/2734527733 >> 172.21.15.162:0/2734527733 conn(0x7f40d0062630 msgr2=0x7f40d0062a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:35.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.172+0000 7f40d597b700 1 -- 172.21.15.162:0/2734527733 shutdown_connections 2020-11-28T21:53:35.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.172+0000 7f40d597b700 1 -- 172.21.15.162:0/2734527733 wait complete. 2020-11-28T21:53:35.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.176+0000 7f40d597b700 1 Processor -- start 2020-11-28T21:53:35.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.176+0000 7f40d597b700 1 -- start start 2020-11-28T21:53:35.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.176+0000 7f40d597b700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f40d00b18c0 0x7f40d0142de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:35.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.176+0000 7f40d597b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f40d00b3950 0x7f40d00abb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:35.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.176+0000 7f40d597b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40d00b44e0 0x7f40d00ac010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:35.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.176+0000 7f40d597b700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d00b5750 con 0x7f40d00b3950 2020-11-28T21:53:35.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.176+0000 7f40d597b700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d00b59f0 con 0x7f40d00b44e0 2020-11-28T21:53:35.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.176+0000 7f40d597b700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d00ab3e0 con 0x7f40d00b18c0 2020-11-28T21:53:35.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.176+0000 7f40d517a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40d00b44e0 0x7f40d00ac010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:35.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.176+0000 7f40cffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f40d00b3950 0x7f40d00abb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:35.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.176+0000 7f40d517a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40d00b44e0 0x7f40d00ac010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42042/0 (socket says 172.21.15.162:42042) 2020-11-28T21:53:35.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.176+0000 7f40d517a700 1 -- 172.21.15.162:0/887117097 learned_addr learned my addr 172.21.15.162:0/887117097 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:35.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.176+0000 7f40d4979700 1 --2- 172.21.15.162:0/887117097 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f40d00b18c0 0x7f40d0142de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:35.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.176+0000 7f40d517a700 1 -- 172.21.15.162:0/887117097 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f40d00b18c0 msgr2=0x7f40d0142de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:35.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.176+0000 7f40d517a700 1 --2- 172.21.15.162:0/887117097 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f40d00b18c0 0x7f40d0142de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:35.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.176+0000 7f40d517a700 1 -- 172.21.15.162:0/887117097 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f40d00b3950 msgr2=0x7f40d00abb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:35.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.176+0000 7f40d517a700 1 --2- 172.21.15.162:0/887117097 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f40d00b3950 0x7f40d00abb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:35.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.176+0000 7f40d517a700 1 -- 172.21.15.162:0/887117097 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d00ab680 con 0x7f40d00b44e0 2020-11-28T21:53:35.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.176+0000 7f40d517a700 1 --2- 172.21.15.162:0/887117097 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40d00b44e0 0x7f40d00ac010 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 rx=0x7f40c80032b0 tx=0x7f40c8016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:35.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.180+0000 7f40cdffb700 1 -- 172.21.15.162:0/887117097 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40c8010660 con 0x7f40d00b44e0 2020-11-28T21:53:35.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.180+0000 7f40cdffb700 1 -- 172.21.15.162:0/887117097 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c80107c0 con 0x7f40d00b44e0 2020-11-28T21:53:35.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.180+0000 7f40cdffb700 1 -- 172.21.15.162:0/887117097 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40c8010b20 con 0x7f40d00b44e0 2020-11-28T21:53:35.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.180+0000 7f40d597b700 1 -- 172.21.15.162:0/887117097 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d01432d0 con 0x7f40d00b44e0 2020-11-28T21:53:35.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.180+0000 7f40d597b700 1 -- 172.21.15.162:0/887117097 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d00b5460 con 0x7f40d00b44e0 2020-11-28T21:53:35.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.180+0000 7f40d597b700 1 -- 172.21.15.162:0/887117097 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40d00ab810 con 0x7f40d00b44e0 2020-11-28T21:53:35.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.180+0000 7f40cdffb700 1 -- 172.21.15.162:0/887117097 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f40c8026030 con 0x7f40d00b44e0 2020-11-28T21:53:35.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.184+0000 7f40cdffb700 1 --2- 172.21.15.162:0/887117097 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f40b0055330 0x7f40b00577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:35.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.184+0000 7f40cdffb700 1 -- 172.21.15.162:0/887117097 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f40c806ed00 con 0x7f40d00b44e0 2020-11-28T21:53:35.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.184+0000 7f40d4979700 1 --2- 172.21.15.162:0/887117097 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f40b0055330 0x7f40b00577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:35.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.184+0000 7f40d4979700 1 --2- 172.21.15.162:0/887117097 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f40b0055330 0x7f40b00577f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f40c000b000 tx=0x7f40c0009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:35.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.188+0000 7f40cdffb700 1 -- 172.21.15.162:0/887117097 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f40c8072030 con 0x7f40d00b44e0 2020-11-28T21:53:35.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:35.324+0000 7f40d597b700 1 -- 172.21.15.162:0/887117097 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f40d0057050 con 0x7f40d00b44e0 2020-11-28T21:53:36.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.188+0000 7f40cdffb700 1 -- 172.21.15.162:0/887117097 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v86) v1 ==== 125+0+0 (secure 0 0 0) 0x7f40c80240a0 con 0x7f40d00b44e0 2020-11-28T21:53:36.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.192+0000 7f40d597b700 1 -- 172.21.15.162:0/887117097 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f40b0055330 msgr2=0x7f40b00577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:36.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.192+0000 7f40d597b700 1 --2- 172.21.15.162:0/887117097 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f40b0055330 0x7f40b00577f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f40c000b000 tx=0x7f40c0009040).stop 2020-11-28T21:53:36.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.192+0000 7f40d597b700 1 -- 172.21.15.162:0/887117097 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40d00b44e0 msgr2=0x7f40d00ac010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:36.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.192+0000 7f40d597b700 1 --2- 172.21.15.162:0/887117097 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40d00b44e0 0x7f40d00ac010 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 rx=0x7f40c80032b0 tx=0x7f40c8016000).stop 2020-11-28T21:53:36.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.192+0000 7f40d597b700 1 -- 172.21.15.162:0/887117097 shutdown_connections 2020-11-28T21:53:36.200 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.192+0000 7f40d597b700 1 --2- 172.21.15.162:0/887117097 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f40b0055330 0x7f40b00577f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:36.200 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.192+0000 7f40d597b700 1 --2- 172.21.15.162:0/887117097 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f40d00b18c0 0x7f40d0142de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:36.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.192+0000 7f40d597b700 1 --2- 172.21.15.162:0/887117097 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40d00b44e0 0x7f40d00ac010 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:36.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.192+0000 7f40d597b700 1 --2- 172.21.15.162:0/887117097 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f40d00b3950 0x7f40d00abb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:36.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.192+0000 7f40d597b700 1 -- 172.21.15.162:0/887117097 >> 172.21.15.162:0/887117097 conn(0x7f40d0062630 msgr2=0x7f40d013f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:36.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.192+0000 7f40d597b700 1 -- 172.21.15.162:0/887117097 shutdown_connections 2020-11-28T21:53:36.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.192+0000 7f40d597b700 1 -- 172.21.15.162:0/887117097 wait complete. 2020-11-28T21:53:36.203 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_data' created 2020-11-28T21:53:36.214 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2020-11-28T21:53:36.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.332+0000 7f941759e700 1 Processor -- start 2020-11-28T21:53:36.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.332+0000 7f941759e700 1 -- start start 2020-11-28T21:53:36.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.336+0000 7f941759e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f94180b2d00 0x7f941805c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:36.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.336+0000 7f941759e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f94180b3ab0 0x7f94180b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:36.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.336+0000 7f941759e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f941805baf0 con 0x7f9418142ad0 2020-11-28T21:53:36.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.336+0000 7f941759e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94180b0e20 con 0x7f94180b2d00 2020-11-28T21:53:36.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.336+0000 7f941759e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f941805bd90 con 0x7f94180b3ab0 2020-11-28T21:53:36.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.336+0000 7f9416d9d700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f9418142ad0 0x7f941813fa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:48892/0 (socket says 172.21.15.162:48892) 2020-11-28T21:53:36.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.336+0000 7f9416d9d700 1 -- 172.21.15.162:0/1389571355 learned_addr learned my addr 172.21.15.162:0/1389571355 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:36.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.336+0000 7f9415d9b700 1 --2- 172.21.15.162:0/1389571355 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f94180b3ab0 0x7f94180b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:36.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.336+0000 7f941659c700 1 --2- 172.21.15.162:0/1389571355 >> v2:172.21.15.118:3300/0 conn(0x7f94180b2d00 0x7f941805c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:36.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.336+0000 7f941559a700 1 -- 172.21.15.162:0/1389571355 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 323393124 0 0) 0x7f941805baf0 con 0x7f9418142ad0 2020-11-28T21:53:36.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.336+0000 7f941559a700 1 -- 172.21.15.162:0/1389571355 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93fc003230 con 0x7f9418142ad0 2020-11-28T21:53:36.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.336+0000 7f941659c700 1 -- 172.21.15.162:0/1389571355 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f94180b3ab0 msgr2=0x7f94180b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:36.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.336+0000 7f941659c700 1 --2- 172.21.15.162:0/1389571355 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f94180b3ab0 0x7f94180b30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:36.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.336+0000 7f941559a700 1 -- 172.21.15.162:0/1389571355 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2589270339 0 0) 0x7f93fc003230 con 0x7f9418142ad0 2020-11-28T21:53:36.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.336+0000 7f941659c700 1 -- 172.21.15.162:0/1389571355 >> v1:172.21.15.162:6789/0 conn(0x7f9418142ad0 legacy=0x7f941813fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:36.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.336+0000 7f941659c700 1 -- 172.21.15.162:0/1389571355 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f941805c030 con 0x7f94180b2d00 2020-11-28T21:53:36.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.336+0000 7f941659c700 1 --2- 172.21.15.162:0/1389571355 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f94180b2d00 0x7f941805c4d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f9400001870 tx=0x7f9410006dc0).ready entity=mon.0 client_cookie=448e2bfc119bf7c3 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:36.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.336+0000 7f941559a700 1 -- 172.21.15.162:0/1389571355 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94000016d0 con 0x7f94180b2d00 2020-11-28T21:53:36.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.336+0000 7f941559a700 1 -- 172.21.15.162:0/1389571355 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f941000d570 con 0x7f94180b2d00 2020-11-28T21:53:36.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.336+0000 7f941559a700 1 -- 172.21.15.162:0/1389571355 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f941000d6e0 con 0x7f94180b2d00 2020-11-28T21:53:36.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.336+0000 7f941759e700 1 -- 172.21.15.162:0/1389571355 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f94180b2d00 msgr2=0x7f941805c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:36.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.336+0000 7f941759e700 1 --2- 172.21.15.162:0/1389571355 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f94180b2d00 0x7f941805c4d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f9400001870 tx=0x7f9410006dc0).stop 2020-11-28T21:53:36.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.336+0000 7f941759e700 1 -- 172.21.15.162:0/1389571355 shutdown_connections 2020-11-28T21:53:36.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.336+0000 7f941759e700 1 --2- 172.21.15.162:0/1389571355 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f94180b2d00 0x7f941805c4d0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:36.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.336+0000 7f941759e700 1 --2- 172.21.15.162:0/1389571355 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f94180b3ab0 0x7f94180b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:36.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.336+0000 7f941759e700 1 -- 172.21.15.162:0/1389571355 >> 172.21.15.162:0/1389571355 conn(0x7f94180604c0 msgr2=0x7f94180608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:36.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.336+0000 7f941759e700 1 -- 172.21.15.162:0/1389571355 shutdown_connections 2020-11-28T21:53:36.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.336+0000 7f941759e700 1 -- 172.21.15.162:0/1389571355 wait complete. 2020-11-28T21:53:36.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.340+0000 7f941759e700 1 Processor -- start 2020-11-28T21:53:36.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.340+0000 7f941759e700 1 -- start start 2020-11-28T21:53:36.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.340+0000 7f941759e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f94180b2d00 0x7f94181421b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:36.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.340+0000 7f941759e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f94180b3ab0 0x7f9418154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:36.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.340+0000 7f941759e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9418142ad0 0x7f94181554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:36.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.340+0000 7f941759e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941814a2f0 con 0x7f94180b2d00 2020-11-28T21:53:36.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.340+0000 7f941759e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94180af290 con 0x7f9418142ad0 2020-11-28T21:53:36.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.340+0000 7f941759e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9418154b90 con 0x7f94180b3ab0 2020-11-28T21:53:36.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.340+0000 7f9416d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9418142ad0 0x7f94181554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:36.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.340+0000 7f9416d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9418142ad0 0x7f94181554b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42056/0 (socket says 172.21.15.162:42056) 2020-11-28T21:53:36.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.340+0000 7f9416d9d700 1 -- 172.21.15.162:0/2353868332 learned_addr learned my addr 172.21.15.162:0/2353868332 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:36.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.340+0000 7f941659c700 1 --2- 172.21.15.162:0/2353868332 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f94180b2d00 0x7f94181421b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:36.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.340+0000 7f9415d9b700 1 --2- 172.21.15.162:0/2353868332 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f94180b3ab0 0x7f9418154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:36.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.340+0000 7f9416d9d700 1 -- 172.21.15.162:0/2353868332 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f94180b3ab0 msgr2=0x7f9418154fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:36.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.340+0000 7f9416d9d700 1 --2- 172.21.15.162:0/2353868332 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f94180b3ab0 0x7f9418154fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:36.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.340+0000 7f9416d9d700 1 -- 172.21.15.162:0/2353868332 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f94180b2d00 msgr2=0x7f94181421b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:36.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.340+0000 7f9416d9d700 1 --2- 172.21.15.162:0/2353868332 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f94180b2d00 0x7f94181421b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:36.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.340+0000 7f9416d9d700 1 -- 172.21.15.162:0/2353868332 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94180ad8f0 con 0x7f9418142ad0 2020-11-28T21:53:36.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.340+0000 7f9416d9d700 1 --2- 172.21.15.162:0/2353868332 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9418142ad0 0x7f94181554b0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0x7f94180aff50 tx=0x7f9400001e50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:36.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.340+0000 7f940f7fe700 1 -- 172.21.15.162:0/2353868332 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9400007170 con 0x7f9418142ad0 2020-11-28T21:53:36.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.340+0000 7f940f7fe700 1 -- 172.21.15.162:0/2353868332 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94000072d0 con 0x7f9418142ad0 2020-11-28T21:53:36.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.340+0000 7f941759e700 1 -- 172.21.15.162:0/2353868332 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94180adb90 con 0x7f9418142ad0 2020-11-28T21:53:36.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.340+0000 7f940f7fe700 1 -- 172.21.15.162:0/2353868332 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94000048f0 con 0x7f9418142ad0 2020-11-28T21:53:36.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.340+0000 7f941759e700 1 -- 172.21.15.162:0/2353868332 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94180ae1f0 con 0x7f9418142ad0 2020-11-28T21:53:36.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.344+0000 7f941759e700 1 -- 172.21.15.162:0/2353868332 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94180b5f00 con 0x7f9418142ad0 2020-11-28T21:53:36.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.344+0000 7f940f7fe700 1 -- 172.21.15.162:0/2353868332 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9400004a50 con 0x7f9418142ad0 2020-11-28T21:53:36.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.344+0000 7f940f7fe700 1 --2- 172.21.15.162:0/2353868332 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f93f00553b0 0x7f93f0057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:36.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.344+0000 7f940f7fe700 1 -- 172.21.15.162:0/2353868332 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5957+0+0 (secure 0 0 0) 0x7f940006d7f0 con 0x7f9418142ad0 2020-11-28T21:53:36.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.344+0000 7f941659c700 1 --2- 172.21.15.162:0/2353868332 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f93f00553b0 0x7f93f0057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:36.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.344+0000 7f941659c700 1 --2- 172.21.15.162:0/2353868332 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f93f00553b0 0x7f93f0057870 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f941001a000 tx=0x7f9410015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:36.355 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.348+0000 7f940f7fe700 1 -- 172.21.15.162:0/2353868332 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9400043ac0 con 0x7f9418142ad0 2020-11-28T21:53:36.537 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.532+0000 7f941759e700 1 -- 172.21.15.162:0/2353868332 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f94180ae1f0 con 0x7f9418142ad0 2020-11-28T21:53:36.550 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.548+0000 7f940f7fe700 1 -- 172.21.15.162:0/2353868332 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 15 and data pool 16 v38) v1 ==== 189+0+0 (secure 0 0 0) 0x7f94180ae1f0 con 0x7f9418142ad0 2020-11-28T21:53:36.557 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.548+0000 7f941759e700 1 -- 172.21.15.162:0/2353868332 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f93f00553b0 msgr2=0x7f93f0057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:36.558 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.548+0000 7f941759e700 1 --2- 172.21.15.162:0/2353868332 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f93f00553b0 0x7f93f0057870 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f941001a000 tx=0x7f9410015000).stop 2020-11-28T21:53:36.558 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.548+0000 7f941759e700 1 -- 172.21.15.162:0/2353868332 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9418142ad0 msgr2=0x7f94181554b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:36.558 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.548+0000 7f941759e700 1 --2- 172.21.15.162:0/2353868332 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9418142ad0 0x7f94181554b0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0x7f94180aff50 tx=0x7f9400001e50).stop 2020-11-28T21:53:36.558 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.548+0000 7f941759e700 1 -- 172.21.15.162:0/2353868332 shutdown_connections 2020-11-28T21:53:36.559 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.548+0000 7f941759e700 1 --2- 172.21.15.162:0/2353868332 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f93f00553b0 0x7f93f0057870 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:36.559 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.548+0000 7f941759e700 1 --2- 172.21.15.162:0/2353868332 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9418142ad0 0x7f94181554b0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:36.559 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.548+0000 7f941759e700 1 --2- 172.21.15.162:0/2353868332 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f94180b2d00 0x7f94181421b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:36.559 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.548+0000 7f941759e700 1 --2- 172.21.15.162:0/2353868332 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f94180b3ab0 0x7f9418154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:36.559 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.548+0000 7f941759e700 1 -- 172.21.15.162:0/2353868332 >> 172.21.15.162:0/2353868332 conn(0x7f94180604c0 msgr2=0x7f94180b82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:36.560 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.548+0000 7f941759e700 1 -- 172.21.15.162:0/2353868332 shutdown_connections 2020-11-28T21:53:36.560 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.548+0000 7f941759e700 1 -- 172.21.15.162:0/2353868332 wait complete. 2020-11-28T21:53:36.560 INFO:teuthology.orchestra.run.smithi162.stderr:new fs with metadata pool 15 and data pool 16 2020-11-28T21:53:36.573 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:53:36.672 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.668+0000 7fa9c6a09700 1 Processor -- start 2020-11-28T21:53:36.673 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.668+0000 7fa9c6a09700 1 -- start start 2020-11-28T21:53:36.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.668+0000 7fa9c6a09700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa9c00603d0 0x7fa9c00b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:36.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.668+0000 7fa9c6a09700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa9c0142a00 0x7fa9c009cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:36.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.668+0000 7fa9c6a09700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9c00b4110 con 0x7fa9c00b4c70 2020-11-28T21:53:36.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.668+0000 7fa9c6a09700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9c0060f60 con 0x7fa9c00603d0 2020-11-28T21:53:36.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.668+0000 7fa9c6a09700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c00b43f0 con 0x7fa9c0142a00 2020-11-28T21:53:36.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.668+0000 7fa9c5a07700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa9c00603d0 0x7fa9c00b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:36.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.668+0000 7fa9c5a07700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa9c00603d0 0x7fa9c00b3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42070/0 (socket says 172.21.15.162:42070) 2020-11-28T21:53:36.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.668+0000 7fa9c5a07700 1 -- 172.21.15.162:0/1571590851 learned_addr learned my addr 172.21.15.162:0/1571590851 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:36.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.668+0000 7fa9c5206700 1 --2- 172.21.15.162:0/1571590851 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa9c0142a00 0x7fa9c009cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:36.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.672+0000 7fa9c5a07700 1 -- 172.21.15.162:0/1571590851 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa9c0142a00 msgr2=0x7fa9c009cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:36.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.672+0000 7fa9c5a07700 1 --2- 172.21.15.162:0/1571590851 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa9c0142a00 0x7fa9c009cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:36.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.672+0000 7fa9c4a05700 1 -- 172.21.15.162:0/1571590851 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 399287182 0 0) 0x7fa9c00b4110 con 0x7fa9c00b4c70 2020-11-28T21:53:36.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.672+0000 7fa9c5a07700 1 -- 172.21.15.162:0/1571590851 >> v1:172.21.15.118:6789/0 conn(0x7fa9c00b4c70 legacy=0x7fa9c0143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:36.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.672+0000 7fa9c5a07700 1 -- 172.21.15.162:0/1571590851 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c00b4690 con 0x7fa9c00603d0 2020-11-28T21:53:36.680 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.672+0000 7fa9c5a07700 1 --2- 172.21.15.162:0/1571590851 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9c00603d0 0x7fa9c00b3a70 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 rx=0x7fa9bc002f30 tx=0x7fa9bc006e80).ready entity=mon.1 client_cookie=9415126e1688013c server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:36.680 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.672+0000 7fa9c4a05700 1 -- 172.21.15.162:0/1571590851 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9bc0097f0 con 0x7fa9c00603d0 2020-11-28T21:53:36.683 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.672+0000 7fa9c4a05700 1 -- 172.21.15.162:0/1571590851 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9bc009950 con 0x7fa9c00603d0 2020-11-28T21:53:36.683 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.672+0000 7fa9c4a05700 1 -- 172.21.15.162:0/1571590851 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9bc009cb0 con 0x7fa9c00603d0 2020-11-28T21:53:36.684 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.672+0000 7fa9c6a09700 1 -- 172.21.15.162:0/1571590851 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9c00603d0 msgr2=0x7fa9c00b3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:36.684 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.672+0000 7fa9c6a09700 1 --2- 172.21.15.162:0/1571590851 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9c00603d0 0x7fa9c00b3a70 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 rx=0x7fa9bc002f30 tx=0x7fa9bc006e80).stop 2020-11-28T21:53:36.685 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.672+0000 7fa9c6a09700 1 -- 172.21.15.162:0/1571590851 shutdown_connections 2020-11-28T21:53:36.685 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.672+0000 7fa9c6a09700 1 --2- 172.21.15.162:0/1571590851 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9c00603d0 0x7fa9c00b3a70 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:36.685 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.672+0000 7fa9c6a09700 1 --2- 172.21.15.162:0/1571590851 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa9c0142a00 0x7fa9c009cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:36.685 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.672+0000 7fa9c6a09700 1 -- 172.21.15.162:0/1571590851 >> 172.21.15.162:0/1571590851 conn(0x7fa9c0066a40 msgr2=0x7fa9c0066e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:36.685 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.672+0000 7fa9c6a09700 1 -- 172.21.15.162:0/1571590851 shutdown_connections 2020-11-28T21:53:36.686 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.672+0000 7fa9c6a09700 1 -- 172.21.15.162:0/1571590851 wait complete. 2020-11-28T21:53:36.686 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.672+0000 7fa9c6a09700 1 Processor -- start 2020-11-28T21:53:36.686 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.672+0000 7fa9c6a09700 1 -- start start 2020-11-28T21:53:36.686 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.676+0000 7fa9c6a09700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa9c00603d0 0x7fa9c01496d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:36.687 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.676+0000 7fa9c6a09700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9c00b4c70 0x7fa9c0151960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:36.687 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.676+0000 7fa9c6a09700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa9c00b2db0 0x7fa9c0151e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:36.687 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.676+0000 7fa9c6a09700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c0146d90 con 0x7fa9c00b2db0 2020-11-28T21:53:36.688 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.676+0000 7fa9c6a09700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c0143e60 con 0x7fa9c00b4c70 2020-11-28T21:53:36.688 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.676+0000 7fa9c6a09700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c0151530 con 0x7fa9c00603d0 2020-11-28T21:53:36.688 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.676+0000 7fa9c5206700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9c00b4c70 0x7fa9c0151960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:36.688 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.676+0000 7fa9c5206700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9c00b4c70 0x7fa9c0151960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42076/0 (socket says 172.21.15.162:42076) 2020-11-28T21:53:36.689 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.676+0000 7fa9c5206700 1 -- 172.21.15.162:0/2135761890 learned_addr learned my addr 172.21.15.162:0/2135761890 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:36.689 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.676+0000 7fa9c5a07700 1 --2- 172.21.15.162:0/2135761890 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa9c00603d0 0x7fa9c01496d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:36.689 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.676+0000 7fa9c6208700 1 --2- 172.21.15.162:0/2135761890 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa9c00b2db0 0x7fa9c0151e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:36.689 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.676+0000 7fa9c5206700 1 -- 172.21.15.162:0/2135761890 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa9c00603d0 msgr2=0x7fa9c01496d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:36.689 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.676+0000 7fa9c5206700 1 --2- 172.21.15.162:0/2135761890 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa9c00603d0 0x7fa9c01496d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:36.690 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.676+0000 7fa9c5206700 1 -- 172.21.15.162:0/2135761890 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa9c00b2db0 msgr2=0x7fa9c0151e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:36.690 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.676+0000 7fa9c5206700 1 --2- 172.21.15.162:0/2135761890 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa9c00b2db0 0x7fa9c0151e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:36.690 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.676+0000 7fa9c5206700 1 -- 172.21.15.162:0/2135761890 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c008fa20 con 0x7fa9c00b4c70 2020-11-28T21:53:36.691 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.676+0000 7fa9c5206700 1 --2- 172.21.15.162:0/2135761890 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9c00b4c70 0x7fa9c0151960 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 rx=0x7fa9b000c000 tx=0x7fa9b0006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:36.691 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.676+0000 7fa9aeffd700 1 -- 172.21.15.162:0/2135761890 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9b000ca10 con 0x7fa9c00b4c70 2020-11-28T21:53:36.691 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.676+0000 7fa9aeffd700 1 -- 172.21.15.162:0/2135761890 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9b000cb70 con 0x7fa9c00b4c70 2020-11-28T21:53:36.691 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.676+0000 7fa9c6a09700 1 -- 172.21.15.162:0/2135761890 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9c008fcc0 con 0x7fa9c00b4c70 2020-11-28T21:53:36.691 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.676+0000 7fa9aeffd700 1 -- 172.21.15.162:0/2135761890 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9b000be70 con 0x7fa9c00b4c70 2020-11-28T21:53:36.692 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.676+0000 7fa9c6a09700 1 -- 172.21.15.162:0/2135761890 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9c0146a10 con 0x7fa9c00b4c70 2020-11-28T21:53:36.692 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.676+0000 7fa9c6a09700 1 -- 172.21.15.162:0/2135761890 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9c0057050 con 0x7fa9c00b4c70 2020-11-28T21:53:36.692 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.680+0000 7fa9aeffd700 1 -- 172.21.15.162:0/2135761890 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa9b001c460 con 0x7fa9c00b4c70 2020-11-28T21:53:36.693 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.680+0000 7fa9aeffd700 1 --2- 172.21.15.162:0/2135761890 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa9a0055440 0x7fa9a0057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:36.693 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.680+0000 7fa9aeffd700 1 -- 172.21.15.162:0/2135761890 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa9b006e3b0 con 0x7fa9c00b4c70 2020-11-28T21:53:36.693 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.680+0000 7fa9c5a07700 1 --2- 172.21.15.162:0/2135761890 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa9a0055440 0x7fa9a0057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:36.694 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.680+0000 7fa9c5a07700 1 --2- 172.21.15.162:0/2135761890 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa9a0055440 0x7fa9a0057900 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7fa9bc01c000 tx=0x7fa9bc015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:36.694 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.684+0000 7fa9aeffd700 1 -- 172.21.15.162:0/2135761890 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa9b0044650 con 0x7fa9c00b4c70 2020-11-28T21:53:36.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.820+0000 7fa9c6a09700 1 -- 172.21.15.162:0/2135761890 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9c0069b60 con 0x7fa9c00b4c70 2020-11-28T21:53:36.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.820+0000 7fa9aeffd700 1 -- 172.21.15.162:0/2135761890 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16150 (secure 0 0 0) 0x7fa9b0047c70 con 0x7fa9c00b4c70 2020-11-28T21:53:36.827 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:53:36.827 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":87,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:53:36.538830+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:53:34.083725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2020-11-28T21:53:35.380493+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:53:36.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.824+0000 7fa9c6a09700 1 -- 172.21.15.162:0/2135761890 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa9a0055440 msgr2=0x7fa9a0057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:36.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.824+0000 7fa9c6a09700 1 --2- 172.21.15.162:0/2135761890 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa9a0055440 0x7fa9a0057900 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7fa9bc01c000 tx=0x7fa9bc015000).stop 2020-11-28T21:53:36.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.824+0000 7fa9c6a09700 1 -- 172.21.15.162:0/2135761890 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9c00b4c70 msgr2=0x7fa9c0151960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:36.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.824+0000 7fa9c6a09700 1 --2- 172.21.15.162:0/2135761890 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9c00b4c70 0x7fa9c0151960 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 rx=0x7fa9b000c000 tx=0x7fa9b0006dc0).stop 2020-11-28T21:53:36.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.824+0000 7fa9c6a09700 1 -- 172.21.15.162:0/2135761890 shutdown_connections 2020-11-28T21:53:36.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.824+0000 7fa9c6a09700 1 --2- 172.21.15.162:0/2135761890 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa9a0055440 0x7fa9a0057900 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:36.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.824+0000 7fa9c6a09700 1 --2- 172.21.15.162:0/2135761890 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa9c00603d0 0x7fa9c01496d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:36.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.824+0000 7fa9c6a09700 1 --2- 172.21.15.162:0/2135761890 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa9c00b2db0 0x7fa9c0151e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:36.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.824+0000 7fa9c6a09700 1 --2- 172.21.15.162:0/2135761890 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9c00b4c70 0x7fa9c0151960 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:36.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.824+0000 7fa9c6a09700 1 -- 172.21.15.162:0/2135761890 >> 172.21.15.162:0/2135761890 conn(0x7fa9c0066a40 msgr2=0x7fa9c0142df0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:36.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.824+0000 7fa9c6a09700 1 -- 172.21.15.162:0/2135761890 shutdown_connections 2020-11-28T21:53:36.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.824+0000 7fa9c6a09700 1 -- 172.21.15.162:0/2135761890 wait complete. 2020-11-28T21:53:36.840 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:53:36.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.948+0000 7f5ea759e700 1 Processor -- start 2020-11-28T21:53:36.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.948+0000 7f5ea759e700 1 -- start start 2020-11-28T21:53:36.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.952+0000 7f5ea759e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5ea8065750 0x7f5ea814a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:36.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.952+0000 7f5ea759e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ea8065050 con 0x7f5ea813f6c0 2020-11-28T21:53:36.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.952+0000 7f5ea759e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ea813fc40 con 0x7f5ea8064c70 2020-11-28T21:53:36.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.952+0000 7f5ea759e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ea813fee0 con 0x7f5ea8065750 2020-11-28T21:53:36.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.952+0000 7f5ea6d9d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5ea8065750 0x7f5ea814a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:36.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.952+0000 7f5ea6d9d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5ea8065750 0x7f5ea814a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42086/0 (socket says 172.21.15.162:42086) 2020-11-28T21:53:36.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.952+0000 7f5ea6d9d700 1 -- 172.21.15.162:0/1659868815 learned_addr learned my addr 172.21.15.162:0/1659868815 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:36.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.952+0000 7f5ea659c700 1 --1- 172.21.15.162:0/1659868815 >> v1:172.21.15.162:6789/0 conn(0x7f5ea8064c70 0x7f5ea8066290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:48928/0 (socket says 172.21.15.162:48928) 2020-11-28T21:53:36.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.952+0000 7f5ea6d9d700 1 -- 172.21.15.162:0/1659868815 >> v1:172.21.15.162:6789/0 conn(0x7f5ea8064c70 legacy=0x7f5ea8066290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:36.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.952+0000 7f5ea6d9d700 1 -- 172.21.15.162:0/1659868815 >> v1:172.21.15.118:6789/0 conn(0x7f5ea813f6c0 legacy=0x7f5ea80669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:36.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.952+0000 7f5ea6d9d700 1 -- 172.21.15.162:0/1659868815 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea8140180 con 0x7f5ea8065750 2020-11-28T21:53:36.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.952+0000 7f5ea6d9d700 1 --2- 172.21.15.162:0/1659868815 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ea8065750 0x7f5ea814a240 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 rx=0x7f5ea80b6b50 tx=0x7f5ea0009270).ready entity=mon.1 client_cookie=3b4558b49e71036b server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:36.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.952+0000 7f5ea559a700 1 -- 172.21.15.162:0/1659868815 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ea000bbc0 con 0x7f5ea8065750 2020-11-28T21:53:36.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.952+0000 7f5ea559a700 1 -- 172.21.15.162:0/1659868815 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea000bd20 con 0x7f5ea8065750 2020-11-28T21:53:36.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.952+0000 7f5ea559a700 1 -- 172.21.15.162:0/1659868815 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ea000be90 con 0x7f5ea8065750 2020-11-28T21:53:36.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.952+0000 7f5ea759e700 1 -- 172.21.15.162:0/1659868815 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ea8065750 msgr2=0x7f5ea814a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:36.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.952+0000 7f5ea759e700 1 --2- 172.21.15.162:0/1659868815 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ea8065750 0x7f5ea814a240 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 rx=0x7f5ea80b6b50 tx=0x7f5ea0009270).stop 2020-11-28T21:53:36.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.952+0000 7f5ea759e700 1 -- 172.21.15.162:0/1659868815 shutdown_connections 2020-11-28T21:53:36.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.952+0000 7f5ea759e700 1 --2- 172.21.15.162:0/1659868815 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ea8065750 0x7f5ea814a240 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:36.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.952+0000 7f5ea759e700 1 -- 172.21.15.162:0/1659868815 >> 172.21.15.162:0/1659868815 conn(0x7f5ea80b5da0 msgr2=0x7f5ea80b6180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:36.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.952+0000 7f5ea759e700 1 -- 172.21.15.162:0/1659868815 shutdown_connections 2020-11-28T21:53:36.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.952+0000 7f5ea759e700 1 -- 172.21.15.162:0/1659868815 wait complete. 2020-11-28T21:53:36.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.952+0000 7f5ea759e700 1 Processor -- start 2020-11-28T21:53:36.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.952+0000 7f5ea759e700 1 -- start start 2020-11-28T21:53:36.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.956+0000 7f5ea759e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5ea8064c70 0x7f5ea813eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:36.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.956+0000 7f5ea759e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5ea8065750 0x7f5ea8144430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:36.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.956+0000 7f5ea759e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ea813f6c0 0x7f5ea81354f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:36.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.956+0000 7f5ea759e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea8144000 con 0x7f5ea8064c70 2020-11-28T21:53:36.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.956+0000 7f5ea759e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea8138ae0 con 0x7f5ea813f6c0 2020-11-28T21:53:36.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.956+0000 7f5ea759e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea8138d80 con 0x7f5ea8065750 2020-11-28T21:53:36.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.956+0000 7f5ea6d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ea813f6c0 0x7f5ea81354f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:36.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.956+0000 7f5ea6d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ea813f6c0 0x7f5ea81354f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42092/0 (socket says 172.21.15.162:42092) 2020-11-28T21:53:36.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.956+0000 7f5ea6d9d700 1 -- 172.21.15.162:0/2632666936 learned_addr learned my addr 172.21.15.162:0/2632666936 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:36.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.956+0000 7f5ea659c700 1 --2- 172.21.15.162:0/2632666936 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5ea8064c70 0x7f5ea813eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:36.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.956+0000 7f5ea5d9b700 1 --2- 172.21.15.162:0/2632666936 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5ea8065750 0x7f5ea8144430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:36.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.956+0000 7f5ea6d9d700 1 -- 172.21.15.162:0/2632666936 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5ea8065750 msgr2=0x7f5ea8144430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:36.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.956+0000 7f5ea6d9d700 1 --2- 172.21.15.162:0/2632666936 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5ea8065750 0x7f5ea8144430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:36.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.956+0000 7f5ea6d9d700 1 -- 172.21.15.162:0/2632666936 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5ea8064c70 msgr2=0x7f5ea813eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:36.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.956+0000 7f5ea6d9d700 1 --2- 172.21.15.162:0/2632666936 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5ea8064c70 0x7f5ea813eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:36.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.956+0000 7f5ea6d9d700 1 -- 172.21.15.162:0/2632666936 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea8145af0 con 0x7f5ea813f6c0 2020-11-28T21:53:36.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.956+0000 7f5ea6d9d700 1 --2- 172.21.15.162:0/2632666936 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ea813f6c0 0x7f5ea81354f0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0x7f5ea80b6b50 tx=0x7f5ea0016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:36.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.956+0000 7f5e9f7fe700 1 -- 172.21.15.162:0/2632666936 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ea001e610 con 0x7f5ea813f6c0 2020-11-28T21:53:36.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.956+0000 7f5e9f7fe700 1 -- 172.21.15.162:0/2632666936 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea001e770 con 0x7f5ea813f6c0 2020-11-28T21:53:36.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.956+0000 7f5ea759e700 1 -- 172.21.15.162:0/2632666936 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ea80b55f0 con 0x7f5ea813f6c0 2020-11-28T21:53:36.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.956+0000 7f5e9f7fe700 1 -- 172.21.15.162:0/2632666936 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ea001ead0 con 0x7f5ea813f6c0 2020-11-28T21:53:36.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.956+0000 7f5ea759e700 1 -- 172.21.15.162:0/2632666936 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea8144ac0 con 0x7f5ea813f6c0 2020-11-28T21:53:36.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.956+0000 7f5ea759e700 1 -- 172.21.15.162:0/2632666936 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ea81456c0 con 0x7f5ea813f6c0 2020-11-28T21:53:36.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.960+0000 7f5e9f7fe700 1 -- 172.21.15.162:0/2632666936 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5ea000f070 con 0x7f5ea813f6c0 2020-11-28T21:53:36.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.960+0000 7f5e9f7fe700 1 --2- 172.21.15.162:0/2632666936 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5e80055330 0x7f5e800577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:36.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.960+0000 7f5e9f7fe700 1 -- 172.21.15.162:0/2632666936 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5ea006f6e0 con 0x7f5ea813f6c0 2020-11-28T21:53:36.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.960+0000 7f5ea659c700 1 --2- 172.21.15.162:0/2632666936 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5e80055330 0x7f5e800577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:36.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.960+0000 7f5ea659c700 1 --2- 172.21.15.162:0/2632666936 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5e80055330 0x7f5e800577f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7f5e9800d000 tx=0x7f5e98007b70).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:36.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:36.964+0000 7f5e9f7fe700 1 -- 172.21.15.162:0/2632666936 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5ea81456c0 con 0x7f5ea813f6c0 2020-11-28T21:53:37.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.108+0000 7f5ea759e700 1 -- 172.21.15.162:0/2632666936 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ea81456c0 con 0x7f5ea813f6c0 2020-11-28T21:53:37.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.108+0000 7f5e9f7fe700 1 -- 172.21.15.162:0/2632666936 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16150 (secure 0 0 0) 0x7f5ea81456c0 con 0x7f5ea813f6c0 2020-11-28T21:53:37.114 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:53:37.114 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":87,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:53:36.538830+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:53:34.083725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2020-11-28T21:53:35.380493+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:53:37.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.112+0000 7f5ea759e700 1 -- 172.21.15.162:0/2632666936 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5e80055330 msgr2=0x7f5e800577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:37.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.112+0000 7f5ea759e700 1 --2- 172.21.15.162:0/2632666936 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5e80055330 0x7f5e800577f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7f5e9800d000 tx=0x7f5e98007b70).stop 2020-11-28T21:53:37.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.112+0000 7f5ea759e700 1 -- 172.21.15.162:0/2632666936 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ea813f6c0 msgr2=0x7f5ea81354f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:37.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.112+0000 7f5ea759e700 1 --2- 172.21.15.162:0/2632666936 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ea813f6c0 0x7f5ea81354f0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0x7f5ea80b6b50 tx=0x7f5ea0016000).stop 2020-11-28T21:53:37.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.112+0000 7f5ea759e700 1 -- 172.21.15.162:0/2632666936 shutdown_connections 2020-11-28T21:53:37.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.112+0000 7f5ea759e700 1 --2- 172.21.15.162:0/2632666936 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5e80055330 0x7f5e800577f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:37.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.112+0000 7f5ea759e700 1 --2- 172.21.15.162:0/2632666936 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ea813f6c0 0x7f5ea81354f0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:37.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.112+0000 7f5ea759e700 1 --2- 172.21.15.162:0/2632666936 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5ea8064c70 0x7f5ea813eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:37.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.112+0000 7f5ea759e700 1 --2- 172.21.15.162:0/2632666936 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5ea8065750 0x7f5ea8144430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:37.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.112+0000 7f5ea759e700 1 -- 172.21.15.162:0/2632666936 >> 172.21.15.162:0/2632666936 conn(0x7f5ea80b5da0 msgr2=0x7f5ea813bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:37.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.112+0000 7f5ea759e700 1 -- 172.21.15.162:0/2632666936 shutdown_connections 2020-11-28T21:53:37.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.112+0000 7f5ea759e700 1 -- 172.21.15.162:0/2632666936 wait complete. 2020-11-28T21:53:37.139 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2020-11-28T21:53:37.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.252+0000 7f311cbdf700 1 Processor -- start 2020-11-28T21:53:37.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.252+0000 7f311cbdf700 1 -- start start 2020-11-28T21:53:37.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.256+0000 7f311cbdf700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f31180b2bb0 0x7f31180b2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:37.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.256+0000 7f311cbdf700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f31180b2f90 0x7f311813c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:37.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.256+0000 7f311cbdf700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3118143690 con 0x7f3118062260 2020-11-28T21:53:37.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.256+0000 7f311cbdf700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31180b39d0 con 0x7f31180b2f90 2020-11-28T21:53:37.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.256+0000 7f311cbdf700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31180b3470 con 0x7f31180b2bb0 2020-11-28T21:53:37.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.256+0000 7f3115d9b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f31180b2f90 0x7f311813c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:37.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.256+0000 7f3115d9b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f31180b2f90 0x7f311813c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42098/0 (socket says 172.21.15.162:42098) 2020-11-28T21:53:37.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.256+0000 7f3115d9b700 1 -- 172.21.15.162:0/3345188702 learned_addr learned my addr 172.21.15.162:0/3345188702 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:37.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.256+0000 7f311659c700 1 --2- 172.21.15.162:0/3345188702 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f31180b2bb0 0x7f31180b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:37.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.256+0000 7f3115d9b700 1 -- 172.21.15.162:0/3345188702 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f31180b2bb0 msgr2=0x7f31180b2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:37.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.256+0000 7f3115d9b700 1 --2- 172.21.15.162:0/3345188702 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f31180b2bb0 0x7f31180b2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:37.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.256+0000 7f3115d9b700 1 -- 172.21.15.162:0/3345188702 >> v1:172.21.15.118:6789/0 conn(0x7f3118062260 legacy=0x7f311813ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:37.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.256+0000 7f3115d9b700 1 -- 172.21.15.162:0/3345188702 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3118143b60 con 0x7f31180b2f90 2020-11-28T21:53:37.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.256+0000 7f3115d9b700 1 --2- 172.21.15.162:0/3345188702 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31180b2f90 0x7f311813c920 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0x7f3100002f30 tx=0x7f3100006e80).ready entity=mon.1 client_cookie=464f65a6f3fcbd2 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:37.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.256+0000 7f311559a700 1 -- 172.21.15.162:0/3345188702 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31000097f0 con 0x7f31180b2f90 2020-11-28T21:53:37.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.256+0000 7f311559a700 1 -- 172.21.15.162:0/3345188702 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3100009950 con 0x7f31180b2f90 2020-11-28T21:53:37.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.256+0000 7f311559a700 1 -- 172.21.15.162:0/3345188702 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3100009cb0 con 0x7f31180b2f90 2020-11-28T21:53:37.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.256+0000 7f311cbdf700 1 -- 172.21.15.162:0/3345188702 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31180b2f90 msgr2=0x7f311813c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:37.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.256+0000 7f311cbdf700 1 --2- 172.21.15.162:0/3345188702 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31180b2f90 0x7f311813c920 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0x7f3100002f30 tx=0x7f3100006e80).stop 2020-11-28T21:53:37.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.256+0000 7f311cbdf700 1 -- 172.21.15.162:0/3345188702 shutdown_connections 2020-11-28T21:53:37.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.256+0000 7f311cbdf700 1 --2- 172.21.15.162:0/3345188702 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f31180b2bb0 0x7f31180b2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:37.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.256+0000 7f311cbdf700 1 --2- 172.21.15.162:0/3345188702 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31180b2f90 0x7f311813c920 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:37.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.256+0000 7f311cbdf700 1 -- 172.21.15.162:0/3345188702 >> 172.21.15.162:0/3345188702 conn(0x7f3118061740 msgr2=0x7f3118061b50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:37.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.256+0000 7f311cbdf700 1 -- 172.21.15.162:0/3345188702 shutdown_connections 2020-11-28T21:53:37.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.256+0000 7f311cbdf700 1 -- 172.21.15.162:0/3345188702 wait complete. 2020-11-28T21:53:37.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.256+0000 7f311cbdf700 1 Processor -- start 2020-11-28T21:53:37.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.256+0000 7f311cbdf700 1 -- start start 2020-11-28T21:53:37.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.260+0000 7f311cbdf700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3118062260 0x7f31181415d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:37.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.260+0000 7f311cbdf700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f31180b2bb0 0x7f3118151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:37.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.260+0000 7f311cbdf700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31180b2f90 0x7f3118151b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:37.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.260+0000 7f311cbdf700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31181428c0 con 0x7f31180b2bb0 2020-11-28T21:53:37.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.260+0000 7f311cbdf700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31180ad3b0 con 0x7f31180b2f90 2020-11-28T21:53:37.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.260+0000 7f311cbdf700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31180ad650 con 0x7f3118062260 2020-11-28T21:53:37.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.260+0000 7f3116d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31180b2f90 0x7f3118151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:37.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.260+0000 7f3116d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31180b2f90 0x7f3118151b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42104/0 (socket says 172.21.15.162:42104) 2020-11-28T21:53:37.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.260+0000 7f3116d9d700 1 -- 172.21.15.162:0/2051864155 learned_addr learned my addr 172.21.15.162:0/2051864155 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:37.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.260+0000 7f3115d9b700 1 --2- 172.21.15.162:0/2051864155 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f31180b2bb0 0x7f3118151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:37.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.260+0000 7f311659c700 1 --2- 172.21.15.162:0/2051864155 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3118062260 0x7f31181415d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:37.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.260+0000 7f3116d9d700 1 -- 172.21.15.162:0/2051864155 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3118062260 msgr2=0x7f31181415d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:37.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.260+0000 7f3116d9d700 1 --2- 172.21.15.162:0/2051864155 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3118062260 0x7f31181415d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:37.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.260+0000 7f3116d9d700 1 -- 172.21.15.162:0/2051864155 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f31180b2bb0 msgr2=0x7f3118151620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:37.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.260+0000 7f3116d9d700 1 --2- 172.21.15.162:0/2051864155 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f31180b2bb0 0x7f3118151620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:37.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.260+0000 7f3116d9d700 1 -- 172.21.15.162:0/2051864155 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31180ad8f0 con 0x7f31180b2f90 2020-11-28T21:53:37.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.260+0000 7f3116d9d700 1 --2- 172.21.15.162:0/2051864155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31180b2f90 0x7f3118151b10 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 rx=0x7f311813f910 tx=0x7f3110007b30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:37.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.260+0000 7f310f7fe700 1 -- 172.21.15.162:0/2051864155 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3110013410 con 0x7f31180b2f90 2020-11-28T21:53:37.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.260+0000 7f310f7fe700 1 -- 172.21.15.162:0/2051864155 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3110013570 con 0x7f31180b2f90 2020-11-28T21:53:37.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.260+0000 7f311cbdf700 1 -- 172.21.15.162:0/2051864155 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31181425b0 con 0x7f31180b2f90 2020-11-28T21:53:37.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.260+0000 7f310f7fe700 1 -- 172.21.15.162:0/2051864155 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31100138d0 con 0x7f31180b2f90 2020-11-28T21:53:37.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.260+0000 7f311cbdf700 1 -- 172.21.15.162:0/2051864155 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31180add90 con 0x7f31180b2f90 2020-11-28T21:53:37.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.260+0000 7f311cbdf700 1 -- 172.21.15.162:0/2051864155 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31180ada80 con 0x7f31180b2f90 2020-11-28T21:53:37.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.260+0000 7f310f7fe700 1 -- 172.21.15.162:0/2051864155 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f3110010030 con 0x7f31180b2f90 2020-11-28T21:53:37.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.264+0000 7f310f7fe700 1 --2- 172.21.15.162:0/2051864155 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f30fc0553b0 0x7f30fc057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:37.273 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.264+0000 7f310f7fe700 1 -- 172.21.15.162:0/2051864155 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3110001e40 con 0x7f31180b2f90 2020-11-28T21:53:37.273 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.264+0000 7f311659c700 1 --2- 172.21.15.162:0/2051864155 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f30fc0553b0 0x7f30fc057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:37.273 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.264+0000 7f311659c700 1 --2- 172.21.15.162:0/2051864155 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f30fc0553b0 0x7f30fc057870 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f310800c000 tx=0x7f3108008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:37.274 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.268+0000 7f310f7fe700 1 -- 172.21.15.162:0/2051864155 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f3110043f90 con 0x7f31180b2f90 2020-11-28T21:53:37.457 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:37.452+0000 7f311cbdf700 1 -- 172.21.15.162:0/2051864155 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f3118057050 con 0x7f31180b2f90 2020-11-28T21:53:38.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.040+0000 7f310f7fe700 1 -- 172.21.15.162:0/2051864155 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v40) v1 ==== 118+0+0 (secure 0 0 0) 0x7f31100475b0 con 0x7f31180b2f90 2020-11-28T21:53:38.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.044+0000 7f311cbdf700 1 -- 172.21.15.162:0/2051864155 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f30fc0553b0 msgr2=0x7f30fc057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:38.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.044+0000 7f311cbdf700 1 --2- 172.21.15.162:0/2051864155 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f30fc0553b0 0x7f30fc057870 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f310800c000 tx=0x7f3108008040).stop 2020-11-28T21:53:38.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.044+0000 7f311cbdf700 1 -- 172.21.15.162:0/2051864155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31180b2f90 msgr2=0x7f3118151b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:38.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.044+0000 7f311cbdf700 1 --2- 172.21.15.162:0/2051864155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31180b2f90 0x7f3118151b10 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 rx=0x7f311813f910 tx=0x7f3110007b30).stop 2020-11-28T21:53:38.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.044+0000 7f311cbdf700 1 -- 172.21.15.162:0/2051864155 shutdown_connections 2020-11-28T21:53:38.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.044+0000 7f311cbdf700 1 --2- 172.21.15.162:0/2051864155 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f30fc0553b0 0x7f30fc057870 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:38.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.044+0000 7f311cbdf700 1 --2- 172.21.15.162:0/2051864155 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3118062260 0x7f31181415d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:38.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.044+0000 7f311cbdf700 1 --2- 172.21.15.162:0/2051864155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31180b2f90 0x7f3118151b10 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:38.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.044+0000 7f311cbdf700 1 --2- 172.21.15.162:0/2051864155 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f31180b2bb0 0x7f3118151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:38.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.044+0000 7f311cbdf700 1 -- 172.21.15.162:0/2051864155 >> 172.21.15.162:0/2051864155 conn(0x7f3118061740 msgr2=0x7f311805c350 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:38.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.044+0000 7f311cbdf700 1 -- 172.21.15.162:0/2051864155 shutdown_connections 2020-11-28T21:53:38.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.044+0000 7f311cbdf700 1 -- 172.21.15.162:0/2051864155 wait complete. 2020-11-28T21:53:38.068 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:53:38.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.184+0000 7f49c5d1f700 1 Processor -- start 2020-11-28T21:53:38.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.184+0000 7f49c5d1f700 1 -- start start 2020-11-28T21:53:38.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.184+0000 7f49c5d1f700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f49c00b9300 0x7f49c0146df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:38.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.184+0000 7f49c5d1f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f49c00b30c0 0x7f49c01472e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:38.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.184+0000 7f49c5d1f700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f49c00b4110 0x7f49c00b6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:38.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.184+0000 7f49c5d1f700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c00b23e0 con 0x7f49c00b4110 2020-11-28T21:53:38.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.184+0000 7f49c5d1f700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c01477d0 con 0x7f49c00b9300 2020-11-28T21:53:38.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.184+0000 7f49c5d1f700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c00b35f0 con 0x7f49c00b30c0 2020-11-28T21:53:38.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.184+0000 7f49bf7fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f49c00b9300 0x7f49c0146df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:38.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.184+0000 7f49bf7fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f49c00b9300 0x7f49c0146df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42110/0 (socket says 172.21.15.162:42110) 2020-11-28T21:53:38.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.184+0000 7f49bf7fe700 1 -- 172.21.15.162:0/1218087535 learned_addr learned my addr 172.21.15.162:0/1218087535 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:38.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.184+0000 7f49bffff700 1 --2- 172.21.15.162:0/1218087535 >> v2:172.21.15.118:3300/0 conn(0x7f49c00b4110 0x7f49c00b6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:38.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.184+0000 7f49beffd700 1 --2- 172.21.15.162:0/1218087535 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f49c00b30c0 0x7f49c01472e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:38.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.184+0000 7f49bf7fe700 1 -- 172.21.15.162:0/1218087535 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f49c00b30c0 msgr2=0x7f49c01472e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:38.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.184+0000 7f49bf7fe700 1 --2- 172.21.15.162:0/1218087535 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f49c00b30c0 0x7f49c01472e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:38.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.184+0000 7f49bf7fe700 1 -- 172.21.15.162:0/1218087535 >> v2:172.21.15.118:3300/0 conn(0x7f49c00b4110 msgr2=0x7f49c00b6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:38.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.184+0000 7f49bf7fe700 1 --2- 172.21.15.162:0/1218087535 >> v2:172.21.15.118:3300/0 conn(0x7f49c00b4110 0x7f49c00b6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:38.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.184+0000 7f49bf7fe700 1 -- 172.21.15.162:0/1218087535 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c00b2640 con 0x7f49c00b9300 2020-11-28T21:53:38.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.184+0000 7f49beffd700 1 --2- 172.21.15.162:0/1218087535 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f49c00b30c0 0x7f49c01472e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:53:38.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.184+0000 7f49bf7fe700 1 --2- 172.21.15.162:0/1218087535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49c00b9300 0x7f49c0146df0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7f49b0002f30 tx=0x7f49b0006e80).ready entity=mon.1 client_cookie=b7265a3411417816 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:38.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.184+0000 7f49be7fc700 1 -- 172.21.15.162:0/1218087535 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49b00097f0 con 0x7f49c00b9300 2020-11-28T21:53:38.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.184+0000 7f49be7fc700 1 -- 172.21.15.162:0/1218087535 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b0009950 con 0x7f49c00b9300 2020-11-28T21:53:38.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.184+0000 7f49be7fc700 1 -- 172.21.15.162:0/1218087535 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49b0009cb0 con 0x7f49c00b9300 2020-11-28T21:53:38.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.184+0000 7f49c5d1f700 1 -- 172.21.15.162:0/1218087535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49c00b9300 msgr2=0x7f49c0146df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:38.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.184+0000 7f49c5d1f700 1 --2- 172.21.15.162:0/1218087535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49c00b9300 0x7f49c0146df0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7f49b0002f30 tx=0x7f49b0006e80).stop 2020-11-28T21:53:38.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.184+0000 7f49c5d1f700 1 -- 172.21.15.162:0/1218087535 shutdown_connections 2020-11-28T21:53:38.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.184+0000 7f49c5d1f700 1 --2- 172.21.15.162:0/1218087535 >> v2:172.21.15.118:3300/0 conn(0x7f49c00b4110 0x7f49c00b6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:38.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.184+0000 7f49c5d1f700 1 --2- 172.21.15.162:0/1218087535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49c00b9300 0x7f49c0146df0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:38.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.184+0000 7f49c5d1f700 1 --2- 172.21.15.162:0/1218087535 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f49c00b30c0 0x7f49c01472e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:38.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.184+0000 7f49c5d1f700 1 -- 172.21.15.162:0/1218087535 >> 172.21.15.162:0/1218087535 conn(0x7f49c0066c70 msgr2=0x7f49c0067050 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:38.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.184+0000 7f49c5d1f700 1 -- 172.21.15.162:0/1218087535 shutdown_connections 2020-11-28T21:53:38.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.184+0000 7f49c5d1f700 1 -- 172.21.15.162:0/1218087535 wait complete. 2020-11-28T21:53:38.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.188+0000 7f49c5d1f700 1 Processor -- start 2020-11-28T21:53:38.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.188+0000 7f49c5d1f700 1 -- start start 2020-11-28T21:53:38.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.188+0000 7f49c5d1f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49c00b30c0 0x7f49c0146ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:38.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.188+0000 7f49c5d1f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f49c00b4110 0x7f49c0158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:38.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.188+0000 7f49c5d1f700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49c00b9300 0x7f49c00640e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:38.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.188+0000 7f49c5d1f700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c0060750 con 0x7f49c00b9300 2020-11-28T21:53:38.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.188+0000 7f49c5d1f700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c0060980 con 0x7f49c00b30c0 2020-11-28T21:53:38.200 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.188+0000 7f49c5d1f700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c0060c20 con 0x7f49c00b4110 2020-11-28T21:53:38.200 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.188+0000 7f49bf7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49c00b30c0 0x7f49c0146ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:38.200 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.188+0000 7f49bf7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49c00b30c0 0x7f49c0146ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42116/0 (socket says 172.21.15.162:42116) 2020-11-28T21:53:38.200 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.188+0000 7f49bf7fe700 1 -- 172.21.15.162:0/3794505472 learned_addr learned my addr 172.21.15.162:0/3794505472 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:38.200 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.188+0000 7f49beffd700 1 --2- 172.21.15.162:0/3794505472 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f49c00b4110 0x7f49c0158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:38.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.188+0000 7f49bffff700 1 --2- 172.21.15.162:0/3794505472 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49c00b9300 0x7f49c00640e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:38.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.188+0000 7f49bf7fe700 1 -- 172.21.15.162:0/3794505472 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f49c00b4110 msgr2=0x7f49c0158ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:38.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.188+0000 7f49bf7fe700 1 --2- 172.21.15.162:0/3794505472 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f49c00b4110 0x7f49c0158ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:38.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.188+0000 7f49bf7fe700 1 -- 172.21.15.162:0/3794505472 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49c00b9300 msgr2=0x7f49c00640e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:38.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.188+0000 7f49bf7fe700 1 --2- 172.21.15.162:0/3794505472 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49c00b9300 0x7f49c00640e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:38.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.188+0000 7f49bf7fe700 1 -- 172.21.15.162:0/3794505472 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c0063c40 con 0x7f49c00b30c0 2020-11-28T21:53:38.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.188+0000 7f49bf7fe700 1 --2- 172.21.15.162:0/3794505472 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49c00b30c0 0x7f49c0146ed0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 rx=0x7f49b0002f30 tx=0x7f49b0015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:38.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.188+0000 7f49bcff9700 1 -- 172.21.15.162:0/3794505472 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49b0011bd0 con 0x7f49c00b30c0 2020-11-28T21:53:38.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.188+0000 7f49bcff9700 1 -- 172.21.15.162:0/3794505472 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b0011d30 con 0x7f49c00b30c0 2020-11-28T21:53:38.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.188+0000 7f49bcff9700 1 -- 172.21.15.162:0/3794505472 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49b0011ea0 con 0x7f49c00b30c0 2020-11-28T21:53:38.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.188+0000 7f49c5d1f700 1 -- 172.21.15.162:0/3794505472 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c0060ef0 con 0x7f49c00b30c0 2020-11-28T21:53:38.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.188+0000 7f49c5d1f700 1 -- 172.21.15.162:0/3794505472 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c0144730 con 0x7f49c00b30c0 2020-11-28T21:53:38.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.192+0000 7f49c5d1f700 1 -- 172.21.15.162:0/3794505472 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49c0063dd0 con 0x7f49c00b30c0 2020-11-28T21:53:38.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.192+0000 7f49bcff9700 1 -- 172.21.15.162:0/3794505472 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f49b001a040 con 0x7f49c00b30c0 2020-11-28T21:53:38.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.192+0000 7f49bcff9700 1 --2- 172.21.15.162:0/3794505472 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4998055330 0x7f49980577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:38.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.192+0000 7f49bcff9700 1 -- 172.21.15.162:0/3794505472 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f49b006f0b0 con 0x7f49c00b30c0 2020-11-28T21:53:38.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.192+0000 7f49beffd700 1 --2- 172.21.15.162:0/3794505472 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4998055330 0x7f49980577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:38.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.192+0000 7f49beffd700 1 --2- 172.21.15.162:0/3794505472 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4998055330 0x7f49980577f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f49b800d000 tx=0x7f49b8008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:38.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.200+0000 7f49bcff9700 1 -- 172.21.15.162:0/3794505472 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f49b0045220 con 0x7f49c00b30c0 2020-11-28T21:53:38.394 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.392+0000 7f49c5d1f700 1 -- 172.21.15.162:0/3794505472 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49c0057050 con 0x7f49c00b30c0 2020-11-28T21:53:38.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.392+0000 7f49bcff9700 1 -- 172.21.15.162:0/3794505472 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+2502 (secure 0 0 0) 0x7f49b0023070 con 0x7f49c00b30c0 2020-11-28T21:53:38.396 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:53:38.397 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":40,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5795,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/2645168518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":2645168518},{"type":"v1","addr":"172.21.15.118:6833","nonce":2645168518}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36},{"gid":6541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1999073284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1999073284},{"type":"v1","addr":"172.21.15.162:6835","nonce":1999073284}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:53:36.539204+0000","modified":"2020-11-28T21:53:38.035411+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6550":{"gid":6550,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/1097065372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1097065372},{"type":"v1","addr":"172.21.15.162:6837","nonce":1097065372}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-28T21:53:38.399 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.392+0000 7f49c5d1f700 1 -- 172.21.15.162:0/3794505472 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4998055330 msgr2=0x7f49980577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:38.399 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.392+0000 7f49c5d1f700 1 --2- 172.21.15.162:0/3794505472 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4998055330 0x7f49980577f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f49b800d000 tx=0x7f49b8008040).stop 2020-11-28T21:53:38.400 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.392+0000 7f49c5d1f700 1 -- 172.21.15.162:0/3794505472 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49c00b30c0 msgr2=0x7f49c0146ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:38.400 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.392+0000 7f49c5d1f700 1 --2- 172.21.15.162:0/3794505472 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49c00b30c0 0x7f49c0146ed0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 rx=0x7f49b0002f30 tx=0x7f49b0015000).stop 2020-11-28T21:53:38.400 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.392+0000 7f49c5d1f700 1 -- 172.21.15.162:0/3794505472 shutdown_connections 2020-11-28T21:53:38.401 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.396+0000 7f49c5d1f700 1 --2- 172.21.15.162:0/3794505472 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4998055330 0x7f49980577f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:38.401 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.396+0000 7f49c5d1f700 1 --2- 172.21.15.162:0/3794505472 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49c00b9300 0x7f49c00640e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:38.402 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.396+0000 7f49c5d1f700 1 --2- 172.21.15.162:0/3794505472 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49c00b30c0 0x7f49c0146ed0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:38.402 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.396+0000 7f49c5d1f700 1 --2- 172.21.15.162:0/3794505472 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f49c00b4110 0x7f49c0158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:38.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.396+0000 7f49c5d1f700 1 -- 172.21.15.162:0/3794505472 >> 172.21.15.162:0/3794505472 conn(0x7f49c0066c70 msgr2=0x7f49c00b0390 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:38.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.396+0000 7f49c5d1f700 1 -- 172.21.15.162:0/3794505472 shutdown_connections 2020-11-28T21:53:38.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.396+0000 7f49c5d1f700 1 -- 172.21.15.162:0/3794505472 wait complete. 2020-11-28T21:53:38.404 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 40 2020-11-28T21:53:38.417 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:53:38.527 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.520+0000 7f3dfdb9a700 1 Processor -- start 2020-11-28T21:53:38.527 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.520+0000 7f3dfdb9a700 1 -- start start 2020-11-28T21:53:38.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.520+0000 7f3dfdb9a700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f3df80615c0 0x7f3df80b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:38.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.520+0000 7f3dfdb9a700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3df80b1790 con 0x7f3df80611e0 2020-11-28T21:53:38.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.520+0000 7f3dfdb9a700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3df80b1a30 con 0x7f3df80b4b40 2020-11-28T21:53:38.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.520+0000 7f3dfdb9a700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3df80b3f90 con 0x7f3df80615c0 2020-11-28T21:53:38.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.520+0000 7f3dfd399700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f3df80b4b40 0x7f3df813ee60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:48970/0 (socket says 172.21.15.162:48970) 2020-11-28T21:53:38.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.520+0000 7f3dfd399700 1 -- 172.21.15.162:0/2438344344 learned_addr learned my addr 172.21.15.162:0/2438344344 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:38.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.520+0000 7f3df7fff700 1 --2- 172.21.15.162:0/2438344344 >> v2:172.21.15.118:3300/0 conn(0x7f3df80615c0 0x7f3df80b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:38.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.520+0000 7f3df77fe700 1 -- 172.21.15.162:0/2438344344 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 813364539 0 0) 0x7f3df80b1a30 con 0x7f3df80b4b40 2020-11-28T21:53:38.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.520+0000 7f3df77fe700 1 -- 172.21.15.162:0/2438344344 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ddc003230 con 0x7f3df80b4b40 2020-11-28T21:53:38.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.524+0000 7f3df77fe700 1 -- 172.21.15.162:0/2438344344 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1786797462 0 0) 0x7f3ddc003230 con 0x7f3df80b4b40 2020-11-28T21:53:38.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.524+0000 7f3df7fff700 1 -- 172.21.15.162:0/2438344344 >> v1:172.21.15.162:6789/0 conn(0x7f3df80b4b40 legacy=0x7f3df813ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:38.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.524+0000 7f3df7fff700 1 -- 172.21.15.162:0/2438344344 >> v1:172.21.15.118:6789/0 conn(0x7f3df80611e0 legacy=0x7f3df80b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:38.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.524+0000 7f3df7fff700 1 -- 172.21.15.162:0/2438344344 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df80b4f60 con 0x7f3df80615c0 2020-11-28T21:53:38.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.524+0000 7f3df7fff700 1 --2- 172.21.15.162:0/2438344344 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3df80615c0 0x7f3df80b10f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f3df0001bf0 tx=0x7f3de4006dc0).ready entity=mon.0 client_cookie=a7adb04fd9705eaa server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:38.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.524+0000 7f3df77fe700 1 -- 172.21.15.162:0/2438344344 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3df0001a50 con 0x7f3df80615c0 2020-11-28T21:53:38.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.524+0000 7f3df77fe700 1 -- 172.21.15.162:0/2438344344 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3de4009670 con 0x7f3df80615c0 2020-11-28T21:53:38.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.524+0000 7f3df77fe700 1 -- 172.21.15.162:0/2438344344 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3de40097e0 con 0x7f3df80615c0 2020-11-28T21:53:38.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.524+0000 7f3dfdb9a700 1 -- 172.21.15.162:0/2438344344 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3df80615c0 msgr2=0x7f3df80b10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:38.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.524+0000 7f3dfdb9a700 1 --2- 172.21.15.162:0/2438344344 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3df80615c0 0x7f3df80b10f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f3df0001bf0 tx=0x7f3de4006dc0).stop 2020-11-28T21:53:38.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.524+0000 7f3dfdb9a700 1 -- 172.21.15.162:0/2438344344 shutdown_connections 2020-11-28T21:53:38.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.524+0000 7f3dfdb9a700 1 --2- 172.21.15.162:0/2438344344 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3df80615c0 0x7f3df80b10f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:38.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.524+0000 7f3dfdb9a700 1 -- 172.21.15.162:0/2438344344 >> 172.21.15.162:0/2438344344 conn(0x7f3df80b54f0 msgr2=0x7f3df80b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:38.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.524+0000 7f3dfdb9a700 1 -- 172.21.15.162:0/2438344344 shutdown_connections 2020-11-28T21:53:38.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.524+0000 7f3dfdb9a700 1 -- 172.21.15.162:0/2438344344 wait complete. 2020-11-28T21:53:38.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.528+0000 7f3dfdb9a700 1 Processor -- start 2020-11-28T21:53:38.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.528+0000 7f3dfdb9a700 1 -- start start 2020-11-28T21:53:38.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.528+0000 7f3dfdb9a700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3df80611e0 0x7f3df8066ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:38.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.528+0000 7f3dfdb9a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3df80615c0 0x7f3df80ab270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:38.543 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.528+0000 7f3dfdb9a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3df80b4b40 0x7f3df8151800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:38.543 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.528+0000 7f3dfdb9a700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df80af140 con 0x7f3df80b4b40 2020-11-28T21:53:38.543 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.528+0000 7f3dfdb9a700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df80af370 con 0x7f3df80615c0 2020-11-28T21:53:38.543 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.528+0000 7f3dfdb9a700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3df80af610 con 0x7f3df80611e0 2020-11-28T21:53:38.544 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.528+0000 7f3df7fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3df80615c0 0x7f3df80ab270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:38.544 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.528+0000 7f3df7fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3df80615c0 0x7f3df80ab270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42132/0 (socket says 172.21.15.162:42132) 2020-11-28T21:53:38.544 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.528+0000 7f3df7fff700 1 -- 172.21.15.162:0/770728438 learned_addr learned my addr 172.21.15.162:0/770728438 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:38.544 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.528+0000 7f3dfd399700 1 --2- 172.21.15.162:0/770728438 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3df80b4b40 0x7f3df8151800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:38.545 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.528+0000 7f3dfcb98700 1 --2- 172.21.15.162:0/770728438 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3df80611e0 0x7f3df8066ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:38.545 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.528+0000 7f3df7fff700 1 -- 172.21.15.162:0/770728438 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3df80611e0 msgr2=0x7f3df8066ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:38.545 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.528+0000 7f3df7fff700 1 --2- 172.21.15.162:0/770728438 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3df80611e0 0x7f3df8066ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:38.545 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.528+0000 7f3df7fff700 1 -- 172.21.15.162:0/770728438 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3df80b4b40 msgr2=0x7f3df8151800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:38.545 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.528+0000 7f3df7fff700 1 --2- 172.21.15.162:0/770728438 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3df80b4b40 0x7f3df8151800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:38.546 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.528+0000 7f3df7fff700 1 -- 172.21.15.162:0/770728438 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df80677f0 con 0x7f3df80615c0 2020-11-28T21:53:38.546 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.528+0000 7f3df7fff700 1 --2- 172.21.15.162:0/770728438 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3df80615c0 0x7f3df80ab270 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 rx=0x7f3df0001bf0 tx=0x7f3de4015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:38.546 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.528+0000 7f3df5ffb700 1 -- 172.21.15.162:0/770728438 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3de4020070 con 0x7f3df80615c0 2020-11-28T21:53:38.546 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.528+0000 7f3dfdb9a700 1 -- 172.21.15.162:0/770728438 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3df8145b60 con 0x7f3df80615c0 2020-11-28T21:53:38.547 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.528+0000 7f3df5ffb700 1 -- 172.21.15.162:0/770728438 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3de4011c90 con 0x7f3df80615c0 2020-11-28T21:53:38.547 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.528+0000 7f3dfdb9a700 1 -- 172.21.15.162:0/770728438 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df8140250 con 0x7f3df80615c0 2020-11-28T21:53:38.547 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.528+0000 7f3df5ffb700 1 -- 172.21.15.162:0/770728438 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3de401c470 con 0x7f3df80615c0 2020-11-28T21:53:38.547 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.532+0000 7f3dfdb9a700 1 -- 172.21.15.162:0/770728438 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3df80af7d0 con 0x7f3df80615c0 2020-11-28T21:53:38.548 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.532+0000 7f3df5ffb700 1 -- 172.21.15.162:0/770728438 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f3de400f070 con 0x7f3df80615c0 2020-11-28T21:53:38.548 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.532+0000 7f3df5ffb700 1 --2- 172.21.15.162:0/770728438 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3de0055350 0x7f3de0057810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:38.548 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.532+0000 7f3df5ffb700 1 -- 172.21.15.162:0/770728438 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3de406f360 con 0x7f3df80615c0 2020-11-28T21:53:38.548 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.532+0000 7f3dfcb98700 1 --2- 172.21.15.162:0/770728438 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3de0055350 0x7f3de0057810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:38.548 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.532+0000 7f3dfcb98700 1 --2- 172.21.15.162:0/770728438 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3de0055350 0x7f3de0057810 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f3dec00d000 tx=0x7f3dec007b60).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:38.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.536+0000 7f3df5ffb700 1 -- 172.21.15.162:0/770728438 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f3de40455c0 con 0x7f3df80615c0 2020-11-28T21:53:38.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.672+0000 7f3dfdb9a700 1 -- 172.21.15.162:0/770728438 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3df8057050 con 0x7f3df80615c0 2020-11-28T21:53:38.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.676+0000 7f3df5ffb700 1 -- 172.21.15.162:0/770728438 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16137 (secure 0 0 0) 0x7f3de4048be0 con 0x7f3df80615c0 2020-11-28T21:53:38.680 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:53:38.680 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":88,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:53:37.541898+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:53:34.083725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2020-11-28T21:53:35.380493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:53:38.682 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.676+0000 7f3dfdb9a700 1 -- 172.21.15.162:0/770728438 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3de0055350 msgr2=0x7f3de0057810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:38.682 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.676+0000 7f3dfdb9a700 1 --2- 172.21.15.162:0/770728438 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3de0055350 0x7f3de0057810 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f3dec00d000 tx=0x7f3dec007b60).stop 2020-11-28T21:53:38.682 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.676+0000 7f3dfdb9a700 1 -- 172.21.15.162:0/770728438 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3df80615c0 msgr2=0x7f3df80ab270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:38.682 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.676+0000 7f3dfdb9a700 1 --2- 172.21.15.162:0/770728438 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3df80615c0 0x7f3df80ab270 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 rx=0x7f3df0001bf0 tx=0x7f3de4015000).stop 2020-11-28T21:53:38.683 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.676+0000 7f3dfdb9a700 1 -- 172.21.15.162:0/770728438 shutdown_connections 2020-11-28T21:53:38.683 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.676+0000 7f3dfdb9a700 1 --2- 172.21.15.162:0/770728438 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3de0055350 0x7f3de0057810 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:38.683 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.676+0000 7f3dfdb9a700 1 --2- 172.21.15.162:0/770728438 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3df80611e0 0x7f3df8066ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:38.684 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.676+0000 7f3dfdb9a700 1 --2- 172.21.15.162:0/770728438 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3df80b4b40 0x7f3df8151800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:38.684 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.676+0000 7f3dfdb9a700 1 --2- 172.21.15.162:0/770728438 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3df80615c0 0x7f3df80ab270 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:38.684 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.676+0000 7f3dfdb9a700 1 -- 172.21.15.162:0/770728438 >> 172.21.15.162:0/770728438 conn(0x7f3df80b54f0 msgr2=0x7f3df813edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:38.685 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.676+0000 7f3dfdb9a700 1 -- 172.21.15.162:0/770728438 shutdown_connections 2020-11-28T21:53:38.685 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.676+0000 7f3dfdb9a700 1 -- 172.21.15.162:0/770728438 wait complete. 2020-11-28T21:53:38.693 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-11-28T21:53:38.709 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:53:38.712 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:53:38.715 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:53:38.744 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:53:38.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f7787048700 1 Processor -- start 2020-11-28T21:53:38.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f7787048700 1 -- start start 2020-11-28T21:53:38.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f7787048700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f77800b2e80 0x7f778005f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:38.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f7787048700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f77800b2560 0x7f778005f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:38.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f7787048700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77800ab0b0 con 0x7f77800acb40 2020-11-28T21:53:38.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f7787048700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f778005fd60 con 0x7f77800b2560 2020-11-28T21:53:38.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f7787048700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77800b3260 con 0x7f77800b2e80 2020-11-28T21:53:38.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f77882aa700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f77800acb40 0x7f7780060d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:48986/0 (socket says 172.21.15.162:48986) 2020-11-28T21:53:38.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f77882aa700 1 -- 172.21.15.162:0/1834811693 learned_addr learned my addr 172.21.15.162:0/1834811693 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:38.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f7786847700 1 --2- 172.21.15.162:0/1834811693 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f77800b2e80 0x7f778005f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:38.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f7786046700 1 --2- 172.21.15.162:0/1834811693 >> v2:172.21.15.118:3300/0 conn(0x7f77800b2560 0x7f778005f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:38.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f7785845700 1 -- 172.21.15.162:0/1834811693 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 422417295 0 0) 0x7f77800ab0b0 con 0x7f77800acb40 2020-11-28T21:53:38.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f7785845700 1 -- 172.21.15.162:0/1834811693 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f776c003230 con 0x7f77800acb40 2020-11-28T21:53:38.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f7785845700 1 -- 172.21.15.162:0/1834811693 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 279299960 0 0) 0x7f776c003230 con 0x7f77800acb40 2020-11-28T21:53:38.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f7785845700 1 -- 172.21.15.162:0/1834811693 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f77800b2e80 msgr2=0x7f778005f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:38.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f7785845700 1 --2- 172.21.15.162:0/1834811693 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f77800b2e80 0x7f778005f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:38.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f7785845700 1 -- 172.21.15.162:0/1834811693 >> v2:172.21.15.118:3300/0 conn(0x7f77800b2560 msgr2=0x7f778005f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:38.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f7785845700 1 --2- 172.21.15.162:0/1834811693 >> v2:172.21.15.118:3300/0 conn(0x7f77800b2560 0x7f778005f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:38.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f7785845700 1 -- 172.21.15.162:0/1834811693 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77800ab2e0 con 0x7f77800acb40 2020-11-28T21:53:38.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f7785845700 1 -- 172.21.15.162:0/1834811693 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f7778001a50 con 0x7f77800acb40 2020-11-28T21:53:38.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f7785845700 1 -- 172.21.15.162:0/1834811693 >> v1:172.21.15.162:6789/0 conn(0x7f77800acb40 legacy=0x7f7780060d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:38.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f7785845700 1 --2- 172.21.15.162:0/1834811693 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f776c003720 0x7f776c003b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:38.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f7785845700 1 --2- 172.21.15.162:0/1834811693 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f77800b2e80 0x7f776c0044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:38.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f7785845700 1 --2- 172.21.15.162:0/1834811693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f776c0049c0 0x7f776c004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:38.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f7785845700 1 -- 172.21.15.162:0/1834811693 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778005fd60 con 0x7f776c003720 2020-11-28T21:53:38.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f7785845700 1 -- 172.21.15.162:0/1834811693 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77800b3260 con 0x7f776c0049c0 2020-11-28T21:53:38.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f7785845700 1 -- 172.21.15.162:0/1834811693 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f776c004130 con 0x7f77800b2e80 2020-11-28T21:53:38.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f77882aa700 1 --2- 172.21.15.162:0/1834811693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f776c0049c0 0x7f776c004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:38.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f7786847700 1 --2- 172.21.15.162:0/1834811693 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f776c003720 0x7f776c003b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:38.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f7786046700 1 --2- 172.21.15.162:0/1834811693 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f77800b2e80 0x7f776c0044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:38.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f77882aa700 1 -- 172.21.15.162:0/1834811693 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f77800b2e80 msgr2=0x7f776c0044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:38.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f77882aa700 1 --2- 172.21.15.162:0/1834811693 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f77800b2e80 0x7f776c0044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:38.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f77882aa700 1 -- 172.21.15.162:0/1834811693 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f776c003720 msgr2=0x7f776c003b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:38.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f77882aa700 1 --2- 172.21.15.162:0/1834811693 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f776c003720 0x7f776c003b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:38.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f77882aa700 1 -- 172.21.15.162:0/1834811693 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f776c003230 con 0x7f776c0049c0 2020-11-28T21:53:38.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f7786046700 1 --2- 172.21.15.162:0/1834811693 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f77800b2e80 0x7f776c0044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:53:38.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f7786847700 1 --2- 172.21.15.162:0/1834811693 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f776c003720 0x7f776c003b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:53:38.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f77882aa700 1 --2- 172.21.15.162:0/1834811693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f776c0049c0 0x7f776c004e40 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 rx=0x7f7778001760 tx=0x7f77800b2480).ready entity=mon.1 client_cookie=165817ae09953ade server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:38.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f7785845700 1 -- 172.21.15.162:0/1834811693 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7778005b70 con 0x7f776c0049c0 2020-11-28T21:53:38.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f7785845700 1 -- 172.21.15.162:0/1834811693 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7778005cd0 con 0x7f776c0049c0 2020-11-28T21:53:38.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f7785845700 1 -- 172.21.15.162:0/1834811693 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7778008360 con 0x7f776c0049c0 2020-11-28T21:53:38.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f7787048700 1 -- 172.21.15.162:0/1834811693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f776c0049c0 msgr2=0x7f776c004e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:38.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f7787048700 1 --2- 172.21.15.162:0/1834811693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f776c0049c0 0x7f776c004e40 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 rx=0x7f7778001760 tx=0x7f77800b2480).stop 2020-11-28T21:53:38.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.812+0000 7f77882aa700 1 -- 172.21.15.162:0/1834811693 reap_dead start 2020-11-28T21:53:38.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.816+0000 7f7787048700 1 -- 172.21.15.162:0/1834811693 shutdown_connections 2020-11-28T21:53:38.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.816+0000 7f7787048700 1 -- 172.21.15.162:0/1834811693 >> 172.21.15.162:0/1834811693 conn(0x7f7780063b60 msgr2=0x7f7780063f40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:38.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.816+0000 7f7787048700 1 -- 172.21.15.162:0/1834811693 shutdown_connections 2020-11-28T21:53:38.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.816+0000 7f7787048700 1 -- 172.21.15.162:0/1834811693 wait complete. 2020-11-28T21:53:38.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.816+0000 7f7787048700 1 Processor -- start 2020-11-28T21:53:38.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.816+0000 7f7787048700 1 -- start start 2020-11-28T21:53:38.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.816+0000 7f7787048700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7780061070 0x7f7780066f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:38.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.816+0000 7f7787048700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7780065db0 0x7f7780066230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:38.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.816+0000 7f7787048700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7780143c20 0x7f77801440a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:38.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.816+0000 7f7787048700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7780061450 con 0x7f7780143c20 2020-11-28T21:53:38.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.816+0000 7f7787048700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77800ae010 con 0x7f7780065db0 2020-11-28T21:53:38.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.816+0000 7f7787048700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7780067780 con 0x7f7780061070 2020-11-28T21:53:38.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.816+0000 7f7786046700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7780065db0 0x7f7780066230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:38.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.816+0000 7f7786046700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7780065db0 0x7f7780066230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42152/0 (socket says 172.21.15.162:42152) 2020-11-28T21:53:38.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.816+0000 7f7786046700 1 -- 172.21.15.162:0/262851061 learned_addr learned my addr 172.21.15.162:0/262851061 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:38.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.816+0000 7f7786847700 1 --2- 172.21.15.162:0/262851061 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7780061070 0x7f7780066f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:38.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.816+0000 7f77882aa700 1 --2- 172.21.15.162:0/262851061 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7780143c20 0x7f77801440a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:38.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.820+0000 7f7786046700 1 -- 172.21.15.162:0/262851061 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7780061070 msgr2=0x7f7780066f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:38.838 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.820+0000 7f7786046700 1 --2- 172.21.15.162:0/262851061 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7780061070 0x7f7780066f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:38.838 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.820+0000 7f7786046700 1 -- 172.21.15.162:0/262851061 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7780143c20 msgr2=0x7f77801440a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:38.838 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.820+0000 7f7786046700 1 --2- 172.21.15.162:0/262851061 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7780143c20 0x7f77801440a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:38.838 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.820+0000 7f7786046700 1 -- 172.21.15.162:0/262851061 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77800af390 con 0x7f7780065db0 2020-11-28T21:53:38.839 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.820+0000 7f7786046700 1 --2- 172.21.15.162:0/262851061 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7780065db0 0x7f7780066230 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 rx=0x7f7770008000 tx=0x7f777000b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:38.839 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.820+0000 7f77777fe700 1 -- 172.21.15.162:0/262851061 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f77700116c0 con 0x7f7780065db0 2020-11-28T21:53:38.839 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.820+0000 7f7787048700 1 -- 172.21.15.162:0/262851061 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77800af630 con 0x7f7780065db0 2020-11-28T21:53:38.839 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.820+0000 7f77777fe700 1 -- 172.21.15.162:0/262851061 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7770011820 con 0x7f7780065db0 2020-11-28T21:53:38.840 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.820+0000 7f77777fe700 1 -- 172.21.15.162:0/262851061 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7770011b80 con 0x7f7780065db0 2020-11-28T21:53:38.840 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.820+0000 7f7787048700 1 -- 172.21.15.162:0/262851061 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7780066720 con 0x7f7780065db0 2020-11-28T21:53:38.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.820+0000 7f7787048700 1 -- 172.21.15.162:0/262851061 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7780057050 con 0x7f7780065db0 2020-11-28T21:53:38.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.820+0000 7f77777fe700 1 -- 172.21.15.162:0/262851061 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7770011ce0 con 0x7f7780065db0 2020-11-28T21:53:38.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.820+0000 7f77777fe700 1 --2- 172.21.15.162:0/262851061 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7768055350 0x7f7768057810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:38.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.820+0000 7f77777fe700 1 -- 172.21.15.162:0/262851061 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f777006dec0 con 0x7f7780065db0 2020-11-28T21:53:38.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.820+0000 7f7786847700 1 --2- 172.21.15.162:0/262851061 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7768055350 0x7f7768057810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:38.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.820+0000 7f7786847700 1 --2- 172.21.15.162:0/262851061 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7768055350 0x7f7768057810 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f777c008000 tx=0x7f777c008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:38.843 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:38.828+0000 7f77777fe700 1 -- 172.21.15.162:0/262851061 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7770042df0 con 0x7f7780065db0 2020-11-28T21:53:39.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.024+0000 7f7787048700 1 -- 172.21.15.162:0/262851061 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f7780068350 con 0x7f7780065db0 2020-11-28T21:53:39.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.032+0000 7f77777fe700 1 -- 172.21.15.162:0/262851061 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v21) v1 ==== 184+0+0 (secure 0 0 0) 0x7f7770047a60 con 0x7f7780065db0 2020-11-28T21:53:39.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.036+0000 7f7787048700 1 -- 172.21.15.162:0/262851061 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7768055350 msgr2=0x7f7768057810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:39.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.036+0000 7f7787048700 1 --2- 172.21.15.162:0/262851061 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7768055350 0x7f7768057810 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f777c008000 tx=0x7f777c008040).stop 2020-11-28T21:53:39.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.036+0000 7f7787048700 1 -- 172.21.15.162:0/262851061 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7780065db0 msgr2=0x7f7780066230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:39.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.036+0000 7f7787048700 1 --2- 172.21.15.162:0/262851061 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7780065db0 0x7f7780066230 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 rx=0x7f7770008000 tx=0x7f777000b040).stop 2020-11-28T21:53:39.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.036+0000 7f7787048700 1 -- 172.21.15.162:0/262851061 shutdown_connections 2020-11-28T21:53:39.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.036+0000 7f7787048700 1 --2- 172.21.15.162:0/262851061 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7768055350 0x7f7768057810 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:39.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.036+0000 7f7787048700 1 --2- 172.21.15.162:0/262851061 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7780061070 0x7f7780066f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:39.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.036+0000 7f7787048700 1 --2- 172.21.15.162:0/262851061 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7780143c20 0x7f77801440a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:39.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.036+0000 7f7787048700 1 --2- 172.21.15.162:0/262851061 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7780065db0 0x7f7780066230 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:39.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.036+0000 7f7787048700 1 -- 172.21.15.162:0/262851061 >> 172.21.15.162:0/262851061 conn(0x7f7780063b60 msgr2=0x7f77800b11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:39.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.036+0000 7f7787048700 1 -- 172.21.15.162:0/262851061 shutdown_connections 2020-11-28T21:53:39.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.036+0000 7f7787048700 1 -- 172.21.15.162:0/262851061 wait complete. 2020-11-28T21:53:39.042 INFO:teuthology.orchestra.run.smithi162.stderr:updated caps for client.0 2020-11-28T21:53:39.053 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-11-28T21:53:39.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.172+0000 7f3a15a09700 1 Processor -- start 2020-11-28T21:53:39.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.172+0000 7f3a15a09700 1 -- start start 2020-11-28T21:53:39.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.172+0000 7f3a15a09700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3a100b2560 0x7f3a10156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:39.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.172+0000 7f3a15a09700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3a1005fa90 0x7f3a101593c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:39.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.172+0000 7f3a15a09700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a100ab0b0 con 0x7f3a100b2e80 2020-11-28T21:53:39.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.172+0000 7f3a15a09700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a100b3260 con 0x7f3a1005fa90 2020-11-28T21:53:39.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.172+0000 7f3a15a09700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a100ab2e0 con 0x7f3a100b2560 2020-11-28T21:53:39.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.172+0000 7f3a15208700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f3a100b2e80 0x7f3a1005f380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:49002/0 (socket says 172.21.15.162:49002) 2020-11-28T21:53:39.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.172+0000 7f3a15208700 1 -- 172.21.15.162:0/1418911309 learned_addr learned my addr 172.21.15.162:0/1418911309 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:39.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.172+0000 7f3a16c6b700 1 --2- 172.21.15.162:0/1418911309 >> v2:172.21.15.162:3300/0 conn(0x7f3a1005fa90 0x7f3a101593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:39.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.172+0000 7f3a14a07700 1 --2- 172.21.15.162:0/1418911309 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3a100b2560 0x7f3a10156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:39.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.172+0000 7f3a03fff700 1 -- 172.21.15.162:0/1418911309 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2833672390 0 0) 0x7f3a100ab0b0 con 0x7f3a100b2e80 2020-11-28T21:53:39.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.172+0000 7f3a03fff700 1 -- 172.21.15.162:0/1418911309 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39fc003230 con 0x7f3a100b2e80 2020-11-28T21:53:39.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.172+0000 7f3a03fff700 1 -- 172.21.15.162:0/1418911309 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3920317286 0 0) 0x7f39fc003230 con 0x7f3a100b2e80 2020-11-28T21:53:39.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.172+0000 7f3a16c6b700 1 -- 172.21.15.162:0/1418911309 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3a100b2560 msgr2=0x7f3a10156ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:39.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.172+0000 7f3a16c6b700 1 --2- 172.21.15.162:0/1418911309 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3a100b2560 0x7f3a10156ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:39.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.172+0000 7f3a16c6b700 1 -- 172.21.15.162:0/1418911309 >> v1:172.21.15.162:6789/0 conn(0x7f3a100b2e80 legacy=0x7f3a1005f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:39.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.172+0000 7f3a16c6b700 1 -- 172.21.15.162:0/1418911309 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a100256d0 con 0x7f3a1005fa90 2020-11-28T21:53:39.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.172+0000 7f3a16c6b700 1 --2- 172.21.15.162:0/1418911309 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a1005fa90 0x7f3a101593c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 rx=0x7f3a0c001870 tx=0x7f3a080091a0).ready entity=mon.1 client_cookie=fe573425a4e85c03 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:39.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.172+0000 7f3a03fff700 1 -- 172.21.15.162:0/1418911309 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a0c0016d0 con 0x7f3a1005fa90 2020-11-28T21:53:39.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.176+0000 7f3a03fff700 1 -- 172.21.15.162:0/1418911309 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a0800b800 con 0x7f3a1005fa90 2020-11-28T21:53:39.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.176+0000 7f3a03fff700 1 -- 172.21.15.162:0/1418911309 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a0800b970 con 0x7f3a1005fa90 2020-11-28T21:53:39.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.176+0000 7f3a15a09700 1 -- 172.21.15.162:0/1418911309 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a1005fa90 msgr2=0x7f3a101593c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:39.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.176+0000 7f3a15a09700 1 --2- 172.21.15.162:0/1418911309 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a1005fa90 0x7f3a101593c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 rx=0x7f3a0c001870 tx=0x7f3a080091a0).stop 2020-11-28T21:53:39.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.176+0000 7f3a15a09700 1 -- 172.21.15.162:0/1418911309 shutdown_connections 2020-11-28T21:53:39.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.176+0000 7f3a15a09700 1 --2- 172.21.15.162:0/1418911309 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a1005fa90 0x7f3a101593c0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:39.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.176+0000 7f3a15a09700 1 --2- 172.21.15.162:0/1418911309 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3a100b2560 0x7f3a10156ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:39.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.176+0000 7f3a15a09700 1 -- 172.21.15.162:0/1418911309 >> 172.21.15.162:0/1418911309 conn(0x7f3a10063b60 msgr2=0x7f3a10063f40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:39.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.176+0000 7f3a15a09700 1 -- 172.21.15.162:0/1418911309 shutdown_connections 2020-11-28T21:53:39.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.176+0000 7f3a15a09700 1 -- 172.21.15.162:0/1418911309 wait complete. 2020-11-28T21:53:39.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.176+0000 7f3a15a09700 1 Processor -- start 2020-11-28T21:53:39.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.176+0000 7f3a15a09700 1 -- start start 2020-11-28T21:53:39.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.176+0000 7f3a15a09700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3a1005fa90 0x7f3a101312b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:39.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.176+0000 7f3a15a09700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3a100b2560 0x7f3a10142010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:39.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.176+0000 7f3a15a09700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a100b2e80 0x7f3a100b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:39.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.176+0000 7f3a15a09700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a100b6030 con 0x7f3a100b2560 2020-11-28T21:53:39.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.176+0000 7f3a15a09700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a10141940 con 0x7f3a100b2e80 2020-11-28T21:53:39.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.176+0000 7f3a15a09700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a100b6e60 con 0x7f3a1005fa90 2020-11-28T21:53:39.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.176+0000 7f3a16c6b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a100b2e80 0x7f3a100b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:39.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.176+0000 7f3a16c6b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a100b2e80 0x7f3a100b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42168/0 (socket says 172.21.15.162:42168) 2020-11-28T21:53:39.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.176+0000 7f3a16c6b700 1 -- 172.21.15.162:0/530574769 learned_addr learned my addr 172.21.15.162:0/530574769 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:39.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.176+0000 7f3a15208700 1 --2- 172.21.15.162:0/530574769 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3a1005fa90 0x7f3a101312b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:39.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.176+0000 7f3a14a07700 1 --2- 172.21.15.162:0/530574769 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3a100b2560 0x7f3a10142010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:39.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.176+0000 7f3a16c6b700 1 -- 172.21.15.162:0/530574769 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3a1005fa90 msgr2=0x7f3a101312b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:39.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.176+0000 7f3a16c6b700 1 --2- 172.21.15.162:0/530574769 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3a1005fa90 0x7f3a101312b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:39.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.176+0000 7f3a16c6b700 1 -- 172.21.15.162:0/530574769 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3a100b2560 msgr2=0x7f3a10142010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:39.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.176+0000 7f3a16c6b700 1 --2- 172.21.15.162:0/530574769 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3a100b2560 0x7f3a10142010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:39.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.176+0000 7f3a16c6b700 1 -- 172.21.15.162:0/530574769 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a100b6850 con 0x7f3a100b2e80 2020-11-28T21:53:39.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.176+0000 7f3a16c6b700 1 --2- 172.21.15.162:0/530574769 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a100b2e80 0x7f3a100b4e90 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 rx=0x7f3a100676e0 tx=0x7f3a08016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:39.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.180+0000 7f3a01ffb700 1 -- 172.21.15.162:0/530574769 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a080119d0 con 0x7f3a100b2e80 2020-11-28T21:53:39.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.180+0000 7f3a01ffb700 1 -- 172.21.15.162:0/530574769 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a08011b30 con 0x7f3a100b2e80 2020-11-28T21:53:39.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.180+0000 7f3a15a09700 1 -- 172.21.15.162:0/530574769 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a10141b70 con 0x7f3a100b2e80 2020-11-28T21:53:39.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.180+0000 7f3a01ffb700 1 -- 172.21.15.162:0/530574769 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a08011e90 con 0x7f3a100b2e80 2020-11-28T21:53:39.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.180+0000 7f3a15a09700 1 -- 172.21.15.162:0/530574769 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a1013be50 con 0x7f3a100b2e80 2020-11-28T21:53:39.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.180+0000 7f3a15a09700 1 -- 172.21.15.162:0/530574769 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a10057050 con 0x7f3a100b2e80 2020-11-28T21:53:39.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.180+0000 7f3a01ffb700 1 -- 172.21.15.162:0/530574769 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f3a08026030 con 0x7f3a100b2e80 2020-11-28T21:53:39.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.180+0000 7f3a01ffb700 1 --2- 172.21.15.162:0/530574769 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f39f0055440 0x7f39f0057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:39.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.180+0000 7f3a15208700 1 --2- 172.21.15.162:0/530574769 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f39f0055440 0x7f39f0057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:39.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.180+0000 7f3a01ffb700 1 -- 172.21.15.162:0/530574769 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3a0806e740 con 0x7f3a100b2e80 2020-11-28T21:53:39.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.180+0000 7f3a15208700 1 --2- 172.21.15.162:0/530574769 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f39f0055440 0x7f39f0057900 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f3a0c00d000 tx=0x7f3a0c000b80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:39.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.188+0000 7f3a01ffb700 1 -- 172.21.15.162:0/530574769 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f3a080449e0 con 0x7f3a100b2e80 2020-11-28T21:53:39.384 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.380+0000 7f3a15a09700 1 -- 172.21.15.162:0/530574769 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f3a10068350 con 0x7f3a100b2e80 2020-11-28T21:53:39.393 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.388+0000 7f3a01ffb700 1 -- 172.21.15.162:0/530574769 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v22) v1 ==== 184+0+0 (secure 0 0 0) 0x7f3a08048000 con 0x7f3a100b2e80 2020-11-28T21:53:39.395 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.392+0000 7f3a15a09700 1 -- 172.21.15.162:0/530574769 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f39f0055440 msgr2=0x7f39f0057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:39.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.392+0000 7f3a15a09700 1 --2- 172.21.15.162:0/530574769 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f39f0055440 0x7f39f0057900 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f3a0c00d000 tx=0x7f3a0c000b80).stop 2020-11-28T21:53:39.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.392+0000 7f3a15a09700 1 -- 172.21.15.162:0/530574769 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a100b2e80 msgr2=0x7f3a100b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:39.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.392+0000 7f3a15a09700 1 --2- 172.21.15.162:0/530574769 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a100b2e80 0x7f3a100b4e90 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 rx=0x7f3a100676e0 tx=0x7f3a08016000).stop 2020-11-28T21:53:39.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.392+0000 7f3a15a09700 1 -- 172.21.15.162:0/530574769 shutdown_connections 2020-11-28T21:53:39.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.392+0000 7f3a15a09700 1 --2- 172.21.15.162:0/530574769 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f39f0055440 0x7f39f0057900 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:39.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.392+0000 7f3a15a09700 1 --2- 172.21.15.162:0/530574769 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3a1005fa90 0x7f3a101312b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:39.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.392+0000 7f3a15a09700 1 --2- 172.21.15.162:0/530574769 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a100b2e80 0x7f3a100b4e90 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:39.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.392+0000 7f3a15a09700 1 --2- 172.21.15.162:0/530574769 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3a100b2560 0x7f3a10142010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:39.398 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.392+0000 7f3a15a09700 1 -- 172.21.15.162:0/530574769 >> 172.21.15.162:0/530574769 conn(0x7f3a10063b60 msgr2=0x7f3a1005f380 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:39.399 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.392+0000 7f3a15a09700 1 -- 172.21.15.162:0/530574769 shutdown_connections 2020-11-28T21:53:39.401 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.392+0000 7f3a15a09700 1 -- 172.21.15.162:0/530574769 wait complete. 2020-11-28T21:53:39.403 INFO:teuthology.orchestra.run.smithi162.stderr:updated caps for client.1 2020-11-28T21:53:39.409 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:53:39.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.516+0000 7f8271c11700 1 Processor -- start 2020-11-28T21:53:39.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.516+0000 7f8271c11700 1 -- start start 2020-11-28T21:53:39.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.516+0000 7f8271c11700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f826c0b1c00 0x7f826c14e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:39.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.516+0000 7f8271c11700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f826c0b0e50 con 0x7f826c143530 2020-11-28T21:53:39.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.516+0000 7f8271c11700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f826c05c090 con 0x7f826c05bcb0 2020-11-28T21:53:39.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.516+0000 7f8271c11700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f826c0b2090 con 0x7f826c0b1c00 2020-11-28T21:53:39.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.516+0000 7f8270c0f700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f826c05bcb0 0x7f826c0b0740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:49016/0 (socket says 172.21.15.162:49016) 2020-11-28T21:53:39.526 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.516+0000 7f8270c0f700 1 -- 172.21.15.162:0/2286570500 learned_addr learned my addr 172.21.15.162:0/2286570500 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:39.526 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.516+0000 7f8271410700 1 --2- 172.21.15.162:0/2286570500 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f826c0b1c00 0x7f826c14e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:39.526 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.516+0000 7f826b7fe700 1 -- 172.21.15.162:0/2286570500 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2850901896 0 0) 0x7f826c05c090 con 0x7f826c05bcb0 2020-11-28T21:53:39.526 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.516+0000 7f826b7fe700 1 -- 172.21.15.162:0/2286570500 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f824c003230 con 0x7f826c05bcb0 2020-11-28T21:53:39.527 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.516+0000 7f826b7fe700 1 -- 172.21.15.162:0/2286570500 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3549120523 0 0) 0x7f826c0b0e50 con 0x7f826c143530 2020-11-28T21:53:39.527 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.516+0000 7f826b7fe700 1 -- 172.21.15.162:0/2286570500 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f826c05c090 con 0x7f826c143530 2020-11-28T21:53:39.527 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.516+0000 7f826b7fe700 1 -- 172.21.15.162:0/2286570500 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1050838003 0 0) 0x7f824c003230 con 0x7f826c05bcb0 2020-11-28T21:53:39.527 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.516+0000 7f826b7fe700 1 -- 172.21.15.162:0/2286570500 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f826c0b1c00 msgr2=0x7f826c14e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:39.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.516+0000 7f826b7fe700 1 --2- 172.21.15.162:0/2286570500 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f826c0b1c00 0x7f826c14e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:39.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.516+0000 7f826b7fe700 1 -- 172.21.15.162:0/2286570500 >> v1:172.21.15.118:6789/0 conn(0x7f826c143530 legacy=0x7f826c13c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:39.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.516+0000 7f826b7fe700 1 -- 172.21.15.162:0/2286570500 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f826c0b22a0 con 0x7f826c05bcb0 2020-11-28T21:53:39.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.516+0000 7f826b7fe700 1 -- 172.21.15.162:0/2286570500 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f825c001a50 con 0x7f826c05bcb0 2020-11-28T21:53:39.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.516+0000 7f826b7fe700 1 -- 172.21.15.162:0/2286570500 >> v1:172.21.15.162:6789/0 conn(0x7f826c05bcb0 legacy=0x7f826c0b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:39.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.516+0000 7f826b7fe700 1 --2- 172.21.15.162:0/2286570500 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f824c0038b0 0x7f824c003cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:39.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.516+0000 7f826b7fe700 1 --2- 172.21.15.162:0/2286570500 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f826c0b1c00 0x7f824c0044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:39.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.516+0000 7f826b7fe700 1 --2- 172.21.15.162:0/2286570500 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f824c0049c0 0x7f824c004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:39.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.516+0000 7f826b7fe700 1 -- 172.21.15.162:0/2286570500 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826c0b2090 con 0x7f824c0038b0 2020-11-28T21:53:39.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.516+0000 7f826b7fe700 1 -- 172.21.15.162:0/2286570500 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824c005330 con 0x7f824c0049c0 2020-11-28T21:53:39.530 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.516+0000 7f826b7fe700 1 -- 172.21.15.162:0/2286570500 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f824c00b750 con 0x7f826c0b1c00 2020-11-28T21:53:39.530 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.520+0000 7f8271410700 1 --2- 172.21.15.162:0/2286570500 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f824c0049c0 0x7f824c004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:39.530 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.520+0000 7f8270c0f700 1 --2- 172.21.15.162:0/2286570500 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f824c0038b0 0x7f824c003cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:39.530 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.520+0000 7f826bfff700 1 --2- 172.21.15.162:0/2286570500 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f826c0b1c00 0x7f824c0044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:39.531 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.520+0000 7f8271410700 1 -- 172.21.15.162:0/2286570500 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f826c0b1c00 msgr2=0x7f824c0044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:39.531 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.520+0000 7f8271410700 1 --2- 172.21.15.162:0/2286570500 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f826c0b1c00 0x7f824c0044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:39.531 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.520+0000 7f8271410700 1 -- 172.21.15.162:0/2286570500 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f824c0038b0 msgr2=0x7f824c003cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:39.531 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.520+0000 7f8271410700 1 --2- 172.21.15.162:0/2286570500 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f824c0038b0 0x7f824c003cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:39.531 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.520+0000 7f8271410700 1 -- 172.21.15.162:0/2286570500 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f824c003230 con 0x7f824c0049c0 2020-11-28T21:53:39.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.520+0000 7f8271410700 1 --2- 172.21.15.162:0/2286570500 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f824c0049c0 0x7f824c004e40 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 rx=0x7f826400c000 tx=0x7f8264008040).ready entity=mon.1 client_cookie=7a22a83e0a1e2172 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:39.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.520+0000 7f826b7fe700 1 -- 172.21.15.162:0/2286570500 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f826400e8a0 con 0x7f824c0049c0 2020-11-28T21:53:39.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.520+0000 7f826b7fe700 1 -- 172.21.15.162:0/2286570500 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f826400ea00 con 0x7f824c0049c0 2020-11-28T21:53:39.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.520+0000 7f826b7fe700 1 -- 172.21.15.162:0/2286570500 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f826400ed60 con 0x7f824c0049c0 2020-11-28T21:53:39.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.520+0000 7f8271c11700 1 -- 172.21.15.162:0/2286570500 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f824c0049c0 msgr2=0x7f824c004e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:39.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.520+0000 7f8271c11700 1 --2- 172.21.15.162:0/2286570500 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f824c0049c0 0x7f824c004e40 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 rx=0x7f826400c000 tx=0x7f8264008040).stop 2020-11-28T21:53:39.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.520+0000 7f8271410700 1 -- 172.21.15.162:0/2286570500 reap_dead start 2020-11-28T21:53:39.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.520+0000 7f8271c11700 1 -- 172.21.15.162:0/2286570500 shutdown_connections 2020-11-28T21:53:39.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.520+0000 7f8271c11700 1 -- 172.21.15.162:0/2286570500 >> 172.21.15.162:0/2286570500 conn(0x7f826c0b5da0 msgr2=0x7f826c0b6180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:39.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.520+0000 7f8271c11700 1 -- 172.21.15.162:0/2286570500 shutdown_connections 2020-11-28T21:53:39.537 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.520+0000 7f8271c11700 1 -- 172.21.15.162:0/2286570500 wait complete. 2020-11-28T21:53:39.537 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.520+0000 7f8271c11700 1 Processor -- start 2020-11-28T21:53:39.537 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.520+0000 7f8271c11700 1 -- start start 2020-11-28T21:53:39.537 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.524+0000 7f8271c11700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f826c148c30 0x7f826c0af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:39.537 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.524+0000 7f8271c11700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f826c146a40 0x7f826c145ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:39.538 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.524+0000 7f8271c11700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f826c13a610 0x7f826c13aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:39.538 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.524+0000 7f8271c11700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826c0af890 con 0x7f826c146a40 2020-11-28T21:53:39.538 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.524+0000 7f8271c11700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826c0ae7c0 con 0x7f826c13a610 2020-11-28T21:53:39.538 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.524+0000 7f8271c11700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f826c0aea60 con 0x7f826c148c30 2020-11-28T21:53:39.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.524+0000 7f8271410700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f826c13a610 0x7f826c13aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:39.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.524+0000 7f8271410700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f826c13a610 0x7f826c13aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42188/0 (socket says 172.21.15.162:42188) 2020-11-28T21:53:39.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.524+0000 7f8271410700 1 -- 172.21.15.162:0/4137171196 learned_addr learned my addr 172.21.15.162:0/4137171196 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:39.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.524+0000 7f826bfff700 1 --2- 172.21.15.162:0/4137171196 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f826c146a40 0x7f826c145ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:39.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.524+0000 7f8270c0f700 1 --2- 172.21.15.162:0/4137171196 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f826c148c30 0x7f826c0af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:39.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.524+0000 7f8271410700 1 -- 172.21.15.162:0/4137171196 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f826c148c30 msgr2=0x7f826c0af3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:39.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.524+0000 7f8271410700 1 --2- 172.21.15.162:0/4137171196 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f826c148c30 0x7f826c0af3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:39.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.524+0000 7f8271410700 1 -- 172.21.15.162:0/4137171196 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f826c146a40 msgr2=0x7f826c145ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:39.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.524+0000 7f8271410700 1 --2- 172.21.15.162:0/4137171196 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f826c146a40 0x7f826c145ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:39.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.524+0000 7f8271410700 1 -- 172.21.15.162:0/4137171196 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f826c0aed00 con 0x7f826c13a610 2020-11-28T21:53:39.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.524+0000 7f8271410700 1 --2- 172.21.15.162:0/4137171196 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f826c13a610 0x7f826c13aa90 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 rx=0x7f825c001c30 tx=0x7f826401a000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:39.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.524+0000 7f8269ffb700 1 -- 172.21.15.162:0/4137171196 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8264012e00 con 0x7f826c13a610 2020-11-28T21:53:39.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.524+0000 7f8269ffb700 1 -- 172.21.15.162:0/4137171196 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8264025470 con 0x7f826c13a610 2020-11-28T21:53:39.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.524+0000 7f8271c11700 1 -- 172.21.15.162:0/4137171196 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f826c145a40 con 0x7f826c13a610 2020-11-28T21:53:39.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.524+0000 7f8269ffb700 1 -- 172.21.15.162:0/4137171196 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82640257d0 con 0x7f826c13a610 2020-11-28T21:53:39.543 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.524+0000 7f8271c11700 1 -- 172.21.15.162:0/4137171196 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f826c1496f0 con 0x7f826c13a610 2020-11-28T21:53:39.543 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.528+0000 7f8271c11700 1 -- 172.21.15.162:0/4137171196 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f826c1491b0 con 0x7f826c13a610 2020-11-28T21:53:39.543 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.528+0000 7f8269ffb700 1 -- 172.21.15.162:0/4137171196 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8264025930 con 0x7f826c13a610 2020-11-28T21:53:39.543 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.528+0000 7f8269ffb700 1 --2- 172.21.15.162:0/4137171196 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f82500553b0 0x7f8250057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:39.544 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.528+0000 7f8269ffb700 1 -- 172.21.15.162:0/4137171196 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f826406f6a0 con 0x7f826c13a610 2020-11-28T21:53:39.544 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.528+0000 7f8270c0f700 1 --2- 172.21.15.162:0/4137171196 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f82500553b0 0x7f8250057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:39.544 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.528+0000 7f8270c0f700 1 --2- 172.21.15.162:0/4137171196 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f82500553b0 0x7f8250057870 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f825c005000 tx=0x7f825c007ac0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:39.545 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.536+0000 7f8269ffb700 1 -- 172.21.15.162:0/4137171196 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8264045920 con 0x7f826c13a610 2020-11-28T21:53:39.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.724+0000 7f8271c11700 1 -- 172.21.15.162:0/4137171196 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f826c145bd0 con 0x7f826c13a610 2020-11-28T21:53:39.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.724+0000 7f8269ffb700 1 -- 172.21.15.162:0/4137171196 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+2502 (secure 0 0 0) 0x7f8264016020 con 0x7f826c13a610 2020-11-28T21:53:39.728 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:53:39.728 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":40,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5795,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/2645168518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":2645168518},{"type":"v1","addr":"172.21.15.118:6833","nonce":2645168518}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36},{"gid":6541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1999073284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1999073284},{"type":"v1","addr":"172.21.15.162:6835","nonce":1999073284}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:53:36.539204+0000","modified":"2020-11-28T21:53:38.035411+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6550":{"gid":6550,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/1097065372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1097065372},{"type":"v1","addr":"172.21.15.162:6837","nonce":1097065372}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-28T21:53:39.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.724+0000 7f8271c11700 1 -- 172.21.15.162:0/4137171196 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f82500553b0 msgr2=0x7f8250057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:39.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.724+0000 7f8271c11700 1 --2- 172.21.15.162:0/4137171196 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f82500553b0 0x7f8250057870 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f825c005000 tx=0x7f825c007ac0).stop 2020-11-28T21:53:39.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.724+0000 7f8271c11700 1 -- 172.21.15.162:0/4137171196 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f826c13a610 msgr2=0x7f826c13aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:39.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.724+0000 7f8271c11700 1 --2- 172.21.15.162:0/4137171196 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f826c13a610 0x7f826c13aa90 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 rx=0x7f825c001c30 tx=0x7f826401a000).stop 2020-11-28T21:53:39.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.724+0000 7f8271c11700 1 -- 172.21.15.162:0/4137171196 shutdown_connections 2020-11-28T21:53:39.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.724+0000 7f8271c11700 1 --2- 172.21.15.162:0/4137171196 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f82500553b0 0x7f8250057870 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:39.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.724+0000 7f8271c11700 1 --2- 172.21.15.162:0/4137171196 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f826c148c30 0x7f826c0af3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:39.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.724+0000 7f8271c11700 1 --2- 172.21.15.162:0/4137171196 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f826c13a610 0x7f826c13aa90 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:39.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.724+0000 7f8271c11700 1 --2- 172.21.15.162:0/4137171196 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f826c146a40 0x7f826c145ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:39.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.724+0000 7f8271c11700 1 -- 172.21.15.162:0/4137171196 >> 172.21.15.162:0/4137171196 conn(0x7f826c0b5da0 msgr2=0x7f826c0b0440 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:39.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.724+0000 7f8271c11700 1 -- 172.21.15.162:0/4137171196 shutdown_connections 2020-11-28T21:53:39.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:39.724+0000 7f8271c11700 1 -- 172.21.15.162:0/4137171196 wait complete. 2020-11-28T21:53:39.732 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 40 2020-11-28T21:53:40.042 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 40, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-28T21:53:36.539204+0000', 'modified': '2020-11-28T21:53:38.035411+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6550}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6550': {'gid': 6550, 'name': 'c', 'rank': 0, 'incarnation': 38, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6837/1097065372', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6836', 'nonce': 1097065372}, {'type': 'v1', 'addr': '172.21.15.162:6837', 'nonce': 1097065372}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-28T21:53:40.043 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-28T21:53:40.044 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-28T21:53:40.173 INFO:teuthology.orchestra.run.smithi162.stdout:{"cluster_fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","whoami":0,"id":6550,"want_state":"up:active","state":"up:active","rank_uptime":3.61928888,"mdsmap_epoch":40,"osdmap_epoch":88,"osdmap_epoch_barrier":88,"uptime":9.0163412760000003} 2020-11-28T21:53:40.173 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", "id": 6550, "mdsmap_epoch": 40, "osdmap_epoch": 88, "osdmap_epoch_barrier": 88, "rank_uptime": 3.61928888, "state": "up:active", "uptime": 9.016341276, "want_state": "up:active", "whoami": 0 } 2020-11-28T21:53:40.174 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:53:40.283 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.280+0000 7f41a0b79700 1 Processor -- start 2020-11-28T21:53:40.283 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.280+0000 7f41a0b79700 1 -- start start 2020-11-28T21:53:40.285 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.280+0000 7f41a0b79700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f419c0b1d70 0x7f419c140fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:40.285 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.280+0000 7f41a0b79700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f419c0b2870 con 0x7f419c0b11b0 2020-11-28T21:53:40.285 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.280+0000 7f41a0b79700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f419c0b22e0 con 0x7f419c0b1990 2020-11-28T21:53:40.285 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.280+0000 7f41a0b79700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f419c146e20 con 0x7f419c0b1d70 2020-11-28T21:53:40.286 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.280+0000 7f419a59c700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f419c0b1990 0x7f419c140890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:49036/0 (socket says 172.21.15.162:49036) 2020-11-28T21:53:40.286 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.280+0000 7f419a59c700 1 -- 172.21.15.162:0/116703390 learned_addr learned my addr 172.21.15.162:0/116703390 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:40.286 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.280+0000 7f419ad9d700 1 --2- 172.21.15.162:0/116703390 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f419c0b1d70 0x7f419c140fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:40.286 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.280+0000 7f419959a700 1 -- 172.21.15.162:0/116703390 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2272489602 0 0) 0x7f419c0b22e0 con 0x7f419c0b1990 2020-11-28T21:53:40.287 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.280+0000 7f419959a700 1 -- 172.21.15.162:0/116703390 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4180003230 con 0x7f419c0b1990 2020-11-28T21:53:40.287 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.280+0000 7f419959a700 1 -- 172.21.15.162:0/116703390 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 330805622 0 0) 0x7f419c0b2870 con 0x7f419c0b11b0 2020-11-28T21:53:40.287 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.280+0000 7f419959a700 1 -- 172.21.15.162:0/116703390 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f419c0b22e0 con 0x7f419c0b11b0 2020-11-28T21:53:40.287 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.280+0000 7f419ad9d700 1 -- 172.21.15.162:0/116703390 >> v1:172.21.15.162:6789/0 conn(0x7f419c0b1990 legacy=0x7f419c140890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:40.287 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.280+0000 7f419ad9d700 1 -- 172.21.15.162:0/116703390 >> v1:172.21.15.118:6789/0 conn(0x7f419c0b11b0 legacy=0x7f419c14a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:40.288 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.280+0000 7f419ad9d700 1 -- 172.21.15.162:0/116703390 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f419c0b0740 con 0x7f419c0b1d70 2020-11-28T21:53:40.290 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.280+0000 7f419ad9d700 1 --2- 172.21.15.162:0/116703390 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f419c0b1d70 0x7f419c140fa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f4194002f30 tx=0x7f4194006dc0).ready entity=mon.2 client_cookie=d252fbde42e3120c server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:40.290 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.280+0000 7f419959a700 1 -- 172.21.15.162:0/116703390 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4194009730 con 0x7f419c0b1d70 2020-11-28T21:53:40.291 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.280+0000 7f419959a700 1 -- 172.21.15.162:0/116703390 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4194009890 con 0x7f419c0b1d70 2020-11-28T21:53:40.291 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.280+0000 7f419959a700 1 -- 172.21.15.162:0/116703390 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4194009bf0 con 0x7f419c0b1d70 2020-11-28T21:53:40.291 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.280+0000 7f41a0b79700 1 -- 172.21.15.162:0/116703390 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f419c0b1d70 msgr2=0x7f419c140fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:40.291 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.280+0000 7f41a0b79700 1 --2- 172.21.15.162:0/116703390 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f419c0b1d70 0x7f419c140fa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f4194002f30 tx=0x7f4194006dc0).stop 2020-11-28T21:53:40.291 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.284+0000 7f41a0b79700 1 -- 172.21.15.162:0/116703390 shutdown_connections 2020-11-28T21:53:40.292 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.284+0000 7f41a0b79700 1 --2- 172.21.15.162:0/116703390 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f419c0b1d70 0x7f419c140fa0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:40.292 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.284+0000 7f41a0b79700 1 -- 172.21.15.162:0/116703390 >> 172.21.15.162:0/116703390 conn(0x7f419c05b970 msgr2=0x7f419c05bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:40.292 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.284+0000 7f41a0b79700 1 -- 172.21.15.162:0/116703390 shutdown_connections 2020-11-28T21:53:40.292 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.284+0000 7f41a0b79700 1 -- 172.21.15.162:0/116703390 wait complete. 2020-11-28T21:53:40.292 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.284+0000 7f41a0b79700 1 Processor -- start 2020-11-28T21:53:40.293 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.284+0000 7f41a0b79700 1 -- start start 2020-11-28T21:53:40.293 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.284+0000 7f41a0b79700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f419c0b11b0 0x7f419c143aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:40.293 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.284+0000 7f41a0b79700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f419c0b1990 0x7f419c143f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:40.293 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.284+0000 7f41a0b79700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f419c0b1d70 0x7f419c1429a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:40.293 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.284+0000 7f41a0b79700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419c1566f0 con 0x7f419c0b11b0 2020-11-28T21:53:40.294 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.284+0000 7f41a0b79700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419c156990 con 0x7f419c0b1d70 2020-11-28T21:53:40.294 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.284+0000 7f41a0b79700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f419c143600 con 0x7f419c0b1990 2020-11-28T21:53:40.294 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.284+0000 7f419ad9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f419c0b1d70 0x7f419c1429a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:40.294 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.284+0000 7f419ad9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f419c0b1d70 0x7f419c1429a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42202/0 (socket says 172.21.15.162:42202) 2020-11-28T21:53:40.295 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.284+0000 7f419ad9d700 1 -- 172.21.15.162:0/1673605811 learned_addr learned my addr 172.21.15.162:0/1673605811 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:40.295 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.284+0000 7f419a59c700 1 --2- 172.21.15.162:0/1673605811 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f419c0b11b0 0x7f419c143aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:40.295 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.284+0000 7f4199d9b700 1 --2- 172.21.15.162:0/1673605811 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f419c0b1990 0x7f419c143f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:40.295 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.284+0000 7f419ad9d700 1 -- 172.21.15.162:0/1673605811 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f419c0b1990 msgr2=0x7f419c143f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:40.295 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.284+0000 7f419ad9d700 1 --2- 172.21.15.162:0/1673605811 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f419c0b1990 0x7f419c143f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:40.296 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.284+0000 7f419ad9d700 1 -- 172.21.15.162:0/1673605811 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f419c0b11b0 msgr2=0x7f419c143aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:40.296 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.284+0000 7f419ad9d700 1 --2- 172.21.15.162:0/1673605811 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f419c0b11b0 0x7f419c143aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:40.296 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.284+0000 7f419ad9d700 1 -- 172.21.15.162:0/1673605811 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f419c0abcc0 con 0x7f419c0b1d70 2020-11-28T21:53:40.296 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.284+0000 7f4199d9b700 1 --2- 172.21.15.162:0/1673605811 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f419c0b1990 0x7f419c143f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:53:40.296 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.284+0000 7f419ad9d700 1 --2- 172.21.15.162:0/1673605811 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f419c0b1d70 0x7f419c1429a0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 rx=0x7f4194002f30 tx=0x7f4194015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:40.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.288+0000 7f418b7fe700 1 -- 172.21.15.162:0/1673605811 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4194011ab0 con 0x7f419c0b1d70 2020-11-28T21:53:40.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.288+0000 7f41a0b79700 1 -- 172.21.15.162:0/1673605811 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f419c0a9c60 con 0x7f419c0b1d70 2020-11-28T21:53:40.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.288+0000 7f418b7fe700 1 -- 172.21.15.162:0/1673605811 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4194011c10 con 0x7f419c0b1d70 2020-11-28T21:53:40.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.288+0000 7f418b7fe700 1 -- 172.21.15.162:0/1673605811 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f419401d450 con 0x7f419c0b1d70 2020-11-28T21:53:40.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.288+0000 7f41a0b79700 1 -- 172.21.15.162:0/1673605811 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f419c0b5a60 con 0x7f419c0b1d70 2020-11-28T21:53:40.298 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.288+0000 7f418b7fe700 1 -- 172.21.15.162:0/1673605811 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4194024030 con 0x7f419c0b1d70 2020-11-28T21:53:40.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.288+0000 7f418b7fe700 1 --2- 172.21.15.162:0/1673605811 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f41780553b0 0x7f4178057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:40.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.288+0000 7f418b7fe700 1 -- 172.21.15.162:0/1673605811 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f419406f1b0 con 0x7f419c0b1d70 2020-11-28T21:53:40.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.288+0000 7f419a59c700 1 --2- 172.21.15.162:0/1673605811 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f41780553b0 0x7f4178057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:40.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.288+0000 7f41a0b79700 1 -- 172.21.15.162:0/1673605811 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f419c1437c0 con 0x7f419c0b1d70 2020-11-28T21:53:40.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.288+0000 7f419a59c700 1 --2- 172.21.15.162:0/1673605811 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f41780553b0 0x7f4178057870 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f418c00d000 tx=0x7f418c009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:40.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.292+0000 7f418b7fe700 1 -- 172.21.15.162:0/1673605811 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4194022080 con 0x7f419c0b1d70 2020-11-28T21:53:40.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.436+0000 7f41a0b79700 1 -- 172.21.15.162:0/1673605811 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f419c057050 con 0x7f419c0b1d70 2020-11-28T21:53:40.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.436+0000 7f418b7fe700 1 -- 172.21.15.162:0/1673605811 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16137 (secure 0 0 0) 0x7f4194045410 con 0x7f419c0b1d70 2020-11-28T21:53:40.440 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:53:40.440 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":88,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:53:37.541898+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:53:34.083725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2020-11-28T21:53:35.380493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:53:40.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.436+0000 7f41a0b79700 1 -- 172.21.15.162:0/1673605811 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f41780553b0 msgr2=0x7f4178057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:40.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.436+0000 7f41a0b79700 1 --2- 172.21.15.162:0/1673605811 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f41780553b0 0x7f4178057870 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f418c00d000 tx=0x7f418c009040).stop 2020-11-28T21:53:40.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.436+0000 7f41a0b79700 1 -- 172.21.15.162:0/1673605811 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f419c0b1d70 msgr2=0x7f419c1429a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:40.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.436+0000 7f41a0b79700 1 --2- 172.21.15.162:0/1673605811 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f419c0b1d70 0x7f419c1429a0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 rx=0x7f4194002f30 tx=0x7f4194015000).stop 2020-11-28T21:53:40.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.436+0000 7f41a0b79700 1 -- 172.21.15.162:0/1673605811 shutdown_connections 2020-11-28T21:53:40.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.436+0000 7f41a0b79700 1 --2- 172.21.15.162:0/1673605811 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f41780553b0 0x7f4178057870 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:40.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.436+0000 7f41a0b79700 1 --2- 172.21.15.162:0/1673605811 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f419c0b1d70 0x7f419c1429a0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:40.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.436+0000 7f41a0b79700 1 --2- 172.21.15.162:0/1673605811 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f419c0b11b0 0x7f419c143aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:40.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.436+0000 7f41a0b79700 1 --2- 172.21.15.162:0/1673605811 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f419c0b1990 0x7f419c143f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:40.444 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.436+0000 7f41a0b79700 1 -- 172.21.15.162:0/1673605811 >> 172.21.15.162:0/1673605811 conn(0x7f419c05b970 msgr2=0x7f419c060290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:40.444 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.436+0000 7f41a0b79700 1 -- 172.21.15.162:0/1673605811 shutdown_connections 2020-11-28T21:53:40.444 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.436+0000 7f41a0b79700 1 -- 172.21.15.162:0/1673605811 wait complete. 2020-11-28T21:53:40.454 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-28T21:53:34.083725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2020-11-28T21:53:35.380493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-28T21:53:40.455 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:53:40.575 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.572+0000 7f29da721700 1 Processor -- start 2020-11-28T21:53:40.576 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.572+0000 7f29da721700 1 -- start start 2020-11-28T21:53:40.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.572+0000 7f29da721700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f29d40b1eb0 0x7f29d40b2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:40.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.572+0000 7f29da721700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f29d4060a70 0x7f29d40afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:40.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.572+0000 7f29da721700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29d4060f00 con 0x7f29d40af6a0 2020-11-28T21:53:40.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.572+0000 7f29da721700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29d41471f0 con 0x7f29d40b1eb0 2020-11-28T21:53:40.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.572+0000 7f29da721700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29d4060090 con 0x7f29d4060a70 2020-11-28T21:53:40.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.572+0000 7f29d8f1e700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f29d40af6a0 0x7f29d4144350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:49050/0 (socket says 172.21.15.162:49050) 2020-11-28T21:53:40.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.572+0000 7f29d8f1e700 1 -- 172.21.15.162:0/390861157 learned_addr learned my addr 172.21.15.162:0/390861157 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:40.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.572+0000 7f29d9f20700 1 --2- 172.21.15.162:0/390861157 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f29d4060a70 0x7f29d40afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:40.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.572+0000 7f29d3fff700 1 -- 172.21.15.162:0/390861157 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 716024632 0 0) 0x7f29d4060f00 con 0x7f29d40af6a0 2020-11-28T21:53:40.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.572+0000 7f29d971f700 1 --2- 172.21.15.162:0/390861157 >> v2:172.21.15.118:3300/0 conn(0x7f29d40b1eb0 0x7f29d40b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:40.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.572+0000 7f29d3fff700 1 -- 172.21.15.162:0/390861157 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29c0003230 con 0x7f29d40af6a0 2020-11-28T21:53:40.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.572+0000 7f29d3fff700 1 -- 172.21.15.162:0/390861157 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2990409272 0 0) 0x7f29c0003230 con 0x7f29d40af6a0 2020-11-28T21:53:40.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.572+0000 7f29d3fff700 1 -- 172.21.15.162:0/390861157 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f29d4060a70 msgr2=0x7f29d40afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:40.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.572+0000 7f29d3fff700 1 --2- 172.21.15.162:0/390861157 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f29d4060a70 0x7f29d40afdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:40.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.572+0000 7f29d3fff700 1 -- 172.21.15.162:0/390861157 >> v2:172.21.15.118:3300/0 conn(0x7f29d40b1eb0 msgr2=0x7f29d40b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:40.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.572+0000 7f29d3fff700 1 --2- 172.21.15.162:0/390861157 >> v2:172.21.15.118:3300/0 conn(0x7f29d40b1eb0 0x7f29d40b2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:40.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.572+0000 7f29d3fff700 1 -- 172.21.15.162:0/390861157 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d4060330 con 0x7f29d40af6a0 2020-11-28T21:53:40.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.572+0000 7f29d3fff700 1 -- 172.21.15.162:0/390861157 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f29c80016d0 con 0x7f29d40af6a0 2020-11-28T21:53:40.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.572+0000 7f29d3fff700 1 -- 172.21.15.162:0/390861157 >> v1:172.21.15.162:6789/0 conn(0x7f29d40af6a0 legacy=0x7f29d4144350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:40.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.572+0000 7f29d3fff700 1 --2- 172.21.15.162:0/390861157 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f29d4060a70 0x7f29c0003720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:40.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.572+0000 7f29d3fff700 1 --2- 172.21.15.162:0/390861157 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f29c0003c30 0x7f29c00044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:40.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.572+0000 7f29d3fff700 1 --2- 172.21.15.162:0/390861157 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29c00049c0 0x7f29c0004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:40.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.572+0000 7f29d3fff700 1 -- 172.21.15.162:0/390861157 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d41471f0 con 0x7f29c0003c30 2020-11-28T21:53:40.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.572+0000 7f29d3fff700 1 -- 172.21.15.162:0/390861157 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d4060090 con 0x7f29c00049c0 2020-11-28T21:53:40.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.572+0000 7f29d3fff700 1 -- 172.21.15.162:0/390861157 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c00041f0 con 0x7f29d4060a70 2020-11-28T21:53:40.586 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.572+0000 7f29d9f20700 1 --2- 172.21.15.162:0/390861157 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29c00049c0 0x7f29c0004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:40.586 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.572+0000 7f29d971f700 1 --2- 172.21.15.162:0/390861157 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f29d4060a70 0x7f29c0003720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:40.586 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.572+0000 7f29d8f1e700 1 --2- 172.21.15.162:0/390861157 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f29c0003c30 0x7f29c00044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:40.586 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.572+0000 7f29d9f20700 1 -- 172.21.15.162:0/390861157 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f29d4060a70 msgr2=0x7f29c0003720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:40.586 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.572+0000 7f29d9f20700 1 --2- 172.21.15.162:0/390861157 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f29d4060a70 0x7f29c0003720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:40.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.572+0000 7f29d9f20700 1 -- 172.21.15.162:0/390861157 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f29c0003c30 msgr2=0x7f29c00044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:40.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.572+0000 7f29d9f20700 1 --2- 172.21.15.162:0/390861157 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f29c0003c30 0x7f29c00044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:40.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.572+0000 7f29d9f20700 1 -- 172.21.15.162:0/390861157 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c0003230 con 0x7f29c00049c0 2020-11-28T21:53:40.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.576+0000 7f29d9f20700 1 --2- 172.21.15.162:0/390861157 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29c00049c0 0x7f29c0004e40 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 rx=0x7f29cc00c000 tx=0x7f29cc0091a0).ready entity=mon.1 client_cookie=7e8708dc2c932f45 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:40.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.576+0000 7f29d3fff700 1 -- 172.21.15.162:0/390861157 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29cc00ed00 con 0x7f29c00049c0 2020-11-28T21:53:40.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.576+0000 7f29d3fff700 1 -- 172.21.15.162:0/390861157 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29cc00ee60 con 0x7f29c00049c0 2020-11-28T21:53:40.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.576+0000 7f29d3fff700 1 -- 172.21.15.162:0/390861157 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29cc0125e0 con 0x7f29c00049c0 2020-11-28T21:53:40.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.576+0000 7f29da721700 1 -- 172.21.15.162:0/390861157 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29c00049c0 msgr2=0x7f29c0004e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:40.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.576+0000 7f29da721700 1 --2- 172.21.15.162:0/390861157 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29c00049c0 0x7f29c0004e40 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 rx=0x7f29cc00c000 tx=0x7f29cc0091a0).stop 2020-11-28T21:53:40.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.576+0000 7f29d9f20700 1 -- 172.21.15.162:0/390861157 reap_dead start 2020-11-28T21:53:40.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.576+0000 7f29da721700 1 -- 172.21.15.162:0/390861157 shutdown_connections 2020-11-28T21:53:40.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.576+0000 7f29da721700 1 -- 172.21.15.162:0/390861157 >> 172.21.15.162:0/390861157 conn(0x7f29d40b57d0 msgr2=0x7f29d40b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:40.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.576+0000 7f29da721700 1 -- 172.21.15.162:0/390861157 shutdown_connections 2020-11-28T21:53:40.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.576+0000 7f29da721700 1 -- 172.21.15.162:0/390861157 wait complete. 2020-11-28T21:53:40.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.576+0000 7f29da721700 1 Processor -- start 2020-11-28T21:53:40.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.576+0000 7f29da721700 1 -- start start 2020-11-28T21:53:40.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.580+0000 7f29da721700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29d414cb80 0x7f29d4144d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:40.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.580+0000 7f29da721700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f29d41467c0 0x7f29d4145200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:40.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.580+0000 7f29da721700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f29d40abbc0 0x7f29d4066b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:40.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.580+0000 7f29da721700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d40636c0 con 0x7f29d41467c0 2020-11-28T21:53:40.594 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.580+0000 7f29da721700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d4146390 con 0x7f29d414cb80 2020-11-28T21:53:40.594 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.580+0000 7f29da721700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29d4156d00 con 0x7f29d40abbc0 2020-11-28T21:53:40.594 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.580+0000 7f29d971f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29d414cb80 0x7f29d4144d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:40.594 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.580+0000 7f29d971f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29d414cb80 0x7f29d4144d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42218/0 (socket says 172.21.15.162:42218) 2020-11-28T21:53:40.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.580+0000 7f29d971f700 1 -- 172.21.15.162:0/2580614996 learned_addr learned my addr 172.21.15.162:0/2580614996 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:40.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.580+0000 7f29d9f20700 1 --2- 172.21.15.162:0/2580614996 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f29d40abbc0 0x7f29d4066b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:40.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.580+0000 7f29d8f1e700 1 --2- 172.21.15.162:0/2580614996 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f29d41467c0 0x7f29d4145200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:40.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.580+0000 7f29d971f700 1 -- 172.21.15.162:0/2580614996 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f29d40abbc0 msgr2=0x7f29d4066b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:40.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.580+0000 7f29d971f700 1 --2- 172.21.15.162:0/2580614996 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f29d40abbc0 0x7f29d4066b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:40.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.580+0000 7f29d971f700 1 -- 172.21.15.162:0/2580614996 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f29d41467c0 msgr2=0x7f29d4145200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:40.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.580+0000 7f29d971f700 1 --2- 172.21.15.162:0/2580614996 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f29d41467c0 0x7f29d4145200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:40.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.580+0000 7f29d971f700 1 -- 172.21.15.162:0/2580614996 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d4156fa0 con 0x7f29d414cb80 2020-11-28T21:53:40.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.580+0000 7f29d971f700 1 --2- 172.21.15.162:0/2580614996 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29d414cb80 0x7f29d4144d10 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 rx=0x7f29d41466a0 tx=0x7f29c4008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:40.597 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.580+0000 7f29d27fc700 1 -- 172.21.15.162:0/2580614996 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29c400e950 con 0x7f29d414cb80 2020-11-28T21:53:40.597 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.580+0000 7f29d27fc700 1 -- 172.21.15.162:0/2580614996 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29c400eab0 con 0x7f29d414cb80 2020-11-28T21:53:40.597 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.580+0000 7f29da721700 1 -- 172.21.15.162:0/2580614996 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29d414cf60 con 0x7f29d414cb80 2020-11-28T21:53:40.597 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.580+0000 7f29d27fc700 1 -- 172.21.15.162:0/2580614996 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29c400ee10 con 0x7f29d414cb80 2020-11-28T21:53:40.598 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.580+0000 7f29da721700 1 -- 172.21.15.162:0/2580614996 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29d40a8630 con 0x7f29d414cb80 2020-11-28T21:53:40.598 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.580+0000 7f29da721700 1 -- 172.21.15.162:0/2580614996 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29d41464d0 con 0x7f29d414cb80 2020-11-28T21:53:40.598 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.580+0000 7f29d27fc700 1 -- 172.21.15.162:0/2580614996 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f29c4014470 con 0x7f29d414cb80 2020-11-28T21:53:40.598 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.584+0000 7f29d27fc700 1 --2- 172.21.15.162:0/2580614996 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f29b4055440 0x7f29b4057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:40.598 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.584+0000 7f29d27fc700 1 -- 172.21.15.162:0/2580614996 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f29c406e790 con 0x7f29d414cb80 2020-11-28T21:53:40.599 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.584+0000 7f29d8f1e700 1 --2- 172.21.15.162:0/2580614996 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f29b4055440 0x7f29b4057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:40.599 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.584+0000 7f29d8f1e700 1 --2- 172.21.15.162:0/2580614996 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f29b4055440 0x7f29b4057900 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f29c800d000 tx=0x7f29c8007040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:40.599 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.584+0000 7f29d27fc700 1 -- 172.21.15.162:0/2580614996 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f29c4044a30 con 0x7f29d414cb80 2020-11-28T21:53:40.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.728+0000 7f29da721700 1 -- 172.21.15.162:0/2580614996 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f29d4057050 con 0x7f29d414cb80 2020-11-28T21:53:40.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.728+0000 7f29d27fc700 1 -- 172.21.15.162:0/2580614996 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v88) v1 ==== 110+0+10 (secure 0 0 0) 0x7f29c4048050 con 0x7f29d414cb80 2020-11-28T21:53:40.732 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:53:40.733 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.728+0000 7f29da721700 1 -- 172.21.15.162:0/2580614996 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f29b4055440 msgr2=0x7f29b4057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:40.733 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.728+0000 7f29da721700 1 --2- 172.21.15.162:0/2580614996 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f29b4055440 0x7f29b4057900 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f29c800d000 tx=0x7f29c8007040).stop 2020-11-28T21:53:40.734 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.728+0000 7f29da721700 1 -- 172.21.15.162:0/2580614996 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29d414cb80 msgr2=0x7f29d4144d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:40.734 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.728+0000 7f29da721700 1 --2- 172.21.15.162:0/2580614996 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29d414cb80 0x7f29d4144d10 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 rx=0x7f29d41466a0 tx=0x7f29c4008040).stop 2020-11-28T21:53:40.734 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.728+0000 7f29da721700 1 -- 172.21.15.162:0/2580614996 shutdown_connections 2020-11-28T21:53:40.734 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.728+0000 7f29da721700 1 --2- 172.21.15.162:0/2580614996 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f29b4055440 0x7f29b4057900 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:40.735 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.728+0000 7f29da721700 1 --2- 172.21.15.162:0/2580614996 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f29d40abbc0 0x7f29d4066b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:40.735 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.728+0000 7f29da721700 1 --2- 172.21.15.162:0/2580614996 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29d414cb80 0x7f29d4144d10 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:40.735 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.728+0000 7f29da721700 1 --2- 172.21.15.162:0/2580614996 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f29d41467c0 0x7f29d4145200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:40.735 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.728+0000 7f29da721700 1 -- 172.21.15.162:0/2580614996 >> 172.21.15.162:0/2580614996 conn(0x7f29d40b57d0 msgr2=0x7f29d40b0120 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:40.735 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.728+0000 7f29da721700 1 -- 172.21.15.162:0/2580614996 shutdown_connections 2020-11-28T21:53:40.736 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.728+0000 7f29da721700 1 -- 172.21.15.162:0/2580614996 wait complete. 2020-11-28T21:53:40.745 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:53:40.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.856+0000 7f4bee895700 1 Processor -- start 2020-11-28T21:53:40.862 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.856+0000 7f4bee895700 1 -- start start 2020-11-28T21:53:40.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.860+0000 7f4bee895700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4be80b3ab0 0x7f4be805c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:40.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.860+0000 7f4bee895700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4be805baf0 con 0x7f4be80b2d00 2020-11-28T21:53:40.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.860+0000 7f4bee895700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4be805bd90 con 0x7f4be80b30e0 2020-11-28T21:53:40.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.860+0000 7f4bee895700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4be80b0e20 con 0x7f4be80b3ab0 2020-11-28T21:53:40.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.860+0000 7f4bee094700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f4be80b30e0 0x7f4be813fa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:49074/0 (socket says 172.21.15.162:49074) 2020-11-28T21:53:40.864 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.860+0000 7f4bee094700 1 -- 172.21.15.162:0/85836707 learned_addr learned my addr 172.21.15.162:0/85836707 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:40.864 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.860+0000 7f4bed092700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4be80b3ab0 0x7f4be805c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:40.864 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.860+0000 7f4bec891700 1 -- 172.21.15.162:0/85836707 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4122445379 0 0) 0x7f4be805bd90 con 0x7f4be80b30e0 2020-11-28T21:53:40.864 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.860+0000 7f4bec891700 1 -- 172.21.15.162:0/85836707 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bd0003230 con 0x7f4be80b30e0 2020-11-28T21:53:40.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.860+0000 7f4bec891700 1 -- 172.21.15.162:0/85836707 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 794766589 0 0) 0x7f4bd0003230 con 0x7f4be80b30e0 2020-11-28T21:53:40.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.860+0000 7f4bed092700 1 -- 172.21.15.162:0/85836707 >> v1:172.21.15.162:6789/0 conn(0x7f4be80b30e0 legacy=0x7f4be813fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:40.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.860+0000 7f4bed092700 1 -- 172.21.15.162:0/85836707 >> v1:172.21.15.118:6789/0 conn(0x7f4be80b2d00 legacy=0x7f4be8142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:40.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.860+0000 7f4bed092700 1 -- 172.21.15.162:0/85836707 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be805c030 con 0x7f4be80b3ab0 2020-11-28T21:53:40.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.860+0000 7f4bed092700 1 --2- 172.21.15.162:0/85836707 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4be80b3ab0 0x7f4be805c4d0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 rx=0x7f4be0001bf0 tx=0x7f4be4006dc0).ready entity=mon.1 client_cookie=66bf4bb8970cfe78 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:40.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.860+0000 7f4bec891700 1 -- 172.21.15.162:0/85836707 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4be0001a50 con 0x7f4be80b3ab0 2020-11-28T21:53:40.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.860+0000 7f4bec891700 1 -- 172.21.15.162:0/85836707 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be4009670 con 0x7f4be80b3ab0 2020-11-28T21:53:40.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.860+0000 7f4bec891700 1 -- 172.21.15.162:0/85836707 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4be40097e0 con 0x7f4be80b3ab0 2020-11-28T21:53:40.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.860+0000 7f4bee895700 1 -- 172.21.15.162:0/85836707 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4be80b3ab0 msgr2=0x7f4be805c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:40.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.860+0000 7f4bee895700 1 --2- 172.21.15.162:0/85836707 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4be80b3ab0 0x7f4be805c4d0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 rx=0x7f4be0001bf0 tx=0x7f4be4006dc0).stop 2020-11-28T21:53:40.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.860+0000 7f4bee895700 1 -- 172.21.15.162:0/85836707 shutdown_connections 2020-11-28T21:53:40.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.860+0000 7f4bee895700 1 --2- 172.21.15.162:0/85836707 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4be80b3ab0 0x7f4be805c4d0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:40.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.860+0000 7f4bee895700 1 -- 172.21.15.162:0/85836707 >> 172.21.15.162:0/85836707 conn(0x7f4be80604c0 msgr2=0x7f4be80608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:40.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.860+0000 7f4bee895700 1 -- 172.21.15.162:0/85836707 shutdown_connections 2020-11-28T21:53:40.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.860+0000 7f4bee895700 1 -- 172.21.15.162:0/85836707 wait complete. 2020-11-28T21:53:40.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.864+0000 7f4bee895700 1 Processor -- start 2020-11-28T21:53:40.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.864+0000 7f4bee895700 1 -- start start 2020-11-28T21:53:40.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.864+0000 7f4bee895700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4be80b2d00 0x7f4be80b5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:40.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.864+0000 7f4bee895700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4be80b30e0 0x7f4be80b90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:40.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.864+0000 7f4bee895700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4be80b3ab0 0x7f4be8142cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:40.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.864+0000 7f4bee895700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be813a3b0 con 0x7f4be80b3ab0 2020-11-28T21:53:40.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.864+0000 7f4bee895700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be80b95e0 con 0x7f4be80b30e0 2020-11-28T21:53:40.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.864+0000 7f4bee895700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be8142850 con 0x7f4be80b2d00 2020-11-28T21:53:40.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.864+0000 7f4bed092700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4be80b30e0 0x7f4be80b90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:40.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.864+0000 7f4bed092700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4be80b30e0 0x7f4be80b90f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42234/0 (socket says 172.21.15.162:42234) 2020-11-28T21:53:40.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.864+0000 7f4bed092700 1 -- 172.21.15.162:0/1822027706 learned_addr learned my addr 172.21.15.162:0/1822027706 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:40.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.864+0000 7f4bee094700 1 --2- 172.21.15.162:0/1822027706 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4be80b3ab0 0x7f4be8142cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:40.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.864+0000 7f4bed893700 1 --2- 172.21.15.162:0/1822027706 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4be80b2d00 0x7f4be80b5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:40.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.864+0000 7f4bed092700 1 -- 172.21.15.162:0/1822027706 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4be80b2d00 msgr2=0x7f4be80b5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:40.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.864+0000 7f4bed092700 1 --2- 172.21.15.162:0/1822027706 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4be80b2d00 0x7f4be80b5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:40.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.864+0000 7f4bed092700 1 -- 172.21.15.162:0/1822027706 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4be80b3ab0 msgr2=0x7f4be8142cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:40.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.864+0000 7f4bed092700 1 --2- 172.21.15.162:0/1822027706 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4be80b3ab0 0x7f4be8142cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:40.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.864+0000 7f4bed092700 1 -- 172.21.15.162:0/1822027706 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be813a890 con 0x7f4be80b30e0 2020-11-28T21:53:40.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.864+0000 7f4bed092700 1 --2- 172.21.15.162:0/1822027706 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4be80b30e0 0x7f4be80b90f0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 rx=0x7f4be0001bf0 tx=0x7f4be4015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:40.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.864+0000 7f4bdaffd700 1 -- 172.21.15.162:0/1822027706 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4be40118d0 con 0x7f4be80b30e0 2020-11-28T21:53:40.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.864+0000 7f4bdaffd700 1 -- 172.21.15.162:0/1822027706 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be4011a30 con 0x7f4be80b30e0 2020-11-28T21:53:40.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.864+0000 7f4bee895700 1 -- 172.21.15.162:0/1822027706 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4be813a5e0 con 0x7f4be80b30e0 2020-11-28T21:53:40.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.864+0000 7f4bdaffd700 1 -- 172.21.15.162:0/1822027706 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4be4011d90 con 0x7f4be80b30e0 2020-11-28T21:53:40.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.864+0000 7f4bee895700 1 -- 172.21.15.162:0/1822027706 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be81439d0 con 0x7f4be80b30e0 2020-11-28T21:53:40.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.868+0000 7f4bee895700 1 -- 172.21.15.162:0/1822027706 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4be8057050 con 0x7f4be80b30e0 2020-11-28T21:53:40.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.868+0000 7f4bdaffd700 1 -- 172.21.15.162:0/1822027706 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4be4017070 con 0x7f4be80b30e0 2020-11-28T21:53:40.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.868+0000 7f4bdaffd700 1 --2- 172.21.15.162:0/1822027706 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4bc80554e0 0x7f4bc80579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:40.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.868+0000 7f4bdaffd700 1 -- 172.21.15.162:0/1822027706 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4be406efd0 con 0x7f4be80b30e0 2020-11-28T21:53:40.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.868+0000 7f4bed893700 1 --2- 172.21.15.162:0/1822027706 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4bc80554e0 0x7f4bc80579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:40.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.868+0000 7f4bed893700 1 --2- 172.21.15.162:0/1822027706 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4bc80554e0 0x7f4bc80579a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f4bdc00d000 tx=0x7f4bdc000b80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:40.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:40.872+0000 7f4bdaffd700 1 -- 172.21.15.162:0/1822027706 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4be40452d0 con 0x7f4be80b30e0 2020-11-28T21:53:41.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.016+0000 7f4bee895700 1 -- 172.21.15.162:0/1822027706 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4be80b6320 con 0x7f4be80b30e0 2020-11-28T21:53:41.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.016+0000 7f4bdaffd700 1 -- 172.21.15.162:0/1822027706 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v88) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4be40488f0 con 0x7f4be80b30e0 2020-11-28T21:53:41.020 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:53:41.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.016+0000 7f4bee895700 1 -- 172.21.15.162:0/1822027706 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4bc80554e0 msgr2=0x7f4bc80579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:41.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.016+0000 7f4bee895700 1 --2- 172.21.15.162:0/1822027706 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4bc80554e0 0x7f4bc80579a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f4bdc00d000 tx=0x7f4bdc000b80).stop 2020-11-28T21:53:41.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.016+0000 7f4bee895700 1 -- 172.21.15.162:0/1822027706 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4be80b30e0 msgr2=0x7f4be80b90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:41.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.016+0000 7f4bee895700 1 --2- 172.21.15.162:0/1822027706 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4be80b30e0 0x7f4be80b90f0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 rx=0x7f4be0001bf0 tx=0x7f4be4015000).stop 2020-11-28T21:53:41.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.016+0000 7f4bee895700 1 -- 172.21.15.162:0/1822027706 shutdown_connections 2020-11-28T21:53:41.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.016+0000 7f4bee895700 1 --2- 172.21.15.162:0/1822027706 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4bc80554e0 0x7f4bc80579a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:41.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.016+0000 7f4bee895700 1 --2- 172.21.15.162:0/1822027706 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4be80b2d00 0x7f4be80b5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:41.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.016+0000 7f4bee895700 1 --2- 172.21.15.162:0/1822027706 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4be80b3ab0 0x7f4be8142cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:41.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.016+0000 7f4bee895700 1 --2- 172.21.15.162:0/1822027706 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4be80b30e0 0x7f4be80b90f0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:41.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.016+0000 7f4bee895700 1 -- 172.21.15.162:0/1822027706 >> 172.21.15.162:0/1822027706 conn(0x7f4be80604c0 msgr2=0x7f4be80b59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:41.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.016+0000 7f4bee895700 1 -- 172.21.15.162:0/1822027706 shutdown_connections 2020-11-28T21:53:41.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.016+0000 7f4bee895700 1 -- 172.21.15.162:0/1822027706 wait complete. 2020-11-28T21:53:41.032 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-28T21:53:41.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.144+0000 7f66d2a7c700 1 Processor -- start 2020-11-28T21:53:41.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.144+0000 7f66d2a7c700 1 -- start start 2020-11-28T21:53:41.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.144+0000 7f66d2a7c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f66cc1404c0 0x7f66cc060450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:41.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.144+0000 7f66d2a7c700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66cc0aff20 con 0x7f66cc0b21f0 2020-11-28T21:53:41.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.144+0000 7f66d2a7c700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66cc0b25d0 con 0x7f66cc060940 2020-11-28T21:53:41.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.144+0000 7f66d2a7c700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66cc0af240 con 0x7f66cc1404c0 2020-11-28T21:53:41.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.144+0000 7f66d1279700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f66cc1404c0 0x7f66cc060450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:41.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.144+0000 7f66d1279700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f66cc1404c0 0x7f66cc060450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42240/0 (socket says 172.21.15.162:42240) 2020-11-28T21:53:41.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.144+0000 7f66d1279700 1 -- 172.21.15.162:0/1675199603 learned_addr learned my addr 172.21.15.162:0/1675199603 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:41.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.144+0000 7f66d1279700 1 -- 172.21.15.162:0/1675199603 >> v1:172.21.15.162:6789/0 conn(0x7f66cc060940 legacy=0x7f66cc14d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:41.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.144+0000 7f66d1279700 1 -- 172.21.15.162:0/1675199603 >> v1:172.21.15.118:6789/0 conn(0x7f66cc0b21f0 legacy=0x7f66cc13fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:41.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.144+0000 7f66d1279700 1 -- 172.21.15.162:0/1675199603 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66cc05fff0 con 0x7f66cc1404c0 2020-11-28T21:53:41.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.144+0000 7f66d0a78700 1 -- 172.21.15.162:0/1675199603 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1099713008 0 0) 0x7f66cc0b25d0 con 0x7f66cc060940 2020-11-28T21:53:41.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.144+0000 7f66d1279700 1 --2- 172.21.15.162:0/1675199603 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66cc1404c0 0x7f66cc060450 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 rx=0x7f66c8002f30 tx=0x7f66c8006e80).ready entity=mon.1 client_cookie=17699ce9cfd3734f server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:41.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.144+0000 7f66d0a78700 1 -- 172.21.15.162:0/1675199603 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66c80097f0 con 0x7f66cc1404c0 2020-11-28T21:53:41.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.148+0000 7f66d0a78700 1 -- 172.21.15.162:0/1675199603 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66c8009950 con 0x7f66cc1404c0 2020-11-28T21:53:41.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.148+0000 7f66d0a78700 1 -- 172.21.15.162:0/1675199603 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66c8009cb0 con 0x7f66cc1404c0 2020-11-28T21:53:41.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.148+0000 7f66d2a7c700 1 -- 172.21.15.162:0/1675199603 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66cc1404c0 msgr2=0x7f66cc060450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:41.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.148+0000 7f66d2a7c700 1 --2- 172.21.15.162:0/1675199603 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66cc1404c0 0x7f66cc060450 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 rx=0x7f66c8002f30 tx=0x7f66c8006e80).stop 2020-11-28T21:53:41.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.148+0000 7f66d2a7c700 1 -- 172.21.15.162:0/1675199603 shutdown_connections 2020-11-28T21:53:41.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.148+0000 7f66d2a7c700 1 --2- 172.21.15.162:0/1675199603 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66cc1404c0 0x7f66cc060450 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:41.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.148+0000 7f66d2a7c700 1 -- 172.21.15.162:0/1675199603 >> 172.21.15.162:0/1675199603 conn(0x7f66cc0b57d0 msgr2=0x7f66cc0b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:41.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.148+0000 7f66d2a7c700 1 -- 172.21.15.162:0/1675199603 shutdown_connections 2020-11-28T21:53:41.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.148+0000 7f66d2a7c700 1 -- 172.21.15.162:0/1675199603 wait complete. 2020-11-28T21:53:41.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.148+0000 7f66d2a7c700 1 Processor -- start 2020-11-28T21:53:41.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.148+0000 7f66d2a7c700 1 -- start start 2020-11-28T21:53:41.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.148+0000 7f66d2a7c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66cc060940 0x7f66cc13d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:41.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.148+0000 7f66d2a7c700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f66cc0b21f0 0x7f66cc143e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:41.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.148+0000 7f66d2a7c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66cc1404c0 0x7f66cc144350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:41.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.148+0000 7f66d2a7c700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66cc141960 con 0x7f66cc060940 2020-11-28T21:53:41.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.148+0000 7f66d2a7c700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66cc142710 con 0x7f66cc1404c0 2020-11-28T21:53:41.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.148+0000 7f66d2a7c700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66cc142470 con 0x7f66cc0b21f0 2020-11-28T21:53:41.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.148+0000 7f66d227b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66cc1404c0 0x7f66cc144350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:41.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.148+0000 7f66d227b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66cc1404c0 0x7f66cc144350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42250/0 (socket says 172.21.15.162:42250) 2020-11-28T21:53:41.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.148+0000 7f66d227b700 1 -- 172.21.15.162:0/927126498 learned_addr learned my addr 172.21.15.162:0/927126498 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:41.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.148+0000 7f66d1a7a700 1 --2- 172.21.15.162:0/927126498 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66cc060940 0x7f66cc13d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:41.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.148+0000 7f66d1279700 1 --2- 172.21.15.162:0/927126498 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f66cc0b21f0 0x7f66cc143e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:41.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.148+0000 7f66d227b700 1 -- 172.21.15.162:0/927126498 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f66cc0b21f0 msgr2=0x7f66cc143e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:41.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.148+0000 7f66d227b700 1 --2- 172.21.15.162:0/927126498 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f66cc0b21f0 0x7f66cc143e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:41.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.148+0000 7f66d227b700 1 -- 172.21.15.162:0/927126498 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66cc060940 msgr2=0x7f66cc13d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:41.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.148+0000 7f66d227b700 1 --2- 172.21.15.162:0/927126498 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66cc060940 0x7f66cc13d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:41.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.148+0000 7f66d227b700 1 -- 172.21.15.162:0/927126498 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66cc141bc0 con 0x7f66cc1404c0 2020-11-28T21:53:41.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.152+0000 7f66d227b700 1 --2- 172.21.15.162:0/927126498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66cc1404c0 0x7f66cc144350 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 rx=0x7f66c4001210 tx=0x7f66c4007d90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:41.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.152+0000 7f66beffd700 1 -- 172.21.15.162:0/927126498 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66c400a7b0 con 0x7f66cc1404c0 2020-11-28T21:53:41.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.152+0000 7f66beffd700 1 -- 172.21.15.162:0/927126498 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66c400a910 con 0x7f66cc1404c0 2020-11-28T21:53:41.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.152+0000 7f66beffd700 1 -- 172.21.15.162:0/927126498 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66c400ac70 con 0x7f66cc1404c0 2020-11-28T21:53:41.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.152+0000 7f66d2a7c700 1 -- 172.21.15.162:0/927126498 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66cc1468d0 con 0x7f66cc1404c0 2020-11-28T21:53:41.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.152+0000 7f66d2a7c700 1 -- 172.21.15.162:0/927126498 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66cc146c00 con 0x7f66cc1404c0 2020-11-28T21:53:41.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.152+0000 7f66d2a7c700 1 -- 172.21.15.162:0/927126498 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66cc141f80 con 0x7f66cc1404c0 2020-11-28T21:53:41.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.152+0000 7f66beffd700 1 -- 172.21.15.162:0/927126498 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f66c4019030 con 0x7f66cc1404c0 2020-11-28T21:53:41.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.152+0000 7f66beffd700 1 --2- 172.21.15.162:0/927126498 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f66ac0553b0 0x7f66ac057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:41.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.152+0000 7f66beffd700 1 -- 172.21.15.162:0/927126498 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f66c406d710 con 0x7f66cc1404c0 2020-11-28T21:53:41.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.152+0000 7f66d1a7a700 1 --2- 172.21.15.162:0/927126498 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f66ac0553b0 0x7f66ac057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:41.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.152+0000 7f66d1a7a700 1 --2- 172.21.15.162:0/927126498 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f66ac0553b0 0x7f66ac057870 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f66c000d000 tx=0x7f66c0007b60).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:41.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.160+0000 7f66beffd700 1 -- 172.21.15.162:0/927126498 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f66c4041df0 con 0x7f66cc1404c0 2020-11-28T21:53:41.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.296+0000 7f66d2a7c700 1 -- 172.21.15.162:0/927126498 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f66cc057050 con 0x7f66cc1404c0 2020-11-28T21:53:41.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.296+0000 7f66beffd700 1 -- 172.21.15.162:0/927126498 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+10 (secure 0 0 0) 0x7f66c4043a20 con 0x7f66cc1404c0 2020-11-28T21:53:41.302 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:53:41.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.300+0000 7f66d2a7c700 1 -- 172.21.15.162:0/927126498 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f66ac0553b0 msgr2=0x7f66ac057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:41.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.300+0000 7f66d2a7c700 1 --2- 172.21.15.162:0/927126498 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f66ac0553b0 0x7f66ac057870 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f66c000d000 tx=0x7f66c0007b60).stop 2020-11-28T21:53:41.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.300+0000 7f66d2a7c700 1 -- 172.21.15.162:0/927126498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66cc1404c0 msgr2=0x7f66cc144350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:41.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.300+0000 7f66d2a7c700 1 --2- 172.21.15.162:0/927126498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66cc1404c0 0x7f66cc144350 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 rx=0x7f66c4001210 tx=0x7f66c4007d90).stop 2020-11-28T21:53:41.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.300+0000 7f66d2a7c700 1 -- 172.21.15.162:0/927126498 shutdown_connections 2020-11-28T21:53:41.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.300+0000 7f66d2a7c700 1 --2- 172.21.15.162:0/927126498 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f66ac0553b0 0x7f66ac057870 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:41.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.300+0000 7f66d2a7c700 1 --2- 172.21.15.162:0/927126498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66cc1404c0 0x7f66cc144350 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:41.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.300+0000 7f66d2a7c700 1 --2- 172.21.15.162:0/927126498 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66cc060940 0x7f66cc13d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:41.306 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.300+0000 7f66d2a7c700 1 --2- 172.21.15.162:0/927126498 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f66cc0b21f0 0x7f66cc143e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:41.306 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.300+0000 7f66d2a7c700 1 -- 172.21.15.162:0/927126498 >> 172.21.15.162:0/927126498 conn(0x7f66cc0b57d0 msgr2=0x7f66cc142eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:41.306 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.300+0000 7f66d2a7c700 1 -- 172.21.15.162:0/927126498 shutdown_connections 2020-11-28T21:53:41.306 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.300+0000 7f66d2a7c700 1 -- 172.21.15.162:0/927126498 wait complete. 2020-11-28T21:53:41.315 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-28T21:53:41.428 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.424+0000 7ff544849700 1 Processor -- start 2020-11-28T21:53:41.428 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.424+0000 7ff544849700 1 -- start start 2020-11-28T21:53:41.429 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.424+0000 7ff544849700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff5400b18c0 0x7ff5400b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:41.429 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.424+0000 7ff544849700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff54014a440 con 0x7ff5400b3950 2020-11-28T21:53:41.430 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.424+0000 7ff544849700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff54014b2f0 con 0x7ff5400b1dd0 2020-11-28T21:53:41.430 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.424+0000 7ff544849700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff54014a670 con 0x7ff5400b18c0 2020-11-28T21:53:41.430 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.424+0000 7ff53effd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff5400b18c0 0x7ff5400b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:41.430 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.424+0000 7ff53ffff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7ff5400b1dd0 0x7ff540146e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:49102/0 (socket says 172.21.15.162:49102) 2020-11-28T21:53:41.431 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.424+0000 7ff53ffff700 1 -- 172.21.15.162:0/2035363956 learned_addr learned my addr 172.21.15.162:0/2035363956 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:41.432 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.424+0000 7ff53e7fc700 1 -- 172.21.15.162:0/2035363956 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3965379756 0 0) 0x7ff54014b2f0 con 0x7ff5400b1dd0 2020-11-28T21:53:41.432 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.424+0000 7ff53e7fc700 1 -- 172.21.15.162:0/2035363956 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff52c003230 con 0x7ff5400b1dd0 2020-11-28T21:53:41.433 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.424+0000 7ff53e7fc700 1 -- 172.21.15.162:0/2035363956 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2174527793 0 0) 0x7ff54014a440 con 0x7ff5400b3950 2020-11-28T21:53:41.433 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.424+0000 7ff53e7fc700 1 -- 172.21.15.162:0/2035363956 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff54014b2f0 con 0x7ff5400b3950 2020-11-28T21:53:41.433 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.424+0000 7ff53effd700 1 -- 172.21.15.162:0/2035363956 >> v1:172.21.15.162:6789/0 conn(0x7ff5400b1dd0 legacy=0x7ff540146e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:41.433 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.424+0000 7ff53effd700 1 -- 172.21.15.162:0/2035363956 >> v1:172.21.15.118:6789/0 conn(0x7ff5400b3950 legacy=0x7ff540060bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:41.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.424+0000 7ff53effd700 1 -- 172.21.15.162:0/2035363956 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff54014a910 con 0x7ff5400b18c0 2020-11-28T21:53:41.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.428+0000 7ff53effd700 1 --2- 172.21.15.162:0/2035363956 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5400b18c0 0x7ff5400b3d30 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 rx=0x7ff5300032b0 tx=0x7ff5300091a0).ready entity=mon.1 client_cookie=f45115f41db00146 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:41.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.428+0000 7ff53e7fc700 1 -- 172.21.15.162:0/2035363956 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff53000b940 con 0x7ff5400b18c0 2020-11-28T21:53:41.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.428+0000 7ff53e7fc700 1 -- 172.21.15.162:0/2035363956 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff53000baa0 con 0x7ff5400b18c0 2020-11-28T21:53:41.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.428+0000 7ff53e7fc700 1 -- 172.21.15.162:0/2035363956 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff53000bc10 con 0x7ff5400b18c0 2020-11-28T21:53:41.435 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.428+0000 7ff544849700 1 -- 172.21.15.162:0/2035363956 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5400b18c0 msgr2=0x7ff5400b3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:41.435 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.428+0000 7ff544849700 1 --2- 172.21.15.162:0/2035363956 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5400b18c0 0x7ff5400b3d30 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 rx=0x7ff5300032b0 tx=0x7ff5300091a0).stop 2020-11-28T21:53:41.435 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.428+0000 7ff544849700 1 -- 172.21.15.162:0/2035363956 shutdown_connections 2020-11-28T21:53:41.435 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.428+0000 7ff544849700 1 --2- 172.21.15.162:0/2035363956 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5400b18c0 0x7ff5400b3d30 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:41.435 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.428+0000 7ff544849700 1 -- 172.21.15.162:0/2035363956 >> 172.21.15.162:0/2035363956 conn(0x7ff540062630 msgr2=0x7ff540062a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:41.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.428+0000 7ff544849700 1 -- 172.21.15.162:0/2035363956 shutdown_connections 2020-11-28T21:53:41.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.428+0000 7ff544849700 1 -- 172.21.15.162:0/2035363956 wait complete. 2020-11-28T21:53:41.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.428+0000 7ff544849700 1 Processor -- start 2020-11-28T21:53:41.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.428+0000 7ff544849700 1 -- start start 2020-11-28T21:53:41.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.432+0000 7ff544849700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff5400b18c0 0x7ff540146240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:41.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.432+0000 7ff544849700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff5400b1dd0 0x7ff5400ab0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:41.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.432+0000 7ff544849700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5400b3950 0x7ff540143930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:41.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.432+0000 7ff544849700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5400b1510 con 0x7ff5400b18c0 2020-11-28T21:53:41.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.432+0000 7ff544849700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5400acaa0 con 0x7ff5400b3950 2020-11-28T21:53:41.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.432+0000 7ff544849700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5400acd40 con 0x7ff5400b1dd0 2020-11-28T21:53:41.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.432+0000 7ff53f7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff5400b18c0 0x7ff540146240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:41.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.432+0000 7ff53f7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff5400b18c0 0x7ff540146240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.162:49248/0 (socket says 172.21.15.162:49248) 2020-11-28T21:53:41.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.432+0000 7ff53f7fe700 1 -- 172.21.15.162:0/509068067 learned_addr learned my addr 172.21.15.162:0/509068067 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:41.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.432+0000 7ff53ffff700 1 --2- 172.21.15.162:0/509068067 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5400b3950 0x7ff540143930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:41.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.432+0000 7ff53effd700 1 --2- 172.21.15.162:0/509068067 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff5400b1dd0 0x7ff5400ab0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:41.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.432+0000 7ff53ffff700 1 -- 172.21.15.162:0/509068067 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff5400b1dd0 msgr2=0x7ff5400ab0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:41.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.432+0000 7ff53ffff700 1 --2- 172.21.15.162:0/509068067 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff5400b1dd0 0x7ff5400ab0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:41.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.432+0000 7ff53ffff700 1 -- 172.21.15.162:0/509068067 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff5400b18c0 msgr2=0x7ff540146240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:41.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.432+0000 7ff53ffff700 1 --2- 172.21.15.162:0/509068067 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff5400b18c0 0x7ff540146240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:41.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.432+0000 7ff53ffff700 1 -- 172.21.15.162:0/509068067 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5400acfe0 con 0x7ff5400b3950 2020-11-28T21:53:41.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.432+0000 7ff53ffff700 1 --2- 172.21.15.162:0/509068067 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5400b3950 0x7ff540143930 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0x7ff5380010d0 tx=0x7ff5380079b0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:41.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.432+0000 7ff53cff9700 1 -- 172.21.15.162:0/509068067 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff5380072e0 con 0x7ff5400b3950 2020-11-28T21:53:41.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.432+0000 7ff544849700 1 -- 172.21.15.162:0/509068067 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5400ad280 con 0x7ff5400b3950 2020-11-28T21:53:41.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.432+0000 7ff544849700 1 -- 172.21.15.162:0/509068067 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff540145e40 con 0x7ff5400b3950 2020-11-28T21:53:41.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.432+0000 7ff53cff9700 1 -- 172.21.15.162:0/509068067 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff538007440 con 0x7ff5400b3950 2020-11-28T21:53:41.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.432+0000 7ff53cff9700 1 -- 172.21.15.162:0/509068067 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff5380057d0 con 0x7ff5400b3950 2020-11-28T21:53:41.444 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.436+0000 7ff544849700 1 -- 172.21.15.162:0/509068067 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff540057050 con 0x7ff5400b3950 2020-11-28T21:53:41.444 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.436+0000 7ff53cff9700 1 -- 172.21.15.162:0/509068067 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff53801a060 con 0x7ff5400b3950 2020-11-28T21:53:41.444 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.436+0000 7ff53cff9700 1 --2- 172.21.15.162:0/509068067 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff5200554e0 0x7ff5200579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:41.445 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.436+0000 7ff53cff9700 1 -- 172.21.15.162:0/509068067 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff53806d600 con 0x7ff5400b3950 2020-11-28T21:53:41.445 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.436+0000 7ff53f7fe700 1 --2- 172.21.15.162:0/509068067 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff5200554e0 0x7ff5200579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:41.445 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.436+0000 7ff53f7fe700 1 --2- 172.21.15.162:0/509068067 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff5200554e0 0x7ff5200579a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7ff53400d000 tx=0x7ff534007a60).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:41.446 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.440+0000 7ff53cff9700 1 -- 172.21.15.162:0/509068067 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff538043900 con 0x7ff5400b3950 2020-11-28T21:53:41.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.580+0000 7ff544849700 1 -- 172.21.15.162:0/509068067 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff54005d250 con 0x7ff5400b3950 2020-11-28T21:53:41.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.580+0000 7ff53cff9700 1 -- 172.21.15.162:0/509068067 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+10 (secure 0 0 0) 0x7ff538018020 con 0x7ff5400b3950 2020-11-28T21:53:41.585 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:53:41.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.584+0000 7ff544849700 1 -- 172.21.15.162:0/509068067 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff5200554e0 msgr2=0x7ff5200579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:41.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.584+0000 7ff544849700 1 --2- 172.21.15.162:0/509068067 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff5200554e0 0x7ff5200579a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7ff53400d000 tx=0x7ff534007a60).stop 2020-11-28T21:53:41.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.584+0000 7ff544849700 1 -- 172.21.15.162:0/509068067 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5400b3950 msgr2=0x7ff540143930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:41.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.584+0000 7ff544849700 1 --2- 172.21.15.162:0/509068067 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5400b3950 0x7ff540143930 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0x7ff5380010d0 tx=0x7ff5380079b0).stop 2020-11-28T21:53:41.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.584+0000 7ff544849700 1 -- 172.21.15.162:0/509068067 shutdown_connections 2020-11-28T21:53:41.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.584+0000 7ff544849700 1 --2- 172.21.15.162:0/509068067 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff5200554e0 0x7ff5200579a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:41.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.584+0000 7ff544849700 1 --2- 172.21.15.162:0/509068067 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5400b3950 0x7ff540143930 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:41.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.584+0000 7ff544849700 1 --2- 172.21.15.162:0/509068067 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff5400b18c0 0x7ff540146240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:41.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.584+0000 7ff544849700 1 --2- 172.21.15.162:0/509068067 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff5400b1dd0 0x7ff5400ab0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:41.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.584+0000 7ff544849700 1 -- 172.21.15.162:0/509068067 >> 172.21.15.162:0/509068067 conn(0x7ff540062630 msgr2=0x7ff5400609a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:41.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.584+0000 7ff544849700 1 -- 172.21.15.162:0/509068067 shutdown_connections 2020-11-28T21:53:41.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.584+0000 7ff544849700 1 -- 172.21.15.162:0/509068067 wait complete. 2020-11-28T21:53:41.604 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-28T21:53:41.604 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:53:41.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.712+0000 7f00906b6700 1 Processor -- start 2020-11-28T21:53:41.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.712+0000 7f00906b6700 1 -- start start 2020-11-28T21:53:41.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.712+0000 7f00906b6700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f00880b69a0 0x7f008813f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:41.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.712+0000 7f00906b6700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f008813fae0 0x7f008813c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:41.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.712+0000 7f00906b6700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0088067e90 con 0x7f008813c900 2020-11-28T21:53:41.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.712+0000 7f00906b6700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00880ae270 con 0x7f00880b69a0 2020-11-28T21:53:41.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.712+0000 7f00906b6700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0088142ed0 con 0x7f008813fae0 2020-11-28T21:53:41.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.712+0000 7f008dc51700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f008813fae0 0x7f008813c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:41.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.712+0000 7f008e452700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f00880b69a0 0x7f008813f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:41.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.712+0000 7f008dc51700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f008813fae0 0x7f008813c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:39492/0 (socket says 172.21.15.162:39492) 2020-11-28T21:53:41.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.712+0000 7f008dc51700 1 -- 172.21.15.162:0/3589827508 learned_addr learned my addr 172.21.15.162:0/3589827508 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:41.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.712+0000 7f008dc51700 1 -- 172.21.15.162:0/3589827508 >> v2:172.21.15.118:3300/0 conn(0x7f00880b69a0 msgr2=0x7f008813f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:41.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.712+0000 7f008dc51700 1 --2- 172.21.15.162:0/3589827508 >> v2:172.21.15.118:3300/0 conn(0x7f00880b69a0 0x7f008813f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:41.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.712+0000 7f008dc51700 1 -- 172.21.15.162:0/3589827508 >> v1:172.21.15.118:6789/0 conn(0x7f008813c900 legacy=0x7f008805f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:41.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.712+0000 7f008dc51700 1 -- 172.21.15.162:0/3589827508 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00880b6d80 con 0x7f008813fae0 2020-11-28T21:53:41.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.712+0000 7f008dc51700 1 --2- 172.21.15.162:0/3589827508 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f008813fae0 0x7f008813c410 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f007c002f30 tx=0x7f007c006e80).ready entity=mon.2 client_cookie=a6251a00830a7d42 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:41.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.712+0000 7f008d450700 1 -- 172.21.15.162:0/3589827508 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f007c0097f0 con 0x7f008813fae0 2020-11-28T21:53:41.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.712+0000 7f008d450700 1 -- 172.21.15.162:0/3589827508 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f007c009950 con 0x7f008813fae0 2020-11-28T21:53:41.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.712+0000 7f008d450700 1 -- 172.21.15.162:0/3589827508 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f007c009cb0 con 0x7f008813fae0 2020-11-28T21:53:41.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.712+0000 7f00906b6700 1 -- 172.21.15.162:0/3589827508 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f008813fae0 msgr2=0x7f008813c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:41.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.712+0000 7f00906b6700 1 --2- 172.21.15.162:0/3589827508 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f008813fae0 0x7f008813c410 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f007c002f30 tx=0x7f007c006e80).stop 2020-11-28T21:53:41.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.716+0000 7f00906b6700 1 -- 172.21.15.162:0/3589827508 shutdown_connections 2020-11-28T21:53:41.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.716+0000 7f00906b6700 1 --2- 172.21.15.162:0/3589827508 >> v2:172.21.15.118:3300/0 conn(0x7f00880b69a0 0x7f008813f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:41.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.716+0000 7f00906b6700 1 --2- 172.21.15.162:0/3589827508 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f008813fae0 0x7f008813c410 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:41.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.716+0000 7f00906b6700 1 -- 172.21.15.162:0/3589827508 >> 172.21.15.162:0/3589827508 conn(0x7f0088062170 msgr2=0x7f00880b5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:41.723 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.716+0000 7f00906b6700 1 -- 172.21.15.162:0/3589827508 shutdown_connections 2020-11-28T21:53:41.723 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.716+0000 7f00906b6700 1 -- 172.21.15.162:0/3589827508 wait complete. 2020-11-28T21:53:41.725 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.716+0000 7f00906b6700 1 Processor -- start 2020-11-28T21:53:41.725 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.716+0000 7f00906b6700 1 -- start start 2020-11-28T21:53:41.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.716+0000 7f00906b6700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f00880b69a0 0x7f0088141cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:41.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.716+0000 7f00906b6700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f008813c900 0x7f008813eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:41.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.716+0000 7f00906b6700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f008813fae0 0x7f008813d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:41.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.716+0000 7f00906b6700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00880b11f0 con 0x7f00880b69a0 2020-11-28T21:53:41.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.716+0000 7f00906b6700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00880b3ca0 con 0x7f008813fae0 2020-11-28T21:53:41.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.716+0000 7f00906b6700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f008813def0 con 0x7f008813c900 2020-11-28T21:53:41.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.716+0000 7f008ec53700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f008813fae0 0x7f008813d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:41.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.716+0000 7f008e452700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f00880b69a0 0x7f0088141cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:41.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.716+0000 7f008e452700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f00880b69a0 0x7f0088141cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.162:49262/0 (socket says 172.21.15.162:49262) 2020-11-28T21:53:41.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.716+0000 7f008ec53700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f008813fae0 0x7f008813d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42278/0 (socket says 172.21.15.162:42278) 2020-11-28T21:53:41.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.716+0000 7f008e452700 1 -- 172.21.15.162:0/1015444930 learned_addr learned my addr 172.21.15.162:0/1015444930 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:41.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.716+0000 7f008dc51700 1 --2- 172.21.15.162:0/1015444930 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f008813c900 0x7f008813eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:41.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.720+0000 7f008ec53700 1 -- 172.21.15.162:0/1015444930 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f008813c900 msgr2=0x7f008813eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:41.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.720+0000 7f008ec53700 1 --2- 172.21.15.162:0/1015444930 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f008813c900 0x7f008813eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:41.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.720+0000 7f008ec53700 1 -- 172.21.15.162:0/1015444930 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f00880b69a0 msgr2=0x7f0088141cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:41.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.720+0000 7f008ec53700 1 --2- 172.21.15.162:0/1015444930 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f00880b69a0 0x7f0088141cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:41.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.720+0000 7f008ec53700 1 -- 172.21.15.162:0/1015444930 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00880b1420 con 0x7f008813fae0 2020-11-28T21:53:41.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.720+0000 7f008e452700 1 --2- 172.21.15.162:0/1015444930 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f00880b69a0 0x7f0088141cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:53:41.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.720+0000 7f008ec53700 1 --2- 172.21.15.162:0/1015444930 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f008813fae0 0x7f008813d540 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 rx=0x7f0088063240 tx=0x7f0080007e80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:41.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.720+0000 7f007b7fe700 1 -- 172.21.15.162:0/1015444930 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f008000a5a0 con 0x7f008813fae0 2020-11-28T21:53:41.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.720+0000 7f00906b6700 1 -- 172.21.15.162:0/1015444930 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f008813e150 con 0x7f008813fae0 2020-11-28T21:53:41.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.720+0000 7f007b7fe700 1 -- 172.21.15.162:0/1015444930 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f008000a700 con 0x7f008813fae0 2020-11-28T21:53:41.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.720+0000 7f007b7fe700 1 -- 172.21.15.162:0/1015444930 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f008000aa60 con 0x7f008813fae0 2020-11-28T21:53:41.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.720+0000 7f00906b6700 1 -- 172.21.15.162:0/1015444930 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0088142810 con 0x7f008813fae0 2020-11-28T21:53:41.733 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.720+0000 7f00906b6700 1 -- 172.21.15.162:0/1015444930 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f008813e2e0 con 0x7f008813fae0 2020-11-28T21:53:41.734 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.720+0000 7f007b7fe700 1 -- 172.21.15.162:0/1015444930 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0080019030 con 0x7f008813fae0 2020-11-28T21:53:41.734 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.720+0000 7f007b7fe700 1 --2- 172.21.15.162:0/1015444930 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f00680553b0 0x7f0068057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:41.734 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.720+0000 7f007b7fe700 1 -- 172.21.15.162:0/1015444930 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f008006d740 con 0x7f008813fae0 2020-11-28T21:53:41.734 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.724+0000 7f008e452700 1 --2- 172.21.15.162:0/1015444930 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f00680553b0 0x7f0068057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:41.735 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.724+0000 7f008e452700 1 --2- 172.21.15.162:0/1015444930 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f00680553b0 0x7f0068057870 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f0084006a50 tx=0x7f0084006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:41.735 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.728+0000 7f007b7fe700 1 -- 172.21.15.162:0/1015444930 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f00800439a0 con 0x7f008813fae0 2020-11-28T21:53:41.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.920+0000 7f00906b6700 1 -- 172.21.15.162:0/1015444930 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00880b15b0 con 0x7f008813fae0 2020-11-28T21:53:41.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.920+0000 7f007b7fe700 1 -- 172.21.15.162:0/1015444930 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+2502 (secure 0 0 0) 0x7f0080046fc0 con 0x7f008813fae0 2020-11-28T21:53:41.924 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:53:41.924 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":40,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5795,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/2645168518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":2645168518},{"type":"v1","addr":"172.21.15.118:6833","nonce":2645168518}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36},{"gid":6541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1999073284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1999073284},{"type":"v1","addr":"172.21.15.162:6835","nonce":1999073284}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:53:36.539204+0000","modified":"2020-11-28T21:53:38.035411+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6550":{"gid":6550,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/1097065372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1097065372},{"type":"v1","addr":"172.21.15.162:6837","nonce":1097065372}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-28T21:53:41.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.920+0000 7f00906b6700 1 -- 172.21.15.162:0/1015444930 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f00680553b0 msgr2=0x7f0068057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:41.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.920+0000 7f00906b6700 1 --2- 172.21.15.162:0/1015444930 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f00680553b0 0x7f0068057870 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f0084006a50 tx=0x7f0084006dc0).stop 2020-11-28T21:53:41.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.920+0000 7f00906b6700 1 -- 172.21.15.162:0/1015444930 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f008813fae0 msgr2=0x7f008813d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:41.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.920+0000 7f00906b6700 1 --2- 172.21.15.162:0/1015444930 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f008813fae0 0x7f008813d540 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 rx=0x7f0088063240 tx=0x7f0080007e80).stop 2020-11-28T21:53:41.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.920+0000 7f00906b6700 1 -- 172.21.15.162:0/1015444930 shutdown_connections 2020-11-28T21:53:41.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.920+0000 7f00906b6700 1 --2- 172.21.15.162:0/1015444930 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f00680553b0 0x7f0068057870 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:41.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.920+0000 7f00906b6700 1 --2- 172.21.15.162:0/1015444930 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f008813fae0 0x7f008813d540 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:41.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.920+0000 7f00906b6700 1 --2- 172.21.15.162:0/1015444930 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f00880b69a0 0x7f0088141cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:41.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.920+0000 7f00906b6700 1 --2- 172.21.15.162:0/1015444930 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f008813c900 0x7f008813eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:41.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.920+0000 7f00906b6700 1 -- 172.21.15.162:0/1015444930 >> 172.21.15.162:0/1015444930 conn(0x7f0088062170 msgr2=0x7f00880b4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:41.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.920+0000 7f00906b6700 1 -- 172.21.15.162:0/1015444930 shutdown_connections 2020-11-28T21:53:41.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:41.920+0000 7f00906b6700 1 -- 172.21.15.162:0/1015444930 wait complete. 2020-11-28T21:53:41.929 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 40 2020-11-28T21:53:42.240 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 40, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-28T21:53:36.539204+0000', 'modified': '2020-11-28T21:53:38.035411+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6550}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6550': {'gid': 6550, 'name': 'c', 'rank': 0, 'incarnation': 38, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6837/1097065372', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6836', 'nonce': 1097065372}, {'type': 'v1', 'addr': '172.21.15.162:6837', 'nonce': 1097065372}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-28T21:53:42.295 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-28T21:53:42.296 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-28T21:53:42.423 INFO:teuthology.orchestra.run.smithi162.stdout:{"cluster_fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","whoami":0,"id":6550,"want_state":"up:active","state":"up:active","rank_uptime":5.8690140059999996,"mdsmap_epoch":40,"osdmap_epoch":88,"osdmap_epoch_barrier":88,"uptime":11.26606028} 2020-11-28T21:53:42.423 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", "id": 6550, "mdsmap_epoch": 40, "osdmap_epoch": 88, "osdmap_epoch_barrier": 88, "rank_uptime": 5.869014006, "state": "up:active", "uptime": 11.26606028, "want_state": "up:active", "whoami": 0 } 2020-11-28T21:53:42.424 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-11-28T21:53:42.424 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-11-28T21:53:42.424 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi192.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-11-28T21:53:42.425 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:53:42.425 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:53:42.433 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:53:42.491 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:53:42.492 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:53:42.546 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:53:42.547 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:53:42.547 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:53:42.548 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:53:42.593 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-11-28T21:53:42.593 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:53:42.637 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:53:42.652 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:53:42.653 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:53:42.711 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:53:42.712 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:53:42.713 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:53:42.713 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:53:42.714 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:2020-11-28T21:53:42.707+0000 7f0aa3e5f0c0 -1 init, newargv = 0x55d914967430 newargc=9 2020-11-28T21:53:42.715 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:ceph-fuse[12215]: starting ceph client 2020-11-28T21:53:42.725 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:ceph-fuse[12215]: starting fuse 2020-11-28T21:53:43.822 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:53:43.838 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:53:43.838 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:53:43.894 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:53:43.895 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:53:43.895 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:53:43.896 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:53:43.941 INFO:teuthology.orchestra.run.smithi192.stdout:49 2020-11-28T21:53:43.942 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2020-11-28T21:53:43.942 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:53:43.943 INFO:teuthology.orchestra.run.smithi192:> sudo python3 -c ' 2020-11-28T21:53:43.943 INFO:teuthology.orchestra.run.smithi192:> import glob 2020-11-28T21:53:43.944 INFO:teuthology.orchestra.run.smithi192:> import re 2020-11-28T21:53:43.944 INFO:teuthology.orchestra.run.smithi192:> import os 2020-11-28T21:53:43.944 INFO:teuthology.orchestra.run.smithi192:> import subprocess 2020-11-28T21:53:43.945 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:53:43.946 INFO:teuthology.orchestra.run.smithi192:> def find_socket(client_name): 2020-11-28T21:53:43.946 INFO:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-11-28T21:53:43.946 INFO:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2020-11-28T21:53:43.947 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:53:43.947 INFO:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2020-11-28T21:53:43.948 INFO:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2020-11-28T21:53:43.948 INFO:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2020-11-28T21:53:43.948 INFO:teuthology.orchestra.run.smithi192:> return files[0] 2020-11-28T21:53:43.949 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:53:43.949 INFO:teuthology.orchestra.run.smithi192:> for f in files: 2020-11-28T21:53:43.950 INFO:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-11-28T21:53:43.950 INFO:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2020-11-28T21:53:43.950 INFO:teuthology.orchestra.run.smithi192:> return f 2020-11-28T21:53:43.951 INFO:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-11-28T21:53:43.951 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:53:43.951 INFO:teuthology.orchestra.run.smithi192:> print(find_socket("client.0")) 2020-11-28T21:53:43.952 INFO:teuthology.orchestra.run.smithi192:> ' 2020-11-28T21:53:44.032 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.0.12215.asok 2020-11-28T21:53:44.036 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.12215.asok 2020-11-28T21:53:44.036 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:53:44.037 INFO:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.12215.asok status 2020-11-28T21:53:44.174 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2020-11-28T21:53:44.174 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2020-11-28T21:53:44.174 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "a83a3572a957356977342f461361c1cd3ec06dbd", 2020-11-28T21:53:44.175 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 15.2.6-595-ga83a3572 (a83a3572a957356977342f461361c1cd3ec06dbd) octopus (stable)", 2020-11-28T21:53:44.175 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "0", 2020-11-28T21:53:44.175 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2020-11-28T21:53:44.175 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-11-28T21:53:44.175 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "12215", 2020-11-28T21:53:44.176 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2020-11-28T21:53:44.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:53:44.176 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2020-11-28T21:53:44.176 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2020-11-28T21:53:44.177 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 6706, 2020-11-28T21:53:44.177 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2020-11-28T21:53:44.177 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2020-11-28T21:53:44.177 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2020-11-28T21:53:44.178 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 6706 2020-11-28T21:53:44.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:53:44.179 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2020-11-28T21:53:44.179 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2020-11-28T21:53:44.179 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2020-11-28T21:53:44.179 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 1558076735 2020-11-28T21:53:44.180 INFO:teuthology.orchestra.run.smithi192.stdout: } 2020-11-28T21:53:44.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:53:44.180 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2020-11-28T21:53:44.180 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2020-11-28T21:53:44.181 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2020-11-28T21:53:44.181 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 1558076735 2020-11-28T21:53:44.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:53:44.182 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.6706 172.21.15.192:0/1558076735", 2020-11-28T21:53:44.182 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "172.21.15.192:0/1558076735", 2020-11-28T21:53:44.182 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2020-11-28T21:53:44.182 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 40, 2020-11-28T21:53:44.183 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 88, 2020-11-28T21:53:44.183 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2020-11-28T21:53:44.183 INFO:teuthology.orchestra.run.smithi192.stdout: "blacklisted": false 2020-11-28T21:53:44.183 INFO:teuthology.orchestra.run.smithi192.stdout:} 2020-11-28T21:53:44.184 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:53:44.185 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:53:44.185 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:53:44.194 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2020-11-28T21:53:44.195 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-28T21:53:44.196 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:53:44.196 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:53:44.254 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:53:44.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.368+0000 7f8a92283700 1 Processor -- start 2020-11-28T21:53:44.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.368+0000 7f8a92283700 1 -- start start 2020-11-28T21:53:44.377 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a92283700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8a8c0994f0 0x7f8a8c0b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:44.377 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a92283700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a8c098f10 con 0x7f8a8c098b30 2020-11-28T21:53:44.377 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a92283700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a8c0b0680 con 0x7f8a8c060710 2020-11-28T21:53:44.378 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a92283700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8c0b1030 con 0x7f8a8c0994f0 2020-11-28T21:53:44.378 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a91a82700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f8a8c060710 0x7f8a8c0b1580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:49130/0 (socket says 172.21.15.162:49130) 2020-11-28T21:53:44.378 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a91a82700 1 -- 172.21.15.162:0/4011058821 learned_addr learned my addr 172.21.15.162:0/4011058821 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:44.378 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a90a80700 1 --2- 172.21.15.162:0/4011058821 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8a8c0994f0 0x7f8a8c0b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:44.379 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a7ffff700 1 -- 172.21.15.162:0/4011058821 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3058693407 0 0) 0x7f8a8c0b0680 con 0x7f8a8c060710 2020-11-28T21:53:44.379 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a7ffff700 1 -- 172.21.15.162:0/4011058821 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a70003230 con 0x7f8a8c060710 2020-11-28T21:53:44.379 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a7ffff700 1 -- 172.21.15.162:0/4011058821 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2740744318 0 0) 0x7f8a8c098f10 con 0x7f8a8c098b30 2020-11-28T21:53:44.379 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a7ffff700 1 -- 172.21.15.162:0/4011058821 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a8c0b0680 con 0x7f8a8c098b30 2020-11-28T21:53:44.380 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a7ffff700 1 -- 172.21.15.162:0/4011058821 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3902825165 0 0) 0x7f8a70003230 con 0x7f8a8c060710 2020-11-28T21:53:44.380 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a7ffff700 1 -- 172.21.15.162:0/4011058821 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8a8c0994f0 msgr2=0x7f8a8c0b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:44.380 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a7ffff700 1 --2- 172.21.15.162:0/4011058821 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8a8c0994f0 0x7f8a8c0b0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:44.380 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a7ffff700 1 -- 172.21.15.162:0/4011058821 >> v1:172.21.15.118:6789/0 conn(0x7f8a8c098b30 legacy=0x7f8a8c060000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:44.381 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a7ffff700 1 -- 172.21.15.162:0/4011058821 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a8c0b0920 con 0x7f8a8c060710 2020-11-28T21:53:44.381 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a7ffff700 1 -- 172.21.15.162:0/4011058821 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f8a84001a50 con 0x7f8a8c060710 2020-11-28T21:53:44.381 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a7ffff700 1 -- 172.21.15.162:0/4011058821 >> v1:172.21.15.162:6789/0 conn(0x7f8a8c060710 legacy=0x7f8a8c0b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:44.381 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a7ffff700 1 --2- 172.21.15.162:0/4011058821 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8a8c0994f0 0x7f8a700038b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:44.381 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a7ffff700 1 --2- 172.21.15.162:0/4011058821 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a70003df0 0x7f8a700044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:44.382 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a7ffff700 1 --2- 172.21.15.162:0/4011058821 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8a700049c0 0x7f8a70004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:44.382 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a7ffff700 1 -- 172.21.15.162:0/4011058821 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8c0b1030 con 0x7f8a700049c0 2020-11-28T21:53:44.382 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a7ffff700 1 -- 172.21.15.162:0/4011058821 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a70005390 con 0x7f8a70003df0 2020-11-28T21:53:44.382 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a7ffff700 1 -- 172.21.15.162:0/4011058821 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7000b7a0 con 0x7f8a8c0994f0 2020-11-28T21:53:44.382 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a90a80700 1 --2- 172.21.15.162:0/4011058821 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a70003df0 0x7f8a700044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:44.383 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a91a82700 1 --2- 172.21.15.162:0/4011058821 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8a700049c0 0x7f8a70004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:44.383 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a91281700 1 --2- 172.21.15.162:0/4011058821 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8a8c0994f0 0x7f8a700038b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:44.383 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a90a80700 1 -- 172.21.15.162:0/4011058821 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8a8c0994f0 msgr2=0x7f8a700038b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:44.383 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a90a80700 1 --2- 172.21.15.162:0/4011058821 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8a8c0994f0 0x7f8a700038b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:44.383 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a90a80700 1 -- 172.21.15.162:0/4011058821 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8a700049c0 msgr2=0x7f8a70004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:44.384 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a90a80700 1 --2- 172.21.15.162:0/4011058821 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8a700049c0 0x7f8a70004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:44.384 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a90a80700 1 -- 172.21.15.162:0/4011058821 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a70003230 con 0x7f8a70003df0 2020-11-28T21:53:44.384 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a91281700 1 --2- 172.21.15.162:0/4011058821 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8a8c0994f0 0x7f8a700038b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:53:44.384 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a91a82700 1 --2- 172.21.15.162:0/4011058821 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8a700049c0 0x7f8a70004e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:53:44.384 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a90a80700 1 --2- 172.21.15.162:0/4011058821 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a70003df0 0x7f8a700044d0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 rx=0x7f8a8800c000 tx=0x7f8a88008040).ready entity=mon.1 client_cookie=9dcc0fd60919c4e9 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:44.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a7ffff700 1 -- 172.21.15.162:0/4011058821 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a8800e9d0 con 0x7f8a70003df0 2020-11-28T21:53:44.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a7ffff700 1 -- 172.21.15.162:0/4011058821 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a8800eb30 con 0x7f8a70003df0 2020-11-28T21:53:44.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a7ffff700 1 -- 172.21.15.162:0/4011058821 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a8800ee90 con 0x7f8a70003df0 2020-11-28T21:53:44.386 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a92283700 1 -- 172.21.15.162:0/4011058821 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a70003df0 msgr2=0x7f8a700044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:44.387 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a92283700 1 --2- 172.21.15.162:0/4011058821 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a70003df0 0x7f8a700044d0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 rx=0x7f8a8800c000 tx=0x7f8a88008040).stop 2020-11-28T21:53:44.387 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a91a82700 1 -- 172.21.15.162:0/4011058821 reap_dead start 2020-11-28T21:53:44.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a92283700 1 -- 172.21.15.162:0/4011058821 shutdown_connections 2020-11-28T21:53:44.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.372+0000 7f8a92283700 1 -- 172.21.15.162:0/4011058821 >> 172.21.15.162:0/4011058821 conn(0x7f8a8c05b970 msgr2=0x7f8a8c05bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:44.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.376+0000 7f8a92283700 1 -- 172.21.15.162:0/4011058821 shutdown_connections 2020-11-28T21:53:44.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.376+0000 7f8a92283700 1 -- 172.21.15.162:0/4011058821 wait complete. 2020-11-28T21:53:44.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.376+0000 7f8a92283700 1 Processor -- start 2020-11-28T21:53:44.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.376+0000 7f8a92283700 1 -- start start 2020-11-28T21:53:44.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.376+0000 7f8a92283700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a8c0aacd0 0x7f8a8c0ac010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:44.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.376+0000 7f8a92283700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8a8c0ac500 0x7f8a8c145860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:44.392 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.376+0000 7f8a92283700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8a8c145d50 0x7f8a8c14b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:44.392 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.376+0000 7f8a92283700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8c14e660 con 0x7f8a8c0ac500 2020-11-28T21:53:44.392 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.376+0000 7f8a92283700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8c0ad830 con 0x7f8a8c0aacd0 2020-11-28T21:53:44.392 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.376+0000 7f8a92283700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8c14cbc0 con 0x7f8a8c145d50 2020-11-28T21:53:44.393 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.376+0000 7f8a91281700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a8c0aacd0 0x7f8a8c0ac010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:44.393 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.376+0000 7f8a91281700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a8c0aacd0 0x7f8a8c0ac010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42294/0 (socket says 172.21.15.162:42294) 2020-11-28T21:53:44.393 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.376+0000 7f8a91281700 1 -- 172.21.15.162:0/647343828 learned_addr learned my addr 172.21.15.162:0/647343828 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:44.393 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.376+0000 7f8a90a80700 1 --2- 172.21.15.162:0/647343828 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8a8c0ac500 0x7f8a8c145860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:44.393 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.376+0000 7f8a91a82700 1 --2- 172.21.15.162:0/647343828 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8a8c145d50 0x7f8a8c14b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:44.394 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.376+0000 7f8a91281700 1 -- 172.21.15.162:0/647343828 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8a8c145d50 msgr2=0x7f8a8c14b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:44.394 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.376+0000 7f8a91281700 1 --2- 172.21.15.162:0/647343828 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8a8c145d50 0x7f8a8c14b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:44.394 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.376+0000 7f8a91281700 1 -- 172.21.15.162:0/647343828 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8a8c0ac500 msgr2=0x7f8a8c145860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:44.394 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.376+0000 7f8a91281700 1 --2- 172.21.15.162:0/647343828 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8a8c0ac500 0x7f8a8c145860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:44.395 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.376+0000 7f8a91281700 1 -- 172.21.15.162:0/647343828 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a8c0ab0b0 con 0x7f8a8c0aacd0 2020-11-28T21:53:44.395 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.376+0000 7f8a91281700 1 --2- 172.21.15.162:0/647343828 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a8c0aacd0 0x7f8a8c0ac010 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 rx=0x7f8a8c14d170 tx=0x7f8a80009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:44.395 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.376+0000 7f8a7effd700 1 -- 172.21.15.162:0/647343828 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a8000b5f0 con 0x7f8a8c0aacd0 2020-11-28T21:53:44.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.376+0000 7f8a7effd700 1 -- 172.21.15.162:0/647343828 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a8000b750 con 0x7f8a8c0aacd0 2020-11-28T21:53:44.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.376+0000 7f8a92283700 1 -- 172.21.15.162:0/647343828 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a8c0a83b0 con 0x7f8a8c0aacd0 2020-11-28T21:53:44.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.376+0000 7f8a7effd700 1 -- 172.21.15.162:0/647343828 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a8000bab0 con 0x7f8a8c0aacd0 2020-11-28T21:53:44.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.376+0000 7f8a92283700 1 -- 172.21.15.162:0/647343828 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a8c0aea50 con 0x7f8a8c0aacd0 2020-11-28T21:53:44.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.380+0000 7f8a92283700 1 -- 172.21.15.162:0/647343828 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a8c057050 con 0x7f8a8c0aacd0 2020-11-28T21:53:44.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.380+0000 7f8a7effd700 1 -- 172.21.15.162:0/647343828 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8a8000bc10 con 0x7f8a8c0aacd0 2020-11-28T21:53:44.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.380+0000 7f8a7effd700 1 --2- 172.21.15.162:0/647343828 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8a680553b0 0x7f8a68057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:44.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.380+0000 7f8a7effd700 1 -- 172.21.15.162:0/647343828 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8a8006e2f0 con 0x7f8a8c0aacd0 2020-11-28T21:53:44.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.380+0000 7f8a90a80700 1 --2- 172.21.15.162:0/647343828 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8a680553b0 0x7f8a68057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:44.398 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.380+0000 7f8a90a80700 1 --2- 172.21.15.162:0/647343828 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8a680553b0 0x7f8a68057870 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f8a8800b000 tx=0x7f8a8801a000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:44.398 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.388+0000 7f8a7effd700 1 -- 172.21.15.162:0/647343828 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8a80044480 con 0x7f8a8c0aacd0 2020-11-28T21:53:44.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.528+0000 7f8a92283700 1 -- 172.21.15.162:0/647343828 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a8c068350 con 0x7f8a8c0aacd0 2020-11-28T21:53:44.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.528+0000 7f8a7effd700 1 -- 172.21.15.162:0/647343828 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16137 (secure 0 0 0) 0x7f8a80047aa0 con 0x7f8a8c0aacd0 2020-11-28T21:53:44.534 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:53:44.534 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":88,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:53:37.541898+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:53:34.083725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2020-11-28T21:53:35.380493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:53:44.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.532+0000 7f8a92283700 1 -- 172.21.15.162:0/647343828 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8a680553b0 msgr2=0x7f8a68057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:44.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.532+0000 7f8a92283700 1 --2- 172.21.15.162:0/647343828 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8a680553b0 0x7f8a68057870 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f8a8800b000 tx=0x7f8a8801a000).stop 2020-11-28T21:53:44.537 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.532+0000 7f8a92283700 1 -- 172.21.15.162:0/647343828 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a8c0aacd0 msgr2=0x7f8a8c0ac010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:44.537 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.532+0000 7f8a92283700 1 --2- 172.21.15.162:0/647343828 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a8c0aacd0 0x7f8a8c0ac010 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 rx=0x7f8a8c14d170 tx=0x7f8a80009040).stop 2020-11-28T21:53:44.537 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.532+0000 7f8a92283700 1 -- 172.21.15.162:0/647343828 shutdown_connections 2020-11-28T21:53:44.537 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.532+0000 7f8a92283700 1 --2- 172.21.15.162:0/647343828 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8a680553b0 0x7f8a68057870 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:44.537 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.532+0000 7f8a92283700 1 --2- 172.21.15.162:0/647343828 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8a8c145d50 0x7f8a8c14b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:44.538 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.532+0000 7f8a92283700 1 --2- 172.21.15.162:0/647343828 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a8c0aacd0 0x7f8a8c0ac010 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:44.538 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.532+0000 7f8a92283700 1 --2- 172.21.15.162:0/647343828 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8a8c0ac500 0x7f8a8c145860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:44.538 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.532+0000 7f8a92283700 1 -- 172.21.15.162:0/647343828 >> 172.21.15.162:0/647343828 conn(0x7f8a8c05b970 msgr2=0x7f8a8c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:44.538 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.532+0000 7f8a92283700 1 -- 172.21.15.162:0/647343828 shutdown_connections 2020-11-28T21:53:44.538 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.532+0000 7f8a92283700 1 -- 172.21.15.162:0/647343828 wait complete. 2020-11-28T21:53:44.547 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-28T21:53:34.083725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2020-11-28T21:53:35.380493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-28T21:53:44.549 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:53:44.663 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.660+0000 7f66114ea700 1 Processor -- start 2020-11-28T21:53:44.663 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.660+0000 7f66114ea700 1 -- start start 2020-11-28T21:53:44.665 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.660+0000 7f66114ea700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f660c0b2290 0x7f660c060a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:44.665 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.660+0000 7f66114ea700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f660c1471f0 con 0x7f660c0b1eb0 2020-11-28T21:53:44.665 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.660+0000 7f66114ea700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f660c060090 con 0x7f660c0af6a0 2020-11-28T21:53:44.666 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.660+0000 7f66114ea700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f660c060330 con 0x7f660c0b2290 2020-11-28T21:53:44.666 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.660+0000 7f660b7fe700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f660c0af6a0 0x7f660c0afdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:49148/0 (socket says 172.21.15.162:49148) 2020-11-28T21:53:44.666 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.660+0000 7f660b7fe700 1 -- 172.21.15.162:0/2647761099 learned_addr learned my addr 172.21.15.162:0/2647761099 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:44.666 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.660+0000 7f6610ce9700 1 --2- 172.21.15.162:0/2647761099 >> v2:172.21.15.118:3300/0 conn(0x7f660c0b2290 0x7f660c060a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:44.666 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.660+0000 7f660affd700 1 -- 172.21.15.162:0/2647761099 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3559255988 0 0) 0x7f660c060090 con 0x7f660c0af6a0 2020-11-28T21:53:44.667 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.660+0000 7f660affd700 1 -- 172.21.15.162:0/2647761099 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65f8003230 con 0x7f660c0af6a0 2020-11-28T21:53:44.667 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.660+0000 7f6610ce9700 1 -- 172.21.15.162:0/2647761099 >> v1:172.21.15.162:6789/0 conn(0x7f660c0af6a0 legacy=0x7f660c0afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:44.667 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.660+0000 7f660affd700 1 -- 172.21.15.162:0/2647761099 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3738693734 0 0) 0x7f65f8003230 con 0x7f660c0af6a0 2020-11-28T21:53:44.667 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.660+0000 7f6610ce9700 1 -- 172.21.15.162:0/2647761099 >> v1:172.21.15.118:6789/0 conn(0x7f660c0b1eb0 legacy=0x7f660c144350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:44.668 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.660+0000 7f6610ce9700 1 -- 172.21.15.162:0/2647761099 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f660c0605d0 con 0x7f660c0b2290 2020-11-28T21:53:44.668 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.660+0000 7f6610ce9700 1 --2- 172.21.15.162:0/2647761099 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f660c0b2290 0x7f660c060a70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f65fc0010d0 tx=0x7f66040091a0).ready entity=mon.0 client_cookie=779f1181f9f25886 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:44.671 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.660+0000 7f660affd700 1 -- 172.21.15.162:0/2647761099 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66000016d0 con 0x7f660c0b2290 2020-11-28T21:53:44.671 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.660+0000 7f660affd700 1 -- 172.21.15.162:0/2647761099 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f660400b860 con 0x7f660c0b2290 2020-11-28T21:53:44.671 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.660+0000 7f660affd700 1 -- 172.21.15.162:0/2647761099 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f660400b9d0 con 0x7f660c0b2290 2020-11-28T21:53:44.671 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.660+0000 7f66114ea700 1 -- 172.21.15.162:0/2647761099 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f660c0b2290 msgr2=0x7f660c060a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:44.671 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.660+0000 7f66114ea700 1 --2- 172.21.15.162:0/2647761099 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f660c0b2290 0x7f660c060a70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f65fc0010d0 tx=0x7f66040091a0).stop 2020-11-28T21:53:44.672 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.664+0000 7f66114ea700 1 -- 172.21.15.162:0/2647761099 shutdown_connections 2020-11-28T21:53:44.672 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.664+0000 7f66114ea700 1 --2- 172.21.15.162:0/2647761099 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f660c0b2290 0x7f660c060a70 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:44.672 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.664+0000 7f66114ea700 1 -- 172.21.15.162:0/2647761099 >> 172.21.15.162:0/2647761099 conn(0x7f660c0b57d0 msgr2=0x7f660c0b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:44.673 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.664+0000 7f66114ea700 1 -- 172.21.15.162:0/2647761099 shutdown_connections 2020-11-28T21:53:44.673 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.664+0000 7f66114ea700 1 -- 172.21.15.162:0/2647761099 wait complete. 2020-11-28T21:53:44.673 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.664+0000 7f66114ea700 1 Processor -- start 2020-11-28T21:53:44.673 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.664+0000 7f66114ea700 1 -- start start 2020-11-28T21:53:44.674 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.664+0000 7f66114ea700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f660c0af6a0 0x7f660c063470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:44.674 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.664+0000 7f66114ea700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f660c0b1eb0 0x7f660c0aa810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:44.674 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.664+0000 7f66114ea700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f660c0b2290 0x7f660c149970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:44.674 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.664+0000 7f66114ea700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660c065040 con 0x7f660c0af6a0 2020-11-28T21:53:44.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.664+0000 7f66114ea700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660c0652e0 con 0x7f660c0b2290 2020-11-28T21:53:44.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.664+0000 7f66114ea700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f660c065580 con 0x7f660c0b1eb0 2020-11-28T21:53:44.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.664+0000 7f6610ce9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f660c0b2290 0x7f660c149970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:44.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.664+0000 7f6610ce9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f660c0b2290 0x7f660c149970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42312/0 (socket says 172.21.15.162:42312) 2020-11-28T21:53:44.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.664+0000 7f6610ce9700 1 -- 172.21.15.162:0/4281463331 learned_addr learned my addr 172.21.15.162:0/4281463331 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:44.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.664+0000 7f660bfff700 1 --2- 172.21.15.162:0/4281463331 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f660c0af6a0 0x7f660c063470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:44.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.664+0000 7f660b7fe700 1 --2- 172.21.15.162:0/4281463331 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f660c0b1eb0 0x7f660c0aa810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:44.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.664+0000 7f6610ce9700 1 -- 172.21.15.162:0/4281463331 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f660c0b1eb0 msgr2=0x7f660c0aa810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:44.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.664+0000 7f6610ce9700 1 --2- 172.21.15.162:0/4281463331 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f660c0b1eb0 0x7f660c0aa810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:44.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.664+0000 7f6610ce9700 1 -- 172.21.15.162:0/4281463331 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f660c0af6a0 msgr2=0x7f660c063470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:44.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.664+0000 7f6610ce9700 1 --2- 172.21.15.162:0/4281463331 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f660c0af6a0 0x7f660c063470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:44.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.664+0000 7f6610ce9700 1 -- 172.21.15.162:0/4281463331 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f660c065820 con 0x7f660c0b2290 2020-11-28T21:53:44.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.664+0000 7f6610ce9700 1 --2- 172.21.15.162:0/4281463331 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f660c0b2290 0x7f660c149970 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 rx=0x7f65fc0010d0 tx=0x7f66040091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:44.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.664+0000 7f66097fa700 1 -- 172.21.15.162:0/4281463331 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6604011c70 con 0x7f660c0b2290 2020-11-28T21:53:44.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.664+0000 7f66097fa700 1 -- 172.21.15.162:0/4281463331 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6604011dd0 con 0x7f660c0b2290 2020-11-28T21:53:44.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.664+0000 7f66097fa700 1 -- 172.21.15.162:0/4281463331 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f660401d640 con 0x7f660c0b2290 2020-11-28T21:53:44.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.664+0000 7f66114ea700 1 -- 172.21.15.162:0/4281463331 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f660c0ab6f0 con 0x7f660c0b2290 2020-11-28T21:53:44.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.664+0000 7f66114ea700 1 -- 172.21.15.162:0/4281463331 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f660c0aaea0 con 0x7f660c0b2290 2020-11-28T21:53:44.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.668+0000 7f66114ea700 1 -- 172.21.15.162:0/4281463331 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f660c0659b0 con 0x7f660c0b2290 2020-11-28T21:53:44.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.668+0000 7f66097fa700 1 -- 172.21.15.162:0/4281463331 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f660400f070 con 0x7f660c0b2290 2020-11-28T21:53:44.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.668+0000 7f66097fa700 1 --2- 172.21.15.162:0/4281463331 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f65ec055470 0x7f65ec057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:44.680 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.668+0000 7f660bfff700 1 --2- 172.21.15.162:0/4281463331 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f65ec055470 0x7f65ec057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:44.680 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.668+0000 7f66097fa700 1 -- 172.21.15.162:0/4281463331 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f660406fb40 con 0x7f660c0b2290 2020-11-28T21:53:44.680 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.668+0000 7f660bfff700 1 --2- 172.21.15.162:0/4281463331 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f65ec055470 0x7f65ec057930 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7f65fc00d000 tx=0x7f65fc007b40).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:44.680 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.676+0000 7f66097fa700 1 -- 172.21.15.162:0/4281463331 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6604045e80 con 0x7f660c0b2290 2020-11-28T21:53:44.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.812+0000 7f66114ea700 1 -- 172.21.15.162:0/4281463331 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f660c057050 con 0x7f660c0b2290 2020-11-28T21:53:44.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.812+0000 7f66097fa700 1 -- 172.21.15.162:0/4281463331 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v88) v1 ==== 110+0+10 (secure 0 0 0) 0x7f6604049460 con 0x7f660c0b2290 2020-11-28T21:53:44.818 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:53:44.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.816+0000 7f66114ea700 1 -- 172.21.15.162:0/4281463331 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f65ec055470 msgr2=0x7f65ec057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:44.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.816+0000 7f66114ea700 1 --2- 172.21.15.162:0/4281463331 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f65ec055470 0x7f65ec057930 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7f65fc00d000 tx=0x7f65fc007b40).stop 2020-11-28T21:53:44.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.816+0000 7f66114ea700 1 -- 172.21.15.162:0/4281463331 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f660c0b2290 msgr2=0x7f660c149970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:44.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.816+0000 7f66114ea700 1 --2- 172.21.15.162:0/4281463331 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f660c0b2290 0x7f660c149970 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 rx=0x7f65fc0010d0 tx=0x7f66040091a0).stop 2020-11-28T21:53:44.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.816+0000 7f66114ea700 1 -- 172.21.15.162:0/4281463331 shutdown_connections 2020-11-28T21:53:44.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.816+0000 7f66114ea700 1 --2- 172.21.15.162:0/4281463331 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f65ec055470 0x7f65ec057930 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:44.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.816+0000 7f66114ea700 1 --2- 172.21.15.162:0/4281463331 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f660c0b2290 0x7f660c149970 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:44.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.816+0000 7f66114ea700 1 --2- 172.21.15.162:0/4281463331 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f660c0af6a0 0x7f660c063470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:44.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.816+0000 7f66114ea700 1 --2- 172.21.15.162:0/4281463331 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f660c0b1eb0 0x7f660c0aa810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:44.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.816+0000 7f66114ea700 1 -- 172.21.15.162:0/4281463331 >> 172.21.15.162:0/4281463331 conn(0x7f660c0b57d0 msgr2=0x7f660c143c40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:44.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.816+0000 7f66114ea700 1 -- 172.21.15.162:0/4281463331 shutdown_connections 2020-11-28T21:53:44.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.816+0000 7f66114ea700 1 -- 172.21.15.162:0/4281463331 wait complete. 2020-11-28T21:53:44.831 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:53:44.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.940+0000 7ff6e5ebb700 1 Processor -- start 2020-11-28T21:53:44.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.940+0000 7ff6e5ebb700 1 -- start start 2020-11-28T21:53:44.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.940+0000 7ff6e5ebb700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6e00af6a0 0x7ff6e00b2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:44.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.940+0000 7ff6e5ebb700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff6e0060a70 0x7ff6e00afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:44.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.940+0000 7ff6e5ebb700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e0060f00 con 0x7ff6e00b1eb0 2020-11-28T21:53:44.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.940+0000 7ff6e5ebb700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6e01471f0 con 0x7ff6e0060a70 2020-11-28T21:53:44.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.940+0000 7ff6e5ebb700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e0060090 con 0x7ff6e00af6a0 2020-11-28T21:53:44.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.940+0000 7ff6e4eb9700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7ff6e00b1eb0 0x7ff6e0144350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:49160/0 (socket says 172.21.15.162:49160) 2020-11-28T21:53:44.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.940+0000 7ff6e4eb9700 1 -- 172.21.15.162:0/4113779198 learned_addr learned my addr 172.21.15.162:0/4113779198 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:44.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.940+0000 7ff6e56ba700 1 --2- 172.21.15.162:0/4113779198 >> v2:172.21.15.118:3300/0 conn(0x7ff6e0060a70 0x7ff6e00afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:44.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.940+0000 7ff6dffff700 1 --2- 172.21.15.162:0/4113779198 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6e00af6a0 0x7ff6e00b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:44.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.940+0000 7ff6df7fe700 1 -- 172.21.15.162:0/4113779198 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1227152377 0 0) 0x7ff6e0060f00 con 0x7ff6e00b1eb0 2020-11-28T21:53:44.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.940+0000 7ff6e56ba700 1 -- 172.21.15.162:0/4113779198 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6e00af6a0 msgr2=0x7ff6e00b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:44.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.940+0000 7ff6e56ba700 1 --2- 172.21.15.162:0/4113779198 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6e00af6a0 0x7ff6e00b2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:44.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.940+0000 7ff6e56ba700 1 -- 172.21.15.162:0/4113779198 >> v1:172.21.15.162:6789/0 conn(0x7ff6e00b1eb0 legacy=0x7ff6e0144350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:44.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.940+0000 7ff6e56ba700 1 -- 172.21.15.162:0/4113779198 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e0060330 con 0x7ff6e0060a70 2020-11-28T21:53:44.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.940+0000 7ff6e56ba700 1 --2- 172.21.15.162:0/4113779198 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff6e0060a70 0x7ff6e00afdc0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7ff6d0002f30 tx=0x7ff6d0006dc0).ready entity=mon.0 client_cookie=ed47917cb6268903 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:44.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.944+0000 7ff6df7fe700 1 -- 172.21.15.162:0/4113779198 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6d0009760 con 0x7ff6e0060a70 2020-11-28T21:53:44.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.944+0000 7ff6df7fe700 1 -- 172.21.15.162:0/4113779198 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d00098c0 con 0x7ff6e0060a70 2020-11-28T21:53:44.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.944+0000 7ff6e5ebb700 1 -- 172.21.15.162:0/4113779198 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff6e0060a70 msgr2=0x7ff6e00afdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:44.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.944+0000 7ff6e5ebb700 1 --2- 172.21.15.162:0/4113779198 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff6e0060a70 0x7ff6e00afdc0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7ff6d0002f30 tx=0x7ff6d0006dc0).stop 2020-11-28T21:53:44.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.944+0000 7ff6e5ebb700 1 -- 172.21.15.162:0/4113779198 shutdown_connections 2020-11-28T21:53:44.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.944+0000 7ff6e5ebb700 1 --2- 172.21.15.162:0/4113779198 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff6e0060a70 0x7ff6e00afdc0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:44.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.944+0000 7ff6e5ebb700 1 --2- 172.21.15.162:0/4113779198 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6e00af6a0 0x7ff6e00b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:44.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.944+0000 7ff6e5ebb700 1 -- 172.21.15.162:0/4113779198 >> 172.21.15.162:0/4113779198 conn(0x7ff6e00b57d0 msgr2=0x7ff6e00b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:44.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.944+0000 7ff6e5ebb700 1 -- 172.21.15.162:0/4113779198 shutdown_connections 2020-11-28T21:53:44.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.944+0000 7ff6e5ebb700 1 -- 172.21.15.162:0/4113779198 wait complete. 2020-11-28T21:53:44.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.944+0000 7ff6e5ebb700 1 Processor -- start 2020-11-28T21:53:44.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.944+0000 7ff6e5ebb700 1 -- start start 2020-11-28T21:53:44.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.944+0000 7ff6e5ebb700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6e0060a70 0x7ff6e00a8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:44.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.944+0000 7ff6e5ebb700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff6e00af6a0 0x7ff6e01436a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:44.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.944+0000 7ff6e5ebb700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6e00b1eb0 0x7ff6e0143b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:44.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.944+0000 7ff6e5ebb700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e0063bb0 con 0x7ff6e00af6a0 2020-11-28T21:53:44.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.944+0000 7ff6e5ebb700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e00abbd0 con 0x7ff6e00b1eb0 2020-11-28T21:53:44.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.944+0000 7ff6e5ebb700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e00abe00 con 0x7ff6e0060a70 2020-11-28T21:53:44.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.944+0000 7ff6e56ba700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6e00b1eb0 0x7ff6e0143b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:44.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.944+0000 7ff6e56ba700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6e00b1eb0 0x7ff6e0143b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42326/0 (socket says 172.21.15.162:42326) 2020-11-28T21:53:44.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.944+0000 7ff6e56ba700 1 -- 172.21.15.162:0/2265118808 learned_addr learned my addr 172.21.15.162:0/2265118808 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:44.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.944+0000 7ff6dffff700 1 --2- 172.21.15.162:0/2265118808 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff6e00af6a0 0x7ff6e01436a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:44.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.944+0000 7ff6e4eb9700 1 --2- 172.21.15.162:0/2265118808 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6e0060a70 0x7ff6e00a8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:44.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.948+0000 7ff6e56ba700 1 -- 172.21.15.162:0/2265118808 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6e0060a70 msgr2=0x7ff6e00a8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:44.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.948+0000 7ff6e56ba700 1 --2- 172.21.15.162:0/2265118808 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6e0060a70 0x7ff6e00a8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:44.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.948+0000 7ff6e56ba700 1 -- 172.21.15.162:0/2265118808 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff6e00af6a0 msgr2=0x7ff6e01436a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:44.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.948+0000 7ff6e56ba700 1 --2- 172.21.15.162:0/2265118808 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff6e00af6a0 0x7ff6e01436a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:44.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.948+0000 7ff6e56ba700 1 -- 172.21.15.162:0/2265118808 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e00631d0 con 0x7ff6e00b1eb0 2020-11-28T21:53:44.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.948+0000 7ff6e56ba700 1 --2- 172.21.15.162:0/2265118808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6e00b1eb0 0x7ff6e0143b90 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 rx=0x7ff6d0002f30 tx=0x7ff6d0015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:44.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.948+0000 7ff6ddffb700 1 -- 172.21.15.162:0/2265118808 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6d0011a90 con 0x7ff6e00b1eb0 2020-11-28T21:53:44.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.948+0000 7ff6ddffb700 1 -- 172.21.15.162:0/2265118808 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d0011bf0 con 0x7ff6e00b1eb0 2020-11-28T21:53:44.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.948+0000 7ff6e5ebb700 1 -- 172.21.15.162:0/2265118808 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6e0063470 con 0x7ff6e00b1eb0 2020-11-28T21:53:44.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.948+0000 7ff6ddffb700 1 -- 172.21.15.162:0/2265118808 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6d001d440 con 0x7ff6e00b1eb0 2020-11-28T21:53:44.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.948+0000 7ff6e5ebb700 1 -- 172.21.15.162:0/2265118808 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6e0066ad0 con 0x7ff6e00b1eb0 2020-11-28T21:53:44.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.948+0000 7ff6e5ebb700 1 -- 172.21.15.162:0/2265118808 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6e0057050 con 0x7ff6e00b1eb0 2020-11-28T21:53:44.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.948+0000 7ff6ddffb700 1 -- 172.21.15.162:0/2265118808 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff6d0024030 con 0x7ff6e00b1eb0 2020-11-28T21:53:44.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.952+0000 7ff6ddffb700 1 --2- 172.21.15.162:0/2265118808 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff6c40553b0 0x7ff6c4057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:44.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.952+0000 7ff6ddffb700 1 -- 172.21.15.162:0/2265118808 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff6d006ee40 con 0x7ff6e00b1eb0 2020-11-28T21:53:44.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.952+0000 7ff6e4eb9700 1 --2- 172.21.15.162:0/2265118808 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff6c40553b0 0x7ff6c4057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:44.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.952+0000 7ff6e4eb9700 1 --2- 172.21.15.162:0/2265118808 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff6c40553b0 0x7ff6c4057870 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7ff6c800d000 tx=0x7ff6c8007bd0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:44.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:44.956+0000 7ff6ddffb700 1 -- 172.21.15.162:0/2265118808 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff6d0045140 con 0x7ff6e00b1eb0 2020-11-28T21:53:45.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.100+0000 7ff6e5ebb700 1 -- 172.21.15.162:0/2265118808 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff6e00ab970 con 0x7ff6e00b1eb0 2020-11-28T21:53:45.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.100+0000 7ff6ddffb700 1 -- 172.21.15.162:0/2265118808 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v88) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff6d0022070 con 0x7ff6e00b1eb0 2020-11-28T21:53:45.104 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:53:45.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.100+0000 7ff6e5ebb700 1 -- 172.21.15.162:0/2265118808 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff6c40553b0 msgr2=0x7ff6c4057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:45.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.100+0000 7ff6e5ebb700 1 --2- 172.21.15.162:0/2265118808 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff6c40553b0 0x7ff6c4057870 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7ff6c800d000 tx=0x7ff6c8007bd0).stop 2020-11-28T21:53:45.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.100+0000 7ff6e5ebb700 1 -- 172.21.15.162:0/2265118808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6e00b1eb0 msgr2=0x7ff6e0143b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:45.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.100+0000 7ff6e5ebb700 1 --2- 172.21.15.162:0/2265118808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6e00b1eb0 0x7ff6e0143b90 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 rx=0x7ff6d0002f30 tx=0x7ff6d0015000).stop 2020-11-28T21:53:45.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.100+0000 7ff6e5ebb700 1 -- 172.21.15.162:0/2265118808 shutdown_connections 2020-11-28T21:53:45.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.100+0000 7ff6e5ebb700 1 --2- 172.21.15.162:0/2265118808 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff6c40553b0 0x7ff6c4057870 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:45.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.100+0000 7ff6e5ebb700 1 --2- 172.21.15.162:0/2265118808 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6e0060a70 0x7ff6e00a8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:45.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.100+0000 7ff6e5ebb700 1 --2- 172.21.15.162:0/2265118808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6e00b1eb0 0x7ff6e0143b90 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:45.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.100+0000 7ff6e5ebb700 1 --2- 172.21.15.162:0/2265118808 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff6e00af6a0 0x7ff6e01436a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:45.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.100+0000 7ff6e5ebb700 1 -- 172.21.15.162:0/2265118808 >> 172.21.15.162:0/2265118808 conn(0x7ff6e00b57d0 msgr2=0x7ff6e0144d10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:45.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.100+0000 7ff6e5ebb700 1 -- 172.21.15.162:0/2265118808 shutdown_connections 2020-11-28T21:53:45.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.100+0000 7ff6e5ebb700 1 -- 172.21.15.162:0/2265118808 wait complete. 2020-11-28T21:53:45.117 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-28T21:53:45.236 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.232+0000 7f8f5ffff700 1 Processor -- start 2020-11-28T21:53:45.236 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.232+0000 7f8f5ffff700 1 -- start start 2020-11-28T21:53:45.237 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.232+0000 7f8f5ffff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8f600b3950 0x7f8f600b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:45.237 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.232+0000 7f8f5ffff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f8f600b18c0 0x7f8f6013dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:45.237 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.232+0000 7f8f5ffff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f600b1f50 con 0x7f8f600b44e0 2020-11-28T21:53:45.238 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.232+0000 7f8f5ffff700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f600aab50 con 0x7f8f600b18c0 2020-11-28T21:53:45.238 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.232+0000 7f8f5ffff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f60143e90 con 0x7f8f600b3950 2020-11-28T21:53:45.239 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.232+0000 7f8f5e7fc700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f8f600b18c0 0x7f8f6013dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:45.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.232+0000 7f8f5e7fc700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f8f600b18c0 0x7f8f6013dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.162:49320/0 (socket says 172.21.15.162:49320) 2020-11-28T21:53:45.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.232+0000 7f8f5e7fc700 1 -- 172.21.15.162:0/1759449501 learned_addr learned my addr 172.21.15.162:0/1759449501 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:45.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.232+0000 7f8f5effd700 1 --2- 172.21.15.162:0/1759449501 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8f600b3950 0x7f8f600b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:45.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.232+0000 7f8f5e7fc700 1 -- 172.21.15.162:0/1759449501 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8f600b3950 msgr2=0x7f8f600b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:45.241 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.232+0000 7f8f5e7fc700 1 --2- 172.21.15.162:0/1759449501 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8f600b3950 0x7f8f600b3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:45.241 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.232+0000 7f8f5e7fc700 1 -- 172.21.15.162:0/1759449501 >> v1:172.21.15.118:6789/0 conn(0x7f8f600b44e0 legacy=0x7f8f600b2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:45.241 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.232+0000 7f8f5e7fc700 1 -- 172.21.15.162:0/1759449501 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f600aa8b0 con 0x7f8f600b18c0 2020-11-28T21:53:45.241 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.236+0000 7f8f5e7fc700 1 --2- 172.21.15.162:0/1759449501 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8f600b18c0 0x7f8f6013dd60 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f8f48002f30 tx=0x7f8f48006e80).ready entity=mon.0 client_cookie=638d972abb4a2025 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:45.241 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.236+0000 7f8f5dffb700 1 -- 172.21.15.162:0/1759449501 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f480097f0 con 0x7f8f600b18c0 2020-11-28T21:53:45.242 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.236+0000 7f8f5dffb700 1 -- 172.21.15.162:0/1759449501 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f48009950 con 0x7f8f600b18c0 2020-11-28T21:53:45.242 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.236+0000 7f8f5ffff700 1 -- 172.21.15.162:0/1759449501 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8f600b18c0 msgr2=0x7f8f6013dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:45.242 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.236+0000 7f8f5dffb700 1 -- 172.21.15.162:0/1759449501 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f48009cb0 con 0x7f8f600b18c0 2020-11-28T21:53:45.242 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.236+0000 7f8f5ffff700 1 --2- 172.21.15.162:0/1759449501 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8f600b18c0 0x7f8f6013dd60 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f8f48002f30 tx=0x7f8f48006e80).stop 2020-11-28T21:53:45.242 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.236+0000 7f8f5ffff700 1 -- 172.21.15.162:0/1759449501 shutdown_connections 2020-11-28T21:53:45.243 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.236+0000 7f8f5ffff700 1 --2- 172.21.15.162:0/1759449501 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8f600b3950 0x7f8f600b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:45.243 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.236+0000 7f8f5ffff700 1 --2- 172.21.15.162:0/1759449501 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8f600b18c0 0x7f8f6013dd60 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:45.243 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.236+0000 7f8f5ffff700 1 -- 172.21.15.162:0/1759449501 >> 172.21.15.162:0/1759449501 conn(0x7f8f60062630 msgr2=0x7f8f60062a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:45.243 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.236+0000 7f8f5ffff700 1 -- 172.21.15.162:0/1759449501 shutdown_connections 2020-11-28T21:53:45.243 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.236+0000 7f8f5ffff700 1 -- 172.21.15.162:0/1759449501 wait complete. 2020-11-28T21:53:45.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.236+0000 7f8f5ffff700 1 Processor -- start 2020-11-28T21:53:45.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.236+0000 7f8f5ffff700 1 -- start start 2020-11-28T21:53:45.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.236+0000 7f8f5ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f600b18c0 0x7f8f600b2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:45.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.236+0000 7f8f5ffff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8f600b3950 0x7f8f600af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:45.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.236+0000 7f8f5ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8f600b44e0 0x7f8f600afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:45.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.236+0000 7f8f5ffff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f600aa280 con 0x7f8f600b44e0 2020-11-28T21:53:45.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.236+0000 7f8f5ffff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f600a9aa0 con 0x7f8f600b18c0 2020-11-28T21:53:45.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.236+0000 7f8f5ffff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f600a9d40 con 0x7f8f600b3950 2020-11-28T21:53:45.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.236+0000 7f8f5effd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f600b18c0 0x7f8f600b2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:45.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.236+0000 7f8f5effd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f600b18c0 0x7f8f600b2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42338/0 (socket says 172.21.15.162:42338) 2020-11-28T21:53:45.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.236+0000 7f8f5effd700 1 -- 172.21.15.162:0/1144484700 learned_addr learned my addr 172.21.15.162:0/1144484700 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:45.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.236+0000 7f8f5f7fe700 1 --2- 172.21.15.162:0/1144484700 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8f600b44e0 0x7f8f600afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:45.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.236+0000 7f8f5e7fc700 1 --2- 172.21.15.162:0/1144484700 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8f600b3950 0x7f8f600af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:45.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.240+0000 7f8f5effd700 1 -- 172.21.15.162:0/1144484700 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8f600b3950 msgr2=0x7f8f600af8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:45.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.240+0000 7f8f5effd700 1 --2- 172.21.15.162:0/1144484700 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8f600b3950 0x7f8f600af8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:45.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.240+0000 7f8f5effd700 1 -- 172.21.15.162:0/1144484700 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8f600b44e0 msgr2=0x7f8f600afdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:45.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.240+0000 7f8f5effd700 1 --2- 172.21.15.162:0/1144484700 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8f600b44e0 0x7f8f600afdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:45.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.240+0000 7f8f5effd700 1 -- 172.21.15.162:0/1144484700 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f600a9fe0 con 0x7f8f600b18c0 2020-11-28T21:53:45.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.240+0000 7f8f5effd700 1 --2- 172.21.15.162:0/1144484700 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f600b18c0 0x7f8f600b2e70 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 rx=0x7f8f600af5e0 tx=0x7f8f50008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:45.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.240+0000 7f8f57fff700 1 -- 172.21.15.162:0/1144484700 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f5000c820 con 0x7f8f600b18c0 2020-11-28T21:53:45.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.240+0000 7f8f57fff700 1 -- 172.21.15.162:0/1144484700 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f5000c980 con 0x7f8f600b18c0 2020-11-28T21:53:45.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.240+0000 7f8f5ffff700 1 -- 172.21.15.162:0/1144484700 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f600ae7a0 con 0x7f8f600b18c0 2020-11-28T21:53:45.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.240+0000 7f8f57fff700 1 -- 172.21.15.162:0/1144484700 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f5000cce0 con 0x7f8f600b18c0 2020-11-28T21:53:45.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.240+0000 7f8f5ffff700 1 -- 172.21.15.162:0/1144484700 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f600b04c0 con 0x7f8f600b18c0 2020-11-28T21:53:45.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.240+0000 7f8f57fff700 1 -- 172.21.15.162:0/1144484700 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8f50022070 con 0x7f8f600b18c0 2020-11-28T21:53:45.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.240+0000 7f8f5ffff700 1 -- 172.21.15.162:0/1144484700 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f60057050 con 0x7f8f600b18c0 2020-11-28T21:53:45.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.240+0000 7f8f57fff700 1 --2- 172.21.15.162:0/1144484700 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8f44055330 0x7f8f440577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:45.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.240+0000 7f8f57fff700 1 -- 172.21.15.162:0/1144484700 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8f5006e250 con 0x7f8f600b18c0 2020-11-28T21:53:45.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.240+0000 7f8f5e7fc700 1 --2- 172.21.15.162:0/1144484700 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8f44055330 0x7f8f440577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:45.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.240+0000 7f8f5e7fc700 1 --2- 172.21.15.162:0/1144484700 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8f44055330 0x7f8f440577f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f8f4801c000 tx=0x7f8f48015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:45.253 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.248+0000 7f8f57fff700 1 -- 172.21.15.162:0/1144484700 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8f50044550 con 0x7f8f600b18c0 2020-11-28T21:53:45.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.392+0000 7f8f5ffff700 1 -- 172.21.15.162:0/1144484700 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8f6005d250 con 0x7f8f600b18c0 2020-11-28T21:53:45.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.392+0000 7f8f57fff700 1 -- 172.21.15.162:0/1144484700 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+10 (secure 0 0 0) 0x7f8f50047b70 con 0x7f8f600b18c0 2020-11-28T21:53:45.398 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:53:45.399 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.396+0000 7f8f5ffff700 1 -- 172.21.15.162:0/1144484700 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8f44055330 msgr2=0x7f8f440577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:45.399 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.396+0000 7f8f5ffff700 1 --2- 172.21.15.162:0/1144484700 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8f44055330 0x7f8f440577f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f8f4801c000 tx=0x7f8f48015000).stop 2020-11-28T21:53:45.400 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.396+0000 7f8f5ffff700 1 -- 172.21.15.162:0/1144484700 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f600b18c0 msgr2=0x7f8f600b2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:45.400 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.396+0000 7f8f5ffff700 1 --2- 172.21.15.162:0/1144484700 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f600b18c0 0x7f8f600b2e70 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 rx=0x7f8f600af5e0 tx=0x7f8f50008040).stop 2020-11-28T21:53:45.400 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.396+0000 7f8f5ffff700 1 -- 172.21.15.162:0/1144484700 shutdown_connections 2020-11-28T21:53:45.400 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.396+0000 7f8f5ffff700 1 --2- 172.21.15.162:0/1144484700 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8f44055330 0x7f8f440577f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:45.401 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.396+0000 7f8f5ffff700 1 --2- 172.21.15.162:0/1144484700 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8f600b44e0 0x7f8f600afdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:45.401 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.396+0000 7f8f5ffff700 1 --2- 172.21.15.162:0/1144484700 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f600b18c0 0x7f8f600b2e70 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:45.401 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.396+0000 7f8f5ffff700 1 --2- 172.21.15.162:0/1144484700 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8f600b3950 0x7f8f600af8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:45.401 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.396+0000 7f8f5ffff700 1 -- 172.21.15.162:0/1144484700 >> 172.21.15.162:0/1144484700 conn(0x7f8f60062630 msgr2=0x7f8f600b4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:45.402 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.396+0000 7f8f5ffff700 1 -- 172.21.15.162:0/1144484700 shutdown_connections 2020-11-28T21:53:45.402 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.396+0000 7f8f5ffff700 1 -- 172.21.15.162:0/1144484700 wait complete. 2020-11-28T21:53:45.412 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-28T21:53:45.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.524+0000 7f21329ab700 1 Processor -- start 2020-11-28T21:53:45.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.524+0000 7f21329ab700 1 -- start start 2020-11-28T21:53:45.531 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.524+0000 7f21329ab700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f212c0b1eb0 0x7f212c0b2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:45.531 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.524+0000 7f21329ab700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f212c0af6a0 0x7f212c060a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:45.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.524+0000 7f21329ab700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f212c1471f0 con 0x7f212c144350 2020-11-28T21:53:45.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.524+0000 7f21329ab700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f212c060090 con 0x7f212c0b1eb0 2020-11-28T21:53:45.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.524+0000 7f21329ab700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f212c060330 con 0x7f212c0af6a0 2020-11-28T21:53:45.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.524+0000 7f21311a8700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f212c0af6a0 0x7f212c060a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:45.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.524+0000 7f21311a8700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f212c0af6a0 0x7f212c060a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42348/0 (socket says 172.21.15.162:42348) 2020-11-28T21:53:45.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.524+0000 7f21311a8700 1 -- 172.21.15.162:0/444807240 learned_addr learned my addr 172.21.15.162:0/444807240 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:45.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.524+0000 7f21319a9700 1 --2- 172.21.15.162:0/444807240 >> v2:172.21.15.118:3300/0 conn(0x7f212c0b1eb0 0x7f212c0b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:45.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.524+0000 7f21311a8700 1 -- 172.21.15.162:0/444807240 >> v2:172.21.15.118:3300/0 conn(0x7f212c0b1eb0 msgr2=0x7f212c0b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:45.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.524+0000 7f21311a8700 1 --2- 172.21.15.162:0/444807240 >> v2:172.21.15.118:3300/0 conn(0x7f212c0b1eb0 0x7f212c0b2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:45.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.524+0000 7f21311a8700 1 -- 172.21.15.162:0/444807240 >> v1:172.21.15.118:6789/0 conn(0x7f212c144350 legacy=0x7f212c14a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:45.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.524+0000 7f21311a8700 1 -- 172.21.15.162:0/444807240 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f212c0605d0 con 0x7f212c0af6a0 2020-11-28T21:53:45.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.528+0000 7f21311a8700 1 --2- 172.21.15.162:0/444807240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f212c0af6a0 0x7f212c060a70 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 rx=0x7f21280032b0 tx=0x7f2128009270).ready entity=mon.1 client_cookie=6d033360263b840 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:45.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.528+0000 7f21309a7700 1 -- 172.21.15.162:0/444807240 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f212800bc00 con 0x7f212c0af6a0 2020-11-28T21:53:45.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.528+0000 7f21309a7700 1 -- 172.21.15.162:0/444807240 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f212800bd60 con 0x7f212c0af6a0 2020-11-28T21:53:45.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.528+0000 7f21309a7700 1 -- 172.21.15.162:0/444807240 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2128011400 con 0x7f212c0af6a0 2020-11-28T21:53:45.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.528+0000 7f21329ab700 1 -- 172.21.15.162:0/444807240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f212c0af6a0 msgr2=0x7f212c060a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:45.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.528+0000 7f21329ab700 1 --2- 172.21.15.162:0/444807240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f212c0af6a0 0x7f212c060a70 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 rx=0x7f21280032b0 tx=0x7f2128009270).stop 2020-11-28T21:53:45.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.528+0000 7f21329ab700 1 -- 172.21.15.162:0/444807240 shutdown_connections 2020-11-28T21:53:45.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.528+0000 7f21329ab700 1 --2- 172.21.15.162:0/444807240 >> v2:172.21.15.118:3300/0 conn(0x7f212c0b1eb0 0x7f212c0b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:45.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.528+0000 7f21329ab700 1 --2- 172.21.15.162:0/444807240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f212c0af6a0 0x7f212c060a70 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:45.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.528+0000 7f21329ab700 1 -- 172.21.15.162:0/444807240 >> 172.21.15.162:0/444807240 conn(0x7f212c0b57d0 msgr2=0x7f212c0b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:45.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.528+0000 7f21329ab700 1 -- 172.21.15.162:0/444807240 shutdown_connections 2020-11-28T21:53:45.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.528+0000 7f21329ab700 1 -- 172.21.15.162:0/444807240 wait complete. 2020-11-28T21:53:45.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.528+0000 7f21329ab700 1 Processor -- start 2020-11-28T21:53:45.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.528+0000 7f21329ab700 1 -- start start 2020-11-28T21:53:45.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.528+0000 7f21329ab700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f212c0af6a0 0x7f212c09e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:45.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.528+0000 7f21329ab700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f212c0b1eb0 0x7f212c159c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:45.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.528+0000 7f21329ab700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f212c144350 0x7f212c15a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:45.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.528+0000 7f21329ab700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f212c0667e0 con 0x7f212c0af6a0 2020-11-28T21:53:45.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.528+0000 7f21329ab700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f212c09f440 con 0x7f212c0b1eb0 2020-11-28T21:53:45.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.528+0000 7f21329ab700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f212c065cc0 con 0x7f212c144350 2020-11-28T21:53:45.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.528+0000 7f21311a8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f212c0b1eb0 0x7f212c159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:45.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.528+0000 7f21319a9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f212c0af6a0 0x7f212c09e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:45.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.528+0000 7f21311a8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f212c0b1eb0 0x7f212c159c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42352/0 (socket says 172.21.15.162:42352) 2020-11-28T21:53:45.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.528+0000 7f21311a8700 1 -- 172.21.15.162:0/727632431 learned_addr learned my addr 172.21.15.162:0/727632431 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:45.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.528+0000 7f21321aa700 1 --2- 172.21.15.162:0/727632431 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f212c144350 0x7f212c15a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:45.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.532+0000 7f21319a9700 1 -- 172.21.15.162:0/727632431 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f212c144350 msgr2=0x7f212c15a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:45.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.532+0000 7f21319a9700 1 --2- 172.21.15.162:0/727632431 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f212c144350 0x7f212c15a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:45.543 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.532+0000 7f21319a9700 1 -- 172.21.15.162:0/727632431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f212c0b1eb0 msgr2=0x7f212c159c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:45.543 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.532+0000 7f21319a9700 1 --2- 172.21.15.162:0/727632431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f212c0b1eb0 0x7f212c159c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:45.543 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.532+0000 7f21319a9700 1 -- 172.21.15.162:0/727632431 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f212c15a660 con 0x7f212c0af6a0 2020-11-28T21:53:45.543 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.532+0000 7f21311a8700 1 --2- 172.21.15.162:0/727632431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f212c0b1eb0 0x7f212c159c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:53:45.544 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.532+0000 7f21319a9700 1 --2- 172.21.15.162:0/727632431 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f212c0af6a0 0x7f212c09e7e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7f211c00a000 tx=0x7f211c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:45.544 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.532+0000 7f2122ffd700 1 -- 172.21.15.162:0/727632431 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f211c00f7e0 con 0x7f212c0af6a0 2020-11-28T21:53:45.544 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.532+0000 7f2122ffd700 1 -- 172.21.15.162:0/727632431 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f211c00f940 con 0x7f212c0af6a0 2020-11-28T21:53:45.544 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.532+0000 7f21329ab700 1 -- 172.21.15.162:0/727632431 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f212c066160 con 0x7f212c0af6a0 2020-11-28T21:53:45.545 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.532+0000 7f2122ffd700 1 -- 172.21.15.162:0/727632431 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f211c00fca0 con 0x7f212c0af6a0 2020-11-28T21:53:45.545 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.532+0000 7f21329ab700 1 -- 172.21.15.162:0/727632431 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f212c09f100 con 0x7f212c0af6a0 2020-11-28T21:53:45.545 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.532+0000 7f21329ab700 1 -- 172.21.15.162:0/727632431 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f212c065e80 con 0x7f212c0af6a0 2020-11-28T21:53:45.545 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.532+0000 7f2122ffd700 1 -- 172.21.15.162:0/727632431 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f211c01a030 con 0x7f212c0af6a0 2020-11-28T21:53:45.545 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.532+0000 7f2122ffd700 1 --2- 172.21.15.162:0/727632431 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f210c0554e0 0x7f210c0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:45.546 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.532+0000 7f2122ffd700 1 -- 172.21.15.162:0/727632431 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f211c06df10 con 0x7f212c0af6a0 2020-11-28T21:53:45.546 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.532+0000 7f21311a8700 1 --2- 172.21.15.162:0/727632431 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f210c0554e0 0x7f210c0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:45.546 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.532+0000 7f21311a8700 1 --2- 172.21.15.162:0/727632431 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f210c0554e0 0x7f210c0579a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7f212801a000 tx=0x7f2128016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:45.548 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.544+0000 7f2122ffd700 1 -- 172.21.15.162:0/727632431 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f211c044210 con 0x7f212c0af6a0 2020-11-28T21:53:45.690 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.688+0000 7f21329ab700 1 -- 172.21.15.162:0/727632431 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f212c057050 con 0x7f212c0af6a0 2020-11-28T21:53:45.691 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.688+0000 7f2122ffd700 1 -- 172.21.15.162:0/727632431 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+10 (secure 0 0 0) 0x7f211c047830 con 0x7f212c0af6a0 2020-11-28T21:53:45.692 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:53:45.693 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.688+0000 7f21329ab700 1 -- 172.21.15.162:0/727632431 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f210c0554e0 msgr2=0x7f210c0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:45.694 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.688+0000 7f21329ab700 1 --2- 172.21.15.162:0/727632431 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f210c0554e0 0x7f210c0579a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7f212801a000 tx=0x7f2128016000).stop 2020-11-28T21:53:45.694 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.688+0000 7f21329ab700 1 -- 172.21.15.162:0/727632431 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f212c0af6a0 msgr2=0x7f212c09e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:45.694 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.688+0000 7f21329ab700 1 --2- 172.21.15.162:0/727632431 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f212c0af6a0 0x7f212c09e7e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7f211c00a000 tx=0x7f211c008040).stop 2020-11-28T21:53:45.694 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.688+0000 7f21329ab700 1 -- 172.21.15.162:0/727632431 shutdown_connections 2020-11-28T21:53:45.695 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.688+0000 7f21329ab700 1 --2- 172.21.15.162:0/727632431 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f210c0554e0 0x7f210c0579a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:45.695 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.688+0000 7f21329ab700 1 --2- 172.21.15.162:0/727632431 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f212c144350 0x7f212c15a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:45.695 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.688+0000 7f21329ab700 1 --2- 172.21.15.162:0/727632431 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f212c0af6a0 0x7f212c09e7e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:45.695 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.688+0000 7f21329ab700 1 --2- 172.21.15.162:0/727632431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f212c0b1eb0 0x7f212c159c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:45.696 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.688+0000 7f21329ab700 1 -- 172.21.15.162:0/727632431 >> 172.21.15.162:0/727632431 conn(0x7f212c0b57d0 msgr2=0x7f212c144d10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:45.696 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.688+0000 7f21329ab700 1 -- 172.21.15.162:0/727632431 shutdown_connections 2020-11-28T21:53:45.696 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.688+0000 7f21329ab700 1 -- 172.21.15.162:0/727632431 wait complete. 2020-11-28T21:53:45.705 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-28T21:53:45.706 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:53:45.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.812+0000 7f1a38bea700 1 Processor -- start 2020-11-28T21:53:45.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.812+0000 7f1a38bea700 1 -- start start 2020-11-28T21:53:45.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.816+0000 7f1a38bea700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1a34143da0 0x7f1a3413d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:45.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.816+0000 7f1a38bea700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1a340b1500 0x7f1a3413da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:45.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.816+0000 7f1a38bea700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a34066df0 con 0x7f1a34066a10 2020-11-28T21:53:45.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.816+0000 7f1a38bea700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a34060720 con 0x7f1a34143da0 2020-11-28T21:53:45.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.816+0000 7f1a38bea700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a340b12b0 con 0x7f1a340b1500 2020-11-28T21:53:45.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.816+0000 7f1a33fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1a340b1500 0x7f1a3413da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:45.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.816+0000 7f1a337fe700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f1a34066a10 0x7f1a34061760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:49204/0 (socket says 172.21.15.162:49204) 2020-11-28T21:53:45.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.816+0000 7f1a337fe700 1 -- 172.21.15.162:0/3778868408 learned_addr learned my addr 172.21.15.162:0/3778868408 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:45.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.816+0000 7f1a32ffd700 1 --2- 172.21.15.162:0/3778868408 >> v2:172.21.15.118:3300/0 conn(0x7f1a34143da0 0x7f1a3413d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:45.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.816+0000 7f1a33fff700 1 -- 172.21.15.162:0/3778868408 >> v2:172.21.15.118:3300/0 conn(0x7f1a34143da0 msgr2=0x7f1a3413d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:45.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.816+0000 7f1a33fff700 1 --2- 172.21.15.162:0/3778868408 >> v2:172.21.15.118:3300/0 conn(0x7f1a34143da0 0x7f1a3413d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:45.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.816+0000 7f1a33fff700 1 -- 172.21.15.162:0/3778868408 >> v1:172.21.15.162:6789/0 conn(0x7f1a34066a10 legacy=0x7f1a34061760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:45.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.816+0000 7f1a327fc700 1 -- 172.21.15.162:0/3778868408 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3360966125 0 0) 0x7f1a34066df0 con 0x7f1a34066a10 2020-11-28T21:53:45.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.816+0000 7f1a33fff700 1 -- 172.21.15.162:0/3778868408 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a340b0d70 con 0x7f1a340b1500 2020-11-28T21:53:45.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.816+0000 7f1a33fff700 1 --2- 172.21.15.162:0/3778868408 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a340b1500 0x7f1a3413da90 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 rx=0x7f1a2c002f30 tx=0x7f1a2c006dc0).ready entity=mon.1 client_cookie=e15e7288f9ecae43 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:45.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.816+0000 7f1a327fc700 1 -- 172.21.15.162:0/3778868408 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a2c0097b0 con 0x7f1a340b1500 2020-11-28T21:53:45.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.816+0000 7f1a327fc700 1 -- 172.21.15.162:0/3778868408 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a2c009910 con 0x7f1a340b1500 2020-11-28T21:53:45.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.816+0000 7f1a327fc700 1 -- 172.21.15.162:0/3778868408 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a2c009c70 con 0x7f1a340b1500 2020-11-28T21:53:45.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.816+0000 7f1a38bea700 1 -- 172.21.15.162:0/3778868408 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a340b1500 msgr2=0x7f1a3413da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:45.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.816+0000 7f1a38bea700 1 --2- 172.21.15.162:0/3778868408 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a340b1500 0x7f1a3413da90 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 rx=0x7f1a2c002f30 tx=0x7f1a2c006dc0).stop 2020-11-28T21:53:45.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.816+0000 7f1a38bea700 1 -- 172.21.15.162:0/3778868408 shutdown_connections 2020-11-28T21:53:45.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.816+0000 7f1a38bea700 1 --2- 172.21.15.162:0/3778868408 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a340b1500 0x7f1a3413da90 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:45.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.816+0000 7f1a38bea700 1 --2- 172.21.15.162:0/3778868408 >> v2:172.21.15.118:3300/0 conn(0x7f1a34143da0 0x7f1a3413d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:45.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.816+0000 7f1a38bea700 1 -- 172.21.15.162:0/3778868408 >> 172.21.15.162:0/3778868408 conn(0x7f1a340b5da0 msgr2=0x7f1a340b6180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:45.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.816+0000 7f1a38bea700 1 -- 172.21.15.162:0/3778868408 shutdown_connections 2020-11-28T21:53:45.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.816+0000 7f1a38bea700 1 -- 172.21.15.162:0/3778868408 wait complete. 2020-11-28T21:53:45.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.820+0000 7f1a38bea700 1 Processor -- start 2020-11-28T21:53:45.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.820+0000 7f1a38bea700 1 -- start start 2020-11-28T21:53:45.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.820+0000 7f1a38bea700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1a34066a10 0x7f1a34150050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:45.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.820+0000 7f1a38bea700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1a340b1500 0x7f1a340af6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:45.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.820+0000 7f1a38bea700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a34143da0 0x7f1a340afb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:45.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.820+0000 7f1a38bea700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a34150540 con 0x7f1a34066a10 2020-11-28T21:53:45.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.820+0000 7f1a38bea700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3413f4c0 con 0x7f1a34143da0 2020-11-28T21:53:45.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.820+0000 7f1a38bea700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a34147560 con 0x7f1a340b1500 2020-11-28T21:53:45.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.820+0000 7f1a33fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a34143da0 0x7f1a340afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:45.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.820+0000 7f1a33fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a34143da0 0x7f1a340afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42364/0 (socket says 172.21.15.162:42364) 2020-11-28T21:53:45.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.820+0000 7f1a33fff700 1 -- 172.21.15.162:0/3634951750 learned_addr learned my addr 172.21.15.162:0/3634951750 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:45.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.820+0000 7f1a337fe700 1 --2- 172.21.15.162:0/3634951750 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1a34066a10 0x7f1a34150050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:45.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.820+0000 7f1a32ffd700 1 --2- 172.21.15.162:0/3634951750 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1a340b1500 0x7f1a340af6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:45.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.820+0000 7f1a33fff700 1 -- 172.21.15.162:0/3634951750 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1a340b1500 msgr2=0x7f1a340af6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:45.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.820+0000 7f1a33fff700 1 --2- 172.21.15.162:0/3634951750 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1a340b1500 0x7f1a340af6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:45.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.820+0000 7f1a33fff700 1 -- 172.21.15.162:0/3634951750 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1a34066a10 msgr2=0x7f1a34150050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:45.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.820+0000 7f1a33fff700 1 --2- 172.21.15.162:0/3634951750 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1a34066a10 0x7f1a34150050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:45.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.820+0000 7f1a33fff700 1 -- 172.21.15.162:0/3634951750 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a34147800 con 0x7f1a34143da0 2020-11-28T21:53:45.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.820+0000 7f1a32ffd700 1 --2- 172.21.15.162:0/3634951750 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1a340b1500 0x7f1a340af6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:53:45.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.820+0000 7f1a33fff700 1 --2- 172.21.15.162:0/3634951750 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a34143da0 0x7f1a340afb90 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 rx=0x7f1a34150680 tx=0x7f1a2c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:45.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.820+0000 7f1a30ff9700 1 -- 172.21.15.162:0/3634951750 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a2c011c80 con 0x7f1a34143da0 2020-11-28T21:53:45.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.820+0000 7f1a30ff9700 1 -- 172.21.15.162:0/3634951750 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a2c011de0 con 0x7f1a34143da0 2020-11-28T21:53:45.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.820+0000 7f1a30ff9700 1 -- 172.21.15.162:0/3634951750 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a2c01d630 con 0x7f1a34143da0 2020-11-28T21:53:45.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.820+0000 7f1a38bea700 1 -- 172.21.15.162:0/3634951750 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a34147aa0 con 0x7f1a34143da0 2020-11-28T21:53:45.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.824+0000 7f1a38bea700 1 -- 172.21.15.162:0/3634951750 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a3413e880 con 0x7f1a34143da0 2020-11-28T21:53:45.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.824+0000 7f1a38bea700 1 -- 172.21.15.162:0/3634951750 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a3413f650 con 0x7f1a34143da0 2020-11-28T21:53:45.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.824+0000 7f1a30ff9700 1 -- 172.21.15.162:0/3634951750 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1a2c024030 con 0x7f1a34143da0 2020-11-28T21:53:45.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.824+0000 7f1a30ff9700 1 --2- 172.21.15.162:0/3634951750 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1a18055470 0x7f1a18057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:45.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.824+0000 7f1a337fe700 1 --2- 172.21.15.162:0/3634951750 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1a18055470 0x7f1a18057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:45.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.824+0000 7f1a30ff9700 1 -- 172.21.15.162:0/3634951750 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1a2c06ed80 con 0x7f1a34143da0 2020-11-28T21:53:45.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.824+0000 7f1a337fe700 1 --2- 172.21.15.162:0/3634951750 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1a18055470 0x7f1a18057930 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f1a2400b000 tx=0x7f1a24009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:45.838 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:45.828+0000 7f1a30ff9700 1 -- 172.21.15.162:0/3634951750 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1a3413f650 con 0x7f1a34143da0 2020-11-28T21:53:46.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:46.012+0000 7f1a38bea700 1 -- 172.21.15.162:0/3634951750 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a3413f600 con 0x7f1a34143da0 2020-11-28T21:53:46.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:46.012+0000 7f1a30ff9700 1 -- 172.21.15.162:0/3634951750 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+2502 (secure 0 0 0) 0x7f1a2c070020 con 0x7f1a34143da0 2020-11-28T21:53:46.018 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:53:46.019 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":40,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5795,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/2645168518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":2645168518},{"type":"v1","addr":"172.21.15.118:6833","nonce":2645168518}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36},{"gid":6541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1999073284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1999073284},{"type":"v1","addr":"172.21.15.162:6835","nonce":1999073284}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:53:36.539204+0000","modified":"2020-11-28T21:53:38.035411+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6550":{"gid":6550,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/1097065372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1097065372},{"type":"v1","addr":"172.21.15.162:6837","nonce":1097065372}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-28T21:53:46.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:46.016+0000 7f1a38bea700 1 -- 172.21.15.162:0/3634951750 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1a18055470 msgr2=0x7f1a18057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:46.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:46.016+0000 7f1a38bea700 1 --2- 172.21.15.162:0/3634951750 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1a18055470 0x7f1a18057930 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f1a2400b000 tx=0x7f1a24009040).stop 2020-11-28T21:53:46.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:46.016+0000 7f1a38bea700 1 -- 172.21.15.162:0/3634951750 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a34143da0 msgr2=0x7f1a340afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:46.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:46.016+0000 7f1a38bea700 1 --2- 172.21.15.162:0/3634951750 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a34143da0 0x7f1a340afb90 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 rx=0x7f1a34150680 tx=0x7f1a2c015000).stop 2020-11-28T21:53:46.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:46.016+0000 7f1a38bea700 1 -- 172.21.15.162:0/3634951750 shutdown_connections 2020-11-28T21:53:46.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:46.016+0000 7f1a38bea700 1 --2- 172.21.15.162:0/3634951750 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1a18055470 0x7f1a18057930 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:46.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:46.016+0000 7f1a38bea700 1 --2- 172.21.15.162:0/3634951750 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a34143da0 0x7f1a340afb90 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:46.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:46.016+0000 7f1a38bea700 1 --2- 172.21.15.162:0/3634951750 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1a34066a10 0x7f1a34150050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:46.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:46.016+0000 7f1a38bea700 1 --2- 172.21.15.162:0/3634951750 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1a340b1500 0x7f1a340af6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:46.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:46.016+0000 7f1a38bea700 1 -- 172.21.15.162:0/3634951750 >> 172.21.15.162:0/3634951750 conn(0x7f1a340b5da0 msgr2=0x7f1a340b4130 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:46.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:46.016+0000 7f1a38bea700 1 -- 172.21.15.162:0/3634951750 shutdown_connections 2020-11-28T21:53:46.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:46.016+0000 7f1a38bea700 1 -- 172.21.15.162:0/3634951750 wait complete. 2020-11-28T21:53:46.024 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 40 2020-11-28T21:53:46.333 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 40, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-28T21:53:36.539204+0000', 'modified': '2020-11-28T21:53:38.035411+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6550}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6550': {'gid': 6550, 'name': 'c', 'rank': 0, 'incarnation': 38, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6837/1097065372', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6836', 'nonce': 1097065372}, {'type': 'v1', 'addr': '172.21.15.162:6837', 'nonce': 1097065372}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-28T21:53:46.333 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-28T21:53:46.334 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-28T21:53:46.461 INFO:teuthology.orchestra.run.smithi162.stdout:{"cluster_fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","whoami":0,"id":6550,"want_state":"up:active","state":"up:active","rank_uptime":9.9079019259999992,"mdsmap_epoch":40,"osdmap_epoch":88,"osdmap_epoch_barrier":88,"uptime":15.304947693000001} 2020-11-28T21:53:46.462 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", "id": 6550, "mdsmap_epoch": 40, "osdmap_epoch": 88, "osdmap_epoch_barrier": 88, "rank_uptime": 9.907901926, "state": "up:active", "uptime": 15.304947693, "want_state": "up:active", "whoami": 0 } 2020-11-28T21:53:46.463 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-11-28T21:53:46.463 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2020-11-28T21:53:46.463 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi096.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2020-11-28T21:53:46.464 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:53:46.464 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:53:46.472 INFO:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2020-11-28T21:53:46.531 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:53:46.532 INFO:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:53:46.587 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:53:46.588 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:53:46.589 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:53:46.589 INFO:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2020-11-28T21:53:46.634 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-11-28T21:53:46.635 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:53:46.678 INFO:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2020-11-28T21:53:46.693 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:53:46.693 INFO:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:53:46.750 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:53:46.751 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:53:46.751 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:53:46.752 INFO:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2020-11-28T21:53:46.753 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:2020-11-28T21:53:46.747+0000 7efd988920c0 -1 init, newargv = 0x564f04c0f2e0 newargc=9ceph-fuse[11636]: starting ceph client 2020-11-28T21:53:46.753 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr: 2020-11-28T21:53:46.764 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[11636]: starting fuse 2020-11-28T21:53:47.859 INFO:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2020-11-28T21:53:47.875 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:53:47.876 INFO:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:53:47.932 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:53:47.933 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:53:47.934 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:53:47.934 INFO:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2020-11-28T21:53:47.983 INFO:teuthology.orchestra.run.smithi096.stdout:49 2020-11-28T21:53:47.983 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2020-11-28T21:53:47.984 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:53:47.985 INFO:teuthology.orchestra.run.smithi096:> sudo python3 -c ' 2020-11-28T21:53:47.985 INFO:teuthology.orchestra.run.smithi096:> import glob 2020-11-28T21:53:47.986 INFO:teuthology.orchestra.run.smithi096:> import re 2020-11-28T21:53:47.987 INFO:teuthology.orchestra.run.smithi096:> import os 2020-11-28T21:53:47.988 INFO:teuthology.orchestra.run.smithi096:> import subprocess 2020-11-28T21:53:47.988 INFO:teuthology.orchestra.run.smithi096:> 2020-11-28T21:53:47.989 INFO:teuthology.orchestra.run.smithi096:> def find_socket(client_name): 2020-11-28T21:53:47.989 INFO:teuthology.orchestra.run.smithi096:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2020-11-28T21:53:47.990 INFO:teuthology.orchestra.run.smithi096:> files = glob.glob(asok_path) 2020-11-28T21:53:47.990 INFO:teuthology.orchestra.run.smithi096:> 2020-11-28T21:53:47.991 INFO:teuthology.orchestra.run.smithi096:> # Given a non-glob path, it better be there 2020-11-28T21:53:47.991 INFO:teuthology.orchestra.run.smithi096:> if "*" not in asok_path: 2020-11-28T21:53:47.991 INFO:teuthology.orchestra.run.smithi096:> assert(len(files) == 1) 2020-11-28T21:53:47.992 INFO:teuthology.orchestra.run.smithi096:> return files[0] 2020-11-28T21:53:47.992 INFO:teuthology.orchestra.run.smithi096:> 2020-11-28T21:53:47.993 INFO:teuthology.orchestra.run.smithi096:> for f in files: 2020-11-28T21:53:47.993 INFO:teuthology.orchestra.run.smithi096:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-11-28T21:53:47.993 INFO:teuthology.orchestra.run.smithi096:> if os.path.exists("/proc/{0}".format(pid)): 2020-11-28T21:53:47.994 INFO:teuthology.orchestra.run.smithi096:> return f 2020-11-28T21:53:47.994 INFO:teuthology.orchestra.run.smithi096:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-11-28T21:53:47.995 INFO:teuthology.orchestra.run.smithi096:> 2020-11-28T21:53:47.995 INFO:teuthology.orchestra.run.smithi096:> print(find_socket("client.1")) 2020-11-28T21:53:47.995 INFO:teuthology.orchestra.run.smithi096:> ' 2020-11-28T21:53:48.074 INFO:teuthology.orchestra.run.smithi096.stdout:/var/run/ceph/ceph-client.1.11636.asok 2020-11-28T21:53:48.079 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.11636.asok 2020-11-28T21:53:48.080 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:53:48.080 INFO:teuthology.orchestra.run.smithi096:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.11636.asok status 2020-11-28T21:53:48.183 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-11-28T21:53:48.184 INFO:teuthology.orchestra.run.smithi096.stdout: "metadata": { 2020-11-28T21:53:48.184 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_sha1": "a83a3572a957356977342f461361c1cd3ec06dbd", 2020-11-28T21:53:48.184 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_version": "ceph version 15.2.6-595-ga83a3572 (a83a3572a957356977342f461361c1cd3ec06dbd) octopus (stable)", 2020-11-28T21:53:48.185 INFO:teuthology.orchestra.run.smithi096.stdout: "entity_id": "1", 2020-11-28T21:53:48.185 INFO:teuthology.orchestra.run.smithi096.stdout: "hostname": "smithi096", 2020-11-28T21:53:48.185 INFO:teuthology.orchestra.run.smithi096.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2020-11-28T21:53:48.185 INFO:teuthology.orchestra.run.smithi096.stdout: "pid": "11636", 2020-11-28T21:53:48.185 INFO:teuthology.orchestra.run.smithi096.stdout: "root": "/" 2020-11-28T21:53:48.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-11-28T21:53:48.186 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_count": 0, 2020-11-28T21:53:48.186 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_pinned_count": 0, 2020-11-28T21:53:48.186 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 6561, 2020-11-28T21:53:48.187 INFO:teuthology.orchestra.run.smithi096.stdout: "inst": { 2020-11-28T21:53:48.187 INFO:teuthology.orchestra.run.smithi096.stdout: "name": { 2020-11-28T21:53:48.187 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "client", 2020-11-28T21:53:48.187 INFO:teuthology.orchestra.run.smithi096.stdout: "num": 6561 2020-11-28T21:53:48.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-11-28T21:53:48.188 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2020-11-28T21:53:48.188 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2020-11-28T21:53:48.188 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2020-11-28T21:53:48.188 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3294740242 2020-11-28T21:53:48.189 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-11-28T21:53:48.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-11-28T21:53:48.189 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2020-11-28T21:53:48.189 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2020-11-28T21:53:48.190 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2020-11-28T21:53:48.190 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3294740242 2020-11-28T21:53:48.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-11-28T21:53:48.190 INFO:teuthology.orchestra.run.smithi096.stdout: "inst_str": "client.6561 172.21.15.96:0/3294740242", 2020-11-28T21:53:48.191 INFO:teuthology.orchestra.run.smithi096.stdout: "addr_str": "172.21.15.96:0/3294740242", 2020-11-28T21:53:48.191 INFO:teuthology.orchestra.run.smithi096.stdout: "inode_count": 1, 2020-11-28T21:53:48.191 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_epoch": 40, 2020-11-28T21:53:48.191 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch": 88, 2020-11-28T21:53:48.191 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch_barrier": 0, 2020-11-28T21:53:48.192 INFO:teuthology.orchestra.run.smithi096.stdout: "blacklisted": false 2020-11-28T21:53:48.192 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-11-28T21:53:48.193 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:53:48.193 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:53:48.194 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:53:48.202 INFO:teuthology.orchestra.run.smithi096.stdout:fuseblk 2020-11-28T21:53:48.202 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-11-28T21:53:48.203 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:53:48.203 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:53:48.262 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:53:48.383 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.380+0000 7fec44caf700 1 Processor -- start 2020-11-28T21:53:48.383 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.380+0000 7fec44caf700 1 -- start start 2020-11-28T21:53:48.384 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.380+0000 7fec44caf700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fec400b11b0 0x7fec40140fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:48.384 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.380+0000 7fec44caf700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fec400b1d70 0x7fec4014a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:48.384 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.380+0000 7fec44caf700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec400b2870 con 0x7fec400b1990 2020-11-28T21:53:48.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.380+0000 7fec44caf700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec40146e20 con 0x7fec400b1d70 2020-11-28T21:53:48.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.380+0000 7fec44caf700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec400b22e0 con 0x7fec400b11b0 2020-11-28T21:53:48.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.380+0000 7fec3ed9d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fec400b1d70 0x7fec4014a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:48.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.380+0000 7fec3ed9d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fec400b1d70 0x7fec4014a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.162:49360/0 (socket says 172.21.15.162:49360) 2020-11-28T21:53:48.386 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.380+0000 7fec3ed9d700 1 -- 172.21.15.162:0/2096562864 learned_addr learned my addr 172.21.15.162:0/2096562864 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:48.386 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.380+0000 7fec3dd9b700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fec400b11b0 0x7fec40140fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:48.388 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.380+0000 7fec3dd9b700 1 -- 172.21.15.162:0/2096562864 >> v2:172.21.15.118:3300/0 conn(0x7fec400b1d70 msgr2=0x7fec4014a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:48.388 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.380+0000 7fec3dd9b700 1 --2- 172.21.15.162:0/2096562864 >> v2:172.21.15.118:3300/0 conn(0x7fec400b1d70 0x7fec4014a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:48.388 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.380+0000 7fec3dd9b700 1 -- 172.21.15.162:0/2096562864 >> v1:172.21.15.118:6789/0 conn(0x7fec400b1990 legacy=0x7fec40140890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:48.388 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.380+0000 7fec3d59a700 1 -- 172.21.15.162:0/2096562864 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 541863805 0 0) 0x7fec400b2870 con 0x7fec400b1990 2020-11-28T21:53:48.389 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.380+0000 7fec3dd9b700 1 -- 172.21.15.162:0/2096562864 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec400b0740 con 0x7fec400b11b0 2020-11-28T21:53:48.389 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.380+0000 7fec3ed9d700 1 --2- 172.21.15.162:0/2096562864 >> v2:172.21.15.118:3300/0 conn(0x7fec400b1d70 0x7fec4014a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:53:48.389 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.380+0000 7fec3dd9b700 1 --2- 172.21.15.162:0/2096562864 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fec400b11b0 0x7fec40140fa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fec28002f30 tx=0x7fec28006dc0).ready entity=mon.2 client_cookie=23a5ef34375f68e5 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:48.389 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.380+0000 7fec3d59a700 1 -- 172.21.15.162:0/2096562864 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec28009760 con 0x7fec400b11b0 2020-11-28T21:53:48.389 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.380+0000 7fec3d59a700 1 -- 172.21.15.162:0/2096562864 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec280098c0 con 0x7fec400b11b0 2020-11-28T21:53:48.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.380+0000 7fec3d59a700 1 -- 172.21.15.162:0/2096562864 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec28009c20 con 0x7fec400b11b0 2020-11-28T21:53:48.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.380+0000 7fec44caf700 1 -- 172.21.15.162:0/2096562864 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fec400b11b0 msgr2=0x7fec40140fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:48.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.380+0000 7fec44caf700 1 --2- 172.21.15.162:0/2096562864 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fec400b11b0 0x7fec40140fa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fec28002f30 tx=0x7fec28006dc0).stop 2020-11-28T21:53:48.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.380+0000 7fec44caf700 1 -- 172.21.15.162:0/2096562864 shutdown_connections 2020-11-28T21:53:48.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.380+0000 7fec44caf700 1 --2- 172.21.15.162:0/2096562864 >> v2:172.21.15.118:3300/0 conn(0x7fec400b1d70 0x7fec4014a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:48.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.380+0000 7fec44caf700 1 --2- 172.21.15.162:0/2096562864 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fec400b11b0 0x7fec40140fa0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:48.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.380+0000 7fec44caf700 1 -- 172.21.15.162:0/2096562864 >> 172.21.15.162:0/2096562864 conn(0x7fec4005b970 msgr2=0x7fec4005bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:48.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.384+0000 7fec44caf700 1 -- 172.21.15.162:0/2096562864 shutdown_connections 2020-11-28T21:53:48.392 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.384+0000 7fec44caf700 1 -- 172.21.15.162:0/2096562864 wait complete. 2020-11-28T21:53:48.395 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.384+0000 7fec44caf700 1 Processor -- start 2020-11-28T21:53:48.395 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.384+0000 7fec44caf700 1 -- start start 2020-11-28T21:53:48.395 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.388+0000 7fec44caf700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec400b11b0 0x7fec401594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:48.395 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.388+0000 7fec44caf700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fec400b1990 0x7fec401599b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:48.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.388+0000 7fec44caf700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fec400b1d70 0x7fec40142920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:48.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.388+0000 7fec44caf700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec400b2f90 con 0x7fec400b1990 2020-11-28T21:53:48.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.388+0000 7fec44caf700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec400b39a0 con 0x7fec400b11b0 2020-11-28T21:53:48.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.388+0000 7fec44caf700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec40159fe0 con 0x7fec400b1d70 2020-11-28T21:53:48.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.388+0000 7fec3e59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec400b11b0 0x7fec401594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:48.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.388+0000 7fec3e59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec400b11b0 0x7fec401594c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42378/0 (socket says 172.21.15.162:42378) 2020-11-28T21:53:48.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.388+0000 7fec3e59c700 1 -- 172.21.15.162:0/3862911653 learned_addr learned my addr 172.21.15.162:0/3862911653 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:48.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.388+0000 7fec3ed9d700 1 --2- 172.21.15.162:0/3862911653 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fec400b1d70 0x7fec40142920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:48.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.388+0000 7fec3dd9b700 1 --2- 172.21.15.162:0/3862911653 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fec400b1990 0x7fec401599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:48.398 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.388+0000 7fec3e59c700 1 -- 172.21.15.162:0/3862911653 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fec400b1d70 msgr2=0x7fec40142920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:48.398 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.388+0000 7fec3e59c700 1 --2- 172.21.15.162:0/3862911653 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fec400b1d70 0x7fec40142920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:48.398 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.388+0000 7fec3e59c700 1 -- 172.21.15.162:0/3862911653 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fec400b1990 msgr2=0x7fec401599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:48.398 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.388+0000 7fec3e59c700 1 --2- 172.21.15.162:0/3862911653 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fec400b1990 0x7fec401599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:48.399 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.388+0000 7fec3e59c700 1 -- 172.21.15.162:0/3862911653 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec4015a280 con 0x7fec400b11b0 2020-11-28T21:53:48.399 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.388+0000 7fec3e59c700 1 --2- 172.21.15.162:0/3862911653 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec400b11b0 0x7fec401594c0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 rx=0x7fec30000e90 tx=0x7fec30007a90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:48.399 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.388+0000 7fec377fe700 1 -- 172.21.15.162:0/3862911653 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec3000f470 con 0x7fec400b11b0 2020-11-28T21:53:48.400 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.388+0000 7fec377fe700 1 -- 172.21.15.162:0/3862911653 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec3000f5d0 con 0x7fec400b11b0 2020-11-28T21:53:48.400 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.388+0000 7fec44caf700 1 -- 172.21.15.162:0/3862911653 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec400b3ff0 con 0x7fec400b11b0 2020-11-28T21:53:48.400 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.388+0000 7fec377fe700 1 -- 172.21.15.162:0/3862911653 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec3000f930 con 0x7fec400b11b0 2020-11-28T21:53:48.401 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.388+0000 7fec44caf700 1 -- 172.21.15.162:0/3862911653 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec40142e10 con 0x7fec400b11b0 2020-11-28T21:53:48.401 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.388+0000 7fec377fe700 1 -- 172.21.15.162:0/3862911653 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fec30019030 con 0x7fec400b11b0 2020-11-28T21:53:48.401 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.392+0000 7fec44caf700 1 -- 172.21.15.162:0/3862911653 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec40143460 con 0x7fec400b11b0 2020-11-28T21:53:48.401 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.392+0000 7fec377fe700 1 --2- 172.21.15.162:0/3862911653 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fec240553b0 0x7fec24057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:48.402 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.392+0000 7fec377fe700 1 -- 172.21.15.162:0/3862911653 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fec3006d3b0 con 0x7fec400b11b0 2020-11-28T21:53:48.402 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.392+0000 7fec3dd9b700 1 --2- 172.21.15.162:0/3862911653 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fec240553b0 0x7fec24057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:48.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.392+0000 7fec3dd9b700 1 --2- 172.21.15.162:0/3862911653 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fec240553b0 0x7fec24057870 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7fec2801a000 tx=0x7fec28015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:48.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.396+0000 7fec377fe700 1 -- 172.21.15.162:0/3862911653 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fec40143460 con 0x7fec400b11b0 2020-11-28T21:53:48.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.580+0000 7fec44caf700 1 -- 172.21.15.162:0/3862911653 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec40143460 con 0x7fec400b11b0 2020-11-28T21:53:48.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.580+0000 7fec377fe700 1 -- 172.21.15.162:0/3862911653 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+2502 (secure 0 0 0) 0x7fec40143460 con 0x7fec400b11b0 2020-11-28T21:53:48.585 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:53:48.586 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":40,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5795,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/2645168518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":2645168518},{"type":"v1","addr":"172.21.15.118:6833","nonce":2645168518}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36},{"gid":6541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1999073284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1999073284},{"type":"v1","addr":"172.21.15.162:6835","nonce":1999073284}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:53:36.539204+0000","modified":"2020-11-28T21:53:38.035411+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6550":{"gid":6550,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/1097065372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1097065372},{"type":"v1","addr":"172.21.15.162:6837","nonce":1097065372}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-28T21:53:48.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.584+0000 7fec44caf700 1 -- 172.21.15.162:0/3862911653 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fec240553b0 msgr2=0x7fec24057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:48.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.584+0000 7fec44caf700 1 --2- 172.21.15.162:0/3862911653 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fec240553b0 0x7fec24057870 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7fec2801a000 tx=0x7fec28015000).stop 2020-11-28T21:53:48.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.584+0000 7fec44caf700 1 -- 172.21.15.162:0/3862911653 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec400b11b0 msgr2=0x7fec401594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:48.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.584+0000 7fec44caf700 1 --2- 172.21.15.162:0/3862911653 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec400b11b0 0x7fec401594c0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 rx=0x7fec30000e90 tx=0x7fec30007a90).stop 2020-11-28T21:53:48.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.584+0000 7fec44caf700 1 -- 172.21.15.162:0/3862911653 shutdown_connections 2020-11-28T21:53:48.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.584+0000 7fec44caf700 1 --2- 172.21.15.162:0/3862911653 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fec240553b0 0x7fec24057870 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:48.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.584+0000 7fec44caf700 1 --2- 172.21.15.162:0/3862911653 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fec400b1d70 0x7fec40142920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:48.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.584+0000 7fec44caf700 1 --2- 172.21.15.162:0/3862911653 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec400b11b0 0x7fec401594c0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:48.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.584+0000 7fec44caf700 1 --2- 172.21.15.162:0/3862911653 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fec400b1990 0x7fec401599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:48.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.584+0000 7fec44caf700 1 -- 172.21.15.162:0/3862911653 >> 172.21.15.162:0/3862911653 conn(0x7fec4005b970 msgr2=0x7fec40149790 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:48.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.584+0000 7fec44caf700 1 -- 172.21.15.162:0/3862911653 shutdown_connections 2020-11-28T21:53:48.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:48.584+0000 7fec44caf700 1 -- 172.21.15.162:0/3862911653 wait complete. 2020-11-28T21:53:48.590 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 40 2020-11-28T21:53:48.599 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump inode 1 2020-11-28T21:53:48.717 INFO:teuthology.orchestra.run.smithi162.stdout:{"path":"/","ino":1,"rdev":0,"ctime":"2020-11-28T21:53:44.250311+0000","btime":"2020-11-28T21:53:36.543430+0000","mode":17407,"uid":0,"gid":0,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":4194304,"stripe_count":1,"object_size":4194304,"pool_id":16,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":0,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2020-11-28T21:53:36.543430+0000","atime":"0.000000","time_warp_seq":0,"change_attr":1,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"0.000000","num_files":0,"num_subdirs":0,"change_attr":0},"rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"2020-11-28T21:53:44.250311+0000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"2020-11-28T21:53:36.543430+0000"},"version":3,"file_data_version":0,"xattr_version":1,"backtrace_version":3,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":2,"is_frozen":false,"is_freezing":false,"pins":{"request":1,"lock":3,"dirfrag":1,"caps":1,"dirtyparent":1,"dirty":1,"authpin":1},"nref":9,"versionlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":1,"num_xlocks":0,"xlock_by":{}},"authlock":{"gather_set":[],"state":"xlockdone","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":1,"xlock_by":{}},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{"gather_set":[],"state":"sync","is_leased":false,"num_rdlocks":1,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"nestlock":{},"flocklock":{},"policylock":{},"states":["auth","dirty","dirtyparent"],"client_caps":[{"client_id":6561,"pending":"pLsXsFs","issued":"pLsXsFs","wanted":"-","last_sent":2},{"client_id":6706,"pending":"pLsXsFs","issued":"pLsXsFs","wanted":"-","last_sent":2}],"loner":-1,"want_loner":-1,"mds_caps_wanted":[]} 2020-11-28T21:53:48.718 DEBUG:tasks.cephfs.filesystem:_json_asok output { "accounted_rstat": { "rbytes": 0, "rctime": "2020-11-28T21:53:36.543430+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "atime": "0.000000", "auth_pins": 2, "auth_state": { "replicas": {} }, "authlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 1, "state": "xlockdone", "xlock_by": {} }, "backtrace_version": 3, "btime": "2020-11-28T21:53:36.543430+0000", "change_attr": 1, "client_caps": [ { "client_id": 6561, "issued": "pLsXsFs", "last_sent": 2, "pending": "pLsXsFs", "wanted": "-" }, { "client_id": 6706, "issued": "pLsXsFs", "last_sent": 2, "pending": "pLsXsFs", "wanted": "-" } ], "client_ranges": [], "ctime": "2020-11-28T21:53:44.250311+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 0, "mtime": "0.000000", "num_files": 0, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 0, "ino": 1, "is_auth": true, "is_freezing": false, "is_frozen": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 4194304, "pool_id": 16, "pool_ns": "", "stripe_count": 1, "stripe_unit": 4194304 }, "linklock": {}, "loner": -1, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 17407, "mtime": "2020-11-28T21:53:36.543430+0000", "nestlock": {}, "nlink": 1, "nref": 9, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "path": "/", "pins": { "authpin": 1, "caps": 1, "dirfrag": 1, "dirty": 1, "dirtyparent": 1, "lock": 3, "request": 1 }, "policylock": {}, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "2020-11-28T21:53:44.250311+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "size": 0, "snaplock": { "gather_set": [], "is_leased": false, "num_rdlocks": 1, "num_wrlocks": 0, "num_xlocks": 0, "state": "sync", "xlock_by": {} }, "states": [ "auth", "dirty", "dirtyparent" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 0, "truncate_size": 18446744073709551615, "uid": 0, "version": 3, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 1, "num_xlocks": 0, "state": "lock", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [] } 2020-11-28T21:53:48.719 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-28T21:53:48.719 INFO:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:53:48.737 INFO:teuthology.orchestra.run.smithi118:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:53:48.756 INFO:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:53:48.776 INFO:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:53:48.793 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-28T21:53:48.794 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:53:48.794 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:53:48.795 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:53:48.843 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2020-11-28T21:53:48.844 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-28T21:53:48.844 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2020-11-28T21:53:48.845 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-28T21:53:48.845 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:53:48.936 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:ceph-fuse[12215]: fuse finished with error 0 and tester_r 0 2020-11-28T21:53:48.938 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:53:48.939 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:53:48.939 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:53:48.947 INFO:teuthology.orchestra.run.smithi192.stdout:ext2/ext3 2020-11-28T21:53:48.948 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-11-28T21:53:49.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-28T21:53:49.065 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:53:49.114 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:53:49.114 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:53:49.115 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:53:49.123 INFO:teuthology.orchestra.run.smithi096.stdout:fuseblk 2020-11-28T21:53:49.123 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-11-28T21:53:49.124 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi096.front.sepia.ceph.com... 2020-11-28T21:53:49.124 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-28T21:53:49.125 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:53:49.197 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[11636]: fuse finished with error 0 and tester_r 0 2020-11-28T21:53:49.199 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:53:49.199 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:53:49.200 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:53:49.247 INFO:teuthology.orchestra.run.smithi096.stdout:ext2/ext3 2020-11-28T21:53:49.248 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-11-28T21:53:49.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-28T21:53:49.352 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:53:49.399 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config rm mon mon_allow_pool_delete 2020-11-28T21:53:49.521 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.516+0000 7f28e43d2700 1 Processor -- start 2020-11-28T21:53:49.522 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.516+0000 7f28e43d2700 1 -- start start 2020-11-28T21:53:49.522 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.520+0000 7f28e43d2700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28dc0b10b0 0x7f28dc060dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:49.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.520+0000 7f28e43d2700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28dc143920 con 0x7f28dc0b1990 2020-11-28T21:53:49.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.520+0000 7f28e43d2700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28dc060290 con 0x7f28dc143440 2020-11-28T21:53:49.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.520+0000 7f28e43d2700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28dc0af7e0 con 0x7f28dc0b10b0 2020-11-28T21:53:49.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.520+0000 7f28e196d700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f28dc143440 0x7f28dc13bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:49232/0 (socket says 172.21.15.162:49232) 2020-11-28T21:53:49.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.520+0000 7f28e196d700 1 -- 172.21.15.162:0/1736527121 learned_addr learned my addr 172.21.15.162:0/1736527121 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:49.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.520+0000 7f28e296f700 1 --2- 172.21.15.162:0/1736527121 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28dc0b10b0 0x7f28dc060dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:49.526 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.520+0000 7f28e116c700 1 -- 172.21.15.162:0/1736527121 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1113270024 0 0) 0x7f28dc060290 con 0x7f28dc143440 2020-11-28T21:53:49.526 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.520+0000 7f28e116c700 1 -- 172.21.15.162:0/1736527121 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28c4003230 con 0x7f28dc143440 2020-11-28T21:53:49.526 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.520+0000 7f28e116c700 1 -- 172.21.15.162:0/1736527121 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1960940649 0 0) 0x7f28dc143920 con 0x7f28dc0b1990 2020-11-28T21:53:49.527 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.520+0000 7f28e116c700 1 -- 172.21.15.162:0/1736527121 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28dc060290 con 0x7f28dc0b1990 2020-11-28T21:53:49.527 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.520+0000 7f28e296f700 1 -- 172.21.15.162:0/1736527121 >> v1:172.21.15.162:6789/0 conn(0x7f28dc143440 legacy=0x7f28dc13bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:49.527 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.520+0000 7f28e296f700 1 -- 172.21.15.162:0/1736527121 >> v1:172.21.15.118:6789/0 conn(0x7f28dc0b1990 legacy=0x7f28dc0606c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:49.527 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.520+0000 7f28e296f700 1 -- 172.21.15.162:0/1736527121 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28dc0b20e0 con 0x7f28dc0b10b0 2020-11-28T21:53:49.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.520+0000 7f28e296f700 1 --2- 172.21.15.162:0/1736527121 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28dc0b10b0 0x7f28dc060dd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f28d40032b0 tx=0x7f28d40091a0).ready entity=mon.2 client_cookie=ba120e6b069bb7fb server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:49.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.520+0000 7f28e116c700 1 -- 172.21.15.162:0/1736527121 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28d400b940 con 0x7f28dc0b10b0 2020-11-28T21:53:49.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.520+0000 7f28e116c700 1 -- 172.21.15.162:0/1736527121 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28d400baa0 con 0x7f28dc0b10b0 2020-11-28T21:53:49.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.520+0000 7f28e116c700 1 -- 172.21.15.162:0/1736527121 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28d400bc10 con 0x7f28dc0b10b0 2020-11-28T21:53:49.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.520+0000 7f28e43d2700 1 -- 172.21.15.162:0/1736527121 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28dc0b10b0 msgr2=0x7f28dc060dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:49.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.520+0000 7f28e43d2700 1 --2- 172.21.15.162:0/1736527121 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28dc0b10b0 0x7f28dc060dd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f28d40032b0 tx=0x7f28d40091a0).stop 2020-11-28T21:53:49.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.520+0000 7f28e43d2700 1 -- 172.21.15.162:0/1736527121 shutdown_connections 2020-11-28T21:53:49.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.520+0000 7f28e43d2700 1 --2- 172.21.15.162:0/1736527121 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28dc0b10b0 0x7f28dc060dd0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:49.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.520+0000 7f28e43d2700 1 -- 172.21.15.162:0/1736527121 >> 172.21.15.162:0/1736527121 conn(0x7f28dc05b970 msgr2=0x7f28dc05bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:49.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.520+0000 7f28e43d2700 1 -- 172.21.15.162:0/1736527121 shutdown_connections 2020-11-28T21:53:49.530 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.520+0000 7f28e43d2700 1 -- 172.21.15.162:0/1736527121 wait complete. 2020-11-28T21:53:49.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.524+0000 7f28e43d2700 1 Processor -- start 2020-11-28T21:53:49.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.524+0000 7f28e43d2700 1 -- start start 2020-11-28T21:53:49.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.524+0000 7f28e43d2700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28dc0b10b0 0x7f28dc145710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:49.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.524+0000 7f28e43d2700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28dc0b1990 0x7f28dc140de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:49.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.524+0000 7f28e43d2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28dc143440 0x7f28dc1412d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:49.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.524+0000 7f28e43d2700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28dc152980 con 0x7f28dc0b10b0 2020-11-28T21:53:49.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.524+0000 7f28e43d2700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28dc1417c0 con 0x7f28dc143440 2020-11-28T21:53:49.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.524+0000 7f28e43d2700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28dc152bb0 con 0x7f28dc0b1990 2020-11-28T21:53:49.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.524+0000 7f28e296f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28dc143440 0x7f28dc1412d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:49.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.524+0000 7f28e296f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28dc143440 0x7f28dc1412d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42396/0 (socket says 172.21.15.162:42396) 2020-11-28T21:53:49.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.524+0000 7f28e296f700 1 -- 172.21.15.162:0/466233838 learned_addr learned my addr 172.21.15.162:0/466233838 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:49.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.524+0000 7f28e216e700 1 --2- 172.21.15.162:0/466233838 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28dc0b10b0 0x7f28dc145710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:49.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.524+0000 7f28e196d700 1 --2- 172.21.15.162:0/466233838 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28dc0b1990 0x7f28dc140de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:49.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.524+0000 7f28e296f700 1 -- 172.21.15.162:0/466233838 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28dc0b1990 msgr2=0x7f28dc140de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:49.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.524+0000 7f28e296f700 1 --2- 172.21.15.162:0/466233838 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28dc0b1990 0x7f28dc140de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:49.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.524+0000 7f28e296f700 1 -- 172.21.15.162:0/466233838 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28dc0b10b0 msgr2=0x7f28dc145710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:49.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.524+0000 7f28e296f700 1 --2- 172.21.15.162:0/466233838 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28dc0b10b0 0x7f28dc145710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:49.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.524+0000 7f28e296f700 1 -- 172.21.15.162:0/466233838 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28dc152e50 con 0x7f28dc143440 2020-11-28T21:53:49.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.524+0000 7f28e296f700 1 --2- 172.21.15.162:0/466233838 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28dc143440 0x7f28dc1412d0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 rx=0x7f28d40032b0 tx=0x7f28d4016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:49.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.524+0000 7f28cb7fe700 1 -- 172.21.15.162:0/466233838 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28d4011bd0 con 0x7f28dc143440 2020-11-28T21:53:49.537 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.524+0000 7f28cb7fe700 1 -- 172.21.15.162:0/466233838 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28d4011d30 con 0x7f28dc143440 2020-11-28T21:53:49.537 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.524+0000 7f28e43d2700 1 -- 172.21.15.162:0/466233838 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28dc140940 con 0x7f28dc143440 2020-11-28T21:53:49.537 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.524+0000 7f28cb7fe700 1 -- 172.21.15.162:0/466233838 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28d4011ea0 con 0x7f28dc143440 2020-11-28T21:53:49.537 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.524+0000 7f28e43d2700 1 -- 172.21.15.162:0/466233838 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28dc0abb00 con 0x7f28dc143440 2020-11-28T21:53:49.538 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.528+0000 7f28cb7fe700 1 -- 172.21.15.162:0/466233838 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f28d4026030 con 0x7f28dc143440 2020-11-28T21:53:49.538 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.528+0000 7f28e43d2700 1 -- 172.21.15.162:0/466233838 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28dc057050 con 0x7f28dc143440 2020-11-28T21:53:49.538 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.528+0000 7f28cb7fe700 1 --2- 172.21.15.162:0/466233838 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f28bc055330 0x7f28bc0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:49.538 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.528+0000 7f28cb7fe700 1 -- 172.21.15.162:0/466233838 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f28d406e980 con 0x7f28dc143440 2020-11-28T21:53:49.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.528+0000 7f28e216e700 1 --2- 172.21.15.162:0/466233838 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f28bc055330 0x7f28bc0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:49.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.528+0000 7f28e216e700 1 --2- 172.21.15.162:0/466233838 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f28bc055330 0x7f28bc0577f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f28d800d000 tx=0x7f28d8000b80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:49.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.532+0000 7f28cb7fe700 1 -- 172.21.15.162:0/466233838 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f28d4044c80 con 0x7f28dc143440 2020-11-28T21:53:49.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.672+0000 7f28e43d2700 1 -- 172.21.15.162:0/466233838 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f28dc140ad0 con 0x7f28dc143440 2020-11-28T21:53:49.684 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.680+0000 7f28cb7fe700 1 -- 172.21.15.162:0/466233838 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f28d40240a0 con 0x7f28dc143440 2020-11-28T21:53:49.687 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.684+0000 7f28e43d2700 1 -- 172.21.15.162:0/466233838 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f28bc055330 msgr2=0x7f28bc0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:49.687 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.684+0000 7f28e43d2700 1 --2- 172.21.15.162:0/466233838 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f28bc055330 0x7f28bc0577f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f28d800d000 tx=0x7f28d8000b80).stop 2020-11-28T21:53:49.687 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.684+0000 7f28e43d2700 1 -- 172.21.15.162:0/466233838 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28dc143440 msgr2=0x7f28dc1412d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:49.687 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.684+0000 7f28e43d2700 1 --2- 172.21.15.162:0/466233838 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28dc143440 0x7f28dc1412d0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 rx=0x7f28d40032b0 tx=0x7f28d4016000).stop 2020-11-28T21:53:49.688 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.684+0000 7f28e43d2700 1 -- 172.21.15.162:0/466233838 shutdown_connections 2020-11-28T21:53:49.688 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.684+0000 7f28e43d2700 1 --2- 172.21.15.162:0/466233838 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f28bc055330 0x7f28bc0577f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:49.688 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.684+0000 7f28e43d2700 1 --2- 172.21.15.162:0/466233838 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28dc143440 0x7f28dc1412d0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:49.688 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.684+0000 7f28e43d2700 1 --2- 172.21.15.162:0/466233838 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28dc0b10b0 0x7f28dc145710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:49.688 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.684+0000 7f28e43d2700 1 --2- 172.21.15.162:0/466233838 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28dc0b1990 0x7f28dc140de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:49.689 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.684+0000 7f28e43d2700 1 -- 172.21.15.162:0/466233838 >> 172.21.15.162:0/466233838 conn(0x7f28dc05b970 msgr2=0x7f28dc0b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:49.689 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.684+0000 7f28e43d2700 1 -- 172.21.15.162:0/466233838 shutdown_connections 2020-11-28T21:53:49.689 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.684+0000 7f28e43d2700 1 -- 172.21.15.162:0/466233838 wait complete. 2020-11-28T21:53:49.701 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode' 2020-11-28T21:53:49.811 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.808+0000 7fa7ea13f700 1 Processor -- start 2020-11-28T21:53:49.812 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.808+0000 7fa7ea13f700 1 -- start start 2020-11-28T21:53:49.812 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.808+0000 7fa7ea13f700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fa7e40603d0 0x7fa7e40b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:49.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.808+0000 7fa7ea13f700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa7e4142a00 0x7fa7e409cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:49.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.808+0000 7fa7ea13f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa7e40b4c70 0x7fa7e409d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:49.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.808+0000 7fa7ea13f700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7e4060f60 con 0x7fa7e40603d0 2020-11-28T21:53:49.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.808+0000 7fa7ea13f700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7e40b4110 con 0x7fa7e4142a00 2020-11-28T21:53:49.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.808+0000 7fa7ea13f700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e40b43f0 con 0x7fa7e40b4c70 2020-11-28T21:53:49.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.808+0000 7fa7e893c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa7e4142a00 0x7fa7e409cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:49.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.808+0000 7fa7e893c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa7e4142a00 0x7fa7e409cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42404/0 (socket says 172.21.15.162:42404) 2020-11-28T21:53:49.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.808+0000 7fa7e993e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa7e40b4c70 0x7fa7e409d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:49.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.808+0000 7fa7e893c700 1 -- 172.21.15.162:0/668227468 learned_addr learned my addr 172.21.15.162:0/668227468 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:49.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.808+0000 7fa7e913d700 1 --2- 172.21.15.162:0/668227468 >> v2:172.21.15.118:3300/0 conn(0x7fa7e40603d0 0x7fa7e40b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:49.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.808+0000 7fa7e893c700 1 -- 172.21.15.162:0/668227468 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa7e40b4c70 msgr2=0x7fa7e409d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:49.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.808+0000 7fa7e893c700 1 --2- 172.21.15.162:0/668227468 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa7e40b4c70 0x7fa7e409d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:49.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.808+0000 7fa7e893c700 1 -- 172.21.15.162:0/668227468 >> v2:172.21.15.118:3300/0 conn(0x7fa7e40603d0 msgr2=0x7fa7e40b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:49.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.808+0000 7fa7e893c700 1 --2- 172.21.15.162:0/668227468 >> v2:172.21.15.118:3300/0 conn(0x7fa7e40603d0 0x7fa7e40b3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:49.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.808+0000 7fa7e893c700 1 -- 172.21.15.162:0/668227468 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e40b4690 con 0x7fa7e4142a00 2020-11-28T21:53:49.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.808+0000 7fa7e893c700 1 --2- 172.21.15.162:0/668227468 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7e4142a00 0x7fa7e409cfc0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 rx=0x7fa7d8002f30 tx=0x7fa7d8006e80).ready entity=mon.1 client_cookie=63d20ad95645dc5c server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:49.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.808+0000 7fa7e3fff700 1 -- 172.21.15.162:0/668227468 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7d8009640 con 0x7fa7e4142a00 2020-11-28T21:53:49.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.808+0000 7fa7e3fff700 1 -- 172.21.15.162:0/668227468 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d80097a0 con 0x7fa7e4142a00 2020-11-28T21:53:49.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.808+0000 7fa7e3fff700 1 -- 172.21.15.162:0/668227468 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7d8009b00 con 0x7fa7e4142a00 2020-11-28T21:53:49.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.808+0000 7fa7ea13f700 1 -- 172.21.15.162:0/668227468 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7e4142a00 msgr2=0x7fa7e409cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:49.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.808+0000 7fa7ea13f700 1 --2- 172.21.15.162:0/668227468 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7e4142a00 0x7fa7e409cfc0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 rx=0x7fa7d8002f30 tx=0x7fa7d8006e80).stop 2020-11-28T21:53:49.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.812+0000 7fa7ea13f700 1 -- 172.21.15.162:0/668227468 shutdown_connections 2020-11-28T21:53:49.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.812+0000 7fa7ea13f700 1 --2- 172.21.15.162:0/668227468 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa7e40b4c70 0x7fa7e409d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:49.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.812+0000 7fa7ea13f700 1 --2- 172.21.15.162:0/668227468 >> v2:172.21.15.118:3300/0 conn(0x7fa7e40603d0 0x7fa7e40b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:49.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.812+0000 7fa7ea13f700 1 --2- 172.21.15.162:0/668227468 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7e4142a00 0x7fa7e409cfc0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:49.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.812+0000 7fa7ea13f700 1 -- 172.21.15.162:0/668227468 >> 172.21.15.162:0/668227468 conn(0x7fa7e4066a40 msgr2=0x7fa7e4066e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:49.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.812+0000 7fa7ea13f700 1 -- 172.21.15.162:0/668227468 shutdown_connections 2020-11-28T21:53:49.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.812+0000 7fa7ea13f700 1 -- 172.21.15.162:0/668227468 wait complete. 2020-11-28T21:53:49.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.812+0000 7fa7ea13f700 1 Processor -- start 2020-11-28T21:53:49.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.812+0000 7fa7ea13f700 1 -- start start 2020-11-28T21:53:49.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.812+0000 7fa7ea13f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa7e40603d0 0x7fa7e41469f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:49.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.812+0000 7fa7ea13f700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa7e40b4c70 0x7fa7e40b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:49.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.812+0000 7fa7ea13f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7e40b0420 0x7fa7e40b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:49.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.812+0000 7fa7ea13f700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e40b2220 con 0x7fa7e40b4c70 2020-11-28T21:53:49.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.812+0000 7fa7ea13f700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e4151f50 con 0x7fa7e40b0420 2020-11-28T21:53:49.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.812+0000 7fa7ea13f700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e4144d10 con 0x7fa7e40603d0 2020-11-28T21:53:49.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.812+0000 7fa7e993e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7e40b0420 0x7fa7e40b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:49.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.816+0000 7fa7e993e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7e40b0420 0x7fa7e40b19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42408/0 (socket says 172.21.15.162:42408) 2020-11-28T21:53:49.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.816+0000 7fa7e993e700 1 -- 172.21.15.162:0/3081299886 learned_addr learned my addr 172.21.15.162:0/3081299886 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:49.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.816+0000 7fa7e893c700 1 --2- 172.21.15.162:0/3081299886 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa7e40b4c70 0x7fa7e40b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:49.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.816+0000 7fa7e913d700 1 --2- 172.21.15.162:0/3081299886 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa7e40603d0 0x7fa7e41469f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:49.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.816+0000 7fa7e993e700 1 -- 172.21.15.162:0/3081299886 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa7e40603d0 msgr2=0x7fa7e41469f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:49.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.816+0000 7fa7e993e700 1 --2- 172.21.15.162:0/3081299886 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa7e40603d0 0x7fa7e41469f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:49.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.816+0000 7fa7e993e700 1 -- 172.21.15.162:0/3081299886 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa7e40b4c70 msgr2=0x7fa7e40b14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:49.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.816+0000 7fa7e993e700 1 --2- 172.21.15.162:0/3081299886 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa7e40b4c70 0x7fa7e40b14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:49.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.816+0000 7fa7e993e700 1 -- 172.21.15.162:0/3081299886 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e41521b0 con 0x7fa7e40b0420 2020-11-28T21:53:49.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.816+0000 7fa7e893c700 1 --2- 172.21.15.162:0/3081299886 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa7e40b4c70 0x7fa7e40b14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:53:49.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.816+0000 7fa7e993e700 1 --2- 172.21.15.162:0/3081299886 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7e40b0420 0x7fa7e40b19e0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 rx=0x7fa7dc006dd0 tx=0x7fa7dc0091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:49.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.816+0000 7fa7e27fc700 1 -- 172.21.15.162:0/3081299886 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7dc00ca00 con 0x7fa7e40b0420 2020-11-28T21:53:49.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.816+0000 7fa7e27fc700 1 -- 172.21.15.162:0/3081299886 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7dc00cb60 con 0x7fa7e40b0420 2020-11-28T21:53:49.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.816+0000 7fa7ea13f700 1 -- 172.21.15.162:0/3081299886 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e40b2480 con 0x7fa7e40b0420 2020-11-28T21:53:49.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.816+0000 7fa7e27fc700 1 -- 172.21.15.162:0/3081299886 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7dc0123f0 con 0x7fa7e40b0420 2020-11-28T21:53:49.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.816+0000 7fa7ea13f700 1 -- 172.21.15.162:0/3081299886 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e40b09a0 con 0x7fa7e40b0420 2020-11-28T21:53:49.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.816+0000 7fa7ea13f700 1 -- 172.21.15.162:0/3081299886 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7e4152340 con 0x7fa7e40b0420 2020-11-28T21:53:49.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.816+0000 7fa7e27fc700 1 -- 172.21.15.162:0/3081299886 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa7dc022070 con 0x7fa7e40b0420 2020-11-28T21:53:49.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.820+0000 7fa7e27fc700 1 --2- 172.21.15.162:0/3081299886 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa7c4055350 0x7fa7c4057810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:49.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.820+0000 7fa7e27fc700 1 -- 172.21.15.162:0/3081299886 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa7dc06e6f0 con 0x7fa7e40b0420 2020-11-28T21:53:49.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.820+0000 7fa7e913d700 1 --2- 172.21.15.162:0/3081299886 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa7c4055350 0x7fa7c4057810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:49.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.820+0000 7fa7e913d700 1 --2- 172.21.15.162:0/3081299886 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa7c4055350 0x7fa7c4057810 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7fa7d400b000 tx=0x7fa7d4006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:49.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:49.824+0000 7fa7e27fc700 1 -- 172.21.15.162:0/3081299886 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa7dc044950 con 0x7fa7e40b0420 2020-11-28T21:53:50.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.020+0000 7fa7ea13f700 1 -- 172.21.15.162:0/3081299886 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode"]} v 0) v1 -- 0x7fa7e4057050 con 0x7fa7e40b0420 2020-11-28T21:53:50.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.804+0000 7fa7e27fc700 1 -- 172.21.15.162:0/3081299886 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode"]}]=0 v316) v1 ==== 126+0+0 (secure 0 0 0) 0x7fa7dc016020 con 0x7fa7e40b0420 2020-11-28T21:53:50.812 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.808+0000 7fa7ea13f700 1 -- 172.21.15.162:0/3081299886 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa7c4055350 msgr2=0x7fa7c4057810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:50.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.808+0000 7fa7ea13f700 1 --2- 172.21.15.162:0/3081299886 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa7c4055350 0x7fa7c4057810 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7fa7d400b000 tx=0x7fa7d4006dc0).stop 2020-11-28T21:53:50.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.808+0000 7fa7ea13f700 1 -- 172.21.15.162:0/3081299886 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7e40b0420 msgr2=0x7fa7e40b19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:50.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.808+0000 7fa7ea13f700 1 --2- 172.21.15.162:0/3081299886 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7e40b0420 0x7fa7e40b19e0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 rx=0x7fa7dc006dd0 tx=0x7fa7dc0091a0).stop 2020-11-28T21:53:50.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.808+0000 7fa7ea13f700 1 -- 172.21.15.162:0/3081299886 shutdown_connections 2020-11-28T21:53:50.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.808+0000 7fa7ea13f700 1 --2- 172.21.15.162:0/3081299886 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa7c4055350 0x7fa7c4057810 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:50.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.808+0000 7fa7ea13f700 1 --2- 172.21.15.162:0/3081299886 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa7e40603d0 0x7fa7e41469f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:50.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.808+0000 7fa7ea13f700 1 --2- 172.21.15.162:0/3081299886 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7e40b0420 0x7fa7e40b19e0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:50.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.808+0000 7fa7ea13f700 1 --2- 172.21.15.162:0/3081299886 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa7e40b4c70 0x7fa7e40b14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:50.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.808+0000 7fa7ea13f700 1 -- 172.21.15.162:0/3081299886 >> 172.21.15.162:0/3081299886 conn(0x7fa7e4066a40 msgr2=0x7fa7e40b6240 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:50.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.808+0000 7fa7ea13f700 1 -- 172.21.15.162:0/3081299886 shutdown_connections 2020-11-28T21:53:50.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.808+0000 7fa7ea13f700 1 -- 172.21.15.162:0/3081299886 wait complete. 2020-11-28T21:53:50.830 INFO:tasks.cephfs_test_runner:test_dump_inode (tasks.cephfs.test_misc.TestMisc) ... ok 2020-11-28T21:53:50.830 INFO:tasks.cephfs_test_runner:Starting test: test_dump_inode_hexademical (tasks.cephfs.test_misc.TestMisc) 2020-11-28T21:53:50.831 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical' 2020-11-28T21:53:50.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.948+0000 7fc9354d1700 1 Processor -- start 2020-11-28T21:53:50.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.948+0000 7fc9354d1700 1 -- start start 2020-11-28T21:53:50.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.948+0000 7fc9354d1700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc93005cbd0 0x7fc9300aac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:50.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.948+0000 7fc9354d1700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fc9300af640 0x7fc9300ae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:50.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.948+0000 7fc9354d1700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9300b0230 con 0x7fc930060cf0 2020-11-28T21:53:50.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.948+0000 7fc9354d1700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9300b12a0 con 0x7fc9300af640 2020-11-28T21:53:50.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.948+0000 7fc9354d1700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9300610d0 con 0x7fc93005cbd0 2020-11-28T21:53:50.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.948+0000 7fc92f7fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc93005cbd0 0x7fc9300aac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:50.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.948+0000 7fc92f7fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc93005cbd0 0x7fc9300aac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42414/0 (socket says 172.21.15.162:42414) 2020-11-28T21:53:50.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.948+0000 7fc92f7fe700 1 -- 172.21.15.162:0/1900396900 learned_addr learned my addr 172.21.15.162:0/1900396900 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:50.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.952+0000 7fc934cd0700 1 --2- 172.21.15.162:0/1900396900 >> v2:172.21.15.118:3300/0 conn(0x7fc9300af640 0x7fc9300ae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:50.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.952+0000 7fc92f7fe700 1 -- 172.21.15.162:0/1900396900 >> v2:172.21.15.118:3300/0 conn(0x7fc9300af640 msgr2=0x7fc9300ae1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:50.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.952+0000 7fc92f7fe700 1 --2- 172.21.15.162:0/1900396900 >> v2:172.21.15.118:3300/0 conn(0x7fc9300af640 0x7fc9300ae1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:50.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.952+0000 7fc92f7fe700 1 -- 172.21.15.162:0/1900396900 >> v1:172.21.15.118:6789/0 conn(0x7fc930060cf0 legacy=0x7fc9300aa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:50.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.952+0000 7fc92f7fe700 1 -- 172.21.15.162:0/1900396900 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9300ab0f0 con 0x7fc93005cbd0 2020-11-28T21:53:50.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.952+0000 7fc92f7fe700 1 --2- 172.21.15.162:0/1900396900 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc93005cbd0 0x7fc9300aac00 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 rx=0x7fc9300ab380 tx=0x7fc924006e80).ready entity=mon.1 client_cookie=2e54dba2d0eea63 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:50.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.952+0000 7fc92effd700 1 -- 172.21.15.162:0/1900396900 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9240097b0 con 0x7fc93005cbd0 2020-11-28T21:53:50.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.952+0000 7fc92effd700 1 -- 172.21.15.162:0/1900396900 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc924009910 con 0x7fc93005cbd0 2020-11-28T21:53:50.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.952+0000 7fc92effd700 1 -- 172.21.15.162:0/1900396900 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc924009a80 con 0x7fc93005cbd0 2020-11-28T21:53:50.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.952+0000 7fc9354d1700 1 -- 172.21.15.162:0/1900396900 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc93005cbd0 msgr2=0x7fc9300aac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:50.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.952+0000 7fc9354d1700 1 --2- 172.21.15.162:0/1900396900 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc93005cbd0 0x7fc9300aac00 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 rx=0x7fc9300ab380 tx=0x7fc924006e80).stop 2020-11-28T21:53:50.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.952+0000 7fc9354d1700 1 -- 172.21.15.162:0/1900396900 shutdown_connections 2020-11-28T21:53:50.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.952+0000 7fc9354d1700 1 --2- 172.21.15.162:0/1900396900 >> v2:172.21.15.118:3300/0 conn(0x7fc9300af640 0x7fc9300ae1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:50.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.952+0000 7fc9354d1700 1 --2- 172.21.15.162:0/1900396900 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc93005cbd0 0x7fc9300aac00 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:50.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.952+0000 7fc9354d1700 1 -- 172.21.15.162:0/1900396900 >> 172.21.15.162:0/1900396900 conn(0x7fc93005b970 msgr2=0x7fc93005bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:50.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.952+0000 7fc9354d1700 1 -- 172.21.15.162:0/1900396900 shutdown_connections 2020-11-28T21:53:50.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.952+0000 7fc9354d1700 1 -- 172.21.15.162:0/1900396900 wait complete. 2020-11-28T21:53:50.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.952+0000 7fc9354d1700 1 Processor -- start 2020-11-28T21:53:50.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.952+0000 7fc9354d1700 1 -- start start 2020-11-28T21:53:50.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.956+0000 7fc9354d1700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc93005cbd0 0x7fc93008fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:50.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.956+0000 7fc9354d1700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc930060cf0 0x7fc93013f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:50.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.956+0000 7fc9354d1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc9300af640 0x7fc93013fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:50.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.956+0000 7fc9354d1700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9301420d0 con 0x7fc930060cf0 2020-11-28T21:53:50.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.956+0000 7fc9354d1700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc930064c40 con 0x7fc9300af640 2020-11-28T21:53:50.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.956+0000 7fc9354d1700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9301403b0 con 0x7fc93005cbd0 2020-11-28T21:53:50.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.956+0000 7fc934cd0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc9300af640 0x7fc93013fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:50.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.956+0000 7fc934cd0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc9300af640 0x7fc93013fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42424/0 (socket says 172.21.15.162:42424) 2020-11-28T21:53:50.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.956+0000 7fc934cd0700 1 -- 172.21.15.162:0/4168070525 learned_addr learned my addr 172.21.15.162:0/4168070525 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:50.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.956+0000 7fc92f7fe700 1 --2- 172.21.15.162:0/4168070525 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc930060cf0 0x7fc93013f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:50.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.956+0000 7fc92ffff700 1 --2- 172.21.15.162:0/4168070525 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc93005cbd0 0x7fc93008fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:50.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.956+0000 7fc934cd0700 1 -- 172.21.15.162:0/4168070525 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc93005cbd0 msgr2=0x7fc93008fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:50.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.956+0000 7fc934cd0700 1 --2- 172.21.15.162:0/4168070525 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc93005cbd0 0x7fc93008fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:50.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.956+0000 7fc934cd0700 1 -- 172.21.15.162:0/4168070525 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc930060cf0 msgr2=0x7fc93013f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:50.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.956+0000 7fc934cd0700 1 --2- 172.21.15.162:0/4168070525 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc930060cf0 0x7fc93013f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:50.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.956+0000 7fc934cd0700 1 -- 172.21.15.162:0/4168070525 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9300a8c00 con 0x7fc9300af640 2020-11-28T21:53:50.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.956+0000 7fc934cd0700 1 --2- 172.21.15.162:0/4168070525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc9300af640 0x7fc93013fec0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 rx=0x7fc92800a000 tx=0x7fc928008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:50.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.956+0000 7fc92d7fa700 1 -- 172.21.15.162:0/4168070525 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc92800b940 con 0x7fc9300af640 2020-11-28T21:53:50.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.956+0000 7fc92d7fa700 1 -- 172.21.15.162:0/4168070525 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc92800baa0 con 0x7fc9300af640 2020-11-28T21:53:50.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.956+0000 7fc92d7fa700 1 -- 172.21.15.162:0/4168070525 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc92800be00 con 0x7fc9300af640 2020-11-28T21:53:50.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.956+0000 7fc9354d1700 1 -- 172.21.15.162:0/4168070525 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9301422b0 con 0x7fc9300af640 2020-11-28T21:53:50.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.956+0000 7fc9354d1700 1 -- 172.21.15.162:0/4168070525 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9300650d0 con 0x7fc9300af640 2020-11-28T21:53:50.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.956+0000 7fc9354d1700 1 -- 172.21.15.162:0/4168070525 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc930057050 con 0x7fc9300af640 2020-11-28T21:53:50.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.960+0000 7fc92d7fa700 1 -- 172.21.15.162:0/4168070525 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc92801c460 con 0x7fc9300af640 2020-11-28T21:53:50.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.960+0000 7fc92d7fa700 1 --2- 172.21.15.162:0/4168070525 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc9100553b0 0x7fc910057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:50.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.960+0000 7fc92ffff700 1 --2- 172.21.15.162:0/4168070525 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc9100553b0 0x7fc910057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:50.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.960+0000 7fc92d7fa700 1 -- 172.21.15.162:0/4168070525 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc92806e610 con 0x7fc9300af640 2020-11-28T21:53:50.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.960+0000 7fc92ffff700 1 --2- 172.21.15.162:0/4168070525 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc9100553b0 0x7fc910057870 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7fc92000c000 tx=0x7fc920008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:50.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:50.964+0000 7fc92d7fa700 1 -- 172.21.15.162:0/4168070525 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc9280447a0 con 0x7fc9300af640 2020-11-28T21:53:51.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.152+0000 7fc9354d1700 1 -- 172.21.15.162:0/4168070525 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical"]} v 0) v1 -- 0x7fc9300b0010 con 0x7fc9300af640 2020-11-28T21:53:51.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.812+0000 7fc92d7fa700 1 -- 172.21.15.162:0/4168070525 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical"]}]=0 v317) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc928047dc0 con 0x7fc9300af640 2020-11-28T21:53:51.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.816+0000 7fc9354d1700 1 -- 172.21.15.162:0/4168070525 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc9100553b0 msgr2=0x7fc910057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:51.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.816+0000 7fc9354d1700 1 --2- 172.21.15.162:0/4168070525 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc9100553b0 0x7fc910057870 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7fc92000c000 tx=0x7fc920008040).stop 2020-11-28T21:53:51.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.816+0000 7fc9354d1700 1 -- 172.21.15.162:0/4168070525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc9300af640 msgr2=0x7fc93013fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:51.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.816+0000 7fc9354d1700 1 --2- 172.21.15.162:0/4168070525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc9300af640 0x7fc93013fec0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 rx=0x7fc92800a000 tx=0x7fc928008040).stop 2020-11-28T21:53:51.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.816+0000 7fc9354d1700 1 -- 172.21.15.162:0/4168070525 shutdown_connections 2020-11-28T21:53:51.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.816+0000 7fc9354d1700 1 --2- 172.21.15.162:0/4168070525 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc9100553b0 0x7fc910057870 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:51.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.816+0000 7fc9354d1700 1 --2- 172.21.15.162:0/4168070525 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc93005cbd0 0x7fc93008fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:51.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.816+0000 7fc9354d1700 1 --2- 172.21.15.162:0/4168070525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc9300af640 0x7fc93013fec0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:51.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.816+0000 7fc9354d1700 1 --2- 172.21.15.162:0/4168070525 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc930060cf0 0x7fc93013f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:51.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.816+0000 7fc9354d1700 1 -- 172.21.15.162:0/4168070525 >> 172.21.15.162:0/4168070525 conn(0x7fc93005b970 msgr2=0x7fc9300b2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:51.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.816+0000 7fc9354d1700 1 -- 172.21.15.162:0/4168070525 shutdown_connections 2020-11-28T21:53:51.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.816+0000 7fc9354d1700 1 -- 172.21.15.162:0/4168070525 wait complete. 2020-11-28T21:53:51.838 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config set mon mon_allow_pool_delete True 2020-11-28T21:53:51.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.960+0000 7fa070a93700 1 Processor -- start 2020-11-28T21:53:51.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.960+0000 7fa070a93700 1 -- start start 2020-11-28T21:53:51.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.960+0000 7fa070a93700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fa06c1433c0 0x7fa06c0605c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:51.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.960+0000 7fa070a93700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa06c067610 0x7fa06c13fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:51.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.960+0000 7fa070a93700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa06c143950 con 0x7fa06c0b9ce0 2020-11-28T21:53:51.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.960+0000 7fa070a93700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa06c0b4340 con 0x7fa06c1433c0 2020-11-28T21:53:51.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.960+0000 7fa070a93700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa06c067ac0 con 0x7fa06c067610 2020-11-28T21:53:51.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.960+0000 7fa06ad9d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa06c067610 0x7fa06c13fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:51.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.960+0000 7fa06a59c700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fa06c0b9ce0 0x7fa06c0b3c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:49272/0 (socket says 172.21.15.162:49272) 2020-11-28T21:53:51.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.960+0000 7fa06a59c700 1 -- 172.21.15.162:0/787399892 learned_addr learned my addr 172.21.15.162:0/787399892 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:51.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.960+0000 7fa069d9b700 1 --2- 172.21.15.162:0/787399892 >> v2:172.21.15.118:3300/0 conn(0x7fa06c1433c0 0x7fa06c0605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:51.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.960+0000 7fa06ad9d700 1 -- 172.21.15.162:0/787399892 >> v2:172.21.15.118:3300/0 conn(0x7fa06c1433c0 msgr2=0x7fa06c0605c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:51.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.960+0000 7fa06ad9d700 1 --2- 172.21.15.162:0/787399892 >> v2:172.21.15.118:3300/0 conn(0x7fa06c1433c0 0x7fa06c0605c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:51.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.960+0000 7fa06ad9d700 1 -- 172.21.15.162:0/787399892 >> v1:172.21.15.162:6789/0 conn(0x7fa06c0b9ce0 legacy=0x7fa06c0b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:51.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.960+0000 7fa06959a700 1 -- 172.21.15.162:0/787399892 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2208753368 0 0) 0x7fa06c143950 con 0x7fa06c0b9ce0 2020-11-28T21:53:51.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.960+0000 7fa06ad9d700 1 -- 172.21.15.162:0/787399892 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa06c0ba0c0 con 0x7fa06c067610 2020-11-28T21:53:51.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.960+0000 7fa06ad9d700 1 --2- 172.21.15.162:0/787399892 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa06c067610 0x7fa06c13fc80 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 rx=0x7fa0640032b0 tx=0x7fa0640091a0).ready entity=mon.1 client_cookie=7680c356252ef91 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:51.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.960+0000 7fa06959a700 1 -- 172.21.15.162:0/787399892 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa06400ba50 con 0x7fa06c067610 2020-11-28T21:53:51.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.960+0000 7fa06959a700 1 -- 172.21.15.162:0/787399892 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa06400bbb0 con 0x7fa06c067610 2020-11-28T21:53:51.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.960+0000 7fa06959a700 1 -- 172.21.15.162:0/787399892 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa06400bd20 con 0x7fa06c067610 2020-11-28T21:53:51.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.960+0000 7fa070a93700 1 -- 172.21.15.162:0/787399892 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa06c067610 msgr2=0x7fa06c13fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:51.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.960+0000 7fa070a93700 1 --2- 172.21.15.162:0/787399892 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa06c067610 0x7fa06c13fc80 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 rx=0x7fa0640032b0 tx=0x7fa0640091a0).stop 2020-11-28T21:53:51.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.960+0000 7fa070a93700 1 -- 172.21.15.162:0/787399892 shutdown_connections 2020-11-28T21:53:51.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.960+0000 7fa070a93700 1 --2- 172.21.15.162:0/787399892 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa06c067610 0x7fa06c13fc80 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:51.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.960+0000 7fa070a93700 1 --2- 172.21.15.162:0/787399892 >> v2:172.21.15.118:3300/0 conn(0x7fa06c1433c0 0x7fa06c0605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:51.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.960+0000 7fa070a93700 1 -- 172.21.15.162:0/787399892 >> 172.21.15.162:0/787399892 conn(0x7fa06c05c430 msgr2=0x7fa06c0b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:51.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.960+0000 7fa070a93700 1 -- 172.21.15.162:0/787399892 shutdown_connections 2020-11-28T21:53:51.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.960+0000 7fa070a93700 1 -- 172.21.15.162:0/787399892 wait complete. 2020-11-28T21:53:51.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.964+0000 7fa070a93700 1 Processor -- start 2020-11-28T21:53:51.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.964+0000 7fa070a93700 1 -- start start 2020-11-28T21:53:51.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.964+0000 7fa070a93700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa06c067610 0x7fa06c0b2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:51.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.964+0000 7fa070a93700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa06c0b9ce0 0x7fa06c0b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:51.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.964+0000 7fa070a93700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa06c1433c0 0x7fa06c144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:51.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.964+0000 7fa070a93700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c0b8c70 con 0x7fa06c0b9ce0 2020-11-28T21:53:51.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.964+0000 7fa070a93700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c09c400 con 0x7fa06c1433c0 2020-11-28T21:53:51.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.964+0000 7fa070a93700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c146bc0 con 0x7fa06c067610 2020-11-28T21:53:51.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.964+0000 7fa06ad9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa06c1433c0 0x7fa06c144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:51.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.964+0000 7fa06ad9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa06c1433c0 0x7fa06c144cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42438/0 (socket says 172.21.15.162:42438) 2020-11-28T21:53:51.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.964+0000 7fa06ad9d700 1 -- 172.21.15.162:0/728084571 learned_addr learned my addr 172.21.15.162:0/728084571 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:51.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.964+0000 7fa069d9b700 1 --2- 172.21.15.162:0/728084571 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa06c0b9ce0 0x7fa06c0b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:51.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.964+0000 7fa06a59c700 1 --2- 172.21.15.162:0/728084571 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa06c067610 0x7fa06c0b2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:51.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.964+0000 7fa06ad9d700 1 -- 172.21.15.162:0/728084571 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa06c067610 msgr2=0x7fa06c0b2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:51.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.964+0000 7fa06ad9d700 1 --2- 172.21.15.162:0/728084571 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa06c067610 0x7fa06c0b2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:51.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.964+0000 7fa06ad9d700 1 -- 172.21.15.162:0/728084571 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa06c0b9ce0 msgr2=0x7fa06c0b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:51.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.964+0000 7fa06ad9d700 1 --2- 172.21.15.162:0/728084571 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa06c0b9ce0 0x7fa06c0b1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:51.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.964+0000 7fa06ad9d700 1 -- 172.21.15.162:0/728084571 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa06c0b6270 con 0x7fa06c1433c0 2020-11-28T21:53:51.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.964+0000 7fa06ad9d700 1 --2- 172.21.15.162:0/728084571 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa06c1433c0 0x7fa06c144cd0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 rx=0x7fa06c144950 tx=0x7fa064016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:51.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.964+0000 7fa05b7fe700 1 -- 172.21.15.162:0/728084571 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa06401f5e0 con 0x7fa06c1433c0 2020-11-28T21:53:51.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.964+0000 7fa05b7fe700 1 -- 172.21.15.162:0/728084571 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa06401f740 con 0x7fa06c1433c0 2020-11-28T21:53:51.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.964+0000 7fa070a93700 1 -- 172.21.15.162:0/728084571 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa06c0b6b10 con 0x7fa06c1433c0 2020-11-28T21:53:51.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.964+0000 7fa05b7fe700 1 -- 172.21.15.162:0/728084571 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa06401faa0 con 0x7fa06c1433c0 2020-11-28T21:53:51.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.964+0000 7fa070a93700 1 -- 172.21.15.162:0/728084571 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa06c0b1fb0 con 0x7fa06c1433c0 2020-11-28T21:53:51.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.968+0000 7fa05b7fe700 1 -- 172.21.15.162:0/728084571 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa06400f070 con 0x7fa06c1433c0 2020-11-28T21:53:51.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.968+0000 7fa070a93700 1 -- 172.21.15.162:0/728084571 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa06c057050 con 0x7fa06c1433c0 2020-11-28T21:53:51.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.968+0000 7fa05b7fe700 1 --2- 172.21.15.162:0/728084571 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa0480554e0 0x7fa0480579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:51.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.968+0000 7fa05b7fe700 1 -- 172.21.15.162:0/728084571 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa06406efb0 con 0x7fa06c1433c0 2020-11-28T21:53:51.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.968+0000 7fa06a59c700 1 --2- 172.21.15.162:0/728084571 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa0480554e0 0x7fa0480579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:51.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.968+0000 7fa06a59c700 1 --2- 172.21.15.162:0/728084571 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa0480554e0 0x7fa0480579a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7fa05c00d000 tx=0x7fa05c001720).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:51.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:51.972+0000 7fa05b7fe700 1 -- 172.21.15.162:0/728084571 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa064048bd0 con 0x7fa06c1433c0 2020-11-28T21:53:52.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.116+0000 7fa070a93700 1 -- 172.21.15.162:0/728084571 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7fa06c068350 con 0x7fa06c1433c0 2020-11-28T21:53:52.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.128+0000 7fa05b7fe700 1 -- 172.21.15.162:0/728084571 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa0640240c0 con 0x7fa06c1433c0 2020-11-28T21:53:52.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.128+0000 7fa070a93700 1 -- 172.21.15.162:0/728084571 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa0480554e0 msgr2=0x7fa0480579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:52.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.128+0000 7fa070a93700 1 --2- 172.21.15.162:0/728084571 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa0480554e0 0x7fa0480579a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7fa05c00d000 tx=0x7fa05c001720).stop 2020-11-28T21:53:52.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.128+0000 7fa070a93700 1 -- 172.21.15.162:0/728084571 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa06c1433c0 msgr2=0x7fa06c144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:52.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.128+0000 7fa070a93700 1 --2- 172.21.15.162:0/728084571 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa06c1433c0 0x7fa06c144cd0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 rx=0x7fa06c144950 tx=0x7fa064016000).stop 2020-11-28T21:53:52.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.128+0000 7fa070a93700 1 -- 172.21.15.162:0/728084571 shutdown_connections 2020-11-28T21:53:52.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.128+0000 7fa070a93700 1 --2- 172.21.15.162:0/728084571 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa0480554e0 0x7fa0480579a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:52.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.128+0000 7fa070a93700 1 --2- 172.21.15.162:0/728084571 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa06c067610 0x7fa06c0b2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:52.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.128+0000 7fa070a93700 1 --2- 172.21.15.162:0/728084571 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa06c1433c0 0x7fa06c144cd0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:52.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.128+0000 7fa070a93700 1 --2- 172.21.15.162:0/728084571 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa06c0b9ce0 0x7fa06c0b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:52.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.128+0000 7fa070a93700 1 -- 172.21.15.162:0/728084571 >> 172.21.15.162:0/728084571 conn(0x7fa06c05c430 msgr2=0x7fa06c09d540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:52.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.128+0000 7fa070a93700 1 -- 172.21.15.162:0/728084571 shutdown_connections 2020-11-28T21:53:52.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.128+0000 7fa070a93700 1 -- 172.21.15.162:0/728084571 wait complete. 2020-11-28T21:53:52.153 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-28T21:53:52.153 INFO:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:53:52.171 INFO:teuthology.orchestra.run.smithi118:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:53:52.192 INFO:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:53:52.210 INFO:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:53:52.228 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-28T21:53:52.229 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:53:52.229 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:53:52.230 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:53:52.282 INFO:teuthology.orchestra.run.smithi192.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2020-11-28T21:53:52.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:53:52.283 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2020-11-28T21:53:52.284 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:53:52.284 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:53:52.284 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:53:52.292 INFO:teuthology.orchestra.run.smithi096.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-11-28T21:53:52.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:53:52.296 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2020-11-28T21:53:52.297 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2020-11-28T21:53:52.417 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.412+0000 7f45ca3db700 1 Processor -- start 2020-11-28T21:53:52.417 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.412+0000 7f45ca3db700 1 -- start start 2020-11-28T21:53:52.419 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.412+0000 7f45ca3db700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f45c40b1860 0x7f45c40b2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:52.419 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.412+0000 7f45ca3db700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45c413d3c0 con 0x7f45c40b2830 2020-11-28T21:53:52.420 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.412+0000 7f45ca3db700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45c413f8d0 con 0x7f45c40ad2c0 2020-11-28T21:53:52.420 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.412+0000 7f45ca3db700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45c413f6e0 con 0x7f45c40b1860 2020-11-28T21:53:52.420 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.412+0000 7f45c8bd8700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f45c40ad2c0 0x7f45c40b1150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:49288/0 (socket says 172.21.15.162:49288) 2020-11-28T21:53:52.421 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.412+0000 7f45c8bd8700 1 -- 172.21.15.162:0/1776203888 learned_addr learned my addr 172.21.15.162:0/1776203888 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:52.421 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.416+0000 7f45c9bda700 1 --2- 172.21.15.162:0/1776203888 >> v2:172.21.15.118:3300/0 conn(0x7f45c40b1860 0x7f45c40b2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:52.421 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.416+0000 7f45c3fff700 1 -- 172.21.15.162:0/1776203888 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 966085238 0 0) 0x7f45c413f8d0 con 0x7f45c40ad2c0 2020-11-28T21:53:52.422 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.416+0000 7f45c3fff700 1 -- 172.21.15.162:0/1776203888 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45b0003230 con 0x7f45c40ad2c0 2020-11-28T21:53:52.422 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.416+0000 7f45c9bda700 1 -- 172.21.15.162:0/1776203888 >> v1:172.21.15.162:6789/0 conn(0x7f45c40ad2c0 legacy=0x7f45c40b1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:52.422 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.416+0000 7f45c9bda700 1 -- 172.21.15.162:0/1776203888 >> v1:172.21.15.118:6789/0 conn(0x7f45c40b2830 legacy=0x7f45c40b1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:52.422 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.416+0000 7f45c9bda700 1 -- 172.21.15.162:0/1776203888 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c413fb70 con 0x7f45c40b1860 2020-11-28T21:53:52.423 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.416+0000 7f45c9bda700 1 --2- 172.21.15.162:0/1776203888 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f45c40b1860 0x7f45c40b2c10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f45bc002f30 tx=0x7f45bc006dc0).ready entity=mon.0 client_cookie=36f31092c3bebf28 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:52.426 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.416+0000 7f45c3fff700 1 -- 172.21.15.162:0/1776203888 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45bc009800 con 0x7f45c40b1860 2020-11-28T21:53:52.426 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.416+0000 7f45c3fff700 1 -- 172.21.15.162:0/1776203888 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45bc009960 con 0x7f45c40b1860 2020-11-28T21:53:52.427 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.416+0000 7f45c3fff700 1 -- 172.21.15.162:0/1776203888 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45bc009cc0 con 0x7f45c40b1860 2020-11-28T21:53:52.427 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.416+0000 7f45ca3db700 1 -- 172.21.15.162:0/1776203888 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f45c40b1860 msgr2=0x7f45c40b2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:52.427 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.416+0000 7f45ca3db700 1 --2- 172.21.15.162:0/1776203888 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f45c40b1860 0x7f45c40b2c10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f45bc002f30 tx=0x7f45bc006dc0).stop 2020-11-28T21:53:52.427 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.416+0000 7f45ca3db700 1 -- 172.21.15.162:0/1776203888 shutdown_connections 2020-11-28T21:53:52.427 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.416+0000 7f45ca3db700 1 --2- 172.21.15.162:0/1776203888 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f45c40b1860 0x7f45c40b2c10 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:52.428 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.416+0000 7f45ca3db700 1 -- 172.21.15.162:0/1776203888 >> 172.21.15.162:0/1776203888 conn(0x7f45c4061550 msgr2=0x7f45c4061960 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:52.428 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.416+0000 7f45ca3db700 1 -- 172.21.15.162:0/1776203888 shutdown_connections 2020-11-28T21:53:52.428 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.416+0000 7f45ca3db700 1 -- 172.21.15.162:0/1776203888 wait complete. 2020-11-28T21:53:52.428 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.416+0000 7f45ca3db700 1 Processor -- start 2020-11-28T21:53:52.428 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.420+0000 7f45ca3db700 1 -- start start 2020-11-28T21:53:52.429 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.420+0000 7f45ca3db700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45c40ad2c0 0x7f45c4066760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:52.429 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.420+0000 7f45ca3db700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f45c40b1860 0x7f45c4143950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:52.429 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.420+0000 7f45ca3db700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f45c40b2830 0x7f45c4148fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:52.429 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.420+0000 7f45ca3db700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c4064470 con 0x7f45c40b1860 2020-11-28T21:53:52.430 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.420+0000 7f45ca3db700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c41523c0 con 0x7f45c40ad2c0 2020-11-28T21:53:52.430 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.420+0000 7f45ca3db700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45c4144330 con 0x7f45c40b2830 2020-11-28T21:53:52.431 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.420+0000 7f45c93d9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45c40ad2c0 0x7f45c4066760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:52.431 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.420+0000 7f45c93d9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45c40ad2c0 0x7f45c4066760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42448/0 (socket says 172.21.15.162:42448) 2020-11-28T21:53:52.431 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.420+0000 7f45c93d9700 1 -- 172.21.15.162:0/3837711029 learned_addr learned my addr 172.21.15.162:0/3837711029 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:52.432 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.420+0000 7f45c8bd8700 1 --2- 172.21.15.162:0/3837711029 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f45c40b1860 0x7f45c4143950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:52.432 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.420+0000 7f45c9bda700 1 --2- 172.21.15.162:0/3837711029 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f45c40b2830 0x7f45c4148fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:52.432 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.420+0000 7f45c93d9700 1 -- 172.21.15.162:0/3837711029 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f45c40b2830 msgr2=0x7f45c4148fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:52.432 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.420+0000 7f45c93d9700 1 --2- 172.21.15.162:0/3837711029 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f45c40b2830 0x7f45c4148fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:52.433 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.420+0000 7f45c93d9700 1 -- 172.21.15.162:0/3837711029 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f45c40b1860 msgr2=0x7f45c4143950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:52.433 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.420+0000 7f45c93d9700 1 --2- 172.21.15.162:0/3837711029 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f45c40b1860 0x7f45c4143950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:52.433 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.420+0000 7f45c93d9700 1 -- 172.21.15.162:0/3837711029 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c41525f0 con 0x7f45c40ad2c0 2020-11-28T21:53:52.433 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.420+0000 7f45c8bd8700 1 --2- 172.21.15.162:0/3837711029 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f45c40b1860 0x7f45c4143950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:53:52.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.420+0000 7f45c93d9700 1 --2- 172.21.15.162:0/3837711029 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45c40ad2c0 0x7f45c4066760 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 rx=0x7f45c4066000 tx=0x7f45b8007c20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:52.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.420+0000 7f45c27fc700 1 -- 172.21.15.162:0/3837711029 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45b800e4a0 con 0x7f45c40ad2c0 2020-11-28T21:53:52.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.420+0000 7f45c27fc700 1 -- 172.21.15.162:0/3837711029 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b800e600 con 0x7f45c40ad2c0 2020-11-28T21:53:52.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.420+0000 7f45c27fc700 1 -- 172.21.15.162:0/3837711029 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45b800e770 con 0x7f45c40ad2c0 2020-11-28T21:53:52.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.420+0000 7f45ca3db700 1 -- 172.21.15.162:0/3837711029 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45c4152890 con 0x7f45c40ad2c0 2020-11-28T21:53:52.435 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.420+0000 7f45ca3db700 1 -- 172.21.15.162:0/3837711029 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45c4143e40 con 0x7f45c40ad2c0 2020-11-28T21:53:52.435 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.420+0000 7f45ca3db700 1 -- 172.21.15.162:0/3837711029 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45c4057050 con 0x7f45c40ad2c0 2020-11-28T21:53:52.435 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.420+0000 7f45c27fc700 1 -- 172.21.15.162:0/3837711029 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f45b8010030 con 0x7f45c40ad2c0 2020-11-28T21:53:52.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.424+0000 7f45c27fc700 1 --2- 172.21.15.162:0/3837711029 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f45a4055330 0x7f45a40577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:52.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.424+0000 7f45c8bd8700 1 --2- 172.21.15.162:0/3837711029 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f45a4055330 0x7f45a40577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:52.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.424+0000 7f45c27fc700 1 -- 172.21.15.162:0/3837711029 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f45b806dc90 con 0x7f45c40ad2c0 2020-11-28T21:53:52.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.424+0000 7f45c8bd8700 1 --2- 172.21.15.162:0/3837711029 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f45a4055330 0x7f45a40577f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7f45b400b000 tx=0x7f45b4009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:52.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.432+0000 7f45c27fc700 1 -- 172.21.15.162:0/3837711029 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f45b8043f10 con 0x7f45c40ad2c0 2020-11-28T21:53:52.578 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.572+0000 7f45ca3db700 1 -- 172.21.15.162:0/3837711029 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f45c4068350 con 0x7f45c40ad2c0 2020-11-28T21:53:52.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.576+0000 7f45c27fc700 1 -- 172.21.15.162:0/3837711029 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v88) v1 ==== 81+0+33744 (secure 0 0 0) 0x7f45b8047530 con 0x7f45c40ad2c0 2020-11-28T21:53:52.581 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:53:52.581 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2020-11-28T21:53:52.582 INFO:teuthology.orchestra.run.smithi162.stdout: "epoch": 88, 2020-11-28T21:53:52.583 INFO:teuthology.orchestra.run.smithi162.stdout: "fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", 2020-11-28T21:53:52.584 INFO:teuthology.orchestra.run.smithi162.stdout: "created": "2020-11-28T21:46:11.724372+0000", 2020-11-28T21:53:52.584 INFO:teuthology.orchestra.run.smithi162.stdout: "modified": "2020-11-28T21:53:37.541898+0000", 2020-11-28T21:53:52.585 INFO:teuthology.orchestra.run.smithi162.stdout: "last_up_change": "2020-11-28T21:46:19.591650+0000", 2020-11-28T21:53:52.585 INFO:teuthology.orchestra.run.smithi162.stdout: "last_in_change": "2020-11-28T21:46:19.591650+0000", 2020-11-28T21:53:52.585 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-11-28T21:53:52.586 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_num": 5799936, 2020-11-28T21:53:52.586 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_set": [ 2020-11-28T21:53:52.587 INFO:teuthology.orchestra.run.smithi162.stdout: "pglog_hardlimit", 2020-11-28T21:53:52.587 INFO:teuthology.orchestra.run.smithi162.stdout: "purged_snapdirs", 2020-11-28T21:53:52.587 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_deletes", 2020-11-28T21:53:52.588 INFO:teuthology.orchestra.run.smithi162.stdout: "sortbitwise" 2020-11-28T21:53:52.588 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:53:52.588 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_version": 4, 2020-11-28T21:53:52.590 INFO:teuthology.orchestra.run.smithi162.stdout: "full_ratio": 0.94999998807907104, 2020-11-28T21:53:52.590 INFO:teuthology.orchestra.run.smithi162.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-11-28T21:53:52.591 INFO:teuthology.orchestra.run.smithi162.stdout: "nearfull_ratio": 0.85000002384185791, 2020-11-28T21:53:52.592 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_snapshot": "", 2020-11-28T21:53:52.592 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_max": 16, 2020-11-28T21:53:52.593 INFO:teuthology.orchestra.run.smithi162.stdout: "max_osd": 8, 2020-11-28T21:53:52.593 INFO:teuthology.orchestra.run.smithi162.stdout: "require_min_compat_client": "jewel", 2020-11-28T21:53:52.594 INFO:teuthology.orchestra.run.smithi162.stdout: "min_compat_client": "jewel", 2020-11-28T21:53:52.594 INFO:teuthology.orchestra.run.smithi162.stdout: "require_osd_release": "octopus", 2020-11-28T21:53:52.595 INFO:teuthology.orchestra.run.smithi162.stdout: "pools": [ 2020-11-28T21:53:52.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.595 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 1, 2020-11-28T21:53:52.596 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "device_health_metrics", 2020-11-28T21:53:52.596 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:46:15.639551+0000", 2020-11-28T21:53:52.596 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:53:52.597 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:53:52.597 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:53:52.597 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:53:52.598 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:53:52.598 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:53:52.598 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:53:52.599 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:53:52.599 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 1, 2020-11-28T21:53:52.599 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 1, 2020-11-28T21:53:52.600 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 1, 2020-11-28T21:53:52.600 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 1, 2020-11-28T21:53:52.600 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 1, 2020-11-28T21:53:52.600 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:53:52.601 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:53:52.601 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:53:52.601 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:53:52.602 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:53:52.602 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:53:52.603 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:53:52.603 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.603 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "16", 2020-11-28T21:53:52.603 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:53:52.603 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:53:52.604 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:53:52.604 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:53:52.604 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:53:52.604 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:53:52.605 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:53:52.605 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:53:52.605 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:53:52.605 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:53:52.605 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:53:52.606 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:53:52.606 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:53:52.606 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:53:52.606 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:53:52.606 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:53:52.607 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:53:52.607 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:53:52.609 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:53:52.610 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:53:52.610 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:53:52.610 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:53:52.611 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:53:52.611 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:53:52.611 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:53:52.612 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:53:52.612 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.612 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:53:52.613 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:53:52.614 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:53:52.615 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:53:52.615 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:53:52.616 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:53:52.616 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:53:52.617 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:53:52.617 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:53:52.617 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:53:52.618 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:53:52.619 INFO:teuthology.orchestra.run.smithi162.stdout: "options": { 2020-11-28T21:53:52.619 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_min": 1 2020-11-28T21:53:52.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.620 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:53:52.620 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr_devicehealth": {} 2020-11-28T21:53:52.621 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:52.621 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.621 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.622 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 2, 2020-11-28T21:53:52.622 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "rbd", 2020-11-28T21:53:52.623 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:46:23.901366+0000", 2020-11-28T21:53:52.623 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:53:52.623 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:53:52.624 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:53:52.624 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:53:52.624 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:53:52.625 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:53:52.625 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:53:52.625 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:53:52.625 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2020-11-28T21:53:52.625 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2020-11-28T21:53:52.626 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2020-11-28T21:53:52.626 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2020-11-28T21:53:52.626 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2020-11-28T21:53:52.626 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:53:52.627 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:53:52.627 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:53:52.628 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:53:52.628 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:53:52.628 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:53:52.629 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:53:52.629 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.630 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "19", 2020-11-28T21:53:52.630 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:53:52.630 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:53:52.631 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:53:52.632 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:53:52.632 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:53:52.633 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:53:52.633 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:53:52.634 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:53:52.634 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:53:52.635 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:53:52.635 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:53:52.635 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:53:52.636 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:53:52.636 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:53:52.637 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:53:52.637 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:53:52.637 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:53:52.638 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:53:52.638 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:53:52.639 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:53:52.639 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:53:52.640 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:53:52.640 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:53:52.640 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:53:52.641 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:53:52.641 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:53:52.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.642 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:53:52.642 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:53:52.643 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:53:52.643 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:53:52.644 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:53:52.644 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:53:52.644 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:53:52.645 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:53:52.645 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:53:52.646 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:53:52.646 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:53:52.646 INFO:teuthology.orchestra.run.smithi162.stdout: "options": {}, 2020-11-28T21:53:52.647 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:53:52.647 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd": {} 2020-11-28T21:53:52.648 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:52.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.649 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 15, 2020-11-28T21:53:52.649 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "cephfs_metadata", 2020-11-28T21:53:52.650 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:53:34.083725+0000", 2020-11-28T21:53:52.650 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:53:52.651 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:53:52.651 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:53:52.651 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:53:52.652 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:53:52.652 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:53:52.653 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:53:52.653 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:53:52.653 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2020-11-28T21:53:52.654 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2020-11-28T21:53:52.654 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2020-11-28T21:53:52.655 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2020-11-28T21:53:52.655 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2020-11-28T21:53:52.655 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:53:52.656 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:53:52.656 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:53:52.657 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:53:52.657 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:53:52.657 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:53:52.658 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:53:52.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.659 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "87", 2020-11-28T21:53:52.659 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:53:52.659 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:53:52.660 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:53:52.660 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:53:52.661 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:53:52.661 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:53:52.661 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:53:52.662 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:53:52.662 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:53:52.663 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:53:52.663 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:53:52.663 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:53:52.664 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:53:52.664 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:53:52.665 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:53:52.665 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:53:52.665 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:53:52.665 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:53:52.666 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:53:52.666 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:53:52.666 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:53:52.666 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:53:52.666 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:53:52.667 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:53:52.667 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:53:52.667 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:53:52.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.667 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:53:52.668 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:53:52.668 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:53:52.668 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:53:52.668 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:53:52.668 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:53:52.669 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:53:52.669 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:53:52.669 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:53:52.669 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:53:52.670 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:53:52.670 INFO:teuthology.orchestra.run.smithi162.stdout: "options": { 2020-11-28T21:53:52.670 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_bias": 4, 2020-11-28T21:53:52.670 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_min": 16, 2020-11-28T21:53:52.670 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_priority": 5 2020-11-28T21:53:52.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.671 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:53:52.671 INFO:teuthology.orchestra.run.smithi162.stdout: "cephfs": { 2020-11-28T21:53:52.671 INFO:teuthology.orchestra.run.smithi162.stdout: "metadata": "cephfs" 2020-11-28T21:53:52.671 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:52.672 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:52.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.672 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 16, 2020-11-28T21:53:52.672 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "cephfs_data", 2020-11-28T21:53:52.673 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:53:35.380493+0000", 2020-11-28T21:53:52.673 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:53:52.673 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:53:52.673 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:53:52.673 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:53:52.674 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:53:52.674 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:53:52.674 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:53:52.674 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:53:52.674 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2020-11-28T21:53:52.675 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2020-11-28T21:53:52.675 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2020-11-28T21:53:52.675 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2020-11-28T21:53:52.675 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2020-11-28T21:53:52.675 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:53:52.676 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:53:52.676 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:53:52.676 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:53:52.676 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:53:52.676 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:53:52.677 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:53:52.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.677 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "88", 2020-11-28T21:53:52.677 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:53:52.678 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:53:52.678 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:53:52.678 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:53:52.678 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:53:52.678 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:53:52.679 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:53:52.679 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:53:52.679 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:53:52.679 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:53:52.679 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:53:52.680 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:53:52.680 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:53:52.680 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:53:52.680 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:53:52.680 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:53:52.681 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:53:52.681 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:53:52.681 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:53:52.681 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:53:52.681 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:53:52.682 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:53:52.682 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:53:52.682 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:53:52.682 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:53:52.683 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:53:52.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.683 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:53:52.683 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:53:52.683 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:53:52.684 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:53:52.684 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:53:52.684 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:53:52.684 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:53:52.684 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:53:52.685 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:53:52.685 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:53:52.685 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:53:52.685 INFO:teuthology.orchestra.run.smithi162.stdout: "options": {}, 2020-11-28T21:53:52.685 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:53:52.686 INFO:teuthology.orchestra.run.smithi162.stdout: "cephfs": { 2020-11-28T21:53:52.686 INFO:teuthology.orchestra.run.smithi162.stdout: "data": "cephfs" 2020-11-28T21:53:52.686 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:52.686 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:52.687 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:52.687 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:53:52.687 INFO:teuthology.orchestra.run.smithi162.stdout: "osds": [ 2020-11-28T21:53:52.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.687 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 0, 2020-11-28T21:53:52.688 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "99c7d9ae-adfa-4fd7-880a-23e6baf3281b", 2020-11-28T21:53:52.688 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:53:52.688 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:53:52.688 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:53:52.688 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:53:52.689 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:53:52.689 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:53:52.689 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:53:52.689 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 86, 2020-11-28T21:53:52.689 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:53:52.690 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:53:52.690 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:53:52.690 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:52.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.690 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:52.691 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6810", 2020-11-28T21:53:52.691 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:53:52.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.691 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:52.692 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6811", 2020-11-28T21:53:52.692 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:53:52.692 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:52.692 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:52.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.693 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:53:52.693 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:52.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.693 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:52.693 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6812", 2020-11-28T21:53:52.694 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:53:52.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.694 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:52.695 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6813", 2020-11-28T21:53:52.695 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:53:52.695 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:52.695 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:52.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.696 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:53:52.696 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:52.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.696 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:52.696 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6816", 2020-11-28T21:53:52.697 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:53:52.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.697 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:52.697 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6817", 2020-11-28T21:53:52.698 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:53:52.698 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:52.698 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:52.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.698 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:53:52.699 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:52.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.699 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:52.699 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6814", 2020-11-28T21:53:52.699 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:53:52.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.700 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:52.700 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6815", 2020-11-28T21:53:52.700 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:53:52.701 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:52.701 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:52.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.701 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6811/12313", 2020-11-28T21:53:52.702 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6813/12313", 2020-11-28T21:53:52.702 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6817/12313", 2020-11-28T21:53:52.702 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6815/12313", 2020-11-28T21:53:52.702 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:53:52.702 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:53:52.703 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:53:52.703 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:52.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.703 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 1, 2020-11-28T21:53:52.704 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "6ce6220a-00ef-4d13-abba-56f87636fb5f", 2020-11-28T21:53:52.704 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:53:52.704 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:53:52.704 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:53:52.704 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:53:52.705 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:53:52.705 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:53:52.705 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:53:52.705 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 84, 2020-11-28T21:53:52.705 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:53:52.706 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:53:52.706 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:53:52.706 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:52.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.706 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:52.707 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6802", 2020-11-28T21:53:52.707 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:53:52.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.707 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:52.708 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6803", 2020-11-28T21:53:52.708 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:53:52.708 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:52.708 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:52.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.709 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:53:52.709 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:52.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.709 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:52.710 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6804", 2020-11-28T21:53:52.710 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:53:52.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.710 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:52.711 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6805", 2020-11-28T21:53:52.711 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:53:52.711 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:52.711 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:52.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.712 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:53:52.712 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:52.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.712 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:52.712 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6808", 2020-11-28T21:53:52.713 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:53:52.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.713 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:52.713 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6809", 2020-11-28T21:53:52.714 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:53:52.714 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:52.714 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:52.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.714 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:53:52.715 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:52.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.715 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:52.715 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6806", 2020-11-28T21:53:52.716 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:53:52.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.716 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:52.716 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6807", 2020-11-28T21:53:52.717 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:53:52.717 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:52.717 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:52.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.717 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6803/12311", 2020-11-28T21:53:52.718 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6805/12311", 2020-11-28T21:53:52.718 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6809/12311", 2020-11-28T21:53:52.718 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6807/12311", 2020-11-28T21:53:52.718 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:53:52.719 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:53:52.719 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:53:52.719 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:52.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.720 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 2, 2020-11-28T21:53:52.720 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "db4e510d-27c1-47fc-a902-8b32e7dfd73e", 2020-11-28T21:53:52.720 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:53:52.720 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:53:52.720 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:53:52.721 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:53:52.721 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:53:52.721 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:53:52.721 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:53:52.721 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 74, 2020-11-28T21:53:52.722 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:53:52.722 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:53:52.722 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:53:52.722 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:52.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.723 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:52.723 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6818", 2020-11-28T21:53:52.723 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:53:52.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.724 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:52.724 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6819", 2020-11-28T21:53:52.724 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:53:52.724 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:52.724 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:52.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.725 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:53:52.725 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:52.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.725 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:52.726 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6820", 2020-11-28T21:53:52.726 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:53:52.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.726 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:52.727 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6821", 2020-11-28T21:53:52.727 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:53:52.727 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:52.727 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:52.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.728 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:53:52.728 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:52.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.728 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:52.729 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6824", 2020-11-28T21:53:52.729 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:53:52.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.729 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:52.730 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6825", 2020-11-28T21:53:52.730 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:53:52.730 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:52.730 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:52.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.731 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:53:52.731 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:52.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.731 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:52.731 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6822", 2020-11-28T21:53:52.732 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:53:52.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.732 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:52.733 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6823", 2020-11-28T21:53:52.733 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:53:52.733 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:52.733 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:52.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.734 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6819/12312", 2020-11-28T21:53:52.734 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6821/12312", 2020-11-28T21:53:52.734 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6825/12312", 2020-11-28T21:53:52.734 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6823/12312", 2020-11-28T21:53:52.734 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:53:52.735 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:53:52.735 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:53:52.735 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:52.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.736 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 3, 2020-11-28T21:53:52.736 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "4094cb87-f04a-45e1-8710-fb4b651949f6", 2020-11-28T21:53:52.736 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:53:52.736 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:53:52.736 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:53:52.737 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:53:52.737 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:53:52.737 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:53:52.737 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:53:52.737 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 86, 2020-11-28T21:53:52.738 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:53:52.738 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:53:52.738 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:53:52.738 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:52.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.739 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:52.739 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6826", 2020-11-28T21:53:52.739 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:53:52.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.740 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:52.740 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6827", 2020-11-28T21:53:52.740 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:53:52.740 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:52.741 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:52.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.741 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:53:52.741 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:52.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.742 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:52.742 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6828", 2020-11-28T21:53:52.742 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:53:52.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.743 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:52.743 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6829", 2020-11-28T21:53:52.743 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:53:52.743 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:52.743 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:52.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.744 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:53:52.744 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:52.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.744 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:52.745 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6832", 2020-11-28T21:53:52.745 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:53:52.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.745 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:52.746 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6833", 2020-11-28T21:53:52.746 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:53:52.746 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:52.746 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:52.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.747 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:53:52.747 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:52.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.747 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:52.747 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6830", 2020-11-28T21:53:52.748 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:53:52.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.748 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:52.749 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6831", 2020-11-28T21:53:52.749 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:53:52.749 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:52.749 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:52.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.750 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6827/12323", 2020-11-28T21:53:52.750 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6829/12323", 2020-11-28T21:53:52.750 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6833/12323", 2020-11-28T21:53:52.750 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6831/12323", 2020-11-28T21:53:52.750 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:53:52.751 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:53:52.751 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:53:52.751 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:52.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.752 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 4, 2020-11-28T21:53:52.752 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "b907bf53-c297-4756-847b-3be96680eef5", 2020-11-28T21:53:52.752 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:53:52.752 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:53:52.752 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:53:52.753 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:53:52.753 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:53:52.753 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:53:52.753 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:53:52.753 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 86, 2020-11-28T21:53:52.754 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:53:52.754 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:53:52.754 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:53:52.754 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:52.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.755 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:52.755 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6800", 2020-11-28T21:53:52.755 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:53:52.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.756 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:52.756 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6801", 2020-11-28T21:53:52.756 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:53:52.756 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:52.756 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:52.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.757 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:53:52.757 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:52.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.757 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:52.758 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6802", 2020-11-28T21:53:52.758 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:53:52.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.759 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:52.759 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6803", 2020-11-28T21:53:52.759 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:53:52.759 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:52.759 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:52.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.760 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:53:52.760 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:52.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.760 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:52.761 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6806", 2020-11-28T21:53:52.761 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:53:52.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.761 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:52.762 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6807", 2020-11-28T21:53:52.762 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:53:52.762 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:52.762 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:52.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.763 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:53:52.763 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:52.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.763 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:52.763 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6804", 2020-11-28T21:53:52.764 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:53:52.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.764 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:52.764 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6805", 2020-11-28T21:53:52.765 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:53:52.765 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:52.765 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:52.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.765 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6801/12527", 2020-11-28T21:53:52.766 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6803/12527", 2020-11-28T21:53:52.766 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6807/12527", 2020-11-28T21:53:52.766 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6805/12527", 2020-11-28T21:53:52.766 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:53:52.766 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:53:52.767 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:53:52.767 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:52.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.767 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 5, 2020-11-28T21:53:52.768 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "55ba305d-0505-4212-aebb-631e8554b440", 2020-11-28T21:53:52.768 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:53:52.768 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:53:52.768 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:53:52.768 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:53:52.769 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:53:52.769 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:53:52.769 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:53:52.769 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 86, 2020-11-28T21:53:52.769 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:53:52.770 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:53:52.770 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:53:52.770 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:52.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.770 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:52.771 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6808", 2020-11-28T21:53:52.771 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:53:52.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.772 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:52.772 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6809", 2020-11-28T21:53:52.772 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:53:52.772 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:52.772 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:52.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.773 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:53:52.773 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:52.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.773 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:52.774 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6810", 2020-11-28T21:53:52.774 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:53:52.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.774 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:52.775 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6811", 2020-11-28T21:53:52.775 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:53:52.775 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:52.775 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:52.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.776 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:53:52.776 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:52.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.776 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:52.776 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6814", 2020-11-28T21:53:52.777 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:53:52.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.777 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:52.777 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6815", 2020-11-28T21:53:52.778 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:53:52.778 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:52.778 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:52.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.778 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:53:52.779 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:52.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.779 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:52.779 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6812", 2020-11-28T21:53:52.779 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:53:52.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.780 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:52.780 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6813", 2020-11-28T21:53:52.780 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:53:52.781 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:52.781 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:52.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.781 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6809/12529", 2020-11-28T21:53:52.781 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6811/12529", 2020-11-28T21:53:52.782 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6815/12529", 2020-11-28T21:53:52.782 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6813/12529", 2020-11-28T21:53:52.782 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:53:52.782 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:53:52.783 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:53:52.783 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:52.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.783 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 6, 2020-11-28T21:53:52.784 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "9ada6051-493b-46b5-ba22-1b401440490f", 2020-11-28T21:53:52.784 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:53:52.784 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:53:52.784 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:53:52.784 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:53:52.785 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:53:52.785 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:53:52.785 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:53:52.785 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 86, 2020-11-28T21:53:52.785 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:53:52.786 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:53:52.786 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:53:52.786 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:52.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.786 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:52.787 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6824", 2020-11-28T21:53:52.787 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:53:52.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.788 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:52.788 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6825", 2020-11-28T21:53:52.788 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:53:52.788 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:52.788 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:52.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.789 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:53:52.789 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:52.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.790 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:52.790 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6826", 2020-11-28T21:53:52.790 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:53:52.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.791 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:52.791 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6827", 2020-11-28T21:53:52.791 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:53:52.791 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:52.792 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:52.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.792 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:53:52.792 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:52.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.793 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:52.793 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6830", 2020-11-28T21:53:52.793 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:53:52.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.794 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:52.794 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6831", 2020-11-28T21:53:52.794 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:53:52.794 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:52.795 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:52.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.795 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:53:52.795 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:52.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.796 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:52.796 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6828", 2020-11-28T21:53:52.796 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:53:52.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.797 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:52.797 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6829", 2020-11-28T21:53:52.797 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:53:52.797 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:52.797 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:52.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.798 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6825/12530", 2020-11-28T21:53:52.798 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6827/12530", 2020-11-28T21:53:52.798 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6831/12530", 2020-11-28T21:53:52.798 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6829/12530", 2020-11-28T21:53:52.799 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:53:52.799 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:53:52.799 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:53:52.799 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:52.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.800 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 7, 2020-11-28T21:53:52.800 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "c328b06d-a216-4cba-aad7-9ad081340613", 2020-11-28T21:53:52.800 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:53:52.801 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:53:52.801 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:53:52.801 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:53:52.801 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:53:52.801 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:53:52.802 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:53:52.802 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 74, 2020-11-28T21:53:52.802 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:53:52.802 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:53:52.803 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:53:52.803 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:52.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.803 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:52.803 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6816", 2020-11-28T21:53:52.804 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:53:52.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.804 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:52.804 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6817", 2020-11-28T21:53:52.805 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:53:52.805 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:52.805 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:52.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.805 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:53:52.806 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:52.806 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.806 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:52.806 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6818", 2020-11-28T21:53:52.806 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:53:52.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.807 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.807 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:52.807 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6819", 2020-11-28T21:53:52.807 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:53:52.808 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:52.808 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:52.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.808 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:53:52.808 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:52.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.809 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:52.809 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6822", 2020-11-28T21:53:52.809 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:53:52.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.810 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.810 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:52.810 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6823", 2020-11-28T21:53:52.810 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:53:52.811 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:52.811 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:52.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.811 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:53:52.811 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:53:52.812 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.812 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:53:52.812 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6820", 2020-11-28T21:53:52.812 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:53:52.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.813 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:53:52.813 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6821", 2020-11-28T21:53:52.813 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:53:52.813 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:52.814 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:52.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.814 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6817/12528", 2020-11-28T21:53:52.814 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6819/12528", 2020-11-28T21:53:52.814 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6823/12528", 2020-11-28T21:53:52.815 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6821/12528", 2020-11-28T21:53:52.815 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:53:52.815 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:53:52.815 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:53:52.815 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:52.816 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:52.816 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:53:52.816 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_xinfo": [ 2020-11-28T21:53:52.816 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.816 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 0, 2020-11-28T21:53:52.817 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:53:52.817 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:53:52.817 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:53:52.817 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:53:52.817 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:53:52.818 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.038052+0000", 2020-11-28T21:53:52.818 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:53:52.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.818 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.819 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 1, 2020-11-28T21:53:52.819 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:53:52.819 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:53:52.819 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:53:52.819 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:53:52.820 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:53:52.820 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:17.947136+0000", 2020-11-28T21:53:52.820 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:53:52.820 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.820 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.821 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 2, 2020-11-28T21:53:52.821 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:53:52.821 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:53:52.821 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:53:52.821 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:53:52.822 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:53:52.822 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.001186+0000", 2020-11-28T21:53:52.822 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:53:52.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.822 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.823 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 3, 2020-11-28T21:53:52.823 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:53:52.823 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:53:52.823 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:53:52.823 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:53:52.824 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:53:52.824 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.199284+0000", 2020-11-28T21:53:52.824 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:53:52.824 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.824 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.825 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 4, 2020-11-28T21:53:52.825 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:53:52.825 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:53:52.825 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:53:52.825 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:53:52.826 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:53:52.826 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.038823+0000", 2020-11-28T21:53:52.826 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:53:52.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.827 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 5, 2020-11-28T21:53:52.827 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:53:52.827 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:53:52.827 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:53:52.827 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:53:52.828 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:53:52.828 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.009322+0000", 2020-11-28T21:53:52.828 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:53:52.828 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.828 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.829 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 6, 2020-11-28T21:53:52.829 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:53:52.829 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:53:52.829 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:53:52.829 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:53:52.830 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:53:52.830 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.238927+0000", 2020-11-28T21:53:52.830 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:53:52.830 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:52.831 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 7, 2020-11-28T21:53:52.831 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:53:52.831 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:53:52.831 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:53:52.832 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:53:52.832 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:53:52.832 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.029839+0000", 2020-11-28T21:53:52.832 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:53:52.832 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:52.833 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:53:52.833 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_upmap": [], 2020-11-28T21:53:52.833 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_upmap_items": [], 2020-11-28T21:53:52.833 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_temp": [], 2020-11-28T21:53:52.833 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_temp": [], 2020-11-28T21:53:52.834 INFO:teuthology.orchestra.run.smithi162.stdout: "blacklist": {}, 2020-11-28T21:53:52.834 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profiles": { 2020-11-28T21:53:52.834 INFO:teuthology.orchestra.run.smithi162.stdout: "default": { 2020-11-28T21:53:52.834 INFO:teuthology.orchestra.run.smithi162.stdout: "crush-failure-domain": "osd", 2020-11-28T21:53:52.834 INFO:teuthology.orchestra.run.smithi162.stdout: "k": "2", 2020-11-28T21:53:52.835 INFO:teuthology.orchestra.run.smithi162.stdout: "m": "1", 2020-11-28T21:53:52.835 INFO:teuthology.orchestra.run.smithi162.stdout: "plugin": "jerasure", 2020-11-28T21:53:52.835 INFO:teuthology.orchestra.run.smithi162.stdout: "ruleset-failure-domain": "osd", 2020-11-28T21:53:52.835 INFO:teuthology.orchestra.run.smithi162.stdout: "technique": "reed_sol_van" 2020-11-28T21:53:52.835 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:52.836 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:52.836 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps_queue": [], 2020-11-28T21:53:52.836 INFO:teuthology.orchestra.run.smithi162.stdout: "new_removed_snaps": [], 2020-11-28T21:53:52.836 INFO:teuthology.orchestra.run.smithi162.stdout: "new_purged_snaps": [], 2020-11-28T21:53:52.836 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_node_flags": {}, 2020-11-28T21:53:52.837 INFO:teuthology.orchestra.run.smithi162.stdout: "device_class_flags": {} 2020-11-28T21:53:52.837 INFO:teuthology.orchestra.run.smithi162.stdout:} 2020-11-28T21:53:52.839 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.576+0000 7f45ca3db700 1 -- 172.21.15.162:0/3837711029 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f45a4055330 msgr2=0x7f45a40577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:52.839 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.576+0000 7f45ca3db700 1 --2- 172.21.15.162:0/3837711029 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f45a4055330 0x7f45a40577f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7f45b400b000 tx=0x7f45b4009040).stop 2020-11-28T21:53:52.839 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.576+0000 7f45ca3db700 1 -- 172.21.15.162:0/3837711029 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45c40ad2c0 msgr2=0x7f45c4066760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:52.839 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.576+0000 7f45ca3db700 1 --2- 172.21.15.162:0/3837711029 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45c40ad2c0 0x7f45c4066760 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 rx=0x7f45c4066000 tx=0x7f45b8007c20).stop 2020-11-28T21:53:52.840 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.576+0000 7f45ca3db700 1 -- 172.21.15.162:0/3837711029 shutdown_connections 2020-11-28T21:53:52.840 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.576+0000 7f45ca3db700 1 --2- 172.21.15.162:0/3837711029 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f45a4055330 0x7f45a40577f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:52.840 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.576+0000 7f45ca3db700 1 --2- 172.21.15.162:0/3837711029 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f45c40b2830 0x7f45c4148fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:52.840 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.576+0000 7f45ca3db700 1 --2- 172.21.15.162:0/3837711029 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45c40ad2c0 0x7f45c4066760 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:52.840 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.576+0000 7f45ca3db700 1 --2- 172.21.15.162:0/3837711029 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f45c40b1860 0x7f45c4143950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:52.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.576+0000 7f45ca3db700 1 -- 172.21.15.162:0/3837711029 >> 172.21.15.162:0/3837711029 conn(0x7f45c4061550 msgr2=0x7f45c4061ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:52.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.576+0000 7f45ca3db700 1 -- 172.21.15.162:0/3837711029 shutdown_connections 2020-11-28T21:53:52.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.576+0000 7f45ca3db700 1 -- 172.21.15.162:0/3837711029 wait complete. 2020-11-28T21:53:52.842 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:53:52.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.980+0000 7f325125b700 1 Processor -- start 2020-11-28T21:53:52.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.980+0000 7f325125b700 1 -- start start 2020-11-28T21:53:52.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.980+0000 7f325125b700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f324c13f6c0 0x7f324c0669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:52.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.980+0000 7f325125b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f324c065750 0x7f324c13c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:52.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.980+0000 7f325125b700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f324c065050 con 0x7f324c064c70 2020-11-28T21:53:52.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.980+0000 7f325125b700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f324c13fc40 con 0x7f324c13f6c0 2020-11-28T21:53:52.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.980+0000 7f325125b700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f324c13fee0 con 0x7f324c065750 2020-11-28T21:53:52.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.980+0000 7f3250a5a700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f324c065750 0x7f324c13c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:52.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.980+0000 7f3250a5a700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f324c065750 0x7f324c13c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42460/0 (socket says 172.21.15.162:42460) 2020-11-28T21:53:52.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.980+0000 7f3250a5a700 1 -- 172.21.15.162:0/2696197615 learned_addr learned my addr 172.21.15.162:0/2696197615 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:52.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.980+0000 7f324b7fe700 1 --2- 172.21.15.162:0/2696197615 >> v2:172.21.15.118:3300/0 conn(0x7f324c13f6c0 0x7f324c0669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:52.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.984+0000 7f3250a5a700 1 -- 172.21.15.162:0/2696197615 >> v2:172.21.15.118:3300/0 conn(0x7f324c13f6c0 msgr2=0x7f324c0669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:52.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.984+0000 7f3250a5a700 1 --2- 172.21.15.162:0/2696197615 >> v2:172.21.15.118:3300/0 conn(0x7f324c13f6c0 0x7f324c0669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:52.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.984+0000 7f3250a5a700 1 -- 172.21.15.162:0/2696197615 >> v1:172.21.15.118:6789/0 conn(0x7f324c064c70 legacy=0x7f324c066290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:52.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.984+0000 7f3250a5a700 1 -- 172.21.15.162:0/2696197615 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f324c140180 con 0x7f324c065750 2020-11-28T21:53:52.992 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.984+0000 7f3250a5a700 1 --2- 172.21.15.162:0/2696197615 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f324c065750 0x7f324c13c260 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 rx=0x7f3244002f30 tx=0x7f3244006e80).ready entity=mon.1 client_cookie=c869d813982dce33 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:52.992 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.984+0000 7f324affd700 1 -- 172.21.15.162:0/2696197615 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f32440097f0 con 0x7f324c065750 2020-11-28T21:53:52.992 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.984+0000 7f324affd700 1 -- 172.21.15.162:0/2696197615 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3244009950 con 0x7f324c065750 2020-11-28T21:53:52.992 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.984+0000 7f324affd700 1 -- 172.21.15.162:0/2696197615 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3244009cb0 con 0x7f324c065750 2020-11-28T21:53:52.993 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.984+0000 7f325125b700 1 -- 172.21.15.162:0/2696197615 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f324c065750 msgr2=0x7f324c13c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:52.993 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.984+0000 7f325125b700 1 --2- 172.21.15.162:0/2696197615 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f324c065750 0x7f324c13c260 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 rx=0x7f3244002f30 tx=0x7f3244006e80).stop 2020-11-28T21:53:52.993 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.984+0000 7f325125b700 1 -- 172.21.15.162:0/2696197615 shutdown_connections 2020-11-28T21:53:52.993 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.984+0000 7f325125b700 1 --2- 172.21.15.162:0/2696197615 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f324c065750 0x7f324c13c260 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:52.993 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.984+0000 7f325125b700 1 --2- 172.21.15.162:0/2696197615 >> v2:172.21.15.118:3300/0 conn(0x7f324c13f6c0 0x7f324c0669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:52.994 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.984+0000 7f325125b700 1 -- 172.21.15.162:0/2696197615 >> 172.21.15.162:0/2696197615 conn(0x7f324c0b5da0 msgr2=0x7f324c0b6180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:52.994 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.984+0000 7f325125b700 1 -- 172.21.15.162:0/2696197615 shutdown_connections 2020-11-28T21:53:52.994 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.984+0000 7f325125b700 1 -- 172.21.15.162:0/2696197615 wait complete. 2020-11-28T21:53:52.994 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.984+0000 7f325125b700 1 Processor -- start 2020-11-28T21:53:52.994 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.984+0000 7f325125b700 1 -- start start 2020-11-28T21:53:52.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.984+0000 7f325125b700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f324c064c70 0x7f324c139200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:52.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.984+0000 7f325125b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f324c065750 0x7f324c140b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:52.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.984+0000 7f325125b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f324c13f6c0 0x7f324c141080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:52.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.984+0000 7f325125b700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324c141570 con 0x7f324c13f6c0 2020-11-28T21:53:52.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.984+0000 7f325125b700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324c0b3f60 con 0x7f324c065750 2020-11-28T21:53:52.996 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.984+0000 7f325125b700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f324c0b47a0 con 0x7f324c064c70 2020-11-28T21:53:52.996 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.988+0000 7f324b7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f324c065750 0x7f324c140b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:52.996 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.988+0000 7f324b7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f324c065750 0x7f324c140b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42464/0 (socket says 172.21.15.162:42464) 2020-11-28T21:53:52.996 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.988+0000 7f324b7fe700 1 -- 172.21.15.162:0/107209301 learned_addr learned my addr 172.21.15.162:0/107209301 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:52.996 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.988+0000 7f324bfff700 1 --2- 172.21.15.162:0/107209301 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f324c064c70 0x7f324c139200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:52.997 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.988+0000 7f3250a5a700 1 --2- 172.21.15.162:0/107209301 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f324c13f6c0 0x7f324c141080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:52.997 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.988+0000 7f324b7fe700 1 -- 172.21.15.162:0/107209301 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f324c064c70 msgr2=0x7f324c139200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:52.997 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.988+0000 7f324b7fe700 1 --2- 172.21.15.162:0/107209301 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f324c064c70 0x7f324c139200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:52.997 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.988+0000 7f324b7fe700 1 -- 172.21.15.162:0/107209301 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f324c13f6c0 msgr2=0x7f324c141080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:52.998 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.988+0000 7f324b7fe700 1 --2- 172.21.15.162:0/107209301 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f324c13f6c0 0x7f324c141080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:52.998 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.988+0000 7f324b7fe700 1 -- 172.21.15.162:0/107209301 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f324c05f420 con 0x7f324c065750 2020-11-28T21:53:52.998 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.988+0000 7f3250a5a700 1 --2- 172.21.15.162:0/107209301 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f324c13f6c0 0x7f324c141080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:53:52.998 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.988+0000 7f324b7fe700 1 --2- 172.21.15.162:0/107209301 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f324c065750 0x7f324c140b90 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 rx=0x7f323400c000 tx=0x7f3234006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:52.998 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.988+0000 7f32497fa700 1 -- 172.21.15.162:0/107209301 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f323400ca10 con 0x7f324c065750 2020-11-28T21:53:52.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.988+0000 7f32497fa700 1 -- 172.21.15.162:0/107209301 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f323400cb70 con 0x7f324c065750 2020-11-28T21:53:52.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.988+0000 7f325125b700 1 -- 172.21.15.162:0/107209301 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f324c064290 con 0x7f324c065750 2020-11-28T21:53:52.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.988+0000 7f32497fa700 1 -- 172.21.15.162:0/107209301 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f32340123f0 con 0x7f324c065750 2020-11-28T21:53:52.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.988+0000 7f325125b700 1 -- 172.21.15.162:0/107209301 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f324c142bb0 con 0x7f324c065750 2020-11-28T21:53:53.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.988+0000 7f325125b700 1 -- 172.21.15.162:0/107209301 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f324c1428f0 con 0x7f324c065750 2020-11-28T21:53:53.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.988+0000 7f32497fa700 1 -- 172.21.15.162:0/107209301 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f3234012550 con 0x7f324c065750 2020-11-28T21:53:53.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.988+0000 7f32497fa700 1 --2- 172.21.15.162:0/107209301 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f32300553b0 0x7f3230057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:53.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.992+0000 7f32497fa700 1 -- 172.21.15.162:0/107209301 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f323406e360 con 0x7f324c065750 2020-11-28T21:53:53.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.992+0000 7f324bfff700 1 --2- 172.21.15.162:0/107209301 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f32300553b0 0x7f3230057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:53.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.992+0000 7f324bfff700 1 --2- 172.21.15.162:0/107209301 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f32300553b0 0x7f3230057870 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f323c00d000 tx=0x7f323c009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:53.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:52.996+0000 7f32497fa700 1 -- 172.21.15.162:0/107209301 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f324c1428f0 con 0x7f324c065750 2020-11-28T21:53:53.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.180+0000 7f325125b700 1 -- 172.21.15.162:0/107209301 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f324c1428f0 con 0x7f324c065750 2020-11-28T21:53:53.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.184+0000 7f32497fa700 1 -- 172.21.15.162:0/107209301 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+2502 (secure 0 0 0) 0x7f324c1428f0 con 0x7f324c065750 2020-11-28T21:53:53.188 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:53:53.189 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":40,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5795,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/2645168518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":2645168518},{"type":"v1","addr":"172.21.15.118:6833","nonce":2645168518}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36},{"gid":6541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1999073284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1999073284},{"type":"v1","addr":"172.21.15.162:6835","nonce":1999073284}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:53:36.539204+0000","modified":"2020-11-28T21:53:38.035411+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6550":{"gid":6550,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/1097065372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1097065372},{"type":"v1","addr":"172.21.15.162:6837","nonce":1097065372}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-28T21:53:53.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.184+0000 7f325125b700 1 -- 172.21.15.162:0/107209301 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f32300553b0 msgr2=0x7f3230057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:53.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.184+0000 7f325125b700 1 --2- 172.21.15.162:0/107209301 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f32300553b0 0x7f3230057870 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f323c00d000 tx=0x7f323c009040).stop 2020-11-28T21:53:53.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.184+0000 7f325125b700 1 -- 172.21.15.162:0/107209301 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f324c065750 msgr2=0x7f324c140b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:53.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.184+0000 7f325125b700 1 --2- 172.21.15.162:0/107209301 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f324c065750 0x7f324c140b90 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 rx=0x7f323400c000 tx=0x7f3234006dc0).stop 2020-11-28T21:53:53.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.184+0000 7f325125b700 1 -- 172.21.15.162:0/107209301 shutdown_connections 2020-11-28T21:53:53.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.184+0000 7f325125b700 1 --2- 172.21.15.162:0/107209301 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f32300553b0 0x7f3230057870 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:53.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.184+0000 7f325125b700 1 --2- 172.21.15.162:0/107209301 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f324c064c70 0x7f324c139200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:53.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.184+0000 7f325125b700 1 --2- 172.21.15.162:0/107209301 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f324c13f6c0 0x7f324c141080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:53.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.184+0000 7f325125b700 1 --2- 172.21.15.162:0/107209301 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f324c065750 0x7f324c140b90 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:53.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.184+0000 7f325125b700 1 -- 172.21.15.162:0/107209301 >> 172.21.15.162:0/107209301 conn(0x7f324c0b5da0 msgr2=0x7f324c06a330 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:53.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.184+0000 7f325125b700 1 -- 172.21.15.162:0/107209301 shutdown_connections 2020-11-28T21:53:53.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.184+0000 7f325125b700 1 -- 172.21.15.162:0/107209301 wait complete. 2020-11-28T21:53:53.197 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 40 2020-11-28T21:53:53.200 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2020-11-28T21:53:53.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.312+0000 7f896218a700 1 Processor -- start 2020-11-28T21:53:53.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.312+0000 7f896218a700 1 -- start start 2020-11-28T21:53:53.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.312+0000 7f896218a700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f895c098b30 0x7f895c0994f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:53.318 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.312+0000 7f896218a700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f895c0b0000 0x7f895c060000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:53.318 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.312+0000 7f896218a700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f895c098f10 con 0x7f895c0604f0 2020-11-28T21:53:53.318 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.312+0000 7f896218a700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f895c0b1030 con 0x7f895c098b30 2020-11-28T21:53:53.318 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.312+0000 7f896218a700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f895c0b0480 con 0x7f895c0b0000 2020-11-28T21:53:53.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.312+0000 7f8961989700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f895c0604f0 0x7f895c14a550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:49314/0 (socket says 172.21.15.162:49314) 2020-11-28T21:53:53.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.312+0000 7f8961989700 1 -- 172.21.15.162:0/1183246331 learned_addr learned my addr 172.21.15.162:0/1183246331 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:53.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.312+0000 7f8961188700 1 --2- 172.21.15.162:0/1183246331 >> v2:172.21.15.162:3300/0 conn(0x7f895c098b30 0x7f895c0994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:53.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.312+0000 7f8960987700 1 --2- 172.21.15.162:0/1183246331 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f895c0b0000 0x7f895c060000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:53.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.312+0000 7f894ffff700 1 -- 172.21.15.162:0/1183246331 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2193546885 0 0) 0x7f895c098f10 con 0x7f895c0604f0 2020-11-28T21:53:53.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.312+0000 7f894ffff700 1 -- 172.21.15.162:0/1183246331 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8940003230 con 0x7f895c0604f0 2020-11-28T21:53:53.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.312+0000 7f8961188700 1 -- 172.21.15.162:0/1183246331 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f895c0b0000 msgr2=0x7f895c060000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:53.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.312+0000 7f8961188700 1 --2- 172.21.15.162:0/1183246331 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f895c0b0000 0x7f895c060000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:53.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.312+0000 7f8961188700 1 -- 172.21.15.162:0/1183246331 >> v1:172.21.15.162:6789/0 conn(0x7f895c0604f0 legacy=0x7f895c14a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:53.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.312+0000 7f8961188700 1 -- 172.21.15.162:0/1183246331 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f895c0b0680 con 0x7f895c098b30 2020-11-28T21:53:53.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.312+0000 7f8961188700 1 --2- 172.21.15.162:0/1183246331 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f895c098b30 0x7f895c0994f0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 rx=0x7f895c099be0 tx=0x7f8958006dc0).ready entity=mon.1 client_cookie=ccf6e16794b88126 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:53.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.316+0000 7f894ffff700 1 -- 172.21.15.162:0/1183246331 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89580096a0 con 0x7f895c098b30 2020-11-28T21:53:53.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.316+0000 7f894ffff700 1 -- 172.21.15.162:0/1183246331 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8958009800 con 0x7f895c098b30 2020-11-28T21:53:53.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.316+0000 7f894ffff700 1 -- 172.21.15.162:0/1183246331 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8958009b60 con 0x7f895c098b30 2020-11-28T21:53:53.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.316+0000 7f896218a700 1 -- 172.21.15.162:0/1183246331 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f895c098b30 msgr2=0x7f895c0994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:53.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.316+0000 7f896218a700 1 --2- 172.21.15.162:0/1183246331 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f895c098b30 0x7f895c0994f0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 rx=0x7f895c099be0 tx=0x7f8958006dc0).stop 2020-11-28T21:53:53.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.316+0000 7f896218a700 1 -- 172.21.15.162:0/1183246331 shutdown_connections 2020-11-28T21:53:53.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.316+0000 7f896218a700 1 --2- 172.21.15.162:0/1183246331 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f895c098b30 0x7f895c0994f0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:53.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.316+0000 7f896218a700 1 --2- 172.21.15.162:0/1183246331 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f895c0b0000 0x7f895c060000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:53.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.316+0000 7f896218a700 1 -- 172.21.15.162:0/1183246331 >> 172.21.15.162:0/1183246331 conn(0x7f895c05b970 msgr2=0x7f895c05bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:53.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.316+0000 7f896218a700 1 -- 172.21.15.162:0/1183246331 shutdown_connections 2020-11-28T21:53:53.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.316+0000 7f896218a700 1 -- 172.21.15.162:0/1183246331 wait complete. 2020-11-28T21:53:53.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.316+0000 7f896218a700 1 Processor -- start 2020-11-28T21:53:53.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.316+0000 7f896218a700 1 -- start start 2020-11-28T21:53:53.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.316+0000 7f896218a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f895c0604f0 0x7f895c0aab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:53.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.316+0000 7f896218a700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f895c098b30 0x7f895c065220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:53.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.316+0000 7f896218a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f895c0b0000 0x7f895c155be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:53.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.316+0000 7f896218a700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895c0adb80 con 0x7f895c0b0000 2020-11-28T21:53:53.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.316+0000 7f896218a700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895c0ad640 con 0x7f895c0604f0 2020-11-28T21:53:53.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.316+0000 7f896218a700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f895c0ad8e0 con 0x7f895c098b30 2020-11-28T21:53:53.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.316+0000 7f8961188700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f895c0604f0 0x7f895c0aab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:53.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.316+0000 7f8961188700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f895c0604f0 0x7f895c0aab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42476/0 (socket says 172.21.15.162:42476) 2020-11-28T21:53:53.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.316+0000 7f8961188700 1 -- 172.21.15.162:0/3565113398 learned_addr learned my addr 172.21.15.162:0/3565113398 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:53.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.316+0000 7f8961989700 1 --2- 172.21.15.162:0/3565113398 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f895c0b0000 0x7f895c155be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:53.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.316+0000 7f8960987700 1 --2- 172.21.15.162:0/3565113398 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f895c098b30 0x7f895c065220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:53.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.320+0000 7f8961188700 1 -- 172.21.15.162:0/3565113398 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f895c098b30 msgr2=0x7f895c065220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:53.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.320+0000 7f8961188700 1 --2- 172.21.15.162:0/3565113398 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f895c098b30 0x7f895c065220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:53.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.320+0000 7f8961188700 1 -- 172.21.15.162:0/3565113398 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f895c0b0000 msgr2=0x7f895c155be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:53.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.320+0000 7f8961188700 1 --2- 172.21.15.162:0/3565113398 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f895c0b0000 0x7f895c155be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:53.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.320+0000 7f8961188700 1 -- 172.21.15.162:0/3565113398 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f895c064d80 con 0x7f895c0604f0 2020-11-28T21:53:53.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.320+0000 7f8961989700 1 --2- 172.21.15.162:0/3565113398 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f895c0b0000 0x7f895c155be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:53:53.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.320+0000 7f8961188700 1 --2- 172.21.15.162:0/3565113398 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f895c0604f0 0x7f895c0aab50 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 rx=0x7f895c099be0 tx=0x7f8958015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:53.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.320+0000 7f894e7fc700 1 -- 172.21.15.162:0/3565113398 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8958011b00 con 0x7f895c0604f0 2020-11-28T21:53:53.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.320+0000 7f894e7fc700 1 -- 172.21.15.162:0/3565113398 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8958011c60 con 0x7f895c0604f0 2020-11-28T21:53:53.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.320+0000 7f896218a700 1 -- 172.21.15.162:0/3565113398 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f895c065710 con 0x7f895c0604f0 2020-11-28T21:53:53.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.320+0000 7f894e7fc700 1 -- 172.21.15.162:0/3565113398 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f895801d450 con 0x7f895c0604f0 2020-11-28T21:53:53.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.320+0000 7f896218a700 1 -- 172.21.15.162:0/3565113398 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f895c0b4480 con 0x7f895c0604f0 2020-11-28T21:53:53.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.320+0000 7f896218a700 1 -- 172.21.15.162:0/3565113398 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f895c057050 con 0x7f895c0604f0 2020-11-28T21:53:53.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.320+0000 7f894e7fc700 1 -- 172.21.15.162:0/3565113398 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8958024030 con 0x7f895c0604f0 2020-11-28T21:53:53.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.320+0000 7f894e7fc700 1 --2- 172.21.15.162:0/3565113398 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f89440553b0 0x7f8944057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:53.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.320+0000 7f894e7fc700 1 -- 172.21.15.162:0/3565113398 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f895806f170 con 0x7f895c0604f0 2020-11-28T21:53:53.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.320+0000 7f8960987700 1 --2- 172.21.15.162:0/3565113398 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f89440553b0 0x7f8944057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:53.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.324+0000 7f8960987700 1 --2- 172.21.15.162:0/3565113398 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f89440553b0 0x7f8944057870 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f895000b000 tx=0x7f8950008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:53.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.328+0000 7f894e7fc700 1 -- 172.21.15.162:0/3565113398 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f89580453d0 con 0x7f895c0604f0 2020-11-28T21:53:53.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.516+0000 7f896218a700 1 -- 172.21.15.162:0/3565113398 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f895c064f10 con 0x7f895c0604f0 2020-11-28T21:53:53.816 INFO:tasks.ceph.mon.b.smithi118.stderr:2020-11-28T21:53:53.809+0000 7ff4e1520700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-11-28T21:53:53.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.828+0000 7f894e7fc700 1 -- 172.21.15.162:0/3565113398 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v41) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8958022020 con 0x7f895c0604f0 2020-11-28T21:53:53.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.832+0000 7f896218a700 1 -- 172.21.15.162:0/3565113398 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f89440553b0 msgr2=0x7f8944057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:53.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.832+0000 7f896218a700 1 --2- 172.21.15.162:0/3565113398 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f89440553b0 0x7f8944057870 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f895000b000 tx=0x7f8950008040).stop 2020-11-28T21:53:53.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.832+0000 7f896218a700 1 -- 172.21.15.162:0/3565113398 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f895c0604f0 msgr2=0x7f895c0aab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:53.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.832+0000 7f896218a700 1 --2- 172.21.15.162:0/3565113398 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f895c0604f0 0x7f895c0aab50 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 rx=0x7f895c099be0 tx=0x7f8958015000).stop 2020-11-28T21:53:53.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.832+0000 7f896218a700 1 -- 172.21.15.162:0/3565113398 shutdown_connections 2020-11-28T21:53:53.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.832+0000 7f896218a700 1 --2- 172.21.15.162:0/3565113398 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f89440553b0 0x7f8944057870 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:53.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.832+0000 7f896218a700 1 --2- 172.21.15.162:0/3565113398 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f895c0b0000 0x7f895c155be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:53.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.832+0000 7f896218a700 1 --2- 172.21.15.162:0/3565113398 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f895c0604f0 0x7f895c0aab50 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:53.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.832+0000 7f896218a700 1 --2- 172.21.15.162:0/3565113398 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f895c098b30 0x7f895c065220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:53.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.832+0000 7f896218a700 1 -- 172.21.15.162:0/3565113398 >> 172.21.15.162:0/3565113398 conn(0x7f895c05b970 msgr2=0x7f895c0a8450 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:53.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.832+0000 7f896218a700 1 -- 172.21.15.162:0/3565113398 shutdown_connections 2020-11-28T21:53:53.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.832+0000 7f896218a700 1 -- 172.21.15.162:0/3565113398 wait complete. 2020-11-28T21:53:53.838 INFO:teuthology.orchestra.run.smithi162.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-11-28T21:53:53.851 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:53:53.852 INFO:tasks.ceph.mds.c.smithi162.stdout:starting mds.c at 2020-11-28T21:53:53.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.956+0000 7f87c08c6700 1 Processor -- start 2020-11-28T21:53:53.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.956+0000 7f87c08c6700 1 -- start start 2020-11-28T21:53:53.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.956+0000 7f87c08c6700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f87bc05cbd0 0x7f87bc0aac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:53.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.956+0000 7f87c08c6700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87bc0b0230 con 0x7f87bc060cf0 2020-11-28T21:53:53.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.956+0000 7f87c08c6700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87bc0ab0f0 con 0x7f87bc0af640 2020-11-28T21:53:53.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.956+0000 7f87c08c6700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87bc0b12a0 con 0x7f87bc05cbd0 2020-11-28T21:53:53.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.960+0000 7f87bbfff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f87bc0af640 0x7f87bc0ae1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:49344/0 (socket says 172.21.15.162:49344) 2020-11-28T21:53:53.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.960+0000 7f87bbfff700 1 -- 172.21.15.162:0/791451147 learned_addr learned my addr 172.21.15.162:0/791451147 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:53.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.960+0000 7f87b2ffd700 1 --2- 172.21.15.162:0/791451147 >> v2:172.21.15.162:3300/0 conn(0x7f87bc05cbd0 0x7f87bc0aac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:53.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.960+0000 7f87baffd700 1 -- 172.21.15.162:0/791451147 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4280070178 0 0) 0x7f87bc0ab0f0 con 0x7f87bc0af640 2020-11-28T21:53:53.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.960+0000 7f87baffd700 1 -- 172.21.15.162:0/791451147 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f879c003230 con 0x7f87bc0af640 2020-11-28T21:53:53.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.960+0000 7f87b2ffd700 1 -- 172.21.15.162:0/791451147 >> v1:172.21.15.162:6789/0 conn(0x7f87bc0af640 legacy=0x7f87bc0ae1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:53.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.960+0000 7f87baffd700 1 -- 172.21.15.162:0/791451147 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3446246627 0 0) 0x7f879c003230 con 0x7f87bc0af640 2020-11-28T21:53:53.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.960+0000 7f87b2ffd700 1 -- 172.21.15.162:0/791451147 >> v1:172.21.15.118:6789/0 conn(0x7f87bc060cf0 legacy=0x7f87bc0aa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:53.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.960+0000 7f87b2ffd700 1 -- 172.21.15.162:0/791451147 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87bc0a8c00 con 0x7f87bc05cbd0 2020-11-28T21:53:53.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.960+0000 7f87b2ffd700 1 --2- 172.21.15.162:0/791451147 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87bc05cbd0 0x7f87bc0aac00 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 rx=0x7f87ac0010d0 tx=0x7f87a4006dc0).ready entity=mon.1 client_cookie=2625dab6e5dd46ae server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:53.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.960+0000 7f87baffd700 1 -- 172.21.15.162:0/791451147 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87b4001a50 con 0x7f87bc05cbd0 2020-11-28T21:53:53.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.960+0000 7f87baffd700 1 -- 172.21.15.162:0/791451147 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a400d5c0 con 0x7f87bc05cbd0 2020-11-28T21:53:53.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.960+0000 7f87c08c6700 1 -- 172.21.15.162:0/791451147 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87bc05cbd0 msgr2=0x7f87bc0aac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:53.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.960+0000 7f87baffd700 1 -- 172.21.15.162:0/791451147 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87a400d730 con 0x7f87bc05cbd0 2020-11-28T21:53:53.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.960+0000 7f87c08c6700 1 --2- 172.21.15.162:0/791451147 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87bc05cbd0 0x7f87bc0aac00 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 rx=0x7f87ac0010d0 tx=0x7f87a4006dc0).stop 2020-11-28T21:53:53.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.960+0000 7f87c08c6700 1 -- 172.21.15.162:0/791451147 shutdown_connections 2020-11-28T21:53:53.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.960+0000 7f87c08c6700 1 --2- 172.21.15.162:0/791451147 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87bc05cbd0 0x7f87bc0aac00 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:53.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.960+0000 7f87c08c6700 1 -- 172.21.15.162:0/791451147 >> 172.21.15.162:0/791451147 conn(0x7f87bc05b970 msgr2=0x7f87bc05bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:53.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.960+0000 7f87c08c6700 1 -- 172.21.15.162:0/791451147 shutdown_connections 2020-11-28T21:53:53.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.960+0000 7f87c08c6700 1 -- 172.21.15.162:0/791451147 wait complete. 2020-11-28T21:53:53.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.960+0000 7f87c08c6700 1 Processor -- start 2020-11-28T21:53:53.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.960+0000 7f87c08c6700 1 -- start start 2020-11-28T21:53:53.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.964+0000 7f87c08c6700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f87bc05cbd0 0x7f87bc090a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:53.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.964+0000 7f87c08c6700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f87bc060cf0 0x7f87bc0921a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:53.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.964+0000 7f87c08c6700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87bc0af640 0x7f87bc14b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:53.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.964+0000 7f87c08c6700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc090380 con 0x7f87bc060cf0 2020-11-28T21:53:53.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.964+0000 7f87c08c6700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc091a80 con 0x7f87bc0af640 2020-11-28T21:53:53.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.964+0000 7f87c08c6700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc093b20 con 0x7f87bc05cbd0 2020-11-28T21:53:53.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.964+0000 7f87bbfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87bc0af640 0x7f87bc14b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:53.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.964+0000 7f87bbfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87bc0af640 0x7f87bc14b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42502/0 (socket says 172.21.15.162:42502) 2020-11-28T21:53:53.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.964+0000 7f87bbfff700 1 -- 172.21.15.162:0/2951738585 learned_addr learned my addr 172.21.15.162:0/2951738585 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:53.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.964+0000 7f87b2ffd700 1 --2- 172.21.15.162:0/2951738585 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f87bc060cf0 0x7f87bc0921a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:53.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.964+0000 7f87bbfff700 1 -- 172.21.15.162:0/2951738585 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f87bc05cbd0 msgr2=0x7f87bc090a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:53.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.964+0000 7f87bbfff700 1 --2- 172.21.15.162:0/2951738585 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f87bc05cbd0 0x7f87bc090a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-28T21:53:53.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.964+0000 7f87bbfff700 1 -- 172.21.15.162:0/2951738585 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f87bc060cf0 msgr2=0x7f87bc0921a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:53.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.964+0000 7f87bbfff700 1 --2- 172.21.15.162:0/2951738585 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f87bc060cf0 0x7f87bc0921a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:53.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.964+0000 7f87bbfff700 1 -- 172.21.15.162:0/2951738585 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87bc0913b0 con 0x7f87bc0af640 2020-11-28T21:53:53.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.964+0000 7f87b2ffd700 1 --2- 172.21.15.162:0/2951738585 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f87bc060cf0 0x7f87bc0921a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:53:53.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.964+0000 7f87bbfff700 1 --2- 172.21.15.162:0/2951738585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87bc0af640 0x7f87bc14b630 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 rx=0x7f87b4001cb0 tx=0x7f87b4002190).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:53.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.964+0000 7f87b97fa700 1 -- 172.21.15.162:0/2951738585 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87b4004a80 con 0x7f87bc0af640 2020-11-28T21:53:53.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.964+0000 7f87b97fa700 1 -- 172.21.15.162:0/2951738585 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b4004be0 con 0x7f87bc0af640 2020-11-28T21:53:53.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.964+0000 7f87c08c6700 1 -- 172.21.15.162:0/2951738585 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87bc092db0 con 0x7f87bc0af640 2020-11-28T21:53:53.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.964+0000 7f87b97fa700 1 -- 172.21.15.162:0/2951738585 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87b4007170 con 0x7f87bc0af640 2020-11-28T21:53:53.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.964+0000 7f87c08c6700 1 -- 172.21.15.162:0/2951738585 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87bc090f20 con 0x7f87bc0af640 2020-11-28T21:53:53.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.964+0000 7f87c08c6700 1 -- 172.21.15.162:0/2951738585 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87bc057050 con 0x7f87bc0af640 2020-11-28T21:53:53.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.964+0000 7f87b97fa700 1 -- 172.21.15.162:0/2951738585 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f87b4010070 con 0x7f87bc0af640 2020-11-28T21:53:53.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.964+0000 7f87b97fa700 1 --2- 172.21.15.162:0/2951738585 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f87a0055440 0x7f87a0057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:53.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.964+0000 7f87b97fa700 1 -- 172.21.15.162:0/2951738585 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f87b406dbd0 con 0x7f87bc0af640 2020-11-28T21:53:53.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.964+0000 7f87bb7fe700 1 --2- 172.21.15.162:0/2951738585 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f87a0055440 0x7f87a0057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:53.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.968+0000 7f87bb7fe700 1 --2- 172.21.15.162:0/2951738585 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f87a0055440 0x7f87a0057900 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7f87ac001190 tx=0x7f87ac007740).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:53.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:53.972+0000 7f87b97fa700 1 -- 172.21.15.162:0/2951738585 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f87b4043de0 con 0x7f87bc0af640 2020-11-28T21:53:54.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.156+0000 7f87c08c6700 1 -- 172.21.15.162:0/2951738585 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87bc0b0010 con 0x7f87bc0af640 2020-11-28T21:53:54.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.156+0000 7f87b97fa700 1 -- 172.21.15.162:0/2951738585 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+2138 (secure 0 0 0) 0x7f87b4019030 con 0x7f87bc0af640 2020-11-28T21:53:54.161 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:53:54.161 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":41,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5795,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/2645168518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":2645168518},{"type":"v1","addr":"172.21.15.118:6833","nonce":2645168518}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36},{"gid":6541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1999073284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1999073284},{"type":"v1","addr":"172.21.15.162:6835","nonce":1999073284}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:53:36.539204+0000","modified":"2020-11-28T21:53:53.815270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-28T21:53:54.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.156+0000 7f87c08c6700 1 -- 172.21.15.162:0/2951738585 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f87a0055440 msgr2=0x7f87a0057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:54.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.156+0000 7f87c08c6700 1 --2- 172.21.15.162:0/2951738585 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f87a0055440 0x7f87a0057900 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7f87ac001190 tx=0x7f87ac007740).stop 2020-11-28T21:53:54.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.156+0000 7f87c08c6700 1 -- 172.21.15.162:0/2951738585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87bc0af640 msgr2=0x7f87bc14b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:54.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.156+0000 7f87c08c6700 1 --2- 172.21.15.162:0/2951738585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87bc0af640 0x7f87bc14b630 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 rx=0x7f87b4001cb0 tx=0x7f87b4002190).stop 2020-11-28T21:53:54.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.160+0000 7f87c08c6700 1 -- 172.21.15.162:0/2951738585 shutdown_connections 2020-11-28T21:53:54.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.160+0000 7f87c08c6700 1 --2- 172.21.15.162:0/2951738585 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f87a0055440 0x7f87a0057900 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:54.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.160+0000 7f87c08c6700 1 --2- 172.21.15.162:0/2951738585 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f87bc05cbd0 0x7f87bc090a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-28T21:53:54.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.160+0000 7f87c08c6700 1 --2- 172.21.15.162:0/2951738585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87bc0af640 0x7f87bc14b630 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:54.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.160+0000 7f87c08c6700 1 --2- 172.21.15.162:0/2951738585 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f87bc060cf0 0x7f87bc0921a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:54.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.160+0000 7f87c08c6700 1 -- 172.21.15.162:0/2951738585 >> 172.21.15.162:0/2951738585 conn(0x7f87bc05b970 msgr2=0x7f87bc0aa410 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:54.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.160+0000 7f87c08c6700 1 -- 172.21.15.162:0/2951738585 shutdown_connections 2020-11-28T21:53:54.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.160+0000 7f87c08c6700 1 -- 172.21.15.162:0/2951738585 wait complete. 2020-11-28T21:53:54.166 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 41 2020-11-28T21:53:54.174 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2020-11-28T21:53:54.287 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.284+0000 7f69891b4700 1 Processor -- start 2020-11-28T21:53:54.288 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.284+0000 7f69891b4700 1 -- start start 2020-11-28T21:53:54.289 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.284+0000 7f69891b4700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f698413fd00 0x7f698413c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:54.289 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.284+0000 7f69891b4700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6984067e90 con 0x7f698413c900 2020-11-28T21:53:54.289 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.284+0000 7f69891b4700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69840b6d80 con 0x7f69840b69a0 2020-11-28T21:53:54.290 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.284+0000 7f69891b4700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69840ae270 con 0x7f698413fd00 2020-11-28T21:53:54.290 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.284+0000 7f6982d9d700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f69840b69a0 0x7f698413f5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:49354/0 (socket says 172.21.15.162:49354) 2020-11-28T21:53:54.290 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.284+0000 7f6982d9d700 1 -- 172.21.15.162:0/3564786035 learned_addr learned my addr 172.21.15.162:0/3564786035 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:54.290 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.284+0000 7f698259c700 1 --2- 172.21.15.162:0/3564786035 >> v2:172.21.15.162:3300/0 conn(0x7f698413fd00 0x7f698413c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:54.292 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.284+0000 7f6981d9b700 1 -- 172.21.15.162:0/3564786035 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3777570829 0 0) 0x7f69840b6d80 con 0x7f69840b69a0 2020-11-28T21:53:54.292 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.284+0000 7f6981d9b700 1 -- 172.21.15.162:0/3564786035 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6968003230 con 0x7f69840b69a0 2020-11-28T21:53:54.292 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.284+0000 7f698259c700 1 -- 172.21.15.162:0/3564786035 >> v1:172.21.15.162:6789/0 conn(0x7f69840b69a0 legacy=0x7f698413f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:54.293 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.284+0000 7f6981d9b700 1 -- 172.21.15.162:0/3564786035 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4077335060 0 0) 0x7f6984067e90 con 0x7f698413c900 2020-11-28T21:53:54.293 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.284+0000 7f698259c700 1 -- 172.21.15.162:0/3564786035 >> v1:172.21.15.118:6789/0 conn(0x7f698413c900 legacy=0x7f6984149f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:54.293 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.284+0000 7f698259c700 1 -- 172.21.15.162:0/3564786035 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6984063600 con 0x7f698413fd00 2020-11-28T21:53:54.293 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.284+0000 7f698259c700 1 --2- 172.21.15.162:0/3564786035 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f698413fd00 0x7f698413c410 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 rx=0x7f6978002f30 tx=0x7f6978006dc0).ready entity=mon.1 client_cookie=183d1d5947afa2e3 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:54.293 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.288+0000 7f6981d9b700 1 -- 172.21.15.162:0/3564786035 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6978009800 con 0x7f698413fd00 2020-11-28T21:53:54.294 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.288+0000 7f6981d9b700 1 -- 172.21.15.162:0/3564786035 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6978009960 con 0x7f698413fd00 2020-11-28T21:53:54.294 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.288+0000 7f6981d9b700 1 -- 172.21.15.162:0/3564786035 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6978009cc0 con 0x7f698413fd00 2020-11-28T21:53:54.294 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.288+0000 7f69891b4700 1 -- 172.21.15.162:0/3564786035 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f698413fd00 msgr2=0x7f698413c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:54.294 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.288+0000 7f69891b4700 1 --2- 172.21.15.162:0/3564786035 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f698413fd00 0x7f698413c410 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 rx=0x7f6978002f30 tx=0x7f6978006dc0).stop 2020-11-28T21:53:54.295 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.288+0000 7f69891b4700 1 -- 172.21.15.162:0/3564786035 shutdown_connections 2020-11-28T21:53:54.295 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.288+0000 7f69891b4700 1 --2- 172.21.15.162:0/3564786035 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f698413fd00 0x7f698413c410 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:54.295 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.288+0000 7f69891b4700 1 -- 172.21.15.162:0/3564786035 >> 172.21.15.162:0/3564786035 conn(0x7f6984062170 msgr2=0x7f69840b5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:54.295 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.288+0000 7f69891b4700 1 -- 172.21.15.162:0/3564786035 shutdown_connections 2020-11-28T21:53:54.296 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.288+0000 7f69891b4700 1 -- 172.21.15.162:0/3564786035 wait complete. 2020-11-28T21:53:54.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.288+0000 7f69891b4700 1 Processor -- start 2020-11-28T21:53:54.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.288+0000 7f69891b4700 1 -- start start 2020-11-28T21:53:54.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.288+0000 7f69891b4700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69840b69a0 0x7f6984145610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:54.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.288+0000 7f69891b4700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f698413c900 0x7f69841509c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:54.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.288+0000 7f69891b4700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f698413fd00 0x7f6984150eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:54.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.288+0000 7f69891b4700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69840b1a20 con 0x7f698413c900 2020-11-28T21:53:54.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.288+0000 7f69891b4700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69840b1780 con 0x7f69840b69a0 2020-11-28T21:53:54.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.288+0000 7f69891b4700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69840b1d30 con 0x7f698413fd00 2020-11-28T21:53:54.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.288+0000 7f6982d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69840b69a0 0x7f6984145610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:54.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.292+0000 7f6982d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69840b69a0 0x7f6984145610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42516/0 (socket says 172.21.15.162:42516) 2020-11-28T21:53:54.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.292+0000 7f698259c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f698413c900 0x7f69841509c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:54.306 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.292+0000 7f6982d9d700 1 -- 172.21.15.162:0/570616536 learned_addr learned my addr 172.21.15.162:0/570616536 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:54.307 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.292+0000 7f698359e700 1 --2- 172.21.15.162:0/570616536 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f698413fd00 0x7f6984150eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:54.307 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.292+0000 7f6982d9d700 1 -- 172.21.15.162:0/570616536 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f698413fd00 msgr2=0x7f6984150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:54.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.292+0000 7f6982d9d700 1 --2- 172.21.15.162:0/570616536 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f698413fd00 0x7f6984150eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:54.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.292+0000 7f6982d9d700 1 -- 172.21.15.162:0/570616536 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f698413c900 msgr2=0x7f69841509c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:54.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.292+0000 7f6982d9d700 1 --2- 172.21.15.162:0/570616536 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f698413c900 0x7f69841509c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:54.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.292+0000 7f6982d9d700 1 -- 172.21.15.162:0/570616536 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69840b1fd0 con 0x7f69840b69a0 2020-11-28T21:53:54.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.292+0000 7f698359e700 1 --2- 172.21.15.162:0/570616536 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f698413fd00 0x7f6984150eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:53:54.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.292+0000 7f6982d9d700 1 --2- 172.21.15.162:0/570616536 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69840b69a0 0x7f6984145610 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 rx=0x7f69740010d0 tx=0x7f6974007a70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:54.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.292+0000 7f6973fff700 1 -- 172.21.15.162:0/570616536 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69740072e0 con 0x7f69840b69a0 2020-11-28T21:53:54.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.292+0000 7f6973fff700 1 -- 172.21.15.162:0/570616536 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6974007440 con 0x7f69840b69a0 2020-11-28T21:53:54.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.292+0000 7f6973fff700 1 -- 172.21.15.162:0/570616536 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69740057d0 con 0x7f69840b69a0 2020-11-28T21:53:54.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.292+0000 7f69891b4700 1 -- 172.21.15.162:0/570616536 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f698413e350 con 0x7f69840b69a0 2020-11-28T21:53:54.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.292+0000 7f69891b4700 1 -- 172.21.15.162:0/570616536 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69840b1400 con 0x7f69840b69a0 2020-11-28T21:53:54.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.292+0000 7f69891b4700 1 -- 172.21.15.162:0/570616536 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69840b2160 con 0x7f69840b69a0 2020-11-28T21:53:54.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.292+0000 7f6973fff700 1 -- 172.21.15.162:0/570616536 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6974010070 con 0x7f69840b69a0 2020-11-28T21:53:54.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.296+0000 7f6973fff700 1 --2- 172.21.15.162:0/570616536 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6960055330 0x7f69600577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:54.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.296+0000 7f6973fff700 1 -- 172.21.15.162:0/570616536 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f697406d660 con 0x7f69840b69a0 2020-11-28T21:53:54.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.296+0000 7f698259c700 1 --2- 172.21.15.162:0/570616536 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6960055330 0x7f69600577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:54.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.296+0000 7f698259c700 1 --2- 172.21.15.162:0/570616536 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6960055330 0x7f69600577f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f6978019000 tx=0x7f6978015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:54.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.300+0000 7f6973fff700 1 -- 172.21.15.162:0/570616536 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6974043870 con 0x7f69840b69a0 2020-11-28T21:53:54.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.484+0000 7f69891b4700 1 -- 172.21.15.162:0/570616536 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6984057050 con 0x7f69840b69a0 2020-11-28T21:53:54.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.828+0000 7f6973fff700 1 -- 172.21.15.162:0/570616536 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v42) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6974018020 con 0x7f69840b69a0 2020-11-28T21:53:54.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.832+0000 7f69891b4700 1 -- 172.21.15.162:0/570616536 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6960055330 msgr2=0x7f69600577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:54.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.832+0000 7f69891b4700 1 --2- 172.21.15.162:0/570616536 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6960055330 0x7f69600577f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f6978019000 tx=0x7f6978015000).stop 2020-11-28T21:53:54.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.832+0000 7f69891b4700 1 -- 172.21.15.162:0/570616536 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69840b69a0 msgr2=0x7f6984145610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:54.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.832+0000 7f69891b4700 1 --2- 172.21.15.162:0/570616536 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69840b69a0 0x7f6984145610 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 rx=0x7f69740010d0 tx=0x7f6974007a70).stop 2020-11-28T21:53:54.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.832+0000 7f69891b4700 1 -- 172.21.15.162:0/570616536 shutdown_connections 2020-11-28T21:53:54.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.832+0000 7f69891b4700 1 --2- 172.21.15.162:0/570616536 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6960055330 0x7f69600577f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:54.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.832+0000 7f69891b4700 1 --2- 172.21.15.162:0/570616536 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f698413fd00 0x7f6984150eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:54.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.832+0000 7f69891b4700 1 --2- 172.21.15.162:0/570616536 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69840b69a0 0x7f6984145610 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:54.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.832+0000 7f69891b4700 1 --2- 172.21.15.162:0/570616536 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f698413c900 0x7f69841509c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:54.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.832+0000 7f69891b4700 1 -- 172.21.15.162:0/570616536 >> 172.21.15.162:0/570616536 conn(0x7f6984062170 msgr2=0x7f69840b5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:54.838 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.832+0000 7f69891b4700 1 -- 172.21.15.162:0/570616536 shutdown_connections 2020-11-28T21:53:54.838 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.832+0000 7f69891b4700 1 -- 172.21.15.162:0/570616536 wait complete. 2020-11-28T21:53:54.854 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-11-28T21:53:54.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.968+0000 7f24da214700 1 Processor -- start 2020-11-28T21:53:54.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.968+0000 7f24da214700 1 -- start start 2020-11-28T21:53:54.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.968+0000 7f24da214700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f24d40b9ce0 0x7f24d41433c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:54.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.972+0000 7f24da214700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24d4143950 con 0x7f24d40605c0 2020-11-28T21:53:54.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.972+0000 7f24da214700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24d4067aa0 con 0x7f24d4067610 2020-11-28T21:53:54.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.972+0000 7f24da214700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24d40b4340 con 0x7f24d40b9ce0 2020-11-28T21:53:54.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.972+0000 7f24d3fff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f24d4067610 0x7f24d413fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:49372/0 (socket says 172.21.15.162:49372) 2020-11-28T21:53:54.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.972+0000 7f24d3fff700 1 -- 172.21.15.162:0/3152969629 learned_addr learned my addr 172.21.15.162:0/3152969629 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:54.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.972+0000 7f24d37fe700 1 --2- 172.21.15.162:0/3152969629 >> v2:172.21.15.162:3300/0 conn(0x7f24d40b9ce0 0x7f24d41433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:54.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.972+0000 7f24d27fc700 1 -- 172.21.15.162:0/3152969629 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3271820168 0 0) 0x7f24d4067aa0 con 0x7f24d4067610 2020-11-28T21:53:54.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.972+0000 7f24d27fc700 1 -- 172.21.15.162:0/3152969629 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24b4003230 con 0x7f24d4067610 2020-11-28T21:53:54.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.972+0000 7f24d37fe700 1 -- 172.21.15.162:0/3152969629 >> v1:172.21.15.162:6789/0 conn(0x7f24d4067610 legacy=0x7f24d413fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:54.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.972+0000 7f24d27fc700 1 -- 172.21.15.162:0/3152969629 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3574438310 0 0) 0x7f24b4003230 con 0x7f24d4067610 2020-11-28T21:53:54.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.972+0000 7f24d37fe700 1 -- 172.21.15.162:0/3152969629 >> v1:172.21.15.118:6789/0 conn(0x7f24d40605c0 legacy=0x7f24d40b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:54.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.972+0000 7f24d37fe700 1 -- 172.21.15.162:0/3152969629 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d40ba0c0 con 0x7f24d40b9ce0 2020-11-28T21:53:54.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.972+0000 7f24d37fe700 1 --2- 172.21.15.162:0/3152969629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24d40b9ce0 0x7f24d41433c0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 rx=0x7f24bc0010d0 tx=0x7f24c40091a0).ready entity=mon.1 client_cookie=b08f66feeb2b0502 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:54.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.972+0000 7f24d27fc700 1 -- 172.21.15.162:0/3152969629 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24cc0016d0 con 0x7f24d40b9ce0 2020-11-28T21:53:54.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.972+0000 7f24d27fc700 1 -- 172.21.15.162:0/3152969629 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24c400b860 con 0x7f24d40b9ce0 2020-11-28T21:53:54.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.972+0000 7f24d27fc700 1 -- 172.21.15.162:0/3152969629 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24c400b9d0 con 0x7f24d40b9ce0 2020-11-28T21:53:54.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.972+0000 7f24da214700 1 -- 172.21.15.162:0/3152969629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24d40b9ce0 msgr2=0x7f24d41433c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:54.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.972+0000 7f24da214700 1 --2- 172.21.15.162:0/3152969629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24d40b9ce0 0x7f24d41433c0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 rx=0x7f24bc0010d0 tx=0x7f24c40091a0).stop 2020-11-28T21:53:54.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.972+0000 7f24da214700 1 -- 172.21.15.162:0/3152969629 shutdown_connections 2020-11-28T21:53:54.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.972+0000 7f24da214700 1 --2- 172.21.15.162:0/3152969629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24d40b9ce0 0x7f24d41433c0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:54.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.972+0000 7f24da214700 1 -- 172.21.15.162:0/3152969629 >> 172.21.15.162:0/3152969629 conn(0x7f24d405c430 msgr2=0x7f24d40b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:54.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.972+0000 7f24da214700 1 -- 172.21.15.162:0/3152969629 shutdown_connections 2020-11-28T21:53:54.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.972+0000 7f24da214700 1 -- 172.21.15.162:0/3152969629 wait complete. 2020-11-28T21:53:54.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.976+0000 7f24da214700 1 Processor -- start 2020-11-28T21:53:54.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.976+0000 7f24da214700 1 -- start start 2020-11-28T21:53:54.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.976+0000 7f24da214700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24d40605c0 0x7f24d409e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:54.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.976+0000 7f24da214700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24d4067610 0x7f24d41550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:54.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.976+0000 7f24da214700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f24d40b9ce0 0x7f24d41555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:54.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.976+0000 7f24da214700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d4145790 con 0x7f24d4067610 2020-11-28T21:53:54.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.976+0000 7f24da214700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d4151820 con 0x7f24d40605c0 2020-11-28T21:53:54.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.976+0000 7f24da214700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24d4143ec0 con 0x7f24d40b9ce0 2020-11-28T21:53:54.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.976+0000 7f24d37fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24d40605c0 0x7f24d409e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:54.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.976+0000 7f24d37fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24d40605c0 0x7f24d409e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42530/0 (socket says 172.21.15.162:42530) 2020-11-28T21:53:54.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.976+0000 7f24d37fe700 1 -- 172.21.15.162:0/220035743 learned_addr learned my addr 172.21.15.162:0/220035743 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:54.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.976+0000 7f24d37fe700 1 -- 172.21.15.162:0/220035743 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f24d40b9ce0 msgr2=0x7f24d41555e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:54.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.976+0000 7f24d37fe700 1 --2- 172.21.15.162:0/220035743 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f24d40b9ce0 0x7f24d41555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-28T21:53:54.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.976+0000 7f24d37fe700 1 -- 172.21.15.162:0/220035743 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24d4067610 msgr2=0x7f24d41550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:54.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.976+0000 7f24d2ffd700 1 --2- 172.21.15.162:0/220035743 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24d4067610 0x7f24d41550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:54.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.976+0000 7f24d37fe700 1 --2- 172.21.15.162:0/220035743 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24d4067610 0x7f24d41550f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:54.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.976+0000 7f24d37fe700 1 -- 172.21.15.162:0/220035743 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d40ae620 con 0x7f24d40605c0 2020-11-28T21:53:54.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.976+0000 7f24d2ffd700 1 --2- 172.21.15.162:0/220035743 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24d4067610 0x7f24d41550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:53:54.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.976+0000 7f24d37fe700 1 --2- 172.21.15.162:0/220035743 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24d40605c0 0x7f24d409e170 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 rx=0x7f24bc0010d0 tx=0x7f24c40091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:54.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.976+0000 7f24d0ff9700 1 -- 172.21.15.162:0/220035743 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24c4011c70 con 0x7f24d40605c0 2020-11-28T21:53:54.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.976+0000 7f24d0ff9700 1 -- 172.21.15.162:0/220035743 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24c4011dd0 con 0x7f24d40605c0 2020-11-28T21:53:54.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.976+0000 7f24da214700 1 -- 172.21.15.162:0/220035743 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24d40ae8c0 con 0x7f24d40605c0 2020-11-28T21:53:54.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.976+0000 7f24d0ff9700 1 -- 172.21.15.162:0/220035743 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24c401d640 con 0x7f24d40605c0 2020-11-28T21:53:54.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.976+0000 7f24da214700 1 -- 172.21.15.162:0/220035743 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d409f130 con 0x7f24d40605c0 2020-11-28T21:53:54.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.976+0000 7f24d0ff9700 1 -- 172.21.15.162:0/220035743 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f24c400f070 con 0x7f24d40605c0 2020-11-28T21:53:54.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.976+0000 7f24da214700 1 -- 172.21.15.162:0/220035743 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24d4057050 con 0x7f24d40605c0 2020-11-28T21:53:54.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.980+0000 7f24d0ff9700 1 --2- 172.21.15.162:0/220035743 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f24b80554e0 0x7f24b80579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:54.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.980+0000 7f24d0ff9700 1 -- 172.21.15.162:0/220035743 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f24c406fcf0 con 0x7f24d40605c0 2020-11-28T21:53:54.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.980+0000 7f24d2ffd700 1 --2- 172.21.15.162:0/220035743 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f24b80554e0 0x7f24b80579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:54.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.980+0000 7f24d2ffd700 1 --2- 172.21.15.162:0/220035743 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f24b80554e0 0x7f24b80579a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f24bc008000 tx=0x7f24bc009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:54.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:54.984+0000 7f24d0ff9700 1 -- 172.21.15.162:0/220035743 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f24c4045e70 con 0x7f24d40605c0 2020-11-28T21:53:55.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.124+0000 7f24da214700 1 -- 172.21.15.162:0/220035743 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f24d4068350 con 0x7f24d40605c0 2020-11-28T21:53:55.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.836+0000 7f24d0ff9700 1 -- 172.21.15.162:0/220035743 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v90) v1 ==== 185+0+0 (secure 0 0 0) 0x7f24c4049450 con 0x7f24d40605c0 2020-11-28T21:53:55.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.840+0000 7f24da214700 1 -- 172.21.15.162:0/220035743 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f24b80554e0 msgr2=0x7f24b80579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:55.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.840+0000 7f24da214700 1 --2- 172.21.15.162:0/220035743 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f24b80554e0 0x7f24b80579a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f24bc008000 tx=0x7f24bc009040).stop 2020-11-28T21:53:55.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.840+0000 7f24da214700 1 -- 172.21.15.162:0/220035743 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24d40605c0 msgr2=0x7f24d409e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:55.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.840+0000 7f24da214700 1 --2- 172.21.15.162:0/220035743 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24d40605c0 0x7f24d409e170 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 rx=0x7f24bc0010d0 tx=0x7f24c40091a0).stop 2020-11-28T21:53:55.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.840+0000 7f24da214700 1 -- 172.21.15.162:0/220035743 shutdown_connections 2020-11-28T21:53:55.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.840+0000 7f24da214700 1 --2- 172.21.15.162:0/220035743 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f24b80554e0 0x7f24b80579a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:55.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.840+0000 7f24da214700 1 --2- 172.21.15.162:0/220035743 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f24d40b9ce0 0x7f24d41555e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-28T21:53:55.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.840+0000 7f24da214700 1 --2- 172.21.15.162:0/220035743 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24d40605c0 0x7f24d409e170 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:55.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.840+0000 7f24da214700 1 --2- 172.21.15.162:0/220035743 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24d4067610 0x7f24d41550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:55.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.840+0000 7f24da214700 1 -- 172.21.15.162:0/220035743 >> 172.21.15.162:0/220035743 conn(0x7f24d405c430 msgr2=0x7f24d405f750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:55.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.840+0000 7f24da214700 1 -- 172.21.15.162:0/220035743 shutdown_connections 2020-11-28T21:53:55.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.840+0000 7f24da214700 1 -- 172.21.15.162:0/220035743 wait complete. 2020-11-28T21:53:55.848 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_metadata' removed 2020-11-28T21:53:55.861 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-11-28T21:53:55.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.980+0000 7f7ffffff700 1 Processor -- start 2020-11-28T21:53:55.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.980+0000 7f7ffffff700 1 -- start start 2020-11-28T21:53:55.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.984+0000 7f7ffffff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f800005bcb0 0x7f8000061300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:55.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.984+0000 7f7ffffff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f80000b21a0 0x7f80000abfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:55.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.984+0000 7f7ffffff700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f800005c090 con 0x7f8000061a50 2020-11-28T21:53:55.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.984+0000 7f7ffffff700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8000143820 con 0x7f80000b21a0 2020-11-28T21:53:55.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.984+0000 7f7ffffff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80000617f0 con 0x7f800005bcb0 2020-11-28T21:53:55.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.984+0000 7f7ffe7fc700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f8000061a50 0x7f80001427d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:49382/0 (socket says 172.21.15.162:49382) 2020-11-28T21:53:55.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.984+0000 7f7ffe7fc700 1 -- 172.21.15.162:0/1162752336 learned_addr learned my addr 172.21.15.162:0/1162752336 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:55.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.984+0000 7f7fff7fe700 1 --2- 172.21.15.162:0/1162752336 >> v2:172.21.15.162:3300/0 conn(0x7f80000b21a0 0x7f80000abfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:55.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.984+0000 7f7ffeffd700 1 --2- 172.21.15.162:0/1162752336 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f800005bcb0 0x7f8000061300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:55.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.984+0000 7f7fff7fe700 1 -- 172.21.15.162:0/1162752336 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f800005bcb0 msgr2=0x7f8000061300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:55.992 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.984+0000 7f7fff7fe700 1 --2- 172.21.15.162:0/1162752336 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f800005bcb0 0x7f8000061300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:55.992 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.984+0000 7f7fff7fe700 1 -- 172.21.15.162:0/1162752336 >> v1:172.21.15.162:6789/0 conn(0x7f8000061a50 legacy=0x7f80001427d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:55.992 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.984+0000 7f7fff7fe700 1 -- 172.21.15.162:0/1162752336 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8000061f60 con 0x7f80000b21a0 2020-11-28T21:53:55.992 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.984+0000 7f7fff7fe700 1 --2- 172.21.15.162:0/1162752336 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f80000b21a0 0x7f80000abfb0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 rx=0x7f7ff80032b0 tx=0x7f7ff80091a0).ready entity=mon.1 client_cookie=7cd46a2ac7bc79b8 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:55.993 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.984+0000 7f7ffdffb700 1 -- 172.21.15.162:0/1162752336 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ff800ba50 con 0x7f80000b21a0 2020-11-28T21:53:55.993 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.984+0000 7f7ffdffb700 1 -- 172.21.15.162:0/1162752336 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ff800bbb0 con 0x7f80000b21a0 2020-11-28T21:53:55.993 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.984+0000 7f7ffdffb700 1 -- 172.21.15.162:0/1162752336 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ff800bd20 con 0x7f80000b21a0 2020-11-28T21:53:55.993 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.984+0000 7f7ffffff700 1 -- 172.21.15.162:0/1162752336 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f80000b21a0 msgr2=0x7f80000abfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:55.994 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.984+0000 7f7ffffff700 1 --2- 172.21.15.162:0/1162752336 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f80000b21a0 0x7f80000abfb0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 rx=0x7f7ff80032b0 tx=0x7f7ff80091a0).stop 2020-11-28T21:53:55.994 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.984+0000 7f7ffffff700 1 -- 172.21.15.162:0/1162752336 shutdown_connections 2020-11-28T21:53:55.994 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.984+0000 7f7ffffff700 1 --2- 172.21.15.162:0/1162752336 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f80000b21a0 0x7f80000abfb0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:55.994 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.984+0000 7f7ffffff700 1 --2- 172.21.15.162:0/1162752336 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f800005bcb0 0x7f8000061300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:55.994 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.984+0000 7f7ffffff700 1 -- 172.21.15.162:0/1162752336 >> 172.21.15.162:0/1162752336 conn(0x7f80000b2830 msgr2=0x7f80000b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:55.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.984+0000 7f7ffffff700 1 -- 172.21.15.162:0/1162752336 shutdown_connections 2020-11-28T21:53:55.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.984+0000 7f7ffffff700 1 -- 172.21.15.162:0/1162752336 wait complete. 2020-11-28T21:53:55.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.988+0000 7f7ffffff700 1 Processor -- start 2020-11-28T21:53:55.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.988+0000 7f7ffffff700 1 -- start start 2020-11-28T21:53:55.998 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.988+0000 7f7ffffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f800005bcb0 0x7f8000142110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:55.998 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.988+0000 7f7ffffff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8000061a50 0x7f80000ae730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:55.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.988+0000 7f7ffffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f80000b21a0 0x7f80001514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:55.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.988+0000 7f7ffffff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8000154a90 con 0x7f80000b21a0 2020-11-28T21:53:55.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.988+0000 7f7ffffff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8000154d30 con 0x7f800005bcb0 2020-11-28T21:53:55.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.988+0000 7f7ffffff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8000154fd0 con 0x7f8000061a50 2020-11-28T21:53:56.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.988+0000 7f7ffeffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f800005bcb0 0x7f8000142110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:56.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.988+0000 7f7ffeffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f800005bcb0 0x7f8000142110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42544/0 (socket says 172.21.15.162:42544) 2020-11-28T21:53:56.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.988+0000 7f7ffeffd700 1 -- 172.21.15.162:0/3540476701 learned_addr learned my addr 172.21.15.162:0/3540476701 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:56.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.988+0000 7f7ffe7fc700 1 --2- 172.21.15.162:0/3540476701 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8000061a50 0x7f80000ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:56.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.988+0000 7f7fff7fe700 1 --2- 172.21.15.162:0/3540476701 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f80000b21a0 0x7f80001514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:56.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.988+0000 7f7ffeffd700 1 -- 172.21.15.162:0/3540476701 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8000061a50 msgr2=0x7f80000ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:56.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.988+0000 7f7ffeffd700 1 --2- 172.21.15.162:0/3540476701 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8000061a50 0x7f80000ae730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:56.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.988+0000 7f7ffeffd700 1 -- 172.21.15.162:0/3540476701 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f80000b21a0 msgr2=0x7f80001514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:56.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.988+0000 7f7ffeffd700 1 --2- 172.21.15.162:0/3540476701 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f80000b21a0 0x7f80001514a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:56.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.988+0000 7f7ffeffd700 1 -- 172.21.15.162:0/3540476701 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8000155270 con 0x7f800005bcb0 2020-11-28T21:53:56.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.988+0000 7f7ffeffd700 1 --2- 172.21.15.162:0/3540476701 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f800005bcb0 0x7f8000142110 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 rx=0x7f7ff0003070 tx=0x7f7ff0008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:56.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.992+0000 7f7ff7fff700 1 -- 172.21.15.162:0/3540476701 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ff0009790 con 0x7f800005bcb0 2020-11-28T21:53:56.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.992+0000 7f7ff7fff700 1 -- 172.21.15.162:0/3540476701 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ff00098f0 con 0x7f800005bcb0 2020-11-28T21:53:56.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.992+0000 7f7ffffff700 1 -- 172.21.15.162:0/3540476701 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80001400b0 con 0x7f800005bcb0 2020-11-28T21:53:56.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.992+0000 7f7ff7fff700 1 -- 172.21.15.162:0/3540476701 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ff0009c50 con 0x7f800005bcb0 2020-11-28T21:53:56.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.992+0000 7f7ffffff700 1 -- 172.21.15.162:0/3540476701 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80000aec20 con 0x7f800005bcb0 2020-11-28T21:53:56.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.992+0000 7f7ffffff700 1 -- 172.21.15.162:0/3540476701 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8000057050 con 0x7f800005bcb0 2020-11-28T21:53:56.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.992+0000 7f7ff7fff700 1 -- 172.21.15.162:0/3540476701 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7ff0013480 con 0x7f800005bcb0 2020-11-28T21:53:56.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.992+0000 7f7ff7fff700 1 --2- 172.21.15.162:0/3540476701 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7fe4055440 0x7f7fe4057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:56.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.992+0000 7f7ff7fff700 1 -- 172.21.15.162:0/3540476701 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5721+0+0 (secure 0 0 0) 0x7f7ff006ed00 con 0x7f800005bcb0 2020-11-28T21:53:56.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.992+0000 7f7ffe7fc700 1 --2- 172.21.15.162:0/3540476701 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7fe4055440 0x7f7fe4057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:56.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:55.992+0000 7f7ffe7fc700 1 --2- 172.21.15.162:0/3540476701 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7fe4055440 0x7f7fe4057900 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f7fe800d000 tx=0x7f7fe8007b70).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:56.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:56.000+0000 7f7ff7fff700 1 -- 172.21.15.162:0/3540476701 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7ff0045100 con 0x7f800005bcb0 2020-11-28T21:53:56.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:56.136+0000 7f7ffffff700 1 -- 172.21.15.162:0/3540476701 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8000068350 con 0x7f800005bcb0 2020-11-28T21:53:56.853 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:56.848+0000 7f7ff7fff700 1 -- 172.21.15.162:0/3540476701 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v91) v1 ==== 173+0+0 (secure 0 0 0) 0x7f7ff0009db0 con 0x7f800005bcb0 2020-11-28T21:53:56.856 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:56.852+0000 7f7ffffff700 1 -- 172.21.15.162:0/3540476701 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7fe4055440 msgr2=0x7f7fe4057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:56.856 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:56.852+0000 7f7ffffff700 1 --2- 172.21.15.162:0/3540476701 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7fe4055440 0x7f7fe4057900 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f7fe800d000 tx=0x7f7fe8007b70).stop 2020-11-28T21:53:56.856 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:56.852+0000 7f7ffffff700 1 -- 172.21.15.162:0/3540476701 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f800005bcb0 msgr2=0x7f8000142110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:56.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:56.852+0000 7f7ffffff700 1 --2- 172.21.15.162:0/3540476701 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f800005bcb0 0x7f8000142110 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 rx=0x7f7ff0003070 tx=0x7f7ff0008040).stop 2020-11-28T21:53:56.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:56.852+0000 7f7ffffff700 1 -- 172.21.15.162:0/3540476701 shutdown_connections 2020-11-28T21:53:56.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:56.852+0000 7f7ffffff700 1 --2- 172.21.15.162:0/3540476701 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7fe4055440 0x7f7fe4057900 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:56.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:56.852+0000 7f7ffffff700 1 --2- 172.21.15.162:0/3540476701 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f80000b21a0 0x7f80001514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:56.858 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:56.852+0000 7f7ffffff700 1 --2- 172.21.15.162:0/3540476701 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f800005bcb0 0x7f8000142110 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:56.858 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:56.852+0000 7f7ffffff700 1 --2- 172.21.15.162:0/3540476701 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8000061a50 0x7f80000ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:56.858 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:56.852+0000 7f7ffffff700 1 -- 172.21.15.162:0/3540476701 >> 172.21.15.162:0/3540476701 conn(0x7f80000b2830 msgr2=0x7f800013a650 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:56.858 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:56.852+0000 7f7ffffff700 1 -- 172.21.15.162:0/3540476701 shutdown_connections 2020-11-28T21:53:56.858 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:56.852+0000 7f7ffffff700 1 -- 172.21.15.162:0/3540476701 wait complete. 2020-11-28T21:53:56.859 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_data' removed 2020-11-28T21:53:56.873 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-28T21:53:56.873 INFO:tasks.ceph.mds.a:Stopping old one... 2020-11-28T21:53:56.874 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-11-28T21:53:56.874 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T21:53:56.875 INFO:tasks.ceph.mds.c:Restarting daemon 2020-11-28T21:53:56.875 INFO:tasks.ceph.mds.c:Stopping old one... 2020-11-28T21:53:56.876 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2020-11-28T21:53:56.877 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T21:53:56.877 INFO:tasks.ceph.mds.b:Restarting daemon 2020-11-28T21:53:56.878 INFO:tasks.ceph.mds.b:Stopping old one... 2020-11-28T21:53:56.878 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-11-28T21:53:56.879 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T21:53:56.888 INFO:tasks.ceph.mds.a:Stopped 2020-11-28T21:53:56.889 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-28T21:53:56.890 INFO:tasks.ceph.mds.c:Stopped 2020-11-28T21:53:56.891 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2020-11-28T21:53:56.893 INFO:tasks.ceph.mds.b:Stopped 2020-11-28T21:53:56.893 INFO:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-11-28T21:53:56.932 INFO:tasks.ceph.mds.a:Started 2020-11-28T21:53:56.932 DEBUG:teuthology.parallel:result is None 2020-11-28T21:53:56.933 INFO:tasks.ceph.mds.c:Started 2020-11-28T21:53:56.933 DEBUG:teuthology.parallel:result is None 2020-11-28T21:53:56.937 INFO:tasks.ceph.mds.b:Started 2020-11-28T21:53:56.937 DEBUG:teuthology.parallel:result is None 2020-11-28T21:53:56.938 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist clear 2020-11-28T21:53:57.003 INFO:tasks.ceph.mds.c.smithi162.stdout:starting mds.c at 2020-11-28T21:53:57.012 INFO:tasks.ceph.mds.a.smithi162.stdout:starting mds.a at 2020-11-28T21:53:57.015 INFO:tasks.ceph.mds.b.smithi118.stdout:starting mds.b at 2020-11-28T21:53:57.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.048+0000 7f291f1fb700 1 Processor -- start 2020-11-28T21:53:57.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.048+0000 7f291f1fb700 1 -- start start 2020-11-28T21:53:57.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.048+0000 7f291f1fb700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f29180611e0 0x7f29180615c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:57.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.048+0000 7f291f1fb700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f29180b4b40 0x7f291813ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:57.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.048+0000 7f291f1fb700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29180b1790 con 0x7f29180b10f0 2020-11-28T21:53:57.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.048+0000 7f291f1fb700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29180b1580 con 0x7f29180611e0 2020-11-28T21:53:57.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.048+0000 7f291f1fb700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29180b1a30 con 0x7f29180b4b40 2020-11-28T21:53:57.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.048+0000 7f291e9fa700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f29180b4b40 0x7f291813ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:57.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.048+0000 7f291e9fa700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f29180b4b40 0x7f291813ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42592/0 (socket says 172.21.15.162:42592) 2020-11-28T21:53:57.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.048+0000 7f291e9fa700 1 -- 172.21.15.162:0/1177923253 learned_addr learned my addr 172.21.15.162:0/1177923253 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:57.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.048+0000 7f291e1f9700 1 --2- 172.21.15.162:0/1177923253 >> v2:172.21.15.118:3300/0 conn(0x7f29180611e0 0x7f29180615c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:57.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.048+0000 7f291e9fa700 1 -- 172.21.15.162:0/1177923253 >> v2:172.21.15.118:3300/0 conn(0x7f29180611e0 msgr2=0x7f29180615c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:57.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.048+0000 7f291e9fa700 1 --2- 172.21.15.162:0/1177923253 >> v2:172.21.15.118:3300/0 conn(0x7f29180611e0 0x7f29180615c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:57.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.048+0000 7f291e9fa700 1 -- 172.21.15.162:0/1177923253 >> v1:172.21.15.118:6789/0 conn(0x7f29180b10f0 legacy=0x7f29180b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:57.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.048+0000 7f291e9fa700 1 -- 172.21.15.162:0/1177923253 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29180b3f90 con 0x7f29180b4b40 2020-11-28T21:53:57.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.048+0000 7f291e9fa700 1 --2- 172.21.15.162:0/1177923253 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29180b4b40 0x7f291813ee60 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 rx=0x7f29100032b0 tx=0x7f2910009270).ready entity=mon.1 client_cookie=b6a7314ada90693a server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:57.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.052+0000 7f291d1f7700 1 -- 172.21.15.162:0/1177923253 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f291000bc00 con 0x7f29180b4b40 2020-11-28T21:53:57.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.052+0000 7f291d1f7700 1 -- 172.21.15.162:0/1177923253 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f291000bd60 con 0x7f29180b4b40 2020-11-28T21:53:57.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.052+0000 7f291d1f7700 1 -- 172.21.15.162:0/1177923253 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2910011400 con 0x7f29180b4b40 2020-11-28T21:53:57.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.052+0000 7f291f1fb700 1 -- 172.21.15.162:0/1177923253 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29180b4b40 msgr2=0x7f291813ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:57.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.052+0000 7f291f1fb700 1 --2- 172.21.15.162:0/1177923253 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29180b4b40 0x7f291813ee60 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 rx=0x7f29100032b0 tx=0x7f2910009270).stop 2020-11-28T21:53:57.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.052+0000 7f291f1fb700 1 -- 172.21.15.162:0/1177923253 shutdown_connections 2020-11-28T21:53:57.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.052+0000 7f291f1fb700 1 --2- 172.21.15.162:0/1177923253 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29180b4b40 0x7f291813ee60 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:57.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.052+0000 7f291f1fb700 1 --2- 172.21.15.162:0/1177923253 >> v2:172.21.15.118:3300/0 conn(0x7f29180611e0 0x7f29180615c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:57.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.052+0000 7f291f1fb700 1 -- 172.21.15.162:0/1177923253 >> 172.21.15.162:0/1177923253 conn(0x7f29180b54f0 msgr2=0x7f29180b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:57.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.052+0000 7f291f1fb700 1 -- 172.21.15.162:0/1177923253 shutdown_connections 2020-11-28T21:53:57.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.052+0000 7f291f1fb700 1 -- 172.21.15.162:0/1177923253 wait complete. 2020-11-28T21:53:57.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.052+0000 7f291f1fb700 1 Processor -- start 2020-11-28T21:53:57.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.052+0000 7f291f1fb700 1 -- start start 2020-11-28T21:53:57.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.052+0000 7f291f1fb700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29180611e0 0x7f291813bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:57.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.052+0000 7f291f1fb700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f29180b10f0 0x7f2918151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:57.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.052+0000 7f291f1fb700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f29180b4b40 0x7f2918152290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:57.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.052+0000 7f291f1fb700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2918142d40 con 0x7f29180b4b40 2020-11-28T21:53:57.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.052+0000 7f291f1fb700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2918099870 con 0x7f29180611e0 2020-11-28T21:53:57.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.052+0000 7f291f1fb700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29180994d0 con 0x7f29180b10f0 2020-11-28T21:53:57.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.052+0000 7f291e1f9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29180611e0 0x7f291813bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:57.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.052+0000 7f291e1f9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29180611e0 0x7f291813bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42594/0 (socket says 172.21.15.162:42594) 2020-11-28T21:53:57.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.052+0000 7f291e1f9700 1 -- 172.21.15.162:0/1128743466 learned_addr learned my addr 172.21.15.162:0/1128743466 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:57.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.052+0000 7f291e9fa700 1 --2- 172.21.15.162:0/1128743466 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f29180b4b40 0x7f2918152290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:57.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.052+0000 7f291d9f8700 1 --2- 172.21.15.162:0/1128743466 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f29180b10f0 0x7f2918151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:57.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.052+0000 7f291e1f9700 1 -- 172.21.15.162:0/1128743466 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f29180b10f0 msgr2=0x7f2918151da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:57.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.052+0000 7f291e1f9700 1 --2- 172.21.15.162:0/1128743466 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f29180b10f0 0x7f2918151da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:57.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.052+0000 7f291e1f9700 1 -- 172.21.15.162:0/1128743466 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f29180b4b40 msgr2=0x7f2918152290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:57.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.052+0000 7f291e1f9700 1 --2- 172.21.15.162:0/1128743466 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f29180b4b40 0x7f2918152290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:57.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.052+0000 7f291e1f9700 1 -- 172.21.15.162:0/1128743466 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2918098ed0 con 0x7f29180611e0 2020-11-28T21:53:57.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.052+0000 7f291e1f9700 1 --2- 172.21.15.162:0/1128743466 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29180611e0 0x7f291813bc60 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 rx=0x7f290800a000 tx=0x7f2908008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:57.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.052+0000 7f29077fe700 1 -- 172.21.15.162:0/1128743466 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29080127e0 con 0x7f29180611e0 2020-11-28T21:53:57.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.056+0000 7f29077fe700 1 -- 172.21.15.162:0/1128743466 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2908012940 con 0x7f29180611e0 2020-11-28T21:53:57.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.056+0000 7f291f1fb700 1 -- 172.21.15.162:0/1128743466 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2918099aa0 con 0x7f29180611e0 2020-11-28T21:53:57.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.056+0000 7f29077fe700 1 -- 172.21.15.162:0/1128743466 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2908012ca0 con 0x7f29180611e0 2020-11-28T21:53:57.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.056+0000 7f291f1fb700 1 -- 172.21.15.162:0/1128743466 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2918142670 con 0x7f29180611e0 2020-11-28T21:53:57.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.056+0000 7f291f1fb700 1 -- 172.21.15.162:0/1128743466 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2918057050 con 0x7f29180611e0 2020-11-28T21:53:57.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.056+0000 7f29077fe700 1 -- 172.21.15.162:0/1128743466 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2908013480 con 0x7f29180611e0 2020-11-28T21:53:57.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.056+0000 7f29077fe700 1 --2- 172.21.15.162:0/1128743466 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f28f8055440 0x7f28f8057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:57.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.056+0000 7f29077fe700 1 -- 172.21.15.162:0/1128743466 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f290806e3f0 con 0x7f29180611e0 2020-11-28T21:53:57.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.056+0000 7f291d9f8700 1 --2- 172.21.15.162:0/1128743466 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f28f8055440 0x7f28f8057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:57.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.056+0000 7f291d9f8700 1 --2- 172.21.15.162:0/1128743466 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f28f8055440 0x7f28f8057900 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f291400d000 tx=0x7f2914000b80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:57.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.064+0000 7f29077fe700 1 -- 172.21.15.162:0/1128743466 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f29080448d0 con 0x7f29180611e0 2020-11-28T21:53:57.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:57.220+0000 7f291f1fb700 1 -- 172.21.15.162:0/1128743466 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f291805ce20 con 0x7f29180611e0 2020-11-28T21:53:58.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.048+0000 7f29077fe700 1 -- 172.21.15.162:0/1128743466 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v94) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2908047ef0 con 0x7f29180611e0 2020-11-28T21:53:58.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.052+0000 7f291f1fb700 1 -- 172.21.15.162:0/1128743466 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f28f8055440 msgr2=0x7f28f8057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:58.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.052+0000 7f291f1fb700 1 --2- 172.21.15.162:0/1128743466 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f28f8055440 0x7f28f8057900 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f291400d000 tx=0x7f2914000b80).stop 2020-11-28T21:53:58.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.052+0000 7f291f1fb700 1 -- 172.21.15.162:0/1128743466 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29180611e0 msgr2=0x7f291813bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:58.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.052+0000 7f291f1fb700 1 --2- 172.21.15.162:0/1128743466 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29180611e0 0x7f291813bc60 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 rx=0x7f290800a000 tx=0x7f2908008040).stop 2020-11-28T21:53:58.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.052+0000 7f291f1fb700 1 -- 172.21.15.162:0/1128743466 shutdown_connections 2020-11-28T21:53:58.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.052+0000 7f291f1fb700 1 --2- 172.21.15.162:0/1128743466 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f28f8055440 0x7f28f8057900 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:58.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.052+0000 7f291f1fb700 1 --2- 172.21.15.162:0/1128743466 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f29180b4b40 0x7f2918152290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:58.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.052+0000 7f291f1fb700 1 --2- 172.21.15.162:0/1128743466 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29180611e0 0x7f291813bc60 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:58.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.052+0000 7f291f1fb700 1 --2- 172.21.15.162:0/1128743466 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f29180b10f0 0x7f2918151da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:58.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.052+0000 7f291f1fb700 1 -- 172.21.15.162:0/1128743466 >> 172.21.15.162:0/1128743466 conn(0x7f29180b54f0 msgr2=0x7f29180b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:58.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.052+0000 7f291f1fb700 1 -- 172.21.15.162:0/1128743466 shutdown_connections 2020-11-28T21:53:58.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.052+0000 7f291f1fb700 1 -- 172.21.15.162:0/1128743466 wait complete. 2020-11-28T21:53:58.060 INFO:teuthology.orchestra.run.smithi162.stderr: removed all blacklist entries 2020-11-28T21:53:58.071 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2020-11-28T21:53:58.071 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth ls --format=json-pretty 2020-11-28T21:53:58.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.200+0000 7f1ef7daf700 1 Processor -- start 2020-11-28T21:53:58.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.200+0000 7f1ef7daf700 1 -- start start 2020-11-28T21:53:58.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.200+0000 7f1ef7daf700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ef00b1990 0x7f1ef0140890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:58.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.200+0000 7f1ef7daf700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1ef00b11b0 0x7f1ef0140d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:58.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.200+0000 7f1ef7daf700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1ef00b1d70 0x7f1ef014a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:58.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.200+0000 7f1ef7daf700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ef0146e20 con 0x7f1ef00b11b0 2020-11-28T21:53:58.224 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.200+0000 7f1ef7daf700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ef00b2870 con 0x7f1ef00b1d70 2020-11-28T21:53:58.224 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.200+0000 7f1ef7daf700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef00b22e0 con 0x7f1ef00b1990 2020-11-28T21:53:58.225 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.200+0000 7f1ef634c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1ef00b1d70 0x7f1ef014a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:58.225 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.200+0000 7f1ef634c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1ef00b1d70 0x7f1ef014a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42608/0 (socket says 172.21.15.162:42608) 2020-11-28T21:53:58.226 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.200+0000 7f1ef634c700 1 -- 172.21.15.162:0/1068800873 learned_addr learned my addr 172.21.15.162:0/1068800873 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:58.227 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.200+0000 7f1ef5b4b700 1 --2- 172.21.15.162:0/1068800873 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ef00b1990 0x7f1ef0140890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:58.227 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.200+0000 7f1ef534a700 1 --2- 172.21.15.162:0/1068800873 >> v2:172.21.15.118:3300/0 conn(0x7f1ef00b11b0 0x7f1ef0140d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:58.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.200+0000 7f1ef634c700 1 -- 172.21.15.162:0/1068800873 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ef00b1990 msgr2=0x7f1ef0140890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:58.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.200+0000 7f1ef634c700 1 --2- 172.21.15.162:0/1068800873 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ef00b1990 0x7f1ef0140890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:58.229 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.200+0000 7f1ef634c700 1 -- 172.21.15.162:0/1068800873 >> v2:172.21.15.118:3300/0 conn(0x7f1ef00b11b0 msgr2=0x7f1ef0140d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:58.230 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.200+0000 7f1ef634c700 1 --2- 172.21.15.162:0/1068800873 >> v2:172.21.15.118:3300/0 conn(0x7f1ef00b11b0 0x7f1ef0140d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:58.230 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.200+0000 7f1ef634c700 1 -- 172.21.15.162:0/1068800873 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef00b0740 con 0x7f1ef00b1d70 2020-11-28T21:53:58.231 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.200+0000 7f1ef5b4b700 1 --2- 172.21.15.162:0/1068800873 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ef00b1990 0x7f1ef0140890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:53:58.231 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.200+0000 7f1ef634c700 1 --2- 172.21.15.162:0/1068800873 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ef00b1d70 0x7f1ef014a770 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 rx=0x7f1ee0002f30 tx=0x7f1ee0006e80).ready entity=mon.1 client_cookie=a4f6f0a58c88f6be server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:58.231 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.204+0000 7f1ef4b49700 1 -- 172.21.15.162:0/1068800873 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ee0009640 con 0x7f1ef00b1d70 2020-11-28T21:53:58.232 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.204+0000 7f1ef4b49700 1 -- 172.21.15.162:0/1068800873 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee00097a0 con 0x7f1ef00b1d70 2020-11-28T21:53:58.232 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.204+0000 7f1ef4b49700 1 -- 172.21.15.162:0/1068800873 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ee0009b00 con 0x7f1ef00b1d70 2020-11-28T21:53:58.232 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.204+0000 7f1ef7daf700 1 -- 172.21.15.162:0/1068800873 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ef00b1d70 msgr2=0x7f1ef014a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:58.233 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.204+0000 7f1ef7daf700 1 --2- 172.21.15.162:0/1068800873 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ef00b1d70 0x7f1ef014a770 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 rx=0x7f1ee0002f30 tx=0x7f1ee0006e80).stop 2020-11-28T21:53:58.233 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.204+0000 7f1ef7daf700 1 -- 172.21.15.162:0/1068800873 shutdown_connections 2020-11-28T21:53:58.233 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.204+0000 7f1ef7daf700 1 --2- 172.21.15.162:0/1068800873 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ef00b1d70 0x7f1ef014a770 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:58.233 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.204+0000 7f1ef7daf700 1 --2- 172.21.15.162:0/1068800873 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ef00b1990 0x7f1ef0140890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:58.234 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.204+0000 7f1ef7daf700 1 --2- 172.21.15.162:0/1068800873 >> v2:172.21.15.118:3300/0 conn(0x7f1ef00b11b0 0x7f1ef0140d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:58.234 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.204+0000 7f1ef7daf700 1 -- 172.21.15.162:0/1068800873 >> 172.21.15.162:0/1068800873 conn(0x7f1ef005b970 msgr2=0x7f1ef005bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:58.235 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.204+0000 7f1ef7daf700 1 -- 172.21.15.162:0/1068800873 shutdown_connections 2020-11-28T21:53:58.235 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.204+0000 7f1ef7daf700 1 -- 172.21.15.162:0/1068800873 wait complete. 2020-11-28T21:53:58.235 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.204+0000 7f1ef7daf700 1 Processor -- start 2020-11-28T21:53:58.236 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.204+0000 7f1ef7daf700 1 -- start start 2020-11-28T21:53:58.236 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.204+0000 7f1ef7daf700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ef00b11b0 0x7f1ef01464a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:58.236 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.204+0000 7f1ef7daf700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1ef00b1990 0x7f1ef0063330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:58.237 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.204+0000 7f1ef7daf700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ef00b1d70 0x7f1ef0066b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:58.237 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.204+0000 7f1ef7daf700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef0142b10 con 0x7f1ef00b1990 2020-11-28T21:53:58.237 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.204+0000 7f1ef7daf700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef0066470 con 0x7f1ef00b1d70 2020-11-28T21:53:58.238 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.204+0000 7f1ef7daf700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef01446f0 con 0x7f1ef00b11b0 2020-11-28T21:53:58.238 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.204+0000 7f1ef634c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ef00b1d70 0x7f1ef0066b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:58.239 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.204+0000 7f1ef634c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ef00b1d70 0x7f1ef0066b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42614/0 (socket says 172.21.15.162:42614) 2020-11-28T21:53:58.239 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.204+0000 7f1ef634c700 1 -- 172.21.15.162:0/1586307150 learned_addr learned my addr 172.21.15.162:0/1586307150 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:58.239 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.204+0000 7f1ef5b4b700 1 --2- 172.21.15.162:0/1586307150 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ef00b11b0 0x7f1ef01464a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:58.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.204+0000 7f1ef534a700 1 --2- 172.21.15.162:0/1586307150 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1ef00b1990 0x7f1ef0063330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:58.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.204+0000 7f1ef634c700 1 -- 172.21.15.162:0/1586307150 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ef00b11b0 msgr2=0x7f1ef01464a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:58.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.204+0000 7f1ef634c700 1 --2- 172.21.15.162:0/1586307150 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ef00b11b0 0x7f1ef01464a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:58.241 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.204+0000 7f1ef634c700 1 -- 172.21.15.162:0/1586307150 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1ef00b1990 msgr2=0x7f1ef0063330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:58.241 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.204+0000 7f1ef634c700 1 --2- 172.21.15.162:0/1586307150 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1ef00b1990 0x7f1ef0063330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:58.241 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.204+0000 7f1ef634c700 1 -- 172.21.15.162:0/1586307150 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef0063820 con 0x7f1ef00b1d70 2020-11-28T21:53:58.242 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.204+0000 7f1ef5b4b700 1 --2- 172.21.15.162:0/1586307150 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ef00b11b0 0x7f1ef01464a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:53:58.242 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.204+0000 7f1ef634c700 1 --2- 172.21.15.162:0/1586307150 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ef00b1d70 0x7f1ef0066b70 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 rx=0x7f1ef0142c80 tx=0x7f1ee0015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:58.242 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.208+0000 7f1ee6ffd700 1 -- 172.21.15.162:0/1586307150 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ee0011c40 con 0x7f1ef00b1d70 2020-11-28T21:53:58.243 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.208+0000 7f1ee6ffd700 1 -- 172.21.15.162:0/1586307150 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee0011da0 con 0x7f1ef00b1d70 2020-11-28T21:53:58.243 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.208+0000 7f1ef7daf700 1 -- 172.21.15.162:0/1586307150 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ef0144450 con 0x7f1ef00b1d70 2020-11-28T21:53:58.243 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.208+0000 7f1ee6ffd700 1 -- 172.21.15.162:0/1586307150 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ee001d640 con 0x7f1ef00b1d70 2020-11-28T21:53:58.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.208+0000 7f1ef7daf700 1 -- 172.21.15.162:0/1586307150 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ef0066050 con 0x7f1ef00b1d70 2020-11-28T21:53:58.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.208+0000 7f1ef7daf700 1 -- 172.21.15.162:0/1586307150 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ef0143230 con 0x7f1ef00b1d70 2020-11-28T21:53:58.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.208+0000 7f1ee6ffd700 1 -- 172.21.15.162:0/1586307150 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1ee000f070 con 0x7f1ef00b1d70 2020-11-28T21:53:58.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.208+0000 7f1ee6ffd700 1 --2- 172.21.15.162:0/1586307150 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1ed80553b0 0x7f1ed8057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:58.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.208+0000 7f1ee6ffd700 1 -- 172.21.15.162:0/1586307150 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f1ee006f470 con 0x7f1ef00b1d70 2020-11-28T21:53:58.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.208+0000 7f1ef5b4b700 1 --2- 172.21.15.162:0/1586307150 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1ed80553b0 0x7f1ed8057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:58.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.208+0000 7f1ef5b4b700 1 --2- 172.21.15.162:0/1586307150 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1ed80553b0 0x7f1ed8057870 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f1ee800b000 tx=0x7f1ee8008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:58.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.212+0000 7f1ee6ffd700 1 -- 172.21.15.162:0/1586307150 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1ee00459d0 con 0x7f1ef00b1d70 2020-11-28T21:53:58.407 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.404+0000 7f1ef7daf700 1 -- 172.21.15.162:0/1586307150 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f1ef0057050 con 0x7f1ef00b1d70 2020-11-28T21:53:58.408 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.404+0000 7f1ee6ffd700 1 -- 172.21.15.162:0/1586307150 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) v1 ==== 80+0+4255 (secure 0 0 0) 0x7f1ee0048fb0 con 0x7f1ef00b1d70 2020-11-28T21:53:58.409 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:53:58.409 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2020-11-28T21:53:58.409 INFO:teuthology.orchestra.run.smithi162.stdout: "auth_dump": [ 2020-11-28T21:53:58.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:58.409 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mds.a", 2020-11-28T21:53:58.410 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfWxUpKxAAIUDwQRKwz8H4Qk9qE356Bg==", 2020-11-28T21:53:58.410 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:53:58.410 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:53:58.410 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:53:58.411 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:53:58.411 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:53:58.411 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:58.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:58.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:58.412 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mds.b", 2020-11-28T21:53:58.412 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfKY2zORAAaO65etMZiYcaTDNX0nsEPw==", 2020-11-28T21:53:58.412 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:53:58.412 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:53:58.412 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:53:58.413 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:53:58.413 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:53:58.413 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:58.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:58.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:58.414 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mds.c", 2020-11-28T21:53:58.414 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfdQbRMxAAWlcjesxTZzZnvKCQneSjjQ==", 2020-11-28T21:53:58.414 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:53:58.414 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:53:58.414 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:53:58.415 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:53:58.415 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:53:58.415 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:58.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:58.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:58.416 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.0", 2020-11-28T21:53:58.416 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQADxcJfF3WLLxAAxsB5YPjH1yjv/vhlRbijqw==", 2020-11-28T21:53:58.416 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:53:58.416 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:53:58.416 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:53:58.417 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:53:58.417 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:58.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:58.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:58.418 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.1", 2020-11-28T21:53:58.418 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAGxcJfKYbBFhAA1x4DzFtkQQv6uWLIdOi1qQ==", 2020-11-28T21:53:58.418 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:53:58.418 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:53:58.418 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:53:58.419 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:53:58.419 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:58.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:58.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:58.420 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.2", 2020-11-28T21:53:58.420 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAIxcJfvcYUOhAASENjM0HfAxxyFe13mmE36Q==", 2020-11-28T21:53:58.420 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:53:58.420 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:53:58.421 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:53:58.421 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:53:58.421 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:58.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:58.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:58.422 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.3", 2020-11-28T21:53:58.422 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQALxcJfc/OHIRAAFYx3s8Ssttt15hjoH04tow==", 2020-11-28T21:53:58.422 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:53:58.422 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:53:58.422 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:53:58.423 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:53:58.423 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:58.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:58.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:58.423 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.4", 2020-11-28T21:53:58.424 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAPxcJffGWqCRAAb8CCN5oUPkdDh6BJH5HEnw==", 2020-11-28T21:53:58.424 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:53:58.424 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:53:58.424 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:53:58.424 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:53:58.425 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:58.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:58.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:58.425 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.5", 2020-11-28T21:53:58.426 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQARxcJfo8GKKRAAfbsCDb4L2wULiKmHyzu7/A==", 2020-11-28T21:53:58.426 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:53:58.426 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:53:58.426 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:53:58.426 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:53:58.427 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:58.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:58.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:58.427 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.6", 2020-11-28T21:53:58.427 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAUxcJf8Y2MEBAAqQX6rX3LK0zodlI2wK4JrA==", 2020-11-28T21:53:58.428 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:53:58.428 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:53:58.428 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:53:58.428 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:53:58.429 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:58.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:58.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:58.429 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.7", 2020-11-28T21:53:58.429 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAWxcJfCLBdMhAAf8QixQMQDrfZOGWN8rHe5g==", 2020-11-28T21:53:58.430 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:53:58.430 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:53:58.430 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:53:58.430 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:53:58.430 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:58.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:58.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:58.431 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "client.0", 2020-11-28T21:53:58.431 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQABxcJfS9vDAxAAUk4FU8FVt4if/CRH5ZisOA==", 2020-11-28T21:53:58.431 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:53:58.432 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:53:58.432 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow r", 2020-11-28T21:53:58.432 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow rw pool=cephfs_data" 2020-11-28T21:53:58.432 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:58.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:58.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:58.433 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "client.1", 2020-11-28T21:53:58.433 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQABxcJfuCU8BhAAXUSoSK3D0XRo58uHNzcjXg==", 2020-11-28T21:53:58.433 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:53:58.433 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:53:58.434 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow r", 2020-11-28T21:53:58.434 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow rw pool=cephfs_data" 2020-11-28T21:53:58.434 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:58.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:58.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:58.435 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "client.admin", 2020-11-28T21:53:58.435 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfdlTWBxAApzAxlVjzdnETQF//Vnsg5w==", 2020-11-28T21:53:58.435 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:53:58.435 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow *", 2020-11-28T21:53:58.435 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:53:58.436 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:53:58.436 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:53:58.436 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:58.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:58.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:58.437 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mgr.x", 2020-11-28T21:53:58.437 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfZhdvKBAAT7zCCsD2OVCHjvddGLxQ/A==", 2020-11-28T21:53:58.437 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:53:58.437 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow *", 2020-11-28T21:53:58.437 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow profile mgr", 2020-11-28T21:53:58.438 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:53:58.438 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:58.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:58.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:58.438 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mgr.y", 2020-11-28T21:53:58.439 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfs8fvJRAAP0Atp074EeMHiTtpCCVHuA==", 2020-11-28T21:53:58.439 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:53:58.439 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow *", 2020-11-28T21:53:58.439 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow profile mgr", 2020-11-28T21:53:58.439 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:53:58.440 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:58.440 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:58.440 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:53:58.440 INFO:teuthology.orchestra.run.smithi162.stdout:} 2020-11-28T21:53:58.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.404+0000 7f1ef7daf700 1 -- 172.21.15.162:0/1586307150 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1ed80553b0 msgr2=0x7f1ed8057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:58.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.404+0000 7f1ef7daf700 1 --2- 172.21.15.162:0/1586307150 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1ed80553b0 0x7f1ed8057870 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f1ee800b000 tx=0x7f1ee8008040).stop 2020-11-28T21:53:58.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.404+0000 7f1ef7daf700 1 -- 172.21.15.162:0/1586307150 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ef00b1d70 msgr2=0x7f1ef0066b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:58.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.404+0000 7f1ef7daf700 1 --2- 172.21.15.162:0/1586307150 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ef00b1d70 0x7f1ef0066b70 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 rx=0x7f1ef0142c80 tx=0x7f1ee0015000).stop 2020-11-28T21:53:58.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.404+0000 7f1ef7daf700 1 -- 172.21.15.162:0/1586307150 shutdown_connections 2020-11-28T21:53:58.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.404+0000 7f1ef7daf700 1 --2- 172.21.15.162:0/1586307150 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1ed80553b0 0x7f1ed8057870 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:58.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.404+0000 7f1ef7daf700 1 --2- 172.21.15.162:0/1586307150 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ef00b11b0 0x7f1ef01464a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:58.444 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.404+0000 7f1ef7daf700 1 --2- 172.21.15.162:0/1586307150 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ef00b1d70 0x7f1ef0066b70 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:58.444 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.404+0000 7f1ef7daf700 1 --2- 172.21.15.162:0/1586307150 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1ef00b1990 0x7f1ef0063330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:58.444 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.404+0000 7f1ef7daf700 1 -- 172.21.15.162:0/1586307150 >> 172.21.15.162:0/1586307150 conn(0x7f1ef005b970 msgr2=0x7f1ef00ad240 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:58.444 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.404+0000 7f1ef7daf700 1 -- 172.21.15.162:0/1586307150 shutdown_connections 2020-11-28T21:53:58.445 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.404+0000 7f1ef7daf700 1 -- 172.21.15.162:0/1586307150 wait complete. 2020-11-28T21:53:58.445 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:53:58.568 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.564+0000 7ff4d20f3700 1 Processor -- start 2020-11-28T21:53:58.568 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.564+0000 7ff4d20f3700 1 -- start start 2020-11-28T21:53:58.570 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.564+0000 7ff4d20f3700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff4cc0994f0 0x7ff4cc0b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:58.570 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.564+0000 7ff4d20f3700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4cc098f10 con 0x7ff4cc098b30 2020-11-28T21:53:58.571 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.564+0000 7ff4d20f3700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4cc0b0680 con 0x7ff4cc060710 2020-11-28T21:53:58.571 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.564+0000 7ff4d20f3700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc0b1030 con 0x7ff4cc0994f0 2020-11-28T21:53:58.571 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.564+0000 7ff4d18f2700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7ff4cc060710 0x7ff4cc0b1580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:49468/0 (socket says 172.21.15.162:49468) 2020-11-28T21:53:58.571 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.564+0000 7ff4d18f2700 1 -- 172.21.15.162:0/3522367795 learned_addr learned my addr 172.21.15.162:0/3522367795 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:58.572 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.564+0000 7ff4d08f0700 1 --2- 172.21.15.162:0/3522367795 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff4cc0994f0 0x7ff4cc0b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:58.572 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.564+0000 7ff4c3fff700 1 -- 172.21.15.162:0/3522367795 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1160587391 0 0) 0x7ff4cc0b0680 con 0x7ff4cc060710 2020-11-28T21:53:58.572 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.564+0000 7ff4c3fff700 1 -- 172.21.15.162:0/3522367795 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4b0003230 con 0x7ff4cc060710 2020-11-28T21:53:58.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.564+0000 7ff4d08f0700 1 -- 172.21.15.162:0/3522367795 >> v1:172.21.15.162:6789/0 conn(0x7ff4cc060710 legacy=0x7ff4cc0b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:58.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.564+0000 7ff4d08f0700 1 -- 172.21.15.162:0/3522367795 >> v1:172.21.15.118:6789/0 conn(0x7ff4cc098b30 legacy=0x7ff4cc060000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:58.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.564+0000 7ff4d08f0700 1 -- 172.21.15.162:0/3522367795 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4cc0b0920 con 0x7ff4cc0994f0 2020-11-28T21:53:58.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.564+0000 7ff4d08f0700 1 --2- 172.21.15.162:0/3522367795 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff4cc0994f0 0x7ff4cc0b0000 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7ff4bc002f30 tx=0x7ff4bc006dc0).ready entity=mon.2 client_cookie=2c9edeca31ee7575 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:58.574 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.564+0000 7ff4c3fff700 1 -- 172.21.15.162:0/3522367795 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4bc0097b0 con 0x7ff4cc0994f0 2020-11-28T21:53:58.574 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.568+0000 7ff4c3fff700 1 -- 172.21.15.162:0/3522367795 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4bc009910 con 0x7ff4cc0994f0 2020-11-28T21:53:58.574 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.568+0000 7ff4c3fff700 1 -- 172.21.15.162:0/3522367795 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4bc009c70 con 0x7ff4cc0994f0 2020-11-28T21:53:58.574 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.568+0000 7ff4d20f3700 1 -- 172.21.15.162:0/3522367795 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff4cc0994f0 msgr2=0x7ff4cc0b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:58.575 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.568+0000 7ff4d20f3700 1 --2- 172.21.15.162:0/3522367795 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff4cc0994f0 0x7ff4cc0b0000 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7ff4bc002f30 tx=0x7ff4bc006dc0).stop 2020-11-28T21:53:58.577 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.568+0000 7ff4d20f3700 1 -- 172.21.15.162:0/3522367795 shutdown_connections 2020-11-28T21:53:58.578 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.568+0000 7ff4d20f3700 1 --2- 172.21.15.162:0/3522367795 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff4cc0994f0 0x7ff4cc0b0000 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:58.578 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.568+0000 7ff4d20f3700 1 -- 172.21.15.162:0/3522367795 >> 172.21.15.162:0/3522367795 conn(0x7ff4cc05b970 msgr2=0x7ff4cc05bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:58.578 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.568+0000 7ff4d20f3700 1 -- 172.21.15.162:0/3522367795 shutdown_connections 2020-11-28T21:53:58.578 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.568+0000 7ff4d20f3700 1 -- 172.21.15.162:0/3522367795 wait complete. 2020-11-28T21:53:58.578 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.568+0000 7ff4d20f3700 1 Processor -- start 2020-11-28T21:53:58.579 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.568+0000 7ff4d20f3700 1 -- start start 2020-11-28T21:53:58.579 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.568+0000 7ff4d20f3700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff4cc060710 0x7ff4cc0ab320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:58.579 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.568+0000 7ff4d20f3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4cc098b30 0x7ff4cc156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:58.579 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.568+0000 7ff4d20f3700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4cc0994f0 0x7ff4cc156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:58.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.568+0000 7ff4d20f3700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc155df0 con 0x7ff4cc0994f0 2020-11-28T21:53:58.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.568+0000 7ff4d20f3700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc14d190 con 0x7ff4cc098b30 2020-11-28T21:53:58.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.568+0000 7ff4d20f3700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc14d430 con 0x7ff4cc060710 2020-11-28T21:53:58.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.568+0000 7ff4d08f0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4cc098b30 0x7ff4cc156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:58.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.568+0000 7ff4d08f0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4cc098b30 0x7ff4cc156310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42630/0 (socket says 172.21.15.162:42630) 2020-11-28T21:53:58.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.568+0000 7ff4d08f0700 1 -- 172.21.15.162:0/2831940814 learned_addr learned my addr 172.21.15.162:0/2831940814 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:58.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.568+0000 7ff4d10f1700 1 --2- 172.21.15.162:0/2831940814 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff4cc060710 0x7ff4cc0ab320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:58.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.568+0000 7ff4d18f2700 1 --2- 172.21.15.162:0/2831940814 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4cc0994f0 0x7ff4cc156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:58.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.568+0000 7ff4d08f0700 1 -- 172.21.15.162:0/2831940814 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff4cc060710 msgr2=0x7ff4cc0ab320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:58.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.568+0000 7ff4d08f0700 1 --2- 172.21.15.162:0/2831940814 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff4cc060710 0x7ff4cc0ab320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:58.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.568+0000 7ff4d08f0700 1 -- 172.21.15.162:0/2831940814 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4cc0994f0 msgr2=0x7ff4cc156800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:58.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.568+0000 7ff4d08f0700 1 --2- 172.21.15.162:0/2831940814 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4cc0994f0 0x7ff4cc156800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:58.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.572+0000 7ff4d08f0700 1 -- 172.21.15.162:0/2831940814 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4cc14d6d0 con 0x7ff4cc098b30 2020-11-28T21:53:58.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.572+0000 7ff4d08f0700 1 --2- 172.21.15.162:0/2831940814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4cc098b30 0x7ff4cc156310 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 rx=0x7ff4bc002f30 tx=0x7ff4bc015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:58.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.572+0000 7ff4c27fc700 1 -- 172.21.15.162:0/2831940814 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4bc0119a0 con 0x7ff4cc098b30 2020-11-28T21:53:58.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.572+0000 7ff4d20f3700 1 -- 172.21.15.162:0/2831940814 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4cc14d970 con 0x7ff4cc098b30 2020-11-28T21:53:58.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.572+0000 7ff4c27fc700 1 -- 172.21.15.162:0/2831940814 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4bc011b00 con 0x7ff4cc098b30 2020-11-28T21:53:58.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.572+0000 7ff4c27fc700 1 -- 172.21.15.162:0/2831940814 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4bc011e60 con 0x7ff4cc098b30 2020-11-28T21:53:58.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.572+0000 7ff4d20f3700 1 -- 172.21.15.162:0/2831940814 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4cc156cf0 con 0x7ff4cc098b30 2020-11-28T21:53:58.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.572+0000 7ff4d20f3700 1 -- 172.21.15.162:0/2831940814 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4cc057050 con 0x7ff4cc098b30 2020-11-28T21:53:58.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.572+0000 7ff4c27fc700 1 -- 172.21.15.162:0/2831940814 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff4bc024030 con 0x7ff4cc098b30 2020-11-28T21:53:58.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.572+0000 7ff4c27fc700 1 --2- 172.21.15.162:0/2831940814 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff4b4055440 0x7ff4b4057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:58.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.572+0000 7ff4c27fc700 1 -- 172.21.15.162:0/2831940814 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5253+0+0 (secure 0 0 0) 0x7ff4bc06ea80 con 0x7ff4cc098b30 2020-11-28T21:53:58.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.572+0000 7ff4d10f1700 1 --2- 172.21.15.162:0/2831940814 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff4b4055440 0x7ff4b4057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:58.586 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.572+0000 7ff4d10f1700 1 --2- 172.21.15.162:0/2831940814 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff4b4055440 0x7ff4b4057900 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7ff4c800d000 tx=0x7ff4c8007b60).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:58.586 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.580+0000 7ff4c27fc700 1 -- 172.21.15.162:0/2831940814 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff4bc045010 con 0x7ff4cc098b30 2020-11-28T21:53:58.725 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.720+0000 7ff4d20f3700 1 -- 172.21.15.162:0/2831940814 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff4cc0b49b0 con 0x7ff4cc098b30 2020-11-28T21:53:58.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.724+0000 7ff4c27fc700 1 -- 172.21.15.162:0/2831940814 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+13278 (secure 0 0 0) 0x7ff4bc022020 con 0x7ff4cc098b30 2020-11-28T21:53:58.727 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:53:58.727 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":94,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:53:58.034554+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:53:58.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.724+0000 7ff4d20f3700 1 -- 172.21.15.162:0/2831940814 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff4b4055440 msgr2=0x7ff4b4057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:58.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.724+0000 7ff4d20f3700 1 --2- 172.21.15.162:0/2831940814 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff4b4055440 0x7ff4b4057900 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7ff4c800d000 tx=0x7ff4c8007b60).stop 2020-11-28T21:53:58.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.724+0000 7ff4d20f3700 1 -- 172.21.15.162:0/2831940814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4cc098b30 msgr2=0x7ff4cc156310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:58.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.724+0000 7ff4d20f3700 1 --2- 172.21.15.162:0/2831940814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4cc098b30 0x7ff4cc156310 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 rx=0x7ff4bc002f30 tx=0x7ff4bc015000).stop 2020-11-28T21:53:58.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.724+0000 7ff4d20f3700 1 -- 172.21.15.162:0/2831940814 shutdown_connections 2020-11-28T21:53:58.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.724+0000 7ff4d20f3700 1 --2- 172.21.15.162:0/2831940814 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff4b4055440 0x7ff4b4057900 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:58.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.724+0000 7ff4d20f3700 1 --2- 172.21.15.162:0/2831940814 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff4cc060710 0x7ff4cc0ab320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:58.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.724+0000 7ff4d20f3700 1 --2- 172.21.15.162:0/2831940814 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4cc0994f0 0x7ff4cc156800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:58.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.724+0000 7ff4d20f3700 1 --2- 172.21.15.162:0/2831940814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4cc098b30 0x7ff4cc156310 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:58.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.724+0000 7ff4d20f3700 1 -- 172.21.15.162:0/2831940814 >> 172.21.15.162:0/2831940814 conn(0x7ff4cc05b970 msgr2=0x7ff4cc148fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:58.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.724+0000 7ff4d20f3700 1 -- 172.21.15.162:0/2831940814 shutdown_connections 2020-11-28T21:53:58.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.724+0000 7ff4d20f3700 1 -- 172.21.15.162:0/2831940814 wait complete. 2020-11-28T21:53:58.741 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-28T21:53:58.742 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:53:58.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.852+0000 7fa024b8a700 1 Processor -- start 2020-11-28T21:53:58.858 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.852+0000 7fa024b8a700 1 -- start start 2020-11-28T21:53:58.859 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.856+0000 7fa024b8a700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa02005b970 0x7fa02005bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:58.859 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.856+0000 7fa024b8a700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa0200b2080 0x7fa020142b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:58.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.856+0000 7fa024b8a700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fa020142ff0 0x7fa0200b0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:58.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.856+0000 7fa024b8a700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0200ac300 con 0x7fa020142ff0 2020-11-28T21:53:58.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.856+0000 7fa024b8a700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0200ac5a0 con 0x7fa0200b2080 2020-11-28T21:53:58.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.856+0000 7fa024b8a700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0200ac840 con 0x7fa02005b970 2020-11-28T21:53:58.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.856+0000 7fa01effd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa0200b2080 0x7fa020142b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:58.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.856+0000 7fa01effd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa0200b2080 0x7fa020142b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42636/0 (socket says 172.21.15.162:42636) 2020-11-28T21:53:58.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.856+0000 7fa01effd700 1 -- 172.21.15.162:0/1155232102 learned_addr learned my addr 172.21.15.162:0/1155232102 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:58.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.856+0000 7fa01f7fe700 1 --2- 172.21.15.162:0/1155232102 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa02005b970 0x7fa02005bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:58.862 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.856+0000 7fa01ffff700 1 --2- 172.21.15.162:0/1155232102 >> v2:172.21.15.118:3300/0 conn(0x7fa020142ff0 0x7fa0200b0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:58.862 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.856+0000 7fa01effd700 1 -- 172.21.15.162:0/1155232102 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa02005b970 msgr2=0x7fa02005bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:58.862 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.856+0000 7fa01effd700 1 --2- 172.21.15.162:0/1155232102 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa02005b970 0x7fa02005bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:58.862 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.856+0000 7fa01effd700 1 -- 172.21.15.162:0/1155232102 >> v2:172.21.15.118:3300/0 conn(0x7fa020142ff0 msgr2=0x7fa0200b0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:58.862 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.856+0000 7fa01effd700 1 --2- 172.21.15.162:0/1155232102 >> v2:172.21.15.118:3300/0 conn(0x7fa020142ff0 0x7fa0200b0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:58.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.856+0000 7fa01effd700 1 -- 172.21.15.162:0/1155232102 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0200acae0 con 0x7fa0200b2080 2020-11-28T21:53:58.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.856+0000 7fa01ffff700 1 --2- 172.21.15.162:0/1155232102 >> v2:172.21.15.118:3300/0 conn(0x7fa020142ff0 0x7fa0200b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:53:58.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.856+0000 7fa01effd700 1 --2- 172.21.15.162:0/1155232102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0200b2080 0x7fa020142b00 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 rx=0x7fa008002f30 tx=0x7fa008006e80).ready entity=mon.1 client_cookie=ef19dc7b5950b598 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:58.864 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.856+0000 7fa01e7fc700 1 -- 172.21.15.162:0/1155232102 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0080097f0 con 0x7fa0200b2080 2020-11-28T21:53:58.864 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.856+0000 7fa01e7fc700 1 -- 172.21.15.162:0/1155232102 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa008009950 con 0x7fa0200b2080 2020-11-28T21:53:58.865 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.856+0000 7fa01e7fc700 1 -- 172.21.15.162:0/1155232102 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa008009cb0 con 0x7fa0200b2080 2020-11-28T21:53:58.865 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.856+0000 7fa024b8a700 1 -- 172.21.15.162:0/1155232102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0200b2080 msgr2=0x7fa020142b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:58.865 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.856+0000 7fa024b8a700 1 --2- 172.21.15.162:0/1155232102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0200b2080 0x7fa020142b00 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 rx=0x7fa008002f30 tx=0x7fa008006e80).stop 2020-11-28T21:53:58.865 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.856+0000 7fa024b8a700 1 -- 172.21.15.162:0/1155232102 shutdown_connections 2020-11-28T21:53:58.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.856+0000 7fa024b8a700 1 --2- 172.21.15.162:0/1155232102 >> v2:172.21.15.118:3300/0 conn(0x7fa020142ff0 0x7fa0200b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:58.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.856+0000 7fa024b8a700 1 --2- 172.21.15.162:0/1155232102 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa02005b970 0x7fa02005bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:58.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.856+0000 7fa024b8a700 1 --2- 172.21.15.162:0/1155232102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0200b2080 0x7fa020142b00 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:58.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.856+0000 7fa024b8a700 1 -- 172.21.15.162:0/1155232102 >> 172.21.15.162:0/1155232102 conn(0x7fa0200b2830 msgr2=0x7fa0200b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:58.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.856+0000 7fa024b8a700 1 -- 172.21.15.162:0/1155232102 shutdown_connections 2020-11-28T21:53:58.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.860+0000 7fa024b8a700 1 -- 172.21.15.162:0/1155232102 wait complete. 2020-11-28T21:53:58.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.860+0000 7fa024b8a700 1 Processor -- start 2020-11-28T21:53:58.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.860+0000 7fa024b8a700 1 -- start start 2020-11-28T21:53:58.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.860+0000 7fa024b8a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa02005b970 0x7fa020148730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:58.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.860+0000 7fa024b8a700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa0200b2080 0x7fa020138020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:58.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.860+0000 7fa024b8a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa020142ff0 0x7fa020138510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:58.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.860+0000 7fa024b8a700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02013bee0 con 0x7fa02005b970 2020-11-28T21:53:58.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.860+0000 7fa024b8a700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0200b49c0 con 0x7fa020142ff0 2020-11-28T21:53:58.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.860+0000 7fa024b8a700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa02013c110 con 0x7fa0200b2080 2020-11-28T21:53:58.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.860+0000 7fa01ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa020142ff0 0x7fa020138510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:58.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.860+0000 7fa01ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa020142ff0 0x7fa020138510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42644/0 (socket says 172.21.15.162:42644) 2020-11-28T21:53:58.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.860+0000 7fa01ffff700 1 -- 172.21.15.162:0/1681812251 learned_addr learned my addr 172.21.15.162:0/1681812251 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:58.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.860+0000 7fa01effd700 1 --2- 172.21.15.162:0/1681812251 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa0200b2080 0x7fa020138020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:58.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.860+0000 7fa01f7fe700 1 --2- 172.21.15.162:0/1681812251 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa02005b970 0x7fa020148730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:58.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.860+0000 7fa01ffff700 1 -- 172.21.15.162:0/1681812251 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa0200b2080 msgr2=0x7fa020138020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:58.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.860+0000 7fa01ffff700 1 --2- 172.21.15.162:0/1681812251 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa0200b2080 0x7fa020138020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:58.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.860+0000 7fa01ffff700 1 -- 172.21.15.162:0/1681812251 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa02005b970 msgr2=0x7fa020148730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:58.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.860+0000 7fa01ffff700 1 --2- 172.21.15.162:0/1681812251 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa02005b970 0x7fa020148730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:58.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.860+0000 7fa01ffff700 1 -- 172.21.15.162:0/1681812251 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0200b4bf0 con 0x7fa020142ff0 2020-11-28T21:53:58.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.860+0000 7fa01ffff700 1 --2- 172.21.15.162:0/1681812251 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa020142ff0 0x7fa020138510 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 rx=0x7fa018006dd0 tx=0x7fa018008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:58.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.860+0000 7fa01cff9700 1 -- 172.21.15.162:0/1681812251 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa01800b9a0 con 0x7fa020142ff0 2020-11-28T21:53:58.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.860+0000 7fa01cff9700 1 -- 172.21.15.162:0/1681812251 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa01800bb00 con 0x7fa020142ff0 2020-11-28T21:53:58.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.860+0000 7fa024b8a700 1 -- 172.21.15.162:0/1681812251 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0200b4e90 con 0x7fa020142ff0 2020-11-28T21:53:58.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.860+0000 7fa01cff9700 1 -- 172.21.15.162:0/1681812251 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa01800be60 con 0x7fa020142ff0 2020-11-28T21:53:58.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.864+0000 7fa024b8a700 1 -- 172.21.15.162:0/1681812251 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa020138a00 con 0x7fa020142ff0 2020-11-28T21:53:58.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.864+0000 7fa024b8a700 1 -- 172.21.15.162:0/1681812251 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0200b5020 con 0x7fa020142ff0 2020-11-28T21:53:58.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.864+0000 7fa01cff9700 1 -- 172.21.15.162:0/1681812251 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa01801c460 con 0x7fa020142ff0 2020-11-28T21:53:58.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.864+0000 7fa01cff9700 1 --2- 172.21.15.162:0/1681812251 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa0040554e0 0x7fa0040579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:58.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.864+0000 7fa01cff9700 1 -- 172.21.15.162:0/1681812251 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fa01806e350 con 0x7fa020142ff0 2020-11-28T21:53:58.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.864+0000 7fa01f7fe700 1 --2- 172.21.15.162:0/1681812251 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa0040554e0 0x7fa0040579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:58.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.864+0000 7fa01f7fe700 1 --2- 172.21.15.162:0/1681812251 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa0040554e0 0x7fa0040579a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7fa01000b000 tx=0x7fa010006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:58.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:58.872+0000 7fa01cff9700 1 -- 172.21.15.162:0/1681812251 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa018044920 con 0x7fa020142ff0 2020-11-28T21:53:59.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.008+0000 7fa024b8a700 1 -- 172.21.15.162:0/1681812251 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fa020057050 con 0x7fa020142ff0 2020-11-28T21:53:59.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.012+0000 7fa01cff9700 1 -- 172.21.15.162:0/1681812251 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v94) v1 ==== 110+0+10 (secure 0 0 0) 0x7fa018047f40 con 0x7fa020142ff0 2020-11-28T21:53:59.015 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:53:59.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.012+0000 7fa024b8a700 1 -- 172.21.15.162:0/1681812251 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa0040554e0 msgr2=0x7fa0040579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:59.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.012+0000 7fa024b8a700 1 --2- 172.21.15.162:0/1681812251 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa0040554e0 0x7fa0040579a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7fa01000b000 tx=0x7fa010006dc0).stop 2020-11-28T21:53:59.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.012+0000 7fa024b8a700 1 -- 172.21.15.162:0/1681812251 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa020142ff0 msgr2=0x7fa020138510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:59.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.012+0000 7fa024b8a700 1 --2- 172.21.15.162:0/1681812251 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa020142ff0 0x7fa020138510 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 rx=0x7fa018006dd0 tx=0x7fa018008040).stop 2020-11-28T21:53:59.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.012+0000 7fa024b8a700 1 -- 172.21.15.162:0/1681812251 shutdown_connections 2020-11-28T21:53:59.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.012+0000 7fa024b8a700 1 --2- 172.21.15.162:0/1681812251 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa0040554e0 0x7fa0040579a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:59.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.012+0000 7fa024b8a700 1 --2- 172.21.15.162:0/1681812251 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa020142ff0 0x7fa020138510 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:59.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.012+0000 7fa024b8a700 1 --2- 172.21.15.162:0/1681812251 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa02005b970 0x7fa020148730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:59.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.012+0000 7fa024b8a700 1 --2- 172.21.15.162:0/1681812251 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa0200b2080 0x7fa020138020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:59.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.012+0000 7fa024b8a700 1 -- 172.21.15.162:0/1681812251 >> 172.21.15.162:0/1681812251 conn(0x7fa0200b2830 msgr2=0x7fa0200608e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:59.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.012+0000 7fa024b8a700 1 -- 172.21.15.162:0/1681812251 shutdown_connections 2020-11-28T21:53:59.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.012+0000 7fa024b8a700 1 -- 172.21.15.162:0/1681812251 wait complete. 2020-11-28T21:53:59.028 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:53:59.139 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.136+0000 7f6302c18700 1 Processor -- start 2020-11-28T21:53:59.139 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.136+0000 7f6302c18700 1 -- start start 2020-11-28T21:53:59.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.136+0000 7f6302c18700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f62fc0b21f0 0x7f62fc13fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:59.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.136+0000 7f6302c18700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f62fc060b60 0x7f62fc14d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:59.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.136+0000 7f6302c18700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62fc0aff20 con 0x7f62fc1402a0 2020-11-28T21:53:59.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.136+0000 7f6302c18700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62fc0af240 con 0x7f62fc060b60 2020-11-28T21:53:59.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.136+0000 7f6302c18700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62fc140750 con 0x7f62fc0b21f0 2020-11-28T21:53:59.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.136+0000 7f6301c16700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f62fc0b21f0 0x7f62fc13fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:59.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.136+0000 7f6301c16700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f62fc0b21f0 0x7f62fc13fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42648/0 (socket says 172.21.15.162:42648) 2020-11-28T21:53:59.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.136+0000 7f6301c16700 1 -- 172.21.15.162:0/3835219569 learned_addr learned my addr 172.21.15.162:0/3835219569 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:59.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.136+0000 7f6302417700 1 --2- 172.21.15.162:0/3835219569 >> v2:172.21.15.118:3300/0 conn(0x7f62fc060b60 0x7f62fc14d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:59.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.136+0000 7f6301c16700 1 -- 172.21.15.162:0/3835219569 >> v2:172.21.15.118:3300/0 conn(0x7f62fc060b60 msgr2=0x7f62fc14d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:59.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.136+0000 7f6301c16700 1 --2- 172.21.15.162:0/3835219569 >> v2:172.21.15.118:3300/0 conn(0x7f62fc060b60 0x7f62fc14d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:59.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.136+0000 7f6301c16700 1 -- 172.21.15.162:0/3835219569 >> v1:172.21.15.118:6789/0 conn(0x7f62fc1402a0 legacy=0x7f62fc060450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:59.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.136+0000 7f6301c16700 1 -- 172.21.15.162:0/3835219569 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62fc0b25d0 con 0x7f62fc0b21f0 2020-11-28T21:53:59.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.136+0000 7f6301c16700 1 --2- 172.21.15.162:0/3835219569 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62fc0b21f0 0x7f62fc13fdb0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 rx=0x7f62fc142fa0 tx=0x7f62f8006e80).ready entity=mon.1 client_cookie=589200585062c488 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:59.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.136+0000 7f6300c14700 1 -- 172.21.15.162:0/3835219569 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62f80097b0 con 0x7f62fc0b21f0 2020-11-28T21:53:59.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.136+0000 7f6300c14700 1 -- 172.21.15.162:0/3835219569 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f8009910 con 0x7f62fc0b21f0 2020-11-28T21:53:59.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.136+0000 7f6300c14700 1 -- 172.21.15.162:0/3835219569 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62f8009a80 con 0x7f62fc0b21f0 2020-11-28T21:53:59.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.136+0000 7f6302c18700 1 -- 172.21.15.162:0/3835219569 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62fc0b21f0 msgr2=0x7f62fc13fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:59.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.136+0000 7f6302c18700 1 --2- 172.21.15.162:0/3835219569 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62fc0b21f0 0x7f62fc13fdb0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 rx=0x7f62fc142fa0 tx=0x7f62f8006e80).stop 2020-11-28T21:53:59.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.136+0000 7f6302c18700 1 -- 172.21.15.162:0/3835219569 shutdown_connections 2020-11-28T21:53:59.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.136+0000 7f6302c18700 1 --2- 172.21.15.162:0/3835219569 >> v2:172.21.15.118:3300/0 conn(0x7f62fc060b60 0x7f62fc14d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:59.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.136+0000 7f6302c18700 1 --2- 172.21.15.162:0/3835219569 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62fc0b21f0 0x7f62fc13fdb0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:59.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.136+0000 7f6302c18700 1 -- 172.21.15.162:0/3835219569 >> 172.21.15.162:0/3835219569 conn(0x7f62fc0b57d0 msgr2=0x7f62fc0b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:59.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.136+0000 7f6302c18700 1 -- 172.21.15.162:0/3835219569 shutdown_connections 2020-11-28T21:53:59.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.136+0000 7f6302c18700 1 -- 172.21.15.162:0/3835219569 wait complete. 2020-11-28T21:53:59.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.140+0000 7f6302c18700 1 Processor -- start 2020-11-28T21:53:59.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.140+0000 7f6302c18700 1 -- start start 2020-11-28T21:53:59.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.140+0000 7f6302c18700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f62fc060b60 0x7f62fc13e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:59.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.140+0000 7f6302c18700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62fc0b21f0 0x7f62fc13ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:59.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.140+0000 7f6302c18700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f62fc1402a0 0x7f62fc13b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:59.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.140+0000 7f6302c18700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62fc1444c0 con 0x7f62fc1402a0 2020-11-28T21:53:59.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.140+0000 7f6302c18700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62fc1469d0 con 0x7f62fc0b21f0 2020-11-28T21:53:59.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.140+0000 7f6302c18700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62fc151d30 con 0x7f62fc060b60 2020-11-28T21:53:59.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.140+0000 7f6301415700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62fc0b21f0 0x7f62fc13ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:59.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.140+0000 7f6301415700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62fc0b21f0 0x7f62fc13ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42656/0 (socket says 172.21.15.162:42656) 2020-11-28T21:53:59.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.140+0000 7f6301415700 1 -- 172.21.15.162:0/2218901046 learned_addr learned my addr 172.21.15.162:0/2218901046 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:59.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.140+0000 7f6301c16700 1 --2- 172.21.15.162:0/2218901046 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f62fc060b60 0x7f62fc13e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:59.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.140+0000 7f6302417700 1 --2- 172.21.15.162:0/2218901046 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f62fc1402a0 0x7f62fc13b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:59.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.140+0000 7f6301415700 1 -- 172.21.15.162:0/2218901046 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f62fc060b60 msgr2=0x7f62fc13e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:59.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.140+0000 7f6301415700 1 --2- 172.21.15.162:0/2218901046 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f62fc060b60 0x7f62fc13e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:59.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.140+0000 7f6301415700 1 -- 172.21.15.162:0/2218901046 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f62fc1402a0 msgr2=0x7f62fc13b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:59.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.140+0000 7f6301415700 1 --2- 172.21.15.162:0/2218901046 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f62fc1402a0 0x7f62fc13b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:59.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.140+0000 7f6301415700 1 -- 172.21.15.162:0/2218901046 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62fc1449d0 con 0x7f62fc0b21f0 2020-11-28T21:53:59.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.140+0000 7f6301c16700 1 --2- 172.21.15.162:0/2218901046 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f62fc060b60 0x7f62fc13e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:53:59.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.140+0000 7f6301415700 1 --2- 172.21.15.162:0/2218901046 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62fc0b21f0 0x7f62fc13ae90 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 rx=0x7f62ec002ef0 tx=0x7f62ec006d60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:59.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.140+0000 7f62eaffd700 1 -- 172.21.15.162:0/2218901046 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62ec00f510 con 0x7f62fc0b21f0 2020-11-28T21:53:59.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.140+0000 7f62eaffd700 1 -- 172.21.15.162:0/2218901046 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62ec00f670 con 0x7f62fc0b21f0 2020-11-28T21:53:59.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.140+0000 7f6302c18700 1 -- 172.21.15.162:0/2218901046 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62fc144c70 con 0x7f62fc0b21f0 2020-11-28T21:53:59.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.140+0000 7f62eaffd700 1 -- 172.21.15.162:0/2218901046 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62ec00f7e0 con 0x7f62fc0b21f0 2020-11-28T21:53:59.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.140+0000 7f6302c18700 1 -- 172.21.15.162:0/2218901046 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62fc13b870 con 0x7f62fc0b21f0 2020-11-28T21:53:59.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.144+0000 7f6302c18700 1 -- 172.21.15.162:0/2218901046 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62fc057050 con 0x7f62fc0b21f0 2020-11-28T21:53:59.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.144+0000 7f62eaffd700 1 -- 172.21.15.162:0/2218901046 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f62ec00f070 con 0x7f62fc0b21f0 2020-11-28T21:53:59.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.144+0000 7f62eaffd700 1 --2- 172.21.15.162:0/2218901046 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f62dc055330 0x7f62dc0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:59.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.144+0000 7f62eaffd700 1 -- 172.21.15.162:0/2218901046 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f62ec06d580 con 0x7f62fc0b21f0 2020-11-28T21:53:59.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.144+0000 7f6301c16700 1 --2- 172.21.15.162:0/2218901046 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f62dc055330 0x7f62dc0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:59.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.144+0000 7f6301c16700 1 --2- 172.21.15.162:0/2218901046 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f62dc055330 0x7f62dc0577f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f62f8019000 tx=0x7f62f8016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:59.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.152+0000 7f62eaffd700 1 -- 172.21.15.162:0/2218901046 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f62ec043bb0 con 0x7f62fc0b21f0 2020-11-28T21:53:59.296 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.292+0000 7f6302c18700 1 -- 172.21.15.162:0/2218901046 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f62fc144e00 con 0x7f62fc0b21f0 2020-11-28T21:53:59.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.292+0000 7f62eaffd700 1 -- 172.21.15.162:0/2218901046 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v94) v1 ==== 92+0+10 (secure 0 0 0) 0x7f62ec0471d0 con 0x7f62fc0b21f0 2020-11-28T21:53:59.297 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:53:59.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.296+0000 7f6302c18700 1 -- 172.21.15.162:0/2218901046 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f62dc055330 msgr2=0x7f62dc0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:59.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.296+0000 7f6302c18700 1 --2- 172.21.15.162:0/2218901046 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f62dc055330 0x7f62dc0577f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f62f8019000 tx=0x7f62f8016000).stop 2020-11-28T21:53:59.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.296+0000 7f6302c18700 1 -- 172.21.15.162:0/2218901046 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62fc0b21f0 msgr2=0x7f62fc13ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:59.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.296+0000 7f6302c18700 1 --2- 172.21.15.162:0/2218901046 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62fc0b21f0 0x7f62fc13ae90 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 rx=0x7f62ec002ef0 tx=0x7f62ec006d60).stop 2020-11-28T21:53:59.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.296+0000 7f6302c18700 1 -- 172.21.15.162:0/2218901046 shutdown_connections 2020-11-28T21:53:59.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.296+0000 7f6302c18700 1 --2- 172.21.15.162:0/2218901046 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f62dc055330 0x7f62dc0577f0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:59.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.296+0000 7f6302c18700 1 --2- 172.21.15.162:0/2218901046 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f62fc060b60 0x7f62fc13e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:59.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.296+0000 7f6302c18700 1 --2- 172.21.15.162:0/2218901046 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f62fc1402a0 0x7f62fc13b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:59.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.296+0000 7f6302c18700 1 --2- 172.21.15.162:0/2218901046 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62fc0b21f0 0x7f62fc13ae90 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:59.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.296+0000 7f6302c18700 1 -- 172.21.15.162:0/2218901046 >> 172.21.15.162:0/2218901046 conn(0x7f62fc0b57d0 msgr2=0x7f62fc0b42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:59.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.296+0000 7f6302c18700 1 -- 172.21.15.162:0/2218901046 shutdown_connections 2020-11-28T21:53:59.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.296+0000 7f6302c18700 1 -- 172.21.15.162:0/2218901046 wait complete. 2020-11-28T21:53:59.314 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2020-11-28T21:53:59.424 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.420+0000 7fde2ec76700 1 Processor -- start 2020-11-28T21:53:59.424 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.420+0000 7fde2ec76700 1 -- start start 2020-11-28T21:53:59.427 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.420+0000 7fde2ec76700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fde28098b30 0x7fde280994f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:59.427 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.420+0000 7fde2ec76700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fde280b0000 0x7fde28060000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:59.427 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.420+0000 7fde2ec76700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde28098f10 con 0x7fde280604f0 2020-11-28T21:53:59.427 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.420+0000 7fde2ec76700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde280b1030 con 0x7fde280b0000 2020-11-28T21:53:59.428 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.420+0000 7fde2ec76700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde280b0480 con 0x7fde28098b30 2020-11-28T21:53:59.428 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.420+0000 7fde2d473700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fde280b0000 0x7fde28060000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:59.428 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.420+0000 7fde2d473700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fde280b0000 0x7fde28060000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42662/0 (socket says 172.21.15.162:42662) 2020-11-28T21:53:59.429 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.420+0000 7fde2d473700 1 -- 172.21.15.162:0/3253556804 learned_addr learned my addr 172.21.15.162:0/3253556804 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:59.429 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.420+0000 7fde2e475700 1 --1- 172.21.15.162:0/3253556804 >> v1:172.21.15.162:6789/0 conn(0x7fde280604f0 0x7fde2814a550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:49510/0 (socket says 172.21.15.162:49510) 2020-11-28T21:53:59.429 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.420+0000 7fde2d473700 1 -- 172.21.15.162:0/3253556804 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fde28098b30 msgr2=0x7fde280994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:59.430 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.420+0000 7fde2dc74700 1 --2- 172.21.15.162:0/3253556804 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fde28098b30 0x7fde280994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:59.430 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.420+0000 7fde2d473700 1 --2- 172.21.15.162:0/3253556804 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fde28098b30 0x7fde280994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:59.430 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.420+0000 7fde2d473700 1 -- 172.21.15.162:0/3253556804 >> v1:172.21.15.162:6789/0 conn(0x7fde280604f0 legacy=0x7fde2814a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:59.430 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.420+0000 7fde2d473700 1 -- 172.21.15.162:0/3253556804 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde280b0680 con 0x7fde280b0000 2020-11-28T21:53:59.431 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.420+0000 7fde2dc74700 1 --2- 172.21.15.162:0/3253556804 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fde28098b30 0x7fde280994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:53:59.431 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.420+0000 7fde2d473700 1 --2- 172.21.15.162:0/3253556804 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde280b0000 0x7fde28060000 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 rx=0x7fde28099be0 tx=0x7fde20009270).ready entity=mon.1 client_cookie=2f007f9ad771e751 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:59.431 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.420+0000 7fde2cc72700 1 -- 172.21.15.162:0/3253556804 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde2000bbc0 con 0x7fde280b0000 2020-11-28T21:53:59.431 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.420+0000 7fde2cc72700 1 -- 172.21.15.162:0/3253556804 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde2000bd20 con 0x7fde280b0000 2020-11-28T21:53:59.432 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.420+0000 7fde2cc72700 1 -- 172.21.15.162:0/3253556804 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde2000be90 con 0x7fde280b0000 2020-11-28T21:53:59.432 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.420+0000 7fde2ec76700 1 -- 172.21.15.162:0/3253556804 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde280b0000 msgr2=0x7fde28060000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:59.432 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.420+0000 7fde2ec76700 1 --2- 172.21.15.162:0/3253556804 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde280b0000 0x7fde28060000 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 rx=0x7fde28099be0 tx=0x7fde20009270).stop 2020-11-28T21:53:59.433 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.424+0000 7fde2ec76700 1 -- 172.21.15.162:0/3253556804 shutdown_connections 2020-11-28T21:53:59.433 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.424+0000 7fde2ec76700 1 --2- 172.21.15.162:0/3253556804 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fde28098b30 0x7fde280994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:59.433 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.424+0000 7fde2ec76700 1 --2- 172.21.15.162:0/3253556804 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde280b0000 0x7fde28060000 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:59.433 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.424+0000 7fde2ec76700 1 -- 172.21.15.162:0/3253556804 >> 172.21.15.162:0/3253556804 conn(0x7fde2805b970 msgr2=0x7fde2805bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:59.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.424+0000 7fde2ec76700 1 -- 172.21.15.162:0/3253556804 shutdown_connections 2020-11-28T21:53:59.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.424+0000 7fde2ec76700 1 -- 172.21.15.162:0/3253556804 wait complete. 2020-11-28T21:53:59.435 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.424+0000 7fde2ec76700 1 Processor -- start 2020-11-28T21:53:59.435 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.424+0000 7fde2ec76700 1 -- start start 2020-11-28T21:53:59.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.424+0000 7fde2ec76700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fde280604f0 0x7fde280aaae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:59.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.424+0000 7fde2ec76700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde28098b30 0x7fde280ac010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:59.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.424+0000 7fde2ec76700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fde280b0000 0x7fde280aec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:59.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.424+0000 7fde2ec76700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde28156180 con 0x7fde280b0000 2020-11-28T21:53:59.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.428+0000 7fde2ec76700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde280ac500 con 0x7fde28098b30 2020-11-28T21:53:59.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.428+0000 7fde2ec76700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde280ae2a0 con 0x7fde280604f0 2020-11-28T21:53:59.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.428+0000 7fde2d473700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde28098b30 0x7fde280ac010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:59.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.428+0000 7fde2d473700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde28098b30 0x7fde280ac010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42670/0 (socket says 172.21.15.162:42670) 2020-11-28T21:53:59.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.428+0000 7fde2d473700 1 -- 172.21.15.162:0/308896803 learned_addr learned my addr 172.21.15.162:0/308896803 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:59.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.428+0000 7fde2dc74700 1 --2- 172.21.15.162:0/308896803 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fde280604f0 0x7fde280aaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:59.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.428+0000 7fde2e475700 1 --2- 172.21.15.162:0/308896803 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fde280b0000 0x7fde280aec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:59.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.428+0000 7fde2d473700 1 -- 172.21.15.162:0/308896803 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fde280604f0 msgr2=0x7fde280aaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:59.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.428+0000 7fde2d473700 1 --2- 172.21.15.162:0/308896803 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fde280604f0 0x7fde280aaae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:59.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.428+0000 7fde2d473700 1 -- 172.21.15.162:0/308896803 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fde280b0000 msgr2=0x7fde280aec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:59.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.428+0000 7fde2d473700 1 --2- 172.21.15.162:0/308896803 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fde280b0000 0x7fde280aec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:59.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.428+0000 7fde2d473700 1 -- 172.21.15.162:0/308896803 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde280ae540 con 0x7fde28098b30 2020-11-28T21:53:59.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.428+0000 7fde2d473700 1 --2- 172.21.15.162:0/308896803 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde28098b30 0x7fde280ac010 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 rx=0x7fde28099be0 tx=0x7fde20016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:59.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.428+0000 7fde1effd700 1 -- 172.21.15.162:0/308896803 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde2001f5e0 con 0x7fde28098b30 2020-11-28T21:53:59.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.428+0000 7fde1effd700 1 -- 172.21.15.162:0/308896803 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde2001f740 con 0x7fde28098b30 2020-11-28T21:53:59.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.428+0000 7fde2ec76700 1 -- 172.21.15.162:0/308896803 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde280ae7e0 con 0x7fde28098b30 2020-11-28T21:53:59.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.428+0000 7fde1effd700 1 -- 172.21.15.162:0/308896803 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde2001faa0 con 0x7fde28098b30 2020-11-28T21:53:59.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.428+0000 7fde2ec76700 1 -- 172.21.15.162:0/308896803 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde280af170 con 0x7fde28098b30 2020-11-28T21:53:59.444 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.428+0000 7fde2ec76700 1 -- 172.21.15.162:0/308896803 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde280634e0 con 0x7fde28098b30 2020-11-28T21:53:59.444 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.428+0000 7fde1effd700 1 -- 172.21.15.162:0/308896803 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fde2000f070 con 0x7fde28098b30 2020-11-28T21:53:59.444 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.428+0000 7fde1effd700 1 --2- 172.21.15.162:0/308896803 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fde08055330 0x7fde080577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:59.444 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.428+0000 7fde1effd700 1 -- 172.21.15.162:0/308896803 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fde2006ea10 con 0x7fde28098b30 2020-11-28T21:53:59.445 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.432+0000 7fde2dc74700 1 --2- 172.21.15.162:0/308896803 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fde08055330 0x7fde080577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:59.445 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.432+0000 7fde2dc74700 1 --2- 172.21.15.162:0/308896803 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fde08055330 0x7fde080577f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7fde1800b000 tx=0x7fde18008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:59.445 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.436+0000 7fde1effd700 1 -- 172.21.15.162:0/308896803 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fde20045040 con 0x7fde28098b30 2020-11-28T21:53:59.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.576+0000 7fde2ec76700 1 -- 172.21.15.162:0/308896803 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fde280ae970 con 0x7fde28098b30 2020-11-28T21:53:59.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.576+0000 7fde1effd700 1 -- 172.21.15.162:0/308896803 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v94) v1 ==== 84+0+143 (secure 0 0 0) 0x7fde20024090 con 0x7fde28098b30 2020-11-28T21:53:59.582 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:53:59.583 INFO:teuthology.orchestra.run.smithi162.stdout:[ 2020-11-28T21:53:59.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:59.583 INFO:teuthology.orchestra.run.smithi162.stdout: "poolnum": 1, 2020-11-28T21:53:59.583 INFO:teuthology.orchestra.run.smithi162.stdout: "poolname": "device_health_metrics" 2020-11-28T21:53:59.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:53:59.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:53:59.584 INFO:teuthology.orchestra.run.smithi162.stdout: "poolnum": 2, 2020-11-28T21:53:59.584 INFO:teuthology.orchestra.run.smithi162.stdout: "poolname": "rbd" 2020-11-28T21:53:59.584 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:53:59.585 INFO:teuthology.orchestra.run.smithi162.stdout:] 2020-11-28T21:53:59.586 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.580+0000 7fde2ec76700 1 -- 172.21.15.162:0/308896803 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fde08055330 msgr2=0x7fde080577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:59.586 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.580+0000 7fde2ec76700 1 --2- 172.21.15.162:0/308896803 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fde08055330 0x7fde080577f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7fde1800b000 tx=0x7fde18008040).stop 2020-11-28T21:53:59.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.580+0000 7fde2ec76700 1 -- 172.21.15.162:0/308896803 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde28098b30 msgr2=0x7fde280ac010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:59.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.580+0000 7fde2ec76700 1 --2- 172.21.15.162:0/308896803 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde28098b30 0x7fde280ac010 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 rx=0x7fde28099be0 tx=0x7fde20016000).stop 2020-11-28T21:53:59.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.580+0000 7fde2ec76700 1 -- 172.21.15.162:0/308896803 shutdown_connections 2020-11-28T21:53:59.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.580+0000 7fde2ec76700 1 --2- 172.21.15.162:0/308896803 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fde08055330 0x7fde080577f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:59.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.580+0000 7fde2ec76700 1 --2- 172.21.15.162:0/308896803 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fde280604f0 0x7fde280aaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:59.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.580+0000 7fde2ec76700 1 --2- 172.21.15.162:0/308896803 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fde280b0000 0x7fde280aec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:59.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.580+0000 7fde2ec76700 1 --2- 172.21.15.162:0/308896803 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde28098b30 0x7fde280ac010 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:59.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.580+0000 7fde2ec76700 1 -- 172.21.15.162:0/308896803 >> 172.21.15.162:0/308896803 conn(0x7fde2805b970 msgr2=0x7fde280b1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:59.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.580+0000 7fde2ec76700 1 -- 172.21.15.162:0/308896803 shutdown_connections 2020-11-28T21:53:59.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.580+0000 7fde2ec76700 1 -- 172.21.15.162:0/308896803 wait complete. 2020-11-28T21:53:59.595 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-11-28T21:53:59.596 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2020-11-28T21:53:59.710 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.704+0000 7fb8b7fff700 1 Processor -- start 2020-11-28T21:53:59.710 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.708+0000 7fb8b7fff700 1 -- start start 2020-11-28T21:53:59.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.708+0000 7fb8b7fff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fb8b813cb50 0x7fb8b8140180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:59.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.708+0000 7fb8b7fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb8b805cca0 0x7fb8b80b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:59.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.708+0000 7fb8b7fff700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8b80b1360 con 0x7fb8b80b23a0 2020-11-28T21:53:59.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.708+0000 7fb8b7fff700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8b80add60 con 0x7fb8b813cb50 2020-11-28T21:53:59.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.708+0000 7fb8b7fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b80ae8c0 con 0x7fb8b805cca0 2020-11-28T21:53:59.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.708+0000 7fb8b77fe700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fb8b80b23a0 0x7fb8b80ac350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:49524/0 (socket says 172.21.15.162:49524) 2020-11-28T21:53:59.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.708+0000 7fb8b77fe700 1 -- 172.21.15.162:0/3350564734 learned_addr learned my addr 172.21.15.162:0/3350564734 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:53:59.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.708+0000 7fb8b6ffd700 1 --2- 172.21.15.162:0/3350564734 >> v2:172.21.15.118:3300/0 conn(0x7fb8b813cb50 0x7fb8b8140180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:59.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.708+0000 7fb8b5ffb700 1 -- 172.21.15.162:0/3350564734 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2375602872 0 0) 0x7fb8b80b1360 con 0x7fb8b80b23a0 2020-11-28T21:53:59.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.708+0000 7fb8b67fc700 1 --2- 172.21.15.162:0/3350564734 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb8b805cca0 0x7fb8b80b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:59.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.708+0000 7fb8b5ffb700 1 -- 172.21.15.162:0/3350564734 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb89c003230 con 0x7fb8b80b23a0 2020-11-28T21:53:59.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.708+0000 7fb8b5ffb700 1 -- 172.21.15.162:0/3350564734 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1866594057 0 0) 0x7fb89c003230 con 0x7fb8b80b23a0 2020-11-28T21:53:59.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.708+0000 7fb8b5ffb700 1 -- 172.21.15.162:0/3350564734 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb8b805cca0 msgr2=0x7fb8b80b1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:59.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.708+0000 7fb8b5ffb700 1 --2- 172.21.15.162:0/3350564734 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb8b805cca0 0x7fb8b80b1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:59.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.708+0000 7fb8b5ffb700 1 -- 172.21.15.162:0/3350564734 >> v2:172.21.15.118:3300/0 conn(0x7fb8b813cb50 msgr2=0x7fb8b8140180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:59.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.708+0000 7fb8b5ffb700 1 --2- 172.21.15.162:0/3350564734 >> v2:172.21.15.118:3300/0 conn(0x7fb8b813cb50 0x7fb8b8140180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:59.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.708+0000 7fb8b5ffb700 1 -- 172.21.15.162:0/3350564734 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b80aeb60 con 0x7fb8b80b23a0 2020-11-28T21:53:59.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.708+0000 7fb8b5ffb700 1 -- 172.21.15.162:0/3350564734 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7fb8b0001a50 con 0x7fb8b80b23a0 2020-11-28T21:53:59.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.708+0000 7fb8b5ffb700 1 -- 172.21.15.162:0/3350564734 >> v1:172.21.15.162:6789/0 conn(0x7fb8b80b23a0 legacy=0x7fb8b80ac350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:59.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.708+0000 7fb8b5ffb700 1 --2- 172.21.15.162:0/3350564734 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb89c003720 0x7fb89c003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:59.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.708+0000 7fb8b5ffb700 1 --2- 172.21.15.162:0/3350564734 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb89c0044d0 0x7fb89c004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:59.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.708+0000 7fb8b5ffb700 1 --2- 172.21.15.162:0/3350564734 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb8b805cca0 0x7fb89c004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:59.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.708+0000 7fb8b5ffb700 1 -- 172.21.15.162:0/3350564734 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b80ae8c0 con 0x7fb89c0044d0 2020-11-28T21:53:59.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.708+0000 7fb8b5ffb700 1 -- 172.21.15.162:0/3350564734 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89c005380 con 0x7fb89c003720 2020-11-28T21:53:59.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.708+0000 7fb8b5ffb700 1 -- 172.21.15.162:0/3350564734 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb89c0042e0 con 0x7fb8b805cca0 2020-11-28T21:53:59.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.708+0000 7fb8b6ffd700 1 --2- 172.21.15.162:0/3350564734 >> v2:172.21.15.118:3300/0 conn(0x7fb8b813cb50 0x7fb8b8140180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:53:59.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.708+0000 7fb8b6ffd700 1 --2- 172.21.15.162:0/3350564734 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb89c003720 0x7fb89c003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:59.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.708+0000 7fb8b67fc700 1 --2- 172.21.15.162:0/3350564734 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb89c0044d0 0x7fb89c004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:59.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.708+0000 7fb8b77fe700 1 --2- 172.21.15.162:0/3350564734 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb8b805cca0 0x7fb89c004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:59.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.712+0000 7fb8b6ffd700 1 -- 172.21.15.162:0/3350564734 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb8b805cca0 msgr2=0x7fb89c004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:59.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.712+0000 7fb8b6ffd700 1 --2- 172.21.15.162:0/3350564734 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb8b805cca0 0x7fb89c004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:59.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.712+0000 7fb8b6ffd700 1 -- 172.21.15.162:0/3350564734 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb89c0044d0 msgr2=0x7fb89c004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:53:59.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.712+0000 7fb8b6ffd700 1 --2- 172.21.15.162:0/3350564734 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb89c0044d0 0x7fb89c004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:59.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.712+0000 7fb8b6ffd700 1 -- 172.21.15.162:0/3350564734 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb89c003230 con 0x7fb89c003720 2020-11-28T21:53:59.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.712+0000 7fb8b6ffd700 1 --2- 172.21.15.162:0/3350564734 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb89c003720 0x7fb89c003b40 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 rx=0x7fb8a800c000 tx=0x7fb8a8008040).ready entity=mon.1 client_cookie=d87be07df7b55d8 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:59.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.712+0000 7fb8b5ffb700 1 -- 172.21.15.162:0/3350564734 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8a800b4d0 con 0x7fb89c003720 2020-11-28T21:53:59.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.712+0000 7fb8b5ffb700 1 -- 172.21.15.162:0/3350564734 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a800ebd0 con 0x7fb89c003720 2020-11-28T21:53:59.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.712+0000 7fb8b5ffb700 1 -- 172.21.15.162:0/3350564734 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8a80123f0 con 0x7fb89c003720 2020-11-28T21:53:59.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.712+0000 7fb8b7fff700 1 -- 172.21.15.162:0/3350564734 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb89c003720 msgr2=0x7fb89c003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:59.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.712+0000 7fb8b7fff700 1 --2- 172.21.15.162:0/3350564734 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb89c003720 0x7fb89c003b40 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 rx=0x7fb8a800c000 tx=0x7fb8a8008040).stop 2020-11-28T21:53:59.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.712+0000 7fb8b77fe700 1 -- 172.21.15.162:0/3350564734 reap_dead start 2020-11-28T21:53:59.725 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.712+0000 7fb8b7fff700 1 -- 172.21.15.162:0/3350564734 shutdown_connections 2020-11-28T21:53:59.725 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.712+0000 7fb8b7fff700 1 -- 172.21.15.162:0/3350564734 >> 172.21.15.162:0/3350564734 conn(0x7fb8b80b57d0 msgr2=0x7fb8b80b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:53:59.725 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.712+0000 7fb8b7fff700 1 -- 172.21.15.162:0/3350564734 shutdown_connections 2020-11-28T21:53:59.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.712+0000 7fb8b7fff700 1 -- 172.21.15.162:0/3350564734 wait complete. 2020-11-28T21:53:59.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.716+0000 7fb8b7fff700 1 Processor -- start 2020-11-28T21:53:59.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.716+0000 7fb8b7fff700 1 -- start start 2020-11-28T21:53:59.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.716+0000 7fb8b7fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb8b80b32d0 0x7fb8b80ab1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:59.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.716+0000 7fb8b7fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb8b80aab70 0x7fb8b814d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:59.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.716+0000 7fb8b7fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8b814da40 0x7fb8b80a8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:59.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.716+0000 7fb8b7fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b80b3c10 con 0x7fb8b80b32d0 2020-11-28T21:53:59.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.716+0000 7fb8b7fff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b80b3e40 con 0x7fb8b814da40 2020-11-28T21:53:59.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.716+0000 7fb8b7fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b80b40e0 con 0x7fb8b80aab70 2020-11-28T21:53:59.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.716+0000 7fb8b77fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8b814da40 0x7fb8b80a8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:59.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.716+0000 7fb8b77fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8b814da40 0x7fb8b80a8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42692/0 (socket says 172.21.15.162:42692) 2020-11-28T21:53:59.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.716+0000 7fb8b77fe700 1 -- 172.21.15.162:0/696951279 learned_addr learned my addr 172.21.15.162:0/696951279 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:53:59.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.716+0000 7fb8b6ffd700 1 --2- 172.21.15.162:0/696951279 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb8b80b32d0 0x7fb8b80ab1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:59.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.716+0000 7fb8b67fc700 1 --2- 172.21.15.162:0/696951279 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb8b80aab70 0x7fb8b814d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:59.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.716+0000 7fb8b77fe700 1 -- 172.21.15.162:0/696951279 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb8b80aab70 msgr2=0x7fb8b814d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:59.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.716+0000 7fb8b77fe700 1 --2- 172.21.15.162:0/696951279 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb8b80aab70 0x7fb8b814d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:59.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.716+0000 7fb8b77fe700 1 -- 172.21.15.162:0/696951279 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb8b80b32d0 msgr2=0x7fb8b80ab1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:53:59.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.716+0000 7fb8b77fe700 1 --2- 172.21.15.162:0/696951279 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb8b80b32d0 0x7fb8b80ab1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:53:59.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.716+0000 7fb8b77fe700 1 -- 172.21.15.162:0/696951279 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b80b4380 con 0x7fb8b814da40 2020-11-28T21:53:59.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.716+0000 7fb8b77fe700 1 --2- 172.21.15.162:0/696951279 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8b814da40 0x7fb8b80a8cc0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 rx=0x7fb8b0000b20 tx=0x7fb8b813f7b0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:59.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.716+0000 7fb8a7fff700 1 -- 172.21.15.162:0/696951279 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8b000fc30 con 0x7fb8b814da40 2020-11-28T21:53:59.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.716+0000 7fb8a7fff700 1 -- 172.21.15.162:0/696951279 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8b000fd90 con 0x7fb8b814da40 2020-11-28T21:53:59.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.716+0000 7fb8b7fff700 1 -- 172.21.15.162:0/696951279 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8b814dec0 con 0x7fb8b814da40 2020-11-28T21:53:59.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.716+0000 7fb8a7fff700 1 -- 172.21.15.162:0/696951279 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8b0012540 con 0x7fb8b814da40 2020-11-28T21:53:59.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.716+0000 7fb8b7fff700 1 -- 172.21.15.162:0/696951279 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8b80b36b0 con 0x7fb8b814da40 2020-11-28T21:53:59.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.720+0000 7fb8b7fff700 1 -- 172.21.15.162:0/696951279 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8b80b4510 con 0x7fb8b814da40 2020-11-28T21:53:59.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.720+0000 7fb8a7fff700 1 -- 172.21.15.162:0/696951279 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb8b00126a0 con 0x7fb8b814da40 2020-11-28T21:53:59.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.720+0000 7fb8a7fff700 1 --2- 172.21.15.162:0/696951279 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb8940554e0 0x7fb8940579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:53:59.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.720+0000 7fb8a7fff700 1 -- 172.21.15.162:0/696951279 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fb8b006d4b0 con 0x7fb8b814da40 2020-11-28T21:53:59.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.720+0000 7fb8b6ffd700 1 --2- 172.21.15.162:0/696951279 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb8940554e0 0x7fb8940579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:53:59.733 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.720+0000 7fb8b6ffd700 1 --2- 172.21.15.162:0/696951279 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb8940554e0 0x7fb8940579a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7fb8a801c000 tx=0x7fb8a801a000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:53:59.733 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.728+0000 7fb8a7fff700 1 -- 172.21.15.162:0/696951279 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb8b0043970 con 0x7fb8b814da40 2020-11-28T21:53:59.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:53:59.868+0000 7fb8b7fff700 1 -- 172.21.15.162:0/696951279 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fb8b8057050 con 0x7fb8b814da40 2020-11-28T21:54:00.886 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:00.884+0000 7fb8a7fff700 1 -- 172.21.15.162:0/696951279 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v95) v1 ==== 133+0+0 (secure 0 0 0) 0x7fb8b0019030 con 0x7fb8b814da40 2020-11-28T21:54:00.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:00.888+0000 7fb8b7fff700 1 -- 172.21.15.162:0/696951279 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb8940554e0 msgr2=0x7fb8940579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:00.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:00.888+0000 7fb8b7fff700 1 --2- 172.21.15.162:0/696951279 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb8940554e0 0x7fb8940579a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7fb8a801c000 tx=0x7fb8a801a000).stop 2020-11-28T21:54:00.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:00.888+0000 7fb8b7fff700 1 -- 172.21.15.162:0/696951279 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8b814da40 msgr2=0x7fb8b80a8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:00.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:00.888+0000 7fb8b7fff700 1 --2- 172.21.15.162:0/696951279 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8b814da40 0x7fb8b80a8cc0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 rx=0x7fb8b0000b20 tx=0x7fb8b813f7b0).stop 2020-11-28T21:54:00.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:00.888+0000 7fb8b7fff700 1 -- 172.21.15.162:0/696951279 shutdown_connections 2020-11-28T21:54:00.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:00.888+0000 7fb8b7fff700 1 --2- 172.21.15.162:0/696951279 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb8940554e0 0x7fb8940579a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:00.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:00.888+0000 7fb8b7fff700 1 --2- 172.21.15.162:0/696951279 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8b814da40 0x7fb8b80a8cc0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:00.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:00.888+0000 7fb8b7fff700 1 --2- 172.21.15.162:0/696951279 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb8b80b32d0 0x7fb8b80ab1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:00.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:00.888+0000 7fb8b7fff700 1 --2- 172.21.15.162:0/696951279 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb8b80aab70 0x7fb8b814d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:00.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:00.888+0000 7fb8b7fff700 1 -- 172.21.15.162:0/696951279 >> 172.21.15.162:0/696951279 conn(0x7fb8b80b57d0 msgr2=0x7fb8b813dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:00.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:00.888+0000 7fb8b7fff700 1 -- 172.21.15.162:0/696951279 shutdown_connections 2020-11-28T21:54:00.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:00.888+0000 7fb8b7fff700 1 -- 172.21.15.162:0/696951279 wait complete. 2020-11-28T21:54:00.894 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_metadata' created 2020-11-28T21:54:00.906 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2020-11-28T21:54:01.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.020+0000 7f2e5cd1e700 1 Processor -- start 2020-11-28T21:54:01.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.020+0000 7f2e5cd1e700 1 -- start start 2020-11-28T21:54:01.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.024+0000 7f2e5cd1e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2e58141e80 0x7f2e580606f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:01.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.024+0000 7f2e5cd1e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2e580ae3d0 0x7f2e5814b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:01.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.024+0000 7f2e5cd1e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e580af6a0 con 0x7f2e580b21f0 2020-11-28T21:54:01.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.024+0000 7f2e5cd1e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e580b25d0 con 0x7f2e58141e80 2020-11-28T21:54:01.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.024+0000 7f2e5cd1e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5805fff0 con 0x7f2e580ae3d0 2020-11-28T21:54:01.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.024+0000 7f2e56ffd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2e58141e80 0x7f2e580606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:01.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.024+0000 7f2e56ffd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2e58141e80 0x7f2e580606f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42698/0 (socket says 172.21.15.162:42698) 2020-11-28T21:54:01.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.024+0000 7f2e56ffd700 1 -- 172.21.15.162:0/308027574 learned_addr learned my addr 172.21.15.162:0/308027574 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:01.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.024+0000 7f2e577fe700 1 --1- 172.21.15.162:0/308027574 >> v1:172.21.15.162:6789/0 conn(0x7f2e580b21f0 0x7f2e581434f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:49540/0 (socket says 172.21.15.162:49540) 2020-11-28T21:54:01.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.024+0000 7f2e57fff700 1 --2- 172.21.15.162:0/308027574 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2e580ae3d0 0x7f2e5814b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:01.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.024+0000 7f2e56ffd700 1 -- 172.21.15.162:0/308027574 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2e580ae3d0 msgr2=0x7f2e5814b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:01.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.024+0000 7f2e56ffd700 1 --2- 172.21.15.162:0/308027574 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2e580ae3d0 0x7f2e5814b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:01.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.024+0000 7f2e56ffd700 1 -- 172.21.15.162:0/308027574 >> v1:172.21.15.162:6789/0 conn(0x7f2e580b21f0 legacy=0x7f2e581434f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:01.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.024+0000 7f2e56ffd700 1 -- 172.21.15.162:0/308027574 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e580af8d0 con 0x7f2e58141e80 2020-11-28T21:54:01.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.024+0000 7f2e56ffd700 1 --2- 172.21.15.162:0/308027574 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e58141e80 0x7f2e580606f0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 rx=0x7f2e480032b0 tx=0x7f2e48009270).ready entity=mon.1 client_cookie=49b3d9add66206d9 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:01.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.024+0000 7f2e567fc700 1 -- 172.21.15.162:0/308027574 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e48009e60 con 0x7f2e58141e80 2020-11-28T21:54:01.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.024+0000 7f2e567fc700 1 -- 172.21.15.162:0/308027574 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e4800bc00 con 0x7f2e58141e80 2020-11-28T21:54:01.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.024+0000 7f2e567fc700 1 -- 172.21.15.162:0/308027574 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e4800bd70 con 0x7f2e58141e80 2020-11-28T21:54:01.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.024+0000 7f2e5cd1e700 1 -- 172.21.15.162:0/308027574 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e58141e80 msgr2=0x7f2e580606f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:01.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.024+0000 7f2e5cd1e700 1 --2- 172.21.15.162:0/308027574 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e58141e80 0x7f2e580606f0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 rx=0x7f2e480032b0 tx=0x7f2e48009270).stop 2020-11-28T21:54:01.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.024+0000 7f2e5cd1e700 1 -- 172.21.15.162:0/308027574 shutdown_connections 2020-11-28T21:54:01.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.024+0000 7f2e5cd1e700 1 --2- 172.21.15.162:0/308027574 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2e580ae3d0 0x7f2e5814b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:01.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.024+0000 7f2e5cd1e700 1 --2- 172.21.15.162:0/308027574 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e58141e80 0x7f2e580606f0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:01.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.024+0000 7f2e5cd1e700 1 -- 172.21.15.162:0/308027574 >> 172.21.15.162:0/308027574 conn(0x7f2e580b57d0 msgr2=0x7f2e580b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:01.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.024+0000 7f2e5cd1e700 1 -- 172.21.15.162:0/308027574 shutdown_connections 2020-11-28T21:54:01.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.024+0000 7f2e5cd1e700 1 -- 172.21.15.162:0/308027574 wait complete. 2020-11-28T21:54:01.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.028+0000 7f2e5cd1e700 1 Processor -- start 2020-11-28T21:54:01.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.028+0000 7f2e5cd1e700 1 -- start start 2020-11-28T21:54:01.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.028+0000 7f2e5cd1e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e580ae3d0 0x7f2e581533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:01.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.028+0000 7f2e5cd1e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2e580b21f0 0x7f2e581538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:01.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.028+0000 7f2e5cd1e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2e58141e80 0x7f2e5813dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:01.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.028+0000 7f2e5cd1e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5809e420 con 0x7f2e580b21f0 2020-11-28T21:54:01.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.028+0000 7f2e5cd1e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e580b4850 con 0x7f2e580ae3d0 2020-11-28T21:54:01.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.028+0000 7f2e5cd1e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e58066e50 con 0x7f2e58141e80 2020-11-28T21:54:01.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.028+0000 7f2e577fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e580ae3d0 0x7f2e581533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:01.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.028+0000 7f2e577fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e580ae3d0 0x7f2e581533f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42702/0 (socket says 172.21.15.162:42702) 2020-11-28T21:54:01.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.028+0000 7f2e577fe700 1 -- 172.21.15.162:0/3310601852 learned_addr learned my addr 172.21.15.162:0/3310601852 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:01.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.028+0000 7f2e57fff700 1 --2- 172.21.15.162:0/3310601852 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2e58141e80 0x7f2e5813dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:01.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.028+0000 7f2e577fe700 1 -- 172.21.15.162:0/3310601852 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2e58141e80 msgr2=0x7f2e5813dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:01.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.028+0000 7f2e577fe700 1 --2- 172.21.15.162:0/3310601852 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2e58141e80 0x7f2e5813dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:01.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.028+0000 7f2e577fe700 1 -- 172.21.15.162:0/3310601852 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2e580b21f0 msgr2=0x7f2e581538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:01.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.028+0000 7f2e577fe700 1 --2- 172.21.15.162:0/3310601852 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2e580b21f0 0x7f2e581538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-28T21:54:01.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.028+0000 7f2e577fe700 1 -- 172.21.15.162:0/3310601852 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e5809e650 con 0x7f2e580ae3d0 2020-11-28T21:54:01.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.028+0000 7f2e577fe700 1 --2- 172.21.15.162:0/3310601852 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e580ae3d0 0x7f2e581533f0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 rx=0x7f2e5805ad70 tx=0x7f2e40007c10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:01.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.028+0000 7f2e54ff9700 1 -- 172.21.15.162:0/3310601852 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e40007320 con 0x7f2e580ae3d0 2020-11-28T21:54:01.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.028+0000 7f2e54ff9700 1 -- 172.21.15.162:0/3310601852 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e40005520 con 0x7f2e580ae3d0 2020-11-28T21:54:01.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.028+0000 7f2e54ff9700 1 -- 172.21.15.162:0/3310601852 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e40005880 con 0x7f2e580ae3d0 2020-11-28T21:54:01.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.028+0000 7f2e5cd1e700 1 -- 172.21.15.162:0/3310601852 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e5809e8f0 con 0x7f2e580ae3d0 2020-11-28T21:54:01.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.028+0000 7f2e5cd1e700 1 -- 172.21.15.162:0/3310601852 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e58063a80 con 0x7f2e580ae3d0 2020-11-28T21:54:01.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.032+0000 7f2e5cd1e700 1 -- 172.21.15.162:0/3310601852 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e58057050 con 0x7f2e580ae3d0 2020-11-28T21:54:01.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.032+0000 7f2e54ff9700 1 -- 172.21.15.162:0/3310601852 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2e40010030 con 0x7f2e580ae3d0 2020-11-28T21:54:01.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.032+0000 7f2e54ff9700 1 --2- 172.21.15.162:0/3310601852 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2e3c055440 0x7f2e3c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:01.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.032+0000 7f2e54ff9700 1 -- 172.21.15.162:0/3310601852 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f2e4006d9d0 con 0x7f2e580ae3d0 2020-11-28T21:54:01.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.032+0000 7f2e56ffd700 1 --2- 172.21.15.162:0/3310601852 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2e3c055440 0x7f2e3c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:01.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.036+0000 7f2e56ffd700 1 --2- 172.21.15.162:0/3310601852 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2e3c055440 0x7f2e3c057900 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7f2e480032b0 tx=0x7f2e48016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:01.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.040+0000 7f2e54ff9700 1 -- 172.21.15.162:0/3310601852 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f2e40043ea0 con 0x7f2e580ae3d0 2020-11-28T21:54:01.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:01.180+0000 7f2e5cd1e700 1 -- 172.21.15.162:0/3310601852 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f2e5805d250 con 0x7f2e580ae3d0 2020-11-28T21:54:02.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.048+0000 7f2e54ff9700 1 -- 172.21.15.162:0/3310601852 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v97) v1 ==== 125+0+0 (secure 0 0 0) 0x7f2e400474c0 con 0x7f2e580ae3d0 2020-11-28T21:54:02.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.052+0000 7f2e5cd1e700 1 -- 172.21.15.162:0/3310601852 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2e3c055440 msgr2=0x7f2e3c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:02.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.052+0000 7f2e5cd1e700 1 --2- 172.21.15.162:0/3310601852 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2e3c055440 0x7f2e3c057900 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7f2e480032b0 tx=0x7f2e48016000).stop 2020-11-28T21:54:02.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.052+0000 7f2e5cd1e700 1 -- 172.21.15.162:0/3310601852 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e580ae3d0 msgr2=0x7f2e581533f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:02.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.052+0000 7f2e5cd1e700 1 --2- 172.21.15.162:0/3310601852 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e580ae3d0 0x7f2e581533f0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 rx=0x7f2e5805ad70 tx=0x7f2e40007c10).stop 2020-11-28T21:54:02.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.052+0000 7f2e5cd1e700 1 -- 172.21.15.162:0/3310601852 shutdown_connections 2020-11-28T21:54:02.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.052+0000 7f2e5cd1e700 1 --2- 172.21.15.162:0/3310601852 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2e3c055440 0x7f2e3c057900 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:02.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.052+0000 7f2e5cd1e700 1 --2- 172.21.15.162:0/3310601852 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2e58141e80 0x7f2e5813dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:02.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.052+0000 7f2e5cd1e700 1 --2- 172.21.15.162:0/3310601852 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e580ae3d0 0x7f2e581533f0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:02.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.052+0000 7f2e5cd1e700 1 --2- 172.21.15.162:0/3310601852 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2e580b21f0 0x7f2e581538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-28T21:54:02.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.052+0000 7f2e5cd1e700 1 -- 172.21.15.162:0/3310601852 >> 172.21.15.162:0/3310601852 conn(0x7f2e580b57d0 msgr2=0x7f2e581408e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:02.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.056+0000 7f2e5cd1e700 1 -- 172.21.15.162:0/3310601852 shutdown_connections 2020-11-28T21:54:02.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.056+0000 7f2e5cd1e700 1 -- 172.21.15.162:0/3310601852 wait complete. 2020-11-28T21:54:02.061 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_data' created 2020-11-28T21:54:02.072 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2020-11-28T21:54:02.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.184+0000 7f4036c6e700 1 Processor -- start 2020-11-28T21:54:02.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.184+0000 7f4036c6e700 1 -- start start 2020-11-28T21:54:02.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.184+0000 7f4036c6e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f40300b21f0 0x7f403013fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:02.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.184+0000 7f4036c6e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f4030060b60 0x7f403014d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:02.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.184+0000 7f4036c6e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40300aff20 con 0x7f40301402a0 2020-11-28T21:54:02.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.184+0000 7f4036c6e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40300af240 con 0x7f4030060b60 2020-11-28T21:54:02.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.184+0000 7f4036c6e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40300b25d0 con 0x7f40300b21f0 2020-11-28T21:54:02.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.184+0000 7f403546b700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f40301402a0 0x7f4030060450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:49556/0 (socket says 172.21.15.162:49556) 2020-11-28T21:54:02.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.184+0000 7f403546b700 1 -- 172.21.15.162:0/1512808315 learned_addr learned my addr 172.21.15.162:0/1512808315 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:02.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.184+0000 7f4035c6c700 1 --2- 172.21.15.162:0/1512808315 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f40300b21f0 0x7f403013fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:02.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.184+0000 7f403646d700 1 --2- 172.21.15.162:0/1512808315 >> v2:172.21.15.118:3300/0 conn(0x7f4030060b60 0x7f403014d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:02.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.184+0000 7f4034c6a700 1 -- 172.21.15.162:0/1512808315 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1663989347 0 0) 0x7f40300aff20 con 0x7f40301402a0 2020-11-28T21:54:02.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.184+0000 7f4034c6a700 1 -- 172.21.15.162:0/1512808315 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4018003230 con 0x7f40301402a0 2020-11-28T21:54:02.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.184+0000 7f4034c6a700 1 -- 172.21.15.162:0/1512808315 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1329421111 0 0) 0x7f4018003230 con 0x7f40301402a0 2020-11-28T21:54:02.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.184+0000 7f4034c6a700 1 -- 172.21.15.162:0/1512808315 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f40300b21f0 msgr2=0x7f403013fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:02.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.184+0000 7f4034c6a700 1 --2- 172.21.15.162:0/1512808315 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f40300b21f0 0x7f403013fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:02.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.184+0000 7f4034c6a700 1 -- 172.21.15.162:0/1512808315 >> v2:172.21.15.118:3300/0 conn(0x7f4030060b60 msgr2=0x7f403014d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:02.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.184+0000 7f4034c6a700 1 --2- 172.21.15.162:0/1512808315 >> v2:172.21.15.118:3300/0 conn(0x7f4030060b60 0x7f403014d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:02.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.184+0000 7f4034c6a700 1 -- 172.21.15.162:0/1512808315 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f403005fff0 con 0x7f40301402a0 2020-11-28T21:54:02.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.184+0000 7f4034c6a700 1 -- 172.21.15.162:0/1512808315 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f40200016d0 con 0x7f40301402a0 2020-11-28T21:54:02.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.184+0000 7f4034c6a700 1 -- 172.21.15.162:0/1512808315 >> v1:172.21.15.162:6789/0 conn(0x7f40301402a0 legacy=0x7f4030060450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:02.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.184+0000 7f4034c6a700 1 --2- 172.21.15.162:0/1512808315 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40180037a0 0x7f4018003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:02.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.184+0000 7f4034c6a700 1 --2- 172.21.15.162:0/1512808315 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f40180044d0 0x7f4018004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:02.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.184+0000 7f4034c6a700 1 --2- 172.21.15.162:0/1512808315 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f40300b21f0 0x7f4018004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:02.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.184+0000 7f4034c6a700 1 -- 172.21.15.162:0/1512808315 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40300af240 con 0x7f40180044d0 2020-11-28T21:54:02.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.184+0000 7f4034c6a700 1 -- 172.21.15.162:0/1512808315 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40300b25d0 con 0x7f40180037a0 2020-11-28T21:54:02.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.184+0000 7f4034c6a700 1 -- 172.21.15.162:0/1512808315 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4018005330 con 0x7f40300b21f0 2020-11-28T21:54:02.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.184+0000 7f4035c6c700 1 --2- 172.21.15.162:0/1512808315 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40180037a0 0x7f4018003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:02.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.184+0000 7f403546b700 1 --2- 172.21.15.162:0/1512808315 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f40180044d0 0x7f4018004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:02.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.184+0000 7f403646d700 1 --2- 172.21.15.162:0/1512808315 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f40300b21f0 0x7f4018004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:02.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.184+0000 7f4035c6c700 1 -- 172.21.15.162:0/1512808315 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f40300b21f0 msgr2=0x7f4018004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:02.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.184+0000 7f4035c6c700 1 --2- 172.21.15.162:0/1512808315 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f40300b21f0 0x7f4018004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:02.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.184+0000 7f4035c6c700 1 -- 172.21.15.162:0/1512808315 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f40180044d0 msgr2=0x7f4018004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:02.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.184+0000 7f4035c6c700 1 --2- 172.21.15.162:0/1512808315 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f40180044d0 0x7f4018004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:02.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.184+0000 7f4035c6c700 1 -- 172.21.15.162:0/1512808315 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4018003230 con 0x7f40180037a0 2020-11-28T21:54:02.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.184+0000 7f4035c6c700 1 --2- 172.21.15.162:0/1512808315 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40180037a0 0x7f4018003bc0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 rx=0x7f402c00c000 tx=0x7f402c008040).ready entity=mon.1 client_cookie=d52e14e834086fde server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:02.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.184+0000 7f4034c6a700 1 -- 172.21.15.162:0/1512808315 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f402c00e970 con 0x7f40180037a0 2020-11-28T21:54:02.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.184+0000 7f4034c6a700 1 -- 172.21.15.162:0/1512808315 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f402c00ead0 con 0x7f40180037a0 2020-11-28T21:54:02.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.184+0000 7f4034c6a700 1 -- 172.21.15.162:0/1512808315 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f402c00ee30 con 0x7f40180037a0 2020-11-28T21:54:02.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.184+0000 7f4036c6e700 1 -- 172.21.15.162:0/1512808315 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40180037a0 msgr2=0x7f4018003bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:02.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.184+0000 7f4036c6e700 1 --2- 172.21.15.162:0/1512808315 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40180037a0 0x7f4018003bc0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 rx=0x7f402c00c000 tx=0x7f402c008040).stop 2020-11-28T21:54:02.200 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.184+0000 7f403646d700 1 -- 172.21.15.162:0/1512808315 reap_dead start 2020-11-28T21:54:02.200 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.188+0000 7f4036c6e700 1 -- 172.21.15.162:0/1512808315 shutdown_connections 2020-11-28T21:54:02.200 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.188+0000 7f4036c6e700 1 -- 172.21.15.162:0/1512808315 >> 172.21.15.162:0/1512808315 conn(0x7f40300b57d0 msgr2=0x7f40300b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:02.200 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.188+0000 7f4036c6e700 1 -- 172.21.15.162:0/1512808315 shutdown_connections 2020-11-28T21:54:02.200 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.188+0000 7f4036c6e700 1 -- 172.21.15.162:0/1512808315 wait complete. 2020-11-28T21:54:02.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.188+0000 7f4036c6e700 1 Processor -- start 2020-11-28T21:54:02.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.188+0000 7f4036c6e700 1 -- start start 2020-11-28T21:54:02.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.188+0000 7f4036c6e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4030151de0 0x7f40301433f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:02.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.188+0000 7f4036c6e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4030146cb0 0x7f4030066370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:02.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.188+0000 7f4036c6e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4030066860 0x7f40300aa530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:02.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.188+0000 7f4036c6e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4030067150 con 0x7f4030146cb0 2020-11-28T21:54:02.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.188+0000 7f4036c6e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40300673f0 con 0x7f4030066860 2020-11-28T21:54:02.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.188+0000 7f4036c6e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40301525b0 con 0x7f4030151de0 2020-11-28T21:54:02.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.188+0000 7f403646d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4030066860 0x7f40300aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:02.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.188+0000 7f403646d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4030066860 0x7f40300aa530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42726/0 (socket says 172.21.15.162:42726) 2020-11-28T21:54:02.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.188+0000 7f403646d700 1 -- 172.21.15.162:0/3691196033 learned_addr learned my addr 172.21.15.162:0/3691196033 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:02.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.188+0000 7f403546b700 1 --2- 172.21.15.162:0/3691196033 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4030146cb0 0x7f4030066370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:02.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.188+0000 7f4035c6c700 1 --2- 172.21.15.162:0/3691196033 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4030151de0 0x7f40301433f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:02.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.188+0000 7f403646d700 1 -- 172.21.15.162:0/3691196033 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4030151de0 msgr2=0x7f40301433f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:02.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.188+0000 7f403646d700 1 --2- 172.21.15.162:0/3691196033 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4030151de0 0x7f40301433f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:02.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.188+0000 7f403646d700 1 -- 172.21.15.162:0/3691196033 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4030146cb0 msgr2=0x7f4030066370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:02.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.188+0000 7f403646d700 1 --2- 172.21.15.162:0/3691196033 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4030146cb0 0x7f4030066370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:02.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.188+0000 7f403646d700 1 -- 172.21.15.162:0/3691196033 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40300ab550 con 0x7f4030066860 2020-11-28T21:54:02.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.188+0000 7f403546b700 1 --2- 172.21.15.162:0/3691196033 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4030146cb0 0x7f4030066370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:54:02.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.188+0000 7f403646d700 1 --2- 172.21.15.162:0/3691196033 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4030066860 0x7f40300aa530 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 rx=0x7f402800b000 tx=0x7f40280091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:02.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.192+0000 7f401effd700 1 -- 172.21.15.162:0/3691196033 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4028019070 con 0x7f4030066860 2020-11-28T21:54:02.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.192+0000 7f401effd700 1 -- 172.21.15.162:0/3691196033 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f402800fab0 con 0x7f4030066860 2020-11-28T21:54:02.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.192+0000 7f4036c6e700 1 -- 172.21.15.162:0/3691196033 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40301521c0 con 0x7f4030066860 2020-11-28T21:54:02.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.192+0000 7f401effd700 1 -- 172.21.15.162:0/3691196033 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f402800fe10 con 0x7f4030066860 2020-11-28T21:54:02.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.192+0000 7f4036c6e700 1 -- 172.21.15.162:0/3691196033 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4030066eb0 con 0x7f4030066860 2020-11-28T21:54:02.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.192+0000 7f4036c6e700 1 -- 172.21.15.162:0/3691196033 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4030152770 con 0x7f4030066860 2020-11-28T21:54:02.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.192+0000 7f401effd700 1 -- 172.21.15.162:0/3691196033 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4028014440 con 0x7f4030066860 2020-11-28T21:54:02.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.192+0000 7f401effd700 1 --2- 172.21.15.162:0/3691196033 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4010055330 0x7f40100577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:02.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.192+0000 7f401effd700 1 -- 172.21.15.162:0/3691196033 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5957+0+0 (secure 0 0 0) 0x7f402806e760 con 0x7f4030066860 2020-11-28T21:54:02.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.192+0000 7f4035c6c700 1 --2- 172.21.15.162:0/3691196033 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4010055330 0x7f40100577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:02.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.192+0000 7f4035c6c700 1 --2- 172.21.15.162:0/3691196033 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4010055330 0x7f40100577f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f402c007000 tx=0x7f402c01a000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:02.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.200+0000 7f401effd700 1 -- 172.21.15.162:0/3691196033 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4028044a70 con 0x7f4030066860 2020-11-28T21:54:02.387 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.384+0000 7f4036c6e700 1 -- 172.21.15.162:0/3691196033 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f4030057050 con 0x7f4030066860 2020-11-28T21:54:02.402 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.396+0000 7f401effd700 1 -- 172.21.15.162:0/3691196033 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 17 and data pool 18 v45) v1 ==== 189+0+0 (secure 0 0 0) 0x7f402806ce60 con 0x7f4030066860 2020-11-28T21:54:02.410 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.400+0000 7f4036c6e700 1 -- 172.21.15.162:0/3691196033 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4010055330 msgr2=0x7f40100577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:02.410 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.400+0000 7f4036c6e700 1 --2- 172.21.15.162:0/3691196033 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4010055330 0x7f40100577f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f402c007000 tx=0x7f402c01a000).stop 2020-11-28T21:54:02.410 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.400+0000 7f4036c6e700 1 -- 172.21.15.162:0/3691196033 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4030066860 msgr2=0x7f40300aa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:02.411 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.400+0000 7f4036c6e700 1 --2- 172.21.15.162:0/3691196033 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4030066860 0x7f40300aa530 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 rx=0x7f402800b000 tx=0x7f40280091a0).stop 2020-11-28T21:54:02.411 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.400+0000 7f4036c6e700 1 -- 172.21.15.162:0/3691196033 shutdown_connections 2020-11-28T21:54:02.411 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.400+0000 7f4036c6e700 1 --2- 172.21.15.162:0/3691196033 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4010055330 0x7f40100577f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:02.412 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.400+0000 7f4036c6e700 1 --2- 172.21.15.162:0/3691196033 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4030151de0 0x7f40301433f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:02.412 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.400+0000 7f4036c6e700 1 --2- 172.21.15.162:0/3691196033 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4030066860 0x7f40300aa530 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:02.412 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.400+0000 7f4036c6e700 1 --2- 172.21.15.162:0/3691196033 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4030146cb0 0x7f4030066370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:02.413 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.400+0000 7f4036c6e700 1 -- 172.21.15.162:0/3691196033 >> 172.21.15.162:0/3691196033 conn(0x7f40300b57d0 msgr2=0x7f40301420c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:02.413 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.400+0000 7f4036c6e700 1 -- 172.21.15.162:0/3691196033 shutdown_connections 2020-11-28T21:54:02.413 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.400+0000 7f4036c6e700 1 -- 172.21.15.162:0/3691196033 wait complete. 2020-11-28T21:54:02.414 INFO:teuthology.orchestra.run.smithi162.stderr:new fs with metadata pool 17 and data pool 18 2020-11-28T21:54:02.424 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:54:02.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.520+0000 7fe8e8ec2700 1 Processor -- start 2020-11-28T21:54:02.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.520+0000 7fe8e8ec2700 1 -- start start 2020-11-28T21:54:02.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.520+0000 7fe8e8ec2700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe8e40b1990 0x7fe8e4140890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:02.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.520+0000 7fe8e8ec2700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8e40b2870 con 0x7fe8e40b1d70 2020-11-28T21:54:02.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.520+0000 7fe8e8ec2700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8e40b22e0 con 0x7fe8e40b11b0 2020-11-28T21:54:02.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.520+0000 7fe8e8ec2700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8e4146e20 con 0x7fe8e40b1990 2020-11-28T21:54:02.526 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.520+0000 7fe8e1d9b700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fe8e40b11b0 0x7fe8e4140d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:49584/0 (socket says 172.21.15.162:49584) 2020-11-28T21:54:02.526 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.520+0000 7fe8e1d9b700 1 -- 172.21.15.162:0/4184998300 learned_addr learned my addr 172.21.15.162:0/4184998300 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:02.526 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.520+0000 7fe8e259c700 1 --2- 172.21.15.162:0/4184998300 >> v2:172.21.15.118:3300/0 conn(0x7fe8e40b1990 0x7fe8e4140890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:02.526 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.520+0000 7fe8e159a700 1 -- 172.21.15.162:0/4184998300 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1398658608 0 0) 0x7fe8e40b22e0 con 0x7fe8e40b11b0 2020-11-28T21:54:02.526 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.520+0000 7fe8e159a700 1 -- 172.21.15.162:0/4184998300 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8c4003230 con 0x7fe8e40b11b0 2020-11-28T21:54:02.527 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.520+0000 7fe8e159a700 1 -- 172.21.15.162:0/4184998300 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1868114468 0 0) 0x7fe8e40b2870 con 0x7fe8e40b1d70 2020-11-28T21:54:02.527 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.520+0000 7fe8e159a700 1 -- 172.21.15.162:0/4184998300 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8e40b22e0 con 0x7fe8e40b1d70 2020-11-28T21:54:02.527 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.520+0000 7fe8e259c700 1 -- 172.21.15.162:0/4184998300 >> v1:172.21.15.162:6789/0 conn(0x7fe8e40b11b0 legacy=0x7fe8e4140d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:02.527 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.520+0000 7fe8e159a700 1 -- 172.21.15.162:0/4184998300 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1495600600 0 0) 0x7fe8c4003230 con 0x7fe8e40b11b0 2020-11-28T21:54:02.527 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.520+0000 7fe8e259c700 1 -- 172.21.15.162:0/4184998300 >> v1:172.21.15.118:6789/0 conn(0x7fe8e40b1d70 legacy=0x7fe8e414a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:02.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.520+0000 7fe8e259c700 1 -- 172.21.15.162:0/4184998300 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e40b0740 con 0x7fe8e40b1990 2020-11-28T21:54:02.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.520+0000 7fe8e259c700 1 --2- 172.21.15.162:0/4184998300 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe8e40b1990 0x7fe8e4140890 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7fe8dc001450 tx=0x7fe8d4006dc0).ready entity=mon.0 client_cookie=6f591c8b495f5d49 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:02.530 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.520+0000 7fe8e159a700 1 -- 172.21.15.162:0/4184998300 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe8dc001a50 con 0x7fe8e40b1990 2020-11-28T21:54:02.531 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.520+0000 7fe8e159a700 1 -- 172.21.15.162:0/4184998300 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d400d610 con 0x7fe8e40b1990 2020-11-28T21:54:02.531 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.520+0000 7fe8e159a700 1 -- 172.21.15.162:0/4184998300 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe8d400d780 con 0x7fe8e40b1990 2020-11-28T21:54:02.531 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.520+0000 7fe8e8ec2700 1 -- 172.21.15.162:0/4184998300 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe8e40b1990 msgr2=0x7fe8e4140890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:02.531 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.520+0000 7fe8e8ec2700 1 --2- 172.21.15.162:0/4184998300 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe8e40b1990 0x7fe8e4140890 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7fe8dc001450 tx=0x7fe8d4006dc0).stop 2020-11-28T21:54:02.531 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.524+0000 7fe8e8ec2700 1 -- 172.21.15.162:0/4184998300 shutdown_connections 2020-11-28T21:54:02.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.524+0000 7fe8e8ec2700 1 --2- 172.21.15.162:0/4184998300 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe8e40b1990 0x7fe8e4140890 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:02.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.524+0000 7fe8e8ec2700 1 -- 172.21.15.162:0/4184998300 >> 172.21.15.162:0/4184998300 conn(0x7fe8e405b970 msgr2=0x7fe8e405bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:02.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.524+0000 7fe8e8ec2700 1 -- 172.21.15.162:0/4184998300 shutdown_connections 2020-11-28T21:54:02.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.524+0000 7fe8e8ec2700 1 -- 172.21.15.162:0/4184998300 wait complete. 2020-11-28T21:54:02.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.524+0000 7fe8e8ec2700 1 Processor -- start 2020-11-28T21:54:02.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.524+0000 7fe8e8ec2700 1 -- start start 2020-11-28T21:54:02.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.524+0000 7fe8e8ec2700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe8e40b11b0 0x7fe8e4145f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:02.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.524+0000 7fe8e8ec2700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe8e40b1990 0x7fe8e4066e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:02.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.524+0000 7fe8e8ec2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8e40b1d70 0x7fe8e41593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:02.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.524+0000 7fe8e8ec2700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e4067330 con 0x7fe8e40b11b0 2020-11-28T21:54:02.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.524+0000 7fe8e8ec2700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e4143300 con 0x7fe8e40b1d70 2020-11-28T21:54:02.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.524+0000 7fe8e8ec2700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e4066700 con 0x7fe8e40b1990 2020-11-28T21:54:02.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.524+0000 7fe8e2d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8e40b1d70 0x7fe8e41593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:02.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.524+0000 7fe8e2d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8e40b1d70 0x7fe8e41593e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42748/0 (socket says 172.21.15.162:42748) 2020-11-28T21:54:02.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.524+0000 7fe8e2d9d700 1 -- 172.21.15.162:0/2842268468 learned_addr learned my addr 172.21.15.162:0/2842268468 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:02.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.524+0000 7fe8e1d9b700 1 --2- 172.21.15.162:0/2842268468 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe8e40b1990 0x7fe8e4066e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:02.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.524+0000 7fe8e259c700 1 --2- 172.21.15.162:0/2842268468 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe8e40b11b0 0x7fe8e4145f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:02.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.524+0000 7fe8e2d9d700 1 -- 172.21.15.162:0/2842268468 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe8e40b1990 msgr2=0x7fe8e4066e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:02.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.524+0000 7fe8e2d9d700 1 --2- 172.21.15.162:0/2842268468 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe8e40b1990 0x7fe8e4066e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:02.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.524+0000 7fe8e2d9d700 1 -- 172.21.15.162:0/2842268468 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe8e40b11b0 msgr2=0x7fe8e4145f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:02.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.524+0000 7fe8e2d9d700 1 --2- 172.21.15.162:0/2842268468 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe8e40b11b0 0x7fe8e4145f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:02.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.524+0000 7fe8e2d9d700 1 -- 172.21.15.162:0/2842268468 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e40669a0 con 0x7fe8e40b1d70 2020-11-28T21:54:02.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.524+0000 7fe8e259c700 1 --2- 172.21.15.162:0/2842268468 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe8e40b11b0 0x7fe8e4145f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:54:02.537 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.524+0000 7fe8e1d9b700 1 --2- 172.21.15.162:0/2842268468 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe8e40b1990 0x7fe8e4066e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:54:02.537 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.524+0000 7fe8e2d9d700 1 --2- 172.21.15.162:0/2842268468 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8e40b1d70 0x7fe8e41593e0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 rx=0x7fe8cc0010d0 tx=0x7fe8cc0079e0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:02.537 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.528+0000 7fe8db7fe700 1 -- 172.21.15.162:0/2842268468 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe8cc00f470 con 0x7fe8e40b1d70 2020-11-28T21:54:02.537 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.528+0000 7fe8db7fe700 1 -- 172.21.15.162:0/2842268468 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8cc00f5d0 con 0x7fe8e40b1d70 2020-11-28T21:54:02.537 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.528+0000 7fe8e8ec2700 1 -- 172.21.15.162:0/2842268468 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8e41598d0 con 0x7fe8e40b1d70 2020-11-28T21:54:02.538 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.528+0000 7fe8db7fe700 1 -- 172.21.15.162:0/2842268468 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe8cc00f930 con 0x7fe8e40b1d70 2020-11-28T21:54:02.538 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.528+0000 7fe8e8ec2700 1 -- 172.21.15.162:0/2842268468 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8e4142e60 con 0x7fe8e40b1d70 2020-11-28T21:54:02.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.528+0000 7fe8db7fe700 1 -- 172.21.15.162:0/2842268468 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe8cc019030 con 0x7fe8e40b1d70 2020-11-28T21:54:02.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.528+0000 7fe8e8ec2700 1 -- 172.21.15.162:0/2842268468 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8e4143cc0 con 0x7fe8e40b1d70 2020-11-28T21:54:02.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.528+0000 7fe8db7fe700 1 --2- 172.21.15.162:0/2842268468 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe8c80554e0 0x7fe8c80579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:02.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.528+0000 7fe8db7fe700 1 -- 172.21.15.162:0/2842268468 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe8cc06d430 con 0x7fe8e40b1d70 2020-11-28T21:54:02.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.528+0000 7fe8e259c700 1 --2- 172.21.15.162:0/2842268468 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe8c80554e0 0x7fe8c80579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:02.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.528+0000 7fe8e259c700 1 --2- 172.21.15.162:0/2842268468 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe8c80554e0 0x7fe8c80579a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7fe8d4018000 tx=0x7fe8d4015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:02.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.536+0000 7fe8db7fe700 1 -- 172.21.15.162:0/2842268468 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe8e4143cc0 con 0x7fe8e40b1d70 2020-11-28T21:54:02.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.672+0000 7fe8e8ec2700 1 -- 172.21.15.162:0/2842268468 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe8e4066560 con 0x7fe8e40b1d70 2020-11-28T21:54:02.680 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.676+0000 7fe8db7fe700 1 -- 172.21.15.162:0/2842268468 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+16150 (secure 0 0 0) 0x7fe8e4066560 con 0x7fe8e40b1d70 2020-11-28T21:54:02.680 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:54:02.681 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":98,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:54:02.387942+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:53:59.925390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2020-11-28T21:54:01.236032+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:54:02.682 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.676+0000 7fe8e8ec2700 1 -- 172.21.15.162:0/2842268468 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe8c80554e0 msgr2=0x7fe8c80579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:02.683 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.676+0000 7fe8e8ec2700 1 --2- 172.21.15.162:0/2842268468 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe8c80554e0 0x7fe8c80579a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7fe8d4018000 tx=0x7fe8d4015000).stop 2020-11-28T21:54:02.683 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.676+0000 7fe8e8ec2700 1 -- 172.21.15.162:0/2842268468 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8e40b1d70 msgr2=0x7fe8e41593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:02.683 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.676+0000 7fe8e8ec2700 1 --2- 172.21.15.162:0/2842268468 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8e40b1d70 0x7fe8e41593e0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 rx=0x7fe8cc0010d0 tx=0x7fe8cc0079e0).stop 2020-11-28T21:54:02.683 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.680+0000 7fe8e8ec2700 1 -- 172.21.15.162:0/2842268468 shutdown_connections 2020-11-28T21:54:02.684 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.680+0000 7fe8e8ec2700 1 --2- 172.21.15.162:0/2842268468 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe8c80554e0 0x7fe8c80579a0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:02.684 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.680+0000 7fe8e8ec2700 1 --2- 172.21.15.162:0/2842268468 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8e40b1d70 0x7fe8e41593e0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:02.684 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.680+0000 7fe8e8ec2700 1 --2- 172.21.15.162:0/2842268468 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe8e40b11b0 0x7fe8e4145f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:02.684 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.680+0000 7fe8e8ec2700 1 --2- 172.21.15.162:0/2842268468 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe8e40b1990 0x7fe8e4066e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:02.685 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.680+0000 7fe8e8ec2700 1 -- 172.21.15.162:0/2842268468 >> 172.21.15.162:0/2842268468 conn(0x7fe8e405b970 msgr2=0x7fe8e40b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:02.685 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.680+0000 7fe8e8ec2700 1 -- 172.21.15.162:0/2842268468 shutdown_connections 2020-11-28T21:54:02.685 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.680+0000 7fe8e8ec2700 1 -- 172.21.15.162:0/2842268468 wait complete. 2020-11-28T21:54:02.700 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:54:02.811 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.808+0000 7fcebffff700 1 Processor -- start 2020-11-28T21:54:02.811 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.808+0000 7fcebffff700 1 -- start start 2020-11-28T21:54:02.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.808+0000 7fcebffff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcec00b0000 0x7fcec0151a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:02.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.808+0000 7fcebffff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcec0098f10 con 0x7fcec00994f0 2020-11-28T21:54:02.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.808+0000 7fcebffff700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcec00b0680 con 0x7fcec0098b30 2020-11-28T21:54:02.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.808+0000 7fcebffff700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcec00b1030 con 0x7fcec00b0000 2020-11-28T21:54:02.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.808+0000 7fcebeffd700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fcec0098b30 0x7fcec0060000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:49596/0 (socket says 172.21.15.162:49596) 2020-11-28T21:54:02.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.808+0000 7fcebeffd700 1 -- 172.21.15.162:0/1691899704 learned_addr learned my addr 172.21.15.162:0/1691899704 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:02.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.808+0000 7fcebf7fe700 1 --2- 172.21.15.162:0/1691899704 >> v2:172.21.15.162:3300/0 conn(0x7fcec00b0000 0x7fcec0151a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:02.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.808+0000 7fcebdffb700 1 -- 172.21.15.162:0/1691899704 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 18724474 0 0) 0x7fcec00b0680 con 0x7fcec0098b30 2020-11-28T21:54:02.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.808+0000 7fcebdffb700 1 -- 172.21.15.162:0/1691899704 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcea4003230 con 0x7fcec0098b30 2020-11-28T21:54:02.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.808+0000 7fcebdffb700 1 -- 172.21.15.162:0/1691899704 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1942320981 0 0) 0x7fcea4003230 con 0x7fcec0098b30 2020-11-28T21:54:02.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.808+0000 7fcebf7fe700 1 -- 172.21.15.162:0/1691899704 >> v1:172.21.15.162:6789/0 conn(0x7fcec0098b30 legacy=0x7fcec0060000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:02.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.808+0000 7fcebf7fe700 1 -- 172.21.15.162:0/1691899704 >> v1:172.21.15.118:6789/0 conn(0x7fcec00994f0 legacy=0x7fcec0060710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:02.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.808+0000 7fcebf7fe700 1 -- 172.21.15.162:0/1691899704 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec00b0920 con 0x7fcec00b0000 2020-11-28T21:54:02.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.808+0000 7fcebf7fe700 1 --2- 172.21.15.162:0/1691899704 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcec00b0000 0x7fcec0151a30 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 rx=0x7fceb0001870 tx=0x7fcea8006dc0).ready entity=mon.1 client_cookie=e04da183901ab2ba server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:02.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.808+0000 7fcebdffb700 1 -- 172.21.15.162:0/1691899704 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fceb00016d0 con 0x7fcec00b0000 2020-11-28T21:54:02.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.808+0000 7fcebdffb700 1 -- 172.21.15.162:0/1691899704 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcea8009670 con 0x7fcec00b0000 2020-11-28T21:54:02.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.808+0000 7fcebdffb700 1 -- 172.21.15.162:0/1691899704 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcea80097e0 con 0x7fcec00b0000 2020-11-28T21:54:02.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.808+0000 7fcebffff700 1 -- 172.21.15.162:0/1691899704 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcec00b0000 msgr2=0x7fcec0151a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:02.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.808+0000 7fcebffff700 1 --2- 172.21.15.162:0/1691899704 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcec00b0000 0x7fcec0151a30 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 rx=0x7fceb0001870 tx=0x7fcea8006dc0).stop 2020-11-28T21:54:02.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.808+0000 7fcebffff700 1 -- 172.21.15.162:0/1691899704 shutdown_connections 2020-11-28T21:54:02.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.808+0000 7fcebffff700 1 --2- 172.21.15.162:0/1691899704 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcec00b0000 0x7fcec0151a30 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:02.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.808+0000 7fcebffff700 1 -- 172.21.15.162:0/1691899704 >> 172.21.15.162:0/1691899704 conn(0x7fcec005b970 msgr2=0x7fcec005bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:02.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.808+0000 7fcebffff700 1 -- 172.21.15.162:0/1691899704 shutdown_connections 2020-11-28T21:54:02.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.808+0000 7fcebffff700 1 -- 172.21.15.162:0/1691899704 wait complete. 2020-11-28T21:54:02.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.812+0000 7fcebffff700 1 Processor -- start 2020-11-28T21:54:02.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.812+0000 7fcebffff700 1 -- start start 2020-11-28T21:54:02.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.816+0000 7fcebffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcec0098b30 0x7fcec00aaf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:02.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.816+0000 7fcebffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcec00994f0 0x7fcec0145b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:02.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.816+0000 7fcebffff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcec00b0000 0x7fcec0146080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:02.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.816+0000 7fcebffff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec00ad4e0 con 0x7fcec00994f0 2020-11-28T21:54:02.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.816+0000 7fcebffff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec00ac960 con 0x7fcec0098b30 2020-11-28T21:54:02.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.816+0000 7fcebffff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec00acc00 con 0x7fcec00b0000 2020-11-28T21:54:02.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.816+0000 7fcebeffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcec0098b30 0x7fcec00aaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:02.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.816+0000 7fcebeffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcec0098b30 0x7fcec00aaf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42758/0 (socket says 172.21.15.162:42758) 2020-11-28T21:54:02.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.816+0000 7fcebeffd700 1 -- 172.21.15.162:0/3068688464 learned_addr learned my addr 172.21.15.162:0/3068688464 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:02.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.816+0000 7fcebe7fc700 1 --2- 172.21.15.162:0/3068688464 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcec00994f0 0x7fcec0145b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:02.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.816+0000 7fcebf7fe700 1 --2- 172.21.15.162:0/3068688464 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcec00b0000 0x7fcec0146080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:02.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.816+0000 7fcebeffd700 1 -- 172.21.15.162:0/3068688464 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcec00b0000 msgr2=0x7fcec0146080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:02.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.816+0000 7fcebeffd700 1 --2- 172.21.15.162:0/3068688464 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcec00b0000 0x7fcec0146080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:02.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.816+0000 7fcebeffd700 1 -- 172.21.15.162:0/3068688464 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcec00994f0 msgr2=0x7fcec0145b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:02.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.816+0000 7fcebeffd700 1 --2- 172.21.15.162:0/3068688464 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcec00994f0 0x7fcec0145b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:02.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.816+0000 7fcebeffd700 1 -- 172.21.15.162:0/3068688464 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec00acea0 con 0x7fcec0098b30 2020-11-28T21:54:02.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.816+0000 7fcebe7fc700 1 --2- 172.21.15.162:0/3068688464 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcec00994f0 0x7fcec0145b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:54:02.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.816+0000 7fcebf7fe700 1 --2- 172.21.15.162:0/3068688464 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcec00b0000 0x7fcec0146080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:54:02.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.816+0000 7fcebeffd700 1 --2- 172.21.15.162:0/3068688464 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcec0098b30 0x7fcec00aaf40 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 rx=0x7fceb0001930 tx=0x7fceb0001280).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:02.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.816+0000 7fceb7fff700 1 -- 172.21.15.162:0/3068688464 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fceb0007170 con 0x7fcec0098b30 2020-11-28T21:54:02.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.816+0000 7fceb7fff700 1 -- 172.21.15.162:0/3068688464 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceb00072d0 con 0x7fcec0098b30 2020-11-28T21:54:02.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.816+0000 7fceb7fff700 1 -- 172.21.15.162:0/3068688464 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fceb00048f0 con 0x7fcec0098b30 2020-11-28T21:54:02.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.816+0000 7fcebffff700 1 -- 172.21.15.162:0/3068688464 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcec00ac2c0 con 0x7fcec0098b30 2020-11-28T21:54:02.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.816+0000 7fcebffff700 1 -- 172.21.15.162:0/3068688464 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec0063340 con 0x7fcec0098b30 2020-11-28T21:54:02.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.816+0000 7fcebffff700 1 -- 172.21.15.162:0/3068688464 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcec0057050 con 0x7fcec0098b30 2020-11-28T21:54:02.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.816+0000 7fceb7fff700 1 -- 172.21.15.162:0/3068688464 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fceb0010070 con 0x7fcec0098b30 2020-11-28T21:54:02.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.820+0000 7fceb7fff700 1 --2- 172.21.15.162:0/3068688464 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fce980553b0 0x7fce98057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:02.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.820+0000 7fcebe7fc700 1 --2- 172.21.15.162:0/3068688464 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fce980553b0 0x7fce98057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:02.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.820+0000 7fceb7fff700 1 -- 172.21.15.162:0/3068688464 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fceb006d880 con 0x7fcec0098b30 2020-11-28T21:54:02.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.820+0000 7fcebe7fc700 1 --2- 172.21.15.162:0/3068688464 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fce980553b0 0x7fce98057870 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7fceb800b000 tx=0x7fceb8009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:02.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.824+0000 7fceb7fff700 1 -- 172.21.15.162:0/3068688464 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fceb0043b20 con 0x7fcec0098b30 2020-11-28T21:54:02.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.964+0000 7fcebffff700 1 -- 172.21.15.162:0/3068688464 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcec0068350 con 0x7fcec0098b30 2020-11-28T21:54:02.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.968+0000 7fceb7fff700 1 -- 172.21.15.162:0/3068688464 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+16150 (secure 0 0 0) 0x7fceb0019030 con 0x7fcec0098b30 2020-11-28T21:54:02.971 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:54:02.971 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":98,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:54:02.387942+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:53:59.925390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2020-11-28T21:54:01.236032+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:54:02.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.968+0000 7fcebffff700 1 -- 172.21.15.162:0/3068688464 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fce980553b0 msgr2=0x7fce98057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:02.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.968+0000 7fcebffff700 1 --2- 172.21.15.162:0/3068688464 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fce980553b0 0x7fce98057870 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7fceb800b000 tx=0x7fceb8009040).stop 2020-11-28T21:54:02.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.968+0000 7fcebffff700 1 -- 172.21.15.162:0/3068688464 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcec0098b30 msgr2=0x7fcec00aaf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:02.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.968+0000 7fcebffff700 1 --2- 172.21.15.162:0/3068688464 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcec0098b30 0x7fcec00aaf40 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 rx=0x7fceb0001930 tx=0x7fceb0001280).stop 2020-11-28T21:54:02.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.968+0000 7fcebffff700 1 -- 172.21.15.162:0/3068688464 shutdown_connections 2020-11-28T21:54:02.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.968+0000 7fcebffff700 1 --2- 172.21.15.162:0/3068688464 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fce980553b0 0x7fce98057870 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:02.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.968+0000 7fcebffff700 1 --2- 172.21.15.162:0/3068688464 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcec00b0000 0x7fcec0146080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:02.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.968+0000 7fcebffff700 1 --2- 172.21.15.162:0/3068688464 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcec0098b30 0x7fcec00aaf40 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:02.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.968+0000 7fcebffff700 1 --2- 172.21.15.162:0/3068688464 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcec00994f0 0x7fcec0145b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:02.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.968+0000 7fcebffff700 1 -- 172.21.15.162:0/3068688464 >> 172.21.15.162:0/3068688464 conn(0x7fcec005b970 msgr2=0x7fcec0149d90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:02.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.968+0000 7fcebffff700 1 -- 172.21.15.162:0/3068688464 shutdown_connections 2020-11-28T21:54:02.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:02.968+0000 7fcebffff700 1 -- 172.21.15.162:0/3068688464 wait complete. 2020-11-28T21:54:02.985 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2020-11-28T21:54:03.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.096+0000 7f8aa49e8700 1 Processor -- start 2020-11-28T21:54:03.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.096+0000 7f8aa49e8700 1 -- start start 2020-11-28T21:54:03.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.096+0000 7f8aa49e8700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8aa00b3950 0x7f8aa00b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:03.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.096+0000 7f8aa49e8700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8aa00b18c0 0x7f8aa013dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:03.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.096+0000 7f8aa49e8700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f8aa00b44e0 0x7f8aa013e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:03.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.096+0000 7f8aa49e8700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aa00aab50 con 0x7f8aa00b44e0 2020-11-28T21:54:03.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.096+0000 7f8aa49e8700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aa00b1f50 con 0x7f8aa00b3950 2020-11-28T21:54:03.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.096+0000 7f8aa49e8700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa0143e90 con 0x7f8aa00b18c0 2020-11-28T21:54:03.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.096+0000 7f8a9ed9d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8aa00b3950 0x7f8aa00b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:03.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.096+0000 7f8a9ed9d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8aa00b3950 0x7f8aa00b3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42766/0 (socket says 172.21.15.162:42766) 2020-11-28T21:54:03.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.096+0000 7f8a9ed9d700 1 -- 172.21.15.162:0/536598144 learned_addr learned my addr 172.21.15.162:0/536598144 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:03.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.100+0000 7f8a9f59e700 1 --2- 172.21.15.162:0/536598144 >> v2:172.21.15.118:3300/0 conn(0x7f8aa00b44e0 0x7f8aa013e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:03.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.100+0000 7f8a9e59c700 1 --2- 172.21.15.162:0/536598144 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8aa00b18c0 0x7f8aa013dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:03.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.100+0000 7f8a9ed9d700 1 -- 172.21.15.162:0/536598144 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8aa00b18c0 msgr2=0x7f8aa013dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:03.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.100+0000 7f8a9ed9d700 1 --2- 172.21.15.162:0/536598144 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8aa00b18c0 0x7f8aa013dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:03.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.100+0000 7f8a9ed9d700 1 -- 172.21.15.162:0/536598144 >> v2:172.21.15.118:3300/0 conn(0x7f8aa00b44e0 msgr2=0x7f8aa013e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:03.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.100+0000 7f8a9ed9d700 1 --2- 172.21.15.162:0/536598144 >> v2:172.21.15.118:3300/0 conn(0x7f8aa00b44e0 0x7f8aa013e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:03.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.100+0000 7f8a9ed9d700 1 -- 172.21.15.162:0/536598144 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa00aa8b0 con 0x7f8aa00b3950 2020-11-28T21:54:03.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.100+0000 7f8a9f59e700 1 --2- 172.21.15.162:0/536598144 >> v2:172.21.15.118:3300/0 conn(0x7f8aa00b44e0 0x7f8aa013e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:54:03.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.100+0000 7f8a9e59c700 1 --2- 172.21.15.162:0/536598144 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8aa00b18c0 0x7f8aa013dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:54:03.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.100+0000 7f8a9ed9d700 1 --2- 172.21.15.162:0/536598144 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8aa00b3950 0x7f8aa00b3d30 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 rx=0x7f8a90002f30 tx=0x7f8a90006e80).ready entity=mon.1 client_cookie=8819358e3db37c31 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:03.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.100+0000 7f8a9dd9b700 1 -- 172.21.15.162:0/536598144 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a900097f0 con 0x7f8aa00b3950 2020-11-28T21:54:03.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.100+0000 7f8a9dd9b700 1 -- 172.21.15.162:0/536598144 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a90009950 con 0x7f8aa00b3950 2020-11-28T21:54:03.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.100+0000 7f8a9dd9b700 1 -- 172.21.15.162:0/536598144 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a90009cb0 con 0x7f8aa00b3950 2020-11-28T21:54:03.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.100+0000 7f8aa49e8700 1 -- 172.21.15.162:0/536598144 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8aa00b3950 msgr2=0x7f8aa00b3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:03.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.100+0000 7f8aa49e8700 1 --2- 172.21.15.162:0/536598144 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8aa00b3950 0x7f8aa00b3d30 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 rx=0x7f8a90002f30 tx=0x7f8a90006e80).stop 2020-11-28T21:54:03.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.100+0000 7f8aa49e8700 1 -- 172.21.15.162:0/536598144 shutdown_connections 2020-11-28T21:54:03.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.100+0000 7f8aa49e8700 1 --2- 172.21.15.162:0/536598144 >> v2:172.21.15.118:3300/0 conn(0x7f8aa00b44e0 0x7f8aa013e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:03.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.100+0000 7f8aa49e8700 1 --2- 172.21.15.162:0/536598144 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8aa00b3950 0x7f8aa00b3d30 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:03.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.100+0000 7f8aa49e8700 1 --2- 172.21.15.162:0/536598144 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8aa00b18c0 0x7f8aa013dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:03.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.100+0000 7f8aa49e8700 1 -- 172.21.15.162:0/536598144 >> 172.21.15.162:0/536598144 conn(0x7f8aa0062630 msgr2=0x7f8aa0062a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:03.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.100+0000 7f8aa49e8700 1 -- 172.21.15.162:0/536598144 shutdown_connections 2020-11-28T21:54:03.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.100+0000 7f8aa49e8700 1 -- 172.21.15.162:0/536598144 wait complete. 2020-11-28T21:54:03.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.100+0000 7f8aa49e8700 1 Processor -- start 2020-11-28T21:54:03.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.100+0000 7f8aa49e8700 1 -- start start 2020-11-28T21:54:03.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.100+0000 7f8aa49e8700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8aa00b18c0 0x7f8aa013ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:03.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.104+0000 7f8aa49e8700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8aa00b3950 0x7f8aa0138d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:03.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.104+0000 7f8aa49e8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8aa00b44e0 0x7f8aa0139200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:03.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.104+0000 7f8aa49e8700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa0140b80 con 0x7f8aa00b18c0 2020-11-28T21:54:03.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.104+0000 7f8aa49e8700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa0140e20 con 0x7f8aa00b44e0 2020-11-28T21:54:03.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.104+0000 7f8aa49e8700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa0152d70 con 0x7f8aa00b3950 2020-11-28T21:54:03.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.104+0000 7f8a9f59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8aa00b44e0 0x7f8aa0139200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:03.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.104+0000 7f8a9f59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8aa00b44e0 0x7f8aa0139200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42774/0 (socket says 172.21.15.162:42774) 2020-11-28T21:54:03.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.104+0000 7f8a9f59e700 1 -- 172.21.15.162:0/4097189046 learned_addr learned my addr 172.21.15.162:0/4097189046 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:03.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.104+0000 7f8a9ed9d700 1 --2- 172.21.15.162:0/4097189046 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8aa00b18c0 0x7f8aa013ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:03.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.104+0000 7f8a9e59c700 1 --2- 172.21.15.162:0/4097189046 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8aa00b3950 0x7f8aa0138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:03.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.104+0000 7f8a9e59c700 1 -- 172.21.15.162:0/4097189046 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8aa00b44e0 msgr2=0x7f8aa0139200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:03.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.104+0000 7f8a9e59c700 1 --2- 172.21.15.162:0/4097189046 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8aa00b44e0 0x7f8aa0139200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:03.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.104+0000 7f8a9e59c700 1 -- 172.21.15.162:0/4097189046 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8aa00b18c0 msgr2=0x7f8aa013ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:03.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.104+0000 7f8a9e59c700 1 --2- 172.21.15.162:0/4097189046 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8aa00b18c0 0x7f8aa013ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:03.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.104+0000 7f8a9e59c700 1 -- 172.21.15.162:0/4097189046 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa01410c0 con 0x7f8aa00b3950 2020-11-28T21:54:03.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.104+0000 7f8a9ed9d700 1 --2- 172.21.15.162:0/4097189046 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8aa00b18c0 0x7f8aa013ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:54:03.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.104+0000 7f8a9e59c700 1 --2- 172.21.15.162:0/4097189046 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8aa00b3950 0x7f8aa0138d10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f8a94007000 tx=0x7f8a94008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:03.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.104+0000 7f8a8ffff700 1 -- 172.21.15.162:0/4097189046 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a9400e510 con 0x7f8aa00b3950 2020-11-28T21:54:03.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.104+0000 7f8aa49e8700 1 -- 172.21.15.162:0/4097189046 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aa013e760 con 0x7f8aa00b3950 2020-11-28T21:54:03.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.104+0000 7f8a8ffff700 1 -- 172.21.15.162:0/4097189046 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a9400c030 con 0x7f8aa00b3950 2020-11-28T21:54:03.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.104+0000 7f8a8ffff700 1 -- 172.21.15.162:0/4097189046 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a9400e860 con 0x7f8aa00b3950 2020-11-28T21:54:03.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.104+0000 7f8aa49e8700 1 -- 172.21.15.162:0/4097189046 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aa014f780 con 0x7f8aa00b3950 2020-11-28T21:54:03.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.104+0000 7f8aa49e8700 1 -- 172.21.15.162:0/4097189046 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aa0152f30 con 0x7f8aa00b3950 2020-11-28T21:54:03.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.108+0000 7f8a8ffff700 1 -- 172.21.15.162:0/4097189046 <== mon.2 v2:172.21.15.118:3301/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8a94019030 con 0x7f8aa00b3950 2020-11-28T21:54:03.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.108+0000 7f8a8ffff700 1 --2- 172.21.15.162:0/4097189046 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8a7c055330 0x7f8a7c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:03.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.108+0000 7f8a8ffff700 1 -- 172.21.15.162:0/4097189046 <== mon.2 v2:172.21.15.118:3301/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8a9406d780 con 0x7f8aa00b3950 2020-11-28T21:54:03.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.108+0000 7f8a9ed9d700 1 --2- 172.21.15.162:0/4097189046 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8a7c055330 0x7f8a7c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:03.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.108+0000 7f8a9ed9d700 1 --2- 172.21.15.162:0/4097189046 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8a7c055330 0x7f8a7c0577f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7f8a9001c000 tx=0x7f8a90015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:03.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.116+0000 7f8a8ffff700 1 -- 172.21.15.162:0/4097189046 <== mon.2 v2:172.21.15.118:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8a94043940 con 0x7f8aa00b3950 2020-11-28T21:54:03.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.300+0000 7f8aa49e8700 1 -- 172.21.15.162:0/4097189046 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8aa0057050 con 0x7f8aa00b3950 2020-11-28T21:54:03.399 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.396+0000 7f8a8ffff700 1 -- 172.21.15.162:0/4097189046 <== mon.2 v2:172.21.15.118:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v46) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8a94046f60 con 0x7f8aa00b3950 2020-11-28T21:54:03.402 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.396+0000 7f8aa49e8700 1 -- 172.21.15.162:0/4097189046 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8a7c055330 msgr2=0x7f8a7c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:03.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.396+0000 7f8aa49e8700 1 --2- 172.21.15.162:0/4097189046 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8a7c055330 0x7f8a7c0577f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7f8a9001c000 tx=0x7f8a90015000).stop 2020-11-28T21:54:03.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.396+0000 7f8aa49e8700 1 -- 172.21.15.162:0/4097189046 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8aa00b3950 msgr2=0x7f8aa0138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:03.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.396+0000 7f8aa49e8700 1 --2- 172.21.15.162:0/4097189046 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8aa00b3950 0x7f8aa0138d10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f8a94007000 tx=0x7f8a94008040).stop 2020-11-28T21:54:03.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.400+0000 7f8aa49e8700 1 -- 172.21.15.162:0/4097189046 shutdown_connections 2020-11-28T21:54:03.404 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.400+0000 7f8aa49e8700 1 --2- 172.21.15.162:0/4097189046 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8a7c055330 0x7f8a7c0577f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:03.404 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.400+0000 7f8aa49e8700 1 --2- 172.21.15.162:0/4097189046 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8aa00b44e0 0x7f8aa0139200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:03.404 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.400+0000 7f8aa49e8700 1 --2- 172.21.15.162:0/4097189046 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8aa00b18c0 0x7f8aa013ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:03.404 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.400+0000 7f8aa49e8700 1 --2- 172.21.15.162:0/4097189046 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8aa00b3950 0x7f8aa0138d10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:03.404 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.400+0000 7f8aa49e8700 1 -- 172.21.15.162:0/4097189046 >> 172.21.15.162:0/4097189046 conn(0x7f8aa0062630 msgr2=0x7f8aa0141fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:03.405 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.400+0000 7f8aa49e8700 1 -- 172.21.15.162:0/4097189046 shutdown_connections 2020-11-28T21:54:03.405 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.400+0000 7f8aa49e8700 1 -- 172.21.15.162:0/4097189046 wait complete. 2020-11-28T21:54:03.418 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:54:03.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.524+0000 7fe2c9fd5700 1 Processor -- start 2020-11-28T21:54:03.530 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.524+0000 7fe2c9fd5700 1 -- start start 2020-11-28T21:54:03.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.528+0000 7fe2c9fd5700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe2c40994f0 0x7fe2c40b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:03.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.528+0000 7fe2c9fd5700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2c4098f10 con 0x7fe2c4060710 2020-11-28T21:54:03.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.528+0000 7fe2c9fd5700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2c40b0680 con 0x7fe2c4098b30 2020-11-28T21:54:03.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.528+0000 7fe2c9fd5700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2c40b1030 con 0x7fe2c40994f0 2020-11-28T21:54:03.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.528+0000 7fe2c3fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe2c40994f0 0x7fe2c40b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:03.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.528+0000 7fe2c8fd3700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fe2c4098b30 0x7fe2c4060000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:49624/0 (socket says 172.21.15.162:49624) 2020-11-28T21:54:03.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.528+0000 7fe2c8fd3700 1 -- 172.21.15.162:0/3061766475 learned_addr learned my addr 172.21.15.162:0/3061766475 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:03.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.528+0000 7fe2c3fff700 1 -- 172.21.15.162:0/3061766475 >> v1:172.21.15.162:6789/0 conn(0x7fe2c4098b30 legacy=0x7fe2c4060000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:03.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.528+0000 7fe2c3fff700 1 -- 172.21.15.162:0/3061766475 >> v1:172.21.15.118:6789/0 conn(0x7fe2c4060710 legacy=0x7fe2c40b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:03.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.528+0000 7fe2c37fe700 1 -- 172.21.15.162:0/3061766475 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 529638758 0 0) 0x7fe2c40b0680 con 0x7fe2c4098b30 2020-11-28T21:54:03.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.528+0000 7fe2c3fff700 1 -- 172.21.15.162:0/3061766475 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c40b0920 con 0x7fe2c40994f0 2020-11-28T21:54:03.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.528+0000 7fe2c3fff700 1 --2- 172.21.15.162:0/3061766475 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2c40994f0 0x7fe2c40b0000 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 rx=0x7fe2b40032b0 tx=0x7fe2b40091a0).ready entity=mon.1 client_cookie=ef99e0e10091ebf2 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:03.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.528+0000 7fe2c37fe700 1 -- 172.21.15.162:0/3061766475 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2b400ba50 con 0x7fe2c40994f0 2020-11-28T21:54:03.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.528+0000 7fe2c37fe700 1 -- 172.21.15.162:0/3061766475 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b400bbb0 con 0x7fe2c40994f0 2020-11-28T21:54:03.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.528+0000 7fe2c9fd5700 1 -- 172.21.15.162:0/3061766475 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2c40994f0 msgr2=0x7fe2c40b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:03.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.528+0000 7fe2c37fe700 1 -- 172.21.15.162:0/3061766475 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2b400bd20 con 0x7fe2c40994f0 2020-11-28T21:54:03.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.528+0000 7fe2c9fd5700 1 --2- 172.21.15.162:0/3061766475 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2c40994f0 0x7fe2c40b0000 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 rx=0x7fe2b40032b0 tx=0x7fe2b40091a0).stop 2020-11-28T21:54:03.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.528+0000 7fe2c9fd5700 1 -- 172.21.15.162:0/3061766475 shutdown_connections 2020-11-28T21:54:03.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.528+0000 7fe2c9fd5700 1 --2- 172.21.15.162:0/3061766475 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2c40994f0 0x7fe2c40b0000 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:03.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.528+0000 7fe2c9fd5700 1 -- 172.21.15.162:0/3061766475 >> 172.21.15.162:0/3061766475 conn(0x7fe2c405b970 msgr2=0x7fe2c405bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:03.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.528+0000 7fe2c9fd5700 1 -- 172.21.15.162:0/3061766475 shutdown_connections 2020-11-28T21:54:03.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.528+0000 7fe2c9fd5700 1 -- 172.21.15.162:0/3061766475 wait complete. 2020-11-28T21:54:03.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.528+0000 7fe2c9fd5700 1 Processor -- start 2020-11-28T21:54:03.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.528+0000 7fe2c9fd5700 1 -- start start 2020-11-28T21:54:03.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.532+0000 7fe2c9fd5700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe2c4060710 0x7fe2c409b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:03.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.532+0000 7fe2c9fd5700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2c4098b30 0x7fe2c41488a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:03.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.532+0000 7fe2c9fd5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2c40994f0 0x7fe2c4148d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:03.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.532+0000 7fe2c9fd5700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c4149280 con 0x7fe2c4098b30 2020-11-28T21:54:03.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.532+0000 7fe2c9fd5700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c413f660 con 0x7fe2c40994f0 2020-11-28T21:54:03.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.532+0000 7fe2c9fd5700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c40aeaf0 con 0x7fe2c4060710 2020-11-28T21:54:03.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.532+0000 7fe2c97d4700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2c40994f0 0x7fe2c4148d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:03.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.532+0000 7fe2c97d4700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2c40994f0 0x7fe2c4148d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42786/0 (socket says 172.21.15.162:42786) 2020-11-28T21:54:03.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.532+0000 7fe2c97d4700 1 -- 172.21.15.162:0/1205833939 learned_addr learned my addr 172.21.15.162:0/1205833939 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:03.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.532+0000 7fe2c8fd3700 1 --2- 172.21.15.162:0/1205833939 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe2c4060710 0x7fe2c409b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:03.543 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.532+0000 7fe2c3fff700 1 --2- 172.21.15.162:0/1205833939 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2c4098b30 0x7fe2c41488a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:03.543 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.532+0000 7fe2c97d4700 1 -- 172.21.15.162:0/1205833939 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe2c4060710 msgr2=0x7fe2c409b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:03.543 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.532+0000 7fe2c97d4700 1 --2- 172.21.15.162:0/1205833939 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe2c4060710 0x7fe2c409b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:03.543 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.532+0000 7fe2c97d4700 1 -- 172.21.15.162:0/1205833939 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2c4098b30 msgr2=0x7fe2c41488a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:03.543 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.532+0000 7fe2c97d4700 1 --2- 172.21.15.162:0/1205833939 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2c4098b30 0x7fe2c41488a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:03.544 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.532+0000 7fe2c97d4700 1 -- 172.21.15.162:0/1205833939 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c41566d0 con 0x7fe2c40994f0 2020-11-28T21:54:03.544 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.532+0000 7fe2c97d4700 1 --2- 172.21.15.162:0/1205833939 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2c40994f0 0x7fe2c4148d90 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 rx=0x7fe2c4061f00 tx=0x7fe2bc007b90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:03.544 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.532+0000 7fe2c1ffb700 1 -- 172.21.15.162:0/1205833939 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2bc013410 con 0x7fe2c40994f0 2020-11-28T21:54:03.544 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.532+0000 7fe2c1ffb700 1 -- 172.21.15.162:0/1205833939 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2bc013570 con 0x7fe2c40994f0 2020-11-28T21:54:03.545 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.532+0000 7fe2c1ffb700 1 -- 172.21.15.162:0/1205833939 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2bc0138d0 con 0x7fe2c40994f0 2020-11-28T21:54:03.545 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.532+0000 7fe2c9fd5700 1 -- 172.21.15.162:0/1205833939 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2c4156970 con 0x7fe2c40994f0 2020-11-28T21:54:03.545 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.532+0000 7fe2c9fd5700 1 -- 172.21.15.162:0/1205833939 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2c4063b30 con 0x7fe2c40994f0 2020-11-28T21:54:03.545 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.532+0000 7fe2c9fd5700 1 -- 172.21.15.162:0/1205833939 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2c4057050 con 0x7fe2c40994f0 2020-11-28T21:54:03.546 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.536+0000 7fe2c1ffb700 1 -- 172.21.15.162:0/1205833939 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe2bc010030 con 0x7fe2c40994f0 2020-11-28T21:54:03.546 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.536+0000 7fe2c1ffb700 1 --2- 172.21.15.162:0/1205833939 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe2a8055440 0x7fe2a8057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:03.546 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.536+0000 7fe2c1ffb700 1 -- 172.21.15.162:0/1205833939 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe2bc001e40 con 0x7fe2c40994f0 2020-11-28T21:54:03.546 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.536+0000 7fe2c8fd3700 1 --2- 172.21.15.162:0/1205833939 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe2a8055440 0x7fe2a8057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:03.547 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.536+0000 7fe2c8fd3700 1 --2- 172.21.15.162:0/1205833939 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe2a8055440 0x7fe2a8057900 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7fe2ac00d000 tx=0x7fe2ac001160).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:03.547 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.540+0000 7fe2c1ffb700 1 -- 172.21.15.162:0/1205833939 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe2bc043f00 con 0x7fe2c40994f0 2020-11-28T21:54:03.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.720+0000 7fe2c9fd5700 1 -- 172.21.15.162:0/1205833939 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2c4068350 con 0x7fe2c40994f0 2020-11-28T21:54:03.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.724+0000 7fe2c1ffb700 1 -- 172.21.15.162:0/1205833939 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+2502 (secure 0 0 0) 0x7fe2bc047520 con 0x7fe2c40994f0 2020-11-28T21:54:03.727 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:54:03.727 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":46,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6657,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.118:6833/1785592854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1785592854},{"type":"v1","addr":"172.21.15.118:6833","nonce":1785592854}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":44},{"gid":6850,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3533607073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3533607073},{"type":"v1","addr":"172.21.15.162:6835","nonce":3533607073}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:54:02.388344+0000","modified":"2020-11-28T21:54:03.391088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6859":{"gid":6859,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/3532128068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3532128068},{"type":"v1","addr":"172.21.15.162:6837","nonce":3532128068}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2020-11-28T21:54:03.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.724+0000 7fe2c9fd5700 1 -- 172.21.15.162:0/1205833939 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe2a8055440 msgr2=0x7fe2a8057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:03.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.724+0000 7fe2c9fd5700 1 --2- 172.21.15.162:0/1205833939 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe2a8055440 0x7fe2a8057900 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7fe2ac00d000 tx=0x7fe2ac001160).stop 2020-11-28T21:54:03.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.724+0000 7fe2c9fd5700 1 -- 172.21.15.162:0/1205833939 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2c40994f0 msgr2=0x7fe2c4148d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:03.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.724+0000 7fe2c9fd5700 1 --2- 172.21.15.162:0/1205833939 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2c40994f0 0x7fe2c4148d90 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 rx=0x7fe2c4061f00 tx=0x7fe2bc007b90).stop 2020-11-28T21:54:03.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.724+0000 7fe2c9fd5700 1 -- 172.21.15.162:0/1205833939 shutdown_connections 2020-11-28T21:54:03.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.724+0000 7fe2c9fd5700 1 --2- 172.21.15.162:0/1205833939 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe2a8055440 0x7fe2a8057900 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:03.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.724+0000 7fe2c9fd5700 1 --2- 172.21.15.162:0/1205833939 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe2c4060710 0x7fe2c409b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:03.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.724+0000 7fe2c9fd5700 1 --2- 172.21.15.162:0/1205833939 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2c40994f0 0x7fe2c4148d90 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:03.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.724+0000 7fe2c9fd5700 1 --2- 172.21.15.162:0/1205833939 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2c4098b30 0x7fe2c41488a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:03.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.724+0000 7fe2c9fd5700 1 -- 172.21.15.162:0/1205833939 >> 172.21.15.162:0/1205833939 conn(0x7fe2c405b970 msgr2=0x7fe2c40ab5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:03.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.724+0000 7fe2c9fd5700 1 -- 172.21.15.162:0/1205833939 shutdown_connections 2020-11-28T21:54:03.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.724+0000 7fe2c9fd5700 1 -- 172.21.15.162:0/1205833939 wait complete. 2020-11-28T21:54:03.732 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 46 2020-11-28T21:54:03.744 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:54:03.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.848+0000 7f4089e11700 1 Processor -- start 2020-11-28T21:54:03.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.848+0000 7f4089e11700 1 -- start start 2020-11-28T21:54:03.852 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.848+0000 7f4089e11700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4084143440 0x7f4084060dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:03.852 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.848+0000 7f4089e11700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f40840b10b0 0x7f408413bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:03.853 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.848+0000 7f4089e11700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4084143920 con 0x7f40840b1990 2020-11-28T21:54:03.853 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.848+0000 7f4089e11700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40840af7e0 con 0x7f40840b10b0 2020-11-28T21:54:03.853 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.848+0000 7f4089e11700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40840b20e0 con 0x7f4084143440 2020-11-28T21:54:03.853 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.848+0000 7f4082ffd700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4084143440 0x7f4084060dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:03.853 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.848+0000 7f4083fff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f40840b10b0 0x7f408413bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:03.854 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.848+0000 7f40837fe700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f40840b1990 0x7f40840606c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.162:39786/0 (socket says 172.21.15.162:39786) 2020-11-28T21:54:03.854 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.848+0000 7f40837fe700 1 -- 172.21.15.162:0/1664035386 learned_addr learned my addr 172.21.15.162:0/1664035386 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:03.855 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.848+0000 7f40827fc700 1 -- 172.21.15.162:0/1664035386 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2681130316 0 0) 0x7f4084143920 con 0x7f40840b1990 2020-11-28T21:54:03.856 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.848+0000 7f40827fc700 1 -- 172.21.15.162:0/1664035386 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4070003230 con 0x7f40840b1990 2020-11-28T21:54:03.856 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.848+0000 7f4083fff700 1 -- 172.21.15.162:0/1664035386 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4084143440 msgr2=0x7f4084060dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:03.856 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.848+0000 7f4083fff700 1 --2- 172.21.15.162:0/1664035386 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4084143440 0x7f4084060dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:03.856 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.848+0000 7f4083fff700 1 -- 172.21.15.162:0/1664035386 >> v1:172.21.15.118:6789/0 conn(0x7f40840b1990 legacy=0x7f40840606c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:03.856 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.848+0000 7f4083fff700 1 -- 172.21.15.162:0/1664035386 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4084060290 con 0x7f40840b10b0 2020-11-28T21:54:03.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.848+0000 7f4082ffd700 1 --2- 172.21.15.162:0/1664035386 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4084143440 0x7f4084060dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:54:03.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.848+0000 7f4083fff700 1 --2- 172.21.15.162:0/1664035386 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f40840b10b0 0x7f408413bfd0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f407c0032b0 tx=0x7f407c0091a0).ready entity=mon.0 client_cookie=5b65740355b2b7dc server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:03.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.848+0000 7f40827fc700 1 -- 172.21.15.162:0/1664035386 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f407c00b940 con 0x7f40840b10b0 2020-11-28T21:54:03.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.848+0000 7f40827fc700 1 -- 172.21.15.162:0/1664035386 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f407c00baa0 con 0x7f40840b10b0 2020-11-28T21:54:03.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.848+0000 7f40827fc700 1 -- 172.21.15.162:0/1664035386 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f407c00bc10 con 0x7f40840b10b0 2020-11-28T21:54:03.858 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.848+0000 7f4089e11700 1 -- 172.21.15.162:0/1664035386 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f40840b10b0 msgr2=0x7f408413bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:03.858 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.848+0000 7f4089e11700 1 --2- 172.21.15.162:0/1664035386 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f40840b10b0 0x7f408413bfd0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f407c0032b0 tx=0x7f407c0091a0).stop 2020-11-28T21:54:03.858 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.852+0000 7f4089e11700 1 -- 172.21.15.162:0/1664035386 shutdown_connections 2020-11-28T21:54:03.858 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.852+0000 7f4089e11700 1 --2- 172.21.15.162:0/1664035386 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f40840b10b0 0x7f408413bfd0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:03.859 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.852+0000 7f4089e11700 1 --2- 172.21.15.162:0/1664035386 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4084143440 0x7f4084060dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:03.859 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.852+0000 7f4089e11700 1 -- 172.21.15.162:0/1664035386 >> 172.21.15.162:0/1664035386 conn(0x7f408405b970 msgr2=0x7f408405bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:03.859 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.852+0000 7f4089e11700 1 -- 172.21.15.162:0/1664035386 shutdown_connections 2020-11-28T21:54:03.859 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.852+0000 7f4089e11700 1 -- 172.21.15.162:0/1664035386 wait complete. 2020-11-28T21:54:03.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.852+0000 7f4089e11700 1 Processor -- start 2020-11-28T21:54:03.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.852+0000 7f4089e11700 1 -- start start 2020-11-28T21:54:03.862 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.852+0000 7f4089e11700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f40840b10b0 0x7f4084144140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:03.862 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.852+0000 7f4089e11700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40840b1990 0x7f4084144800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:03.862 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.852+0000 7f4089e11700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4084143440 0x7f4084144cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:03.862 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.852+0000 7f4089e11700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40841527d0 con 0x7f40840b10b0 2020-11-28T21:54:03.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.852+0000 7f4089e11700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40840aafe0 con 0x7f40840b1990 2020-11-28T21:54:03.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.852+0000 7f4089e11700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40840ab280 con 0x7f4084143440 2020-11-28T21:54:03.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.852+0000 7f4082ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40840b1990 0x7f4084144800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:03.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.852+0000 7f4082ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40840b1990 0x7f4084144800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42802/0 (socket says 172.21.15.162:42802) 2020-11-28T21:54:03.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.852+0000 7f4082ffd700 1 -- 172.21.15.162:0/3015049817 learned_addr learned my addr 172.21.15.162:0/3015049817 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:03.864 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.852+0000 7f40837fe700 1 --2- 172.21.15.162:0/3015049817 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f40840b10b0 0x7f4084144140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:03.864 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.852+0000 7f4083fff700 1 --2- 172.21.15.162:0/3015049817 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4084143440 0x7f4084144cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:03.864 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.852+0000 7f4082ffd700 1 -- 172.21.15.162:0/3015049817 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4084143440 msgr2=0x7f4084144cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:03.864 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.852+0000 7f4082ffd700 1 --2- 172.21.15.162:0/3015049817 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4084143440 0x7f4084144cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:03.864 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.852+0000 7f4082ffd700 1 -- 172.21.15.162:0/3015049817 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f40840b10b0 msgr2=0x7f4084144140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:03.865 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.856+0000 7f4082ffd700 1 --2- 172.21.15.162:0/3015049817 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f40840b10b0 0x7f4084144140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:03.865 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.856+0000 7f4082ffd700 1 -- 172.21.15.162:0/3015049817 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4084152470 con 0x7f40840b1990 2020-11-28T21:54:03.865 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.856+0000 7f4082ffd700 1 --2- 172.21.15.162:0/3015049817 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40840b1990 0x7f4084144800 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 rx=0x7f407400c000 tx=0x7f4074006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:03.865 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.856+0000 7f4080ff9700 1 -- 172.21.15.162:0/3015049817 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f407400cae0 con 0x7f40840b1990 2020-11-28T21:54:03.865 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.856+0000 7f4080ff9700 1 -- 172.21.15.162:0/3015049817 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f407400cc40 con 0x7f40840b1990 2020-11-28T21:54:03.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.856+0000 7f4089e11700 1 -- 172.21.15.162:0/3015049817 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4084146b80 con 0x7f40840b1990 2020-11-28T21:54:03.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.856+0000 7f4080ff9700 1 -- 172.21.15.162:0/3015049817 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40740123f0 con 0x7f40840b1990 2020-11-28T21:54:03.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.856+0000 7f4089e11700 1 -- 172.21.15.162:0/3015049817 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4084145400 con 0x7f40840b1990 2020-11-28T21:54:03.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.856+0000 7f4089e11700 1 -- 172.21.15.162:0/3015049817 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40840ab440 con 0x7f40840b1990 2020-11-28T21:54:03.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.856+0000 7f4080ff9700 1 -- 172.21.15.162:0/3015049817 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4074012550 con 0x7f40840b1990 2020-11-28T21:54:03.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.856+0000 7f4080ff9700 1 --2- 172.21.15.162:0/3015049817 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4064055330 0x7f40640577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:03.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.856+0000 7f40837fe700 1 --2- 172.21.15.162:0/3015049817 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4064055330 0x7f40640577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:03.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.856+0000 7f4080ff9700 1 -- 172.21.15.162:0/3015049817 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f407406e640 con 0x7f40840b1990 2020-11-28T21:54:03.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.860+0000 7f40837fe700 1 --2- 172.21.15.162:0/3015049817 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4064055330 0x7f40640577f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f407800d000 tx=0x7f4078009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:03.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:03.864+0000 7f4080ff9700 1 -- 172.21.15.162:0/3015049817 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4074044750 con 0x7f40840b1990 2020-11-28T21:54:04.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.008+0000 7f4089e11700 1 -- 172.21.15.162:0/3015049817 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4084057050 con 0x7f40840b1990 2020-11-28T21:54:04.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.008+0000 7f4080ff9700 1 -- 172.21.15.162:0/3015049817 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+16137 (secure 0 0 0) 0x7f4074047d70 con 0x7f40840b1990 2020-11-28T21:54:04.013 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:54:04.013 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":99,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:54:03.391732+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:53:59.925390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2020-11-28T21:54:01.236032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:54:04.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.012+0000 7f4089e11700 1 -- 172.21.15.162:0/3015049817 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4064055330 msgr2=0x7f40640577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:04.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.012+0000 7f4089e11700 1 --2- 172.21.15.162:0/3015049817 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4064055330 0x7f40640577f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f407800d000 tx=0x7f4078009040).stop 2020-11-28T21:54:04.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.012+0000 7f4089e11700 1 -- 172.21.15.162:0/3015049817 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40840b1990 msgr2=0x7f4084144800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:04.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.012+0000 7f4089e11700 1 --2- 172.21.15.162:0/3015049817 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40840b1990 0x7f4084144800 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 rx=0x7f407400c000 tx=0x7f4074006dc0).stop 2020-11-28T21:54:04.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.012+0000 7f4089e11700 1 -- 172.21.15.162:0/3015049817 shutdown_connections 2020-11-28T21:54:04.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.012+0000 7f4089e11700 1 --2- 172.21.15.162:0/3015049817 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4064055330 0x7f40640577f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:04.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.012+0000 7f4089e11700 1 --2- 172.21.15.162:0/3015049817 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4084143440 0x7f4084144cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:04.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.012+0000 7f4089e11700 1 --2- 172.21.15.162:0/3015049817 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f40840b10b0 0x7f4084144140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:04.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.012+0000 7f4089e11700 1 --2- 172.21.15.162:0/3015049817 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40840b1990 0x7f4084144800 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:04.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.012+0000 7f4089e11700 1 -- 172.21.15.162:0/3015049817 >> 172.21.15.162:0/3015049817 conn(0x7f408405b970 msgr2=0x7f40840af9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:04.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.012+0000 7f4089e11700 1 -- 172.21.15.162:0/3015049817 shutdown_connections 2020-11-28T21:54:04.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.012+0000 7f4089e11700 1 -- 172.21.15.162:0/3015049817 wait complete. 2020-11-28T21:54:04.027 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-11-28T21:54:04.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.144+0000 7f073b59e700 1 Processor -- start 2020-11-28T21:54:04.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.144+0000 7f073b59e700 1 -- start start 2020-11-28T21:54:04.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.148+0000 7f073b59e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f073c0b2e80 0x7f073c05f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:04.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.148+0000 7f073b59e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f073c0b2560 0x7f073c05f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:04.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.148+0000 7f073b59e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f073c0ab0b0 con 0x7f073c0acb40 2020-11-28T21:54:04.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.148+0000 7f073b59e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f073c05fd60 con 0x7f073c0b2560 2020-11-28T21:54:04.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.148+0000 7f073b59e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f073c0b3260 con 0x7f073c0b2e80 2020-11-28T21:54:04.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.148+0000 7f073ad9d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f073c0b2e80 0x7f073c05f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:04.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.148+0000 7f0740a5e700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f073c0acb40 0x7f073c060d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:49656/0 (socket says 172.21.15.162:49656) 2020-11-28T21:54:04.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.148+0000 7f0740a5e700 1 -- 172.21.15.162:0/2029906622 learned_addr learned my addr 172.21.15.162:0/2029906622 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:04.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.148+0000 7f073a59c700 1 --2- 172.21.15.162:0/2029906622 >> v2:172.21.15.118:3300/0 conn(0x7f073c0b2560 0x7f073c05f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:04.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.148+0000 7f073ad9d700 1 -- 172.21.15.162:0/2029906622 >> v2:172.21.15.118:3300/0 conn(0x7f073c0b2560 msgr2=0x7f073c05f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:04.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.148+0000 7f073ad9d700 1 --2- 172.21.15.162:0/2029906622 >> v2:172.21.15.118:3300/0 conn(0x7f073c0b2560 0x7f073c05f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:04.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.148+0000 7f073ad9d700 1 -- 172.21.15.162:0/2029906622 >> v1:172.21.15.162:6789/0 conn(0x7f073c0acb40 legacy=0x7f073c060d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:04.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.148+0000 7f073ad9d700 1 -- 172.21.15.162:0/2029906622 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f073c0ab2e0 con 0x7f073c0b2e80 2020-11-28T21:54:04.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.148+0000 7f073a59c700 1 --2- 172.21.15.162:0/2029906622 >> v2:172.21.15.118:3300/0 conn(0x7f073c0b2560 0x7f073c05f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:54:04.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.148+0000 7f0739d9b700 1 -- 172.21.15.162:0/2029906622 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2060377814 0 0) 0x7f073c0ab0b0 con 0x7f073c0acb40 2020-11-28T21:54:04.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.148+0000 7f073ad9d700 1 --2- 172.21.15.162:0/2029906622 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f073c0b2e80 0x7f073c05f380 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 rx=0x7f072c002f30 tx=0x7f072c006dc0).ready entity=mon.1 client_cookie=fec97b61c064a7d0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:04.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.148+0000 7f0739d9b700 1 -- 172.21.15.162:0/2029906622 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f072c0097f0 con 0x7f073c0b2e80 2020-11-28T21:54:04.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.148+0000 7f0739d9b700 1 -- 172.21.15.162:0/2029906622 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f072c009950 con 0x7f073c0b2e80 2020-11-28T21:54:04.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.148+0000 7f0739d9b700 1 -- 172.21.15.162:0/2029906622 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f072c009cb0 con 0x7f073c0b2e80 2020-11-28T21:54:04.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.148+0000 7f073b59e700 1 -- 172.21.15.162:0/2029906622 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f073c0b2e80 msgr2=0x7f073c05f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:04.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.148+0000 7f073b59e700 1 --2- 172.21.15.162:0/2029906622 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f073c0b2e80 0x7f073c05f380 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 rx=0x7f072c002f30 tx=0x7f072c006dc0).stop 2020-11-28T21:54:04.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.148+0000 7f073b59e700 1 -- 172.21.15.162:0/2029906622 shutdown_connections 2020-11-28T21:54:04.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.148+0000 7f073b59e700 1 --2- 172.21.15.162:0/2029906622 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f073c0b2e80 0x7f073c05f380 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:04.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.148+0000 7f073b59e700 1 --2- 172.21.15.162:0/2029906622 >> v2:172.21.15.118:3300/0 conn(0x7f073c0b2560 0x7f073c05f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:04.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.148+0000 7f073b59e700 1 -- 172.21.15.162:0/2029906622 >> 172.21.15.162:0/2029906622 conn(0x7f073c063b60 msgr2=0x7f073c063f40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:04.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.148+0000 7f073b59e700 1 -- 172.21.15.162:0/2029906622 shutdown_connections 2020-11-28T21:54:04.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.148+0000 7f073b59e700 1 -- 172.21.15.162:0/2029906622 wait complete. 2020-11-28T21:54:04.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.152+0000 7f073b59e700 1 Processor -- start 2020-11-28T21:54:04.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.152+0000 7f073b59e700 1 -- start start 2020-11-28T21:54:04.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.152+0000 7f073b59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f073c0acb40 0x7f073c0b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:04.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.152+0000 7f073b59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f073c0b2560 0x7f073c064a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:04.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.152+0000 7f073b59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f073c0b2e80 0x7f073c13d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:04.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.152+0000 7f073b59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073c140810 con 0x7f073c0acb40 2020-11-28T21:54:04.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.152+0000 7f073b59e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073c140ab0 con 0x7f073c0b2e80 2020-11-28T21:54:04.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.152+0000 7f073b59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f073c140d50 con 0x7f073c0b2560 2020-11-28T21:54:04.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.152+0000 7f0740a5e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f073c0b2e80 0x7f073c13d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:04.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.152+0000 7f0740a5e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f073c0b2e80 0x7f073c13d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42818/0 (socket says 172.21.15.162:42818) 2020-11-28T21:54:04.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.152+0000 7f0740a5e700 1 -- 172.21.15.162:0/1889648331 learned_addr learned my addr 172.21.15.162:0/1889648331 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:04.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.152+0000 7f073ad9d700 1 --2- 172.21.15.162:0/1889648331 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f073c0acb40 0x7f073c0b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:04.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.152+0000 7f073a59c700 1 --2- 172.21.15.162:0/1889648331 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f073c0b2560 0x7f073c064a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:04.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.152+0000 7f0740a5e700 1 -- 172.21.15.162:0/1889648331 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f073c0b2560 msgr2=0x7f073c064a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:04.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.152+0000 7f0740a5e700 1 --2- 172.21.15.162:0/1889648331 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f073c0b2560 0x7f073c064a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:04.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.152+0000 7f0740a5e700 1 -- 172.21.15.162:0/1889648331 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f073c0acb40 msgr2=0x7f073c0b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:04.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.152+0000 7f0740a5e700 1 --2- 172.21.15.162:0/1889648331 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f073c0acb40 0x7f073c0b4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:04.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.152+0000 7f0740a5e700 1 -- 172.21.15.162:0/1889648331 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f073c140ff0 con 0x7f073c0b2e80 2020-11-28T21:54:04.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.152+0000 7f0740a5e700 1 --2- 172.21.15.162:0/1889648331 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f073c0b2e80 0x7f073c13d1a0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 rx=0x7f0734001210 tx=0x7f0734007d50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:04.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.152+0000 7f072b7fe700 1 -- 172.21.15.162:0/1889648331 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f073400ce60 con 0x7f073c0b2e80 2020-11-28T21:54:04.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.152+0000 7f072b7fe700 1 -- 172.21.15.162:0/1889648331 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0734005b70 con 0x7f073c0b2e80 2020-11-28T21:54:04.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.152+0000 7f073b59e700 1 -- 172.21.15.162:0/1889648331 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f073c0b5270 con 0x7f073c0b2e80 2020-11-28T21:54:04.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.152+0000 7f072b7fe700 1 -- 172.21.15.162:0/1889648331 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0734004870 con 0x7f073c0b2e80 2020-11-28T21:54:04.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.152+0000 7f073b59e700 1 -- 172.21.15.162:0/1889648331 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f073c141490 con 0x7f073c0b2e80 2020-11-28T21:54:04.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.156+0000 7f072b7fe700 1 -- 172.21.15.162:0/1889648331 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0734010070 con 0x7f073c0b2e80 2020-11-28T21:54:04.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.156+0000 7f073b59e700 1 -- 172.21.15.162:0/1889648331 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f073c141180 con 0x7f073c0b2e80 2020-11-28T21:54:04.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.156+0000 7f072b7fe700 1 --2- 172.21.15.162:0/1889648331 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0720055470 0x7f0720057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:04.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.156+0000 7f072b7fe700 1 -- 172.21.15.162:0/1889648331 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f073406da00 con 0x7f073c0b2e80 2020-11-28T21:54:04.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.156+0000 7f073ad9d700 1 --2- 172.21.15.162:0/1889648331 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0720055470 0x7f0720057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:04.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.156+0000 7f073ad9d700 1 --2- 172.21.15.162:0/1889648331 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0720055470 0x7f0720057930 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f072c01b000 tx=0x7f072c015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:04.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.160+0000 7f072b7fe700 1 -- 172.21.15.162:0/1889648331 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0734043bf0 con 0x7f073c0b2e80 2020-11-28T21:54:04.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.348+0000 7f073b59e700 1 -- 172.21.15.162:0/1889648331 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f073c057050 con 0x7f073c0b2e80 2020-11-28T21:54:04.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.356+0000 7f072b7fe700 1 -- 172.21.15.162:0/1889648331 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v23) v1 ==== 184+0+0 (secure 0 0 0) 0x7f0734047210 con 0x7f073c0b2e80 2020-11-28T21:54:04.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.360+0000 7f073b59e700 1 -- 172.21.15.162:0/1889648331 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0720055470 msgr2=0x7f0720057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:04.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.360+0000 7f073b59e700 1 --2- 172.21.15.162:0/1889648331 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0720055470 0x7f0720057930 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f072c01b000 tx=0x7f072c015000).stop 2020-11-28T21:54:04.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.360+0000 7f073b59e700 1 -- 172.21.15.162:0/1889648331 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f073c0b2e80 msgr2=0x7f073c13d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:04.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.360+0000 7f073b59e700 1 --2- 172.21.15.162:0/1889648331 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f073c0b2e80 0x7f073c13d1a0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 rx=0x7f0734001210 tx=0x7f0734007d50).stop 2020-11-28T21:54:04.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.360+0000 7f073b59e700 1 -- 172.21.15.162:0/1889648331 shutdown_connections 2020-11-28T21:54:04.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.360+0000 7f073b59e700 1 --2- 172.21.15.162:0/1889648331 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0720055470 0x7f0720057930 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:04.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.360+0000 7f073b59e700 1 --2- 172.21.15.162:0/1889648331 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f073c0b2e80 0x7f073c13d1a0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:04.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.360+0000 7f073b59e700 1 --2- 172.21.15.162:0/1889648331 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f073c0acb40 0x7f073c0b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:04.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.360+0000 7f073b59e700 1 --2- 172.21.15.162:0/1889648331 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f073c0b2560 0x7f073c064a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:04.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.360+0000 7f073b59e700 1 -- 172.21.15.162:0/1889648331 >> 172.21.15.162:0/1889648331 conn(0x7f073c063b60 msgr2=0x7f073c133570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:04.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.360+0000 7f073b59e700 1 -- 172.21.15.162:0/1889648331 shutdown_connections 2020-11-28T21:54:04.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.360+0000 7f073b59e700 1 -- 172.21.15.162:0/1889648331 wait complete. 2020-11-28T21:54:04.367 INFO:teuthology.orchestra.run.smithi162.stderr:updated caps for client.0 2020-11-28T21:54:04.377 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-11-28T21:54:04.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.496+0000 7f4c89db8700 1 Processor -- start 2020-11-28T21:54:04.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.496+0000 7f4c89db8700 1 -- start start 2020-11-28T21:54:04.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.496+0000 7f4c89db8700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4c8405fa90 0x7f4c8415a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:04.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.496+0000 7f4c89db8700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c840ab0b0 con 0x7f4c840b2e80 2020-11-28T21:54:04.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.496+0000 7f4c89db8700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c840b3260 con 0x7f4c840b2560 2020-11-28T21:54:04.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.496+0000 7f4c89db8700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c840ab2e0 con 0x7f4c8405fa90 2020-11-28T21:54:04.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.496+0000 7f4c88db6700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f4c840b2560 0x7f4c84156ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:49668/0 (socket says 172.21.15.162:49668) 2020-11-28T21:54:04.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.496+0000 7f4c88db6700 1 -- 172.21.15.162:0/1774160741 learned_addr learned my addr 172.21.15.162:0/1774160741 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:04.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.496+0000 7f4c8b01a700 1 --2- 172.21.15.162:0/1774160741 >> v2:172.21.15.162:3300/0 conn(0x7f4c8405fa90 0x7f4c8415a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:04.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.496+0000 7f4c7bfff700 1 -- 172.21.15.162:0/1774160741 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3929622150 0 0) 0x7f4c840b3260 con 0x7f4c840b2560 2020-11-28T21:54:04.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.496+0000 7f4c7bfff700 1 -- 172.21.15.162:0/1774160741 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c70003230 con 0x7f4c840b2560 2020-11-28T21:54:04.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.496+0000 7f4c8b01a700 1 -- 172.21.15.162:0/1774160741 >> v1:172.21.15.162:6789/0 conn(0x7f4c840b2560 legacy=0x7f4c84156ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:04.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.496+0000 7f4c8b01a700 1 -- 172.21.15.162:0/1774160741 >> v1:172.21.15.118:6789/0 conn(0x7f4c840b2e80 legacy=0x7f4c8405f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:04.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.496+0000 7f4c8b01a700 1 -- 172.21.15.162:0/1774160741 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c840256d0 con 0x7f4c8405fa90 2020-11-28T21:54:04.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.496+0000 7f4c8b01a700 1 --2- 172.21.15.162:0/1774160741 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c8405fa90 0x7f4c8415a5f0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 rx=0x7f4c80002f30 tx=0x7f4c80006dc0).ready entity=mon.1 client_cookie=63d88176a6d6be6a server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:04.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.496+0000 7f4c7bfff700 1 -- 172.21.15.162:0/1774160741 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c80009800 con 0x7f4c8405fa90 2020-11-28T21:54:04.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.496+0000 7f4c7bfff700 1 -- 172.21.15.162:0/1774160741 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c80009960 con 0x7f4c8405fa90 2020-11-28T21:54:04.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.496+0000 7f4c7bfff700 1 -- 172.21.15.162:0/1774160741 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c80009cc0 con 0x7f4c8405fa90 2020-11-28T21:54:04.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.496+0000 7f4c89db8700 1 -- 172.21.15.162:0/1774160741 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c8405fa90 msgr2=0x7f4c8415a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:04.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.496+0000 7f4c89db8700 1 --2- 172.21.15.162:0/1774160741 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c8405fa90 0x7f4c8415a5f0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 rx=0x7f4c80002f30 tx=0x7f4c80006dc0).stop 2020-11-28T21:54:04.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.496+0000 7f4c89db8700 1 -- 172.21.15.162:0/1774160741 shutdown_connections 2020-11-28T21:54:04.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.496+0000 7f4c89db8700 1 --2- 172.21.15.162:0/1774160741 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c8405fa90 0x7f4c8415a5f0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:04.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.500+0000 7f4c89db8700 1 -- 172.21.15.162:0/1774160741 >> 172.21.15.162:0/1774160741 conn(0x7f4c84063b60 msgr2=0x7f4c84063f40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:04.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.500+0000 7f4c89db8700 1 -- 172.21.15.162:0/1774160741 shutdown_connections 2020-11-28T21:54:04.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.500+0000 7f4c89db8700 1 -- 172.21.15.162:0/1774160741 wait complete. 2020-11-28T21:54:04.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.500+0000 7f4c89db8700 1 Processor -- start 2020-11-28T21:54:04.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.500+0000 7f4c89db8700 1 -- start start 2020-11-28T21:54:04.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.500+0000 7f4c89db8700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4c8405fa90 0x7f4c841408a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:04.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.500+0000 7f4c89db8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c840b2560 0x7f4c84140d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:04.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.500+0000 7f4c89db8700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c840b2e80 0x7f4c84143e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:04.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.500+0000 7f4c89db8700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c84132310 con 0x7f4c840b2e80 2020-11-28T21:54:04.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.500+0000 7f4c89db8700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c840997f0 con 0x7f4c840b2560 2020-11-28T21:54:04.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.500+0000 7f4c89db8700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c841417f0 con 0x7f4c8405fa90 2020-11-28T21:54:04.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.500+0000 7f4c88db6700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c840b2560 0x7f4c84140d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:04.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.500+0000 7f4c88db6700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c840b2560 0x7f4c84140d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42830/0 (socket says 172.21.15.162:42830) 2020-11-28T21:54:04.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.500+0000 7f4c88db6700 1 -- 172.21.15.162:0/3807554396 learned_addr learned my addr 172.21.15.162:0/3807554396 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:04.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.500+0000 7f4c895b7700 1 --2- 172.21.15.162:0/3807554396 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4c8405fa90 0x7f4c841408a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:04.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.500+0000 7f4c8b01a700 1 --2- 172.21.15.162:0/3807554396 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c840b2e80 0x7f4c84143e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:04.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.500+0000 7f4c88db6700 1 -- 172.21.15.162:0/3807554396 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4c8405fa90 msgr2=0x7f4c841408a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:04.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.500+0000 7f4c88db6700 1 --2- 172.21.15.162:0/3807554396 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4c8405fa90 0x7f4c841408a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:04.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.500+0000 7f4c88db6700 1 -- 172.21.15.162:0/3807554396 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c840b2e80 msgr2=0x7f4c84143e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:04.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.500+0000 7f4c88db6700 1 --2- 172.21.15.162:0/3807554396 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c840b2e80 0x7f4c84143e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:04.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.500+0000 7f4c88db6700 1 -- 172.21.15.162:0/3807554396 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c84141550 con 0x7f4c840b2560 2020-11-28T21:54:04.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.500+0000 7f4c88db6700 1 --2- 172.21.15.162:0/3807554396 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c840b2560 0x7f4c84140d90 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 rx=0x7f4c840ae1a0 tx=0x7f4c74007a70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:04.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.500+0000 7f4c79ffb700 1 -- 172.21.15.162:0/3807554396 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c740072e0 con 0x7f4c840b2560 2020-11-28T21:54:04.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.500+0000 7f4c79ffb700 1 -- 172.21.15.162:0/3807554396 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c74007440 con 0x7f4c840b2560 2020-11-28T21:54:04.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.500+0000 7f4c89db8700 1 -- 172.21.15.162:0/3807554396 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c840ae7e0 con 0x7f4c840b2560 2020-11-28T21:54:04.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.500+0000 7f4c79ffb700 1 -- 172.21.15.162:0/3807554396 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c740057d0 con 0x7f4c840b2560 2020-11-28T21:54:04.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.500+0000 7f4c89db8700 1 -- 172.21.15.162:0/3807554396 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c84134130 con 0x7f4c840b2560 2020-11-28T21:54:04.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.504+0000 7f4c79ffb700 1 -- 172.21.15.162:0/3807554396 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4c74010070 con 0x7f4c840b2560 2020-11-28T21:54:04.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.504+0000 7f4c89db8700 1 -- 172.21.15.162:0/3807554396 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c840ae970 con 0x7f4c840b2560 2020-11-28T21:54:04.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.504+0000 7f4c79ffb700 1 --2- 172.21.15.162:0/3807554396 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4c64055320 0x7f4c640577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:04.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.504+0000 7f4c79ffb700 1 -- 172.21.15.162:0/3807554396 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4c7406d460 con 0x7f4c840b2560 2020-11-28T21:54:04.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.504+0000 7f4c895b7700 1 --2- 172.21.15.162:0/3807554396 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4c64055320 0x7f4c640577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:04.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.504+0000 7f4c895b7700 1 --2- 172.21.15.162:0/3807554396 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4c64055320 0x7f4c640577e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f4c7c00d000 tx=0x7f4c7c009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:04.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.512+0000 7f4c79ffb700 1 -- 172.21.15.162:0/3807554396 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4c74043760 con 0x7f4c840b2560 2020-11-28T21:54:04.700 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.696+0000 7f4c89db8700 1 -- 172.21.15.162:0/3807554396 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f4c84057050 con 0x7f4c840b2560 2020-11-28T21:54:04.709 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.704+0000 7f4c79ffb700 1 -- 172.21.15.162:0/3807554396 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v24) v1 ==== 184+0+0 (secure 0 0 0) 0x7f4c74046d80 con 0x7f4c840b2560 2020-11-28T21:54:04.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.708+0000 7f4c89db8700 1 -- 172.21.15.162:0/3807554396 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4c64055320 msgr2=0x7f4c640577e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:04.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.708+0000 7f4c89db8700 1 --2- 172.21.15.162:0/3807554396 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4c64055320 0x7f4c640577e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f4c7c00d000 tx=0x7f4c7c009040).stop 2020-11-28T21:54:04.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.708+0000 7f4c89db8700 1 -- 172.21.15.162:0/3807554396 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c840b2560 msgr2=0x7f4c84140d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:04.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.708+0000 7f4c89db8700 1 --2- 172.21.15.162:0/3807554396 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c840b2560 0x7f4c84140d90 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 rx=0x7f4c840ae1a0 tx=0x7f4c74007a70).stop 2020-11-28T21:54:04.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.708+0000 7f4c89db8700 1 -- 172.21.15.162:0/3807554396 shutdown_connections 2020-11-28T21:54:04.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.708+0000 7f4c89db8700 1 --2- 172.21.15.162:0/3807554396 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4c64055320 0x7f4c640577e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:04.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.708+0000 7f4c89db8700 1 --2- 172.21.15.162:0/3807554396 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4c8405fa90 0x7f4c841408a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:04.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.708+0000 7f4c89db8700 1 --2- 172.21.15.162:0/3807554396 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c840b2e80 0x7f4c84143e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:04.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.708+0000 7f4c89db8700 1 --2- 172.21.15.162:0/3807554396 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c840b2560 0x7f4c84140d90 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:04.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.708+0000 7f4c89db8700 1 -- 172.21.15.162:0/3807554396 >> 172.21.15.162:0/3807554396 conn(0x7f4c84063b60 msgr2=0x7f4c8405f380 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:04.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.708+0000 7f4c89db8700 1 -- 172.21.15.162:0/3807554396 shutdown_connections 2020-11-28T21:54:04.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.708+0000 7f4c89db8700 1 -- 172.21.15.162:0/3807554396 wait complete. 2020-11-28T21:54:04.715 INFO:teuthology.orchestra.run.smithi162.stderr:updated caps for client.1 2020-11-28T21:54:04.729 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:54:04.839 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.836+0000 7f8e5142e700 1 Processor -- start 2020-11-28T21:54:04.839 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.836+0000 7f8e5142e700 1 -- start start 2020-11-28T21:54:04.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.836+0000 7f8e5142e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8e4c0b11b0 0x7f8e4c140fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:04.843 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.836+0000 7f8e5142e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e4c0b2870 con 0x7f8e4c0b1990 2020-11-28T21:54:04.843 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.836+0000 7f8e5142e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e4c0b22e0 con 0x7f8e4c0b1d70 2020-11-28T21:54:04.843 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.836+0000 7f8e5142e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c146e20 con 0x7f8e4c0b11b0 2020-11-28T21:54:04.843 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.836+0000 7f8e4b7fe700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f8e4c0b1d70 0x7f8e4c14a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:49680/0 (socket says 172.21.15.162:49680) 2020-11-28T21:54:04.844 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.836+0000 7f8e4b7fe700 1 -- 172.21.15.162:0/3540731013 learned_addr learned my addr 172.21.15.162:0/3540731013 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:04.844 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.836+0000 7f8e49ffb700 1 -- 172.21.15.162:0/3540731013 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2376824754 0 0) 0x7f8e4c0b22e0 con 0x7f8e4c0b1d70 2020-11-28T21:54:04.844 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.836+0000 7f8e4a7fc700 1 --2- 172.21.15.162:0/3540731013 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8e4c0b11b0 0x7f8e4c140fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:04.844 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.836+0000 7f8e49ffb700 1 -- 172.21.15.162:0/3540731013 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e30003230 con 0x7f8e4c0b1d70 2020-11-28T21:54:04.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.836+0000 7f8e49ffb700 1 -- 172.21.15.162:0/3540731013 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 552026469 0 0) 0x7f8e30003230 con 0x7f8e4c0b1d70 2020-11-28T21:54:04.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.836+0000 7f8e49ffb700 1 -- 172.21.15.162:0/3540731013 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8e4c0b11b0 msgr2=0x7f8e4c140fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:04.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.836+0000 7f8e49ffb700 1 --2- 172.21.15.162:0/3540731013 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8e4c0b11b0 0x7f8e4c140fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:04.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.836+0000 7f8e49ffb700 1 -- 172.21.15.162:0/3540731013 >> v1:172.21.15.118:6789/0 conn(0x7f8e4c0b1990 legacy=0x7f8e4c140890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:04.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.836+0000 7f8e49ffb700 1 -- 172.21.15.162:0/3540731013 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4c0b0740 con 0x7f8e4c0b1d70 2020-11-28T21:54:04.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.836+0000 7f8e49ffb700 1 -- 172.21.15.162:0/3540731013 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f8e44001a50 con 0x7f8e4c0b1d70 2020-11-28T21:54:04.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.836+0000 7f8e49ffb700 1 -- 172.21.15.162:0/3540731013 >> v1:172.21.15.162:6789/0 conn(0x7f8e4c0b1d70 legacy=0x7f8e4c14a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:04.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.836+0000 7f8e49ffb700 1 --2- 172.21.15.162:0/3540731013 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e30003720 0x7f8e30003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:04.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.836+0000 7f8e49ffb700 1 --2- 172.21.15.162:0/3540731013 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8e300044d0 0x7f8e30004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:04.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.836+0000 7f8e49ffb700 1 --2- 172.21.15.162:0/3540731013 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8e4c0b11b0 0x7f8e30004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:04.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.836+0000 7f8e49ffb700 1 -- 172.21.15.162:0/3540731013 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e30005350 con 0x7f8e300044d0 2020-11-28T21:54:04.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.836+0000 7f8e49ffb700 1 -- 172.21.15.162:0/3540731013 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e30004290 con 0x7f8e30003720 2020-11-28T21:54:04.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.836+0000 7f8e49ffb700 1 -- 172.21.15.162:0/3540731013 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3000b720 con 0x7f8e4c0b11b0 2020-11-28T21:54:04.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.836+0000 7f8e4affd700 1 --2- 172.21.15.162:0/3540731013 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e30003720 0x7f8e30003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:04.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.836+0000 7f8e4b7fe700 1 --2- 172.21.15.162:0/3540731013 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8e4c0b11b0 0x7f8e30004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:04.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.836+0000 7f8e4a7fc700 1 --2- 172.21.15.162:0/3540731013 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8e300044d0 0x7f8e30004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:04.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.836+0000 7f8e4affd700 1 -- 172.21.15.162:0/3540731013 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8e4c0b11b0 msgr2=0x7f8e30004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:04.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.836+0000 7f8e4affd700 1 --2- 172.21.15.162:0/3540731013 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8e4c0b11b0 0x7f8e30004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:04.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.836+0000 7f8e4affd700 1 -- 172.21.15.162:0/3540731013 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8e300044d0 msgr2=0x7f8e30004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:04.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.836+0000 7f8e4affd700 1 --2- 172.21.15.162:0/3540731013 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8e300044d0 0x7f8e30004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:04.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.836+0000 7f8e4affd700 1 -- 172.21.15.162:0/3540731013 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e30003230 con 0x7f8e30003720 2020-11-28T21:54:04.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.836+0000 7f8e4affd700 1 --2- 172.21.15.162:0/3540731013 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e30003720 0x7f8e30003b40 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 rx=0x7f8e4c0b2ce0 tx=0x7f8e3c007b60).ready entity=mon.1 client_cookie=31f3df82bbdf5b68 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:04.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.836+0000 7f8e49ffb700 1 -- 172.21.15.162:0/3540731013 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e3c00c870 con 0x7f8e30003720 2020-11-28T21:54:04.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.836+0000 7f8e49ffb700 1 -- 172.21.15.162:0/3540731013 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e3c00c9d0 con 0x7f8e30003720 2020-11-28T21:54:04.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.836+0000 7f8e49ffb700 1 -- 172.21.15.162:0/3540731013 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e3c00cd30 con 0x7f8e30003720 2020-11-28T21:54:04.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.836+0000 7f8e5142e700 1 -- 172.21.15.162:0/3540731013 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e30003720 msgr2=0x7f8e30003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:04.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.836+0000 7f8e5142e700 1 --2- 172.21.15.162:0/3540731013 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e30003720 0x7f8e30003b40 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 rx=0x7f8e4c0b2ce0 tx=0x7f8e3c007b60).stop 2020-11-28T21:54:04.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.836+0000 7f8e4b7fe700 1 -- 172.21.15.162:0/3540731013 reap_dead start 2020-11-28T21:54:04.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.840+0000 7f8e5142e700 1 -- 172.21.15.162:0/3540731013 shutdown_connections 2020-11-28T21:54:04.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.840+0000 7f8e5142e700 1 -- 172.21.15.162:0/3540731013 >> 172.21.15.162:0/3540731013 conn(0x7f8e4c05b970 msgr2=0x7f8e4c05bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:04.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.840+0000 7f8e5142e700 1 -- 172.21.15.162:0/3540731013 shutdown_connections 2020-11-28T21:54:04.852 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.840+0000 7f8e5142e700 1 -- 172.21.15.162:0/3540731013 wait complete. 2020-11-28T21:54:04.854 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.840+0000 7f8e5142e700 1 Processor -- start 2020-11-28T21:54:04.854 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.840+0000 7f8e5142e700 1 -- start start 2020-11-28T21:54:04.855 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.844+0000 7f8e5142e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8e4c1438a0 0x7f8e4c143c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:04.855 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.844+0000 7f8e5142e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8e4c144260 0x7f8e4c141670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:04.855 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.844+0000 7f8e5142e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e4c141b60 0x7f8e4c063e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:04.855 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.844+0000 7f8e5142e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c066b30 con 0x7f8e4c144260 2020-11-28T21:54:04.856 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.844+0000 7f8e5142e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c066d90 con 0x7f8e4c141b60 2020-11-28T21:54:04.856 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.844+0000 7f8e5142e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c1566d0 con 0x7f8e4c1438a0 2020-11-28T21:54:04.856 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.844+0000 7f8e4b7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e4c141b60 0x7f8e4c063e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:04.856 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.844+0000 7f8e4a7fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8e4c144260 0x7f8e4c141670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:04.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.844+0000 7f8e4a7fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8e4c144260 0x7f8e4c141670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.162:49836/0 (socket says 172.21.15.162:49836) 2020-11-28T21:54:04.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.844+0000 7f8e4a7fc700 1 -- 172.21.15.162:0/3328023174 learned_addr learned my addr 172.21.15.162:0/3328023174 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:04.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.844+0000 7f8e4affd700 1 --2- 172.21.15.162:0/3328023174 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8e4c1438a0 0x7f8e4c143c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:04.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.844+0000 7f8e4b7fe700 1 -- 172.21.15.162:0/3328023174 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8e4c1438a0 msgr2=0x7f8e4c143c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:04.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.844+0000 7f8e4b7fe700 1 --2- 172.21.15.162:0/3328023174 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8e4c1438a0 0x7f8e4c143c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:04.858 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.844+0000 7f8e4b7fe700 1 -- 172.21.15.162:0/3328023174 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8e4c144260 msgr2=0x7f8e4c141670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:04.858 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.844+0000 7f8e4b7fe700 1 --2- 172.21.15.162:0/3328023174 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8e4c144260 0x7f8e4c141670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:04.858 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.844+0000 7f8e4b7fe700 1 -- 172.21.15.162:0/3328023174 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4c156970 con 0x7f8e4c141b60 2020-11-28T21:54:04.858 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.844+0000 7f8e4a7fc700 1 --2- 172.21.15.162:0/3328023174 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8e4c144260 0x7f8e4c141670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:54:04.859 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.844+0000 7f8e4b7fe700 1 --2- 172.21.15.162:0/3328023174 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e4c141b60 0x7f8e4c063e00 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 rx=0x7f8e44000b20 tx=0x7f8e4c061370).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:04.859 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.844+0000 7f8e3bfff700 1 -- 172.21.15.162:0/3328023174 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e4400fb00 con 0x7f8e4c141b60 2020-11-28T21:54:04.859 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.844+0000 7f8e3bfff700 1 -- 172.21.15.162:0/3328023174 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e4400fc60 con 0x7f8e4c141b60 2020-11-28T21:54:04.859 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.844+0000 7f8e5142e700 1 -- 172.21.15.162:0/3328023174 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e4c065c70 con 0x7f8e4c141b60 2020-11-28T21:54:04.859 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.844+0000 7f8e3bfff700 1 -- 172.21.15.162:0/3328023174 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e44012350 con 0x7f8e4c141b60 2020-11-28T21:54:04.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.844+0000 7f8e5142e700 1 -- 172.21.15.162:0/3328023174 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e4c142210 con 0x7f8e4c141b60 2020-11-28T21:54:04.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.844+0000 7f8e5142e700 1 -- 172.21.15.162:0/3328023174 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e4c0655d0 con 0x7f8e4c141b60 2020-11-28T21:54:04.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.848+0000 7f8e3bfff700 1 -- 172.21.15.162:0/3328023174 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8e440124b0 con 0x7f8e4c141b60 2020-11-28T21:54:04.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.848+0000 7f8e3bfff700 1 --2- 172.21.15.162:0/3328023174 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8e280553b0 0x7f8e28057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:04.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.848+0000 7f8e3bfff700 1 -- 172.21.15.162:0/3328023174 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8e4406d790 con 0x7f8e4c141b60 2020-11-28T21:54:04.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.848+0000 7f8e4affd700 1 --2- 172.21.15.162:0/3328023174 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8e280553b0 0x7f8e28057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:04.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.848+0000 7f8e4affd700 1 --2- 172.21.15.162:0/3328023174 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8e280553b0 0x7f8e28057870 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f8e3c019000 tx=0x7f8e3c016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:04.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:04.852+0000 7f8e3bfff700 1 -- 172.21.15.162:0/3328023174 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8e44043a30 con 0x7f8e4c141b60 2020-11-28T21:54:05.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.032+0000 7f8e5142e700 1 -- 172.21.15.162:0/3328023174 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8e4c057050 con 0x7f8e4c141b60 2020-11-28T21:54:05.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.036+0000 7f8e3bfff700 1 -- 172.21.15.162:0/3328023174 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+2502 (secure 0 0 0) 0x7f8e44019020 con 0x7f8e4c141b60 2020-11-28T21:54:05.039 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:54:05.039 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":46,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6657,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.118:6833/1785592854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1785592854},{"type":"v1","addr":"172.21.15.118:6833","nonce":1785592854}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":44},{"gid":6850,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3533607073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3533607073},{"type":"v1","addr":"172.21.15.162:6835","nonce":3533607073}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:54:02.388344+0000","modified":"2020-11-28T21:54:03.391088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6859":{"gid":6859,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/3532128068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3532128068},{"type":"v1","addr":"172.21.15.162:6837","nonce":3532128068}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2020-11-28T21:54:05.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.036+0000 7f8e5142e700 1 -- 172.21.15.162:0/3328023174 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8e280553b0 msgr2=0x7f8e28057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:05.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.036+0000 7f8e5142e700 1 --2- 172.21.15.162:0/3328023174 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8e280553b0 0x7f8e28057870 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f8e3c019000 tx=0x7f8e3c016000).stop 2020-11-28T21:54:05.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.036+0000 7f8e5142e700 1 -- 172.21.15.162:0/3328023174 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e4c141b60 msgr2=0x7f8e4c063e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:05.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.036+0000 7f8e5142e700 1 --2- 172.21.15.162:0/3328023174 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e4c141b60 0x7f8e4c063e00 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 rx=0x7f8e44000b20 tx=0x7f8e4c061370).stop 2020-11-28T21:54:05.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.036+0000 7f8e5142e700 1 -- 172.21.15.162:0/3328023174 shutdown_connections 2020-11-28T21:54:05.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.036+0000 7f8e5142e700 1 --2- 172.21.15.162:0/3328023174 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8e280553b0 0x7f8e28057870 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:05.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.036+0000 7f8e5142e700 1 --2- 172.21.15.162:0/3328023174 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8e4c1438a0 0x7f8e4c143c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:05.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.036+0000 7f8e5142e700 1 --2- 172.21.15.162:0/3328023174 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e4c141b60 0x7f8e4c063e00 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:05.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.036+0000 7f8e5142e700 1 --2- 172.21.15.162:0/3328023174 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8e4c144260 0x7f8e4c141670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:05.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.036+0000 7f8e5142e700 1 -- 172.21.15.162:0/3328023174 >> 172.21.15.162:0/3328023174 conn(0x7f8e4c05b970 msgr2=0x7f8e4c0b34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:05.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.040+0000 7f8e5142e700 1 -- 172.21.15.162:0/3328023174 shutdown_connections 2020-11-28T21:54:05.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.040+0000 7f8e5142e700 1 -- 172.21.15.162:0/3328023174 wait complete. 2020-11-28T21:54:05.045 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 46 2020-11-28T21:54:05.361 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-28T21:54:02.388344+0000', 'modified': '2020-11-28T21:54:03.391088+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6859}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6859': {'gid': 6859, 'name': 'a', 'rank': 0, 'incarnation': 45, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6837/3532128068', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6836', 'nonce': 3532128068}, {'type': 'v1', 'addr': '172.21.15.162:6837', 'nonce': 3532128068}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-28T21:54:05.361 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-28T21:54:05.362 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-28T21:54:05.490 INFO:teuthology.orchestra.run.smithi162.stdout:{"cluster_fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","whoami":0,"id":6859,"want_state":"up:active","state":"up:active","rank_uptime":3.0873335260000001,"mdsmap_epoch":46,"osdmap_epoch":99,"osdmap_epoch_barrier":98,"uptime":8.467711542} 2020-11-28T21:54:05.491 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", "id": 6859, "mdsmap_epoch": 46, "osdmap_epoch": 99, "osdmap_epoch_barrier": 98, "rank_uptime": 3.087333526, "state": "up:active", "uptime": 8.467711542, "want_state": "up:active", "whoami": 0 } 2020-11-28T21:54:05.491 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:54:05.600 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.596+0000 7f25c0a8d700 1 Processor -- start 2020-11-28T21:54:05.600 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.596+0000 7f25c0a8d700 1 -- start start 2020-11-28T21:54:05.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.600+0000 7f25c0a8d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f25bc0b10f0 0x7f25bc13ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:05.603 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.600+0000 7f25c0a8d700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25bc0b1520 con 0x7f25bc0611e0 2020-11-28T21:54:05.603 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.600+0000 7f25c0a8d700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25bc0b1790 con 0x7f25bc0615c0 2020-11-28T21:54:05.603 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.600+0000 7f25c0a8d700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25bc0b1a30 con 0x7f25bc0b10f0 2020-11-28T21:54:05.603 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.600+0000 7f25baffd700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f25bc0615c0 0x7f25bc0b4b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:49700/0 (socket says 172.21.15.162:49700) 2020-11-28T21:54:05.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.600+0000 7f25baffd700 1 -- 172.21.15.162:0/1882777094 learned_addr learned my addr 172.21.15.162:0/1882777094 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:05.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.600+0000 7f25bbfff700 1 --2- 172.21.15.162:0/1882777094 >> v2:172.21.15.162:3300/0 conn(0x7f25bc0b10f0 0x7f25bc13ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:05.606 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.600+0000 7f25ba7fc700 1 -- 172.21.15.162:0/1882777094 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1912009098 0 0) 0x7f25bc0b1790 con 0x7f25bc0615c0 2020-11-28T21:54:05.606 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.600+0000 7f25ba7fc700 1 -- 172.21.15.162:0/1882777094 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25a8003230 con 0x7f25bc0615c0 2020-11-28T21:54:05.606 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.600+0000 7f25bbfff700 1 -- 172.21.15.162:0/1882777094 >> v1:172.21.15.162:6789/0 conn(0x7f25bc0615c0 legacy=0x7f25bc0b4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:05.606 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.600+0000 7f25bbfff700 1 -- 172.21.15.162:0/1882777094 >> v1:172.21.15.118:6789/0 conn(0x7f25bc0611e0 legacy=0x7f25bc0b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:05.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.600+0000 7f25bbfff700 1 -- 172.21.15.162:0/1882777094 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25bc0b3f90 con 0x7f25bc0b10f0 2020-11-28T21:54:05.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.600+0000 7f25bbfff700 1 --2- 172.21.15.162:0/1882777094 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25bc0b10f0 0x7f25bc13ee60 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 rx=0x7f25b4002f30 tx=0x7f25b4006dc0).ready entity=mon.1 client_cookie=bd79fc9331854caf server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:05.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.600+0000 7f25ba7fc700 1 -- 172.21.15.162:0/1882777094 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f25b4009800 con 0x7f25bc0b10f0 2020-11-28T21:54:05.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.600+0000 7f25ba7fc700 1 -- 172.21.15.162:0/1882777094 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b4009960 con 0x7f25bc0b10f0 2020-11-28T21:54:05.608 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.600+0000 7f25ba7fc700 1 -- 172.21.15.162:0/1882777094 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f25b4009cc0 con 0x7f25bc0b10f0 2020-11-28T21:54:05.608 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.600+0000 7f25c0a8d700 1 -- 172.21.15.162:0/1882777094 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25bc0b10f0 msgr2=0x7f25bc13ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:05.608 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.600+0000 7f25c0a8d700 1 --2- 172.21.15.162:0/1882777094 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25bc0b10f0 0x7f25bc13ee60 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 rx=0x7f25b4002f30 tx=0x7f25b4006dc0).stop 2020-11-28T21:54:05.608 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.600+0000 7f25c0a8d700 1 -- 172.21.15.162:0/1882777094 shutdown_connections 2020-11-28T21:54:05.608 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.600+0000 7f25c0a8d700 1 --2- 172.21.15.162:0/1882777094 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25bc0b10f0 0x7f25bc13ee60 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:05.609 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.600+0000 7f25c0a8d700 1 -- 172.21.15.162:0/1882777094 >> 172.21.15.162:0/1882777094 conn(0x7f25bc0b54f0 msgr2=0x7f25bc0b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:05.609 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.600+0000 7f25c0a8d700 1 -- 172.21.15.162:0/1882777094 shutdown_connections 2020-11-28T21:54:05.609 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.600+0000 7f25c0a8d700 1 -- 172.21.15.162:0/1882777094 wait complete. 2020-11-28T21:54:05.609 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.600+0000 7f25c0a8d700 1 Processor -- start 2020-11-28T21:54:05.610 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.600+0000 7f25c0a8d700 1 -- start start 2020-11-28T21:54:05.612 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.604+0000 7f25c0a8d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25bc0611e0 0x7f25bc09a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:05.612 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.604+0000 7f25c0a8d700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f25bc0615c0 0x7f25bc13ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:05.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.604+0000 7f25c0a8d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f25bc0b10f0 0x7f25bc12e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:05.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.604+0000 7f25c0a8d700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc063db0 con 0x7f25bc0b10f0 2020-11-28T21:54:05.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.604+0000 7f25c0a8d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc09ad10 con 0x7f25bc0611e0 2020-11-28T21:54:05.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.604+0000 7f25c0a8d700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc067b10 con 0x7f25bc0615c0 2020-11-28T21:54:05.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.604+0000 7f25bb7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25bc0611e0 0x7f25bc09a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:05.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.604+0000 7f25bb7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25bc0611e0 0x7f25bc09a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42860/0 (socket says 172.21.15.162:42860) 2020-11-28T21:54:05.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.604+0000 7f25bb7fe700 1 -- 172.21.15.162:0/3101097620 learned_addr learned my addr 172.21.15.162:0/3101097620 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:05.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.604+0000 7f25baffd700 1 --2- 172.21.15.162:0/3101097620 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f25bc0615c0 0x7f25bc13ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:05.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.604+0000 7f25bbfff700 1 --2- 172.21.15.162:0/3101097620 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f25bc0b10f0 0x7f25bc12e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:05.615 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.604+0000 7f25bb7fe700 1 -- 172.21.15.162:0/3101097620 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f25bc0615c0 msgr2=0x7f25bc13ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:05.615 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.604+0000 7f25bb7fe700 1 --2- 172.21.15.162:0/3101097620 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f25bc0615c0 0x7f25bc13ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:05.615 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.604+0000 7f25bb7fe700 1 -- 172.21.15.162:0/3101097620 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f25bc0b10f0 msgr2=0x7f25bc12e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:05.615 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.604+0000 7f25bb7fe700 1 --2- 172.21.15.162:0/3101097620 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f25bc0b10f0 0x7f25bc12e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:05.616 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.604+0000 7f25bb7fe700 1 -- 172.21.15.162:0/3101097620 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25bc09af40 con 0x7f25bc0611e0 2020-11-28T21:54:05.616 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.604+0000 7f25bbfff700 1 --2- 172.21.15.162:0/3101097620 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f25bc0b10f0 0x7f25bc12e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:54:05.616 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.604+0000 7f25bb7fe700 1 --2- 172.21.15.162:0/3101097620 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25bc0611e0 0x7f25bc09a270 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 rx=0x7f25bc060310 tx=0x7f25ac001460).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:05.616 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.604+0000 7f25b8ff9700 1 -- 172.21.15.162:0/3101097620 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f25ac00a7b0 con 0x7f25bc0611e0 2020-11-28T21:54:05.616 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.604+0000 7f25b8ff9700 1 -- 172.21.15.162:0/3101097620 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25ac00a910 con 0x7f25bc0611e0 2020-11-28T21:54:05.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.604+0000 7f25c0a8d700 1 -- 172.21.15.162:0/3101097620 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25bc067870 con 0x7f25bc0611e0 2020-11-28T21:54:05.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.604+0000 7f25b8ff9700 1 -- 172.21.15.162:0/3101097620 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f25ac00ac70 con 0x7f25bc0611e0 2020-11-28T21:54:05.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.604+0000 7f25c0a8d700 1 -- 172.21.15.162:0/3101097620 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25bc064fc0 con 0x7f25bc0611e0 2020-11-28T21:54:05.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.604+0000 7f25c0a8d700 1 -- 172.21.15.162:0/3101097620 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25bc099fd0 con 0x7f25bc0611e0 2020-11-28T21:54:05.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.604+0000 7f25b8ff9700 1 -- 172.21.15.162:0/3101097620 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f25ac019030 con 0x7f25bc0611e0 2020-11-28T21:54:05.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.608+0000 7f25b8ff9700 1 --2- 172.21.15.162:0/3101097620 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f259c0554e0 0x7f259c0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:05.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.608+0000 7f25b8ff9700 1 -- 172.21.15.162:0/3101097620 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f25ac06d880 con 0x7f25bc0611e0 2020-11-28T21:54:05.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.608+0000 7f25baffd700 1 --2- 172.21.15.162:0/3101097620 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f259c0554e0 0x7f259c0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:05.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.608+0000 7f25baffd700 1 --2- 172.21.15.162:0/3101097620 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f259c0554e0 0x7f259c0579a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7f25b000d000 tx=0x7f25b0009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:05.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.612+0000 7f25b8ff9700 1 -- 172.21.15.162:0/3101097620 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f25bc099fd0 con 0x7f25bc0611e0 2020-11-28T21:54:05.756 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.752+0000 7f25c0a8d700 1 -- 172.21.15.162:0/3101097620 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25bc099fd0 con 0x7f25bc0611e0 2020-11-28T21:54:05.758 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.756+0000 7f25b8ff9700 1 -- 172.21.15.162:0/3101097620 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+16137 (secure 0 0 0) 0x7f25bc099fd0 con 0x7f25bc0611e0 2020-11-28T21:54:05.759 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:54:05.759 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":99,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:54:03.391732+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:53:59.925390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2020-11-28T21:54:01.236032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:54:05.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.756+0000 7f25c0a8d700 1 -- 172.21.15.162:0/3101097620 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f259c0554e0 msgr2=0x7f259c0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:05.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.756+0000 7f25c0a8d700 1 --2- 172.21.15.162:0/3101097620 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f259c0554e0 0x7f259c0579a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7f25b000d000 tx=0x7f25b0009040).stop 2020-11-28T21:54:05.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.756+0000 7f25c0a8d700 1 -- 172.21.15.162:0/3101097620 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25bc0611e0 msgr2=0x7f25bc09a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:05.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.756+0000 7f25c0a8d700 1 --2- 172.21.15.162:0/3101097620 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25bc0611e0 0x7f25bc09a270 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 rx=0x7f25bc060310 tx=0x7f25ac001460).stop 2020-11-28T21:54:05.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.756+0000 7f25c0a8d700 1 -- 172.21.15.162:0/3101097620 shutdown_connections 2020-11-28T21:54:05.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.756+0000 7f25c0a8d700 1 --2- 172.21.15.162:0/3101097620 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f259c0554e0 0x7f259c0579a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:05.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.756+0000 7f25c0a8d700 1 --2- 172.21.15.162:0/3101097620 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f25bc0b10f0 0x7f25bc12e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:05.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.756+0000 7f25c0a8d700 1 --2- 172.21.15.162:0/3101097620 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25bc0611e0 0x7f25bc09a270 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:05.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.756+0000 7f25c0a8d700 1 --2- 172.21.15.162:0/3101097620 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f25bc0615c0 0x7f25bc13ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:05.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.756+0000 7f25c0a8d700 1 -- 172.21.15.162:0/3101097620 >> 172.21.15.162:0/3101097620 conn(0x7f25bc0b54f0 msgr2=0x7f25bc05c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:05.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.760+0000 7f25c0a8d700 1 -- 172.21.15.162:0/3101097620 shutdown_connections 2020-11-28T21:54:05.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.760+0000 7f25c0a8d700 1 -- 172.21.15.162:0/3101097620 wait complete. 2020-11-28T21:54:05.778 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-28T21:53:59.925390+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2020-11-28T21:54:01.236032+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-28T21:54:05.780 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:54:05.898 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.896+0000 7f9071bf9700 1 Processor -- start 2020-11-28T21:54:05.899 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.896+0000 7f9071bf9700 1 -- start start 2020-11-28T21:54:05.900 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.896+0000 7f9071bf9700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f906c0b9ce0 0x7f906c1433c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:05.900 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.896+0000 7f9071bf9700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f906c067610 0x7f906c13fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:05.900 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.896+0000 7f9071bf9700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f906c143950 con 0x7f906c0605c0 2020-11-28T21:54:05.901 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.896+0000 7f9071bf9700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f906c0b4340 con 0x7f906c067610 2020-11-28T21:54:05.901 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.896+0000 7f9071bf9700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f906c0ba0c0 con 0x7f906c0b9ce0 2020-11-28T21:54:05.901 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.896+0000 7f906b7fe700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f906c0b9ce0 0x7f906c1433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:05.901 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.896+0000 7f906bfff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f906c067610 0x7f906c13fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:05.902 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.896+0000 7f906b7fe700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f906c0b9ce0 0x7f906c1433c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:40092/0 (socket says 172.21.15.162:40092) 2020-11-28T21:54:05.902 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.896+0000 7f906b7fe700 1 -- 172.21.15.162:0/2935387724 learned_addr learned my addr 172.21.15.162:0/2935387724 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:05.903 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.896+0000 7f906bfff700 1 -- 172.21.15.162:0/2935387724 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f906c0b9ce0 msgr2=0x7f906c1433c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:05.903 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.896+0000 7f906bfff700 1 --2- 172.21.15.162:0/2935387724 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f906c0b9ce0 0x7f906c1433c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:05.903 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.896+0000 7f906bfff700 1 -- 172.21.15.162:0/2935387724 >> v1:172.21.15.118:6789/0 conn(0x7f906c0605c0 legacy=0x7f906c0b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:05.903 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.896+0000 7f906bfff700 1 -- 172.21.15.162:0/2935387724 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f906c067aa0 con 0x7f906c067610 2020-11-28T21:54:05.903 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.896+0000 7f906b7fe700 1 --2- 172.21.15.162:0/2935387724 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f906c0b9ce0 0x7f906c1433c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:54:05.904 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.896+0000 7f906bfff700 1 --2- 172.21.15.162:0/2935387724 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f906c067610 0x7f906c13fc80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f90640032b0 tx=0x7f90640091a0).ready entity=mon.0 client_cookie=b6a41339c147054d server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:05.905 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.900+0000 7f906a7fc700 1 -- 172.21.15.162:0/2935387724 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f906400b9a0 con 0x7f906c067610 2020-11-28T21:54:05.905 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.900+0000 7f906a7fc700 1 -- 172.21.15.162:0/2935387724 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f906400bb00 con 0x7f906c067610 2020-11-28T21:54:05.905 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.900+0000 7f906a7fc700 1 -- 172.21.15.162:0/2935387724 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f906400bc70 con 0x7f906c067610 2020-11-28T21:54:05.906 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.900+0000 7f9071bf9700 1 -- 172.21.15.162:0/2935387724 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f906c067610 msgr2=0x7f906c13fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:05.906 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.900+0000 7f9071bf9700 1 --2- 172.21.15.162:0/2935387724 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f906c067610 0x7f906c13fc80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f90640032b0 tx=0x7f90640091a0).stop 2020-11-28T21:54:05.906 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.900+0000 7f9071bf9700 1 -- 172.21.15.162:0/2935387724 shutdown_connections 2020-11-28T21:54:05.906 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.900+0000 7f9071bf9700 1 --2- 172.21.15.162:0/2935387724 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f906c067610 0x7f906c13fc80 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:05.906 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.900+0000 7f9071bf9700 1 --2- 172.21.15.162:0/2935387724 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f906c0b9ce0 0x7f906c1433c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:05.907 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.900+0000 7f9071bf9700 1 -- 172.21.15.162:0/2935387724 >> 172.21.15.162:0/2935387724 conn(0x7f906c05c430 msgr2=0x7f906c0b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:05.907 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.900+0000 7f9071bf9700 1 -- 172.21.15.162:0/2935387724 shutdown_connections 2020-11-28T21:54:05.907 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.900+0000 7f9071bf9700 1 -- 172.21.15.162:0/2935387724 wait complete. 2020-11-28T21:54:05.908 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.904+0000 7f9071bf9700 1 Processor -- start 2020-11-28T21:54:05.908 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.904+0000 7f9071bf9700 1 -- start start 2020-11-28T21:54:05.910 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.904+0000 7f9071bf9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f906c0605c0 0x7f906c141fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:05.910 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.904+0000 7f9071bf9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f906c067610 0x7f906c1550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:05.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.904+0000 7f9071bf9700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f906c0b9ce0 0x7f906c1555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:05.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.904+0000 7f9071bf9700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906c0b1510 con 0x7f906c067610 2020-11-28T21:54:05.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.904+0000 7f9071bf9700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906c0b17b0 con 0x7f906c0605c0 2020-11-28T21:54:05.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.904+0000 7f9071bf9700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f906c0b1a50 con 0x7f906c0b9ce0 2020-11-28T21:54:05.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.904+0000 7f906b7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f906c0605c0 0x7f906c141fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:05.912 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.904+0000 7f906b7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f906c0605c0 0x7f906c141fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42874/0 (socket says 172.21.15.162:42874) 2020-11-28T21:54:05.912 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.904+0000 7f906b7fe700 1 -- 172.21.15.162:0/3588720088 learned_addr learned my addr 172.21.15.162:0/3588720088 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:05.912 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.904+0000 7f906bfff700 1 --2- 172.21.15.162:0/3588720088 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f906c0b9ce0 0x7f906c1555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:05.912 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.904+0000 7f906affd700 1 --2- 172.21.15.162:0/3588720088 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f906c067610 0x7f906c1550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:05.912 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.904+0000 7f906b7fe700 1 -- 172.21.15.162:0/3588720088 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f906c0b9ce0 msgr2=0x7f906c1555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:05.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.904+0000 7f906b7fe700 1 --2- 172.21.15.162:0/3588720088 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f906c0b9ce0 0x7f906c1555a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:05.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.904+0000 7f906b7fe700 1 -- 172.21.15.162:0/3588720088 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f906c067610 msgr2=0x7f906c1550b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:05.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.904+0000 7f906b7fe700 1 --2- 172.21.15.162:0/3588720088 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f906c067610 0x7f906c1550b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:05.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.904+0000 7f906b7fe700 1 -- 172.21.15.162:0/3588720088 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f906c0b1cf0 con 0x7f906c0605c0 2020-11-28T21:54:05.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.904+0000 7f906affd700 1 --2- 172.21.15.162:0/3588720088 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f906c067610 0x7f906c1550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:54:05.914 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.904+0000 7f906b7fe700 1 --2- 172.21.15.162:0/3588720088 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f906c0605c0 0x7f906c141fd0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 rx=0x7f905c00c000 tx=0x7f905c006e20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:05.914 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.904+0000 7f9068ff9700 1 -- 172.21.15.162:0/3588720088 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f905c00cb40 con 0x7f906c0605c0 2020-11-28T21:54:05.914 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.904+0000 7f9071bf9700 1 -- 172.21.15.162:0/3588720088 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f906c0b0020 con 0x7f906c0605c0 2020-11-28T21:54:05.914 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.904+0000 7f9068ff9700 1 -- 172.21.15.162:0/3588720088 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f905c00cca0 con 0x7f906c0605c0 2020-11-28T21:54:05.916 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.904+0000 7f9071bf9700 1 -- 172.21.15.162:0/3588720088 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f906c0ad780 con 0x7f906c0605c0 2020-11-28T21:54:05.916 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.904+0000 7f9068ff9700 1 -- 172.21.15.162:0/3588720088 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f905c0123f0 con 0x7f906c0605c0 2020-11-28T21:54:05.917 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.908+0000 7f9071bf9700 1 -- 172.21.15.162:0/3588720088 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f906c057050 con 0x7f906c0605c0 2020-11-28T21:54:05.917 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.908+0000 7f9068ff9700 1 -- 172.21.15.162:0/3588720088 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f905c012550 con 0x7f906c0605c0 2020-11-28T21:54:05.918 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.908+0000 7f9068ff9700 1 --2- 172.21.15.162:0/3588720088 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f904c055500 0x7f904c0579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:05.918 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.908+0000 7f9068ff9700 1 -- 172.21.15.162:0/3588720088 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f905c06e600 con 0x7f906c0605c0 2020-11-28T21:54:05.918 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.908+0000 7f906affd700 1 --2- 172.21.15.162:0/3588720088 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f904c055500 0x7f904c0579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:05.918 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.908+0000 7f906affd700 1 --2- 172.21.15.162:0/3588720088 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f904c055500 0x7f904c0579c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f906000b000 tx=0x7f9060009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:05.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:05.916+0000 7f9068ff9700 1 -- 172.21.15.162:0/3588720088 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f905c044800 con 0x7f906c0605c0 2020-11-28T21:54:06.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.056+0000 7f9071bf9700 1 -- 172.21.15.162:0/3588720088 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f906c0b1e80 con 0x7f906c0605c0 2020-11-28T21:54:06.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.056+0000 7f9068ff9700 1 -- 172.21.15.162:0/3588720088 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v99) v1 ==== 110+0+10 (secure 0 0 0) 0x7f905c047e20 con 0x7f906c0605c0 2020-11-28T21:54:06.060 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:54:06.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.060+0000 7f9071bf9700 1 -- 172.21.15.162:0/3588720088 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f904c055500 msgr2=0x7f904c0579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:06.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.060+0000 7f9071bf9700 1 --2- 172.21.15.162:0/3588720088 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f904c055500 0x7f904c0579c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f906000b000 tx=0x7f9060009040).stop 2020-11-28T21:54:06.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.060+0000 7f9071bf9700 1 -- 172.21.15.162:0/3588720088 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f906c0605c0 msgr2=0x7f906c141fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:06.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.060+0000 7f9071bf9700 1 --2- 172.21.15.162:0/3588720088 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f906c0605c0 0x7f906c141fd0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 rx=0x7f905c00c000 tx=0x7f905c006e20).stop 2020-11-28T21:54:06.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.060+0000 7f9071bf9700 1 -- 172.21.15.162:0/3588720088 shutdown_connections 2020-11-28T21:54:06.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.060+0000 7f9071bf9700 1 --2- 172.21.15.162:0/3588720088 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f904c055500 0x7f904c0579c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:06.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.060+0000 7f9071bf9700 1 --2- 172.21.15.162:0/3588720088 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f906c0b9ce0 0x7f906c1555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:06.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.060+0000 7f9071bf9700 1 --2- 172.21.15.162:0/3588720088 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f906c0605c0 0x7f906c141fd0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:06.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.060+0000 7f9071bf9700 1 --2- 172.21.15.162:0/3588720088 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f906c067610 0x7f906c1550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:06.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.060+0000 7f9071bf9700 1 -- 172.21.15.162:0/3588720088 >> 172.21.15.162:0/3588720088 conn(0x7f906c05c430 msgr2=0x7f906c0b7fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:06.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.060+0000 7f9071bf9700 1 -- 172.21.15.162:0/3588720088 shutdown_connections 2020-11-28T21:54:06.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.060+0000 7f9071bf9700 1 -- 172.21.15.162:0/3588720088 wait complete. 2020-11-28T21:54:06.078 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:54:06.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.188+0000 7fa3e9b4b700 1 Processor -- start 2020-11-28T21:54:06.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.188+0000 7fa3e9b4b700 1 -- start start 2020-11-28T21:54:06.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.188+0000 7fa3e9b4b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa3e41404c0 0x7fa3e4060450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:06.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.188+0000 7fa3e9b4b700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3e40aff20 con 0x7fa3e40b21f0 2020-11-28T21:54:06.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.188+0000 7fa3e9b4b700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3e40b25d0 con 0x7fa3e4060940 2020-11-28T21:54:06.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.188+0000 7fa3e9b4b700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3e40af240 con 0x7fa3e41404c0 2020-11-28T21:54:06.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.188+0000 7fa3e3fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa3e41404c0 0x7fa3e4060450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:06.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.188+0000 7fa3e3fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa3e41404c0 0x7fa3e4060450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42884/0 (socket says 172.21.15.162:42884) 2020-11-28T21:54:06.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.188+0000 7fa3e934a700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fa3e4060940 0x7fa3e414d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:49730/0 (socket says 172.21.15.162:49730) 2020-11-28T21:54:06.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.188+0000 7fa3e3fff700 1 -- 172.21.15.162:0/2849849439 learned_addr learned my addr 172.21.15.162:0/2849849439 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:06.200 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.192+0000 7fa3e37fe700 1 -- 172.21.15.162:0/2849849439 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1198649182 0 0) 0x7fa3e40b25d0 con 0x7fa3e4060940 2020-11-28T21:54:06.200 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.192+0000 7fa3e37fe700 1 -- 172.21.15.162:0/2849849439 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3d0003230 con 0x7fa3e4060940 2020-11-28T21:54:06.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.192+0000 7fa3e3fff700 1 -- 172.21.15.162:0/2849849439 >> v1:172.21.15.162:6789/0 conn(0x7fa3e4060940 legacy=0x7fa3e414d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:06.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.192+0000 7fa3e3fff700 1 -- 172.21.15.162:0/2849849439 >> v1:172.21.15.118:6789/0 conn(0x7fa3e40b21f0 legacy=0x7fa3e413fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:06.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.192+0000 7fa3e3fff700 1 -- 172.21.15.162:0/2849849439 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e405fff0 con 0x7fa3e41404c0 2020-11-28T21:54:06.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.192+0000 7fa3e3fff700 1 --2- 172.21.15.162:0/2849849439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3e41404c0 0x7fa3e4060450 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 rx=0x7fa3d8002f30 tx=0x7fa3d8006e80).ready entity=mon.1 client_cookie=b562391eb1a22735 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:06.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.192+0000 7fa3e37fe700 1 -- 172.21.15.162:0/2849849439 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3d80097f0 con 0x7fa3e41404c0 2020-11-28T21:54:06.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.192+0000 7fa3e37fe700 1 -- 172.21.15.162:0/2849849439 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d8009950 con 0x7fa3e41404c0 2020-11-28T21:54:06.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.192+0000 7fa3e9b4b700 1 -- 172.21.15.162:0/2849849439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3e41404c0 msgr2=0x7fa3e4060450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:06.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.192+0000 7fa3e37fe700 1 -- 172.21.15.162:0/2849849439 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3d8009ac0 con 0x7fa3e41404c0 2020-11-28T21:54:06.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.192+0000 7fa3e9b4b700 1 --2- 172.21.15.162:0/2849849439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3e41404c0 0x7fa3e4060450 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 rx=0x7fa3d8002f30 tx=0x7fa3d8006e80).stop 2020-11-28T21:54:06.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.192+0000 7fa3e9b4b700 1 -- 172.21.15.162:0/2849849439 shutdown_connections 2020-11-28T21:54:06.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.192+0000 7fa3e9b4b700 1 --2- 172.21.15.162:0/2849849439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3e41404c0 0x7fa3e4060450 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:06.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.192+0000 7fa3e9b4b700 1 -- 172.21.15.162:0/2849849439 >> 172.21.15.162:0/2849849439 conn(0x7fa3e40b57d0 msgr2=0x7fa3e40b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:06.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.192+0000 7fa3e9b4b700 1 -- 172.21.15.162:0/2849849439 shutdown_connections 2020-11-28T21:54:06.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.192+0000 7fa3e9b4b700 1 -- 172.21.15.162:0/2849849439 wait complete. 2020-11-28T21:54:06.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.192+0000 7fa3e9b4b700 1 Processor -- start 2020-11-28T21:54:06.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.192+0000 7fa3e9b4b700 1 -- start start 2020-11-28T21:54:06.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.192+0000 7fa3e9b4b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3e4060940 0x7fa3e41497f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:06.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.192+0000 7fa3e9b4b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa3e40b21f0 0x7fa3e4145dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:06.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.192+0000 7fa3e9b4b700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa3e41404c0 0x7fa3e413df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:06.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.192+0000 7fa3e9b4b700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e4146970 con 0x7fa3e40b21f0 2020-11-28T21:54:06.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.192+0000 7fa3e9b4b700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e40a8c30 con 0x7fa3e4060940 2020-11-28T21:54:06.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.192+0000 7fa3e9b4b700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e40a9090 con 0x7fa3e41404c0 2020-11-28T21:54:06.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.192+0000 7fa3e8b49700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3e4060940 0x7fa3e41497f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:06.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.192+0000 7fa3e8b49700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3e4060940 0x7fa3e41497f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42888/0 (socket says 172.21.15.162:42888) 2020-11-28T21:54:06.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.192+0000 7fa3e8b49700 1 -- 172.21.15.162:0/587750031 learned_addr learned my addr 172.21.15.162:0/587750031 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:06.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.192+0000 7fa3e934a700 1 --2- 172.21.15.162:0/587750031 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa3e41404c0 0x7fa3e413df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:06.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.192+0000 7fa3e3fff700 1 --2- 172.21.15.162:0/587750031 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa3e40b21f0 0x7fa3e4145dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:06.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.192+0000 7fa3e8b49700 1 -- 172.21.15.162:0/587750031 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa3e41404c0 msgr2=0x7fa3e413df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:06.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.192+0000 7fa3e8b49700 1 --2- 172.21.15.162:0/587750031 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa3e41404c0 0x7fa3e413df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:06.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.192+0000 7fa3e8b49700 1 -- 172.21.15.162:0/587750031 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa3e40b21f0 msgr2=0x7fa3e4145dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:06.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.192+0000 7fa3e8b49700 1 --2- 172.21.15.162:0/587750031 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa3e40b21f0 0x7fa3e4145dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:06.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.192+0000 7fa3e8b49700 1 -- 172.21.15.162:0/587750031 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e4146ba0 con 0x7fa3e4060940 2020-11-28T21:54:06.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.192+0000 7fa3e3fff700 1 --2- 172.21.15.162:0/587750031 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa3e40b21f0 0x7fa3e4145dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:54:06.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.196+0000 7fa3e8b49700 1 --2- 172.21.15.162:0/587750031 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3e4060940 0x7fa3e41497f0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 rx=0x7fa3e405ad70 tx=0x7fa3d4007c20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:06.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.196+0000 7fa3e1ffb700 1 -- 172.21.15.162:0/587750031 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3d400e4a0 con 0x7fa3e4060940 2020-11-28T21:54:06.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.196+0000 7fa3e1ffb700 1 -- 172.21.15.162:0/587750031 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d400e600 con 0x7fa3e4060940 2020-11-28T21:54:06.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.196+0000 7fa3e9b4b700 1 -- 172.21.15.162:0/587750031 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3e4146e40 con 0x7fa3e4060940 2020-11-28T21:54:06.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.196+0000 7fa3e1ffb700 1 -- 172.21.15.162:0/587750031 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3d400e960 con 0x7fa3e4060940 2020-11-28T21:54:06.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.196+0000 7fa3e9b4b700 1 -- 172.21.15.162:0/587750031 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3e40ab9f0 con 0x7fa3e4060940 2020-11-28T21:54:06.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.196+0000 7fa3e9b4b700 1 -- 172.21.15.162:0/587750031 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3e4057050 con 0x7fa3e4060940 2020-11-28T21:54:06.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.196+0000 7fa3e1ffb700 1 -- 172.21.15.162:0/587750031 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa3d4019030 con 0x7fa3e4060940 2020-11-28T21:54:06.212 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.196+0000 7fa3e1ffb700 1 --2- 172.21.15.162:0/587750031 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa3c4055330 0x7fa3c40577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:06.212 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.196+0000 7fa3e3fff700 1 --2- 172.21.15.162:0/587750031 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa3c4055330 0x7fa3c40577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:06.212 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.196+0000 7fa3e1ffb700 1 -- 172.21.15.162:0/587750031 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa3d406d6b0 con 0x7fa3e4060940 2020-11-28T21:54:06.212 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.200+0000 7fa3e3fff700 1 --2- 172.21.15.162:0/587750031 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa3c4055330 0x7fa3c40577f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7fa3d8017000 tx=0x7fa3d8016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:06.213 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.204+0000 7fa3e1ffb700 1 -- 172.21.15.162:0/587750031 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa3d40438a0 con 0x7fa3e4060940 2020-11-28T21:54:06.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.348+0000 7fa3e9b4b700 1 -- 172.21.15.162:0/587750031 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa3e405d250 con 0x7fa3e4060940 2020-11-28T21:54:06.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.348+0000 7fa3e1ffb700 1 -- 172.21.15.162:0/587750031 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v99) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa3d4046ec0 con 0x7fa3e4060940 2020-11-28T21:54:06.351 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:54:06.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.348+0000 7fa3e9b4b700 1 -- 172.21.15.162:0/587750031 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa3c4055330 msgr2=0x7fa3c40577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:06.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.348+0000 7fa3e9b4b700 1 --2- 172.21.15.162:0/587750031 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa3c4055330 0x7fa3c40577f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7fa3d8017000 tx=0x7fa3d8016000).stop 2020-11-28T21:54:06.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.348+0000 7fa3e9b4b700 1 -- 172.21.15.162:0/587750031 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3e4060940 msgr2=0x7fa3e41497f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:06.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.348+0000 7fa3e9b4b700 1 --2- 172.21.15.162:0/587750031 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3e4060940 0x7fa3e41497f0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 rx=0x7fa3e405ad70 tx=0x7fa3d4007c20).stop 2020-11-28T21:54:06.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.348+0000 7fa3e9b4b700 1 -- 172.21.15.162:0/587750031 shutdown_connections 2020-11-28T21:54:06.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.348+0000 7fa3e9b4b700 1 --2- 172.21.15.162:0/587750031 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa3c4055330 0x7fa3c40577f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:06.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.348+0000 7fa3e9b4b700 1 --2- 172.21.15.162:0/587750031 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa3e41404c0 0x7fa3e413df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:06.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.348+0000 7fa3e9b4b700 1 --2- 172.21.15.162:0/587750031 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3e4060940 0x7fa3e41497f0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:06.355 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.348+0000 7fa3e9b4b700 1 --2- 172.21.15.162:0/587750031 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa3e40b21f0 0x7fa3e4145dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:06.355 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.348+0000 7fa3e9b4b700 1 -- 172.21.15.162:0/587750031 >> 172.21.15.162:0/587750031 conn(0x7fa3e40b57d0 msgr2=0x7fa3e40b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:06.355 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.348+0000 7fa3e9b4b700 1 -- 172.21.15.162:0/587750031 shutdown_connections 2020-11-28T21:54:06.355 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.348+0000 7fa3e9b4b700 1 -- 172.21.15.162:0/587750031 wait complete. 2020-11-28T21:54:06.364 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-28T21:54:06.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.476+0000 7f3a45e44700 1 Processor -- start 2020-11-28T21:54:06.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.476+0000 7f3a45e44700 1 -- start start 2020-11-28T21:54:06.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.476+0000 7f3a45e44700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3a4013d0c0 0x7f3a400b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:06.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.476+0000 7f3a45e44700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f3a400b13c0 0x7f3a40099b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:06.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.476+0000 7f3a45e44700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a4006aca0 con 0x7f3a4009a050 2020-11-28T21:54:06.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.476+0000 7f3a45e44700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a4013d4a0 con 0x7f3a400b13c0 2020-11-28T21:54:06.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.476+0000 7f3a45e44700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a40098da0 con 0x7f3a4013d0c0 2020-11-28T21:54:06.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.476+0000 7f3a44e42700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3a4013d0c0 0x7f3a400b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:06.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.476+0000 7f3a44e42700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3a4013d0c0 0x7f3a400b0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42900/0 (socket says 172.21.15.162:42900) 2020-11-28T21:54:06.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.480+0000 7f3a44e42700 1 -- 172.21.15.162:0/1509488950 learned_addr learned my addr 172.21.15.162:0/1509488950 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:06.485 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.480+0000 7f3a3ffff700 1 --2- 172.21.15.162:0/1509488950 >> v2:172.21.15.118:3300/0 conn(0x7f3a400b13c0 0x7f3a40099b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:06.485 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.480+0000 7f3a3ffff700 1 -- 172.21.15.162:0/1509488950 >> v2:172.21.15.162:3300/0 conn(0x7f3a4013d0c0 msgr2=0x7f3a400b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:06.485 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.480+0000 7f3a3ffff700 1 --2- 172.21.15.162:0/1509488950 >> v2:172.21.15.162:3300/0 conn(0x7f3a4013d0c0 0x7f3a400b0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:06.485 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.480+0000 7f3a3f7fe700 1 -- 172.21.15.162:0/1509488950 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 996956531 0 0) 0x7f3a4006aca0 con 0x7f3a4009a050 2020-11-28T21:54:06.486 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.480+0000 7f3a3ffff700 1 -- 172.21.15.162:0/1509488950 >> v1:172.21.15.118:6789/0 conn(0x7f3a4009a050 legacy=0x7f3a4005bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:06.486 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.480+0000 7f3a3ffff700 1 -- 172.21.15.162:0/1509488950 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a4006af40 con 0x7f3a400b13c0 2020-11-28T21:54:06.486 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.480+0000 7f3a44e42700 1 --2- 172.21.15.162:0/1509488950 >> v2:172.21.15.162:3300/0 conn(0x7f3a4013d0c0 0x7f3a400b0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:54:06.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.480+0000 7f3a3ffff700 1 --2- 172.21.15.162:0/1509488950 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3a400b13c0 0x7f3a40099b60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f3a34002f30 tx=0x7f3a34006dc0).ready entity=mon.0 client_cookie=8c33dbd1107bd8c5 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:06.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.480+0000 7f3a3f7fe700 1 -- 172.21.15.162:0/1509488950 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a340097b0 con 0x7f3a400b13c0 2020-11-28T21:54:06.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.480+0000 7f3a3f7fe700 1 -- 172.21.15.162:0/1509488950 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a34009910 con 0x7f3a400b13c0 2020-11-28T21:54:06.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.480+0000 7f3a3f7fe700 1 -- 172.21.15.162:0/1509488950 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a34009c70 con 0x7f3a400b13c0 2020-11-28T21:54:06.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.480+0000 7f3a45e44700 1 -- 172.21.15.162:0/1509488950 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3a400b13c0 msgr2=0x7f3a40099b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:06.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.480+0000 7f3a45e44700 1 --2- 172.21.15.162:0/1509488950 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3a400b13c0 0x7f3a40099b60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f3a34002f30 tx=0x7f3a34006dc0).stop 2020-11-28T21:54:06.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.480+0000 7f3a45e44700 1 -- 172.21.15.162:0/1509488950 shutdown_connections 2020-11-28T21:54:06.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.480+0000 7f3a45e44700 1 --2- 172.21.15.162:0/1509488950 >> v2:172.21.15.162:3300/0 conn(0x7f3a4013d0c0 0x7f3a400b0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:06.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.480+0000 7f3a45e44700 1 --2- 172.21.15.162:0/1509488950 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3a400b13c0 0x7f3a40099b60 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:06.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.480+0000 7f3a45e44700 1 -- 172.21.15.162:0/1509488950 >> 172.21.15.162:0/1509488950 conn(0x7f3a400b5520 msgr2=0x7f3a400b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:06.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.480+0000 7f3a45e44700 1 -- 172.21.15.162:0/1509488950 shutdown_connections 2020-11-28T21:54:06.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.480+0000 7f3a45e44700 1 -- 172.21.15.162:0/1509488950 wait complete. 2020-11-28T21:54:06.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.480+0000 7f3a45e44700 1 Processor -- start 2020-11-28T21:54:06.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.480+0000 7f3a45e44700 1 -- start start 2020-11-28T21:54:06.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.484+0000 7f3a45e44700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3a4009a050 0x7f3a40154f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:06.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.484+0000 7f3a45e44700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a400b13c0 0x7f3a400ae480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:06.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.484+0000 7f3a45e44700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3a4013d0c0 0x7f3a40060760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:06.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.484+0000 7f3a45e44700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a400b5300 con 0x7f3a4009a050 2020-11-28T21:54:06.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.484+0000 7f3a45e44700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a400b50a0 con 0x7f3a400b13c0 2020-11-28T21:54:06.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.484+0000 7f3a45e44700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a400ae970 con 0x7f3a4013d0c0 2020-11-28T21:54:06.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.484+0000 7f3a3ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a400b13c0 0x7f3a400ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:06.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.484+0000 7f3a3ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a400b13c0 0x7f3a400ae480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42904/0 (socket says 172.21.15.162:42904) 2020-11-28T21:54:06.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.484+0000 7f3a3ffff700 1 -- 172.21.15.162:0/1568010018 learned_addr learned my addr 172.21.15.162:0/1568010018 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:06.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.484+0000 7f3a44e42700 1 --2- 172.21.15.162:0/1568010018 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3a4009a050 0x7f3a40154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:06.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.484+0000 7f3a45643700 1 --2- 172.21.15.162:0/1568010018 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3a4013d0c0 0x7f3a40060760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:06.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.484+0000 7f3a3ffff700 1 -- 172.21.15.162:0/1568010018 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3a4013d0c0 msgr2=0x7f3a40060760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:06.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.484+0000 7f3a3ffff700 1 --2- 172.21.15.162:0/1568010018 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3a4013d0c0 0x7f3a40060760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:06.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.484+0000 7f3a3ffff700 1 -- 172.21.15.162:0/1568010018 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3a4009a050 msgr2=0x7f3a40154f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:06.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.484+0000 7f3a3ffff700 1 --2- 172.21.15.162:0/1568010018 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3a4009a050 0x7f3a40154f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:06.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.484+0000 7f3a3ffff700 1 -- 172.21.15.162:0/1568010018 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a400aee70 con 0x7f3a400b13c0 2020-11-28T21:54:06.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.484+0000 7f3a3ffff700 1 --2- 172.21.15.162:0/1568010018 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a400b13c0 0x7f3a400ae480 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 rx=0x7f3a400b4c30 tx=0x7f3a34015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:06.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.484+0000 7f3a3dffb700 1 -- 172.21.15.162:0/1568010018 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a340119a0 con 0x7f3a400b13c0 2020-11-28T21:54:06.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.484+0000 7f3a3dffb700 1 -- 172.21.15.162:0/1568010018 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a34011b00 con 0x7f3a400b13c0 2020-11-28T21:54:06.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.484+0000 7f3a3dffb700 1 -- 172.21.15.162:0/1568010018 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a34011e60 con 0x7f3a400b13c0 2020-11-28T21:54:06.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.484+0000 7f3a45e44700 1 -- 172.21.15.162:0/1568010018 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a40136100 con 0x7f3a400b13c0 2020-11-28T21:54:06.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.484+0000 7f3a45e44700 1 -- 172.21.15.162:0/1568010018 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a400b3b30 con 0x7f3a400b13c0 2020-11-28T21:54:06.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.484+0000 7f3a45e44700 1 -- 172.21.15.162:0/1568010018 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a401553f0 con 0x7f3a400b13c0 2020-11-28T21:54:06.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.488+0000 7f3a3dffb700 1 -- 172.21.15.162:0/1568010018 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f3a34024030 con 0x7f3a400b13c0 2020-11-28T21:54:06.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.488+0000 7f3a3dffb700 1 --2- 172.21.15.162:0/1568010018 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3a200554e0 0x7f3a200579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:06.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.488+0000 7f3a3dffb700 1 -- 172.21.15.162:0/1568010018 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3a3406ecf0 con 0x7f3a400b13c0 2020-11-28T21:54:06.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.488+0000 7f3a44e42700 1 --2- 172.21.15.162:0/1568010018 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3a200554e0 0x7f3a200579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:06.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.488+0000 7f3a44e42700 1 --2- 172.21.15.162:0/1568010018 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3a200554e0 0x7f3a200579a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7f3a30008000 tx=0x7f3a30008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:06.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.492+0000 7f3a3dffb700 1 -- 172.21.15.162:0/1568010018 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f3a401553f0 con 0x7f3a400b13c0 2020-11-28T21:54:06.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.636+0000 7f3a45e44700 1 -- 172.21.15.162:0/1568010018 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3a401553f0 con 0x7f3a400b13c0 2020-11-28T21:54:06.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.636+0000 7f3a3dffb700 1 -- 172.21.15.162:0/1568010018 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) v1 ==== 104+0+10 (secure 0 0 0) 0x7f3a401553f0 con 0x7f3a400b13c0 2020-11-28T21:54:06.641 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:54:06.644 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.640+0000 7f3a45e44700 1 -- 172.21.15.162:0/1568010018 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3a200554e0 msgr2=0x7f3a200579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:06.644 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.640+0000 7f3a45e44700 1 --2- 172.21.15.162:0/1568010018 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3a200554e0 0x7f3a200579a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7f3a30008000 tx=0x7f3a30008040).stop 2020-11-28T21:54:06.645 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.640+0000 7f3a45e44700 1 -- 172.21.15.162:0/1568010018 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a400b13c0 msgr2=0x7f3a400ae480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:06.645 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.640+0000 7f3a45e44700 1 --2- 172.21.15.162:0/1568010018 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a400b13c0 0x7f3a400ae480 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 rx=0x7f3a400b4c30 tx=0x7f3a34015000).stop 2020-11-28T21:54:06.646 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.640+0000 7f3a45e44700 1 -- 172.21.15.162:0/1568010018 shutdown_connections 2020-11-28T21:54:06.646 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.640+0000 7f3a45e44700 1 --2- 172.21.15.162:0/1568010018 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3a200554e0 0x7f3a200579a0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:06.646 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.640+0000 7f3a45e44700 1 --2- 172.21.15.162:0/1568010018 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3a4013d0c0 0x7f3a40060760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:06.647 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.640+0000 7f3a45e44700 1 --2- 172.21.15.162:0/1568010018 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3a4009a050 0x7f3a40154f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:06.647 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.640+0000 7f3a45e44700 1 --2- 172.21.15.162:0/1568010018 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a400b13c0 0x7f3a400ae480 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:06.648 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.640+0000 7f3a45e44700 1 -- 172.21.15.162:0/1568010018 >> 172.21.15.162:0/1568010018 conn(0x7f3a400b5520 msgr2=0x7f3a40132300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:06.648 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.640+0000 7f3a45e44700 1 -- 172.21.15.162:0/1568010018 shutdown_connections 2020-11-28T21:54:06.649 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.640+0000 7f3a45e44700 1 -- 172.21.15.162:0/1568010018 wait complete. 2020-11-28T21:54:06.656 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-28T21:54:06.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.760+0000 7f1e8f8d8700 1 Processor -- start 2020-11-28T21:54:06.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.760+0000 7f1e8f8d8700 1 -- start start 2020-11-28T21:54:06.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.760+0000 7f1e8f8d8700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1e8805ccc0 0x7f1e8813fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:06.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.760+0000 7f1e8f8d8700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1e880b1200 0x7f1e881401d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:06.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.760+0000 7f1e8f8d8700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1e880ac240 0x7f1e88142d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:06.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.760+0000 7f1e8f8d8700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e88061630 con 0x7f1e880ac240 2020-11-28T21:54:06.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.760+0000 7f1e8f8d8700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e8805d0a0 con 0x7f1e880b1200 2020-11-28T21:54:06.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.760+0000 7f1e8f8d8700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e880aa710 con 0x7f1e8805ccc0 2020-11-28T21:54:06.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.760+0000 7f1e8ce73700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1e880b1200 0x7f1e881401d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:06.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.760+0000 7f1e8ce73700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1e880b1200 0x7f1e881401d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42910/0 (socket says 172.21.15.162:42910) 2020-11-28T21:54:06.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.760+0000 7f1e8ce73700 1 -- 172.21.15.162:0/3241034849 learned_addr learned my addr 172.21.15.162:0/3241034849 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:06.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.760+0000 7f1e8d674700 1 --2- 172.21.15.162:0/3241034849 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1e8805ccc0 0x7f1e8813fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:06.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.760+0000 7f1e8de75700 1 --2- 172.21.15.162:0/3241034849 >> v2:172.21.15.118:3300/0 conn(0x7f1e880ac240 0x7f1e88142d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:06.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.760+0000 7f1e8ce73700 1 -- 172.21.15.162:0/3241034849 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1e8805ccc0 msgr2=0x7f1e8813fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:06.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.760+0000 7f1e8ce73700 1 --2- 172.21.15.162:0/3241034849 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1e8805ccc0 0x7f1e8813fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:06.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.760+0000 7f1e8ce73700 1 -- 172.21.15.162:0/3241034849 >> v2:172.21.15.118:3300/0 conn(0x7f1e880ac240 msgr2=0x7f1e88142d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:06.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.760+0000 7f1e8ce73700 1 --2- 172.21.15.162:0/3241034849 >> v2:172.21.15.118:3300/0 conn(0x7f1e880ac240 0x7f1e88142d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:06.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.760+0000 7f1e8ce73700 1 -- 172.21.15.162:0/3241034849 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e88061860 con 0x7f1e880b1200 2020-11-28T21:54:06.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.760+0000 7f1e8ce73700 1 --2- 172.21.15.162:0/3241034849 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e880b1200 0x7f1e881401d0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 rx=0x7f1e84002f30 tx=0x7f1e84006e80).ready entity=mon.1 client_cookie=85cd1a44a1bcfa17 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:06.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.760+0000 7f1e77fff700 1 -- 172.21.15.162:0/3241034849 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e840097f0 con 0x7f1e880b1200 2020-11-28T21:54:06.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.760+0000 7f1e77fff700 1 -- 172.21.15.162:0/3241034849 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e84009950 con 0x7f1e880b1200 2020-11-28T21:54:06.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.760+0000 7f1e77fff700 1 -- 172.21.15.162:0/3241034849 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e84009cb0 con 0x7f1e880b1200 2020-11-28T21:54:06.776 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.760+0000 7f1e8f8d8700 1 -- 172.21.15.162:0/3241034849 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e880b1200 msgr2=0x7f1e881401d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:06.776 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.760+0000 7f1e8f8d8700 1 --2- 172.21.15.162:0/3241034849 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e880b1200 0x7f1e881401d0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 rx=0x7f1e84002f30 tx=0x7f1e84006e80).stop 2020-11-28T21:54:06.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.764+0000 7f1e8f8d8700 1 -- 172.21.15.162:0/3241034849 shutdown_connections 2020-11-28T21:54:06.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.764+0000 7f1e8f8d8700 1 --2- 172.21.15.162:0/3241034849 >> v2:172.21.15.118:3300/0 conn(0x7f1e880ac240 0x7f1e88142d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:06.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.764+0000 7f1e8f8d8700 1 --2- 172.21.15.162:0/3241034849 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1e8805ccc0 0x7f1e8813fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:06.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.764+0000 7f1e8f8d8700 1 --2- 172.21.15.162:0/3241034849 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e880b1200 0x7f1e881401d0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:06.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.764+0000 7f1e8f8d8700 1 -- 172.21.15.162:0/3241034849 >> 172.21.15.162:0/3241034849 conn(0x7f1e880b6220 msgr2=0x7f1e880b21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:06.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.764+0000 7f1e8f8d8700 1 -- 172.21.15.162:0/3241034849 shutdown_connections 2020-11-28T21:54:06.780 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.764+0000 7f1e8f8d8700 1 -- 172.21.15.162:0/3241034849 wait complete. 2020-11-28T21:54:06.785 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.764+0000 7f1e8f8d8700 1 Processor -- start 2020-11-28T21:54:06.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.764+0000 7f1e8f8d8700 1 -- start start 2020-11-28T21:54:06.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.768+0000 7f1e8f8d8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e8805ccc0 0x7f1e8813cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:06.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.768+0000 7f1e8f8d8700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1e880ac240 0x7f1e881399c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:06.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.768+0000 7f1e8f8d8700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1e880b1200 0x7f1e88139eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:06.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.768+0000 7f1e8f8d8700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8813d8e0 con 0x7f1e880b1200 2020-11-28T21:54:06.788 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.768+0000 7f1e8f8d8700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e880ae8c0 con 0x7f1e8805ccc0 2020-11-28T21:54:06.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.768+0000 7f1e8f8d8700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e880ad4b0 con 0x7f1e880ac240 2020-11-28T21:54:06.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.768+0000 7f1e8d674700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e8805ccc0 0x7f1e8813cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:06.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.768+0000 7f1e8d674700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e8805ccc0 0x7f1e8813cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42918/0 (socket says 172.21.15.162:42918) 2020-11-28T21:54:06.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.768+0000 7f1e8ce73700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1e880ac240 0x7f1e881399c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:06.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.768+0000 7f1e8d674700 1 -- 172.21.15.162:0/2270233602 learned_addr learned my addr 172.21.15.162:0/2270233602 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:06.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.768+0000 7f1e8de75700 1 --2- 172.21.15.162:0/2270233602 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1e880b1200 0x7f1e88139eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:06.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.768+0000 7f1e8d674700 1 -- 172.21.15.162:0/2270233602 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1e880ac240 msgr2=0x7f1e881399c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:06.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.768+0000 7f1e8d674700 1 --2- 172.21.15.162:0/2270233602 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1e880ac240 0x7f1e881399c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:06.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.768+0000 7f1e8d674700 1 -- 172.21.15.162:0/2270233602 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1e880b1200 msgr2=0x7f1e88139eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:06.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.768+0000 7f1e8d674700 1 --2- 172.21.15.162:0/2270233602 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1e880b1200 0x7f1e88139eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:06.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.768+0000 7f1e8d674700 1 -- 172.21.15.162:0/2270233602 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e880ad750 con 0x7f1e8805ccc0 2020-11-28T21:54:06.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.768+0000 7f1e8d674700 1 --2- 172.21.15.162:0/2270233602 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e8805ccc0 0x7f1e8813cff0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 rx=0x7f1e78003070 tx=0x7f1e78008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:06.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.768+0000 7f1e767fc700 1 -- 172.21.15.162:0/2270233602 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e78009730 con 0x7f1e8805ccc0 2020-11-28T21:54:06.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.768+0000 7f1e767fc700 1 -- 172.21.15.162:0/2270233602 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e78009890 con 0x7f1e8805ccc0 2020-11-28T21:54:06.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.768+0000 7f1e8f8d8700 1 -- 172.21.15.162:0/2270233602 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e8813a3a0 con 0x7f1e8805ccc0 2020-11-28T21:54:06.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.768+0000 7f1e767fc700 1 -- 172.21.15.162:0/2270233602 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e78009bf0 con 0x7f1e8805ccc0 2020-11-28T21:54:06.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.768+0000 7f1e8f8d8700 1 -- 172.21.15.162:0/2270233602 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e880af770 con 0x7f1e8805ccc0 2020-11-28T21:54:06.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.772+0000 7f1e8f8d8700 1 -- 172.21.15.162:0/2270233602 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e880aea80 con 0x7f1e8805ccc0 2020-11-28T21:54:06.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.772+0000 7f1e767fc700 1 -- 172.21.15.162:0/2270233602 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1e78009d50 con 0x7f1e8805ccc0 2020-11-28T21:54:06.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.772+0000 7f1e767fc700 1 --2- 172.21.15.162:0/2270233602 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1e68055440 0x7f1e68057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:06.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.772+0000 7f1e767fc700 1 -- 172.21.15.162:0/2270233602 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1e7806e280 con 0x7f1e8805ccc0 2020-11-28T21:54:06.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.772+0000 7f1e8ce73700 1 --2- 172.21.15.162:0/2270233602 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1e68055440 0x7f1e68057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:06.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.772+0000 7f1e8ce73700 1 --2- 172.21.15.162:0/2270233602 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1e68055440 0x7f1e68057900 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f1e8401c000 tx=0x7f1e84015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:06.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.780+0000 7f1e767fc700 1 -- 172.21.15.162:0/2270233602 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1e78044580 con 0x7f1e8805ccc0 2020-11-28T21:54:06.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.920+0000 7f1e8f8d8700 1 -- 172.21.15.162:0/2270233602 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1e880ad8e0 con 0x7f1e8805ccc0 2020-11-28T21:54:06.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.920+0000 7f1e767fc700 1 -- 172.21.15.162:0/2270233602 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) v1 ==== 100+0+10 (secure 0 0 0) 0x7f1e78047ba0 con 0x7f1e8805ccc0 2020-11-28T21:54:06.926 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:54:06.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.924+0000 7f1e8f8d8700 1 -- 172.21.15.162:0/2270233602 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1e68055440 msgr2=0x7f1e68057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:06.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.924+0000 7f1e8f8d8700 1 --2- 172.21.15.162:0/2270233602 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1e68055440 0x7f1e68057900 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f1e8401c000 tx=0x7f1e84015000).stop 2020-11-28T21:54:06.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.924+0000 7f1e8f8d8700 1 -- 172.21.15.162:0/2270233602 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e8805ccc0 msgr2=0x7f1e8813cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:06.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.924+0000 7f1e8f8d8700 1 --2- 172.21.15.162:0/2270233602 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e8805ccc0 0x7f1e8813cff0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 rx=0x7f1e78003070 tx=0x7f1e78008040).stop 2020-11-28T21:54:06.931 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.924+0000 7f1e8f8d8700 1 -- 172.21.15.162:0/2270233602 shutdown_connections 2020-11-28T21:54:06.931 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.924+0000 7f1e8f8d8700 1 --2- 172.21.15.162:0/2270233602 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1e68055440 0x7f1e68057900 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:06.931 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.924+0000 7f1e8f8d8700 1 --2- 172.21.15.162:0/2270233602 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1e880b1200 0x7f1e88139eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:06.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.924+0000 7f1e8f8d8700 1 --2- 172.21.15.162:0/2270233602 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e8805ccc0 0x7f1e8813cff0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:06.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.924+0000 7f1e8f8d8700 1 --2- 172.21.15.162:0/2270233602 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1e880ac240 0x7f1e881399c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:06.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.924+0000 7f1e8f8d8700 1 -- 172.21.15.162:0/2270233602 >> 172.21.15.162:0/2270233602 conn(0x7f1e880b6220 msgr2=0x7f1e8805b970 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:06.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.924+0000 7f1e8f8d8700 1 -- 172.21.15.162:0/2270233602 shutdown_connections 2020-11-28T21:54:06.933 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:06.924+0000 7f1e8f8d8700 1 -- 172.21.15.162:0/2270233602 wait complete. 2020-11-28T21:54:06.938 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-28T21:54:06.939 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:54:07.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.048+0000 7fb7898b0700 1 Processor -- start 2020-11-28T21:54:07.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.048+0000 7fb7898b0700 1 -- start start 2020-11-28T21:54:07.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.048+0000 7fb7898b0700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb7840b19f0 0x7fb7840b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:07.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.048+0000 7fb7898b0700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fb7840b1f00 0x7fb7840b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:07.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.048+0000 7fb7898b0700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7840b7700 con 0x7fb7840b7320 2020-11-28T21:54:07.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.048+0000 7fb7898b0700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb784067b50 con 0x7fb7840b1f00 2020-11-28T21:54:07.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.048+0000 7fb7898b0700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb784067df0 con 0x7fb7840b19f0 2020-11-28T21:54:07.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.048+0000 7fb7827fc700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb7840b19f0 0x7fb7840b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:07.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.048+0000 7fb782ffd700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fb7840b7320 0x7fb78413d9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:49768/0 (socket says 172.21.15.162:49768) 2020-11-28T21:54:07.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.048+0000 7fb782ffd700 1 -- 172.21.15.162:0/2032417078 learned_addr learned my addr 172.21.15.162:0/2032417078 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:07.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.048+0000 7fb7837fe700 1 --2- 172.21.15.162:0/2032417078 >> v2:172.21.15.118:3300/0 conn(0x7fb7840b1f00 0x7fb7840b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:07.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.048+0000 7fb781ffb700 1 -- 172.21.15.162:0/2032417078 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3989004087 0 0) 0x7fb7840b7700 con 0x7fb7840b7320 2020-11-28T21:54:07.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.048+0000 7fb781ffb700 1 -- 172.21.15.162:0/2032417078 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb764003230 con 0x7fb7840b7320 2020-11-28T21:54:07.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.048+0000 7fb7827fc700 1 -- 172.21.15.162:0/2032417078 >> v2:172.21.15.118:3300/0 conn(0x7fb7840b1f00 msgr2=0x7fb7840b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:07.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.048+0000 7fb7827fc700 1 --2- 172.21.15.162:0/2032417078 >> v2:172.21.15.118:3300/0 conn(0x7fb7840b1f00 0x7fb7840b2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:07.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.048+0000 7fb7827fc700 1 -- 172.21.15.162:0/2032417078 >> v1:172.21.15.162:6789/0 conn(0x7fb7840b7320 legacy=0x7fb78413d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:07.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.048+0000 7fb7827fc700 1 -- 172.21.15.162:0/2032417078 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb784065c80 con 0x7fb7840b19f0 2020-11-28T21:54:07.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.048+0000 7fb7827fc700 1 --2- 172.21.15.162:0/2032417078 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7840b19f0 0x7fb7840b27b0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 rx=0x7fb76c0032b0 tx=0x7fb76c0091a0).ready entity=mon.1 client_cookie=ea055bf5639ef7a6 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:07.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.048+0000 7fb781ffb700 1 -- 172.21.15.162:0/2032417078 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb76c00b9a0 con 0x7fb7840b19f0 2020-11-28T21:54:07.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.048+0000 7fb781ffb700 1 -- 172.21.15.162:0/2032417078 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb76c00bb00 con 0x7fb7840b19f0 2020-11-28T21:54:07.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.048+0000 7fb7898b0700 1 -- 172.21.15.162:0/2032417078 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7840b19f0 msgr2=0x7fb7840b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:07.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.048+0000 7fb781ffb700 1 -- 172.21.15.162:0/2032417078 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb76c00bcb0 con 0x7fb7840b19f0 2020-11-28T21:54:07.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.048+0000 7fb7898b0700 1 --2- 172.21.15.162:0/2032417078 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7840b19f0 0x7fb7840b27b0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 rx=0x7fb76c0032b0 tx=0x7fb76c0091a0).stop 2020-11-28T21:54:07.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.048+0000 7fb7898b0700 1 -- 172.21.15.162:0/2032417078 shutdown_connections 2020-11-28T21:54:07.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.048+0000 7fb7898b0700 1 --2- 172.21.15.162:0/2032417078 >> v2:172.21.15.118:3300/0 conn(0x7fb7840b1f00 0x7fb7840b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:07.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.048+0000 7fb7898b0700 1 --2- 172.21.15.162:0/2032417078 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7840b19f0 0x7fb7840b27b0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:07.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.048+0000 7fb7898b0700 1 -- 172.21.15.162:0/2032417078 >> 172.21.15.162:0/2032417078 conn(0x7fb784062830 msgr2=0x7fb784062c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:07.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.048+0000 7fb7898b0700 1 -- 172.21.15.162:0/2032417078 shutdown_connections 2020-11-28T21:54:07.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.048+0000 7fb7898b0700 1 -- 172.21.15.162:0/2032417078 wait complete. 2020-11-28T21:54:07.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.052+0000 7fb7898b0700 1 Processor -- start 2020-11-28T21:54:07.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.052+0000 7fb7898b0700 1 -- start start 2020-11-28T21:54:07.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.052+0000 7fb7898b0700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb7840b19f0 0x7fb7840b5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:07.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.052+0000 7fb7898b0700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb7840b1f00 0x7fb784135930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:07.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.052+0000 7fb7898b0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7840b7320 0x7fb784135e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:07.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.052+0000 7fb7898b0700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78414f700 con 0x7fb7840b1f00 2020-11-28T21:54:07.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.052+0000 7fb7898b0700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb784147250 con 0x7fb7840b7320 2020-11-28T21:54:07.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.052+0000 7fb7898b0700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb784136310 con 0x7fb7840b19f0 2020-11-28T21:54:07.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.052+0000 7fb7837fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7840b7320 0x7fb784135e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:07.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.052+0000 7fb7837fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7840b7320 0x7fb784135e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42934/0 (socket says 172.21.15.162:42934) 2020-11-28T21:54:07.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.052+0000 7fb782ffd700 1 --2- 172.21.15.162:0/2828389603 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb7840b19f0 0x7fb7840b5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:07.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.052+0000 7fb7837fe700 1 -- 172.21.15.162:0/2828389603 learned_addr learned my addr 172.21.15.162:0/2828389603 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:07.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.052+0000 7fb7827fc700 1 --2- 172.21.15.162:0/2828389603 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb7840b1f00 0x7fb784135930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:07.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.052+0000 7fb7837fe700 1 -- 172.21.15.162:0/2828389603 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb7840b19f0 msgr2=0x7fb7840b5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:07.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.052+0000 7fb7837fe700 1 --2- 172.21.15.162:0/2828389603 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb7840b19f0 0x7fb7840b5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:07.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.052+0000 7fb7837fe700 1 -- 172.21.15.162:0/2828389603 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb7840b1f00 msgr2=0x7fb784135930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:07.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.052+0000 7fb7837fe700 1 --2- 172.21.15.162:0/2828389603 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb7840b1f00 0x7fb784135930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:07.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.052+0000 7fb7837fe700 1 -- 172.21.15.162:0/2828389603 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb784147940 con 0x7fb7840b7320 2020-11-28T21:54:07.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.052+0000 7fb7837fe700 1 --2- 172.21.15.162:0/2828389603 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7840b7320 0x7fb784135e20 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 rx=0x7fb7840ad120 tx=0x7fb774008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:07.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.052+0000 7fb77bfff700 1 -- 172.21.15.162:0/2828389603 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7740127e0 con 0x7fb7840b7320 2020-11-28T21:54:07.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.052+0000 7fb77bfff700 1 -- 172.21.15.162:0/2828389603 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb774012940 con 0x7fb7840b7320 2020-11-28T21:54:07.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.052+0000 7fb7898b0700 1 -- 172.21.15.162:0/2828389603 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb784147480 con 0x7fb7840b7320 2020-11-28T21:54:07.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.052+0000 7fb77bfff700 1 -- 172.21.15.162:0/2828389603 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb774012ca0 con 0x7fb7840b7320 2020-11-28T21:54:07.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.052+0000 7fb7898b0700 1 -- 172.21.15.162:0/2828389603 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb784147be0 con 0x7fb7840b7320 2020-11-28T21:54:07.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.056+0000 7fb7898b0700 1 -- 172.21.15.162:0/2828389603 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb784147610 con 0x7fb7840b7320 2020-11-28T21:54:07.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.056+0000 7fb77bfff700 1 -- 172.21.15.162:0/2828389603 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb774013480 con 0x7fb7840b7320 2020-11-28T21:54:07.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.056+0000 7fb77bfff700 1 --2- 172.21.15.162:0/2828389603 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb7680553b0 0x7fb768057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:07.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.056+0000 7fb77bfff700 1 -- 172.21.15.162:0/2828389603 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb77406e6c0 con 0x7fb7840b7320 2020-11-28T21:54:07.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.056+0000 7fb782ffd700 1 --2- 172.21.15.162:0/2828389603 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb7680553b0 0x7fb768057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:07.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.056+0000 7fb782ffd700 1 --2- 172.21.15.162:0/2828389603 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb7680553b0 0x7fb768057870 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7fb77c00d000 tx=0x7fb77c009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:07.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.060+0000 7fb77bfff700 1 -- 172.21.15.162:0/2828389603 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb774044850 con 0x7fb7840b7320 2020-11-28T21:54:07.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.248+0000 7fb7898b0700 1 -- 172.21.15.162:0/2828389603 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb784057050 con 0x7fb7840b7320 2020-11-28T21:54:07.253 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.248+0000 7fb77bfff700 1 -- 172.21.15.162:0/2828389603 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+2502 (secure 0 0 0) 0x7fb774047e70 con 0x7fb7840b7320 2020-11-28T21:54:07.253 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:54:07.254 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":46,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6657,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.118:6833/1785592854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1785592854},{"type":"v1","addr":"172.21.15.118:6833","nonce":1785592854}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":44},{"gid":6850,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3533607073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3533607073},{"type":"v1","addr":"172.21.15.162:6835","nonce":3533607073}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:54:02.388344+0000","modified":"2020-11-28T21:54:03.391088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6859":{"gid":6859,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/3532128068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3532128068},{"type":"v1","addr":"172.21.15.162:6837","nonce":3532128068}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2020-11-28T21:54:07.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.252+0000 7fb7898b0700 1 -- 172.21.15.162:0/2828389603 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb7680553b0 msgr2=0x7fb768057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:07.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.252+0000 7fb7898b0700 1 --2- 172.21.15.162:0/2828389603 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb7680553b0 0x7fb768057870 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7fb77c00d000 tx=0x7fb77c009040).stop 2020-11-28T21:54:07.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.252+0000 7fb7898b0700 1 -- 172.21.15.162:0/2828389603 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7840b7320 msgr2=0x7fb784135e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:07.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.252+0000 7fb7898b0700 1 --2- 172.21.15.162:0/2828389603 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7840b7320 0x7fb784135e20 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 rx=0x7fb7840ad120 tx=0x7fb774008040).stop 2020-11-28T21:54:07.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.252+0000 7fb7898b0700 1 -- 172.21.15.162:0/2828389603 shutdown_connections 2020-11-28T21:54:07.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.252+0000 7fb7898b0700 1 --2- 172.21.15.162:0/2828389603 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb7680553b0 0x7fb768057870 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:07.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.252+0000 7fb7898b0700 1 --2- 172.21.15.162:0/2828389603 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb7840b19f0 0x7fb7840b5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:07.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.252+0000 7fb7898b0700 1 --2- 172.21.15.162:0/2828389603 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7840b7320 0x7fb784135e20 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:07.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.252+0000 7fb7898b0700 1 --2- 172.21.15.162:0/2828389603 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb7840b1f00 0x7fb784135930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:07.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.252+0000 7fb7898b0700 1 -- 172.21.15.162:0/2828389603 >> 172.21.15.162:0/2828389603 conn(0x7fb784062830 msgr2=0x7fb7840b3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:07.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.252+0000 7fb7898b0700 1 -- 172.21.15.162:0/2828389603 shutdown_connections 2020-11-28T21:54:07.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:07.252+0000 7fb7898b0700 1 -- 172.21.15.162:0/2828389603 wait complete. 2020-11-28T21:54:07.262 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 46 2020-11-28T21:54:07.574 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-28T21:54:02.388344+0000', 'modified': '2020-11-28T21:54:03.391088+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6859}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6859': {'gid': 6859, 'name': 'a', 'rank': 0, 'incarnation': 45, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6837/3532128068', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6836', 'nonce': 3532128068}, {'type': 'v1', 'addr': '172.21.15.162:6837', 'nonce': 3532128068}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-28T21:54:07.575 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-28T21:54:07.576 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-28T21:54:07.706 INFO:teuthology.orchestra.run.smithi162.stdout:{"cluster_fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","whoami":0,"id":6859,"want_state":"up:active","state":"up:active","rank_uptime":5.3024220580000003,"mdsmap_epoch":46,"osdmap_epoch":99,"osdmap_epoch_barrier":98,"uptime":10.682798361} 2020-11-28T21:54:07.706 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", "id": 6859, "mdsmap_epoch": 46, "osdmap_epoch": 99, "osdmap_epoch_barrier": 98, "rank_uptime": 5.302422058, "state": "up:active", "uptime": 10.682798361, "want_state": "up:active", "whoami": 0 } 2020-11-28T21:54:07.707 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-11-28T21:54:07.708 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-11-28T21:54:07.708 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi192.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-11-28T21:54:07.708 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:07.709 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:54:07.717 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:54:07.775 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:54:07.775 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:54:07.830 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:54:07.832 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:54:07.833 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:07.834 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:54:07.882 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-11-28T21:54:07.883 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:54:07.926 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:54:07.943 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:54:07.944 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:54:07.999 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:54:08.000 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:54:08.001 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:08.001 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:54:08.004 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:2020-11-28T21:54:07.998+0000 7f63d341b0c0 -1 init, newargv = 0x563a3cf4e490 newargc=9ceph-fuse[12299 2020-11-28T21:54:08.004 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:]: starting ceph client 2020-11-28T21:54:08.014 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:ceph-fuse[12299]: starting fuse 2020-11-28T21:54:08.785 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:54:08.789 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:54:08.792 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:54:08.795 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:54:09.111 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:54:09.127 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:54:09.127 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:54:09.183 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:54:09.184 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:54:09.185 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:09.186 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:54:09.234 INFO:teuthology.orchestra.run.smithi192.stdout:49 2020-11-28T21:54:09.235 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2020-11-28T21:54:09.236 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:09.237 INFO:teuthology.orchestra.run.smithi192:> sudo python3 -c ' 2020-11-28T21:54:09.238 INFO:teuthology.orchestra.run.smithi192:> import glob 2020-11-28T21:54:09.238 INFO:teuthology.orchestra.run.smithi192:> import re 2020-11-28T21:54:09.239 INFO:teuthology.orchestra.run.smithi192:> import os 2020-11-28T21:54:09.240 INFO:teuthology.orchestra.run.smithi192:> import subprocess 2020-11-28T21:54:09.241 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:54:09.242 INFO:teuthology.orchestra.run.smithi192:> def find_socket(client_name): 2020-11-28T21:54:09.243 INFO:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-11-28T21:54:09.243 INFO:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2020-11-28T21:54:09.244 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:54:09.245 INFO:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2020-11-28T21:54:09.245 INFO:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2020-11-28T21:54:09.246 INFO:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2020-11-28T21:54:09.247 INFO:teuthology.orchestra.run.smithi192:> return files[0] 2020-11-28T21:54:09.248 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:54:09.248 INFO:teuthology.orchestra.run.smithi192:> for f in files: 2020-11-28T21:54:09.249 INFO:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-11-28T21:54:09.250 INFO:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2020-11-28T21:54:09.250 INFO:teuthology.orchestra.run.smithi192:> return f 2020-11-28T21:54:09.251 INFO:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-11-28T21:54:09.252 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:54:09.253 INFO:teuthology.orchestra.run.smithi192:> print(find_socket("client.0")) 2020-11-28T21:54:09.254 INFO:teuthology.orchestra.run.smithi192:> ' 2020-11-28T21:54:09.325 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.0.12299.asok 2020-11-28T21:54:09.330 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.12299.asok 2020-11-28T21:54:09.331 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:09.331 INFO:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.12299.asok status 2020-11-28T21:54:09.427 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2020-11-28T21:54:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2020-11-28T21:54:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "a83a3572a957356977342f461361c1cd3ec06dbd", 2020-11-28T21:54:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 15.2.6-595-ga83a3572 (a83a3572a957356977342f461361c1cd3ec06dbd) octopus (stable)", 2020-11-28T21:54:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "0", 2020-11-28T21:54:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2020-11-28T21:54:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-11-28T21:54:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "12299", 2020-11-28T21:54:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2020-11-28T21:54:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:54:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2020-11-28T21:54:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2020-11-28T21:54:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 6801, 2020-11-28T21:54:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2020-11-28T21:54:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2020-11-28T21:54:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2020-11-28T21:54:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 6801 2020-11-28T21:54:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:54:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2020-11-28T21:54:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2020-11-28T21:54:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2020-11-28T21:54:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 1947164564 2020-11-28T21:54:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: } 2020-11-28T21:54:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:54:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2020-11-28T21:54:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2020-11-28T21:54:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2020-11-28T21:54:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 1947164564 2020-11-28T21:54:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:54:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.6801 172.21.15.192:0/1947164564", 2020-11-28T21:54:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "172.21.15.192:0/1947164564", 2020-11-28T21:54:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2020-11-28T21:54:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 46, 2020-11-28T21:54:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 99, 2020-11-28T21:54:09.435 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2020-11-28T21:54:09.435 INFO:teuthology.orchestra.run.smithi192.stdout: "blacklisted": false 2020-11-28T21:54:09.435 INFO:teuthology.orchestra.run.smithi192.stdout:} 2020-11-28T21:54:09.437 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:09.438 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:54:09.438 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:54:09.444 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2020-11-28T21:54:09.445 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-28T21:54:09.446 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:09.446 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:54:09.504 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:54:09.620 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.616+0000 7f684e8e9700 1 Processor -- start 2020-11-28T21:54:09.621 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.616+0000 7f684e8e9700 1 -- start start 2020-11-28T21:54:09.622 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.616+0000 7f684e8e9700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f68480b69a0 0x7f684805b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:09.622 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.616+0000 7f684e8e9700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f68480acac0 0x7f684814d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:09.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.616+0000 7f684e8e9700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f684805fe30 con 0x7f68480b0e90 2020-11-28T21:54:09.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.620+0000 7f684e8e9700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68480b6d80 con 0x7f68480acac0 2020-11-28T21:54:09.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.620+0000 7f684e8e9700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68480b1bf0 con 0x7f68480b69a0 2020-11-28T21:54:09.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.620+0000 7f6847fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f68480b69a0 0x7f684805b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:09.624 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.620+0000 7f6847fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f68480b69a0 0x7f684805b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42938/0 (socket says 172.21.15.162:42938) 2020-11-28T21:54:09.624 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.620+0000 7f6847fff700 1 -- 172.21.15.162:0/3768341152 learned_addr learned my addr 172.21.15.162:0/3768341152 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:09.624 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.620+0000 7f684ce86700 1 --2- 172.21.15.162:0/3768341152 >> v2:172.21.15.118:3300/0 conn(0x7f68480acac0 0x7f684814d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:09.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.620+0000 7f6847fff700 1 -- 172.21.15.162:0/3768341152 >> v2:172.21.15.118:3300/0 conn(0x7f68480acac0 msgr2=0x7f684814d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:09.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.620+0000 7f6847fff700 1 --2- 172.21.15.162:0/3768341152 >> v2:172.21.15.118:3300/0 conn(0x7f68480acac0 0x7f684814d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:09.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.620+0000 7f6847fff700 1 -- 172.21.15.162:0/3768341152 >> v1:172.21.15.118:6789/0 conn(0x7f68480b0e90 legacy=0x7f68480ac3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:09.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.620+0000 7f6847fff700 1 -- 172.21.15.162:0/3768341152 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6848060060 con 0x7f68480b69a0 2020-11-28T21:54:09.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.620+0000 7f6847fff700 1 --2- 172.21.15.162:0/3768341152 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68480b69a0 0x7f684805b970 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 rx=0x7f6838002f30 tx=0x7f6838006e80).ready entity=mon.1 client_cookie=f0c63bd809f09d8c server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:09.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.620+0000 7f6846ffd700 1 -- 172.21.15.162:0/3768341152 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68380097f0 con 0x7f68480b69a0 2020-11-28T21:54:09.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.620+0000 7f6846ffd700 1 -- 172.21.15.162:0/3768341152 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6838009950 con 0x7f68480b69a0 2020-11-28T21:54:09.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.620+0000 7f6846ffd700 1 -- 172.21.15.162:0/3768341152 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6838009cb0 con 0x7f68480b69a0 2020-11-28T21:54:09.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.620+0000 7f684e8e9700 1 -- 172.21.15.162:0/3768341152 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68480b69a0 msgr2=0x7f684805b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:09.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.620+0000 7f684e8e9700 1 --2- 172.21.15.162:0/3768341152 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68480b69a0 0x7f684805b970 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 rx=0x7f6838002f30 tx=0x7f6838006e80).stop 2020-11-28T21:54:09.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.620+0000 7f684e8e9700 1 -- 172.21.15.162:0/3768341152 shutdown_connections 2020-11-28T21:54:09.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.620+0000 7f684e8e9700 1 --2- 172.21.15.162:0/3768341152 >> v2:172.21.15.118:3300/0 conn(0x7f68480acac0 0x7f684814d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:09.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.620+0000 7f684e8e9700 1 --2- 172.21.15.162:0/3768341152 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68480b69a0 0x7f684805b970 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:09.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.620+0000 7f684e8e9700 1 -- 172.21.15.162:0/3768341152 >> 172.21.15.162:0/3768341152 conn(0x7f68480621c0 msgr2=0x7f68480b5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:09.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.620+0000 7f684e8e9700 1 -- 172.21.15.162:0/3768341152 shutdown_connections 2020-11-28T21:54:09.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.620+0000 7f684e8e9700 1 -- 172.21.15.162:0/3768341152 wait complete. 2020-11-28T21:54:09.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.624+0000 7f684e8e9700 1 Processor -- start 2020-11-28T21:54:09.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.624+0000 7f684e8e9700 1 -- start start 2020-11-28T21:54:09.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.624+0000 7f684e8e9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68480acac0 0x7f6848154d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:09.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.624+0000 7f684e8e9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f68480b0e90 0x7f6848065a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:09.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.624+0000 7f684e8e9700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f68480b69a0 0x7f6848066220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:09.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.624+0000 7f684e8e9700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68480a46e0 con 0x7f68480b0e90 2020-11-28T21:54:09.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.624+0000 7f684e8e9700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68480aa9b0 con 0x7f68480acac0 2020-11-28T21:54:09.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.624+0000 7f684e8e9700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68480aac10 con 0x7f68480b69a0 2020-11-28T21:54:09.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.624+0000 7f6847fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68480acac0 0x7f6848154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:09.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.624+0000 7f6847fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68480acac0 0x7f6848154d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42946/0 (socket says 172.21.15.162:42946) 2020-11-28T21:54:09.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.624+0000 7f6847fff700 1 -- 172.21.15.162:0/18308216 learned_addr learned my addr 172.21.15.162:0/18308216 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:09.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.624+0000 7f68477fe700 1 --2- 172.21.15.162:0/18308216 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f68480b0e90 0x7f6848065a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:09.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.624+0000 7f684ce86700 1 --2- 172.21.15.162:0/18308216 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f68480b69a0 0x7f6848066220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:09.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.624+0000 7f6847fff700 1 -- 172.21.15.162:0/18308216 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f68480b69a0 msgr2=0x7f6848066220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:09.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.624+0000 7f6847fff700 1 --2- 172.21.15.162:0/18308216 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f68480b69a0 0x7f6848066220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:09.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.624+0000 7f6847fff700 1 -- 172.21.15.162:0/18308216 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f68480b0e90 msgr2=0x7f6848065a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:09.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.624+0000 7f6847fff700 1 --2- 172.21.15.162:0/18308216 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f68480b0e90 0x7f6848065a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:09.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.624+0000 7f6847fff700 1 -- 172.21.15.162:0/18308216 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68480aaeb0 con 0x7f68480acac0 2020-11-28T21:54:09.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.624+0000 7f6847fff700 1 --2- 172.21.15.162:0/18308216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68480acac0 0x7f6848154d60 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 rx=0x7f68480a3430 tx=0x7f6838015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:09.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.624+0000 7f68457fa700 1 -- 172.21.15.162:0/18308216 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f683800bee0 con 0x7f68480acac0 2020-11-28T21:54:09.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.624+0000 7f68457fa700 1 -- 172.21.15.162:0/18308216 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6838011b40 con 0x7f68480acac0 2020-11-28T21:54:09.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.624+0000 7f684e8e9700 1 -- 172.21.15.162:0/18308216 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6848067220 con 0x7f68480acac0 2020-11-28T21:54:09.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.624+0000 7f68457fa700 1 -- 172.21.15.162:0/18308216 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6838011ea0 con 0x7f68480acac0 2020-11-28T21:54:09.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.624+0000 7f684e8e9700 1 -- 172.21.15.162:0/18308216 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6848066930 con 0x7f68480acac0 2020-11-28T21:54:09.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.628+0000 7f684e8e9700 1 -- 172.21.15.162:0/18308216 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68480673b0 con 0x7f68480acac0 2020-11-28T21:54:09.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.628+0000 7f68457fa700 1 -- 172.21.15.162:0/18308216 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6838025030 con 0x7f68480acac0 2020-11-28T21:54:09.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.628+0000 7f68457fa700 1 --2- 172.21.15.162:0/18308216 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f682c055500 0x7f682c0579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:09.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.628+0000 7f68457fa700 1 -- 172.21.15.162:0/18308216 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f683806ef20 con 0x7f68480acac0 2020-11-28T21:54:09.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.628+0000 7f68477fe700 1 --2- 172.21.15.162:0/18308216 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f682c055500 0x7f682c0579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:09.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.628+0000 7f68477fe700 1 --2- 172.21.15.162:0/18308216 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f682c055500 0x7f682c0579c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f683000d000 tx=0x7f6830007b60).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:09.639 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.636+0000 7f68457fa700 1 -- 172.21.15.162:0/18308216 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6838045220 con 0x7f68480acac0 2020-11-28T21:54:09.783 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.780+0000 7f684e8e9700 1 -- 172.21.15.162:0/18308216 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6848066930 con 0x7f68480acac0 2020-11-28T21:54:09.784 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.780+0000 7f68457fa700 1 -- 172.21.15.162:0/18308216 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+16137 (secure 0 0 0) 0x7f6848066930 con 0x7f68480acac0 2020-11-28T21:54:09.785 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:54:09.785 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":99,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:54:03.391732+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:53:59.925390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2020-11-28T21:54:01.236032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:54:09.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.784+0000 7f684e8e9700 1 -- 172.21.15.162:0/18308216 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f682c055500 msgr2=0x7f682c0579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:09.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.784+0000 7f684e8e9700 1 --2- 172.21.15.162:0/18308216 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f682c055500 0x7f682c0579c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f683000d000 tx=0x7f6830007b60).stop 2020-11-28T21:54:09.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.784+0000 7f684e8e9700 1 -- 172.21.15.162:0/18308216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68480acac0 msgr2=0x7f6848154d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:09.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.784+0000 7f684e8e9700 1 --2- 172.21.15.162:0/18308216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68480acac0 0x7f6848154d60 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 rx=0x7f68480a3430 tx=0x7f6838015000).stop 2020-11-28T21:54:09.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.784+0000 7f684e8e9700 1 -- 172.21.15.162:0/18308216 shutdown_connections 2020-11-28T21:54:09.788 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.784+0000 7f684e8e9700 1 --2- 172.21.15.162:0/18308216 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f682c055500 0x7f682c0579c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:09.788 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.784+0000 7f684e8e9700 1 --2- 172.21.15.162:0/18308216 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f68480b69a0 0x7f6848066220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:09.788 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.784+0000 7f684e8e9700 1 --2- 172.21.15.162:0/18308216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68480acac0 0x7f6848154d60 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:09.788 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.784+0000 7f684e8e9700 1 --2- 172.21.15.162:0/18308216 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f68480b0e90 0x7f6848065a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:09.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.784+0000 7f684e8e9700 1 -- 172.21.15.162:0/18308216 >> 172.21.15.162:0/18308216 conn(0x7f68480621c0 msgr2=0x7f68480ac200 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:09.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.784+0000 7f684e8e9700 1 -- 172.21.15.162:0/18308216 shutdown_connections 2020-11-28T21:54:09.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.784+0000 7f684e8e9700 1 -- 172.21.15.162:0/18308216 wait complete. 2020-11-28T21:54:09.799 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-28T21:53:59.925390+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2020-11-28T21:54:01.236032+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-28T21:54:09.800 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:54:09.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.916+0000 7f4a39874700 1 Processor -- start 2020-11-28T21:54:09.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.916+0000 7f4a39874700 1 -- start start 2020-11-28T21:54:09.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.916+0000 7f4a39874700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f4a34061280 0x7f4a34061f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:09.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.916+0000 7f4a39874700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a34142bb0 con 0x7f4a341437c0 2020-11-28T21:54:09.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.916+0000 7f4a39874700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a34142de0 con 0x7f4a3413fc50 2020-11-28T21:54:09.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.916+0000 7f4a39874700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a34143080 con 0x7f4a34061280 2020-11-28T21:54:09.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.916+0000 7f4a39073700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f4a3413fc50 0x7f4a340ae4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:49800/0 (socket says 172.21.15.162:49800) 2020-11-28T21:54:09.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.916+0000 7f4a39073700 1 -- 172.21.15.162:0/1330318779 learned_addr learned my addr 172.21.15.162:0/1330318779 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:09.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.916+0000 7f4a38872700 1 --2- 172.21.15.162:0/1330318779 >> v2:172.21.15.118:3300/0 conn(0x7f4a34061280 0x7f4a34061f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:09.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.916+0000 7f4a337fe700 1 -- 172.21.15.162:0/1330318779 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3925811220 0 0) 0x7f4a34142bb0 con 0x7f4a341437c0 2020-11-28T21:54:09.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.916+0000 7f4a337fe700 1 -- 172.21.15.162:0/1330318779 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a20003230 con 0x7f4a341437c0 2020-11-28T21:54:09.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.916+0000 7f4a38872700 1 -- 172.21.15.162:0/1330318779 >> v1:172.21.15.162:6789/0 conn(0x7f4a3413fc50 legacy=0x7f4a340ae4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:09.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.916+0000 7f4a38872700 1 -- 172.21.15.162:0/1330318779 >> v1:172.21.15.118:6789/0 conn(0x7f4a341437c0 legacy=0x7f4a3413f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:09.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.916+0000 7f4a38872700 1 -- 172.21.15.162:0/1330318779 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a34143320 con 0x7f4a34061280 2020-11-28T21:54:09.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.920+0000 7f4a38872700 1 --2- 172.21.15.162:0/1330318779 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4a34061280 0x7f4a34061f20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f4a28002f30 tx=0x7f4a28006dc0).ready entity=mon.0 client_cookie=6d8d96671ee5e764 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:09.931 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.924+0000 7f4a337fe700 1 -- 172.21.15.162:0/1330318779 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a28009800 con 0x7f4a34061280 2020-11-28T21:54:09.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.924+0000 7f4a337fe700 1 -- 172.21.15.162:0/1330318779 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a28009960 con 0x7f4a34061280 2020-11-28T21:54:09.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.924+0000 7f4a337fe700 1 -- 172.21.15.162:0/1330318779 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a28009cc0 con 0x7f4a34061280 2020-11-28T21:54:09.933 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.924+0000 7f4a39874700 1 -- 172.21.15.162:0/1330318779 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4a34061280 msgr2=0x7f4a34061f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:09.933 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.924+0000 7f4a39874700 1 --2- 172.21.15.162:0/1330318779 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4a34061280 0x7f4a34061f20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f4a28002f30 tx=0x7f4a28006dc0).stop 2020-11-28T21:54:09.933 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.924+0000 7f4a39874700 1 -- 172.21.15.162:0/1330318779 shutdown_connections 2020-11-28T21:54:09.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.924+0000 7f4a39874700 1 --2- 172.21.15.162:0/1330318779 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4a34061280 0x7f4a34061f20 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:09.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.924+0000 7f4a39874700 1 -- 172.21.15.162:0/1330318779 >> 172.21.15.162:0/1330318779 conn(0x7f4a340b5de0 msgr2=0x7f4a340b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:09.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.924+0000 7f4a39874700 1 -- 172.21.15.162:0/1330318779 shutdown_connections 2020-11-28T21:54:09.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.924+0000 7f4a39874700 1 -- 172.21.15.162:0/1330318779 wait complete. 2020-11-28T21:54:09.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.928+0000 7f4a39874700 1 Processor -- start 2020-11-28T21:54:09.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.928+0000 7f4a39874700 1 -- start start 2020-11-28T21:54:09.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.928+0000 7f4a39874700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4a34061280 0x7f4a340b4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:09.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.928+0000 7f4a39874700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a3413fc50 0x7f4a3413c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:09.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.928+0000 7f4a39874700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4a34065fb0 0x7f4a3413c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:09.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.928+0000 7f4a39874700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a34066c30 con 0x7f4a34061280 2020-11-28T21:54:09.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.928+0000 7f4a39874700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a34066e90 con 0x7f4a3413fc50 2020-11-28T21:54:09.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.928+0000 7f4a39874700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a34067130 con 0x7f4a34065fb0 2020-11-28T21:54:09.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.928+0000 7f4a33fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a3413fc50 0x7f4a3413c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:09.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.928+0000 7f4a33fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a3413fc50 0x7f4a3413c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42962/0 (socket says 172.21.15.162:42962) 2020-11-28T21:54:09.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.928+0000 7f4a33fff700 1 -- 172.21.15.162:0/2350992663 learned_addr learned my addr 172.21.15.162:0/2350992663 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:09.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.928+0000 7f4a39073700 1 --2- 172.21.15.162:0/2350992663 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4a34065fb0 0x7f4a3413c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:09.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.928+0000 7f4a38872700 1 --2- 172.21.15.162:0/2350992663 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4a34061280 0x7f4a340b4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:09.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.928+0000 7f4a33fff700 1 -- 172.21.15.162:0/2350992663 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4a34065fb0 msgr2=0x7f4a3413c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:09.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.928+0000 7f4a33fff700 1 --2- 172.21.15.162:0/2350992663 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4a34065fb0 0x7f4a3413c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:09.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.928+0000 7f4a33fff700 1 -- 172.21.15.162:0/2350992663 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4a34061280 msgr2=0x7f4a340b4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:09.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.928+0000 7f4a33fff700 1 --2- 172.21.15.162:0/2350992663 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4a34061280 0x7f4a340b4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:09.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.928+0000 7f4a33fff700 1 -- 172.21.15.162:0/2350992663 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a340673d0 con 0x7f4a3413fc50 2020-11-28T21:54:09.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.928+0000 7f4a33fff700 1 --2- 172.21.15.162:0/2350992663 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a3413fc50 0x7f4a3413c050 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 rx=0x7f4a240010d0 tx=0x7f4a24007a60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:09.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.928+0000 7f4a31ffb700 1 -- 172.21.15.162:0/2350992663 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a24013410 con 0x7f4a3413fc50 2020-11-28T21:54:09.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.932+0000 7f4a31ffb700 1 -- 172.21.15.162:0/2350992663 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a24013570 con 0x7f4a3413fc50 2020-11-28T21:54:09.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.932+0000 7f4a39874700 1 -- 172.21.15.162:0/2350992663 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a3413ca30 con 0x7f4a3413fc50 2020-11-28T21:54:09.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.932+0000 7f4a31ffb700 1 -- 172.21.15.162:0/2350992663 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a240138d0 con 0x7f4a3413fc50 2020-11-28T21:54:09.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.932+0000 7f4a39874700 1 -- 172.21.15.162:0/2350992663 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a340aa6a0 con 0x7f4a3413fc50 2020-11-28T21:54:09.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.932+0000 7f4a39874700 1 -- 172.21.15.162:0/2350992663 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a34057050 con 0x7f4a3413fc50 2020-11-28T21:54:09.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.932+0000 7f4a31ffb700 1 -- 172.21.15.162:0/2350992663 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4a24010030 con 0x7f4a3413fc50 2020-11-28T21:54:09.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.932+0000 7f4a31ffb700 1 --2- 172.21.15.162:0/2350992663 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4a140553b0 0x7f4a14057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:09.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.932+0000 7f4a31ffb700 1 -- 172.21.15.162:0/2350992663 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4a2406dc60 con 0x7f4a3413fc50 2020-11-28T21:54:09.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.932+0000 7f4a38872700 1 --2- 172.21.15.162:0/2350992663 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4a140553b0 0x7f4a14057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:09.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.932+0000 7f4a38872700 1 --2- 172.21.15.162:0/2350992663 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4a140553b0 0x7f4a14057870 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f4a2801a000 tx=0x7f4a28015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:09.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:09.940+0000 7f4a31ffb700 1 -- 172.21.15.162:0/2350992663 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4a24043dd0 con 0x7f4a3413fc50 2020-11-28T21:54:10.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.084+0000 7f4a39874700 1 -- 172.21.15.162:0/2350992663 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f4a34069c00 con 0x7f4a3413fc50 2020-11-28T21:54:10.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.084+0000 7f4a31ffb700 1 -- 172.21.15.162:0/2350992663 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v99) v1 ==== 110+0+10 (secure 0 0 0) 0x7f4a240473f0 con 0x7f4a3413fc50 2020-11-28T21:54:10.088 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:54:10.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.088+0000 7f4a39874700 1 -- 172.21.15.162:0/2350992663 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4a140553b0 msgr2=0x7f4a14057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:10.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.088+0000 7f4a39874700 1 --2- 172.21.15.162:0/2350992663 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4a140553b0 0x7f4a14057870 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f4a2801a000 tx=0x7f4a28015000).stop 2020-11-28T21:54:10.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.088+0000 7f4a39874700 1 -- 172.21.15.162:0/2350992663 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a3413fc50 msgr2=0x7f4a3413c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:10.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.088+0000 7f4a39874700 1 --2- 172.21.15.162:0/2350992663 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a3413fc50 0x7f4a3413c050 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 rx=0x7f4a240010d0 tx=0x7f4a24007a60).stop 2020-11-28T21:54:10.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.088+0000 7f4a39874700 1 -- 172.21.15.162:0/2350992663 shutdown_connections 2020-11-28T21:54:10.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.088+0000 7f4a39874700 1 --2- 172.21.15.162:0/2350992663 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4a140553b0 0x7f4a14057870 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:10.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.088+0000 7f4a39874700 1 --2- 172.21.15.162:0/2350992663 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4a34065fb0 0x7f4a3413c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:10.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.088+0000 7f4a39874700 1 --2- 172.21.15.162:0/2350992663 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4a34061280 0x7f4a340b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:10.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.088+0000 7f4a39874700 1 --2- 172.21.15.162:0/2350992663 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a3413fc50 0x7f4a3413c050 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:10.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.088+0000 7f4a39874700 1 -- 172.21.15.162:0/2350992663 >> 172.21.15.162:0/2350992663 conn(0x7f4a340b5de0 msgr2=0x7f4a340b5450 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:10.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.088+0000 7f4a39874700 1 -- 172.21.15.162:0/2350992663 shutdown_connections 2020-11-28T21:54:10.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.088+0000 7f4a39874700 1 -- 172.21.15.162:0/2350992663 wait complete. 2020-11-28T21:54:10.101 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:54:10.217 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.212+0000 7fe217fff700 1 Processor -- start 2020-11-28T21:54:10.217 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.212+0000 7fe217fff700 1 -- start start 2020-11-28T21:54:10.224 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.212+0000 7fe217fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe2180b3950 0x7fe2180b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:10.224 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.212+0000 7fe217fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe2180b18c0 0x7fe21813dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:10.225 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.212+0000 7fe217fff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2180b1f50 con 0x7fe2180b44e0 2020-11-28T21:54:10.225 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.212+0000 7fe217fff700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2180aab50 con 0x7fe2180b18c0 2020-11-28T21:54:10.225 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.212+0000 7fe217fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe218143e90 con 0x7fe2180b3950 2020-11-28T21:54:10.225 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.216+0000 7fe2167fc700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe2180b18c0 0x7fe21813dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:10.226 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.216+0000 7fe2167fc700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe2180b18c0 0x7fe21813dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42968/0 (socket says 172.21.15.162:42968) 2020-11-28T21:54:10.226 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.216+0000 7fe2167fc700 1 -- 172.21.15.162:0/894981704 learned_addr learned my addr 172.21.15.162:0/894981704 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:10.226 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.216+0000 7fe216ffd700 1 --2- 172.21.15.162:0/894981704 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe2180b3950 0x7fe2180b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:10.226 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.216+0000 7fe2167fc700 1 -- 172.21.15.162:0/894981704 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe2180b3950 msgr2=0x7fe2180b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:10.227 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.216+0000 7fe2167fc700 1 --2- 172.21.15.162:0/894981704 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe2180b3950 0x7fe2180b3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:10.227 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.216+0000 7fe2167fc700 1 -- 172.21.15.162:0/894981704 >> v1:172.21.15.118:6789/0 conn(0x7fe2180b44e0 legacy=0x7fe2180b2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:10.227 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.216+0000 7fe2167fc700 1 -- 172.21.15.162:0/894981704 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2180aa8b0 con 0x7fe2180b18c0 2020-11-28T21:54:10.227 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.216+0000 7fe2167fc700 1 --2- 172.21.15.162:0/894981704 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2180b18c0 0x7fe21813dd60 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 rx=0x7fe208002f30 tx=0x7fe208006e80).ready entity=mon.1 client_cookie=f287900ff9eb9f89 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:10.227 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.216+0000 7fe215ffb700 1 -- 172.21.15.162:0/894981704 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2080097f0 con 0x7fe2180b18c0 2020-11-28T21:54:10.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.216+0000 7fe215ffb700 1 -- 172.21.15.162:0/894981704 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe208009950 con 0x7fe2180b18c0 2020-11-28T21:54:10.229 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.216+0000 7fe215ffb700 1 -- 172.21.15.162:0/894981704 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe208009cb0 con 0x7fe2180b18c0 2020-11-28T21:54:10.229 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.216+0000 7fe217fff700 1 -- 172.21.15.162:0/894981704 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2180b18c0 msgr2=0x7fe21813dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:10.229 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.216+0000 7fe217fff700 1 --2- 172.21.15.162:0/894981704 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2180b18c0 0x7fe21813dd60 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 rx=0x7fe208002f30 tx=0x7fe208006e80).stop 2020-11-28T21:54:10.231 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.216+0000 7fe217fff700 1 -- 172.21.15.162:0/894981704 shutdown_connections 2020-11-28T21:54:10.232 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.216+0000 7fe217fff700 1 --2- 172.21.15.162:0/894981704 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe2180b3950 0x7fe2180b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:10.234 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.216+0000 7fe217fff700 1 --2- 172.21.15.162:0/894981704 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2180b18c0 0x7fe21813dd60 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:10.234 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.216+0000 7fe217fff700 1 -- 172.21.15.162:0/894981704 >> 172.21.15.162:0/894981704 conn(0x7fe218062630 msgr2=0x7fe218062a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:10.234 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.216+0000 7fe217fff700 1 -- 172.21.15.162:0/894981704 shutdown_connections 2020-11-28T21:54:10.235 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.216+0000 7fe217fff700 1 -- 172.21.15.162:0/894981704 wait complete. 2020-11-28T21:54:10.235 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.216+0000 7fe217fff700 1 Processor -- start 2020-11-28T21:54:10.235 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.216+0000 7fe217fff700 1 -- start start 2020-11-28T21:54:10.238 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.220+0000 7fe217fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe2180b18c0 0x7fe2180acfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:10.238 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.220+0000 7fe217fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2180b3950 0x7fe21814f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:10.238 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.220+0000 7fe217fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2180b44e0 0x7fe21814fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:10.238 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.220+0000 7fe217fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2180abdb0 con 0x7fe2180b3950 2020-11-28T21:54:10.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.220+0000 7fe217fff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2180ac590 con 0x7fe2180b44e0 2020-11-28T21:54:10.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.220+0000 7fe217fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe21813f570 con 0x7fe2180b18c0 2020-11-28T21:54:10.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.220+0000 7fe2177fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2180b44e0 0x7fe21814fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:10.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.220+0000 7fe2177fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2180b44e0 0x7fe21814fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42976/0 (socket says 172.21.15.162:42976) 2020-11-28T21:54:10.241 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.220+0000 7fe2177fe700 1 -- 172.21.15.162:0/1952022447 learned_addr learned my addr 172.21.15.162:0/1952022447 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:10.241 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.220+0000 7fe216ffd700 1 --2- 172.21.15.162:0/1952022447 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe2180b18c0 0x7fe2180acfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:10.241 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.220+0000 7fe2167fc700 1 --2- 172.21.15.162:0/1952022447 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2180b3950 0x7fe21814f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:10.241 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.220+0000 7fe2177fe700 1 -- 172.21.15.162:0/1952022447 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe2180b18c0 msgr2=0x7fe2180acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:10.241 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.220+0000 7fe2177fe700 1 --2- 172.21.15.162:0/1952022447 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe2180b18c0 0x7fe2180acfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:10.242 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.220+0000 7fe2177fe700 1 -- 172.21.15.162:0/1952022447 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2180b3950 msgr2=0x7fe21814f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:10.242 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.220+0000 7fe2177fe700 1 --2- 172.21.15.162:0/1952022447 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2180b3950 0x7fe21814f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:10.242 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.220+0000 7fe2177fe700 1 -- 172.21.15.162:0/1952022447 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2180ab870 con 0x7fe2180b44e0 2020-11-28T21:54:10.242 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.220+0000 7fe2177fe700 1 --2- 172.21.15.162:0/1952022447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2180b44e0 0x7fe21814fd40 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 rx=0x7fe2180b42a0 tx=0x7fe210007f40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:10.243 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.220+0000 7fe20ffff700 1 -- 172.21.15.162:0/1952022447 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe21000a5a0 con 0x7fe2180b44e0 2020-11-28T21:54:10.243 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.220+0000 7fe20ffff700 1 -- 172.21.15.162:0/1952022447 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe21000a700 con 0x7fe2180b44e0 2020-11-28T21:54:10.243 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.220+0000 7fe217fff700 1 -- 172.21.15.162:0/1952022447 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2180abb10 con 0x7fe2180b44e0 2020-11-28T21:54:10.243 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.220+0000 7fe20ffff700 1 -- 172.21.15.162:0/1952022447 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe21000aa60 con 0x7fe2180b44e0 2020-11-28T21:54:10.243 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.220+0000 7fe217fff700 1 -- 172.21.15.162:0/1952022447 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe218150230 con 0x7fe2180b44e0 2020-11-28T21:54:10.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.220+0000 7fe217fff700 1 -- 172.21.15.162:0/1952022447 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe218057050 con 0x7fe2180b44e0 2020-11-28T21:54:10.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.224+0000 7fe20ffff700 1 -- 172.21.15.162:0/1952022447 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe210019030 con 0x7fe2180b44e0 2020-11-28T21:54:10.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.224+0000 7fe20ffff700 1 --2- 172.21.15.162:0/1952022447 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe1fc055350 0x7fe1fc057810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:10.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.224+0000 7fe20ffff700 1 -- 172.21.15.162:0/1952022447 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe21006d670 con 0x7fe2180b44e0 2020-11-28T21:54:10.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.224+0000 7fe216ffd700 1 --2- 172.21.15.162:0/1952022447 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe1fc055350 0x7fe1fc057810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:10.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.224+0000 7fe216ffd700 1 --2- 172.21.15.162:0/1952022447 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe1fc055350 0x7fe1fc057810 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7fe200008000 tx=0x7fe200008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:10.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.228+0000 7fe20ffff700 1 -- 172.21.15.162:0/1952022447 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe2100438d0 con 0x7fe2180b44e0 2020-11-28T21:54:10.372 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.368+0000 7fe217fff700 1 -- 172.21.15.162:0/1952022447 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe2180ac720 con 0x7fe2180b44e0 2020-11-28T21:54:10.373 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.368+0000 7fe20ffff700 1 -- 172.21.15.162:0/1952022447 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v99) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe210046ef0 con 0x7fe2180b44e0 2020-11-28T21:54:10.373 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:54:10.375 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.372+0000 7fe217fff700 1 -- 172.21.15.162:0/1952022447 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe1fc055350 msgr2=0x7fe1fc057810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:10.375 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.372+0000 7fe217fff700 1 --2- 172.21.15.162:0/1952022447 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe1fc055350 0x7fe1fc057810 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7fe200008000 tx=0x7fe200008040).stop 2020-11-28T21:54:10.375 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.372+0000 7fe217fff700 1 -- 172.21.15.162:0/1952022447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2180b44e0 msgr2=0x7fe21814fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:10.375 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.372+0000 7fe217fff700 1 --2- 172.21.15.162:0/1952022447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2180b44e0 0x7fe21814fd40 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 rx=0x7fe2180b42a0 tx=0x7fe210007f40).stop 2020-11-28T21:54:10.376 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.372+0000 7fe217fff700 1 -- 172.21.15.162:0/1952022447 shutdown_connections 2020-11-28T21:54:10.376 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.372+0000 7fe217fff700 1 --2- 172.21.15.162:0/1952022447 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe1fc055350 0x7fe1fc057810 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:10.376 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.372+0000 7fe217fff700 1 --2- 172.21.15.162:0/1952022447 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe2180b18c0 0x7fe2180acfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:10.376 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.372+0000 7fe217fff700 1 --2- 172.21.15.162:0/1952022447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2180b44e0 0x7fe21814fd40 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:10.376 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.372+0000 7fe217fff700 1 --2- 172.21.15.162:0/1952022447 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2180b3950 0x7fe21814f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:10.377 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.372+0000 7fe217fff700 1 -- 172.21.15.162:0/1952022447 >> 172.21.15.162:0/1952022447 conn(0x7fe218062630 msgr2=0x7fe218142bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:10.377 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.372+0000 7fe217fff700 1 -- 172.21.15.162:0/1952022447 shutdown_connections 2020-11-28T21:54:10.377 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.372+0000 7fe217fff700 1 -- 172.21.15.162:0/1952022447 wait complete. 2020-11-28T21:54:10.386 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-28T21:54:10.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.500+0000 7f0cecd33700 1 Processor -- start 2020-11-28T21:54:10.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.500+0000 7f0cecd33700 1 -- start start 2020-11-28T21:54:10.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.500+0000 7f0cecd33700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0ce80ac500 0x7f0ce805fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:10.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.500+0000 7f0cecd33700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f0ce8141b80 0x7f0ce813f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:10.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.500+0000 7f0cecd33700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ce80ac8e0 con 0x7f0ce80ae9d0 2020-11-28T21:54:10.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.500+0000 7f0cecd33700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce813ccf0 con 0x7f0ce8141b80 2020-11-28T21:54:10.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.500+0000 7f0cecd33700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce80b25c0 con 0x7f0ce80ac500 2020-11-28T21:54:10.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.500+0000 7f0ce6ffd700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f0ce80ae9d0 0x7f0ce80b1eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:49824/0 (socket says 172.21.15.162:49824) 2020-11-28T21:54:10.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.500+0000 7f0ce6ffd700 1 -- 172.21.15.162:0/3556396553 learned_addr learned my addr 172.21.15.162:0/3556396553 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:10.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.500+0000 7f0ce7fff700 1 --2- 172.21.15.162:0/3556396553 >> v2:172.21.15.118:3300/0 conn(0x7f0ce8141b80 0x7f0ce813f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:10.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.500+0000 7f0ce77fe700 1 --2- 172.21.15.162:0/3556396553 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0ce80ac500 0x7f0ce805fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:10.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.500+0000 7f0ce67fc700 1 -- 172.21.15.162:0/3556396553 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1565084132 0 0) 0x7f0ce80ac8e0 con 0x7f0ce80ae9d0 2020-11-28T21:54:10.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.500+0000 7f0ce67fc700 1 -- 172.21.15.162:0/3556396553 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cd4003230 con 0x7f0ce80ae9d0 2020-11-28T21:54:10.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.504+0000 7f0ce7fff700 1 -- 172.21.15.162:0/3556396553 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0ce80ac500 msgr2=0x7f0ce805fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:10.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.504+0000 7f0ce7fff700 1 --2- 172.21.15.162:0/3556396553 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0ce80ac500 0x7f0ce805fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:10.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.504+0000 7f0ce7fff700 1 -- 172.21.15.162:0/3556396553 >> v1:172.21.15.162:6789/0 conn(0x7f0ce80ae9d0 legacy=0x7f0ce80b1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:10.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.504+0000 7f0ce7fff700 1 -- 172.21.15.162:0/3556396553 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce8060690 con 0x7f0ce8141b80 2020-11-28T21:54:10.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.504+0000 7f0ce7fff700 1 --2- 172.21.15.162:0/3556396553 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0ce8141b80 0x7f0ce813f560 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7f0ce00032b0 tx=0x7f0ce00091a0).ready entity=mon.0 client_cookie=86c9a4932583214b server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:10.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.504+0000 7f0ce67fc700 1 -- 172.21.15.162:0/3556396553 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ce000b940 con 0x7f0ce8141b80 2020-11-28T21:54:10.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.504+0000 7f0ce67fc700 1 -- 172.21.15.162:0/3556396553 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce000baa0 con 0x7f0ce8141b80 2020-11-28T21:54:10.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.504+0000 7f0ce67fc700 1 -- 172.21.15.162:0/3556396553 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ce000bc10 con 0x7f0ce8141b80 2020-11-28T21:54:10.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.504+0000 7f0cecd33700 1 -- 172.21.15.162:0/3556396553 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0ce8141b80 msgr2=0x7f0ce813f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:10.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.504+0000 7f0cecd33700 1 --2- 172.21.15.162:0/3556396553 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0ce8141b80 0x7f0ce813f560 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7f0ce00032b0 tx=0x7f0ce00091a0).stop 2020-11-28T21:54:10.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.504+0000 7f0cecd33700 1 -- 172.21.15.162:0/3556396553 shutdown_connections 2020-11-28T21:54:10.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.504+0000 7f0cecd33700 1 --2- 172.21.15.162:0/3556396553 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0ce8141b80 0x7f0ce813f560 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:10.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.504+0000 7f0cecd33700 1 --2- 172.21.15.162:0/3556396553 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0ce80ac500 0x7f0ce805fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:10.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.504+0000 7f0cecd33700 1 -- 172.21.15.162:0/3556396553 >> 172.21.15.162:0/3556396553 conn(0x7f0ce80b57d0 msgr2=0x7f0ce80b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:10.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.504+0000 7f0cecd33700 1 -- 172.21.15.162:0/3556396553 shutdown_connections 2020-11-28T21:54:10.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.504+0000 7f0cecd33700 1 -- 172.21.15.162:0/3556396553 wait complete. 2020-11-28T21:54:10.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.504+0000 7f0cecd33700 1 Processor -- start 2020-11-28T21:54:10.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.504+0000 7f0cecd33700 1 -- start start 2020-11-28T21:54:10.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.508+0000 7f0cecd33700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0ce80ac500 0x7f0ce81446f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:10.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.508+0000 7f0cecd33700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ce80ae9d0 0x7f0ce813d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:10.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.508+0000 7f0cecd33700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0ce8141b80 0x7f0ce8154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:10.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.508+0000 7f0cecd33700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce813ea30 con 0x7f0ce80ac500 2020-11-28T21:54:10.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.508+0000 7f0cecd33700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce813ec90 con 0x7f0ce80ae9d0 2020-11-28T21:54:10.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.508+0000 7f0cecd33700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce813ce50 con 0x7f0ce8141b80 2020-11-28T21:54:10.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.508+0000 7f0ce6ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ce80ae9d0 0x7f0ce813d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:10.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.508+0000 7f0ce6ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ce80ae9d0 0x7f0ce813d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42990/0 (socket says 172.21.15.162:42990) 2020-11-28T21:54:10.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.508+0000 7f0ce6ffd700 1 -- 172.21.15.162:0/2771895468 learned_addr learned my addr 172.21.15.162:0/2771895468 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:10.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.508+0000 7f0ce77fe700 1 --2- 172.21.15.162:0/2771895468 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0ce80ac500 0x7f0ce81446f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:10.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.508+0000 7f0ce7fff700 1 --2- 172.21.15.162:0/2771895468 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0ce8141b80 0x7f0ce8154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:10.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.508+0000 7f0ce6ffd700 1 -- 172.21.15.162:0/2771895468 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0ce8141b80 msgr2=0x7f0ce8154d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:10.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.508+0000 7f0ce6ffd700 1 --2- 172.21.15.162:0/2771895468 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0ce8141b80 0x7f0ce8154d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:10.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.508+0000 7f0ce6ffd700 1 -- 172.21.15.162:0/2771895468 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0ce80ac500 msgr2=0x7f0ce81446f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:10.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.508+0000 7f0ce6ffd700 1 --2- 172.21.15.162:0/2771895468 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0ce80ac500 0x7f0ce81446f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:10.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.508+0000 7f0ce6ffd700 1 -- 172.21.15.162:0/2771895468 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce813d0f0 con 0x7f0ce80ae9d0 2020-11-28T21:54:10.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.508+0000 7f0ce6ffd700 1 --2- 172.21.15.162:0/2771895468 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ce80ae9d0 0x7f0ce813d590 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 rx=0x7f0cdc0010d0 tx=0x7f0cdc007aa0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:10.521 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.508+0000 7f0ce4ff9700 1 -- 172.21.15.162:0/2771895468 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cdc0072e0 con 0x7f0ce80ae9d0 2020-11-28T21:54:10.521 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.508+0000 7f0ce4ff9700 1 -- 172.21.15.162:0/2771895468 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cdc007440 con 0x7f0ce80ae9d0 2020-11-28T21:54:10.521 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.508+0000 7f0cecd33700 1 -- 172.21.15.162:0/2771895468 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ce8144be0 con 0x7f0ce80ae9d0 2020-11-28T21:54:10.521 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.508+0000 7f0ce4ff9700 1 -- 172.21.15.162:0/2771895468 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cdc0057d0 con 0x7f0ce80ae9d0 2020-11-28T21:54:10.521 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.508+0000 7f0cecd33700 1 -- 172.21.15.162:0/2771895468 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce81453e0 con 0x7f0ce80ae9d0 2020-11-28T21:54:10.522 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.512+0000 7f0cecd33700 1 -- 172.21.15.162:0/2771895468 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ce813d280 con 0x7f0ce80ae9d0 2020-11-28T21:54:10.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.512+0000 7f0ce4ff9700 1 -- 172.21.15.162:0/2771895468 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0cdc010070 con 0x7f0ce80ae9d0 2020-11-28T21:54:10.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.512+0000 7f0ce4ff9700 1 --2- 172.21.15.162:0/2771895468 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0cc8055330 0x7f0cc80577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:10.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.512+0000 7f0ce4ff9700 1 -- 172.21.15.162:0/2771895468 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0cdc06d720 con 0x7f0ce80ae9d0 2020-11-28T21:54:10.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.512+0000 7f0ce77fe700 1 --2- 172.21.15.162:0/2771895468 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0cc8055330 0x7f0cc80577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:10.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.512+0000 7f0ce77fe700 1 --2- 172.21.15.162:0/2771895468 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0cc8055330 0x7f0cc80577f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f0cd800b000 tx=0x7f0cd8006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:10.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.520+0000 7f0ce4ff9700 1 -- 172.21.15.162:0/2771895468 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0cdc043980 con 0x7f0ce80ae9d0 2020-11-28T21:54:10.667 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.664+0000 7f0cecd33700 1 -- 172.21.15.162:0/2771895468 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0ce8057050 con 0x7f0ce80ae9d0 2020-11-28T21:54:10.667 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.664+0000 7f0ce4ff9700 1 -- 172.21.15.162:0/2771895468 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) v1 ==== 104+0+10 (secure 0 0 0) 0x7f0cdc046fa0 con 0x7f0ce80ae9d0 2020-11-28T21:54:10.668 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:54:10.669 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.664+0000 7f0cecd33700 1 -- 172.21.15.162:0/2771895468 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0cc8055330 msgr2=0x7f0cc80577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:10.670 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.664+0000 7f0cecd33700 1 --2- 172.21.15.162:0/2771895468 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0cc8055330 0x7f0cc80577f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f0cd800b000 tx=0x7f0cd8006dc0).stop 2020-11-28T21:54:10.670 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.664+0000 7f0cecd33700 1 -- 172.21.15.162:0/2771895468 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ce80ae9d0 msgr2=0x7f0ce813d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:10.670 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.664+0000 7f0cecd33700 1 --2- 172.21.15.162:0/2771895468 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ce80ae9d0 0x7f0ce813d590 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 rx=0x7f0cdc0010d0 tx=0x7f0cdc007aa0).stop 2020-11-28T21:54:10.670 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.664+0000 7f0cecd33700 1 -- 172.21.15.162:0/2771895468 shutdown_connections 2020-11-28T21:54:10.671 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.664+0000 7f0cecd33700 1 --2- 172.21.15.162:0/2771895468 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0cc8055330 0x7f0cc80577f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:10.671 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.664+0000 7f0cecd33700 1 --2- 172.21.15.162:0/2771895468 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0ce8141b80 0x7f0ce8154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:10.671 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.664+0000 7f0cecd33700 1 --2- 172.21.15.162:0/2771895468 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0ce80ac500 0x7f0ce81446f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:10.671 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.664+0000 7f0cecd33700 1 --2- 172.21.15.162:0/2771895468 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ce80ae9d0 0x7f0ce813d590 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:10.672 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.664+0000 7f0cecd33700 1 -- 172.21.15.162:0/2771895468 >> 172.21.15.162:0/2771895468 conn(0x7f0ce80b57d0 msgr2=0x7f0ce81413d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:10.672 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.664+0000 7f0cecd33700 1 -- 172.21.15.162:0/2771895468 shutdown_connections 2020-11-28T21:54:10.672 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.664+0000 7f0cecd33700 1 -- 172.21.15.162:0/2771895468 wait complete. 2020-11-28T21:54:10.681 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-28T21:54:10.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.800+0000 7f695b59e700 1 Processor -- start 2020-11-28T21:54:10.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.800+0000 7f695b59e700 1 -- start start 2020-11-28T21:54:10.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.800+0000 7f695b59e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f695c0b3ab0 0x7f695c05c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:10.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.800+0000 7f695b59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f695c0b30e0 0x7f695c13fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:10.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.800+0000 7f695b59e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f695c05baf0 con 0x7f695c0b2d00 2020-11-28T21:54:10.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.800+0000 7f695b59e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f695c0b0e20 con 0x7f695c0b3ab0 2020-11-28T21:54:10.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.800+0000 7f695b59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f695c05bd90 con 0x7f695c0b30e0 2020-11-28T21:54:10.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.800+0000 7f6959d9b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f695c0b3ab0 0x7f695c05c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:10.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.800+0000 7f6959d9b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f695c0b3ab0 0x7f695c05c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42998/0 (socket says 172.21.15.162:42998) 2020-11-28T21:54:10.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.800+0000 7f6959d9b700 1 -- 172.21.15.162:0/2076564914 learned_addr learned my addr 172.21.15.162:0/2076564914 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:10.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.800+0000 7f695ad9d700 1 --2- 172.21.15.162:0/2076564914 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f695c0b30e0 0x7f695c13fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:10.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.800+0000 7f6959d9b700 1 -- 172.21.15.162:0/2076564914 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f695c0b30e0 msgr2=0x7f695c13fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:10.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.800+0000 7f6959d9b700 1 --2- 172.21.15.162:0/2076564914 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f695c0b30e0 0x7f695c13fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:10.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.800+0000 7f6959d9b700 1 -- 172.21.15.162:0/2076564914 >> v1:172.21.15.118:6789/0 conn(0x7f695c0b2d00 legacy=0x7f695c142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:10.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.800+0000 7f6959d9b700 1 -- 172.21.15.162:0/2076564914 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f695c05c030 con 0x7f695c0b3ab0 2020-11-28T21:54:10.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.800+0000 7f6959d9b700 1 --2- 172.21.15.162:0/2076564914 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f695c0b3ab0 0x7f695c05c4d0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 rx=0x7f6950002f30 tx=0x7f6950006e80).ready entity=mon.1 client_cookie=9cbed266f4d5f319 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:10.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.800+0000 7f695959a700 1 -- 172.21.15.162:0/2076564914 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69500097f0 con 0x7f695c0b3ab0 2020-11-28T21:54:10.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.800+0000 7f695959a700 1 -- 172.21.15.162:0/2076564914 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6950009950 con 0x7f695c0b3ab0 2020-11-28T21:54:10.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.800+0000 7f695959a700 1 -- 172.21.15.162:0/2076564914 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6950009cb0 con 0x7f695c0b3ab0 2020-11-28T21:54:10.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.800+0000 7f695b59e700 1 -- 172.21.15.162:0/2076564914 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f695c0b3ab0 msgr2=0x7f695c05c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:10.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.800+0000 7f695b59e700 1 --2- 172.21.15.162:0/2076564914 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f695c0b3ab0 0x7f695c05c4d0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 rx=0x7f6950002f30 tx=0x7f6950006e80).stop 2020-11-28T21:54:10.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.804+0000 7f695b59e700 1 -- 172.21.15.162:0/2076564914 shutdown_connections 2020-11-28T21:54:10.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.804+0000 7f695b59e700 1 --2- 172.21.15.162:0/2076564914 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f695c0b30e0 0x7f695c13fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:10.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.804+0000 7f695b59e700 1 --2- 172.21.15.162:0/2076564914 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f695c0b3ab0 0x7f695c05c4d0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:10.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.804+0000 7f695b59e700 1 -- 172.21.15.162:0/2076564914 >> 172.21.15.162:0/2076564914 conn(0x7f695c0604c0 msgr2=0x7f695c0608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:10.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.804+0000 7f695b59e700 1 -- 172.21.15.162:0/2076564914 shutdown_connections 2020-11-28T21:54:10.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.804+0000 7f695b59e700 1 -- 172.21.15.162:0/2076564914 wait complete. 2020-11-28T21:54:10.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.804+0000 7f695b59e700 1 Processor -- start 2020-11-28T21:54:10.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.804+0000 7f695b59e700 1 -- start start 2020-11-28T21:54:10.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.808+0000 7f695b59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f695c0b2d00 0x7f695c139700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:10.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.808+0000 7f695b59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f695c0b30e0 0x7f695c0b84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:10.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.808+0000 7f695b59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f695c0b3ab0 0x7f695c0b8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:10.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.808+0000 7f695b59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f695c0b8b80 con 0x7f695c0b30e0 2020-11-28T21:54:10.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.808+0000 7f695b59e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f695c0ad070 con 0x7f695c0b2d00 2020-11-28T21:54:10.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.808+0000 7f695b59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f695c0ad310 con 0x7f695c0b3ab0 2020-11-28T21:54:10.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.808+0000 7f695a59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f695c0b2d00 0x7f695c139700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:10.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.808+0000 7f695a59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f695c0b2d00 0x7f695c139700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43000/0 (socket says 172.21.15.162:43000) 2020-11-28T21:54:10.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.808+0000 7f695a59c700 1 -- 172.21.15.162:0/694489676 learned_addr learned my addr 172.21.15.162:0/694489676 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:10.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.808+0000 7f695ad9d700 1 --2- 172.21.15.162:0/694489676 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f695c0b3ab0 0x7f695c0b8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:10.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.808+0000 7f6959d9b700 1 --2- 172.21.15.162:0/694489676 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f695c0b30e0 0x7f695c0b84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:10.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.808+0000 7f695a59c700 1 -- 172.21.15.162:0/694489676 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f695c0b3ab0 msgr2=0x7f695c0b8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:10.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.808+0000 7f695a59c700 1 --2- 172.21.15.162:0/694489676 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f695c0b3ab0 0x7f695c0b8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:10.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.808+0000 7f695a59c700 1 -- 172.21.15.162:0/694489676 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f695c0b30e0 msgr2=0x7f695c0b84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:10.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.808+0000 7f695a59c700 1 --2- 172.21.15.162:0/694489676 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f695c0b30e0 0x7f695c0b84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:10.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.808+0000 7f695a59c700 1 -- 172.21.15.162:0/694489676 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f695c0ad5b0 con 0x7f695c0b2d00 2020-11-28T21:54:10.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.808+0000 7f695a59c700 1 --2- 172.21.15.162:0/694489676 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f695c0b2d00 0x7f695c139700 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 rx=0x7f695c0acde0 tx=0x7f694c007c20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:10.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.808+0000 7f694b7fe700 1 -- 172.21.15.162:0/694489676 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f694c013410 con 0x7f695c0b2d00 2020-11-28T21:54:10.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.808+0000 7f694b7fe700 1 -- 172.21.15.162:0/694489676 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f694c013570 con 0x7f695c0b2d00 2020-11-28T21:54:10.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.808+0000 7f695b59e700 1 -- 172.21.15.162:0/694489676 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f695c0afa40 con 0x7f695c0b2d00 2020-11-28T21:54:10.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.808+0000 7f694b7fe700 1 -- 172.21.15.162:0/694489676 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f694c0138d0 con 0x7f695c0b2d00 2020-11-28T21:54:10.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.808+0000 7f695b59e700 1 -- 172.21.15.162:0/694489676 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f695c0ada50 con 0x7f695c0b2d00 2020-11-28T21:54:10.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.808+0000 7f695b59e700 1 -- 172.21.15.162:0/694489676 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f695c057050 con 0x7f695c0b2d00 2020-11-28T21:54:10.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.812+0000 7f694b7fe700 1 -- 172.21.15.162:0/694489676 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f694c010030 con 0x7f695c0b2d00 2020-11-28T21:54:10.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.812+0000 7f694b7fe700 1 --2- 172.21.15.162:0/694489676 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6938055470 0x7f6938057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:10.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.812+0000 7f694b7fe700 1 -- 172.21.15.162:0/694489676 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f694c06ddf0 con 0x7f695c0b2d00 2020-11-28T21:54:10.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.812+0000 7f6959d9b700 1 --2- 172.21.15.162:0/694489676 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6938055470 0x7f6938057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:10.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.812+0000 7f6959d9b700 1 --2- 172.21.15.162:0/694489676 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6938055470 0x7f6938057930 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f695001c000 tx=0x7f6950015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:10.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.816+0000 7f694b7fe700 1 -- 172.21.15.162:0/694489676 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f694c043fe0 con 0x7f695c0b2d00 2020-11-28T21:54:10.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.956+0000 7f695b59e700 1 -- 172.21.15.162:0/694489676 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f695c0b6320 con 0x7f695c0b2d00 2020-11-28T21:54:10.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.956+0000 7f694b7fe700 1 -- 172.21.15.162:0/694489676 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) v1 ==== 100+0+10 (secure 0 0 0) 0x7f694c047600 con 0x7f695c0b2d00 2020-11-28T21:54:10.962 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:54:10.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.960+0000 7f695b59e700 1 -- 172.21.15.162:0/694489676 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6938055470 msgr2=0x7f6938057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:10.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.960+0000 7f695b59e700 1 --2- 172.21.15.162:0/694489676 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6938055470 0x7f6938057930 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f695001c000 tx=0x7f6950015000).stop 2020-11-28T21:54:10.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.960+0000 7f695b59e700 1 -- 172.21.15.162:0/694489676 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f695c0b2d00 msgr2=0x7f695c139700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:10.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.960+0000 7f695b59e700 1 --2- 172.21.15.162:0/694489676 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f695c0b2d00 0x7f695c139700 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 rx=0x7f695c0acde0 tx=0x7f694c007c20).stop 2020-11-28T21:54:10.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.960+0000 7f695b59e700 1 -- 172.21.15.162:0/694489676 shutdown_connections 2020-11-28T21:54:10.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.960+0000 7f695b59e700 1 --2- 172.21.15.162:0/694489676 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6938055470 0x7f6938057930 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:10.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.960+0000 7f695b59e700 1 --2- 172.21.15.162:0/694489676 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f695c0b3ab0 0x7f695c0b8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:10.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.960+0000 7f695b59e700 1 --2- 172.21.15.162:0/694489676 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f695c0b2d00 0x7f695c139700 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:10.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.960+0000 7f695b59e700 1 --2- 172.21.15.162:0/694489676 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f695c0b30e0 0x7f695c0b84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:10.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.960+0000 7f695b59e700 1 -- 172.21.15.162:0/694489676 >> 172.21.15.162:0/694489676 conn(0x7f695c0604c0 msgr2=0x7f695c13bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:10.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.960+0000 7f695b59e700 1 -- 172.21.15.162:0/694489676 shutdown_connections 2020-11-28T21:54:10.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:10.960+0000 7f695b59e700 1 -- 172.21.15.162:0/694489676 wait complete. 2020-11-28T21:54:10.975 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-28T21:54:10.976 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:54:11.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.084+0000 7fcc7b59e700 1 Processor -- start 2020-11-28T21:54:11.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.084+0000 7fcc7b59e700 1 -- start start 2020-11-28T21:54:11.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.084+0000 7fcc7b59e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcc7c0b0000 0x7fcc7c151a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:11.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.084+0000 7fcc7b59e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc7c098f10 con 0x7fcc7c098b30 2020-11-28T21:54:11.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.084+0000 7fcc7b59e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc7c0b0680 con 0x7fcc7c0994f0 2020-11-28T21:54:11.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.084+0000 7fcc7b59e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc7c0b1030 con 0x7fcc7c0b0000 2020-11-28T21:54:11.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.084+0000 7fcc7ad9d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcc7c0b0000 0x7fcc7c151a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:11.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.084+0000 7fcc79d9b700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fcc7c0994f0 0x7fcc7c060710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:49852/0 (socket says 172.21.15.162:49852) 2020-11-28T21:54:11.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.084+0000 7fcc79d9b700 1 -- 172.21.15.162:0/1774774072 learned_addr learned my addr 172.21.15.162:0/1774774072 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:11.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.084+0000 7fcc7959a700 1 -- 172.21.15.162:0/1774774072 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2496814004 0 0) 0x7fcc7c0b0680 con 0x7fcc7c0994f0 2020-11-28T21:54:11.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.084+0000 7fcc7959a700 1 -- 172.21.15.162:0/1774774072 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc60003230 con 0x7fcc7c0994f0 2020-11-28T21:54:11.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.084+0000 7fcc7ad9d700 1 -- 172.21.15.162:0/1774774072 >> v1:172.21.15.162:6789/0 conn(0x7fcc7c0994f0 legacy=0x7fcc7c060710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:11.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.084+0000 7fcc7959a700 1 -- 172.21.15.162:0/1774774072 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2472379843 0 0) 0x7fcc7c098f10 con 0x7fcc7c098b30 2020-11-28T21:54:11.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.084+0000 7fcc7ad9d700 1 -- 172.21.15.162:0/1774774072 >> v1:172.21.15.118:6789/0 conn(0x7fcc7c098b30 legacy=0x7fcc7c060000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:11.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.088+0000 7fcc7ad9d700 1 -- 172.21.15.162:0/1774774072 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc7c0b0920 con 0x7fcc7c0b0000 2020-11-28T21:54:11.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.088+0000 7fcc7ad9d700 1 --2- 172.21.15.162:0/1774774072 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcc7c0b0000 0x7fcc7c151a30 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 rx=0x7fcc740032b0 tx=0x7fcc740091a0).ready entity=mon.1 client_cookie=c2d4d17c38ba5a2e server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:11.098 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.088+0000 7fcc7959a700 1 -- 172.21.15.162:0/1774774072 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc7400b9a0 con 0x7fcc7c0b0000 2020-11-28T21:54:11.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.088+0000 7fcc7959a700 1 -- 172.21.15.162:0/1774774072 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc7400bb00 con 0x7fcc7c0b0000 2020-11-28T21:54:11.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.088+0000 7fcc7959a700 1 -- 172.21.15.162:0/1774774072 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc7400bc70 con 0x7fcc7c0b0000 2020-11-28T21:54:11.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.088+0000 7fcc7b59e700 1 -- 172.21.15.162:0/1774774072 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcc7c0b0000 msgr2=0x7fcc7c151a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:11.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.088+0000 7fcc7b59e700 1 --2- 172.21.15.162:0/1774774072 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcc7c0b0000 0x7fcc7c151a30 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 rx=0x7fcc740032b0 tx=0x7fcc740091a0).stop 2020-11-28T21:54:11.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.088+0000 7fcc7b59e700 1 -- 172.21.15.162:0/1774774072 shutdown_connections 2020-11-28T21:54:11.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.088+0000 7fcc7b59e700 1 --2- 172.21.15.162:0/1774774072 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcc7c0b0000 0x7fcc7c151a30 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:11.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.088+0000 7fcc7b59e700 1 -- 172.21.15.162:0/1774774072 >> 172.21.15.162:0/1774774072 conn(0x7fcc7c05b970 msgr2=0x7fcc7c05bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:11.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.088+0000 7fcc7b59e700 1 -- 172.21.15.162:0/1774774072 shutdown_connections 2020-11-28T21:54:11.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.088+0000 7fcc7b59e700 1 -- 172.21.15.162:0/1774774072 wait complete. 2020-11-28T21:54:11.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.088+0000 7fcc7b59e700 1 Processor -- start 2020-11-28T21:54:11.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.088+0000 7fcc7b59e700 1 -- start start 2020-11-28T21:54:11.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.088+0000 7fcc7b59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcc7c098b30 0x7fcc7c0ab370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:11.104 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.088+0000 7fcc7b59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcc7c0994f0 0x7fcc7c0ad080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:11.104 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.088+0000 7fcc7b59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcc7c0b0000 0x7fcc7c0ad570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:11.104 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.088+0000 7fcc7b59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7c1560f0 con 0x7fcc7c098b30 2020-11-28T21:54:11.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.088+0000 7fcc7b59e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7c156390 con 0x7fcc7c0994f0 2020-11-28T21:54:11.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.088+0000 7fcc7b59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7c156630 con 0x7fcc7c0b0000 2020-11-28T21:54:11.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.088+0000 7fcc79d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcc7c0994f0 0x7fcc7c0ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:11.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.088+0000 7fcc79d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcc7c0994f0 0x7fcc7c0ad080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43014/0 (socket says 172.21.15.162:43014) 2020-11-28T21:54:11.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.088+0000 7fcc79d9b700 1 -- 172.21.15.162:0/2497247041 learned_addr learned my addr 172.21.15.162:0/2497247041 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:11.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.088+0000 7fcc7ad9d700 1 --2- 172.21.15.162:0/2497247041 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcc7c0b0000 0x7fcc7c0ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:11.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.088+0000 7fcc7a59c700 1 --2- 172.21.15.162:0/2497247041 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcc7c098b30 0x7fcc7c0ab370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:11.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.088+0000 7fcc79d9b700 1 -- 172.21.15.162:0/2497247041 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcc7c0b0000 msgr2=0x7fcc7c0ad570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:11.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.088+0000 7fcc79d9b700 1 --2- 172.21.15.162:0/2497247041 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcc7c0b0000 0x7fcc7c0ad570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:11.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.088+0000 7fcc79d9b700 1 -- 172.21.15.162:0/2497247041 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcc7c098b30 msgr2=0x7fcc7c0ab370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:11.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.088+0000 7fcc79d9b700 1 --2- 172.21.15.162:0/2497247041 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcc7c098b30 0x7fcc7c0ab370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:11.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.088+0000 7fcc79d9b700 1 -- 172.21.15.162:0/2497247041 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc7c1568d0 con 0x7fcc7c0994f0 2020-11-28T21:54:11.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.092+0000 7fcc79d9b700 1 --2- 172.21.15.162:0/2497247041 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcc7c0994f0 0x7fcc7c0ad080 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 rx=0x7fcc700010d0 tx=0x7fcc70007a70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:11.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.092+0000 7fcc6b7fe700 1 -- 172.21.15.162:0/2497247041 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc700072e0 con 0x7fcc7c0994f0 2020-11-28T21:54:11.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.092+0000 7fcc6b7fe700 1 -- 172.21.15.162:0/2497247041 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc70007440 con 0x7fcc7c0994f0 2020-11-28T21:54:11.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.092+0000 7fcc7b59e700 1 -- 172.21.15.162:0/2497247041 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc7c0634e0 con 0x7fcc7c0994f0 2020-11-28T21:54:11.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.092+0000 7fcc6b7fe700 1 -- 172.21.15.162:0/2497247041 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc700057d0 con 0x7fcc7c0994f0 2020-11-28T21:54:11.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.092+0000 7fcc7b59e700 1 -- 172.21.15.162:0/2497247041 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc7c0adc80 con 0x7fcc7c0994f0 2020-11-28T21:54:11.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.092+0000 7fcc7b59e700 1 -- 172.21.15.162:0/2497247041 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc7c155eb0 con 0x7fcc7c0994f0 2020-11-28T21:54:11.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.092+0000 7fcc6b7fe700 1 -- 172.21.15.162:0/2497247041 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fcc70010070 con 0x7fcc7c0994f0 2020-11-28T21:54:11.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.092+0000 7fcc6b7fe700 1 --2- 172.21.15.162:0/2497247041 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcc580553b0 0x7fcc58057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:11.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.092+0000 7fcc6b7fe700 1 -- 172.21.15.162:0/2497247041 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fcc7006d660 con 0x7fcc7c0994f0 2020-11-28T21:54:11.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.092+0000 7fcc7a59c700 1 --2- 172.21.15.162:0/2497247041 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcc580553b0 0x7fcc58057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:11.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.096+0000 7fcc7a59c700 1 --2- 172.21.15.162:0/2497247041 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcc580553b0 0x7fcc58057870 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7fcc6c00b000 tx=0x7fcc6c009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:11.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.096+0000 7fcc6b7fe700 1 -- 172.21.15.162:0/2497247041 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fcc7c155eb0 con 0x7fcc7c0994f0 2020-11-28T21:54:11.288 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.284+0000 7fcc7b59e700 1 -- 172.21.15.162:0/2497247041 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc7c155eb0 con 0x7fcc7c0994f0 2020-11-28T21:54:11.289 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.284+0000 7fcc6b7fe700 1 -- 172.21.15.162:0/2497247041 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+2502 (secure 0 0 0) 0x7fcc7c155eb0 con 0x7fcc7c0994f0 2020-11-28T21:54:11.289 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:54:11.289 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":46,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6657,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.118:6833/1785592854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1785592854},{"type":"v1","addr":"172.21.15.118:6833","nonce":1785592854}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":44},{"gid":6850,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3533607073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3533607073},{"type":"v1","addr":"172.21.15.162:6835","nonce":3533607073}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:54:02.388344+0000","modified":"2020-11-28T21:54:03.391088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6859":{"gid":6859,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/3532128068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3532128068},{"type":"v1","addr":"172.21.15.162:6837","nonce":3532128068}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2020-11-28T21:54:11.291 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.288+0000 7fcc7b59e700 1 -- 172.21.15.162:0/2497247041 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcc580553b0 msgr2=0x7fcc58057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:11.291 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.288+0000 7fcc7b59e700 1 --2- 172.21.15.162:0/2497247041 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcc580553b0 0x7fcc58057870 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7fcc6c00b000 tx=0x7fcc6c009040).stop 2020-11-28T21:54:11.291 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.288+0000 7fcc7b59e700 1 -- 172.21.15.162:0/2497247041 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcc7c0994f0 msgr2=0x7fcc7c0ad080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:11.291 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.288+0000 7fcc7b59e700 1 --2- 172.21.15.162:0/2497247041 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcc7c0994f0 0x7fcc7c0ad080 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 rx=0x7fcc700010d0 tx=0x7fcc70007a70).stop 2020-11-28T21:54:11.292 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.288+0000 7fcc7b59e700 1 -- 172.21.15.162:0/2497247041 shutdown_connections 2020-11-28T21:54:11.292 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.288+0000 7fcc7b59e700 1 --2- 172.21.15.162:0/2497247041 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcc580553b0 0x7fcc58057870 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:11.292 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.288+0000 7fcc7b59e700 1 --2- 172.21.15.162:0/2497247041 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcc7c0b0000 0x7fcc7c0ad570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:11.292 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.288+0000 7fcc7b59e700 1 --2- 172.21.15.162:0/2497247041 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcc7c098b30 0x7fcc7c0ab370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:11.292 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.288+0000 7fcc7b59e700 1 --2- 172.21.15.162:0/2497247041 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcc7c0994f0 0x7fcc7c0ad080 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:11.293 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.288+0000 7fcc7b59e700 1 -- 172.21.15.162:0/2497247041 >> 172.21.15.162:0/2497247041 conn(0x7fcc7c05b970 msgr2=0x7fcc7c0b4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:11.293 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.288+0000 7fcc7b59e700 1 -- 172.21.15.162:0/2497247041 shutdown_connections 2020-11-28T21:54:11.293 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:11.288+0000 7fcc7b59e700 1 -- 172.21.15.162:0/2497247041 wait complete. 2020-11-28T21:54:11.294 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 46 2020-11-28T21:54:11.605 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-28T21:54:02.388344+0000', 'modified': '2020-11-28T21:54:03.391088+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6859}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6859': {'gid': 6859, 'name': 'a', 'rank': 0, 'incarnation': 45, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6837/3532128068', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6836', 'nonce': 3532128068}, {'type': 'v1', 'addr': '172.21.15.162:6837', 'nonce': 3532128068}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-28T21:54:11.606 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-28T21:54:11.606 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-28T21:54:11.737 INFO:teuthology.orchestra.run.smithi162.stdout:{"cluster_fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","whoami":0,"id":6859,"want_state":"up:active","state":"up:active","rank_uptime":9.3335151780000007,"mdsmap_epoch":46,"osdmap_epoch":99,"osdmap_epoch_barrier":98,"uptime":14.713891497000001} 2020-11-28T21:54:11.738 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", "id": 6859, "mdsmap_epoch": 46, "osdmap_epoch": 99, "osdmap_epoch_barrier": 98, "rank_uptime": 9.333515178, "state": "up:active", "uptime": 14.713891497, "want_state": "up:active", "whoami": 0 } 2020-11-28T21:54:11.739 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-11-28T21:54:11.739 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2020-11-28T21:54:11.740 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi096.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2020-11-28T21:54:11.741 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:11.741 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:54:11.750 INFO:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2020-11-28T21:54:11.808 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:54:11.809 INFO:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:54:11.863 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:54:11.864 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:54:11.865 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:11.865 INFO:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2020-11-28T21:54:11.911 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-11-28T21:54:11.911 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:54:11.954 INFO:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2020-11-28T21:54:11.970 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:54:11.970 INFO:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:54:12.029 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:54:12.030 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:54:12.030 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:12.031 INFO:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2020-11-28T21:54:12.033 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:2020-11-28T21:54:12.026+0000 7f2ce886b0c0 -1 init, newargv = 0x56074d535ba0 newargc=9 2020-11-28T21:54:12.033 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[11722]: starting ceph client 2020-11-28T21:54:12.044 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[11722]: starting fuse 2020-11-28T21:54:13.139 INFO:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2020-11-28T21:54:13.156 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:54:13.157 INFO:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:54:13.208 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:54:13.209 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:54:13.210 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:13.210 INFO:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2020-11-28T21:54:13.259 INFO:teuthology.orchestra.run.smithi096.stdout:49 2020-11-28T21:54:13.259 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2020-11-28T21:54:13.260 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:13.260 INFO:teuthology.orchestra.run.smithi096:> sudo python3 -c ' 2020-11-28T21:54:13.261 INFO:teuthology.orchestra.run.smithi096:> import glob 2020-11-28T21:54:13.262 INFO:teuthology.orchestra.run.smithi096:> import re 2020-11-28T21:54:13.262 INFO:teuthology.orchestra.run.smithi096:> import os 2020-11-28T21:54:13.263 INFO:teuthology.orchestra.run.smithi096:> import subprocess 2020-11-28T21:54:13.264 INFO:teuthology.orchestra.run.smithi096:> 2020-11-28T21:54:13.264 INFO:teuthology.orchestra.run.smithi096:> def find_socket(client_name): 2020-11-28T21:54:13.265 INFO:teuthology.orchestra.run.smithi096:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2020-11-28T21:54:13.266 INFO:teuthology.orchestra.run.smithi096:> files = glob.glob(asok_path) 2020-11-28T21:54:13.267 INFO:teuthology.orchestra.run.smithi096:> 2020-11-28T21:54:13.267 INFO:teuthology.orchestra.run.smithi096:> # Given a non-glob path, it better be there 2020-11-28T21:54:13.268 INFO:teuthology.orchestra.run.smithi096:> if "*" not in asok_path: 2020-11-28T21:54:13.269 INFO:teuthology.orchestra.run.smithi096:> assert(len(files) == 1) 2020-11-28T21:54:13.270 INFO:teuthology.orchestra.run.smithi096:> return files[0] 2020-11-28T21:54:13.270 INFO:teuthology.orchestra.run.smithi096:> 2020-11-28T21:54:13.271 INFO:teuthology.orchestra.run.smithi096:> for f in files: 2020-11-28T21:54:13.272 INFO:teuthology.orchestra.run.smithi096:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-11-28T21:54:13.273 INFO:teuthology.orchestra.run.smithi096:> if os.path.exists("/proc/{0}".format(pid)): 2020-11-28T21:54:13.273 INFO:teuthology.orchestra.run.smithi096:> return f 2020-11-28T21:54:13.274 INFO:teuthology.orchestra.run.smithi096:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-11-28T21:54:13.275 INFO:teuthology.orchestra.run.smithi096:> 2020-11-28T21:54:13.275 INFO:teuthology.orchestra.run.smithi096:> print(find_socket("client.1")) 2020-11-28T21:54:13.276 INFO:teuthology.orchestra.run.smithi096:> ' 2020-11-28T21:54:13.350 INFO:teuthology.orchestra.run.smithi096.stdout:/var/run/ceph/ceph-client.1.11722.asok 2020-11-28T21:54:13.355 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.11722.asok 2020-11-28T21:54:13.356 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:13.356 INFO:teuthology.orchestra.run.smithi096:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.11722.asok status 2020-11-28T21:54:13.459 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-11-28T21:54:13.470 INFO:teuthology.orchestra.run.smithi096.stdout: "metadata": { 2020-11-28T21:54:13.471 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_sha1": "a83a3572a957356977342f461361c1cd3ec06dbd", 2020-11-28T21:54:13.471 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_version": "ceph version 15.2.6-595-ga83a3572 (a83a3572a957356977342f461361c1cd3ec06dbd) octopus (stable)", 2020-11-28T21:54:13.471 INFO:teuthology.orchestra.run.smithi096.stdout: "entity_id": "1", 2020-11-28T21:54:13.471 INFO:teuthology.orchestra.run.smithi096.stdout: "hostname": "smithi096", 2020-11-28T21:54:13.471 INFO:teuthology.orchestra.run.smithi096.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2020-11-28T21:54:13.472 INFO:teuthology.orchestra.run.smithi096.stdout: "pid": "11722", 2020-11-28T21:54:13.472 INFO:teuthology.orchestra.run.smithi096.stdout: "root": "/" 2020-11-28T21:54:13.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-11-28T21:54:13.472 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_count": 0, 2020-11-28T21:54:13.472 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_pinned_count": 0, 2020-11-28T21:54:13.473 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 6852, 2020-11-28T21:54:13.473 INFO:teuthology.orchestra.run.smithi096.stdout: "inst": { 2020-11-28T21:54:13.473 INFO:teuthology.orchestra.run.smithi096.stdout: "name": { 2020-11-28T21:54:13.473 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "client", 2020-11-28T21:54:13.473 INFO:teuthology.orchestra.run.smithi096.stdout: "num": 6852 2020-11-28T21:54:13.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-11-28T21:54:13.474 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2020-11-28T21:54:13.474 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2020-11-28T21:54:13.474 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2020-11-28T21:54:13.474 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1652715507 2020-11-28T21:54:13.475 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-11-28T21:54:13.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-11-28T21:54:13.475 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2020-11-28T21:54:13.475 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2020-11-28T21:54:13.475 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2020-11-28T21:54:13.476 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1652715507 2020-11-28T21:54:13.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-11-28T21:54:13.476 INFO:teuthology.orchestra.run.smithi096.stdout: "inst_str": "client.6852 172.21.15.96:0/1652715507", 2020-11-28T21:54:13.476 INFO:teuthology.orchestra.run.smithi096.stdout: "addr_str": "172.21.15.96:0/1652715507", 2020-11-28T21:54:13.476 INFO:teuthology.orchestra.run.smithi096.stdout: "inode_count": 1, 2020-11-28T21:54:13.477 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_epoch": 46, 2020-11-28T21:54:13.477 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch": 99, 2020-11-28T21:54:13.477 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch_barrier": 0, 2020-11-28T21:54:13.477 INFO:teuthology.orchestra.run.smithi096.stdout: "blacklisted": false 2020-11-28T21:54:13.477 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-11-28T21:54:13.479 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:13.479 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:54:13.479 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:54:13.487 INFO:teuthology.orchestra.run.smithi096.stdout:fuseblk 2020-11-28T21:54:13.488 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-11-28T21:54:13.488 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:13.489 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:54:13.546 INFO:teuthology.orchestra.run.smithi192:> cd /home/ubuntu/cephtest/mnt.0 && sudo mkdir -p foo 2020-11-28T21:54:13.577 INFO:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-28T21:54:13.578 INFO:teuthology.orchestra.run.smithi192:> import os 2020-11-28T21:54:13.578 INFO:teuthology.orchestra.run.smithi192:> import stat 2020-11-28T21:54:13.579 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:54:13.579 INFO:teuthology.orchestra.run.smithi192:> print(os.stat("/home/ubuntu/cephtest/mnt.0/foo").st_ino) 2020-11-28T21:54:13.580 INFO:teuthology.orchestra.run.smithi192:> ' 2020-11-28T21:54:13.703 INFO:teuthology.orchestra.run.smithi192.stdout:1099511627776 2020-11-28T21:54:13.890 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:54:14.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.004+0000 7f195359e700 1 Processor -- start 2020-11-28T21:54:14.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.004+0000 7f195359e700 1 -- start start 2020-11-28T21:54:14.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.004+0000 7f195359e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f19540994f0 0x7f19540b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:14.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.004+0000 7f195359e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1954060710 0x7f19540b1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:14.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.004+0000 7f195359e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1954098f10 con 0x7f1954098b30 2020-11-28T21:54:14.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.004+0000 7f195359e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19540b1030 con 0x7f1954060710 2020-11-28T21:54:14.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.004+0000 7f195359e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19540b0680 con 0x7f19540994f0 2020-11-28T21:54:14.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.004+0000 7f1952d9d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1954060710 0x7f19540b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:14.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.004+0000 7f195259c700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f1954098b30 0x7f1954060000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:49864/0 (socket says 172.21.15.162:49864) 2020-11-28T21:54:14.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.004+0000 7f195259c700 1 -- 172.21.15.162:0/2172821347 learned_addr learned my addr 172.21.15.162:0/2172821347 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:14.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.004+0000 7f1951d9b700 1 --2- 172.21.15.162:0/2172821347 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f19540994f0 0x7f19540b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:14.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.004+0000 7f1952d9d700 1 -- 172.21.15.162:0/2172821347 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f19540994f0 msgr2=0x7f19540b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:14.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.004+0000 7f1952d9d700 1 --2- 172.21.15.162:0/2172821347 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f19540994f0 0x7f19540b0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:14.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.004+0000 7f1952d9d700 1 -- 172.21.15.162:0/2172821347 >> v1:172.21.15.162:6789/0 conn(0x7f1954098b30 legacy=0x7f1954060000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:14.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.004+0000 7f1952d9d700 1 -- 172.21.15.162:0/2172821347 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19540b0920 con 0x7f1954060710 2020-11-28T21:54:14.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.004+0000 7f195159a700 1 -- 172.21.15.162:0/2172821347 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3437649556 0 0) 0x7f1954098f10 con 0x7f1954098b30 2020-11-28T21:54:14.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.004+0000 7f1952d9d700 1 --2- 172.21.15.162:0/2172821347 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1954060710 0x7f19540b1580 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 rx=0x7f194c002f30 tx=0x7f194c006dc0).ready entity=mon.1 client_cookie=2255d592d91a5a5d server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:14.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.004+0000 7f195159a700 1 -- 172.21.15.162:0/2172821347 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f194c009760 con 0x7f1954060710 2020-11-28T21:54:14.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.004+0000 7f195159a700 1 -- 172.21.15.162:0/2172821347 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f194c0098c0 con 0x7f1954060710 2020-11-28T21:54:14.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.004+0000 7f195159a700 1 -- 172.21.15.162:0/2172821347 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f194c009c20 con 0x7f1954060710 2020-11-28T21:54:14.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.004+0000 7f195359e700 1 -- 172.21.15.162:0/2172821347 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1954060710 msgr2=0x7f19540b1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:14.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.004+0000 7f195359e700 1 --2- 172.21.15.162:0/2172821347 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1954060710 0x7f19540b1580 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 rx=0x7f194c002f30 tx=0x7f194c006dc0).stop 2020-11-28T21:54:14.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.008+0000 7f195359e700 1 -- 172.21.15.162:0/2172821347 shutdown_connections 2020-11-28T21:54:14.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.008+0000 7f195359e700 1 --2- 172.21.15.162:0/2172821347 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1954060710 0x7f19540b1580 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:14.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.008+0000 7f195359e700 1 --2- 172.21.15.162:0/2172821347 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f19540994f0 0x7f19540b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:14.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.008+0000 7f195359e700 1 -- 172.21.15.162:0/2172821347 >> 172.21.15.162:0/2172821347 conn(0x7f195405b970 msgr2=0x7f195405bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:14.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.008+0000 7f195359e700 1 -- 172.21.15.162:0/2172821347 shutdown_connections 2020-11-28T21:54:14.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.008+0000 7f195359e700 1 -- 172.21.15.162:0/2172821347 wait complete. 2020-11-28T21:54:14.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.008+0000 7f195359e700 1 Processor -- start 2020-11-28T21:54:14.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.008+0000 7f195359e700 1 -- start start 2020-11-28T21:54:14.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.008+0000 7f195359e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1954060710 0x7f19540666a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:14.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.008+0000 7f195359e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1954098b30 0x7f1954064400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:14.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.008+0000 7f195359e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f19540994f0 0x7f1954155f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:14.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.008+0000 7f195359e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1954155a40 con 0x7f1954060710 2020-11-28T21:54:14.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.008+0000 7f195359e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195409bcd0 con 0x7f1954098b30 2020-11-28T21:54:14.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.008+0000 7f195359e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f195409bf70 con 0x7f19540994f0 2020-11-28T21:54:14.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.008+0000 7f1951d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1954098b30 0x7f1954064400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:14.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.008+0000 7f1951d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1954098b30 0x7f1954064400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43028/0 (socket says 172.21.15.162:43028) 2020-11-28T21:54:14.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.008+0000 7f1951d9b700 1 -- 172.21.15.162:0/2933127333 learned_addr learned my addr 172.21.15.162:0/2933127333 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:14.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.008+0000 7f195259c700 1 --2- 172.21.15.162:0/2933127333 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1954060710 0x7f19540666a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:14.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.012+0000 7f1952d9d700 1 --2- 172.21.15.162:0/2933127333 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f19540994f0 0x7f1954155f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:14.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.012+0000 7f1951d9b700 1 -- 172.21.15.162:0/2933127333 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f19540994f0 msgr2=0x7f1954155f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:14.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.012+0000 7f1951d9b700 1 --2- 172.21.15.162:0/2933127333 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f19540994f0 0x7f1954155f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:14.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.012+0000 7f1951d9b700 1 -- 172.21.15.162:0/2933127333 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1954060710 msgr2=0x7f19540666a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:14.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.012+0000 7f1951d9b700 1 --2- 172.21.15.162:0/2933127333 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1954060710 0x7f19540666a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:14.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.012+0000 7f1951d9b700 1 -- 172.21.15.162:0/2933127333 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f195409c210 con 0x7f1954098b30 2020-11-28T21:54:14.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.012+0000 7f1951d9b700 1 --2- 172.21.15.162:0/2933127333 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1954098b30 0x7f1954064400 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 rx=0x7f1944006dd0 tx=0x7f19440091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:14.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.012+0000 7f19437fe700 1 -- 172.21.15.162:0/2933127333 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f194400ca00 con 0x7f1954098b30 2020-11-28T21:54:14.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.012+0000 7f19437fe700 1 -- 172.21.15.162:0/2933127333 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f194400cb60 con 0x7f1954098b30 2020-11-28T21:54:14.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.012+0000 7f195359e700 1 -- 172.21.15.162:0/2933127333 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f195409c4b0 con 0x7f1954098b30 2020-11-28T21:54:14.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.012+0000 7f19437fe700 1 -- 172.21.15.162:0/2933127333 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19440123f0 con 0x7f1954098b30 2020-11-28T21:54:14.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.012+0000 7f195359e700 1 -- 172.21.15.162:0/2933127333 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1954064be0 con 0x7f1954098b30 2020-11-28T21:54:14.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.012+0000 7f195359e700 1 -- 172.21.15.162:0/2933127333 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1954057050 con 0x7f1954098b30 2020-11-28T21:54:14.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.012+0000 7f19437fe700 1 -- 172.21.15.162:0/2933127333 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1944022070 con 0x7f1954098b30 2020-11-28T21:54:14.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.012+0000 7f19437fe700 1 --2- 172.21.15.162:0/2933127333 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1930055330 0x7f19300577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:14.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.012+0000 7f19437fe700 1 -- 172.21.15.162:0/2933127333 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f194406e610 con 0x7f1954098b30 2020-11-28T21:54:14.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.012+0000 7f195259c700 1 --2- 172.21.15.162:0/2933127333 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1930055330 0x7f19300577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:14.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.016+0000 7f195259c700 1 --2- 172.21.15.162:0/2933127333 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1930055330 0x7f19300577f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f194800d000 tx=0x7f1948001160).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:14.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.020+0000 7f19437fe700 1 -- 172.21.15.162:0/2933127333 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1944044910 con 0x7f1954098b30 2020-11-28T21:54:14.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.208+0000 7f195359e700 1 -- 172.21.15.162:0/2933127333 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1954068350 con 0x7f1954098b30 2020-11-28T21:54:14.212 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.208+0000 7f19437fe700 1 -- 172.21.15.162:0/2933127333 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+2502 (secure 0 0 0) 0x7f1944016070 con 0x7f1954098b30 2020-11-28T21:54:14.212 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:54:14.212 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":46,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6657,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.118:6833/1785592854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1785592854},{"type":"v1","addr":"172.21.15.118:6833","nonce":1785592854}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":44},{"gid":6850,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3533607073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3533607073},{"type":"v1","addr":"172.21.15.162:6835","nonce":3533607073}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:54:02.388344+0000","modified":"2020-11-28T21:54:03.391088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6859":{"gid":6859,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/3532128068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3532128068},{"type":"v1","addr":"172.21.15.162:6837","nonce":3532128068}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2020-11-28T21:54:14.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.208+0000 7f195359e700 1 -- 172.21.15.162:0/2933127333 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1930055330 msgr2=0x7f19300577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:14.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.208+0000 7f195359e700 1 --2- 172.21.15.162:0/2933127333 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1930055330 0x7f19300577f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f194800d000 tx=0x7f1948001160).stop 2020-11-28T21:54:14.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.208+0000 7f195359e700 1 -- 172.21.15.162:0/2933127333 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1954098b30 msgr2=0x7f1954064400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:14.216 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.208+0000 7f195359e700 1 --2- 172.21.15.162:0/2933127333 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1954098b30 0x7f1954064400 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 rx=0x7f1944006dd0 tx=0x7f19440091a0).stop 2020-11-28T21:54:14.216 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.212+0000 7f195359e700 1 -- 172.21.15.162:0/2933127333 shutdown_connections 2020-11-28T21:54:14.217 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.212+0000 7f195359e700 1 --2- 172.21.15.162:0/2933127333 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1930055330 0x7f19300577f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:14.217 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.212+0000 7f195359e700 1 --2- 172.21.15.162:0/2933127333 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f19540994f0 0x7f1954155f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:14.217 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.212+0000 7f195359e700 1 --2- 172.21.15.162:0/2933127333 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1954060710 0x7f19540666a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:14.218 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.212+0000 7f195359e700 1 --2- 172.21.15.162:0/2933127333 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1954098b30 0x7f1954064400 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:14.218 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.212+0000 7f195359e700 1 -- 172.21.15.162:0/2933127333 >> 172.21.15.162:0/2933127333 conn(0x7f195405b970 msgr2=0x7f19540ab5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:14.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.212+0000 7f195359e700 1 -- 172.21.15.162:0/2933127333 shutdown_connections 2020-11-28T21:54:14.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.212+0000 7f195359e700 1 -- 172.21.15.162:0/2933127333 wait complete. 2020-11-28T21:54:14.220 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 46 2020-11-28T21:54:14.226 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump inode 0x10000000000 2020-11-28T21:54:14.339 INFO:teuthology.orchestra.run.smithi162.stdout:{"path":"/foo","ino":1099511627776,"rdev":0,"ctime":"2020-11-28T21:54:13.565738+0000","btime":"2020-11-28T21:54:13.565738+0000","mode":16877,"uid":0,"gid":0,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2020-11-28T21:54:13.565738+0000","atime":"2020-11-28T21:54:13.565738+0000","time_warp_seq":0,"change_attr":0,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"0.000000","num_files":0,"num_subdirs":0,"change_attr":0},"rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"0.000000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"0.000000"},"version":2,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"dirfrag":1,"caps":1,"dirtyparent":1,"dirty":1},"nref":4,"versionlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{"gather_set":[],"state":"excl","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"linklock":{},"dirfragtreelock":{},"filelock":{"gather_set":[],"state":"excl","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"xattrlock":{"gather_set":[],"state":"excl","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"states":["auth","dirty","dirtyparent","dirtypool"],"client_caps":[{"client_id":6801,"pending":"pAsxLsXsxFsx","issued":"pAsxLsXsxFsx","wanted":"-","last_sent":1}],"loner":6801,"want_loner":6801,"mds_caps_wanted":[]} 2020-11-28T21:54:14.340 DEBUG:tasks.cephfs.filesystem:_json_asok output { "accounted_rstat": { "rbytes": 0, "rctime": "0.000000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "atime": "2020-11-28T21:54:13.565738+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "xlock_by": {} }, "backtrace_version": 2, "btime": "2020-11-28T21:54:13.565738+0000", "change_attr": 0, "client_caps": [ { "client_id": 6801, "issued": "pAsxLsXsxFsx", "last_sent": 1, "pending": "pAsxLsXsxFsx", "wanted": "-" } ], "client_ranges": [], "ctime": "2020-11-28T21:54:13.565738+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 0, "mtime": "0.000000", "num_files": 0, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "xlock_by": {} }, "flocklock": {}, "gid": 0, "ino": 1099511627776, "is_auth": true, "is_freezing": false, "is_frozen": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": 6801, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16877, "mtime": "2020-11-28T21:54:13.565738+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "xlock_by": {} }, "nlink": 1, "nref": 4, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "path": "/foo", "pins": { "caps": 1, "dirfrag": 1, "dirty": 1, "dirtyparent": 1, "request": 0 }, "policylock": {}, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "0.000000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty", "dirtyparent", "dirtypool" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 0, "version": 2, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "xlock_by": {} }, "want_loner": 6801, "xattr_version": 1, "xattrlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "xlock_by": {} }, "xattrs": [] } 2020-11-28T21:54:14.341 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-28T21:54:14.341 INFO:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:54:14.359 INFO:teuthology.orchestra.run.smithi118:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:54:14.378 INFO:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:54:14.397 INFO:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:54:14.415 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-28T21:54:14.416 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:14.416 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:54:14.417 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:54:14.467 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2020-11-28T21:54:14.468 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-28T21:54:14.468 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2020-11-28T21:54:14.469 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-28T21:54:14.469 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:54:14.548 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:ceph-fuse[12299]: fuse finished with error 0 and tester_r 0 2020-11-28T21:54:14.550 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:14.551 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:54:14.551 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:54:14.598 INFO:teuthology.orchestra.run.smithi192.stdout:ext2/ext3 2020-11-28T21:54:14.599 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-11-28T21:54:14.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-28T21:54:14.603 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:54:14.651 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:14.651 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:54:14.652 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:54:14.659 INFO:teuthology.orchestra.run.smithi096.stdout:fuseblk 2020-11-28T21:54:14.660 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-11-28T21:54:14.661 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi096.front.sepia.ceph.com... 2020-11-28T21:54:14.661 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-28T21:54:14.662 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:54:14.734 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[11722]: fuse finished with error 0 and tester_r 0 2020-11-28T21:54:14.736 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:14.736 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:54:14.737 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:54:14.783 INFO:teuthology.orchestra.run.smithi096.stdout:ext2/ext3 2020-11-28T21:54:14.784 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-11-28T21:54:14.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-28T21:54:14.828 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:54:14.834 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config rm mon mon_allow_pool_delete 2020-11-28T21:54:14.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.952+0000 7fb118c4e700 1 Processor -- start 2020-11-28T21:54:14.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.952+0000 7fb118c4e700 1 -- start start 2020-11-28T21:54:14.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.952+0000 7fb118c4e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb1140af9e0 0x7fb11405c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:14.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.952+0000 7fb118c4e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fb1140b04a0 0x7fb1140ae3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:14.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.952+0000 7fb118c4e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1140afdc0 con 0x7fb1140adb60 2020-11-28T21:54:14.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.952+0000 7fb118c4e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1140ae890 con 0x7fb1140b04a0 2020-11-28T21:54:14.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.952+0000 7fb118c4e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb11405baf0 con 0x7fb1140af9e0 2020-11-28T21:54:14.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.952+0000 7fb1137fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb1140af9e0 0x7fb11405c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:14.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.952+0000 7fb1137fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb1140af9e0 0x7fb11405c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43036/0 (socket says 172.21.15.162:43036) 2020-11-28T21:54:14.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.952+0000 7fb1137fe700 1 -- 172.21.15.162:0/1161500021 learned_addr learned my addr 172.21.15.162:0/1161500021 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:14.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.952+0000 7fb112ffd700 1 --2- 172.21.15.162:0/1161500021 >> v2:172.21.15.118:3300/0 conn(0x7fb1140b04a0 0x7fb1140ae3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:14.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.952+0000 7fb1137fe700 1 -- 172.21.15.162:0/1161500021 >> v2:172.21.15.118:3300/0 conn(0x7fb1140b04a0 msgr2=0x7fb1140ae3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:14.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.952+0000 7fb1137fe700 1 --2- 172.21.15.162:0/1161500021 >> v2:172.21.15.118:3300/0 conn(0x7fb1140b04a0 0x7fb1140ae3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:14.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.952+0000 7fb1137fe700 1 -- 172.21.15.162:0/1161500021 >> v1:172.21.15.118:6789/0 conn(0x7fb1140adb60 legacy=0x7fb114143f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:14.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.952+0000 7fb1137fe700 1 -- 172.21.15.162:0/1161500021 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb11405bd90 con 0x7fb1140af9e0 2020-11-28T21:54:14.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.952+0000 7fb1137fe700 1 --2- 172.21.15.162:0/1161500021 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb1140af9e0 0x7fb11405c650 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 rx=0x7fb1140ad610 tx=0x7fb104006e80).ready entity=mon.1 client_cookie=ba37c6645ea792c1 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:14.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.952+0000 7fb1127fc700 1 -- 172.21.15.162:0/1161500021 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1040097b0 con 0x7fb1140af9e0 2020-11-28T21:54:14.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.952+0000 7fb1127fc700 1 -- 172.21.15.162:0/1161500021 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb104009910 con 0x7fb1140af9e0 2020-11-28T21:54:14.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.952+0000 7fb1127fc700 1 -- 172.21.15.162:0/1161500021 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb104009a80 con 0x7fb1140af9e0 2020-11-28T21:54:14.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.952+0000 7fb118c4e700 1 -- 172.21.15.162:0/1161500021 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb1140af9e0 msgr2=0x7fb11405c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:14.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.952+0000 7fb118c4e700 1 --2- 172.21.15.162:0/1161500021 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb1140af9e0 0x7fb11405c650 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 rx=0x7fb1140ad610 tx=0x7fb104006e80).stop 2020-11-28T21:54:14.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.956+0000 7fb118c4e700 1 -- 172.21.15.162:0/1161500021 shutdown_connections 2020-11-28T21:54:14.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.956+0000 7fb118c4e700 1 --2- 172.21.15.162:0/1161500021 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb1140af9e0 0x7fb11405c650 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:14.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.956+0000 7fb118c4e700 1 --2- 172.21.15.162:0/1161500021 >> v2:172.21.15.118:3300/0 conn(0x7fb1140b04a0 0x7fb1140ae3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:14.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.956+0000 7fb118c4e700 1 -- 172.21.15.162:0/1161500021 >> 172.21.15.162:0/1161500021 conn(0x7fb1140b17a0 msgr2=0x7fb1140b1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:14.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.956+0000 7fb118c4e700 1 -- 172.21.15.162:0/1161500021 shutdown_connections 2020-11-28T21:54:14.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.956+0000 7fb118c4e700 1 -- 172.21.15.162:0/1161500021 wait complete. 2020-11-28T21:54:14.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.956+0000 7fb118c4e700 1 Processor -- start 2020-11-28T21:54:14.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.956+0000 7fb118c4e700 1 -- start start 2020-11-28T21:54:14.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.956+0000 7fb118c4e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb1140adb60 0x7fb1140b4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:14.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.956+0000 7fb118c4e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb1140af9e0 0x7fb114151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:14.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.956+0000 7fb118c4e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb1140b04a0 0x7fb114155980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:14.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.956+0000 7fb118c4e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb114063b70 con 0x7fb1140adb60 2020-11-28T21:54:14.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.956+0000 7fb118c4e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb114155050 con 0x7fb1140af9e0 2020-11-28T21:54:14.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.956+0000 7fb118c4e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb114067290 con 0x7fb1140b04a0 2020-11-28T21:54:14.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.956+0000 7fb112ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb1140af9e0 0x7fb114151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:14.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.956+0000 7fb1137fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb1140adb60 0x7fb1140b4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:14.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.956+0000 7fb112ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb1140af9e0 0x7fb114151aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43042/0 (socket says 172.21.15.162:43042) 2020-11-28T21:54:14.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.956+0000 7fb112ffd700 1 -- 172.21.15.162:0/2187788379 learned_addr learned my addr 172.21.15.162:0/2187788379 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:14.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.956+0000 7fb113fff700 1 --2- 172.21.15.162:0/2187788379 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb1140b04a0 0x7fb114155980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:14.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.960+0000 7fb112ffd700 1 -- 172.21.15.162:0/2187788379 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb1140b04a0 msgr2=0x7fb114155980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:14.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.960+0000 7fb112ffd700 1 --2- 172.21.15.162:0/2187788379 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb1140b04a0 0x7fb114155980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:14.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.960+0000 7fb112ffd700 1 -- 172.21.15.162:0/2187788379 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb1140adb60 msgr2=0x7fb1140b4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:14.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.960+0000 7fb112ffd700 1 --2- 172.21.15.162:0/2187788379 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb1140adb60 0x7fb1140b4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:14.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.960+0000 7fb112ffd700 1 -- 172.21.15.162:0/2187788379 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb114155240 con 0x7fb1140af9e0 2020-11-28T21:54:14.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.960+0000 7fb1137fe700 1 --2- 172.21.15.162:0/2187788379 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb1140adb60 0x7fb1140b4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:54:14.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.960+0000 7fb112ffd700 1 --2- 172.21.15.162:0/2187788379 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb1140af9e0 0x7fb114151aa0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 rx=0x7fb10800a000 tx=0x7fb108008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:14.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.960+0000 7fb110ff9700 1 -- 172.21.15.162:0/2187788379 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb10800f710 con 0x7fb1140af9e0 2020-11-28T21:54:14.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.960+0000 7fb110ff9700 1 -- 172.21.15.162:0/2187788379 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10800f870 con 0x7fb1140af9e0 2020-11-28T21:54:14.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.960+0000 7fb118c4e700 1 -- 172.21.15.162:0/2187788379 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1141554e0 con 0x7fb1140af9e0 2020-11-28T21:54:14.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.960+0000 7fb110ff9700 1 -- 172.21.15.162:0/2187788379 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb10800fbd0 con 0x7fb1140af9e0 2020-11-28T21:54:14.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.960+0000 7fb118c4e700 1 -- 172.21.15.162:0/2187788379 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb114155ef0 con 0x7fb1140af9e0 2020-11-28T21:54:14.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.960+0000 7fb110ff9700 1 -- 172.21.15.162:0/2187788379 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb10801a030 con 0x7fb1140af9e0 2020-11-28T21:54:14.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.960+0000 7fb118c4e700 1 -- 172.21.15.162:0/2187788379 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb114155670 con 0x7fb1140af9e0 2020-11-28T21:54:14.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.960+0000 7fb110ff9700 1 --2- 172.21.15.162:0/2187788379 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb0f40554e0 0x7fb0f40579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:14.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.960+0000 7fb110ff9700 1 -- 172.21.15.162:0/2187788379 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb10806da90 con 0x7fb1140af9e0 2020-11-28T21:54:14.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.960+0000 7fb1137fe700 1 --2- 172.21.15.162:0/2187788379 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb0f40554e0 0x7fb0f40579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:14.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.960+0000 7fb1137fe700 1 --2- 172.21.15.162:0/2187788379 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb0f40554e0 0x7fb0f40579a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7fb10401b000 tx=0x7fb104016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:14.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:14.968+0000 7fb110ff9700 1 -- 172.21.15.162:0/2187788379 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb108042df0 con 0x7fb1140af9e0 2020-11-28T21:54:15.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.108+0000 7fb118c4e700 1 -- 172.21.15.162:0/2187788379 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb114057050 con 0x7fb1140af9e0 2020-11-28T21:54:15.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.116+0000 7fb110ff9700 1 -- 172.21.15.162:0/2187788379 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb108043e40 con 0x7fb1140af9e0 2020-11-28T21:54:15.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.116+0000 7fb118c4e700 1 -- 172.21.15.162:0/2187788379 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb0f40554e0 msgr2=0x7fb0f40579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:15.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.116+0000 7fb118c4e700 1 --2- 172.21.15.162:0/2187788379 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb0f40554e0 0x7fb0f40579a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7fb10401b000 tx=0x7fb104016000).stop 2020-11-28T21:54:15.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.116+0000 7fb118c4e700 1 -- 172.21.15.162:0/2187788379 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb1140af9e0 msgr2=0x7fb114151aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:15.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.116+0000 7fb118c4e700 1 --2- 172.21.15.162:0/2187788379 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb1140af9e0 0x7fb114151aa0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 rx=0x7fb10800a000 tx=0x7fb108008040).stop 2020-11-28T21:54:15.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.116+0000 7fb118c4e700 1 -- 172.21.15.162:0/2187788379 shutdown_connections 2020-11-28T21:54:15.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.116+0000 7fb118c4e700 1 --2- 172.21.15.162:0/2187788379 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb0f40554e0 0x7fb0f40579a0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:15.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.116+0000 7fb118c4e700 1 --2- 172.21.15.162:0/2187788379 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb1140b04a0 0x7fb114155980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:15.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.116+0000 7fb118c4e700 1 --2- 172.21.15.162:0/2187788379 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb1140adb60 0x7fb1140b4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:15.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.116+0000 7fb118c4e700 1 --2- 172.21.15.162:0/2187788379 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb1140af9e0 0x7fb114151aa0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:15.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.120+0000 7fb118c4e700 1 -- 172.21.15.162:0/2187788379 >> 172.21.15.162:0/2187788379 conn(0x7fb1140b17a0 msgr2=0x7fb1140ac500 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:15.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.120+0000 7fb118c4e700 1 -- 172.21.15.162:0/2187788379 shutdown_connections 2020-11-28T21:54:15.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.120+0000 7fb118c4e700 1 -- 172.21.15.162:0/2187788379 wait complete. 2020-11-28T21:54:15.136 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical' 2020-11-28T21:54:15.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.244+0000 7f95c7fff700 1 Processor -- start 2020-11-28T21:54:15.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.244+0000 7f95c7fff700 1 -- start start 2020-11-28T21:54:15.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.244+0000 7f95c7fff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f95c80ab990 0x7f95c805f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:15.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.244+0000 7f95c7fff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95c80a85c0 con 0x7f95c80ab1f0 2020-11-28T21:54:15.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.244+0000 7f95c7fff700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95c81448e0 con 0x7f95c80aae10 2020-11-28T21:54:15.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.244+0000 7f95c7fff700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95c8144b80 con 0x7f95c80ab990 2020-11-28T21:54:15.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.248+0000 7f95c6ffd700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f95c80aae10 0x7f95c8140800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:49892/0 (socket says 172.21.15.162:49892) 2020-11-28T21:54:15.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.248+0000 7f95c6ffd700 1 -- 172.21.15.162:0/3007720662 learned_addr learned my addr 172.21.15.162:0/3007720662 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:15.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.248+0000 7f95c67fc700 1 --2- 172.21.15.162:0/3007720662 >> v2:172.21.15.118:3300/0 conn(0x7f95c80ab990 0x7f95c805f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:15.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.248+0000 7f95c5ffb700 1 -- 172.21.15.162:0/3007720662 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1846294676 0 0) 0x7f95c81448e0 con 0x7f95c80aae10 2020-11-28T21:54:15.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.248+0000 7f95c5ffb700 1 -- 172.21.15.162:0/3007720662 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95a8003230 con 0x7f95c80aae10 2020-11-28T21:54:15.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.248+0000 7f95c5ffb700 1 -- 172.21.15.162:0/3007720662 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3927422205 0 0) 0x7f95a8003230 con 0x7f95c80aae10 2020-11-28T21:54:15.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.248+0000 7f95c5ffb700 1 -- 172.21.15.162:0/3007720662 >> v2:172.21.15.118:3300/0 conn(0x7f95c80ab990 msgr2=0x7f95c805f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:15.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.248+0000 7f95c5ffb700 1 --2- 172.21.15.162:0/3007720662 >> v2:172.21.15.118:3300/0 conn(0x7f95c80ab990 0x7f95c805f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:15.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.248+0000 7f95c5ffb700 1 -- 172.21.15.162:0/3007720662 >> v1:172.21.15.118:6789/0 conn(0x7f95c80ab1f0 legacy=0x7f95c8143ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:15.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.248+0000 7f95c5ffb700 1 -- 172.21.15.162:0/3007720662 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c80af6b0 con 0x7f95c80aae10 2020-11-28T21:54:15.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.248+0000 7f95c5ffb700 1 -- 172.21.15.162:0/3007720662 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f95b80016d0 con 0x7f95c80aae10 2020-11-28T21:54:15.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.248+0000 7f95c5ffb700 1 -- 172.21.15.162:0/3007720662 >> v1:172.21.15.162:6789/0 conn(0x7f95c80aae10 legacy=0x7f95c8140800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:15.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.248+0000 7f95c5ffb700 1 --2- 172.21.15.162:0/3007720662 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f95a8003720 0x7f95a8003b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:15.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.248+0000 7f95c5ffb700 1 --2- 172.21.15.162:0/3007720662 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f95a80044d0 0x7f95a8004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:15.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.248+0000 7f95c5ffb700 1 --2- 172.21.15.162:0/3007720662 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95a8004e40 0x7f95a800b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:15.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.248+0000 7f95c5ffb700 1 -- 172.21.15.162:0/3007720662 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c8144b80 con 0x7f95a80044d0 2020-11-28T21:54:15.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.248+0000 7f95c5ffb700 1 -- 172.21.15.162:0/3007720662 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a80041c0 con 0x7f95a8004e40 2020-11-28T21:54:15.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.248+0000 7f95c5ffb700 1 -- 172.21.15.162:0/3007720662 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95a800bc10 con 0x7f95a8003720 2020-11-28T21:54:15.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.248+0000 7f95c77fe700 1 --2- 172.21.15.162:0/3007720662 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95a8004e40 0x7f95a800b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:15.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.248+0000 7f95c67fc700 1 --2- 172.21.15.162:0/3007720662 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f95a80044d0 0x7f95a8004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:15.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.248+0000 7f95c6ffd700 1 --2- 172.21.15.162:0/3007720662 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f95a8003720 0x7f95a8003b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:15.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.248+0000 7f95c77fe700 1 -- 172.21.15.162:0/3007720662 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f95a8003720 msgr2=0x7f95a8003b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:15.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.248+0000 7f95c77fe700 1 --2- 172.21.15.162:0/3007720662 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f95a8003720 0x7f95a8003b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:15.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.248+0000 7f95c77fe700 1 -- 172.21.15.162:0/3007720662 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f95a80044d0 msgr2=0x7f95a8004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:15.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.248+0000 7f95c77fe700 1 --2- 172.21.15.162:0/3007720662 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f95a80044d0 0x7f95a8004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:15.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.248+0000 7f95c77fe700 1 -- 172.21.15.162:0/3007720662 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a8003230 con 0x7f95a8004e40 2020-11-28T21:54:15.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.248+0000 7f95c6ffd700 1 --2- 172.21.15.162:0/3007720662 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f95a8003720 0x7f95a8003b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:54:15.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.248+0000 7f95c77fe700 1 -- 172.21.15.162:0/3007720662 reap_dead start 2020-11-28T21:54:15.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.248+0000 7f95c77fe700 1 --2- 172.21.15.162:0/3007720662 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95a8004e40 0x7f95a800b720 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 rx=0x7f95c0001360 tx=0x7f95c0007ec0).ready entity=mon.1 client_cookie=71bff62e1c782ea server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:15.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.248+0000 7f95c5ffb700 1 -- 172.21.15.162:0/3007720662 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95c000cc00 con 0x7f95a8004e40 2020-11-28T21:54:15.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.248+0000 7f95c5ffb700 1 -- 172.21.15.162:0/3007720662 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c000cd60 con 0x7f95a8004e40 2020-11-28T21:54:15.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.248+0000 7f95c5ffb700 1 -- 172.21.15.162:0/3007720662 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95c0013600 con 0x7f95a8004e40 2020-11-28T21:54:15.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.248+0000 7f95c7fff700 1 -- 172.21.15.162:0/3007720662 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95a8004e40 msgr2=0x7f95a800b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:15.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.248+0000 7f95c7fff700 1 --2- 172.21.15.162:0/3007720662 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95a8004e40 0x7f95a800b720 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 rx=0x7f95c0001360 tx=0x7f95c0007ec0).stop 2020-11-28T21:54:15.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.248+0000 7f95c7fff700 1 -- 172.21.15.162:0/3007720662 shutdown_connections 2020-11-28T21:54:15.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.248+0000 7f95c7fff700 1 --2- 172.21.15.162:0/3007720662 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95a8004e40 0x7f95a800b720 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:15.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.248+0000 7f95c7fff700 1 -- 172.21.15.162:0/3007720662 >> 172.21.15.162:0/3007720662 conn(0x7f95c805b970 msgr2=0x7f95c805bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:15.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.248+0000 7f95c7fff700 1 -- 172.21.15.162:0/3007720662 shutdown_connections 2020-11-28T21:54:15.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.248+0000 7f95c7fff700 1 -- 172.21.15.162:0/3007720662 wait complete. 2020-11-28T21:54:15.273 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.252+0000 7f95c7fff700 1 Processor -- start 2020-11-28T21:54:15.273 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.252+0000 7f95c7fff700 1 -- start start 2020-11-28T21:54:15.273 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.252+0000 7f95c7fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f95a8004e40 0x7f95c8063680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:15.274 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.252+0000 7f95c7fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95c8067100 0x7f95c80ad820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:15.274 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.252+0000 7f95c7fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f95c80add10 0x7f95c8154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:15.274 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.252+0000 7f95c7fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c8154a90 con 0x7f95a8004e40 2020-11-28T21:54:15.274 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.252+0000 7f95c7fff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c80ae5c0 con 0x7f95c8067100 2020-11-28T21:54:15.275 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.252+0000 7f95c7fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95c80aecf0 con 0x7f95c80add10 2020-11-28T21:54:15.275 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.252+0000 7f95c67fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95c8067100 0x7f95c80ad820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:15.275 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.252+0000 7f95c67fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95c8067100 0x7f95c80ad820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43062/0 (socket says 172.21.15.162:43062) 2020-11-28T21:54:15.276 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.252+0000 7f95c6ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f95a8004e40 0x7f95c8063680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:15.276 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.252+0000 7f95c67fc700 1 -- 172.21.15.162:0/3171758144 learned_addr learned my addr 172.21.15.162:0/3171758144 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:15.276 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.252+0000 7f95c77fe700 1 --2- 172.21.15.162:0/3171758144 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f95c80add10 0x7f95c8154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:15.277 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.252+0000 7f95c67fc700 1 -- 172.21.15.162:0/3171758144 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f95c80add10 msgr2=0x7f95c8154e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:15.277 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.252+0000 7f95c67fc700 1 --2- 172.21.15.162:0/3171758144 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f95c80add10 0x7f95c8154e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:15.277 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.252+0000 7f95c67fc700 1 -- 172.21.15.162:0/3171758144 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f95a8004e40 msgr2=0x7f95c8063680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:15.277 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.252+0000 7f95c67fc700 1 --2- 172.21.15.162:0/3171758144 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f95a8004e40 0x7f95c8063680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:15.278 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.252+0000 7f95c67fc700 1 -- 172.21.15.162:0/3171758144 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c80ae7f0 con 0x7f95c8067100 2020-11-28T21:54:15.278 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.252+0000 7f95c67fc700 1 --2- 172.21.15.162:0/3171758144 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95c8067100 0x7f95c80ad820 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 rx=0x7f95b0006750 tx=0x7f95b0008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:15.278 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.252+0000 7f95bffff700 1 -- 172.21.15.162:0/3171758144 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95b000f910 con 0x7f95c8067100 2020-11-28T21:54:15.278 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.252+0000 7f95bffff700 1 -- 172.21.15.162:0/3171758144 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95b000fa70 con 0x7f95c8067100 2020-11-28T21:54:15.279 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.252+0000 7f95c7fff700 1 -- 172.21.15.162:0/3171758144 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95c80aea90 con 0x7f95c8067100 2020-11-28T21:54:15.279 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.252+0000 7f95bffff700 1 -- 172.21.15.162:0/3171758144 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95b000fbe0 con 0x7f95c8067100 2020-11-28T21:54:15.279 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.252+0000 7f95c7fff700 1 -- 172.21.15.162:0/3171758144 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95c80aa690 con 0x7f95c8067100 2020-11-28T21:54:15.279 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.256+0000 7f95c7fff700 1 -- 172.21.15.162:0/3171758144 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95c80ae340 con 0x7f95c8067100 2020-11-28T21:54:15.279 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.256+0000 7f95bffff700 1 -- 172.21.15.162:0/3171758144 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f95b0024070 con 0x7f95c8067100 2020-11-28T21:54:15.281 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.256+0000 7f95bffff700 1 --2- 172.21.15.162:0/3171758144 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f95ac0553b0 0x7f95ac057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:15.281 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.256+0000 7f95bffff700 1 -- 172.21.15.162:0/3171758144 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f95b006e760 con 0x7f95c8067100 2020-11-28T21:54:15.281 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.256+0000 7f95c6ffd700 1 --2- 172.21.15.162:0/3171758144 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f95ac0553b0 0x7f95ac057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:15.281 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.256+0000 7f95c6ffd700 1 --2- 172.21.15.162:0/3171758144 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f95ac0553b0 0x7f95ac057870 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f95b800d000 tx=0x7f95b8007040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:15.282 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.264+0000 7f95bffff700 1 -- 172.21.15.162:0/3171758144 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f95b0044950 con 0x7f95c8067100 2020-11-28T21:54:15.453 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:15.448+0000 7f95c7fff700 1 -- 172.21.15.162:0/3171758144 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical"]} v 0) v1 -- 0x7f95c8057050 con 0x7f95c8067100 2020-11-28T21:54:16.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.008+0000 7f95bffff700 1 -- 172.21.15.162:0/3171758144 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical"]}]=0 v341) v1 ==== 138+0+0 (secure 0 0 0) 0x7f95b0047f70 con 0x7f95c8067100 2020-11-28T21:54:16.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.012+0000 7f95c7fff700 1 -- 172.21.15.162:0/3171758144 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f95ac0553b0 msgr2=0x7f95ac057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:16.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.012+0000 7f95c7fff700 1 --2- 172.21.15.162:0/3171758144 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f95ac0553b0 0x7f95ac057870 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f95b800d000 tx=0x7f95b8007040).stop 2020-11-28T21:54:16.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.012+0000 7f95c7fff700 1 -- 172.21.15.162:0/3171758144 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95c8067100 msgr2=0x7f95c80ad820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:16.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.012+0000 7f95c7fff700 1 --2- 172.21.15.162:0/3171758144 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95c8067100 0x7f95c80ad820 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 rx=0x7f95b0006750 tx=0x7f95b0008040).stop 2020-11-28T21:54:16.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.012+0000 7f95c7fff700 1 -- 172.21.15.162:0/3171758144 shutdown_connections 2020-11-28T21:54:16.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.012+0000 7f95c7fff700 1 --2- 172.21.15.162:0/3171758144 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f95ac0553b0 0x7f95ac057870 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:16.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.012+0000 7f95c7fff700 1 --2- 172.21.15.162:0/3171758144 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f95c80add10 0x7f95c8154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:16.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.012+0000 7f95c7fff700 1 --2- 172.21.15.162:0/3171758144 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f95a8004e40 0x7f95c8063680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:16.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.012+0000 7f95c7fff700 1 --2- 172.21.15.162:0/3171758144 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95c8067100 0x7f95c80ad820 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:16.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.012+0000 7f95c7fff700 1 -- 172.21.15.162:0/3171758144 >> 172.21.15.162:0/3171758144 conn(0x7f95c805b970 msgr2=0x7f95c813ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:16.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.012+0000 7f95c7fff700 1 -- 172.21.15.162:0/3171758144 shutdown_connections 2020-11-28T21:54:16.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.012+0000 7f95c7fff700 1 -- 172.21.15.162:0/3171758144 wait complete. 2020-11-28T21:54:16.029 INFO:tasks.cephfs_test_runner:test_dump_inode_hexademical (tasks.cephfs.test_misc.TestMisc) ... ok 2020-11-28T21:54:16.029 INFO:tasks.cephfs_test_runner:Starting test: test_filtered_df (tasks.cephfs.test_misc.TestMisc) 2020-11-28T21:54:16.030 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_misc.TestMisc.test_filtered_df' 2020-11-28T21:54:16.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.144+0000 7f28dcf92700 1 Processor -- start 2020-11-28T21:54:16.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.144+0000 7f28dcf92700 1 -- start start 2020-11-28T21:54:16.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.148+0000 7f28dcf92700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f28d80b39a0 0x7f28d8062050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:16.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.148+0000 7f28dcf92700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28d8146cf0 con 0x7f28d80610b0 2020-11-28T21:54:16.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.148+0000 7f28dcf92700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28d80b2fd0 con 0x7f28d8062cd0 2020-11-28T21:54:16.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.148+0000 7f28dcf92700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28d8143990 con 0x7f28d80b39a0 2020-11-28T21:54:16.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.148+0000 7f28d6ffd700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f28d8062cd0 0x7f28d8145ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:49912/0 (socket says 172.21.15.162:49912) 2020-11-28T21:54:16.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.148+0000 7f28d6ffd700 1 -- 172.21.15.162:0/3714577410 learned_addr learned my addr 172.21.15.162:0/3714577410 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:16.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.148+0000 7f28d77fe700 1 --2- 172.21.15.162:0/3714577410 >> v2:172.21.15.162:3300/0 conn(0x7f28d80b39a0 0x7f28d8062050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:16.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.148+0000 7f28d67fc700 1 -- 172.21.15.162:0/3714577410 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1079653679 0 0) 0x7f28d80b2fd0 con 0x7f28d8062cd0 2020-11-28T21:54:16.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.148+0000 7f28d67fc700 1 -- 172.21.15.162:0/3714577410 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28b8003230 con 0x7f28d8062cd0 2020-11-28T21:54:16.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.148+0000 7f28d77fe700 1 -- 172.21.15.162:0/3714577410 >> v1:172.21.15.162:6789/0 conn(0x7f28d8062cd0 legacy=0x7f28d8145ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:16.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.148+0000 7f28d77fe700 1 -- 172.21.15.162:0/3714577410 >> v1:172.21.15.118:6789/0 conn(0x7f28d80610b0 legacy=0x7f28d81465e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:16.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.148+0000 7f28d77fe700 1 -- 172.21.15.162:0/3714577410 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d80b3270 con 0x7f28d80b39a0 2020-11-28T21:54:16.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.148+0000 7f28d77fe700 1 --2- 172.21.15.162:0/3714577410 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28d80b39a0 0x7f28d8062050 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 rx=0x7f28c8002f30 tx=0x7f28c8006dc0).ready entity=mon.1 client_cookie=27aa38851221116e server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:16.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.148+0000 7f28d67fc700 1 -- 172.21.15.162:0/3714577410 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28c8009800 con 0x7f28d80b39a0 2020-11-28T21:54:16.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.148+0000 7f28d67fc700 1 -- 172.21.15.162:0/3714577410 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c8009960 con 0x7f28d80b39a0 2020-11-28T21:54:16.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.148+0000 7f28d67fc700 1 -- 172.21.15.162:0/3714577410 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28c8009cc0 con 0x7f28d80b39a0 2020-11-28T21:54:16.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.148+0000 7f28dcf92700 1 -- 172.21.15.162:0/3714577410 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28d80b39a0 msgr2=0x7f28d8062050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:16.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.148+0000 7f28dcf92700 1 --2- 172.21.15.162:0/3714577410 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28d80b39a0 0x7f28d8062050 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 rx=0x7f28c8002f30 tx=0x7f28c8006dc0).stop 2020-11-28T21:54:16.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.148+0000 7f28dcf92700 1 -- 172.21.15.162:0/3714577410 shutdown_connections 2020-11-28T21:54:16.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.148+0000 7f28dcf92700 1 --2- 172.21.15.162:0/3714577410 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28d80b39a0 0x7f28d8062050 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:16.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.148+0000 7f28dcf92700 1 -- 172.21.15.162:0/3714577410 >> 172.21.15.162:0/3714577410 conn(0x7f28d805b970 msgr2=0x7f28d805bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:16.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.148+0000 7f28dcf92700 1 -- 172.21.15.162:0/3714577410 shutdown_connections 2020-11-28T21:54:16.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.148+0000 7f28dcf92700 1 -- 172.21.15.162:0/3714577410 wait complete. 2020-11-28T21:54:16.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.148+0000 7f28dcf92700 1 Processor -- start 2020-11-28T21:54:16.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.148+0000 7f28dcf92700 1 -- start start 2020-11-28T21:54:16.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.152+0000 7f28dcf92700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28d80610b0 0x7f28d8141e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:16.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.152+0000 7f28dcf92700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28d8062cd0 0x7f28d813fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:16.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.152+0000 7f28dcf92700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28d80b39a0 0x7f28d813ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:16.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.152+0000 7f28dcf92700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d8142b00 con 0x7f28d80610b0 2020-11-28T21:54:16.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.152+0000 7f28dcf92700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d8148c30 con 0x7f28d80b39a0 2020-11-28T21:54:16.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.152+0000 7f28dcf92700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28d80ae1c0 con 0x7f28d8062cd0 2020-11-28T21:54:16.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.152+0000 7f28d7fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28d80b39a0 0x7f28d813ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:16.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.152+0000 7f28d7fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28d80b39a0 0x7f28d813ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43076/0 (socket says 172.21.15.162:43076) 2020-11-28T21:54:16.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.152+0000 7f28d7fff700 1 -- 172.21.15.162:0/788904346 learned_addr learned my addr 172.21.15.162:0/788904346 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:16.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.152+0000 7f28d6ffd700 1 --2- 172.21.15.162:0/788904346 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28d8062cd0 0x7f28d813fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:16.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.152+0000 7f28d77fe700 1 --2- 172.21.15.162:0/788904346 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28d80610b0 0x7f28d8141e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:16.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.152+0000 7f28d7fff700 1 -- 172.21.15.162:0/788904346 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28d8062cd0 msgr2=0x7f28d813fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:16.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.152+0000 7f28d7fff700 1 --2- 172.21.15.162:0/788904346 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28d8062cd0 0x7f28d813fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:16.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.152+0000 7f28d7fff700 1 -- 172.21.15.162:0/788904346 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28d80610b0 msgr2=0x7f28d8141e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:16.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.152+0000 7f28d7fff700 1 --2- 172.21.15.162:0/788904346 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28d80610b0 0x7f28d8141e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:16.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.152+0000 7f28d7fff700 1 -- 172.21.15.162:0/788904346 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d80ae460 con 0x7f28d80b39a0 2020-11-28T21:54:16.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.152+0000 7f28d6ffd700 1 --2- 172.21.15.162:0/788904346 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28d8062cd0 0x7f28d813fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:54:16.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.152+0000 7f28d7fff700 1 --2- 172.21.15.162:0/788904346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28d80b39a0 0x7f28d813ff80 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 rx=0x7f28d8068650 tx=0x7f28d0007f10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:16.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.152+0000 7f28d4ff9700 1 -- 172.21.15.162:0/788904346 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28d000a7b0 con 0x7f28d80b39a0 2020-11-28T21:54:16.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.152+0000 7f28d4ff9700 1 -- 172.21.15.162:0/788904346 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28d000a910 con 0x7f28d80b39a0 2020-11-28T21:54:16.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.152+0000 7f28dcf92700 1 -- 172.21.15.162:0/788904346 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28d8140470 con 0x7f28d80b39a0 2020-11-28T21:54:16.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.152+0000 7f28d4ff9700 1 -- 172.21.15.162:0/788904346 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28d000ac70 con 0x7f28d80b39a0 2020-11-28T21:54:16.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.152+0000 7f28dcf92700 1 -- 172.21.15.162:0/788904346 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28d81486b0 con 0x7f28d80b39a0 2020-11-28T21:54:16.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.152+0000 7f28dcf92700 1 -- 172.21.15.162:0/788904346 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28d8149c30 con 0x7f28d80b39a0 2020-11-28T21:54:16.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.152+0000 7f28d4ff9700 1 -- 172.21.15.162:0/788904346 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f28d0019030 con 0x7f28d80b39a0 2020-11-28T21:54:16.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.156+0000 7f28d4ff9700 1 --2- 172.21.15.162:0/788904346 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f28bc0553b0 0x7f28bc057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:16.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.156+0000 7f28d4ff9700 1 -- 172.21.15.162:0/788904346 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f28d006d7f0 con 0x7f28d80b39a0 2020-11-28T21:54:16.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.156+0000 7f28d77fe700 1 --2- 172.21.15.162:0/788904346 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f28bc0553b0 0x7f28bc057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:16.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.156+0000 7f28d77fe700 1 --2- 172.21.15.162:0/788904346 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f28bc0553b0 0x7f28bc057870 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7f28c8018000 tx=0x7f28c8015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:16.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.160+0000 7f28d4ff9700 1 -- 172.21.15.162:0/788904346 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f28d0043af0 con 0x7f28d80b39a0 2020-11-28T21:54:16.355 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:16.352+0000 7f28dcf92700 1 -- 172.21.15.162:0/788904346 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_filtered_df"]} v 0) v1 -- 0x7f28d8057050 con 0x7f28d80b39a0 2020-11-28T21:54:17.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.008+0000 7f28d4ff9700 1 -- 172.21.15.162:0/788904346 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_filtered_df"]}]=0 v342) v1 ==== 130+0+0 (secure 0 0 0) 0x7f28d0047110 con 0x7f28d80b39a0 2020-11-28T21:54:17.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.012+0000 7f28dcf92700 1 -- 172.21.15.162:0/788904346 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f28bc0553b0 msgr2=0x7f28bc057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:17.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.012+0000 7f28dcf92700 1 --2- 172.21.15.162:0/788904346 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f28bc0553b0 0x7f28bc057870 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7f28c8018000 tx=0x7f28c8015000).stop 2020-11-28T21:54:17.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.012+0000 7f28dcf92700 1 -- 172.21.15.162:0/788904346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28d80b39a0 msgr2=0x7f28d813ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:17.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.012+0000 7f28dcf92700 1 --2- 172.21.15.162:0/788904346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28d80b39a0 0x7f28d813ff80 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 rx=0x7f28d8068650 tx=0x7f28d0007f10).stop 2020-11-28T21:54:17.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.012+0000 7f28dcf92700 1 -- 172.21.15.162:0/788904346 shutdown_connections 2020-11-28T21:54:17.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.012+0000 7f28dcf92700 1 --2- 172.21.15.162:0/788904346 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f28bc0553b0 0x7f28bc057870 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:17.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.012+0000 7f28dcf92700 1 --2- 172.21.15.162:0/788904346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28d80b39a0 0x7f28d813ff80 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:17.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.012+0000 7f28dcf92700 1 --2- 172.21.15.162:0/788904346 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28d80610b0 0x7f28d8141e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:17.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.012+0000 7f28dcf92700 1 --2- 172.21.15.162:0/788904346 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28d8062cd0 0x7f28d813fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:17.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.012+0000 7f28dcf92700 1 -- 172.21.15.162:0/788904346 >> 172.21.15.162:0/788904346 conn(0x7f28d805b970 msgr2=0x7f28d80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:17.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.012+0000 7f28dcf92700 1 -- 172.21.15.162:0/788904346 shutdown_connections 2020-11-28T21:54:17.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.012+0000 7f28dcf92700 1 -- 172.21.15.162:0/788904346 wait complete. 2020-11-28T21:54:17.033 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config set mon mon_allow_pool_delete True 2020-11-28T21:54:17.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.160+0000 7fd782b2c700 1 Processor -- start 2020-11-28T21:54:17.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.160+0000 7fd782b2c700 1 -- start start 2020-11-28T21:54:17.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.160+0000 7fd782b2c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd77c05cb20 0x7fd77c141220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:17.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.160+0000 7fd782b2c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd77c0b00f0 0x7fd77c0ab4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:17.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.160+0000 7fd782b2c700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd77c0ab990 0x7fd77c096ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:17.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.160+0000 7fd782b2c700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd77c141710 con 0x7fd77c0b00f0 2020-11-28T21:54:17.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.160+0000 7fd782b2c700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd77c0aa820 con 0x7fd77c05cb20 2020-11-28T21:54:17.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.160+0000 7fd782b2c700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c0aaac0 con 0x7fd77c0ab990 2020-11-28T21:54:17.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.160+0000 7fd7808c8700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd77c05cb20 0x7fd77c141220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:17.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.160+0000 7fd7808c8700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd77c05cb20 0x7fd77c141220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43082/0 (socket says 172.21.15.162:43082) 2020-11-28T21:54:17.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.160+0000 7fd7808c8700 1 -- 172.21.15.162:0/3429718976 learned_addr learned my addr 172.21.15.162:0/3429718976 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:17.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.160+0000 7fd7810c9700 1 --2- 172.21.15.162:0/3429718976 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd77c0ab990 0x7fd77c096ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:17.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.160+0000 7fd77bfff700 1 --2- 172.21.15.162:0/3429718976 >> v2:172.21.15.118:3300/0 conn(0x7fd77c0b00f0 0x7fd77c0ab4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:17.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.160+0000 7fd7808c8700 1 -- 172.21.15.162:0/3429718976 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd77c0ab990 msgr2=0x7fd77c096ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:17.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.160+0000 7fd7808c8700 1 --2- 172.21.15.162:0/3429718976 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd77c0ab990 0x7fd77c096ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:17.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.160+0000 7fd7808c8700 1 -- 172.21.15.162:0/3429718976 >> v2:172.21.15.118:3300/0 conn(0x7fd77c0b00f0 msgr2=0x7fd77c0ab4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:17.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.160+0000 7fd7808c8700 1 --2- 172.21.15.162:0/3429718976 >> v2:172.21.15.118:3300/0 conn(0x7fd77c0b00f0 0x7fd77c0ab4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:17.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.160+0000 7fd7808c8700 1 -- 172.21.15.162:0/3429718976 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd77c0aad60 con 0x7fd77c05cb20 2020-11-28T21:54:17.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.160+0000 7fd7808c8700 1 --2- 172.21.15.162:0/3429718976 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd77c05cb20 0x7fd77c141220 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 rx=0x7fd770002f30 tx=0x7fd770006e80).ready entity=mon.1 client_cookie=51f7335b30bdc3fa server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:17.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.160+0000 7fd77b7fe700 1 -- 172.21.15.162:0/3429718976 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd770009640 con 0x7fd77c05cb20 2020-11-28T21:54:17.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.160+0000 7fd77b7fe700 1 -- 172.21.15.162:0/3429718976 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7700097a0 con 0x7fd77c05cb20 2020-11-28T21:54:17.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.160+0000 7fd77b7fe700 1 -- 172.21.15.162:0/3429718976 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd770009b00 con 0x7fd77c05cb20 2020-11-28T21:54:17.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.160+0000 7fd782b2c700 1 -- 172.21.15.162:0/3429718976 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd77c05cb20 msgr2=0x7fd77c141220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:17.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.160+0000 7fd782b2c700 1 --2- 172.21.15.162:0/3429718976 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd77c05cb20 0x7fd77c141220 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 rx=0x7fd770002f30 tx=0x7fd770006e80).stop 2020-11-28T21:54:17.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.164+0000 7fd782b2c700 1 -- 172.21.15.162:0/3429718976 shutdown_connections 2020-11-28T21:54:17.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.164+0000 7fd782b2c700 1 --2- 172.21.15.162:0/3429718976 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd77c0ab990 0x7fd77c096ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:17.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.164+0000 7fd782b2c700 1 --2- 172.21.15.162:0/3429718976 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd77c05cb20 0x7fd77c141220 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:17.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.164+0000 7fd782b2c700 1 --2- 172.21.15.162:0/3429718976 >> v2:172.21.15.118:3300/0 conn(0x7fd77c0b00f0 0x7fd77c0ab4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:17.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.164+0000 7fd782b2c700 1 -- 172.21.15.162:0/3429718976 >> 172.21.15.162:0/3429718976 conn(0x7fd77c0b6220 msgr2=0x7fd77c0b21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:17.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.164+0000 7fd782b2c700 1 -- 172.21.15.162:0/3429718976 shutdown_connections 2020-11-28T21:54:17.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.164+0000 7fd782b2c700 1 -- 172.21.15.162:0/3429718976 wait complete. 2020-11-28T21:54:17.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.164+0000 7fd782b2c700 1 Processor -- start 2020-11-28T21:54:17.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.164+0000 7fd782b2c700 1 -- start start 2020-11-28T21:54:17.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.164+0000 7fd782b2c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd77c05cb20 0x7fd77c0b47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:17.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.164+0000 7fd782b2c700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd77c0ab990 0x7fd77c0b3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:17.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.164+0000 7fd782b2c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd77c0b00f0 0x7fd77c060560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:17.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.164+0000 7fd782b2c700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c0b3840 con 0x7fd77c0b00f0 2020-11-28T21:54:17.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.164+0000 7fd782b2c700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c1521d0 con 0x7fd77c05cb20 2020-11-28T21:54:17.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.164+0000 7fd782b2c700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c152470 con 0x7fd77c0ab990 2020-11-28T21:54:17.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.164+0000 7fd7808c8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd77c05cb20 0x7fd77c0b47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:17.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.164+0000 7fd7808c8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd77c05cb20 0x7fd77c0b47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43088/0 (socket says 172.21.15.162:43088) 2020-11-28T21:54:17.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.164+0000 7fd7808c8700 1 -- 172.21.15.162:0/3933407439 learned_addr learned my addr 172.21.15.162:0/3933407439 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:17.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.168+0000 7fd77bfff700 1 --2- 172.21.15.162:0/3933407439 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd77c0ab990 0x7fd77c0b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:17.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.168+0000 7fd7810c9700 1 --2- 172.21.15.162:0/3933407439 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd77c0b00f0 0x7fd77c060560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:17.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.168+0000 7fd7808c8700 1 -- 172.21.15.162:0/3933407439 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd77c0ab990 msgr2=0x7fd77c0b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:17.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.168+0000 7fd7808c8700 1 --2- 172.21.15.162:0/3933407439 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd77c0ab990 0x7fd77c0b3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:17.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.168+0000 7fd7808c8700 1 -- 172.21.15.162:0/3933407439 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd77c0b00f0 msgr2=0x7fd77c060560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:17.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.168+0000 7fd7808c8700 1 --2- 172.21.15.162:0/3933407439 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd77c0b00f0 0x7fd77c060560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:17.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.168+0000 7fd7808c8700 1 -- 172.21.15.162:0/3933407439 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd77c0b4e00 con 0x7fd77c05cb20 2020-11-28T21:54:17.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.168+0000 7fd7810c9700 1 --2- 172.21.15.162:0/3933407439 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd77c0b00f0 0x7fd77c060560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:54:17.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.168+0000 7fd7808c8700 1 --2- 172.21.15.162:0/3933407439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd77c05cb20 0x7fd77c0b47f0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 rx=0x7fd770002f30 tx=0x7fd770015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:17.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.168+0000 7fd779ffb700 1 -- 172.21.15.162:0/3933407439 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd770011ca0 con 0x7fd77c05cb20 2020-11-28T21:54:17.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.168+0000 7fd782b2c700 1 -- 172.21.15.162:0/3933407439 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd77c0b50a0 con 0x7fd77c05cb20 2020-11-28T21:54:17.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.168+0000 7fd779ffb700 1 -- 172.21.15.162:0/3933407439 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd770011e00 con 0x7fd77c05cb20 2020-11-28T21:54:17.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.168+0000 7fd779ffb700 1 -- 172.21.15.162:0/3933407439 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd77001d640 con 0x7fd77c05cb20 2020-11-28T21:54:17.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.168+0000 7fd782b2c700 1 -- 172.21.15.162:0/3933407439 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd77c09b460 con 0x7fd77c05cb20 2020-11-28T21:54:17.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.168+0000 7fd782b2c700 1 -- 172.21.15.162:0/3933407439 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd77c0b5230 con 0x7fd77c05cb20 2020-11-28T21:54:17.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.168+0000 7fd779ffb700 1 -- 172.21.15.162:0/3933407439 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd77000f070 con 0x7fd77c05cb20 2020-11-28T21:54:17.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.168+0000 7fd779ffb700 1 --2- 172.21.15.162:0/3933407439 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd7640554e0 0x7fd7640579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:17.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.168+0000 7fd779ffb700 1 -- 172.21.15.162:0/3933407439 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd77006f9f0 con 0x7fd77c05cb20 2020-11-28T21:54:17.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.168+0000 7fd77bfff700 1 --2- 172.21.15.162:0/3933407439 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd7640554e0 0x7fd7640579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:17.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.172+0000 7fd77bfff700 1 --2- 172.21.15.162:0/3933407439 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd7640554e0 0x7fd7640579a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7fd76800b000 tx=0x7fd768006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:17.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.176+0000 7fd779ffb700 1 -- 172.21.15.162:0/3933407439 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd770071e60 con 0x7fd77c05cb20 2020-11-28T21:54:17.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.320+0000 7fd782b2c700 1 -- 172.21.15.162:0/3933407439 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7fd77c057050 con 0x7fd77c05cb20 2020-11-28T21:54:17.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.332+0000 7fd779ffb700 1 -- 172.21.15.162:0/3933407439 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd770049730 con 0x7fd77c05cb20 2020-11-28T21:54:17.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.332+0000 7fd782b2c700 1 -- 172.21.15.162:0/3933407439 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd7640554e0 msgr2=0x7fd7640579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:17.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.332+0000 7fd782b2c700 1 --2- 172.21.15.162:0/3933407439 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd7640554e0 0x7fd7640579a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7fd76800b000 tx=0x7fd768006dc0).stop 2020-11-28T21:54:17.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.332+0000 7fd782b2c700 1 -- 172.21.15.162:0/3933407439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd77c05cb20 msgr2=0x7fd77c0b47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:17.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.332+0000 7fd782b2c700 1 --2- 172.21.15.162:0/3933407439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd77c05cb20 0x7fd77c0b47f0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 rx=0x7fd770002f30 tx=0x7fd770015000).stop 2020-11-28T21:54:17.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.332+0000 7fd782b2c700 1 -- 172.21.15.162:0/3933407439 shutdown_connections 2020-11-28T21:54:17.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.332+0000 7fd782b2c700 1 --2- 172.21.15.162:0/3933407439 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd7640554e0 0x7fd7640579a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:17.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.332+0000 7fd782b2c700 1 --2- 172.21.15.162:0/3933407439 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd77c0b00f0 0x7fd77c060560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:17.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.332+0000 7fd782b2c700 1 --2- 172.21.15.162:0/3933407439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd77c05cb20 0x7fd77c0b47f0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:17.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.332+0000 7fd782b2c700 1 --2- 172.21.15.162:0/3933407439 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd77c0ab990 0x7fd77c0b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:17.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.332+0000 7fd782b2c700 1 -- 172.21.15.162:0/3933407439 >> 172.21.15.162:0/3933407439 conn(0x7fd77c0b6220 msgr2=0x7fd77c094760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:17.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.332+0000 7fd782b2c700 1 -- 172.21.15.162:0/3933407439 shutdown_connections 2020-11-28T21:54:17.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.332+0000 7fd782b2c700 1 -- 172.21.15.162:0/3933407439 wait complete. 2020-11-28T21:54:17.349 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-28T21:54:17.350 INFO:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:54:17.368 INFO:teuthology.orchestra.run.smithi118:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:54:17.386 INFO:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:54:17.404 INFO:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:54:17.422 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-28T21:54:17.423 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:17.423 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:54:17.424 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:54:17.469 INFO:teuthology.orchestra.run.smithi192.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2020-11-28T21:54:17.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:54:17.470 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2020-11-28T21:54:17.471 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:17.471 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:54:17.472 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:54:17.478 INFO:teuthology.orchestra.run.smithi096.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-11-28T21:54:17.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:54:17.480 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2020-11-28T21:54:17.480 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2020-11-28T21:54:17.601 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.596+0000 7fa38d97f700 1 Processor -- start 2020-11-28T21:54:17.601 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.596+0000 7fa38d97f700 1 -- start start 2020-11-28T21:54:17.605 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.596+0000 7fa38d97f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa3880994f0 0x7fa3880b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:17.606 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.596+0000 7fa38d97f700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa388098f10 con 0x7fa388060710 2020-11-28T21:54:17.606 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.596+0000 7fa38d97f700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3880b0680 con 0x7fa388098b30 2020-11-28T21:54:17.606 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.596+0000 7fa38d97f700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3880b1030 con 0x7fa3880994f0 2020-11-28T21:54:17.606 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.596+0000 7fa38c97d700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fa388098b30 0x7fa388060000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:49944/0 (socket says 172.21.15.162:49944) 2020-11-28T21:54:17.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.596+0000 7fa38c97d700 1 -- 172.21.15.162:0/3402225396 learned_addr learned my addr 172.21.15.162:0/3402225396 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:17.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.600+0000 7fa387fff700 1 --2- 172.21.15.162:0/3402225396 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa3880994f0 0x7fa3880b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:17.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.600+0000 7fa3877fe700 1 -- 172.21.15.162:0/3402225396 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2678125272 0 0) 0x7fa3880b0680 con 0x7fa388098b30 2020-11-28T21:54:17.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.600+0000 7fa3877fe700 1 -- 172.21.15.162:0/3402225396 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa36c003230 con 0x7fa388098b30 2020-11-28T21:54:17.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.600+0000 7fa3877fe700 1 -- 172.21.15.162:0/3402225396 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3786423456 0 0) 0x7fa388098f10 con 0x7fa388060710 2020-11-28T21:54:17.608 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.600+0000 7fa3877fe700 1 -- 172.21.15.162:0/3402225396 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3880b0680 con 0x7fa388060710 2020-11-28T21:54:17.608 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.600+0000 7fa3877fe700 1 -- 172.21.15.162:0/3402225396 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2493565223 0 0) 0x7fa36c003230 con 0x7fa388098b30 2020-11-28T21:54:17.608 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.600+0000 7fa3877fe700 1 -- 172.21.15.162:0/3402225396 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa3880994f0 msgr2=0x7fa3880b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:17.608 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.600+0000 7fa3877fe700 1 --2- 172.21.15.162:0/3402225396 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa3880994f0 0x7fa3880b0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:17.608 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.600+0000 7fa3877fe700 1 -- 172.21.15.162:0/3402225396 >> v1:172.21.15.118:6789/0 conn(0x7fa388060710 legacy=0x7fa3880b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:17.609 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.600+0000 7fa3877fe700 1 -- 172.21.15.162:0/3402225396 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3880b0920 con 0x7fa388098b30 2020-11-28T21:54:17.609 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.600+0000 7fa3877fe700 1 -- 172.21.15.162:0/3402225396 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7fa380001a50 con 0x7fa388098b30 2020-11-28T21:54:17.609 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.600+0000 7fa3877fe700 1 -- 172.21.15.162:0/3402225396 >> v1:172.21.15.162:6789/0 conn(0x7fa388098b30 legacy=0x7fa388060000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:17.609 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.600+0000 7fa3877fe700 1 --2- 172.21.15.162:0/3402225396 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa3880994f0 0x7fa36c0038b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:17.609 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.600+0000 7fa3877fe700 1 --2- 172.21.15.162:0/3402225396 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa36c003da0 0x7fa36c0044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:17.610 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.600+0000 7fa3877fe700 1 --2- 172.21.15.162:0/3402225396 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa36c0049c0 0x7fa36c004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:17.610 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.600+0000 7fa3877fe700 1 -- 172.21.15.162:0/3402225396 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3880b1030 con 0x7fa36c0049c0 2020-11-28T21:54:17.610 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.600+0000 7fa3877fe700 1 -- 172.21.15.162:0/3402225396 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36c005330 con 0x7fa36c003da0 2020-11-28T21:54:17.610 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.600+0000 7fa3877fe700 1 -- 172.21.15.162:0/3402225396 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa36c00b750 con 0x7fa3880994f0 2020-11-28T21:54:17.611 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.600+0000 7fa387fff700 1 --2- 172.21.15.162:0/3402225396 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa36c003da0 0x7fa36c0044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:17.611 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.600+0000 7fa38c97d700 1 --2- 172.21.15.162:0/3402225396 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa3880994f0 0x7fa36c0038b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:17.611 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.600+0000 7fa38d17e700 1 --2- 172.21.15.162:0/3402225396 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa36c0049c0 0x7fa36c004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:17.611 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.600+0000 7fa387fff700 1 -- 172.21.15.162:0/3402225396 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa3880994f0 msgr2=0x7fa36c0038b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:17.612 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.600+0000 7fa387fff700 1 --2- 172.21.15.162:0/3402225396 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa3880994f0 0x7fa36c0038b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:17.612 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.600+0000 7fa387fff700 1 -- 172.21.15.162:0/3402225396 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa36c0049c0 msgr2=0x7fa36c004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:17.612 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.600+0000 7fa387fff700 1 --2- 172.21.15.162:0/3402225396 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa36c0049c0 0x7fa36c004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:17.612 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.600+0000 7fa387fff700 1 -- 172.21.15.162:0/3402225396 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa36c003230 con 0x7fa36c003da0 2020-11-28T21:54:17.612 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.600+0000 7fa38d17e700 1 --2- 172.21.15.162:0/3402225396 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa36c0049c0 0x7fa36c004e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:54:17.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.600+0000 7fa387fff700 1 --2- 172.21.15.162:0/3402225396 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa36c003da0 0x7fa36c0044d0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 rx=0x7fa37800c000 tx=0x7fa378008040).ready entity=mon.1 client_cookie=62a4e179e30c070c server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:17.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.600+0000 7fa3877fe700 1 -- 172.21.15.162:0/3402225396 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa37800e9d0 con 0x7fa36c003da0 2020-11-28T21:54:17.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.600+0000 7fa3877fe700 1 -- 172.21.15.162:0/3402225396 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa37800eb30 con 0x7fa36c003da0 2020-11-28T21:54:17.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.600+0000 7fa3877fe700 1 -- 172.21.15.162:0/3402225396 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa37800ee90 con 0x7fa36c003da0 2020-11-28T21:54:17.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.600+0000 7fa38d97f700 1 -- 172.21.15.162:0/3402225396 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa36c003da0 msgr2=0x7fa36c0044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:17.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.600+0000 7fa38d97f700 1 --2- 172.21.15.162:0/3402225396 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa36c003da0 0x7fa36c0044d0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 rx=0x7fa37800c000 tx=0x7fa378008040).stop 2020-11-28T21:54:17.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.600+0000 7fa38d17e700 1 -- 172.21.15.162:0/3402225396 reap_dead start 2020-11-28T21:54:17.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.600+0000 7fa38d97f700 1 -- 172.21.15.162:0/3402225396 shutdown_connections 2020-11-28T21:54:17.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.600+0000 7fa38d97f700 1 -- 172.21.15.162:0/3402225396 >> 172.21.15.162:0/3402225396 conn(0x7fa38805b970 msgr2=0x7fa38805bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:17.615 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.600+0000 7fa38d97f700 1 -- 172.21.15.162:0/3402225396 shutdown_connections 2020-11-28T21:54:17.615 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.600+0000 7fa38d97f700 1 -- 172.21.15.162:0/3402225396 wait complete. 2020-11-28T21:54:17.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.604+0000 7fa38d97f700 1 Processor -- start 2020-11-28T21:54:17.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.604+0000 7fa38d97f700 1 -- start start 2020-11-28T21:54:17.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.604+0000 7fa38d97f700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa3880aaf40 0x7fa388063340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:17.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.604+0000 7fa38d97f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3880ada60 0x7fa3881592e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:17.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.604+0000 7fa38d97f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa388063830 0x7fa3881597d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:17.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.604+0000 7fa38d97f700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3880ac010 con 0x7fa3880aaf40 2020-11-28T21:54:17.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.604+0000 7fa38d97f700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3880ab320 con 0x7fa3880ada60 2020-11-28T21:54:17.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.604+0000 7fa38d97f700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa388063e60 con 0x7fa388063830 2020-11-28T21:54:17.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.604+0000 7fa387fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3880ada60 0x7fa3881592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:17.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.604+0000 7fa387fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3880ada60 0x7fa3881592e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43112/0 (socket says 172.21.15.162:43112) 2020-11-28T21:54:17.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.604+0000 7fa387fff700 1 -- 172.21.15.162:0/2701295090 learned_addr learned my addr 172.21.15.162:0/2701295090 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:17.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.604+0000 7fa38c97d700 1 --2- 172.21.15.162:0/2701295090 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa3880aaf40 0x7fa388063340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:17.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.604+0000 7fa38d17e700 1 --2- 172.21.15.162:0/2701295090 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa388063830 0x7fa3881597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:17.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.604+0000 7fa387fff700 1 -- 172.21.15.162:0/2701295090 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa388063830 msgr2=0x7fa3881597d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:17.620 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.604+0000 7fa387fff700 1 --2- 172.21.15.162:0/2701295090 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa388063830 0x7fa3881597d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:17.620 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.604+0000 7fa387fff700 1 -- 172.21.15.162:0/2701295090 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa3880aaf40 msgr2=0x7fa388063340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:17.620 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.604+0000 7fa387fff700 1 --2- 172.21.15.162:0/2701295090 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa3880aaf40 0x7fa388063340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:17.620 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.604+0000 7fa387fff700 1 -- 172.21.15.162:0/2701295090 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3880aeb00 con 0x7fa3880ada60 2020-11-28T21:54:17.620 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.604+0000 7fa387fff700 1 --2- 172.21.15.162:0/2701295090 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3880ada60 0x7fa3881592e0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 rx=0x7fa37800c000 tx=0x7fa37801a000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:17.621 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.604+0000 7fa385ffb700 1 -- 172.21.15.162:0/2701295090 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa37801c430 con 0x7fa3880ada60 2020-11-28T21:54:17.621 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.604+0000 7fa385ffb700 1 -- 172.21.15.162:0/2701295090 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa37801c590 con 0x7fa3880ada60 2020-11-28T21:54:17.621 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.604+0000 7fa38d97f700 1 -- 172.21.15.162:0/2701295090 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3880ab9b0 con 0x7fa3880ada60 2020-11-28T21:54:17.621 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.604+0000 7fa385ffb700 1 -- 172.21.15.162:0/2701295090 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa37801c8f0 con 0x7fa3880ada60 2020-11-28T21:54:17.621 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.604+0000 7fa38d97f700 1 -- 172.21.15.162:0/2701295090 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3880ad660 con 0x7fa3880ada60 2020-11-28T21:54:17.622 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.608+0000 7fa38d97f700 1 -- 172.21.15.162:0/2701295090 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3880aec90 con 0x7fa3880ada60 2020-11-28T21:54:17.622 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.608+0000 7fa385ffb700 1 -- 172.21.15.162:0/2701295090 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa37801ca50 con 0x7fa3880ada60 2020-11-28T21:54:17.622 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.608+0000 7fa385ffb700 1 --2- 172.21.15.162:0/2701295090 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa360055330 0x7fa3600577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:17.622 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.608+0000 7fa385ffb700 1 -- 172.21.15.162:0/2701295090 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa37806fa70 con 0x7fa3880ada60 2020-11-28T21:54:17.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.608+0000 7fa38c97d700 1 --2- 172.21.15.162:0/2701295090 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa360055330 0x7fa3600577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:17.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.608+0000 7fa38c97d700 1 --2- 172.21.15.162:0/2701295090 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa360055330 0x7fa3600577f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7fa38000c000 tx=0x7fa380008d90).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:17.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.612+0000 7fa385ffb700 1 -- 172.21.15.162:0/2701295090 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa378045db0 con 0x7fa3880ada60 2020-11-28T21:54:17.756 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.752+0000 7fa38d97f700 1 -- 172.21.15.162:0/2701295090 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fa388057050 con 0x7fa3880ada60 2020-11-28T21:54:17.758 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.756+0000 7fa385ffb700 1 -- 172.21.15.162:0/2701295090 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v99) v1 ==== 81+0+33744 (secure 0 0 0) 0x7fa37806fe60 con 0x7fa3880ada60 2020-11-28T21:54:17.759 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:54:17.760 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2020-11-28T21:54:17.760 INFO:teuthology.orchestra.run.smithi162.stdout: "epoch": 99, 2020-11-28T21:54:17.760 INFO:teuthology.orchestra.run.smithi162.stdout: "fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", 2020-11-28T21:54:17.760 INFO:teuthology.orchestra.run.smithi162.stdout: "created": "2020-11-28T21:46:11.724372+0000", 2020-11-28T21:54:17.761 INFO:teuthology.orchestra.run.smithi162.stdout: "modified": "2020-11-28T21:54:03.391732+0000", 2020-11-28T21:54:17.761 INFO:teuthology.orchestra.run.smithi162.stdout: "last_up_change": "2020-11-28T21:46:19.591650+0000", 2020-11-28T21:54:17.761 INFO:teuthology.orchestra.run.smithi162.stdout: "last_in_change": "2020-11-28T21:46:19.591650+0000", 2020-11-28T21:54:17.762 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-11-28T21:54:17.762 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_num": 5799936, 2020-11-28T21:54:17.762 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_set": [ 2020-11-28T21:54:17.762 INFO:teuthology.orchestra.run.smithi162.stdout: "pglog_hardlimit", 2020-11-28T21:54:17.763 INFO:teuthology.orchestra.run.smithi162.stdout: "purged_snapdirs", 2020-11-28T21:54:17.763 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_deletes", 2020-11-28T21:54:17.763 INFO:teuthology.orchestra.run.smithi162.stdout: "sortbitwise" 2020-11-28T21:54:17.763 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:54:17.764 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_version": 4, 2020-11-28T21:54:17.764 INFO:teuthology.orchestra.run.smithi162.stdout: "full_ratio": 0.94999998807907104, 2020-11-28T21:54:17.764 INFO:teuthology.orchestra.run.smithi162.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-11-28T21:54:17.764 INFO:teuthology.orchestra.run.smithi162.stdout: "nearfull_ratio": 0.85000002384185791, 2020-11-28T21:54:17.764 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_snapshot": "", 2020-11-28T21:54:17.765 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_max": 18, 2020-11-28T21:54:17.765 INFO:teuthology.orchestra.run.smithi162.stdout: "max_osd": 8, 2020-11-28T21:54:17.765 INFO:teuthology.orchestra.run.smithi162.stdout: "require_min_compat_client": "jewel", 2020-11-28T21:54:17.765 INFO:teuthology.orchestra.run.smithi162.stdout: "min_compat_client": "jewel", 2020-11-28T21:54:17.766 INFO:teuthology.orchestra.run.smithi162.stdout: "require_osd_release": "octopus", 2020-11-28T21:54:17.766 INFO:teuthology.orchestra.run.smithi162.stdout: "pools": [ 2020-11-28T21:54:17.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.766 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 1, 2020-11-28T21:54:17.767 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "device_health_metrics", 2020-11-28T21:54:17.767 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:46:15.639551+0000", 2020-11-28T21:54:17.767 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:54:17.767 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:54:17.768 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:54:17.768 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:54:17.768 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:54:17.769 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:54:17.769 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:54:17.769 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:54:17.769 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 1, 2020-11-28T21:54:17.769 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 1, 2020-11-28T21:54:17.770 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 1, 2020-11-28T21:54:17.770 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 1, 2020-11-28T21:54:17.770 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 1, 2020-11-28T21:54:17.771 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:54:17.771 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:54:17.771 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:54:17.772 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:54:17.772 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:54:17.773 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:54:17.773 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:54:17.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.773 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "16", 2020-11-28T21:54:17.774 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:54:17.774 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:54:17.774 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:54:17.774 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:54:17.774 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:54:17.775 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:54:17.775 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:54:17.775 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:54:17.776 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:54:17.776 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:54:17.776 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:54:17.776 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:54:17.776 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:54:17.777 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:54:17.777 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:54:17.777 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:54:17.778 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:54:17.778 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:54:17.779 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:54:17.779 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:54:17.779 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:54:17.779 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:54:17.780 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:54:17.780 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:54:17.780 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:54:17.781 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:54:17.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.781 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:54:17.781 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:54:17.782 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:54:17.782 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:54:17.782 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:54:17.783 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:54:17.783 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:54:17.783 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:54:17.783 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:54:17.784 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:54:17.784 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:54:17.784 INFO:teuthology.orchestra.run.smithi162.stdout: "options": { 2020-11-28T21:54:17.784 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_min": 1 2020-11-28T21:54:17.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.785 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:54:17.785 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr_devicehealth": {} 2020-11-28T21:54:17.786 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:17.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.787 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 2, 2020-11-28T21:54:17.787 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "rbd", 2020-11-28T21:54:17.789 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:46:23.901366+0000", 2020-11-28T21:54:17.789 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:54:17.790 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:54:17.790 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:54:17.791 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:54:17.791 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:54:17.792 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:54:17.793 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:54:17.793 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:54:17.794 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2020-11-28T21:54:17.794 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2020-11-28T21:54:17.795 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2020-11-28T21:54:17.795 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2020-11-28T21:54:17.796 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2020-11-28T21:54:17.796 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:54:17.797 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:54:17.797 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:54:17.797 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:54:17.798 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:54:17.798 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:54:17.799 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:54:17.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.800 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "19", 2020-11-28T21:54:17.800 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:54:17.800 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:54:17.801 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:54:17.801 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:54:17.802 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:54:17.802 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:54:17.803 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:54:17.803 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:54:17.803 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:54:17.804 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:54:17.804 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:54:17.805 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:54:17.805 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:54:17.806 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:54:17.806 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:54:17.806 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:54:17.807 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:54:17.807 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:54:17.808 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:54:17.808 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:54:17.808 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:54:17.809 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:54:17.809 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:54:17.810 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:54:17.810 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:54:17.811 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:54:17.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.811 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:54:17.812 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:54:17.812 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:54:17.813 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:54:17.813 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:54:17.814 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:54:17.814 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:54:17.814 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:54:17.815 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:54:17.815 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:54:17.816 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:54:17.816 INFO:teuthology.orchestra.run.smithi162.stdout: "options": {}, 2020-11-28T21:54:17.816 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:54:17.817 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd": {} 2020-11-28T21:54:17.817 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:17.817 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.817 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.817 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 17, 2020-11-28T21:54:17.818 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "cephfs_metadata", 2020-11-28T21:54:17.818 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:53:59.925390+0000", 2020-11-28T21:54:17.818 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:54:17.818 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:54:17.818 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:54:17.819 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:54:17.819 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:54:17.819 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:54:17.819 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:54:17.819 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:54:17.820 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2020-11-28T21:54:17.820 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2020-11-28T21:54:17.820 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2020-11-28T21:54:17.820 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2020-11-28T21:54:17.821 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2020-11-28T21:54:17.821 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:54:17.821 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:54:17.821 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:54:17.822 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:54:17.822 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:54:17.822 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:54:17.822 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:54:17.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.823 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "98", 2020-11-28T21:54:17.823 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:54:17.823 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:54:17.823 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:54:17.823 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:54:17.824 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:54:17.824 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:54:17.824 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:54:17.824 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:54:17.824 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:54:17.825 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:54:17.825 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:54:17.825 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:54:17.825 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:54:17.826 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:54:17.826 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:54:17.826 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:54:17.826 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:54:17.827 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:54:17.827 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:54:17.827 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:54:17.827 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:54:17.827 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:54:17.828 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:54:17.828 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:54:17.828 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:54:17.828 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:54:17.828 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.829 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:54:17.829 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:54:17.829 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:54:17.829 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:54:17.829 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:54:17.830 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:54:17.830 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:54:17.830 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:54:17.830 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:54:17.831 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:54:17.831 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:54:17.831 INFO:teuthology.orchestra.run.smithi162.stdout: "options": { 2020-11-28T21:54:17.831 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_bias": 4, 2020-11-28T21:54:17.831 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_min": 16, 2020-11-28T21:54:17.832 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_priority": 5 2020-11-28T21:54:17.832 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.832 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:54:17.832 INFO:teuthology.orchestra.run.smithi162.stdout: "cephfs": { 2020-11-28T21:54:17.832 INFO:teuthology.orchestra.run.smithi162.stdout: "metadata": "cephfs" 2020-11-28T21:54:17.833 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:17.833 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:17.833 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.833 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.833 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 18, 2020-11-28T21:54:17.834 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "cephfs_data", 2020-11-28T21:54:17.834 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:54:01.236032+0000", 2020-11-28T21:54:17.834 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:54:17.834 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:54:17.834 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:54:17.835 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:54:17.835 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:54:17.835 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:54:17.835 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:54:17.836 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:54:17.836 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2020-11-28T21:54:17.836 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2020-11-28T21:54:17.836 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2020-11-28T21:54:17.837 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2020-11-28T21:54:17.837 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2020-11-28T21:54:17.837 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:54:17.837 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:54:17.837 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:54:17.838 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:54:17.838 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:54:17.838 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:54:17.838 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:54:17.838 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.839 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "99", 2020-11-28T21:54:17.839 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:54:17.839 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:54:17.839 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:54:17.840 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:54:17.840 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:54:17.840 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:54:17.840 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:54:17.840 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:54:17.841 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:54:17.841 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:54:17.841 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:54:17.841 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:54:17.841 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:54:17.842 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:54:17.842 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:54:17.842 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:54:17.842 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:54:17.843 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:54:17.843 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:54:17.843 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:54:17.843 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:54:17.843 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:54:17.844 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:54:17.844 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:54:17.844 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:54:17.844 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:54:17.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.845 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:54:17.845 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:54:17.845 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:54:17.845 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:54:17.846 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:54:17.846 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:54:17.846 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:54:17.846 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:54:17.846 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:54:17.847 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:54:17.847 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:54:17.847 INFO:teuthology.orchestra.run.smithi162.stdout: "options": {}, 2020-11-28T21:54:17.847 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:54:17.847 INFO:teuthology.orchestra.run.smithi162.stdout: "cephfs": { 2020-11-28T21:54:17.848 INFO:teuthology.orchestra.run.smithi162.stdout: "data": "cephfs" 2020-11-28T21:54:17.848 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:17.848 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:17.848 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:17.848 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:54:17.849 INFO:teuthology.orchestra.run.smithi162.stdout: "osds": [ 2020-11-28T21:54:17.849 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.849 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 0, 2020-11-28T21:54:17.849 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "99c7d9ae-adfa-4fd7-880a-23e6baf3281b", 2020-11-28T21:54:17.849 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:54:17.850 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:54:17.850 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:54:17.850 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:54:17.850 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:54:17.850 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:54:17.851 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:54:17.851 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 97, 2020-11-28T21:54:17.851 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:54:17.851 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:54:17.851 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:54:17.852 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:17.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.852 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:17.852 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6810", 2020-11-28T21:54:17.852 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:54:17.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.853 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:17.853 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6811", 2020-11-28T21:54:17.853 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:54:17.854 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:17.854 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:17.854 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.854 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:54:17.855 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:17.855 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.855 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:17.855 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6812", 2020-11-28T21:54:17.855 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:54:17.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.856 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:17.856 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6813", 2020-11-28T21:54:17.856 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:54:17.857 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:17.857 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:17.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.857 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:54:17.857 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:17.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.858 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:17.858 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6816", 2020-11-28T21:54:17.858 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:54:17.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.859 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:17.859 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6817", 2020-11-28T21:54:17.859 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:54:17.860 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:17.860 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:17.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.860 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:54:17.861 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:17.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.861 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:17.861 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6814", 2020-11-28T21:54:17.861 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:54:17.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.862 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:17.862 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6815", 2020-11-28T21:54:17.862 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:54:17.863 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:17.863 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:17.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.863 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6811/12313", 2020-11-28T21:54:17.863 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6813/12313", 2020-11-28T21:54:17.864 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6817/12313", 2020-11-28T21:54:17.864 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6815/12313", 2020-11-28T21:54:17.864 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:54:17.864 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:54:17.864 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:54:17.865 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:17.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.865 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 1, 2020-11-28T21:54:17.865 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "6ce6220a-00ef-4d13-abba-56f87636fb5f", 2020-11-28T21:54:17.866 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:54:17.866 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:54:17.866 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:54:17.866 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:54:17.866 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:54:17.867 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:54:17.867 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:54:17.867 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 96, 2020-11-28T21:54:17.867 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:54:17.867 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:54:17.868 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:54:17.868 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:17.868 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.868 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:17.868 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6802", 2020-11-28T21:54:17.869 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:54:17.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.869 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:17.869 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6803", 2020-11-28T21:54:17.870 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:54:17.870 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:17.870 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:17.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.870 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:54:17.871 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:17.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.871 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:17.871 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6804", 2020-11-28T21:54:17.871 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:54:17.872 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.872 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.872 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:17.872 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6805", 2020-11-28T21:54:17.872 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:54:17.873 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:17.873 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:17.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.873 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:54:17.873 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:17.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.874 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:17.874 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6808", 2020-11-28T21:54:17.874 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:54:17.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.875 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:17.875 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6809", 2020-11-28T21:54:17.875 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:54:17.875 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:17.876 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:17.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.876 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:54:17.876 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:17.876 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.877 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:17.877 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6806", 2020-11-28T21:54:17.877 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:54:17.877 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.877 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.878 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:17.878 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6807", 2020-11-28T21:54:17.878 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:54:17.878 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:17.878 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:17.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.879 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6803/12311", 2020-11-28T21:54:17.879 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6805/12311", 2020-11-28T21:54:17.879 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6809/12311", 2020-11-28T21:54:17.879 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6807/12311", 2020-11-28T21:54:17.880 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:54:17.880 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:54:17.880 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:54:17.880 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:17.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.881 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 2, 2020-11-28T21:54:17.881 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "db4e510d-27c1-47fc-a902-8b32e7dfd73e", 2020-11-28T21:54:17.881 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:54:17.881 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:54:17.882 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:54:17.882 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:54:17.882 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:54:17.882 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:54:17.882 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:54:17.883 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 97, 2020-11-28T21:54:17.883 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:54:17.883 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:54:17.883 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:54:17.883 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:17.884 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.884 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:17.884 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6818", 2020-11-28T21:54:17.884 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:54:17.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.885 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:17.885 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6819", 2020-11-28T21:54:17.885 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:54:17.885 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:17.886 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:17.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.886 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:54:17.886 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:17.886 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.887 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:17.887 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6820", 2020-11-28T21:54:17.887 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:54:17.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.888 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:17.888 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6821", 2020-11-28T21:54:17.888 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:54:17.888 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:17.889 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:17.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.889 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:54:17.889 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:17.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.890 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:17.890 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6824", 2020-11-28T21:54:17.890 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:54:17.890 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.890 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.891 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:17.891 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6825", 2020-11-28T21:54:17.891 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:54:17.891 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:17.891 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:17.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.892 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:54:17.892 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:17.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.892 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:17.893 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6822", 2020-11-28T21:54:17.893 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:54:17.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.894 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:17.894 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6823", 2020-11-28T21:54:17.894 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:54:17.894 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:17.895 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:17.895 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.895 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6819/12312", 2020-11-28T21:54:17.895 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6821/12312", 2020-11-28T21:54:17.895 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6825/12312", 2020-11-28T21:54:17.896 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6823/12312", 2020-11-28T21:54:17.896 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:54:17.896 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:54:17.896 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:54:17.896 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:17.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.897 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 3, 2020-11-28T21:54:17.897 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "4094cb87-f04a-45e1-8710-fb4b651949f6", 2020-11-28T21:54:17.897 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:54:17.898 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:54:17.898 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:54:17.898 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:54:17.898 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:54:17.898 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:54:17.899 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:54:17.899 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 86, 2020-11-28T21:54:17.899 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:54:17.899 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:54:17.899 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:54:17.900 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:17.900 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.900 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:17.900 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6826", 2020-11-28T21:54:17.900 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:54:17.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.901 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:17.901 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6827", 2020-11-28T21:54:17.901 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:54:17.902 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:17.902 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:17.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.902 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:54:17.902 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:17.903 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.903 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:17.903 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6828", 2020-11-28T21:54:17.903 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:54:17.904 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.904 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.904 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:17.904 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6829", 2020-11-28T21:54:17.904 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:54:17.905 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:17.905 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:17.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.905 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:54:17.905 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:17.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.906 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:17.906 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6832", 2020-11-28T21:54:17.906 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:54:17.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.907 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.907 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:17.907 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6833", 2020-11-28T21:54:17.907 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:54:17.908 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:17.908 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:17.908 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.908 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:54:17.908 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:17.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.909 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:17.909 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6830", 2020-11-28T21:54:17.909 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:54:17.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.910 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:17.910 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6831", 2020-11-28T21:54:17.910 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:54:17.910 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:17.911 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:17.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.911 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6827/12323", 2020-11-28T21:54:17.911 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6829/12323", 2020-11-28T21:54:17.911 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6833/12323", 2020-11-28T21:54:17.912 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6831/12323", 2020-11-28T21:54:17.912 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:54:17.912 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:54:17.912 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:54:17.912 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:17.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.913 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 4, 2020-11-28T21:54:17.913 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "b907bf53-c297-4756-847b-3be96680eef5", 2020-11-28T21:54:17.913 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:54:17.914 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:54:17.914 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:54:17.914 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:54:17.914 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:54:17.914 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:54:17.915 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:54:17.915 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 97, 2020-11-28T21:54:17.915 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:54:17.915 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:54:17.915 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:54:17.916 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:17.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.916 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:17.916 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6800", 2020-11-28T21:54:17.916 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:54:17.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.917 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:17.917 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6801", 2020-11-28T21:54:17.917 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:54:17.918 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:17.918 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:17.918 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.918 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:54:17.918 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:17.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.919 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:17.919 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6802", 2020-11-28T21:54:17.919 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:54:17.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.920 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:17.920 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6803", 2020-11-28T21:54:17.920 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:54:17.920 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:17.921 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:17.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.921 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:54:17.921 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:17.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.922 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:17.922 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6806", 2020-11-28T21:54:17.922 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:54:17.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.923 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:17.923 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6807", 2020-11-28T21:54:17.923 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:54:17.923 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:17.924 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:17.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.924 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:54:17.924 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:17.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.925 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:17.925 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6804", 2020-11-28T21:54:17.926 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:54:17.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.927 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:17.927 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6805", 2020-11-28T21:54:17.927 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:54:17.928 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:17.928 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:17.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.929 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6801/12527", 2020-11-28T21:54:17.929 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6803/12527", 2020-11-28T21:54:17.929 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6807/12527", 2020-11-28T21:54:17.929 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6805/12527", 2020-11-28T21:54:17.930 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:54:17.930 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:54:17.930 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:54:17.930 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:17.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.931 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 5, 2020-11-28T21:54:17.931 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "55ba305d-0505-4212-aebb-631e8554b440", 2020-11-28T21:54:17.931 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:54:17.931 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:54:17.932 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:54:17.932 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:54:17.932 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:54:17.932 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:54:17.932 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:54:17.933 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 97, 2020-11-28T21:54:17.933 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:54:17.933 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:54:17.933 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:54:17.933 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:17.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.934 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:17.934 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6808", 2020-11-28T21:54:17.934 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:54:17.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.935 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:17.935 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6809", 2020-11-28T21:54:17.935 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:54:17.935 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:17.936 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:17.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.936 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:54:17.936 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:17.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.937 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:17.937 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6810", 2020-11-28T21:54:17.937 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:54:17.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.938 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:17.938 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6811", 2020-11-28T21:54:17.938 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:54:17.938 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:17.938 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:17.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.939 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:54:17.939 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:17.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.939 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:17.940 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6814", 2020-11-28T21:54:17.940 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:54:17.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.941 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:17.941 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6815", 2020-11-28T21:54:17.941 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:54:17.941 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:17.942 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:17.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.942 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:54:17.942 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:17.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.943 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:17.943 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6812", 2020-11-28T21:54:17.943 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:54:17.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.944 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:17.944 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6813", 2020-11-28T21:54:17.944 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:54:17.944 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:17.945 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:17.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.945 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6809/12529", 2020-11-28T21:54:17.945 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6811/12529", 2020-11-28T21:54:17.946 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6815/12529", 2020-11-28T21:54:17.946 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6813/12529", 2020-11-28T21:54:17.946 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:54:17.946 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:54:17.946 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:54:17.947 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:17.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.947 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 6, 2020-11-28T21:54:17.947 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "9ada6051-493b-46b5-ba22-1b401440490f", 2020-11-28T21:54:17.948 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:54:17.948 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:54:17.948 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:54:17.948 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:54:17.948 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:54:17.949 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:54:17.949 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:54:17.949 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 97, 2020-11-28T21:54:17.949 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:54:17.950 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:54:17.950 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:54:17.950 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:17.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.950 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:17.951 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6824", 2020-11-28T21:54:17.951 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:54:17.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.951 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:17.952 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6825", 2020-11-28T21:54:17.952 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:54:17.952 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:17.952 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:17.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.953 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:54:17.953 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:17.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.953 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:17.953 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6826", 2020-11-28T21:54:17.954 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:54:17.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.954 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:17.954 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6827", 2020-11-28T21:54:17.955 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:54:17.955 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:17.955 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:17.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.955 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:54:17.956 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:17.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.956 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:17.956 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6830", 2020-11-28T21:54:17.956 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:54:17.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.957 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:17.957 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6831", 2020-11-28T21:54:17.957 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:54:17.958 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:17.958 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:17.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.958 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:54:17.958 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:17.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.959 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:17.959 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6828", 2020-11-28T21:54:17.959 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:54:17.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.960 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:17.960 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6829", 2020-11-28T21:54:17.960 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:54:17.960 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:17.961 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:17.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.961 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6825/12530", 2020-11-28T21:54:17.961 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6827/12530", 2020-11-28T21:54:17.961 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6831/12530", 2020-11-28T21:54:17.962 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6829/12530", 2020-11-28T21:54:17.962 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:54:17.962 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:54:17.962 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:54:17.962 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:17.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.963 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 7, 2020-11-28T21:54:17.963 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "c328b06d-a216-4cba-aad7-9ad081340613", 2020-11-28T21:54:17.963 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:54:17.964 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:54:17.964 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:54:17.964 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:54:17.964 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:54:17.964 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:54:17.965 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:54:17.965 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 97, 2020-11-28T21:54:17.965 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:54:17.965 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:54:17.965 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:54:17.966 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:17.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.966 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:17.966 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6816", 2020-11-28T21:54:17.966 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:54:17.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.967 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:17.967 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6817", 2020-11-28T21:54:17.967 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:54:17.968 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:17.968 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:17.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.968 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:54:17.968 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:17.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.969 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:17.969 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6818", 2020-11-28T21:54:17.969 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:54:17.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.970 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:17.970 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6819", 2020-11-28T21:54:17.970 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:54:17.970 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:17.971 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:17.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.971 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:54:17.971 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:17.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.972 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:17.972 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6822", 2020-11-28T21:54:17.972 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:54:17.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.972 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:17.973 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6823", 2020-11-28T21:54:17.973 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:54:17.973 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:17.973 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:17.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.974 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:54:17.974 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:17.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.974 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:17.974 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6820", 2020-11-28T21:54:17.975 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:54:17.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.975 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:17.975 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6821", 2020-11-28T21:54:17.976 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:54:17.976 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:17.976 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:17.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.976 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6817/12528", 2020-11-28T21:54:17.977 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6819/12528", 2020-11-28T21:54:17.977 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6823/12528", 2020-11-28T21:54:17.977 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6821/12528", 2020-11-28T21:54:17.977 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:54:17.977 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:54:17.978 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:54:17.978 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:17.978 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:17.978 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:54:17.978 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_xinfo": [ 2020-11-28T21:54:17.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.979 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 0, 2020-11-28T21:54:17.979 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:54:17.979 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:54:17.979 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:54:17.980 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:54:17.980 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:54:17.980 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.038052+0000", 2020-11-28T21:54:17.980 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:54:17.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.981 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 1, 2020-11-28T21:54:17.981 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:54:17.981 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:54:17.982 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:54:17.982 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:54:17.982 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:54:17.982 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:17.947136+0000", 2020-11-28T21:54:17.982 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:54:17.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.983 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 2, 2020-11-28T21:54:17.983 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:54:17.983 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:54:17.984 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:54:17.984 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:54:17.984 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:54:17.984 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.001186+0000", 2020-11-28T21:54:17.984 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:54:17.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.985 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 3, 2020-11-28T21:54:17.985 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:54:17.985 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:54:17.986 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:54:17.986 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:54:17.986 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:54:17.986 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.199284+0000", 2020-11-28T21:54:17.986 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:54:17.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.987 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 4, 2020-11-28T21:54:17.987 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:54:17.987 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:54:17.988 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:54:17.988 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:54:17.988 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:54:17.988 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.038823+0000", 2020-11-28T21:54:17.988 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:54:17.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.989 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 5, 2020-11-28T21:54:17.989 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:54:17.989 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:54:17.989 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:54:17.990 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:54:17.990 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:54:17.990 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.009322+0000", 2020-11-28T21:54:17.990 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:54:17.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.991 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 6, 2020-11-28T21:54:17.991 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:54:17.991 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:54:17.991 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:54:17.992 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:54:17.992 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:54:17.992 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.238927+0000", 2020-11-28T21:54:17.992 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:54:17.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:17.993 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 7, 2020-11-28T21:54:17.993 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:54:17.993 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:54:17.994 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:54:17.994 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:54:17.994 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:54:17.994 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.029839+0000", 2020-11-28T21:54:17.994 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:54:17.995 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:17.995 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:54:17.995 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_upmap": [], 2020-11-28T21:54:17.995 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_upmap_items": [], 2020-11-28T21:54:17.995 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_temp": [], 2020-11-28T21:54:17.996 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_temp": [], 2020-11-28T21:54:17.996 INFO:teuthology.orchestra.run.smithi162.stdout: "blacklist": {}, 2020-11-28T21:54:17.996 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profiles": { 2020-11-28T21:54:17.996 INFO:teuthology.orchestra.run.smithi162.stdout: "default": { 2020-11-28T21:54:17.996 INFO:teuthology.orchestra.run.smithi162.stdout: "crush-failure-domain": "osd", 2020-11-28T21:54:17.997 INFO:teuthology.orchestra.run.smithi162.stdout: "k": "2", 2020-11-28T21:54:17.997 INFO:teuthology.orchestra.run.smithi162.stdout: "m": "1", 2020-11-28T21:54:17.997 INFO:teuthology.orchestra.run.smithi162.stdout: "plugin": "jerasure", 2020-11-28T21:54:17.997 INFO:teuthology.orchestra.run.smithi162.stdout: "ruleset-failure-domain": "osd", 2020-11-28T21:54:17.997 INFO:teuthology.orchestra.run.smithi162.stdout: "technique": "reed_sol_van" 2020-11-28T21:54:17.998 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:17.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:17.998 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps_queue": [], 2020-11-28T21:54:17.998 INFO:teuthology.orchestra.run.smithi162.stdout: "new_removed_snaps": [], 2020-11-28T21:54:17.999 INFO:teuthology.orchestra.run.smithi162.stdout: "new_purged_snaps": [], 2020-11-28T21:54:17.999 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_node_flags": {}, 2020-11-28T21:54:17.999 INFO:teuthology.orchestra.run.smithi162.stdout: "device_class_flags": {} 2020-11-28T21:54:17.999 INFO:teuthology.orchestra.run.smithi162.stdout:} 2020-11-28T21:54:18.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.756+0000 7fa38d97f700 1 -- 172.21.15.162:0/2701295090 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa360055330 msgr2=0x7fa3600577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:18.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.756+0000 7fa38d97f700 1 --2- 172.21.15.162:0/2701295090 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa360055330 0x7fa3600577f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7fa38000c000 tx=0x7fa380008d90).stop 2020-11-28T21:54:18.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.756+0000 7fa38d97f700 1 -- 172.21.15.162:0/2701295090 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3880ada60 msgr2=0x7fa3881592e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:18.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.756+0000 7fa38d97f700 1 --2- 172.21.15.162:0/2701295090 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3880ada60 0x7fa3881592e0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 rx=0x7fa37800c000 tx=0x7fa37801a000).stop 2020-11-28T21:54:18.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.756+0000 7fa38d97f700 1 -- 172.21.15.162:0/2701295090 shutdown_connections 2020-11-28T21:54:18.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.756+0000 7fa38d97f700 1 --2- 172.21.15.162:0/2701295090 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa360055330 0x7fa3600577f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:18.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.756+0000 7fa38d97f700 1 --2- 172.21.15.162:0/2701295090 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa388063830 0x7fa3881597d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:18.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.756+0000 7fa38d97f700 1 --2- 172.21.15.162:0/2701295090 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa3880aaf40 0x7fa388063340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:18.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.756+0000 7fa38d97f700 1 --2- 172.21.15.162:0/2701295090 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3880ada60 0x7fa3881592e0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:18.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.756+0000 7fa38d97f700 1 -- 172.21.15.162:0/2701295090 >> 172.21.15.162:0/2701295090 conn(0x7fa38805b970 msgr2=0x7fa3880b1450 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:18.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.756+0000 7fa38d97f700 1 -- 172.21.15.162:0/2701295090 shutdown_connections 2020-11-28T21:54:18.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:17.756+0000 7fa38d97f700 1 -- 172.21.15.162:0/2701295090 wait complete. 2020-11-28T21:54:18.004 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:54:18.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.116+0000 7fee58b97700 1 Processor -- start 2020-11-28T21:54:18.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.116+0000 7fee58b97700 1 -- start start 2020-11-28T21:54:18.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.120+0000 7fee58b97700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fee540b69a0 0x7fee5413f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:18.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.120+0000 7fee58b97700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fee5413cb20 0x7fee54149f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:18.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.120+0000 7fee58b97700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee54067e90 con 0x7fee5413fae0 2020-11-28T21:54:18.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.120+0000 7fee58b97700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee540ae270 con 0x7fee540b69a0 2020-11-28T21:54:18.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.120+0000 7fee58b97700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee54142ed0 con 0x7fee5413cb20 2020-11-28T21:54:18.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.120+0000 7fee52d9d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fee5413cb20 0x7fee54149f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:18.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.120+0000 7fee52d9d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fee5413cb20 0x7fee54149f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43120/0 (socket says 172.21.15.162:43120) 2020-11-28T21:54:18.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.120+0000 7fee52d9d700 1 -- 172.21.15.162:0/3676865723 learned_addr learned my addr 172.21.15.162:0/3676865723 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:18.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.120+0000 7fee5259c700 1 --2- 172.21.15.162:0/3676865723 >> v2:172.21.15.118:3300/0 conn(0x7fee540b69a0 0x7fee5413f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:18.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.120+0000 7fee52d9d700 1 -- 172.21.15.162:0/3676865723 >> v2:172.21.15.118:3300/0 conn(0x7fee540b69a0 msgr2=0x7fee5413f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:18.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.120+0000 7fee52d9d700 1 --2- 172.21.15.162:0/3676865723 >> v2:172.21.15.118:3300/0 conn(0x7fee540b69a0 0x7fee5413f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:18.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.120+0000 7fee52d9d700 1 -- 172.21.15.162:0/3676865723 >> v1:172.21.15.118:6789/0 conn(0x7fee5413fae0 legacy=0x7fee5413c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:18.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.120+0000 7fee52d9d700 1 -- 172.21.15.162:0/3676865723 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee540b6d80 con 0x7fee5413cb20 2020-11-28T21:54:18.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.120+0000 7fee52d9d700 1 --2- 172.21.15.162:0/3676865723 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee5413cb20 0x7fee54149f60 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 rx=0x7fee4c0032b0 tx=0x7fee4c009270).ready entity=mon.1 client_cookie=e2d0e01b73125398 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:18.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.120+0000 7fee51d9b700 1 -- 172.21.15.162:0/3676865723 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee4c009e60 con 0x7fee5413cb20 2020-11-28T21:54:18.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.120+0000 7fee51d9b700 1 -- 172.21.15.162:0/3676865723 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee4c00bc00 con 0x7fee5413cb20 2020-11-28T21:54:18.129 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.120+0000 7fee51d9b700 1 -- 172.21.15.162:0/3676865723 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee4c00bd70 con 0x7fee5413cb20 2020-11-28T21:54:18.129 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.120+0000 7fee58b97700 1 -- 172.21.15.162:0/3676865723 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee5413cb20 msgr2=0x7fee54149f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:18.129 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.120+0000 7fee58b97700 1 --2- 172.21.15.162:0/3676865723 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee5413cb20 0x7fee54149f60 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 rx=0x7fee4c0032b0 tx=0x7fee4c009270).stop 2020-11-28T21:54:18.129 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.120+0000 7fee58b97700 1 -- 172.21.15.162:0/3676865723 shutdown_connections 2020-11-28T21:54:18.129 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.120+0000 7fee58b97700 1 --2- 172.21.15.162:0/3676865723 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee5413cb20 0x7fee54149f60 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:18.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.120+0000 7fee58b97700 1 --2- 172.21.15.162:0/3676865723 >> v2:172.21.15.118:3300/0 conn(0x7fee540b69a0 0x7fee5413f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:18.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.120+0000 7fee58b97700 1 -- 172.21.15.162:0/3676865723 >> 172.21.15.162:0/3676865723 conn(0x7fee54062170 msgr2=0x7fee540b5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:18.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.120+0000 7fee58b97700 1 -- 172.21.15.162:0/3676865723 shutdown_connections 2020-11-28T21:54:18.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.120+0000 7fee58b97700 1 -- 172.21.15.162:0/3676865723 wait complete. 2020-11-28T21:54:18.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.124+0000 7fee58b97700 1 Processor -- start 2020-11-28T21:54:18.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.124+0000 7fee58b97700 1 -- start start 2020-11-28T21:54:18.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.128+0000 7fee58b97700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee540b69a0 0x7fee540b1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:18.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.128+0000 7fee58b97700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee5413cb20 0x7fee5414e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:18.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.128+0000 7fee58b97700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee5413fae0 0x7fee5414ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:18.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.128+0000 7fee58b97700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5414f1c0 con 0x7fee5413cb20 2020-11-28T21:54:18.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.128+0000 7fee58b97700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee54151dd0 con 0x7fee540b69a0 2020-11-28T21:54:18.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.128+0000 7fee58b97700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee54152070 con 0x7fee5413fae0 2020-11-28T21:54:18.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.128+0000 7fee5259c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee540b69a0 0x7fee540b1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:18.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.128+0000 7fee5259c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee540b69a0 0x7fee540b1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43122/0 (socket says 172.21.15.162:43122) 2020-11-28T21:54:18.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.128+0000 7fee5259c700 1 -- 172.21.15.162:0/61144940 learned_addr learned my addr 172.21.15.162:0/61144940 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:18.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.128+0000 7fee52d9d700 1 --2- 172.21.15.162:0/61144940 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee5413fae0 0x7fee5414ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:18.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.128+0000 7fee49d9b700 1 --2- 172.21.15.162:0/61144940 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee5413cb20 0x7fee5414e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:18.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.128+0000 7fee5259c700 1 -- 172.21.15.162:0/61144940 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee5413fae0 msgr2=0x7fee5414ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:18.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.128+0000 7fee5259c700 1 --2- 172.21.15.162:0/61144940 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee5413fae0 0x7fee5414ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:18.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.128+0000 7fee5259c700 1 -- 172.21.15.162:0/61144940 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee5413cb20 msgr2=0x7fee5414e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:18.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.128+0000 7fee5259c700 1 --2- 172.21.15.162:0/61144940 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee5413cb20 0x7fee5414e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:18.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.128+0000 7fee5259c700 1 -- 172.21.15.162:0/61144940 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee54152310 con 0x7fee540b69a0 2020-11-28T21:54:18.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.128+0000 7fee5259c700 1 --2- 172.21.15.162:0/61144940 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee540b69a0 0x7fee540b1370 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 rx=0x7fee4400c000 tx=0x7fee44006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:18.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.128+0000 7fee4bfff700 1 -- 172.21.15.162:0/61144940 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee4400ca10 con 0x7fee540b69a0 2020-11-28T21:54:18.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.128+0000 7fee4bfff700 1 -- 172.21.15.162:0/61144940 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee4400cb70 con 0x7fee540b69a0 2020-11-28T21:54:18.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.128+0000 7fee58b97700 1 -- 172.21.15.162:0/61144940 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee541525b0 con 0x7fee540b69a0 2020-11-28T21:54:18.138 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.128+0000 7fee4bfff700 1 -- 172.21.15.162:0/61144940 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee44011450 con 0x7fee540b69a0 2020-11-28T21:54:18.138 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.128+0000 7fee58b97700 1 -- 172.21.15.162:0/61144940 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee54152a50 con 0x7fee540b69a0 2020-11-28T21:54:18.139 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.128+0000 7fee58b97700 1 -- 172.21.15.162:0/61144940 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee54152740 con 0x7fee540b69a0 2020-11-28T21:54:18.139 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.132+0000 7fee4bfff700 1 -- 172.21.15.162:0/61144940 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fee440115b0 con 0x7fee540b69a0 2020-11-28T21:54:18.139 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.132+0000 7fee4bfff700 1 --2- 172.21.15.162:0/61144940 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fee38055440 0x7fee38057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:18.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.132+0000 7fee4bfff700 1 -- 172.21.15.162:0/61144940 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fee4406fcd0 con 0x7fee540b69a0 2020-11-28T21:54:18.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.132+0000 7fee49d9b700 1 --2- 172.21.15.162:0/61144940 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fee38055440 0x7fee38057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:18.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.132+0000 7fee49d9b700 1 --2- 172.21.15.162:0/61144940 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fee38055440 0x7fee38057900 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7fee3c00d000 tx=0x7fee3c007b40).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:18.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.140+0000 7fee4bfff700 1 -- 172.21.15.162:0/61144940 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fee44045f70 con 0x7fee540b69a0 2020-11-28T21:54:18.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.324+0000 7fee58b97700 1 -- 172.21.15.162:0/61144940 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee54057050 con 0x7fee540b69a0 2020-11-28T21:54:18.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.324+0000 7fee4bfff700 1 -- 172.21.15.162:0/61144940 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+2502 (secure 0 0 0) 0x7fee4401f070 con 0x7fee540b69a0 2020-11-28T21:54:18.330 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:54:18.331 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":46,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6657,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.118:6833/1785592854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1785592854},{"type":"v1","addr":"172.21.15.118:6833","nonce":1785592854}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":44},{"gid":6850,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3533607073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3533607073},{"type":"v1","addr":"172.21.15.162:6835","nonce":3533607073}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:54:02.388344+0000","modified":"2020-11-28T21:54:03.391088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6859":{"gid":6859,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/3532128068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3532128068},{"type":"v1","addr":"172.21.15.162:6837","nonce":3532128068}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2020-11-28T21:54:18.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.328+0000 7fee58b97700 1 -- 172.21.15.162:0/61144940 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fee38055440 msgr2=0x7fee38057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:18.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.328+0000 7fee58b97700 1 --2- 172.21.15.162:0/61144940 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fee38055440 0x7fee38057900 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7fee3c00d000 tx=0x7fee3c007b40).stop 2020-11-28T21:54:18.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.328+0000 7fee58b97700 1 -- 172.21.15.162:0/61144940 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee540b69a0 msgr2=0x7fee540b1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:18.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.328+0000 7fee58b97700 1 --2- 172.21.15.162:0/61144940 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee540b69a0 0x7fee540b1370 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 rx=0x7fee4400c000 tx=0x7fee44006dc0).stop 2020-11-28T21:54:18.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.328+0000 7fee58b97700 1 -- 172.21.15.162:0/61144940 shutdown_connections 2020-11-28T21:54:18.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.328+0000 7fee58b97700 1 --2- 172.21.15.162:0/61144940 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fee38055440 0x7fee38057900 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:18.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.328+0000 7fee58b97700 1 --2- 172.21.15.162:0/61144940 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee5413fae0 0x7fee5414ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:18.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.328+0000 7fee58b97700 1 --2- 172.21.15.162:0/61144940 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee540b69a0 0x7fee540b1370 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:18.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.328+0000 7fee58b97700 1 --2- 172.21.15.162:0/61144940 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee5413cb20 0x7fee5414e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:18.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.328+0000 7fee58b97700 1 -- 172.21.15.162:0/61144940 >> 172.21.15.162:0/61144940 conn(0x7fee54062170 msgr2=0x7fee540b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:18.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.328+0000 7fee58b97700 1 -- 172.21.15.162:0/61144940 shutdown_connections 2020-11-28T21:54:18.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.328+0000 7fee58b97700 1 -- 172.21.15.162:0/61144940 wait complete. 2020-11-28T21:54:18.339 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 46 2020-11-28T21:54:18.349 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2020-11-28T21:54:18.460 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.456+0000 7fa851ec8700 1 Processor -- start 2020-11-28T21:54:18.460 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.456+0000 7fa851ec8700 1 -- start start 2020-11-28T21:54:18.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.456+0000 7fa851ec8700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa84c05b970 0x7fa84c0ac3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:18.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.456+0000 7fa851ec8700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa84c05fe30 con 0x7fa84c0b69a0 2020-11-28T21:54:18.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.456+0000 7fa851ec8700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa84c0b6d80 con 0x7fa84c0ac8a0 2020-11-28T21:54:18.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.456+0000 7fa851ec8700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa84c05bdd0 con 0x7fa84c05b970 2020-11-28T21:54:18.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.456+0000 7fa84affd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa84c05b970 0x7fa84c0ac3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:18.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.456+0000 7fa84bfff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fa84c0ac8a0 0x7fa84c14d450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:49978/0 (socket says 172.21.15.162:49978) 2020-11-28T21:54:18.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.456+0000 7fa84affd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa84c05b970 0x7fa84c0ac3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43132/0 (socket says 172.21.15.162:43132) 2020-11-28T21:54:18.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.456+0000 7fa84bfff700 1 -- 172.21.15.162:0/1970718025 learned_addr learned my addr 172.21.15.162:0/1970718025 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:18.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.456+0000 7fa84affd700 1 -- 172.21.15.162:0/1970718025 >> v1:172.21.15.162:6789/0 conn(0x7fa84c0ac8a0 legacy=0x7fa84c14d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:18.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.456+0000 7fa84affd700 1 -- 172.21.15.162:0/1970718025 >> v1:172.21.15.118:6789/0 conn(0x7fa84c0b69a0 legacy=0x7fa84c0b0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:18.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.456+0000 7fa84a7fc700 1 -- 172.21.15.162:0/1970718025 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2965522514 0 0) 0x7fa84c0b6d80 con 0x7fa84c0ac8a0 2020-11-28T21:54:18.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.456+0000 7fa84affd700 1 -- 172.21.15.162:0/1970718025 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa84c0b1bf0 con 0x7fa84c05b970 2020-11-28T21:54:18.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.456+0000 7fa84affd700 1 --2- 172.21.15.162:0/1970718025 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa84c05b970 0x7fa84c0ac3b0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 rx=0x7fa840002f30 tx=0x7fa840006dc0).ready entity=mon.1 client_cookie=ab1024ec35922ce0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:18.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.460+0000 7fa84a7fc700 1 -- 172.21.15.162:0/1970718025 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8400097f0 con 0x7fa84c05b970 2020-11-28T21:54:18.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.460+0000 7fa84a7fc700 1 -- 172.21.15.162:0/1970718025 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa840009950 con 0x7fa84c05b970 2020-11-28T21:54:18.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.460+0000 7fa84a7fc700 1 -- 172.21.15.162:0/1970718025 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa840009cb0 con 0x7fa84c05b970 2020-11-28T21:54:18.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.460+0000 7fa851ec8700 1 -- 172.21.15.162:0/1970718025 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa84c05b970 msgr2=0x7fa84c0ac3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:18.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.460+0000 7fa851ec8700 1 --2- 172.21.15.162:0/1970718025 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa84c05b970 0x7fa84c0ac3b0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 rx=0x7fa840002f30 tx=0x7fa840006dc0).stop 2020-11-28T21:54:18.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.460+0000 7fa851ec8700 1 -- 172.21.15.162:0/1970718025 shutdown_connections 2020-11-28T21:54:18.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.460+0000 7fa851ec8700 1 --2- 172.21.15.162:0/1970718025 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa84c05b970 0x7fa84c0ac3b0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:18.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.460+0000 7fa851ec8700 1 -- 172.21.15.162:0/1970718025 >> 172.21.15.162:0/1970718025 conn(0x7fa84c0621c0 msgr2=0x7fa84c0b5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:18.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.460+0000 7fa851ec8700 1 -- 172.21.15.162:0/1970718025 shutdown_connections 2020-11-28T21:54:18.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.460+0000 7fa851ec8700 1 -- 172.21.15.162:0/1970718025 wait complete. 2020-11-28T21:54:18.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.464+0000 7fa851ec8700 1 Processor -- start 2020-11-28T21:54:18.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.464+0000 7fa851ec8700 1 -- start start 2020-11-28T21:54:18.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.464+0000 7fa851ec8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa84c05b970 0x7fa84c154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:18.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.464+0000 7fa851ec8700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa84c0ac8a0 0x7fa84c1482a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:18.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.464+0000 7fa851ec8700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa84c0b69a0 0x7fa84c0adba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:18.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.464+0000 7fa851ec8700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c152690 con 0x7fa84c0b69a0 2020-11-28T21:54:18.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.464+0000 7fa851ec8700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c148790 con 0x7fa84c05b970 2020-11-28T21:54:18.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.464+0000 7fa851ec8700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c1523f0 con 0x7fa84c0ac8a0 2020-11-28T21:54:18.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.464+0000 7fa84b7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa84c05b970 0x7fa84c154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:18.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.464+0000 7fa84b7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa84c05b970 0x7fa84c154f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43136/0 (socket says 172.21.15.162:43136) 2020-11-28T21:54:18.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.464+0000 7fa84b7fe700 1 -- 172.21.15.162:0/4131814380 learned_addr learned my addr 172.21.15.162:0/4131814380 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:18.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.464+0000 7fa84affd700 1 --2- 172.21.15.162:0/4131814380 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa84c0ac8a0 0x7fa84c1482a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:18.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.464+0000 7fa84bfff700 1 --2- 172.21.15.162:0/4131814380 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa84c0b69a0 0x7fa84c0adba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:18.485 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.464+0000 7fa84b7fe700 1 -- 172.21.15.162:0/4131814380 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa84c0ac8a0 msgr2=0x7fa84c1482a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:18.485 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.464+0000 7fa84b7fe700 1 --2- 172.21.15.162:0/4131814380 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa84c0ac8a0 0x7fa84c1482a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:18.486 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.464+0000 7fa84b7fe700 1 -- 172.21.15.162:0/4131814380 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa84c0b69a0 msgr2=0x7fa84c0adba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:18.486 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.464+0000 7fa84b7fe700 1 --2- 172.21.15.162:0/4131814380 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa84c0b69a0 0x7fa84c0adba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:18.486 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.464+0000 7fa84b7fe700 1 -- 172.21.15.162:0/4131814380 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa84c155610 con 0x7fa84c05b970 2020-11-28T21:54:18.487 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.464+0000 7fa84b7fe700 1 --2- 172.21.15.162:0/4131814380 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa84c05b970 0x7fa84c154f60 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 rx=0x7fa84c06b330 tx=0x7fa83c007c20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:18.487 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.464+0000 7fa848ff9700 1 -- 172.21.15.162:0/4131814380 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa83c013410 con 0x7fa84c05b970 2020-11-28T21:54:18.487 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.464+0000 7fa848ff9700 1 -- 172.21.15.162:0/4131814380 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa83c013570 con 0x7fa84c05b970 2020-11-28T21:54:18.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.464+0000 7fa851ec8700 1 -- 172.21.15.162:0/4131814380 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa84c1558b0 con 0x7fa84c05b970 2020-11-28T21:54:18.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.464+0000 7fa848ff9700 1 -- 172.21.15.162:0/4131814380 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa83c0138d0 con 0x7fa84c05b970 2020-11-28T21:54:18.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.464+0000 7fa851ec8700 1 -- 172.21.15.162:0/4131814380 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa84c0ae090 con 0x7fa84c05b970 2020-11-28T21:54:18.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.468+0000 7fa851ec8700 1 -- 172.21.15.162:0/4131814380 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa84c155a40 con 0x7fa84c05b970 2020-11-28T21:54:18.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.468+0000 7fa848ff9700 1 -- 172.21.15.162:0/4131814380 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa83c010030 con 0x7fa84c05b970 2020-11-28T21:54:18.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.468+0000 7fa848ff9700 1 --2- 172.21.15.162:0/4131814380 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa82c055330 0x7fa82c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:18.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.468+0000 7fa848ff9700 1 -- 172.21.15.162:0/4131814380 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa83c06dde0 con 0x7fa84c05b970 2020-11-28T21:54:18.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.468+0000 7fa84affd700 1 --2- 172.21.15.162:0/4131814380 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa82c055330 0x7fa82c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:18.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.468+0000 7fa84affd700 1 --2- 172.21.15.162:0/4131814380 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa82c055330 0x7fa82c0577f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7fa84001b000 tx=0x7fa840015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:18.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.476+0000 7fa848ff9700 1 -- 172.21.15.162:0/4131814380 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa83c043fd0 con 0x7fa84c05b970 2020-11-28T21:54:18.663 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:18.660+0000 7fa851ec8700 1 -- 172.21.15.162:0/4131814380 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa84c057050 con 0x7fa84c05b970 2020-11-28T21:54:19.330 INFO:tasks.ceph.mon.b.smithi118.stderr:2020-11-28T21:54:19.325+0000 7ff4e1520700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-11-28T21:54:19.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.336+0000 7fa848ff9700 1 -- 172.21.15.162:0/4131814380 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v47) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa83c0475f0 con 0x7fa84c05b970 2020-11-28T21:54:19.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.340+0000 7fa851ec8700 1 -- 172.21.15.162:0/4131814380 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa82c055330 msgr2=0x7fa82c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:19.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.340+0000 7fa851ec8700 1 --2- 172.21.15.162:0/4131814380 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa82c055330 0x7fa82c0577f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7fa84001b000 tx=0x7fa840015000).stop 2020-11-28T21:54:19.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.340+0000 7fa851ec8700 1 -- 172.21.15.162:0/4131814380 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa84c05b970 msgr2=0x7fa84c154f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:19.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.340+0000 7fa851ec8700 1 --2- 172.21.15.162:0/4131814380 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa84c05b970 0x7fa84c154f60 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 rx=0x7fa84c06b330 tx=0x7fa83c007c20).stop 2020-11-28T21:54:19.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.340+0000 7fa851ec8700 1 -- 172.21.15.162:0/4131814380 shutdown_connections 2020-11-28T21:54:19.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.340+0000 7fa851ec8700 1 --2- 172.21.15.162:0/4131814380 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa82c055330 0x7fa82c0577f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:19.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.340+0000 7fa851ec8700 1 --2- 172.21.15.162:0/4131814380 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa84c0b69a0 0x7fa84c0adba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:19.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.340+0000 7fa851ec8700 1 --2- 172.21.15.162:0/4131814380 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa84c05b970 0x7fa84c154f60 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:19.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.340+0000 7fa851ec8700 1 --2- 172.21.15.162:0/4131814380 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa84c0ac8a0 0x7fa84c1482a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:19.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.340+0000 7fa851ec8700 1 -- 172.21.15.162:0/4131814380 >> 172.21.15.162:0/4131814380 conn(0x7fa84c0621c0 msgr2=0x7fa84c06a280 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:19.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.340+0000 7fa851ec8700 1 -- 172.21.15.162:0/4131814380 shutdown_connections 2020-11-28T21:54:19.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.340+0000 7fa851ec8700 1 -- 172.21.15.162:0/4131814380 wait complete. 2020-11-28T21:54:19.347 INFO:teuthology.orchestra.run.smithi162.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-11-28T21:54:19.366 INFO:tasks.ceph.mds.a.smithi162.stdout:starting mds.a at 2020-11-28T21:54:19.366 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:54:19.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.468+0000 7fa0ab59e700 1 Processor -- start 2020-11-28T21:54:19.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.472+0000 7fa0ab59e700 1 -- start start 2020-11-28T21:54:19.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.472+0000 7fa0ab59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa0ac0a9560 0x7fa0ac0a9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:19.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.472+0000 7fa0ab59e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fa0ac0b16a0 0x7fa0ac0b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:19.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.472+0000 7fa0ab59e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0ac0a8890 con 0x7fa0ac0b2010 2020-11-28T21:54:19.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.472+0000 7fa0ab59e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0ac0a86a0 con 0x7fa0ac0b16a0 2020-11-28T21:54:19.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.472+0000 7fa0ab59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac0a9e30 con 0x7fa0ac0a9560 2020-11-28T21:54:19.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.472+0000 7fa0aad9d700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fa0ac0b2010 0x7fa0ac05f820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:50002/0 (socket says 172.21.15.162:50002) 2020-11-28T21:54:19.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.472+0000 7fa0aad9d700 1 -- 172.21.15.162:0/3333744699 learned_addr learned my addr 172.21.15.162:0/3333744699 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:19.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.472+0000 7fa0aa59c700 1 --2- 172.21.15.162:0/3333744699 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa0ac0a9560 0x7fa0ac0a9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:19.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.472+0000 7fa0a9d9b700 1 --2- 172.21.15.162:0/3333744699 >> v2:172.21.15.118:3300/0 conn(0x7fa0ac0b16a0 0x7fa0ac0b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:19.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.472+0000 7fa0a959a700 1 -- 172.21.15.162:0/3333744699 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1958357143 0 0) 0x7fa0ac0a8890 con 0x7fa0ac0b2010 2020-11-28T21:54:19.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.472+0000 7fa0a959a700 1 -- 172.21.15.162:0/3333744699 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa090003230 con 0x7fa0ac0b2010 2020-11-28T21:54:19.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.472+0000 7fa0aa59c700 1 -- 172.21.15.162:0/3333744699 >> v2:172.21.15.118:3300/0 conn(0x7fa0ac0b16a0 msgr2=0x7fa0ac0b1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:19.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.472+0000 7fa0aa59c700 1 --2- 172.21.15.162:0/3333744699 >> v2:172.21.15.118:3300/0 conn(0x7fa0ac0b16a0 0x7fa0ac0b1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:19.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.472+0000 7fa0aa59c700 1 -- 172.21.15.162:0/3333744699 >> v1:172.21.15.162:6789/0 conn(0x7fa0ac0b2010 legacy=0x7fa0ac05f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:19.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.472+0000 7fa0aa59c700 1 -- 172.21.15.162:0/3333744699 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ac0b02f0 con 0x7fa0ac0a9560 2020-11-28T21:54:19.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.472+0000 7fa0aa59c700 1 --2- 172.21.15.162:0/3333744699 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa0ac0a9560 0x7fa0ac0a9940 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fa09c002f30 tx=0x7fa09c006dc0).ready entity=mon.2 client_cookie=b122855a5e3a9e8c server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:19.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.472+0000 7fa0a959a700 1 -- 172.21.15.162:0/3333744699 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa09c0096a0 con 0x7fa0ac0a9560 2020-11-28T21:54:19.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.472+0000 7fa0a959a700 1 -- 172.21.15.162:0/3333744699 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa09c009800 con 0x7fa0ac0a9560 2020-11-28T21:54:19.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.472+0000 7fa0a959a700 1 -- 172.21.15.162:0/3333744699 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa09c009b60 con 0x7fa0ac0a9560 2020-11-28T21:54:19.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.472+0000 7fa0ab59e700 1 -- 172.21.15.162:0/3333744699 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa0ac0a9560 msgr2=0x7fa0ac0a9940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:19.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.472+0000 7fa0ab59e700 1 --2- 172.21.15.162:0/3333744699 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa0ac0a9560 0x7fa0ac0a9940 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fa09c002f30 tx=0x7fa09c006dc0).stop 2020-11-28T21:54:19.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.472+0000 7fa0ab59e700 1 -- 172.21.15.162:0/3333744699 shutdown_connections 2020-11-28T21:54:19.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.472+0000 7fa0ab59e700 1 --2- 172.21.15.162:0/3333744699 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa0ac0a9560 0x7fa0ac0a9940 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:19.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.472+0000 7fa0ab59e700 1 --2- 172.21.15.162:0/3333744699 >> v2:172.21.15.118:3300/0 conn(0x7fa0ac0b16a0 0x7fa0ac0b1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:19.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.472+0000 7fa0ab59e700 1 -- 172.21.15.162:0/3333744699 >> 172.21.15.162:0/3333744699 conn(0x7fa0ac05b970 msgr2=0x7fa0ac05bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:19.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.472+0000 7fa0ab59e700 1 -- 172.21.15.162:0/3333744699 shutdown_connections 2020-11-28T21:54:19.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.472+0000 7fa0ab59e700 1 -- 172.21.15.162:0/3333744699 wait complete. 2020-11-28T21:54:19.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.476+0000 7fa0ab59e700 1 Processor -- start 2020-11-28T21:54:19.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.476+0000 7fa0ab59e700 1 -- start start 2020-11-28T21:54:19.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.476+0000 7fa0ab59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa0ac0a9560 0x7fa0ac146c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:19.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.476+0000 7fa0ab59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa0ac0b16a0 0x7fa0ac0ad1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:19.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.476+0000 7fa0ab59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0ac0b2010 0x7fa0ac0ad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:19.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.476+0000 7fa0ab59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac0af4e0 con 0x7fa0ac0b16a0 2020-11-28T21:54:19.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.476+0000 7fa0ab59e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac0adbd0 con 0x7fa0ac0b2010 2020-11-28T21:54:19.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.476+0000 7fa0ab59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac1475c0 con 0x7fa0ac0a9560 2020-11-28T21:54:19.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.476+0000 7fa0aad9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0ac0b2010 0x7fa0ac0ad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:19.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.476+0000 7fa0aad9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0ac0b2010 0x7fa0ac0ad6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43162/0 (socket says 172.21.15.162:43162) 2020-11-28T21:54:19.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.476+0000 7fa0aad9d700 1 -- 172.21.15.162:0/1675893493 learned_addr learned my addr 172.21.15.162:0/1675893493 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:19.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.476+0000 7fa0aa59c700 1 --2- 172.21.15.162:0/1675893493 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa0ac0a9560 0x7fa0ac146c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:19.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.476+0000 7fa0a9d9b700 1 --2- 172.21.15.162:0/1675893493 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa0ac0b16a0 0x7fa0ac0ad1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:19.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.476+0000 7fa0aad9d700 1 -- 172.21.15.162:0/1675893493 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa0ac0a9560 msgr2=0x7fa0ac146c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:19.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.476+0000 7fa0aad9d700 1 --2- 172.21.15.162:0/1675893493 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa0ac0a9560 0x7fa0ac146c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:19.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.476+0000 7fa0aad9d700 1 -- 172.21.15.162:0/1675893493 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa0ac0b16a0 msgr2=0x7fa0ac0ad1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:19.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.476+0000 7fa0aad9d700 1 --2- 172.21.15.162:0/1675893493 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa0ac0b16a0 0x7fa0ac0ad1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:19.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.476+0000 7fa0aad9d700 1 -- 172.21.15.162:0/1675893493 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ac0aa860 con 0x7fa0ac0b2010 2020-11-28T21:54:19.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.476+0000 7fa0aa59c700 1 --2- 172.21.15.162:0/1675893493 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa0ac0a9560 0x7fa0ac146c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:54:19.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.476+0000 7fa0aad9d700 1 --2- 172.21.15.162:0/1675893493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0ac0b2010 0x7fa0ac0ad6e0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 rx=0x7fa0a4001450 tx=0x7fa0a4007dd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:19.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.476+0000 7fa09b7fe700 1 -- 172.21.15.162:0/1675893493 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0a4005960 con 0x7fa0ac0b2010 2020-11-28T21:54:19.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.476+0000 7fa09b7fe700 1 -- 172.21.15.162:0/1675893493 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a4005ac0 con 0x7fa0ac0b2010 2020-11-28T21:54:19.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.476+0000 7fa0ab59e700 1 -- 172.21.15.162:0/1675893493 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0ac0af710 con 0x7fa0ac0b2010 2020-11-28T21:54:19.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.480+0000 7fa09b7fe700 1 -- 172.21.15.162:0/1675893493 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0a4005e20 con 0x7fa0ac0b2010 2020-11-28T21:54:19.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.480+0000 7fa0ab59e700 1 -- 172.21.15.162:0/1675893493 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0ac08fdb0 con 0x7fa0ac0b2010 2020-11-28T21:54:19.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.480+0000 7fa0ab59e700 1 -- 172.21.15.162:0/1675893493 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0ac057050 con 0x7fa0ac0b2010 2020-11-28T21:54:19.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.480+0000 7fa09b7fe700 1 -- 172.21.15.162:0/1675893493 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa0a4010030 con 0x7fa0ac0b2010 2020-11-28T21:54:19.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.480+0000 7fa09b7fe700 1 --2- 172.21.15.162:0/1675893493 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa088055500 0x7fa0880579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:19.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.480+0000 7fa09b7fe700 1 -- 172.21.15.162:0/1675893493 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fa0a406dea0 con 0x7fa0ac0b2010 2020-11-28T21:54:19.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.480+0000 7fa0aa59c700 1 --2- 172.21.15.162:0/1675893493 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa088055500 0x7fa0880579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:19.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.480+0000 7fa0aa59c700 1 --2- 172.21.15.162:0/1675893493 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa088055500 0x7fa0880579c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7fa09c01b000 tx=0x7fa09c015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:19.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.488+0000 7fa09b7fe700 1 -- 172.21.15.162:0/1675893493 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa0a4044150 con 0x7fa0ac0b2010 2020-11-28T21:54:19.680 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.676+0000 7fa0ab59e700 1 -- 172.21.15.162:0/1675893493 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0ac068350 con 0x7fa0ac0b2010 2020-11-28T21:54:19.681 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.676+0000 7fa09b7fe700 1 -- 172.21.15.162:0/1675893493 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+2139 (secure 0 0 0) 0x7fa0a4047770 con 0x7fa0ac0b2010 2020-11-28T21:54:19.682 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:54:19.682 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":47,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6657,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.118:6833/1785592854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1785592854},{"type":"v1","addr":"172.21.15.118:6833","nonce":1785592854}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":44},{"gid":6850,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3533607073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3533607073},{"type":"v1","addr":"172.21.15.162:6835","nonce":3533607073}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43}],"filesystems":[{"mdsmap":{"epoch":47,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:54:02.388344+0000","modified":"2020-11-28T21:54:19.330024+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2020-11-28T21:54:19.684 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.680+0000 7fa0ab59e700 1 -- 172.21.15.162:0/1675893493 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa088055500 msgr2=0x7fa0880579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:19.684 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.680+0000 7fa0ab59e700 1 --2- 172.21.15.162:0/1675893493 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa088055500 0x7fa0880579c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7fa09c01b000 tx=0x7fa09c015000).stop 2020-11-28T21:54:19.685 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.680+0000 7fa0ab59e700 1 -- 172.21.15.162:0/1675893493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0ac0b2010 msgr2=0x7fa0ac0ad6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:19.685 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.680+0000 7fa0ab59e700 1 --2- 172.21.15.162:0/1675893493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0ac0b2010 0x7fa0ac0ad6e0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 rx=0x7fa0a4001450 tx=0x7fa0a4007dd0).stop 2020-11-28T21:54:19.685 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.680+0000 7fa0ab59e700 1 -- 172.21.15.162:0/1675893493 shutdown_connections 2020-11-28T21:54:19.685 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.680+0000 7fa0ab59e700 1 --2- 172.21.15.162:0/1675893493 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa088055500 0x7fa0880579c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:19.686 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.680+0000 7fa0ab59e700 1 --2- 172.21.15.162:0/1675893493 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa0ac0a9560 0x7fa0ac146c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:19.686 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.680+0000 7fa0ab59e700 1 --2- 172.21.15.162:0/1675893493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0ac0b2010 0x7fa0ac0ad6e0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:19.686 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.680+0000 7fa0ab59e700 1 --2- 172.21.15.162:0/1675893493 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa0ac0b16a0 0x7fa0ac0ad1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:19.686 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.680+0000 7fa0ab59e700 1 -- 172.21.15.162:0/1675893493 >> 172.21.15.162:0/1675893493 conn(0x7fa0ac05b970 msgr2=0x7fa0ac099ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:19.687 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.680+0000 7fa0ab59e700 1 -- 172.21.15.162:0/1675893493 shutdown_connections 2020-11-28T21:54:19.687 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.680+0000 7fa0ab59e700 1 -- 172.21.15.162:0/1675893493 wait complete. 2020-11-28T21:54:19.687 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 47 2020-11-28T21:54:19.700 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2020-11-28T21:54:19.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.812+0000 7f5fab238700 1 Processor -- start 2020-11-28T21:54:19.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.812+0000 7f5fab238700 1 -- start start 2020-11-28T21:54:19.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.812+0000 7f5fab238700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5fa4060900 0x7f5fa40b16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:19.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.812+0000 7f5fab238700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5fa405c840 0x7f5fa40b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:19.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.812+0000 7f5fab238700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5fa405c1e0 0x7f5fa4142a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:19.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.812+0000 7f5fab238700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fa4060ce0 con 0x7f5fa4060900 2020-11-28T21:54:19.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.812+0000 7f5fab238700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fa4146360 con 0x7f5fa405c840 2020-11-28T21:54:19.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.812+0000 7f5fab238700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa4146600 con 0x7f5fa405c1e0 2020-11-28T21:54:19.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.812+0000 7f5fa9a35700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5fa405c840 0x7f5fa40b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:19.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.812+0000 7f5fa9a35700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5fa405c840 0x7f5fa40b1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43170/0 (socket says 172.21.15.162:43170) 2020-11-28T21:54:19.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.812+0000 7f5fa9a35700 1 -- 172.21.15.162:0/2381740494 learned_addr learned my addr 172.21.15.162:0/2381740494 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:19.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.812+0000 7f5faa236700 1 --2- 172.21.15.162:0/2381740494 >> v2:172.21.15.118:3300/0 conn(0x7f5fa4060900 0x7f5fa40b16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:19.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.812+0000 7f5faaa37700 1 --2- 172.21.15.162:0/2381740494 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5fa405c1e0 0x7f5fa4142a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:19.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.812+0000 7f5fa9a35700 1 -- 172.21.15.162:0/2381740494 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5fa405c1e0 msgr2=0x7f5fa4142a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:19.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.812+0000 7f5fa9a35700 1 --2- 172.21.15.162:0/2381740494 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5fa405c1e0 0x7f5fa4142a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:19.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.812+0000 7f5fa9a35700 1 -- 172.21.15.162:0/2381740494 >> v2:172.21.15.118:3300/0 conn(0x7f5fa4060900 msgr2=0x7f5fa40b16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:19.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.812+0000 7f5fa9a35700 1 --2- 172.21.15.162:0/2381740494 >> v2:172.21.15.118:3300/0 conn(0x7f5fa4060900 0x7f5fa40b16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:19.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.812+0000 7f5fa9a35700 1 -- 172.21.15.162:0/2381740494 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa41468a0 con 0x7f5fa405c840 2020-11-28T21:54:19.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.812+0000 7f5faaa37700 1 --2- 172.21.15.162:0/2381740494 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5fa405c1e0 0x7f5fa4142a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:54:19.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.812+0000 7f5fa9a35700 1 --2- 172.21.15.162:0/2381740494 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fa405c840 0x7f5fa40b1be0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 rx=0x7f5fa0002f30 tx=0x7f5fa0006e80).ready entity=mon.1 client_cookie=b82c4b9be7817933 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:19.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.812+0000 7f5fa9234700 1 -- 172.21.15.162:0/2381740494 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fa00097f0 con 0x7f5fa405c840 2020-11-28T21:54:19.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.812+0000 7f5fa9234700 1 -- 172.21.15.162:0/2381740494 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa0009950 con 0x7f5fa405c840 2020-11-28T21:54:19.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.812+0000 7f5fa9234700 1 -- 172.21.15.162:0/2381740494 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fa0009cb0 con 0x7f5fa405c840 2020-11-28T21:54:19.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.812+0000 7f5fab238700 1 -- 172.21.15.162:0/2381740494 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fa405c840 msgr2=0x7f5fa40b1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:19.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.812+0000 7f5fab238700 1 --2- 172.21.15.162:0/2381740494 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fa405c840 0x7f5fa40b1be0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 rx=0x7f5fa0002f30 tx=0x7f5fa0006e80).stop 2020-11-28T21:54:19.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.812+0000 7f5fab238700 1 -- 172.21.15.162:0/2381740494 shutdown_connections 2020-11-28T21:54:19.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.812+0000 7f5fab238700 1 --2- 172.21.15.162:0/2381740494 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5fa405c1e0 0x7f5fa4142a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:19.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.812+0000 7f5fab238700 1 --2- 172.21.15.162:0/2381740494 >> v2:172.21.15.118:3300/0 conn(0x7f5fa4060900 0x7f5fa40b16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:19.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.812+0000 7f5fab238700 1 --2- 172.21.15.162:0/2381740494 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fa405c840 0x7f5fa40b1be0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:19.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.812+0000 7f5fab238700 1 -- 172.21.15.162:0/2381740494 >> 172.21.15.162:0/2381740494 conn(0x7f5fa406aa70 msgr2=0x7f5fa4066af0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:19.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.812+0000 7f5fab238700 1 -- 172.21.15.162:0/2381740494 shutdown_connections 2020-11-28T21:54:19.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.812+0000 7f5fab238700 1 -- 172.21.15.162:0/2381740494 wait complete. 2020-11-28T21:54:19.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.816+0000 7f5fab238700 1 Processor -- start 2020-11-28T21:54:19.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.816+0000 7f5fab238700 1 -- start start 2020-11-28T21:54:19.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.816+0000 7f5fab238700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fa405c1e0 0x7f5fa4154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:19.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.816+0000 7f5fab238700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5fa405c840 0x7f5fa4149490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:19.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.816+0000 7f5fab238700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5fa4060900 0x7f5fa41544f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:19.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.816+0000 7f5fab238700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa40af920 con 0x7f5fa4060900 2020-11-28T21:54:19.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.816+0000 7f5fab238700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa4153df0 con 0x7f5fa405c1e0 2020-11-28T21:54:19.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.816+0000 7f5fab238700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa4145c40 con 0x7f5fa405c840 2020-11-28T21:54:19.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.816+0000 7f5faa236700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fa405c1e0 0x7f5fa4154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:19.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.816+0000 7f5faa236700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fa405c1e0 0x7f5fa4154e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43174/0 (socket says 172.21.15.162:43174) 2020-11-28T21:54:19.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.816+0000 7f5faa236700 1 -- 172.21.15.162:0/3860691100 learned_addr learned my addr 172.21.15.162:0/3860691100 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:19.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.816+0000 7f5fa9a35700 1 --2- 172.21.15.162:0/3860691100 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5fa405c840 0x7f5fa4149490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:19.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.816+0000 7f5faaa37700 1 --2- 172.21.15.162:0/3860691100 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5fa4060900 0x7f5fa41544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:19.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.816+0000 7f5faa236700 1 -- 172.21.15.162:0/3860691100 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5fa405c840 msgr2=0x7f5fa4149490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:19.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.816+0000 7f5faa236700 1 --2- 172.21.15.162:0/3860691100 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5fa405c840 0x7f5fa4149490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:19.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.816+0000 7f5faa236700 1 -- 172.21.15.162:0/3860691100 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5fa4060900 msgr2=0x7f5fa41544f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:19.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.816+0000 7f5faa236700 1 --2- 172.21.15.162:0/3860691100 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5fa4060900 0x7f5fa41544f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:19.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.816+0000 7f5faa236700 1 -- 172.21.15.162:0/3860691100 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa41459a0 con 0x7f5fa405c1e0 2020-11-28T21:54:19.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.816+0000 7f5fa9a35700 1 --2- 172.21.15.162:0/3860691100 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5fa405c840 0x7f5fa4149490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:54:19.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.816+0000 7f5faa236700 1 --2- 172.21.15.162:0/3860691100 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fa405c1e0 0x7f5fa4154e40 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 rx=0x7f5fa40ad4f0 tx=0x7f5f9c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:19.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.816+0000 7f5f937fe700 1 -- 172.21.15.162:0/3860691100 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f9c00bad0 con 0x7f5fa405c1e0 2020-11-28T21:54:19.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.816+0000 7f5f937fe700 1 -- 172.21.15.162:0/3860691100 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f9c00bc30 con 0x7f5fa405c1e0 2020-11-28T21:54:19.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.816+0000 7f5fab238700 1 -- 172.21.15.162:0/3860691100 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fa4154050 con 0x7f5fa405c1e0 2020-11-28T21:54:19.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.816+0000 7f5f937fe700 1 -- 172.21.15.162:0/3860691100 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f9c012420 con 0x7f5fa405c1e0 2020-11-28T21:54:19.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.816+0000 7f5fab238700 1 -- 172.21.15.162:0/3860691100 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fa4145f10 con 0x7f5fa405c1e0 2020-11-28T21:54:19.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.820+0000 7f5fab238700 1 -- 172.21.15.162:0/3860691100 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fa4057050 con 0x7f5fa405c1e0 2020-11-28T21:54:19.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.820+0000 7f5f937fe700 1 -- 172.21.15.162:0/3860691100 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5f9c012580 con 0x7f5fa405c1e0 2020-11-28T21:54:19.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.820+0000 7f5f937fe700 1 --2- 172.21.15.162:0/3860691100 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5f84055330 0x7f5f840577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:19.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.820+0000 7f5f937fe700 1 -- 172.21.15.162:0/3860691100 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f5f9c06e5a0 con 0x7f5fa405c1e0 2020-11-28T21:54:19.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.820+0000 7f5fa9a35700 1 --2- 172.21.15.162:0/3860691100 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5f84055330 0x7f5f840577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:19.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.820+0000 7f5fa9a35700 1 --2- 172.21.15.162:0/3860691100 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5f84055330 0x7f5f840577f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f5fa001a000 tx=0x7f5fa0015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:19.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:19.824+0000 7f5f937fe700 1 -- 172.21.15.162:0/3860691100 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5f9c044750 con 0x7f5fa405c1e0 2020-11-28T21:54:20.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.008+0000 7f5fab238700 1 -- 172.21.15.162:0/3860691100 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5fa405d250 con 0x7f5fa405c1e0 2020-11-28T21:54:20.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.336+0000 7f5f937fe700 1 -- 172.21.15.162:0/3860691100 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v48) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5f9c047d70 con 0x7f5fa405c1e0 2020-11-28T21:54:20.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.340+0000 7f5fab238700 1 -- 172.21.15.162:0/3860691100 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5f84055330 msgr2=0x7f5f840577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:20.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.340+0000 7f5fab238700 1 --2- 172.21.15.162:0/3860691100 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5f84055330 0x7f5f840577f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f5fa001a000 tx=0x7f5fa0015000).stop 2020-11-28T21:54:20.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.340+0000 7f5fab238700 1 -- 172.21.15.162:0/3860691100 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fa405c1e0 msgr2=0x7f5fa4154e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:20.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.340+0000 7f5fab238700 1 --2- 172.21.15.162:0/3860691100 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fa405c1e0 0x7f5fa4154e40 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 rx=0x7f5fa40ad4f0 tx=0x7f5f9c008040).stop 2020-11-28T21:54:20.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.340+0000 7f5fab238700 1 -- 172.21.15.162:0/3860691100 shutdown_connections 2020-11-28T21:54:20.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.340+0000 7f5fab238700 1 --2- 172.21.15.162:0/3860691100 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5f84055330 0x7f5f840577f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:20.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.340+0000 7f5fab238700 1 --2- 172.21.15.162:0/3860691100 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5fa4060900 0x7f5fa41544f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:20.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.340+0000 7f5fab238700 1 --2- 172.21.15.162:0/3860691100 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fa405c1e0 0x7f5fa4154e40 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:20.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.340+0000 7f5fab238700 1 --2- 172.21.15.162:0/3860691100 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5fa405c840 0x7f5fa4149490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:20.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.340+0000 7f5fab238700 1 -- 172.21.15.162:0/3860691100 >> 172.21.15.162:0/3860691100 conn(0x7f5fa406aa70 msgr2=0x7f5fa4142b40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:20.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.340+0000 7f5fab238700 1 -- 172.21.15.162:0/3860691100 shutdown_connections 2020-11-28T21:54:20.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.340+0000 7f5fab238700 1 -- 172.21.15.162:0/3860691100 wait complete. 2020-11-28T21:54:20.362 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-11-28T21:54:20.485 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.480+0000 7f6a66cdb700 1 Processor -- start 2020-11-28T21:54:20.486 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.480+0000 7f6a66cdb700 1 -- start start 2020-11-28T21:54:20.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.484+0000 7f6a66cdb700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f6a600b2d00 0x7f6a6005c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:20.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.484+0000 7f6a66cdb700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6a600b30e0 0x7f6a6013fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:20.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.484+0000 7f6a66cdb700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a6005baf0 con 0x7f6a600b3ab0 2020-11-28T21:54:20.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.484+0000 7f6a66cdb700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a600b0e20 con 0x7f6a600b2d00 2020-11-28T21:54:20.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.484+0000 7f6a66cdb700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a6005bd90 con 0x7f6a600b30e0 2020-11-28T21:54:20.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.484+0000 7f6a664da700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6a600b30e0 0x7f6a6013fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:20.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.484+0000 7f6a664da700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6a600b30e0 0x7f6a6013fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43186/0 (socket says 172.21.15.162:43186) 2020-11-28T21:54:20.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.484+0000 7f6a654d8700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f6a600b3ab0 0x7f6a60142ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:50028/0 (socket says 172.21.15.162:50028) 2020-11-28T21:54:20.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.484+0000 7f6a664da700 1 -- 172.21.15.162:0/1435758735 learned_addr learned my addr 172.21.15.162:0/1435758735 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:20.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.484+0000 7f6a65cd9700 1 --2- 172.21.15.162:0/1435758735 >> v2:172.21.15.118:3300/0 conn(0x7f6a600b2d00 0x7f6a6005c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:20.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.484+0000 7f6a664da700 1 -- 172.21.15.162:0/1435758735 >> v2:172.21.15.118:3300/0 conn(0x7f6a600b2d00 msgr2=0x7f6a6005c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:20.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.484+0000 7f6a664da700 1 --2- 172.21.15.162:0/1435758735 >> v2:172.21.15.118:3300/0 conn(0x7f6a600b2d00 0x7f6a6005c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:20.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.484+0000 7f6a664da700 1 -- 172.21.15.162:0/1435758735 >> v1:172.21.15.162:6789/0 conn(0x7f6a600b3ab0 legacy=0x7f6a60142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:20.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.484+0000 7f6a664da700 1 -- 172.21.15.162:0/1435758735 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a6005c030 con 0x7f6a600b30e0 2020-11-28T21:54:20.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.484+0000 7f6a664da700 1 --2- 172.21.15.162:0/1435758735 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a600b30e0 0x7f6a6013fa70 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 rx=0x7f6a580032b0 tx=0x7f6a58009270).ready entity=mon.1 client_cookie=b816e46bed84153d server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:20.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.484+0000 7f6a64cd7700 1 -- 172.21.15.162:0/1435758735 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a5800bc00 con 0x7f6a600b30e0 2020-11-28T21:54:20.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.484+0000 7f6a64cd7700 1 -- 172.21.15.162:0/1435758735 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a5800bd60 con 0x7f6a600b30e0 2020-11-28T21:54:20.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.484+0000 7f6a64cd7700 1 -- 172.21.15.162:0/1435758735 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a58011400 con 0x7f6a600b30e0 2020-11-28T21:54:20.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.484+0000 7f6a66cdb700 1 -- 172.21.15.162:0/1435758735 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a600b30e0 msgr2=0x7f6a6013fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:20.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.484+0000 7f6a66cdb700 1 --2- 172.21.15.162:0/1435758735 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a600b30e0 0x7f6a6013fa70 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 rx=0x7f6a580032b0 tx=0x7f6a58009270).stop 2020-11-28T21:54:20.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.484+0000 7f6a66cdb700 1 -- 172.21.15.162:0/1435758735 shutdown_connections 2020-11-28T21:54:20.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.484+0000 7f6a66cdb700 1 --2- 172.21.15.162:0/1435758735 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a600b30e0 0x7f6a6013fa70 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:20.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.484+0000 7f6a66cdb700 1 --2- 172.21.15.162:0/1435758735 >> v2:172.21.15.118:3300/0 conn(0x7f6a600b2d00 0x7f6a6005c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:20.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.484+0000 7f6a66cdb700 1 -- 172.21.15.162:0/1435758735 >> 172.21.15.162:0/1435758735 conn(0x7f6a600604c0 msgr2=0x7f6a600608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:20.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.484+0000 7f6a66cdb700 1 -- 172.21.15.162:0/1435758735 shutdown_connections 2020-11-28T21:54:20.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.484+0000 7f6a66cdb700 1 -- 172.21.15.162:0/1435758735 wait complete. 2020-11-28T21:54:20.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.488+0000 7f6a66cdb700 1 Processor -- start 2020-11-28T21:54:20.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.488+0000 7f6a66cdb700 1 -- start start 2020-11-28T21:54:20.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.488+0000 7f6a66cdb700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6a600b2d00 0x7f6a6013b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:20.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.488+0000 7f6a66cdb700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6a600b30e0 0x7f6a60155160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:20.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.488+0000 7f6a66cdb700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a600b3ab0 0x7f6a60155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:20.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.488+0000 7f6a66cdb700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a60151940 con 0x7f6a600b30e0 2020-11-28T21:54:20.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.488+0000 7f6a66cdb700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a600b8ca0 con 0x7f6a600b3ab0 2020-11-28T21:54:20.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.488+0000 7f6a66cdb700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a600b84c0 con 0x7f6a600b2d00 2020-11-28T21:54:20.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.488+0000 7f6a664da700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a600b3ab0 0x7f6a60155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:20.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.488+0000 7f6a664da700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a600b3ab0 0x7f6a60155650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43192/0 (socket says 172.21.15.162:43192) 2020-11-28T21:54:20.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.488+0000 7f6a664da700 1 -- 172.21.15.162:0/3802777541 learned_addr learned my addr 172.21.15.162:0/3802777541 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:20.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.488+0000 7f6a65cd9700 1 --2- 172.21.15.162:0/3802777541 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6a600b2d00 0x7f6a6013b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:20.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.488+0000 7f6a654d8700 1 --2- 172.21.15.162:0/3802777541 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6a600b30e0 0x7f6a60155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:20.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.488+0000 7f6a664da700 1 -- 172.21.15.162:0/3802777541 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6a600b2d00 msgr2=0x7f6a6013b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:20.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.488+0000 7f6a664da700 1 --2- 172.21.15.162:0/3802777541 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6a600b2d00 0x7f6a6013b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:20.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.488+0000 7f6a664da700 1 -- 172.21.15.162:0/3802777541 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6a600b30e0 msgr2=0x7f6a60155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:20.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.488+0000 7f6a664da700 1 --2- 172.21.15.162:0/3802777541 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6a600b30e0 0x7f6a60155160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:20.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.488+0000 7f6a664da700 1 -- 172.21.15.162:0/3802777541 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a600b8760 con 0x7f6a600b3ab0 2020-11-28T21:54:20.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.488+0000 7f6a664da700 1 --2- 172.21.15.162:0/3802777541 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a600b3ab0 0x7f6a60155650 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 rx=0x7f6a580032b0 tx=0x7f6a58016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:20.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.488+0000 7f6a4effd700 1 -- 172.21.15.162:0/3802777541 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a5801f670 con 0x7f6a600b3ab0 2020-11-28T21:54:20.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.488+0000 7f6a4effd700 1 -- 172.21.15.162:0/3802777541 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a5801f7d0 con 0x7f6a600b3ab0 2020-11-28T21:54:20.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.488+0000 7f6a4effd700 1 -- 172.21.15.162:0/3802777541 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a5801fb30 con 0x7f6a600b3ab0 2020-11-28T21:54:20.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.488+0000 7f6a66cdb700 1 -- 172.21.15.162:0/3802777541 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a600b8a00 con 0x7f6a600b3ab0 2020-11-28T21:54:20.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.488+0000 7f6a66cdb700 1 -- 172.21.15.162:0/3802777541 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a600b8140 con 0x7f6a600b3ab0 2020-11-28T21:54:20.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.492+0000 7f6a66cdb700 1 -- 172.21.15.162:0/3802777541 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a60057050 con 0x7f6a600b3ab0 2020-11-28T21:54:20.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.492+0000 7f6a4effd700 1 -- 172.21.15.162:0/3802777541 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6a5800f070 con 0x7f6a600b3ab0 2020-11-28T21:54:20.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.492+0000 7f6a4effd700 1 --2- 172.21.15.162:0/3802777541 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6a400553b0 0x7f6a40057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:20.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.492+0000 7f6a4effd700 1 -- 172.21.15.162:0/3802777541 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f6a5806f2b0 con 0x7f6a600b3ab0 2020-11-28T21:54:20.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.492+0000 7f6a65cd9700 1 --2- 172.21.15.162:0/3802777541 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6a400553b0 0x7f6a40057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:20.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.496+0000 7f6a65cd9700 1 --2- 172.21.15.162:0/3802777541 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6a400553b0 0x7f6a40057870 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7f6a5c00c000 tx=0x7f6a5c008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:20.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.500+0000 7f6a4effd700 1 -- 172.21.15.162:0/3802777541 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6a580454c0 con 0x7f6a600b3ab0 2020-11-28T21:54:20.647 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:20.644+0000 7f6a66cdb700 1 -- 172.21.15.162:0/3802777541 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6a6005d250 con 0x7f6a600b3ab0 2020-11-28T21:54:21.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.348+0000 7f6a4effd700 1 -- 172.21.15.162:0/3802777541 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v101) v1 ==== 185+0+0 (secure 0 0 0) 0x7f6a580150e0 con 0x7f6a600b3ab0 2020-11-28T21:54:21.356 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.352+0000 7f6a66cdb700 1 -- 172.21.15.162:0/3802777541 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6a400553b0 msgr2=0x7f6a40057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:21.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.352+0000 7f6a66cdb700 1 --2- 172.21.15.162:0/3802777541 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6a400553b0 0x7f6a40057870 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7f6a5c00c000 tx=0x7f6a5c008040).stop 2020-11-28T21:54:21.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.352+0000 7f6a66cdb700 1 -- 172.21.15.162:0/3802777541 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a600b3ab0 msgr2=0x7f6a60155650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:21.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.352+0000 7f6a66cdb700 1 --2- 172.21.15.162:0/3802777541 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a600b3ab0 0x7f6a60155650 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 rx=0x7f6a580032b0 tx=0x7f6a58016000).stop 2020-11-28T21:54:21.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.352+0000 7f6a66cdb700 1 -- 172.21.15.162:0/3802777541 shutdown_connections 2020-11-28T21:54:21.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.352+0000 7f6a66cdb700 1 --2- 172.21.15.162:0/3802777541 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6a400553b0 0x7f6a40057870 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:21.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.352+0000 7f6a66cdb700 1 --2- 172.21.15.162:0/3802777541 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6a600b2d00 0x7f6a6013b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:21.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.352+0000 7f6a66cdb700 1 --2- 172.21.15.162:0/3802777541 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a600b3ab0 0x7f6a60155650 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:21.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.352+0000 7f6a66cdb700 1 --2- 172.21.15.162:0/3802777541 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6a600b30e0 0x7f6a60155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:21.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.352+0000 7f6a66cdb700 1 -- 172.21.15.162:0/3802777541 >> 172.21.15.162:0/3802777541 conn(0x7f6a600604c0 msgr2=0x7f6a60146770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:21.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.352+0000 7f6a66cdb700 1 -- 172.21.15.162:0/3802777541 shutdown_connections 2020-11-28T21:54:21.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.352+0000 7f6a66cdb700 1 -- 172.21.15.162:0/3802777541 wait complete. 2020-11-28T21:54:21.359 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_metadata' removed 2020-11-28T21:54:21.373 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-11-28T21:54:21.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.492+0000 7f79f3fff700 1 Processor -- start 2020-11-28T21:54:21.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.492+0000 7f79f3fff700 1 -- start start 2020-11-28T21:54:21.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.492+0000 7f79f3fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f79f40b3ab0 0x7f79f405c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:21.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.492+0000 7f79f3fff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f405baf0 con 0x7f79f40b2d00 2020-11-28T21:54:21.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.492+0000 7f79f3fff700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f405bd90 con 0x7f79f40b30e0 2020-11-28T21:54:21.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.492+0000 7f79f3fff700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f40b0e20 con 0x7f79f40b3ab0 2020-11-28T21:54:21.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.492+0000 7f79f37fe700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f79f40b30e0 0x7f79f413fa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:50042/0 (socket says 172.21.15.162:50042) 2020-11-28T21:54:21.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.492+0000 7f79f37fe700 1 -- 172.21.15.162:0/3319252633 learned_addr learned my addr 172.21.15.162:0/3319252633 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:21.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.492+0000 7f79f27fc700 1 --2- 172.21.15.162:0/3319252633 >> v2:172.21.15.162:3300/0 conn(0x7f79f40b3ab0 0x7f79f405c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:21.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.492+0000 7f79f1ffb700 1 -- 172.21.15.162:0/3319252633 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1542652114 0 0) 0x7f79f405bd90 con 0x7f79f40b30e0 2020-11-28T21:54:21.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.492+0000 7f79f1ffb700 1 -- 172.21.15.162:0/3319252633 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79d8003230 con 0x7f79f40b30e0 2020-11-28T21:54:21.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.492+0000 7f79f1ffb700 1 -- 172.21.15.162:0/3319252633 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2355698732 0 0) 0x7f79f405baf0 con 0x7f79f40b2d00 2020-11-28T21:54:21.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.492+0000 7f79f1ffb700 1 -- 172.21.15.162:0/3319252633 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79f405bd90 con 0x7f79f40b2d00 2020-11-28T21:54:21.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.492+0000 7f79f27fc700 1 -- 172.21.15.162:0/3319252633 >> v1:172.21.15.162:6789/0 conn(0x7f79f40b30e0 legacy=0x7f79f413fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:21.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.492+0000 7f79f27fc700 1 -- 172.21.15.162:0/3319252633 >> v1:172.21.15.118:6789/0 conn(0x7f79f40b2d00 legacy=0x7f79f4142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:21.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.492+0000 7f79f27fc700 1 -- 172.21.15.162:0/3319252633 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f405c030 con 0x7f79f40b3ab0 2020-11-28T21:54:21.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.496+0000 7f79f27fc700 1 --2- 172.21.15.162:0/3319252633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79f40b3ab0 0x7f79f405c4d0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 rx=0x7f79e8002f30 tx=0x7f79e8006dc0).ready entity=mon.1 client_cookie=d6c1f085d2278583 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:21.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.496+0000 7f79f1ffb700 1 -- 172.21.15.162:0/3319252633 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79e8009740 con 0x7f79f40b3ab0 2020-11-28T21:54:21.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.496+0000 7f79f1ffb700 1 -- 172.21.15.162:0/3319252633 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e80098a0 con 0x7f79f40b3ab0 2020-11-28T21:54:21.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.496+0000 7f79f1ffb700 1 -- 172.21.15.162:0/3319252633 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79e8009c00 con 0x7f79f40b3ab0 2020-11-28T21:54:21.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.496+0000 7f79f3fff700 1 -- 172.21.15.162:0/3319252633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79f40b3ab0 msgr2=0x7f79f405c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:21.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.496+0000 7f79f3fff700 1 --2- 172.21.15.162:0/3319252633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79f40b3ab0 0x7f79f405c4d0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 rx=0x7f79e8002f30 tx=0x7f79e8006dc0).stop 2020-11-28T21:54:21.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.496+0000 7f79f3fff700 1 -- 172.21.15.162:0/3319252633 shutdown_connections 2020-11-28T21:54:21.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.496+0000 7f79f3fff700 1 --2- 172.21.15.162:0/3319252633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79f40b3ab0 0x7f79f405c4d0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:21.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.496+0000 7f79f3fff700 1 -- 172.21.15.162:0/3319252633 >> 172.21.15.162:0/3319252633 conn(0x7f79f40604c0 msgr2=0x7f79f40608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:21.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.496+0000 7f79f3fff700 1 -- 172.21.15.162:0/3319252633 shutdown_connections 2020-11-28T21:54:21.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.496+0000 7f79f3fff700 1 -- 172.21.15.162:0/3319252633 wait complete. 2020-11-28T21:54:21.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.496+0000 7f79f3fff700 1 Processor -- start 2020-11-28T21:54:21.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.496+0000 7f79f3fff700 1 -- start start 2020-11-28T21:54:21.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.500+0000 7f79f3fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f79f40b2d00 0x7f79f4145770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:21.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.500+0000 7f79f3fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79f40b30e0 0x7f79f4151bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:21.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.500+0000 7f79f3fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f79f40b3ab0 0x7f79f41520c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:21.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.500+0000 7f79f3fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f41525b0 con 0x7f79f40b2d00 2020-11-28T21:54:21.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.500+0000 7f79f3fff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f4146860 con 0x7f79f40b30e0 2020-11-28T21:54:21.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.500+0000 7f79f3fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f4140970 con 0x7f79f40b3ab0 2020-11-28T21:54:21.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.500+0000 7f79f27fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79f40b30e0 0x7f79f4151bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:21.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.500+0000 7f79f27fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79f40b30e0 0x7f79f4151bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43206/0 (socket says 172.21.15.162:43206) 2020-11-28T21:54:21.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.500+0000 7f79f27fc700 1 -- 172.21.15.162:0/1869594003 learned_addr learned my addr 172.21.15.162:0/1869594003 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:21.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.500+0000 7f79f37fe700 1 --2- 172.21.15.162:0/1869594003 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f79f40b3ab0 0x7f79f41520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:21.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.500+0000 7f79f2ffd700 1 --2- 172.21.15.162:0/1869594003 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f79f40b2d00 0x7f79f4145770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:21.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.500+0000 7f79f27fc700 1 -- 172.21.15.162:0/1869594003 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f79f40b3ab0 msgr2=0x7f79f41520c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:21.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.500+0000 7f79f27fc700 1 --2- 172.21.15.162:0/1869594003 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f79f40b3ab0 0x7f79f41520c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:21.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.500+0000 7f79f27fc700 1 -- 172.21.15.162:0/1869594003 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f79f40b2d00 msgr2=0x7f79f4145770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:21.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.500+0000 7f79f27fc700 1 --2- 172.21.15.162:0/1869594003 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f79f40b2d00 0x7f79f4145770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:21.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.500+0000 7f79f27fc700 1 -- 172.21.15.162:0/1869594003 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f40b8ca0 con 0x7f79f40b30e0 2020-11-28T21:54:21.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.500+0000 7f79f27fc700 1 --2- 172.21.15.162:0/1869594003 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79f40b30e0 0x7f79f4151bd0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 rx=0x7f79e8002f30 tx=0x7f79e8015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:21.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.500+0000 7f79e3fff700 1 -- 172.21.15.162:0/1869594003 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79e8011b60 con 0x7f79f40b30e0 2020-11-28T21:54:21.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.500+0000 7f79e3fff700 1 -- 172.21.15.162:0/1869594003 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e8011cc0 con 0x7f79f40b30e0 2020-11-28T21:54:21.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.500+0000 7f79f3fff700 1 -- 172.21.15.162:0/1869594003 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f40b8f40 con 0x7f79f40b30e0 2020-11-28T21:54:21.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.500+0000 7f79e3fff700 1 -- 172.21.15.162:0/1869594003 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79e801d640 con 0x7f79f40b30e0 2020-11-28T21:54:21.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.500+0000 7f79f3fff700 1 -- 172.21.15.162:0/1869594003 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f41516c0 con 0x7f79f40b30e0 2020-11-28T21:54:21.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.500+0000 7f79f3fff700 1 -- 172.21.15.162:0/1869594003 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79d40033e0 con 0x7f79f40b30e0 2020-11-28T21:54:21.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.500+0000 7f79e3fff700 1 -- 172.21.15.162:0/1869594003 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f79e800f070 con 0x7f79f40b30e0 2020-11-28T21:54:21.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.504+0000 7f79e3fff700 1 --2- 172.21.15.162:0/1869594003 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f79d0055330 0x7f79d00577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:21.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.504+0000 7f79e3fff700 1 -- 172.21.15.162:0/1869594003 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5721+0+0 (secure 0 0 0) 0x7f79e806eab0 con 0x7f79f40b30e0 2020-11-28T21:54:21.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.504+0000 7f79f2ffd700 1 --2- 172.21.15.162:0/1869594003 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f79d0055330 0x7f79d00577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:21.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.504+0000 7f79f2ffd700 1 --2- 172.21.15.162:0/1869594003 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f79d0055330 0x7f79d00577f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f79e400d000 tx=0x7f79e40079f0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:21.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.508+0000 7f79e3fff700 1 -- 172.21.15.162:0/1869594003 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f79e8044f10 con 0x7f79f40b30e0 2020-11-28T21:54:21.655 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:21.652+0000 7f79f3fff700 1 -- 172.21.15.162:0/1869594003 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f79d4002300 con 0x7f79f40b30e0 2020-11-28T21:54:22.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.356+0000 7f79e3fff700 1 -- 172.21.15.162:0/1869594003 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v102) v1 ==== 173+0+0 (secure 0 0 0) 0x7f79e8022070 con 0x7f79f40b30e0 2020-11-28T21:54:22.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.360+0000 7f79f3fff700 1 -- 172.21.15.162:0/1869594003 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f79d0055330 msgr2=0x7f79d00577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:22.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.360+0000 7f79f3fff700 1 --2- 172.21.15.162:0/1869594003 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f79d0055330 0x7f79d00577f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f79e400d000 tx=0x7f79e40079f0).stop 2020-11-28T21:54:22.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.360+0000 7f79f3fff700 1 -- 172.21.15.162:0/1869594003 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79f40b30e0 msgr2=0x7f79f4151bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:22.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.360+0000 7f79f3fff700 1 --2- 172.21.15.162:0/1869594003 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79f40b30e0 0x7f79f4151bd0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 rx=0x7f79e8002f30 tx=0x7f79e8015000).stop 2020-11-28T21:54:22.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.360+0000 7f79f3fff700 1 -- 172.21.15.162:0/1869594003 shutdown_connections 2020-11-28T21:54:22.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.360+0000 7f79f3fff700 1 --2- 172.21.15.162:0/1869594003 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f79d0055330 0x7f79d00577f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:22.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.360+0000 7f79f3fff700 1 --2- 172.21.15.162:0/1869594003 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f79f40b3ab0 0x7f79f41520c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:22.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.360+0000 7f79f3fff700 1 --2- 172.21.15.162:0/1869594003 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f79f40b2d00 0x7f79f4145770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:22.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.360+0000 7f79f3fff700 1 --2- 172.21.15.162:0/1869594003 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79f40b30e0 0x7f79f4151bd0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:22.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.360+0000 7f79f3fff700 1 -- 172.21.15.162:0/1869594003 >> 172.21.15.162:0/1869594003 conn(0x7f79f40604c0 msgr2=0x7f79f40b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:22.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.360+0000 7f79f3fff700 1 -- 172.21.15.162:0/1869594003 shutdown_connections 2020-11-28T21:54:22.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.360+0000 7f79f3fff700 1 -- 172.21.15.162:0/1869594003 wait complete. 2020-11-28T21:54:22.367 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_data' removed 2020-11-28T21:54:22.380 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-28T21:54:22.380 INFO:tasks.ceph.mds.a:Stopping old one... 2020-11-28T21:54:22.381 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-11-28T21:54:22.381 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T21:54:22.382 INFO:tasks.ceph.mds.c:Restarting daemon 2020-11-28T21:54:22.382 INFO:tasks.ceph.mds.c:Stopping old one... 2020-11-28T21:54:22.382 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2020-11-28T21:54:22.383 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T21:54:22.383 INFO:tasks.ceph.mds.b:Restarting daemon 2020-11-28T21:54:22.383 INFO:tasks.ceph.mds.b:Stopping old one... 2020-11-28T21:54:22.384 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-11-28T21:54:22.384 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T21:54:22.393 INFO:tasks.ceph.mds.a:Stopped 2020-11-28T21:54:22.394 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-28T21:54:22.395 INFO:tasks.ceph.mds.c:Stopped 2020-11-28T21:54:22.395 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2020-11-28T21:54:22.396 INFO:tasks.ceph.mds.b:Stopped 2020-11-28T21:54:22.397 INFO:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-11-28T21:54:22.438 INFO:tasks.ceph.mds.a:Started 2020-11-28T21:54:22.439 DEBUG:teuthology.parallel:result is None 2020-11-28T21:54:22.440 INFO:tasks.ceph.mds.c:Started 2020-11-28T21:54:22.441 DEBUG:teuthology.parallel:result is None 2020-11-28T21:54:22.441 INFO:tasks.ceph.mds.b:Started 2020-11-28T21:54:22.442 DEBUG:teuthology.parallel:result is None 2020-11-28T21:54:22.442 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist clear 2020-11-28T21:54:22.519 INFO:tasks.ceph.mds.b.smithi118.stdout:starting mds.b at 2020-11-28T21:54:22.526 INFO:tasks.ceph.mds.c.smithi162.stdout:starting mds.c at 2020-11-28T21:54:22.534 INFO:tasks.ceph.mds.a.smithi162.stdout:starting mds.a at 2020-11-28T21:54:22.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.552+0000 7f447e5d7700 1 Processor -- start 2020-11-28T21:54:22.557 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.552+0000 7f447e5d7700 1 -- start start 2020-11-28T21:54:22.560 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.556+0000 7f447e5d7700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f447805c080 0x7f447805c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:22.560 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.556+0000 7f447e5d7700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f44780acc40 0x7f447813f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:22.560 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.556+0000 7f447e5d7700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f44780afc50 0x7f44780ae830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:22.560 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.556+0000 7f447e5d7700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44780b27f0 con 0x7f447805c080 2020-11-28T21:54:22.561 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.556+0000 7f447e5d7700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44780a8760 con 0x7f44780afc50 2020-11-28T21:54:22.561 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.556+0000 7f447e5d7700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447813fbd0 con 0x7f44780acc40 2020-11-28T21:54:22.561 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.556+0000 7f447cb74700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f44780afc50 0x7f44780ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:22.561 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.556+0000 7f447cb74700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f44780afc50 0x7f44780ae830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43238/0 (socket says 172.21.15.162:43238) 2020-11-28T21:54:22.562 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.556+0000 7f447cb74700 1 -- 172.21.15.162:0/3435988883 learned_addr learned my addr 172.21.15.162:0/3435988883 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:22.562 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.556+0000 7f44777fe700 1 --2- 172.21.15.162:0/3435988883 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f44780acc40 0x7f447813f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:22.562 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.556+0000 7f4477fff700 1 --2- 172.21.15.162:0/3435988883 >> v2:172.21.15.118:3300/0 conn(0x7f447805c080 0x7f447805c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:22.562 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.556+0000 7f447cb74700 1 -- 172.21.15.162:0/3435988883 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f44780acc40 msgr2=0x7f447813f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:22.562 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.556+0000 7f447cb74700 1 --2- 172.21.15.162:0/3435988883 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f44780acc40 0x7f447813f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:22.563 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.556+0000 7f447cb74700 1 -- 172.21.15.162:0/3435988883 >> v2:172.21.15.118:3300/0 conn(0x7f447805c080 msgr2=0x7f447805c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:22.563 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.556+0000 7f447cb74700 1 --2- 172.21.15.162:0/3435988883 >> v2:172.21.15.118:3300/0 conn(0x7f447805c080 0x7f447805c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:22.563 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.556+0000 7f447cb74700 1 -- 172.21.15.162:0/3435988883 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44780b01b0 con 0x7f44780afc50 2020-11-28T21:54:22.563 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.556+0000 7f447cb74700 1 --2- 172.21.15.162:0/3435988883 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44780afc50 0x7f44780ae830 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 rx=0x7f44700032b0 tx=0x7f4470009270).ready entity=mon.1 client_cookie=66ec9a6d536c1c53 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:22.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.556+0000 7f4476ffd700 1 -- 172.21.15.162:0/3435988883 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4470009e60 con 0x7f44780afc50 2020-11-28T21:54:22.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.556+0000 7f4476ffd700 1 -- 172.21.15.162:0/3435988883 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f447000bc00 con 0x7f44780afc50 2020-11-28T21:54:22.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.556+0000 7f4476ffd700 1 -- 172.21.15.162:0/3435988883 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f447000bd70 con 0x7f44780afc50 2020-11-28T21:54:22.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.556+0000 7f447e5d7700 1 -- 172.21.15.162:0/3435988883 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44780afc50 msgr2=0x7f44780ae830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:22.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.556+0000 7f447e5d7700 1 --2- 172.21.15.162:0/3435988883 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44780afc50 0x7f44780ae830 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 rx=0x7f44700032b0 tx=0x7f4470009270).stop 2020-11-28T21:54:22.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.556+0000 7f447e5d7700 1 -- 172.21.15.162:0/3435988883 shutdown_connections 2020-11-28T21:54:22.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.556+0000 7f447e5d7700 1 --2- 172.21.15.162:0/3435988883 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44780afc50 0x7f44780ae830 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:22.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.556+0000 7f447e5d7700 1 --2- 172.21.15.162:0/3435988883 >> v2:172.21.15.118:3300/0 conn(0x7f447805c080 0x7f447805c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:22.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.556+0000 7f447e5d7700 1 --2- 172.21.15.162:0/3435988883 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f44780acc40 0x7f447813f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:22.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.556+0000 7f447e5d7700 1 -- 172.21.15.162:0/3435988883 >> 172.21.15.162:0/3435988883 conn(0x7f447805fe50 msgr2=0x7f447805b970 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:22.566 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.556+0000 7f447e5d7700 1 -- 172.21.15.162:0/3435988883 shutdown_connections 2020-11-28T21:54:22.566 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.556+0000 7f447e5d7700 1 -- 172.21.15.162:0/3435988883 wait complete. 2020-11-28T21:54:22.568 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.556+0000 7f447e5d7700 1 Processor -- start 2020-11-28T21:54:22.569 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.556+0000 7f447e5d7700 1 -- start start 2020-11-28T21:54:22.569 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.560+0000 7f447e5d7700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f447805c080 0x7f4478144e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:22.569 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.560+0000 7f447e5d7700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44780acc40 0x7f44780a9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:22.569 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.560+0000 7f447e5d7700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f44780afc50 0x7f44780aa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:22.569 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.560+0000 7f447e5d7700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4478146ba0 con 0x7f44780acc40 2020-11-28T21:54:22.570 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.560+0000 7f447e5d7700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447813a460 con 0x7f447805c080 2020-11-28T21:54:22.570 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.560+0000 7f447e5d7700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44780aa6d0 con 0x7f44780afc50 2020-11-28T21:54:22.570 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.560+0000 7f4477fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f447805c080 0x7f4478144e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:22.570 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.560+0000 7f4477fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f447805c080 0x7f4478144e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43240/0 (socket says 172.21.15.162:43240) 2020-11-28T21:54:22.571 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.560+0000 7f4477fff700 1 -- 172.21.15.162:0/1998240971 learned_addr learned my addr 172.21.15.162:0/1998240971 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:22.571 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.560+0000 7f447cb74700 1 --2- 172.21.15.162:0/1998240971 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f44780afc50 0x7f44780aa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:22.571 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.560+0000 7f44777fe700 1 --2- 172.21.15.162:0/1998240971 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44780acc40 0x7f44780a9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:22.571 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.560+0000 7f4477fff700 1 -- 172.21.15.162:0/1998240971 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f44780afc50 msgr2=0x7f44780aa1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:22.572 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.560+0000 7f4477fff700 1 --2- 172.21.15.162:0/1998240971 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f44780afc50 0x7f44780aa1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:22.572 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.560+0000 7f4477fff700 1 -- 172.21.15.162:0/1998240971 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44780acc40 msgr2=0x7f44780a9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:22.572 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.560+0000 7f4477fff700 1 --2- 172.21.15.162:0/1998240971 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44780acc40 0x7f44780a9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:22.572 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.560+0000 7f4477fff700 1 -- 172.21.15.162:0/1998240971 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4478146900 con 0x7f447805c080 2020-11-28T21:54:22.572 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.560+0000 7f4477fff700 1 --2- 172.21.15.162:0/1998240971 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f447805c080 0x7f4478144e00 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 rx=0x7f4478143f70 tx=0x7f4468008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:22.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.560+0000 7f44757fa700 1 -- 172.21.15.162:0/1998240971 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f446800bee0 con 0x7f447805c080 2020-11-28T21:54:22.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.560+0000 7f44757fa700 1 -- 172.21.15.162:0/1998240971 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4468009980 con 0x7f447805c080 2020-11-28T21:54:22.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.560+0000 7f447e5d7700 1 -- 172.21.15.162:0/1998240971 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f447813a650 con 0x7f447805c080 2020-11-28T21:54:22.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.560+0000 7f44757fa700 1 -- 172.21.15.162:0/1998240971 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4468009ce0 con 0x7f447805c080 2020-11-28T21:54:22.574 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.560+0000 7f447e5d7700 1 -- 172.21.15.162:0/1998240971 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f447813ad90 con 0x7f447805c080 2020-11-28T21:54:22.574 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.564+0000 7f447e5d7700 1 -- 172.21.15.162:0/1998240971 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4478057050 con 0x7f447805c080 2020-11-28T21:54:22.574 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.564+0000 7f44757fa700 1 -- 172.21.15.162:0/1998240971 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4468013480 con 0x7f447805c080 2020-11-28T21:54:22.574 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.564+0000 7f44757fa700 1 --2- 172.21.15.162:0/1998240971 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f44580554e0 0x7f44580579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:22.574 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.564+0000 7f44757fa700 1 -- 172.21.15.162:0/1998240971 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f446806e1a0 con 0x7f447805c080 2020-11-28T21:54:22.575 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.564+0000 7f44777fe700 1 --2- 172.21.15.162:0/1998240971 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f44580554e0 0x7f44580579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:22.575 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.564+0000 7f44777fe700 1 --2- 172.21.15.162:0/1998240971 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f44580554e0 0x7f44580579a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f446c00b000 tx=0x7f446c006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:22.575 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.568+0000 7f44757fa700 1 -- 172.21.15.162:0/1998240971 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4468044610 con 0x7f447805c080 2020-11-28T21:54:22.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:22.724+0000 7f447e5d7700 1 -- 172.21.15.162:0/1998240971 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f44780b1d60 con 0x7f447805c080 2020-11-28T21:54:23.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.548+0000 7f44757fa700 1 -- 172.21.15.162:0/1998240971 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v105) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4468047c30 con 0x7f447805c080 2020-11-28T21:54:23.555 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.552+0000 7f447e5d7700 1 -- 172.21.15.162:0/1998240971 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f44580554e0 msgr2=0x7f44580579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:23.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.552+0000 7f447e5d7700 1 --2- 172.21.15.162:0/1998240971 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f44580554e0 0x7f44580579a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f446c00b000 tx=0x7f446c006dc0).stop 2020-11-28T21:54:23.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.552+0000 7f447e5d7700 1 -- 172.21.15.162:0/1998240971 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f447805c080 msgr2=0x7f4478144e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:23.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.552+0000 7f447e5d7700 1 --2- 172.21.15.162:0/1998240971 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f447805c080 0x7f4478144e00 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 rx=0x7f4478143f70 tx=0x7f4468008040).stop 2020-11-28T21:54:23.557 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.552+0000 7f447e5d7700 1 -- 172.21.15.162:0/1998240971 shutdown_connections 2020-11-28T21:54:23.557 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.552+0000 7f447e5d7700 1 --2- 172.21.15.162:0/1998240971 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f44580554e0 0x7f44580579a0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:23.557 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.552+0000 7f447e5d7700 1 --2- 172.21.15.162:0/1998240971 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f44780afc50 0x7f44780aa1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:23.558 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.552+0000 7f447e5d7700 1 --2- 172.21.15.162:0/1998240971 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f447805c080 0x7f4478144e00 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:23.558 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.552+0000 7f447e5d7700 1 --2- 172.21.15.162:0/1998240971 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44780acc40 0x7f44780a9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:23.558 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.552+0000 7f447e5d7700 1 -- 172.21.15.162:0/1998240971 >> 172.21.15.162:0/1998240971 conn(0x7f447805fe50 msgr2=0x7f44780abf80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:23.558 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.552+0000 7f447e5d7700 1 -- 172.21.15.162:0/1998240971 shutdown_connections 2020-11-28T21:54:23.559 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.552+0000 7f447e5d7700 1 -- 172.21.15.162:0/1998240971 wait complete. 2020-11-28T21:54:23.559 INFO:teuthology.orchestra.run.smithi162.stderr: removed all blacklist entries 2020-11-28T21:54:23.572 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2020-11-28T21:54:23.573 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth ls --format=json-pretty 2020-11-28T21:54:23.685 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.680+0000 7fd72ffff700 1 Processor -- start 2020-11-28T21:54:23.686 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.680+0000 7fd72ffff700 1 -- start start 2020-11-28T21:54:23.688 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.684+0000 7fd72ffff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd7300a9560 0x7fd7300a9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:23.688 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.684+0000 7fd72ffff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd7300b16a0 0x7fd7300b1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:23.688 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.684+0000 7fd72ffff700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7300a8890 con 0x7fd7300b1ff0 2020-11-28T21:54:23.688 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.684+0000 7fd72ffff700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7300a86a0 con 0x7fd7300a9560 2020-11-28T21:54:23.689 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.684+0000 7fd72ffff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7300a9e30 con 0x7fd7300b16a0 2020-11-28T21:54:23.689 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.684+0000 7fd72effd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd7300a9560 0x7fd7300a9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:23.689 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.684+0000 7fd72f7fe700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fd7300b1ff0 0x7fd73005f820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:50096/0 (socket says 172.21.15.162:50096) 2020-11-28T21:54:23.689 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.684+0000 7fd72f7fe700 1 -- 172.21.15.162:0/3696416080 learned_addr learned my addr 172.21.15.162:0/3696416080 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:23.689 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.684+0000 7fd72e7fc700 1 --2- 172.21.15.162:0/3696416080 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd7300b16a0 0x7fd7300b1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:23.690 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.684+0000 7fd72dffb700 1 -- 172.21.15.162:0/3696416080 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4198679647 0 0) 0x7fd7300a8890 con 0x7fd7300b1ff0 2020-11-28T21:54:23.690 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.684+0000 7fd72dffb700 1 -- 172.21.15.162:0/3696416080 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd710003230 con 0x7fd7300b1ff0 2020-11-28T21:54:23.690 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.684+0000 7fd72effd700 1 -- 172.21.15.162:0/3696416080 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd7300b16a0 msgr2=0x7fd7300b1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:23.690 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.684+0000 7fd72effd700 1 --2- 172.21.15.162:0/3696416080 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd7300b16a0 0x7fd7300b1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:23.690 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.684+0000 7fd72effd700 1 -- 172.21.15.162:0/3696416080 >> v1:172.21.15.162:6789/0 conn(0x7fd7300b1ff0 legacy=0x7fd73005f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:23.691 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.684+0000 7fd72effd700 1 -- 172.21.15.162:0/3696416080 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7300b02f0 con 0x7fd7300a9560 2020-11-28T21:54:23.691 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.684+0000 7fd72effd700 1 --2- 172.21.15.162:0/3696416080 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7300a9560 0x7fd7300a9940 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 rx=0x7fd7300a8640 tx=0x7fd720006dc0).ready entity=mon.1 client_cookie=f1d7aa154463f4ec server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:23.691 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.684+0000 7fd72dffb700 1 -- 172.21.15.162:0/3696416080 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7200096a0 con 0x7fd7300a9560 2020-11-28T21:54:23.692 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.684+0000 7fd72dffb700 1 -- 172.21.15.162:0/3696416080 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd720009800 con 0x7fd7300a9560 2020-11-28T21:54:23.692 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.684+0000 7fd72dffb700 1 -- 172.21.15.162:0/3696416080 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd720009b60 con 0x7fd7300a9560 2020-11-28T21:54:23.692 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.684+0000 7fd72ffff700 1 -- 172.21.15.162:0/3696416080 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7300a9560 msgr2=0x7fd7300a9940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:23.692 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.684+0000 7fd72ffff700 1 --2- 172.21.15.162:0/3696416080 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7300a9560 0x7fd7300a9940 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 rx=0x7fd7300a8640 tx=0x7fd720006dc0).stop 2020-11-28T21:54:23.693 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.684+0000 7fd72ffff700 1 -- 172.21.15.162:0/3696416080 shutdown_connections 2020-11-28T21:54:23.693 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.684+0000 7fd72ffff700 1 --2- 172.21.15.162:0/3696416080 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7300a9560 0x7fd7300a9940 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:23.694 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.684+0000 7fd72ffff700 1 --2- 172.21.15.162:0/3696416080 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd7300b16a0 0x7fd7300b1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:23.694 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.684+0000 7fd72ffff700 1 -- 172.21.15.162:0/3696416080 >> 172.21.15.162:0/3696416080 conn(0x7fd73005b970 msgr2=0x7fd73005bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:23.694 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.684+0000 7fd72ffff700 1 -- 172.21.15.162:0/3696416080 shutdown_connections 2020-11-28T21:54:23.694 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.684+0000 7fd72ffff700 1 -- 172.21.15.162:0/3696416080 wait complete. 2020-11-28T21:54:23.695 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.688+0000 7fd72ffff700 1 Processor -- start 2020-11-28T21:54:23.695 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.688+0000 7fd72ffff700 1 -- start start 2020-11-28T21:54:23.697 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.692+0000 7fd72ffff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd7300a9560 0x7fd73013ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:23.697 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.692+0000 7fd72ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd7300b16a0 0x7fd730140a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:23.697 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.692+0000 7fd72ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7300b1ff0 0x7fd7300ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:23.697 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.692+0000 7fd72ffff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd730140390 con 0x7fd7300b16a0 2020-11-28T21:54:23.698 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.692+0000 7fd72ffff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7300ac240 con 0x7fd7300b1ff0 2020-11-28T21:54:23.698 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.692+0000 7fd72ffff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd730141330 con 0x7fd7300a9560 2020-11-28T21:54:23.698 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.692+0000 7fd72f7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7300b1ff0 0x7fd7300ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:23.698 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.692+0000 7fd72f7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7300b1ff0 0x7fd7300ad870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43260/0 (socket says 172.21.15.162:43260) 2020-11-28T21:54:23.699 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.692+0000 7fd72f7fe700 1 -- 172.21.15.162:0/2207694476 learned_addr learned my addr 172.21.15.162:0/2207694476 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:23.699 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.692+0000 7fd72e7fc700 1 --2- 172.21.15.162:0/2207694476 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd7300b16a0 0x7fd730140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:23.699 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.692+0000 7fd72effd700 1 --2- 172.21.15.162:0/2207694476 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd7300a9560 0x7fd73013ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:23.699 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.692+0000 7fd72f7fe700 1 -- 172.21.15.162:0/2207694476 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd7300a9560 msgr2=0x7fd73013ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:23.699 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.692+0000 7fd72f7fe700 1 --2- 172.21.15.162:0/2207694476 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd7300a9560 0x7fd73013ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:23.700 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.692+0000 7fd72f7fe700 1 -- 172.21.15.162:0/2207694476 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd7300b16a0 msgr2=0x7fd730140a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:23.700 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.692+0000 7fd72f7fe700 1 --2- 172.21.15.162:0/2207694476 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd7300b16a0 0x7fd730140a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:23.700 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.692+0000 7fd72f7fe700 1 -- 172.21.15.162:0/2207694476 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7301415d0 con 0x7fd7300b1ff0 2020-11-28T21:54:23.700 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.692+0000 7fd72f7fe700 1 --2- 172.21.15.162:0/2207694476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7300b1ff0 0x7fd7300ad870 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 rx=0x7fd728001450 tx=0x7fd728007dd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:23.701 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.692+0000 7fd727fff700 1 -- 172.21.15.162:0/2207694476 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd728005960 con 0x7fd7300b1ff0 2020-11-28T21:54:23.701 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.692+0000 7fd727fff700 1 -- 172.21.15.162:0/2207694476 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd728005ac0 con 0x7fd7300b1ff0 2020-11-28T21:54:23.701 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.692+0000 7fd727fff700 1 -- 172.21.15.162:0/2207694476 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd728005e20 con 0x7fd7300b1ff0 2020-11-28T21:54:23.702 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.692+0000 7fd72ffff700 1 -- 172.21.15.162:0/2207694476 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7301405c0 con 0x7fd7300b1ff0 2020-11-28T21:54:23.702 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.692+0000 7fd72ffff700 1 -- 172.21.15.162:0/2207694476 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7300abc10 con 0x7fd7300b1ff0 2020-11-28T21:54:23.702 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.692+0000 7fd72ffff700 1 -- 172.21.15.162:0/2207694476 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd730057050 con 0x7fd7300b1ff0 2020-11-28T21:54:23.703 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.696+0000 7fd727fff700 1 -- 172.21.15.162:0/2207694476 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd728010030 con 0x7fd7300b1ff0 2020-11-28T21:54:23.703 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.696+0000 7fd727fff700 1 --2- 172.21.15.162:0/2207694476 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd7140554e0 0x7fd7140579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:23.704 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.696+0000 7fd727fff700 1 -- 172.21.15.162:0/2207694476 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fd72806dc80 con 0x7fd7300b1ff0 2020-11-28T21:54:23.704 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.696+0000 7fd72effd700 1 --2- 172.21.15.162:0/2207694476 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd7140554e0 0x7fd7140579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:23.704 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.696+0000 7fd72effd700 1 --2- 172.21.15.162:0/2207694476 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd7140554e0 0x7fd7140579a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7fd72001b000 tx=0x7fd720015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:23.706 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.700+0000 7fd727fff700 1 -- 172.21.15.162:0/2207694476 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd7280441a0 con 0x7fd7300b1ff0 2020-11-28T21:54:23.900 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.896+0000 7fd72ffff700 1 -- 172.21.15.162:0/2207694476 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd7300b2520 con 0x7fd7300b1ff0 2020-11-28T21:54:23.902 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.896+0000 7fd727fff700 1 -- 172.21.15.162:0/2207694476 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v24) v1 ==== 80+0+4255 (secure 0 0 0) 0x7fd7280477c0 con 0x7fd7300b1ff0 2020-11-28T21:54:23.903 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:54:23.904 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2020-11-28T21:54:23.904 INFO:teuthology.orchestra.run.smithi162.stdout: "auth_dump": [ 2020-11-28T21:54:23.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:23.905 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mds.a", 2020-11-28T21:54:23.906 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfWxUpKxAAIUDwQRKwz8H4Qk9qE356Bg==", 2020-11-28T21:54:23.906 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:54:23.907 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:54:23.908 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:54:23.909 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:54:23.910 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:54:23.911 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:23.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:23.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:23.911 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mds.b", 2020-11-28T21:54:23.912 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfKY2zORAAaO65etMZiYcaTDNX0nsEPw==", 2020-11-28T21:54:23.912 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:54:23.912 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:54:23.912 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:54:23.912 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:54:23.913 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:54:23.913 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:23.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:23.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:23.913 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mds.c", 2020-11-28T21:54:23.914 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfdQbRMxAAWlcjesxTZzZnvKCQneSjjQ==", 2020-11-28T21:54:23.914 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:54:23.914 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:54:23.914 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:54:23.914 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:54:23.915 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:54:23.915 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:23.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:23.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:23.916 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.0", 2020-11-28T21:54:23.916 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQADxcJfF3WLLxAAxsB5YPjH1yjv/vhlRbijqw==", 2020-11-28T21:54:23.916 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:54:23.916 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:54:23.916 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:54:23.917 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:54:23.917 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:23.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:23.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:23.917 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.1", 2020-11-28T21:54:23.918 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAGxcJfKYbBFhAA1x4DzFtkQQv6uWLIdOi1qQ==", 2020-11-28T21:54:23.918 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:54:23.918 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:54:23.918 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:54:23.918 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:54:23.919 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:23.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:23.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:23.919 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.2", 2020-11-28T21:54:23.919 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAIxcJfvcYUOhAASENjM0HfAxxyFe13mmE36Q==", 2020-11-28T21:54:23.920 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:54:23.920 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:54:23.920 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:54:23.920 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:54:23.921 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:23.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:23.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:23.921 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.3", 2020-11-28T21:54:23.922 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQALxcJfc/OHIRAAFYx3s8Ssttt15hjoH04tow==", 2020-11-28T21:54:23.922 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:54:23.922 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:54:23.922 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:54:23.922 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:54:23.923 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:23.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:23.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:23.923 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.4", 2020-11-28T21:54:23.923 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAPxcJffGWqCRAAb8CCN5oUPkdDh6BJH5HEnw==", 2020-11-28T21:54:23.924 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:54:23.924 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:54:23.924 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:54:23.924 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:54:23.924 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:23.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:23.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:23.925 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.5", 2020-11-28T21:54:23.925 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQARxcJfo8GKKRAAfbsCDb4L2wULiKmHyzu7/A==", 2020-11-28T21:54:23.925 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:54:23.925 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:54:23.926 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:54:23.926 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:54:23.926 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:23.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:23.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:23.927 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.6", 2020-11-28T21:54:23.927 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAUxcJf8Y2MEBAAqQX6rX3LK0zodlI2wK4JrA==", 2020-11-28T21:54:23.927 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:54:23.927 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:54:23.927 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:54:23.928 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:54:23.928 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:23.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:23.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:23.928 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.7", 2020-11-28T21:54:23.929 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAWxcJfCLBdMhAAf8QixQMQDrfZOGWN8rHe5g==", 2020-11-28T21:54:23.929 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:54:23.929 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:54:23.929 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:54:23.929 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:54:23.930 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:23.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:23.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:23.930 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "client.0", 2020-11-28T21:54:23.930 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQABxcJfS9vDAxAAUk4FU8FVt4if/CRH5ZisOA==", 2020-11-28T21:54:23.931 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:54:23.931 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:54:23.931 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow r", 2020-11-28T21:54:23.931 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow rw pool=cephfs_data" 2020-11-28T21:54:23.932 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:23.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:23.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:23.932 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "client.1", 2020-11-28T21:54:23.933 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQABxcJfuCU8BhAAXUSoSK3D0XRo58uHNzcjXg==", 2020-11-28T21:54:23.933 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:54:23.933 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:54:23.933 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow r", 2020-11-28T21:54:23.933 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow rw pool=cephfs_data" 2020-11-28T21:54:23.934 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:23.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:23.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:23.934 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "client.admin", 2020-11-28T21:54:23.934 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfdlTWBxAApzAxlVjzdnETQF//Vnsg5w==", 2020-11-28T21:54:23.935 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:54:23.935 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow *", 2020-11-28T21:54:23.935 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:54:23.935 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:54:23.935 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:54:23.936 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:23.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:23.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:23.936 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mgr.x", 2020-11-28T21:54:23.936 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfZhdvKBAAT7zCCsD2OVCHjvddGLxQ/A==", 2020-11-28T21:54:23.937 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:54:23.937 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow *", 2020-11-28T21:54:23.937 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow profile mgr", 2020-11-28T21:54:23.937 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:54:23.937 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:23.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:23.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:23.938 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mgr.y", 2020-11-28T21:54:23.938 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfs8fvJRAAP0Atp074EeMHiTtpCCVHuA==", 2020-11-28T21:54:23.938 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:54:23.938 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow *", 2020-11-28T21:54:23.939 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow profile mgr", 2020-11-28T21:54:23.939 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:54:23.939 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:23.939 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:23.939 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:23.940 INFO:teuthology.orchestra.run.smithi162.stdout:} 2020-11-28T21:54:23.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.900+0000 7fd72ffff700 1 -- 172.21.15.162:0/2207694476 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd7140554e0 msgr2=0x7fd7140579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:23.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.900+0000 7fd72ffff700 1 --2- 172.21.15.162:0/2207694476 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd7140554e0 0x7fd7140579a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7fd72001b000 tx=0x7fd720015000).stop 2020-11-28T21:54:23.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.900+0000 7fd72ffff700 1 -- 172.21.15.162:0/2207694476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7300b1ff0 msgr2=0x7fd7300ad870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:23.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.900+0000 7fd72ffff700 1 --2- 172.21.15.162:0/2207694476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7300b1ff0 0x7fd7300ad870 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 rx=0x7fd728001450 tx=0x7fd728007dd0).stop 2020-11-28T21:54:23.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.900+0000 7fd72ffff700 1 -- 172.21.15.162:0/2207694476 shutdown_connections 2020-11-28T21:54:23.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.900+0000 7fd72ffff700 1 --2- 172.21.15.162:0/2207694476 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd7140554e0 0x7fd7140579a0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:23.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.900+0000 7fd72ffff700 1 --2- 172.21.15.162:0/2207694476 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd7300a9560 0x7fd73013ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:23.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.900+0000 7fd72ffff700 1 --2- 172.21.15.162:0/2207694476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7300b1ff0 0x7fd7300ad870 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:23.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.900+0000 7fd72ffff700 1 --2- 172.21.15.162:0/2207694476 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd7300b16a0 0x7fd730140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:23.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.900+0000 7fd72ffff700 1 -- 172.21.15.162:0/2207694476 >> 172.21.15.162:0/2207694476 conn(0x7fd73005b970 msgr2=0x7fd7300af010 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:23.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.900+0000 7fd72ffff700 1 -- 172.21.15.162:0/2207694476 shutdown_connections 2020-11-28T21:54:23.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:23.900+0000 7fd72ffff700 1 -- 172.21.15.162:0/2207694476 wait complete. 2020-11-28T21:54:23.944 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:54:24.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.056+0000 7fbcfd93e700 1 Processor -- start 2020-11-28T21:54:24.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.056+0000 7fbcfd93e700 1 -- start start 2020-11-28T21:54:24.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.056+0000 7fbcfd93e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fbcf80994f0 0x7fbcf80b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:24.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.056+0000 7fbcfd93e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcf8098f10 con 0x7fbcf8060710 2020-11-28T21:54:24.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.056+0000 7fbcfd93e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcf80b0680 con 0x7fbcf8098b30 2020-11-28T21:54:24.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.056+0000 7fbcfd93e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcf80b1030 con 0x7fbcf80994f0 2020-11-28T21:54:24.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.056+0000 7fbcfc93c700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fbcf8098b30 0x7fbcf8060000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:50110/0 (socket says 172.21.15.162:50110) 2020-11-28T21:54:24.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.056+0000 7fbcfc93c700 1 -- 172.21.15.162:0/1359050273 learned_addr learned my addr 172.21.15.162:0/1359050273 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:24.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.056+0000 7fbcf7fff700 1 --2- 172.21.15.162:0/1359050273 >> v2:172.21.15.118:3300/0 conn(0x7fbcf80994f0 0x7fbcf80b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:24.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.056+0000 7fbcf77fe700 1 -- 172.21.15.162:0/1359050273 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2553338744 0 0) 0x7fbcf80b0680 con 0x7fbcf8098b30 2020-11-28T21:54:24.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.056+0000 7fbcf77fe700 1 -- 172.21.15.162:0/1359050273 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcdc003230 con 0x7fbcf8098b30 2020-11-28T21:54:24.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.056+0000 7fbcf77fe700 1 -- 172.21.15.162:0/1359050273 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3098140259 0 0) 0x7fbcdc003230 con 0x7fbcf8098b30 2020-11-28T21:54:24.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.056+0000 7fbcf7fff700 1 -- 172.21.15.162:0/1359050273 >> v1:172.21.15.162:6789/0 conn(0x7fbcf8098b30 legacy=0x7fbcf8060000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:24.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.056+0000 7fbcf7fff700 1 -- 172.21.15.162:0/1359050273 >> v1:172.21.15.118:6789/0 conn(0x7fbcf8060710 legacy=0x7fbcf80b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:24.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.056+0000 7fbcf7fff700 1 -- 172.21.15.162:0/1359050273 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf80b0920 con 0x7fbcf80994f0 2020-11-28T21:54:24.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.056+0000 7fbcf7fff700 1 --2- 172.21.15.162:0/1359050273 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbcf80994f0 0x7fbcf80b0000 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7fbcf00010d0 tx=0x7fbce0006dc0).ready entity=mon.0 client_cookie=f3c6cd9432ae369e server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:24.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.056+0000 7fbcf77fe700 1 -- 172.21.15.162:0/1359050273 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbce8001a50 con 0x7fbcf80994f0 2020-11-28T21:54:24.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.060+0000 7fbcf77fe700 1 -- 172.21.15.162:0/1359050273 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce0009620 con 0x7fbcf80994f0 2020-11-28T21:54:24.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.060+0000 7fbcf77fe700 1 -- 172.21.15.162:0/1359050273 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbce0009790 con 0x7fbcf80994f0 2020-11-28T21:54:24.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.060+0000 7fbcfd93e700 1 -- 172.21.15.162:0/1359050273 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbcf80994f0 msgr2=0x7fbcf80b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:24.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.060+0000 7fbcfd93e700 1 --2- 172.21.15.162:0/1359050273 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbcf80994f0 0x7fbcf80b0000 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7fbcf00010d0 tx=0x7fbce0006dc0).stop 2020-11-28T21:54:24.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.060+0000 7fbcfd93e700 1 -- 172.21.15.162:0/1359050273 shutdown_connections 2020-11-28T21:54:24.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.060+0000 7fbcfd93e700 1 --2- 172.21.15.162:0/1359050273 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbcf80994f0 0x7fbcf80b0000 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:24.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.060+0000 7fbcfd93e700 1 -- 172.21.15.162:0/1359050273 >> 172.21.15.162:0/1359050273 conn(0x7fbcf805b970 msgr2=0x7fbcf805bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:24.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.060+0000 7fbcfd93e700 1 -- 172.21.15.162:0/1359050273 shutdown_connections 2020-11-28T21:54:24.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.060+0000 7fbcfd93e700 1 -- 172.21.15.162:0/1359050273 wait complete. 2020-11-28T21:54:24.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.060+0000 7fbcfd93e700 1 Processor -- start 2020-11-28T21:54:24.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.060+0000 7fbcfd93e700 1 -- start start 2020-11-28T21:54:24.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.060+0000 7fbcfd93e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbcf8060710 0x7fbcf814ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:24.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.060+0000 7fbcfd93e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbcf8098b30 0x7fbcf80acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:24.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.060+0000 7fbcfd93e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcf80994f0 0x7fbcf80ad490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:24.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.060+0000 7fbcfd93e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf815a480 con 0x7fbcf8098b30 2020-11-28T21:54:24.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.060+0000 7fbcfd93e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf814d760 con 0x7fbcf80994f0 2020-11-28T21:54:24.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.060+0000 7fbcfd93e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf815a1e0 con 0x7fbcf8060710 2020-11-28T21:54:24.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.060+0000 7fbcfd13d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcf80994f0 0x7fbcf80ad490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:24.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.060+0000 7fbcfd13d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcf80994f0 0x7fbcf80ad490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43272/0 (socket says 172.21.15.162:43272) 2020-11-28T21:54:24.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.060+0000 7fbcfd13d700 1 -- 172.21.15.162:0/4028208883 learned_addr learned my addr 172.21.15.162:0/4028208883 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:24.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.060+0000 7fbcf7fff700 1 --2- 172.21.15.162:0/4028208883 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbcf8098b30 0x7fbcf80acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:24.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.060+0000 7fbcfc93c700 1 --2- 172.21.15.162:0/4028208883 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbcf8060710 0x7fbcf814ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:24.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.060+0000 7fbcfd13d700 1 -- 172.21.15.162:0/4028208883 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbcf8060710 msgr2=0x7fbcf814ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:24.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.060+0000 7fbcfd13d700 1 --2- 172.21.15.162:0/4028208883 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbcf8060710 0x7fbcf814ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:24.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.060+0000 7fbcfd13d700 1 -- 172.21.15.162:0/4028208883 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbcf8098b30 msgr2=0x7fbcf80acfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:24.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.060+0000 7fbcfd13d700 1 --2- 172.21.15.162:0/4028208883 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbcf8098b30 0x7fbcf80acfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:24.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.060+0000 7fbcfd13d700 1 -- 172.21.15.162:0/4028208883 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf8159f00 con 0x7fbcf80994f0 2020-11-28T21:54:24.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.064+0000 7fbcfd13d700 1 --2- 172.21.15.162:0/4028208883 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcf80994f0 0x7fbcf80ad490 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 rx=0x7fbcf8063c00 tx=0x7fbcf0007b00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:24.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.064+0000 7fbcf5ffb700 1 -- 172.21.15.162:0/4028208883 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbcf000f470 con 0x7fbcf80994f0 2020-11-28T21:54:24.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.064+0000 7fbcfd93e700 1 -- 172.21.15.162:0/4028208883 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcf814d330 con 0x7fbcf80994f0 2020-11-28T21:54:24.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.064+0000 7fbcf5ffb700 1 -- 172.21.15.162:0/4028208883 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcf000f5d0 con 0x7fbcf80994f0 2020-11-28T21:54:24.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.064+0000 7fbcfd93e700 1 -- 172.21.15.162:0/4028208883 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcf814c240 con 0x7fbcf80994f0 2020-11-28T21:54:24.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.064+0000 7fbcf5ffb700 1 -- 172.21.15.162:0/4028208883 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbcf000f930 con 0x7fbcf80994f0 2020-11-28T21:54:24.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.064+0000 7fbcfd93e700 1 -- 172.21.15.162:0/4028208883 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcf8057050 con 0x7fbcf80994f0 2020-11-28T21:54:24.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.064+0000 7fbcf5ffb700 1 -- 172.21.15.162:0/4028208883 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fbcf0019030 con 0x7fbcf80994f0 2020-11-28T21:54:24.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.064+0000 7fbcf5ffb700 1 --2- 172.21.15.162:0/4028208883 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbcd0055440 0x7fbcd0057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:24.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.064+0000 7fbcf5ffb700 1 -- 172.21.15.162:0/4028208883 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fbcf006cfe0 con 0x7fbcf80994f0 2020-11-28T21:54:24.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.068+0000 7fbcfc93c700 1 --2- 172.21.15.162:0/4028208883 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbcd0055440 0x7fbcd0057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:24.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.068+0000 7fbcfc93c700 1 --2- 172.21.15.162:0/4028208883 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbcd0055440 0x7fbcd0057900 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7fbce800d000 tx=0x7fbce8000f00).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:24.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.072+0000 7fbcf5ffb700 1 -- 172.21.15.162:0/4028208883 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fbcf0041df0 con 0x7fbcf80994f0 2020-11-28T21:54:24.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.212+0000 7fbcfd93e700 1 -- 172.21.15.162:0/4028208883 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbcf805c6f0 con 0x7fbcf80994f0 2020-11-28T21:54:24.217 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.212+0000 7fbcf5ffb700 1 -- 172.21.15.162:0/4028208883 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+13279 (secure 0 0 0) 0x7fbcf00243f0 con 0x7fbcf80994f0 2020-11-28T21:54:24.218 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:54:24.218 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":105,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:54:23.542721+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:54:24.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.216+0000 7fbcfd93e700 1 -- 172.21.15.162:0/4028208883 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbcd0055440 msgr2=0x7fbcd0057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:24.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.216+0000 7fbcfd93e700 1 --2- 172.21.15.162:0/4028208883 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbcd0055440 0x7fbcd0057900 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7fbce800d000 tx=0x7fbce8000f00).stop 2020-11-28T21:54:24.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.216+0000 7fbcfd93e700 1 -- 172.21.15.162:0/4028208883 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcf80994f0 msgr2=0x7fbcf80ad490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:24.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.216+0000 7fbcfd93e700 1 --2- 172.21.15.162:0/4028208883 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcf80994f0 0x7fbcf80ad490 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 rx=0x7fbcf8063c00 tx=0x7fbcf0007b00).stop 2020-11-28T21:54:24.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.216+0000 7fbcfd93e700 1 -- 172.21.15.162:0/4028208883 shutdown_connections 2020-11-28T21:54:24.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.216+0000 7fbcfd93e700 1 --2- 172.21.15.162:0/4028208883 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbcd0055440 0x7fbcd0057900 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:24.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.216+0000 7fbcfd93e700 1 --2- 172.21.15.162:0/4028208883 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbcf8060710 0x7fbcf814ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:24.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.216+0000 7fbcfd93e700 1 --2- 172.21.15.162:0/4028208883 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcf80994f0 0x7fbcf80ad490 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:24.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.216+0000 7fbcfd93e700 1 --2- 172.21.15.162:0/4028208883 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbcf8098b30 0x7fbcf80acfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:24.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.216+0000 7fbcfd93e700 1 -- 172.21.15.162:0/4028208883 >> 172.21.15.162:0/4028208883 conn(0x7fbcf805b970 msgr2=0x7fbcf8148970 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:24.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.216+0000 7fbcfd93e700 1 -- 172.21.15.162:0/4028208883 shutdown_connections 2020-11-28T21:54:24.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.216+0000 7fbcfd93e700 1 -- 172.21.15.162:0/4028208883 wait complete. 2020-11-28T21:54:24.231 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-28T21:54:24.232 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:54:24.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.344+0000 7f411f23b700 1 Processor -- start 2020-11-28T21:54:24.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.344+0000 7f411f23b700 1 -- start start 2020-11-28T21:54:24.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.344+0000 7f411f23b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f41180b1eb0 0x7f41180b2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:24.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.344+0000 7f411f23b700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4118060a70 0x7f41180afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:24.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.344+0000 7f411f23b700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4118060f00 con 0x7f41180af6a0 2020-11-28T21:54:24.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.344+0000 7f411f23b700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41181471f0 con 0x7f41180b1eb0 2020-11-28T21:54:24.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.344+0000 7f411f23b700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4118060090 con 0x7f4118060a70 2020-11-28T21:54:24.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.344+0000 7f411e239700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f41180b1eb0 0x7f41180b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:24.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.344+0000 7f411e239700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f41180b1eb0 0x7f41180b2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43278/0 (socket says 172.21.15.162:43278) 2020-11-28T21:54:24.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.344+0000 7f411e239700 1 -- 172.21.15.162:0/1335597089 learned_addr learned my addr 172.21.15.162:0/1335597089 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:24.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.344+0000 7f411ea3a700 1 --2- 172.21.15.162:0/1335597089 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4118060a70 0x7f41180afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:24.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.344+0000 7f411e239700 1 -- 172.21.15.162:0/1335597089 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4118060a70 msgr2=0x7f41180afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:24.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.344+0000 7f411e239700 1 --2- 172.21.15.162:0/1335597089 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4118060a70 0x7f41180afdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:24.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.344+0000 7f411e239700 1 -- 172.21.15.162:0/1335597089 >> v1:172.21.15.118:6789/0 conn(0x7f41180af6a0 legacy=0x7f4118144350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:24.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.344+0000 7f411e239700 1 -- 172.21.15.162:0/1335597089 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4118060330 con 0x7f41180b1eb0 2020-11-28T21:54:24.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.348+0000 7f411e239700 1 --2- 172.21.15.162:0/1335597089 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41180b1eb0 0x7f41180b2290 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 rx=0x7f4108002f30 tx=0x7f4108006e80).ready entity=mon.1 client_cookie=34ce365a496cbf20 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:24.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.348+0000 7f411d237700 1 -- 172.21.15.162:0/1335597089 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f41080097f0 con 0x7f41180b1eb0 2020-11-28T21:54:24.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.348+0000 7f411d237700 1 -- 172.21.15.162:0/1335597089 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4108009950 con 0x7f41180b1eb0 2020-11-28T21:54:24.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.348+0000 7f411d237700 1 -- 172.21.15.162:0/1335597089 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4108009cb0 con 0x7f41180b1eb0 2020-11-28T21:54:24.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.348+0000 7f411f23b700 1 -- 172.21.15.162:0/1335597089 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41180b1eb0 msgr2=0x7f41180b2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:24.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.348+0000 7f411f23b700 1 --2- 172.21.15.162:0/1335597089 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41180b1eb0 0x7f41180b2290 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 rx=0x7f4108002f30 tx=0x7f4108006e80).stop 2020-11-28T21:54:24.356 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.348+0000 7f411f23b700 1 -- 172.21.15.162:0/1335597089 shutdown_connections 2020-11-28T21:54:24.356 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.348+0000 7f411f23b700 1 --2- 172.21.15.162:0/1335597089 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4118060a70 0x7f41180afdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:24.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.348+0000 7f411f23b700 1 --2- 172.21.15.162:0/1335597089 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41180b1eb0 0x7f41180b2290 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:24.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.348+0000 7f411f23b700 1 -- 172.21.15.162:0/1335597089 >> 172.21.15.162:0/1335597089 conn(0x7f41180b57d0 msgr2=0x7f41180b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:24.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.348+0000 7f411f23b700 1 -- 172.21.15.162:0/1335597089 shutdown_connections 2020-11-28T21:54:24.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.348+0000 7f411f23b700 1 -- 172.21.15.162:0/1335597089 wait complete. 2020-11-28T21:54:24.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.348+0000 7f411f23b700 1 Processor -- start 2020-11-28T21:54:24.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.348+0000 7f411f23b700 1 -- start start 2020-11-28T21:54:24.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.348+0000 7f411f23b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4118060a70 0x7f41180b3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:24.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.348+0000 7f411f23b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41180af6a0 0x7f41180aa310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:24.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.348+0000 7f411f23b700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f41180b1eb0 0x7f41180aa800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:24.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.352+0000 7f411f23b700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41180a9c40 con 0x7f4118060a70 2020-11-28T21:54:24.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.352+0000 7f411f23b700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411814b340 con 0x7f41180af6a0 2020-11-28T21:54:24.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.352+0000 7f411f23b700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4118156d70 con 0x7f41180b1eb0 2020-11-28T21:54:24.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.352+0000 7f411da38700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41180af6a0 0x7f41180aa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:24.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.352+0000 7f411da38700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41180af6a0 0x7f41180aa310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43284/0 (socket says 172.21.15.162:43284) 2020-11-28T21:54:24.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.352+0000 7f411da38700 1 -- 172.21.15.162:0/4248939979 learned_addr learned my addr 172.21.15.162:0/4248939979 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:24.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.352+0000 7f411ea3a700 1 --2- 172.21.15.162:0/4248939979 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f41180b1eb0 0x7f41180aa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:24.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.352+0000 7f411e239700 1 --2- 172.21.15.162:0/4248939979 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4118060a70 0x7f41180b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:24.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.352+0000 7f411da38700 1 -- 172.21.15.162:0/4248939979 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f41180b1eb0 msgr2=0x7f41180aa800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:24.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.352+0000 7f411da38700 1 --2- 172.21.15.162:0/4248939979 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f41180b1eb0 0x7f41180aa800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:24.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.352+0000 7f411da38700 1 -- 172.21.15.162:0/4248939979 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4118060a70 msgr2=0x7f41180b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:24.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.352+0000 7f411da38700 1 --2- 172.21.15.162:0/4248939979 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4118060a70 0x7f41180b3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:24.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.352+0000 7f411da38700 1 -- 172.21.15.162:0/4248939979 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41180a9e70 con 0x7f41180af6a0 2020-11-28T21:54:24.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.352+0000 7f411da38700 1 --2- 172.21.15.162:0/4248939979 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41180af6a0 0x7f41180aa310 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 rx=0x7f41181561a0 tx=0x7f410c007bb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:24.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.352+0000 7f41177fe700 1 -- 172.21.15.162:0/4248939979 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f410c007240 con 0x7f41180af6a0 2020-11-28T21:54:24.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.352+0000 7f41177fe700 1 -- 172.21.15.162:0/4248939979 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f410c0073a0 con 0x7f41180af6a0 2020-11-28T21:54:24.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.352+0000 7f411f23b700 1 -- 172.21.15.162:0/4248939979 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4118157010 con 0x7f41180af6a0 2020-11-28T21:54:24.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.352+0000 7f41177fe700 1 -- 172.21.15.162:0/4248939979 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f410c0066d0 con 0x7f41180af6a0 2020-11-28T21:54:24.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.352+0000 7f411f23b700 1 -- 172.21.15.162:0/4248939979 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41180b34f0 con 0x7f41180af6a0 2020-11-28T21:54:24.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.352+0000 7f411f23b700 1 -- 172.21.15.162:0/4248939979 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4118057050 con 0x7f41180af6a0 2020-11-28T21:54:24.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.352+0000 7f41177fe700 1 -- 172.21.15.162:0/4248939979 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f410c010070 con 0x7f41180af6a0 2020-11-28T21:54:24.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.352+0000 7f41177fe700 1 --2- 172.21.15.162:0/4248939979 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f40f80554e0 0x7f40f80579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:24.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.356+0000 7f41177fe700 1 -- 172.21.15.162:0/4248939979 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f410c06d440 con 0x7f41180af6a0 2020-11-28T21:54:24.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.356+0000 7f411e239700 1 --2- 172.21.15.162:0/4248939979 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f40f80554e0 0x7f40f80579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:24.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.356+0000 7f411e239700 1 --2- 172.21.15.162:0/4248939979 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f40f80554e0 0x7f40f80579a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f410801a000 tx=0x7f4108015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:24.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.360+0000 7f41177fe700 1 -- 172.21.15.162:0/4248939979 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f410c0439d0 con 0x7f41180af6a0 2020-11-28T21:54:24.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.500+0000 7f411f23b700 1 -- 172.21.15.162:0/4248939979 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f41180a89f0 con 0x7f41180af6a0 2020-11-28T21:54:24.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.500+0000 7f41177fe700 1 -- 172.21.15.162:0/4248939979 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v105) v1 ==== 110+0+10 (secure 0 0 0) 0x7f410c018020 con 0x7f41180af6a0 2020-11-28T21:54:24.506 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:54:24.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.504+0000 7f411f23b700 1 -- 172.21.15.162:0/4248939979 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f40f80554e0 msgr2=0x7f40f80579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:24.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.504+0000 7f411f23b700 1 --2- 172.21.15.162:0/4248939979 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f40f80554e0 0x7f40f80579a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f410801a000 tx=0x7f4108015000).stop 2020-11-28T21:54:24.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.504+0000 7f411f23b700 1 -- 172.21.15.162:0/4248939979 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41180af6a0 msgr2=0x7f41180aa310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:24.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.504+0000 7f411f23b700 1 --2- 172.21.15.162:0/4248939979 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41180af6a0 0x7f41180aa310 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 rx=0x7f41181561a0 tx=0x7f410c007bb0).stop 2020-11-28T21:54:24.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.504+0000 7f411f23b700 1 -- 172.21.15.162:0/4248939979 shutdown_connections 2020-11-28T21:54:24.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.504+0000 7f411f23b700 1 --2- 172.21.15.162:0/4248939979 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f40f80554e0 0x7f40f80579a0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:24.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.504+0000 7f411f23b700 1 --2- 172.21.15.162:0/4248939979 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f41180b1eb0 0x7f41180aa800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:24.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.504+0000 7f411f23b700 1 --2- 172.21.15.162:0/4248939979 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4118060a70 0x7f41180b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:24.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.504+0000 7f411f23b700 1 --2- 172.21.15.162:0/4248939979 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41180af6a0 0x7f41180aa310 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:24.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.504+0000 7f411f23b700 1 -- 172.21.15.162:0/4248939979 >> 172.21.15.162:0/4248939979 conn(0x7f41180b57d0 msgr2=0x7f41180abce0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:24.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.504+0000 7f411f23b700 1 -- 172.21.15.162:0/4248939979 shutdown_connections 2020-11-28T21:54:24.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.504+0000 7f411f23b700 1 -- 172.21.15.162:0/4248939979 wait complete. 2020-11-28T21:54:24.520 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:54:24.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.636+0000 7fe94d65c700 1 Processor -- start 2020-11-28T21:54:24.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.636+0000 7fe94d65c700 1 -- start start 2020-11-28T21:54:24.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.636+0000 7fe94d65c700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe94813cb50 0x7fe948140180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:24.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.636+0000 7fe94d65c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe94805cca0 0x7fe9480b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:24.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.636+0000 7fe94d65c700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9480b1360 con 0x7fe9480b23a0 2020-11-28T21:54:24.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.636+0000 7fe94d65c700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9480add60 con 0x7fe94805cca0 2020-11-28T21:54:24.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.636+0000 7fe94d65c700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9480ae8c0 con 0x7fe94813cb50 2020-11-28T21:54:24.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.636+0000 7fe9477fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe94805cca0 0x7fe9480b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:24.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.636+0000 7fe9477fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe94805cca0 0x7fe9480b1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43294/0 (socket says 172.21.15.162:43294) 2020-11-28T21:54:24.642 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.636+0000 7fe9477fe700 1 -- 172.21.15.162:0/772941888 learned_addr learned my addr 172.21.15.162:0/772941888 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:24.642 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.636+0000 7fe947fff700 1 --2- 172.21.15.162:0/772941888 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe94813cb50 0x7fe948140180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:24.643 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.636+0000 7fe9477fe700 1 -- 172.21.15.162:0/772941888 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe94813cb50 msgr2=0x7fe948140180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:24.643 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.636+0000 7fe9477fe700 1 --2- 172.21.15.162:0/772941888 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe94813cb50 0x7fe948140180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:24.644 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.636+0000 7fe9477fe700 1 -- 172.21.15.162:0/772941888 >> v1:172.21.15.118:6789/0 conn(0x7fe9480b23a0 legacy=0x7fe9480ac350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:24.644 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.636+0000 7fe9477fe700 1 -- 172.21.15.162:0/772941888 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9480aeb60 con 0x7fe94805cca0 2020-11-28T21:54:24.644 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.636+0000 7fe9477fe700 1 --2- 172.21.15.162:0/772941888 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe94805cca0 0x7fe9480b1eb0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 rx=0x7fe93c002f30 tx=0x7fe93c006e80).ready entity=mon.1 client_cookie=d004b7b28b60d6f6 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:24.644 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.636+0000 7fe946ffd700 1 -- 172.21.15.162:0/772941888 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe93c0097f0 con 0x7fe94805cca0 2020-11-28T21:54:24.645 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.636+0000 7fe946ffd700 1 -- 172.21.15.162:0/772941888 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe93c009950 con 0x7fe94805cca0 2020-11-28T21:54:24.645 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.636+0000 7fe946ffd700 1 -- 172.21.15.162:0/772941888 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe93c009cb0 con 0x7fe94805cca0 2020-11-28T21:54:24.645 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.636+0000 7fe94d65c700 1 -- 172.21.15.162:0/772941888 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe94805cca0 msgr2=0x7fe9480b1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:24.645 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.636+0000 7fe94d65c700 1 --2- 172.21.15.162:0/772941888 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe94805cca0 0x7fe9480b1eb0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 rx=0x7fe93c002f30 tx=0x7fe93c006e80).stop 2020-11-28T21:54:24.645 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.640+0000 7fe94d65c700 1 -- 172.21.15.162:0/772941888 shutdown_connections 2020-11-28T21:54:24.646 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.640+0000 7fe94d65c700 1 --2- 172.21.15.162:0/772941888 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe94813cb50 0x7fe948140180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:24.646 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.640+0000 7fe94d65c700 1 --2- 172.21.15.162:0/772941888 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe94805cca0 0x7fe9480b1eb0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:24.646 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.640+0000 7fe94d65c700 1 -- 172.21.15.162:0/772941888 >> 172.21.15.162:0/772941888 conn(0x7fe9480b57d0 msgr2=0x7fe9480b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:24.646 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.640+0000 7fe94d65c700 1 -- 172.21.15.162:0/772941888 shutdown_connections 2020-11-28T21:54:24.646 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.640+0000 7fe94d65c700 1 -- 172.21.15.162:0/772941888 wait complete. 2020-11-28T21:54:24.649 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.640+0000 7fe94d65c700 1 Processor -- start 2020-11-28T21:54:24.649 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.640+0000 7fe94d65c700 1 -- start start 2020-11-28T21:54:24.649 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.644+0000 7fe94d65c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe94805cca0 0x7fe948060400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:24.649 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.644+0000 7fe94d65c700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe94813cb50 0x7fe948140bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:24.649 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.644+0000 7fe94d65c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe948153990 0x7fe94814f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:24.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.644+0000 7fe94d65c700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9481410a0 con 0x7fe94805cca0 2020-11-28T21:54:24.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.644+0000 7fe94d65c700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9481412c0 con 0x7fe948153990 2020-11-28T21:54:24.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.644+0000 7fe94d65c700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe948152fb0 con 0x7fe94813cb50 2020-11-28T21:54:24.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.644+0000 7fe94ce5b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe948153990 0x7fe94814f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:24.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.644+0000 7fe94ce5b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe948153990 0x7fe94814f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43302/0 (socket says 172.21.15.162:43302) 2020-11-28T21:54:24.651 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.644+0000 7fe94ce5b700 1 -- 172.21.15.162:0/2195230679 learned_addr learned my addr 172.21.15.162:0/2195230679 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:24.651 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.644+0000 7fe9477fe700 1 --2- 172.21.15.162:0/2195230679 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe94813cb50 0x7fe948140bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:24.651 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.644+0000 7fe947fff700 1 --2- 172.21.15.162:0/2195230679 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe94805cca0 0x7fe948060400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:24.651 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.644+0000 7fe94ce5b700 1 -- 172.21.15.162:0/2195230679 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe94813cb50 msgr2=0x7fe948140bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:24.652 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.644+0000 7fe94ce5b700 1 --2- 172.21.15.162:0/2195230679 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe94813cb50 0x7fe948140bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:24.652 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.644+0000 7fe94ce5b700 1 -- 172.21.15.162:0/2195230679 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe94805cca0 msgr2=0x7fe948060400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:24.652 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.644+0000 7fe94ce5b700 1 --2- 172.21.15.162:0/2195230679 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe94805cca0 0x7fe948060400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:24.652 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.644+0000 7fe94ce5b700 1 -- 172.21.15.162:0/2195230679 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe948153250 con 0x7fe948153990 2020-11-28T21:54:24.652 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.644+0000 7fe9477fe700 1 --2- 172.21.15.162:0/2195230679 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe94813cb50 0x7fe948140bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:54:24.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.644+0000 7fe94ce5b700 1 --2- 172.21.15.162:0/2195230679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe948153990 0x7fe94814f9e0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 rx=0x7fe9481411e0 tx=0x7fe940007ee0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:24.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.644+0000 7fe9457fa700 1 -- 172.21.15.162:0/2195230679 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe94000a770 con 0x7fe948153990 2020-11-28T21:54:24.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.644+0000 7fe9457fa700 1 -- 172.21.15.162:0/2195230679 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe94000a8d0 con 0x7fe948153990 2020-11-28T21:54:24.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.644+0000 7fe94d65c700 1 -- 172.21.15.162:0/2195230679 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9481534f0 con 0x7fe948153990 2020-11-28T21:54:24.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.644+0000 7fe9457fa700 1 -- 172.21.15.162:0/2195230679 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe94000ac30 con 0x7fe948153990 2020-11-28T21:54:24.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.644+0000 7fe94d65c700 1 -- 172.21.15.162:0/2195230679 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9481431f0 con 0x7fe948153990 2020-11-28T21:54:24.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.648+0000 7fe94d65c700 1 -- 172.21.15.162:0/2195230679 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe948150070 con 0x7fe948153990 2020-11-28T21:54:24.655 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.648+0000 7fe9457fa700 1 -- 172.21.15.162:0/2195230679 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe940019030 con 0x7fe948153990 2020-11-28T21:54:24.655 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.648+0000 7fe9457fa700 1 --2- 172.21.15.162:0/2195230679 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe9280553b0 0x7fe928057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:24.655 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.648+0000 7fe9457fa700 1 -- 172.21.15.162:0/2195230679 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fe94006d670 con 0x7fe948153990 2020-11-28T21:54:24.655 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.648+0000 7fe947fff700 1 --2- 172.21.15.162:0/2195230679 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe9280553b0 0x7fe928057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:24.656 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.648+0000 7fe947fff700 1 --2- 172.21.15.162:0/2195230679 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe9280553b0 0x7fe928057870 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7fe93800b000 tx=0x7fe938006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:24.656 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.652+0000 7fe9457fa700 1 -- 172.21.15.162:0/2195230679 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe940043b10 con 0x7fe948153990 2020-11-28T21:54:24.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.792+0000 7fe94d65c700 1 -- 172.21.15.162:0/2195230679 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe948153680 con 0x7fe948153990 2020-11-28T21:54:24.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.792+0000 7fe9457fa700 1 -- 172.21.15.162:0/2195230679 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v105) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe940047130 con 0x7fe948153990 2020-11-28T21:54:24.795 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:54:24.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.792+0000 7fe94d65c700 1 -- 172.21.15.162:0/2195230679 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe9280553b0 msgr2=0x7fe928057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:24.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.792+0000 7fe94d65c700 1 --2- 172.21.15.162:0/2195230679 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe9280553b0 0x7fe928057870 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7fe93800b000 tx=0x7fe938006dc0).stop 2020-11-28T21:54:24.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.792+0000 7fe94d65c700 1 -- 172.21.15.162:0/2195230679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe948153990 msgr2=0x7fe94814f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:24.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.792+0000 7fe94d65c700 1 --2- 172.21.15.162:0/2195230679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe948153990 0x7fe94814f9e0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 rx=0x7fe9481411e0 tx=0x7fe940007ee0).stop 2020-11-28T21:54:24.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.792+0000 7fe94d65c700 1 -- 172.21.15.162:0/2195230679 shutdown_connections 2020-11-28T21:54:24.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.792+0000 7fe94d65c700 1 --2- 172.21.15.162:0/2195230679 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe9280553b0 0x7fe928057870 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:24.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.792+0000 7fe94d65c700 1 --2- 172.21.15.162:0/2195230679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe948153990 0x7fe94814f9e0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:24.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.792+0000 7fe94d65c700 1 --2- 172.21.15.162:0/2195230679 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe94805cca0 0x7fe948060400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:24.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.792+0000 7fe94d65c700 1 --2- 172.21.15.162:0/2195230679 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe94813cb50 0x7fe948140bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:24.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.792+0000 7fe94d65c700 1 -- 172.21.15.162:0/2195230679 >> 172.21.15.162:0/2195230679 conn(0x7fe9480b57d0 msgr2=0x7fe94813c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:24.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.792+0000 7fe94d65c700 1 -- 172.21.15.162:0/2195230679 shutdown_connections 2020-11-28T21:54:24.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.792+0000 7fe94d65c700 1 -- 172.21.15.162:0/2195230679 wait complete. 2020-11-28T21:54:24.808 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2020-11-28T21:54:24.916 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.912+0000 7fe49a43d700 1 Processor -- start 2020-11-28T21:54:24.916 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.912+0000 7fe49a43d700 1 -- start start 2020-11-28T21:54:24.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.916+0000 7fe49a43d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe4940b1870 0x7fe494142f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:24.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.916+0000 7fe49a43d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe494143440 0x7fe4940b0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:24.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.916+0000 7fe49a43d700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4940b1c50 con 0x7fe4940b4720 2020-11-28T21:54:24.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.916+0000 7fe49a43d700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4940615a0 con 0x7fe494143440 2020-11-28T21:54:24.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.916+0000 7fe49a43d700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4940afff0 con 0x7fe4940b1870 2020-11-28T21:54:24.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.916+0000 7fe49943b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe4940b1870 0x7fe494142f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:24.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.916+0000 7fe49943b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe4940b1870 0x7fe494142f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43306/0 (socket says 172.21.15.162:43306) 2020-11-28T21:54:24.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.916+0000 7fe49943b700 1 -- 172.21.15.162:0/300951570 learned_addr learned my addr 172.21.15.162:0/300951570 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:24.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.916+0000 7fe498c3a700 1 --2- 172.21.15.162:0/300951570 >> v2:172.21.15.118:3300/0 conn(0x7fe494143440 0x7fe4940b0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:24.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.916+0000 7fe49943b700 1 -- 172.21.15.162:0/300951570 >> v2:172.21.15.118:3300/0 conn(0x7fe494143440 msgr2=0x7fe4940b0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:24.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.916+0000 7fe49943b700 1 --2- 172.21.15.162:0/300951570 >> v2:172.21.15.118:3300/0 conn(0x7fe494143440 0x7fe4940b0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:24.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.916+0000 7fe49943b700 1 -- 172.21.15.162:0/300951570 >> v1:172.21.15.118:6789/0 conn(0x7fe4940b4720 legacy=0x7fe4940b0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:24.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.916+0000 7fe49943b700 1 -- 172.21.15.162:0/300951570 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe494060a60 con 0x7fe4940b1870 2020-11-28T21:54:24.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.916+0000 7fe49943b700 1 --2- 172.21.15.162:0/300951570 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4940b1870 0x7fe494142f50 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 rx=0x7fe49406ac20 tx=0x7fe490006e80).ready entity=mon.1 client_cookie=a97ff7a67aba84af server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:24.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.916+0000 7fe483fff700 1 -- 172.21.15.162:0/300951570 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe4900097b0 con 0x7fe4940b1870 2020-11-28T21:54:24.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.916+0000 7fe483fff700 1 -- 172.21.15.162:0/300951570 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe490009910 con 0x7fe4940b1870 2020-11-28T21:54:24.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.916+0000 7fe483fff700 1 -- 172.21.15.162:0/300951570 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe490009a80 con 0x7fe4940b1870 2020-11-28T21:54:24.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.916+0000 7fe49a43d700 1 -- 172.21.15.162:0/300951570 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4940b1870 msgr2=0x7fe494142f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:24.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.916+0000 7fe49a43d700 1 --2- 172.21.15.162:0/300951570 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4940b1870 0x7fe494142f50 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 rx=0x7fe49406ac20 tx=0x7fe490006e80).stop 2020-11-28T21:54:24.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.916+0000 7fe49a43d700 1 -- 172.21.15.162:0/300951570 shutdown_connections 2020-11-28T21:54:24.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.916+0000 7fe49a43d700 1 --2- 172.21.15.162:0/300951570 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4940b1870 0x7fe494142f50 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:24.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.916+0000 7fe49a43d700 1 --2- 172.21.15.162:0/300951570 >> v2:172.21.15.118:3300/0 conn(0x7fe494143440 0x7fe4940b0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:24.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.916+0000 7fe49a43d700 1 -- 172.21.15.162:0/300951570 >> 172.21.15.162:0/300951570 conn(0x7fe4940b54f0 msgr2=0x7fe4940b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:24.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.916+0000 7fe49a43d700 1 -- 172.21.15.162:0/300951570 shutdown_connections 2020-11-28T21:54:24.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.916+0000 7fe49a43d700 1 -- 172.21.15.162:0/300951570 wait complete. 2020-11-28T21:54:24.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.920+0000 7fe49a43d700 1 Processor -- start 2020-11-28T21:54:24.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.920+0000 7fe49a43d700 1 -- start start 2020-11-28T21:54:24.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.920+0000 7fe49a43d700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe4940b1870 0x7fe49413fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:24.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.920+0000 7fe49a43d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4940b4720 0x7fe494098620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:24.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.920+0000 7fe49a43d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe494143440 0x7fe494098b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:24.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.920+0000 7fe49a43d700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49413c490 con 0x7fe494143440 2020-11-28T21:54:24.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.920+0000 7fe49a43d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4940b1110 con 0x7fe4940b4720 2020-11-28T21:54:24.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.920+0000 7fe49a43d700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe49413b560 con 0x7fe4940b1870 2020-11-28T21:54:24.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.920+0000 7fe498c3a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4940b4720 0x7fe494098620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:24.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.920+0000 7fe498c3a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4940b4720 0x7fe494098620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43314/0 (socket says 172.21.15.162:43314) 2020-11-28T21:54:24.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.920+0000 7fe498c3a700 1 -- 172.21.15.162:0/1033625045 learned_addr learned my addr 172.21.15.162:0/1033625045 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:24.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.920+0000 7fe49943b700 1 --2- 172.21.15.162:0/1033625045 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe4940b1870 0x7fe49413fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:24.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.920+0000 7fe499c3c700 1 --2- 172.21.15.162:0/1033625045 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe494143440 0x7fe494098b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:24.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.924+0000 7fe498c3a700 1 -- 172.21.15.162:0/1033625045 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe4940b1870 msgr2=0x7fe49413fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:24.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.924+0000 7fe498c3a700 1 --2- 172.21.15.162:0/1033625045 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe4940b1870 0x7fe49413fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:24.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.924+0000 7fe498c3a700 1 -- 172.21.15.162:0/1033625045 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe494143440 msgr2=0x7fe494098b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:24.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.924+0000 7fe498c3a700 1 --2- 172.21.15.162:0/1033625045 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe494143440 0x7fe494098b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:24.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.924+0000 7fe498c3a700 1 -- 172.21.15.162:0/1033625045 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe49413af60 con 0x7fe4940b4720 2020-11-28T21:54:24.931 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.924+0000 7fe498c3a700 1 --2- 172.21.15.162:0/1033625045 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4940b4720 0x7fe494098620 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 rx=0x7fe49413c5d0 tx=0x7fe484008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:24.931 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.924+0000 7fe4827fc700 1 -- 172.21.15.162:0/1033625045 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe484011850 con 0x7fe4940b4720 2020-11-28T21:54:24.931 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.924+0000 7fe4827fc700 1 -- 172.21.15.162:0/1033625045 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4840119b0 con 0x7fe4940b4720 2020-11-28T21:54:24.931 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.924+0000 7fe4827fc700 1 -- 172.21.15.162:0/1033625045 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe484011d10 con 0x7fe4940b4720 2020-11-28T21:54:24.931 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.924+0000 7fe49a43d700 1 -- 172.21.15.162:0/1033625045 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe494145e70 con 0x7fe4940b4720 2020-11-28T21:54:24.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.924+0000 7fe49a43d700 1 -- 172.21.15.162:0/1033625045 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe49413bff0 con 0x7fe4940b4720 2020-11-28T21:54:24.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.924+0000 7fe49a43d700 1 -- 172.21.15.162:0/1033625045 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe49413bc30 con 0x7fe4940b4720 2020-11-28T21:54:24.933 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.924+0000 7fe4827fc700 1 -- 172.21.15.162:0/1033625045 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe48401c460 con 0x7fe4940b4720 2020-11-28T21:54:24.933 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.924+0000 7fe4827fc700 1 --2- 172.21.15.162:0/1033625045 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe474055330 0x7fe4740577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:24.933 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.924+0000 7fe4827fc700 1 -- 172.21.15.162:0/1033625045 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fe48406df00 con 0x7fe4940b4720 2020-11-28T21:54:24.933 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.924+0000 7fe49943b700 1 --2- 172.21.15.162:0/1033625045 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe474055330 0x7fe4740577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:24.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.924+0000 7fe49943b700 1 --2- 172.21.15.162:0/1033625045 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe474055330 0x7fe4740577f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7fe49001b000 tx=0x7fe490016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:24.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:24.932+0000 7fe4827fc700 1 -- 172.21.15.162:0/1033625045 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe4840444d0 con 0x7fe4940b4720 2020-11-28T21:54:25.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.072+0000 7fe49a43d700 1 -- 172.21.15.162:0/1033625045 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe494057050 con 0x7fe4940b4720 2020-11-28T21:54:25.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.072+0000 7fe4827fc700 1 -- 172.21.15.162:0/1033625045 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v105) v1 ==== 84+0+143 (secure 0 0 0) 0x7fe484047af0 con 0x7fe4940b4720 2020-11-28T21:54:25.077 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:54:25.077 INFO:teuthology.orchestra.run.smithi162.stdout:[ 2020-11-28T21:54:25.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:25.078 INFO:teuthology.orchestra.run.smithi162.stdout: "poolnum": 1, 2020-11-28T21:54:25.078 INFO:teuthology.orchestra.run.smithi162.stdout: "poolname": "device_health_metrics" 2020-11-28T21:54:25.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:25.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:25.078 INFO:teuthology.orchestra.run.smithi162.stdout: "poolnum": 2, 2020-11-28T21:54:25.079 INFO:teuthology.orchestra.run.smithi162.stdout: "poolname": "rbd" 2020-11-28T21:54:25.079 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:25.079 INFO:teuthology.orchestra.run.smithi162.stdout:] 2020-11-28T21:54:25.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.076+0000 7fe49a43d700 1 -- 172.21.15.162:0/1033625045 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe474055330 msgr2=0x7fe4740577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:25.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.076+0000 7fe49a43d700 1 --2- 172.21.15.162:0/1033625045 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe474055330 0x7fe4740577f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7fe49001b000 tx=0x7fe490016000).stop 2020-11-28T21:54:25.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.076+0000 7fe49a43d700 1 -- 172.21.15.162:0/1033625045 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4940b4720 msgr2=0x7fe494098620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:25.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.076+0000 7fe49a43d700 1 --2- 172.21.15.162:0/1033625045 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4940b4720 0x7fe494098620 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 rx=0x7fe49413c5d0 tx=0x7fe484008040).stop 2020-11-28T21:54:25.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.076+0000 7fe49a43d700 1 -- 172.21.15.162:0/1033625045 shutdown_connections 2020-11-28T21:54:25.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.076+0000 7fe49a43d700 1 --2- 172.21.15.162:0/1033625045 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe474055330 0x7fe4740577f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:25.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.076+0000 7fe49a43d700 1 --2- 172.21.15.162:0/1033625045 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe4940b1870 0x7fe49413fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:25.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.076+0000 7fe49a43d700 1 --2- 172.21.15.162:0/1033625045 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe494143440 0x7fe494098b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:25.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.076+0000 7fe49a43d700 1 --2- 172.21.15.162:0/1033625045 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4940b4720 0x7fe494098620 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:25.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.076+0000 7fe49a43d700 1 -- 172.21.15.162:0/1033625045 >> 172.21.15.162:0/1033625045 conn(0x7fe4940b54f0 msgr2=0x7fe4940b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:25.083 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.076+0000 7fe49a43d700 1 -- 172.21.15.162:0/1033625045 shutdown_connections 2020-11-28T21:54:25.083 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.076+0000 7fe49a43d700 1 -- 172.21.15.162:0/1033625045 wait complete. 2020-11-28T21:54:25.091 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-11-28T21:54:25.092 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2020-11-28T21:54:25.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.204+0000 7f4e848f0700 1 Processor -- start 2020-11-28T21:54:25.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.204+0000 7f4e848f0700 1 -- start start 2020-11-28T21:54:25.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.204+0000 7f4e848f0700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4e80098860 0x7f4e80098c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:25.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.204+0000 7f4e848f0700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f4e800610e0 0x7f4e80147aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:25.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.204+0000 7f4e848f0700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e800b0da0 con 0x7f4e8006ae10 2020-11-28T21:54:25.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.204+0000 7f4e848f0700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e80146ed0 con 0x7f4e800610e0 2020-11-28T21:54:25.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.204+0000 7f4e848f0700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e801470c0 con 0x7f4e80098860 2020-11-28T21:54:25.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.204+0000 7f4e7e59c700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f4e8006ae10 0x7f4e8014a420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:50166/0 (socket says 172.21.15.162:50166) 2020-11-28T21:54:25.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.204+0000 7f4e7e59c700 1 -- 172.21.15.162:0/1432739188 learned_addr learned my addr 172.21.15.162:0/1432739188 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:25.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.204+0000 7f4e7f59e700 1 --2- 172.21.15.162:0/1432739188 >> v2:172.21.15.118:3300/0 conn(0x7f4e800610e0 0x7f4e80147aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:25.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.204+0000 7f4e7ed9d700 1 --2- 172.21.15.162:0/1432739188 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4e80098860 0x7f4e80098c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:25.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.204+0000 7f4e7dd9b700 1 -- 172.21.15.162:0/1432739188 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3257942574 0 0) 0x7f4e800b0da0 con 0x7f4e8006ae10 2020-11-28T21:54:25.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.204+0000 7f4e7dd9b700 1 -- 172.21.15.162:0/1432739188 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e64003230 con 0x7f4e8006ae10 2020-11-28T21:54:25.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.204+0000 7f4e7f59e700 1 -- 172.21.15.162:0/1432739188 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4e80098860 msgr2=0x7f4e80098c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:25.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.204+0000 7f4e7f59e700 1 --2- 172.21.15.162:0/1432739188 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4e80098860 0x7f4e80098c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:25.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.204+0000 7f4e7f59e700 1 -- 172.21.15.162:0/1432739188 >> v1:172.21.15.162:6789/0 conn(0x7f4e8006ae10 legacy=0x7f4e8014a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:25.212 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.204+0000 7f4e7f59e700 1 -- 172.21.15.162:0/1432739188 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e80147360 con 0x7f4e800610e0 2020-11-28T21:54:25.212 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.204+0000 7f4e7f59e700 1 --2- 172.21.15.162:0/1432739188 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4e800610e0 0x7f4e80147aa0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f4e780032b0 tx=0x7f4e780091a0).ready entity=mon.0 client_cookie=1d1a394854e507f6 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:25.212 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.204+0000 7f4e7dd9b700 1 -- 172.21.15.162:0/1432739188 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e7800b940 con 0x7f4e800610e0 2020-11-28T21:54:25.212 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.204+0000 7f4e7dd9b700 1 -- 172.21.15.162:0/1432739188 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7800baa0 con 0x7f4e800610e0 2020-11-28T21:54:25.212 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.204+0000 7f4e848f0700 1 -- 172.21.15.162:0/1432739188 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4e800610e0 msgr2=0x7f4e80147aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:25.213 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.204+0000 7f4e7dd9b700 1 -- 172.21.15.162:0/1432739188 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e7800bc10 con 0x7f4e800610e0 2020-11-28T21:54:25.213 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.204+0000 7f4e848f0700 1 --2- 172.21.15.162:0/1432739188 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4e800610e0 0x7f4e80147aa0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f4e780032b0 tx=0x7f4e780091a0).stop 2020-11-28T21:54:25.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.208+0000 7f4e848f0700 1 -- 172.21.15.162:0/1432739188 shutdown_connections 2020-11-28T21:54:25.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.208+0000 7f4e848f0700 1 --2- 172.21.15.162:0/1432739188 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4e800610e0 0x7f4e80147aa0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:25.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.208+0000 7f4e848f0700 1 --2- 172.21.15.162:0/1432739188 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4e80098860 0x7f4e80098c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:25.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.208+0000 7f4e848f0700 1 -- 172.21.15.162:0/1432739188 >> 172.21.15.162:0/1432739188 conn(0x7f4e800b5520 msgr2=0x7f4e800b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:25.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.208+0000 7f4e848f0700 1 -- 172.21.15.162:0/1432739188 shutdown_connections 2020-11-28T21:54:25.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.208+0000 7f4e848f0700 1 -- 172.21.15.162:0/1432739188 wait complete. 2020-11-28T21:54:25.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.208+0000 7f4e848f0700 1 Processor -- start 2020-11-28T21:54:25.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.208+0000 7f4e848f0700 1 -- start start 2020-11-28T21:54:25.217 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.212+0000 7f4e848f0700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4e800610e0 0x7f4e80159120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:25.217 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.212+0000 7f4e848f0700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4e8006ae10 0x7f4e8013efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:25.217 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.212+0000 7f4e848f0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e80098860 0x7f4e8013f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:25.217 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.212+0000 7f4e848f0700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e800a8f40 con 0x7f4e8006ae10 2020-11-28T21:54:25.217 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.212+0000 7f4e848f0700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e80159610 con 0x7f4e80098860 2020-11-28T21:54:25.218 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.212+0000 7f4e848f0700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e80148810 con 0x7f4e800610e0 2020-11-28T21:54:25.218 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.212+0000 7f4e7f59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e80098860 0x7f4e8013f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:25.218 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.212+0000 7f4e7f59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e80098860 0x7f4e8013f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43328/0 (socket says 172.21.15.162:43328) 2020-11-28T21:54:25.218 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.212+0000 7f4e7f59e700 1 -- 172.21.15.162:0/2607339500 learned_addr learned my addr 172.21.15.162:0/2607339500 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:25.218 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.212+0000 7f4e7ed9d700 1 --2- 172.21.15.162:0/2607339500 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4e800610e0 0x7f4e80159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:25.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.212+0000 7f4e7e59c700 1 --2- 172.21.15.162:0/2607339500 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4e8006ae10 0x7f4e8013efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:25.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.212+0000 7f4e7f59e700 1 -- 172.21.15.162:0/2607339500 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4e800610e0 msgr2=0x7f4e80159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:25.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.212+0000 7f4e7f59e700 1 --2- 172.21.15.162:0/2607339500 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4e800610e0 0x7f4e80159120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:25.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.212+0000 7f4e7f59e700 1 -- 172.21.15.162:0/2607339500 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4e8006ae10 msgr2=0x7f4e8013efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:25.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.212+0000 7f4e7f59e700 1 --2- 172.21.15.162:0/2607339500 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4e8006ae10 0x7f4e8013efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:25.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.212+0000 7f4e7f59e700 1 -- 172.21.15.162:0/2607339500 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e800a8c80 con 0x7f4e80098860 2020-11-28T21:54:25.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.212+0000 7f4e7f59e700 1 --2- 172.21.15.162:0/2607339500 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e80098860 0x7f4e8013f4d0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 rx=0x7f4e780032b0 tx=0x7f4e78016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:25.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.212+0000 7f4e6ffff700 1 -- 172.21.15.162:0/2607339500 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e78011d90 con 0x7f4e80098860 2020-11-28T21:54:25.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.212+0000 7f4e6ffff700 1 -- 172.21.15.162:0/2607339500 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7801f3f0 con 0x7f4e80098860 2020-11-28T21:54:25.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.212+0000 7f4e848f0700 1 -- 172.21.15.162:0/2607339500 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e801463e0 con 0x7f4e80098860 2020-11-28T21:54:25.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.212+0000 7f4e6ffff700 1 -- 172.21.15.162:0/2607339500 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e7801f750 con 0x7f4e80098860 2020-11-28T21:54:25.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.212+0000 7f4e848f0700 1 -- 172.21.15.162:0/2607339500 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e800af5c0 con 0x7f4e80098860 2020-11-28T21:54:25.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.212+0000 7f4e848f0700 1 -- 172.21.15.162:0/2607339500 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e80148980 con 0x7f4e80098860 2020-11-28T21:54:25.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.216+0000 7f4e6ffff700 1 -- 172.21.15.162:0/2607339500 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4e78026030 con 0x7f4e80098860 2020-11-28T21:54:25.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.216+0000 7f4e6ffff700 1 --2- 172.21.15.162:0/2607339500 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4e5c055330 0x7f4e5c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:25.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.216+0000 7f4e6ffff700 1 -- 172.21.15.162:0/2607339500 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f4e7806e740 con 0x7f4e80098860 2020-11-28T21:54:25.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.216+0000 7f4e7ed9d700 1 --2- 172.21.15.162:0/2607339500 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4e5c055330 0x7f4e5c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:25.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.216+0000 7f4e7ed9d700 1 --2- 172.21.15.162:0/2607339500 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4e5c055330 0x7f4e5c0577f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7f4e7000b000 tx=0x7f4e70006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:25.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.220+0000 7f4e6ffff700 1 -- 172.21.15.162:0/2607339500 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4e78044d70 con 0x7f4e80098860 2020-11-28T21:54:25.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.360+0000 7f4e848f0700 1 -- 172.21.15.162:0/2607339500 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f4e80057050 con 0x7f4e80098860 2020-11-28T21:54:25.557 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.552+0000 7f4e6ffff700 1 -- 172.21.15.162:0/2607339500 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v106) v1 ==== 133+0+0 (secure 0 0 0) 0x7f4e780240a0 con 0x7f4e80098860 2020-11-28T21:54:25.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.560+0000 7f4e848f0700 1 -- 172.21.15.162:0/2607339500 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4e5c055330 msgr2=0x7f4e5c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:25.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.560+0000 7f4e848f0700 1 --2- 172.21.15.162:0/2607339500 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4e5c055330 0x7f4e5c0577f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7f4e7000b000 tx=0x7f4e70006dc0).stop 2020-11-28T21:54:25.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.560+0000 7f4e848f0700 1 -- 172.21.15.162:0/2607339500 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e80098860 msgr2=0x7f4e8013f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:25.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.560+0000 7f4e848f0700 1 --2- 172.21.15.162:0/2607339500 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e80098860 0x7f4e8013f4d0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 rx=0x7f4e780032b0 tx=0x7f4e78016000).stop 2020-11-28T21:54:25.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.560+0000 7f4e848f0700 1 -- 172.21.15.162:0/2607339500 shutdown_connections 2020-11-28T21:54:25.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.560+0000 7f4e848f0700 1 --2- 172.21.15.162:0/2607339500 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4e5c055330 0x7f4e5c0577f0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:25.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.560+0000 7f4e848f0700 1 --2- 172.21.15.162:0/2607339500 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4e800610e0 0x7f4e80159120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:25.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.560+0000 7f4e848f0700 1 --2- 172.21.15.162:0/2607339500 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e80098860 0x7f4e8013f4d0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:25.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.560+0000 7f4e848f0700 1 --2- 172.21.15.162:0/2607339500 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4e8006ae10 0x7f4e8013efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:25.566 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.560+0000 7f4e848f0700 1 -- 172.21.15.162:0/2607339500 >> 172.21.15.162:0/2607339500 conn(0x7f4e800b5520 msgr2=0x7f4e8005b970 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:25.566 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.560+0000 7f4e848f0700 1 -- 172.21.15.162:0/2607339500 shutdown_connections 2020-11-28T21:54:25.567 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.560+0000 7f4e848f0700 1 -- 172.21.15.162:0/2607339500 wait complete. 2020-11-28T21:54:25.567 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_metadata' created 2020-11-28T21:54:25.578 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2020-11-28T21:54:25.693 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.688+0000 7f9b2cb73700 1 Processor -- start 2020-11-28T21:54:25.693 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.688+0000 7f9b2cb73700 1 -- start start 2020-11-28T21:54:25.694 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b2cb73700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f9b280afec0 0x7f9b280b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:25.695 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b2cb73700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b28060500 con 0x7f9b28099d30 2020-11-28T21:54:25.695 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b2cb73700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b280607a0 con 0x7f9b280a8df0 2020-11-28T21:54:25.695 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b2cb73700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b28060ce0 con 0x7f9b280afec0 2020-11-28T21:54:25.695 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b26ffd700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f9b280a8df0 0x7f9b28099620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:50180/0 (socket says 172.21.15.162:50180) 2020-11-28T21:54:25.696 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b26ffd700 1 -- 172.21.15.162:0/366797924 learned_addr learned my addr 172.21.15.162:0/366797924 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:25.696 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b277fe700 1 --2- 172.21.15.162:0/366797924 >> v2:172.21.15.118:3300/0 conn(0x7f9b280afec0 0x7f9b280b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:25.699 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b267fc700 1 -- 172.21.15.162:0/366797924 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2943520639 0 0) 0x7f9b280607a0 con 0x7f9b280a8df0 2020-11-28T21:54:25.699 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b267fc700 1 -- 172.21.15.162:0/366797924 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b14003230 con 0x7f9b280a8df0 2020-11-28T21:54:25.699 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b267fc700 1 -- 172.21.15.162:0/366797924 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1291719273 0 0) 0x7f9b14003230 con 0x7f9b280a8df0 2020-11-28T21:54:25.699 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b267fc700 1 -- 172.21.15.162:0/366797924 >> v2:172.21.15.118:3300/0 conn(0x7f9b280afec0 msgr2=0x7f9b280b02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:25.700 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b267fc700 1 --2- 172.21.15.162:0/366797924 >> v2:172.21.15.118:3300/0 conn(0x7f9b280afec0 0x7f9b280b02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:25.700 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b267fc700 1 -- 172.21.15.162:0/366797924 >> v1:172.21.15.118:6789/0 conn(0x7f9b28099d30 legacy=0x7f9b28148d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:25.700 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b267fc700 1 -- 172.21.15.162:0/366797924 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b28060a40 con 0x7f9b280a8df0 2020-11-28T21:54:25.700 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b267fc700 1 -- 172.21.15.162:0/366797924 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f9b1c0016d0 con 0x7f9b280a8df0 2020-11-28T21:54:25.701 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b267fc700 1 -- 172.21.15.162:0/366797924 >> v1:172.21.15.162:6789/0 conn(0x7f9b280a8df0 legacy=0x7f9b28099620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:25.701 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b267fc700 1 --2- 172.21.15.162:0/366797924 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b14003720 0x7f9b14003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:25.701 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b267fc700 1 --2- 172.21.15.162:0/366797924 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b140044d0 0x7f9b14004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:25.701 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b267fc700 1 --2- 172.21.15.162:0/366797924 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9b14004e40 0x7f9b1400b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:25.701 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b267fc700 1 -- 172.21.15.162:0/366797924 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b28060ce0 con 0x7f9b14003720 2020-11-28T21:54:25.702 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b267fc700 1 -- 172.21.15.162:0/366797924 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b14004250 con 0x7f9b140044d0 2020-11-28T21:54:25.702 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b267fc700 1 -- 172.21.15.162:0/366797924 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1400bc10 con 0x7f9b14004e40 2020-11-28T21:54:25.702 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b26ffd700 1 --2- 172.21.15.162:0/366797924 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b140044d0 0x7f9b14004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:25.702 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b277fe700 1 --2- 172.21.15.162:0/366797924 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b14003720 0x7f9b14003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:25.702 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b26ffd700 1 -- 172.21.15.162:0/366797924 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9b14004e40 msgr2=0x7f9b1400b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:25.703 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b27fff700 1 --2- 172.21.15.162:0/366797924 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9b14004e40 0x7f9b1400b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:25.703 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b26ffd700 1 --2- 172.21.15.162:0/366797924 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9b14004e40 0x7f9b1400b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:25.703 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b26ffd700 1 -- 172.21.15.162:0/366797924 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b14003720 msgr2=0x7f9b14003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:25.703 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b26ffd700 1 --2- 172.21.15.162:0/366797924 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b14003720 0x7f9b14003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:25.704 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b26ffd700 1 -- 172.21.15.162:0/366797924 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b14003230 con 0x7f9b140044d0 2020-11-28T21:54:25.704 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b277fe700 1 --2- 172.21.15.162:0/366797924 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b14003720 0x7f9b14003b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:54:25.704 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b27fff700 1 --2- 172.21.15.162:0/366797924 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9b14004e40 0x7f9b1400b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:54:25.704 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b27fff700 1 -- 172.21.15.162:0/366797924 reap_dead start 2020-11-28T21:54:25.704 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b26ffd700 1 --2- 172.21.15.162:0/366797924 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b140044d0 0x7f9b14004950 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 rx=0x7f9b1c0013e0 tx=0x7f9b28060690).ready entity=mon.1 client_cookie=6a6cc7e66dbe718a server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:25.705 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b267fc700 1 -- 172.21.15.162:0/366797924 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b1c00ce00 con 0x7f9b140044d0 2020-11-28T21:54:25.705 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b267fc700 1 -- 172.21.15.162:0/366797924 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b1c0072e0 con 0x7f9b140044d0 2020-11-28T21:54:25.705 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b267fc700 1 -- 172.21.15.162:0/366797924 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b1c0057d0 con 0x7f9b140044d0 2020-11-28T21:54:25.705 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b2cb73700 1 -- 172.21.15.162:0/366797924 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b140044d0 msgr2=0x7f9b14004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:25.705 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b2cb73700 1 --2- 172.21.15.162:0/366797924 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b140044d0 0x7f9b14004950 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 rx=0x7f9b1c0013e0 tx=0x7f9b28060690).stop 2020-11-28T21:54:25.706 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b2cb73700 1 -- 172.21.15.162:0/366797924 shutdown_connections 2020-11-28T21:54:25.706 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b2cb73700 1 --2- 172.21.15.162:0/366797924 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b140044d0 0x7f9b14004950 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:25.706 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b2cb73700 1 -- 172.21.15.162:0/366797924 >> 172.21.15.162:0/366797924 conn(0x7f9b280b5520 msgr2=0x7f9b280b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:25.706 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b2cb73700 1 -- 172.21.15.162:0/366797924 shutdown_connections 2020-11-28T21:54:25.706 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.692+0000 7f9b2cb73700 1 -- 172.21.15.162:0/366797924 wait complete. 2020-11-28T21:54:25.707 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.696+0000 7f9b2cb73700 1 Processor -- start 2020-11-28T21:54:25.707 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.696+0000 7f9b2cb73700 1 -- start start 2020-11-28T21:54:25.709 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.696+0000 7f9b2cb73700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9b140044d0 0x7f9b280adbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:25.709 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.696+0000 7f9b2cb73700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b28097960 0x7f9b2814d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:25.709 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.696+0000 7f9b2cb73700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b280ae3c0 0x7f9b28096d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:25.710 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.696+0000 7f9b2cb73700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2814e300 con 0x7f9b280ae3c0 2020-11-28T21:54:25.710 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.696+0000 7f9b2cb73700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b280af970 con 0x7f9b28097960 2020-11-28T21:54:25.710 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.696+0000 7f9b2cb73700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b280af1d0 con 0x7f9b140044d0 2020-11-28T21:54:25.710 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.696+0000 7f9b26ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b28097960 0x7f9b2814d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:25.710 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.696+0000 7f9b277fe700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9b140044d0 0x7f9b280adbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:25.711 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.696+0000 7f9b26ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b28097960 0x7f9b2814d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43348/0 (socket says 172.21.15.162:43348) 2020-11-28T21:54:25.711 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.696+0000 7f9b26ffd700 1 -- 172.21.15.162:0/3793449776 learned_addr learned my addr 172.21.15.162:0/3793449776 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:25.711 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.696+0000 7f9b27fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b280ae3c0 0x7f9b28096d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:25.711 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.696+0000 7f9b26ffd700 1 -- 172.21.15.162:0/3793449776 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9b140044d0 msgr2=0x7f9b280adbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:25.711 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.696+0000 7f9b26ffd700 1 --2- 172.21.15.162:0/3793449776 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9b140044d0 0x7f9b280adbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:25.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.696+0000 7f9b26ffd700 1 -- 172.21.15.162:0/3793449776 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b280ae3c0 msgr2=0x7f9b28096d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:25.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.696+0000 7f9b26ffd700 1 --2- 172.21.15.162:0/3793449776 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b280ae3c0 0x7f9b28096d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:25.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.696+0000 7f9b26ffd700 1 -- 172.21.15.162:0/3793449776 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b280afba0 con 0x7f9b28097960 2020-11-28T21:54:25.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.696+0000 7f9b26ffd700 1 --2- 172.21.15.162:0/3793449776 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b28097960 0x7f9b2814d350 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 rx=0x7f9b2814e440 tx=0x7f9b1c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:25.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.700+0000 7f9b24ff9700 1 -- 172.21.15.162:0/3793449776 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b1c004950 con 0x7f9b28097960 2020-11-28T21:54:25.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.700+0000 7f9b24ff9700 1 -- 172.21.15.162:0/3793449776 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b1c004ab0 con 0x7f9b28097960 2020-11-28T21:54:25.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.700+0000 7f9b2cb73700 1 -- 172.21.15.162:0/3793449776 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b280af430 con 0x7f9b28097960 2020-11-28T21:54:25.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.700+0000 7f9b24ff9700 1 -- 172.21.15.162:0/3793449776 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b1c004e10 con 0x7f9b28097960 2020-11-28T21:54:25.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.700+0000 7f9b2cb73700 1 -- 172.21.15.162:0/3793449776 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b280ae840 con 0x7f9b28097960 2020-11-28T21:54:25.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.700+0000 7f9b2cb73700 1 -- 172.21.15.162:0/3793449776 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b280af5c0 con 0x7f9b28097960 2020-11-28T21:54:25.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.700+0000 7f9b24ff9700 1 -- 172.21.15.162:0/3793449776 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9b1c026070 con 0x7f9b28097960 2020-11-28T21:54:25.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.700+0000 7f9b24ff9700 1 --2- 172.21.15.162:0/3793449776 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9b080553f0 0x7f9b080578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:25.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.700+0000 7f9b24ff9700 1 -- 172.21.15.162:0/3793449776 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f9b1c06e430 con 0x7f9b28097960 2020-11-28T21:54:25.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.700+0000 7f9b277fe700 1 --2- 172.21.15.162:0/3793449776 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9b080553f0 0x7f9b080578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:25.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.700+0000 7f9b277fe700 1 --2- 172.21.15.162:0/3793449776 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9b080553f0 0x7f9b080578b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f9b18009000 tx=0x7f9b18006e50).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:25.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.708+0000 7f9b24ff9700 1 -- 172.21.15.162:0/3793449776 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9b1c044900 con 0x7f9b28097960 2020-11-28T21:54:25.853 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:25.848+0000 7f9b2cb73700 1 -- 172.21.15.162:0/3793449776 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f9b28057050 con 0x7f9b28097960 2020-11-28T21:54:26.567 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.564+0000 7f9b24ff9700 1 -- 172.21.15.162:0/3793449776 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v107) v1 ==== 125+0+0 (secure 0 0 0) 0x7f9b1c047f20 con 0x7f9b28097960 2020-11-28T21:54:26.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.568+0000 7f9b2cb73700 1 -- 172.21.15.162:0/3793449776 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9b080553f0 msgr2=0x7f9b080578b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:26.574 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.568+0000 7f9b2cb73700 1 --2- 172.21.15.162:0/3793449776 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9b080553f0 0x7f9b080578b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f9b18009000 tx=0x7f9b18006e50).stop 2020-11-28T21:54:26.574 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.568+0000 7f9b2cb73700 1 -- 172.21.15.162:0/3793449776 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b28097960 msgr2=0x7f9b2814d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:26.574 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.568+0000 7f9b2cb73700 1 --2- 172.21.15.162:0/3793449776 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b28097960 0x7f9b2814d350 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 rx=0x7f9b2814e440 tx=0x7f9b1c015000).stop 2020-11-28T21:54:26.574 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.568+0000 7f9b2cb73700 1 -- 172.21.15.162:0/3793449776 shutdown_connections 2020-11-28T21:54:26.575 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.568+0000 7f9b2cb73700 1 --2- 172.21.15.162:0/3793449776 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9b080553f0 0x7f9b080578b0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:26.575 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.568+0000 7f9b2cb73700 1 --2- 172.21.15.162:0/3793449776 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9b140044d0 0x7f9b280adbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:26.575 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.568+0000 7f9b2cb73700 1 --2- 172.21.15.162:0/3793449776 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b280ae3c0 0x7f9b28096d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:26.575 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.568+0000 7f9b2cb73700 1 --2- 172.21.15.162:0/3793449776 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b28097960 0x7f9b2814d350 secure :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 rx=0x7f9b2814e440 tx=0x7f9b1c015000).stop 2020-11-28T21:54:26.575 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.568+0000 7f9b2cb73700 1 -- 172.21.15.162:0/3793449776 >> 172.21.15.162:0/3793449776 conn(0x7f9b280b5520 msgr2=0x7f9b28066180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:26.576 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.572+0000 7f9b2cb73700 1 -- 172.21.15.162:0/3793449776 shutdown_connections 2020-11-28T21:54:26.576 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.572+0000 7f9b2cb73700 1 -- 172.21.15.162:0/3793449776 wait complete. 2020-11-28T21:54:26.576 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_data' created 2020-11-28T21:54:26.590 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2020-11-28T21:54:26.702 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.700+0000 7f0d2b2aa700 1 Processor -- start 2020-11-28T21:54:26.703 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.700+0000 7f0d2b2aa700 1 -- start start 2020-11-28T21:54:26.704 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.700+0000 7f0d2b2aa700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0d240b2d00 0x7f0d2405c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:26.704 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.700+0000 7f0d2b2aa700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0d240b30e0 0x7f0d2413fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:26.705 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.700+0000 7f0d2b2aa700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d2405baf0 con 0x7f0d240b3ab0 2020-11-28T21:54:26.705 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.700+0000 7f0d2b2aa700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d240b0e20 con 0x7f0d240b30e0 2020-11-28T21:54:26.705 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.700+0000 7f0d2b2aa700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2405bd90 con 0x7f0d240b2d00 2020-11-28T21:54:26.705 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.700+0000 7f0d2aaa9700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0d240b30e0 0x7f0d2413fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:26.705 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.700+0000 7f0d29aa7700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f0d240b3ab0 0x7f0d24142ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:50200/0 (socket says 172.21.15.162:50200) 2020-11-28T21:54:26.706 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.700+0000 7f0d29aa7700 1 -- 172.21.15.162:0/2816765014 learned_addr learned my addr 172.21.15.162:0/2816765014 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:26.706 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.700+0000 7f0d2a2a8700 1 --2- 172.21.15.162:0/2816765014 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0d240b2d00 0x7f0d2405c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:26.707 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.700+0000 7f0d2aaa9700 1 -- 172.21.15.162:0/2816765014 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0d240b2d00 msgr2=0x7f0d2405c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:26.708 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.700+0000 7f0d2aaa9700 1 --2- 172.21.15.162:0/2816765014 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0d240b2d00 0x7f0d2405c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:26.708 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.700+0000 7f0d2aaa9700 1 -- 172.21.15.162:0/2816765014 >> v1:172.21.15.162:6789/0 conn(0x7f0d240b3ab0 legacy=0x7f0d24142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:26.708 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.700+0000 7f0d292a6700 1 -- 172.21.15.162:0/2816765014 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1982858725 0 0) 0x7f0d2405baf0 con 0x7f0d240b3ab0 2020-11-28T21:54:26.708 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.700+0000 7f0d2aaa9700 1 -- 172.21.15.162:0/2816765014 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d2405c030 con 0x7f0d240b30e0 2020-11-28T21:54:26.708 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.700+0000 7f0d2a2a8700 1 --2- 172.21.15.162:0/2816765014 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0d240b2d00 0x7f0d2405c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:54:26.709 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.700+0000 7f0d2aaa9700 1 --2- 172.21.15.162:0/2816765014 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d240b30e0 0x7f0d2413fa70 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 rx=0x7f0d1c0032b0 tx=0x7f0d1c0091a0).ready entity=mon.1 client_cookie=480cc9d4defb656d server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:26.709 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.700+0000 7f0d292a6700 1 -- 172.21.15.162:0/2816765014 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d1c00ba50 con 0x7f0d240b30e0 2020-11-28T21:54:26.709 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.700+0000 7f0d292a6700 1 -- 172.21.15.162:0/2816765014 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d1c00bbb0 con 0x7f0d240b30e0 2020-11-28T21:54:26.709 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.700+0000 7f0d292a6700 1 -- 172.21.15.162:0/2816765014 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d1c00bd20 con 0x7f0d240b30e0 2020-11-28T21:54:26.710 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.700+0000 7f0d2b2aa700 1 -- 172.21.15.162:0/2816765014 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d240b30e0 msgr2=0x7f0d2413fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:26.710 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.700+0000 7f0d2b2aa700 1 --2- 172.21.15.162:0/2816765014 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d240b30e0 0x7f0d2413fa70 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 rx=0x7f0d1c0032b0 tx=0x7f0d1c0091a0).stop 2020-11-28T21:54:26.710 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.704+0000 7f0d2b2aa700 1 -- 172.21.15.162:0/2816765014 shutdown_connections 2020-11-28T21:54:26.710 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.704+0000 7f0d2b2aa700 1 --2- 172.21.15.162:0/2816765014 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d240b30e0 0x7f0d2413fa70 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:26.710 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.704+0000 7f0d2b2aa700 1 --2- 172.21.15.162:0/2816765014 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0d240b2d00 0x7f0d2405c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:26.711 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.704+0000 7f0d2b2aa700 1 -- 172.21.15.162:0/2816765014 >> 172.21.15.162:0/2816765014 conn(0x7f0d240604c0 msgr2=0x7f0d240608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:26.711 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.704+0000 7f0d2b2aa700 1 -- 172.21.15.162:0/2816765014 shutdown_connections 2020-11-28T21:54:26.711 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.704+0000 7f0d2b2aa700 1 -- 172.21.15.162:0/2816765014 wait complete. 2020-11-28T21:54:26.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.704+0000 7f0d2b2aa700 1 Processor -- start 2020-11-28T21:54:26.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.704+0000 7f0d2b2aa700 1 -- start start 2020-11-28T21:54:26.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.708+0000 7f0d2b2aa700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d240b2d00 0x7f0d2413f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:26.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.708+0000 7f0d2b2aa700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0d240b30e0 0x7f0d241427e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:26.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.708+0000 7f0d2b2aa700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0d240b3ab0 0x7f0d240b5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:26.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.708+0000 7f0d2b2aa700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d240b40b0 con 0x7f0d240b30e0 2020-11-28T21:54:26.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.708+0000 7f0d2b2aa700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d24142cd0 con 0x7f0d240b2d00 2020-11-28T21:54:26.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.708+0000 7f0d2b2aa700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d240b42e0 con 0x7f0d240b3ab0 2020-11-28T21:54:26.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.708+0000 7f0d2a2a8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d240b2d00 0x7f0d2413f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:26.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.708+0000 7f0d2a2a8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d240b2d00 0x7f0d2413f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43360/0 (socket says 172.21.15.162:43360) 2020-11-28T21:54:26.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.708+0000 7f0d2a2a8700 1 -- 172.21.15.162:0/3997502436 learned_addr learned my addr 172.21.15.162:0/3997502436 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:26.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.708+0000 7f0d2aaa9700 1 --2- 172.21.15.162:0/3997502436 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0d240b3ab0 0x7f0d240b5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:26.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.708+0000 7f0d29aa7700 1 --2- 172.21.15.162:0/3997502436 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0d240b30e0 0x7f0d241427e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:26.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.708+0000 7f0d2a2a8700 1 -- 172.21.15.162:0/3997502436 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0d240b3ab0 msgr2=0x7f0d240b5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:26.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.708+0000 7f0d2a2a8700 1 --2- 172.21.15.162:0/3997502436 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0d240b3ab0 0x7f0d240b5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:26.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.708+0000 7f0d2a2a8700 1 -- 172.21.15.162:0/3997502436 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0d240b30e0 msgr2=0x7f0d241427e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:26.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.708+0000 7f0d2a2a8700 1 --2- 172.21.15.162:0/3997502436 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0d240b30e0 0x7f0d241427e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:26.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.708+0000 7f0d2a2a8700 1 -- 172.21.15.162:0/3997502436 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d240b4580 con 0x7f0d240b2d00 2020-11-28T21:54:26.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.708+0000 7f0d2a2a8700 1 --2- 172.21.15.162:0/3997502436 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d240b2d00 0x7f0d2413f080 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 rx=0x7f0d20006910 tx=0x7f0d20008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:26.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.708+0000 7f0d137fe700 1 -- 172.21.15.162:0/3997502436 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d20009980 con 0x7f0d240b2d00 2020-11-28T21:54:26.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.708+0000 7f0d137fe700 1 -- 172.21.15.162:0/3997502436 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d20009ae0 con 0x7f0d240b2d00 2020-11-28T21:54:26.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.708+0000 7f0d137fe700 1 -- 172.21.15.162:0/3997502436 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d20009e40 con 0x7f0d240b2d00 2020-11-28T21:54:26.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.708+0000 7f0d2b2aa700 1 -- 172.21.15.162:0/3997502436 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d24142340 con 0x7f0d240b2d00 2020-11-28T21:54:26.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.708+0000 7f0d2b2aa700 1 -- 172.21.15.162:0/3997502436 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d24140a90 con 0x7f0d240b2d00 2020-11-28T21:54:26.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.708+0000 7f0d2b2aa700 1 -- 172.21.15.162:0/3997502436 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d24140da0 con 0x7f0d240b2d00 2020-11-28T21:54:26.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.712+0000 7f0d137fe700 1 -- 172.21.15.162:0/3997502436 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0d20013480 con 0x7f0d240b2d00 2020-11-28T21:54:26.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.712+0000 7f0d137fe700 1 --2- 172.21.15.162:0/3997502436 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0d040554e0 0x7f0d040579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:26.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.712+0000 7f0d137fe700 1 -- 172.21.15.162:0/3997502436 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5957+0+0 (secure 0 0 0) 0x7f0d2006e580 con 0x7f0d240b2d00 2020-11-28T21:54:26.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.712+0000 7f0d29aa7700 1 --2- 172.21.15.162:0/3997502436 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0d040554e0 0x7f0d040579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:26.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.712+0000 7f0d29aa7700 1 --2- 172.21.15.162:0/3997502436 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0d040554e0 0x7f0d040579a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7f0d1400d000 tx=0x7f0d14007bd0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:26.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.716+0000 7f0d137fe700 1 -- 172.21.15.162:0/3997502436 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0d20044890 con 0x7f0d240b2d00 2020-11-28T21:54:26.906 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.904+0000 7f0d2b2aa700 1 -- 172.21.15.162:0/3997502436 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f0d240b5040 con 0x7f0d240b2d00 2020-11-28T21:54:26.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.916+0000 7f0d137fe700 1 -- 172.21.15.162:0/3997502436 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 19 and data pool 20 v50) v1 ==== 189+0+0 (secure 0 0 0) 0x7f0d240b5040 con 0x7f0d240b2d00 2020-11-28T21:54:26.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.920+0000 7f0d2b2aa700 1 -- 172.21.15.162:0/3997502436 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0d040554e0 msgr2=0x7f0d040579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:26.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.920+0000 7f0d2b2aa700 1 --2- 172.21.15.162:0/3997502436 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0d040554e0 0x7f0d040579a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7f0d1400d000 tx=0x7f0d14007bd0).stop 2020-11-28T21:54:26.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.920+0000 7f0d2b2aa700 1 -- 172.21.15.162:0/3997502436 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d240b2d00 msgr2=0x7f0d2413f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:26.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.920+0000 7f0d2b2aa700 1 --2- 172.21.15.162:0/3997502436 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d240b2d00 0x7f0d2413f080 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 rx=0x7f0d20006910 tx=0x7f0d20008040).stop 2020-11-28T21:54:26.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.924+0000 7f0d2b2aa700 1 -- 172.21.15.162:0/3997502436 shutdown_connections 2020-11-28T21:54:26.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.924+0000 7f0d2b2aa700 1 --2- 172.21.15.162:0/3997502436 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0d040554e0 0x7f0d040579a0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:26.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.924+0000 7f0d2b2aa700 1 --2- 172.21.15.162:0/3997502436 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0d240b3ab0 0x7f0d240b5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:26.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.924+0000 7f0d2b2aa700 1 --2- 172.21.15.162:0/3997502436 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d240b2d00 0x7f0d2413f080 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:26.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.924+0000 7f0d2b2aa700 1 --2- 172.21.15.162:0/3997502436 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0d240b30e0 0x7f0d241427e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:26.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.924+0000 7f0d2b2aa700 1 -- 172.21.15.162:0/3997502436 >> 172.21.15.162:0/3997502436 conn(0x7f0d240604c0 msgr2=0x7f0d24140090 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:26.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.924+0000 7f0d2b2aa700 1 -- 172.21.15.162:0/3997502436 shutdown_connections 2020-11-28T21:54:26.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:26.924+0000 7f0d2b2aa700 1 -- 172.21.15.162:0/3997502436 wait complete. 2020-11-28T21:54:26.929 INFO:teuthology.orchestra.run.smithi162.stderr:new fs with metadata pool 19 and data pool 20 2020-11-28T21:54:26.942 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:54:27.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.040+0000 7f79e175a700 1 Processor -- start 2020-11-28T21:54:27.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.040+0000 7f79e175a700 1 -- start start 2020-11-28T21:54:27.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.040+0000 7f79e175a700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f79dc0b1990 0x7f79dc140890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:27.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.040+0000 7f79e175a700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f79dc0b11b0 0x7f79dc140d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:27.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.040+0000 7f79e175a700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f79dc0b1d70 0x7f79dc14a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:27.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.040+0000 7f79e175a700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79dc146e20 con 0x7f79dc0b11b0 2020-11-28T21:54:27.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.040+0000 7f79e175a700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79dc0b2870 con 0x7f79dc0b1d70 2020-11-28T21:54:27.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.040+0000 7f79e175a700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc0b22e0 con 0x7f79dc0b1990 2020-11-28T21:54:27.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.040+0000 7f79db7fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f79dc0b1d70 0x7f79dc14a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:27.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.040+0000 7f79db7fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f79dc0b1d70 0x7f79dc14a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43386/0 (socket says 172.21.15.162:43386) 2020-11-28T21:54:27.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.040+0000 7f79db7fe700 1 -- 172.21.15.162:0/1081838544 learned_addr learned my addr 172.21.15.162:0/1081838544 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:27.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.040+0000 7f79daffd700 1 --2- 172.21.15.162:0/1081838544 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f79dc0b1990 0x7f79dc140890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:27.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.040+0000 7f79da7fc700 1 --2- 172.21.15.162:0/1081838544 >> v2:172.21.15.118:3300/0 conn(0x7f79dc0b11b0 0x7f79dc140d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:27.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.040+0000 7f79db7fe700 1 -- 172.21.15.162:0/1081838544 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f79dc0b1990 msgr2=0x7f79dc140890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:27.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.040+0000 7f79db7fe700 1 --2- 172.21.15.162:0/1081838544 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f79dc0b1990 0x7f79dc140890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:27.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.040+0000 7f79db7fe700 1 -- 172.21.15.162:0/1081838544 >> v2:172.21.15.118:3300/0 conn(0x7f79dc0b11b0 msgr2=0x7f79dc140d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:27.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.040+0000 7f79db7fe700 1 --2- 172.21.15.162:0/1081838544 >> v2:172.21.15.118:3300/0 conn(0x7f79dc0b11b0 0x7f79dc140d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:27.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.040+0000 7f79db7fe700 1 -- 172.21.15.162:0/1081838544 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79dc0b0740 con 0x7f79dc0b1d70 2020-11-28T21:54:27.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.040+0000 7f79daffd700 1 --2- 172.21.15.162:0/1081838544 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f79dc0b1990 0x7f79dc140890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:54:27.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.040+0000 7f79da7fc700 1 --2- 172.21.15.162:0/1081838544 >> v2:172.21.15.118:3300/0 conn(0x7f79dc0b11b0 0x7f79dc140d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:54:27.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.040+0000 7f79db7fe700 1 --2- 172.21.15.162:0/1081838544 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79dc0b1d70 0x7f79dc14a770 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 rx=0x7f79cc002f30 tx=0x7f79cc006e80).ready entity=mon.1 client_cookie=7209050b94331bb server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:27.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.040+0000 7f79d9ffb700 1 -- 172.21.15.162:0/1081838544 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79cc0097f0 con 0x7f79dc0b1d70 2020-11-28T21:54:27.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.040+0000 7f79d9ffb700 1 -- 172.21.15.162:0/1081838544 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79cc009950 con 0x7f79dc0b1d70 2020-11-28T21:54:27.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.040+0000 7f79d9ffb700 1 -- 172.21.15.162:0/1081838544 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79cc009cb0 con 0x7f79dc0b1d70 2020-11-28T21:54:27.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.040+0000 7f79e175a700 1 -- 172.21.15.162:0/1081838544 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79dc0b1d70 msgr2=0x7f79dc14a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:27.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.040+0000 7f79e175a700 1 --2- 172.21.15.162:0/1081838544 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79dc0b1d70 0x7f79dc14a770 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 rx=0x7f79cc002f30 tx=0x7f79cc006e80).stop 2020-11-28T21:54:27.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.040+0000 7f79e175a700 1 -- 172.21.15.162:0/1081838544 shutdown_connections 2020-11-28T21:54:27.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.040+0000 7f79e175a700 1 --2- 172.21.15.162:0/1081838544 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79dc0b1d70 0x7f79dc14a770 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:27.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.040+0000 7f79e175a700 1 --2- 172.21.15.162:0/1081838544 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f79dc0b1990 0x7f79dc140890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:27.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.040+0000 7f79e175a700 1 --2- 172.21.15.162:0/1081838544 >> v2:172.21.15.118:3300/0 conn(0x7f79dc0b11b0 0x7f79dc140d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:27.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.040+0000 7f79e175a700 1 -- 172.21.15.162:0/1081838544 >> 172.21.15.162:0/1081838544 conn(0x7f79dc05b970 msgr2=0x7f79dc05bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:27.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.040+0000 7f79e175a700 1 -- 172.21.15.162:0/1081838544 shutdown_connections 2020-11-28T21:54:27.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.040+0000 7f79e175a700 1 -- 172.21.15.162:0/1081838544 wait complete. 2020-11-28T21:54:27.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.044+0000 7f79e175a700 1 Processor -- start 2020-11-28T21:54:27.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.044+0000 7f79e175a700 1 -- start start 2020-11-28T21:54:27.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.044+0000 7f79e175a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f79dc0b11b0 0x7f79dc0b4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:27.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.044+0000 7f79e175a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79dc0b1990 0x7f79dc13b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:27.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.044+0000 7f79e175a700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f79dc0b1d70 0x7f79dc13bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:27.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.044+0000 7f79e175a700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc0ae350 con 0x7f79dc0b11b0 2020-11-28T21:54:27.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.044+0000 7f79e175a700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc064110 con 0x7f79dc0b1990 2020-11-28T21:54:27.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.044+0000 7f79e175a700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc0ae580 con 0x7f79dc0b1d70 2020-11-28T21:54:27.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.044+0000 7f79da7fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79dc0b1990 0x7f79dc13b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:27.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.044+0000 7f79da7fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79dc0b1990 0x7f79dc13b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43388/0 (socket says 172.21.15.162:43388) 2020-11-28T21:54:27.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.044+0000 7f79da7fc700 1 -- 172.21.15.162:0/147089101 learned_addr learned my addr 172.21.15.162:0/147089101 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:27.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.044+0000 7f79daffd700 1 --2- 172.21.15.162:0/147089101 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f79dc0b11b0 0x7f79dc0b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:27.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.044+0000 7f79db7fe700 1 --2- 172.21.15.162:0/147089101 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f79dc0b1d70 0x7f79dc13bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:27.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.044+0000 7f79da7fc700 1 -- 172.21.15.162:0/147089101 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f79dc0b1d70 msgr2=0x7f79dc13bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:27.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.044+0000 7f79da7fc700 1 --2- 172.21.15.162:0/147089101 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f79dc0b1d70 0x7f79dc13bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:27.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.044+0000 7f79da7fc700 1 -- 172.21.15.162:0/147089101 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f79dc0b11b0 msgr2=0x7f79dc0b4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:27.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.044+0000 7f79da7fc700 1 --2- 172.21.15.162:0/147089101 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f79dc0b11b0 0x7f79dc0b4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:27.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.044+0000 7f79da7fc700 1 -- 172.21.15.162:0/147089101 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79dc13c1b0 con 0x7f79dc0b1990 2020-11-28T21:54:27.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.044+0000 7f79da7fc700 1 --2- 172.21.15.162:0/147089101 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79dc0b1990 0x7f79dc13b7d0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 rx=0x7f79d4006dd0 tx=0x7f79d4008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:27.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.044+0000 7f79d3fff700 1 -- 172.21.15.162:0/147089101 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79d400cad0 con 0x7f79dc0b1990 2020-11-28T21:54:27.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.044+0000 7f79d3fff700 1 -- 172.21.15.162:0/147089101 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d400cc30 con 0x7f79dc0b1990 2020-11-28T21:54:27.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.044+0000 7f79e175a700 1 -- 172.21.15.162:0/147089101 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79dc0672b0 con 0x7f79dc0b1990 2020-11-28T21:54:27.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.044+0000 7f79d3fff700 1 -- 172.21.15.162:0/147089101 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79d4013430 con 0x7f79dc0b1990 2020-11-28T21:54:27.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.044+0000 7f79e175a700 1 -- 172.21.15.162:0/147089101 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79dc0af3e0 con 0x7f79dc0b1990 2020-11-28T21:54:27.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.048+0000 7f79e175a700 1 -- 172.21.15.162:0/147089101 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79dc057050 con 0x7f79dc0b1990 2020-11-28T21:54:27.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.048+0000 7f79d3fff700 1 -- 172.21.15.162:0/147089101 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f79d4013590 con 0x7f79dc0b1990 2020-11-28T21:54:27.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.048+0000 7f79d3fff700 1 --2- 172.21.15.162:0/147089101 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f79c00553b0 0x7f79c0057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:27.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.048+0000 7f79d3fff700 1 -- 172.21.15.162:0/147089101 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f79d406e5a0 con 0x7f79dc0b1990 2020-11-28T21:54:27.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.048+0000 7f79daffd700 1 --2- 172.21.15.162:0/147089101 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f79c00553b0 0x7f79c0057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:27.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.048+0000 7f79daffd700 1 --2- 172.21.15.162:0/147089101 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f79c00553b0 0x7f79c0057870 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7f79c400c000 tx=0x7f79c4008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:27.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.056+0000 7f79d3fff700 1 -- 172.21.15.162:0/147089101 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f79d40447a0 con 0x7f79dc0b1990 2020-11-28T21:54:27.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.196+0000 7f79e175a700 1 -- 172.21.15.162:0/147089101 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79dc05d530 con 0x7f79dc0b1990 2020-11-28T21:54:27.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.196+0000 7f79d3fff700 1 -- 172.21.15.162:0/147089101 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+16161 (secure 0 0 0) 0x7f79d4047dc0 con 0x7f79dc0b1990 2020-11-28T21:54:27.202 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:54:27.202 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":108,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:54:26.907740+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:54:25.418443+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2020-11-28T21:54:25.913773+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:54:27.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.200+0000 7f79e175a700 1 -- 172.21.15.162:0/147089101 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f79c00553b0 msgr2=0x7f79c0057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:27.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.200+0000 7f79e175a700 1 --2- 172.21.15.162:0/147089101 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f79c00553b0 0x7f79c0057870 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7f79c400c000 tx=0x7f79c4008040).stop 2020-11-28T21:54:27.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.200+0000 7f79e175a700 1 -- 172.21.15.162:0/147089101 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79dc0b1990 msgr2=0x7f79dc13b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:27.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.200+0000 7f79e175a700 1 --2- 172.21.15.162:0/147089101 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79dc0b1990 0x7f79dc13b7d0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 rx=0x7f79d4006dd0 tx=0x7f79d4008040).stop 2020-11-28T21:54:27.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.200+0000 7f79e175a700 1 -- 172.21.15.162:0/147089101 shutdown_connections 2020-11-28T21:54:27.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.200+0000 7f79e175a700 1 --2- 172.21.15.162:0/147089101 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f79c00553b0 0x7f79c0057870 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:27.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.200+0000 7f79e175a700 1 --2- 172.21.15.162:0/147089101 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f79dc0b1d70 0x7f79dc13bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:27.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.200+0000 7f79e175a700 1 --2- 172.21.15.162:0/147089101 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f79dc0b11b0 0x7f79dc0b4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:27.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.200+0000 7f79e175a700 1 --2- 172.21.15.162:0/147089101 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79dc0b1990 0x7f79dc13b7d0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:27.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.200+0000 7f79e175a700 1 -- 172.21.15.162:0/147089101 >> 172.21.15.162:0/147089101 conn(0x7f79dc05b970 msgr2=0x7f79dc0b4dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:27.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.200+0000 7f79e175a700 1 -- 172.21.15.162:0/147089101 shutdown_connections 2020-11-28T21:54:27.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.200+0000 7f79e175a700 1 -- 172.21.15.162:0/147089101 wait complete. 2020-11-28T21:54:27.215 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:54:27.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.324+0000 7f451e4bf700 1 Processor -- start 2020-11-28T21:54:27.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.324+0000 7f451e4bf700 1 -- start start 2020-11-28T21:54:27.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.324+0000 7f451e4bf700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4518142a00 0x7f45180b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:27.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.324+0000 7f451e4bf700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f45180b4c70 0x7f4518143020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:27.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.324+0000 7f451e4bf700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45180b4110 con 0x7f45180603d0 2020-11-28T21:54:27.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.324+0000 7f451e4bf700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4518060f60 con 0x7f45180b4c70 2020-11-28T21:54:27.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.324+0000 7f451e4bf700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45180b43f0 con 0x7f4518142a00 2020-11-28T21:54:27.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.324+0000 7f451ccbc700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4518142a00 0x7f45180b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:27.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.324+0000 7f451dcbe700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f45180b4c70 0x7f4518143020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:27.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.324+0000 7f451ccbc700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4518142a00 0x7f45180b3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:40620/0 (socket says 172.21.15.162:40620) 2020-11-28T21:54:27.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.324+0000 7f451ccbc700 1 -- 172.21.15.162:0/843939656 learned_addr learned my addr 172.21.15.162:0/843939656 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:27.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.324+0000 7f451ccbc700 1 -- 172.21.15.162:0/843939656 >> v2:172.21.15.118:3300/0 conn(0x7f45180b4c70 msgr2=0x7f4518143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:27.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.324+0000 7f451ccbc700 1 --2- 172.21.15.162:0/843939656 >> v2:172.21.15.118:3300/0 conn(0x7f45180b4c70 0x7f4518143020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:27.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.324+0000 7f451ccbc700 1 -- 172.21.15.162:0/843939656 >> v1:172.21.15.118:6789/0 conn(0x7f45180603d0 legacy=0x7f451809cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:27.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.324+0000 7f451ccbc700 1 -- 172.21.15.162:0/843939656 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45180b4690 con 0x7f4518142a00 2020-11-28T21:54:27.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.324+0000 7f451ccbc700 1 --2- 172.21.15.162:0/843939656 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4518142a00 0x7f45180b3a70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f4514002f30 tx=0x7f4514006e80).ready entity=mon.2 client_cookie=d38d6ffe8774e0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:27.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.328+0000 7f450ffff700 1 -- 172.21.15.162:0/843939656 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45140097f0 con 0x7f4518142a00 2020-11-28T21:54:27.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.328+0000 7f450ffff700 1 -- 172.21.15.162:0/843939656 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4514009950 con 0x7f4518142a00 2020-11-28T21:54:27.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.328+0000 7f450ffff700 1 -- 172.21.15.162:0/843939656 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4514009cb0 con 0x7f4518142a00 2020-11-28T21:54:27.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.328+0000 7f451e4bf700 1 -- 172.21.15.162:0/843939656 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4518142a00 msgr2=0x7f45180b3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:27.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.328+0000 7f451e4bf700 1 --2- 172.21.15.162:0/843939656 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4518142a00 0x7f45180b3a70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f4514002f30 tx=0x7f4514006e80).stop 2020-11-28T21:54:27.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.328+0000 7f451e4bf700 1 -- 172.21.15.162:0/843939656 shutdown_connections 2020-11-28T21:54:27.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.328+0000 7f451e4bf700 1 --2- 172.21.15.162:0/843939656 >> v2:172.21.15.118:3300/0 conn(0x7f45180b4c70 0x7f4518143020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:27.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.328+0000 7f451e4bf700 1 --2- 172.21.15.162:0/843939656 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4518142a00 0x7f45180b3a70 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:27.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.328+0000 7f451e4bf700 1 -- 172.21.15.162:0/843939656 >> 172.21.15.162:0/843939656 conn(0x7f4518066a40 msgr2=0x7f4518066e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:27.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.328+0000 7f451e4bf700 1 -- 172.21.15.162:0/843939656 shutdown_connections 2020-11-28T21:54:27.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.328+0000 7f451e4bf700 1 -- 172.21.15.162:0/843939656 wait complete. 2020-11-28T21:54:27.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.328+0000 7f451e4bf700 1 Processor -- start 2020-11-28T21:54:27.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.328+0000 7f451e4bf700 1 -- start start 2020-11-28T21:54:27.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.328+0000 7f451e4bf700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f45180603d0 0x7f45180b1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:27.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.328+0000 7f451e4bf700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f45180b4c70 0x7f4518151bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:27.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.328+0000 7f451e4bf700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45180b8640 0x7f45181520c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:27.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.328+0000 7f451e4bf700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45181525b0 con 0x7f45180603d0 2020-11-28T21:54:27.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.328+0000 7f451e4bf700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45180ae920 con 0x7f45180b8640 2020-11-28T21:54:27.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.328+0000 7f451e4bf700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45180ae3e0 con 0x7f45180b4c70 2020-11-28T21:54:27.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.328+0000 7f451dcbe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45180b8640 0x7f45181520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:27.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.328+0000 7f451dcbe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45180b8640 0x7f45181520c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43406/0 (socket says 172.21.15.162:43406) 2020-11-28T21:54:27.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.328+0000 7f451dcbe700 1 -- 172.21.15.162:0/1899658901 learned_addr learned my addr 172.21.15.162:0/1899658901 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:27.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.328+0000 7f451ccbc700 1 --2- 172.21.15.162:0/1899658901 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f45180b4c70 0x7f4518151bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:27.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.328+0000 7f451d4bd700 1 --2- 172.21.15.162:0/1899658901 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f45180603d0 0x7f45180b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:27.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.328+0000 7f451dcbe700 1 -- 172.21.15.162:0/1899658901 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f45180b4c70 msgr2=0x7f4518151bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:27.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.328+0000 7f451dcbe700 1 --2- 172.21.15.162:0/1899658901 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f45180b4c70 0x7f4518151bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:27.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.328+0000 7f451dcbe700 1 -- 172.21.15.162:0/1899658901 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f45180603d0 msgr2=0x7f45180b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:27.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.328+0000 7f451dcbe700 1 --2- 172.21.15.162:0/1899658901 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f45180603d0 0x7f45180b1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:27.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.328+0000 7f451dcbe700 1 -- 172.21.15.162:0/1899658901 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45180ae680 con 0x7f45180b8640 2020-11-28T21:54:27.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.328+0000 7f451dcbe700 1 --2- 172.21.15.162:0/1899658901 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45180b8640 0x7f45181520c0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 rx=0x7f4510006dd0 tx=0x7f45100091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:27.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.332+0000 7f450e7fc700 1 -- 172.21.15.162:0/1899658901 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f451000c950 con 0x7f45180b8640 2020-11-28T21:54:27.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.332+0000 7f450e7fc700 1 -- 172.21.15.162:0/1899658901 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f451000cab0 con 0x7f45180b8640 2020-11-28T21:54:27.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.332+0000 7f450e7fc700 1 -- 172.21.15.162:0/1899658901 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f451000ce10 con 0x7f45180b8640 2020-11-28T21:54:27.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.332+0000 7f451e4bf700 1 -- 172.21.15.162:0/1899658901 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4518154bb0 con 0x7f45180b8640 2020-11-28T21:54:27.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.332+0000 7f451e4bf700 1 -- 172.21.15.162:0/1899658901 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4518151580 con 0x7f45180b8640 2020-11-28T21:54:27.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.332+0000 7f451e4bf700 1 -- 172.21.15.162:0/1899658901 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45180b1db0 con 0x7f45180b8640 2020-11-28T21:54:27.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.332+0000 7f450e7fc700 1 -- 172.21.15.162:0/1899658901 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4510022070 con 0x7f45180b8640 2020-11-28T21:54:27.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.332+0000 7f450e7fc700 1 --2- 172.21.15.162:0/1899658901 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f45000554e0 0x7f45000579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:27.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.332+0000 7f450e7fc700 1 -- 172.21.15.162:0/1899658901 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f451006e690 con 0x7f45180b8640 2020-11-28T21:54:27.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.332+0000 7f451d4bd700 1 --2- 172.21.15.162:0/1899658901 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f45000554e0 0x7f45000579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:27.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.332+0000 7f451d4bd700 1 --2- 172.21.15.162:0/1899658901 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f45000554e0 0x7f45000579a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7f450800d000 tx=0x7f4508000b80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:27.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.340+0000 7f450e7fc700 1 -- 172.21.15.162:0/1899658901 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f45100448f0 con 0x7f45180b8640 2020-11-28T21:54:27.485 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.480+0000 7f451e4bf700 1 -- 172.21.15.162:0/1899658901 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4518154d40 con 0x7f45180b8640 2020-11-28T21:54:27.486 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.484+0000 7f450e7fc700 1 -- 172.21.15.162:0/1899658901 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+16161 (secure 0 0 0) 0x7f4510016020 con 0x7f45180b8640 2020-11-28T21:54:27.486 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:54:27.487 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":108,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:54:26.907740+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:54:25.418443+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2020-11-28T21:54:25.913773+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:54:27.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.484+0000 7f451e4bf700 1 -- 172.21.15.162:0/1899658901 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f45000554e0 msgr2=0x7f45000579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:27.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.484+0000 7f451e4bf700 1 --2- 172.21.15.162:0/1899658901 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f45000554e0 0x7f45000579a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7f450800d000 tx=0x7f4508000b80).stop 2020-11-28T21:54:27.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.484+0000 7f451e4bf700 1 -- 172.21.15.162:0/1899658901 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45180b8640 msgr2=0x7f45181520c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:27.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.484+0000 7f451e4bf700 1 --2- 172.21.15.162:0/1899658901 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45180b8640 0x7f45181520c0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 rx=0x7f4510006dd0 tx=0x7f45100091a0).stop 2020-11-28T21:54:27.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.484+0000 7f451e4bf700 1 -- 172.21.15.162:0/1899658901 shutdown_connections 2020-11-28T21:54:27.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.484+0000 7f451e4bf700 1 --2- 172.21.15.162:0/1899658901 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f45000554e0 0x7f45000579a0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:27.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.484+0000 7f451e4bf700 1 --2- 172.21.15.162:0/1899658901 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45180b8640 0x7f45181520c0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:27.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.484+0000 7f451e4bf700 1 --2- 172.21.15.162:0/1899658901 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f45180603d0 0x7f45180b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:27.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.484+0000 7f451e4bf700 1 --2- 172.21.15.162:0/1899658901 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f45180b4c70 0x7f4518151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:27.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.484+0000 7f451e4bf700 1 -- 172.21.15.162:0/1899658901 >> 172.21.15.162:0/1899658901 conn(0x7f4518066a40 msgr2=0x7f451809cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:27.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.484+0000 7f451e4bf700 1 -- 172.21.15.162:0/1899658901 shutdown_connections 2020-11-28T21:54:27.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.484+0000 7f451e4bf700 1 -- 172.21.15.162:0/1899658901 wait complete. 2020-11-28T21:54:27.499 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2020-11-28T21:54:27.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.616+0000 7fa7de436700 1 Processor -- start 2020-11-28T21:54:27.620 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.616+0000 7fa7de436700 1 -- start start 2020-11-28T21:54:27.621 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.616+0000 7fa7de436700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa7d80b2d00 0x7fa7d805c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:27.621 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.616+0000 7fa7de436700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7d805baf0 con 0x7fa7d80b30e0 2020-11-28T21:54:27.621 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.616+0000 7fa7de436700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7d805bd90 con 0x7fa7d80b3ab0 2020-11-28T21:54:27.621 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.616+0000 7fa7de436700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7d80b0e20 con 0x7fa7d80b2d00 2020-11-28T21:54:27.622 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.616+0000 7fa7dcc33700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fa7d80b3ab0 0x7fa7d8142ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:50256/0 (socket says 172.21.15.162:50256) 2020-11-28T21:54:27.622 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.616+0000 7fa7dd434700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa7d80b2d00 0x7fa7d805c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:27.622 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.616+0000 7fa7dcc33700 1 -- 172.21.15.162:0/4186869768 learned_addr learned my addr 172.21.15.162:0/4186869768 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:27.624 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.616+0000 7fa7cffff700 1 -- 172.21.15.162:0/4186869768 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 603427696 0 0) 0x7fa7d805bd90 con 0x7fa7d80b3ab0 2020-11-28T21:54:27.624 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.616+0000 7fa7cffff700 1 -- 172.21.15.162:0/4186869768 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7bc003230 con 0x7fa7d80b3ab0 2020-11-28T21:54:27.624 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.616+0000 7fa7dd434700 1 -- 172.21.15.162:0/4186869768 >> v1:172.21.15.162:6789/0 conn(0x7fa7d80b3ab0 legacy=0x7fa7d8142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:27.624 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.616+0000 7fa7dd434700 1 -- 172.21.15.162:0/4186869768 >> v1:172.21.15.118:6789/0 conn(0x7fa7d80b30e0 legacy=0x7fa7d813fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:27.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.620+0000 7fa7dd434700 1 -- 172.21.15.162:0/4186869768 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d805c030 con 0x7fa7d80b2d00 2020-11-28T21:54:27.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.620+0000 7fa7dd434700 1 --2- 172.21.15.162:0/4186869768 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7d80b2d00 0x7fa7d805c4d0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 rx=0x7fa7c8002f30 tx=0x7fa7c8006dc0).ready entity=mon.1 client_cookie=2c41701ac748a7da server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:27.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.620+0000 7fa7cffff700 1 -- 172.21.15.162:0/4186869768 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7c8009800 con 0x7fa7d80b2d00 2020-11-28T21:54:27.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.620+0000 7fa7cffff700 1 -- 172.21.15.162:0/4186869768 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c8009960 con 0x7fa7d80b2d00 2020-11-28T21:54:27.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.620+0000 7fa7cffff700 1 -- 172.21.15.162:0/4186869768 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7c8009cc0 con 0x7fa7d80b2d00 2020-11-28T21:54:27.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.620+0000 7fa7de436700 1 -- 172.21.15.162:0/4186869768 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7d80b2d00 msgr2=0x7fa7d805c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:27.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.620+0000 7fa7de436700 1 --2- 172.21.15.162:0/4186869768 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7d80b2d00 0x7fa7d805c4d0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 rx=0x7fa7c8002f30 tx=0x7fa7c8006dc0).stop 2020-11-28T21:54:27.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.620+0000 7fa7de436700 1 -- 172.21.15.162:0/4186869768 shutdown_connections 2020-11-28T21:54:27.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.620+0000 7fa7de436700 1 --2- 172.21.15.162:0/4186869768 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7d80b2d00 0x7fa7d805c4d0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:27.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.620+0000 7fa7de436700 1 -- 172.21.15.162:0/4186869768 >> 172.21.15.162:0/4186869768 conn(0x7fa7d80604c0 msgr2=0x7fa7d80608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:27.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.620+0000 7fa7de436700 1 -- 172.21.15.162:0/4186869768 shutdown_connections 2020-11-28T21:54:27.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.620+0000 7fa7de436700 1 -- 172.21.15.162:0/4186869768 wait complete. 2020-11-28T21:54:27.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.620+0000 7fa7de436700 1 Processor -- start 2020-11-28T21:54:27.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.620+0000 7fa7de436700 1 -- start start 2020-11-28T21:54:27.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.624+0000 7fa7de436700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7d80b2d00 0x7fa7d81434d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:27.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.624+0000 7fa7de436700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa7d80b30e0 0x7fa7d8140930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:27.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.624+0000 7fa7de436700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa7d80b3ab0 0x7fa7d8140e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:27.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.624+0000 7fa7de436700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d80b9e10 con 0x7fa7d80b3ab0 2020-11-28T21:54:27.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.624+0000 7fa7de436700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d80b9870 con 0x7fa7d80b2d00 2020-11-28T21:54:27.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.624+0000 7fa7de436700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d80b4ba0 con 0x7fa7d80b30e0 2020-11-28T21:54:27.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.624+0000 7fa7dd434700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7d80b2d00 0x7fa7d81434d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:27.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.624+0000 7fa7dd434700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7d80b2d00 0x7fa7d81434d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43416/0 (socket says 172.21.15.162:43416) 2020-11-28T21:54:27.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.624+0000 7fa7dd434700 1 -- 172.21.15.162:0/4184584202 learned_addr learned my addr 172.21.15.162:0/4184584202 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:27.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.624+0000 7fa7ddc35700 1 --2- 172.21.15.162:0/4184584202 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa7d80b3ab0 0x7fa7d8140e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:27.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.624+0000 7fa7dcc33700 1 --2- 172.21.15.162:0/4184584202 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa7d80b30e0 0x7fa7d8140930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:27.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.624+0000 7fa7dd434700 1 -- 172.21.15.162:0/4184584202 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa7d80b30e0 msgr2=0x7fa7d8140930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:27.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.624+0000 7fa7dd434700 1 --2- 172.21.15.162:0/4184584202 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa7d80b30e0 0x7fa7d8140930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:27.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.624+0000 7fa7dd434700 1 -- 172.21.15.162:0/4184584202 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa7d80b3ab0 msgr2=0x7fa7d8140e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:27.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.624+0000 7fa7dd434700 1 --2- 172.21.15.162:0/4184584202 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa7d80b3ab0 0x7fa7d8140e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:27.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.624+0000 7fa7dd434700 1 -- 172.21.15.162:0/4184584202 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d80b4e40 con 0x7fa7d80b2d00 2020-11-28T21:54:27.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.624+0000 7fa7ddc35700 1 --2- 172.21.15.162:0/4184584202 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa7d80b3ab0 0x7fa7d8140e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:54:27.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.624+0000 7fa7dcc33700 1 --2- 172.21.15.162:0/4184584202 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa7d80b30e0 0x7fa7d8140930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:54:27.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.624+0000 7fa7dd434700 1 --2- 172.21.15.162:0/4184584202 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7d80b2d00 0x7fa7d81434d0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 rx=0x7fa7c8002f30 tx=0x7fa7c8015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:27.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.624+0000 7fa7ce7fc700 1 -- 172.21.15.162:0/4184584202 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7c8011ce0 con 0x7fa7d80b2d00 2020-11-28T21:54:27.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.624+0000 7fa7ce7fc700 1 -- 172.21.15.162:0/4184584202 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c8011e40 con 0x7fa7d80b2d00 2020-11-28T21:54:27.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.624+0000 7fa7de436700 1 -- 172.21.15.162:0/4184584202 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7d8142ea0 con 0x7fa7d80b2d00 2020-11-28T21:54:27.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.624+0000 7fa7ce7fc700 1 -- 172.21.15.162:0/4184584202 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7c801d630 con 0x7fa7d80b2d00 2020-11-28T21:54:27.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.624+0000 7fa7de436700 1 -- 172.21.15.162:0/4184584202 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7d80b8270 con 0x7fa7d80b2d00 2020-11-28T21:54:27.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.624+0000 7fa7de436700 1 -- 172.21.15.162:0/4184584202 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7d80b9a70 con 0x7fa7d80b2d00 2020-11-28T21:54:27.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.624+0000 7fa7ce7fc700 1 -- 172.21.15.162:0/4184584202 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa7c8024030 con 0x7fa7d80b2d00 2020-11-28T21:54:27.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.624+0000 7fa7ce7fc700 1 --2- 172.21.15.162:0/4184584202 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa7c0055330 0x7fa7c00577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:27.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.628+0000 7fa7dcc33700 1 --2- 172.21.15.162:0/4184584202 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa7c0055330 0x7fa7c00577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:27.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.628+0000 7fa7ce7fc700 1 -- 172.21.15.162:0/4184584202 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa7c806ed80 con 0x7fa7d80b2d00 2020-11-28T21:54:27.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.628+0000 7fa7dcc33700 1 --2- 172.21.15.162:0/4184584202 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa7c0055330 0x7fa7c00577f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7fa7d4008000 tx=0x7fa7d4009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:27.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.632+0000 7fa7ce7fc700 1 -- 172.21.15.162:0/4184584202 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa7c8045080 con 0x7fa7d80b2d00 2020-11-28T21:54:27.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.816+0000 7fa7de436700 1 -- 172.21.15.162:0/4184584202 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa7d80b9610 con 0x7fa7d80b2d00 2020-11-28T21:54:27.918 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.916+0000 7fa7ce7fc700 1 -- 172.21.15.162:0/4184584202 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v51) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa7c80700b0 con 0x7fa7d80b2d00 2020-11-28T21:54:27.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.916+0000 7fa7de436700 1 -- 172.21.15.162:0/4184584202 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa7c0055330 msgr2=0x7fa7c00577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:27.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.916+0000 7fa7de436700 1 --2- 172.21.15.162:0/4184584202 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa7c0055330 0x7fa7c00577f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7fa7d4008000 tx=0x7fa7d4009040).stop 2020-11-28T21:54:27.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.916+0000 7fa7de436700 1 -- 172.21.15.162:0/4184584202 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7d80b2d00 msgr2=0x7fa7d81434d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:27.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.916+0000 7fa7de436700 1 --2- 172.21.15.162:0/4184584202 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7d80b2d00 0x7fa7d81434d0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 rx=0x7fa7c8002f30 tx=0x7fa7c8015000).stop 2020-11-28T21:54:27.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.920+0000 7fa7de436700 1 -- 172.21.15.162:0/4184584202 shutdown_connections 2020-11-28T21:54:27.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.920+0000 7fa7de436700 1 --2- 172.21.15.162:0/4184584202 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa7c0055330 0x7fa7c00577f0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:27.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.920+0000 7fa7de436700 1 --2- 172.21.15.162:0/4184584202 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa7d80b3ab0 0x7fa7d8140e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:27.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.920+0000 7fa7de436700 1 --2- 172.21.15.162:0/4184584202 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7d80b2d00 0x7fa7d81434d0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:27.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.920+0000 7fa7de436700 1 --2- 172.21.15.162:0/4184584202 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa7d80b30e0 0x7fa7d8140930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:27.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.920+0000 7fa7de436700 1 -- 172.21.15.162:0/4184584202 >> 172.21.15.162:0/4184584202 conn(0x7fa7d80604c0 msgr2=0x7fa7d813bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:27.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.920+0000 7fa7de436700 1 -- 172.21.15.162:0/4184584202 shutdown_connections 2020-11-28T21:54:27.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:27.920+0000 7fa7de436700 1 -- 172.21.15.162:0/4184584202 wait complete. 2020-11-28T21:54:27.938 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:54:28.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.044+0000 7f5eb5bf3700 1 Processor -- start 2020-11-28T21:54:28.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.044+0000 7f5eb5bf3700 1 -- start start 2020-11-28T21:54:28.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.044+0000 7f5eb5bf3700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5eb00603d0 0x7f5eb00b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:28.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.044+0000 7f5eb5bf3700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5eb0142a00 0x7f5eb009cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:28.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.044+0000 7f5eb5bf3700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eb00b4110 con 0x7f5eb00b4c70 2020-11-28T21:54:28.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.044+0000 7f5eb5bf3700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eb0060f60 con 0x7f5eb0142a00 2020-11-28T21:54:28.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.044+0000 7f5eb5bf3700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eb00b43f0 con 0x7f5eb00603d0 2020-11-28T21:54:28.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.044+0000 7f5eb4bf1700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5eb00603d0 0x7f5eb00b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:28.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.044+0000 7f5eb4bf1700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5eb00603d0 0x7f5eb00b3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43424/0 (socket says 172.21.15.162:43424) 2020-11-28T21:54:28.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.044+0000 7f5eb4bf1700 1 -- 172.21.15.162:0/3242824209 learned_addr learned my addr 172.21.15.162:0/3242824209 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:28.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.044+0000 7f5eaffff700 1 --2- 172.21.15.162:0/3242824209 >> v2:172.21.15.118:3300/0 conn(0x7f5eb0142a00 0x7f5eb009cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:28.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.044+0000 7f5eb4bf1700 1 -- 172.21.15.162:0/3242824209 >> v2:172.21.15.118:3300/0 conn(0x7f5eb0142a00 msgr2=0x7f5eb009cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:28.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.044+0000 7f5eb4bf1700 1 --2- 172.21.15.162:0/3242824209 >> v2:172.21.15.118:3300/0 conn(0x7f5eb0142a00 0x7f5eb009cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:28.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.044+0000 7f5eb4bf1700 1 -- 172.21.15.162:0/3242824209 >> v1:172.21.15.118:6789/0 conn(0x7f5eb00b4c70 legacy=0x7f5eb0143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:28.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.044+0000 7f5eb4bf1700 1 -- 172.21.15.162:0/3242824209 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb00b4690 con 0x7f5eb00603d0 2020-11-28T21:54:28.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.044+0000 7f5eb4bf1700 1 --2- 172.21.15.162:0/3242824209 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5eb00603d0 0x7f5eb00b3a70 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 rx=0x7f5ea00032b0 tx=0x7f5ea0009270).ready entity=mon.1 client_cookie=cde05cb1524d037a server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:28.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.044+0000 7f5eaf7fe700 1 -- 172.21.15.162:0/3242824209 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ea0009e60 con 0x7f5eb00603d0 2020-11-28T21:54:28.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.044+0000 7f5eaf7fe700 1 -- 172.21.15.162:0/3242824209 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea000bc00 con 0x7f5eb00603d0 2020-11-28T21:54:28.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.044+0000 7f5eaf7fe700 1 -- 172.21.15.162:0/3242824209 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ea000bd70 con 0x7f5eb00603d0 2020-11-28T21:54:28.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.044+0000 7f5eb5bf3700 1 -- 172.21.15.162:0/3242824209 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5eb00603d0 msgr2=0x7f5eb00b3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:28.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.044+0000 7f5eb5bf3700 1 --2- 172.21.15.162:0/3242824209 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5eb00603d0 0x7f5eb00b3a70 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 rx=0x7f5ea00032b0 tx=0x7f5ea0009270).stop 2020-11-28T21:54:28.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.048+0000 7f5eb5bf3700 1 -- 172.21.15.162:0/3242824209 shutdown_connections 2020-11-28T21:54:28.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.048+0000 7f5eb5bf3700 1 --2- 172.21.15.162:0/3242824209 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5eb00603d0 0x7f5eb00b3a70 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:28.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.048+0000 7f5eb5bf3700 1 --2- 172.21.15.162:0/3242824209 >> v2:172.21.15.118:3300/0 conn(0x7f5eb0142a00 0x7f5eb009cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:28.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.048+0000 7f5eb5bf3700 1 -- 172.21.15.162:0/3242824209 >> 172.21.15.162:0/3242824209 conn(0x7f5eb0066a40 msgr2=0x7f5eb0066e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:28.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.048+0000 7f5eb5bf3700 1 -- 172.21.15.162:0/3242824209 shutdown_connections 2020-11-28T21:54:28.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.048+0000 7f5eb5bf3700 1 -- 172.21.15.162:0/3242824209 wait complete. 2020-11-28T21:54:28.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.048+0000 7f5eb5bf3700 1 Processor -- start 2020-11-28T21:54:28.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.048+0000 7f5eb5bf3700 1 -- start start 2020-11-28T21:54:28.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.052+0000 7f5eb5bf3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5eb00603d0 0x7f5eb009dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:28.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.052+0000 7f5eb5bf3700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5eb00b4c70 0x7f5eb0155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:28.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.052+0000 7f5eb5bf3700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5eb00adab0 0x7f5eb00b7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:28.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.052+0000 7f5eb5bf3700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb00ae1f0 con 0x7f5eb00adab0 2020-11-28T21:54:28.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.052+0000 7f5eb5bf3700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb00adf50 con 0x7f5eb00603d0 2020-11-28T21:54:28.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.052+0000 7f5eb5bf3700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb0154df0 con 0x7f5eb00b4c70 2020-11-28T21:54:28.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.052+0000 7f5eb4bf1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5eb00603d0 0x7f5eb009dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:28.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.052+0000 7f5eb4bf1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5eb00603d0 0x7f5eb009dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43430/0 (socket says 172.21.15.162:43430) 2020-11-28T21:54:28.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.052+0000 7f5eb4bf1700 1 -- 172.21.15.162:0/4142966131 learned_addr learned my addr 172.21.15.162:0/4142966131 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:28.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.052+0000 7f5eaffff700 1 --2- 172.21.15.162:0/4142966131 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5eb00b4c70 0x7f5eb0155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:28.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.052+0000 7f5eb53f2700 1 --2- 172.21.15.162:0/4142966131 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5eb00adab0 0x7f5eb00b7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:28.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.052+0000 7f5eb4bf1700 1 -- 172.21.15.162:0/4142966131 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5eb00b4c70 msgr2=0x7f5eb0155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:28.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.052+0000 7f5eb4bf1700 1 --2- 172.21.15.162:0/4142966131 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5eb00b4c70 0x7f5eb0155530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:28.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.052+0000 7f5eb4bf1700 1 -- 172.21.15.162:0/4142966131 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5eb00adab0 msgr2=0x7f5eb00b7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:28.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.052+0000 7f5eb4bf1700 1 --2- 172.21.15.162:0/4142966131 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5eb00adab0 0x7f5eb00b7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:28.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.052+0000 7f5eb4bf1700 1 -- 172.21.15.162:0/4142966131 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb0155090 con 0x7f5eb00603d0 2020-11-28T21:54:28.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.052+0000 7f5eb4bf1700 1 --2- 172.21.15.162:0/4142966131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5eb00603d0 0x7f5eb009dc40 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 rx=0x7f5ea00032b0 tx=0x7f5ea0016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:28.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.052+0000 7f5eadffb700 1 -- 172.21.15.162:0/4142966131 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ea00107a0 con 0x7f5eb00603d0 2020-11-28T21:54:28.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.052+0000 7f5eadffb700 1 -- 172.21.15.162:0/4142966131 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea0010900 con 0x7f5eb00603d0 2020-11-28T21:54:28.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.052+0000 7f5eb5bf3700 1 -- 172.21.15.162:0/4142966131 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eb009e130 con 0x7f5eb00603d0 2020-11-28T21:54:28.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.052+0000 7f5eadffb700 1 -- 172.21.15.162:0/4142966131 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ea0010c60 con 0x7f5eb00603d0 2020-11-28T21:54:28.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.052+0000 7f5eb5bf3700 1 -- 172.21.15.162:0/4142966131 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eb0155a20 con 0x7f5eb00603d0 2020-11-28T21:54:28.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.056+0000 7f5eb5bf3700 1 -- 172.21.15.162:0/4142966131 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eb00ad7e0 con 0x7f5eb00603d0 2020-11-28T21:54:28.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.056+0000 7f5eadffb700 1 -- 172.21.15.162:0/4142966131 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5ea0026030 con 0x7f5eb00603d0 2020-11-28T21:54:28.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.056+0000 7f5eadffb700 1 --2- 172.21.15.162:0/4142966131 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5e90055500 0x7f5e900579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:28.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.056+0000 7f5eaffff700 1 --2- 172.21.15.162:0/4142966131 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5e90055500 0x7f5e900579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:28.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.056+0000 7f5eadffb700 1 -- 172.21.15.162:0/4142966131 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5ea006eff0 con 0x7f5eb00603d0 2020-11-28T21:54:28.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.056+0000 7f5eaffff700 1 --2- 172.21.15.162:0/4142966131 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5e90055500 0x7f5e900579c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7f5ea400b000 tx=0x7f5ea4006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:28.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.064+0000 7f5eadffb700 1 -- 172.21.15.162:0/4142966131 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5eb00ad7e0 con 0x7f5eb00603d0 2020-11-28T21:54:28.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.252+0000 7f5eb5bf3700 1 -- 172.21.15.162:0/4142966131 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5eb00ad7e0 con 0x7f5eb00603d0 2020-11-28T21:54:28.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.252+0000 7f5eadffb700 1 -- 172.21.15.162:0/4142966131 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+2499 (secure 0 0 0) 0x7f5eb00ad7e0 con 0x7f5eb00603d0 2020-11-28T21:54:28.256 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:54:28.257 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":51,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6182,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2775691244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2775691244},{"type":"v1","addr":"172.21.15.162:6835","nonce":2775691244}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":49},{"gid":6942,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/1896314020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1896314020},{"type":"v1","addr":"172.21.15.118:6833","nonce":1896314020}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":49}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:54:26.908249+0000","modified":"2020-11-28T21:54:27.911138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7177},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7177":{"gid":7177,"name":"a","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/493776989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":493776989},{"type":"v1","addr":"172.21.15.162:6837","nonce":493776989}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2020-11-28T21:54:28.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.256+0000 7f5eb5bf3700 1 -- 172.21.15.162:0/4142966131 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5e90055500 msgr2=0x7f5e900579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:28.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.256+0000 7f5eb5bf3700 1 --2- 172.21.15.162:0/4142966131 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5e90055500 0x7f5e900579c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7f5ea400b000 tx=0x7f5ea4006dc0).stop 2020-11-28T21:54:28.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.256+0000 7f5eb5bf3700 1 -- 172.21.15.162:0/4142966131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5eb00603d0 msgr2=0x7f5eb009dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:28.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.256+0000 7f5eb5bf3700 1 --2- 172.21.15.162:0/4142966131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5eb00603d0 0x7f5eb009dc40 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 rx=0x7f5ea00032b0 tx=0x7f5ea0016000).stop 2020-11-28T21:54:28.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.256+0000 7f5eb5bf3700 1 -- 172.21.15.162:0/4142966131 shutdown_connections 2020-11-28T21:54:28.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.256+0000 7f5eb5bf3700 1 --2- 172.21.15.162:0/4142966131 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5e90055500 0x7f5e900579c0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:28.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.256+0000 7f5eb5bf3700 1 --2- 172.21.15.162:0/4142966131 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5eb00adab0 0x7f5eb00b7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:28.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.256+0000 7f5eb5bf3700 1 --2- 172.21.15.162:0/4142966131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5eb00603d0 0x7f5eb009dc40 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:28.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.256+0000 7f5eb5bf3700 1 --2- 172.21.15.162:0/4142966131 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5eb00b4c70 0x7f5eb0155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:28.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.256+0000 7f5eb5bf3700 1 -- 172.21.15.162:0/4142966131 >> 172.21.15.162:0/4142966131 conn(0x7f5eb0066a40 msgr2=0x7f5eb0142df0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:28.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.256+0000 7f5eb5bf3700 1 -- 172.21.15.162:0/4142966131 shutdown_connections 2020-11-28T21:54:28.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.256+0000 7f5eb5bf3700 1 -- 172.21.15.162:0/4142966131 wait complete. 2020-11-28T21:54:28.261 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 51 2020-11-28T21:54:28.270 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:54:28.378 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.376+0000 7f47f4227700 1 Processor -- start 2020-11-28T21:54:28.379 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.376+0000 7f47f4227700 1 -- start start 2020-11-28T21:54:28.380 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.376+0000 7f47f4227700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f47ec0b1990 0x7f47ec140890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:28.380 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.376+0000 7f47f4227700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47ec0b2870 con 0x7f47ec0b11b0 2020-11-28T21:54:28.380 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.376+0000 7f47f4227700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47ec0b22e0 con 0x7f47ec0b1d70 2020-11-28T21:54:28.381 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.376+0000 7f47f4227700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47ec146e20 con 0x7f47ec0b1990 2020-11-28T21:54:28.381 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.376+0000 7f47f27c4700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f47ec0b1d70 0x7f47ec14a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:50286/0 (socket says 172.21.15.162:50286) 2020-11-28T21:54:28.381 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.376+0000 7f47f27c4700 1 -- 172.21.15.162:0/510526424 learned_addr learned my addr 172.21.15.162:0/510526424 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:28.381 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.376+0000 7f47f1fc3700 1 --2- 172.21.15.162:0/510526424 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f47ec0b1990 0x7f47ec140890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:28.381 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.376+0000 7f47f0fc1700 1 -- 172.21.15.162:0/510526424 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3065683142 0 0) 0x7f47ec0b22e0 con 0x7f47ec0b1d70 2020-11-28T21:54:28.382 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.376+0000 7f47f0fc1700 1 -- 172.21.15.162:0/510526424 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47cc003230 con 0x7f47ec0b1d70 2020-11-28T21:54:28.382 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.376+0000 7f47f1fc3700 1 -- 172.21.15.162:0/510526424 >> v1:172.21.15.162:6789/0 conn(0x7f47ec0b1d70 legacy=0x7f47ec14a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:28.382 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.376+0000 7f47f0fc1700 1 -- 172.21.15.162:0/510526424 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3885909814 0 0) 0x7f47ec0b2870 con 0x7f47ec0b11b0 2020-11-28T21:54:28.382 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.376+0000 7f47f1fc3700 1 -- 172.21.15.162:0/510526424 >> v1:172.21.15.118:6789/0 conn(0x7f47ec0b11b0 legacy=0x7f47ec140d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:28.383 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.376+0000 7f47f1fc3700 1 -- 172.21.15.162:0/510526424 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ec0b0740 con 0x7f47ec0b1990 2020-11-28T21:54:28.383 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.376+0000 7f47f1fc3700 1 --2- 172.21.15.162:0/510526424 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f47ec0b1990 0x7f47ec140890 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f47d4002f30 tx=0x7f47d4006dc0).ready entity=mon.2 client_cookie=f4652d64eb7d9b05 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:28.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.376+0000 7f47f0fc1700 1 -- 172.21.15.162:0/510526424 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47d40097b0 con 0x7f47ec0b1990 2020-11-28T21:54:28.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.376+0000 7f47f0fc1700 1 -- 172.21.15.162:0/510526424 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47d4009910 con 0x7f47ec0b1990 2020-11-28T21:54:28.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.376+0000 7f47f4227700 1 -- 172.21.15.162:0/510526424 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f47ec0b1990 msgr2=0x7f47ec140890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:28.386 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.376+0000 7f47f4227700 1 --2- 172.21.15.162:0/510526424 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f47ec0b1990 0x7f47ec140890 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f47d4002f30 tx=0x7f47d4006dc0).stop 2020-11-28T21:54:28.386 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.376+0000 7f47f4227700 1 -- 172.21.15.162:0/510526424 shutdown_connections 2020-11-28T21:54:28.386 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.376+0000 7f47f4227700 1 --2- 172.21.15.162:0/510526424 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f47ec0b1990 0x7f47ec140890 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:28.386 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.376+0000 7f47f4227700 1 -- 172.21.15.162:0/510526424 >> 172.21.15.162:0/510526424 conn(0x7f47ec05b970 msgr2=0x7f47ec05bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:28.387 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.376+0000 7f47f4227700 1 -- 172.21.15.162:0/510526424 shutdown_connections 2020-11-28T21:54:28.387 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.380+0000 7f47f4227700 1 -- 172.21.15.162:0/510526424 wait complete. 2020-11-28T21:54:28.387 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.380+0000 7f47f4227700 1 Processor -- start 2020-11-28T21:54:28.388 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.380+0000 7f47f4227700 1 -- start start 2020-11-28T21:54:28.388 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.380+0000 7f47f4227700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f47ec0b11b0 0x7f47ec066420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:28.388 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.380+0000 7f47f4227700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47ec0b1990 0x7f47ec0aead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:28.388 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.380+0000 7f47f4227700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f47ec0b1d70 0x7f47ec1560c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:28.388 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.380+0000 7f47f4227700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ec066230 con 0x7f47ec0b1d70 2020-11-28T21:54:28.389 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.380+0000 7f47f4227700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ec143c00 con 0x7f47ec0b1990 2020-11-28T21:54:28.389 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.380+0000 7f47f4227700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47ec143ea0 con 0x7f47ec0b11b0 2020-11-28T21:54:28.389 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.380+0000 7f47f17c2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47ec0b1990 0x7f47ec0aead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:28.389 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.380+0000 7f47f17c2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47ec0b1990 0x7f47ec0aead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43446/0 (socket says 172.21.15.162:43446) 2020-11-28T21:54:28.389 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.380+0000 7f47f17c2700 1 -- 172.21.15.162:0/456815722 learned_addr learned my addr 172.21.15.162:0/456815722 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:28.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.380+0000 7f47f1fc3700 1 --2- 172.21.15.162:0/456815722 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f47ec0b11b0 0x7f47ec066420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:28.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.380+0000 7f47f27c4700 1 --2- 172.21.15.162:0/456815722 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f47ec0b1d70 0x7f47ec1560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:28.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.380+0000 7f47f17c2700 1 -- 172.21.15.162:0/456815722 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f47ec0b11b0 msgr2=0x7f47ec066420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:28.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.380+0000 7f47f17c2700 1 --2- 172.21.15.162:0/456815722 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f47ec0b11b0 0x7f47ec066420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:28.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.380+0000 7f47f17c2700 1 -- 172.21.15.162:0/456815722 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f47ec0b1d70 msgr2=0x7f47ec1560c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:28.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.380+0000 7f47f17c2700 1 --2- 172.21.15.162:0/456815722 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f47ec0b1d70 0x7f47ec1560c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:28.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.380+0000 7f47f17c2700 1 -- 172.21.15.162:0/456815722 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ec144140 con 0x7f47ec0b1990 2020-11-28T21:54:28.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.380+0000 7f47f17c2700 1 --2- 172.21.15.162:0/456815722 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47ec0b1990 0x7f47ec0aead0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 rx=0x7f47dc000f00 tx=0x7f47dc007dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:28.392 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.380+0000 7f47eaffd700 1 -- 172.21.15.162:0/456815722 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47dc00a7b0 con 0x7f47ec0b1990 2020-11-28T21:54:28.392 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.380+0000 7f47eaffd700 1 -- 172.21.15.162:0/456815722 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47dc00a910 con 0x7f47ec0b1990 2020-11-28T21:54:28.392 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.380+0000 7f47f4227700 1 -- 172.21.15.162:0/456815722 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47ec155c20 con 0x7f47ec0b1990 2020-11-28T21:54:28.392 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.380+0000 7f47eaffd700 1 -- 172.21.15.162:0/456815722 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47dc00ac70 con 0x7f47ec0b1990 2020-11-28T21:54:28.392 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.380+0000 7f47f4227700 1 -- 172.21.15.162:0/456815722 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47ec0aefc0 con 0x7f47ec0b1990 2020-11-28T21:54:28.394 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.384+0000 7f47f4227700 1 -- 172.21.15.162:0/456815722 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47ec155db0 con 0x7f47ec0b1990 2020-11-28T21:54:28.394 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.384+0000 7f47eaffd700 1 -- 172.21.15.162:0/456815722 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f47dc019030 con 0x7f47ec0b1990 2020-11-28T21:54:28.394 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.384+0000 7f47eaffd700 1 --2- 172.21.15.162:0/456815722 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f47d00553b0 0x7f47d0057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:28.394 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.384+0000 7f47eaffd700 1 -- 172.21.15.162:0/456815722 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f47dc06d690 con 0x7f47ec0b1990 2020-11-28T21:54:28.395 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.384+0000 7f47f1fc3700 1 --2- 172.21.15.162:0/456815722 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f47d00553b0 0x7f47d0057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:28.395 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.384+0000 7f47f1fc3700 1 --2- 172.21.15.162:0/456815722 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f47d00553b0 0x7f47d0057870 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f47d401a000 tx=0x7f47d4015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:28.395 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.388+0000 7f47eaffd700 1 -- 172.21.15.162:0/456815722 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f47dc041df0 con 0x7f47ec0b1990 2020-11-28T21:54:28.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.532+0000 7f47f4227700 1 -- 172.21.15.162:0/456815722 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f47ec057050 con 0x7f47ec0b1990 2020-11-28T21:54:28.538 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.536+0000 7f47eaffd700 1 -- 172.21.15.162:0/456815722 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+16148 (secure 0 0 0) 0x7f47dc024f30 con 0x7f47ec0b1990 2020-11-28T21:54:28.539 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:54:28.539 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":109,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:54:27.911877+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:54:25.418443+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2020-11-28T21:54:25.913773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:54:28.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.536+0000 7f47f4227700 1 -- 172.21.15.162:0/456815722 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f47d00553b0 msgr2=0x7f47d0057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:28.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.536+0000 7f47f4227700 1 --2- 172.21.15.162:0/456815722 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f47d00553b0 0x7f47d0057870 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f47d401a000 tx=0x7f47d4015000).stop 2020-11-28T21:54:28.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.536+0000 7f47f4227700 1 -- 172.21.15.162:0/456815722 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47ec0b1990 msgr2=0x7f47ec0aead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:28.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.536+0000 7f47f4227700 1 --2- 172.21.15.162:0/456815722 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47ec0b1990 0x7f47ec0aead0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 rx=0x7f47dc000f00 tx=0x7f47dc007dc0).stop 2020-11-28T21:54:28.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.536+0000 7f47f4227700 1 -- 172.21.15.162:0/456815722 shutdown_connections 2020-11-28T21:54:28.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.536+0000 7f47f4227700 1 --2- 172.21.15.162:0/456815722 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f47d00553b0 0x7f47d0057870 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:28.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.536+0000 7f47f4227700 1 --2- 172.21.15.162:0/456815722 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f47ec0b11b0 0x7f47ec066420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:28.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.536+0000 7f47f4227700 1 --2- 172.21.15.162:0/456815722 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f47ec0b1d70 0x7f47ec1560c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:28.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.536+0000 7f47f4227700 1 --2- 172.21.15.162:0/456815722 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47ec0b1990 0x7f47ec0aead0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:28.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.536+0000 7f47f4227700 1 -- 172.21.15.162:0/456815722 >> 172.21.15.162:0/456815722 conn(0x7f47ec05b970 msgr2=0x7f47ec140660 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:28.543 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.536+0000 7f47f4227700 1 -- 172.21.15.162:0/456815722 shutdown_connections 2020-11-28T21:54:28.543 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.536+0000 7f47f4227700 1 -- 172.21.15.162:0/456815722 wait complete. 2020-11-28T21:54:28.553 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-11-28T21:54:28.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.672+0000 7fad8d703700 1 Processor -- start 2020-11-28T21:54:28.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.672+0000 7fad8d703700 1 -- start start 2020-11-28T21:54:28.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.672+0000 7fad8d703700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fad880b55a0 0x7fad880ad6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:28.680 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.672+0000 7fad8d703700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fad880a86d0 0x7fad88153080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:28.680 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.672+0000 7fad8d703700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad880b5980 con 0x7fad88080500 2020-11-28T21:54:28.680 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.672+0000 7fad8d703700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad880afe80 con 0x7fad880b55a0 2020-11-28T21:54:28.680 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.672+0000 7fad8d703700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad880809a0 con 0x7fad880a86d0 2020-11-28T21:54:28.680 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.672+0000 7fad877fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fad880b55a0 0x7fad880ad6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:28.681 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.672+0000 7fad877fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fad880b55a0 0x7fad880ad6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43454/0 (socket says 172.21.15.162:43454) 2020-11-28T21:54:28.681 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.672+0000 7fad877fe700 1 -- 172.21.15.162:0/1923149339 learned_addr learned my addr 172.21.15.162:0/1923149339 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:28.681 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.672+0000 7fad87fff700 1 --2- 172.21.15.162:0/1923149339 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fad880a86d0 0x7fad88153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:28.681 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.672+0000 7fad877fe700 1 -- 172.21.15.162:0/1923149339 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fad880a86d0 msgr2=0x7fad88153080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:28.682 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.672+0000 7fad877fe700 1 --2- 172.21.15.162:0/1923149339 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fad880a86d0 0x7fad88153080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:28.682 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.672+0000 7fad877fe700 1 -- 172.21.15.162:0/1923149339 >> v1:172.21.15.118:6789/0 conn(0x7fad88080500 legacy=0x7fad880adbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:28.682 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.672+0000 7fad877fe700 1 -- 172.21.15.162:0/1923149339 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad880af740 con 0x7fad880b55a0 2020-11-28T21:54:28.682 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.672+0000 7fad877fe700 1 --2- 172.21.15.162:0/1923149339 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad880b55a0 0x7fad880ad6c0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 rx=0x7fad78002f30 tx=0x7fad78006e80).ready entity=mon.1 client_cookie=34583350c3c20168 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:28.682 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.672+0000 7fad867fc700 1 -- 172.21.15.162:0/1923149339 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad780097f0 con 0x7fad880b55a0 2020-11-28T21:54:28.683 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.672+0000 7fad867fc700 1 -- 172.21.15.162:0/1923149339 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad78009950 con 0x7fad880b55a0 2020-11-28T21:54:28.683 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.672+0000 7fad867fc700 1 -- 172.21.15.162:0/1923149339 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad78009cb0 con 0x7fad880b55a0 2020-11-28T21:54:28.683 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.672+0000 7fad8d703700 1 -- 172.21.15.162:0/1923149339 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad880b55a0 msgr2=0x7fad880ad6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:28.683 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.672+0000 7fad8d703700 1 --2- 172.21.15.162:0/1923149339 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad880b55a0 0x7fad880ad6c0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 rx=0x7fad78002f30 tx=0x7fad78006e80).stop 2020-11-28T21:54:28.683 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.676+0000 7fad8d703700 1 -- 172.21.15.162:0/1923149339 shutdown_connections 2020-11-28T21:54:28.684 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.676+0000 7fad8d703700 1 --2- 172.21.15.162:0/1923149339 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fad880a86d0 0x7fad88153080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:28.684 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.676+0000 7fad8d703700 1 --2- 172.21.15.162:0/1923149339 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad880b55a0 0x7fad880ad6c0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:28.684 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.676+0000 7fad8d703700 1 -- 172.21.15.162:0/1923149339 >> 172.21.15.162:0/1923149339 conn(0x7fad880626d0 msgr2=0x7fad88062ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:28.684 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.676+0000 7fad8d703700 1 -- 172.21.15.162:0/1923149339 shutdown_connections 2020-11-28T21:54:28.685 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.676+0000 7fad8d703700 1 -- 172.21.15.162:0/1923149339 wait complete. 2020-11-28T21:54:28.687 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.676+0000 7fad8d703700 1 Processor -- start 2020-11-28T21:54:28.687 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.676+0000 7fad8d703700 1 -- start start 2020-11-28T21:54:28.688 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.680+0000 7fad8d703700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad88080500 0x7fad880adb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:28.688 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.680+0000 7fad8d703700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fad880a86d0 0x7fad88152130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:28.688 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.680+0000 7fad8d703700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fad880b55a0 0x7fad88152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:28.688 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.680+0000 7fad8d703700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad88063db0 con 0x7fad880b55a0 2020-11-28T21:54:28.688 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.680+0000 7fad8d703700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad88137be0 con 0x7fad88080500 2020-11-28T21:54:28.689 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.680+0000 7fad8d703700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad88137e80 con 0x7fad880a86d0 2020-11-28T21:54:28.689 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.680+0000 7fad877fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad88080500 0x7fad880adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:28.689 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.680+0000 7fad877fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad88080500 0x7fad880adb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43458/0 (socket says 172.21.15.162:43458) 2020-11-28T21:54:28.689 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.680+0000 7fad877fe700 1 -- 172.21.15.162:0/838867787 learned_addr learned my addr 172.21.15.162:0/838867787 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:28.689 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.680+0000 7fad87fff700 1 --2- 172.21.15.162:0/838867787 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fad880b55a0 0x7fad88152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:28.690 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.680+0000 7fad86ffd700 1 --2- 172.21.15.162:0/838867787 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fad880a86d0 0x7fad88152130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:28.690 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.680+0000 7fad877fe700 1 -- 172.21.15.162:0/838867787 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fad880a86d0 msgr2=0x7fad88152130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:28.690 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.680+0000 7fad877fe700 1 --2- 172.21.15.162:0/838867787 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fad880a86d0 0x7fad88152130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:28.690 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.680+0000 7fad877fe700 1 -- 172.21.15.162:0/838867787 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fad880b55a0 msgr2=0x7fad88152620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:28.691 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.680+0000 7fad877fe700 1 --2- 172.21.15.162:0/838867787 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fad880b55a0 0x7fad88152620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:28.691 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.680+0000 7fad877fe700 1 -- 172.21.15.162:0/838867787 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad88138120 con 0x7fad88080500 2020-11-28T21:54:28.691 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.680+0000 7fad877fe700 1 --2- 172.21.15.162:0/838867787 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad88080500 0x7fad880adb20 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 rx=0x7fad78002f30 tx=0x7fad78015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:28.691 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.680+0000 7fad6ffff700 1 -- 172.21.15.162:0/838867787 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad78009c50 con 0x7fad88080500 2020-11-28T21:54:28.691 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.680+0000 7fad6ffff700 1 -- 172.21.15.162:0/838867787 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7801e460 con 0x7fad88080500 2020-11-28T21:54:28.692 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.680+0000 7fad6ffff700 1 -- 172.21.15.162:0/838867787 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad7801e7c0 con 0x7fad88080500 2020-11-28T21:54:28.692 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.680+0000 7fad8d703700 1 -- 172.21.15.162:0/838867787 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad881383c0 con 0x7fad88080500 2020-11-28T21:54:28.692 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.680+0000 7fad8d703700 1 -- 172.21.15.162:0/838867787 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad88152b10 con 0x7fad88080500 2020-11-28T21:54:28.692 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.680+0000 7fad8d703700 1 -- 172.21.15.162:0/838867787 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad88057050 con 0x7fad88080500 2020-11-28T21:54:28.692 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.684+0000 7fad6ffff700 1 -- 172.21.15.162:0/838867787 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fad78025030 con 0x7fad88080500 2020-11-28T21:54:28.693 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.684+0000 7fad6ffff700 1 --2- 172.21.15.162:0/838867787 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fad680553b0 0x7fad68057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:28.693 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.684+0000 7fad6ffff700 1 -- 172.21.15.162:0/838867787 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fad7806f340 con 0x7fad88080500 2020-11-28T21:54:28.693 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.684+0000 7fad86ffd700 1 --2- 172.21.15.162:0/838867787 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fad680553b0 0x7fad68057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:28.693 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.684+0000 7fad86ffd700 1 --2- 172.21.15.162:0/838867787 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fad680553b0 0x7fad68057870 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7fad7c00b000 tx=0x7fad7c009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:28.694 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.688+0000 7fad6ffff700 1 -- 172.21.15.162:0/838867787 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fad78045640 con 0x7fad88080500 2020-11-28T21:54:28.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.876+0000 7fad8d703700 1 -- 172.21.15.162:0/838867787 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fad8805d250 con 0x7fad88080500 2020-11-28T21:54:28.888 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.884+0000 7fad6ffff700 1 -- 172.21.15.162:0/838867787 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v25) v1 ==== 184+0+0 (secure 0 0 0) 0x7fad780230e0 con 0x7fad88080500 2020-11-28T21:54:28.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.888+0000 7fad8d703700 1 -- 172.21.15.162:0/838867787 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fad680553b0 msgr2=0x7fad68057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:28.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.888+0000 7fad8d703700 1 --2- 172.21.15.162:0/838867787 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fad680553b0 0x7fad68057870 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7fad7c00b000 tx=0x7fad7c009040).stop 2020-11-28T21:54:28.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.888+0000 7fad8d703700 1 -- 172.21.15.162:0/838867787 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad88080500 msgr2=0x7fad880adb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:28.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.888+0000 7fad8d703700 1 --2- 172.21.15.162:0/838867787 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad88080500 0x7fad880adb20 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 rx=0x7fad78002f30 tx=0x7fad78015000).stop 2020-11-28T21:54:28.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.888+0000 7fad8d703700 1 -- 172.21.15.162:0/838867787 shutdown_connections 2020-11-28T21:54:28.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.888+0000 7fad8d703700 1 --2- 172.21.15.162:0/838867787 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fad680553b0 0x7fad68057870 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:28.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.888+0000 7fad8d703700 1 --2- 172.21.15.162:0/838867787 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fad880b55a0 0x7fad88152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:28.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.888+0000 7fad8d703700 1 --2- 172.21.15.162:0/838867787 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad88080500 0x7fad880adb20 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:28.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.888+0000 7fad8d703700 1 --2- 172.21.15.162:0/838867787 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fad880a86d0 0x7fad88152130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:28.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.888+0000 7fad8d703700 1 -- 172.21.15.162:0/838867787 >> 172.21.15.162:0/838867787 conn(0x7fad880626d0 msgr2=0x7fad88128120 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:28.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.888+0000 7fad8d703700 1 -- 172.21.15.162:0/838867787 shutdown_connections 2020-11-28T21:54:28.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:28.888+0000 7fad8d703700 1 -- 172.21.15.162:0/838867787 wait complete. 2020-11-28T21:54:28.894 INFO:teuthology.orchestra.run.smithi162.stderr:updated caps for client.0 2020-11-28T21:54:28.903 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-11-28T21:54:29.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.048+0000 7f5f03b05700 1 Processor -- start 2020-11-28T21:54:29.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.048+0000 7f5f03b05700 1 -- start start 2020-11-28T21:54:29.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.048+0000 7f5f03b05700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5efc080500 0x7f5efc0addd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:29.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.048+0000 7f5f03b05700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5efc0a86d0 0x7f5efc153080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:29.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.048+0000 7f5f03b05700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5efc0b5980 con 0x7f5efc0b55a0 2020-11-28T21:54:29.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.048+0000 7f5f03b05700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5efc0afe80 con 0x7f5efc0a86d0 2020-11-28T21:54:29.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.048+0000 7f5f03b05700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc0809a0 con 0x7f5efc080500 2020-11-28T21:54:29.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.048+0000 7f5f020a2700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f5efc0b55a0 0x7f5efc0ad6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:50310/0 (socket says 172.21.15.162:50310) 2020-11-28T21:54:29.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.048+0000 7f5f020a2700 1 -- 172.21.15.162:0/3378416461 learned_addr learned my addr 172.21.15.162:0/3378416461 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:29.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.048+0000 7f5f018a1700 1 --2- 172.21.15.162:0/3378416461 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5efc080500 0x7f5efc0addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:29.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.048+0000 7f5f028a3700 1 --2- 172.21.15.162:0/3378416461 >> v2:172.21.15.118:3300/0 conn(0x7f5efc0a86d0 0x7f5efc153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:29.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.048+0000 7f5f010a0700 1 -- 172.21.15.162:0/3378416461 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2591475438 0 0) 0x7f5efc0b5980 con 0x7f5efc0b55a0 2020-11-28T21:54:29.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.048+0000 7f5f010a0700 1 -- 172.21.15.162:0/3378416461 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ee8003230 con 0x7f5efc0b55a0 2020-11-28T21:54:29.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.048+0000 7f5f028a3700 1 -- 172.21.15.162:0/3378416461 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5efc080500 msgr2=0x7f5efc0addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:29.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.048+0000 7f5f028a3700 1 --2- 172.21.15.162:0/3378416461 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5efc080500 0x7f5efc0addd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:29.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.048+0000 7f5f028a3700 1 -- 172.21.15.162:0/3378416461 >> v1:172.21.15.162:6789/0 conn(0x7f5efc0b55a0 legacy=0x7f5efc0ad6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:29.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.048+0000 7f5f010a0700 1 -- 172.21.15.162:0/3378416461 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 514820237 0 0) 0x7f5ee8003230 con 0x7f5efc0b55a0 2020-11-28T21:54:29.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.048+0000 7f5f028a3700 1 -- 172.21.15.162:0/3378416461 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5efc0af740 con 0x7f5efc0a86d0 2020-11-28T21:54:29.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.048+0000 7f5f028a3700 1 --2- 172.21.15.162:0/3378416461 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5efc0a86d0 0x7f5efc153080 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f5ef8001870 tx=0x7f5ef40091a0).ready entity=mon.0 client_cookie=1d8ea7cd8b1b4b85 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:29.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.048+0000 7f5f010a0700 1 -- 172.21.15.162:0/3378416461 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ef80016d0 con 0x7f5efc0a86d0 2020-11-28T21:54:29.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.048+0000 7f5f010a0700 1 -- 172.21.15.162:0/3378416461 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef400b800 con 0x7f5efc0a86d0 2020-11-28T21:54:29.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.048+0000 7f5f010a0700 1 -- 172.21.15.162:0/3378416461 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ef400b970 con 0x7f5efc0a86d0 2020-11-28T21:54:29.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.048+0000 7f5f03b05700 1 -- 172.21.15.162:0/3378416461 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5efc0a86d0 msgr2=0x7f5efc153080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:29.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.048+0000 7f5f03b05700 1 --2- 172.21.15.162:0/3378416461 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5efc0a86d0 0x7f5efc153080 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f5ef8001870 tx=0x7f5ef40091a0).stop 2020-11-28T21:54:29.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.052+0000 7f5f03b05700 1 -- 172.21.15.162:0/3378416461 shutdown_connections 2020-11-28T21:54:29.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.052+0000 7f5f03b05700 1 --2- 172.21.15.162:0/3378416461 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5efc0a86d0 0x7f5efc153080 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:29.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.052+0000 7f5f03b05700 1 --2- 172.21.15.162:0/3378416461 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5efc080500 0x7f5efc0addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:29.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.052+0000 7f5f03b05700 1 -- 172.21.15.162:0/3378416461 >> 172.21.15.162:0/3378416461 conn(0x7f5efc0626d0 msgr2=0x7f5efc062ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:29.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.052+0000 7f5f03b05700 1 -- 172.21.15.162:0/3378416461 shutdown_connections 2020-11-28T21:54:29.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.052+0000 7f5f03b05700 1 -- 172.21.15.162:0/3378416461 wait complete. 2020-11-28T21:54:29.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.052+0000 7f5f03b05700 1 Processor -- start 2020-11-28T21:54:29.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.052+0000 7f5f03b05700 1 -- start start 2020-11-28T21:54:29.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.052+0000 7f5f03b05700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5efc080500 0x7f5efc0b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:29.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.052+0000 7f5f03b05700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5efc0a86d0 0x7f5efc1500f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:29.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.052+0000 7f5f03b05700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5efc0b55a0 0x7f5efc1505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:29.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.052+0000 7f5f03b05700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc150ad0 con 0x7f5efc080500 2020-11-28T21:54:29.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.052+0000 7f5f03b05700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc065640 con 0x7f5efc0b55a0 2020-11-28T21:54:29.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.052+0000 7f5f03b05700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc065c30 con 0x7f5efc0a86d0 2020-11-28T21:54:29.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.056+0000 7f5f028a3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5efc0b55a0 0x7f5efc1505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:29.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.056+0000 7f5f028a3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5efc0b55a0 0x7f5efc1505e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43476/0 (socket says 172.21.15.162:43476) 2020-11-28T21:54:29.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.056+0000 7f5f028a3700 1 -- 172.21.15.162:0/453656714 learned_addr learned my addr 172.21.15.162:0/453656714 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:29.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.056+0000 7f5f018a1700 1 --2- 172.21.15.162:0/453656714 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5efc0a86d0 0x7f5efc1500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:29.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.056+0000 7f5f020a2700 1 --2- 172.21.15.162:0/453656714 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5efc080500 0x7f5efc0b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:29.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.056+0000 7f5f028a3700 1 -- 172.21.15.162:0/453656714 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5efc0a86d0 msgr2=0x7f5efc1500f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:29.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.056+0000 7f5f028a3700 1 --2- 172.21.15.162:0/453656714 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5efc0a86d0 0x7f5efc1500f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:29.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.056+0000 7f5f028a3700 1 -- 172.21.15.162:0/453656714 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5efc080500 msgr2=0x7f5efc0b3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:29.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.056+0000 7f5f028a3700 1 --2- 172.21.15.162:0/453656714 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5efc080500 0x7f5efc0b3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:29.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.056+0000 7f5f028a3700 1 -- 172.21.15.162:0/453656714 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5efc065ed0 con 0x7f5efc0b55a0 2020-11-28T21:54:29.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.056+0000 7f5f028a3700 1 --2- 172.21.15.162:0/453656714 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5efc0b55a0 0x7f5efc1505e0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 rx=0x7f5ef8001870 tx=0x7f5ef4016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:29.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.056+0000 7f5ef2ffd700 1 -- 172.21.15.162:0/453656714 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ef40119e0 con 0x7f5efc0b55a0 2020-11-28T21:54:29.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.056+0000 7f5ef2ffd700 1 -- 172.21.15.162:0/453656714 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef4011b40 con 0x7f5efc0b55a0 2020-11-28T21:54:29.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.056+0000 7f5f03b05700 1 -- 172.21.15.162:0/453656714 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5efc0645e0 con 0x7f5efc0b55a0 2020-11-28T21:54:29.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.056+0000 7f5ef2ffd700 1 -- 172.21.15.162:0/453656714 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ef4011ea0 con 0x7f5efc0b55a0 2020-11-28T21:54:29.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.056+0000 7f5f03b05700 1 -- 172.21.15.162:0/453656714 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5efc0676a0 con 0x7f5efc0b55a0 2020-11-28T21:54:29.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.056+0000 7f5ef2ffd700 1 -- 172.21.15.162:0/453656714 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5ef4026030 con 0x7f5efc0b55a0 2020-11-28T21:54:29.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.056+0000 7f5f03b05700 1 -- 172.21.15.162:0/453656714 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5efc0a2ad0 con 0x7f5efc0b55a0 2020-11-28T21:54:29.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.056+0000 7f5ef2ffd700 1 --2- 172.21.15.162:0/453656714 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5ee40554d0 0x7f5ee4057990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:29.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.056+0000 7f5ef2ffd700 1 -- 172.21.15.162:0/453656714 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5ef406e7b0 con 0x7f5efc0b55a0 2020-11-28T21:54:29.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.056+0000 7f5f020a2700 1 --2- 172.21.15.162:0/453656714 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5ee40554d0 0x7f5ee4057990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:29.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.060+0000 7f5f020a2700 1 --2- 172.21.15.162:0/453656714 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5ee40554d0 0x7f5ee4057990 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f5ef800d000 tx=0x7f5ef8000b80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:29.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.064+0000 7f5ef2ffd700 1 -- 172.21.15.162:0/453656714 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5ef4044b30 con 0x7f5efc0b55a0 2020-11-28T21:54:29.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.252+0000 7f5f03b05700 1 -- 172.21.15.162:0/453656714 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f5efc065840 con 0x7f5efc0b55a0 2020-11-28T21:54:29.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.260+0000 7f5ef2ffd700 1 -- 172.21.15.162:0/453656714 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v26) v1 ==== 184+0+0 (secure 0 0 0) 0x7f5ef40150e0 con 0x7f5efc0b55a0 2020-11-28T21:54:29.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.260+0000 7f5f03b05700 1 -- 172.21.15.162:0/453656714 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5ee40554d0 msgr2=0x7f5ee4057990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:29.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.260+0000 7f5f03b05700 1 --2- 172.21.15.162:0/453656714 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5ee40554d0 0x7f5ee4057990 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f5ef800d000 tx=0x7f5ef8000b80).stop 2020-11-28T21:54:29.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.260+0000 7f5f03b05700 1 -- 172.21.15.162:0/453656714 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5efc0b55a0 msgr2=0x7f5efc1505e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:29.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.260+0000 7f5f03b05700 1 --2- 172.21.15.162:0/453656714 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5efc0b55a0 0x7f5efc1505e0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 rx=0x7f5ef8001870 tx=0x7f5ef4016000).stop 2020-11-28T21:54:29.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.260+0000 7f5f03b05700 1 -- 172.21.15.162:0/453656714 shutdown_connections 2020-11-28T21:54:29.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.260+0000 7f5f03b05700 1 --2- 172.21.15.162:0/453656714 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5ee40554d0 0x7f5ee4057990 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:29.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.260+0000 7f5f03b05700 1 --2- 172.21.15.162:0/453656714 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5efc0b55a0 0x7f5efc1505e0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:29.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.260+0000 7f5f03b05700 1 --2- 172.21.15.162:0/453656714 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5efc080500 0x7f5efc0b3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:29.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.260+0000 7f5f03b05700 1 --2- 172.21.15.162:0/453656714 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5efc0a86d0 0x7f5efc1500f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:29.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.260+0000 7f5f03b05700 1 -- 172.21.15.162:0/453656714 >> 172.21.15.162:0/453656714 conn(0x7f5efc0626d0 msgr2=0x7f5efc0ad5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:29.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.260+0000 7f5f03b05700 1 -- 172.21.15.162:0/453656714 shutdown_connections 2020-11-28T21:54:29.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.260+0000 7f5f03b05700 1 -- 172.21.15.162:0/453656714 wait complete. 2020-11-28T21:54:29.268 INFO:teuthology.orchestra.run.smithi162.stderr:updated caps for client.1 2020-11-28T21:54:29.276 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:54:29.401 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.396+0000 7ffaee136700 1 Processor -- start 2020-11-28T21:54:29.402 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.396+0000 7ffaee136700 1 -- start start 2020-11-28T21:54:29.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.400+0000 7ffaee136700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ffae80af640 0x7ffae80aac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:29.404 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.400+0000 7ffaee136700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffae80b0230 con 0x7ffae805cbd0 2020-11-28T21:54:29.404 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.400+0000 7ffaee136700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffae80ab0f0 con 0x7ffae8060cf0 2020-11-28T21:54:29.404 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.400+0000 7ffaee136700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffae80b12a0 con 0x7ffae80af640 2020-11-28T21:54:29.404 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.400+0000 7ffaed134700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7ffae8060cf0 0x7ffae80aa4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:50326/0 (socket says 172.21.15.162:50326) 2020-11-28T21:54:29.405 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.400+0000 7ffaed134700 1 -- 172.21.15.162:0/647321616 learned_addr learned my addr 172.21.15.162:0/647321616 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:29.405 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.400+0000 7ffaed935700 1 --2- 172.21.15.162:0/647321616 >> v2:172.21.15.118:3300/0 conn(0x7ffae80af640 0x7ffae80aac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:29.405 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.400+0000 7ffadffff700 1 -- 172.21.15.162:0/647321616 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1189563716 0 0) 0x7ffae80ab0f0 con 0x7ffae8060cf0 2020-11-28T21:54:29.405 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.400+0000 7ffadffff700 1 -- 172.21.15.162:0/647321616 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffacc003230 con 0x7ffae8060cf0 2020-11-28T21:54:29.405 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.400+0000 7ffaed935700 1 -- 172.21.15.162:0/647321616 >> v1:172.21.15.162:6789/0 conn(0x7ffae8060cf0 legacy=0x7ffae80aa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:29.406 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.400+0000 7ffaed935700 1 -- 172.21.15.162:0/647321616 >> v1:172.21.15.118:6789/0 conn(0x7ffae805cbd0 legacy=0x7ffae80ae1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:29.406 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.400+0000 7ffadffff700 1 -- 172.21.15.162:0/647321616 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3684516970 0 0) 0x7ffae80b0230 con 0x7ffae805cbd0 2020-11-28T21:54:29.406 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.400+0000 7ffaed935700 1 -- 172.21.15.162:0/647321616 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae80a8c00 con 0x7ffae80af640 2020-11-28T21:54:29.406 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.400+0000 7ffaed935700 1 --2- 172.21.15.162:0/647321616 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffae80af640 0x7ffae80aac00 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7ffae00032b0 tx=0x7ffae00091a0).ready entity=mon.0 client_cookie=688fc2c44a3a4c27 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:29.409 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.400+0000 7ffadffff700 1 -- 172.21.15.162:0/647321616 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffae000b9a0 con 0x7ffae80af640 2020-11-28T21:54:29.409 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.400+0000 7ffadffff700 1 -- 172.21.15.162:0/647321616 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae000bb00 con 0x7ffae80af640 2020-11-28T21:54:29.409 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.400+0000 7ffadffff700 1 -- 172.21.15.162:0/647321616 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffae000bc70 con 0x7ffae80af640 2020-11-28T21:54:29.410 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.400+0000 7ffaee136700 1 -- 172.21.15.162:0/647321616 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffae80af640 msgr2=0x7ffae80aac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:29.410 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.400+0000 7ffaee136700 1 --2- 172.21.15.162:0/647321616 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffae80af640 0x7ffae80aac00 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7ffae00032b0 tx=0x7ffae00091a0).stop 2020-11-28T21:54:29.410 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.400+0000 7ffaee136700 1 -- 172.21.15.162:0/647321616 shutdown_connections 2020-11-28T21:54:29.410 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.400+0000 7ffaee136700 1 --2- 172.21.15.162:0/647321616 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffae80af640 0x7ffae80aac00 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:29.411 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.400+0000 7ffaee136700 1 -- 172.21.15.162:0/647321616 >> 172.21.15.162:0/647321616 conn(0x7ffae805b970 msgr2=0x7ffae805bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:29.411 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.400+0000 7ffaee136700 1 -- 172.21.15.162:0/647321616 shutdown_connections 2020-11-28T21:54:29.411 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.400+0000 7ffaee136700 1 -- 172.21.15.162:0/647321616 wait complete. 2020-11-28T21:54:29.411 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.404+0000 7ffaee136700 1 Processor -- start 2020-11-28T21:54:29.412 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.404+0000 7ffaee136700 1 -- start start 2020-11-28T21:54:29.412 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.404+0000 7ffaee136700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffae805cbd0 0x7ffae80abf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:29.412 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.404+0000 7ffaee136700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffae8060cf0 0x7ffae808fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:29.412 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.404+0000 7ffaee136700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffae80af640 0x7ffae8090230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:29.413 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.404+0000 7ffaee136700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae8066f40 con 0x7ffae8060cf0 2020-11-28T21:54:29.413 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.404+0000 7ffaee136700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae8090720 con 0x7ffae805cbd0 2020-11-28T21:54:29.413 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.404+0000 7ffaee136700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffae8091fd0 con 0x7ffae80af640 2020-11-28T21:54:29.413 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.404+0000 7ffaed134700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffae805cbd0 0x7ffae80abf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:29.413 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.404+0000 7ffaed134700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffae805cbd0 0x7ffae80abf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43486/0 (socket says 172.21.15.162:43486) 2020-11-28T21:54:29.414 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.404+0000 7ffaed134700 1 -- 172.21.15.162:0/1380864343 learned_addr learned my addr 172.21.15.162:0/1380864343 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:29.414 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.404+0000 7ffaed935700 1 --2- 172.21.15.162:0/1380864343 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffae80af640 0x7ffae8090230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:29.414 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.404+0000 7ffaed134700 1 -- 172.21.15.162:0/1380864343 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffae80af640 msgr2=0x7ffae8090230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:29.414 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.404+0000 7ffaec933700 1 --2- 172.21.15.162:0/1380864343 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffae8060cf0 0x7ffae808fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:29.414 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.404+0000 7ffaed134700 1 --2- 172.21.15.162:0/1380864343 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffae80af640 0x7ffae8090230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:29.415 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.404+0000 7ffaed134700 1 -- 172.21.15.162:0/1380864343 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffae8060cf0 msgr2=0x7ffae808fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:29.415 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.404+0000 7ffaed134700 1 --2- 172.21.15.162:0/1380864343 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffae8060cf0 0x7ffae808fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:29.415 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.404+0000 7ffaed134700 1 -- 172.21.15.162:0/1380864343 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae8092270 con 0x7ffae805cbd0 2020-11-28T21:54:29.415 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.404+0000 7ffaec933700 1 --2- 172.21.15.162:0/1380864343 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffae8060cf0 0x7ffae808fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:54:29.415 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.404+0000 7ffaed935700 1 --2- 172.21.15.162:0/1380864343 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffae80af640 0x7ffae8090230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:54:29.416 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.404+0000 7ffaed134700 1 --2- 172.21.15.162:0/1380864343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffae805cbd0 0x7ffae80abf70 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 rx=0x7ffae8067080 tx=0x7ffae4007a70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:29.416 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.404+0000 7ffade7fc700 1 -- 172.21.15.162:0/1380864343 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffae40072e0 con 0x7ffae805cbd0 2020-11-28T21:54:29.416 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.404+0000 7ffade7fc700 1 -- 172.21.15.162:0/1380864343 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae4007440 con 0x7ffae805cbd0 2020-11-28T21:54:29.416 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.404+0000 7ffaee136700 1 -- 172.21.15.162:0/1380864343 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffae8092510 con 0x7ffae805cbd0 2020-11-28T21:54:29.416 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.404+0000 7ffade7fc700 1 -- 172.21.15.162:0/1380864343 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffae40057d0 con 0x7ffae805cbd0 2020-11-28T21:54:29.417 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.404+0000 7ffaee136700 1 -- 172.21.15.162:0/1380864343 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffae8066010 con 0x7ffae805cbd0 2020-11-28T21:54:29.418 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.404+0000 7ffaee136700 1 -- 172.21.15.162:0/1380864343 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffae8057050 con 0x7ffae805cbd0 2020-11-28T21:54:29.418 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.408+0000 7ffade7fc700 1 -- 172.21.15.162:0/1380864343 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ffae4010070 con 0x7ffae805cbd0 2020-11-28T21:54:29.418 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.408+0000 7ffade7fc700 1 --2- 172.21.15.162:0/1380864343 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ffad00553b0 0x7ffad0057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:29.418 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.408+0000 7ffade7fc700 1 -- 172.21.15.162:0/1380864343 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ffae406d750 con 0x7ffae805cbd0 2020-11-28T21:54:29.419 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.408+0000 7ffaec933700 1 --2- 172.21.15.162:0/1380864343 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ffad00553b0 0x7ffad0057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:29.419 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.408+0000 7ffaec933700 1 --2- 172.21.15.162:0/1380864343 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ffad00553b0 0x7ffad0057870 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7ffad8008000 tx=0x7ffad8009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:29.419 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.412+0000 7ffade7fc700 1 -- 172.21.15.162:0/1380864343 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ffae4043950 con 0x7ffae805cbd0 2020-11-28T21:54:29.600 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.596+0000 7ffaee136700 1 -- 172.21.15.162:0/1380864343 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffae80b0010 con 0x7ffae805cbd0 2020-11-28T21:54:29.600 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.596+0000 7ffade7fc700 1 -- 172.21.15.162:0/1380864343 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+2499 (secure 0 0 0) 0x7ffae4018070 con 0x7ffae805cbd0 2020-11-28T21:54:29.601 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:54:29.601 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":51,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6182,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2775691244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2775691244},{"type":"v1","addr":"172.21.15.162:6835","nonce":2775691244}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":49},{"gid":6942,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/1896314020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1896314020},{"type":"v1","addr":"172.21.15.118:6833","nonce":1896314020}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":49}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:54:26.908249+0000","modified":"2020-11-28T21:54:27.911138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7177},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7177":{"gid":7177,"name":"a","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/493776989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":493776989},{"type":"v1","addr":"172.21.15.162:6837","nonce":493776989}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2020-11-28T21:54:29.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.600+0000 7ffaee136700 1 -- 172.21.15.162:0/1380864343 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ffad00553b0 msgr2=0x7ffad0057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:29.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.600+0000 7ffaee136700 1 --2- 172.21.15.162:0/1380864343 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ffad00553b0 0x7ffad0057870 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7ffad8008000 tx=0x7ffad8009040).stop 2020-11-28T21:54:29.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.600+0000 7ffaee136700 1 -- 172.21.15.162:0/1380864343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffae805cbd0 msgr2=0x7ffae80abf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:29.603 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.600+0000 7ffaee136700 1 --2- 172.21.15.162:0/1380864343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffae805cbd0 0x7ffae80abf70 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 rx=0x7ffae8067080 tx=0x7ffae4007a70).stop 2020-11-28T21:54:29.603 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.600+0000 7ffaee136700 1 -- 172.21.15.162:0/1380864343 shutdown_connections 2020-11-28T21:54:29.603 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.600+0000 7ffaee136700 1 --2- 172.21.15.162:0/1380864343 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ffad00553b0 0x7ffad0057870 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:29.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.600+0000 7ffaee136700 1 --2- 172.21.15.162:0/1380864343 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffae80af640 0x7ffae8090230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:29.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.600+0000 7ffaee136700 1 --2- 172.21.15.162:0/1380864343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffae805cbd0 0x7ffae80abf70 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:29.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.600+0000 7ffaee136700 1 --2- 172.21.15.162:0/1380864343 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffae8060cf0 0x7ffae808fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:29.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.600+0000 7ffaee136700 1 -- 172.21.15.162:0/1380864343 >> 172.21.15.162:0/1380864343 conn(0x7ffae805b970 msgr2=0x7ffae80b4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:29.605 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.600+0000 7ffaee136700 1 -- 172.21.15.162:0/1380864343 shutdown_connections 2020-11-28T21:54:29.605 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:29.600+0000 7ffaee136700 1 -- 172.21.15.162:0/1380864343 wait complete. 2020-11-28T21:54:29.605 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 51 2020-11-28T21:54:29.918 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 51, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-28T21:54:26.908249+0000', 'modified': '2020-11-28T21:54:27.911138+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7177}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7177': {'gid': 7177, 'name': 'a', 'rank': 0, 'incarnation': 50, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6837/493776989', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6836', 'nonce': 493776989}, {'type': 'v1', 'addr': '172.21.15.162:6837', 'nonce': 493776989}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-28T21:54:29.919 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-28T21:54:29.920 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-28T21:54:30.051 INFO:teuthology.orchestra.run.smithi162.stdout:{"cluster_fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","whoami":0,"id":7177,"want_state":"up:active","state":"up:active","rank_uptime":3.1281223140000001,"mdsmap_epoch":51,"osdmap_epoch":109,"osdmap_epoch_barrier":108,"uptime":7.5045530390000001} 2020-11-28T21:54:30.052 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", "id": 7177, "mdsmap_epoch": 51, "osdmap_epoch": 109, "osdmap_epoch_barrier": 108, "rank_uptime": 3.128122314, "state": "up:active", "uptime": 7.504553039, "want_state": "up:active", "whoami": 0 } 2020-11-28T21:54:30.053 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:54:30.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.160+0000 7f3262e48700 1 Processor -- start 2020-11-28T21:54:30.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.160+0000 7f3262e48700 1 -- start start 2020-11-28T21:54:30.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.160+0000 7f3262e48700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f325c1443f0 0x7f325c05f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:30.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.160+0000 7f3262e48700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f325c0a85c0 con 0x7f325c0ab990 2020-11-28T21:54:30.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.160+0000 7f3262e48700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f325c1448e0 con 0x7f325c0aae10 2020-11-28T21:54:30.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.160+0000 7f3262e48700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f325c144b80 con 0x7f325c1443f0 2020-11-28T21:54:30.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.160+0000 7f3261e46700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f325c0aae10 0x7f325c140800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:50336/0 (socket says 172.21.15.162:50336) 2020-11-28T21:54:30.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.160+0000 7f3261e46700 1 -- 172.21.15.162:0/3735303246 learned_addr learned my addr 172.21.15.162:0/3735303246 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:30.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.160+0000 7f3262647700 1 --2- 172.21.15.162:0/3735303246 >> v2:172.21.15.118:3300/0 conn(0x7f325c1443f0 0x7f325c05f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:30.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.160+0000 7f3260e44700 1 -- 172.21.15.162:0/3735303246 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 888916749 0 0) 0x7f325c1448e0 con 0x7f325c0aae10 2020-11-28T21:54:30.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.160+0000 7f3260e44700 1 -- 172.21.15.162:0/3735303246 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3244003230 con 0x7f325c0aae10 2020-11-28T21:54:30.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.160+0000 7f3262647700 1 -- 172.21.15.162:0/3735303246 >> v1:172.21.15.162:6789/0 conn(0x7f325c0aae10 legacy=0x7f325c140800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:30.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.160+0000 7f3262647700 1 -- 172.21.15.162:0/3735303246 >> v1:172.21.15.118:6789/0 conn(0x7f325c0ab990 legacy=0x7f325c143ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:30.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.160+0000 7f3262647700 1 -- 172.21.15.162:0/3735303246 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f325c0af6b0 con 0x7f325c1443f0 2020-11-28T21:54:30.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.164+0000 7f3262647700 1 --2- 172.21.15.162:0/3735303246 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f325c1443f0 0x7f325c05f820 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f3254002f30 tx=0x7f3254006dc0).ready entity=mon.0 client_cookie=6a8cadfb49b37117 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:30.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.164+0000 7f3260e44700 1 -- 172.21.15.162:0/3735303246 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3254009800 con 0x7f325c1443f0 2020-11-28T21:54:30.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.164+0000 7f3260e44700 1 -- 172.21.15.162:0/3735303246 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3254009960 con 0x7f325c1443f0 2020-11-28T21:54:30.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.164+0000 7f3260e44700 1 -- 172.21.15.162:0/3735303246 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3254009cc0 con 0x7f325c1443f0 2020-11-28T21:54:30.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.164+0000 7f3262e48700 1 -- 172.21.15.162:0/3735303246 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f325c1443f0 msgr2=0x7f325c05f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:30.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.164+0000 7f3262e48700 1 --2- 172.21.15.162:0/3735303246 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f325c1443f0 0x7f325c05f820 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f3254002f30 tx=0x7f3254006dc0).stop 2020-11-28T21:54:30.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.164+0000 7f3262e48700 1 -- 172.21.15.162:0/3735303246 shutdown_connections 2020-11-28T21:54:30.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.164+0000 7f3262e48700 1 --2- 172.21.15.162:0/3735303246 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f325c1443f0 0x7f325c05f820 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:30.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.164+0000 7f3262e48700 1 -- 172.21.15.162:0/3735303246 >> 172.21.15.162:0/3735303246 conn(0x7f325c05b970 msgr2=0x7f325c05bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:30.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.164+0000 7f3262e48700 1 -- 172.21.15.162:0/3735303246 shutdown_connections 2020-11-28T21:54:30.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.164+0000 7f3262e48700 1 -- 172.21.15.162:0/3735303246 wait complete. 2020-11-28T21:54:30.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.164+0000 7f3262e48700 1 Processor -- start 2020-11-28T21:54:30.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.164+0000 7f3262e48700 1 -- start start 2020-11-28T21:54:30.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.164+0000 7f3262e48700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f325c0aae10 0x7f325c145110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:30.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.164+0000 7f3262e48700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f325c0ab990 0x7f325c151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:30.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.164+0000 7f3262e48700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f325c1443f0 0x7f325c152040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:30.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.164+0000 7f3262e48700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325c09a090 con 0x7f325c0aae10 2020-11-28T21:54:30.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.164+0000 7f3262e48700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325c099df0 con 0x7f325c0ab990 2020-11-28T21:54:30.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.164+0000 7f3262e48700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f325c1516b0 con 0x7f325c1443f0 2020-11-28T21:54:30.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.164+0000 7f3261645700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f325c0ab990 0x7f325c151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:30.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.164+0000 7f3261645700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f325c0ab990 0x7f325c151b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43500/0 (socket says 172.21.15.162:43500) 2020-11-28T21:54:30.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.164+0000 7f3261645700 1 -- 172.21.15.162:0/812446572 learned_addr learned my addr 172.21.15.162:0/812446572 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:30.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.164+0000 7f3261e46700 1 --2- 172.21.15.162:0/812446572 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f325c0aae10 0x7f325c145110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:30.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.164+0000 7f3262647700 1 --2- 172.21.15.162:0/812446572 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f325c1443f0 0x7f325c152040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:30.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.168+0000 7f3261645700 1 -- 172.21.15.162:0/812446572 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f325c1443f0 msgr2=0x7f325c152040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:30.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.168+0000 7f3261645700 1 --2- 172.21.15.162:0/812446572 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f325c1443f0 0x7f325c152040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:30.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.168+0000 7f3261645700 1 -- 172.21.15.162:0/812446572 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f325c0aae10 msgr2=0x7f325c145110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:30.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.168+0000 7f3261645700 1 --2- 172.21.15.162:0/812446572 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f325c0aae10 0x7f325c145110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:30.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.168+0000 7f3261645700 1 -- 172.21.15.162:0/812446572 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f325c141550 con 0x7f325c0ab990 2020-11-28T21:54:30.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.168+0000 7f3261e46700 1 --2- 172.21.15.162:0/812446572 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f325c0aae10 0x7f325c145110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:54:30.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.168+0000 7f3262647700 1 --2- 172.21.15.162:0/812446572 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f325c1443f0 0x7f325c152040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:54:30.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.168+0000 7f3261645700 1 --2- 172.21.15.162:0/812446572 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f325c0ab990 0x7f325c151b50 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 rx=0x7f325c0b0ac0 tx=0x7f3258001460).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:30.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.168+0000 7f324affd700 1 -- 172.21.15.162:0/812446572 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f325800a7b0 con 0x7f325c0ab990 2020-11-28T21:54:30.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.168+0000 7f324affd700 1 -- 172.21.15.162:0/812446572 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f325800a910 con 0x7f325c0ab990 2020-11-28T21:54:30.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.168+0000 7f3262e48700 1 -- 172.21.15.162:0/812446572 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f325c0ae650 con 0x7f325c0ab990 2020-11-28T21:54:30.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.168+0000 7f324affd700 1 -- 172.21.15.162:0/812446572 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f325800ac70 con 0x7f325c0ab990 2020-11-28T21:54:30.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.168+0000 7f3262e48700 1 -- 172.21.15.162:0/812446572 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f325c0aefa0 con 0x7f325c0ab990 2020-11-28T21:54:30.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.168+0000 7f3262e48700 1 -- 172.21.15.162:0/812446572 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f325c099c50 con 0x7f325c0ab990 2020-11-28T21:54:30.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.168+0000 7f324affd700 1 -- 172.21.15.162:0/812446572 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f3258019030 con 0x7f325c0ab990 2020-11-28T21:54:30.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.172+0000 7f324affd700 1 --2- 172.21.15.162:0/812446572 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f323c055330 0x7f323c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:30.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.172+0000 7f324affd700 1 -- 172.21.15.162:0/812446572 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f325806d950 con 0x7f325c0ab990 2020-11-28T21:54:30.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.172+0000 7f3261e46700 1 --2- 172.21.15.162:0/812446572 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f323c055330 0x7f323c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:30.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.172+0000 7f3261e46700 1 --2- 172.21.15.162:0/812446572 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f323c055330 0x7f323c0577f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7f324c008000 tx=0x7f324c009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:30.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.176+0000 7f324affd700 1 -- 172.21.15.162:0/812446572 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f325c099c50 con 0x7f325c0ab990 2020-11-28T21:54:30.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.316+0000 7f3262e48700 1 -- 172.21.15.162:0/812446572 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f325c099c50 con 0x7f325c0ab990 2020-11-28T21:54:30.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.320+0000 7f324affd700 1 -- 172.21.15.162:0/812446572 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+16148 (secure 0 0 0) 0x7f325c099c50 con 0x7f325c0ab990 2020-11-28T21:54:30.322 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:54:30.322 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":109,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:54:27.911877+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:54:25.418443+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2020-11-28T21:54:25.913773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:54:30.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.320+0000 7f3262e48700 1 -- 172.21.15.162:0/812446572 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f323c055330 msgr2=0x7f323c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:30.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.320+0000 7f3262e48700 1 --2- 172.21.15.162:0/812446572 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f323c055330 0x7f323c0577f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7f324c008000 tx=0x7f324c009040).stop 2020-11-28T21:54:30.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.320+0000 7f3262e48700 1 -- 172.21.15.162:0/812446572 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f325c0ab990 msgr2=0x7f325c151b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:30.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.320+0000 7f3262e48700 1 --2- 172.21.15.162:0/812446572 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f325c0ab990 0x7f325c151b50 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 rx=0x7f325c0b0ac0 tx=0x7f3258001460).stop 2020-11-28T21:54:30.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.320+0000 7f3262e48700 1 -- 172.21.15.162:0/812446572 shutdown_connections 2020-11-28T21:54:30.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.320+0000 7f3262e48700 1 --2- 172.21.15.162:0/812446572 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f323c055330 0x7f323c0577f0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:30.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.320+0000 7f3262e48700 1 --2- 172.21.15.162:0/812446572 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f325c1443f0 0x7f325c152040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:30.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.320+0000 7f3262e48700 1 --2- 172.21.15.162:0/812446572 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f325c0aae10 0x7f325c145110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:30.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.320+0000 7f3262e48700 1 --2- 172.21.15.162:0/812446572 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f325c0ab990 0x7f325c151b50 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:30.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.320+0000 7f3262e48700 1 -- 172.21.15.162:0/812446572 >> 172.21.15.162:0/812446572 conn(0x7f325c05b970 msgr2=0x7f325c0b36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:30.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.320+0000 7f3262e48700 1 -- 172.21.15.162:0/812446572 shutdown_connections 2020-11-28T21:54:30.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.320+0000 7f3262e48700 1 -- 172.21.15.162:0/812446572 wait complete. 2020-11-28T21:54:30.336 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-28T21:54:25.418443+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2020-11-28T21:54:25.913773+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-28T21:54:30.337 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:54:30.456 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.452+0000 7f788759e700 1 Processor -- start 2020-11-28T21:54:30.457 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.452+0000 7f788759e700 1 -- start start 2020-11-28T21:54:30.458 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.456+0000 7f788759e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f788813d0c0 0x7f78880b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:30.459 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.456+0000 7f788759e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f78880b13c0 0x7f7888099b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:30.459 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.456+0000 7f788759e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f788806aca0 con 0x7f788809a050 2020-11-28T21:54:30.459 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.456+0000 7f788759e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f788813d4a0 con 0x7f788813d0c0 2020-11-28T21:54:30.459 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.456+0000 7f788759e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7888098da0 con 0x7f78880b13c0 2020-11-28T21:54:30.460 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.456+0000 7f7886d9d700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f788809a050 0x7f788805bf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:50354/0 (socket says 172.21.15.162:50354) 2020-11-28T21:54:30.460 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.456+0000 7f7886d9d700 1 -- 172.21.15.162:0/2701757662 learned_addr learned my addr 172.21.15.162:0/2701757662 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:30.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.456+0000 7f788559a700 1 -- 172.21.15.162:0/2701757662 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1870106445 0 0) 0x7f788806aca0 con 0x7f788809a050 2020-11-28T21:54:30.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.456+0000 7f788659c700 1 --2- 172.21.15.162:0/2701757662 >> v2:172.21.15.118:3300/0 conn(0x7f788813d0c0 0x7f78880b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:30.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.456+0000 7f7885d9b700 1 --2- 172.21.15.162:0/2701757662 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f78880b13c0 0x7f7888099b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:30.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.456+0000 7f788559a700 1 -- 172.21.15.162:0/2701757662 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f786c003230 con 0x7f788809a050 2020-11-28T21:54:30.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.456+0000 7f788559a700 1 -- 172.21.15.162:0/2701757662 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 963934757 0 0) 0x7f786c003230 con 0x7f788809a050 2020-11-28T21:54:30.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.456+0000 7f788559a700 1 -- 172.21.15.162:0/2701757662 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f78880b13c0 msgr2=0x7f7888099b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:30.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.456+0000 7f788559a700 1 --2- 172.21.15.162:0/2701757662 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f78880b13c0 0x7f7888099b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:30.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.456+0000 7f788559a700 1 -- 172.21.15.162:0/2701757662 >> v2:172.21.15.118:3300/0 conn(0x7f788813d0c0 msgr2=0x7f78880b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:30.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.456+0000 7f788559a700 1 --2- 172.21.15.162:0/2701757662 >> v2:172.21.15.118:3300/0 conn(0x7f788813d0c0 0x7f78880b0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:30.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.456+0000 7f788559a700 1 -- 172.21.15.162:0/2701757662 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f788806af40 con 0x7f788809a050 2020-11-28T21:54:30.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.456+0000 7f788559a700 1 -- 172.21.15.162:0/2701757662 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f7880001a50 con 0x7f788809a050 2020-11-28T21:54:30.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.456+0000 7f788559a700 1 -- 172.21.15.162:0/2701757662 >> v1:172.21.15.162:6789/0 conn(0x7f788809a050 legacy=0x7f788805bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:30.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.456+0000 7f788559a700 1 --2- 172.21.15.162:0/2701757662 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f786c0037a0 0x7f786c003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:30.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.456+0000 7f788559a700 1 --2- 172.21.15.162:0/2701757662 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f786c0044d0 0x7f786c004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:30.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.456+0000 7f788559a700 1 --2- 172.21.15.162:0/2701757662 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f78880b13c0 0x7f786c004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:30.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.456+0000 7f788559a700 1 -- 172.21.15.162:0/2701757662 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788813d4a0 con 0x7f786c0037a0 2020-11-28T21:54:30.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.456+0000 7f788559a700 1 -- 172.21.15.162:0/2701757662 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7888098da0 con 0x7f786c0044d0 2020-11-28T21:54:30.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.456+0000 7f788559a700 1 -- 172.21.15.162:0/2701757662 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f786c005330 con 0x7f78880b13c0 2020-11-28T21:54:30.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.456+0000 7f788659c700 1 --2- 172.21.15.162:0/2701757662 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f786c0037a0 0x7f786c003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:30.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.456+0000 7f7885d9b700 1 --2- 172.21.15.162:0/2701757662 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f786c0044d0 0x7f786c004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:30.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.456+0000 7f7886d9d700 1 --2- 172.21.15.162:0/2701757662 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f78880b13c0 0x7f786c004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:30.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.456+0000 7f788659c700 1 -- 172.21.15.162:0/2701757662 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f78880b13c0 msgr2=0x7f786c004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:30.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.456+0000 7f788659c700 1 --2- 172.21.15.162:0/2701757662 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f78880b13c0 0x7f786c004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:30.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.456+0000 7f788659c700 1 -- 172.21.15.162:0/2701757662 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f786c0044d0 msgr2=0x7f786c004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:30.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.456+0000 7f788659c700 1 --2- 172.21.15.162:0/2701757662 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f786c0044d0 0x7f786c004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:30.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.456+0000 7f788659c700 1 -- 172.21.15.162:0/2701757662 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786c003230 con 0x7f786c0037a0 2020-11-28T21:54:30.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.456+0000 7f7885d9b700 1 --2- 172.21.15.162:0/2701757662 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f786c0044d0 0x7f786c004950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:54:30.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.456+0000 7f788659c700 1 --2- 172.21.15.162:0/2701757662 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f786c0037a0 0x7f786c003bc0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f787800c000 tx=0x7f7878008040).ready entity=mon.0 client_cookie=81984c5465175cc1 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:30.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.456+0000 7f788559a700 1 -- 172.21.15.162:0/2701757662 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f787800e970 con 0x7f786c0037a0 2020-11-28T21:54:30.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.456+0000 7f788559a700 1 -- 172.21.15.162:0/2701757662 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f787800ead0 con 0x7f786c0037a0 2020-11-28T21:54:30.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.456+0000 7f788559a700 1 -- 172.21.15.162:0/2701757662 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f787800ee30 con 0x7f786c0037a0 2020-11-28T21:54:30.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.460+0000 7f788759e700 1 -- 172.21.15.162:0/2701757662 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f786c0037a0 msgr2=0x7f786c003bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:30.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.460+0000 7f788759e700 1 --2- 172.21.15.162:0/2701757662 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f786c0037a0 0x7f786c003bc0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f787800c000 tx=0x7f7878008040).stop 2020-11-28T21:54:30.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.460+0000 7f7886d9d700 1 -- 172.21.15.162:0/2701757662 reap_dead start 2020-11-28T21:54:30.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.460+0000 7f788759e700 1 -- 172.21.15.162:0/2701757662 shutdown_connections 2020-11-28T21:54:30.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.460+0000 7f788759e700 1 -- 172.21.15.162:0/2701757662 >> 172.21.15.162:0/2701757662 conn(0x7f78880b5520 msgr2=0x7f78880b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:30.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.460+0000 7f788759e700 1 -- 172.21.15.162:0/2701757662 shutdown_connections 2020-11-28T21:54:30.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.460+0000 7f788759e700 1 -- 172.21.15.162:0/2701757662 wait complete. 2020-11-28T21:54:30.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.460+0000 7f788759e700 1 Processor -- start 2020-11-28T21:54:30.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.460+0000 7f788759e700 1 -- start start 2020-11-28T21:54:30.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.464+0000 7f788759e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78880aba40 0x7f78880a9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:30.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.464+0000 7f788759e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f78880670c0 0x7f78880a9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:30.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.464+0000 7f788759e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7888067750 0x7f788814e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:30.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.464+0000 7f788759e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7888066440 con 0x7f7888067750 2020-11-28T21:54:30.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.464+0000 7f788759e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78880666e0 con 0x7f78880aba40 2020-11-28T21:54:30.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.464+0000 7f788759e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7888066980 con 0x7f78880670c0 2020-11-28T21:54:30.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.464+0000 7f788659c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78880aba40 0x7f78880a9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:30.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.464+0000 7f788659c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78880aba40 0x7f78880a9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43518/0 (socket says 172.21.15.162:43518) 2020-11-28T21:54:30.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.464+0000 7f788659c700 1 -- 172.21.15.162:0/760917558 learned_addr learned my addr 172.21.15.162:0/760917558 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:30.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.464+0000 7f7885d9b700 1 --2- 172.21.15.162:0/760917558 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f78880670c0 0x7f78880a9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:30.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.464+0000 7f7886d9d700 1 --2- 172.21.15.162:0/760917558 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7888067750 0x7f788814e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:30.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.464+0000 7f788659c700 1 -- 172.21.15.162:0/760917558 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f78880670c0 msgr2=0x7f78880a9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:30.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.464+0000 7f788659c700 1 --2- 172.21.15.162:0/760917558 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f78880670c0 0x7f78880a9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:30.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.464+0000 7f788659c700 1 -- 172.21.15.162:0/760917558 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7888067750 msgr2=0x7f788814e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:30.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.464+0000 7f788659c700 1 --2- 172.21.15.162:0/760917558 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7888067750 0x7f788814e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:30.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.464+0000 7f788659c700 1 -- 172.21.15.162:0/760917558 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7888066c20 con 0x7f78880aba40 2020-11-28T21:54:30.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.464+0000 7f788659c700 1 --2- 172.21.15.162:0/760917558 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78880aba40 0x7f78880a9330 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 rx=0x7f787800c000 tx=0x7f787801a000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:30.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.464+0000 7f78777fe700 1 -- 172.21.15.162:0/760917558 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f787801cd90 con 0x7f78880aba40 2020-11-28T21:54:30.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.464+0000 7f78777fe700 1 -- 172.21.15.162:0/760917558 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7878022400 con 0x7f78880aba40 2020-11-28T21:54:30.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.464+0000 7f78777fe700 1 -- 172.21.15.162:0/760917558 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7878022760 con 0x7f78880aba40 2020-11-28T21:54:30.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.464+0000 7f788759e700 1 -- 172.21.15.162:0/760917558 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78880ac2b0 con 0x7f78880aba40 2020-11-28T21:54:30.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.464+0000 7f788759e700 1 -- 172.21.15.162:0/760917558 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f788814e760 con 0x7f78880aba40 2020-11-28T21:54:30.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.468+0000 7f788759e700 1 -- 172.21.15.162:0/760917558 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7888066db0 con 0x7f78880aba40 2020-11-28T21:54:30.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.468+0000 7f78777fe700 1 -- 172.21.15.162:0/760917558 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f78780228c0 con 0x7f78880aba40 2020-11-28T21:54:30.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.468+0000 7f78777fe700 1 --2- 172.21.15.162:0/760917558 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f78640553b0 0x7f7864057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:30.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.468+0000 7f78777fe700 1 -- 172.21.15.162:0/760917558 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f78780715c0 con 0x7f78880aba40 2020-11-28T21:54:30.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.468+0000 7f7885d9b700 1 --2- 172.21.15.162:0/760917558 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f78640553b0 0x7f7864057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:30.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.468+0000 7f7885d9b700 1 --2- 172.21.15.162:0/760917558 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f78640553b0 0x7f7864057870 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f787c008000 tx=0x7f787c008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:30.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.476+0000 7f78777fe700 1 -- 172.21.15.162:0/760917558 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f78780477b0 con 0x7f78880aba40 2020-11-28T21:54:30.621 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.616+0000 7f788759e700 1 -- 172.21.15.162:0/760917558 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f7888057050 con 0x7f78880aba40 2020-11-28T21:54:30.621 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.616+0000 7f78777fe700 1 -- 172.21.15.162:0/760917558 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v109) v1 ==== 110+0+10 (secure 0 0 0) 0x7f7878016020 con 0x7f78880aba40 2020-11-28T21:54:30.622 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:54:30.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.620+0000 7f788759e700 1 -- 172.21.15.162:0/760917558 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f78640553b0 msgr2=0x7f7864057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:30.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.620+0000 7f788759e700 1 --2- 172.21.15.162:0/760917558 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f78640553b0 0x7f7864057870 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f787c008000 tx=0x7f787c008040).stop 2020-11-28T21:54:30.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.620+0000 7f788759e700 1 -- 172.21.15.162:0/760917558 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78880aba40 msgr2=0x7f78880a9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:30.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.620+0000 7f788759e700 1 --2- 172.21.15.162:0/760917558 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78880aba40 0x7f78880a9330 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 rx=0x7f787800c000 tx=0x7f787801a000).stop 2020-11-28T21:54:30.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.620+0000 7f788759e700 1 -- 172.21.15.162:0/760917558 shutdown_connections 2020-11-28T21:54:30.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.620+0000 7f788759e700 1 --2- 172.21.15.162:0/760917558 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f78640553b0 0x7f7864057870 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:30.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.620+0000 7f788759e700 1 --2- 172.21.15.162:0/760917558 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7888067750 0x7f788814e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:30.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.620+0000 7f788759e700 1 --2- 172.21.15.162:0/760917558 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78880aba40 0x7f78880a9330 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:30.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.620+0000 7f788759e700 1 --2- 172.21.15.162:0/760917558 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f78880670c0 0x7f78880a9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:30.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.620+0000 7f788759e700 1 -- 172.21.15.162:0/760917558 >> 172.21.15.162:0/760917558 conn(0x7f78880b5520 msgr2=0x7f78880b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:30.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.620+0000 7f788759e700 1 -- 172.21.15.162:0/760917558 shutdown_connections 2020-11-28T21:54:30.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.620+0000 7f788759e700 1 -- 172.21.15.162:0/760917558 wait complete. 2020-11-28T21:54:30.642 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:54:30.761 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.756+0000 7f1096935700 1 Processor -- start 2020-11-28T21:54:30.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.760+0000 7f1096935700 1 -- start start 2020-11-28T21:54:30.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.760+0000 7f1096935700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1090060200 0x7f1090143580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:30.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.760+0000 7f1096935700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1090064da0 con 0x7f10900649c0 2020-11-28T21:54:30.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.760+0000 7f1096935700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1090142ba0 con 0x7f10900b0bf0 2020-11-28T21:54:30.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.760+0000 7f1096935700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10901429b0 con 0x7f1090060200 2020-11-28T21:54:30.766 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.760+0000 7f1094ed2700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f10900b0bf0 0x7f109013fa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:50374/0 (socket says 172.21.15.162:50374) 2020-11-28T21:54:30.767 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.760+0000 7f1094ed2700 1 -- 172.21.15.162:0/2427282073 learned_addr learned my addr 172.21.15.162:0/2427282073 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:30.767 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.760+0000 7f108f7fe700 1 --2- 172.21.15.162:0/2427282073 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1090060200 0x7f1090143580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:30.767 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.760+0000 7f108effd700 1 -- 172.21.15.162:0/2427282073 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 430621581 0 0) 0x7f1090142ba0 con 0x7f10900b0bf0 2020-11-28T21:54:30.767 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.760+0000 7f108effd700 1 -- 172.21.15.162:0/2427282073 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f107c003230 con 0x7f10900b0bf0 2020-11-28T21:54:30.767 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.760+0000 7f108effd700 1 -- 172.21.15.162:0/2427282073 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4250436717 0 0) 0x7f1090064da0 con 0x7f10900649c0 2020-11-28T21:54:30.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.760+0000 7f108effd700 1 -- 172.21.15.162:0/2427282073 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1090142ba0 con 0x7f10900649c0 2020-11-28T21:54:30.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.760+0000 7f108effd700 1 -- 172.21.15.162:0/2427282073 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3024235091 0 0) 0x7f107c003230 con 0x7f10900b0bf0 2020-11-28T21:54:30.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.760+0000 7f108effd700 1 -- 172.21.15.162:0/2427282073 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1090060200 msgr2=0x7f1090143580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:30.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.760+0000 7f108effd700 1 --2- 172.21.15.162:0/2427282073 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1090060200 0x7f1090143580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:30.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.760+0000 7f108effd700 1 -- 172.21.15.162:0/2427282073 >> v1:172.21.15.118:6789/0 conn(0x7f10900649c0 legacy=0x7f109005b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:30.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.760+0000 7f108effd700 1 -- 172.21.15.162:0/2427282073 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1090142e40 con 0x7f10900b0bf0 2020-11-28T21:54:30.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.760+0000 7f108effd700 1 -- 172.21.15.162:0/2427282073 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f1088001a50 con 0x7f10900b0bf0 2020-11-28T21:54:30.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.760+0000 7f108effd700 1 -- 172.21.15.162:0/2427282073 >> v1:172.21.15.162:6789/0 conn(0x7f10900b0bf0 legacy=0x7f109013fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:30.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.760+0000 7f108effd700 1 --2- 172.21.15.162:0/2427282073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f107c0038b0 0x7f107c003cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:30.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.760+0000 7f108effd700 1 --2- 172.21.15.162:0/2427282073 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f107c0044d0 0x7f107c004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:30.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.760+0000 7f108effd700 1 --2- 172.21.15.162:0/2427282073 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1090060200 0x7f107c004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:30.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.760+0000 7f108effd700 1 -- 172.21.15.162:0/2427282073 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107c005330 con 0x7f107c0044d0 2020-11-28T21:54:30.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.760+0000 7f108effd700 1 -- 172.21.15.162:0/2427282073 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107c00b720 con 0x7f107c0038b0 2020-11-28T21:54:30.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.760+0000 7f108effd700 1 -- 172.21.15.162:0/2427282073 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f107c00b890 con 0x7f1090060200 2020-11-28T21:54:30.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.760+0000 7f108ffff700 1 --2- 172.21.15.162:0/2427282073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f107c0038b0 0x7f107c003cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:30.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.760+0000 7f1094ed2700 1 --2- 172.21.15.162:0/2427282073 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1090060200 0x7f107c004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:30.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.760+0000 7f108f7fe700 1 --2- 172.21.15.162:0/2427282073 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f107c0044d0 0x7f107c004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:30.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.760+0000 7f108ffff700 1 -- 172.21.15.162:0/2427282073 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1090060200 msgr2=0x7f107c004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:30.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.760+0000 7f108ffff700 1 --2- 172.21.15.162:0/2427282073 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1090060200 0x7f107c004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:30.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.760+0000 7f108ffff700 1 -- 172.21.15.162:0/2427282073 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f107c0044d0 msgr2=0x7f107c004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:30.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.760+0000 7f108ffff700 1 --2- 172.21.15.162:0/2427282073 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f107c0044d0 0x7f107c004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:30.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.760+0000 7f108ffff700 1 -- 172.21.15.162:0/2427282073 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f107c003230 con 0x7f107c0038b0 2020-11-28T21:54:30.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.760+0000 7f108f7fe700 1 --2- 172.21.15.162:0/2427282073 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f107c0044d0 0x7f107c004950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:54:30.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.760+0000 7f108ffff700 1 --2- 172.21.15.162:0/2427282073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f107c0038b0 0x7f107c003cb0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 rx=0x7f10800010d0 tx=0x7f1080007b60).ready entity=mon.1 client_cookie=4b83b1f1b821585d server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:30.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.760+0000 7f108effd700 1 -- 172.21.15.162:0/2427282073 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f108000c870 con 0x7f107c0038b0 2020-11-28T21:54:30.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.760+0000 7f108effd700 1 -- 172.21.15.162:0/2427282073 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108000c9d0 con 0x7f107c0038b0 2020-11-28T21:54:30.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.760+0000 7f108effd700 1 -- 172.21.15.162:0/2427282073 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f108000cd30 con 0x7f107c0038b0 2020-11-28T21:54:30.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.760+0000 7f1096935700 1 -- 172.21.15.162:0/2427282073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f107c0038b0 msgr2=0x7f107c003cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:30.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.760+0000 7f1096935700 1 --2- 172.21.15.162:0/2427282073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f107c0038b0 0x7f107c003cb0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 rx=0x7f10800010d0 tx=0x7f1080007b60).stop 2020-11-28T21:54:30.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.760+0000 7f1094ed2700 1 -- 172.21.15.162:0/2427282073 reap_dead start 2020-11-28T21:54:30.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.764+0000 7f1096935700 1 -- 172.21.15.162:0/2427282073 shutdown_connections 2020-11-28T21:54:30.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.764+0000 7f1096935700 1 -- 172.21.15.162:0/2427282073 >> 172.21.15.162:0/2427282073 conn(0x7f1090061740 msgr2=0x7f1090061b50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:30.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.764+0000 7f1096935700 1 -- 172.21.15.162:0/2427282073 shutdown_connections 2020-11-28T21:54:30.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.764+0000 7f1096935700 1 -- 172.21.15.162:0/2427282073 wait complete. 2020-11-28T21:54:30.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.764+0000 7f1096935700 1 Processor -- start 2020-11-28T21:54:30.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.764+0000 7f1096935700 1 -- start start 2020-11-28T21:54:30.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.768+0000 7f1096935700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f10901447a0 0x7f1090144b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:30.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.768+0000 7f1096935700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f109013c870 0x7f1090154b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:30.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.768+0000 7f1096935700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1090155070 0x7f1090155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:30.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.768+0000 7f1096935700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10900af8e0 con 0x7f109013c870 2020-11-28T21:54:30.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.768+0000 7f1096935700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1090155a10 con 0x7f1090155070 2020-11-28T21:54:30.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.768+0000 7f1096935700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1090152140 con 0x7f10901447a0 2020-11-28T21:54:30.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.768+0000 7f108ffff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f10901447a0 0x7f1090144b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:30.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.768+0000 7f1094ed2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1090155070 0x7f1090155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:30.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.768+0000 7f108ffff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f10901447a0 0x7f1090144b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:40762/0 (socket says 172.21.15.162:40762) 2020-11-28T21:54:30.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.768+0000 7f1094ed2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1090155070 0x7f1090155520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43542/0 (socket says 172.21.15.162:43542) 2020-11-28T21:54:30.780 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.768+0000 7f108ffff700 1 -- 172.21.15.162:0/534548739 learned_addr learned my addr 172.21.15.162:0/534548739 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:30.780 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.768+0000 7f108f7fe700 1 --2- 172.21.15.162:0/534548739 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f109013c870 0x7f1090154b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:30.780 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.768+0000 7f1094ed2700 1 -- 172.21.15.162:0/534548739 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f10901447a0 msgr2=0x7f1090144b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:30.780 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.768+0000 7f1094ed2700 1 --2- 172.21.15.162:0/534548739 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f10901447a0 0x7f1090144b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:30.780 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.768+0000 7f1094ed2700 1 -- 172.21.15.162:0/534548739 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f109013c870 msgr2=0x7f1090154b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:30.781 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.768+0000 7f1094ed2700 1 --2- 172.21.15.162:0/534548739 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f109013c870 0x7f1090154b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:30.781 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.768+0000 7f1094ed2700 1 -- 172.21.15.162:0/534548739 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f109013c090 con 0x7f1090155070 2020-11-28T21:54:30.781 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.768+0000 7f1094ed2700 1 --2- 172.21.15.162:0/534548739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1090155070 0x7f1090155520 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 rx=0x7f1088000b20 tx=0x7f1090142d30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:30.781 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.768+0000 7f108d7fa700 1 -- 172.21.15.162:0/534548739 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f108800fb50 con 0x7f1090155070 2020-11-28T21:54:30.781 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.768+0000 7f108d7fa700 1 -- 172.21.15.162:0/534548739 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108800fcb0 con 0x7f1090155070 2020-11-28T21:54:30.782 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.768+0000 7f108d7fa700 1 -- 172.21.15.162:0/534548739 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f10880012f0 con 0x7f1090155070 2020-11-28T21:54:30.782 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.768+0000 7f1096935700 1 -- 172.21.15.162:0/534548739 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f109013bdf0 con 0x7f1090155070 2020-11-28T21:54:30.782 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.768+0000 7f1096935700 1 -- 172.21.15.162:0/534548739 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10901450f0 con 0x7f1090155070 2020-11-28T21:54:30.782 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.768+0000 7f1096935700 1 -- 172.21.15.162:0/534548739 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1090057050 con 0x7f1090155070 2020-11-28T21:54:30.782 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.768+0000 7f108d7fa700 1 -- 172.21.15.162:0/534548739 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f108801c360 con 0x7f1090155070 2020-11-28T21:54:30.783 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.772+0000 7f108d7fa700 1 --2- 172.21.15.162:0/534548739 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1070055470 0x7f1070057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:30.783 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.772+0000 7f108d7fa700 1 -- 172.21.15.162:0/534548739 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f108806d890 con 0x7f1090155070 2020-11-28T21:54:30.783 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.772+0000 7f108ffff700 1 --2- 172.21.15.162:0/534548739 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1070055470 0x7f1070057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:30.783 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.772+0000 7f108ffff700 1 --2- 172.21.15.162:0/534548739 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1070055470 0x7f1070057930 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7f1080019000 tx=0x7f1080017000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:30.784 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.776+0000 7f108d7fa700 1 -- 172.21.15.162:0/534548739 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1088043a20 con 0x7f1090155070 2020-11-28T21:54:30.916 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.912+0000 7f1096935700 1 -- 172.21.15.162:0/534548739 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f10900684e0 con 0x7f1090155070 2020-11-28T21:54:30.917 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.912+0000 7f108d7fa700 1 -- 172.21.15.162:0/534548739 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v109) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1088019030 con 0x7f1090155070 2020-11-28T21:54:30.917 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:54:30.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.916+0000 7f1096935700 1 -- 172.21.15.162:0/534548739 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1070055470 msgr2=0x7f1070057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:30.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.916+0000 7f1096935700 1 --2- 172.21.15.162:0/534548739 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1070055470 0x7f1070057930 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7f1080019000 tx=0x7f1080017000).stop 2020-11-28T21:54:30.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.916+0000 7f1096935700 1 -- 172.21.15.162:0/534548739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1090155070 msgr2=0x7f1090155520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:30.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.916+0000 7f1096935700 1 --2- 172.21.15.162:0/534548739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1090155070 0x7f1090155520 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 rx=0x7f1088000b20 tx=0x7f1090142d30).stop 2020-11-28T21:54:30.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.916+0000 7f1096935700 1 -- 172.21.15.162:0/534548739 shutdown_connections 2020-11-28T21:54:30.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.916+0000 7f1096935700 1 --2- 172.21.15.162:0/534548739 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1070055470 0x7f1070057930 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:30.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.916+0000 7f1096935700 1 --2- 172.21.15.162:0/534548739 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f10901447a0 0x7f1090144b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:30.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.916+0000 7f1096935700 1 --2- 172.21.15.162:0/534548739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1090155070 0x7f1090155520 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:30.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.916+0000 7f1096935700 1 --2- 172.21.15.162:0/534548739 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f109013c870 0x7f1090154b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:30.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.916+0000 7f1096935700 1 -- 172.21.15.162:0/534548739 >> 172.21.15.162:0/534548739 conn(0x7f1090061740 msgr2=0x7f10900b2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:30.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.916+0000 7f1096935700 1 -- 172.21.15.162:0/534548739 shutdown_connections 2020-11-28T21:54:30.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:30.916+0000 7f1096935700 1 -- 172.21.15.162:0/534548739 wait complete. 2020-11-28T21:54:30.938 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-28T21:54:31.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.052+0000 7fc11759e700 1 Processor -- start 2020-11-28T21:54:31.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.052+0000 7fc11759e700 1 -- start start 2020-11-28T21:54:31.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.052+0000 7fc11759e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fc1180b1eb0 0x7fc1180b2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:31.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.052+0000 7fc11759e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc1180af6a0 0x7fc118060a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:31.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.052+0000 7fc11759e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1181471f0 con 0x7fc118144350 2020-11-28T21:54:31.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.052+0000 7fc11759e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc118060090 con 0x7fc1180b1eb0 2020-11-28T21:54:31.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.052+0000 7fc11759e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc118060330 con 0x7fc1180af6a0 2020-11-28T21:54:31.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.052+0000 7fc116d9d700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fc118144350 0x7fc11814a440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:50392/0 (socket says 172.21.15.162:50392) 2020-11-28T21:54:31.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.052+0000 7fc116d9d700 1 -- 172.21.15.162:0/2923378462 learned_addr learned my addr 172.21.15.162:0/2923378462 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:31.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.052+0000 7fc11659c700 1 --2- 172.21.15.162:0/2923378462 >> v2:172.21.15.118:3300/0 conn(0x7fc1180b1eb0 0x7fc1180b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:31.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.052+0000 7fc10ffff700 1 --2- 172.21.15.162:0/2923378462 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc1180af6a0 0x7fc118060a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:31.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.052+0000 7fc115d9b700 1 -- 172.21.15.162:0/2923378462 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2266457367 0 0) 0x7fc1181471f0 con 0x7fc118144350 2020-11-28T21:54:31.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.052+0000 7fc115d9b700 1 -- 172.21.15.162:0/2923378462 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0f8003230 con 0x7fc118144350 2020-11-28T21:54:31.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.052+0000 7fc115d9b700 1 -- 172.21.15.162:0/2923378462 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1758300427 0 0) 0x7fc0f8003230 con 0x7fc118144350 2020-11-28T21:54:31.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.052+0000 7fc115d9b700 1 -- 172.21.15.162:0/2923378462 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc1180af6a0 msgr2=0x7fc118060a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:31.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.052+0000 7fc115d9b700 1 --2- 172.21.15.162:0/2923378462 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc1180af6a0 0x7fc118060a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:31.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.052+0000 7fc115d9b700 1 -- 172.21.15.162:0/2923378462 >> v2:172.21.15.118:3300/0 conn(0x7fc1180b1eb0 msgr2=0x7fc1180b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:31.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.052+0000 7fc115d9b700 1 --2- 172.21.15.162:0/2923378462 >> v2:172.21.15.118:3300/0 conn(0x7fc1180b1eb0 0x7fc1180b2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:31.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.052+0000 7fc115d9b700 1 -- 172.21.15.162:0/2923378462 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1180605d0 con 0x7fc118144350 2020-11-28T21:54:31.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.052+0000 7fc115d9b700 1 -- 172.21.15.162:0/2923378462 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7fc110001a50 con 0x7fc118144350 2020-11-28T21:54:31.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.052+0000 7fc115d9b700 1 -- 172.21.15.162:0/2923378462 >> v1:172.21.15.162:6789/0 conn(0x7fc118144350 legacy=0x7fc11814a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:31.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.052+0000 7fc115d9b700 1 --2- 172.21.15.162:0/2923378462 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc0f80037a0 0x7fc0f8003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:31.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.052+0000 7fc115d9b700 1 --2- 172.21.15.162:0/2923378462 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0f80044d0 0x7fc0f8004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:31.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.052+0000 7fc115d9b700 1 --2- 172.21.15.162:0/2923378462 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc1180af6a0 0x7fc0f8004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:31.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.052+0000 7fc115d9b700 1 -- 172.21.15.162:0/2923378462 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc118060090 con 0x7fc0f80037a0 2020-11-28T21:54:31.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.052+0000 7fc115d9b700 1 -- 172.21.15.162:0/2923378462 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc118060330 con 0x7fc0f80044d0 2020-11-28T21:54:31.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.052+0000 7fc115d9b700 1 -- 172.21.15.162:0/2923378462 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f8005330 con 0x7fc1180af6a0 2020-11-28T21:54:31.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.052+0000 7fc10ffff700 1 --2- 172.21.15.162:0/2923378462 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0f80044d0 0x7fc0f8004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:31.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.052+0000 7fc11659c700 1 --2- 172.21.15.162:0/2923378462 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc0f80037a0 0x7fc0f8003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:31.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.052+0000 7fc116d9d700 1 --2- 172.21.15.162:0/2923378462 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc1180af6a0 0x7fc0f8004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:31.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.052+0000 7fc10ffff700 1 -- 172.21.15.162:0/2923378462 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc1180af6a0 msgr2=0x7fc0f8004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:31.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.052+0000 7fc10ffff700 1 --2- 172.21.15.162:0/2923378462 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc1180af6a0 0x7fc0f8004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:31.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.052+0000 7fc10ffff700 1 -- 172.21.15.162:0/2923378462 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc0f80037a0 msgr2=0x7fc0f8003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:31.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.052+0000 7fc10ffff700 1 --2- 172.21.15.162:0/2923378462 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc0f80037a0 0x7fc0f8003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:31.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.052+0000 7fc10ffff700 1 -- 172.21.15.162:0/2923378462 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f8003230 con 0x7fc0f80044d0 2020-11-28T21:54:31.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.052+0000 7fc10ffff700 1 --2- 172.21.15.162:0/2923378462 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0f80044d0 0x7fc0f8004950 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 rx=0x7fc10000c000 tx=0x7fc100008040).ready entity=mon.1 client_cookie=5b608234cc88aa0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:31.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.052+0000 7fc115d9b700 1 -- 172.21.15.162:0/2923378462 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc10000e970 con 0x7fc0f80044d0 2020-11-28T21:54:31.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.052+0000 7fc115d9b700 1 -- 172.21.15.162:0/2923378462 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc10000ead0 con 0x7fc0f80044d0 2020-11-28T21:54:31.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.052+0000 7fc115d9b700 1 -- 172.21.15.162:0/2923378462 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc10000ee30 con 0x7fc0f80044d0 2020-11-28T21:54:31.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.052+0000 7fc11759e700 1 -- 172.21.15.162:0/2923378462 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0f80044d0 msgr2=0x7fc0f8004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:31.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.052+0000 7fc11759e700 1 --2- 172.21.15.162:0/2923378462 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0f80044d0 0x7fc0f8004950 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 rx=0x7fc10000c000 tx=0x7fc100008040).stop 2020-11-28T21:54:31.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.052+0000 7fc116d9d700 1 -- 172.21.15.162:0/2923378462 reap_dead start 2020-11-28T21:54:31.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.056+0000 7fc11759e700 1 -- 172.21.15.162:0/2923378462 shutdown_connections 2020-11-28T21:54:31.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.056+0000 7fc11759e700 1 -- 172.21.15.162:0/2923378462 >> 172.21.15.162:0/2923378462 conn(0x7fc1180b57d0 msgr2=0x7fc1180b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:31.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.056+0000 7fc11759e700 1 -- 172.21.15.162:0/2923378462 shutdown_connections 2020-11-28T21:54:31.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.056+0000 7fc11759e700 1 -- 172.21.15.162:0/2923378462 wait complete. 2020-11-28T21:54:31.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.056+0000 7fc11759e700 1 Processor -- start 2020-11-28T21:54:31.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.056+0000 7fc11759e700 1 -- start start 2020-11-28T21:54:31.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.056+0000 7fc11759e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc118159940 0x7fc11815a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:31.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.056+0000 7fc11759e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc118066e00 0x7fc118063a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:31.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.056+0000 7fc11759e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc118143780 0x7fc118143c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:31.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.056+0000 7fc11759e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc118159d20 con 0x7fc118159940 2020-11-28T21:54:31.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.056+0000 7fc11759e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1180636c0 con 0x7fc118143780 2020-11-28T21:54:31.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.056+0000 7fc11759e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc118065420 con 0x7fc118066e00 2020-11-28T21:54:31.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.060+0000 7fc116d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc118143780 0x7fc118143c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:31.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.060+0000 7fc116d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc118143780 0x7fc118143c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43562/0 (socket says 172.21.15.162:43562) 2020-11-28T21:54:31.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.060+0000 7fc116d9d700 1 -- 172.21.15.162:0/212721547 learned_addr learned my addr 172.21.15.162:0/212721547 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:31.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.060+0000 7fc10ffff700 1 --2- 172.21.15.162:0/212721547 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc118066e00 0x7fc118063a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:31.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.060+0000 7fc11659c700 1 --2- 172.21.15.162:0/212721547 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc118159940 0x7fc11815a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:31.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.060+0000 7fc116d9d700 1 -- 172.21.15.162:0/212721547 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc118066e00 msgr2=0x7fc118063a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:31.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.060+0000 7fc116d9d700 1 --2- 172.21.15.162:0/212721547 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc118066e00 0x7fc118063a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:31.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.060+0000 7fc116d9d700 1 -- 172.21.15.162:0/212721547 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc118159940 msgr2=0x7fc11815a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:31.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.060+0000 7fc116d9d700 1 --2- 172.21.15.162:0/212721547 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc118159940 0x7fc11815a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:31.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.060+0000 7fc116d9d700 1 -- 172.21.15.162:0/212721547 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1180656c0 con 0x7fc118143780 2020-11-28T21:54:31.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.060+0000 7fc116d9d700 1 --2- 172.21.15.162:0/212721547 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc118143780 0x7fc118143c00 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 rx=0x7fc1100011d0 tx=0x7fc110002120).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:31.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.060+0000 7fc10f7fe700 1 -- 172.21.15.162:0/212721547 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc11000cad0 con 0x7fc118143780 2020-11-28T21:54:31.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.060+0000 7fc10f7fe700 1 -- 172.21.15.162:0/212721547 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11000cc30 con 0x7fc118143780 2020-11-28T21:54:31.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.060+0000 7fc11759e700 1 -- 172.21.15.162:0/212721547 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1181440f0 con 0x7fc118143780 2020-11-28T21:54:31.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.060+0000 7fc10f7fe700 1 -- 172.21.15.162:0/212721547 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc110014430 con 0x7fc118143780 2020-11-28T21:54:31.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.060+0000 7fc11759e700 1 -- 172.21.15.162:0/212721547 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc118063340 con 0x7fc118143780 2020-11-28T21:54:31.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.060+0000 7fc11759e700 1 -- 172.21.15.162:0/212721547 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc118057050 con 0x7fc118143780 2020-11-28T21:54:31.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.060+0000 7fc10f7fe700 1 -- 172.21.15.162:0/212721547 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc11001d450 con 0x7fc118143780 2020-11-28T21:54:31.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.064+0000 7fc10f7fe700 1 --2- 172.21.15.162:0/212721547 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc0fc0554e0 0x7fc0fc0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:31.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.064+0000 7fc11659c700 1 --2- 172.21.15.162:0/212721547 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc0fc0554e0 0x7fc0fc0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:31.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.064+0000 7fc10f7fe700 1 -- 172.21.15.162:0/212721547 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc11006ddf0 con 0x7fc118143780 2020-11-28T21:54:31.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.064+0000 7fc11659c700 1 --2- 172.21.15.162:0/212721547 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc0fc0554e0 0x7fc0fc0579a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7fc108008000 tx=0x7fc108008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:31.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.068+0000 7fc10f7fe700 1 -- 172.21.15.162:0/212721547 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc110044090 con 0x7fc118143780 2020-11-28T21:54:31.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.208+0000 7fc11759e700 1 -- 172.21.15.162:0/212721547 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc118064740 con 0x7fc118143780 2020-11-28T21:54:31.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.208+0000 7fc10f7fe700 1 -- 172.21.15.162:0/212721547 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v109) v1 ==== 104+0+10 (secure 0 0 0) 0x7fc110001510 con 0x7fc118143780 2020-11-28T21:54:31.211 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:54:31.212 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.208+0000 7fc11759e700 1 -- 172.21.15.162:0/212721547 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc0fc0554e0 msgr2=0x7fc0fc0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:31.213 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.208+0000 7fc11759e700 1 --2- 172.21.15.162:0/212721547 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc0fc0554e0 0x7fc0fc0579a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7fc108008000 tx=0x7fc108008040).stop 2020-11-28T21:54:31.213 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.208+0000 7fc11759e700 1 -- 172.21.15.162:0/212721547 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc118143780 msgr2=0x7fc118143c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:31.213 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.208+0000 7fc11759e700 1 --2- 172.21.15.162:0/212721547 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc118143780 0x7fc118143c00 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 rx=0x7fc1100011d0 tx=0x7fc110002120).stop 2020-11-28T21:54:31.213 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.208+0000 7fc11759e700 1 -- 172.21.15.162:0/212721547 shutdown_connections 2020-11-28T21:54:31.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.208+0000 7fc11759e700 1 --2- 172.21.15.162:0/212721547 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc0fc0554e0 0x7fc0fc0579a0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:31.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.208+0000 7fc11759e700 1 --2- 172.21.15.162:0/212721547 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc118143780 0x7fc118143c00 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:31.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.208+0000 7fc11759e700 1 --2- 172.21.15.162:0/212721547 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc118159940 0x7fc11815a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:31.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.208+0000 7fc11759e700 1 --2- 172.21.15.162:0/212721547 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc118066e00 0x7fc118063a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:31.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.208+0000 7fc11759e700 1 -- 172.21.15.162:0/212721547 >> 172.21.15.162:0/212721547 conn(0x7fc1180b57d0 msgr2=0x7fc1180ad5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:31.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.208+0000 7fc11759e700 1 -- 172.21.15.162:0/212721547 shutdown_connections 2020-11-28T21:54:31.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.208+0000 7fc11759e700 1 -- 172.21.15.162:0/212721547 wait complete. 2020-11-28T21:54:31.224 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-28T21:54:31.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.340+0000 7fd2f8a3b700 1 Processor -- start 2020-11-28T21:54:31.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.340+0000 7fd2f8a3b700 1 -- start start 2020-11-28T21:54:31.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.344+0000 7fd2f8a3b700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2f405bcb0 0x7fd2f4061300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:31.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.344+0000 7fd2f8a3b700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd2f40b21a0 0x7fd2f40abfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:31.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.344+0000 7fd2f8a3b700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2f405c090 con 0x7fd2f4061a50 2020-11-28T21:54:31.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.344+0000 7fd2f8a3b700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2f4143820 con 0x7fd2f40b21a0 2020-11-28T21:54:31.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.344+0000 7fd2f8a3b700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f40617f0 con 0x7fd2f405bcb0 2020-11-28T21:54:31.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.344+0000 7fd2f359e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd2f40b21a0 0x7fd2f40abfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:31.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.344+0000 7fd2f359e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd2f40b21a0 0x7fd2f40abfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.162:50558/0 (socket says 172.21.15.162:50558) 2020-11-28T21:54:31.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.344+0000 7fd2f359e700 1 -- 172.21.15.162:0/1578194554 learned_addr learned my addr 172.21.15.162:0/1578194554 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:31.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.344+0000 7fd2f2d9d700 1 --2- 172.21.15.162:0/1578194554 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2f405bcb0 0x7fd2f4061300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:31.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.344+0000 7fd2f359e700 1 -- 172.21.15.162:0/1578194554 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2f405bcb0 msgr2=0x7fd2f4061300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:31.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.344+0000 7fd2f359e700 1 --2- 172.21.15.162:0/1578194554 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2f405bcb0 0x7fd2f4061300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:31.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.344+0000 7fd2f359e700 1 -- 172.21.15.162:0/1578194554 >> v1:172.21.15.118:6789/0 conn(0x7fd2f4061a50 legacy=0x7fd2f41427d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:31.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.344+0000 7fd2f359e700 1 -- 172.21.15.162:0/1578194554 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f4061f60 con 0x7fd2f40b21a0 2020-11-28T21:54:31.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.344+0000 7fd2f359e700 1 --2- 172.21.15.162:0/1578194554 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2f40b21a0 0x7fd2f40abfb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7fd2ec0032b0 tx=0x7fd2ec009270).ready entity=mon.0 client_cookie=7ed7307d48060721 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:31.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.344+0000 7fd2f1d9b700 1 -- 172.21.15.162:0/1578194554 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2ec009e60 con 0x7fd2f40b21a0 2020-11-28T21:54:31.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.344+0000 7fd2f1d9b700 1 -- 172.21.15.162:0/1578194554 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2ec00bc00 con 0x7fd2f40b21a0 2020-11-28T21:54:31.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.344+0000 7fd2f1d9b700 1 -- 172.21.15.162:0/1578194554 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2ec00bd70 con 0x7fd2f40b21a0 2020-11-28T21:54:31.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.344+0000 7fd2f8a3b700 1 -- 172.21.15.162:0/1578194554 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2f40b21a0 msgr2=0x7fd2f40abfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:31.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.344+0000 7fd2f8a3b700 1 --2- 172.21.15.162:0/1578194554 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2f40b21a0 0x7fd2f40abfb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7fd2ec0032b0 tx=0x7fd2ec009270).stop 2020-11-28T21:54:31.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.344+0000 7fd2f8a3b700 1 -- 172.21.15.162:0/1578194554 shutdown_connections 2020-11-28T21:54:31.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.344+0000 7fd2f8a3b700 1 --2- 172.21.15.162:0/1578194554 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2f40b21a0 0x7fd2f40abfb0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:31.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.344+0000 7fd2f8a3b700 1 --2- 172.21.15.162:0/1578194554 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2f405bcb0 0x7fd2f4061300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:31.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.344+0000 7fd2f8a3b700 1 -- 172.21.15.162:0/1578194554 >> 172.21.15.162:0/1578194554 conn(0x7fd2f40b2830 msgr2=0x7fd2f40b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:31.355 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.344+0000 7fd2f8a3b700 1 -- 172.21.15.162:0/1578194554 shutdown_connections 2020-11-28T21:54:31.355 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.344+0000 7fd2f8a3b700 1 -- 172.21.15.162:0/1578194554 wait complete. 2020-11-28T21:54:31.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.348+0000 7fd2f8a3b700 1 Processor -- start 2020-11-28T21:54:31.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.348+0000 7fd2f8a3b700 1 -- start start 2020-11-28T21:54:31.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.348+0000 7fd2f8a3b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2f405bcb0 0x7fd2f4142110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:31.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.348+0000 7fd2f8a3b700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2f4061a50 0x7fd2f40ae730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:31.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.348+0000 7fd2f8a3b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2f40b21a0 0x7fd2f41514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:31.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.348+0000 7fd2f8a3b700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f4154a90 con 0x7fd2f40b21a0 2020-11-28T21:54:31.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.348+0000 7fd2f8a3b700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f4154d30 con 0x7fd2f405bcb0 2020-11-28T21:54:31.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.348+0000 7fd2f8a3b700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f4154fd0 con 0x7fd2f4061a50 2020-11-28T21:54:31.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.348+0000 7fd2f2d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2f405bcb0 0x7fd2f4142110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:31.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.348+0000 7fd2f2d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2f405bcb0 0x7fd2f4142110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43572/0 (socket says 172.21.15.162:43572) 2020-11-28T21:54:31.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.348+0000 7fd2f2d9d700 1 -- 172.21.15.162:0/4092166553 learned_addr learned my addr 172.21.15.162:0/4092166553 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:31.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.348+0000 7fd2f359e700 1 --2- 172.21.15.162:0/4092166553 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2f40b21a0 0x7fd2f41514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:31.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.348+0000 7fd2f259c700 1 --2- 172.21.15.162:0/4092166553 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2f4061a50 0x7fd2f40ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:31.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.352+0000 7fd2f2d9d700 1 -- 172.21.15.162:0/4092166553 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2f4061a50 msgr2=0x7fd2f40ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:31.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.352+0000 7fd2f2d9d700 1 --2- 172.21.15.162:0/4092166553 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2f4061a50 0x7fd2f40ae730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:31.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.352+0000 7fd2f2d9d700 1 -- 172.21.15.162:0/4092166553 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2f40b21a0 msgr2=0x7fd2f41514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:31.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.352+0000 7fd2f2d9d700 1 --2- 172.21.15.162:0/4092166553 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2f40b21a0 0x7fd2f41514a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:31.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.352+0000 7fd2f2d9d700 1 -- 172.21.15.162:0/4092166553 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f4155270 con 0x7fd2f405bcb0 2020-11-28T21:54:31.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.352+0000 7fd2f2d9d700 1 --2- 172.21.15.162:0/4092166553 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2f405bcb0 0x7fd2f4142110 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 rx=0x7fd2e400c000 tx=0x7fd2e4006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:31.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.352+0000 7fd2e3fff700 1 -- 172.21.15.162:0/4092166553 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2e400ca10 con 0x7fd2f405bcb0 2020-11-28T21:54:31.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.352+0000 7fd2e3fff700 1 -- 172.21.15.162:0/4092166553 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e400cb70 con 0x7fd2f405bcb0 2020-11-28T21:54:31.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.352+0000 7fd2f8a3b700 1 -- 172.21.15.162:0/4092166553 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2f41400b0 con 0x7fd2f405bcb0 2020-11-28T21:54:31.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.352+0000 7fd2e3fff700 1 -- 172.21.15.162:0/4092166553 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2e40123f0 con 0x7fd2f405bcb0 2020-11-28T21:54:31.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.352+0000 7fd2f8a3b700 1 -- 172.21.15.162:0/4092166553 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2f40aec20 con 0x7fd2f405bcb0 2020-11-28T21:54:31.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.352+0000 7fd2f8a3b700 1 -- 172.21.15.162:0/4092166553 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2f4155400 con 0x7fd2f405bcb0 2020-11-28T21:54:31.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.352+0000 7fd2e3fff700 1 -- 172.21.15.162:0/4092166553 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd2e4012550 con 0x7fd2f405bcb0 2020-11-28T21:54:31.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.352+0000 7fd2e3fff700 1 --2- 172.21.15.162:0/4092166553 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd2d0055440 0x7fd2d0057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:31.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.356+0000 7fd2e3fff700 1 -- 172.21.15.162:0/4092166553 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd2e406f0a0 con 0x7fd2f405bcb0 2020-11-28T21:54:31.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.356+0000 7fd2f259c700 1 --2- 172.21.15.162:0/4092166553 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd2d0055440 0x7fd2d0057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:31.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.356+0000 7fd2f259c700 1 --2- 172.21.15.162:0/4092166553 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd2d0055440 0x7fd2d0057900 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7fd2e800d000 tx=0x7fd2e8007b20).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:31.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.360+0000 7fd2e3fff700 1 -- 172.21.15.162:0/4092166553 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd2e4045340 con 0x7fd2f405bcb0 2020-11-28T21:54:31.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.500+0000 7fd2f8a3b700 1 -- 172.21.15.162:0/4092166553 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd2f4057050 con 0x7fd2f405bcb0 2020-11-28T21:54:31.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.500+0000 7fd2e3fff700 1 -- 172.21.15.162:0/4092166553 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v109) v1 ==== 100+0+10 (secure 0 0 0) 0x7fd2e4048960 con 0x7fd2f405bcb0 2020-11-28T21:54:31.504 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:54:31.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.504+0000 7fd2f8a3b700 1 -- 172.21.15.162:0/4092166553 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd2d0055440 msgr2=0x7fd2d0057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:31.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.504+0000 7fd2f8a3b700 1 --2- 172.21.15.162:0/4092166553 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd2d0055440 0x7fd2d0057900 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7fd2e800d000 tx=0x7fd2e8007b20).stop 2020-11-28T21:54:31.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.504+0000 7fd2f8a3b700 1 -- 172.21.15.162:0/4092166553 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2f405bcb0 msgr2=0x7fd2f4142110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:31.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.504+0000 7fd2f8a3b700 1 --2- 172.21.15.162:0/4092166553 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2f405bcb0 0x7fd2f4142110 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 rx=0x7fd2e400c000 tx=0x7fd2e4006dc0).stop 2020-11-28T21:54:31.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.504+0000 7fd2f8a3b700 1 -- 172.21.15.162:0/4092166553 shutdown_connections 2020-11-28T21:54:31.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.504+0000 7fd2f8a3b700 1 --2- 172.21.15.162:0/4092166553 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd2d0055440 0x7fd2d0057900 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:31.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.504+0000 7fd2f8a3b700 1 --2- 172.21.15.162:0/4092166553 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2f40b21a0 0x7fd2f41514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:31.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.504+0000 7fd2f8a3b700 1 --2- 172.21.15.162:0/4092166553 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2f405bcb0 0x7fd2f4142110 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:31.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.504+0000 7fd2f8a3b700 1 --2- 172.21.15.162:0/4092166553 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2f4061a50 0x7fd2f40ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:31.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.504+0000 7fd2f8a3b700 1 -- 172.21.15.162:0/4092166553 >> 172.21.15.162:0/4092166553 conn(0x7fd2f40b2830 msgr2=0x7fd2f413a650 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:31.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.504+0000 7fd2f8a3b700 1 -- 172.21.15.162:0/4092166553 shutdown_connections 2020-11-28T21:54:31.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.504+0000 7fd2f8a3b700 1 -- 172.21.15.162:0/4092166553 wait complete. 2020-11-28T21:54:31.522 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-28T21:54:31.522 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:54:31.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.628+0000 7fc60e86a700 1 Processor -- start 2020-11-28T21:54:31.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.628+0000 7fc60e86a700 1 -- start start 2020-11-28T21:54:31.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.628+0000 7fc60e86a700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc608098b30 0x7fc6080994f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:31.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.628+0000 7fc60e86a700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fc6080b0000 0x7fc608060000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:31.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.628+0000 7fc60e86a700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc608098f10 con 0x7fc6080604f0 2020-11-28T21:54:31.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.628+0000 7fc60e86a700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6080b1030 con 0x7fc6080b0000 2020-11-28T21:54:31.639 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.628+0000 7fc60e86a700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6080b0480 con 0x7fc608098b30 2020-11-28T21:54:31.639 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.628+0000 7fc60d868700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc608098b30 0x7fc6080994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:31.639 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.628+0000 7fc60e069700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fc6080604f0 0x7fc60814a550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:50428/0 (socket says 172.21.15.162:50428) 2020-11-28T21:54:31.639 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.628+0000 7fc60e069700 1 -- 172.21.15.162:0/2062654970 learned_addr learned my addr 172.21.15.162:0/2062654970 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:31.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.628+0000 7fc60d067700 1 --2- 172.21.15.162:0/2062654970 >> v2:172.21.15.118:3300/0 conn(0x7fc6080b0000 0x7fc608060000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:31.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.628+0000 7fc60d868700 1 -- 172.21.15.162:0/2062654970 >> v2:172.21.15.118:3300/0 conn(0x7fc6080b0000 msgr2=0x7fc608060000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:31.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.628+0000 7fc60d868700 1 --2- 172.21.15.162:0/2062654970 >> v2:172.21.15.118:3300/0 conn(0x7fc6080b0000 0x7fc608060000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:31.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.628+0000 7fc60c866700 1 -- 172.21.15.162:0/2062654970 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2380671 0 0) 0x7fc608098f10 con 0x7fc6080604f0 2020-11-28T21:54:31.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.628+0000 7fc60d868700 1 -- 172.21.15.162:0/2062654970 >> v1:172.21.15.162:6789/0 conn(0x7fc6080604f0 legacy=0x7fc60814a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:31.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.628+0000 7fc60d868700 1 -- 172.21.15.162:0/2062654970 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6080b0680 con 0x7fc608098b30 2020-11-28T21:54:31.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.628+0000 7fc60d067700 1 --2- 172.21.15.162:0/2062654970 >> v2:172.21.15.118:3300/0 conn(0x7fc6080b0000 0x7fc608060000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:54:31.642 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.628+0000 7fc60d868700 1 --2- 172.21.15.162:0/2062654970 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc608098b30 0x7fc6080994f0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 rx=0x7fc5fc002f30 tx=0x7fc5fc006dc0).ready entity=mon.1 client_cookie=e11c4f0f8ea78206 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:31.642 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.628+0000 7fc60c866700 1 -- 172.21.15.162:0/2062654970 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5fc0097f0 con 0x7fc608098b30 2020-11-28T21:54:31.642 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.628+0000 7fc60c866700 1 -- 172.21.15.162:0/2062654970 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5fc009950 con 0x7fc608098b30 2020-11-28T21:54:31.642 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.628+0000 7fc60c866700 1 -- 172.21.15.162:0/2062654970 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5fc009cb0 con 0x7fc608098b30 2020-11-28T21:54:31.642 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.628+0000 7fc60e86a700 1 -- 172.21.15.162:0/2062654970 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc608098b30 msgr2=0x7fc6080994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:31.643 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.628+0000 7fc60e86a700 1 --2- 172.21.15.162:0/2062654970 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc608098b30 0x7fc6080994f0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 rx=0x7fc5fc002f30 tx=0x7fc5fc006dc0).stop 2020-11-28T21:54:31.643 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.628+0000 7fc60e86a700 1 -- 172.21.15.162:0/2062654970 shutdown_connections 2020-11-28T21:54:31.643 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.628+0000 7fc60e86a700 1 --2- 172.21.15.162:0/2062654970 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc608098b30 0x7fc6080994f0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:31.643 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.628+0000 7fc60e86a700 1 --2- 172.21.15.162:0/2062654970 >> v2:172.21.15.118:3300/0 conn(0x7fc6080b0000 0x7fc608060000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:31.643 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.628+0000 7fc60e86a700 1 -- 172.21.15.162:0/2062654970 >> 172.21.15.162:0/2062654970 conn(0x7fc60805b970 msgr2=0x7fc60805bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:31.644 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.628+0000 7fc60e86a700 1 -- 172.21.15.162:0/2062654970 shutdown_connections 2020-11-28T21:54:31.644 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.628+0000 7fc60e86a700 1 -- 172.21.15.162:0/2062654970 wait complete. 2020-11-28T21:54:31.644 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.632+0000 7fc60e86a700 1 Processor -- start 2020-11-28T21:54:31.644 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.632+0000 7fc60e86a700 1 -- start start 2020-11-28T21:54:31.644 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.632+0000 7fc60e86a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc6080604f0 0x7fc608064c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:31.645 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.632+0000 7fc60e86a700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc608098b30 0x7fc608155be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:31.645 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.632+0000 7fc60e86a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6080b0000 0x7fc6081560d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:31.645 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.632+0000 7fc60e86a700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc608064740 con 0x7fc6080604f0 2020-11-28T21:54:31.645 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.632+0000 7fc60e86a700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc608065830 con 0x7fc6080b0000 2020-11-28T21:54:31.645 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.632+0000 7fc60e86a700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6080634c0 con 0x7fc608098b30 2020-11-28T21:54:31.646 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.632+0000 7fc60e069700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6080b0000 0x7fc6081560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:31.646 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.632+0000 7fc60e069700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6080b0000 0x7fc6081560d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43590/0 (socket says 172.21.15.162:43590) 2020-11-28T21:54:31.646 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.632+0000 7fc60e069700 1 -- 172.21.15.162:0/2383432453 learned_addr learned my addr 172.21.15.162:0/2383432453 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:31.646 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.632+0000 7fc60d067700 1 --2- 172.21.15.162:0/2383432453 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc608098b30 0x7fc608155be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:31.647 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.632+0000 7fc60d868700 1 --2- 172.21.15.162:0/2383432453 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc6080604f0 0x7fc608064c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:31.647 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.632+0000 7fc60e069700 1 -- 172.21.15.162:0/2383432453 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc608098b30 msgr2=0x7fc608155be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:31.647 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.632+0000 7fc60e069700 1 --2- 172.21.15.162:0/2383432453 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc608098b30 0x7fc608155be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:31.647 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.632+0000 7fc60e069700 1 -- 172.21.15.162:0/2383432453 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc6080604f0 msgr2=0x7fc608064c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:31.647 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.632+0000 7fc60e069700 1 --2- 172.21.15.162:0/2383432453 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc6080604f0 0x7fc608064c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:31.648 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.632+0000 7fc60e069700 1 -- 172.21.15.162:0/2383432453 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6080b4150 con 0x7fc6080b0000 2020-11-28T21:54:31.648 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.632+0000 7fc60e069700 1 --2- 172.21.15.162:0/2383432453 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6080b0000 0x7fc6081560d0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 rx=0x7fc600001210 tx=0x7fc600007d90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:31.648 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.632+0000 7fc5faffd700 1 -- 172.21.15.162:0/2383432453 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc60000ce60 con 0x7fc6080b0000 2020-11-28T21:54:31.648 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.632+0000 7fc5faffd700 1 -- 172.21.15.162:0/2383432453 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc600005b70 con 0x7fc6080b0000 2020-11-28T21:54:31.649 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.632+0000 7fc5faffd700 1 -- 172.21.15.162:0/2383432453 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc600004870 con 0x7fc6080b0000 2020-11-28T21:54:31.649 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.632+0000 7fc60e86a700 1 -- 172.21.15.162:0/2383432453 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6080644a0 con 0x7fc6080b0000 2020-11-28T21:54:31.649 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.632+0000 7fc60e86a700 1 -- 172.21.15.162:0/2383432453 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc608065520 con 0x7fc6080b0000 2020-11-28T21:54:31.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.632+0000 7fc60e86a700 1 -- 172.21.15.162:0/2383432453 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc608063320 con 0x7fc6080b0000 2020-11-28T21:54:31.651 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.636+0000 7fc5faffd700 1 -- 172.21.15.162:0/2383432453 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc600010070 con 0x7fc6080b0000 2020-11-28T21:54:31.651 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.636+0000 7fc5faffd700 1 --2- 172.21.15.162:0/2383432453 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc5f0055350 0x7fc5f0057810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:31.651 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.636+0000 7fc5faffd700 1 -- 172.21.15.162:0/2383432453 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc60006d850 con 0x7fc6080b0000 2020-11-28T21:54:31.651 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.636+0000 7fc60d868700 1 --2- 172.21.15.162:0/2383432453 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc5f0055350 0x7fc5f0057810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:31.652 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.636+0000 7fc60d868700 1 --2- 172.21.15.162:0/2383432453 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc5f0055350 0x7fc5f0057810 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7fc5fc01b000 tx=0x7fc5fc015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:31.652 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.640+0000 7fc5faffd700 1 -- 172.21.15.162:0/2383432453 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc608063320 con 0x7fc6080b0000 2020-11-28T21:54:31.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.824+0000 7fc60e86a700 1 -- 172.21.15.162:0/2383432453 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc608063320 con 0x7fc6080b0000 2020-11-28T21:54:31.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.824+0000 7fc5faffd700 1 -- 172.21.15.162:0/2383432453 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+2499 (secure 0 0 0) 0x7fc608063320 con 0x7fc6080b0000 2020-11-28T21:54:31.830 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:54:31.830 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":51,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6182,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2775691244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2775691244},{"type":"v1","addr":"172.21.15.162:6835","nonce":2775691244}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":49},{"gid":6942,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/1896314020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1896314020},{"type":"v1","addr":"172.21.15.118:6833","nonce":1896314020}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":49}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:54:26.908249+0000","modified":"2020-11-28T21:54:27.911138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7177},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7177":{"gid":7177,"name":"a","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/493776989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":493776989},{"type":"v1","addr":"172.21.15.162:6837","nonce":493776989}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2020-11-28T21:54:31.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.828+0000 7fc60e86a700 1 -- 172.21.15.162:0/2383432453 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc5f0055350 msgr2=0x7fc5f0057810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:31.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.828+0000 7fc60e86a700 1 --2- 172.21.15.162:0/2383432453 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc5f0055350 0x7fc5f0057810 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7fc5fc01b000 tx=0x7fc5fc015000).stop 2020-11-28T21:54:31.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.828+0000 7fc60e86a700 1 -- 172.21.15.162:0/2383432453 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6080b0000 msgr2=0x7fc6081560d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:31.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.828+0000 7fc60e86a700 1 --2- 172.21.15.162:0/2383432453 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6080b0000 0x7fc6081560d0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 rx=0x7fc600001210 tx=0x7fc600007d90).stop 2020-11-28T21:54:31.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.828+0000 7fc60e86a700 1 -- 172.21.15.162:0/2383432453 shutdown_connections 2020-11-28T21:54:31.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.828+0000 7fc60e86a700 1 --2- 172.21.15.162:0/2383432453 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc5f0055350 0x7fc5f0057810 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:31.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.828+0000 7fc60e86a700 1 --2- 172.21.15.162:0/2383432453 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6080b0000 0x7fc6081560d0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:31.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.828+0000 7fc60e86a700 1 --2- 172.21.15.162:0/2383432453 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc6080604f0 0x7fc608064c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:31.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.828+0000 7fc60e86a700 1 --2- 172.21.15.162:0/2383432453 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc608098b30 0x7fc608155be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:31.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.828+0000 7fc60e86a700 1 -- 172.21.15.162:0/2383432453 >> 172.21.15.162:0/2383432453 conn(0x7fc60805b970 msgr2=0x7fc6080a8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:31.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.828+0000 7fc60e86a700 1 -- 172.21.15.162:0/2383432453 shutdown_connections 2020-11-28T21:54:31.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:31.828+0000 7fc60e86a700 1 -- 172.21.15.162:0/2383432453 wait complete. 2020-11-28T21:54:31.836 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 51 2020-11-28T21:54:32.145 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 51, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-28T21:54:26.908249+0000', 'modified': '2020-11-28T21:54:27.911138+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7177}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7177': {'gid': 7177, 'name': 'a', 'rank': 0, 'incarnation': 50, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6837/493776989', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6836', 'nonce': 493776989}, {'type': 'v1', 'addr': '172.21.15.162:6837', 'nonce': 493776989}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-28T21:54:32.145 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-28T21:54:32.146 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-28T21:54:32.274 INFO:teuthology.orchestra.run.smithi162.stdout:{"cluster_fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","whoami":0,"id":7177,"want_state":"up:active","state":"up:active","rank_uptime":5.351349055,"mdsmap_epoch":51,"osdmap_epoch":109,"osdmap_epoch_barrier":108,"uptime":9.7277775000000002} 2020-11-28T21:54:32.275 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", "id": 7177, "mdsmap_epoch": 51, "osdmap_epoch": 109, "osdmap_epoch_barrier": 108, "rank_uptime": 5.351349055, "state": "up:active", "uptime": 9.7277775, "want_state": "up:active", "whoami": 0 } 2020-11-28T21:54:32.275 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-11-28T21:54:32.275 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-11-28T21:54:32.276 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi192.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-11-28T21:54:32.276 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:32.277 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:54:32.284 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:54:32.342 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:54:32.343 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:54:32.398 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:54:32.399 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:54:32.400 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:32.400 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:54:32.445 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-11-28T21:54:32.445 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:54:32.490 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:54:32.506 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:54:32.507 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:54:32.562 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:54:32.564 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:54:32.565 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:32.565 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:54:32.567 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:2020-11-28T21:54:32.561+0000 7ff7212d70c0 -1 init, newargv = 0x5601d8340a00 newargc=9ceph-fuse[12403 2020-11-28T21:54:32.568 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:]: starting ceph client 2020-11-28T21:54:32.577 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:ceph-fuse[12403]: starting fuse 2020-11-28T21:54:33.674 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:54:33.691 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:54:33.692 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:54:33.746 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:54:33.747 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:54:33.748 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:33.748 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:54:33.795 INFO:teuthology.orchestra.run.smithi192.stdout:49 2020-11-28T21:54:33.796 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2020-11-28T21:54:33.797 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:33.798 INFO:teuthology.orchestra.run.smithi192:> sudo python3 -c ' 2020-11-28T21:54:33.798 INFO:teuthology.orchestra.run.smithi192:> import glob 2020-11-28T21:54:33.799 INFO:teuthology.orchestra.run.smithi192:> import re 2020-11-28T21:54:33.799 INFO:teuthology.orchestra.run.smithi192:> import os 2020-11-28T21:54:33.800 INFO:teuthology.orchestra.run.smithi192:> import subprocess 2020-11-28T21:54:33.800 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:54:33.801 INFO:teuthology.orchestra.run.smithi192:> def find_socket(client_name): 2020-11-28T21:54:33.801 INFO:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-11-28T21:54:33.802 INFO:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2020-11-28T21:54:33.802 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:54:33.803 INFO:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2020-11-28T21:54:33.803 INFO:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2020-11-28T21:54:33.803 INFO:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2020-11-28T21:54:33.804 INFO:teuthology.orchestra.run.smithi192:> return files[0] 2020-11-28T21:54:33.804 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:54:33.805 INFO:teuthology.orchestra.run.smithi192:> for f in files: 2020-11-28T21:54:33.805 INFO:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-11-28T21:54:33.806 INFO:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2020-11-28T21:54:33.806 INFO:teuthology.orchestra.run.smithi192:> return f 2020-11-28T21:54:33.807 INFO:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-11-28T21:54:33.807 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:54:33.807 INFO:teuthology.orchestra.run.smithi192:> print(find_socket("client.0")) 2020-11-28T21:54:33.808 INFO:teuthology.orchestra.run.smithi192:> ' 2020-11-28T21:54:33.884 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.0.12403.asok 2020-11-28T21:54:33.888 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.12403.asok 2020-11-28T21:54:33.889 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:33.889 INFO:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.12403.asok status 2020-11-28T21:54:33.987 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2020-11-28T21:54:33.987 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2020-11-28T21:54:33.987 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "a83a3572a957356977342f461361c1cd3ec06dbd", 2020-11-28T21:54:33.988 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 15.2.6-595-ga83a3572 (a83a3572a957356977342f461361c1cd3ec06dbd) octopus (stable)", 2020-11-28T21:54:33.988 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "0", 2020-11-28T21:54:33.988 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2020-11-28T21:54:33.989 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-11-28T21:54:33.989 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "12403", 2020-11-28T21:54:33.990 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2020-11-28T21:54:33.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:54:33.991 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2020-11-28T21:54:33.991 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2020-11-28T21:54:33.991 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 7327, 2020-11-28T21:54:33.992 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2020-11-28T21:54:33.992 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2020-11-28T21:54:33.993 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2020-11-28T21:54:33.993 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 7327 2020-11-28T21:54:33.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:54:33.994 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2020-11-28T21:54:33.995 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2020-11-28T21:54:33.995 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2020-11-28T21:54:33.995 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 4117707454 2020-11-28T21:54:33.996 INFO:teuthology.orchestra.run.smithi192.stdout: } 2020-11-28T21:54:33.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:54:33.997 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2020-11-28T21:54:33.997 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2020-11-28T21:54:33.998 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2020-11-28T21:54:33.998 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 4117707454 2020-11-28T21:54:33.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:54:33.999 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.7327 172.21.15.192:0/4117707454", 2020-11-28T21:54:33.999 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "172.21.15.192:0/4117707454", 2020-11-28T21:54:34.000 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2020-11-28T21:54:34.000 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 51, 2020-11-28T21:54:34.001 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 109, 2020-11-28T21:54:34.001 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2020-11-28T21:54:34.003 INFO:teuthology.orchestra.run.smithi192.stdout: "blacklisted": false 2020-11-28T21:54:34.003 INFO:teuthology.orchestra.run.smithi192.stdout:} 2020-11-28T21:54:34.005 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:34.006 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:54:34.007 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:54:34.015 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2020-11-28T21:54:34.016 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-28T21:54:34.016 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:34.017 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:54:34.073 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:54:34.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.184+0000 7f4fbbfff700 1 Processor -- start 2020-11-28T21:54:34.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.184+0000 7f4fbbfff700 1 -- start start 2020-11-28T21:54:34.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.188+0000 7f4fbbfff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4fbc13f400 0x7f4fbc09b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:34.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.188+0000 7f4fbbfff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f4fbc0b1940 0x7f4fbc13fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:34.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.188+0000 7f4fbbfff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fbc097a90 con 0x7f4fbc0b45d0 2020-11-28T21:54:34.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.188+0000 7f4fbbfff700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fbc098180 con 0x7f4fbc0b1940 2020-11-28T21:54:34.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.188+0000 7f4fbbfff700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fbc0b49b0 con 0x7f4fbc13f400 2020-11-28T21:54:34.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.188+0000 7f4fba7fc700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4fbc13f400 0x7f4fbc09b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:34.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.188+0000 7f4fba7fc700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4fbc13f400 0x7f4fbc09b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43598/0 (socket says 172.21.15.162:43598) 2020-11-28T21:54:34.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.188+0000 7f4fba7fc700 1 -- 172.21.15.162:0/3952808312 learned_addr learned my addr 172.21.15.162:0/3952808312 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:34.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.188+0000 7f4fbb7fe700 1 --2- 172.21.15.162:0/3952808312 >> v2:172.21.15.118:3300/0 conn(0x7f4fbc0b1940 0x7f4fbc13fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:34.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.188+0000 7f4fba7fc700 1 -- 172.21.15.162:0/3952808312 >> v2:172.21.15.118:3300/0 conn(0x7f4fbc0b1940 msgr2=0x7f4fbc13fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:34.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.188+0000 7f4fba7fc700 1 --2- 172.21.15.162:0/3952808312 >> v2:172.21.15.118:3300/0 conn(0x7f4fbc0b1940 0x7f4fbc13fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:34.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.188+0000 7f4fba7fc700 1 -- 172.21.15.162:0/3952808312 >> v1:172.21.15.118:6789/0 conn(0x7f4fbc0b45d0 legacy=0x7f4fbc0afe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:34.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.188+0000 7f4fba7fc700 1 -- 172.21.15.162:0/3952808312 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fbc0987e0 con 0x7f4fbc13f400 2020-11-28T21:54:34.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.188+0000 7f4fba7fc700 1 --2- 172.21.15.162:0/3952808312 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4fbc13f400 0x7f4fbc09b330 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 rx=0x7f4fac0032b0 tx=0x7f4fac009270).ready entity=mon.1 client_cookie=3c6d535e69443619 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:34.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.188+0000 7f4fb9ffb700 1 -- 172.21.15.162:0/3952808312 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fac009e60 con 0x7f4fbc13f400 2020-11-28T21:54:34.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.188+0000 7f4fb9ffb700 1 -- 172.21.15.162:0/3952808312 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fac00bc00 con 0x7f4fbc13f400 2020-11-28T21:54:34.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.188+0000 7f4fb9ffb700 1 -- 172.21.15.162:0/3952808312 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fac00bd70 con 0x7f4fbc13f400 2020-11-28T21:54:34.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.188+0000 7f4fbbfff700 1 -- 172.21.15.162:0/3952808312 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4fbc13f400 msgr2=0x7f4fbc09b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:34.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.188+0000 7f4fbbfff700 1 --2- 172.21.15.162:0/3952808312 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4fbc13f400 0x7f4fbc09b330 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 rx=0x7f4fac0032b0 tx=0x7f4fac009270).stop 2020-11-28T21:54:34.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.188+0000 7f4fbbfff700 1 -- 172.21.15.162:0/3952808312 shutdown_connections 2020-11-28T21:54:34.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.188+0000 7f4fbbfff700 1 --2- 172.21.15.162:0/3952808312 >> v2:172.21.15.118:3300/0 conn(0x7f4fbc0b1940 0x7f4fbc13fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:34.200 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.188+0000 7f4fbbfff700 1 --2- 172.21.15.162:0/3952808312 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4fbc13f400 0x7f4fbc09b330 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:34.200 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.188+0000 7f4fbbfff700 1 -- 172.21.15.162:0/3952808312 >> 172.21.15.162:0/3952808312 conn(0x7f4fbc0b54f0 msgr2=0x7f4fbc0b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:34.200 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.188+0000 7f4fbbfff700 1 -- 172.21.15.162:0/3952808312 shutdown_connections 2020-11-28T21:54:34.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.188+0000 7f4fbbfff700 1 -- 172.21.15.162:0/3952808312 wait complete. 2020-11-28T21:54:34.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.192+0000 7f4fbbfff700 1 Processor -- start 2020-11-28T21:54:34.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.192+0000 7f4fbbfff700 1 -- start start 2020-11-28T21:54:34.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.192+0000 7f4fbbfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4fbc0b1940 0x7f4fbc0674e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:34.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.192+0000 7f4fbbfff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4fbc0b45d0 0x7f4fbc0aa4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:34.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.192+0000 7f4fbbfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4fbc13f400 0x7f4fbc14e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:34.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.192+0000 7f4fbbfff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc0aad80 con 0x7f4fbc0b1940 2020-11-28T21:54:34.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.192+0000 7f4fbbfff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc0ac970 con 0x7f4fbc13f400 2020-11-28T21:54:34.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.192+0000 7f4fbbfff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc065ee0 con 0x7f4fbc0b45d0 2020-11-28T21:54:34.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.192+0000 7f4fbb7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4fbc13f400 0x7f4fbc14e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:34.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.192+0000 7f4fbb7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4fbc13f400 0x7f4fbc14e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43604/0 (socket says 172.21.15.162:43604) 2020-11-28T21:54:34.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.192+0000 7f4fbb7fe700 1 -- 172.21.15.162:0/1878464443 learned_addr learned my addr 172.21.15.162:0/1878464443 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:34.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.192+0000 7f4fbaffd700 1 --2- 172.21.15.162:0/1878464443 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4fbc0b1940 0x7f4fbc0674e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:34.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.192+0000 7f4fba7fc700 1 --2- 172.21.15.162:0/1878464443 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4fbc0b45d0 0x7f4fbc0aa4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:34.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.192+0000 7f4fbb7fe700 1 -- 172.21.15.162:0/1878464443 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4fbc0b45d0 msgr2=0x7f4fbc0aa4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:34.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.192+0000 7f4fbb7fe700 1 --2- 172.21.15.162:0/1878464443 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4fbc0b45d0 0x7f4fbc0aa4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:34.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.192+0000 7f4fbb7fe700 1 -- 172.21.15.162:0/1878464443 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4fbc0b1940 msgr2=0x7f4fbc0674e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:34.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.192+0000 7f4fbb7fe700 1 --2- 172.21.15.162:0/1878464443 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4fbc0b1940 0x7f4fbc0674e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:34.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.192+0000 7f4fbb7fe700 1 -- 172.21.15.162:0/1878464443 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fbc066180 con 0x7f4fbc13f400 2020-11-28T21:54:34.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.192+0000 7f4fbb7fe700 1 --2- 172.21.15.162:0/1878464443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4fbc13f400 0x7f4fbc14e230 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 rx=0x7f4fbc065300 tx=0x7f4fb4006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:34.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.192+0000 7f4fb3fff700 1 -- 172.21.15.162:0/1878464443 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fb400cae0 con 0x7f4fbc13f400 2020-11-28T21:54:34.211 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.192+0000 7f4fb3fff700 1 -- 172.21.15.162:0/1878464443 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb400cc40 con 0x7f4fbc13f400 2020-11-28T21:54:34.212 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.192+0000 7f4fbbfff700 1 -- 172.21.15.162:0/1878464443 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fbc0668c0 con 0x7f4fbc13f400 2020-11-28T21:54:34.212 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.192+0000 7f4fb3fff700 1 -- 172.21.15.162:0/1878464443 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fb40123f0 con 0x7f4fbc13f400 2020-11-28T21:54:34.212 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.192+0000 7f4fbbfff700 1 -- 172.21.15.162:0/1878464443 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fbc065a60 con 0x7f4fbc13f400 2020-11-28T21:54:34.213 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.192+0000 7f4fbbfff700 1 -- 172.21.15.162:0/1878464443 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fbc057050 con 0x7f4fbc13f400 2020-11-28T21:54:34.213 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.192+0000 7f4fb3fff700 1 -- 172.21.15.162:0/1878464443 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4fb4012550 con 0x7f4fbc13f400 2020-11-28T21:54:34.213 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.192+0000 7f4fb3fff700 1 --2- 172.21.15.162:0/1878464443 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4f940553b0 0x7f4f94057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:34.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.196+0000 7f4fb3fff700 1 -- 172.21.15.162:0/1878464443 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4fb406e4d0 con 0x7f4fbc13f400 2020-11-28T21:54:34.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.196+0000 7f4fbaffd700 1 --2- 172.21.15.162:0/1878464443 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4f940553b0 0x7f4f94057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:34.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.196+0000 7f4fbaffd700 1 --2- 172.21.15.162:0/1878464443 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4f940553b0 0x7f4f94057870 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f4fa400b000 tx=0x7f4fa4009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:34.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.200+0000 7f4fb3fff700 1 -- 172.21.15.162:0/1878464443 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4fb40446d0 con 0x7f4fbc13f400 2020-11-28T21:54:34.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.340+0000 7f4fbbfff700 1 -- 172.21.15.162:0/1878464443 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4fbc05d5e0 con 0x7f4fbc13f400 2020-11-28T21:54:34.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.340+0000 7f4fb3fff700 1 -- 172.21.15.162:0/1878464443 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+16148 (secure 0 0 0) 0x7f4fb4047cf0 con 0x7f4fbc13f400 2020-11-28T21:54:34.344 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:54:34.344 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":109,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:54:27.911877+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:54:25.418443+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2020-11-28T21:54:25.913773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:54:34.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.340+0000 7f4fbbfff700 1 -- 172.21.15.162:0/1878464443 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4f940553b0 msgr2=0x7f4f94057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:34.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.340+0000 7f4fbbfff700 1 --2- 172.21.15.162:0/1878464443 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4f940553b0 0x7f4f94057870 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f4fa400b000 tx=0x7f4fa4009040).stop 2020-11-28T21:54:34.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.340+0000 7f4fbbfff700 1 -- 172.21.15.162:0/1878464443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4fbc13f400 msgr2=0x7f4fbc14e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:34.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.340+0000 7f4fbbfff700 1 --2- 172.21.15.162:0/1878464443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4fbc13f400 0x7f4fbc14e230 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 rx=0x7f4fbc065300 tx=0x7f4fb4006dc0).stop 2020-11-28T21:54:34.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.344+0000 7f4fbbfff700 1 -- 172.21.15.162:0/1878464443 shutdown_connections 2020-11-28T21:54:34.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.344+0000 7f4fbbfff700 1 --2- 172.21.15.162:0/1878464443 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4f940553b0 0x7f4f94057870 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:34.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.344+0000 7f4fbbfff700 1 --2- 172.21.15.162:0/1878464443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4fbc13f400 0x7f4fbc14e230 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:34.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.344+0000 7f4fbbfff700 1 --2- 172.21.15.162:0/1878464443 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4fbc0b1940 0x7f4fbc0674e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:34.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.344+0000 7f4fbbfff700 1 --2- 172.21.15.162:0/1878464443 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4fbc0b45d0 0x7f4fbc0aa4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:34.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.344+0000 7f4fbbfff700 1 -- 172.21.15.162:0/1878464443 >> 172.21.15.162:0/1878464443 conn(0x7f4fbc0b54f0 msgr2=0x7f4fbc1416b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:34.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.344+0000 7f4fbbfff700 1 -- 172.21.15.162:0/1878464443 shutdown_connections 2020-11-28T21:54:34.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.344+0000 7f4fbbfff700 1 -- 172.21.15.162:0/1878464443 wait complete. 2020-11-28T21:54:34.358 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-28T21:54:25.418443+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2020-11-28T21:54:25.913773+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-28T21:54:34.360 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:54:34.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.484+0000 7fc416c0a700 1 Processor -- start 2020-11-28T21:54:34.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.484+0000 7fc416c0a700 1 -- start start 2020-11-28T21:54:34.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.488+0000 7fc416c0a700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc4100649c0 0x7fc410143580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:34.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.488+0000 7fc416c0a700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc4100b0bf0 0x7fc41013fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:34.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.488+0000 7fc416c0a700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc410064da0 con 0x7fc410060200 2020-11-28T21:54:34.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.488+0000 7fc416c0a700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4101429b0 con 0x7fc4100649c0 2020-11-28T21:54:34.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.488+0000 7fc416c0a700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc410142ba0 con 0x7fc4100b0bf0 2020-11-28T21:54:34.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.488+0000 7fc4149a6700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc4100649c0 0x7fc410143580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:34.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.488+0000 7fc4149a6700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc4100649c0 0x7fc410143580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43608/0 (socket says 172.21.15.162:43608) 2020-11-28T21:54:34.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.488+0000 7fc4149a6700 1 -- 172.21.15.162:0/2207234379 learned_addr learned my addr 172.21.15.162:0/2207234379 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:34.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.488+0000 7fc4151a7700 1 --2- 172.21.15.162:0/2207234379 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc4100b0bf0 0x7fc41013fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:34.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.488+0000 7fc4149a6700 1 -- 172.21.15.162:0/2207234379 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc4100b0bf0 msgr2=0x7fc41013fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:34.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.488+0000 7fc4149a6700 1 --2- 172.21.15.162:0/2207234379 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc4100b0bf0 0x7fc41013fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:34.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.488+0000 7fc4149a6700 1 -- 172.21.15.162:0/2207234379 >> v1:172.21.15.118:6789/0 conn(0x7fc410060200 legacy=0x7fc41005b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:34.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.488+0000 7fc4149a6700 1 -- 172.21.15.162:0/2207234379 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc410142e40 con 0x7fc4100649c0 2020-11-28T21:54:34.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.488+0000 7fc4149a6700 1 --2- 172.21.15.162:0/2207234379 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4100649c0 0x7fc410143580 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 rx=0x7fc4000032b0 tx=0x7fc400009270).ready entity=mon.1 client_cookie=c4e62b1832991765 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:34.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.488+0000 7fc40f7fe700 1 -- 172.21.15.162:0/2207234379 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc40000bc00 con 0x7fc4100649c0 2020-11-28T21:54:34.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.488+0000 7fc40f7fe700 1 -- 172.21.15.162:0/2207234379 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc40000bd60 con 0x7fc4100649c0 2020-11-28T21:54:34.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.488+0000 7fc40f7fe700 1 -- 172.21.15.162:0/2207234379 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc400011400 con 0x7fc4100649c0 2020-11-28T21:54:34.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.488+0000 7fc416c0a700 1 -- 172.21.15.162:0/2207234379 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4100649c0 msgr2=0x7fc410143580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:34.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.488+0000 7fc416c0a700 1 --2- 172.21.15.162:0/2207234379 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4100649c0 0x7fc410143580 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 rx=0x7fc4000032b0 tx=0x7fc400009270).stop 2020-11-28T21:54:34.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.488+0000 7fc416c0a700 1 -- 172.21.15.162:0/2207234379 shutdown_connections 2020-11-28T21:54:34.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.488+0000 7fc416c0a700 1 --2- 172.21.15.162:0/2207234379 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc4100b0bf0 0x7fc41013fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:34.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.488+0000 7fc416c0a700 1 --2- 172.21.15.162:0/2207234379 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4100649c0 0x7fc410143580 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:34.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.488+0000 7fc416c0a700 1 -- 172.21.15.162:0/2207234379 >> 172.21.15.162:0/2207234379 conn(0x7fc410061740 msgr2=0x7fc410061b50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:34.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.492+0000 7fc416c0a700 1 -- 172.21.15.162:0/2207234379 shutdown_connections 2020-11-28T21:54:34.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.492+0000 7fc416c0a700 1 -- 172.21.15.162:0/2207234379 wait complete. 2020-11-28T21:54:34.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.492+0000 7fc416c0a700 1 Processor -- start 2020-11-28T21:54:34.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.492+0000 7fc416c0a700 1 -- start start 2020-11-28T21:54:34.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.496+0000 7fc416c0a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc410060200 0x7fc410066420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:34.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.496+0000 7fc416c0a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4100649c0 0x7fc4100ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:34.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.496+0000 7fc416c0a700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc4100b0bf0 0x7fc4100ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:34.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.496+0000 7fc416c0a700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc410066e60 con 0x7fc410060200 2020-11-28T21:54:34.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.496+0000 7fc416c0a700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4101526c0 con 0x7fc4100649c0 2020-11-28T21:54:34.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.496+0000 7fc416c0a700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc410152960 con 0x7fc4100b0bf0 2020-11-28T21:54:34.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.496+0000 7fc40ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4100649c0 0x7fc4100ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:34.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.496+0000 7fc4149a6700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc410060200 0x7fc410066420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:34.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.496+0000 7fc40ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4100649c0 0x7fc4100ae1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43616/0 (socket says 172.21.15.162:43616) 2020-11-28T21:54:34.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.496+0000 7fc40ffff700 1 -- 172.21.15.162:0/1750224573 learned_addr learned my addr 172.21.15.162:0/1750224573 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:34.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.496+0000 7fc4151a7700 1 --2- 172.21.15.162:0/1750224573 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc4100b0bf0 0x7fc4100ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:34.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.496+0000 7fc40ffff700 1 -- 172.21.15.162:0/1750224573 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc4100b0bf0 msgr2=0x7fc4100ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:34.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.496+0000 7fc40ffff700 1 --2- 172.21.15.162:0/1750224573 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc4100b0bf0 0x7fc4100ae6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:34.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.496+0000 7fc40ffff700 1 -- 172.21.15.162:0/1750224573 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc410060200 msgr2=0x7fc410066420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:34.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.496+0000 7fc40ffff700 1 --2- 172.21.15.162:0/1750224573 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc410060200 0x7fc410066420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:34.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.496+0000 7fc40ffff700 1 -- 172.21.15.162:0/1750224573 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc410140fd0 con 0x7fc4100649c0 2020-11-28T21:54:34.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.496+0000 7fc4151a7700 1 --2- 172.21.15.162:0/1750224573 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc4100b0bf0 0x7fc4100ae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:54:34.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.496+0000 7fc40ffff700 1 --2- 172.21.15.162:0/1750224573 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4100649c0 0x7fc4100ae1f0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 rx=0x7fc410063400 tx=0x7fc404007bb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:34.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.496+0000 7fc40dffb700 1 -- 172.21.15.162:0/1750224573 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc404007030 con 0x7fc4100649c0 2020-11-28T21:54:34.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.496+0000 7fc40dffb700 1 -- 172.21.15.162:0/1750224573 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc404007190 con 0x7fc4100649c0 2020-11-28T21:54:34.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.496+0000 7fc40dffb700 1 -- 172.21.15.162:0/1750224573 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4040064e0 con 0x7fc4100649c0 2020-11-28T21:54:34.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.496+0000 7fc416c0a700 1 -- 172.21.15.162:0/1750224573 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc41014f690 con 0x7fc4100649c0 2020-11-28T21:54:34.521 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.496+0000 7fc416c0a700 1 -- 172.21.15.162:0/1750224573 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4100aeeb0 con 0x7fc4100649c0 2020-11-28T21:54:34.521 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.496+0000 7fc416c0a700 1 -- 172.21.15.162:0/1750224573 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc410067060 con 0x7fc4100649c0 2020-11-28T21:54:34.521 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.500+0000 7fc40dffb700 1 -- 172.21.15.162:0/1750224573 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc404010070 con 0x7fc4100649c0 2020-11-28T21:54:34.522 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.500+0000 7fc40dffb700 1 --2- 172.21.15.162:0/1750224573 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc3f00554e0 0x7fc3f00579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:34.522 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.500+0000 7fc40dffb700 1 -- 172.21.15.162:0/1750224573 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc40406d4d0 con 0x7fc4100649c0 2020-11-28T21:54:34.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.500+0000 7fc4149a6700 1 --2- 172.21.15.162:0/1750224573 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc3f00554e0 0x7fc3f00579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:34.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.500+0000 7fc4149a6700 1 --2- 172.21.15.162:0/1750224573 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc3f00554e0 0x7fc3f00579a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7fc40001a000 tx=0x7fc400016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:34.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.504+0000 7fc40dffb700 1 -- 172.21.15.162:0/1750224573 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc4040437d0 con 0x7fc4100649c0 2020-11-28T21:54:34.651 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.644+0000 7fc416c0a700 1 -- 172.21.15.162:0/1750224573 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fc4100ae030 con 0x7fc4100649c0 2020-11-28T21:54:34.651 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.648+0000 7fc40dffb700 1 -- 172.21.15.162:0/1750224573 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v109) v1 ==== 110+0+10 (secure 0 0 0) 0x7fc404018030 con 0x7fc4100649c0 2020-11-28T21:54:34.651 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:54:34.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.648+0000 7fc416c0a700 1 -- 172.21.15.162:0/1750224573 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc3f00554e0 msgr2=0x7fc3f00579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:34.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.648+0000 7fc416c0a700 1 --2- 172.21.15.162:0/1750224573 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc3f00554e0 0x7fc3f00579a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7fc40001a000 tx=0x7fc400016000).stop 2020-11-28T21:54:34.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.648+0000 7fc416c0a700 1 -- 172.21.15.162:0/1750224573 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4100649c0 msgr2=0x7fc4100ae1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:34.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.648+0000 7fc416c0a700 1 --2- 172.21.15.162:0/1750224573 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4100649c0 0x7fc4100ae1f0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 rx=0x7fc410063400 tx=0x7fc404007bb0).stop 2020-11-28T21:54:34.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.648+0000 7fc416c0a700 1 -- 172.21.15.162:0/1750224573 shutdown_connections 2020-11-28T21:54:34.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.648+0000 7fc416c0a700 1 --2- 172.21.15.162:0/1750224573 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc3f00554e0 0x7fc3f00579a0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:34.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.648+0000 7fc416c0a700 1 --2- 172.21.15.162:0/1750224573 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc4100b0bf0 0x7fc4100ae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:34.655 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.648+0000 7fc416c0a700 1 --2- 172.21.15.162:0/1750224573 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc410060200 0x7fc410066420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:34.655 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.648+0000 7fc416c0a700 1 --2- 172.21.15.162:0/1750224573 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4100649c0 0x7fc4100ae1f0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:34.655 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.648+0000 7fc416c0a700 1 -- 172.21.15.162:0/1750224573 >> 172.21.15.162:0/1750224573 conn(0x7fc410061740 msgr2=0x7fc41013e480 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:34.655 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.648+0000 7fc416c0a700 1 -- 172.21.15.162:0/1750224573 shutdown_connections 2020-11-28T21:54:34.655 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.648+0000 7fc416c0a700 1 -- 172.21.15.162:0/1750224573 wait complete. 2020-11-28T21:54:34.670 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:54:34.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.784+0000 7fe4faa82700 1 Processor -- start 2020-11-28T21:54:34.788 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.784+0000 7fe4faa82700 1 -- start start 2020-11-28T21:54:34.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.784+0000 7fe4faa82700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe4f413cf30 0x7fe4f40ac350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:34.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.784+0000 7fe4faa82700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4f40b1360 con 0x7fe4f405cca0 2020-11-28T21:54:34.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.784+0000 7fe4faa82700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4f40b25c0 con 0x7fe4f413cb50 2020-11-28T21:54:34.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.784+0000 7fe4faa82700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f40add60 con 0x7fe4f413cf30 2020-11-28T21:54:34.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.784+0000 7fe4f9a80700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fe4f413cb50 0x7fe4f4140180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:50468/0 (socket says 172.21.15.162:50468) 2020-11-28T21:54:34.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.784+0000 7fe4f9a80700 1 -- 172.21.15.162:0/3556653087 learned_addr learned my addr 172.21.15.162:0/3556653087 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:34.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.784+0000 7fe4fa281700 1 --2- 172.21.15.162:0/3556653087 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe4f413cf30 0x7fe4f40ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:34.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.784+0000 7fe4f8a7e700 1 -- 172.21.15.162:0/3556653087 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 274838649 0 0) 0x7fe4f40b25c0 con 0x7fe4f413cb50 2020-11-28T21:54:34.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.784+0000 7fe4f8a7e700 1 -- 172.21.15.162:0/3556653087 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4dc003230 con 0x7fe4f413cb50 2020-11-28T21:54:34.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.784+0000 7fe4f8a7e700 1 -- 172.21.15.162:0/3556653087 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3603824809 0 0) 0x7fe4f40b1360 con 0x7fe4f405cca0 2020-11-28T21:54:34.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.784+0000 7fe4f8a7e700 1 -- 172.21.15.162:0/3556653087 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4f40b25c0 con 0x7fe4f405cca0 2020-11-28T21:54:34.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.788+0000 7fe4fa281700 1 -- 172.21.15.162:0/3556653087 >> v1:172.21.15.162:6789/0 conn(0x7fe4f413cb50 legacy=0x7fe4f4140180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:34.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.788+0000 7fe4fa281700 1 -- 172.21.15.162:0/3556653087 >> v1:172.21.15.118:6789/0 conn(0x7fe4f405cca0 legacy=0x7fe4f40b1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:34.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.788+0000 7fe4fa281700 1 -- 172.21.15.162:0/3556653087 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f40ae8c0 con 0x7fe4f413cf30 2020-11-28T21:54:34.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.788+0000 7fe4fa281700 1 --2- 172.21.15.162:0/3556653087 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe4f413cf30 0x7fe4f40ac350 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fe4ec0032b0 tx=0x7fe4ec0091a0).ready entity=mon.2 client_cookie=66855c65e0d6573e server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:34.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.788+0000 7fe4f8a7e700 1 -- 172.21.15.162:0/3556653087 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe4ec00b940 con 0x7fe4f413cf30 2020-11-28T21:54:34.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.788+0000 7fe4f8a7e700 1 -- 172.21.15.162:0/3556653087 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4ec00baa0 con 0x7fe4f413cf30 2020-11-28T21:54:34.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.788+0000 7fe4f8a7e700 1 -- 172.21.15.162:0/3556653087 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe4ec00bc10 con 0x7fe4f413cf30 2020-11-28T21:54:34.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.788+0000 7fe4faa82700 1 -- 172.21.15.162:0/3556653087 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe4f413cf30 msgr2=0x7fe4f40ac350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:34.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.788+0000 7fe4faa82700 1 --2- 172.21.15.162:0/3556653087 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe4f413cf30 0x7fe4f40ac350 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fe4ec0032b0 tx=0x7fe4ec0091a0).stop 2020-11-28T21:54:34.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.788+0000 7fe4faa82700 1 -- 172.21.15.162:0/3556653087 shutdown_connections 2020-11-28T21:54:34.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.788+0000 7fe4faa82700 1 --2- 172.21.15.162:0/3556653087 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe4f413cf30 0x7fe4f40ac350 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:34.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.788+0000 7fe4faa82700 1 -- 172.21.15.162:0/3556653087 >> 172.21.15.162:0/3556653087 conn(0x7fe4f40b57d0 msgr2=0x7fe4f40b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:34.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.788+0000 7fe4faa82700 1 -- 172.21.15.162:0/3556653087 shutdown_connections 2020-11-28T21:54:34.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.788+0000 7fe4faa82700 1 -- 172.21.15.162:0/3556653087 wait complete. 2020-11-28T21:54:34.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.788+0000 7fe4faa82700 1 Processor -- start 2020-11-28T21:54:34.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.788+0000 7fe4faa82700 1 -- start start 2020-11-28T21:54:34.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.792+0000 7fe4faa82700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe4f405cca0 0x7fe4f4146fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:34.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.792+0000 7fe4faa82700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4f413cb50 0x7fe4f40aaa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:34.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.792+0000 7fe4faa82700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe4f413cf30 0x7fe4f4144170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:34.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.792+0000 7fe4faa82700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f40abe20 con 0x7fe4f405cca0 2020-11-28T21:54:34.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.792+0000 7fe4faa82700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f40ab340 con 0x7fe4f413cb50 2020-11-28T21:54:34.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.792+0000 7fe4faa82700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f40ab5e0 con 0x7fe4f413cf30 2020-11-28T21:54:34.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.792+0000 7fe4f927f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4f413cb50 0x7fe4f40aaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:34.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.792+0000 7fe4f927f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4f413cb50 0x7fe4f40aaa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43628/0 (socket says 172.21.15.162:43628) 2020-11-28T21:54:34.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.792+0000 7fe4f927f700 1 -- 172.21.15.162:0/3016172384 learned_addr learned my addr 172.21.15.162:0/3016172384 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:34.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.792+0000 7fe4fa281700 1 --2- 172.21.15.162:0/3016172384 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe4f413cf30 0x7fe4f4144170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:34.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.792+0000 7fe4f9a80700 1 --2- 172.21.15.162:0/3016172384 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe4f405cca0 0x7fe4f4146fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:34.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.792+0000 7fe4f927f700 1 -- 172.21.15.162:0/3016172384 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe4f413cf30 msgr2=0x7fe4f4144170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:34.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.792+0000 7fe4f927f700 1 --2- 172.21.15.162:0/3016172384 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe4f413cf30 0x7fe4f4144170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:34.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.792+0000 7fe4f927f700 1 -- 172.21.15.162:0/3016172384 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe4f405cca0 msgr2=0x7fe4f4146fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:34.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.792+0000 7fe4f927f700 1 --2- 172.21.15.162:0/3016172384 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe4f405cca0 0x7fe4f4146fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:34.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.792+0000 7fe4f927f700 1 -- 172.21.15.162:0/3016172384 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f4141d70 con 0x7fe4f413cb50 2020-11-28T21:54:34.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.792+0000 7fe4f927f700 1 --2- 172.21.15.162:0/3016172384 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4f413cb50 0x7fe4f40aaa20 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 rx=0x7fe4e40010d0 tx=0x7fe4e4007a60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:34.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.792+0000 7fe4e2ffd700 1 -- 172.21.15.162:0/3016172384 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe4e400e4a0 con 0x7fe4f413cb50 2020-11-28T21:54:34.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.792+0000 7fe4e2ffd700 1 -- 172.21.15.162:0/3016172384 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e400e600 con 0x7fe4f413cb50 2020-11-28T21:54:34.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.792+0000 7fe4faa82700 1 -- 172.21.15.162:0/3016172384 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4f40ab880 con 0x7fe4f413cb50 2020-11-28T21:54:34.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.792+0000 7fe4e2ffd700 1 -- 172.21.15.162:0/3016172384 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe4e400e960 con 0x7fe4f413cb50 2020-11-28T21:54:34.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.792+0000 7fe4faa82700 1 -- 172.21.15.162:0/3016172384 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4f4146cf0 con 0x7fe4f413cb50 2020-11-28T21:54:34.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.792+0000 7fe4faa82700 1 -- 172.21.15.162:0/3016172384 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4f4141f00 con 0x7fe4f413cb50 2020-11-28T21:54:34.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.796+0000 7fe4e2ffd700 1 -- 172.21.15.162:0/3016172384 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe4e4019030 con 0x7fe4f413cb50 2020-11-28T21:54:34.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.796+0000 7fe4e2ffd700 1 --2- 172.21.15.162:0/3016172384 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe4d4055440 0x7fe4d4057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:34.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.796+0000 7fe4e2ffd700 1 -- 172.21.15.162:0/3016172384 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe4e406d420 con 0x7fe4f413cb50 2020-11-28T21:54:34.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.796+0000 7fe4f9a80700 1 --2- 172.21.15.162:0/3016172384 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe4d4055440 0x7fe4d4057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:34.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.796+0000 7fe4f9a80700 1 --2- 172.21.15.162:0/3016172384 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe4d4055440 0x7fe4d4057900 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7fe4f000b000 tx=0x7fe4f0009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:34.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.800+0000 7fe4e2ffd700 1 -- 172.21.15.162:0/3016172384 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe4e4043720 con 0x7fe4f413cb50 2020-11-28T21:54:34.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.940+0000 7fe4faa82700 1 -- 172.21.15.162:0/3016172384 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe4f4057050 con 0x7fe4f413cb50 2020-11-28T21:54:34.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.940+0000 7fe4e2ffd700 1 -- 172.21.15.162:0/3016172384 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v109) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe4e4017020 con 0x7fe4f413cb50 2020-11-28T21:54:34.945 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:54:34.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.944+0000 7fe4faa82700 1 -- 172.21.15.162:0/3016172384 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe4d4055440 msgr2=0x7fe4d4057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:34.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.944+0000 7fe4faa82700 1 --2- 172.21.15.162:0/3016172384 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe4d4055440 0x7fe4d4057900 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7fe4f000b000 tx=0x7fe4f0009040).stop 2020-11-28T21:54:34.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.944+0000 7fe4faa82700 1 -- 172.21.15.162:0/3016172384 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4f413cb50 msgr2=0x7fe4f40aaa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:34.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.944+0000 7fe4faa82700 1 --2- 172.21.15.162:0/3016172384 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4f413cb50 0x7fe4f40aaa20 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 rx=0x7fe4e40010d0 tx=0x7fe4e4007a60).stop 2020-11-28T21:54:34.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.944+0000 7fe4faa82700 1 -- 172.21.15.162:0/3016172384 shutdown_connections 2020-11-28T21:54:34.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.944+0000 7fe4faa82700 1 --2- 172.21.15.162:0/3016172384 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe4d4055440 0x7fe4d4057900 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:34.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.944+0000 7fe4faa82700 1 --2- 172.21.15.162:0/3016172384 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe4f413cf30 0x7fe4f4144170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:34.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.944+0000 7fe4faa82700 1 --2- 172.21.15.162:0/3016172384 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe4f405cca0 0x7fe4f4146fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:34.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.944+0000 7fe4faa82700 1 --2- 172.21.15.162:0/3016172384 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4f413cb50 0x7fe4f40aaa20 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:34.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.944+0000 7fe4faa82700 1 -- 172.21.15.162:0/3016172384 >> 172.21.15.162:0/3016172384 conn(0x7fe4f40b57d0 msgr2=0x7fe4f41436a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:34.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.944+0000 7fe4faa82700 1 -- 172.21.15.162:0/3016172384 shutdown_connections 2020-11-28T21:54:34.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:34.944+0000 7fe4faa82700 1 -- 172.21.15.162:0/3016172384 wait complete. 2020-11-28T21:54:34.962 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-28T21:54:35.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.084+0000 7f1527d02700 1 Processor -- start 2020-11-28T21:54:35.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.084+0000 7f1527d02700 1 -- start start 2020-11-28T21:54:35.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.084+0000 7f1527d02700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f15200b9ce0 0x7f15201433c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:35.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.084+0000 7f1527d02700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f15200605c0 0x7f1520067610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:35.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.084+0000 7f1527d02700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1520143950 con 0x7f15200b3c30 2020-11-28T21:54:35.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.084+0000 7f1527d02700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15200ba0c0 con 0x7f15200b9ce0 2020-11-28T21:54:35.094 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.084+0000 7f1527d02700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1520066c30 con 0x7f15200605c0 2020-11-28T21:54:35.094 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.084+0000 7f1525a9e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f15200b9ce0 0x7f15201433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:35.095 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.084+0000 7f152629f700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f15200b3c30 0x7f152013fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:50484/0 (socket says 172.21.15.162:50484) 2020-11-28T21:54:35.095 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.084+0000 7f152629f700 1 -- 172.21.15.162:0/2462869765 learned_addr learned my addr 172.21.15.162:0/2462869765 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:35.095 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.084+0000 7f152529d700 1 --2- 172.21.15.162:0/2462869765 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f15200605c0 0x7f1520067610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:35.095 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.084+0000 7f1525a9e700 1 -- 172.21.15.162:0/2462869765 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f15200605c0 msgr2=0x7f1520067610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:35.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.084+0000 7f1525a9e700 1 --2- 172.21.15.162:0/2462869765 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f15200605c0 0x7f1520067610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:35.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.084+0000 7f1524a9c700 1 -- 172.21.15.162:0/2462869765 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 80921657 0 0) 0x7f1520143950 con 0x7f15200b3c30 2020-11-28T21:54:35.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.084+0000 7f1525a9e700 1 -- 172.21.15.162:0/2462869765 >> v1:172.21.15.162:6789/0 conn(0x7f15200b3c30 legacy=0x7f152013fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:35.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.084+0000 7f1525a9e700 1 -- 172.21.15.162:0/2462869765 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1520066ed0 con 0x7f15200b9ce0 2020-11-28T21:54:35.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.084+0000 7f1525a9e700 1 --2- 172.21.15.162:0/2462869765 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15200b9ce0 0x7f15201433c0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 rx=0x7f151c002f30 tx=0x7f151c006d60).ready entity=mon.1 client_cookie=bcf423d88ced36bd server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:35.098 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.084+0000 7f1524a9c700 1 -- 172.21.15.162:0/2462869765 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f151c0097f0 con 0x7f15200b9ce0 2020-11-28T21:54:35.098 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.084+0000 7f1524a9c700 1 -- 172.21.15.162:0/2462869765 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f151c009950 con 0x7f15200b9ce0 2020-11-28T21:54:35.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.084+0000 7f1524a9c700 1 -- 172.21.15.162:0/2462869765 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f151c009ac0 con 0x7f15200b9ce0 2020-11-28T21:54:35.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.084+0000 7f1527d02700 1 -- 172.21.15.162:0/2462869765 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15200b9ce0 msgr2=0x7f15201433c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:35.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.084+0000 7f1527d02700 1 --2- 172.21.15.162:0/2462869765 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15200b9ce0 0x7f15201433c0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 rx=0x7f151c002f30 tx=0x7f151c006d60).stop 2020-11-28T21:54:35.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.088+0000 7f1527d02700 1 -- 172.21.15.162:0/2462869765 shutdown_connections 2020-11-28T21:54:35.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.088+0000 7f1527d02700 1 --2- 172.21.15.162:0/2462869765 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15200b9ce0 0x7f15201433c0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:35.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.088+0000 7f1527d02700 1 --2- 172.21.15.162:0/2462869765 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f15200605c0 0x7f1520067610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:35.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.088+0000 7f1527d02700 1 -- 172.21.15.162:0/2462869765 >> 172.21.15.162:0/2462869765 conn(0x7f152005c430 msgr2=0x7f15200b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:35.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.088+0000 7f1527d02700 1 -- 172.21.15.162:0/2462869765 shutdown_connections 2020-11-28T21:54:35.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.088+0000 7f1527d02700 1 -- 172.21.15.162:0/2462869765 wait complete. 2020-11-28T21:54:35.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.088+0000 7f1527d02700 1 Processor -- start 2020-11-28T21:54:35.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.088+0000 7f1527d02700 1 -- start start 2020-11-28T21:54:35.104 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.088+0000 7f1527d02700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f15200605c0 0x7f15200b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:35.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.088+0000 7f1527d02700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15200b3c30 0x7f15200ad4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:35.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.088+0000 7f1527d02700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f15200b9ce0 0x7f1520149450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:35.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.088+0000 7f1527d02700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15200ad0b0 con 0x7f15200605c0 2020-11-28T21:54:35.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.088+0000 7f1527d02700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1520149220 con 0x7f15200b3c30 2020-11-28T21:54:35.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.088+0000 7f1527d02700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15201486a0 con 0x7f15200b9ce0 2020-11-28T21:54:35.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.088+0000 7f152529d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15200b3c30 0x7f15200ad4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:35.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.088+0000 7f152529d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15200b3c30 0x7f15200ad4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43644/0 (socket says 172.21.15.162:43644) 2020-11-28T21:54:35.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.088+0000 7f152529d700 1 -- 172.21.15.162:0/467175656 learned_addr learned my addr 172.21.15.162:0/467175656 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:35.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.088+0000 7f1525a9e700 1 --2- 172.21.15.162:0/467175656 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f15200605c0 0x7f15200b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:35.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.088+0000 7f152629f700 1 --2- 172.21.15.162:0/467175656 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f15200b9ce0 0x7f1520149450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:35.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.092+0000 7f152529d700 1 -- 172.21.15.162:0/467175656 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f15200b9ce0 msgr2=0x7f1520149450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:35.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.092+0000 7f152529d700 1 --2- 172.21.15.162:0/467175656 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f15200b9ce0 0x7f1520149450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:35.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.092+0000 7f152529d700 1 -- 172.21.15.162:0/467175656 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f15200605c0 msgr2=0x7f15200b8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:35.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.092+0000 7f152529d700 1 --2- 172.21.15.162:0/467175656 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f15200605c0 0x7f15200b8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:35.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.092+0000 7f152529d700 1 -- 172.21.15.162:0/467175656 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1520148940 con 0x7f15200b3c30 2020-11-28T21:54:35.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.092+0000 7f152629f700 1 --2- 172.21.15.162:0/467175656 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f15200b9ce0 0x7f1520149450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:54:35.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.092+0000 7f152529d700 1 --2- 172.21.15.162:0/467175656 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15200b3c30 0x7f15200ad4a0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 rx=0x7f1510003070 tx=0x7f1510008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:35.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.092+0000 7f150effd700 1 -- 172.21.15.162:0/467175656 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1510009980 con 0x7f15200b3c30 2020-11-28T21:54:35.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.092+0000 7f150effd700 1 -- 172.21.15.162:0/467175656 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1510009ae0 con 0x7f15200b3c30 2020-11-28T21:54:35.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.092+0000 7f1527d02700 1 -- 172.21.15.162:0/467175656 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15200af9f0 con 0x7f15200b3c30 2020-11-28T21:54:35.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.092+0000 7f150effd700 1 -- 172.21.15.162:0/467175656 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1510009e40 con 0x7f15200b3c30 2020-11-28T21:54:35.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.092+0000 7f1527d02700 1 -- 172.21.15.162:0/467175656 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15200adaa0 con 0x7f15200b3c30 2020-11-28T21:54:35.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.092+0000 7f1527d02700 1 -- 172.21.15.162:0/467175656 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1520148ad0 con 0x7f15200b3c30 2020-11-28T21:54:35.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.092+0000 7f150effd700 1 -- 172.21.15.162:0/467175656 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1510013480 con 0x7f15200b3c30 2020-11-28T21:54:35.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.092+0000 7f150effd700 1 --2- 172.21.15.162:0/467175656 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f15000553b0 0x7f1500057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:35.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.092+0000 7f1525a9e700 1 --2- 172.21.15.162:0/467175656 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f15000553b0 0x7f1500057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:35.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.092+0000 7f150effd700 1 -- 172.21.15.162:0/467175656 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f151006e960 con 0x7f15200b3c30 2020-11-28T21:54:35.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.096+0000 7f1525a9e700 1 --2- 172.21.15.162:0/467175656 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f15000553b0 0x7f1500057870 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7f151c01b000 tx=0x7f151c016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:35.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.104+0000 7f150effd700 1 -- 172.21.15.162:0/467175656 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1510044af0 con 0x7f15200b3c30 2020-11-28T21:54:35.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.244+0000 7f1527d02700 1 -- 172.21.15.162:0/467175656 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1520057050 con 0x7f15200b3c30 2020-11-28T21:54:35.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.244+0000 7f150effd700 1 -- 172.21.15.162:0/467175656 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v109) v1 ==== 104+0+10 (secure 0 0 0) 0x7f1510048110 con 0x7f15200b3c30 2020-11-28T21:54:35.248 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:54:35.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.244+0000 7f1527d02700 1 -- 172.21.15.162:0/467175656 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f15000553b0 msgr2=0x7f1500057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:35.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.244+0000 7f1527d02700 1 --2- 172.21.15.162:0/467175656 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f15000553b0 0x7f1500057870 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7f151c01b000 tx=0x7f151c016000).stop 2020-11-28T21:54:35.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.244+0000 7f1527d02700 1 -- 172.21.15.162:0/467175656 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15200b3c30 msgr2=0x7f15200ad4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:35.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.244+0000 7f1527d02700 1 --2- 172.21.15.162:0/467175656 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15200b3c30 0x7f15200ad4a0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 rx=0x7f1510003070 tx=0x7f1510008040).stop 2020-11-28T21:54:35.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.248+0000 7f1527d02700 1 -- 172.21.15.162:0/467175656 shutdown_connections 2020-11-28T21:54:35.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.248+0000 7f1527d02700 1 --2- 172.21.15.162:0/467175656 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f15000553b0 0x7f1500057870 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:35.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.248+0000 7f1527d02700 1 --2- 172.21.15.162:0/467175656 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f15200b9ce0 0x7f1520149450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:35.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.248+0000 7f1527d02700 1 --2- 172.21.15.162:0/467175656 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f15200605c0 0x7f15200b8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:35.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.248+0000 7f1527d02700 1 --2- 172.21.15.162:0/467175656 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15200b3c30 0x7f15200ad4a0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:35.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.248+0000 7f1527d02700 1 -- 172.21.15.162:0/467175656 >> 172.21.15.162:0/467175656 conn(0x7f152005c430 msgr2=0x7f152009d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:35.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.248+0000 7f1527d02700 1 -- 172.21.15.162:0/467175656 shutdown_connections 2020-11-28T21:54:35.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.248+0000 7f1527d02700 1 -- 172.21.15.162:0/467175656 wait complete. 2020-11-28T21:54:35.262 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-28T21:54:35.377 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.372+0000 7ff8c9790700 1 Processor -- start 2020-11-28T21:54:35.378 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.372+0000 7ff8c9790700 1 -- start start 2020-11-28T21:54:35.379 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.376+0000 7ff8c9790700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff8c40b3950 0x7ff8c40b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:35.380 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.376+0000 7ff8c9790700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8c40b1f50 con 0x7ff8c40b18c0 2020-11-28T21:54:35.380 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.376+0000 7ff8c9790700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8c4143e90 con 0x7ff8c40b44e0 2020-11-28T21:54:35.380 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.376+0000 7ff8c9790700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8c40aab50 con 0x7ff8c40b3950 2020-11-28T21:54:35.380 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.376+0000 7ff8c8f8f700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7ff8c40b44e0 0x7ff8c40b2c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:50498/0 (socket says 172.21.15.162:50498) 2020-11-28T21:54:35.381 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.376+0000 7ff8c8f8f700 1 -- 172.21.15.162:0/3308041660 learned_addr learned my addr 172.21.15.162:0/3308041660 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:35.381 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.376+0000 7ff8c3fff700 1 --2- 172.21.15.162:0/3308041660 >> v2:172.21.15.162:3300/0 conn(0x7ff8c40b3950 0x7ff8c40b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:35.381 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.376+0000 7ff8c2ffd700 1 -- 172.21.15.162:0/3308041660 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2726788068 0 0) 0x7ff8c4143e90 con 0x7ff8c40b44e0 2020-11-28T21:54:35.381 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.376+0000 7ff8c2ffd700 1 -- 172.21.15.162:0/3308041660 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8a8003230 con 0x7ff8c40b44e0 2020-11-28T21:54:35.381 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.376+0000 7ff8c3fff700 1 -- 172.21.15.162:0/3308041660 >> v1:172.21.15.162:6789/0 conn(0x7ff8c40b44e0 legacy=0x7ff8c40b2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:35.382 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.376+0000 7ff8c3fff700 1 -- 172.21.15.162:0/3308041660 >> v1:172.21.15.118:6789/0 conn(0x7ff8c40b18c0 legacy=0x7ff8c413dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:35.382 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.376+0000 7ff8c3fff700 1 -- 172.21.15.162:0/3308041660 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c40aa8b0 con 0x7ff8c40b3950 2020-11-28T21:54:35.382 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.376+0000 7ff8c3fff700 1 --2- 172.21.15.162:0/3308041660 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8c40b3950 0x7ff8c40b3d30 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 rx=0x7ff8b40032b0 tx=0x7ff8b40091a0).ready entity=mon.1 client_cookie=9a256215b0055b8 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:35.382 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.376+0000 7ff8c2ffd700 1 -- 172.21.15.162:0/3308041660 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8b400b9a0 con 0x7ff8c40b3950 2020-11-28T21:54:35.383 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.376+0000 7ff8c2ffd700 1 -- 172.21.15.162:0/3308041660 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8b400bb00 con 0x7ff8c40b3950 2020-11-28T21:54:35.383 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.376+0000 7ff8c2ffd700 1 -- 172.21.15.162:0/3308041660 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8b400bc70 con 0x7ff8c40b3950 2020-11-28T21:54:35.383 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.376+0000 7ff8c9790700 1 -- 172.21.15.162:0/3308041660 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8c40b3950 msgr2=0x7ff8c40b3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:35.383 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.376+0000 7ff8c9790700 1 --2- 172.21.15.162:0/3308041660 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8c40b3950 0x7ff8c40b3d30 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 rx=0x7ff8b40032b0 tx=0x7ff8b40091a0).stop 2020-11-28T21:54:35.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.376+0000 7ff8c9790700 1 -- 172.21.15.162:0/3308041660 shutdown_connections 2020-11-28T21:54:35.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.376+0000 7ff8c9790700 1 --2- 172.21.15.162:0/3308041660 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8c40b3950 0x7ff8c40b3d30 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:35.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.376+0000 7ff8c9790700 1 -- 172.21.15.162:0/3308041660 >> 172.21.15.162:0/3308041660 conn(0x7ff8c4062630 msgr2=0x7ff8c4062a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:35.386 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.376+0000 7ff8c9790700 1 -- 172.21.15.162:0/3308041660 shutdown_connections 2020-11-28T21:54:35.386 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.376+0000 7ff8c9790700 1 -- 172.21.15.162:0/3308041660 wait complete. 2020-11-28T21:54:35.386 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.380+0000 7ff8c9790700 1 Processor -- start 2020-11-28T21:54:35.386 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.380+0000 7ff8c9790700 1 -- start start 2020-11-28T21:54:35.387 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.380+0000 7ff8c9790700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff8c40b18c0 0x7ff8c4142c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:35.387 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.380+0000 7ff8c9790700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8c40b3950 0x7ff8c4151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:35.387 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.380+0000 7ff8c9790700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8c40b44e0 0x7ff8c4152090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:35.387 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.380+0000 7ff8c9790700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c41517b0 con 0x7ff8c40b3950 2020-11-28T21:54:35.387 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.380+0000 7ff8c9790700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c4147bd0 con 0x7ff8c40b44e0 2020-11-28T21:54:35.388 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.380+0000 7ff8c9790700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c405f490 con 0x7ff8c40b18c0 2020-11-28T21:54:35.388 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.380+0000 7ff8c8f8f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8c40b44e0 0x7ff8c4152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:35.388 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.380+0000 7ff8c8f8f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8c40b44e0 0x7ff8c4152090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43660/0 (socket says 172.21.15.162:43660) 2020-11-28T21:54:35.388 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.380+0000 7ff8c8f8f700 1 -- 172.21.15.162:0/3963794059 learned_addr learned my addr 172.21.15.162:0/3963794059 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:35.388 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.380+0000 7ff8c3fff700 1 --2- 172.21.15.162:0/3963794059 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff8c40b18c0 0x7ff8c4142c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:35.389 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.380+0000 7ff8c37fe700 1 --2- 172.21.15.162:0/3963794059 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8c40b3950 0x7ff8c4151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:35.389 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.380+0000 7ff8c8f8f700 1 -- 172.21.15.162:0/3963794059 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff8c40b18c0 msgr2=0x7ff8c4142c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:35.389 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.380+0000 7ff8c8f8f700 1 --2- 172.21.15.162:0/3963794059 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff8c40b18c0 0x7ff8c4142c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:35.389 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.380+0000 7ff8c8f8f700 1 -- 172.21.15.162:0/3963794059 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8c40b3950 msgr2=0x7ff8c4151ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:35.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.380+0000 7ff8c8f8f700 1 --2- 172.21.15.162:0/3963794059 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8c40b3950 0x7ff8c4151ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:35.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.380+0000 7ff8c8f8f700 1 -- 172.21.15.162:0/3963794059 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c405f730 con 0x7ff8c40b44e0 2020-11-28T21:54:35.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.380+0000 7ff8c3fff700 1 --2- 172.21.15.162:0/3963794059 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff8c40b18c0 0x7ff8c4142c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:54:35.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.380+0000 7ff8c8f8f700 1 --2- 172.21.15.162:0/3963794059 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8c40b44e0 0x7ff8c4152090 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 rx=0x7ff8bc0010d0 tx=0x7ff8bc007a50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:35.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.380+0000 7ff8c17fa700 1 -- 172.21.15.162:0/3963794059 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8bc0072e0 con 0x7ff8c40b44e0 2020-11-28T21:54:35.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.380+0000 7ff8c17fa700 1 -- 172.21.15.162:0/3963794059 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8bc007440 con 0x7ff8c40b44e0 2020-11-28T21:54:35.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.380+0000 7ff8c17fa700 1 -- 172.21.15.162:0/3963794059 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8bc0057d0 con 0x7ff8c40b44e0 2020-11-28T21:54:35.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.380+0000 7ff8c9790700 1 -- 172.21.15.162:0/3963794059 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c405f9d0 con 0x7ff8c40b44e0 2020-11-28T21:54:35.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.380+0000 7ff8c9790700 1 -- 172.21.15.162:0/3963794059 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c4061060 con 0x7ff8c40b44e0 2020-11-28T21:54:35.392 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.384+0000 7ff8c9790700 1 -- 172.21.15.162:0/3963794059 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8c4151900 con 0x7ff8c40b44e0 2020-11-28T21:54:35.392 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.384+0000 7ff8c17fa700 1 -- 172.21.15.162:0/3963794059 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff8bc010070 con 0x7ff8c40b44e0 2020-11-28T21:54:35.393 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.384+0000 7ff8c17fa700 1 --2- 172.21.15.162:0/3963794059 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff89c0553b0 0x7ff89c057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:35.393 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.384+0000 7ff8c17fa700 1 -- 172.21.15.162:0/3963794059 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff8bc06d910 con 0x7ff8c40b44e0 2020-11-28T21:54:35.393 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.384+0000 7ff8c3fff700 1 --2- 172.21.15.162:0/3963794059 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff89c0553b0 0x7ff89c057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:35.393 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.384+0000 7ff8c3fff700 1 --2- 172.21.15.162:0/3963794059 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff89c0553b0 0x7ff89c057870 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7ff8b4019000 tx=0x7ff8b4016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:35.394 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.388+0000 7ff8c17fa700 1 -- 172.21.15.162:0/3963794059 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff8bc043c10 con 0x7ff8c40b44e0 2020-11-28T21:54:35.531 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.528+0000 7ff8c9790700 1 -- 172.21.15.162:0/3963794059 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff8c4151900 con 0x7ff8c40b44e0 2020-11-28T21:54:35.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.528+0000 7ff8c17fa700 1 -- 172.21.15.162:0/3963794059 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v109) v1 ==== 100+0+10 (secure 0 0 0) 0x7ff8c4151900 con 0x7ff8c40b44e0 2020-11-28T21:54:35.532 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:54:35.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.528+0000 7ff8c9790700 1 -- 172.21.15.162:0/3963794059 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff89c0553b0 msgr2=0x7ff89c057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:35.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.528+0000 7ff8c9790700 1 --2- 172.21.15.162:0/3963794059 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff89c0553b0 0x7ff89c057870 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7ff8b4019000 tx=0x7ff8b4016000).stop 2020-11-28T21:54:35.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.528+0000 7ff8c9790700 1 -- 172.21.15.162:0/3963794059 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8c40b44e0 msgr2=0x7ff8c4152090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:35.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.528+0000 7ff8c9790700 1 --2- 172.21.15.162:0/3963794059 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8c40b44e0 0x7ff8c4152090 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 rx=0x7ff8bc0010d0 tx=0x7ff8bc007a50).stop 2020-11-28T21:54:35.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.528+0000 7ff8c9790700 1 -- 172.21.15.162:0/3963794059 shutdown_connections 2020-11-28T21:54:35.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.528+0000 7ff8c9790700 1 --2- 172.21.15.162:0/3963794059 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff89c0553b0 0x7ff89c057870 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:35.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.528+0000 7ff8c9790700 1 --2- 172.21.15.162:0/3963794059 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff8c40b18c0 0x7ff8c4142c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:35.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.528+0000 7ff8c9790700 1 --2- 172.21.15.162:0/3963794059 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8c40b44e0 0x7ff8c4152090 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:35.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.532+0000 7ff8c9790700 1 --2- 172.21.15.162:0/3963794059 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8c40b3950 0x7ff8c4151ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:35.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.532+0000 7ff8c9790700 1 -- 172.21.15.162:0/3963794059 >> 172.21.15.162:0/3963794059 conn(0x7ff8c4062630 msgr2=0x7ff8c4066b20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:35.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.532+0000 7ff8c9790700 1 -- 172.21.15.162:0/3963794059 shutdown_connections 2020-11-28T21:54:35.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.532+0000 7ff8c9790700 1 -- 172.21.15.162:0/3963794059 wait complete. 2020-11-28T21:54:35.546 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-28T21:54:35.546 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:54:35.658 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.656+0000 7f3483583700 1 Processor -- start 2020-11-28T21:54:35.659 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.656+0000 7f3483583700 1 -- start start 2020-11-28T21:54:35.663 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.656+0000 7f3483583700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f347c0b9320 0x7f347c0b4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:35.663 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.656+0000 7f3483583700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f347c143620 0x7f347c09d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:35.664 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.656+0000 7f3483583700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f347c0b9700 con 0x7f347c09d560 2020-11-28T21:54:35.665 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.656+0000 7f3483583700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f347c142c40 con 0x7f347c143620 2020-11-28T21:54:35.665 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.656+0000 7f3483583700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f347c142ee0 con 0x7f347c0b9320 2020-11-28T21:54:35.665 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.656+0000 7f3481d80700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f347c143620 0x7f347c09d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:35.665 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.656+0000 7f3481d80700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f347c143620 0x7f347c09d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43664/0 (socket says 172.21.15.162:43664) 2020-11-28T21:54:35.667 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.656+0000 7f3481d80700 1 -- 172.21.15.162:0/2337101126 learned_addr learned my addr 172.21.15.162:0/2337101126 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:35.668 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.656+0000 7f3482581700 1 --2- 172.21.15.162:0/2337101126 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f347c0b9320 0x7f347c0b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:35.668 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.656+0000 7f3481d80700 1 -- 172.21.15.162:0/2337101126 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f347c0b9320 msgr2=0x7f347c0b4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:35.668 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.656+0000 7f3481d80700 1 --2- 172.21.15.162:0/2337101126 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f347c0b9320 0x7f347c0b4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:35.668 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.656+0000 7f3481d80700 1 -- 172.21.15.162:0/2337101126 >> v1:172.21.15.118:6789/0 conn(0x7f347c09d560 legacy=0x7f347c060230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:35.668 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.656+0000 7f3481d80700 1 -- 172.21.15.162:0/2337101126 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f347c143180 con 0x7f347c143620 2020-11-28T21:54:35.669 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.656+0000 7f3482581700 1 --2- 172.21.15.162:0/2337101126 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f347c0b9320 0x7f347c0b4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:54:35.669 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.656+0000 7f3481d80700 1 --2- 172.21.15.162:0/2337101126 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f347c143620 0x7f347c09d070 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 rx=0x7f347c05fff0 tx=0x7f346c006e80).ready entity=mon.1 client_cookie=42c5a71a71a5e69a server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:35.669 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.656+0000 7f348157f700 1 -- 172.21.15.162:0/2337101126 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f346c0097b0 con 0x7f347c143620 2020-11-28T21:54:35.669 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.656+0000 7f348157f700 1 -- 172.21.15.162:0/2337101126 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f346c009910 con 0x7f347c143620 2020-11-28T21:54:35.670 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.656+0000 7f348157f700 1 -- 172.21.15.162:0/2337101126 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f346c009ac0 con 0x7f347c143620 2020-11-28T21:54:35.670 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.656+0000 7f3483583700 1 -- 172.21.15.162:0/2337101126 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f347c143620 msgr2=0x7f347c09d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:35.670 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.656+0000 7f3483583700 1 --2- 172.21.15.162:0/2337101126 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f347c143620 0x7f347c09d070 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 rx=0x7f347c05fff0 tx=0x7f346c006e80).stop 2020-11-28T21:54:35.670 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.660+0000 7f3483583700 1 -- 172.21.15.162:0/2337101126 shutdown_connections 2020-11-28T21:54:35.670 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.660+0000 7f3483583700 1 --2- 172.21.15.162:0/2337101126 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f347c0b9320 0x7f347c0b4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:35.671 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.660+0000 7f3483583700 1 --2- 172.21.15.162:0/2337101126 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f347c143620 0x7f347c09d070 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:35.671 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.660+0000 7f3483583700 1 -- 172.21.15.162:0/2337101126 >> 172.21.15.162:0/2337101126 conn(0x7f347c066a40 msgr2=0x7f347c066e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:35.671 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.660+0000 7f3483583700 1 -- 172.21.15.162:0/2337101126 shutdown_connections 2020-11-28T21:54:35.671 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.660+0000 7f3483583700 1 -- 172.21.15.162:0/2337101126 wait complete. 2020-11-28T21:54:35.674 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.664+0000 7f3483583700 1 Processor -- start 2020-11-28T21:54:35.674 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.664+0000 7f3483583700 1 -- start start 2020-11-28T21:54:35.674 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.664+0000 7f3483583700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f347c09d560 0x7f347c0ad540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:35.674 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.664+0000 7f3483583700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f347c0b9320 0x7f347c13e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:35.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.664+0000 7f3483583700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f347c143620 0x7f347c0b1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:35.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.664+0000 7f3483583700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347c0b12e0 con 0x7f347c0b9320 2020-11-28T21:54:35.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.664+0000 7f3483583700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347c1526d0 con 0x7f347c09d560 2020-11-28T21:54:35.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.664+0000 7f3483583700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f347c152930 con 0x7f347c143620 2020-11-28T21:54:35.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.664+0000 7f3482581700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f347c09d560 0x7f347c0ad540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:35.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.664+0000 7f3482581700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f347c09d560 0x7f347c0ad540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43670/0 (socket says 172.21.15.162:43670) 2020-11-28T21:54:35.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.664+0000 7f3482581700 1 -- 172.21.15.162:0/2247962227 learned_addr learned my addr 172.21.15.162:0/2247962227 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:35.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.664+0000 7f3481d80700 1 --2- 172.21.15.162:0/2247962227 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f347c0b9320 0x7f347c13e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:35.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.664+0000 7f3482d82700 1 --2- 172.21.15.162:0/2247962227 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f347c143620 0x7f347c0b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:35.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.664+0000 7f3482581700 1 -- 172.21.15.162:0/2247962227 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f347c143620 msgr2=0x7f347c0b1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:35.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.664+0000 7f3482581700 1 --2- 172.21.15.162:0/2247962227 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f347c143620 0x7f347c0b1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:35.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.664+0000 7f3482581700 1 -- 172.21.15.162:0/2247962227 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f347c0b9320 msgr2=0x7f347c13e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:35.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.664+0000 7f3482581700 1 --2- 172.21.15.162:0/2247962227 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f347c0b9320 0x7f347c13e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:35.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.664+0000 7f3482581700 1 -- 172.21.15.162:0/2247962227 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f347c0b14d0 con 0x7f347c09d560 2020-11-28T21:54:35.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.664+0000 7f3482d82700 1 --2- 172.21.15.162:0/2247962227 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f347c143620 0x7f347c0b1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:54:35.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.664+0000 7f3482581700 1 --2- 172.21.15.162:0/2247962227 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f347c09d560 0x7f347c0ad540 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 rx=0x7f3474007000 tx=0x7f3474008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:35.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.668+0000 7f346b7fe700 1 -- 172.21.15.162:0/2247962227 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f347400a8e0 con 0x7f347c09d560 2020-11-28T21:54:35.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.668+0000 7f346b7fe700 1 -- 172.21.15.162:0/2247962227 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f347400aa40 con 0x7f347c09d560 2020-11-28T21:54:35.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.668+0000 7f346b7fe700 1 -- 172.21.15.162:0/2247962227 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f347400ada0 con 0x7f347c09d560 2020-11-28T21:54:35.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.668+0000 7f3483583700 1 -- 172.21.15.162:0/2247962227 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f347c0ad2f0 con 0x7f347c09d560 2020-11-28T21:54:35.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.668+0000 7f3483583700 1 -- 172.21.15.162:0/2247962227 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f347c0b3470 con 0x7f347c09d560 2020-11-28T21:54:35.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.668+0000 7f346b7fe700 1 -- 172.21.15.162:0/2247962227 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f347401b460 con 0x7f347c09d560 2020-11-28T21:54:35.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.668+0000 7f346b7fe700 1 --2- 172.21.15.162:0/2247962227 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f345c055380 0x7f345c057840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:35.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.668+0000 7f346b7fe700 1 -- 172.21.15.162:0/2247962227 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f347406e2e0 con 0x7f347c09d560 2020-11-28T21:54:35.680 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.668+0000 7f3481d80700 1 --2- 172.21.15.162:0/2247962227 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f345c055380 0x7f345c057840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:35.680 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.668+0000 7f3483583700 1 -- 172.21.15.162:0/2247962227 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f347c152af0 con 0x7f347c09d560 2020-11-28T21:54:35.680 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.668+0000 7f3481d80700 1 --2- 172.21.15.162:0/2247962227 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f345c055380 0x7f345c057840 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7f346c01c000 tx=0x7f346c016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:35.681 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.676+0000 7f346b7fe700 1 -- 172.21.15.162:0/2247962227 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f3474044580 con 0x7f347c09d560 2020-11-28T21:54:35.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.856+0000 7f3483583700 1 -- 172.21.15.162:0/2247962227 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f347c0b3470 con 0x7f347c09d560 2020-11-28T21:54:35.862 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.860+0000 7f346b7fe700 1 -- 172.21.15.162:0/2247962227 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+2499 (secure 0 0 0) 0x7f347c0b3470 con 0x7f347c09d560 2020-11-28T21:54:35.862 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:54:35.863 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":51,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6182,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2775691244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2775691244},{"type":"v1","addr":"172.21.15.162:6835","nonce":2775691244}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":49},{"gid":6942,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/1896314020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1896314020},{"type":"v1","addr":"172.21.15.118:6833","nonce":1896314020}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":49}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:54:26.908249+0000","modified":"2020-11-28T21:54:27.911138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7177},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7177":{"gid":7177,"name":"a","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/493776989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":493776989},{"type":"v1","addr":"172.21.15.162:6837","nonce":493776989}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2020-11-28T21:54:35.864 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.860+0000 7f3483583700 1 -- 172.21.15.162:0/2247962227 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f345c055380 msgr2=0x7f345c057840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:35.864 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.860+0000 7f3483583700 1 --2- 172.21.15.162:0/2247962227 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f345c055380 0x7f345c057840 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7f346c01c000 tx=0x7f346c016000).stop 2020-11-28T21:54:35.864 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.860+0000 7f3483583700 1 -- 172.21.15.162:0/2247962227 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f347c09d560 msgr2=0x7f347c0ad540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:35.865 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.860+0000 7f3483583700 1 --2- 172.21.15.162:0/2247962227 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f347c09d560 0x7f347c0ad540 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 rx=0x7f3474007000 tx=0x7f3474008040).stop 2020-11-28T21:54:35.865 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.860+0000 7f3483583700 1 -- 172.21.15.162:0/2247962227 shutdown_connections 2020-11-28T21:54:35.865 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.860+0000 7f3483583700 1 --2- 172.21.15.162:0/2247962227 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f345c055380 0x7f345c057840 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:35.865 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.860+0000 7f3483583700 1 --2- 172.21.15.162:0/2247962227 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f347c143620 0x7f347c0b1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:35.865 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.860+0000 7f3483583700 1 --2- 172.21.15.162:0/2247962227 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f347c09d560 0x7f347c0ad540 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:35.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.860+0000 7f3483583700 1 --2- 172.21.15.162:0/2247962227 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f347c0b9320 0x7f347c13e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:35.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.860+0000 7f3483583700 1 -- 172.21.15.162:0/2247962227 >> 172.21.15.162:0/2247962227 conn(0x7f347c066a40 msgr2=0x7f347c09e610 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:35.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.860+0000 7f3483583700 1 -- 172.21.15.162:0/2247962227 shutdown_connections 2020-11-28T21:54:35.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:35.860+0000 7f3483583700 1 -- 172.21.15.162:0/2247962227 wait complete. 2020-11-28T21:54:35.867 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 51 2020-11-28T21:54:36.178 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 51, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-28T21:54:26.908249+0000', 'modified': '2020-11-28T21:54:27.911138+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7177}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7177': {'gid': 7177, 'name': 'a', 'rank': 0, 'incarnation': 50, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6837/493776989', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6836', 'nonce': 493776989}, {'type': 'v1', 'addr': '172.21.15.162:6837', 'nonce': 493776989}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-28T21:54:36.179 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-28T21:54:36.180 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-28T21:54:36.310 INFO:teuthology.orchestra.run.smithi162.stdout:{"cluster_fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","whoami":0,"id":7177,"want_state":"up:active","state":"up:active","rank_uptime":9.3875102500000001,"mdsmap_epoch":51,"osdmap_epoch":109,"osdmap_epoch_barrier":108,"uptime":13.763943249} 2020-11-28T21:54:36.311 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", "id": 7177, "mdsmap_epoch": 51, "osdmap_epoch": 109, "osdmap_epoch_barrier": 108, "rank_uptime": 9.38751025, "state": "up:active", "uptime": 13.763943249, "want_state": "up:active", "whoami": 0 } 2020-11-28T21:54:36.311 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-11-28T21:54:36.312 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2020-11-28T21:54:36.312 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi096.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2020-11-28T21:54:36.312 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:36.313 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:54:36.320 INFO:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2020-11-28T21:54:36.377 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:54:36.377 INFO:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:54:36.440 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:54:36.447 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:54:36.448 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:36.449 INFO:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2020-11-28T21:54:36.495 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-11-28T21:54:36.496 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:54:36.542 INFO:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2020-11-28T21:54:36.558 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:54:36.559 INFO:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:54:36.617 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[11822]: starting ceph client 2020-11-28T21:54:36.618 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:2020-11-28T21:54:36.609+0000 7f6fbe6560c0 -1 init, newargv = 0x55e5a418e410 newargc=9 2020-11-28T21:54:36.620 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:54:36.621 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:54:36.622 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:36.623 INFO:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2020-11-28T21:54:36.631 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[11822]: starting fuse 2020-11-28T21:54:36.646 INFO:teuthology.orchestra.run.smithi096.stdout:49 2020-11-28T21:54:36.647 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2020-11-28T21:54:36.648 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:36.648 INFO:teuthology.orchestra.run.smithi096:> sudo python3 -c ' 2020-11-28T21:54:36.649 INFO:teuthology.orchestra.run.smithi096:> import glob 2020-11-28T21:54:36.650 INFO:teuthology.orchestra.run.smithi096:> import re 2020-11-28T21:54:36.651 INFO:teuthology.orchestra.run.smithi096:> import os 2020-11-28T21:54:36.652 INFO:teuthology.orchestra.run.smithi096:> import subprocess 2020-11-28T21:54:36.652 INFO:teuthology.orchestra.run.smithi096:> 2020-11-28T21:54:36.653 INFO:teuthology.orchestra.run.smithi096:> def find_socket(client_name): 2020-11-28T21:54:36.654 INFO:teuthology.orchestra.run.smithi096:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2020-11-28T21:54:36.655 INFO:teuthology.orchestra.run.smithi096:> files = glob.glob(asok_path) 2020-11-28T21:54:36.655 INFO:teuthology.orchestra.run.smithi096:> 2020-11-28T21:54:36.656 INFO:teuthology.orchestra.run.smithi096:> # Given a non-glob path, it better be there 2020-11-28T21:54:36.657 INFO:teuthology.orchestra.run.smithi096:> if "*" not in asok_path: 2020-11-28T21:54:36.658 INFO:teuthology.orchestra.run.smithi096:> assert(len(files) == 1) 2020-11-28T21:54:36.658 INFO:teuthology.orchestra.run.smithi096:> return files[0] 2020-11-28T21:54:36.659 INFO:teuthology.orchestra.run.smithi096:> 2020-11-28T21:54:36.660 INFO:teuthology.orchestra.run.smithi096:> for f in files: 2020-11-28T21:54:36.661 INFO:teuthology.orchestra.run.smithi096:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-11-28T21:54:36.661 INFO:teuthology.orchestra.run.smithi096:> if os.path.exists("/proc/{0}".format(pid)): 2020-11-28T21:54:36.662 INFO:teuthology.orchestra.run.smithi096:> return f 2020-11-28T21:54:36.663 INFO:teuthology.orchestra.run.smithi096:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-11-28T21:54:36.664 INFO:teuthology.orchestra.run.smithi096:> 2020-11-28T21:54:36.664 INFO:teuthology.orchestra.run.smithi096:> print(find_socket("client.1")) 2020-11-28T21:54:36.665 INFO:teuthology.orchestra.run.smithi096:> ' 2020-11-28T21:54:36.739 INFO:teuthology.orchestra.run.smithi096.stdout:/var/run/ceph/ceph-client.1.11822.asok 2020-11-28T21:54:36.747 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.11822.asok 2020-11-28T21:54:36.747 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:36.748 INFO:teuthology.orchestra.run.smithi096:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.11822.asok status 2020-11-28T21:54:36.859 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-11-28T21:54:36.860 INFO:teuthology.orchestra.run.smithi096.stdout: "metadata": { 2020-11-28T21:54:36.860 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_sha1": "a83a3572a957356977342f461361c1cd3ec06dbd", 2020-11-28T21:54:36.860 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_version": "ceph version 15.2.6-595-ga83a3572 (a83a3572a957356977342f461361c1cd3ec06dbd) octopus (stable)", 2020-11-28T21:54:36.861 INFO:teuthology.orchestra.run.smithi096.stdout: "entity_id": "1", 2020-11-28T21:54:36.861 INFO:teuthology.orchestra.run.smithi096.stdout: "hostname": "smithi096", 2020-11-28T21:54:36.861 INFO:teuthology.orchestra.run.smithi096.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2020-11-28T21:54:36.861 INFO:teuthology.orchestra.run.smithi096.stdout: "pid": "11822", 2020-11-28T21:54:36.861 INFO:teuthology.orchestra.run.smithi096.stdout: "root": "/" 2020-11-28T21:54:36.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-11-28T21:54:36.862 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_count": 0, 2020-11-28T21:54:36.862 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_pinned_count": 0, 2020-11-28T21:54:36.862 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 7378, 2020-11-28T21:54:36.862 INFO:teuthology.orchestra.run.smithi096.stdout: "inst": { 2020-11-28T21:54:36.863 INFO:teuthology.orchestra.run.smithi096.stdout: "name": { 2020-11-28T21:54:36.863 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "client", 2020-11-28T21:54:36.863 INFO:teuthology.orchestra.run.smithi096.stdout: "num": 7378 2020-11-28T21:54:36.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-11-28T21:54:36.863 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2020-11-28T21:54:36.864 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2020-11-28T21:54:36.864 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2020-11-28T21:54:36.864 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 515783770 2020-11-28T21:54:36.864 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-11-28T21:54:36.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-11-28T21:54:36.865 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2020-11-28T21:54:36.865 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2020-11-28T21:54:36.865 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2020-11-28T21:54:36.865 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 515783770 2020-11-28T21:54:36.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-11-28T21:54:36.866 INFO:teuthology.orchestra.run.smithi096.stdout: "inst_str": "client.7378 172.21.15.96:0/515783770", 2020-11-28T21:54:36.866 INFO:teuthology.orchestra.run.smithi096.stdout: "addr_str": "172.21.15.96:0/515783770", 2020-11-28T21:54:36.866 INFO:teuthology.orchestra.run.smithi096.stdout: "inode_count": 1, 2020-11-28T21:54:36.866 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_epoch": 51, 2020-11-28T21:54:36.867 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch": 109, 2020-11-28T21:54:36.867 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch_barrier": 0, 2020-11-28T21:54:36.867 INFO:teuthology.orchestra.run.smithi096.stdout: "blacklisted": false 2020-11-28T21:54:36.867 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-11-28T21:54:36.870 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:36.871 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:54:36.871 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:54:36.879 INFO:teuthology.orchestra.run.smithi096.stdout:fuseblk 2020-11-28T21:54:36.879 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-11-28T21:54:36.880 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:36.880 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:54:36.939 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early df --format=json-pretty 2020-11-28T21:54:37.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.056+0000 7fbc7288c700 1 Processor -- start 2020-11-28T21:54:37.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.056+0000 7fbc7288c700 1 -- start start 2020-11-28T21:54:37.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.056+0000 7fbc7288c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fbc6c0994f0 0x7fbc6c0b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:37.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.056+0000 7fbc7288c700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc6c098f10 con 0x7fbc6c098b30 2020-11-28T21:54:37.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.056+0000 7fbc7288c700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc6c0b0680 con 0x7fbc6c060710 2020-11-28T21:54:37.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.056+0000 7fbc7288c700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc6c0b1030 con 0x7fbc6c0994f0 2020-11-28T21:54:37.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.056+0000 7fbc7208b700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fbc6c060710 0x7fbc6c0b1580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:50526/0 (socket says 172.21.15.162:50526) 2020-11-28T21:54:37.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.056+0000 7fbc7208b700 1 -- 172.21.15.162:0/4239946474 learned_addr learned my addr 172.21.15.162:0/4239946474 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:37.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.056+0000 7fbc71089700 1 --2- 172.21.15.162:0/4239946474 >> v2:172.21.15.118:3300/0 conn(0x7fbc6c0994f0 0x7fbc6c0b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:37.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.056+0000 7fbc70888700 1 -- 172.21.15.162:0/4239946474 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 643969288 0 0) 0x7fbc6c0b0680 con 0x7fbc6c060710 2020-11-28T21:54:37.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.056+0000 7fbc70888700 1 -- 172.21.15.162:0/4239946474 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc58003230 con 0x7fbc6c060710 2020-11-28T21:54:37.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.056+0000 7fbc70888700 1 -- 172.21.15.162:0/4239946474 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 707292353 0 0) 0x7fbc58003230 con 0x7fbc6c060710 2020-11-28T21:54:37.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.056+0000 7fbc70888700 1 -- 172.21.15.162:0/4239946474 >> v2:172.21.15.118:3300/0 conn(0x7fbc6c0994f0 msgr2=0x7fbc6c0b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:37.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.056+0000 7fbc70888700 1 --2- 172.21.15.162:0/4239946474 >> v2:172.21.15.118:3300/0 conn(0x7fbc6c0994f0 0x7fbc6c0b0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:37.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.056+0000 7fbc70888700 1 -- 172.21.15.162:0/4239946474 >> v1:172.21.15.118:6789/0 conn(0x7fbc6c098b30 legacy=0x7fbc6c060000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:37.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.056+0000 7fbc70888700 1 -- 172.21.15.162:0/4239946474 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc6c0b0920 con 0x7fbc6c060710 2020-11-28T21:54:37.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.056+0000 7fbc70888700 1 -- 172.21.15.162:0/4239946474 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7fbc5c0016d0 con 0x7fbc6c060710 2020-11-28T21:54:37.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.056+0000 7fbc70888700 1 -- 172.21.15.162:0/4239946474 >> v1:172.21.15.162:6789/0 conn(0x7fbc6c060710 legacy=0x7fbc6c0b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:37.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.056+0000 7fbc70888700 1 --2- 172.21.15.162:0/4239946474 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbc58003720 0x7fbc58003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:37.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.056+0000 7fbc70888700 1 --2- 172.21.15.162:0/4239946474 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbc580044d0 0x7fbc58004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:37.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.056+0000 7fbc70888700 1 --2- 172.21.15.162:0/4239946474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc58004e40 0x7fbc5800b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:37.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.056+0000 7fbc70888700 1 -- 172.21.15.162:0/4239946474 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6c0b1030 con 0x7fbc58003720 2020-11-28T21:54:37.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.056+0000 7fbc70888700 1 -- 172.21.15.162:0/4239946474 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc58004250 con 0x7fbc58004e40 2020-11-28T21:54:37.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.056+0000 7fbc70888700 1 -- 172.21.15.162:0/4239946474 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5800bc10 con 0x7fbc580044d0 2020-11-28T21:54:37.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.056+0000 7fbc7208b700 1 --2- 172.21.15.162:0/4239946474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc58004e40 0x7fbc5800b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:37.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.056+0000 7fbc71089700 1 --2- 172.21.15.162:0/4239946474 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbc580044d0 0x7fbc58004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:37.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.056+0000 7fbc7188a700 1 --2- 172.21.15.162:0/4239946474 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbc58003720 0x7fbc58003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:37.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.060+0000 7fbc7208b700 1 -- 172.21.15.162:0/4239946474 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbc580044d0 msgr2=0x7fbc58004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:37.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.060+0000 7fbc7208b700 1 --2- 172.21.15.162:0/4239946474 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbc580044d0 0x7fbc58004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:37.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.060+0000 7fbc7208b700 1 -- 172.21.15.162:0/4239946474 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbc58003720 msgr2=0x7fbc58003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:37.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.060+0000 7fbc7208b700 1 --2- 172.21.15.162:0/4239946474 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbc58003720 0x7fbc58003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:37.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.060+0000 7fbc7208b700 1 -- 172.21.15.162:0/4239946474 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc58003230 con 0x7fbc58004e40 2020-11-28T21:54:37.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.060+0000 7fbc7208b700 1 -- 172.21.15.162:0/4239946474 reap_dead start 2020-11-28T21:54:37.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.060+0000 7fbc7208b700 1 --2- 172.21.15.162:0/4239946474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc58004e40 0x7fbc5800b720 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 rx=0x7fbc5c0013e0 tx=0x7fbc5c007770).ready entity=mon.1 client_cookie=1b31306487fbc818 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:37.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.060+0000 7fbc70888700 1 -- 172.21.15.162:0/4239946474 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc5c0070d0 con 0x7fbc58004e40 2020-11-28T21:54:37.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.060+0000 7fbc70888700 1 -- 172.21.15.162:0/4239946474 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc5c007230 con 0x7fbc58004e40 2020-11-28T21:54:37.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.060+0000 7fbc70888700 1 -- 172.21.15.162:0/4239946474 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc5c0055e0 con 0x7fbc58004e40 2020-11-28T21:54:37.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.060+0000 7fbc7288c700 1 -- 172.21.15.162:0/4239946474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc58004e40 msgr2=0x7fbc5800b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:37.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.060+0000 7fbc7288c700 1 --2- 172.21.15.162:0/4239946474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc58004e40 0x7fbc5800b720 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 rx=0x7fbc5c0013e0 tx=0x7fbc5c007770).stop 2020-11-28T21:54:37.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.060+0000 7fbc7288c700 1 -- 172.21.15.162:0/4239946474 shutdown_connections 2020-11-28T21:54:37.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.060+0000 7fbc7288c700 1 --2- 172.21.15.162:0/4239946474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc58004e40 0x7fbc5800b720 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:37.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.060+0000 7fbc7288c700 1 -- 172.21.15.162:0/4239946474 >> 172.21.15.162:0/4239946474 conn(0x7fbc6c05b970 msgr2=0x7fbc6c05bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:37.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.060+0000 7fbc7288c700 1 -- 172.21.15.162:0/4239946474 shutdown_connections 2020-11-28T21:54:37.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.060+0000 7fbc7288c700 1 -- 172.21.15.162:0/4239946474 wait complete. 2020-11-28T21:54:37.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.060+0000 7fbc7288c700 1 Processor -- start 2020-11-28T21:54:37.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.060+0000 7fbc7288c700 1 -- start start 2020-11-28T21:54:37.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.060+0000 7fbc7288c700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbc58004e40 0x7fbc6c0ac800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:37.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.060+0000 7fbc7288c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc6c159c00 0x7fbc6c066fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:37.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.060+0000 7fbc7288c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbc6c0674c0 0x7fbc6c0ad740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:37.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.060+0000 7fbc7288c700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6c0aea30 con 0x7fbc6c0674c0 2020-11-28T21:54:37.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.060+0000 7fbc7288c700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6c155aa0 con 0x7fbc6c159c00 2020-11-28T21:54:37.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.060+0000 7fbc7288c700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6c09b250 con 0x7fbc58004e40 2020-11-28T21:54:37.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.060+0000 7fbc71089700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc6c159c00 0x7fbc6c066fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:37.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.060+0000 7fbc71089700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc6c159c00 0x7fbc6c066fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43692/0 (socket says 172.21.15.162:43692) 2020-11-28T21:54:37.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.060+0000 7fbc71089700 1 -- 172.21.15.162:0/664966403 learned_addr learned my addr 172.21.15.162:0/664966403 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:37.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.060+0000 7fbc7208b700 1 --2- 172.21.15.162:0/664966403 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbc6c0674c0 0x7fbc6c0ad740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:37.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.060+0000 7fbc7188a700 1 --2- 172.21.15.162:0/664966403 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbc58004e40 0x7fbc6c0ac800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:37.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.060+0000 7fbc71089700 1 -- 172.21.15.162:0/664966403 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbc58004e40 msgr2=0x7fbc6c0ac800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:37.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.064+0000 7fbc71089700 1 --2- 172.21.15.162:0/664966403 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbc58004e40 0x7fbc6c0ac800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:37.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.064+0000 7fbc71089700 1 -- 172.21.15.162:0/664966403 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbc6c0674c0 msgr2=0x7fbc6c0ad740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:37.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.064+0000 7fbc71089700 1 --2- 172.21.15.162:0/664966403 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbc6c0674c0 0x7fbc6c0ad740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:37.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.064+0000 7fbc71089700 1 -- 172.21.15.162:0/664966403 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc6c063d80 con 0x7fbc6c159c00 2020-11-28T21:54:37.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.064+0000 7fbc7208b700 1 --2- 172.21.15.162:0/664966403 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbc6c0674c0 0x7fbc6c0ad740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:54:37.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.064+0000 7fbc7188a700 1 --2- 172.21.15.162:0/664966403 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbc58004e40 0x7fbc6c0ac800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:54:37.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.064+0000 7fbc71089700 1 --2- 172.21.15.162:0/664966403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc6c159c00 0x7fbc6c066fd0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 rx=0x7fbc6c063ab0 tx=0x7fbc640091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:37.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.064+0000 7fbc6affd700 1 -- 172.21.15.162:0/664966403 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc6400fca0 con 0x7fbc6c159c00 2020-11-28T21:54:37.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.064+0000 7fbc6affd700 1 -- 172.21.15.162:0/664966403 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc6400fe00 con 0x7fbc6c159c00 2020-11-28T21:54:37.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.064+0000 7fbc7288c700 1 -- 172.21.15.162:0/664966403 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc6c0637e0 con 0x7fbc6c159c00 2020-11-28T21:54:37.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.064+0000 7fbc6affd700 1 -- 172.21.15.162:0/664966403 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc64014630 con 0x7fbc6c159c00 2020-11-28T21:54:37.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.064+0000 7fbc7288c700 1 -- 172.21.15.162:0/664966403 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc6c063340 con 0x7fbc6c159c00 2020-11-28T21:54:37.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.064+0000 7fbc7288c700 1 -- 172.21.15.162:0/664966403 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc6c064410 con 0x7fbc6c159c00 2020-11-28T21:54:37.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.064+0000 7fbc6affd700 1 -- 172.21.15.162:0/664966403 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fbc64024070 con 0x7fbc6c159c00 2020-11-28T21:54:37.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.064+0000 7fbc6affd700 1 --2- 172.21.15.162:0/664966403 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbc4c0553b0 0x7fbc4c057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:37.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.064+0000 7fbc6affd700 1 -- 172.21.15.162:0/664966403 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbc6406ebc0 con 0x7fbc6c159c00 2020-11-28T21:54:37.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.064+0000 7fbc7188a700 1 --2- 172.21.15.162:0/664966403 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbc4c0553b0 0x7fbc4c057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:37.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.068+0000 7fbc7188a700 1 --2- 172.21.15.162:0/664966403 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbc4c0553b0 0x7fbc4c057870 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7fbc60003000 tx=0x7fbc60009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:37.083 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.072+0000 7fbc6affd700 1 -- 172.21.15.162:0/664966403 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fbc64044db0 con 0x7fbc6c159c00 2020-11-28T21:54:37.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.264+0000 7fbc7288c700 1 -- 172.21.15.162:0/664966403 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7fbc6c057050 con 0x7fbc6c159c00 2020-11-28T21:54:37.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.264+0000 7fbc6affd700 1 -- 172.21.15.162:0/664966403 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1936 (secure 0 0 0) 0x7fbc640483d0 con 0x7fbc6c159c00 2020-11-28T21:54:37.269 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:54:37.269 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2020-11-28T21:54:37.270 INFO:teuthology.orchestra.run.smithi162.stdout: "stats": { 2020-11-28T21:54:37.270 INFO:teuthology.orchestra.run.smithi162.stdout: "total_bytes": 773094113280, 2020-11-28T21:54:37.270 INFO:teuthology.orchestra.run.smithi162.stdout: "total_avail_bytes": 764493172736, 2020-11-28T21:54:37.270 INFO:teuthology.orchestra.run.smithi162.stdout: "total_used_bytes": 11005952, 2020-11-28T21:54:37.271 INFO:teuthology.orchestra.run.smithi162.stdout: "total_used_raw_bytes": 8600940544, 2020-11-28T21:54:37.271 INFO:teuthology.orchestra.run.smithi162.stdout: "total_used_raw_ratio": 0.011125347577035427, 2020-11-28T21:54:37.271 INFO:teuthology.orchestra.run.smithi162.stdout: "num_osds": 8, 2020-11-28T21:54:37.271 INFO:teuthology.orchestra.run.smithi162.stdout: "num_per_pool_osds": 8, 2020-11-28T21:54:37.272 INFO:teuthology.orchestra.run.smithi162.stdout: "num_per_pool_omap_osds": 8 2020-11-28T21:54:37.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:37.272 INFO:teuthology.orchestra.run.smithi162.stdout: "stats_by_class": { 2020-11-28T21:54:37.272 INFO:teuthology.orchestra.run.smithi162.stdout: "ssd": { 2020-11-28T21:54:37.272 INFO:teuthology.orchestra.run.smithi162.stdout: "total_bytes": 773094113280, 2020-11-28T21:54:37.273 INFO:teuthology.orchestra.run.smithi162.stdout: "total_avail_bytes": 764493172736, 2020-11-28T21:54:37.273 INFO:teuthology.orchestra.run.smithi162.stdout: "total_used_bytes": 11005952, 2020-11-28T21:54:37.273 INFO:teuthology.orchestra.run.smithi162.stdout: "total_used_raw_bytes": 8600940544, 2020-11-28T21:54:37.273 INFO:teuthology.orchestra.run.smithi162.stdout: "total_used_raw_ratio": 0.011125347577035427 2020-11-28T21:54:37.273 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:37.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:37.274 INFO:teuthology.orchestra.run.smithi162.stdout: "pools": [ 2020-11-28T21:54:37.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:37.274 INFO:teuthology.orchestra.run.smithi162.stdout: "name": "device_health_metrics", 2020-11-28T21:54:37.274 INFO:teuthology.orchestra.run.smithi162.stdout: "id": 1, 2020-11-28T21:54:37.275 INFO:teuthology.orchestra.run.smithi162.stdout: "stats": { 2020-11-28T21:54:37.275 INFO:teuthology.orchestra.run.smithi162.stdout: "stored": 0, 2020-11-28T21:54:37.275 INFO:teuthology.orchestra.run.smithi162.stdout: "objects": 2, 2020-11-28T21:54:37.275 INFO:teuthology.orchestra.run.smithi162.stdout: "kb_used": 0, 2020-11-28T21:54:37.276 INFO:teuthology.orchestra.run.smithi162.stdout: "bytes_used": 0, 2020-11-28T21:54:37.276 INFO:teuthology.orchestra.run.smithi162.stdout: "percent_used": 0, 2020-11-28T21:54:37.276 INFO:teuthology.orchestra.run.smithi162.stdout: "max_avail": 362919165952 2020-11-28T21:54:37.276 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:37.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:37.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:37.278 INFO:teuthology.orchestra.run.smithi162.stdout: "name": "rbd", 2020-11-28T21:54:37.278 INFO:teuthology.orchestra.run.smithi162.stdout: "id": 2, 2020-11-28T21:54:37.279 INFO:teuthology.orchestra.run.smithi162.stdout: "stats": { 2020-11-28T21:54:37.279 INFO:teuthology.orchestra.run.smithi162.stdout: "stored": 0, 2020-11-28T21:54:37.279 INFO:teuthology.orchestra.run.smithi162.stdout: "objects": 0, 2020-11-28T21:54:37.279 INFO:teuthology.orchestra.run.smithi162.stdout: "kb_used": 0, 2020-11-28T21:54:37.279 INFO:teuthology.orchestra.run.smithi162.stdout: "bytes_used": 0, 2020-11-28T21:54:37.280 INFO:teuthology.orchestra.run.smithi162.stdout: "percent_used": 0, 2020-11-28T21:54:37.280 INFO:teuthology.orchestra.run.smithi162.stdout: "max_avail": 362919165952 2020-11-28T21:54:37.280 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:37.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:37.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:37.281 INFO:teuthology.orchestra.run.smithi162.stdout: "name": "cephfs_metadata", 2020-11-28T21:54:37.281 INFO:teuthology.orchestra.run.smithi162.stdout: "id": 19, 2020-11-28T21:54:37.281 INFO:teuthology.orchestra.run.smithi162.stdout: "stats": { 2020-11-28T21:54:37.281 INFO:teuthology.orchestra.run.smithi162.stdout: "stored": 2741, 2020-11-28T21:54:37.282 INFO:teuthology.orchestra.run.smithi162.stdout: "objects": 22, 2020-11-28T21:54:37.282 INFO:teuthology.orchestra.run.smithi162.stdout: "kb_used": 64, 2020-11-28T21:54:37.282 INFO:teuthology.orchestra.run.smithi162.stdout: "bytes_used": 65536, 2020-11-28T21:54:37.282 INFO:teuthology.orchestra.run.smithi162.stdout: "percent_used": 9.0290072307652736e-08, 2020-11-28T21:54:37.282 INFO:teuthology.orchestra.run.smithi162.stdout: "max_avail": 362919165952 2020-11-28T21:54:37.283 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:37.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:37.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:37.283 INFO:teuthology.orchestra.run.smithi162.stdout: "name": "cephfs_data", 2020-11-28T21:54:37.283 INFO:teuthology.orchestra.run.smithi162.stdout: "id": 20, 2020-11-28T21:54:37.284 INFO:teuthology.orchestra.run.smithi162.stdout: "stats": { 2020-11-28T21:54:37.284 INFO:teuthology.orchestra.run.smithi162.stdout: "stored": 0, 2020-11-28T21:54:37.284 INFO:teuthology.orchestra.run.smithi162.stdout: "objects": 0, 2020-11-28T21:54:37.284 INFO:teuthology.orchestra.run.smithi162.stdout: "kb_used": 0, 2020-11-28T21:54:37.284 INFO:teuthology.orchestra.run.smithi162.stdout: "bytes_used": 0, 2020-11-28T21:54:37.285 INFO:teuthology.orchestra.run.smithi162.stdout: "percent_used": 0, 2020-11-28T21:54:37.285 INFO:teuthology.orchestra.run.smithi162.stdout: "max_avail": 362919165952 2020-11-28T21:54:37.285 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:37.285 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:37.285 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:37.286 INFO:teuthology.orchestra.run.smithi162.stdout:} 2020-11-28T21:54:37.287 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.268+0000 7fbc7288c700 1 -- 172.21.15.162:0/664966403 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbc4c0553b0 msgr2=0x7fbc4c057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:37.287 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.268+0000 7fbc7288c700 1 --2- 172.21.15.162:0/664966403 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbc4c0553b0 0x7fbc4c057870 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7fbc60003000 tx=0x7fbc60009040).stop 2020-11-28T21:54:37.288 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.268+0000 7fbc7288c700 1 -- 172.21.15.162:0/664966403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc6c159c00 msgr2=0x7fbc6c066fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:37.288 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.268+0000 7fbc7288c700 1 --2- 172.21.15.162:0/664966403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc6c159c00 0x7fbc6c066fd0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 rx=0x7fbc6c063ab0 tx=0x7fbc640091a0).stop 2020-11-28T21:54:37.288 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.268+0000 7fbc7288c700 1 -- 172.21.15.162:0/664966403 shutdown_connections 2020-11-28T21:54:37.288 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.268+0000 7fbc7288c700 1 --2- 172.21.15.162:0/664966403 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbc4c0553b0 0x7fbc4c057870 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:37.289 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.268+0000 7fbc7288c700 1 --2- 172.21.15.162:0/664966403 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbc58004e40 0x7fbc6c0ac800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:37.289 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.268+0000 7fbc7288c700 1 --2- 172.21.15.162:0/664966403 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbc6c0674c0 0x7fbc6c0ad740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:37.289 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.268+0000 7fbc7288c700 1 --2- 172.21.15.162:0/664966403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc6c159c00 0x7fbc6c066fd0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:37.289 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.268+0000 7fbc7288c700 1 -- 172.21.15.162:0/664966403 >> 172.21.15.162:0/664966403 conn(0x7fbc6c05b970 msgr2=0x7fbc6c14a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:37.289 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.268+0000 7fbc7288c700 1 -- 172.21.15.162:0/664966403 shutdown_connections 2020-11-28T21:54:37.290 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.268+0000 7fbc7288c700 1 -- 172.21.15.162:0/664966403 wait complete. 2020-11-28T21:54:37.290 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data size -f json-pretty 2020-11-28T21:54:37.402 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.400+0000 7fa4d3fa6700 1 Processor -- start 2020-11-28T21:54:37.402 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.400+0000 7fa4d3fa6700 1 -- start start 2020-11-28T21:54:37.404 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.400+0000 7fa4d3fa6700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fa4cc0ae230 0x7fa4cc0a8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:37.404 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.400+0000 7fa4d3fa6700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa4cc142dc0 0x7fa4cc149c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:37.404 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.400+0000 7fa4d3fa6700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4cc05f380 con 0x7fa4cc0ade50 2020-11-28T21:54:37.404 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.400+0000 7fa4d3fa6700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4cc0acda0 con 0x7fa4cc0ae230 2020-11-28T21:54:37.404 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.400+0000 7fa4d3fa6700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4cc0a8a60 con 0x7fa4cc142dc0 2020-11-28T21:54:37.405 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.400+0000 7fa4d2543700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa4cc142dc0 0x7fa4cc149c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:37.405 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.400+0000 7fa4d2543700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa4cc142dc0 0x7fa4cc149c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43700/0 (socket says 172.21.15.162:43700) 2020-11-28T21:54:37.405 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.400+0000 7fa4d2543700 1 -- 172.21.15.162:0/2788416145 learned_addr learned my addr 172.21.15.162:0/2788416145 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:37.405 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.400+0000 7fa4d1541700 1 --2- 172.21.15.162:0/2788416145 >> v2:172.21.15.118:3300/0 conn(0x7fa4cc0ae230 0x7fa4cc0a8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:37.407 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.400+0000 7fa4d2543700 1 -- 172.21.15.162:0/2788416145 >> v2:172.21.15.118:3300/0 conn(0x7fa4cc0ae230 msgr2=0x7fa4cc0a8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:37.407 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.400+0000 7fa4d2543700 1 --2- 172.21.15.162:0/2788416145 >> v2:172.21.15.118:3300/0 conn(0x7fa4cc0ae230 0x7fa4cc0a8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:37.407 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.400+0000 7fa4d2543700 1 -- 172.21.15.162:0/2788416145 >> v1:172.21.15.118:6789/0 conn(0x7fa4cc0ade50 legacy=0x7fa4cc1426b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:37.407 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.400+0000 7fa4d2543700 1 -- 172.21.15.162:0/2788416145 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4cc061240 con 0x7fa4cc142dc0 2020-11-28T21:54:37.408 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.400+0000 7fa4d2543700 1 --2- 172.21.15.162:0/2788416145 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4cc142dc0 0x7fa4cc149c10 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 rx=0x7fa4c8002f30 tx=0x7fa4c8006e80).ready entity=mon.1 client_cookie=c6368affb4e84829 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:37.408 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.400+0000 7fa4d0d40700 1 -- 172.21.15.162:0/2788416145 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4c80097f0 con 0x7fa4cc142dc0 2020-11-28T21:54:37.408 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.400+0000 7fa4d0d40700 1 -- 172.21.15.162:0/2788416145 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c8009950 con 0x7fa4cc142dc0 2020-11-28T21:54:37.408 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.400+0000 7fa4d0d40700 1 -- 172.21.15.162:0/2788416145 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4c8009cb0 con 0x7fa4cc142dc0 2020-11-28T21:54:37.409 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.400+0000 7fa4d3fa6700 1 -- 172.21.15.162:0/2788416145 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4cc142dc0 msgr2=0x7fa4cc149c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:37.409 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.400+0000 7fa4d3fa6700 1 --2- 172.21.15.162:0/2788416145 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4cc142dc0 0x7fa4cc149c10 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 rx=0x7fa4c8002f30 tx=0x7fa4c8006e80).stop 2020-11-28T21:54:37.409 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.400+0000 7fa4d3fa6700 1 -- 172.21.15.162:0/2788416145 shutdown_connections 2020-11-28T21:54:37.409 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.400+0000 7fa4d3fa6700 1 --2- 172.21.15.162:0/2788416145 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4cc142dc0 0x7fa4cc149c10 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:37.409 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.400+0000 7fa4d3fa6700 1 --2- 172.21.15.162:0/2788416145 >> v2:172.21.15.118:3300/0 conn(0x7fa4cc0ae230 0x7fa4cc0a8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:37.410 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.400+0000 7fa4d3fa6700 1 -- 172.21.15.162:0/2788416145 >> 172.21.15.162:0/2788416145 conn(0x7fa4cc05c3f0 msgr2=0x7fa4cc0b4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:37.410 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.404+0000 7fa4d3fa6700 1 -- 172.21.15.162:0/2788416145 shutdown_connections 2020-11-28T21:54:37.410 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.404+0000 7fa4d3fa6700 1 -- 172.21.15.162:0/2788416145 wait complete. 2020-11-28T21:54:37.410 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.404+0000 7fa4d3fa6700 1 Processor -- start 2020-11-28T21:54:37.411 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.404+0000 7fa4d3fa6700 1 -- start start 2020-11-28T21:54:37.413 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.404+0000 7fa4d3fa6700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa4cc0ade50 0x7fa4cc0ab3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:37.413 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.404+0000 7fa4d3fa6700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa4cc0ae230 0x7fa4cc152020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:37.413 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.404+0000 7fa4d3fa6700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4cc142dc0 0x7fa4cc152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:37.414 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.404+0000 7fa4d3fa6700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4cc152a00 con 0x7fa4cc0ae230 2020-11-28T21:54:37.414 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.404+0000 7fa4d3fa6700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4cc13d190 con 0x7fa4cc142dc0 2020-11-28T21:54:37.414 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.404+0000 7fa4d3fa6700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4cc13d3f0 con 0x7fa4cc0ade50 2020-11-28T21:54:37.414 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.404+0000 7fa4d2543700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4cc142dc0 0x7fa4cc152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:37.414 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.404+0000 7fa4d1541700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa4cc0ae230 0x7fa4cc152020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:37.415 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.404+0000 7fa4d1541700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa4cc0ae230 0x7fa4cc152020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.162:50694/0 (socket says 172.21.15.162:50694) 2020-11-28T21:54:37.415 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.404+0000 7fa4d1541700 1 -- 172.21.15.162:0/2984447200 learned_addr learned my addr 172.21.15.162:0/2984447200 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:37.415 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.404+0000 7fa4d2543700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4cc142dc0 0x7fa4cc152510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43708/0 (socket says 172.21.15.162:43708) 2020-11-28T21:54:37.415 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.404+0000 7fa4d1d42700 1 --2- 172.21.15.162:0/2984447200 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa4cc0ade50 0x7fa4cc0ab3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:37.416 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.404+0000 7fa4d2543700 1 -- 172.21.15.162:0/2984447200 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa4cc0ade50 msgr2=0x7fa4cc0ab3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:37.416 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.404+0000 7fa4d2543700 1 --2- 172.21.15.162:0/2984447200 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa4cc0ade50 0x7fa4cc0ab3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:37.416 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.404+0000 7fa4d2543700 1 -- 172.21.15.162:0/2984447200 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa4cc0ae230 msgr2=0x7fa4cc152020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:37.416 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.404+0000 7fa4d2543700 1 --2- 172.21.15.162:0/2984447200 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa4cc0ae230 0x7fa4cc152020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:37.416 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.404+0000 7fa4d2543700 1 -- 172.21.15.162:0/2984447200 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4cc13d690 con 0x7fa4cc142dc0 2020-11-28T21:54:37.417 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.404+0000 7fa4d1541700 1 --2- 172.21.15.162:0/2984447200 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa4cc0ae230 0x7fa4cc152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:54:37.417 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.404+0000 7fa4d1d42700 1 --2- 172.21.15.162:0/2984447200 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa4cc0ade50 0x7fa4cc0ab3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:54:37.417 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.404+0000 7fa4d2543700 1 --2- 172.21.15.162:0/2984447200 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4cc142dc0 0x7fa4cc152510 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 rx=0x7fa4cc13d0b0 tx=0x7fa4c8015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:37.417 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.404+0000 7fa4baffd700 1 -- 172.21.15.162:0/2984447200 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4c8011b40 con 0x7fa4cc142dc0 2020-11-28T21:54:37.417 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.404+0000 7fa4d3fa6700 1 -- 172.21.15.162:0/2984447200 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4cc151b80 con 0x7fa4cc142dc0 2020-11-28T21:54:37.418 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.404+0000 7fa4baffd700 1 -- 172.21.15.162:0/2984447200 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c8011ca0 con 0x7fa4cc142dc0 2020-11-28T21:54:37.418 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.404+0000 7fa4baffd700 1 -- 172.21.15.162:0/2984447200 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4c801e460 con 0x7fa4cc142dc0 2020-11-28T21:54:37.418 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.404+0000 7fa4d3fa6700 1 -- 172.21.15.162:0/2984447200 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4cc0b4380 con 0x7fa4cc142dc0 2020-11-28T21:54:37.418 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.408+0000 7fa4d3fa6700 1 -- 172.21.15.162:0/2984447200 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4cc151d10 con 0x7fa4cc142dc0 2020-11-28T21:54:37.418 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.408+0000 7fa4baffd700 1 -- 172.21.15.162:0/2984447200 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa4c801a040 con 0x7fa4cc142dc0 2020-11-28T21:54:37.419 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.408+0000 7fa4baffd700 1 --2- 172.21.15.162:0/2984447200 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa4ac0554e0 0x7fa4ac0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:37.419 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.408+0000 7fa4baffd700 1 -- 172.21.15.162:0/2984447200 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa4c806f200 con 0x7fa4cc142dc0 2020-11-28T21:54:37.419 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.408+0000 7fa4d1d42700 1 --2- 172.21.15.162:0/2984447200 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa4ac0554e0 0x7fa4ac0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:37.419 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.408+0000 7fa4d1d42700 1 --2- 172.21.15.162:0/2984447200 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa4ac0554e0 0x7fa4ac0579a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7fa4c400d000 tx=0x7fa4c4009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:37.420 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.416+0000 7fa4baffd700 1 -- 172.21.15.162:0/2984447200 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa4c8045460 con 0x7fa4cc142dc0 2020-11-28T21:54:37.561 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.556+0000 7fa4d3fa6700 1 -- 172.21.15.162:0/2984447200 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "size", "format": "json-pretty"} v 0) v1 -- 0x7fa4cc057050 con 0x7fa4cc142dc0 2020-11-28T21:54:37.561 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.556+0000 7fa4baffd700 1 -- 172.21.15.162:0/2984447200 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "size", "format": "json-pretty"}]=0 v109) v1 ==== 123+0+64 (secure 0 0 0) 0x7fa4c8048a80 con 0x7fa4cc142dc0 2020-11-28T21:54:37.562 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:54:37.562 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2020-11-28T21:54:37.562 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": "cephfs_data", 2020-11-28T21:54:37.563 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_id": 20, 2020-11-28T21:54:37.563 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2 2020-11-28T21:54:37.563 INFO:teuthology.orchestra.run.smithi162.stdout:} 2020-11-28T21:54:37.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.560+0000 7fa4d3fa6700 1 -- 172.21.15.162:0/2984447200 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa4ac0554e0 msgr2=0x7fa4ac0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:37.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.560+0000 7fa4d3fa6700 1 --2- 172.21.15.162:0/2984447200 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa4ac0554e0 0x7fa4ac0579a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7fa4c400d000 tx=0x7fa4c4009040).stop 2020-11-28T21:54:37.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.560+0000 7fa4d3fa6700 1 -- 172.21.15.162:0/2984447200 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4cc142dc0 msgr2=0x7fa4cc152510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:37.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.560+0000 7fa4d3fa6700 1 --2- 172.21.15.162:0/2984447200 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4cc142dc0 0x7fa4cc152510 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 rx=0x7fa4cc13d0b0 tx=0x7fa4c8015000).stop 2020-11-28T21:54:37.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.560+0000 7fa4d3fa6700 1 -- 172.21.15.162:0/2984447200 shutdown_connections 2020-11-28T21:54:37.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.560+0000 7fa4d3fa6700 1 --2- 172.21.15.162:0/2984447200 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa4ac0554e0 0x7fa4ac0579a0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:37.566 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.560+0000 7fa4d3fa6700 1 --2- 172.21.15.162:0/2984447200 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa4cc0ade50 0x7fa4cc0ab3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:37.566 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.560+0000 7fa4d3fa6700 1 --2- 172.21.15.162:0/2984447200 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4cc142dc0 0x7fa4cc152510 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:37.566 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.560+0000 7fa4d3fa6700 1 --2- 172.21.15.162:0/2984447200 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa4cc0ae230 0x7fa4cc152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:37.567 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.560+0000 7fa4d3fa6700 1 -- 172.21.15.162:0/2984447200 >> 172.21.15.162:0/2984447200 conn(0x7fa4cc05c3f0 msgr2=0x7fa4cc0b5400 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:37.567 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.560+0000 7fa4d3fa6700 1 -- 172.21.15.162:0/2984447200 shutdown_connections 2020-11-28T21:54:37.567 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:37.560+0000 7fa4d3fa6700 1 -- 172.21.15.162:0/2984447200 wait complete. 2020-11-28T21:54:37.575 INFO:teuthology.orchestra.run.smithi192:> cd /home/ubuntu/cephtest/mnt.0 && sudo df . 2020-11-28T21:54:37.593 INFO:teuthology.orchestra.run.smithi192.stdout:Filesystem 1K-blocks Used Available Use% Mounted on 2020-11-28T21:54:37.593 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-fuse 354410496 0 354410496 0% /home/ubuntu/cephtest/mnt.0 2020-11-28T21:54:37.594 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-28T21:54:37.595 INFO:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:54:37.614 INFO:teuthology.orchestra.run.smithi118:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:54:37.634 INFO:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:54:37.652 INFO:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:54:37.670 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-28T21:54:37.671 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:37.672 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:54:37.672 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:54:37.720 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2020-11-28T21:54:37.720 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-28T21:54:37.721 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2020-11-28T21:54:37.721 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-28T21:54:37.722 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:54:37.800 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:ceph-fuse[12403]: fuse finished with error 0 and tester_r 0 2020-11-28T21:54:37.802 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:37.803 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:54:37.803 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:54:37.855 INFO:teuthology.orchestra.run.smithi192.stdout:ext2/ext3 2020-11-28T21:54:37.855 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-11-28T21:54:37.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-28T21:54:37.967 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:54:38.014 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:38.015 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:54:38.015 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:54:38.023 INFO:teuthology.orchestra.run.smithi096.stdout:fuseblk 2020-11-28T21:54:38.024 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-11-28T21:54:38.024 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi096.front.sepia.ceph.com... 2020-11-28T21:54:38.025 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-28T21:54:38.025 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:54:38.097 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[11822]: fuse finished with error 0 and tester_r 0 2020-11-28T21:54:38.099 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:38.100 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:54:38.100 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:54:38.147 INFO:teuthology.orchestra.run.smithi096.stdout:ext2/ext3 2020-11-28T21:54:38.148 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-11-28T21:54:38.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-28T21:54:38.210 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:54:38.218 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config rm mon mon_allow_pool_delete 2020-11-28T21:54:38.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.336+0000 7f5ea5bc0700 1 Processor -- start 2020-11-28T21:54:38.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.336+0000 7f5ea5bc0700 1 -- start start 2020-11-28T21:54:38.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.340+0000 7f5ea5bc0700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5ea013f6c0 0x7f5ea00669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:38.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.340+0000 7f5ea5bc0700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ea0065050 con 0x7f5ea0064c70 2020-11-28T21:54:38.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.340+0000 7f5ea5bc0700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ea013fc40 con 0x7f5ea0065750 2020-11-28T21:54:38.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.340+0000 7f5ea5bc0700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ea013fee0 con 0x7f5ea013f6c0 2020-11-28T21:54:38.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.340+0000 7f5ea53bf700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f5ea0065750 0x7f5ea013c260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:50560/0 (socket says 172.21.15.162:50560) 2020-11-28T21:54:38.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.340+0000 7f5ea53bf700 1 -- 172.21.15.162:0/144807761 learned_addr learned my addr 172.21.15.162:0/144807761 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:38.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.340+0000 7f5e9ffff700 1 --2- 172.21.15.162:0/144807761 >> v2:172.21.15.118:3300/0 conn(0x7f5ea013f6c0 0x7f5ea00669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:38.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.340+0000 7f5e9f7fe700 1 -- 172.21.15.162:0/144807761 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1893164983 0 0) 0x7f5ea013fc40 con 0x7f5ea0065750 2020-11-28T21:54:38.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.340+0000 7f5e9f7fe700 1 -- 172.21.15.162:0/144807761 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e8c003230 con 0x7f5ea0065750 2020-11-28T21:54:38.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.340+0000 7f5e9f7fe700 1 -- 172.21.15.162:0/144807761 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3161762779 0 0) 0x7f5ea0065050 con 0x7f5ea0064c70 2020-11-28T21:54:38.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.340+0000 7f5e9f7fe700 1 -- 172.21.15.162:0/144807761 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ea013fc40 con 0x7f5ea0064c70 2020-11-28T21:54:38.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.340+0000 7f5e9f7fe700 1 -- 172.21.15.162:0/144807761 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1892318477 0 0) 0x7f5e8c003230 con 0x7f5ea0065750 2020-11-28T21:54:38.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.340+0000 7f5e9f7fe700 1 -- 172.21.15.162:0/144807761 >> v2:172.21.15.118:3300/0 conn(0x7f5ea013f6c0 msgr2=0x7f5ea00669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:38.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.340+0000 7f5e9f7fe700 1 --2- 172.21.15.162:0/144807761 >> v2:172.21.15.118:3300/0 conn(0x7f5ea013f6c0 0x7f5ea00669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:38.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.340+0000 7f5e9f7fe700 1 -- 172.21.15.162:0/144807761 >> v1:172.21.15.118:6789/0 conn(0x7f5ea0064c70 legacy=0x7f5ea0066290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:38.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.340+0000 7f5e9f7fe700 1 -- 172.21.15.162:0/144807761 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea0140180 con 0x7f5ea0065750 2020-11-28T21:54:38.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.340+0000 7f5e9f7fe700 1 -- 172.21.15.162:0/144807761 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f5e98001a50 con 0x7f5ea0065750 2020-11-28T21:54:38.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.340+0000 7f5e9f7fe700 1 -- 172.21.15.162:0/144807761 >> v1:172.21.15.162:6789/0 conn(0x7f5ea0065750 legacy=0x7f5ea013c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:38.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.340+0000 7f5e9f7fe700 1 --2- 172.21.15.162:0/144807761 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5e8c0038b0 0x7f5e8c003cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:38.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.340+0000 7f5e9f7fe700 1 --2- 172.21.15.162:0/144807761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e8c0044d0 0x7f5e8c004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:38.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.340+0000 7f5e9f7fe700 1 --2- 172.21.15.162:0/144807761 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5e8c004e40 0x7f5e8c00b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:38.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.340+0000 7f5e9f7fe700 1 -- 172.21.15.162:0/144807761 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea013fee0 con 0x7f5e8c0038b0 2020-11-28T21:54:38.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.340+0000 7f5e9f7fe700 1 -- 172.21.15.162:0/144807761 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c00bc10 con 0x7f5e8c0044d0 2020-11-28T21:54:38.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.340+0000 7f5e9f7fe700 1 -- 172.21.15.162:0/144807761 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c00bd50 con 0x7f5e8c004e40 2020-11-28T21:54:38.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.340+0000 7f5e9ffff700 1 --2- 172.21.15.162:0/144807761 >> v2:172.21.15.118:3300/0 conn(0x7f5ea013f6c0 0x7f5ea00669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:54:38.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.340+0000 7f5e9ffff700 1 --2- 172.21.15.162:0/144807761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e8c0044d0 0x7f5e8c004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:38.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.340+0000 7f5ea4bbe700 1 --2- 172.21.15.162:0/144807761 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5e8c0038b0 0x7f5e8c003cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:38.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.340+0000 7f5ea53bf700 1 --2- 172.21.15.162:0/144807761 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5e8c004e40 0x7f5e8c00b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:38.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.340+0000 7f5e9ffff700 1 -- 172.21.15.162:0/144807761 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5e8c004e40 msgr2=0x7f5e8c00b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:38.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.340+0000 7f5e9ffff700 1 --2- 172.21.15.162:0/144807761 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5e8c004e40 0x7f5e8c00b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:38.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.340+0000 7f5e9ffff700 1 -- 172.21.15.162:0/144807761 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5e8c0038b0 msgr2=0x7f5e8c003cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:38.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.340+0000 7f5e9ffff700 1 --2- 172.21.15.162:0/144807761 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5e8c0038b0 0x7f5e8c003cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:38.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.340+0000 7f5e9ffff700 1 -- 172.21.15.162:0/144807761 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e8c003230 con 0x7f5e8c0044d0 2020-11-28T21:54:38.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.340+0000 7f5ea4bbe700 1 --2- 172.21.15.162:0/144807761 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5e8c0038b0 0x7f5e8c003cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:54:38.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.340+0000 7f5ea53bf700 1 -- 172.21.15.162:0/144807761 reap_dead start 2020-11-28T21:54:38.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.340+0000 7f5e9ffff700 1 --2- 172.21.15.162:0/144807761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e8c0044d0 0x7f5e8c004950 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 rx=0x7f5e8c0054a0 tx=0x7f5e90008040).ready entity=mon.1 client_cookie=68a73740519d6c17 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:38.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.340+0000 7f5e9f7fe700 1 -- 172.21.15.162:0/144807761 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e9000ec30 con 0x7f5e8c0044d0 2020-11-28T21:54:38.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.340+0000 7f5e9f7fe700 1 -- 172.21.15.162:0/144807761 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e9000ed90 con 0x7f5e8c0044d0 2020-11-28T21:54:38.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.340+0000 7f5e9f7fe700 1 -- 172.21.15.162:0/144807761 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e900125e0 con 0x7f5e8c0044d0 2020-11-28T21:54:38.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.340+0000 7f5ea5bc0700 1 -- 172.21.15.162:0/144807761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e8c0044d0 msgr2=0x7f5e8c004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:38.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.340+0000 7f5ea5bc0700 1 --2- 172.21.15.162:0/144807761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e8c0044d0 0x7f5e8c004950 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 rx=0x7f5e8c0054a0 tx=0x7f5e90008040).stop 2020-11-28T21:54:38.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.344+0000 7f5ea5bc0700 1 -- 172.21.15.162:0/144807761 shutdown_connections 2020-11-28T21:54:38.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.344+0000 7f5ea5bc0700 1 --2- 172.21.15.162:0/144807761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e8c0044d0 0x7f5e8c004950 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:38.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.344+0000 7f5ea5bc0700 1 -- 172.21.15.162:0/144807761 >> 172.21.15.162:0/144807761 conn(0x7f5ea00b5da0 msgr2=0x7f5ea00b6180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:38.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.344+0000 7f5ea5bc0700 1 -- 172.21.15.162:0/144807761 shutdown_connections 2020-11-28T21:54:38.355 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.344+0000 7f5ea5bc0700 1 -- 172.21.15.162:0/144807761 wait complete. 2020-11-28T21:54:38.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.344+0000 7f5ea5bc0700 1 Processor -- start 2020-11-28T21:54:38.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.344+0000 7f5ea5bc0700 1 -- start start 2020-11-28T21:54:38.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.344+0000 7f5ea5bc0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e8c0044d0 0x7f5ea005bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:38.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.344+0000 7f5ea5bc0700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5ea0144dd0 0x7f5ea00b5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:38.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.344+0000 7f5ea5bc0700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5ea00b5730 0x7f5ea013d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:38.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.344+0000 7f5ea5bc0700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea00b4220 con 0x7f5ea0144dd0 2020-11-28T21:54:38.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.344+0000 7f5ea5bc0700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea00b3bb0 con 0x7f5e8c0044d0 2020-11-28T21:54:38.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.344+0000 7f5ea5bc0700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea0144660 con 0x7f5ea00b5730 2020-11-28T21:54:38.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.344+0000 7f5ea4bbe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e8c0044d0 0x7f5ea005bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:38.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.344+0000 7f5ea4bbe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e8c0044d0 0x7f5ea005bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43724/0 (socket says 172.21.15.162:43724) 2020-11-28T21:54:38.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.344+0000 7f5ea4bbe700 1 -- 172.21.15.162:0/1842192173 learned_addr learned my addr 172.21.15.162:0/1842192173 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:38.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.344+0000 7f5ea53bf700 1 --2- 172.21.15.162:0/1842192173 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5ea00b5730 0x7f5ea013d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:38.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.344+0000 7f5e9ffff700 1 --2- 172.21.15.162:0/1842192173 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5ea0144dd0 0x7f5ea00b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:38.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.344+0000 7f5ea4bbe700 1 -- 172.21.15.162:0/1842192173 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5ea00b5730 msgr2=0x7f5ea013d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:38.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.344+0000 7f5ea4bbe700 1 --2- 172.21.15.162:0/1842192173 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5ea00b5730 0x7f5ea013d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:38.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.344+0000 7f5ea4bbe700 1 -- 172.21.15.162:0/1842192173 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5ea0144dd0 msgr2=0x7f5ea00b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:38.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.344+0000 7f5ea4bbe700 1 --2- 172.21.15.162:0/1842192173 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5ea0144dd0 0x7f5ea00b5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:38.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.344+0000 7f5ea4bbe700 1 -- 172.21.15.162:0/1842192173 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea0145eb0 con 0x7f5e8c0044d0 2020-11-28T21:54:38.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.344+0000 7f5ea4bbe700 1 --2- 172.21.15.162:0/1842192173 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e8c0044d0 0x7f5ea005bcb0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 rx=0x7f5e9400b000 tx=0x7f5e94009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:38.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.348+0000 7f5e9dffb700 1 -- 172.21.15.162:0/1842192173 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e9400c790 con 0x7f5e8c0044d0 2020-11-28T21:54:38.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.348+0000 7f5e9dffb700 1 -- 172.21.15.162:0/1842192173 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e9400c8f0 con 0x7f5e8c0044d0 2020-11-28T21:54:38.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.348+0000 7f5ea5bc0700 1 -- 172.21.15.162:0/1842192173 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ea01442b0 con 0x7f5e8c0044d0 2020-11-28T21:54:38.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.348+0000 7f5e9dffb700 1 -- 172.21.15.162:0/1842192173 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e9400cc50 con 0x7f5e8c0044d0 2020-11-28T21:54:38.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.348+0000 7f5ea5bc0700 1 -- 172.21.15.162:0/1842192173 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea00ad480 con 0x7f5e8c0044d0 2020-11-28T21:54:38.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.348+0000 7f5ea5bc0700 1 -- 172.21.15.162:0/1842192173 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ea0057050 con 0x7f5e8c0044d0 2020-11-28T21:54:38.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.348+0000 7f5e9dffb700 1 -- 172.21.15.162:0/1842192173 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5e9401b030 con 0x7f5e8c0044d0 2020-11-28T21:54:38.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.348+0000 7f5e9dffb700 1 --2- 172.21.15.162:0/1842192173 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5e80055440 0x7f5e80057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:38.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.348+0000 7f5e9dffb700 1 -- 172.21.15.162:0/1842192173 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5e9406df50 con 0x7f5e8c0044d0 2020-11-28T21:54:38.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.348+0000 7f5e9ffff700 1 --2- 172.21.15.162:0/1842192173 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5e80055440 0x7f5e80057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:38.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.352+0000 7f5e9ffff700 1 --2- 172.21.15.162:0/1842192173 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5e80055440 0x7f5e80057900 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f5e9001c000 tx=0x7f5e9001a000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:38.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.352+0000 7f5e9dffb700 1 -- 172.21.15.162:0/1842192173 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5e94044250 con 0x7f5e8c0044d0 2020-11-28T21:54:38.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.496+0000 7f5ea5bc0700 1 -- 172.21.15.162:0/1842192173 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f5ea0144820 con 0x7f5e8c0044d0 2020-11-28T21:54:38.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.504+0000 7f5e9dffb700 1 -- 172.21.15.162:0/1842192173 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5e94047870 con 0x7f5e8c0044d0 2020-11-28T21:54:38.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.508+0000 7f5ea5bc0700 1 -- 172.21.15.162:0/1842192173 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5e80055440 msgr2=0x7f5e80057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:38.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.508+0000 7f5ea5bc0700 1 --2- 172.21.15.162:0/1842192173 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5e80055440 0x7f5e80057900 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f5e9001c000 tx=0x7f5e9001a000).stop 2020-11-28T21:54:38.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.508+0000 7f5ea5bc0700 1 -- 172.21.15.162:0/1842192173 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e8c0044d0 msgr2=0x7f5ea005bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:38.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.508+0000 7f5ea5bc0700 1 --2- 172.21.15.162:0/1842192173 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e8c0044d0 0x7f5ea005bcb0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 rx=0x7f5e9400b000 tx=0x7f5e94009040).stop 2020-11-28T21:54:38.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.508+0000 7f5ea5bc0700 1 -- 172.21.15.162:0/1842192173 shutdown_connections 2020-11-28T21:54:38.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.508+0000 7f5ea5bc0700 1 --2- 172.21.15.162:0/1842192173 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5e80055440 0x7f5e80057900 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:38.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.508+0000 7f5ea5bc0700 1 --2- 172.21.15.162:0/1842192173 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5ea00b5730 0x7f5ea013d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:38.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.508+0000 7f5ea5bc0700 1 --2- 172.21.15.162:0/1842192173 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e8c0044d0 0x7f5ea005bcb0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:38.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.508+0000 7f5ea5bc0700 1 --2- 172.21.15.162:0/1842192173 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5ea0144dd0 0x7f5ea00b5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:38.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.508+0000 7f5ea5bc0700 1 -- 172.21.15.162:0/1842192173 >> 172.21.15.162:0/1842192173 conn(0x7f5ea00b5da0 msgr2=0x7f5ea00b08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:38.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.508+0000 7f5ea5bc0700 1 -- 172.21.15.162:0/1842192173 shutdown_connections 2020-11-28T21:54:38.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.508+0000 7f5ea5bc0700 1 -- 172.21.15.162:0/1842192173 wait complete. 2020-11-28T21:54:38.525 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Ended test tasks.cephfs.test_misc.TestMisc.test_filtered_df' 2020-11-28T21:54:38.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.624+0000 7f62a759e700 1 Processor -- start 2020-11-28T21:54:38.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.624+0000 7f62a759e700 1 -- start start 2020-11-28T21:54:38.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.624+0000 7f62a759e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f62a80b5bf0 0x7f62a8142980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:38.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.624+0000 7f62a759e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f62a80b3a60 0x7f62a813fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:38.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.624+0000 7f62a759e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62a805f380 con 0x7f62a8060cb0 2020-11-28T21:54:38.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.624+0000 7f62a759e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62a8143930 con 0x7f62a80b5bf0 2020-11-28T21:54:38.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.624+0000 7f62a759e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62a805f5b0 con 0x7f62a80b3a60 2020-11-28T21:54:38.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.624+0000 7f62a659c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f62a80b5bf0 0x7f62a8142980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:38.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.624+0000 7f62a659c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f62a80b5bf0 0x7f62a8142980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43732/0 (socket says 172.21.15.162:43732) 2020-11-28T21:54:38.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.624+0000 7f62a659c700 1 -- 172.21.15.162:0/254800619 learned_addr learned my addr 172.21.15.162:0/254800619 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:38.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.624+0000 7f62a6d9d700 1 --2- 172.21.15.162:0/254800619 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f62a80b3a60 0x7f62a813fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:38.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.628+0000 7f62a559a700 1 -- 172.21.15.162:0/254800619 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1029087811 0 0) 0x7f62a805f380 con 0x7f62a8060cb0 2020-11-28T21:54:38.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.628+0000 7f62a659c700 1 -- 172.21.15.162:0/254800619 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f62a80b3a60 msgr2=0x7f62a813fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:38.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.628+0000 7f62a659c700 1 --2- 172.21.15.162:0/254800619 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f62a80b3a60 0x7f62a813fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:38.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.628+0000 7f62a659c700 1 -- 172.21.15.162:0/254800619 >> v1:172.21.15.118:6789/0 conn(0x7f62a8060cb0 legacy=0x7f62a8060230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:38.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.628+0000 7f62a659c700 1 -- 172.21.15.162:0/254800619 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a805f850 con 0x7f62a80b5bf0 2020-11-28T21:54:38.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.628+0000 7f62a659c700 1 --2- 172.21.15.162:0/254800619 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62a80b5bf0 0x7f62a8142980 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 rx=0x7f629c002f30 tx=0x7f629c006e80).ready entity=mon.1 client_cookie=490f0347cd09303 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:38.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.628+0000 7f62a559a700 1 -- 172.21.15.162:0/254800619 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f629c0097f0 con 0x7f62a80b5bf0 2020-11-28T21:54:38.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.628+0000 7f62a559a700 1 -- 172.21.15.162:0/254800619 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629c009950 con 0x7f62a80b5bf0 2020-11-28T21:54:38.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.628+0000 7f62a559a700 1 -- 172.21.15.162:0/254800619 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f629c009cb0 con 0x7f62a80b5bf0 2020-11-28T21:54:38.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.628+0000 7f62a759e700 1 -- 172.21.15.162:0/254800619 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62a80b5bf0 msgr2=0x7f62a8142980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:38.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.628+0000 7f62a759e700 1 --2- 172.21.15.162:0/254800619 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62a80b5bf0 0x7f62a8142980 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 rx=0x7f629c002f30 tx=0x7f629c006e80).stop 2020-11-28T21:54:38.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.628+0000 7f62a759e700 1 -- 172.21.15.162:0/254800619 shutdown_connections 2020-11-28T21:54:38.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.628+0000 7f62a759e700 1 --2- 172.21.15.162:0/254800619 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f62a80b3a60 0x7f62a813fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:38.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.628+0000 7f62a759e700 1 --2- 172.21.15.162:0/254800619 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62a80b5bf0 0x7f62a8142980 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:38.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.628+0000 7f62a759e700 1 -- 172.21.15.162:0/254800619 >> 172.21.15.162:0/254800619 conn(0x7f62a805b970 msgr2=0x7f62a805bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:38.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.628+0000 7f62a759e700 1 -- 172.21.15.162:0/254800619 shutdown_connections 2020-11-28T21:54:38.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.628+0000 7f62a759e700 1 -- 172.21.15.162:0/254800619 wait complete. 2020-11-28T21:54:38.639 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.632+0000 7f62a759e700 1 Processor -- start 2020-11-28T21:54:38.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.632+0000 7f62a759e700 1 -- start start 2020-11-28T21:54:38.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.632+0000 7f62a759e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f62a8060cb0 0x7f62a80b0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:38.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.632+0000 7f62a759e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f62a80b3a60 0x7f62a81553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:38.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.632+0000 7f62a759e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62a80b5bf0 0x7f62a809df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:38.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.632+0000 7f62a759e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a80af520 con 0x7f62a80b3a60 2020-11-28T21:54:38.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.632+0000 7f62a759e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a81558c0 con 0x7f62a80b5bf0 2020-11-28T21:54:38.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.632+0000 7f62a759e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62a80acee0 con 0x7f62a8060cb0 2020-11-28T21:54:38.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.632+0000 7f62a6d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62a80b5bf0 0x7f62a809df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:38.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.632+0000 7f62a6d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62a80b5bf0 0x7f62a809df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43740/0 (socket says 172.21.15.162:43740) 2020-11-28T21:54:38.642 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.632+0000 7f62a6d9d700 1 -- 172.21.15.162:0/556060155 learned_addr learned my addr 172.21.15.162:0/556060155 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:38.642 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.632+0000 7f62a5d9b700 1 --2- 172.21.15.162:0/556060155 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f62a80b3a60 0x7f62a81553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:38.642 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.632+0000 7f62a659c700 1 --2- 172.21.15.162:0/556060155 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f62a8060cb0 0x7f62a80b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:38.642 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.632+0000 7f62a6d9d700 1 -- 172.21.15.162:0/556060155 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f62a8060cb0 msgr2=0x7f62a80b0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:38.643 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.632+0000 7f62a6d9d700 1 --2- 172.21.15.162:0/556060155 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f62a8060cb0 0x7f62a80b0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:38.643 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.632+0000 7f62a6d9d700 1 -- 172.21.15.162:0/556060155 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f62a80b3a60 msgr2=0x7f62a81553d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:38.643 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.632+0000 7f62a6d9d700 1 --2- 172.21.15.162:0/556060155 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f62a80b3a60 0x7f62a81553d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:38.643 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.632+0000 7f62a6d9d700 1 -- 172.21.15.162:0/556060155 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a80ad180 con 0x7f62a80b5bf0 2020-11-28T21:54:38.644 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.632+0000 7f62a6d9d700 1 --2- 172.21.15.162:0/556060155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62a80b5bf0 0x7f62a809df20 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 rx=0x7f62a0006dd0 tx=0x7f62a00091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:38.644 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.632+0000 7f62977fe700 1 -- 172.21.15.162:0/556060155 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62a000c950 con 0x7f62a80b5bf0 2020-11-28T21:54:38.644 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.632+0000 7f62977fe700 1 -- 172.21.15.162:0/556060155 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62a000cab0 con 0x7f62a80b5bf0 2020-11-28T21:54:38.644 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.632+0000 7f62977fe700 1 -- 172.21.15.162:0/556060155 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62a000ce10 con 0x7f62a80b5bf0 2020-11-28T21:54:38.644 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.632+0000 7f62a759e700 1 -- 172.21.15.162:0/556060155 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62a80ad420 con 0x7f62a80b5bf0 2020-11-28T21:54:38.645 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.632+0000 7f62a759e700 1 -- 172.21.15.162:0/556060155 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62a8140870 con 0x7f62a80b5bf0 2020-11-28T21:54:38.649 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.636+0000 7f62a759e700 1 -- 172.21.15.162:0/556060155 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62a8057050 con 0x7f62a80b5bf0 2020-11-28T21:54:38.649 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.636+0000 7f62977fe700 1 -- 172.21.15.162:0/556060155 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f62a0022070 con 0x7f62a80b5bf0 2020-11-28T21:54:38.649 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.636+0000 7f62977fe700 1 --2- 172.21.15.162:0/556060155 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f62840554e0 0x7f62840579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:38.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.636+0000 7f62977fe700 1 -- 172.21.15.162:0/556060155 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f62a006e5a0 con 0x7f62a80b5bf0 2020-11-28T21:54:38.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.636+0000 7f62a659c700 1 --2- 172.21.15.162:0/556060155 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f62840554e0 0x7f62840579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:38.651 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.636+0000 7f62a659c700 1 --2- 172.21.15.162:0/556060155 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f62840554e0 0x7f62840579a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f629c01c000 tx=0x7f629c015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:38.651 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.644+0000 7f62977fe700 1 -- 172.21.15.162:0/556060155 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f62a00448a0 con 0x7f62a80b5bf0 2020-11-28T21:54:38.840 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:38.836+0000 7f62a759e700 1 -- 172.21.15.162:0/556060155 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_filtered_df"]} v 0) v1 -- 0x7f62a8068350 con 0x7f62a80b5bf0 2020-11-28T21:54:38.921 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:54:38.924 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:54:38.926 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:54:38.928 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:54:39.854 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.852+0000 7f62977fe700 1 -- 172.21.15.162:0/556060155 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_filtered_df"]}]=0 v363) v1 ==== 127+0+0 (secure 0 0 0) 0x7f62a0016070 con 0x7f62a80b5bf0 2020-11-28T21:54:39.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.852+0000 7f62a759e700 1 -- 172.21.15.162:0/556060155 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f62840554e0 msgr2=0x7f62840579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:39.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.852+0000 7f62a759e700 1 --2- 172.21.15.162:0/556060155 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f62840554e0 0x7f62840579a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f629c01c000 tx=0x7f629c015000).stop 2020-11-28T21:54:39.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.852+0000 7f62a759e700 1 -- 172.21.15.162:0/556060155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62a80b5bf0 msgr2=0x7f62a809df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:39.858 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.852+0000 7f62a759e700 1 --2- 172.21.15.162:0/556060155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62a80b5bf0 0x7f62a809df20 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 rx=0x7f62a0006dd0 tx=0x7f62a00091a0).stop 2020-11-28T21:54:39.858 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.852+0000 7f62a759e700 1 -- 172.21.15.162:0/556060155 shutdown_connections 2020-11-28T21:54:39.859 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.852+0000 7f62a759e700 1 --2- 172.21.15.162:0/556060155 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f62840554e0 0x7f62840579a0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:39.859 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.852+0000 7f62a759e700 1 --2- 172.21.15.162:0/556060155 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f62a8060cb0 0x7f62a80b0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:39.859 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.852+0000 7f62a759e700 1 --2- 172.21.15.162:0/556060155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62a80b5bf0 0x7f62a809df20 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:39.859 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.852+0000 7f62a759e700 1 --2- 172.21.15.162:0/556060155 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f62a80b3a60 0x7f62a81553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:39.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.852+0000 7f62a759e700 1 -- 172.21.15.162:0/556060155 >> 172.21.15.162:0/556060155 conn(0x7f62a805b970 msgr2=0x7f62a8140e00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:39.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.856+0000 7f62a759e700 1 -- 172.21.15.162:0/556060155 shutdown_connections 2020-11-28T21:54:39.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.856+0000 7f62a759e700 1 -- 172.21.15.162:0/556060155 wait complete. 2020-11-28T21:54:39.873 INFO:tasks.cephfs_test_runner:test_filtered_df (tasks.cephfs.test_misc.TestMisc) ... ok 2020-11-28T21:54:39.874 INFO:tasks.cephfs_test_runner:Starting test: test_fs_new (tasks.cephfs.test_misc.TestMisc) 2020-11-28T21:54:39.874 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_misc.TestMisc.test_fs_new' 2020-11-28T21:54:39.997 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.992+0000 7f361b7af700 1 Processor -- start 2020-11-28T21:54:39.998 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.992+0000 7f361b7af700 1 -- start start 2020-11-28T21:54:40.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f361b7af700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f36140b1990 0x7f36140606c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:40.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f361b7af700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f36140b10b0 0x7f361413bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:40.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f361b7af700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3614143920 con 0x7f3614143440 2020-11-28T21:54:40.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f361b7af700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36140af7e0 con 0x7f36140b10b0 2020-11-28T21:54:40.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f361b7af700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36140b20e0 con 0x7f36140b1990 2020-11-28T21:54:40.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f3618d4a700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f3614143440 0x7f3614060bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:50590/0 (socket says 172.21.15.162:50590) 2020-11-28T21:54:40.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f3618d4a700 1 -- 172.21.15.162:0/891447183 learned_addr learned my addr 172.21.15.162:0/891447183 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:40.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f3619d4c700 1 --2- 172.21.15.162:0/891447183 >> v2:172.21.15.118:3300/0 conn(0x7f36140b10b0 0x7f361413bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:40.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f361954b700 1 --2- 172.21.15.162:0/891447183 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f36140b1990 0x7f36140606c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:40.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f360bfff700 1 -- 172.21.15.162:0/891447183 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1141267346 0 0) 0x7f3614143920 con 0x7f3614143440 2020-11-28T21:54:40.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f360bfff700 1 -- 172.21.15.162:0/891447183 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35f8003230 con 0x7f3614143440 2020-11-28T21:54:40.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f360bfff700 1 -- 172.21.15.162:0/891447183 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1008713173 0 0) 0x7f35f8003230 con 0x7f3614143440 2020-11-28T21:54:40.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f360bfff700 1 -- 172.21.15.162:0/891447183 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f36140b1990 msgr2=0x7f36140606c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:40.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f360bfff700 1 --2- 172.21.15.162:0/891447183 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f36140b1990 0x7f36140606c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:40.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f360bfff700 1 -- 172.21.15.162:0/891447183 >> v2:172.21.15.118:3300/0 conn(0x7f36140b10b0 msgr2=0x7f361413bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:40.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f360bfff700 1 --2- 172.21.15.162:0/891447183 >> v2:172.21.15.118:3300/0 conn(0x7f36140b10b0 0x7f361413bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:40.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f360bfff700 1 -- 172.21.15.162:0/891447183 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3614060290 con 0x7f3614143440 2020-11-28T21:54:40.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f360bfff700 1 -- 172.21.15.162:0/891447183 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f3610001a50 con 0x7f3614143440 2020-11-28T21:54:40.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f360bfff700 1 -- 172.21.15.162:0/891447183 >> v1:172.21.15.162:6789/0 conn(0x7f3614143440 legacy=0x7f3614060bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:40.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f360bfff700 1 --2- 172.21.15.162:0/891447183 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f35f8003720 0x7f35f8003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:40.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f360bfff700 1 --2- 172.21.15.162:0/891447183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35f80044d0 0x7f35f8004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:40.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f360bfff700 1 --2- 172.21.15.162:0/891447183 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f36140b1990 0x7f35f8004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:40.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f360bfff700 1 -- 172.21.15.162:0/891447183 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36140af7e0 con 0x7f35f8003720 2020-11-28T21:54:40.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f360bfff700 1 -- 172.21.15.162:0/891447183 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36140b20e0 con 0x7f35f80044d0 2020-11-28T21:54:40.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f360bfff700 1 -- 172.21.15.162:0/891447183 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35f8004260 con 0x7f36140b1990 2020-11-28T21:54:40.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f3618d4a700 1 --2- 172.21.15.162:0/891447183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35f80044d0 0x7f35f8004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:40.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f361954b700 1 --2- 172.21.15.162:0/891447183 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f35f8003720 0x7f35f8003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:40.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f3618d4a700 1 -- 172.21.15.162:0/891447183 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f36140b1990 msgr2=0x7f35f8004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:40.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f3618d4a700 1 --2- 172.21.15.162:0/891447183 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f36140b1990 0x7f35f8004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-28T21:54:40.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f3618d4a700 1 -- 172.21.15.162:0/891447183 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f35f8003720 msgr2=0x7f35f8003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:40.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f3618d4a700 1 --2- 172.21.15.162:0/891447183 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f35f8003720 0x7f35f8003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:40.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f3618d4a700 1 -- 172.21.15.162:0/891447183 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f8003230 con 0x7f35f80044d0 2020-11-28T21:54:40.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f361954b700 1 --2- 172.21.15.162:0/891447183 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f35f8003720 0x7f35f8003b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:54:40.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f3618d4a700 1 --2- 172.21.15.162:0/891447183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35f80044d0 0x7f35f8004950 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 rx=0x7f3610001760 tx=0x7f36140aca60).ready entity=mon.1 client_cookie=f25d0864aa33a54a server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:40.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f360bfff700 1 -- 172.21.15.162:0/891447183 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3610005b70 con 0x7f35f80044d0 2020-11-28T21:54:40.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f360bfff700 1 -- 172.21.15.162:0/891447183 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3610005cd0 con 0x7f35f80044d0 2020-11-28T21:54:40.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f360bfff700 1 -- 172.21.15.162:0/891447183 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36100072c0 con 0x7f35f80044d0 2020-11-28T21:54:40.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f361b7af700 1 -- 172.21.15.162:0/891447183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35f80044d0 msgr2=0x7f35f8004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:40.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f361b7af700 1 --2- 172.21.15.162:0/891447183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35f80044d0 0x7f35f8004950 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 rx=0x7f3610001760 tx=0x7f36140aca60).stop 2020-11-28T21:54:40.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f3619d4c700 1 -- 172.21.15.162:0/891447183 reap_dead start 2020-11-28T21:54:40.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f361b7af700 1 -- 172.21.15.162:0/891447183 shutdown_connections 2020-11-28T21:54:40.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f361b7af700 1 -- 172.21.15.162:0/891447183 >> 172.21.15.162:0/891447183 conn(0x7f361405b970 msgr2=0x7f361405bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:40.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f361b7af700 1 -- 172.21.15.162:0/891447183 shutdown_connections 2020-11-28T21:54:40.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:39.996+0000 7f361b7af700 1 -- 172.21.15.162:0/891447183 wait complete. 2020-11-28T21:54:40.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.000+0000 7f361b7af700 1 Processor -- start 2020-11-28T21:54:40.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.000+0000 7f361b7af700 1 -- start start 2020-11-28T21:54:40.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.000+0000 7f361b7af700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3614063880 0x7f3614155960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:40.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.000+0000 7f361b7af700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3614152030 0x7f36141524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:40.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.000+0000 7f361b7af700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3614149800 0x7f3614149c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:40.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.000+0000 7f361b7af700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36140aba80 con 0x7f3614152030 2020-11-28T21:54:40.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.000+0000 7f361b7af700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3614063c60 con 0x7f3614149800 2020-11-28T21:54:40.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.000+0000 7f361b7af700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36141529a0 con 0x7f3614063880 2020-11-28T21:54:40.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.000+0000 7f3619d4c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3614149800 0x7f3614149c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:40.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.000+0000 7f3619d4c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3614149800 0x7f3614149c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43762/0 (socket says 172.21.15.162:43762) 2020-11-28T21:54:40.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.000+0000 7f3619d4c700 1 -- 172.21.15.162:0/2378768846 learned_addr learned my addr 172.21.15.162:0/2378768846 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:40.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.000+0000 7f3618d4a700 1 --2- 172.21.15.162:0/2378768846 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3614152030 0x7f36141524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:40.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.000+0000 7f361954b700 1 --2- 172.21.15.162:0/2378768846 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3614063880 0x7f3614155960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:40.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.000+0000 7f3619d4c700 1 -- 172.21.15.162:0/2378768846 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3614063880 msgr2=0x7f3614155960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:40.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.000+0000 7f3619d4c700 1 --2- 172.21.15.162:0/2378768846 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3614063880 0x7f3614155960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:40.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.000+0000 7f3619d4c700 1 -- 172.21.15.162:0/2378768846 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3614152030 msgr2=0x7f36141524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:40.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.000+0000 7f3619d4c700 1 --2- 172.21.15.162:0/2378768846 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3614152030 0x7f36141524b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:40.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.000+0000 7f3619d4c700 1 -- 172.21.15.162:0/2378768846 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36140ae9e0 con 0x7f3614149800 2020-11-28T21:54:40.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.000+0000 7f3619d4c700 1 --2- 172.21.15.162:0/2378768846 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3614149800 0x7f3614149c80 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 rx=0x7f3614151f10 tx=0x7f360c006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:40.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.000+0000 7f360a7fc700 1 -- 172.21.15.162:0/2378768846 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f360c016070 con 0x7f3614149800 2020-11-28T21:54:40.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.000+0000 7f361b7af700 1 -- 172.21.15.162:0/2378768846 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36140aec80 con 0x7f3614149800 2020-11-28T21:54:40.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.000+0000 7f360a7fc700 1 -- 172.21.15.162:0/2378768846 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f360c011750 con 0x7f3614149800 2020-11-28T21:54:40.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.000+0000 7f360a7fc700 1 -- 172.21.15.162:0/2378768846 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f360c011ab0 con 0x7f3614149800 2020-11-28T21:54:40.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.000+0000 7f361b7af700 1 -- 172.21.15.162:0/2378768846 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3614151b20 con 0x7f3614149800 2020-11-28T21:54:40.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.004+0000 7f361b7af700 1 -- 172.21.15.162:0/2378768846 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36140ae840 con 0x7f3614149800 2020-11-28T21:54:40.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.004+0000 7f360a7fc700 1 -- 172.21.15.162:0/2378768846 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f360c011c10 con 0x7f3614149800 2020-11-28T21:54:40.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.004+0000 7f360a7fc700 1 --2- 172.21.15.162:0/2378768846 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f35fc055330 0x7f35fc0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:40.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.004+0000 7f360a7fc700 1 -- 172.21.15.162:0/2378768846 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f360c06db10 con 0x7f3614149800 2020-11-28T21:54:40.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.004+0000 7f361954b700 1 --2- 172.21.15.162:0/2378768846 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f35fc055330 0x7f35fc0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:40.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.004+0000 7f361954b700 1 --2- 172.21.15.162:0/2378768846 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f35fc055330 0x7f35fc0577f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f3604008000 tx=0x7f3604008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:40.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.008+0000 7f360a7fc700 1 -- 172.21.15.162:0/2378768846 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f36140ae840 con 0x7f3614149800 2020-11-28T21:54:40.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.196+0000 7f361b7af700 1 -- 172.21.15.162:0/2378768846 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_fs_new"]} v 0) v1 -- 0x7f36140ae840 con 0x7f3614149800 2020-11-28T21:54:40.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.852+0000 7f360a7fc700 1 -- 172.21.15.162:0/2378768846 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_fs_new"]}]=0 v364) v1 ==== 125+0+0 (secure 0 0 0) 0x7f36140ae840 con 0x7f3614149800 2020-11-28T21:54:40.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.856+0000 7f361b7af700 1 -- 172.21.15.162:0/2378768846 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f35fc055330 msgr2=0x7f35fc0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:40.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.856+0000 7f361b7af700 1 --2- 172.21.15.162:0/2378768846 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f35fc055330 0x7f35fc0577f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f3604008000 tx=0x7f3604008040).stop 2020-11-28T21:54:40.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.856+0000 7f361b7af700 1 -- 172.21.15.162:0/2378768846 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3614149800 msgr2=0x7f3614149c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:40.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.856+0000 7f361b7af700 1 --2- 172.21.15.162:0/2378768846 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3614149800 0x7f3614149c80 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 rx=0x7f3614151f10 tx=0x7f360c006dc0).stop 2020-11-28T21:54:40.862 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.856+0000 7f361b7af700 1 -- 172.21.15.162:0/2378768846 shutdown_connections 2020-11-28T21:54:40.862 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.856+0000 7f361b7af700 1 --2- 172.21.15.162:0/2378768846 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f35fc055330 0x7f35fc0577f0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:40.862 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.856+0000 7f361b7af700 1 --2- 172.21.15.162:0/2378768846 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3614063880 0x7f3614155960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:40.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.856+0000 7f361b7af700 1 --2- 172.21.15.162:0/2378768846 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3614149800 0x7f3614149c80 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:40.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.856+0000 7f361b7af700 1 --2- 172.21.15.162:0/2378768846 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3614152030 0x7f36141524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:40.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.856+0000 7f361b7af700 1 -- 172.21.15.162:0/2378768846 >> 172.21.15.162:0/2378768846 conn(0x7f361405b970 msgr2=0x7f36140b3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:40.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.856+0000 7f361b7af700 1 -- 172.21.15.162:0/2378768846 shutdown_connections 2020-11-28T21:54:40.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.856+0000 7f361b7af700 1 -- 172.21.15.162:0/2378768846 wait complete. 2020-11-28T21:54:40.877 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config set mon mon_allow_pool_delete True 2020-11-28T21:54:40.997 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.992+0000 7fd326a30700 1 Processor -- start 2020-11-28T21:54:40.997 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.992+0000 7fd326a30700 1 -- start start 2020-11-28T21:54:40.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.996+0000 7fd326a30700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd32005bcb0 0x7fd320061300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:41.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.996+0000 7fd326a30700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd320061a50 0x7fd3201427d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:41.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.996+0000 7fd326a30700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd32005c090 con 0x7fd3200b21a0 2020-11-28T21:54:41.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.996+0000 7fd326a30700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd320143820 con 0x7fd320061a50 2020-11-28T21:54:41.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.996+0000 7fd326a30700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3200617f0 con 0x7fd32005bcb0 2020-11-28T21:54:41.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.996+0000 7fd32522d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd320061a50 0x7fd3201427d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:41.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.996+0000 7fd32622f700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fd3200b21a0 0x7fd3200abfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.162:40762/0 (socket says 172.21.15.162:40762) 2020-11-28T21:54:41.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.996+0000 7fd32622f700 1 -- 172.21.15.162:0/758239275 learned_addr learned my addr 172.21.15.162:0/758239275 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:41.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.996+0000 7fd325a2e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd32005bcb0 0x7fd320061300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:41.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.996+0000 7fd324a2c700 1 -- 172.21.15.162:0/758239275 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3629407936 0 0) 0x7fd32005c090 con 0x7fd3200b21a0 2020-11-28T21:54:41.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.996+0000 7fd324a2c700 1 -- 172.21.15.162:0/758239275 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd308003230 con 0x7fd3200b21a0 2020-11-28T21:54:41.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.996+0000 7fd325a2e700 1 -- 172.21.15.162:0/758239275 >> v2:172.21.15.118:3300/0 conn(0x7fd320061a50 msgr2=0x7fd3201427d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:41.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.996+0000 7fd325a2e700 1 --2- 172.21.15.162:0/758239275 >> v2:172.21.15.118:3300/0 conn(0x7fd320061a50 0x7fd3201427d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:41.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.996+0000 7fd325a2e700 1 -- 172.21.15.162:0/758239275 >> v1:172.21.15.118:6789/0 conn(0x7fd3200b21a0 legacy=0x7fd3200abfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:41.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.996+0000 7fd325a2e700 1 -- 172.21.15.162:0/758239275 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd320061f60 con 0x7fd32005bcb0 2020-11-28T21:54:41.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.996+0000 7fd32522d700 1 --2- 172.21.15.162:0/758239275 >> v2:172.21.15.118:3300/0 conn(0x7fd320061a50 0x7fd3201427d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:54:41.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.996+0000 7fd325a2e700 1 --2- 172.21.15.162:0/758239275 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd32005bcb0 0x7fd320061300 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fd31c002f30 tx=0x7fd31c006dc0).ready entity=mon.2 client_cookie=ba6561d54261549a server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:41.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.996+0000 7fd324a2c700 1 -- 172.21.15.162:0/758239275 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd31c0096a0 con 0x7fd32005bcb0 2020-11-28T21:54:41.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.996+0000 7fd324a2c700 1 -- 172.21.15.162:0/758239275 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd31c009800 con 0x7fd32005bcb0 2020-11-28T21:54:41.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.996+0000 7fd324a2c700 1 -- 172.21.15.162:0/758239275 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd31c009b60 con 0x7fd32005bcb0 2020-11-28T21:54:41.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.996+0000 7fd326a30700 1 -- 172.21.15.162:0/758239275 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd32005bcb0 msgr2=0x7fd320061300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:41.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.996+0000 7fd326a30700 1 --2- 172.21.15.162:0/758239275 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd32005bcb0 0x7fd320061300 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fd31c002f30 tx=0x7fd31c006dc0).stop 2020-11-28T21:54:41.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.996+0000 7fd326a30700 1 -- 172.21.15.162:0/758239275 shutdown_connections 2020-11-28T21:54:41.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.996+0000 7fd326a30700 1 --2- 172.21.15.162:0/758239275 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd32005bcb0 0x7fd320061300 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:41.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.996+0000 7fd326a30700 1 --2- 172.21.15.162:0/758239275 >> v2:172.21.15.118:3300/0 conn(0x7fd320061a50 0x7fd3201427d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:41.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.996+0000 7fd326a30700 1 -- 172.21.15.162:0/758239275 >> 172.21.15.162:0/758239275 conn(0x7fd3200b2830 msgr2=0x7fd3200b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:41.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.996+0000 7fd326a30700 1 -- 172.21.15.162:0/758239275 shutdown_connections 2020-11-28T21:54:41.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:40.996+0000 7fd326a30700 1 -- 172.21.15.162:0/758239275 wait complete. 2020-11-28T21:54:41.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.000+0000 7fd326a30700 1 Processor -- start 2020-11-28T21:54:41.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.000+0000 7fd326a30700 1 -- start start 2020-11-28T21:54:41.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.000+0000 7fd326a30700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd32005bcb0 0x7fd32013eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:41.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.000+0000 7fd326a30700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd320061a50 0x7fd320151a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:41.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.000+0000 7fd326a30700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd3200b21a0 0x7fd320151f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:41.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.000+0000 7fd326a30700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3200afcb0 con 0x7fd3200b21a0 2020-11-28T21:54:41.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.000+0000 7fd326a30700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3201513a0 con 0x7fd320061a50 2020-11-28T21:54:41.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.000+0000 7fd326a30700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd32013f880 con 0x7fd32005bcb0 2020-11-28T21:54:41.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.000+0000 7fd32522d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd320061a50 0x7fd320151a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:41.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.000+0000 7fd32522d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd320061a50 0x7fd320151a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43774/0 (socket says 172.21.15.162:43774) 2020-11-28T21:54:41.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.000+0000 7fd32522d700 1 -- 172.21.15.162:0/2477490625 learned_addr learned my addr 172.21.15.162:0/2477490625 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:41.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.000+0000 7fd32622f700 1 --2- 172.21.15.162:0/2477490625 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd3200b21a0 0x7fd320151f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:41.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.000+0000 7fd325a2e700 1 --2- 172.21.15.162:0/2477490625 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd32005bcb0 0x7fd32013eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:41.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.000+0000 7fd32522d700 1 -- 172.21.15.162:0/2477490625 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd32005bcb0 msgr2=0x7fd32013eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:41.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.000+0000 7fd32522d700 1 --2- 172.21.15.162:0/2477490625 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd32005bcb0 0x7fd32013eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:41.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.000+0000 7fd32522d700 1 -- 172.21.15.162:0/2477490625 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd3200b21a0 msgr2=0x7fd320151f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:41.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.000+0000 7fd32522d700 1 --2- 172.21.15.162:0/2477490625 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd3200b21a0 0x7fd320151f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:41.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.000+0000 7fd32522d700 1 -- 172.21.15.162:0/2477490625 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd320151590 con 0x7fd320061a50 2020-11-28T21:54:41.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.000+0000 7fd32522d700 1 --2- 172.21.15.162:0/2477490625 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd320061a50 0x7fd320151a30 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 rx=0x7fd31000c000 tx=0x7fd310008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:41.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.004+0000 7fd30effd700 1 -- 172.21.15.162:0/2477490625 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd31000cb50 con 0x7fd320061a50 2020-11-28T21:54:41.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.004+0000 7fd30effd700 1 -- 172.21.15.162:0/2477490625 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd31000ccb0 con 0x7fd320061a50 2020-11-28T21:54:41.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.004+0000 7fd326a30700 1 -- 172.21.15.162:0/2477490625 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd32013d570 con 0x7fd320061a50 2020-11-28T21:54:41.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.004+0000 7fd30effd700 1 -- 172.21.15.162:0/2477490625 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3100125e0 con 0x7fd320061a50 2020-11-28T21:54:41.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.004+0000 7fd326a30700 1 -- 172.21.15.162:0/2477490625 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd32013e990 con 0x7fd320061a50 2020-11-28T21:54:41.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.004+0000 7fd326a30700 1 -- 172.21.15.162:0/2477490625 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd320057050 con 0x7fd320061a50 2020-11-28T21:54:41.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.004+0000 7fd30effd700 1 -- 172.21.15.162:0/2477490625 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd310022070 con 0x7fd320061a50 2020-11-28T21:54:41.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.004+0000 7fd30effd700 1 --2- 172.21.15.162:0/2477490625 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd3000553b0 0x7fd300057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:41.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.004+0000 7fd30effd700 1 -- 172.21.15.162:0/2477490625 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd31006e5c0 con 0x7fd320061a50 2020-11-28T21:54:41.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.004+0000 7fd325a2e700 1 --2- 172.21.15.162:0/2477490625 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd3000553b0 0x7fd300057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:41.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.004+0000 7fd325a2e700 1 --2- 172.21.15.162:0/2477490625 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd3000553b0 0x7fd300057870 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7fd31c019000 tx=0x7fd31c015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:41.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.012+0000 7fd30effd700 1 -- 172.21.15.162:0/2477490625 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd3100448c0 con 0x7fd320061a50 2020-11-28T21:54:41.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.160+0000 7fd326a30700 1 -- 172.21.15.162:0/2477490625 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7fd320068350 con 0x7fd320061a50 2020-11-28T21:54:41.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.168+0000 7fd30effd700 1 -- 172.21.15.162:0/2477490625 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd310047ee0 con 0x7fd320061a50 2020-11-28T21:54:41.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.172+0000 7fd326a30700 1 -- 172.21.15.162:0/2477490625 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd3000553b0 msgr2=0x7fd300057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:41.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.172+0000 7fd326a30700 1 --2- 172.21.15.162:0/2477490625 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd3000553b0 0x7fd300057870 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7fd31c019000 tx=0x7fd31c015000).stop 2020-11-28T21:54:41.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.172+0000 7fd326a30700 1 -- 172.21.15.162:0/2477490625 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd320061a50 msgr2=0x7fd320151a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:41.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.172+0000 7fd326a30700 1 --2- 172.21.15.162:0/2477490625 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd320061a50 0x7fd320151a30 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 rx=0x7fd31000c000 tx=0x7fd310008040).stop 2020-11-28T21:54:41.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.172+0000 7fd326a30700 1 -- 172.21.15.162:0/2477490625 shutdown_connections 2020-11-28T21:54:41.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.172+0000 7fd326a30700 1 --2- 172.21.15.162:0/2477490625 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd3000553b0 0x7fd300057870 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:41.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.172+0000 7fd326a30700 1 --2- 172.21.15.162:0/2477490625 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd32005bcb0 0x7fd32013eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:41.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.172+0000 7fd326a30700 1 --2- 172.21.15.162:0/2477490625 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd3200b21a0 0x7fd320151f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:41.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.172+0000 7fd326a30700 1 --2- 172.21.15.162:0/2477490625 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd320061a50 0x7fd320151a30 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:41.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.172+0000 7fd326a30700 1 -- 172.21.15.162:0/2477490625 >> 172.21.15.162:0/2477490625 conn(0x7fd3200b2830 msgr2=0x7fd3200b4d30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:41.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.172+0000 7fd326a30700 1 -- 172.21.15.162:0/2477490625 shutdown_connections 2020-11-28T21:54:41.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.172+0000 7fd326a30700 1 -- 172.21.15.162:0/2477490625 wait complete. 2020-11-28T21:54:41.195 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-28T21:54:41.196 INFO:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:54:41.215 INFO:teuthology.orchestra.run.smithi118:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:54:41.234 INFO:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:54:41.251 INFO:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:54:41.269 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-28T21:54:41.270 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:41.270 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:54:41.271 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:54:41.322 INFO:teuthology.orchestra.run.smithi192.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2020-11-28T21:54:41.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:54:41.325 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2020-11-28T21:54:41.325 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:41.326 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:54:41.327 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:54:41.334 INFO:teuthology.orchestra.run.smithi096.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-11-28T21:54:41.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:54:41.337 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2020-11-28T21:54:41.338 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2020-11-28T21:54:41.460 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.456+0000 7fd42a9c2700 1 Processor -- start 2020-11-28T21:54:41.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.456+0000 7fd42a9c2700 1 -- start start 2020-11-28T21:54:41.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.456+0000 7fd42a9c2700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd4240aae10 0x7fd42405f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:41.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.456+0000 7fd42a9c2700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd4240ab990 0x7fd4240ab1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:41.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.456+0000 7fd42a9c2700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd424140800 0x7fd424140c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:41.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.456+0000 7fd42a9c2700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4240a85c0 con 0x7fd4240aae10 2020-11-28T21:54:41.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.456+0000 7fd42a9c2700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4241448e0 con 0x7fd4240ab990 2020-11-28T21:54:41.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.456+0000 7fd42a9c2700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd424144b80 con 0x7fd424140800 2020-11-28T21:54:41.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.456+0000 7fd4291bf700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd4240ab990 0x7fd4240ab1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:41.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.456+0000 7fd4291bf700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd4240ab990 0x7fd4240ab1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43780/0 (socket says 172.21.15.162:43780) 2020-11-28T21:54:41.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.456+0000 7fd4291bf700 1 -- 172.21.15.162:0/3545551386 learned_addr learned my addr 172.21.15.162:0/3545551386 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:41.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.460+0000 7fd42a1c1700 1 --2- 172.21.15.162:0/3545551386 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd424140800 0x7fd424140c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:41.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.460+0000 7fd4299c0700 1 --2- 172.21.15.162:0/3545551386 >> v2:172.21.15.118:3300/0 conn(0x7fd4240aae10 0x7fd42405f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:41.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.460+0000 7fd4291bf700 1 -- 172.21.15.162:0/3545551386 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd424140800 msgr2=0x7fd424140c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:41.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.460+0000 7fd4291bf700 1 --2- 172.21.15.162:0/3545551386 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd424140800 0x7fd424140c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:41.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.460+0000 7fd4291bf700 1 -- 172.21.15.162:0/3545551386 >> v2:172.21.15.118:3300/0 conn(0x7fd4240aae10 msgr2=0x7fd42405f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:41.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.460+0000 7fd4291bf700 1 --2- 172.21.15.162:0/3545551386 >> v2:172.21.15.118:3300/0 conn(0x7fd4240aae10 0x7fd42405f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:41.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.460+0000 7fd4291bf700 1 -- 172.21.15.162:0/3545551386 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4240af6b0 con 0x7fd4240ab990 2020-11-28T21:54:41.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.460+0000 7fd4299c0700 1 --2- 172.21.15.162:0/3545551386 >> v2:172.21.15.118:3300/0 conn(0x7fd4240aae10 0x7fd42405f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:54:41.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.460+0000 7fd4291bf700 1 --2- 172.21.15.162:0/3545551386 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4240ab990 0x7fd4240ab1f0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 rx=0x7fd414002f30 tx=0x7fd414006e80).ready entity=mon.1 client_cookie=abf5d4a880e46fc9 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:41.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.460+0000 7fd4289be700 1 -- 172.21.15.162:0/3545551386 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4140097f0 con 0x7fd4240ab990 2020-11-28T21:54:41.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.460+0000 7fd4289be700 1 -- 172.21.15.162:0/3545551386 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd414009950 con 0x7fd4240ab990 2020-11-28T21:54:41.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.460+0000 7fd4289be700 1 -- 172.21.15.162:0/3545551386 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd414009cb0 con 0x7fd4240ab990 2020-11-28T21:54:41.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.460+0000 7fd42a9c2700 1 -- 172.21.15.162:0/3545551386 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4240ab990 msgr2=0x7fd4240ab1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:41.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.460+0000 7fd42a9c2700 1 --2- 172.21.15.162:0/3545551386 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4240ab990 0x7fd4240ab1f0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 rx=0x7fd414002f30 tx=0x7fd414006e80).stop 2020-11-28T21:54:41.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.460+0000 7fd42a9c2700 1 -- 172.21.15.162:0/3545551386 shutdown_connections 2020-11-28T21:54:41.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.460+0000 7fd42a9c2700 1 --2- 172.21.15.162:0/3545551386 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd424140800 0x7fd424140c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:41.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.460+0000 7fd42a9c2700 1 --2- 172.21.15.162:0/3545551386 >> v2:172.21.15.118:3300/0 conn(0x7fd4240aae10 0x7fd42405f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:41.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.460+0000 7fd42a9c2700 1 --2- 172.21.15.162:0/3545551386 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4240ab990 0x7fd4240ab1f0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:41.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.460+0000 7fd42a9c2700 1 -- 172.21.15.162:0/3545551386 >> 172.21.15.162:0/3545551386 conn(0x7fd42405b970 msgr2=0x7fd42405bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:41.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.460+0000 7fd42a9c2700 1 -- 172.21.15.162:0/3545551386 shutdown_connections 2020-11-28T21:54:41.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.460+0000 7fd42a9c2700 1 -- 172.21.15.162:0/3545551386 wait complete. 2020-11-28T21:54:41.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.460+0000 7fd42a9c2700 1 Processor -- start 2020-11-28T21:54:41.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.460+0000 7fd42a9c2700 1 -- start start 2020-11-28T21:54:41.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.460+0000 7fd42a9c2700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd4240aae10 0x7fd4240b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:41.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.460+0000 7fd42a9c2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4240ab990 0x7fd4240b53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:41.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.464+0000 7fd42a9c2700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd424140800 0x7fd424155410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:41.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.464+0000 7fd42a9c2700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd424145f40 con 0x7fd424140800 2020-11-28T21:54:41.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.464+0000 7fd42a9c2700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd424151ac0 con 0x7fd4240ab990 2020-11-28T21:54:41.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.464+0000 7fd42a9c2700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd424147970 con 0x7fd4240aae10 2020-11-28T21:54:41.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.464+0000 7fd4291bf700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4240ab990 0x7fd4240b53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:41.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.464+0000 7fd4291bf700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4240ab990 0x7fd4240b53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43790/0 (socket says 172.21.15.162:43790) 2020-11-28T21:54:41.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.464+0000 7fd42a1c1700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd424140800 0x7fd424155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:41.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.464+0000 7fd4299c0700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd4240aae10 0x7fd4240b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:41.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.464+0000 7fd4291bf700 1 -- 172.21.15.162:0/2337190997 learned_addr learned my addr 172.21.15.162:0/2337190997 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:41.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.464+0000 7fd4291bf700 1 -- 172.21.15.162:0/2337190997 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd4240aae10 msgr2=0x7fd4240b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:41.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.464+0000 7fd4291bf700 1 --2- 172.21.15.162:0/2337190997 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd4240aae10 0x7fd4240b4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:41.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.464+0000 7fd4291bf700 1 -- 172.21.15.162:0/2337190997 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd424140800 msgr2=0x7fd424155410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:41.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.464+0000 7fd4291bf700 1 --2- 172.21.15.162:0/2337190997 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd424140800 0x7fd424155410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:41.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.464+0000 7fd4291bf700 1 -- 172.21.15.162:0/2337190997 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd424147250 con 0x7fd4240ab990 2020-11-28T21:54:41.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.464+0000 7fd4291bf700 1 --2- 172.21.15.162:0/2337190997 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4240ab990 0x7fd4240b53d0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 rx=0x7fd414002f30 tx=0x7fd414015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:41.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.464+0000 7fd412ffd700 1 -- 172.21.15.162:0/2337190997 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd414011b80 con 0x7fd4240ab990 2020-11-28T21:54:41.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.464+0000 7fd412ffd700 1 -- 172.21.15.162:0/2337190997 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd414011ce0 con 0x7fd4240ab990 2020-11-28T21:54:41.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.464+0000 7fd42a9c2700 1 -- 172.21.15.162:0/2337190997 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd424146fa0 con 0x7fd4240ab990 2020-11-28T21:54:41.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.464+0000 7fd412ffd700 1 -- 172.21.15.162:0/2337190997 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd41401e650 con 0x7fd4240ab990 2020-11-28T21:54:41.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.464+0000 7fd42a9c2700 1 -- 172.21.15.162:0/2337190997 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd424145ab0 con 0x7fd4240ab990 2020-11-28T21:54:41.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.464+0000 7fd42a9c2700 1 -- 172.21.15.162:0/2337190997 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4241473e0 con 0x7fd4240ab990 2020-11-28T21:54:41.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.468+0000 7fd412ffd700 1 -- 172.21.15.162:0/2337190997 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd414025030 con 0x7fd4240ab990 2020-11-28T21:54:41.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.468+0000 7fd412ffd700 1 --2- 172.21.15.162:0/2337190997 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd4040553b0 0x7fd404057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:41.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.468+0000 7fd412ffd700 1 -- 172.21.15.162:0/2337190997 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd41406f1c0 con 0x7fd4240ab990 2020-11-28T21:54:41.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.468+0000 7fd4299c0700 1 --2- 172.21.15.162:0/2337190997 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd4040553b0 0x7fd404057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:41.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.468+0000 7fd4299c0700 1 --2- 172.21.15.162:0/2337190997 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd4040553b0 0x7fd404057870 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7fd42000c000 tx=0x7fd420008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:41.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.472+0000 7fd412ffd700 1 -- 172.21.15.162:0/2337190997 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd414045420 con 0x7fd4240ab990 2020-11-28T21:54:41.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.616+0000 7fd42a9c2700 1 -- 172.21.15.162:0/2337190997 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fd424057050 con 0x7fd4240ab990 2020-11-28T21:54:41.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.616+0000 7fd412ffd700 1 -- 172.21.15.162:0/2337190997 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v109) v1 ==== 81+0+33755 (secure 0 0 0) 0x7fd414048a40 con 0x7fd4240ab990 2020-11-28T21:54:41.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.620+0000 7fd42a9c2700 1 -- 172.21.15.162:0/2337190997 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd4040553b0 msgr2=0x7fd404057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:41.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.620+0000 7fd42a9c2700 1 --2- 172.21.15.162:0/2337190997 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd4040553b0 0x7fd404057870 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7fd42000c000 tx=0x7fd420008040).stop 2020-11-28T21:54:41.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.620+0000 7fd42a9c2700 1 -- 172.21.15.162:0/2337190997 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4240ab990 msgr2=0x7fd4240b53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:41.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.620+0000 7fd42a9c2700 1 --2- 172.21.15.162:0/2337190997 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4240ab990 0x7fd4240b53d0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 rx=0x7fd414002f30 tx=0x7fd414015000).stop 2020-11-28T21:54:41.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.620+0000 7fd42a9c2700 1 -- 172.21.15.162:0/2337190997 shutdown_connections 2020-11-28T21:54:41.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.620+0000 7fd42a9c2700 1 --2- 172.21.15.162:0/2337190997 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd4040553b0 0x7fd404057870 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:41.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.620+0000 7fd42a9c2700 1 --2- 172.21.15.162:0/2337190997 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd4240aae10 0x7fd4240b4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:41.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.620+0000 7fd42a9c2700 1 --2- 172.21.15.162:0/2337190997 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd424140800 0x7fd424155410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:41.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.620+0000 7fd42a9c2700 1 --2- 172.21.15.162:0/2337190997 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4240ab990 0x7fd4240b53d0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:41.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.620+0000 7fd42a9c2700 1 -- 172.21.15.162:0/2337190997 >> 172.21.15.162:0/2337190997 conn(0x7fd42405b970 msgr2=0x7fd424143c90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:41.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.620+0000 7fd42a9c2700 1 -- 172.21.15.162:0/2337190997 shutdown_connections 2020-11-28T21:54:41.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:41.620+0000 7fd42a9c2700 1 -- 172.21.15.162:0/2337190997 wait complete. 2020-11-28T21:54:41.629 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:54:41.630 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2020-11-28T21:54:41.631 INFO:teuthology.orchestra.run.smithi162.stdout: "epoch": 109, 2020-11-28T21:54:41.631 INFO:teuthology.orchestra.run.smithi162.stdout: "fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", 2020-11-28T21:54:41.632 INFO:teuthology.orchestra.run.smithi162.stdout: "created": "2020-11-28T21:46:11.724372+0000", 2020-11-28T21:54:41.632 INFO:teuthology.orchestra.run.smithi162.stdout: "modified": "2020-11-28T21:54:27.911877+0000", 2020-11-28T21:54:41.632 INFO:teuthology.orchestra.run.smithi162.stdout: "last_up_change": "2020-11-28T21:46:19.591650+0000", 2020-11-28T21:54:41.632 INFO:teuthology.orchestra.run.smithi162.stdout: "last_in_change": "2020-11-28T21:46:19.591650+0000", 2020-11-28T21:54:41.633 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-11-28T21:54:41.633 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_num": 5799936, 2020-11-28T21:54:41.633 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_set": [ 2020-11-28T21:54:41.633 INFO:teuthology.orchestra.run.smithi162.stdout: "pglog_hardlimit", 2020-11-28T21:54:41.633 INFO:teuthology.orchestra.run.smithi162.stdout: "purged_snapdirs", 2020-11-28T21:54:41.634 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_deletes", 2020-11-28T21:54:41.634 INFO:teuthology.orchestra.run.smithi162.stdout: "sortbitwise" 2020-11-28T21:54:41.634 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:54:41.634 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_version": 4, 2020-11-28T21:54:41.635 INFO:teuthology.orchestra.run.smithi162.stdout: "full_ratio": 0.94999998807907104, 2020-11-28T21:54:41.635 INFO:teuthology.orchestra.run.smithi162.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-11-28T21:54:41.635 INFO:teuthology.orchestra.run.smithi162.stdout: "nearfull_ratio": 0.85000002384185791, 2020-11-28T21:54:41.635 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_snapshot": "", 2020-11-28T21:54:41.636 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_max": 20, 2020-11-28T21:54:41.636 INFO:teuthology.orchestra.run.smithi162.stdout: "max_osd": 8, 2020-11-28T21:54:41.636 INFO:teuthology.orchestra.run.smithi162.stdout: "require_min_compat_client": "jewel", 2020-11-28T21:54:41.636 INFO:teuthology.orchestra.run.smithi162.stdout: "min_compat_client": "jewel", 2020-11-28T21:54:41.636 INFO:teuthology.orchestra.run.smithi162.stdout: "require_osd_release": "octopus", 2020-11-28T21:54:41.637 INFO:teuthology.orchestra.run.smithi162.stdout: "pools": [ 2020-11-28T21:54:41.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.637 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 1, 2020-11-28T21:54:41.637 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "device_health_metrics", 2020-11-28T21:54:41.638 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:46:15.639551+0000", 2020-11-28T21:54:41.639 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:54:41.639 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:54:41.639 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:54:41.640 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:54:41.640 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:54:41.640 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:54:41.640 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:54:41.640 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:54:41.641 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 1, 2020-11-28T21:54:41.641 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 1, 2020-11-28T21:54:41.641 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 1, 2020-11-28T21:54:41.641 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 1, 2020-11-28T21:54:41.642 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 1, 2020-11-28T21:54:41.642 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:54:41.642 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:54:41.642 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:54:41.642 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:54:41.643 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:54:41.643 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:54:41.643 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:54:41.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.644 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "16", 2020-11-28T21:54:41.644 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:54:41.644 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:54:41.644 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:54:41.645 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:54:41.645 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:54:41.646 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:54:41.646 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:54:41.647 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:54:41.647 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:54:41.647 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:54:41.648 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:54:41.648 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:54:41.648 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:54:41.649 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:54:41.649 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:54:41.649 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:54:41.650 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:54:41.650 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:54:41.650 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:54:41.651 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:54:41.651 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:54:41.651 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:54:41.652 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:54:41.652 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:54:41.652 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:54:41.653 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:54:41.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.653 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:54:41.654 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:54:41.654 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:54:41.654 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:54:41.655 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:54:41.655 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:54:41.655 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:54:41.656 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:54:41.656 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:54:41.656 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:54:41.657 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:54:41.657 INFO:teuthology.orchestra.run.smithi162.stdout: "options": { 2020-11-28T21:54:41.657 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_min": 1 2020-11-28T21:54:41.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.658 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:54:41.658 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr_devicehealth": {} 2020-11-28T21:54:41.659 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:41.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.660 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 2, 2020-11-28T21:54:41.660 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "rbd", 2020-11-28T21:54:41.660 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:46:23.901366+0000", 2020-11-28T21:54:41.661 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:54:41.661 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:54:41.661 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:54:41.662 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:54:41.662 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:54:41.662 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:54:41.663 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:54:41.663 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:54:41.663 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2020-11-28T21:54:41.664 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2020-11-28T21:54:41.664 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2020-11-28T21:54:41.664 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2020-11-28T21:54:41.665 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2020-11-28T21:54:41.665 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:54:41.665 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:54:41.665 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:54:41.666 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:54:41.666 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:54:41.666 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:54:41.667 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:54:41.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.667 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "19", 2020-11-28T21:54:41.668 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:54:41.668 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:54:41.668 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:54:41.669 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:54:41.669 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:54:41.669 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:54:41.670 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:54:41.670 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:54:41.670 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:54:41.671 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:54:41.672 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:54:41.672 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:54:41.673 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:54:41.673 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:54:41.673 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:54:41.674 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:54:41.674 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:54:41.675 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:54:41.675 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:54:41.675 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:54:41.676 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:54:41.676 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:54:41.676 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:54:41.677 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:54:41.677 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:54:41.677 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:54:41.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.678 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:54:41.678 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:54:41.678 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:54:41.679 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:54:41.680 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:54:41.680 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:54:41.681 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:54:41.681 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:54:41.681 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:54:41.682 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:54:41.682 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:54:41.682 INFO:teuthology.orchestra.run.smithi162.stdout: "options": {}, 2020-11-28T21:54:41.683 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:54:41.683 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd": {} 2020-11-28T21:54:41.683 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:41.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.684 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 19, 2020-11-28T21:54:41.685 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "cephfs_metadata", 2020-11-28T21:54:41.685 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:54:25.418443+0000", 2020-11-28T21:54:41.685 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:54:41.686 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:54:41.686 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:54:41.686 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:54:41.687 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:54:41.687 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:54:41.687 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:54:41.688 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:54:41.688 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2020-11-28T21:54:41.688 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2020-11-28T21:54:41.688 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2020-11-28T21:54:41.689 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2020-11-28T21:54:41.689 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2020-11-28T21:54:41.689 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:54:41.690 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:54:41.690 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:54:41.690 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:54:41.691 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:54:41.691 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:54:41.692 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:54:41.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.692 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "108", 2020-11-28T21:54:41.693 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:54:41.693 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:54:41.693 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:54:41.694 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:54:41.694 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:54:41.694 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:54:41.695 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:54:41.695 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:54:41.695 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:54:41.696 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:54:41.696 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:54:41.696 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:54:41.697 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:54:41.697 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:54:41.698 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:54:41.698 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:54:41.699 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:54:41.699 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:54:41.699 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:54:41.700 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:54:41.700 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:54:41.701 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:54:41.701 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:54:41.702 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:54:41.702 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:54:41.702 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:54:41.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.703 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:54:41.704 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:54:41.704 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:54:41.704 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:54:41.705 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:54:41.705 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:54:41.706 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:54:41.706 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:54:41.707 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:54:41.707 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:54:41.707 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:54:41.708 INFO:teuthology.orchestra.run.smithi162.stdout: "options": { 2020-11-28T21:54:41.708 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_bias": 4, 2020-11-28T21:54:41.709 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_min": 16, 2020-11-28T21:54:41.709 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_priority": 5 2020-11-28T21:54:41.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.710 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:54:41.710 INFO:teuthology.orchestra.run.smithi162.stdout: "cephfs": { 2020-11-28T21:54:41.710 INFO:teuthology.orchestra.run.smithi162.stdout: "metadata": "cephfs" 2020-11-28T21:54:41.711 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:41.711 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:41.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.712 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 20, 2020-11-28T21:54:41.712 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "cephfs_data", 2020-11-28T21:54:41.712 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:54:25.913773+0000", 2020-11-28T21:54:41.712 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:54:41.713 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:54:41.713 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:54:41.713 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:54:41.714 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:54:41.714 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:54:41.714 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:54:41.715 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:54:41.715 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2020-11-28T21:54:41.715 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2020-11-28T21:54:41.716 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2020-11-28T21:54:41.716 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2020-11-28T21:54:41.717 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2020-11-28T21:54:41.717 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:54:41.718 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:54:41.718 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:54:41.719 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:54:41.719 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:54:41.720 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:54:41.720 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:54:41.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.721 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "109", 2020-11-28T21:54:41.721 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:54:41.722 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:54:41.722 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:54:41.723 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:54:41.723 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:54:41.724 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:54:41.724 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:54:41.725 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:54:41.725 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:54:41.725 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:54:41.725 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:54:41.726 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:54:41.726 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:54:41.727 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:54:41.727 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:54:41.727 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:54:41.727 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:54:41.728 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:54:41.728 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:54:41.728 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:54:41.729 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:54:41.729 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:54:41.729 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:54:41.729 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:54:41.730 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:54:41.730 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:54:41.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.730 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:54:41.731 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:54:41.731 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:54:41.731 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:54:41.732 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:54:41.732 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:54:41.732 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:54:41.733 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:54:41.733 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:54:41.733 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:54:41.734 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:54:41.734 INFO:teuthology.orchestra.run.smithi162.stdout: "options": {}, 2020-11-28T21:54:41.735 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:54:41.735 INFO:teuthology.orchestra.run.smithi162.stdout: "cephfs": { 2020-11-28T21:54:41.736 INFO:teuthology.orchestra.run.smithi162.stdout: "data": "cephfs" 2020-11-28T21:54:41.736 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:41.737 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:41.737 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:41.738 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:54:41.738 INFO:teuthology.orchestra.run.smithi162.stdout: "osds": [ 2020-11-28T21:54:41.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.739 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 0, 2020-11-28T21:54:41.740 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "99c7d9ae-adfa-4fd7-880a-23e6baf3281b", 2020-11-28T21:54:41.740 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:54:41.741 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:54:41.741 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:54:41.742 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:54:41.742 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:54:41.742 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:54:41.743 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:54:41.744 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 107, 2020-11-28T21:54:41.744 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:54:41.745 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:54:41.746 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:54:41.746 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:41.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.747 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:41.747 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6810", 2020-11-28T21:54:41.748 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:54:41.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.749 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:41.750 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6811", 2020-11-28T21:54:41.750 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:54:41.751 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:41.751 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:41.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.752 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:54:41.753 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:41.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.754 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:41.754 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6812", 2020-11-28T21:54:41.755 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:54:41.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.756 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:41.757 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6813", 2020-11-28T21:54:41.757 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:54:41.758 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:41.758 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:41.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.759 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:54:41.760 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:41.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.761 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:41.761 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6816", 2020-11-28T21:54:41.762 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:54:41.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.763 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:41.763 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6817", 2020-11-28T21:54:41.764 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:54:41.764 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:41.765 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:41.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.766 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:54:41.766 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:41.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.767 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:41.768 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6814", 2020-11-28T21:54:41.768 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:54:41.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.769 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:41.770 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6815", 2020-11-28T21:54:41.770 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:54:41.771 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:41.771 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:41.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.782 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6811/12313", 2020-11-28T21:54:41.783 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6813/12313", 2020-11-28T21:54:41.783 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6817/12313", 2020-11-28T21:54:41.783 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6815/12313", 2020-11-28T21:54:41.783 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:54:41.784 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:54:41.784 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:54:41.784 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:41.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.785 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 1, 2020-11-28T21:54:41.785 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "6ce6220a-00ef-4d13-abba-56f87636fb5f", 2020-11-28T21:54:41.785 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:54:41.785 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:54:41.785 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:54:41.786 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:54:41.786 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:54:41.786 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:54:41.786 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:54:41.787 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 106, 2020-11-28T21:54:41.787 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:54:41.787 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:54:41.787 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:54:41.787 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:41.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.788 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:41.788 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6802", 2020-11-28T21:54:41.788 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:54:41.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.789 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:41.789 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6803", 2020-11-28T21:54:41.789 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:54:41.790 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:41.790 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:41.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.790 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:54:41.790 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:41.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.791 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:41.791 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6804", 2020-11-28T21:54:41.791 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:54:41.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.792 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:41.792 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6805", 2020-11-28T21:54:41.792 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:54:41.792 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:41.793 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:41.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.793 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:54:41.793 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:41.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.794 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:41.794 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6808", 2020-11-28T21:54:41.794 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:54:41.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.795 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:41.795 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6809", 2020-11-28T21:54:41.795 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:54:41.795 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:41.796 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:41.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.796 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:54:41.796 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:41.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.797 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:41.797 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6806", 2020-11-28T21:54:41.797 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:54:41.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.798 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:41.798 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6807", 2020-11-28T21:54:41.798 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:54:41.798 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:41.798 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:41.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.799 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6803/12311", 2020-11-28T21:54:41.799 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6805/12311", 2020-11-28T21:54:41.799 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6809/12311", 2020-11-28T21:54:41.799 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6807/12311", 2020-11-28T21:54:41.800 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:54:41.800 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:54:41.800 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:54:41.800 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:41.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.801 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 2, 2020-11-28T21:54:41.801 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "db4e510d-27c1-47fc-a902-8b32e7dfd73e", 2020-11-28T21:54:41.801 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:54:41.802 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:54:41.802 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:54:41.802 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:54:41.802 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:54:41.802 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:54:41.803 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:54:41.803 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 107, 2020-11-28T21:54:41.803 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:54:41.803 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:54:41.803 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:54:41.804 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:41.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.804 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:41.804 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6818", 2020-11-28T21:54:41.804 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:54:41.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.805 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:41.805 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6819", 2020-11-28T21:54:41.806 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:54:41.806 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:41.806 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:41.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.807 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:54:41.807 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:41.807 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.807 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:41.807 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6820", 2020-11-28T21:54:41.808 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:54:41.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.808 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:41.808 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6821", 2020-11-28T21:54:41.809 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:54:41.809 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:41.809 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:41.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.810 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:54:41.810 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:41.810 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.810 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:41.810 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6824", 2020-11-28T21:54:41.811 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:54:41.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.811 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:41.811 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6825", 2020-11-28T21:54:41.812 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:54:41.812 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:41.812 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:41.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.812 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:54:41.813 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:41.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.813 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:41.813 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6822", 2020-11-28T21:54:41.814 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:54:41.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.814 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:41.814 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6823", 2020-11-28T21:54:41.815 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:54:41.815 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:41.815 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:41.815 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.815 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6819/12312", 2020-11-28T21:54:41.816 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6821/12312", 2020-11-28T21:54:41.816 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6825/12312", 2020-11-28T21:54:41.816 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6823/12312", 2020-11-28T21:54:41.816 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:54:41.816 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:54:41.817 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:54:41.817 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:41.817 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.817 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.817 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 3, 2020-11-28T21:54:41.818 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "4094cb87-f04a-45e1-8710-fb4b651949f6", 2020-11-28T21:54:41.818 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:54:41.818 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:54:41.818 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:54:41.819 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:54:41.819 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:54:41.819 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:54:41.819 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:54:41.819 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 107, 2020-11-28T21:54:41.820 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:54:41.820 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:54:41.820 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:54:41.820 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:41.820 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.821 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:41.821 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6826", 2020-11-28T21:54:41.821 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:54:41.821 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.821 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.822 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:41.822 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6827", 2020-11-28T21:54:41.822 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:54:41.822 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:41.822 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:41.823 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.823 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:54:41.823 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:41.823 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.823 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:41.824 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6828", 2020-11-28T21:54:41.824 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:54:41.824 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.824 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.824 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:41.825 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6829", 2020-11-28T21:54:41.825 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:54:41.825 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:41.825 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:41.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.826 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:54:41.826 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:41.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.826 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:41.827 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6832", 2020-11-28T21:54:41.827 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:54:41.827 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.827 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.827 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:41.828 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6833", 2020-11-28T21:54:41.828 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:54:41.828 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:41.828 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:41.829 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.829 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:54:41.829 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:41.829 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.829 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:41.830 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6830", 2020-11-28T21:54:41.830 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:54:41.830 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.830 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.830 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:41.831 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6831", 2020-11-28T21:54:41.831 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:54:41.831 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:41.831 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:41.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.832 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6827/12323", 2020-11-28T21:54:41.832 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6829/12323", 2020-11-28T21:54:41.832 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6833/12323", 2020-11-28T21:54:41.832 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6831/12323", 2020-11-28T21:54:41.832 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:54:41.833 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:54:41.833 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:54:41.833 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:41.833 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.833 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.834 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 4, 2020-11-28T21:54:41.834 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "b907bf53-c297-4756-847b-3be96680eef5", 2020-11-28T21:54:41.834 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:54:41.834 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:54:41.834 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:54:41.835 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:54:41.835 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:54:41.835 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:54:41.835 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:54:41.835 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 106, 2020-11-28T21:54:41.836 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:54:41.836 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:54:41.836 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:54:41.836 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:41.836 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.837 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:41.837 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6800", 2020-11-28T21:54:41.837 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:54:41.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.838 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.838 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:41.838 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6801", 2020-11-28T21:54:41.838 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:54:41.838 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:41.839 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:41.839 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.839 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:54:41.839 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:41.839 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.840 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:41.840 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6802", 2020-11-28T21:54:41.840 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:54:41.840 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.841 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.841 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:41.841 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6803", 2020-11-28T21:54:41.841 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:54:41.841 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:41.842 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:41.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.842 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:54:41.842 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:41.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.843 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:41.843 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6806", 2020-11-28T21:54:41.843 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:54:41.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.844 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:41.844 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6807", 2020-11-28T21:54:41.844 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:54:41.844 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:41.844 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:41.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.845 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:54:41.845 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:41.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.846 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:41.846 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6804", 2020-11-28T21:54:41.846 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:54:41.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.847 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:41.847 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6805", 2020-11-28T21:54:41.847 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:54:41.847 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:41.847 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:41.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.848 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6801/12527", 2020-11-28T21:54:41.848 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6803/12527", 2020-11-28T21:54:41.848 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6807/12527", 2020-11-28T21:54:41.848 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6805/12527", 2020-11-28T21:54:41.849 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:54:41.849 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:54:41.849 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:54:41.849 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:41.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.850 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 5, 2020-11-28T21:54:41.850 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "55ba305d-0505-4212-aebb-631e8554b440", 2020-11-28T21:54:41.851 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:54:41.851 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:54:41.851 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:54:41.851 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:54:41.851 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:54:41.852 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:54:41.852 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:54:41.852 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 107, 2020-11-28T21:54:41.852 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:54:41.852 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:54:41.853 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:54:41.853 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:41.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.853 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:41.854 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6808", 2020-11-28T21:54:41.854 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:54:41.854 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.854 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.854 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:41.855 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6809", 2020-11-28T21:54:41.855 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:54:41.855 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:41.855 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:41.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.856 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:54:41.856 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:41.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.856 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:41.856 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6810", 2020-11-28T21:54:41.857 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:54:41.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.857 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:41.857 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6811", 2020-11-28T21:54:41.858 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:54:41.858 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:41.858 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:41.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.859 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:54:41.859 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:41.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.859 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:41.859 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6814", 2020-11-28T21:54:41.860 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:54:41.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.860 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.860 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:41.860 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6815", 2020-11-28T21:54:41.861 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:54:41.861 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:41.861 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:41.861 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.861 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:54:41.862 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:41.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.862 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:41.862 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6812", 2020-11-28T21:54:41.863 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:54:41.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.863 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:41.863 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6813", 2020-11-28T21:54:41.864 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:54:41.864 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:41.864 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:41.864 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.864 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6809/12529", 2020-11-28T21:54:41.865 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6811/12529", 2020-11-28T21:54:41.865 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6815/12529", 2020-11-28T21:54:41.865 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6813/12529", 2020-11-28T21:54:41.865 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:54:41.865 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:54:41.866 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:54:41.866 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:41.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.866 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 6, 2020-11-28T21:54:41.867 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "9ada6051-493b-46b5-ba22-1b401440490f", 2020-11-28T21:54:41.867 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:54:41.867 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:54:41.867 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:54:41.868 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:54:41.868 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:54:41.868 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:54:41.868 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:54:41.868 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 107, 2020-11-28T21:54:41.869 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:54:41.869 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:54:41.869 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:54:41.869 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:41.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.870 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:41.870 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6824", 2020-11-28T21:54:41.870 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:54:41.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.871 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:41.871 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6825", 2020-11-28T21:54:41.871 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:54:41.871 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:41.872 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:41.872 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.872 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:54:41.872 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:41.872 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.873 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:41.873 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6826", 2020-11-28T21:54:41.873 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:54:41.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.873 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.874 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:41.874 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6827", 2020-11-28T21:54:41.874 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:54:41.874 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:41.875 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:41.875 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.875 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:54:41.875 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:41.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.876 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:41.876 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6830", 2020-11-28T21:54:41.876 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:54:41.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.877 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.877 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:41.877 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6831", 2020-11-28T21:54:41.877 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:54:41.877 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:41.878 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:41.878 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.878 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:54:41.878 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:41.878 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.879 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:41.879 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6828", 2020-11-28T21:54:41.879 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:54:41.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.880 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:41.880 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6829", 2020-11-28T21:54:41.880 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:54:41.880 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:41.881 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:41.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.881 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6825/12530", 2020-11-28T21:54:41.881 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6827/12530", 2020-11-28T21:54:41.881 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6831/12530", 2020-11-28T21:54:41.882 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6829/12530", 2020-11-28T21:54:41.882 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:54:41.882 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:54:41.882 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:54:41.882 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:41.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.883 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 7, 2020-11-28T21:54:41.883 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "c328b06d-a216-4cba-aad7-9ad081340613", 2020-11-28T21:54:41.883 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:54:41.884 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:54:41.884 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:54:41.884 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:54:41.884 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:54:41.884 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:54:41.885 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:54:41.885 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 107, 2020-11-28T21:54:41.885 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:54:41.885 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:54:41.886 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:54:41.886 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:41.886 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.886 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:41.887 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6816", 2020-11-28T21:54:41.887 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:54:41.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.887 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:41.888 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6817", 2020-11-28T21:54:41.888 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:54:41.888 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:41.888 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:41.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.889 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:54:41.889 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:41.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.889 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:41.889 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6818", 2020-11-28T21:54:41.890 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:54:41.890 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.890 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.890 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:41.891 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6819", 2020-11-28T21:54:41.891 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:54:41.891 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:41.891 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:41.891 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.892 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:54:41.892 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:41.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.892 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:41.892 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6822", 2020-11-28T21:54:41.893 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:54:41.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.893 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:41.893 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6823", 2020-11-28T21:54:41.894 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:54:41.894 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:41.894 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:41.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.894 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:54:41.895 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:54:41.895 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.895 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:54:41.895 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6820", 2020-11-28T21:54:41.895 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:54:41.896 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.896 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.896 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:54:41.896 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6821", 2020-11-28T21:54:41.897 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:54:41.897 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:41.897 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:41.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.897 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6817/12528", 2020-11-28T21:54:41.898 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6819/12528", 2020-11-28T21:54:41.898 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6823/12528", 2020-11-28T21:54:41.898 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6821/12528", 2020-11-28T21:54:41.898 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:54:41.899 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:54:41.899 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:54:41.899 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:41.899 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:41.899 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:54:41.900 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_xinfo": [ 2020-11-28T21:54:41.900 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.900 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 0, 2020-11-28T21:54:41.900 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:54:41.900 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:54:41.901 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:54:41.901 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:54:41.901 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:54:41.901 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.038052+0000", 2020-11-28T21:54:41.902 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:54:41.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.902 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 1, 2020-11-28T21:54:41.903 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:54:41.903 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:54:41.903 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:54:41.903 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:54:41.903 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:54:41.904 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:17.947136+0000", 2020-11-28T21:54:41.904 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:54:41.904 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.904 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.905 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 2, 2020-11-28T21:54:41.905 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:54:41.905 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:54:41.905 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:54:41.906 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:54:41.906 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:54:41.906 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.001186+0000", 2020-11-28T21:54:41.906 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:54:41.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.907 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.907 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 3, 2020-11-28T21:54:41.907 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:54:41.907 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:54:41.908 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:54:41.908 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:54:41.908 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:54:41.908 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.199284+0000", 2020-11-28T21:54:41.909 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:54:41.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.909 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 4, 2020-11-28T21:54:41.910 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:54:41.910 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:54:41.910 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:54:41.910 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:54:41.910 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:54:41.911 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.038823+0000", 2020-11-28T21:54:41.911 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:54:41.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.912 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 5, 2020-11-28T21:54:41.912 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:54:41.912 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:54:41.912 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:54:41.913 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:54:41.913 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:54:41.913 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.009322+0000", 2020-11-28T21:54:41.913 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:54:41.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.914 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 6, 2020-11-28T21:54:41.914 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:54:41.914 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:54:41.915 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:54:41.915 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:54:41.915 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:54:41.915 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.238927+0000", 2020-11-28T21:54:41.916 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:54:41.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:41.916 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 7, 2020-11-28T21:54:41.917 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:54:41.917 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:54:41.917 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:54:41.917 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:54:41.918 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:54:41.918 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.029839+0000", 2020-11-28T21:54:41.918 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:54:41.918 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:41.919 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:54:41.919 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_upmap": [], 2020-11-28T21:54:41.919 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_upmap_items": [], 2020-11-28T21:54:41.919 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_temp": [], 2020-11-28T21:54:41.919 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_temp": [], 2020-11-28T21:54:41.920 INFO:teuthology.orchestra.run.smithi162.stdout: "blacklist": {}, 2020-11-28T21:54:41.920 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profiles": { 2020-11-28T21:54:41.920 INFO:teuthology.orchestra.run.smithi162.stdout: "default": { 2020-11-28T21:54:41.920 INFO:teuthology.orchestra.run.smithi162.stdout: "crush-failure-domain": "osd", 2020-11-28T21:54:41.921 INFO:teuthology.orchestra.run.smithi162.stdout: "k": "2", 2020-11-28T21:54:41.921 INFO:teuthology.orchestra.run.smithi162.stdout: "m": "1", 2020-11-28T21:54:41.921 INFO:teuthology.orchestra.run.smithi162.stdout: "plugin": "jerasure", 2020-11-28T21:54:41.921 INFO:teuthology.orchestra.run.smithi162.stdout: "ruleset-failure-domain": "osd", 2020-11-28T21:54:41.922 INFO:teuthology.orchestra.run.smithi162.stdout: "technique": "reed_sol_van" 2020-11-28T21:54:41.922 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:41.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:41.923 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps_queue": [], 2020-11-28T21:54:41.923 INFO:teuthology.orchestra.run.smithi162.stdout: "new_removed_snaps": [], 2020-11-28T21:54:41.923 INFO:teuthology.orchestra.run.smithi162.stdout: "new_purged_snaps": [], 2020-11-28T21:54:41.924 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_node_flags": {}, 2020-11-28T21:54:41.924 INFO:teuthology.orchestra.run.smithi162.stdout: "device_class_flags": {} 2020-11-28T21:54:41.924 INFO:teuthology.orchestra.run.smithi162.stdout:} 2020-11-28T21:54:41.927 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:54:42.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.044+0000 7fe4a2b39700 1 Processor -- start 2020-11-28T21:54:42.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.044+0000 7fe4a2b39700 1 -- start start 2020-11-28T21:54:42.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.048+0000 7fe4a2b39700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe49c0b4bd0 0x7fe49c098660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:42.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.048+0000 7fe4a2b39700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe49c0b33f0 0x7fe49c09ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:42.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.048+0000 7fe4a2b39700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe49c0b4fb0 con 0x7fe49c09b250 2020-11-28T21:54:42.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.048+0000 7fe4a2b39700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe49c098b50 con 0x7fe49c0b4bd0 2020-11-28T21:54:42.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.048+0000 7fe4a2b39700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe49c142a60 con 0x7fe49c0b33f0 2020-11-28T21:54:42.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.048+0000 7fe4a2338700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fe49c09b250 0x7fe49c142ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:50642/0 (socket says 172.21.15.162:50642) 2020-11-28T21:54:42.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.048+0000 7fe4a1336700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe49c0b33f0 0x7fe49c09ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:42.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.048+0000 7fe4a2338700 1 -- 172.21.15.162:0/3130823984 learned_addr learned my addr 172.21.15.162:0/3130823984 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:42.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.048+0000 7fe4a1b37700 1 --2- 172.21.15.162:0/3130823984 >> v2:172.21.15.118:3300/0 conn(0x7fe49c0b4bd0 0x7fe49c098660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:42.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.048+0000 7fe4a1336700 1 -- 172.21.15.162:0/3130823984 >> v2:172.21.15.118:3300/0 conn(0x7fe49c0b4bd0 msgr2=0x7fe49c098660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:42.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.048+0000 7fe4a1336700 1 --2- 172.21.15.162:0/3130823984 >> v2:172.21.15.118:3300/0 conn(0x7fe49c0b4bd0 0x7fe49c098660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:42.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.048+0000 7fe4a1336700 1 -- 172.21.15.162:0/3130823984 >> v1:172.21.15.162:6789/0 conn(0x7fe49c09b250 legacy=0x7fe49c142ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:42.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.048+0000 7fe4a1336700 1 -- 172.21.15.162:0/3130823984 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe49c09b670 con 0x7fe49c0b33f0 2020-11-28T21:54:42.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.048+0000 7fe4a0b35700 1 -- 172.21.15.162:0/3130823984 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2897099800 0 0) 0x7fe49c0b4fb0 con 0x7fe49c09b250 2020-11-28T21:54:42.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.048+0000 7fe4a1b37700 1 --2- 172.21.15.162:0/3130823984 >> v2:172.21.15.118:3300/0 conn(0x7fe49c0b4bd0 0x7fe49c098660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:54:42.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.048+0000 7fe4a1336700 1 --2- 172.21.15.162:0/3130823984 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe49c0b33f0 0x7fe49c09ad60 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 rx=0x7fe490002f30 tx=0x7fe490006dc0).ready entity=mon.1 client_cookie=bf1efee49b67ae6e server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:42.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.048+0000 7fe4a0b35700 1 -- 172.21.15.162:0/3130823984 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe4900097f0 con 0x7fe49c0b33f0 2020-11-28T21:54:42.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.048+0000 7fe4a0b35700 1 -- 172.21.15.162:0/3130823984 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe490009950 con 0x7fe49c0b33f0 2020-11-28T21:54:42.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.048+0000 7fe4a0b35700 1 -- 172.21.15.162:0/3130823984 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe490009cb0 con 0x7fe49c0b33f0 2020-11-28T21:54:42.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.048+0000 7fe4a2b39700 1 -- 172.21.15.162:0/3130823984 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe49c0b33f0 msgr2=0x7fe49c09ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:42.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.048+0000 7fe4a2b39700 1 --2- 172.21.15.162:0/3130823984 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe49c0b33f0 0x7fe49c09ad60 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 rx=0x7fe490002f30 tx=0x7fe490006dc0).stop 2020-11-28T21:54:42.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.048+0000 7fe4a2b39700 1 -- 172.21.15.162:0/3130823984 shutdown_connections 2020-11-28T21:54:42.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.048+0000 7fe4a2b39700 1 --2- 172.21.15.162:0/3130823984 >> v2:172.21.15.118:3300/0 conn(0x7fe49c0b4bd0 0x7fe49c098660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:42.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.048+0000 7fe4a2b39700 1 --2- 172.21.15.162:0/3130823984 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe49c0b33f0 0x7fe49c09ad60 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:42.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.048+0000 7fe4a2b39700 1 -- 172.21.15.162:0/3130823984 >> 172.21.15.162:0/3130823984 conn(0x7fe49c0b54f0 msgr2=0x7fe49c0b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:42.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.048+0000 7fe4a2b39700 1 -- 172.21.15.162:0/3130823984 shutdown_connections 2020-11-28T21:54:42.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.048+0000 7fe4a2b39700 1 -- 172.21.15.162:0/3130823984 wait complete. 2020-11-28T21:54:42.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.052+0000 7fe4a2b39700 1 Processor -- start 2020-11-28T21:54:42.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.052+0000 7fe4a2b39700 1 -- start start 2020-11-28T21:54:42.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.052+0000 7fe4a2b39700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe49c09b250 0x7fe49c0677c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:42.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.052+0000 7fe4a2b39700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe49c0b33f0 0x7fe49c067010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:42.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.052+0000 7fe4a2b39700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe49c0b4bd0 0x7fe49c091910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:42.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.052+0000 7fe4a2b39700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c08ff50 con 0x7fe49c09b250 2020-11-28T21:54:42.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.052+0000 7fe4a2b39700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c065f20 con 0x7fe49c0b4bd0 2020-11-28T21:54:42.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.052+0000 7fe4a2b39700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c0661c0 con 0x7fe49c0b33f0 2020-11-28T21:54:42.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.052+0000 7fe4a2338700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe49c0b4bd0 0x7fe49c091910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:42.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.052+0000 7fe4a2338700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe49c0b4bd0 0x7fe49c091910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43804/0 (socket says 172.21.15.162:43804) 2020-11-28T21:54:42.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.052+0000 7fe4a2338700 1 -- 172.21.15.162:0/212587333 learned_addr learned my addr 172.21.15.162:0/212587333 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:42.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.052+0000 7fe4a1b37700 1 --2- 172.21.15.162:0/212587333 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe49c09b250 0x7fe49c0677c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:42.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.052+0000 7fe4a1336700 1 --2- 172.21.15.162:0/212587333 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe49c0b33f0 0x7fe49c067010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:42.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.052+0000 7fe4a2338700 1 -- 172.21.15.162:0/212587333 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe49c0b33f0 msgr2=0x7fe49c067010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:42.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.052+0000 7fe4a2338700 1 --2- 172.21.15.162:0/212587333 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe49c0b33f0 0x7fe49c067010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:42.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.052+0000 7fe4a2338700 1 -- 172.21.15.162:0/212587333 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe49c09b250 msgr2=0x7fe49c0677c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:42.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.052+0000 7fe4a2338700 1 --2- 172.21.15.162:0/212587333 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe49c09b250 0x7fe49c0677c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:42.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.052+0000 7fe4a2338700 1 -- 172.21.15.162:0/212587333 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe49c066460 con 0x7fe49c0b4bd0 2020-11-28T21:54:42.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.052+0000 7fe4a2338700 1 --2- 172.21.15.162:0/212587333 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe49c0b4bd0 0x7fe49c091910 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 rx=0x7fe494000f00 tx=0x7fe494007d50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:42.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.052+0000 7fe49affd700 1 -- 172.21.15.162:0/212587333 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe49400ce60 con 0x7fe49c0b4bd0 2020-11-28T21:54:42.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.052+0000 7fe49affd700 1 -- 172.21.15.162:0/212587333 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe494005b70 con 0x7fe49c0b4bd0 2020-11-28T21:54:42.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.052+0000 7fe4a2b39700 1 -- 172.21.15.162:0/212587333 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe49c067500 con 0x7fe49c0b4bd0 2020-11-28T21:54:42.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.052+0000 7fe49affd700 1 -- 172.21.15.162:0/212587333 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe494007110 con 0x7fe49c0b4bd0 2020-11-28T21:54:42.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.052+0000 7fe4a2b39700 1 -- 172.21.15.162:0/212587333 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe49c064b50 con 0x7fe49c0b4bd0 2020-11-28T21:54:42.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.056+0000 7fe4a2b39700 1 -- 172.21.15.162:0/212587333 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe49c091710 con 0x7fe49c0b4bd0 2020-11-28T21:54:42.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.056+0000 7fe49affd700 1 -- 172.21.15.162:0/212587333 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe494010030 con 0x7fe49c0b4bd0 2020-11-28T21:54:42.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.056+0000 7fe49affd700 1 --2- 172.21.15.162:0/212587333 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe47c0553b0 0x7fe47c057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:42.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.056+0000 7fe49affd700 1 -- 172.21.15.162:0/212587333 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe49406df30 con 0x7fe49c0b4bd0 2020-11-28T21:54:42.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.056+0000 7fe4a1b37700 1 --2- 172.21.15.162:0/212587333 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe47c0553b0 0x7fe47c057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:42.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.056+0000 7fe4a1b37700 1 --2- 172.21.15.162:0/212587333 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe47c0553b0 0x7fe47c057870 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7fe48c00b000 tx=0x7fe48c008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:42.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.064+0000 7fe49affd700 1 -- 172.21.15.162:0/212587333 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe494044230 con 0x7fe49c0b4bd0 2020-11-28T21:54:42.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.252+0000 7fe4a2b39700 1 -- 172.21.15.162:0/212587333 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe49c057050 con 0x7fe49c0b4bd0 2020-11-28T21:54:42.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.252+0000 7fe49affd700 1 -- 172.21.15.162:0/212587333 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+2499 (secure 0 0 0) 0x7fe494047850 con 0x7fe49c0b4bd0 2020-11-28T21:54:42.256 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:54:42.256 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":51,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6182,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2775691244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2775691244},{"type":"v1","addr":"172.21.15.162:6835","nonce":2775691244}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":49},{"gid":6942,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/1896314020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1896314020},{"type":"v1","addr":"172.21.15.118:6833","nonce":1896314020}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":49}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:54:26.908249+0000","modified":"2020-11-28T21:54:27.911138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7177},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7177":{"gid":7177,"name":"a","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/493776989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":493776989},{"type":"v1","addr":"172.21.15.162:6837","nonce":493776989}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2020-11-28T21:54:42.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.256+0000 7fe4a2b39700 1 -- 172.21.15.162:0/212587333 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe47c0553b0 msgr2=0x7fe47c057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:42.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.256+0000 7fe4a2b39700 1 --2- 172.21.15.162:0/212587333 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe47c0553b0 0x7fe47c057870 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7fe48c00b000 tx=0x7fe48c008040).stop 2020-11-28T21:54:42.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.256+0000 7fe4a2b39700 1 -- 172.21.15.162:0/212587333 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe49c0b4bd0 msgr2=0x7fe49c091910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:42.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.256+0000 7fe4a2b39700 1 --2- 172.21.15.162:0/212587333 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe49c0b4bd0 0x7fe49c091910 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 rx=0x7fe494000f00 tx=0x7fe494007d50).stop 2020-11-28T21:54:42.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.256+0000 7fe4a2b39700 1 -- 172.21.15.162:0/212587333 shutdown_connections 2020-11-28T21:54:42.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.256+0000 7fe4a2b39700 1 --2- 172.21.15.162:0/212587333 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe47c0553b0 0x7fe47c057870 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:42.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.256+0000 7fe4a2b39700 1 --2- 172.21.15.162:0/212587333 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe49c0b4bd0 0x7fe49c091910 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:42.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.256+0000 7fe4a2b39700 1 --2- 172.21.15.162:0/212587333 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe49c09b250 0x7fe49c0677c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:42.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.256+0000 7fe4a2b39700 1 --2- 172.21.15.162:0/212587333 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe49c0b33f0 0x7fe49c067010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:42.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.256+0000 7fe4a2b39700 1 -- 172.21.15.162:0/212587333 >> 172.21.15.162:0/212587333 conn(0x7fe49c0b54f0 msgr2=0x7fe49c0b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:42.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.256+0000 7fe4a2b39700 1 -- 172.21.15.162:0/212587333 shutdown_connections 2020-11-28T21:54:42.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.256+0000 7fe4a2b39700 1 -- 172.21.15.162:0/212587333 wait complete. 2020-11-28T21:54:42.262 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 51 2020-11-28T21:54:42.274 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2020-11-28T21:54:42.386 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.384+0000 7f321515f700 1 Processor -- start 2020-11-28T21:54:42.387 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.384+0000 7f321515f700 1 -- start start 2020-11-28T21:54:42.388 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.384+0000 7f321515f700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f32100ac790 0x7f3210142bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:42.388 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.384+0000 7f321515f700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f32100acca0 0x7f32100b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:42.389 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.384+0000 7f321515f700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32100b07e0 con 0x7f32100ac3b0 2020-11-28T21:54:42.389 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.384+0000 7f321515f700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32100abe90 con 0x7f32100acca0 2020-11-28T21:54:42.389 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.384+0000 7f321515f700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32100b0a80 con 0x7f32100ac790 2020-11-28T21:54:42.389 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.384+0000 7f320f7fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f32100ac790 0x7f3210142bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:42.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.384+0000 7f320f7fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f32100ac790 0x7f3210142bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43812/0 (socket says 172.21.15.162:43812) 2020-11-28T21:54:42.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.384+0000 7f320f7fe700 1 -- 172.21.15.162:0/1242933211 learned_addr learned my addr 172.21.15.162:0/1242933211 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:42.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.384+0000 7f321495e700 1 --2- 172.21.15.162:0/1242933211 >> v2:172.21.15.118:3300/0 conn(0x7f32100acca0 0x7f32100b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:42.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.384+0000 7f320f7fe700 1 -- 172.21.15.162:0/1242933211 >> v2:172.21.15.118:3300/0 conn(0x7f32100acca0 msgr2=0x7f32100b2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:42.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.384+0000 7f320f7fe700 1 --2- 172.21.15.162:0/1242933211 >> v2:172.21.15.118:3300/0 conn(0x7f32100acca0 0x7f32100b2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:42.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.384+0000 7f320f7fe700 1 -- 172.21.15.162:0/1242933211 >> v1:172.21.15.118:6789/0 conn(0x7f32100ac3b0 legacy=0x7f32100ab780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:42.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.384+0000 7f320f7fe700 1 -- 172.21.15.162:0/1242933211 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32100b0d20 con 0x7f32100ac790 2020-11-28T21:54:42.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.384+0000 7f320f7fe700 1 --2- 172.21.15.162:0/1242933211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32100ac790 0x7f3210142bb0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 rx=0x7f31f8002f30 tx=0x7f31f8006e80).ready entity=mon.1 client_cookie=5d9beea2da31cabc server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:42.393 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.384+0000 7f320effd700 1 -- 172.21.15.162:0/1242933211 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31f80097f0 con 0x7f32100ac790 2020-11-28T21:54:42.393 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.384+0000 7f320effd700 1 -- 172.21.15.162:0/1242933211 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f8009950 con 0x7f32100ac790 2020-11-28T21:54:42.394 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.384+0000 7f320effd700 1 -- 172.21.15.162:0/1242933211 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31f8009cb0 con 0x7f32100ac790 2020-11-28T21:54:42.394 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.384+0000 7f321515f700 1 -- 172.21.15.162:0/1242933211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32100ac790 msgr2=0x7f3210142bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:42.394 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.384+0000 7f321515f700 1 --2- 172.21.15.162:0/1242933211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32100ac790 0x7f3210142bb0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 rx=0x7f31f8002f30 tx=0x7f31f8006e80).stop 2020-11-28T21:54:42.394 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.388+0000 7f321515f700 1 -- 172.21.15.162:0/1242933211 shutdown_connections 2020-11-28T21:54:42.395 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.388+0000 7f321515f700 1 --2- 172.21.15.162:0/1242933211 >> v2:172.21.15.118:3300/0 conn(0x7f32100acca0 0x7f32100b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:42.395 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.388+0000 7f321515f700 1 --2- 172.21.15.162:0/1242933211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32100ac790 0x7f3210142bb0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:42.395 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.388+0000 7f321515f700 1 -- 172.21.15.162:0/1242933211 >> 172.21.15.162:0/1242933211 conn(0x7f32100b6870 msgr2=0x7f32100b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:42.395 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.388+0000 7f321515f700 1 -- 172.21.15.162:0/1242933211 shutdown_connections 2020-11-28T21:54:42.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.388+0000 7f321515f700 1 -- 172.21.15.162:0/1242933211 wait complete. 2020-11-28T21:54:42.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.388+0000 7f321515f700 1 Processor -- start 2020-11-28T21:54:42.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.388+0000 7f321515f700 1 -- start start 2020-11-28T21:54:42.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.388+0000 7f321515f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f32100ac3b0 0x7f3210142fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:42.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.388+0000 7f321515f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32100ac790 0x7f321014eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:42.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.388+0000 7f321515f700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32100acca0 0x7f321014eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:42.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.388+0000 7f321515f700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f321008f910 con 0x7f32100acca0 2020-11-28T21:54:42.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.388+0000 7f321515f700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f321008fbb0 con 0x7f32100ac790 2020-11-28T21:54:42.398 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.388+0000 7f321515f700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f321008fe50 con 0x7f32100ac3b0 2020-11-28T21:54:42.398 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.388+0000 7f320f7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32100ac790 0x7f321014eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:42.398 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.388+0000 7f321495e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32100acca0 0x7f321014eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:42.398 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.388+0000 7f320f7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32100ac790 0x7f321014eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43818/0 (socket says 172.21.15.162:43818) 2020-11-28T21:54:42.398 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.388+0000 7f320f7fe700 1 -- 172.21.15.162:0/1688110506 learned_addr learned my addr 172.21.15.162:0/1688110506 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:42.399 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.388+0000 7f320ffff700 1 --2- 172.21.15.162:0/1688110506 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f32100ac3b0 0x7f3210142fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:42.399 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.388+0000 7f320f7fe700 1 -- 172.21.15.162:0/1688110506 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f32100ac3b0 msgr2=0x7f3210142fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:42.399 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.388+0000 7f320f7fe700 1 --2- 172.21.15.162:0/1688110506 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f32100ac3b0 0x7f3210142fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:42.399 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.388+0000 7f320f7fe700 1 -- 172.21.15.162:0/1688110506 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32100acca0 msgr2=0x7f321014eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:42.399 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.388+0000 7f320f7fe700 1 --2- 172.21.15.162:0/1688110506 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32100acca0 0x7f321014eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:42.400 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.388+0000 7f320f7fe700 1 -- 172.21.15.162:0/1688110506 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f321014e660 con 0x7f32100ac790 2020-11-28T21:54:42.400 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.388+0000 7f320f7fe700 1 --2- 172.21.15.162:0/1688110506 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32100ac790 0x7f321014eb00 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 rx=0x7f31f8002f30 tx=0x7f31f8015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:42.401 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.392+0000 7f320d7fa700 1 -- 172.21.15.162:0/1688110506 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31f8011c10 con 0x7f32100ac790 2020-11-28T21:54:42.401 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.392+0000 7f320d7fa700 1 -- 172.21.15.162:0/1688110506 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f8011d70 con 0x7f32100ac790 2020-11-28T21:54:42.402 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.392+0000 7f321515f700 1 -- 172.21.15.162:0/1688110506 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3210093ac0 con 0x7f32100ac790 2020-11-28T21:54:42.402 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.392+0000 7f320d7fa700 1 -- 172.21.15.162:0/1688110506 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31f801e650 con 0x7f32100ac790 2020-11-28T21:54:42.402 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.392+0000 7f321515f700 1 -- 172.21.15.162:0/1688110506 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32100a9710 con 0x7f32100ac790 2020-11-28T21:54:42.402 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.392+0000 7f321515f700 1 -- 172.21.15.162:0/1688110506 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32100a99f0 con 0x7f32100ac790 2020-11-28T21:54:42.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.392+0000 7f320d7fa700 1 -- 172.21.15.162:0/1688110506 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f31f801a040 con 0x7f32100ac790 2020-11-28T21:54:42.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.392+0000 7f320d7fa700 1 --2- 172.21.15.162:0/1688110506 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f31f4055440 0x7f31f4057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:42.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.392+0000 7f320d7fa700 1 -- 172.21.15.162:0/1688110506 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f31f806eed0 con 0x7f32100ac790 2020-11-28T21:54:42.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.392+0000 7f320ffff700 1 --2- 172.21.15.162:0/1688110506 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f31f4055440 0x7f31f4057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:42.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.396+0000 7f320ffff700 1 --2- 172.21.15.162:0/1688110506 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f31f4055440 0x7f31f4057900 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7f320000d000 tx=0x7f3200007b60).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:42.404 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.396+0000 7f320d7fa700 1 -- 172.21.15.162:0/1688110506 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f31f80451d0 con 0x7f32100ac790 2020-11-28T21:54:42.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:42.580+0000 7f321515f700 1 -- 172.21.15.162:0/1688110506 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f321014e7f0 con 0x7f32100ac790 2020-11-28T21:54:43.168 INFO:tasks.ceph.mon.b.smithi118.stderr:2020-11-28T21:54:43.165+0000 7ff4e1520700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-11-28T21:54:43.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.172+0000 7f320d7fa700 1 -- 172.21.15.162:0/1688110506 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v52) v1 ==== 161+0+0 (secure 0 0 0) 0x7f31f8023090 con 0x7f32100ac790 2020-11-28T21:54:43.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.176+0000 7f321515f700 1 -- 172.21.15.162:0/1688110506 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f31f4055440 msgr2=0x7f31f4057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:43.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.176+0000 7f321515f700 1 --2- 172.21.15.162:0/1688110506 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f31f4055440 0x7f31f4057900 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7f320000d000 tx=0x7f3200007b60).stop 2020-11-28T21:54:43.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.176+0000 7f321515f700 1 -- 172.21.15.162:0/1688110506 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32100ac790 msgr2=0x7f321014eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:43.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.176+0000 7f321515f700 1 --2- 172.21.15.162:0/1688110506 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32100ac790 0x7f321014eb00 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 rx=0x7f31f8002f30 tx=0x7f31f8015000).stop 2020-11-28T21:54:43.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.176+0000 7f321515f700 1 -- 172.21.15.162:0/1688110506 shutdown_connections 2020-11-28T21:54:43.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.176+0000 7f321515f700 1 --2- 172.21.15.162:0/1688110506 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f31f4055440 0x7f31f4057900 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:43.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.176+0000 7f321515f700 1 --2- 172.21.15.162:0/1688110506 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f32100ac3b0 0x7f3210142fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:43.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.176+0000 7f321515f700 1 --2- 172.21.15.162:0/1688110506 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32100acca0 0x7f321014eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:43.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.176+0000 7f321515f700 1 --2- 172.21.15.162:0/1688110506 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32100ac790 0x7f321014eb00 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:43.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.176+0000 7f321515f700 1 -- 172.21.15.162:0/1688110506 >> 172.21.15.162:0/1688110506 conn(0x7f32100b6870 msgr2=0x7f321005fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:43.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.176+0000 7f321515f700 1 -- 172.21.15.162:0/1688110506 shutdown_connections 2020-11-28T21:54:43.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.176+0000 7f321515f700 1 -- 172.21.15.162:0/1688110506 wait complete. 2020-11-28T21:54:43.183 INFO:teuthology.orchestra.run.smithi162.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-11-28T21:54:43.197 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:54:43.203 INFO:tasks.ceph.mds.a.smithi162.stdout:starting mds.a at 2020-11-28T21:54:43.306 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d07dd8700 1 Processor -- start 2020-11-28T21:54:43.306 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d07dd8700 1 -- start start 2020-11-28T21:54:43.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d07dd8700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f0d000b1d70 0x7f0d00140fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:43.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d07dd8700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d000b2870 con 0x7f0d000b11b0 2020-11-28T21:54:43.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d07dd8700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d000b22e0 con 0x7f0d000b1990 2020-11-28T21:54:43.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d07dd8700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d00146e20 con 0x7f0d000b1d70 2020-11-28T21:54:43.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d05b74700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f0d000b1990 0x7f0d00140890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:50684/0 (socket says 172.21.15.162:50684) 2020-11-28T21:54:43.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d05b74700 1 -- 172.21.15.162:0/1864031039 learned_addr learned my addr 172.21.15.162:0/1864031039 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:43.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d06375700 1 --2- 172.21.15.162:0/1864031039 >> v2:172.21.15.118:3300/0 conn(0x7f0d000b1d70 0x7f0d00140fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:43.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d04b72700 1 -- 172.21.15.162:0/1864031039 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 968854917 0 0) 0x7f0d000b22e0 con 0x7f0d000b1990 2020-11-28T21:54:43.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d04b72700 1 -- 172.21.15.162:0/1864031039 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ce4003230 con 0x7f0d000b1990 2020-11-28T21:54:43.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d04b72700 1 -- 172.21.15.162:0/1864031039 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3540896851 0 0) 0x7f0ce4003230 con 0x7f0d000b1990 2020-11-28T21:54:43.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d04b72700 1 -- 172.21.15.162:0/1864031039 >> v2:172.21.15.118:3300/0 conn(0x7f0d000b1d70 msgr2=0x7f0d00140fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:43.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d04b72700 1 --2- 172.21.15.162:0/1864031039 >> v2:172.21.15.118:3300/0 conn(0x7f0d000b1d70 0x7f0d00140fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:43.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d04b72700 1 -- 172.21.15.162:0/1864031039 >> v1:172.21.15.118:6789/0 conn(0x7f0d000b11b0 legacy=0x7f0d0014a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:43.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d04b72700 1 -- 172.21.15.162:0/1864031039 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d000b0740 con 0x7f0d000b1990 2020-11-28T21:54:43.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d04b72700 1 -- 172.21.15.162:0/1864031039 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f0cf40016d0 con 0x7f0d000b1990 2020-11-28T21:54:43.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d04b72700 1 -- 172.21.15.162:0/1864031039 >> v1:172.21.15.162:6789/0 conn(0x7f0d000b1990 legacy=0x7f0d00140890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:43.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d04b72700 1 --2- 172.21.15.162:0/1864031039 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0ce4003720 0x7f0ce4003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:43.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d04b72700 1 --2- 172.21.15.162:0/1864031039 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ce40044d0 0x7f0ce4004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:43.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d04b72700 1 --2- 172.21.15.162:0/1864031039 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0ce4004e40 0x7f0ce400b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:43.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d04b72700 1 -- 172.21.15.162:0/1864031039 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce4004250 con 0x7f0ce4004e40 2020-11-28T21:54:43.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d04b72700 1 -- 172.21.15.162:0/1864031039 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce400bc10 con 0x7f0ce40044d0 2020-11-28T21:54:43.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d04b72700 1 -- 172.21.15.162:0/1864031039 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce400bd50 con 0x7f0ce4003720 2020-11-28T21:54:43.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d05373700 1 --2- 172.21.15.162:0/1864031039 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ce40044d0 0x7f0ce4004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:43.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d05b74700 1 --2- 172.21.15.162:0/1864031039 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0ce4003720 0x7f0ce4003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:43.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d06375700 1 --2- 172.21.15.162:0/1864031039 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0ce4004e40 0x7f0ce400b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:43.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d05373700 1 -- 172.21.15.162:0/1864031039 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0ce4003720 msgr2=0x7f0ce4003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:43.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d05373700 1 --2- 172.21.15.162:0/1864031039 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0ce4003720 0x7f0ce4003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:43.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d05373700 1 -- 172.21.15.162:0/1864031039 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0ce4004e40 msgr2=0x7f0ce400b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:43.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d05373700 1 --2- 172.21.15.162:0/1864031039 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0ce4004e40 0x7f0ce400b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:43.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d05373700 1 -- 172.21.15.162:0/1864031039 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce4003230 con 0x7f0ce40044d0 2020-11-28T21:54:43.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d06375700 1 --2- 172.21.15.162:0/1864031039 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0ce4004e40 0x7f0ce400b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:54:43.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d06375700 1 -- 172.21.15.162:0/1864031039 reap_dead start 2020-11-28T21:54:43.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d05373700 1 --2- 172.21.15.162:0/1864031039 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ce40044d0 0x7f0ce4004950 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 rx=0x7f0d0014a650 tx=0x7f0cfc007b60).ready entity=mon.1 client_cookie=2c47cb6b89d9804b server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:43.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d04b72700 1 -- 172.21.15.162:0/1864031039 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cfc00c870 con 0x7f0ce40044d0 2020-11-28T21:54:43.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d04b72700 1 -- 172.21.15.162:0/1864031039 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cfc00c9d0 con 0x7f0ce40044d0 2020-11-28T21:54:43.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d04b72700 1 -- 172.21.15.162:0/1864031039 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cfc00cd30 con 0x7f0ce40044d0 2020-11-28T21:54:43.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d07dd8700 1 -- 172.21.15.162:0/1864031039 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ce40044d0 msgr2=0x7f0ce4004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:43.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d07dd8700 1 --2- 172.21.15.162:0/1864031039 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ce40044d0 0x7f0ce4004950 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 rx=0x7f0d0014a650 tx=0x7f0cfc007b60).stop 2020-11-28T21:54:43.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d07dd8700 1 -- 172.21.15.162:0/1864031039 shutdown_connections 2020-11-28T21:54:43.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d07dd8700 1 --2- 172.21.15.162:0/1864031039 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ce40044d0 0x7f0ce4004950 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:43.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.304+0000 7f0d07dd8700 1 -- 172.21.15.162:0/1864031039 >> 172.21.15.162:0/1864031039 conn(0x7f0d0005b970 msgr2=0x7f0d0005bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:43.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.308+0000 7f0d07dd8700 1 -- 172.21.15.162:0/1864031039 shutdown_connections 2020-11-28T21:54:43.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.308+0000 7f0d07dd8700 1 -- 172.21.15.162:0/1864031039 wait complete. 2020-11-28T21:54:43.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.308+0000 7f0d07dd8700 1 Processor -- start 2020-11-28T21:54:43.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.308+0000 7f0d07dd8700 1 -- start start 2020-11-28T21:54:43.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.308+0000 7f0d07dd8700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0ce40044d0 0x7f0d000aa8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:43.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.308+0000 7f0d07dd8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d00155d20 0x7f0d00157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:43.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.308+0000 7f0d07dd8700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0d000ab4d0 0x7f0d00158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:43.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.308+0000 7f0d07dd8700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d00157170 con 0x7f0d000ab4d0 2020-11-28T21:54:43.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.308+0000 7f0d07dd8700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d001573a0 con 0x7f0d00155d20 2020-11-28T21:54:43.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.308+0000 7f0d07dd8700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d00157640 con 0x7f0ce40044d0 2020-11-28T21:54:43.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.308+0000 7f0d05373700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d00155d20 0x7f0d00157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:43.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.308+0000 7f0d05373700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d00155d20 0x7f0d00157d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43854/0 (socket says 172.21.15.162:43854) 2020-11-28T21:54:43.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.308+0000 7f0d05373700 1 -- 172.21.15.162:0/2730537515 learned_addr learned my addr 172.21.15.162:0/2730537515 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:43.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.308+0000 7f0d05b74700 1 --2- 172.21.15.162:0/2730537515 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0ce40044d0 0x7f0d000aa8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:43.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.308+0000 7f0d06375700 1 --2- 172.21.15.162:0/2730537515 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0d000ab4d0 0x7f0d00158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:43.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.308+0000 7f0d05373700 1 -- 172.21.15.162:0/2730537515 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0ce40044d0 msgr2=0x7f0d000aa8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:43.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.308+0000 7f0d05373700 1 --2- 172.21.15.162:0/2730537515 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0ce40044d0 0x7f0d000aa8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:43.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.308+0000 7f0d05373700 1 -- 172.21.15.162:0/2730537515 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0d000ab4d0 msgr2=0x7f0d00158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:43.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.308+0000 7f0d05373700 1 --2- 172.21.15.162:0/2730537515 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0d000ab4d0 0x7f0d00158270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:43.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.308+0000 7f0d05373700 1 -- 172.21.15.162:0/2730537515 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d001578e0 con 0x7f0d00155d20 2020-11-28T21:54:43.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.308+0000 7f0d05b74700 1 --2- 172.21.15.162:0/2730537515 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0ce40044d0 0x7f0d000aa8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:54:43.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.308+0000 7f0d06375700 1 --2- 172.21.15.162:0/2730537515 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0d000ab4d0 0x7f0d00158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:54:43.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.308+0000 7f0d05373700 1 --2- 172.21.15.162:0/2730537515 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d00155d20 0x7f0d00157d80 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 rx=0x7f0d00145880 tx=0x7f0cfc016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:43.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.308+0000 7f0cf2ffd700 1 -- 172.21.15.162:0/2730537515 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cfc001640 con 0x7f0d00155d20 2020-11-28T21:54:43.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.308+0000 7f0cf2ffd700 1 -- 172.21.15.162:0/2730537515 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cfc007030 con 0x7f0d00155d20 2020-11-28T21:54:43.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.308+0000 7f0d07dd8700 1 -- 172.21.15.162:0/2730537515 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d00149d60 con 0x7f0d00155d20 2020-11-28T21:54:43.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.308+0000 7f0cf2ffd700 1 -- 172.21.15.162:0/2730537515 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cfc007390 con 0x7f0d00155d20 2020-11-28T21:54:43.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.308+0000 7f0d07dd8700 1 -- 172.21.15.162:0/2730537515 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d00158900 con 0x7f0d00155d20 2020-11-28T21:54:43.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.312+0000 7f0cf2ffd700 1 -- 172.21.15.162:0/2730537515 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0cfc023020 con 0x7f0d00155d20 2020-11-28T21:54:43.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.312+0000 7f0cf2ffd700 1 --2- 172.21.15.162:0/2730537515 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0ce80554e0 0x7f0ce80579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:43.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.312+0000 7f0cf2ffd700 1 -- 172.21.15.162:0/2730537515 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f0cfc06ef90 con 0x7f0d00155d20 2020-11-28T21:54:43.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.312+0000 7f0d05b74700 1 --2- 172.21.15.162:0/2730537515 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0ce80554e0 0x7f0ce80579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:43.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.312+0000 7f0d07dd8700 1 -- 172.21.15.162:0/2730537515 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d00157a70 con 0x7f0d00155d20 2020-11-28T21:54:43.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.312+0000 7f0d05b74700 1 --2- 172.21.15.162:0/2730537515 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0ce80554e0 0x7f0ce80579a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f0cf4005000 tx=0x7f0cf4009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:43.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.320+0000 7f0cf2ffd700 1 -- 172.21.15.162:0/2730537515 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0cfc06adf0 con 0x7f0d00155d20 2020-11-28T21:54:43.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.504+0000 7f0d07dd8700 1 -- 172.21.15.162:0/2730537515 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d00157a70 con 0x7f0d00155d20 2020-11-28T21:54:43.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.504+0000 7f0cf2ffd700 1 -- 172.21.15.162:0/2730537515 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+2139 (secure 0 0 0) 0x7f0cfc045240 con 0x7f0d00155d20 2020-11-28T21:54:43.508 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:54:43.508 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":52,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6182,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2775691244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2775691244},{"type":"v1","addr":"172.21.15.162:6835","nonce":2775691244}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":49},{"gid":6942,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/1896314020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1896314020},{"type":"v1","addr":"172.21.15.118:6833","nonce":1896314020}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":49}],"filesystems":[{"mdsmap":{"epoch":52,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:54:26.908249+0000","modified":"2020-11-28T21:54:43.168533+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2020-11-28T21:54:43.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.504+0000 7f0d07dd8700 1 -- 172.21.15.162:0/2730537515 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0ce80554e0 msgr2=0x7f0ce80579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:43.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.504+0000 7f0d07dd8700 1 --2- 172.21.15.162:0/2730537515 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0ce80554e0 0x7f0ce80579a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f0cf4005000 tx=0x7f0cf4009040).stop 2020-11-28T21:54:43.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.504+0000 7f0d07dd8700 1 -- 172.21.15.162:0/2730537515 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d00155d20 msgr2=0x7f0d00157d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:43.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.508+0000 7f0d07dd8700 1 --2- 172.21.15.162:0/2730537515 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d00155d20 0x7f0d00157d80 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 rx=0x7f0d00145880 tx=0x7f0cfc016000).stop 2020-11-28T21:54:43.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.508+0000 7f0d07dd8700 1 -- 172.21.15.162:0/2730537515 shutdown_connections 2020-11-28T21:54:43.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.508+0000 7f0d07dd8700 1 --2- 172.21.15.162:0/2730537515 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0ce80554e0 0x7f0ce80579a0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:43.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.508+0000 7f0d07dd8700 1 --2- 172.21.15.162:0/2730537515 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0ce40044d0 0x7f0d000aa8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:43.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.508+0000 7f0d07dd8700 1 --2- 172.21.15.162:0/2730537515 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0d000ab4d0 0x7f0d00158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:43.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.508+0000 7f0d07dd8700 1 --2- 172.21.15.162:0/2730537515 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d00155d20 0x7f0d00157d80 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:43.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.508+0000 7f0d07dd8700 1 -- 172.21.15.162:0/2730537515 >> 172.21.15.162:0/2730537515 conn(0x7f0d0005b970 msgr2=0x7f0d0013fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:43.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.508+0000 7f0d07dd8700 1 -- 172.21.15.162:0/2730537515 shutdown_connections 2020-11-28T21:54:43.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.508+0000 7f0d07dd8700 1 -- 172.21.15.162:0/2730537515 wait complete. 2020-11-28T21:54:43.513 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 52 2020-11-28T21:54:43.522 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2020-11-28T21:54:43.684 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.680+0000 7ff0ac818700 1 Processor -- start 2020-11-28T21:54:43.684 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.680+0000 7ff0ac818700 1 -- start start 2020-11-28T21:54:43.688 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.680+0000 7ff0ac818700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff0a8146330 0x7ff0a80b1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:43.689 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.680+0000 7ff0ac818700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff0a806aa70 0x7ff0a805b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:43.689 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.680+0000 7ff0ac818700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0a8146b00 con 0x7ff0a80b1bf0 2020-11-28T21:54:43.690 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.680+0000 7ff0ac818700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0a8146da0 con 0x7ff0a806aa70 2020-11-28T21:54:43.690 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.680+0000 7ff0ac818700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0a80b6610 con 0x7ff0a8146330 2020-11-28T21:54:43.690 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.684+0000 7ff0a659c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff0a8146330 0x7ff0a80b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:43.691 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.684+0000 7ff0a659c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff0a8146330 0x7ff0a80b1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43860/0 (socket says 172.21.15.162:43860) 2020-11-28T21:54:43.691 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.684+0000 7ff0a659c700 1 -- 172.21.15.162:0/315729620 learned_addr learned my addr 172.21.15.162:0/315729620 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:43.692 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.684+0000 7ff0a5d9b700 1 --2- 172.21.15.162:0/315729620 >> v2:172.21.15.118:3300/0 conn(0x7ff0a806aa70 0x7ff0a805b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:43.692 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.684+0000 7ff0a659c700 1 -- 172.21.15.162:0/315729620 >> v2:172.21.15.118:3300/0 conn(0x7ff0a806aa70 msgr2=0x7ff0a805b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:43.692 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.684+0000 7ff0a659c700 1 --2- 172.21.15.162:0/315729620 >> v2:172.21.15.118:3300/0 conn(0x7ff0a806aa70 0x7ff0a805b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:43.693 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.684+0000 7ff0a659c700 1 -- 172.21.15.162:0/315729620 >> v1:172.21.15.118:6789/0 conn(0x7ff0a80b1bf0 legacy=0x7ff0a805c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:43.693 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.684+0000 7ff0a659c700 1 -- 172.21.15.162:0/315729620 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a80b0f80 con 0x7ff0a8146330 2020-11-28T21:54:43.694 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.684+0000 7ff0a659c700 1 --2- 172.21.15.162:0/315729620 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0a8146330 0x7ff0a80b1470 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 rx=0x7ff090002f30 tx=0x7ff090006e80).ready entity=mon.1 client_cookie=91f7ca0bfe06ecbd server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:43.694 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.684+0000 7ff0a559a700 1 -- 172.21.15.162:0/315729620 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0900097f0 con 0x7ff0a8146330 2020-11-28T21:54:43.695 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.684+0000 7ff0a559a700 1 -- 172.21.15.162:0/315729620 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff090009950 con 0x7ff0a8146330 2020-11-28T21:54:43.695 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.684+0000 7ff0a559a700 1 -- 172.21.15.162:0/315729620 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff090009cb0 con 0x7ff0a8146330 2020-11-28T21:54:43.695 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.684+0000 7ff0ac818700 1 -- 172.21.15.162:0/315729620 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0a8146330 msgr2=0x7ff0a80b1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:43.696 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.684+0000 7ff0ac818700 1 --2- 172.21.15.162:0/315729620 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0a8146330 0x7ff0a80b1470 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 rx=0x7ff090002f30 tx=0x7ff090006e80).stop 2020-11-28T21:54:43.696 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.684+0000 7ff0ac818700 1 -- 172.21.15.162:0/315729620 shutdown_connections 2020-11-28T21:54:43.696 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.684+0000 7ff0ac818700 1 --2- 172.21.15.162:0/315729620 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0a8146330 0x7ff0a80b1470 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:43.697 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.684+0000 7ff0ac818700 1 --2- 172.21.15.162:0/315729620 >> v2:172.21.15.118:3300/0 conn(0x7ff0a806aa70 0x7ff0a805b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:43.697 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.684+0000 7ff0ac818700 1 -- 172.21.15.162:0/315729620 >> 172.21.15.162:0/315729620 conn(0x7ff0a80b56f0 msgr2=0x7ff0a80b5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:43.697 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.684+0000 7ff0ac818700 1 -- 172.21.15.162:0/315729620 shutdown_connections 2020-11-28T21:54:43.698 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.684+0000 7ff0ac818700 1 -- 172.21.15.162:0/315729620 wait complete. 2020-11-28T21:54:43.703 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.688+0000 7ff0ac818700 1 Processor -- start 2020-11-28T21:54:43.703 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.688+0000 7ff0ac818700 1 -- start start 2020-11-28T21:54:43.703 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.688+0000 7ff0ac818700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0a806aa70 0x7ff0a80abf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:43.704 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.688+0000 7ff0ac818700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff0a80b1bf0 0x7ff0a80a87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:43.704 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.688+0000 7ff0ac818700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff0a8146330 0x7ff0a80a8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:43.704 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.688+0000 7ff0ac818700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a8151920 con 0x7ff0a80b1bf0 2020-11-28T21:54:43.705 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.688+0000 7ff0ac818700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a8151bc0 con 0x7ff0a806aa70 2020-11-28T21:54:43.705 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.688+0000 7ff0ac818700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a8151e60 con 0x7ff0a8146330 2020-11-28T21:54:43.705 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.688+0000 7ff0a659c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0a806aa70 0x7ff0a80abf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:43.706 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.688+0000 7ff0a659c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0a806aa70 0x7ff0a80abf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43866/0 (socket says 172.21.15.162:43866) 2020-11-28T21:54:43.706 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.688+0000 7ff0a659c700 1 -- 172.21.15.162:0/1335066609 learned_addr learned my addr 172.21.15.162:0/1335066609 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:43.707 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.688+0000 7ff0a5d9b700 1 --2- 172.21.15.162:0/1335066609 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff0a80b1bf0 0x7ff0a80a87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:43.707 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.688+0000 7ff0a6d9d700 1 --2- 172.21.15.162:0/1335066609 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff0a8146330 0x7ff0a80a8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:43.707 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.688+0000 7ff0a659c700 1 -- 172.21.15.162:0/1335066609 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff0a8146330 msgr2=0x7ff0a80a8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:43.708 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.688+0000 7ff0a659c700 1 --2- 172.21.15.162:0/1335066609 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff0a8146330 0x7ff0a80a8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:43.708 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.688+0000 7ff0a659c700 1 -- 172.21.15.162:0/1335066609 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff0a80b1bf0 msgr2=0x7ff0a80a87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:43.708 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.688+0000 7ff0a659c700 1 --2- 172.21.15.162:0/1335066609 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff0a80b1bf0 0x7ff0a80a87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:43.709 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.688+0000 7ff0a659c700 1 -- 172.21.15.162:0/1335066609 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a8152100 con 0x7ff0a806aa70 2020-11-28T21:54:43.709 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.688+0000 7ff0a5d9b700 1 --2- 172.21.15.162:0/1335066609 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff0a80b1bf0 0x7ff0a80a87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:54:43.709 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.688+0000 7ff0a6d9d700 1 --2- 172.21.15.162:0/1335066609 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff0a8146330 0x7ff0a80a8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:54:43.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.688+0000 7ff0a659c700 1 --2- 172.21.15.162:0/1335066609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0a806aa70 0x7ff0a80abf30 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 rx=0x7ff090002f30 tx=0x7ff090015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:43.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.688+0000 7ff09f7fe700 1 -- 172.21.15.162:0/1335066609 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff090011ca0 con 0x7ff0a806aa70 2020-11-28T21:54:43.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.688+0000 7ff09f7fe700 1 -- 172.21.15.162:0/1335066609 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff090011e00 con 0x7ff0a806aa70 2020-11-28T21:54:43.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.688+0000 7ff0ac818700 1 -- 172.21.15.162:0/1335066609 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0a81523a0 con 0x7ff0a806aa70 2020-11-28T21:54:43.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.688+0000 7ff09f7fe700 1 -- 172.21.15.162:0/1335066609 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff09001e650 con 0x7ff0a806aa70 2020-11-28T21:54:43.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.688+0000 7ff0ac818700 1 -- 172.21.15.162:0/1335066609 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0a80ab030 con 0x7ff0a806aa70 2020-11-28T21:54:43.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.692+0000 7ff0ac818700 1 -- 172.21.15.162:0/1335066609 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0a8152530 con 0x7ff0a806aa70 2020-11-28T21:54:43.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.692+0000 7ff09f7fe700 1 -- 172.21.15.162:0/1335066609 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff090025030 con 0x7ff0a806aa70 2020-11-28T21:54:43.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.692+0000 7ff09f7fe700 1 --2- 172.21.15.162:0/1335066609 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff08c0553f0 0x7ff08c0578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:43.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.692+0000 7ff09f7fe700 1 -- 172.21.15.162:0/1335066609 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7ff09006f340 con 0x7ff0a806aa70 2020-11-28T21:54:43.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.692+0000 7ff0a5d9b700 1 --2- 172.21.15.162:0/1335066609 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff08c0553f0 0x7ff08c0578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:43.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.692+0000 7ff0a5d9b700 1 --2- 172.21.15.162:0/1335066609 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff08c0553f0 0x7ff08c0578b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7ff09800b000 tx=0x7ff098008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:43.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.696+0000 7ff09f7fe700 1 -- 172.21.15.162:0/1335066609 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff0900456b0 con 0x7ff0a806aa70 2020-11-28T21:54:43.886 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:43.884+0000 7ff0ac818700 1 -- 172.21.15.162:0/1335066609 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff0a8057050 con 0x7ff0a806aa70 2020-11-28T21:54:44.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:44.864+0000 7ff09f7fe700 1 -- 172.21.15.162:0/1335066609 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v53) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff090048c10 con 0x7ff0a806aa70 2020-11-28T21:54:44.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:44.868+0000 7ff0ac818700 1 -- 172.21.15.162:0/1335066609 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff08c0553f0 msgr2=0x7ff08c0578b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:44.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:44.868+0000 7ff0ac818700 1 --2- 172.21.15.162:0/1335066609 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff08c0553f0 0x7ff08c0578b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7ff09800b000 tx=0x7ff098008040).stop 2020-11-28T21:54:44.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:44.868+0000 7ff0ac818700 1 -- 172.21.15.162:0/1335066609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0a806aa70 msgr2=0x7ff0a80abf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:44.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:44.868+0000 7ff0ac818700 1 --2- 172.21.15.162:0/1335066609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0a806aa70 0x7ff0a80abf30 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 rx=0x7ff090002f30 tx=0x7ff090015000).stop 2020-11-28T21:54:44.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:44.868+0000 7ff0ac818700 1 -- 172.21.15.162:0/1335066609 shutdown_connections 2020-11-28T21:54:44.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:44.868+0000 7ff0ac818700 1 --2- 172.21.15.162:0/1335066609 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff08c0553f0 0x7ff08c0578b0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:44.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:44.868+0000 7ff0ac818700 1 --2- 172.21.15.162:0/1335066609 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff0a8146330 0x7ff0a80a8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:44.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:44.868+0000 7ff0ac818700 1 --2- 172.21.15.162:0/1335066609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0a806aa70 0x7ff0a80abf30 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:44.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:44.868+0000 7ff0ac818700 1 --2- 172.21.15.162:0/1335066609 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff0a80b1bf0 0x7ff0a80a87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:44.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:44.868+0000 7ff0ac818700 1 -- 172.21.15.162:0/1335066609 >> 172.21.15.162:0/1335066609 conn(0x7ff0a80b56f0 msgr2=0x7ff0a8144b00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:44.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:44.868+0000 7ff0ac818700 1 -- 172.21.15.162:0/1335066609 shutdown_connections 2020-11-28T21:54:44.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:44.872+0000 7ff0ac818700 1 -- 172.21.15.162:0/1335066609 wait complete. 2020-11-28T21:54:44.889 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-11-28T21:54:45.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.012+0000 7f4cfaf2f700 1 Processor -- start 2020-11-28T21:54:45.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.012+0000 7f4cfaf2f700 1 -- start start 2020-11-28T21:54:45.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.016+0000 7f4cfaf2f700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4cf405ccc0 0x7f4cf4097ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:45.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.016+0000 7f4cfaf2f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4cf409a990 0x7f4cf4060620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:45.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.016+0000 7f4cfaf2f700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cf40ad400 con 0x7f4cf40a8f10 2020-11-28T21:54:45.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.016+0000 7f4cfaf2f700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cf4067bb0 con 0x7f4cf405ccc0 2020-11-28T21:54:45.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.016+0000 7f4cfaf2f700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf405d0a0 con 0x7f4cf409a990 2020-11-28T21:54:45.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.016+0000 7f4cf9f2d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4cf405ccc0 0x7f4cf4097ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:45.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.016+0000 7f4cf972c700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f4cf40a8f10 0x7f4cf405ff10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:50720/0 (socket says 172.21.15.162:50720) 2020-11-28T21:54:45.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.016+0000 7f4cf972c700 1 -- 172.21.15.162:0/1418837853 learned_addr learned my addr 172.21.15.162:0/1418837853 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:45.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.016+0000 7f4cfa72e700 1 --2- 172.21.15.162:0/1418837853 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4cf409a990 0x7f4cf4060620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:45.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.016+0000 7f4cf9f2d700 1 -- 172.21.15.162:0/1418837853 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4cf409a990 msgr2=0x7f4cf4060620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:45.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.016+0000 7f4cf9f2d700 1 --2- 172.21.15.162:0/1418837853 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4cf409a990 0x7f4cf4060620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:45.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.016+0000 7f4cf9f2d700 1 -- 172.21.15.162:0/1418837853 >> v1:172.21.15.162:6789/0 conn(0x7f4cf40a8f10 legacy=0x7f4cf405ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:45.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.016+0000 7f4cf8f2b700 1 -- 172.21.15.162:0/1418837853 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2060510140 0 0) 0x7f4cf40ad400 con 0x7f4cf40a8f10 2020-11-28T21:54:45.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.016+0000 7f4cf9f2d700 1 -- 172.21.15.162:0/1418837853 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf40b1260 con 0x7f4cf405ccc0 2020-11-28T21:54:45.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.016+0000 7f4cf9f2d700 1 --2- 172.21.15.162:0/1418837853 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4cf405ccc0 0x7f4cf4097ac0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 rx=0x7f4cf0002f30 tx=0x7f4cf0006dc0).ready entity=mon.1 client_cookie=1675cb71c5287da5 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:45.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.016+0000 7f4cf8f2b700 1 -- 172.21.15.162:0/1418837853 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4cf00097f0 con 0x7f4cf405ccc0 2020-11-28T21:54:45.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.016+0000 7f4cf8f2b700 1 -- 172.21.15.162:0/1418837853 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf0009950 con 0x7f4cf405ccc0 2020-11-28T21:54:45.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.016+0000 7f4cf8f2b700 1 -- 172.21.15.162:0/1418837853 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4cf0009cb0 con 0x7f4cf405ccc0 2020-11-28T21:54:45.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.016+0000 7f4cfaf2f700 1 -- 172.21.15.162:0/1418837853 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4cf405ccc0 msgr2=0x7f4cf4097ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:45.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.016+0000 7f4cfaf2f700 1 --2- 172.21.15.162:0/1418837853 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4cf405ccc0 0x7f4cf4097ac0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 rx=0x7f4cf0002f30 tx=0x7f4cf0006dc0).stop 2020-11-28T21:54:45.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.016+0000 7f4cfaf2f700 1 -- 172.21.15.162:0/1418837853 shutdown_connections 2020-11-28T21:54:45.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.016+0000 7f4cfaf2f700 1 --2- 172.21.15.162:0/1418837853 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4cf405ccc0 0x7f4cf4097ac0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:45.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.016+0000 7f4cfaf2f700 1 --2- 172.21.15.162:0/1418837853 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4cf409a990 0x7f4cf4060620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:45.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.016+0000 7f4cfaf2f700 1 -- 172.21.15.162:0/1418837853 >> 172.21.15.162:0/1418837853 conn(0x7f4cf40b1fe0 msgr2=0x7f4cf40b23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:45.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.016+0000 7f4cfaf2f700 1 -- 172.21.15.162:0/1418837853 shutdown_connections 2020-11-28T21:54:45.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.016+0000 7f4cfaf2f700 1 -- 172.21.15.162:0/1418837853 wait complete. 2020-11-28T21:54:45.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.020+0000 7f4cfaf2f700 1 Processor -- start 2020-11-28T21:54:45.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.020+0000 7f4cfaf2f700 1 -- start start 2020-11-28T21:54:45.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.020+0000 7f4cfaf2f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4cf405ccc0 0x7f4cf40ab230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:45.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.020+0000 7f4cfaf2f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4cf409a990 0x7f4cf4062d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:45.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.020+0000 7f4cfaf2f700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4cf40a8f10 0x7f4cf4147340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:45.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.020+0000 7f4cfaf2f700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf4067900 con 0x7f4cf40a8f10 2020-11-28T21:54:45.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.020+0000 7f4cfaf2f700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf40aad50 con 0x7f4cf409a990 2020-11-28T21:54:45.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.020+0000 7f4cfaf2f700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf4065300 con 0x7f4cf405ccc0 2020-11-28T21:54:45.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.020+0000 7f4cf972c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4cf409a990 0x7f4cf4062d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:45.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.020+0000 7f4cf9f2d700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4cf405ccc0 0x7f4cf40ab230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:45.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.020+0000 7f4cf972c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4cf409a990 0x7f4cf4062d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43884/0 (socket says 172.21.15.162:43884) 2020-11-28T21:54:45.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.020+0000 7f4cf972c700 1 -- 172.21.15.162:0/650620693 learned_addr learned my addr 172.21.15.162:0/650620693 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:45.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.020+0000 7f4cfa72e700 1 --2- 172.21.15.162:0/650620693 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4cf40a8f10 0x7f4cf4147340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:45.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.020+0000 7f4cf972c700 1 -- 172.21.15.162:0/650620693 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4cf405ccc0 msgr2=0x7f4cf40ab230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:45.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.020+0000 7f4cf972c700 1 --2- 172.21.15.162:0/650620693 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4cf405ccc0 0x7f4cf40ab230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:45.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.020+0000 7f4cf972c700 1 -- 172.21.15.162:0/650620693 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4cf40a8f10 msgr2=0x7f4cf4147340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:45.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.020+0000 7f4cf972c700 1 --2- 172.21.15.162:0/650620693 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4cf40a8f10 0x7f4cf4147340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:45.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.020+0000 7f4cf972c700 1 -- 172.21.15.162:0/650620693 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf40aa840 con 0x7f4cf409a990 2020-11-28T21:54:45.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.020+0000 7f4cf972c700 1 --2- 172.21.15.162:0/650620693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4cf409a990 0x7f4cf4062d20 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 rx=0x7f4ce4000e90 tx=0x7f4ce4007cb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:45.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.020+0000 7f4ce2ffd700 1 -- 172.21.15.162:0/650620693 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ce40072e0 con 0x7f4cf409a990 2020-11-28T21:54:45.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.020+0000 7f4cfaf2f700 1 -- 172.21.15.162:0/650620693 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cf40abf50 con 0x7f4cf409a990 2020-11-28T21:54:45.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.020+0000 7f4ce2ffd700 1 -- 172.21.15.162:0/650620693 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce4007440 con 0x7f4cf409a990 2020-11-28T21:54:45.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.020+0000 7f4ce2ffd700 1 -- 172.21.15.162:0/650620693 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ce40057d0 con 0x7f4cf409a990 2020-11-28T21:54:45.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.020+0000 7f4cfaf2f700 1 -- 172.21.15.162:0/650620693 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cf4144bd0 con 0x7f4cf409a990 2020-11-28T21:54:45.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.024+0000 7f4cfaf2f700 1 -- 172.21.15.162:0/650620693 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cf40aaf10 con 0x7f4cf409a990 2020-11-28T21:54:45.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.024+0000 7f4ce2ffd700 1 -- 172.21.15.162:0/650620693 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4ce4010070 con 0x7f4cf409a990 2020-11-28T21:54:45.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.024+0000 7f4ce2ffd700 1 --2- 172.21.15.162:0/650620693 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4cd40553b0 0x7f4cd4057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:45.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.024+0000 7f4ce2ffd700 1 -- 172.21.15.162:0/650620693 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f4ce406d700 con 0x7f4cf409a990 2020-11-28T21:54:45.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.024+0000 7f4cf9f2d700 1 --2- 172.21.15.162:0/650620693 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4cd40553b0 0x7f4cd4057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:45.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.024+0000 7f4cf9f2d700 1 --2- 172.21.15.162:0/650620693 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4cd40553b0 0x7f4cd4057870 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f4cf001b000 tx=0x7f4cf0015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:45.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.032+0000 7f4ce2ffd700 1 -- 172.21.15.162:0/650620693 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4ce4043820 con 0x7f4cf409a990 2020-11-28T21:54:45.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.172+0000 7f4cfaf2f700 1 -- 172.21.15.162:0/650620693 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4cf4057050 con 0x7f4cf409a990 2020-11-28T21:54:45.889 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.884+0000 7f4ce2ffd700 1 -- 172.21.15.162:0/650620693 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v111) v1 ==== 185+0+0 (secure 0 0 0) 0x7f4ce4046e40 con 0x7f4cf409a990 2020-11-28T21:54:45.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.888+0000 7f4cfaf2f700 1 -- 172.21.15.162:0/650620693 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4cd40553b0 msgr2=0x7f4cd4057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:45.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.888+0000 7f4cfaf2f700 1 --2- 172.21.15.162:0/650620693 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4cd40553b0 0x7f4cd4057870 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f4cf001b000 tx=0x7f4cf0015000).stop 2020-11-28T21:54:45.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.888+0000 7f4cfaf2f700 1 -- 172.21.15.162:0/650620693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4cf409a990 msgr2=0x7f4cf4062d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:45.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.888+0000 7f4cfaf2f700 1 --2- 172.21.15.162:0/650620693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4cf409a990 0x7f4cf4062d20 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 rx=0x7f4ce4000e90 tx=0x7f4ce4007cb0).stop 2020-11-28T21:54:45.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.888+0000 7f4cfaf2f700 1 -- 172.21.15.162:0/650620693 shutdown_connections 2020-11-28T21:54:45.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.888+0000 7f4cfaf2f700 1 --2- 172.21.15.162:0/650620693 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4cd40553b0 0x7f4cd4057870 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:45.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.888+0000 7f4cfaf2f700 1 --2- 172.21.15.162:0/650620693 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4cf405ccc0 0x7f4cf40ab230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:45.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.888+0000 7f4cfaf2f700 1 --2- 172.21.15.162:0/650620693 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4cf40a8f10 0x7f4cf4147340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:45.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.888+0000 7f4cfaf2f700 1 --2- 172.21.15.162:0/650620693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4cf409a990 0x7f4cf4062d20 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:45.896 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.888+0000 7f4cfaf2f700 1 -- 172.21.15.162:0/650620693 >> 172.21.15.162:0/650620693 conn(0x7f4cf40b1fe0 msgr2=0x7f4cf40b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:45.896 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.892+0000 7f4cfaf2f700 1 -- 172.21.15.162:0/650620693 shutdown_connections 2020-11-28T21:54:45.896 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:45.892+0000 7f4cfaf2f700 1 -- 172.21.15.162:0/650620693 wait complete. 2020-11-28T21:54:45.897 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_metadata' removed 2020-11-28T21:54:45.910 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-11-28T21:54:46.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.028+0000 7f95d4998700 1 Processor -- start 2020-11-28T21:54:46.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.028+0000 7f95d4998700 1 -- start start 2020-11-28T21:54:46.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.032+0000 7f95d4998700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f95d013dfc0 0x7f95d00b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:46.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.032+0000 7f95d4998700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f95d0140590 0x7f95d00b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:46.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.032+0000 7f95d4998700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95d00b1190 con 0x7f95d013dbe0 2020-11-28T21:54:46.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.032+0000 7f95d4998700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95d00b0040 con 0x7f95d013dfc0 2020-11-28T21:54:46.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.032+0000 7f95d4998700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95d00b1420 con 0x7f95d0140590 2020-11-28T21:54:46.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.032+0000 7f95cf59e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f95d0140590 0x7f95d00b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:46.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.032+0000 7f95ced9d700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f95d013dbe0 0x7f95d0060920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:50732/0 (socket says 172.21.15.162:50732) 2020-11-28T21:54:46.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.032+0000 7f95cf59e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f95d0140590 0x7f95d00b0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43892/0 (socket says 172.21.15.162:43892) 2020-11-28T21:54:46.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.032+0000 7f95ced9d700 1 -- 172.21.15.162:0/3169773246 learned_addr learned my addr 172.21.15.162:0/3169773246 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:46.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.032+0000 7f95ce59c700 1 --2- 172.21.15.162:0/3169773246 >> v2:172.21.15.118:3300/0 conn(0x7f95d013dfc0 0x7f95d00b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:46.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.032+0000 7f95cdd9b700 1 -- 172.21.15.162:0/3169773246 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3242628383 0 0) 0x7f95d00b1190 con 0x7f95d013dbe0 2020-11-28T21:54:46.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.032+0000 7f95cdd9b700 1 -- 172.21.15.162:0/3169773246 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95b8003230 con 0x7f95d013dbe0 2020-11-28T21:54:46.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.032+0000 7f95cf59e700 1 -- 172.21.15.162:0/3169773246 >> v2:172.21.15.118:3300/0 conn(0x7f95d013dfc0 msgr2=0x7f95d00b20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:46.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.032+0000 7f95cf59e700 1 --2- 172.21.15.162:0/3169773246 >> v2:172.21.15.118:3300/0 conn(0x7f95d013dfc0 0x7f95d00b20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:46.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.032+0000 7f95cdd9b700 1 -- 172.21.15.162:0/3169773246 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1129221096 0 0) 0x7f95b8003230 con 0x7f95d013dbe0 2020-11-28T21:54:46.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.032+0000 7f95cf59e700 1 -- 172.21.15.162:0/3169773246 >> v1:172.21.15.162:6789/0 conn(0x7f95d013dbe0 legacy=0x7f95d0060920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:46.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.032+0000 7f95cf59e700 1 -- 172.21.15.162:0/3169773246 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d00b16c0 con 0x7f95d0140590 2020-11-28T21:54:46.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.032+0000 7f95cf59e700 1 --2- 172.21.15.162:0/3169773246 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95d0140590 0x7f95d00b0ca0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 rx=0x7f95c0001870 tx=0x7f95c80091a0).ready entity=mon.1 client_cookie=461abf0e7380f433 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:46.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.032+0000 7f95cdd9b700 1 -- 172.21.15.162:0/3169773246 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95c00016d0 con 0x7f95d0140590 2020-11-28T21:54:46.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.032+0000 7f95cdd9b700 1 -- 172.21.15.162:0/3169773246 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c800b800 con 0x7f95d0140590 2020-11-28T21:54:46.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.032+0000 7f95cdd9b700 1 -- 172.21.15.162:0/3169773246 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95c800b970 con 0x7f95d0140590 2020-11-28T21:54:46.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.032+0000 7f95d4998700 1 -- 172.21.15.162:0/3169773246 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95d0140590 msgr2=0x7f95d00b0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:46.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.032+0000 7f95d4998700 1 --2- 172.21.15.162:0/3169773246 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95d0140590 0x7f95d00b0ca0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 rx=0x7f95c0001870 tx=0x7f95c80091a0).stop 2020-11-28T21:54:46.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.032+0000 7f95d4998700 1 -- 172.21.15.162:0/3169773246 shutdown_connections 2020-11-28T21:54:46.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.032+0000 7f95d4998700 1 --2- 172.21.15.162:0/3169773246 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95d0140590 0x7f95d00b0ca0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:46.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.032+0000 7f95d4998700 1 --2- 172.21.15.162:0/3169773246 >> v2:172.21.15.118:3300/0 conn(0x7f95d013dfc0 0x7f95d00b20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:46.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.032+0000 7f95d4998700 1 -- 172.21.15.162:0/3169773246 >> 172.21.15.162:0/3169773246 conn(0x7f95d00b2830 msgr2=0x7f95d00b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:46.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.032+0000 7f95d4998700 1 -- 172.21.15.162:0/3169773246 shutdown_connections 2020-11-28T21:54:46.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.032+0000 7f95d4998700 1 -- 172.21.15.162:0/3169773246 wait complete. 2020-11-28T21:54:46.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.032+0000 7f95d4998700 1 Processor -- start 2020-11-28T21:54:46.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.032+0000 7f95d4998700 1 -- start start 2020-11-28T21:54:46.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.036+0000 7f95d4998700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f95d013dbe0 0x7f95d0151e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:46.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.036+0000 7f95d4998700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95d013dfc0 0x7f95d0154920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:46.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.036+0000 7f95d4998700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f95d0140590 0x7f95d013c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:46.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.036+0000 7f95d4998700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d01514c0 con 0x7f95d0140590 2020-11-28T21:54:46.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.036+0000 7f95d4998700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d01516f0 con 0x7f95d013dfc0 2020-11-28T21:54:46.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.036+0000 7f95d4998700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d0151990 con 0x7f95d013dbe0 2020-11-28T21:54:46.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.036+0000 7f95ce59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95d013dfc0 0x7f95d0154920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:46.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.036+0000 7f95ce59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95d013dfc0 0x7f95d0154920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43896/0 (socket says 172.21.15.162:43896) 2020-11-28T21:54:46.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.036+0000 7f95ced9d700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f95d013dbe0 0x7f95d0151e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:46.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.036+0000 7f95ce59c700 1 -- 172.21.15.162:0/3831505416 learned_addr learned my addr 172.21.15.162:0/3831505416 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:46.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.036+0000 7f95cf59e700 1 --2- 172.21.15.162:0/3831505416 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f95d0140590 0x7f95d013c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:46.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.036+0000 7f95ce59c700 1 -- 172.21.15.162:0/3831505416 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f95d013dbe0 msgr2=0x7f95d0151e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:46.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.036+0000 7f95ce59c700 1 --2- 172.21.15.162:0/3831505416 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f95d013dbe0 0x7f95d0151e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:46.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.036+0000 7f95ce59c700 1 -- 172.21.15.162:0/3831505416 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f95d0140590 msgr2=0x7f95d013c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:46.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.036+0000 7f95ce59c700 1 --2- 172.21.15.162:0/3831505416 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f95d0140590 0x7f95d013c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:46.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.036+0000 7f95ce59c700 1 -- 172.21.15.162:0/3831505416 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d00ac8b0 con 0x7f95d013dfc0 2020-11-28T21:54:46.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.036+0000 7f95ce59c700 1 --2- 172.21.15.162:0/3831505416 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95d013dfc0 0x7f95d0154920 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 rx=0x7f95d01513e0 tx=0x7f95bc008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:46.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.036+0000 7f95c7fff700 1 -- 172.21.15.162:0/3831505416 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95bc00c820 con 0x7f95d013dfc0 2020-11-28T21:54:46.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.036+0000 7f95c7fff700 1 -- 172.21.15.162:0/3831505416 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95bc00c980 con 0x7f95d013dfc0 2020-11-28T21:54:46.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.036+0000 7f95c7fff700 1 -- 172.21.15.162:0/3831505416 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95bc00cce0 con 0x7f95d013dfc0 2020-11-28T21:54:46.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.036+0000 7f95d4998700 1 -- 172.21.15.162:0/3831505416 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95d01424c0 con 0x7f95d013dfc0 2020-11-28T21:54:46.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.036+0000 7f95d4998700 1 -- 172.21.15.162:0/3831505416 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d013d2c0 con 0x7f95d013dfc0 2020-11-28T21:54:46.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.036+0000 7f95d4998700 1 -- 172.21.15.162:0/3831505416 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95d0057050 con 0x7f95d013dfc0 2020-11-28T21:54:46.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.040+0000 7f95c7fff700 1 -- 172.21.15.162:0/3831505416 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f95bc022070 con 0x7f95d013dfc0 2020-11-28T21:54:46.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.040+0000 7f95c7fff700 1 --2- 172.21.15.162:0/3831505416 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f95ac0553b0 0x7f95ac057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:46.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.040+0000 7f95c7fff700 1 -- 172.21.15.162:0/3831505416 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5721+0+0 (secure 0 0 0) 0x7f95bc06e1f0 con 0x7f95d013dfc0 2020-11-28T21:54:46.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.040+0000 7f95ced9d700 1 --2- 172.21.15.162:0/3831505416 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f95ac0553b0 0x7f95ac057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:46.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.040+0000 7f95ced9d700 1 --2- 172.21.15.162:0/3831505416 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f95ac0553b0 0x7f95ac057870 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f95c000d000 tx=0x7f95c0001140).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:46.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.048+0000 7f95c7fff700 1 -- 172.21.15.162:0/3831505416 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f95bc044540 con 0x7f95d013dfc0 2020-11-28T21:54:46.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.188+0000 7f95d4998700 1 -- 172.21.15.162:0/3831505416 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f95d0068350 con 0x7f95d013dfc0 2020-11-28T21:54:46.898 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.896+0000 7f95c7fff700 1 -- 172.21.15.162:0/3831505416 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v112) v1 ==== 173+0+0 (secure 0 0 0) 0x7f95bc047b60 con 0x7f95d013dfc0 2020-11-28T21:54:46.901 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.896+0000 7f95d4998700 1 -- 172.21.15.162:0/3831505416 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f95ac0553b0 msgr2=0x7f95ac057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:46.901 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.896+0000 7f95d4998700 1 --2- 172.21.15.162:0/3831505416 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f95ac0553b0 0x7f95ac057870 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f95c000d000 tx=0x7f95c0001140).stop 2020-11-28T21:54:46.901 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.896+0000 7f95d4998700 1 -- 172.21.15.162:0/3831505416 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95d013dfc0 msgr2=0x7f95d0154920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:46.902 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.896+0000 7f95d4998700 1 --2- 172.21.15.162:0/3831505416 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95d013dfc0 0x7f95d0154920 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 rx=0x7f95d01513e0 tx=0x7f95bc008040).stop 2020-11-28T21:54:46.902 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.896+0000 7f95d4998700 1 -- 172.21.15.162:0/3831505416 shutdown_connections 2020-11-28T21:54:46.902 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.896+0000 7f95d4998700 1 --2- 172.21.15.162:0/3831505416 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f95ac0553b0 0x7f95ac057870 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:46.902 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.896+0000 7f95d4998700 1 --2- 172.21.15.162:0/3831505416 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f95d013dbe0 0x7f95d0151e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:46.902 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.896+0000 7f95d4998700 1 --2- 172.21.15.162:0/3831505416 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f95d0140590 0x7f95d013c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:46.903 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.896+0000 7f95d4998700 1 --2- 172.21.15.162:0/3831505416 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95d013dfc0 0x7f95d0154920 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:46.903 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.896+0000 7f95d4998700 1 -- 172.21.15.162:0/3831505416 >> 172.21.15.162:0/3831505416 conn(0x7f95d00b2830 msgr2=0x7f95d01409a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:46.903 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.896+0000 7f95d4998700 1 -- 172.21.15.162:0/3831505416 shutdown_connections 2020-11-28T21:54:46.903 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:46.896+0000 7f95d4998700 1 -- 172.21.15.162:0/3831505416 wait complete. 2020-11-28T21:54:46.904 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_data' removed 2020-11-28T21:54:46.915 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-28T21:54:46.916 INFO:tasks.ceph.mds.a:Stopping old one... 2020-11-28T21:54:46.916 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-11-28T21:54:46.916 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T21:54:46.916 INFO:tasks.ceph.mds.c:Restarting daemon 2020-11-28T21:54:46.917 INFO:tasks.ceph.mds.c:Stopping old one... 2020-11-28T21:54:46.917 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2020-11-28T21:54:46.917 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T21:54:46.918 INFO:tasks.ceph.mds.b:Restarting daemon 2020-11-28T21:54:46.918 INFO:tasks.ceph.mds.b:Stopping old one... 2020-11-28T21:54:46.918 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-11-28T21:54:46.918 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T21:54:46.927 INFO:tasks.ceph.mds.c:Stopped 2020-11-28T21:54:46.928 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2020-11-28T21:54:46.929 INFO:tasks.ceph.mds.a:Stopped 2020-11-28T21:54:46.929 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-28T21:54:46.932 INFO:tasks.ceph.mds.b:Stopped 2020-11-28T21:54:46.933 INFO:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-11-28T21:54:46.974 INFO:tasks.ceph.mds.c:Started 2020-11-28T21:54:46.975 DEBUG:teuthology.parallel:result is None 2020-11-28T21:54:46.976 INFO:tasks.ceph.mds.a:Started 2020-11-28T21:54:46.976 DEBUG:teuthology.parallel:result is None 2020-11-28T21:54:46.981 INFO:tasks.ceph.mds.b:Started 2020-11-28T21:54:46.981 DEBUG:teuthology.parallel:result is None 2020-11-28T21:54:46.982 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist clear 2020-11-28T21:54:47.048 INFO:tasks.ceph.mds.a.smithi162.stdout:starting mds.a at 2020-11-28T21:54:47.055 INFO:tasks.ceph.mds.c.smithi162.stdout:starting mds.c at 2020-11-28T21:54:47.058 INFO:tasks.ceph.mds.b.smithi118.stdout:starting mds.b at 2020-11-28T21:54:47.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.104+0000 7fa217fff700 1 Processor -- start 2020-11-28T21:54:47.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.104+0000 7fa217fff700 1 -- start start 2020-11-28T21:54:47.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.104+0000 7fa217fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa21813f6c0 0x7fa2180669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:47.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.104+0000 7fa217fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa218065750 0x7fa21813c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:47.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.104+0000 7fa217fff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa218065050 con 0x7fa218064c70 2020-11-28T21:54:47.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.104+0000 7fa217fff700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa21813fc40 con 0x7fa218065750 2020-11-28T21:54:47.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.104+0000 7fa217fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa21813fee0 con 0x7fa21813f6c0 2020-11-28T21:54:47.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.104+0000 7fa2177fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa218065750 0x7fa21813c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:47.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.104+0000 7fa216ffd700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fa218064c70 0x7fa218066290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.162:40918/0 (socket says 172.21.15.162:40918) 2020-11-28T21:54:47.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.104+0000 7fa216ffd700 1 -- 172.21.15.162:0/2713424285 learned_addr learned my addr 172.21.15.162:0/2713424285 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:47.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.104+0000 7fa2167fc700 1 --2- 172.21.15.162:0/2713424285 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa21813f6c0 0x7fa2180669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:47.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.108+0000 7fa215ffb700 1 -- 172.21.15.162:0/2713424285 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2079225339 0 0) 0x7fa218065050 con 0x7fa218064c70 2020-11-28T21:54:47.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.108+0000 7fa215ffb700 1 -- 172.21.15.162:0/2713424285 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1f8003230 con 0x7fa218064c70 2020-11-28T21:54:47.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.108+0000 7fa2177fe700 1 -- 172.21.15.162:0/2713424285 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa21813f6c0 msgr2=0x7fa2180669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:47.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.108+0000 7fa2177fe700 1 --2- 172.21.15.162:0/2713424285 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa21813f6c0 0x7fa2180669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:47.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.108+0000 7fa2177fe700 1 -- 172.21.15.162:0/2713424285 >> v1:172.21.15.118:6789/0 conn(0x7fa218064c70 legacy=0x7fa218066290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:47.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.108+0000 7fa2177fe700 1 -- 172.21.15.162:0/2713424285 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa218140180 con 0x7fa218065750 2020-11-28T21:54:47.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.108+0000 7fa2177fe700 1 --2- 172.21.15.162:0/2713424285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa218065750 0x7fa21813c260 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 rx=0x7fa208002f30 tx=0x7fa208006dc0).ready entity=mon.1 client_cookie=3f6d9812ea8072da server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:47.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.108+0000 7fa215ffb700 1 -- 172.21.15.162:0/2713424285 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2080096a0 con 0x7fa218065750 2020-11-28T21:54:47.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.108+0000 7fa215ffb700 1 -- 172.21.15.162:0/2713424285 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa208009800 con 0x7fa218065750 2020-11-28T21:54:47.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.108+0000 7fa215ffb700 1 -- 172.21.15.162:0/2713424285 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa208009b60 con 0x7fa218065750 2020-11-28T21:54:47.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.108+0000 7fa217fff700 1 -- 172.21.15.162:0/2713424285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa218065750 msgr2=0x7fa21813c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:47.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.108+0000 7fa217fff700 1 --2- 172.21.15.162:0/2713424285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa218065750 0x7fa21813c260 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 rx=0x7fa208002f30 tx=0x7fa208006dc0).stop 2020-11-28T21:54:47.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.108+0000 7fa217fff700 1 -- 172.21.15.162:0/2713424285 shutdown_connections 2020-11-28T21:54:47.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.108+0000 7fa217fff700 1 --2- 172.21.15.162:0/2713424285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa218065750 0x7fa21813c260 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:47.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.108+0000 7fa217fff700 1 --2- 172.21.15.162:0/2713424285 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa21813f6c0 0x7fa2180669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:47.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.108+0000 7fa217fff700 1 -- 172.21.15.162:0/2713424285 >> 172.21.15.162:0/2713424285 conn(0x7fa2180b5da0 msgr2=0x7fa2180b6180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:47.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.108+0000 7fa217fff700 1 -- 172.21.15.162:0/2713424285 shutdown_connections 2020-11-28T21:54:47.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.108+0000 7fa217fff700 1 -- 172.21.15.162:0/2713424285 wait complete. 2020-11-28T21:54:47.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.108+0000 7fa217fff700 1 Processor -- start 2020-11-28T21:54:47.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.108+0000 7fa217fff700 1 -- start start 2020-11-28T21:54:47.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.112+0000 7fa217fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa218064c70 0x7fa218144c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:47.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.112+0000 7fa217fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa218065750 0x7fa2181410e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:47.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.112+0000 7fa217fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa21813f6c0 0x7fa2181415d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:47.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.112+0000 7fa217fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa218145c70 con 0x7fa218064c70 2020-11-28T21:54:47.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.112+0000 7fa217fff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2180b0b80 con 0x7fa218065750 2020-11-28T21:54:47.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.112+0000 7fa217fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa218145ea0 con 0x7fa21813f6c0 2020-11-28T21:54:47.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.112+0000 7fa2167fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa218065750 0x7fa2181410e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:47.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.112+0000 7fa2167fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa218065750 0x7fa2181410e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43934/0 (socket says 172.21.15.162:43934) 2020-11-28T21:54:47.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.112+0000 7fa2167fc700 1 -- 172.21.15.162:0/2192156018 learned_addr learned my addr 172.21.15.162:0/2192156018 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:47.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.112+0000 7fa2177fe700 1 --2- 172.21.15.162:0/2192156018 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa21813f6c0 0x7fa2181415d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:47.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.112+0000 7fa216ffd700 1 --2- 172.21.15.162:0/2192156018 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa218064c70 0x7fa218144c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:47.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.112+0000 7fa2167fc700 1 -- 172.21.15.162:0/2192156018 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa21813f6c0 msgr2=0x7fa2181415d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:47.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.112+0000 7fa2167fc700 1 --2- 172.21.15.162:0/2192156018 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa21813f6c0 0x7fa2181415d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:47.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.112+0000 7fa2167fc700 1 -- 172.21.15.162:0/2192156018 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa218064c70 msgr2=0x7fa218144c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:47.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.112+0000 7fa2167fc700 1 --2- 172.21.15.162:0/2192156018 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa218064c70 0x7fa218144c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:47.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.112+0000 7fa2167fc700 1 -- 172.21.15.162:0/2192156018 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa218146140 con 0x7fa218065750 2020-11-28T21:54:47.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.112+0000 7fa216ffd700 1 --2- 172.21.15.162:0/2192156018 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa218064c70 0x7fa218144c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:54:47.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.112+0000 7fa2167fc700 1 --2- 172.21.15.162:0/2192156018 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa218065750 0x7fa2181410e0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 rx=0x7fa21000c000 tx=0x7fa210006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:47.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.112+0000 7fa20ffff700 1 -- 172.21.15.162:0/2192156018 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa21000ca10 con 0x7fa218065750 2020-11-28T21:54:47.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.112+0000 7fa20ffff700 1 -- 172.21.15.162:0/2192156018 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa21000cb70 con 0x7fa218065750 2020-11-28T21:54:47.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.112+0000 7fa217fff700 1 -- 172.21.15.162:0/2192156018 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2180b0440 con 0x7fa218065750 2020-11-28T21:54:47.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.112+0000 7fa20ffff700 1 -- 172.21.15.162:0/2192156018 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2100123f0 con 0x7fa218065750 2020-11-28T21:54:47.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.112+0000 7fa217fff700 1 -- 172.21.15.162:0/2192156018 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa218141ac0 con 0x7fa218065750 2020-11-28T21:54:47.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.116+0000 7fa217fff700 1 -- 172.21.15.162:0/2192156018 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa218057050 con 0x7fa218065750 2020-11-28T21:54:47.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.116+0000 7fa20ffff700 1 -- 172.21.15.162:0/2192156018 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa210012550 con 0x7fa218065750 2020-11-28T21:54:47.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.116+0000 7fa20ffff700 1 --2- 172.21.15.162:0/2192156018 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa1fc055350 0x7fa1fc057810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:47.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.116+0000 7fa20ffff700 1 -- 172.21.15.162:0/2192156018 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5339+0+0 (secure 0 0 0) 0x7fa21006e100 con 0x7fa218065750 2020-11-28T21:54:47.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.116+0000 7fa216ffd700 1 --2- 172.21.15.162:0/2192156018 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa1fc055350 0x7fa1fc057810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:47.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.116+0000 7fa216ffd700 1 --2- 172.21.15.162:0/2192156018 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa1fc055350 0x7fa1fc057810 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7fa20000b000 tx=0x7fa200009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:47.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.124+0000 7fa20ffff700 1 -- 172.21.15.162:0/2192156018 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa2100445e0 con 0x7fa218065750 2020-11-28T21:54:47.281 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:47.276+0000 7fa217fff700 1 -- 172.21.15.162:0/2192156018 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fa218063480 con 0x7fa218065750 2020-11-28T21:54:48.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.088+0000 7fa20ffff700 1 -- 172.21.15.162:0/2192156018 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v115) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa210047c00 con 0x7fa218065750 2020-11-28T21:54:48.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.088+0000 7fa217fff700 1 -- 172.21.15.162:0/2192156018 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa1fc055350 msgr2=0x7fa1fc057810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:48.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.088+0000 7fa217fff700 1 --2- 172.21.15.162:0/2192156018 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa1fc055350 0x7fa1fc057810 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7fa20000b000 tx=0x7fa200009040).stop 2020-11-28T21:54:48.094 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.088+0000 7fa217fff700 1 -- 172.21.15.162:0/2192156018 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa218065750 msgr2=0x7fa2181410e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:48.094 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.088+0000 7fa217fff700 1 --2- 172.21.15.162:0/2192156018 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa218065750 0x7fa2181410e0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 rx=0x7fa21000c000 tx=0x7fa210006dc0).stop 2020-11-28T21:54:48.094 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.088+0000 7fa217fff700 1 -- 172.21.15.162:0/2192156018 shutdown_connections 2020-11-28T21:54:48.094 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.088+0000 7fa217fff700 1 --2- 172.21.15.162:0/2192156018 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa1fc055350 0x7fa1fc057810 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:48.095 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.088+0000 7fa217fff700 1 --2- 172.21.15.162:0/2192156018 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa21813f6c0 0x7fa2181415d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:48.095 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.088+0000 7fa217fff700 1 --2- 172.21.15.162:0/2192156018 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa218064c70 0x7fa218144c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:48.095 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.088+0000 7fa217fff700 1 --2- 172.21.15.162:0/2192156018 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa218065750 0x7fa2181410e0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:48.095 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.088+0000 7fa217fff700 1 -- 172.21.15.162:0/2192156018 >> 172.21.15.162:0/2192156018 conn(0x7fa2180b5da0 msgr2=0x7fa21813d560 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:48.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.088+0000 7fa217fff700 1 -- 172.21.15.162:0/2192156018 shutdown_connections 2020-11-28T21:54:48.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.088+0000 7fa217fff700 1 -- 172.21.15.162:0/2192156018 wait complete. 2020-11-28T21:54:48.097 INFO:teuthology.orchestra.run.smithi162.stderr: removed all blacklist entries 2020-11-28T21:54:48.110 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2020-11-28T21:54:48.110 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth ls --format=json-pretty 2020-11-28T21:54:48.226 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.224+0000 7f2b2ed6e700 1 Processor -- start 2020-11-28T21:54:48.226 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.224+0000 7f2b2ed6e700 1 -- start start 2020-11-28T21:54:48.227 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.224+0000 7f2b2ed6e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2b280ae710 0x7f2b280aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:48.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.224+0000 7f2b2ed6e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f2b280af8f0 0x7f2b2813eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:48.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.224+0000 7f2b2ed6e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b280b0730 con 0x7f2b280b01f0 2020-11-28T21:54:48.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.224+0000 7f2b2ed6e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b280b6680 con 0x7f2b280af8f0 2020-11-28T21:54:48.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.224+0000 7f2b2ed6e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b280ad7f0 con 0x7f2b280ae710 2020-11-28T21:54:48.229 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.224+0000 7f2b2cb0a700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2b280ae710 0x7f2b280aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:48.229 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.224+0000 7f2b27fff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f2b280af8f0 0x7f2b2813eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:48.229 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.224+0000 7f2b2cb0a700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2b280ae710 0x7f2b280aeaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:41166/0 (socket says 172.21.15.162:41166) 2020-11-28T21:54:48.229 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.224+0000 7f2b2cb0a700 1 -- 172.21.15.162:0/1765134236 learned_addr learned my addr 172.21.15.162:0/1765134236 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:48.229 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.224+0000 7f2b2d30b700 1 --1- 172.21.15.162:0/1765134236 >> v1:172.21.15.118:6789/0 conn(0x7f2b280b01f0 0x7f2b280acd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.162:40936/0 (socket says 172.21.15.162:40936) 2020-11-28T21:54:48.231 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.224+0000 7f2b277fe700 1 -- 172.21.15.162:0/1765134236 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2331516284 0 0) 0x7f2b280b0730 con 0x7f2b280b01f0 2020-11-28T21:54:48.231 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.224+0000 7f2b277fe700 1 -- 172.21.15.162:0/1765134236 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b14003230 con 0x7f2b280b01f0 2020-11-28T21:54:48.231 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.224+0000 7f2b27fff700 1 -- 172.21.15.162:0/1765134236 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2b280ae710 msgr2=0x7f2b280aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:48.232 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.224+0000 7f2b27fff700 1 --2- 172.21.15.162:0/1765134236 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2b280ae710 0x7f2b280aeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:48.232 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.224+0000 7f2b27fff700 1 -- 172.21.15.162:0/1765134236 >> v1:172.21.15.118:6789/0 conn(0x7f2b280b01f0 legacy=0x7f2b280acd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:48.232 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.224+0000 7f2b27fff700 1 -- 172.21.15.162:0/1765134236 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b280b1990 con 0x7f2b280af8f0 2020-11-28T21:54:48.232 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.224+0000 7f2b2cb0a700 1 --2- 172.21.15.162:0/1765134236 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2b280ae710 0x7f2b280aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:54:48.233 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.224+0000 7f2b27fff700 1 --2- 172.21.15.162:0/1765134236 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b280af8f0 0x7f2b2813eff0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f2b1c002f30 tx=0x7f2b1c006dc0).ready entity=mon.0 client_cookie=bb5b5dc669687c40 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:48.233 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.224+0000 7f2b277fe700 1 -- 172.21.15.162:0/1765134236 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b1c0096a0 con 0x7f2b280af8f0 2020-11-28T21:54:48.233 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.224+0000 7f2b277fe700 1 -- 172.21.15.162:0/1765134236 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b1c009800 con 0x7f2b280af8f0 2020-11-28T21:54:48.233 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.224+0000 7f2b277fe700 1 -- 172.21.15.162:0/1765134236 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b1c009b60 con 0x7f2b280af8f0 2020-11-28T21:54:48.233 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.224+0000 7f2b2ed6e700 1 -- 172.21.15.162:0/1765134236 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b280af8f0 msgr2=0x7f2b2813eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:48.234 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.224+0000 7f2b2ed6e700 1 --2- 172.21.15.162:0/1765134236 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b280af8f0 0x7f2b2813eff0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f2b1c002f30 tx=0x7f2b1c006dc0).stop 2020-11-28T21:54:48.234 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.228+0000 7f2b2ed6e700 1 -- 172.21.15.162:0/1765134236 shutdown_connections 2020-11-28T21:54:48.234 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.228+0000 7f2b2ed6e700 1 --2- 172.21.15.162:0/1765134236 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2b280ae710 0x7f2b280aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:48.234 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.228+0000 7f2b2ed6e700 1 --2- 172.21.15.162:0/1765134236 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b280af8f0 0x7f2b2813eff0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:48.234 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.228+0000 7f2b2ed6e700 1 -- 172.21.15.162:0/1765134236 >> 172.21.15.162:0/1765134236 conn(0x7f2b280b56f0 msgr2=0x7f2b280b5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:48.235 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.228+0000 7f2b2ed6e700 1 -- 172.21.15.162:0/1765134236 shutdown_connections 2020-11-28T21:54:48.235 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.228+0000 7f2b2ed6e700 1 -- 172.21.15.162:0/1765134236 wait complete. 2020-11-28T21:54:48.237 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.228+0000 7f2b2ed6e700 1 Processor -- start 2020-11-28T21:54:48.237 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.228+0000 7f2b2ed6e700 1 -- start start 2020-11-28T21:54:48.237 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.228+0000 7f2b2ed6e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b280ae710 0x7f2b2813d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:48.238 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.228+0000 7f2b2ed6e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b280af8f0 0x7f2b281520d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:48.238 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.228+0000 7f2b2ed6e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2b280b01f0 0x7f2b281369d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:48.238 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.228+0000 7f2b2ed6e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b28140d60 con 0x7f2b280ae710 2020-11-28T21:54:48.238 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.228+0000 7f2b2ed6e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b28140fc0 con 0x7f2b280af8f0 2020-11-28T21:54:48.239 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.228+0000 7f2b2ed6e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b28141260 con 0x7f2b280b01f0 2020-11-28T21:54:48.239 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.232+0000 7f2b27fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b280af8f0 0x7f2b281520d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:48.239 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.232+0000 7f2b27fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b280af8f0 0x7f2b281520d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43952/0 (socket says 172.21.15.162:43952) 2020-11-28T21:54:48.239 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.232+0000 7f2b2d30b700 1 --2- 172.21.15.162:0/1811290475 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2b280b01f0 0x7f2b281369d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:48.239 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.232+0000 7f2b27fff700 1 -- 172.21.15.162:0/1811290475 learned_addr learned my addr 172.21.15.162:0/1811290475 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:48.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.232+0000 7f2b2cb0a700 1 --2- 172.21.15.162:0/1811290475 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b280ae710 0x7f2b2813d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:48.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.232+0000 7f2b27fff700 1 -- 172.21.15.162:0/1811290475 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2b280b01f0 msgr2=0x7f2b281369d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:48.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.232+0000 7f2b27fff700 1 --2- 172.21.15.162:0/1811290475 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2b280b01f0 0x7f2b281369d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:48.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.232+0000 7f2b27fff700 1 -- 172.21.15.162:0/1811290475 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b280ae710 msgr2=0x7f2b2813d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:48.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.232+0000 7f2b27fff700 1 --2- 172.21.15.162:0/1811290475 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b280ae710 0x7f2b2813d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:48.241 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.232+0000 7f2b27fff700 1 -- 172.21.15.162:0/1811290475 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b28141500 con 0x7f2b280af8f0 2020-11-28T21:54:48.241 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.232+0000 7f2b27fff700 1 --2- 172.21.15.162:0/1811290475 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b280af8f0 0x7f2b281520d0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 rx=0x7f2b1c002f30 tx=0x7f2b1c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:48.241 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.232+0000 7f2b25ffb700 1 -- 172.21.15.162:0/1811290475 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b1c011c30 con 0x7f2b280af8f0 2020-11-28T21:54:48.241 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.232+0000 7f2b25ffb700 1 -- 172.21.15.162:0/1811290475 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b1c011d90 con 0x7f2b280af8f0 2020-11-28T21:54:48.241 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.232+0000 7f2b2ed6e700 1 -- 172.21.15.162:0/1811290475 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b281417a0 con 0x7f2b280af8f0 2020-11-28T21:54:48.242 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.232+0000 7f2b25ffb700 1 -- 172.21.15.162:0/1811290475 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b1c01d640 con 0x7f2b280af8f0 2020-11-28T21:54:48.242 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.232+0000 7f2b2ed6e700 1 -- 172.21.15.162:0/1811290475 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b2813de10 con 0x7f2b280af8f0 2020-11-28T21:54:48.243 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.232+0000 7f2b2ed6e700 1 -- 172.21.15.162:0/1811290475 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b28057050 con 0x7f2b280af8f0 2020-11-28T21:54:48.243 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.236+0000 7f2b25ffb700 1 -- 172.21.15.162:0/1811290475 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2b1c024030 con 0x7f2b280af8f0 2020-11-28T21:54:48.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.236+0000 7f2b25ffb700 1 --2- 172.21.15.162:0/1811290475 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2b080554e0 0x7f2b080579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:48.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.236+0000 7f2b25ffb700 1 -- 172.21.15.162:0/1811290475 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f2b1c06eb50 con 0x7f2b280af8f0 2020-11-28T21:54:48.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.236+0000 7f2b2cb0a700 1 --2- 172.21.15.162:0/1811290475 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2b080554e0 0x7f2b080579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:48.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.236+0000 7f2b2cb0a700 1 --2- 172.21.15.162:0/1811290475 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2b080554e0 0x7f2b080579a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7f2b1800c000 tx=0x7f2b18006f10).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:48.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.244+0000 7f2b25ffb700 1 -- 172.21.15.162:0/1811290475 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f2b1c045180 con 0x7f2b280af8f0 2020-11-28T21:54:48.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.432+0000 7f2b2ed6e700 1 -- 172.21.15.162:0/1811290475 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f2b28141930 con 0x7f2b280af8f0 2020-11-28T21:54:48.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.436+0000 7f2b25ffb700 1 -- 172.21.15.162:0/1811290475 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v26) v1 ==== 80+0+4255 (secure 0 0 0) 0x7f2b1c022070 con 0x7f2b280af8f0 2020-11-28T21:54:48.438 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:54:48.439 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2020-11-28T21:54:48.439 INFO:teuthology.orchestra.run.smithi162.stdout: "auth_dump": [ 2020-11-28T21:54:48.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:48.439 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mds.a", 2020-11-28T21:54:48.440 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfWxUpKxAAIUDwQRKwz8H4Qk9qE356Bg==", 2020-11-28T21:54:48.440 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:54:48.440 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:54:48.440 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:54:48.440 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:54:48.441 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:54:48.441 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:48.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:48.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:48.441 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mds.b", 2020-11-28T21:54:48.442 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfKY2zORAAaO65etMZiYcaTDNX0nsEPw==", 2020-11-28T21:54:48.442 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:54:48.442 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:54:48.443 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:54:48.443 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:54:48.443 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:54:48.443 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:48.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:48.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:48.444 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mds.c", 2020-11-28T21:54:48.444 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfdQbRMxAAWlcjesxTZzZnvKCQneSjjQ==", 2020-11-28T21:54:48.444 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:54:48.445 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:54:48.445 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:54:48.445 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:54:48.445 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:54:48.445 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:48.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:48.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:48.446 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.0", 2020-11-28T21:54:48.446 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQADxcJfF3WLLxAAxsB5YPjH1yjv/vhlRbijqw==", 2020-11-28T21:54:48.447 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:54:48.447 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:54:48.447 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:54:48.448 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:54:48.448 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:48.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:48.448 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:48.449 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.1", 2020-11-28T21:54:48.449 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAGxcJfKYbBFhAA1x4DzFtkQQv6uWLIdOi1qQ==", 2020-11-28T21:54:48.449 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:54:48.449 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:54:48.450 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:54:48.450 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:54:48.450 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:48.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:48.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:48.451 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.2", 2020-11-28T21:54:48.451 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAIxcJfvcYUOhAASENjM0HfAxxyFe13mmE36Q==", 2020-11-28T21:54:48.451 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:54:48.452 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:54:48.452 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:54:48.452 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:54:48.452 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:48.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:48.453 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:48.453 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.3", 2020-11-28T21:54:48.453 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQALxcJfc/OHIRAAFYx3s8Ssttt15hjoH04tow==", 2020-11-28T21:54:48.454 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:54:48.454 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:54:48.454 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:54:48.454 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:54:48.455 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:48.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:48.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:48.455 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.4", 2020-11-28T21:54:48.455 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAPxcJffGWqCRAAb8CCN5oUPkdDh6BJH5HEnw==", 2020-11-28T21:54:48.456 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:54:48.456 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:54:48.456 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:54:48.456 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:54:48.456 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:48.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:48.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:48.457 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.5", 2020-11-28T21:54:48.457 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQARxcJfo8GKKRAAfbsCDb4L2wULiKmHyzu7/A==", 2020-11-28T21:54:48.457 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:54:48.458 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:54:48.458 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:54:48.458 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:54:48.458 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:48.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:48.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:48.459 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.6", 2020-11-28T21:54:48.459 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAUxcJf8Y2MEBAAqQX6rX3LK0zodlI2wK4JrA==", 2020-11-28T21:54:48.459 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:54:48.460 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:54:48.460 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:54:48.460 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:54:48.460 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:48.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:48.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:48.461 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.7", 2020-11-28T21:54:48.462 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAWxcJfCLBdMhAAf8QixQMQDrfZOGWN8rHe5g==", 2020-11-28T21:54:48.462 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:54:48.462 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:54:48.462 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:54:48.463 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:54:48.463 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:48.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:48.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:48.464 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "client.0", 2020-11-28T21:54:48.464 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQABxcJfS9vDAxAAUk4FU8FVt4if/CRH5ZisOA==", 2020-11-28T21:54:48.464 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:54:48.464 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:54:48.464 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow r", 2020-11-28T21:54:48.465 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow rw pool=cephfs_data" 2020-11-28T21:54:48.465 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:48.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:48.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:48.466 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "client.1", 2020-11-28T21:54:48.466 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQABxcJfuCU8BhAAXUSoSK3D0XRo58uHNzcjXg==", 2020-11-28T21:54:48.466 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:54:48.466 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:54:48.467 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow r", 2020-11-28T21:54:48.467 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow rw pool=cephfs_data" 2020-11-28T21:54:48.467 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:48.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:48.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:48.468 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "client.admin", 2020-11-28T21:54:48.468 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfdlTWBxAApzAxlVjzdnETQF//Vnsg5w==", 2020-11-28T21:54:48.468 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:54:48.468 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow *", 2020-11-28T21:54:48.469 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:54:48.469 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:54:48.469 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:54:48.469 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:48.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:48.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:48.470 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mgr.x", 2020-11-28T21:54:48.470 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfZhdvKBAAT7zCCsD2OVCHjvddGLxQ/A==", 2020-11-28T21:54:48.470 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:54:48.471 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow *", 2020-11-28T21:54:48.472 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow profile mgr", 2020-11-28T21:54:48.472 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:54:48.472 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:48.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:48.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:48.473 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mgr.y", 2020-11-28T21:54:48.473 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfs8fvJRAAP0Atp074EeMHiTtpCCVHuA==", 2020-11-28T21:54:48.473 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:54:48.474 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow *", 2020-11-28T21:54:48.474 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow profile mgr", 2020-11-28T21:54:48.474 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:54:48.475 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:48.475 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:48.475 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:54:48.475 INFO:teuthology.orchestra.run.smithi162.stdout:} 2020-11-28T21:54:48.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.436+0000 7f2b2ed6e700 1 -- 172.21.15.162:0/1811290475 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2b080554e0 msgr2=0x7f2b080579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:48.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.436+0000 7f2b2ed6e700 1 --2- 172.21.15.162:0/1811290475 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2b080554e0 0x7f2b080579a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7f2b1800c000 tx=0x7f2b18006f10).stop 2020-11-28T21:54:48.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.436+0000 7f2b2ed6e700 1 -- 172.21.15.162:0/1811290475 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b280af8f0 msgr2=0x7f2b281520d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:48.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.436+0000 7f2b2ed6e700 1 --2- 172.21.15.162:0/1811290475 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b280af8f0 0x7f2b281520d0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 rx=0x7f2b1c002f30 tx=0x7f2b1c015000).stop 2020-11-28T21:54:48.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.436+0000 7f2b2ed6e700 1 -- 172.21.15.162:0/1811290475 shutdown_connections 2020-11-28T21:54:48.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.436+0000 7f2b2ed6e700 1 --2- 172.21.15.162:0/1811290475 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2b080554e0 0x7f2b080579a0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:48.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.436+0000 7f2b2ed6e700 1 --2- 172.21.15.162:0/1811290475 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2b280b01f0 0x7f2b281369d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:48.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.436+0000 7f2b2ed6e700 1 --2- 172.21.15.162:0/1811290475 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b280ae710 0x7f2b2813d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:48.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.436+0000 7f2b2ed6e700 1 --2- 172.21.15.162:0/1811290475 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b280af8f0 0x7f2b281520d0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:48.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.436+0000 7f2b2ed6e700 1 -- 172.21.15.162:0/1811290475 >> 172.21.15.162:0/1811290475 conn(0x7f2b280b56f0 msgr2=0x7f2b281399b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:48.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.436+0000 7f2b2ed6e700 1 -- 172.21.15.162:0/1811290475 shutdown_connections 2020-11-28T21:54:48.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.436+0000 7f2b2ed6e700 1 -- 172.21.15.162:0/1811290475 wait complete. 2020-11-28T21:54:48.484 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:54:48.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.600+0000 7f9f34047700 1 Processor -- start 2020-11-28T21:54:48.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.600+0000 7f9f34047700 1 -- start start 2020-11-28T21:54:48.603 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.600+0000 7f9f34047700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9f2c0b1ae0 0x7f9f2c13cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:48.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.600+0000 7f9f34047700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f9f2c13f9e0 0x7f9f2c13d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:48.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.600+0000 7f9f34047700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f2c0b1ec0 con 0x7f9f2c0aef20 2020-11-28T21:54:48.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.600+0000 7f9f34047700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f2c13fef0 con 0x7f9f2c13f9e0 2020-11-28T21:54:48.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.600+0000 7f9f34047700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f2c140190 con 0x7f9f2c0b1ae0 2020-11-28T21:54:48.605 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.600+0000 7f9f31de3700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9f2c0b1ae0 0x7f9f2c13cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:48.605 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.600+0000 7f9f315e2700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f9f2c0aef20 0x7f9f2c0ac2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:50800/0 (socket says 172.21.15.162:50800) 2020-11-28T21:54:48.605 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.600+0000 7f9f315e2700 1 -- 172.21.15.162:0/4293886044 learned_addr learned my addr 172.21.15.162:0/4293886044 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:48.605 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.600+0000 7f9f325e4700 1 --2- 172.21.15.162:0/4293886044 >> v2:172.21.15.118:3300/0 conn(0x7f9f2c13f9e0 0x7f9f2c13d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:48.605 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.600+0000 7f9f31de3700 1 -- 172.21.15.162:0/4293886044 >> v2:172.21.15.118:3300/0 conn(0x7f9f2c13f9e0 msgr2=0x7f9f2c13d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:48.606 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.600+0000 7f9f31de3700 1 --2- 172.21.15.162:0/4293886044 >> v2:172.21.15.118:3300/0 conn(0x7f9f2c13f9e0 0x7f9f2c13d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:48.606 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.600+0000 7f9f31de3700 1 -- 172.21.15.162:0/4293886044 >> v1:172.21.15.162:6789/0 conn(0x7f9f2c0aef20 legacy=0x7f9f2c0ac2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:48.606 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.600+0000 7f9f31de3700 1 -- 172.21.15.162:0/4293886044 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2c140430 con 0x7f9f2c0b1ae0 2020-11-28T21:54:48.606 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.600+0000 7f9f30de1700 1 -- 172.21.15.162:0/4293886044 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 215895012 0 0) 0x7f9f2c0b1ec0 con 0x7f9f2c0aef20 2020-11-28T21:54:48.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.600+0000 7f9f325e4700 1 --2- 172.21.15.162:0/4293886044 >> v2:172.21.15.118:3300/0 conn(0x7f9f2c13f9e0 0x7f9f2c13d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:54:48.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.600+0000 7f9f31de3700 1 --2- 172.21.15.162:0/4293886044 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f2c0b1ae0 0x7f9f2c13cc90 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 rx=0x7f9f200032b0 tx=0x7f9f200091a0).ready entity=mon.1 client_cookie=be2e07508710eaa server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:48.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.600+0000 7f9f30de1700 1 -- 172.21.15.162:0/4293886044 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f2000ba50 con 0x7f9f2c0b1ae0 2020-11-28T21:54:48.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.600+0000 7f9f30de1700 1 -- 172.21.15.162:0/4293886044 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2000bbb0 con 0x7f9f2c0b1ae0 2020-11-28T21:54:48.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.600+0000 7f9f30de1700 1 -- 172.21.15.162:0/4293886044 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f2000bd20 con 0x7f9f2c0b1ae0 2020-11-28T21:54:48.608 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.600+0000 7f9f34047700 1 -- 172.21.15.162:0/4293886044 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f2c0b1ae0 msgr2=0x7f9f2c13cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:48.608 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.600+0000 7f9f34047700 1 --2- 172.21.15.162:0/4293886044 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f2c0b1ae0 0x7f9f2c13cc90 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 rx=0x7f9f200032b0 tx=0x7f9f200091a0).stop 2020-11-28T21:54:48.611 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.600+0000 7f9f34047700 1 -- 172.21.15.162:0/4293886044 shutdown_connections 2020-11-28T21:54:48.611 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.600+0000 7f9f34047700 1 --2- 172.21.15.162:0/4293886044 >> v2:172.21.15.118:3300/0 conn(0x7f9f2c13f9e0 0x7f9f2c13d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:48.611 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.600+0000 7f9f34047700 1 --2- 172.21.15.162:0/4293886044 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f2c0b1ae0 0x7f9f2c13cc90 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:48.612 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.600+0000 7f9f34047700 1 -- 172.21.15.162:0/4293886044 >> 172.21.15.162:0/4293886044 conn(0x7f9f2c05b970 msgr2=0x7f9f2c05bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:48.612 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.600+0000 7f9f34047700 1 -- 172.21.15.162:0/4293886044 shutdown_connections 2020-11-28T21:54:48.612 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.600+0000 7f9f34047700 1 -- 172.21.15.162:0/4293886044 wait complete. 2020-11-28T21:54:48.612 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.604+0000 7f9f34047700 1 Processor -- start 2020-11-28T21:54:48.612 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.604+0000 7f9f34047700 1 -- start start 2020-11-28T21:54:48.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.604+0000 7f9f34047700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f2c0aef20 0x7f9f2c140d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:48.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.604+0000 7f9f34047700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9f2c0b1ae0 0x7f9f2c064c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:48.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.604+0000 7f9f34047700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9f2c13f9e0 0x7f9f2c0661a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:48.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.604+0000 7f9f34047700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c141700 con 0x7f9f2c0b1ae0 2020-11-28T21:54:48.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.604+0000 7f9f34047700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c1419a0 con 0x7f9f2c0aef20 2020-11-28T21:54:48.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.604+0000 7f9f34047700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c065d00 con 0x7f9f2c13f9e0 2020-11-28T21:54:48.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.604+0000 7f9f31de3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f2c0aef20 0x7f9f2c140d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:48.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.604+0000 7f9f31de3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f2c0aef20 0x7f9f2c140d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43964/0 (socket says 172.21.15.162:43964) 2020-11-28T21:54:48.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.604+0000 7f9f31de3700 1 -- 172.21.15.162:0/1456236057 learned_addr learned my addr 172.21.15.162:0/1456236057 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:48.615 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.604+0000 7f9f325e4700 1 --2- 172.21.15.162:0/1456236057 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9f2c13f9e0 0x7f9f2c0661a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:48.616 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.604+0000 7f9f315e2700 1 --2- 172.21.15.162:0/1456236057 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9f2c0b1ae0 0x7f9f2c064c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:48.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.604+0000 7f9f31de3700 1 -- 172.21.15.162:0/1456236057 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9f2c13f9e0 msgr2=0x7f9f2c0661a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:48.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.604+0000 7f9f31de3700 1 --2- 172.21.15.162:0/1456236057 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9f2c13f9e0 0x7f9f2c0661a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:48.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.604+0000 7f9f31de3700 1 -- 172.21.15.162:0/1456236057 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9f2c0b1ae0 msgr2=0x7f9f2c064c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:48.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.604+0000 7f9f31de3700 1 --2- 172.21.15.162:0/1456236057 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9f2c0b1ae0 0x7f9f2c064c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:48.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.604+0000 7f9f31de3700 1 -- 172.21.15.162:0/1456236057 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2c1451f0 con 0x7f9f2c0aef20 2020-11-28T21:54:48.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.604+0000 7f9f315e2700 1 --2- 172.21.15.162:0/1456236057 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9f2c0b1ae0 0x7f9f2c064c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:54:48.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.604+0000 7f9f31de3700 1 --2- 172.21.15.162:0/1456236057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f2c0aef20 0x7f9f2c140d80 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 rx=0x7f9f200032b0 tx=0x7f9f20016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:48.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.604+0000 7f9f2affd700 1 -- 172.21.15.162:0/1456236057 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f2001f5e0 con 0x7f9f2c0aef20 2020-11-28T21:54:48.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.604+0000 7f9f2affd700 1 -- 172.21.15.162:0/1456236057 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2001f740 con 0x7f9f2c0aef20 2020-11-28T21:54:48.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.604+0000 7f9f34047700 1 -- 172.21.15.162:0/1456236057 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f2c067520 con 0x7f9f2c0aef20 2020-11-28T21:54:48.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.604+0000 7f9f2affd700 1 -- 172.21.15.162:0/1456236057 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f2001faa0 con 0x7f9f2c0aef20 2020-11-28T21:54:48.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.604+0000 7f9f34047700 1 -- 172.21.15.162:0/1456236057 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f2c066830 con 0x7f9f2c0aef20 2020-11-28T21:54:48.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.604+0000 7f9f34047700 1 -- 172.21.15.162:0/1456236057 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f2c141b10 con 0x7f9f2c0aef20 2020-11-28T21:54:48.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.608+0000 7f9f2affd700 1 -- 172.21.15.162:0/1456236057 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9f2000f070 con 0x7f9f2c0aef20 2020-11-28T21:54:48.620 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.608+0000 7f9f2affd700 1 --2- 172.21.15.162:0/1456236057 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9f14055330 0x7f9f140577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:48.620 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.608+0000 7f9f2affd700 1 -- 172.21.15.162:0/1456236057 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f9f2006ebf0 con 0x7f9f2c0aef20 2020-11-28T21:54:48.620 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.608+0000 7f9f315e2700 1 --2- 172.21.15.162:0/1456236057 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9f14055330 0x7f9f140577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:48.620 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.608+0000 7f9f315e2700 1 --2- 172.21.15.162:0/1456236057 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9f14055330 0x7f9f140577f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7f9f18008000 tx=0x7f9f18009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:48.621 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.612+0000 7f9f2affd700 1 -- 172.21.15.162:0/1456236057 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9f20045220 con 0x7f9f2c0aef20 2020-11-28T21:54:48.751 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.748+0000 7f9f34047700 1 -- 172.21.15.162:0/1456236057 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f2c0676b0 con 0x7f9f2c0aef20 2020-11-28T21:54:48.752 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.748+0000 7f9f2affd700 1 -- 172.21.15.162:0/1456236057 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) v1 ==== 74+0+13287 (secure 0 0 0) 0x7f9f20030400 con 0x7f9f2c0aef20 2020-11-28T21:54:48.753 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:54:48.753 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":115,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:54:48.075886+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:54:48.755 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.752+0000 7f9f34047700 1 -- 172.21.15.162:0/1456236057 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9f14055330 msgr2=0x7f9f140577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:48.755 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.752+0000 7f9f34047700 1 --2- 172.21.15.162:0/1456236057 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9f14055330 0x7f9f140577f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7f9f18008000 tx=0x7f9f18009040).stop 2020-11-28T21:54:48.755 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.752+0000 7f9f34047700 1 -- 172.21.15.162:0/1456236057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f2c0aef20 msgr2=0x7f9f2c140d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:48.756 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.752+0000 7f9f34047700 1 --2- 172.21.15.162:0/1456236057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f2c0aef20 0x7f9f2c140d80 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 rx=0x7f9f200032b0 tx=0x7f9f20016000).stop 2020-11-28T21:54:48.756 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.752+0000 7f9f34047700 1 -- 172.21.15.162:0/1456236057 shutdown_connections 2020-11-28T21:54:48.756 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.752+0000 7f9f34047700 1 --2- 172.21.15.162:0/1456236057 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9f14055330 0x7f9f140577f0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:48.756 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.752+0000 7f9f34047700 1 --2- 172.21.15.162:0/1456236057 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9f2c13f9e0 0x7f9f2c0661a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:48.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.752+0000 7f9f34047700 1 --2- 172.21.15.162:0/1456236057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f2c0aef20 0x7f9f2c140d80 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:48.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.752+0000 7f9f34047700 1 --2- 172.21.15.162:0/1456236057 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9f2c0b1ae0 0x7f9f2c064c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:48.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.752+0000 7f9f34047700 1 -- 172.21.15.162:0/1456236057 >> 172.21.15.162:0/1456236057 conn(0x7f9f2c05b970 msgr2=0x7f9f2c0b2080 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:48.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.752+0000 7f9f34047700 1 -- 172.21.15.162:0/1456236057 shutdown_connections 2020-11-28T21:54:48.758 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.752+0000 7f9f34047700 1 -- 172.21.15.162:0/1456236057 wait complete. 2020-11-28T21:54:48.766 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-28T21:54:48.767 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:54:48.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.880+0000 7ffb8ce84700 1 Processor -- start 2020-11-28T21:54:48.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.880+0000 7ffb8ce84700 1 -- start start 2020-11-28T21:54:48.886 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.880+0000 7ffb8ce84700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ffb8813cb50 0x7ffb88140180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:48.887 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.880+0000 7ffb8ce84700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffb8805cca0 0x7ffb880b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:48.887 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.880+0000 7ffb8ce84700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ffb880b23a0 0x7ffb880ac350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:48.887 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.880+0000 7ffb8ce84700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb880add60 con 0x7ffb880b23a0 2020-11-28T21:54:48.887 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.880+0000 7ffb8ce84700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb880b1360 con 0x7ffb8813cb50 2020-11-28T21:54:48.888 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.880+0000 7ffb8ce84700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb880ae8c0 con 0x7ffb8805cca0 2020-11-28T21:54:48.888 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.884+0000 7ffb877fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ffb8813cb50 0x7ffb88140180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:48.888 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.884+0000 7ffb877fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ffb8813cb50 0x7ffb88140180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43970/0 (socket says 172.21.15.162:43970) 2020-11-28T21:54:48.888 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.884+0000 7ffb877fe700 1 -- 172.21.15.162:0/3851255229 learned_addr learned my addr 172.21.15.162:0/3851255229 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:48.889 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.884+0000 7ffb86ffd700 1 --2- 172.21.15.162:0/3851255229 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffb8805cca0 0x7ffb880b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:48.889 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.884+0000 7ffb87fff700 1 --2- 172.21.15.162:0/3851255229 >> v2:172.21.15.118:3300/0 conn(0x7ffb880b23a0 0x7ffb880ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:48.889 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.884+0000 7ffb877fe700 1 -- 172.21.15.162:0/3851255229 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffb8805cca0 msgr2=0x7ffb880b1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:48.889 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.884+0000 7ffb877fe700 1 --2- 172.21.15.162:0/3851255229 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffb8805cca0 0x7ffb880b1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:48.889 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.884+0000 7ffb877fe700 1 -- 172.21.15.162:0/3851255229 >> v2:172.21.15.118:3300/0 conn(0x7ffb880b23a0 msgr2=0x7ffb880ac350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:48.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.884+0000 7ffb877fe700 1 --2- 172.21.15.162:0/3851255229 >> v2:172.21.15.118:3300/0 conn(0x7ffb880b23a0 0x7ffb880ac350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:48.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.884+0000 7ffb877fe700 1 -- 172.21.15.162:0/3851255229 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb880aeb60 con 0x7ffb8813cb50 2020-11-28T21:54:48.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.884+0000 7ffb877fe700 1 --2- 172.21.15.162:0/3851255229 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb8813cb50 0x7ffb88140180 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 rx=0x7ffb78002f30 tx=0x7ffb78006e80).ready entity=mon.1 client_cookie=50aee88c53907779 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:48.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.884+0000 7ffb867fc700 1 -- 172.21.15.162:0/3851255229 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb780097f0 con 0x7ffb8813cb50 2020-11-28T21:54:48.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.884+0000 7ffb867fc700 1 -- 172.21.15.162:0/3851255229 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb78009950 con 0x7ffb8813cb50 2020-11-28T21:54:48.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.884+0000 7ffb867fc700 1 -- 172.21.15.162:0/3851255229 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb78009cb0 con 0x7ffb8813cb50 2020-11-28T21:54:48.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.884+0000 7ffb8ce84700 1 -- 172.21.15.162:0/3851255229 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb8813cb50 msgr2=0x7ffb88140180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:48.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.884+0000 7ffb8ce84700 1 --2- 172.21.15.162:0/3851255229 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb8813cb50 0x7ffb88140180 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 rx=0x7ffb78002f30 tx=0x7ffb78006e80).stop 2020-11-28T21:54:48.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.884+0000 7ffb8ce84700 1 -- 172.21.15.162:0/3851255229 shutdown_connections 2020-11-28T21:54:48.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.884+0000 7ffb8ce84700 1 --2- 172.21.15.162:0/3851255229 >> v2:172.21.15.118:3300/0 conn(0x7ffb880b23a0 0x7ffb880ac350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:48.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.884+0000 7ffb8ce84700 1 --2- 172.21.15.162:0/3851255229 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb8813cb50 0x7ffb88140180 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:48.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.884+0000 7ffb8ce84700 1 --2- 172.21.15.162:0/3851255229 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffb8805cca0 0x7ffb880b1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:48.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.884+0000 7ffb8ce84700 1 -- 172.21.15.162:0/3851255229 >> 172.21.15.162:0/3851255229 conn(0x7ffb880b57d0 msgr2=0x7ffb880b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:48.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.884+0000 7ffb8ce84700 1 -- 172.21.15.162:0/3851255229 shutdown_connections 2020-11-28T21:54:48.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.884+0000 7ffb8ce84700 1 -- 172.21.15.162:0/3851255229 wait complete. 2020-11-28T21:54:48.896 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.888+0000 7ffb8ce84700 1 Processor -- start 2020-11-28T21:54:48.896 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.888+0000 7ffb8ce84700 1 -- start start 2020-11-28T21:54:48.896 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.888+0000 7ffb8ce84700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffb8805cca0 0x7ffb88140ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:48.896 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.888+0000 7ffb8ce84700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffb8813cb50 0x7ffb8814ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:48.896 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.888+0000 7ffb8ce84700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb880634b0 0x7ffb88141cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:48.897 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.888+0000 7ffb8ce84700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb88063a30 con 0x7ffb8805cca0 2020-11-28T21:54:48.897 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.888+0000 7ffb8ce84700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb88063cd0 con 0x7ffb880634b0 2020-11-28T21:54:48.897 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.888+0000 7ffb8ce84700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb88063f70 con 0x7ffb8813cb50 2020-11-28T21:54:48.897 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.888+0000 7ffb87fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb880634b0 0x7ffb88141cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:48.897 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.888+0000 7ffb877fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffb8805cca0 0x7ffb88140ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:48.898 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.888+0000 7ffb87fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb880634b0 0x7ffb88141cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43978/0 (socket says 172.21.15.162:43978) 2020-11-28T21:54:48.898 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.888+0000 7ffb87fff700 1 -- 172.21.15.162:0/1830070382 learned_addr learned my addr 172.21.15.162:0/1830070382 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:48.898 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.888+0000 7ffb86ffd700 1 --2- 172.21.15.162:0/1830070382 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffb8813cb50 0x7ffb8814ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:48.898 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.888+0000 7ffb87fff700 1 -- 172.21.15.162:0/1830070382 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffb8813cb50 msgr2=0x7ffb8814ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:48.898 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.888+0000 7ffb87fff700 1 --2- 172.21.15.162:0/1830070382 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffb8813cb50 0x7ffb8814ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:48.899 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.888+0000 7ffb87fff700 1 -- 172.21.15.162:0/1830070382 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffb8805cca0 msgr2=0x7ffb88140ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:48.899 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.888+0000 7ffb87fff700 1 --2- 172.21.15.162:0/1830070382 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffb8805cca0 0x7ffb88140ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:48.899 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.888+0000 7ffb87fff700 1 -- 172.21.15.162:0/1830070382 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb88141820 con 0x7ffb880634b0 2020-11-28T21:54:48.899 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.888+0000 7ffb877fe700 1 --2- 172.21.15.162:0/1830070382 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffb8805cca0 0x7ffb88140ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:54:48.899 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.888+0000 7ffb86ffd700 1 --2- 172.21.15.162:0/1830070382 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffb8813cb50 0x7ffb8814ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:54:48.900 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.888+0000 7ffb87fff700 1 --2- 172.21.15.162:0/1830070382 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb880634b0 0x7ffb88141cc0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 rx=0x7ffb8000a000 tx=0x7ffb80008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:48.900 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.888+0000 7ffb84ff9700 1 -- 172.21.15.162:0/1830070382 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb8000b980 con 0x7ffb880634b0 2020-11-28T21:54:48.900 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.888+0000 7ffb84ff9700 1 -- 172.21.15.162:0/1830070382 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb8000bae0 con 0x7ffb880634b0 2020-11-28T21:54:48.900 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.888+0000 7ffb84ff9700 1 -- 172.21.15.162:0/1830070382 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb8000be40 con 0x7ffb880634b0 2020-11-28T21:54:48.900 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.888+0000 7ffb8ce84700 1 -- 172.21.15.162:0/1830070382 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb8814f480 con 0x7ffb880634b0 2020-11-28T21:54:48.901 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.892+0000 7ffb8ce84700 1 -- 172.21.15.162:0/1830070382 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb881423d0 con 0x7ffb880634b0 2020-11-28T21:54:48.902 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.892+0000 7ffb8ce84700 1 -- 172.21.15.162:0/1830070382 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb881419b0 con 0x7ffb880634b0 2020-11-28T21:54:48.902 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.892+0000 7ffb84ff9700 1 -- 172.21.15.162:0/1830070382 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ffb8001c460 con 0x7ffb880634b0 2020-11-28T21:54:48.902 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.892+0000 7ffb84ff9700 1 --2- 172.21.15.162:0/1830070382 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ffb6c0553b0 0x7ffb6c057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:48.902 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.892+0000 7ffb84ff9700 1 -- 172.21.15.162:0/1830070382 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5253+0+0 (secure 0 0 0) 0x7ffb8006e2a0 con 0x7ffb880634b0 2020-11-28T21:54:48.903 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.892+0000 7ffb877fe700 1 --2- 172.21.15.162:0/1830070382 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ffb6c0553b0 0x7ffb6c057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:48.903 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.892+0000 7ffb877fe700 1 --2- 172.21.15.162:0/1830070382 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ffb6c0553b0 0x7ffb6c057870 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7ffb7801c000 tx=0x7ffb78015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:48.903 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:48.900+0000 7ffb84ff9700 1 -- 172.21.15.162:0/1830070382 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ffb800447d0 con 0x7ffb880634b0 2020-11-28T21:54:49.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.036+0000 7ffb8ce84700 1 -- 172.21.15.162:0/1830070382 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7ffb88057050 con 0x7ffb880634b0 2020-11-28T21:54:49.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.036+0000 7ffb84ff9700 1 -- 172.21.15.162:0/1830070382 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v115) v1 ==== 110+0+10 (secure 0 0 0) 0x7ffb80047df0 con 0x7ffb880634b0 2020-11-28T21:54:49.039 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:54:49.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.036+0000 7ffb8ce84700 1 -- 172.21.15.162:0/1830070382 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ffb6c0553b0 msgr2=0x7ffb6c057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:49.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.036+0000 7ffb8ce84700 1 --2- 172.21.15.162:0/1830070382 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ffb6c0553b0 0x7ffb6c057870 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7ffb7801c000 tx=0x7ffb78015000).stop 2020-11-28T21:54:49.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.036+0000 7ffb8ce84700 1 -- 172.21.15.162:0/1830070382 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb880634b0 msgr2=0x7ffb88141cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:49.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.036+0000 7ffb8ce84700 1 --2- 172.21.15.162:0/1830070382 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb880634b0 0x7ffb88141cc0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 rx=0x7ffb8000a000 tx=0x7ffb80008040).stop 2020-11-28T21:54:49.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.036+0000 7ffb8ce84700 1 -- 172.21.15.162:0/1830070382 shutdown_connections 2020-11-28T21:54:49.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.036+0000 7ffb8ce84700 1 --2- 172.21.15.162:0/1830070382 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ffb6c0553b0 0x7ffb6c057870 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:49.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.036+0000 7ffb8ce84700 1 --2- 172.21.15.162:0/1830070382 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb880634b0 0x7ffb88141cc0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:49.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.036+0000 7ffb8ce84700 1 --2- 172.21.15.162:0/1830070382 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffb8805cca0 0x7ffb88140ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:49.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.036+0000 7ffb8ce84700 1 --2- 172.21.15.162:0/1830070382 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffb8813cb50 0x7ffb8814ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:49.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.036+0000 7ffb8ce84700 1 -- 172.21.15.162:0/1830070382 >> 172.21.15.162:0/1830070382 conn(0x7ffb880b57d0 msgr2=0x7ffb880b42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:49.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.036+0000 7ffb8ce84700 1 -- 172.21.15.162:0/1830070382 shutdown_connections 2020-11-28T21:54:49.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.036+0000 7ffb8ce84700 1 -- 172.21.15.162:0/1830070382 wait complete. 2020-11-28T21:54:49.053 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:54:49.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.172+0000 7f2eb5a91700 1 Processor -- start 2020-11-28T21:54:49.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.172+0000 7f2eb5a91700 1 -- start start 2020-11-28T21:54:49.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.172+0000 7f2eb5a91700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2eb005bcb0 0x7f2eb0061300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:49.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.172+0000 7f2eb5a91700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2eb00b21a0 0x7f2eb00abfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:49.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.172+0000 7f2eb5a91700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2eb005c090 con 0x7f2eb0061a50 2020-11-28T21:54:49.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.172+0000 7f2eb5a91700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eb0143820 con 0x7f2eb00b21a0 2020-11-28T21:54:49.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.172+0000 7f2eb5a91700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb00617f0 con 0x7f2eb005bcb0 2020-11-28T21:54:49.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.176+0000 7f2eb5290700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2eb00b21a0 0x7f2eb00abfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:49.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.176+0000 7f2eaffff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f2eb0061a50 0x7f2eb01427d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:50828/0 (socket says 172.21.15.162:50828) 2020-11-28T21:54:49.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.176+0000 7f2eaffff700 1 -- 172.21.15.162:0/1713090864 learned_addr learned my addr 172.21.15.162:0/1713090864 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:49.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.176+0000 7f2eb4a8f700 1 --2- 172.21.15.162:0/1713090864 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2eb005bcb0 0x7f2eb0061300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:49.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.176+0000 7f2eaf7fe700 1 -- 172.21.15.162:0/1713090864 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3113400406 0 0) 0x7f2eb005c090 con 0x7f2eb0061a50 2020-11-28T21:54:49.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.176+0000 7f2eb5290700 1 -- 172.21.15.162:0/1713090864 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2eb005bcb0 msgr2=0x7f2eb0061300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:49.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.176+0000 7f2eb5290700 1 --2- 172.21.15.162:0/1713090864 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2eb005bcb0 0x7f2eb0061300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:49.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.176+0000 7f2eb5290700 1 -- 172.21.15.162:0/1713090864 >> v1:172.21.15.162:6789/0 conn(0x7f2eb0061a50 legacy=0x7f2eb01427d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:49.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.176+0000 7f2eb5290700 1 -- 172.21.15.162:0/1713090864 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb0061f60 con 0x7f2eb00b21a0 2020-11-28T21:54:49.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.176+0000 7f2eb5290700 1 --2- 172.21.15.162:0/1713090864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2eb00b21a0 0x7f2eb00abfb0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 rx=0x7f2ea80032b0 tx=0x7f2ea80091a0).ready entity=mon.1 client_cookie=25b95b5541518f90 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:49.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.176+0000 7f2eaf7fe700 1 -- 172.21.15.162:0/1713090864 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2ea800ba50 con 0x7f2eb00b21a0 2020-11-28T21:54:49.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.176+0000 7f2eaf7fe700 1 -- 172.21.15.162:0/1713090864 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea800bbb0 con 0x7f2eb00b21a0 2020-11-28T21:54:49.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.176+0000 7f2eaf7fe700 1 -- 172.21.15.162:0/1713090864 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2ea800bd20 con 0x7f2eb00b21a0 2020-11-28T21:54:49.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.176+0000 7f2eb5a91700 1 -- 172.21.15.162:0/1713090864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2eb00b21a0 msgr2=0x7f2eb00abfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:49.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.176+0000 7f2eb5a91700 1 --2- 172.21.15.162:0/1713090864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2eb00b21a0 0x7f2eb00abfb0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 rx=0x7f2ea80032b0 tx=0x7f2ea80091a0).stop 2020-11-28T21:54:49.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.176+0000 7f2eb5a91700 1 -- 172.21.15.162:0/1713090864 shutdown_connections 2020-11-28T21:54:49.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.176+0000 7f2eb5a91700 1 --2- 172.21.15.162:0/1713090864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2eb00b21a0 0x7f2eb00abfb0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:49.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.176+0000 7f2eb5a91700 1 --2- 172.21.15.162:0/1713090864 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2eb005bcb0 0x7f2eb0061300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:49.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.176+0000 7f2eb5a91700 1 -- 172.21.15.162:0/1713090864 >> 172.21.15.162:0/1713090864 conn(0x7f2eb00b2830 msgr2=0x7f2eb00b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:49.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.176+0000 7f2eb5a91700 1 -- 172.21.15.162:0/1713090864 shutdown_connections 2020-11-28T21:54:49.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.176+0000 7f2eb5a91700 1 -- 172.21.15.162:0/1713090864 wait complete. 2020-11-28T21:54:49.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.176+0000 7f2eb5a91700 1 Processor -- start 2020-11-28T21:54:49.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.176+0000 7f2eb5a91700 1 -- start start 2020-11-28T21:54:49.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.180+0000 7f2eb5a91700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2eb005bcb0 0x7f2eb00af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:49.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.180+0000 7f2eb5a91700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2eb0061a50 0x7f2eb01522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:49.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.180+0000 7f2eb5a91700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2eb00b21a0 0x7f2eb01442f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:49.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.180+0000 7f2eb5a91700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb013d000 con 0x7f2eb0061a50 2020-11-28T21:54:49.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.180+0000 7f2eb5a91700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb00ae5b0 con 0x7f2eb005bcb0 2020-11-28T21:54:49.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.180+0000 7f2eb5a91700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb00ae850 con 0x7f2eb00b21a0 2020-11-28T21:54:49.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.180+0000 7f2eb4a8f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2eb005bcb0 0x7f2eb00af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:49.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.180+0000 7f2eb4a8f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2eb005bcb0 0x7f2eb00af4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:43990/0 (socket says 172.21.15.162:43990) 2020-11-28T21:54:49.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.180+0000 7f2eb4a8f700 1 -- 172.21.15.162:0/4241864078 learned_addr learned my addr 172.21.15.162:0/4241864078 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:49.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.180+0000 7f2eb5290700 1 --2- 172.21.15.162:0/4241864078 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2eb00b21a0 0x7f2eb01442f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:49.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.180+0000 7f2eaffff700 1 --2- 172.21.15.162:0/4241864078 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2eb0061a50 0x7f2eb01522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:49.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.180+0000 7f2eb4a8f700 1 -- 172.21.15.162:0/4241864078 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2eb00b21a0 msgr2=0x7f2eb01442f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:49.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.180+0000 7f2eb4a8f700 1 --2- 172.21.15.162:0/4241864078 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2eb00b21a0 0x7f2eb01442f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:49.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.180+0000 7f2eb4a8f700 1 -- 172.21.15.162:0/4241864078 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2eb0061a50 msgr2=0x7f2eb01522b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:49.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.180+0000 7f2eb4a8f700 1 --2- 172.21.15.162:0/4241864078 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2eb0061a50 0x7f2eb01522b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:49.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.180+0000 7f2eb4a8f700 1 -- 172.21.15.162:0/4241864078 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb00aeaf0 con 0x7f2eb005bcb0 2020-11-28T21:54:49.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.180+0000 7f2eb4a8f700 1 --2- 172.21.15.162:0/4241864078 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2eb005bcb0 0x7f2eb00af4d0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 rx=0x7f2ea0003070 tx=0x7f2ea0008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:49.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.180+0000 7f2eadffb700 1 -- 172.21.15.162:0/4241864078 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2ea0009790 con 0x7f2eb005bcb0 2020-11-28T21:54:49.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.180+0000 7f2eadffb700 1 -- 172.21.15.162:0/4241864078 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea00098f0 con 0x7f2eb005bcb0 2020-11-28T21:54:49.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.180+0000 7f2eb5a91700 1 -- 172.21.15.162:0/4241864078 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eb00aed90 con 0x7f2eb005bcb0 2020-11-28T21:54:49.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.180+0000 7f2eadffb700 1 -- 172.21.15.162:0/4241864078 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2ea0009c50 con 0x7f2eb005bcb0 2020-11-28T21:54:49.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.180+0000 7f2eb5a91700 1 -- 172.21.15.162:0/4241864078 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eb0144a00 con 0x7f2eb005bcb0 2020-11-28T21:54:49.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.180+0000 7f2eb5a91700 1 -- 172.21.15.162:0/4241864078 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eb00af1e0 con 0x7f2eb005bcb0 2020-11-28T21:54:49.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.184+0000 7f2eadffb700 1 -- 172.21.15.162:0/4241864078 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2ea0013480 con 0x7f2eb005bcb0 2020-11-28T21:54:49.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.184+0000 7f2eadffb700 1 --2- 172.21.15.162:0/4241864078 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2e940553b0 0x7f2e94057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:49.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.184+0000 7f2eadffb700 1 -- 172.21.15.162:0/4241864078 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f2ea006e0a0 con 0x7f2eb005bcb0 2020-11-28T21:54:49.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.184+0000 7f2eaffff700 1 --2- 172.21.15.162:0/4241864078 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2e940553b0 0x7f2e94057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:49.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.184+0000 7f2eaffff700 1 --2- 172.21.15.162:0/4241864078 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2e940553b0 0x7f2e94057870 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7f2e9800d000 tx=0x7f2e980079f0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:49.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.188+0000 7f2eadffb700 1 -- 172.21.15.162:0/4241864078 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f2ea0044560 con 0x7f2eb005bcb0 2020-11-28T21:54:49.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.328+0000 7f2eb5a91700 1 -- 172.21.15.162:0/4241864078 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2eb00aef20 con 0x7f2eb005bcb0 2020-11-28T21:54:49.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.328+0000 7f2eadffb700 1 -- 172.21.15.162:0/4241864078 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v115) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2ea0009db0 con 0x7f2eb005bcb0 2020-11-28T21:54:49.333 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:54:49.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.332+0000 7f2eb5a91700 1 -- 172.21.15.162:0/4241864078 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2e940553b0 msgr2=0x7f2e94057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:49.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.332+0000 7f2eb5a91700 1 --2- 172.21.15.162:0/4241864078 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2e940553b0 0x7f2e94057870 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7f2e9800d000 tx=0x7f2e980079f0).stop 2020-11-28T21:54:49.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.332+0000 7f2eb5a91700 1 -- 172.21.15.162:0/4241864078 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2eb005bcb0 msgr2=0x7f2eb00af4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:49.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.332+0000 7f2eb5a91700 1 --2- 172.21.15.162:0/4241864078 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2eb005bcb0 0x7f2eb00af4d0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 rx=0x7f2ea0003070 tx=0x7f2ea0008040).stop 2020-11-28T21:54:49.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.332+0000 7f2eb5a91700 1 -- 172.21.15.162:0/4241864078 shutdown_connections 2020-11-28T21:54:49.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.332+0000 7f2eb5a91700 1 --2- 172.21.15.162:0/4241864078 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2e940553b0 0x7f2e94057870 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:49.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.332+0000 7f2eb5a91700 1 --2- 172.21.15.162:0/4241864078 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2eb00b21a0 0x7f2eb01442f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:49.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.332+0000 7f2eb5a91700 1 --2- 172.21.15.162:0/4241864078 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2eb005bcb0 0x7f2eb00af4d0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:49.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.332+0000 7f2eb5a91700 1 --2- 172.21.15.162:0/4241864078 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2eb0061a50 0x7f2eb01522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:49.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.332+0000 7f2eb5a91700 1 -- 172.21.15.162:0/4241864078 >> 172.21.15.162:0/4241864078 conn(0x7f2eb00b2830 msgr2=0x7f2eb00abfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:49.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.332+0000 7f2eb5a91700 1 -- 172.21.15.162:0/4241864078 shutdown_connections 2020-11-28T21:54:49.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.332+0000 7f2eb5a91700 1 -- 172.21.15.162:0/4241864078 wait complete. 2020-11-28T21:54:49.351 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2020-11-28T21:54:49.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.460+0000 7f4086283700 1 Processor -- start 2020-11-28T21:54:49.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.460+0000 7f4086283700 1 -- start start 2020-11-28T21:54:49.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.460+0000 7f4086283700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f408013fd00 0x7f408013c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:49.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.460+0000 7f4086283700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4080067e90 con 0x7f40800b69a0 2020-11-28T21:54:49.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.460+0000 7f4086283700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40800b6d80 con 0x7f408013c900 2020-11-28T21:54:49.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.460+0000 7f4086283700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40800ae270 con 0x7f408013fd00 2020-11-28T21:54:49.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.460+0000 7f4084820700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f408013c900 0x7f4080149f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:50846/0 (socket says 172.21.15.162:50846) 2020-11-28T21:54:49.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.460+0000 7f4084820700 1 -- 172.21.15.162:0/2501780266 learned_addr learned my addr 172.21.15.162:0/2501780266 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:49.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.460+0000 7f407f7fe700 1 --2- 172.21.15.162:0/2501780266 >> v2:172.21.15.118:3300/0 conn(0x7f408013fd00 0x7f408013c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:49.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.460+0000 7f407effd700 1 -- 172.21.15.162:0/2501780266 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3187230878 0 0) 0x7f40800b6d80 con 0x7f408013c900 2020-11-28T21:54:49.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.460+0000 7f407effd700 1 -- 172.21.15.162:0/2501780266 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f406c003230 con 0x7f408013c900 2020-11-28T21:54:49.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.460+0000 7f407effd700 1 -- 172.21.15.162:0/2501780266 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2982100698 0 0) 0x7f406c003230 con 0x7f408013c900 2020-11-28T21:54:49.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.460+0000 7f407effd700 1 -- 172.21.15.162:0/2501780266 >> v2:172.21.15.118:3300/0 conn(0x7f408013fd00 msgr2=0x7f408013c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:49.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.460+0000 7f407effd700 1 --2- 172.21.15.162:0/2501780266 >> v2:172.21.15.118:3300/0 conn(0x7f408013fd00 0x7f408013c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:49.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.460+0000 7f407effd700 1 -- 172.21.15.162:0/2501780266 >> v1:172.21.15.118:6789/0 conn(0x7f40800b69a0 legacy=0x7f408013f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:49.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.460+0000 7f407effd700 1 -- 172.21.15.162:0/2501780266 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4080063600 con 0x7f408013c900 2020-11-28T21:54:49.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.460+0000 7f407effd700 1 -- 172.21.15.162:0/2501780266 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f4078001a50 con 0x7f408013c900 2020-11-28T21:54:49.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.460+0000 7f407effd700 1 -- 172.21.15.162:0/2501780266 >> v1:172.21.15.162:6789/0 conn(0x7f408013c900 legacy=0x7f4080149f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:49.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.460+0000 7f407effd700 1 --2- 172.21.15.162:0/2501780266 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f406c003720 0x7f406c003b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:49.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.460+0000 7f407effd700 1 --2- 172.21.15.162:0/2501780266 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f406c004050 0x7f406c0044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:49.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.460+0000 7f407effd700 1 --2- 172.21.15.162:0/2501780266 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f406c004a60 0x7f406c004ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:49.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.460+0000 7f407effd700 1 -- 172.21.15.162:0/2501780266 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406c00fe00 con 0x7f406c003720 2020-11-28T21:54:49.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.460+0000 7f407effd700 1 -- 172.21.15.162:0/2501780266 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406c00ff70 con 0x7f406c004050 2020-11-28T21:54:49.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.460+0000 7f407effd700 1 -- 172.21.15.162:0/2501780266 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f406c010130 con 0x7f406c004a60 2020-11-28T21:54:49.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.460+0000 7f407f7fe700 1 --2- 172.21.15.162:0/2501780266 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f406c004050 0x7f406c0044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:49.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.460+0000 7f4084820700 1 --2- 172.21.15.162:0/2501780266 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f406c004a60 0x7f406c004ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:49.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.460+0000 7f407ffff700 1 --2- 172.21.15.162:0/2501780266 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f406c003720 0x7f406c003b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:49.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.460+0000 7f407f7fe700 1 -- 172.21.15.162:0/2501780266 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f406c004a60 msgr2=0x7f406c004ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:49.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.460+0000 7f407f7fe700 1 --2- 172.21.15.162:0/2501780266 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f406c004a60 0x7f406c004ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:49.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.460+0000 7f407f7fe700 1 -- 172.21.15.162:0/2501780266 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f406c003720 msgr2=0x7f406c003b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:49.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.460+0000 7f407f7fe700 1 --2- 172.21.15.162:0/2501780266 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f406c003720 0x7f406c003b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:49.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.460+0000 7f407f7fe700 1 -- 172.21.15.162:0/2501780266 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f406c003230 con 0x7f406c004050 2020-11-28T21:54:49.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.460+0000 7f407ffff700 1 --2- 172.21.15.162:0/2501780266 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f406c003720 0x7f406c003b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:54:49.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.460+0000 7f4084820700 1 --2- 172.21.15.162:0/2501780266 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f406c004a60 0x7f406c004ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:54:49.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.460+0000 7f4084820700 1 -- 172.21.15.162:0/2501780266 reap_dead start 2020-11-28T21:54:49.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.460+0000 7f407f7fe700 1 --2- 172.21.15.162:0/2501780266 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f406c004050 0x7f406c0044d0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 rx=0x7f407400c000 tx=0x7f4074008040).ready entity=mon.1 client_cookie=f4159e8a9e64800a server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:49.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.460+0000 7f407effd700 1 -- 172.21.15.162:0/2501780266 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f407400e9d0 con 0x7f406c004050 2020-11-28T21:54:49.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.464+0000 7f407effd700 1 -- 172.21.15.162:0/2501780266 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f407400eb30 con 0x7f406c004050 2020-11-28T21:54:49.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.464+0000 7f407effd700 1 -- 172.21.15.162:0/2501780266 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f407400ee90 con 0x7f406c004050 2020-11-28T21:54:49.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.464+0000 7f4086283700 1 -- 172.21.15.162:0/2501780266 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f406c004050 msgr2=0x7f406c0044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:49.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.464+0000 7f4086283700 1 --2- 172.21.15.162:0/2501780266 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f406c004050 0x7f406c0044d0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 rx=0x7f407400c000 tx=0x7f4074008040).stop 2020-11-28T21:54:49.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.464+0000 7f4086283700 1 -- 172.21.15.162:0/2501780266 shutdown_connections 2020-11-28T21:54:49.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.464+0000 7f4086283700 1 --2- 172.21.15.162:0/2501780266 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f406c004050 0x7f406c0044d0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:49.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.464+0000 7f4086283700 1 -- 172.21.15.162:0/2501780266 >> 172.21.15.162:0/2501780266 conn(0x7f4080062170 msgr2=0x7f40800b5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:49.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.464+0000 7f4086283700 1 -- 172.21.15.162:0/2501780266 shutdown_connections 2020-11-28T21:54:49.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.464+0000 7f4086283700 1 -- 172.21.15.162:0/2501780266 wait complete. 2020-11-28T21:54:49.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.464+0000 7f4086283700 1 Processor -- start 2020-11-28T21:54:49.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.464+0000 7f4086283700 1 -- start start 2020-11-28T21:54:49.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.464+0000 7f4086283700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f406c004050 0x7f40800b2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:49.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.464+0000 7f4086283700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f408014d3b0 0x7f4080064a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:49.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.464+0000 7f4086283700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f408013ea80 0x7f4080064f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:49.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.464+0000 7f4086283700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408013f1c0 con 0x7f408014d3b0 2020-11-28T21:54:49.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.464+0000 7f4086283700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40800645f0 con 0x7f406c004050 2020-11-28T21:54:49.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.464+0000 7f4086283700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40800656d0 con 0x7f408013ea80 2020-11-28T21:54:49.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.464+0000 7f407ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f406c004050 0x7f40800b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:49.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.464+0000 7f407ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f406c004050 0x7f40800b2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44010/0 (socket says 172.21.15.162:44010) 2020-11-28T21:54:49.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.464+0000 7f407ffff700 1 -- 172.21.15.162:0/2949851023 learned_addr learned my addr 172.21.15.162:0/2949851023 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:49.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.468+0000 7f407f7fe700 1 --2- 172.21.15.162:0/2949851023 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f408014d3b0 0x7f4080064a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:49.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.468+0000 7f4084820700 1 --2- 172.21.15.162:0/2949851023 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f408013ea80 0x7f4080064f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:49.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.468+0000 7f407ffff700 1 -- 172.21.15.162:0/2949851023 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f408013ea80 msgr2=0x7f4080064f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:49.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.468+0000 7f407ffff700 1 --2- 172.21.15.162:0/2949851023 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f408013ea80 0x7f4080064f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:49.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.468+0000 7f407ffff700 1 -- 172.21.15.162:0/2949851023 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f408014d3b0 msgr2=0x7f4080064a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:49.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.468+0000 7f407ffff700 1 --2- 172.21.15.162:0/2949851023 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f408014d3b0 0x7f4080064a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:49.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.468+0000 7f407ffff700 1 -- 172.21.15.162:0/2949851023 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f408013f3f0 con 0x7f406c004050 2020-11-28T21:54:49.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.468+0000 7f407ffff700 1 --2- 172.21.15.162:0/2949851023 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f406c004050 0x7f40800b2050 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 rx=0x7f4080066bc0 tx=0x7f4070009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:49.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.468+0000 7f407d7fa700 1 -- 172.21.15.162:0/2949851023 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f407000c770 con 0x7f406c004050 2020-11-28T21:54:49.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.468+0000 7f407d7fa700 1 -- 172.21.15.162:0/2949851023 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f407000c8d0 con 0x7f406c004050 2020-11-28T21:54:49.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.468+0000 7f4086283700 1 -- 172.21.15.162:0/2949851023 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4080067680 con 0x7f406c004050 2020-11-28T21:54:49.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.468+0000 7f407d7fa700 1 -- 172.21.15.162:0/2949851023 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f407000cc30 con 0x7f406c004050 2020-11-28T21:54:49.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.468+0000 7f4086283700 1 -- 172.21.15.162:0/2949851023 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f408013e8c0 con 0x7f406c004050 2020-11-28T21:54:49.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.468+0000 7f4086283700 1 -- 172.21.15.162:0/2949851023 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4080064760 con 0x7f406c004050 2020-11-28T21:54:49.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.468+0000 7f407d7fa700 1 -- 172.21.15.162:0/2949851023 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4070014430 con 0x7f406c004050 2020-11-28T21:54:49.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.472+0000 7f407d7fa700 1 --2- 172.21.15.162:0/2949851023 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4060055330 0x7f40600577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:49.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.472+0000 7f407d7fa700 1 -- 172.21.15.162:0/2949851023 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f407006daf0 con 0x7f406c004050 2020-11-28T21:54:49.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.472+0000 7f407f7fe700 1 --2- 172.21.15.162:0/2949851023 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4060055330 0x7f40600577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:49.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.472+0000 7f407f7fe700 1 --2- 172.21.15.162:0/2949851023 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4060055330 0x7f40600577f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f4074008000 tx=0x7f407401a000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:49.485 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.476+0000 7f407d7fa700 1 -- 172.21.15.162:0/2949851023 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4070043fb0 con 0x7f406c004050 2020-11-28T21:54:49.622 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.620+0000 7f4086283700 1 -- 172.21.15.162:0/2949851023 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f408013e8c0 con 0x7f406c004050 2020-11-28T21:54:49.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.620+0000 7f407d7fa700 1 -- 172.21.15.162:0/2949851023 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v115) v1 ==== 84+0+143 (secure 0 0 0) 0x7f40700475d0 con 0x7f406c004050 2020-11-28T21:54:49.623 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:54:49.624 INFO:teuthology.orchestra.run.smithi162.stdout:[ 2020-11-28T21:54:49.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:49.624 INFO:teuthology.orchestra.run.smithi162.stdout: "poolnum": 1, 2020-11-28T21:54:49.624 INFO:teuthology.orchestra.run.smithi162.stdout: "poolname": "device_health_metrics" 2020-11-28T21:54:49.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:54:49.625 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:54:49.625 INFO:teuthology.orchestra.run.smithi162.stdout: "poolnum": 2, 2020-11-28T21:54:49.625 INFO:teuthology.orchestra.run.smithi162.stdout: "poolname": "rbd" 2020-11-28T21:54:49.625 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:54:49.625 INFO:teuthology.orchestra.run.smithi162.stdout:] 2020-11-28T21:54:49.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.620+0000 7f4086283700 1 -- 172.21.15.162:0/2949851023 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4060055330 msgr2=0x7f40600577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:49.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.620+0000 7f4086283700 1 --2- 172.21.15.162:0/2949851023 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4060055330 0x7f40600577f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f4074008000 tx=0x7f407401a000).stop 2020-11-28T21:54:49.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.620+0000 7f4086283700 1 -- 172.21.15.162:0/2949851023 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f406c004050 msgr2=0x7f40800b2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:49.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.620+0000 7f4086283700 1 --2- 172.21.15.162:0/2949851023 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f406c004050 0x7f40800b2050 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 rx=0x7f4080066bc0 tx=0x7f4070009040).stop 2020-11-28T21:54:49.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.620+0000 7f4086283700 1 -- 172.21.15.162:0/2949851023 shutdown_connections 2020-11-28T21:54:49.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.620+0000 7f4086283700 1 --2- 172.21.15.162:0/2949851023 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4060055330 0x7f40600577f0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:49.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.620+0000 7f4086283700 1 --2- 172.21.15.162:0/2949851023 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f408013ea80 0x7f4080064f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:49.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.620+0000 7f4086283700 1 --2- 172.21.15.162:0/2949851023 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f406c004050 0x7f40800b2050 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:49.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.620+0000 7f4086283700 1 --2- 172.21.15.162:0/2949851023 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f408014d3b0 0x7f4080064a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:49.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.620+0000 7f4086283700 1 -- 172.21.15.162:0/2949851023 >> 172.21.15.162:0/2949851023 conn(0x7f4080062170 msgr2=0x7f40800b3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:49.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.620+0000 7f4086283700 1 -- 172.21.15.162:0/2949851023 shutdown_connections 2020-11-28T21:54:49.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.620+0000 7f4086283700 1 -- 172.21.15.162:0/2949851023 wait complete. 2020-11-28T21:54:49.637 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-11-28T21:54:49.638 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2020-11-28T21:54:49.753 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.748+0000 7fd524865700 1 Processor -- start 2020-11-28T21:54:49.753 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.748+0000 7fd524865700 1 -- start start 2020-11-28T21:54:49.756 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.748+0000 7fd524865700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd52005b970 0x7fd5201405b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:49.756 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.752+0000 7fd524865700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd52005bd50 0x7fd520141010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:49.756 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.752+0000 7fd524865700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd5200b1be0 0x7fd5200b2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:49.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.752+0000 7fd524865700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5200b2560 con 0x7fd52005bd50 2020-11-28T21:54:49.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.752+0000 7fd524865700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5200613d0 con 0x7fd52005b970 2020-11-28T21:54:49.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.752+0000 7fd524865700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd520061670 con 0x7fd5200b1be0 2020-11-28T21:54:49.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.752+0000 7fd51f7fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd52005b970 0x7fd5201405b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:49.758 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.752+0000 7fd51f7fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd52005b970 0x7fd5201405b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44020/0 (socket says 172.21.15.162:44020) 2020-11-28T21:54:49.758 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.752+0000 7fd51f7fe700 1 -- 172.21.15.162:0/2538967311 learned_addr learned my addr 172.21.15.162:0/2538967311 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:49.758 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.752+0000 7fd51ffff700 1 --2- 172.21.15.162:0/2538967311 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd5200b1be0 0x7fd5200b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:49.758 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.752+0000 7fd51effd700 1 --2- 172.21.15.162:0/2538967311 >> v2:172.21.15.118:3300/0 conn(0x7fd52005bd50 0x7fd520141010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:49.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.752+0000 7fd51f7fe700 1 -- 172.21.15.162:0/2538967311 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd5200b1be0 msgr2=0x7fd5200b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:49.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.752+0000 7fd51f7fe700 1 --2- 172.21.15.162:0/2538967311 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd5200b1be0 0x7fd5200b2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:49.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.752+0000 7fd51f7fe700 1 -- 172.21.15.162:0/2538967311 >> v2:172.21.15.118:3300/0 conn(0x7fd52005bd50 msgr2=0x7fd520141010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:49.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.752+0000 7fd51f7fe700 1 --2- 172.21.15.162:0/2538967311 >> v2:172.21.15.118:3300/0 conn(0x7fd52005bd50 0x7fd520141010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:49.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.752+0000 7fd51f7fe700 1 -- 172.21.15.162:0/2538967311 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd520061910 con 0x7fd52005b970 2020-11-28T21:54:49.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.752+0000 7fd51f7fe700 1 --2- 172.21.15.162:0/2538967311 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd52005b970 0x7fd5201405b0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 rx=0x7fd508002f30 tx=0x7fd508006e80).ready entity=mon.1 client_cookie=8a9077eb7b228640 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:49.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.752+0000 7fd51e7fc700 1 -- 172.21.15.162:0/2538967311 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd508009640 con 0x7fd52005b970 2020-11-28T21:54:49.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.752+0000 7fd51e7fc700 1 -- 172.21.15.162:0/2538967311 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5080097a0 con 0x7fd52005b970 2020-11-28T21:54:49.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.752+0000 7fd51e7fc700 1 -- 172.21.15.162:0/2538967311 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd508009b00 con 0x7fd52005b970 2020-11-28T21:54:49.761 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.752+0000 7fd524865700 1 -- 172.21.15.162:0/2538967311 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd52005b970 msgr2=0x7fd5201405b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:49.761 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.752+0000 7fd524865700 1 --2- 172.21.15.162:0/2538967311 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd52005b970 0x7fd5201405b0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 rx=0x7fd508002f30 tx=0x7fd508006e80).stop 2020-11-28T21:54:49.761 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.752+0000 7fd524865700 1 -- 172.21.15.162:0/2538967311 shutdown_connections 2020-11-28T21:54:49.761 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.752+0000 7fd524865700 1 --2- 172.21.15.162:0/2538967311 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd5200b1be0 0x7fd5200b2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:49.761 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.752+0000 7fd524865700 1 --2- 172.21.15.162:0/2538967311 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd52005b970 0x7fd5201405b0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:49.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.752+0000 7fd524865700 1 --2- 172.21.15.162:0/2538967311 >> v2:172.21.15.118:3300/0 conn(0x7fd52005bd50 0x7fd520141010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:49.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.752+0000 7fd524865700 1 -- 172.21.15.162:0/2538967311 >> 172.21.15.162:0/2538967311 conn(0x7fd5200b2830 msgr2=0x7fd5200b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:49.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.752+0000 7fd524865700 1 -- 172.21.15.162:0/2538967311 shutdown_connections 2020-11-28T21:54:49.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.752+0000 7fd524865700 1 -- 172.21.15.162:0/2538967311 wait complete. 2020-11-28T21:54:49.765 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.752+0000 7fd524865700 1 Processor -- start 2020-11-28T21:54:49.765 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.752+0000 7fd524865700 1 -- start start 2020-11-28T21:54:49.766 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.752+0000 7fd524865700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd52005b970 0x7fd5200b5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:49.766 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.752+0000 7fd524865700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd52005bd50 0x7fd5201381a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:49.766 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.752+0000 7fd524865700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5200b1be0 0x7fd520138690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:49.766 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.752+0000 7fd524865700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5200abbb0 con 0x7fd52005bd50 2020-11-28T21:54:49.767 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.752+0000 7fd524865700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5200ac500 con 0x7fd5200b1be0 2020-11-28T21:54:49.767 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.752+0000 7fd524865700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5200ac7a0 con 0x7fd52005b970 2020-11-28T21:54:49.767 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.752+0000 7fd51ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5200b1be0 0x7fd520138690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:49.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.752+0000 7fd51ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5200b1be0 0x7fd520138690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44028/0 (socket says 172.21.15.162:44028) 2020-11-28T21:54:49.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.752+0000 7fd51ffff700 1 -- 172.21.15.162:0/3825902913 learned_addr learned my addr 172.21.15.162:0/3825902913 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:49.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.752+0000 7fd51f7fe700 1 --2- 172.21.15.162:0/3825902913 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd52005b970 0x7fd5200b5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:49.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.756+0000 7fd51effd700 1 --2- 172.21.15.162:0/3825902913 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd52005bd50 0x7fd5201381a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:49.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.756+0000 7fd51ffff700 1 -- 172.21.15.162:0/3825902913 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd52005b970 msgr2=0x7fd5200b5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:49.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.756+0000 7fd51ffff700 1 --2- 172.21.15.162:0/3825902913 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd52005b970 0x7fd5200b5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:49.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.756+0000 7fd51ffff700 1 -- 172.21.15.162:0/3825902913 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd52005bd50 msgr2=0x7fd5201381a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:49.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.756+0000 7fd51ffff700 1 --2- 172.21.15.162:0/3825902913 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd52005bd50 0x7fd5201381a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:49.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.756+0000 7fd51ffff700 1 -- 172.21.15.162:0/3825902913 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5201422e0 con 0x7fd5200b1be0 2020-11-28T21:54:49.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.756+0000 7fd51effd700 1 --2- 172.21.15.162:0/3825902913 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd52005bd50 0x7fd5201381a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:54:49.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.756+0000 7fd51ffff700 1 --2- 172.21.15.162:0/3825902913 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5200b1be0 0x7fd520138690 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 rx=0x7fd518006dd0 tx=0x7fd5180091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:49.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.756+0000 7fd51cff9700 1 -- 172.21.15.162:0/3825902913 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd51800c950 con 0x7fd5200b1be0 2020-11-28T21:54:49.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.756+0000 7fd51cff9700 1 -- 172.21.15.162:0/3825902913 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd51800cab0 con 0x7fd5200b1be0 2020-11-28T21:54:49.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.756+0000 7fd524865700 1 -- 172.21.15.162:0/3825902913 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5200acc40 con 0x7fd5200b1be0 2020-11-28T21:54:49.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.756+0000 7fd51cff9700 1 -- 172.21.15.162:0/3825902913 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd51800ce10 con 0x7fd5200b1be0 2020-11-28T21:54:49.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.756+0000 7fd524865700 1 -- 172.21.15.162:0/3825902913 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5200ab840 con 0x7fd5200b1be0 2020-11-28T21:54:49.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.756+0000 7fd524865700 1 -- 172.21.15.162:0/3825902913 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5200ac960 con 0x7fd5200b1be0 2020-11-28T21:54:49.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.756+0000 7fd51cff9700 1 -- 172.21.15.162:0/3825902913 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd518022070 con 0x7fd5200b1be0 2020-11-28T21:54:49.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.756+0000 7fd51cff9700 1 --2- 172.21.15.162:0/3825902913 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd5040553b0 0x7fd504057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:49.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.756+0000 7fd51f7fe700 1 --2- 172.21.15.162:0/3825902913 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd5040553b0 0x7fd504057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:49.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.756+0000 7fd51cff9700 1 -- 172.21.15.162:0/3825902913 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fd51806e1c0 con 0x7fd5200b1be0 2020-11-28T21:54:49.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.760+0000 7fd51f7fe700 1 --2- 172.21.15.162:0/3825902913 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd5040553b0 0x7fd504057870 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7fd50801b000 tx=0x7fd508015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:49.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.760+0000 7fd51cff9700 1 -- 172.21.15.162:0/3825902913 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd5180447f0 con 0x7fd5200b1be0 2020-11-28T21:54:49.903 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:49.900+0000 7fd524865700 1 -- 172.21.15.162:0/3825902913 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fd5200abd20 con 0x7fd5200b1be0 2020-11-28T21:54:50.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.016+0000 7fd51cff9700 1 -- 172.21.15.162:0/3825902913 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v116) v1 ==== 133+0+0 (secure 0 0 0) 0x7fd518016020 con 0x7fd5200b1be0 2020-11-28T21:54:50.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.020+0000 7fd524865700 1 -- 172.21.15.162:0/3825902913 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd5040553b0 msgr2=0x7fd504057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:50.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.020+0000 7fd524865700 1 --2- 172.21.15.162:0/3825902913 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd5040553b0 0x7fd504057870 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7fd50801b000 tx=0x7fd508015000).stop 2020-11-28T21:54:50.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.020+0000 7fd524865700 1 -- 172.21.15.162:0/3825902913 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5200b1be0 msgr2=0x7fd520138690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:50.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.020+0000 7fd524865700 1 --2- 172.21.15.162:0/3825902913 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5200b1be0 0x7fd520138690 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 rx=0x7fd518006dd0 tx=0x7fd5180091a0).stop 2020-11-28T21:54:50.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.020+0000 7fd524865700 1 -- 172.21.15.162:0/3825902913 shutdown_connections 2020-11-28T21:54:50.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.020+0000 7fd524865700 1 --2- 172.21.15.162:0/3825902913 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd5040553b0 0x7fd504057870 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:50.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.020+0000 7fd524865700 1 --2- 172.21.15.162:0/3825902913 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd52005b970 0x7fd5200b5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:50.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.020+0000 7fd524865700 1 --2- 172.21.15.162:0/3825902913 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5200b1be0 0x7fd520138690 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:50.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.020+0000 7fd524865700 1 --2- 172.21.15.162:0/3825902913 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd52005bd50 0x7fd5201381a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:50.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.020+0000 7fd524865700 1 -- 172.21.15.162:0/3825902913 >> 172.21.15.162:0/3825902913 conn(0x7fd5200b2830 msgr2=0x7fd5200b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:50.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.020+0000 7fd524865700 1 -- 172.21.15.162:0/3825902913 shutdown_connections 2020-11-28T21:54:50.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.020+0000 7fd524865700 1 -- 172.21.15.162:0/3825902913 wait complete. 2020-11-28T21:54:50.030 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_metadata' created 2020-11-28T21:54:50.039 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2020-11-28T21:54:50.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.152+0000 7f931e936700 1 Processor -- start 2020-11-28T21:54:50.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.152+0000 7f931e936700 1 -- start start 2020-11-28T21:54:50.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.152+0000 7f931e936700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f931809bcd0 0x7f9318060920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:50.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.152+0000 7f931e936700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f93180b2180 0x7f93180611d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:50.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.152+0000 7f931e936700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9318146d00 con 0x7f931805bb10 2020-11-28T21:54:50.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.152+0000 7f931e936700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93180ac2c0 con 0x7f931809bcd0 2020-11-28T21:54:50.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.152+0000 7f931e936700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93180ac020 con 0x7f93180b2180 2020-11-28T21:54:50.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.152+0000 7f931e135700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f93180b2180 0x7f93180611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:50.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.152+0000 7f931e135700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f93180b2180 0x7f93180611d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44036/0 (socket says 172.21.15.162:44036) 2020-11-28T21:54:50.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.152+0000 7f931e135700 1 -- 172.21.15.162:0/1637555911 learned_addr learned my addr 172.21.15.162:0/1637555911 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:50.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.152+0000 7f931e135700 1 -- 172.21.15.162:0/1637555911 >> v2:172.21.15.118:3300/0 conn(0x7f931809bcd0 msgr2=0x7f9318060920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:50.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.152+0000 7f931d133700 1 --2- 172.21.15.162:0/1637555911 >> v2:172.21.15.118:3300/0 conn(0x7f931809bcd0 0x7f9318060920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:50.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.152+0000 7f931e135700 1 --2- 172.21.15.162:0/1637555911 >> v2:172.21.15.118:3300/0 conn(0x7f931809bcd0 0x7f9318060920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:50.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.152+0000 7f931e135700 1 -- 172.21.15.162:0/1637555911 >> v1:172.21.15.118:6789/0 conn(0x7f931805bb10 legacy=0x7f9318147400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:50.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.152+0000 7f931e135700 1 -- 172.21.15.162:0/1637555911 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9318146f60 con 0x7f93180b2180 2020-11-28T21:54:50.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.152+0000 7f931d133700 1 --2- 172.21.15.162:0/1637555911 >> v2:172.21.15.118:3300/0 conn(0x7f931809bcd0 0x7f9318060920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:54:50.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.152+0000 7f931e135700 1 --2- 172.21.15.162:0/1637555911 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93180b2180 0x7f93180611d0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 rx=0x7f93180ad0c0 tx=0x7f9310006e80).ready entity=mon.1 client_cookie=d699ba00b2a1b3e8 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:50.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.152+0000 7f931c932700 1 -- 172.21.15.162:0/1637555911 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93100097b0 con 0x7f93180b2180 2020-11-28T21:54:50.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.152+0000 7f931c932700 1 -- 172.21.15.162:0/1637555911 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9310009910 con 0x7f93180b2180 2020-11-28T21:54:50.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.152+0000 7f931c932700 1 -- 172.21.15.162:0/1637555911 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9310009a80 con 0x7f93180b2180 2020-11-28T21:54:50.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.152+0000 7f931e936700 1 -- 172.21.15.162:0/1637555911 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93180b2180 msgr2=0x7f93180611d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:50.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.152+0000 7f931e936700 1 --2- 172.21.15.162:0/1637555911 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93180b2180 0x7f93180611d0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 rx=0x7f93180ad0c0 tx=0x7f9310006e80).stop 2020-11-28T21:54:50.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.152+0000 7f931e936700 1 -- 172.21.15.162:0/1637555911 shutdown_connections 2020-11-28T21:54:50.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.152+0000 7f931e936700 1 --2- 172.21.15.162:0/1637555911 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93180b2180 0x7f93180611d0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:50.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.152+0000 7f931e936700 1 --2- 172.21.15.162:0/1637555911 >> v2:172.21.15.118:3300/0 conn(0x7f931809bcd0 0x7f9318060920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:50.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.152+0000 7f931e936700 1 -- 172.21.15.162:0/1637555911 >> 172.21.15.162:0/1637555911 conn(0x7f93180b2830 msgr2=0x7f93180b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:50.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.152+0000 7f931e936700 1 -- 172.21.15.162:0/1637555911 shutdown_connections 2020-11-28T21:54:50.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.152+0000 7f931e936700 1 -- 172.21.15.162:0/1637555911 wait complete. 2020-11-28T21:54:50.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.156+0000 7f931e936700 1 Processor -- start 2020-11-28T21:54:50.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.156+0000 7f931e936700 1 -- start start 2020-11-28T21:54:50.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.156+0000 7f931e936700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f931805bb10 0x7f93180aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:50.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.156+0000 7f931e936700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f931809bcd0 0x7f9318142e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:50.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.156+0000 7f931e936700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f93180b2180 0x7f9318159770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:50.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.156+0000 7f931e936700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93180ae8f0 con 0x7f931809bcd0 2020-11-28T21:54:50.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.156+0000 7f931e936700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93180ae110 con 0x7f931805bb10 2020-11-28T21:54:50.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.156+0000 7f931e936700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93180ae3b0 con 0x7f93180b2180 2020-11-28T21:54:50.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.156+0000 7f931d934700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f931805bb10 0x7f93180aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:50.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.156+0000 7f931d934700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f931805bb10 0x7f93180aeaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44038/0 (socket says 172.21.15.162:44038) 2020-11-28T21:54:50.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.156+0000 7f931d934700 1 -- 172.21.15.162:0/927450116 learned_addr learned my addr 172.21.15.162:0/927450116 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:50.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.156+0000 7f931e135700 1 --2- 172.21.15.162:0/927450116 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f93180b2180 0x7f9318159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:50.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.156+0000 7f931d133700 1 --2- 172.21.15.162:0/927450116 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f931809bcd0 0x7f9318142e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:50.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.156+0000 7f931d934700 1 -- 172.21.15.162:0/927450116 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f93180b2180 msgr2=0x7f9318159770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:50.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.156+0000 7f931d934700 1 --2- 172.21.15.162:0/927450116 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f93180b2180 0x7f9318159770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:50.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.156+0000 7f931d934700 1 -- 172.21.15.162:0/927450116 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f931809bcd0 msgr2=0x7f9318142e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:50.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.156+0000 7f931d934700 1 --2- 172.21.15.162:0/927450116 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f931809bcd0 0x7f9318142e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:50.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.156+0000 7f931d934700 1 -- 172.21.15.162:0/927450116 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93180ae650 con 0x7f931805bb10 2020-11-28T21:54:50.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.156+0000 7f931d934700 1 --2- 172.21.15.162:0/927450116 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f931805bb10 0x7f93180aeaa0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 rx=0x7f93180b00f0 tx=0x7f930c00cdf0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:50.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.156+0000 7f9316ffd700 1 -- 172.21.15.162:0/927450116 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f930c0040b0 con 0x7f931805bb10 2020-11-28T21:54:50.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.156+0000 7f9316ffd700 1 -- 172.21.15.162:0/927450116 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f930c00a7b0 con 0x7f931805bb10 2020-11-28T21:54:50.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.156+0000 7f931e936700 1 -- 172.21.15.162:0/927450116 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9318140aa0 con 0x7f931805bb10 2020-11-28T21:54:50.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.156+0000 7f9316ffd700 1 -- 172.21.15.162:0/927450116 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f930c00ab10 con 0x7f931805bb10 2020-11-28T21:54:50.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.156+0000 7f931e936700 1 -- 172.21.15.162:0/927450116 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9318143300 con 0x7f931805bb10 2020-11-28T21:54:50.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.156+0000 7f931e936700 1 -- 172.21.15.162:0/927450116 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93181435e0 con 0x7f931805bb10 2020-11-28T21:54:50.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.160+0000 7f9316ffd700 1 -- 172.21.15.162:0/927450116 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f930c019030 con 0x7f931805bb10 2020-11-28T21:54:50.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.160+0000 7f9316ffd700 1 --2- 172.21.15.162:0/927450116 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f93000554e0 0x7f93000579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:50.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.160+0000 7f9316ffd700 1 -- 172.21.15.162:0/927450116 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f930c06d440 con 0x7f931805bb10 2020-11-28T21:54:50.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.160+0000 7f931d133700 1 --2- 172.21.15.162:0/927450116 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f93000554e0 0x7f93000579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:50.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.160+0000 7f931d133700 1 --2- 172.21.15.162:0/927450116 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f93000554e0 0x7f93000579a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f930400d000 tx=0x7f9304006f90).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:50.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.164+0000 7f9316ffd700 1 -- 172.21.15.162:0/927450116 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f930c041df0 con 0x7f931805bb10 2020-11-28T21:54:50.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:50.304+0000 7f931e936700 1 -- 172.21.15.162:0/927450116 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f9318057050 con 0x7f931805bb10 2020-11-28T21:54:51.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.016+0000 7f9316ffd700 1 -- 172.21.15.162:0/927450116 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v117) v1 ==== 125+0+0 (secure 0 0 0) 0x7f930c0439c0 con 0x7f931805bb10 2020-11-28T21:54:51.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.024+0000 7f931e936700 1 -- 172.21.15.162:0/927450116 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f93000554e0 msgr2=0x7f93000579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:51.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.024+0000 7f931e936700 1 --2- 172.21.15.162:0/927450116 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f93000554e0 0x7f93000579a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f930400d000 tx=0x7f9304006f90).stop 2020-11-28T21:54:51.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.024+0000 7f931e936700 1 -- 172.21.15.162:0/927450116 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f931805bb10 msgr2=0x7f93180aeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:51.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.024+0000 7f931e936700 1 --2- 172.21.15.162:0/927450116 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f931805bb10 0x7f93180aeaa0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 rx=0x7f93180b00f0 tx=0x7f930c00cdf0).stop 2020-11-28T21:54:51.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.024+0000 7f931e936700 1 -- 172.21.15.162:0/927450116 shutdown_connections 2020-11-28T21:54:51.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.024+0000 7f931e936700 1 --2- 172.21.15.162:0/927450116 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f93000554e0 0x7f93000579a0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:51.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.024+0000 7f931e936700 1 --2- 172.21.15.162:0/927450116 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f93180b2180 0x7f9318159770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:51.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.024+0000 7f931e936700 1 --2- 172.21.15.162:0/927450116 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f931805bb10 0x7f93180aeaa0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:51.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.024+0000 7f931e936700 1 --2- 172.21.15.162:0/927450116 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f931809bcd0 0x7f9318142e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:51.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.024+0000 7f931e936700 1 -- 172.21.15.162:0/927450116 >> 172.21.15.162:0/927450116 conn(0x7f93180b2830 msgr2=0x7f93180b1520 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:51.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.024+0000 7f931e936700 1 -- 172.21.15.162:0/927450116 shutdown_connections 2020-11-28T21:54:51.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.024+0000 7f931e936700 1 -- 172.21.15.162:0/927450116 wait complete. 2020-11-28T21:54:51.030 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_data' created 2020-11-28T21:54:51.042 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2020-11-28T21:54:51.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.160+0000 7fa5f68c0700 1 Processor -- start 2020-11-28T21:54:51.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.160+0000 7fa5f68c0700 1 -- start start 2020-11-28T21:54:51.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.160+0000 7fa5f68c0700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fa5f00b2d00 0x7fa5f005c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:51.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.160+0000 7fa5f68c0700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa5f00b3ab0 0x7fa5f00b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:51.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.160+0000 7fa5f68c0700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa5f0142ad0 0x7fa5f0142f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:51.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.160+0000 7fa5f68c0700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5f00b0e20 con 0x7fa5f00b2d00 2020-11-28T21:54:51.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.160+0000 7fa5f68c0700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5f005baf0 con 0x7fa5f0142ad0 2020-11-28T21:54:51.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.160+0000 7fa5f68c0700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f005bd90 con 0x7fa5f00b3ab0 2020-11-28T21:54:51.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.164+0000 7fa5f60bf700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa5f0142ad0 0x7fa5f0142f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:51.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.164+0000 7fa5f60bf700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa5f0142ad0 0x7fa5f0142f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44050/0 (socket says 172.21.15.162:44050) 2020-11-28T21:54:51.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.164+0000 7fa5f60bf700 1 -- 172.21.15.162:0/2942758832 learned_addr learned my addr 172.21.15.162:0/2942758832 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:51.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.164+0000 7fa5f50bd700 1 --2- 172.21.15.162:0/2942758832 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa5f00b3ab0 0x7fa5f00b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:51.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.164+0000 7fa5f58be700 1 --2- 172.21.15.162:0/2942758832 >> v2:172.21.15.118:3300/0 conn(0x7fa5f00b2d00 0x7fa5f005c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:51.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.164+0000 7fa5f60bf700 1 -- 172.21.15.162:0/2942758832 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa5f00b3ab0 msgr2=0x7fa5f00b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:51.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.164+0000 7fa5f60bf700 1 --2- 172.21.15.162:0/2942758832 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa5f00b3ab0 0x7fa5f00b30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:51.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.164+0000 7fa5f60bf700 1 -- 172.21.15.162:0/2942758832 >> v2:172.21.15.118:3300/0 conn(0x7fa5f00b2d00 msgr2=0x7fa5f005c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:51.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.164+0000 7fa5f60bf700 1 --2- 172.21.15.162:0/2942758832 >> v2:172.21.15.118:3300/0 conn(0x7fa5f00b2d00 0x7fa5f005c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:51.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.164+0000 7fa5f60bf700 1 -- 172.21.15.162:0/2942758832 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f005c030 con 0x7fa5f0142ad0 2020-11-28T21:54:51.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.164+0000 7fa5f58be700 1 --2- 172.21.15.162:0/2942758832 >> v2:172.21.15.118:3300/0 conn(0x7fa5f00b2d00 0x7fa5f005c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:54:51.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.164+0000 7fa5f50bd700 1 --2- 172.21.15.162:0/2942758832 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa5f00b3ab0 0x7fa5f00b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:54:51.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.164+0000 7fa5f60bf700 1 --2- 172.21.15.162:0/2942758832 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5f0142ad0 0x7fa5f0142f60 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 rx=0x7fa5e00032b0 tx=0x7fa5e0009270).ready entity=mon.1 client_cookie=8e19a212b15aae26 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:51.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.164+0000 7fa5f48bc700 1 -- 172.21.15.162:0/2942758832 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5e000bc00 con 0x7fa5f0142ad0 2020-11-28T21:54:51.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.164+0000 7fa5f48bc700 1 -- 172.21.15.162:0/2942758832 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5e000bd60 con 0x7fa5f0142ad0 2020-11-28T21:54:51.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.164+0000 7fa5f48bc700 1 -- 172.21.15.162:0/2942758832 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5e0011400 con 0x7fa5f0142ad0 2020-11-28T21:54:51.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.164+0000 7fa5f68c0700 1 -- 172.21.15.162:0/2942758832 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5f0142ad0 msgr2=0x7fa5f0142f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:51.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.164+0000 7fa5f68c0700 1 --2- 172.21.15.162:0/2942758832 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5f0142ad0 0x7fa5f0142f60 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 rx=0x7fa5e00032b0 tx=0x7fa5e0009270).stop 2020-11-28T21:54:51.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.164+0000 7fa5f68c0700 1 -- 172.21.15.162:0/2942758832 shutdown_connections 2020-11-28T21:54:51.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.164+0000 7fa5f68c0700 1 --2- 172.21.15.162:0/2942758832 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5f0142ad0 0x7fa5f0142f60 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:51.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.164+0000 7fa5f68c0700 1 --2- 172.21.15.162:0/2942758832 >> v2:172.21.15.118:3300/0 conn(0x7fa5f00b2d00 0x7fa5f005c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:51.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.164+0000 7fa5f68c0700 1 --2- 172.21.15.162:0/2942758832 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa5f00b3ab0 0x7fa5f00b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:51.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.164+0000 7fa5f68c0700 1 -- 172.21.15.162:0/2942758832 >> 172.21.15.162:0/2942758832 conn(0x7fa5f00604c0 msgr2=0x7fa5f00608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:51.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.164+0000 7fa5f68c0700 1 -- 172.21.15.162:0/2942758832 shutdown_connections 2020-11-28T21:54:51.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.164+0000 7fa5f68c0700 1 -- 172.21.15.162:0/2942758832 wait complete. 2020-11-28T21:54:51.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.168+0000 7fa5f68c0700 1 Processor -- start 2020-11-28T21:54:51.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.168+0000 7fa5f68c0700 1 -- start start 2020-11-28T21:54:51.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.168+0000 7fa5f68c0700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa5f00b2d00 0x7fa5f00b53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:51.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.168+0000 7fa5f68c0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5f00b3ab0 0x7fa5f013d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:51.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.168+0000 7fa5f68c0700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa5f0142ad0 0x7fa5f013ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:51.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.168+0000 7fa5f68c0700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f01394d0 con 0x7fa5f0142ad0 2020-11-28T21:54:51.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.168+0000 7fa5f68c0700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f013a730 con 0x7fa5f00b3ab0 2020-11-28T21:54:51.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.168+0000 7fa5f68c0700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f013a1f0 con 0x7fa5f00b2d00 2020-11-28T21:54:51.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.168+0000 7fa5f50bd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5f00b3ab0 0x7fa5f013d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:51.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.168+0000 7fa5f50bd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5f00b3ab0 0x7fa5f013d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44054/0 (socket says 172.21.15.162:44054) 2020-11-28T21:54:51.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.168+0000 7fa5f50bd700 1 -- 172.21.15.162:0/651532025 learned_addr learned my addr 172.21.15.162:0/651532025 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:51.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.168+0000 7fa5f58be700 1 --2- 172.21.15.162:0/651532025 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa5f00b2d00 0x7fa5f00b53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:51.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.168+0000 7fa5f50bd700 1 -- 172.21.15.162:0/651532025 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa5f00b2d00 msgr2=0x7fa5f00b53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:51.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.168+0000 7fa5f50bd700 1 --2- 172.21.15.162:0/651532025 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa5f00b2d00 0x7fa5f00b53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:51.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.168+0000 7fa5f60bf700 1 --2- 172.21.15.162:0/651532025 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa5f0142ad0 0x7fa5f013ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:51.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.168+0000 7fa5f50bd700 1 -- 172.21.15.162:0/651532025 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa5f0142ad0 msgr2=0x7fa5f013ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:51.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.168+0000 7fa5f50bd700 1 --2- 172.21.15.162:0/651532025 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa5f0142ad0 0x7fa5f013ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:51.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.168+0000 7fa5f50bd700 1 -- 172.21.15.162:0/651532025 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f013a490 con 0x7fa5f00b3ab0 2020-11-28T21:54:51.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.168+0000 7fa5f60bf700 1 --2- 172.21.15.162:0/651532025 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa5f0142ad0 0x7fa5f013ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:54:51.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.168+0000 7fa5f50bd700 1 --2- 172.21.15.162:0/651532025 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5f00b3ab0 0x7fa5f013d8c0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 rx=0x7fa5e400a000 tx=0x7fa5e4008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:51.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.168+0000 7fa5eeffd700 1 -- 172.21.15.162:0/651532025 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5e40127e0 con 0x7fa5f00b3ab0 2020-11-28T21:54:51.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.168+0000 7fa5eeffd700 1 -- 172.21.15.162:0/651532025 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5e4012940 con 0x7fa5f00b3ab0 2020-11-28T21:54:51.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.168+0000 7fa5f68c0700 1 -- 172.21.15.162:0/651532025 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5f0138940 con 0x7fa5f00b3ab0 2020-11-28T21:54:51.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.168+0000 7fa5eeffd700 1 -- 172.21.15.162:0/651532025 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5e4012ca0 con 0x7fa5f00b3ab0 2020-11-28T21:54:51.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.168+0000 7fa5f68c0700 1 -- 172.21.15.162:0/651532025 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5f013e2a0 con 0x7fa5f00b3ab0 2020-11-28T21:54:51.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.172+0000 7fa5eeffd700 1 -- 172.21.15.162:0/651532025 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa5e4013480 con 0x7fa5f00b3ab0 2020-11-28T21:54:51.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.172+0000 7fa5f68c0700 1 -- 172.21.15.162:0/651532025 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5f00b7280 con 0x7fa5f00b3ab0 2020-11-28T21:54:51.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.172+0000 7fa5eeffd700 1 --2- 172.21.15.162:0/651532025 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa5d0055440 0x7fa5d0057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:51.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.172+0000 7fa5eeffd700 1 -- 172.21.15.162:0/651532025 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5957+0+0 (secure 0 0 0) 0x7fa5e406e730 con 0x7fa5f00b3ab0 2020-11-28T21:54:51.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.172+0000 7fa5f58be700 1 --2- 172.21.15.162:0/651532025 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa5d0055440 0x7fa5d0057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:51.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.172+0000 7fa5f58be700 1 --2- 172.21.15.162:0/651532025 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa5d0055440 0x7fa5d0057900 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7fa5e800c000 tx=0x7fa5e8008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:51.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.176+0000 7fa5eeffd700 1 -- 172.21.15.162:0/651532025 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa5f00b7280 con 0x7fa5f00b3ab0 2020-11-28T21:54:51.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.364+0000 7fa5f68c0700 1 -- 172.21.15.162:0/651532025 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fa5f013d600 con 0x7fa5f00b3ab0 2020-11-28T21:54:51.381 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.380+0000 7fa5eeffd700 1 -- 172.21.15.162:0/651532025 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 21 and data pool 22 v55) v1 ==== 189+0+0 (secure 0 0 0) 0x7fa5f013d600 con 0x7fa5f00b3ab0 2020-11-28T21:54:51.389 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.384+0000 7fa5f68c0700 1 -- 172.21.15.162:0/651532025 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa5d0055440 msgr2=0x7fa5d0057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:51.389 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.384+0000 7fa5f68c0700 1 --2- 172.21.15.162:0/651532025 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa5d0055440 0x7fa5d0057900 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7fa5e800c000 tx=0x7fa5e8008040).stop 2020-11-28T21:54:51.389 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.384+0000 7fa5f68c0700 1 -- 172.21.15.162:0/651532025 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5f00b3ab0 msgr2=0x7fa5f013d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:51.389 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.384+0000 7fa5f68c0700 1 --2- 172.21.15.162:0/651532025 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5f00b3ab0 0x7fa5f013d8c0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 rx=0x7fa5e400a000 tx=0x7fa5e4008040).stop 2020-11-28T21:54:51.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.384+0000 7fa5f68c0700 1 -- 172.21.15.162:0/651532025 shutdown_connections 2020-11-28T21:54:51.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.384+0000 7fa5f68c0700 1 --2- 172.21.15.162:0/651532025 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa5d0055440 0x7fa5d0057900 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:51.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.384+0000 7fa5f68c0700 1 --2- 172.21.15.162:0/651532025 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa5f00b2d00 0x7fa5f00b53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:51.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.384+0000 7fa5f68c0700 1 --2- 172.21.15.162:0/651532025 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa5f0142ad0 0x7fa5f013ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:51.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.384+0000 7fa5f68c0700 1 --2- 172.21.15.162:0/651532025 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5f00b3ab0 0x7fa5f013d8c0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:51.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.384+0000 7fa5f68c0700 1 -- 172.21.15.162:0/651532025 >> 172.21.15.162:0/651532025 conn(0x7fa5f00604c0 msgr2=0x7fa5f013b160 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:51.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.384+0000 7fa5f68c0700 1 -- 172.21.15.162:0/651532025 shutdown_connections 2020-11-28T21:54:51.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.384+0000 7fa5f68c0700 1 -- 172.21.15.162:0/651532025 wait complete. 2020-11-28T21:54:51.392 INFO:teuthology.orchestra.run.smithi162.stderr:new fs with metadata pool 21 and data pool 22 2020-11-28T21:54:51.406 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:54:51.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.500+0000 7f54bc210700 1 Processor -- start 2020-11-28T21:54:51.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.500+0000 7f54bc210700 1 -- start start 2020-11-28T21:54:51.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.500+0000 7f54bc210700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f54b40b7320 0x7f54b413d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:51.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.500+0000 7f54bc210700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f54b40b19f0 0x7f54b413dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:51.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.500+0000 7f54bc210700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f54b40b1f00 0x7f54b40b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:51.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.500+0000 7f54bc210700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54b40b7700 con 0x7f54b40b19f0 2020-11-28T21:54:51.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.500+0000 7f54bc210700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54b4067b50 con 0x7f54b40b1f00 2020-11-28T21:54:51.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.500+0000 7f54bc210700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b4067df0 con 0x7f54b40b7320 2020-11-28T21:54:51.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.500+0000 7f54ba7ad700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f54b40b1f00 0x7f54b40b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:51.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.500+0000 7f54ba7ad700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f54b40b1f00 0x7f54b40b27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44068/0 (socket says 172.21.15.162:44068) 2020-11-28T21:54:51.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.500+0000 7f54ba7ad700 1 -- 172.21.15.162:0/2718394629 learned_addr learned my addr 172.21.15.162:0/2718394629 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:51.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.500+0000 7f54b9fac700 1 --2- 172.21.15.162:0/2718394629 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f54b40b7320 0x7f54b413d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:51.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.500+0000 7f54b97ab700 1 --2- 172.21.15.162:0/2718394629 >> v2:172.21.15.118:3300/0 conn(0x7f54b40b19f0 0x7f54b413dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:51.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.500+0000 7f54ba7ad700 1 -- 172.21.15.162:0/2718394629 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f54b40b7320 msgr2=0x7f54b413d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:51.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.500+0000 7f54ba7ad700 1 --2- 172.21.15.162:0/2718394629 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f54b40b7320 0x7f54b413d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:51.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.500+0000 7f54ba7ad700 1 -- 172.21.15.162:0/2718394629 >> v2:172.21.15.118:3300/0 conn(0x7f54b40b19f0 msgr2=0x7f54b413dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:51.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.500+0000 7f54ba7ad700 1 --2- 172.21.15.162:0/2718394629 >> v2:172.21.15.118:3300/0 conn(0x7f54b40b19f0 0x7f54b413dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:51.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.500+0000 7f54ba7ad700 1 -- 172.21.15.162:0/2718394629 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b4065c80 con 0x7f54b40b1f00 2020-11-28T21:54:51.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.500+0000 7f54ba7ad700 1 --2- 172.21.15.162:0/2718394629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54b40b1f00 0x7f54b40b27b0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 rx=0x7f54a4002f30 tx=0x7f54a4006e80).ready entity=mon.1 client_cookie=eadc9ee1c91714ba server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:51.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.500+0000 7f54b8faa700 1 -- 172.21.15.162:0/2718394629 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54a4009640 con 0x7f54b40b1f00 2020-11-28T21:54:51.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.500+0000 7f54b8faa700 1 -- 172.21.15.162:0/2718394629 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a40097a0 con 0x7f54b40b1f00 2020-11-28T21:54:51.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.500+0000 7f54b8faa700 1 -- 172.21.15.162:0/2718394629 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54a4009b00 con 0x7f54b40b1f00 2020-11-28T21:54:51.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.500+0000 7f54bc210700 1 -- 172.21.15.162:0/2718394629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54b40b1f00 msgr2=0x7f54b40b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:51.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.500+0000 7f54bc210700 1 --2- 172.21.15.162:0/2718394629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54b40b1f00 0x7f54b40b27b0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 rx=0x7f54a4002f30 tx=0x7f54a4006e80).stop 2020-11-28T21:54:51.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.504+0000 7f54bc210700 1 -- 172.21.15.162:0/2718394629 shutdown_connections 2020-11-28T21:54:51.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.504+0000 7f54bc210700 1 --2- 172.21.15.162:0/2718394629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54b40b1f00 0x7f54b40b27b0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:51.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.504+0000 7f54bc210700 1 --2- 172.21.15.162:0/2718394629 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f54b40b7320 0x7f54b413d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:51.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.504+0000 7f54bc210700 1 --2- 172.21.15.162:0/2718394629 >> v2:172.21.15.118:3300/0 conn(0x7f54b40b19f0 0x7f54b413dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:51.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.504+0000 7f54bc210700 1 -- 172.21.15.162:0/2718394629 >> 172.21.15.162:0/2718394629 conn(0x7f54b4062830 msgr2=0x7f54b4062c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:51.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.504+0000 7f54bc210700 1 -- 172.21.15.162:0/2718394629 shutdown_connections 2020-11-28T21:54:51.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.504+0000 7f54bc210700 1 -- 172.21.15.162:0/2718394629 wait complete. 2020-11-28T21:54:51.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.504+0000 7f54bc210700 1 Processor -- start 2020-11-28T21:54:51.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.504+0000 7f54bc210700 1 -- start start 2020-11-28T21:54:51.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.504+0000 7f54bc210700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54b40b19f0 0x7f54b413e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:51.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.504+0000 7f54bc210700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f54b40b1f00 0x7f54b4147800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:51.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.504+0000 7f54bc210700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f54b40b7320 0x7f54b4152d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:51.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.504+0000 7f54bc210700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b40b0b40 con 0x7f54b40b7320 2020-11-28T21:54:51.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.504+0000 7f54bc210700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b4150570 con 0x7f54b40b19f0 2020-11-28T21:54:51.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.504+0000 7f54bc210700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b40b0600 con 0x7f54b40b1f00 2020-11-28T21:54:51.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.504+0000 7f54b9fac700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54b40b19f0 0x7f54b413e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:51.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.504+0000 7f54b9fac700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54b40b19f0 0x7f54b413e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44074/0 (socket says 172.21.15.162:44074) 2020-11-28T21:54:51.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.504+0000 7f54b9fac700 1 -- 172.21.15.162:0/450017151 learned_addr learned my addr 172.21.15.162:0/450017151 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:51.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.508+0000 7f54b97ab700 1 --2- 172.21.15.162:0/450017151 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f54b40b1f00 0x7f54b4147800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:51.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.508+0000 7f54ba7ad700 1 --2- 172.21.15.162:0/450017151 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f54b40b7320 0x7f54b4152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:51.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.508+0000 7f54b9fac700 1 -- 172.21.15.162:0/450017151 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f54b40b1f00 msgr2=0x7f54b4147800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:51.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.508+0000 7f54b9fac700 1 --2- 172.21.15.162:0/450017151 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f54b40b1f00 0x7f54b4147800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:51.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.508+0000 7f54b9fac700 1 -- 172.21.15.162:0/450017151 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f54b40b7320 msgr2=0x7f54b4152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:51.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.508+0000 7f54b9fac700 1 --2- 172.21.15.162:0/450017151 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f54b40b7320 0x7f54b4152d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:51.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.508+0000 7f54b9fac700 1 -- 172.21.15.162:0/450017151 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b40b08a0 con 0x7f54b40b19f0 2020-11-28T21:54:51.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.508+0000 7f54b9fac700 1 --2- 172.21.15.162:0/450017151 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54b40b19f0 0x7f54b413e6d0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 rx=0x7f54ac00c000 tx=0x7f54ac0091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:51.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.508+0000 7f54b2ffd700 1 -- 172.21.15.162:0/450017151 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54ac00cb50 con 0x7f54b40b19f0 2020-11-28T21:54:51.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.508+0000 7f54b2ffd700 1 -- 172.21.15.162:0/450017151 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54ac00ccb0 con 0x7f54b40b19f0 2020-11-28T21:54:51.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.508+0000 7f54bc210700 1 -- 172.21.15.162:0/450017151 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54b40b1080 con 0x7f54b40b19f0 2020-11-28T21:54:51.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.508+0000 7f54b2ffd700 1 -- 172.21.15.162:0/450017151 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54ac0125e0 con 0x7f54b40b19f0 2020-11-28T21:54:51.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.508+0000 7f54bc210700 1 -- 172.21.15.162:0/450017151 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54b4144b10 con 0x7f54b40b19f0 2020-11-28T21:54:51.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.508+0000 7f54bc210700 1 -- 172.21.15.162:0/450017151 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54b4057050 con 0x7f54b40b19f0 2020-11-28T21:54:51.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.508+0000 7f54b2ffd700 1 -- 172.21.15.162:0/450017151 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f54ac022070 con 0x7f54b40b19f0 2020-11-28T21:54:51.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.512+0000 7f54b2ffd700 1 --2- 172.21.15.162:0/450017151 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f54940553b0 0x7f5494057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:51.521 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.512+0000 7f54b97ab700 1 --2- 172.21.15.162:0/450017151 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f54940553b0 0x7f5494057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:51.521 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.512+0000 7f54b2ffd700 1 -- 172.21.15.162:0/450017151 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f54ac06e740 con 0x7f54b40b19f0 2020-11-28T21:54:51.521 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.512+0000 7f54b97ab700 1 --2- 172.21.15.162:0/450017151 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f54940553b0 0x7f5494057870 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7f54a800b000 tx=0x7f54a8006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:51.522 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.520+0000 7f54b2ffd700 1 -- 172.21.15.162:0/450017151 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f54ac0449a0 con 0x7f54b40b19f0 2020-11-28T21:54:51.665 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.664+0000 7f54bc210700 1 -- 172.21.15.162:0/450017151 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f54b40b2520 con 0x7f54b40b19f0 2020-11-28T21:54:51.667 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.664+0000 7f54b2ffd700 1 -- 172.21.15.162:0/450017151 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+16161 (secure 0 0 0) 0x7f54ac047fc0 con 0x7f54b40b19f0 2020-11-28T21:54:51.668 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:54:51.668 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":118,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:54:51.370747+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:54:49.956401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2020-11-28T21:54:50.370123+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:54:51.669 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.664+0000 7f54bc210700 1 -- 172.21.15.162:0/450017151 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f54940553b0 msgr2=0x7f5494057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:51.669 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.664+0000 7f54bc210700 1 --2- 172.21.15.162:0/450017151 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f54940553b0 0x7f5494057870 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7f54a800b000 tx=0x7f54a8006dc0).stop 2020-11-28T21:54:51.669 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.664+0000 7f54bc210700 1 -- 172.21.15.162:0/450017151 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54b40b19f0 msgr2=0x7f54b413e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:51.670 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.664+0000 7f54bc210700 1 --2- 172.21.15.162:0/450017151 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54b40b19f0 0x7f54b413e6d0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 rx=0x7f54ac00c000 tx=0x7f54ac0091a0).stop 2020-11-28T21:54:51.671 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.664+0000 7f54bc210700 1 -- 172.21.15.162:0/450017151 shutdown_connections 2020-11-28T21:54:51.671 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.668+0000 7f54bc210700 1 --2- 172.21.15.162:0/450017151 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f54940553b0 0x7f5494057870 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:51.672 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.668+0000 7f54bc210700 1 --2- 172.21.15.162:0/450017151 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f54b40b7320 0x7f54b4152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:51.672 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.668+0000 7f54bc210700 1 --2- 172.21.15.162:0/450017151 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54b40b19f0 0x7f54b413e6d0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:51.673 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.668+0000 7f54bc210700 1 --2- 172.21.15.162:0/450017151 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f54b40b1f00 0x7f54b4147800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:51.673 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.668+0000 7f54bc210700 1 -- 172.21.15.162:0/450017151 >> 172.21.15.162:0/450017151 conn(0x7f54b4062830 msgr2=0x7f54b4062c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:51.673 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.668+0000 7f54bc210700 1 -- 172.21.15.162:0/450017151 shutdown_connections 2020-11-28T21:54:51.674 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.668+0000 7f54bc210700 1 -- 172.21.15.162:0/450017151 wait complete. 2020-11-28T21:54:51.682 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:54:51.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.792+0000 7fc229058700 1 Processor -- start 2020-11-28T21:54:51.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.792+0000 7fc229058700 1 -- start start 2020-11-28T21:54:51.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.792+0000 7fc229058700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fc2240994f0 0x7fc2240b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:51.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.792+0000 7fc229058700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc224098f10 con 0x7fc224098b30 2020-11-28T21:54:51.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.792+0000 7fc229058700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2240b0680 con 0x7fc224060710 2020-11-28T21:54:51.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.792+0000 7fc229058700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2240b1030 con 0x7fc2240994f0 2020-11-28T21:54:51.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.792+0000 7fc228857700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fc224060710 0x7fc2240b1580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:50928/0 (socket says 172.21.15.162:50928) 2020-11-28T21:54:51.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.792+0000 7fc228857700 1 -- 172.21.15.162:0/346605848 learned_addr learned my addr 172.21.15.162:0/346605848 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:51.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.792+0000 7fc2237fe700 1 --2- 172.21.15.162:0/346605848 >> v2:172.21.15.118:3300/0 conn(0x7fc2240994f0 0x7fc2240b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:51.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.792+0000 7fc222ffd700 1 -- 172.21.15.162:0/346605848 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1833783364 0 0) 0x7fc2240b0680 con 0x7fc224060710 2020-11-28T21:54:51.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.792+0000 7fc222ffd700 1 -- 172.21.15.162:0/346605848 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc210003230 con 0x7fc224060710 2020-11-28T21:54:51.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.792+0000 7fc222ffd700 1 -- 172.21.15.162:0/346605848 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3315792372 0 0) 0x7fc210003230 con 0x7fc224060710 2020-11-28T21:54:51.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.792+0000 7fc222ffd700 1 -- 172.21.15.162:0/346605848 >> v2:172.21.15.118:3300/0 conn(0x7fc2240994f0 msgr2=0x7fc2240b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:51.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.792+0000 7fc222ffd700 1 --2- 172.21.15.162:0/346605848 >> v2:172.21.15.118:3300/0 conn(0x7fc2240994f0 0x7fc2240b0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:51.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.792+0000 7fc222ffd700 1 -- 172.21.15.162:0/346605848 >> v1:172.21.15.118:6789/0 conn(0x7fc224098b30 legacy=0x7fc224060000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:51.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.792+0000 7fc222ffd700 1 -- 172.21.15.162:0/346605848 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2240b0920 con 0x7fc224060710 2020-11-28T21:54:51.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.792+0000 7fc222ffd700 1 -- 172.21.15.162:0/346605848 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7fc21c001a50 con 0x7fc224060710 2020-11-28T21:54:51.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.792+0000 7fc222ffd700 1 -- 172.21.15.162:0/346605848 >> v1:172.21.15.162:6789/0 conn(0x7fc224060710 legacy=0x7fc2240b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:51.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.792+0000 7fc222ffd700 1 --2- 172.21.15.162:0/346605848 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc210003720 0x7fc210003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:51.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.792+0000 7fc222ffd700 1 --2- 172.21.15.162:0/346605848 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc2100044d0 0x7fc210004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:51.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.792+0000 7fc222ffd700 1 --2- 172.21.15.162:0/346605848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc210004e40 0x7fc21000b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:51.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.792+0000 7fc222ffd700 1 -- 172.21.15.162:0/346605848 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2240b1030 con 0x7fc2100044d0 2020-11-28T21:54:51.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.792+0000 7fc222ffd700 1 -- 172.21.15.162:0/346605848 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc210004250 con 0x7fc210004e40 2020-11-28T21:54:51.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.792+0000 7fc222ffd700 1 -- 172.21.15.162:0/346605848 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc21000bc10 con 0x7fc210003720 2020-11-28T21:54:51.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.792+0000 7fc228857700 1 --2- 172.21.15.162:0/346605848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc210004e40 0x7fc21000b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:51.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.796+0000 7fc223fff700 1 --2- 172.21.15.162:0/346605848 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc210003720 0x7fc210003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:51.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.796+0000 7fc2237fe700 1 --2- 172.21.15.162:0/346605848 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc2100044d0 0x7fc210004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:51.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.796+0000 7fc228857700 1 -- 172.21.15.162:0/346605848 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc210003720 msgr2=0x7fc210003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:51.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.796+0000 7fc228857700 1 --2- 172.21.15.162:0/346605848 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc210003720 0x7fc210003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:51.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.796+0000 7fc228857700 1 -- 172.21.15.162:0/346605848 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc2100044d0 msgr2=0x7fc210004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:51.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.796+0000 7fc228857700 1 --2- 172.21.15.162:0/346605848 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc2100044d0 0x7fc210004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:51.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.796+0000 7fc228857700 1 -- 172.21.15.162:0/346605848 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc210003230 con 0x7fc210004e40 2020-11-28T21:54:51.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.796+0000 7fc2237fe700 1 --2- 172.21.15.162:0/346605848 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc2100044d0 0x7fc210004950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:54:51.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.796+0000 7fc228857700 1 -- 172.21.15.162:0/346605848 reap_dead start 2020-11-28T21:54:51.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.796+0000 7fc228857700 1 --2- 172.21.15.162:0/346605848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc210004e40 0x7fc21000b720 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 rx=0x7fc21c001760 tx=0x7fc21c007af0).ready entity=mon.1 client_cookie=eab54c8e9e9adcd server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:51.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.796+0000 7fc222ffd700 1 -- 172.21.15.162:0/346605848 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc21c005c40 con 0x7fc210004e40 2020-11-28T21:54:51.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.796+0000 7fc222ffd700 1 -- 172.21.15.162:0/346605848 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc21c005da0 con 0x7fc210004e40 2020-11-28T21:54:51.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.796+0000 7fc222ffd700 1 -- 172.21.15.162:0/346605848 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc21c0072c0 con 0x7fc210004e40 2020-11-28T21:54:51.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.796+0000 7fc229058700 1 -- 172.21.15.162:0/346605848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc210004e40 msgr2=0x7fc21000b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:51.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.796+0000 7fc229058700 1 --2- 172.21.15.162:0/346605848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc210004e40 0x7fc21000b720 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 rx=0x7fc21c001760 tx=0x7fc21c007af0).stop 2020-11-28T21:54:51.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.796+0000 7fc229058700 1 -- 172.21.15.162:0/346605848 shutdown_connections 2020-11-28T21:54:51.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.796+0000 7fc229058700 1 --2- 172.21.15.162:0/346605848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc210004e40 0x7fc21000b720 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:51.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.796+0000 7fc229058700 1 -- 172.21.15.162:0/346605848 >> 172.21.15.162:0/346605848 conn(0x7fc22405b970 msgr2=0x7fc22405bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:51.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.796+0000 7fc229058700 1 -- 172.21.15.162:0/346605848 shutdown_connections 2020-11-28T21:54:51.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.796+0000 7fc229058700 1 -- 172.21.15.162:0/346605848 wait complete. 2020-11-28T21:54:51.812 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.796+0000 7fc229058700 1 Processor -- start 2020-11-28T21:54:51.812 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.796+0000 7fc229058700 1 -- start start 2020-11-28T21:54:51.812 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.800+0000 7fc229058700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc210004e40 0x7fc224159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:51.812 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.800+0000 7fc229058700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2240670b0 0x7fc224065450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:51.812 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.800+0000 7fc229058700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc224065940 0x7fc224147520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:51.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.800+0000 7fc229058700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc224159960 con 0x7fc224065940 2020-11-28T21:54:51.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.800+0000 7fc229058700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc224159b90 con 0x7fc2240670b0 2020-11-28T21:54:51.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.800+0000 7fc229058700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc224159e30 con 0x7fc210004e40 2020-11-28T21:54:51.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.800+0000 7fc2237fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2240670b0 0x7fc224065450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:51.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.800+0000 7fc2237fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2240670b0 0x7fc224065450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44098/0 (socket says 172.21.15.162:44098) 2020-11-28T21:54:51.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.800+0000 7fc2237fe700 1 -- 172.21.15.162:0/1877433729 learned_addr learned my addr 172.21.15.162:0/1877433729 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:51.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.800+0000 7fc223fff700 1 --2- 172.21.15.162:0/1877433729 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc210004e40 0x7fc224159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:51.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.800+0000 7fc228857700 1 --2- 172.21.15.162:0/1877433729 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc224065940 0x7fc224147520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:51.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.800+0000 7fc2237fe700 1 -- 172.21.15.162:0/1877433729 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc210004e40 msgr2=0x7fc224159470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:51.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.800+0000 7fc2237fe700 1 --2- 172.21.15.162:0/1877433729 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc210004e40 0x7fc224159470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:51.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.800+0000 7fc2237fe700 1 -- 172.21.15.162:0/1877433729 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc224065940 msgr2=0x7fc224147520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:51.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.800+0000 7fc2237fe700 1 --2- 172.21.15.162:0/1877433729 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc224065940 0x7fc224147520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:51.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.800+0000 7fc2237fe700 1 -- 172.21.15.162:0/1877433729 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2240644f0 con 0x7fc2240670b0 2020-11-28T21:54:51.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.800+0000 7fc2237fe700 1 --2- 172.21.15.162:0/1877433729 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2240670b0 0x7fc224065450 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 rx=0x7fc218008000 tx=0x7fc218008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:51.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.800+0000 7fc2217fa700 1 -- 172.21.15.162:0/1877433729 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc21800e8a0 con 0x7fc2240670b0 2020-11-28T21:54:51.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.800+0000 7fc2217fa700 1 -- 172.21.15.162:0/1877433729 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc21800ea00 con 0x7fc2240670b0 2020-11-28T21:54:51.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.800+0000 7fc2217fa700 1 -- 172.21.15.162:0/1877433729 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc21800ed60 con 0x7fc2240670b0 2020-11-28T21:54:51.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.800+0000 7fc229058700 1 -- 172.21.15.162:0/1877433729 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2240636c0 con 0x7fc2240670b0 2020-11-28T21:54:51.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.800+0000 7fc229058700 1 -- 172.21.15.162:0/1877433729 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc224063e60 con 0x7fc2240670b0 2020-11-28T21:54:51.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.800+0000 7fc2217fa700 1 -- 172.21.15.162:0/1877433729 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc218014470 con 0x7fc2240670b0 2020-11-28T21:54:51.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.800+0000 7fc229058700 1 -- 172.21.15.162:0/1877433729 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc224159fa0 con 0x7fc2240670b0 2020-11-28T21:54:51.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.804+0000 7fc2217fa700 1 --2- 172.21.15.162:0/1877433729 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc204055330 0x7fc2040577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:51.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.804+0000 7fc223fff700 1 --2- 172.21.15.162:0/1877433729 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc204055330 0x7fc2040577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:51.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.804+0000 7fc2217fa700 1 -- 172.21.15.162:0/1877433729 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc21806e360 con 0x7fc2240670b0 2020-11-28T21:54:51.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.804+0000 7fc223fff700 1 --2- 172.21.15.162:0/1877433729 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc204055330 0x7fc2040577f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7fc214003000 tx=0x7fc214009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:51.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.808+0000 7fc2217fa700 1 -- 172.21.15.162:0/1877433729 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc218044600 con 0x7fc2240670b0 2020-11-28T21:54:51.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.956+0000 7fc229058700 1 -- 172.21.15.162:0/1877433729 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc224057050 con 0x7fc2240670b0 2020-11-28T21:54:51.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.956+0000 7fc2217fa700 1 -- 172.21.15.162:0/1877433729 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+16161 (secure 0 0 0) 0x7fc218047c20 con 0x7fc2240670b0 2020-11-28T21:54:51.959 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:54:51.959 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":118,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:54:51.370747+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:54:49.956401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2020-11-28T21:54:50.370123+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:54:51.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.956+0000 7fc229058700 1 -- 172.21.15.162:0/1877433729 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc204055330 msgr2=0x7fc2040577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:51.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.956+0000 7fc229058700 1 --2- 172.21.15.162:0/1877433729 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc204055330 0x7fc2040577f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7fc214003000 tx=0x7fc214009040).stop 2020-11-28T21:54:51.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.956+0000 7fc229058700 1 -- 172.21.15.162:0/1877433729 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2240670b0 msgr2=0x7fc224065450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:51.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.956+0000 7fc229058700 1 --2- 172.21.15.162:0/1877433729 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2240670b0 0x7fc224065450 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 rx=0x7fc218008000 tx=0x7fc218008040).stop 2020-11-28T21:54:51.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.956+0000 7fc229058700 1 -- 172.21.15.162:0/1877433729 shutdown_connections 2020-11-28T21:54:51.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.956+0000 7fc229058700 1 --2- 172.21.15.162:0/1877433729 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc204055330 0x7fc2040577f0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:51.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.956+0000 7fc229058700 1 --2- 172.21.15.162:0/1877433729 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc210004e40 0x7fc224159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:51.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.956+0000 7fc229058700 1 --2- 172.21.15.162:0/1877433729 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc224065940 0x7fc224147520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:51.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.956+0000 7fc229058700 1 --2- 172.21.15.162:0/1877433729 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2240670b0 0x7fc224065450 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:51.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.956+0000 7fc229058700 1 -- 172.21.15.162:0/1877433729 >> 172.21.15.162:0/1877433729 conn(0x7fc22405b970 msgr2=0x7fc2240ab690 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:51.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.956+0000 7fc229058700 1 -- 172.21.15.162:0/1877433729 shutdown_connections 2020-11-28T21:54:51.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:51.956+0000 7fc229058700 1 -- 172.21.15.162:0/1877433729 wait complete. 2020-11-28T21:54:51.974 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2020-11-28T21:54:52.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.092+0000 7f6b5fa31700 1 Processor -- start 2020-11-28T21:54:52.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.092+0000 7f6b5fa31700 1 -- start start 2020-11-28T21:54:52.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.092+0000 7f6b5fa31700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6b580ae230 0x7f6b580a8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:52.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.092+0000 7f6b5fa31700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6b58142dc0 0x7f6b58149c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:52.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.096+0000 7f6b5fa31700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b5805f380 con 0x7f6b580ade50 2020-11-28T21:54:52.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.096+0000 7f6b5fa31700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b580acda0 con 0x7f6b580ae230 2020-11-28T21:54:52.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.096+0000 7f6b5fa31700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b580a8a60 con 0x7f6b58142dc0 2020-11-28T21:54:52.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.096+0000 7f6b5cfcc700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6b580ae230 0x7f6b580a8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:52.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.096+0000 7f6b5cfcc700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6b580ae230 0x7f6b580a8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44102/0 (socket says 172.21.15.162:44102) 2020-11-28T21:54:52.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.096+0000 7f6b5d7cd700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f6b580ade50 0x7f6b581426b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:50946/0 (socket says 172.21.15.162:50946) 2020-11-28T21:54:52.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.096+0000 7f6b5cfcc700 1 -- 172.21.15.162:0/2170046446 learned_addr learned my addr 172.21.15.162:0/2170046446 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:52.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.096+0000 7f6b5dfce700 1 --2- 172.21.15.162:0/2170046446 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6b58142dc0 0x7f6b58149c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:52.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.096+0000 7f6b5cfcc700 1 -- 172.21.15.162:0/2170046446 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6b58142dc0 msgr2=0x7f6b58149c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:52.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.096+0000 7f6b5cfcc700 1 --2- 172.21.15.162:0/2170046446 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6b58142dc0 0x7f6b58149c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:52.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.096+0000 7f6b5cfcc700 1 -- 172.21.15.162:0/2170046446 >> v1:172.21.15.162:6789/0 conn(0x7f6b580ade50 legacy=0x7f6b581426b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:52.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.096+0000 7f6b5cfcc700 1 -- 172.21.15.162:0/2170046446 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b58061240 con 0x7f6b580ae230 2020-11-28T21:54:52.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.096+0000 7f6b5dfce700 1 --2- 172.21.15.162:0/2170046446 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6b58142dc0 0x7f6b58149c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:54:52.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.096+0000 7f6b5cfcc700 1 --2- 172.21.15.162:0/2170046446 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b580ae230 0x7f6b580a8570 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 rx=0x7f6b54002f30 tx=0x7f6b54006e80).ready entity=mon.1 client_cookie=8b18fd3c7ed18336 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:52.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.096+0000 7f6b43fff700 1 -- 172.21.15.162:0/2170046446 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b540097f0 con 0x7f6b580ae230 2020-11-28T21:54:52.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.096+0000 7f6b43fff700 1 -- 172.21.15.162:0/2170046446 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b54009950 con 0x7f6b580ae230 2020-11-28T21:54:52.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.096+0000 7f6b43fff700 1 -- 172.21.15.162:0/2170046446 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b54009cb0 con 0x7f6b580ae230 2020-11-28T21:54:52.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.096+0000 7f6b5fa31700 1 -- 172.21.15.162:0/2170046446 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b580ae230 msgr2=0x7f6b580a8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:52.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.096+0000 7f6b5fa31700 1 --2- 172.21.15.162:0/2170046446 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b580ae230 0x7f6b580a8570 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 rx=0x7f6b54002f30 tx=0x7f6b54006e80).stop 2020-11-28T21:54:52.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.096+0000 7f6b5fa31700 1 -- 172.21.15.162:0/2170046446 shutdown_connections 2020-11-28T21:54:52.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.096+0000 7f6b5fa31700 1 --2- 172.21.15.162:0/2170046446 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6b58142dc0 0x7f6b58149c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:52.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.096+0000 7f6b5fa31700 1 --2- 172.21.15.162:0/2170046446 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b580ae230 0x7f6b580a8570 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:52.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.096+0000 7f6b5fa31700 1 -- 172.21.15.162:0/2170046446 >> 172.21.15.162:0/2170046446 conn(0x7f6b5805c3f0 msgr2=0x7f6b580b4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:52.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.096+0000 7f6b5fa31700 1 -- 172.21.15.162:0/2170046446 shutdown_connections 2020-11-28T21:54:52.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.096+0000 7f6b5fa31700 1 -- 172.21.15.162:0/2170046446 wait complete. 2020-11-28T21:54:52.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.096+0000 7f6b5fa31700 1 Processor -- start 2020-11-28T21:54:52.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.096+0000 7f6b5fa31700 1 -- start start 2020-11-28T21:54:52.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.096+0000 7f6b5fa31700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b580ade50 0x7f6b58155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:52.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.096+0000 7f6b5fa31700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6b580ae230 0x7f6b580a9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:52.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.096+0000 7f6b5fa31700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6b58142dc0 0x7f6b580a9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:52.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.096+0000 7f6b5fa31700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5813b880 con 0x7f6b580ae230 2020-11-28T21:54:52.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.096+0000 7f6b5fa31700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b58138860 con 0x7f6b580ade50 2020-11-28T21:54:52.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.096+0000 7f6b5fa31700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b58138b00 con 0x7f6b58142dc0 2020-11-28T21:54:52.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.096+0000 7f6b5d7cd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b580ade50 0x7f6b58155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:52.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.096+0000 7f6b5d7cd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b580ade50 0x7f6b58155a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44108/0 (socket says 172.21.15.162:44108) 2020-11-28T21:54:52.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.096+0000 7f6b5d7cd700 1 -- 172.21.15.162:0/3642901573 learned_addr learned my addr 172.21.15.162:0/3642901573 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:52.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.100+0000 7f6b5cfcc700 1 --2- 172.21.15.162:0/3642901573 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6b580ae230 0x7f6b580a9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:52.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.100+0000 7f6b5dfce700 1 --2- 172.21.15.162:0/3642901573 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6b58142dc0 0x7f6b580a9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:52.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.100+0000 7f6b5d7cd700 1 -- 172.21.15.162:0/3642901573 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6b58142dc0 msgr2=0x7f6b580a9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:52.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.100+0000 7f6b5d7cd700 1 --2- 172.21.15.162:0/3642901573 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6b58142dc0 0x7f6b580a9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:52.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.100+0000 7f6b5d7cd700 1 -- 172.21.15.162:0/3642901573 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6b580ae230 msgr2=0x7f6b580a9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:52.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.100+0000 7f6b5d7cd700 1 --2- 172.21.15.162:0/3642901573 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6b580ae230 0x7f6b580a9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:52.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.100+0000 7f6b5d7cd700 1 -- 172.21.15.162:0/3642901573 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b58138da0 con 0x7f6b580ade50 2020-11-28T21:54:52.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.100+0000 7f6b5dfce700 1 --2- 172.21.15.162:0/3642901573 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6b58142dc0 0x7f6b580a9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:54:52.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.100+0000 7f6b5d7cd700 1 --2- 172.21.15.162:0/3642901573 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b580ade50 0x7f6b58155a40 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 rx=0x7f6b5813b9c0 tx=0x7f6b4c007bd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:52.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.100+0000 7f6b4affd700 1 -- 172.21.15.162:0/3642901573 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b4c0072e0 con 0x7f6b580ade50 2020-11-28T21:54:52.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.100+0000 7f6b5fa31700 1 -- 172.21.15.162:0/3642901573 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b58155f30 con 0x7f6b580ade50 2020-11-28T21:54:52.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.100+0000 7f6b4affd700 1 -- 172.21.15.162:0/3642901573 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b4c005520 con 0x7f6b580ade50 2020-11-28T21:54:52.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.100+0000 7f6b4affd700 1 -- 172.21.15.162:0/3642901573 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b4c005880 con 0x7f6b580ade50 2020-11-28T21:54:52.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.100+0000 7f6b5fa31700 1 -- 172.21.15.162:0/3642901573 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b58064710 con 0x7f6b580ade50 2020-11-28T21:54:52.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.100+0000 7f6b5fa31700 1 -- 172.21.15.162:0/3642901573 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b58057050 con 0x7f6b580ade50 2020-11-28T21:54:52.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.100+0000 7f6b4affd700 1 -- 172.21.15.162:0/3642901573 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6b4c010070 con 0x7f6b580ade50 2020-11-28T21:54:52.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.100+0000 7f6b4affd700 1 --2- 172.21.15.162:0/3642901573 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6b34055350 0x7f6b34057810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:52.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.100+0000 7f6b4affd700 1 -- 172.21.15.162:0/3642901573 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6b4c06d620 con 0x7f6b580ade50 2020-11-28T21:54:52.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.100+0000 7f6b5cfcc700 1 --2- 172.21.15.162:0/3642901573 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6b34055350 0x7f6b34057810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:52.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.104+0000 7f6b5cfcc700 1 --2- 172.21.15.162:0/3642901573 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6b34055350 0x7f6b34057810 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f6b5401c000 tx=0x7f6b54015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:52.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.112+0000 7f6b4affd700 1 -- 172.21.15.162:0/3642901573 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6b4c043880 con 0x7f6b580ade50 2020-11-28T21:54:52.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:52.312+0000 7f6b5fa31700 1 -- 172.21.15.162:0/3642901573 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f6b5805c7d0 con 0x7f6b580ade50 2020-11-28T21:54:53.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.028+0000 7f6b4affd700 1 -- 172.21.15.162:0/3642901573 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v57) v1 ==== 118+0+0 (secure 0 0 0) 0x7f6b4c046ea0 con 0x7f6b580ade50 2020-11-28T21:54:53.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.032+0000 7f6b5fa31700 1 -- 172.21.15.162:0/3642901573 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6b34055350 msgr2=0x7f6b34057810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:53.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.032+0000 7f6b5fa31700 1 --2- 172.21.15.162:0/3642901573 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6b34055350 0x7f6b34057810 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f6b5401c000 tx=0x7f6b54015000).stop 2020-11-28T21:54:53.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.032+0000 7f6b5fa31700 1 -- 172.21.15.162:0/3642901573 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b580ade50 msgr2=0x7f6b58155a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:53.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.032+0000 7f6b5fa31700 1 --2- 172.21.15.162:0/3642901573 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b580ade50 0x7f6b58155a40 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 rx=0x7f6b5813b9c0 tx=0x7f6b4c007bd0).stop 2020-11-28T21:54:53.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.032+0000 7f6b5fa31700 1 -- 172.21.15.162:0/3642901573 shutdown_connections 2020-11-28T21:54:53.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.032+0000 7f6b5fa31700 1 --2- 172.21.15.162:0/3642901573 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6b34055350 0x7f6b34057810 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:53.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.032+0000 7f6b5fa31700 1 --2- 172.21.15.162:0/3642901573 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6b58142dc0 0x7f6b580a9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:53.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.032+0000 7f6b5fa31700 1 --2- 172.21.15.162:0/3642901573 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b580ade50 0x7f6b58155a40 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:53.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.032+0000 7f6b5fa31700 1 --2- 172.21.15.162:0/3642901573 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6b580ae230 0x7f6b580a9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:53.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.032+0000 7f6b5fa31700 1 -- 172.21.15.162:0/3642901573 >> 172.21.15.162:0/3642901573 conn(0x7f6b5805c3f0 msgr2=0x7f6b580b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:53.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.032+0000 7f6b5fa31700 1 -- 172.21.15.162:0/3642901573 shutdown_connections 2020-11-28T21:54:53.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.032+0000 7f6b5fa31700 1 -- 172.21.15.162:0/3642901573 wait complete. 2020-11-28T21:54:53.054 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:54:53.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.156+0000 7f59b359e700 1 Processor -- start 2020-11-28T21:54:53.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.156+0000 7f59b359e700 1 -- start start 2020-11-28T21:54:53.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.160+0000 7f59b359e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f59b40994f0 0x7f59b40b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:53.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.160+0000 7f59b359e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f59b4060710 0x7f59b40b1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:53.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.160+0000 7f59b359e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59b4098f10 con 0x7f59b4098b30 2020-11-28T21:54:53.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.160+0000 7f59b359e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59b40b1030 con 0x7f59b4060710 2020-11-28T21:54:53.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.160+0000 7f59b359e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59b40b0680 con 0x7f59b40994f0 2020-11-28T21:54:53.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.160+0000 7f59b1d9b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f59b40994f0 0x7f59b40b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:53.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.160+0000 7f59b1d9b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f59b40994f0 0x7f59b40b0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44116/0 (socket says 172.21.15.162:44116) 2020-11-28T21:54:53.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.160+0000 7f59b1d9b700 1 -- 172.21.15.162:0/1542642112 learned_addr learned my addr 172.21.15.162:0/1542642112 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:53.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.160+0000 7f59b2d9d700 1 --2- 172.21.15.162:0/1542642112 >> v2:172.21.15.118:3300/0 conn(0x7f59b4060710 0x7f59b40b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:53.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.160+0000 7f59b1d9b700 1 -- 172.21.15.162:0/1542642112 >> v2:172.21.15.118:3300/0 conn(0x7f59b4060710 msgr2=0x7f59b40b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:53.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.160+0000 7f59b1d9b700 1 --2- 172.21.15.162:0/1542642112 >> v2:172.21.15.118:3300/0 conn(0x7f59b4060710 0x7f59b40b1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:53.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.160+0000 7f59b1d9b700 1 -- 172.21.15.162:0/1542642112 >> v1:172.21.15.162:6789/0 conn(0x7f59b4098b30 legacy=0x7f59b4060000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:53.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.160+0000 7f59b1d9b700 1 -- 172.21.15.162:0/1542642112 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b40b0920 con 0x7f59b40994f0 2020-11-28T21:54:53.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.160+0000 7f59b2d9d700 1 --2- 172.21.15.162:0/1542642112 >> v2:172.21.15.118:3300/0 conn(0x7f59b4060710 0x7f59b40b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:54:53.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.160+0000 7f59b1d9b700 1 --2- 172.21.15.162:0/1542642112 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59b40994f0 0x7f59b40b0000 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 rx=0x7f59a8002f30 tx=0x7f59a8006e80).ready entity=mon.1 client_cookie=6c2ec76ab592db7f server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:53.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.160+0000 7f59b159a700 1 -- 172.21.15.162:0/1542642112 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59a80097f0 con 0x7f59b40994f0 2020-11-28T21:54:53.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.160+0000 7f59b159a700 1 -- 172.21.15.162:0/1542642112 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59a8009950 con 0x7f59b40994f0 2020-11-28T21:54:53.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.160+0000 7f59b159a700 1 -- 172.21.15.162:0/1542642112 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59a8009cb0 con 0x7f59b40994f0 2020-11-28T21:54:53.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.160+0000 7f59b359e700 1 -- 172.21.15.162:0/1542642112 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59b40994f0 msgr2=0x7f59b40b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:53.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.160+0000 7f59b359e700 1 --2- 172.21.15.162:0/1542642112 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59b40994f0 0x7f59b40b0000 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 rx=0x7f59a8002f30 tx=0x7f59a8006e80).stop 2020-11-28T21:54:53.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.160+0000 7f59b359e700 1 -- 172.21.15.162:0/1542642112 shutdown_connections 2020-11-28T21:54:53.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.160+0000 7f59b359e700 1 --2- 172.21.15.162:0/1542642112 >> v2:172.21.15.118:3300/0 conn(0x7f59b4060710 0x7f59b40b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:53.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.160+0000 7f59b359e700 1 --2- 172.21.15.162:0/1542642112 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59b40994f0 0x7f59b40b0000 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:53.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.160+0000 7f59b359e700 1 -- 172.21.15.162:0/1542642112 >> 172.21.15.162:0/1542642112 conn(0x7f59b405b970 msgr2=0x7f59b405bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:53.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.160+0000 7f59b359e700 1 -- 172.21.15.162:0/1542642112 shutdown_connections 2020-11-28T21:54:53.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.160+0000 7f59b359e700 1 -- 172.21.15.162:0/1542642112 wait complete. 2020-11-28T21:54:53.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.160+0000 7f59b359e700 1 Processor -- start 2020-11-28T21:54:53.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.160+0000 7f59b359e700 1 -- start start 2020-11-28T21:54:53.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.164+0000 7f59b359e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59b4060710 0x7f59b4155a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:53.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.164+0000 7f59b359e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f59b4098b30 0x7f59b40aec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:53.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.164+0000 7f59b359e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f59b40994f0 0x7f59b40ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:53.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.164+0000 7f59b359e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b40633e0 con 0x7f59b4098b30 2020-11-28T21:54:53.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.164+0000 7f59b359e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b4063680 con 0x7f59b4060710 2020-11-28T21:54:53.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.164+0000 7f59b359e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59b4063920 con 0x7f59b40994f0 2020-11-28T21:54:53.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.164+0000 7f59b259c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59b4060710 0x7f59b4155a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:53.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.164+0000 7f59b259c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59b4060710 0x7f59b4155a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44126/0 (socket says 172.21.15.162:44126) 2020-11-28T21:54:53.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.164+0000 7f59b259c700 1 -- 172.21.15.162:0/1675357827 learned_addr learned my addr 172.21.15.162:0/1675357827 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:53.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.164+0000 7f59b1d9b700 1 --2- 172.21.15.162:0/1675357827 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f59b4098b30 0x7f59b40aec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:53.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.164+0000 7f59b2d9d700 1 --2- 172.21.15.162:0/1675357827 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f59b40994f0 0x7f59b40ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:53.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.164+0000 7f59b259c700 1 -- 172.21.15.162:0/1675357827 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f59b40994f0 msgr2=0x7f59b40ae640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:53.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.164+0000 7f59b259c700 1 --2- 172.21.15.162:0/1675357827 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f59b40994f0 0x7f59b40ae640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:53.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.164+0000 7f59b259c700 1 -- 172.21.15.162:0/1675357827 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f59b4098b30 msgr2=0x7f59b40aec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:53.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.164+0000 7f59b259c700 1 --2- 172.21.15.162:0/1675357827 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f59b4098b30 0x7f59b40aec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:53.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.164+0000 7f59b259c700 1 -- 172.21.15.162:0/1675357827 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b40aad80 con 0x7f59b4060710 2020-11-28T21:54:53.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.164+0000 7f59b2d9d700 1 --2- 172.21.15.162:0/1675357827 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f59b40994f0 0x7f59b40ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:54:53.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.164+0000 7f59b259c700 1 --2- 172.21.15.162:0/1675357827 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59b4060710 0x7f59b4155a30 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 rx=0x7f59b4061f00 tx=0x7f59a4007c20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:53.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.164+0000 7f59a37fe700 1 -- 172.21.15.162:0/1675357827 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59a400e4a0 con 0x7f59b4060710 2020-11-28T21:54:53.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.164+0000 7f59a37fe700 1 -- 172.21.15.162:0/1675357827 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59a400e600 con 0x7f59b4060710 2020-11-28T21:54:53.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.164+0000 7f59a37fe700 1 -- 172.21.15.162:0/1675357827 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59a400e960 con 0x7f59b4060710 2020-11-28T21:54:53.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.164+0000 7f59b359e700 1 -- 172.21.15.162:0/1675357827 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59b40ab030 con 0x7f59b4060710 2020-11-28T21:54:53.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.164+0000 7f59b359e700 1 -- 172.21.15.162:0/1675357827 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59b40ada00 con 0x7f59b4060710 2020-11-28T21:54:53.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.164+0000 7f59b359e700 1 -- 172.21.15.162:0/1675357827 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59b40ae020 con 0x7f59b4060710 2020-11-28T21:54:53.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.168+0000 7f59a37fe700 1 -- 172.21.15.162:0/1675357827 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f59a4010030 con 0x7f59b4060710 2020-11-28T21:54:53.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.168+0000 7f59a37fe700 1 --2- 172.21.15.162:0/1675357827 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5990055330 0x7f59900577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:53.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.168+0000 7f59b1d9b700 1 --2- 172.21.15.162:0/1675357827 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5990055330 0x7f59900577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:53.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.168+0000 7f59a37fe700 1 -- 172.21.15.162:0/1675357827 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f59a406e1b0 con 0x7f59b4060710 2020-11-28T21:54:53.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.168+0000 7f59b1d9b700 1 --2- 172.21.15.162:0/1675357827 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5990055330 0x7f59900577f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f59a801c000 tx=0x7f59a8015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:53.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.176+0000 7f59a37fe700 1 -- 172.21.15.162:0/1675357827 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f59a40444b0 con 0x7f59b4060710 2020-11-28T21:54:53.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.360+0000 7f59b359e700 1 -- 172.21.15.162:0/1675357827 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59b4063ae0 con 0x7f59b4060710 2020-11-28T21:54:53.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.360+0000 7f59a37fe700 1 -- 172.21.15.162:0/1675357827 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+2504 (secure 0 0 0) 0x7f59a4047ad0 con 0x7f59b4060710 2020-11-28T21:54:53.365 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:54:53.365 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":57,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7257,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/1527184432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1527184432},{"type":"v1","addr":"172.21.15.118:6833","nonce":1527184432}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":54},{"gid":7477,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6835/2749838911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2749838911},{"type":"v1","addr":"172.21.15.162:6835","nonce":2749838911}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":55}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:54:51.371126+0000","modified":"2020-11-28T21:54:53.015631+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7489},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7489":{"gid":7489,"name":"c","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/1061393314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1061393314},{"type":"v1","addr":"172.21.15.162:6837","nonce":1061393314}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2020-11-28T21:54:53.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.364+0000 7f59b359e700 1 -- 172.21.15.162:0/1675357827 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5990055330 msgr2=0x7f59900577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:53.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.364+0000 7f59b359e700 1 --2- 172.21.15.162:0/1675357827 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5990055330 0x7f59900577f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f59a801c000 tx=0x7f59a8015000).stop 2020-11-28T21:54:53.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.364+0000 7f59b359e700 1 -- 172.21.15.162:0/1675357827 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59b4060710 msgr2=0x7f59b4155a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:53.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.364+0000 7f59b359e700 1 --2- 172.21.15.162:0/1675357827 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59b4060710 0x7f59b4155a30 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 rx=0x7f59b4061f00 tx=0x7f59a4007c20).stop 2020-11-28T21:54:53.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.364+0000 7f59b359e700 1 -- 172.21.15.162:0/1675357827 shutdown_connections 2020-11-28T21:54:53.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.364+0000 7f59b359e700 1 --2- 172.21.15.162:0/1675357827 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5990055330 0x7f59900577f0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:53.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.364+0000 7f59b359e700 1 --2- 172.21.15.162:0/1675357827 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f59b40994f0 0x7f59b40ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:53.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.364+0000 7f59b359e700 1 --2- 172.21.15.162:0/1675357827 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59b4060710 0x7f59b4155a30 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:53.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.364+0000 7f59b359e700 1 --2- 172.21.15.162:0/1675357827 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f59b4098b30 0x7f59b40aec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:53.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.364+0000 7f59b359e700 1 -- 172.21.15.162:0/1675357827 >> 172.21.15.162:0/1675357827 conn(0x7f59b405b970 msgr2=0x7f59b40b43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:53.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.364+0000 7f59b359e700 1 -- 172.21.15.162:0/1675357827 shutdown_connections 2020-11-28T21:54:53.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.364+0000 7f59b359e700 1 -- 172.21.15.162:0/1675357827 wait complete. 2020-11-28T21:54:53.370 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 57 2020-11-28T21:54:53.384 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:54:53.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.488+0000 7f396d7a3700 1 Processor -- start 2020-11-28T21:54:53.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.488+0000 7f396d7a3700 1 -- start start 2020-11-28T21:54:53.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.492+0000 7f396d7a3700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f39680b3800 0x7f39680b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:53.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.492+0000 7f396d7a3700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f39680676b0 0x7f3968144150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:53.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.492+0000 7f396d7a3700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f3968144640 0x7f396813d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:53.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.492+0000 7f396d7a3700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3968143d20 con 0x7f3968144640 2020-11-28T21:54:53.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.492+0000 7f396d7a3700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3968062fe0 con 0x7f39680676b0 2020-11-28T21:54:53.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.492+0000 7f396d7a3700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39680b3150 con 0x7f39680b3800 2020-11-28T21:54:53.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.492+0000 7f39677fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f39680676b0 0x7f3968144150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:53.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.492+0000 7f39677fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f39680676b0 0x7f3968144150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44130/0 (socket says 172.21.15.162:44130) 2020-11-28T21:54:53.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.492+0000 7f39677fe700 1 -- 172.21.15.162:0/3353798399 learned_addr learned my addr 172.21.15.162:0/3353798399 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:53.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.492+0000 7f3967fff700 1 --2- 172.21.15.162:0/3353798399 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f39680b3800 0x7f39680b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:53.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.492+0000 7f396cfa2700 1 --2- 172.21.15.162:0/3353798399 >> v2:172.21.15.118:3300/0 conn(0x7f3968144640 0x7f396813d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:53.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.492+0000 7f39677fe700 1 -- 172.21.15.162:0/3353798399 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f39680b3800 msgr2=0x7f39680b3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:53.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.492+0000 7f39677fe700 1 --2- 172.21.15.162:0/3353798399 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f39680b3800 0x7f39680b3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:53.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.492+0000 7f39677fe700 1 -- 172.21.15.162:0/3353798399 >> v2:172.21.15.118:3300/0 conn(0x7f3968144640 msgr2=0x7f396813d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:53.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.492+0000 7f39677fe700 1 --2- 172.21.15.162:0/3353798399 >> v2:172.21.15.118:3300/0 conn(0x7f3968144640 0x7f396813d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:53.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.492+0000 7f39677fe700 1 -- 172.21.15.162:0/3353798399 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3968144af0 con 0x7f39680676b0 2020-11-28T21:54:53.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.492+0000 7f396cfa2700 1 --2- 172.21.15.162:0/3353798399 >> v2:172.21.15.118:3300/0 conn(0x7f3968144640 0x7f396813d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:54:53.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.492+0000 7f39677fe700 1 --2- 172.21.15.162:0/3353798399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f39680676b0 0x7f3968144150 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 rx=0x7f395c002f30 tx=0x7f395c006e80).ready entity=mon.1 client_cookie=6aee7dd47355d2db server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:53.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.492+0000 7f3966ffd700 1 -- 172.21.15.162:0/3353798399 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f395c0097f0 con 0x7f39680676b0 2020-11-28T21:54:53.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.492+0000 7f3966ffd700 1 -- 172.21.15.162:0/3353798399 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f395c009950 con 0x7f39680676b0 2020-11-28T21:54:53.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.492+0000 7f396d7a3700 1 -- 172.21.15.162:0/3353798399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f39680676b0 msgr2=0x7f3968144150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:53.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.492+0000 7f3966ffd700 1 -- 172.21.15.162:0/3353798399 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f395c009cb0 con 0x7f39680676b0 2020-11-28T21:54:53.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.492+0000 7f396d7a3700 1 --2- 172.21.15.162:0/3353798399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f39680676b0 0x7f3968144150 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 rx=0x7f395c002f30 tx=0x7f395c006e80).stop 2020-11-28T21:54:53.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.492+0000 7f396d7a3700 1 -- 172.21.15.162:0/3353798399 shutdown_connections 2020-11-28T21:54:53.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.492+0000 7f396d7a3700 1 --2- 172.21.15.162:0/3353798399 >> v2:172.21.15.118:3300/0 conn(0x7f3968144640 0x7f396813d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:53.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.492+0000 7f396d7a3700 1 --2- 172.21.15.162:0/3353798399 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f39680b3800 0x7f39680b3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:53.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.492+0000 7f396d7a3700 1 --2- 172.21.15.162:0/3353798399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f39680676b0 0x7f3968144150 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:53.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.492+0000 7f396d7a3700 1 -- 172.21.15.162:0/3353798399 >> 172.21.15.162:0/3353798399 conn(0x7f396805b970 msgr2=0x7f396805bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:53.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.492+0000 7f396d7a3700 1 -- 172.21.15.162:0/3353798399 shutdown_connections 2020-11-28T21:54:53.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.492+0000 7f396d7a3700 1 -- 172.21.15.162:0/3353798399 wait complete. 2020-11-28T21:54:53.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.492+0000 7f396d7a3700 1 Processor -- start 2020-11-28T21:54:53.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.492+0000 7f396d7a3700 1 -- start start 2020-11-28T21:54:53.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.496+0000 7f396d7a3700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f39680676b0 0x7f396805fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:53.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.496+0000 7f396d7a3700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f39680b3800 0x7f39680ad210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:53.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.496+0000 7f396d7a3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3968144640 0x7f39681521c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:53.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.496+0000 7f396d7a3700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3968142bd0 con 0x7f39680676b0 2020-11-28T21:54:53.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.496+0000 7f396d7a3700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3968155ad0 con 0x7f3968144640 2020-11-28T21:54:53.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.496+0000 7f396d7a3700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3968155d70 con 0x7f39680b3800 2020-11-28T21:54:53.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.496+0000 7f396cfa2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3968144640 0x7f39681521c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:53.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.496+0000 7f396cfa2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3968144640 0x7f39681521c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44140/0 (socket says 172.21.15.162:44140) 2020-11-28T21:54:53.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.496+0000 7f396cfa2700 1 -- 172.21.15.162:0/860237204 learned_addr learned my addr 172.21.15.162:0/860237204 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:53.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.496+0000 7f3967fff700 1 --2- 172.21.15.162:0/860237204 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f39680676b0 0x7f396805fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:53.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.496+0000 7f39677fe700 1 --2- 172.21.15.162:0/860237204 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f39680b3800 0x7f39680ad210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:53.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.496+0000 7f396cfa2700 1 -- 172.21.15.162:0/860237204 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f39680b3800 msgr2=0x7f39680ad210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:53.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.496+0000 7f396cfa2700 1 --2- 172.21.15.162:0/860237204 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f39680b3800 0x7f39680ad210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:53.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.496+0000 7f396cfa2700 1 -- 172.21.15.162:0/860237204 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f39680676b0 msgr2=0x7f396805fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:53.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.496+0000 7f396cfa2700 1 --2- 172.21.15.162:0/860237204 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f39680676b0 0x7f396805fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:53.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.496+0000 7f396cfa2700 1 -- 172.21.15.162:0/860237204 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3968151a80 con 0x7f3968144640 2020-11-28T21:54:53.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.496+0000 7f396cfa2700 1 --2- 172.21.15.162:0/860237204 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3968144640 0x7f39681521c0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 rx=0x7f3960007000 tx=0x7f3960008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:53.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.496+0000 7f39657fa700 1 -- 172.21.15.162:0/860237204 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3960007790 con 0x7f3968144640 2020-11-28T21:54:53.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.496+0000 7f39657fa700 1 -- 172.21.15.162:0/860237204 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39600078f0 con 0x7f3968144640 2020-11-28T21:54:53.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.496+0000 7f396d7a3700 1 -- 172.21.15.162:0/860237204 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3968151d20 con 0x7f3968144640 2020-11-28T21:54:53.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.496+0000 7f39657fa700 1 -- 172.21.15.162:0/860237204 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3960007c50 con 0x7f3968144640 2020-11-28T21:54:53.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.496+0000 7f396d7a3700 1 -- 172.21.15.162:0/860237204 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39680ad920 con 0x7f3968144640 2020-11-28T21:54:53.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.500+0000 7f396d7a3700 1 -- 172.21.15.162:0/860237204 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3968057050 con 0x7f3968144640 2020-11-28T21:54:53.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.500+0000 7f39657fa700 1 -- 172.21.15.162:0/860237204 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f3960019030 con 0x7f3968144640 2020-11-28T21:54:53.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.500+0000 7f39657fa700 1 --2- 172.21.15.162:0/860237204 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f39480554e0 0x7f39480579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:53.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.500+0000 7f39657fa700 1 -- 172.21.15.162:0/860237204 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f396006de00 con 0x7f3968144640 2020-11-28T21:54:53.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.500+0000 7f3967fff700 1 --2- 172.21.15.162:0/860237204 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f39480554e0 0x7f39480579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:53.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.500+0000 7f3967fff700 1 --2- 172.21.15.162:0/860237204 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f39480554e0 0x7f39480579a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f395800c000 tx=0x7f3958008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:53.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.504+0000 7f39657fa700 1 -- 172.21.15.162:0/860237204 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f3960044100 con 0x7f3968144640 2020-11-28T21:54:53.652 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.648+0000 7f396d7a3700 1 -- 172.21.15.162:0/860237204 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3968151eb0 con 0x7f3968144640 2020-11-28T21:54:53.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.652+0000 7f39657fa700 1 -- 172.21.15.162:0/860237204 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+16148 (secure 0 0 0) 0x7f3960047720 con 0x7f3968144640 2020-11-28T21:54:53.655 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:54:53.655 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":119,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:54:52.374598+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:54:49.956401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2020-11-28T21:54:50.370123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:54:53.656 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.652+0000 7f396d7a3700 1 -- 172.21.15.162:0/860237204 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f39480554e0 msgr2=0x7f39480579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:53.657 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.652+0000 7f396d7a3700 1 --2- 172.21.15.162:0/860237204 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f39480554e0 0x7f39480579a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f395800c000 tx=0x7f3958008040).stop 2020-11-28T21:54:53.657 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.652+0000 7f396d7a3700 1 -- 172.21.15.162:0/860237204 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3968144640 msgr2=0x7f39681521c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:53.657 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.652+0000 7f396d7a3700 1 --2- 172.21.15.162:0/860237204 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3968144640 0x7f39681521c0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 rx=0x7f3960007000 tx=0x7f3960008040).stop 2020-11-28T21:54:53.657 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.652+0000 7f396d7a3700 1 -- 172.21.15.162:0/860237204 shutdown_connections 2020-11-28T21:54:53.658 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.652+0000 7f396d7a3700 1 --2- 172.21.15.162:0/860237204 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f39480554e0 0x7f39480579a0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:53.658 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.652+0000 7f396d7a3700 1 --2- 172.21.15.162:0/860237204 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3968144640 0x7f39681521c0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:53.658 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.652+0000 7f396d7a3700 1 --2- 172.21.15.162:0/860237204 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f39680676b0 0x7f396805fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:53.658 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.652+0000 7f396d7a3700 1 --2- 172.21.15.162:0/860237204 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f39680b3800 0x7f39680ad210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:53.659 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.652+0000 7f396d7a3700 1 -- 172.21.15.162:0/860237204 >> 172.21.15.162:0/860237204 conn(0x7f396805b970 msgr2=0x7f39680b1790 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:53.659 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.652+0000 7f396d7a3700 1 -- 172.21.15.162:0/860237204 shutdown_connections 2020-11-28T21:54:53.659 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.652+0000 7f396d7a3700 1 -- 172.21.15.162:0/860237204 wait complete. 2020-11-28T21:54:53.668 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-11-28T21:54:53.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.788+0000 7f6650dab700 1 Processor -- start 2020-11-28T21:54:53.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.788+0000 7f6650dab700 1 -- start start 2020-11-28T21:54:53.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.788+0000 7f6650dab700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f664c0b2560 0x7f664c156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:53.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.788+0000 7f6650dab700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f664c0ab0b0 con 0x7f664c05fa90 2020-11-28T21:54:53.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.788+0000 7f6650dab700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f664c0b3260 con 0x7f664c0b2e80 2020-11-28T21:54:53.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.788+0000 7f6650dab700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f664c0ab2e0 con 0x7f664c0b2560 2020-11-28T21:54:53.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.788+0000 7f664bfff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f664c0b2e80 0x7f664c05f380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:50988/0 (socket says 172.21.15.162:50988) 2020-11-28T21:54:53.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.788+0000 7f664bfff700 1 -- 172.21.15.162:0/1251672592 learned_addr learned my addr 172.21.15.162:0/1251672592 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:53.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.788+0000 7f664b7fe700 1 --2- 172.21.15.162:0/1251672592 >> v2:172.21.15.118:3300/0 conn(0x7f664c0b2560 0x7f664c156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:53.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.788+0000 7f664affd700 1 -- 172.21.15.162:0/1251672592 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 281752033 0 0) 0x7f664c0b3260 con 0x7f664c0b2e80 2020-11-28T21:54:53.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.788+0000 7f664affd700 1 -- 172.21.15.162:0/1251672592 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6634003230 con 0x7f664c0b2e80 2020-11-28T21:54:53.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.788+0000 7f664affd700 1 -- 172.21.15.162:0/1251672592 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2565951259 0 0) 0x7f6634003230 con 0x7f664c0b2e80 2020-11-28T21:54:53.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.788+0000 7f664affd700 1 -- 172.21.15.162:0/1251672592 >> v2:172.21.15.118:3300/0 conn(0x7f664c0b2560 msgr2=0x7f664c156ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:53.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.788+0000 7f664affd700 1 --2- 172.21.15.162:0/1251672592 >> v2:172.21.15.118:3300/0 conn(0x7f664c0b2560 0x7f664c156ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:53.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.788+0000 7f664affd700 1 -- 172.21.15.162:0/1251672592 >> v1:172.21.15.118:6789/0 conn(0x7f664c05fa90 legacy=0x7f664c1593c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:53.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.788+0000 7f664affd700 1 -- 172.21.15.162:0/1251672592 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f664c0256d0 con 0x7f664c0b2e80 2020-11-28T21:54:53.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.788+0000 7f664affd700 1 -- 172.21.15.162:0/1251672592 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f663c0016d0 con 0x7f664c0b2e80 2020-11-28T21:54:53.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.788+0000 7f664affd700 1 -- 172.21.15.162:0/1251672592 >> v1:172.21.15.162:6789/0 conn(0x7f664c0b2e80 legacy=0x7f664c05f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:53.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.788+0000 7f664affd700 1 --2- 172.21.15.162:0/1251672592 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6634003720 0x7f6634003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:53.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.788+0000 7f664affd700 1 --2- 172.21.15.162:0/1251672592 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f66340044d0 0x7f6634004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:53.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.788+0000 7f664affd700 1 --2- 172.21.15.162:0/1251672592 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6634004e40 0x7f663400b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:53.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.792+0000 7f664affd700 1 -- 172.21.15.162:0/1251672592 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664c0ab2e0 con 0x7f6634003720 2020-11-28T21:54:53.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.792+0000 7f664affd700 1 -- 172.21.15.162:0/1251672592 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6634004250 con 0x7f6634004e40 2020-11-28T21:54:53.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.792+0000 7f664affd700 1 -- 172.21.15.162:0/1251672592 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663404fc20 con 0x7f66340044d0 2020-11-28T21:54:53.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.792+0000 7f665200d700 1 --2- 172.21.15.162:0/1251672592 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6634004e40 0x7f663400b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:53.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.792+0000 7f664b7fe700 1 --2- 172.21.15.162:0/1251672592 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f66340044d0 0x7f6634004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:53.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.792+0000 7f664bfff700 1 --2- 172.21.15.162:0/1251672592 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6634003720 0x7f6634003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:53.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.792+0000 7f665200d700 1 -- 172.21.15.162:0/1251672592 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f66340044d0 msgr2=0x7f6634004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:53.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.792+0000 7f665200d700 1 --2- 172.21.15.162:0/1251672592 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f66340044d0 0x7f6634004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:53.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.792+0000 7f665200d700 1 -- 172.21.15.162:0/1251672592 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6634003720 msgr2=0x7f6634003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:53.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.792+0000 7f665200d700 1 --2- 172.21.15.162:0/1251672592 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6634003720 0x7f6634003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:53.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.792+0000 7f665200d700 1 -- 172.21.15.162:0/1251672592 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6634003230 con 0x7f6634004e40 2020-11-28T21:54:53.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.792+0000 7f665200d700 1 -- 172.21.15.162:0/1251672592 reap_dead start 2020-11-28T21:54:53.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.792+0000 7f665200d700 1 --2- 172.21.15.162:0/1251672592 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6634004e40 0x7f663400b720 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 rx=0x7f664c063710 tx=0x7f6644007ee0).ready entity=mon.1 client_cookie=95505ff34a583ef server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:53.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.792+0000 7f664affd700 1 -- 172.21.15.162:0/1251672592 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f664400c9d0 con 0x7f6634004e40 2020-11-28T21:54:53.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.792+0000 7f664affd700 1 -- 172.21.15.162:0/1251672592 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f664400cb30 con 0x7f6634004e40 2020-11-28T21:54:53.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.792+0000 7f664affd700 1 -- 172.21.15.162:0/1251672592 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f664400ce90 con 0x7f6634004e40 2020-11-28T21:54:53.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.792+0000 7f6650dab700 1 -- 172.21.15.162:0/1251672592 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6634004e40 msgr2=0x7f663400b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:53.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.792+0000 7f6650dab700 1 --2- 172.21.15.162:0/1251672592 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6634004e40 0x7f663400b720 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 rx=0x7f664c063710 tx=0x7f6644007ee0).stop 2020-11-28T21:54:53.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.792+0000 7f6650dab700 1 -- 172.21.15.162:0/1251672592 shutdown_connections 2020-11-28T21:54:53.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.792+0000 7f6650dab700 1 --2- 172.21.15.162:0/1251672592 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6634004e40 0x7f663400b720 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:53.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.792+0000 7f6650dab700 1 -- 172.21.15.162:0/1251672592 >> 172.21.15.162:0/1251672592 conn(0x7f664c063b60 msgr2=0x7f664c063f40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:53.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.792+0000 7f6650dab700 1 -- 172.21.15.162:0/1251672592 shutdown_connections 2020-11-28T21:54:53.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.792+0000 7f6650dab700 1 -- 172.21.15.162:0/1251672592 wait complete. 2020-11-28T21:54:53.811 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.796+0000 7f6650dab700 1 Processor -- start 2020-11-28T21:54:53.811 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.796+0000 7f6650dab700 1 -- start start 2020-11-28T21:54:53.811 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.796+0000 7f6650dab700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6634004e40 0x7f664c13d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:53.812 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.796+0000 7f6650dab700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f664c0af850 0x7f664c08fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:53.812 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.796+0000 7f6650dab700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f664c13a6d0 0x7f664c13ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:53.812 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.796+0000 7f6650dab700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664c13b070 con 0x7f664c13a6d0 2020-11-28T21:54:53.812 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.796+0000 7f6650dab700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664c09c0c0 con 0x7f664c0af850 2020-11-28T21:54:53.812 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.796+0000 7f6650dab700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f664c09ad50 con 0x7f6634004e40 2020-11-28T21:54:53.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.796+0000 7f664b7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f664c0af850 0x7f664c08fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:53.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.796+0000 7f664b7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f664c0af850 0x7f664c08fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44156/0 (socket says 172.21.15.162:44156) 2020-11-28T21:54:53.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.796+0000 7f664b7fe700 1 -- 172.21.15.162:0/1775480504 learned_addr learned my addr 172.21.15.162:0/1775480504 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:53.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.796+0000 7f664bfff700 1 --2- 172.21.15.162:0/1775480504 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6634004e40 0x7f664c13d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:53.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.796+0000 7f665200d700 1 --2- 172.21.15.162:0/1775480504 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f664c13a6d0 0x7f664c13ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:53.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.796+0000 7f664b7fe700 1 -- 172.21.15.162:0/1775480504 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6634004e40 msgr2=0x7f664c13d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:53.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.796+0000 7f664b7fe700 1 --2- 172.21.15.162:0/1775480504 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6634004e40 0x7f664c13d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:53.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.796+0000 7f664b7fe700 1 -- 172.21.15.162:0/1775480504 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f664c13a6d0 msgr2=0x7f664c13ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:53.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.796+0000 7f664b7fe700 1 --2- 172.21.15.162:0/1775480504 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f664c13a6d0 0x7f664c13ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:53.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.796+0000 7f664b7fe700 1 -- 172.21.15.162:0/1775480504 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f664c09aff0 con 0x7f664c0af850 2020-11-28T21:54:53.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.796+0000 7f665200d700 1 --2- 172.21.15.162:0/1775480504 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f664c13a6d0 0x7f664c13ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:54:53.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.796+0000 7f664bfff700 1 --2- 172.21.15.162:0/1775480504 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6634004e40 0x7f664c13d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:54:53.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.796+0000 7f664b7fe700 1 --2- 172.21.15.162:0/1775480504 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f664c0af850 0x7f664c08fdb0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 rx=0x7f6640003070 tx=0x7f6640008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:53.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.796+0000 7f6648ff9700 1 -- 172.21.15.162:0/1775480504 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f664000ea90 con 0x7f664c0af850 2020-11-28T21:54:53.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.796+0000 7f6648ff9700 1 -- 172.21.15.162:0/1775480504 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f664000ebf0 con 0x7f664c0af850 2020-11-28T21:54:53.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.796+0000 7f6650dab700 1 -- 172.21.15.162:0/1775480504 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f664c08f910 con 0x7f664c0af850 2020-11-28T21:54:53.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.796+0000 7f6648ff9700 1 -- 172.21.15.162:0/1775480504 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6640014470 con 0x7f664c0af850 2020-11-28T21:54:53.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.796+0000 7f6650dab700 1 -- 172.21.15.162:0/1775480504 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f664c0902a0 con 0x7f664c0af850 2020-11-28T21:54:53.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.800+0000 7f6650dab700 1 -- 172.21.15.162:0/1775480504 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f664c09ab90 con 0x7f664c0af850 2020-11-28T21:54:53.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.800+0000 7f6648ff9700 1 -- 172.21.15.162:0/1775480504 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f66400145d0 con 0x7f664c0af850 2020-11-28T21:54:53.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.800+0000 7f6648ff9700 1 --2- 172.21.15.162:0/1775480504 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f662c055320 0x7f662c0577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:53.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.800+0000 7f664bfff700 1 --2- 172.21.15.162:0/1775480504 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f662c055320 0x7f662c0577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:53.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.800+0000 7f6648ff9700 1 -- 172.21.15.162:0/1775480504 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f664006ea90 con 0x7f664c0af850 2020-11-28T21:54:53.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.800+0000 7f664bfff700 1 --2- 172.21.15.162:0/1775480504 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f662c055320 0x7f662c0577e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7f663c00d000 tx=0x7f663c009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:53.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.808+0000 7f6648ff9700 1 -- 172.21.15.162:0/1775480504 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6640044c20 con 0x7f664c0af850 2020-11-28T21:54:53.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:53.996+0000 7f6650dab700 1 -- 172.21.15.162:0/1775480504 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f664c08faa0 con 0x7f664c0af850 2020-11-28T21:54:54.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.004+0000 7f6648ff9700 1 -- 172.21.15.162:0/1775480504 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v27) v1 ==== 184+0+0 (secure 0 0 0) 0x7f6640048240 con 0x7f664c0af850 2020-11-28T21:54:54.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.008+0000 7f6650dab700 1 -- 172.21.15.162:0/1775480504 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f662c055320 msgr2=0x7f662c0577e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:54.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.008+0000 7f6650dab700 1 --2- 172.21.15.162:0/1775480504 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f662c055320 0x7f662c0577e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7f663c00d000 tx=0x7f663c009040).stop 2020-11-28T21:54:54.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.008+0000 7f6650dab700 1 -- 172.21.15.162:0/1775480504 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f664c0af850 msgr2=0x7f664c08fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:54.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.008+0000 7f6650dab700 1 --2- 172.21.15.162:0/1775480504 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f664c0af850 0x7f664c08fdb0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 rx=0x7f6640003070 tx=0x7f6640008040).stop 2020-11-28T21:54:54.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.008+0000 7f6650dab700 1 -- 172.21.15.162:0/1775480504 shutdown_connections 2020-11-28T21:54:54.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.008+0000 7f6650dab700 1 --2- 172.21.15.162:0/1775480504 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f662c055320 0x7f662c0577e0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:54.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.008+0000 7f6650dab700 1 --2- 172.21.15.162:0/1775480504 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6634004e40 0x7f664c13d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:54.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.008+0000 7f6650dab700 1 --2- 172.21.15.162:0/1775480504 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f664c13a6d0 0x7f664c13ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:54.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.008+0000 7f6650dab700 1 --2- 172.21.15.162:0/1775480504 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f664c0af850 0x7f664c08fdb0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:54.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.008+0000 7f6650dab700 1 -- 172.21.15.162:0/1775480504 >> 172.21.15.162:0/1775480504 conn(0x7f664c063b60 msgr2=0x7f664c0ab4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:54.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.008+0000 7f6650dab700 1 -- 172.21.15.162:0/1775480504 shutdown_connections 2020-11-28T21:54:54.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.008+0000 7f6650dab700 1 -- 172.21.15.162:0/1775480504 wait complete. 2020-11-28T21:54:54.014 INFO:teuthology.orchestra.run.smithi162.stderr:updated caps for client.0 2020-11-28T21:54:54.023 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-11-28T21:54:54.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.148+0000 7f54ba47b700 1 Processor -- start 2020-11-28T21:54:54.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.148+0000 7f54ba47b700 1 -- start start 2020-11-28T21:54:54.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.148+0000 7f54ba47b700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f54b40b2e80 0x7f54b405f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:54.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.148+0000 7f54ba47b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f54b40acb40 0x7f54b405f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:54.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.148+0000 7f54ba47b700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54b40ab0b0 con 0x7f54b40b2560 2020-11-28T21:54:54.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.148+0000 7f54ba47b700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54b405fd60 con 0x7f54b40b2e80 2020-11-28T21:54:54.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.148+0000 7f54ba47b700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54b40b3260 con 0x7f54b40acb40 2020-11-28T21:54:54.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.148+0000 7f54bb6dd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f54b40acb40 0x7f54b405f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:54.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.148+0000 7f54b9479700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f54b40b2560 0x7f54b4060d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51008/0 (socket says 172.21.15.162:51008) 2020-11-28T21:54:54.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.148+0000 7f54b9479700 1 -- 172.21.15.162:0/1423377996 learned_addr learned my addr 172.21.15.162:0/1423377996 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:54.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.148+0000 7f54b9c7a700 1 --2- 172.21.15.162:0/1423377996 >> v2:172.21.15.118:3300/0 conn(0x7f54b40b2e80 0x7f54b405f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:54.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.148+0000 7f54bb6dd700 1 -- 172.21.15.162:0/1423377996 >> v2:172.21.15.118:3300/0 conn(0x7f54b40b2e80 msgr2=0x7f54b405f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:54.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.148+0000 7f54bb6dd700 1 --2- 172.21.15.162:0/1423377996 >> v2:172.21.15.118:3300/0 conn(0x7f54b40b2e80 0x7f54b405f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:54.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.148+0000 7f54bb6dd700 1 -- 172.21.15.162:0/1423377996 >> v1:172.21.15.162:6789/0 conn(0x7f54b40b2560 legacy=0x7f54b4060d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:54.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.148+0000 7f54bb6dd700 1 -- 172.21.15.162:0/1423377996 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b40ab2e0 con 0x7f54b40acb40 2020-11-28T21:54:54.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.148+0000 7f54b8c78700 1 -- 172.21.15.162:0/1423377996 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3491967512 0 0) 0x7f54b40ab0b0 con 0x7f54b40b2560 2020-11-28T21:54:54.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.152+0000 7f54bb6dd700 1 --2- 172.21.15.162:0/1423377996 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54b40acb40 0x7f54b405f870 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 rx=0x7f54ac0032b0 tx=0x7f54ac0091a0).ready entity=mon.1 client_cookie=318c397113f137a server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:54.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.152+0000 7f54b8c78700 1 -- 172.21.15.162:0/1423377996 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54ac00b9a0 con 0x7f54b40acb40 2020-11-28T21:54:54.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.152+0000 7f54b8c78700 1 -- 172.21.15.162:0/1423377996 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54ac00bb00 con 0x7f54b40acb40 2020-11-28T21:54:54.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.152+0000 7f54b8c78700 1 -- 172.21.15.162:0/1423377996 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54ac00bc70 con 0x7f54b40acb40 2020-11-28T21:54:54.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.152+0000 7f54ba47b700 1 -- 172.21.15.162:0/1423377996 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54b40acb40 msgr2=0x7f54b405f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:54.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.152+0000 7f54ba47b700 1 --2- 172.21.15.162:0/1423377996 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54b40acb40 0x7f54b405f870 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 rx=0x7f54ac0032b0 tx=0x7f54ac0091a0).stop 2020-11-28T21:54:54.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.152+0000 7f54ba47b700 1 -- 172.21.15.162:0/1423377996 shutdown_connections 2020-11-28T21:54:54.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.152+0000 7f54ba47b700 1 --2- 172.21.15.162:0/1423377996 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54b40acb40 0x7f54b405f870 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:54.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.152+0000 7f54ba47b700 1 --2- 172.21.15.162:0/1423377996 >> v2:172.21.15.118:3300/0 conn(0x7f54b40b2e80 0x7f54b405f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:54.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.152+0000 7f54ba47b700 1 -- 172.21.15.162:0/1423377996 >> 172.21.15.162:0/1423377996 conn(0x7f54b4063b60 msgr2=0x7f54b4063f40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:54.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.152+0000 7f54ba47b700 1 -- 172.21.15.162:0/1423377996 shutdown_connections 2020-11-28T21:54:54.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.152+0000 7f54ba47b700 1 -- 172.21.15.162:0/1423377996 wait complete. 2020-11-28T21:54:54.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.152+0000 7f54ba47b700 1 Processor -- start 2020-11-28T21:54:54.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.152+0000 7f54ba47b700 1 -- start start 2020-11-28T21:54:54.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.156+0000 7f54ba47b700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f54b40acb40 0x7f54b4066d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:54.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.156+0000 7f54ba47b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f54b40b2560 0x7f54b413e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:54.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.156+0000 7f54ba47b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54b40b2e80 0x7f54b413e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:54.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.156+0000 7f54ba47b700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b41429e0 con 0x7f54b40b2560 2020-11-28T21:54:54.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.156+0000 7f54ba47b700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b4132c10 con 0x7f54b40b2e80 2020-11-28T21:54:54.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.156+0000 7f54ba47b700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b4132eb0 con 0x7f54b40acb40 2020-11-28T21:54:54.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.156+0000 7f54bb6dd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54b40b2e80 0x7f54b413e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:54.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.156+0000 7f54b9c7a700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f54b40acb40 0x7f54b4066d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:54.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.156+0000 7f54bb6dd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54b40b2e80 0x7f54b413e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44174/0 (socket says 172.21.15.162:44174) 2020-11-28T21:54:54.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.156+0000 7f54bb6dd700 1 -- 172.21.15.162:0/3638752583 learned_addr learned my addr 172.21.15.162:0/3638752583 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:54.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.156+0000 7f54b9479700 1 --2- 172.21.15.162:0/3638752583 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f54b40b2560 0x7f54b413e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:54.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.156+0000 7f54bb6dd700 1 -- 172.21.15.162:0/3638752583 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f54b40acb40 msgr2=0x7f54b4066d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:54.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.156+0000 7f54bb6dd700 1 --2- 172.21.15.162:0/3638752583 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f54b40acb40 0x7f54b4066d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:54.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.156+0000 7f54bb6dd700 1 -- 172.21.15.162:0/3638752583 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f54b40b2560 msgr2=0x7f54b413e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:54.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.156+0000 7f54bb6dd700 1 --2- 172.21.15.162:0/3638752583 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f54b40b2560 0x7f54b413e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:54.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.156+0000 7f54bb6dd700 1 -- 172.21.15.162:0/3638752583 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b4142540 con 0x7f54b40b2e80 2020-11-28T21:54:54.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.156+0000 7f54b9c7a700 1 --2- 172.21.15.162:0/3638752583 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f54b40acb40 0x7f54b4066d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:54:54.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.156+0000 7f54bb6dd700 1 --2- 172.21.15.162:0/3638752583 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54b40b2e80 0x7f54b413e990 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 rx=0x7f54b4142b20 tx=0x7f54ac016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:54.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.156+0000 7f54aa7fc700 1 -- 172.21.15.162:0/3638752583 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54ac01f3f0 con 0x7f54b40b2e80 2020-11-28T21:54:54.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.156+0000 7f54ba47b700 1 -- 172.21.15.162:0/3638752583 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54b413ee80 con 0x7f54b40b2e80 2020-11-28T21:54:54.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.156+0000 7f54aa7fc700 1 -- 172.21.15.162:0/3638752583 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54ac01f550 con 0x7f54b40b2e80 2020-11-28T21:54:54.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.156+0000 7f54aa7fc700 1 -- 172.21.15.162:0/3638752583 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54ac01f8b0 con 0x7f54b40b2e80 2020-11-28T21:54:54.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.156+0000 7f54ba47b700 1 -- 172.21.15.162:0/3638752583 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54b4134180 con 0x7f54b40b2e80 2020-11-28T21:54:54.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.160+0000 7f54ba47b700 1 -- 172.21.15.162:0/3638752583 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54b4057050 con 0x7f54b40b2e80 2020-11-28T21:54:54.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.160+0000 7f54aa7fc700 1 -- 172.21.15.162:0/3638752583 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f54ac026030 con 0x7f54b40b2e80 2020-11-28T21:54:54.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.160+0000 7f54aa7fc700 1 --2- 172.21.15.162:0/3638752583 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f549c055440 0x7f549c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:54.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.160+0000 7f54aa7fc700 1 -- 172.21.15.162:0/3638752583 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f54ac06eb90 con 0x7f54b40b2e80 2020-11-28T21:54:54.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.160+0000 7f54b9c7a700 1 --2- 172.21.15.162:0/3638752583 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f549c055440 0x7f549c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:54.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.160+0000 7f54b9c7a700 1 --2- 172.21.15.162:0/3638752583 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f549c055440 0x7f549c057900 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f54b000b000 tx=0x7f54b0006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:54.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.168+0000 7f54aa7fc700 1 -- 172.21.15.162:0/3638752583 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f54ac044d90 con 0x7f54b40b2e80 2020-11-28T21:54:54.355 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.352+0000 7f54ba47b700 1 -- 172.21.15.162:0/3638752583 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f54b4068350 con 0x7f54b40b2e80 2020-11-28T21:54:54.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.360+0000 7f54aa7fc700 1 -- 172.21.15.162:0/3638752583 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v28) v1 ==== 184+0+0 (secure 0 0 0) 0x7f54ac024020 con 0x7f54b40b2e80 2020-11-28T21:54:54.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.364+0000 7f54ba47b700 1 -- 172.21.15.162:0/3638752583 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f549c055440 msgr2=0x7f549c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:54.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.364+0000 7f54ba47b700 1 --2- 172.21.15.162:0/3638752583 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f549c055440 0x7f549c057900 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f54b000b000 tx=0x7f54b0006dc0).stop 2020-11-28T21:54:54.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.364+0000 7f54ba47b700 1 -- 172.21.15.162:0/3638752583 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54b40b2e80 msgr2=0x7f54b413e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:54.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.364+0000 7f54ba47b700 1 --2- 172.21.15.162:0/3638752583 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54b40b2e80 0x7f54b413e990 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 rx=0x7f54b4142b20 tx=0x7f54ac016000).stop 2020-11-28T21:54:54.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.364+0000 7f54ba47b700 1 -- 172.21.15.162:0/3638752583 shutdown_connections 2020-11-28T21:54:54.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.364+0000 7f54ba47b700 1 --2- 172.21.15.162:0/3638752583 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f549c055440 0x7f549c057900 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:54.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.364+0000 7f54ba47b700 1 --2- 172.21.15.162:0/3638752583 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f54b40acb40 0x7f54b4066d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:54.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.364+0000 7f54ba47b700 1 --2- 172.21.15.162:0/3638752583 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54b40b2e80 0x7f54b413e990 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:54.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.364+0000 7f54ba47b700 1 --2- 172.21.15.162:0/3638752583 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f54b40b2560 0x7f54b413e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:54.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.364+0000 7f54ba47b700 1 -- 172.21.15.162:0/3638752583 >> 172.21.15.162:0/3638752583 conn(0x7f54b4063b60 msgr2=0x7f54b4063f40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:54.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.364+0000 7f54ba47b700 1 -- 172.21.15.162:0/3638752583 shutdown_connections 2020-11-28T21:54:54.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.364+0000 7f54ba47b700 1 -- 172.21.15.162:0/3638752583 wait complete. 2020-11-28T21:54:54.370 INFO:teuthology.orchestra.run.smithi162.stderr:updated caps for client.1 2020-11-28T21:54:54.384 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:54:54.485 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.480+0000 7f605ebd7700 1 Processor -- start 2020-11-28T21:54:54.486 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.484+0000 7f605ebd7700 1 -- start start 2020-11-28T21:54:54.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.484+0000 7f605ebd7700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6058146330 0x7f60580b1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:54.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.484+0000 7f605ebd7700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f60580b1bf0 0x7f605805c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:54.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.484+0000 7f605ebd7700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6058146b00 con 0x7f605806aa70 2020-11-28T21:54:54.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.484+0000 7f605ebd7700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6058146da0 con 0x7f60580b1bf0 2020-11-28T21:54:54.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.484+0000 7f605ebd7700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60580b6610 con 0x7f6058146330 2020-11-28T21:54:54.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.484+0000 7f605c973700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6058146330 0x7f60580b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:54.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.484+0000 7f6057fff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f605806aa70 0x7f605805b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51022/0 (socket says 172.21.15.162:51022) 2020-11-28T21:54:54.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.484+0000 7f6057fff700 1 -- 172.21.15.162:0/2884033968 learned_addr learned my addr 172.21.15.162:0/2884033968 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:54.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.484+0000 7f605d174700 1 --2- 172.21.15.162:0/2884033968 >> v2:172.21.15.118:3300/0 conn(0x7f60580b1bf0 0x7f605805c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:54.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.484+0000 7f60577fe700 1 -- 172.21.15.162:0/2884033968 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1235305956 0 0) 0x7f6058146b00 con 0x7f605806aa70 2020-11-28T21:54:54.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.484+0000 7f60577fe700 1 -- 172.21.15.162:0/2884033968 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6038003230 con 0x7f605806aa70 2020-11-28T21:54:54.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.484+0000 7f605c973700 1 -- 172.21.15.162:0/2884033968 >> v2:172.21.15.118:3300/0 conn(0x7f60580b1bf0 msgr2=0x7f605805c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:54.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.484+0000 7f605c973700 1 --2- 172.21.15.162:0/2884033968 >> v2:172.21.15.118:3300/0 conn(0x7f60580b1bf0 0x7f605805c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:54.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.484+0000 7f605c973700 1 -- 172.21.15.162:0/2884033968 >> v1:172.21.15.162:6789/0 conn(0x7f605806aa70 legacy=0x7f605805b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:54.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.484+0000 7f605c973700 1 -- 172.21.15.162:0/2884033968 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60580b0f80 con 0x7f6058146330 2020-11-28T21:54:54.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.484+0000 7f605c973700 1 --2- 172.21.15.162:0/2884033968 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6058146330 0x7f60580b1470 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 rx=0x7f60480032b0 tx=0x7f60480091a0).ready entity=mon.1 client_cookie=dde7de6e4bdc4b8d server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:54.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.484+0000 7f60577fe700 1 -- 172.21.15.162:0/2884033968 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f604800b940 con 0x7f6058146330 2020-11-28T21:54:54.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.484+0000 7f60577fe700 1 -- 172.21.15.162:0/2884033968 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f604800baa0 con 0x7f6058146330 2020-11-28T21:54:54.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.484+0000 7f605ebd7700 1 -- 172.21.15.162:0/2884033968 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6058146330 msgr2=0x7f60580b1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:54.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.484+0000 7f60577fe700 1 -- 172.21.15.162:0/2884033968 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f604800bc10 con 0x7f6058146330 2020-11-28T21:54:54.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.484+0000 7f605ebd7700 1 --2- 172.21.15.162:0/2884033968 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6058146330 0x7f60580b1470 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 rx=0x7f60480032b0 tx=0x7f60480091a0).stop 2020-11-28T21:54:54.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.484+0000 7f605ebd7700 1 -- 172.21.15.162:0/2884033968 shutdown_connections 2020-11-28T21:54:54.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.484+0000 7f605ebd7700 1 --2- 172.21.15.162:0/2884033968 >> v2:172.21.15.118:3300/0 conn(0x7f60580b1bf0 0x7f605805c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:54.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.484+0000 7f605ebd7700 1 --2- 172.21.15.162:0/2884033968 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6058146330 0x7f60580b1470 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:54.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.484+0000 7f605ebd7700 1 -- 172.21.15.162:0/2884033968 >> 172.21.15.162:0/2884033968 conn(0x7f60580b56f0 msgr2=0x7f60580b5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:54.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.484+0000 7f605ebd7700 1 -- 172.21.15.162:0/2884033968 shutdown_connections 2020-11-28T21:54:54.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.484+0000 7f605ebd7700 1 -- 172.21.15.162:0/2884033968 wait complete. 2020-11-28T21:54:54.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.488+0000 7f605ebd7700 1 Processor -- start 2020-11-28T21:54:54.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.488+0000 7f605ebd7700 1 -- start start 2020-11-28T21:54:54.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.488+0000 7f605ebd7700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f605806aa70 0x7f605813f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:54.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.488+0000 7f605ebd7700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60580b1bf0 0x7f60580ab5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:54.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.488+0000 7f605ebd7700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6058146330 0x7f60580abae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:54.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.488+0000 7f605ebd7700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6058155230 con 0x7f605806aa70 2020-11-28T21:54:54.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.488+0000 7f605ebd7700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60581554d0 con 0x7f60580b1bf0 2020-11-28T21:54:54.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.488+0000 7f605ebd7700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6058155770 con 0x7f6058146330 2020-11-28T21:54:54.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.488+0000 7f6057fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60580b1bf0 0x7f60580ab5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:54.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.488+0000 7f6057fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60580b1bf0 0x7f60580ab5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44188/0 (socket says 172.21.15.162:44188) 2020-11-28T21:54:54.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.488+0000 7f6057fff700 1 -- 172.21.15.162:0/168423315 learned_addr learned my addr 172.21.15.162:0/168423315 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:54.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.488+0000 7f605c973700 1 --2- 172.21.15.162:0/168423315 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f605806aa70 0x7f605813f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:54.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.488+0000 7f605d174700 1 --2- 172.21.15.162:0/168423315 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6058146330 0x7f60580abae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:54.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.488+0000 7f6057fff700 1 -- 172.21.15.162:0/168423315 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6058146330 msgr2=0x7f60580abae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:54.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.488+0000 7f6057fff700 1 --2- 172.21.15.162:0/168423315 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6058146330 0x7f60580abae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:54.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.488+0000 7f6057fff700 1 -- 172.21.15.162:0/168423315 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f605806aa70 msgr2=0x7f605813f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:54.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.488+0000 7f6057fff700 1 --2- 172.21.15.162:0/168423315 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f605806aa70 0x7f605813f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:54.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.488+0000 7f6057fff700 1 -- 172.21.15.162:0/168423315 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6058155a10 con 0x7f60580b1bf0 2020-11-28T21:54:54.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.488+0000 7f605d174700 1 --2- 172.21.15.162:0/168423315 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6058146330 0x7f60580abae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:54:54.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.488+0000 7f6057fff700 1 --2- 172.21.15.162:0/168423315 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60580b1bf0 0x7f60580ab5f0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 rx=0x7f60400010d0 tx=0x7f6040007a30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:54.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.488+0000 7f6055ffb700 1 -- 172.21.15.162:0/168423315 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f60400072e0 con 0x7f60580b1bf0 2020-11-28T21:54:54.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.488+0000 7f6055ffb700 1 -- 172.21.15.162:0/168423315 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6040007440 con 0x7f60580b1bf0 2020-11-28T21:54:54.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.488+0000 7f6055ffb700 1 -- 172.21.15.162:0/168423315 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f60400057d0 con 0x7f60580b1bf0 2020-11-28T21:54:54.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.488+0000 7f605ebd7700 1 -- 172.21.15.162:0/168423315 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6058141ff0 con 0x7f60580b1bf0 2020-11-28T21:54:54.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.488+0000 7f605ebd7700 1 -- 172.21.15.162:0/168423315 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60580abfd0 con 0x7f60580b1bf0 2020-11-28T21:54:54.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.492+0000 7f605ebd7700 1 -- 172.21.15.162:0/168423315 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6058057050 con 0x7f60580b1bf0 2020-11-28T21:54:54.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.492+0000 7f6055ffb700 1 -- 172.21.15.162:0/168423315 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6040010070 con 0x7f60580b1bf0 2020-11-28T21:54:54.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.492+0000 7f6055ffb700 1 --2- 172.21.15.162:0/168423315 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f603c055440 0x7f603c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:54.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.492+0000 7f6055ffb700 1 -- 172.21.15.162:0/168423315 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f604006d530 con 0x7f60580b1bf0 2020-11-28T21:54:54.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.492+0000 7f605c973700 1 --2- 172.21.15.162:0/168423315 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f603c055440 0x7f603c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:54.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.492+0000 7f605c973700 1 --2- 172.21.15.162:0/168423315 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f603c055440 0x7f603c057900 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f604801c000 tx=0x7f6048016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:54.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.500+0000 7f6055ffb700 1 -- 172.21.15.162:0/168423315 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6040043830 con 0x7f60580b1bf0 2020-11-28T21:54:54.690 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.688+0000 7f605ebd7700 1 -- 172.21.15.162:0/168423315 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6058068350 con 0x7f60580b1bf0 2020-11-28T21:54:54.691 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.688+0000 7f6055ffb700 1 -- 172.21.15.162:0/168423315 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+2504 (secure 0 0 0) 0x7f6040018070 con 0x7f60580b1bf0 2020-11-28T21:54:54.691 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:54:54.691 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":57,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7257,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/1527184432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1527184432},{"type":"v1","addr":"172.21.15.118:6833","nonce":1527184432}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":54},{"gid":7477,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6835/2749838911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2749838911},{"type":"v1","addr":"172.21.15.162:6835","nonce":2749838911}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":55}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:54:51.371126+0000","modified":"2020-11-28T21:54:53.015631+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7489},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7489":{"gid":7489,"name":"c","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/1061393314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1061393314},{"type":"v1","addr":"172.21.15.162:6837","nonce":1061393314}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2020-11-28T21:54:54.693 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.688+0000 7f605ebd7700 1 -- 172.21.15.162:0/168423315 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f603c055440 msgr2=0x7f603c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:54.693 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.688+0000 7f605ebd7700 1 --2- 172.21.15.162:0/168423315 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f603c055440 0x7f603c057900 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f604801c000 tx=0x7f6048016000).stop 2020-11-28T21:54:54.694 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.688+0000 7f605ebd7700 1 -- 172.21.15.162:0/168423315 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60580b1bf0 msgr2=0x7f60580ab5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:54.694 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.688+0000 7f605ebd7700 1 --2- 172.21.15.162:0/168423315 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60580b1bf0 0x7f60580ab5f0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 rx=0x7f60400010d0 tx=0x7f6040007a30).stop 2020-11-28T21:54:54.694 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.688+0000 7f605ebd7700 1 -- 172.21.15.162:0/168423315 shutdown_connections 2020-11-28T21:54:54.694 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.688+0000 7f605ebd7700 1 --2- 172.21.15.162:0/168423315 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f603c055440 0x7f603c057900 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:54.695 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.688+0000 7f605ebd7700 1 --2- 172.21.15.162:0/168423315 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6058146330 0x7f60580abae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:54.695 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.688+0000 7f605ebd7700 1 --2- 172.21.15.162:0/168423315 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f605806aa70 0x7f605813f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:54.695 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.688+0000 7f605ebd7700 1 --2- 172.21.15.162:0/168423315 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60580b1bf0 0x7f60580ab5f0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:54.695 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.688+0000 7f605ebd7700 1 -- 172.21.15.162:0/168423315 >> 172.21.15.162:0/168423315 conn(0x7f60580b56f0 msgr2=0x7f60580aeda0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:54.696 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.688+0000 7f605ebd7700 1 -- 172.21.15.162:0/168423315 shutdown_connections 2020-11-28T21:54:54.696 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:54.688+0000 7f605ebd7700 1 -- 172.21.15.162:0/168423315 wait complete. 2020-11-28T21:54:54.696 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 57 2020-11-28T21:54:55.007 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 57, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-28T21:54:51.371126+0000', 'modified': '2020-11-28T21:54:53.015631+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7489}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7489': {'gid': 7489, 'name': 'c', 'rank': 0, 'incarnation': 55, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6837/1061393314', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6836', 'nonce': 1061393314}, {'type': 'v1', 'addr': '172.21.15.162:6837', 'nonce': 1061393314}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-28T21:54:55.007 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-28T21:54:55.008 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-28T21:54:55.142 INFO:teuthology.orchestra.run.smithi162.stdout:{"cluster_fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","whoami":0,"id":7489,"want_state":"up:active","state":"up:active","rank_uptime":3.753668276,"mdsmap_epoch":57,"osdmap_epoch":119,"osdmap_epoch_barrier":119,"uptime":8.0723697019999996} 2020-11-28T21:54:55.143 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", "id": 7489, "mdsmap_epoch": 57, "osdmap_epoch": 119, "osdmap_epoch_barrier": 119, "rank_uptime": 3.753668276, "state": "up:active", "uptime": 8.072369702, "want_state": "up:active", "whoami": 0 } 2020-11-28T21:54:55.143 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:54:55.253 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.248+0000 7f1bdb264700 1 Processor -- start 2020-11-28T21:54:55.253 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.248+0000 7f1bdb264700 1 -- start start 2020-11-28T21:54:55.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.252+0000 7f1bdb264700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1bd40603d0 0x7f1bd40b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:55.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.252+0000 7f1bdb264700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1bd40b4c70 0x7f1bd4143020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:55.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.252+0000 7f1bdb264700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bd40b4110 con 0x7f1bd4142a00 2020-11-28T21:54:55.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.252+0000 7f1bdb264700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bd4060f60 con 0x7f1bd40603d0 2020-11-28T21:54:55.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.252+0000 7f1bdb264700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd40b43f0 con 0x7f1bd40b4c70 2020-11-28T21:54:55.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.252+0000 7f1bda262700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1bd40603d0 0x7f1bd40b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:55.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.252+0000 7f1bdaa63700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1bd40b4c70 0x7f1bd4143020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:55.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.252+0000 7f1bda262700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1bd40603d0 0x7f1bd40b3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44194/0 (socket says 172.21.15.162:44194) 2020-11-28T21:54:55.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.252+0000 7f1bda262700 1 -- 172.21.15.162:0/1967257927 learned_addr learned my addr 172.21.15.162:0/1967257927 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:55.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.252+0000 7f1bd9260700 1 -- 172.21.15.162:0/1967257927 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2180589051 0 0) 0x7f1bd40b4110 con 0x7f1bd4142a00 2020-11-28T21:54:55.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.252+0000 7f1bda262700 1 -- 172.21.15.162:0/1967257927 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1bd40b4c70 msgr2=0x7f1bd4143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:55.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.252+0000 7f1bda262700 1 --2- 172.21.15.162:0/1967257927 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1bd40b4c70 0x7f1bd4143020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:55.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.252+0000 7f1bda262700 1 -- 172.21.15.162:0/1967257927 >> v1:172.21.15.118:6789/0 conn(0x7f1bd4142a00 legacy=0x7f1bd409cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:55.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.252+0000 7f1bda262700 1 -- 172.21.15.162:0/1967257927 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd40b4690 con 0x7f1bd40603d0 2020-11-28T21:54:55.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.252+0000 7f1bdaa63700 1 --2- 172.21.15.162:0/1967257927 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1bd40b4c70 0x7f1bd4143020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:54:55.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.252+0000 7f1bda262700 1 --2- 172.21.15.162:0/1967257927 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1bd40603d0 0x7f1bd40b3a70 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 rx=0x7f1bd0002f30 tx=0x7f1bd0006e80).ready entity=mon.1 client_cookie=ba5cca67da68866a server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:55.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.252+0000 7f1bd9260700 1 -- 172.21.15.162:0/1967257927 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1bd00097f0 con 0x7f1bd40603d0 2020-11-28T21:54:55.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.252+0000 7f1bd9260700 1 -- 172.21.15.162:0/1967257927 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd0009950 con 0x7f1bd40603d0 2020-11-28T21:54:55.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.252+0000 7f1bdb264700 1 -- 172.21.15.162:0/1967257927 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1bd40603d0 msgr2=0x7f1bd40b3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:55.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.252+0000 7f1bd9260700 1 -- 172.21.15.162:0/1967257927 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1bd0009cb0 con 0x7f1bd40603d0 2020-11-28T21:54:55.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.252+0000 7f1bdb264700 1 --2- 172.21.15.162:0/1967257927 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1bd40603d0 0x7f1bd40b3a70 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 rx=0x7f1bd0002f30 tx=0x7f1bd0006e80).stop 2020-11-28T21:54:55.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.252+0000 7f1bdb264700 1 -- 172.21.15.162:0/1967257927 shutdown_connections 2020-11-28T21:54:55.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.252+0000 7f1bdb264700 1 --2- 172.21.15.162:0/1967257927 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1bd40b4c70 0x7f1bd4143020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:55.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.252+0000 7f1bdb264700 1 --2- 172.21.15.162:0/1967257927 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1bd40603d0 0x7f1bd40b3a70 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:55.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.252+0000 7f1bdb264700 1 -- 172.21.15.162:0/1967257927 >> 172.21.15.162:0/1967257927 conn(0x7f1bd4066a40 msgr2=0x7f1bd4066e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:55.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.252+0000 7f1bdb264700 1 -- 172.21.15.162:0/1967257927 shutdown_connections 2020-11-28T21:54:55.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.252+0000 7f1bdb264700 1 -- 172.21.15.162:0/1967257927 wait complete. 2020-11-28T21:54:55.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.256+0000 7f1bdb264700 1 Processor -- start 2020-11-28T21:54:55.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.256+0000 7f1bdb264700 1 -- start start 2020-11-28T21:54:55.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.256+0000 7f1bdb264700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1bd40603d0 0x7f1bd40af870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:55.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.256+0000 7f1bdb264700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1bd40b4c70 0x7f1bd40b7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:55.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.256+0000 7f1bdb264700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1bd40abab0 0x7f1bd40abf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:55.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.256+0000 7f1bdb264700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd4152570 con 0x7f1bd40603d0 2020-11-28T21:54:55.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.256+0000 7f1bdb264700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd4152060 con 0x7f1bd40b4c70 2020-11-28T21:54:55.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.256+0000 7f1bdb264700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd4140520 con 0x7f1bd40abab0 2020-11-28T21:54:55.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.256+0000 7f1bd9a61700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1bd40b4c70 0x7f1bd40b7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:55.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.256+0000 7f1bd9a61700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1bd40b4c70 0x7f1bd40b7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44200/0 (socket says 172.21.15.162:44200) 2020-11-28T21:54:55.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.256+0000 7f1bd9a61700 1 -- 172.21.15.162:0/2658447806 learned_addr learned my addr 172.21.15.162:0/2658447806 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:55.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.256+0000 7f1bdaa63700 1 --2- 172.21.15.162:0/2658447806 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1bd40abab0 0x7f1bd40abf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:55.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.256+0000 7f1bda262700 1 --2- 172.21.15.162:0/2658447806 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1bd40603d0 0x7f1bd40af870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:55.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.256+0000 7f1bd9a61700 1 -- 172.21.15.162:0/2658447806 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1bd40abab0 msgr2=0x7f1bd40abf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:55.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.256+0000 7f1bd9a61700 1 --2- 172.21.15.162:0/2658447806 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1bd40abab0 0x7f1bd40abf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:55.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.256+0000 7f1bd9a61700 1 -- 172.21.15.162:0/2658447806 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1bd40603d0 msgr2=0x7f1bd40af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:55.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.256+0000 7f1bd9a61700 1 --2- 172.21.15.162:0/2658447806 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1bd40603d0 0x7f1bd40af870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:55.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.256+0000 7f1bd9a61700 1 -- 172.21.15.162:0/2658447806 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd41522d0 con 0x7f1bd40b4c70 2020-11-28T21:54:55.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.256+0000 7f1bd9a61700 1 --2- 172.21.15.162:0/2658447806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1bd40b4c70 0x7f1bd40b7fa0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 rx=0x7f1bc4000e90 tx=0x7f1bc4007c20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:55.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.256+0000 7f1bc37fe700 1 -- 172.21.15.162:0/2658447806 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1bc400e4a0 con 0x7f1bd40b4c70 2020-11-28T21:54:55.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.256+0000 7f1bc37fe700 1 -- 172.21.15.162:0/2658447806 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bc400e600 con 0x7f1bd40b4c70 2020-11-28T21:54:55.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.256+0000 7f1bdb264700 1 -- 172.21.15.162:0/2658447806 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bd40b8650 con 0x7f1bd40b4c70 2020-11-28T21:54:55.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.256+0000 7f1bc37fe700 1 -- 172.21.15.162:0/2658447806 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1bc400e960 con 0x7f1bd40b4c70 2020-11-28T21:54:55.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.256+0000 7f1bdb264700 1 -- 172.21.15.162:0/2658447806 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bd40b8d00 con 0x7f1bd40b4c70 2020-11-28T21:54:55.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.260+0000 7f1bdb264700 1 -- 172.21.15.162:0/2658447806 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bd40b8490 con 0x7f1bd40b4c70 2020-11-28T21:54:55.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.260+0000 7f1bc37fe700 1 -- 172.21.15.162:0/2658447806 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1bc4010030 con 0x7f1bd40b4c70 2020-11-28T21:54:55.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.260+0000 7f1bc37fe700 1 --2- 172.21.15.162:0/2658447806 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1bb40554e0 0x7f1bb40579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:55.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.260+0000 7f1bda262700 1 --2- 172.21.15.162:0/2658447806 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1bb40554e0 0x7f1bb40579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:55.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.260+0000 7f1bc37fe700 1 -- 172.21.15.162:0/2658447806 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1bc406ddf0 con 0x7f1bd40b4c70 2020-11-28T21:54:55.273 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.260+0000 7f1bda262700 1 --2- 172.21.15.162:0/2658447806 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1bb40554e0 0x7f1bb40579a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f1bd001c000 tx=0x7f1bd0015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:55.273 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.268+0000 7f1bc37fe700 1 -- 172.21.15.162:0/2658447806 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1bc4043fe0 con 0x7f1bd40b4c70 2020-11-28T21:54:55.412 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.408+0000 7f1bdb264700 1 -- 172.21.15.162:0/2658447806 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1bd4057050 con 0x7f1bd40b4c70 2020-11-28T21:54:55.413 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.412+0000 7f1bc37fe700 1 -- 172.21.15.162:0/2658447806 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+16148 (secure 0 0 0) 0x7f1bc4047600 con 0x7f1bd40b4c70 2020-11-28T21:54:55.414 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:54:55.415 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":119,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:54:52.374598+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:54:49.956401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2020-11-28T21:54:50.370123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:54:55.416 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.412+0000 7f1bdb264700 1 -- 172.21.15.162:0/2658447806 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1bb40554e0 msgr2=0x7f1bb40579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:55.416 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.412+0000 7f1bdb264700 1 --2- 172.21.15.162:0/2658447806 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1bb40554e0 0x7f1bb40579a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f1bd001c000 tx=0x7f1bd0015000).stop 2020-11-28T21:54:55.416 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.412+0000 7f1bdb264700 1 -- 172.21.15.162:0/2658447806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1bd40b4c70 msgr2=0x7f1bd40b7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:55.417 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.412+0000 7f1bdb264700 1 --2- 172.21.15.162:0/2658447806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1bd40b4c70 0x7f1bd40b7fa0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 rx=0x7f1bc4000e90 tx=0x7f1bc4007c20).stop 2020-11-28T21:54:55.417 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.412+0000 7f1bdb264700 1 -- 172.21.15.162:0/2658447806 shutdown_connections 2020-11-28T21:54:55.417 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.412+0000 7f1bdb264700 1 --2- 172.21.15.162:0/2658447806 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1bb40554e0 0x7f1bb40579a0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:55.417 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.412+0000 7f1bdb264700 1 --2- 172.21.15.162:0/2658447806 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1bd40abab0 0x7f1bd40abf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:55.418 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.412+0000 7f1bdb264700 1 --2- 172.21.15.162:0/2658447806 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1bd40603d0 0x7f1bd40af870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:55.418 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.412+0000 7f1bdb264700 1 --2- 172.21.15.162:0/2658447806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1bd40b4c70 0x7f1bd40b7fa0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:55.418 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.412+0000 7f1bdb264700 1 -- 172.21.15.162:0/2658447806 >> 172.21.15.162:0/2658447806 conn(0x7f1bd4066a40 msgr2=0x7f1bd4141770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:55.418 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.412+0000 7f1bdb264700 1 -- 172.21.15.162:0/2658447806 shutdown_connections 2020-11-28T21:54:55.419 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.412+0000 7f1bdb264700 1 -- 172.21.15.162:0/2658447806 wait complete. 2020-11-28T21:54:55.429 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-28T21:54:49.956401+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2020-11-28T21:54:50.370123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-28T21:54:55.430 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:54:55.547 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.544+0000 7fac3f59e700 1 Processor -- start 2020-11-28T21:54:55.548 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.544+0000 7fac3f59e700 1 -- start start 2020-11-28T21:54:55.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.544+0000 7fac3f59e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fac400b2d00 0x7fac4005c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:55.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.544+0000 7fac3f59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fac400b3ab0 0x7fac400b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:55.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.544+0000 7fac3f59e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac4005baf0 con 0x7fac40142ad0 2020-11-28T21:54:55.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.544+0000 7fac3f59e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac400b0e20 con 0x7fac400b2d00 2020-11-28T21:54:55.550 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.544+0000 7fac3f59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac4005bd90 con 0x7fac400b3ab0 2020-11-28T21:54:55.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.544+0000 7fac3dd9b700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fac400b3ab0 0x7fac400b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:55.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.544+0000 7fac3dd9b700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fac400b3ab0 0x7fac400b30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:41430/0 (socket says 172.21.15.162:41430) 2020-11-28T21:54:55.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.544+0000 7fac3dd9b700 1 -- 172.21.15.162:0/1201289693 learned_addr learned my addr 172.21.15.162:0/1201289693 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:55.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.544+0000 7fac3e59c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fac400b2d00 0x7fac4005c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:55.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.548+0000 7fac3d59a700 1 -- 172.21.15.162:0/1201289693 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4284718651 0 0) 0x7fac4005baf0 con 0x7fac40142ad0 2020-11-28T21:54:55.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.548+0000 7fac3dd9b700 1 -- 172.21.15.162:0/1201289693 >> v2:172.21.15.118:3300/0 conn(0x7fac400b2d00 msgr2=0x7fac4005c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:55.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.548+0000 7fac3dd9b700 1 --2- 172.21.15.162:0/1201289693 >> v2:172.21.15.118:3300/0 conn(0x7fac400b2d00 0x7fac4005c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:55.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.548+0000 7fac3dd9b700 1 -- 172.21.15.162:0/1201289693 >> v1:172.21.15.118:6789/0 conn(0x7fac40142ad0 legacy=0x7fac4013fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:55.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.548+0000 7fac3dd9b700 1 -- 172.21.15.162:0/1201289693 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac4005c030 con 0x7fac400b3ab0 2020-11-28T21:54:55.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.548+0000 7fac3dd9b700 1 --2- 172.21.15.162:0/1201289693 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fac400b3ab0 0x7fac400b30e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fac30002f30 tx=0x7fac30006e80).ready entity=mon.2 client_cookie=b9328e66677e6849 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:55.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.548+0000 7fac3d59a700 1 -- 172.21.15.162:0/1201289693 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac300097f0 con 0x7fac400b3ab0 2020-11-28T21:54:55.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.548+0000 7fac3d59a700 1 -- 172.21.15.162:0/1201289693 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac30009950 con 0x7fac400b3ab0 2020-11-28T21:54:55.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.548+0000 7fac3d59a700 1 -- 172.21.15.162:0/1201289693 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac30009cb0 con 0x7fac400b3ab0 2020-11-28T21:54:55.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.548+0000 7fac3f59e700 1 -- 172.21.15.162:0/1201289693 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fac400b3ab0 msgr2=0x7fac400b30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:55.555 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.548+0000 7fac3f59e700 1 --2- 172.21.15.162:0/1201289693 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fac400b3ab0 0x7fac400b30e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fac30002f30 tx=0x7fac30006e80).stop 2020-11-28T21:54:55.555 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.548+0000 7fac3f59e700 1 -- 172.21.15.162:0/1201289693 shutdown_connections 2020-11-28T21:54:55.555 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.548+0000 7fac3f59e700 1 --2- 172.21.15.162:0/1201289693 >> v2:172.21.15.118:3300/0 conn(0x7fac400b2d00 0x7fac4005c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:55.555 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.548+0000 7fac3f59e700 1 --2- 172.21.15.162:0/1201289693 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fac400b3ab0 0x7fac400b30e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:55.555 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.548+0000 7fac3f59e700 1 -- 172.21.15.162:0/1201289693 >> 172.21.15.162:0/1201289693 conn(0x7fac400604c0 msgr2=0x7fac400608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:55.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.548+0000 7fac3f59e700 1 -- 172.21.15.162:0/1201289693 shutdown_connections 2020-11-28T21:54:55.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.548+0000 7fac3f59e700 1 -- 172.21.15.162:0/1201289693 wait complete. 2020-11-28T21:54:55.558 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.552+0000 7fac3f59e700 1 Processor -- start 2020-11-28T21:54:55.559 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.552+0000 7fac3f59e700 1 -- start start 2020-11-28T21:54:55.559 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.552+0000 7fac3f59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fac400b2d00 0x7fac4013f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:55.559 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.552+0000 7fac3f59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fac400b3ab0 0x7fac40144d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:55.559 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.552+0000 7fac3f59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac40142ad0 0x7fac40148b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:55.560 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.552+0000 7fac3f59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac40148170 con 0x7fac400b3ab0 2020-11-28T21:54:55.560 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.552+0000 7fac3f59e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac40151940 con 0x7fac40142ad0 2020-11-28T21:54:55.560 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.552+0000 7fac3f59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac40151be0 con 0x7fac400b2d00 2020-11-28T21:54:55.560 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.552+0000 7fac3ed9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac40142ad0 0x7fac40148b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:55.560 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.552+0000 7fac3ed9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac40142ad0 0x7fac40148b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44216/0 (socket says 172.21.15.162:44216) 2020-11-28T21:54:55.561 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.552+0000 7fac3ed9d700 1 -- 172.21.15.162:0/4236522091 learned_addr learned my addr 172.21.15.162:0/4236522091 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:55.561 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.552+0000 7fac3e59c700 1 --2- 172.21.15.162:0/4236522091 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fac400b2d00 0x7fac4013f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:55.561 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.552+0000 7fac3dd9b700 1 --2- 172.21.15.162:0/4236522091 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fac400b3ab0 0x7fac40144d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:55.561 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.552+0000 7fac3ed9d700 1 -- 172.21.15.162:0/4236522091 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fac400b2d00 msgr2=0x7fac4013f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:55.561 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.552+0000 7fac3ed9d700 1 --2- 172.21.15.162:0/4236522091 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fac400b2d00 0x7fac4013f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:55.563 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.552+0000 7fac3ed9d700 1 -- 172.21.15.162:0/4236522091 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fac400b3ab0 msgr2=0x7fac40144d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:55.563 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.552+0000 7fac3ed9d700 1 --2- 172.21.15.162:0/4236522091 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fac400b3ab0 0x7fac40144d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:55.563 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.552+0000 7fac3ed9d700 1 -- 172.21.15.162:0/4236522091 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac401483d0 con 0x7fac40142ad0 2020-11-28T21:54:55.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.552+0000 7fac3ed9d700 1 --2- 172.21.15.162:0/4236522091 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac40142ad0 0x7fac40148b10 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 rx=0x7fac38000f00 tx=0x7fac38007f60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:55.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.552+0000 7fac377fe700 1 -- 172.21.15.162:0/4236522091 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac38006a70 con 0x7fac40142ad0 2020-11-28T21:54:55.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.552+0000 7fac377fe700 1 -- 172.21.15.162:0/4236522091 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac38006bd0 con 0x7fac40142ad0 2020-11-28T21:54:55.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.552+0000 7fac3f59e700 1 -- 172.21.15.162:0/4236522091 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac40148670 con 0x7fac40142ad0 2020-11-28T21:54:55.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.552+0000 7fac377fe700 1 -- 172.21.15.162:0/4236522091 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac38007030 con 0x7fac40142ad0 2020-11-28T21:54:55.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.552+0000 7fac3f59e700 1 -- 172.21.15.162:0/4236522091 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac40149220 con 0x7fac40142ad0 2020-11-28T21:54:55.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.556+0000 7fac3f59e700 1 -- 172.21.15.162:0/4236522091 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac40148800 con 0x7fac40142ad0 2020-11-28T21:54:55.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.556+0000 7fac377fe700 1 -- 172.21.15.162:0/4236522091 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fac38019030 con 0x7fac40142ad0 2020-11-28T21:54:55.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.556+0000 7fac377fe700 1 --2- 172.21.15.162:0/4236522091 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fac240553b0 0x7fac24057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:55.566 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.556+0000 7fac377fe700 1 -- 172.21.15.162:0/4236522091 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fac3806d620 con 0x7fac40142ad0 2020-11-28T21:54:55.566 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.556+0000 7fac3e59c700 1 --2- 172.21.15.162:0/4236522091 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fac240553b0 0x7fac24057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:55.566 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.556+0000 7fac3e59c700 1 --2- 172.21.15.162:0/4236522091 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fac240553b0 0x7fac24057870 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7fac2800b000 tx=0x7fac28006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:55.567 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.560+0000 7fac377fe700 1 -- 172.21.15.162:0/4236522091 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fac38041df0 con 0x7fac40142ad0 2020-11-28T21:54:55.707 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.704+0000 7fac3f59e700 1 -- 172.21.15.162:0/4236522091 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fac40057050 con 0x7fac40142ad0 2020-11-28T21:54:55.708 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.704+0000 7fac377fe700 1 -- 172.21.15.162:0/4236522091 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v119) v1 ==== 110+0+10 (secure 0 0 0) 0x7fac38017020 con 0x7fac40142ad0 2020-11-28T21:54:55.708 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:54:55.710 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.708+0000 7fac3f59e700 1 -- 172.21.15.162:0/4236522091 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fac240553b0 msgr2=0x7fac24057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:55.710 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.708+0000 7fac3f59e700 1 --2- 172.21.15.162:0/4236522091 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fac240553b0 0x7fac24057870 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7fac2800b000 tx=0x7fac28006dc0).stop 2020-11-28T21:54:55.711 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.708+0000 7fac3f59e700 1 -- 172.21.15.162:0/4236522091 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac40142ad0 msgr2=0x7fac40148b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:55.711 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.708+0000 7fac3f59e700 1 --2- 172.21.15.162:0/4236522091 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac40142ad0 0x7fac40148b10 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 rx=0x7fac38000f00 tx=0x7fac38007f60).stop 2020-11-28T21:54:55.711 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.708+0000 7fac3f59e700 1 -- 172.21.15.162:0/4236522091 shutdown_connections 2020-11-28T21:54:55.711 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.708+0000 7fac3f59e700 1 --2- 172.21.15.162:0/4236522091 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fac240553b0 0x7fac24057870 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:55.711 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.708+0000 7fac3f59e700 1 --2- 172.21.15.162:0/4236522091 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fac400b2d00 0x7fac4013f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:55.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.708+0000 7fac3f59e700 1 --2- 172.21.15.162:0/4236522091 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac40142ad0 0x7fac40148b10 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:55.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.708+0000 7fac3f59e700 1 --2- 172.21.15.162:0/4236522091 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fac400b3ab0 0x7fac40144d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:55.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.708+0000 7fac3f59e700 1 -- 172.21.15.162:0/4236522091 >> 172.21.15.162:0/4236522091 conn(0x7fac400604c0 msgr2=0x7fac400b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:55.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.708+0000 7fac3f59e700 1 -- 172.21.15.162:0/4236522091 shutdown_connections 2020-11-28T21:54:55.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.708+0000 7fac3f59e700 1 -- 172.21.15.162:0/4236522091 wait complete. 2020-11-28T21:54:55.721 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:54:55.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.831+0000 7f26d89a8700 1 Processor -- start 2020-11-28T21:54:55.838 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.831+0000 7f26d89a8700 1 -- start start 2020-11-28T21:54:55.839 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.835+0000 7f26d89a8700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f26d40b2290 0x7f26d4060a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:55.839 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.835+0000 7f26d89a8700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26d41471f0 con 0x7f26d40af6a0 2020-11-28T21:54:55.840 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.835+0000 7f26d89a8700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26d4060090 con 0x7f26d40b1eb0 2020-11-28T21:54:55.840 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.835+0000 7f26d89a8700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26d4060330 con 0x7f26d40b2290 2020-11-28T21:54:55.840 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.835+0000 7f26d37fe700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f26d40b1eb0 0x7f26d4144350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51064/0 (socket says 172.21.15.162:51064) 2020-11-28T21:54:55.840 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.835+0000 7f26d37fe700 1 -- 172.21.15.162:0/2228459238 learned_addr learned my addr 172.21.15.162:0/2228459238 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:55.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.835+0000 7f26d3fff700 1 --2- 172.21.15.162:0/2228459238 >> v2:172.21.15.118:3300/0 conn(0x7f26d40b2290 0x7f26d4060a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:55.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.835+0000 7f26d27fc700 1 -- 172.21.15.162:0/2228459238 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 545790019 0 0) 0x7f26d4060090 con 0x7f26d40b1eb0 2020-11-28T21:54:55.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.835+0000 7f26d27fc700 1 -- 172.21.15.162:0/2228459238 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26b4003230 con 0x7f26d40b1eb0 2020-11-28T21:54:55.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.835+0000 7f26d27fc700 1 -- 172.21.15.162:0/2228459238 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 64717160 0 0) 0x7f26d41471f0 con 0x7f26d40af6a0 2020-11-28T21:54:55.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.835+0000 7f26d27fc700 1 -- 172.21.15.162:0/2228459238 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26d4060090 con 0x7f26d40af6a0 2020-11-28T21:54:55.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.835+0000 7f26d3fff700 1 -- 172.21.15.162:0/2228459238 >> v1:172.21.15.162:6789/0 conn(0x7f26d40b1eb0 legacy=0x7f26d4144350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:55.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.835+0000 7f26d3fff700 1 -- 172.21.15.162:0/2228459238 >> v1:172.21.15.118:6789/0 conn(0x7f26d40af6a0 legacy=0x7f26d40afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:55.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.835+0000 7f26d3fff700 1 -- 172.21.15.162:0/2228459238 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d40605d0 con 0x7f26d40b2290 2020-11-28T21:54:55.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.835+0000 7f26d3fff700 1 --2- 172.21.15.162:0/2228459238 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26d40b2290 0x7f26d4060a70 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f26cc002f30 tx=0x7f26cc006dc0).ready entity=mon.0 client_cookie=ca46f0b445b11001 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:55.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.835+0000 7f26d27fc700 1 -- 172.21.15.162:0/2228459238 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26cc009740 con 0x7f26d40b2290 2020-11-28T21:54:55.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.835+0000 7f26d27fc700 1 -- 172.21.15.162:0/2228459238 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26cc0098a0 con 0x7f26d40b2290 2020-11-28T21:54:55.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.835+0000 7f26d27fc700 1 -- 172.21.15.162:0/2228459238 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26cc009c00 con 0x7f26d40b2290 2020-11-28T21:54:55.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.835+0000 7f26d89a8700 1 -- 172.21.15.162:0/2228459238 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26d40b2290 msgr2=0x7f26d4060a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:55.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.835+0000 7f26d89a8700 1 --2- 172.21.15.162:0/2228459238 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26d40b2290 0x7f26d4060a70 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f26cc002f30 tx=0x7f26cc006dc0).stop 2020-11-28T21:54:55.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.835+0000 7f26d89a8700 1 -- 172.21.15.162:0/2228459238 shutdown_connections 2020-11-28T21:54:55.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.835+0000 7f26d89a8700 1 --2- 172.21.15.162:0/2228459238 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26d40b2290 0x7f26d4060a70 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:55.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.835+0000 7f26d89a8700 1 -- 172.21.15.162:0/2228459238 >> 172.21.15.162:0/2228459238 conn(0x7f26d40b57d0 msgr2=0x7f26d40b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:55.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.835+0000 7f26d89a8700 1 -- 172.21.15.162:0/2228459238 shutdown_connections 2020-11-28T21:54:55.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.835+0000 7f26d89a8700 1 -- 172.21.15.162:0/2228459238 wait complete. 2020-11-28T21:54:55.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.839+0000 7f26d89a8700 1 Processor -- start 2020-11-28T21:54:55.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.839+0000 7f26d89a8700 1 -- start start 2020-11-28T21:54:55.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.839+0000 7f26d89a8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26d40af6a0 0x7f26d40ab910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:55.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.839+0000 7f26d89a8700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26d40b1eb0 0x7f26d4159c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:55.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.839+0000 7f26d89a8700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f26d40b2290 0x7f26d415a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:55.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.839+0000 7f26d89a8700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d40aada0 con 0x7f26d40b1eb0 2020-11-28T21:54:55.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.839+0000 7f26d89a8700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d415a660 con 0x7f26d40af6a0 2020-11-28T21:54:55.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.839+0000 7f26d89a8700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26d4146c10 con 0x7f26d40b2290 2020-11-28T21:54:55.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.839+0000 7f26d37fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26d40af6a0 0x7f26d40ab910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:55.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.839+0000 7f26d37fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26d40af6a0 0x7f26d40ab910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44228/0 (socket says 172.21.15.162:44228) 2020-11-28T21:54:55.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.839+0000 7f26d37fe700 1 -- 172.21.15.162:0/559789439 learned_addr learned my addr 172.21.15.162:0/559789439 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:55.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.839+0000 7f26d3fff700 1 --2- 172.21.15.162:0/559789439 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f26d40b2290 0x7f26d415a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:55.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.839+0000 7f26d2ffd700 1 --2- 172.21.15.162:0/559789439 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26d40b1eb0 0x7f26d4159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:55.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.839+0000 7f26d37fe700 1 -- 172.21.15.162:0/559789439 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f26d40b2290 msgr2=0x7f26d415a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:55.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.839+0000 7f26d37fe700 1 --2- 172.21.15.162:0/559789439 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f26d40b2290 0x7f26d415a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:55.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.839+0000 7f26d37fe700 1 -- 172.21.15.162:0/559789439 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26d40b1eb0 msgr2=0x7f26d4159c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:55.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.839+0000 7f26d37fe700 1 --2- 172.21.15.162:0/559789439 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26d40b1eb0 0x7f26d4159c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:55.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.839+0000 7f26d37fe700 1 -- 172.21.15.162:0/559789439 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d4146eb0 con 0x7f26d40af6a0 2020-11-28T21:54:55.852 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.839+0000 7f26d37fe700 1 --2- 172.21.15.162:0/559789439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26d40af6a0 0x7f26d40ab910 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 rx=0x7f26c4001410 tx=0x7f26c4007e50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:55.852 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.839+0000 7f26d0ff9700 1 -- 172.21.15.162:0/559789439 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26c400ce70 con 0x7f26d40af6a0 2020-11-28T21:54:55.852 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.839+0000 7f26d0ff9700 1 -- 172.21.15.162:0/559789439 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26c4005b70 con 0x7f26d40af6a0 2020-11-28T21:54:55.852 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.839+0000 7f26d89a8700 1 -- 172.21.15.162:0/559789439 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26d4147150 con 0x7f26d40af6a0 2020-11-28T21:54:55.852 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.839+0000 7f26d0ff9700 1 -- 172.21.15.162:0/559789439 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26c4007110 con 0x7f26d40af6a0 2020-11-28T21:54:55.853 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.839+0000 7f26d89a8700 1 -- 172.21.15.162:0/559789439 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26d4067170 con 0x7f26d40af6a0 2020-11-28T21:54:55.853 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.843+0000 7f26d89a8700 1 -- 172.21.15.162:0/559789439 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26d4057050 con 0x7f26d40af6a0 2020-11-28T21:54:55.854 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.843+0000 7f26d0ff9700 1 -- 172.21.15.162:0/559789439 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f26c4010030 con 0x7f26d40af6a0 2020-11-28T21:54:55.854 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.843+0000 7f26d0ff9700 1 --2- 172.21.15.162:0/559789439 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f26b8055330 0x7f26b80577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:55.854 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.843+0000 7f26d0ff9700 1 -- 172.21.15.162:0/559789439 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f26c406df30 con 0x7f26d40af6a0 2020-11-28T21:54:55.854 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.843+0000 7f26d2ffd700 1 --2- 172.21.15.162:0/559789439 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f26b8055330 0x7f26b80577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:55.855 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.843+0000 7f26d2ffd700 1 --2- 172.21.15.162:0/559789439 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f26b8055330 0x7f26b80577f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7f26bc00d000 tx=0x7f26bc0079f0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:55.855 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.851+0000 7f26d0ff9700 1 -- 172.21.15.162:0/559789439 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f26c4044190 con 0x7f26d40af6a0 2020-11-28T21:54:55.998 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.991+0000 7f26d89a8700 1 -- 172.21.15.162:0/559789439 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f26d40a89f0 con 0x7f26d40af6a0 2020-11-28T21:54:55.998 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.991+0000 7f26d0ff9700 1 -- 172.21.15.162:0/559789439 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v119) v1 ==== 92+0+10 (secure 0 0 0) 0x7f26c40477b0 con 0x7f26d40af6a0 2020-11-28T21:54:55.998 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:54:56.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.995+0000 7f26d89a8700 1 -- 172.21.15.162:0/559789439 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f26b8055330 msgr2=0x7f26b80577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:56.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.995+0000 7f26d89a8700 1 --2- 172.21.15.162:0/559789439 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f26b8055330 0x7f26b80577f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7f26bc00d000 tx=0x7f26bc0079f0).stop 2020-11-28T21:54:56.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.995+0000 7f26d89a8700 1 -- 172.21.15.162:0/559789439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26d40af6a0 msgr2=0x7f26d40ab910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:56.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.995+0000 7f26d89a8700 1 --2- 172.21.15.162:0/559789439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26d40af6a0 0x7f26d40ab910 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 rx=0x7f26c4001410 tx=0x7f26c4007e50).stop 2020-11-28T21:54:56.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.995+0000 7f26d89a8700 1 -- 172.21.15.162:0/559789439 shutdown_connections 2020-11-28T21:54:56.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.995+0000 7f26d89a8700 1 --2- 172.21.15.162:0/559789439 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f26b8055330 0x7f26b80577f0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:56.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.995+0000 7f26d89a8700 1 --2- 172.21.15.162:0/559789439 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f26d40b2290 0x7f26d415a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:56.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.995+0000 7f26d89a8700 1 --2- 172.21.15.162:0/559789439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26d40af6a0 0x7f26d40ab910 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:56.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.995+0000 7f26d89a8700 1 --2- 172.21.15.162:0/559789439 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26d40b1eb0 0x7f26d4159c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:56.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.995+0000 7f26d89a8700 1 -- 172.21.15.162:0/559789439 >> 172.21.15.162:0/559789439 conn(0x7f26d40b57d0 msgr2=0x7f26d40b01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:56.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.995+0000 7f26d89a8700 1 -- 172.21.15.162:0/559789439 shutdown_connections 2020-11-28T21:54:56.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:55.995+0000 7f26d89a8700 1 -- 172.21.15.162:0/559789439 wait complete. 2020-11-28T21:54:56.021 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-28T21:54:56.139 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.135+0000 7ffa813a8700 1 Processor -- start 2020-11-28T21:54:56.139 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.135+0000 7ffa813a8700 1 -- start start 2020-11-28T21:54:56.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.135+0000 7ffa813a8700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ffa7c0b21f0 0x7ffa7c13fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:56.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.135+0000 7ffa813a8700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ffa7c060b60 0x7ffa7c14d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:56.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.135+0000 7ffa813a8700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa7c0aff20 con 0x7ffa7c1402a0 2020-11-28T21:54:56.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.135+0000 7ffa813a8700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa7c0af240 con 0x7ffa7c060b60 2020-11-28T21:54:56.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.135+0000 7ffa813a8700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa7c140750 con 0x7ffa7c0b21f0 2020-11-28T21:54:56.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.135+0000 7ffa7b7fe700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7ffa7c1402a0 0x7ffa7c060450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51078/0 (socket says 172.21.15.162:51078) 2020-11-28T21:54:56.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.135+0000 7ffa7b7fe700 1 -- 172.21.15.162:0/2191784204 learned_addr learned my addr 172.21.15.162:0/2191784204 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:56.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.135+0000 7ffa7bfff700 1 --2- 172.21.15.162:0/2191784204 >> v2:172.21.15.162:3300/0 conn(0x7ffa7c0b21f0 0x7ffa7c13fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:56.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.135+0000 7ffa80ba7700 1 --2- 172.21.15.162:0/2191784204 >> v2:172.21.15.118:3300/0 conn(0x7ffa7c060b60 0x7ffa7c14d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:56.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.135+0000 7ffa7affd700 1 -- 172.21.15.162:0/2191784204 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 267741459 0 0) 0x7ffa7c0aff20 con 0x7ffa7c1402a0 2020-11-28T21:54:56.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.135+0000 7ffa7affd700 1 -- 172.21.15.162:0/2191784204 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa60003230 con 0x7ffa7c1402a0 2020-11-28T21:54:56.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.135+0000 7ffa7bfff700 1 -- 172.21.15.162:0/2191784204 >> v2:172.21.15.118:3300/0 conn(0x7ffa7c060b60 msgr2=0x7ffa7c14d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:56.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.135+0000 7ffa7bfff700 1 --2- 172.21.15.162:0/2191784204 >> v2:172.21.15.118:3300/0 conn(0x7ffa7c060b60 0x7ffa7c14d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:56.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.135+0000 7ffa7bfff700 1 -- 172.21.15.162:0/2191784204 >> v1:172.21.15.162:6789/0 conn(0x7ffa7c1402a0 legacy=0x7ffa7c060450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:56.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.135+0000 7ffa7bfff700 1 -- 172.21.15.162:0/2191784204 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa7c0b25d0 con 0x7ffa7c0b21f0 2020-11-28T21:54:56.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.135+0000 7ffa7bfff700 1 --2- 172.21.15.162:0/2191784204 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa7c0b21f0 0x7ffa7c13fdb0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 rx=0x7ffa6c0032b0 tx=0x7ffa6c0091a0).ready entity=mon.1 client_cookie=67b769d6562ecc7d server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:56.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.135+0000 7ffa7affd700 1 -- 172.21.15.162:0/2191784204 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa6c00b9a0 con 0x7ffa7c0b21f0 2020-11-28T21:54:56.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.135+0000 7ffa7affd700 1 -- 172.21.15.162:0/2191784204 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa6c00bb00 con 0x7ffa7c0b21f0 2020-11-28T21:54:56.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.135+0000 7ffa7affd700 1 -- 172.21.15.162:0/2191784204 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa6c00bc70 con 0x7ffa7c0b21f0 2020-11-28T21:54:56.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.135+0000 7ffa813a8700 1 -- 172.21.15.162:0/2191784204 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa7c0b21f0 msgr2=0x7ffa7c13fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:56.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.135+0000 7ffa813a8700 1 --2- 172.21.15.162:0/2191784204 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa7c0b21f0 0x7ffa7c13fdb0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 rx=0x7ffa6c0032b0 tx=0x7ffa6c0091a0).stop 2020-11-28T21:54:56.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.139+0000 7ffa813a8700 1 -- 172.21.15.162:0/2191784204 shutdown_connections 2020-11-28T21:54:56.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.139+0000 7ffa813a8700 1 --2- 172.21.15.162:0/2191784204 >> v2:172.21.15.118:3300/0 conn(0x7ffa7c060b60 0x7ffa7c14d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:56.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.139+0000 7ffa813a8700 1 --2- 172.21.15.162:0/2191784204 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa7c0b21f0 0x7ffa7c13fdb0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:56.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.139+0000 7ffa813a8700 1 -- 172.21.15.162:0/2191784204 >> 172.21.15.162:0/2191784204 conn(0x7ffa7c0b57d0 msgr2=0x7ffa7c0b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:56.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.139+0000 7ffa813a8700 1 -- 172.21.15.162:0/2191784204 shutdown_connections 2020-11-28T21:54:56.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.139+0000 7ffa813a8700 1 -- 172.21.15.162:0/2191784204 wait complete. 2020-11-28T21:54:56.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.139+0000 7ffa813a8700 1 Processor -- start 2020-11-28T21:54:56.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.139+0000 7ffa813a8700 1 -- start start 2020-11-28T21:54:56.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.139+0000 7ffa813a8700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffa7c060b60 0x7ffa7c0ab650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:56.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.139+0000 7ffa813a8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa7c0b21f0 0x7ffa7c1408c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:56.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.139+0000 7ffa813a8700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffa7c1402a0 0x7ffa7c0aa130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:56.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.139+0000 7ffa813a8700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7c1528f0 con 0x7ffa7c060b60 2020-11-28T21:54:56.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.139+0000 7ffa813a8700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7c0aac70 con 0x7ffa7c0b21f0 2020-11-28T21:54:56.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.139+0000 7ffa813a8700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7c0aaf10 con 0x7ffa7c1402a0 2020-11-28T21:54:56.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.139+0000 7ffa7b7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa7c0b21f0 0x7ffa7c1408c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:56.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.139+0000 7ffa7b7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa7c0b21f0 0x7ffa7c1408c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44240/0 (socket says 172.21.15.162:44240) 2020-11-28T21:54:56.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.139+0000 7ffa7b7fe700 1 -- 172.21.15.162:0/240029633 learned_addr learned my addr 172.21.15.162:0/240029633 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:56.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.139+0000 7ffa80ba7700 1 --2- 172.21.15.162:0/240029633 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffa7c1402a0 0x7ffa7c0aa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:56.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.139+0000 7ffa7bfff700 1 --2- 172.21.15.162:0/240029633 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffa7c060b60 0x7ffa7c0ab650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:56.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.143+0000 7ffa7b7fe700 1 -- 172.21.15.162:0/240029633 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffa7c1402a0 msgr2=0x7ffa7c0aa130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:56.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.143+0000 7ffa7b7fe700 1 --2- 172.21.15.162:0/240029633 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffa7c1402a0 0x7ffa7c0aa130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:56.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.143+0000 7ffa7b7fe700 1 -- 172.21.15.162:0/240029633 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffa7c060b60 msgr2=0x7ffa7c0ab650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:56.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.143+0000 7ffa7b7fe700 1 --2- 172.21.15.162:0/240029633 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffa7c060b60 0x7ffa7c0ab650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:56.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.143+0000 7ffa7b7fe700 1 -- 172.21.15.162:0/240029633 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa7c0ab1b0 con 0x7ffa7c0b21f0 2020-11-28T21:54:56.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.143+0000 7ffa7b7fe700 1 --2- 172.21.15.162:0/240029633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa7c0b21f0 0x7ffa7c1408c0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 rx=0x7ffa7c0a9d40 tx=0x7ffa64007a70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:56.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.143+0000 7ffa797fa700 1 -- 172.21.15.162:0/240029633 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa640072e0 con 0x7ffa7c0b21f0 2020-11-28T21:54:56.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.143+0000 7ffa797fa700 1 -- 172.21.15.162:0/240029633 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa64007440 con 0x7ffa7c0b21f0 2020-11-28T21:54:56.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.143+0000 7ffa813a8700 1 -- 172.21.15.162:0/240029633 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa7c142790 con 0x7ffa7c0b21f0 2020-11-28T21:54:56.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.143+0000 7ffa797fa700 1 -- 172.21.15.162:0/240029633 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa640057d0 con 0x7ffa7c0b21f0 2020-11-28T21:54:56.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.143+0000 7ffa813a8700 1 -- 172.21.15.162:0/240029633 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa7c1518f0 con 0x7ffa7c0b21f0 2020-11-28T21:54:56.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.143+0000 7ffa813a8700 1 -- 172.21.15.162:0/240029633 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa7c0ab340 con 0x7ffa7c0b21f0 2020-11-28T21:54:56.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.143+0000 7ffa797fa700 1 -- 172.21.15.162:0/240029633 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ffa64010070 con 0x7ffa7c0b21f0 2020-11-28T21:54:56.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.143+0000 7ffa797fa700 1 --2- 172.21.15.162:0/240029633 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ffa54055440 0x7ffa54057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:56.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.143+0000 7ffa797fa700 1 -- 172.21.15.162:0/240029633 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ffa6406d520 con 0x7ffa7c0b21f0 2020-11-28T21:54:56.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.143+0000 7ffa7bfff700 1 --2- 172.21.15.162:0/240029633 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ffa54055440 0x7ffa54057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:56.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.143+0000 7ffa7bfff700 1 --2- 172.21.15.162:0/240029633 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ffa54055440 0x7ffa54057900 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7ffa6c01b000 tx=0x7ffa6c016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:56.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.151+0000 7ffa797fa700 1 -- 172.21.15.162:0/240029633 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ffa64043780 con 0x7ffa7c0b21f0 2020-11-28T21:54:56.294 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.291+0000 7ffa813a8700 1 -- 172.21.15.162:0/240029633 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffa7c057050 con 0x7ffa7c0b21f0 2020-11-28T21:54:56.294 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.291+0000 7ffa797fa700 1 -- 172.21.15.162:0/240029633 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v119) v1 ==== 104+0+10 (secure 0 0 0) 0x7ffa64018020 con 0x7ffa7c0b21f0 2020-11-28T21:54:56.295 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:54:56.295 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.291+0000 7ffa813a8700 1 -- 172.21.15.162:0/240029633 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ffa54055440 msgr2=0x7ffa54057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:56.296 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.291+0000 7ffa813a8700 1 --2- 172.21.15.162:0/240029633 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ffa54055440 0x7ffa54057900 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7ffa6c01b000 tx=0x7ffa6c016000).stop 2020-11-28T21:54:56.296 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.291+0000 7ffa813a8700 1 -- 172.21.15.162:0/240029633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa7c0b21f0 msgr2=0x7ffa7c1408c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:56.296 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.291+0000 7ffa813a8700 1 --2- 172.21.15.162:0/240029633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa7c0b21f0 0x7ffa7c1408c0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 rx=0x7ffa7c0a9d40 tx=0x7ffa64007a70).stop 2020-11-28T21:54:56.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.291+0000 7ffa813a8700 1 -- 172.21.15.162:0/240029633 shutdown_connections 2020-11-28T21:54:56.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.291+0000 7ffa813a8700 1 --2- 172.21.15.162:0/240029633 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ffa54055440 0x7ffa54057900 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:56.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.291+0000 7ffa813a8700 1 --2- 172.21.15.162:0/240029633 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffa7c1402a0 0x7ffa7c0aa130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:56.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.291+0000 7ffa813a8700 1 --2- 172.21.15.162:0/240029633 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffa7c060b60 0x7ffa7c0ab650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:56.298 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.291+0000 7ffa813a8700 1 --2- 172.21.15.162:0/240029633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa7c0b21f0 0x7ffa7c1408c0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:56.298 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.291+0000 7ffa813a8700 1 -- 172.21.15.162:0/240029633 >> 172.21.15.162:0/240029633 conn(0x7ffa7c0b57d0 msgr2=0x7ffa7c13ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:56.298 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.291+0000 7ffa813a8700 1 -- 172.21.15.162:0/240029633 shutdown_connections 2020-11-28T21:54:56.298 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.291+0000 7ffa813a8700 1 -- 172.21.15.162:0/240029633 wait complete. 2020-11-28T21:54:56.312 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-28T21:54:56.428 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.423+0000 7faae6009700 1 Processor -- start 2020-11-28T21:54:56.428 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.423+0000 7faae6009700 1 -- start start 2020-11-28T21:54:56.430 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.423+0000 7faae6009700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7faae00b3c90 0x7faae0142a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:56.431 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.423+0000 7faae6009700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7faae00b18c0 0x7faae0142f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:56.431 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.427+0000 7faae6009700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faae00aac00 con 0x7faae005f380 2020-11-28T21:54:56.431 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.427+0000 7faae6009700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faae013cf80 con 0x7faae00b18c0 2020-11-28T21:54:56.431 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.427+0000 7faae6009700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faae00b4070 con 0x7faae00b3c90 2020-11-28T21:54:56.431 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.427+0000 7faae5007700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7faae00b3c90 0x7faae0142a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:56.432 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.427+0000 7faae5007700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7faae00b3c90 0x7faae0142a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44248/0 (socket says 172.21.15.162:44248) 2020-11-28T21:54:56.432 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.427+0000 7faae5007700 1 -- 172.21.15.162:0/1636544791 learned_addr learned my addr 172.21.15.162:0/1636544791 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:56.432 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.427+0000 7faae4806700 1 --2- 172.21.15.162:0/1636544791 >> v2:172.21.15.118:3300/0 conn(0x7faae00b18c0 0x7faae0142f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:56.432 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.427+0000 7faae5007700 1 -- 172.21.15.162:0/1636544791 >> v2:172.21.15.118:3300/0 conn(0x7faae00b18c0 msgr2=0x7faae0142f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:56.433 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.427+0000 7faae5007700 1 --2- 172.21.15.162:0/1636544791 >> v2:172.21.15.118:3300/0 conn(0x7faae00b18c0 0x7faae0142f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:56.433 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.427+0000 7faae5007700 1 -- 172.21.15.162:0/1636544791 >> v1:172.21.15.118:6789/0 conn(0x7faae005f380 legacy=0x7faae0146050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:56.433 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.427+0000 7faae5007700 1 -- 172.21.15.162:0/1636544791 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae0060a10 con 0x7faae00b3c90 2020-11-28T21:54:56.433 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.427+0000 7faae5007700 1 --2- 172.21.15.162:0/1636544791 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faae00b3c90 0x7faae0142a80 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 rx=0x7faae0067dc0 tx=0x7faadc006e80).ready entity=mon.1 client_cookie=867e464bfe7b8b78 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:56.433 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.427+0000 7faacffff700 1 -- 172.21.15.162:0/1636544791 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faadc0097b0 con 0x7faae00b3c90 2020-11-28T21:54:56.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.427+0000 7faacffff700 1 -- 172.21.15.162:0/1636544791 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faadc009910 con 0x7faae00b3c90 2020-11-28T21:54:56.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.427+0000 7faacffff700 1 -- 172.21.15.162:0/1636544791 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faadc009a80 con 0x7faae00b3c90 2020-11-28T21:54:56.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.427+0000 7faae6009700 1 -- 172.21.15.162:0/1636544791 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faae00b3c90 msgr2=0x7faae0142a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:56.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.427+0000 7faae6009700 1 --2- 172.21.15.162:0/1636544791 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faae00b3c90 0x7faae0142a80 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 rx=0x7faae0067dc0 tx=0x7faadc006e80).stop 2020-11-28T21:54:56.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.427+0000 7faae6009700 1 -- 172.21.15.162:0/1636544791 shutdown_connections 2020-11-28T21:54:56.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.427+0000 7faae6009700 1 --2- 172.21.15.162:0/1636544791 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faae00b3c90 0x7faae0142a80 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:56.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.427+0000 7faae6009700 1 --2- 172.21.15.162:0/1636544791 >> v2:172.21.15.118:3300/0 conn(0x7faae00b18c0 0x7faae0142f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:56.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.427+0000 7faae6009700 1 -- 172.21.15.162:0/1636544791 >> 172.21.15.162:0/1636544791 conn(0x7faae0062630 msgr2=0x7faae0062a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:56.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.427+0000 7faae6009700 1 -- 172.21.15.162:0/1636544791 shutdown_connections 2020-11-28T21:54:56.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.427+0000 7faae6009700 1 -- 172.21.15.162:0/1636544791 wait complete. 2020-11-28T21:54:56.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.427+0000 7faae6009700 1 Processor -- start 2020-11-28T21:54:56.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.427+0000 7faae6009700 1 -- start start 2020-11-28T21:54:56.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.431+0000 7faae6009700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faae005f380 0x7faae0065b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:56.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.431+0000 7faae6009700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faae00b18c0 0x7faae0145ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:56.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.431+0000 7faae6009700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faae00b3c90 0x7faae01461d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:56.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.431+0000 7faae6009700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae013eed0 con 0x7faae00b18c0 2020-11-28T21:54:56.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.431+0000 7faae6009700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae013f170 con 0x7faae00b3c90 2020-11-28T21:54:56.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.431+0000 7faae6009700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faae013e1b0 con 0x7faae005f380 2020-11-28T21:54:56.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.431+0000 7faae5808700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faae00b3c90 0x7faae01461d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:56.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.431+0000 7faae5808700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faae00b3c90 0x7faae01461d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44258/0 (socket says 172.21.15.162:44258) 2020-11-28T21:54:56.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.431+0000 7faae5808700 1 -- 172.21.15.162:0/2600253297 learned_addr learned my addr 172.21.15.162:0/2600253297 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:56.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.431+0000 7faae5007700 1 --2- 172.21.15.162:0/2600253297 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faae005f380 0x7faae0065b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:56.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.431+0000 7faae4806700 1 --2- 172.21.15.162:0/2600253297 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faae00b18c0 0x7faae0145ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:56.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.431+0000 7faae5808700 1 -- 172.21.15.162:0/2600253297 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faae005f380 msgr2=0x7faae0065b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:56.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.431+0000 7faae5808700 1 --2- 172.21.15.162:0/2600253297 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faae005f380 0x7faae0065b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:56.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.431+0000 7faae5808700 1 -- 172.21.15.162:0/2600253297 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faae00b18c0 msgr2=0x7faae0145ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:56.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.431+0000 7faae5808700 1 --2- 172.21.15.162:0/2600253297 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faae00b18c0 0x7faae0145ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:56.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.431+0000 7faae5808700 1 -- 172.21.15.162:0/2600253297 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae013e4c0 con 0x7faae00b3c90 2020-11-28T21:54:56.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.431+0000 7faae5808700 1 --2- 172.21.15.162:0/2600253297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faae00b3c90 0x7faae01461d0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 rx=0x7faad80022e0 tx=0x7faad800cb40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:56.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.431+0000 7faace7fc700 1 -- 172.21.15.162:0/2600253297 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faad8010510 con 0x7faae00b3c90 2020-11-28T21:54:56.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.431+0000 7faace7fc700 1 -- 172.21.15.162:0/2600253297 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faad8010670 con 0x7faae00b3c90 2020-11-28T21:54:56.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.431+0000 7faae6009700 1 -- 172.21.15.162:0/2600253297 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faae01466c0 con 0x7faae00b3c90 2020-11-28T21:54:56.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.431+0000 7faace7fc700 1 -- 172.21.15.162:0/2600253297 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faad80107e0 con 0x7faae00b3c90 2020-11-28T21:54:56.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.431+0000 7faae6009700 1 -- 172.21.15.162:0/2600253297 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faae013fe70 con 0x7faae00b3c90 2020-11-28T21:54:56.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.431+0000 7faae6009700 1 -- 172.21.15.162:0/2600253297 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faae013e710 con 0x7faae00b3c90 2020-11-28T21:54:56.444 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.435+0000 7faace7fc700 1 -- 172.21.15.162:0/2600253297 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7faad8010070 con 0x7faae00b3c90 2020-11-28T21:54:56.445 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.435+0000 7faace7fc700 1 --2- 172.21.15.162:0/2600253297 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7faac0055440 0x7faac0057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:56.445 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.435+0000 7faace7fc700 1 -- 172.21.15.162:0/2600253297 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6073+0+0 (secure 0 0 0) 0x7faad806d7b0 con 0x7faae00b3c90 2020-11-28T21:54:56.445 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.435+0000 7faae5007700 1 --2- 172.21.15.162:0/2600253297 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7faac0055440 0x7faac0057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:56.445 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.435+0000 7faae5007700 1 --2- 172.21.15.162:0/2600253297 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7faac0055440 0x7faac0057900 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7faadc01c000 tx=0x7faadc016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:56.445 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.439+0000 7faace7fc700 1 -- 172.21.15.162:0/2600253297 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7faad8043ab0 con 0x7faae00b3c90 2020-11-28T21:54:56.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.575+0000 7faae6009700 1 -- 172.21.15.162:0/2600253297 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faae0057050 con 0x7faae00b3c90 2020-11-28T21:54:56.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.575+0000 7faace7fc700 1 -- 172.21.15.162:0/2600253297 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v119) v1 ==== 100+0+10 (secure 0 0 0) 0x7faad80470d0 con 0x7faae00b3c90 2020-11-28T21:54:56.581 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:54:56.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.579+0000 7faae6009700 1 -- 172.21.15.162:0/2600253297 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7faac0055440 msgr2=0x7faac0057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:56.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.579+0000 7faae6009700 1 --2- 172.21.15.162:0/2600253297 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7faac0055440 0x7faac0057900 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7faadc01c000 tx=0x7faadc016000).stop 2020-11-28T21:54:56.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.579+0000 7faae6009700 1 -- 172.21.15.162:0/2600253297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faae00b3c90 msgr2=0x7faae01461d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:56.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.579+0000 7faae6009700 1 --2- 172.21.15.162:0/2600253297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faae00b3c90 0x7faae01461d0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 rx=0x7faad80022e0 tx=0x7faad800cb40).stop 2020-11-28T21:54:56.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.579+0000 7faae6009700 1 -- 172.21.15.162:0/2600253297 shutdown_connections 2020-11-28T21:54:56.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.579+0000 7faae6009700 1 --2- 172.21.15.162:0/2600253297 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7faac0055440 0x7faac0057900 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:56.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.579+0000 7faae6009700 1 --2- 172.21.15.162:0/2600253297 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faae005f380 0x7faae0065b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:56.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.579+0000 7faae6009700 1 --2- 172.21.15.162:0/2600253297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faae00b3c90 0x7faae01461d0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:56.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.579+0000 7faae6009700 1 --2- 172.21.15.162:0/2600253297 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faae00b18c0 0x7faae0145ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:56.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.579+0000 7faae6009700 1 -- 172.21.15.162:0/2600253297 >> 172.21.15.162:0/2600253297 conn(0x7faae0062630 msgr2=0x7faae00b2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:56.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.579+0000 7faae6009700 1 -- 172.21.15.162:0/2600253297 shutdown_connections 2020-11-28T21:54:56.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.579+0000 7faae6009700 1 -- 172.21.15.162:0/2600253297 wait complete. 2020-11-28T21:54:56.595 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-28T21:54:56.595 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:54:56.704 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.699+0000 7fb4df59e700 1 Processor -- start 2020-11-28T21:54:56.705 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.699+0000 7fb4df59e700 1 -- start start 2020-11-28T21:54:56.706 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.703+0000 7fb4df59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb4e00603d0 0x7fb4e00b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:56.706 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.703+0000 7fb4df59e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb4e0142a00 0x7fb4e009cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:56.706 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.703+0000 7fb4df59e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fb4e00b4c70 0x7fb4e009d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:56.707 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.703+0000 7fb4df59e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4e0060f60 con 0x7fb4e00b4c70 2020-11-28T21:54:56.707 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.703+0000 7fb4df59e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4e00b4110 con 0x7fb4e0142a00 2020-11-28T21:54:56.707 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.703+0000 7fb4df59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e00b43f0 con 0x7fb4e00603d0 2020-11-28T21:54:56.707 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.703+0000 7fb4ddd9b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb4e0142a00 0x7fb4e009cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:56.707 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.703+0000 7fb4ddd9b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb4e0142a00 0x7fb4e009cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44264/0 (socket says 172.21.15.162:44264) 2020-11-28T21:54:56.708 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.703+0000 7fb4ddd9b700 1 -- 172.21.15.162:0/2687498899 learned_addr learned my addr 172.21.15.162:0/2687498899 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:56.708 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.703+0000 7fb4ded9d700 1 --2- 172.21.15.162:0/2687498899 >> v2:172.21.15.118:3300/0 conn(0x7fb4e00b4c70 0x7fb4e009d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:56.708 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.703+0000 7fb4de59c700 1 --2- 172.21.15.162:0/2687498899 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb4e00603d0 0x7fb4e00b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:56.709 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.703+0000 7fb4ddd9b700 1 -- 172.21.15.162:0/2687498899 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb4e00603d0 msgr2=0x7fb4e00b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:56.709 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.703+0000 7fb4ddd9b700 1 --2- 172.21.15.162:0/2687498899 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb4e00603d0 0x7fb4e00b3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:56.710 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.703+0000 7fb4ddd9b700 1 -- 172.21.15.162:0/2687498899 >> v2:172.21.15.118:3300/0 conn(0x7fb4e00b4c70 msgr2=0x7fb4e009d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:56.710 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.703+0000 7fb4ddd9b700 1 --2- 172.21.15.162:0/2687498899 >> v2:172.21.15.118:3300/0 conn(0x7fb4e00b4c70 0x7fb4e009d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:56.710 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.703+0000 7fb4ddd9b700 1 -- 172.21.15.162:0/2687498899 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e00b4690 con 0x7fb4e0142a00 2020-11-28T21:54:56.710 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.703+0000 7fb4ddd9b700 1 --2- 172.21.15.162:0/2687498899 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4e0142a00 0x7fb4e009cfc0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 rx=0x7fb4d0002f30 tx=0x7fb4d0006e80).ready entity=mon.1 client_cookie=9bb70448aa1b035f server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:56.711 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.703+0000 7fb4dd59a700 1 -- 172.21.15.162:0/2687498899 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb4d0009640 con 0x7fb4e0142a00 2020-11-28T21:54:56.711 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.703+0000 7fb4dd59a700 1 -- 172.21.15.162:0/2687498899 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4d00097a0 con 0x7fb4e0142a00 2020-11-28T21:54:56.711 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.703+0000 7fb4dd59a700 1 -- 172.21.15.162:0/2687498899 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb4d0009b00 con 0x7fb4e0142a00 2020-11-28T21:54:56.711 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.703+0000 7fb4df59e700 1 -- 172.21.15.162:0/2687498899 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4e0142a00 msgr2=0x7fb4e009cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:56.711 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.703+0000 7fb4df59e700 1 --2- 172.21.15.162:0/2687498899 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4e0142a00 0x7fb4e009cfc0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 rx=0x7fb4d0002f30 tx=0x7fb4d0006e80).stop 2020-11-28T21:54:56.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.703+0000 7fb4df59e700 1 -- 172.21.15.162:0/2687498899 shutdown_connections 2020-11-28T21:54:56.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.703+0000 7fb4df59e700 1 --2- 172.21.15.162:0/2687498899 >> v2:172.21.15.118:3300/0 conn(0x7fb4e00b4c70 0x7fb4e009d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:56.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.703+0000 7fb4df59e700 1 --2- 172.21.15.162:0/2687498899 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb4e00603d0 0x7fb4e00b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:56.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.703+0000 7fb4df59e700 1 --2- 172.21.15.162:0/2687498899 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4e0142a00 0x7fb4e009cfc0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:56.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.703+0000 7fb4df59e700 1 -- 172.21.15.162:0/2687498899 >> 172.21.15.162:0/2687498899 conn(0x7fb4e0066a40 msgr2=0x7fb4e0066e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:56.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.703+0000 7fb4df59e700 1 -- 172.21.15.162:0/2687498899 shutdown_connections 2020-11-28T21:54:56.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.703+0000 7fb4df59e700 1 -- 172.21.15.162:0/2687498899 wait complete. 2020-11-28T21:54:56.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.707+0000 7fb4df59e700 1 Processor -- start 2020-11-28T21:54:56.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.707+0000 7fb4df59e700 1 -- start start 2020-11-28T21:54:56.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.707+0000 7fb4df59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb4e00603d0 0x7fb4e0144890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:56.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.707+0000 7fb4df59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4e00b4c70 0x7fb4e0151530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:56.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.707+0000 7fb4df59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb4e0144d80 0x7fb4e0151a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:56.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.707+0000 7fb4df59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e0146d90 con 0x7fb4e00603d0 2020-11-28T21:54:56.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.707+0000 7fb4df59e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e00ad130 con 0x7fb4e00b4c70 2020-11-28T21:54:56.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.707+0000 7fb4df59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e00aca60 con 0x7fb4e0144d80 2020-11-28T21:54:56.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.707+0000 7fb4ddd9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4e00b4c70 0x7fb4e0151530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:56.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.707+0000 7fb4ddd9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4e00b4c70 0x7fb4e0151530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44270/0 (socket says 172.21.15.162:44270) 2020-11-28T21:54:56.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.707+0000 7fb4ddd9b700 1 -- 172.21.15.162:0/2075503041 learned_addr learned my addr 172.21.15.162:0/2075503041 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:56.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.707+0000 7fb4de59c700 1 --2- 172.21.15.162:0/2075503041 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb4e00603d0 0x7fb4e0144890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:56.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.707+0000 7fb4ded9d700 1 --2- 172.21.15.162:0/2075503041 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb4e0144d80 0x7fb4e0151a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:56.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.707+0000 7fb4ddd9b700 1 -- 172.21.15.162:0/2075503041 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb4e0144d80 msgr2=0x7fb4e0151a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:56.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.707+0000 7fb4ddd9b700 1 --2- 172.21.15.162:0/2075503041 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb4e0144d80 0x7fb4e0151a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:56.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.707+0000 7fb4ddd9b700 1 -- 172.21.15.162:0/2075503041 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb4e00603d0 msgr2=0x7fb4e0144890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:56.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.707+0000 7fb4ddd9b700 1 --2- 172.21.15.162:0/2075503041 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb4e00603d0 0x7fb4e0144890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:56.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.707+0000 7fb4ddd9b700 1 -- 172.21.15.162:0/2075503041 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e00ad360 con 0x7fb4e00b4c70 2020-11-28T21:54:56.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.707+0000 7fb4ddd9b700 1 --2- 172.21.15.162:0/2075503041 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4e00b4c70 0x7fb4e0151530 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 rx=0x7fb4e00b3120 tx=0x7fb4d0015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:56.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.707+0000 7fb4cf7fe700 1 -- 172.21.15.162:0/2075503041 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb4d0011bb0 con 0x7fb4e00b4c70 2020-11-28T21:54:56.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.707+0000 7fb4cf7fe700 1 -- 172.21.15.162:0/2075503041 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4d0011d10 con 0x7fb4e00b4c70 2020-11-28T21:54:56.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.707+0000 7fb4df59e700 1 -- 172.21.15.162:0/2075503041 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4e00ad600 con 0x7fb4e00b4c70 2020-11-28T21:54:56.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.707+0000 7fb4cf7fe700 1 -- 172.21.15.162:0/2075503041 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb4d001d640 con 0x7fb4e00b4c70 2020-11-28T21:54:56.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.707+0000 7fb4df59e700 1 -- 172.21.15.162:0/2075503041 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4e0143d80 con 0x7fb4e00b4c70 2020-11-28T21:54:56.720 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.711+0000 7fb4df59e700 1 -- 172.21.15.162:0/2075503041 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4e00ad790 con 0x7fb4e00b4c70 2020-11-28T21:54:56.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.711+0000 7fb4cf7fe700 1 -- 172.21.15.162:0/2075503041 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb4d000f070 con 0x7fb4e00b4c70 2020-11-28T21:54:56.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.711+0000 7fb4cf7fe700 1 --2- 172.21.15.162:0/2075503041 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb4bc055330 0x7fb4bc0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:56.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.711+0000 7fb4cf7fe700 1 -- 172.21.15.162:0/2075503041 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb4d006f940 con 0x7fb4e00b4c70 2020-11-28T21:54:56.721 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.711+0000 7fb4de59c700 1 --2- 172.21.15.162:0/2075503041 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb4bc055330 0x7fb4bc0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:56.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.711+0000 7fb4de59c700 1 --2- 172.21.15.162:0/2075503041 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb4bc055330 0x7fb4bc0577f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7fb4d400b000 tx=0x7fb4d4006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:56.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.715+0000 7fb4cf7fe700 1 -- 172.21.15.162:0/2075503041 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb4d0045be0 con 0x7fb4e00b4c70 2020-11-28T21:54:56.906 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.903+0000 7fb4df59e700 1 -- 172.21.15.162:0/2075503041 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4e0057050 con 0x7fb4e00b4c70 2020-11-28T21:54:56.907 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.903+0000 7fb4cf7fe700 1 -- 172.21.15.162:0/2075503041 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+2504 (secure 0 0 0) 0x7fb4d0049200 con 0x7fb4e00b4c70 2020-11-28T21:54:56.907 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:54:56.907 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":57,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7257,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/1527184432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1527184432},{"type":"v1","addr":"172.21.15.118:6833","nonce":1527184432}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":54},{"gid":7477,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6835/2749838911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2749838911},{"type":"v1","addr":"172.21.15.162:6835","nonce":2749838911}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":55}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:54:51.371126+0000","modified":"2020-11-28T21:54:53.015631+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7489},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7489":{"gid":7489,"name":"c","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/1061393314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1061393314},{"type":"v1","addr":"172.21.15.162:6837","nonce":1061393314}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2020-11-28T21:54:56.908 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.903+0000 7fb4df59e700 1 -- 172.21.15.162:0/2075503041 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb4bc055330 msgr2=0x7fb4bc0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:56.909 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.903+0000 7fb4df59e700 1 --2- 172.21.15.162:0/2075503041 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb4bc055330 0x7fb4bc0577f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7fb4d400b000 tx=0x7fb4d4006dc0).stop 2020-11-28T21:54:56.909 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.903+0000 7fb4df59e700 1 -- 172.21.15.162:0/2075503041 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4e00b4c70 msgr2=0x7fb4e0151530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:56.909 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.903+0000 7fb4df59e700 1 --2- 172.21.15.162:0/2075503041 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4e00b4c70 0x7fb4e0151530 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 rx=0x7fb4e00b3120 tx=0x7fb4d0015000).stop 2020-11-28T21:54:56.910 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.903+0000 7fb4df59e700 1 -- 172.21.15.162:0/2075503041 shutdown_connections 2020-11-28T21:54:56.910 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.903+0000 7fb4df59e700 1 --2- 172.21.15.162:0/2075503041 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb4bc055330 0x7fb4bc0577f0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:56.910 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.903+0000 7fb4df59e700 1 --2- 172.21.15.162:0/2075503041 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb4e0144d80 0x7fb4e0151a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:56.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.903+0000 7fb4df59e700 1 --2- 172.21.15.162:0/2075503041 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb4e00603d0 0x7fb4e0144890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:56.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.903+0000 7fb4df59e700 1 --2- 172.21.15.162:0/2075503041 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4e00b4c70 0x7fb4e0151530 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:56.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.903+0000 7fb4df59e700 1 -- 172.21.15.162:0/2075503041 >> 172.21.15.162:0/2075503041 conn(0x7fb4e0066a40 msgr2=0x7fb4e0141800 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:56.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.903+0000 7fb4df59e700 1 -- 172.21.15.162:0/2075503041 shutdown_connections 2020-11-28T21:54:56.912 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:56.903+0000 7fb4df59e700 1 -- 172.21.15.162:0/2075503041 wait complete. 2020-11-28T21:54:56.912 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 57 2020-11-28T21:54:57.223 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 57, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-28T21:54:51.371126+0000', 'modified': '2020-11-28T21:54:53.015631+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7489}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7489': {'gid': 7489, 'name': 'c', 'rank': 0, 'incarnation': 55, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6837/1061393314', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6836', 'nonce': 1061393314}, {'type': 'v1', 'addr': '172.21.15.162:6837', 'nonce': 1061393314}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-28T21:54:57.224 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-28T21:54:57.224 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-28T21:54:57.355 INFO:teuthology.orchestra.run.smithi162.stdout:{"cluster_fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","whoami":0,"id":7489,"want_state":"up:active","state":"up:active","rank_uptime":5.969510251,"mdsmap_epoch":57,"osdmap_epoch":119,"osdmap_epoch_barrier":119,"uptime":10.288212137} 2020-11-28T21:54:57.355 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", "id": 7489, "mdsmap_epoch": 57, "osdmap_epoch": 119, "osdmap_epoch_barrier": 119, "rank_uptime": 5.969510251, "state": "up:active", "uptime": 10.288212137, "want_state": "up:active", "whoami": 0 } 2020-11-28T21:54:57.356 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-11-28T21:54:57.356 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-11-28T21:54:57.357 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi192.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-11-28T21:54:57.357 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:57.357 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:54:57.365 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:54:57.423 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:54:57.423 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:54:57.478 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:54:57.479 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:54:57.480 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:57.480 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:54:57.526 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-11-28T21:54:57.527 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:54:57.571 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:54:57.583 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:54:57.584 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:54:57.632 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:54:57.633 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:54:57.634 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:57.634 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:54:57.645 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:2020-11-28T21:54:57.641+0000 7f10c85de0c0 -1 init, newargv = 0x560712ff9b40 newargc=9ceph-fuse[ 2020-11-28T21:54:57.645 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:12493]: starting ceph client 2020-11-28T21:54:57.657 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:ceph-fuse[12493]: starting fuse 2020-11-28T21:54:58.756 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:54:58.772 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:54:58.773 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:54:58.829 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:54:58.830 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:54:58.831 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:58.831 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:54:58.877 INFO:teuthology.orchestra.run.smithi192.stdout:49 2020-11-28T21:54:58.878 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2020-11-28T21:54:58.878 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:58.879 INFO:teuthology.orchestra.run.smithi192:> sudo python3 -c ' 2020-11-28T21:54:58.879 INFO:teuthology.orchestra.run.smithi192:> import glob 2020-11-28T21:54:58.879 INFO:teuthology.orchestra.run.smithi192:> import re 2020-11-28T21:54:58.880 INFO:teuthology.orchestra.run.smithi192:> import os 2020-11-28T21:54:58.880 INFO:teuthology.orchestra.run.smithi192:> import subprocess 2020-11-28T21:54:58.881 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:54:58.881 INFO:teuthology.orchestra.run.smithi192:> def find_socket(client_name): 2020-11-28T21:54:58.881 INFO:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-11-28T21:54:58.882 INFO:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2020-11-28T21:54:58.882 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:54:58.882 INFO:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2020-11-28T21:54:58.883 INFO:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2020-11-28T21:54:58.883 INFO:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2020-11-28T21:54:58.884 INFO:teuthology.orchestra.run.smithi192:> return files[0] 2020-11-28T21:54:58.884 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:54:58.884 INFO:teuthology.orchestra.run.smithi192:> for f in files: 2020-11-28T21:54:58.885 INFO:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-11-28T21:54:58.885 INFO:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2020-11-28T21:54:58.885 INFO:teuthology.orchestra.run.smithi192:> return f 2020-11-28T21:54:58.886 INFO:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-11-28T21:54:58.886 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:54:58.887 INFO:teuthology.orchestra.run.smithi192:> print(find_socket("client.0")) 2020-11-28T21:54:58.887 INFO:teuthology.orchestra.run.smithi192:> ' 2020-11-28T21:54:58.968 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.0.12493.asok 2020-11-28T21:54:58.973 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.12493.asok 2020-11-28T21:54:58.973 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:58.974 INFO:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.12493.asok status 2020-11-28T21:54:59.078 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2020-11-28T21:54:59.078 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2020-11-28T21:54:59.078 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "a83a3572a957356977342f461361c1cd3ec06dbd", 2020-11-28T21:54:59.079 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 15.2.6-595-ga83a3572 (a83a3572a957356977342f461361c1cd3ec06dbd) octopus (stable)", 2020-11-28T21:54:59.079 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "0", 2020-11-28T21:54:59.079 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2020-11-28T21:54:59.079 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-11-28T21:54:59.079 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "12493", 2020-11-28T21:54:59.080 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2020-11-28T21:54:59.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:54:59.080 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2020-11-28T21:54:59.080 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2020-11-28T21:54:59.080 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 7654, 2020-11-28T21:54:59.081 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2020-11-28T21:54:59.081 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2020-11-28T21:54:59.081 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2020-11-28T21:54:59.081 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 7654 2020-11-28T21:54:59.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:54:59.082 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2020-11-28T21:54:59.082 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2020-11-28T21:54:59.082 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2020-11-28T21:54:59.082 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 2600853361 2020-11-28T21:54:59.083 INFO:teuthology.orchestra.run.smithi192.stdout: } 2020-11-28T21:54:59.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:54:59.083 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2020-11-28T21:54:59.083 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2020-11-28T21:54:59.083 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2020-11-28T21:54:59.084 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 2600853361 2020-11-28T21:54:59.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:54:59.084 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.7654 172.21.15.192:0/2600853361", 2020-11-28T21:54:59.085 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "172.21.15.192:0/2600853361", 2020-11-28T21:54:59.086 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2020-11-28T21:54:59.086 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 57, 2020-11-28T21:54:59.086 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 119, 2020-11-28T21:54:59.086 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2020-11-28T21:54:59.087 INFO:teuthology.orchestra.run.smithi192.stdout: "blacklisted": false 2020-11-28T21:54:59.087 INFO:teuthology.orchestra.run.smithi192.stdout:} 2020-11-28T21:54:59.088 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:59.088 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:54:59.089 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:54:59.095 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2020-11-28T21:54:59.097 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-28T21:54:59.097 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:54:59.098 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:54:59.157 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:54:59.277 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.271+0000 7f0b42e1c700 1 Processor -- start 2020-11-28T21:54:59.277 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.271+0000 7f0b42e1c700 1 -- start start 2020-11-28T21:54:59.279 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.275+0000 7f0b42e1c700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b3c0a9bc0 0x7f0b3c0a9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:59.279 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.275+0000 7f0b42e1c700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b3c0b0f30 con 0x7f0b3c0b1fd0 2020-11-28T21:54:59.280 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.275+0000 7f0b42e1c700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b3c0af980 con 0x7f0b3c0aa490 2020-11-28T21:54:59.280 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.275+0000 7f0b42e1c700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c061030 con 0x7f0b3c0a9bc0 2020-11-28T21:54:59.280 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.275+0000 7f0b41619700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f0b3c0aa490 0x7f0b3c0afb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51120/0 (socket says 172.21.15.162:51120) 2020-11-28T21:54:59.280 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.275+0000 7f0b41619700 1 -- 172.21.15.162:0/2214333737 learned_addr learned my addr 172.21.15.162:0/2214333737 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:54:59.280 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.275+0000 7f0b41e1a700 1 --2- 172.21.15.162:0/2214333737 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b3c0a9bc0 0x7f0b3c0a9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:59.281 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.275+0000 7f0b40e18700 1 -- 172.21.15.162:0/2214333737 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2733218894 0 0) 0x7f0b3c0af980 con 0x7f0b3c0aa490 2020-11-28T21:54:59.281 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.275+0000 7f0b40e18700 1 -- 172.21.15.162:0/2214333737 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b24003230 con 0x7f0b3c0aa490 2020-11-28T21:54:59.281 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.275+0000 7f0b40e18700 1 -- 172.21.15.162:0/2214333737 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4160157718 0 0) 0x7f0b24003230 con 0x7f0b3c0aa490 2020-11-28T21:54:59.281 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.275+0000 7f0b40e18700 1 -- 172.21.15.162:0/2214333737 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b3c0a9bc0 msgr2=0x7f0b3c0a9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:59.281 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.275+0000 7f0b40e18700 1 --2- 172.21.15.162:0/2214333737 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b3c0a9bc0 0x7f0b3c0a9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:59.282 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.275+0000 7f0b40e18700 1 -- 172.21.15.162:0/2214333737 >> v1:172.21.15.118:6789/0 conn(0x7f0b3c0b1fd0 legacy=0x7f0b3c0aab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:59.282 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.275+0000 7f0b40e18700 1 -- 172.21.15.162:0/2214333737 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b3c0af6e0 con 0x7f0b3c0aa490 2020-11-28T21:54:59.282 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.275+0000 7f0b40e18700 1 -- 172.21.15.162:0/2214333737 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f0b2c0016d0 con 0x7f0b3c0aa490 2020-11-28T21:54:59.282 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.275+0000 7f0b40e18700 1 -- 172.21.15.162:0/2214333737 >> v1:172.21.15.162:6789/0 conn(0x7f0b3c0aa490 legacy=0x7f0b3c0afb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:59.283 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.275+0000 7f0b40e18700 1 --2- 172.21.15.162:0/2214333737 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b3c0a9bc0 0x7f0b24003720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:59.283 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.275+0000 7f0b40e18700 1 --2- 172.21.15.162:0/2214333737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b24003c10 0x7f0b240044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:59.283 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.275+0000 7f0b40e18700 1 --2- 172.21.15.162:0/2214333737 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0b24004090 0x7f0b240049c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:59.283 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.275+0000 7f0b40e18700 1 -- 172.21.15.162:0/2214333737 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c061030 con 0x7f0b24004090 2020-11-28T21:54:59.283 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.275+0000 7f0b40e18700 1 -- 172.21.15.162:0/2214333737 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b24004fe0 con 0x7f0b24003c10 2020-11-28T21:54:59.284 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.275+0000 7f0b40e18700 1 -- 172.21.15.162:0/2214333737 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b240051a0 con 0x7f0b3c0a9bc0 2020-11-28T21:54:59.284 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.275+0000 7f0b41619700 1 --2- 172.21.15.162:0/2214333737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b24003c10 0x7f0b240044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:59.284 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.275+0000 7f0b4261b700 1 --2- 172.21.15.162:0/2214333737 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0b24004090 0x7f0b240049c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:59.284 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.275+0000 7f0b41619700 1 -- 172.21.15.162:0/2214333737 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b3c0a9bc0 msgr2=0x7f0b24003720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:59.284 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.275+0000 7f0b41619700 1 --2- 172.21.15.162:0/2214333737 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b3c0a9bc0 0x7f0b24003720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-28T21:54:59.285 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.275+0000 7f0b41619700 1 -- 172.21.15.162:0/2214333737 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0b24004090 msgr2=0x7f0b240049c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:59.285 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.275+0000 7f0b41619700 1 --2- 172.21.15.162:0/2214333737 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0b24004090 0x7f0b240049c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:59.285 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.275+0000 7f0b41619700 1 -- 172.21.15.162:0/2214333737 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b24003230 con 0x7f0b24003c10 2020-11-28T21:54:59.285 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.275+0000 7f0b4261b700 1 --2- 172.21.15.162:0/2214333737 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0b24004090 0x7f0b240049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:54:59.285 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.275+0000 7f0b41619700 1 --2- 172.21.15.162:0/2214333737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b24003c10 0x7f0b240044d0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 rx=0x7f0b2c0013e0 tx=0x7f0b2c007770).ready entity=mon.1 client_cookie=a187c7fbac983414 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:59.289 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.275+0000 7f0b40e18700 1 -- 172.21.15.162:0/2214333737 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b2c0070d0 con 0x7f0b24003c10 2020-11-28T21:54:59.289 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.275+0000 7f0b40e18700 1 -- 172.21.15.162:0/2214333737 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b2c007230 con 0x7f0b24003c10 2020-11-28T21:54:59.290 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.275+0000 7f0b40e18700 1 -- 172.21.15.162:0/2214333737 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b2c0055e0 con 0x7f0b24003c10 2020-11-28T21:54:59.290 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.275+0000 7f0b42e1c700 1 -- 172.21.15.162:0/2214333737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b24003c10 msgr2=0x7f0b240044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:59.290 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.275+0000 7f0b42e1c700 1 --2- 172.21.15.162:0/2214333737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b24003c10 0x7f0b240044d0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 rx=0x7f0b2c0013e0 tx=0x7f0b2c007770).stop 2020-11-28T21:54:59.290 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.275+0000 7f0b4261b700 1 -- 172.21.15.162:0/2214333737 reap_dead start 2020-11-28T21:54:59.291 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.275+0000 7f0b42e1c700 1 -- 172.21.15.162:0/2214333737 shutdown_connections 2020-11-28T21:54:59.291 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.275+0000 7f0b42e1c700 1 -- 172.21.15.162:0/2214333737 >> 172.21.15.162:0/2214333737 conn(0x7f0b3c05b970 msgr2=0x7f0b3c05bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:59.291 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.275+0000 7f0b42e1c700 1 -- 172.21.15.162:0/2214333737 shutdown_connections 2020-11-28T21:54:59.291 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.275+0000 7f0b42e1c700 1 -- 172.21.15.162:0/2214333737 wait complete. 2020-11-28T21:54:59.291 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.279+0000 7f0b42e1c700 1 Processor -- start 2020-11-28T21:54:59.292 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.279+0000 7f0b42e1c700 1 -- start start 2020-11-28T21:54:59.292 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.279+0000 7f0b42e1c700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b3c0aabe0 0x7f0b3c150dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:59.292 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.279+0000 7f0b42e1c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0b3c065690 0x7f0b3c0991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:59.292 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.279+0000 7f0b42e1c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b3c0996e0 0x7f0b3c1406e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:59.292 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.279+0000 7f0b42e1c700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c064aa0 con 0x7f0b3c065690 2020-11-28T21:54:59.293 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.279+0000 7f0b42e1c700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c098dc0 con 0x7f0b3c0996e0 2020-11-28T21:54:59.293 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.279+0000 7f0b42e1c700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c0aafc0 con 0x7f0b3c0aabe0 2020-11-28T21:54:59.293 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.279+0000 7f0b4261b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b3c0996e0 0x7f0b3c1406e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:59.293 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.279+0000 7f0b4261b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b3c0996e0 0x7f0b3c1406e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44290/0 (socket says 172.21.15.162:44290) 2020-11-28T21:54:59.293 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.279+0000 7f0b4261b700 1 -- 172.21.15.162:0/1082288207 learned_addr learned my addr 172.21.15.162:0/1082288207 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:59.294 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.279+0000 7f0b41619700 1 --2- 172.21.15.162:0/1082288207 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0b3c065690 0x7f0b3c0991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:59.294 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.279+0000 7f0b41e1a700 1 --2- 172.21.15.162:0/1082288207 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b3c0aabe0 0x7f0b3c150dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:59.294 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.279+0000 7f0b4261b700 1 -- 172.21.15.162:0/1082288207 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b3c0aabe0 msgr2=0x7f0b3c150dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:59.294 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.279+0000 7f0b4261b700 1 --2- 172.21.15.162:0/1082288207 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b3c0aabe0 0x7f0b3c150dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:59.294 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.279+0000 7f0b4261b700 1 -- 172.21.15.162:0/1082288207 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0b3c065690 msgr2=0x7f0b3c0991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:59.295 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.279+0000 7f0b4261b700 1 --2- 172.21.15.162:0/1082288207 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0b3c065690 0x7f0b3c0991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:59.295 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.279+0000 7f0b4261b700 1 -- 172.21.15.162:0/1082288207 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b3c064cd0 con 0x7f0b3c0996e0 2020-11-28T21:54:59.295 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.279+0000 7f0b41619700 1 --2- 172.21.15.162:0/1082288207 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0b3c065690 0x7f0b3c0991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:54:59.295 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.279+0000 7f0b41e1a700 1 --2- 172.21.15.162:0/1082288207 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b3c0aabe0 0x7f0b3c150dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:54:59.296 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.279+0000 7f0b4261b700 1 --2- 172.21.15.162:0/1082288207 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b3c0996e0 0x7f0b3c1406e0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 rx=0x7f0b2c0010d0 tx=0x7f0b34009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:59.296 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.279+0000 7f0b2affd700 1 -- 172.21.15.162:0/1082288207 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b3400b570 con 0x7f0b3c0996e0 2020-11-28T21:54:59.296 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.279+0000 7f0b2affd700 1 -- 172.21.15.162:0/1082288207 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b3400b6d0 con 0x7f0b3c0996e0 2020-11-28T21:54:59.296 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.279+0000 7f0b42e1c700 1 -- 172.21.15.162:0/1082288207 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b3c064f70 con 0x7f0b3c0996e0 2020-11-28T21:54:59.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.279+0000 7f0b2affd700 1 -- 172.21.15.162:0/1082288207 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b3400ba30 con 0x7f0b3c0996e0 2020-11-28T21:54:59.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.279+0000 7f0b42e1c700 1 -- 172.21.15.162:0/1082288207 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b3c0916d0 con 0x7f0b3c0996e0 2020-11-28T21:54:59.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.283+0000 7f0b2affd700 1 -- 172.21.15.162:0/1082288207 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0b3401a030 con 0x7f0b3c0996e0 2020-11-28T21:54:59.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.283+0000 7f0b42e1c700 1 -- 172.21.15.162:0/1082288207 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b3c057050 con 0x7f0b3c0996e0 2020-11-28T21:54:59.298 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.283+0000 7f0b2affd700 1 --2- 172.21.15.162:0/1082288207 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0b1c055440 0x7f0b1c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:59.298 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.283+0000 7f0b41e1a700 1 --2- 172.21.15.162:0/1082288207 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0b1c055440 0x7f0b1c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:59.298 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.283+0000 7f0b2affd700 1 -- 172.21.15.162:0/1082288207 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0b3406dcc0 con 0x7f0b3c0996e0 2020-11-28T21:54:59.298 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.283+0000 7f0b41e1a700 1 --2- 172.21.15.162:0/1082288207 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0b1c055440 0x7f0b1c057900 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f0b38002000 tx=0x7f0b38008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:59.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.287+0000 7f0b2affd700 1 -- 172.21.15.162:0/1082288207 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0b34047860 con 0x7f0b3c0996e0 2020-11-28T21:54:59.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.431+0000 7f0b42e1c700 1 -- 172.21.15.162:0/1082288207 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0b3c0b09b0 con 0x7f0b3c0996e0 2020-11-28T21:54:59.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.431+0000 7f0b2affd700 1 -- 172.21.15.162:0/1082288207 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+16148 (secure 0 0 0) 0x7f0b34047450 con 0x7f0b3c0996e0 2020-11-28T21:54:59.437 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:54:59.437 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":119,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:54:52.374598+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:54:49.956401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2020-11-28T21:54:50.370123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:54:59.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.435+0000 7f0b42e1c700 1 -- 172.21.15.162:0/1082288207 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0b1c055440 msgr2=0x7f0b1c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:59.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.435+0000 7f0b42e1c700 1 --2- 172.21.15.162:0/1082288207 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0b1c055440 0x7f0b1c057900 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f0b38002000 tx=0x7f0b38008040).stop 2020-11-28T21:54:59.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.435+0000 7f0b42e1c700 1 -- 172.21.15.162:0/1082288207 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b3c0996e0 msgr2=0x7f0b3c1406e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:59.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.435+0000 7f0b42e1c700 1 --2- 172.21.15.162:0/1082288207 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b3c0996e0 0x7f0b3c1406e0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 rx=0x7f0b2c0010d0 tx=0x7f0b34009040).stop 2020-11-28T21:54:59.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.435+0000 7f0b42e1c700 1 -- 172.21.15.162:0/1082288207 shutdown_connections 2020-11-28T21:54:59.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.435+0000 7f0b42e1c700 1 --2- 172.21.15.162:0/1082288207 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0b1c055440 0x7f0b1c057900 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:59.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.435+0000 7f0b42e1c700 1 --2- 172.21.15.162:0/1082288207 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b3c0aabe0 0x7f0b3c150dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:59.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.435+0000 7f0b42e1c700 1 --2- 172.21.15.162:0/1082288207 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b3c0996e0 0x7f0b3c1406e0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:59.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.435+0000 7f0b42e1c700 1 --2- 172.21.15.162:0/1082288207 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0b3c065690 0x7f0b3c0991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:59.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.435+0000 7f0b42e1c700 1 -- 172.21.15.162:0/1082288207 >> 172.21.15.162:0/1082288207 conn(0x7f0b3c05b970 msgr2=0x7f0b3c142d40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:59.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.435+0000 7f0b42e1c700 1 -- 172.21.15.162:0/1082288207 shutdown_connections 2020-11-28T21:54:59.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.435+0000 7f0b42e1c700 1 -- 172.21.15.162:0/1082288207 wait complete. 2020-11-28T21:54:59.455 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-28T21:54:49.956401+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2020-11-28T21:54:50.370123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-28T21:54:59.456 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:54:59.576 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.571+0000 7fdf9359e700 1 Processor -- start 2020-11-28T21:54:59.576 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.571+0000 7fdf9359e700 1 -- start start 2020-11-28T21:54:59.578 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.571+0000 7fdf9359e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fdf9405cca0 0x7fdf940b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:59.578 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.571+0000 7fdf9359e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf940b1360 con 0x7fdf940b23a0 2020-11-28T21:54:59.578 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.571+0000 7fdf9359e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf940ae8c0 con 0x7fdf9413cb50 2020-11-28T21:54:59.579 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.571+0000 7fdf9359e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf940add60 con 0x7fdf9405cca0 2020-11-28T21:54:59.579 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.571+0000 7fdf91d9b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fdf9405cca0 0x7fdf940b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:59.579 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.571+0000 7fdf91d9b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fdf9405cca0 0x7fdf940b1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44298/0 (socket says 172.21.15.162:44298) 2020-11-28T21:54:59.579 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.571+0000 7fdf91d9b700 1 -- 172.21.15.162:0/2713764789 learned_addr learned my addr 172.21.15.162:0/2713764789 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:59.579 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.571+0000 7fdf9259c700 1 --1- 172.21.15.162:0/2713764789 >> v1:172.21.15.162:6789/0 conn(0x7fdf9413cb50 0x7fdf94140180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51140/0 (socket says 172.21.15.162:51140) 2020-11-28T21:54:59.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.575+0000 7fdf91d9b700 1 -- 172.21.15.162:0/2713764789 >> v1:172.21.15.162:6789/0 conn(0x7fdf9413cb50 legacy=0x7fdf94140180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:59.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.575+0000 7fdf91d9b700 1 -- 172.21.15.162:0/2713764789 >> v1:172.21.15.118:6789/0 conn(0x7fdf940b23a0 legacy=0x7fdf940ac350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:59.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.575+0000 7fdf91d9b700 1 -- 172.21.15.162:0/2713764789 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf940aeb60 con 0x7fdf9405cca0 2020-11-28T21:54:59.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.575+0000 7fdf91d9b700 1 --2- 172.21.15.162:0/2713764789 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf9405cca0 0x7fdf940b1eb0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 rx=0x7fdf84002f30 tx=0x7fdf84006e80).ready entity=mon.1 client_cookie=1e03f133319173d4 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:59.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.575+0000 7fdf9159a700 1 -- 172.21.15.162:0/2713764789 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf840097f0 con 0x7fdf9405cca0 2020-11-28T21:54:59.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.575+0000 7fdf9159a700 1 -- 172.21.15.162:0/2713764789 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf84009950 con 0x7fdf9405cca0 2020-11-28T21:54:59.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.575+0000 7fdf9159a700 1 -- 172.21.15.162:0/2713764789 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf84009cb0 con 0x7fdf9405cca0 2020-11-28T21:54:59.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.575+0000 7fdf9359e700 1 -- 172.21.15.162:0/2713764789 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf9405cca0 msgr2=0x7fdf940b1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:59.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.575+0000 7fdf9359e700 1 --2- 172.21.15.162:0/2713764789 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf9405cca0 0x7fdf940b1eb0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 rx=0x7fdf84002f30 tx=0x7fdf84006e80).stop 2020-11-28T21:54:59.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.575+0000 7fdf9359e700 1 -- 172.21.15.162:0/2713764789 shutdown_connections 2020-11-28T21:54:59.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.575+0000 7fdf9359e700 1 --2- 172.21.15.162:0/2713764789 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf9405cca0 0x7fdf940b1eb0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:59.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.575+0000 7fdf9359e700 1 -- 172.21.15.162:0/2713764789 >> 172.21.15.162:0/2713764789 conn(0x7fdf940b57d0 msgr2=0x7fdf940b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:59.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.575+0000 7fdf9359e700 1 -- 172.21.15.162:0/2713764789 shutdown_connections 2020-11-28T21:54:59.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.575+0000 7fdf9359e700 1 -- 172.21.15.162:0/2713764789 wait complete. 2020-11-28T21:54:59.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.575+0000 7fdf9359e700 1 Processor -- start 2020-11-28T21:54:59.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.575+0000 7fdf9359e700 1 -- start start 2020-11-28T21:54:59.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.575+0000 7fdf9359e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdf9405cca0 0x7fdf94140ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:59.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.575+0000 7fdf9359e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf9413cb50 0x7fdf940b3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:59.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.575+0000 7fdf9359e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdf940b3a10 0x7fdf9412b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:59.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.575+0000 7fdf9359e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf941415a0 con 0x7fdf940b3a10 2020-11-28T21:54:59.597 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.575+0000 7fdf9359e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9413a7b0 con 0x7fdf9413cb50 2020-11-28T21:54:59.597 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.575+0000 7fdf9359e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9413a510 con 0x7fdf9405cca0 2020-11-28T21:54:59.598 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.575+0000 7fdf91d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf9413cb50 0x7fdf940b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:59.598 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.575+0000 7fdf91d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf9413cb50 0x7fdf940b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44302/0 (socket says 172.21.15.162:44302) 2020-11-28T21:54:59.598 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.575+0000 7fdf91d9b700 1 -- 172.21.15.162:0/3573834145 learned_addr learned my addr 172.21.15.162:0/3573834145 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:59.598 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.575+0000 7fdf9259c700 1 --2- 172.21.15.162:0/3573834145 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdf9405cca0 0x7fdf94140ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:59.598 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.575+0000 7fdf92d9d700 1 --2- 172.21.15.162:0/3573834145 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdf940b3a10 0x7fdf9412b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:59.599 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.579+0000 7fdf91d9b700 1 -- 172.21.15.162:0/3573834145 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdf9405cca0 msgr2=0x7fdf94140ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:59.599 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.579+0000 7fdf91d9b700 1 --2- 172.21.15.162:0/3573834145 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdf9405cca0 0x7fdf94140ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:59.599 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.579+0000 7fdf91d9b700 1 -- 172.21.15.162:0/3573834145 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdf940b3a10 msgr2=0x7fdf9412b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:59.599 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.579+0000 7fdf91d9b700 1 --2- 172.21.15.162:0/3573834145 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdf940b3a10 0x7fdf9412b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:59.599 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.579+0000 7fdf91d9b700 1 -- 172.21.15.162:0/3573834145 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf940b44b0 con 0x7fdf9413cb50 2020-11-28T21:54:59.600 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.579+0000 7fdf91d9b700 1 --2- 172.21.15.162:0/3573834145 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf9413cb50 0x7fdf940b3520 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 rx=0x7fdf84002f30 tx=0x7fdf84015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:59.600 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.579+0000 7fdf8b7fe700 1 -- 172.21.15.162:0/3573834145 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf84011df0 con 0x7fdf9413cb50 2020-11-28T21:54:59.600 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.579+0000 7fdf8b7fe700 1 -- 172.21.15.162:0/3573834145 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf8401e460 con 0x7fdf9413cb50 2020-11-28T21:54:59.600 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.579+0000 7fdf9359e700 1 -- 172.21.15.162:0/3573834145 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf9413c930 con 0x7fdf9413cb50 2020-11-28T21:54:59.600 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.579+0000 7fdf8b7fe700 1 -- 172.21.15.162:0/3573834145 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf8401e7c0 con 0x7fdf9413cb50 2020-11-28T21:54:59.601 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.579+0000 7fdf9359e700 1 -- 172.21.15.162:0/3573834145 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf940b4950 con 0x7fdf9413cb50 2020-11-28T21:54:59.601 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.579+0000 7fdf9359e700 1 -- 172.21.15.162:0/3573834145 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf940b4640 con 0x7fdf9413cb50 2020-11-28T21:54:59.601 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.579+0000 7fdf8b7fe700 1 -- 172.21.15.162:0/3573834145 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fdf8401a040 con 0x7fdf9413cb50 2020-11-28T21:54:59.601 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.579+0000 7fdf8b7fe700 1 --2- 172.21.15.162:0/3573834145 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fdf780554e0 0x7fdf780579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:59.601 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.579+0000 7fdf8b7fe700 1 -- 172.21.15.162:0/3573834145 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdf8406f1e0 con 0x7fdf9413cb50 2020-11-28T21:54:59.603 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.579+0000 7fdf9259c700 1 --2- 172.21.15.162:0/3573834145 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fdf780554e0 0x7fdf780579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:59.603 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.579+0000 7fdf9259c700 1 --2- 172.21.15.162:0/3573834145 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fdf780554e0 0x7fdf780579a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7fdf7c00b000 tx=0x7fdf7c000b80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:59.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.587+0000 7fdf8b7fe700 1 -- 172.21.15.162:0/3573834145 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fdf84045440 con 0x7fdf9413cb50 2020-11-28T21:54:59.734 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.731+0000 7fdf9359e700 1 -- 172.21.15.162:0/3573834145 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fdf94057050 con 0x7fdf9413cb50 2020-11-28T21:54:59.735 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.731+0000 7fdf8b7fe700 1 -- 172.21.15.162:0/3573834145 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v119) v1 ==== 110+0+10 (secure 0 0 0) 0x7fdf84023020 con 0x7fdf9413cb50 2020-11-28T21:54:59.735 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:54:59.736 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.731+0000 7fdf9359e700 1 -- 172.21.15.162:0/3573834145 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fdf780554e0 msgr2=0x7fdf780579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:59.737 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.731+0000 7fdf9359e700 1 --2- 172.21.15.162:0/3573834145 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fdf780554e0 0x7fdf780579a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7fdf7c00b000 tx=0x7fdf7c000b80).stop 2020-11-28T21:54:59.737 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.731+0000 7fdf9359e700 1 -- 172.21.15.162:0/3573834145 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf9413cb50 msgr2=0x7fdf940b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:59.737 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.731+0000 7fdf9359e700 1 --2- 172.21.15.162:0/3573834145 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf9413cb50 0x7fdf940b3520 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 rx=0x7fdf84002f30 tx=0x7fdf84015000).stop 2020-11-28T21:54:59.738 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.731+0000 7fdf9359e700 1 -- 172.21.15.162:0/3573834145 shutdown_connections 2020-11-28T21:54:59.738 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.731+0000 7fdf9359e700 1 --2- 172.21.15.162:0/3573834145 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fdf780554e0 0x7fdf780579a0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:59.738 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.731+0000 7fdf9359e700 1 --2- 172.21.15.162:0/3573834145 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdf9405cca0 0x7fdf94140ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:59.739 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.731+0000 7fdf9359e700 1 --2- 172.21.15.162:0/3573834145 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdf940b3a10 0x7fdf9412b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:59.739 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.731+0000 7fdf9359e700 1 --2- 172.21.15.162:0/3573834145 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf9413cb50 0x7fdf940b3520 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:59.739 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.731+0000 7fdf9359e700 1 -- 172.21.15.162:0/3573834145 >> 172.21.15.162:0/3573834145 conn(0x7fdf940b57d0 msgr2=0x7fdf940ac270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:59.739 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.731+0000 7fdf9359e700 1 -- 172.21.15.162:0/3573834145 shutdown_connections 2020-11-28T21:54:59.739 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.731+0000 7fdf9359e700 1 -- 172.21.15.162:0/3573834145 wait complete. 2020-11-28T21:54:59.749 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:54:59.865 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.859+0000 7f5a97fff700 1 Processor -- start 2020-11-28T21:54:59.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.859+0000 7f5a97fff700 1 -- start start 2020-11-28T21:54:59.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.863+0000 7f5a97fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5a980ad240 0x7f5a98140040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:59.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.863+0000 7f5a97fff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5a980b1900 0x7f5a980b1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:59.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.863+0000 7f5a97fff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a980b1080 con 0x7f5a98142aa0 2020-11-28T21:54:59.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.863+0000 7f5a97fff700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a980b1490 con 0x7f5a980b1900 2020-11-28T21:54:59.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.863+0000 7f5a97fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9813f660 con 0x7f5a980ad240 2020-11-28T21:54:59.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.863+0000 7f5a96ffd700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5a980ad240 0x7f5a98140040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:59.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.863+0000 7f5a977fe700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5a980b1900 0x7f5a980b1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:59.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.863+0000 7f5a96ffd700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5a980ad240 0x7f5a98140040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:41534/0 (socket says 172.21.15.162:41534) 2020-11-28T21:54:59.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.863+0000 7f5a977fe700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5a980b1900 0x7f5a980b1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.162:51302/0 (socket says 172.21.15.162:51302) 2020-11-28T21:54:59.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.863+0000 7f5a96ffd700 1 -- 172.21.15.162:0/2503232740 learned_addr learned my addr 172.21.15.162:0/2503232740 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:59.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.863+0000 7f5a96ffd700 1 -- 172.21.15.162:0/2503232740 >> v2:172.21.15.118:3300/0 conn(0x7f5a980b1900 msgr2=0x7f5a980b1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:54:59.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.863+0000 7f5a96ffd700 1 --2- 172.21.15.162:0/2503232740 >> v2:172.21.15.118:3300/0 conn(0x7f5a980b1900 0x7f5a980b1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:59.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.863+0000 7f5a96ffd700 1 -- 172.21.15.162:0/2503232740 >> v1:172.21.15.118:6789/0 conn(0x7f5a98142aa0 legacy=0x7f5a980ad620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:59.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.863+0000 7f5a96ffd700 1 -- 172.21.15.162:0/2503232740 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a9813f900 con 0x7f5a980ad240 2020-11-28T21:54:59.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.863+0000 7f5a96ffd700 1 --2- 172.21.15.162:0/2503232740 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5a980ad240 0x7f5a98140040 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f5a880032b0 tx=0x7f5a88009270).ready entity=mon.2 client_cookie=5d1d5694edcaac2e server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:59.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.863+0000 7f5a95ffb700 1 -- 172.21.15.162:0/2503232740 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a88009e60 con 0x7f5a980ad240 2020-11-28T21:54:59.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.863+0000 7f5a95ffb700 1 -- 172.21.15.162:0/2503232740 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a8800bc00 con 0x7f5a980ad240 2020-11-28T21:54:59.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.863+0000 7f5a97fff700 1 -- 172.21.15.162:0/2503232740 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5a980ad240 msgr2=0x7f5a98140040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:59.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.863+0000 7f5a95ffb700 1 -- 172.21.15.162:0/2503232740 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a8800bd70 con 0x7f5a980ad240 2020-11-28T21:54:59.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.863+0000 7f5a97fff700 1 --2- 172.21.15.162:0/2503232740 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5a980ad240 0x7f5a98140040 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f5a880032b0 tx=0x7f5a88009270).stop 2020-11-28T21:54:59.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.863+0000 7f5a97fff700 1 -- 172.21.15.162:0/2503232740 shutdown_connections 2020-11-28T21:54:59.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.863+0000 7f5a97fff700 1 --2- 172.21.15.162:0/2503232740 >> v2:172.21.15.118:3300/0 conn(0x7f5a980b1900 0x7f5a980b1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:59.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.863+0000 7f5a97fff700 1 --2- 172.21.15.162:0/2503232740 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5a980ad240 0x7f5a98140040 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:59.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.863+0000 7f5a97fff700 1 -- 172.21.15.162:0/2503232740 >> 172.21.15.162:0/2503232740 conn(0x7f5a980b2830 msgr2=0x7f5a980b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:54:59.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.863+0000 7f5a97fff700 1 -- 172.21.15.162:0/2503232740 shutdown_connections 2020-11-28T21:54:59.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.863+0000 7f5a97fff700 1 -- 172.21.15.162:0/2503232740 wait complete. 2020-11-28T21:54:59.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.867+0000 7f5a97fff700 1 Processor -- start 2020-11-28T21:54:59.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.867+0000 7f5a97fff700 1 -- start start 2020-11-28T21:54:59.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.867+0000 7f5a97fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5a980ad240 0x7f5a980b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:59.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.867+0000 7f5a97fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5a980b1900 0x7f5a98135ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:59.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.867+0000 7f5a97fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5a98142aa0 0x7f5a981363b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:59.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.867+0000 7f5a97fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a98141390 con 0x7f5a980b1900 2020-11-28T21:54:59.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.867+0000 7f5a97fff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a98141630 con 0x7f5a98142aa0 2020-11-28T21:54:59.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.867+0000 7f5a97fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a981418d0 con 0x7f5a980ad240 2020-11-28T21:54:59.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.867+0000 7f5a977fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5a98142aa0 0x7f5a981363b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:59.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.867+0000 7f5a977fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5a98142aa0 0x7f5a981363b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44320/0 (socket says 172.21.15.162:44320) 2020-11-28T21:54:59.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.867+0000 7f5a977fe700 1 -- 172.21.15.162:0/3704832983 learned_addr learned my addr 172.21.15.162:0/3704832983 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:54:59.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.867+0000 7f5a96ffd700 1 --2- 172.21.15.162:0/3704832983 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5a980ad240 0x7f5a980b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:59.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.867+0000 7f5a967fc700 1 --2- 172.21.15.162:0/3704832983 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5a980b1900 0x7f5a98135ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:59.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.867+0000 7f5a977fe700 1 -- 172.21.15.162:0/3704832983 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5a980ad240 msgr2=0x7f5a980b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:59.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.867+0000 7f5a977fe700 1 --2- 172.21.15.162:0/3704832983 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5a980ad240 0x7f5a980b3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:59.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.867+0000 7f5a977fe700 1 -- 172.21.15.162:0/3704832983 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5a980b1900 msgr2=0x7f5a98135ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:54:59.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.867+0000 7f5a977fe700 1 --2- 172.21.15.162:0/3704832983 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5a980b1900 0x7f5a98135ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:54:59.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.867+0000 7f5a977fe700 1 -- 172.21.15.162:0/3704832983 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a98141b70 con 0x7f5a98142aa0 2020-11-28T21:54:59.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.867+0000 7f5a977fe700 1 --2- 172.21.15.162:0/3704832983 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5a98142aa0 0x7f5a981363b0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 rx=0x7f5a90006a50 tx=0x7f5a90006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:59.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.867+0000 7f5a87fff700 1 -- 172.21.15.162:0/3704832983 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a9000ca10 con 0x7f5a98142aa0 2020-11-28T21:54:59.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.867+0000 7f5a87fff700 1 -- 172.21.15.162:0/3704832983 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a9000be90 con 0x7f5a98142aa0 2020-11-28T21:54:59.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.867+0000 7f5a97fff700 1 -- 172.21.15.162:0/3704832983 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a981455c0 con 0x7f5a98142aa0 2020-11-28T21:54:59.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.867+0000 7f5a87fff700 1 -- 172.21.15.162:0/3704832983 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a9000cd60 con 0x7f5a98142aa0 2020-11-28T21:54:59.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.867+0000 7f5a97fff700 1 -- 172.21.15.162:0/3704832983 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a98141ff0 con 0x7f5a98142aa0 2020-11-28T21:54:59.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.871+0000 7f5a97fff700 1 -- 172.21.15.162:0/3704832983 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a98141d00 con 0x7f5a98142aa0 2020-11-28T21:54:59.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.871+0000 7f5a87fff700 1 -- 172.21.15.162:0/3704832983 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5a9001c460 con 0x7f5a98142aa0 2020-11-28T21:54:59.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.871+0000 7f5a87fff700 1 --2- 172.21.15.162:0/3704832983 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5a740554e0 0x7f5a740579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:54:59.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.871+0000 7f5a87fff700 1 -- 172.21.15.162:0/3704832983 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5a9006e320 con 0x7f5a98142aa0 2020-11-28T21:54:59.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.871+0000 7f5a96ffd700 1 --2- 172.21.15.162:0/3704832983 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5a740554e0 0x7f5a740579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:54:59.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.871+0000 7f5a96ffd700 1 --2- 172.21.15.162:0/3704832983 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5a740554e0 0x7f5a740579a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7f5a8801a000 tx=0x7f5a88016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:54:59.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:54:59.879+0000 7f5a87fff700 1 -- 172.21.15.162:0/3704832983 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5a900445c0 con 0x7f5a98142aa0 2020-11-28T21:55:00.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.019+0000 7f5a97fff700 1 -- 172.21.15.162:0/3704832983 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5a98057050 con 0x7f5a98142aa0 2020-11-28T21:55:00.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.019+0000 7f5a87fff700 1 -- 172.21.15.162:0/3704832983 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v119) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5a90047be0 con 0x7f5a98142aa0 2020-11-28T21:55:00.024 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:55:00.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.023+0000 7f5a97fff700 1 -- 172.21.15.162:0/3704832983 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5a740554e0 msgr2=0x7f5a740579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:00.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.023+0000 7f5a97fff700 1 --2- 172.21.15.162:0/3704832983 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5a740554e0 0x7f5a740579a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7f5a8801a000 tx=0x7f5a88016000).stop 2020-11-28T21:55:00.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.023+0000 7f5a97fff700 1 -- 172.21.15.162:0/3704832983 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5a98142aa0 msgr2=0x7f5a981363b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:00.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.023+0000 7f5a97fff700 1 --2- 172.21.15.162:0/3704832983 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5a98142aa0 0x7f5a981363b0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 rx=0x7f5a90006a50 tx=0x7f5a90006dc0).stop 2020-11-28T21:55:00.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.023+0000 7f5a97fff700 1 -- 172.21.15.162:0/3704832983 shutdown_connections 2020-11-28T21:55:00.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.023+0000 7f5a97fff700 1 --2- 172.21.15.162:0/3704832983 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5a740554e0 0x7f5a740579a0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:00.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.023+0000 7f5a97fff700 1 --2- 172.21.15.162:0/3704832983 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5a980ad240 0x7f5a980b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:00.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.023+0000 7f5a97fff700 1 --2- 172.21.15.162:0/3704832983 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5a98142aa0 0x7f5a981363b0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:00.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.023+0000 7f5a97fff700 1 --2- 172.21.15.162:0/3704832983 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5a980b1900 0x7f5a98135ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:00.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.023+0000 7f5a97fff700 1 -- 172.21.15.162:0/3704832983 >> 172.21.15.162:0/3704832983 conn(0x7f5a980b2830 msgr2=0x7f5a9813e790 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:00.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.023+0000 7f5a97fff700 1 -- 172.21.15.162:0/3704832983 shutdown_connections 2020-11-28T21:55:00.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.023+0000 7f5a97fff700 1 -- 172.21.15.162:0/3704832983 wait complete. 2020-11-28T21:55:00.036 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-28T21:55:00.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.147+0000 7fef5ec3b700 1 Processor -- start 2020-11-28T21:55:00.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.147+0000 7fef5ec3b700 1 -- start start 2020-11-28T21:55:00.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.147+0000 7fef5ec3b700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fef580b18c0 0x7fef580b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:00.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.147+0000 7fef5ec3b700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fef580b44e0 0x7fef580b2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:00.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.147+0000 7fef5ec3b700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef580b1f50 con 0x7fef580b3950 2020-11-28T21:55:00.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.147+0000 7fef5ec3b700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef580aab50 con 0x7fef580b18c0 2020-11-28T21:55:00.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.147+0000 7fef5ec3b700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef58143e90 con 0x7fef580b44e0 2020-11-28T21:55:00.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.151+0000 7fef5dc39700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fef580b3950 0x7fef5813dd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51168/0 (socket says 172.21.15.162:51168) 2020-11-28T21:55:00.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.151+0000 7fef5dc39700 1 -- 172.21.15.162:0/3426817078 learned_addr learned my addr 172.21.15.162:0/3426817078 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:00.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.151+0000 7fef5e43a700 1 --2- 172.21.15.162:0/3426817078 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fef580b44e0 0x7fef580b2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:00.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.151+0000 7fef5d438700 1 --2- 172.21.15.162:0/3426817078 >> v2:172.21.15.118:3300/0 conn(0x7fef580b18c0 0x7fef580b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:00.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.151+0000 7fef5cc37700 1 -- 172.21.15.162:0/3426817078 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2936769097 0 0) 0x7fef580b1f50 con 0x7fef580b3950 2020-11-28T21:55:00.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.151+0000 7fef5cc37700 1 -- 172.21.15.162:0/3426817078 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef44003230 con 0x7fef580b3950 2020-11-28T21:55:00.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.151+0000 7fef5e43a700 1 -- 172.21.15.162:0/3426817078 >> v2:172.21.15.118:3300/0 conn(0x7fef580b18c0 msgr2=0x7fef580b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:00.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.151+0000 7fef5e43a700 1 --2- 172.21.15.162:0/3426817078 >> v2:172.21.15.118:3300/0 conn(0x7fef580b18c0 0x7fef580b3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:00.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.151+0000 7fef5e43a700 1 -- 172.21.15.162:0/3426817078 >> v1:172.21.15.162:6789/0 conn(0x7fef580b3950 legacy=0x7fef5813dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:00.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.151+0000 7fef5e43a700 1 -- 172.21.15.162:0/3426817078 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef580aa8b0 con 0x7fef580b44e0 2020-11-28T21:55:00.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.151+0000 7fef5e43a700 1 --2- 172.21.15.162:0/3426817078 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fef580b44e0 0x7fef580b2c10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fef48002f30 tx=0x7fef48006dc0).ready entity=mon.2 client_cookie=30097329b8aa3b11 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:00.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.151+0000 7fef5cc37700 1 -- 172.21.15.162:0/3426817078 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef480096a0 con 0x7fef580b44e0 2020-11-28T21:55:00.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.151+0000 7fef5cc37700 1 -- 172.21.15.162:0/3426817078 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef48009800 con 0x7fef580b44e0 2020-11-28T21:55:00.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.151+0000 7fef5cc37700 1 -- 172.21.15.162:0/3426817078 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef48009b60 con 0x7fef580b44e0 2020-11-28T21:55:00.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.151+0000 7fef5ec3b700 1 -- 172.21.15.162:0/3426817078 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fef580b44e0 msgr2=0x7fef580b2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:00.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.151+0000 7fef5ec3b700 1 --2- 172.21.15.162:0/3426817078 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fef580b44e0 0x7fef580b2c10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fef48002f30 tx=0x7fef48006dc0).stop 2020-11-28T21:55:00.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.151+0000 7fef5ec3b700 1 -- 172.21.15.162:0/3426817078 shutdown_connections 2020-11-28T21:55:00.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.151+0000 7fef5ec3b700 1 --2- 172.21.15.162:0/3426817078 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fef580b44e0 0x7fef580b2c10 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:00.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.151+0000 7fef5ec3b700 1 --2- 172.21.15.162:0/3426817078 >> v2:172.21.15.118:3300/0 conn(0x7fef580b18c0 0x7fef580b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:00.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.151+0000 7fef5ec3b700 1 -- 172.21.15.162:0/3426817078 >> 172.21.15.162:0/3426817078 conn(0x7fef58062630 msgr2=0x7fef58062a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:00.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.151+0000 7fef5ec3b700 1 -- 172.21.15.162:0/3426817078 shutdown_connections 2020-11-28T21:55:00.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.151+0000 7fef5ec3b700 1 -- 172.21.15.162:0/3426817078 wait complete. 2020-11-28T21:55:00.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.151+0000 7fef5ec3b700 1 Processor -- start 2020-11-28T21:55:00.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.151+0000 7fef5ec3b700 1 -- start start 2020-11-28T21:55:00.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.155+0000 7fef5ec3b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fef580b18c0 0x7fef580ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:00.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.155+0000 7fef5ec3b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef580b3950 0x7fef58147850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:00.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.155+0000 7fef5ec3b700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fef580b44e0 0x7fef580ac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:00.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.155+0000 7fef5ec3b700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef581392a0 con 0x7fef580b18c0 2020-11-28T21:55:00.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.155+0000 7fef5ec3b700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef580ae6c0 con 0x7fef580b3950 2020-11-28T21:55:00.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.155+0000 7fef5ec3b700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef580ae950 con 0x7fef580b44e0 2020-11-28T21:55:00.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.155+0000 7fef5d438700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef580b3950 0x7fef58147850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:00.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.155+0000 7fef5d438700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef580b3950 0x7fef58147850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44330/0 (socket says 172.21.15.162:44330) 2020-11-28T21:55:00.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.155+0000 7fef5d438700 1 -- 172.21.15.162:0/3222866389 learned_addr learned my addr 172.21.15.162:0/3222866389 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:00.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.155+0000 7fef5e43a700 1 --2- 172.21.15.162:0/3222866389 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fef580b44e0 0x7fef580ac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:00.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.155+0000 7fef5dc39700 1 --2- 172.21.15.162:0/3222866389 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fef580b18c0 0x7fef580ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:00.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.155+0000 7fef5d438700 1 -- 172.21.15.162:0/3222866389 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fef580b44e0 msgr2=0x7fef580ac1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:00.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.155+0000 7fef5d438700 1 --2- 172.21.15.162:0/3222866389 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fef580b44e0 0x7fef580ac1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:00.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.155+0000 7fef5d438700 1 -- 172.21.15.162:0/3222866389 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fef580b18c0 msgr2=0x7fef580ae1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:00.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.155+0000 7fef5d438700 1 --2- 172.21.15.162:0/3222866389 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fef580b18c0 0x7fef580ae1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:00.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.155+0000 7fef5d438700 1 -- 172.21.15.162:0/3222866389 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef580aebf0 con 0x7fef580b3950 2020-11-28T21:55:00.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.155+0000 7fef5dc39700 1 --2- 172.21.15.162:0/3222866389 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fef580b18c0 0x7fef580ae1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:55:00.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.155+0000 7fef5d438700 1 --2- 172.21.15.162:0/3222866389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef580b3950 0x7fef58147850 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 rx=0x7fef4c00c000 tx=0x7fef4c006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:00.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.155+0000 7fef56ffd700 1 -- 172.21.15.162:0/3222866389 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef4c00ca10 con 0x7fef580b3950 2020-11-28T21:55:00.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.155+0000 7fef56ffd700 1 -- 172.21.15.162:0/3222866389 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef4c00cb70 con 0x7fef580b3950 2020-11-28T21:55:00.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.155+0000 7fef56ffd700 1 -- 172.21.15.162:0/3222866389 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef4c0123f0 con 0x7fef580b3950 2020-11-28T21:55:00.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.155+0000 7fef5ec3b700 1 -- 172.21.15.162:0/3222866389 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef58144fb0 con 0x7fef580b3950 2020-11-28T21:55:00.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.155+0000 7fef5ec3b700 1 -- 172.21.15.162:0/3222866389 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef58147d40 con 0x7fef580b3950 2020-11-28T21:55:00.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.155+0000 7fef5ec3b700 1 -- 172.21.15.162:0/3222866389 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef58057050 con 0x7fef580b3950 2020-11-28T21:55:00.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.155+0000 7fef56ffd700 1 -- 172.21.15.162:0/3222866389 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fef4c012550 con 0x7fef580b3950 2020-11-28T21:55:00.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.159+0000 7fef56ffd700 1 --2- 172.21.15.162:0/3222866389 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fef38055440 0x7fef38057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:00.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.159+0000 7fef56ffd700 1 -- 172.21.15.162:0/3222866389 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fef4c06e580 con 0x7fef580b3950 2020-11-28T21:55:00.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.159+0000 7fef5dc39700 1 --2- 172.21.15.162:0/3222866389 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fef38055440 0x7fef38057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:00.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.159+0000 7fef5dc39700 1 --2- 172.21.15.162:0/3222866389 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fef38055440 0x7fef38057900 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7fef5000b000 tx=0x7fef50009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:00.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.163+0000 7fef56ffd700 1 -- 172.21.15.162:0/3222866389 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fef4c044710 con 0x7fef580b3950 2020-11-28T21:55:00.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.303+0000 7fef5ec3b700 1 -- 172.21.15.162:0/3222866389 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fef5805d250 con 0x7fef580b3950 2020-11-28T21:55:00.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.303+0000 7fef56ffd700 1 -- 172.21.15.162:0/3222866389 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v119) v1 ==== 104+0+10 (secure 0 0 0) 0x7fef4c047d30 con 0x7fef580b3950 2020-11-28T21:55:00.309 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:55:00.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.307+0000 7fef5ec3b700 1 -- 172.21.15.162:0/3222866389 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fef38055440 msgr2=0x7fef38057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:00.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.307+0000 7fef5ec3b700 1 --2- 172.21.15.162:0/3222866389 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fef38055440 0x7fef38057900 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7fef5000b000 tx=0x7fef50009040).stop 2020-11-28T21:55:00.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.307+0000 7fef5ec3b700 1 -- 172.21.15.162:0/3222866389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef580b3950 msgr2=0x7fef58147850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:00.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.307+0000 7fef5ec3b700 1 --2- 172.21.15.162:0/3222866389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef580b3950 0x7fef58147850 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 rx=0x7fef4c00c000 tx=0x7fef4c006dc0).stop 2020-11-28T21:55:00.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.307+0000 7fef5ec3b700 1 -- 172.21.15.162:0/3222866389 shutdown_connections 2020-11-28T21:55:00.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.307+0000 7fef5ec3b700 1 --2- 172.21.15.162:0/3222866389 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fef38055440 0x7fef38057900 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:00.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.307+0000 7fef5ec3b700 1 --2- 172.21.15.162:0/3222866389 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fef580b44e0 0x7fef580ac1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:00.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.307+0000 7fef5ec3b700 1 --2- 172.21.15.162:0/3222866389 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fef580b18c0 0x7fef580ae1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:00.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.307+0000 7fef5ec3b700 1 --2- 172.21.15.162:0/3222866389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef580b3950 0x7fef58147850 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:00.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.307+0000 7fef5ec3b700 1 -- 172.21.15.162:0/3222866389 >> 172.21.15.162:0/3222866389 conn(0x7fef58062630 msgr2=0x7fef58062a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:00.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.307+0000 7fef5ec3b700 1 -- 172.21.15.162:0/3222866389 shutdown_connections 2020-11-28T21:55:00.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.307+0000 7fef5ec3b700 1 -- 172.21.15.162:0/3222866389 wait complete. 2020-11-28T21:55:00.328 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-28T21:55:00.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.467+0000 7f8b38d46700 1 Processor -- start 2020-11-28T21:55:00.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.467+0000 7f8b38d46700 1 -- start start 2020-11-28T21:55:00.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.467+0000 7f8b38d46700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f8b340afec0 0x7f8b340b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:00.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.467+0000 7f8b38d46700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8b34099d30 0x7f8b34148d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:00.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.467+0000 7f8b38d46700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b34060500 con 0x7f8b340a8df0 2020-11-28T21:55:00.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.467+0000 7f8b38d46700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b34060ce0 con 0x7f8b340afec0 2020-11-28T21:55:00.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.467+0000 7f8b38d46700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b340607a0 con 0x7f8b34099d30 2020-11-28T21:55:00.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.467+0000 7f8b33fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8b34099d30 0x7f8b34148d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:00.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.467+0000 7f8b32ffd700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f8b340a8df0 0x7f8b34099620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51182/0 (socket says 172.21.15.162:51182) 2020-11-28T21:55:00.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.467+0000 7f8b32ffd700 1 -- 172.21.15.162:0/2452360296 learned_addr learned my addr 172.21.15.162:0/2452360296 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:00.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.467+0000 7f8b337fe700 1 --2- 172.21.15.162:0/2452360296 >> v2:172.21.15.118:3300/0 conn(0x7f8b340afec0 0x7f8b340b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:00.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.467+0000 7f8b33fff700 1 -- 172.21.15.162:0/2452360296 >> v2:172.21.15.118:3300/0 conn(0x7f8b340afec0 msgr2=0x7f8b340b02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:00.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.467+0000 7f8b33fff700 1 --2- 172.21.15.162:0/2452360296 >> v2:172.21.15.118:3300/0 conn(0x7f8b340afec0 0x7f8b340b02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:00.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.467+0000 7f8b33fff700 1 -- 172.21.15.162:0/2452360296 >> v1:172.21.15.162:6789/0 conn(0x7f8b340a8df0 legacy=0x7f8b34099620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:00.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.467+0000 7f8b33fff700 1 -- 172.21.15.162:0/2452360296 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b34060a40 con 0x7f8b34099d30 2020-11-28T21:55:00.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.467+0000 7f8b33fff700 1 --2- 172.21.15.162:0/2452360296 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b34099d30 0x7f8b34148d80 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 rx=0x7f8b2c0032b0 tx=0x7f8b2c0091a0).ready entity=mon.1 client_cookie=46fdea797463430a server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:00.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.467+0000 7f8b327fc700 1 -- 172.21.15.162:0/2452360296 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b2c00ba50 con 0x7f8b34099d30 2020-11-28T21:55:00.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.467+0000 7f8b327fc700 1 -- 172.21.15.162:0/2452360296 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b2c00bbb0 con 0x7f8b34099d30 2020-11-28T21:55:00.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.467+0000 7f8b327fc700 1 -- 172.21.15.162:0/2452360296 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b2c00bd20 con 0x7f8b34099d30 2020-11-28T21:55:00.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.467+0000 7f8b38d46700 1 -- 172.21.15.162:0/2452360296 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b34099d30 msgr2=0x7f8b34148d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:00.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.467+0000 7f8b38d46700 1 --2- 172.21.15.162:0/2452360296 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b34099d30 0x7f8b34148d80 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 rx=0x7f8b2c0032b0 tx=0x7f8b2c0091a0).stop 2020-11-28T21:55:00.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.467+0000 7f8b38d46700 1 -- 172.21.15.162:0/2452360296 shutdown_connections 2020-11-28T21:55:00.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.467+0000 7f8b38d46700 1 --2- 172.21.15.162:0/2452360296 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b34099d30 0x7f8b34148d80 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:00.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.467+0000 7f8b38d46700 1 --2- 172.21.15.162:0/2452360296 >> v2:172.21.15.118:3300/0 conn(0x7f8b340afec0 0x7f8b340b02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:00.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.467+0000 7f8b38d46700 1 -- 172.21.15.162:0/2452360296 >> 172.21.15.162:0/2452360296 conn(0x7f8b340b5520 msgr2=0x7f8b340b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:00.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.467+0000 7f8b38d46700 1 -- 172.21.15.162:0/2452360296 shutdown_connections 2020-11-28T21:55:00.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.467+0000 7f8b38d46700 1 -- 172.21.15.162:0/2452360296 wait complete. 2020-11-28T21:55:00.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.471+0000 7f8b38d46700 1 Processor -- start 2020-11-28T21:55:00.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.471+0000 7f8b38d46700 1 -- start start 2020-11-28T21:55:00.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.471+0000 7f8b38d46700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8b34099d30 0x7f8b34147990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:00.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.471+0000 7f8b38d46700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b340a8df0 0x7f8b340a9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:00.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.471+0000 7f8b38d46700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8b340afec0 0x7f8b340ac310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:00.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.471+0000 7f8b38d46700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b340a9450 con 0x7f8b34099d30 2020-11-28T21:55:00.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.471+0000 7f8b38d46700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b340a96f0 con 0x7f8b340a8df0 2020-11-28T21:55:00.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.471+0000 7f8b38d46700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b340a9990 con 0x7f8b340afec0 2020-11-28T21:55:00.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.471+0000 7f8b32ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b340a8df0 0x7f8b340a9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:00.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.471+0000 7f8b32ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b340a8df0 0x7f8b340a9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44346/0 (socket says 172.21.15.162:44346) 2020-11-28T21:55:00.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.471+0000 7f8b32ffd700 1 -- 172.21.15.162:0/2188098040 learned_addr learned my addr 172.21.15.162:0/2188098040 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:00.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.471+0000 7f8b33fff700 1 --2- 172.21.15.162:0/2188098040 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8b340afec0 0x7f8b340ac310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:00.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.471+0000 7f8b337fe700 1 --2- 172.21.15.162:0/2188098040 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8b34099d30 0x7f8b34147990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:00.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.471+0000 7f8b32ffd700 1 -- 172.21.15.162:0/2188098040 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8b340afec0 msgr2=0x7f8b340ac310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:00.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.471+0000 7f8b32ffd700 1 --2- 172.21.15.162:0/2188098040 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8b340afec0 0x7f8b340ac310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:00.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.471+0000 7f8b32ffd700 1 -- 172.21.15.162:0/2188098040 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8b34099d30 msgr2=0x7f8b34147990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:00.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.471+0000 7f8b32ffd700 1 --2- 172.21.15.162:0/2188098040 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8b34099d30 0x7f8b34147990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:00.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.471+0000 7f8b32ffd700 1 -- 172.21.15.162:0/2188098040 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b3414ad90 con 0x7f8b340a8df0 2020-11-28T21:55:00.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.471+0000 7f8b337fe700 1 --2- 172.21.15.162:0/2188098040 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8b34099d30 0x7f8b34147990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:55:00.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.471+0000 7f8b32ffd700 1 --2- 172.21.15.162:0/2188098040 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b340a8df0 0x7f8b340a9e30 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 rx=0x7f8b340629b0 tx=0x7f8b20007c10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:00.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.471+0000 7f8b30ff9700 1 -- 172.21.15.162:0/2188098040 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b200072e0 con 0x7f8b340a8df0 2020-11-28T21:55:00.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.471+0000 7f8b30ff9700 1 -- 172.21.15.162:0/2188098040 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b20007440 con 0x7f8b340a8df0 2020-11-28T21:55:00.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.471+0000 7f8b38d46700 1 -- 172.21.15.162:0/2188098040 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b340aa790 con 0x7f8b340a8df0 2020-11-28T21:55:00.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.471+0000 7f8b30ff9700 1 -- 172.21.15.162:0/2188098040 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b200057d0 con 0x7f8b340a8df0 2020-11-28T21:55:00.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.471+0000 7f8b38d46700 1 -- 172.21.15.162:0/2188098040 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b340ac800 con 0x7f8b340a8df0 2020-11-28T21:55:00.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.471+0000 7f8b38d46700 1 -- 172.21.15.162:0/2188098040 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b340a9b50 con 0x7f8b340a8df0 2020-11-28T21:55:00.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.475+0000 7f8b30ff9700 1 -- 172.21.15.162:0/2188098040 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8b20010070 con 0x7f8b340a8df0 2020-11-28T21:55:00.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.475+0000 7f8b30ff9700 1 --2- 172.21.15.162:0/2188098040 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8b10055440 0x7f8b10057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:00.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.475+0000 7f8b30ff9700 1 -- 172.21.15.162:0/2188098040 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8b2006d550 con 0x7f8b340a8df0 2020-11-28T21:55:00.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.475+0000 7f8b337fe700 1 --2- 172.21.15.162:0/2188098040 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8b10055440 0x7f8b10057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:00.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.475+0000 7f8b337fe700 1 --2- 172.21.15.162:0/2188098040 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8b10055440 0x7f8b10057900 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f8b2400c000 tx=0x7f8b24008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:00.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.483+0000 7f8b30ff9700 1 -- 172.21.15.162:0/2188098040 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8b20043850 con 0x7f8b340a8df0 2020-11-28T21:55:00.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.623+0000 7f8b38d46700 1 -- 172.21.15.162:0/2188098040 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8b34057050 con 0x7f8b340a8df0 2020-11-28T21:55:00.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.623+0000 7f8b30ff9700 1 -- 172.21.15.162:0/2188098040 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v119) v1 ==== 100+0+10 (secure 0 0 0) 0x7f8b20046e70 con 0x7f8b340a8df0 2020-11-28T21:55:00.630 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:55:00.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.627+0000 7f8b38d46700 1 -- 172.21.15.162:0/2188098040 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8b10055440 msgr2=0x7f8b10057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:00.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.627+0000 7f8b38d46700 1 --2- 172.21.15.162:0/2188098040 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8b10055440 0x7f8b10057900 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f8b2400c000 tx=0x7f8b24008040).stop 2020-11-28T21:55:00.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.627+0000 7f8b38d46700 1 -- 172.21.15.162:0/2188098040 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b340a8df0 msgr2=0x7f8b340a9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:00.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.627+0000 7f8b38d46700 1 --2- 172.21.15.162:0/2188098040 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b340a8df0 0x7f8b340a9e30 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 rx=0x7f8b340629b0 tx=0x7f8b20007c10).stop 2020-11-28T21:55:00.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.627+0000 7f8b38d46700 1 -- 172.21.15.162:0/2188098040 shutdown_connections 2020-11-28T21:55:00.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.627+0000 7f8b38d46700 1 --2- 172.21.15.162:0/2188098040 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8b10055440 0x7f8b10057900 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:00.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.627+0000 7f8b38d46700 1 --2- 172.21.15.162:0/2188098040 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8b340afec0 0x7f8b340ac310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:00.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.627+0000 7f8b38d46700 1 --2- 172.21.15.162:0/2188098040 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8b34099d30 0x7f8b34147990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:00.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.627+0000 7f8b38d46700 1 --2- 172.21.15.162:0/2188098040 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b340a8df0 0x7f8b340a9e30 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:00.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.627+0000 7f8b38d46700 1 -- 172.21.15.162:0/2188098040 >> 172.21.15.162:0/2188098040 conn(0x7f8b340b5520 msgr2=0x7f8b34099370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:00.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.627+0000 7f8b38d46700 1 -- 172.21.15.162:0/2188098040 shutdown_connections 2020-11-28T21:55:00.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.627+0000 7f8b38d46700 1 -- 172.21.15.162:0/2188098040 wait complete. 2020-11-28T21:55:00.650 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-28T21:55:00.651 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:55:00.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.759+0000 7f2f97fff700 1 Processor -- start 2020-11-28T21:55:00.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.759+0000 7f2f97fff700 1 -- start start 2020-11-28T21:55:00.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.763+0000 7f2f97fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2f980603d0 0x7f2f980b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:00.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.763+0000 7f2f97fff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f2f980b4c70 0x7f2f98143020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:00.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.763+0000 7f2f97fff700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f980b4110 con 0x7f2f98142a00 2020-11-28T21:55:00.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.763+0000 7f2f97fff700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f98060f60 con 0x7f2f980b4c70 2020-11-28T21:55:00.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.763+0000 7f2f97fff700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f980b43f0 con 0x7f2f980603d0 2020-11-28T21:55:00.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.763+0000 7f2f96ffd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2f980603d0 0x7f2f980b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:00.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.763+0000 7f2f967fc700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f2f98142a00 0x7f2f9809cfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51194/0 (socket says 172.21.15.162:51194) 2020-11-28T21:55:00.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.763+0000 7f2f967fc700 1 -- 172.21.15.162:0/354663631 learned_addr learned my addr 172.21.15.162:0/354663631 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:00.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.763+0000 7f2f977fe700 1 --2- 172.21.15.162:0/354663631 >> v2:172.21.15.118:3300/0 conn(0x7f2f980b4c70 0x7f2f98143020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:00.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.763+0000 7f2f95ffb700 1 -- 172.21.15.162:0/354663631 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2660813203 0 0) 0x7f2f980b4110 con 0x7f2f98142a00 2020-11-28T21:55:00.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.763+0000 7f2f95ffb700 1 -- 172.21.15.162:0/354663631 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f7c003230 con 0x7f2f98142a00 2020-11-28T21:55:00.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.763+0000 7f2f96ffd700 1 -- 172.21.15.162:0/354663631 >> v2:172.21.15.118:3300/0 conn(0x7f2f980b4c70 msgr2=0x7f2f98143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:00.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.763+0000 7f2f96ffd700 1 --2- 172.21.15.162:0/354663631 >> v2:172.21.15.118:3300/0 conn(0x7f2f980b4c70 0x7f2f98143020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:00.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.763+0000 7f2f96ffd700 1 -- 172.21.15.162:0/354663631 >> v1:172.21.15.162:6789/0 conn(0x7f2f98142a00 legacy=0x7f2f9809cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:00.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.763+0000 7f2f96ffd700 1 -- 172.21.15.162:0/354663631 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f980b4690 con 0x7f2f980603d0 2020-11-28T21:55:00.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.763+0000 7f2f96ffd700 1 --2- 172.21.15.162:0/354663631 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f980603d0 0x7f2f980b3a70 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 rx=0x7f2f88002f30 tx=0x7f2f88006dc0).ready entity=mon.1 client_cookie=6ef3c1a64e8fe7af server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:00.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.763+0000 7f2f95ffb700 1 -- 172.21.15.162:0/354663631 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f880096f0 con 0x7f2f980603d0 2020-11-28T21:55:00.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.763+0000 7f2f95ffb700 1 -- 172.21.15.162:0/354663631 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f88009850 con 0x7f2f980603d0 2020-11-28T21:55:00.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.763+0000 7f2f95ffb700 1 -- 172.21.15.162:0/354663631 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f88009bb0 con 0x7f2f980603d0 2020-11-28T21:55:00.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.763+0000 7f2f97fff700 1 -- 172.21.15.162:0/354663631 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f980603d0 msgr2=0x7f2f980b3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:00.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.763+0000 7f2f97fff700 1 --2- 172.21.15.162:0/354663631 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f980603d0 0x7f2f980b3a70 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 rx=0x7f2f88002f30 tx=0x7f2f88006dc0).stop 2020-11-28T21:55:00.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.763+0000 7f2f97fff700 1 -- 172.21.15.162:0/354663631 shutdown_connections 2020-11-28T21:55:00.780 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.763+0000 7f2f97fff700 1 --2- 172.21.15.162:0/354663631 >> v2:172.21.15.118:3300/0 conn(0x7f2f980b4c70 0x7f2f98143020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:00.780 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.763+0000 7f2f97fff700 1 --2- 172.21.15.162:0/354663631 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f980603d0 0x7f2f980b3a70 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:00.780 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.763+0000 7f2f97fff700 1 -- 172.21.15.162:0/354663631 >> 172.21.15.162:0/354663631 conn(0x7f2f98066a40 msgr2=0x7f2f98066e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:00.781 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.763+0000 7f2f97fff700 1 -- 172.21.15.162:0/354663631 shutdown_connections 2020-11-28T21:55:00.781 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.763+0000 7f2f97fff700 1 -- 172.21.15.162:0/354663631 wait complete. 2020-11-28T21:55:00.781 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.767+0000 7f2f97fff700 1 Processor -- start 2020-11-28T21:55:00.782 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.767+0000 7f2f97fff700 1 -- start start 2020-11-28T21:55:00.782 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.767+0000 7f2f97fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f980603d0 0x7f2f98144d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:00.783 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.767+0000 7f2f97fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2f980b4c70 0x7f2f98143f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:00.783 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.767+0000 7f2f97fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2f98147270 0x7f2f98147f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:00.784 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.767+0000 7f2f97fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f98149ae0 con 0x7f2f98147270 2020-11-28T21:55:00.784 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.767+0000 7f2f97fff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f98149d10 con 0x7f2f980603d0 2020-11-28T21:55:00.784 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.767+0000 7f2f97fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f980afa50 con 0x7f2f980b4c70 2020-11-28T21:55:00.785 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.767+0000 7f2f96ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f980603d0 0x7f2f98144d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:00.785 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.767+0000 7f2f96ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f980603d0 0x7f2f98144d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44356/0 (socket says 172.21.15.162:44356) 2020-11-28T21:55:00.785 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.767+0000 7f2f96ffd700 1 -- 172.21.15.162:0/3423391443 learned_addr learned my addr 172.21.15.162:0/3423391443 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:00.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.767+0000 7f2f967fc700 1 --2- 172.21.15.162:0/3423391443 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2f980b4c70 0x7f2f98143f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:00.788 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.767+0000 7f2f977fe700 1 --2- 172.21.15.162:0/3423391443 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2f98147270 0x7f2f98147f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:00.788 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.767+0000 7f2f96ffd700 1 -- 172.21.15.162:0/3423391443 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2f980b4c70 msgr2=0x7f2f98143f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:00.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.767+0000 7f2f96ffd700 1 --2- 172.21.15.162:0/3423391443 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2f980b4c70 0x7f2f98143f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:00.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.767+0000 7f2f96ffd700 1 -- 172.21.15.162:0/3423391443 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2f98147270 msgr2=0x7f2f98147f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:00.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.767+0000 7f2f96ffd700 1 --2- 172.21.15.162:0/3423391443 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2f98147270 0x7f2f98147f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:00.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.767+0000 7f2f96ffd700 1 -- 172.21.15.162:0/3423391443 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f980afcf0 con 0x7f2f980603d0 2020-11-28T21:55:00.791 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.767+0000 7f2f967fc700 1 --2- 172.21.15.162:0/3423391443 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2f980b4c70 0x7f2f98143f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:55:00.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.767+0000 7f2f977fe700 1 --2- 172.21.15.162:0/3423391443 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2f98147270 0x7f2f98147f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:55:00.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.767+0000 7f2f96ffd700 1 --2- 172.21.15.162:0/3423391443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f980603d0 0x7f2f98144d30 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 rx=0x7f2f88002f30 tx=0x7f2f88015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:00.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.771+0000 7f2f87fff700 1 -- 172.21.15.162:0/3423391443 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f88011c60 con 0x7f2f980603d0 2020-11-28T21:55:00.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.771+0000 7f2f87fff700 1 -- 172.21.15.162:0/3423391443 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f88011dc0 con 0x7f2f980603d0 2020-11-28T21:55:00.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.771+0000 7f2f97fff700 1 -- 172.21.15.162:0/3423391443 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f980aff90 con 0x7f2f980603d0 2020-11-28T21:55:00.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.771+0000 7f2f87fff700 1 -- 172.21.15.162:0/3423391443 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f8801c5a0 con 0x7f2f980603d0 2020-11-28T21:55:00.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.771+0000 7f2f97fff700 1 -- 172.21.15.162:0/3423391443 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f980aedf0 con 0x7f2f980603d0 2020-11-28T21:55:00.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.771+0000 7f2f97fff700 1 -- 172.21.15.162:0/3423391443 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f98144a90 con 0x7f2f980603d0 2020-11-28T21:55:00.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.771+0000 7f2f87fff700 1 -- 172.21.15.162:0/3423391443 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2f8800f070 con 0x7f2f980603d0 2020-11-28T21:55:00.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.771+0000 7f2f87fff700 1 --2- 172.21.15.162:0/3423391443 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2f740554e0 0x7f2f740579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:00.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.771+0000 7f2f87fff700 1 -- 172.21.15.162:0/3423391443 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2f8806ee60 con 0x7f2f980603d0 2020-11-28T21:55:00.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.771+0000 7f2f967fc700 1 --2- 172.21.15.162:0/3423391443 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2f740554e0 0x7f2f740579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:00.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.775+0000 7f2f967fc700 1 --2- 172.21.15.162:0/3423391443 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2f740554e0 0x7f2f740579a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7f2f8c00b000 tx=0x7f2f8c009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:00.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.783+0000 7f2f87fff700 1 -- 172.21.15.162:0/3423391443 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f2f98144a90 con 0x7f2f980603d0 2020-11-28T21:55:00.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.971+0000 7f2f97fff700 1 -- 172.21.15.162:0/3423391443 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f98144a90 con 0x7f2f980603d0 2020-11-28T21:55:00.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.971+0000 7f2f87fff700 1 -- 172.21.15.162:0/3423391443 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+2504 (secure 0 0 0) 0x7f2f98144a90 con 0x7f2f980603d0 2020-11-28T21:55:00.976 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:00.976 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":57,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7257,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/1527184432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1527184432},{"type":"v1","addr":"172.21.15.118:6833","nonce":1527184432}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":54},{"gid":7477,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6835/2749838911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2749838911},{"type":"v1","addr":"172.21.15.162:6835","nonce":2749838911}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":55}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:54:51.371126+0000","modified":"2020-11-28T21:54:53.015631+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7489},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7489":{"gid":7489,"name":"c","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/1061393314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1061393314},{"type":"v1","addr":"172.21.15.162:6837","nonce":1061393314}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2020-11-28T21:55:00.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.971+0000 7f2f97fff700 1 -- 172.21.15.162:0/3423391443 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2f740554e0 msgr2=0x7f2f740579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:00.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.971+0000 7f2f97fff700 1 --2- 172.21.15.162:0/3423391443 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2f740554e0 0x7f2f740579a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7f2f8c00b000 tx=0x7f2f8c009040).stop 2020-11-28T21:55:00.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.971+0000 7f2f97fff700 1 -- 172.21.15.162:0/3423391443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f980603d0 msgr2=0x7f2f98144d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:00.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.971+0000 7f2f97fff700 1 --2- 172.21.15.162:0/3423391443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f980603d0 0x7f2f98144d30 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 rx=0x7f2f88002f30 tx=0x7f2f88015000).stop 2020-11-28T21:55:00.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.971+0000 7f2f97fff700 1 -- 172.21.15.162:0/3423391443 shutdown_connections 2020-11-28T21:55:00.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.971+0000 7f2f97fff700 1 --2- 172.21.15.162:0/3423391443 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2f740554e0 0x7f2f740579a0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:00.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.971+0000 7f2f97fff700 1 --2- 172.21.15.162:0/3423391443 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2f98147270 0x7f2f98147f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:00.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.971+0000 7f2f97fff700 1 --2- 172.21.15.162:0/3423391443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f980603d0 0x7f2f98144d30 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:00.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.971+0000 7f2f97fff700 1 --2- 172.21.15.162:0/3423391443 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2f980b4c70 0x7f2f98143f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:00.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.971+0000 7f2f97fff700 1 -- 172.21.15.162:0/3423391443 >> 172.21.15.162:0/3423391443 conn(0x7f2f98066a40 msgr2=0x7f2f980b5fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:00.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.971+0000 7f2f97fff700 1 -- 172.21.15.162:0/3423391443 shutdown_connections 2020-11-28T21:55:00.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:00.971+0000 7f2f97fff700 1 -- 172.21.15.162:0/3423391443 wait complete. 2020-11-28T21:55:00.987 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 57 2020-11-28T21:55:01.292 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 57, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-28T21:54:51.371126+0000', 'modified': '2020-11-28T21:54:53.015631+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7489}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7489': {'gid': 7489, 'name': 'c', 'rank': 0, 'incarnation': 55, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6837/1061393314', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6836', 'nonce': 1061393314}, {'type': 'v1', 'addr': '172.21.15.162:6837', 'nonce': 1061393314}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-28T21:55:01.293 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-28T21:55:01.294 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-28T21:55:01.425 INFO:teuthology.orchestra.run.smithi162.stdout:{"cluster_fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","whoami":0,"id":7489,"want_state":"up:active","state":"up:active","rank_uptime":10.039939186,"mdsmap_epoch":57,"osdmap_epoch":119,"osdmap_epoch_barrier":119,"uptime":14.358642315000001} 2020-11-28T21:55:01.425 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", "id": 7489, "mdsmap_epoch": 57, "osdmap_epoch": 119, "osdmap_epoch_barrier": 119, "rank_uptime": 10.039939186, "state": "up:active", "uptime": 14.358642315, "want_state": "up:active", "whoami": 0 } 2020-11-28T21:55:01.426 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-11-28T21:55:01.426 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2020-11-28T21:55:01.427 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi096.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2020-11-28T21:55:01.427 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:01.427 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:55:01.435 INFO:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2020-11-28T21:55:01.492 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:55:01.493 INFO:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:55:01.547 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:55:01.549 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:55:01.549 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:01.550 INFO:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2020-11-28T21:55:01.597 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-11-28T21:55:01.598 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:55:01.643 INFO:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2020-11-28T21:55:01.660 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:55:01.661 INFO:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:55:01.714 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:55:01.715 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:55:01.716 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:01.716 INFO:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2020-11-28T21:55:01.725 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:2020-11-28T21:55:01.717+0000 7fc81f1200c0 -1 init, newargv = 0x55fc2621d6d0 newargc=9ceph-fuse[11908 2020-11-28T21:55:01.725 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:]: starting ceph client 2020-11-28T21:55:01.736 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[11908]: starting fuse 2020-11-28T21:55:02.830 INFO:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2020-11-28T21:55:02.846 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:55:02.847 INFO:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:55:02.903 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:55:02.904 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:55:02.905 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:02.906 INFO:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2020-11-28T21:55:02.951 INFO:teuthology.orchestra.run.smithi096.stdout:49 2020-11-28T21:55:02.951 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2020-11-28T21:55:02.952 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:02.952 INFO:teuthology.orchestra.run.smithi096:> sudo python3 -c ' 2020-11-28T21:55:02.953 INFO:teuthology.orchestra.run.smithi096:> import glob 2020-11-28T21:55:02.953 INFO:teuthology.orchestra.run.smithi096:> import re 2020-11-28T21:55:02.953 INFO:teuthology.orchestra.run.smithi096:> import os 2020-11-28T21:55:02.954 INFO:teuthology.orchestra.run.smithi096:> import subprocess 2020-11-28T21:55:02.954 INFO:teuthology.orchestra.run.smithi096:> 2020-11-28T21:55:02.955 INFO:teuthology.orchestra.run.smithi096:> def find_socket(client_name): 2020-11-28T21:55:02.955 INFO:teuthology.orchestra.run.smithi096:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2020-11-28T21:55:02.955 INFO:teuthology.orchestra.run.smithi096:> files = glob.glob(asok_path) 2020-11-28T21:55:02.956 INFO:teuthology.orchestra.run.smithi096:> 2020-11-28T21:55:02.956 INFO:teuthology.orchestra.run.smithi096:> # Given a non-glob path, it better be there 2020-11-28T21:55:02.957 INFO:teuthology.orchestra.run.smithi096:> if "*" not in asok_path: 2020-11-28T21:55:02.957 INFO:teuthology.orchestra.run.smithi096:> assert(len(files) == 1) 2020-11-28T21:55:02.957 INFO:teuthology.orchestra.run.smithi096:> return files[0] 2020-11-28T21:55:02.958 INFO:teuthology.orchestra.run.smithi096:> 2020-11-28T21:55:02.958 INFO:teuthology.orchestra.run.smithi096:> for f in files: 2020-11-28T21:55:02.959 INFO:teuthology.orchestra.run.smithi096:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-11-28T21:55:02.960 INFO:teuthology.orchestra.run.smithi096:> if os.path.exists("/proc/{0}".format(pid)): 2020-11-28T21:55:02.960 INFO:teuthology.orchestra.run.smithi096:> return f 2020-11-28T21:55:02.961 INFO:teuthology.orchestra.run.smithi096:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-11-28T21:55:02.961 INFO:teuthology.orchestra.run.smithi096:> 2020-11-28T21:55:02.961 INFO:teuthology.orchestra.run.smithi096:> print(find_socket("client.1")) 2020-11-28T21:55:02.962 INFO:teuthology.orchestra.run.smithi096:> ' 2020-11-28T21:55:03.043 INFO:teuthology.orchestra.run.smithi096.stdout:/var/run/ceph/ceph-client.1.11908.asok 2020-11-28T21:55:03.050 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.11908.asok 2020-11-28T21:55:03.050 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:03.052 INFO:teuthology.orchestra.run.smithi096:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.11908.asok status 2020-11-28T21:55:03.156 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-11-28T21:55:03.157 INFO:teuthology.orchestra.run.smithi096.stdout: "metadata": { 2020-11-28T21:55:03.157 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_sha1": "a83a3572a957356977342f461361c1cd3ec06dbd", 2020-11-28T21:55:03.158 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_version": "ceph version 15.2.6-595-ga83a3572 (a83a3572a957356977342f461361c1cd3ec06dbd) octopus (stable)", 2020-11-28T21:55:03.158 INFO:teuthology.orchestra.run.smithi096.stdout: "entity_id": "1", 2020-11-28T21:55:03.158 INFO:teuthology.orchestra.run.smithi096.stdout: "hostname": "smithi096", 2020-11-28T21:55:03.159 INFO:teuthology.orchestra.run.smithi096.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2020-11-28T21:55:03.160 INFO:teuthology.orchestra.run.smithi096.stdout: "pid": "11908", 2020-11-28T21:55:03.160 INFO:teuthology.orchestra.run.smithi096.stdout: "root": "/" 2020-11-28T21:55:03.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-11-28T21:55:03.161 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_count": 0, 2020-11-28T21:55:03.161 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_pinned_count": 0, 2020-11-28T21:55:03.162 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 6533, 2020-11-28T21:55:03.162 INFO:teuthology.orchestra.run.smithi096.stdout: "inst": { 2020-11-28T21:55:03.163 INFO:teuthology.orchestra.run.smithi096.stdout: "name": { 2020-11-28T21:55:03.163 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "client", 2020-11-28T21:55:03.164 INFO:teuthology.orchestra.run.smithi096.stdout: "num": 6533 2020-11-28T21:55:03.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-11-28T21:55:03.165 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2020-11-28T21:55:03.165 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2020-11-28T21:55:03.165 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2020-11-28T21:55:03.166 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1742087161 2020-11-28T21:55:03.167 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-11-28T21:55:03.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-11-28T21:55:03.167 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2020-11-28T21:55:03.168 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2020-11-28T21:55:03.168 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2020-11-28T21:55:03.169 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1742087161 2020-11-28T21:55:03.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-11-28T21:55:03.169 INFO:teuthology.orchestra.run.smithi096.stdout: "inst_str": "client.6533 172.21.15.96:0/1742087161", 2020-11-28T21:55:03.170 INFO:teuthology.orchestra.run.smithi096.stdout: "addr_str": "172.21.15.96:0/1742087161", 2020-11-28T21:55:03.170 INFO:teuthology.orchestra.run.smithi096.stdout: "inode_count": 1, 2020-11-28T21:55:03.171 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_epoch": 57, 2020-11-28T21:55:03.171 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch": 119, 2020-11-28T21:55:03.171 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch_barrier": 0, 2020-11-28T21:55:03.172 INFO:teuthology.orchestra.run.smithi096.stdout: "blacklisted": false 2020-11-28T21:55:03.172 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-11-28T21:55:03.174 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:03.175 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:55:03.175 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:55:03.183 INFO:teuthology.orchestra.run.smithi096.stdout:fuseblk 2020-11-28T21:55:03.184 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-11-28T21:55:03.184 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:03.185 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:55:03.241 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:03.242 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:55:03.243 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:55:03.251 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2020-11-28T21:55:03.252 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-28T21:55:03.253 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:03.253 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:55:03.254 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:55:03.303 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2020-11-28T21:55:03.304 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-28T21:55:03.304 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2020-11-28T21:55:03.305 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-28T21:55:03.305 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:55:03.376 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:ceph-fuse[12493]: fuse finished with error 0 and tester_r 0 2020-11-28T21:55:03.378 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:03.379 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:55:03.379 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:55:03.426 INFO:teuthology.orchestra.run.smithi192.stdout:ext2/ext3 2020-11-28T21:55:03.427 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-11-28T21:55:03.428 INFO:teuthology.orchestra.run:waiting for 900 2020-11-28T21:55:03.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-28T21:55:03.447 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:55:03.495 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:03.495 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:55:03.495 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:55:03.503 INFO:teuthology.orchestra.run.smithi096.stdout:fuseblk 2020-11-28T21:55:03.504 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-11-28T21:55:03.505 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:03.505 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:55:03.506 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:55:03.552 INFO:teuthology.orchestra.run.smithi096.stdout:fuseblk 2020-11-28T21:55:03.554 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-11-28T21:55:03.555 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi096.front.sepia.ceph.com... 2020-11-28T21:55:03.555 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-28T21:55:03.556 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:55:03.629 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[11908]: fuse finished with error 0 and tester_r 0 2020-11-28T21:55:03.631 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:03.632 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:55:03.632 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:55:03.679 INFO:teuthology.orchestra.run.smithi096.stdout:ext2/ext3 2020-11-28T21:55:03.680 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-11-28T21:55:03.681 INFO:teuthology.orchestra.run:waiting for 900 2020-11-28T21:55:03.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-28T21:55:03.716 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:55:03.769 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-11-28T21:55:03.770 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T21:55:03.771 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2020-11-28T21:55:03.771 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T21:55:03.772 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-11-28T21:55:03.772 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T21:55:03.786 INFO:tasks.ceph.mds.b:Stopped 2020-11-28T21:55:03.787 DEBUG:teuthology.parallel:result is None 2020-11-28T21:55:03.790 INFO:tasks.ceph.mds.a:Stopped 2020-11-28T21:55:03.790 INFO:tasks.ceph.mds.c:Stopped 2020-11-28T21:55:03.790 DEBUG:teuthology.parallel:result is None 2020-11-28T21:55:03.791 DEBUG:teuthology.parallel:result is None 2020-11-28T21:55:03.791 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early mds fail a 2020-11-28T21:55:03.792 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early mds fail c 2020-11-28T21:55:03.793 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early mds fail b 2020-11-28T21:55:03.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.935+0000 7ff9a359e700 1 Processor -- start 2020-11-28T21:55:03.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.935+0000 7ff9a359e700 1 -- start start 2020-11-28T21:55:03.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.939+0000 7ff9a359e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff9a40b1870 0x7ff9a4142f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:03.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.939+0000 7ff9a359e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff9a4143440 0x7ff9a40b0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:03.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.939+0000 7ff9a359e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff9a40b4720 0x7ff9a40b0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:03.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.939+0000 7ff9a359e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9a40615a0 con 0x7ff9a40b4720 2020-11-28T21:55:03.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.939+0000 7ff9a359e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9a40afff0 con 0x7ff9a4143440 2020-11-28T21:55:03.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.939+0000 7ff9a359e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a40b1c50 con 0x7ff9a40b1870 2020-11-28T21:55:03.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.939+0000 7ff9a1d9b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff9a4143440 0x7ff9a40b0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:03.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.939+0000 7ff9a1d9b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff9a4143440 0x7ff9a40b0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44366/0 (socket says 172.21.15.162:44366) 2020-11-28T21:55:03.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.939+0000 7ff9a1d9b700 1 -- 172.21.15.162:0/1158728123 learned_addr learned my addr 172.21.15.162:0/1158728123 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:03.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.939+0000 7ff9a259c700 1 --2- 172.21.15.162:0/1158728123 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff9a40b1870 0x7ff9a4142f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:03.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.939+0000 7ff9a2d9d700 1 --2- 172.21.15.162:0/1158728123 >> v2:172.21.15.118:3300/0 conn(0x7ff9a40b4720 0x7ff9a40b0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:03.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.939+0000 7ff9a1d9b700 1 -- 172.21.15.162:0/1158728123 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff9a40b1870 msgr2=0x7ff9a4142f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:03.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.939+0000 7ff9a1d9b700 1 --2- 172.21.15.162:0/1158728123 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff9a40b1870 0x7ff9a4142f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:03.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.939+0000 7ff9a1d9b700 1 -- 172.21.15.162:0/1158728123 >> v2:172.21.15.118:3300/0 conn(0x7ff9a40b4720 msgr2=0x7ff9a40b0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:03.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.939+0000 7ff9a1d9b700 1 --2- 172.21.15.162:0/1158728123 >> v2:172.21.15.118:3300/0 conn(0x7ff9a40b4720 0x7ff9a40b0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:03.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.939+0000 7ff9a1d9b700 1 -- 172.21.15.162:0/1158728123 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a4060a60 con 0x7ff9a4143440 2020-11-28T21:55:03.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.939+0000 7ff9a2d9d700 1 --2- 172.21.15.162:0/1158728123 >> v2:172.21.15.118:3300/0 conn(0x7ff9a40b4720 0x7ff9a40b0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:55:03.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.939+0000 7ff9a1d9b700 1 --2- 172.21.15.162:0/1158728123 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9a4143440 0x7ff9a40b0830 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 rx=0x7ff994002f30 tx=0x7ff994006e80).ready entity=mon.1 client_cookie=af9ab5686d8fab06 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:03.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.939+0000 7ff9a159a700 1 -- 172.21.15.162:0/1158728123 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9940097f0 con 0x7ff9a4143440 2020-11-28T21:55:03.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.939+0000 7ff9a159a700 1 -- 172.21.15.162:0/1158728123 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff994009950 con 0x7ff9a4143440 2020-11-28T21:55:03.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.939+0000 7ff9a159a700 1 -- 172.21.15.162:0/1158728123 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff994009cb0 con 0x7ff9a4143440 2020-11-28T21:55:03.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.939+0000 7ff9a359e700 1 -- 172.21.15.162:0/1158728123 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9a4143440 msgr2=0x7ff9a40b0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:03.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.939+0000 7ff9a359e700 1 --2- 172.21.15.162:0/1158728123 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9a4143440 0x7ff9a40b0830 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 rx=0x7ff994002f30 tx=0x7ff994006e80).stop 2020-11-28T21:55:03.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.939+0000 7ff9a359e700 1 -- 172.21.15.162:0/1158728123 shutdown_connections 2020-11-28T21:55:03.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.939+0000 7ff9a359e700 1 --2- 172.21.15.162:0/1158728123 >> v2:172.21.15.118:3300/0 conn(0x7ff9a40b4720 0x7ff9a40b0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:03.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.939+0000 7ff9a359e700 1 --2- 172.21.15.162:0/1158728123 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff9a40b1870 0x7ff9a4142f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:03.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.939+0000 7ff9a359e700 1 --2- 172.21.15.162:0/1158728123 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9a4143440 0x7ff9a40b0830 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:03.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.939+0000 7ff9a359e700 1 -- 172.21.15.162:0/1158728123 >> 172.21.15.162:0/1158728123 conn(0x7ff9a40b54f0 msgr2=0x7ff9a40b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:03.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.939+0000 7ff9a359e700 1 -- 172.21.15.162:0/1158728123 shutdown_connections 2020-11-28T21:55:03.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.939+0000 7ff9a359e700 1 -- 172.21.15.162:0/1158728123 wait complete. 2020-11-28T21:55:03.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.939+0000 7ff9a359e700 1 Processor -- start 2020-11-28T21:55:03.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.939+0000 7ff9a359e700 1 -- start start 2020-11-28T21:55:03.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.943+0000 7ff9a359e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff9a40b1870 0x7ff9a4145740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:03.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.943+0000 7ff9a359e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9a40b4720 0x7ff9a4098590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:03.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.943+0000 7ff9a359e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9a4143440 0x7ff9a4098a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:03.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.943+0000 7ff9a359e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a409a060 con 0x7ff9a40b4720 2020-11-28T21:55:03.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.943+0000 7ff9a359e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a409a250 con 0x7ff9a4143440 2020-11-28T21:55:03.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.943+0000 7ff9a359e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a409a4f0 con 0x7ff9a40b1870 2020-11-28T21:55:03.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.943+0000 7ff9a2d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9a4143440 0x7ff9a4098a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:03.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.943+0000 7ff9a2d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9a4143440 0x7ff9a4098a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44372/0 (socket says 172.21.15.162:44372) 2020-11-28T21:55:03.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.943+0000 7ff9a2d9d700 1 -- 172.21.15.162:0/1609276242 learned_addr learned my addr 172.21.15.162:0/1609276242 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:03.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.943+0000 7ff9a1d9b700 1 --2- 172.21.15.162:0/1609276242 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9a40b4720 0x7ff9a4098590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:03.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.943+0000 7ff9a259c700 1 --2- 172.21.15.162:0/1609276242 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff9a40b1870 0x7ff9a4145740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:03.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.943+0000 7ff9a2d9d700 1 -- 172.21.15.162:0/1609276242 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff9a40b1870 msgr2=0x7ff9a4145740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:03.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.943+0000 7ff9a2d9d700 1 --2- 172.21.15.162:0/1609276242 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff9a40b1870 0x7ff9a4145740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:03.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.943+0000 7ff9a2d9d700 1 -- 172.21.15.162:0/1609276242 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9a40b4720 msgr2=0x7ff9a4098590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:03.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.943+0000 7ff9a2d9d700 1 --2- 172.21.15.162:0/1609276242 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9a40b4720 0x7ff9a4098590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:03.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.943+0000 7ff9a2d9d700 1 -- 172.21.15.162:0/1609276242 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a409a790 con 0x7ff9a4143440 2020-11-28T21:55:03.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.943+0000 7ff9a259c700 1 --2- 172.21.15.162:0/1609276242 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff9a40b1870 0x7ff9a4145740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:55:03.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.943+0000 7ff9a2d9d700 1 --2- 172.21.15.162:0/1609276242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9a4143440 0x7ff9a4098a80 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 rx=0x7ff99c006c90 tx=0x7ff99c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:03.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.943+0000 7ff99b7fe700 1 -- 172.21.15.162:0/1609276242 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff99c00c940 con 0x7ff9a4143440 2020-11-28T21:55:03.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.943+0000 7ff99b7fe700 1 -- 172.21.15.162:0/1609276242 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99c00caa0 con 0x7ff9a4143440 2020-11-28T21:55:03.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.943+0000 7ff9a359e700 1 -- 172.21.15.162:0/1609276242 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9a409ac30 con 0x7ff9a4143440 2020-11-28T21:55:03.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.943+0000 7ff99b7fe700 1 -- 172.21.15.162:0/1609276242 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff99c00ce00 con 0x7ff9a4143440 2020-11-28T21:55:03.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.943+0000 7ff9a359e700 1 -- 172.21.15.162:0/1609276242 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9a409b3e0 con 0x7ff9a4143440 2020-11-28T21:55:03.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.943+0000 7ff9a359e700 1 -- 172.21.15.162:0/1609276242 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9a40af7e0 con 0x7ff9a4143440 2020-11-28T21:55:03.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.943+0000 7ff99b7fe700 1 -- 172.21.15.162:0/1609276242 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff99c01d460 con 0x7ff9a4143440 2020-11-28T21:55:03.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.943+0000 7ff99b7fe700 1 --2- 172.21.15.162:0/1609276242 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff988055440 0x7ff988057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:03.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.943+0000 7ff99b7fe700 1 -- 172.21.15.162:0/1609276242 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff99c06e4c0 con 0x7ff9a4143440 2020-11-28T21:55:03.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.943+0000 7ff9a259c700 1 --2- 172.21.15.162:0/1609276242 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff988055440 0x7ff988057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:03.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.943+0000 7ff9a259c700 1 --2- 172.21.15.162:0/1609276242 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff988055440 0x7ff988057900 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7ff98c008000 tx=0x7ff98c008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:03.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.951+0000 7ff99b7fe700 1 -- 172.21.15.162:0/1609276242 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff99c044760 con 0x7ff9a4143440 2020-11-28T21:55:03.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.951+0000 7fb786d49700 1 Processor -- start 2020-11-28T21:55:03.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.951+0000 7fb786d49700 1 -- start start 2020-11-28T21:55:03.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.951+0000 7fb786d49700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb78005ce00 0x7fb7800b1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:03.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.951+0000 7fb786d49700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb780146b00 con 0x7fb780146330 2020-11-28T21:55:03.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.951+0000 7fb786d49700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb780146da0 con 0x7fb78006aa70 2020-11-28T21:55:03.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.951+0000 7fb786d49700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7800b6610 con 0x7fb78005ce00 2020-11-28T21:55:03.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.951+0000 7fb77ffff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fb78006aa70 0x7fb78005c6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51222/0 (socket says 172.21.15.162:51222) 2020-11-28T21:55:03.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.951+0000 7fb77ffff700 1 -- 172.21.15.162:0/1968642979 learned_addr learned my addr 172.21.15.162:0/1968642979 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:03.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.951+0000 7fb7852e6700 1 --2- 172.21.15.162:0/1968642979 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb78005ce00 0x7fb7800b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:03.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.951+0000 7fb77f7fe700 1 -- 172.21.15.162:0/1968642979 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2103093958 0 0) 0x7fb780146da0 con 0x7fb78006aa70 2020-11-28T21:55:03.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.951+0000 7fb77f7fe700 1 -- 172.21.15.162:0/1968642979 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb760003230 con 0x7fb78006aa70 2020-11-28T21:55:03.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.951+0000 7fb77f7fe700 1 -- 172.21.15.162:0/1968642979 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3186258400 0 0) 0x7fb760003230 con 0x7fb78006aa70 2020-11-28T21:55:03.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.951+0000 7fb77f7fe700 1 -- 172.21.15.162:0/1968642979 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb78005ce00 msgr2=0x7fb7800b1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:03.992 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.951+0000 7fb77f7fe700 1 --2- 172.21.15.162:0/1968642979 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb78005ce00 0x7fb7800b1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:03.992 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.951+0000 7fb77f7fe700 1 -- 172.21.15.162:0/1968642979 >> v1:172.21.15.118:6789/0 conn(0x7fb780146330 legacy=0x7fb78005b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:03.992 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.951+0000 7fb77f7fe700 1 -- 172.21.15.162:0/1968642979 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7800b0f80 con 0x7fb78006aa70 2020-11-28T21:55:03.992 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.951+0000 7fb77f7fe700 1 -- 172.21.15.162:0/1968642979 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7fb7680016d0 con 0x7fb78006aa70 2020-11-28T21:55:03.992 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.951+0000 7fb77f7fe700 1 -- 172.21.15.162:0/1968642979 >> v1:172.21.15.162:6789/0 conn(0x7fb78006aa70 legacy=0x7fb78005c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:03.993 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.951+0000 7fb77f7fe700 1 --2- 172.21.15.162:0/1968642979 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb760003720 0x7fb760003b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:03.993 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.951+0000 7fb77f7fe700 1 --2- 172.21.15.162:0/1968642979 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb78005ce00 0x7fb7600044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:03.993 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.951+0000 7fb77f7fe700 1 --2- 172.21.15.162:0/1968642979 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7600049c0 0x7fb760004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:03.993 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.951+0000 7fb77f7fe700 1 -- 172.21.15.162:0/1968642979 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7800b6610 con 0x7fb760003720 2020-11-28T21:55:03.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.951+0000 7fb77f7fe700 1 -- 172.21.15.162:0/1968642979 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb760004130 con 0x7fb7600049c0 2020-11-28T21:55:03.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.951+0000 7fb77f7fe700 1 -- 172.21.15.162:0/1968642979 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7600042f0 con 0x7fb78005ce00 2020-11-28T21:55:03.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.951+0000 7fb7852e6700 1 --2- 172.21.15.162:0/1968642979 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7600049c0 0x7fb760004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:03.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.951+0000 7fb77ffff700 1 --2- 172.21.15.162:0/1968642979 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb78005ce00 0x7fb7600044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:03.996 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.951+0000 7fb784ae5700 1 --2- 172.21.15.162:0/1968642979 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb760003720 0x7fb760003b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:03.996 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.951+0000 7fb7852e6700 1 -- 172.21.15.162:0/1968642979 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb78005ce00 msgr2=0x7fb7600044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:03.996 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.951+0000 7fb7852e6700 1 --2- 172.21.15.162:0/1968642979 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb78005ce00 0x7fb7600044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:03.996 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.951+0000 7fb7852e6700 1 -- 172.21.15.162:0/1968642979 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb760003720 msgr2=0x7fb760003b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:03.997 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.951+0000 7fb7852e6700 1 --2- 172.21.15.162:0/1968642979 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb760003720 0x7fb760003b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:03.997 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.951+0000 7fb7852e6700 1 -- 172.21.15.162:0/1968642979 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb760003230 con 0x7fb7600049c0 2020-11-28T21:55:03.997 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.951+0000 7fb77ffff700 1 --2- 172.21.15.162:0/1968642979 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb78005ce00 0x7fb7600044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:55:03.997 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.951+0000 7fb7852e6700 1 --2- 172.21.15.162:0/1968642979 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7600049c0 0x7fb760004e40 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 rx=0x7fb77000c000 tx=0x7fb770008040).ready entity=mon.1 client_cookie=f7fff26ba90d27be server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:03.997 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.951+0000 7fb77f7fe700 1 -- 172.21.15.162:0/1968642979 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb77000e9d0 con 0x7fb7600049c0 2020-11-28T21:55:03.998 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.951+0000 7fb77f7fe700 1 -- 172.21.15.162:0/1968642979 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77000eb30 con 0x7fb7600049c0 2020-11-28T21:55:03.998 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.951+0000 7fb77f7fe700 1 -- 172.21.15.162:0/1968642979 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb77000ee90 con 0x7fb7600049c0 2020-11-28T21:55:03.998 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.951+0000 7fb786d49700 1 -- 172.21.15.162:0/1968642979 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7600049c0 msgr2=0x7fb760004e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:03.998 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.951+0000 7fb786d49700 1 --2- 172.21.15.162:0/1968642979 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7600049c0 0x7fb760004e40 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 rx=0x7fb77000c000 tx=0x7fb770008040).stop 2020-11-28T21:55:03.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.951+0000 7fb7852e6700 1 -- 172.21.15.162:0/1968642979 reap_dead start 2020-11-28T21:55:03.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.955+0000 7fb786d49700 1 -- 172.21.15.162:0/1968642979 shutdown_connections 2020-11-28T21:55:03.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.955+0000 7fb786d49700 1 -- 172.21.15.162:0/1968642979 >> 172.21.15.162:0/1968642979 conn(0x7fb7800b56f0 msgr2=0x7fb7800b5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:03.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.955+0000 7fb786d49700 1 -- 172.21.15.162:0/1968642979 shutdown_connections 2020-11-28T21:55:04.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.955+0000 7fb786d49700 1 -- 172.21.15.162:0/1968642979 wait complete. 2020-11-28T21:55:04.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.955+0000 7fb786d49700 1 Processor -- start 2020-11-28T21:55:04.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.955+0000 7fb786d49700 1 -- start start 2020-11-28T21:55:04.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.955+0000 7fb786d49700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb780149970 0x7fb7800678a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:04.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.955+0000 7fb786d49700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb780141e40 0x7fb780063d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:04.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.955+0000 7fb786d49700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb7801435b0 0x7fb78013f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:04.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.955+0000 7fb786d49700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb780149d50 con 0x7fb7801435b0 2020-11-28T21:55:04.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.955+0000 7fb786d49700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7800a8a90 con 0x7fb780149970 2020-11-28T21:55:04.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.955+0000 7fb786d49700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7800a8d30 con 0x7fb780141e40 2020-11-28T21:55:04.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.955+0000 7fb784ae5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb780149970 0x7fb7800678a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:04.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.955+0000 7fb784ae5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb780149970 0x7fb7800678a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44388/0 (socket says 172.21.15.162:44388) 2020-11-28T21:55:04.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.955+0000 7fb784ae5700 1 -- 172.21.15.162:0/4033472110 learned_addr learned my addr 172.21.15.162:0/4033472110 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:04.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.955+0000 7fb77ffff700 1 --2- 172.21.15.162:0/4033472110 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb780141e40 0x7fb780063d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:04.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.955+0000 7fb7852e6700 1 --2- 172.21.15.162:0/4033472110 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb7801435b0 0x7fb78013f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:04.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.955+0000 7fb784ae5700 1 -- 172.21.15.162:0/4033472110 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb780141e40 msgr2=0x7fb780063d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:04.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.955+0000 7fb784ae5700 1 --2- 172.21.15.162:0/4033472110 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb780141e40 0x7fb780063d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:04.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.955+0000 7fb784ae5700 1 -- 172.21.15.162:0/4033472110 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb7801435b0 msgr2=0x7fb78013f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:04.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.955+0000 7fb784ae5700 1 --2- 172.21.15.162:0/4033472110 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb7801435b0 0x7fb78013f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:04.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.955+0000 7fb784ae5700 1 -- 172.21.15.162:0/4033472110 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7801557f0 con 0x7fb780149970 2020-11-28T21:55:04.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.955+0000 7fb784ae5700 1 --2- 172.21.15.162:0/4033472110 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb780149970 0x7fb7800678a0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 rx=0x7fb778008000 tx=0x7fb778009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:04.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.955+0000 7fb77dffb700 1 -- 172.21.15.162:0/4033472110 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb77800b8b0 con 0x7fb780149970 2020-11-28T21:55:04.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.955+0000 7fb77dffb700 1 -- 172.21.15.162:0/4033472110 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77800ba10 con 0x7fb780149970 2020-11-28T21:55:04.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.955+0000 7fb786d49700 1 -- 172.21.15.162:0/4033472110 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7800ac4d0 con 0x7fb780149970 2020-11-28T21:55:04.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.955+0000 7fb77dffb700 1 -- 172.21.15.162:0/4033472110 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb77800bd70 con 0x7fb780149970 2020-11-28T21:55:04.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.955+0000 7fb786d49700 1 -- 172.21.15.162:0/4033472110 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7800641f0 con 0x7fb780149970 2020-11-28T21:55:04.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.959+0000 7fb786d49700 1 -- 172.21.15.162:0/4033472110 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb780057050 con 0x7fb780149970 2020-11-28T21:55:04.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.959+0000 7fb77dffb700 1 -- 172.21.15.162:0/4033472110 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb77801c460 con 0x7fb780149970 2020-11-28T21:55:04.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.959+0000 7fb77dffb700 1 --2- 172.21.15.162:0/4033472110 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb764055440 0x7fb764057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:04.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.959+0000 7fb77dffb700 1 -- 172.21.15.162:0/4033472110 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb77806e4d0 con 0x7fb780149970 2020-11-28T21:55:04.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.959+0000 7fb77ffff700 1 --2- 172.21.15.162:0/4033472110 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb764055440 0x7fb764057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:04.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.959+0000 7fb77ffff700 1 --2- 172.21.15.162:0/4033472110 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb764055440 0x7fb764057900 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7fb76800c000 tx=0x7fb768009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:04.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.963+0000 7fb77dffb700 1 -- 172.21.15.162:0/4033472110 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb778044770 con 0x7fb780149970 2020-11-28T21:55:04.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.967+0000 7fb6fa304700 1 Processor -- start 2020-11-28T21:55:04.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.967+0000 7fb6fa304700 1 -- start start 2020-11-28T21:55:04.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.967+0000 7fb6fa304700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb6ec046c40 0x7fb6ec040b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:04.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.967+0000 7fb6fa304700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb6ec041f90 0x7fb6ec0d2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:04.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.967+0000 7fb6fa304700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fb6ec03f840 0x7fb6ec0ba8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:04.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.967+0000 7fb6fa304700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6ec0d1ae0 con 0x7fb6ec03f840 2020-11-28T21:55:04.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.967+0000 7fb6fa304700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6ec047020 con 0x7fb6ec046c40 2020-11-28T21:55:04.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.967+0000 7fb6fa304700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ec006030 con 0x7fb6ec041f90 2020-11-28T21:55:04.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.967+0000 7fb6f9302700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb6ec046c40 0x7fb6ec040b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:04.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.967+0000 7fb6f9302700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb6ec046c40 0x7fb6ec040b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44396/0 (socket says 172.21.15.162:44396) 2020-11-28T21:55:04.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.967+0000 7fb6f9302700 1 -- 172.21.15.162:0/3583464350 learned_addr learned my addr 172.21.15.162:0/3583464350 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:04.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.967+0000 7fb6f9b03700 1 --2- 172.21.15.162:0/3583464350 >> v2:172.21.15.118:3300/0 conn(0x7fb6ec03f840 0x7fb6ec0ba8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:04.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.967+0000 7fb6f8b01700 1 --2- 172.21.15.162:0/3583464350 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb6ec041f90 0x7fb6ec0d2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:04.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.967+0000 7fb6f9302700 1 -- 172.21.15.162:0/3583464350 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb6ec041f90 msgr2=0x7fb6ec0d2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:04.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.967+0000 7fb6f9302700 1 --2- 172.21.15.162:0/3583464350 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb6ec041f90 0x7fb6ec0d2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:04.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.967+0000 7fb6f9302700 1 -- 172.21.15.162:0/3583464350 >> v2:172.21.15.118:3300/0 conn(0x7fb6ec03f840 msgr2=0x7fb6ec0ba8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:04.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.967+0000 7fb6f9302700 1 --2- 172.21.15.162:0/3583464350 >> v2:172.21.15.118:3300/0 conn(0x7fb6ec03f840 0x7fb6ec0ba8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:04.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.967+0000 7fb6f9302700 1 -- 172.21.15.162:0/3583464350 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6ec0062d0 con 0x7fb6ec046c40 2020-11-28T21:55:04.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.967+0000 7fb6f9302700 1 --2- 172.21.15.162:0/3583464350 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6ec046c40 0x7fb6ec040b90 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 rx=0x7fb6f0002f30 tx=0x7fb6f0006e80).ready entity=mon.1 client_cookie=4ccdd823891d427 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:04.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.967+0000 7fb6e3fff700 1 -- 172.21.15.162:0/3583464350 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6f0009640 con 0x7fb6ec046c40 2020-11-28T21:55:04.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.967+0000 7fb6e3fff700 1 -- 172.21.15.162:0/3583464350 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6f00097a0 con 0x7fb6ec046c40 2020-11-28T21:55:04.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.967+0000 7fb6e3fff700 1 -- 172.21.15.162:0/3583464350 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6f0009b00 con 0x7fb6ec046c40 2020-11-28T21:55:04.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.967+0000 7fb6fa304700 1 -- 172.21.15.162:0/3583464350 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6ec046c40 msgr2=0x7fb6ec040b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:04.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.967+0000 7fb6fa304700 1 --2- 172.21.15.162:0/3583464350 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6ec046c40 0x7fb6ec040b90 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 rx=0x7fb6f0002f30 tx=0x7fb6f0006e80).stop 2020-11-28T21:55:04.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.967+0000 7fb6fa304700 1 -- 172.21.15.162:0/3583464350 shutdown_connections 2020-11-28T21:55:04.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.967+0000 7fb6fa304700 1 --2- 172.21.15.162:0/3583464350 >> v2:172.21.15.118:3300/0 conn(0x7fb6ec03f840 0x7fb6ec0ba8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:04.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.967+0000 7fb6fa304700 1 --2- 172.21.15.162:0/3583464350 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6ec046c40 0x7fb6ec040b90 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:04.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.967+0000 7fb6fa304700 1 --2- 172.21.15.162:0/3583464350 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb6ec041f90 0x7fb6ec0d2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:04.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.967+0000 7fb6fa304700 1 -- 172.21.15.162:0/3583464350 >> 172.21.15.162:0/3583464350 conn(0x7fb6ec009de0 msgr2=0x7fb6ec0461f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:04.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.967+0000 7fb6fa304700 1 -- 172.21.15.162:0/3583464350 shutdown_connections 2020-11-28T21:55:04.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.967+0000 7fb6fa304700 1 -- 172.21.15.162:0/3583464350 wait complete. 2020-11-28T21:55:04.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.971+0000 7fb6fa304700 1 Processor -- start 2020-11-28T21:55:04.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.971+0000 7fb6fa304700 1 -- start start 2020-11-28T21:55:04.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.971+0000 7fb6fa304700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb6ec03f840 0x7fb6ec0d8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:04.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.971+0000 7fb6fa304700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6ec041f90 0x7fb6ec03a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:04.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.971+0000 7fb6fa304700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb6ec046c40 0x7fb6ec03a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:04.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.971+0000 7fb6fa304700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ec0d6fc0 con 0x7fb6ec046c40 2020-11-28T21:55:04.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.971+0000 7fb6fa304700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ec03cb70 con 0x7fb6ec041f90 2020-11-28T21:55:04.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.971+0000 7fb6fa304700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ec0d6a80 con 0x7fb6ec03f840 2020-11-28T21:55:04.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.971+0000 7fb6f8b01700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6ec041f90 0x7fb6ec03a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:04.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.971+0000 7fb6f8b01700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6ec041f90 0x7fb6ec03a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44404/0 (socket says 172.21.15.162:44404) 2020-11-28T21:55:04.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.971+0000 7fb6f8b01700 1 -- 172.21.15.162:0/1002720391 learned_addr learned my addr 172.21.15.162:0/1002720391 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:04.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.971+0000 7fb6f9302700 1 --2- 172.21.15.162:0/1002720391 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb6ec03f840 0x7fb6ec0d8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:04.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.971+0000 7fb6f9b03700 1 --2- 172.21.15.162:0/1002720391 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb6ec046c40 0x7fb6ec03a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:04.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.971+0000 7fb6f8b01700 1 -- 172.21.15.162:0/1002720391 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb6ec03f840 msgr2=0x7fb6ec0d8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:04.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.971+0000 7fb6f8b01700 1 --2- 172.21.15.162:0/1002720391 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb6ec03f840 0x7fb6ec0d8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:04.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.971+0000 7fb6f8b01700 1 -- 172.21.15.162:0/1002720391 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb6ec046c40 msgr2=0x7fb6ec03a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:04.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.971+0000 7fb6f8b01700 1 --2- 172.21.15.162:0/1002720391 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb6ec046c40 0x7fb6ec03a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:04.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.971+0000 7fb6f8b01700 1 -- 172.21.15.162:0/1002720391 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6ec0d6d20 con 0x7fb6ec041f90 2020-11-28T21:55:04.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.971+0000 7fb6f8b01700 1 --2- 172.21.15.162:0/1002720391 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6ec041f90 0x7fb6ec03a040 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 rx=0x7fb6e400c000 tx=0x7fb6e4006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:04.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.971+0000 7fb6e27fc700 1 -- 172.21.15.162:0/1002720391 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6e400ca80 con 0x7fb6ec041f90 2020-11-28T21:55:04.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.971+0000 7fb6e27fc700 1 -- 172.21.15.162:0/1002720391 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6e400cbe0 con 0x7fb6ec041f90 2020-11-28T21:55:04.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.971+0000 7fb6fa304700 1 -- 172.21.15.162:0/1002720391 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6ec0d8fa0 con 0x7fb6ec041f90 2020-11-28T21:55:04.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.971+0000 7fb6e27fc700 1 -- 172.21.15.162:0/1002720391 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6e40123f0 con 0x7fb6ec041f90 2020-11-28T21:55:04.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.971+0000 7fb6fa304700 1 -- 172.21.15.162:0/1002720391 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6ec03aa20 con 0x7fb6ec041f90 2020-11-28T21:55:04.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.975+0000 7fb6fa304700 1 -- 172.21.15.162:0/1002720391 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6ec004cc0 con 0x7fb6ec041f90 2020-11-28T21:55:04.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.975+0000 7fb6e27fc700 1 -- 172.21.15.162:0/1002720391 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb6e4012550 con 0x7fb6ec041f90 2020-11-28T21:55:04.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.975+0000 7fb6e27fc700 1 --2- 172.21.15.162:0/1002720391 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb6dc055720 0x7fb6dc057be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:04.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.975+0000 7fb6e27fc700 1 -- 172.21.15.162:0/1002720391 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb6e406e370 con 0x7fb6ec041f90 2020-11-28T21:55:04.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.975+0000 7fb6f9302700 1 --2- 172.21.15.162:0/1002720391 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb6dc055720 0x7fb6dc057be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:04.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.975+0000 7fb6f9302700 1 --2- 172.21.15.162:0/1002720391 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb6dc055720 0x7fb6dc057be0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7fb6f0019000 tx=0x7fb6f0015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:04.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:03.979+0000 7fb6e27fc700 1 -- 172.21.15.162:0/1002720391 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb6e4044610 con 0x7fb6ec041f90 2020-11-28T21:55:04.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:04.147+0000 7ff9a359e700 1 -- 172.21.15.162:0/1609276242 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "c"} v 0) v1 -- 0x7ff9a4057050 con 0x7ff9a4143440 2020-11-28T21:55:04.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:04.151+0000 7fb786d49700 1 -- 172.21.15.162:0/4033472110 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "b"} v 0) v1 -- 0x7fb780155980 con 0x7fb780149970 2020-11-28T21:55:04.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:04.175+0000 7fb6fa304700 1 -- 172.21.15.162:0/1002720391 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "a"} v 0) v1 -- 0x7fb6ec0083e0 con 0x7fb6ec041f90 2020-11-28T21:55:04.835 INFO:tasks.ceph.mon.b.smithi118.stderr:2020-11-28T21:55:04.832+0000 7ff4e1520700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-11-28T21:55:04.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:04.843+0000 7ff99b7fe700 1 -- 172.21.15.162:0/1609276242 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "c"}]=0 failed mds gid 7489 v58) v1 ==== 95+0+0 (secure 0 0 0) 0x7ff99c047d80 con 0x7ff9a4143440 2020-11-28T21:55:04.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:04.843+0000 7fb77dffb700 1 -- 172.21.15.162:0/4033472110 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "b"}]=0 failed mds gid 7257 v58) v1 ==== 95+0+0 (secure 0 0 0) 0x7fb778047d90 con 0x7fb780149970 2020-11-28T21:55:04.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:04.843+0000 7fb6e27fc700 1 -- 172.21.15.162:0/1002720391 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "a"}]=0 failed mds gid 7477 v58) v1 ==== 95+0+0 (secure 0 0 0) 0x7fb6e4047c30 con 0x7fb6ec041f90 2020-11-28T21:55:04.853 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:04.847+0000 7ff9a359e700 1 -- 172.21.15.162:0/1609276242 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff988055440 msgr2=0x7ff988057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:04.854 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:04.847+0000 7ff9a359e700 1 --2- 172.21.15.162:0/1609276242 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff988055440 0x7ff988057900 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7ff98c008000 tx=0x7ff98c008040).stop 2020-11-28T21:55:04.854 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:04.847+0000 7ff9a359e700 1 -- 172.21.15.162:0/1609276242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9a4143440 msgr2=0x7ff9a4098a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:04.854 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:04.847+0000 7ff9a359e700 1 --2- 172.21.15.162:0/1609276242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9a4143440 0x7ff9a4098a80 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 rx=0x7ff99c006c90 tx=0x7ff99c008040).stop 2020-11-28T21:55:04.854 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:04.847+0000 7ff9a359e700 1 -- 172.21.15.162:0/1609276242 shutdown_connections 2020-11-28T21:55:04.855 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:04.847+0000 7ff9a359e700 1 --2- 172.21.15.162:0/1609276242 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff988055440 0x7ff988057900 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:04.855 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:04.847+0000 7ff9a359e700 1 --2- 172.21.15.162:0/1609276242 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff9a40b1870 0x7ff9a4145740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:04.855 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:04.847+0000 7ff9a359e700 1 --2- 172.21.15.162:0/1609276242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9a4143440 0x7ff9a4098a80 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:04.855 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:04.847+0000 7ff9a359e700 1 --2- 172.21.15.162:0/1609276242 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9a40b4720 0x7ff9a4098590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:04.855 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:04.847+0000 7ff9a359e700 1 -- 172.21.15.162:0/1609276242 >> 172.21.15.162:0/1609276242 conn(0x7ff9a40b54f0 msgr2=0x7ff9a4141f10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:04.856 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:04.847+0000 7ff9a359e700 1 -- 172.21.15.162:0/1609276242 shutdown_connections 2020-11-28T21:55:04.856 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:04.847+0000 7ff9a359e700 1 -- 172.21.15.162:0/1609276242 wait complete. 2020-11-28T21:55:04.856 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:04.847+0000 7fb786d49700 1 -- 172.21.15.162:0/4033472110 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb764055440 msgr2=0x7fb764057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:04.856 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:04.847+0000 7fb786d49700 1 --2- 172.21.15.162:0/4033472110 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb764055440 0x7fb764057900 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7fb76800c000 tx=0x7fb768009040).stop 2020-11-28T21:55:04.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:04.847+0000 7fb786d49700 1 -- 172.21.15.162:0/4033472110 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb780149970 msgr2=0x7fb7800678a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:04.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:04.847+0000 7fb786d49700 1 --2- 172.21.15.162:0/4033472110 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb780149970 0x7fb7800678a0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 rx=0x7fb778008000 tx=0x7fb778009040).stop 2020-11-28T21:55:04.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:04.847+0000 7fb786d49700 1 -- 172.21.15.162:0/4033472110 shutdown_connections 2020-11-28T21:55:04.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:04.847+0000 7fb786d49700 1 --2- 172.21.15.162:0/4033472110 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb764055440 0x7fb764057900 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:04.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:04.847+0000 7fb786d49700 1 --2- 172.21.15.162:0/4033472110 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb7801435b0 0x7fb78013f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:04.858 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:04.847+0000 7fb786d49700 1 --2- 172.21.15.162:0/4033472110 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb780149970 0x7fb7800678a0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:04.858 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:04.847+0000 7fb786d49700 1 --2- 172.21.15.162:0/4033472110 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb780141e40 0x7fb780063d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:04.858 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:04.847+0000 7fb786d49700 1 -- 172.21.15.162:0/4033472110 >> 172.21.15.162:0/4033472110 conn(0x7fb7800b56f0 msgr2=0x7fb780145820 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:04.858 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:04.847+0000 7fb786d49700 1 -- 172.21.15.162:0/4033472110 shutdown_connections 2020-11-28T21:55:04.858 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:04.847+0000 7fb786d49700 1 -- 172.21.15.162:0/4033472110 wait complete. 2020-11-28T21:55:04.859 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:04.847+0000 7fb6fa304700 1 -- 172.21.15.162:0/1002720391 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb6dc055720 msgr2=0x7fb6dc057be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:04.859 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:04.847+0000 7fb6fa304700 1 --2- 172.21.15.162:0/1002720391 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb6dc055720 0x7fb6dc057be0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7fb6f0019000 tx=0x7fb6f0015000).stop 2020-11-28T21:55:04.859 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:04.847+0000 7fb6fa304700 1 -- 172.21.15.162:0/1002720391 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6ec041f90 msgr2=0x7fb6ec03a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:04.859 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:04.847+0000 7fb6fa304700 1 --2- 172.21.15.162:0/1002720391 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6ec041f90 0x7fb6ec03a040 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 rx=0x7fb6e400c000 tx=0x7fb6e4006dc0).stop 2020-11-28T21:55:04.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:04.847+0000 7fb6fa304700 1 -- 172.21.15.162:0/1002720391 shutdown_connections 2020-11-28T21:55:04.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:04.847+0000 7fb6fa304700 1 --2- 172.21.15.162:0/1002720391 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb6dc055720 0x7fb6dc057be0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:04.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:04.847+0000 7fb6fa304700 1 --2- 172.21.15.162:0/1002720391 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb6ec03f840 0x7fb6ec0d8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:04.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:04.851+0000 7fb6fa304700 1 --2- 172.21.15.162:0/1002720391 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb6ec046c40 0x7fb6ec03a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:04.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:04.851+0000 7fb6fa304700 1 --2- 172.21.15.162:0/1002720391 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6ec041f90 0x7fb6ec03a040 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:04.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:04.851+0000 7fb6fa304700 1 -- 172.21.15.162:0/1002720391 >> 172.21.15.162:0/1002720391 conn(0x7fb6ec009de0 msgr2=0x7fb6ec0b9770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:04.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:04.851+0000 7fb6fa304700 1 -- 172.21.15.162:0/1002720391 shutdown_connections 2020-11-28T21:55:04.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:04.851+0000 7fb6fa304700 1 -- 172.21.15.162:0/1002720391 wait complete. 2020-11-28T21:55:04.862 INFO:teuthology.orchestra.run.smithi162.stderr:failed mds gid 7489 2020-11-28T21:55:04.862 INFO:teuthology.orchestra.run.smithi162.stderr:failed mds gid 7257 2020-11-28T21:55:04.862 INFO:teuthology.orchestra.run.smithi162.stderr:failed mds gid 7477 2020-11-28T21:55:04.870 DEBUG:teuthology.parallel:result is '' 2020-11-28T21:55:04.870 DEBUG:teuthology.parallel:result is '' 2020-11-28T21:55:04.871 DEBUG:teuthology.parallel:result is '' 2020-11-28T21:55:04.872 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2020-11-28T21:55:05.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.031+0000 7f1d8cf0f700 1 Processor -- start 2020-11-28T21:55:05.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.031+0000 7f1d8cf0f700 1 -- start start 2020-11-28T21:55:05.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.031+0000 7f1d8cf0f700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1d880b1990 0x7f1d88140890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:05.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.031+0000 7f1d8cf0f700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1d880b11b0 0x7f1d88140d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:05.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.031+0000 7f1d8cf0f700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d880b2870 con 0x7f1d880b1d70 2020-11-28T21:55:05.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.031+0000 7f1d8cf0f700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d88146e20 con 0x7f1d880b1990 2020-11-28T21:55:05.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.031+0000 7f1d8cf0f700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d880b22e0 con 0x7f1d880b11b0 2020-11-28T21:55:05.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.031+0000 7f1d85d9b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1d880b11b0 0x7f1d88140d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:05.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.031+0000 7f1d85d9b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1d880b11b0 0x7f1d88140d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44414/0 (socket says 172.21.15.162:44414) 2020-11-28T21:55:05.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.031+0000 7f1d85d9b700 1 -- 172.21.15.162:0/1672035542 learned_addr learned my addr 172.21.15.162:0/1672035542 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:05.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.035+0000 7f1d8659c700 1 --2- 172.21.15.162:0/1672035542 >> v2:172.21.15.118:3300/0 conn(0x7f1d880b1990 0x7f1d88140890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:05.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.035+0000 7f1d85d9b700 1 -- 172.21.15.162:0/1672035542 >> v2:172.21.15.118:3300/0 conn(0x7f1d880b1990 msgr2=0x7f1d88140890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:05.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.035+0000 7f1d85d9b700 1 --2- 172.21.15.162:0/1672035542 >> v2:172.21.15.118:3300/0 conn(0x7f1d880b1990 0x7f1d88140890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:05.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.035+0000 7f1d85d9b700 1 -- 172.21.15.162:0/1672035542 >> v1:172.21.15.118:6789/0 conn(0x7f1d880b1d70 legacy=0x7f1d8814a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:05.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.035+0000 7f1d85d9b700 1 -- 172.21.15.162:0/1672035542 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d880b0740 con 0x7f1d880b11b0 2020-11-28T21:55:05.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.035+0000 7f1d85d9b700 1 --2- 172.21.15.162:0/1672035542 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d880b11b0 0x7f1d88140d80 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 rx=0x7f1d78002f30 tx=0x7f1d78006e80).ready entity=mon.1 client_cookie=bdf83e2bf1954cc5 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:05.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.035+0000 7f1d8559a700 1 -- 172.21.15.162:0/1672035542 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d780097f0 con 0x7f1d880b11b0 2020-11-28T21:55:05.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.035+0000 7f1d8559a700 1 -- 172.21.15.162:0/1672035542 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d78009950 con 0x7f1d880b11b0 2020-11-28T21:55:05.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.035+0000 7f1d8559a700 1 -- 172.21.15.162:0/1672035542 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d78009cb0 con 0x7f1d880b11b0 2020-11-28T21:55:05.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.035+0000 7f1d8cf0f700 1 -- 172.21.15.162:0/1672035542 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d880b11b0 msgr2=0x7f1d88140d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:05.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.035+0000 7f1d8cf0f700 1 --2- 172.21.15.162:0/1672035542 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d880b11b0 0x7f1d88140d80 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 rx=0x7f1d78002f30 tx=0x7f1d78006e80).stop 2020-11-28T21:55:05.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.035+0000 7f1d8cf0f700 1 -- 172.21.15.162:0/1672035542 shutdown_connections 2020-11-28T21:55:05.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.035+0000 7f1d8cf0f700 1 --2- 172.21.15.162:0/1672035542 >> v2:172.21.15.118:3300/0 conn(0x7f1d880b1990 0x7f1d88140890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:05.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.035+0000 7f1d8cf0f700 1 --2- 172.21.15.162:0/1672035542 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d880b11b0 0x7f1d88140d80 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:05.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.035+0000 7f1d8cf0f700 1 -- 172.21.15.162:0/1672035542 >> 172.21.15.162:0/1672035542 conn(0x7f1d8805b970 msgr2=0x7f1d8805bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:05.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.035+0000 7f1d8cf0f700 1 -- 172.21.15.162:0/1672035542 shutdown_connections 2020-11-28T21:55:05.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.035+0000 7f1d8cf0f700 1 -- 172.21.15.162:0/1672035542 wait complete. 2020-11-28T21:55:05.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.035+0000 7f1d8cf0f700 1 Processor -- start 2020-11-28T21:55:05.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.035+0000 7f1d8cf0f700 1 -- start start 2020-11-28T21:55:05.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.035+0000 7f1d8cf0f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1d880b11b0 0x7f1d880b4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:05.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.035+0000 7f1d8cf0f700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1d880b1990 0x7f1d8813f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:05.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.035+0000 7f1d8cf0f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d880b1d70 0x7f1d8813fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:05.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.035+0000 7f1d8cf0f700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8814e220 con 0x7f1d880b1990 2020-11-28T21:55:05.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.035+0000 7f1d8cf0f700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d881402e0 con 0x7f1d880b1d70 2020-11-28T21:55:05.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.035+0000 7f1d8cf0f700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8814e480 con 0x7f1d880b11b0 2020-11-28T21:55:05.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.035+0000 7f1d86d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d880b1d70 0x7f1d8813fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:05.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.039+0000 7f1d86d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d880b1d70 0x7f1d8813fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44420/0 (socket says 172.21.15.162:44420) 2020-11-28T21:55:05.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.039+0000 7f1d86d9d700 1 -- 172.21.15.162:0/263707703 learned_addr learned my addr 172.21.15.162:0/263707703 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:05.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.039+0000 7f1d85d9b700 1 --2- 172.21.15.162:0/263707703 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1d880b1990 0x7f1d8813f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:05.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.039+0000 7f1d8659c700 1 --2- 172.21.15.162:0/263707703 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1d880b11b0 0x7f1d880b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:05.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.039+0000 7f1d86d9d700 1 -- 172.21.15.162:0/263707703 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1d880b11b0 msgr2=0x7f1d880b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:05.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.039+0000 7f1d86d9d700 1 --2- 172.21.15.162:0/263707703 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1d880b11b0 0x7f1d880b4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:05.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.039+0000 7f1d86d9d700 1 -- 172.21.15.162:0/263707703 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1d880b1990 msgr2=0x7f1d8813f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:05.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.039+0000 7f1d86d9d700 1 --2- 172.21.15.162:0/263707703 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1d880b1990 0x7f1d8813f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:05.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.039+0000 7f1d86d9d700 1 -- 172.21.15.162:0/263707703 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d88143fc0 con 0x7f1d880b1d70 2020-11-28T21:55:05.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.039+0000 7f1d8659c700 1 --2- 172.21.15.162:0/263707703 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1d880b11b0 0x7f1d880b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:55:05.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.039+0000 7f1d86d9d700 1 --2- 172.21.15.162:0/263707703 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d880b1d70 0x7f1d8813fdf0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 rx=0x7f1d8814a650 tx=0x7f1d80007f70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:05.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.039+0000 7f1d7f7fe700 1 -- 172.21.15.162:0/263707703 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d80003080 con 0x7f1d880b1d70 2020-11-28T21:55:05.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.039+0000 7f1d7f7fe700 1 -- 172.21.15.162:0/263707703 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8000a7b0 con 0x7f1d880b1d70 2020-11-28T21:55:05.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.039+0000 7f1d7f7fe700 1 -- 172.21.15.162:0/263707703 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d8000ab10 con 0x7f1d880b1d70 2020-11-28T21:55:05.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.039+0000 7f1d8cf0f700 1 -- 172.21.15.162:0/263707703 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d88144260 con 0x7f1d880b1d70 2020-11-28T21:55:05.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.039+0000 7f1d8cf0f700 1 -- 172.21.15.162:0/263707703 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d8814a690 con 0x7f1d880b1d70 2020-11-28T21:55:05.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.039+0000 7f1d8cf0f700 1 -- 172.21.15.162:0/263707703 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d88057050 con 0x7f1d880b1d70 2020-11-28T21:55:05.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.039+0000 7f1d7f7fe700 1 -- 172.21.15.162:0/263707703 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1d80019030 con 0x7f1d880b1d70 2020-11-28T21:55:05.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.043+0000 7f1d7f7fe700 1 --2- 172.21.15.162:0/263707703 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1d6c055330 0x7f1d6c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:05.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.043+0000 7f1d7f7fe700 1 -- 172.21.15.162:0/263707703 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f1d8006d700 con 0x7f1d880b1d70 2020-11-28T21:55:05.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.043+0000 7f1d8659c700 1 --2- 172.21.15.162:0/263707703 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1d6c055330 0x7f1d6c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:05.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.043+0000 7f1d8659c700 1 --2- 172.21.15.162:0/263707703 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1d6c055330 0x7f1d6c0577f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f1d7000c000 tx=0x7f1d70008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:05.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.047+0000 7f1d7f7fe700 1 -- 172.21.15.162:0/263707703 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1d80043910 con 0x7f1d880b1d70 2020-11-28T21:55:05.234 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.231+0000 7f1d8cf0f700 1 -- 172.21.15.162:0/263707703 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1d8805d530 con 0x7f1d880b1d70 2020-11-28T21:55:05.859 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.855+0000 7f1d7f7fe700 1 -- 172.21.15.162:0/263707703 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v59) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1d80046f30 con 0x7f1d880b1d70 2020-11-28T21:55:05.862 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.855+0000 7f1d8cf0f700 1 -- 172.21.15.162:0/263707703 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1d6c055330 msgr2=0x7f1d6c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:05.864 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.855+0000 7f1d8cf0f700 1 --2- 172.21.15.162:0/263707703 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1d6c055330 0x7f1d6c0577f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f1d7000c000 tx=0x7f1d70008040).stop 2020-11-28T21:55:05.865 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.855+0000 7f1d8cf0f700 1 -- 172.21.15.162:0/263707703 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d880b1d70 msgr2=0x7f1d8813fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:05.865 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.855+0000 7f1d8cf0f700 1 --2- 172.21.15.162:0/263707703 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d880b1d70 0x7f1d8813fdf0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 rx=0x7f1d8814a650 tx=0x7f1d80007f70).stop 2020-11-28T21:55:05.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.855+0000 7f1d8cf0f700 1 -- 172.21.15.162:0/263707703 shutdown_connections 2020-11-28T21:55:05.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.855+0000 7f1d8cf0f700 1 --2- 172.21.15.162:0/263707703 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1d6c055330 0x7f1d6c0577f0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:05.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.855+0000 7f1d8cf0f700 1 --2- 172.21.15.162:0/263707703 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1d880b11b0 0x7f1d880b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:05.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.855+0000 7f1d8cf0f700 1 --2- 172.21.15.162:0/263707703 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d880b1d70 0x7f1d8813fdf0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:05.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.855+0000 7f1d8cf0f700 1 --2- 172.21.15.162:0/263707703 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1d880b1990 0x7f1d8813f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:05.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.855+0000 7f1d8cf0f700 1 -- 172.21.15.162:0/263707703 >> 172.21.15.162:0/263707703 conn(0x7f1d8805b970 msgr2=0x7f1d88146f70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:05.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.855+0000 7f1d8cf0f700 1 -- 172.21.15.162:0/263707703 shutdown_connections 2020-11-28T21:55:05.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:05.855+0000 7f1d8cf0f700 1 -- 172.21.15.162:0/263707703 wait complete. 2020-11-28T21:55:05.881 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-11-28T21:55:06.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.003+0000 7f8b6a910700 1 Processor -- start 2020-11-28T21:55:06.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.003+0000 7f8b6a910700 1 -- start start 2020-11-28T21:55:06.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.007+0000 7f8b6a910700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8b640a8df0 0x7f8b640b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:06.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.007+0000 7f8b6a910700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b64060500 con 0x7f8b64099d30 2020-11-28T21:55:06.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.007+0000 7f8b6a910700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b640607a0 con 0x7f8b640afec0 2020-11-28T21:55:06.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.007+0000 7f8b6a910700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b64060ce0 con 0x7f8b640a8df0 2020-11-28T21:55:06.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.007+0000 7f8b6990e700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f8b640afec0 0x7f8b64099620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51268/0 (socket says 172.21.15.162:51268) 2020-11-28T21:55:06.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.007+0000 7f8b6990e700 1 -- 172.21.15.162:0/418537753 learned_addr learned my addr 172.21.15.162:0/418537753 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:06.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.007+0000 7f8b6910d700 1 --2- 172.21.15.162:0/418537753 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8b640a8df0 0x7f8b640b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:06.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.007+0000 7f8b6890c700 1 -- 172.21.15.162:0/418537753 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3175349522 0 0) 0x7f8b640607a0 con 0x7f8b640afec0 2020-11-28T21:55:06.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.007+0000 7f8b6890c700 1 -- 172.21.15.162:0/418537753 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b48004340 con 0x7f8b640afec0 2020-11-28T21:55:06.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.007+0000 7f8b6890c700 1 -- 172.21.15.162:0/418537753 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1922760830 0 0) 0x7f8b64060500 con 0x7f8b64099d30 2020-11-28T21:55:06.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.007+0000 7f8b6890c700 1 -- 172.21.15.162:0/418537753 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b640607a0 con 0x7f8b64099d30 2020-11-28T21:55:06.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.007+0000 7f8b6890c700 1 -- 172.21.15.162:0/418537753 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2916542789 0 0) 0x7f8b48004340 con 0x7f8b640afec0 2020-11-28T21:55:06.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.007+0000 7f8b6890c700 1 -- 172.21.15.162:0/418537753 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8b640a8df0 msgr2=0x7f8b640b02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:06.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.007+0000 7f8b6890c700 1 --2- 172.21.15.162:0/418537753 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8b640a8df0 0x7f8b640b02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:06.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.007+0000 7f8b6890c700 1 -- 172.21.15.162:0/418537753 >> v1:172.21.15.118:6789/0 conn(0x7f8b64099d30 legacy=0x7f8b64148d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:06.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.007+0000 7f8b6890c700 1 -- 172.21.15.162:0/418537753 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b64060a40 con 0x7f8b640afec0 2020-11-28T21:55:06.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.007+0000 7f8b6890c700 1 -- 172.21.15.162:0/418537753 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f8b58001a50 con 0x7f8b640afec0 2020-11-28T21:55:06.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.007+0000 7f8b6890c700 1 -- 172.21.15.162:0/418537753 >> v1:172.21.15.162:6789/0 conn(0x7f8b640afec0 legacy=0x7f8b64099620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:06.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.007+0000 7f8b6890c700 1 --2- 172.21.15.162:0/418537753 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8b480049c0 0x7f8b48004dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:06.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.007+0000 7f8b6890c700 1 --2- 172.21.15.162:0/418537753 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8b640a8df0 0x7f8b480055e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:06.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.007+0000 7f8b6890c700 1 --2- 172.21.15.162:0/418537753 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b48005ad0 0x7f8b48005f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:06.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.007+0000 7f8b6890c700 1 -- 172.21.15.162:0/418537753 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b64060ce0 con 0x7f8b480049c0 2020-11-28T21:55:06.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.007+0000 7f8b6890c700 1 -- 172.21.15.162:0/418537753 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b48006440 con 0x7f8b48005ad0 2020-11-28T21:55:06.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.007+0000 7f8b6890c700 1 -- 172.21.15.162:0/418537753 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b48003230 con 0x7f8b640a8df0 2020-11-28T21:55:06.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.007+0000 7f8b6a10f700 1 --2- 172.21.15.162:0/418537753 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b48005ad0 0x7f8b48005f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:06.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.007+0000 7f8b6910d700 1 --2- 172.21.15.162:0/418537753 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8b640a8df0 0x7f8b480055e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:06.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.007+0000 7f8b6990e700 1 --2- 172.21.15.162:0/418537753 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8b480049c0 0x7f8b48004dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:06.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.007+0000 7f8b6a10f700 1 -- 172.21.15.162:0/418537753 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8b640a8df0 msgr2=0x7f8b480055e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:06.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.007+0000 7f8b6a10f700 1 --2- 172.21.15.162:0/418537753 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8b640a8df0 0x7f8b480055e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:06.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.007+0000 7f8b6a10f700 1 -- 172.21.15.162:0/418537753 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8b480049c0 msgr2=0x7f8b48004dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:06.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.007+0000 7f8b6a10f700 1 --2- 172.21.15.162:0/418537753 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8b480049c0 0x7f8b48004dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:06.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.007+0000 7f8b6a10f700 1 -- 172.21.15.162:0/418537753 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b48004340 con 0x7f8b48005ad0 2020-11-28T21:55:06.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.007+0000 7f8b6990e700 1 --2- 172.21.15.162:0/418537753 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8b480049c0 0x7f8b48004dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:55:06.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.007+0000 7f8b6a10f700 1 --2- 172.21.15.162:0/418537753 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b48005ad0 0x7f8b48005f50 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 rx=0x7f8b5c0010d0 tx=0x7f8b5c007b60).ready entity=mon.1 client_cookie=fd4b0b131e600417 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:06.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.007+0000 7f8b6890c700 1 -- 172.21.15.162:0/418537753 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b5c00c870 con 0x7f8b48005ad0 2020-11-28T21:55:06.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.007+0000 7f8b6890c700 1 -- 172.21.15.162:0/418537753 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b5c00c9d0 con 0x7f8b48005ad0 2020-11-28T21:55:06.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.007+0000 7f8b6a910700 1 -- 172.21.15.162:0/418537753 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b48005ad0 msgr2=0x7f8b48005f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:06.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.007+0000 7f8b6890c700 1 -- 172.21.15.162:0/418537753 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b5c00cd30 con 0x7f8b48005ad0 2020-11-28T21:55:06.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.007+0000 7f8b6a910700 1 --2- 172.21.15.162:0/418537753 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b48005ad0 0x7f8b48005f50 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 rx=0x7f8b5c0010d0 tx=0x7f8b5c007b60).stop 2020-11-28T21:55:06.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.007+0000 7f8b6a10f700 1 -- 172.21.15.162:0/418537753 reap_dead start 2020-11-28T21:55:06.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.011+0000 7f8b6a910700 1 -- 172.21.15.162:0/418537753 shutdown_connections 2020-11-28T21:55:06.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.011+0000 7f8b6a910700 1 -- 172.21.15.162:0/418537753 >> 172.21.15.162:0/418537753 conn(0x7f8b640b5520 msgr2=0x7f8b640b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:06.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.011+0000 7f8b6a910700 1 -- 172.21.15.162:0/418537753 shutdown_connections 2020-11-28T21:55:06.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.011+0000 7f8b6a910700 1 -- 172.21.15.162:0/418537753 wait complete. 2020-11-28T21:55:06.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.011+0000 7f8b6a910700 1 Processor -- start 2020-11-28T21:55:06.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.011+0000 7f8b6a910700 1 -- start start 2020-11-28T21:55:06.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.011+0000 7f8b6a910700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8b64147e50 0x7f8b64155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:06.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.011+0000 7f8b6a910700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8b640ab430 0x7f8b640aa510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:06.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.011+0000 7f8b6a910700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b640aaa00 0x7f8b64151660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:06.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.011+0000 7f8b6a910700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b640ac0a0 con 0x7f8b64147e50 2020-11-28T21:55:06.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.011+0000 7f8b6a910700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b64148230 con 0x7f8b640aaa00 2020-11-28T21:55:06.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.011+0000 7f8b6a910700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b64146cf0 con 0x7f8b640ab430 2020-11-28T21:55:06.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.011+0000 7f8b6a10f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b640aaa00 0x7f8b64151660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:06.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.011+0000 7f8b6a10f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b640aaa00 0x7f8b64151660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44440/0 (socket says 172.21.15.162:44440) 2020-11-28T21:55:06.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.011+0000 7f8b6a10f700 1 -- 172.21.15.162:0/2041709154 learned_addr learned my addr 172.21.15.162:0/2041709154 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:06.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.011+0000 7f8b6990e700 1 --2- 172.21.15.162:0/2041709154 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8b64147e50 0x7f8b64155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:06.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.011+0000 7f8b6910d700 1 --2- 172.21.15.162:0/2041709154 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8b640ab430 0x7f8b640aa510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:06.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.011+0000 7f8b6a10f700 1 -- 172.21.15.162:0/2041709154 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8b640ab430 msgr2=0x7f8b640aa510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:06.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.011+0000 7f8b6a10f700 1 --2- 172.21.15.162:0/2041709154 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8b640ab430 0x7f8b640aa510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:06.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.011+0000 7f8b6a10f700 1 -- 172.21.15.162:0/2041709154 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8b64147e50 msgr2=0x7f8b64155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:06.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.011+0000 7f8b6a10f700 1 --2- 172.21.15.162:0/2041709154 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8b64147e50 0x7f8b64155510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:06.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.011+0000 7f8b6a10f700 1 -- 172.21.15.162:0/2041709154 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b64155b50 con 0x7f8b640aaa00 2020-11-28T21:55:06.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.011+0000 7f8b6910d700 1 --2- 172.21.15.162:0/2041709154 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8b640ab430 0x7f8b640aa510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:55:06.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.011+0000 7f8b6a10f700 1 --2- 172.21.15.162:0/2041709154 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b640aaa00 0x7f8b64151660 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 rx=0x7f8b640ad4c0 tx=0x7f8b5c017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:06.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.015+0000 7f8b62ffd700 1 -- 172.21.15.162:0/2041709154 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b5c007470 con 0x7f8b640aaa00 2020-11-28T21:55:06.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.015+0000 7f8b62ffd700 1 -- 172.21.15.162:0/2041709154 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b5c012960 con 0x7f8b640aaa00 2020-11-28T21:55:06.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.015+0000 7f8b6a910700 1 -- 172.21.15.162:0/2041709154 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b640acc00 con 0x7f8b640aaa00 2020-11-28T21:55:06.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.015+0000 7f8b62ffd700 1 -- 172.21.15.162:0/2041709154 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b5c012cc0 con 0x7f8b640aaa00 2020-11-28T21:55:06.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.015+0000 7f8b6a910700 1 -- 172.21.15.162:0/2041709154 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b640ad140 con 0x7f8b640aaa00 2020-11-28T21:55:06.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.015+0000 7f8b6a910700 1 -- 172.21.15.162:0/2041709154 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b64057050 con 0x7f8b640aaa00 2020-11-28T21:55:06.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.015+0000 7f8b62ffd700 1 -- 172.21.15.162:0/2041709154 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8b5c010070 con 0x7f8b640aaa00 2020-11-28T21:55:06.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.015+0000 7f8b62ffd700 1 --2- 172.21.15.162:0/2041709154 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8b4c0554e0 0x7f8b4c0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:06.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.015+0000 7f8b62ffd700 1 -- 172.21.15.162:0/2041709154 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f8b5c06e4c0 con 0x7f8b640aaa00 2020-11-28T21:55:06.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.015+0000 7f8b6990e700 1 --2- 172.21.15.162:0/2041709154 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8b4c0554e0 0x7f8b4c0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:06.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.019+0000 7f8b6990e700 1 --2- 172.21.15.162:0/2041709154 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8b4c0554e0 0x7f8b4c0579a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7f8b5800d000 tx=0x7f8b58007040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:06.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.023+0000 7f8b62ffd700 1 -- 172.21.15.162:0/2041709154 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8b5c044770 con 0x7f8b640aaa00 2020-11-28T21:55:06.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.167+0000 7f8b6a910700 1 -- 172.21.15.162:0/2041709154 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8b6405d680 con 0x7f8b640aaa00 2020-11-28T21:55:06.862 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.859+0000 7f8b62ffd700 1 -- 172.21.15.162:0/2041709154 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v123) v1 ==== 185+0+0 (secure 0 0 0) 0x7f8b5c047d90 con 0x7f8b640aaa00 2020-11-28T21:55:06.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.863+0000 7f8b6a910700 1 -- 172.21.15.162:0/2041709154 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8b4c0554e0 msgr2=0x7f8b4c0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:06.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.863+0000 7f8b6a910700 1 --2- 172.21.15.162:0/2041709154 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8b4c0554e0 0x7f8b4c0579a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7f8b5800d000 tx=0x7f8b58007040).stop 2020-11-28T21:55:06.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.863+0000 7f8b6a910700 1 -- 172.21.15.162:0/2041709154 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b640aaa00 msgr2=0x7f8b64151660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:06.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.863+0000 7f8b6a910700 1 --2- 172.21.15.162:0/2041709154 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b640aaa00 0x7f8b64151660 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 rx=0x7f8b640ad4c0 tx=0x7f8b5c017000).stop 2020-11-28T21:55:06.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.863+0000 7f8b6a910700 1 -- 172.21.15.162:0/2041709154 shutdown_connections 2020-11-28T21:55:06.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.863+0000 7f8b6a910700 1 --2- 172.21.15.162:0/2041709154 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8b4c0554e0 0x7f8b4c0579a0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:06.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.863+0000 7f8b6a910700 1 --2- 172.21.15.162:0/2041709154 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b640aaa00 0x7f8b64151660 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:06.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.863+0000 7f8b6a910700 1 --2- 172.21.15.162:0/2041709154 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8b64147e50 0x7f8b64155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:06.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.863+0000 7f8b6a910700 1 --2- 172.21.15.162:0/2041709154 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8b640ab430 0x7f8b640aa510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:06.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.863+0000 7f8b6a910700 1 -- 172.21.15.162:0/2041709154 >> 172.21.15.162:0/2041709154 conn(0x7f8b640b5520 msgr2=0x7f8b640b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:06.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.863+0000 7f8b6a910700 1 -- 172.21.15.162:0/2041709154 shutdown_connections 2020-11-28T21:55:06.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:06.863+0000 7f8b6a910700 1 -- 172.21.15.162:0/2041709154 wait complete. 2020-11-28T21:55:06.870 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_metadata' removed 2020-11-28T21:55:06.883 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2020-11-28T21:55:07.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.003+0000 7f109e5c0700 1 Processor -- start 2020-11-28T21:55:07.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.003+0000 7f109e5c0700 1 -- start start 2020-11-28T21:55:07.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.003+0000 7f109e5c0700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f10980b21f0 0x7f10981434f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:07.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.003+0000 7f109e5c0700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f10980ae3d0 0x7f109814b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:07.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.003+0000 7f109e5c0700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10980af6a0 con 0x7f1098141e80 2020-11-28T21:55:07.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.003+0000 7f109e5c0700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10980b25d0 con 0x7f10980b21f0 2020-11-28T21:55:07.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.003+0000 7f109e5c0700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f109805fff0 con 0x7f10980ae3d0 2020-11-28T21:55:07.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.003+0000 7f109ddbf700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f10980ae3d0 0x7f109814b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:07.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.003+0000 7f109ddbf700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f10980ae3d0 0x7f109814b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:41672/0 (socket says 172.21.15.162:41672) 2020-11-28T21:55:07.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.003+0000 7f109ddbf700 1 -- 172.21.15.162:0/19535503 learned_addr learned my addr 172.21.15.162:0/19535503 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:07.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.003+0000 7f109d5be700 1 --2- 172.21.15.162:0/19535503 >> v2:172.21.15.118:3300/0 conn(0x7f10980b21f0 0x7f10981434f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:07.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.007+0000 7f109ddbf700 1 -- 172.21.15.162:0/19535503 >> v2:172.21.15.118:3300/0 conn(0x7f10980b21f0 msgr2=0x7f10981434f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:07.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.007+0000 7f109ddbf700 1 --2- 172.21.15.162:0/19535503 >> v2:172.21.15.118:3300/0 conn(0x7f10980b21f0 0x7f10981434f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:07.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.007+0000 7f109ddbf700 1 -- 172.21.15.162:0/19535503 >> v1:172.21.15.118:6789/0 conn(0x7f1098141e80 legacy=0x7f10980606f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:07.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.007+0000 7f109ddbf700 1 -- 172.21.15.162:0/19535503 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10980af8d0 con 0x7f10980ae3d0 2020-11-28T21:55:07.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.007+0000 7f108ffff700 1 -- 172.21.15.162:0/19535503 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3225884893 0 0) 0x7f10980af6a0 con 0x7f1098141e80 2020-11-28T21:55:07.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.007+0000 7f109ddbf700 1 --2- 172.21.15.162:0/19535503 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f10980ae3d0 0x7f109814b250 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f10900032b0 tx=0x7f1090009270).ready entity=mon.2 client_cookie=bebbadd12c368c3 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:07.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.007+0000 7f108ffff700 1 -- 172.21.15.162:0/19535503 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1090009e60 con 0x7f10980ae3d0 2020-11-28T21:55:07.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.007+0000 7f108ffff700 1 -- 172.21.15.162:0/19535503 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109000bc00 con 0x7f10980ae3d0 2020-11-28T21:55:07.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.007+0000 7f108ffff700 1 -- 172.21.15.162:0/19535503 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f109000bd70 con 0x7f10980ae3d0 2020-11-28T21:55:07.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.007+0000 7f109e5c0700 1 -- 172.21.15.162:0/19535503 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f10980ae3d0 msgr2=0x7f109814b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:07.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.007+0000 7f109e5c0700 1 --2- 172.21.15.162:0/19535503 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f10980ae3d0 0x7f109814b250 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f10900032b0 tx=0x7f1090009270).stop 2020-11-28T21:55:07.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.007+0000 7f109e5c0700 1 -- 172.21.15.162:0/19535503 shutdown_connections 2020-11-28T21:55:07.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.007+0000 7f109e5c0700 1 --2- 172.21.15.162:0/19535503 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f10980ae3d0 0x7f109814b250 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:07.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.007+0000 7f109e5c0700 1 --2- 172.21.15.162:0/19535503 >> v2:172.21.15.118:3300/0 conn(0x7f10980b21f0 0x7f10981434f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:07.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.007+0000 7f109e5c0700 1 -- 172.21.15.162:0/19535503 >> 172.21.15.162:0/19535503 conn(0x7f10980b57d0 msgr2=0x7f10980b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:07.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.007+0000 7f109e5c0700 1 -- 172.21.15.162:0/19535503 shutdown_connections 2020-11-28T21:55:07.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.007+0000 7f109e5c0700 1 -- 172.21.15.162:0/19535503 wait complete. 2020-11-28T21:55:07.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.007+0000 7f109e5c0700 1 Processor -- start 2020-11-28T21:55:07.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.007+0000 7f109e5c0700 1 -- start start 2020-11-28T21:55:07.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.011+0000 7f109e5c0700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f10980ae3d0 0x7f10980aafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:07.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.011+0000 7f109e5c0700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f10980b21f0 0x7f109814f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:07.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.011+0000 7f109e5c0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1098141e80 0x7f1098064b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:07.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.011+0000 7f109e5c0700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1098063b80 con 0x7f10980b21f0 2020-11-28T21:55:07.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.011+0000 7f109e5c0700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1098063e20 con 0x7f1098141e80 2020-11-28T21:55:07.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.011+0000 7f109e5c0700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10980640c0 con 0x7f10980ae3d0 2020-11-28T21:55:07.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.011+0000 7f109ddbf700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1098141e80 0x7f1098064b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:07.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.011+0000 7f109ddbf700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1098141e80 0x7f1098064b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44452/0 (socket says 172.21.15.162:44452) 2020-11-28T21:55:07.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.011+0000 7f109ddbf700 1 -- 172.21.15.162:0/607634184 learned_addr learned my addr 172.21.15.162:0/607634184 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:07.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.011+0000 7f109d5be700 1 --2- 172.21.15.162:0/607634184 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f10980ae3d0 0x7f10980aafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:07.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.011+0000 7f109cdbd700 1 --2- 172.21.15.162:0/607634184 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f10980b21f0 0x7f109814f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:07.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.011+0000 7f109ddbf700 1 -- 172.21.15.162:0/607634184 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f10980ae3d0 msgr2=0x7f10980aafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:07.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.011+0000 7f109ddbf700 1 --2- 172.21.15.162:0/607634184 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f10980ae3d0 0x7f10980aafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:07.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.011+0000 7f109ddbf700 1 -- 172.21.15.162:0/607634184 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f10980b21f0 msgr2=0x7f109814f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:07.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.011+0000 7f109ddbf700 1 --2- 172.21.15.162:0/607634184 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f10980b21f0 0x7f109814f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:07.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.011+0000 7f109ddbf700 1 -- 172.21.15.162:0/607634184 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1098064680 con 0x7f1098141e80 2020-11-28T21:55:07.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.011+0000 7f109ddbf700 1 --2- 172.21.15.162:0/607634184 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1098141e80 0x7f1098064b20 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 rx=0x7f10900032b0 tx=0x7f1090016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:07.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.011+0000 7f108e7fc700 1 -- 172.21.15.162:0/607634184 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f10900107a0 con 0x7f1098141e80 2020-11-28T21:55:07.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.011+0000 7f108e7fc700 1 -- 172.21.15.162:0/607634184 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1090010900 con 0x7f1098141e80 2020-11-28T21:55:07.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.011+0000 7f109e5c0700 1 -- 172.21.15.162:0/607634184 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1098065010 con 0x7f1098141e80 2020-11-28T21:55:07.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.011+0000 7f108e7fc700 1 -- 172.21.15.162:0/607634184 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1090010c60 con 0x7f1098141e80 2020-11-28T21:55:07.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.011+0000 7f109e5c0700 1 -- 172.21.15.162:0/607634184 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1098065630 con 0x7f1098141e80 2020-11-28T21:55:07.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.015+0000 7f109e5c0700 1 -- 172.21.15.162:0/607634184 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1098063470 con 0x7f1098141e80 2020-11-28T21:55:07.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.015+0000 7f108e7fc700 1 -- 172.21.15.162:0/607634184 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f109000f070 con 0x7f1098141e80 2020-11-28T21:55:07.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.015+0000 7f108e7fc700 1 --2- 172.21.15.162:0/607634184 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1080055440 0x7f1080057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:07.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.015+0000 7f108e7fc700 1 -- 172.21.15.162:0/607634184 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5721+0+0 (secure 0 0 0) 0x7f109006f4e0 con 0x7f1098141e80 2020-11-28T21:55:07.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.015+0000 7f109d5be700 1 --2- 172.21.15.162:0/607634184 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1080055440 0x7f1080057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:07.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.015+0000 7f109d5be700 1 --2- 172.21.15.162:0/607634184 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1080055440 0x7f1080057900 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f108800b000 tx=0x7f1088006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:07.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.019+0000 7f108e7fc700 1 -- 172.21.15.162:0/607634184 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1098063470 con 0x7f1098141e80 2020-11-28T21:55:07.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.163+0000 7f109e5c0700 1 -- 172.21.15.162:0/607634184 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f1098063470 con 0x7f1098141e80 2020-11-28T21:55:07.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.875+0000 7f108e7fc700 1 -- 172.21.15.162:0/607634184 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v124) v1 ==== 133+0+0 (secure 0 0 0) 0x7f1098063470 con 0x7f1098141e80 2020-11-28T21:55:07.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.879+0000 7f109e5c0700 1 -- 172.21.15.162:0/607634184 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1080055440 msgr2=0x7f1080057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:07.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.879+0000 7f109e5c0700 1 --2- 172.21.15.162:0/607634184 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1080055440 0x7f1080057900 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f108800b000 tx=0x7f1088006dc0).stop 2020-11-28T21:55:07.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.879+0000 7f109e5c0700 1 -- 172.21.15.162:0/607634184 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1098141e80 msgr2=0x7f1098064b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:07.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.879+0000 7f109e5c0700 1 --2- 172.21.15.162:0/607634184 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1098141e80 0x7f1098064b20 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 rx=0x7f10900032b0 tx=0x7f1090016000).stop 2020-11-28T21:55:07.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.879+0000 7f109e5c0700 1 -- 172.21.15.162:0/607634184 shutdown_connections 2020-11-28T21:55:07.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.879+0000 7f109e5c0700 1 --2- 172.21.15.162:0/607634184 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1080055440 0x7f1080057900 secure :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 rx=0x7f108800b000 tx=0x7f1088006dc0).stop 2020-11-28T21:55:07.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.879+0000 7f109e5c0700 1 --2- 172.21.15.162:0/607634184 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f10980ae3d0 0x7f10980aafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:07.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.879+0000 7f109e5c0700 1 --2- 172.21.15.162:0/607634184 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1098141e80 0x7f1098064b20 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:07.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.879+0000 7f109e5c0700 1 --2- 172.21.15.162:0/607634184 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f10980b21f0 0x7f109814f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:07.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.879+0000 7f109e5c0700 1 -- 172.21.15.162:0/607634184 >> 172.21.15.162:0/607634184 conn(0x7f10980b57d0 msgr2=0x7f109805b970 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:07.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.879+0000 7f109e5c0700 1 -- 172.21.15.162:0/607634184 shutdown_connections 2020-11-28T21:55:07.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:07.879+0000 7f109e5c0700 1 -- 172.21.15.162:0/607634184 wait complete. 2020-11-28T21:55:07.884 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_metadata' created 2020-11-28T21:55:07.898 INFO:teuthology.orchestra.run.smithi192:> sudo rados -p cephfs_metadata put key /etc/fstab 2020-11-28T21:55:07.951 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.933+0000 7fd209fc1c00 1 Processor -- start 2020-11-28T21:55:07.951 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.933+0000 7fd209fc1c00 1 -- start start 2020-11-28T21:55:07.951 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.933+0000 7fd209fc1c00 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562db0d95b20 0x562db0d90660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:07.951 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.933+0000 7fd209fc1c00 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562db0d9c800 0x562db0d90b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:07.952 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.933+0000 7fd209fc1c00 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x562db0d9f0e0 0x562db0daba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:07.952 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.933+0000 7fd209fc1c00 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x562db0c88450 con 0x562db0d9c800 2020-11-28T21:55:07.952 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.933+0000 7fd209fc1c00 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x562db0d91410 con 0x562db0d95b20 2020-11-28T21:55:07.952 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.933+0000 7fd209fc1c00 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x562db0dabf80 con 0x562db0d9f0e0 2020-11-28T21:55:07.952 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.933+0000 7fd1fbeb5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562db0d95b20 0x562db0d90660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:07.953 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.933+0000 7fd1fbeb5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562db0d95b20 0x562db0d90660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.192:50326/0 (socket says 172.21.15.192:50326) 2020-11-28T21:55:07.953 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.933+0000 7fd1fbeb5700 1 -- 172.21.15.192:0/130657301 learned_addr learned my addr 172.21.15.192:0/130657301 (peer_addr_for_me v2:172.21.15.192:0/0) 2020-11-28T21:55:07.953 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.933+0000 7fd1fb6b4700 1 --2- 172.21.15.192:0/130657301 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562db0d9c800 0x562db0d90b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:07.953 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.933+0000 7fd1fc6b6700 1 --2- 172.21.15.192:0/130657301 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x562db0d9f0e0 0x562db0daba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:07.953 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.933+0000 7fd1fbeb5700 1 -- 172.21.15.192:0/130657301 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x562db0d9f0e0 msgr2=0x562db0daba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:07.954 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.937+0000 7fd1fbeb5700 1 --2- 172.21.15.192:0/130657301 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x562db0d9f0e0 0x562db0daba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:07.954 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.937+0000 7fd1fbeb5700 1 -- 172.21.15.192:0/130657301 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562db0d9c800 msgr2=0x562db0d90b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:07.954 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.937+0000 7fd1fbeb5700 1 --2- 172.21.15.192:0/130657301 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562db0d9c800 0x562db0d90b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:07.954 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.937+0000 7fd1fbeb5700 1 -- 172.21.15.192:0/130657301 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562db0caec80 con 0x562db0d95b20 2020-11-28T21:55:07.955 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.937+0000 7fd1fbeb5700 1 --2- 172.21.15.192:0/130657301 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562db0d95b20 0x562db0d90660 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 rx=0x7fd1f0002f30 tx=0x7fd1f0015000).ready entity=mon.1 client_cookie=83c5d41ad2bb31de server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:07.955 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.937+0000 7fd1f9eb1700 1 -- 172.21.15.192:0/130657301 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1f001d630 con 0x562db0d95b20 2020-11-28T21:55:07.955 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.937+0000 7fd1f9eb1700 1 -- 172.21.15.192:0/130657301 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f001d790 con 0x562db0d95b20 2020-11-28T21:55:07.955 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.937+0000 7fd1f9eb1700 1 -- 172.21.15.192:0/130657301 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1f001daf0 con 0x562db0d95b20 2020-11-28T21:55:07.955 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.937+0000 7fd209fc1c00 1 -- 172.21.15.192:0/130657301 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562db0d95b20 msgr2=0x562db0d90660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:07.956 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.937+0000 7fd209fc1c00 1 --2- 172.21.15.192:0/130657301 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562db0d95b20 0x562db0d90660 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 rx=0x7fd1f0002f30 tx=0x7fd1f0015000).stop 2020-11-28T21:55:07.956 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.937+0000 7fd209fc1c00 1 -- 172.21.15.192:0/130657301 shutdown_connections 2020-11-28T21:55:07.956 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.937+0000 7fd209fc1c00 1 --2- 172.21.15.192:0/130657301 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x562db0d9f0e0 0x562db0daba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:07.956 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.937+0000 7fd209fc1c00 1 --2- 172.21.15.192:0/130657301 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562db0d95b20 0x562db0d90660 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:07.956 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.937+0000 7fd209fc1c00 1 --2- 172.21.15.192:0/130657301 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562db0d9c800 0x562db0d90b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:07.957 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.937+0000 7fd209fc1c00 1 -- 172.21.15.192:0/130657301 >> 172.21.15.192:0/130657301 conn(0x562db0d8f270 msgr2=0x562db0d9b040 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:07.957 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.937+0000 7fd209fc1c00 1 -- 172.21.15.192:0/130657301 shutdown_connections 2020-11-28T21:55:07.957 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.937+0000 7fd209fc1c00 1 -- 172.21.15.192:0/130657301 wait complete. 2020-11-28T21:55:07.957 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.937+0000 7fd209fc1c00 1 Processor -- start 2020-11-28T21:55:07.957 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.937+0000 7fd209fc1c00 1 -- start start 2020-11-28T21:55:07.958 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.937+0000 7fd209fc1c00 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562db0d95b20 0x562db0dab5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:07.958 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.937+0000 7fd209fc1c00 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x562db0d9c800 0x562db0daba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:07.958 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.937+0000 7fd209fc1c00 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562db0d9f0e0 0x562db0da68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:07.958 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.937+0000 7fd209fc1c00 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x562db0da0430 con 0x562db0d9f0e0 2020-11-28T21:55:07.959 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.937+0000 7fd209fc1c00 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x562db0da7010 con 0x562db0d95b20 2020-11-28T21:55:07.959 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.937+0000 7fd209fc1c00 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x562db0da71d0 con 0x562db0d9c800 2020-11-28T21:55:07.959 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.937+0000 7fd1fbeb5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562db0d95b20 0x562db0dab5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:07.959 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.937+0000 7fd1fbeb5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562db0d95b20 0x562db0dab5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.192:50332/0 (socket says 172.21.15.192:50332) 2020-11-28T21:55:07.959 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.937+0000 7fd1fbeb5700 1 -- 172.21.15.192:0/3190315684 learned_addr learned my addr 172.21.15.192:0/3190315684 (peer_addr_for_me v2:172.21.15.192:0/0) 2020-11-28T21:55:07.960 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.937+0000 7fd1fc6b6700 1 --2- 172.21.15.192:0/3190315684 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562db0d9f0e0 0x562db0da68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:07.960 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.937+0000 7fd1fb6b4700 1 --2- 172.21.15.192:0/3190315684 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x562db0d9c800 0x562db0daba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:07.960 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.937+0000 7fd1fbeb5700 1 -- 172.21.15.192:0/3190315684 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x562db0d9c800 msgr2=0x562db0daba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:07.960 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.937+0000 7fd1fbeb5700 1 --2- 172.21.15.192:0/3190315684 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x562db0d9c800 0x562db0daba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:07.961 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.937+0000 7fd1fbeb5700 1 -- 172.21.15.192:0/3190315684 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562db0d9f0e0 msgr2=0x562db0da68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:07.961 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.937+0000 7fd1fbeb5700 1 --2- 172.21.15.192:0/3190315684 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562db0d9f0e0 0x562db0da68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:07.961 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.937+0000 7fd1fbeb5700 1 -- 172.21.15.192:0/3190315684 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562db0cab740 con 0x562db0d95b20 2020-11-28T21:55:07.961 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.937+0000 7fd1fbeb5700 1 --2- 172.21.15.192:0/3190315684 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562db0d95b20 0x562db0dab5a0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 rx=0x7fd1f0002f30 tx=0x7fd1f001c000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:07.963 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.937+0000 7fd1f96b0700 1 -- 172.21.15.192:0/3190315684 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1f00052f0 con 0x562db0d95b20 2020-11-28T21:55:07.964 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.937+0000 7fd1f96b0700 1 -- 172.21.15.192:0/3190315684 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f0012ab0 con 0x562db0d95b20 2020-11-28T21:55:07.964 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.937+0000 7fd1f96b0700 1 -- 172.21.15.192:0/3190315684 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1f0012e10 con 0x562db0d95b20 2020-11-28T21:55:07.964 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.937+0000 7fd209fc1c00 1 -- 172.21.15.192:0/3190315684 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562db0c811e0 con 0x562db0d95b20 2020-11-28T21:55:07.965 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.937+0000 7fd209fc1c00 1 -- 172.21.15.192:0/3190315684 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562db0db3100 con 0x562db0d95b20 2020-11-28T21:55:07.965 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.941+0000 7fd1f96b0700 1 -- 172.21.15.192:0/3190315684 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd1f001a040 con 0x562db0d95b20 2020-11-28T21:55:07.966 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.941+0000 7fd1f96b0700 1 --2- 172.21.15.192:0/3190315684 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd1e8053fe0 0x7fd1e80564a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:07.966 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.941+0000 7fd1f96b0700 1 -- 172.21.15.192:0/3190315684 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6075+0+0 (secure 0 0 0) 0x7fd1f0070410 con 0x562db0d95b20 2020-11-28T21:55:07.967 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.941+0000 7fd209fc1c00 1 --2- 172.21.15.192:0/3190315684 >> [v2:172.21.15.118:6824/12530,v1:172.21.15.118:6825/12530] conn(0x562db0db3b20 0x562db0db5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:07.967 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.941+0000 7fd209fc1c00 1 -- 172.21.15.192:0/3190315684 --> [v2:172.21.15.118:6824/12530,v1:172.21.15.118:6825/12530] -- osd_op(unknown.0.0:1 23.0 23:0620e6f9:::key:head [writefull 0~617 in=617b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x562db0db6400 con 0x562db0db3b20 2020-11-28T21:55:07.967 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.941+0000 7fd1fb6b4700 1 --2- 172.21.15.192:0/3190315684 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd1e8053fe0 0x7fd1e80564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:07.968 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.941+0000 7fd1fc6b6700 1 --2- 172.21.15.192:0/3190315684 >> [v2:172.21.15.118:6824/12530,v1:172.21.15.118:6825/12530] conn(0x562db0db3b20 0x562db0db5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:07.968 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.941+0000 7fd1fb6b4700 1 --2- 172.21.15.192:0/3190315684 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd1e8053fe0 0x7fd1e80564a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7fd1ec007000 tx=0x7fd1ec007040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:07.969 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:07.941+0000 7fd1fc6b6700 1 --2- 172.21.15.192:0/3190315684 >> [v2:172.21.15.118:6824/12530,v1:172.21.15.118:6825/12530] conn(0x562db0db3b20 0x562db0db5f10 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:08.964 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:55:08.966 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:55:08.969 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:55:08.971 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:55:09.506 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:09.505+0000 7fd1f96b0700 1 -- 172.21.15.192:0/3190315684 <== osd.6 v2:172.21.15.118:6824/12530 1 ==== osd_map(125..125 src has 1..125) v4 ==== 320+0+0 (crc 0 0 0) 0x7fd1f4009dd0 con 0x562db0db3b20 2020-11-28T21:55:09.512 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:09.509+0000 7fd1fc6b6700 1 -- 172.21.15.192:0/3190315684 <== osd.6 v2:172.21.15.118:6824/12530 2 ==== osd_op_reply(1 key [writefull 0~617] v125'1 uv1 ondisk = 0) v8 ==== 147+0+0 (crc 0 0 0) 0x7fd1f4012040 con 0x562db0db3b20 2020-11-28T21:55:09.512 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:09.509+0000 7fd209fc1c00 1 -- 172.21.15.192:0/3190315684 >> [v2:172.21.15.118:6824/12530,v1:172.21.15.118:6825/12530] conn(0x562db0db3b20 msgr2=0x562db0db5f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:09.512 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:09.509+0000 7fd209fc1c00 1 --2- 172.21.15.192:0/3190315684 >> [v2:172.21.15.118:6824/12530,v1:172.21.15.118:6825/12530] conn(0x562db0db3b20 0x562db0db5f10 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:09.512 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:09.509+0000 7fd209fc1c00 1 -- 172.21.15.192:0/3190315684 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd1e8053fe0 msgr2=0x7fd1e80564a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:09.513 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:09.509+0000 7fd209fc1c00 1 --2- 172.21.15.192:0/3190315684 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd1e8053fe0 0x7fd1e80564a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7fd1ec007000 tx=0x7fd1ec007040).stop 2020-11-28T21:55:09.513 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:09.509+0000 7fd209fc1c00 1 -- 172.21.15.192:0/3190315684 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562db0d95b20 msgr2=0x562db0dab5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:09.513 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:09.509+0000 7fd209fc1c00 1 --2- 172.21.15.192:0/3190315684 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562db0d95b20 0x562db0dab5a0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 rx=0x7fd1f0002f30 tx=0x7fd1f001c000).stop 2020-11-28T21:55:09.513 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:09.509+0000 7fd1fc6b6700 1 -- 172.21.15.192:0/3190315684 reap_dead start 2020-11-28T21:55:09.513 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:09.509+0000 7fd209fc1c00 1 -- 172.21.15.192:0/3190315684 shutdown_connections 2020-11-28T21:55:09.514 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:09.509+0000 7fd209fc1c00 1 -- 172.21.15.192:0/3190315684 >> 172.21.15.192:0/3190315684 conn(0x562db0d8f270 msgr2=0x562db0da0930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:09.514 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:09.509+0000 7fd209fc1c00 1 -- 172.21.15.192:0/3190315684 shutdown_connections 2020-11-28T21:55:09.514 INFO:teuthology.orchestra.run.smithi192.stderr:2020-11-28T21:55:09.509+0000 7fd209fc1c00 1 -- 172.21.15.192:0/3190315684 wait complete. 2020-11-28T21:55:09.516 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early df --format=json-pretty 2020-11-28T21:55:09.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.627+0000 7fee4f22f700 1 Processor -- start 2020-11-28T21:55:09.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.627+0000 7fee4f22f700 1 -- start start 2020-11-28T21:55:09.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.627+0000 7fee4f22f700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fee480b19f0 0x7fee48142a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:09.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.627+0000 7fee4f22f700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fee480b3230 0x7fee48142f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:09.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.627+0000 7fee4f22f700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee48140440 con 0x7fee48060e50 2020-11-28T21:55:09.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.627+0000 7fee4f22f700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee480b7160 con 0x7fee480b19f0 2020-11-28T21:55:09.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.627+0000 7fee4f22f700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee480b7400 con 0x7fee480b3230 2020-11-28T21:55:09.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.627+0000 7fee4cfcb700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fee48060e50 0x7fee480b2880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51302/0 (socket says 172.21.15.162:51302) 2020-11-28T21:55:09.639 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.627+0000 7fee4cfcb700 1 -- 172.21.15.162:0/2286583257 learned_addr learned my addr 172.21.15.162:0/2286583257 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:09.639 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.627+0000 7fee4d7cc700 1 --2- 172.21.15.162:0/2286583257 >> v2:172.21.15.162:3300/0 conn(0x7fee480b3230 0x7fee48142f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:09.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.627+0000 7fee47fff700 1 --2- 172.21.15.162:0/2286583257 >> v2:172.21.15.118:3300/0 conn(0x7fee480b19f0 0x7fee48142a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:09.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.627+0000 7fee477fe700 1 -- 172.21.15.162:0/2286583257 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2311904712 0 0) 0x7fee38001450 con 0x7fee48060e50 2020-11-28T21:55:09.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.627+0000 7fee477fe700 1 -- 172.21.15.162:0/2286583257 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee34003230 con 0x7fee48060e50 2020-11-28T21:55:09.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.627+0000 7fee4d7cc700 1 -- 172.21.15.162:0/2286583257 >> v2:172.21.15.118:3300/0 conn(0x7fee480b19f0 msgr2=0x7fee48142a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:09.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.627+0000 7fee4d7cc700 1 --2- 172.21.15.162:0/2286583257 >> v2:172.21.15.118:3300/0 conn(0x7fee480b19f0 0x7fee48142a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:09.642 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.627+0000 7fee4d7cc700 1 -- 172.21.15.162:0/2286583257 >> v1:172.21.15.162:6789/0 conn(0x7fee48060e50 legacy=0x7fee480b2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:09.642 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.627+0000 7fee4d7cc700 1 -- 172.21.15.162:0/2286583257 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee480b76a0 con 0x7fee480b3230 2020-11-28T21:55:09.643 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.627+0000 7fee4d7cc700 1 --2- 172.21.15.162:0/2286583257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee480b3230 0x7fee48142f00 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 rx=0x7fee48060290 tx=0x7fee40006dc0).ready entity=mon.1 client_cookie=973641ddf9660ad2 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:09.643 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.631+0000 7fee477fe700 1 -- 172.21.15.162:0/2286583257 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee40009770 con 0x7fee480b3230 2020-11-28T21:55:09.643 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.631+0000 7fee477fe700 1 -- 172.21.15.162:0/2286583257 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee400098d0 con 0x7fee480b3230 2020-11-28T21:55:09.644 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.631+0000 7fee477fe700 1 -- 172.21.15.162:0/2286583257 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee40009c30 con 0x7fee480b3230 2020-11-28T21:55:09.644 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.631+0000 7fee4f22f700 1 -- 172.21.15.162:0/2286583257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee480b3230 msgr2=0x7fee48142f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:09.645 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.631+0000 7fee4f22f700 1 --2- 172.21.15.162:0/2286583257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee480b3230 0x7fee48142f00 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 rx=0x7fee48060290 tx=0x7fee40006dc0).stop 2020-11-28T21:55:09.646 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.631+0000 7fee4f22f700 1 -- 172.21.15.162:0/2286583257 shutdown_connections 2020-11-28T21:55:09.646 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.631+0000 7fee4f22f700 1 --2- 172.21.15.162:0/2286583257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee480b3230 0x7fee48142f00 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:09.646 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.631+0000 7fee4f22f700 1 --2- 172.21.15.162:0/2286583257 >> v2:172.21.15.118:3300/0 conn(0x7fee480b19f0 0x7fee48142a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:09.646 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.631+0000 7fee4f22f700 1 -- 172.21.15.162:0/2286583257 >> 172.21.15.162:0/2286583257 conn(0x7fee48062830 msgr2=0x7fee48062c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:09.647 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.631+0000 7fee4f22f700 1 -- 172.21.15.162:0/2286583257 shutdown_connections 2020-11-28T21:55:09.647 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.631+0000 7fee4f22f700 1 -- 172.21.15.162:0/2286583257 wait complete. 2020-11-28T21:55:09.647 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.631+0000 7fee4f22f700 1 Processor -- start 2020-11-28T21:55:09.647 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.631+0000 7fee4f22f700 1 -- start start 2020-11-28T21:55:09.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.631+0000 7fee4f22f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee48060e50 0x7fee480b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:09.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.631+0000 7fee4f22f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee480b19f0 0x7fee480acb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:09.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.631+0000 7fee4f22f700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee480b3230 0x7fee480ad070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:09.651 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.631+0000 7fee4f22f700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee480ac070 con 0x7fee480b3230 2020-11-28T21:55:09.651 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.631+0000 7fee4f22f700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee480ac720 con 0x7fee48060e50 2020-11-28T21:55:09.651 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.631+0000 7fee4f22f700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee480abdd0 con 0x7fee480b19f0 2020-11-28T21:55:09.651 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.631+0000 7fee4cfcb700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee48060e50 0x7fee480b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:09.651 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.631+0000 7fee4cfcb700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee48060e50 0x7fee480b57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44464/0 (socket says 172.21.15.162:44464) 2020-11-28T21:55:09.652 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.631+0000 7fee4cfcb700 1 -- 172.21.15.162:0/2891604849 learned_addr learned my addr 172.21.15.162:0/2891604849 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:09.652 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.631+0000 7fee4d7cc700 1 --2- 172.21.15.162:0/2891604849 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee480b3230 0x7fee480ad070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:09.652 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.631+0000 7fee47fff700 1 --2- 172.21.15.162:0/2891604849 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee480b19f0 0x7fee480acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:09.652 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.635+0000 7fee4cfcb700 1 -- 172.21.15.162:0/2891604849 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee480b19f0 msgr2=0x7fee480acb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:09.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.635+0000 7fee4cfcb700 1 --2- 172.21.15.162:0/2891604849 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee480b19f0 0x7fee480acb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:09.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.635+0000 7fee4cfcb700 1 -- 172.21.15.162:0/2891604849 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee480b3230 msgr2=0x7fee480ad070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:09.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.635+0000 7fee4cfcb700 1 --2- 172.21.15.162:0/2891604849 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee480b3230 0x7fee480ad070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:09.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.635+0000 7fee4cfcb700 1 -- 172.21.15.162:0/2891604849 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee48146040 con 0x7fee48060e50 2020-11-28T21:55:09.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.635+0000 7fee47fff700 1 --2- 172.21.15.162:0/2891604849 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee480b19f0 0x7fee480acb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:55:09.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.635+0000 7fee4cfcb700 1 --2- 172.21.15.162:0/2891604849 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee48060e50 0x7fee480b57a0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 rx=0x7fee380015e0 tx=0x7fee38007c90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:09.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.635+0000 7fee45ffb700 1 -- 172.21.15.162:0/2891604849 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee3800ce70 con 0x7fee48060e50 2020-11-28T21:55:09.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.635+0000 7fee45ffb700 1 -- 172.21.15.162:0/2891604849 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee38005b70 con 0x7fee48060e50 2020-11-28T21:55:09.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.635+0000 7fee4f22f700 1 -- 172.21.15.162:0/2891604849 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee481441d0 con 0x7fee48060e50 2020-11-28T21:55:09.655 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.635+0000 7fee45ffb700 1 -- 172.21.15.162:0/2891604849 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee38007110 con 0x7fee48060e50 2020-11-28T21:55:09.655 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.635+0000 7fee4f22f700 1 -- 172.21.15.162:0/2891604849 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee480aae20 con 0x7fee48060e50 2020-11-28T21:55:09.655 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.635+0000 7fee4f22f700 1 -- 172.21.15.162:0/2891604849 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee48057050 con 0x7fee48060e50 2020-11-28T21:55:09.655 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.635+0000 7fee45ffb700 1 -- 172.21.15.162:0/2891604849 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fee38010030 con 0x7fee48060e50 2020-11-28T21:55:09.655 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.635+0000 7fee45ffb700 1 --2- 172.21.15.162:0/2891604849 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fee280553b0 0x7fee28057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:09.656 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.639+0000 7fee47fff700 1 --2- 172.21.15.162:0/2891604849 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fee280553b0 0x7fee28057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:09.656 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.639+0000 7fee45ffb700 1 -- 172.21.15.162:0/2891604849 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6075+0+0 (secure 0 0 0) 0x7fee3806de80 con 0x7fee48060e50 2020-11-28T21:55:09.656 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.639+0000 7fee47fff700 1 --2- 172.21.15.162:0/2891604849 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fee280553b0 0x7fee28057870 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7fee3c00c000 tx=0x7fee3c008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:09.656 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.639+0000 7fee45ffb700 1 -- 172.21.15.162:0/2891604849 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fee38044070 con 0x7fee48060e50 2020-11-28T21:55:09.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.831+0000 7fee4f22f700 1 -- 172.21.15.162:0/2891604849 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7fee48140440 con 0x7fee48060e50 2020-11-28T21:55:09.838 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.835+0000 7fee45ffb700 1 -- 172.21.15.162:0/2891604849 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1588 (secure 0 0 0) 0x7fee48140440 con 0x7fee48060e50 2020-11-28T21:55:09.838 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:09.839 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2020-11-28T21:55:09.839 INFO:teuthology.orchestra.run.smithi162.stdout: "stats": { 2020-11-28T21:55:09.839 INFO:teuthology.orchestra.run.smithi162.stdout: "total_bytes": 773094113280, 2020-11-28T21:55:09.839 INFO:teuthology.orchestra.run.smithi162.stdout: "total_avail_bytes": 764491964416, 2020-11-28T21:55:09.839 INFO:teuthology.orchestra.run.smithi162.stdout: "total_used_bytes": 12214272, 2020-11-28T21:55:09.840 INFO:teuthology.orchestra.run.smithi162.stdout: "total_used_raw_bytes": 8602148864, 2020-11-28T21:55:09.840 INFO:teuthology.orchestra.run.smithi162.stdout: "total_used_raw_ratio": 0.011126910336315632, 2020-11-28T21:55:09.840 INFO:teuthology.orchestra.run.smithi162.stdout: "num_osds": 8, 2020-11-28T21:55:09.840 INFO:teuthology.orchestra.run.smithi162.stdout: "num_per_pool_osds": 8, 2020-11-28T21:55:09.840 INFO:teuthology.orchestra.run.smithi162.stdout: "num_per_pool_omap_osds": 8 2020-11-28T21:55:09.841 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:09.841 INFO:teuthology.orchestra.run.smithi162.stdout: "stats_by_class": { 2020-11-28T21:55:09.841 INFO:teuthology.orchestra.run.smithi162.stdout: "ssd": { 2020-11-28T21:55:09.841 INFO:teuthology.orchestra.run.smithi162.stdout: "total_bytes": 773094113280, 2020-11-28T21:55:09.842 INFO:teuthology.orchestra.run.smithi162.stdout: "total_avail_bytes": 764491964416, 2020-11-28T21:55:09.842 INFO:teuthology.orchestra.run.smithi162.stdout: "total_used_bytes": 12214272, 2020-11-28T21:55:09.842 INFO:teuthology.orchestra.run.smithi162.stdout: "total_used_raw_bytes": 8602148864, 2020-11-28T21:55:09.842 INFO:teuthology.orchestra.run.smithi162.stdout: "total_used_raw_ratio": 0.011126910336315632 2020-11-28T21:55:09.842 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:09.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:09.843 INFO:teuthology.orchestra.run.smithi162.stdout: "pools": [ 2020-11-28T21:55:09.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:09.843 INFO:teuthology.orchestra.run.smithi162.stdout: "name": "device_health_metrics", 2020-11-28T21:55:09.843 INFO:teuthology.orchestra.run.smithi162.stdout: "id": 1, 2020-11-28T21:55:09.844 INFO:teuthology.orchestra.run.smithi162.stdout: "stats": { 2020-11-28T21:55:09.844 INFO:teuthology.orchestra.run.smithi162.stdout: "stored": 0, 2020-11-28T21:55:09.844 INFO:teuthology.orchestra.run.smithi162.stdout: "objects": 2, 2020-11-28T21:55:09.844 INFO:teuthology.orchestra.run.smithi162.stdout: "kb_used": 0, 2020-11-28T21:55:09.844 INFO:teuthology.orchestra.run.smithi162.stdout: "bytes_used": 0, 2020-11-28T21:55:09.845 INFO:teuthology.orchestra.run.smithi162.stdout: "percent_used": 0, 2020-11-28T21:55:09.845 INFO:teuthology.orchestra.run.smithi162.stdout: "max_avail": 362918510592 2020-11-28T21:55:09.845 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:09.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:09.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:09.846 INFO:teuthology.orchestra.run.smithi162.stdout: "name": "rbd", 2020-11-28T21:55:09.846 INFO:teuthology.orchestra.run.smithi162.stdout: "id": 2, 2020-11-28T21:55:09.846 INFO:teuthology.orchestra.run.smithi162.stdout: "stats": { 2020-11-28T21:55:09.846 INFO:teuthology.orchestra.run.smithi162.stdout: "stored": 0, 2020-11-28T21:55:09.846 INFO:teuthology.orchestra.run.smithi162.stdout: "objects": 0, 2020-11-28T21:55:09.847 INFO:teuthology.orchestra.run.smithi162.stdout: "kb_used": 0, 2020-11-28T21:55:09.847 INFO:teuthology.orchestra.run.smithi162.stdout: "bytes_used": 0, 2020-11-28T21:55:09.847 INFO:teuthology.orchestra.run.smithi162.stdout: "percent_used": 0, 2020-11-28T21:55:09.847 INFO:teuthology.orchestra.run.smithi162.stdout: "max_avail": 362918510592 2020-11-28T21:55:09.847 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:09.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:09.848 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:09.848 INFO:teuthology.orchestra.run.smithi162.stdout: "name": "cephfs_data", 2020-11-28T21:55:09.848 INFO:teuthology.orchestra.run.smithi162.stdout: "id": 22, 2020-11-28T21:55:09.848 INFO:teuthology.orchestra.run.smithi162.stdout: "stats": { 2020-11-28T21:55:09.849 INFO:teuthology.orchestra.run.smithi162.stdout: "stored": 0, 2020-11-28T21:55:09.849 INFO:teuthology.orchestra.run.smithi162.stdout: "objects": 0, 2020-11-28T21:55:09.849 INFO:teuthology.orchestra.run.smithi162.stdout: "kb_used": 0, 2020-11-28T21:55:09.849 INFO:teuthology.orchestra.run.smithi162.stdout: "bytes_used": 0, 2020-11-28T21:55:09.849 INFO:teuthology.orchestra.run.smithi162.stdout: "percent_used": 0, 2020-11-28T21:55:09.850 INFO:teuthology.orchestra.run.smithi162.stdout: "max_avail": 362918510592 2020-11-28T21:55:09.850 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:09.850 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:09.850 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:09.850 INFO:teuthology.orchestra.run.smithi162.stdout:} 2020-11-28T21:55:09.852 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.835+0000 7fee4f22f700 1 -- 172.21.15.162:0/2891604849 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fee280553b0 msgr2=0x7fee28057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:09.852 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.835+0000 7fee4f22f700 1 --2- 172.21.15.162:0/2891604849 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fee280553b0 0x7fee28057870 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7fee3c00c000 tx=0x7fee3c008040).stop 2020-11-28T21:55:09.852 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.835+0000 7fee4f22f700 1 -- 172.21.15.162:0/2891604849 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee48060e50 msgr2=0x7fee480b57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:09.852 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.835+0000 7fee4f22f700 1 --2- 172.21.15.162:0/2891604849 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee48060e50 0x7fee480b57a0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 rx=0x7fee380015e0 tx=0x7fee38007c90).stop 2020-11-28T21:55:09.853 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.835+0000 7fee4f22f700 1 -- 172.21.15.162:0/2891604849 shutdown_connections 2020-11-28T21:55:09.853 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.835+0000 7fee4f22f700 1 --2- 172.21.15.162:0/2891604849 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fee280553b0 0x7fee28057870 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:09.853 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.835+0000 7fee4f22f700 1 --2- 172.21.15.162:0/2891604849 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee480b3230 0x7fee480ad070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:09.853 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.835+0000 7fee4f22f700 1 --2- 172.21.15.162:0/2891604849 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee48060e50 0x7fee480b57a0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:09.853 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.835+0000 7fee4f22f700 1 --2- 172.21.15.162:0/2891604849 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee480b19f0 0x7fee480acb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:09.854 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.835+0000 7fee4f22f700 1 -- 172.21.15.162:0/2891604849 >> 172.21.15.162:0/2891604849 conn(0x7fee48062830 msgr2=0x7fee480b3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:09.854 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.835+0000 7fee4f22f700 1 -- 172.21.15.162:0/2891604849 shutdown_connections 2020-11-28T21:55:09.854 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:09.835+0000 7fee4f22f700 1 -- 172.21.15.162:0/2891604849 wait complete. 2020-11-28T21:55:09.858 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30)... 2020-11-28T21:55:14.859 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early df --format=json-pretty 2020-11-28T21:55:14.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.975+0000 7f12cf59e700 1 Processor -- start 2020-11-28T21:55:14.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.975+0000 7f12cf59e700 1 -- start start 2020-11-28T21:55:14.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.975+0000 7f12cf59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12d00615c0 0x7f12d00b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:14.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.975+0000 7f12cf59e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12d00b1790 con 0x7f12d00611e0 2020-11-28T21:55:14.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.975+0000 7f12cf59e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12d00b1a30 con 0x7f12d00b4b40 2020-11-28T21:55:14.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.975+0000 7f12cf59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d00b3f90 con 0x7f12d00615c0 2020-11-28T21:55:14.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.975+0000 7f12ced9d700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f12d00b4b40 0x7f12d013ee60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51320/0 (socket says 172.21.15.162:51320) 2020-11-28T21:55:14.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.975+0000 7f12ced9d700 1 -- 172.21.15.162:0/3112841061 learned_addr learned my addr 172.21.15.162:0/3112841061 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:14.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.975+0000 7f12c7fff700 1 --2- 172.21.15.162:0/3112841061 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12d00615c0 0x7f12d00b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:14.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.975+0000 7f12cdd9b700 1 -- 172.21.15.162:0/3112841061 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1026281399 0 0) 0x7f12d00b1a30 con 0x7f12d00b4b40 2020-11-28T21:55:14.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.975+0000 7f12cdd9b700 1 -- 172.21.15.162:0/3112841061 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12b0003230 con 0x7f12d00b4b40 2020-11-28T21:55:14.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.975+0000 7f12cdd9b700 1 -- 172.21.15.162:0/3112841061 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2494398731 0 0) 0x7f12d00b1790 con 0x7f12d00611e0 2020-11-28T21:55:14.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.975+0000 7f12cdd9b700 1 -- 172.21.15.162:0/3112841061 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12d00b1a30 con 0x7f12d00611e0 2020-11-28T21:55:14.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.975+0000 7f12c7fff700 1 -- 172.21.15.162:0/3112841061 >> v1:172.21.15.162:6789/0 conn(0x7f12d00b4b40 legacy=0x7f12d013ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:14.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.975+0000 7f12c7fff700 1 -- 172.21.15.162:0/3112841061 >> v1:172.21.15.118:6789/0 conn(0x7f12d00611e0 legacy=0x7f12d00b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:14.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.975+0000 7f12c7fff700 1 -- 172.21.15.162:0/3112841061 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d00b4fb0 con 0x7f12d00615c0 2020-11-28T21:55:14.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.975+0000 7f12c7fff700 1 --2- 172.21.15.162:0/3112841061 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12d00615c0 0x7f12d00b10f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f12b8002f30 tx=0x7f12b8006dc0).ready entity=mon.2 client_cookie=7bb4ff0760d455ce server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:14.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.975+0000 7f12cdd9b700 1 -- 172.21.15.162:0/3112841061 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12b8009730 con 0x7f12d00615c0 2020-11-28T21:55:14.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.975+0000 7f12cdd9b700 1 -- 172.21.15.162:0/3112841061 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b8009890 con 0x7f12d00615c0 2020-11-28T21:55:14.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.975+0000 7f12cdd9b700 1 -- 172.21.15.162:0/3112841061 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12b8009bf0 con 0x7f12d00615c0 2020-11-28T21:55:14.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.975+0000 7f12cf59e700 1 -- 172.21.15.162:0/3112841061 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12d00615c0 msgr2=0x7f12d00b10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:14.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.975+0000 7f12cf59e700 1 --2- 172.21.15.162:0/3112841061 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12d00615c0 0x7f12d00b10f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f12b8002f30 tx=0x7f12b8006dc0).stop 2020-11-28T21:55:14.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.975+0000 7f12cf59e700 1 -- 172.21.15.162:0/3112841061 shutdown_connections 2020-11-28T21:55:14.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.975+0000 7f12cf59e700 1 --2- 172.21.15.162:0/3112841061 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12d00615c0 0x7f12d00b10f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:14.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.975+0000 7f12cf59e700 1 -- 172.21.15.162:0/3112841061 >> 172.21.15.162:0/3112841061 conn(0x7f12d00b54f0 msgr2=0x7f12d00b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:14.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.979+0000 7f12cf59e700 1 -- 172.21.15.162:0/3112841061 shutdown_connections 2020-11-28T21:55:14.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.979+0000 7f12cf59e700 1 -- 172.21.15.162:0/3112841061 wait complete. 2020-11-28T21:55:14.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.979+0000 7f12cf59e700 1 Processor -- start 2020-11-28T21:55:14.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.979+0000 7f12cf59e700 1 -- start start 2020-11-28T21:55:14.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.979+0000 7f12cf59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12d00611e0 0x7f12d012e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:14.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.979+0000 7f12cf59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12d00615c0 0x7f12d00ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:14.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.979+0000 7f12cf59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f12d00b4b40 0x7f12d00650e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:14.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.979+0000 7f12cf59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d0064600 con 0x7f12d00b4b40 2020-11-28T21:55:14.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.979+0000 7f12cf59e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d0099000 con 0x7f12d00611e0 2020-11-28T21:55:14.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.979+0000 7f12cf59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d0099230 con 0x7f12d00615c0 2020-11-28T21:55:14.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.979+0000 7f12ce59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12d00611e0 0x7f12d012e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:14.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.979+0000 7f12ce59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12d00611e0 0x7f12d012e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44478/0 (socket says 172.21.15.162:44478) 2020-11-28T21:55:14.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.979+0000 7f12ce59c700 1 -- 172.21.15.162:0/3900716769 learned_addr learned my addr 172.21.15.162:0/3900716769 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:14.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.979+0000 7f12ced9d700 1 --2- 172.21.15.162:0/3900716769 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f12d00b4b40 0x7f12d00650e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:14.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.979+0000 7f12c7fff700 1 --2- 172.21.15.162:0/3900716769 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12d00615c0 0x7f12d00ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:14.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.979+0000 7f12ce59c700 1 -- 172.21.15.162:0/3900716769 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12d00615c0 msgr2=0x7f12d00ae8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:14.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.979+0000 7f12ce59c700 1 --2- 172.21.15.162:0/3900716769 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12d00615c0 0x7f12d00ae8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:14.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.979+0000 7f12ce59c700 1 -- 172.21.15.162:0/3900716769 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f12d00b4b40 msgr2=0x7f12d00650e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:14.992 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.979+0000 7f12ce59c700 1 --2- 172.21.15.162:0/3900716769 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f12d00b4b40 0x7f12d00650e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:14.992 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.979+0000 7f12ce59c700 1 -- 172.21.15.162:0/3900716769 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d00994d0 con 0x7f12d00611e0 2020-11-28T21:55:14.992 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.979+0000 7f12ce59c700 1 --2- 172.21.15.162:0/3900716769 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12d00611e0 0x7f12d012e670 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 rx=0x7f12c00010d0 tx=0x7f12c0007ab0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:14.992 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.979+0000 7f12c77fe700 1 -- 172.21.15.162:0/3900716769 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12c000f470 con 0x7f12d00611e0 2020-11-28T21:55:14.993 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.979+0000 7f12c77fe700 1 -- 172.21.15.162:0/3900716769 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c000f5d0 con 0x7f12d00611e0 2020-11-28T21:55:14.993 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.979+0000 7f12cf59e700 1 -- 172.21.15.162:0/3900716769 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12d00af390 con 0x7f12d00611e0 2020-11-28T21:55:14.993 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.979+0000 7f12c77fe700 1 -- 172.21.15.162:0/3900716769 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12c000f930 con 0x7f12d00611e0 2020-11-28T21:55:14.993 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.979+0000 7f12cf59e700 1 -- 172.21.15.162:0/3900716769 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12d0067b50 con 0x7f12d00611e0 2020-11-28T21:55:14.994 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.983+0000 7f12cf59e700 1 -- 172.21.15.162:0/3900716769 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12d012e380 con 0x7f12d00611e0 2020-11-28T21:55:14.994 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.983+0000 7f12c77fe700 1 -- 172.21.15.162:0/3900716769 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f12c0019030 con 0x7f12d00611e0 2020-11-28T21:55:14.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.983+0000 7f12c77fe700 1 --2- 172.21.15.162:0/3900716769 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f12b40554e0 0x7f12b40579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:14.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.983+0000 7f12c77fe700 1 -- 172.21.15.162:0/3900716769 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6075+0+0 (secure 0 0 0) 0x7f12c006d480 con 0x7f12d00611e0 2020-11-28T21:55:14.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.983+0000 7f12c7fff700 1 --2- 172.21.15.162:0/3900716769 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f12b40554e0 0x7f12b40579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:14.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.983+0000 7f12c7fff700 1 --2- 172.21.15.162:0/3900716769 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f12b40554e0 0x7f12b40579a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f12b801b000 tx=0x7f12b8015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:14.996 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:14.991+0000 7f12c77fe700 1 -- 172.21.15.162:0/3900716769 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f12c00436e0 con 0x7f12d00611e0 2020-11-28T21:55:15.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.183+0000 7f12cf59e700 1 -- 172.21.15.162:0/3900716769 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7f12d0057050 con 0x7f12d00611e0 2020-11-28T21:55:15.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.183+0000 7f12c77fe700 1 -- 172.21.15.162:0/3900716769 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1932 (secure 0 0 0) 0x7f12c0007bd0 con 0x7f12d00611e0 2020-11-28T21:55:15.188 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:15.188 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2020-11-28T21:55:15.188 INFO:teuthology.orchestra.run.smithi162.stdout: "stats": { 2020-11-28T21:55:15.188 INFO:teuthology.orchestra.run.smithi162.stdout: "total_bytes": 773094113280, 2020-11-28T21:55:15.189 INFO:teuthology.orchestra.run.smithi162.stdout: "total_avail_bytes": 764491591680, 2020-11-28T21:55:15.189 INFO:teuthology.orchestra.run.smithi162.stdout: "total_used_bytes": 12587008, 2020-11-28T21:55:15.189 INFO:teuthology.orchestra.run.smithi162.stdout: "total_used_raw_bytes": 8602521600, 2020-11-28T21:55:15.189 INFO:teuthology.orchestra.run.smithi162.stdout: "total_used_raw_ratio": 0.011127392761409283, 2020-11-28T21:55:15.190 INFO:teuthology.orchestra.run.smithi162.stdout: "num_osds": 8, 2020-11-28T21:55:15.190 INFO:teuthology.orchestra.run.smithi162.stdout: "num_per_pool_osds": 8, 2020-11-28T21:55:15.190 INFO:teuthology.orchestra.run.smithi162.stdout: "num_per_pool_omap_osds": 8 2020-11-28T21:55:15.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:15.190 INFO:teuthology.orchestra.run.smithi162.stdout: "stats_by_class": { 2020-11-28T21:55:15.191 INFO:teuthology.orchestra.run.smithi162.stdout: "ssd": { 2020-11-28T21:55:15.191 INFO:teuthology.orchestra.run.smithi162.stdout: "total_bytes": 773094113280, 2020-11-28T21:55:15.191 INFO:teuthology.orchestra.run.smithi162.stdout: "total_avail_bytes": 764491591680, 2020-11-28T21:55:15.191 INFO:teuthology.orchestra.run.smithi162.stdout: "total_used_bytes": 12587008, 2020-11-28T21:55:15.191 INFO:teuthology.orchestra.run.smithi162.stdout: "total_used_raw_bytes": 8602521600, 2020-11-28T21:55:15.192 INFO:teuthology.orchestra.run.smithi162.stdout: "total_used_raw_ratio": 0.011127392761409283 2020-11-28T21:55:15.192 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:15.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:15.192 INFO:teuthology.orchestra.run.smithi162.stdout: "pools": [ 2020-11-28T21:55:15.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:15.193 INFO:teuthology.orchestra.run.smithi162.stdout: "name": "device_health_metrics", 2020-11-28T21:55:15.193 INFO:teuthology.orchestra.run.smithi162.stdout: "id": 1, 2020-11-28T21:55:15.193 INFO:teuthology.orchestra.run.smithi162.stdout: "stats": { 2020-11-28T21:55:15.193 INFO:teuthology.orchestra.run.smithi162.stdout: "stored": 0, 2020-11-28T21:55:15.193 INFO:teuthology.orchestra.run.smithi162.stdout: "objects": 2, 2020-11-28T21:55:15.194 INFO:teuthology.orchestra.run.smithi162.stdout: "kb_used": 0, 2020-11-28T21:55:15.194 INFO:teuthology.orchestra.run.smithi162.stdout: "bytes_used": 0, 2020-11-28T21:55:15.194 INFO:teuthology.orchestra.run.smithi162.stdout: "percent_used": 0, 2020-11-28T21:55:15.194 INFO:teuthology.orchestra.run.smithi162.stdout: "max_avail": 362918412288 2020-11-28T21:55:15.195 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:15.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:15.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:15.195 INFO:teuthology.orchestra.run.smithi162.stdout: "name": "rbd", 2020-11-28T21:55:15.195 INFO:teuthology.orchestra.run.smithi162.stdout: "id": 2, 2020-11-28T21:55:15.196 INFO:teuthology.orchestra.run.smithi162.stdout: "stats": { 2020-11-28T21:55:15.196 INFO:teuthology.orchestra.run.smithi162.stdout: "stored": 0, 2020-11-28T21:55:15.196 INFO:teuthology.orchestra.run.smithi162.stdout: "objects": 0, 2020-11-28T21:55:15.196 INFO:teuthology.orchestra.run.smithi162.stdout: "kb_used": 0, 2020-11-28T21:55:15.196 INFO:teuthology.orchestra.run.smithi162.stdout: "bytes_used": 0, 2020-11-28T21:55:15.197 INFO:teuthology.orchestra.run.smithi162.stdout: "percent_used": 0, 2020-11-28T21:55:15.197 INFO:teuthology.orchestra.run.smithi162.stdout: "max_avail": 362918412288 2020-11-28T21:55:15.197 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:15.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:15.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:15.198 INFO:teuthology.orchestra.run.smithi162.stdout: "name": "cephfs_data", 2020-11-28T21:55:15.198 INFO:teuthology.orchestra.run.smithi162.stdout: "id": 22, 2020-11-28T21:55:15.198 INFO:teuthology.orchestra.run.smithi162.stdout: "stats": { 2020-11-28T21:55:15.198 INFO:teuthology.orchestra.run.smithi162.stdout: "stored": 0, 2020-11-28T21:55:15.199 INFO:teuthology.orchestra.run.smithi162.stdout: "objects": 0, 2020-11-28T21:55:15.199 INFO:teuthology.orchestra.run.smithi162.stdout: "kb_used": 0, 2020-11-28T21:55:15.199 INFO:teuthology.orchestra.run.smithi162.stdout: "bytes_used": 0, 2020-11-28T21:55:15.199 INFO:teuthology.orchestra.run.smithi162.stdout: "percent_used": 0, 2020-11-28T21:55:15.199 INFO:teuthology.orchestra.run.smithi162.stdout: "max_avail": 362918412288 2020-11-28T21:55:15.200 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:15.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:15.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:15.200 INFO:teuthology.orchestra.run.smithi162.stdout: "name": "cephfs_metadata", 2020-11-28T21:55:15.200 INFO:teuthology.orchestra.run.smithi162.stdout: "id": 23, 2020-11-28T21:55:15.201 INFO:teuthology.orchestra.run.smithi162.stdout: "stats": { 2020-11-28T21:55:15.201 INFO:teuthology.orchestra.run.smithi162.stdout: "stored": 617, 2020-11-28T21:55:15.201 INFO:teuthology.orchestra.run.smithi162.stdout: "objects": 1, 2020-11-28T21:55:15.201 INFO:teuthology.orchestra.run.smithi162.stdout: "kb_used": 8, 2020-11-28T21:55:15.202 INFO:teuthology.orchestra.run.smithi162.stdout: "bytes_used": 8192, 2020-11-28T21:55:15.202 INFO:teuthology.orchestra.run.smithi162.stdout: "percent_used": 1.1286283019273924e-08, 2020-11-28T21:55:15.202 INFO:teuthology.orchestra.run.smithi162.stdout: "max_avail": 362918412288 2020-11-28T21:55:15.202 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:15.203 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:15.203 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:15.203 INFO:teuthology.orchestra.run.smithi162.stdout:} 2020-11-28T21:55:15.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.183+0000 7f12cf59e700 1 -- 172.21.15.162:0/3900716769 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f12b40554e0 msgr2=0x7f12b40579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:15.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.183+0000 7f12cf59e700 1 --2- 172.21.15.162:0/3900716769 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f12b40554e0 0x7f12b40579a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f12b801b000 tx=0x7f12b8015000).stop 2020-11-28T21:55:15.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.183+0000 7f12cf59e700 1 -- 172.21.15.162:0/3900716769 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12d00611e0 msgr2=0x7f12d012e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:15.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.183+0000 7f12cf59e700 1 --2- 172.21.15.162:0/3900716769 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12d00611e0 0x7f12d012e670 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 rx=0x7f12c00010d0 tx=0x7f12c0007ab0).stop 2020-11-28T21:55:15.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.187+0000 7f12cf59e700 1 -- 172.21.15.162:0/3900716769 shutdown_connections 2020-11-28T21:55:15.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.187+0000 7f12cf59e700 1 --2- 172.21.15.162:0/3900716769 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f12b40554e0 0x7f12b40579a0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:15.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.187+0000 7f12cf59e700 1 --2- 172.21.15.162:0/3900716769 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f12d00b4b40 0x7f12d00650e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:15.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.187+0000 7f12cf59e700 1 --2- 172.21.15.162:0/3900716769 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12d00611e0 0x7f12d012e670 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:15.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.187+0000 7f12cf59e700 1 --2- 172.21.15.162:0/3900716769 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12d00615c0 0x7f12d00ae8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:15.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.187+0000 7f12cf59e700 1 -- 172.21.15.162:0/3900716769 >> 172.21.15.162:0/3900716769 conn(0x7f12d00b54f0 msgr2=0x7f12d0134b20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:15.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.187+0000 7f12cf59e700 1 -- 172.21.15.162:0/3900716769 shutdown_connections 2020-11-28T21:55:15.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.187+0000 7f12cf59e700 1 -- 172.21.15.162:0/3900716769 wait complete. 2020-11-28T21:55:15.207 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s 2020-11-28T21:55:15.208 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data 2020-11-28T21:55:15.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.327+0000 7fb63c165700 1 Processor -- start 2020-11-28T21:55:15.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.327+0000 7fb63c165700 1 -- start start 2020-11-28T21:55:15.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.327+0000 7fb63c165700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb6341433c0 0x7fb6340605c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:15.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.327+0000 7fb63c165700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb634143950 con 0x7fb6340b9ce0 2020-11-28T21:55:15.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.327+0000 7fb63c165700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6340ba0c0 con 0x7fb634067610 2020-11-28T21:55:15.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.327+0000 7fb63c165700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6340b4340 con 0x7fb6341433c0 2020-11-28T21:55:15.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.327+0000 7fb63a702700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fb634067610 0x7fb63413fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51334/0 (socket says 172.21.15.162:51334) 2020-11-28T21:55:15.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.327+0000 7fb63a702700 1 -- 172.21.15.162:0/3997121627 learned_addr learned my addr 172.21.15.162:0/3997121627 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:15.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.327+0000 7fb639700700 1 --2- 172.21.15.162:0/3997121627 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb6341433c0 0x7fb6340605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:15.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.327+0000 7fb638eff700 1 -- 172.21.15.162:0/3997121627 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3799598548 0 0) 0x7fb6340ba0c0 con 0x7fb634067610 2020-11-28T21:55:15.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.327+0000 7fb638eff700 1 -- 172.21.15.162:0/3997121627 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb61c003230 con 0x7fb634067610 2020-11-28T21:55:15.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.327+0000 7fb638eff700 1 -- 172.21.15.162:0/3997121627 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 767324004 0 0) 0x7fb61c003230 con 0x7fb634067610 2020-11-28T21:55:15.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.327+0000 7fb638eff700 1 -- 172.21.15.162:0/3997121627 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb6341433c0 msgr2=0x7fb6340605c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:15.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.327+0000 7fb638eff700 1 --2- 172.21.15.162:0/3997121627 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb6341433c0 0x7fb6340605c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:15.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.327+0000 7fb638eff700 1 -- 172.21.15.162:0/3997121627 >> v1:172.21.15.118:6789/0 conn(0x7fb6340b9ce0 legacy=0x7fb6340b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:15.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.327+0000 7fb638eff700 1 -- 172.21.15.162:0/3997121627 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb634066c30 con 0x7fb634067610 2020-11-28T21:55:15.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.327+0000 7fb638eff700 1 -- 172.21.15.162:0/3997121627 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7fb62c001a50 con 0x7fb634067610 2020-11-28T21:55:15.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.327+0000 7fb638eff700 1 -- 172.21.15.162:0/3997121627 >> v1:172.21.15.162:6789/0 conn(0x7fb634067610 legacy=0x7fb63413fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:15.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.327+0000 7fb638eff700 1 --2- 172.21.15.162:0/3997121627 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb61c003750 0x7fb61c003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:15.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.327+0000 7fb638eff700 1 --2- 172.21.15.162:0/3997121627 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb6341433c0 0x7fb61c0044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:15.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.327+0000 7fb638eff700 1 --2- 172.21.15.162:0/3997121627 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb61c0049c0 0x7fb61c004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:15.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.327+0000 7fb638eff700 1 -- 172.21.15.162:0/3997121627 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6340b4340 con 0x7fb61c003750 2020-11-28T21:55:15.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.327+0000 7fb638eff700 1 -- 172.21.15.162:0/3997121627 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c004160 con 0x7fb61c0049c0 2020-11-28T21:55:15.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.327+0000 7fb638eff700 1 -- 172.21.15.162:0/3997121627 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c004320 con 0x7fb6341433c0 2020-11-28T21:55:15.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.327+0000 7fb63a702700 1 --2- 172.21.15.162:0/3997121627 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb61c0049c0 0x7fb61c004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:15.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.327+0000 7fb639f01700 1 --2- 172.21.15.162:0/3997121627 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb61c003750 0x7fb61c003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:15.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.327+0000 7fb639700700 1 --2- 172.21.15.162:0/3997121627 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb6341433c0 0x7fb61c0044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:15.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.327+0000 7fb63a702700 1 -- 172.21.15.162:0/3997121627 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb6341433c0 msgr2=0x7fb61c0044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:15.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.327+0000 7fb63a702700 1 --2- 172.21.15.162:0/3997121627 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb6341433c0 0x7fb61c0044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:15.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.327+0000 7fb63a702700 1 -- 172.21.15.162:0/3997121627 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb61c003750 msgr2=0x7fb61c003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:15.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.327+0000 7fb63a702700 1 --2- 172.21.15.162:0/3997121627 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb61c003750 0x7fb61c003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:15.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.327+0000 7fb63a702700 1 -- 172.21.15.162:0/3997121627 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb61c003230 con 0x7fb61c0049c0 2020-11-28T21:55:15.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.327+0000 7fb639f01700 1 --2- 172.21.15.162:0/3997121627 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb61c003750 0x7fb61c003b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:55:15.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.327+0000 7fb639700700 1 --2- 172.21.15.162:0/3997121627 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb6341433c0 0x7fb61c0044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:55:15.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.327+0000 7fb63a702700 1 --2- 172.21.15.162:0/3997121627 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb61c0049c0 0x7fb61c004e40 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 rx=0x7fb62c001760 tx=0x7fb63405fe30).ready entity=mon.1 client_cookie=725840d0b6af6cf3 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:15.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.327+0000 7fb638eff700 1 -- 172.21.15.162:0/3997121627 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb62c005b70 con 0x7fb61c0049c0 2020-11-28T21:55:15.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.327+0000 7fb638eff700 1 -- 172.21.15.162:0/3997121627 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb62c005cd0 con 0x7fb61c0049c0 2020-11-28T21:55:15.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.327+0000 7fb638eff700 1 -- 172.21.15.162:0/3997121627 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb62c0072c0 con 0x7fb61c0049c0 2020-11-28T21:55:15.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.327+0000 7fb63c165700 1 -- 172.21.15.162:0/3997121627 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb61c0049c0 msgr2=0x7fb61c004e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:15.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.327+0000 7fb63c165700 1 --2- 172.21.15.162:0/3997121627 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb61c0049c0 0x7fb61c004e40 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 rx=0x7fb62c001760 tx=0x7fb63405fe30).stop 2020-11-28T21:55:15.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.327+0000 7fb63a702700 1 -- 172.21.15.162:0/3997121627 reap_dead start 2020-11-28T21:55:15.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.331+0000 7fb63c165700 1 -- 172.21.15.162:0/3997121627 shutdown_connections 2020-11-28T21:55:15.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.331+0000 7fb63c165700 1 -- 172.21.15.162:0/3997121627 >> 172.21.15.162:0/3997121627 conn(0x7fb63405c430 msgr2=0x7fb6340b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:15.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.331+0000 7fb63c165700 1 -- 172.21.15.162:0/3997121627 shutdown_connections 2020-11-28T21:55:15.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.331+0000 7fb63c165700 1 -- 172.21.15.162:0/3997121627 wait complete. 2020-11-28T21:55:15.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.331+0000 7fb63c165700 1 Processor -- start 2020-11-28T21:55:15.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.331+0000 7fb63c165700 1 -- start start 2020-11-28T21:55:15.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.331+0000 7fb63c165700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb634147400 0x7fb6340af280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:15.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.331+0000 7fb63c165700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6340af770 0x7fb634144650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:15.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.331+0000 7fb63c165700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb634144b40 0x7fb634144fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:15.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.331+0000 7fb63c165700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb634152b70 con 0x7fb634144b40 2020-11-28T21:55:15.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.331+0000 7fb63c165700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6341477e0 con 0x7fb6340af770 2020-11-28T21:55:15.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.331+0000 7fb63c165700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6340afe40 con 0x7fb634147400 2020-11-28T21:55:15.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.335+0000 7fb639f01700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb634147400 0x7fb6340af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:15.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.335+0000 7fb639700700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6340af770 0x7fb634144650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:15.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.335+0000 7fb63a702700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb634144b40 0x7fb634144fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:15.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.335+0000 7fb639f01700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb634147400 0x7fb6340af280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:41722/0 (socket says 172.21.15.162:41722) 2020-11-28T21:55:15.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.335+0000 7fb639f01700 1 -- 172.21.15.162:0/1047951825 learned_addr learned my addr 172.21.15.162:0/1047951825 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:15.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.335+0000 7fb63a702700 1 -- 172.21.15.162:0/1047951825 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb634147400 msgr2=0x7fb6340af280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:15.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.335+0000 7fb63a702700 1 --2- 172.21.15.162:0/1047951825 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb634147400 0x7fb6340af280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:15.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.335+0000 7fb63a702700 1 -- 172.21.15.162:0/1047951825 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6340af770 msgr2=0x7fb634144650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:15.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.335+0000 7fb63a702700 1 --2- 172.21.15.162:0/1047951825 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6340af770 0x7fb634144650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:15.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.335+0000 7fb63a702700 1 -- 172.21.15.162:0/1047951825 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6341441f0 con 0x7fb634144b40 2020-11-28T21:55:15.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.335+0000 7fb639700700 1 --2- 172.21.15.162:0/1047951825 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6340af770 0x7fb634144650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:55:15.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.335+0000 7fb63a702700 1 --2- 172.21.15.162:0/1047951825 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb634144b40 0x7fb634144fc0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7fb6340b91a0 tx=0x7fb62c013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:15.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.335+0000 7fb622ffd700 1 -- 172.21.15.162:0/1047951825 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb62c017070 con 0x7fb634144b40 2020-11-28T21:55:15.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.335+0000 7fb622ffd700 1 -- 172.21.15.162:0/1047951825 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb62c0074d0 con 0x7fb634144b40 2020-11-28T21:55:15.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.335+0000 7fb63c165700 1 -- 172.21.15.162:0/1047951825 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb634151f30 con 0x7fb634144b40 2020-11-28T21:55:15.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.335+0000 7fb622ffd700 1 -- 172.21.15.162:0/1047951825 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb62c01a040 con 0x7fb634144b40 2020-11-28T21:55:15.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.335+0000 7fb63c165700 1 -- 172.21.15.162:0/1047951825 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb634147a40 con 0x7fb634144b40 2020-11-28T21:55:15.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.335+0000 7fb63c165700 1 -- 172.21.15.162:0/1047951825 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb634057050 con 0x7fb634144b40 2020-11-28T21:55:15.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.339+0000 7fb622ffd700 1 -- 172.21.15.162:0/1047951825 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb62c010070 con 0x7fb634144b40 2020-11-28T21:55:15.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.339+0000 7fb622ffd700 1 --2- 172.21.15.162:0/1047951825 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb6140554e0 0x7fb6140579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:15.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.339+0000 7fb622ffd700 1 -- 172.21.15.162:0/1047951825 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6075+0+0 (secure 0 0 0) 0x7fb62c06eb70 con 0x7fb634144b40 2020-11-28T21:55:15.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.339+0000 7fb639f01700 1 --2- 172.21.15.162:0/1047951825 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb6140554e0 0x7fb6140579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:15.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.339+0000 7fb639f01700 1 --2- 172.21.15.162:0/1047951825 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb6140554e0 0x7fb6140579a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7fb630009000 tx=0x7fb63000c040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:15.355 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.347+0000 7fb622ffd700 1 -- 172.21.15.162:0/1047951825 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb62c044dd0 con 0x7fb634144b40 2020-11-28T21:55:15.537 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.531+0000 7fb63c165700 1 -- 172.21.15.162:0/1047951825 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb634146e40 con 0x7fb634144b40 2020-11-28T21:55:15.538 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.535+0000 7fb622ffd700 1 -- 172.21.15.162:0/1047951825 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=-22 pool 'cephfs_metadata' already contains some objects. Use an empty pool instead. v59) v1 ==== 209+0+0 (secure 0 0 0) 0x7fb62c0483f0 con 0x7fb634144b40 2020-11-28T21:55:15.539 INFO:teuthology.orchestra.run.smithi162.stderr:Error EINVAL: pool 'cephfs_metadata' already contains some objects. Use an empty pool instead. 2020-11-28T21:55:15.544 DEBUG:teuthology.orchestra.run:got remote process result: 22 2020-11-28T21:55:15.544 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2020-11-28T21:55:15.657 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.651+0000 7f3d96164700 1 Processor -- start 2020-11-28T21:55:15.658 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.655+0000 7f3d96164700 1 -- start start 2020-11-28T21:55:15.660 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.655+0000 7f3d96164700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3d90060d10 0x7f3d90098ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:15.660 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.655+0000 7f3d96164700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f3d900b09f0 0x7f3d90098fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:15.660 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.655+0000 7f3d96164700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d900afec0 con 0x7f3d90099600 2020-11-28T21:55:15.660 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.655+0000 7f3d96164700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d900608b0 con 0x7f3d900b09f0 2020-11-28T21:55:15.661 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.655+0000 7f3d96164700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d900b0160 con 0x7f3d90060d10 2020-11-28T21:55:15.661 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.655+0000 7f3d95162700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3d90060d10 0x7f3d90098ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:15.661 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.655+0000 7f3d95162700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3d90060d10 0x7f3d90098ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44506/0 (socket says 172.21.15.162:44506) 2020-11-28T21:55:15.661 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.655+0000 7f3d95162700 1 -- 172.21.15.162:0/3443374056 learned_addr learned my addr 172.21.15.162:0/3443374056 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:15.662 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.655+0000 7f3d94961700 1 --2- 172.21.15.162:0/3443374056 >> v2:172.21.15.118:3300/0 conn(0x7f3d900b09f0 0x7f3d90098fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:15.662 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.655+0000 7f3d95162700 1 -- 172.21.15.162:0/3443374056 >> v2:172.21.15.118:3300/0 conn(0x7f3d900b09f0 msgr2=0x7f3d90098fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:15.662 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.655+0000 7f3d95162700 1 --2- 172.21.15.162:0/3443374056 >> v2:172.21.15.118:3300/0 conn(0x7f3d900b09f0 0x7f3d90098fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:15.662 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.655+0000 7f3d95162700 1 -- 172.21.15.162:0/3443374056 >> v1:172.21.15.118:6789/0 conn(0x7f3d90099600 legacy=0x7f3d90099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:15.663 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.655+0000 7f3d95162700 1 -- 172.21.15.162:0/3443374056 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d900b0400 con 0x7f3d90060d10 2020-11-28T21:55:15.663 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.655+0000 7f3d94961700 1 --2- 172.21.15.162:0/3443374056 >> v2:172.21.15.118:3300/0 conn(0x7f3d900b09f0 0x7f3d90098fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:55:15.663 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.655+0000 7f3d95162700 1 --2- 172.21.15.162:0/3443374056 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d90060d10 0x7f3d90098ac0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 rx=0x7f3d8c002f30 tx=0x7f3d8c006e80).ready entity=mon.1 client_cookie=ac33cfd3ca560b76 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:15.663 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.655+0000 7f3d7ffff700 1 -- 172.21.15.162:0/3443374056 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d8c0097f0 con 0x7f3d90060d10 2020-11-28T21:55:15.663 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.655+0000 7f3d7ffff700 1 -- 172.21.15.162:0/3443374056 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d8c009950 con 0x7f3d90060d10 2020-11-28T21:55:15.664 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.655+0000 7f3d7ffff700 1 -- 172.21.15.162:0/3443374056 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d8c009cb0 con 0x7f3d90060d10 2020-11-28T21:55:15.664 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.655+0000 7f3d96164700 1 -- 172.21.15.162:0/3443374056 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d90060d10 msgr2=0x7f3d90098ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:15.664 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.655+0000 7f3d96164700 1 --2- 172.21.15.162:0/3443374056 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d90060d10 0x7f3d90098ac0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 rx=0x7f3d8c002f30 tx=0x7f3d8c006e80).stop 2020-11-28T21:55:15.669 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.655+0000 7f3d96164700 1 -- 172.21.15.162:0/3443374056 shutdown_connections 2020-11-28T21:55:15.669 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.655+0000 7f3d96164700 1 --2- 172.21.15.162:0/3443374056 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d90060d10 0x7f3d90098ac0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:15.669 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.655+0000 7f3d96164700 1 --2- 172.21.15.162:0/3443374056 >> v2:172.21.15.118:3300/0 conn(0x7f3d900b09f0 0x7f3d90098fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:15.670 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.655+0000 7f3d96164700 1 -- 172.21.15.162:0/3443374056 >> 172.21.15.162:0/3443374056 conn(0x7f3d900b5520 msgr2=0x7f3d900b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:15.670 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.655+0000 7f3d96164700 1 -- 172.21.15.162:0/3443374056 shutdown_connections 2020-11-28T21:55:15.670 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.655+0000 7f3d96164700 1 -- 172.21.15.162:0/3443374056 wait complete. 2020-11-28T21:55:15.671 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.659+0000 7f3d96164700 1 Processor -- start 2020-11-28T21:55:15.671 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.659+0000 7f3d96164700 1 -- start start 2020-11-28T21:55:15.671 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.659+0000 7f3d96164700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3d90060d10 0x7f3d901483d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:15.672 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.659+0000 7f3d96164700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d90099600 0x7f3d900aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:15.672 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.659+0000 7f3d96164700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d900b09f0 0x7f3d90151990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:15.672 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.659+0000 7f3d96164700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d900af0c0 con 0x7f3d90099600 2020-11-28T21:55:15.673 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.659+0000 7f3d96164700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d90090260 con 0x7f3d900b09f0 2020-11-28T21:55:15.673 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.659+0000 7f3d96164700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d90090500 con 0x7f3d90060d10 2020-11-28T21:55:15.673 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.659+0000 7f3d95963700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d900b09f0 0x7f3d90151990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:15.673 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.659+0000 7f3d95963700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d900b09f0 0x7f3d90151990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44516/0 (socket says 172.21.15.162:44516) 2020-11-28T21:55:15.674 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.659+0000 7f3d94961700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d90099600 0x7f3d900aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:15.674 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.659+0000 7f3d95963700 1 -- 172.21.15.162:0/3672712098 learned_addr learned my addr 172.21.15.162:0/3672712098 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:15.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.659+0000 7f3d95162700 1 --2- 172.21.15.162:0/3672712098 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3d90060d10 0x7f3d901483d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:15.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.659+0000 7f3d95963700 1 -- 172.21.15.162:0/3672712098 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3d90060d10 msgr2=0x7f3d901483d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:15.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.659+0000 7f3d95963700 1 --2- 172.21.15.162:0/3672712098 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3d90060d10 0x7f3d901483d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:15.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.659+0000 7f3d95963700 1 -- 172.21.15.162:0/3672712098 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d90099600 msgr2=0x7f3d900aebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:15.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.659+0000 7f3d95963700 1 --2- 172.21.15.162:0/3672712098 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d90099600 0x7f3d900aebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:15.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.659+0000 7f3d95963700 1 -- 172.21.15.162:0/3672712098 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d900ae490 con 0x7f3d900b09f0 2020-11-28T21:55:15.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.659+0000 7f3d95963700 1 --2- 172.21.15.162:0/3672712098 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d900b09f0 0x7f3d90151990 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 rx=0x7f3d90060790 tx=0x7f3d88007f10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:15.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.659+0000 7f3d7e7fc700 1 -- 172.21.15.162:0/3672712098 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d8800a5a0 con 0x7f3d900b09f0 2020-11-28T21:55:15.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.659+0000 7f3d96164700 1 -- 172.21.15.162:0/3672712098 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d900ae730 con 0x7f3d900b09f0 2020-11-28T21:55:15.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.659+0000 7f3d7e7fc700 1 -- 172.21.15.162:0/3672712098 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d8800a700 con 0x7f3d900b09f0 2020-11-28T21:55:15.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.659+0000 7f3d7e7fc700 1 -- 172.21.15.162:0/3672712098 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d8800aa60 con 0x7f3d900b09f0 2020-11-28T21:55:15.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.659+0000 7f3d96164700 1 -- 172.21.15.162:0/3672712098 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d90151e80 con 0x7f3d900b09f0 2020-11-28T21:55:15.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.659+0000 7f3d96164700 1 -- 172.21.15.162:0/3672712098 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d90057050 con 0x7f3d900b09f0 2020-11-28T21:55:15.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.659+0000 7f3d7e7fc700 1 -- 172.21.15.162:0/3672712098 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f3d88019030 con 0x7f3d900b09f0 2020-11-28T21:55:15.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.663+0000 7f3d7e7fc700 1 --2- 172.21.15.162:0/3672712098 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3d70055440 0x7f3d70057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:15.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.663+0000 7f3d7e7fc700 1 -- 172.21.15.162:0/3672712098 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6075+0+0 (secure 0 0 0) 0x7f3d8806d850 con 0x7f3d900b09f0 2020-11-28T21:55:15.680 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.663+0000 7f3d95162700 1 --2- 172.21.15.162:0/3672712098 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3d70055440 0x7f3d70057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:15.680 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.663+0000 7f3d95162700 1 --2- 172.21.15.162:0/3672712098 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3d70055440 0x7f3d70057900 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7f3d8c01c000 tx=0x7f3d8c015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:15.681 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.667+0000 7f3d7e7fc700 1 -- 172.21.15.162:0/3672712098 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f3d88043ab0 con 0x7f3d900b09f0 2020-11-28T21:55:15.856 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.851+0000 7f3d96164700 1 -- 172.21.15.162:0/3672712098 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f3d900ae8c0 con 0x7f3d900b09f0 2020-11-28T21:55:15.858 INFO:tasks.ceph.mon.b.smithi118.stderr:2020-11-28T21:55:15.852+0000 7ff4ded1b700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-11-28T21:55:15.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.863+0000 7f3d7e7fc700 1 -- 172.21.15.162:0/3672712098 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 23 and data pool 22 v60) v1 ==== 189+0+0 (secure 0 0 0) 0x7f3d880470d0 con 0x7f3d900b09f0 2020-11-28T21:55:15.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.867+0000 7f3d96164700 1 -- 172.21.15.162:0/3672712098 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3d70055440 msgr2=0x7f3d70057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:15.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.867+0000 7f3d96164700 1 --2- 172.21.15.162:0/3672712098 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3d70055440 0x7f3d70057900 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7f3d8c01c000 tx=0x7f3d8c015000).stop 2020-11-28T21:55:15.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.867+0000 7f3d96164700 1 -- 172.21.15.162:0/3672712098 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d900b09f0 msgr2=0x7f3d90151990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:15.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.867+0000 7f3d96164700 1 --2- 172.21.15.162:0/3672712098 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d900b09f0 0x7f3d90151990 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 rx=0x7f3d90060790 tx=0x7f3d88007f10).stop 2020-11-28T21:55:15.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.867+0000 7f3d96164700 1 -- 172.21.15.162:0/3672712098 shutdown_connections 2020-11-28T21:55:15.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.867+0000 7f3d96164700 1 --2- 172.21.15.162:0/3672712098 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3d70055440 0x7f3d70057900 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:15.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.867+0000 7f3d96164700 1 --2- 172.21.15.162:0/3672712098 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3d90060d10 0x7f3d901483d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:15.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.867+0000 7f3d96164700 1 --2- 172.21.15.162:0/3672712098 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d900b09f0 0x7f3d90151990 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:15.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.867+0000 7f3d96164700 1 --2- 172.21.15.162:0/3672712098 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d90099600 0x7f3d900aebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:15.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.867+0000 7f3d96164700 1 -- 172.21.15.162:0/3672712098 >> 172.21.15.162:0/3672712098 conn(0x7f3d900b5520 msgr2=0x7f3d900999e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:15.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.867+0000 7f3d96164700 1 -- 172.21.15.162:0/3672712098 shutdown_connections 2020-11-28T21:55:15.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.867+0000 7f3d96164700 1 -- 172.21.15.162:0/3672712098 wait complete. 2020-11-28T21:55:15.876 INFO:teuthology.orchestra.run.smithi162.stderr:new fs with metadata pool 23 and data pool 22 2020-11-28T21:55:15.886 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2020-11-28T21:55:16.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.999+0000 7f5b9d0de700 1 Processor -- start 2020-11-28T21:55:16.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.999+0000 7f5b9d0de700 1 -- start start 2020-11-28T21:55:16.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.999+0000 7f5b9d0de700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5b9805bcb0 0x7f5b9805c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:16.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.999+0000 7f5b9d0de700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5b980b2300 0x7f5b980add90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:16.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.999+0000 7f5b9d0de700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b980ad6c0 con 0x7f5b980b0820 2020-11-28T21:55:16.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.999+0000 7f5b9d0de700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b980ae520 con 0x7f5b980b2300 2020-11-28T21:55:16.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:15.999+0000 7f5b9d0de700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b9805c090 con 0x7f5b9805bcb0 2020-11-28T21:55:16.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.003+0000 7f5b97fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5b9805bcb0 0x7f5b9805c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:16.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.003+0000 7f5b9c8dd700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f5b980b0820 0x7f5b98147750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51368/0 (socket says 172.21.15.162:51368) 2020-11-28T21:55:16.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.003+0000 7f5b9c8dd700 1 -- 172.21.15.162:0/2919088699 learned_addr learned my addr 172.21.15.162:0/2919088699 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:16.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.003+0000 7f5b977fe700 1 --2- 172.21.15.162:0/2919088699 >> v2:172.21.15.118:3300/0 conn(0x7f5b980b2300 0x7f5b980add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:16.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.003+0000 7f5b96ffd700 1 -- 172.21.15.162:0/2919088699 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 650283156 0 0) 0x7f5b980ad6c0 con 0x7f5b980b0820 2020-11-28T21:55:16.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.003+0000 7f5b97fff700 1 -- 172.21.15.162:0/2919088699 >> v2:172.21.15.118:3300/0 conn(0x7f5b980b2300 msgr2=0x7f5b980add90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:16.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.003+0000 7f5b97fff700 1 --2- 172.21.15.162:0/2919088699 >> v2:172.21.15.118:3300/0 conn(0x7f5b980b2300 0x7f5b980add90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:16.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.003+0000 7f5b97fff700 1 -- 172.21.15.162:0/2919088699 >> v1:172.21.15.162:6789/0 conn(0x7f5b980b0820 legacy=0x7f5b98147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:16.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.003+0000 7f5b97fff700 1 -- 172.21.15.162:0/2919088699 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b980ae800 con 0x7f5b9805bcb0 2020-11-28T21:55:16.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.003+0000 7f5b97fff700 1 --2- 172.21.15.162:0/2919088699 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b9805bcb0 0x7f5b9805c6a0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 rx=0x7f5b80002f30 tx=0x7f5b80006dc0).ready entity=mon.1 client_cookie=c4e71cf2aea1d00f server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:16.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.003+0000 7f5b96ffd700 1 -- 172.21.15.162:0/2919088699 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b800097b0 con 0x7f5b9805bcb0 2020-11-28T21:55:16.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.003+0000 7f5b96ffd700 1 -- 172.21.15.162:0/2919088699 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b80009910 con 0x7f5b9805bcb0 2020-11-28T21:55:16.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.003+0000 7f5b96ffd700 1 -- 172.21.15.162:0/2919088699 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b80009c70 con 0x7f5b9805bcb0 2020-11-28T21:55:16.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.003+0000 7f5b9d0de700 1 -- 172.21.15.162:0/2919088699 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b9805bcb0 msgr2=0x7f5b9805c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:16.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.003+0000 7f5b9d0de700 1 --2- 172.21.15.162:0/2919088699 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b9805bcb0 0x7f5b9805c6a0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 rx=0x7f5b80002f30 tx=0x7f5b80006dc0).stop 2020-11-28T21:55:16.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.003+0000 7f5b9d0de700 1 -- 172.21.15.162:0/2919088699 shutdown_connections 2020-11-28T21:55:16.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.003+0000 7f5b9d0de700 1 --2- 172.21.15.162:0/2919088699 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b9805bcb0 0x7f5b9805c6a0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:16.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.003+0000 7f5b9d0de700 1 --2- 172.21.15.162:0/2919088699 >> v2:172.21.15.118:3300/0 conn(0x7f5b980b2300 0x7f5b980add90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:16.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.003+0000 7f5b9d0de700 1 -- 172.21.15.162:0/2919088699 >> 172.21.15.162:0/2919088699 conn(0x7f5b980b17a0 msgr2=0x7f5b980b1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:16.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.003+0000 7f5b9d0de700 1 -- 172.21.15.162:0/2919088699 shutdown_connections 2020-11-28T21:55:16.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.003+0000 7f5b9d0de700 1 -- 172.21.15.162:0/2919088699 wait complete. 2020-11-28T21:55:16.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.003+0000 7f5b9d0de700 1 Processor -- start 2020-11-28T21:55:16.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.003+0000 7f5b9d0de700 1 -- start start 2020-11-28T21:55:16.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.007+0000 7f5b9d0de700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b9805bcb0 0x7f5b98064ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:16.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.007+0000 7f5b9d0de700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5b980b0820 0x7f5b98159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:16.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.007+0000 7f5b9d0de700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b980b2300 0x7f5b98159830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:16.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.007+0000 7f5b9d0de700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b980674d0 con 0x7f5b9805bcb0 2020-11-28T21:55:16.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.007+0000 7f5b9d0de700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9814e440 con 0x7f5b980b2300 2020-11-28T21:55:16.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.007+0000 7f5b9d0de700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b98065860 con 0x7f5b980b0820 2020-11-28T21:55:16.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.007+0000 7f5b9c8dd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b980b2300 0x7f5b98159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:16.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.007+0000 7f5b9c8dd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b980b2300 0x7f5b98159830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44528/0 (socket says 172.21.15.162:44528) 2020-11-28T21:55:16.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.007+0000 7f5b9c8dd700 1 -- 172.21.15.162:0/3510143881 learned_addr learned my addr 172.21.15.162:0/3510143881 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:16.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.007+0000 7f5b97fff700 1 --2- 172.21.15.162:0/3510143881 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b9805bcb0 0x7f5b98064ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:16.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.007+0000 7f5b977fe700 1 --2- 172.21.15.162:0/3510143881 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5b980b0820 0x7f5b98159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:16.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.007+0000 7f5b977fe700 1 -- 172.21.15.162:0/3510143881 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b980b2300 msgr2=0x7f5b98159830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:16.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.007+0000 7f5b977fe700 1 --2- 172.21.15.162:0/3510143881 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b980b2300 0x7f5b98159830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:16.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.007+0000 7f5b977fe700 1 -- 172.21.15.162:0/3510143881 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b9805bcb0 msgr2=0x7f5b98064ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:16.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.007+0000 7f5b977fe700 1 --2- 172.21.15.162:0/3510143881 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b9805bcb0 0x7f5b98064ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:16.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.007+0000 7f5b977fe700 1 -- 172.21.15.162:0/3510143881 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b981497a0 con 0x7f5b980b0820 2020-11-28T21:55:16.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.007+0000 7f5b9c8dd700 1 --2- 172.21.15.162:0/3510143881 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b980b2300 0x7f5b98159830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:55:16.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.007+0000 7f5b977fe700 1 --2- 172.21.15.162:0/3510143881 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5b980b0820 0x7f5b98159340 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f5b98065700 tx=0x7f5b88006dc0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:16.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.007+0000 7f5b957fa700 1 -- 172.21.15.162:0/3510143881 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b8800c7e0 con 0x7f5b980b0820 2020-11-28T21:55:16.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.007+0000 7f5b957fa700 1 -- 172.21.15.162:0/3510143881 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b8800c940 con 0x7f5b980b0820 2020-11-28T21:55:16.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.007+0000 7f5b9d0de700 1 -- 172.21.15.162:0/3510143881 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b98149a40 con 0x7f5b980b0820 2020-11-28T21:55:16.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.007+0000 7f5b957fa700 1 -- 172.21.15.162:0/3510143881 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b8800cca0 con 0x7f5b980b0820 2020-11-28T21:55:16.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.007+0000 7f5b9d0de700 1 -- 172.21.15.162:0/3510143881 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b9814da30 con 0x7f5b980b0820 2020-11-28T21:55:16.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.007+0000 7f5b9d0de700 1 -- 172.21.15.162:0/3510143881 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b98057050 con 0x7f5b980b0820 2020-11-28T21:55:16.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.011+0000 7f5b957fa700 1 -- 172.21.15.162:0/3510143881 <== mon.2 v2:172.21.15.118:3301/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5b8801b410 con 0x7f5b980b0820 2020-11-28T21:55:16.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.011+0000 7f5b957fa700 1 --2- 172.21.15.162:0/3510143881 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5b7c0553b0 0x7f5b7c057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:16.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.011+0000 7f5b957fa700 1 -- 172.21.15.162:0/3510143881 <== mon.2 v2:172.21.15.118:3301/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f5b8806e020 con 0x7f5b980b0820 2020-11-28T21:55:16.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.011+0000 7f5b97fff700 1 --2- 172.21.15.162:0/3510143881 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5b7c0553b0 0x7f5b7c057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:16.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.011+0000 7f5b97fff700 1 --2- 172.21.15.162:0/3510143881 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5b7c0553b0 0x7f5b7c057870 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7f5b8001a000 tx=0x7f5b80015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:16.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.019+0000 7f5b957fa700 1 -- 172.21.15.162:0/3510143881 <== mon.2 v2:172.21.15.118:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5b880441d0 con 0x7f5b980b0820 2020-11-28T21:55:16.216 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.211+0000 7f5b9d0de700 1 -- 172.21.15.162:0/3510143881 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5b9805d250 con 0x7f5b980b0820 2020-11-28T21:55:16.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.915+0000 7f5b957fa700 1 -- 172.21.15.162:0/3510143881 <== mon.2 v2:172.21.15.118:3301/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v61) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5b880477f0 con 0x7f5b980b0820 2020-11-28T21:55:16.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.919+0000 7f5b9d0de700 1 -- 172.21.15.162:0/3510143881 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5b7c0553b0 msgr2=0x7f5b7c057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:16.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.919+0000 7f5b9d0de700 1 --2- 172.21.15.162:0/3510143881 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5b7c0553b0 0x7f5b7c057870 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7f5b8001a000 tx=0x7f5b80015000).stop 2020-11-28T21:55:16.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.919+0000 7f5b9d0de700 1 -- 172.21.15.162:0/3510143881 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5b980b0820 msgr2=0x7f5b98159340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:16.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.919+0000 7f5b9d0de700 1 --2- 172.21.15.162:0/3510143881 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5b980b0820 0x7f5b98159340 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f5b98065700 tx=0x7f5b88006dc0).stop 2020-11-28T21:55:16.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.919+0000 7f5b9d0de700 1 -- 172.21.15.162:0/3510143881 shutdown_connections 2020-11-28T21:55:16.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.919+0000 7f5b9d0de700 1 --2- 172.21.15.162:0/3510143881 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5b7c0553b0 0x7f5b7c057870 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:16.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.919+0000 7f5b9d0de700 1 --2- 172.21.15.162:0/3510143881 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b980b2300 0x7f5b98159830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:16.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.919+0000 7f5b9d0de700 1 --2- 172.21.15.162:0/3510143881 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b9805bcb0 0x7f5b98064ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:16.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.919+0000 7f5b9d0de700 1 --2- 172.21.15.162:0/3510143881 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5b980b0820 0x7f5b98159340 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:16.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.919+0000 7f5b9d0de700 1 -- 172.21.15.162:0/3510143881 >> 172.21.15.162:0/3510143881 conn(0x7f5b980b17a0 msgr2=0x7f5b980b0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:16.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.923+0000 7f5b9d0de700 1 -- 172.21.15.162:0/3510143881 shutdown_connections 2020-11-28T21:55:16.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:16.923+0000 7f5b9d0de700 1 -- 172.21.15.162:0/3510143881 wait complete. 2020-11-28T21:55:16.942 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-11-28T21:55:17.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.063+0000 7feff65fa700 1 Processor -- start 2020-11-28T21:55:17.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.063+0000 7feff65fa700 1 -- start start 2020-11-28T21:55:17.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.063+0000 7feff65fa700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7feff00b2d00 0x7feff005c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:17.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.063+0000 7feff65fa700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feff005baf0 con 0x7feff00b30e0 2020-11-28T21:55:17.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.063+0000 7feff65fa700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feff005bd90 con 0x7feff00b3ab0 2020-11-28T21:55:17.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.063+0000 7feff65fa700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feff00b0e20 con 0x7feff00b2d00 2020-11-28T21:55:17.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.063+0000 7feff55f8700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7feff00b2d00 0x7feff005c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:17.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.063+0000 7feff55f8700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7feff00b2d00 0x7feff005c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44534/0 (socket says 172.21.15.162:44534) 2020-11-28T21:55:17.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.063+0000 7feff55f8700 1 -- 172.21.15.162:0/161735203 learned_addr learned my addr 172.21.15.162:0/161735203 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:17.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.063+0000 7feff4df7700 1 --1- 172.21.15.162:0/161735203 >> v1:172.21.15.162:6789/0 conn(0x7feff00b3ab0 0x7feff0142ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51380/0 (socket says 172.21.15.162:51380) 2020-11-28T21:55:17.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.063+0000 7feff55f8700 1 -- 172.21.15.162:0/161735203 >> v1:172.21.15.162:6789/0 conn(0x7feff00b3ab0 legacy=0x7feff0142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:17.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.063+0000 7feff55f8700 1 -- 172.21.15.162:0/161735203 >> v1:172.21.15.118:6789/0 conn(0x7feff00b30e0 legacy=0x7feff013fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:17.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.063+0000 7feff55f8700 1 -- 172.21.15.162:0/161735203 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff005c030 con 0x7feff00b2d00 2020-11-28T21:55:17.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.063+0000 7feff55f8700 1 --2- 172.21.15.162:0/161735203 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feff00b2d00 0x7feff005c4d0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 rx=0x7fefec002f30 tx=0x7fefec006e80).ready entity=mon.1 client_cookie=7c2f3d8f922a4341 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:17.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.063+0000 7fefe3fff700 1 -- 172.21.15.162:0/161735203 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefec0097f0 con 0x7feff00b2d00 2020-11-28T21:55:17.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.063+0000 7fefe3fff700 1 -- 172.21.15.162:0/161735203 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefec009950 con 0x7feff00b2d00 2020-11-28T21:55:17.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.063+0000 7fefe3fff700 1 -- 172.21.15.162:0/161735203 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefec009cb0 con 0x7feff00b2d00 2020-11-28T21:55:17.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.063+0000 7feff65fa700 1 -- 172.21.15.162:0/161735203 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feff00b2d00 msgr2=0x7feff005c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:17.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.063+0000 7feff65fa700 1 --2- 172.21.15.162:0/161735203 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feff00b2d00 0x7feff005c4d0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 rx=0x7fefec002f30 tx=0x7fefec006e80).stop 2020-11-28T21:55:17.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.063+0000 7feff65fa700 1 -- 172.21.15.162:0/161735203 shutdown_connections 2020-11-28T21:55:17.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.063+0000 7feff65fa700 1 --2- 172.21.15.162:0/161735203 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feff00b2d00 0x7feff005c4d0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:17.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.063+0000 7feff65fa700 1 -- 172.21.15.162:0/161735203 >> 172.21.15.162:0/161735203 conn(0x7feff00604c0 msgr2=0x7feff00608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:17.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.063+0000 7feff65fa700 1 -- 172.21.15.162:0/161735203 shutdown_connections 2020-11-28T21:55:17.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.063+0000 7feff65fa700 1 -- 172.21.15.162:0/161735203 wait complete. 2020-11-28T21:55:17.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.067+0000 7feff65fa700 1 Processor -- start 2020-11-28T21:55:17.086 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.067+0000 7feff65fa700 1 -- start start 2020-11-28T21:55:17.086 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.067+0000 7feff65fa700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7feff00b2d00 0x7feff00afad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:17.086 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.067+0000 7feff65fa700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feff00b30e0 0x7feff0155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:17.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.067+0000 7feff65fa700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feff00b3ab0 0x7feff0155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:17.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.067+0000 7feff65fa700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff00ad080 con 0x7feff00b3ab0 2020-11-28T21:55:17.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.067+0000 7feff65fa700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff0146400 con 0x7feff00b30e0 2020-11-28T21:55:17.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.067+0000 7feff65fa700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff01466a0 con 0x7feff00b2d00 2020-11-28T21:55:17.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.067+0000 7feff4df7700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feff00b30e0 0x7feff0155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:17.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.067+0000 7feff4df7700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feff00b30e0 0x7feff0155580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44542/0 (socket says 172.21.15.162:44542) 2020-11-28T21:55:17.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.067+0000 7feff4df7700 1 -- 172.21.15.162:0/2995884644 learned_addr learned my addr 172.21.15.162:0/2995884644 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:17.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.067+0000 7feff55f8700 1 --2- 172.21.15.162:0/2995884644 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7feff00b2d00 0x7feff00afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:17.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.067+0000 7feff5df9700 1 --2- 172.21.15.162:0/2995884644 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feff00b3ab0 0x7feff0155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:17.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.067+0000 7feff4df7700 1 -- 172.21.15.162:0/2995884644 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7feff00b2d00 msgr2=0x7feff00afad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:17.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.067+0000 7feff4df7700 1 --2- 172.21.15.162:0/2995884644 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7feff00b2d00 0x7feff00afad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:17.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.067+0000 7feff4df7700 1 -- 172.21.15.162:0/2995884644 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feff00b3ab0 msgr2=0x7feff0155a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:17.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.067+0000 7feff4df7700 1 --2- 172.21.15.162:0/2995884644 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feff00b3ab0 0x7feff0155a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:17.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.067+0000 7feff4df7700 1 -- 172.21.15.162:0/2995884644 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff01550e0 con 0x7feff00b30e0 2020-11-28T21:55:17.094 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.071+0000 7feff4df7700 1 --2- 172.21.15.162:0/2995884644 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feff00b30e0 0x7feff0155580 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 rx=0x7feff00b0740 tx=0x7fefe4007be0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:17.094 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.071+0000 7fefe27fc700 1 -- 172.21.15.162:0/2995884644 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefe40072e0 con 0x7feff00b30e0 2020-11-28T21:55:17.095 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.071+0000 7fefe27fc700 1 -- 172.21.15.162:0/2995884644 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe4005520 con 0x7feff00b30e0 2020-11-28T21:55:17.095 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.071+0000 7feff65fa700 1 -- 172.21.15.162:0/2995884644 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feff0151a10 con 0x7feff00b30e0 2020-11-28T21:55:17.095 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.071+0000 7fefe27fc700 1 -- 172.21.15.162:0/2995884644 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefe4005880 con 0x7feff00b30e0 2020-11-28T21:55:17.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.071+0000 7feff65fa700 1 -- 172.21.15.162:0/2995884644 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feff0151d20 con 0x7feff00b30e0 2020-11-28T21:55:17.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.071+0000 7feff65fa700 1 -- 172.21.15.162:0/2995884644 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feff0146860 con 0x7feff00b30e0 2020-11-28T21:55:17.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.071+0000 7fefe27fc700 1 -- 172.21.15.162:0/2995884644 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fefe401a060 con 0x7feff00b30e0 2020-11-28T21:55:17.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.071+0000 7fefe27fc700 1 --2- 172.21.15.162:0/2995884644 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fefd8055330 0x7fefd80577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:17.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.071+0000 7fefe27fc700 1 -- 172.21.15.162:0/2995884644 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fefe406d590 con 0x7feff00b30e0 2020-11-28T21:55:17.098 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.071+0000 7feff55f8700 1 --2- 172.21.15.162:0/2995884644 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fefd8055330 0x7fefd80577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:17.098 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.071+0000 7feff55f8700 1 --2- 172.21.15.162:0/2995884644 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fefd8055330 0x7fefd80577f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7fefec01c000 tx=0x7fefec015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:17.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.079+0000 7fefe27fc700 1 -- 172.21.15.162:0/2995884644 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fefe4043840 con 0x7feff00b30e0 2020-11-28T21:55:17.224 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.219+0000 7feff65fa700 1 -- 172.21.15.162:0/2995884644 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7feff0057050 con 0x7feff00b30e0 2020-11-28T21:55:17.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.935+0000 7fefe27fc700 1 -- 172.21.15.162:0/2995884644 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v128) v1 ==== 185+0+0 (secure 0 0 0) 0x7fefe4018020 con 0x7feff00b30e0 2020-11-28T21:55:17.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.939+0000 7feff65fa700 1 -- 172.21.15.162:0/2995884644 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fefd8055330 msgr2=0x7fefd80577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:17.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.939+0000 7feff65fa700 1 --2- 172.21.15.162:0/2995884644 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fefd8055330 0x7fefd80577f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7fefec01c000 tx=0x7fefec015000).stop 2020-11-28T21:55:17.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.939+0000 7feff65fa700 1 -- 172.21.15.162:0/2995884644 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feff00b30e0 msgr2=0x7feff0155580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:17.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.939+0000 7feff65fa700 1 --2- 172.21.15.162:0/2995884644 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feff00b30e0 0x7feff0155580 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 rx=0x7feff00b0740 tx=0x7fefe4007be0).stop 2020-11-28T21:55:17.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.939+0000 7feff65fa700 1 -- 172.21.15.162:0/2995884644 shutdown_connections 2020-11-28T21:55:17.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.939+0000 7feff65fa700 1 --2- 172.21.15.162:0/2995884644 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fefd8055330 0x7fefd80577f0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:17.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.939+0000 7feff65fa700 1 --2- 172.21.15.162:0/2995884644 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7feff00b2d00 0x7feff00afad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:17.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.939+0000 7feff65fa700 1 --2- 172.21.15.162:0/2995884644 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feff00b3ab0 0x7feff0155a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:17.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.939+0000 7feff65fa700 1 --2- 172.21.15.162:0/2995884644 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feff00b30e0 0x7feff0155580 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:17.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.939+0000 7feff65fa700 1 -- 172.21.15.162:0/2995884644 >> 172.21.15.162:0/2995884644 conn(0x7feff00604c0 msgr2=0x7feff013ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:17.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.939+0000 7feff65fa700 1 -- 172.21.15.162:0/2995884644 shutdown_connections 2020-11-28T21:55:17.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:17.939+0000 7feff65fa700 1 -- 172.21.15.162:0/2995884644 wait complete. 2020-11-28T21:55:17.946 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_metadata' removed 2020-11-28T21:55:17.959 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2020-11-28T21:55:18.086 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.083+0000 7fc39f59e700 1 Processor -- start 2020-11-28T21:55:18.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.083+0000 7fc39f59e700 1 -- start start 2020-11-28T21:55:18.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.083+0000 7fc39f59e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fc3a00afec0 0x7fc3a00b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:18.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.083+0000 7fc39f59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc3a00a8df0 0x7fc3a0099620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:18.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.083+0000 7fc39f59e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc3a0099b10 0x7fc3a0099f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:18.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.083+0000 7fc39f59e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3a0060ce0 con 0x7fc3a00afec0 2020-11-28T21:55:18.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.083+0000 7fc39f59e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3a0060500 con 0x7fc3a0099b10 2020-11-28T21:55:18.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.083+0000 7fc39f59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a00607a0 con 0x7fc3a00a8df0 2020-11-28T21:55:18.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.083+0000 7fc39ed9d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc3a0099b10 0x7fc3a0099f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:18.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.083+0000 7fc39ed9d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc3a0099b10 0x7fc3a0099f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44552/0 (socket says 172.21.15.162:44552) 2020-11-28T21:55:18.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.083+0000 7fc39ed9d700 1 -- 172.21.15.162:0/363085355 learned_addr learned my addr 172.21.15.162:0/363085355 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:18.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.083+0000 7fc39dd9b700 1 --2- 172.21.15.162:0/363085355 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc3a00a8df0 0x7fc3a0099620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:18.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.083+0000 7fc39e59c700 1 --2- 172.21.15.162:0/363085355 >> v2:172.21.15.118:3300/0 conn(0x7fc3a00afec0 0x7fc3a00b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:18.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.083+0000 7fc39ed9d700 1 -- 172.21.15.162:0/363085355 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc3a00a8df0 msgr2=0x7fc3a0099620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:18.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.083+0000 7fc39ed9d700 1 --2- 172.21.15.162:0/363085355 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc3a00a8df0 0x7fc3a0099620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:18.094 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.083+0000 7fc39ed9d700 1 -- 172.21.15.162:0/363085355 >> v2:172.21.15.118:3300/0 conn(0x7fc3a00afec0 msgr2=0x7fc3a00b02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:18.094 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.083+0000 7fc39ed9d700 1 --2- 172.21.15.162:0/363085355 >> v2:172.21.15.118:3300/0 conn(0x7fc3a00afec0 0x7fc3a00b02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:18.094 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.083+0000 7fc39ed9d700 1 -- 172.21.15.162:0/363085355 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a0060a40 con 0x7fc3a0099b10 2020-11-28T21:55:18.095 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.083+0000 7fc39ed9d700 1 --2- 172.21.15.162:0/363085355 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3a0099b10 0x7fc3a0099f80 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 rx=0x7fc398002f30 tx=0x7fc398006e80).ready entity=mon.1 client_cookie=9829742bb2708058 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:18.095 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.083+0000 7fc39d59a700 1 -- 172.21.15.162:0/363085355 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc398014070 con 0x7fc3a0099b10 2020-11-28T21:55:18.095 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.083+0000 7fc39d59a700 1 -- 172.21.15.162:0/363085355 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3980099e0 con 0x7fc3a0099b10 2020-11-28T21:55:18.095 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.083+0000 7fc39d59a700 1 -- 172.21.15.162:0/363085355 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc398009d40 con 0x7fc3a0099b10 2020-11-28T21:55:18.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.083+0000 7fc39f59e700 1 -- 172.21.15.162:0/363085355 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3a0099b10 msgr2=0x7fc3a0099f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:18.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.087+0000 7fc39f59e700 1 --2- 172.21.15.162:0/363085355 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3a0099b10 0x7fc3a0099f80 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 rx=0x7fc398002f30 tx=0x7fc398006e80).stop 2020-11-28T21:55:18.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.087+0000 7fc39f59e700 1 -- 172.21.15.162:0/363085355 shutdown_connections 2020-11-28T21:55:18.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.087+0000 7fc39f59e700 1 --2- 172.21.15.162:0/363085355 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3a0099b10 0x7fc3a0099f80 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:18.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.087+0000 7fc39f59e700 1 --2- 172.21.15.162:0/363085355 >> v2:172.21.15.118:3300/0 conn(0x7fc3a00afec0 0x7fc3a00b02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:18.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.087+0000 7fc39f59e700 1 --2- 172.21.15.162:0/363085355 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc3a00a8df0 0x7fc3a0099620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:18.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.087+0000 7fc39f59e700 1 -- 172.21.15.162:0/363085355 >> 172.21.15.162:0/363085355 conn(0x7fc3a00b5520 msgr2=0x7fc3a00b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:18.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.087+0000 7fc39f59e700 1 -- 172.21.15.162:0/363085355 shutdown_connections 2020-11-28T21:55:18.098 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.087+0000 7fc39f59e700 1 -- 172.21.15.162:0/363085355 wait complete. 2020-11-28T21:55:18.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.087+0000 7fc39f59e700 1 Processor -- start 2020-11-28T21:55:18.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.087+0000 7fc39f59e700 1 -- start start 2020-11-28T21:55:18.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.091+0000 7fc39f59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc3a0099b10 0x7fc3a0065cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:18.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.091+0000 7fc39f59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc3a00a8df0 0x7fc3a0140560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:18.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.091+0000 7fc39f59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3a00afec0 0x7fc3a00a92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:18.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.091+0000 7fc39f59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a013f210 con 0x7fc3a00a8df0 2020-11-28T21:55:18.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.091+0000 7fc39f59e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a0067a60 con 0x7fc3a00afec0 2020-11-28T21:55:18.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.091+0000 7fc39f59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a0067d00 con 0x7fc3a0099b10 2020-11-28T21:55:18.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.091+0000 7fc39ed9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3a00afec0 0x7fc3a00a92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:18.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.091+0000 7fc39ed9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3a00afec0 0x7fc3a00a92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44558/0 (socket says 172.21.15.162:44558) 2020-11-28T21:55:18.104 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.091+0000 7fc39ed9d700 1 -- 172.21.15.162:0/1774259225 learned_addr learned my addr 172.21.15.162:0/1774259225 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:18.104 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.091+0000 7fc39dd9b700 1 --2- 172.21.15.162:0/1774259225 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc3a00a8df0 0x7fc3a0140560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:18.104 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.091+0000 7fc39e59c700 1 --2- 172.21.15.162:0/1774259225 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc3a0099b10 0x7fc3a0065cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:18.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.091+0000 7fc39ed9d700 1 -- 172.21.15.162:0/1774259225 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc3a0099b10 msgr2=0x7fc3a0065cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:18.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.091+0000 7fc39ed9d700 1 --2- 172.21.15.162:0/1774259225 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc3a0099b10 0x7fc3a0065cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:18.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.091+0000 7fc39ed9d700 1 -- 172.21.15.162:0/1774259225 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc3a00a8df0 msgr2=0x7fc3a0140560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:18.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.091+0000 7fc39ed9d700 1 --2- 172.21.15.162:0/1774259225 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc3a00a8df0 0x7fc3a0140560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:18.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.091+0000 7fc39ed9d700 1 -- 172.21.15.162:0/1774259225 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a0152b70 con 0x7fc3a00afec0 2020-11-28T21:55:18.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.091+0000 7fc39ed9d700 1 --2- 172.21.15.162:0/1774259225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3a00afec0 0x7fc3a00a92d0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 rx=0x7fc398002f30 tx=0x7fc398016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:18.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.091+0000 7fc38f7fe700 1 -- 172.21.15.162:0/1774259225 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc398010d40 con 0x7fc3a00afec0 2020-11-28T21:55:18.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.091+0000 7fc38f7fe700 1 -- 172.21.15.162:0/1774259225 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc39801c440 con 0x7fc3a00afec0 2020-11-28T21:55:18.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.091+0000 7fc39f59e700 1 -- 172.21.15.162:0/1774259225 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3a0152e10 con 0x7fc3a00afec0 2020-11-28T21:55:18.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.091+0000 7fc38f7fe700 1 -- 172.21.15.162:0/1774259225 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc398010ea0 con 0x7fc3a00afec0 2020-11-28T21:55:18.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.091+0000 7fc39f59e700 1 -- 172.21.15.162:0/1774259225 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3a00676e0 con 0x7fc3a00afec0 2020-11-28T21:55:18.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.091+0000 7fc39f59e700 1 -- 172.21.15.162:0/1774259225 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3a0057050 con 0x7fc3a00afec0 2020-11-28T21:55:18.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.091+0000 7fc38f7fe700 1 -- 172.21.15.162:0/1774259225 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc39800e040 con 0x7fc3a00afec0 2020-11-28T21:55:18.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.095+0000 7fc38f7fe700 1 --2- 172.21.15.162:0/1774259225 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc37c055330 0x7fc37c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:18.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.095+0000 7fc38f7fe700 1 -- 172.21.15.162:0/1774259225 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5721+0+0 (secure 0 0 0) 0x7fc39806f9f0 con 0x7fc3a00afec0 2020-11-28T21:55:18.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.095+0000 7fc39e59c700 1 --2- 172.21.15.162:0/1774259225 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc37c055330 0x7fc37c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:18.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.095+0000 7fc39e59c700 1 --2- 172.21.15.162:0/1774259225 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc37c055330 0x7fc37c0577f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7fc39000c000 tx=0x7fc390008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:18.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.099+0000 7fc38f7fe700 1 -- 172.21.15.162:0/1774259225 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc398071e60 con 0x7fc3a00afec0 2020-11-28T21:55:18.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.243+0000 7fc39f59e700 1 -- 172.21.15.162:0/1774259225 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fc3a0152fa0 con 0x7fc3a00afec0 2020-11-28T21:55:18.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.939+0000 7fc38f7fe700 1 -- 172.21.15.162:0/1774259225 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v129) v1 ==== 133+0+0 (secure 0 0 0) 0x7fc398045e50 con 0x7fc3a00afec0 2020-11-28T21:55:18.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.947+0000 7fc39f59e700 1 -- 172.21.15.162:0/1774259225 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc37c055330 msgr2=0x7fc37c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:18.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.947+0000 7fc39f59e700 1 --2- 172.21.15.162:0/1774259225 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc37c055330 0x7fc37c0577f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7fc39000c000 tx=0x7fc390008040).stop 2020-11-28T21:55:18.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.947+0000 7fc39f59e700 1 -- 172.21.15.162:0/1774259225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3a00afec0 msgr2=0x7fc3a00a92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:18.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.947+0000 7fc39f59e700 1 --2- 172.21.15.162:0/1774259225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3a00afec0 0x7fc3a00a92d0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 rx=0x7fc398002f30 tx=0x7fc398016000).stop 2020-11-28T21:55:18.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.947+0000 7fc39f59e700 1 -- 172.21.15.162:0/1774259225 shutdown_connections 2020-11-28T21:55:18.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.947+0000 7fc39f59e700 1 --2- 172.21.15.162:0/1774259225 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc37c055330 0x7fc37c0577f0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:18.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.947+0000 7fc39f59e700 1 --2- 172.21.15.162:0/1774259225 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc3a0099b10 0x7fc3a0065cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:18.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.947+0000 7fc39f59e700 1 --2- 172.21.15.162:0/1774259225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3a00afec0 0x7fc3a00a92d0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:18.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.947+0000 7fc39f59e700 1 --2- 172.21.15.162:0/1774259225 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc3a00a8df0 0x7fc3a0140560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:18.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.947+0000 7fc39f59e700 1 -- 172.21.15.162:0/1774259225 >> 172.21.15.162:0/1774259225 conn(0x7fc3a00b5520 msgr2=0x7fc3a0144a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:18.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.947+0000 7fc39f59e700 1 -- 172.21.15.162:0/1774259225 shutdown_connections 2020-11-28T21:55:18.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:18.947+0000 7fc39f59e700 1 -- 172.21.15.162:0/1774259225 wait complete. 2020-11-28T21:55:18.953 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_metadata' created 2020-11-28T21:55:18.965 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data 2020-11-28T21:55:19.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.079+0000 7f659cad9700 1 Processor -- start 2020-11-28T21:55:19.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.079+0000 7f659cad9700 1 -- start start 2020-11-28T21:55:19.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.079+0000 7f659cad9700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6598066a50 0x7f65981409d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:19.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.079+0000 7f659cad9700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f65980b37d0 0x7f65980b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:19.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.079+0000 7f659cad9700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65980b1dd0 con 0x7f6598066e30 2020-11-28T21:55:19.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.079+0000 7f659cad9700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6598143f20 con 0x7f6598066a50 2020-11-28T21:55:19.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.079+0000 7f659cad9700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65980b2000 con 0x7f65980b37d0 2020-11-28T21:55:19.086 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.079+0000 7f659659c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6598066a50 0x7f65981409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:19.086 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.079+0000 7f659659c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6598066a50 0x7f65981409d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44562/0 (socket says 172.21.15.162:44562) 2020-11-28T21:55:19.086 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.079+0000 7f659659c700 1 -- 172.21.15.162:0/518542636 learned_addr learned my addr 172.21.15.162:0/518542636 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:19.086 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.079+0000 7f6596d9d700 1 --2- 172.21.15.162:0/518542636 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f65980b37d0 0x7f65980b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:19.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.079+0000 7f659659c700 1 -- 172.21.15.162:0/518542636 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f65980b37d0 msgr2=0x7f65980b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:19.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.079+0000 7f659659c700 1 --2- 172.21.15.162:0/518542636 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f65980b37d0 0x7f65980b3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:19.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.079+0000 7f659659c700 1 -- 172.21.15.162:0/518542636 >> v1:172.21.15.118:6789/0 conn(0x7f6598066e30 legacy=0x7f6598144620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:19.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.079+0000 7f659659c700 1 -- 172.21.15.162:0/518542636 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6598144180 con 0x7f6598066a50 2020-11-28T21:55:19.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.079+0000 7f659659c700 1 --2- 172.21.15.162:0/518542636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6598066a50 0x7f65981409d0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 rx=0x7f659805c9b0 tx=0x7f6588006e80).ready entity=mon.1 client_cookie=7e4719f3e2d7264e server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:19.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.079+0000 7f659559a700 1 -- 172.21.15.162:0/518542636 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65880097b0 con 0x7f6598066a50 2020-11-28T21:55:19.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.079+0000 7f659559a700 1 -- 172.21.15.162:0/518542636 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6588009910 con 0x7f6598066a50 2020-11-28T21:55:19.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.079+0000 7f659559a700 1 -- 172.21.15.162:0/518542636 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6588009c70 con 0x7f6598066a50 2020-11-28T21:55:19.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.079+0000 7f659cad9700 1 -- 172.21.15.162:0/518542636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6598066a50 msgr2=0x7f65981409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:19.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.079+0000 7f659cad9700 1 --2- 172.21.15.162:0/518542636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6598066a50 0x7f65981409d0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 rx=0x7f659805c9b0 tx=0x7f6588006e80).stop 2020-11-28T21:55:19.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.083+0000 7f659cad9700 1 -- 172.21.15.162:0/518542636 shutdown_connections 2020-11-28T21:55:19.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.083+0000 7f659cad9700 1 --2- 172.21.15.162:0/518542636 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f65980b37d0 0x7f65980b3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:19.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.083+0000 7f659cad9700 1 --2- 172.21.15.162:0/518542636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6598066a50 0x7f65981409d0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:19.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.083+0000 7f659cad9700 1 -- 172.21.15.162:0/518542636 >> 172.21.15.162:0/518542636 conn(0x7f659805b970 msgr2=0x7f659805bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:19.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.083+0000 7f659cad9700 1 -- 172.21.15.162:0/518542636 shutdown_connections 2020-11-28T21:55:19.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.083+0000 7f659cad9700 1 -- 172.21.15.162:0/518542636 wait complete. 2020-11-28T21:55:19.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.083+0000 7f659cad9700 1 Processor -- start 2020-11-28T21:55:19.094 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.083+0000 7f659cad9700 1 -- start start 2020-11-28T21:55:19.094 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.083+0000 7f659cad9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6598066a50 0x7f659809a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:19.094 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.083+0000 7f659cad9700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6598066e30 0x7f6598153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:19.094 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.083+0000 7f659cad9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65980b37d0 0x7f65980abe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:19.095 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.083+0000 7f659cad9700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659813d6a0 con 0x7f6598066a50 2020-11-28T21:55:19.095 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.083+0000 7f659cad9700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6598099cb0 con 0x7f65980b37d0 2020-11-28T21:55:19.095 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.083+0000 7f659cad9700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f659809bb80 con 0x7f6598066e30 2020-11-28T21:55:19.095 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.083+0000 7f6596d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65980b37d0 0x7f65980abe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:19.095 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.083+0000 7f6596d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65980b37d0 0x7f65980abe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44572/0 (socket says 172.21.15.162:44572) 2020-11-28T21:55:19.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.083+0000 7f6596d9d700 1 -- 172.21.15.162:0/3527979243 learned_addr learned my addr 172.21.15.162:0/3527979243 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:19.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.083+0000 7f659659c700 1 --2- 172.21.15.162:0/3527979243 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6598066a50 0x7f659809a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:19.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.083+0000 7f6595d9b700 1 --2- 172.21.15.162:0/3527979243 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6598066e30 0x7f6598153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:19.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.083+0000 7f6596d9d700 1 -- 172.21.15.162:0/3527979243 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6598066e30 msgr2=0x7f6598153730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:19.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.083+0000 7f6596d9d700 1 --2- 172.21.15.162:0/3527979243 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6598066e30 0x7f6598153730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:19.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.083+0000 7f6596d9d700 1 -- 172.21.15.162:0/3527979243 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6598066a50 msgr2=0x7f659809a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:19.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.083+0000 7f6596d9d700 1 --2- 172.21.15.162:0/3527979243 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6598066a50 0x7f659809a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:19.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.087+0000 7f6596d9d700 1 -- 172.21.15.162:0/3527979243 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6598099a80 con 0x7f65980b37d0 2020-11-28T21:55:19.098 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.087+0000 7f6596d9d700 1 --2- 172.21.15.162:0/3527979243 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65980b37d0 0x7f65980abe40 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 rx=0x7f6590006dd0 tx=0x7f65900091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:19.098 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.087+0000 7f65877fe700 1 -- 172.21.15.162:0/3527979243 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f659000ca00 con 0x7f65980b37d0 2020-11-28T21:55:19.098 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.087+0000 7f65877fe700 1 -- 172.21.15.162:0/3527979243 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659000cb60 con 0x7f65980b37d0 2020-11-28T21:55:19.098 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.087+0000 7f659cad9700 1 -- 172.21.15.162:0/3527979243 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f659809cea0 con 0x7f65980b37d0 2020-11-28T21:55:19.098 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.087+0000 7f65877fe700 1 -- 172.21.15.162:0/3527979243 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65900123f0 con 0x7f65980b37d0 2020-11-28T21:55:19.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.087+0000 7f659cad9700 1 -- 172.21.15.162:0/3527979243 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f659809a980 con 0x7f65980b37d0 2020-11-28T21:55:19.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.087+0000 7f659cad9700 1 -- 172.21.15.162:0/3527979243 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6598057050 con 0x7f65980b37d0 2020-11-28T21:55:19.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.087+0000 7f65877fe700 1 -- 172.21.15.162:0/3527979243 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6590022070 con 0x7f65980b37d0 2020-11-28T21:55:19.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.087+0000 7f65877fe700 1 --2- 172.21.15.162:0/3527979243 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6574055500 0x7f65740579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:19.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.087+0000 7f65877fe700 1 -- 172.21.15.162:0/3527979243 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6075+0+0 (secure 0 0 0) 0x7f659006e630 con 0x7f65980b37d0 2020-11-28T21:55:19.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.087+0000 7f659659c700 1 --2- 172.21.15.162:0/3527979243 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6574055500 0x7f65740579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:19.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.087+0000 7f659659c700 1 --2- 172.21.15.162:0/3527979243 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6574055500 0x7f65740579c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7f658801a000 tx=0x7f6588015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:19.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.095+0000 7f65877fe700 1 -- 172.21.15.162:0/3527979243 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6590044890 con 0x7f65980b37d0 2020-11-28T21:55:19.284 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.279+0000 7f659cad9700 1 -- 172.21.15.162:0/3527979243 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6598067a80 con 0x7f65980b37d0 2020-11-28T21:55:19.286 INFO:tasks.ceph.mon.b.smithi118.stderr:2020-11-28T21:55:19.284+0000 7ff4ded1b700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-11-28T21:55:19.298 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.295+0000 7f65877fe700 1 -- 172.21.15.162:0/3527979243 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 24 and data pool 22 v62) v1 ==== 174+0+0 (secure 0 0 0) 0x7f6590016020 con 0x7f65980b37d0 2020-11-28T21:55:19.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.299+0000 7f659cad9700 1 -- 172.21.15.162:0/3527979243 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6574055500 msgr2=0x7f65740579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:19.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.299+0000 7f659cad9700 1 --2- 172.21.15.162:0/3527979243 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6574055500 0x7f65740579c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7f658801a000 tx=0x7f6588015000).stop 2020-11-28T21:55:19.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.299+0000 7f659cad9700 1 -- 172.21.15.162:0/3527979243 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65980b37d0 msgr2=0x7f65980abe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:19.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.299+0000 7f659cad9700 1 --2- 172.21.15.162:0/3527979243 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65980b37d0 0x7f65980abe40 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 rx=0x7f6590006dd0 tx=0x7f65900091a0).stop 2020-11-28T21:55:19.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.299+0000 7f659cad9700 1 -- 172.21.15.162:0/3527979243 shutdown_connections 2020-11-28T21:55:19.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.299+0000 7f659cad9700 1 --2- 172.21.15.162:0/3527979243 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6574055500 0x7f65740579c0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:19.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.299+0000 7f659cad9700 1 --2- 172.21.15.162:0/3527979243 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65980b37d0 0x7f65980abe40 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:19.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.299+0000 7f659cad9700 1 --2- 172.21.15.162:0/3527979243 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6598066a50 0x7f659809a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:19.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.299+0000 7f659cad9700 1 --2- 172.21.15.162:0/3527979243 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6598066e30 0x7f6598153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:19.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.299+0000 7f659cad9700 1 -- 172.21.15.162:0/3527979243 >> 172.21.15.162:0/3527979243 conn(0x7f659805b970 msgr2=0x7f6598144540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:19.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.299+0000 7f659cad9700 1 -- 172.21.15.162:0/3527979243 shutdown_connections 2020-11-28T21:55:19.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.299+0000 7f659cad9700 1 -- 172.21.15.162:0/3527979243 wait complete. 2020-11-28T21:55:19.305 INFO:teuthology.orchestra.run.smithi162.stderr:new fs with metadata pool 24 and data pool 22 2020-11-28T21:55:19.316 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-28T21:55:19.316 INFO:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:55:19.335 INFO:teuthology.orchestra.run.smithi118:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:55:19.350 INFO:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:55:19.366 INFO:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:55:19.385 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-28T21:55:19.385 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:19.386 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:55:19.386 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:55:19.434 INFO:teuthology.orchestra.run.smithi192.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2020-11-28T21:55:19.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:55:19.436 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2020-11-28T21:55:19.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-28T21:55:19.437 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:55:19.481 INFO:teuthology.orchestra.run.smithi192.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2020-11-28T21:55:19.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:55:19.483 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:19.483 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:55:19.484 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:55:19.490 INFO:teuthology.orchestra.run.smithi096.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-11-28T21:55:19.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:55:19.492 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2020-11-28T21:55:19.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-28T21:55:19.493 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:55:19.542 INFO:teuthology.orchestra.run.smithi096.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-11-28T21:55:19.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:55:19.544 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config rm mon mon_allow_pool_delete 2020-11-28T21:55:19.667 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.663+0000 7f12b48e0700 1 Processor -- start 2020-11-28T21:55:19.668 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.663+0000 7f12b48e0700 1 -- start start 2020-11-28T21:55:19.669 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.663+0000 7f12b48e0700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12b0143530 0x7f12b00b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:19.669 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.663+0000 7f12b48e0700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f12b013c630 0x7f12b013cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:19.669 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.663+0000 7f12b48e0700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12b00b0e50 con 0x7f12b005bcb0 2020-11-28T21:55:19.669 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.663+0000 7f12b48e0700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12b005fe40 con 0x7f12b013c630 2020-11-28T21:55:19.670 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.663+0000 7f12b48e0700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b005c090 con 0x7f12b0143530 2020-11-28T21:55:19.671 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.667+0000 7f12aeffd700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12b0143530 0x7f12b00b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:19.672 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.667+0000 7f12affff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f12b013c630 0x7f12b013cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:19.672 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.667+0000 7f12aeffd700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12b0143530 0x7f12b00b1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:41802/0 (socket says 172.21.15.162:41802) 2020-11-28T21:55:19.672 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.667+0000 7f12aeffd700 1 -- 172.21.15.162:0/1726254854 learned_addr learned my addr 172.21.15.162:0/1726254854 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:19.672 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.667+0000 7f12af7fe700 1 --1- 172.21.15.162:0/1726254854 >> v1:172.21.15.118:6789/0 conn(0x7f12b005bcb0 0x7f12b00b0740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.162:41568/0 (socket says 172.21.15.162:41568) 2020-11-28T21:55:19.672 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.667+0000 7f12aeffd700 1 -- 172.21.15.162:0/1726254854 >> v2:172.21.15.118:3300/0 conn(0x7f12b013c630 msgr2=0x7f12b013cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:19.673 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.667+0000 7f12aeffd700 1 --2- 172.21.15.162:0/1726254854 >> v2:172.21.15.118:3300/0 conn(0x7f12b013c630 0x7f12b013cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:19.673 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.667+0000 7f12aeffd700 1 -- 172.21.15.162:0/1726254854 >> v1:172.21.15.118:6789/0 conn(0x7f12b005bcb0 legacy=0x7f12b00b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:19.673 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.667+0000 7f12aeffd700 1 -- 172.21.15.162:0/1726254854 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b00b22a0 con 0x7f12b0143530 2020-11-28T21:55:19.673 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.667+0000 7f12aeffd700 1 --2- 172.21.15.162:0/1726254854 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12b0143530 0x7f12b00b1c00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f1298002f30 tx=0x7f1298006e80).ready entity=mon.2 client_cookie=92082fdf66df9671 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:19.674 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.667+0000 7f12ae7fc700 1 -- 172.21.15.162:0/1726254854 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12980097f0 con 0x7f12b0143530 2020-11-28T21:55:19.674 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.667+0000 7f12ae7fc700 1 -- 172.21.15.162:0/1726254854 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1298009950 con 0x7f12b0143530 2020-11-28T21:55:19.674 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.667+0000 7f12ae7fc700 1 -- 172.21.15.162:0/1726254854 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1298009cb0 con 0x7f12b0143530 2020-11-28T21:55:19.674 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.667+0000 7f12b48e0700 1 -- 172.21.15.162:0/1726254854 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12b0143530 msgr2=0x7f12b00b1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:19.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.667+0000 7f12b48e0700 1 --2- 172.21.15.162:0/1726254854 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12b0143530 0x7f12b00b1c00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f1298002f30 tx=0x7f1298006e80).stop 2020-11-28T21:55:19.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.667+0000 7f12b48e0700 1 -- 172.21.15.162:0/1726254854 shutdown_connections 2020-11-28T21:55:19.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.667+0000 7f12b48e0700 1 --2- 172.21.15.162:0/1726254854 >> v2:172.21.15.118:3300/0 conn(0x7f12b013c630 0x7f12b013cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:19.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.667+0000 7f12b48e0700 1 --2- 172.21.15.162:0/1726254854 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12b0143530 0x7f12b00b1c00 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:19.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.667+0000 7f12b48e0700 1 -- 172.21.15.162:0/1726254854 >> 172.21.15.162:0/1726254854 conn(0x7f12b00b5da0 msgr2=0x7f12b00b6180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:19.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.667+0000 7f12b48e0700 1 -- 172.21.15.162:0/1726254854 shutdown_connections 2020-11-28T21:55:19.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.667+0000 7f12b48e0700 1 -- 172.21.15.162:0/1726254854 wait complete. 2020-11-28T21:55:19.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.667+0000 7f12b48e0700 1 Processor -- start 2020-11-28T21:55:19.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.667+0000 7f12b48e0700 1 -- start start 2020-11-28T21:55:19.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.671+0000 7f12b48e0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12b005bcb0 0x7f12b0146a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:19.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.671+0000 7f12b48e0700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f12b013c630 0x7f12b0148dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:19.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.671+0000 7f12b48e0700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12b0143530 0x7f12b00ad170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:19.680 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.671+0000 7f12b48e0700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b00ae760 con 0x7f12b013c630 2020-11-28T21:55:19.680 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.671+0000 7f12b48e0700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b00aef70 con 0x7f12b005bcb0 2020-11-28T21:55:19.680 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.671+0000 7f12b48e0700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b0140180 con 0x7f12b0143530 2020-11-28T21:55:19.680 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.671+0000 7f12af7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12b005bcb0 0x7f12b0146a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:19.681 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.671+0000 7f12af7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12b005bcb0 0x7f12b0146a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44582/0 (socket says 172.21.15.162:44582) 2020-11-28T21:55:19.681 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.671+0000 7f12af7fe700 1 -- 172.21.15.162:0/1794762691 learned_addr learned my addr 172.21.15.162:0/1794762691 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:19.681 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.671+0000 7f12affff700 1 --2- 172.21.15.162:0/1794762691 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12b0143530 0x7f12b00ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:19.681 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.671+0000 7f12aeffd700 1 --2- 172.21.15.162:0/1794762691 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f12b013c630 0x7f12b0148dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:19.682 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.671+0000 7f12af7fe700 1 -- 172.21.15.162:0/1794762691 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12b0143530 msgr2=0x7f12b00ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:19.682 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.671+0000 7f12af7fe700 1 --2- 172.21.15.162:0/1794762691 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12b0143530 0x7f12b00ad170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:19.682 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.671+0000 7f12af7fe700 1 -- 172.21.15.162:0/1794762691 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f12b013c630 msgr2=0x7f12b0148dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:19.682 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.671+0000 7f12af7fe700 1 --2- 172.21.15.162:0/1794762691 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f12b013c630 0x7f12b0148dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:19.682 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.671+0000 7f12af7fe700 1 -- 172.21.15.162:0/1794762691 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b0140420 con 0x7f12b005bcb0 2020-11-28T21:55:19.683 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.671+0000 7f12af7fe700 1 --2- 172.21.15.162:0/1794762691 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12b005bcb0 0x7f12b0146a70 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 rx=0x7f12b00b6b90 tx=0x7f12a0007f90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:19.683 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.671+0000 7f12acff9700 1 -- 172.21.15.162:0/1794762691 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12a0005ab0 con 0x7f12b005bcb0 2020-11-28T21:55:19.683 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.671+0000 7f12b48e0700 1 -- 172.21.15.162:0/1794762691 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12b0144f30 con 0x7f12b005bcb0 2020-11-28T21:55:19.683 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.671+0000 7f12acff9700 1 -- 172.21.15.162:0/1794762691 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a0005c10 con 0x7f12b005bcb0 2020-11-28T21:55:19.683 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.671+0000 7f12acff9700 1 -- 172.21.15.162:0/1794762691 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12a0007110 con 0x7f12b005bcb0 2020-11-28T21:55:19.684 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.671+0000 7f12b48e0700 1 -- 172.21.15.162:0/1794762691 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12b013fe20 con 0x7f12b005bcb0 2020-11-28T21:55:19.684 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.671+0000 7f12b48e0700 1 -- 172.21.15.162:0/1794762691 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12b0145590 con 0x7f12b005bcb0 2020-11-28T21:55:19.684 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.675+0000 7f12acff9700 1 -- 172.21.15.162:0/1794762691 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f12a0010030 con 0x7f12b005bcb0 2020-11-28T21:55:19.684 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.675+0000 7f12acff9700 1 --2- 172.21.15.162:0/1794762691 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f12940553b0 0x7f1294057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:19.684 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.675+0000 7f12acff9700 1 -- 172.21.15.162:0/1794762691 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f12a006e0e0 con 0x7f12b005bcb0 2020-11-28T21:55:19.685 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.675+0000 7f12aeffd700 1 --2- 172.21.15.162:0/1794762691 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f12940553b0 0x7f1294057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:19.685 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.675+0000 7f12aeffd700 1 --2- 172.21.15.162:0/1794762691 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f12940553b0 0x7f1294057870 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7f129801c000 tx=0x7f1298015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:19.686 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.679+0000 7f12acff9700 1 -- 172.21.15.162:0/1794762691 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f12a00442b0 con 0x7f12b005bcb0 2020-11-28T21:55:19.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.827+0000 7f12b48e0700 1 -- 172.21.15.162:0/1794762691 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f12b0057050 con 0x7f12b005bcb0 2020-11-28T21:55:19.840 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.835+0000 7f12acff9700 1 -- 172.21.15.162:0/1794762691 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f12a00478d0 con 0x7f12b005bcb0 2020-11-28T21:55:19.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.839+0000 7f12b48e0700 1 -- 172.21.15.162:0/1794762691 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f12940553b0 msgr2=0x7f1294057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:19.843 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.839+0000 7f12b48e0700 1 --2- 172.21.15.162:0/1794762691 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f12940553b0 0x7f1294057870 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7f129801c000 tx=0x7f1298015000).stop 2020-11-28T21:55:19.843 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.839+0000 7f12b48e0700 1 -- 172.21.15.162:0/1794762691 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12b005bcb0 msgr2=0x7f12b0146a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:19.843 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.839+0000 7f12b48e0700 1 --2- 172.21.15.162:0/1794762691 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12b005bcb0 0x7f12b0146a70 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 rx=0x7f12b00b6b90 tx=0x7f12a0007f90).stop 2020-11-28T21:55:19.844 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.839+0000 7f12b48e0700 1 -- 172.21.15.162:0/1794762691 shutdown_connections 2020-11-28T21:55:19.844 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.839+0000 7f12b48e0700 1 --2- 172.21.15.162:0/1794762691 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f12940553b0 0x7f1294057870 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:19.844 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.839+0000 7f12b48e0700 1 --2- 172.21.15.162:0/1794762691 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12b0143530 0x7f12b00ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:19.844 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.839+0000 7f12b48e0700 1 --2- 172.21.15.162:0/1794762691 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12b005bcb0 0x7f12b0146a70 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:19.844 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.839+0000 7f12b48e0700 1 --2- 172.21.15.162:0/1794762691 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f12b013c630 0x7f12b0148dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:19.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.839+0000 7f12b48e0700 1 -- 172.21.15.162:0/1794762691 >> 172.21.15.162:0/1794762691 conn(0x7f12b00b5da0 msgr2=0x7f12b01412a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:19.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.839+0000 7f12b48e0700 1 -- 172.21.15.162:0/1794762691 shutdown_connections 2020-11-28T21:55:19.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.839+0000 7f12b48e0700 1 -- 172.21.15.162:0/1794762691 wait complete. 2020-11-28T21:55:19.857 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Ended test tasks.cephfs.test_misc.TestMisc.test_fs_new' 2020-11-28T21:55:19.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.963+0000 7fd2d2f9a700 1 Processor -- start 2020-11-28T21:55:19.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.963+0000 7fd2d2f9a700 1 -- start start 2020-11-28T21:55:19.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.963+0000 7fd2d2f9a700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2cc1431f0 0x7fd2cc067090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:19.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.963+0000 7fd2d2f9a700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2cc0b1c00 con 0x7fd2cc0b3cf0 2020-11-28T21:55:19.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.963+0000 7fd2d2f9a700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2cc0b6460 con 0x7fd2cc066cb0 2020-11-28T21:55:19.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.963+0000 7fd2d2f9a700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc143670 con 0x7fd2cc1431f0 2020-11-28T21:55:19.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.963+0000 7fd2d0d36700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fd2cc066cb0 0x7fd2cc0b5d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51434/0 (socket says 172.21.15.162:51434) 2020-11-28T21:55:19.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.963+0000 7fd2d0d36700 1 -- 172.21.15.162:0/3461565472 learned_addr learned my addr 172.21.15.162:0/3461565472 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:19.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.963+0000 7fd2d1537700 1 --2- 172.21.15.162:0/3461565472 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2cc1431f0 0x7fd2cc067090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:19.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.963+0000 7fd2cb7fe700 1 -- 172.21.15.162:0/3461565472 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 291832010 0 0) 0x7fd2cc0b6460 con 0x7fd2cc066cb0 2020-11-28T21:55:19.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.963+0000 7fd2cb7fe700 1 -- 172.21.15.162:0/3461565472 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2ac003230 con 0x7fd2cc066cb0 2020-11-28T21:55:19.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.963+0000 7fd2cb7fe700 1 -- 172.21.15.162:0/3461565472 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 35446676 0 0) 0x7fd2cc0b1c00 con 0x7fd2cc0b3cf0 2020-11-28T21:55:19.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.963+0000 7fd2cb7fe700 1 -- 172.21.15.162:0/3461565472 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2cc0b6460 con 0x7fd2cc0b3cf0 2020-11-28T21:55:19.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.963+0000 7fd2d1537700 1 -- 172.21.15.162:0/3461565472 >> v1:172.21.15.162:6789/0 conn(0x7fd2cc066cb0 legacy=0x7fd2cc0b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:19.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.963+0000 7fd2d1537700 1 -- 172.21.15.162:0/3461565472 >> v1:172.21.15.118:6789/0 conn(0x7fd2cc0b3cf0 legacy=0x7fd2cc142ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:19.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.963+0000 7fd2d1537700 1 -- 172.21.15.162:0/3461565472 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2cc143910 con 0x7fd2cc1431f0 2020-11-28T21:55:19.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.963+0000 7fd2d1537700 1 --2- 172.21.15.162:0/3461565472 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2cc1431f0 0x7fd2cc067090 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7fd2c40032b0 tx=0x7fd2c40091a0).ready entity=mon.2 client_cookie=1039105763dbd216 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:19.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.967+0000 7fd2cb7fe700 1 -- 172.21.15.162:0/3461565472 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2c400b940 con 0x7fd2cc1431f0 2020-11-28T21:55:19.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.967+0000 7fd2cb7fe700 1 -- 172.21.15.162:0/3461565472 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c400baa0 con 0x7fd2cc1431f0 2020-11-28T21:55:19.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.967+0000 7fd2cb7fe700 1 -- 172.21.15.162:0/3461565472 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2c400bc10 con 0x7fd2cc1431f0 2020-11-28T21:55:19.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.967+0000 7fd2d2f9a700 1 -- 172.21.15.162:0/3461565472 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2cc1431f0 msgr2=0x7fd2cc067090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:19.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.967+0000 7fd2d2f9a700 1 --2- 172.21.15.162:0/3461565472 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2cc1431f0 0x7fd2cc067090 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7fd2c40032b0 tx=0x7fd2c40091a0).stop 2020-11-28T21:55:19.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.967+0000 7fd2d2f9a700 1 -- 172.21.15.162:0/3461565472 shutdown_connections 2020-11-28T21:55:19.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.967+0000 7fd2d2f9a700 1 --2- 172.21.15.162:0/3461565472 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2cc1431f0 0x7fd2cc067090 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:19.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.967+0000 7fd2d2f9a700 1 -- 172.21.15.162:0/3461565472 >> 172.21.15.162:0/3461565472 conn(0x7fd2cc05fe10 msgr2=0x7fd2cc06aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:19.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.967+0000 7fd2d2f9a700 1 -- 172.21.15.162:0/3461565472 shutdown_connections 2020-11-28T21:55:19.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.967+0000 7fd2d2f9a700 1 -- 172.21.15.162:0/3461565472 wait complete. 2020-11-28T21:55:19.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.967+0000 7fd2d2f9a700 1 Processor -- start 2020-11-28T21:55:19.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.967+0000 7fd2d2f9a700 1 -- start start 2020-11-28T21:55:19.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.971+0000 7fd2d2f9a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2cc066cb0 0x7fd2cc1550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:19.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.971+0000 7fd2d2f9a700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2cc0b3cf0 0x7fd2cc144750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:19.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.971+0000 7fd2d2f9a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2cc1431f0 0x7fd2cc144c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:19.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.971+0000 7fd2d2f9a700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc13d6a0 con 0x7fd2cc1431f0 2020-11-28T21:55:19.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.971+0000 7fd2d2f9a700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc155590 con 0x7fd2cc066cb0 2020-11-28T21:55:19.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.971+0000 7fd2d2f9a700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc0b0120 con 0x7fd2cc0b3cf0 2020-11-28T21:55:19.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.971+0000 7fd2d0d36700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2cc066cb0 0x7fd2cc1550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:19.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.971+0000 7fd2d0d36700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2cc066cb0 0x7fd2cc1550a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44596/0 (socket says 172.21.15.162:44596) 2020-11-28T21:55:19.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.971+0000 7fd2d0d36700 1 -- 172.21.15.162:0/555235396 learned_addr learned my addr 172.21.15.162:0/555235396 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:19.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.971+0000 7fd2cbfff700 1 --2- 172.21.15.162:0/555235396 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2cc0b3cf0 0x7fd2cc144750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:19.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.971+0000 7fd2d1537700 1 --2- 172.21.15.162:0/555235396 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2cc1431f0 0x7fd2cc144c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:19.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.971+0000 7fd2d0d36700 1 -- 172.21.15.162:0/555235396 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2cc0b3cf0 msgr2=0x7fd2cc144750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:19.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.971+0000 7fd2d0d36700 1 --2- 172.21.15.162:0/555235396 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2cc0b3cf0 0x7fd2cc144750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:19.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.971+0000 7fd2d0d36700 1 -- 172.21.15.162:0/555235396 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2cc1431f0 msgr2=0x7fd2cc144c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:19.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.971+0000 7fd2d0d36700 1 --2- 172.21.15.162:0/555235396 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2cc1431f0 0x7fd2cc144c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:19.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.971+0000 7fd2d0d36700 1 -- 172.21.15.162:0/555235396 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2cc13d160 con 0x7fd2cc066cb0 2020-11-28T21:55:19.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.971+0000 7fd2cbfff700 1 --2- 172.21.15.162:0/555235396 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2cc0b3cf0 0x7fd2cc144750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:55:19.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.971+0000 7fd2d1537700 1 --2- 172.21.15.162:0/555235396 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2cc1431f0 0x7fd2cc144c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:55:19.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.971+0000 7fd2d0d36700 1 --2- 172.21.15.162:0/555235396 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2cc066cb0 0x7fd2cc1550a0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 rx=0x7fd2bc001110 tx=0x7fd2bc007a90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:19.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.971+0000 7fd2c9ffb700 1 -- 172.21.15.162:0/555235396 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2bc0072e0 con 0x7fd2cc066cb0 2020-11-28T21:55:19.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.971+0000 7fd2c9ffb700 1 -- 172.21.15.162:0/555235396 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2bc007440 con 0x7fd2cc066cb0 2020-11-28T21:55:19.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.971+0000 7fd2d2f9a700 1 -- 172.21.15.162:0/555235396 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2cc13d400 con 0x7fd2cc066cb0 2020-11-28T21:55:19.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.971+0000 7fd2c9ffb700 1 -- 172.21.15.162:0/555235396 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2bc0057d0 con 0x7fd2cc066cb0 2020-11-28T21:55:19.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.971+0000 7fd2d2f9a700 1 -- 172.21.15.162:0/555235396 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2cc0afc80 con 0x7fd2cc066cb0 2020-11-28T21:55:19.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.971+0000 7fd2d2f9a700 1 -- 172.21.15.162:0/555235396 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2cc155720 con 0x7fd2cc066cb0 2020-11-28T21:55:19.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.971+0000 7fd2c9ffb700 1 -- 172.21.15.162:0/555235396 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd2bc01a060 con 0x7fd2cc066cb0 2020-11-28T21:55:19.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.975+0000 7fd2c9ffb700 1 --2- 172.21.15.162:0/555235396 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd2b0055440 0x7fd2b0057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:19.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.975+0000 7fd2c9ffb700 1 -- 172.21.15.162:0/555235396 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fd2bc06d450 con 0x7fd2cc066cb0 2020-11-28T21:55:19.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.975+0000 7fd2cbfff700 1 --2- 172.21.15.162:0/555235396 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd2b0055440 0x7fd2b0057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:19.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.975+0000 7fd2cbfff700 1 --2- 172.21.15.162:0/555235396 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd2b0055440 0x7fd2b0057900 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7fd2b400d000 tx=0x7fd2b40079d0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:19.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:19.979+0000 7fd2c9ffb700 1 -- 172.21.15.162:0/555235396 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd2cc155720 con 0x7fd2cc066cb0 2020-11-28T21:55:20.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:20.179+0000 7fd2d2f9a700 1 -- 172.21.15.162:0/555235396 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_fs_new"]} v 0) v1 -- 0x7fd2cc1556d0 con 0x7fd2cc066cb0 2020-11-28T21:55:20.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:20.955+0000 7fd2c9ffb700 1 -- 172.21.15.162:0/555235396 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_fs_new"]}]=0 v400) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd2bc018020 con 0x7fd2cc066cb0 2020-11-28T21:55:20.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:20.959+0000 7fd2d2f9a700 1 -- 172.21.15.162:0/555235396 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd2b0055440 msgr2=0x7fd2b0057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:20.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:20.959+0000 7fd2d2f9a700 1 --2- 172.21.15.162:0/555235396 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd2b0055440 0x7fd2b0057900 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7fd2b400d000 tx=0x7fd2b40079d0).stop 2020-11-28T21:55:20.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:20.959+0000 7fd2d2f9a700 1 -- 172.21.15.162:0/555235396 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2cc066cb0 msgr2=0x7fd2cc1550a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:20.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:20.959+0000 7fd2d2f9a700 1 --2- 172.21.15.162:0/555235396 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2cc066cb0 0x7fd2cc1550a0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 rx=0x7fd2bc001110 tx=0x7fd2bc007a90).stop 2020-11-28T21:55:20.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:20.959+0000 7fd2d2f9a700 1 -- 172.21.15.162:0/555235396 shutdown_connections 2020-11-28T21:55:20.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:20.959+0000 7fd2d2f9a700 1 --2- 172.21.15.162:0/555235396 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd2b0055440 0x7fd2b0057900 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:20.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:20.959+0000 7fd2d2f9a700 1 --2- 172.21.15.162:0/555235396 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2cc1431f0 0x7fd2cc144c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:20.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:20.959+0000 7fd2d2f9a700 1 --2- 172.21.15.162:0/555235396 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2cc066cb0 0x7fd2cc1550a0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:20.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:20.959+0000 7fd2d2f9a700 1 --2- 172.21.15.162:0/555235396 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2cc0b3cf0 0x7fd2cc144750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:20.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:20.959+0000 7fd2d2f9a700 1 -- 172.21.15.162:0/555235396 >> 172.21.15.162:0/555235396 conn(0x7fd2cc05fe10 msgr2=0x7fd2cc149280 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:20.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:20.959+0000 7fd2d2f9a700 1 -- 172.21.15.162:0/555235396 shutdown_connections 2020-11-28T21:55:20.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:20.959+0000 7fd2d2f9a700 1 -- 172.21.15.162:0/555235396 wait complete. 2020-11-28T21:55:20.980 INFO:tasks.cephfs_test_runner:test_fs_new (tasks.cephfs.test_misc.TestMisc) ... ok 2020-11-28T21:55:20.980 INFO:tasks.cephfs_test_runner:Starting test: test_getattr_caps (tasks.cephfs.test_misc.TestMisc) 2020-11-28T21:55:20.981 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_misc.TestMisc.test_getattr_caps' 2020-11-28T21:55:21.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.095+0000 7f5f994ff700 1 Processor -- start 2020-11-28T21:55:21.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.095+0000 7f5f994ff700 1 -- start start 2020-11-28T21:55:21.104 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.095+0000 7f5f994ff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5f940aa7d0 0x7f5f940b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:21.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.095+0000 7f5f994ff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5f940aabb0 0x7f5f94067930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:21.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.095+0000 7f5f994ff700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f94065b00 con 0x7f5f940b1c20 2020-11-28T21:55:21.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.095+0000 7f5f994ff700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f9413e7f0 con 0x7f5f940aa7d0 2020-11-28T21:55:21.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.095+0000 7f5f994ff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f94066270 con 0x7f5f940aabb0 2020-11-28T21:55:21.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.099+0000 7f5f92ffd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5f940aa7d0 0x7f5f940b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:21.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.099+0000 7f5f92ffd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5f940aa7d0 0x7f5f940b2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44604/0 (socket says 172.21.15.162:44604) 2020-11-28T21:55:21.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.099+0000 7f5f92ffd700 1 -- 172.21.15.162:0/222840942 learned_addr learned my addr 172.21.15.162:0/222840942 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:21.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.099+0000 7f5f927fc700 1 --1- 172.21.15.162:0/222840942 >> v1:172.21.15.162:6789/0 conn(0x7f5f940b1c20 0x7f5f94140ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51450/0 (socket says 172.21.15.162:51450) 2020-11-28T21:55:21.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.099+0000 7f5f937fe700 1 --2- 172.21.15.162:0/222840942 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5f940aabb0 0x7f5f94067930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:21.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.099+0000 7f5f92ffd700 1 -- 172.21.15.162:0/222840942 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5f940aabb0 msgr2=0x7f5f94067930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:21.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.099+0000 7f5f92ffd700 1 --2- 172.21.15.162:0/222840942 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5f940aabb0 0x7f5f94067930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:21.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.099+0000 7f5f92ffd700 1 -- 172.21.15.162:0/222840942 >> v1:172.21.15.162:6789/0 conn(0x7f5f940b1c20 legacy=0x7f5f94140ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:21.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.099+0000 7f5f92ffd700 1 -- 172.21.15.162:0/222840942 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f94067e20 con 0x7f5f940aa7d0 2020-11-28T21:55:21.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.099+0000 7f5f937fe700 1 --2- 172.21.15.162:0/222840942 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5f940aabb0 0x7f5f94067930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:55:21.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.099+0000 7f5f92ffd700 1 --2- 172.21.15.162:0/222840942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f940aa7d0 0x7f5f940b2370 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 rx=0x7f5f940663b0 tx=0x7f5f84009270).ready entity=mon.1 client_cookie=47306e53dc75ee41 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:21.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.099+0000 7f5f91ffb700 1 -- 172.21.15.162:0/222840942 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f8400bbc0 con 0x7f5f940aa7d0 2020-11-28T21:55:21.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.099+0000 7f5f91ffb700 1 -- 172.21.15.162:0/222840942 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f8400bd20 con 0x7f5f940aa7d0 2020-11-28T21:55:21.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.099+0000 7f5f91ffb700 1 -- 172.21.15.162:0/222840942 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f8400be90 con 0x7f5f940aa7d0 2020-11-28T21:55:21.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.099+0000 7f5f994ff700 1 -- 172.21.15.162:0/222840942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f940aa7d0 msgr2=0x7f5f940b2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:21.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.099+0000 7f5f994ff700 1 --2- 172.21.15.162:0/222840942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f940aa7d0 0x7f5f940b2370 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 rx=0x7f5f940663b0 tx=0x7f5f84009270).stop 2020-11-28T21:55:21.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.099+0000 7f5f994ff700 1 -- 172.21.15.162:0/222840942 shutdown_connections 2020-11-28T21:55:21.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.099+0000 7f5f994ff700 1 --2- 172.21.15.162:0/222840942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f940aa7d0 0x7f5f940b2370 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:21.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.099+0000 7f5f994ff700 1 --2- 172.21.15.162:0/222840942 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5f940aabb0 0x7f5f94067930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:21.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.099+0000 7f5f994ff700 1 -- 172.21.15.162:0/222840942 >> 172.21.15.162:0/222840942 conn(0x7f5f94062b50 msgr2=0x7f5f94062f30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:21.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.099+0000 7f5f994ff700 1 -- 172.21.15.162:0/222840942 shutdown_connections 2020-11-28T21:55:21.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.099+0000 7f5f994ff700 1 -- 172.21.15.162:0/222840942 wait complete. 2020-11-28T21:55:21.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.103+0000 7f5f994ff700 1 Processor -- start 2020-11-28T21:55:21.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.103+0000 7f5f994ff700 1 -- start start 2020-11-28T21:55:21.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.103+0000 7f5f994ff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f940aa7d0 0x7f5f940ad340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:21.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.103+0000 7f5f994ff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5f940aabb0 0x7f5f9413d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:21.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.103+0000 7f5f994ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f940b1c20 0x7f5f94143a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:21.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.103+0000 7f5f994ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f940ac5b0 con 0x7f5f940b1c20 2020-11-28T21:55:21.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.103+0000 7f5f994ff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f940acd90 con 0x7f5f940aa7d0 2020-11-28T21:55:21.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.103+0000 7f5f994ff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9413d630 con 0x7f5f940aabb0 2020-11-28T21:55:21.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.103+0000 7f5f92ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f940aa7d0 0x7f5f940ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:21.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.103+0000 7f5f92ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f940aa7d0 0x7f5f940ad340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44610/0 (socket says 172.21.15.162:44610) 2020-11-28T21:55:21.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.103+0000 7f5f92ffd700 1 -- 172.21.15.162:0/573258029 learned_addr learned my addr 172.21.15.162:0/573258029 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:21.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.103+0000 7f5f937fe700 1 --2- 172.21.15.162:0/573258029 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f940b1c20 0x7f5f94143a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:21.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.103+0000 7f5f927fc700 1 --2- 172.21.15.162:0/573258029 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5f940aabb0 0x7f5f9413d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:21.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.103+0000 7f5f92ffd700 1 -- 172.21.15.162:0/573258029 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5f940aabb0 msgr2=0x7f5f9413d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:21.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.103+0000 7f5f92ffd700 1 --2- 172.21.15.162:0/573258029 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5f940aabb0 0x7f5f9413d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:21.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.103+0000 7f5f92ffd700 1 -- 172.21.15.162:0/573258029 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f940b1c20 msgr2=0x7f5f94143a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:21.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.103+0000 7f5f92ffd700 1 --2- 172.21.15.162:0/573258029 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f940b1c20 0x7f5f94143a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:21.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.103+0000 7f5f92ffd700 1 -- 172.21.15.162:0/573258029 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f940ac310 con 0x7f5f940aa7d0 2020-11-28T21:55:21.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.103+0000 7f5f92ffd700 1 --2- 172.21.15.162:0/573258029 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f940aa7d0 0x7f5f940ad340 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 rx=0x7f5f9413d7c0 tx=0x7f5f84016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:21.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.103+0000 7f5f8bfff700 1 -- 172.21.15.162:0/573258029 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f8401e610 con 0x7f5f940aa7d0 2020-11-28T21:55:21.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.103+0000 7f5f8bfff700 1 -- 172.21.15.162:0/573258029 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f8401e770 con 0x7f5f940aa7d0 2020-11-28T21:55:21.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.103+0000 7f5f994ff700 1 -- 172.21.15.162:0/573258029 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f940acaf0 con 0x7f5f940aa7d0 2020-11-28T21:55:21.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.103+0000 7f5f8bfff700 1 -- 172.21.15.162:0/573258029 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f8401ead0 con 0x7f5f940aa7d0 2020-11-28T21:55:21.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.103+0000 7f5f994ff700 1 -- 172.21.15.162:0/573258029 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f94143f10 con 0x7f5f940aa7d0 2020-11-28T21:55:21.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.107+0000 7f5f8bfff700 1 -- 172.21.15.162:0/573258029 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5f8400f070 con 0x7f5f940aa7d0 2020-11-28T21:55:21.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.107+0000 7f5f994ff700 1 -- 172.21.15.162:0/573258029 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f94057050 con 0x7f5f940aa7d0 2020-11-28T21:55:21.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.107+0000 7f5f8bfff700 1 --2- 172.21.15.162:0/573258029 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5f78055440 0x7f5f78057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:21.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.107+0000 7f5f8bfff700 1 -- 172.21.15.162:0/573258029 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f5f8406f690 con 0x7f5f940aa7d0 2020-11-28T21:55:21.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.107+0000 7f5f927fc700 1 --2- 172.21.15.162:0/573258029 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5f78055440 0x7f5f78057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:21.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.107+0000 7f5f927fc700 1 --2- 172.21.15.162:0/573258029 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5f78055440 0x7f5f78057900 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f5f7c00b000 tx=0x7f5f7c009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:21.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.111+0000 7f5f8bfff700 1 -- 172.21.15.162:0/573258029 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5f84045940 con 0x7f5f940aa7d0 2020-11-28T21:55:21.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.299+0000 7f5f994ff700 1 -- 172.21.15.162:0/573258029 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_getattr_caps"]} v 0) v1 -- 0x7f5f94044350 con 0x7f5f940aa7d0 2020-11-28T21:55:21.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.963+0000 7f5f8bfff700 1 -- 172.21.15.162:0/573258029 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_getattr_caps"]}]=0 v401) v1 ==== 131+0+0 (secure 0 0 0) 0x7f5f84015020 con 0x7f5f940aa7d0 2020-11-28T21:55:21.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.967+0000 7f5f994ff700 1 -- 172.21.15.162:0/573258029 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5f78055440 msgr2=0x7f5f78057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:21.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.967+0000 7f5f994ff700 1 --2- 172.21.15.162:0/573258029 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5f78055440 0x7f5f78057900 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f5f7c00b000 tx=0x7f5f7c009040).stop 2020-11-28T21:55:21.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.967+0000 7f5f994ff700 1 -- 172.21.15.162:0/573258029 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f940aa7d0 msgr2=0x7f5f940ad340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:21.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.967+0000 7f5f994ff700 1 --2- 172.21.15.162:0/573258029 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f940aa7d0 0x7f5f940ad340 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 rx=0x7f5f9413d7c0 tx=0x7f5f84016000).stop 2020-11-28T21:55:21.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.967+0000 7f5f994ff700 1 -- 172.21.15.162:0/573258029 shutdown_connections 2020-11-28T21:55:21.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.967+0000 7f5f994ff700 1 --2- 172.21.15.162:0/573258029 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5f78055440 0x7f5f78057900 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:21.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.967+0000 7f5f994ff700 1 --2- 172.21.15.162:0/573258029 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f940b1c20 0x7f5f94143a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:21.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.967+0000 7f5f994ff700 1 --2- 172.21.15.162:0/573258029 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f940aa7d0 0x7f5f940ad340 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:21.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.967+0000 7f5f994ff700 1 --2- 172.21.15.162:0/573258029 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5f940aabb0 0x7f5f9413d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:21.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.967+0000 7f5f994ff700 1 -- 172.21.15.162:0/573258029 >> 172.21.15.162:0/573258029 conn(0x7f5f94062b50 msgr2=0x7f5f940663f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:21.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.967+0000 7f5f994ff700 1 -- 172.21.15.162:0/573258029 shutdown_connections 2020-11-28T21:55:21.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:21.967+0000 7f5f994ff700 1 -- 172.21.15.162:0/573258029 wait complete. 2020-11-28T21:55:21.988 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config set mon mon_allow_pool_delete True 2020-11-28T21:55:22.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.107+0000 7f8c7f08f700 1 Processor -- start 2020-11-28T21:55:22.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.107+0000 7f8c7f08f700 1 -- start start 2020-11-28T21:55:22.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.107+0000 7f8c7f08f700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8c780b2d00 0x7f8c7805c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:22.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.107+0000 7f8c7f08f700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f8c780b30e0 0x7f8c7813fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:22.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.107+0000 7f8c7f08f700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c7805baf0 con 0x7f8c780b3ab0 2020-11-28T21:55:22.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.107+0000 7f8c7f08f700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c780b0e20 con 0x7f8c780b30e0 2020-11-28T21:55:22.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.107+0000 7f8c7f08f700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c7805bd90 con 0x7f8c780b2d00 2020-11-28T21:55:22.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.107+0000 7f8c7e08d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8c780b2d00 0x7f8c7805c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:22.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.107+0000 7f8c7d88c700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f8c780b3ab0 0x7f8c78142ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51464/0 (socket says 172.21.15.162:51464) 2020-11-28T21:55:22.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.107+0000 7f8c7d88c700 1 -- 172.21.15.162:0/3352228573 learned_addr learned my addr 172.21.15.162:0/3352228573 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:22.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.107+0000 7f8c7e88e700 1 --2- 172.21.15.162:0/3352228573 >> v2:172.21.15.118:3300/0 conn(0x7f8c780b30e0 0x7f8c7813fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:22.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.107+0000 7f8c7e08d700 1 -- 172.21.15.162:0/3352228573 >> v2:172.21.15.118:3300/0 conn(0x7f8c780b30e0 msgr2=0x7f8c7813fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:22.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.107+0000 7f8c7e08d700 1 --2- 172.21.15.162:0/3352228573 >> v2:172.21.15.118:3300/0 conn(0x7f8c780b30e0 0x7f8c7813fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:22.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.107+0000 7f8c7e08d700 1 -- 172.21.15.162:0/3352228573 >> v1:172.21.15.162:6789/0 conn(0x7f8c780b3ab0 legacy=0x7f8c78142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:22.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.107+0000 7f8c7e08d700 1 -- 172.21.15.162:0/3352228573 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c7805c030 con 0x7f8c780b2d00 2020-11-28T21:55:22.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.107+0000 7f8c7e88e700 1 --2- 172.21.15.162:0/3352228573 >> v2:172.21.15.118:3300/0 conn(0x7f8c780b30e0 0x7f8c7813fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:55:22.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.107+0000 7f8c7e08d700 1 --2- 172.21.15.162:0/3352228573 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c780b2d00 0x7f8c7805c4d0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 rx=0x7f8c68002f30 tx=0x7f8c68006dc0).ready entity=mon.1 client_cookie=8b56591a7690aba1 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:22.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.107+0000 7f8c7d08b700 1 -- 172.21.15.162:0/3352228573 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c680097f0 con 0x7f8c780b2d00 2020-11-28T21:55:22.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.107+0000 7f8c7d08b700 1 -- 172.21.15.162:0/3352228573 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c68009950 con 0x7f8c780b2d00 2020-11-28T21:55:22.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.107+0000 7f8c7d08b700 1 -- 172.21.15.162:0/3352228573 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c68009cb0 con 0x7f8c780b2d00 2020-11-28T21:55:22.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.107+0000 7f8c7f08f700 1 -- 172.21.15.162:0/3352228573 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c780b2d00 msgr2=0x7f8c7805c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:22.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.107+0000 7f8c7f08f700 1 --2- 172.21.15.162:0/3352228573 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c780b2d00 0x7f8c7805c4d0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 rx=0x7f8c68002f30 tx=0x7f8c68006dc0).stop 2020-11-28T21:55:22.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.111+0000 7f8c7f08f700 1 -- 172.21.15.162:0/3352228573 shutdown_connections 2020-11-28T21:55:22.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.111+0000 7f8c7f08f700 1 --2- 172.21.15.162:0/3352228573 >> v2:172.21.15.118:3300/0 conn(0x7f8c780b30e0 0x7f8c7813fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:22.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.111+0000 7f8c7f08f700 1 --2- 172.21.15.162:0/3352228573 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c780b2d00 0x7f8c7805c4d0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:22.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.111+0000 7f8c7f08f700 1 -- 172.21.15.162:0/3352228573 >> 172.21.15.162:0/3352228573 conn(0x7f8c780604c0 msgr2=0x7f8c780608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:22.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.111+0000 7f8c7f08f700 1 -- 172.21.15.162:0/3352228573 shutdown_connections 2020-11-28T21:55:22.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.111+0000 7f8c7f08f700 1 -- 172.21.15.162:0/3352228573 wait complete. 2020-11-28T21:55:22.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.111+0000 7f8c7f08f700 1 Processor -- start 2020-11-28T21:55:22.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.111+0000 7f8c7f08f700 1 -- start start 2020-11-28T21:55:22.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.111+0000 7f8c7f08f700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8c780b2d00 0x7f8c7813a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:22.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.111+0000 7f8c7f08f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c780b30e0 0x7f8c780b8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:22.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.111+0000 7f8c7f08f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8c780b3ab0 0x7f8c780b90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:22.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.111+0000 7f8c7f08f700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c781414f0 con 0x7f8c780b2d00 2020-11-28T21:55:22.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.111+0000 7f8c7f08f700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c78142a00 con 0x7f8c780b30e0 2020-11-28T21:55:22.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.111+0000 7f8c7f08f700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c78142ca0 con 0x7f8c780b3ab0 2020-11-28T21:55:22.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.111+0000 7f8c7d88c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c780b30e0 0x7f8c780b8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:22.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.111+0000 7f8c7d88c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c780b30e0 0x7f8c780b8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44624/0 (socket says 172.21.15.162:44624) 2020-11-28T21:55:22.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.111+0000 7f8c7d88c700 1 -- 172.21.15.162:0/1608916956 learned_addr learned my addr 172.21.15.162:0/1608916956 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:22.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.111+0000 7f8c7e08d700 1 --2- 172.21.15.162:0/1608916956 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8c780b2d00 0x7f8c7813a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:22.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.111+0000 7f8c7e88e700 1 --2- 172.21.15.162:0/1608916956 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8c780b3ab0 0x7f8c780b90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:22.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.111+0000 7f8c7d88c700 1 -- 172.21.15.162:0/1608916956 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8c780b3ab0 msgr2=0x7f8c780b90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:22.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.111+0000 7f8c7d88c700 1 --2- 172.21.15.162:0/1608916956 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8c780b3ab0 0x7f8c780b90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:22.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.111+0000 7f8c7d88c700 1 -- 172.21.15.162:0/1608916956 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8c780b2d00 msgr2=0x7f8c7813a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:22.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.111+0000 7f8c7d88c700 1 --2- 172.21.15.162:0/1608916956 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8c780b2d00 0x7f8c7813a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:22.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.111+0000 7f8c7d88c700 1 -- 172.21.15.162:0/1608916956 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c78142f40 con 0x7f8c780b30e0 2020-11-28T21:55:22.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.111+0000 7f8c7e08d700 1 --2- 172.21.15.162:0/1608916956 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8c780b2d00 0x7f8c7813a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:55:22.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.111+0000 7f8c7d88c700 1 --2- 172.21.15.162:0/1608916956 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c780b30e0 0x7f8c780b8a00 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 rx=0x7f8c780acde0 tx=0x7f8c74007c00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:22.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.111+0000 7f8c677fe700 1 -- 172.21.15.162:0/1608916956 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c74007320 con 0x7f8c780b30e0 2020-11-28T21:55:22.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.111+0000 7f8c677fe700 1 -- 172.21.15.162:0/1608916956 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c74007480 con 0x7f8c780b30e0 2020-11-28T21:55:22.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.111+0000 7f8c7f08f700 1 -- 172.21.15.162:0/1608916956 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c781431e0 con 0x7f8c780b30e0 2020-11-28T21:55:22.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.111+0000 7f8c677fe700 1 -- 172.21.15.162:0/1608916956 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c74005220 con 0x7f8c780b30e0 2020-11-28T21:55:22.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.111+0000 7f8c7f08f700 1 -- 172.21.15.162:0/1608916956 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c78139d10 con 0x7f8c780b30e0 2020-11-28T21:55:22.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.115+0000 7f8c677fe700 1 -- 172.21.15.162:0/1608916956 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8c7401b070 con 0x7f8c780b30e0 2020-11-28T21:55:22.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.115+0000 7f8c7f08f700 1 -- 172.21.15.162:0/1608916956 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c78143370 con 0x7f8c780b30e0 2020-11-28T21:55:22.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.115+0000 7f8c677fe700 1 --2- 172.21.15.162:0/1608916956 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8c580554e0 0x7f8c580579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:22.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.115+0000 7f8c677fe700 1 -- 172.21.15.162:0/1608916956 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f8c7406d4c0 con 0x7f8c780b30e0 2020-11-28T21:55:22.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.115+0000 7f8c7e08d700 1 --2- 172.21.15.162:0/1608916956 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8c580554e0 0x7f8c580579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:22.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.115+0000 7f8c7e08d700 1 --2- 172.21.15.162:0/1608916956 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8c580554e0 0x7f8c580579a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f8c68019000 tx=0x7f8c68015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:22.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.119+0000 7f8c677fe700 1 -- 172.21.15.162:0/1608916956 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8c740436f0 con 0x7f8c780b30e0 2020-11-28T21:55:22.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.259+0000 7f8c7f08f700 1 -- 172.21.15.162:0/1608916956 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7f8c78057050 con 0x7f8c780b30e0 2020-11-28T21:55:22.273 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.267+0000 7f8c677fe700 1 -- 172.21.15.162:0/1608916956 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8c74046d10 con 0x7f8c780b30e0 2020-11-28T21:55:22.279 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.271+0000 7f8c7f08f700 1 -- 172.21.15.162:0/1608916956 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8c580554e0 msgr2=0x7f8c580579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:22.279 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.271+0000 7f8c7f08f700 1 --2- 172.21.15.162:0/1608916956 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8c580554e0 0x7f8c580579a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f8c68019000 tx=0x7f8c68015000).stop 2020-11-28T21:55:22.280 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.271+0000 7f8c7f08f700 1 -- 172.21.15.162:0/1608916956 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c780b30e0 msgr2=0x7f8c780b8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:22.280 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.271+0000 7f8c7f08f700 1 --2- 172.21.15.162:0/1608916956 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c780b30e0 0x7f8c780b8a00 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 rx=0x7f8c780acde0 tx=0x7f8c74007c00).stop 2020-11-28T21:55:22.280 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.271+0000 7f8c7f08f700 1 -- 172.21.15.162:0/1608916956 shutdown_connections 2020-11-28T21:55:22.281 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.271+0000 7f8c7f08f700 1 --2- 172.21.15.162:0/1608916956 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8c580554e0 0x7f8c580579a0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:22.281 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.271+0000 7f8c7f08f700 1 --2- 172.21.15.162:0/1608916956 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8c780b3ab0 0x7f8c780b90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:22.282 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.271+0000 7f8c7f08f700 1 --2- 172.21.15.162:0/1608916956 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8c780b2d00 0x7f8c7813a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:22.282 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.271+0000 7f8c7f08f700 1 --2- 172.21.15.162:0/1608916956 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c780b30e0 0x7f8c780b8a00 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:22.283 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.271+0000 7f8c7f08f700 1 -- 172.21.15.162:0/1608916956 >> 172.21.15.162:0/1608916956 conn(0x7f8c780604c0 msgr2=0x7f8c780608a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:22.284 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.271+0000 7f8c7f08f700 1 -- 172.21.15.162:0/1608916956 shutdown_connections 2020-11-28T21:55:22.284 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.271+0000 7f8c7f08f700 1 -- 172.21.15.162:0/1608916956 wait complete. 2020-11-28T21:55:22.290 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-28T21:55:22.290 INFO:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:55:22.309 INFO:teuthology.orchestra.run.smithi118:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:55:22.328 INFO:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:55:22.345 INFO:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:55:22.363 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-28T21:55:22.364 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:22.364 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:55:22.364 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:55:22.410 INFO:teuthology.orchestra.run.smithi192.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2020-11-28T21:55:22.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:55:22.412 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2020-11-28T21:55:22.412 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:22.413 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:55:22.413 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:55:22.420 INFO:teuthology.orchestra.run.smithi096.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-11-28T21:55:22.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:55:22.422 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2020-11-28T21:55:22.422 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2020-11-28T21:55:22.545 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.539+0000 7f082f3c5700 1 Processor -- start 2020-11-28T21:55:22.545 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.539+0000 7f082f3c5700 1 -- start start 2020-11-28T21:55:22.547 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.543+0000 7f082f3c5700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f082813f400 0x7f082809b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:22.548 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.543+0000 7f082f3c5700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f08280b1940 0x7f082813fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:22.548 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.543+0000 7f082f3c5700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0828097a90 con 0x7f08280b45d0 2020-11-28T21:55:22.548 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.543+0000 7f082f3c5700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0828098180 con 0x7f082813f400 2020-11-28T21:55:22.548 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.543+0000 7f082f3c5700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08280b49b0 con 0x7f08280b1940 2020-11-28T21:55:22.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.543+0000 7f082e3c3700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f08280b45d0 0x7f08280afe90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.162:41622/0 (socket says 172.21.15.162:41622) 2020-11-28T21:55:22.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.543+0000 7f082e3c3700 1 -- 172.21.15.162:0/2942276397 learned_addr learned my addr 172.21.15.162:0/2942276397 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:22.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.543+0000 7f082ebc4700 1 --2- 172.21.15.162:0/2942276397 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f08280b1940 0x7f082813fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:22.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.543+0000 7f082dbc2700 1 --2- 172.21.15.162:0/2942276397 >> v2:172.21.15.118:3300/0 conn(0x7f082813f400 0x7f082809b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:22.550 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.543+0000 7f082d3c1700 1 -- 172.21.15.162:0/2942276397 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1272491856 0 0) 0x7f0828097a90 con 0x7f08280b45d0 2020-11-28T21:55:22.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.543+0000 7f082d3c1700 1 -- 172.21.15.162:0/2942276397 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0810003230 con 0x7f08280b45d0 2020-11-28T21:55:22.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.543+0000 7f082ebc4700 1 -- 172.21.15.162:0/2942276397 >> v2:172.21.15.118:3300/0 conn(0x7f082813f400 msgr2=0x7f082809b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:22.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.543+0000 7f082ebc4700 1 --2- 172.21.15.162:0/2942276397 >> v2:172.21.15.118:3300/0 conn(0x7f082813f400 0x7f082809b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:22.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.543+0000 7f082ebc4700 1 -- 172.21.15.162:0/2942276397 >> v1:172.21.15.118:6789/0 conn(0x7f08280b45d0 legacy=0x7f08280afe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:22.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.543+0000 7f082ebc4700 1 -- 172.21.15.162:0/2942276397 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08280987e0 con 0x7f08280b1940 2020-11-28T21:55:22.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.543+0000 7f082ebc4700 1 --2- 172.21.15.162:0/2942276397 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f08280b1940 0x7f082813fab0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f08200032b0 tx=0x7f08200091a0).ready entity=mon.2 client_cookie=b7f1364d9a897a4 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:22.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.543+0000 7f082d3c1700 1 -- 172.21.15.162:0/2942276397 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f082000b940 con 0x7f08280b1940 2020-11-28T21:55:22.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.543+0000 7f082d3c1700 1 -- 172.21.15.162:0/2942276397 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f082000baa0 con 0x7f08280b1940 2020-11-28T21:55:22.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.543+0000 7f082d3c1700 1 -- 172.21.15.162:0/2942276397 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f082000bc10 con 0x7f08280b1940 2020-11-28T21:55:22.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.543+0000 7f082f3c5700 1 -- 172.21.15.162:0/2942276397 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f08280b1940 msgr2=0x7f082813fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:22.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.543+0000 7f082f3c5700 1 --2- 172.21.15.162:0/2942276397 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f08280b1940 0x7f082813fab0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f08200032b0 tx=0x7f08200091a0).stop 2020-11-28T21:55:22.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.547+0000 7f082f3c5700 1 -- 172.21.15.162:0/2942276397 shutdown_connections 2020-11-28T21:55:22.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.547+0000 7f082f3c5700 1 --2- 172.21.15.162:0/2942276397 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f08280b1940 0x7f082813fab0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:22.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.547+0000 7f082f3c5700 1 --2- 172.21.15.162:0/2942276397 >> v2:172.21.15.118:3300/0 conn(0x7f082813f400 0x7f082809b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:22.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.547+0000 7f082f3c5700 1 -- 172.21.15.162:0/2942276397 >> 172.21.15.162:0/2942276397 conn(0x7f08280b54f0 msgr2=0x7f08280b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:22.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.547+0000 7f082f3c5700 1 -- 172.21.15.162:0/2942276397 shutdown_connections 2020-11-28T21:55:22.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.547+0000 7f082f3c5700 1 -- 172.21.15.162:0/2942276397 wait complete. 2020-11-28T21:55:22.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.547+0000 7f082f3c5700 1 Processor -- start 2020-11-28T21:55:22.557 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.547+0000 7f082f3c5700 1 -- start start 2020-11-28T21:55:22.557 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.551+0000 7f082f3c5700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f08280b1940 0x7f08280b2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:22.557 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.551+0000 7f082f3c5700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f08280b45d0 0x7f082813ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:22.557 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.551+0000 7f082f3c5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f082813f400 0x7f082813b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:22.557 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.551+0000 7f082f3c5700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0828065e70 con 0x7f08280b45d0 2020-11-28T21:55:22.558 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.551+0000 7f082f3c5700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0828063a80 con 0x7f082813f400 2020-11-28T21:55:22.558 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.551+0000 7f082f3c5700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0828063d20 con 0x7f08280b1940 2020-11-28T21:55:22.558 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.551+0000 7f082ebc4700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f082813f400 0x7f082813b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:22.558 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.551+0000 7f082ebc4700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f082813f400 0x7f082813b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44640/0 (socket says 172.21.15.162:44640) 2020-11-28T21:55:22.558 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.551+0000 7f082ebc4700 1 -- 172.21.15.162:0/3072653656 learned_addr learned my addr 172.21.15.162:0/3072653656 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:22.559 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.551+0000 7f082e3c3700 1 --2- 172.21.15.162:0/3072653656 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f08280b1940 0x7f08280b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:22.559 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.551+0000 7f082dbc2700 1 --2- 172.21.15.162:0/3072653656 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f08280b45d0 0x7f082813ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:22.559 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.551+0000 7f082ebc4700 1 -- 172.21.15.162:0/3072653656 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f08280b1940 msgr2=0x7f08280b2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:22.559 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.551+0000 7f082ebc4700 1 --2- 172.21.15.162:0/3072653656 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f08280b1940 0x7f08280b2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:22.560 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.551+0000 7f082ebc4700 1 -- 172.21.15.162:0/3072653656 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f08280b45d0 msgr2=0x7f082813ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:22.560 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.551+0000 7f082ebc4700 1 --2- 172.21.15.162:0/3072653656 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f08280b45d0 0x7f082813ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:22.560 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.551+0000 7f082ebc4700 1 -- 172.21.15.162:0/3072653656 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0828063fc0 con 0x7f082813f400 2020-11-28T21:55:22.561 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.551+0000 7f082ebc4700 1 --2- 172.21.15.162:0/3072653656 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f082813f400 0x7f082813b260 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 rx=0x7f08200032b0 tx=0x7f0820016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:22.561 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.551+0000 7f08177fe700 1 -- 172.21.15.162:0/3072653656 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0820011d90 con 0x7f082813f400 2020-11-28T21:55:22.561 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.551+0000 7f08177fe700 1 -- 172.21.15.162:0/3072653656 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f082001f3f0 con 0x7f082813f400 2020-11-28T21:55:22.561 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.551+0000 7f08177fe700 1 -- 172.21.15.162:0/3072653656 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f082001f750 con 0x7f082813f400 2020-11-28T21:55:22.561 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.551+0000 7f082f3c5700 1 -- 172.21.15.162:0/3072653656 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f082813b750 con 0x7f082813f400 2020-11-28T21:55:22.562 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.551+0000 7f082f3c5700 1 -- 172.21.15.162:0/3072653656 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0828144810 con 0x7f082813f400 2020-11-28T21:55:22.562 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.551+0000 7f082f3c5700 1 -- 172.21.15.162:0/3072653656 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0828057050 con 0x7f082813f400 2020-11-28T21:55:22.563 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.555+0000 7f08177fe700 1 -- 172.21.15.162:0/3072653656 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0820026030 con 0x7f082813f400 2020-11-28T21:55:22.563 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.555+0000 7f08177fe700 1 --2- 172.21.15.162:0/3072653656 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f08080553b0 0x7f0808057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:22.563 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.555+0000 7f08177fe700 1 -- 172.21.15.162:0/3072653656 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f082006eba0 con 0x7f082813f400 2020-11-28T21:55:22.563 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.555+0000 7f082e3c3700 1 --2- 172.21.15.162:0/3072653656 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f08080553b0 0x7f0808057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:22.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.555+0000 7f082e3c3700 1 --2- 172.21.15.162:0/3072653656 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f08080553b0 0x7f0808057870 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7f082400d000 tx=0x7f0824009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:22.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.559+0000 7f08177fe700 1 -- 172.21.15.162:0/3072653656 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0820044d40 con 0x7f082813f400 2020-11-28T21:55:22.704 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.699+0000 7f082f3c5700 1 -- 172.21.15.162:0/3072653656 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f082805d5e0 con 0x7f082813f400 2020-11-28T21:55:22.705 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.703+0000 7f08177fe700 1 -- 172.21.15.162:0/3072653656 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v131) v1 ==== 81+0+33911 (secure 0 0 0) 0x7f0820024090 con 0x7f082813f400 2020-11-28T21:55:22.708 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:22.709 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2020-11-28T21:55:22.709 INFO:teuthology.orchestra.run.smithi162.stdout: "epoch": 131, 2020-11-28T21:55:22.709 INFO:teuthology.orchestra.run.smithi162.stdout: "fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", 2020-11-28T21:55:22.709 INFO:teuthology.orchestra.run.smithi162.stdout: "created": "2020-11-28T21:46:11.724372+0000", 2020-11-28T21:55:22.710 INFO:teuthology.orchestra.run.smithi162.stdout: "modified": "2020-11-28T21:55:20.290602+0000", 2020-11-28T21:55:22.710 INFO:teuthology.orchestra.run.smithi162.stdout: "last_up_change": "2020-11-28T21:46:19.591650+0000", 2020-11-28T21:55:22.710 INFO:teuthology.orchestra.run.smithi162.stdout: "last_in_change": "2020-11-28T21:46:19.591650+0000", 2020-11-28T21:55:22.710 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-11-28T21:55:22.710 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_num": 5799936, 2020-11-28T21:55:22.711 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_set": [ 2020-11-28T21:55:22.711 INFO:teuthology.orchestra.run.smithi162.stdout: "pglog_hardlimit", 2020-11-28T21:55:22.711 INFO:teuthology.orchestra.run.smithi162.stdout: "purged_snapdirs", 2020-11-28T21:55:22.711 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_deletes", 2020-11-28T21:55:22.711 INFO:teuthology.orchestra.run.smithi162.stdout: "sortbitwise" 2020-11-28T21:55:22.712 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:55:22.712 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_version": 4, 2020-11-28T21:55:22.712 INFO:teuthology.orchestra.run.smithi162.stdout: "full_ratio": 0.94999998807907104, 2020-11-28T21:55:22.712 INFO:teuthology.orchestra.run.smithi162.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-11-28T21:55:22.713 INFO:teuthology.orchestra.run.smithi162.stdout: "nearfull_ratio": 0.85000002384185791, 2020-11-28T21:55:22.713 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_snapshot": "", 2020-11-28T21:55:22.713 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_max": 24, 2020-11-28T21:55:22.713 INFO:teuthology.orchestra.run.smithi162.stdout: "max_osd": 8, 2020-11-28T21:55:22.714 INFO:teuthology.orchestra.run.smithi162.stdout: "require_min_compat_client": "jewel", 2020-11-28T21:55:22.714 INFO:teuthology.orchestra.run.smithi162.stdout: "min_compat_client": "jewel", 2020-11-28T21:55:22.714 INFO:teuthology.orchestra.run.smithi162.stdout: "require_osd_release": "octopus", 2020-11-28T21:55:22.714 INFO:teuthology.orchestra.run.smithi162.stdout: "pools": [ 2020-11-28T21:55:22.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.715 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 1, 2020-11-28T21:55:22.715 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "device_health_metrics", 2020-11-28T21:55:22.715 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:46:15.639551+0000", 2020-11-28T21:55:22.715 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:55:22.716 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:55:22.716 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:55:22.716 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:55:22.716 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:55:22.716 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:55:22.717 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:55:22.717 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:55:22.717 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 1, 2020-11-28T21:55:22.717 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 1, 2020-11-28T21:55:22.717 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 1, 2020-11-28T21:55:22.718 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 1, 2020-11-28T21:55:22.718 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 1, 2020-11-28T21:55:22.718 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:55:22.718 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:55:22.718 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:55:22.719 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:55:22.719 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:55:22.719 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:55:22.719 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:55:22.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.720 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "16", 2020-11-28T21:55:22.720 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:55:22.720 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:55:22.720 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:55:22.720 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:55:22.721 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:55:22.721 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:55:22.721 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:55:22.721 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:55:22.721 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:55:22.722 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:55:22.722 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:55:22.722 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:55:22.722 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:55:22.722 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:55:22.723 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:55:22.723 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:55:22.723 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:55:22.724 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:55:22.724 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:55:22.724 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:55:22.724 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:55:22.724 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:55:22.725 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:55:22.725 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:55:22.725 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:55:22.725 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:55:22.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.726 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:55:22.726 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:55:22.726 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:55:22.726 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:55:22.726 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:55:22.727 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:55:22.727 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:55:22.727 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:55:22.727 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:55:22.727 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:55:22.728 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:55:22.728 INFO:teuthology.orchestra.run.smithi162.stdout: "options": { 2020-11-28T21:55:22.728 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_min": 1 2020-11-28T21:55:22.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.728 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:55:22.729 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr_devicehealth": {} 2020-11-28T21:55:22.729 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:22.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.729 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 2, 2020-11-28T21:55:22.730 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "rbd", 2020-11-28T21:55:22.730 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:46:23.901366+0000", 2020-11-28T21:55:22.730 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:55:22.730 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:55:22.730 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:55:22.731 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:55:22.731 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:55:22.731 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:55:22.731 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:55:22.731 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:55:22.732 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2020-11-28T21:55:22.732 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2020-11-28T21:55:22.732 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2020-11-28T21:55:22.732 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2020-11-28T21:55:22.732 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2020-11-28T21:55:22.733 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:55:22.733 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:55:22.733 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:55:22.733 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:55:22.734 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:55:22.734 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:55:22.734 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:55:22.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.735 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "19", 2020-11-28T21:55:22.735 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:55:22.735 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:55:22.735 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:55:22.735 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:55:22.736 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:55:22.736 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:55:22.736 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:55:22.736 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:55:22.736 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:55:22.737 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:55:22.737 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:55:22.737 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:55:22.737 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:55:22.737 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:55:22.738 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:55:22.738 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:55:22.738 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:55:22.738 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:55:22.739 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:55:22.739 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:55:22.739 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:55:22.739 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:55:22.739 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:55:22.740 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:55:22.740 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:55:22.740 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:55:22.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.740 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:55:22.741 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:55:22.741 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:55:22.741 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:55:22.741 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:55:22.741 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:55:22.742 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:55:22.742 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:55:22.742 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:55:22.742 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:55:22.742 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:55:22.743 INFO:teuthology.orchestra.run.smithi162.stdout: "options": {}, 2020-11-28T21:55:22.743 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:55:22.743 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd": {} 2020-11-28T21:55:22.743 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:22.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.744 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 22, 2020-11-28T21:55:22.744 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "cephfs_data", 2020-11-28T21:55:22.744 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:54:50.370123+0000", 2020-11-28T21:55:22.744 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:55:22.745 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:55:22.745 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:55:22.745 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:55:22.745 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:55:22.745 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:55:22.746 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:55:22.746 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:55:22.746 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2020-11-28T21:55:22.746 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2020-11-28T21:55:22.746 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2020-11-28T21:55:22.747 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2020-11-28T21:55:22.747 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2020-11-28T21:55:22.747 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:55:22.747 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:55:22.747 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:55:22.748 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:55:22.748 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:55:22.748 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:55:22.748 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:55:22.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.749 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "130", 2020-11-28T21:55:22.749 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:55:22.749 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:55:22.749 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:55:22.750 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:55:22.750 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:55:22.750 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:55:22.750 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:55:22.750 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:55:22.751 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:55:22.751 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:55:22.751 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:55:22.751 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:55:22.751 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:55:22.752 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:55:22.752 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:55:22.752 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:55:22.752 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:55:22.752 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:55:22.753 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:55:22.753 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:55:22.753 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:55:22.753 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:55:22.753 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:55:22.754 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:55:22.754 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:55:22.754 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:55:22.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.754 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:55:22.755 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:55:22.755 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:55:22.755 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:55:22.755 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:55:22.755 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:55:22.756 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:55:22.756 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:55:22.756 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:55:22.756 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:55:22.756 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:55:22.757 INFO:teuthology.orchestra.run.smithi162.stdout: "options": {}, 2020-11-28T21:55:22.757 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:55:22.757 INFO:teuthology.orchestra.run.smithi162.stdout: "cephfs": { 2020-11-28T21:55:22.757 INFO:teuthology.orchestra.run.smithi162.stdout: "data": "cephfs" 2020-11-28T21:55:22.758 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:22.758 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:22.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.758 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 24, 2020-11-28T21:55:22.759 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "cephfs_metadata", 2020-11-28T21:55:22.759 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:55:18.304204+0000", 2020-11-28T21:55:22.759 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:55:22.759 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:55:22.759 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:55:22.760 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:55:22.760 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:55:22.760 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:55:22.760 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:55:22.760 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:55:22.761 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2020-11-28T21:55:22.761 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2020-11-28T21:55:22.761 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2020-11-28T21:55:22.761 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2020-11-28T21:55:22.761 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2020-11-28T21:55:22.762 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:55:22.762 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:55:22.762 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:55:22.762 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:55:22.762 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:55:22.763 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:55:22.763 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:55:22.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.763 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "131", 2020-11-28T21:55:22.763 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:55:22.764 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:55:22.764 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:55:22.764 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:55:22.764 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:55:22.764 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:55:22.765 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:55:22.765 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:55:22.765 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:55:22.765 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:55:22.765 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:55:22.766 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:55:22.766 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:55:22.766 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:55:22.766 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:55:22.766 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:55:22.767 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:55:22.767 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:55:22.767 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:55:22.767 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:55:22.767 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:55:22.768 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:55:22.768 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:55:22.768 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:55:22.768 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:55:22.768 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:55:22.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.769 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:55:22.769 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:55:22.769 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:55:22.769 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:55:22.770 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:55:22.770 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:55:22.770 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:55:22.770 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:55:22.770 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:55:22.771 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:55:22.771 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:55:22.771 INFO:teuthology.orchestra.run.smithi162.stdout: "options": { 2020-11-28T21:55:22.771 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_bias": 4, 2020-11-28T21:55:22.771 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_min": 16, 2020-11-28T21:55:22.772 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_priority": 5 2020-11-28T21:55:22.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.772 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:55:22.772 INFO:teuthology.orchestra.run.smithi162.stdout: "cephfs": { 2020-11-28T21:55:22.772 INFO:teuthology.orchestra.run.smithi162.stdout: "metadata": "cephfs" 2020-11-28T21:55:22.773 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:22.773 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:22.773 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:22.773 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:55:22.773 INFO:teuthology.orchestra.run.smithi162.stdout: "osds": [ 2020-11-28T21:55:22.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.774 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 0, 2020-11-28T21:55:22.774 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "99c7d9ae-adfa-4fd7-880a-23e6baf3281b", 2020-11-28T21:55:22.774 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:55:22.774 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:55:22.775 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:55:22.775 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:55:22.775 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:55:22.775 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:55:22.775 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:55:22.776 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 107, 2020-11-28T21:55:22.776 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:55:22.776 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:55:22.776 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:55:22.776 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:22.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.777 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:22.777 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6810", 2020-11-28T21:55:22.777 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:55:22.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.778 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:22.778 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6811", 2020-11-28T21:55:22.778 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:55:22.778 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:22.779 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:22.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.779 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:55:22.779 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:22.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.780 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:22.780 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6812", 2020-11-28T21:55:22.780 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:55:22.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.781 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:22.781 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6813", 2020-11-28T21:55:22.781 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:55:22.781 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:22.781 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:22.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.782 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:55:22.782 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:22.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.782 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:22.783 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6816", 2020-11-28T21:55:22.783 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:55:22.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.783 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:22.784 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6817", 2020-11-28T21:55:22.784 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:55:22.784 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:22.784 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:22.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.785 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:55:22.785 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:22.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.786 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:22.786 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6814", 2020-11-28T21:55:22.786 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:55:22.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.787 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:22.787 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6815", 2020-11-28T21:55:22.787 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:55:22.787 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:22.787 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:22.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.788 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6811/12313", 2020-11-28T21:55:22.788 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6813/12313", 2020-11-28T21:55:22.788 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6817/12313", 2020-11-28T21:55:22.788 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6815/12313", 2020-11-28T21:55:22.789 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:55:22.789 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:55:22.789 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:55:22.789 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:22.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.790 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 1, 2020-11-28T21:55:22.790 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "6ce6220a-00ef-4d13-abba-56f87636fb5f", 2020-11-28T21:55:22.790 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:55:22.790 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:55:22.791 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:55:22.791 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:55:22.791 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:55:22.791 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:55:22.791 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:55:22.792 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 129, 2020-11-28T21:55:22.792 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:55:22.792 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:55:22.792 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:55:22.792 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:22.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.793 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:22.793 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6802", 2020-11-28T21:55:22.793 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:55:22.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.794 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:22.794 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6803", 2020-11-28T21:55:22.794 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:55:22.795 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:22.795 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:22.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.795 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:55:22.795 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:22.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.796 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:22.796 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6804", 2020-11-28T21:55:22.796 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:55:22.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.797 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:22.797 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6805", 2020-11-28T21:55:22.797 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:55:22.797 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:22.798 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:22.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.798 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:55:22.798 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:22.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.799 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:22.799 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6808", 2020-11-28T21:55:22.799 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:55:22.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.800 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:22.800 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6809", 2020-11-28T21:55:22.800 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:55:22.800 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:22.800 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:22.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.801 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:55:22.801 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:22.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.801 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:22.802 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6806", 2020-11-28T21:55:22.802 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:55:22.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.803 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:22.803 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6807", 2020-11-28T21:55:22.803 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:55:22.803 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:22.803 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:22.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.804 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6803/12311", 2020-11-28T21:55:22.804 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6805/12311", 2020-11-28T21:55:22.804 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6809/12311", 2020-11-28T21:55:22.804 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6807/12311", 2020-11-28T21:55:22.805 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:55:22.805 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:55:22.805 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:55:22.805 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:22.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.806 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 2, 2020-11-28T21:55:22.806 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "db4e510d-27c1-47fc-a902-8b32e7dfd73e", 2020-11-28T21:55:22.806 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:55:22.806 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:55:22.807 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:55:22.807 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:55:22.807 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:55:22.807 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:55:22.807 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:55:22.808 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 124, 2020-11-28T21:55:22.808 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:55:22.808 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:55:22.808 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:55:22.808 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:22.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.809 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:22.809 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6818", 2020-11-28T21:55:22.809 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:55:22.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.810 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.810 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:22.810 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6819", 2020-11-28T21:55:22.810 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:55:22.811 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:22.811 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:22.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.811 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:55:22.811 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:22.812 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.812 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:22.812 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6820", 2020-11-28T21:55:22.812 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:55:22.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.812 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.813 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:22.813 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6821", 2020-11-28T21:55:22.813 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:55:22.813 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:22.813 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:22.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.814 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:55:22.814 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:22.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.814 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:22.815 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6824", 2020-11-28T21:55:22.815 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:55:22.815 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.815 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.815 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:22.816 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6825", 2020-11-28T21:55:22.816 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:55:22.816 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:22.816 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:22.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.817 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:55:22.817 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:22.817 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.817 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:22.818 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6822", 2020-11-28T21:55:22.818 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:55:22.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.818 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.818 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:22.819 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6823", 2020-11-28T21:55:22.819 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:55:22.819 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:22.819 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:22.819 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.820 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6819/12312", 2020-11-28T21:55:22.820 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6821/12312", 2020-11-28T21:55:22.820 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6825/12312", 2020-11-28T21:55:22.820 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6823/12312", 2020-11-28T21:55:22.820 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:55:22.821 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:55:22.821 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:55:22.821 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:22.821 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.821 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.822 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 3, 2020-11-28T21:55:22.822 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "4094cb87-f04a-45e1-8710-fb4b651949f6", 2020-11-28T21:55:22.822 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:55:22.822 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:55:22.822 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:55:22.823 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:55:22.823 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:55:22.823 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:55:22.823 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:55:22.823 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 129, 2020-11-28T21:55:22.824 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:55:22.824 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:55:22.824 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:55:22.824 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:22.824 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.825 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:22.825 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6826", 2020-11-28T21:55:22.825 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:55:22.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.825 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.826 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:22.826 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6827", 2020-11-28T21:55:22.826 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:55:22.826 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:22.827 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:22.827 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.827 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:55:22.827 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:22.827 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.828 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:22.828 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6828", 2020-11-28T21:55:22.828 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:55:22.828 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.828 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.829 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:22.829 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6829", 2020-11-28T21:55:22.829 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:55:22.829 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:22.829 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:22.830 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.830 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:55:22.830 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:22.830 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.830 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:22.831 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6832", 2020-11-28T21:55:22.831 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:55:22.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.831 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:22.832 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6833", 2020-11-28T21:55:22.832 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:55:22.832 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:22.832 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:22.833 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.833 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:55:22.833 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:22.833 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.833 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:22.834 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6830", 2020-11-28T21:55:22.834 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:55:22.834 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.834 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.834 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:22.835 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6831", 2020-11-28T21:55:22.835 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:55:22.835 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:22.835 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:22.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.836 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6827/12323", 2020-11-28T21:55:22.836 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6829/12323", 2020-11-28T21:55:22.836 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6833/12323", 2020-11-28T21:55:22.836 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6831/12323", 2020-11-28T21:55:22.836 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:55:22.837 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:55:22.837 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:55:22.837 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:22.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.838 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 4, 2020-11-28T21:55:22.838 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "b907bf53-c297-4756-847b-3be96680eef5", 2020-11-28T21:55:22.838 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:55:22.838 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:55:22.838 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:55:22.839 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:55:22.839 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:55:22.839 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:55:22.839 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:55:22.839 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 129, 2020-11-28T21:55:22.840 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:55:22.840 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:55:22.840 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:55:22.840 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:22.840 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.841 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:22.841 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6800", 2020-11-28T21:55:22.841 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:55:22.841 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.841 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.842 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:22.842 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6801", 2020-11-28T21:55:22.842 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:55:22.842 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:22.843 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:22.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.843 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:55:22.843 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:22.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.844 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:22.844 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6802", 2020-11-28T21:55:22.844 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:55:22.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.845 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:22.845 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6803", 2020-11-28T21:55:22.845 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:55:22.845 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:22.845 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:22.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.846 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:55:22.846 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:22.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.846 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:22.847 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6806", 2020-11-28T21:55:22.847 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:55:22.847 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.847 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.847 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:22.848 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6807", 2020-11-28T21:55:22.848 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:55:22.848 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:22.848 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:22.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.849 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:55:22.849 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:22.849 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.849 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:22.849 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6804", 2020-11-28T21:55:22.850 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:55:22.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.850 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:22.851 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6805", 2020-11-28T21:55:22.851 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:55:22.851 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:22.851 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:22.851 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.852 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6801/12527", 2020-11-28T21:55:22.852 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6803/12527", 2020-11-28T21:55:22.852 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6807/12527", 2020-11-28T21:55:22.852 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6805/12527", 2020-11-28T21:55:22.852 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:55:22.853 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:55:22.853 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:55:22.853 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:22.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.854 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 5, 2020-11-28T21:55:22.854 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "55ba305d-0505-4212-aebb-631e8554b440", 2020-11-28T21:55:22.854 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:55:22.854 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:55:22.854 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:55:22.855 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:55:22.855 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:55:22.855 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:55:22.855 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:55:22.855 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 129, 2020-11-28T21:55:22.856 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:55:22.856 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:55:22.856 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:55:22.856 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:22.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.857 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:22.857 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6808", 2020-11-28T21:55:22.857 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:55:22.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.858 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:22.858 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6809", 2020-11-28T21:55:22.858 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:55:22.858 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:22.858 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:22.859 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.859 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:55:22.859 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:22.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.859 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:22.860 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6810", 2020-11-28T21:55:22.860 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:55:22.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.860 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.860 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:22.861 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6811", 2020-11-28T21:55:22.861 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:55:22.861 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:22.861 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:22.861 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.862 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:55:22.862 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:22.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.862 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:22.863 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6814", 2020-11-28T21:55:22.863 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:55:22.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.863 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:22.864 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6815", 2020-11-28T21:55:22.864 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:55:22.864 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:22.864 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:22.864 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.865 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:55:22.865 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:22.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.865 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:22.865 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6812", 2020-11-28T21:55:22.866 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:55:22.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.866 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:22.866 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6813", 2020-11-28T21:55:22.867 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:55:22.867 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:22.867 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:22.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.867 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6809/12529", 2020-11-28T21:55:22.868 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6811/12529", 2020-11-28T21:55:22.868 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6815/12529", 2020-11-28T21:55:22.868 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6813/12529", 2020-11-28T21:55:22.868 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:55:22.868 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:55:22.869 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:55:22.869 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:22.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.869 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 6, 2020-11-28T21:55:22.870 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "9ada6051-493b-46b5-ba22-1b401440490f", 2020-11-28T21:55:22.870 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:55:22.870 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:55:22.870 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:55:22.870 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:55:22.871 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:55:22.871 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:55:22.871 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:55:22.871 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 124, 2020-11-28T21:55:22.871 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:55:22.872 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:55:22.872 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:55:22.872 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:22.872 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.872 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:22.873 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6824", 2020-11-28T21:55:22.873 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:55:22.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.873 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.873 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:22.874 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6825", 2020-11-28T21:55:22.874 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:55:22.874 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:22.874 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:22.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.875 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:55:22.875 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:22.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.875 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:22.875 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6826", 2020-11-28T21:55:22.876 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:55:22.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.876 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.876 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:22.876 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6827", 2020-11-28T21:55:22.877 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:55:22.877 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:22.877 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:22.877 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.877 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:55:22.878 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:22.878 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.878 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:22.878 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6830", 2020-11-28T21:55:22.878 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:55:22.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.879 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.879 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:22.879 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6831", 2020-11-28T21:55:22.879 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:55:22.880 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:22.880 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:22.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.880 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:55:22.880 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:22.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.881 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:22.881 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6828", 2020-11-28T21:55:22.881 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:55:22.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.882 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.882 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:22.882 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6829", 2020-11-28T21:55:22.882 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:55:22.882 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:22.883 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:22.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.883 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6825/12530", 2020-11-28T21:55:22.883 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6827/12530", 2020-11-28T21:55:22.883 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6831/12530", 2020-11-28T21:55:22.884 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6829/12530", 2020-11-28T21:55:22.884 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:55:22.884 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:55:22.884 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:55:22.884 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:22.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.885 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 7, 2020-11-28T21:55:22.885 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "c328b06d-a216-4cba-aad7-9ad081340613", 2020-11-28T21:55:22.885 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:55:22.886 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:55:22.886 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:55:22.886 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:55:22.886 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:55:22.886 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:55:22.887 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:55:22.887 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 129, 2020-11-28T21:55:22.887 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:55:22.887 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:55:22.887 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:55:22.888 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:22.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.888 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:22.888 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6816", 2020-11-28T21:55:22.888 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:55:22.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.889 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:22.889 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6817", 2020-11-28T21:55:22.889 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:55:22.890 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:22.890 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:22.890 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.890 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:55:22.890 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:22.891 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.891 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:22.891 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6818", 2020-11-28T21:55:22.891 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:55:22.891 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.892 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:22.892 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6819", 2020-11-28T21:55:22.892 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:55:22.892 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:22.893 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:22.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.893 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:55:22.893 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:22.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.894 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:22.894 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6822", 2020-11-28T21:55:22.894 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:55:22.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.895 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:22.895 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6823", 2020-11-28T21:55:22.895 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:55:22.895 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:22.895 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:22.896 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.896 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:55:22.896 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:22.896 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.896 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:22.897 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6820", 2020-11-28T21:55:22.897 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:55:22.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.897 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:22.898 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6821", 2020-11-28T21:55:22.898 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:55:22.898 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:22.898 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:22.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.899 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6817/12528", 2020-11-28T21:55:22.899 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6819/12528", 2020-11-28T21:55:22.899 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6823/12528", 2020-11-28T21:55:22.899 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6821/12528", 2020-11-28T21:55:22.899 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:55:22.900 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:55:22.900 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:55:22.900 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:22.900 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:22.900 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:55:22.901 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_xinfo": [ 2020-11-28T21:55:22.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.901 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 0, 2020-11-28T21:55:22.901 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:55:22.901 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:55:22.902 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:55:22.902 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:55:22.902 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:55:22.902 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.038052+0000", 2020-11-28T21:55:22.902 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:55:22.903 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.903 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.903 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 1, 2020-11-28T21:55:22.903 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:55:22.903 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:55:22.904 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:55:22.904 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:55:22.904 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:55:22.904 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:17.947136+0000", 2020-11-28T21:55:22.904 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:55:22.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.905 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 2, 2020-11-28T21:55:22.905 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:55:22.905 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:55:22.906 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:55:22.906 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:55:22.906 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:55:22.906 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.001186+0000", 2020-11-28T21:55:22.906 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:55:22.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.907 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.907 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 3, 2020-11-28T21:55:22.907 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:55:22.907 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:55:22.908 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:55:22.908 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:55:22.908 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:55:22.908 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.199284+0000", 2020-11-28T21:55:22.909 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:55:22.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.909 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 4, 2020-11-28T21:55:22.909 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:55:22.910 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:55:22.910 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:55:22.910 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:55:22.910 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:55:22.910 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.038823+0000", 2020-11-28T21:55:22.911 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:55:22.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.911 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 5, 2020-11-28T21:55:22.911 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:55:22.912 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:55:22.912 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:55:22.912 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:55:22.912 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:55:22.912 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.009322+0000", 2020-11-28T21:55:22.913 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:55:22.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.913 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 6, 2020-11-28T21:55:22.913 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:55:22.914 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:55:22.914 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:55:22.914 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:55:22.914 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:55:22.915 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.238927+0000", 2020-11-28T21:55:22.915 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:55:22.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:22.915 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 7, 2020-11-28T21:55:22.916 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:55:22.916 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:55:22.916 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:55:22.916 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:55:22.916 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:55:22.917 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.029839+0000", 2020-11-28T21:55:22.917 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:55:22.917 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:22.917 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:55:22.917 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_upmap": [], 2020-11-28T21:55:22.918 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_upmap_items": [], 2020-11-28T21:55:22.918 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_temp": [], 2020-11-28T21:55:22.918 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_temp": [], 2020-11-28T21:55:22.918 INFO:teuthology.orchestra.run.smithi162.stdout: "blacklist": { 2020-11-28T21:55:22.919 INFO:teuthology.orchestra.run.smithi162.stdout: "172.21.15.162:6837/1061393314": "2020-11-29T21:55:04.152214+0000", 2020-11-28T21:55:22.919 INFO:teuthology.orchestra.run.smithi162.stdout: "172.21.15.162:6836/1061393314": "2020-11-29T21:55:04.152214+0000" 2020-11-28T21:55:22.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.919 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profiles": { 2020-11-28T21:55:22.919 INFO:teuthology.orchestra.run.smithi162.stdout: "default": { 2020-11-28T21:55:22.920 INFO:teuthology.orchestra.run.smithi162.stdout: "crush-failure-domain": "osd", 2020-11-28T21:55:22.920 INFO:teuthology.orchestra.run.smithi162.stdout: "k": "2", 2020-11-28T21:55:22.920 INFO:teuthology.orchestra.run.smithi162.stdout: "m": "1", 2020-11-28T21:55:22.920 INFO:teuthology.orchestra.run.smithi162.stdout: "plugin": "jerasure", 2020-11-28T21:55:22.920 INFO:teuthology.orchestra.run.smithi162.stdout: "ruleset-failure-domain": "osd", 2020-11-28T21:55:22.921 INFO:teuthology.orchestra.run.smithi162.stdout: "technique": "reed_sol_van" 2020-11-28T21:55:22.921 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:22.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:22.921 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps_queue": [], 2020-11-28T21:55:22.921 INFO:teuthology.orchestra.run.smithi162.stdout: "new_removed_snaps": [], 2020-11-28T21:55:22.922 INFO:teuthology.orchestra.run.smithi162.stdout: "new_purged_snaps": [], 2020-11-28T21:55:22.922 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_node_flags": {}, 2020-11-28T21:55:22.922 INFO:teuthology.orchestra.run.smithi162.stdout: "device_class_flags": {} 2020-11-28T21:55:22.922 INFO:teuthology.orchestra.run.smithi162.stdout:} 2020-11-28T21:55:22.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.703+0000 7f082f3c5700 1 -- 172.21.15.162:0/3072653656 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f08080553b0 msgr2=0x7f0808057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:22.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.703+0000 7f082f3c5700 1 --2- 172.21.15.162:0/3072653656 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f08080553b0 0x7f0808057870 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7f082400d000 tx=0x7f0824009040).stop 2020-11-28T21:55:22.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.703+0000 7f082f3c5700 1 -- 172.21.15.162:0/3072653656 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f082813f400 msgr2=0x7f082813b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:22.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.703+0000 7f082f3c5700 1 --2- 172.21.15.162:0/3072653656 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f082813f400 0x7f082813b260 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 rx=0x7f08200032b0 tx=0x7f0820016000).stop 2020-11-28T21:55:22.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.703+0000 7f082f3c5700 1 -- 172.21.15.162:0/3072653656 shutdown_connections 2020-11-28T21:55:22.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.703+0000 7f082f3c5700 1 --2- 172.21.15.162:0/3072653656 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f08080553b0 0x7f0808057870 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:22.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.703+0000 7f082f3c5700 1 --2- 172.21.15.162:0/3072653656 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f08280b1940 0x7f08280b2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:22.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.703+0000 7f082f3c5700 1 --2- 172.21.15.162:0/3072653656 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f082813f400 0x7f082813b260 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:22.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.703+0000 7f082f3c5700 1 --2- 172.21.15.162:0/3072653656 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f08280b45d0 0x7f082813ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:22.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.703+0000 7f082f3c5700 1 -- 172.21.15.162:0/3072653656 >> 172.21.15.162:0/3072653656 conn(0x7f08280b54f0 msgr2=0x7f0828140f70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:22.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.703+0000 7f082f3c5700 1 -- 172.21.15.162:0/3072653656 shutdown_connections 2020-11-28T21:55:22.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:22.703+0000 7f082f3c5700 1 -- 172.21.15.162:0/3072653656 wait complete. 2020-11-28T21:55:22.926 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:55:23.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.039+0000 7fed18fa4700 1 Processor -- start 2020-11-28T21:55:23.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.039+0000 7fed18fa4700 1 -- start start 2020-11-28T21:55:23.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.039+0000 7fed18fa4700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fed14142a00 0x7fed140b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:23.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.039+0000 7fed18fa4700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fed140b4c70 0x7fed14143020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:23.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.039+0000 7fed18fa4700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed140b4110 con 0x7fed140603d0 2020-11-28T21:55:23.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.039+0000 7fed18fa4700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed14060f60 con 0x7fed140b4c70 2020-11-28T21:55:23.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.039+0000 7fed18fa4700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed140b43f0 con 0x7fed14142a00 2020-11-28T21:55:23.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.039+0000 7fed12ffd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fed14142a00 0x7fed140b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:23.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.043+0000 7fed137fe700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fed140603d0 0x7fed1409cfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51488/0 (socket says 172.21.15.162:51488) 2020-11-28T21:55:23.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.043+0000 7fed137fe700 1 -- 172.21.15.162:0/1647518011 learned_addr learned my addr 172.21.15.162:0/1647518011 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:23.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.043+0000 7fed13fff700 1 --2- 172.21.15.162:0/1647518011 >> v2:172.21.15.118:3300/0 conn(0x7fed140b4c70 0x7fed14143020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:23.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.043+0000 7fed12ffd700 1 -- 172.21.15.162:0/1647518011 >> v2:172.21.15.118:3300/0 conn(0x7fed140b4c70 msgr2=0x7fed14143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:23.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.043+0000 7fed12ffd700 1 --2- 172.21.15.162:0/1647518011 >> v2:172.21.15.118:3300/0 conn(0x7fed140b4c70 0x7fed14143020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:23.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.043+0000 7fed12ffd700 1 -- 172.21.15.162:0/1647518011 >> v1:172.21.15.162:6789/0 conn(0x7fed140603d0 legacy=0x7fed1409cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:23.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.043+0000 7fed127fc700 1 -- 172.21.15.162:0/1647518011 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3004271030 0 0) 0x7fed140b4110 con 0x7fed140603d0 2020-11-28T21:55:23.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.043+0000 7fed12ffd700 1 -- 172.21.15.162:0/1647518011 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed140b4690 con 0x7fed14142a00 2020-11-28T21:55:23.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.043+0000 7fed12ffd700 1 --2- 172.21.15.162:0/1647518011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed14142a00 0x7fed140b3a70 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 rx=0x7fed04002f30 tx=0x7fed04006dc0).ready entity=mon.1 client_cookie=5f504cfaa49c19ec server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:23.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.043+0000 7fed127fc700 1 -- 172.21.15.162:0/1647518011 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed040097b0 con 0x7fed14142a00 2020-11-28T21:55:23.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.043+0000 7fed127fc700 1 -- 172.21.15.162:0/1647518011 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed04009910 con 0x7fed14142a00 2020-11-28T21:55:23.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.043+0000 7fed127fc700 1 -- 172.21.15.162:0/1647518011 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed04009c70 con 0x7fed14142a00 2020-11-28T21:55:23.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.043+0000 7fed18fa4700 1 -- 172.21.15.162:0/1647518011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed14142a00 msgr2=0x7fed140b3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:23.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.043+0000 7fed18fa4700 1 --2- 172.21.15.162:0/1647518011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed14142a00 0x7fed140b3a70 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 rx=0x7fed04002f30 tx=0x7fed04006dc0).stop 2020-11-28T21:55:23.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.043+0000 7fed18fa4700 1 -- 172.21.15.162:0/1647518011 shutdown_connections 2020-11-28T21:55:23.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.043+0000 7fed18fa4700 1 --2- 172.21.15.162:0/1647518011 >> v2:172.21.15.118:3300/0 conn(0x7fed140b4c70 0x7fed14143020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:23.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.043+0000 7fed18fa4700 1 --2- 172.21.15.162:0/1647518011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed14142a00 0x7fed140b3a70 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:23.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.043+0000 7fed18fa4700 1 -- 172.21.15.162:0/1647518011 >> 172.21.15.162:0/1647518011 conn(0x7fed14066a40 msgr2=0x7fed14066e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:23.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.043+0000 7fed18fa4700 1 -- 172.21.15.162:0/1647518011 shutdown_connections 2020-11-28T21:55:23.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.043+0000 7fed18fa4700 1 -- 172.21.15.162:0/1647518011 wait complete. 2020-11-28T21:55:23.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.047+0000 7fed18fa4700 1 Processor -- start 2020-11-28T21:55:23.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.047+0000 7fed18fa4700 1 -- start start 2020-11-28T21:55:23.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.047+0000 7fed18fa4700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fed140603d0 0x7fed140b2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:23.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.047+0000 7fed18fa4700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed140b4c70 0x7fed14155250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:23.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.047+0000 7fed18fa4700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed14145cd0 0x7fed14151390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:23.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.047+0000 7fed18fa4700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed141548e0 con 0x7fed14145cd0 2020-11-28T21:55:23.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.047+0000 7fed18fa4700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1409f1a0 con 0x7fed140b4c70 2020-11-28T21:55:23.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.047+0000 7fed18fa4700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed14155740 con 0x7fed140603d0 2020-11-28T21:55:23.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.047+0000 7fed12ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed140b4c70 0x7fed14155250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:23.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.047+0000 7fed12ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed140b4c70 0x7fed14155250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44652/0 (socket says 172.21.15.162:44652) 2020-11-28T21:55:23.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.047+0000 7fed12ffd700 1 -- 172.21.15.162:0/26982391 learned_addr learned my addr 172.21.15.162:0/26982391 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:23.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.047+0000 7fed137fe700 1 --2- 172.21.15.162:0/26982391 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fed140603d0 0x7fed140b2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:23.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.047+0000 7fed13fff700 1 --2- 172.21.15.162:0/26982391 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed14145cd0 0x7fed14151390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:23.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.047+0000 7fed12ffd700 1 -- 172.21.15.162:0/26982391 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fed140603d0 msgr2=0x7fed140b2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:23.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.047+0000 7fed12ffd700 1 --2- 172.21.15.162:0/26982391 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fed140603d0 0x7fed140b2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:23.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.047+0000 7fed12ffd700 1 -- 172.21.15.162:0/26982391 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed14145cd0 msgr2=0x7fed14151390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:23.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.047+0000 7fed12ffd700 1 --2- 172.21.15.162:0/26982391 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed14145cd0 0x7fed14151390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:23.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.047+0000 7fed12ffd700 1 -- 172.21.15.162:0/26982391 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed14146250 con 0x7fed140b4c70 2020-11-28T21:55:23.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.047+0000 7fed12ffd700 1 --2- 172.21.15.162:0/26982391 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed140b4c70 0x7fed14155250 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 rx=0x7fed14145570 tx=0x7fed04015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:23.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.051+0000 7fed10ff9700 1 -- 172.21.15.162:0/26982391 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed04009e20 con 0x7fed140b4c70 2020-11-28T21:55:23.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.051+0000 7fed10ff9700 1 -- 172.21.15.162:0/26982391 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed04011ca0 con 0x7fed140b4c70 2020-11-28T21:55:23.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.051+0000 7fed18fa4700 1 -- 172.21.15.162:0/26982391 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed14154b10 con 0x7fed140b4c70 2020-11-28T21:55:23.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.051+0000 7fed10ff9700 1 -- 172.21.15.162:0/26982391 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed0401d440 con 0x7fed140b4c70 2020-11-28T21:55:23.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.051+0000 7fed18fa4700 1 -- 172.21.15.162:0/26982391 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed14145110 con 0x7fed140b4c70 2020-11-28T21:55:23.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.051+0000 7fed10ff9700 1 -- 172.21.15.162:0/26982391 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fed04024030 con 0x7fed140b4c70 2020-11-28T21:55:23.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.051+0000 7fed18fa4700 1 -- 172.21.15.162:0/26982391 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed14151a20 con 0x7fed140b4c70 2020-11-28T21:55:23.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.051+0000 7fed10ff9700 1 --2- 172.21.15.162:0/26982391 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fecf8055440 0x7fecf8057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:23.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.051+0000 7fed137fe700 1 --2- 172.21.15.162:0/26982391 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fecf8055440 0x7fecf8057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:23.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.051+0000 7fed10ff9700 1 -- 172.21.15.162:0/26982391 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fed0406eef0 con 0x7fed140b4c70 2020-11-28T21:55:23.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.051+0000 7fed137fe700 1 --2- 172.21.15.162:0/26982391 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fecf8055440 0x7fecf8057900 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7fecfc00b000 tx=0x7fecfc009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:23.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.055+0000 7fed10ff9700 1 -- 172.21.15.162:0/26982391 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fed04045090 con 0x7fed140b4c70 2020-11-28T21:55:23.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.239+0000 7fed18fa4700 1 -- 172.21.15.162:0/26982391 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed14092e00 con 0x7fed140b4c70 2020-11-28T21:55:23.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.243+0000 7fed10ff9700 1 -- 172.21.15.162:0/26982391 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+1426 (secure 0 0 0) 0x7fed04022070 con 0x7fed140b4c70 2020-11-28T21:55:23.247 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:23.247 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":62,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:55:19.286981+0000","modified":"2020-11-28T21:55:19.287109+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[],"up":{},"failed":[],"damaged":[],"stopped":[],"info":{},"data_pools":[22],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2020-11-28T21:55:23.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.243+0000 7fed18fa4700 1 -- 172.21.15.162:0/26982391 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fecf8055440 msgr2=0x7fecf8057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:23.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.243+0000 7fed18fa4700 1 --2- 172.21.15.162:0/26982391 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fecf8055440 0x7fecf8057900 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7fecfc00b000 tx=0x7fecfc009040).stop 2020-11-28T21:55:23.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.243+0000 7fed18fa4700 1 -- 172.21.15.162:0/26982391 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed140b4c70 msgr2=0x7fed14155250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:23.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.243+0000 7fed18fa4700 1 --2- 172.21.15.162:0/26982391 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed140b4c70 0x7fed14155250 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 rx=0x7fed14145570 tx=0x7fed04015000).stop 2020-11-28T21:55:23.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.243+0000 7fed18fa4700 1 -- 172.21.15.162:0/26982391 shutdown_connections 2020-11-28T21:55:23.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.243+0000 7fed18fa4700 1 --2- 172.21.15.162:0/26982391 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fecf8055440 0x7fecf8057900 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:23.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.243+0000 7fed18fa4700 1 --2- 172.21.15.162:0/26982391 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fed140603d0 0x7fed140b2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:23.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.243+0000 7fed18fa4700 1 --2- 172.21.15.162:0/26982391 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed14145cd0 0x7fed14151390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:23.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.243+0000 7fed18fa4700 1 --2- 172.21.15.162:0/26982391 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed140b4c70 0x7fed14155250 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:23.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.243+0000 7fed18fa4700 1 -- 172.21.15.162:0/26982391 >> 172.21.15.162:0/26982391 conn(0x7fed14066a40 msgr2=0x7fed14142df0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:23.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.243+0000 7fed18fa4700 1 -- 172.21.15.162:0/26982391 shutdown_connections 2020-11-28T21:55:23.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.243+0000 7fed18fa4700 1 -- 172.21.15.162:0/26982391 wait complete. 2020-11-28T21:55:23.251 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 62 2020-11-28T21:55:23.259 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2020-11-28T21:55:23.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.371+0000 7ff194b79700 1 Processor -- start 2020-11-28T21:55:23.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.371+0000 7ff194b79700 1 -- start start 2020-11-28T21:55:23.376 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.371+0000 7ff194b79700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff1900603d0 0x7ff1900b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:23.377 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.371+0000 7ff194b79700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff190142a00 0x7ff19009cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:23.377 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.371+0000 7ff194b79700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1900b4110 con 0x7ff1900b4c70 2020-11-28T21:55:23.377 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.371+0000 7ff194b79700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff190060f60 con 0x7ff190142a00 2020-11-28T21:55:23.377 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.371+0000 7ff194b79700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1900b43f0 con 0x7ff1900603d0 2020-11-28T21:55:23.378 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.371+0000 7ff18f7fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff1900603d0 0x7ff1900b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:23.378 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.371+0000 7ff18f7fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff1900603d0 0x7ff1900b3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44658/0 (socket says 172.21.15.162:44658) 2020-11-28T21:55:23.378 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.371+0000 7ff18f7fe700 1 -- 172.21.15.162:0/1593342321 learned_addr learned my addr 172.21.15.162:0/1593342321 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:23.378 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.371+0000 7ff18ffff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7ff1900b4c70 0x7ff190143020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51506/0 (socket says 172.21.15.162:51506) 2020-11-28T21:55:23.378 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.371+0000 7ff18effd700 1 --2- 172.21.15.162:0/1593342321 >> v2:172.21.15.118:3300/0 conn(0x7ff190142a00 0x7ff19009cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:23.379 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.371+0000 7ff18f7fe700 1 -- 172.21.15.162:0/1593342321 >> v2:172.21.15.118:3300/0 conn(0x7ff190142a00 msgr2=0x7ff19009cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:23.379 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.371+0000 7ff18f7fe700 1 --2- 172.21.15.162:0/1593342321 >> v2:172.21.15.118:3300/0 conn(0x7ff190142a00 0x7ff19009cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:23.379 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.371+0000 7ff18f7fe700 1 -- 172.21.15.162:0/1593342321 >> v1:172.21.15.162:6789/0 conn(0x7ff1900b4c70 legacy=0x7ff190143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:23.379 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.371+0000 7ff18f7fe700 1 -- 172.21.15.162:0/1593342321 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1900b4690 con 0x7ff1900603d0 2020-11-28T21:55:23.379 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.371+0000 7ff18f7fe700 1 --2- 172.21.15.162:0/1593342321 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1900603d0 0x7ff1900b3a70 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 rx=0x7ff180002f30 tx=0x7ff180006e80).ready entity=mon.1 client_cookie=1406f0bc5732622f server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:23.380 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.371+0000 7ff18e7fc700 1 -- 172.21.15.162:0/1593342321 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1800097f0 con 0x7ff1900603d0 2020-11-28T21:55:23.380 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.371+0000 7ff18e7fc700 1 -- 172.21.15.162:0/1593342321 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff180009950 con 0x7ff1900603d0 2020-11-28T21:55:23.380 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.371+0000 7ff18e7fc700 1 -- 172.21.15.162:0/1593342321 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff180009cb0 con 0x7ff1900603d0 2020-11-28T21:55:23.380 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.371+0000 7ff194b79700 1 -- 172.21.15.162:0/1593342321 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1900603d0 msgr2=0x7ff1900b3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:23.381 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.371+0000 7ff194b79700 1 --2- 172.21.15.162:0/1593342321 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1900603d0 0x7ff1900b3a70 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 rx=0x7ff180002f30 tx=0x7ff180006e80).stop 2020-11-28T21:55:23.383 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.371+0000 7ff194b79700 1 -- 172.21.15.162:0/1593342321 shutdown_connections 2020-11-28T21:55:23.384 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.371+0000 7ff194b79700 1 --2- 172.21.15.162:0/1593342321 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1900603d0 0x7ff1900b3a70 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:23.384 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.371+0000 7ff194b79700 1 --2- 172.21.15.162:0/1593342321 >> v2:172.21.15.118:3300/0 conn(0x7ff190142a00 0x7ff19009cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:23.384 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.371+0000 7ff194b79700 1 -- 172.21.15.162:0/1593342321 >> 172.21.15.162:0/1593342321 conn(0x7ff190066a40 msgr2=0x7ff190066e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:23.384 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.371+0000 7ff194b79700 1 -- 172.21.15.162:0/1593342321 shutdown_connections 2020-11-28T21:55:23.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.371+0000 7ff194b79700 1 -- 172.21.15.162:0/1593342321 wait complete. 2020-11-28T21:55:23.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.375+0000 7ff194b79700 1 Processor -- start 2020-11-28T21:55:23.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.375+0000 7ff194b79700 1 -- start start 2020-11-28T21:55:23.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.375+0000 7ff194b79700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff1900603d0 0x7ff19009e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:23.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.375+0000 7ff194b79700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff1900b4c70 0x7ff190155570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:23.386 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.375+0000 7ff194b79700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1900abb00 0x7ff1900abf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:23.386 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.375+0000 7ff194b79700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff190155a60 con 0x7ff1900603d0 2020-11-28T21:55:23.386 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.375+0000 7ff194b79700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1900b1160 con 0x7ff1900abb00 2020-11-28T21:55:23.386 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.375+0000 7ff194b79700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1900b1400 con 0x7ff1900b4c70 2020-11-28T21:55:23.386 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.375+0000 7ff18ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1900abb00 0x7ff1900abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:23.387 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.375+0000 7ff18ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1900abb00 0x7ff1900abf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44666/0 (socket says 172.21.15.162:44666) 2020-11-28T21:55:23.387 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.375+0000 7ff18ffff700 1 -- 172.21.15.162:0/2727591651 learned_addr learned my addr 172.21.15.162:0/2727591651 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:23.387 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.375+0000 7ff18effd700 1 --2- 172.21.15.162:0/2727591651 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff1900b4c70 0x7ff190155570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:23.387 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.375+0000 7ff18f7fe700 1 --2- 172.21.15.162:0/2727591651 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff1900603d0 0x7ff19009e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:23.387 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.375+0000 7ff18ffff700 1 -- 172.21.15.162:0/2727591651 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff1900b4c70 msgr2=0x7ff190155570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:23.388 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.375+0000 7ff18ffff700 1 --2- 172.21.15.162:0/2727591651 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff1900b4c70 0x7ff190155570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:23.388 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.375+0000 7ff18ffff700 1 -- 172.21.15.162:0/2727591651 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff1900603d0 msgr2=0x7ff19009e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:23.388 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.375+0000 7ff18ffff700 1 --2- 172.21.15.162:0/2727591651 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff1900603d0 0x7ff19009e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:23.388 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.375+0000 7ff18ffff700 1 -- 172.21.15.162:0/2727591651 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1900b16a0 con 0x7ff1900abb00 2020-11-28T21:55:23.388 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.375+0000 7ff18f7fe700 1 --2- 172.21.15.162:0/2727591651 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff1900603d0 0x7ff19009e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:55:23.389 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.375+0000 7ff18ffff700 1 --2- 172.21.15.162:0/2727591651 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1900abb00 0x7ff1900abf90 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 rx=0x7ff1900b91f0 tx=0x7ff188007f10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:23.389 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.375+0000 7ff18cff9700 1 -- 172.21.15.162:0/2727591651 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff188005ab0 con 0x7ff1900abb00 2020-11-28T21:55:23.389 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.375+0000 7ff18cff9700 1 -- 172.21.15.162:0/2727591651 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff188005c10 con 0x7ff1900abb00 2020-11-28T21:55:23.389 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.375+0000 7ff194b79700 1 -- 172.21.15.162:0/2727591651 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1900b1940 con 0x7ff1900abb00 2020-11-28T21:55:23.389 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.375+0000 7ff18cff9700 1 -- 172.21.15.162:0/2727591651 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff188007110 con 0x7ff1900abb00 2020-11-28T21:55:23.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.375+0000 7ff194b79700 1 -- 172.21.15.162:0/2727591651 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1900b8dc0 con 0x7ff1900abb00 2020-11-28T21:55:23.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.375+0000 7ff194b79700 1 -- 172.21.15.162:0/2727591651 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1900b1ad0 con 0x7ff1900abb00 2020-11-28T21:55:23.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.379+0000 7ff18cff9700 1 -- 172.21.15.162:0/2727591651 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff188010030 con 0x7ff1900abb00 2020-11-28T21:55:23.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.379+0000 7ff18cff9700 1 --2- 172.21.15.162:0/2727591651 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff1700554e0 0x7ff1700579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:23.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.379+0000 7ff18cff9700 1 -- 172.21.15.162:0/2727591651 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6159+0+0 (secure 0 0 0) 0x7ff18806df90 con 0x7ff1900abb00 2020-11-28T21:55:23.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.379+0000 7ff18f7fe700 1 --2- 172.21.15.162:0/2727591651 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff1700554e0 0x7ff1700579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:23.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.379+0000 7ff18f7fe700 1 --2- 172.21.15.162:0/2727591651 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff1700554e0 0x7ff1700579a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7ff180017000 tx=0x7ff180015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:23.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.387+0000 7ff18cff9700 1 -- 172.21.15.162:0/2727591651 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff1880441e0 con 0x7ff1900abb00 2020-11-28T21:55:23.574 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.571+0000 7ff194b79700 1 -- 172.21.15.162:0/2727591651 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff190057050 con 0x7ff1900abb00 2020-11-28T21:55:23.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.975+0000 7ff18cff9700 1 -- 172.21.15.162:0/2727591651 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v63) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff188047800 con 0x7ff1900abb00 2020-11-28T21:55:23.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.979+0000 7ff194b79700 1 -- 172.21.15.162:0/2727591651 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff1700554e0 msgr2=0x7ff1700579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:23.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.979+0000 7ff194b79700 1 --2- 172.21.15.162:0/2727591651 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff1700554e0 0x7ff1700579a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7ff180017000 tx=0x7ff180015000).stop 2020-11-28T21:55:23.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.979+0000 7ff194b79700 1 -- 172.21.15.162:0/2727591651 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1900abb00 msgr2=0x7ff1900abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:23.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.979+0000 7ff194b79700 1 --2- 172.21.15.162:0/2727591651 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1900abb00 0x7ff1900abf90 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 rx=0x7ff1900b91f0 tx=0x7ff188007f10).stop 2020-11-28T21:55:23.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.979+0000 7ff194b79700 1 -- 172.21.15.162:0/2727591651 shutdown_connections 2020-11-28T21:55:23.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.979+0000 7ff194b79700 1 --2- 172.21.15.162:0/2727591651 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff1700554e0 0x7ff1700579a0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:23.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.979+0000 7ff194b79700 1 --2- 172.21.15.162:0/2727591651 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1900abb00 0x7ff1900abf90 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:23.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.979+0000 7ff194b79700 1 --2- 172.21.15.162:0/2727591651 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff1900603d0 0x7ff19009e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:23.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.979+0000 7ff194b79700 1 --2- 172.21.15.162:0/2727591651 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff1900b4c70 0x7ff190155570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:23.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.979+0000 7ff194b79700 1 -- 172.21.15.162:0/2727591651 >> 172.21.15.162:0/2727591651 conn(0x7ff190066a40 msgr2=0x7ff19013de10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:23.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.979+0000 7ff194b79700 1 -- 172.21.15.162:0/2727591651 shutdown_connections 2020-11-28T21:55:23.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:23.979+0000 7ff194b79700 1 -- 172.21.15.162:0/2727591651 wait complete. 2020-11-28T21:55:23.987 INFO:teuthology.orchestra.run.smithi162.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-11-28T21:55:23.998 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:55:24.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.115+0000 7ff67ae11700 1 Processor -- start 2020-11-28T21:55:24.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.115+0000 7ff67ae11700 1 -- start start 2020-11-28T21:55:24.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.115+0000 7ff67ae11700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff674098b30 0x7ff6740994f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:24.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.115+0000 7ff67ae11700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff6740b0000 0x7ff674060000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:24.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.115+0000 7ff67ae11700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff674098f10 con 0x7ff6740604f0 2020-11-28T21:55:24.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.115+0000 7ff67ae11700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6740b1030 con 0x7ff674098b30 2020-11-28T21:55:24.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.115+0000 7ff67ae11700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6740b0480 con 0x7ff6740b0000 2020-11-28T21:55:24.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.115+0000 7ff67960e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff6740b0000 0x7ff674060000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:24.129 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.115+0000 7ff67960e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff6740b0000 0x7ff674060000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44672/0 (socket says 172.21.15.162:44672) 2020-11-28T21:55:24.129 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.115+0000 7ff67960e700 1 -- 172.21.15.162:0/1019740110 learned_addr learned my addr 172.21.15.162:0/1019740110 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:24.129 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.115+0000 7ff679e0f700 1 --2- 172.21.15.162:0/1019740110 >> v2:172.21.15.118:3300/0 conn(0x7ff674098b30 0x7ff6740994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:24.129 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.115+0000 7ff67960e700 1 -- 172.21.15.162:0/1019740110 >> v2:172.21.15.118:3300/0 conn(0x7ff674098b30 msgr2=0x7ff6740994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:24.129 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.115+0000 7ff67960e700 1 --2- 172.21.15.162:0/1019740110 >> v2:172.21.15.118:3300/0 conn(0x7ff674098b30 0x7ff6740994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:24.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.115+0000 7ff67960e700 1 -- 172.21.15.162:0/1019740110 >> v1:172.21.15.118:6789/0 conn(0x7ff6740604f0 legacy=0x7ff67414a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:24.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.115+0000 7ff67960e700 1 -- 172.21.15.162:0/1019740110 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6740b0680 con 0x7ff6740b0000 2020-11-28T21:55:24.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.115+0000 7ff679e0f700 1 --2- 172.21.15.162:0/1019740110 >> v2:172.21.15.118:3300/0 conn(0x7ff674098b30 0x7ff6740994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:55:24.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.115+0000 7ff67960e700 1 --2- 172.21.15.162:0/1019740110 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6740b0000 0x7ff674060000 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 rx=0x7ff674061f00 tx=0x7ff670006e80).ready entity=mon.1 client_cookie=6581124be3ad9caf server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:24.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.115+0000 7ff678e0d700 1 -- 172.21.15.162:0/1019740110 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6700097b0 con 0x7ff6740b0000 2020-11-28T21:55:24.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.115+0000 7ff678e0d700 1 -- 172.21.15.162:0/1019740110 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff670009910 con 0x7ff6740b0000 2020-11-28T21:55:24.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.115+0000 7ff678e0d700 1 -- 172.21.15.162:0/1019740110 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff670009a80 con 0x7ff6740b0000 2020-11-28T21:55:24.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.115+0000 7ff67ae11700 1 -- 172.21.15.162:0/1019740110 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6740b0000 msgr2=0x7ff674060000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:24.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.115+0000 7ff67ae11700 1 --2- 172.21.15.162:0/1019740110 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6740b0000 0x7ff674060000 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 rx=0x7ff674061f00 tx=0x7ff670006e80).stop 2020-11-28T21:55:24.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.119+0000 7ff67ae11700 1 -- 172.21.15.162:0/1019740110 shutdown_connections 2020-11-28T21:55:24.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.119+0000 7ff67ae11700 1 --2- 172.21.15.162:0/1019740110 >> v2:172.21.15.118:3300/0 conn(0x7ff674098b30 0x7ff6740994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:24.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.119+0000 7ff67ae11700 1 --2- 172.21.15.162:0/1019740110 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6740b0000 0x7ff674060000 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:24.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.119+0000 7ff67ae11700 1 -- 172.21.15.162:0/1019740110 >> 172.21.15.162:0/1019740110 conn(0x7ff67405b970 msgr2=0x7ff67405bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:24.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.119+0000 7ff67ae11700 1 -- 172.21.15.162:0/1019740110 shutdown_connections 2020-11-28T21:55:24.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.119+0000 7ff67ae11700 1 -- 172.21.15.162:0/1019740110 wait complete. 2020-11-28T21:55:24.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.119+0000 7ff67ae11700 1 Processor -- start 2020-11-28T21:55:24.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.119+0000 7ff67ae11700 1 -- start start 2020-11-28T21:55:24.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.119+0000 7ff67ae11700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6740604f0 0x7ff6740aa880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:24.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.119+0000 7ff67ae11700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff674098b30 0x7ff6740ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:24.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.119+0000 7ff67ae11700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6740b0000 0x7ff6740addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:24.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.119+0000 7ff67ae11700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff674065de0 con 0x7ff674098b30 2020-11-28T21:55:24.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.119+0000 7ff67ae11700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff674063ea0 con 0x7ff6740604f0 2020-11-28T21:55:24.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.119+0000 7ff67ae11700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6740636c0 con 0x7ff6740b0000 2020-11-28T21:55:24.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.119+0000 7ff679e0f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6740604f0 0x7ff6740aa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:24.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.119+0000 7ff679e0f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6740604f0 0x7ff6740aa880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44680/0 (socket says 172.21.15.162:44680) 2020-11-28T21:55:24.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.119+0000 7ff679e0f700 1 -- 172.21.15.162:0/2324675474 learned_addr learned my addr 172.21.15.162:0/2324675474 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:24.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.119+0000 7ff67960e700 1 --2- 172.21.15.162:0/2324675474 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff674098b30 0x7ff6740ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:24.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.119+0000 7ff67a610700 1 --2- 172.21.15.162:0/2324675474 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6740b0000 0x7ff6740addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:24.138 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.119+0000 7ff679e0f700 1 -- 172.21.15.162:0/2324675474 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6740b0000 msgr2=0x7ff6740addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:24.138 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.119+0000 7ff679e0f700 1 --2- 172.21.15.162:0/2324675474 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6740b0000 0x7ff6740addd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:24.139 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.119+0000 7ff679e0f700 1 -- 172.21.15.162:0/2324675474 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff674098b30 msgr2=0x7ff6740ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:24.139 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.119+0000 7ff679e0f700 1 --2- 172.21.15.162:0/2324675474 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff674098b30 0x7ff6740ad8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:24.139 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.119+0000 7ff679e0f700 1 -- 172.21.15.162:0/2324675474 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff674063960 con 0x7ff6740604f0 2020-11-28T21:55:24.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.119+0000 7ff67a610700 1 --2- 172.21.15.162:0/2324675474 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6740b0000 0x7ff6740addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:55:24.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.119+0000 7ff679e0f700 1 --2- 172.21.15.162:0/2324675474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6740604f0 0x7ff6740aa880 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 rx=0x7ff674064e50 tx=0x7ff664008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:24.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.119+0000 7ff662ffd700 1 -- 172.21.15.162:0/2324675474 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff664010580 con 0x7ff6740604f0 2020-11-28T21:55:24.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.119+0000 7ff662ffd700 1 -- 172.21.15.162:0/2324675474 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6640106e0 con 0x7ff6740604f0 2020-11-28T21:55:24.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.119+0000 7ff67ae11700 1 -- 172.21.15.162:0/2324675474 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff674063c00 con 0x7ff6740604f0 2020-11-28T21:55:24.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.119+0000 7ff662ffd700 1 -- 172.21.15.162:0/2324675474 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff664010a40 con 0x7ff6740604f0 2020-11-28T21:55:24.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.119+0000 7ff67ae11700 1 -- 172.21.15.162:0/2324675474 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6740ab620 con 0x7ff6740604f0 2020-11-28T21:55:24.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.123+0000 7ff662ffd700 1 -- 172.21.15.162:0/2324675474 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff664010ba0 con 0x7ff6740604f0 2020-11-28T21:55:24.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.123+0000 7ff67ae11700 1 -- 172.21.15.162:0/2324675474 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff674057050 con 0x7ff6740604f0 2020-11-28T21:55:24.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.123+0000 7ff662ffd700 1 --2- 172.21.15.162:0/2324675474 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff654055330 0x7ff6540577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:24.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.123+0000 7ff67960e700 1 --2- 172.21.15.162:0/2324675474 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff654055330 0x7ff6540577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:24.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.123+0000 7ff662ffd700 1 -- 172.21.15.162:0/2324675474 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6159+0+0 (secure 0 0 0) 0x7ff66406e1c0 con 0x7ff6740604f0 2020-11-28T21:55:24.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.123+0000 7ff67960e700 1 --2- 172.21.15.162:0/2324675474 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff654055330 0x7ff6540577f0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7ff67001c000 tx=0x7ff670016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:24.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.127+0000 7ff662ffd700 1 -- 172.21.15.162:0/2324675474 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff664044300 con 0x7ff6740604f0 2020-11-28T21:55:24.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.311+0000 7ff67ae11700 1 -- 172.21.15.162:0/2324675474 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff674155ee0 con 0x7ff6740604f0 2020-11-28T21:55:24.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.311+0000 7ff662ffd700 1 -- 172.21.15.162:0/2324675474 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+1426 (secure 0 0 0) 0x7ff664047920 con 0x7ff6740604f0 2020-11-28T21:55:24.316 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:24.317 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":63,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":63,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:55:19.286981+0000","modified":"2020-11-28T21:55:23.966323+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[],"up":{},"failed":[],"damaged":[],"stopped":[],"info":{},"data_pools":[22],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2020-11-28T21:55:24.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.315+0000 7ff67ae11700 1 -- 172.21.15.162:0/2324675474 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff654055330 msgr2=0x7ff6540577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:24.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.315+0000 7ff67ae11700 1 --2- 172.21.15.162:0/2324675474 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff654055330 0x7ff6540577f0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7ff67001c000 tx=0x7ff670016000).stop 2020-11-28T21:55:24.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.315+0000 7ff67ae11700 1 -- 172.21.15.162:0/2324675474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6740604f0 msgr2=0x7ff6740aa880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:24.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.315+0000 7ff67ae11700 1 --2- 172.21.15.162:0/2324675474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6740604f0 0x7ff6740aa880 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 rx=0x7ff674064e50 tx=0x7ff664008040).stop 2020-11-28T21:55:24.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.315+0000 7ff67ae11700 1 -- 172.21.15.162:0/2324675474 shutdown_connections 2020-11-28T21:55:24.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.315+0000 7ff67ae11700 1 --2- 172.21.15.162:0/2324675474 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff654055330 0x7ff6540577f0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:24.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.315+0000 7ff67ae11700 1 --2- 172.21.15.162:0/2324675474 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6740b0000 0x7ff6740addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:24.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.315+0000 7ff67ae11700 1 --2- 172.21.15.162:0/2324675474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6740604f0 0x7ff6740aa880 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:24.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.315+0000 7ff67ae11700 1 --2- 172.21.15.162:0/2324675474 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff674098b30 0x7ff6740ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:24.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.315+0000 7ff67ae11700 1 -- 172.21.15.162:0/2324675474 >> 172.21.15.162:0/2324675474 conn(0x7ff67405b970 msgr2=0x7ff6741494b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:24.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.315+0000 7ff67ae11700 1 -- 172.21.15.162:0/2324675474 shutdown_connections 2020-11-28T21:55:24.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.315+0000 7ff67ae11700 1 -- 172.21.15.162:0/2324675474 wait complete. 2020-11-28T21:55:24.324 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 63 2020-11-28T21:55:24.337 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2020-11-28T21:55:24.453 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.451+0000 7ffaaccbc700 1 Processor -- start 2020-11-28T21:55:24.454 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.451+0000 7ffaaccbc700 1 -- start start 2020-11-28T21:55:24.456 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.451+0000 7ffaaccbc700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ffaa80b5bf0 0x7ffaa8142980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:24.456 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.451+0000 7ffaaccbc700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ffaa8060cb0 0x7ffaa80b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:24.457 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.451+0000 7ffaaccbc700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffaa8060230 0x7ffaa813fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:24.457 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.451+0000 7ffaaccbc700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaa805f380 con 0x7ffaa8060cb0 2020-11-28T21:55:24.457 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.451+0000 7ffaaccbc700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaa8143930 con 0x7ffaa80b5bf0 2020-11-28T21:55:24.457 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.451+0000 7ffaaccbc700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa80b3f50 con 0x7ffaa8060230 2020-11-28T21:55:24.457 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.451+0000 7ffaa77fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ffaa80b5bf0 0x7ffaa8142980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:24.458 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.451+0000 7ffaa77fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ffaa80b5bf0 0x7ffaa8142980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44686/0 (socket says 172.21.15.162:44686) 2020-11-28T21:55:24.458 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.451+0000 7ffaa77fe700 1 -- 172.21.15.162:0/96275005 learned_addr learned my addr 172.21.15.162:0/96275005 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:24.458 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.451+0000 7ffaa6ffd700 1 --2- 172.21.15.162:0/96275005 >> v2:172.21.15.118:3300/0 conn(0x7ffaa8060cb0 0x7ffaa80b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:24.458 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.451+0000 7ffaa7fff700 1 --2- 172.21.15.162:0/96275005 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffaa8060230 0x7ffaa813fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:24.458 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.451+0000 7ffaa77fe700 1 -- 172.21.15.162:0/96275005 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffaa8060230 msgr2=0x7ffaa813fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:24.459 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.451+0000 7ffaa77fe700 1 --2- 172.21.15.162:0/96275005 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffaa8060230 0x7ffaa813fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:24.459 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.451+0000 7ffaa77fe700 1 -- 172.21.15.162:0/96275005 >> v2:172.21.15.118:3300/0 conn(0x7ffaa8060cb0 msgr2=0x7ffaa80b3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:24.459 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.451+0000 7ffaa77fe700 1 --2- 172.21.15.162:0/96275005 >> v2:172.21.15.118:3300/0 conn(0x7ffaa8060cb0 0x7ffaa80b3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:24.459 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.451+0000 7ffaa77fe700 1 -- 172.21.15.162:0/96275005 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa805f5b0 con 0x7ffaa80b5bf0 2020-11-28T21:55:24.459 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.451+0000 7ffaa6ffd700 1 --2- 172.21.15.162:0/96275005 >> v2:172.21.15.118:3300/0 conn(0x7ffaa8060cb0 0x7ffaa80b3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:55:24.460 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.451+0000 7ffaa7fff700 1 --2- 172.21.15.162:0/96275005 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffaa8060230 0x7ffaa813fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:55:24.460 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.451+0000 7ffaa77fe700 1 --2- 172.21.15.162:0/96275005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffaa80b5bf0 0x7ffaa8142980 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 rx=0x7ffa98002f30 tx=0x7ffa98006e80).ready entity=mon.1 client_cookie=51102c45b555b656 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:24.460 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.451+0000 7ffaa67fc700 1 -- 172.21.15.162:0/96275005 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa980097f0 con 0x7ffaa80b5bf0 2020-11-28T21:55:24.460 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.451+0000 7ffaa67fc700 1 -- 172.21.15.162:0/96275005 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa98009950 con 0x7ffaa80b5bf0 2020-11-28T21:55:24.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.451+0000 7ffaa67fc700 1 -- 172.21.15.162:0/96275005 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa98009ac0 con 0x7ffaa80b5bf0 2020-11-28T21:55:24.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.451+0000 7ffaaccbc700 1 -- 172.21.15.162:0/96275005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffaa80b5bf0 msgr2=0x7ffaa8142980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:24.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.451+0000 7ffaaccbc700 1 --2- 172.21.15.162:0/96275005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffaa80b5bf0 0x7ffaa8142980 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 rx=0x7ffa98002f30 tx=0x7ffa98006e80).stop 2020-11-28T21:55:24.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.451+0000 7ffaaccbc700 1 -- 172.21.15.162:0/96275005 shutdown_connections 2020-11-28T21:55:24.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.451+0000 7ffaaccbc700 1 --2- 172.21.15.162:0/96275005 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffaa8060230 0x7ffaa813fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:24.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.451+0000 7ffaaccbc700 1 --2- 172.21.15.162:0/96275005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffaa80b5bf0 0x7ffaa8142980 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:24.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.451+0000 7ffaaccbc700 1 --2- 172.21.15.162:0/96275005 >> v2:172.21.15.118:3300/0 conn(0x7ffaa8060cb0 0x7ffaa80b3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:24.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.451+0000 7ffaaccbc700 1 -- 172.21.15.162:0/96275005 >> 172.21.15.162:0/96275005 conn(0x7ffaa805b970 msgr2=0x7ffaa805bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:24.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.451+0000 7ffaaccbc700 1 -- 172.21.15.162:0/96275005 shutdown_connections 2020-11-28T21:55:24.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.451+0000 7ffaaccbc700 1 -- 172.21.15.162:0/96275005 wait complete. 2020-11-28T21:55:24.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.451+0000 7ffaaccbc700 1 Processor -- start 2020-11-28T21:55:24.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.451+0000 7ffaaccbc700 1 -- start start 2020-11-28T21:55:24.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.455+0000 7ffaaccbc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffaa8060230 0x7ffaa80b84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:24.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.455+0000 7ffaaccbc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffaa8060cb0 0x7ffaa809ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:24.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.455+0000 7ffaaccbc700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffaa80b5bf0 0x7ffaa8143d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:24.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.455+0000 7ffaaccbc700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa80aefe0 con 0x7ffaa8060cb0 2020-11-28T21:55:24.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.455+0000 7ffaaccbc700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa80afab0 con 0x7ffaa8060230 2020-11-28T21:55:24.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.455+0000 7ffaaccbc700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa809f130 con 0x7ffaa80b5bf0 2020-11-28T21:55:24.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.455+0000 7ffaa77fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffaa8060230 0x7ffaa80b84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:24.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.455+0000 7ffaa77fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffaa8060230 0x7ffaa80b84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44692/0 (socket says 172.21.15.162:44692) 2020-11-28T21:55:24.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.455+0000 7ffaa77fe700 1 -- 172.21.15.162:0/1419793337 learned_addr learned my addr 172.21.15.162:0/1419793337 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:24.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.455+0000 7ffaa7fff700 1 --2- 172.21.15.162:0/1419793337 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffaa80b5bf0 0x7ffaa8143d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:24.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.455+0000 7ffaa6ffd700 1 --2- 172.21.15.162:0/1419793337 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffaa8060cb0 0x7ffaa809ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:24.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.455+0000 7ffaa77fe700 1 -- 172.21.15.162:0/1419793337 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffaa80b5bf0 msgr2=0x7ffaa8143d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:24.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.455+0000 7ffaa77fe700 1 --2- 172.21.15.162:0/1419793337 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffaa80b5bf0 0x7ffaa8143d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:24.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.455+0000 7ffaa77fe700 1 -- 172.21.15.162:0/1419793337 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffaa8060cb0 msgr2=0x7ffaa809ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:24.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.455+0000 7ffaa77fe700 1 --2- 172.21.15.162:0/1419793337 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffaa8060cb0 0x7ffaa809ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:24.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.455+0000 7ffaa77fe700 1 -- 172.21.15.162:0/1419793337 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa80afce0 con 0x7ffaa8060230 2020-11-28T21:55:24.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.455+0000 7ffaa6ffd700 1 --2- 172.21.15.162:0/1419793337 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffaa8060cb0 0x7ffaa809ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:55:24.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.455+0000 7ffaa77fe700 1 --2- 172.21.15.162:0/1419793337 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffaa8060230 0x7ffaa80b84a0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 rx=0x7ffaa8151670 tx=0x7ffa98016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:24.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.455+0000 7ffaa4ff9700 1 -- 172.21.15.162:0/1419793337 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa98011e70 con 0x7ffaa8060230 2020-11-28T21:55:24.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.455+0000 7ffaa4ff9700 1 -- 172.21.15.162:0/1419793337 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa98020400 con 0x7ffaa8060230 2020-11-28T21:55:24.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.455+0000 7ffaaccbc700 1 -- 172.21.15.162:0/1419793337 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaa809d800 con 0x7ffaa8060230 2020-11-28T21:55:24.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.455+0000 7ffaa4ff9700 1 -- 172.21.15.162:0/1419793337 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa98020760 con 0x7ffaa8060230 2020-11-28T21:55:24.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.455+0000 7ffaaccbc700 1 -- 172.21.15.162:0/1419793337 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaa81441f0 con 0x7ffaa8060230 2020-11-28T21:55:24.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.455+0000 7ffaaccbc700 1 -- 172.21.15.162:0/1419793337 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaa809eab0 con 0x7ffaa8060230 2020-11-28T21:55:24.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.455+0000 7ffaa4ff9700 1 -- 172.21.15.162:0/1419793337 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ffa98027030 con 0x7ffaa8060230 2020-11-28T21:55:24.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.459+0000 7ffaa4ff9700 1 --2- 172.21.15.162:0/1419793337 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ffa8c0554e0 0x7ffa8c0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:24.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.459+0000 7ffaa6ffd700 1 --2- 172.21.15.162:0/1419793337 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ffa8c0554e0 0x7ffa8c0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:24.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.459+0000 7ffaa4ff9700 1 -- 172.21.15.162:0/1419793337 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6159+0+0 (secure 0 0 0) 0x7ffa9806ed80 con 0x7ffaa8060230 2020-11-28T21:55:24.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.459+0000 7ffaa6ffd700 1 --2- 172.21.15.162:0/1419793337 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ffa8c0554e0 0x7ffa8c0579a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7ffa9000b000 tx=0x7ffa90008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:24.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.467+0000 7ffaa4ff9700 1 -- 172.21.15.162:0/1419793337 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ffa98044f20 con 0x7ffaa8060230 2020-11-28T21:55:24.657 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.651+0000 7ffaaccbc700 1 -- 172.21.15.162:0/1419793337 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ffaa8057050 con 0x7ffaa8060230 2020-11-28T21:55:24.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.979+0000 7ffaa4ff9700 1 -- 172.21.15.162:0/1419793337 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v64) v1 ==== 104+0+0 (secure 0 0 0) 0x7ffa980150a0 con 0x7ffaa8060230 2020-11-28T21:55:24.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.983+0000 7ffaaccbc700 1 -- 172.21.15.162:0/1419793337 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ffa8c0554e0 msgr2=0x7ffa8c0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:24.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.983+0000 7ffaaccbc700 1 --2- 172.21.15.162:0/1419793337 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ffa8c0554e0 0x7ffa8c0579a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7ffa9000b000 tx=0x7ffa90008040).stop 2020-11-28T21:55:24.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.983+0000 7ffaaccbc700 1 -- 172.21.15.162:0/1419793337 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffaa8060230 msgr2=0x7ffaa80b84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:24.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.983+0000 7ffaaccbc700 1 --2- 172.21.15.162:0/1419793337 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffaa8060230 0x7ffaa80b84a0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 rx=0x7ffaa8151670 tx=0x7ffa98016000).stop 2020-11-28T21:55:24.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.983+0000 7ffaaccbc700 1 -- 172.21.15.162:0/1419793337 shutdown_connections 2020-11-28T21:55:24.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.983+0000 7ffaaccbc700 1 --2- 172.21.15.162:0/1419793337 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ffa8c0554e0 0x7ffa8c0579a0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:24.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.983+0000 7ffaaccbc700 1 --2- 172.21.15.162:0/1419793337 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffaa80b5bf0 0x7ffaa8143d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:24.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.983+0000 7ffaaccbc700 1 --2- 172.21.15.162:0/1419793337 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffaa8060230 0x7ffaa80b84a0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:24.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.983+0000 7ffaaccbc700 1 --2- 172.21.15.162:0/1419793337 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffaa8060cb0 0x7ffaa809ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:24.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.983+0000 7ffaaccbc700 1 -- 172.21.15.162:0/1419793337 >> 172.21.15.162:0/1419793337 conn(0x7ffaa805b970 msgr2=0x7ffaa8140e00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:24.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.983+0000 7ffaaccbc700 1 -- 172.21.15.162:0/1419793337 shutdown_connections 2020-11-28T21:55:24.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:24.983+0000 7ffaaccbc700 1 -- 172.21.15.162:0/1419793337 wait complete. 2020-11-28T21:55:25.005 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-11-28T21:55:25.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.127+0000 7f147c932700 1 Processor -- start 2020-11-28T21:55:25.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.127+0000 7f147c932700 1 -- start start 2020-11-28T21:55:25.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.131+0000 7f147c932700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14780b3950 0x7f14780b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:25.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.131+0000 7f147c932700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f14780b44e0 0x7f14780b2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:25.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.131+0000 7f147c932700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14780b1f50 con 0x7f14780b18c0 2020-11-28T21:55:25.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.131+0000 7f147c932700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14780aab50 con 0x7f14780b44e0 2020-11-28T21:55:25.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.131+0000 7f147c932700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1478143e90 con 0x7f14780b3950 2020-11-28T21:55:25.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.131+0000 7f1476ffd700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f14780b18c0 0x7f147813dd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51544/0 (socket says 172.21.15.162:51544) 2020-11-28T21:55:25.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.131+0000 7f1476ffd700 1 -- 172.21.15.162:0/1310692264 learned_addr learned my addr 172.21.15.162:0/1310692264 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:25.138 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.131+0000 7f1477fff700 1 --2- 172.21.15.162:0/1310692264 >> v2:172.21.15.118:3300/0 conn(0x7f14780b44e0 0x7f14780b2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:25.138 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.131+0000 7f14777fe700 1 --2- 172.21.15.162:0/1310692264 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14780b3950 0x7f14780b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:25.138 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.131+0000 7f14767fc700 1 -- 172.21.15.162:0/1310692264 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3371883661 0 0) 0x7f14780b1f50 con 0x7f14780b18c0 2020-11-28T21:55:25.138 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.131+0000 7f14767fc700 1 -- 172.21.15.162:0/1310692264 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1464003230 con 0x7f14780b18c0 2020-11-28T21:55:25.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.131+0000 7f1477fff700 1 -- 172.21.15.162:0/1310692264 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14780b3950 msgr2=0x7f14780b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:25.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.131+0000 7f1477fff700 1 --2- 172.21.15.162:0/1310692264 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14780b3950 0x7f14780b3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:25.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.131+0000 7f14767fc700 1 -- 172.21.15.162:0/1310692264 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1236231184 0 0) 0x7f1464003230 con 0x7f14780b18c0 2020-11-28T21:55:25.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.131+0000 7f1477fff700 1 -- 172.21.15.162:0/1310692264 >> v1:172.21.15.162:6789/0 conn(0x7f14780b18c0 legacy=0x7f147813dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:25.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.131+0000 7f1477fff700 1 -- 172.21.15.162:0/1310692264 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14780aa8b0 con 0x7f14780b44e0 2020-11-28T21:55:25.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.131+0000 7f14777fe700 1 --2- 172.21.15.162:0/1310692264 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14780b3950 0x7f14780b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:55:25.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.131+0000 7f1477fff700 1 --2- 172.21.15.162:0/1310692264 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14780b44e0 0x7f14780b2c10 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7f146c001870 tx=0x7f14700091a0).ready entity=mon.0 client_cookie=ad3b99c512ed00d4 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:25.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.131+0000 7f14767fc700 1 -- 172.21.15.162:0/1310692264 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f146c0016d0 con 0x7f14780b44e0 2020-11-28T21:55:25.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.131+0000 7f14767fc700 1 -- 172.21.15.162:0/1310692264 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f147000b800 con 0x7f14780b44e0 2020-11-28T21:55:25.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.131+0000 7f14767fc700 1 -- 172.21.15.162:0/1310692264 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f147000b970 con 0x7f14780b44e0 2020-11-28T21:55:25.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.131+0000 7f147c932700 1 -- 172.21.15.162:0/1310692264 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14780b44e0 msgr2=0x7f14780b2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:25.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.131+0000 7f147c932700 1 --2- 172.21.15.162:0/1310692264 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14780b44e0 0x7f14780b2c10 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7f146c001870 tx=0x7f14700091a0).stop 2020-11-28T21:55:25.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.131+0000 7f147c932700 1 -- 172.21.15.162:0/1310692264 shutdown_connections 2020-11-28T21:55:25.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.131+0000 7f147c932700 1 --2- 172.21.15.162:0/1310692264 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14780b44e0 0x7f14780b2c10 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:25.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.131+0000 7f147c932700 1 --2- 172.21.15.162:0/1310692264 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14780b3950 0x7f14780b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:25.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.131+0000 7f147c932700 1 -- 172.21.15.162:0/1310692264 >> 172.21.15.162:0/1310692264 conn(0x7f1478062630 msgr2=0x7f1478062a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:25.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.131+0000 7f147c932700 1 -- 172.21.15.162:0/1310692264 shutdown_connections 2020-11-28T21:55:25.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.131+0000 7f147c932700 1 -- 172.21.15.162:0/1310692264 wait complete. 2020-11-28T21:55:25.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.135+0000 7f147c932700 1 Processor -- start 2020-11-28T21:55:25.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.135+0000 7f147c932700 1 -- start start 2020-11-28T21:55:25.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.135+0000 7f147c932700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14780b18c0 0x7f14780b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:25.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.135+0000 7f147c932700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14780b3950 0x7f1478147010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:25.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.135+0000 7f147c932700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14780b44e0 0x7f1478147500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:25.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.135+0000 7f147c932700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14780aed80 con 0x7f14780b3950 2020-11-28T21:55:25.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.135+0000 7f147c932700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14780af280 con 0x7f14780b18c0 2020-11-28T21:55:25.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.135+0000 7f147c932700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1478146850 con 0x7f14780b44e0 2020-11-28T21:55:25.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.135+0000 7f14777fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14780b18c0 0x7f14780b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:25.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.135+0000 7f14777fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14780b18c0 0x7f14780b7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44706/0 (socket says 172.21.15.162:44706) 2020-11-28T21:55:25.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.135+0000 7f14777fe700 1 -- 172.21.15.162:0/442088759 learned_addr learned my addr 172.21.15.162:0/442088759 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:25.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.135+0000 7f1477fff700 1 --2- 172.21.15.162:0/442088759 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14780b44e0 0x7f1478147500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:25.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.135+0000 7f1476ffd700 1 --2- 172.21.15.162:0/442088759 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14780b3950 0x7f1478147010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:25.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.135+0000 7f14777fe700 1 -- 172.21.15.162:0/442088759 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14780b44e0 msgr2=0x7f1478147500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:25.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.135+0000 7f14777fe700 1 --2- 172.21.15.162:0/442088759 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14780b44e0 0x7f1478147500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:25.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.135+0000 7f14777fe700 1 -- 172.21.15.162:0/442088759 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14780b3950 msgr2=0x7f1478147010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:25.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.135+0000 7f14777fe700 1 --2- 172.21.15.162:0/442088759 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14780b3950 0x7f1478147010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:25.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.135+0000 7f14777fe700 1 -- 172.21.15.162:0/442088759 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14780ae3a0 con 0x7f14780b18c0 2020-11-28T21:55:25.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.135+0000 7f1477fff700 1 --2- 172.21.15.162:0/442088759 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14780b44e0 0x7f1478147500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:55:25.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.135+0000 7f1476ffd700 1 --2- 172.21.15.162:0/442088759 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14780b3950 0x7f1478147010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:55:25.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.135+0000 7f14777fe700 1 --2- 172.21.15.162:0/442088759 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14780b18c0 0x7f14780b7230 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 rx=0x7f146800c000 tx=0x7f1468006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:25.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.135+0000 7f1474ff9700 1 -- 172.21.15.162:0/442088759 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f146800cda0 con 0x7f14780b18c0 2020-11-28T21:55:25.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.135+0000 7f147c932700 1 -- 172.21.15.162:0/442088759 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14780ae640 con 0x7f14780b18c0 2020-11-28T21:55:25.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.135+0000 7f147c932700 1 -- 172.21.15.162:0/442088759 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14780aef70 con 0x7f14780b18c0 2020-11-28T21:55:25.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.135+0000 7f1474ff9700 1 -- 172.21.15.162:0/442088759 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f146801b410 con 0x7f14780b18c0 2020-11-28T21:55:25.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.135+0000 7f1474ff9700 1 -- 172.21.15.162:0/442088759 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f146801b770 con 0x7f14780b18c0 2020-11-28T21:55:25.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.139+0000 7f147c932700 1 -- 172.21.15.162:0/442088759 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1478057050 con 0x7f14780b18c0 2020-11-28T21:55:25.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.139+0000 7f1474ff9700 1 -- 172.21.15.162:0/442088759 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f146801b8d0 con 0x7f14780b18c0 2020-11-28T21:55:25.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.139+0000 7f1474ff9700 1 --2- 172.21.15.162:0/442088759 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f14580553b0 0x7f1458057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:25.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.139+0000 7f1474ff9700 1 -- 172.21.15.162:0/442088759 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f146806f0d0 con 0x7f14780b18c0 2020-11-28T21:55:25.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.139+0000 7f1476ffd700 1 --2- 172.21.15.162:0/442088759 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f14580553b0 0x7f1458057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:25.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.139+0000 7f1476ffd700 1 --2- 172.21.15.162:0/442088759 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f14580553b0 0x7f1458057870 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7f146c00b000 tx=0x7f146c009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:25.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.147+0000 7f1474ff9700 1 -- 172.21.15.162:0/442088759 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1468045270 con 0x7f14780b18c0 2020-11-28T21:55:25.295 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.291+0000 7f147c932700 1 -- 172.21.15.162:0/442088759 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f147805d250 con 0x7f14780b18c0 2020-11-28T21:55:26.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.995+0000 7f1474ff9700 1 -- 172.21.15.162:0/442088759 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v132) v1 ==== 185+0+0 (secure 0 0 0) 0x7f1468048890 con 0x7f14780b18c0 2020-11-28T21:55:26.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.999+0000 7f147c932700 1 -- 172.21.15.162:0/442088759 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f14580553b0 msgr2=0x7f1458057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:26.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.999+0000 7f147c932700 1 --2- 172.21.15.162:0/442088759 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f14580553b0 0x7f1458057870 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7f146c00b000 tx=0x7f146c009040).stop 2020-11-28T21:55:26.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.999+0000 7f147c932700 1 -- 172.21.15.162:0/442088759 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14780b18c0 msgr2=0x7f14780b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:26.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.999+0000 7f147c932700 1 --2- 172.21.15.162:0/442088759 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14780b18c0 0x7f14780b7230 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 rx=0x7f146800c000 tx=0x7f1468006dc0).stop 2020-11-28T21:55:26.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.999+0000 7f147c932700 1 -- 172.21.15.162:0/442088759 shutdown_connections 2020-11-28T21:55:26.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.999+0000 7f147c932700 1 --2- 172.21.15.162:0/442088759 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f14580553b0 0x7f1458057870 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:26.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.999+0000 7f147c932700 1 --2- 172.21.15.162:0/442088759 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14780b44e0 0x7f1478147500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:26.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.999+0000 7f147c932700 1 --2- 172.21.15.162:0/442088759 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14780b18c0 0x7f14780b7230 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:26.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.999+0000 7f147c932700 1 --2- 172.21.15.162:0/442088759 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14780b3950 0x7f1478147010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:26.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.999+0000 7f147c932700 1 -- 172.21.15.162:0/442088759 >> 172.21.15.162:0/442088759 conn(0x7f1478062630 msgr2=0x7f14780b2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:26.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.999+0000 7f147c932700 1 -- 172.21.15.162:0/442088759 shutdown_connections 2020-11-28T21:55:26.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:25.999+0000 7f147c932700 1 -- 172.21.15.162:0/442088759 wait complete. 2020-11-28T21:55:26.006 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_metadata' removed 2020-11-28T21:55:26.018 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-11-28T21:55:26.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.135+0000 7fd9fd068700 1 Processor -- start 2020-11-28T21:55:26.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.135+0000 7fd9fd068700 1 -- start start 2020-11-28T21:55:26.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.139+0000 7fd9fd068700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd9f80b2d00 0x7fd9f805c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:26.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.139+0000 7fd9fd068700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd9f80b3ab0 0x7fd9f80b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:26.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.139+0000 7fd9fd068700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9f805baf0 con 0x7fd9f8142ad0 2020-11-28T21:55:26.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.139+0000 7fd9fd068700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9f80b0e20 con 0x7fd9f80b2d00 2020-11-28T21:55:26.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.139+0000 7fd9fd068700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f805bd90 con 0x7fd9f80b3ab0 2020-11-28T21:55:26.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.139+0000 7fd9f77fe700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd9f80b3ab0 0x7fd9f80b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:26.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.139+0000 7fd9f77fe700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd9f80b3ab0 0x7fd9f80b30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:41940/0 (socket says 172.21.15.162:41940) 2020-11-28T21:55:26.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.139+0000 7fd9f7fff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd9f80b2d00 0x7fd9f805c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:26.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.139+0000 7fd9f77fe700 1 -- 172.21.15.162:0/3127291109 learned_addr learned my addr 172.21.15.162:0/3127291109 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:26.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.139+0000 7fd9fc867700 1 --1- 172.21.15.162:0/3127291109 >> v1:172.21.15.118:6789/0 conn(0x7fd9f8142ad0 0x7fd9f813fa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.162:41710/0 (socket says 172.21.15.162:41710) 2020-11-28T21:55:26.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.139+0000 7fd9f77fe700 1 -- 172.21.15.162:0/3127291109 >> v2:172.21.15.118:3300/0 conn(0x7fd9f80b2d00 msgr2=0x7fd9f805c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:26.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.139+0000 7fd9f77fe700 1 --2- 172.21.15.162:0/3127291109 >> v2:172.21.15.118:3300/0 conn(0x7fd9f80b2d00 0x7fd9f805c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:26.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.139+0000 7fd9f77fe700 1 -- 172.21.15.162:0/3127291109 >> v1:172.21.15.118:6789/0 conn(0x7fd9f8142ad0 legacy=0x7fd9f813fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:26.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.139+0000 7fd9f77fe700 1 -- 172.21.15.162:0/3127291109 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f805c030 con 0x7fd9f80b3ab0 2020-11-28T21:55:26.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.139+0000 7fd9f77fe700 1 --2- 172.21.15.162:0/3127291109 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd9f80b3ab0 0x7fd9f80b30e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fd9e8002f30 tx=0x7fd9e8006e80).ready entity=mon.2 client_cookie=14731e15d1ac4afd server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:26.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.139+0000 7fd9f6ffd700 1 -- 172.21.15.162:0/3127291109 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9e80097f0 con 0x7fd9f80b3ab0 2020-11-28T21:55:26.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.139+0000 7fd9f6ffd700 1 -- 172.21.15.162:0/3127291109 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9e8009950 con 0x7fd9f80b3ab0 2020-11-28T21:55:26.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.139+0000 7fd9f6ffd700 1 -- 172.21.15.162:0/3127291109 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9e8009cb0 con 0x7fd9f80b3ab0 2020-11-28T21:55:26.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.139+0000 7fd9fd068700 1 -- 172.21.15.162:0/3127291109 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd9f80b3ab0 msgr2=0x7fd9f80b30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:26.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.139+0000 7fd9fd068700 1 --2- 172.21.15.162:0/3127291109 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd9f80b3ab0 0x7fd9f80b30e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fd9e8002f30 tx=0x7fd9e8006e80).stop 2020-11-28T21:55:26.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.139+0000 7fd9fd068700 1 -- 172.21.15.162:0/3127291109 shutdown_connections 2020-11-28T21:55:26.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.139+0000 7fd9fd068700 1 --2- 172.21.15.162:0/3127291109 >> v2:172.21.15.118:3300/0 conn(0x7fd9f80b2d00 0x7fd9f805c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:26.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.139+0000 7fd9fd068700 1 --2- 172.21.15.162:0/3127291109 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd9f80b3ab0 0x7fd9f80b30e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:26.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.139+0000 7fd9fd068700 1 -- 172.21.15.162:0/3127291109 >> 172.21.15.162:0/3127291109 conn(0x7fd9f80604c0 msgr2=0x7fd9f80608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:26.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.139+0000 7fd9fd068700 1 -- 172.21.15.162:0/3127291109 shutdown_connections 2020-11-28T21:55:26.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.139+0000 7fd9fd068700 1 -- 172.21.15.162:0/3127291109 wait complete. 2020-11-28T21:55:26.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.143+0000 7fd9fd068700 1 Processor -- start 2020-11-28T21:55:26.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.143+0000 7fd9fd068700 1 -- start start 2020-11-28T21:55:26.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.143+0000 7fd9fd068700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9f80b2d00 0x7fd9f8146a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:26.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.143+0000 7fd9fd068700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9f80b3ab0 0x7fd9f80b5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:26.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.143+0000 7fd9fd068700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd9f8142ad0 0x7fd9f80b4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:26.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.143+0000 7fd9fd068700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f8146520 con 0x7fd9f80b3ab0 2020-11-28T21:55:26.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.143+0000 7fd9fd068700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f80b44f0 con 0x7fd9f80b2d00 2020-11-28T21:55:26.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.143+0000 7fd9fd068700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f8145b40 con 0x7fd9f8142ad0 2020-11-28T21:55:26.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.143+0000 7fd9f7fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9f80b2d00 0x7fd9f8146a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:26.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.143+0000 7fd9fc867700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd9f8142ad0 0x7fd9f80b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:26.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.143+0000 7fd9f7fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9f80b2d00 0x7fd9f8146a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44720/0 (socket says 172.21.15.162:44720) 2020-11-28T21:55:26.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.143+0000 7fd9fc867700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd9f8142ad0 0x7fd9f80b4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:41948/0 (socket says 172.21.15.162:41948) 2020-11-28T21:55:26.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.143+0000 7fd9f77fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9f80b3ab0 0x7fd9f80b5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:26.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.143+0000 7fd9fc867700 1 -- 172.21.15.162:0/3081925232 learned_addr learned my addr 172.21.15.162:0/3081925232 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:26.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.143+0000 7fd9fc867700 1 -- 172.21.15.162:0/3081925232 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9f80b2d00 msgr2=0x7fd9f8146a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:26.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.143+0000 7fd9fc867700 1 --2- 172.21.15.162:0/3081925232 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9f80b2d00 0x7fd9f8146a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:26.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.143+0000 7fd9fc867700 1 -- 172.21.15.162:0/3081925232 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9f80b3ab0 msgr2=0x7fd9f80b5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:26.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.143+0000 7fd9fc867700 1 --2- 172.21.15.162:0/3081925232 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9f80b3ab0 0x7fd9f80b5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:26.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.143+0000 7fd9fc867700 1 -- 172.21.15.162:0/3081925232 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f8145de0 con 0x7fd9f8142ad0 2020-11-28T21:55:26.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.143+0000 7fd9f77fe700 1 --2- 172.21.15.162:0/3081925232 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9f80b3ab0 0x7fd9f80b5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:55:26.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.143+0000 7fd9fc867700 1 --2- 172.21.15.162:0/3081925232 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd9f8142ad0 0x7fd9f80b4760 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7fd9f8146660 tx=0x7fd9f0007f70).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:26.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.143+0000 7fd9f57fa700 1 -- 172.21.15.162:0/3081925232 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9f0002040 con 0x7fd9f8142ad0 2020-11-28T21:55:26.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.143+0000 7fd9f57fa700 1 -- 172.21.15.162:0/3081925232 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f0005ab0 con 0x7fd9f8142ad0 2020-11-28T21:55:26.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.143+0000 7fd9fd068700 1 -- 172.21.15.162:0/3081925232 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9f8146080 con 0x7fd9f8142ad0 2020-11-28T21:55:26.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.143+0000 7fd9f57fa700 1 -- 172.21.15.162:0/3081925232 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9f0005e10 con 0x7fd9f8142ad0 2020-11-28T21:55:26.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.143+0000 7fd9fd068700 1 -- 172.21.15.162:0/3081925232 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9f80b5810 con 0x7fd9f8142ad0 2020-11-28T21:55:26.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.147+0000 7fd9fd068700 1 -- 172.21.15.162:0/3081925232 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9f8057050 con 0x7fd9f8142ad0 2020-11-28T21:55:26.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.147+0000 7fd9f57fa700 1 -- 172.21.15.162:0/3081925232 <== mon.2 v2:172.21.15.118:3301/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd9f0010030 con 0x7fd9f8142ad0 2020-11-28T21:55:26.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.147+0000 7fd9f57fa700 1 --2- 172.21.15.162:0/3081925232 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd9d40554e0 0x7fd9d40579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:26.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.147+0000 7fd9f7fff700 1 --2- 172.21.15.162:0/3081925232 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd9d40554e0 0x7fd9d40579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:26.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.147+0000 7fd9f57fa700 1 -- 172.21.15.162:0/3081925232 <== mon.2 v2:172.21.15.118:3301/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5721+0+0 (secure 0 0 0) 0x7fd9f006dd10 con 0x7fd9f8142ad0 2020-11-28T21:55:26.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.147+0000 7fd9f7fff700 1 --2- 172.21.15.162:0/3081925232 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd9d40554e0 0x7fd9d40579a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7fd9e400b000 tx=0x7fd9e4006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:26.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.155+0000 7fd9f57fa700 1 -- 172.21.15.162:0/3081925232 <== mon.2 v2:172.21.15.118:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd9f00440d0 con 0x7fd9f8142ad0 2020-11-28T21:55:26.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:26.299+0000 7fd9fd068700 1 -- 172.21.15.162:0/3081925232 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd9f805d250 con 0x7fd9f8142ad0 2020-11-28T21:55:27.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.007+0000 7fd9f57fa700 1 -- 172.21.15.162:0/3081925232 <== mon.2 v2:172.21.15.118:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v133) v1 ==== 173+0+0 (secure 0 0 0) 0x7fd9f00476f0 con 0x7fd9f8142ad0 2020-11-28T21:55:27.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.007+0000 7fd9fd068700 1 -- 172.21.15.162:0/3081925232 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd9d40554e0 msgr2=0x7fd9d40579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:27.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.007+0000 7fd9fd068700 1 --2- 172.21.15.162:0/3081925232 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd9d40554e0 0x7fd9d40579a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7fd9e400b000 tx=0x7fd9e4006dc0).stop 2020-11-28T21:55:27.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.007+0000 7fd9fd068700 1 -- 172.21.15.162:0/3081925232 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd9f8142ad0 msgr2=0x7fd9f80b4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:27.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.007+0000 7fd9fd068700 1 --2- 172.21.15.162:0/3081925232 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd9f8142ad0 0x7fd9f80b4760 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7fd9f8146660 tx=0x7fd9f0007f70).stop 2020-11-28T21:55:27.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.007+0000 7fd9fd068700 1 -- 172.21.15.162:0/3081925232 shutdown_connections 2020-11-28T21:55:27.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.007+0000 7fd9fd068700 1 --2- 172.21.15.162:0/3081925232 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd9d40554e0 0x7fd9d40579a0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:27.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.007+0000 7fd9fd068700 1 --2- 172.21.15.162:0/3081925232 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd9f8142ad0 0x7fd9f80b4760 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:27.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.007+0000 7fd9fd068700 1 --2- 172.21.15.162:0/3081925232 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9f80b2d00 0x7fd9f8146a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:27.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.007+0000 7fd9fd068700 1 --2- 172.21.15.162:0/3081925232 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9f80b3ab0 0x7fd9f80b5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:27.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.007+0000 7fd9fd068700 1 -- 172.21.15.162:0/3081925232 >> 172.21.15.162:0/3081925232 conn(0x7fd9f80604c0 msgr2=0x7fd9f80b9040 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:27.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.011+0000 7fd9fd068700 1 -- 172.21.15.162:0/3081925232 shutdown_connections 2020-11-28T21:55:27.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.011+0000 7fd9fd068700 1 -- 172.21.15.162:0/3081925232 wait complete. 2020-11-28T21:55:27.017 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_data' removed 2020-11-28T21:55:27.030 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-28T21:55:27.031 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-28T21:55:27.031 INFO:tasks.ceph.mds.c:Restarting daemon 2020-11-28T21:55:27.032 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2020-11-28T21:55:27.032 INFO:tasks.ceph.mds.b:Restarting daemon 2020-11-28T21:55:27.033 INFO:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-11-28T21:55:27.035 INFO:tasks.ceph.mds.a:Started 2020-11-28T21:55:27.035 DEBUG:teuthology.parallel:result is None 2020-11-28T21:55:27.036 INFO:tasks.ceph.mds.b:Started 2020-11-28T21:55:27.037 INFO:tasks.ceph.mds.c:Started 2020-11-28T21:55:27.037 DEBUG:teuthology.parallel:result is None 2020-11-28T21:55:27.038 DEBUG:teuthology.parallel:result is None 2020-11-28T21:55:27.038 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist clear 2020-11-28T21:55:27.101 INFO:tasks.ceph.mds.a.smithi162.stdout:starting mds.a at 2020-11-28T21:55:27.111 INFO:tasks.ceph.mds.b.smithi118.stdout:starting mds.b at 2020-11-28T21:55:27.115 INFO:tasks.ceph.mds.c.smithi162.stdout:starting mds.c at 2020-11-28T21:55:27.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.147+0000 7f0ec3cdc700 1 Processor -- start 2020-11-28T21:55:27.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.147+0000 7f0ec3cdc700 1 -- start start 2020-11-28T21:55:27.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.147+0000 7f0ec3cdc700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0ebc0af1f0 0x7f0ebc13f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:27.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.147+0000 7f0ec3cdc700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f0ebc06aef0 0x7f0ebc13fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:27.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.147+0000 7f0ec3cdc700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ebc0af910 con 0x7f0ebc140350 2020-11-28T21:55:27.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.147+0000 7f0ec3cdc700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ebc0b66f0 con 0x7f0ebc06aef0 2020-11-28T21:55:27.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.147+0000 7f0ec3cdc700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ebc1438d0 con 0x7f0ebc0af1f0 2020-11-28T21:55:27.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.147+0000 7f0ec1a78700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0ebc0af1f0 0x7f0ebc13f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:27.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.147+0000 7f0ec2279700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f0ebc140350 0x7f0ebc0acb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51596/0 (socket says 172.21.15.162:51596) 2020-11-28T21:55:27.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.147+0000 7f0ec2279700 1 -- 172.21.15.162:0/1997360466 learned_addr learned my addr 172.21.15.162:0/1997360466 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:27.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.147+0000 7f0ec1a78700 1 --2- 172.21.15.162:0/1997360466 >> v2:172.21.15.162:3300/0 conn(0x7f0ebc0af1f0 0x7f0ebc13f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44756/0 (socket says 172.21.15.162:44756) 2020-11-28T21:55:27.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.147+0000 7f0ec1277700 1 --2- 172.21.15.162:0/1997360466 >> v2:172.21.15.118:3300/0 conn(0x7f0ebc06aef0 0x7f0ebc13fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:27.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.151+0000 7f0ec1a78700 1 -- 172.21.15.162:0/1997360466 >> v2:172.21.15.118:3300/0 conn(0x7f0ebc06aef0 msgr2=0x7f0ebc13fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:27.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.151+0000 7f0ec1a78700 1 --2- 172.21.15.162:0/1997360466 >> v2:172.21.15.118:3300/0 conn(0x7f0ebc06aef0 0x7f0ebc13fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:27.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.151+0000 7f0ec1a78700 1 -- 172.21.15.162:0/1997360466 >> v1:172.21.15.162:6789/0 conn(0x7f0ebc140350 legacy=0x7f0ebc0acb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:27.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.151+0000 7f0ec1a78700 1 -- 172.21.15.162:0/1997360466 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ebc143b70 con 0x7f0ebc0af1f0 2020-11-28T21:55:27.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.151+0000 7f0ec0a76700 1 -- 172.21.15.162:0/1997360466 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4122202940 0 0) 0x7f0ebc0af910 con 0x7f0ebc140350 2020-11-28T21:55:27.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.151+0000 7f0ec1a78700 1 --2- 172.21.15.162:0/1997360466 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ebc0af1f0 0x7f0ebc13f970 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 rx=0x7f0eac002f30 tx=0x7f0eac006dc0).ready entity=mon.1 client_cookie=b515f49b94870e63 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:27.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.151+0000 7f0ec0a76700 1 -- 172.21.15.162:0/1997360466 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0eac00fa10 con 0x7f0ebc0af1f0 2020-11-28T21:55:27.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.151+0000 7f0ec0a76700 1 -- 172.21.15.162:0/1997360466 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eac00fb70 con 0x7f0ebc0af1f0 2020-11-28T21:55:27.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.151+0000 7f0ec3cdc700 1 -- 172.21.15.162:0/1997360466 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ebc0af1f0 msgr2=0x7f0ebc13f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:27.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.151+0000 7f0ec0a76700 1 -- 172.21.15.162:0/1997360466 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0eac010470 con 0x7f0ebc0af1f0 2020-11-28T21:55:27.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.151+0000 7f0ec3cdc700 1 --2- 172.21.15.162:0/1997360466 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ebc0af1f0 0x7f0ebc13f970 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 rx=0x7f0eac002f30 tx=0x7f0eac006dc0).stop 2020-11-28T21:55:27.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.151+0000 7f0ec3cdc700 1 -- 172.21.15.162:0/1997360466 shutdown_connections 2020-11-28T21:55:27.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.151+0000 7f0ec3cdc700 1 --2- 172.21.15.162:0/1997360466 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ebc0af1f0 0x7f0ebc13f970 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:27.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.151+0000 7f0ec3cdc700 1 --2- 172.21.15.162:0/1997360466 >> v2:172.21.15.118:3300/0 conn(0x7f0ebc06aef0 0x7f0ebc13fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:27.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.151+0000 7f0ec3cdc700 1 -- 172.21.15.162:0/1997360466 >> 172.21.15.162:0/1997360466 conn(0x7f0ebc0b56f0 msgr2=0x7f0ebc0b5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:27.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.151+0000 7f0ec3cdc700 1 -- 172.21.15.162:0/1997360466 shutdown_connections 2020-11-28T21:55:27.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.151+0000 7f0ec3cdc700 1 -- 172.21.15.162:0/1997360466 wait complete. 2020-11-28T21:55:27.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.151+0000 7f0ec3cdc700 1 Processor -- start 2020-11-28T21:55:27.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.151+0000 7f0ec3cdc700 1 -- start start 2020-11-28T21:55:27.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.151+0000 7f0ec3cdc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ebc06aef0 0x7f0ebc0b4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:27.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.151+0000 7f0ec3cdc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0ebc0af1f0 0x7f0ebc0b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:27.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.151+0000 7f0ec3cdc700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0ebc140350 0x7f0ebc1414a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:27.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.151+0000 7f0ec3cdc700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ebc0b50c0 con 0x7f0ebc0af1f0 2020-11-28T21:55:27.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.151+0000 7f0ec3cdc700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ebc0b5360 con 0x7f0ebc06aef0 2020-11-28T21:55:27.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.151+0000 7f0ec3cdc700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ebc140d60 con 0x7f0ebc140350 2020-11-28T21:55:27.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.151+0000 7f0ec1a78700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ebc06aef0 0x7f0ebc0b4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:27.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.151+0000 7f0ec1a78700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ebc06aef0 0x7f0ebc0b4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44758/0 (socket says 172.21.15.162:44758) 2020-11-28T21:55:27.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.151+0000 7f0ec1a78700 1 -- 172.21.15.162:0/252268945 learned_addr learned my addr 172.21.15.162:0/252268945 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:27.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.151+0000 7f0ec2279700 1 --2- 172.21.15.162:0/252268945 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0ebc140350 0x7f0ebc1414a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:27.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.151+0000 7f0ec1277700 1 --2- 172.21.15.162:0/252268945 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0ebc0af1f0 0x7f0ebc0b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:27.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.155+0000 7f0ec1a78700 1 -- 172.21.15.162:0/252268945 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0ebc140350 msgr2=0x7f0ebc1414a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:27.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.155+0000 7f0ec1a78700 1 --2- 172.21.15.162:0/252268945 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0ebc140350 0x7f0ebc1414a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:27.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.155+0000 7f0ec1a78700 1 -- 172.21.15.162:0/252268945 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0ebc0af1f0 msgr2=0x7f0ebc0b0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:27.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.155+0000 7f0ec1a78700 1 --2- 172.21.15.162:0/252268945 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0ebc0af1f0 0x7f0ebc0b0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:27.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.155+0000 7f0ec1a78700 1 -- 172.21.15.162:0/252268945 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ebc141000 con 0x7f0ebc06aef0 2020-11-28T21:55:27.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.155+0000 7f0ec1a78700 1 --2- 172.21.15.162:0/252268945 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ebc06aef0 0x7f0ebc0b4720 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 rx=0x7f0eac002f30 tx=0x7f0eac017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:27.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.155+0000 7f0eaaffd700 1 -- 172.21.15.162:0/252268945 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0eac01a640 con 0x7f0ebc06aef0 2020-11-28T21:55:27.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.155+0000 7f0eaaffd700 1 -- 172.21.15.162:0/252268945 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eac01e070 con 0x7f0ebc06aef0 2020-11-28T21:55:27.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.155+0000 7f0ec3cdc700 1 -- 172.21.15.162:0/252268945 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ebc0b1270 con 0x7f0ebc06aef0 2020-11-28T21:55:27.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.155+0000 7f0eaaffd700 1 -- 172.21.15.162:0/252268945 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0eac01a990 con 0x7f0ebc06aef0 2020-11-28T21:55:27.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.155+0000 7f0ec3cdc700 1 -- 172.21.15.162:0/252268945 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ebc0aa3e0 con 0x7f0ebc06aef0 2020-11-28T21:55:27.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.155+0000 7f0eaaffd700 1 -- 172.21.15.162:0/252268945 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0eac02a070 con 0x7f0ebc06aef0 2020-11-28T21:55:27.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.155+0000 7f0ec3cdc700 1 -- 172.21.15.162:0/252268945 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ebc141190 con 0x7f0ebc06aef0 2020-11-28T21:55:27.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.155+0000 7f0eaaffd700 1 --2- 172.21.15.162:0/252268945 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0e9c055330 0x7f0e9c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:27.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.155+0000 7f0eaaffd700 1 -- 172.21.15.162:0/252268945 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f0eac06f4f0 con 0x7f0ebc06aef0 2020-11-28T21:55:27.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.155+0000 7f0ec1277700 1 --2- 172.21.15.162:0/252268945 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0e9c055330 0x7f0e9c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:27.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.159+0000 7f0ec1277700 1 --2- 172.21.15.162:0/252268945 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0e9c055330 0x7f0e9c0577f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7f0eb800b000 tx=0x7f0eb8006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:27.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.159+0000 7f0eaaffd700 1 -- 172.21.15.162:0/252268945 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0eac045ad0 con 0x7f0ebc06aef0 2020-11-28T21:55:27.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:27.315+0000 7f0ec3cdc700 1 -- 172.21.15.162:0/252268945 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f0ebc057050 con 0x7f0ebc06aef0 2020-11-28T21:55:28.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.131+0000 7f0eaaffd700 1 -- 172.21.15.162:0/252268945 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v134) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0eac06de60 con 0x7f0ebc06aef0 2020-11-28T21:55:28.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.135+0000 7f0ec3cdc700 1 -- 172.21.15.162:0/252268945 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0e9c055330 msgr2=0x7f0e9c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:28.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.135+0000 7f0ec3cdc700 1 --2- 172.21.15.162:0/252268945 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0e9c055330 0x7f0e9c0577f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7f0eb800b000 tx=0x7f0eb8006dc0).stop 2020-11-28T21:55:28.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.135+0000 7f0ec3cdc700 1 -- 172.21.15.162:0/252268945 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ebc06aef0 msgr2=0x7f0ebc0b4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:28.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.135+0000 7f0ec3cdc700 1 --2- 172.21.15.162:0/252268945 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ebc06aef0 0x7f0ebc0b4720 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 rx=0x7f0eac002f30 tx=0x7f0eac017000).stop 2020-11-28T21:55:28.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.135+0000 7f0ec3cdc700 1 -- 172.21.15.162:0/252268945 shutdown_connections 2020-11-28T21:55:28.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.135+0000 7f0ec3cdc700 1 --2- 172.21.15.162:0/252268945 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0e9c055330 0x7f0e9c0577f0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:28.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.135+0000 7f0ec3cdc700 1 --2- 172.21.15.162:0/252268945 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0ebc140350 0x7f0ebc1414a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:28.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.135+0000 7f0ec3cdc700 1 --2- 172.21.15.162:0/252268945 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ebc06aef0 0x7f0ebc0b4720 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:28.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.135+0000 7f0ec3cdc700 1 --2- 172.21.15.162:0/252268945 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0ebc0af1f0 0x7f0ebc0b0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:28.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.135+0000 7f0ec3cdc700 1 -- 172.21.15.162:0/252268945 >> 172.21.15.162:0/252268945 conn(0x7f0ebc0b56f0 msgr2=0x7f0ebc13e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:28.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.135+0000 7f0ec3cdc700 1 -- 172.21.15.162:0/252268945 shutdown_connections 2020-11-28T21:55:28.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.135+0000 7f0ec3cdc700 1 -- 172.21.15.162:0/252268945 wait complete. 2020-11-28T21:55:28.146 INFO:teuthology.orchestra.run.smithi162.stderr: removed all blacklist entries 2020-11-28T21:55:28.155 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2020-11-28T21:55:28.156 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth ls --format=json-pretty 2020-11-28T21:55:28.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.263+0000 7f30e3d1f700 1 Processor -- start 2020-11-28T21:55:28.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.263+0000 7f30e3d1f700 1 -- start start 2020-11-28T21:55:28.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.267+0000 7f30e3d1f700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f30dc0608f0 0x7f30dc13c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:28.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.267+0000 7f30e3d1f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f30dc143730 0x7f30dc13ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:28.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.267+0000 7f30e3d1f700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30dc0ba0b0 con 0x7f30dc0b6130 2020-11-28T21:55:28.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.267+0000 7f30e3d1f700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30dc0b9e10 con 0x7f30dc0608f0 2020-11-28T21:55:28.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.267+0000 7f30e3d1f700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30dc142b80 con 0x7f30dc143730 2020-11-28T21:55:28.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.267+0000 7f30e1abb700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f30dc0608f0 0x7f30dc13c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:28.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.267+0000 7f30e12ba700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f30dc0b6130 0x7f30dc143020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51612/0 (socket says 172.21.15.162:51612) 2020-11-28T21:55:28.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.267+0000 7f30e12ba700 1 -- 172.21.15.162:0/3681459345 learned_addr learned my addr 172.21.15.162:0/3681459345 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:28.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.267+0000 7f30e22bc700 1 --2- 172.21.15.162:0/3681459345 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f30dc143730 0x7f30dc13ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:28.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.267+0000 7f30e1abb700 1 -- 172.21.15.162:0/3681459345 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f30dc143730 msgr2=0x7f30dc13ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:28.273 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.267+0000 7f30e1abb700 1 --2- 172.21.15.162:0/3681459345 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f30dc143730 0x7f30dc13ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:28.273 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.267+0000 7f30e1abb700 1 -- 172.21.15.162:0/3681459345 >> v1:172.21.15.162:6789/0 conn(0x7f30dc0b6130 legacy=0x7f30dc143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:28.273 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.267+0000 7f30e1abb700 1 -- 172.21.15.162:0/3681459345 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30dc0b9b30 con 0x7f30dc0608f0 2020-11-28T21:55:28.273 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.267+0000 7f30e22bc700 1 --2- 172.21.15.162:0/3681459345 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f30dc143730 0x7f30dc13ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:55:28.274 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.267+0000 7f30e1abb700 1 --2- 172.21.15.162:0/3681459345 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30dc0608f0 0x7f30dc13c910 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 rx=0x7f30cc002f30 tx=0x7f30cc006dc0).ready entity=mon.1 client_cookie=6ecb11a970be9163 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:28.275 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.267+0000 7f30e0ab9700 1 -- 172.21.15.162:0/3681459345 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30cc0097f0 con 0x7f30dc0608f0 2020-11-28T21:55:28.275 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.267+0000 7f30e0ab9700 1 -- 172.21.15.162:0/3681459345 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30cc009950 con 0x7f30dc0608f0 2020-11-28T21:55:28.275 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.267+0000 7f30e0ab9700 1 -- 172.21.15.162:0/3681459345 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30cc009cb0 con 0x7f30dc0608f0 2020-11-28T21:55:28.276 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.267+0000 7f30e3d1f700 1 -- 172.21.15.162:0/3681459345 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30dc0608f0 msgr2=0x7f30dc13c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:28.276 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.267+0000 7f30e3d1f700 1 --2- 172.21.15.162:0/3681459345 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30dc0608f0 0x7f30dc13c910 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 rx=0x7f30cc002f30 tx=0x7f30cc006dc0).stop 2020-11-28T21:55:28.276 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.267+0000 7f30e3d1f700 1 -- 172.21.15.162:0/3681459345 shutdown_connections 2020-11-28T21:55:28.276 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.267+0000 7f30e3d1f700 1 --2- 172.21.15.162:0/3681459345 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30dc0608f0 0x7f30dc13c910 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:28.277 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.267+0000 7f30e3d1f700 1 --2- 172.21.15.162:0/3681459345 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f30dc143730 0x7f30dc13ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:28.277 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.267+0000 7f30e3d1f700 1 -- 172.21.15.162:0/3681459345 >> 172.21.15.162:0/3681459345 conn(0x7f30dc066c70 msgr2=0x7f30dc067050 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:28.277 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.267+0000 7f30e3d1f700 1 -- 172.21.15.162:0/3681459345 shutdown_connections 2020-11-28T21:55:28.278 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.267+0000 7f30e3d1f700 1 -- 172.21.15.162:0/3681459345 wait complete. 2020-11-28T21:55:28.278 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.271+0000 7f30e3d1f700 1 Processor -- start 2020-11-28T21:55:28.278 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.271+0000 7f30e3d1f700 1 -- start start 2020-11-28T21:55:28.280 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.271+0000 7f30e3d1f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30dc0608f0 0x7f30dc142280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:28.280 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.271+0000 7f30e3d1f700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f30dc0b6130 0x7f30dc151810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:28.281 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.271+0000 7f30e3d1f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f30dc143730 0x7f30dc151d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:28.281 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.271+0000 7f30e3d1f700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30dc154f60 con 0x7f30dc0b6130 2020-11-28T21:55:28.281 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.271+0000 7f30e3d1f700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30dc155200 con 0x7f30dc0608f0 2020-11-28T21:55:28.281 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.271+0000 7f30e3d1f700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30dc1554a0 con 0x7f30dc143730 2020-11-28T21:55:28.281 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.275+0000 7f30e1abb700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30dc0608f0 0x7f30dc142280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:28.282 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.275+0000 7f30e1abb700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30dc0608f0 0x7f30dc142280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44772/0 (socket says 172.21.15.162:44772) 2020-11-28T21:55:28.282 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.275+0000 7f30e1abb700 1 -- 172.21.15.162:0/461671293 learned_addr learned my addr 172.21.15.162:0/461671293 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:28.282 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.275+0000 7f30e22bc700 1 --2- 172.21.15.162:0/461671293 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f30dc143730 0x7f30dc151d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:28.282 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.275+0000 7f30e12ba700 1 --2- 172.21.15.162:0/461671293 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f30dc0b6130 0x7f30dc151810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:28.283 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.275+0000 7f30e1abb700 1 -- 172.21.15.162:0/461671293 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f30dc143730 msgr2=0x7f30dc151d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:28.283 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.275+0000 7f30e1abb700 1 --2- 172.21.15.162:0/461671293 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f30dc143730 0x7f30dc151d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:28.283 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.275+0000 7f30e1abb700 1 -- 172.21.15.162:0/461671293 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f30dc0b6130 msgr2=0x7f30dc151810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:28.283 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.275+0000 7f30e1abb700 1 --2- 172.21.15.162:0/461671293 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f30dc0b6130 0x7f30dc151810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:28.283 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.275+0000 7f30e1abb700 1 -- 172.21.15.162:0/461671293 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30dc155740 con 0x7f30dc0608f0 2020-11-28T21:55:28.284 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.275+0000 7f30e12ba700 1 --2- 172.21.15.162:0/461671293 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f30dc0b6130 0x7f30dc151810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:55:28.284 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.275+0000 7f30e1abb700 1 --2- 172.21.15.162:0/461671293 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30dc0608f0 0x7f30dc142280 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 rx=0x7f30cc002f30 tx=0x7f30cc015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:28.284 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.275+0000 7f30daffd700 1 -- 172.21.15.162:0/461671293 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30cc011db0 con 0x7f30dc0608f0 2020-11-28T21:55:28.284 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.275+0000 7f30daffd700 1 -- 172.21.15.162:0/461671293 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30cc01d450 con 0x7f30dc0608f0 2020-11-28T21:55:28.284 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.275+0000 7f30e3d1f700 1 -- 172.21.15.162:0/461671293 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30dc1521f0 con 0x7f30dc0608f0 2020-11-28T21:55:28.285 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.275+0000 7f30daffd700 1 -- 172.21.15.162:0/461671293 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30cc01d7b0 con 0x7f30dc0608f0 2020-11-28T21:55:28.285 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.275+0000 7f30e3d1f700 1 -- 172.21.15.162:0/461671293 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30dc0b2050 con 0x7f30dc0608f0 2020-11-28T21:55:28.285 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.275+0000 7f30e3d1f700 1 -- 172.21.15.162:0/461671293 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30dc057050 con 0x7f30dc0608f0 2020-11-28T21:55:28.285 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.275+0000 7f30daffd700 1 -- 172.21.15.162:0/461671293 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f30cc024030 con 0x7f30dc0608f0 2020-11-28T21:55:28.286 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.275+0000 7f30daffd700 1 --2- 172.21.15.162:0/461671293 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f30bc055440 0x7f30bc057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:28.287 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.275+0000 7f30daffd700 1 -- 172.21.15.162:0/461671293 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f30cc06f150 con 0x7f30dc0608f0 2020-11-28T21:55:28.287 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.275+0000 7f30e12ba700 1 --2- 172.21.15.162:0/461671293 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f30bc055440 0x7f30bc057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:28.287 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.279+0000 7f30e12ba700 1 --2- 172.21.15.162:0/461671293 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f30bc055440 0x7f30bc057900 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7f30d0008000 tx=0x7f30d0009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:28.287 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.279+0000 7f30daffd700 1 -- 172.21.15.162:0/461671293 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f30cc045670 con 0x7f30dc0608f0 2020-11-28T21:55:28.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.467+0000 7f30e3d1f700 1 -- 172.21.15.162:0/461671293 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f30dc05d530 con 0x7f30dc0608f0 2020-11-28T21:55:28.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.471+0000 7f30daffd700 1 -- 172.21.15.162:0/461671293 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v28) v1 ==== 80+0+4255 (secure 0 0 0) 0x7f30cc022020 con 0x7f30dc0608f0 2020-11-28T21:55:28.474 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:28.474 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2020-11-28T21:55:28.474 INFO:teuthology.orchestra.run.smithi162.stdout: "auth_dump": [ 2020-11-28T21:55:28.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:28.475 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mds.a", 2020-11-28T21:55:28.475 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfWxUpKxAAIUDwQRKwz8H4Qk9qE356Bg==", 2020-11-28T21:55:28.475 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:55:28.475 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:55:28.476 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:55:28.476 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:55:28.476 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:55:28.476 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:28.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:28.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:28.477 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mds.b", 2020-11-28T21:55:28.477 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfKY2zORAAaO65etMZiYcaTDNX0nsEPw==", 2020-11-28T21:55:28.477 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:55:28.478 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:55:28.478 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:55:28.478 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:55:28.478 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:55:28.479 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:28.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:28.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:28.479 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mds.c", 2020-11-28T21:55:28.479 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfdQbRMxAAWlcjesxTZzZnvKCQneSjjQ==", 2020-11-28T21:55:28.480 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:55:28.480 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:55:28.480 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:55:28.480 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:55:28.480 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:55:28.481 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:28.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:28.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:28.481 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.0", 2020-11-28T21:55:28.481 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQADxcJfF3WLLxAAxsB5YPjH1yjv/vhlRbijqw==", 2020-11-28T21:55:28.481 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:55:28.482 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:55:28.482 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:55:28.482 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:55:28.482 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:28.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:28.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:28.483 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.1", 2020-11-28T21:55:28.483 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAGxcJfKYbBFhAA1x4DzFtkQQv6uWLIdOi1qQ==", 2020-11-28T21:55:28.483 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:55:28.483 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:55:28.484 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:55:28.484 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:55:28.484 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:28.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:28.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:28.485 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.2", 2020-11-28T21:55:28.485 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAIxcJfvcYUOhAASENjM0HfAxxyFe13mmE36Q==", 2020-11-28T21:55:28.485 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:55:28.485 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:55:28.486 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:55:28.486 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:55:28.486 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:28.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:28.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:28.487 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.3", 2020-11-28T21:55:28.487 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQALxcJfc/OHIRAAFYx3s8Ssttt15hjoH04tow==", 2020-11-28T21:55:28.487 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:55:28.487 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:55:28.488 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:55:28.488 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:55:28.488 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:28.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:28.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:28.489 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.4", 2020-11-28T21:55:28.489 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAPxcJffGWqCRAAb8CCN5oUPkdDh6BJH5HEnw==", 2020-11-28T21:55:28.489 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:55:28.489 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:55:28.489 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:55:28.490 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:55:28.490 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:28.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:28.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:28.491 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.5", 2020-11-28T21:55:28.491 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQARxcJfo8GKKRAAfbsCDb4L2wULiKmHyzu7/A==", 2020-11-28T21:55:28.491 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:55:28.491 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:55:28.492 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:55:28.492 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:55:28.492 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:28.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:28.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:28.493 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.6", 2020-11-28T21:55:28.493 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAUxcJf8Y2MEBAAqQX6rX3LK0zodlI2wK4JrA==", 2020-11-28T21:55:28.493 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:55:28.493 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:55:28.493 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:55:28.494 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:55:28.494 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:28.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:28.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:28.495 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.7", 2020-11-28T21:55:28.495 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAWxcJfCLBdMhAAf8QixQMQDrfZOGWN8rHe5g==", 2020-11-28T21:55:28.495 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:55:28.495 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:55:28.495 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:55:28.496 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:55:28.496 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:28.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:28.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:28.496 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "client.0", 2020-11-28T21:55:28.497 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQABxcJfS9vDAxAAUk4FU8FVt4if/CRH5ZisOA==", 2020-11-28T21:55:28.497 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:55:28.497 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:55:28.497 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow r", 2020-11-28T21:55:28.497 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow rw pool=cephfs_data" 2020-11-28T21:55:28.498 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:28.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:28.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:28.498 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "client.1", 2020-11-28T21:55:28.499 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQABxcJfuCU8BhAAXUSoSK3D0XRo58uHNzcjXg==", 2020-11-28T21:55:28.499 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:55:28.499 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:55:28.499 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow r", 2020-11-28T21:55:28.499 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow rw pool=cephfs_data" 2020-11-28T21:55:28.500 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:28.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:28.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:28.500 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "client.admin", 2020-11-28T21:55:28.501 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfdlTWBxAApzAxlVjzdnETQF//Vnsg5w==", 2020-11-28T21:55:28.501 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:55:28.501 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow *", 2020-11-28T21:55:28.501 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:55:28.501 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:55:28.502 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:55:28.502 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:28.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:28.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:28.502 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mgr.x", 2020-11-28T21:55:28.503 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfZhdvKBAAT7zCCsD2OVCHjvddGLxQ/A==", 2020-11-28T21:55:28.503 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:55:28.503 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow *", 2020-11-28T21:55:28.503 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow profile mgr", 2020-11-28T21:55:28.504 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:55:28.504 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:28.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:28.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:28.504 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mgr.y", 2020-11-28T21:55:28.505 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfs8fvJRAAP0Atp074EeMHiTtpCCVHuA==", 2020-11-28T21:55:28.505 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:55:28.505 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow *", 2020-11-28T21:55:28.505 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow profile mgr", 2020-11-28T21:55:28.505 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:55:28.506 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:28.506 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:28.506 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:28.506 INFO:teuthology.orchestra.run.smithi162.stdout:} 2020-11-28T21:55:28.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.471+0000 7f30e3d1f700 1 -- 172.21.15.162:0/461671293 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f30bc055440 msgr2=0x7f30bc057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:28.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.471+0000 7f30e3d1f700 1 --2- 172.21.15.162:0/461671293 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f30bc055440 0x7f30bc057900 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7f30d0008000 tx=0x7f30d0009040).stop 2020-11-28T21:55:28.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.471+0000 7f30e3d1f700 1 -- 172.21.15.162:0/461671293 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30dc0608f0 msgr2=0x7f30dc142280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:28.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.471+0000 7f30e3d1f700 1 --2- 172.21.15.162:0/461671293 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30dc0608f0 0x7f30dc142280 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 rx=0x7f30cc002f30 tx=0x7f30cc015000).stop 2020-11-28T21:55:28.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.471+0000 7f30e3d1f700 1 -- 172.21.15.162:0/461671293 shutdown_connections 2020-11-28T21:55:28.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.471+0000 7f30e3d1f700 1 --2- 172.21.15.162:0/461671293 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f30bc055440 0x7f30bc057900 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:28.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.471+0000 7f30e3d1f700 1 --2- 172.21.15.162:0/461671293 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f30dc143730 0x7f30dc151d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:28.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.471+0000 7f30e3d1f700 1 --2- 172.21.15.162:0/461671293 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30dc0608f0 0x7f30dc142280 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:28.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.471+0000 7f30e3d1f700 1 --2- 172.21.15.162:0/461671293 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f30dc0b6130 0x7f30dc151810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:28.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.471+0000 7f30e3d1f700 1 -- 172.21.15.162:0/461671293 >> 172.21.15.162:0/461671293 conn(0x7f30dc066c70 msgr2=0x7f30dc0b4030 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:28.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.471+0000 7f30e3d1f700 1 -- 172.21.15.162:0/461671293 shutdown_connections 2020-11-28T21:55:28.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.471+0000 7f30e3d1f700 1 -- 172.21.15.162:0/461671293 wait complete. 2020-11-28T21:55:28.511 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:55:28.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.619+0000 7ff9ebfff700 1 Processor -- start 2020-11-28T21:55:28.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.619+0000 7ff9ebfff700 1 -- start start 2020-11-28T21:55:28.624 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.619+0000 7ff9ebfff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff9ec0994f0 0x7ff9ec0b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:28.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.619+0000 7ff9ebfff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff9ec060710 0x7ff9ec0b1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:28.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.619+0000 7ff9ebfff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9ec098f10 con 0x7ff9ec098b30 2020-11-28T21:55:28.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.619+0000 7ff9ebfff700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9ec0b1030 con 0x7ff9ec0994f0 2020-11-28T21:55:28.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.619+0000 7ff9ebfff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ec0b0680 con 0x7ff9ec060710 2020-11-28T21:55:28.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.619+0000 7ff9ea7fc700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff9ec0994f0 0x7ff9ec0b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:28.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.619+0000 7ff9ea7fc700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff9ec0994f0 0x7ff9ec0b0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44784/0 (socket says 172.21.15.162:44784) 2020-11-28T21:55:28.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.619+0000 7ff9ea7fc700 1 -- 172.21.15.162:0/428307779 learned_addr learned my addr 172.21.15.162:0/428307779 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:28.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.619+0000 7ff9eb7fe700 1 --2- 172.21.15.162:0/428307779 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff9ec060710 0x7ff9ec0b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:28.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.619+0000 7ff9ea7fc700 1 -- 172.21.15.162:0/428307779 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff9ec060710 msgr2=0x7ff9ec0b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:28.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.619+0000 7ff9ea7fc700 1 --2- 172.21.15.162:0/428307779 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff9ec060710 0x7ff9ec0b1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:28.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.619+0000 7ff9ea7fc700 1 -- 172.21.15.162:0/428307779 >> v1:172.21.15.118:6789/0 conn(0x7ff9ec098b30 legacy=0x7ff9ec060000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:28.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.619+0000 7ff9ea7fc700 1 -- 172.21.15.162:0/428307779 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ec0b0920 con 0x7ff9ec0994f0 2020-11-28T21:55:28.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.619+0000 7ff9ea7fc700 1 --2- 172.21.15.162:0/428307779 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9ec0994f0 0x7ff9ec0b0000 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 rx=0x7ff9d4002f30 tx=0x7ff9d4006e80).ready entity=mon.1 client_cookie=eea6730028fa04ee server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:28.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.623+0000 7ff9e9ffb700 1 -- 172.21.15.162:0/428307779 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9d40097f0 con 0x7ff9ec0994f0 2020-11-28T21:55:28.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.623+0000 7ff9e9ffb700 1 -- 172.21.15.162:0/428307779 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9d4009950 con 0x7ff9ec0994f0 2020-11-28T21:55:28.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.623+0000 7ff9e9ffb700 1 -- 172.21.15.162:0/428307779 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9d4009ac0 con 0x7ff9ec0994f0 2020-11-28T21:55:28.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.623+0000 7ff9ebfff700 1 -- 172.21.15.162:0/428307779 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9ec0994f0 msgr2=0x7ff9ec0b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:28.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.623+0000 7ff9ebfff700 1 --2- 172.21.15.162:0/428307779 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9ec0994f0 0x7ff9ec0b0000 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 rx=0x7ff9d4002f30 tx=0x7ff9d4006e80).stop 2020-11-28T21:55:28.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.623+0000 7ff9ebfff700 1 -- 172.21.15.162:0/428307779 shutdown_connections 2020-11-28T21:55:28.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.623+0000 7ff9ebfff700 1 --2- 172.21.15.162:0/428307779 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff9ec060710 0x7ff9ec0b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:28.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.623+0000 7ff9ebfff700 1 --2- 172.21.15.162:0/428307779 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9ec0994f0 0x7ff9ec0b0000 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:28.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.623+0000 7ff9ebfff700 1 -- 172.21.15.162:0/428307779 >> 172.21.15.162:0/428307779 conn(0x7ff9ec05b970 msgr2=0x7ff9ec05bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:28.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.623+0000 7ff9ebfff700 1 -- 172.21.15.162:0/428307779 shutdown_connections 2020-11-28T21:55:28.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.623+0000 7ff9ebfff700 1 -- 172.21.15.162:0/428307779 wait complete. 2020-11-28T21:55:28.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.623+0000 7ff9ebfff700 1 Processor -- start 2020-11-28T21:55:28.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.623+0000 7ff9ebfff700 1 -- start start 2020-11-28T21:55:28.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.623+0000 7ff9ebfff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff9ec060710 0x7ff9ec0aaff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:28.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.623+0000 7ff9ebfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9ec098b30 0x7ff9ec155ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:28.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.623+0000 7ff9ebfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9ec0994f0 0x7ff9ec0ac750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:28.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.623+0000 7ff9ebfff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ec0638c0 con 0x7ff9ec0994f0 2020-11-28T21:55:28.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.623+0000 7ff9ebfff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ec063b60 con 0x7ff9ec098b30 2020-11-28T21:55:28.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.623+0000 7ff9ebfff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ec0ac010 con 0x7ff9ec060710 2020-11-28T21:55:28.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.623+0000 7ff9ea7fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9ec098b30 0x7ff9ec155ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:28.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.623+0000 7ff9eaffd700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff9ec060710 0x7ff9ec0aaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:28.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.623+0000 7ff9ea7fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9ec098b30 0x7ff9ec155ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44790/0 (socket says 172.21.15.162:44790) 2020-11-28T21:55:28.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.623+0000 7ff9eaffd700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff9ec060710 0x7ff9ec0aaff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:42010/0 (socket says 172.21.15.162:42010) 2020-11-28T21:55:28.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.623+0000 7ff9ea7fc700 1 -- 172.21.15.162:0/3804180486 learned_addr learned my addr 172.21.15.162:0/3804180486 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:28.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.623+0000 7ff9eb7fe700 1 --2- 172.21.15.162:0/3804180486 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9ec0994f0 0x7ff9ec0ac750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:28.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.627+0000 7ff9ea7fc700 1 -- 172.21.15.162:0/3804180486 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff9ec060710 msgr2=0x7ff9ec0aaff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:28.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.627+0000 7ff9ea7fc700 1 --2- 172.21.15.162:0/3804180486 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff9ec060710 0x7ff9ec0aaff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:28.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.627+0000 7ff9ea7fc700 1 -- 172.21.15.162:0/3804180486 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9ec0994f0 msgr2=0x7ff9ec0ac750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:28.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.627+0000 7ff9ea7fc700 1 --2- 172.21.15.162:0/3804180486 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9ec0994f0 0x7ff9ec0ac750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:28.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.627+0000 7ff9ea7fc700 1 -- 172.21.15.162:0/3804180486 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ec0ac2b0 con 0x7ff9ec098b30 2020-11-28T21:55:28.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.627+0000 7ff9ea7fc700 1 --2- 172.21.15.162:0/3804180486 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9ec098b30 0x7ff9ec155ce0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 rx=0x7ff9d4002f30 tx=0x7ff9d4016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:28.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.627+0000 7ff9e3fff700 1 -- 172.21.15.162:0/3804180486 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9d4011de0 con 0x7ff9ec098b30 2020-11-28T21:55:28.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.627+0000 7ff9e3fff700 1 -- 172.21.15.162:0/3804180486 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9d4020400 con 0x7ff9ec098b30 2020-11-28T21:55:28.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.627+0000 7ff9ebfff700 1 -- 172.21.15.162:0/3804180486 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9ec065aa0 con 0x7ff9ec098b30 2020-11-28T21:55:28.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.627+0000 7ff9e3fff700 1 -- 172.21.15.162:0/3804180486 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9d4020760 con 0x7ff9ec098b30 2020-11-28T21:55:28.639 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.627+0000 7ff9ebfff700 1 -- 172.21.15.162:0/3804180486 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9ec0ad340 con 0x7ff9ec098b30 2020-11-28T21:55:28.639 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.627+0000 7ff9ebfff700 1 -- 172.21.15.162:0/3804180486 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9ec0ac440 con 0x7ff9ec098b30 2020-11-28T21:55:28.639 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.627+0000 7ff9e3fff700 1 -- 172.21.15.162:0/3804180486 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff9d4027030 con 0x7ff9ec098b30 2020-11-28T21:55:28.639 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.627+0000 7ff9e3fff700 1 --2- 172.21.15.162:0/3804180486 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff9d0055330 0x7ff9d00577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:28.639 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.627+0000 7ff9e3fff700 1 -- 172.21.15.162:0/3804180486 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5253+0+0 (secure 0 0 0) 0x7ff9d406e7b0 con 0x7ff9ec098b30 2020-11-28T21:55:28.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.627+0000 7ff9eaffd700 1 --2- 172.21.15.162:0/3804180486 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff9d0055330 0x7ff9d00577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:28.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.631+0000 7ff9eaffd700 1 --2- 172.21.15.162:0/3804180486 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff9d0055330 0x7ff9d00577f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7ff9dc00d000 tx=0x7ff9dc000f00).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:28.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.635+0000 7ff9e3fff700 1 -- 172.21.15.162:0/3804180486 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff9d4044de0 con 0x7ff9ec098b30 2020-11-28T21:55:28.782 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.779+0000 7ff9ebfff700 1 -- 172.21.15.162:0/3804180486 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9ec057050 con 0x7ff9ec098b30 2020-11-28T21:55:28.783 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.779+0000 7ff9e3fff700 1 -- 172.21.15.162:0/3804180486 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+13287 (secure 0 0 0) 0x7ff9d4025090 con 0x7ff9ec098b30 2020-11-28T21:55:28.784 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:28.784 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":134,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:55:28.119737+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:55:28.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.783+0000 7ff9ebfff700 1 -- 172.21.15.162:0/3804180486 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff9d0055330 msgr2=0x7ff9d00577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:28.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.783+0000 7ff9ebfff700 1 --2- 172.21.15.162:0/3804180486 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff9d0055330 0x7ff9d00577f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7ff9dc00d000 tx=0x7ff9dc000f00).stop 2020-11-28T21:55:28.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.783+0000 7ff9ebfff700 1 -- 172.21.15.162:0/3804180486 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9ec098b30 msgr2=0x7ff9ec155ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:28.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.783+0000 7ff9ebfff700 1 --2- 172.21.15.162:0/3804180486 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9ec098b30 0x7ff9ec155ce0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 rx=0x7ff9d4002f30 tx=0x7ff9d4016000).stop 2020-11-28T21:55:28.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.783+0000 7ff9ebfff700 1 -- 172.21.15.162:0/3804180486 shutdown_connections 2020-11-28T21:55:28.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.783+0000 7ff9ebfff700 1 --2- 172.21.15.162:0/3804180486 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff9d0055330 0x7ff9d00577f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:28.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.783+0000 7ff9ebfff700 1 --2- 172.21.15.162:0/3804180486 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff9ec060710 0x7ff9ec0aaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:28.788 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.783+0000 7ff9ebfff700 1 --2- 172.21.15.162:0/3804180486 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9ec0994f0 0x7ff9ec0ac750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:28.788 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.783+0000 7ff9ebfff700 1 --2- 172.21.15.162:0/3804180486 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9ec098b30 0x7ff9ec155ce0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:28.788 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.783+0000 7ff9ebfff700 1 -- 172.21.15.162:0/3804180486 >> 172.21.15.162:0/3804180486 conn(0x7ff9ec05b970 msgr2=0x7ff9ec148ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:28.788 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.783+0000 7ff9ebfff700 1 -- 172.21.15.162:0/3804180486 shutdown_connections 2020-11-28T21:55:28.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.783+0000 7ff9ebfff700 1 -- 172.21.15.162:0/3804180486 wait complete. 2020-11-28T21:55:28.798 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-28T21:55:28.798 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:55:28.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.911+0000 7f87a34ad700 1 Processor -- start 2020-11-28T21:55:28.914 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.911+0000 7f87a34ad700 1 -- start start 2020-11-28T21:55:28.915 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.911+0000 7f87a34ad700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f879c0b9ce0 0x7f879c1433c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:28.915 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.911+0000 7f87a34ad700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f879c143950 con 0x7f879c0605c0 2020-11-28T21:55:28.916 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.911+0000 7f87a34ad700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f879c067aa0 con 0x7f879c067610 2020-11-28T21:55:28.916 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.911+0000 7f87a34ad700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f879c0b4340 con 0x7f879c0b9ce0 2020-11-28T21:55:28.916 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.911+0000 7f87a1249700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f879c0b9ce0 0x7f879c1433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:28.916 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.911+0000 7f87a1a4a700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f879c067610 0x7f879c13fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51642/0 (socket says 172.21.15.162:51642) 2020-11-28T21:55:28.916 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.911+0000 7f87a1a4a700 1 -- 172.21.15.162:0/3601515141 learned_addr learned my addr 172.21.15.162:0/3601515141 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:28.917 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.911+0000 7f878ffff700 1 -- 172.21.15.162:0/3601515141 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3503090705 0 0) 0x7f879c067aa0 con 0x7f879c067610 2020-11-28T21:55:28.918 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.911+0000 7f87a1249700 1 -- 172.21.15.162:0/3601515141 >> v1:172.21.15.162:6789/0 conn(0x7f879c067610 legacy=0x7f879c13fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:28.918 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.911+0000 7f87a1249700 1 -- 172.21.15.162:0/3601515141 >> v1:172.21.15.118:6789/0 conn(0x7f879c0605c0 legacy=0x7f879c0b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:28.918 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.911+0000 7f87a1249700 1 -- 172.21.15.162:0/3601515141 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f879c0ba0c0 con 0x7f879c0b9ce0 2020-11-28T21:55:28.918 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.911+0000 7f87a1249700 1 --2- 172.21.15.162:0/3601515141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f879c0b9ce0 0x7f879c1433c0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 rx=0x7f8798002f30 tx=0x7f8798006d60).ready entity=mon.1 client_cookie=5a840038b6469eec server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:28.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.911+0000 7f878ffff700 1 -- 172.21.15.162:0/3601515141 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87980097f0 con 0x7f879c0b9ce0 2020-11-28T21:55:28.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.911+0000 7f878ffff700 1 -- 172.21.15.162:0/3601515141 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8798009950 con 0x7f879c0b9ce0 2020-11-28T21:55:28.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.911+0000 7f878ffff700 1 -- 172.21.15.162:0/3601515141 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8798009ac0 con 0x7f879c0b9ce0 2020-11-28T21:55:28.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.911+0000 7f87a34ad700 1 -- 172.21.15.162:0/3601515141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f879c0b9ce0 msgr2=0x7f879c1433c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:28.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.911+0000 7f87a34ad700 1 --2- 172.21.15.162:0/3601515141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f879c0b9ce0 0x7f879c1433c0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 rx=0x7f8798002f30 tx=0x7f8798006d60).stop 2020-11-28T21:55:28.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.915+0000 7f87a34ad700 1 -- 172.21.15.162:0/3601515141 shutdown_connections 2020-11-28T21:55:28.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.915+0000 7f87a34ad700 1 --2- 172.21.15.162:0/3601515141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f879c0b9ce0 0x7f879c1433c0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:28.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.915+0000 7f87a34ad700 1 -- 172.21.15.162:0/3601515141 >> 172.21.15.162:0/3601515141 conn(0x7f879c05c430 msgr2=0x7f879c0b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:28.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.915+0000 7f87a34ad700 1 -- 172.21.15.162:0/3601515141 shutdown_connections 2020-11-28T21:55:28.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.915+0000 7f87a34ad700 1 -- 172.21.15.162:0/3601515141 wait complete. 2020-11-28T21:55:28.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.915+0000 7f87a34ad700 1 Processor -- start 2020-11-28T21:55:28.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.915+0000 7f87a34ad700 1 -- start start 2020-11-28T21:55:28.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.915+0000 7f87a34ad700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f879c0605c0 0x7f879c1423e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:28.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.915+0000 7f87a34ad700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f879c067610 0x7f879c155800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:28.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.915+0000 7f87a34ad700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f879c0b9ce0 0x7f879c09e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:28.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.915+0000 7f87a34ad700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879c155cf0 con 0x7f879c0605c0 2020-11-28T21:55:28.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.915+0000 7f87a34ad700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879c0b2180 con 0x7f879c067610 2020-11-28T21:55:28.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.915+0000 7f87a34ad700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f879c09de30 con 0x7f879c0b9ce0 2020-11-28T21:55:28.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.915+0000 7f87a1249700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f879c0605c0 0x7f879c1423e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:28.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.915+0000 7f87a0a48700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f879c067610 0x7f879c155800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:28.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.915+0000 7f87a1249700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f879c0605c0 0x7f879c1423e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.162:51788/0 (socket says 172.21.15.162:51788) 2020-11-28T21:55:28.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.915+0000 7f87a0a48700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f879c067610 0x7f879c155800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44802/0 (socket says 172.21.15.162:44802) 2020-11-28T21:55:28.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.915+0000 7f87a1249700 1 -- 172.21.15.162:0/2660258849 learned_addr learned my addr 172.21.15.162:0/2660258849 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:28.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.915+0000 7f87a1a4a700 1 --2- 172.21.15.162:0/2660258849 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f879c0b9ce0 0x7f879c09e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:28.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.919+0000 7f87a0a48700 1 -- 172.21.15.162:0/2660258849 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f879c0b9ce0 msgr2=0x7f879c09e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:28.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.919+0000 7f87a0a48700 1 --2- 172.21.15.162:0/2660258849 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f879c0b9ce0 0x7f879c09e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:28.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.919+0000 7f87a0a48700 1 -- 172.21.15.162:0/2660258849 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f879c0605c0 msgr2=0x7f879c1423e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:28.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.919+0000 7f87a0a48700 1 --2- 172.21.15.162:0/2660258849 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f879c0605c0 0x7f879c1423e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:28.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.919+0000 7f87a0a48700 1 -- 172.21.15.162:0/2660258849 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f879c144fb0 con 0x7f879c067610 2020-11-28T21:55:28.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.919+0000 7f87a1a4a700 1 --2- 172.21.15.162:0/2660258849 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f879c0b9ce0 0x7f879c09e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:55:28.926 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.919+0000 7f87a0a48700 1 --2- 172.21.15.162:0/2660258849 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f879c067610 0x7f879c155800 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 rx=0x7f879c0b05c0 tx=0x7f8790007b20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:28.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.919+0000 7f878e7fc700 1 -- 172.21.15.162:0/2660258849 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8790007240 con 0x7f879c067610 2020-11-28T21:55:28.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.919+0000 7f878e7fc700 1 -- 172.21.15.162:0/2660258849 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87900073a0 con 0x7f879c067610 2020-11-28T21:55:28.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.919+0000 7f878e7fc700 1 -- 172.21.15.162:0/2660258849 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87900066d0 con 0x7f879c067610 2020-11-28T21:55:28.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.919+0000 7f87a34ad700 1 -- 172.21.15.162:0/2660258849 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f879c145250 con 0x7f879c067610 2020-11-28T21:55:28.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.919+0000 7f87a34ad700 1 -- 172.21.15.162:0/2660258849 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f879c0b0920 con 0x7f879c067610 2020-11-28T21:55:28.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.919+0000 7f87a34ad700 1 -- 172.21.15.162:0/2660258849 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f879c057050 con 0x7f879c067610 2020-11-28T21:55:28.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.919+0000 7f878e7fc700 1 -- 172.21.15.162:0/2660258849 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8790010070 con 0x7f879c067610 2020-11-28T21:55:28.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.923+0000 7f878e7fc700 1 --2- 172.21.15.162:0/2660258849 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f87840554e0 0x7f87840579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:28.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.923+0000 7f878e7fc700 1 -- 172.21.15.162:0/2660258849 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f8790001e40 con 0x7f879c067610 2020-11-28T21:55:28.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.923+0000 7f87a1249700 1 --2- 172.21.15.162:0/2660258849 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f87840554e0 0x7f87840579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:28.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.923+0000 7f87a1249700 1 --2- 172.21.15.162:0/2660258849 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f87840554e0 0x7f87840579a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f879801b000 tx=0x7f8798016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:28.931 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:28.927+0000 7f878e7fc700 1 -- 172.21.15.162:0/2660258849 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f87900438b0 con 0x7f879c067610 2020-11-28T21:55:29.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.071+0000 7f87a34ad700 1 -- 172.21.15.162:0/2660258849 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f879c068350 con 0x7f879c067610 2020-11-28T21:55:29.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.071+0000 7f878e7fc700 1 -- 172.21.15.162:0/2660258849 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v134) v1 ==== 110+0+10 (secure 0 0 0) 0x7f8790018020 con 0x7f879c067610 2020-11-28T21:55:29.075 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:55:29.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.071+0000 7f87a34ad700 1 -- 172.21.15.162:0/2660258849 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f87840554e0 msgr2=0x7f87840579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:29.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.071+0000 7f87a34ad700 1 --2- 172.21.15.162:0/2660258849 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f87840554e0 0x7f87840579a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f879801b000 tx=0x7f8798016000).stop 2020-11-28T21:55:29.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.071+0000 7f87a34ad700 1 -- 172.21.15.162:0/2660258849 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f879c067610 msgr2=0x7f879c155800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:29.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.071+0000 7f87a34ad700 1 --2- 172.21.15.162:0/2660258849 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f879c067610 0x7f879c155800 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 rx=0x7f879c0b05c0 tx=0x7f8790007b20).stop 2020-11-28T21:55:29.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.071+0000 7f87a34ad700 1 -- 172.21.15.162:0/2660258849 shutdown_connections 2020-11-28T21:55:29.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.071+0000 7f87a34ad700 1 --2- 172.21.15.162:0/2660258849 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f87840554e0 0x7f87840579a0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:29.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.071+0000 7f87a34ad700 1 --2- 172.21.15.162:0/2660258849 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f879c0b9ce0 0x7f879c09e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:29.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.071+0000 7f87a34ad700 1 --2- 172.21.15.162:0/2660258849 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f879c0605c0 0x7f879c1423e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:29.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.071+0000 7f87a34ad700 1 --2- 172.21.15.162:0/2660258849 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f879c067610 0x7f879c155800 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:29.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.071+0000 7f87a34ad700 1 -- 172.21.15.162:0/2660258849 >> 172.21.15.162:0/2660258849 conn(0x7f879c05c430 msgr2=0x7f879c05f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:29.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.071+0000 7f87a34ad700 1 -- 172.21.15.162:0/2660258849 shutdown_connections 2020-11-28T21:55:29.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.071+0000 7f87a34ad700 1 -- 172.21.15.162:0/2660258849 wait complete. 2020-11-28T21:55:29.087 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:55:29.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.199+0000 7f0255bf0700 1 Processor -- start 2020-11-28T21:55:29.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.199+0000 7f0255bf0700 1 -- start start 2020-11-28T21:55:29.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.199+0000 7f0255bf0700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f02500b3df0 0x7f02500b2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:29.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.199+0000 7f0255bf0700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f025005b970 0x7f025014ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:29.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.199+0000 7f0255bf0700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f02500b2c50 0x7f025014b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:29.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.199+0000 7f0255bf0700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02500b41d0 con 0x7f02500b2c50 2020-11-28T21:55:29.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.199+0000 7f0255bf0700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02500b3160 con 0x7f025005b970 2020-11-28T21:55:29.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.199+0000 7f0255bf0700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02500b3400 con 0x7f02500b3df0 2020-11-28T21:55:29.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.199+0000 7f024ffff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f025005b970 0x7f025014ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:29.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.199+0000 7f024ffff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f025005b970 0x7f025014ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44810/0 (socket says 172.21.15.162:44810) 2020-11-28T21:55:29.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.199+0000 7f024ffff700 1 -- 172.21.15.162:0/2462412374 learned_addr learned my addr 172.21.15.162:0/2462412374 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:29.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.199+0000 7f0254bee700 1 --2- 172.21.15.162:0/2462412374 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f02500b3df0 0x7f02500b2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:29.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.199+0000 7f02553ef700 1 --2- 172.21.15.162:0/2462412374 >> v2:172.21.15.118:3300/0 conn(0x7f02500b2c50 0x7f025014b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:29.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.199+0000 7f024ffff700 1 -- 172.21.15.162:0/2462412374 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f02500b3df0 msgr2=0x7f02500b2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:29.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.199+0000 7f024ffff700 1 --2- 172.21.15.162:0/2462412374 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f02500b3df0 0x7f02500b2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:29.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.199+0000 7f024ffff700 1 -- 172.21.15.162:0/2462412374 >> v2:172.21.15.118:3300/0 conn(0x7f02500b2c50 msgr2=0x7f025014b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:29.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.199+0000 7f024ffff700 1 --2- 172.21.15.162:0/2462412374 >> v2:172.21.15.118:3300/0 conn(0x7f02500b2c50 0x7f025014b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:29.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.199+0000 7f024ffff700 1 -- 172.21.15.162:0/2462412374 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f025014a670 con 0x7f025005b970 2020-11-28T21:55:29.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.203+0000 7f024ffff700 1 --2- 172.21.15.162:0/2462412374 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f025005b970 0x7f025014ab10 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 rx=0x7f0244002f30 tx=0x7f0244006e80).ready entity=mon.1 client_cookie=a02824a1565bab02 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:29.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.203+0000 7f024f7fe700 1 -- 172.21.15.162:0/2462412374 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02440097f0 con 0x7f025005b970 2020-11-28T21:55:29.209 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.203+0000 7f024f7fe700 1 -- 172.21.15.162:0/2462412374 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0244009950 con 0x7f025005b970 2020-11-28T21:55:29.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.203+0000 7f024f7fe700 1 -- 172.21.15.162:0/2462412374 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0244009cb0 con 0x7f025005b970 2020-11-28T21:55:29.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.203+0000 7f0255bf0700 1 -- 172.21.15.162:0/2462412374 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f025005b970 msgr2=0x7f025014ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:29.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.203+0000 7f0255bf0700 1 --2- 172.21.15.162:0/2462412374 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f025005b970 0x7f025014ab10 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 rx=0x7f0244002f30 tx=0x7f0244006e80).stop 2020-11-28T21:55:29.212 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.203+0000 7f0255bf0700 1 -- 172.21.15.162:0/2462412374 shutdown_connections 2020-11-28T21:55:29.212 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.203+0000 7f0255bf0700 1 --2- 172.21.15.162:0/2462412374 >> v2:172.21.15.118:3300/0 conn(0x7f02500b2c50 0x7f025014b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:29.213 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.203+0000 7f0255bf0700 1 --2- 172.21.15.162:0/2462412374 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f02500b3df0 0x7f02500b2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:29.213 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.203+0000 7f0255bf0700 1 --2- 172.21.15.162:0/2462412374 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f025005b970 0x7f025014ab10 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:29.213 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.203+0000 7f0255bf0700 1 -- 172.21.15.162:0/2462412374 >> 172.21.15.162:0/2462412374 conn(0x7f02500604c0 msgr2=0x7f02500608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:29.213 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.203+0000 7f0255bf0700 1 -- 172.21.15.162:0/2462412374 shutdown_connections 2020-11-28T21:55:29.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.203+0000 7f0255bf0700 1 -- 172.21.15.162:0/2462412374 wait complete. 2020-11-28T21:55:29.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.203+0000 7f0255bf0700 1 Processor -- start 2020-11-28T21:55:29.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.203+0000 7f0255bf0700 1 -- start start 2020-11-28T21:55:29.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.207+0000 7f0255bf0700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f02500b2c50 0x7f02500af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:29.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.207+0000 7f0255bf0700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f02500b3df0 0x7f0250156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:29.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.207+0000 7f0255bf0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0250144e40 0x7f02501565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:29.216 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.207+0000 7f0255bf0700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0250155c50 con 0x7f02500b3df0 2020-11-28T21:55:29.216 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.207+0000 7f0255bf0700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0250147460 con 0x7f0250144e40 2020-11-28T21:55:29.216 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.207+0000 7f0255bf0700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0250147700 con 0x7f02500b2c50 2020-11-28T21:55:29.216 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.207+0000 7f02553ef700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0250144e40 0x7f02501565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:29.217 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.207+0000 7f02553ef700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0250144e40 0x7f02501565f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44818/0 (socket says 172.21.15.162:44818) 2020-11-28T21:55:29.217 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.207+0000 7f02553ef700 1 -- 172.21.15.162:0/4078558564 learned_addr learned my addr 172.21.15.162:0/4078558564 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:29.217 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.207+0000 7f0254bee700 1 --2- 172.21.15.162:0/4078558564 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f02500b2c50 0x7f02500af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:29.217 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.207+0000 7f024ffff700 1 --2- 172.21.15.162:0/4078558564 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f02500b3df0 0x7f0250156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:29.217 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.207+0000 7f02553ef700 1 -- 172.21.15.162:0/4078558564 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f02500b2c50 msgr2=0x7f02500af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:29.218 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.207+0000 7f02553ef700 1 --2- 172.21.15.162:0/4078558564 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f02500b2c50 0x7f02500af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:29.218 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.207+0000 7f02553ef700 1 -- 172.21.15.162:0/4078558564 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f02500b3df0 msgr2=0x7f0250156100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:29.218 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.207+0000 7f02553ef700 1 --2- 172.21.15.162:0/4078558564 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f02500b3df0 0x7f0250156100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:29.218 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.207+0000 7f02553ef700 1 -- 172.21.15.162:0/4078558564 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02501479a0 con 0x7f0250144e40 2020-11-28T21:55:29.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.207+0000 7f0254bee700 1 --2- 172.21.15.162:0/4078558564 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f02500b2c50 0x7f02500af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:55:29.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.207+0000 7f02553ef700 1 --2- 172.21.15.162:0/4078558564 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0250144e40 0x7f02501565f0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 rx=0x7f0250147fc0 tx=0x7f0240008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:29.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.207+0000 7f024dffb700 1 -- 172.21.15.162:0/4078558564 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0240011850 con 0x7f0250144e40 2020-11-28T21:55:29.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.207+0000 7f024dffb700 1 -- 172.21.15.162:0/4078558564 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02400119b0 con 0x7f0250144e40 2020-11-28T21:55:29.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.207+0000 7f0255bf0700 1 -- 172.21.15.162:0/4078558564 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0250147c40 con 0x7f0250144e40 2020-11-28T21:55:29.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.207+0000 7f024dffb700 1 -- 172.21.15.162:0/4078558564 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0240011d10 con 0x7f0250144e40 2020-11-28T21:55:29.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.207+0000 7f0255bf0700 1 -- 172.21.15.162:0/4078558564 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02500ad100 con 0x7f0250144e40 2020-11-28T21:55:29.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.207+0000 7f0255bf0700 1 -- 172.21.15.162:0/4078558564 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0250147dd0 con 0x7f0250144e40 2020-11-28T21:55:29.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.207+0000 7f024dffb700 1 -- 172.21.15.162:0/4078558564 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f024001c460 con 0x7f0250144e40 2020-11-28T21:55:29.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.211+0000 7f024dffb700 1 --2- 172.21.15.162:0/4078558564 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0230055330 0x7f02300577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:29.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.211+0000 7f024dffb700 1 -- 172.21.15.162:0/4078558564 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f024006e0c0 con 0x7f0250144e40 2020-11-28T21:55:29.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.211+0000 7f0254bee700 1 --2- 172.21.15.162:0/4078558564 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0230055330 0x7f02300577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:29.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.211+0000 7f0254bee700 1 --2- 172.21.15.162:0/4078558564 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0230055330 0x7f02300577f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f024800c000 tx=0x7f0248008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:29.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.219+0000 7f024dffb700 1 -- 172.21.15.162:0/4078558564 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0240044690 con 0x7f0250144e40 2020-11-28T21:55:29.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.359+0000 7f0255bf0700 1 -- 172.21.15.162:0/4078558564 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0250057050 con 0x7f0250144e40 2020-11-28T21:55:29.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.363+0000 7f024dffb700 1 -- 172.21.15.162:0/4078558564 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v134) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0240047cb0 con 0x7f0250144e40 2020-11-28T21:55:29.365 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:55:29.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.363+0000 7f0255bf0700 1 -- 172.21.15.162:0/4078558564 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0230055330 msgr2=0x7f02300577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:29.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.363+0000 7f0255bf0700 1 --2- 172.21.15.162:0/4078558564 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0230055330 0x7f02300577f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f024800c000 tx=0x7f0248008040).stop 2020-11-28T21:55:29.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.363+0000 7f0255bf0700 1 -- 172.21.15.162:0/4078558564 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0250144e40 msgr2=0x7f02501565f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:29.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.363+0000 7f0255bf0700 1 --2- 172.21.15.162:0/4078558564 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0250144e40 0x7f02501565f0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 rx=0x7f0250147fc0 tx=0x7f0240008040).stop 2020-11-28T21:55:29.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.363+0000 7f0255bf0700 1 -- 172.21.15.162:0/4078558564 shutdown_connections 2020-11-28T21:55:29.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.363+0000 7f0255bf0700 1 --2- 172.21.15.162:0/4078558564 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0230055330 0x7f02300577f0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:29.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.363+0000 7f0255bf0700 1 --2- 172.21.15.162:0/4078558564 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f02500b2c50 0x7f02500af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:29.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.363+0000 7f0255bf0700 1 --2- 172.21.15.162:0/4078558564 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0250144e40 0x7f02501565f0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:29.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.363+0000 7f0255bf0700 1 --2- 172.21.15.162:0/4078558564 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f02500b3df0 0x7f0250156100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:29.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.363+0000 7f0255bf0700 1 -- 172.21.15.162:0/4078558564 >> 172.21.15.162:0/4078558564 conn(0x7f02500604c0 msgr2=0x7f0250142be0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:29.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.363+0000 7f0255bf0700 1 -- 172.21.15.162:0/4078558564 shutdown_connections 2020-11-28T21:55:29.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.363+0000 7f0255bf0700 1 -- 172.21.15.162:0/4078558564 wait complete. 2020-11-28T21:55:29.385 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2020-11-28T21:55:29.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.487+0000 7f3d30d58700 1 Processor -- start 2020-11-28T21:55:29.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.487+0000 7f3d30d58700 1 -- start start 2020-11-28T21:55:29.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d30d58700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f3d2c064c70 0x7f3d2c066290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:29.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d30d58700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d2c065050 con 0x7f3d2c13f6c0 2020-11-28T21:55:29.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d30d58700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d2c13fc40 con 0x7f3d2c065750 2020-11-28T21:55:29.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d30d58700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d2c13fee0 con 0x7f3d2c064c70 2020-11-28T21:55:29.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d2bfff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f3d2c065750 0x7f3d2c13c260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51670/0 (socket says 172.21.15.162:51670) 2020-11-28T21:55:29.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d2bfff700 1 -- 172.21.15.162:0/1721834306 learned_addr learned my addr 172.21.15.162:0/1721834306 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:29.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d2b7fe700 1 --2- 172.21.15.162:0/1721834306 >> v2:172.21.15.118:3300/0 conn(0x7f3d2c064c70 0x7f3d2c066290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:29.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d2affd700 1 -- 172.21.15.162:0/1721834306 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1248325193 0 0) 0x7f3d2c13fc40 con 0x7f3d2c065750 2020-11-28T21:55:29.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d2affd700 1 -- 172.21.15.162:0/1721834306 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d0c003230 con 0x7f3d2c065750 2020-11-28T21:55:29.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d2affd700 1 -- 172.21.15.162:0/1721834306 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3033582627 0 0) 0x7f3d0c003230 con 0x7f3d2c065750 2020-11-28T21:55:29.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d2affd700 1 -- 172.21.15.162:0/1721834306 >> v2:172.21.15.118:3300/0 conn(0x7f3d2c064c70 msgr2=0x7f3d2c066290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:29.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d2affd700 1 --2- 172.21.15.162:0/1721834306 >> v2:172.21.15.118:3300/0 conn(0x7f3d2c064c70 0x7f3d2c066290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:29.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d2affd700 1 -- 172.21.15.162:0/1721834306 >> v1:172.21.15.118:6789/0 conn(0x7f3d2c13f6c0 legacy=0x7f3d2c066780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:29.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d2affd700 1 -- 172.21.15.162:0/1721834306 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d2c140180 con 0x7f3d2c065750 2020-11-28T21:55:29.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d2affd700 1 -- 172.21.15.162:0/1721834306 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f3d24001a50 con 0x7f3d2c065750 2020-11-28T21:55:29.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d2affd700 1 -- 172.21.15.162:0/1721834306 >> v1:172.21.15.162:6789/0 conn(0x7f3d2c065750 legacy=0x7f3d2c13c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:29.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d2affd700 1 --2- 172.21.15.162:0/1721834306 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d0c003720 0x7f3d0c003b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:29.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d2affd700 1 --2- 172.21.15.162:0/1721834306 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3d0c0044d0 0x7f3d0c004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:29.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d2affd700 1 --2- 172.21.15.162:0/1721834306 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d0c004e40 0x7f3d0c00b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:29.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d2affd700 1 -- 172.21.15.162:0/1721834306 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2c13fee0 con 0x7f3d0c003720 2020-11-28T21:55:29.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d2affd700 1 -- 172.21.15.162:0/1721834306 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0c0041c0 con 0x7f3d0c004e40 2020-11-28T21:55:29.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d2affd700 1 -- 172.21.15.162:0/1721834306 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0c00de20 con 0x7f3d0c0044d0 2020-11-28T21:55:29.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d2bfff700 1 --2- 172.21.15.162:0/1721834306 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d0c004e40 0x7f3d0c00b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:29.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d2b7fe700 1 --2- 172.21.15.162:0/1721834306 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d0c003720 0x7f3d0c003b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:29.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d23fff700 1 --2- 172.21.15.162:0/1721834306 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3d0c0044d0 0x7f3d0c004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:29.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d2bfff700 1 -- 172.21.15.162:0/1721834306 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3d0c0044d0 msgr2=0x7f3d0c004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:29.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d2bfff700 1 --2- 172.21.15.162:0/1721834306 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3d0c0044d0 0x7f3d0c004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:29.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d2bfff700 1 -- 172.21.15.162:0/1721834306 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d0c003720 msgr2=0x7f3d0c003b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:29.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d2bfff700 1 --2- 172.21.15.162:0/1721834306 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d0c003720 0x7f3d0c003b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:29.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d2bfff700 1 -- 172.21.15.162:0/1721834306 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d0c003230 con 0x7f3d0c004e40 2020-11-28T21:55:29.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d2b7fe700 1 --2- 172.21.15.162:0/1721834306 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d0c003720 0x7f3d0c003b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:55:29.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d2bfff700 1 -- 172.21.15.162:0/1721834306 reap_dead start 2020-11-28T21:55:29.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d2bfff700 1 --2- 172.21.15.162:0/1721834306 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d0c004e40 0x7f3d0c00b720 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 rx=0x7f3d24001760 tx=0x7f3d2c140310).ready entity=mon.1 client_cookie=bc7a41688ad2303a server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:29.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d2affd700 1 -- 172.21.15.162:0/1721834306 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d24005c40 con 0x7f3d0c004e40 2020-11-28T21:55:29.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d2affd700 1 -- 172.21.15.162:0/1721834306 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d24005da0 con 0x7f3d0c004e40 2020-11-28T21:55:29.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d2affd700 1 -- 172.21.15.162:0/1721834306 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d240072c0 con 0x7f3d0c004e40 2020-11-28T21:55:29.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d30d58700 1 -- 172.21.15.162:0/1721834306 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d0c004e40 msgr2=0x7f3d0c00b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:29.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d30d58700 1 --2- 172.21.15.162:0/1721834306 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d0c004e40 0x7f3d0c00b720 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 rx=0x7f3d24001760 tx=0x7f3d2c140310).stop 2020-11-28T21:55:29.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d30d58700 1 -- 172.21.15.162:0/1721834306 shutdown_connections 2020-11-28T21:55:29.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d30d58700 1 --2- 172.21.15.162:0/1721834306 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d0c004e40 0x7f3d0c00b720 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:29.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d30d58700 1 -- 172.21.15.162:0/1721834306 >> 172.21.15.162:0/1721834306 conn(0x7f3d2c0b5da0 msgr2=0x7f3d2c0b6180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:29.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d30d58700 1 -- 172.21.15.162:0/1721834306 shutdown_connections 2020-11-28T21:55:29.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.491+0000 7f3d30d58700 1 -- 172.21.15.162:0/1721834306 wait complete. 2020-11-28T21:55:29.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.495+0000 7f3d30d58700 1 Processor -- start 2020-11-28T21:55:29.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.495+0000 7f3d30d58700 1 -- start start 2020-11-28T21:55:29.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.495+0000 7f3d30d58700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d0c004e40 0x7f3d2c14e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:29.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.495+0000 7f3d30d58700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d2c061a00 0x7f3d2c150c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:29.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.495+0000 7f3d30d58700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3d2c151150 0x7f3d2c0af6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:29.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.495+0000 7f3d30d58700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2c0b5280 con 0x7f3d0c004e40 2020-11-28T21:55:29.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.495+0000 7f3d30d58700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2c14d870 con 0x7f3d2c061a00 2020-11-28T21:55:29.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.499+0000 7f3d30d58700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2c14db10 con 0x7f3d2c151150 2020-11-28T21:55:29.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.499+0000 7f3d23fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d2c061a00 0x7f3d2c150c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:29.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.499+0000 7f3d2b7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d0c004e40 0x7f3d2c14e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:29.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.499+0000 7f3d23fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d2c061a00 0x7f3d2c150c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44836/0 (socket says 172.21.15.162:44836) 2020-11-28T21:55:29.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.499+0000 7f3d23fff700 1 -- 172.21.15.162:0/3779032134 learned_addr learned my addr 172.21.15.162:0/3779032134 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:29.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.499+0000 7f3d2bfff700 1 --2- 172.21.15.162:0/3779032134 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3d2c151150 0x7f3d2c0af6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:29.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.499+0000 7f3d2b7fe700 1 -- 172.21.15.162:0/3779032134 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3d2c151150 msgr2=0x7f3d2c0af6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:29.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.499+0000 7f3d2b7fe700 1 --2- 172.21.15.162:0/3779032134 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3d2c151150 0x7f3d2c0af6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:29.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.499+0000 7f3d2b7fe700 1 -- 172.21.15.162:0/3779032134 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d2c061a00 msgr2=0x7f3d2c150c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:29.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.499+0000 7f3d2b7fe700 1 --2- 172.21.15.162:0/3779032134 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d2c061a00 0x7f3d2c150c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:29.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.499+0000 7f3d2b7fe700 1 -- 172.21.15.162:0/3779032134 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d2c14ddb0 con 0x7f3d0c004e40 2020-11-28T21:55:29.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.499+0000 7f3d23fff700 1 --2- 172.21.15.162:0/3779032134 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d2c061a00 0x7f3d2c150c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:55:29.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.499+0000 7f3d2b7fe700 1 --2- 172.21.15.162:0/3779032134 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d0c004e40 0x7f3d2c14e250 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f3d1c008000 tx=0x7f3d1c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:29.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.499+0000 7f3d297fa700 1 -- 172.21.15.162:0/3779032134 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d1c014680 con 0x7f3d0c004e40 2020-11-28T21:55:29.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.499+0000 7f3d297fa700 1 -- 172.21.15.162:0/3779032134 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d1c0147e0 con 0x7f3d0c004e40 2020-11-28T21:55:29.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.499+0000 7f3d30d58700 1 -- 172.21.15.162:0/3779032134 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d2c0ae220 con 0x7f3d0c004e40 2020-11-28T21:55:29.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.499+0000 7f3d297fa700 1 -- 172.21.15.162:0/3779032134 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d1c014b40 con 0x7f3d0c004e40 2020-11-28T21:55:29.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.499+0000 7f3d30d58700 1 -- 172.21.15.162:0/3779032134 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d2c0b59b0 con 0x7f3d0c004e40 2020-11-28T21:55:29.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.499+0000 7f3d30d58700 1 -- 172.21.15.162:0/3779032134 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d2c061de0 con 0x7f3d0c004e40 2020-11-28T21:55:29.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.499+0000 7f3d297fa700 1 -- 172.21.15.162:0/3779032134 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f3d1c024070 con 0x7f3d0c004e40 2020-11-28T21:55:29.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.503+0000 7f3d297fa700 1 --2- 172.21.15.162:0/3779032134 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3d100553b0 0x7f3d10057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:29.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.503+0000 7f3d23fff700 1 --2- 172.21.15.162:0/3779032134 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3d100553b0 0x7f3d10057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:29.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.503+0000 7f3d297fa700 1 -- 172.21.15.162:0/3779032134 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f3d1c06e810 con 0x7f3d0c004e40 2020-11-28T21:55:29.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.503+0000 7f3d23fff700 1 --2- 172.21.15.162:0/3779032134 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3d100553b0 0x7f3d10057870 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7f3d1400d000 tx=0x7f3d1400c040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:29.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.511+0000 7f3d297fa700 1 -- 172.21.15.162:0/3779032134 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f3d1c044e40 con 0x7f3d0c004e40 2020-11-28T21:55:29.660 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.655+0000 7f3d30d58700 1 -- 172.21.15.162:0/3779032134 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3d2c057050 con 0x7f3d0c004e40 2020-11-28T21:55:29.661 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.655+0000 7f3d297fa700 1 -- 172.21.15.162:0/3779032134 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v134) v1 ==== 84+0+143 (secure 0 0 0) 0x7f3d1c02b020 con 0x7f3d0c004e40 2020-11-28T21:55:29.661 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:29.661 INFO:teuthology.orchestra.run.smithi162.stdout:[ 2020-11-28T21:55:29.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:29.662 INFO:teuthology.orchestra.run.smithi162.stdout: "poolnum": 1, 2020-11-28T21:55:29.662 INFO:teuthology.orchestra.run.smithi162.stdout: "poolname": "device_health_metrics" 2020-11-28T21:55:29.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:29.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:29.663 INFO:teuthology.orchestra.run.smithi162.stdout: "poolnum": 2, 2020-11-28T21:55:29.663 INFO:teuthology.orchestra.run.smithi162.stdout: "poolname": "rbd" 2020-11-28T21:55:29.663 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:29.663 INFO:teuthology.orchestra.run.smithi162.stdout:] 2020-11-28T21:55:29.664 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.659+0000 7f3d30d58700 1 -- 172.21.15.162:0/3779032134 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3d100553b0 msgr2=0x7f3d10057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:29.665 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.659+0000 7f3d30d58700 1 --2- 172.21.15.162:0/3779032134 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3d100553b0 0x7f3d10057870 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7f3d1400d000 tx=0x7f3d1400c040).stop 2020-11-28T21:55:29.665 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.659+0000 7f3d30d58700 1 -- 172.21.15.162:0/3779032134 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d0c004e40 msgr2=0x7f3d2c14e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:29.665 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.659+0000 7f3d30d58700 1 --2- 172.21.15.162:0/3779032134 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d0c004e40 0x7f3d2c14e250 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f3d1c008000 tx=0x7f3d1c008040).stop 2020-11-28T21:55:29.665 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.659+0000 7f3d30d58700 1 -- 172.21.15.162:0/3779032134 shutdown_connections 2020-11-28T21:55:29.665 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.659+0000 7f3d30d58700 1 --2- 172.21.15.162:0/3779032134 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3d100553b0 0x7f3d10057870 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:29.666 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.659+0000 7f3d30d58700 1 --2- 172.21.15.162:0/3779032134 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3d2c151150 0x7f3d2c0af6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:29.666 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.659+0000 7f3d30d58700 1 --2- 172.21.15.162:0/3779032134 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d0c004e40 0x7f3d2c14e250 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:29.666 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.659+0000 7f3d30d58700 1 --2- 172.21.15.162:0/3779032134 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d2c061a00 0x7f3d2c150c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:29.666 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.659+0000 7f3d30d58700 1 -- 172.21.15.162:0/3779032134 >> 172.21.15.162:0/3779032134 conn(0x7f3d2c0b5da0 msgr2=0x7f3d2c139540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:29.666 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.659+0000 7f3d30d58700 1 -- 172.21.15.162:0/3779032134 shutdown_connections 2020-11-28T21:55:29.667 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.659+0000 7f3d30d58700 1 -- 172.21.15.162:0/3779032134 wait complete. 2020-11-28T21:55:29.677 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-11-28T21:55:29.678 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2020-11-28T21:55:29.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.787+0000 7f32d08aa700 1 Processor -- start 2020-11-28T21:55:29.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.787+0000 7f32d08aa700 1 -- start start 2020-11-28T21:55:29.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.791+0000 7f32d08aa700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f32cc0b18c0 0x7f32cc0b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:29.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.791+0000 7f32d08aa700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f32cc0b44e0 0x7f32cc0b2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:29.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.791+0000 7f32d08aa700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32cc0b1f50 con 0x7f32cc0b3950 2020-11-28T21:55:29.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.791+0000 7f32d08aa700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32cc0aab50 con 0x7f32cc0b44e0 2020-11-28T21:55:29.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.791+0000 7f32d08aa700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32cc143e90 con 0x7f32cc0b18c0 2020-11-28T21:55:29.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.791+0000 7f32ca59c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f32cc0b18c0 0x7f32cc0b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:29.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.791+0000 7f32cad9d700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f32cc0b3950 0x7f32cc13dd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51686/0 (socket says 172.21.15.162:51686) 2020-11-28T21:55:29.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.791+0000 7f32ca59c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f32cc0b18c0 0x7f32cc0b3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44842/0 (socket says 172.21.15.162:44842) 2020-11-28T21:55:29.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.791+0000 7f32ca59c700 1 -- 172.21.15.162:0/2961679254 learned_addr learned my addr 172.21.15.162:0/2961679254 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:29.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.791+0000 7f32cb59e700 1 --2- 172.21.15.162:0/2961679254 >> v2:172.21.15.118:3300/0 conn(0x7f32cc0b44e0 0x7f32cc0b2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:29.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.791+0000 7f32ca59c700 1 -- 172.21.15.162:0/2961679254 >> v2:172.21.15.118:3300/0 conn(0x7f32cc0b44e0 msgr2=0x7f32cc0b2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:29.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.791+0000 7f32ca59c700 1 --2- 172.21.15.162:0/2961679254 >> v2:172.21.15.118:3300/0 conn(0x7f32cc0b44e0 0x7f32cc0b2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:29.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.791+0000 7f32ca59c700 1 -- 172.21.15.162:0/2961679254 >> v1:172.21.15.162:6789/0 conn(0x7f32cc0b3950 legacy=0x7f32cc13dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:29.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.791+0000 7f32ca59c700 1 -- 172.21.15.162:0/2961679254 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32cc0aa8b0 con 0x7f32cc0b18c0 2020-11-28T21:55:29.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.791+0000 7f32cb59e700 1 --2- 172.21.15.162:0/2961679254 >> v2:172.21.15.118:3300/0 conn(0x7f32cc0b44e0 0x7f32cc0b2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:55:29.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.791+0000 7f32ca59c700 1 --2- 172.21.15.162:0/2961679254 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32cc0b18c0 0x7f32cc0b3d30 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 rx=0x7f32c0002f30 tx=0x7f32c0006e80).ready entity=mon.1 client_cookie=49e5498db0ec3f8 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:29.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.791+0000 7f32c9d9b700 1 -- 172.21.15.162:0/2961679254 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f32c00097f0 con 0x7f32cc0b18c0 2020-11-28T21:55:29.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.791+0000 7f32c9d9b700 1 -- 172.21.15.162:0/2961679254 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c0009950 con 0x7f32cc0b18c0 2020-11-28T21:55:29.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.791+0000 7f32c9d9b700 1 -- 172.21.15.162:0/2961679254 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f32c0009cb0 con 0x7f32cc0b18c0 2020-11-28T21:55:29.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.791+0000 7f32d08aa700 1 -- 172.21.15.162:0/2961679254 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32cc0b18c0 msgr2=0x7f32cc0b3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:29.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.791+0000 7f32d08aa700 1 --2- 172.21.15.162:0/2961679254 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32cc0b18c0 0x7f32cc0b3d30 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 rx=0x7f32c0002f30 tx=0x7f32c0006e80).stop 2020-11-28T21:55:29.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.791+0000 7f32d08aa700 1 -- 172.21.15.162:0/2961679254 shutdown_connections 2020-11-28T21:55:29.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.791+0000 7f32d08aa700 1 --2- 172.21.15.162:0/2961679254 >> v2:172.21.15.118:3300/0 conn(0x7f32cc0b44e0 0x7f32cc0b2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:29.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.791+0000 7f32d08aa700 1 --2- 172.21.15.162:0/2961679254 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32cc0b18c0 0x7f32cc0b3d30 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:29.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.791+0000 7f32d08aa700 1 -- 172.21.15.162:0/2961679254 >> 172.21.15.162:0/2961679254 conn(0x7f32cc062630 msgr2=0x7f32cc062a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:29.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.791+0000 7f32d08aa700 1 -- 172.21.15.162:0/2961679254 shutdown_connections 2020-11-28T21:55:29.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.791+0000 7f32d08aa700 1 -- 172.21.15.162:0/2961679254 wait complete. 2020-11-28T21:55:29.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.791+0000 7f32d08aa700 1 Processor -- start 2020-11-28T21:55:29.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.791+0000 7f32d08aa700 1 -- start start 2020-11-28T21:55:29.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.795+0000 7f32d08aa700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32cc0b18c0 0x7f32cc0b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:29.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.795+0000 7f32d08aa700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32cc0b3950 0x7f32cc152f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:29.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.795+0000 7f32d08aa700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f32cc0b44e0 0x7f32cc153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:29.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.795+0000 7f32d08aa700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32cc140f30 con 0x7f32cc0b18c0 2020-11-28T21:55:29.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.795+0000 7f32d08aa700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32cc140940 con 0x7f32cc0b3950 2020-11-28T21:55:29.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.795+0000 7f32d08aa700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32cc141160 con 0x7f32cc0b44e0 2020-11-28T21:55:29.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.795+0000 7f32ca59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32cc0b3950 0x7f32cc152f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:29.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.795+0000 7f32ca59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32cc0b3950 0x7f32cc152f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44850/0 (socket says 172.21.15.162:44850) 2020-11-28T21:55:29.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.795+0000 7f32ca59c700 1 -- 172.21.15.162:0/909209620 learned_addr learned my addr 172.21.15.162:0/909209620 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:29.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.795+0000 7f32cb59e700 1 --2- 172.21.15.162:0/909209620 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f32cc0b44e0 0x7f32cc153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:29.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.795+0000 7f32cad9d700 1 --2- 172.21.15.162:0/909209620 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32cc0b18c0 0x7f32cc0b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:29.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.795+0000 7f32ca59c700 1 -- 172.21.15.162:0/909209620 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f32cc0b44e0 msgr2=0x7f32cc153460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:29.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.795+0000 7f32ca59c700 1 --2- 172.21.15.162:0/909209620 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f32cc0b44e0 0x7f32cc153460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:29.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.795+0000 7f32ca59c700 1 -- 172.21.15.162:0/909209620 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32cc0b18c0 msgr2=0x7f32cc0b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:29.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.795+0000 7f32ca59c700 1 --2- 172.21.15.162:0/909209620 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32cc0b18c0 0x7f32cc0b75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:29.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.795+0000 7f32ca59c700 1 -- 172.21.15.162:0/909209620 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32cc141400 con 0x7f32cc0b3950 2020-11-28T21:55:29.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.795+0000 7f32ca59c700 1 --2- 172.21.15.162:0/909209620 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32cc0b3950 0x7f32cc152f70 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 rx=0x7f32cc0ad350 tx=0x7f32c0015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:29.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.795+0000 7f32bbfff700 1 -- 172.21.15.162:0/909209620 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f32c0009c50 con 0x7f32cc0b3950 2020-11-28T21:55:29.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.795+0000 7f32d08aa700 1 -- 172.21.15.162:0/909209620 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32cc13f850 con 0x7f32cc0b3950 2020-11-28T21:55:29.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.795+0000 7f32bbfff700 1 -- 172.21.15.162:0/909209620 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c001e460 con 0x7f32cc0b3950 2020-11-28T21:55:29.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.795+0000 7f32d08aa700 1 -- 172.21.15.162:0/909209620 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32cc0abfa0 con 0x7f32cc0b3950 2020-11-28T21:55:29.810 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.795+0000 7f32bbfff700 1 -- 172.21.15.162:0/909209620 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f32c001e7c0 con 0x7f32cc0b3950 2020-11-28T21:55:29.810 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.795+0000 7f32d08aa700 1 -- 172.21.15.162:0/909209620 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32cc141590 con 0x7f32cc0b3950 2020-11-28T21:55:29.810 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.795+0000 7f32bbfff700 1 -- 172.21.15.162:0/909209620 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f32c0025030 con 0x7f32cc0b3950 2020-11-28T21:55:29.810 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.799+0000 7f32bbfff700 1 --2- 172.21.15.162:0/909209620 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f32a8055330 0x7f32a80577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:29.810 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.799+0000 7f32bbfff700 1 -- 172.21.15.162:0/909209620 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f32c006f200 con 0x7f32cc0b3950 2020-11-28T21:55:29.811 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.799+0000 7f32cad9d700 1 --2- 172.21.15.162:0/909209620 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f32a8055330 0x7f32a80577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:29.811 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.799+0000 7f32cad9d700 1 --2- 172.21.15.162:0/909209620 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f32a8055330 0x7f32a80577f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f32bc00b000 tx=0x7f32bc007160).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:29.812 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.803+0000 7f32bbfff700 1 -- 172.21.15.162:0/909209620 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f32c00457d0 con 0x7f32cc0b3950 2020-11-28T21:55:29.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:29.939+0000 7f32d08aa700 1 -- 172.21.15.162:0/909209620 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f32cc057050 con 0x7f32cc0b3950 2020-11-28T21:55:30.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.031+0000 7f32bbfff700 1 -- 172.21.15.162:0/909209620 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v135) v1 ==== 133+0+0 (secure 0 0 0) 0x7f32c0023020 con 0x7f32cc0b3950 2020-11-28T21:55:30.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.035+0000 7f32d08aa700 1 -- 172.21.15.162:0/909209620 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f32a8055330 msgr2=0x7f32a80577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:30.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.035+0000 7f32d08aa700 1 --2- 172.21.15.162:0/909209620 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f32a8055330 0x7f32a80577f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f32bc00b000 tx=0x7f32bc007160).stop 2020-11-28T21:55:30.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.035+0000 7f32d08aa700 1 -- 172.21.15.162:0/909209620 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32cc0b3950 msgr2=0x7f32cc152f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:30.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.035+0000 7f32d08aa700 1 --2- 172.21.15.162:0/909209620 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32cc0b3950 0x7f32cc152f70 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 rx=0x7f32cc0ad350 tx=0x7f32c0015000).stop 2020-11-28T21:55:30.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.035+0000 7f32d08aa700 1 -- 172.21.15.162:0/909209620 shutdown_connections 2020-11-28T21:55:30.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.035+0000 7f32d08aa700 1 --2- 172.21.15.162:0/909209620 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f32a8055330 0x7f32a80577f0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:30.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.035+0000 7f32d08aa700 1 --2- 172.21.15.162:0/909209620 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f32cc0b44e0 0x7f32cc153460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:30.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.035+0000 7f32d08aa700 1 --2- 172.21.15.162:0/909209620 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32cc0b18c0 0x7f32cc0b75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:30.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.035+0000 7f32d08aa700 1 --2- 172.21.15.162:0/909209620 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32cc0b3950 0x7f32cc152f70 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:30.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.035+0000 7f32d08aa700 1 -- 172.21.15.162:0/909209620 >> 172.21.15.162:0/909209620 conn(0x7f32cc062630 msgr2=0x7f32cc0b4d30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:30.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.035+0000 7f32d08aa700 1 -- 172.21.15.162:0/909209620 shutdown_connections 2020-11-28T21:55:30.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.035+0000 7f32d08aa700 1 -- 172.21.15.162:0/909209620 wait complete. 2020-11-28T21:55:30.043 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_metadata' created 2020-11-28T21:55:30.053 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2020-11-28T21:55:30.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.163+0000 7f8dd3fff700 1 Processor -- start 2020-11-28T21:55:30.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.163+0000 7f8dd3fff700 1 -- start start 2020-11-28T21:55:30.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd3fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8dd40b2290 0x7f8dd4060a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:30.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd3fff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dd41471f0 con 0x7f8dd40af6a0 2020-11-28T21:55:30.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd3fff700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dd4060090 con 0x7f8dd40b1eb0 2020-11-28T21:55:30.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd3fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd4060330 con 0x7f8dd40b2290 2020-11-28T21:55:30.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd2ffd700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f8dd40b1eb0 0x7f8dd4144350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51700/0 (socket says 172.21.15.162:51700) 2020-11-28T21:55:30.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd2ffd700 1 -- 172.21.15.162:0/1492262808 learned_addr learned my addr 172.21.15.162:0/1492262808 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:30.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd37fe700 1 --2- 172.21.15.162:0/1492262808 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8dd40b2290 0x7f8dd4060a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:30.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd1ffb700 1 -- 172.21.15.162:0/1492262808 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3445854035 0 0) 0x7f8dd4060090 con 0x7f8dd40b1eb0 2020-11-28T21:55:30.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd1ffb700 1 -- 172.21.15.162:0/1492262808 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8db8003230 con 0x7f8dd40b1eb0 2020-11-28T21:55:30.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd1ffb700 1 -- 172.21.15.162:0/1492262808 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 743812458 0 0) 0x7f8db8003230 con 0x7f8dd40b1eb0 2020-11-28T21:55:30.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd1ffb700 1 -- 172.21.15.162:0/1492262808 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8dd40b2290 msgr2=0x7f8dd4060a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:30.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd1ffb700 1 --2- 172.21.15.162:0/1492262808 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8dd40b2290 0x7f8dd4060a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:30.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd1ffb700 1 -- 172.21.15.162:0/1492262808 >> v1:172.21.15.118:6789/0 conn(0x7f8dd40af6a0 legacy=0x7f8dd40afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:30.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd1ffb700 1 -- 172.21.15.162:0/1492262808 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd40605d0 con 0x7f8dd40b1eb0 2020-11-28T21:55:30.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd1ffb700 1 -- 172.21.15.162:0/1492262808 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f8dc4001a50 con 0x7f8dd40b1eb0 2020-11-28T21:55:30.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd1ffb700 1 -- 172.21.15.162:0/1492262808 >> v1:172.21.15.162:6789/0 conn(0x7f8dd40b1eb0 legacy=0x7f8dd4144350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:30.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd1ffb700 1 --2- 172.21.15.162:0/1492262808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8db8003720 0x7f8db8003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:30.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd1ffb700 1 --2- 172.21.15.162:0/1492262808 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8dd40b2290 0x7f8db80044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:30.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd1ffb700 1 --2- 172.21.15.162:0/1492262808 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8db80049c0 0x7f8db8004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:30.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd1ffb700 1 -- 172.21.15.162:0/1492262808 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd4060330 con 0x7f8db80049c0 2020-11-28T21:55:30.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd1ffb700 1 -- 172.21.15.162:0/1492262808 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db80041c0 con 0x7f8db8003720 2020-11-28T21:55:30.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd1ffb700 1 -- 172.21.15.162:0/1492262808 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db800b720 con 0x7f8dd40b2290 2020-11-28T21:55:30.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd2ffd700 1 --2- 172.21.15.162:0/1492262808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8db8003720 0x7f8db8003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:30.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd37fe700 1 --2- 172.21.15.162:0/1492262808 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8db80049c0 0x7f8db8004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:30.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd2ffd700 1 -- 172.21.15.162:0/1492262808 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8dd40b2290 msgr2=0x7f8db80044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:30.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd27fc700 1 --2- 172.21.15.162:0/1492262808 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8dd40b2290 0x7f8db80044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:30.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd2ffd700 1 --2- 172.21.15.162:0/1492262808 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8dd40b2290 0x7f8db80044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:30.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd2ffd700 1 -- 172.21.15.162:0/1492262808 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8db80049c0 msgr2=0x7f8db8004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:30.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd2ffd700 1 --2- 172.21.15.162:0/1492262808 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8db80049c0 0x7f8db8004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:30.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd2ffd700 1 -- 172.21.15.162:0/1492262808 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db8003230 con 0x7f8db8003720 2020-11-28T21:55:30.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd37fe700 1 --2- 172.21.15.162:0/1492262808 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8db80049c0 0x7f8db8004e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:55:30.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd27fc700 1 --2- 172.21.15.162:0/1492262808 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8dd40b2290 0x7f8db80044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:55:30.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd2ffd700 1 --2- 172.21.15.162:0/1492262808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8db8003720 0x7f8db8003b40 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 rx=0x7f8dc4001760 tx=0x7f8dc4007af0).ready entity=mon.1 client_cookie=24ea175e75a0c5fb server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:30.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd1ffb700 1 -- 172.21.15.162:0/1492262808 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8dc4005b70 con 0x7f8db8003720 2020-11-28T21:55:30.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd1ffb700 1 -- 172.21.15.162:0/1492262808 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc4005cd0 con 0x7f8db8003720 2020-11-28T21:55:30.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd1ffb700 1 -- 172.21.15.162:0/1492262808 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8dc40072c0 con 0x7f8db8003720 2020-11-28T21:55:30.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd3fff700 1 -- 172.21.15.162:0/1492262808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8db8003720 msgr2=0x7f8db8003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:30.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd3fff700 1 --2- 172.21.15.162:0/1492262808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8db8003720 0x7f8db8003b40 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 rx=0x7f8dc4001760 tx=0x7f8dc4007af0).stop 2020-11-28T21:55:30.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd37fe700 1 -- 172.21.15.162:0/1492262808 reap_dead start 2020-11-28T21:55:30.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd3fff700 1 -- 172.21.15.162:0/1492262808 shutdown_connections 2020-11-28T21:55:30.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd3fff700 1 -- 172.21.15.162:0/1492262808 >> 172.21.15.162:0/1492262808 conn(0x7f8dd40b57d0 msgr2=0x7f8dd40b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:30.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd3fff700 1 -- 172.21.15.162:0/1492262808 shutdown_connections 2020-11-28T21:55:30.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.167+0000 7f8dd3fff700 1 -- 172.21.15.162:0/1492262808 wait complete. 2020-11-28T21:55:30.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.171+0000 7f8dd3fff700 1 Processor -- start 2020-11-28T21:55:30.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.171+0000 7f8dd3fff700 1 -- start start 2020-11-28T21:55:30.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.171+0000 7f8dd3fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8dd4145aa0 0x7f8dd409e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:30.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.171+0000 7f8dd3fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8dd4066a50 0x7f8dd409e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:30.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.171+0000 7f8dd3fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8dd4159c40 0x7f8dd415a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:30.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.171+0000 7f8dd3fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd409f0f0 con 0x7f8dd4159c40 2020-11-28T21:55:30.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.171+0000 7f8dd3fff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd4159810 con 0x7f8dd4145aa0 2020-11-28T21:55:30.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.171+0000 7f8dd3fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd409f320 con 0x7f8dd4066a50 2020-11-28T21:55:30.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.171+0000 7f8dd2ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8dd4145aa0 0x7f8dd409e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:30.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.171+0000 7f8dd2ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8dd4145aa0 0x7f8dd409e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44868/0 (socket says 172.21.15.162:44868) 2020-11-28T21:55:30.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.171+0000 7f8dd2ffd700 1 -- 172.21.15.162:0/2229210670 learned_addr learned my addr 172.21.15.162:0/2229210670 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:30.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.171+0000 7f8dd27fc700 1 --2- 172.21.15.162:0/2229210670 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8dd4066a50 0x7f8dd409e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:30.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.171+0000 7f8dd37fe700 1 --2- 172.21.15.162:0/2229210670 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8dd4159c40 0x7f8dd415a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:30.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.171+0000 7f8dd2ffd700 1 -- 172.21.15.162:0/2229210670 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8dd4066a50 msgr2=0x7f8dd409e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:30.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.171+0000 7f8dd2ffd700 1 --2- 172.21.15.162:0/2229210670 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8dd4066a50 0x7f8dd409e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:30.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.171+0000 7f8dd2ffd700 1 -- 172.21.15.162:0/2229210670 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8dd4159c40 msgr2=0x7f8dd415a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:30.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.171+0000 7f8dd2ffd700 1 --2- 172.21.15.162:0/2229210670 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8dd4159c40 0x7f8dd415a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:30.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.171+0000 7f8dd2ffd700 1 -- 172.21.15.162:0/2229210670 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd4065f30 con 0x7f8dd4145aa0 2020-11-28T21:55:30.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.171+0000 7f8dd2ffd700 1 --2- 172.21.15.162:0/2229210670 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8dd4145aa0 0x7f8dd409e460 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 rx=0x7f8dc4001760 tx=0x7f8dc4015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:30.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.171+0000 7f8dc3fff700 1 -- 172.21.15.162:0/2229210670 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8dc4007760 con 0x7f8dd4145aa0 2020-11-28T21:55:30.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.171+0000 7f8dc3fff700 1 -- 172.21.15.162:0/2229210670 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc4013450 con 0x7f8dd4145aa0 2020-11-28T21:55:30.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.171+0000 7f8dc3fff700 1 -- 172.21.15.162:0/2229210670 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8dc40137b0 con 0x7f8dd4145aa0 2020-11-28T21:55:30.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.171+0000 7f8dd3fff700 1 -- 172.21.15.162:0/2229210670 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dd40661d0 con 0x7f8dd4145aa0 2020-11-28T21:55:30.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.171+0000 7f8dd3fff700 1 -- 172.21.15.162:0/2229210670 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dd40a12a0 con 0x7f8dd4145aa0 2020-11-28T21:55:30.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.175+0000 7f8dc3fff700 1 -- 172.21.15.162:0/2229210670 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8dc4026020 con 0x7f8dd4145aa0 2020-11-28T21:55:30.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.175+0000 7f8dc3fff700 1 --2- 172.21.15.162:0/2229210670 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8db0055470 0x7f8db0057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:30.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.175+0000 7f8dd3fff700 1 -- 172.21.15.162:0/2229210670 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dd4159810 con 0x7f8dd4145aa0 2020-11-28T21:55:30.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.175+0000 7f8dc3fff700 1 -- 172.21.15.162:0/2229210670 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f8dc406f330 con 0x7f8dd4145aa0 2020-11-28T21:55:30.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.175+0000 7f8dd27fc700 1 --2- 172.21.15.162:0/2229210670 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8db0055470 0x7f8db0057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:30.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.175+0000 7f8dd27fc700 1 --2- 172.21.15.162:0/2229210670 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8db0055470 0x7f8db0057930 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f8dc800e000 tx=0x7f8dc8009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:30.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.183+0000 7f8dc3fff700 1 -- 172.21.15.162:0/2229210670 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8dc4045760 con 0x7f8dd4145aa0 2020-11-28T21:55:30.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:30.323+0000 7f8dd3fff700 1 -- 172.21.15.162:0/2229210670 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f8dd4057050 con 0x7f8dd4145aa0 2020-11-28T21:55:31.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.043+0000 7f8dc3fff700 1 -- 172.21.15.162:0/2229210670 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v136) v1 ==== 125+0+0 (secure 0 0 0) 0x7f8dc402d020 con 0x7f8dd4145aa0 2020-11-28T21:55:31.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.047+0000 7f8dd3fff700 1 -- 172.21.15.162:0/2229210670 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8db0055470 msgr2=0x7f8db0057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:31.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.047+0000 7f8dd3fff700 1 --2- 172.21.15.162:0/2229210670 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8db0055470 0x7f8db0057930 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f8dc800e000 tx=0x7f8dc8009040).stop 2020-11-28T21:55:31.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.047+0000 7f8dd3fff700 1 -- 172.21.15.162:0/2229210670 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8dd4145aa0 msgr2=0x7f8dd409e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:31.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.047+0000 7f8dd3fff700 1 --2- 172.21.15.162:0/2229210670 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8dd4145aa0 0x7f8dd409e460 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 rx=0x7f8dc4001760 tx=0x7f8dc4015000).stop 2020-11-28T21:55:31.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.047+0000 7f8dd3fff700 1 -- 172.21.15.162:0/2229210670 shutdown_connections 2020-11-28T21:55:31.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.047+0000 7f8dd3fff700 1 --2- 172.21.15.162:0/2229210670 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8db0055470 0x7f8db0057930 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:31.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.047+0000 7f8dd3fff700 1 --2- 172.21.15.162:0/2229210670 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8dd4159c40 0x7f8dd415a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:31.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.047+0000 7f8dd3fff700 1 --2- 172.21.15.162:0/2229210670 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8dd4145aa0 0x7f8dd409e460 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:31.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.047+0000 7f8dd3fff700 1 --2- 172.21.15.162:0/2229210670 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8dd4066a50 0x7f8dd409e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:31.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.047+0000 7f8dd3fff700 1 -- 172.21.15.162:0/2229210670 >> 172.21.15.162:0/2229210670 conn(0x7f8dd40b57d0 msgr2=0x7f8dd41469e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:31.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.047+0000 7f8dd3fff700 1 -- 172.21.15.162:0/2229210670 shutdown_connections 2020-11-28T21:55:31.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.047+0000 7f8dd3fff700 1 -- 172.21.15.162:0/2229210670 wait complete. 2020-11-28T21:55:31.056 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_data' created 2020-11-28T21:55:31.068 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2020-11-28T21:55:31.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.175+0000 7f236759e700 1 Processor -- start 2020-11-28T21:55:31.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.175+0000 7f236759e700 1 -- start start 2020-11-28T21:55:31.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.179+0000 7f236759e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23680b1eb0 0x7f23680b2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:31.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.179+0000 7f236759e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2368060a70 0x7f23680afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:31.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.179+0000 7f236759e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2368060f00 con 0x7f23680af6a0 2020-11-28T21:55:31.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.179+0000 7f236759e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23681471f0 con 0x7f2368060a70 2020-11-28T21:55:31.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.179+0000 7f236759e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2368060090 con 0x7f23680b1eb0 2020-11-28T21:55:31.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.179+0000 7f2366d9d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2368060a70 0x7f23680afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:31.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.179+0000 7f2365d9b700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f23680af6a0 0x7f2368144350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51720/0 (socket says 172.21.15.162:51720) 2020-11-28T21:55:31.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.179+0000 7f2365d9b700 1 -- 172.21.15.162:0/3492949986 learned_addr learned my addr 172.21.15.162:0/3492949986 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:31.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.179+0000 7f236659c700 1 --2- 172.21.15.162:0/3492949986 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23680b1eb0 0x7f23680b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:31.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.179+0000 7f2366d9d700 1 -- 172.21.15.162:0/3492949986 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23680b1eb0 msgr2=0x7f23680b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:31.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.179+0000 7f2366d9d700 1 --2- 172.21.15.162:0/3492949986 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23680b1eb0 0x7f23680b2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:31.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.179+0000 7f2366d9d700 1 -- 172.21.15.162:0/3492949986 >> v1:172.21.15.162:6789/0 conn(0x7f23680af6a0 legacy=0x7f2368144350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:31.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.179+0000 7f236559a700 1 -- 172.21.15.162:0/3492949986 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 979793275 0 0) 0x7f2368060f00 con 0x7f23680af6a0 2020-11-28T21:55:31.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.179+0000 7f2366d9d700 1 -- 172.21.15.162:0/3492949986 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2368060330 con 0x7f2368060a70 2020-11-28T21:55:31.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.179+0000 7f2366d9d700 1 --2- 172.21.15.162:0/3492949986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2368060a70 0x7f23680afdc0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 rx=0x7f23600032b0 tx=0x7f23600091a0).ready entity=mon.1 client_cookie=a222aa219b1705af server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:31.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.179+0000 7f236559a700 1 -- 172.21.15.162:0/3492949986 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f236000ba50 con 0x7f2368060a70 2020-11-28T21:55:31.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.179+0000 7f236559a700 1 -- 172.21.15.162:0/3492949986 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236000bbb0 con 0x7f2368060a70 2020-11-28T21:55:31.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.179+0000 7f236559a700 1 -- 172.21.15.162:0/3492949986 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f236000bd20 con 0x7f2368060a70 2020-11-28T21:55:31.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.179+0000 7f236759e700 1 -- 172.21.15.162:0/3492949986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2368060a70 msgr2=0x7f23680afdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:31.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.179+0000 7f236759e700 1 --2- 172.21.15.162:0/3492949986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2368060a70 0x7f23680afdc0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 rx=0x7f23600032b0 tx=0x7f23600091a0).stop 2020-11-28T21:55:31.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.179+0000 7f236759e700 1 -- 172.21.15.162:0/3492949986 shutdown_connections 2020-11-28T21:55:31.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.179+0000 7f236759e700 1 --2- 172.21.15.162:0/3492949986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2368060a70 0x7f23680afdc0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:31.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.179+0000 7f236759e700 1 --2- 172.21.15.162:0/3492949986 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23680b1eb0 0x7f23680b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:31.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.179+0000 7f236759e700 1 -- 172.21.15.162:0/3492949986 >> 172.21.15.162:0/3492949986 conn(0x7f23680b57d0 msgr2=0x7f23680b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:31.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.179+0000 7f236759e700 1 -- 172.21.15.162:0/3492949986 shutdown_connections 2020-11-28T21:55:31.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.179+0000 7f236759e700 1 -- 172.21.15.162:0/3492949986 wait complete. 2020-11-28T21:55:31.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.179+0000 7f236759e700 1 Processor -- start 2020-11-28T21:55:31.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.179+0000 7f236759e700 1 -- start start 2020-11-28T21:55:31.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.183+0000 7f236759e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2368060a70 0x7f236809ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:31.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.183+0000 7f236759e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23680af6a0 0x7f2368066ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:31.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.183+0000 7f236759e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23680b1eb0 0x7f23680659e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:31.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.183+0000 7f236759e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236809e420 con 0x7f2368060a70 2020-11-28T21:55:31.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.183+0000 7f236759e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236809e6c0 con 0x7f23680af6a0 2020-11-28T21:55:31.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.183+0000 7f236759e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f236809e960 con 0x7f23680b1eb0 2020-11-28T21:55:31.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.183+0000 7f2365d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23680af6a0 0x7f2368066ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:31.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.183+0000 7f2365d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23680af6a0 0x7f2368066ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44884/0 (socket says 172.21.15.162:44884) 2020-11-28T21:55:31.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.183+0000 7f2365d9b700 1 -- 172.21.15.162:0/3153765702 learned_addr learned my addr 172.21.15.162:0/3153765702 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:31.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.183+0000 7f236659c700 1 --2- 172.21.15.162:0/3153765702 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2368060a70 0x7f236809ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:31.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.183+0000 7f2366d9d700 1 --2- 172.21.15.162:0/3153765702 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23680b1eb0 0x7f23680659e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:31.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.183+0000 7f2365d9b700 1 -- 172.21.15.162:0/3153765702 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23680b1eb0 msgr2=0x7f23680659e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:31.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.183+0000 7f2365d9b700 1 --2- 172.21.15.162:0/3153765702 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23680b1eb0 0x7f23680659e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:31.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.183+0000 7f2365d9b700 1 -- 172.21.15.162:0/3153765702 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2368060a70 msgr2=0x7f236809ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:31.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.183+0000 7f2365d9b700 1 --2- 172.21.15.162:0/3153765702 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2368060a70 0x7f236809ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:31.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.183+0000 7f2365d9b700 1 -- 172.21.15.162:0/3153765702 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2368066360 con 0x7f23680af6a0 2020-11-28T21:55:31.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.183+0000 7f2365d9b700 1 --2- 172.21.15.162:0/3153765702 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23680af6a0 0x7f2368066ce0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 rx=0x7f2358000e90 tx=0x7f2358007a50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:31.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.183+0000 7f235f7fe700 1 -- 172.21.15.162:0/3153765702 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23580072e0 con 0x7f23680af6a0 2020-11-28T21:55:31.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.183+0000 7f235f7fe700 1 -- 172.21.15.162:0/3153765702 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2358007440 con 0x7f23680af6a0 2020-11-28T21:55:31.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.183+0000 7f236759e700 1 -- 172.21.15.162:0/3153765702 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23680a0f00 con 0x7f23680af6a0 2020-11-28T21:55:31.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.183+0000 7f235f7fe700 1 -- 172.21.15.162:0/3153765702 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23580057d0 con 0x7f23680af6a0 2020-11-28T21:55:31.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.183+0000 7f236759e700 1 -- 172.21.15.162:0/3153765702 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23680660f0 con 0x7f23680af6a0 2020-11-28T21:55:31.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.187+0000 7f236759e700 1 -- 172.21.15.162:0/3153765702 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2368057050 con 0x7f23680af6a0 2020-11-28T21:55:31.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.187+0000 7f235f7fe700 1 -- 172.21.15.162:0/3153765702 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2358010070 con 0x7f23680af6a0 2020-11-28T21:55:31.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.187+0000 7f235f7fe700 1 --2- 172.21.15.162:0/3153765702 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f234c0554e0 0x7f234c0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:31.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.187+0000 7f235f7fe700 1 -- 172.21.15.162:0/3153765702 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5957+0+0 (secure 0 0 0) 0x7f235806d670 con 0x7f23680af6a0 2020-11-28T21:55:31.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.187+0000 7f236659c700 1 --2- 172.21.15.162:0/3153765702 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f234c0554e0 0x7f234c0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:31.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.187+0000 7f236659c700 1 --2- 172.21.15.162:0/3153765702 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f234c0554e0 0x7f234c0579a0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f235000c000 tx=0x7f2350008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:31.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.191+0000 7f235f7fe700 1 -- 172.21.15.162:0/3153765702 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f23580438d0 con 0x7f23680af6a0 2020-11-28T21:55:31.378 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.375+0000 7f236759e700 1 -- 172.21.15.162:0/3153765702 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f23680a89f0 con 0x7f23680af6a0 2020-11-28T21:55:31.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.387+0000 7f235f7fe700 1 -- 172.21.15.162:0/3153765702 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 25 and data pool 26 v66) v1 ==== 189+0+0 (secure 0 0 0) 0x7f2358046ef0 con 0x7f23680af6a0 2020-11-28T21:55:31.395 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.391+0000 7f236759e700 1 -- 172.21.15.162:0/3153765702 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f234c0554e0 msgr2=0x7f234c0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:31.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.391+0000 7f236759e700 1 --2- 172.21.15.162:0/3153765702 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f234c0554e0 0x7f234c0579a0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f235000c000 tx=0x7f2350008040).stop 2020-11-28T21:55:31.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.391+0000 7f236759e700 1 -- 172.21.15.162:0/3153765702 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23680af6a0 msgr2=0x7f2368066ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:31.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.391+0000 7f236759e700 1 --2- 172.21.15.162:0/3153765702 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23680af6a0 0x7f2368066ce0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 rx=0x7f2358000e90 tx=0x7f2358007a50).stop 2020-11-28T21:55:31.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.391+0000 7f236759e700 1 -- 172.21.15.162:0/3153765702 shutdown_connections 2020-11-28T21:55:31.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.391+0000 7f236759e700 1 --2- 172.21.15.162:0/3153765702 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f234c0554e0 0x7f234c0579a0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:31.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.391+0000 7f236759e700 1 --2- 172.21.15.162:0/3153765702 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23680b1eb0 0x7f23680659e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:31.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.391+0000 7f236759e700 1 --2- 172.21.15.162:0/3153765702 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2368060a70 0x7f236809ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:31.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.391+0000 7f236759e700 1 --2- 172.21.15.162:0/3153765702 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23680af6a0 0x7f2368066ce0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:31.398 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.391+0000 7f236759e700 1 -- 172.21.15.162:0/3153765702 >> 172.21.15.162:0/3153765702 conn(0x7f23680b57d0 msgr2=0x7f23680b3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:31.398 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.391+0000 7f236759e700 1 -- 172.21.15.162:0/3153765702 shutdown_connections 2020-11-28T21:55:31.399 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.391+0000 7f236759e700 1 -- 172.21.15.162:0/3153765702 wait complete. 2020-11-28T21:55:31.403 INFO:teuthology.orchestra.run.smithi162.stderr:new fs with metadata pool 25 and data pool 26 2020-11-28T21:55:31.415 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:55:31.546 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.543+0000 7ff00ead4700 1 Processor -- start 2020-11-28T21:55:31.546 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.543+0000 7ff00ead4700 1 -- start start 2020-11-28T21:55:31.548 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.543+0000 7ff00ead4700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff008140350 0x7ff0080b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:31.548 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.543+0000 7ff00ead4700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0080b4930 con 0x7ff008060d10 2020-11-28T21:55:31.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.543+0000 7ff00ead4700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0080b2170 con 0x7ff0080b60e0 2020-11-28T21:55:31.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.543+0000 7ff00ead4700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0080b6660 con 0x7ff008140350 2020-11-28T21:55:31.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.543+0000 7ff00d071700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff008140350 0x7ff0080b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:31.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.543+0000 7ff007fff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7ff0080b60e0 0x7ff00813fc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51748/0 (socket says 172.21.15.162:51748) 2020-11-28T21:55:31.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.543+0000 7ff00d071700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff008140350 0x7ff0080b52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44906/0 (socket says 172.21.15.162:44906) 2020-11-28T21:55:31.550 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.543+0000 7ff007fff700 1 -- 172.21.15.162:0/2690446943 learned_addr learned my addr 172.21.15.162:0/2690446943 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:31.550 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.543+0000 7ff00d071700 1 -- 172.21.15.162:0/2690446943 >> v1:172.21.15.162:6789/0 conn(0x7ff0080b60e0 legacy=0x7ff00813fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:31.550 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.543+0000 7ff00d071700 1 -- 172.21.15.162:0/2690446943 >> v1:172.21.15.118:6789/0 conn(0x7ff008060d10 legacy=0x7ff008146180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:31.550 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.543+0000 7ff00d071700 1 -- 172.21.15.162:0/2690446943 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0080b4b60 con 0x7ff008140350 2020-11-28T21:55:31.550 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.543+0000 7ff00d071700 1 --2- 172.21.15.162:0/2690446943 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff008140350 0x7ff0080b52a0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 rx=0x7feff8002f30 tx=0x7feff8006dc0).ready entity=mon.1 client_cookie=6108741bdfdd4e39 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:31.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.543+0000 7ff0077fe700 1 -- 172.21.15.162:0/2690446943 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feff80097f0 con 0x7ff008140350 2020-11-28T21:55:31.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.543+0000 7ff0077fe700 1 -- 172.21.15.162:0/2690446943 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff8009950 con 0x7ff008140350 2020-11-28T21:55:31.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.543+0000 7ff0077fe700 1 -- 172.21.15.162:0/2690446943 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feff8009cb0 con 0x7ff008140350 2020-11-28T21:55:31.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.543+0000 7ff00ead4700 1 -- 172.21.15.162:0/2690446943 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff008140350 msgr2=0x7ff0080b52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:31.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.543+0000 7ff00ead4700 1 --2- 172.21.15.162:0/2690446943 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff008140350 0x7ff0080b52a0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 rx=0x7feff8002f30 tx=0x7feff8006dc0).stop 2020-11-28T21:55:31.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.543+0000 7ff00ead4700 1 -- 172.21.15.162:0/2690446943 shutdown_connections 2020-11-28T21:55:31.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.543+0000 7ff00ead4700 1 --2- 172.21.15.162:0/2690446943 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff008140350 0x7ff0080b52a0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:31.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.543+0000 7ff00ead4700 1 -- 172.21.15.162:0/2690446943 >> 172.21.15.162:0/2690446943 conn(0x7ff00805fe70 msgr2=0x7ff00806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:31.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.543+0000 7ff00ead4700 1 -- 172.21.15.162:0/2690446943 shutdown_connections 2020-11-28T21:55:31.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.543+0000 7ff00ead4700 1 -- 172.21.15.162:0/2690446943 wait complete. 2020-11-28T21:55:31.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.547+0000 7ff00ead4700 1 Processor -- start 2020-11-28T21:55:31.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.547+0000 7ff00ead4700 1 -- start start 2020-11-28T21:55:31.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.551+0000 7ff00ead4700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff008060d10 0x7ff0080aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:31.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.551+0000 7ff00ead4700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff0080b60e0 0x7ff008147240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:31.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.551+0000 7ff00ead4700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff008140350 0x7ff0080b7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:31.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.551+0000 7ff00ead4700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0080ba310 con 0x7ff008140350 2020-11-28T21:55:31.557 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.551+0000 7ff00ead4700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0080b9930 con 0x7ff008060d10 2020-11-28T21:55:31.557 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.551+0000 7ff00ead4700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0080b9bd0 con 0x7ff0080b60e0 2020-11-28T21:55:31.557 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.551+0000 7ff00c870700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff008060d10 0x7ff0080aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:31.557 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.551+0000 7ff00c870700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff008060d10 0x7ff0080aeaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44908/0 (socket says 172.21.15.162:44908) 2020-11-28T21:55:31.557 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.551+0000 7ff00c870700 1 -- 172.21.15.162:0/3936163726 learned_addr learned my addr 172.21.15.162:0/3936163726 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:31.558 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.551+0000 7ff007fff700 1 --2- 172.21.15.162:0/3936163726 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff0080b60e0 0x7ff008147240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:31.558 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.551+0000 7ff00d071700 1 --2- 172.21.15.162:0/3936163726 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff008140350 0x7ff0080b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:31.558 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.551+0000 7ff00c870700 1 -- 172.21.15.162:0/3936163726 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff0080b60e0 msgr2=0x7ff008147240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:31.558 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.551+0000 7ff00c870700 1 --2- 172.21.15.162:0/3936163726 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff0080b60e0 0x7ff008147240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:31.558 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.551+0000 7ff00c870700 1 -- 172.21.15.162:0/3936163726 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff008140350 msgr2=0x7ff0080b7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:31.559 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.551+0000 7ff00c870700 1 --2- 172.21.15.162:0/3936163726 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff008140350 0x7ff0080b7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:31.559 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.551+0000 7ff00c870700 1 -- 172.21.15.162:0/3936163726 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0080b9e70 con 0x7ff008060d10 2020-11-28T21:55:31.559 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.551+0000 7ff00d071700 1 --2- 172.21.15.162:0/3936163726 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff008140350 0x7ff0080b7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:55:31.559 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.551+0000 7ff007fff700 1 --2- 172.21.15.162:0/3936163726 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff0080b60e0 0x7ff008147240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:55:31.560 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.551+0000 7ff00c870700 1 --2- 172.21.15.162:0/3936163726 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff008060d10 0x7ff0080aeaa0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 rx=0x7ff008066d90 tx=0x7feff0007f60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:31.560 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.551+0000 7ff005ffb700 1 -- 172.21.15.162:0/3936163726 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feff0006a70 con 0x7ff008060d10 2020-11-28T21:55:31.560 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.551+0000 7ff005ffb700 1 -- 172.21.15.162:0/3936163726 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff0006bd0 con 0x7ff008060d10 2020-11-28T21:55:31.560 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.551+0000 7ff00ead4700 1 -- 172.21.15.162:0/3936163726 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff008147730 con 0x7ff008060d10 2020-11-28T21:55:31.561 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.551+0000 7ff005ffb700 1 -- 172.21.15.162:0/3936163726 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feff0007030 con 0x7ff008060d10 2020-11-28T21:55:31.561 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.551+0000 7ff00ead4700 1 -- 172.21.15.162:0/3936163726 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0080b72d0 con 0x7ff008060d10 2020-11-28T21:55:31.562 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.551+0000 7ff00ead4700 1 -- 172.21.15.162:0/3936163726 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff008057050 con 0x7ff008060d10 2020-11-28T21:55:31.562 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.551+0000 7ff005ffb700 1 -- 172.21.15.162:0/3936163726 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7feff0010030 con 0x7ff008060d10 2020-11-28T21:55:31.562 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.555+0000 7ff005ffb700 1 --2- 172.21.15.162:0/3936163726 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fefe00554e0 0x7fefe00579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:31.562 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.555+0000 7ff005ffb700 1 -- 172.21.15.162:0/3936163726 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6073+0+0 (secure 0 0 0) 0x7feff006e060 con 0x7ff008060d10 2020-11-28T21:55:31.563 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.555+0000 7ff007fff700 1 --2- 172.21.15.162:0/3936163726 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fefe00554e0 0x7fefe00579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:31.563 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.555+0000 7ff007fff700 1 --2- 172.21.15.162:0/3936163726 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fefe00554e0 0x7fefe00579a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7ff00000b000 tx=0x7ff000009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:31.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.559+0000 7ff005ffb700 1 -- 172.21.15.162:0/3936163726 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7feff0044360 con 0x7ff008060d10 2020-11-28T21:55:31.704 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.699+0000 7ff00ead4700 1 -- 172.21.15.162:0/3936163726 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff0080ba000 con 0x7ff008060d10 2020-11-28T21:55:31.704 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.699+0000 7ff005ffb700 1 -- 172.21.15.162:0/3936163726 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+16161 (secure 0 0 0) 0x7feff0047980 con 0x7ff008060d10 2020-11-28T21:55:31.705 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:31.705 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":137,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:55:31.379479+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:55:30.002238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2020-11-28T21:55:30.389418+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:55:31.706 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.703+0000 7ff00ead4700 1 -- 172.21.15.162:0/3936163726 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fefe00554e0 msgr2=0x7fefe00579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:31.706 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.703+0000 7ff00ead4700 1 --2- 172.21.15.162:0/3936163726 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fefe00554e0 0x7fefe00579a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7ff00000b000 tx=0x7ff000009040).stop 2020-11-28T21:55:31.707 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.703+0000 7ff00ead4700 1 -- 172.21.15.162:0/3936163726 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff008060d10 msgr2=0x7ff0080aeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:31.707 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.703+0000 7ff00ead4700 1 --2- 172.21.15.162:0/3936163726 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff008060d10 0x7ff0080aeaa0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 rx=0x7ff008066d90 tx=0x7feff0007f60).stop 2020-11-28T21:55:31.707 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.703+0000 7ff00ead4700 1 -- 172.21.15.162:0/3936163726 shutdown_connections 2020-11-28T21:55:31.707 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.703+0000 7ff00ead4700 1 --2- 172.21.15.162:0/3936163726 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fefe00554e0 0x7fefe00579a0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:31.707 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.703+0000 7ff00ead4700 1 --2- 172.21.15.162:0/3936163726 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff008140350 0x7ff0080b7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:31.708 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.703+0000 7ff00ead4700 1 --2- 172.21.15.162:0/3936163726 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff008060d10 0x7ff0080aeaa0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:31.708 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.703+0000 7ff00ead4700 1 --2- 172.21.15.162:0/3936163726 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff0080b60e0 0x7ff008147240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:31.708 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.703+0000 7ff00ead4700 1 -- 172.21.15.162:0/3936163726 >> 172.21.15.162:0/3936163726 conn(0x7ff00805fe70 msgr2=0x7ff008148400 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:31.708 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.703+0000 7ff00ead4700 1 -- 172.21.15.162:0/3936163726 shutdown_connections 2020-11-28T21:55:31.708 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.703+0000 7ff00ead4700 1 -- 172.21.15.162:0/3936163726 wait complete. 2020-11-28T21:55:31.718 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:55:31.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.835+0000 7f8e56e17700 1 Processor -- start 2020-11-28T21:55:31.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.835+0000 7f8e56e17700 1 -- start start 2020-11-28T21:55:31.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.839+0000 7f8e56e17700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8e500b19f0 0x7f8e500b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:31.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.839+0000 7f8e56e17700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8e500b1f00 0x7f8e500b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:31.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.839+0000 7f8e56e17700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e500b7700 con 0x7f8e500b7320 2020-11-28T21:55:31.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.839+0000 7f8e56e17700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e50067b50 con 0x7f8e500b1f00 2020-11-28T21:55:31.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.839+0000 7f8e56e17700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e50067df0 con 0x7f8e500b19f0 2020-11-28T21:55:31.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.839+0000 7f8e553b4700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8e500b1f00 0x7f8e500b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:31.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.839+0000 7f8e553b4700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8e500b1f00 0x7f8e500b2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44920/0 (socket says 172.21.15.162:44920) 2020-11-28T21:55:31.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.839+0000 7f8e553b4700 1 -- 172.21.15.162:0/4196284483 learned_addr learned my addr 172.21.15.162:0/4196284483 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:31.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.839+0000 7f8e4ffff700 1 --2- 172.21.15.162:0/4196284483 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8e500b19f0 0x7f8e500b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:31.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.839+0000 7f8e553b4700 1 -- 172.21.15.162:0/4196284483 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8e500b19f0 msgr2=0x7f8e500b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:31.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.839+0000 7f8e553b4700 1 --2- 172.21.15.162:0/4196284483 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8e500b19f0 0x7f8e500b27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:31.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.839+0000 7f8e553b4700 1 -- 172.21.15.162:0/4196284483 >> v1:172.21.15.118:6789/0 conn(0x7f8e500b7320 legacy=0x7f8e5013d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:31.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.839+0000 7f8e553b4700 1 -- 172.21.15.162:0/4196284483 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e50065c80 con 0x7f8e500b1f00 2020-11-28T21:55:31.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.839+0000 7f8e553b4700 1 --2- 172.21.15.162:0/4196284483 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e500b1f00 0x7f8e500b2ca0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 rx=0x7f8e44002f30 tx=0x7f8e44006e80).ready entity=mon.1 client_cookie=c3296ce29cb4b9a6 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:31.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.839+0000 7f8e4f7fe700 1 -- 172.21.15.162:0/4196284483 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e440097f0 con 0x7f8e500b1f00 2020-11-28T21:55:31.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.839+0000 7f8e4f7fe700 1 -- 172.21.15.162:0/4196284483 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e44009950 con 0x7f8e500b1f00 2020-11-28T21:55:31.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.839+0000 7f8e4f7fe700 1 -- 172.21.15.162:0/4196284483 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e44009cb0 con 0x7f8e500b1f00 2020-11-28T21:55:31.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.839+0000 7f8e56e17700 1 -- 172.21.15.162:0/4196284483 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e500b1f00 msgr2=0x7f8e500b2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:31.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.839+0000 7f8e56e17700 1 --2- 172.21.15.162:0/4196284483 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e500b1f00 0x7f8e500b2ca0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 rx=0x7f8e44002f30 tx=0x7f8e44006e80).stop 2020-11-28T21:55:31.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.839+0000 7f8e56e17700 1 -- 172.21.15.162:0/4196284483 shutdown_connections 2020-11-28T21:55:31.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.839+0000 7f8e56e17700 1 --2- 172.21.15.162:0/4196284483 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e500b1f00 0x7f8e500b2ca0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:31.852 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.839+0000 7f8e56e17700 1 --2- 172.21.15.162:0/4196284483 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8e500b19f0 0x7f8e500b27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:31.852 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.839+0000 7f8e56e17700 1 -- 172.21.15.162:0/4196284483 >> 172.21.15.162:0/4196284483 conn(0x7f8e50062830 msgr2=0x7f8e50062c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:31.852 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.839+0000 7f8e56e17700 1 -- 172.21.15.162:0/4196284483 shutdown_connections 2020-11-28T21:55:31.852 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.839+0000 7f8e56e17700 1 -- 172.21.15.162:0/4196284483 wait complete. 2020-11-28T21:55:31.858 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.843+0000 7f8e56e17700 1 Processor -- start 2020-11-28T21:55:31.858 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.843+0000 7f8e56e17700 1 -- start start 2020-11-28T21:55:31.858 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.843+0000 7f8e56e17700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8e500b19f0 0x7f8e50147ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:31.858 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.843+0000 7f8e56e17700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e500b1f00 0x7f8e501446d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:31.859 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.843+0000 7f8e56e17700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8e500b7320 0x7f8e50141020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:31.859 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.843+0000 7f8e56e17700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e500b0870 con 0x7f8e500b19f0 2020-11-28T21:55:31.859 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.843+0000 7f8e56e17700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e50142fa0 con 0x7f8e500b1f00 2020-11-28T21:55:31.859 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.843+0000 7f8e56e17700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e50143240 con 0x7f8e500b7320 2020-11-28T21:55:31.859 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.843+0000 7f8e4ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e500b1f00 0x7f8e501446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:31.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.843+0000 7f8e54bb3700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8e500b19f0 0x7f8e50147ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:31.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.843+0000 7f8e54bb3700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8e500b19f0 0x7f8e50147ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.162:51910/0 (socket says 172.21.15.162:51910) 2020-11-28T21:55:31.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.843+0000 7f8e54bb3700 1 -- 172.21.15.162:0/1571892681 learned_addr learned my addr 172.21.15.162:0/1571892681 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:31.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.843+0000 7f8e553b4700 1 --2- 172.21.15.162:0/1571892681 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8e500b7320 0x7f8e50141020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:31.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.843+0000 7f8e4ffff700 1 -- 172.21.15.162:0/1571892681 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8e500b7320 msgr2=0x7f8e50141020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:31.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.843+0000 7f8e4ffff700 1 --2- 172.21.15.162:0/1571892681 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8e500b7320 0x7f8e50141020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:31.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.843+0000 7f8e4ffff700 1 -- 172.21.15.162:0/1571892681 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8e500b19f0 msgr2=0x7f8e50147ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:31.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.843+0000 7f8e4ffff700 1 --2- 172.21.15.162:0/1571892681 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8e500b19f0 0x7f8e50147ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:31.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.843+0000 7f8e4ffff700 1 -- 172.21.15.162:0/1571892681 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e500b05d0 con 0x7f8e500b1f00 2020-11-28T21:55:31.862 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.843+0000 7f8e54bb3700 1 --2- 172.21.15.162:0/1571892681 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8e500b19f0 0x7f8e50147ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:55:31.862 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.843+0000 7f8e4ffff700 1 --2- 172.21.15.162:0/1571892681 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e500b1f00 0x7f8e501446d0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 rx=0x7f8e4000c000 tx=0x7f8e40006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:31.862 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.843+0000 7f8e4dffb700 1 -- 172.21.15.162:0/1571892681 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e4000ca10 con 0x7f8e500b1f00 2020-11-28T21:55:31.862 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.843+0000 7f8e4dffb700 1 -- 172.21.15.162:0/1571892681 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e4000cb70 con 0x7f8e500b1f00 2020-11-28T21:55:31.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.843+0000 7f8e56e17700 1 -- 172.21.15.162:0/1571892681 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e50144bc0 con 0x7f8e500b1f00 2020-11-28T21:55:31.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.843+0000 7f8e4dffb700 1 -- 172.21.15.162:0/1571892681 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e4000be40 con 0x7f8e500b1f00 2020-11-28T21:55:31.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.843+0000 7f8e56e17700 1 -- 172.21.15.162:0/1571892681 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e500b0f80 con 0x7f8e500b1f00 2020-11-28T21:55:31.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.847+0000 7f8e56e17700 1 -- 172.21.15.162:0/1571892681 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e2c0033e0 con 0x7f8e500b1f00 2020-11-28T21:55:31.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.847+0000 7f8e4dffb700 1 -- 172.21.15.162:0/1571892681 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8e4001c460 con 0x7f8e500b1f00 2020-11-28T21:55:31.864 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.847+0000 7f8e4dffb700 1 --2- 172.21.15.162:0/1571892681 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8e30055440 0x7f8e30057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:31.864 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.847+0000 7f8e4dffb700 1 -- 172.21.15.162:0/1571892681 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8e4006e1f0 con 0x7f8e500b1f00 2020-11-28T21:55:31.864 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.847+0000 7f8e54bb3700 1 --2- 172.21.15.162:0/1571892681 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8e30055440 0x7f8e30057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:31.864 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.847+0000 7f8e54bb3700 1 --2- 172.21.15.162:0/1571892681 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8e30055440 0x7f8e30057900 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f8e4800d000 tx=0x7f8e48009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:31.864 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.851+0000 7f8e4dffb700 1 -- 172.21.15.162:0/1571892681 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8e40044490 con 0x7f8e500b1f00 2020-11-28T21:55:31.997 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.991+0000 7f8e56e17700 1 -- 172.21.15.162:0/1571892681 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8e2c002270 con 0x7f8e500b1f00 2020-11-28T21:55:31.998 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.995+0000 7f8e4dffb700 1 -- 172.21.15.162:0/1571892681 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+16161 (secure 0 0 0) 0x7f8e40047ab0 con 0x7f8e500b1f00 2020-11-28T21:55:31.999 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:31.999 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":137,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:55:31.379479+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:55:30.002238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2020-11-28T21:55:30.389418+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:55:32.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.995+0000 7f8e56e17700 1 -- 172.21.15.162:0/1571892681 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8e30055440 msgr2=0x7f8e30057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:32.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.995+0000 7f8e56e17700 1 --2- 172.21.15.162:0/1571892681 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8e30055440 0x7f8e30057900 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f8e4800d000 tx=0x7f8e48009040).stop 2020-11-28T21:55:32.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.995+0000 7f8e56e17700 1 -- 172.21.15.162:0/1571892681 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e500b1f00 msgr2=0x7f8e501446d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:32.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.995+0000 7f8e56e17700 1 --2- 172.21.15.162:0/1571892681 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e500b1f00 0x7f8e501446d0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 rx=0x7f8e4000c000 tx=0x7f8e40006dc0).stop 2020-11-28T21:55:32.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.995+0000 7f8e56e17700 1 -- 172.21.15.162:0/1571892681 shutdown_connections 2020-11-28T21:55:32.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.995+0000 7f8e56e17700 1 --2- 172.21.15.162:0/1571892681 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8e30055440 0x7f8e30057900 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:32.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.995+0000 7f8e56e17700 1 --2- 172.21.15.162:0/1571892681 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8e500b7320 0x7f8e50141020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:32.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.995+0000 7f8e56e17700 1 --2- 172.21.15.162:0/1571892681 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8e500b19f0 0x7f8e50147ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:32.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.995+0000 7f8e56e17700 1 --2- 172.21.15.162:0/1571892681 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e500b1f00 0x7f8e501446d0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:32.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.995+0000 7f8e56e17700 1 -- 172.21.15.162:0/1571892681 >> 172.21.15.162:0/1571892681 conn(0x7f8e50062830 msgr2=0x7f8e5013c510 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:32.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.995+0000 7f8e56e17700 1 -- 172.21.15.162:0/1571892681 shutdown_connections 2020-11-28T21:55:32.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:31.995+0000 7f8e56e17700 1 -- 172.21.15.162:0/1571892681 wait complete. 2020-11-28T21:55:32.013 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2020-11-28T21:55:32.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.123+0000 7f612f2fa700 1 Processor -- start 2020-11-28T21:55:32.129 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.123+0000 7f612f2fa700 1 -- start start 2020-11-28T21:55:32.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.127+0000 7f612f2fa700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f61281433c0 0x7f61280605c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:32.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.127+0000 7f612f2fa700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6128067610 0x7f612813fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:32.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.127+0000 7f612f2fa700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6128143950 con 0x7f61280b9ce0 2020-11-28T21:55:32.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.127+0000 7f612f2fa700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61280b4340 con 0x7f61281433c0 2020-11-28T21:55:32.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.127+0000 7f612f2fa700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6128067ac0 con 0x7f6128067610 2020-11-28T21:55:32.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.127+0000 7f612c895700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f61281433c0 0x7f61280605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:32.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.127+0000 7f612c895700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f61281433c0 0x7f61280605c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44932/0 (socket says 172.21.15.162:44932) 2020-11-28T21:55:32.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.127+0000 7f612c895700 1 -- 172.21.15.162:0/1625793280 learned_addr learned my addr 172.21.15.162:0/1625793280 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:32.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.127+0000 7f612d897700 1 --2- 172.21.15.162:0/1625793280 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6128067610 0x7f612813fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:32.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.127+0000 7f612c895700 1 -- 172.21.15.162:0/1625793280 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6128067610 msgr2=0x7f612813fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:32.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.127+0000 7f612c895700 1 --2- 172.21.15.162:0/1625793280 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6128067610 0x7f612813fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:32.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.127+0000 7f612c895700 1 -- 172.21.15.162:0/1625793280 >> v1:172.21.15.118:6789/0 conn(0x7f61280b9ce0 legacy=0x7f61280b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:32.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.127+0000 7f612c895700 1 -- 172.21.15.162:0/1625793280 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61280ba0c0 con 0x7f61281433c0 2020-11-28T21:55:32.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.127+0000 7f612c895700 1 --2- 172.21.15.162:0/1625793280 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61281433c0 0x7f61280605c0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 rx=0x7f611c002f30 tx=0x7f611c006e80).ready entity=mon.1 client_cookie=d19ac7054a98bf95 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:32.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.127+0000 7f611bfff700 1 -- 172.21.15.162:0/1625793280 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f611c0097f0 con 0x7f61281433c0 2020-11-28T21:55:32.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.127+0000 7f611bfff700 1 -- 172.21.15.162:0/1625793280 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f611c009950 con 0x7f61281433c0 2020-11-28T21:55:32.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.127+0000 7f611bfff700 1 -- 172.21.15.162:0/1625793280 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f611c009cb0 con 0x7f61281433c0 2020-11-28T21:55:32.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.127+0000 7f612f2fa700 1 -- 172.21.15.162:0/1625793280 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61281433c0 msgr2=0x7f61280605c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:32.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.127+0000 7f612f2fa700 1 --2- 172.21.15.162:0/1625793280 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61281433c0 0x7f61280605c0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 rx=0x7f611c002f30 tx=0x7f611c006e80).stop 2020-11-28T21:55:32.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.127+0000 7f612f2fa700 1 -- 172.21.15.162:0/1625793280 shutdown_connections 2020-11-28T21:55:32.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.127+0000 7f612f2fa700 1 --2- 172.21.15.162:0/1625793280 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6128067610 0x7f612813fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:32.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.127+0000 7f612f2fa700 1 --2- 172.21.15.162:0/1625793280 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61281433c0 0x7f61280605c0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:32.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.127+0000 7f612f2fa700 1 -- 172.21.15.162:0/1625793280 >> 172.21.15.162:0/1625793280 conn(0x7f612805c430 msgr2=0x7f61280b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:32.138 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.127+0000 7f612f2fa700 1 -- 172.21.15.162:0/1625793280 shutdown_connections 2020-11-28T21:55:32.138 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.127+0000 7f612f2fa700 1 -- 172.21.15.162:0/1625793280 wait complete. 2020-11-28T21:55:32.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.127+0000 7f612f2fa700 1 Processor -- start 2020-11-28T21:55:32.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.127+0000 7f612f2fa700 1 -- start start 2020-11-28T21:55:32.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.131+0000 7f612f2fa700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6128067610 0x7f61280b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:32.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.131+0000 7f612f2fa700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61280b9ce0 0x7f61281543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:32.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.131+0000 7f612f2fa700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f61281433c0 0x7f61281548a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:32.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.131+0000 7f612f2fa700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61280b6610 con 0x7f61281433c0 2020-11-28T21:55:32.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.131+0000 7f612f2fa700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6128152ee0 con 0x7f61280b9ce0 2020-11-28T21:55:32.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.131+0000 7f612f2fa700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6128154d90 con 0x7f6128067610 2020-11-28T21:55:32.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.131+0000 7f612c895700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61280b9ce0 0x7f61281543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:32.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.131+0000 7f612c895700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61280b9ce0 0x7f61281543b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44938/0 (socket says 172.21.15.162:44938) 2020-11-28T21:55:32.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.131+0000 7f612c895700 1 -- 172.21.15.162:0/1863334232 learned_addr learned my addr 172.21.15.162:0/1863334232 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:32.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.131+0000 7f612d096700 1 --2- 172.21.15.162:0/1863334232 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6128067610 0x7f61280b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:32.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.131+0000 7f612d897700 1 --2- 172.21.15.162:0/1863334232 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f61281433c0 0x7f61281548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:32.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.131+0000 7f612c895700 1 -- 172.21.15.162:0/1863334232 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6128067610 msgr2=0x7f61280b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:32.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.131+0000 7f612c895700 1 --2- 172.21.15.162:0/1863334232 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6128067610 0x7f61280b57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:32.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.131+0000 7f612c895700 1 -- 172.21.15.162:0/1863334232 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f61281433c0 msgr2=0x7f61281548a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:32.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.131+0000 7f612c895700 1 --2- 172.21.15.162:0/1863334232 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f61281433c0 0x7f61281548a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:32.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.131+0000 7f612c895700 1 -- 172.21.15.162:0/1863334232 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61281525e0 con 0x7f61280b9ce0 2020-11-28T21:55:32.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.131+0000 7f612c895700 1 --2- 172.21.15.162:0/1863334232 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61280b9ce0 0x7f61281543b0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 rx=0x7f6128144870 tx=0x7f611c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:32.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.131+0000 7f611a7fc700 1 -- 172.21.15.162:0/1863334232 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f611c006bb0 con 0x7f61280b9ce0 2020-11-28T21:55:32.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.131+0000 7f611a7fc700 1 -- 172.21.15.162:0/1863334232 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f611c006750 con 0x7f61280b9ce0 2020-11-28T21:55:32.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.131+0000 7f612f2fa700 1 -- 172.21.15.162:0/1863334232 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6128152880 con 0x7f61280b9ce0 2020-11-28T21:55:32.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.131+0000 7f611a7fc700 1 -- 172.21.15.162:0/1863334232 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f611c011cf0 con 0x7f61280b9ce0 2020-11-28T21:55:32.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.131+0000 7f612f2fa700 1 -- 172.21.15.162:0/1863334232 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61280b6290 con 0x7f61280b9ce0 2020-11-28T21:55:32.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.131+0000 7f612f2fa700 1 -- 172.21.15.162:0/1863334232 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6128057050 con 0x7f61280b9ce0 2020-11-28T21:55:32.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.135+0000 7f611a7fc700 1 -- 172.21.15.162:0/1863334232 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f611c025030 con 0x7f61280b9ce0 2020-11-28T21:55:32.151 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.135+0000 7f611a7fc700 1 --2- 172.21.15.162:0/1863334232 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f61100554e0 0x7f61100579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:32.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.135+0000 7f611a7fc700 1 -- 172.21.15.162:0/1863334232 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f611c06f010 con 0x7f61280b9ce0 2020-11-28T21:55:32.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.135+0000 7f612d096700 1 --2- 172.21.15.162:0/1863334232 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f61100554e0 0x7f61100579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:32.152 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.135+0000 7f612d096700 1 --2- 172.21.15.162:0/1863334232 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f61100554e0 0x7f61100579a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7f612400d000 tx=0x7f6124007b10).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:32.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.143+0000 7f611a7fc700 1 -- 172.21.15.162:0/1863334232 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f611c045310 con 0x7f61280b9ce0 2020-11-28T21:55:32.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:32.327+0000 7f612f2fa700 1 -- 172.21.15.162:0/1863334232 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f6128068350 con 0x7f61280b9ce0 2020-11-28T21:55:33.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.047+0000 7f611a7fc700 1 -- 172.21.15.162:0/1863334232 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v68) v1 ==== 118+0+0 (secure 0 0 0) 0x7f611c023070 con 0x7f61280b9ce0 2020-11-28T21:55:33.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.047+0000 7f612f2fa700 1 -- 172.21.15.162:0/1863334232 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f61100554e0 msgr2=0x7f61100579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:33.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.047+0000 7f612f2fa700 1 --2- 172.21.15.162:0/1863334232 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f61100554e0 0x7f61100579a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7f612400d000 tx=0x7f6124007b10).stop 2020-11-28T21:55:33.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.047+0000 7f612f2fa700 1 -- 172.21.15.162:0/1863334232 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61280b9ce0 msgr2=0x7f61281543b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:33.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.047+0000 7f612f2fa700 1 --2- 172.21.15.162:0/1863334232 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61280b9ce0 0x7f61281543b0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 rx=0x7f6128144870 tx=0x7f611c015000).stop 2020-11-28T21:55:33.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.047+0000 7f612f2fa700 1 -- 172.21.15.162:0/1863334232 shutdown_connections 2020-11-28T21:55:33.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.047+0000 7f612f2fa700 1 --2- 172.21.15.162:0/1863334232 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f61100554e0 0x7f61100579a0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:33.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.047+0000 7f612f2fa700 1 --2- 172.21.15.162:0/1863334232 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6128067610 0x7f61280b57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:33.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.047+0000 7f612f2fa700 1 --2- 172.21.15.162:0/1863334232 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f61281433c0 0x7f61281548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:33.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.047+0000 7f612f2fa700 1 --2- 172.21.15.162:0/1863334232 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61280b9ce0 0x7f61281543b0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:33.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.047+0000 7f612f2fa700 1 -- 172.21.15.162:0/1863334232 >> 172.21.15.162:0/1863334232 conn(0x7f612805c430 msgr2=0x7f612809d350 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:33.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.051+0000 7f612f2fa700 1 -- 172.21.15.162:0/1863334232 shutdown_connections 2020-11-28T21:55:33.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.051+0000 7f612f2fa700 1 -- 172.21.15.162:0/1863334232 wait complete. 2020-11-28T21:55:33.070 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:55:33.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.187+0000 7fb3fb3d8700 1 Processor -- start 2020-11-28T21:55:33.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.187+0000 7fb3fb3d8700 1 -- start start 2020-11-28T21:55:33.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.187+0000 7fb3fb3d8700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb3f40994f0 0x7fb3f40b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:33.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.187+0000 7fb3fb3d8700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3f4098f10 con 0x7fb3f4098b30 2020-11-28T21:55:33.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.187+0000 7fb3fb3d8700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3f40b0680 con 0x7fb3f4060710 2020-11-28T21:55:33.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.187+0000 7fb3fb3d8700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3f40b1030 con 0x7fb3f40994f0 2020-11-28T21:55:33.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.187+0000 7fb3f9bd5700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb3f40994f0 0x7fb3f40b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:33.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.187+0000 7fb3f9bd5700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb3f40994f0 0x7fb3f40b0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44948/0 (socket says 172.21.15.162:44948) 2020-11-28T21:55:33.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.187+0000 7fb3fabd7700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fb3f4060710 0x7fb3f40b1580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51790/0 (socket says 172.21.15.162:51790) 2020-11-28T21:55:33.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.187+0000 7fb3f9bd5700 1 -- 172.21.15.162:0/3048333171 learned_addr learned my addr 172.21.15.162:0/3048333171 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:33.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.187+0000 7fb3f9bd5700 1 -- 172.21.15.162:0/3048333171 >> v1:172.21.15.162:6789/0 conn(0x7fb3f4060710 legacy=0x7fb3f40b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:33.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.187+0000 7fb3f9bd5700 1 -- 172.21.15.162:0/3048333171 >> v1:172.21.15.118:6789/0 conn(0x7fb3f4098b30 legacy=0x7fb3f4060000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:33.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.187+0000 7fb3f93d4700 1 -- 172.21.15.162:0/3048333171 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 564098171 0 0) 0x7fb3f40b0680 con 0x7fb3f4060710 2020-11-28T21:55:33.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.187+0000 7fb3f9bd5700 1 -- 172.21.15.162:0/3048333171 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f40b0920 con 0x7fb3f40994f0 2020-11-28T21:55:33.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.187+0000 7fb3f9bd5700 1 --2- 172.21.15.162:0/3048333171 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3f40994f0 0x7fb3f40b0000 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 rx=0x7fb3e8002f30 tx=0x7fb3e8006e80).ready entity=mon.1 client_cookie=fc2b8ad72ca63d98 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:33.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.187+0000 7fb3f93d4700 1 -- 172.21.15.162:0/3048333171 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3e80097f0 con 0x7fb3f40994f0 2020-11-28T21:55:33.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.187+0000 7fb3f93d4700 1 -- 172.21.15.162:0/3048333171 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e8009950 con 0x7fb3f40994f0 2020-11-28T21:55:33.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.187+0000 7fb3f93d4700 1 -- 172.21.15.162:0/3048333171 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3e8009cb0 con 0x7fb3f40994f0 2020-11-28T21:55:33.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.187+0000 7fb3fb3d8700 1 -- 172.21.15.162:0/3048333171 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3f40994f0 msgr2=0x7fb3f40b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:33.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.187+0000 7fb3fb3d8700 1 --2- 172.21.15.162:0/3048333171 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3f40994f0 0x7fb3f40b0000 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 rx=0x7fb3e8002f30 tx=0x7fb3e8006e80).stop 2020-11-28T21:55:33.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.187+0000 7fb3fb3d8700 1 -- 172.21.15.162:0/3048333171 shutdown_connections 2020-11-28T21:55:33.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.187+0000 7fb3fb3d8700 1 --2- 172.21.15.162:0/3048333171 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3f40994f0 0x7fb3f40b0000 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:33.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.187+0000 7fb3fb3d8700 1 -- 172.21.15.162:0/3048333171 >> 172.21.15.162:0/3048333171 conn(0x7fb3f405b970 msgr2=0x7fb3f405bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:33.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.187+0000 7fb3fb3d8700 1 -- 172.21.15.162:0/3048333171 shutdown_connections 2020-11-28T21:55:33.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.191+0000 7fb3fb3d8700 1 -- 172.21.15.162:0/3048333171 wait complete. 2020-11-28T21:55:33.200 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.191+0000 7fb3fb3d8700 1 Processor -- start 2020-11-28T21:55:33.200 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.191+0000 7fb3fb3d8700 1 -- start start 2020-11-28T21:55:33.200 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.191+0000 7fb3fb3d8700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb3f4060710 0x7fb3f40acc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:33.200 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.191+0000 7fb3fb3d8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3f4098b30 0x7fb3f40ad450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:33.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.191+0000 7fb3fb3d8700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb3f40994f0 0x7fb3f41458f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:33.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.191+0000 7fb3fb3d8700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f4149030 con 0x7fb3f40994f0 2020-11-28T21:55:33.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.191+0000 7fb3fb3d8700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f41492d0 con 0x7fb3f4098b30 2020-11-28T21:55:33.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.191+0000 7fb3fb3d8700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f4149570 con 0x7fb3f4060710 2020-11-28T21:55:33.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.191+0000 7fb3f9bd5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3f4098b30 0x7fb3f40ad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:33.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.191+0000 7fb3f9bd5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3f4098b30 0x7fb3f40ad450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44952/0 (socket says 172.21.15.162:44952) 2020-11-28T21:55:33.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.191+0000 7fb3f9bd5700 1 -- 172.21.15.162:0/343625649 learned_addr learned my addr 172.21.15.162:0/343625649 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:33.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.191+0000 7fb3fa3d6700 1 --2- 172.21.15.162:0/343625649 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb3f4060710 0x7fb3f40acc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:33.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.191+0000 7fb3fabd7700 1 --2- 172.21.15.162:0/343625649 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb3f40994f0 0x7fb3f41458f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:33.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.191+0000 7fb3f9bd5700 1 -- 172.21.15.162:0/343625649 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb3f4060710 msgr2=0x7fb3f40acc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:33.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.191+0000 7fb3f9bd5700 1 --2- 172.21.15.162:0/343625649 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb3f4060710 0x7fb3f40acc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:33.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.191+0000 7fb3f9bd5700 1 -- 172.21.15.162:0/343625649 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb3f40994f0 msgr2=0x7fb3f41458f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:33.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.191+0000 7fb3f9bd5700 1 --2- 172.21.15.162:0/343625649 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb3f40994f0 0x7fb3f41458f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:33.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.191+0000 7fb3f9bd5700 1 -- 172.21.15.162:0/343625649 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f40ab930 con 0x7fb3f4098b30 2020-11-28T21:55:33.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.191+0000 7fb3fa3d6700 1 --2- 172.21.15.162:0/343625649 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb3f4060710 0x7fb3f40acc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:55:33.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.191+0000 7fb3f9bd5700 1 --2- 172.21.15.162:0/343625649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3f4098b30 0x7fb3f40ad450 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 rx=0x7fb3e8002f30 tx=0x7fb3e8015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:33.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.195+0000 7fb3f37fe700 1 -- 172.21.15.162:0/343625649 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3e8011d10 con 0x7fb3f4098b30 2020-11-28T21:55:33.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.195+0000 7fb3f37fe700 1 -- 172.21.15.162:0/343625649 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e8011e70 con 0x7fb3f4098b30 2020-11-28T21:55:33.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.195+0000 7fb3f37fe700 1 -- 172.21.15.162:0/343625649 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3e801e650 con 0x7fb3f4098b30 2020-11-28T21:55:33.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.195+0000 7fb3fb3d8700 1 -- 172.21.15.162:0/343625649 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f40aeba0 con 0x7fb3f4098b30 2020-11-28T21:55:33.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.195+0000 7fb3fb3d8700 1 -- 172.21.15.162:0/343625649 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f4146000 con 0x7fb3f4098b30 2020-11-28T21:55:33.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.195+0000 7fb3fb3d8700 1 -- 172.21.15.162:0/343625649 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3f4057050 con 0x7fb3f4098b30 2020-11-28T21:55:33.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.195+0000 7fb3f37fe700 1 -- 172.21.15.162:0/343625649 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb3e801a040 con 0x7fb3f4098b30 2020-11-28T21:55:33.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.195+0000 7fb3f37fe700 1 --2- 172.21.15.162:0/343625649 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb3d40554e0 0x7fb3d40579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:33.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.195+0000 7fb3f37fe700 1 -- 172.21.15.162:0/343625649 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb3e806f150 con 0x7fb3f4098b30 2020-11-28T21:55:33.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.195+0000 7fb3fa3d6700 1 --2- 172.21.15.162:0/343625649 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb3d40554e0 0x7fb3d40579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:33.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.195+0000 7fb3fa3d6700 1 --2- 172.21.15.162:0/343625649 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb3d40554e0 0x7fb3d40579a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7fb3e400b000 tx=0x7fb3e4009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:33.210 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.207+0000 7fb3f37fe700 1 -- 172.21.15.162:0/343625649 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb3e8045450 con 0x7fb3f4098b30 2020-11-28T21:55:33.392 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.387+0000 7fb3fb3d8700 1 -- 172.21.15.162:0/343625649 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3f4067380 con 0x7fb3f4098b30 2020-11-28T21:55:33.393 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.387+0000 7fb3f37fe700 1 -- 172.21.15.162:0/343625649 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+2504 (secure 0 0 0) 0x7fb3e8023020 con 0x7fb3f4098b30 2020-11-28T21:55:33.393 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:33.394 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":68,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7611,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6835/2127102267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2127102267},{"type":"v1","addr":"172.21.15.162:6835","nonce":2127102267}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":66},{"gid":7623,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/1473328171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1473328171},{"type":"v1","addr":"172.21.15.118:6833","nonce":1473328171}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:55:31.379864+0000","modified":"2020-11-28T21:55:33.042143+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7888":{"gid":7888,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/2667777645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2667777645},{"type":"v1","addr":"172.21.15.162:6837","nonce":2667777645}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2020-11-28T21:55:33.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.391+0000 7fb3fb3d8700 1 -- 172.21.15.162:0/343625649 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb3d40554e0 msgr2=0x7fb3d40579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:33.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.391+0000 7fb3fb3d8700 1 --2- 172.21.15.162:0/343625649 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb3d40554e0 0x7fb3d40579a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7fb3e400b000 tx=0x7fb3e4009040).stop 2020-11-28T21:55:33.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.391+0000 7fb3fb3d8700 1 -- 172.21.15.162:0/343625649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3f4098b30 msgr2=0x7fb3f40ad450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:33.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.391+0000 7fb3fb3d8700 1 --2- 172.21.15.162:0/343625649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3f4098b30 0x7fb3f40ad450 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 rx=0x7fb3e8002f30 tx=0x7fb3e8015000).stop 2020-11-28T21:55:33.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.391+0000 7fb3fb3d8700 1 -- 172.21.15.162:0/343625649 shutdown_connections 2020-11-28T21:55:33.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.391+0000 7fb3fb3d8700 1 --2- 172.21.15.162:0/343625649 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb3d40554e0 0x7fb3d40579a0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:33.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.391+0000 7fb3fb3d8700 1 --2- 172.21.15.162:0/343625649 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb3f4060710 0x7fb3f40acc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:33.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.391+0000 7fb3fb3d8700 1 --2- 172.21.15.162:0/343625649 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb3f40994f0 0x7fb3f41458f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:33.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.391+0000 7fb3fb3d8700 1 --2- 172.21.15.162:0/343625649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3f4098b30 0x7fb3f40ad450 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:33.398 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.391+0000 7fb3fb3d8700 1 -- 172.21.15.162:0/343625649 >> 172.21.15.162:0/343625649 conn(0x7fb3f405b970 msgr2=0x7fb3f4149d00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:33.398 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.391+0000 7fb3fb3d8700 1 -- 172.21.15.162:0/343625649 shutdown_connections 2020-11-28T21:55:33.398 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.391+0000 7fb3fb3d8700 1 -- 172.21.15.162:0/343625649 wait complete. 2020-11-28T21:55:33.399 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 68 2020-11-28T21:55:33.411 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:55:33.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.515+0000 7fa029d24700 1 Processor -- start 2020-11-28T21:55:33.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.515+0000 7fa029d24700 1 -- start start 2020-11-28T21:55:33.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.515+0000 7fa029d24700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fa024142a00 0x7fa0240b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:33.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.515+0000 7fa029d24700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0240b4110 con 0x7fa0240603d0 2020-11-28T21:55:33.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.515+0000 7fa029d24700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0240b43f0 con 0x7fa0240b4c70 2020-11-28T21:55:33.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.515+0000 7fa029d24700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa024060f60 con 0x7fa024142a00 2020-11-28T21:55:33.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.515+0000 7fa029523700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fa0240b4c70 0x7fa024143020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51806/0 (socket says 172.21.15.162:51806) 2020-11-28T21:55:33.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.515+0000 7fa029523700 1 -- 172.21.15.162:0/4186175834 learned_addr learned my addr 172.21.15.162:0/4186175834 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:33.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.515+0000 7fa023fff700 1 --2- 172.21.15.162:0/4186175834 >> v2:172.21.15.118:3300/0 conn(0x7fa024142a00 0x7fa0240b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:33.522 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.515+0000 7fa0237fe700 1 -- 172.21.15.162:0/4186175834 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2356802220 0 0) 0x7fa0240b43f0 con 0x7fa0240b4c70 2020-11-28T21:55:33.522 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.515+0000 7fa0237fe700 1 -- 172.21.15.162:0/4186175834 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa010003230 con 0x7fa0240b4c70 2020-11-28T21:55:33.522 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.515+0000 7fa0237fe700 1 -- 172.21.15.162:0/4186175834 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3271189768 0 0) 0x7fa0240b4110 con 0x7fa0240603d0 2020-11-28T21:55:33.522 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.515+0000 7fa0237fe700 1 -- 172.21.15.162:0/4186175834 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0240b43f0 con 0x7fa0240603d0 2020-11-28T21:55:33.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.515+0000 7fa0237fe700 1 -- 172.21.15.162:0/4186175834 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1795335642 0 0) 0x7fa010003230 con 0x7fa0240b4c70 2020-11-28T21:55:33.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.515+0000 7fa023fff700 1 -- 172.21.15.162:0/4186175834 >> v1:172.21.15.162:6789/0 conn(0x7fa0240b4c70 legacy=0x7fa024143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:33.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.515+0000 7fa023fff700 1 -- 172.21.15.162:0/4186175834 >> v1:172.21.15.118:6789/0 conn(0x7fa0240603d0 legacy=0x7fa02409cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:33.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.515+0000 7fa023fff700 1 -- 172.21.15.162:0/4186175834 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0240b4690 con 0x7fa024142a00 2020-11-28T21:55:33.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.515+0000 7fa023fff700 1 --2- 172.21.15.162:0/4186175834 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa024142a00 0x7fa0240b3a70 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7fa01c001490 tx=0x7fa014006dc0).ready entity=mon.0 client_cookie=6791247a35a0abbe server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:33.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.515+0000 7fa0237fe700 1 -- 172.21.15.162:0/4186175834 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa01c001a90 con 0x7fa024142a00 2020-11-28T21:55:33.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.515+0000 7fa0237fe700 1 -- 172.21.15.162:0/4186175834 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa01400d610 con 0x7fa024142a00 2020-11-28T21:55:33.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.515+0000 7fa0237fe700 1 -- 172.21.15.162:0/4186175834 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa01400d780 con 0x7fa024142a00 2020-11-28T21:55:33.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.515+0000 7fa029d24700 1 -- 172.21.15.162:0/4186175834 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa024142a00 msgr2=0x7fa0240b3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:33.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.515+0000 7fa029d24700 1 --2- 172.21.15.162:0/4186175834 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa024142a00 0x7fa0240b3a70 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7fa01c001490 tx=0x7fa014006dc0).stop 2020-11-28T21:55:33.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.515+0000 7fa029d24700 1 -- 172.21.15.162:0/4186175834 shutdown_connections 2020-11-28T21:55:33.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.515+0000 7fa029d24700 1 --2- 172.21.15.162:0/4186175834 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa024142a00 0x7fa0240b3a70 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:33.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.515+0000 7fa029d24700 1 -- 172.21.15.162:0/4186175834 >> 172.21.15.162:0/4186175834 conn(0x7fa024066a40 msgr2=0x7fa024066e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:33.526 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.519+0000 7fa029d24700 1 -- 172.21.15.162:0/4186175834 shutdown_connections 2020-11-28T21:55:33.526 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.519+0000 7fa029d24700 1 -- 172.21.15.162:0/4186175834 wait complete. 2020-11-28T21:55:33.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.519+0000 7fa029d24700 1 Processor -- start 2020-11-28T21:55:33.530 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.519+0000 7fa029d24700 1 -- start start 2020-11-28T21:55:33.530 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.519+0000 7fa029d24700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa0240603d0 0x7fa0240b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:33.531 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.519+0000 7fa029d24700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa0240b4c70 0x7fa0241444e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:33.531 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.519+0000 7fa029d24700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0240b2f50 0x7fa0241449d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:33.531 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.519+0000 7fa029d24700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa024146d90 con 0x7fa0240603d0 2020-11-28T21:55:33.531 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.519+0000 7fa029d24700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02408fc50 con 0x7fa0240b2f50 2020-11-28T21:55:33.531 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.519+0000 7fa029d24700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa024143e80 con 0x7fa0240b4c70 2020-11-28T21:55:33.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.519+0000 7fa029523700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0240b2f50 0x7fa0241449d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:33.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.519+0000 7fa029523700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0240b2f50 0x7fa0241449d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44968/0 (socket says 172.21.15.162:44968) 2020-11-28T21:55:33.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.519+0000 7fa028d22700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa0240603d0 0x7fa0240b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:33.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.519+0000 7fa023fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa0240b4c70 0x7fa0241444e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:33.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.519+0000 7fa029523700 1 -- 172.21.15.162:0/3227610772 learned_addr learned my addr 172.21.15.162:0/3227610772 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:33.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.519+0000 7fa029523700 1 -- 172.21.15.162:0/3227610772 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa0240b4c70 msgr2=0x7fa0241444e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:33.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.519+0000 7fa029523700 1 --2- 172.21.15.162:0/3227610772 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa0240b4c70 0x7fa0241444e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:33.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.519+0000 7fa029523700 1 -- 172.21.15.162:0/3227610772 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa0240603d0 msgr2=0x7fa0240b2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:33.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.519+0000 7fa029523700 1 --2- 172.21.15.162:0/3227610772 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa0240603d0 0x7fa0240b2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:33.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.519+0000 7fa029523700 1 -- 172.21.15.162:0/3227610772 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa024146af0 con 0x7fa0240b2f50 2020-11-28T21:55:33.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.523+0000 7fa029523700 1 --2- 172.21.15.162:0/3227610772 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0240b2f50 0x7fa0241449d0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 rx=0x7fa024149ef0 tx=0x7fa01c0015b0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:33.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.523+0000 7fa021ffb700 1 -- 172.21.15.162:0/3227610772 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa01c004a80 con 0x7fa0240b2f50 2020-11-28T21:55:33.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.523+0000 7fa021ffb700 1 -- 172.21.15.162:0/3227610772 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa01c004be0 con 0x7fa0240b2f50 2020-11-28T21:55:33.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.523+0000 7fa029d24700 1 -- 172.21.15.162:0/3227610772 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa02408f9b0 con 0x7fa0240b2f50 2020-11-28T21:55:33.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.523+0000 7fa021ffb700 1 -- 172.21.15.162:0/3227610772 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa01c007170 con 0x7fa0240b2f50 2020-11-28T21:55:33.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.523+0000 7fa029d24700 1 -- 172.21.15.162:0/3227610772 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa024149a90 con 0x7fa0240b2f50 2020-11-28T21:55:33.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.523+0000 7fa029d24700 1 -- 172.21.15.162:0/3227610772 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa024155a30 con 0x7fa0240b2f50 2020-11-28T21:55:33.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.523+0000 7fa021ffb700 1 -- 172.21.15.162:0/3227610772 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa01c010070 con 0x7fa0240b2f50 2020-11-28T21:55:33.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.523+0000 7fa021ffb700 1 --2- 172.21.15.162:0/3227610772 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa004055440 0x7fa004057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:33.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.523+0000 7fa021ffb700 1 -- 172.21.15.162:0/3227610772 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa01c06dae0 con 0x7fa0240b2f50 2020-11-28T21:55:33.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.523+0000 7fa028d22700 1 --2- 172.21.15.162:0/3227610772 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa004055440 0x7fa004057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:33.537 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.527+0000 7fa028d22700 1 --2- 172.21.15.162:0/3227610772 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa004055440 0x7fa004057900 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7fa01800d000 tx=0x7fa0180079d0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:33.537 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.531+0000 7fa021ffb700 1 -- 172.21.15.162:0/3227610772 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa01c043cd0 con 0x7fa0240b2f50 2020-11-28T21:55:33.673 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.671+0000 7fa029d24700 1 -- 172.21.15.162:0/3227610772 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa024057050 con 0x7fa0240b2f50 2020-11-28T21:55:33.674 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.671+0000 7fa021ffb700 1 -- 172.21.15.162:0/3227610772 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+16148 (secure 0 0 0) 0x7fa01c0472f0 con 0x7fa0240b2f50 2020-11-28T21:55:33.675 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:33.675 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":138,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:55:32.383252+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:55:30.002238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2020-11-28T21:55:30.389418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:55:33.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.671+0000 7fa029d24700 1 -- 172.21.15.162:0/3227610772 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa004055440 msgr2=0x7fa004057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:33.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.671+0000 7fa029d24700 1 --2- 172.21.15.162:0/3227610772 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa004055440 0x7fa004057900 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7fa01800d000 tx=0x7fa0180079d0).stop 2020-11-28T21:55:33.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.671+0000 7fa029d24700 1 -- 172.21.15.162:0/3227610772 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0240b2f50 msgr2=0x7fa0241449d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:33.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.671+0000 7fa029d24700 1 --2- 172.21.15.162:0/3227610772 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0240b2f50 0x7fa0241449d0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 rx=0x7fa024149ef0 tx=0x7fa01c0015b0).stop 2020-11-28T21:55:33.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.671+0000 7fa029d24700 1 -- 172.21.15.162:0/3227610772 shutdown_connections 2020-11-28T21:55:33.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.671+0000 7fa029d24700 1 --2- 172.21.15.162:0/3227610772 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa004055440 0x7fa004057900 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:33.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.671+0000 7fa029d24700 1 --2- 172.21.15.162:0/3227610772 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0240b2f50 0x7fa0241449d0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:33.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.671+0000 7fa029d24700 1 --2- 172.21.15.162:0/3227610772 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa0240603d0 0x7fa0240b2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:33.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.671+0000 7fa029d24700 1 --2- 172.21.15.162:0/3227610772 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa0240b4c70 0x7fa0241444e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:33.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.671+0000 7fa029d24700 1 -- 172.21.15.162:0/3227610772 >> 172.21.15.162:0/3227610772 conn(0x7fa024066a40 msgr2=0x7fa024142df0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:33.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.671+0000 7fa029d24700 1 -- 172.21.15.162:0/3227610772 shutdown_connections 2020-11-28T21:55:33.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.671+0000 7fa029d24700 1 -- 172.21.15.162:0/3227610772 wait complete. 2020-11-28T21:55:33.689 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-11-28T21:55:33.812 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.803+0000 7f2c791af700 1 Processor -- start 2020-11-28T21:55:33.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.803+0000 7f2c791af700 1 -- start start 2020-11-28T21:55:33.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.803+0000 7f2c791af700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2c7405fa90 0x7f2c7415a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:33.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.803+0000 7f2c791af700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c740ab0b0 con 0x7f2c740b2e80 2020-11-28T21:55:33.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.803+0000 7f2c791af700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c740b3260 con 0x7f2c740b2560 2020-11-28T21:55:33.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.803+0000 7f2c791af700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c740ab2e0 con 0x7f2c7405fa90 2020-11-28T21:55:33.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.803+0000 7f2c73fff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f2c740b2560 0x7f2c74156ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51816/0 (socket says 172.21.15.162:51816) 2020-11-28T21:55:33.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.803+0000 7f2c73fff700 1 -- 172.21.15.162:0/775855209 learned_addr learned my addr 172.21.15.162:0/775855209 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:33.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.803+0000 7f2c7a411700 1 --2- 172.21.15.162:0/775855209 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2c7405fa90 0x7f2c7415a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:33.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.803+0000 7f2c737fe700 1 -- 172.21.15.162:0/775855209 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1077700563 0 0) 0x7f2c740b3260 con 0x7f2c740b2560 2020-11-28T21:55:33.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.803+0000 7f2c737fe700 1 -- 172.21.15.162:0/775855209 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c5c003230 con 0x7f2c740b2560 2020-11-28T21:55:33.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.803+0000 7f2c737fe700 1 -- 172.21.15.162:0/775855209 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2116379531 0 0) 0x7f2c5c003230 con 0x7f2c740b2560 2020-11-28T21:55:33.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.803+0000 7f2c737fe700 1 -- 172.21.15.162:0/775855209 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2c7405fa90 msgr2=0x7f2c7415a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:33.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.803+0000 7f2c737fe700 1 --2- 172.21.15.162:0/775855209 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2c7405fa90 0x7f2c7415a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:33.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.803+0000 7f2c737fe700 1 -- 172.21.15.162:0/775855209 >> v1:172.21.15.118:6789/0 conn(0x7f2c740b2e80 legacy=0x7f2c7405f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:33.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.803+0000 7f2c737fe700 1 -- 172.21.15.162:0/775855209 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c740256d0 con 0x7f2c740b2560 2020-11-28T21:55:33.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.803+0000 7f2c737fe700 1 -- 172.21.15.162:0/775855209 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f2c680016d0 con 0x7f2c740b2560 2020-11-28T21:55:33.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.803+0000 7f2c737fe700 1 -- 172.21.15.162:0/775855209 >> v1:172.21.15.162:6789/0 conn(0x7f2c740b2560 legacy=0x7f2c74156ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:33.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.803+0000 7f2c737fe700 1 --2- 172.21.15.162:0/775855209 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c5c003720 0x7f2c5c003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:33.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.803+0000 7f2c737fe700 1 --2- 172.21.15.162:0/775855209 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2c7405fa90 0x7f2c5c0044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:33.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.803+0000 7f2c737fe700 1 --2- 172.21.15.162:0/775855209 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2c5c0049c0 0x7f2c5c004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:33.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.803+0000 7f2c737fe700 1 -- 172.21.15.162:0/775855209 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c740ab2e0 con 0x7f2c5c0049c0 2020-11-28T21:55:33.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.803+0000 7f2c737fe700 1 -- 172.21.15.162:0/775855209 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c004210 con 0x7f2c5c003720 2020-11-28T21:55:33.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.803+0000 7f2c737fe700 1 -- 172.21.15.162:0/775855209 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c04f730 con 0x7f2c7405fa90 2020-11-28T21:55:33.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.807+0000 7f2c789ae700 1 --2- 172.21.15.162:0/775855209 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c5c003720 0x7f2c5c003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:33.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.807+0000 7f2c73fff700 1 --2- 172.21.15.162:0/775855209 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2c7405fa90 0x7f2c5c0044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:33.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.807+0000 7f2c7a411700 1 --2- 172.21.15.162:0/775855209 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2c5c0049c0 0x7f2c5c004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:33.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.807+0000 7f2c789ae700 1 -- 172.21.15.162:0/775855209 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2c7405fa90 msgr2=0x7f2c5c0044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:33.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.807+0000 7f2c789ae700 1 --2- 172.21.15.162:0/775855209 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2c7405fa90 0x7f2c5c0044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:33.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.807+0000 7f2c789ae700 1 -- 172.21.15.162:0/775855209 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2c5c0049c0 msgr2=0x7f2c5c004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:33.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.807+0000 7f2c789ae700 1 --2- 172.21.15.162:0/775855209 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2c5c0049c0 0x7f2c5c004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:33.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.807+0000 7f2c789ae700 1 -- 172.21.15.162:0/775855209 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c5c003230 con 0x7f2c5c003720 2020-11-28T21:55:33.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.807+0000 7f2c7a411700 1 --2- 172.21.15.162:0/775855209 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2c5c0049c0 0x7f2c5c004e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:55:33.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.807+0000 7f2c789ae700 1 --2- 172.21.15.162:0/775855209 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c5c003720 0x7f2c5c003b40 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 rx=0x7f2c74063710 tx=0x7f2c64007b60).ready entity=mon.1 client_cookie=e05e80cdb73e1c2f server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:33.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.807+0000 7f2c737fe700 1 -- 172.21.15.162:0/775855209 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c6400c660 con 0x7f2c5c003720 2020-11-28T21:55:33.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.807+0000 7f2c737fe700 1 -- 172.21.15.162:0/775855209 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c6400c7c0 con 0x7f2c5c003720 2020-11-28T21:55:33.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.807+0000 7f2c737fe700 1 -- 172.21.15.162:0/775855209 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c6400cb20 con 0x7f2c5c003720 2020-11-28T21:55:33.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.807+0000 7f2c791af700 1 -- 172.21.15.162:0/775855209 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c5c003720 msgr2=0x7f2c5c003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:33.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.807+0000 7f2c791af700 1 --2- 172.21.15.162:0/775855209 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c5c003720 0x7f2c5c003b40 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 rx=0x7f2c74063710 tx=0x7f2c64007b60).stop 2020-11-28T21:55:33.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.807+0000 7f2c7a411700 1 -- 172.21.15.162:0/775855209 reap_dead start 2020-11-28T21:55:33.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.807+0000 7f2c791af700 1 -- 172.21.15.162:0/775855209 shutdown_connections 2020-11-28T21:55:33.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.807+0000 7f2c791af700 1 -- 172.21.15.162:0/775855209 >> 172.21.15.162:0/775855209 conn(0x7f2c74063b60 msgr2=0x7f2c74063f40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:33.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.807+0000 7f2c791af700 1 -- 172.21.15.162:0/775855209 shutdown_connections 2020-11-28T21:55:33.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.807+0000 7f2c791af700 1 -- 172.21.15.162:0/775855209 wait complete. 2020-11-28T21:55:33.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.811+0000 7f2c791af700 1 Processor -- start 2020-11-28T21:55:33.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.811+0000 7f2c791af700 1 -- start start 2020-11-28T21:55:33.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.811+0000 7f2c791af700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c74132820 0x7f2c74066b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:33.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.811+0000 7f2c791af700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2c7409ad90 0x7f2c74067060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:33.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.811+0000 7f2c791af700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2c7412f260 0x7f2c7412f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:33.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.811+0000 7f2c791af700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c740655c0 con 0x7f2c7412f260 2020-11-28T21:55:33.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.811+0000 7f2c791af700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c74132c00 con 0x7f2c74132820 2020-11-28T21:55:33.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.811+0000 7f2c791af700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c74065360 con 0x7f2c7409ad90 2020-11-28T21:55:33.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.811+0000 7f2c789ae700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c74132820 0x7f2c74066b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:33.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.811+0000 7f2c789ae700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c74132820 0x7f2c74066b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44986/0 (socket says 172.21.15.162:44986) 2020-11-28T21:55:33.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.811+0000 7f2c73fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2c7409ad90 0x7f2c74067060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:33.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.811+0000 7f2c789ae700 1 -- 172.21.15.162:0/3016778776 learned_addr learned my addr 172.21.15.162:0/3016778776 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:33.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.811+0000 7f2c7a411700 1 --2- 172.21.15.162:0/3016778776 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2c7412f260 0x7f2c7412f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:33.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.811+0000 7f2c789ae700 1 -- 172.21.15.162:0/3016778776 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2c7409ad90 msgr2=0x7f2c74067060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:33.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.811+0000 7f2c789ae700 1 --2- 172.21.15.162:0/3016778776 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2c7409ad90 0x7f2c74067060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:33.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.811+0000 7f2c789ae700 1 -- 172.21.15.162:0/3016778776 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2c7412f260 msgr2=0x7f2c7412f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:33.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.811+0000 7f2c789ae700 1 --2- 172.21.15.162:0/3016778776 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2c7412f260 0x7f2c7412f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:33.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.811+0000 7f2c789ae700 1 -- 172.21.15.162:0/3016778776 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c74065020 con 0x7f2c74132820 2020-11-28T21:55:33.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.811+0000 7f2c73fff700 1 --2- 172.21.15.162:0/3016778776 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2c7409ad90 0x7f2c74067060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:55:33.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.811+0000 7f2c789ae700 1 --2- 172.21.15.162:0/3016778776 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c74132820 0x7f2c74066b70 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 rx=0x7f2c7409a790 tx=0x7f2c64016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:33.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.811+0000 7f2c717fa700 1 -- 172.21.15.162:0/3016778776 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c64013a80 con 0x7f2c74132820 2020-11-28T21:55:33.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.811+0000 7f2c791af700 1 -- 172.21.15.162:0/3016778776 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c740648e0 con 0x7f2c74132820 2020-11-28T21:55:33.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.811+0000 7f2c717fa700 1 -- 172.21.15.162:0/3016778776 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c64013be0 con 0x7f2c74132820 2020-11-28T21:55:33.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.811+0000 7f2c717fa700 1 -- 172.21.15.162:0/3016778776 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c6401d410 con 0x7f2c74132820 2020-11-28T21:55:33.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.811+0000 7f2c791af700 1 -- 172.21.15.162:0/3016778776 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c74067550 con 0x7f2c74132820 2020-11-28T21:55:33.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.815+0000 7f2c791af700 1 -- 172.21.15.162:0/3016778776 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c74057050 con 0x7f2c74132820 2020-11-28T21:55:33.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.815+0000 7f2c717fa700 1 -- 172.21.15.162:0/3016778776 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2c64023020 con 0x7f2c74132820 2020-11-28T21:55:33.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.815+0000 7f2c717fa700 1 --2- 172.21.15.162:0/3016778776 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2c54055320 0x7f2c540577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:33.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.815+0000 7f2c73fff700 1 --2- 172.21.15.162:0/3016778776 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2c54055320 0x7f2c540577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:33.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.815+0000 7f2c717fa700 1 -- 172.21.15.162:0/3016778776 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2c6406eef0 con 0x7f2c74132820 2020-11-28T21:55:33.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.815+0000 7f2c73fff700 1 --2- 172.21.15.162:0/3016778776 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2c54055320 0x7f2c540577e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f2c68005000 tx=0x7f2c68009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:33.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:33.823+0000 7f2c717fa700 1 -- 172.21.15.162:0/3016778776 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f2c64045060 con 0x7f2c74132820 2020-11-28T21:55:34.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.015+0000 7f2c791af700 1 -- 172.21.15.162:0/3016778776 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f2c74057050 con 0x7f2c74132820 2020-11-28T21:55:34.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.023+0000 7f2c717fa700 1 -- 172.21.15.162:0/3016778776 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v29) v1 ==== 184+0+0 (secure 0 0 0) 0x7f2c64048680 con 0x7f2c74132820 2020-11-28T21:55:34.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.023+0000 7f2c791af700 1 -- 172.21.15.162:0/3016778776 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2c54055320 msgr2=0x7f2c540577e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:34.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.023+0000 7f2c791af700 1 --2- 172.21.15.162:0/3016778776 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2c54055320 0x7f2c540577e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f2c68005000 tx=0x7f2c68009040).stop 2020-11-28T21:55:34.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.023+0000 7f2c791af700 1 -- 172.21.15.162:0/3016778776 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c74132820 msgr2=0x7f2c74066b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:34.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.023+0000 7f2c791af700 1 --2- 172.21.15.162:0/3016778776 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c74132820 0x7f2c74066b70 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 rx=0x7f2c7409a790 tx=0x7f2c64016000).stop 2020-11-28T21:55:34.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.023+0000 7f2c791af700 1 -- 172.21.15.162:0/3016778776 shutdown_connections 2020-11-28T21:55:34.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.023+0000 7f2c791af700 1 --2- 172.21.15.162:0/3016778776 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2c54055320 0x7f2c540577e0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:34.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.023+0000 7f2c791af700 1 --2- 172.21.15.162:0/3016778776 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2c7412f260 0x7f2c7412f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:34.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.023+0000 7f2c791af700 1 --2- 172.21.15.162:0/3016778776 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c74132820 0x7f2c74066b70 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:34.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.027+0000 7f2c791af700 1 --2- 172.21.15.162:0/3016778776 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2c7409ad90 0x7f2c74067060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:34.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.027+0000 7f2c791af700 1 -- 172.21.15.162:0/3016778776 >> 172.21.15.162:0/3016778776 conn(0x7f2c74063b60 msgr2=0x7f2c740b3260 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:34.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.027+0000 7f2c791af700 1 -- 172.21.15.162:0/3016778776 shutdown_connections 2020-11-28T21:55:34.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.027+0000 7f2c791af700 1 -- 172.21.15.162:0/3016778776 wait complete. 2020-11-28T21:55:34.032 INFO:teuthology.orchestra.run.smithi162.stderr:updated caps for client.0 2020-11-28T21:55:34.046 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-11-28T21:55:34.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.159+0000 7f373d7e7700 1 Processor -- start 2020-11-28T21:55:34.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.159+0000 7f373d7e7700 1 -- start start 2020-11-28T21:55:34.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.159+0000 7f373d7e7700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f37380b2560 0x7f3738156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:34.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.159+0000 7f373d7e7700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f373805fa90 0x7f37381593c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:34.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.159+0000 7f373d7e7700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37380ab0b0 con 0x7f37380b2e80 2020-11-28T21:55:34.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.159+0000 7f373d7e7700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37380b3260 con 0x7f37380b2560 2020-11-28T21:55:34.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.159+0000 7f373d7e7700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37380ab2e0 con 0x7f373805fa90 2020-11-28T21:55:34.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.159+0000 7f3737fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f37380b2560 0x7f3738156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:34.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.159+0000 7f3737fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f37380b2560 0x7f3738156ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44994/0 (socket says 172.21.15.162:44994) 2020-11-28T21:55:34.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.159+0000 7f3737fff700 1 -- 172.21.15.162:0/2998732766 learned_addr learned my addr 172.21.15.162:0/2998732766 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:34.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.159+0000 7f373ea49700 1 --2- 172.21.15.162:0/2998732766 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f373805fa90 0x7f37381593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:34.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.163+0000 7f3737fff700 1 -- 172.21.15.162:0/2998732766 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f373805fa90 msgr2=0x7f37381593c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:34.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.163+0000 7f3737fff700 1 --2- 172.21.15.162:0/2998732766 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f373805fa90 0x7f37381593c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:34.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.163+0000 7f3737fff700 1 -- 172.21.15.162:0/2998732766 >> v1:172.21.15.118:6789/0 conn(0x7f37380b2e80 legacy=0x7f373805f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:34.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.163+0000 7f3737fff700 1 -- 172.21.15.162:0/2998732766 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37380256d0 con 0x7f37380b2560 2020-11-28T21:55:34.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.163+0000 7f3737fff700 1 --2- 172.21.15.162:0/2998732766 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37380b2560 0x7f3738156ec0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 rx=0x7f3720002f30 tx=0x7f3720006e80).ready entity=mon.1 client_cookie=8cef7987c86bd3c6 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:34.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.163+0000 7f37377fe700 1 -- 172.21.15.162:0/2998732766 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f37200097f0 con 0x7f37380b2560 2020-11-28T21:55:34.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.163+0000 7f37377fe700 1 -- 172.21.15.162:0/2998732766 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3720009950 con 0x7f37380b2560 2020-11-28T21:55:34.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.163+0000 7f37377fe700 1 -- 172.21.15.162:0/2998732766 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3720009cb0 con 0x7f37380b2560 2020-11-28T21:55:34.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.163+0000 7f373d7e7700 1 -- 172.21.15.162:0/2998732766 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37380b2560 msgr2=0x7f3738156ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:34.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.163+0000 7f373d7e7700 1 --2- 172.21.15.162:0/2998732766 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37380b2560 0x7f3738156ec0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 rx=0x7f3720002f30 tx=0x7f3720006e80).stop 2020-11-28T21:55:34.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.163+0000 7f373d7e7700 1 -- 172.21.15.162:0/2998732766 shutdown_connections 2020-11-28T21:55:34.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.163+0000 7f373d7e7700 1 --2- 172.21.15.162:0/2998732766 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f373805fa90 0x7f37381593c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:34.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.163+0000 7f373d7e7700 1 --2- 172.21.15.162:0/2998732766 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37380b2560 0x7f3738156ec0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:34.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.163+0000 7f373d7e7700 1 -- 172.21.15.162:0/2998732766 >> 172.21.15.162:0/2998732766 conn(0x7f3738063b60 msgr2=0x7f3738063f40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:34.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.163+0000 7f373d7e7700 1 -- 172.21.15.162:0/2998732766 shutdown_connections 2020-11-28T21:55:34.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.163+0000 7f373d7e7700 1 -- 172.21.15.162:0/2998732766 wait complete. 2020-11-28T21:55:34.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.163+0000 7f373d7e7700 1 Processor -- start 2020-11-28T21:55:34.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.163+0000 7f373d7e7700 1 -- start start 2020-11-28T21:55:34.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.167+0000 7f373d7e7700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f373805fa90 0x7f3738065440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:34.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.167+0000 7f373d7e7700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37380b2560 0x7f3738065930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:34.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.167+0000 7f373d7e7700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f37380b2e80 0x7f37380ae1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:34.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.167+0000 7f373d7e7700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37380995b0 con 0x7f37380b2560 2020-11-28T21:55:34.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.167+0000 7f373d7e7700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3738099f20 con 0x7f373805fa90 2020-11-28T21:55:34.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.167+0000 7f373d7e7700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37380999e0 con 0x7f37380b2e80 2020-11-28T21:55:34.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.167+0000 7f373cfe6700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f373805fa90 0x7f3738065440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:34.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.167+0000 7f373ea49700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f37380b2e80 0x7f37380ae1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:34.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.167+0000 7f3737fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37380b2560 0x7f3738065930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:34.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.167+0000 7f373ea49700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f37380b2e80 0x7f37380ae1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:42224/0 (socket says 172.21.15.162:42224) 2020-11-28T21:55:34.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.167+0000 7f373cfe6700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f373805fa90 0x7f3738065440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45002/0 (socket says 172.21.15.162:45002) 2020-11-28T21:55:34.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.167+0000 7f373ea49700 1 -- 172.21.15.162:0/3261114210 learned_addr learned my addr 172.21.15.162:0/3261114210 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:34.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.167+0000 7f373ea49700 1 -- 172.21.15.162:0/3261114210 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f373805fa90 msgr2=0x7f3738065440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:34.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.167+0000 7f373ea49700 1 --2- 172.21.15.162:0/3261114210 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f373805fa90 0x7f3738065440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:34.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.167+0000 7f373ea49700 1 -- 172.21.15.162:0/3261114210 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37380b2560 msgr2=0x7f3738065930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:34.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.167+0000 7f373ea49700 1 --2- 172.21.15.162:0/3261114210 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37380b2560 0x7f3738065930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:34.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.167+0000 7f373ea49700 1 -- 172.21.15.162:0/3261114210 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3738099c80 con 0x7f37380b2e80 2020-11-28T21:55:34.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.167+0000 7f373cfe6700 1 --2- 172.21.15.162:0/3261114210 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f373805fa90 0x7f3738065440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:55:34.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.167+0000 7f3737fff700 1 --2- 172.21.15.162:0/3261114210 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37380b2560 0x7f3738065930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:55:34.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.167+0000 7f373ea49700 1 --2- 172.21.15.162:0/3261114210 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f37380b2e80 0x7f37380ae1a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f373809be80 tx=0x7f3728006dc0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:34.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.167+0000 7f37357fa700 1 -- 172.21.15.162:0/3261114210 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f372800ca10 con 0x7f37380b2e80 2020-11-28T21:55:34.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.167+0000 7f37357fa700 1 -- 172.21.15.162:0/3261114210 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f372800cb70 con 0x7f37380b2e80 2020-11-28T21:55:34.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.167+0000 7f373d7e7700 1 -- 172.21.15.162:0/3261114210 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3738080950 con 0x7f37380b2e80 2020-11-28T21:55:34.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.167+0000 7f37357fa700 1 -- 172.21.15.162:0/3261114210 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f372800ced0 con 0x7f37380b2e80 2020-11-28T21:55:34.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.167+0000 7f373d7e7700 1 -- 172.21.15.162:0/3261114210 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3738099230 con 0x7f37380b2e80 2020-11-28T21:55:34.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.167+0000 7f373d7e7700 1 -- 172.21.15.162:0/3261114210 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3738057050 con 0x7f37380b2e80 2020-11-28T21:55:34.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.171+0000 7f37357fa700 1 -- 172.21.15.162:0/3261114210 <== mon.2 v2:172.21.15.118:3301/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f372801c460 con 0x7f37380b2e80 2020-11-28T21:55:34.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.171+0000 7f37357fa700 1 --2- 172.21.15.162:0/3261114210 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f371c0553b0 0x7f371c057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:34.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.171+0000 7f37357fa700 1 -- 172.21.15.162:0/3261114210 <== mon.2 v2:172.21.15.118:3301/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f372806e2e0 con 0x7f37380b2e80 2020-11-28T21:55:34.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.171+0000 7f373cfe6700 1 --2- 172.21.15.162:0/3261114210 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f371c0553b0 0x7f371c057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:34.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.171+0000 7f373cfe6700 1 --2- 172.21.15.162:0/3261114210 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f371c0553b0 0x7f371c057870 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7f373000d000 tx=0x7f3730009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:34.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.179+0000 7f37357fa700 1 -- 172.21.15.162:0/3261114210 <== mon.2 v2:172.21.15.118:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f3728044580 con 0x7f37380b2e80 2020-11-28T21:55:34.400 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.395+0000 7f373d7e7700 1 -- 172.21.15.162:0/3261114210 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f3738064070 con 0x7f37380b2e80 2020-11-28T21:55:34.410 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.407+0000 7f37357fa700 1 -- 172.21.15.162:0/3261114210 <== mon.2 v2:172.21.15.118:3301/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v30) v1 ==== 184+0+0 (secure 0 0 0) 0x7f3728047ba0 con 0x7f37380b2e80 2020-11-28T21:55:34.412 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.407+0000 7f373d7e7700 1 -- 172.21.15.162:0/3261114210 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f371c0553b0 msgr2=0x7f371c057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:34.412 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.407+0000 7f373d7e7700 1 --2- 172.21.15.162:0/3261114210 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f371c0553b0 0x7f371c057870 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7f373000d000 tx=0x7f3730009040).stop 2020-11-28T21:55:34.412 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.407+0000 7f373d7e7700 1 -- 172.21.15.162:0/3261114210 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f37380b2e80 msgr2=0x7f37380ae1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:34.412 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.407+0000 7f373d7e7700 1 --2- 172.21.15.162:0/3261114210 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f37380b2e80 0x7f37380ae1a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f373809be80 tx=0x7f3728006dc0).stop 2020-11-28T21:55:34.413 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.407+0000 7f373d7e7700 1 -- 172.21.15.162:0/3261114210 shutdown_connections 2020-11-28T21:55:34.413 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.407+0000 7f373d7e7700 1 --2- 172.21.15.162:0/3261114210 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f371c0553b0 0x7f371c057870 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:34.413 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.407+0000 7f373d7e7700 1 --2- 172.21.15.162:0/3261114210 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f37380b2e80 0x7f37380ae1a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:34.413 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.407+0000 7f373d7e7700 1 --2- 172.21.15.162:0/3261114210 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f373805fa90 0x7f3738065440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:34.414 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.407+0000 7f373d7e7700 1 --2- 172.21.15.162:0/3261114210 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37380b2560 0x7f3738065930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:34.414 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.407+0000 7f373d7e7700 1 -- 172.21.15.162:0/3261114210 >> 172.21.15.162:0/3261114210 conn(0x7f3738063b60 msgr2=0x7f37380b6930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:34.414 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.407+0000 7f373d7e7700 1 -- 172.21.15.162:0/3261114210 shutdown_connections 2020-11-28T21:55:34.414 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.407+0000 7f373d7e7700 1 -- 172.21.15.162:0/3261114210 wait complete. 2020-11-28T21:55:34.415 INFO:teuthology.orchestra.run.smithi162.stderr:updated caps for client.1 2020-11-28T21:55:34.425 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:55:34.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.531+0000 7fe96ae45700 1 Processor -- start 2020-11-28T21:55:34.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.531+0000 7fe96ae45700 1 -- start start 2020-11-28T21:55:34.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.531+0000 7fe96ae45700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe964066bb0 0x7fe9641406c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:34.537 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.531+0000 7fe96ae45700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe964140bb0 con 0x7fe964062cd0 2020-11-28T21:55:34.537 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.531+0000 7fe96ae45700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9640b3230 con 0x7fe9640b3a50 2020-11-28T21:55:34.537 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.531+0000 7fe96ae45700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe964144bf0 con 0x7fe964066bb0 2020-11-28T21:55:34.537 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.531+0000 7fe969e43700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fe9640b3a50 0x7fe9641444e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51850/0 (socket says 172.21.15.162:51850) 2020-11-28T21:55:34.538 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.531+0000 7fe969e43700 1 -- 172.21.15.162:0/3100219040 learned_addr learned my addr 172.21.15.162:0/3100219040 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:34.538 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.531+0000 7fe96a644700 1 --2- 172.21.15.162:0/3100219040 >> v2:172.21.15.118:3300/0 conn(0x7fe964066bb0 0x7fe9641406c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:34.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.531+0000 7fe968e41700 1 -- 172.21.15.162:0/3100219040 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1450443205 0 0) 0x7fe9640b3230 con 0x7fe9640b3a50 2020-11-28T21:55:34.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.531+0000 7fe968e41700 1 -- 172.21.15.162:0/3100219040 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe94c003230 con 0x7fe9640b3a50 2020-11-28T21:55:34.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.531+0000 7fe96a644700 1 -- 172.21.15.162:0/3100219040 >> v1:172.21.15.162:6789/0 conn(0x7fe9640b3a50 legacy=0x7fe9641444e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:34.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.531+0000 7fe96a644700 1 -- 172.21.15.162:0/3100219040 >> v1:172.21.15.118:6789/0 conn(0x7fe964062cd0 legacy=0x7fe9640664a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:34.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.531+0000 7fe96a644700 1 -- 172.21.15.162:0/3100219040 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe964143da0 con 0x7fe964066bb0 2020-11-28T21:55:34.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.535+0000 7fe96a644700 1 --2- 172.21.15.162:0/3100219040 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe964066bb0 0x7fe9641406c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7fe95c0032b0 tx=0x7fe95c0091a0).ready entity=mon.0 client_cookie=b8b1e8cd6d913e8b server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:34.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.535+0000 7fe968e41700 1 -- 172.21.15.162:0/3100219040 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe95c00b9a0 con 0x7fe964066bb0 2020-11-28T21:55:34.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.535+0000 7fe968e41700 1 -- 172.21.15.162:0/3100219040 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95c00bb00 con 0x7fe964066bb0 2020-11-28T21:55:34.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.535+0000 7fe968e41700 1 -- 172.21.15.162:0/3100219040 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe95c00bc70 con 0x7fe964066bb0 2020-11-28T21:55:34.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.535+0000 7fe96ae45700 1 -- 172.21.15.162:0/3100219040 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe964066bb0 msgr2=0x7fe9641406c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:34.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.535+0000 7fe96ae45700 1 --2- 172.21.15.162:0/3100219040 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe964066bb0 0x7fe9641406c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7fe95c0032b0 tx=0x7fe95c0091a0).stop 2020-11-28T21:55:34.543 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.535+0000 7fe96ae45700 1 -- 172.21.15.162:0/3100219040 shutdown_connections 2020-11-28T21:55:34.543 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.535+0000 7fe96ae45700 1 --2- 172.21.15.162:0/3100219040 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe964066bb0 0x7fe9641406c0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:34.544 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.535+0000 7fe96ae45700 1 -- 172.21.15.162:0/3100219040 >> 172.21.15.162:0/3100219040 conn(0x7fe96405b970 msgr2=0x7fe96405bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:34.544 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.535+0000 7fe96ae45700 1 -- 172.21.15.162:0/3100219040 shutdown_connections 2020-11-28T21:55:34.544 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.535+0000 7fe96ae45700 1 -- 172.21.15.162:0/3100219040 wait complete. 2020-11-28T21:55:34.547 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.539+0000 7fe96ae45700 1 Processor -- start 2020-11-28T21:55:34.547 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.539+0000 7fe96ae45700 1 -- start start 2020-11-28T21:55:34.547 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.539+0000 7fe96ae45700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe964062cd0 0x7fe9640952f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:34.548 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.539+0000 7fe96ae45700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe964066bb0 0x7fe9640b0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:34.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.539+0000 7fe96ae45700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9640b3a50 0x7fe9640b0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:34.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.539+0000 7fe96ae45700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9640ad8c0 con 0x7fe964062cd0 2020-11-28T21:55:34.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.539+0000 7fe96ae45700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe964140910 con 0x7fe9640b3a50 2020-11-28T21:55:34.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.539+0000 7fe96ae45700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9641528e0 con 0x7fe964066bb0 2020-11-28T21:55:34.550 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.539+0000 7fe96a644700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9640b3a50 0x7fe9640b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:34.550 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.539+0000 7fe96a644700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9640b3a50 0x7fe9640b0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45016/0 (socket says 172.21.15.162:45016) 2020-11-28T21:55:34.550 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.539+0000 7fe96a644700 1 -- 172.21.15.162:0/3842130750 learned_addr learned my addr 172.21.15.162:0/3842130750 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:34.550 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.539+0000 7fe969642700 1 --2- 172.21.15.162:0/3842130750 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe964066bb0 0x7fe9640b0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:34.550 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.539+0000 7fe969e43700 1 --2- 172.21.15.162:0/3842130750 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe964062cd0 0x7fe9640952f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:34.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.539+0000 7fe96a644700 1 -- 172.21.15.162:0/3842130750 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe964066bb0 msgr2=0x7fe9640b0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:34.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.539+0000 7fe96a644700 1 --2- 172.21.15.162:0/3842130750 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe964066bb0 0x7fe9640b0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:34.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.539+0000 7fe96a644700 1 -- 172.21.15.162:0/3842130750 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe964062cd0 msgr2=0x7fe9640952f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:34.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.539+0000 7fe96a644700 1 --2- 172.21.15.162:0/3842130750 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe964062cd0 0x7fe9640952f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:34.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.539+0000 7fe96a644700 1 -- 172.21.15.162:0/3842130750 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9640aee80 con 0x7fe9640b3a50 2020-11-28T21:55:34.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.539+0000 7fe969642700 1 --2- 172.21.15.162:0/3842130750 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe964066bb0 0x7fe9640b0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:55:34.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.539+0000 7fe969e43700 1 --2- 172.21.15.162:0/3842130750 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe964062cd0 0x7fe9640952f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:55:34.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.539+0000 7fe96a644700 1 --2- 172.21.15.162:0/3842130750 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9640b3a50 0x7fe9640b0670 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 rx=0x7fe95c0032b0 tx=0x7fe95c016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:34.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.539+0000 7fe952ffd700 1 -- 172.21.15.162:0/3842130750 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe95c01f5e0 con 0x7fe9640b3a50 2020-11-28T21:55:34.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.539+0000 7fe952ffd700 1 -- 172.21.15.162:0/3842130750 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95c01f740 con 0x7fe9640b3a50 2020-11-28T21:55:34.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.539+0000 7fe96ae45700 1 -- 172.21.15.162:0/3842130750 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9640af120 con 0x7fe9640b3a50 2020-11-28T21:55:34.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.539+0000 7fe952ffd700 1 -- 172.21.15.162:0/3842130750 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe95c01faa0 con 0x7fe9640b3a50 2020-11-28T21:55:34.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.539+0000 7fe96ae45700 1 -- 172.21.15.162:0/3842130750 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9640af5c0 con 0x7fe9640b3a50 2020-11-28T21:55:34.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.539+0000 7fe96ae45700 1 -- 172.21.15.162:0/3842130750 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe964095b20 con 0x7fe9640b3a50 2020-11-28T21:55:34.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.543+0000 7fe952ffd700 1 -- 172.21.15.162:0/3842130750 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe95c026030 con 0x7fe9640b3a50 2020-11-28T21:55:34.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.543+0000 7fe952ffd700 1 --2- 172.21.15.162:0/3842130750 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe944055330 0x7fe9440577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:34.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.543+0000 7fe952ffd700 1 -- 172.21.15.162:0/3842130750 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe95c06f010 con 0x7fe9640b3a50 2020-11-28T21:55:34.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.543+0000 7fe969e43700 1 --2- 172.21.15.162:0/3842130750 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe944055330 0x7fe9440577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:34.555 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.543+0000 7fe969e43700 1 --2- 172.21.15.162:0/3842130750 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe944055330 0x7fe9440577f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7fe95400b000 tx=0x7fe954009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:34.555 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.547+0000 7fe952ffd700 1 -- 172.21.15.162:0/3842130750 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe95c045310 con 0x7fe9640b3a50 2020-11-28T21:55:34.734 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.731+0000 7fe96ae45700 1 -- 172.21.15.162:0/3842130750 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe964152aa0 con 0x7fe9640b3a50 2020-11-28T21:55:34.734 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.731+0000 7fe952ffd700 1 -- 172.21.15.162:0/3842130750 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+2504 (secure 0 0 0) 0x7fe95c024090 con 0x7fe9640b3a50 2020-11-28T21:55:34.735 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:34.735 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":68,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7611,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6835/2127102267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2127102267},{"type":"v1","addr":"172.21.15.162:6835","nonce":2127102267}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":66},{"gid":7623,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/1473328171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1473328171},{"type":"v1","addr":"172.21.15.118:6833","nonce":1473328171}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:55:31.379864+0000","modified":"2020-11-28T21:55:33.042143+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7888":{"gid":7888,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/2667777645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2667777645},{"type":"v1","addr":"172.21.15.162:6837","nonce":2667777645}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2020-11-28T21:55:34.736 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.731+0000 7fe96ae45700 1 -- 172.21.15.162:0/3842130750 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe944055330 msgr2=0x7fe9440577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:34.737 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.731+0000 7fe96ae45700 1 --2- 172.21.15.162:0/3842130750 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe944055330 0x7fe9440577f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7fe95400b000 tx=0x7fe954009040).stop 2020-11-28T21:55:34.737 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.731+0000 7fe96ae45700 1 -- 172.21.15.162:0/3842130750 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9640b3a50 msgr2=0x7fe9640b0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:34.737 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.731+0000 7fe96ae45700 1 --2- 172.21.15.162:0/3842130750 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9640b3a50 0x7fe9640b0670 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 rx=0x7fe95c0032b0 tx=0x7fe95c016000).stop 2020-11-28T21:55:34.737 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.731+0000 7fe96ae45700 1 -- 172.21.15.162:0/3842130750 shutdown_connections 2020-11-28T21:55:34.738 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.731+0000 7fe96ae45700 1 --2- 172.21.15.162:0/3842130750 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe944055330 0x7fe9440577f0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:34.738 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.731+0000 7fe96ae45700 1 --2- 172.21.15.162:0/3842130750 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9640b3a50 0x7fe9640b0670 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:34.738 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.731+0000 7fe96ae45700 1 --2- 172.21.15.162:0/3842130750 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe964062cd0 0x7fe9640952f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:34.738 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.731+0000 7fe96ae45700 1 --2- 172.21.15.162:0/3842130750 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe964066bb0 0x7fe9640b0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:34.739 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.731+0000 7fe96ae45700 1 -- 172.21.15.162:0/3842130750 >> 172.21.15.162:0/3842130750 conn(0x7fe96405b970 msgr2=0x7fe964141ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:34.739 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.731+0000 7fe96ae45700 1 -- 172.21.15.162:0/3842130750 shutdown_connections 2020-11-28T21:55:34.739 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:34.731+0000 7fe96ae45700 1 -- 172.21.15.162:0/3842130750 wait complete. 2020-11-28T21:55:34.741 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 68 2020-11-28T21:55:35.049 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-28T21:55:31.379864+0000', 'modified': '2020-11-28T21:55:33.042143+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7888}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7888': {'gid': 7888, 'name': 'c', 'rank': 0, 'incarnation': 66, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6837/2667777645', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6836', 'nonce': 2667777645}, {'type': 'v1', 'addr': '172.21.15.162:6837', 'nonce': 2667777645}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-28T21:55:35.050 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-28T21:55:35.051 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-28T21:55:35.178 INFO:teuthology.orchestra.run.smithi162.stdout:{"cluster_fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","whoami":0,"id":7888,"want_state":"up:active","state":"up:active","rank_uptime":3.7842082019999999,"mdsmap_epoch":68,"osdmap_epoch":138,"osdmap_epoch_barrier":138,"uptime":8.0518394769999997} 2020-11-28T21:55:35.178 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", "id": 7888, "mdsmap_epoch": 68, "osdmap_epoch": 138, "osdmap_epoch_barrier": 138, "rank_uptime": 3.784208202, "state": "up:active", "uptime": 8.051839477, "want_state": "up:active", "whoami": 0 } 2020-11-28T21:55:35.179 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:55:35.284 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.279+0000 7f1553294700 1 Processor -- start 2020-11-28T21:55:35.284 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.279+0000 7f1553294700 1 -- start start 2020-11-28T21:55:35.286 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.279+0000 7f1553294700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f154c05bcb0 0x7f154c143530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:35.286 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.279+0000 7f1553294700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f154c13c630 0x7f154c13cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:35.287 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.279+0000 7f1553294700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f154c0b0e50 con 0x7f154c0b1c00 2020-11-28T21:55:35.287 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.279+0000 7f1553294700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f154c05fe40 con 0x7f154c13c630 2020-11-28T21:55:35.287 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.279+0000 7f1553294700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f154c0b2080 con 0x7f154c05bcb0 2020-11-28T21:55:35.287 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.279+0000 7f1551a91700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f154c0b1c00 0x7f154c0b0740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51868/0 (socket says 172.21.15.162:51868) 2020-11-28T21:55:35.288 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.279+0000 7f1551a91700 1 -- 172.21.15.162:0/1635009109 learned_addr learned my addr 172.21.15.162:0/1635009109 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:35.288 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.283+0000 7f1552292700 1 --2- 172.21.15.162:0/1635009109 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f154c05bcb0 0x7f154c143530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:35.288 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.283+0000 7f1552a93700 1 --2- 172.21.15.162:0/1635009109 >> v2:172.21.15.118:3300/0 conn(0x7f154c13c630 0x7f154c13cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:35.288 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.283+0000 7f1551290700 1 -- 172.21.15.162:0/1635009109 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1697159381 0 0) 0x7f154c0b0e50 con 0x7f154c0b1c00 2020-11-28T21:55:35.288 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.283+0000 7f1551290700 1 -- 172.21.15.162:0/1635009109 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1534003230 con 0x7f154c0b1c00 2020-11-28T21:55:35.289 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.283+0000 7f1552a93700 1 -- 172.21.15.162:0/1635009109 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f154c05bcb0 msgr2=0x7f154c143530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:35.289 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.283+0000 7f1552a93700 1 --2- 172.21.15.162:0/1635009109 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f154c05bcb0 0x7f154c143530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:35.289 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.283+0000 7f1552a93700 1 -- 172.21.15.162:0/1635009109 >> v1:172.21.15.162:6789/0 conn(0x7f154c0b1c00 legacy=0x7f154c0b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:35.289 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.283+0000 7f1552a93700 1 -- 172.21.15.162:0/1635009109 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f154c05c090 con 0x7f154c13c630 2020-11-28T21:55:35.290 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.283+0000 7f1552a93700 1 --2- 172.21.15.162:0/1635009109 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f154c13c630 0x7f154c13cac0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7f15440032b0 tx=0x7f15440091a0).ready entity=mon.0 client_cookie=ba58f38f625ba251 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:35.293 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.283+0000 7f1551290700 1 -- 172.21.15.162:0/1635009109 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f154400b940 con 0x7f154c13c630 2020-11-28T21:55:35.293 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.283+0000 7f1551290700 1 -- 172.21.15.162:0/1635009109 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f154400baa0 con 0x7f154c13c630 2020-11-28T21:55:35.293 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.283+0000 7f1551290700 1 -- 172.21.15.162:0/1635009109 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f154400bc10 con 0x7f154c13c630 2020-11-28T21:55:35.294 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.283+0000 7f1553294700 1 -- 172.21.15.162:0/1635009109 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f154c13c630 msgr2=0x7f154c13cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:35.294 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.283+0000 7f1553294700 1 --2- 172.21.15.162:0/1635009109 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f154c13c630 0x7f154c13cac0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7f15440032b0 tx=0x7f15440091a0).stop 2020-11-28T21:55:35.294 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.283+0000 7f1553294700 1 -- 172.21.15.162:0/1635009109 shutdown_connections 2020-11-28T21:55:35.294 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.283+0000 7f1553294700 1 --2- 172.21.15.162:0/1635009109 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f154c13c630 0x7f154c13cac0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:35.295 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.283+0000 7f1553294700 1 --2- 172.21.15.162:0/1635009109 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f154c05bcb0 0x7f154c143530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:35.296 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.283+0000 7f1553294700 1 -- 172.21.15.162:0/1635009109 >> 172.21.15.162:0/1635009109 conn(0x7f154c0b5da0 msgr2=0x7f154c0b6180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:35.296 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.283+0000 7f1553294700 1 -- 172.21.15.162:0/1635009109 shutdown_connections 2020-11-28T21:55:35.296 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.283+0000 7f1553294700 1 -- 172.21.15.162:0/1635009109 wait complete. 2020-11-28T21:55:35.296 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.283+0000 7f1553294700 1 Processor -- start 2020-11-28T21:55:35.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.283+0000 7f1553294700 1 -- start start 2020-11-28T21:55:35.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.287+0000 7f1553294700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f154c05bcb0 0x7f154c060040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:35.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.287+0000 7f1553294700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f154c0b1c00 0x7f154c1366a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:35.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.287+0000 7f1553294700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f154c13c630 0x7f154c060f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:35.298 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.287+0000 7f1553294700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154c140c00 con 0x7f154c0b1c00 2020-11-28T21:55:35.298 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.287+0000 7f1553294700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154c0b55c0 con 0x7f154c05bcb0 2020-11-28T21:55:35.298 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.287+0000 7f1553294700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f154c1393f0 con 0x7f154c13c630 2020-11-28T21:55:35.298 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.287+0000 7f1552292700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f154c05bcb0 0x7f154c060040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:35.298 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.287+0000 7f1552292700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f154c05bcb0 0x7f154c060040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45028/0 (socket says 172.21.15.162:45028) 2020-11-28T21:55:35.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.287+0000 7f1552292700 1 -- 172.21.15.162:0/3182197269 learned_addr learned my addr 172.21.15.162:0/3182197269 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:35.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.287+0000 7f1551a91700 1 --2- 172.21.15.162:0/3182197269 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f154c0b1c00 0x7f154c1366a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:35.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.287+0000 7f1552a93700 1 --2- 172.21.15.162:0/3182197269 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f154c13c630 0x7f154c060f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:35.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.287+0000 7f1552292700 1 -- 172.21.15.162:0/3182197269 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f154c13c630 msgr2=0x7f154c060f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:35.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.287+0000 7f1552292700 1 --2- 172.21.15.162:0/3182197269 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f154c13c630 0x7f154c060f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:35.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.287+0000 7f1552292700 1 -- 172.21.15.162:0/3182197269 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f154c0b1c00 msgr2=0x7f154c1366a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:35.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.287+0000 7f1552292700 1 --2- 172.21.15.162:0/3182197269 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f154c0b1c00 0x7f154c1366a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:35.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.287+0000 7f1552292700 1 -- 172.21.15.162:0/3182197269 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f154c0b57f0 con 0x7f154c05bcb0 2020-11-28T21:55:35.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.287+0000 7f1552292700 1 --2- 172.21.15.162:0/3182197269 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f154c05bcb0 0x7f154c060040 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 rx=0x7f154c140eb0 tx=0x7f1548006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:35.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.287+0000 7f153b7fe700 1 -- 172.21.15.162:0/3182197269 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f154800cae0 con 0x7f154c05bcb0 2020-11-28T21:55:35.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.287+0000 7f153b7fe700 1 -- 172.21.15.162:0/3182197269 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f154800cc40 con 0x7f154c05bcb0 2020-11-28T21:55:35.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.287+0000 7f1553294700 1 -- 172.21.15.162:0/3182197269 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f154c0b5a90 con 0x7f154c05bcb0 2020-11-28T21:55:35.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.287+0000 7f153b7fe700 1 -- 172.21.15.162:0/3182197269 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15480123f0 con 0x7f154c05bcb0 2020-11-28T21:55:35.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.287+0000 7f1553294700 1 -- 172.21.15.162:0/3182197269 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f154c1373b0 con 0x7f154c05bcb0 2020-11-28T21:55:35.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.287+0000 7f1553294700 1 -- 172.21.15.162:0/3182197269 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f154c057050 con 0x7f154c05bcb0 2020-11-28T21:55:35.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.287+0000 7f153b7fe700 1 -- 172.21.15.162:0/3182197269 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1548012550 con 0x7f154c05bcb0 2020-11-28T21:55:35.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.287+0000 7f153b7fe700 1 --2- 172.21.15.162:0/3182197269 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f152c055440 0x7f152c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:35.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.287+0000 7f153b7fe700 1 -- 172.21.15.162:0/3182197269 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f154806e4a0 con 0x7f154c05bcb0 2020-11-28T21:55:35.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.287+0000 7f1551a91700 1 --2- 172.21.15.162:0/3182197269 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f152c055440 0x7f152c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:35.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.291+0000 7f1551a91700 1 --2- 172.21.15.162:0/3182197269 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f152c055440 0x7f152c057900 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7f153c00b000 tx=0x7f153c009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:35.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.295+0000 7f153b7fe700 1 -- 172.21.15.162:0/3182197269 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f15480446a0 con 0x7f154c05bcb0 2020-11-28T21:55:35.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.431+0000 7f1553294700 1 -- 172.21.15.162:0/3182197269 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f154c139530 con 0x7f154c05bcb0 2020-11-28T21:55:35.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.435+0000 7f153b7fe700 1 -- 172.21.15.162:0/3182197269 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+16148 (secure 0 0 0) 0x7f1548047cc0 con 0x7f154c05bcb0 2020-11-28T21:55:35.438 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:35.438 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":138,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:55:32.383252+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:55:30.002238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2020-11-28T21:55:30.389418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:55:35.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.435+0000 7f1553294700 1 -- 172.21.15.162:0/3182197269 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f152c055440 msgr2=0x7f152c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:35.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.435+0000 7f1553294700 1 --2- 172.21.15.162:0/3182197269 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f152c055440 0x7f152c057900 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7f153c00b000 tx=0x7f153c009040).stop 2020-11-28T21:55:35.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.435+0000 7f1553294700 1 -- 172.21.15.162:0/3182197269 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f154c05bcb0 msgr2=0x7f154c060040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:35.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.435+0000 7f1553294700 1 --2- 172.21.15.162:0/3182197269 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f154c05bcb0 0x7f154c060040 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 rx=0x7f154c140eb0 tx=0x7f1548006dc0).stop 2020-11-28T21:55:35.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.435+0000 7f1553294700 1 -- 172.21.15.162:0/3182197269 shutdown_connections 2020-11-28T21:55:35.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.435+0000 7f1553294700 1 --2- 172.21.15.162:0/3182197269 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f152c055440 0x7f152c057900 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:35.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.435+0000 7f1553294700 1 --2- 172.21.15.162:0/3182197269 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f154c13c630 0x7f154c060f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:35.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.435+0000 7f1553294700 1 --2- 172.21.15.162:0/3182197269 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f154c05bcb0 0x7f154c060040 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:35.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.435+0000 7f1553294700 1 --2- 172.21.15.162:0/3182197269 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f154c0b1c00 0x7f154c1366a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:35.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.435+0000 7f1553294700 1 -- 172.21.15.162:0/3182197269 >> 172.21.15.162:0/3182197269 conn(0x7f154c0b5da0 msgr2=0x7f154c13b140 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:35.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.435+0000 7f1553294700 1 -- 172.21.15.162:0/3182197269 shutdown_connections 2020-11-28T21:55:35.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.435+0000 7f1553294700 1 -- 172.21.15.162:0/3182197269 wait complete. 2020-11-28T21:55:35.455 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-28T21:55:30.002238+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '137', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2020-11-28T21:55:30.389418+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '138', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-28T21:55:35.457 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:55:35.569 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.563+0000 7ff6c0b78700 1 Processor -- start 2020-11-28T21:55:35.569 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.563+0000 7ff6c0b78700 1 -- start start 2020-11-28T21:55:35.571 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.567+0000 7ff6c0b78700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6bc0b3c90 0x7ff6bc142a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:35.572 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.567+0000 7ff6c0b78700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6bc0aac00 con 0x7ff6bc05f380 2020-11-28T21:55:35.572 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.567+0000 7ff6c0b78700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6bc060a10 con 0x7ff6bc0b18c0 2020-11-28T21:55:35.572 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.567+0000 7ff6c0b78700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6bc13cf80 con 0x7ff6bc0b3c90 2020-11-28T21:55:35.572 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.567+0000 7ff6baffd700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7ff6bc0b18c0 0x7ff6bc142f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51880/0 (socket says 172.21.15.162:51880) 2020-11-28T21:55:35.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.567+0000 7ff6baffd700 1 -- 172.21.15.162:0/2793103680 learned_addr learned my addr 172.21.15.162:0/2793103680 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:35.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.567+0000 7ff6bb7fe700 1 --2- 172.21.15.162:0/2793103680 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6bc0b3c90 0x7ff6bc142a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:35.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.567+0000 7ff6ba7fc700 1 -- 172.21.15.162:0/2793103680 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3219961288 0 0) 0x7ff6bc060a10 con 0x7ff6bc0b18c0 2020-11-28T21:55:35.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.567+0000 7ff6ba7fc700 1 -- 172.21.15.162:0/2793103680 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6a8003230 con 0x7ff6bc0b18c0 2020-11-28T21:55:35.574 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.567+0000 7ff6ba7fc700 1 -- 172.21.15.162:0/2793103680 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 804018301 0 0) 0x7ff6bc0aac00 con 0x7ff6bc05f380 2020-11-28T21:55:35.574 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.567+0000 7ff6ba7fc700 1 -- 172.21.15.162:0/2793103680 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6bc060a10 con 0x7ff6bc05f380 2020-11-28T21:55:35.574 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.567+0000 7ff6ba7fc700 1 -- 172.21.15.162:0/2793103680 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2015530367 0 0) 0x7ff6a8003230 con 0x7ff6bc0b18c0 2020-11-28T21:55:35.574 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.567+0000 7ff6bb7fe700 1 -- 172.21.15.162:0/2793103680 >> v1:172.21.15.162:6789/0 conn(0x7ff6bc0b18c0 legacy=0x7ff6bc142f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:35.575 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.567+0000 7ff6bb7fe700 1 -- 172.21.15.162:0/2793103680 >> v1:172.21.15.118:6789/0 conn(0x7ff6bc05f380 legacy=0x7ff6bc146050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:35.575 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.567+0000 7ff6bb7fe700 1 -- 172.21.15.162:0/2793103680 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6bc0b4070 con 0x7ff6bc0b3c90 2020-11-28T21:55:35.575 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.567+0000 7ff6bb7fe700 1 --2- 172.21.15.162:0/2793103680 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6bc0b3c90 0x7ff6bc142a80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7ff6b00010d0 tx=0x7ff6ac006dc0).ready entity=mon.2 client_cookie=9b29db74456877e1 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:35.578 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.567+0000 7ff6ba7fc700 1 -- 172.21.15.162:0/2793103680 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6b00016d0 con 0x7ff6bc0b3c90 2020-11-28T21:55:35.578 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.567+0000 7ff6ba7fc700 1 -- 172.21.15.162:0/2793103680 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6ac00d5c0 con 0x7ff6bc0b3c90 2020-11-28T21:55:35.578 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.567+0000 7ff6ba7fc700 1 -- 172.21.15.162:0/2793103680 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6ac00d730 con 0x7ff6bc0b3c90 2020-11-28T21:55:35.578 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.567+0000 7ff6c0b78700 1 -- 172.21.15.162:0/2793103680 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6bc0b3c90 msgr2=0x7ff6bc142a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:35.579 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.567+0000 7ff6c0b78700 1 --2- 172.21.15.162:0/2793103680 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6bc0b3c90 0x7ff6bc142a80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7ff6b00010d0 tx=0x7ff6ac006dc0).stop 2020-11-28T21:55:35.579 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.567+0000 7ff6c0b78700 1 -- 172.21.15.162:0/2793103680 shutdown_connections 2020-11-28T21:55:35.579 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.567+0000 7ff6c0b78700 1 --2- 172.21.15.162:0/2793103680 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6bc0b3c90 0x7ff6bc142a80 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:35.579 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.567+0000 7ff6c0b78700 1 -- 172.21.15.162:0/2793103680 >> 172.21.15.162:0/2793103680 conn(0x7ff6bc062630 msgr2=0x7ff6bc062a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:35.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.567+0000 7ff6c0b78700 1 -- 172.21.15.162:0/2793103680 shutdown_connections 2020-11-28T21:55:35.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.567+0000 7ff6c0b78700 1 -- 172.21.15.162:0/2793103680 wait complete. 2020-11-28T21:55:35.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.571+0000 7ff6c0b78700 1 Processor -- start 2020-11-28T21:55:35.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.571+0000 7ff6c0b78700 1 -- start start 2020-11-28T21:55:35.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.571+0000 7ff6c0b78700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff6bc05f380 0x7ff6bc145d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:35.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.571+0000 7ff6c0b78700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6bc0b18c0 0x7ff6bc13ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:35.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.571+0000 7ff6c0b78700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6bc0b3c90 0x7ff6bc13f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:35.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.571+0000 7ff6c0b78700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6bc1457e0 con 0x7ff6bc05f380 2020-11-28T21:55:35.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.571+0000 7ff6c0b78700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6bc13a460 con 0x7ff6bc0b3c90 2020-11-28T21:55:35.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.571+0000 7ff6c0b78700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6bc145540 con 0x7ff6bc0b18c0 2020-11-28T21:55:35.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.571+0000 7ff6bbfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6bc0b3c90 0x7ff6bc13f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:35.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.571+0000 7ff6bbfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6bc0b3c90 0x7ff6bc13f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45044/0 (socket says 172.21.15.162:45044) 2020-11-28T21:55:35.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.571+0000 7ff6bbfff700 1 -- 172.21.15.162:0/1158185919 learned_addr learned my addr 172.21.15.162:0/1158185919 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:35.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.571+0000 7ff6baffd700 1 --2- 172.21.15.162:0/1158185919 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6bc0b18c0 0x7ff6bc13ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:35.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.571+0000 7ff6bb7fe700 1 --2- 172.21.15.162:0/1158185919 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff6bc05f380 0x7ff6bc145d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:35.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.571+0000 7ff6bbfff700 1 -- 172.21.15.162:0/1158185919 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6bc0b18c0 msgr2=0x7ff6bc13ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:35.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.571+0000 7ff6bbfff700 1 --2- 172.21.15.162:0/1158185919 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6bc0b18c0 0x7ff6bc13ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:35.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.571+0000 7ff6bbfff700 1 -- 172.21.15.162:0/1158185919 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff6bc05f380 msgr2=0x7ff6bc145d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:35.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.571+0000 7ff6bbfff700 1 --2- 172.21.15.162:0/1158185919 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff6bc05f380 0x7ff6bc145d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:35.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.571+0000 7ff6bbfff700 1 -- 172.21.15.162:0/1158185919 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6bc13fff0 con 0x7ff6bc0b3c90 2020-11-28T21:55:35.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.571+0000 7ff6bbfff700 1 --2- 172.21.15.162:0/1158185919 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6bc0b3c90 0x7ff6bc13f350 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 rx=0x7ff6b4001450 tx=0x7ff6b4007da0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:35.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.571+0000 7ff6b8ff9700 1 -- 172.21.15.162:0/1158185919 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6b4002240 con 0x7ff6bc0b3c90 2020-11-28T21:55:35.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.571+0000 7ff6b8ff9700 1 -- 172.21.15.162:0/1158185919 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6b40023a0 con 0x7ff6bc0b3c90 2020-11-28T21:55:35.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.575+0000 7ff6c0b78700 1 -- 172.21.15.162:0/1158185919 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6bc13d0d0 con 0x7ff6bc0b3c90 2020-11-28T21:55:35.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.575+0000 7ff6b8ff9700 1 -- 172.21.15.162:0/1158185919 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6b4001800 con 0x7ff6bc0b3c90 2020-11-28T21:55:35.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.575+0000 7ff6c0b78700 1 -- 172.21.15.162:0/1158185919 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6bc1452e0 con 0x7ff6bc0b3c90 2020-11-28T21:55:35.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.575+0000 7ff6c0b78700 1 -- 172.21.15.162:0/1158185919 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6bc057050 con 0x7ff6bc0b3c90 2020-11-28T21:55:35.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.575+0000 7ff6b8ff9700 1 -- 172.21.15.162:0/1158185919 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff6b4019030 con 0x7ff6bc0b3c90 2020-11-28T21:55:35.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.575+0000 7ff6b8ff9700 1 --2- 172.21.15.162:0/1158185919 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff69c055440 0x7ff69c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:35.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.575+0000 7ff6b8ff9700 1 -- 172.21.15.162:0/1158185919 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff6b406d6a0 con 0x7ff6bc0b3c90 2020-11-28T21:55:35.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.575+0000 7ff6bb7fe700 1 --2- 172.21.15.162:0/1158185919 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff69c055440 0x7ff69c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:35.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.575+0000 7ff6bb7fe700 1 --2- 172.21.15.162:0/1158185919 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff69c055440 0x7ff69c057900 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7ff6ac01b000 tx=0x7ff6ac015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:35.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.583+0000 7ff6b8ff9700 1 -- 172.21.15.162:0/1158185919 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff6b40439a0 con 0x7ff6bc0b3c90 2020-11-28T21:55:35.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.723+0000 7ff6c0b78700 1 -- 172.21.15.162:0/1158185919 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7ff6bc060030 con 0x7ff6bc0b3c90 2020-11-28T21:55:35.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.723+0000 7ff6b8ff9700 1 -- 172.21.15.162:0/1158185919 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v138) v1 ==== 110+0+10 (secure 0 0 0) 0x7ff6b4046fc0 con 0x7ff6bc0b3c90 2020-11-28T21:55:35.729 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:55:35.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.727+0000 7ff6c0b78700 1 -- 172.21.15.162:0/1158185919 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff69c055440 msgr2=0x7ff69c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:35.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.727+0000 7ff6c0b78700 1 --2- 172.21.15.162:0/1158185919 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff69c055440 0x7ff69c057900 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7ff6ac01b000 tx=0x7ff6ac015000).stop 2020-11-28T21:55:35.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.727+0000 7ff6c0b78700 1 -- 172.21.15.162:0/1158185919 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6bc0b3c90 msgr2=0x7ff6bc13f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:35.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.727+0000 7ff6c0b78700 1 --2- 172.21.15.162:0/1158185919 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6bc0b3c90 0x7ff6bc13f350 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 rx=0x7ff6b4001450 tx=0x7ff6b4007da0).stop 2020-11-28T21:55:35.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.727+0000 7ff6c0b78700 1 -- 172.21.15.162:0/1158185919 shutdown_connections 2020-11-28T21:55:35.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.727+0000 7ff6c0b78700 1 --2- 172.21.15.162:0/1158185919 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff69c055440 0x7ff69c057900 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:35.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.727+0000 7ff6c0b78700 1 --2- 172.21.15.162:0/1158185919 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6bc0b3c90 0x7ff6bc13f350 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:35.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.727+0000 7ff6c0b78700 1 --2- 172.21.15.162:0/1158185919 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff6bc05f380 0x7ff6bc145d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:35.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.727+0000 7ff6c0b78700 1 --2- 172.21.15.162:0/1158185919 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6bc0b18c0 0x7ff6bc13ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:35.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.727+0000 7ff6c0b78700 1 -- 172.21.15.162:0/1158185919 >> 172.21.15.162:0/1158185919 conn(0x7ff6bc062630 msgr2=0x7ff6bc13a810 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:35.733 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.727+0000 7ff6c0b78700 1 -- 172.21.15.162:0/1158185919 shutdown_connections 2020-11-28T21:55:35.733 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.727+0000 7ff6c0b78700 1 -- 172.21.15.162:0/1158185919 wait complete. 2020-11-28T21:55:35.742 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:55:35.862 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.859+0000 7f0c95e5c700 1 Processor -- start 2020-11-28T21:55:35.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.859+0000 7f0c95e5c700 1 -- start start 2020-11-28T21:55:35.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.859+0000 7f0c95e5c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f0c901433c0 0x7f0c900605c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:35.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.859+0000 7f0c95e5c700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c90143950 con 0x7f0c900b9ce0 2020-11-28T21:55:35.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.859+0000 7f0c95e5c700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c900ba0c0 con 0x7f0c90067610 2020-11-28T21:55:35.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.859+0000 7f0c95e5c700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c900b4340 con 0x7f0c901433c0 2020-11-28T21:55:35.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.859+0000 7f0c8ffff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f0c90067610 0x7f0c9013fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51896/0 (socket says 172.21.15.162:51896) 2020-11-28T21:55:35.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.859+0000 7f0c8ffff700 1 -- 172.21.15.162:0/1951684062 learned_addr learned my addr 172.21.15.162:0/1951684062 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:35.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.859+0000 7f0c8effd700 1 --2- 172.21.15.162:0/1951684062 >> v2:172.21.15.118:3300/0 conn(0x7f0c901433c0 0x7f0c900605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:35.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.859+0000 7f0c8e7fc700 1 -- 172.21.15.162:0/1951684062 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4190410189 0 0) 0x7f0c900ba0c0 con 0x7f0c90067610 2020-11-28T21:55:35.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.859+0000 7f0c8e7fc700 1 -- 172.21.15.162:0/1951684062 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c7c003230 con 0x7f0c90067610 2020-11-28T21:55:35.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.859+0000 7f0c8e7fc700 1 -- 172.21.15.162:0/1951684062 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3858125722 0 0) 0x7f0c90143950 con 0x7f0c900b9ce0 2020-11-28T21:55:35.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.859+0000 7f0c8e7fc700 1 -- 172.21.15.162:0/1951684062 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c900ba0c0 con 0x7f0c900b9ce0 2020-11-28T21:55:35.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.859+0000 7f0c8effd700 1 -- 172.21.15.162:0/1951684062 >> v1:172.21.15.162:6789/0 conn(0x7f0c90067610 legacy=0x7f0c9013fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:35.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.859+0000 7f0c8effd700 1 -- 172.21.15.162:0/1951684062 >> v1:172.21.15.118:6789/0 conn(0x7f0c900b9ce0 legacy=0x7f0c900b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:35.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.859+0000 7f0c8e7fc700 1 -- 172.21.15.162:0/1951684062 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1571310400 0 0) 0x7f0c7c003230 con 0x7f0c90067610 2020-11-28T21:55:35.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.859+0000 7f0c8effd700 1 -- 172.21.15.162:0/1951684062 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c90066c30 con 0x7f0c901433c0 2020-11-28T21:55:35.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.859+0000 7f0c8effd700 1 --2- 172.21.15.162:0/1951684062 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0c901433c0 0x7f0c900605c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7f0c84002f30 tx=0x7f0c84006dc0).ready entity=mon.0 client_cookie=b1a8e6d8c8f68927 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:35.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.859+0000 7f0c8e7fc700 1 -- 172.21.15.162:0/1951684062 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c84009740 con 0x7f0c901433c0 2020-11-28T21:55:35.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.859+0000 7f0c8e7fc700 1 -- 172.21.15.162:0/1951684062 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c840098a0 con 0x7f0c901433c0 2020-11-28T21:55:35.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.859+0000 7f0c8e7fc700 1 -- 172.21.15.162:0/1951684062 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c84009c00 con 0x7f0c901433c0 2020-11-28T21:55:35.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.859+0000 7f0c95e5c700 1 -- 172.21.15.162:0/1951684062 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0c901433c0 msgr2=0x7f0c900605c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:35.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.859+0000 7f0c95e5c700 1 --2- 172.21.15.162:0/1951684062 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0c901433c0 0x7f0c900605c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7f0c84002f30 tx=0x7f0c84006dc0).stop 2020-11-28T21:55:35.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.863+0000 7f0c95e5c700 1 -- 172.21.15.162:0/1951684062 shutdown_connections 2020-11-28T21:55:35.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.863+0000 7f0c95e5c700 1 --2- 172.21.15.162:0/1951684062 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0c901433c0 0x7f0c900605c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:35.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.863+0000 7f0c95e5c700 1 -- 172.21.15.162:0/1951684062 >> 172.21.15.162:0/1951684062 conn(0x7f0c9005c430 msgr2=0x7f0c900b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:35.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.863+0000 7f0c95e5c700 1 -- 172.21.15.162:0/1951684062 shutdown_connections 2020-11-28T21:55:35.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.863+0000 7f0c95e5c700 1 -- 172.21.15.162:0/1951684062 wait complete. 2020-11-28T21:55:35.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.863+0000 7f0c95e5c700 1 Processor -- start 2020-11-28T21:55:35.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.863+0000 7f0c95e5c700 1 -- start start 2020-11-28T21:55:35.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.863+0000 7f0c95e5c700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0c90067610 0x7f0c900ac930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:35.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.863+0000 7f0c95e5c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0c900b9ce0 0x7f0c90155130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:35.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.863+0000 7f0c95e5c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c901433c0 0x7f0c900ae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:35.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.863+0000 7f0c95e5c700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c900adec0 con 0x7f0c900b9ce0 2020-11-28T21:55:35.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.863+0000 7f0c95e5c700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c900ace20 con 0x7f0c901433c0 2020-11-28T21:55:35.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.863+0000 7f0c95e5c700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c90155900 con 0x7f0c90067610 2020-11-28T21:55:35.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.863+0000 7f0c8ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c901433c0 0x7f0c900ae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:35.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.863+0000 7f0c8f7fe700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0c90067610 0x7f0c900ac930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:35.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.863+0000 7f0c8ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c901433c0 0x7f0c900ae2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45058/0 (socket says 172.21.15.162:45058) 2020-11-28T21:55:35.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.863+0000 7f0c8ffff700 1 -- 172.21.15.162:0/2531373540 learned_addr learned my addr 172.21.15.162:0/2531373540 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:35.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.863+0000 7f0c8effd700 1 --2- 172.21.15.162:0/2531373540 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0c900b9ce0 0x7f0c90155130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:35.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.867+0000 7f0c8ffff700 1 -- 172.21.15.162:0/2531373540 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0c90067610 msgr2=0x7f0c900ac930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:35.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.867+0000 7f0c8ffff700 1 --2- 172.21.15.162:0/2531373540 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0c90067610 0x7f0c900ac930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:35.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.867+0000 7f0c8ffff700 1 -- 172.21.15.162:0/2531373540 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0c900b9ce0 msgr2=0x7f0c90155130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:35.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.867+0000 7f0c8ffff700 1 --2- 172.21.15.162:0/2531373540 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0c900b9ce0 0x7f0c90155130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:35.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.867+0000 7f0c8ffff700 1 -- 172.21.15.162:0/2531373540 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c90155ba0 con 0x7f0c901433c0 2020-11-28T21:55:35.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.867+0000 7f0c8ffff700 1 --2- 172.21.15.162:0/2531373540 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c901433c0 0x7f0c900ae2b0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 rx=0x7f0c88000e90 tx=0x7f0c88001740).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:35.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.867+0000 7f0c8cff9700 1 -- 172.21.15.162:0/2531373540 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c88007170 con 0x7f0c901433c0 2020-11-28T21:55:35.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.867+0000 7f0c8cff9700 1 -- 172.21.15.162:0/2531373540 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c880072d0 con 0x7f0c901433c0 2020-11-28T21:55:35.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.867+0000 7f0c95e5c700 1 -- 172.21.15.162:0/2531373540 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c90155620 con 0x7f0c901433c0 2020-11-28T21:55:35.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.867+0000 7f0c8cff9700 1 -- 172.21.15.162:0/2531373540 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c880048f0 con 0x7f0c901433c0 2020-11-28T21:55:35.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.867+0000 7f0c95e5c700 1 -- 172.21.15.162:0/2531373540 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c9013fef0 con 0x7f0c901433c0 2020-11-28T21:55:35.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.867+0000 7f0c8cff9700 1 -- 172.21.15.162:0/2531373540 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0c88004a50 con 0x7f0c901433c0 2020-11-28T21:55:35.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.867+0000 7f0c95e5c700 1 -- 172.21.15.162:0/2531373540 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c90155d30 con 0x7f0c901433c0 2020-11-28T21:55:35.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.867+0000 7f0c8cff9700 1 --2- 172.21.15.162:0/2531373540 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0c700554e0 0x7f0c700579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:35.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.867+0000 7f0c8f7fe700 1 --2- 172.21.15.162:0/2531373540 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0c700554e0 0x7f0c700579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:35.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.867+0000 7f0c8cff9700 1 -- 172.21.15.162:0/2531373540 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0c8806d940 con 0x7f0c901433c0 2020-11-28T21:55:35.886 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.867+0000 7f0c8f7fe700 1 --2- 172.21.15.162:0/2531373540 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0c700554e0 0x7f0c700579a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7f0c8000d000 tx=0x7f0c80007d60).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:35.886 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:35.875+0000 7f0c8cff9700 1 -- 172.21.15.162:0/2531373540 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0c88043b30 con 0x7f0c901433c0 2020-11-28T21:55:36.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.019+0000 7f0c95e5c700 1 -- 172.21.15.162:0/2531373540 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0c90057050 con 0x7f0c901433c0 2020-11-28T21:55:36.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.019+0000 7f0c8cff9700 1 -- 172.21.15.162:0/2531373540 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v138) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0c88001860 con 0x7f0c901433c0 2020-11-28T21:55:36.024 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:55:36.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.023+0000 7f0c95e5c700 1 -- 172.21.15.162:0/2531373540 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0c700554e0 msgr2=0x7f0c700579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:36.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.023+0000 7f0c95e5c700 1 --2- 172.21.15.162:0/2531373540 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0c700554e0 0x7f0c700579a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7f0c8000d000 tx=0x7f0c80007d60).stop 2020-11-28T21:55:36.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.023+0000 7f0c95e5c700 1 -- 172.21.15.162:0/2531373540 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c901433c0 msgr2=0x7f0c900ae2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:36.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.023+0000 7f0c95e5c700 1 --2- 172.21.15.162:0/2531373540 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c901433c0 0x7f0c900ae2b0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 rx=0x7f0c88000e90 tx=0x7f0c88001740).stop 2020-11-28T21:55:36.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.023+0000 7f0c95e5c700 1 -- 172.21.15.162:0/2531373540 shutdown_connections 2020-11-28T21:55:36.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.023+0000 7f0c95e5c700 1 --2- 172.21.15.162:0/2531373540 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0c700554e0 0x7f0c700579a0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:36.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.023+0000 7f0c95e5c700 1 --2- 172.21.15.162:0/2531373540 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0c90067610 0x7f0c900ac930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:36.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.023+0000 7f0c95e5c700 1 --2- 172.21.15.162:0/2531373540 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c901433c0 0x7f0c900ae2b0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:36.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.023+0000 7f0c95e5c700 1 --2- 172.21.15.162:0/2531373540 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0c900b9ce0 0x7f0c90155130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:36.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.023+0000 7f0c95e5c700 1 -- 172.21.15.162:0/2531373540 >> 172.21.15.162:0/2531373540 conn(0x7f0c9005c430 msgr2=0x7f0c9005f750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:36.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.023+0000 7f0c95e5c700 1 -- 172.21.15.162:0/2531373540 shutdown_connections 2020-11-28T21:55:36.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.023+0000 7f0c95e5c700 1 -- 172.21.15.162:0/2531373540 wait complete. 2020-11-28T21:55:36.043 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-28T21:55:36.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.155+0000 7ff8bbfff700 1 Processor -- start 2020-11-28T21:55:36.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.155+0000 7ff8bbfff700 1 -- start start 2020-11-28T21:55:36.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.159+0000 7ff8bbfff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff8bc13d0c0 0x7ff8bc0b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:36.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.159+0000 7ff8bbfff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff8bc0b13c0 0x7ff8bc099b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:36.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.159+0000 7ff8bbfff700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8bc06aca0 con 0x7ff8bc09a050 2020-11-28T21:55:36.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.159+0000 7ff8bbfff700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8bc13d4a0 con 0x7ff8bc13d0c0 2020-11-28T21:55:36.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.159+0000 7ff8bbfff700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8bc098da0 con 0x7ff8bc0b13c0 2020-11-28T21:55:36.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.159+0000 7ff8bb7fe700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7ff8bc09a050 0x7ff8bc05bf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51910/0 (socket says 172.21.15.162:51910) 2020-11-28T21:55:36.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.159+0000 7ff8bb7fe700 1 -- 172.21.15.162:0/3742761581 learned_addr learned my addr 172.21.15.162:0/3742761581 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:36.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.159+0000 7ff8ba7fc700 1 --2- 172.21.15.162:0/3742761581 >> v2:172.21.15.162:3300/0 conn(0x7ff8bc0b13c0 0x7ff8bc099b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:36.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.159+0000 7ff8baffd700 1 --2- 172.21.15.162:0/3742761581 >> v2:172.21.15.118:3300/0 conn(0x7ff8bc13d0c0 0x7ff8bc0b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:36.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.159+0000 7ff8b9ffb700 1 -- 172.21.15.162:0/3742761581 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1422399964 0 0) 0x7ff8bc06aca0 con 0x7ff8bc09a050 2020-11-28T21:55:36.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.159+0000 7ff8b9ffb700 1 -- 172.21.15.162:0/3742761581 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff89c003230 con 0x7ff8bc09a050 2020-11-28T21:55:36.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.159+0000 7ff8ba7fc700 1 -- 172.21.15.162:0/3742761581 >> v2:172.21.15.118:3300/0 conn(0x7ff8bc13d0c0 msgr2=0x7ff8bc0b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:36.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.159+0000 7ff8ba7fc700 1 --2- 172.21.15.162:0/3742761581 >> v2:172.21.15.118:3300/0 conn(0x7ff8bc13d0c0 0x7ff8bc0b0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:36.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.159+0000 7ff8ba7fc700 1 -- 172.21.15.162:0/3742761581 >> v1:172.21.15.162:6789/0 conn(0x7ff8bc09a050 legacy=0x7ff8bc05bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:36.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.159+0000 7ff8ba7fc700 1 -- 172.21.15.162:0/3742761581 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8bc06af40 con 0x7ff8bc0b13c0 2020-11-28T21:55:36.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.159+0000 7ff8ba7fc700 1 --2- 172.21.15.162:0/3742761581 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8bc0b13c0 0x7ff8bc099b60 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 rx=0x7ff8a4002f30 tx=0x7ff8a4006dc0).ready entity=mon.1 client_cookie=b45fa1c2936e0fee server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:36.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.159+0000 7ff8b9ffb700 1 -- 172.21.15.162:0/3742761581 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8a40096f0 con 0x7ff8bc0b13c0 2020-11-28T21:55:36.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.159+0000 7ff8b9ffb700 1 -- 172.21.15.162:0/3742761581 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8a4009850 con 0x7ff8bc0b13c0 2020-11-28T21:55:36.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.159+0000 7ff8b9ffb700 1 -- 172.21.15.162:0/3742761581 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8a4009bb0 con 0x7ff8bc0b13c0 2020-11-28T21:55:36.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.159+0000 7ff8bbfff700 1 -- 172.21.15.162:0/3742761581 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8bc0b13c0 msgr2=0x7ff8bc099b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:36.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.159+0000 7ff8bbfff700 1 --2- 172.21.15.162:0/3742761581 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8bc0b13c0 0x7ff8bc099b60 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 rx=0x7ff8a4002f30 tx=0x7ff8a4006dc0).stop 2020-11-28T21:55:36.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.159+0000 7ff8bbfff700 1 -- 172.21.15.162:0/3742761581 shutdown_connections 2020-11-28T21:55:36.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.159+0000 7ff8bbfff700 1 --2- 172.21.15.162:0/3742761581 >> v2:172.21.15.118:3300/0 conn(0x7ff8bc13d0c0 0x7ff8bc0b0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:36.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.159+0000 7ff8bbfff700 1 --2- 172.21.15.162:0/3742761581 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8bc0b13c0 0x7ff8bc099b60 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:36.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.159+0000 7ff8bbfff700 1 -- 172.21.15.162:0/3742761581 >> 172.21.15.162:0/3742761581 conn(0x7ff8bc0b5520 msgr2=0x7ff8bc0b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:36.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.159+0000 7ff8bbfff700 1 -- 172.21.15.162:0/3742761581 shutdown_connections 2020-11-28T21:55:36.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.159+0000 7ff8bbfff700 1 -- 172.21.15.162:0/3742761581 wait complete. 2020-11-28T21:55:36.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.163+0000 7ff8bbfff700 1 Processor -- start 2020-11-28T21:55:36.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.163+0000 7ff8bbfff700 1 -- start start 2020-11-28T21:55:36.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.163+0000 7ff8bbfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8bc09a050 0x7ff8bc0b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:36.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.163+0000 7ff8bbfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8bc0b13c0 0x7ff8bc14eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:36.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.163+0000 7ff8bbfff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff8bc13d0c0 0x7ff8bc150ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:36.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.163+0000 7ff8bbfff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc0b4e40 con 0x7ff8bc0b13c0 2020-11-28T21:55:36.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.163+0000 7ff8bbfff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc0b5070 con 0x7ff8bc09a050 2020-11-28T21:55:36.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.163+0000 7ff8bbfff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc14e600 con 0x7ff8bc13d0c0 2020-11-28T21:55:36.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.163+0000 7ff8baffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8bc09a050 0x7ff8bc0b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:36.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.163+0000 7ff8baffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8bc09a050 0x7ff8bc0b43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45068/0 (socket says 172.21.15.162:45068) 2020-11-28T21:55:36.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.163+0000 7ff8baffd700 1 -- 172.21.15.162:0/619067274 learned_addr learned my addr 172.21.15.162:0/619067274 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:36.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.163+0000 7ff8ba7fc700 1 --2- 172.21.15.162:0/619067274 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8bc0b13c0 0x7ff8bc14eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:36.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.163+0000 7ff8bb7fe700 1 --2- 172.21.15.162:0/619067274 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff8bc13d0c0 0x7ff8bc150ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:36.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.167+0000 7ff8baffd700 1 -- 172.21.15.162:0/619067274 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff8bc13d0c0 msgr2=0x7ff8bc150ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:36.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.167+0000 7ff8baffd700 1 --2- 172.21.15.162:0/619067274 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff8bc13d0c0 0x7ff8bc150ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:36.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.167+0000 7ff8baffd700 1 -- 172.21.15.162:0/619067274 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8bc0b13c0 msgr2=0x7ff8bc14eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:36.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.167+0000 7ff8baffd700 1 --2- 172.21.15.162:0/619067274 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8bc0b13c0 0x7ff8bc14eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:36.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.167+0000 7ff8baffd700 1 -- 172.21.15.162:0/619067274 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8bc14e330 con 0x7ff8bc09a050 2020-11-28T21:55:36.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.167+0000 7ff8baffd700 1 --2- 172.21.15.162:0/619067274 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8bc09a050 0x7ff8bc0b43f0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 rx=0x7ff8ac00c000 tx=0x7ff8ac0091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:36.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.167+0000 7ff8b3fff700 1 -- 172.21.15.162:0/619067274 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8ac00c8f0 con 0x7ff8bc09a050 2020-11-28T21:55:36.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.167+0000 7ff8b3fff700 1 -- 172.21.15.162:0/619067274 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8ac00ca50 con 0x7ff8bc09a050 2020-11-28T21:55:36.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.167+0000 7ff8b3fff700 1 -- 172.21.15.162:0/619067274 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8ac00cdb0 con 0x7ff8bc09a050 2020-11-28T21:55:36.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.167+0000 7ff8bbfff700 1 -- 172.21.15.162:0/619067274 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8bc13cb80 con 0x7ff8bc09a050 2020-11-28T21:55:36.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.167+0000 7ff8bbfff700 1 -- 172.21.15.162:0/619067274 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8bc140160 con 0x7ff8bc09a050 2020-11-28T21:55:36.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.167+0000 7ff8bbfff700 1 -- 172.21.15.162:0/619067274 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8bc14e7c0 con 0x7ff8bc09a050 2020-11-28T21:55:36.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.167+0000 7ff8b3fff700 1 -- 172.21.15.162:0/619067274 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff8ac022070 con 0x7ff8bc09a050 2020-11-28T21:55:36.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.167+0000 7ff8b3fff700 1 --2- 172.21.15.162:0/619067274 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff8a00554e0 0x7ff8a00579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:36.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.167+0000 7ff8b3fff700 1 -- 172.21.15.162:0/619067274 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff8ac06e520 con 0x7ff8bc09a050 2020-11-28T21:55:36.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.167+0000 7ff8ba7fc700 1 --2- 172.21.15.162:0/619067274 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff8a00554e0 0x7ff8a00579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:36.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.167+0000 7ff8ba7fc700 1 --2- 172.21.15.162:0/619067274 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff8a00554e0 0x7ff8a00579a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7ff8a401b000 tx=0x7ff8a4015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:36.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.179+0000 7ff8b3fff700 1 -- 172.21.15.162:0/619067274 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff8ac044820 con 0x7ff8bc09a050 2020-11-28T21:55:36.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.311+0000 7ff8bbfff700 1 -- 172.21.15.162:0/619067274 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff8bc057050 con 0x7ff8bc09a050 2020-11-28T21:55:36.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.311+0000 7ff8b3fff700 1 -- 172.21.15.162:0/619067274 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v138) v1 ==== 104+0+10 (secure 0 0 0) 0x7ff8ac047e40 con 0x7ff8bc09a050 2020-11-28T21:55:36.317 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:55:36.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.315+0000 7ff8bbfff700 1 -- 172.21.15.162:0/619067274 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff8a00554e0 msgr2=0x7ff8a00579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:36.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.315+0000 7ff8bbfff700 1 --2- 172.21.15.162:0/619067274 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff8a00554e0 0x7ff8a00579a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7ff8a401b000 tx=0x7ff8a4015000).stop 2020-11-28T21:55:36.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.315+0000 7ff8bbfff700 1 -- 172.21.15.162:0/619067274 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8bc09a050 msgr2=0x7ff8bc0b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:36.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.315+0000 7ff8bbfff700 1 --2- 172.21.15.162:0/619067274 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8bc09a050 0x7ff8bc0b43f0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 rx=0x7ff8ac00c000 tx=0x7ff8ac0091a0).stop 2020-11-28T21:55:36.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.315+0000 7ff8bbfff700 1 -- 172.21.15.162:0/619067274 shutdown_connections 2020-11-28T21:55:36.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.315+0000 7ff8bbfff700 1 --2- 172.21.15.162:0/619067274 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff8a00554e0 0x7ff8a00579a0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:36.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.315+0000 7ff8bbfff700 1 --2- 172.21.15.162:0/619067274 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff8bc13d0c0 0x7ff8bc150ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:36.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.315+0000 7ff8bbfff700 1 --2- 172.21.15.162:0/619067274 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8bc09a050 0x7ff8bc0b43f0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:36.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.315+0000 7ff8bbfff700 1 --2- 172.21.15.162:0/619067274 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8bc0b13c0 0x7ff8bc14eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:36.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.315+0000 7ff8bbfff700 1 -- 172.21.15.162:0/619067274 >> 172.21.15.162:0/619067274 conn(0x7ff8bc0b5520 msgr2=0x7ff8bc13a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:36.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.315+0000 7ff8bbfff700 1 -- 172.21.15.162:0/619067274 shutdown_connections 2020-11-28T21:55:36.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.315+0000 7ff8bbfff700 1 -- 172.21.15.162:0/619067274 wait complete. 2020-11-28T21:55:36.330 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-28T21:55:36.445 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.439+0000 7fd73c8d3700 1 Processor -- start 2020-11-28T21:55:36.445 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.439+0000 7fd73c8d3700 1 -- start start 2020-11-28T21:55:36.448 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.443+0000 7fd73c8d3700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd7380b1eb0 0x7fd7380b2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:36.449 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.443+0000 7fd73c8d3700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd7380af6a0 0x7fd738060a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:36.449 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.443+0000 7fd73c8d3700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7381471f0 con 0x7fd738144350 2020-11-28T21:55:36.449 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.443+0000 7fd73c8d3700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd738060090 con 0x7fd7380b1eb0 2020-11-28T21:55:36.449 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.443+0000 7fd73c8d3700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd738060330 con 0x7fd7380af6a0 2020-11-28T21:55:36.450 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.443+0000 7fd7377fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd7380b1eb0 0x7fd7380b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:36.450 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.443+0000 7fd7377fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd7380b1eb0 0x7fd7380b2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45078/0 (socket says 172.21.15.162:45078) 2020-11-28T21:55:36.450 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.443+0000 7fd736ffd700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd7380af6a0 0x7fd738060a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:36.451 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.443+0000 7fd7377fe700 1 -- 172.21.15.162:0/2302360138 learned_addr learned my addr 172.21.15.162:0/2302360138 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:36.451 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.443+0000 7fd7377fe700 1 -- 172.21.15.162:0/2302360138 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd7380af6a0 msgr2=0x7fd738060a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:36.452 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.443+0000 7fd7377fe700 1 --2- 172.21.15.162:0/2302360138 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd7380af6a0 0x7fd738060a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:36.452 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.443+0000 7fd7377fe700 1 -- 172.21.15.162:0/2302360138 >> v1:172.21.15.118:6789/0 conn(0x7fd738144350 legacy=0x7fd73814a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:36.452 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.443+0000 7fd7377fe700 1 -- 172.21.15.162:0/2302360138 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7380605d0 con 0x7fd7380b1eb0 2020-11-28T21:55:36.452 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.443+0000 7fd7377fe700 1 --2- 172.21.15.162:0/2302360138 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7380b1eb0 0x7fd7380b2290 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 rx=0x7fd728002f30 tx=0x7fd728006e80).ready entity=mon.1 client_cookie=69300e337f2a2987 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:36.452 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.443+0000 7fd7367fc700 1 -- 172.21.15.162:0/2302360138 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7280097f0 con 0x7fd7380b1eb0 2020-11-28T21:55:36.453 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.443+0000 7fd7367fc700 1 -- 172.21.15.162:0/2302360138 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd728009950 con 0x7fd7380b1eb0 2020-11-28T21:55:36.453 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.443+0000 7fd7367fc700 1 -- 172.21.15.162:0/2302360138 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd728009cb0 con 0x7fd7380b1eb0 2020-11-28T21:55:36.453 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.443+0000 7fd73c8d3700 1 -- 172.21.15.162:0/2302360138 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7380b1eb0 msgr2=0x7fd7380b2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:36.453 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.443+0000 7fd73c8d3700 1 --2- 172.21.15.162:0/2302360138 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7380b1eb0 0x7fd7380b2290 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 rx=0x7fd728002f30 tx=0x7fd728006e80).stop 2020-11-28T21:55:36.453 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.443+0000 7fd73c8d3700 1 -- 172.21.15.162:0/2302360138 shutdown_connections 2020-11-28T21:55:36.454 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.443+0000 7fd73c8d3700 1 --2- 172.21.15.162:0/2302360138 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7380b1eb0 0x7fd7380b2290 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:36.454 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.443+0000 7fd73c8d3700 1 --2- 172.21.15.162:0/2302360138 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd7380af6a0 0x7fd738060a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:36.454 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.443+0000 7fd73c8d3700 1 -- 172.21.15.162:0/2302360138 >> 172.21.15.162:0/2302360138 conn(0x7fd7380b57d0 msgr2=0x7fd7380b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:36.454 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.443+0000 7fd73c8d3700 1 -- 172.21.15.162:0/2302360138 shutdown_connections 2020-11-28T21:55:36.455 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.443+0000 7fd73c8d3700 1 -- 172.21.15.162:0/2302360138 wait complete. 2020-11-28T21:55:36.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.443+0000 7fd73c8d3700 1 Processor -- start 2020-11-28T21:55:36.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.443+0000 7fd73c8d3700 1 -- start start 2020-11-28T21:55:36.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.447+0000 7fd73c8d3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7380af6a0 0x7fd7381566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:36.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.447+0000 7fd73c8d3700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd7380b1eb0 0x7fd738146b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:36.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.447+0000 7fd73c8d3700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd738144350 0x7fd738063b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:36.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.447+0000 7fd73c8d3700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73815ab40 con 0x7fd738144350 2020-11-28T21:55:36.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.447+0000 7fd73c8d3700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73815a630 con 0x7fd7380af6a0 2020-11-28T21:55:36.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.447+0000 7fd73c8d3700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd73814a450 con 0x7fd7380b1eb0 2020-11-28T21:55:36.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.447+0000 7fd7377fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7380af6a0 0x7fd7381566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:36.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.447+0000 7fd7377fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7380af6a0 0x7fd7381566f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45082/0 (socket says 172.21.15.162:45082) 2020-11-28T21:55:36.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.447+0000 7fd7377fe700 1 -- 172.21.15.162:0/3421486726 learned_addr learned my addr 172.21.15.162:0/3421486726 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:36.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.447+0000 7fd736ffd700 1 --2- 172.21.15.162:0/3421486726 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd7380b1eb0 0x7fd738146b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:36.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.447+0000 7fd737fff700 1 --2- 172.21.15.162:0/3421486726 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd738144350 0x7fd738063b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:36.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.447+0000 7fd7377fe700 1 -- 172.21.15.162:0/3421486726 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd7380b1eb0 msgr2=0x7fd738146b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:36.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.447+0000 7fd7377fe700 1 --2- 172.21.15.162:0/3421486726 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd7380b1eb0 0x7fd738146b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:36.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.447+0000 7fd7377fe700 1 -- 172.21.15.162:0/3421486726 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd738144350 msgr2=0x7fd738063b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:36.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.447+0000 7fd7377fe700 1 --2- 172.21.15.162:0/3421486726 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd738144350 0x7fd738063b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:36.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.447+0000 7fd7377fe700 1 -- 172.21.15.162:0/3421486726 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd73815a8a0 con 0x7fd7380af6a0 2020-11-28T21:55:36.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.447+0000 7fd7377fe700 1 --2- 172.21.15.162:0/3421486726 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7380af6a0 0x7fd7381566f0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 rx=0x7fd738063830 tx=0x7fd728015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:36.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.447+0000 7fd734ff9700 1 -- 172.21.15.162:0/3421486726 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd728011bb0 con 0x7fd7380af6a0 2020-11-28T21:55:36.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.447+0000 7fd734ff9700 1 -- 172.21.15.162:0/3421486726 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd728011d10 con 0x7fd7380af6a0 2020-11-28T21:55:36.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.447+0000 7fd73c8d3700 1 -- 172.21.15.162:0/3421486726 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7380aae90 con 0x7fd7380af6a0 2020-11-28T21:55:36.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.447+0000 7fd734ff9700 1 -- 172.21.15.162:0/3421486726 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd72801e650 con 0x7fd7380af6a0 2020-11-28T21:55:36.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.447+0000 7fd73c8d3700 1 -- 172.21.15.162:0/3421486726 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7380abb90 con 0x7fd7380af6a0 2020-11-28T21:55:36.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.451+0000 7fd73c8d3700 1 -- 172.21.15.162:0/3421486726 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd738057050 con 0x7fd7380af6a0 2020-11-28T21:55:36.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.451+0000 7fd734ff9700 1 -- 172.21.15.162:0/3421486726 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd728025030 con 0x7fd7380af6a0 2020-11-28T21:55:36.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.451+0000 7fd734ff9700 1 --2- 172.21.15.162:0/3421486726 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd7180554e0 0x7fd7180579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:36.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.451+0000 7fd734ff9700 1 -- 172.21.15.162:0/3421486726 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd72806f110 con 0x7fd7380af6a0 2020-11-28T21:55:36.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.451+0000 7fd736ffd700 1 --2- 172.21.15.162:0/3421486726 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd7180554e0 0x7fd7180579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:36.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.451+0000 7fd736ffd700 1 --2- 172.21.15.162:0/3421486726 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd7180554e0 0x7fd7180579a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7fd72c00b000 tx=0x7fd72c006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:36.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.455+0000 7fd734ff9700 1 -- 172.21.15.162:0/3421486726 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd728045410 con 0x7fd7380af6a0 2020-11-28T21:55:36.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.599+0000 7fd73c8d3700 1 -- 172.21.15.162:0/3421486726 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd7380abb90 con 0x7fd7380af6a0 2020-11-28T21:55:36.603 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.599+0000 7fd734ff9700 1 -- 172.21.15.162:0/3421486726 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v138) v1 ==== 100+0+10 (secure 0 0 0) 0x7fd7380abb90 con 0x7fd7380af6a0 2020-11-28T21:55:36.604 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:55:36.606 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.599+0000 7fd73c8d3700 1 -- 172.21.15.162:0/3421486726 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd7180554e0 msgr2=0x7fd7180579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:36.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.599+0000 7fd73c8d3700 1 --2- 172.21.15.162:0/3421486726 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd7180554e0 0x7fd7180579a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7fd72c00b000 tx=0x7fd72c006dc0).stop 2020-11-28T21:55:36.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.599+0000 7fd73c8d3700 1 -- 172.21.15.162:0/3421486726 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7380af6a0 msgr2=0x7fd7381566f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:36.608 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.599+0000 7fd73c8d3700 1 --2- 172.21.15.162:0/3421486726 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7380af6a0 0x7fd7381566f0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 rx=0x7fd738063830 tx=0x7fd728015000).stop 2020-11-28T21:55:36.608 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.599+0000 7fd73c8d3700 1 -- 172.21.15.162:0/3421486726 shutdown_connections 2020-11-28T21:55:36.609 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.599+0000 7fd73c8d3700 1 --2- 172.21.15.162:0/3421486726 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd7180554e0 0x7fd7180579a0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:36.609 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.599+0000 7fd73c8d3700 1 --2- 172.21.15.162:0/3421486726 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd738144350 0x7fd738063b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:36.610 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.599+0000 7fd73c8d3700 1 --2- 172.21.15.162:0/3421486726 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7380af6a0 0x7fd7381566f0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:36.610 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.599+0000 7fd73c8d3700 1 --2- 172.21.15.162:0/3421486726 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd7380b1eb0 0x7fd738146b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:36.611 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.599+0000 7fd73c8d3700 1 -- 172.21.15.162:0/3421486726 >> 172.21.15.162:0/3421486726 conn(0x7fd7380b57d0 msgr2=0x7fd7380b36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:36.611 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.599+0000 7fd73c8d3700 1 -- 172.21.15.162:0/3421486726 shutdown_connections 2020-11-28T21:55:36.611 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.599+0000 7fd73c8d3700 1 -- 172.21.15.162:0/3421486726 wait complete. 2020-11-28T21:55:36.617 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-28T21:55:36.618 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:55:36.724 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.719+0000 7f23be1cd700 1 Processor -- start 2020-11-28T21:55:36.725 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.719+0000 7f23be1cd700 1 -- start start 2020-11-28T21:55:36.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.719+0000 7f23be1cd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f23b80b1990 0x7f23b8140890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:36.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.719+0000 7f23be1cd700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f23b80b11b0 0x7f23b8140d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:36.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.719+0000 7f23be1cd700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23b80b2870 con 0x7f23b80b1d70 2020-11-28T21:55:36.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.719+0000 7f23be1cd700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23b8146e20 con 0x7f23b80b11b0 2020-11-28T21:55:36.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.719+0000 7f23be1cd700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23b80b22e0 con 0x7f23b80b1990 2020-11-28T21:55:36.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.723+0000 7f23b7fff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f23b80b1d70 0x7f23b814a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51936/0 (socket says 172.21.15.162:51936) 2020-11-28T21:55:36.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.723+0000 7f23b7fff700 1 -- 172.21.15.162:0/220999305 learned_addr learned my addr 172.21.15.162:0/220999305 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:36.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.723+0000 7f23b77fe700 1 --2- 172.21.15.162:0/220999305 >> v2:172.21.15.162:3300/0 conn(0x7f23b80b1990 0x7f23b8140890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:36.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.723+0000 7f23b6ffd700 1 --2- 172.21.15.162:0/220999305 >> v2:172.21.15.118:3300/0 conn(0x7f23b80b11b0 0x7f23b8140d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:36.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.723+0000 7f23b77fe700 1 -- 172.21.15.162:0/220999305 >> v2:172.21.15.118:3300/0 conn(0x7f23b80b11b0 msgr2=0x7f23b8140d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:36.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.723+0000 7f23b77fe700 1 --2- 172.21.15.162:0/220999305 >> v2:172.21.15.118:3300/0 conn(0x7f23b80b11b0 0x7f23b8140d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:36.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.723+0000 7f23b77fe700 1 -- 172.21.15.162:0/220999305 >> v1:172.21.15.162:6789/0 conn(0x7f23b80b1d70 legacy=0x7f23b814a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:36.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.723+0000 7f23b77fe700 1 -- 172.21.15.162:0/220999305 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b80b0740 con 0x7f23b80b1990 2020-11-28T21:55:36.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.723+0000 7f23b77fe700 1 --2- 172.21.15.162:0/220999305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23b80b1990 0x7f23b8140890 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 rx=0x7f23ac002f30 tx=0x7f23ac006dc0).ready entity=mon.1 client_cookie=67edaba0a98870be server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:36.733 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.723+0000 7f23b67fc700 1 -- 172.21.15.162:0/220999305 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23ac0097b0 con 0x7f23b80b1990 2020-11-28T21:55:36.733 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.723+0000 7f23b67fc700 1 -- 172.21.15.162:0/220999305 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23ac009910 con 0x7f23b80b1990 2020-11-28T21:55:36.733 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.723+0000 7f23b67fc700 1 -- 172.21.15.162:0/220999305 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23ac009c70 con 0x7f23b80b1990 2020-11-28T21:55:36.734 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.723+0000 7f23be1cd700 1 -- 172.21.15.162:0/220999305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23b80b1990 msgr2=0x7f23b8140890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:36.735 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.723+0000 7f23be1cd700 1 --2- 172.21.15.162:0/220999305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23b80b1990 0x7f23b8140890 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 rx=0x7f23ac002f30 tx=0x7f23ac006dc0).stop 2020-11-28T21:55:36.740 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.723+0000 7f23be1cd700 1 -- 172.21.15.162:0/220999305 shutdown_connections 2020-11-28T21:55:36.741 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.723+0000 7f23be1cd700 1 --2- 172.21.15.162:0/220999305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23b80b1990 0x7f23b8140890 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:36.741 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.723+0000 7f23be1cd700 1 --2- 172.21.15.162:0/220999305 >> v2:172.21.15.118:3300/0 conn(0x7f23b80b11b0 0x7f23b8140d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:36.742 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.723+0000 7f23be1cd700 1 -- 172.21.15.162:0/220999305 >> 172.21.15.162:0/220999305 conn(0x7f23b805b970 msgr2=0x7f23b805bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:36.742 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.723+0000 7f23be1cd700 1 -- 172.21.15.162:0/220999305 shutdown_connections 2020-11-28T21:55:36.743 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.723+0000 7f23be1cd700 1 -- 172.21.15.162:0/220999305 wait complete. 2020-11-28T21:55:36.743 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.727+0000 7f23be1cd700 1 Processor -- start 2020-11-28T21:55:36.743 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.727+0000 7f23be1cd700 1 -- start start 2020-11-28T21:55:36.744 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.727+0000 7f23be1cd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f23b80b11b0 0x7f23b8066620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:36.744 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.727+0000 7f23be1cd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23b80b1990 0x7f23b8066b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:36.745 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.727+0000 7f23be1cd700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23b80b1d70 0x7f23b8064340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:36.745 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.727+0000 7f23be1cd700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b8143310 con 0x7f23b80b11b0 2020-11-28T21:55:36.745 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.727+0000 7f23be1cd700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b815a3f0 con 0x7f23b80b1990 2020-11-28T21:55:36.746 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.727+0000 7f23be1cd700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b8159ee0 con 0x7f23b80b1d70 2020-11-28T21:55:36.746 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.727+0000 7f23b6ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23b80b1990 0x7f23b8066b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:36.747 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.727+0000 7f23b6ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23b80b1990 0x7f23b8066b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45096/0 (socket says 172.21.15.162:45096) 2020-11-28T21:55:36.747 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.727+0000 7f23b6ffd700 1 -- 172.21.15.162:0/604252092 learned_addr learned my addr 172.21.15.162:0/604252092 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:36.747 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.727+0000 7f23b77fe700 1 --2- 172.21.15.162:0/604252092 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f23b80b11b0 0x7f23b8066620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:36.748 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.727+0000 7f23b7fff700 1 --2- 172.21.15.162:0/604252092 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23b80b1d70 0x7f23b8064340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:36.748 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.727+0000 7f23b6ffd700 1 -- 172.21.15.162:0/604252092 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23b80b1d70 msgr2=0x7f23b8064340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:36.748 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.727+0000 7f23b6ffd700 1 --2- 172.21.15.162:0/604252092 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23b80b1d70 0x7f23b8064340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:36.749 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.727+0000 7f23b6ffd700 1 -- 172.21.15.162:0/604252092 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f23b80b11b0 msgr2=0x7f23b8066620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:36.749 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.727+0000 7f23b6ffd700 1 --2- 172.21.15.162:0/604252092 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f23b80b11b0 0x7f23b8066620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:36.749 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.727+0000 7f23b6ffd700 1 -- 172.21.15.162:0/604252092 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b815a150 con 0x7f23b80b1990 2020-11-28T21:55:36.750 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.727+0000 7f23b6ffd700 1 --2- 172.21.15.162:0/604252092 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23b80b1990 0x7f23b8066b10 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 rx=0x7f23b815a070 tx=0x7f23a80091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:36.750 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.727+0000 7f23b4ff9700 1 -- 172.21.15.162:0/604252092 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23a800ee70 con 0x7f23b80b1990 2020-11-28T21:55:36.751 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.727+0000 7f23b4ff9700 1 -- 172.21.15.162:0/604252092 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a800cb00 con 0x7f23b80b1990 2020-11-28T21:55:36.752 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.727+0000 7f23be1cd700 1 -- 172.21.15.162:0/604252092 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23b80670f0 con 0x7f23b80b1990 2020-11-28T21:55:36.752 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.727+0000 7f23b4ff9700 1 -- 172.21.15.162:0/604252092 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23a800ce60 con 0x7f23b80b1990 2020-11-28T21:55:36.753 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.727+0000 7f23be1cd700 1 -- 172.21.15.162:0/604252092 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b8143fa0 con 0x7f23b80b1990 2020-11-28T21:55:36.756 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.731+0000 7f23be1cd700 1 -- 172.21.15.162:0/604252092 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23b8057050 con 0x7f23b80b1990 2020-11-28T21:55:36.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.731+0000 7f23b4ff9700 1 -- 172.21.15.162:0/604252092 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f23a80123f0 con 0x7f23b80b1990 2020-11-28T21:55:36.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.731+0000 7f23b4ff9700 1 --2- 172.21.15.162:0/604252092 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2398055330 0x7f23980577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:36.758 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.731+0000 7f23b4ff9700 1 -- 172.21.15.162:0/604252092 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f23a806e510 con 0x7f23b80b1990 2020-11-28T21:55:36.758 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.731+0000 7f23b77fe700 1 --2- 172.21.15.162:0/604252092 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2398055330 0x7f23980577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:36.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.731+0000 7f23b77fe700 1 --2- 172.21.15.162:0/604252092 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2398055330 0x7f23980577f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7f23ac01a000 tx=0x7f23ac015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:36.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.735+0000 7f23b4ff9700 1 -- 172.21.15.162:0/604252092 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f23a80447b0 con 0x7f23b80b1990 2020-11-28T21:55:36.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.919+0000 7f23be1cd700 1 -- 172.21.15.162:0/604252092 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23b80b50b0 con 0x7f23b80b1990 2020-11-28T21:55:36.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.919+0000 7f23b4ff9700 1 -- 172.21.15.162:0/604252092 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+2504 (secure 0 0 0) 0x7f23a8047dd0 con 0x7f23b80b1990 2020-11-28T21:55:36.924 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:36.925 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":68,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7611,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6835/2127102267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2127102267},{"type":"v1","addr":"172.21.15.162:6835","nonce":2127102267}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":66},{"gid":7623,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/1473328171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1473328171},{"type":"v1","addr":"172.21.15.118:6833","nonce":1473328171}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:55:31.379864+0000","modified":"2020-11-28T21:55:33.042143+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7888":{"gid":7888,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/2667777645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2667777645},{"type":"v1","addr":"172.21.15.162:6837","nonce":2667777645}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2020-11-28T21:55:36.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.919+0000 7f23be1cd700 1 -- 172.21.15.162:0/604252092 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2398055330 msgr2=0x7f23980577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:36.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.919+0000 7f23be1cd700 1 --2- 172.21.15.162:0/604252092 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2398055330 0x7f23980577f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7f23ac01a000 tx=0x7f23ac015000).stop 2020-11-28T21:55:36.927 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.923+0000 7f23be1cd700 1 -- 172.21.15.162:0/604252092 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23b80b1990 msgr2=0x7f23b8066b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:36.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.923+0000 7f23be1cd700 1 --2- 172.21.15.162:0/604252092 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23b80b1990 0x7f23b8066b10 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 rx=0x7f23b815a070 tx=0x7f23a80091a0).stop 2020-11-28T21:55:36.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.923+0000 7f23be1cd700 1 -- 172.21.15.162:0/604252092 shutdown_connections 2020-11-28T21:55:36.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.923+0000 7f23be1cd700 1 --2- 172.21.15.162:0/604252092 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2398055330 0x7f23980577f0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:36.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.923+0000 7f23be1cd700 1 --2- 172.21.15.162:0/604252092 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23b80b1d70 0x7f23b8064340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:36.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.923+0000 7f23be1cd700 1 --2- 172.21.15.162:0/604252092 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f23b80b11b0 0x7f23b8066620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:36.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.923+0000 7f23be1cd700 1 --2- 172.21.15.162:0/604252092 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23b80b1990 0x7f23b8066b10 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:36.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.923+0000 7f23be1cd700 1 -- 172.21.15.162:0/604252092 >> 172.21.15.162:0/604252092 conn(0x7f23b805b970 msgr2=0x7f23b81491a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:36.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.923+0000 7f23be1cd700 1 -- 172.21.15.162:0/604252092 shutdown_connections 2020-11-28T21:55:36.931 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:36.923+0000 7f23be1cd700 1 -- 172.21.15.162:0/604252092 wait complete. 2020-11-28T21:55:36.932 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 68 2020-11-28T21:55:37.241 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-28T21:55:31.379864+0000', 'modified': '2020-11-28T21:55:33.042143+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7888}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7888': {'gid': 7888, 'name': 'c', 'rank': 0, 'incarnation': 66, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6837/2667777645', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6836', 'nonce': 2667777645}, {'type': 'v1', 'addr': '172.21.15.162:6837', 'nonce': 2667777645}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-28T21:55:37.241 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-28T21:55:37.242 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-28T21:55:37.374 INFO:teuthology.orchestra.run.smithi162.stdout:{"cluster_fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","whoami":0,"id":7888,"want_state":"up:active","state":"up:active","rank_uptime":5.9789465870000003,"mdsmap_epoch":68,"osdmap_epoch":138,"osdmap_epoch_barrier":138,"uptime":10.246568792} 2020-11-28T21:55:37.375 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", "id": 7888, "mdsmap_epoch": 68, "osdmap_epoch": 138, "osdmap_epoch_barrier": 138, "rank_uptime": 5.978946587, "state": "up:active", "uptime": 10.246568792, "want_state": "up:active", "whoami": 0 } 2020-11-28T21:55:37.376 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-11-28T21:55:37.377 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-11-28T21:55:37.377 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi192.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-11-28T21:55:37.378 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:37.379 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:55:37.392 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:55:37.453 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:55:37.453 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:55:37.512 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:55:37.513 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:55:37.514 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:37.514 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:55:37.565 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-11-28T21:55:37.566 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:55:37.610 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:55:37.627 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:55:37.627 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:55:37.681 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:55:37.683 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:55:37.683 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:37.684 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:55:37.687 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:2020-11-28T21:55:37.680+0000 7f130b9aa0c0 -1 init, newargv = 0x557ecc712b40 newargc=9ceph-fuse[12612]: starting ceph client 2020-11-28T21:55:37.687 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr: 2020-11-28T21:55:37.699 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:ceph-fuse[12612]: starting fuse 2020-11-28T21:55:38.793 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:55:38.809 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:55:38.810 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:55:38.867 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:55:38.868 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:55:38.868 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:38.869 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:55:38.918 INFO:teuthology.orchestra.run.smithi192.stdout:49 2020-11-28T21:55:38.918 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2020-11-28T21:55:38.919 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:38.919 INFO:teuthology.orchestra.run.smithi192:> sudo python3 -c ' 2020-11-28T21:55:38.920 INFO:teuthology.orchestra.run.smithi192:> import glob 2020-11-28T21:55:38.920 INFO:teuthology.orchestra.run.smithi192:> import re 2020-11-28T21:55:38.921 INFO:teuthology.orchestra.run.smithi192:> import os 2020-11-28T21:55:38.921 INFO:teuthology.orchestra.run.smithi192:> import subprocess 2020-11-28T21:55:38.922 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:55:38.922 INFO:teuthology.orchestra.run.smithi192:> def find_socket(client_name): 2020-11-28T21:55:38.923 INFO:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-11-28T21:55:38.923 INFO:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2020-11-28T21:55:38.924 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:55:38.924 INFO:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2020-11-28T21:55:38.925 INFO:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2020-11-28T21:55:38.925 INFO:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2020-11-28T21:55:38.926 INFO:teuthology.orchestra.run.smithi192:> return files[0] 2020-11-28T21:55:38.926 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:55:38.927 INFO:teuthology.orchestra.run.smithi192:> for f in files: 2020-11-28T21:55:38.927 INFO:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-11-28T21:55:38.927 INFO:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2020-11-28T21:55:38.928 INFO:teuthology.orchestra.run.smithi192:> return f 2020-11-28T21:55:38.928 INFO:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-11-28T21:55:38.929 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:55:38.929 INFO:teuthology.orchestra.run.smithi192:> print(find_socket("client.0")) 2020-11-28T21:55:38.930 INFO:teuthology.orchestra.run.smithi192:> ' 2020-11-28T21:55:39.010 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.0.12612.asok 2020-11-28T21:55:39.015 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.12612.asok 2020-11-28T21:55:39.016 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:39.016 INFO:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.12612.asok status 2020-11-28T21:55:39.086 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:55:39.088 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:55:39.091 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:55:39.095 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:55:39.116 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2020-11-28T21:55:39.116 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2020-11-28T21:55:39.116 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "a83a3572a957356977342f461361c1cd3ec06dbd", 2020-11-28T21:55:39.116 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 15.2.6-595-ga83a3572 (a83a3572a957356977342f461361c1cd3ec06dbd) octopus (stable)", 2020-11-28T21:55:39.117 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "0", 2020-11-28T21:55:39.117 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2020-11-28T21:55:39.117 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-11-28T21:55:39.117 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "12612", 2020-11-28T21:55:39.117 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2020-11-28T21:55:39.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:55:39.118 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2020-11-28T21:55:39.119 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2020-11-28T21:55:39.119 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 6773, 2020-11-28T21:55:39.119 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2020-11-28T21:55:39.119 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2020-11-28T21:55:39.120 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2020-11-28T21:55:39.120 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 6773 2020-11-28T21:55:39.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:55:39.120 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2020-11-28T21:55:39.120 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2020-11-28T21:55:39.121 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2020-11-28T21:55:39.121 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 1610149874 2020-11-28T21:55:39.121 INFO:teuthology.orchestra.run.smithi192.stdout: } 2020-11-28T21:55:39.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:55:39.121 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2020-11-28T21:55:39.122 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2020-11-28T21:55:39.122 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2020-11-28T21:55:39.122 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 1610149874 2020-11-28T21:55:39.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:55:39.122 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.6773 172.21.15.192:0/1610149874", 2020-11-28T21:55:39.123 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "172.21.15.192:0/1610149874", 2020-11-28T21:55:39.123 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2020-11-28T21:55:39.123 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 68, 2020-11-28T21:55:39.123 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 138, 2020-11-28T21:55:39.126 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2020-11-28T21:55:39.126 INFO:teuthology.orchestra.run.smithi192.stdout: "blacklisted": false 2020-11-28T21:55:39.126 INFO:teuthology.orchestra.run.smithi192.stdout:} 2020-11-28T21:55:39.128 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:39.128 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:55:39.129 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:55:39.182 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2020-11-28T21:55:39.183 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-28T21:55:39.183 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:39.184 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:55:39.240 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:55:39.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.355+0000 7f5f98df8700 1 Processor -- start 2020-11-28T21:55:39.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.355+0000 7f5f98df8700 1 -- start start 2020-11-28T21:55:39.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.355+0000 7f5f98df8700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5f94060e50 0x7f5f940b2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:39.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.355+0000 7f5f98df8700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f94140440 con 0x7f5f940b19f0 2020-11-28T21:55:39.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.355+0000 7f5f98df8700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f94146e00 con 0x7f5f940b2d70 2020-11-28T21:55:39.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.355+0000 7f5f98df8700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f940b7160 con 0x7f5f94060e50 2020-11-28T21:55:39.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.355+0000 7f5f9259c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5f94060e50 0x7f5f940b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:39.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.355+0000 7f5f9259c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5f94060e50 0x7f5f940b2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45104/0 (socket says 172.21.15.162:45104) 2020-11-28T21:55:39.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.355+0000 7f5f9259c700 1 -- 172.21.15.162:0/313337174 learned_addr learned my addr 172.21.15.162:0/313337174 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:39.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.355+0000 7f5f92d9d700 1 --1- 172.21.15.162:0/313337174 >> v1:172.21.15.162:6789/0 conn(0x7f5f940b2d70 0x7f5f941466f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51952/0 (socket says 172.21.15.162:51952) 2020-11-28T21:55:39.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.355+0000 7f5f9259c700 1 -- 172.21.15.162:0/313337174 >> v1:172.21.15.162:6789/0 conn(0x7f5f940b2d70 legacy=0x7f5f941466f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:39.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.355+0000 7f5f9259c700 1 -- 172.21.15.162:0/313337174 >> v1:172.21.15.118:6789/0 conn(0x7f5f940b19f0 legacy=0x7f5f94145fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:39.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.355+0000 7f5f9259c700 1 -- 172.21.15.162:0/313337174 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f940b7400 con 0x7f5f94060e50 2020-11-28T21:55:39.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.355+0000 7f5f9259c700 1 --2- 172.21.15.162:0/313337174 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f94060e50 0x7f5f940b2880 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 rx=0x7f5f84002f30 tx=0x7f5f84006e80).ready entity=mon.1 client_cookie=33e77027ef4b9c1a server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:39.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.355+0000 7f5f9159a700 1 -- 172.21.15.162:0/313337174 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f840097f0 con 0x7f5f94060e50 2020-11-28T21:55:39.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.355+0000 7f5f9159a700 1 -- 172.21.15.162:0/313337174 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f84009950 con 0x7f5f94060e50 2020-11-28T21:55:39.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.355+0000 7f5f9159a700 1 -- 172.21.15.162:0/313337174 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f84009cb0 con 0x7f5f94060e50 2020-11-28T21:55:39.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.355+0000 7f5f98df8700 1 -- 172.21.15.162:0/313337174 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f94060e50 msgr2=0x7f5f940b2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:39.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.355+0000 7f5f98df8700 1 --2- 172.21.15.162:0/313337174 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f94060e50 0x7f5f940b2880 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 rx=0x7f5f84002f30 tx=0x7f5f84006e80).stop 2020-11-28T21:55:39.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.355+0000 7f5f98df8700 1 -- 172.21.15.162:0/313337174 shutdown_connections 2020-11-28T21:55:39.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.355+0000 7f5f98df8700 1 --2- 172.21.15.162:0/313337174 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f94060e50 0x7f5f940b2880 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:39.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.355+0000 7f5f98df8700 1 -- 172.21.15.162:0/313337174 >> 172.21.15.162:0/313337174 conn(0x7f5f94062830 msgr2=0x7f5f94062c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:39.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.355+0000 7f5f98df8700 1 -- 172.21.15.162:0/313337174 shutdown_connections 2020-11-28T21:55:39.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.355+0000 7f5f98df8700 1 -- 172.21.15.162:0/313337174 wait complete. 2020-11-28T21:55:39.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.359+0000 7f5f98df8700 1 Processor -- start 2020-11-28T21:55:39.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.359+0000 7f5f98df8700 1 -- start start 2020-11-28T21:55:39.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.359+0000 7f5f98df8700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5f94060e50 0x7f5f941450a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:39.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.359+0000 7f5f98df8700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f940b19f0 0x7f5f94142bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:39.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.359+0000 7f5f98df8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f940b2d70 0x7f5f941430e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:39.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.359+0000 7f5f98df8700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f940add60 con 0x7f5f940b19f0 2020-11-28T21:55:39.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.359+0000 7f5f98df8700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f940ac8a0 con 0x7f5f940b2d70 2020-11-28T21:55:39.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.359+0000 7f5f98df8700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f940acb00 con 0x7f5f94060e50 2020-11-28T21:55:39.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.359+0000 7f5f92d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f940b2d70 0x7f5f941430e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:39.369 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.359+0000 7f5f92d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f940b2d70 0x7f5f941430e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45110/0 (socket says 172.21.15.162:45110) 2020-11-28T21:55:39.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.359+0000 7f5f92d9d700 1 -- 172.21.15.162:0/1445600352 learned_addr learned my addr 172.21.15.162:0/1445600352 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:39.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.359+0000 7f5f9259c700 1 --2- 172.21.15.162:0/1445600352 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5f94060e50 0x7f5f941450a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:39.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.359+0000 7f5f91d9b700 1 --2- 172.21.15.162:0/1445600352 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f940b19f0 0x7f5f94142bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:39.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.359+0000 7f5f92d9d700 1 -- 172.21.15.162:0/1445600352 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5f94060e50 msgr2=0x7f5f941450a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:39.370 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.359+0000 7f5f92d9d700 1 --2- 172.21.15.162:0/1445600352 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5f94060e50 0x7f5f941450a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:39.371 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.359+0000 7f5f92d9d700 1 -- 172.21.15.162:0/1445600352 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f940b19f0 msgr2=0x7f5f94142bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:39.371 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.359+0000 7f5f92d9d700 1 --2- 172.21.15.162:0/1445600352 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f940b19f0 0x7f5f94142bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:39.371 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.359+0000 7f5f92d9d700 1 -- 172.21.15.162:0/1445600352 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f940ad620 con 0x7f5f940b2d70 2020-11-28T21:55:39.371 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.359+0000 7f5f92d9d700 1 --2- 172.21.15.162:0/1445600352 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f940b2d70 0x7f5f941430e0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 rx=0x7f5f941423d0 tx=0x7f5f7c007b90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:39.372 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.359+0000 7f5f8b7fe700 1 -- 172.21.15.162:0/1445600352 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f7c0072e0 con 0x7f5f940b2d70 2020-11-28T21:55:39.372 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.359+0000 7f5f8b7fe700 1 -- 172.21.15.162:0/1445600352 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f7c005520 con 0x7f5f940b2d70 2020-11-28T21:55:39.372 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.359+0000 7f5f98df8700 1 -- 172.21.15.162:0/1445600352 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f940ad8c0 con 0x7f5f940b2d70 2020-11-28T21:55:39.372 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.359+0000 7f5f8b7fe700 1 -- 172.21.15.162:0/1445600352 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f7c005880 con 0x7f5f940b2d70 2020-11-28T21:55:39.372 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.359+0000 7f5f98df8700 1 -- 172.21.15.162:0/1445600352 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f94145b30 con 0x7f5f940b2d70 2020-11-28T21:55:39.373 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.363+0000 7f5f98df8700 1 -- 172.21.15.162:0/1445600352 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f94143c30 con 0x7f5f940b2d70 2020-11-28T21:55:39.373 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.363+0000 7f5f8b7fe700 1 -- 172.21.15.162:0/1445600352 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5f7c01a060 con 0x7f5f940b2d70 2020-11-28T21:55:39.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.363+0000 7f5f8b7fe700 1 --2- 172.21.15.162:0/1445600352 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5f78055330 0x7f5f780577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:39.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.363+0000 7f5f8b7fe700 1 -- 172.21.15.162:0/1445600352 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5f7c06d5c0 con 0x7f5f940b2d70 2020-11-28T21:55:39.374 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.363+0000 7f5f9259c700 1 --2- 172.21.15.162:0/1445600352 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5f78055330 0x7f5f780577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:39.375 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.363+0000 7f5f9259c700 1 --2- 172.21.15.162:0/1445600352 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5f78055330 0x7f5f780577f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f5f8401a000 tx=0x7f5f84015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:39.375 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.367+0000 7f5f8b7fe700 1 -- 172.21.15.162:0/1445600352 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5f94143c30 con 0x7f5f940b2d70 2020-11-28T21:55:39.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.511+0000 7f5f98df8700 1 -- 172.21.15.162:0/1445600352 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f94143c30 con 0x7f5f940b2d70 2020-11-28T21:55:39.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.511+0000 7f5f8b7fe700 1 -- 172.21.15.162:0/1445600352 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+16148 (secure 0 0 0) 0x7f5f94143c30 con 0x7f5f940b2d70 2020-11-28T21:55:39.516 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:39.517 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":138,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:55:32.383252+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:55:30.002238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2020-11-28T21:55:30.389418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:55:39.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.515+0000 7f5f98df8700 1 -- 172.21.15.162:0/1445600352 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5f78055330 msgr2=0x7f5f780577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:39.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.515+0000 7f5f98df8700 1 --2- 172.21.15.162:0/1445600352 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5f78055330 0x7f5f780577f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f5f8401a000 tx=0x7f5f84015000).stop 2020-11-28T21:55:39.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.515+0000 7f5f98df8700 1 -- 172.21.15.162:0/1445600352 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f940b2d70 msgr2=0x7f5f941430e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:39.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.515+0000 7f5f98df8700 1 --2- 172.21.15.162:0/1445600352 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f940b2d70 0x7f5f941430e0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 rx=0x7f5f941423d0 tx=0x7f5f7c007b90).stop 2020-11-28T21:55:39.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.515+0000 7f5f98df8700 1 -- 172.21.15.162:0/1445600352 shutdown_connections 2020-11-28T21:55:39.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.515+0000 7f5f98df8700 1 --2- 172.21.15.162:0/1445600352 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5f78055330 0x7f5f780577f0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:39.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.515+0000 7f5f98df8700 1 --2- 172.21.15.162:0/1445600352 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5f94060e50 0x7f5f941450a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:39.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.515+0000 7f5f98df8700 1 --2- 172.21.15.162:0/1445600352 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f940b2d70 0x7f5f941430e0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:39.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.515+0000 7f5f98df8700 1 --2- 172.21.15.162:0/1445600352 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f940b19f0 0x7f5f94142bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:39.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.515+0000 7f5f98df8700 1 -- 172.21.15.162:0/1445600352 >> 172.21.15.162:0/1445600352 conn(0x7f5f94062830 msgr2=0x7f5f941461b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:39.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.515+0000 7f5f98df8700 1 -- 172.21.15.162:0/1445600352 shutdown_connections 2020-11-28T21:55:39.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.515+0000 7f5f98df8700 1 -- 172.21.15.162:0/1445600352 wait complete. 2020-11-28T21:55:39.530 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-28T21:55:30.002238+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '137', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2020-11-28T21:55:30.389418+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '138', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-28T21:55:39.531 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:55:39.645 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.639+0000 7f755ffff700 1 Processor -- start 2020-11-28T21:55:39.645 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.639+0000 7f755ffff700 1 -- start start 2020-11-28T21:55:39.647 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.643+0000 7f755ffff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f75600ad240 0x7f7560140040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:39.647 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.643+0000 7f755ffff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7560142aa0 0x7f75600ad620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:39.648 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.643+0000 7f755ffff700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75600b1080 con 0x7f75600adb10 2020-11-28T21:55:39.648 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.643+0000 7f755ffff700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75600b1490 con 0x7f75600ad240 2020-11-28T21:55:39.648 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.643+0000 7f755ffff700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f756013f660 con 0x7f7560142aa0 2020-11-28T21:55:39.648 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.643+0000 7f755f7fe700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f75600adb10 0x7f75600b1900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51966/0 (socket says 172.21.15.162:51966) 2020-11-28T21:55:39.648 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.643+0000 7f755f7fe700 1 -- 172.21.15.162:0/1944282 learned_addr learned my addr 172.21.15.162:0/1944282 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:39.649 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.643+0000 7f755e7fc700 1 --2- 172.21.15.162:0/1944282 >> v2:172.21.15.162:3300/0 conn(0x7f7560142aa0 0x7f75600ad620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:39.649 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.643+0000 7f755effd700 1 --2- 172.21.15.162:0/1944282 >> v2:172.21.15.118:3300/0 conn(0x7f75600ad240 0x7f7560140040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:39.649 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.643+0000 7f755e7fc700 1 -- 172.21.15.162:0/1944282 >> v2:172.21.15.118:3300/0 conn(0x7f75600ad240 msgr2=0x7f7560140040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:39.649 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.643+0000 7f755e7fc700 1 --2- 172.21.15.162:0/1944282 >> v2:172.21.15.118:3300/0 conn(0x7f75600ad240 0x7f7560140040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:39.649 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.643+0000 7f755e7fc700 1 -- 172.21.15.162:0/1944282 >> v1:172.21.15.162:6789/0 conn(0x7f75600adb10 legacy=0x7f75600b1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:39.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.643+0000 7f755e7fc700 1 -- 172.21.15.162:0/1944282 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f756013f900 con 0x7f7560142aa0 2020-11-28T21:55:39.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.643+0000 7f755e7fc700 1 --2- 172.21.15.162:0/1944282 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7560142aa0 0x7f75600ad620 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 rx=0x7f7550002f30 tx=0x7f7550006dc0).ready entity=mon.1 client_cookie=ed80edf7210df427 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:39.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.643+0000 7f755dffb700 1 -- 172.21.15.162:0/1944282 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75500097f0 con 0x7f7560142aa0 2020-11-28T21:55:39.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.643+0000 7f755dffb700 1 -- 172.21.15.162:0/1944282 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7550009950 con 0x7f7560142aa0 2020-11-28T21:55:39.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.643+0000 7f755dffb700 1 -- 172.21.15.162:0/1944282 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7550009cb0 con 0x7f7560142aa0 2020-11-28T21:55:39.651 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.643+0000 7f755ffff700 1 -- 172.21.15.162:0/1944282 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7560142aa0 msgr2=0x7f75600ad620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:39.651 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.643+0000 7f755ffff700 1 --2- 172.21.15.162:0/1944282 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7560142aa0 0x7f75600ad620 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 rx=0x7f7550002f30 tx=0x7f7550006dc0).stop 2020-11-28T21:55:39.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.643+0000 7f755ffff700 1 -- 172.21.15.162:0/1944282 shutdown_connections 2020-11-28T21:55:39.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.643+0000 7f755ffff700 1 --2- 172.21.15.162:0/1944282 >> v2:172.21.15.118:3300/0 conn(0x7f75600ad240 0x7f7560140040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:39.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.643+0000 7f755ffff700 1 --2- 172.21.15.162:0/1944282 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7560142aa0 0x7f75600ad620 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:39.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.643+0000 7f755ffff700 1 -- 172.21.15.162:0/1944282 >> 172.21.15.162:0/1944282 conn(0x7f75600b2830 msgr2=0x7f75600b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:39.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.643+0000 7f755ffff700 1 -- 172.21.15.162:0/1944282 shutdown_connections 2020-11-28T21:55:39.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.643+0000 7f755ffff700 1 -- 172.21.15.162:0/1944282 wait complete. 2020-11-28T21:55:39.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.647+0000 7f755ffff700 1 Processor -- start 2020-11-28T21:55:39.655 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.647+0000 7f755ffff700 1 -- start start 2020-11-28T21:55:39.655 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.647+0000 7f755ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75600ad240 0x7f75600b5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:39.655 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.647+0000 7f755ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f75600adb10 0x7f7560142470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:39.655 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.647+0000 7f755ffff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7560142aa0 0x7f7560063930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:39.655 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.647+0000 7f755effd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75600ad240 0x7f75600b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:39.656 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.647+0000 7f755ffff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7560064d10 con 0x7f75600adb10 2020-11-28T21:55:39.656 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.647+0000 7f755ffff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756013c5c0 con 0x7f75600ad240 2020-11-28T21:55:39.656 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.647+0000 7f755ffff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f756013c860 con 0x7f7560142aa0 2020-11-28T21:55:39.656 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.647+0000 7f755effd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75600ad240 0x7f75600b5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45124/0 (socket says 172.21.15.162:45124) 2020-11-28T21:55:39.657 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.647+0000 7f755effd700 1 -- 172.21.15.162:0/581956876 learned_addr learned my addr 172.21.15.162:0/581956876 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:39.657 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.647+0000 7f755f7fe700 1 --2- 172.21.15.162:0/581956876 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7560142aa0 0x7f7560063930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:39.657 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.647+0000 7f755e7fc700 1 --2- 172.21.15.162:0/581956876 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f75600adb10 0x7f7560142470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:39.657 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.647+0000 7f755effd700 1 -- 172.21.15.162:0/581956876 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7560142aa0 msgr2=0x7f7560063930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:39.657 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.647+0000 7f755effd700 1 --2- 172.21.15.162:0/581956876 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7560142aa0 0x7f7560063930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:39.658 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.647+0000 7f755effd700 1 -- 172.21.15.162:0/581956876 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f75600adb10 msgr2=0x7f7560142470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:39.658 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.647+0000 7f755effd700 1 --2- 172.21.15.162:0/581956876 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f75600adb10 0x7f7560142470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:39.658 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.647+0000 7f755effd700 1 -- 172.21.15.162:0/581956876 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75600abea0 con 0x7f75600ad240 2020-11-28T21:55:39.658 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.647+0000 7f755effd700 1 --2- 172.21.15.162:0/581956876 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75600ad240 0x7f75600b5670 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 rx=0x7f7554003070 tx=0x7f7554008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:39.658 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.647+0000 7f754ffff700 1 -- 172.21.15.162:0/581956876 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7554009980 con 0x7f75600ad240 2020-11-28T21:55:39.659 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.647+0000 7f755ffff700 1 -- 172.21.15.162:0/581956876 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f756013c2f0 con 0x7f75600ad240 2020-11-28T21:55:39.659 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.647+0000 7f754ffff700 1 -- 172.21.15.162:0/581956876 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7554009ae0 con 0x7f75600ad240 2020-11-28T21:55:39.659 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.647+0000 7f754ffff700 1 -- 172.21.15.162:0/581956876 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7554009e40 con 0x7f75600ad240 2020-11-28T21:55:39.659 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.647+0000 7f755ffff700 1 -- 172.21.15.162:0/581956876 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7560137920 con 0x7f75600ad240 2020-11-28T21:55:39.660 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.651+0000 7f755ffff700 1 -- 172.21.15.162:0/581956876 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7560057050 con 0x7f75600ad240 2020-11-28T21:55:39.660 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.651+0000 7f754ffff700 1 -- 172.21.15.162:0/581956876 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7554013480 con 0x7f75600ad240 2020-11-28T21:55:39.660 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.651+0000 7f754ffff700 1 --2- 172.21.15.162:0/581956876 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f753c055330 0x7f753c0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:39.661 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.651+0000 7f754ffff700 1 -- 172.21.15.162:0/581956876 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f755406e5c0 con 0x7f75600ad240 2020-11-28T21:55:39.661 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.651+0000 7f755e7fc700 1 --2- 172.21.15.162:0/581956876 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f753c055330 0x7f753c0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:39.661 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.651+0000 7f755e7fc700 1 --2- 172.21.15.162:0/581956876 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f753c055330 0x7f753c0577f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7f7550016000 tx=0x7f7550015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:39.661 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.655+0000 7f754ffff700 1 -- 172.21.15.162:0/581956876 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f75540447c0 con 0x7f75600ad240 2020-11-28T21:55:39.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.799+0000 7f755ffff700 1 -- 172.21.15.162:0/581956876 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f7560068350 con 0x7f75600ad240 2020-11-28T21:55:39.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.799+0000 7f754ffff700 1 -- 172.21.15.162:0/581956876 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v138) v1 ==== 110+0+10 (secure 0 0 0) 0x7f7554047de0 con 0x7f75600ad240 2020-11-28T21:55:39.804 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:55:39.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.799+0000 7f755ffff700 1 -- 172.21.15.162:0/581956876 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f753c055330 msgr2=0x7f753c0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:39.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.799+0000 7f755ffff700 1 --2- 172.21.15.162:0/581956876 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f753c055330 0x7f753c0577f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7f7550016000 tx=0x7f7550015000).stop 2020-11-28T21:55:39.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.799+0000 7f755ffff700 1 -- 172.21.15.162:0/581956876 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75600ad240 msgr2=0x7f75600b5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:39.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.799+0000 7f755ffff700 1 --2- 172.21.15.162:0/581956876 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75600ad240 0x7f75600b5670 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 rx=0x7f7554003070 tx=0x7f7554008040).stop 2020-11-28T21:55:39.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.803+0000 7f755ffff700 1 -- 172.21.15.162:0/581956876 shutdown_connections 2020-11-28T21:55:39.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.803+0000 7f755ffff700 1 --2- 172.21.15.162:0/581956876 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f753c055330 0x7f753c0577f0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:39.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.803+0000 7f755ffff700 1 --2- 172.21.15.162:0/581956876 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7560142aa0 0x7f7560063930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:39.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.803+0000 7f755ffff700 1 --2- 172.21.15.162:0/581956876 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75600ad240 0x7f75600b5670 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:39.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.803+0000 7f755ffff700 1 --2- 172.21.15.162:0/581956876 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f75600adb10 0x7f7560142470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:39.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.803+0000 7f755ffff700 1 -- 172.21.15.162:0/581956876 >> 172.21.15.162:0/581956876 conn(0x7f75600b2830 msgr2=0x7f7560061a60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:39.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.803+0000 7f755ffff700 1 -- 172.21.15.162:0/581956876 shutdown_connections 2020-11-28T21:55:39.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.803+0000 7f755ffff700 1 -- 172.21.15.162:0/581956876 wait complete. 2020-11-28T21:55:39.817 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:55:39.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.927+0000 7fa8eb59e700 1 Processor -- start 2020-11-28T21:55:39.933 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.927+0000 7fa8eb59e700 1 -- start start 2020-11-28T21:55:39.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8eb59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa8ec0b21f0 0x7fa8ec1434f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:39.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8eb59e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fa8ec141e80 0x7fa8ec0606f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:39.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8eb59e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8ec0af6a0 con 0x7fa8ec0ae3d0 2020-11-28T21:55:39.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8eb59e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8ec0b25d0 con 0x7fa8ec141e80 2020-11-28T21:55:39.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8eb59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ec05fff0 con 0x7fa8ec0b21f0 2020-11-28T21:55:39.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8ead9d700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fa8ec0ae3d0 0x7fa8ec143f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51980/0 (socket says 172.21.15.162:51980) 2020-11-28T21:55:39.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8ead9d700 1 -- 172.21.15.162:0/3774751766 learned_addr learned my addr 172.21.15.162:0/3774751766 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:39.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8ea59c700 1 --2- 172.21.15.162:0/3774751766 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa8ec0b21f0 0x7fa8ec1434f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:39.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8e3fff700 1 --2- 172.21.15.162:0/3774751766 >> v2:172.21.15.118:3300/0 conn(0x7fa8ec141e80 0x7fa8ec0606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:39.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8e9d9b700 1 -- 172.21.15.162:0/3774751766 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 802250004 0 0) 0x7fa8ec0af6a0 con 0x7fa8ec0ae3d0 2020-11-28T21:55:39.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8e9d9b700 1 -- 172.21.15.162:0/3774751766 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8cc003230 con 0x7fa8ec0ae3d0 2020-11-28T21:55:39.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8e9d9b700 1 -- 172.21.15.162:0/3774751766 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2295893614 0 0) 0x7fa8cc003230 con 0x7fa8ec0ae3d0 2020-11-28T21:55:39.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8e9d9b700 1 -- 172.21.15.162:0/3774751766 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa8ec0b21f0 msgr2=0x7fa8ec1434f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:39.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8e9d9b700 1 --2- 172.21.15.162:0/3774751766 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa8ec0b21f0 0x7fa8ec1434f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:39.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8e9d9b700 1 -- 172.21.15.162:0/3774751766 >> v2:172.21.15.118:3300/0 conn(0x7fa8ec141e80 msgr2=0x7fa8ec0606f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:39.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8e9d9b700 1 --2- 172.21.15.162:0/3774751766 >> v2:172.21.15.118:3300/0 conn(0x7fa8ec141e80 0x7fa8ec0606f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:39.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8e9d9b700 1 -- 172.21.15.162:0/3774751766 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8ec0af8d0 con 0x7fa8ec0ae3d0 2020-11-28T21:55:39.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8e9d9b700 1 -- 172.21.15.162:0/3774751766 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7fa8e4001a50 con 0x7fa8ec0ae3d0 2020-11-28T21:55:39.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8e9d9b700 1 -- 172.21.15.162:0/3774751766 >> v1:172.21.15.162:6789/0 conn(0x7fa8ec0ae3d0 legacy=0x7fa8ec143f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:39.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8e9d9b700 1 --2- 172.21.15.162:0/3774751766 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa8ec0b21f0 0x7fa8cc003720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:39.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8e9d9b700 1 --2- 172.21.15.162:0/3774751766 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa8cc003c30 0x7fa8cc0044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:39.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8e9d9b700 1 --2- 172.21.15.162:0/3774751766 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8cc0049c0 0x7fa8cc004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:39.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8e9d9b700 1 -- 172.21.15.162:0/3774751766 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ec0b25d0 con 0x7fa8cc003c30 2020-11-28T21:55:39.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8e9d9b700 1 -- 172.21.15.162:0/3774751766 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ec05fff0 con 0x7fa8cc0049c0 2020-11-28T21:55:39.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8e9d9b700 1 -- 172.21.15.162:0/3774751766 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc0041f0 con 0x7fa8ec0b21f0 2020-11-28T21:55:39.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8ead9d700 1 --2- 172.21.15.162:0/3774751766 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8cc0049c0 0x7fa8cc004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:39.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8e3fff700 1 --2- 172.21.15.162:0/3774751766 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa8cc003c30 0x7fa8cc0044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:39.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8ea59c700 1 --2- 172.21.15.162:0/3774751766 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa8ec0b21f0 0x7fa8cc003720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:39.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8ead9d700 1 -- 172.21.15.162:0/3774751766 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa8ec0b21f0 msgr2=0x7fa8cc003720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:39.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8ead9d700 1 --2- 172.21.15.162:0/3774751766 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa8ec0b21f0 0x7fa8cc003720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:39.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8ead9d700 1 -- 172.21.15.162:0/3774751766 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa8cc003c30 msgr2=0x7fa8cc0044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:39.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8ead9d700 1 --2- 172.21.15.162:0/3774751766 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa8cc003c30 0x7fa8cc0044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:39.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8ead9d700 1 -- 172.21.15.162:0/3774751766 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8cc003230 con 0x7fa8cc0049c0 2020-11-28T21:55:39.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8e3fff700 1 --2- 172.21.15.162:0/3774751766 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa8cc003c30 0x7fa8cc0044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:55:39.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8ead9d700 1 --2- 172.21.15.162:0/3774751766 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8cc0049c0 0x7fa8cc004e40 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 rx=0x7fa8e4001760 tx=0x7fa8e4007b40).ready entity=mon.1 client_cookie=546cb88709dbc938 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:39.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8e9d9b700 1 -- 172.21.15.162:0/3774751766 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8e4005b70 con 0x7fa8cc0049c0 2020-11-28T21:55:39.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8e9d9b700 1 -- 172.21.15.162:0/3774751766 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8e4005cd0 con 0x7fa8cc0049c0 2020-11-28T21:55:39.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8e9d9b700 1 -- 172.21.15.162:0/3774751766 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8e40072c0 con 0x7fa8cc0049c0 2020-11-28T21:55:39.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8eb59e700 1 -- 172.21.15.162:0/3774751766 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8cc0049c0 msgr2=0x7fa8cc004e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:39.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8eb59e700 1 --2- 172.21.15.162:0/3774751766 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8cc0049c0 0x7fa8cc004e40 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 rx=0x7fa8e4001760 tx=0x7fa8e4007b40).stop 2020-11-28T21:55:39.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8ead9d700 1 -- 172.21.15.162:0/3774751766 reap_dead start 2020-11-28T21:55:39.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8eb59e700 1 -- 172.21.15.162:0/3774751766 shutdown_connections 2020-11-28T21:55:39.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8eb59e700 1 -- 172.21.15.162:0/3774751766 >> 172.21.15.162:0/3774751766 conn(0x7fa8ec0b57d0 msgr2=0x7fa8ec0b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:39.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8eb59e700 1 -- 172.21.15.162:0/3774751766 shutdown_connections 2020-11-28T21:55:39.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.931+0000 7fa8eb59e700 1 -- 172.21.15.162:0/3774751766 wait complete. 2020-11-28T21:55:39.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.935+0000 7fa8eb59e700 1 Processor -- start 2020-11-28T21:55:39.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.935+0000 7fa8eb59e700 1 -- start start 2020-11-28T21:55:39.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.935+0000 7fa8eb59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8ec0b3fd0 0x7fa8ec140230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:39.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.935+0000 7fa8eb59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa8ec0ac1c0 0x7fa8ec140720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:39.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.935+0000 7fa8eb59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa8ec1550e0 0x7fa8ec155560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:39.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.935+0000 7fa8eb59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ec140c10 con 0x7fa8ec0ac1c0 2020-11-28T21:55:39.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.935+0000 7fa8eb59e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ec0ac970 con 0x7fa8ec0b3fd0 2020-11-28T21:55:39.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.935+0000 7fa8eb59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ec0b43b0 con 0x7fa8ec1550e0 2020-11-28T21:55:39.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.935+0000 7fa8ea59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8ec0b3fd0 0x7fa8ec140230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:39.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.935+0000 7fa8ea59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8ec0b3fd0 0x7fa8ec140230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45144/0 (socket says 172.21.15.162:45144) 2020-11-28T21:55:39.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.935+0000 7fa8ea59c700 1 -- 172.21.15.162:0/338138677 learned_addr learned my addr 172.21.15.162:0/338138677 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:39.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.935+0000 7fa8e3fff700 1 --2- 172.21.15.162:0/338138677 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa8ec0ac1c0 0x7fa8ec140720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:39.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.935+0000 7fa8ead9d700 1 --2- 172.21.15.162:0/338138677 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa8ec1550e0 0x7fa8ec155560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:39.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.935+0000 7fa8ea59c700 1 -- 172.21.15.162:0/338138677 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa8ec1550e0 msgr2=0x7fa8ec155560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:39.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.935+0000 7fa8ea59c700 1 --2- 172.21.15.162:0/338138677 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa8ec1550e0 0x7fa8ec155560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:39.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.935+0000 7fa8ea59c700 1 -- 172.21.15.162:0/338138677 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa8ec0ac1c0 msgr2=0x7fa8ec140720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:39.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.935+0000 7fa8ea59c700 1 --2- 172.21.15.162:0/338138677 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa8ec0ac1c0 0x7fa8ec140720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:39.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.935+0000 7fa8ea59c700 1 -- 172.21.15.162:0/338138677 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8ec065220 con 0x7fa8ec0b3fd0 2020-11-28T21:55:39.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.935+0000 7fa8e3fff700 1 --2- 172.21.15.162:0/338138677 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa8ec0ac1c0 0x7fa8ec140720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:55:39.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.935+0000 7fa8ead9d700 1 --2- 172.21.15.162:0/338138677 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa8ec1550e0 0x7fa8ec155560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:55:39.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.935+0000 7fa8ea59c700 1 --2- 172.21.15.162:0/338138677 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8ec0b3fd0 0x7fa8ec140230 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 rx=0x7fa8ec140110 tx=0x7fa8d4008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:39.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.935+0000 7fa8e37fe700 1 -- 172.21.15.162:0/338138677 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8d400f980 con 0x7fa8ec0b3fd0 2020-11-28T21:55:39.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.935+0000 7fa8e37fe700 1 -- 172.21.15.162:0/338138677 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d400fae0 con 0x7fa8ec0b3fd0 2020-11-28T21:55:39.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.935+0000 7fa8eb59e700 1 -- 172.21.15.162:0/338138677 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8ec13fdd0 con 0x7fa8ec0b3fd0 2020-11-28T21:55:39.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.935+0000 7fa8e37fe700 1 -- 172.21.15.162:0/338138677 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8d400fc50 con 0x7fa8ec0b3fd0 2020-11-28T21:55:39.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.935+0000 7fa8eb59e700 1 -- 172.21.15.162:0/338138677 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8ec0ac6d0 con 0x7fa8ec0b3fd0 2020-11-28T21:55:39.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.939+0000 7fa8eb59e700 1 -- 172.21.15.162:0/338138677 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8ec057050 con 0x7fa8ec0b3fd0 2020-11-28T21:55:39.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.939+0000 7fa8e37fe700 1 -- 172.21.15.162:0/338138677 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa8d4024070 con 0x7fa8ec0b3fd0 2020-11-28T21:55:39.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.939+0000 7fa8e37fe700 1 --2- 172.21.15.162:0/338138677 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa8d00553b0 0x7fa8d0057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:39.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.939+0000 7fa8e3fff700 1 --2- 172.21.15.162:0/338138677 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa8d00553b0 0x7fa8d0057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:39.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.939+0000 7fa8e37fe700 1 -- 172.21.15.162:0/338138677 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa8d406ef50 con 0x7fa8ec0b3fd0 2020-11-28T21:55:39.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.939+0000 7fa8e3fff700 1 --2- 172.21.15.162:0/338138677 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa8d00553b0 0x7fa8d0057870 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7fa8dc008000 tx=0x7fa8dc008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:39.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:39.943+0000 7fa8e37fe700 1 -- 172.21.15.162:0/338138677 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa8d4045140 con 0x7fa8ec0b3fd0 2020-11-28T21:55:40.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.083+0000 7fa8eb59e700 1 -- 172.21.15.162:0/338138677 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa8ec067380 con 0x7fa8ec0b3fd0 2020-11-28T21:55:40.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.083+0000 7fa8e37fe700 1 -- 172.21.15.162:0/338138677 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v138) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa8d4048760 con 0x7fa8ec0b3fd0 2020-11-28T21:55:40.088 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:55:40.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.087+0000 7fa8eb59e700 1 -- 172.21.15.162:0/338138677 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa8d00553b0 msgr2=0x7fa8d0057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:40.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.087+0000 7fa8eb59e700 1 --2- 172.21.15.162:0/338138677 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa8d00553b0 0x7fa8d0057870 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7fa8dc008000 tx=0x7fa8dc008040).stop 2020-11-28T21:55:40.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.087+0000 7fa8eb59e700 1 -- 172.21.15.162:0/338138677 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8ec0b3fd0 msgr2=0x7fa8ec140230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:40.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.087+0000 7fa8eb59e700 1 --2- 172.21.15.162:0/338138677 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8ec0b3fd0 0x7fa8ec140230 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 rx=0x7fa8ec140110 tx=0x7fa8d4008040).stop 2020-11-28T21:55:40.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.087+0000 7fa8eb59e700 1 -- 172.21.15.162:0/338138677 shutdown_connections 2020-11-28T21:55:40.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.087+0000 7fa8eb59e700 1 --2- 172.21.15.162:0/338138677 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa8d00553b0 0x7fa8d0057870 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:40.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.087+0000 7fa8eb59e700 1 --2- 172.21.15.162:0/338138677 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa8ec1550e0 0x7fa8ec155560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:40.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.087+0000 7fa8eb59e700 1 --2- 172.21.15.162:0/338138677 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8ec0b3fd0 0x7fa8ec140230 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:40.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.087+0000 7fa8eb59e700 1 --2- 172.21.15.162:0/338138677 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa8ec0ac1c0 0x7fa8ec140720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:40.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.087+0000 7fa8eb59e700 1 -- 172.21.15.162:0/338138677 >> 172.21.15.162:0/338138677 conn(0x7fa8ec0b57d0 msgr2=0x7fa8ec142ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:40.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.087+0000 7fa8eb59e700 1 -- 172.21.15.162:0/338138677 shutdown_connections 2020-11-28T21:55:40.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.087+0000 7fa8eb59e700 1 -- 172.21.15.162:0/338138677 wait complete. 2020-11-28T21:55:40.102 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-28T21:55:40.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.219+0000 7f45167c0700 1 Processor -- start 2020-11-28T21:55:40.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.219+0000 7f45167c0700 1 -- start start 2020-11-28T21:55:40.229 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.219+0000 7f45167c0700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f45100b3ab0 0x7f451005c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:40.230 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.219+0000 7f45167c0700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f45100b30e0 0x7f451013fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:40.230 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.219+0000 7f45167c0700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f451005baf0 con 0x7f45100b2d00 2020-11-28T21:55:40.230 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.219+0000 7f45167c0700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45100b0e20 con 0x7f45100b3ab0 2020-11-28T21:55:40.231 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.219+0000 7f45167c0700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f451005bd90 con 0x7f45100b30e0 2020-11-28T21:55:40.231 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.219+0000 7f45157be700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f45100b2d00 0x7f4510142ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:51996/0 (socket says 172.21.15.162:51996) 2020-11-28T21:55:40.231 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.219+0000 7f45157be700 1 -- 172.21.15.162:0/2301755800 learned_addr learned my addr 172.21.15.162:0/2301755800 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:40.232 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.219+0000 7f4514fbd700 1 --2- 172.21.15.162:0/2301755800 >> v2:172.21.15.118:3300/0 conn(0x7f45100b3ab0 0x7f451005c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:40.232 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.219+0000 7f4515fbf700 1 --2- 172.21.15.162:0/2301755800 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f45100b30e0 0x7f451013fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:40.232 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.219+0000 7f4507fff700 1 -- 172.21.15.162:0/2301755800 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3556192494 0 0) 0x7f451005baf0 con 0x7f45100b2d00 2020-11-28T21:55:40.232 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.219+0000 7f4507fff700 1 -- 172.21.15.162:0/2301755800 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44f4003230 con 0x7f45100b2d00 2020-11-28T21:55:40.232 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.219+0000 7f4507fff700 1 -- 172.21.15.162:0/2301755800 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3099178802 0 0) 0x7f44f4003230 con 0x7f45100b2d00 2020-11-28T21:55:40.233 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.219+0000 7f4507fff700 1 -- 172.21.15.162:0/2301755800 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f45100b30e0 msgr2=0x7f451013fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:40.233 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.219+0000 7f4507fff700 1 --2- 172.21.15.162:0/2301755800 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f45100b30e0 0x7f451013fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:40.233 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.219+0000 7f4507fff700 1 -- 172.21.15.162:0/2301755800 >> v2:172.21.15.118:3300/0 conn(0x7f45100b3ab0 msgr2=0x7f451005c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:40.233 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.219+0000 7f4507fff700 1 --2- 172.21.15.162:0/2301755800 >> v2:172.21.15.118:3300/0 conn(0x7f45100b3ab0 0x7f451005c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:40.234 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.219+0000 7f4507fff700 1 -- 172.21.15.162:0/2301755800 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f451005c030 con 0x7f45100b2d00 2020-11-28T21:55:40.234 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.219+0000 7f4507fff700 1 -- 172.21.15.162:0/2301755800 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f450c0016d0 con 0x7f45100b2d00 2020-11-28T21:55:40.234 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.219+0000 7f4507fff700 1 -- 172.21.15.162:0/2301755800 >> v1:172.21.15.162:6789/0 conn(0x7f45100b2d00 legacy=0x7f4510142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:40.235 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.219+0000 7f4507fff700 1 --2- 172.21.15.162:0/2301755800 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44f4003720 0x7f44f4003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:40.235 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.219+0000 7f4507fff700 1 --2- 172.21.15.162:0/2301755800 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f45100b30e0 0x7f44f40044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:40.235 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.219+0000 7f4507fff700 1 --2- 172.21.15.162:0/2301755800 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44f40049c0 0x7f44f4004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:40.235 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.219+0000 7f4507fff700 1 -- 172.21.15.162:0/2301755800 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45100b0e20 con 0x7f44f40049c0 2020-11-28T21:55:40.236 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.219+0000 7f4507fff700 1 -- 172.21.15.162:0/2301755800 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f451005bd90 con 0x7f44f4003720 2020-11-28T21:55:40.236 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.219+0000 7f4507fff700 1 -- 172.21.15.162:0/2301755800 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44f40041c0 con 0x7f45100b30e0 2020-11-28T21:55:40.236 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.219+0000 7f45157be700 1 --2- 172.21.15.162:0/2301755800 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44f4003720 0x7f44f4003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:40.236 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.219+0000 7f4514fbd700 1 --2- 172.21.15.162:0/2301755800 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f45100b30e0 0x7f44f40044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:40.236 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.223+0000 7f4515fbf700 1 --2- 172.21.15.162:0/2301755800 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44f40049c0 0x7f44f4004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:40.237 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.223+0000 7f45157be700 1 -- 172.21.15.162:0/2301755800 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f45100b30e0 msgr2=0x7f44f40044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:40.237 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.223+0000 7f45157be700 1 --2- 172.21.15.162:0/2301755800 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f45100b30e0 0x7f44f40044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:40.237 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.223+0000 7f45157be700 1 -- 172.21.15.162:0/2301755800 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44f40049c0 msgr2=0x7f44f4004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:40.237 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.223+0000 7f45157be700 1 --2- 172.21.15.162:0/2301755800 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44f40049c0 0x7f44f4004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:40.237 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.223+0000 7f45157be700 1 -- 172.21.15.162:0/2301755800 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f4003230 con 0x7f44f4003720 2020-11-28T21:55:40.238 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.223+0000 7f45157be700 1 --2- 172.21.15.162:0/2301755800 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44f4003720 0x7f44f4003b40 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 rx=0x7f450c0013e0 tx=0x7f450c0077c0).ready entity=mon.1 client_cookie=a61ce318cc93b1c server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:40.238 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.223+0000 7f4507fff700 1 -- 172.21.15.162:0/2301755800 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f450c0070d0 con 0x7f44f4003720 2020-11-28T21:55:40.238 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.223+0000 7f4507fff700 1 -- 172.21.15.162:0/2301755800 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f450c007230 con 0x7f44f4003720 2020-11-28T21:55:40.238 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.223+0000 7f4507fff700 1 -- 172.21.15.162:0/2301755800 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f450c0055e0 con 0x7f44f4003720 2020-11-28T21:55:40.239 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.223+0000 7f45167c0700 1 -- 172.21.15.162:0/2301755800 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44f4003720 msgr2=0x7f44f4003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:40.239 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.223+0000 7f45167c0700 1 --2- 172.21.15.162:0/2301755800 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44f4003720 0x7f44f4003b40 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 rx=0x7f450c0013e0 tx=0x7f450c0077c0).stop 2020-11-28T21:55:40.239 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.223+0000 7f4515fbf700 1 -- 172.21.15.162:0/2301755800 reap_dead start 2020-11-28T21:55:40.239 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.223+0000 7f45167c0700 1 -- 172.21.15.162:0/2301755800 shutdown_connections 2020-11-28T21:55:40.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.223+0000 7f45167c0700 1 -- 172.21.15.162:0/2301755800 >> 172.21.15.162:0/2301755800 conn(0x7f45100604c0 msgr2=0x7f45100608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:40.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.223+0000 7f45167c0700 1 -- 172.21.15.162:0/2301755800 shutdown_connections 2020-11-28T21:55:40.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.223+0000 7f45167c0700 1 -- 172.21.15.162:0/2301755800 wait complete. 2020-11-28T21:55:40.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.227+0000 7f45167c0700 1 Processor -- start 2020-11-28T21:55:40.241 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.227+0000 7f45167c0700 1 -- start start 2020-11-28T21:55:40.243 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.227+0000 7f45167c0700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f45101420f0 0x7f45100aea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:40.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.227+0000 7f45167c0700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4510143ed0 0x7f451013ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:40.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.227+0000 7f45167c0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f451013b070 0x7f45100af680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:40.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.227+0000 7f45167c0700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4510145640 con 0x7f45101420f0 2020-11-28T21:55:40.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.227+0000 7f45167c0700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45101424d0 con 0x7f451013b070 2020-11-28T21:55:40.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.227+0000 7f45167c0700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45100aef10 con 0x7f4510143ed0 2020-11-28T21:55:40.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.227+0000 7f4515fbf700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f451013b070 0x7f45100af680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:40.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.227+0000 7f4515fbf700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f451013b070 0x7f45100af680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45166/0 (socket says 172.21.15.162:45166) 2020-11-28T21:55:40.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.227+0000 7f4515fbf700 1 -- 172.21.15.162:0/3401719774 learned_addr learned my addr 172.21.15.162:0/3401719774 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:40.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.227+0000 7f45157be700 1 --2- 172.21.15.162:0/3401719774 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f45101420f0 0x7f45100aea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:40.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.227+0000 7f4514fbd700 1 --2- 172.21.15.162:0/3401719774 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4510143ed0 0x7f451013ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:40.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.227+0000 7f4515fbf700 1 -- 172.21.15.162:0/3401719774 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4510143ed0 msgr2=0x7f451013ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:40.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.227+0000 7f4515fbf700 1 --2- 172.21.15.162:0/3401719774 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4510143ed0 0x7f451013ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:40.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.227+0000 7f4515fbf700 1 -- 172.21.15.162:0/3401719774 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f45101420f0 msgr2=0x7f45100aea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:40.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.227+0000 7f4515fbf700 1 --2- 172.21.15.162:0/3401719774 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f45101420f0 0x7f45100aea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:40.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.227+0000 7f4515fbf700 1 -- 172.21.15.162:0/3401719774 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4510145870 con 0x7f451013b070 2020-11-28T21:55:40.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.227+0000 7f45157be700 1 --2- 172.21.15.162:0/3401719774 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f45101420f0 0x7f45100aea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:55:40.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.227+0000 7f4514fbd700 1 --2- 172.21.15.162:0/3401719774 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4510143ed0 0x7f451013ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:55:40.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.227+0000 7f4515fbf700 1 --2- 172.21.15.162:0/3401719774 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f451013b070 0x7f45100af680 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 rx=0x7f450800b000 tx=0x7f4508008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:40.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.227+0000 7f45067fc700 1 -- 172.21.15.162:0/3401719774 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f450800ec30 con 0x7f451013b070 2020-11-28T21:55:40.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.227+0000 7f45067fc700 1 -- 172.21.15.162:0/3401719774 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f450800ed90 con 0x7f451013b070 2020-11-28T21:55:40.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.227+0000 7f45167c0700 1 -- 172.21.15.162:0/3401719774 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4510145b10 con 0x7f451013b070 2020-11-28T21:55:40.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.227+0000 7f45067fc700 1 -- 172.21.15.162:0/3401719774 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4508014660 con 0x7f451013b070 2020-11-28T21:55:40.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.227+0000 7f45167c0700 1 -- 172.21.15.162:0/3401719774 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f451013b4f0 con 0x7f451013b070 2020-11-28T21:55:40.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.231+0000 7f45167c0700 1 -- 172.21.15.162:0/3401719774 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4510057050 con 0x7f451013b070 2020-11-28T21:55:40.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.231+0000 7f45067fc700 1 -- 172.21.15.162:0/3401719774 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f45080147c0 con 0x7f451013b070 2020-11-28T21:55:40.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.231+0000 7f45067fc700 1 --2- 172.21.15.162:0/3401719774 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f44f8055350 0x7f44f8057810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:40.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.231+0000 7f45067fc700 1 -- 172.21.15.162:0/3401719774 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f450806eae0 con 0x7f451013b070 2020-11-28T21:55:40.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.231+0000 7f45157be700 1 --2- 172.21.15.162:0/3401719774 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f44f8055350 0x7f44f8057810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:40.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.231+0000 7f45157be700 1 --2- 172.21.15.162:0/3401719774 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f44f8055350 0x7f44f8057810 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7f450c017000 tx=0x7f450c015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:40.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.235+0000 7f45067fc700 1 -- 172.21.15.162:0/3401719774 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4508044de0 con 0x7f451013b070 2020-11-28T21:55:40.379 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.375+0000 7f45167c0700 1 -- 172.21.15.162:0/3401719774 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f451005d250 con 0x7f451013b070 2020-11-28T21:55:40.380 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.375+0000 7f45067fc700 1 -- 172.21.15.162:0/3401719774 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v138) v1 ==== 104+0+10 (secure 0 0 0) 0x7f450806e030 con 0x7f451013b070 2020-11-28T21:55:40.381 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:55:40.383 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.379+0000 7f45167c0700 1 -- 172.21.15.162:0/3401719774 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f44f8055350 msgr2=0x7f44f8057810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:40.384 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.379+0000 7f45167c0700 1 --2- 172.21.15.162:0/3401719774 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f44f8055350 0x7f44f8057810 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7f450c017000 tx=0x7f450c015000).stop 2020-11-28T21:55:40.384 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.379+0000 7f45167c0700 1 -- 172.21.15.162:0/3401719774 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f451013b070 msgr2=0x7f45100af680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:40.384 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.379+0000 7f45167c0700 1 --2- 172.21.15.162:0/3401719774 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f451013b070 0x7f45100af680 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 rx=0x7f450800b000 tx=0x7f4508008040).stop 2020-11-28T21:55:40.384 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.379+0000 7f45167c0700 1 -- 172.21.15.162:0/3401719774 shutdown_connections 2020-11-28T21:55:40.384 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.379+0000 7f45167c0700 1 --2- 172.21.15.162:0/3401719774 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f44f8055350 0x7f44f8057810 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:40.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.379+0000 7f45167c0700 1 --2- 172.21.15.162:0/3401719774 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f451013b070 0x7f45100af680 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:40.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.379+0000 7f45167c0700 1 --2- 172.21.15.162:0/3401719774 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f45101420f0 0x7f45100aea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:40.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.379+0000 7f45167c0700 1 --2- 172.21.15.162:0/3401719774 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4510143ed0 0x7f451013ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:40.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.379+0000 7f45167c0700 1 -- 172.21.15.162:0/3401719774 >> 172.21.15.162:0/3401719774 conn(0x7f45100604c0 msgr2=0x7f45100b3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:40.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.379+0000 7f45167c0700 1 -- 172.21.15.162:0/3401719774 shutdown_connections 2020-11-28T21:55:40.386 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.379+0000 7f45167c0700 1 -- 172.21.15.162:0/3401719774 wait complete. 2020-11-28T21:55:40.400 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-28T21:55:40.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.511+0000 7f294e0dc700 1 Processor -- start 2020-11-28T21:55:40.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.511+0000 7f294e0dc700 1 -- start start 2020-11-28T21:55:40.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.511+0000 7f294e0dc700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f29481433c0 0x7f29480605c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:40.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.511+0000 7f294e0dc700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2948067610 0x7f294813fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:40.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.511+0000 7f294e0dc700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2948143950 con 0x7f29480b9ce0 2020-11-28T21:55:40.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.511+0000 7f294e0dc700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29480b4340 con 0x7f29481433c0 2020-11-28T21:55:40.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.511+0000 7f294e0dc700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2948067ac0 con 0x7f2948067610 2020-11-28T21:55:40.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.511+0000 7f29477fe700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f29480b9ce0 0x7f29480b3c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:52018/0 (socket says 172.21.15.162:52018) 2020-11-28T21:55:40.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.511+0000 7f29477fe700 1 -- 172.21.15.162:0/2814922450 learned_addr learned my addr 172.21.15.162:0/2814922450 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:40.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.515+0000 7f2946ffd700 1 --2- 172.21.15.162:0/2814922450 >> v2:172.21.15.118:3300/0 conn(0x7f29481433c0 0x7f29480605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:40.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.515+0000 7f2947fff700 1 --2- 172.21.15.162:0/2814922450 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2948067610 0x7f294813fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:40.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.515+0000 7f29467fc700 1 -- 172.21.15.162:0/2814922450 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2199706299 0 0) 0x7f2948143950 con 0x7f29480b9ce0 2020-11-28T21:55:40.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.515+0000 7f29467fc700 1 -- 172.21.15.162:0/2814922450 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2928003230 con 0x7f29480b9ce0 2020-11-28T21:55:40.521 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.515+0000 7f29467fc700 1 -- 172.21.15.162:0/2814922450 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 823868878 0 0) 0x7f2928003230 con 0x7f29480b9ce0 2020-11-28T21:55:40.522 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.515+0000 7f2947fff700 1 -- 172.21.15.162:0/2814922450 >> v2:172.21.15.118:3300/0 conn(0x7f29481433c0 msgr2=0x7f29480605c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:40.522 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.515+0000 7f2947fff700 1 --2- 172.21.15.162:0/2814922450 >> v2:172.21.15.118:3300/0 conn(0x7f29481433c0 0x7f29480605c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:40.522 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.515+0000 7f2947fff700 1 -- 172.21.15.162:0/2814922450 >> v1:172.21.15.162:6789/0 conn(0x7f29480b9ce0 legacy=0x7f29480b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:40.522 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.515+0000 7f2947fff700 1 -- 172.21.15.162:0/2814922450 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29480ba0c0 con 0x7f2948067610 2020-11-28T21:55:40.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.515+0000 7f2946ffd700 1 --2- 172.21.15.162:0/2814922450 >> v2:172.21.15.118:3300/0 conn(0x7f29481433c0 0x7f29480605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:55:40.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.515+0000 7f2947fff700 1 --2- 172.21.15.162:0/2814922450 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2948067610 0x7f294813fc80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f29380019f0 tx=0x7f29400091a0).ready entity=mon.2 client_cookie=f7a1cea46187b932 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:40.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.515+0000 7f29467fc700 1 -- 172.21.15.162:0/2814922450 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2938001850 con 0x7f2948067610 2020-11-28T21:55:40.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.515+0000 7f29467fc700 1 -- 172.21.15.162:0/2814922450 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f294000b800 con 0x7f2948067610 2020-11-28T21:55:40.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.515+0000 7f29467fc700 1 -- 172.21.15.162:0/2814922450 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f294000b970 con 0x7f2948067610 2020-11-28T21:55:40.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.515+0000 7f294e0dc700 1 -- 172.21.15.162:0/2814922450 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2948067610 msgr2=0x7f294813fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:40.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.515+0000 7f294e0dc700 1 --2- 172.21.15.162:0/2814922450 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2948067610 0x7f294813fc80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f29380019f0 tx=0x7f29400091a0).stop 2020-11-28T21:55:40.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.515+0000 7f294e0dc700 1 -- 172.21.15.162:0/2814922450 shutdown_connections 2020-11-28T21:55:40.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.515+0000 7f294e0dc700 1 --2- 172.21.15.162:0/2814922450 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2948067610 0x7f294813fc80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:40.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.515+0000 7f294e0dc700 1 --2- 172.21.15.162:0/2814922450 >> v2:172.21.15.118:3300/0 conn(0x7f29481433c0 0x7f29480605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:40.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.515+0000 7f294e0dc700 1 -- 172.21.15.162:0/2814922450 >> 172.21.15.162:0/2814922450 conn(0x7f294805c430 msgr2=0x7f29480b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:40.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.515+0000 7f294e0dc700 1 -- 172.21.15.162:0/2814922450 shutdown_connections 2020-11-28T21:55:40.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.515+0000 7f294e0dc700 1 -- 172.21.15.162:0/2814922450 wait complete. 2020-11-28T21:55:40.526 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.515+0000 7f294e0dc700 1 Processor -- start 2020-11-28T21:55:40.526 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.515+0000 7f294e0dc700 1 -- start start 2020-11-28T21:55:40.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.519+0000 7f294e0dc700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2948067610 0x7f2948141bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:40.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.519+0000 7f294e0dc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f29480b9ce0 0x7f2948152010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:40.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.519+0000 7f294e0dc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29481433c0 0x7f2948154c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:40.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.519+0000 7f294e0dc700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29481516e0 con 0x7f29480b9ce0 2020-11-28T21:55:40.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.519+0000 7f294e0dc700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29480ae800 con 0x7f29481433c0 2020-11-28T21:55:40.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.519+0000 7f294e0dc700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29480aeaa0 con 0x7f2948067610 2020-11-28T21:55:40.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.519+0000 7f2947fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29481433c0 0x7f2948154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:40.530 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.519+0000 7f2947fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29481433c0 0x7f2948154c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45182/0 (socket says 172.21.15.162:45182) 2020-11-28T21:55:40.530 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.519+0000 7f2947fff700 1 -- 172.21.15.162:0/3395337340 learned_addr learned my addr 172.21.15.162:0/3395337340 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:40.530 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.519+0000 7f2946ffd700 1 --2- 172.21.15.162:0/3395337340 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f29480b9ce0 0x7f2948152010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:40.530 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.519+0000 7f29477fe700 1 --2- 172.21.15.162:0/3395337340 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2948067610 0x7f2948141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:40.531 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.519+0000 7f2947fff700 1 -- 172.21.15.162:0/3395337340 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2948067610 msgr2=0x7f2948141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:40.531 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.519+0000 7f2947fff700 1 --2- 172.21.15.162:0/3395337340 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2948067610 0x7f2948141bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:40.531 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.519+0000 7f2947fff700 1 -- 172.21.15.162:0/3395337340 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f29480b9ce0 msgr2=0x7f2948152010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:40.531 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.519+0000 7f2947fff700 1 --2- 172.21.15.162:0/3395337340 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f29480b9ce0 0x7f2948152010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:40.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.519+0000 7f2947fff700 1 -- 172.21.15.162:0/3395337340 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29481518d0 con 0x7f29481433c0 2020-11-28T21:55:40.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.519+0000 7f2946ffd700 1 --2- 172.21.15.162:0/3395337340 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f29480b9ce0 0x7f2948152010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:55:40.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.519+0000 7f29477fe700 1 --2- 172.21.15.162:0/3395337340 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2948067610 0x7f2948141bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:55:40.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.519+0000 7f2947fff700 1 --2- 172.21.15.162:0/3395337340 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29481433c0 0x7f2948154c30 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 rx=0x7f29380019f0 tx=0x7f2940016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:40.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.519+0000 7f2944ff9700 1 -- 172.21.15.162:0/3395337340 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2940011ce0 con 0x7f29481433c0 2020-11-28T21:55:40.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.519+0000 7f294e0dc700 1 -- 172.21.15.162:0/3395337340 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2948151b70 con 0x7f29481433c0 2020-11-28T21:55:40.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.519+0000 7f2944ff9700 1 -- 172.21.15.162:0/3395337340 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2940011e40 con 0x7f29481433c0 2020-11-28T21:55:40.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.519+0000 7f2944ff9700 1 -- 172.21.15.162:0/3395337340 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2940021630 con 0x7f29481433c0 2020-11-28T21:55:40.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.519+0000 7f294e0dc700 1 -- 172.21.15.162:0/3395337340 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2948152500 con 0x7f29481433c0 2020-11-28T21:55:40.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.519+0000 7f2944ff9700 1 -- 172.21.15.162:0/3395337340 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2940027020 con 0x7f29481433c0 2020-11-28T21:55:40.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.523+0000 7f294e0dc700 1 -- 172.21.15.162:0/3395337340 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2948151d00 con 0x7f29481433c0 2020-11-28T21:55:40.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.523+0000 7f2944ff9700 1 --2- 172.21.15.162:0/3395337340 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f292c0554e0 0x7f292c0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:40.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.523+0000 7f2944ff9700 1 -- 172.21.15.162:0/3395337340 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f294006f080 con 0x7f29481433c0 2020-11-28T21:55:40.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.523+0000 7f29477fe700 1 --2- 172.21.15.162:0/3395337340 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f292c0554e0 0x7f292c0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:40.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.523+0000 7f29477fe700 1 --2- 172.21.15.162:0/3395337340 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f292c0554e0 0x7f292c0579a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7f293800b000 tx=0x7f2938009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:40.535 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.527+0000 7f2944ff9700 1 -- 172.21.15.162:0/3395337340 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f29400451f0 con 0x7f29481433c0 2020-11-28T21:55:40.672 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.667+0000 7f294e0dc700 1 -- 172.21.15.162:0/3395337340 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2948057050 con 0x7f29481433c0 2020-11-28T21:55:40.673 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.667+0000 7f2944ff9700 1 -- 172.21.15.162:0/3395337340 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v138) v1 ==== 100+0+10 (secure 0 0 0) 0x7f29400150a0 con 0x7f29481433c0 2020-11-28T21:55:40.673 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:55:40.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.671+0000 7f294e0dc700 1 -- 172.21.15.162:0/3395337340 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f292c0554e0 msgr2=0x7f292c0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:40.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.671+0000 7f294e0dc700 1 --2- 172.21.15.162:0/3395337340 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f292c0554e0 0x7f292c0579a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7f293800b000 tx=0x7f2938009040).stop 2020-11-28T21:55:40.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.671+0000 7f294e0dc700 1 -- 172.21.15.162:0/3395337340 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29481433c0 msgr2=0x7f2948154c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:40.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.671+0000 7f294e0dc700 1 --2- 172.21.15.162:0/3395337340 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29481433c0 0x7f2948154c30 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 rx=0x7f29380019f0 tx=0x7f2940016000).stop 2020-11-28T21:55:40.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.671+0000 7f294e0dc700 1 -- 172.21.15.162:0/3395337340 shutdown_connections 2020-11-28T21:55:40.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.671+0000 7f294e0dc700 1 --2- 172.21.15.162:0/3395337340 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f292c0554e0 0x7f292c0579a0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:40.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.671+0000 7f294e0dc700 1 --2- 172.21.15.162:0/3395337340 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2948067610 0x7f2948141bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:40.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.671+0000 7f294e0dc700 1 --2- 172.21.15.162:0/3395337340 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29481433c0 0x7f2948154c30 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:40.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.671+0000 7f294e0dc700 1 --2- 172.21.15.162:0/3395337340 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f29480b9ce0 0x7f2948152010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:40.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.671+0000 7f294e0dc700 1 -- 172.21.15.162:0/3395337340 >> 172.21.15.162:0/3395337340 conn(0x7f294805c430 msgr2=0x7f294813e640 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:40.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.671+0000 7f294e0dc700 1 -- 172.21.15.162:0/3395337340 shutdown_connections 2020-11-28T21:55:40.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.671+0000 7f294e0dc700 1 -- 172.21.15.162:0/3395337340 wait complete. 2020-11-28T21:55:40.694 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-28T21:55:40.695 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:55:40.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.799+0000 7f48b4fa5700 1 Processor -- start 2020-11-28T21:55:40.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.799+0000 7f48b4fa5700 1 -- start start 2020-11-28T21:55:40.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.799+0000 7f48b4fa5700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f48b00603d0 0x7f48b00b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:40.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.799+0000 7f48b4fa5700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48b00b4110 con 0x7f48b00b4c70 2020-11-28T21:55:40.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.799+0000 7f48b4fa5700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48b00b43f0 con 0x7f48b0142a00 2020-11-28T21:55:40.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.799+0000 7f48b4fa5700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b0060f60 con 0x7f48b00603d0 2020-11-28T21:55:40.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.799+0000 7f48aeffd700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f48b0142a00 0x7f48b009cfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:52034/0 (socket says 172.21.15.162:52034) 2020-11-28T21:55:40.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.799+0000 7f48aeffd700 1 -- 172.21.15.162:0/2571299479 learned_addr learned my addr 172.21.15.162:0/2571299479 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:40.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.799+0000 7f48af7fe700 1 --2- 172.21.15.162:0/2571299479 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f48b00603d0 0x7f48b00b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:40.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.803+0000 7f48ae7fc700 1 -- 172.21.15.162:0/2571299479 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4259589168 0 0) 0x7f48b00b43f0 con 0x7f48b0142a00 2020-11-28T21:55:40.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.803+0000 7f48ae7fc700 1 -- 172.21.15.162:0/2571299479 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4890003230 con 0x7f48b0142a00 2020-11-28T21:55:40.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.803+0000 7f48ae7fc700 1 -- 172.21.15.162:0/2571299479 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3183525208 0 0) 0x7f48b00b4110 con 0x7f48b00b4c70 2020-11-28T21:55:40.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.803+0000 7f48af7fe700 1 -- 172.21.15.162:0/2571299479 >> v1:172.21.15.162:6789/0 conn(0x7f48b0142a00 legacy=0x7f48b009cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:40.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.803+0000 7f48af7fe700 1 -- 172.21.15.162:0/2571299479 >> v1:172.21.15.118:6789/0 conn(0x7f48b00b4c70 legacy=0x7f48b0143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:40.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.803+0000 7f48af7fe700 1 -- 172.21.15.162:0/2571299479 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b00b4690 con 0x7f48b00603d0 2020-11-28T21:55:40.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.803+0000 7f48af7fe700 1 --2- 172.21.15.162:0/2571299479 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f48b00603d0 0x7f48b00b3a70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f48a00032b0 tx=0x7f48a00091a0).ready entity=mon.2 client_cookie=58e6aca3bd073532 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:40.810 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.803+0000 7f48ae7fc700 1 -- 172.21.15.162:0/2571299479 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48a000b9f0 con 0x7f48b00603d0 2020-11-28T21:55:40.810 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.803+0000 7f48ae7fc700 1 -- 172.21.15.162:0/2571299479 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a000bb50 con 0x7f48b00603d0 2020-11-28T21:55:40.810 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.803+0000 7f48ae7fc700 1 -- 172.21.15.162:0/2571299479 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48a000bcc0 con 0x7f48b00603d0 2020-11-28T21:55:40.811 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.803+0000 7f48b4fa5700 1 -- 172.21.15.162:0/2571299479 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f48b00603d0 msgr2=0x7f48b00b3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:40.811 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.803+0000 7f48b4fa5700 1 --2- 172.21.15.162:0/2571299479 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f48b00603d0 0x7f48b00b3a70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f48a00032b0 tx=0x7f48a00091a0).stop 2020-11-28T21:55:40.811 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.803+0000 7f48b4fa5700 1 -- 172.21.15.162:0/2571299479 shutdown_connections 2020-11-28T21:55:40.812 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.803+0000 7f48b4fa5700 1 --2- 172.21.15.162:0/2571299479 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f48b00603d0 0x7f48b00b3a70 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:40.812 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.803+0000 7f48b4fa5700 1 -- 172.21.15.162:0/2571299479 >> 172.21.15.162:0/2571299479 conn(0x7f48b0066a40 msgr2=0x7f48b0066e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:40.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.803+0000 7f48b4fa5700 1 -- 172.21.15.162:0/2571299479 shutdown_connections 2020-11-28T21:55:40.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.803+0000 7f48b4fa5700 1 -- 172.21.15.162:0/2571299479 wait complete. 2020-11-28T21:55:40.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.803+0000 7f48b4fa5700 1 Processor -- start 2020-11-28T21:55:40.813 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.803+0000 7f48b4fa5700 1 -- start start 2020-11-28T21:55:40.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.803+0000 7f48b4fa5700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f48b00603d0 0x7f48b00b1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:40.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.803+0000 7f48b4fa5700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48b00b4c70 0x7f48b00ad880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:40.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.803+0000 7f48b4fa5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48b00b3170 0x7f48b01516a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:40.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.803+0000 7f48b4fa5700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b00b1e80 con 0x7f48b00b4c70 2020-11-28T21:55:40.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.803+0000 7f48b4fa5700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b0151fc0 con 0x7f48b00b3170 2020-11-28T21:55:40.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.803+0000 7f48b4fa5700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b00b1250 con 0x7f48b00603d0 2020-11-28T21:55:40.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.807+0000 7f48affff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48b00b3170 0x7f48b01516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:40.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.807+0000 7f48affff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48b00b3170 0x7f48b01516a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45196/0 (socket says 172.21.15.162:45196) 2020-11-28T21:55:40.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.807+0000 7f48affff700 1 -- 172.21.15.162:0/991931776 learned_addr learned my addr 172.21.15.162:0/991931776 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:40.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.807+0000 7f48af7fe700 1 --2- 172.21.15.162:0/991931776 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f48b00603d0 0x7f48b00b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:40.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.807+0000 7f48aeffd700 1 --2- 172.21.15.162:0/991931776 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48b00b4c70 0x7f48b00ad880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:40.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.807+0000 7f48affff700 1 -- 172.21.15.162:0/991931776 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f48b00603d0 msgr2=0x7f48b00b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:40.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.807+0000 7f48affff700 1 --2- 172.21.15.162:0/991931776 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f48b00603d0 0x7f48b00b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:40.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.807+0000 7f48affff700 1 -- 172.21.15.162:0/991931776 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48b00b4c70 msgr2=0x7f48b00ad880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:40.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.807+0000 7f48affff700 1 --2- 172.21.15.162:0/991931776 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48b00b4c70 0x7f48b00ad880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:40.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.807+0000 7f48affff700 1 -- 172.21.15.162:0/991931776 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b00b14f0 con 0x7f48b00b3170 2020-11-28T21:55:40.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.807+0000 7f48affff700 1 --2- 172.21.15.162:0/991931776 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48b00b3170 0x7f48b01516a0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 rx=0x7f48a8000b80 tx=0x7f48a80079e0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:40.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.807+0000 7f48acff9700 1 -- 172.21.15.162:0/991931776 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48a800e4a0 con 0x7f48b00b3170 2020-11-28T21:55:40.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.807+0000 7f48acff9700 1 -- 172.21.15.162:0/991931776 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a800e600 con 0x7f48b00b3170 2020-11-28T21:55:40.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.807+0000 7f48acff9700 1 -- 172.21.15.162:0/991931776 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48a800e960 con 0x7f48b00b3170 2020-11-28T21:55:40.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.807+0000 7f48b4fa5700 1 -- 172.21.15.162:0/991931776 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48b00add70 con 0x7f48b00b3170 2020-11-28T21:55:40.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.807+0000 7f48b4fa5700 1 -- 172.21.15.162:0/991931776 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b00ace10 con 0x7f48b00b3170 2020-11-28T21:55:40.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.807+0000 7f48b4fa5700 1 -- 172.21.15.162:0/991931776 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48b0152180 con 0x7f48b00b3170 2020-11-28T21:55:40.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.807+0000 7f48acff9700 1 -- 172.21.15.162:0/991931776 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f48a8010030 con 0x7f48b00b3170 2020-11-28T21:55:40.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.811+0000 7f48acff9700 1 --2- 172.21.15.162:0/991931776 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4894055440 0x7f4894057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:40.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.811+0000 7f48acff9700 1 -- 172.21.15.162:0/991931776 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f48a806de00 con 0x7f48b00b3170 2020-11-28T21:55:40.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.811+0000 7f48af7fe700 1 --2- 172.21.15.162:0/991931776 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4894055440 0x7f4894057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:40.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.811+0000 7f48af7fe700 1 --2- 172.21.15.162:0/991931776 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4894055440 0x7f4894057900 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7f48a0019000 tx=0x7f48a0016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:40.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.815+0000 7f48acff9700 1 -- 172.21.15.162:0/991931776 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f48a8044100 con 0x7f48b00b3170 2020-11-28T21:55:41.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.995+0000 7f48b4fa5700 1 -- 172.21.15.162:0/991931776 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48b0057050 con 0x7f48b00b3170 2020-11-28T21:55:41.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.999+0000 7f48acff9700 1 -- 172.21.15.162:0/991931776 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+2504 (secure 0 0 0) 0x7f48a8047720 con 0x7f48b00b3170 2020-11-28T21:55:41.002 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:41.003 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":68,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7611,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6835/2127102267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2127102267},{"type":"v1","addr":"172.21.15.162:6835","nonce":2127102267}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":66},{"gid":7623,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/1473328171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1473328171},{"type":"v1","addr":"172.21.15.118:6833","nonce":1473328171}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:55:31.379864+0000","modified":"2020-11-28T21:55:33.042143+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7888":{"gid":7888,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/2667777645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2667777645},{"type":"v1","addr":"172.21.15.162:6837","nonce":2667777645}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2020-11-28T21:55:41.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.999+0000 7f48b4fa5700 1 -- 172.21.15.162:0/991931776 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4894055440 msgr2=0x7f4894057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:41.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.999+0000 7f48b4fa5700 1 --2- 172.21.15.162:0/991931776 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4894055440 0x7f4894057900 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7f48a0019000 tx=0x7f48a0016000).stop 2020-11-28T21:55:41.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.999+0000 7f48b4fa5700 1 -- 172.21.15.162:0/991931776 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48b00b3170 msgr2=0x7f48b01516a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:41.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.999+0000 7f48b4fa5700 1 --2- 172.21.15.162:0/991931776 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48b00b3170 0x7f48b01516a0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 rx=0x7f48a8000b80 tx=0x7f48a80079e0).stop 2020-11-28T21:55:41.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.999+0000 7f48b4fa5700 1 -- 172.21.15.162:0/991931776 shutdown_connections 2020-11-28T21:55:41.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.999+0000 7f48b4fa5700 1 --2- 172.21.15.162:0/991931776 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4894055440 0x7f4894057900 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:41.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.999+0000 7f48b4fa5700 1 --2- 172.21.15.162:0/991931776 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f48b00603d0 0x7f48b00b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:41.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.999+0000 7f48b4fa5700 1 --2- 172.21.15.162:0/991931776 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48b00b3170 0x7f48b01516a0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:41.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.999+0000 7f48b4fa5700 1 --2- 172.21.15.162:0/991931776 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48b00b4c70 0x7f48b00ad880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:41.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.999+0000 7f48b4fa5700 1 -- 172.21.15.162:0/991931776 >> 172.21.15.162:0/991931776 conn(0x7f48b0066a40 msgr2=0x7f48b00b5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:41.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.999+0000 7f48b4fa5700 1 -- 172.21.15.162:0/991931776 shutdown_connections 2020-11-28T21:55:41.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:40.999+0000 7f48b4fa5700 1 -- 172.21.15.162:0/991931776 wait complete. 2020-11-28T21:55:41.007 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 68 2020-11-28T21:55:41.318 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-28T21:55:31.379864+0000', 'modified': '2020-11-28T21:55:33.042143+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7888}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7888': {'gid': 7888, 'name': 'c', 'rank': 0, 'incarnation': 66, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6837/2667777645', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6836', 'nonce': 2667777645}, {'type': 'v1', 'addr': '172.21.15.162:6837', 'nonce': 2667777645}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-28T21:55:41.319 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-28T21:55:41.320 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-28T21:55:41.449 INFO:teuthology.orchestra.run.smithi162.stdout:{"cluster_fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","whoami":0,"id":7888,"want_state":"up:active","state":"up:active","rank_uptime":10.054829174,"mdsmap_epoch":68,"osdmap_epoch":138,"osdmap_epoch_barrier":138,"uptime":14.322453854999999} 2020-11-28T21:55:41.449 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", "id": 7888, "mdsmap_epoch": 68, "osdmap_epoch": 138, "osdmap_epoch_barrier": 138, "rank_uptime": 10.054829174, "state": "up:active", "uptime": 14.322453855, "want_state": "up:active", "whoami": 0 } 2020-11-28T21:55:41.450 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-11-28T21:55:41.451 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2020-11-28T21:55:41.451 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi096.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2020-11-28T21:55:41.451 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:41.452 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:55:41.460 INFO:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2020-11-28T21:55:41.516 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:55:41.516 INFO:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:55:41.570 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:55:41.572 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:55:41.573 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:41.574 INFO:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2020-11-28T21:55:41.623 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-11-28T21:55:41.624 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:55:41.667 INFO:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2020-11-28T21:55:41.684 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:55:41.685 INFO:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:55:41.739 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:55:41.740 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:55:41.741 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:41.741 INFO:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2020-11-28T21:55:41.743 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:2020-11-28T21:55:41.736+0000 7fc5ff49c0c0 -1 init, newargv = 0x55960a95a450 newargc=9 2020-11-28T21:55:41.744 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[12003]: starting ceph client 2020-11-28T21:55:41.754 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[12003]: starting fuse 2020-11-28T21:55:42.851 INFO:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2020-11-28T21:55:42.867 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:55:42.868 INFO:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:55:42.922 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:55:42.923 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:55:42.924 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:42.924 INFO:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2020-11-28T21:55:42.971 INFO:teuthology.orchestra.run.smithi096.stdout:49 2020-11-28T21:55:42.971 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2020-11-28T21:55:42.972 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:42.972 INFO:teuthology.orchestra.run.smithi096:> sudo python3 -c ' 2020-11-28T21:55:42.972 INFO:teuthology.orchestra.run.smithi096:> import glob 2020-11-28T21:55:42.973 INFO:teuthology.orchestra.run.smithi096:> import re 2020-11-28T21:55:42.973 INFO:teuthology.orchestra.run.smithi096:> import os 2020-11-28T21:55:42.974 INFO:teuthology.orchestra.run.smithi096:> import subprocess 2020-11-28T21:55:42.974 INFO:teuthology.orchestra.run.smithi096:> 2020-11-28T21:55:42.975 INFO:teuthology.orchestra.run.smithi096:> def find_socket(client_name): 2020-11-28T21:55:42.975 INFO:teuthology.orchestra.run.smithi096:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2020-11-28T21:55:42.976 INFO:teuthology.orchestra.run.smithi096:> files = glob.glob(asok_path) 2020-11-28T21:55:42.976 INFO:teuthology.orchestra.run.smithi096:> 2020-11-28T21:55:42.976 INFO:teuthology.orchestra.run.smithi096:> # Given a non-glob path, it better be there 2020-11-28T21:55:42.977 INFO:teuthology.orchestra.run.smithi096:> if "*" not in asok_path: 2020-11-28T21:55:42.977 INFO:teuthology.orchestra.run.smithi096:> assert(len(files) == 1) 2020-11-28T21:55:42.978 INFO:teuthology.orchestra.run.smithi096:> return files[0] 2020-11-28T21:55:42.978 INFO:teuthology.orchestra.run.smithi096:> 2020-11-28T21:55:42.979 INFO:teuthology.orchestra.run.smithi096:> for f in files: 2020-11-28T21:55:42.979 INFO:teuthology.orchestra.run.smithi096:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-11-28T21:55:42.979 INFO:teuthology.orchestra.run.smithi096:> if os.path.exists("/proc/{0}".format(pid)): 2020-11-28T21:55:42.980 INFO:teuthology.orchestra.run.smithi096:> return f 2020-11-28T21:55:42.980 INFO:teuthology.orchestra.run.smithi096:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-11-28T21:55:42.981 INFO:teuthology.orchestra.run.smithi096:> 2020-11-28T21:55:42.981 INFO:teuthology.orchestra.run.smithi096:> print(find_socket("client.1")) 2020-11-28T21:55:42.982 INFO:teuthology.orchestra.run.smithi096:> ' 2020-11-28T21:55:43.060 INFO:teuthology.orchestra.run.smithi096.stdout:/var/run/ceph/ceph-client.1.12003.asok 2020-11-28T21:55:43.065 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.12003.asok 2020-11-28T21:55:43.065 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:43.066 INFO:teuthology.orchestra.run.smithi096:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.12003.asok status 2020-11-28T21:55:43.163 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-11-28T21:55:43.163 INFO:teuthology.orchestra.run.smithi096.stdout: "metadata": { 2020-11-28T21:55:43.164 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_sha1": "a83a3572a957356977342f461361c1cd3ec06dbd", 2020-11-28T21:55:43.164 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_version": "ceph version 15.2.6-595-ga83a3572 (a83a3572a957356977342f461361c1cd3ec06dbd) octopus (stable)", 2020-11-28T21:55:43.164 INFO:teuthology.orchestra.run.smithi096.stdout: "entity_id": "1", 2020-11-28T21:55:43.165 INFO:teuthology.orchestra.run.smithi096.stdout: "hostname": "smithi096", 2020-11-28T21:55:43.165 INFO:teuthology.orchestra.run.smithi096.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2020-11-28T21:55:43.165 INFO:teuthology.orchestra.run.smithi096.stdout: "pid": "12003", 2020-11-28T21:55:43.165 INFO:teuthology.orchestra.run.smithi096.stdout: "root": "/" 2020-11-28T21:55:43.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-11-28T21:55:43.166 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_count": 0, 2020-11-28T21:55:43.166 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_pinned_count": 0, 2020-11-28T21:55:43.166 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 8107, 2020-11-28T21:55:43.167 INFO:teuthology.orchestra.run.smithi096.stdout: "inst": { 2020-11-28T21:55:43.167 INFO:teuthology.orchestra.run.smithi096.stdout: "name": { 2020-11-28T21:55:43.167 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "client", 2020-11-28T21:55:43.167 INFO:teuthology.orchestra.run.smithi096.stdout: "num": 8107 2020-11-28T21:55:43.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-11-28T21:55:43.168 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2020-11-28T21:55:43.168 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2020-11-28T21:55:43.168 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2020-11-28T21:55:43.169 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 10694537 2020-11-28T21:55:43.169 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-11-28T21:55:43.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-11-28T21:55:43.169 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2020-11-28T21:55:43.170 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2020-11-28T21:55:43.170 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2020-11-28T21:55:43.170 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 10694537 2020-11-28T21:55:43.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-11-28T21:55:43.171 INFO:teuthology.orchestra.run.smithi096.stdout: "inst_str": "client.8107 172.21.15.96:0/10694537", 2020-11-28T21:55:43.171 INFO:teuthology.orchestra.run.smithi096.stdout: "addr_str": "172.21.15.96:0/10694537", 2020-11-28T21:55:43.171 INFO:teuthology.orchestra.run.smithi096.stdout: "inode_count": 1, 2020-11-28T21:55:43.172 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_epoch": 68, 2020-11-28T21:55:43.172 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch": 138, 2020-11-28T21:55:43.172 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch_barrier": 0, 2020-11-28T21:55:43.172 INFO:teuthology.orchestra.run.smithi096.stdout: "blacklisted": false 2020-11-28T21:55:43.172 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-11-28T21:55:43.175 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:43.175 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:55:43.176 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:55:43.182 INFO:teuthology.orchestra.run.smithi096.stdout:fuseblk 2020-11-28T21:55:43.183 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-11-28T21:55:43.184 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:43.184 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:55:43.242 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:43.242 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:55:43.243 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:55:43.292 INFO:teuthology.orchestra.run.smithi096.stdout:fuseblk 2020-11-28T21:55:43.293 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-11-28T21:55:43.294 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:43.295 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:55:43.295 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:55:43.349 INFO:teuthology.orchestra.run.smithi096.stdout:fuseblk 2020-11-28T21:55:43.349 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-11-28T21:55:43.350 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi096.front.sepia.ceph.com... 2020-11-28T21:55:43.350 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-28T21:55:43.351 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:55:43.421 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[12003]: fuse finished with error 0 and tester_r 0 2020-11-28T21:55:43.423 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:43.424 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:55:43.425 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:55:43.471 INFO:teuthology.orchestra.run.smithi096.stdout:ext2/ext3 2020-11-28T21:55:43.472 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-11-28T21:55:43.473 INFO:teuthology.orchestra.run:waiting for 900 2020-11-28T21:55:43.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-28T21:55:43.539 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:55:43.547 INFO:teuthology.orchestra.run.smithi096:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-28T21:55:43.591 INFO:teuthology.orchestra.run.smithi118:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-28T21:55:43.594 INFO:teuthology.orchestra.run.smithi162:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-28T21:55:43.596 INFO:teuthology.orchestra.run.smithi192:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-28T21:55:43.618 INFO:teuthology.orchestra.run.smithi096.stdout:[global] 2020-11-28T21:55:43.619 INFO:teuthology.orchestra.run.smithi096.stdout: chdir = "" 2020-11-28T21:55:43.619 INFO:teuthology.orchestra.run.smithi096.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-28T21:55:43.620 INFO:teuthology.orchestra.run.smithi096.stdout: auth supported = cephx 2020-11-28T21:55:43.620 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:43.621 INFO:teuthology.orchestra.run.smithi096.stdout: filestore xattr use omap = true 2020-11-28T21:55:43.621 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:43.621 INFO:teuthology.orchestra.run.smithi096.stdout: mon clock drift allowed = 1.000 2020-11-28T21:55:43.622 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:43.622 INFO:teuthology.orchestra.run.smithi096.stdout: osd crush chooseleaf type = 0 2020-11-28T21:55:43.622 INFO:teuthology.orchestra.run.smithi096.stdout: auth debug = true 2020-11-28T21:55:43.623 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:43.624 INFO:teuthology.orchestra.run.smithi096.stdout: ms die on old message = true 2020-11-28T21:55:43.624 INFO:teuthology.orchestra.run.smithi096.stdout: ms die on bug = true 2020-11-28T21:55:43.624 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:43.624 INFO:teuthology.orchestra.run.smithi096.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-28T21:55:43.624 INFO:teuthology.orchestra.run.smithi096.stdout: mon pg warn max object skew = 0 2020-11-28T21:55:43.625 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:43.625 INFO:teuthology.orchestra.run.smithi096.stdout: # disable pg_autoscaler by default for new pools 2020-11-28T21:55:43.625 INFO:teuthology.orchestra.run.smithi096.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-28T21:55:43.625 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:43.626 INFO:teuthology.orchestra.run.smithi096.stdout: osd pool default size = 2 2020-11-28T21:55:43.626 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:43.626 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd allow primary affinity = true 2020-11-28T21:55:43.626 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd allow pg remap = true 2020-11-28T21:55:43.626 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on legacy crush tunables = false 2020-11-28T21:55:43.627 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on crush straw calc version zero = false 2020-11-28T21:55:43.627 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on no sortbitwise = false 2020-11-28T21:55:43.627 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on osd down out interval zero = false 2020-11-28T21:55:43.627 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on too few osds = false 2020-11-28T21:55:43.627 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-28T21:55:43.628 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-28T21:55:43.628 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:43.628 INFO:teuthology.orchestra.run.smithi096.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-28T21:55:43.628 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:43.628 INFO:teuthology.orchestra.run.smithi096.stdout: osd default data pool replay window = 5 2020-11-28T21:55:43.629 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:43.629 INFO:teuthology.orchestra.run.smithi096.stdout: mon allow pool delete = true 2020-11-28T21:55:43.629 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:43.629 INFO:teuthology.orchestra.run.smithi096.stdout: mon cluster log file level = debug 2020-11-28T21:55:43.629 INFO:teuthology.orchestra.run.smithi096.stdout: debug asserts on shutdown = true 2020-11-28T21:55:43.630 INFO:teuthology.orchestra.run.smithi096.stdout: mon host = "172.21.15.162,172.21.15.118,[v2:172.21.15.118:3301,v1:172.21.15.118:6790]" 2020-11-28T21:55:43.630 INFO:teuthology.orchestra.run.smithi096.stdout: fsid = 7c321bcf-cd33-40f7-a39f-da5470b386cf 2020-11-28T21:55:43.630 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:43.630 INFO:teuthology.orchestra.run.smithi096.stdout:[osd] 2020-11-28T21:55:43.631 INFO:teuthology.orchestra.run.smithi096.stdout: osd journal size = 100 2020-11-28T21:55:43.631 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:43.631 INFO:teuthology.orchestra.run.smithi096.stdout: osd scrub load threshold = 5.0 2020-11-28T21:55:43.631 INFO:teuthology.orchestra.run.smithi096.stdout: osd scrub max interval = 600 2020-11-28T21:55:43.631 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:43.632 INFO:teuthology.orchestra.run.smithi096.stdout: osd recover clone overlap = true 2020-11-28T21:55:43.632 INFO:teuthology.orchestra.run.smithi096.stdout: osd recovery max chunk = 1048576 2020-11-28T21:55:43.632 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:43.632 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug shutdown = true 2020-11-28T21:55:43.632 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug op order = true 2020-11-28T21:55:43.633 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug verify stray on activate = true 2020-11-28T21:55:43.633 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:43.633 INFO:teuthology.orchestra.run.smithi096.stdout: osd open classes on start = true 2020-11-28T21:55:43.633 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug pg log writeout = true 2020-11-28T21:55:43.633 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:43.634 INFO:teuthology.orchestra.run.smithi096.stdout: osd deep scrub update digest min age = 30 2020-11-28T21:55:43.634 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:43.634 INFO:teuthology.orchestra.run.smithi096.stdout: osd map max advance = 10 2020-11-28T21:55:43.634 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:43.634 INFO:teuthology.orchestra.run.smithi096.stdout: journal zero on create = true 2020-11-28T21:55:43.635 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:43.635 INFO:teuthology.orchestra.run.smithi096.stdout: filestore ondisk finisher threads = 3 2020-11-28T21:55:43.635 INFO:teuthology.orchestra.run.smithi096.stdout: filestore apply finisher threads = 3 2020-11-28T21:55:43.635 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:43.635 INFO:teuthology.orchestra.run.smithi096.stdout: bdev debug aio = true 2020-11-28T21:55:43.635 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug misdirected ops = true 2020-11-28T21:55:43.636 INFO:teuthology.orchestra.run.smithi096.stdout: bluestore block size = 96636764160 2020-11-28T21:55:43.636 INFO:teuthology.orchestra.run.smithi096.stdout: bluestore compression mode = aggressive 2020-11-28T21:55:43.636 INFO:teuthology.orchestra.run.smithi096.stdout: bluestore fsck on mount = True 2020-11-28T21:55:43.636 INFO:teuthology.orchestra.run.smithi096.stdout: debug bluefs = 20 2020-11-28T21:55:43.636 INFO:teuthology.orchestra.run.smithi096.stdout: debug bluestore = 20 2020-11-28T21:55:43.637 INFO:teuthology.orchestra.run.smithi096.stdout: debug filestore = 20 2020-11-28T21:55:43.637 INFO:teuthology.orchestra.run.smithi096.stdout: debug journal = 20 2020-11-28T21:55:43.637 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 20 2020-11-28T21:55:43.637 INFO:teuthology.orchestra.run.smithi096.stdout: debug osd = 25 2020-11-28T21:55:43.637 INFO:teuthology.orchestra.run.smithi096.stdout: debug rocksdb = 10 2020-11-28T21:55:43.638 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd backfillfull_ratio = 0.85 2020-11-28T21:55:43.638 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd full ratio = 0.9 2020-11-28T21:55:43.638 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd nearfull ratio = 0.8 2020-11-28T21:55:43.638 INFO:teuthology.orchestra.run.smithi096.stdout: osd failsafe full ratio = 0.95 2020-11-28T21:55:43.638 INFO:teuthology.orchestra.run.smithi096.stdout: osd objectstore = bluestore 2020-11-28T21:55:43.639 INFO:teuthology.orchestra.run.smithi096.stdout: osd op complaint time = 180 2020-11-28T21:55:43.639 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:43.639 INFO:teuthology.orchestra.run.smithi096.stdout:[mgr] 2020-11-28T21:55:43.639 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2020-11-28T21:55:43.639 INFO:teuthology.orchestra.run.smithi096.stdout: debug mgr = 20 2020-11-28T21:55:43.640 INFO:teuthology.orchestra.run.smithi096.stdout: debug mon = 20 2020-11-28T21:55:43.640 INFO:teuthology.orchestra.run.smithi096.stdout: debug auth = 20 2020-11-28T21:55:43.640 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min pgs per osd = 4 2020-11-28T21:55:43.640 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min bytes per osd = 10 2020-11-28T21:55:43.640 INFO:teuthology.orchestra.run.smithi096.stdout: mgr/telemetry/nag = false 2020-11-28T21:55:43.641 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:43.641 INFO:teuthology.orchestra.run.smithi096.stdout:[mon] 2020-11-28T21:55:43.641 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2020-11-28T21:55:43.641 INFO:teuthology.orchestra.run.smithi096.stdout: debug mon = 20 2020-11-28T21:55:43.641 INFO:teuthology.orchestra.run.smithi096.stdout: debug paxos = 20 2020-11-28T21:55:43.642 INFO:teuthology.orchestra.run.smithi096.stdout: debug auth = 20 2020-11-28T21:55:43.642 INFO:teuthology.orchestra.run.smithi096.stdout: mon data avail warn = 5 2020-11-28T21:55:43.642 INFO:teuthology.orchestra.run.smithi096.stdout: mon mgr mkfs grace = 240 2020-11-28T21:55:43.642 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min pgs per osd = 4 2020-11-28T21:55:43.642 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd reporter subtree level = osd 2020-11-28T21:55:43.643 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd prime pg temp = true 2020-11-28T21:55:43.643 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min bytes per osd = 10 2020-11-28T21:55:43.643 INFO:teuthology.orchestra.run.smithi096.stdout: mon op complaint time = 120 2020-11-28T21:55:43.643 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:43.643 INFO:teuthology.orchestra.run.smithi096.stdout:[client] 2020-11-28T21:55:43.644 INFO:teuthology.orchestra.run.smithi096.stdout: rgw cache enabled = true 2020-11-28T21:55:43.644 INFO:teuthology.orchestra.run.smithi096.stdout: rgw enable ops log = true 2020-11-28T21:55:43.644 INFO:teuthology.orchestra.run.smithi096.stdout: rgw enable usage log = true 2020-11-28T21:55:43.644 INFO:teuthology.orchestra.run.smithi096.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-28T21:55:43.644 INFO:teuthology.orchestra.run.smithi096.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-28T21:55:43.645 INFO:teuthology.orchestra.run.smithi096.stdout: client mount timeout = 600 2020-11-28T21:55:43.645 INFO:teuthology.orchestra.run.smithi096.stdout: debug client = 20 2020-11-28T21:55:43.645 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2020-11-28T21:55:43.645 INFO:teuthology.orchestra.run.smithi096.stdout: rados mon op timeout = 15m 2020-11-28T21:55:43.645 INFO:teuthology.orchestra.run.smithi096.stdout: rados osd op timeout = 15m 2020-11-28T21:55:43.646 INFO:teuthology.orchestra.run.smithi096.stdout: client debug getattr caps = true 2020-11-28T21:55:43.646 INFO:teuthology.orchestra.run.smithi096.stdout:[mon.a] 2020-11-28T21:55:43.646 INFO:teuthology.orchestra.run.smithi096.stdout:[mon.b] 2020-11-28T21:55:43.646 INFO:teuthology.orchestra.run.smithi096.stdout:[mon.c] 2020-11-28T21:55:43.646 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.a] 2020-11-28T21:55:43.647 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.c] 2020-11-28T21:55:43.647 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.b] 2020-11-28T21:55:43.647 INFO:teuthology.orchestra.run.smithi096.stdout:[mds] 2020-11-28T21:55:43.647 INFO:teuthology.orchestra.run.smithi096.stdout: debug mds = 20 2020-11-28T21:55:43.647 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2020-11-28T21:55:43.648 INFO:teuthology.orchestra.run.smithi096.stdout: mds bal frag = True 2020-11-28T21:55:43.648 INFO:teuthology.orchestra.run.smithi096.stdout: mds bal fragment size max = 10000 2020-11-28T21:55:43.648 INFO:teuthology.orchestra.run.smithi096.stdout: mds bal merge size = 5 2020-11-28T21:55:43.648 INFO:teuthology.orchestra.run.smithi096.stdout: mds bal split bits = 3 2020-11-28T21:55:43.648 INFO:teuthology.orchestra.run.smithi096.stdout: mds bal split size = 100 2020-11-28T21:55:43.649 INFO:teuthology.orchestra.run.smithi096.stdout: mds debug frag = True 2020-11-28T21:55:43.649 INFO:teuthology.orchestra.run.smithi096.stdout: mds debug scatterstat = True 2020-11-28T21:55:43.649 INFO:teuthology.orchestra.run.smithi096.stdout: mds op complaint time = 180 2020-11-28T21:55:43.649 INFO:teuthology.orchestra.run.smithi096.stdout: mds verify scatter = True 2020-11-28T21:55:43.649 INFO:teuthology.orchestra.run.smithi096.stdout: osd op complaint time = 180 2020-11-28T21:55:43.650 INFO:teuthology.orchestra.run.smithi096.stdout: rados mon op timeout = 15m 2020-11-28T21:55:43.650 INFO:teuthology.orchestra.run.smithi096.stdout: rados osd op timeout = 15m 2020-11-28T21:55:43.652 INFO:teuthology.orchestra.run.smithi192.stdout:[global] 2020-11-28T21:55:43.652 INFO:teuthology.orchestra.run.smithi192.stdout: chdir = "" 2020-11-28T21:55:43.653 INFO:teuthology.orchestra.run.smithi192.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-28T21:55:43.653 INFO:teuthology.orchestra.run.smithi192.stdout: auth supported = cephx 2020-11-28T21:55:43.653 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:43.653 INFO:teuthology.orchestra.run.smithi192.stdout: filestore xattr use omap = true 2020-11-28T21:55:43.654 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:43.654 INFO:teuthology.orchestra.run.smithi192.stdout: mon clock drift allowed = 1.000 2020-11-28T21:55:43.654 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:43.654 INFO:teuthology.orchestra.run.smithi192.stdout: osd crush chooseleaf type = 0 2020-11-28T21:55:43.654 INFO:teuthology.orchestra.run.smithi192.stdout: auth debug = true 2020-11-28T21:55:43.655 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:43.655 INFO:teuthology.orchestra.run.smithi192.stdout: ms die on old message = true 2020-11-28T21:55:43.655 INFO:teuthology.orchestra.run.smithi192.stdout: ms die on bug = true 2020-11-28T21:55:43.655 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:43.655 INFO:teuthology.orchestra.run.smithi192.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-28T21:55:43.656 INFO:teuthology.orchestra.run.smithi192.stdout: mon pg warn max object skew = 0 2020-11-28T21:55:43.656 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:43.656 INFO:teuthology.orchestra.run.smithi192.stdout: # disable pg_autoscaler by default for new pools 2020-11-28T21:55:43.656 INFO:teuthology.orchestra.run.smithi192.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-28T21:55:43.656 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:43.657 INFO:teuthology.orchestra.run.smithi192.stdout: osd pool default size = 2 2020-11-28T21:55:43.657 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:43.657 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd allow primary affinity = true 2020-11-28T21:55:43.657 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd allow pg remap = true 2020-11-28T21:55:43.657 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on legacy crush tunables = false 2020-11-28T21:55:43.658 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on crush straw calc version zero = false 2020-11-28T21:55:43.658 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on no sortbitwise = false 2020-11-28T21:55:43.658 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on osd down out interval zero = false 2020-11-28T21:55:43.658 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on too few osds = false 2020-11-28T21:55:43.658 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-28T21:55:43.659 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-28T21:55:43.659 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:43.659 INFO:teuthology.orchestra.run.smithi192.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-28T21:55:43.659 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:43.659 INFO:teuthology.orchestra.run.smithi192.stdout: osd default data pool replay window = 5 2020-11-28T21:55:43.660 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:43.660 INFO:teuthology.orchestra.run.smithi192.stdout: mon allow pool delete = true 2020-11-28T21:55:43.660 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:43.660 INFO:teuthology.orchestra.run.smithi192.stdout: mon cluster log file level = debug 2020-11-28T21:55:43.660 INFO:teuthology.orchestra.run.smithi192.stdout: debug asserts on shutdown = true 2020-11-28T21:55:43.661 INFO:teuthology.orchestra.run.smithi192.stdout: mon host = "172.21.15.162,172.21.15.118,[v2:172.21.15.118:3301,v1:172.21.15.118:6790]" 2020-11-28T21:55:43.661 INFO:teuthology.orchestra.run.smithi192.stdout: fsid = 7c321bcf-cd33-40f7-a39f-da5470b386cf 2020-11-28T21:55:43.661 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:43.661 INFO:teuthology.orchestra.run.smithi192.stdout:[osd] 2020-11-28T21:55:43.661 INFO:teuthology.orchestra.run.smithi192.stdout: osd journal size = 100 2020-11-28T21:55:43.662 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:43.662 INFO:teuthology.orchestra.run.smithi192.stdout: osd scrub load threshold = 5.0 2020-11-28T21:55:43.662 INFO:teuthology.orchestra.run.smithi192.stdout: osd scrub max interval = 600 2020-11-28T21:55:43.662 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:43.663 INFO:teuthology.orchestra.run.smithi192.stdout: osd recover clone overlap = true 2020-11-28T21:55:43.663 INFO:teuthology.orchestra.run.smithi192.stdout: osd recovery max chunk = 1048576 2020-11-28T21:55:43.663 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:43.663 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug shutdown = true 2020-11-28T21:55:43.663 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug op order = true 2020-11-28T21:55:43.664 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug verify stray on activate = true 2020-11-28T21:55:43.664 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:43.664 INFO:teuthology.orchestra.run.smithi192.stdout: osd open classes on start = true 2020-11-28T21:55:43.664 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug pg log writeout = true 2020-11-28T21:55:43.664 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:43.665 INFO:teuthology.orchestra.run.smithi192.stdout: osd deep scrub update digest min age = 30 2020-11-28T21:55:43.665 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:43.665 INFO:teuthology.orchestra.run.smithi192.stdout: osd map max advance = 10 2020-11-28T21:55:43.665 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:43.665 INFO:teuthology.orchestra.run.smithi192.stdout: journal zero on create = true 2020-11-28T21:55:43.666 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:43.666 INFO:teuthology.orchestra.run.smithi192.stdout: filestore ondisk finisher threads = 3 2020-11-28T21:55:43.666 INFO:teuthology.orchestra.run.smithi192.stdout: filestore apply finisher threads = 3 2020-11-28T21:55:43.666 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:43.666 INFO:teuthology.orchestra.run.smithi192.stdout: bdev debug aio = true 2020-11-28T21:55:43.667 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug misdirected ops = true 2020-11-28T21:55:43.667 INFO:teuthology.orchestra.run.smithi192.stdout: bluestore block size = 96636764160 2020-11-28T21:55:43.667 INFO:teuthology.orchestra.run.smithi192.stdout: bluestore compression mode = aggressive 2020-11-28T21:55:43.667 INFO:teuthology.orchestra.run.smithi192.stdout: bluestore fsck on mount = True 2020-11-28T21:55:43.667 INFO:teuthology.orchestra.run.smithi192.stdout: debug bluefs = 20 2020-11-28T21:55:43.668 INFO:teuthology.orchestra.run.smithi192.stdout: debug bluestore = 20 2020-11-28T21:55:43.668 INFO:teuthology.orchestra.run.smithi192.stdout: debug filestore = 20 2020-11-28T21:55:43.668 INFO:teuthology.orchestra.run.smithi192.stdout: debug journal = 20 2020-11-28T21:55:43.668 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 20 2020-11-28T21:55:43.668 INFO:teuthology.orchestra.run.smithi192.stdout: debug osd = 25 2020-11-28T21:55:43.669 INFO:teuthology.orchestra.run.smithi192.stdout: debug rocksdb = 10 2020-11-28T21:55:43.669 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd backfillfull_ratio = 0.85 2020-11-28T21:55:43.669 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd full ratio = 0.9 2020-11-28T21:55:43.669 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd nearfull ratio = 0.8 2020-11-28T21:55:43.669 INFO:teuthology.orchestra.run.smithi192.stdout: osd failsafe full ratio = 0.95 2020-11-28T21:55:43.670 INFO:teuthology.orchestra.run.smithi192.stdout: osd objectstore = bluestore 2020-11-28T21:55:43.670 INFO:teuthology.orchestra.run.smithi192.stdout: osd op complaint time = 180 2020-11-28T21:55:43.670 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:43.670 INFO:teuthology.orchestra.run.smithi192.stdout:[mgr] 2020-11-28T21:55:43.670 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2020-11-28T21:55:43.671 INFO:teuthology.orchestra.run.smithi192.stdout: debug mgr = 20 2020-11-28T21:55:43.671 INFO:teuthology.orchestra.run.smithi192.stdout: debug mon = 20 2020-11-28T21:55:43.671 INFO:teuthology.orchestra.run.smithi192.stdout: debug auth = 20 2020-11-28T21:55:43.671 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min pgs per osd = 4 2020-11-28T21:55:43.671 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min bytes per osd = 10 2020-11-28T21:55:43.672 INFO:teuthology.orchestra.run.smithi192.stdout: mgr/telemetry/nag = false 2020-11-28T21:55:43.672 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:43.672 INFO:teuthology.orchestra.run.smithi192.stdout:[mon] 2020-11-28T21:55:43.672 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2020-11-28T21:55:43.672 INFO:teuthology.orchestra.run.smithi192.stdout: debug mon = 20 2020-11-28T21:55:43.673 INFO:teuthology.orchestra.run.smithi192.stdout: debug paxos = 20 2020-11-28T21:55:43.673 INFO:teuthology.orchestra.run.smithi192.stdout: debug auth = 20 2020-11-28T21:55:43.673 INFO:teuthology.orchestra.run.smithi192.stdout: mon data avail warn = 5 2020-11-28T21:55:43.673 INFO:teuthology.orchestra.run.smithi192.stdout: mon mgr mkfs grace = 240 2020-11-28T21:55:43.673 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min pgs per osd = 4 2020-11-28T21:55:43.674 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd reporter subtree level = osd 2020-11-28T21:55:43.674 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd prime pg temp = true 2020-11-28T21:55:43.674 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min bytes per osd = 10 2020-11-28T21:55:43.674 INFO:teuthology.orchestra.run.smithi192.stdout: mon op complaint time = 120 2020-11-28T21:55:43.674 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:43.675 INFO:teuthology.orchestra.run.smithi192.stdout:[client] 2020-11-28T21:55:43.675 INFO:teuthology.orchestra.run.smithi192.stdout: rgw cache enabled = true 2020-11-28T21:55:43.675 INFO:teuthology.orchestra.run.smithi192.stdout: rgw enable ops log = true 2020-11-28T21:55:43.675 INFO:teuthology.orchestra.run.smithi192.stdout: rgw enable usage log = true 2020-11-28T21:55:43.675 INFO:teuthology.orchestra.run.smithi192.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-28T21:55:43.676 INFO:teuthology.orchestra.run.smithi192.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-28T21:55:43.676 INFO:teuthology.orchestra.run.smithi192.stdout: client mount timeout = 600 2020-11-28T21:55:43.676 INFO:teuthology.orchestra.run.smithi192.stdout: debug client = 20 2020-11-28T21:55:43.676 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2020-11-28T21:55:43.676 INFO:teuthology.orchestra.run.smithi192.stdout: rados mon op timeout = 15m 2020-11-28T21:55:43.676 INFO:teuthology.orchestra.run.smithi192.stdout: rados osd op timeout = 15m 2020-11-28T21:55:43.677 INFO:teuthology.orchestra.run.smithi192.stdout: client debug getattr caps = true 2020-11-28T21:55:43.677 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.a] 2020-11-28T21:55:43.677 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.b] 2020-11-28T21:55:43.677 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.c] 2020-11-28T21:55:43.677 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.a] 2020-11-28T21:55:43.678 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.c] 2020-11-28T21:55:43.678 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.b] 2020-11-28T21:55:43.678 INFO:teuthology.orchestra.run.smithi192.stdout:[mds] 2020-11-28T21:55:43.678 INFO:teuthology.orchestra.run.smithi192.stdout: debug mds = 20 2020-11-28T21:55:43.679 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2020-11-28T21:55:43.679 INFO:teuthology.orchestra.run.smithi192.stdout: mds bal frag = True 2020-11-28T21:55:43.679 INFO:teuthology.orchestra.run.smithi192.stdout: mds bal fragment size max = 10000 2020-11-28T21:55:43.679 INFO:teuthology.orchestra.run.smithi192.stdout: mds bal merge size = 5 2020-11-28T21:55:43.679 INFO:teuthology.orchestra.run.smithi192.stdout: mds bal split bits = 3 2020-11-28T21:55:43.680 INFO:teuthology.orchestra.run.smithi192.stdout: mds bal split size = 100 2020-11-28T21:55:43.680 INFO:teuthology.orchestra.run.smithi192.stdout: mds debug frag = True 2020-11-28T21:55:43.680 INFO:teuthology.orchestra.run.smithi192.stdout: mds debug scatterstat = True 2020-11-28T21:55:43.680 INFO:teuthology.orchestra.run.smithi192.stdout: mds op complaint time = 180 2020-11-28T21:55:43.680 INFO:teuthology.orchestra.run.smithi192.stdout: mds verify scatter = True 2020-11-28T21:55:43.681 INFO:teuthology.orchestra.run.smithi192.stdout: osd op complaint time = 180 2020-11-28T21:55:43.681 INFO:teuthology.orchestra.run.smithi192.stdout: rados mon op timeout = 15m 2020-11-28T21:55:43.681 INFO:teuthology.orchestra.run.smithi192.stdout: rados osd op timeout = 15m 2020-11-28T21:55:43.681 INFO:teuthology.orchestra.run.smithi118.stdout:[global] 2020-11-28T21:55:43.681 INFO:teuthology.orchestra.run.smithi118.stdout: chdir = "" 2020-11-28T21:55:43.682 INFO:teuthology.orchestra.run.smithi118.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-28T21:55:43.682 INFO:teuthology.orchestra.run.smithi118.stdout: auth supported = cephx 2020-11-28T21:55:43.682 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:43.682 INFO:teuthology.orchestra.run.smithi118.stdout: filestore xattr use omap = true 2020-11-28T21:55:43.682 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:43.683 INFO:teuthology.orchestra.run.smithi118.stdout: mon clock drift allowed = 1.000 2020-11-28T21:55:43.683 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:43.683 INFO:teuthology.orchestra.run.smithi118.stdout: osd crush chooseleaf type = 0 2020-11-28T21:55:43.683 INFO:teuthology.orchestra.run.smithi118.stdout: auth debug = true 2020-11-28T21:55:43.683 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:43.684 INFO:teuthology.orchestra.run.smithi118.stdout: ms die on old message = true 2020-11-28T21:55:43.684 INFO:teuthology.orchestra.run.smithi118.stdout: ms die on bug = true 2020-11-28T21:55:43.684 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:43.684 INFO:teuthology.orchestra.run.smithi118.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-28T21:55:43.684 INFO:teuthology.orchestra.run.smithi118.stdout: mon pg warn max object skew = 0 2020-11-28T21:55:43.685 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:43.685 INFO:teuthology.orchestra.run.smithi118.stdout: # disable pg_autoscaler by default for new pools 2020-11-28T21:55:43.685 INFO:teuthology.orchestra.run.smithi118.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-28T21:55:43.685 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:43.685 INFO:teuthology.orchestra.run.smithi118.stdout: osd pool default size = 2 2020-11-28T21:55:43.686 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:43.686 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd allow primary affinity = true 2020-11-28T21:55:43.686 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd allow pg remap = true 2020-11-28T21:55:43.686 INFO:teuthology.orchestra.run.smithi118.stdout: mon warn on legacy crush tunables = false 2020-11-28T21:55:43.686 INFO:teuthology.orchestra.run.smithi118.stdout: mon warn on crush straw calc version zero = false 2020-11-28T21:55:43.687 INFO:teuthology.orchestra.run.smithi118.stdout: mon warn on no sortbitwise = false 2020-11-28T21:55:43.687 INFO:teuthology.orchestra.run.smithi118.stdout: mon warn on osd down out interval zero = false 2020-11-28T21:55:43.687 INFO:teuthology.orchestra.run.smithi118.stdout: mon warn on too few osds = false 2020-11-28T21:55:43.687 INFO:teuthology.orchestra.run.smithi118.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-28T21:55:43.687 INFO:teuthology.orchestra.run.smithi118.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-28T21:55:43.688 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:43.688 INFO:teuthology.orchestra.run.smithi118.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-28T21:55:43.688 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:43.688 INFO:teuthology.orchestra.run.smithi118.stdout: osd default data pool replay window = 5 2020-11-28T21:55:43.688 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:43.689 INFO:teuthology.orchestra.run.smithi118.stdout: mon allow pool delete = true 2020-11-28T21:55:43.689 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:43.689 INFO:teuthology.orchestra.run.smithi118.stdout: mon cluster log file level = debug 2020-11-28T21:55:43.689 INFO:teuthology.orchestra.run.smithi118.stdout: debug asserts on shutdown = true 2020-11-28T21:55:43.689 INFO:teuthology.orchestra.run.smithi118.stdout: mon host = "172.21.15.162,172.21.15.118,[v2:172.21.15.118:3301,v1:172.21.15.118:6790]" 2020-11-28T21:55:43.690 INFO:teuthology.orchestra.run.smithi118.stdout: fsid = 7c321bcf-cd33-40f7-a39f-da5470b386cf 2020-11-28T21:55:43.690 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:43.690 INFO:teuthology.orchestra.run.smithi118.stdout:[osd] 2020-11-28T21:55:43.690 INFO:teuthology.orchestra.run.smithi118.stdout: osd journal size = 100 2020-11-28T21:55:43.690 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:43.691 INFO:teuthology.orchestra.run.smithi118.stdout: osd scrub load threshold = 5.0 2020-11-28T21:55:43.691 INFO:teuthology.orchestra.run.smithi118.stdout: osd scrub max interval = 600 2020-11-28T21:55:43.691 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:43.691 INFO:teuthology.orchestra.run.smithi118.stdout: osd recover clone overlap = true 2020-11-28T21:55:43.691 INFO:teuthology.orchestra.run.smithi118.stdout: osd recovery max chunk = 1048576 2020-11-28T21:55:43.692 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:43.692 INFO:teuthology.orchestra.run.smithi118.stdout: osd debug shutdown = true 2020-11-28T21:55:43.692 INFO:teuthology.orchestra.run.smithi118.stdout: osd debug op order = true 2020-11-28T21:55:43.692 INFO:teuthology.orchestra.run.smithi118.stdout: osd debug verify stray on activate = true 2020-11-28T21:55:43.692 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:43.693 INFO:teuthology.orchestra.run.smithi118.stdout: osd open classes on start = true 2020-11-28T21:55:43.693 INFO:teuthology.orchestra.run.smithi118.stdout: osd debug pg log writeout = true 2020-11-28T21:55:43.693 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:43.693 INFO:teuthology.orchestra.run.smithi118.stdout: osd deep scrub update digest min age = 30 2020-11-28T21:55:43.693 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:43.694 INFO:teuthology.orchestra.run.smithi118.stdout: osd map max advance = 10 2020-11-28T21:55:43.694 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:43.694 INFO:teuthology.orchestra.run.smithi118.stdout: journal zero on create = true 2020-11-28T21:55:43.694 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:43.694 INFO:teuthology.orchestra.run.smithi118.stdout: filestore ondisk finisher threads = 3 2020-11-28T21:55:43.695 INFO:teuthology.orchestra.run.smithi118.stdout: filestore apply finisher threads = 3 2020-11-28T21:55:43.695 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:43.695 INFO:teuthology.orchestra.run.smithi118.stdout: bdev debug aio = true 2020-11-28T21:55:43.695 INFO:teuthology.orchestra.run.smithi118.stdout: osd debug misdirected ops = true 2020-11-28T21:55:43.695 INFO:teuthology.orchestra.run.smithi118.stdout: bluestore block size = 96636764160 2020-11-28T21:55:43.696 INFO:teuthology.orchestra.run.smithi118.stdout: bluestore compression mode = aggressive 2020-11-28T21:55:43.696 INFO:teuthology.orchestra.run.smithi118.stdout: bluestore fsck on mount = True 2020-11-28T21:55:43.696 INFO:teuthology.orchestra.run.smithi118.stdout: debug bluefs = 20 2020-11-28T21:55:43.696 INFO:teuthology.orchestra.run.smithi118.stdout: debug bluestore = 20 2020-11-28T21:55:43.696 INFO:teuthology.orchestra.run.smithi118.stdout: debug filestore = 20 2020-11-28T21:55:43.697 INFO:teuthology.orchestra.run.smithi118.stdout: debug journal = 20 2020-11-28T21:55:43.697 INFO:teuthology.orchestra.run.smithi118.stdout: debug ms = 20 2020-11-28T21:55:43.697 INFO:teuthology.orchestra.run.smithi118.stdout: debug osd = 25 2020-11-28T21:55:43.697 INFO:teuthology.orchestra.run.smithi118.stdout: debug rocksdb = 10 2020-11-28T21:55:43.697 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd backfillfull_ratio = 0.85 2020-11-28T21:55:43.698 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd full ratio = 0.9 2020-11-28T21:55:43.698 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd nearfull ratio = 0.8 2020-11-28T21:55:43.698 INFO:teuthology.orchestra.run.smithi118.stdout: osd failsafe full ratio = 0.95 2020-11-28T21:55:43.698 INFO:teuthology.orchestra.run.smithi118.stdout: osd objectstore = bluestore 2020-11-28T21:55:43.698 INFO:teuthology.orchestra.run.smithi118.stdout: osd op complaint time = 180 2020-11-28T21:55:43.699 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:43.699 INFO:teuthology.orchestra.run.smithi118.stdout:[mgr] 2020-11-28T21:55:43.699 INFO:teuthology.orchestra.run.smithi118.stdout: debug ms = 1 2020-11-28T21:55:43.699 INFO:teuthology.orchestra.run.smithi118.stdout: debug mgr = 20 2020-11-28T21:55:43.699 INFO:teuthology.orchestra.run.smithi118.stdout: debug mon = 20 2020-11-28T21:55:43.700 INFO:teuthology.orchestra.run.smithi118.stdout: debug auth = 20 2020-11-28T21:55:43.700 INFO:teuthology.orchestra.run.smithi118.stdout: mon reweight min pgs per osd = 4 2020-11-28T21:55:43.700 INFO:teuthology.orchestra.run.smithi118.stdout: mon reweight min bytes per osd = 10 2020-11-28T21:55:43.700 INFO:teuthology.orchestra.run.smithi118.stdout: mgr/telemetry/nag = false 2020-11-28T21:55:43.700 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:43.700 INFO:teuthology.orchestra.run.smithi118.stdout:[mon] 2020-11-28T21:55:43.701 INFO:teuthology.orchestra.run.smithi118.stdout: debug ms = 1 2020-11-28T21:55:43.701 INFO:teuthology.orchestra.run.smithi118.stdout: debug mon = 20 2020-11-28T21:55:43.701 INFO:teuthology.orchestra.run.smithi118.stdout: debug paxos = 20 2020-11-28T21:55:43.701 INFO:teuthology.orchestra.run.smithi118.stdout: debug auth = 20 2020-11-28T21:55:43.701 INFO:teuthology.orchestra.run.smithi118.stdout: mon data avail warn = 5 2020-11-28T21:55:43.702 INFO:teuthology.orchestra.run.smithi118.stdout: mon mgr mkfs grace = 240 2020-11-28T21:55:43.702 INFO:teuthology.orchestra.run.smithi118.stdout: mon reweight min pgs per osd = 4 2020-11-28T21:55:43.702 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd reporter subtree level = osd 2020-11-28T21:55:43.702 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd prime pg temp = true 2020-11-28T21:55:43.702 INFO:teuthology.orchestra.run.smithi118.stdout: mon reweight min bytes per osd = 10 2020-11-28T21:55:43.703 INFO:teuthology.orchestra.run.smithi118.stdout: mon op complaint time = 120 2020-11-28T21:55:43.703 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:43.703 INFO:teuthology.orchestra.run.smithi118.stdout:[client] 2020-11-28T21:55:43.703 INFO:teuthology.orchestra.run.smithi118.stdout: rgw cache enabled = true 2020-11-28T21:55:43.703 INFO:teuthology.orchestra.run.smithi118.stdout: rgw enable ops log = true 2020-11-28T21:55:43.704 INFO:teuthology.orchestra.run.smithi118.stdout: rgw enable usage log = true 2020-11-28T21:55:43.704 INFO:teuthology.orchestra.run.smithi118.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-28T21:55:43.704 INFO:teuthology.orchestra.run.smithi118.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-28T21:55:43.704 INFO:teuthology.orchestra.run.smithi118.stdout: client mount timeout = 600 2020-11-28T21:55:43.704 INFO:teuthology.orchestra.run.smithi118.stdout: debug client = 20 2020-11-28T21:55:43.705 INFO:teuthology.orchestra.run.smithi118.stdout: debug ms = 1 2020-11-28T21:55:43.705 INFO:teuthology.orchestra.run.smithi118.stdout: rados mon op timeout = 15m 2020-11-28T21:55:43.705 INFO:teuthology.orchestra.run.smithi118.stdout: rados osd op timeout = 15m 2020-11-28T21:55:43.705 INFO:teuthology.orchestra.run.smithi118.stdout: client debug getattr caps = true 2020-11-28T21:55:43.705 INFO:teuthology.orchestra.run.smithi118.stdout:[mon.a] 2020-11-28T21:55:43.706 INFO:teuthology.orchestra.run.smithi118.stdout:[mon.b] 2020-11-28T21:55:43.706 INFO:teuthology.orchestra.run.smithi118.stdout:[mon.c] 2020-11-28T21:55:43.706 INFO:teuthology.orchestra.run.smithi118.stdout:[mds.a] 2020-11-28T21:55:43.706 INFO:teuthology.orchestra.run.smithi118.stdout:[mds.c] 2020-11-28T21:55:43.706 INFO:teuthology.orchestra.run.smithi118.stdout:[mds.b] 2020-11-28T21:55:43.707 INFO:teuthology.orchestra.run.smithi118.stdout:[mds] 2020-11-28T21:55:43.707 INFO:teuthology.orchestra.run.smithi118.stdout: debug mds = 20 2020-11-28T21:55:43.707 INFO:teuthology.orchestra.run.smithi118.stdout: debug ms = 1 2020-11-28T21:55:43.707 INFO:teuthology.orchestra.run.smithi118.stdout: mds bal frag = True 2020-11-28T21:55:43.707 INFO:teuthology.orchestra.run.smithi118.stdout: mds bal fragment size max = 10000 2020-11-28T21:55:43.708 INFO:teuthology.orchestra.run.smithi118.stdout: mds bal merge size = 5 2020-11-28T21:55:43.708 INFO:teuthology.orchestra.run.smithi118.stdout: mds bal split bits = 3 2020-11-28T21:55:43.708 INFO:teuthology.orchestra.run.smithi118.stdout: mds bal split size = 100 2020-11-28T21:55:43.708 INFO:teuthology.orchestra.run.smithi118.stdout: mds debug frag = True 2020-11-28T21:55:43.708 INFO:teuthology.orchestra.run.smithi118.stdout: mds debug scatterstat = True 2020-11-28T21:55:43.709 INFO:teuthology.orchestra.run.smithi118.stdout: mds op complaint time = 180 2020-11-28T21:55:43.709 INFO:teuthology.orchestra.run.smithi118.stdout: mds verify scatter = True 2020-11-28T21:55:43.709 INFO:teuthology.orchestra.run.smithi118.stdout: osd op complaint time = 180 2020-11-28T21:55:43.709 INFO:teuthology.orchestra.run.smithi118.stdout: rados mon op timeout = 15m 2020-11-28T21:55:43.709 INFO:teuthology.orchestra.run.smithi118.stdout: rados osd op timeout = 15m 2020-11-28T21:55:43.710 INFO:teuthology.orchestra.run.smithi162.stdout:[global] 2020-11-28T21:55:43.710 INFO:teuthology.orchestra.run.smithi162.stdout: chdir = "" 2020-11-28T21:55:43.710 INFO:teuthology.orchestra.run.smithi162.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-28T21:55:43.710 INFO:teuthology.orchestra.run.smithi162.stdout: auth supported = cephx 2020-11-28T21:55:43.710 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:43.711 INFO:teuthology.orchestra.run.smithi162.stdout: filestore xattr use omap = true 2020-11-28T21:55:43.711 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:43.711 INFO:teuthology.orchestra.run.smithi162.stdout: mon clock drift allowed = 1.000 2020-11-28T21:55:43.711 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:43.711 INFO:teuthology.orchestra.run.smithi162.stdout: osd crush chooseleaf type = 0 2020-11-28T21:55:43.712 INFO:teuthology.orchestra.run.smithi162.stdout: auth debug = true 2020-11-28T21:55:43.712 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:43.712 INFO:teuthology.orchestra.run.smithi162.stdout: ms die on old message = true 2020-11-28T21:55:43.712 INFO:teuthology.orchestra.run.smithi162.stdout: ms die on bug = true 2020-11-28T21:55:43.712 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:43.713 INFO:teuthology.orchestra.run.smithi162.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-28T21:55:43.713 INFO:teuthology.orchestra.run.smithi162.stdout: mon pg warn max object skew = 0 2020-11-28T21:55:43.713 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:43.713 INFO:teuthology.orchestra.run.smithi162.stdout: # disable pg_autoscaler by default for new pools 2020-11-28T21:55:43.713 INFO:teuthology.orchestra.run.smithi162.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-28T21:55:43.714 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:43.714 INFO:teuthology.orchestra.run.smithi162.stdout: osd pool default size = 2 2020-11-28T21:55:43.714 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:43.714 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd allow primary affinity = true 2020-11-28T21:55:43.714 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd allow pg remap = true 2020-11-28T21:55:43.715 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on legacy crush tunables = false 2020-11-28T21:55:43.715 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on crush straw calc version zero = false 2020-11-28T21:55:43.715 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on no sortbitwise = false 2020-11-28T21:55:43.715 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on osd down out interval zero = false 2020-11-28T21:55:43.715 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on too few osds = false 2020-11-28T21:55:43.716 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-28T21:55:43.716 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-28T21:55:43.716 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:43.716 INFO:teuthology.orchestra.run.smithi162.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-28T21:55:43.716 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:43.717 INFO:teuthology.orchestra.run.smithi162.stdout: osd default data pool replay window = 5 2020-11-28T21:55:43.717 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:43.717 INFO:teuthology.orchestra.run.smithi162.stdout: mon allow pool delete = true 2020-11-28T21:55:43.717 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:43.717 INFO:teuthology.orchestra.run.smithi162.stdout: mon cluster log file level = debug 2020-11-28T21:55:43.717 INFO:teuthology.orchestra.run.smithi162.stdout: debug asserts on shutdown = true 2020-11-28T21:55:43.718 INFO:teuthology.orchestra.run.smithi162.stdout: mon host = "172.21.15.162,172.21.15.118,[v2:172.21.15.118:3301,v1:172.21.15.118:6790]" 2020-11-28T21:55:43.718 INFO:teuthology.orchestra.run.smithi162.stdout: fsid = 7c321bcf-cd33-40f7-a39f-da5470b386cf 2020-11-28T21:55:43.718 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:43.718 INFO:teuthology.orchestra.run.smithi162.stdout:[osd] 2020-11-28T21:55:43.718 INFO:teuthology.orchestra.run.smithi162.stdout: osd journal size = 100 2020-11-28T21:55:43.719 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:43.719 INFO:teuthology.orchestra.run.smithi162.stdout: osd scrub load threshold = 5.0 2020-11-28T21:55:43.719 INFO:teuthology.orchestra.run.smithi162.stdout: osd scrub max interval = 600 2020-11-28T21:55:43.719 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:43.719 INFO:teuthology.orchestra.run.smithi162.stdout: osd recover clone overlap = true 2020-11-28T21:55:43.720 INFO:teuthology.orchestra.run.smithi162.stdout: osd recovery max chunk = 1048576 2020-11-28T21:55:43.720 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:43.720 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug shutdown = true 2020-11-28T21:55:43.720 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug op order = true 2020-11-28T21:55:43.720 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug verify stray on activate = true 2020-11-28T21:55:43.721 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:43.721 INFO:teuthology.orchestra.run.smithi162.stdout: osd open classes on start = true 2020-11-28T21:55:43.721 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug pg log writeout = true 2020-11-28T21:55:43.721 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:43.721 INFO:teuthology.orchestra.run.smithi162.stdout: osd deep scrub update digest min age = 30 2020-11-28T21:55:43.722 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:43.722 INFO:teuthology.orchestra.run.smithi162.stdout: osd map max advance = 10 2020-11-28T21:55:43.722 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:43.722 INFO:teuthology.orchestra.run.smithi162.stdout: journal zero on create = true 2020-11-28T21:55:43.722 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:43.723 INFO:teuthology.orchestra.run.smithi162.stdout: filestore ondisk finisher threads = 3 2020-11-28T21:55:43.723 INFO:teuthology.orchestra.run.smithi162.stdout: filestore apply finisher threads = 3 2020-11-28T21:55:43.723 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:43.723 INFO:teuthology.orchestra.run.smithi162.stdout: bdev debug aio = true 2020-11-28T21:55:43.723 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug misdirected ops = true 2020-11-28T21:55:43.724 INFO:teuthology.orchestra.run.smithi162.stdout: bluestore block size = 96636764160 2020-11-28T21:55:43.724 INFO:teuthology.orchestra.run.smithi162.stdout: bluestore compression mode = aggressive 2020-11-28T21:55:43.724 INFO:teuthology.orchestra.run.smithi162.stdout: bluestore fsck on mount = True 2020-11-28T21:55:43.724 INFO:teuthology.orchestra.run.smithi162.stdout: debug bluefs = 20 2020-11-28T21:55:43.724 INFO:teuthology.orchestra.run.smithi162.stdout: debug bluestore = 20 2020-11-28T21:55:43.725 INFO:teuthology.orchestra.run.smithi162.stdout: debug filestore = 20 2020-11-28T21:55:43.725 INFO:teuthology.orchestra.run.smithi162.stdout: debug journal = 20 2020-11-28T21:55:43.725 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 20 2020-11-28T21:55:43.725 INFO:teuthology.orchestra.run.smithi162.stdout: debug osd = 25 2020-11-28T21:55:43.725 INFO:teuthology.orchestra.run.smithi162.stdout: debug rocksdb = 10 2020-11-28T21:55:43.726 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd backfillfull_ratio = 0.85 2020-11-28T21:55:43.726 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd full ratio = 0.9 2020-11-28T21:55:43.726 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd nearfull ratio = 0.8 2020-11-28T21:55:43.726 INFO:teuthology.orchestra.run.smithi162.stdout: osd failsafe full ratio = 0.95 2020-11-28T21:55:43.726 INFO:teuthology.orchestra.run.smithi162.stdout: osd objectstore = bluestore 2020-11-28T21:55:43.727 INFO:teuthology.orchestra.run.smithi162.stdout: osd op complaint time = 180 2020-11-28T21:55:43.727 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:43.727 INFO:teuthology.orchestra.run.smithi162.stdout:[mgr] 2020-11-28T21:55:43.727 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2020-11-28T21:55:43.727 INFO:teuthology.orchestra.run.smithi162.stdout: debug mgr = 20 2020-11-28T21:55:43.728 INFO:teuthology.orchestra.run.smithi162.stdout: debug mon = 20 2020-11-28T21:55:43.728 INFO:teuthology.orchestra.run.smithi162.stdout: debug auth = 20 2020-11-28T21:55:43.728 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min pgs per osd = 4 2020-11-28T21:55:43.728 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min bytes per osd = 10 2020-11-28T21:55:43.728 INFO:teuthology.orchestra.run.smithi162.stdout: mgr/telemetry/nag = false 2020-11-28T21:55:43.729 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:43.729 INFO:teuthology.orchestra.run.smithi162.stdout:[mon] 2020-11-28T21:55:43.729 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2020-11-28T21:55:43.729 INFO:teuthology.orchestra.run.smithi162.stdout: debug mon = 20 2020-11-28T21:55:43.729 INFO:teuthology.orchestra.run.smithi162.stdout: debug paxos = 20 2020-11-28T21:55:43.730 INFO:teuthology.orchestra.run.smithi162.stdout: debug auth = 20 2020-11-28T21:55:43.730 INFO:teuthology.orchestra.run.smithi162.stdout: mon data avail warn = 5 2020-11-28T21:55:43.730 INFO:teuthology.orchestra.run.smithi162.stdout: mon mgr mkfs grace = 240 2020-11-28T21:55:43.730 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min pgs per osd = 4 2020-11-28T21:55:43.730 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd reporter subtree level = osd 2020-11-28T21:55:43.731 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd prime pg temp = true 2020-11-28T21:55:43.731 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min bytes per osd = 10 2020-11-28T21:55:43.731 INFO:teuthology.orchestra.run.smithi162.stdout: mon op complaint time = 120 2020-11-28T21:55:43.731 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:43.731 INFO:teuthology.orchestra.run.smithi162.stdout:[client] 2020-11-28T21:55:43.732 INFO:teuthology.orchestra.run.smithi162.stdout: rgw cache enabled = true 2020-11-28T21:55:43.732 INFO:teuthology.orchestra.run.smithi162.stdout: rgw enable ops log = true 2020-11-28T21:55:43.732 INFO:teuthology.orchestra.run.smithi162.stdout: rgw enable usage log = true 2020-11-28T21:55:43.732 INFO:teuthology.orchestra.run.smithi162.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-28T21:55:43.732 INFO:teuthology.orchestra.run.smithi162.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-28T21:55:43.733 INFO:teuthology.orchestra.run.smithi162.stdout: client mount timeout = 600 2020-11-28T21:55:43.733 INFO:teuthology.orchestra.run.smithi162.stdout: debug client = 20 2020-11-28T21:55:43.733 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2020-11-28T21:55:43.733 INFO:teuthology.orchestra.run.smithi162.stdout: rados mon op timeout = 15m 2020-11-28T21:55:43.733 INFO:teuthology.orchestra.run.smithi162.stdout: rados osd op timeout = 15m 2020-11-28T21:55:43.733 INFO:teuthology.orchestra.run.smithi162.stdout: client debug getattr caps = true 2020-11-28T21:55:43.734 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.a] 2020-11-28T21:55:43.734 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.b] 2020-11-28T21:55:43.734 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.c] 2020-11-28T21:55:43.734 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.a] 2020-11-28T21:55:43.734 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.c] 2020-11-28T21:55:43.735 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.b] 2020-11-28T21:55:43.735 INFO:teuthology.orchestra.run.smithi162.stdout:[mds] 2020-11-28T21:55:43.735 INFO:teuthology.orchestra.run.smithi162.stdout: debug mds = 20 2020-11-28T21:55:43.735 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2020-11-28T21:55:43.735 INFO:teuthology.orchestra.run.smithi162.stdout: mds bal frag = True 2020-11-28T21:55:43.736 INFO:teuthology.orchestra.run.smithi162.stdout: mds bal fragment size max = 10000 2020-11-28T21:55:43.736 INFO:teuthology.orchestra.run.smithi162.stdout: mds bal merge size = 5 2020-11-28T21:55:43.736 INFO:teuthology.orchestra.run.smithi162.stdout: mds bal split bits = 3 2020-11-28T21:55:43.736 INFO:teuthology.orchestra.run.smithi162.stdout: mds bal split size = 100 2020-11-28T21:55:43.736 INFO:teuthology.orchestra.run.smithi162.stdout: mds debug frag = True 2020-11-28T21:55:43.737 INFO:teuthology.orchestra.run.smithi162.stdout: mds debug scatterstat = True 2020-11-28T21:55:43.737 INFO:teuthology.orchestra.run.smithi162.stdout: mds op complaint time = 180 2020-11-28T21:55:43.737 INFO:teuthology.orchestra.run.smithi162.stdout: mds verify scatter = True 2020-11-28T21:55:43.737 INFO:teuthology.orchestra.run.smithi162.stdout: osd op complaint time = 180 2020-11-28T21:55:43.737 INFO:teuthology.orchestra.run.smithi162.stdout: rados mon op timeout = 15m 2020-11-28T21:55:43.738 INFO:teuthology.orchestra.run.smithi162.stdout: rados osd op timeout = 15m 2020-11-28T21:55:43.738 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:55:43.855 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.851+0000 7f903e186700 1 Processor -- start 2020-11-28T21:55:43.856 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.851+0000 7f903e186700 1 -- start start 2020-11-28T21:55:43.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.851+0000 7f903e186700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f903809a9b0 0x7f90380ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:43.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.851+0000 7f903e186700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90380b2a10 con 0x7f90380b2630 2020-11-28T21:55:43.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.851+0000 7f903e186700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90380608b0 con 0x7f90380a9030 2020-11-28T21:55:43.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.851+0000 7f903e186700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90380ad5d0 con 0x7f903809a9b0 2020-11-28T21:55:43.858 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.851+0000 7f903d985700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f90380a9030 0x7f90380ae750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:52048/0 (socket says 172.21.15.162:52048) 2020-11-28T21:55:43.858 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.851+0000 7f903d985700 1 -- 172.21.15.162:0/2956256809 learned_addr learned my addr 172.21.15.162:0/2956256809 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:43.858 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.851+0000 7f903c983700 1 --2- 172.21.15.162:0/2956256809 >> v2:172.21.15.118:3300/0 conn(0x7f903809a9b0 0x7f90380ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:43.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.855+0000 7f902bfff700 1 -- 172.21.15.162:0/2956256809 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3088634729 0 0) 0x7f90380608b0 con 0x7f90380a9030 2020-11-28T21:55:43.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.855+0000 7f902bfff700 1 -- 172.21.15.162:0/2956256809 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9020003230 con 0x7f90380a9030 2020-11-28T21:55:43.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.855+0000 7f902bfff700 1 -- 172.21.15.162:0/2956256809 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3124980938 0 0) 0x7f90380b2a10 con 0x7f90380b2630 2020-11-28T21:55:43.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.855+0000 7f902bfff700 1 -- 172.21.15.162:0/2956256809 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90380608b0 con 0x7f90380b2630 2020-11-28T21:55:43.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.855+0000 7f903c983700 1 -- 172.21.15.162:0/2956256809 >> v1:172.21.15.162:6789/0 conn(0x7f90380a9030 legacy=0x7f90380ae750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:43.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.855+0000 7f903c983700 1 -- 172.21.15.162:0/2956256809 >> v1:172.21.15.118:6789/0 conn(0x7f90380b2630 legacy=0x7f90380af450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:43.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.855+0000 7f903c983700 1 -- 172.21.15.162:0/2956256809 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9038061620 con 0x7f903809a9b0 2020-11-28T21:55:43.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.855+0000 7f903c983700 1 --2- 172.21.15.162:0/2956256809 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f903809a9b0 0x7f90380ad0e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f9034002f30 tx=0x7f9034006dc0).ready entity=mon.0 client_cookie=1875f4d56e6260f5 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:43.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.855+0000 7f902bfff700 1 -- 172.21.15.162:0/2956256809 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9034009740 con 0x7f903809a9b0 2020-11-28T21:55:43.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.855+0000 7f902bfff700 1 -- 172.21.15.162:0/2956256809 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90340098a0 con 0x7f903809a9b0 2020-11-28T21:55:43.862 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.855+0000 7f902bfff700 1 -- 172.21.15.162:0/2956256809 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9034009c00 con 0x7f903809a9b0 2020-11-28T21:55:43.862 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.855+0000 7f903e186700 1 -- 172.21.15.162:0/2956256809 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f903809a9b0 msgr2=0x7f90380ad0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:43.862 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.855+0000 7f903e186700 1 --2- 172.21.15.162:0/2956256809 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f903809a9b0 0x7f90380ad0e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f9034002f30 tx=0x7f9034006dc0).stop 2020-11-28T21:55:43.862 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.855+0000 7f903e186700 1 -- 172.21.15.162:0/2956256809 shutdown_connections 2020-11-28T21:55:43.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.855+0000 7f903e186700 1 --2- 172.21.15.162:0/2956256809 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f903809a9b0 0x7f90380ad0e0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:43.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.855+0000 7f903e186700 1 -- 172.21.15.162:0/2956256809 >> 172.21.15.162:0/2956256809 conn(0x7f903806a280 msgr2=0x7f903806a690 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:43.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.855+0000 7f903e186700 1 -- 172.21.15.162:0/2956256809 shutdown_connections 2020-11-28T21:55:43.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.855+0000 7f903e186700 1 -- 172.21.15.162:0/2956256809 wait complete. 2020-11-28T21:55:43.865 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.855+0000 7f903e186700 1 Processor -- start 2020-11-28T21:55:43.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.855+0000 7f903e186700 1 -- start start 2020-11-28T21:55:43.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.859+0000 7f903e186700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f903809a9b0 0x7f9038065320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:43.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.859+0000 7f903e186700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f90380a9030 0x7f903808fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:43.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.859+0000 7f903e186700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f90380b2630 0x7f903808ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:43.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.859+0000 7f903e186700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9038065c80 con 0x7f90380b2630 2020-11-28T21:55:43.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.859+0000 7f903e186700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9038065f20 con 0x7f903809a9b0 2020-11-28T21:55:43.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.859+0000 7f903e186700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9038146380 con 0x7f90380a9030 2020-11-28T21:55:43.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.859+0000 7f903d184700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f903809a9b0 0x7f9038065320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:43.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.859+0000 7f903d184700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f903809a9b0 0x7f9038065320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45206/0 (socket says 172.21.15.162:45206) 2020-11-28T21:55:43.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.859+0000 7f903d184700 1 -- 172.21.15.162:0/2982618216 learned_addr learned my addr 172.21.15.162:0/2982618216 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:43.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.859+0000 7f903c983700 1 --2- 172.21.15.162:0/2982618216 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f90380a9030 0x7f903808fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:43.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.859+0000 7f903d985700 1 --2- 172.21.15.162:0/2982618216 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f90380b2630 0x7f903808ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:43.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.859+0000 7f903d184700 1 -- 172.21.15.162:0/2982618216 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f90380a9030 msgr2=0x7f903808fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:43.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.859+0000 7f903d184700 1 --2- 172.21.15.162:0/2982618216 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f90380a9030 0x7f903808fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:43.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.859+0000 7f903d184700 1 -- 172.21.15.162:0/2982618216 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f90380b2630 msgr2=0x7f903808ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:43.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.859+0000 7f903d184700 1 --2- 172.21.15.162:0/2982618216 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f90380b2630 0x7f903808ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:43.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.859+0000 7f903d184700 1 -- 172.21.15.162:0/2982618216 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9038066f20 con 0x7f903809a9b0 2020-11-28T21:55:43.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.859+0000 7f903c983700 1 --2- 172.21.15.162:0/2982618216 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f90380a9030 0x7f903808fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:55:43.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.859+0000 7f903d184700 1 --2- 172.21.15.162:0/2982618216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f903809a9b0 0x7f9038065320 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 rx=0x7f902c001450 tx=0x7f902c007e30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:43.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.859+0000 7f902a7fc700 1 -- 172.21.15.162:0/2982618216 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f902c002240 con 0x7f903809a9b0 2020-11-28T21:55:43.870 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.859+0000 7f902a7fc700 1 -- 172.21.15.162:0/2982618216 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f902c0023a0 con 0x7f903809a9b0 2020-11-28T21:55:43.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.859+0000 7f903e186700 1 -- 172.21.15.162:0/2982618216 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9038146820 con 0x7f903809a9b0 2020-11-28T21:55:43.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.859+0000 7f902a7fc700 1 -- 172.21.15.162:0/2982618216 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f902c00a790 con 0x7f903809a9b0 2020-11-28T21:55:43.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.859+0000 7f903e186700 1 -- 172.21.15.162:0/2982618216 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9038153730 con 0x7f903809a9b0 2020-11-28T21:55:43.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.859+0000 7f903e186700 1 -- 172.21.15.162:0/2982618216 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9038057050 con 0x7f903809a9b0 2020-11-28T21:55:43.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.859+0000 7f902a7fc700 1 -- 172.21.15.162:0/2982618216 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f902c010070 con 0x7f903809a9b0 2020-11-28T21:55:43.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.859+0000 7f902a7fc700 1 --2- 172.21.15.162:0/2982618216 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f90180554e0 0x7f90180579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:43.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.863+0000 7f903c983700 1 --2- 172.21.15.162:0/2982618216 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f90180554e0 0x7f90180579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:43.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.863+0000 7f902a7fc700 1 -- 172.21.15.162:0/2982618216 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f902c004020 con 0x7f903809a9b0 2020-11-28T21:55:43.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.863+0000 7f903c983700 1 --2- 172.21.15.162:0/2982618216 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f90180554e0 0x7f90180579a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f9034019000 tx=0x7f9034015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:43.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:43.867+0000 7f902a7fc700 1 -- 172.21.15.162:0/2982618216 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f902c043790 con 0x7f903809a9b0 2020-11-28T21:55:44.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.007+0000 7f903e186700 1 -- 172.21.15.162:0/2982618216 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90380982a0 con 0x7f903809a9b0 2020-11-28T21:55:44.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.007+0000 7f902a7fc700 1 -- 172.21.15.162:0/2982618216 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+16148 (secure 0 0 0) 0x7f902c046db0 con 0x7f903809a9b0 2020-11-28T21:55:44.015 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:44.015 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":138,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:55:32.383252+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:55:30.002238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2020-11-28T21:55:30.389418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:55:44.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.011+0000 7f903e186700 1 -- 172.21.15.162:0/2982618216 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f90180554e0 msgr2=0x7f90180579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:44.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.011+0000 7f903e186700 1 --2- 172.21.15.162:0/2982618216 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f90180554e0 0x7f90180579a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f9034019000 tx=0x7f9034015000).stop 2020-11-28T21:55:44.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.011+0000 7f903e186700 1 -- 172.21.15.162:0/2982618216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f903809a9b0 msgr2=0x7f9038065320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:44.019 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.011+0000 7f903e186700 1 --2- 172.21.15.162:0/2982618216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f903809a9b0 0x7f9038065320 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 rx=0x7f902c001450 tx=0x7f902c007e30).stop 2020-11-28T21:55:44.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.011+0000 7f903e186700 1 -- 172.21.15.162:0/2982618216 shutdown_connections 2020-11-28T21:55:44.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.011+0000 7f903e186700 1 --2- 172.21.15.162:0/2982618216 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f90180554e0 0x7f90180579a0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:44.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.011+0000 7f903e186700 1 --2- 172.21.15.162:0/2982618216 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f90380b2630 0x7f903808ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:44.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.011+0000 7f903e186700 1 --2- 172.21.15.162:0/2982618216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f903809a9b0 0x7f9038065320 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:44.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.011+0000 7f903e186700 1 --2- 172.21.15.162:0/2982618216 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f90380a9030 0x7f903808fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:44.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.011+0000 7f903e186700 1 -- 172.21.15.162:0/2982618216 >> 172.21.15.162:0/2982618216 conn(0x7f903806a280 msgr2=0x7f90380b5fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:44.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.011+0000 7f903e186700 1 -- 172.21.15.162:0/2982618216 shutdown_connections 2020-11-28T21:55:44.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.011+0000 7f903e186700 1 -- 172.21.15.162:0/2982618216 wait complete. 2020-11-28T21:55:44.032 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-28T21:55:30.002238+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '137', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2020-11-28T21:55:30.389418+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '138', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-28T21:55:44.033 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:55:44.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.143+0000 7f3de6644700 1 Processor -- start 2020-11-28T21:55:44.150 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.147+0000 7f3de6644700 1 -- start start 2020-11-28T21:55:44.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.147+0000 7f3de6644700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f3de006a420 0x7f3de0142910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:44.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.147+0000 7f3de6644700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3de0061c60 0x7f3de00ad580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:44.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.147+0000 7f3de6644700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3de0060560 con 0x7f3de00b09a0 2020-11-28T21:55:44.153 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.147+0000 7f3de6644700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3de013ce20 con 0x7f3de006a420 2020-11-28T21:55:44.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.147+0000 7f3de6644700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3de00b0eb0 con 0x7f3de0061c60 2020-11-28T21:55:44.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.147+0000 7f3de5e43700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3de0061c60 0x7f3de00ad580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:44.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.147+0000 7f3de4e41700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f3de00b09a0 0x7f3de005fe50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:52058/0 (socket says 172.21.15.162:52058) 2020-11-28T21:55:44.154 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.147+0000 7f3de5e43700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3de0061c60 0x7f3de00ad580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45218/0 (socket says 172.21.15.162:45218) 2020-11-28T21:55:44.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.147+0000 7f3de4e41700 1 -- 172.21.15.162:0/3090160868 learned_addr learned my addr 172.21.15.162:0/3090160868 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:44.155 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.147+0000 7f3de5642700 1 --2- 172.21.15.162:0/3090160868 >> v2:172.21.15.118:3300/0 conn(0x7f3de006a420 0x7f3de0142910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:44.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.147+0000 7f3de5e43700 1 -- 172.21.15.162:0/3090160868 >> v2:172.21.15.118:3300/0 conn(0x7f3de006a420 msgr2=0x7f3de0142910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:44.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.147+0000 7f3de5e43700 1 --2- 172.21.15.162:0/3090160868 >> v2:172.21.15.118:3300/0 conn(0x7f3de006a420 0x7f3de0142910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:44.156 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.147+0000 7f3de5e43700 1 -- 172.21.15.162:0/3090160868 >> v1:172.21.15.162:6789/0 conn(0x7f3de00b09a0 legacy=0x7f3de005fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:44.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.147+0000 7f3de5e43700 1 -- 172.21.15.162:0/3090160868 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de00b1150 con 0x7f3de0061c60 2020-11-28T21:55:44.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.147+0000 7f3de5e43700 1 --2- 172.21.15.162:0/3090160868 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3de0061c60 0x7f3de00ad580 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 rx=0x7f3dd80032b0 tx=0x7f3dd80091a0).ready entity=mon.1 client_cookie=9af14974425843a7 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:44.157 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.147+0000 7f3dd7fff700 1 -- 172.21.15.162:0/3090160868 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dd800ba50 con 0x7f3de0061c60 2020-11-28T21:55:44.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.147+0000 7f3dd7fff700 1 -- 172.21.15.162:0/3090160868 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd800bbb0 con 0x7f3de0061c60 2020-11-28T21:55:44.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.147+0000 7f3dd7fff700 1 -- 172.21.15.162:0/3090160868 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dd800bd20 con 0x7f3de0061c60 2020-11-28T21:55:44.158 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.147+0000 7f3de6644700 1 -- 172.21.15.162:0/3090160868 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3de0061c60 msgr2=0x7f3de00ad580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:44.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.147+0000 7f3de6644700 1 --2- 172.21.15.162:0/3090160868 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3de0061c60 0x7f3de00ad580 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 rx=0x7f3dd80032b0 tx=0x7f3dd80091a0).stop 2020-11-28T21:55:44.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.147+0000 7f3de6644700 1 -- 172.21.15.162:0/3090160868 shutdown_connections 2020-11-28T21:55:44.159 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.147+0000 7f3de6644700 1 --2- 172.21.15.162:0/3090160868 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3de0061c60 0x7f3de00ad580 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:44.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.147+0000 7f3de6644700 1 --2- 172.21.15.162:0/3090160868 >> v2:172.21.15.118:3300/0 conn(0x7f3de006a420 0x7f3de0142910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:44.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.147+0000 7f3de6644700 1 -- 172.21.15.162:0/3090160868 >> 172.21.15.162:0/3090160868 conn(0x7f3de00568a0 msgr2=0x7f3de0061550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:44.160 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.147+0000 7f3de6644700 1 -- 172.21.15.162:0/3090160868 shutdown_connections 2020-11-28T21:55:44.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.147+0000 7f3de6644700 1 -- 172.21.15.162:0/3090160868 wait complete. 2020-11-28T21:55:44.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.151+0000 7f3de6644700 1 Processor -- start 2020-11-28T21:55:44.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.151+0000 7f3de6644700 1 -- start start 2020-11-28T21:55:44.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.151+0000 7f3de6644700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3de0061c60 0x7f3de0140be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:44.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.151+0000 7f3de6644700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3de006a420 0x7f3de013f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:44.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.151+0000 7f3de6644700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3de00b09a0 0x7f3de013fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:44.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.151+0000 7f3de6644700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de0143ae0 con 0x7f3de00b09a0 2020-11-28T21:55:44.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.151+0000 7f3de6644700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de0154b80 con 0x7f3de006a420 2020-11-28T21:55:44.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.151+0000 7f3de6644700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de013ebc0 con 0x7f3de0061c60 2020-11-28T21:55:44.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.151+0000 7f3de4e41700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3de006a420 0x7f3de013f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:44.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.151+0000 7f3de4e41700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3de006a420 0x7f3de013f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45222/0 (socket says 172.21.15.162:45222) 2020-11-28T21:55:44.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.151+0000 7f3de4e41700 1 -- 172.21.15.162:0/3792438106 learned_addr learned my addr 172.21.15.162:0/3792438106 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:44.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.151+0000 7f3de5642700 1 --2- 172.21.15.162:0/3792438106 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3de0061c60 0x7f3de0140be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:44.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.151+0000 7f3de5e43700 1 --2- 172.21.15.162:0/3792438106 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3de00b09a0 0x7f3de013fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:44.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.155+0000 7f3de4e41700 1 -- 172.21.15.162:0/3792438106 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3de0061c60 msgr2=0x7f3de0140be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:44.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.155+0000 7f3de4e41700 1 --2- 172.21.15.162:0/3792438106 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3de0061c60 0x7f3de0140be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:44.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.155+0000 7f3de4e41700 1 -- 172.21.15.162:0/3792438106 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3de00b09a0 msgr2=0x7f3de013fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:44.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.155+0000 7f3de4e41700 1 --2- 172.21.15.162:0/3792438106 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3de00b09a0 0x7f3de013fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:44.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.155+0000 7f3de4e41700 1 -- 172.21.15.162:0/3792438106 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de013e290 con 0x7f3de006a420 2020-11-28T21:55:44.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.155+0000 7f3de4e41700 1 --2- 172.21.15.162:0/3792438106 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3de006a420 0x7f3de013f7f0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 rx=0x7f3de0061090 tx=0x7f3dd0007c30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:44.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.155+0000 7f3dd67fc700 1 -- 172.21.15.162:0/3792438106 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dd0007380 con 0x7f3de006a420 2020-11-28T21:55:44.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.155+0000 7f3dd67fc700 1 -- 172.21.15.162:0/3792438106 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd0005030 con 0x7f3de006a420 2020-11-28T21:55:44.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.155+0000 7f3de6644700 1 -- 172.21.15.162:0/3792438106 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3de01401d0 con 0x7f3de006a420 2020-11-28T21:55:44.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.155+0000 7f3dd67fc700 1 -- 172.21.15.162:0/3792438106 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dd0005390 con 0x7f3de006a420 2020-11-28T21:55:44.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.155+0000 7f3de6644700 1 -- 172.21.15.162:0/3792438106 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de013de90 con 0x7f3de006a420 2020-11-28T21:55:44.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.155+0000 7f3de6644700 1 -- 172.21.15.162:0/3792438106 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3de0057050 con 0x7f3de006a420 2020-11-28T21:55:44.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.155+0000 7f3dd67fc700 1 -- 172.21.15.162:0/3792438106 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f3dd001b070 con 0x7f3de006a420 2020-11-28T21:55:44.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.155+0000 7f3dd67fc700 1 --2- 172.21.15.162:0/3792438106 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3dc80553b0 0x7f3dc8057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:44.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.155+0000 7f3dd67fc700 1 -- 172.21.15.162:0/3792438106 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3dd006d440 con 0x7f3de006a420 2020-11-28T21:55:44.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.155+0000 7f3de5642700 1 --2- 172.21.15.162:0/3792438106 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3dc80553b0 0x7f3dc8057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:44.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.159+0000 7f3de5642700 1 --2- 172.21.15.162:0/3792438106 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3dc80553b0 0x7f3dc8057870 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7f3ddc00c000 tx=0x7f3ddc008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:44.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.163+0000 7f3dd67fc700 1 -- 172.21.15.162:0/3792438106 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f3dd0043740 con 0x7f3de006a420 2020-11-28T21:55:44.306 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.303+0000 7f3de6644700 1 -- 172.21.15.162:0/3792438106 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f3de0068350 con 0x7f3de006a420 2020-11-28T21:55:44.307 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.303+0000 7f3dd67fc700 1 -- 172.21.15.162:0/3792438106 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v138) v1 ==== 110+0+10 (secure 0 0 0) 0x7f3dd0046d60 con 0x7f3de006a420 2020-11-28T21:55:44.308 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:55:44.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.307+0000 7f3de6644700 1 -- 172.21.15.162:0/3792438106 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3dc80553b0 msgr2=0x7f3dc8057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:44.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.307+0000 7f3de6644700 1 --2- 172.21.15.162:0/3792438106 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3dc80553b0 0x7f3dc8057870 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7f3ddc00c000 tx=0x7f3ddc008040).stop 2020-11-28T21:55:44.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.307+0000 7f3de6644700 1 -- 172.21.15.162:0/3792438106 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3de006a420 msgr2=0x7f3de013f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:44.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.307+0000 7f3de6644700 1 --2- 172.21.15.162:0/3792438106 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3de006a420 0x7f3de013f7f0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 rx=0x7f3de0061090 tx=0x7f3dd0007c30).stop 2020-11-28T21:55:44.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.307+0000 7f3de6644700 1 -- 172.21.15.162:0/3792438106 shutdown_connections 2020-11-28T21:55:44.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.307+0000 7f3de6644700 1 --2- 172.21.15.162:0/3792438106 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3dc80553b0 0x7f3dc8057870 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:44.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.307+0000 7f3de6644700 1 --2- 172.21.15.162:0/3792438106 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3de0061c60 0x7f3de0140be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:44.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.307+0000 7f3de6644700 1 --2- 172.21.15.162:0/3792438106 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3de00b09a0 0x7f3de013fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:44.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.307+0000 7f3de6644700 1 --2- 172.21.15.162:0/3792438106 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3de006a420 0x7f3de013f7f0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:44.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.307+0000 7f3de6644700 1 -- 172.21.15.162:0/3792438106 >> 172.21.15.162:0/3792438106 conn(0x7f3de00568a0 msgr2=0x7f3de00af520 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:44.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.307+0000 7f3de6644700 1 -- 172.21.15.162:0/3792438106 shutdown_connections 2020-11-28T21:55:44.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.307+0000 7f3de6644700 1 -- 172.21.15.162:0/3792438106 wait complete. 2020-11-28T21:55:44.328 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:55:44.447 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.443+0000 7fb7d44ad700 1 Processor -- start 2020-11-28T21:55:44.447 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.443+0000 7fb7d44ad700 1 -- start start 2020-11-28T21:55:44.449 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.443+0000 7fb7d44ad700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb7cc06acd0 0x7fb7cc13fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:44.449 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.443+0000 7fb7d44ad700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb7cc140280 0x7fb7cc143450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:44.450 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.443+0000 7fb7d44ad700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7cc06b0b0 con 0x7fb7cc0b52f0 2020-11-28T21:55:44.450 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.443+0000 7fb7d44ad700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7cc061100 con 0x7fb7cc140280 2020-11-28T21:55:44.450 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.443+0000 7fb7d44ad700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc060580 con 0x7fb7cc06acd0 2020-11-28T21:55:44.450 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.443+0000 7fb7d2a4a700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb7cc140280 0x7fb7cc143450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:44.450 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.443+0000 7fb7d2a4a700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb7cc140280 0x7fb7cc143450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45232/0 (socket says 172.21.15.162:45232) 2020-11-28T21:55:44.451 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.443+0000 7fb7d2a4a700 1 -- 172.21.15.162:0/1217066049 learned_addr learned my addr 172.21.15.162:0/1217066049 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:44.451 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.443+0000 7fb7d2249700 1 --2- 172.21.15.162:0/1217066049 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb7cc06acd0 0x7fb7cc13fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:44.451 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.443+0000 7fb7d2a4a700 1 -- 172.21.15.162:0/1217066049 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb7cc06acd0 msgr2=0x7fb7cc13fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:44.451 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.443+0000 7fb7d2a4a700 1 --2- 172.21.15.162:0/1217066049 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb7cc06acd0 0x7fb7cc13fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:44.451 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.443+0000 7fb7d2a4a700 1 -- 172.21.15.162:0/1217066049 >> v1:172.21.15.118:6789/0 conn(0x7fb7cc0b52f0 legacy=0x7fb7cc142d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:44.452 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.443+0000 7fb7d2a4a700 1 -- 172.21.15.162:0/1217066049 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7cc060820 con 0x7fb7cc140280 2020-11-28T21:55:44.452 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.443+0000 7fb7d2a4a700 1 --2- 172.21.15.162:0/1217066049 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7cc140280 0x7fb7cc143450 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 rx=0x7fb7c40032b0 tx=0x7fb7c4009270).ready entity=mon.1 client_cookie=9f7ed5ea82238614 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:44.452 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.443+0000 7fb7d1247700 1 -- 172.21.15.162:0/1217066049 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7c400bc00 con 0x7fb7cc140280 2020-11-28T21:55:44.452 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.443+0000 7fb7d1247700 1 -- 172.21.15.162:0/1217066049 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7c400bd60 con 0x7fb7cc140280 2020-11-28T21:55:44.453 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.443+0000 7fb7d1247700 1 -- 172.21.15.162:0/1217066049 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7c4011400 con 0x7fb7cc140280 2020-11-28T21:55:44.453 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.443+0000 7fb7d44ad700 1 -- 172.21.15.162:0/1217066049 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7cc140280 msgr2=0x7fb7cc143450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:44.453 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.443+0000 7fb7d44ad700 1 --2- 172.21.15.162:0/1217066049 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7cc140280 0x7fb7cc143450 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 rx=0x7fb7c40032b0 tx=0x7fb7c4009270).stop 2020-11-28T21:55:44.456 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.447+0000 7fb7d44ad700 1 -- 172.21.15.162:0/1217066049 shutdown_connections 2020-11-28T21:55:44.456 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.447+0000 7fb7d44ad700 1 --2- 172.21.15.162:0/1217066049 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7cc140280 0x7fb7cc143450 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:44.457 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.447+0000 7fb7d44ad700 1 --2- 172.21.15.162:0/1217066049 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb7cc06acd0 0x7fb7cc13fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:44.457 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.447+0000 7fb7d44ad700 1 -- 172.21.15.162:0/1217066049 >> 172.21.15.162:0/1217066049 conn(0x7fb7cc0568a0 msgr2=0x7fb7cc06a280 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:44.457 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.447+0000 7fb7d44ad700 1 -- 172.21.15.162:0/1217066049 shutdown_connections 2020-11-28T21:55:44.457 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.447+0000 7fb7d44ad700 1 -- 172.21.15.162:0/1217066049 wait complete. 2020-11-28T21:55:44.457 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.447+0000 7fb7d44ad700 1 Processor -- start 2020-11-28T21:55:44.458 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.447+0000 7fb7d44ad700 1 -- start start 2020-11-28T21:55:44.458 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.447+0000 7fb7d44ad700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7cc06acd0 0x7fb7cc0b09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:44.458 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.447+0000 7fb7d44ad700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb7cc0b52f0 0x7fb7cc0b8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:44.458 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.447+0000 7fb7d44ad700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb7cc140280 0x7fb7cc0b3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:44.459 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.447+0000 7fb7d44ad700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc0b0830 con 0x7fb7cc140280 2020-11-28T21:55:44.459 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.447+0000 7fb7d44ad700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc149810 con 0x7fb7cc06acd0 2020-11-28T21:55:44.459 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.447+0000 7fb7d44ad700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc152900 con 0x7fb7cc0b52f0 2020-11-28T21:55:44.459 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.447+0000 7fb7d2249700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7cc06acd0 0x7fb7cc0b09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:44.459 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.447+0000 7fb7d2249700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7cc06acd0 0x7fb7cc0b09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45236/0 (socket says 172.21.15.162:45236) 2020-11-28T21:55:44.460 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.447+0000 7fb7d2249700 1 -- 172.21.15.162:0/1761935457 learned_addr learned my addr 172.21.15.162:0/1761935457 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:44.460 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.447+0000 7fb7d2a4a700 1 --2- 172.21.15.162:0/1761935457 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb7cc140280 0x7fb7cc0b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:44.460 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.447+0000 7fb7d1a48700 1 --2- 172.21.15.162:0/1761935457 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb7cc0b52f0 0x7fb7cc0b8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:44.460 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.447+0000 7fb7d2249700 1 -- 172.21.15.162:0/1761935457 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb7cc0b52f0 msgr2=0x7fb7cc0b8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:44.460 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.447+0000 7fb7d2249700 1 --2- 172.21.15.162:0/1761935457 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb7cc0b52f0 0x7fb7cc0b8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:44.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.447+0000 7fb7d2249700 1 -- 172.21.15.162:0/1761935457 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb7cc140280 msgr2=0x7fb7cc0b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:44.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.447+0000 7fb7d2249700 1 --2- 172.21.15.162:0/1761935457 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb7cc140280 0x7fb7cc0b3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:44.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.447+0000 7fb7d2249700 1 -- 172.21.15.162:0/1761935457 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7cc149a40 con 0x7fb7cc06acd0 2020-11-28T21:55:44.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.451+0000 7fb7d2249700 1 --2- 172.21.15.162:0/1761935457 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7cc06acd0 0x7fb7cc0b09d0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 rx=0x7fb7cc152bd0 tx=0x7fb7bc008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:44.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.451+0000 7fb7bb7fe700 1 -- 172.21.15.162:0/1761935457 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7bc009790 con 0x7fb7cc06acd0 2020-11-28T21:55:44.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.451+0000 7fb7d44ad700 1 -- 172.21.15.162:0/1761935457 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7cc149ce0 con 0x7fb7cc06acd0 2020-11-28T21:55:44.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.451+0000 7fb7bb7fe700 1 -- 172.21.15.162:0/1761935457 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7bc0098f0 con 0x7fb7cc06acd0 2020-11-28T21:55:44.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.451+0000 7fb7bb7fe700 1 -- 172.21.15.162:0/1761935457 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7bc009c50 con 0x7fb7cc06acd0 2020-11-28T21:55:44.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.451+0000 7fb7d44ad700 1 -- 172.21.15.162:0/1761935457 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7cc09c420 con 0x7fb7cc06acd0 2020-11-28T21:55:44.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.451+0000 7fb7d44ad700 1 -- 172.21.15.162:0/1761935457 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7cc0b2e70 con 0x7fb7cc06acd0 2020-11-28T21:55:44.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.451+0000 7fb7bb7fe700 1 -- 172.21.15.162:0/1761935457 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb7bc013480 con 0x7fb7cc06acd0 2020-11-28T21:55:44.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.451+0000 7fb7bb7fe700 1 --2- 172.21.15.162:0/1761935457 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb7ac055440 0x7fb7ac057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:44.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.451+0000 7fb7d1a48700 1 --2- 172.21.15.162:0/1761935457 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb7ac055440 0x7fb7ac057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:44.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.451+0000 7fb7bb7fe700 1 -- 172.21.15.162:0/1761935457 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb7bc06e500 con 0x7fb7cc06acd0 2020-11-28T21:55:44.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.455+0000 7fb7d1a48700 1 --2- 172.21.15.162:0/1761935457 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb7ac055440 0x7fb7ac057900 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7fb7c800d000 tx=0x7fb7c8000b80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:44.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.459+0000 7fb7bb7fe700 1 -- 172.21.15.162:0/1761935457 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb7bc0447a0 con 0x7fb7cc06acd0 2020-11-28T21:55:44.599 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.595+0000 7fb7d44ad700 1 -- 172.21.15.162:0/1761935457 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb7cc0b3690 con 0x7fb7cc06acd0 2020-11-28T21:55:44.599 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.595+0000 7fb7bb7fe700 1 -- 172.21.15.162:0/1761935457 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v138) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb7bc009db0 con 0x7fb7cc06acd0 2020-11-28T21:55:44.601 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:55:44.603 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.595+0000 7fb7d44ad700 1 -- 172.21.15.162:0/1761935457 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb7ac055440 msgr2=0x7fb7ac057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:44.603 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.595+0000 7fb7d44ad700 1 --2- 172.21.15.162:0/1761935457 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb7ac055440 0x7fb7ac057900 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7fb7c800d000 tx=0x7fb7c8000b80).stop 2020-11-28T21:55:44.603 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.595+0000 7fb7d44ad700 1 -- 172.21.15.162:0/1761935457 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7cc06acd0 msgr2=0x7fb7cc0b09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:44.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.595+0000 7fb7d44ad700 1 --2- 172.21.15.162:0/1761935457 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7cc06acd0 0x7fb7cc0b09d0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 rx=0x7fb7cc152bd0 tx=0x7fb7bc008040).stop 2020-11-28T21:55:44.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.595+0000 7fb7d44ad700 1 -- 172.21.15.162:0/1761935457 shutdown_connections 2020-11-28T21:55:44.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.595+0000 7fb7d44ad700 1 --2- 172.21.15.162:0/1761935457 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb7ac055440 0x7fb7ac057900 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:44.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.595+0000 7fb7d44ad700 1 --2- 172.21.15.162:0/1761935457 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb7cc140280 0x7fb7cc0b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:44.604 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.595+0000 7fb7d44ad700 1 --2- 172.21.15.162:0/1761935457 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7cc06acd0 0x7fb7cc0b09d0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:44.605 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.595+0000 7fb7d44ad700 1 --2- 172.21.15.162:0/1761935457 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb7cc0b52f0 0x7fb7cc0b8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:44.605 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.595+0000 7fb7d44ad700 1 -- 172.21.15.162:0/1761935457 >> 172.21.15.162:0/1761935457 conn(0x7fb7cc0568a0 msgr2=0x7fb7cc0b4530 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:44.605 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.595+0000 7fb7d44ad700 1 -- 172.21.15.162:0/1761935457 shutdown_connections 2020-11-28T21:55:44.605 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.595+0000 7fb7d44ad700 1 -- 172.21.15.162:0/1761935457 wait complete. 2020-11-28T21:55:44.617 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-28T21:55:44.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.727+0000 7fbd2bd5e700 1 Processor -- start 2020-11-28T21:55:44.733 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.727+0000 7fbd2bd5e700 1 -- start start 2020-11-28T21:55:44.734 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.731+0000 7fbd2bd5e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbd24140550 0x7fbd24142ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:44.735 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.731+0000 7fbd2bd5e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd24140bf0 con 0x7fbd240b67d0 2020-11-28T21:55:44.735 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.731+0000 7fbd2bd5e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd240564f0 con 0x7fbd24056110 2020-11-28T21:55:44.735 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.731+0000 7fbd2bd5e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd24140980 con 0x7fbd24140550 2020-11-28T21:55:44.735 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.731+0000 7fbd2a2fb700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbd24140550 0x7fbd24142ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:44.735 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.731+0000 7fbd2a2fb700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbd24140550 0x7fbd24142ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45246/0 (socket says 172.21.15.162:45246) 2020-11-28T21:55:44.736 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.731+0000 7fbd2a2fb700 1 -- 172.21.15.162:0/383910477 learned_addr learned my addr 172.21.15.162:0/383910477 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:44.736 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.731+0000 7fbd29afa700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fbd24056110 0x7fbd24146d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:52086/0 (socket says 172.21.15.162:52086) 2020-11-28T21:55:44.736 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.731+0000 7fbd2a2fb700 1 -- 172.21.15.162:0/383910477 >> v1:172.21.15.162:6789/0 conn(0x7fbd24056110 legacy=0x7fbd24146d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:44.736 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.731+0000 7fbd2a2fb700 1 -- 172.21.15.162:0/383910477 >> v1:172.21.15.118:6789/0 conn(0x7fbd240b67d0 legacy=0x7fbd2413fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:44.736 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.731+0000 7fbd2a2fb700 1 -- 172.21.15.162:0/383910477 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd240b4110 con 0x7fbd24140550 2020-11-28T21:55:44.737 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.731+0000 7fbd2a2fb700 1 --2- 172.21.15.162:0/383910477 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd24140550 0x7fbd24142ef0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 rx=0x7fbd240ad090 tx=0x7fbd1c009270).ready entity=mon.1 client_cookie=a80150f13cd0fc2 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:44.737 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.731+0000 7fbd28af8700 1 -- 172.21.15.162:0/383910477 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd1c00bbc0 con 0x7fbd24140550 2020-11-28T21:55:44.737 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.731+0000 7fbd28af8700 1 -- 172.21.15.162:0/383910477 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd1c00bd20 con 0x7fbd24140550 2020-11-28T21:55:44.737 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.731+0000 7fbd28af8700 1 -- 172.21.15.162:0/383910477 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd1c00be90 con 0x7fbd24140550 2020-11-28T21:55:44.737 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.731+0000 7fbd2bd5e700 1 -- 172.21.15.162:0/383910477 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd24140550 msgr2=0x7fbd24142ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:44.738 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.731+0000 7fbd2bd5e700 1 --2- 172.21.15.162:0/383910477 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd24140550 0x7fbd24142ef0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 rx=0x7fbd240ad090 tx=0x7fbd1c009270).stop 2020-11-28T21:55:44.740 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.731+0000 7fbd2bd5e700 1 -- 172.21.15.162:0/383910477 shutdown_connections 2020-11-28T21:55:44.741 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.731+0000 7fbd2bd5e700 1 --2- 172.21.15.162:0/383910477 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd24140550 0x7fbd24142ef0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:44.741 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.731+0000 7fbd2bd5e700 1 -- 172.21.15.162:0/383910477 >> 172.21.15.162:0/383910477 conn(0x7fbd24060730 msgr2=0x7fbd24060b10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:44.741 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.731+0000 7fbd2bd5e700 1 -- 172.21.15.162:0/383910477 shutdown_connections 2020-11-28T21:55:44.741 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.731+0000 7fbd2bd5e700 1 -- 172.21.15.162:0/383910477 wait complete. 2020-11-28T21:55:44.741 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.735+0000 7fbd2bd5e700 1 Processor -- start 2020-11-28T21:55:44.742 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.735+0000 7fbd2bd5e700 1 -- start start 2020-11-28T21:55:44.742 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.735+0000 7fbd2bd5e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd24056110 0x7fbd24145370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:44.742 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.735+0000 7fbd2bd5e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbd240b67d0 0x7fbd241558b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:44.742 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.735+0000 7fbd2bd5e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbd24140550 0x7fbd24155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:44.743 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.735+0000 7fbd2bd5e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd24156290 con 0x7fbd24140550 2020-11-28T21:55:44.743 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.735+0000 7fbd2bd5e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd24143370 con 0x7fbd24056110 2020-11-28T21:55:44.743 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.735+0000 7fbd2bd5e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd24143610 con 0x7fbd240b67d0 2020-11-28T21:55:44.744 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.735+0000 7fbd29afa700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd24056110 0x7fbd24145370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:44.744 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.735+0000 7fbd29afa700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd24056110 0x7fbd24145370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45248/0 (socket says 172.21.15.162:45248) 2020-11-28T21:55:44.744 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.735+0000 7fbd29afa700 1 -- 172.21.15.162:0/2338951965 learned_addr learned my addr 172.21.15.162:0/2338951965 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:44.744 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.735+0000 7fbd2a2fb700 1 --2- 172.21.15.162:0/2338951965 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbd24140550 0x7fbd24155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:44.745 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.735+0000 7fbd292f9700 1 --2- 172.21.15.162:0/2338951965 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbd240b67d0 0x7fbd241558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:44.745 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.735+0000 7fbd29afa700 1 -- 172.21.15.162:0/2338951965 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbd240b67d0 msgr2=0x7fbd241558b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:44.745 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.735+0000 7fbd29afa700 1 --2- 172.21.15.162:0/2338951965 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbd240b67d0 0x7fbd241558b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:44.745 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.735+0000 7fbd29afa700 1 -- 172.21.15.162:0/2338951965 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbd24140550 msgr2=0x7fbd24155da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:44.745 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.735+0000 7fbd29afa700 1 --2- 172.21.15.162:0/2338951965 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbd24140550 0x7fbd24155da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:44.746 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.735+0000 7fbd29afa700 1 -- 172.21.15.162:0/2338951965 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd241438b0 con 0x7fbd24056110 2020-11-28T21:55:44.746 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.735+0000 7fbd2a2fb700 1 --2- 172.21.15.162:0/2338951965 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbd24140550 0x7fbd24155da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:55:44.746 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.735+0000 7fbd29afa700 1 --2- 172.21.15.162:0/2338951965 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd24056110 0x7fbd24145370 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 rx=0x7fbd241563d0 tx=0x7fbd20007c10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:44.746 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.735+0000 7fbd16ffd700 1 -- 172.21.15.162:0/2338951965 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd200073b0 con 0x7fbd24056110 2020-11-28T21:55:44.746 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.735+0000 7fbd16ffd700 1 -- 172.21.15.162:0/2338951965 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd20005520 con 0x7fbd24056110 2020-11-28T21:55:44.747 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.735+0000 7fbd2bd5e700 1 -- 172.21.15.162:0/2338951965 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd24155410 con 0x7fbd24056110 2020-11-28T21:55:44.747 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.735+0000 7fbd16ffd700 1 -- 172.21.15.162:0/2338951965 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd20005880 con 0x7fbd24056110 2020-11-28T21:55:44.748 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.735+0000 7fbd2bd5e700 1 -- 172.21.15.162:0/2338951965 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd24151dc0 con 0x7fbd24056110 2020-11-28T21:55:44.749 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.739+0000 7fbd2bd5e700 1 -- 172.21.15.162:0/2338951965 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd24057050 con 0x7fbd24056110 2020-11-28T21:55:44.750 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.739+0000 7fbd16ffd700 1 -- 172.21.15.162:0/2338951965 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fbd20010070 con 0x7fbd24056110 2020-11-28T21:55:44.750 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.739+0000 7fbd16ffd700 1 --2- 172.21.15.162:0/2338951965 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbd040553b0 0x7fbd04057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:44.750 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.739+0000 7fbd292f9700 1 --2- 172.21.15.162:0/2338951965 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbd040553b0 0x7fbd04057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:44.750 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.739+0000 7fbd16ffd700 1 -- 172.21.15.162:0/2338951965 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbd2006d5e0 con 0x7fbd24056110 2020-11-28T21:55:44.751 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.739+0000 7fbd292f9700 1 --2- 172.21.15.162:0/2338951965 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbd040553b0 0x7fbd04057870 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7fbd1800d000 tx=0x7fbd18007b40).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:44.751 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.747+0000 7fbd16ffd700 1 -- 172.21.15.162:0/2338951965 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fbd200437d0 con 0x7fbd24056110 2020-11-28T21:55:44.889 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.883+0000 7fbd2bd5e700 1 -- 172.21.15.162:0/2338951965 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbd2405d5a0 con 0x7fbd24056110 2020-11-28T21:55:44.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.887+0000 7fbd16ffd700 1 -- 172.21.15.162:0/2338951965 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v138) v1 ==== 104+0+10 (secure 0 0 0) 0x7fbd20018020 con 0x7fbd24056110 2020-11-28T21:55:44.890 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:55:44.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.887+0000 7fbd2bd5e700 1 -- 172.21.15.162:0/2338951965 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbd040553b0 msgr2=0x7fbd04057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:44.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.887+0000 7fbd2bd5e700 1 --2- 172.21.15.162:0/2338951965 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbd040553b0 0x7fbd04057870 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7fbd1800d000 tx=0x7fbd18007b40).stop 2020-11-28T21:55:44.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.887+0000 7fbd2bd5e700 1 -- 172.21.15.162:0/2338951965 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd24056110 msgr2=0x7fbd24145370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:44.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.887+0000 7fbd2bd5e700 1 --2- 172.21.15.162:0/2338951965 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd24056110 0x7fbd24145370 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 rx=0x7fbd241563d0 tx=0x7fbd20007c10).stop 2020-11-28T21:55:44.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.887+0000 7fbd2bd5e700 1 -- 172.21.15.162:0/2338951965 shutdown_connections 2020-11-28T21:55:44.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.887+0000 7fbd2bd5e700 1 --2- 172.21.15.162:0/2338951965 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbd040553b0 0x7fbd04057870 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:44.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.887+0000 7fbd2bd5e700 1 --2- 172.21.15.162:0/2338951965 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbd24140550 0x7fbd24155da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:44.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.887+0000 7fbd2bd5e700 1 --2- 172.21.15.162:0/2338951965 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd24056110 0x7fbd24145370 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:44.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.887+0000 7fbd2bd5e700 1 --2- 172.21.15.162:0/2338951965 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbd240b67d0 0x7fbd241558b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:44.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.887+0000 7fbd2bd5e700 1 -- 172.21.15.162:0/2338951965 >> 172.21.15.162:0/2338951965 conn(0x7fbd24060730 msgr2=0x7fbd240b7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:44.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.887+0000 7fbd2bd5e700 1 -- 172.21.15.162:0/2338951965 shutdown_connections 2020-11-28T21:55:44.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:44.887+0000 7fbd2bd5e700 1 -- 172.21.15.162:0/2338951965 wait complete. 2020-11-28T21:55:44.905 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-28T21:55:45.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.015+0000 7f5f1b59e700 1 Processor -- start 2020-11-28T21:55:45.020 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.015+0000 7f5f1b59e700 1 -- start start 2020-11-28T21:55:45.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.015+0000 7f5f1b59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5f1c0b3d20 0x7f5f1c1405c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:45.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.015+0000 7f5f1b59e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f1c0671a0 con 0x7f5f1c146bc0 2020-11-28T21:55:45.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.015+0000 7f5f1b59e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f1c0b3a30 con 0x7f5f1c066dc0 2020-11-28T21:55:45.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.015+0000 7f5f1b59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c0b64a0 con 0x7f5f1c0b3d20 2020-11-28T21:55:45.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.015+0000 7f5f1a59c700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f5f1c066dc0 0x7f5f1c1464b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:52100/0 (socket says 172.21.15.162:52100) 2020-11-28T21:55:45.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.015+0000 7f5f1a59c700 1 -- 172.21.15.162:0/3168448448 learned_addr learned my addr 172.21.15.162:0/3168448448 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:45.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.019+0000 7f5f1ad9d700 1 --2- 172.21.15.162:0/3168448448 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5f1c0b3d20 0x7f5f1c1405c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:45.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.019+0000 7f5f1959a700 1 -- 172.21.15.162:0/3168448448 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3911479785 0 0) 0x7f5f1c0b3a30 con 0x7f5f1c066dc0 2020-11-28T21:55:45.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.019+0000 7f5f1959a700 1 -- 172.21.15.162:0/3168448448 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f00003230 con 0x7f5f1c066dc0 2020-11-28T21:55:45.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.019+0000 7f5f1959a700 1 -- 172.21.15.162:0/3168448448 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 834121097 0 0) 0x7f5f00003230 con 0x7f5f1c066dc0 2020-11-28T21:55:45.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.019+0000 7f5f1959a700 1 -- 172.21.15.162:0/3168448448 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5f1c0b3d20 msgr2=0x7f5f1c1405c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:45.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.019+0000 7f5f1959a700 1 --2- 172.21.15.162:0/3168448448 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5f1c0b3d20 0x7f5f1c1405c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:45.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.019+0000 7f5f1959a700 1 -- 172.21.15.162:0/3168448448 >> v1:172.21.15.118:6789/0 conn(0x7f5f1c146bc0 legacy=0x7f5f1c13feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:45.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.019+0000 7f5f1959a700 1 -- 172.21.15.162:0/3168448448 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1c0b2120 con 0x7f5f1c066dc0 2020-11-28T21:55:45.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.019+0000 7f5f1959a700 1 -- 172.21.15.162:0/3168448448 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f5f100016d0 con 0x7f5f1c066dc0 2020-11-28T21:55:45.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.019+0000 7f5f1959a700 1 -- 172.21.15.162:0/3168448448 >> v1:172.21.15.162:6789/0 conn(0x7f5f1c066dc0 legacy=0x7f5f1c1464b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:45.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.019+0000 7f5f1959a700 1 --2- 172.21.15.162:0/3168448448 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f00003720 0x7f5f00003b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:45.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.019+0000 7f5f1959a700 1 --2- 172.21.15.162:0/3168448448 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f00004080 0x7f5f00004550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:45.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.019+0000 7f5f1959a700 1 --2- 172.21.15.162:0/3168448448 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5f1c0b3d20 0x7f5f00004bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:45.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.019+0000 7f5f1959a700 1 -- 172.21.15.162:0/3168448448 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c0b64a0 con 0x7f5f00004080 2020-11-28T21:55:45.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.019+0000 7f5f1959a700 1 -- 172.21.15.162:0/3168448448 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f000050f0 con 0x7f5f00003720 2020-11-28T21:55:45.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.019+0000 7f5f1959a700 1 -- 172.21.15.162:0/3168448448 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f000052b0 con 0x7f5f1c0b3d20 2020-11-28T21:55:45.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.019+0000 7f5f1a59c700 1 --2- 172.21.15.162:0/3168448448 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f00003720 0x7f5f00003b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:45.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.019+0000 7f5f1ad9d700 1 --2- 172.21.15.162:0/3168448448 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5f1c0b3d20 0x7f5f00004bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:45.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.019+0000 7f5f19d9b700 1 --2- 172.21.15.162:0/3168448448 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f00004080 0x7f5f00004550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:45.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.019+0000 7f5f1a59c700 1 -- 172.21.15.162:0/3168448448 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5f1c0b3d20 msgr2=0x7f5f00004bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:45.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.019+0000 7f5f1a59c700 1 --2- 172.21.15.162:0/3168448448 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5f1c0b3d20 0x7f5f00004bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:45.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.019+0000 7f5f1a59c700 1 -- 172.21.15.162:0/3168448448 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f00004080 msgr2=0x7f5f00004550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:45.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.019+0000 7f5f1a59c700 1 --2- 172.21.15.162:0/3168448448 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f00004080 0x7f5f00004550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:45.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.019+0000 7f5f1a59c700 1 -- 172.21.15.162:0/3168448448 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f00003230 con 0x7f5f00003720 2020-11-28T21:55:45.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.019+0000 7f5f1a59c700 1 --2- 172.21.15.162:0/3168448448 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f00003720 0x7f5f00003b20 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 rx=0x7f5f100013e0 tx=0x7f5f1c0b22b0).ready entity=mon.1 client_cookie=cedacff943afeda5 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:45.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.019+0000 7f5f1959a700 1 -- 172.21.15.162:0/3168448448 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f100070d0 con 0x7f5f00003720 2020-11-28T21:55:45.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.019+0000 7f5f1959a700 1 -- 172.21.15.162:0/3168448448 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f10007230 con 0x7f5f00003720 2020-11-28T21:55:45.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.019+0000 7f5f1959a700 1 -- 172.21.15.162:0/3168448448 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f100055e0 con 0x7f5f00003720 2020-11-28T21:55:45.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.019+0000 7f5f1b59e700 1 -- 172.21.15.162:0/3168448448 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f00003720 msgr2=0x7f5f00003b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:45.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.019+0000 7f5f1b59e700 1 --2- 172.21.15.162:0/3168448448 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f00003720 0x7f5f00003b20 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 rx=0x7f5f100013e0 tx=0x7f5f1c0b22b0).stop 2020-11-28T21:55:45.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.019+0000 7f5f1ad9d700 1 -- 172.21.15.162:0/3168448448 reap_dead start 2020-11-28T21:55:45.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.019+0000 7f5f1b59e700 1 -- 172.21.15.162:0/3168448448 shutdown_connections 2020-11-28T21:55:45.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.019+0000 7f5f1b59e700 1 -- 172.21.15.162:0/3168448448 >> 172.21.15.162:0/3168448448 conn(0x7f5f1c0606c0 msgr2=0x7f5f1c060aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:45.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.019+0000 7f5f1b59e700 1 -- 172.21.15.162:0/3168448448 shutdown_connections 2020-11-28T21:55:45.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.019+0000 7f5f1b59e700 1 -- 172.21.15.162:0/3168448448 wait complete. 2020-11-28T21:55:45.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.023+0000 7f5f1b59e700 1 Processor -- start 2020-11-28T21:55:45.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.023+0000 7f5f1b59e700 1 -- start start 2020-11-28T21:55:45.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.023+0000 7f5f1b59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f1c13cd60 0x7f5f1c0b8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:45.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.023+0000 7f5f1b59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f1c13f1b0 0x7f5f1c0b94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:45.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.023+0000 7f5f1b59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5f1c152150 0x7f5f1c1525d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:45.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.023+0000 7f5f1b59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c0ae610 con 0x7f5f1c13f1b0 2020-11-28T21:55:45.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.023+0000 7f5f1b59e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c13d140 con 0x7f5f1c13cd60 2020-11-28T21:55:45.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.023+0000 7f5f1b59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c0ae3b0 con 0x7f5f1c152150 2020-11-28T21:55:45.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.023+0000 7f5f1a59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f1c13cd60 0x7f5f1c0b8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:45.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.023+0000 7f5f1a59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f1c13cd60 0x7f5f1c0b8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45268/0 (socket says 172.21.15.162:45268) 2020-11-28T21:55:45.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.023+0000 7f5f1a59c700 1 -- 172.21.15.162:0/306181816 learned_addr learned my addr 172.21.15.162:0/306181816 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:45.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.023+0000 7f5f1ad9d700 1 --2- 172.21.15.162:0/306181816 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5f1c152150 0x7f5f1c1525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:45.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.023+0000 7f5f19d9b700 1 --2- 172.21.15.162:0/306181816 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f1c13f1b0 0x7f5f1c0b94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:45.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.023+0000 7f5f1a59c700 1 -- 172.21.15.162:0/306181816 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5f1c152150 msgr2=0x7f5f1c1525d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:45.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.023+0000 7f5f1a59c700 1 --2- 172.21.15.162:0/306181816 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5f1c152150 0x7f5f1c1525d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:45.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.023+0000 7f5f1a59c700 1 -- 172.21.15.162:0/306181816 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f1c13f1b0 msgr2=0x7f5f1c0b94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:45.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.023+0000 7f5f1a59c700 1 --2- 172.21.15.162:0/306181816 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f1c13f1b0 0x7f5f1c0b94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:45.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.023+0000 7f5f1a59c700 1 -- 172.21.15.162:0/306181816 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1c156010 con 0x7f5f1c13cd60 2020-11-28T21:55:45.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.023+0000 7f5f1a59c700 1 --2- 172.21.15.162:0/306181816 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f1c13cd60 0x7f5f1c0b8ff0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 rx=0x7f5f100013e0 tx=0x7f5f10015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:45.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.023+0000 7f5f0b7fe700 1 -- 172.21.15.162:0/306181816 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f100049b0 con 0x7f5f1c13cd60 2020-11-28T21:55:45.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.023+0000 7f5f0b7fe700 1 -- 172.21.15.162:0/306181816 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f10004b10 con 0x7f5f1c13cd60 2020-11-28T21:55:45.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.023+0000 7f5f1b59e700 1 -- 172.21.15.162:0/306181816 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f1c1562b0 con 0x7f5f1c13cd60 2020-11-28T21:55:45.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.023+0000 7f5f0b7fe700 1 -- 172.21.15.162:0/306181816 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f10004e70 con 0x7f5f1c13cd60 2020-11-28T21:55:45.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.023+0000 7f5f1b59e700 1 -- 172.21.15.162:0/306181816 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f1c0b9b70 con 0x7f5f1c13cd60 2020-11-28T21:55:45.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.027+0000 7f5f1b59e700 1 -- 172.21.15.162:0/306181816 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f1c151f90 con 0x7f5f1c13cd60 2020-11-28T21:55:45.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.027+0000 7f5f0b7fe700 1 -- 172.21.15.162:0/306181816 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5f10027070 con 0x7f5f1c13cd60 2020-11-28T21:55:45.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.027+0000 7f5f0b7fe700 1 --2- 172.21.15.162:0/306181816 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5ef8055440 0x7f5ef8057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:45.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.027+0000 7f5f0b7fe700 1 -- 172.21.15.162:0/306181816 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5f1006ea30 con 0x7f5f1c13cd60 2020-11-28T21:55:45.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.027+0000 7f5f19d9b700 1 --2- 172.21.15.162:0/306181816 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5ef8055440 0x7f5ef8057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:45.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.027+0000 7f5f19d9b700 1 --2- 172.21.15.162:0/306181816 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5ef8055440 0x7f5ef8057900 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7f5f0c003000 tx=0x7f5f0c007b40).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:45.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.035+0000 7f5f0b7fe700 1 -- 172.21.15.162:0/306181816 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5f10044d40 con 0x7f5f1c13cd60 2020-11-28T21:55:45.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.175+0000 7f5f1b59e700 1 -- 172.21.15.162:0/306181816 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5f1c057050 con 0x7f5f1c13cd60 2020-11-28T21:55:45.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.175+0000 7f5f0b7fe700 1 -- 172.21.15.162:0/306181816 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v138) v1 ==== 100+0+10 (secure 0 0 0) 0x7f5f10048360 con 0x7f5f1c13cd60 2020-11-28T21:55:45.182 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:55:45.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.179+0000 7f5f1b59e700 1 -- 172.21.15.162:0/306181816 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5ef8055440 msgr2=0x7f5ef8057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:45.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.179+0000 7f5f1b59e700 1 --2- 172.21.15.162:0/306181816 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5ef8055440 0x7f5ef8057900 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7f5f0c003000 tx=0x7f5f0c007b40).stop 2020-11-28T21:55:45.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.179+0000 7f5f1b59e700 1 -- 172.21.15.162:0/306181816 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f1c13cd60 msgr2=0x7f5f1c0b8ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:45.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.179+0000 7f5f1b59e700 1 --2- 172.21.15.162:0/306181816 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f1c13cd60 0x7f5f1c0b8ff0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 rx=0x7f5f100013e0 tx=0x7f5f10015000).stop 2020-11-28T21:55:45.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.179+0000 7f5f1b59e700 1 -- 172.21.15.162:0/306181816 shutdown_connections 2020-11-28T21:55:45.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.179+0000 7f5f1b59e700 1 --2- 172.21.15.162:0/306181816 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5ef8055440 0x7f5ef8057900 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:45.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.179+0000 7f5f1b59e700 1 --2- 172.21.15.162:0/306181816 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5f1c152150 0x7f5f1c1525d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:45.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.179+0000 7f5f1b59e700 1 --2- 172.21.15.162:0/306181816 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f1c13cd60 0x7f5f1c0b8ff0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:45.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.179+0000 7f5f1b59e700 1 --2- 172.21.15.162:0/306181816 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f1c13f1b0 0x7f5f1c0b94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:45.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.179+0000 7f5f1b59e700 1 -- 172.21.15.162:0/306181816 >> 172.21.15.162:0/306181816 conn(0x7f5f1c0606c0 msgr2=0x7f5f1c1462f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:45.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.179+0000 7f5f1b59e700 1 -- 172.21.15.162:0/306181816 shutdown_connections 2020-11-28T21:55:45.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.179+0000 7f5f1b59e700 1 -- 172.21.15.162:0/306181816 wait complete. 2020-11-28T21:55:45.195 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-28T21:55:45.196 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:55:45.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.299+0000 7fca7741b700 1 Processor -- start 2020-11-28T21:55:45.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.299+0000 7fca7741b700 1 -- start start 2020-11-28T21:55:45.306 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.303+0000 7fca7741b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fca700b3260 0x7fca70063200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:45.306 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.303+0000 7fca7741b700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca701417c0 con 0x7fca700b7c50 2020-11-28T21:55:45.307 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.303+0000 7fca7741b700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca70144d00 con 0x7fca70140840 2020-11-28T21:55:45.307 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.303+0000 7fca7741b700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca70061fe0 con 0x7fca700b3260 2020-11-28T21:55:45.307 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.303+0000 7fca749b6700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fca700b3260 0x7fca70063200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:45.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.303+0000 7fca759b8700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fca70140840 0x7fca7013a230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:52124/0 (socket says 172.21.15.162:52124) 2020-11-28T21:55:45.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.303+0000 7fca759b8700 1 -- 172.21.15.162:0/1697062842 learned_addr learned my addr 172.21.15.162:0/1697062842 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:45.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.303+0000 7fca749b6700 1 -- 172.21.15.162:0/1697062842 >> v1:172.21.15.162:6789/0 conn(0x7fca70140840 legacy=0x7fca7013a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:45.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.303+0000 7fca749b6700 1 -- 172.21.15.162:0/1697062842 >> v1:172.21.15.118:6789/0 conn(0x7fca700b7c50 legacy=0x7fca701445f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:45.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.303+0000 7fca749b6700 1 -- 172.21.15.162:0/1697062842 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca700b8030 con 0x7fca700b3260 2020-11-28T21:55:45.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.303+0000 7fca67fff700 1 -- 172.21.15.162:0/1697062842 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3061661653 0 0) 0x7fca70144d00 con 0x7fca70140840 2020-11-28T21:55:45.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.303+0000 7fca749b6700 1 --2- 172.21.15.162:0/1697062842 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca700b3260 0x7fca70063200 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 rx=0x7fca60002f30 tx=0x7fca60006dc0).ready entity=mon.1 client_cookie=b8035ab76866e97c server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:45.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.303+0000 7fca67fff700 1 -- 172.21.15.162:0/1697062842 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca600097f0 con 0x7fca700b3260 2020-11-28T21:55:45.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.303+0000 7fca67fff700 1 -- 172.21.15.162:0/1697062842 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca60009950 con 0x7fca700b3260 2020-11-28T21:55:45.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.303+0000 7fca67fff700 1 -- 172.21.15.162:0/1697062842 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca60009cb0 con 0x7fca700b3260 2020-11-28T21:55:45.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.303+0000 7fca7741b700 1 -- 172.21.15.162:0/1697062842 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca700b3260 msgr2=0x7fca70063200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:45.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.303+0000 7fca7741b700 1 --2- 172.21.15.162:0/1697062842 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca700b3260 0x7fca70063200 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 rx=0x7fca60002f30 tx=0x7fca60006dc0).stop 2020-11-28T21:55:45.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.303+0000 7fca7741b700 1 -- 172.21.15.162:0/1697062842 shutdown_connections 2020-11-28T21:55:45.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.303+0000 7fca7741b700 1 --2- 172.21.15.162:0/1697062842 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca700b3260 0x7fca70063200 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:45.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.303+0000 7fca7741b700 1 -- 172.21.15.162:0/1697062842 >> 172.21.15.162:0/1697062842 conn(0x7fca700568a0 msgr2=0x7fca700b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:45.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.303+0000 7fca7741b700 1 -- 172.21.15.162:0/1697062842 shutdown_connections 2020-11-28T21:55:45.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.303+0000 7fca7741b700 1 -- 172.21.15.162:0/1697062842 wait complete. 2020-11-28T21:55:45.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.307+0000 7fca7741b700 1 Processor -- start 2020-11-28T21:55:45.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.307+0000 7fca7741b700 1 -- start start 2020-11-28T21:55:45.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.307+0000 7fca7741b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fca700b3260 0x7fca7005ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:45.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.307+0000 7fca7741b700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fca700b7c50 0x7fca70065590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:45.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.307+0000 7fca7741b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca70140840 0x7fca700b53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:45.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.307+0000 7fca7741b700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca70060f30 con 0x7fca700b3260 2020-11-28T21:55:45.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.307+0000 7fca7741b700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7005f380 con 0x7fca70140840 2020-11-28T21:55:45.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.307+0000 7fca7741b700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca7005f5b0 con 0x7fca700b7c50 2020-11-28T21:55:45.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.307+0000 7fca759b8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca70140840 0x7fca700b53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:45.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.307+0000 7fca759b8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca70140840 0x7fca700b53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45286/0 (socket says 172.21.15.162:45286) 2020-11-28T21:55:45.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.307+0000 7fca759b8700 1 -- 172.21.15.162:0/2583358883 learned_addr learned my addr 172.21.15.162:0/2583358883 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:45.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.307+0000 7fca749b6700 1 --2- 172.21.15.162:0/2583358883 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fca700b7c50 0x7fca70065590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:45.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.307+0000 7fca751b7700 1 --2- 172.21.15.162:0/2583358883 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fca700b3260 0x7fca7005ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:45.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.307+0000 7fca759b8700 1 -- 172.21.15.162:0/2583358883 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fca700b7c50 msgr2=0x7fca70065590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:45.318 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.307+0000 7fca759b8700 1 --2- 172.21.15.162:0/2583358883 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fca700b7c50 0x7fca70065590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:45.318 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.307+0000 7fca759b8700 1 -- 172.21.15.162:0/2583358883 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fca700b3260 msgr2=0x7fca7005ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:45.318 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.307+0000 7fca759b8700 1 --2- 172.21.15.162:0/2583358883 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fca700b3260 0x7fca7005ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:45.318 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.307+0000 7fca759b8700 1 -- 172.21.15.162:0/2583358883 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca700b6930 con 0x7fca70140840 2020-11-28T21:55:45.318 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.307+0000 7fca759b8700 1 --2- 172.21.15.162:0/2583358883 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca70140840 0x7fca700b53e0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 rx=0x7fca68000f00 tx=0x7fca68007f70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:45.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.307+0000 7fca667fc700 1 -- 172.21.15.162:0/2583358883 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca68005b70 con 0x7fca70140840 2020-11-28T21:55:45.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.307+0000 7fca667fc700 1 -- 172.21.15.162:0/2583358883 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca68005cd0 con 0x7fca70140840 2020-11-28T21:55:45.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.307+0000 7fca7741b700 1 -- 172.21.15.162:0/2583358883 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca700b6bd0 con 0x7fca70140840 2020-11-28T21:55:45.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.307+0000 7fca667fc700 1 -- 172.21.15.162:0/2583358883 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca68007300 con 0x7fca70140840 2020-11-28T21:55:45.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.307+0000 7fca7741b700 1 -- 172.21.15.162:0/2583358883 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca700660a0 con 0x7fca70140840 2020-11-28T21:55:45.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.311+0000 7fca7741b700 1 -- 172.21.15.162:0/2583358883 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca700b6780 con 0x7fca70140840 2020-11-28T21:55:45.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.311+0000 7fca667fc700 1 -- 172.21.15.162:0/2583358883 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fca68010030 con 0x7fca70140840 2020-11-28T21:55:45.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.311+0000 7fca667fc700 1 --2- 172.21.15.162:0/2583358883 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fca58055330 0x7fca580577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:45.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.311+0000 7fca667fc700 1 -- 172.21.15.162:0/2583358883 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fca6806dd90 con 0x7fca70140840 2020-11-28T21:55:45.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.311+0000 7fca751b7700 1 --2- 172.21.15.162:0/2583358883 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fca58055330 0x7fca580577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:45.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.311+0000 7fca751b7700 1 --2- 172.21.15.162:0/2583358883 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fca58055330 0x7fca580577f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7fca6c00d000 tx=0x7fca6c000b80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:45.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.319+0000 7fca667fc700 1 -- 172.21.15.162:0/2583358883 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fca68044090 con 0x7fca70140840 2020-11-28T21:55:45.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.503+0000 7fca7741b700 1 -- 172.21.15.162:0/2583358883 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca70057050 con 0x7fca70140840 2020-11-28T21:55:45.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.507+0000 7fca667fc700 1 -- 172.21.15.162:0/2583358883 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+2504 (secure 0 0 0) 0x7fca680476b0 con 0x7fca70140840 2020-11-28T21:55:45.510 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:45.510 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":68,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7611,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6835/2127102267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2127102267},{"type":"v1","addr":"172.21.15.162:6835","nonce":2127102267}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":66},{"gid":7623,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/1473328171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1473328171},{"type":"v1","addr":"172.21.15.118:6833","nonce":1473328171}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:55:31.379864+0000","modified":"2020-11-28T21:55:33.042143+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7888":{"gid":7888,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/2667777645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2667777645},{"type":"v1","addr":"172.21.15.162:6837","nonce":2667777645}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2020-11-28T21:55:45.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.507+0000 7fca7741b700 1 -- 172.21.15.162:0/2583358883 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fca58055330 msgr2=0x7fca580577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:45.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.507+0000 7fca7741b700 1 --2- 172.21.15.162:0/2583358883 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fca58055330 0x7fca580577f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7fca6c00d000 tx=0x7fca6c000b80).stop 2020-11-28T21:55:45.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.507+0000 7fca7741b700 1 -- 172.21.15.162:0/2583358883 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca70140840 msgr2=0x7fca700b53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:45.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.507+0000 7fca7741b700 1 --2- 172.21.15.162:0/2583358883 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca70140840 0x7fca700b53e0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 rx=0x7fca68000f00 tx=0x7fca68007f70).stop 2020-11-28T21:55:45.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.507+0000 7fca7741b700 1 -- 172.21.15.162:0/2583358883 shutdown_connections 2020-11-28T21:55:45.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.507+0000 7fca7741b700 1 --2- 172.21.15.162:0/2583358883 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fca58055330 0x7fca580577f0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:45.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.507+0000 7fca7741b700 1 --2- 172.21.15.162:0/2583358883 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca70140840 0x7fca700b53e0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:45.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.507+0000 7fca7741b700 1 --2- 172.21.15.162:0/2583358883 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fca700b3260 0x7fca7005ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:45.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.507+0000 7fca7741b700 1 --2- 172.21.15.162:0/2583358883 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fca700b7c50 0x7fca70065590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:45.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.507+0000 7fca7741b700 1 -- 172.21.15.162:0/2583358883 >> 172.21.15.162:0/2583358883 conn(0x7fca700568a0 msgr2=0x7fca7013fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:45.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.507+0000 7fca7741b700 1 -- 172.21.15.162:0/2583358883 shutdown_connections 2020-11-28T21:55:45.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:45.507+0000 7fca7741b700 1 -- 172.21.15.162:0/2583358883 wait complete. 2020-11-28T21:55:45.514 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 68 2020-11-28T21:55:45.826 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-28T21:55:31.379864+0000', 'modified': '2020-11-28T21:55:33.042143+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7888}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7888': {'gid': 7888, 'name': 'c', 'rank': 0, 'incarnation': 66, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6837/2667777645', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6836', 'nonce': 2667777645}, {'type': 'v1', 'addr': '172.21.15.162:6837', 'nonce': 2667777645}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-28T21:55:45.827 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-28T21:55:45.827 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-28T21:55:45.957 INFO:teuthology.orchestra.run.smithi162.stdout:{"cluster_fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","whoami":0,"id":7888,"want_state":"up:active","state":"up:active","rank_uptime":14.563596965,"mdsmap_epoch":68,"osdmap_epoch":138,"osdmap_epoch_barrier":138,"uptime":18.831221573000001} 2020-11-28T21:55:45.958 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", "id": 7888, "mdsmap_epoch": 68, "osdmap_epoch": 138, "osdmap_epoch_barrier": 138, "rank_uptime": 14.563596965, "state": "up:active", "uptime": 18.831221573, "want_state": "up:active", "whoami": 0 } 2020-11-28T21:55:45.958 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-11-28T21:55:45.959 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2020-11-28T21:55:45.959 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi096.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2020-11-28T21:55:45.959 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:45.960 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:55:45.968 INFO:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2020-11-28T21:55:46.027 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:55:46.028 INFO:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:55:46.083 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:55:46.084 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:55:46.084 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:46.085 INFO:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2020-11-28T21:55:46.138 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-11-28T21:55:46.138 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:55:46.183 INFO:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2020-11-28T21:55:46.198 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:55:46.199 INFO:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:55:46.254 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:55:46.256 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:55:46.256 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:46.257 INFO:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2020-11-28T21:55:46.261 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:2020-11-28T21:55:46.256+0000 7f8dfc4960c0 -1 init, newargv = 0x55804904e000 newargc=9ceph-fuse[ 2020-11-28T21:55:46.262 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:12086]: starting ceph client 2020-11-28T21:55:46.272 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[12086]: starting fuse 2020-11-28T21:55:47.367 INFO:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2020-11-28T21:55:47.383 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:55:47.384 INFO:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:55:47.439 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:55:47.440 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:55:47.441 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:47.441 INFO:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2020-11-28T21:55:47.491 INFO:teuthology.orchestra.run.smithi096.stdout:49 2020-11-28T21:55:47.491 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2020-11-28T21:55:47.492 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:47.492 INFO:teuthology.orchestra.run.smithi096:> sudo python3 -c ' 2020-11-28T21:55:47.492 INFO:teuthology.orchestra.run.smithi096:> import glob 2020-11-28T21:55:47.493 INFO:teuthology.orchestra.run.smithi096:> import re 2020-11-28T21:55:47.493 INFO:teuthology.orchestra.run.smithi096:> import os 2020-11-28T21:55:47.494 INFO:teuthology.orchestra.run.smithi096:> import subprocess 2020-11-28T21:55:47.494 INFO:teuthology.orchestra.run.smithi096:> 2020-11-28T21:55:47.495 INFO:teuthology.orchestra.run.smithi096:> def find_socket(client_name): 2020-11-28T21:55:47.495 INFO:teuthology.orchestra.run.smithi096:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2020-11-28T21:55:47.496 INFO:teuthology.orchestra.run.smithi096:> files = glob.glob(asok_path) 2020-11-28T21:55:47.496 INFO:teuthology.orchestra.run.smithi096:> 2020-11-28T21:55:47.496 INFO:teuthology.orchestra.run.smithi096:> # Given a non-glob path, it better be there 2020-11-28T21:55:47.497 INFO:teuthology.orchestra.run.smithi096:> if "*" not in asok_path: 2020-11-28T21:55:47.497 INFO:teuthology.orchestra.run.smithi096:> assert(len(files) == 1) 2020-11-28T21:55:47.498 INFO:teuthology.orchestra.run.smithi096:> return files[0] 2020-11-28T21:55:47.498 INFO:teuthology.orchestra.run.smithi096:> 2020-11-28T21:55:47.499 INFO:teuthology.orchestra.run.smithi096:> for f in files: 2020-11-28T21:55:47.499 INFO:teuthology.orchestra.run.smithi096:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-11-28T21:55:47.500 INFO:teuthology.orchestra.run.smithi096:> if os.path.exists("/proc/{0}".format(pid)): 2020-11-28T21:55:47.500 INFO:teuthology.orchestra.run.smithi096:> return f 2020-11-28T21:55:47.501 INFO:teuthology.orchestra.run.smithi096:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-11-28T21:55:47.501 INFO:teuthology.orchestra.run.smithi096:> 2020-11-28T21:55:47.502 INFO:teuthology.orchestra.run.smithi096:> print(find_socket("client.1")) 2020-11-28T21:55:47.502 INFO:teuthology.orchestra.run.smithi096:> ' 2020-11-28T21:55:47.580 INFO:teuthology.orchestra.run.smithi096.stdout:/var/run/ceph/ceph-client.1.12086.asok 2020-11-28T21:55:47.585 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.12086.asok 2020-11-28T21:55:47.586 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:47.586 INFO:teuthology.orchestra.run.smithi096:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.12086.asok status 2020-11-28T21:55:47.685 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-11-28T21:55:47.685 INFO:teuthology.orchestra.run.smithi096.stdout: "metadata": { 2020-11-28T21:55:47.685 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_sha1": "a83a3572a957356977342f461361c1cd3ec06dbd", 2020-11-28T21:55:47.685 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_version": "ceph version 15.2.6-595-ga83a3572 (a83a3572a957356977342f461361c1cd3ec06dbd) octopus (stable)", 2020-11-28T21:55:47.686 INFO:teuthology.orchestra.run.smithi096.stdout: "entity_id": "1", 2020-11-28T21:55:47.686 INFO:teuthology.orchestra.run.smithi096.stdout: "hostname": "smithi096", 2020-11-28T21:55:47.686 INFO:teuthology.orchestra.run.smithi096.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2020-11-28T21:55:47.686 INFO:teuthology.orchestra.run.smithi096.stdout: "pid": "12086", 2020-11-28T21:55:47.686 INFO:teuthology.orchestra.run.smithi096.stdout: "root": "/" 2020-11-28T21:55:47.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-11-28T21:55:47.687 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_count": 0, 2020-11-28T21:55:47.687 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_pinned_count": 0, 2020-11-28T21:55:47.687 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 8161, 2020-11-28T21:55:47.687 INFO:teuthology.orchestra.run.smithi096.stdout: "inst": { 2020-11-28T21:55:47.688 INFO:teuthology.orchestra.run.smithi096.stdout: "name": { 2020-11-28T21:55:47.688 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "client", 2020-11-28T21:55:47.688 INFO:teuthology.orchestra.run.smithi096.stdout: "num": 8161 2020-11-28T21:55:47.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-11-28T21:55:47.688 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2020-11-28T21:55:47.689 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2020-11-28T21:55:47.689 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2020-11-28T21:55:47.689 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3495313937 2020-11-28T21:55:47.689 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-11-28T21:55:47.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-11-28T21:55:47.690 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2020-11-28T21:55:47.690 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2020-11-28T21:55:47.690 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2020-11-28T21:55:47.690 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3495313937 2020-11-28T21:55:47.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-11-28T21:55:47.691 INFO:teuthology.orchestra.run.smithi096.stdout: "inst_str": "client.8161 172.21.15.96:0/3495313937", 2020-11-28T21:55:47.691 INFO:teuthology.orchestra.run.smithi096.stdout: "addr_str": "172.21.15.96:0/3495313937", 2020-11-28T21:55:47.691 INFO:teuthology.orchestra.run.smithi096.stdout: "inode_count": 1, 2020-11-28T21:55:47.691 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_epoch": 68, 2020-11-28T21:55:47.691 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch": 138, 2020-11-28T21:55:47.692 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch_barrier": 0, 2020-11-28T21:55:47.692 INFO:teuthology.orchestra.run.smithi096.stdout: "blacklisted": false 2020-11-28T21:55:47.692 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-11-28T21:55:47.693 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:47.694 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:55:47.694 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:55:47.701 INFO:teuthology.orchestra.run.smithi096.stdout:fuseblk 2020-11-28T21:55:47.702 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-11-28T21:55:47.702 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:47.703 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:55:47.766 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:47.767 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:55:47.767 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:55:47.775 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2020-11-28T21:55:47.776 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-28T21:55:47.777 INFO:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-28T21:55:47.777 INFO:teuthology.orchestra.run.smithi192:> import time 2020-11-28T21:55:47.778 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:55:47.778 INFO:teuthology.orchestra.run.smithi192:> with open("/home/ubuntu/cephtest/mnt.0/testfile", '"'"'w'"'"') as f: 2020-11-28T21:55:47.778 INFO:teuthology.orchestra.run.smithi192:> f.write('"'"'content'"'"') 2020-11-28T21:55:47.779 INFO:teuthology.orchestra.run.smithi192:> f.flush() 2020-11-28T21:55:47.779 INFO:teuthology.orchestra.run.smithi192:> f.write('"'"'content2'"'"') 2020-11-28T21:55:47.780 INFO:teuthology.orchestra.run.smithi192:> while True: 2020-11-28T21:55:47.781 INFO:teuthology.orchestra.run.smithi192:> time.sleep(1) 2020-11-28T21:55:47.781 INFO:teuthology.orchestra.run.smithi192:> ' 2020-11-28T21:55:47.821 INFO:teuthology.orchestra.run.smithi192:> sudo ls /home/ubuntu/cephtest/mnt.0/testfile 2020-11-28T21:55:47.844 INFO:teuthology.orchestra.run.smithi192.stderr:ls: cannot access '/home/ubuntu/cephtest/mnt.0/testfile': No such file or directory 2020-11-28T21:55:47.845 DEBUG:teuthology.orchestra.run:got remote process result: 2 2020-11-28T21:55:48.847 INFO:teuthology.orchestra.run.smithi192:> sudo ls /home/ubuntu/cephtest/mnt.0/testfile 2020-11-28T21:55:48.862 INFO:teuthology.orchestra.run.smithi192.stdout:/home/ubuntu/cephtest/mnt.0/testfile 2020-11-28T21:55:48.866 DEBUG:tasks.cephfs.mount:File testfile became visible from 0 after 1s 2020-11-28T21:55:48.867 INFO:teuthology.orchestra.run.smithi096:> sudo ls /home/ubuntu/cephtest/mnt.1/testfile 2020-11-28T21:55:48.912 INFO:teuthology.orchestra.run.smithi096.stdout:/home/ubuntu/cephtest/mnt.1/testfile 2020-11-28T21:55:48.913 DEBUG:tasks.cephfs.mount:File testfile became visible from 1 after 0s 2020-11-28T21:55:48.914 INFO:teuthology.orchestra.run.smithi096:> cd /home/ubuntu/cephtest/mnt.1 && sudo cat testfile 2020-11-28T21:55:49.008 INFO:teuthology.orchestra.run.smithi096.stdout:content 2020-11-28T21:55:49.024 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-28T21:55:49.024 INFO:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:55:49.042 INFO:teuthology.orchestra.run.smithi118:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:55:49.060 INFO:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:55:49.078 INFO:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:55:49.097 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-28T21:55:49.098 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:49.098 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:55:49.098 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:55:49.147 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2020-11-28T21:55:49.148 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-28T21:55:49.149 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2020-11-28T21:55:49.149 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-28T21:55:49.149 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:55:49.228 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:ceph-fuse[12612]: fuse finished with error 0 and tester_r 0 2020-11-28T21:55:49.230 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:49.231 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:55:49.231 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:55:49.278 INFO:teuthology.orchestra.run.smithi192.stdout:ext2/ext3 2020-11-28T21:55:49.279 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-11-28T21:55:49.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-28T21:55:49.296 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:55:49.346 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:49.347 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:55:49.348 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:55:49.355 INFO:teuthology.orchestra.run.smithi096.stdout:fuseblk 2020-11-28T21:55:49.356 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-11-28T21:55:49.357 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi096.front.sepia.ceph.com... 2020-11-28T21:55:49.357 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-28T21:55:49.358 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:55:49.425 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[12086]: fuse finished with error 0 and tester_r 0 2020-11-28T21:55:49.427 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:49.427 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:55:49.428 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:55:49.502 INFO:teuthology.orchestra.run.smithi096.stdout:ext2/ext3 2020-11-28T21:55:49.503 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-11-28T21:55:49.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-28T21:55:49.504 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:55:49.552 INFO:teuthology.orchestra.run.smithi096:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-28T21:55:49.595 INFO:teuthology.orchestra.run.smithi118:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-28T21:55:49.598 INFO:teuthology.orchestra.run.smithi162:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-28T21:55:49.602 INFO:teuthology.orchestra.run.smithi192:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-28T21:55:49.625 INFO:teuthology.orchestra.run.smithi096.stdout:[global] 2020-11-28T21:55:49.626 INFO:teuthology.orchestra.run.smithi096.stdout: chdir = "" 2020-11-28T21:55:49.626 INFO:teuthology.orchestra.run.smithi096.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-28T21:55:49.626 INFO:teuthology.orchestra.run.smithi096.stdout: auth supported = cephx 2020-11-28T21:55:49.626 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:49.627 INFO:teuthology.orchestra.run.smithi096.stdout: filestore xattr use omap = true 2020-11-28T21:55:49.627 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:49.627 INFO:teuthology.orchestra.run.smithi096.stdout: mon clock drift allowed = 1.000 2020-11-28T21:55:49.627 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:49.627 INFO:teuthology.orchestra.run.smithi096.stdout: osd crush chooseleaf type = 0 2020-11-28T21:55:49.628 INFO:teuthology.orchestra.run.smithi096.stdout: auth debug = true 2020-11-28T21:55:49.628 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:49.628 INFO:teuthology.orchestra.run.smithi096.stdout: ms die on old message = true 2020-11-28T21:55:49.628 INFO:teuthology.orchestra.run.smithi096.stdout: ms die on bug = true 2020-11-28T21:55:49.628 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:49.629 INFO:teuthology.orchestra.run.smithi096.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-28T21:55:49.629 INFO:teuthology.orchestra.run.smithi096.stdout: mon pg warn max object skew = 0 2020-11-28T21:55:49.629 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:49.629 INFO:teuthology.orchestra.run.smithi096.stdout: # disable pg_autoscaler by default for new pools 2020-11-28T21:55:49.630 INFO:teuthology.orchestra.run.smithi096.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-28T21:55:49.630 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:49.630 INFO:teuthology.orchestra.run.smithi096.stdout: osd pool default size = 2 2020-11-28T21:55:49.630 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:49.630 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd allow primary affinity = true 2020-11-28T21:55:49.631 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd allow pg remap = true 2020-11-28T21:55:49.631 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on legacy crush tunables = false 2020-11-28T21:55:49.631 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on crush straw calc version zero = false 2020-11-28T21:55:49.631 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on no sortbitwise = false 2020-11-28T21:55:49.632 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on osd down out interval zero = false 2020-11-28T21:55:49.632 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on too few osds = false 2020-11-28T21:55:49.632 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-28T21:55:49.632 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-28T21:55:49.632 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:49.633 INFO:teuthology.orchestra.run.smithi096.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-28T21:55:49.633 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:49.633 INFO:teuthology.orchestra.run.smithi096.stdout: osd default data pool replay window = 5 2020-11-28T21:55:49.633 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:49.633 INFO:teuthology.orchestra.run.smithi096.stdout: mon allow pool delete = true 2020-11-28T21:55:49.634 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:49.634 INFO:teuthology.orchestra.run.smithi096.stdout: mon cluster log file level = debug 2020-11-28T21:55:49.634 INFO:teuthology.orchestra.run.smithi096.stdout: debug asserts on shutdown = true 2020-11-28T21:55:49.634 INFO:teuthology.orchestra.run.smithi096.stdout: mon host = "172.21.15.162,172.21.15.118,[v2:172.21.15.118:3301,v1:172.21.15.118:6790]" 2020-11-28T21:55:49.634 INFO:teuthology.orchestra.run.smithi096.stdout: fsid = 7c321bcf-cd33-40f7-a39f-da5470b386cf 2020-11-28T21:55:49.635 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:49.635 INFO:teuthology.orchestra.run.smithi096.stdout:[osd] 2020-11-28T21:55:49.635 INFO:teuthology.orchestra.run.smithi096.stdout: osd journal size = 100 2020-11-28T21:55:49.635 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:49.635 INFO:teuthology.orchestra.run.smithi096.stdout: osd scrub load threshold = 5.0 2020-11-28T21:55:49.636 INFO:teuthology.orchestra.run.smithi096.stdout: osd scrub max interval = 600 2020-11-28T21:55:49.636 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:49.636 INFO:teuthology.orchestra.run.smithi096.stdout: osd recover clone overlap = true 2020-11-28T21:55:49.636 INFO:teuthology.orchestra.run.smithi096.stdout: osd recovery max chunk = 1048576 2020-11-28T21:55:49.636 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:49.637 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug shutdown = true 2020-11-28T21:55:49.637 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug op order = true 2020-11-28T21:55:49.637 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug verify stray on activate = true 2020-11-28T21:55:49.637 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:49.637 INFO:teuthology.orchestra.run.smithi096.stdout: osd open classes on start = true 2020-11-28T21:55:49.638 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug pg log writeout = true 2020-11-28T21:55:49.638 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:49.638 INFO:teuthology.orchestra.run.smithi096.stdout: osd deep scrub update digest min age = 30 2020-11-28T21:55:49.638 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:49.639 INFO:teuthology.orchestra.run.smithi096.stdout: osd map max advance = 10 2020-11-28T21:55:49.639 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:49.639 INFO:teuthology.orchestra.run.smithi096.stdout: journal zero on create = true 2020-11-28T21:55:49.639 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:49.639 INFO:teuthology.orchestra.run.smithi096.stdout: filestore ondisk finisher threads = 3 2020-11-28T21:55:49.640 INFO:teuthology.orchestra.run.smithi096.stdout: filestore apply finisher threads = 3 2020-11-28T21:55:49.640 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:49.640 INFO:teuthology.orchestra.run.smithi096.stdout: bdev debug aio = true 2020-11-28T21:55:49.640 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug misdirected ops = true 2020-11-28T21:55:49.640 INFO:teuthology.orchestra.run.smithi096.stdout: bluestore block size = 96636764160 2020-11-28T21:55:49.641 INFO:teuthology.orchestra.run.smithi096.stdout: bluestore compression mode = aggressive 2020-11-28T21:55:49.641 INFO:teuthology.orchestra.run.smithi096.stdout: bluestore fsck on mount = True 2020-11-28T21:55:49.641 INFO:teuthology.orchestra.run.smithi096.stdout: debug bluefs = 20 2020-11-28T21:55:49.641 INFO:teuthology.orchestra.run.smithi096.stdout: debug bluestore = 20 2020-11-28T21:55:49.641 INFO:teuthology.orchestra.run.smithi096.stdout: debug filestore = 20 2020-11-28T21:55:49.642 INFO:teuthology.orchestra.run.smithi096.stdout: debug journal = 20 2020-11-28T21:55:49.642 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 20 2020-11-28T21:55:49.642 INFO:teuthology.orchestra.run.smithi096.stdout: debug osd = 25 2020-11-28T21:55:49.642 INFO:teuthology.orchestra.run.smithi096.stdout: debug rocksdb = 10 2020-11-28T21:55:49.642 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd backfillfull_ratio = 0.85 2020-11-28T21:55:49.643 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd full ratio = 0.9 2020-11-28T21:55:49.643 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd nearfull ratio = 0.8 2020-11-28T21:55:49.643 INFO:teuthology.orchestra.run.smithi096.stdout: osd failsafe full ratio = 0.95 2020-11-28T21:55:49.643 INFO:teuthology.orchestra.run.smithi096.stdout: osd objectstore = bluestore 2020-11-28T21:55:49.643 INFO:teuthology.orchestra.run.smithi096.stdout: osd op complaint time = 180 2020-11-28T21:55:49.644 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:49.644 INFO:teuthology.orchestra.run.smithi096.stdout:[mgr] 2020-11-28T21:55:49.644 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2020-11-28T21:55:49.644 INFO:teuthology.orchestra.run.smithi096.stdout: debug mgr = 20 2020-11-28T21:55:49.645 INFO:teuthology.orchestra.run.smithi096.stdout: debug mon = 20 2020-11-28T21:55:49.645 INFO:teuthology.orchestra.run.smithi096.stdout: debug auth = 20 2020-11-28T21:55:49.645 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min pgs per osd = 4 2020-11-28T21:55:49.645 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min bytes per osd = 10 2020-11-28T21:55:49.645 INFO:teuthology.orchestra.run.smithi096.stdout: mgr/telemetry/nag = false 2020-11-28T21:55:49.646 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:49.646 INFO:teuthology.orchestra.run.smithi096.stdout:[mon] 2020-11-28T21:55:49.646 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2020-11-28T21:55:49.646 INFO:teuthology.orchestra.run.smithi096.stdout: debug mon = 20 2020-11-28T21:55:49.646 INFO:teuthology.orchestra.run.smithi096.stdout: debug paxos = 20 2020-11-28T21:55:49.647 INFO:teuthology.orchestra.run.smithi096.stdout: debug auth = 20 2020-11-28T21:55:49.647 INFO:teuthology.orchestra.run.smithi096.stdout: mon data avail warn = 5 2020-11-28T21:55:49.647 INFO:teuthology.orchestra.run.smithi096.stdout: mon mgr mkfs grace = 240 2020-11-28T21:55:49.647 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min pgs per osd = 4 2020-11-28T21:55:49.647 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd reporter subtree level = osd 2020-11-28T21:55:49.648 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd prime pg temp = true 2020-11-28T21:55:49.648 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min bytes per osd = 10 2020-11-28T21:55:49.648 INFO:teuthology.orchestra.run.smithi096.stdout: mon op complaint time = 120 2020-11-28T21:55:49.648 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-28T21:55:49.648 INFO:teuthology.orchestra.run.smithi096.stdout:[client] 2020-11-28T21:55:49.649 INFO:teuthology.orchestra.run.smithi096.stdout: rgw cache enabled = true 2020-11-28T21:55:49.649 INFO:teuthology.orchestra.run.smithi096.stdout: rgw enable ops log = true 2020-11-28T21:55:49.649 INFO:teuthology.orchestra.run.smithi096.stdout: rgw enable usage log = true 2020-11-28T21:55:49.649 INFO:teuthology.orchestra.run.smithi096.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-28T21:55:49.649 INFO:teuthology.orchestra.run.smithi096.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-28T21:55:49.650 INFO:teuthology.orchestra.run.smithi096.stdout: client mount timeout = 600 2020-11-28T21:55:49.650 INFO:teuthology.orchestra.run.smithi096.stdout: debug client = 20 2020-11-28T21:55:49.650 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2020-11-28T21:55:49.650 INFO:teuthology.orchestra.run.smithi096.stdout: rados mon op timeout = 15m 2020-11-28T21:55:49.651 INFO:teuthology.orchestra.run.smithi096.stdout: rados osd op timeout = 15m 2020-11-28T21:55:49.651 INFO:teuthology.orchestra.run.smithi096.stdout:[mon.a] 2020-11-28T21:55:49.651 INFO:teuthology.orchestra.run.smithi096.stdout:[mon.b] 2020-11-28T21:55:49.651 INFO:teuthology.orchestra.run.smithi096.stdout:[mon.c] 2020-11-28T21:55:49.651 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.a] 2020-11-28T21:55:49.652 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.c] 2020-11-28T21:55:49.652 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.b] 2020-11-28T21:55:49.652 INFO:teuthology.orchestra.run.smithi096.stdout:[mds] 2020-11-28T21:55:49.652 INFO:teuthology.orchestra.run.smithi096.stdout: debug mds = 20 2020-11-28T21:55:49.652 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2020-11-28T21:55:49.653 INFO:teuthology.orchestra.run.smithi096.stdout: mds bal frag = True 2020-11-28T21:55:49.653 INFO:teuthology.orchestra.run.smithi096.stdout: mds bal fragment size max = 10000 2020-11-28T21:55:49.653 INFO:teuthology.orchestra.run.smithi096.stdout: mds bal merge size = 5 2020-11-28T21:55:49.653 INFO:teuthology.orchestra.run.smithi096.stdout: mds bal split bits = 3 2020-11-28T21:55:49.653 INFO:teuthology.orchestra.run.smithi096.stdout: mds bal split size = 100 2020-11-28T21:55:49.654 INFO:teuthology.orchestra.run.smithi096.stdout: mds debug frag = True 2020-11-28T21:55:49.654 INFO:teuthology.orchestra.run.smithi096.stdout: mds debug scatterstat = True 2020-11-28T21:55:49.654 INFO:teuthology.orchestra.run.smithi096.stdout: mds op complaint time = 180 2020-11-28T21:55:49.654 INFO:teuthology.orchestra.run.smithi096.stdout: mds verify scatter = True 2020-11-28T21:55:49.654 INFO:teuthology.orchestra.run.smithi096.stdout: osd op complaint time = 180 2020-11-28T21:55:49.655 INFO:teuthology.orchestra.run.smithi096.stdout: rados mon op timeout = 15m 2020-11-28T21:55:49.655 INFO:teuthology.orchestra.run.smithi096.stdout: rados osd op timeout = 15m 2020-11-28T21:55:49.657 INFO:teuthology.orchestra.run.smithi192.stdout:[global] 2020-11-28T21:55:49.658 INFO:teuthology.orchestra.run.smithi192.stdout: chdir = "" 2020-11-28T21:55:49.658 INFO:teuthology.orchestra.run.smithi192.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-28T21:55:49.658 INFO:teuthology.orchestra.run.smithi192.stdout: auth supported = cephx 2020-11-28T21:55:49.658 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:49.659 INFO:teuthology.orchestra.run.smithi192.stdout: filestore xattr use omap = true 2020-11-28T21:55:49.659 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:49.659 INFO:teuthology.orchestra.run.smithi192.stdout: mon clock drift allowed = 1.000 2020-11-28T21:55:49.659 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:49.660 INFO:teuthology.orchestra.run.smithi192.stdout: osd crush chooseleaf type = 0 2020-11-28T21:55:49.660 INFO:teuthology.orchestra.run.smithi192.stdout: auth debug = true 2020-11-28T21:55:49.660 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:49.660 INFO:teuthology.orchestra.run.smithi192.stdout: ms die on old message = true 2020-11-28T21:55:49.660 INFO:teuthology.orchestra.run.smithi192.stdout: ms die on bug = true 2020-11-28T21:55:49.661 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:49.661 INFO:teuthology.orchestra.run.smithi192.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-28T21:55:49.661 INFO:teuthology.orchestra.run.smithi192.stdout: mon pg warn max object skew = 0 2020-11-28T21:55:49.661 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:49.662 INFO:teuthology.orchestra.run.smithi192.stdout: # disable pg_autoscaler by default for new pools 2020-11-28T21:55:49.662 INFO:teuthology.orchestra.run.smithi192.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-28T21:55:49.662 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:49.662 INFO:teuthology.orchestra.run.smithi192.stdout: osd pool default size = 2 2020-11-28T21:55:49.662 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:49.663 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd allow primary affinity = true 2020-11-28T21:55:49.663 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd allow pg remap = true 2020-11-28T21:55:49.663 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on legacy crush tunables = false 2020-11-28T21:55:49.663 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on crush straw calc version zero = false 2020-11-28T21:55:49.663 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on no sortbitwise = false 2020-11-28T21:55:49.664 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on osd down out interval zero = false 2020-11-28T21:55:49.664 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on too few osds = false 2020-11-28T21:55:49.664 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-28T21:55:49.664 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-28T21:55:49.664 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:49.665 INFO:teuthology.orchestra.run.smithi192.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-28T21:55:49.665 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:49.665 INFO:teuthology.orchestra.run.smithi192.stdout: osd default data pool replay window = 5 2020-11-28T21:55:49.665 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:49.665 INFO:teuthology.orchestra.run.smithi192.stdout: mon allow pool delete = true 2020-11-28T21:55:49.666 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:49.666 INFO:teuthology.orchestra.run.smithi192.stdout: mon cluster log file level = debug 2020-11-28T21:55:49.666 INFO:teuthology.orchestra.run.smithi192.stdout: debug asserts on shutdown = true 2020-11-28T21:55:49.666 INFO:teuthology.orchestra.run.smithi192.stdout: mon host = "172.21.15.162,172.21.15.118,[v2:172.21.15.118:3301,v1:172.21.15.118:6790]" 2020-11-28T21:55:49.667 INFO:teuthology.orchestra.run.smithi192.stdout: fsid = 7c321bcf-cd33-40f7-a39f-da5470b386cf 2020-11-28T21:55:49.667 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:49.667 INFO:teuthology.orchestra.run.smithi192.stdout:[osd] 2020-11-28T21:55:49.667 INFO:teuthology.orchestra.run.smithi192.stdout: osd journal size = 100 2020-11-28T21:55:49.667 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:49.668 INFO:teuthology.orchestra.run.smithi192.stdout: osd scrub load threshold = 5.0 2020-11-28T21:55:49.668 INFO:teuthology.orchestra.run.smithi192.stdout: osd scrub max interval = 600 2020-11-28T21:55:49.668 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:49.668 INFO:teuthology.orchestra.run.smithi192.stdout: osd recover clone overlap = true 2020-11-28T21:55:49.668 INFO:teuthology.orchestra.run.smithi192.stdout: osd recovery max chunk = 1048576 2020-11-28T21:55:49.669 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:49.669 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug shutdown = true 2020-11-28T21:55:49.669 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug op order = true 2020-11-28T21:55:49.669 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug verify stray on activate = true 2020-11-28T21:55:49.669 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:49.670 INFO:teuthology.orchestra.run.smithi192.stdout: osd open classes on start = true 2020-11-28T21:55:49.670 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug pg log writeout = true 2020-11-28T21:55:49.670 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:49.670 INFO:teuthology.orchestra.run.smithi192.stdout: osd deep scrub update digest min age = 30 2020-11-28T21:55:49.670 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:49.671 INFO:teuthology.orchestra.run.smithi192.stdout: osd map max advance = 10 2020-11-28T21:55:49.671 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:49.671 INFO:teuthology.orchestra.run.smithi192.stdout: journal zero on create = true 2020-11-28T21:55:49.671 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:49.671 INFO:teuthology.orchestra.run.smithi192.stdout: filestore ondisk finisher threads = 3 2020-11-28T21:55:49.672 INFO:teuthology.orchestra.run.smithi192.stdout: filestore apply finisher threads = 3 2020-11-28T21:55:49.672 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:49.672 INFO:teuthology.orchestra.run.smithi192.stdout: bdev debug aio = true 2020-11-28T21:55:49.672 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug misdirected ops = true 2020-11-28T21:55:49.673 INFO:teuthology.orchestra.run.smithi192.stdout: bluestore block size = 96636764160 2020-11-28T21:55:49.673 INFO:teuthology.orchestra.run.smithi192.stdout: bluestore compression mode = aggressive 2020-11-28T21:55:49.673 INFO:teuthology.orchestra.run.smithi192.stdout: bluestore fsck on mount = True 2020-11-28T21:55:49.673 INFO:teuthology.orchestra.run.smithi192.stdout: debug bluefs = 20 2020-11-28T21:55:49.674 INFO:teuthology.orchestra.run.smithi192.stdout: debug bluestore = 20 2020-11-28T21:55:49.674 INFO:teuthology.orchestra.run.smithi192.stdout: debug filestore = 20 2020-11-28T21:55:49.674 INFO:teuthology.orchestra.run.smithi192.stdout: debug journal = 20 2020-11-28T21:55:49.674 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 20 2020-11-28T21:55:49.675 INFO:teuthology.orchestra.run.smithi192.stdout: debug osd = 25 2020-11-28T21:55:49.675 INFO:teuthology.orchestra.run.smithi192.stdout: debug rocksdb = 10 2020-11-28T21:55:49.675 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd backfillfull_ratio = 0.85 2020-11-28T21:55:49.675 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd full ratio = 0.9 2020-11-28T21:55:49.675 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd nearfull ratio = 0.8 2020-11-28T21:55:49.676 INFO:teuthology.orchestra.run.smithi192.stdout: osd failsafe full ratio = 0.95 2020-11-28T21:55:49.676 INFO:teuthology.orchestra.run.smithi192.stdout: osd objectstore = bluestore 2020-11-28T21:55:49.676 INFO:teuthology.orchestra.run.smithi192.stdout: osd op complaint time = 180 2020-11-28T21:55:49.676 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:49.676 INFO:teuthology.orchestra.run.smithi192.stdout:[mgr] 2020-11-28T21:55:49.677 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2020-11-28T21:55:49.677 INFO:teuthology.orchestra.run.smithi192.stdout: debug mgr = 20 2020-11-28T21:55:49.677 INFO:teuthology.orchestra.run.smithi192.stdout: debug mon = 20 2020-11-28T21:55:49.677 INFO:teuthology.orchestra.run.smithi192.stdout: debug auth = 20 2020-11-28T21:55:49.677 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min pgs per osd = 4 2020-11-28T21:55:49.678 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min bytes per osd = 10 2020-11-28T21:55:49.678 INFO:teuthology.orchestra.run.smithi192.stdout: mgr/telemetry/nag = false 2020-11-28T21:55:49.678 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:49.678 INFO:teuthology.orchestra.run.smithi192.stdout:[mon] 2020-11-28T21:55:49.678 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2020-11-28T21:55:49.679 INFO:teuthology.orchestra.run.smithi192.stdout: debug mon = 20 2020-11-28T21:55:49.679 INFO:teuthology.orchestra.run.smithi192.stdout: debug paxos = 20 2020-11-28T21:55:49.679 INFO:teuthology.orchestra.run.smithi192.stdout: debug auth = 20 2020-11-28T21:55:49.679 INFO:teuthology.orchestra.run.smithi192.stdout: mon data avail warn = 5 2020-11-28T21:55:49.679 INFO:teuthology.orchestra.run.smithi192.stdout: mon mgr mkfs grace = 240 2020-11-28T21:55:49.680 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min pgs per osd = 4 2020-11-28T21:55:49.680 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd reporter subtree level = osd 2020-11-28T21:55:49.680 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd prime pg temp = true 2020-11-28T21:55:49.680 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min bytes per osd = 10 2020-11-28T21:55:49.680 INFO:teuthology.orchestra.run.smithi192.stdout: mon op complaint time = 120 2020-11-28T21:55:49.681 INFO:teuthology.orchestra.run.smithi192.stdout: 2020-11-28T21:55:49.681 INFO:teuthology.orchestra.run.smithi192.stdout:[client] 2020-11-28T21:55:49.681 INFO:teuthology.orchestra.run.smithi192.stdout: rgw cache enabled = true 2020-11-28T21:55:49.681 INFO:teuthology.orchestra.run.smithi192.stdout: rgw enable ops log = true 2020-11-28T21:55:49.681 INFO:teuthology.orchestra.run.smithi192.stdout: rgw enable usage log = true 2020-11-28T21:55:49.682 INFO:teuthology.orchestra.run.smithi192.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-28T21:55:49.682 INFO:teuthology.orchestra.run.smithi192.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-28T21:55:49.682 INFO:teuthology.orchestra.run.smithi192.stdout: client mount timeout = 600 2020-11-28T21:55:49.682 INFO:teuthology.orchestra.run.smithi192.stdout: debug client = 20 2020-11-28T21:55:49.682 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2020-11-28T21:55:49.683 INFO:teuthology.orchestra.run.smithi192.stdout: rados mon op timeout = 15m 2020-11-28T21:55:49.683 INFO:teuthology.orchestra.run.smithi192.stdout: rados osd op timeout = 15m 2020-11-28T21:55:49.683 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.a] 2020-11-28T21:55:49.683 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.b] 2020-11-28T21:55:49.684 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.c] 2020-11-28T21:55:49.684 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.a] 2020-11-28T21:55:49.684 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.c] 2020-11-28T21:55:49.684 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.b] 2020-11-28T21:55:49.684 INFO:teuthology.orchestra.run.smithi192.stdout:[mds] 2020-11-28T21:55:49.685 INFO:teuthology.orchestra.run.smithi192.stdout: debug mds = 20 2020-11-28T21:55:49.685 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2020-11-28T21:55:49.685 INFO:teuthology.orchestra.run.smithi192.stdout: mds bal frag = True 2020-11-28T21:55:49.685 INFO:teuthology.orchestra.run.smithi192.stdout: mds bal fragment size max = 10000 2020-11-28T21:55:49.685 INFO:teuthology.orchestra.run.smithi192.stdout: mds bal merge size = 5 2020-11-28T21:55:49.686 INFO:teuthology.orchestra.run.smithi192.stdout: mds bal split bits = 3 2020-11-28T21:55:49.686 INFO:teuthology.orchestra.run.smithi192.stdout: mds bal split size = 100 2020-11-28T21:55:49.686 INFO:teuthology.orchestra.run.smithi192.stdout: mds debug frag = True 2020-11-28T21:55:49.686 INFO:teuthology.orchestra.run.smithi192.stdout: mds debug scatterstat = True 2020-11-28T21:55:49.686 INFO:teuthology.orchestra.run.smithi192.stdout: mds op complaint time = 180 2020-11-28T21:55:49.687 INFO:teuthology.orchestra.run.smithi192.stdout: mds verify scatter = True 2020-11-28T21:55:49.687 INFO:teuthology.orchestra.run.smithi192.stdout: osd op complaint time = 180 2020-11-28T21:55:49.687 INFO:teuthology.orchestra.run.smithi192.stdout: rados mon op timeout = 15m 2020-11-28T21:55:49.687 INFO:teuthology.orchestra.run.smithi192.stdout: rados osd op timeout = 15m 2020-11-28T21:55:49.687 INFO:teuthology.orchestra.run.smithi162.stdout:[global] 2020-11-28T21:55:49.688 INFO:teuthology.orchestra.run.smithi162.stdout: chdir = "" 2020-11-28T21:55:49.688 INFO:teuthology.orchestra.run.smithi162.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-28T21:55:49.688 INFO:teuthology.orchestra.run.smithi162.stdout: auth supported = cephx 2020-11-28T21:55:49.688 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:49.688 INFO:teuthology.orchestra.run.smithi162.stdout: filestore xattr use omap = true 2020-11-28T21:55:49.689 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:49.689 INFO:teuthology.orchestra.run.smithi162.stdout: mon clock drift allowed = 1.000 2020-11-28T21:55:49.689 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:49.689 INFO:teuthology.orchestra.run.smithi162.stdout: osd crush chooseleaf type = 0 2020-11-28T21:55:49.689 INFO:teuthology.orchestra.run.smithi162.stdout: auth debug = true 2020-11-28T21:55:49.690 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:49.690 INFO:teuthology.orchestra.run.smithi162.stdout: ms die on old message = true 2020-11-28T21:55:49.690 INFO:teuthology.orchestra.run.smithi162.stdout: ms die on bug = true 2020-11-28T21:55:49.690 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:49.691 INFO:teuthology.orchestra.run.smithi162.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-28T21:55:49.691 INFO:teuthology.orchestra.run.smithi162.stdout: mon pg warn max object skew = 0 2020-11-28T21:55:49.691 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:49.691 INFO:teuthology.orchestra.run.smithi162.stdout: # disable pg_autoscaler by default for new pools 2020-11-28T21:55:49.691 INFO:teuthology.orchestra.run.smithi162.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-28T21:55:49.692 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:49.692 INFO:teuthology.orchestra.run.smithi162.stdout: osd pool default size = 2 2020-11-28T21:55:49.692 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:49.692 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd allow primary affinity = true 2020-11-28T21:55:49.692 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd allow pg remap = true 2020-11-28T21:55:49.693 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on legacy crush tunables = false 2020-11-28T21:55:49.693 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on crush straw calc version zero = false 2020-11-28T21:55:49.693 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on no sortbitwise = false 2020-11-28T21:55:49.693 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on osd down out interval zero = false 2020-11-28T21:55:49.693 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on too few osds = false 2020-11-28T21:55:49.694 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-28T21:55:49.694 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-28T21:55:49.694 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:49.694 INFO:teuthology.orchestra.run.smithi162.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-28T21:55:49.694 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:49.695 INFO:teuthology.orchestra.run.smithi162.stdout: osd default data pool replay window = 5 2020-11-28T21:55:49.695 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:49.695 INFO:teuthology.orchestra.run.smithi162.stdout: mon allow pool delete = true 2020-11-28T21:55:49.695 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:49.695 INFO:teuthology.orchestra.run.smithi162.stdout: mon cluster log file level = debug 2020-11-28T21:55:49.696 INFO:teuthology.orchestra.run.smithi162.stdout: debug asserts on shutdown = true 2020-11-28T21:55:49.696 INFO:teuthology.orchestra.run.smithi162.stdout: mon host = "172.21.15.162,172.21.15.118,[v2:172.21.15.118:3301,v1:172.21.15.118:6790]" 2020-11-28T21:55:49.696 INFO:teuthology.orchestra.run.smithi162.stdout: fsid = 7c321bcf-cd33-40f7-a39f-da5470b386cf 2020-11-28T21:55:49.696 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:49.696 INFO:teuthology.orchestra.run.smithi162.stdout:[osd] 2020-11-28T21:55:49.697 INFO:teuthology.orchestra.run.smithi162.stdout: osd journal size = 100 2020-11-28T21:55:49.697 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:49.697 INFO:teuthology.orchestra.run.smithi162.stdout: osd scrub load threshold = 5.0 2020-11-28T21:55:49.697 INFO:teuthology.orchestra.run.smithi162.stdout: osd scrub max interval = 600 2020-11-28T21:55:49.697 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:49.698 INFO:teuthology.orchestra.run.smithi162.stdout: osd recover clone overlap = true 2020-11-28T21:55:49.698 INFO:teuthology.orchestra.run.smithi162.stdout: osd recovery max chunk = 1048576 2020-11-28T21:55:49.698 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:49.698 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug shutdown = true 2020-11-28T21:55:49.698 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug op order = true 2020-11-28T21:55:49.699 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug verify stray on activate = true 2020-11-28T21:55:49.699 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:49.699 INFO:teuthology.orchestra.run.smithi162.stdout: osd open classes on start = true 2020-11-28T21:55:49.699 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug pg log writeout = true 2020-11-28T21:55:49.699 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:49.700 INFO:teuthology.orchestra.run.smithi162.stdout: osd deep scrub update digest min age = 30 2020-11-28T21:55:49.700 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:49.700 INFO:teuthology.orchestra.run.smithi162.stdout: osd map max advance = 10 2020-11-28T21:55:49.700 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:49.700 INFO:teuthology.orchestra.run.smithi162.stdout: journal zero on create = true 2020-11-28T21:55:49.701 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:49.701 INFO:teuthology.orchestra.run.smithi162.stdout: filestore ondisk finisher threads = 3 2020-11-28T21:55:49.701 INFO:teuthology.orchestra.run.smithi162.stdout: filestore apply finisher threads = 3 2020-11-28T21:55:49.701 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:49.701 INFO:teuthology.orchestra.run.smithi162.stdout: bdev debug aio = true 2020-11-28T21:55:49.702 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug misdirected ops = true 2020-11-28T21:55:49.702 INFO:teuthology.orchestra.run.smithi162.stdout: bluestore block size = 96636764160 2020-11-28T21:55:49.702 INFO:teuthology.orchestra.run.smithi162.stdout: bluestore compression mode = aggressive 2020-11-28T21:55:49.702 INFO:teuthology.orchestra.run.smithi162.stdout: bluestore fsck on mount = True 2020-11-28T21:55:49.702 INFO:teuthology.orchestra.run.smithi162.stdout: debug bluefs = 20 2020-11-28T21:55:49.703 INFO:teuthology.orchestra.run.smithi162.stdout: debug bluestore = 20 2020-11-28T21:55:49.703 INFO:teuthology.orchestra.run.smithi162.stdout: debug filestore = 20 2020-11-28T21:55:49.703 INFO:teuthology.orchestra.run.smithi162.stdout: debug journal = 20 2020-11-28T21:55:49.703 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 20 2020-11-28T21:55:49.703 INFO:teuthology.orchestra.run.smithi162.stdout: debug osd = 25 2020-11-28T21:55:49.704 INFO:teuthology.orchestra.run.smithi162.stdout: debug rocksdb = 10 2020-11-28T21:55:49.704 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd backfillfull_ratio = 0.85 2020-11-28T21:55:49.704 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd full ratio = 0.9 2020-11-28T21:55:49.704 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd nearfull ratio = 0.8 2020-11-28T21:55:49.704 INFO:teuthology.orchestra.run.smithi162.stdout: osd failsafe full ratio = 0.95 2020-11-28T21:55:49.705 INFO:teuthology.orchestra.run.smithi162.stdout: osd objectstore = bluestore 2020-11-28T21:55:49.705 INFO:teuthology.orchestra.run.smithi162.stdout: osd op complaint time = 180 2020-11-28T21:55:49.705 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:49.705 INFO:teuthology.orchestra.run.smithi162.stdout:[mgr] 2020-11-28T21:55:49.705 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2020-11-28T21:55:49.706 INFO:teuthology.orchestra.run.smithi162.stdout: debug mgr = 20 2020-11-28T21:55:49.706 INFO:teuthology.orchestra.run.smithi162.stdout: debug mon = 20 2020-11-28T21:55:49.706 INFO:teuthology.orchestra.run.smithi162.stdout: debug auth = 20 2020-11-28T21:55:49.706 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min pgs per osd = 4 2020-11-28T21:55:49.706 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min bytes per osd = 10 2020-11-28T21:55:49.707 INFO:teuthology.orchestra.run.smithi162.stdout: mgr/telemetry/nag = false 2020-11-28T21:55:49.707 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:49.707 INFO:teuthology.orchestra.run.smithi162.stdout:[mon] 2020-11-28T21:55:49.707 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2020-11-28T21:55:49.707 INFO:teuthology.orchestra.run.smithi162.stdout: debug mon = 20 2020-11-28T21:55:49.708 INFO:teuthology.orchestra.run.smithi162.stdout: debug paxos = 20 2020-11-28T21:55:49.708 INFO:teuthology.orchestra.run.smithi162.stdout: debug auth = 20 2020-11-28T21:55:49.708 INFO:teuthology.orchestra.run.smithi162.stdout: mon data avail warn = 5 2020-11-28T21:55:49.708 INFO:teuthology.orchestra.run.smithi162.stdout: mon mgr mkfs grace = 240 2020-11-28T21:55:49.708 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min pgs per osd = 4 2020-11-28T21:55:49.709 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd reporter subtree level = osd 2020-11-28T21:55:49.709 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd prime pg temp = true 2020-11-28T21:55:49.709 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min bytes per osd = 10 2020-11-28T21:55:49.709 INFO:teuthology.orchestra.run.smithi162.stdout: mon op complaint time = 120 2020-11-28T21:55:49.709 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:49.709 INFO:teuthology.orchestra.run.smithi162.stdout:[client] 2020-11-28T21:55:49.710 INFO:teuthology.orchestra.run.smithi162.stdout: rgw cache enabled = true 2020-11-28T21:55:49.710 INFO:teuthology.orchestra.run.smithi162.stdout: rgw enable ops log = true 2020-11-28T21:55:49.710 INFO:teuthology.orchestra.run.smithi162.stdout: rgw enable usage log = true 2020-11-28T21:55:49.710 INFO:teuthology.orchestra.run.smithi162.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-28T21:55:49.710 INFO:teuthology.orchestra.run.smithi162.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-28T21:55:49.711 INFO:teuthology.orchestra.run.smithi162.stdout: client mount timeout = 600 2020-11-28T21:55:49.711 INFO:teuthology.orchestra.run.smithi162.stdout: debug client = 20 2020-11-28T21:55:49.711 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2020-11-28T21:55:49.711 INFO:teuthology.orchestra.run.smithi162.stdout: rados mon op timeout = 15m 2020-11-28T21:55:49.711 INFO:teuthology.orchestra.run.smithi162.stdout: rados osd op timeout = 15m 2020-11-28T21:55:49.712 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.a] 2020-11-28T21:55:49.712 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.b] 2020-11-28T21:55:49.712 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.c] 2020-11-28T21:55:49.712 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.a] 2020-11-28T21:55:49.712 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.c] 2020-11-28T21:55:49.713 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.b] 2020-11-28T21:55:49.713 INFO:teuthology.orchestra.run.smithi162.stdout:[mds] 2020-11-28T21:55:49.713 INFO:teuthology.orchestra.run.smithi162.stdout: debug mds = 20 2020-11-28T21:55:49.713 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2020-11-28T21:55:49.713 INFO:teuthology.orchestra.run.smithi162.stdout: mds bal frag = True 2020-11-28T21:55:49.714 INFO:teuthology.orchestra.run.smithi162.stdout: mds bal fragment size max = 10000 2020-11-28T21:55:49.714 INFO:teuthology.orchestra.run.smithi162.stdout: mds bal merge size = 5 2020-11-28T21:55:49.714 INFO:teuthology.orchestra.run.smithi162.stdout: mds bal split bits = 3 2020-11-28T21:55:49.714 INFO:teuthology.orchestra.run.smithi162.stdout: mds bal split size = 100 2020-11-28T21:55:49.714 INFO:teuthology.orchestra.run.smithi162.stdout: mds debug frag = True 2020-11-28T21:55:49.715 INFO:teuthology.orchestra.run.smithi162.stdout: mds debug scatterstat = True 2020-11-28T21:55:49.715 INFO:teuthology.orchestra.run.smithi162.stdout: mds op complaint time = 180 2020-11-28T21:55:49.715 INFO:teuthology.orchestra.run.smithi162.stdout: mds verify scatter = True 2020-11-28T21:55:49.716 INFO:teuthology.orchestra.run.smithi162.stdout: osd op complaint time = 180 2020-11-28T21:55:49.716 INFO:teuthology.orchestra.run.smithi162.stdout: rados mon op timeout = 15m 2020-11-28T21:55:49.716 INFO:teuthology.orchestra.run.smithi162.stdout: rados osd op timeout = 15m 2020-11-28T21:55:49.716 INFO:teuthology.orchestra.run.smithi118.stdout:[global] 2020-11-28T21:55:49.716 INFO:teuthology.orchestra.run.smithi118.stdout: chdir = "" 2020-11-28T21:55:49.717 INFO:teuthology.orchestra.run.smithi118.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-28T21:55:49.717 INFO:teuthology.orchestra.run.smithi118.stdout: auth supported = cephx 2020-11-28T21:55:49.717 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:49.717 INFO:teuthology.orchestra.run.smithi118.stdout: filestore xattr use omap = true 2020-11-28T21:55:49.718 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:49.718 INFO:teuthology.orchestra.run.smithi118.stdout: mon clock drift allowed = 1.000 2020-11-28T21:55:49.718 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:49.718 INFO:teuthology.orchestra.run.smithi118.stdout: osd crush chooseleaf type = 0 2020-11-28T21:55:49.718 INFO:teuthology.orchestra.run.smithi118.stdout: auth debug = true 2020-11-28T21:55:49.719 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:49.719 INFO:teuthology.orchestra.run.smithi118.stdout: ms die on old message = true 2020-11-28T21:55:49.719 INFO:teuthology.orchestra.run.smithi118.stdout: ms die on bug = true 2020-11-28T21:55:49.719 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:49.719 INFO:teuthology.orchestra.run.smithi118.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-28T21:55:49.720 INFO:teuthology.orchestra.run.smithi118.stdout: mon pg warn max object skew = 0 2020-11-28T21:55:49.720 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:49.720 INFO:teuthology.orchestra.run.smithi118.stdout: # disable pg_autoscaler by default for new pools 2020-11-28T21:55:49.720 INFO:teuthology.orchestra.run.smithi118.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-28T21:55:49.720 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:49.721 INFO:teuthology.orchestra.run.smithi118.stdout: osd pool default size = 2 2020-11-28T21:55:49.721 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:49.721 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd allow primary affinity = true 2020-11-28T21:55:49.721 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd allow pg remap = true 2020-11-28T21:55:49.721 INFO:teuthology.orchestra.run.smithi118.stdout: mon warn on legacy crush tunables = false 2020-11-28T21:55:49.722 INFO:teuthology.orchestra.run.smithi118.stdout: mon warn on crush straw calc version zero = false 2020-11-28T21:55:49.722 INFO:teuthology.orchestra.run.smithi118.stdout: mon warn on no sortbitwise = false 2020-11-28T21:55:49.722 INFO:teuthology.orchestra.run.smithi118.stdout: mon warn on osd down out interval zero = false 2020-11-28T21:55:49.722 INFO:teuthology.orchestra.run.smithi118.stdout: mon warn on too few osds = false 2020-11-28T21:55:49.722 INFO:teuthology.orchestra.run.smithi118.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-28T21:55:49.723 INFO:teuthology.orchestra.run.smithi118.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-28T21:55:49.723 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:49.723 INFO:teuthology.orchestra.run.smithi118.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-28T21:55:49.723 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:49.723 INFO:teuthology.orchestra.run.smithi118.stdout: osd default data pool replay window = 5 2020-11-28T21:55:49.724 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:49.724 INFO:teuthology.orchestra.run.smithi118.stdout: mon allow pool delete = true 2020-11-28T21:55:49.724 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:49.724 INFO:teuthology.orchestra.run.smithi118.stdout: mon cluster log file level = debug 2020-11-28T21:55:49.724 INFO:teuthology.orchestra.run.smithi118.stdout: debug asserts on shutdown = true 2020-11-28T21:55:49.725 INFO:teuthology.orchestra.run.smithi118.stdout: mon host = "172.21.15.162,172.21.15.118,[v2:172.21.15.118:3301,v1:172.21.15.118:6790]" 2020-11-28T21:55:49.725 INFO:teuthology.orchestra.run.smithi118.stdout: fsid = 7c321bcf-cd33-40f7-a39f-da5470b386cf 2020-11-28T21:55:49.725 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:49.726 INFO:teuthology.orchestra.run.smithi118.stdout:[osd] 2020-11-28T21:55:49.726 INFO:teuthology.orchestra.run.smithi118.stdout: osd journal size = 100 2020-11-28T21:55:49.726 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:49.726 INFO:teuthology.orchestra.run.smithi118.stdout: osd scrub load threshold = 5.0 2020-11-28T21:55:49.726 INFO:teuthology.orchestra.run.smithi118.stdout: osd scrub max interval = 600 2020-11-28T21:55:49.727 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:49.727 INFO:teuthology.orchestra.run.smithi118.stdout: osd recover clone overlap = true 2020-11-28T21:55:49.727 INFO:teuthology.orchestra.run.smithi118.stdout: osd recovery max chunk = 1048576 2020-11-28T21:55:49.727 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:49.727 INFO:teuthology.orchestra.run.smithi118.stdout: osd debug shutdown = true 2020-11-28T21:55:49.728 INFO:teuthology.orchestra.run.smithi118.stdout: osd debug op order = true 2020-11-28T21:55:49.728 INFO:teuthology.orchestra.run.smithi118.stdout: osd debug verify stray on activate = true 2020-11-28T21:55:49.728 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:49.728 INFO:teuthology.orchestra.run.smithi118.stdout: osd open classes on start = true 2020-11-28T21:55:49.728 INFO:teuthology.orchestra.run.smithi118.stdout: osd debug pg log writeout = true 2020-11-28T21:55:49.729 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:49.729 INFO:teuthology.orchestra.run.smithi118.stdout: osd deep scrub update digest min age = 30 2020-11-28T21:55:49.729 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:49.729 INFO:teuthology.orchestra.run.smithi118.stdout: osd map max advance = 10 2020-11-28T21:55:49.729 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:49.730 INFO:teuthology.orchestra.run.smithi118.stdout: journal zero on create = true 2020-11-28T21:55:49.730 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:49.730 INFO:teuthology.orchestra.run.smithi118.stdout: filestore ondisk finisher threads = 3 2020-11-28T21:55:49.730 INFO:teuthology.orchestra.run.smithi118.stdout: filestore apply finisher threads = 3 2020-11-28T21:55:49.730 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:49.731 INFO:teuthology.orchestra.run.smithi118.stdout: bdev debug aio = true 2020-11-28T21:55:49.731 INFO:teuthology.orchestra.run.smithi118.stdout: osd debug misdirected ops = true 2020-11-28T21:55:49.731 INFO:teuthology.orchestra.run.smithi118.stdout: bluestore block size = 96636764160 2020-11-28T21:55:49.731 INFO:teuthology.orchestra.run.smithi118.stdout: bluestore compression mode = aggressive 2020-11-28T21:55:49.731 INFO:teuthology.orchestra.run.smithi118.stdout: bluestore fsck on mount = True 2020-11-28T21:55:49.732 INFO:teuthology.orchestra.run.smithi118.stdout: debug bluefs = 20 2020-11-28T21:55:49.732 INFO:teuthology.orchestra.run.smithi118.stdout: debug bluestore = 20 2020-11-28T21:55:49.732 INFO:teuthology.orchestra.run.smithi118.stdout: debug filestore = 20 2020-11-28T21:55:49.732 INFO:teuthology.orchestra.run.smithi118.stdout: debug journal = 20 2020-11-28T21:55:49.732 INFO:teuthology.orchestra.run.smithi118.stdout: debug ms = 20 2020-11-28T21:55:49.733 INFO:teuthology.orchestra.run.smithi118.stdout: debug osd = 25 2020-11-28T21:55:49.733 INFO:teuthology.orchestra.run.smithi118.stdout: debug rocksdb = 10 2020-11-28T21:55:49.733 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd backfillfull_ratio = 0.85 2020-11-28T21:55:49.733 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd full ratio = 0.9 2020-11-28T21:55:49.733 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd nearfull ratio = 0.8 2020-11-28T21:55:49.734 INFO:teuthology.orchestra.run.smithi118.stdout: osd failsafe full ratio = 0.95 2020-11-28T21:55:49.734 INFO:teuthology.orchestra.run.smithi118.stdout: osd objectstore = bluestore 2020-11-28T21:55:49.734 INFO:teuthology.orchestra.run.smithi118.stdout: osd op complaint time = 180 2020-11-28T21:55:49.734 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:49.734 INFO:teuthology.orchestra.run.smithi118.stdout:[mgr] 2020-11-28T21:55:49.735 INFO:teuthology.orchestra.run.smithi118.stdout: debug ms = 1 2020-11-28T21:55:49.735 INFO:teuthology.orchestra.run.smithi118.stdout: debug mgr = 20 2020-11-28T21:55:49.735 INFO:teuthology.orchestra.run.smithi118.stdout: debug mon = 20 2020-11-28T21:55:49.735 INFO:teuthology.orchestra.run.smithi118.stdout: debug auth = 20 2020-11-28T21:55:49.735 INFO:teuthology.orchestra.run.smithi118.stdout: mon reweight min pgs per osd = 4 2020-11-28T21:55:49.736 INFO:teuthology.orchestra.run.smithi118.stdout: mon reweight min bytes per osd = 10 2020-11-28T21:55:49.736 INFO:teuthology.orchestra.run.smithi118.stdout: mgr/telemetry/nag = false 2020-11-28T21:55:49.736 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:49.736 INFO:teuthology.orchestra.run.smithi118.stdout:[mon] 2020-11-28T21:55:49.736 INFO:teuthology.orchestra.run.smithi118.stdout: debug ms = 1 2020-11-28T21:55:49.737 INFO:teuthology.orchestra.run.smithi118.stdout: debug mon = 20 2020-11-28T21:55:49.737 INFO:teuthology.orchestra.run.smithi118.stdout: debug paxos = 20 2020-11-28T21:55:49.737 INFO:teuthology.orchestra.run.smithi118.stdout: debug auth = 20 2020-11-28T21:55:49.737 INFO:teuthology.orchestra.run.smithi118.stdout: mon data avail warn = 5 2020-11-28T21:55:49.737 INFO:teuthology.orchestra.run.smithi118.stdout: mon mgr mkfs grace = 240 2020-11-28T21:55:49.738 INFO:teuthology.orchestra.run.smithi118.stdout: mon reweight min pgs per osd = 4 2020-11-28T21:55:49.738 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd reporter subtree level = osd 2020-11-28T21:55:49.738 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd prime pg temp = true 2020-11-28T21:55:49.738 INFO:teuthology.orchestra.run.smithi118.stdout: mon reweight min bytes per osd = 10 2020-11-28T21:55:49.738 INFO:teuthology.orchestra.run.smithi118.stdout: mon op complaint time = 120 2020-11-28T21:55:49.739 INFO:teuthology.orchestra.run.smithi118.stdout: 2020-11-28T21:55:49.739 INFO:teuthology.orchestra.run.smithi118.stdout:[client] 2020-11-28T21:55:49.739 INFO:teuthology.orchestra.run.smithi118.stdout: rgw cache enabled = true 2020-11-28T21:55:49.739 INFO:teuthology.orchestra.run.smithi118.stdout: rgw enable ops log = true 2020-11-28T21:55:49.739 INFO:teuthology.orchestra.run.smithi118.stdout: rgw enable usage log = true 2020-11-28T21:55:49.740 INFO:teuthology.orchestra.run.smithi118.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-28T21:55:49.740 INFO:teuthology.orchestra.run.smithi118.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-28T21:55:49.740 INFO:teuthology.orchestra.run.smithi118.stdout: client mount timeout = 600 2020-11-28T21:55:49.740 INFO:teuthology.orchestra.run.smithi118.stdout: debug client = 20 2020-11-28T21:55:49.740 INFO:teuthology.orchestra.run.smithi118.stdout: debug ms = 1 2020-11-28T21:55:49.741 INFO:teuthology.orchestra.run.smithi118.stdout: rados mon op timeout = 15m 2020-11-28T21:55:49.741 INFO:teuthology.orchestra.run.smithi118.stdout: rados osd op timeout = 15m 2020-11-28T21:55:49.741 INFO:teuthology.orchestra.run.smithi118.stdout:[mon.a] 2020-11-28T21:55:49.741 INFO:teuthology.orchestra.run.smithi118.stdout:[mon.b] 2020-11-28T21:55:49.741 INFO:teuthology.orchestra.run.smithi118.stdout:[mon.c] 2020-11-28T21:55:49.742 INFO:teuthology.orchestra.run.smithi118.stdout:[mds.a] 2020-11-28T21:55:49.742 INFO:teuthology.orchestra.run.smithi118.stdout:[mds.c] 2020-11-28T21:55:49.742 INFO:teuthology.orchestra.run.smithi118.stdout:[mds.b] 2020-11-28T21:55:49.742 INFO:teuthology.orchestra.run.smithi118.stdout:[mds] 2020-11-28T21:55:49.742 INFO:teuthology.orchestra.run.smithi118.stdout: debug mds = 20 2020-11-28T21:55:49.743 INFO:teuthology.orchestra.run.smithi118.stdout: debug ms = 1 2020-11-28T21:55:49.743 INFO:teuthology.orchestra.run.smithi118.stdout: mds bal frag = True 2020-11-28T21:55:49.743 INFO:teuthology.orchestra.run.smithi118.stdout: mds bal fragment size max = 10000 2020-11-28T21:55:49.743 INFO:teuthology.orchestra.run.smithi118.stdout: mds bal merge size = 5 2020-11-28T21:55:49.743 INFO:teuthology.orchestra.run.smithi118.stdout: mds bal split bits = 3 2020-11-28T21:55:49.744 INFO:teuthology.orchestra.run.smithi118.stdout: mds bal split size = 100 2020-11-28T21:55:49.744 INFO:teuthology.orchestra.run.smithi118.stdout: mds debug frag = True 2020-11-28T21:55:49.744 INFO:teuthology.orchestra.run.smithi118.stdout: mds debug scatterstat = True 2020-11-28T21:55:49.744 INFO:teuthology.orchestra.run.smithi118.stdout: mds op complaint time = 180 2020-11-28T21:55:49.744 INFO:teuthology.orchestra.run.smithi118.stdout: mds verify scatter = True 2020-11-28T21:55:49.745 INFO:teuthology.orchestra.run.smithi118.stdout: osd op complaint time = 180 2020-11-28T21:55:49.745 INFO:teuthology.orchestra.run.smithi118.stdout: rados mon op timeout = 15m 2020-11-28T21:55:49.745 INFO:teuthology.orchestra.run.smithi118.stdout: rados osd op timeout = 15m 2020-11-28T21:55:49.746 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config rm mon mon_allow_pool_delete 2020-11-28T21:55:49.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.862+0000 7f84f759e700 1 Processor -- start 2020-11-28T21:55:49.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.862+0000 7f84f759e700 1 -- start start 2020-11-28T21:55:49.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.862+0000 7f84f759e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84f8060360 0x7f84f805cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:49.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.862+0000 7f84f759e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f84f8060740 0x7f84f8142d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:49.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.862+0000 7f84f759e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84f805c270 con 0x7f84f80a8ec0 2020-11-28T21:55:49.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.862+0000 7f84f759e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84f80a9b30 con 0x7f84f8060740 2020-11-28T21:55:49.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.862+0000 7f84f759e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84f8060f40 con 0x7f84f8060360 2020-11-28T21:55:49.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.862+0000 7f84f659c700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84f8060360 0x7f84f805cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:49.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.862+0000 7f84f659c700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84f8060360 0x7f84f805cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:42514/0 (socket says 172.21.15.162:42514) 2020-11-28T21:55:49.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.862+0000 7f84f659c700 1 -- 172.21.15.162:0/2180216653 learned_addr learned my addr 172.21.15.162:0/2180216653 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:49.869 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.862+0000 7f84f6d9d700 1 --2- 172.21.15.162:0/2180216653 >> v2:172.21.15.118:3300/0 conn(0x7f84f8060740 0x7f84f8142d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:49.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.862+0000 7f84f659c700 1 -- 172.21.15.162:0/2180216653 >> v2:172.21.15.118:3300/0 conn(0x7f84f8060740 msgr2=0x7f84f8142d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:49.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.862+0000 7f84f659c700 1 --2- 172.21.15.162:0/2180216653 >> v2:172.21.15.118:3300/0 conn(0x7f84f8060740 0x7f84f8142d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:49.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.862+0000 7f84f659c700 1 -- 172.21.15.162:0/2180216653 >> v1:172.21.15.118:6789/0 conn(0x7f84f80a8ec0 legacy=0x7f84f80a9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:49.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.862+0000 7f84f659c700 1 -- 172.21.15.162:0/2180216653 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f805c500 con 0x7f84f8060360 2020-11-28T21:55:49.871 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.862+0000 7f84f659c700 1 --2- 172.21.15.162:0/2180216653 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84f8060360 0x7f84f805cc40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f84e8002f30 tx=0x7f84e8006e80).ready entity=mon.2 client_cookie=b1d22f90751b74dc server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:49.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.862+0000 7f84f559a700 1 -- 172.21.15.162:0/2180216653 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84e80097f0 con 0x7f84f8060360 2020-11-28T21:55:49.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.862+0000 7f84f559a700 1 -- 172.21.15.162:0/2180216653 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e8009950 con 0x7f84f8060360 2020-11-28T21:55:49.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.862+0000 7f84f559a700 1 -- 172.21.15.162:0/2180216653 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84e8009cb0 con 0x7f84f8060360 2020-11-28T21:55:49.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.862+0000 7f84f759e700 1 -- 172.21.15.162:0/2180216653 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84f8060360 msgr2=0x7f84f805cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:49.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.862+0000 7f84f759e700 1 --2- 172.21.15.162:0/2180216653 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84f8060360 0x7f84f805cc40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f84e8002f30 tx=0x7f84e8006e80).stop 2020-11-28T21:55:49.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.862+0000 7f84f759e700 1 -- 172.21.15.162:0/2180216653 shutdown_connections 2020-11-28T21:55:49.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.862+0000 7f84f759e700 1 --2- 172.21.15.162:0/2180216653 >> v2:172.21.15.118:3300/0 conn(0x7f84f8060740 0x7f84f8142d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:49.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.862+0000 7f84f759e700 1 --2- 172.21.15.162:0/2180216653 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84f8060360 0x7f84f805cc40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:49.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.862+0000 7f84f759e700 1 -- 172.21.15.162:0/2180216653 >> 172.21.15.162:0/2180216653 conn(0x7f84f80b5520 msgr2=0x7f84f80b5930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:49.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.866+0000 7f84f759e700 1 -- 172.21.15.162:0/2180216653 shutdown_connections 2020-11-28T21:55:49.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.866+0000 7f84f759e700 1 -- 172.21.15.162:0/2180216653 wait complete. 2020-11-28T21:55:49.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.866+0000 7f84f759e700 1 Processor -- start 2020-11-28T21:55:49.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.866+0000 7f84f759e700 1 -- start start 2020-11-28T21:55:49.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.866+0000 7f84f759e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84f8060360 0x7f84f8066860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:49.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.866+0000 7f84f759e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84f8060740 0x7f84f8098660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:49.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.866+0000 7f84f759e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f84f80a8ec0 0x7f84f8155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:49.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.866+0000 7f84f759e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f8098b50 con 0x7f84f80a8ec0 2020-11-28T21:55:49.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.866+0000 7f84f759e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f8099f60 con 0x7f84f8060360 2020-11-28T21:55:49.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.866+0000 7f84f759e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84f8155450 con 0x7f84f8060740 2020-11-28T21:55:49.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.866+0000 7f84f659c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84f8060360 0x7f84f8066860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:49.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.866+0000 7f84f659c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84f8060360 0x7f84f8066860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45296/0 (socket says 172.21.15.162:45296) 2020-11-28T21:55:49.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.866+0000 7f84f659c700 1 -- 172.21.15.162:0/453037715 learned_addr learned my addr 172.21.15.162:0/453037715 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:49.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.866+0000 7f84f6d9d700 1 --2- 172.21.15.162:0/453037715 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f84f80a8ec0 0x7f84f8155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:49.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.866+0000 7f84f5d9b700 1 --2- 172.21.15.162:0/453037715 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84f8060740 0x7f84f8098660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:49.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.866+0000 7f84f659c700 1 -- 172.21.15.162:0/453037715 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84f8060740 msgr2=0x7f84f8098660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:49.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.866+0000 7f84f659c700 1 --2- 172.21.15.162:0/453037715 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84f8060740 0x7f84f8098660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:49.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.866+0000 7f84f659c700 1 -- 172.21.15.162:0/453037715 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f84f80a8ec0 msgr2=0x7f84f8155b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:49.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.866+0000 7f84f659c700 1 --2- 172.21.15.162:0/453037715 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f84f80a8ec0 0x7f84f8155b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:49.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.866+0000 7f84f659c700 1 -- 172.21.15.162:0/453037715 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f81556f0 con 0x7f84f8060360 2020-11-28T21:55:49.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.866+0000 7f84f6d9d700 1 --2- 172.21.15.162:0/453037715 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f84f80a8ec0 0x7f84f8155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:55:49.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.866+0000 7f84f659c700 1 --2- 172.21.15.162:0/453037715 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84f8060360 0x7f84f8066860 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 rx=0x7f84e8002f30 tx=0x7f84e8015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:49.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.866+0000 7f84e77fe700 1 -- 172.21.15.162:0/453037715 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84e8011ca0 con 0x7f84f8060360 2020-11-28T21:55:49.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.866+0000 7f84e77fe700 1 -- 172.21.15.162:0/453037715 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e8011e00 con 0x7f84f8060360 2020-11-28T21:55:49.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.866+0000 7f84f759e700 1 -- 172.21.15.162:0/453037715 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84f8156080 con 0x7f84f8060360 2020-11-28T21:55:49.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.866+0000 7f84e77fe700 1 -- 172.21.15.162:0/453037715 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84e801e650 con 0x7f84f8060360 2020-11-28T21:55:49.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.866+0000 7f84f759e700 1 -- 172.21.15.162:0/453037715 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84f8066f80 con 0x7f84f8060360 2020-11-28T21:55:49.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.870+0000 7f84f759e700 1 -- 172.21.15.162:0/453037715 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84f81552a0 con 0x7f84f8060360 2020-11-28T21:55:49.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.870+0000 7f84e77fe700 1 -- 172.21.15.162:0/453037715 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f84e8025030 con 0x7f84f8060360 2020-11-28T21:55:49.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.870+0000 7f84e77fe700 1 --2- 172.21.15.162:0/453037715 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f84d4055330 0x7f84d40577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:49.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.870+0000 7f84e77fe700 1 -- 172.21.15.162:0/453037715 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f84e806f330 con 0x7f84f8060360 2020-11-28T21:55:49.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.870+0000 7f84f5d9b700 1 --2- 172.21.15.162:0/453037715 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f84d4055330 0x7f84d40577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:49.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.870+0000 7f84f5d9b700 1 --2- 172.21.15.162:0/453037715 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f84d4055330 0x7f84d40577f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7f84ec00b000 tx=0x7f84ec009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:49.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:49.878+0000 7f84e77fe700 1 -- 172.21.15.162:0/453037715 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f84e8045630 con 0x7f84f8060360 2020-11-28T21:55:50.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.022+0000 7f84f759e700 1 -- 172.21.15.162:0/453037715 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f84f8155880 con 0x7f84f8060360 2020-11-28T21:55:50.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.030+0000 7f84e77fe700 1 -- 172.21.15.162:0/453037715 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f84e8048c50 con 0x7f84f8060360 2020-11-28T21:55:50.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.034+0000 7f84f759e700 1 -- 172.21.15.162:0/453037715 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f84d4055330 msgr2=0x7f84d40577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:50.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.034+0000 7f84f759e700 1 --2- 172.21.15.162:0/453037715 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f84d4055330 0x7f84d40577f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7f84ec00b000 tx=0x7f84ec009040).stop 2020-11-28T21:55:50.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.034+0000 7f84f759e700 1 -- 172.21.15.162:0/453037715 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84f8060360 msgr2=0x7f84f8066860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:50.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.034+0000 7f84f759e700 1 --2- 172.21.15.162:0/453037715 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84f8060360 0x7f84f8066860 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 rx=0x7f84e8002f30 tx=0x7f84e8015000).stop 2020-11-28T21:55:50.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.034+0000 7f84f759e700 1 -- 172.21.15.162:0/453037715 shutdown_connections 2020-11-28T21:55:50.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.034+0000 7f84f759e700 1 --2- 172.21.15.162:0/453037715 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f84d4055330 0x7f84d40577f0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:50.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.034+0000 7f84f759e700 1 --2- 172.21.15.162:0/453037715 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f84f80a8ec0 0x7f84f8155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:50.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.034+0000 7f84f759e700 1 --2- 172.21.15.162:0/453037715 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84f8060360 0x7f84f8066860 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:50.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.034+0000 7f84f759e700 1 --2- 172.21.15.162:0/453037715 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84f8060740 0x7f84f8098660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:50.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.034+0000 7f84f759e700 1 -- 172.21.15.162:0/453037715 >> 172.21.15.162:0/453037715 conn(0x7f84f80b5520 msgr2=0x7f84f80aa620 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:50.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.034+0000 7f84f759e700 1 -- 172.21.15.162:0/453037715 shutdown_connections 2020-11-28T21:55:50.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.034+0000 7f84f759e700 1 -- 172.21.15.162:0/453037715 wait complete. 2020-11-28T21:55:50.056 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Ended test tasks.cephfs.test_misc.TestMisc.test_getattr_caps' 2020-11-28T21:55:50.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.170+0000 7fe845ae5700 1 Processor -- start 2020-11-28T21:55:50.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.170+0000 7fe845ae5700 1 -- start start 2020-11-28T21:55:50.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.170+0000 7fe845ae5700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe8400b5430 0x7fe840143360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:50.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.170+0000 7fe845ae5700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8400b8040 con 0x7fe8400b5a70 2020-11-28T21:55:50.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.170+0000 7fe845ae5700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8400b8720 con 0x7fe8400b5e50 2020-11-28T21:55:50.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.170+0000 7fe845ae5700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8400b7600 con 0x7fe8400b5430 2020-11-28T21:55:50.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.170+0000 7fe83ffff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fe8400b5e50 0x7fe84009d220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:52150/0 (socket says 172.21.15.162:52150) 2020-11-28T21:55:50.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.170+0000 7fe83ffff700 1 -- 172.21.15.162:0/1480628050 learned_addr learned my addr 172.21.15.162:0/1480628050 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:50.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.170+0000 7fe8452e4700 1 --2- 172.21.15.162:0/1480628050 >> v2:172.21.15.118:3300/0 conn(0x7fe8400b5430 0x7fe840143360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:50.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.170+0000 7fe83f7fe700 1 -- 172.21.15.162:0/1480628050 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2165832343 0 0) 0x7fe8400b8720 con 0x7fe8400b5e50 2020-11-28T21:55:50.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.170+0000 7fe83f7fe700 1 -- 172.21.15.162:0/1480628050 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe820003230 con 0x7fe8400b5e50 2020-11-28T21:55:50.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.170+0000 7fe83f7fe700 1 -- 172.21.15.162:0/1480628050 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1301862550 0 0) 0x7fe820003230 con 0x7fe8400b5e50 2020-11-28T21:55:50.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.170+0000 7fe83f7fe700 1 -- 172.21.15.162:0/1480628050 >> v2:172.21.15.118:3300/0 conn(0x7fe8400b5430 msgr2=0x7fe840143360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:50.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.170+0000 7fe83f7fe700 1 --2- 172.21.15.162:0/1480628050 >> v2:172.21.15.118:3300/0 conn(0x7fe8400b5430 0x7fe840143360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:50.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.170+0000 7fe83f7fe700 1 -- 172.21.15.162:0/1480628050 >> v1:172.21.15.118:6789/0 conn(0x7fe8400b5a70 legacy=0x7fe8400b6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:50.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.170+0000 7fe83f7fe700 1 -- 172.21.15.162:0/1480628050 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe840142980 con 0x7fe8400b5e50 2020-11-28T21:55:50.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.170+0000 7fe83f7fe700 1 -- 172.21.15.162:0/1480628050 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7fe8300016d0 con 0x7fe8400b5e50 2020-11-28T21:55:50.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.170+0000 7fe83f7fe700 1 -- 172.21.15.162:0/1480628050 >> v1:172.21.15.162:6789/0 conn(0x7fe8400b5e50 legacy=0x7fe84009d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:50.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.170+0000 7fe83f7fe700 1 --2- 172.21.15.162:0/1480628050 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe820003720 0x7fe820003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:50.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.170+0000 7fe83f7fe700 1 --2- 172.21.15.162:0/1480628050 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8200044d0 0x7fe820004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:50.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.170+0000 7fe83f7fe700 1 --2- 172.21.15.162:0/1480628050 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe820004e40 0x7fe82000b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:50.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.170+0000 7fe83f7fe700 1 -- 172.21.15.162:0/1480628050 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8400b7600 con 0x7fe820003720 2020-11-28T21:55:50.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.170+0000 7fe83f7fe700 1 -- 172.21.15.162:0/1480628050 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe820005350 con 0x7fe8200044d0 2020-11-28T21:55:50.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.170+0000 7fe83f7fe700 1 -- 172.21.15.162:0/1480628050 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe82000bc40 con 0x7fe820004e40 2020-11-28T21:55:50.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.170+0000 7fe83ffff700 1 --2- 172.21.15.162:0/1480628050 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8200044d0 0x7fe820004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:50.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.170+0000 7fe844ae3700 1 --2- 172.21.15.162:0/1480628050 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe820003720 0x7fe820003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:50.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.170+0000 7fe83ffff700 1 -- 172.21.15.162:0/1480628050 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe820004e40 msgr2=0x7fe82000b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:50.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.170+0000 7fe83ffff700 1 --2- 172.21.15.162:0/1480628050 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe820004e40 0x7fe82000b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-28T21:55:50.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.170+0000 7fe83ffff700 1 -- 172.21.15.162:0/1480628050 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe820003720 msgr2=0x7fe820003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:50.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.170+0000 7fe83ffff700 1 --2- 172.21.15.162:0/1480628050 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe820003720 0x7fe820003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:50.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.170+0000 7fe83ffff700 1 -- 172.21.15.162:0/1480628050 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe820003230 con 0x7fe8200044d0 2020-11-28T21:55:50.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.170+0000 7fe8452e4700 1 -- 172.21.15.162:0/1480628050 reap_dead start 2020-11-28T21:55:50.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.170+0000 7fe83ffff700 1 --2- 172.21.15.162:0/1480628050 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8200044d0 0x7fe820004950 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 rx=0x7fe8300013e0 tx=0x7fe840142b10).ready entity=mon.1 client_cookie=6498195968f18e27 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:50.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.170+0000 7fe83f7fe700 1 -- 172.21.15.162:0/1480628050 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe83000ce00 con 0x7fe8200044d0 2020-11-28T21:55:50.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.170+0000 7fe83f7fe700 1 -- 172.21.15.162:0/1480628050 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8300072e0 con 0x7fe8200044d0 2020-11-28T21:55:50.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.170+0000 7fe83f7fe700 1 -- 172.21.15.162:0/1480628050 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe8300057d0 con 0x7fe8200044d0 2020-11-28T21:55:50.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.170+0000 7fe845ae5700 1 -- 172.21.15.162:0/1480628050 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8200044d0 msgr2=0x7fe820004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:50.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.170+0000 7fe845ae5700 1 --2- 172.21.15.162:0/1480628050 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8200044d0 0x7fe820004950 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 rx=0x7fe8300013e0 tx=0x7fe840142b10).stop 2020-11-28T21:55:50.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.174+0000 7fe845ae5700 1 -- 172.21.15.162:0/1480628050 shutdown_connections 2020-11-28T21:55:50.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.174+0000 7fe845ae5700 1 --2- 172.21.15.162:0/1480628050 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8200044d0 0x7fe820004950 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:50.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.174+0000 7fe845ae5700 1 -- 172.21.15.162:0/1480628050 >> 172.21.15.162:0/1480628050 conn(0x7fe84005b970 msgr2=0x7fe84005bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:50.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.174+0000 7fe845ae5700 1 -- 172.21.15.162:0/1480628050 shutdown_connections 2020-11-28T21:55:50.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.174+0000 7fe845ae5700 1 -- 172.21.15.162:0/1480628050 wait complete. 2020-11-28T21:55:50.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.174+0000 7fe845ae5700 1 Processor -- start 2020-11-28T21:55:50.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.174+0000 7fe845ae5700 1 -- start start 2020-11-28T21:55:50.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.174+0000 7fe845ae5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8200044d0 0x7fe8401493d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:50.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.174+0000 7fe845ae5700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe84005fe50 0x7fe840144fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:50.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.174+0000 7fe845ae5700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe8401454d0 0x7fe840145ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:50.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.174+0000 7fe845ae5700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe840152670 con 0x7fe8401454d0 2020-11-28T21:55:50.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.174+0000 7fe845ae5700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8401519f0 con 0x7fe8200044d0 2020-11-28T21:55:50.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.174+0000 7fe845ae5700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe840151c90 con 0x7fe84005fe50 2020-11-28T21:55:50.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.174+0000 7fe844ae3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8200044d0 0x7fe8401493d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:50.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.174+0000 7fe844ae3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8200044d0 0x7fe8401493d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45316/0 (socket says 172.21.15.162:45316) 2020-11-28T21:55:50.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.174+0000 7fe844ae3700 1 -- 172.21.15.162:0/3182606230 learned_addr learned my addr 172.21.15.162:0/3182606230 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:50.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.174+0000 7fe83ffff700 1 --2- 172.21.15.162:0/3182606230 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe84005fe50 0x7fe840144fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:50.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.174+0000 7fe8452e4700 1 --2- 172.21.15.162:0/3182606230 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe8401454d0 0x7fe840145ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:50.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.174+0000 7fe844ae3700 1 -- 172.21.15.162:0/3182606230 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe84005fe50 msgr2=0x7fe840144fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:50.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.174+0000 7fe844ae3700 1 --2- 172.21.15.162:0/3182606230 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe84005fe50 0x7fe840144fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:50.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.174+0000 7fe844ae3700 1 -- 172.21.15.162:0/3182606230 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe8401454d0 msgr2=0x7fe840145ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:50.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.174+0000 7fe844ae3700 1 --2- 172.21.15.162:0/3182606230 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe8401454d0 0x7fe840145ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:50.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.174+0000 7fe844ae3700 1 -- 172.21.15.162:0/3182606230 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe840151f30 con 0x7fe8200044d0 2020-11-28T21:55:50.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.174+0000 7fe844ae3700 1 --2- 172.21.15.162:0/3182606230 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8200044d0 0x7fe8401493d0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 rx=0x7fe83800b000 tx=0x7fe838009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:50.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.174+0000 7fe83dffb700 1 -- 172.21.15.162:0/3182606230 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe83800c5e0 con 0x7fe8200044d0 2020-11-28T21:55:50.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.174+0000 7fe83dffb700 1 -- 172.21.15.162:0/3182606230 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe83800c740 con 0x7fe8200044d0 2020-11-28T21:55:50.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.174+0000 7fe845ae5700 1 -- 172.21.15.162:0/3182606230 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8401521d0 con 0x7fe8200044d0 2020-11-28T21:55:50.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.174+0000 7fe83dffb700 1 -- 172.21.15.162:0/3182606230 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe83800caa0 con 0x7fe8200044d0 2020-11-28T21:55:50.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.174+0000 7fe845ae5700 1 -- 172.21.15.162:0/3182606230 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8400b1ee0 con 0x7fe8200044d0 2020-11-28T21:55:50.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.178+0000 7fe845ae5700 1 -- 172.21.15.162:0/3182606230 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe840152360 con 0x7fe8200044d0 2020-11-28T21:55:50.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.178+0000 7fe83dffb700 1 -- 172.21.15.162:0/3182606230 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe83800cc00 con 0x7fe8200044d0 2020-11-28T21:55:50.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.178+0000 7fe83dffb700 1 --2- 172.21.15.162:0/3182606230 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe8240554e0 0x7fe8240579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:50.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.178+0000 7fe83dffb700 1 -- 172.21.15.162:0/3182606230 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe83806df70 con 0x7fe8200044d0 2020-11-28T21:55:50.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.178+0000 7fe83ffff700 1 --2- 172.21.15.162:0/3182606230 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe8240554e0 0x7fe8240579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:50.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.178+0000 7fe83ffff700 1 --2- 172.21.15.162:0/3182606230 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe8240554e0 0x7fe8240579a0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7fe830019000 tx=0x7fe830013000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:50.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.186+0000 7fe83dffb700 1 -- 172.21.15.162:0/3182606230 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe838044210 con 0x7fe8200044d0 2020-11-28T21:55:50.375 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.370+0000 7fe845ae5700 1 -- 172.21.15.162:0/3182606230 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_getattr_caps"]} v 0) v1 -- 0x7fe840057050 con 0x7fe8200044d0 2020-11-28T21:55:50.878 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.874+0000 7fe83dffb700 1 -- 172.21.15.162:0/3182606230 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_getattr_caps"]}]=0 v426) v1 ==== 128+0+0 (secure 0 0 0) 0x7fe838047830 con 0x7fe8200044d0 2020-11-28T21:55:50.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.874+0000 7fe845ae5700 1 -- 172.21.15.162:0/3182606230 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe8240554e0 msgr2=0x7fe8240579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:50.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.874+0000 7fe845ae5700 1 --2- 172.21.15.162:0/3182606230 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe8240554e0 0x7fe8240579a0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7fe830019000 tx=0x7fe830013000).stop 2020-11-28T21:55:50.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.874+0000 7fe845ae5700 1 -- 172.21.15.162:0/3182606230 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8200044d0 msgr2=0x7fe8401493d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:50.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.874+0000 7fe845ae5700 1 --2- 172.21.15.162:0/3182606230 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8200044d0 0x7fe8401493d0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 rx=0x7fe83800b000 tx=0x7fe838009040).stop 2020-11-28T21:55:50.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.878+0000 7fe845ae5700 1 -- 172.21.15.162:0/3182606230 shutdown_connections 2020-11-28T21:55:50.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.878+0000 7fe845ae5700 1 --2- 172.21.15.162:0/3182606230 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe8240554e0 0x7fe8240579a0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:50.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.878+0000 7fe845ae5700 1 --2- 172.21.15.162:0/3182606230 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe8401454d0 0x7fe840145ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:50.886 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.878+0000 7fe845ae5700 1 --2- 172.21.15.162:0/3182606230 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8200044d0 0x7fe8401493d0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:50.886 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.878+0000 7fe845ae5700 1 --2- 172.21.15.162:0/3182606230 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe84005fe50 0x7fe840144fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:50.886 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.878+0000 7fe845ae5700 1 -- 172.21.15.162:0/3182606230 >> 172.21.15.162:0/3182606230 conn(0x7fe84005b970 msgr2=0x7fe8400b8920 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:50.886 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.878+0000 7fe845ae5700 1 -- 172.21.15.162:0/3182606230 shutdown_connections 2020-11-28T21:55:50.886 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:50.878+0000 7fe845ae5700 1 -- 172.21.15.162:0/3182606230 wait complete. 2020-11-28T21:55:50.899 INFO:tasks.cephfs_test_runner:test_getattr_caps (tasks.cephfs.test_misc.TestMisc) ... ok 2020-11-28T21:55:50.900 INFO:tasks.cephfs_test_runner:Starting test: test_root_rctime (tasks.cephfs.test_misc.TestMisc) 2020-11-28T21:55:50.901 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_misc.TestMisc.test_root_rctime' 2020-11-28T21:55:51.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.014+0000 7f76a1901700 1 Processor -- start 2020-11-28T21:55:51.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.014+0000 7f76a1901700 1 -- start start 2020-11-28T21:55:51.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.018+0000 7f76a1901700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f769c064c70 0x7f769c066290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:51.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.018+0000 7f76a1901700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f769c065750 0x7f769c13c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:51.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.018+0000 7f76a1901700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f769c065050 con 0x7f769c13f6c0 2020-11-28T21:55:51.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.018+0000 7f76a1901700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f769c13fc40 con 0x7f769c064c70 2020-11-28T21:55:51.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.018+0000 7f76a1901700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f769c13fee0 con 0x7f769c065750 2020-11-28T21:55:51.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.018+0000 7f76a1100700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f769c065750 0x7f769c13c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:51.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.018+0000 7f769bfff700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f769c13f6c0 0x7f769c066780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.162:42316/0 (socket says 172.21.15.162:42316) 2020-11-28T21:55:51.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.018+0000 7f769bfff700 1 -- 172.21.15.162:0/107711625 learned_addr learned my addr 172.21.15.162:0/107711625 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:51.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.018+0000 7f76a08ff700 1 --2- 172.21.15.162:0/107711625 >> v2:172.21.15.118:3300/0 conn(0x7f769c064c70 0x7f769c066290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:51.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.018+0000 7f769b7fe700 1 -- 172.21.15.162:0/107711625 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3216560882 0 0) 0x7f769c065050 con 0x7f769c13f6c0 2020-11-28T21:55:51.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.018+0000 7f769b7fe700 1 -- 172.21.15.162:0/107711625 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7680003230 con 0x7f769c13f6c0 2020-11-28T21:55:51.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.018+0000 7f76a1100700 1 -- 172.21.15.162:0/107711625 >> v2:172.21.15.118:3300/0 conn(0x7f769c064c70 msgr2=0x7f769c066290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:51.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.018+0000 7f76a1100700 1 --2- 172.21.15.162:0/107711625 >> v2:172.21.15.118:3300/0 conn(0x7f769c064c70 0x7f769c066290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:51.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.018+0000 7f76a1100700 1 -- 172.21.15.162:0/107711625 >> v1:172.21.15.118:6789/0 conn(0x7f769c13f6c0 legacy=0x7f769c066780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:51.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.018+0000 7f76a1100700 1 -- 172.21.15.162:0/107711625 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f769c140180 con 0x7f769c065750 2020-11-28T21:55:51.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.018+0000 7f76a1100700 1 --2- 172.21.15.162:0/107711625 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f769c065750 0x7f769c13c260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f76940032b0 tx=0x7f76940091a0).ready entity=mon.2 client_cookie=b639b520b46106a6 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:51.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.018+0000 7f769b7fe700 1 -- 172.21.15.162:0/107711625 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f769400b940 con 0x7f769c065750 2020-11-28T21:55:51.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.018+0000 7f769b7fe700 1 -- 172.21.15.162:0/107711625 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f769400baa0 con 0x7f769c065750 2020-11-28T21:55:51.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.018+0000 7f769b7fe700 1 -- 172.21.15.162:0/107711625 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f769400bc10 con 0x7f769c065750 2020-11-28T21:55:51.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.018+0000 7f76a1901700 1 -- 172.21.15.162:0/107711625 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f769c065750 msgr2=0x7f769c13c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:51.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.018+0000 7f76a1901700 1 --2- 172.21.15.162:0/107711625 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f769c065750 0x7f769c13c260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f76940032b0 tx=0x7f76940091a0).stop 2020-11-28T21:55:51.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.018+0000 7f76a1901700 1 -- 172.21.15.162:0/107711625 shutdown_connections 2020-11-28T21:55:51.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.018+0000 7f76a1901700 1 --2- 172.21.15.162:0/107711625 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f769c065750 0x7f769c13c260 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:51.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.018+0000 7f76a1901700 1 --2- 172.21.15.162:0/107711625 >> v2:172.21.15.118:3300/0 conn(0x7f769c064c70 0x7f769c066290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:51.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.018+0000 7f76a1901700 1 -- 172.21.15.162:0/107711625 >> 172.21.15.162:0/107711625 conn(0x7f769c0b5da0 msgr2=0x7f769c0b6180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:51.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.018+0000 7f76a1901700 1 -- 172.21.15.162:0/107711625 shutdown_connections 2020-11-28T21:55:51.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.018+0000 7f76a1901700 1 -- 172.21.15.162:0/107711625 wait complete. 2020-11-28T21:55:51.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.022+0000 7f76a1901700 1 Processor -- start 2020-11-28T21:55:51.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.022+0000 7f76a1901700 1 -- start start 2020-11-28T21:55:51.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.022+0000 7f76a1901700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f769c064c70 0x7f769c0ad4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:51.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.022+0000 7f76a1901700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f769c065750 0x7f769c13d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:51.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.022+0000 7f76a1901700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f769c13f6c0 0x7f769c155360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:51.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.022+0000 7f76a1901700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769c13f2e0 con 0x7f769c064c70 2020-11-28T21:55:51.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.022+0000 7f76a1901700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769c0b5860 con 0x7f769c13f6c0 2020-11-28T21:55:51.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.022+0000 7f76a1901700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f769c13c460 con 0x7f769c065750 2020-11-28T21:55:51.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.022+0000 7f76a1100700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f769c13f6c0 0x7f769c155360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:51.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.022+0000 7f76a1100700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f769c13f6c0 0x7f769c155360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45334/0 (socket says 172.21.15.162:45334) 2020-11-28T21:55:51.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.022+0000 7f76a1100700 1 -- 172.21.15.162:0/2913427234 learned_addr learned my addr 172.21.15.162:0/2913427234 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:51.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.022+0000 7f76a08ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f769c064c70 0x7f769c0ad4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:51.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.022+0000 7f769bfff700 1 --2- 172.21.15.162:0/2913427234 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f769c065750 0x7f769c13d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:51.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.022+0000 7f76a1100700 1 -- 172.21.15.162:0/2913427234 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f769c065750 msgr2=0x7f769c13d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:51.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.022+0000 7f76a1100700 1 --2- 172.21.15.162:0/2913427234 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f769c065750 0x7f769c13d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:51.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.022+0000 7f76a1100700 1 -- 172.21.15.162:0/2913427234 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f769c064c70 msgr2=0x7f769c0ad4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:51.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.022+0000 7f76a1100700 1 --2- 172.21.15.162:0/2913427234 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f769c064c70 0x7f769c0ad4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:51.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.022+0000 7f76a1100700 1 -- 172.21.15.162:0/2913427234 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f769c0affd0 con 0x7f769c13f6c0 2020-11-28T21:55:51.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.022+0000 7f769bfff700 1 --2- 172.21.15.162:0/2913427234 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f769c065750 0x7f769c13d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:55:51.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.022+0000 7f76a1100700 1 --2- 172.21.15.162:0/2913427234 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f769c13f6c0 0x7f769c155360 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 rx=0x7f76940032b0 tx=0x7f7694016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:51.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.022+0000 7f7699ffb700 1 -- 172.21.15.162:0/2913427234 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76940205f0 con 0x7f769c13f6c0 2020-11-28T21:55:51.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.022+0000 7f7699ffb700 1 -- 172.21.15.162:0/2913427234 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7694020750 con 0x7f769c13f6c0 2020-11-28T21:55:51.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.022+0000 7f76a1901700 1 -- 172.21.15.162:0/2913427234 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f769c0b0220 con 0x7f769c13f6c0 2020-11-28T21:55:51.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.022+0000 7f7699ffb700 1 -- 172.21.15.162:0/2913427234 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7694020ab0 con 0x7f769c13f6c0 2020-11-28T21:55:51.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.022+0000 7f76a1901700 1 -- 172.21.15.162:0/2913427234 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f769c1559f0 con 0x7f769c13f6c0 2020-11-28T21:55:51.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.022+0000 7f76a1901700 1 -- 172.21.15.162:0/2913427234 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f769c057050 con 0x7f769c13f6c0 2020-11-28T21:55:51.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.026+0000 7f7699ffb700 1 -- 172.21.15.162:0/2913427234 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7694027030 con 0x7f769c13f6c0 2020-11-28T21:55:51.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.026+0000 7f7699ffb700 1 --2- 172.21.15.162:0/2913427234 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f76840554e0 0x7f76840579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:51.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.026+0000 7f7699ffb700 1 -- 172.21.15.162:0/2913427234 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f769406f2e0 con 0x7f769c13f6c0 2020-11-28T21:55:51.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.026+0000 7f76a08ff700 1 --2- 172.21.15.162:0/2913427234 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f76840554e0 0x7f76840579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:51.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.026+0000 7f76a08ff700 1 --2- 172.21.15.162:0/2913427234 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f76840554e0 0x7f76840579a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f769000b000 tx=0x7f7690006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:51.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.034+0000 7f7699ffb700 1 -- 172.21.15.162:0/2913427234 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7694045620 con 0x7f769c13f6c0 2020-11-28T21:55:51.227 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.222+0000 7f76a1901700 1 -- 172.21.15.162:0/2913427234 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_root_rctime"]} v 0) v1 -- 0x7f769c0b59a0 con 0x7f769c13f6c0 2020-11-28T21:55:51.887 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.882+0000 7f7699ffb700 1 -- 172.21.15.162:0/2913427234 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_root_rctime"]}]=0 v427) v1 ==== 130+0+0 (secure 0 0 0) 0x7f76940150c0 con 0x7f769c13f6c0 2020-11-28T21:55:51.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.886+0000 7f76a1901700 1 -- 172.21.15.162:0/2913427234 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f76840554e0 msgr2=0x7f76840579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:51.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.886+0000 7f76a1901700 1 --2- 172.21.15.162:0/2913427234 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f76840554e0 0x7f76840579a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f769000b000 tx=0x7f7690006dc0).stop 2020-11-28T21:55:51.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.886+0000 7f76a1901700 1 -- 172.21.15.162:0/2913427234 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f769c13f6c0 msgr2=0x7f769c155360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:51.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.886+0000 7f76a1901700 1 --2- 172.21.15.162:0/2913427234 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f769c13f6c0 0x7f769c155360 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 rx=0x7f76940032b0 tx=0x7f7694016000).stop 2020-11-28T21:55:51.892 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.886+0000 7f76a1901700 1 -- 172.21.15.162:0/2913427234 shutdown_connections 2020-11-28T21:55:51.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.886+0000 7f76a1901700 1 --2- 172.21.15.162:0/2913427234 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f76840554e0 0x7f76840579a0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:51.893 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.886+0000 7f76a1901700 1 --2- 172.21.15.162:0/2913427234 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f769c13f6c0 0x7f769c155360 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:51.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.886+0000 7f76a1901700 1 --2- 172.21.15.162:0/2913427234 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f769c064c70 0x7f769c0ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:51.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.886+0000 7f76a1901700 1 --2- 172.21.15.162:0/2913427234 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f769c065750 0x7f769c13d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:51.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.886+0000 7f76a1901700 1 -- 172.21.15.162:0/2913427234 >> 172.21.15.162:0/2913427234 conn(0x7f769c0b5da0 msgr2=0x7f769c05b970 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:51.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.886+0000 7f76a1901700 1 -- 172.21.15.162:0/2913427234 shutdown_connections 2020-11-28T21:55:51.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:51.886+0000 7f76a1901700 1 -- 172.21.15.162:0/2913427234 wait complete. 2020-11-28T21:55:51.908 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config set mon mon_allow_pool_delete True 2020-11-28T21:55:52.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.026+0000 7fbaeb367700 1 Processor -- start 2020-11-28T21:55:52.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.026+0000 7fbaeb367700 1 -- start start 2020-11-28T21:55:52.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.026+0000 7fbaeb367700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbae40b21f0 0x7fbae413fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:52.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.026+0000 7fbaeb367700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbae41402a0 0x7fbae4060450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:52.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.026+0000 7fbaeb367700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbae40aff20 con 0x7fbae4060940 2020-11-28T21:55:52.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.026+0000 7fbaeb367700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbae40af240 con 0x7fbae40b21f0 2020-11-28T21:55:52.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.026+0000 7fbaeb367700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbae40b25d0 con 0x7fbae41402a0 2020-11-28T21:55:52.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.026+0000 7fbaeab66700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fbae4060940 0x7fbae4146030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:52186/0 (socket says 172.21.15.162:52186) 2020-11-28T21:55:52.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.026+0000 7fbaeab66700 1 -- 172.21.15.162:0/2491271773 learned_addr learned my addr 172.21.15.162:0/2491271773 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:52.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.026+0000 7fbaea365700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbae40b21f0 0x7fbae413fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:52.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.026+0000 7fbae9b64700 1 --2- 172.21.15.162:0/2491271773 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbae41402a0 0x7fbae4060450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:52.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.026+0000 7fbae9363700 1 -- 172.21.15.162:0/2491271773 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4078907403 0 0) 0x7fbae40aff20 con 0x7fbae4060940 2020-11-28T21:55:52.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.026+0000 7fbae9363700 1 -- 172.21.15.162:0/2491271773 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbacc003230 con 0x7fbae4060940 2020-11-28T21:55:52.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.026+0000 7fbaea365700 1 -- 172.21.15.162:0/2491271773 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbae41402a0 msgr2=0x7fbae4060450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:52.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.026+0000 7fbaea365700 1 --2- 172.21.15.162:0/2491271773 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbae41402a0 0x7fbae4060450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:52.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.030+0000 7fbaea365700 1 -- 172.21.15.162:0/2491271773 >> v1:172.21.15.162:6789/0 conn(0x7fbae4060940 legacy=0x7fbae4146030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:52.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.030+0000 7fbae9363700 1 -- 172.21.15.162:0/2491271773 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 292744656 0 0) 0x7fbacc003230 con 0x7fbae4060940 2020-11-28T21:55:52.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.030+0000 7fbaea365700 1 -- 172.21.15.162:0/2491271773 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae405fff0 con 0x7fbae40b21f0 2020-11-28T21:55:52.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.030+0000 7fbae9b64700 1 --2- 172.21.15.162:0/2491271773 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbae41402a0 0x7fbae4060450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:55:52.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.030+0000 7fbaea365700 1 --2- 172.21.15.162:0/2491271773 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbae40b21f0 0x7fbae413fdb0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 rx=0x7fbadc001bf0 tx=0x7fbae0006dc0).ready entity=mon.1 client_cookie=aca4bd74b4941f74 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:52.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.030+0000 7fbae9363700 1 -- 172.21.15.162:0/2491271773 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbadc001a50 con 0x7fbae40b21f0 2020-11-28T21:55:52.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.030+0000 7fbae9363700 1 -- 172.21.15.162:0/2491271773 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbae000d570 con 0x7fbae40b21f0 2020-11-28T21:55:52.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.030+0000 7fbae9363700 1 -- 172.21.15.162:0/2491271773 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbae000d6e0 con 0x7fbae40b21f0 2020-11-28T21:55:52.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.030+0000 7fbaeb367700 1 -- 172.21.15.162:0/2491271773 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbae40b21f0 msgr2=0x7fbae413fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:52.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.030+0000 7fbaeb367700 1 --2- 172.21.15.162:0/2491271773 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbae40b21f0 0x7fbae413fdb0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 rx=0x7fbadc001bf0 tx=0x7fbae0006dc0).stop 2020-11-28T21:55:52.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.030+0000 7fbaeb367700 1 -- 172.21.15.162:0/2491271773 shutdown_connections 2020-11-28T21:55:52.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.030+0000 7fbaeb367700 1 --2- 172.21.15.162:0/2491271773 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbae40b21f0 0x7fbae413fdb0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:52.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.030+0000 7fbaeb367700 1 --2- 172.21.15.162:0/2491271773 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbae41402a0 0x7fbae4060450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:52.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.030+0000 7fbaeb367700 1 -- 172.21.15.162:0/2491271773 >> 172.21.15.162:0/2491271773 conn(0x7fbae40b57d0 msgr2=0x7fbae40b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:52.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.030+0000 7fbaeb367700 1 -- 172.21.15.162:0/2491271773 shutdown_connections 2020-11-28T21:55:52.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.030+0000 7fbaeb367700 1 -- 172.21.15.162:0/2491271773 wait complete. 2020-11-28T21:55:52.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.030+0000 7fbaeb367700 1 Processor -- start 2020-11-28T21:55:52.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.030+0000 7fbaeb367700 1 -- start start 2020-11-28T21:55:52.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.034+0000 7fbaeb367700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbae4060940 0x7fbae40b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:52.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.034+0000 7fbaeb367700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbae40b21f0 0x7fbae41416d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:52.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.034+0000 7fbaeb367700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbae41402a0 0x7fbae4141bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:52.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.034+0000 7fbaeb367700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae40ac370 con 0x7fbae4060940 2020-11-28T21:55:52.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.034+0000 7fbaeb367700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae4152a30 con 0x7fbae41402a0 2020-11-28T21:55:52.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.034+0000 7fbaeb367700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbae4151db0 con 0x7fbae40b21f0 2020-11-28T21:55:52.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.034+0000 7fbaeab66700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbae41402a0 0x7fbae4141bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:52.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.034+0000 7fbaeab66700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbae41402a0 0x7fbae4141bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45348/0 (socket says 172.21.15.162:45348) 2020-11-28T21:55:52.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.034+0000 7fbaeab66700 1 -- 172.21.15.162:0/2436496656 learned_addr learned my addr 172.21.15.162:0/2436496656 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:52.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.034+0000 7fbae9b64700 1 --2- 172.21.15.162:0/2436496656 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbae40b21f0 0x7fbae41416d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:52.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.034+0000 7fbaea365700 1 --2- 172.21.15.162:0/2436496656 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbae4060940 0x7fbae40b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:52.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.034+0000 7fbaeab66700 1 -- 172.21.15.162:0/2436496656 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbae40b21f0 msgr2=0x7fbae41416d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:52.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.034+0000 7fbaeab66700 1 --2- 172.21.15.162:0/2436496656 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbae40b21f0 0x7fbae41416d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:52.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.034+0000 7fbaeab66700 1 -- 172.21.15.162:0/2436496656 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbae4060940 msgr2=0x7fbae40b2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:52.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.034+0000 7fbaeab66700 1 --2- 172.21.15.162:0/2436496656 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbae4060940 0x7fbae40b2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:52.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.034+0000 7fbaeab66700 1 -- 172.21.15.162:0/2436496656 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae4152050 con 0x7fbae41402a0 2020-11-28T21:55:52.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.034+0000 7fbaea365700 1 --2- 172.21.15.162:0/2436496656 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbae4060940 0x7fbae40b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:55:52.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.034+0000 7fbaeab66700 1 --2- 172.21.15.162:0/2436496656 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbae41402a0 0x7fbae4141bc0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 rx=0x7fbadc001cb0 tx=0x7fbadc001570).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:52.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.034+0000 7fbad77fe700 1 -- 172.21.15.162:0/2436496656 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbadc004a80 con 0x7fbae41402a0 2020-11-28T21:55:52.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.034+0000 7fbad77fe700 1 -- 172.21.15.162:0/2436496656 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbadc004be0 con 0x7fbae41402a0 2020-11-28T21:55:52.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.034+0000 7fbaeb367700 1 -- 172.21.15.162:0/2436496656 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbae41522f0 con 0x7fbae41402a0 2020-11-28T21:55:52.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.034+0000 7fbad77fe700 1 -- 172.21.15.162:0/2436496656 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbadc007170 con 0x7fbae41402a0 2020-11-28T21:55:52.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.034+0000 7fbaeb367700 1 -- 172.21.15.162:0/2436496656 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbae41420b0 con 0x7fbae41402a0 2020-11-28T21:55:52.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.034+0000 7fbad77fe700 1 -- 172.21.15.162:0/2436496656 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fbadc010070 con 0x7fbae41402a0 2020-11-28T21:55:52.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.034+0000 7fbad77fe700 1 --2- 172.21.15.162:0/2436496656 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbac4055330 0x7fbac40577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:52.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.034+0000 7fbad77fe700 1 -- 172.21.15.162:0/2436496656 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbadc06dce0 con 0x7fbae41402a0 2020-11-28T21:55:52.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.034+0000 7fbaeb367700 1 -- 172.21.15.162:0/2436496656 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbae4057050 con 0x7fbae41402a0 2020-11-28T21:55:52.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.034+0000 7fbaea365700 1 --2- 172.21.15.162:0/2436496656 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbac4055330 0x7fbac40577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:52.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.038+0000 7fbaea365700 1 --2- 172.21.15.162:0/2436496656 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbac4055330 0x7fbac40577f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7fbae0009000 tx=0x7fbae0015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:52.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.042+0000 7fbad77fe700 1 -- 172.21.15.162:0/2436496656 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fbadc06bdf0 con 0x7fbae41402a0 2020-11-28T21:55:52.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.186+0000 7fbaeb367700 1 -- 172.21.15.162:0/2436496656 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7fbae405d250 con 0x7fbae41402a0 2020-11-28T21:55:52.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.194+0000 7fbad77fe700 1 -- 172.21.15.162:0/2436496656 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbadc04b070 con 0x7fbae41402a0 2020-11-28T21:55:52.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.198+0000 7fbaeb367700 1 -- 172.21.15.162:0/2436496656 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbac4055330 msgr2=0x7fbac40577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:52.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.198+0000 7fbaeb367700 1 --2- 172.21.15.162:0/2436496656 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbac4055330 0x7fbac40577f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7fbae0009000 tx=0x7fbae0015000).stop 2020-11-28T21:55:52.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.198+0000 7fbaeb367700 1 -- 172.21.15.162:0/2436496656 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbae41402a0 msgr2=0x7fbae4141bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:52.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.198+0000 7fbaeb367700 1 --2- 172.21.15.162:0/2436496656 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbae41402a0 0x7fbae4141bc0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 rx=0x7fbadc001cb0 tx=0x7fbadc001570).stop 2020-11-28T21:55:52.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.198+0000 7fbaeb367700 1 -- 172.21.15.162:0/2436496656 shutdown_connections 2020-11-28T21:55:52.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.198+0000 7fbaeb367700 1 --2- 172.21.15.162:0/2436496656 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbac4055330 0x7fbac40577f0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:52.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.198+0000 7fbaeb367700 1 --2- 172.21.15.162:0/2436496656 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbae41402a0 0x7fbae4141bc0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:52.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.198+0000 7fbaeb367700 1 --2- 172.21.15.162:0/2436496656 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbae4060940 0x7fbae40b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:52.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.198+0000 7fbaeb367700 1 --2- 172.21.15.162:0/2436496656 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbae40b21f0 0x7fbae41416d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:52.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.198+0000 7fbaeb367700 1 -- 172.21.15.162:0/2436496656 >> 172.21.15.162:0/2436496656 conn(0x7fbae40b57d0 msgr2=0x7fbae413ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:52.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.198+0000 7fbaeb367700 1 -- 172.21.15.162:0/2436496656 shutdown_connections 2020-11-28T21:55:52.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.198+0000 7fbaeb367700 1 -- 172.21.15.162:0/2436496656 wait complete. 2020-11-28T21:55:52.218 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-28T21:55:52.219 INFO:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:55:52.236 INFO:teuthology.orchestra.run.smithi118:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:55:52.255 INFO:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:55:52.273 INFO:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:55:52.294 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-28T21:55:52.295 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:52.296 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:55:52.296 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:55:52.345 INFO:teuthology.orchestra.run.smithi192.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2020-11-28T21:55:52.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:55:52.347 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2020-11-28T21:55:52.347 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:55:52.348 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:55:52.348 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:55:52.358 INFO:teuthology.orchestra.run.smithi096.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-11-28T21:55:52.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:55:52.360 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2020-11-28T21:55:52.360 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2020-11-28T21:55:52.486 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.482+0000 7fa4effff700 1 Processor -- start 2020-11-28T21:55:52.486 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.482+0000 7fa4effff700 1 -- start start 2020-11-28T21:55:52.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.482+0000 7fa4effff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa4f0066a10 0x7fa4f0061760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:52.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.482+0000 7fa4effff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa4f0143da0 0x7fa4f0061c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:52.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.482+0000 7fa4effff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4f0066df0 con 0x7fa4f00b1500 2020-11-28T21:55:52.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.482+0000 7fa4effff700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4f0060100 con 0x7fa4f0143da0 2020-11-28T21:55:52.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.482+0000 7fa4effff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f00b12b0 con 0x7fa4f0066a10 2020-11-28T21:55:52.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.482+0000 7fa4ee7fc700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa4f0143da0 0x7fa4f0061c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:52.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.482+0000 7fa4ee7fc700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa4f0143da0 0x7fa4f0061c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45354/0 (socket says 172.21.15.162:45354) 2020-11-28T21:55:52.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.482+0000 7fa4ee7fc700 1 -- 172.21.15.162:0/2015242283 learned_addr learned my addr 172.21.15.162:0/2015242283 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:52.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.482+0000 7fa4eeffd700 1 --2- 172.21.15.162:0/2015242283 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa4f0066a10 0x7fa4f0061760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:52.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.482+0000 7fa4ee7fc700 1 -- 172.21.15.162:0/2015242283 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa4f0066a10 msgr2=0x7fa4f0061760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:52.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.482+0000 7fa4ee7fc700 1 --2- 172.21.15.162:0/2015242283 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa4f0066a10 0x7fa4f0061760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:52.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.482+0000 7fa4ee7fc700 1 -- 172.21.15.162:0/2015242283 >> v1:172.21.15.118:6789/0 conn(0x7fa4f00b1500 legacy=0x7fa4f005b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:52.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.482+0000 7fa4ee7fc700 1 -- 172.21.15.162:0/2015242283 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f00b0d70 con 0x7fa4f0143da0 2020-11-28T21:55:52.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.482+0000 7fa4ee7fc700 1 --2- 172.21.15.162:0/2015242283 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4f0143da0 0x7fa4f0061c50 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 rx=0x7fa4e4002f30 tx=0x7fa4e4006e80).ready entity=mon.1 client_cookie=65368bab2b3e907e server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:52.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.482+0000 7fa4edffb700 1 -- 172.21.15.162:0/2015242283 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4e40097f0 con 0x7fa4f0143da0 2020-11-28T21:55:52.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.482+0000 7fa4edffb700 1 -- 172.21.15.162:0/2015242283 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e4009950 con 0x7fa4f0143da0 2020-11-28T21:55:52.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.482+0000 7fa4edffb700 1 -- 172.21.15.162:0/2015242283 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4e4009cb0 con 0x7fa4f0143da0 2020-11-28T21:55:52.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.482+0000 7fa4effff700 1 -- 172.21.15.162:0/2015242283 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4f0143da0 msgr2=0x7fa4f0061c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:52.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.482+0000 7fa4effff700 1 --2- 172.21.15.162:0/2015242283 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4f0143da0 0x7fa4f0061c50 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 rx=0x7fa4e4002f30 tx=0x7fa4e4006e80).stop 2020-11-28T21:55:52.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.486+0000 7fa4effff700 1 -- 172.21.15.162:0/2015242283 shutdown_connections 2020-11-28T21:55:52.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.486+0000 7fa4effff700 1 --2- 172.21.15.162:0/2015242283 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa4f0066a10 0x7fa4f0061760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:52.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.486+0000 7fa4effff700 1 --2- 172.21.15.162:0/2015242283 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4f0143da0 0x7fa4f0061c50 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:52.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.486+0000 7fa4effff700 1 -- 172.21.15.162:0/2015242283 >> 172.21.15.162:0/2015242283 conn(0x7fa4f00b5da0 msgr2=0x7fa4f00b6180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:52.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.486+0000 7fa4effff700 1 -- 172.21.15.162:0/2015242283 shutdown_connections 2020-11-28T21:55:52.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.486+0000 7fa4effff700 1 -- 172.21.15.162:0/2015242283 wait complete. 2020-11-28T21:55:52.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.486+0000 7fa4effff700 1 Processor -- start 2020-11-28T21:55:52.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.486+0000 7fa4effff700 1 -- start start 2020-11-28T21:55:52.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.486+0000 7fa4effff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa4f0066a10 0x7fa4f013bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:52.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.486+0000 7fa4effff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa4f00b1500 0x7fa4f0135720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:52.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.486+0000 7fa4effff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4f0143da0 0x7fa4f0135c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:52.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.486+0000 7fa4effff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f00b49f0 con 0x7fa4f00b1500 2020-11-28T21:55:52.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.486+0000 7fa4effff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f00b52a0 con 0x7fa4f0143da0 2020-11-28T21:55:52.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.486+0000 7fa4effff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f00b5540 con 0x7fa4f0066a10 2020-11-28T21:55:52.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.486+0000 7fa4eeffd700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa4f0066a10 0x7fa4f013bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:52.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.486+0000 7fa4ef7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4f0143da0 0x7fa4f0135c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:52.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.486+0000 7fa4eeffd700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa4f0066a10 0x7fa4f013bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:42582/0 (socket says 172.21.15.162:42582) 2020-11-28T21:55:52.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.486+0000 7fa4eeffd700 1 -- 172.21.15.162:0/2128620389 learned_addr learned my addr 172.21.15.162:0/2128620389 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:52.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.486+0000 7fa4ef7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4f0143da0 0x7fa4f0135c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45362/0 (socket says 172.21.15.162:45362) 2020-11-28T21:55:52.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.486+0000 7fa4ee7fc700 1 --2- 172.21.15.162:0/2128620389 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa4f00b1500 0x7fa4f0135720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:52.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.486+0000 7fa4eeffd700 1 -- 172.21.15.162:0/2128620389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4f0143da0 msgr2=0x7fa4f0135c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:52.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.486+0000 7fa4eeffd700 1 --2- 172.21.15.162:0/2128620389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4f0143da0 0x7fa4f0135c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:52.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.486+0000 7fa4eeffd700 1 -- 172.21.15.162:0/2128620389 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa4f00b1500 msgr2=0x7fa4f0135720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:52.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.486+0000 7fa4eeffd700 1 --2- 172.21.15.162:0/2128620389 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa4f00b1500 0x7fa4f0135720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:52.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.486+0000 7fa4eeffd700 1 -- 172.21.15.162:0/2128620389 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f013a5c0 con 0x7fa4f0066a10 2020-11-28T21:55:52.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.486+0000 7fa4ef7fe700 1 --2- 172.21.15.162:0/2128620389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4f0143da0 0x7fa4f0135c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:55:52.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.490+0000 7fa4eeffd700 1 --2- 172.21.15.162:0/2128620389 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa4f0066a10 0x7fa4f013bf40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7fa4e000c000 tx=0x7fa4e0006dc0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:52.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.490+0000 7fa4dffff700 1 -- 172.21.15.162:0/2128620389 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4e000ca10 con 0x7fa4f0066a10 2020-11-28T21:55:52.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.490+0000 7fa4dffff700 1 -- 172.21.15.162:0/2128620389 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e000cb70 con 0x7fa4f0066a10 2020-11-28T21:55:52.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.490+0000 7fa4effff700 1 -- 172.21.15.162:0/2128620389 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4f0136100 con 0x7fa4f0066a10 2020-11-28T21:55:52.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.490+0000 7fa4dffff700 1 -- 172.21.15.162:0/2128620389 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4e00123f0 con 0x7fa4f0066a10 2020-11-28T21:55:52.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.490+0000 7fa4effff700 1 -- 172.21.15.162:0/2128620389 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4f00b4670 con 0x7fa4f0066a10 2020-11-28T21:55:52.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.490+0000 7fa4effff700 1 -- 172.21.15.162:0/2128620389 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4f013ac90 con 0x7fa4f0066a10 2020-11-28T21:55:52.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.490+0000 7fa4dffff700 1 -- 172.21.15.162:0/2128620389 <== mon.2 v2:172.21.15.118:3301/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa4e0012550 con 0x7fa4f0066a10 2020-11-28T21:55:52.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.490+0000 7fa4dffff700 1 --2- 172.21.15.162:0/2128620389 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa4cc055330 0x7fa4cc0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:52.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.490+0000 7fa4ee7fc700 1 --2- 172.21.15.162:0/2128620389 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa4cc055330 0x7fa4cc0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:52.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.490+0000 7fa4dffff700 1 -- 172.21.15.162:0/2128620389 <== mon.2 v2:172.21.15.118:3301/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa4e006e390 con 0x7fa4f0066a10 2020-11-28T21:55:52.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.494+0000 7fa4ee7fc700 1 --2- 172.21.15.162:0/2128620389 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa4cc055330 0x7fa4cc0577f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7fa4e401c000 tx=0x7fa4e4015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:52.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.502+0000 7fa4dffff700 1 -- 172.21.15.162:0/2128620389 <== mon.2 v2:172.21.15.118:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa4f013ac90 con 0x7fa4f0066a10 2020-11-28T21:55:52.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.646+0000 7fa4effff700 1 -- 172.21.15.162:0/2128620389 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fa4f013ac90 con 0x7fa4f0066a10 2020-11-28T21:55:52.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.646+0000 7fa4dffff700 1 -- 172.21.15.162:0/2128620389 <== mon.2 v2:172.21.15.118:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v138) v1 ==== 81+0+33755 (secure 0 0 0) 0x7fa4f013ac90 con 0x7fa4f0066a10 2020-11-28T21:55:52.654 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:52.654 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2020-11-28T21:55:52.654 INFO:teuthology.orchestra.run.smithi162.stdout: "epoch": 138, 2020-11-28T21:55:52.655 INFO:teuthology.orchestra.run.smithi162.stdout: "fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", 2020-11-28T21:55:52.655 INFO:teuthology.orchestra.run.smithi162.stdout: "created": "2020-11-28T21:46:11.724372+0000", 2020-11-28T21:55:52.655 INFO:teuthology.orchestra.run.smithi162.stdout: "modified": "2020-11-28T21:55:32.383252+0000", 2020-11-28T21:55:52.655 INFO:teuthology.orchestra.run.smithi162.stdout: "last_up_change": "2020-11-28T21:46:19.591650+0000", 2020-11-28T21:55:52.655 INFO:teuthology.orchestra.run.smithi162.stdout: "last_in_change": "2020-11-28T21:46:19.591650+0000", 2020-11-28T21:55:52.656 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-11-28T21:55:52.656 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_num": 5799936, 2020-11-28T21:55:52.656 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_set": [ 2020-11-28T21:55:52.656 INFO:teuthology.orchestra.run.smithi162.stdout: "pglog_hardlimit", 2020-11-28T21:55:52.656 INFO:teuthology.orchestra.run.smithi162.stdout: "purged_snapdirs", 2020-11-28T21:55:52.657 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_deletes", 2020-11-28T21:55:52.657 INFO:teuthology.orchestra.run.smithi162.stdout: "sortbitwise" 2020-11-28T21:55:52.657 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:55:52.657 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_version": 4, 2020-11-28T21:55:52.657 INFO:teuthology.orchestra.run.smithi162.stdout: "full_ratio": 0.94999998807907104, 2020-11-28T21:55:52.658 INFO:teuthology.orchestra.run.smithi162.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-11-28T21:55:52.658 INFO:teuthology.orchestra.run.smithi162.stdout: "nearfull_ratio": 0.85000002384185791, 2020-11-28T21:55:52.658 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_snapshot": "", 2020-11-28T21:55:52.658 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_max": 26, 2020-11-28T21:55:52.659 INFO:teuthology.orchestra.run.smithi162.stdout: "max_osd": 8, 2020-11-28T21:55:52.659 INFO:teuthology.orchestra.run.smithi162.stdout: "require_min_compat_client": "jewel", 2020-11-28T21:55:52.659 INFO:teuthology.orchestra.run.smithi162.stdout: "min_compat_client": "jewel", 2020-11-28T21:55:52.659 INFO:teuthology.orchestra.run.smithi162.stdout: "require_osd_release": "octopus", 2020-11-28T21:55:52.659 INFO:teuthology.orchestra.run.smithi162.stdout: "pools": [ 2020-11-28T21:55:52.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.660 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 1, 2020-11-28T21:55:52.660 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "device_health_metrics", 2020-11-28T21:55:52.660 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:46:15.639551+0000", 2020-11-28T21:55:52.660 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:55:52.661 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:55:52.661 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:55:52.661 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:55:52.661 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:55:52.661 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:55:52.662 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:55:52.662 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:55:52.662 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 1, 2020-11-28T21:55:52.662 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 1, 2020-11-28T21:55:52.662 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 1, 2020-11-28T21:55:52.663 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 1, 2020-11-28T21:55:52.663 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 1, 2020-11-28T21:55:52.663 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:55:52.663 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:55:52.663 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:55:52.664 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:55:52.664 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:55:52.664 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:55:52.664 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:55:52.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.665 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "16", 2020-11-28T21:55:52.665 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:55:52.665 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:55:52.665 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:55:52.665 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:55:52.666 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:55:52.666 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:55:52.666 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:55:52.666 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:55:52.666 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:55:52.667 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:55:52.667 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:55:52.667 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:55:52.668 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:55:52.668 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:55:52.668 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:55:52.668 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:55:52.669 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:55:52.669 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:55:52.669 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:55:52.669 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:55:52.669 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:55:52.670 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:55:52.670 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:55:52.670 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:55:52.670 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:55:52.670 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:55:52.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.671 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:55:52.671 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:55:52.671 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:55:52.671 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:55:52.672 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:55:52.672 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:55:52.672 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:55:52.672 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:55:52.672 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:55:52.673 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:55:52.673 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:55:52.673 INFO:teuthology.orchestra.run.smithi162.stdout: "options": { 2020-11-28T21:55:52.673 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_min": 1 2020-11-28T21:55:52.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.674 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:55:52.674 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr_devicehealth": {} 2020-11-28T21:55:52.674 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:52.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.675 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 2, 2020-11-28T21:55:52.675 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "rbd", 2020-11-28T21:55:52.675 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:46:23.901366+0000", 2020-11-28T21:55:52.675 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:55:52.675 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:55:52.676 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:55:52.676 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:55:52.676 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:55:52.676 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:55:52.677 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:55:52.677 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:55:52.677 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2020-11-28T21:55:52.677 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2020-11-28T21:55:52.677 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2020-11-28T21:55:52.678 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2020-11-28T21:55:52.678 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2020-11-28T21:55:52.678 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:55:52.678 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:55:52.679 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:55:52.679 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:55:52.679 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:55:52.679 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:55:52.679 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:55:52.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.680 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "19", 2020-11-28T21:55:52.680 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:55:52.680 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:55:52.680 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:55:52.680 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:55:52.681 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:55:52.681 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:55:52.681 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:55:52.681 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:55:52.681 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:55:52.682 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:55:52.682 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:55:52.682 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:55:52.682 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:55:52.682 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:55:52.683 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:55:52.683 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:55:52.683 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:55:52.683 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:55:52.683 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:55:52.684 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:55:52.684 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:55:52.684 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:55:52.684 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:55:52.684 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:55:52.685 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:55:52.685 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:55:52.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.685 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:55:52.685 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:55:52.686 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:55:52.686 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:55:52.686 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:55:52.686 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:55:52.686 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:55:52.687 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:55:52.687 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:55:52.687 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:55:52.687 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:55:52.687 INFO:teuthology.orchestra.run.smithi162.stdout: "options": {}, 2020-11-28T21:55:52.688 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:55:52.688 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd": {} 2020-11-28T21:55:52.688 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:52.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.689 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 25, 2020-11-28T21:55:52.689 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "cephfs_metadata", 2020-11-28T21:55:52.689 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:55:30.002238+0000", 2020-11-28T21:55:52.689 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:55:52.689 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:55:52.690 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:55:52.690 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:55:52.690 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:55:52.690 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:55:52.690 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:55:52.691 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:55:52.691 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2020-11-28T21:55:52.691 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2020-11-28T21:55:52.691 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2020-11-28T21:55:52.691 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2020-11-28T21:55:52.692 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2020-11-28T21:55:52.692 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:55:52.692 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:55:52.692 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:55:52.692 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:55:52.693 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:55:52.693 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:55:52.693 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:55:52.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.693 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "137", 2020-11-28T21:55:52.694 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:55:52.694 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:55:52.694 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:55:52.694 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:55:52.694 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:55:52.695 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:55:52.695 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:55:52.695 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:55:52.695 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:55:52.695 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:55:52.696 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:55:52.696 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:55:52.696 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:55:52.696 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:55:52.696 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:55:52.696 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:55:52.697 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:55:52.697 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:55:52.697 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:55:52.697 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:55:52.697 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:55:52.698 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:55:52.698 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:55:52.698 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:55:52.698 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:55:52.698 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:55:52.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.699 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:55:52.699 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:55:52.699 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:55:52.699 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:55:52.700 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:55:52.700 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:55:52.700 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:55:52.700 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:55:52.700 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:55:52.701 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:55:52.701 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:55:52.701 INFO:teuthology.orchestra.run.smithi162.stdout: "options": { 2020-11-28T21:55:52.701 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_bias": 4, 2020-11-28T21:55:52.701 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_min": 16, 2020-11-28T21:55:52.702 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_priority": 5 2020-11-28T21:55:52.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.702 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:55:52.702 INFO:teuthology.orchestra.run.smithi162.stdout: "cephfs": { 2020-11-28T21:55:52.702 INFO:teuthology.orchestra.run.smithi162.stdout: "metadata": "cephfs" 2020-11-28T21:55:52.703 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:52.703 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:52.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.703 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 26, 2020-11-28T21:55:52.704 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "cephfs_data", 2020-11-28T21:55:52.704 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:55:30.389418+0000", 2020-11-28T21:55:52.704 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:55:52.704 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:55:52.704 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:55:52.705 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:55:52.705 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:55:52.705 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:55:52.705 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:55:52.705 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:55:52.706 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2020-11-28T21:55:52.706 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2020-11-28T21:55:52.706 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2020-11-28T21:55:52.706 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2020-11-28T21:55:52.706 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2020-11-28T21:55:52.707 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:55:52.707 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:55:52.707 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:55:52.707 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:55:52.707 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:55:52.708 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:55:52.708 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:55:52.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.708 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "138", 2020-11-28T21:55:52.708 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:55:52.709 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:55:52.709 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:55:52.709 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:55:52.709 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:55:52.709 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:55:52.710 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:55:52.710 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:55:52.710 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:55:52.710 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:55:52.710 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:55:52.711 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:55:52.711 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:55:52.711 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:55:52.711 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:55:52.711 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:55:52.712 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:55:52.712 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:55:52.712 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:55:52.712 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:55:52.712 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:55:52.713 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:55:52.713 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:55:52.713 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:55:52.713 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:55:52.713 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:55:52.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.714 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:55:52.714 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:55:52.714 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:55:52.714 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:55:52.715 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:55:52.715 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:55:52.715 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:55:52.715 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:55:52.715 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:55:52.715 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:55:52.716 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:55:52.716 INFO:teuthology.orchestra.run.smithi162.stdout: "options": {}, 2020-11-28T21:55:52.716 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:55:52.716 INFO:teuthology.orchestra.run.smithi162.stdout: "cephfs": { 2020-11-28T21:55:52.716 INFO:teuthology.orchestra.run.smithi162.stdout: "data": "cephfs" 2020-11-28T21:55:52.717 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:52.717 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:52.717 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:52.717 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:55:52.717 INFO:teuthology.orchestra.run.smithi162.stdout: "osds": [ 2020-11-28T21:55:52.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.718 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 0, 2020-11-28T21:55:52.718 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "99c7d9ae-adfa-4fd7-880a-23e6baf3281b", 2020-11-28T21:55:52.718 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:55:52.718 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:55:52.719 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:55:52.719 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:55:52.719 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:55:52.719 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:55:52.719 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:55:52.720 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 136, 2020-11-28T21:55:52.720 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:55:52.720 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:55:52.720 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:55:52.720 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:52.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.721 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:52.721 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6810", 2020-11-28T21:55:52.721 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:55:52.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.722 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:52.722 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6811", 2020-11-28T21:55:52.722 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:55:52.722 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:52.723 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:52.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.723 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:55:52.723 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:52.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.724 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:52.724 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6812", 2020-11-28T21:55:52.724 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:55:52.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.725 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:52.725 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6813", 2020-11-28T21:55:52.725 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:55:52.725 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:52.725 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:52.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.726 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:55:52.726 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:52.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.726 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:52.727 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6816", 2020-11-28T21:55:52.727 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:55:52.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.727 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:52.728 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6817", 2020-11-28T21:55:52.728 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:55:52.728 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:52.728 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:52.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.729 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:55:52.729 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:52.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.729 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:52.729 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6814", 2020-11-28T21:55:52.730 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:55:52.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.730 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:52.730 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6815", 2020-11-28T21:55:52.731 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:55:52.731 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:52.731 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:52.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.731 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6811/12313", 2020-11-28T21:55:52.731 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6813/12313", 2020-11-28T21:55:52.732 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6817/12313", 2020-11-28T21:55:52.732 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6815/12313", 2020-11-28T21:55:52.732 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:55:52.732 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:55:52.732 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:55:52.733 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:52.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.733 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 1, 2020-11-28T21:55:52.733 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "6ce6220a-00ef-4d13-abba-56f87636fb5f", 2020-11-28T21:55:52.734 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:55:52.734 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:55:52.734 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:55:52.734 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:55:52.734 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:55:52.735 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:55:52.735 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:55:52.735 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 136, 2020-11-28T21:55:52.735 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:55:52.735 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:55:52.736 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:55:52.736 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:52.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.736 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:52.736 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6802", 2020-11-28T21:55:52.737 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:55:52.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.737 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:52.737 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6803", 2020-11-28T21:55:52.738 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:55:52.738 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:52.738 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:52.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.738 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:55:52.739 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:52.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.739 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:52.739 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6804", 2020-11-28T21:55:52.739 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:55:52.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.740 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:52.740 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6805", 2020-11-28T21:55:52.740 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:55:52.741 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:52.741 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:52.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.741 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:55:52.741 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:52.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.742 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:52.742 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6808", 2020-11-28T21:55:52.742 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:55:52.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.743 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:52.743 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6809", 2020-11-28T21:55:52.743 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:55:52.743 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:52.743 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:52.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.744 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:55:52.744 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:52.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.744 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:52.745 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6806", 2020-11-28T21:55:52.745 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:55:52.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.745 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:52.746 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6807", 2020-11-28T21:55:52.746 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:55:52.746 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:52.746 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:52.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.747 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6803/12311", 2020-11-28T21:55:52.747 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6805/12311", 2020-11-28T21:55:52.747 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6809/12311", 2020-11-28T21:55:52.747 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6807/12311", 2020-11-28T21:55:52.747 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:55:52.748 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:55:52.748 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:55:52.748 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:52.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.749 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 2, 2020-11-28T21:55:52.749 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "db4e510d-27c1-47fc-a902-8b32e7dfd73e", 2020-11-28T21:55:52.749 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:55:52.749 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:55:52.749 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:55:52.750 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:55:52.750 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:55:52.750 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:55:52.750 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:55:52.750 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 124, 2020-11-28T21:55:52.751 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:55:52.751 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:55:52.751 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:55:52.751 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:52.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.751 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:52.752 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6818", 2020-11-28T21:55:52.752 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:55:52.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.752 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:52.753 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6819", 2020-11-28T21:55:52.753 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:55:52.753 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:52.753 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:52.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.754 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:55:52.754 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:52.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.754 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:52.754 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6820", 2020-11-28T21:55:52.755 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:55:52.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.755 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:52.755 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6821", 2020-11-28T21:55:52.756 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:55:52.756 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:52.756 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:52.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.756 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:55:52.757 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:52.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.757 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:52.757 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6824", 2020-11-28T21:55:52.757 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:55:52.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.758 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:52.758 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6825", 2020-11-28T21:55:52.758 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:55:52.759 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:52.759 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:52.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.759 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:55:52.759 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:52.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.760 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:52.760 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6822", 2020-11-28T21:55:52.760 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:55:52.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.761 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:52.761 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6823", 2020-11-28T21:55:52.761 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:55:52.761 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:52.762 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:52.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.762 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6819/12312", 2020-11-28T21:55:52.762 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6821/12312", 2020-11-28T21:55:52.762 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6825/12312", 2020-11-28T21:55:52.763 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6823/12312", 2020-11-28T21:55:52.763 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:55:52.763 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:55:52.763 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:55:52.763 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:52.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.764 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 3, 2020-11-28T21:55:52.764 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "4094cb87-f04a-45e1-8710-fb4b651949f6", 2020-11-28T21:55:52.764 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:55:52.764 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:55:52.765 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:55:52.765 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:55:52.765 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:55:52.765 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:55:52.765 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:55:52.766 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 136, 2020-11-28T21:55:52.766 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:55:52.766 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:55:52.766 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:55:52.766 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:52.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.767 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:52.767 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6826", 2020-11-28T21:55:52.767 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:55:52.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.768 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:52.768 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6827", 2020-11-28T21:55:52.768 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:55:52.768 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:52.769 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:52.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.769 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:55:52.769 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:52.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.770 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:52.770 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6828", 2020-11-28T21:55:52.770 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:55:52.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.771 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:52.771 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6829", 2020-11-28T21:55:52.771 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:55:52.771 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:52.771 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:52.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.772 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:55:52.772 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:52.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.772 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:52.773 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6832", 2020-11-28T21:55:52.773 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:55:52.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.773 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:52.774 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6833", 2020-11-28T21:55:52.774 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:55:52.774 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:52.774 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:52.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.775 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:55:52.775 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:52.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.775 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:52.775 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6830", 2020-11-28T21:55:52.776 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:55:52.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.776 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:52.776 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6831", 2020-11-28T21:55:52.777 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:55:52.777 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:52.777 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:52.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.777 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6827/12323", 2020-11-28T21:55:52.778 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6829/12323", 2020-11-28T21:55:52.778 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6833/12323", 2020-11-28T21:55:52.778 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6831/12323", 2020-11-28T21:55:52.778 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:55:52.778 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:55:52.779 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:55:52.779 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:52.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.779 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 4, 2020-11-28T21:55:52.780 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "b907bf53-c297-4756-847b-3be96680eef5", 2020-11-28T21:55:52.780 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:55:52.780 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:55:52.780 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:55:52.780 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:55:52.781 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:55:52.781 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:55:52.781 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:55:52.781 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 136, 2020-11-28T21:55:52.781 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:55:52.781 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:55:52.782 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:55:52.782 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:52.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.782 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:52.783 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6800", 2020-11-28T21:55:52.783 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:55:52.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.784 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:52.784 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6801", 2020-11-28T21:55:52.785 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:55:52.785 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:52.785 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:52.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.786 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:55:52.786 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:52.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.787 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:52.787 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6802", 2020-11-28T21:55:52.788 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:55:52.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.789 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:52.789 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6803", 2020-11-28T21:55:52.790 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:55:52.790 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:52.791 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:52.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.791 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:55:52.792 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:52.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.793 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:52.793 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6806", 2020-11-28T21:55:52.793 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:55:52.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.794 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:52.795 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6807", 2020-11-28T21:55:52.795 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:55:52.796 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:52.796 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:52.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.797 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:55:52.797 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:52.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.798 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:52.798 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6804", 2020-11-28T21:55:52.799 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:55:52.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.800 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:52.800 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6805", 2020-11-28T21:55:52.800 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:55:52.801 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:52.801 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:52.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.802 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6801/12527", 2020-11-28T21:55:52.802 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6803/12527", 2020-11-28T21:55:52.803 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6807/12527", 2020-11-28T21:55:52.803 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6805/12527", 2020-11-28T21:55:52.803 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:55:52.804 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:55:52.804 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:55:52.804 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:52.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.805 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 5, 2020-11-28T21:55:52.806 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "55ba305d-0505-4212-aebb-631e8554b440", 2020-11-28T21:55:52.806 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:55:52.807 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:55:52.807 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:55:52.807 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:55:52.808 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:55:52.808 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:55:52.808 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:55:52.809 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 136, 2020-11-28T21:55:52.809 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:55:52.810 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:55:52.810 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:55:52.810 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:52.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.811 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:52.811 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6808", 2020-11-28T21:55:52.812 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:55:52.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.813 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:52.813 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6809", 2020-11-28T21:55:52.814 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:55:52.814 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:52.814 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:52.815 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.815 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:55:52.816 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:52.816 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.816 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:52.817 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6810", 2020-11-28T21:55:52.817 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:55:52.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.818 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.818 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:52.819 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6811", 2020-11-28T21:55:52.819 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:55:52.819 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:52.820 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:52.820 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.821 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:55:52.821 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:52.821 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.822 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:52.822 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6814", 2020-11-28T21:55:52.823 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:55:52.823 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.823 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.824 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:52.824 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6815", 2020-11-28T21:55:52.824 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:55:52.825 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:52.825 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:52.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.826 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:55:52.826 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:52.827 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.827 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:52.828 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6812", 2020-11-28T21:55:52.828 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:55:52.828 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.829 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.829 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:52.829 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6813", 2020-11-28T21:55:52.830 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:55:52.830 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:52.831 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:52.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.831 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6809/12529", 2020-11-28T21:55:52.832 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6811/12529", 2020-11-28T21:55:52.832 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6815/12529", 2020-11-28T21:55:52.833 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6813/12529", 2020-11-28T21:55:52.833 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:55:52.833 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:55:52.834 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:55:52.834 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:52.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.835 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.835 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 6, 2020-11-28T21:55:52.836 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "9ada6051-493b-46b5-ba22-1b401440490f", 2020-11-28T21:55:52.836 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:55:52.836 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:55:52.837 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:55:52.837 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:55:52.837 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:55:52.838 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:55:52.838 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:55:52.839 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 124, 2020-11-28T21:55:52.839 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:55:52.839 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:55:52.840 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:55:52.840 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:52.841 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.841 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:52.841 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6824", 2020-11-28T21:55:52.842 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:55:52.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.843 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:52.843 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6825", 2020-11-28T21:55:52.844 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:55:52.844 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:52.845 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:52.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.845 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:55:52.846 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:52.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.846 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:52.846 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6826", 2020-11-28T21:55:52.847 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:55:52.847 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.847 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.847 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:52.848 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6827", 2020-11-28T21:55:52.848 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:55:52.848 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:52.849 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:52.849 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.849 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:55:52.849 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:52.849 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.850 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:52.850 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6830", 2020-11-28T21:55:52.850 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:55:52.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.851 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:52.851 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6831", 2020-11-28T21:55:52.851 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:55:52.851 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:52.852 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:52.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.852 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:55:52.852 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:52.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.853 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:52.853 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6828", 2020-11-28T21:55:52.853 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:55:52.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.854 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.854 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:52.854 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6829", 2020-11-28T21:55:52.854 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:55:52.854 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:52.855 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:52.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.855 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6825/12530", 2020-11-28T21:55:52.855 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6827/12530", 2020-11-28T21:55:52.855 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6831/12530", 2020-11-28T21:55:52.856 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6829/12530", 2020-11-28T21:55:52.856 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:55:52.856 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:55:52.856 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:55:52.856 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:52.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.857 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 7, 2020-11-28T21:55:52.857 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "c328b06d-a216-4cba-aad7-9ad081340613", 2020-11-28T21:55:52.858 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:55:52.858 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:55:52.858 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:55:52.858 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:55:52.859 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:55:52.859 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:55:52.860 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:55:52.860 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 136, 2020-11-28T21:55:52.860 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:55:52.861 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:55:52.861 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:55:52.861 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:52.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.862 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:52.862 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6816", 2020-11-28T21:55:52.862 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:55:52.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.863 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:52.863 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6817", 2020-11-28T21:55:52.863 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:55:52.864 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:52.864 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:52.864 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.864 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:55:52.864 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:52.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.865 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:52.865 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6818", 2020-11-28T21:55:52.865 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:55:52.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.866 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:52.866 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6819", 2020-11-28T21:55:52.866 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:55:52.866 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:52.867 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:52.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.867 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:55:52.867 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:52.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.868 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:52.868 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6822", 2020-11-28T21:55:52.868 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:55:52.868 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.868 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.869 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:52.869 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6823", 2020-11-28T21:55:52.869 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:55:52.869 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:52.869 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:52.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.870 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:55:52.870 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:55:52.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.871 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:55:52.871 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6820", 2020-11-28T21:55:52.871 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:55:52.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.871 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:55:52.872 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6821", 2020-11-28T21:55:52.872 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:55:52.872 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:52.872 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:52.872 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.873 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6817/12528", 2020-11-28T21:55:52.873 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6819/12528", 2020-11-28T21:55:52.873 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6823/12528", 2020-11-28T21:55:52.873 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6821/12528", 2020-11-28T21:55:52.874 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:55:52.874 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:55:52.874 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:55:52.874 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:52.874 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:52.875 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:55:52.875 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_xinfo": [ 2020-11-28T21:55:52.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.875 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 0, 2020-11-28T21:55:52.875 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:55:52.876 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:55:52.876 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:55:52.876 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:55:52.876 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:55:52.877 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.038052+0000", 2020-11-28T21:55:52.877 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:55:52.877 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.877 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.877 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 1, 2020-11-28T21:55:52.878 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:55:52.878 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:55:52.878 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:55:52.878 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:55:52.878 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:55:52.879 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:17.947136+0000", 2020-11-28T21:55:52.879 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:55:52.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.879 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.879 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 2, 2020-11-28T21:55:52.880 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:55:52.880 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:55:52.880 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:55:52.880 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:55:52.880 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:55:52.881 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.001186+0000", 2020-11-28T21:55:52.881 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:55:52.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.882 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 3, 2020-11-28T21:55:52.882 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:55:52.882 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:55:52.882 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:55:52.882 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:55:52.883 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:55:52.883 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.199284+0000", 2020-11-28T21:55:52.883 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:55:52.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.884 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 4, 2020-11-28T21:55:52.884 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:55:52.884 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:55:52.884 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:55:52.884 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:55:52.885 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:55:52.885 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.038823+0000", 2020-11-28T21:55:52.885 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:55:52.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.886 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 5, 2020-11-28T21:55:52.886 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:55:52.886 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:55:52.886 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:55:52.886 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:55:52.887 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:55:52.887 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.009322+0000", 2020-11-28T21:55:52.887 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:55:52.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.888 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 6, 2020-11-28T21:55:52.888 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:55:52.888 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:55:52.888 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:55:52.888 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:55:52.889 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:55:52.889 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.238927+0000", 2020-11-28T21:55:52.889 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:55:52.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:52.890 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 7, 2020-11-28T21:55:52.890 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:55:52.890 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:55:52.890 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:55:52.890 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:55:52.891 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:55:52.891 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.029839+0000", 2020-11-28T21:55:52.891 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:55:52.891 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:52.891 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:55:52.892 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_upmap": [], 2020-11-28T21:55:52.892 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_upmap_items": [], 2020-11-28T21:55:52.892 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_temp": [], 2020-11-28T21:55:52.892 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_temp": [], 2020-11-28T21:55:52.892 INFO:teuthology.orchestra.run.smithi162.stdout: "blacklist": {}, 2020-11-28T21:55:52.893 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profiles": { 2020-11-28T21:55:52.893 INFO:teuthology.orchestra.run.smithi162.stdout: "default": { 2020-11-28T21:55:52.893 INFO:teuthology.orchestra.run.smithi162.stdout: "crush-failure-domain": "osd", 2020-11-28T21:55:52.893 INFO:teuthology.orchestra.run.smithi162.stdout: "k": "2", 2020-11-28T21:55:52.893 INFO:teuthology.orchestra.run.smithi162.stdout: "m": "1", 2020-11-28T21:55:52.894 INFO:teuthology.orchestra.run.smithi162.stdout: "plugin": "jerasure", 2020-11-28T21:55:52.894 INFO:teuthology.orchestra.run.smithi162.stdout: "ruleset-failure-domain": "osd", 2020-11-28T21:55:52.894 INFO:teuthology.orchestra.run.smithi162.stdout: "technique": "reed_sol_van" 2020-11-28T21:55:52.894 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:52.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:52.898 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps_queue": [], 2020-11-28T21:55:52.898 INFO:teuthology.orchestra.run.smithi162.stdout: "new_removed_snaps": [], 2020-11-28T21:55:52.898 INFO:teuthology.orchestra.run.smithi162.stdout: "new_purged_snaps": [], 2020-11-28T21:55:52.899 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_node_flags": {}, 2020-11-28T21:55:52.899 INFO:teuthology.orchestra.run.smithi162.stdout: "device_class_flags": {} 2020-11-28T21:55:52.899 INFO:teuthology.orchestra.run.smithi162.stdout:} 2020-11-28T21:55:52.901 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.650+0000 7fa4effff700 1 -- 172.21.15.162:0/2128620389 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa4cc055330 msgr2=0x7fa4cc0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:52.901 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.650+0000 7fa4effff700 1 --2- 172.21.15.162:0/2128620389 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa4cc055330 0x7fa4cc0577f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7fa4e401c000 tx=0x7fa4e4015000).stop 2020-11-28T21:55:52.901 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.650+0000 7fa4effff700 1 -- 172.21.15.162:0/2128620389 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa4f0066a10 msgr2=0x7fa4f013bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:52.901 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.650+0000 7fa4effff700 1 --2- 172.21.15.162:0/2128620389 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa4f0066a10 0x7fa4f013bf40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7fa4e000c000 tx=0x7fa4e0006dc0).stop 2020-11-28T21:55:52.902 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.650+0000 7fa4effff700 1 -- 172.21.15.162:0/2128620389 shutdown_connections 2020-11-28T21:55:52.902 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.650+0000 7fa4effff700 1 --2- 172.21.15.162:0/2128620389 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa4cc055330 0x7fa4cc0577f0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:52.902 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.650+0000 7fa4effff700 1 --2- 172.21.15.162:0/2128620389 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa4f0066a10 0x7fa4f013bf40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:52.902 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.650+0000 7fa4effff700 1 --2- 172.21.15.162:0/2128620389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4f0143da0 0x7fa4f0135c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:52.903 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.650+0000 7fa4effff700 1 --2- 172.21.15.162:0/2128620389 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa4f00b1500 0x7fa4f0135720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:52.903 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.650+0000 7fa4effff700 1 -- 172.21.15.162:0/2128620389 >> 172.21.15.162:0/2128620389 conn(0x7fa4f00b5da0 msgr2=0x7fa4f013d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:52.903 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.650+0000 7fa4effff700 1 -- 172.21.15.162:0/2128620389 shutdown_connections 2020-11-28T21:55:52.903 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:52.650+0000 7fa4effff700 1 -- 172.21.15.162:0/2128620389 wait complete. 2020-11-28T21:55:52.904 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:55:53.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.018+0000 7f8362ecf700 1 Processor -- start 2020-11-28T21:55:53.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.018+0000 7f8362ecf700 1 -- start start 2020-11-28T21:55:53.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.018+0000 7f8362ecf700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f835c0b1990 0x7f835c140890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:53.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.018+0000 7f8362ecf700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f835c0b11b0 0x7f835c140d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:53.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.018+0000 7f8362ecf700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f835c0b2870 con 0x7f835c0b1d70 2020-11-28T21:55:53.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.018+0000 7f8362ecf700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f835c146e20 con 0x7f835c0b1990 2020-11-28T21:55:53.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.018+0000 7f8362ecf700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f835c0b22e0 con 0x7f835c0b11b0 2020-11-28T21:55:53.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.018+0000 7f835bfff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f835c0b11b0 0x7f835c140d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:53.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.018+0000 7f835bfff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f835c0b11b0 0x7f835c140d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45370/0 (socket says 172.21.15.162:45370) 2020-11-28T21:55:53.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.018+0000 7f8360c6b700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f835c0b1990 0x7f835c140890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:53.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.018+0000 7f835bfff700 1 -- 172.21.15.162:0/3587983931 learned_addr learned my addr 172.21.15.162:0/3587983931 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:53.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.018+0000 7f835bfff700 1 -- 172.21.15.162:0/3587983931 >> v2:172.21.15.118:3300/0 conn(0x7f835c0b1990 msgr2=0x7f835c140890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:53.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.018+0000 7f835bfff700 1 --2- 172.21.15.162:0/3587983931 >> v2:172.21.15.118:3300/0 conn(0x7f835c0b1990 0x7f835c140890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:53.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.018+0000 7f835bfff700 1 -- 172.21.15.162:0/3587983931 >> v1:172.21.15.118:6789/0 conn(0x7f835c0b1d70 legacy=0x7f835c14a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:53.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.018+0000 7f835bfff700 1 -- 172.21.15.162:0/3587983931 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f835c0b0740 con 0x7f835c0b11b0 2020-11-28T21:55:53.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.018+0000 7f835bfff700 1 --2- 172.21.15.162:0/3587983931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f835c0b11b0 0x7f835c140d80 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 rx=0x7f834c002f30 tx=0x7f834c006e80).ready entity=mon.1 client_cookie=1f848f77f1bc42ab server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:53.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.018+0000 7f835b7fe700 1 -- 172.21.15.162:0/3587983931 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f834c0097f0 con 0x7f835c0b11b0 2020-11-28T21:55:53.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.018+0000 7f835b7fe700 1 -- 172.21.15.162:0/3587983931 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f834c009950 con 0x7f835c0b11b0 2020-11-28T21:55:53.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.018+0000 7f835b7fe700 1 -- 172.21.15.162:0/3587983931 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f834c009cb0 con 0x7f835c0b11b0 2020-11-28T21:55:53.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.018+0000 7f8362ecf700 1 -- 172.21.15.162:0/3587983931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f835c0b11b0 msgr2=0x7f835c140d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:53.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.018+0000 7f8362ecf700 1 --2- 172.21.15.162:0/3587983931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f835c0b11b0 0x7f835c140d80 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 rx=0x7f834c002f30 tx=0x7f834c006e80).stop 2020-11-28T21:55:53.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.022+0000 7f8362ecf700 1 -- 172.21.15.162:0/3587983931 shutdown_connections 2020-11-28T21:55:53.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.022+0000 7f8362ecf700 1 --2- 172.21.15.162:0/3587983931 >> v2:172.21.15.118:3300/0 conn(0x7f835c0b1990 0x7f835c140890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:53.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.022+0000 7f8362ecf700 1 --2- 172.21.15.162:0/3587983931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f835c0b11b0 0x7f835c140d80 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:53.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.022+0000 7f8362ecf700 1 -- 172.21.15.162:0/3587983931 >> 172.21.15.162:0/3587983931 conn(0x7f835c05b970 msgr2=0x7f835c05bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:53.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.022+0000 7f8362ecf700 1 -- 172.21.15.162:0/3587983931 shutdown_connections 2020-11-28T21:55:53.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.022+0000 7f8362ecf700 1 -- 172.21.15.162:0/3587983931 wait complete. 2020-11-28T21:55:53.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.022+0000 7f8362ecf700 1 Processor -- start 2020-11-28T21:55:53.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.022+0000 7f8362ecf700 1 -- start start 2020-11-28T21:55:53.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.022+0000 7f8362ecf700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f835c0b11b0 0x7f835c0b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:53.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.022+0000 7f8362ecf700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f835c0b1990 0x7f835c063330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:53.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.022+0000 7f8362ecf700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f835c0b1d70 0x7f835c13b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:53.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.022+0000 7f8362ecf700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835c0aec10 con 0x7f835c0b1990 2020-11-28T21:55:53.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.022+0000 7f8362ecf700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835c0aeeb0 con 0x7f835c0b1d70 2020-11-28T21:55:53.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.022+0000 7f8362ecf700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f835c0af150 con 0x7f835c0b11b0 2020-11-28T21:55:53.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.022+0000 7f836146c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f835c0b1d70 0x7f835c13b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:53.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.022+0000 7f836146c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f835c0b1d70 0x7f835c13b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45376/0 (socket says 172.21.15.162:45376) 2020-11-28T21:55:53.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.022+0000 7f836146c700 1 -- 172.21.15.162:0/3879679665 learned_addr learned my addr 172.21.15.162:0/3879679665 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:53.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.022+0000 7f8360c6b700 1 --2- 172.21.15.162:0/3879679665 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f835c0b11b0 0x7f835c0b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:53.035 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.022+0000 7f835bfff700 1 --2- 172.21.15.162:0/3879679665 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f835c0b1990 0x7f835c063330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:53.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.022+0000 7f836146c700 1 -- 172.21.15.162:0/3879679665 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f835c0b11b0 msgr2=0x7f835c0b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:53.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.022+0000 7f836146c700 1 --2- 172.21.15.162:0/3879679665 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f835c0b11b0 0x7f835c0b3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:53.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.022+0000 7f836146c700 1 -- 172.21.15.162:0/3879679665 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f835c0b1990 msgr2=0x7f835c063330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:53.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.022+0000 7f836146c700 1 --2- 172.21.15.162:0/3879679665 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f835c0b1990 0x7f835c063330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:53.036 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.022+0000 7f836146c700 1 -- 172.21.15.162:0/3879679665 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f835c0af3f0 con 0x7f835c0b1d70 2020-11-28T21:55:53.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.026+0000 7f836146c700 1 --2- 172.21.15.162:0/3879679665 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f835c0b1d70 0x7f835c13b8c0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 rx=0x7f835c14a650 tx=0x7f8344007bf0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:53.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.026+0000 7f8359ffb700 1 -- 172.21.15.162:0/3879679665 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f834400e4a0 con 0x7f835c0b1d70 2020-11-28T21:55:53.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.026+0000 7f8359ffb700 1 -- 172.21.15.162:0/3879679665 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f834400e600 con 0x7f835c0b1d70 2020-11-28T21:55:53.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.026+0000 7f8359ffb700 1 -- 172.21.15.162:0/3879679665 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f834400e960 con 0x7f835c0b1d70 2020-11-28T21:55:53.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.026+0000 7f8362ecf700 1 -- 172.21.15.162:0/3879679665 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f835c0b46f0 con 0x7f835c0b1d70 2020-11-28T21:55:53.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.026+0000 7f8362ecf700 1 -- 172.21.15.162:0/3879679665 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f835c13bdb0 con 0x7f835c0b1d70 2020-11-28T21:55:53.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.026+0000 7f8362ecf700 1 -- 172.21.15.162:0/3879679665 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f835c0af580 con 0x7f835c0b1d70 2020-11-28T21:55:53.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.026+0000 7f8359ffb700 1 -- 172.21.15.162:0/3879679665 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8344010030 con 0x7f835c0b1d70 2020-11-28T21:55:53.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.026+0000 7f8359ffb700 1 --2- 172.21.15.162:0/3879679665 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8334055500 0x7f83340579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:53.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.026+0000 7f8360c6b700 1 --2- 172.21.15.162:0/3879679665 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8334055500 0x7f83340579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:53.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.026+0000 7f8359ffb700 1 -- 172.21.15.162:0/3879679665 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f834406dd10 con 0x7f835c0b1d70 2020-11-28T21:55:53.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.030+0000 7f8360c6b700 1 --2- 172.21.15.162:0/3879679665 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8334055500 0x7f83340579c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7f835400b000 tx=0x7f8354008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:53.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.030+0000 7f8359ffb700 1 -- 172.21.15.162:0/3879679665 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8344044010 con 0x7f835c0b1d70 2020-11-28T21:55:53.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.218+0000 7f8362ecf700 1 -- 172.21.15.162:0/3879679665 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f835c057050 con 0x7f835c0b1d70 2020-11-28T21:55:53.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.218+0000 7f8359ffb700 1 -- 172.21.15.162:0/3879679665 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+2504 (secure 0 0 0) 0x7f8344047630 con 0x7f835c0b1d70 2020-11-28T21:55:53.224 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:53.224 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":68,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7611,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6835/2127102267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2127102267},{"type":"v1","addr":"172.21.15.162:6835","nonce":2127102267}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":66},{"gid":7623,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/1473328171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1473328171},{"type":"v1","addr":"172.21.15.118:6833","nonce":1473328171}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:55:31.379864+0000","modified":"2020-11-28T21:55:33.042143+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7888":{"gid":7888,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/2667777645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2667777645},{"type":"v1","addr":"172.21.15.162:6837","nonce":2667777645}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2020-11-28T21:55:53.225 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.218+0000 7f8362ecf700 1 -- 172.21.15.162:0/3879679665 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8334055500 msgr2=0x7f83340579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:53.226 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.218+0000 7f8362ecf700 1 --2- 172.21.15.162:0/3879679665 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8334055500 0x7f83340579c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7f835400b000 tx=0x7f8354008040).stop 2020-11-28T21:55:53.226 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.218+0000 7f8362ecf700 1 -- 172.21.15.162:0/3879679665 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f835c0b1d70 msgr2=0x7f835c13b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:53.226 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.218+0000 7f8362ecf700 1 --2- 172.21.15.162:0/3879679665 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f835c0b1d70 0x7f835c13b8c0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 rx=0x7f835c14a650 tx=0x7f8344007bf0).stop 2020-11-28T21:55:53.226 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.222+0000 7f8362ecf700 1 -- 172.21.15.162:0/3879679665 shutdown_connections 2020-11-28T21:55:53.227 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.222+0000 7f8362ecf700 1 --2- 172.21.15.162:0/3879679665 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8334055500 0x7f83340579c0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:53.227 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.222+0000 7f8362ecf700 1 --2- 172.21.15.162:0/3879679665 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f835c0b11b0 0x7f835c0b3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:53.227 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.222+0000 7f8362ecf700 1 --2- 172.21.15.162:0/3879679665 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f835c0b1d70 0x7f835c13b8c0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:53.227 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.222+0000 7f8362ecf700 1 --2- 172.21.15.162:0/3879679665 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f835c0b1990 0x7f835c063330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:53.227 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.222+0000 7f8362ecf700 1 -- 172.21.15.162:0/3879679665 >> 172.21.15.162:0/3879679665 conn(0x7f835c05b970 msgr2=0x7f835c144e00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:53.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.222+0000 7f8362ecf700 1 -- 172.21.15.162:0/3879679665 shutdown_connections 2020-11-28T21:55:53.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.222+0000 7f8362ecf700 1 -- 172.21.15.162:0/3879679665 wait complete. 2020-11-28T21:55:53.228 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 68 2020-11-28T21:55:53.237 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2020-11-28T21:55:53.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.342+0000 7faee2a84700 1 Processor -- start 2020-11-28T21:55:53.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.342+0000 7faee2a84700 1 -- start start 2020-11-28T21:55:53.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.346+0000 7faee2a84700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7faedc098b30 0x7faedc0994f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:53.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.346+0000 7faee2a84700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faedc0b0000 0x7faedc060000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:53.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.346+0000 7faee2a84700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7faedc0604f0 0x7faedc060980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:53.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.346+0000 7faee2a84700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faedc0b1030 con 0x7faedc0604f0 2020-11-28T21:55:53.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.346+0000 7faee2a84700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faedc0b0480 con 0x7faedc098b30 2020-11-28T21:55:53.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.346+0000 7faee2a84700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faedc098f10 con 0x7faedc0b0000 2020-11-28T21:55:53.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.346+0000 7faee1a82700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7faedc098b30 0x7faedc0994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:53.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.346+0000 7faee1a82700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7faedc098b30 0x7faedc0994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45380/0 (socket says 172.21.15.162:45380) 2020-11-28T21:55:53.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.346+0000 7faee1a82700 1 -- 172.21.15.162:0/3821429155 learned_addr learned my addr 172.21.15.162:0/3821429155 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:53.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.346+0000 7faee1281700 1 --2- 172.21.15.162:0/3821429155 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faedc0b0000 0x7faedc060000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:53.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.346+0000 7faee2283700 1 --2- 172.21.15.162:0/3821429155 >> v2:172.21.15.118:3300/0 conn(0x7faedc0604f0 0x7faedc060980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:53.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.346+0000 7faee1a82700 1 -- 172.21.15.162:0/3821429155 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faedc0b0000 msgr2=0x7faedc060000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:53.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.346+0000 7faee1a82700 1 --2- 172.21.15.162:0/3821429155 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faedc0b0000 0x7faedc060000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:53.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.346+0000 7faee1a82700 1 -- 172.21.15.162:0/3821429155 >> v2:172.21.15.118:3300/0 conn(0x7faedc0604f0 msgr2=0x7faedc060980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:53.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.346+0000 7faee1a82700 1 --2- 172.21.15.162:0/3821429155 >> v2:172.21.15.118:3300/0 conn(0x7faedc0604f0 0x7faedc060980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:53.355 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.346+0000 7faee1a82700 1 -- 172.21.15.162:0/3821429155 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faedc0b0680 con 0x7faedc098b30 2020-11-28T21:55:53.355 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.346+0000 7faee1a82700 1 --2- 172.21.15.162:0/3821429155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faedc098b30 0x7faedc0994f0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 rx=0x7faedc099be0 tx=0x7faec4006e80).ready entity=mon.1 client_cookie=a3f192190199bb95 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:53.355 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.346+0000 7faee0a80700 1 -- 172.21.15.162:0/3821429155 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faec4009600 con 0x7faedc098b30 2020-11-28T21:55:53.355 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.346+0000 7faee0a80700 1 -- 172.21.15.162:0/3821429155 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec4009760 con 0x7faedc098b30 2020-11-28T21:55:53.355 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.346+0000 7faee0a80700 1 -- 172.21.15.162:0/3821429155 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faec4009ac0 con 0x7faedc098b30 2020-11-28T21:55:53.356 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.346+0000 7faee2a84700 1 -- 172.21.15.162:0/3821429155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faedc098b30 msgr2=0x7faedc0994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:53.356 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.346+0000 7faee2a84700 1 --2- 172.21.15.162:0/3821429155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faedc098b30 0x7faedc0994f0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 rx=0x7faedc099be0 tx=0x7faec4006e80).stop 2020-11-28T21:55:53.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.346+0000 7faee2a84700 1 -- 172.21.15.162:0/3821429155 shutdown_connections 2020-11-28T21:55:53.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.346+0000 7faee2a84700 1 --2- 172.21.15.162:0/3821429155 >> v2:172.21.15.118:3300/0 conn(0x7faedc0604f0 0x7faedc060980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:53.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.346+0000 7faee2a84700 1 --2- 172.21.15.162:0/3821429155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faedc098b30 0x7faedc0994f0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:53.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.346+0000 7faee2a84700 1 --2- 172.21.15.162:0/3821429155 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faedc0b0000 0x7faedc060000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:53.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.346+0000 7faee2a84700 1 -- 172.21.15.162:0/3821429155 >> 172.21.15.162:0/3821429155 conn(0x7faedc05b970 msgr2=0x7faedc05bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:53.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.346+0000 7faee2a84700 1 -- 172.21.15.162:0/3821429155 shutdown_connections 2020-11-28T21:55:53.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.346+0000 7faee2a84700 1 -- 172.21.15.162:0/3821429155 wait complete. 2020-11-28T21:55:53.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.350+0000 7faee2a84700 1 Processor -- start 2020-11-28T21:55:53.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.350+0000 7faee2a84700 1 -- start start 2020-11-28T21:55:53.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.350+0000 7faee2a84700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faedc0604f0 0x7faedc0adf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:53.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.350+0000 7faee2a84700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faedc098b30 0x7faedc063ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:53.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.350+0000 7faee2a84700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faedc0b0000 0x7faedc155b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:53.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.350+0000 7faee2a84700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faedc063fa0 con 0x7faedc098b30 2020-11-28T21:55:53.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.350+0000 7faee2a84700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faedc0633e0 con 0x7faedc0604f0 2020-11-28T21:55:53.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.350+0000 7faee2a84700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faedc0aea90 con 0x7faedc0b0000 2020-11-28T21:55:53.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.350+0000 7faee1a82700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faedc0604f0 0x7faedc0adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:53.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.350+0000 7faee1a82700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faedc0604f0 0x7faedc0adf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45384/0 (socket says 172.21.15.162:45384) 2020-11-28T21:55:53.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.350+0000 7faee1a82700 1 -- 172.21.15.162:0/1235208423 learned_addr learned my addr 172.21.15.162:0/1235208423 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:53.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.350+0000 7faee2283700 1 --2- 172.21.15.162:0/1235208423 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faedc0b0000 0x7faedc155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:53.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.350+0000 7faee1281700 1 --2- 172.21.15.162:0/1235208423 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faedc098b30 0x7faedc063ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:53.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.350+0000 7faee1a82700 1 -- 172.21.15.162:0/1235208423 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faedc0b0000 msgr2=0x7faedc155b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:53.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.350+0000 7faee1a82700 1 --2- 172.21.15.162:0/1235208423 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faedc0b0000 0x7faedc155b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:53.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.350+0000 7faee1a82700 1 -- 172.21.15.162:0/1235208423 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faedc098b30 msgr2=0x7faedc063ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:53.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.350+0000 7faee1a82700 1 --2- 172.21.15.162:0/1235208423 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faedc098b30 0x7faedc063ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:53.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.350+0000 7faee1a82700 1 -- 172.21.15.162:0/1235208423 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faedc063610 con 0x7faedc0604f0 2020-11-28T21:55:53.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.350+0000 7faee2283700 1 --2- 172.21.15.162:0/1235208423 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faedc0b0000 0x7faedc155b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:55:53.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.350+0000 7faee1a82700 1 --2- 172.21.15.162:0/1235208423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faedc0604f0 0x7faedc0adf50 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 rx=0x7faedc099be0 tx=0x7faec4015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:53.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.350+0000 7faedaffd700 1 -- 172.21.15.162:0/1235208423 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faec4011b00 con 0x7faedc0604f0 2020-11-28T21:55:53.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.350+0000 7faedaffd700 1 -- 172.21.15.162:0/1235208423 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec4011c60 con 0x7faedc0604f0 2020-11-28T21:55:53.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.350+0000 7faee2a84700 1 -- 172.21.15.162:0/1235208423 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faedc0aed30 con 0x7faedc0604f0 2020-11-28T21:55:53.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.350+0000 7faedaffd700 1 -- 172.21.15.162:0/1235208423 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faec401e460 con 0x7faedc0604f0 2020-11-28T21:55:53.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.350+0000 7faee2a84700 1 -- 172.21.15.162:0/1235208423 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faedc0ad4a0 con 0x7faedc0604f0 2020-11-28T21:55:53.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.354+0000 7faee2a84700 1 -- 172.21.15.162:0/1235208423 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faedc057050 con 0x7faedc0604f0 2020-11-28T21:55:53.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.354+0000 7faedaffd700 1 -- 172.21.15.162:0/1235208423 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7faec4025030 con 0x7faedc0604f0 2020-11-28T21:55:53.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.354+0000 7faedaffd700 1 --2- 172.21.15.162:0/1235208423 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7faec00553b0 0x7faec0057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:53.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.354+0000 7faedaffd700 1 -- 172.21.15.162:0/1235208423 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6073+0+0 (secure 0 0 0) 0x7faec406f270 con 0x7faedc0604f0 2020-11-28T21:55:53.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.354+0000 7faee1281700 1 --2- 172.21.15.162:0/1235208423 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7faec00553b0 0x7faec0057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:53.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.354+0000 7faee1281700 1 --2- 172.21.15.162:0/1235208423 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7faec00553b0 0x7faec0057870 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7faed400b000 tx=0x7faed4008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:53.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.358+0000 7faedaffd700 1 -- 172.21.15.162:0/1235208423 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7faec40454d0 con 0x7faedc0604f0 2020-11-28T21:55:53.547 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.542+0000 7faee2a84700 1 -- 172.21.15.162:0/1235208423 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7faedc068350 con 0x7faedc0604f0 2020-11-28T21:55:53.885 INFO:tasks.ceph.mon.b.smithi118.stderr:2020-11-28T21:55:53.880+0000 7ff4e1520700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-11-28T21:55:53.900 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.894+0000 7faedaffd700 1 -- 172.21.15.162:0/1235208423 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v69) v1 ==== 161+0+0 (secure 0 0 0) 0x7faec4023090 con 0x7faedc0604f0 2020-11-28T21:55:53.904 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.898+0000 7faee2a84700 1 -- 172.21.15.162:0/1235208423 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7faec00553b0 msgr2=0x7faec0057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:53.904 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.898+0000 7faee2a84700 1 --2- 172.21.15.162:0/1235208423 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7faec00553b0 0x7faec0057870 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7faed400b000 tx=0x7faed4008040).stop 2020-11-28T21:55:53.904 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.898+0000 7faee2a84700 1 -- 172.21.15.162:0/1235208423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faedc0604f0 msgr2=0x7faedc0adf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:53.904 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.898+0000 7faee2a84700 1 --2- 172.21.15.162:0/1235208423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faedc0604f0 0x7faedc0adf50 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 rx=0x7faedc099be0 tx=0x7faec4015000).stop 2020-11-28T21:55:53.905 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.898+0000 7faee2a84700 1 -- 172.21.15.162:0/1235208423 shutdown_connections 2020-11-28T21:55:53.905 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.898+0000 7faee2a84700 1 --2- 172.21.15.162:0/1235208423 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7faec00553b0 0x7faec0057870 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:53.905 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.898+0000 7faee2a84700 1 --2- 172.21.15.162:0/1235208423 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faedc0b0000 0x7faedc155b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:53.905 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.898+0000 7faee2a84700 1 --2- 172.21.15.162:0/1235208423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faedc0604f0 0x7faedc0adf50 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:53.906 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.898+0000 7faee2a84700 1 --2- 172.21.15.162:0/1235208423 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faedc098b30 0x7faedc063ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:53.906 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.898+0000 7faee2a84700 1 -- 172.21.15.162:0/1235208423 >> 172.21.15.162:0/1235208423 conn(0x7faedc05b970 msgr2=0x7faedc0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:53.906 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.898+0000 7faee2a84700 1 -- 172.21.15.162:0/1235208423 shutdown_connections 2020-11-28T21:55:53.906 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:53.898+0000 7faee2a84700 1 -- 172.21.15.162:0/1235208423 wait complete. 2020-11-28T21:55:53.907 INFO:teuthology.orchestra.run.smithi162.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-11-28T21:55:53.921 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:55:53.930 INFO:tasks.ceph.mds.c.smithi162.stdout:starting mds.c at 2020-11-28T21:55:54.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.022+0000 7f18a8f46700 1 Processor -- start 2020-11-28T21:55:54.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.022+0000 7f18a8f46700 1 -- start start 2020-11-28T21:55:54.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.022+0000 7f18a8f46700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f18a40611e0 0x7f18a40615c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:54.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.022+0000 7f18a8f46700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f18a40b10f0 0x7f18a40b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:54.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.022+0000 7f18a8f46700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18a40b1790 con 0x7f18a40b4920 2020-11-28T21:55:54.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.022+0000 7f18a8f46700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18a40b1580 con 0x7f18a40b10f0 2020-11-28T21:55:54.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.022+0000 7f18a8f46700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18a40b1a30 con 0x7f18a40611e0 2020-11-28T21:55:54.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.022+0000 7f18a3fff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f18a40b4920 0x7f18a413ee60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:52252/0 (socket says 172.21.15.162:52252) 2020-11-28T21:55:54.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.022+0000 7f18a3fff700 1 -- 172.21.15.162:0/3439784320 learned_addr learned my addr 172.21.15.162:0/3439784320 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:54.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.022+0000 7f18a37fe700 1 --2- 172.21.15.162:0/3439784320 >> v2:172.21.15.162:3300/0 conn(0x7f18a40611e0 0x7f18a40615c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:54.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.026+0000 7f18a2ffd700 1 --2- 172.21.15.162:0/3439784320 >> v2:172.21.15.118:3300/0 conn(0x7f18a40b10f0 0x7f18a40b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:54.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.026+0000 7f18a37fe700 1 -- 172.21.15.162:0/3439784320 >> v2:172.21.15.118:3300/0 conn(0x7f18a40b10f0 msgr2=0x7f18a40b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:54.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.026+0000 7f18a37fe700 1 --2- 172.21.15.162:0/3439784320 >> v2:172.21.15.118:3300/0 conn(0x7f18a40b10f0 0x7f18a40b4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:54.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.026+0000 7f18a37fe700 1 -- 172.21.15.162:0/3439784320 >> v1:172.21.15.162:6789/0 conn(0x7f18a40b4920 legacy=0x7f18a413ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:54.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.026+0000 7f18a27fc700 1 -- 172.21.15.162:0/3439784320 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 90715327 0 0) 0x7f18a40b1790 con 0x7f18a40b4920 2020-11-28T21:55:54.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.026+0000 7f18a37fe700 1 -- 172.21.15.162:0/3439784320 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a40b3f90 con 0x7f18a40611e0 2020-11-28T21:55:54.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.026+0000 7f18a37fe700 1 --2- 172.21.15.162:0/3439784320 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18a40611e0 0x7f18a40615c0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 rx=0x7f1894002f30 tx=0x7f1894006dc0).ready entity=mon.1 client_cookie=a34630761d1918c7 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:54.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.026+0000 7f18a27fc700 1 -- 172.21.15.162:0/3439784320 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18940097f0 con 0x7f18a40611e0 2020-11-28T21:55:54.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.026+0000 7f18a27fc700 1 -- 172.21.15.162:0/3439784320 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1894009950 con 0x7f18a40611e0 2020-11-28T21:55:54.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.026+0000 7f18a27fc700 1 -- 172.21.15.162:0/3439784320 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1894009cb0 con 0x7f18a40611e0 2020-11-28T21:55:54.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.026+0000 7f18a8f46700 1 -- 172.21.15.162:0/3439784320 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18a40611e0 msgr2=0x7f18a40615c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:54.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.026+0000 7f18a8f46700 1 --2- 172.21.15.162:0/3439784320 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18a40611e0 0x7f18a40615c0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 rx=0x7f1894002f30 tx=0x7f1894006dc0).stop 2020-11-28T21:55:54.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.026+0000 7f18a8f46700 1 -- 172.21.15.162:0/3439784320 shutdown_connections 2020-11-28T21:55:54.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.026+0000 7f18a8f46700 1 --2- 172.21.15.162:0/3439784320 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18a40611e0 0x7f18a40615c0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:54.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.026+0000 7f18a8f46700 1 --2- 172.21.15.162:0/3439784320 >> v2:172.21.15.118:3300/0 conn(0x7f18a40b10f0 0x7f18a40b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:54.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.026+0000 7f18a8f46700 1 -- 172.21.15.162:0/3439784320 >> 172.21.15.162:0/3439784320 conn(0x7f18a40b54f0 msgr2=0x7f18a40b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:54.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.026+0000 7f18a8f46700 1 -- 172.21.15.162:0/3439784320 shutdown_connections 2020-11-28T21:55:54.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.026+0000 7f18a8f46700 1 -- 172.21.15.162:0/3439784320 wait complete. 2020-11-28T21:55:54.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.030+0000 7f18a8f46700 1 Processor -- start 2020-11-28T21:55:54.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.030+0000 7f18a8f46700 1 -- start start 2020-11-28T21:55:54.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.030+0000 7f18a8f46700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f18a40611e0 0x7f18a413bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:54.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.030+0000 7f18a8f46700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f18a40b10f0 0x7f18a4065c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:54.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.030+0000 7f18a8f46700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18a40b4920 0x7f18a4066130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:54.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.030+0000 7f18a8f46700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a4065870 con 0x7f18a40b10f0 2020-11-28T21:55:54.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.030+0000 7f18a8f46700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a4137500 con 0x7f18a40b4920 2020-11-28T21:55:54.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.030+0000 7f18a8f46700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18a41377a0 con 0x7f18a40611e0 2020-11-28T21:55:54.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.030+0000 7f18a3fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18a40b4920 0x7f18a4066130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:54.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.030+0000 7f18a3fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18a40b4920 0x7f18a4066130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45414/0 (socket says 172.21.15.162:45414) 2020-11-28T21:55:54.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.030+0000 7f18a3fff700 1 -- 172.21.15.162:0/2374657405 learned_addr learned my addr 172.21.15.162:0/2374657405 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:54.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.030+0000 7f18a3fff700 1 -- 172.21.15.162:0/2374657405 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f18a40611e0 msgr2=0x7f18a413bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:54.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.030+0000 7f18a3fff700 1 --2- 172.21.15.162:0/2374657405 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f18a40611e0 0x7f18a413bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-28T21:55:54.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.030+0000 7f18a3fff700 1 -- 172.21.15.162:0/2374657405 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f18a40b10f0 msgr2=0x7f18a4065c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:54.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.030+0000 7f18a3fff700 1 --2- 172.21.15.162:0/2374657405 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f18a40b10f0 0x7f18a4065c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-28T21:55:54.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.030+0000 7f18a3fff700 1 -- 172.21.15.162:0/2374657405 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a413b530 con 0x7f18a40b4920 2020-11-28T21:55:54.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.030+0000 7f18a3fff700 1 --2- 172.21.15.162:0/2374657405 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18a40b4920 0x7f18a4066130 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 rx=0x7f189c000f00 tx=0x7f189c007f70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:54.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.030+0000 7f18a0ff9700 1 -- 172.21.15.162:0/2374657405 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f189c005b70 con 0x7f18a40b4920 2020-11-28T21:55:54.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.030+0000 7f18a0ff9700 1 -- 172.21.15.162:0/2374657405 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f189c005cd0 con 0x7f18a40b4920 2020-11-28T21:55:54.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.030+0000 7f18a8f46700 1 -- 172.21.15.162:0/2374657405 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18a409b440 con 0x7f18a40b4920 2020-11-28T21:55:54.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.030+0000 7f18a0ff9700 1 -- 172.21.15.162:0/2374657405 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f189c007300 con 0x7f18a40b4920 2020-11-28T21:55:54.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.030+0000 7f18a8f46700 1 -- 172.21.15.162:0/2374657405 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18a4137e60 con 0x7f18a40b4920 2020-11-28T21:55:54.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.034+0000 7f18a0ff9700 1 -- 172.21.15.162:0/2374657405 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f189c010030 con 0x7f18a40b4920 2020-11-28T21:55:54.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.034+0000 7f18a8f46700 1 -- 172.21.15.162:0/2374657405 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18a4065a40 con 0x7f18a40b4920 2020-11-28T21:55:54.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.034+0000 7f18a0ff9700 1 --2- 172.21.15.162:0/2374657405 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1884055440 0x7f1884057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:54.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.034+0000 7f18a0ff9700 1 -- 172.21.15.162:0/2374657405 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f189c06e0d0 con 0x7f18a40b4920 2020-11-28T21:55:54.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.034+0000 7f18a37fe700 1 --2- 172.21.15.162:0/2374657405 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1884055440 0x7f1884057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:54.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.034+0000 7f18a37fe700 1 --2- 172.21.15.162:0/2374657405 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1884055440 0x7f1884057900 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7f1894002f30 tx=0x7f1894015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:54.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.038+0000 7f18a0ff9700 1 -- 172.21.15.162:0/2374657405 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f189c044210 con 0x7f18a40b4920 2020-11-28T21:55:54.225 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.218+0000 7f18a8f46700 1 -- 172.21.15.162:0/2374657405 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18a4137960 con 0x7f18a40b4920 2020-11-28T21:55:54.225 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.218+0000 7f18a0ff9700 1 -- 172.21.15.162:0/2374657405 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+2141 (secure 0 0 0) 0x7f189c047830 con 0x7f18a40b4920 2020-11-28T21:55:54.226 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:54.226 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":69,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7611,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6835/2127102267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2127102267},{"type":"v1","addr":"172.21.15.162:6835","nonce":2127102267}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":66},{"gid":7623,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/1473328171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1473328171},{"type":"v1","addr":"172.21.15.118:6833","nonce":1473328171}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65}],"filesystems":[{"mdsmap":{"epoch":69,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:55:31.379864+0000","modified":"2020-11-28T21:55:53.885713+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":139,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2020-11-28T21:55:54.227 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.222+0000 7f18a8f46700 1 -- 172.21.15.162:0/2374657405 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1884055440 msgr2=0x7f1884057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:54.227 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.222+0000 7f18a8f46700 1 --2- 172.21.15.162:0/2374657405 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1884055440 0x7f1884057900 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7f1894002f30 tx=0x7f1894015000).stop 2020-11-28T21:55:54.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.222+0000 7f18a8f46700 1 -- 172.21.15.162:0/2374657405 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18a40b4920 msgr2=0x7f18a4066130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:54.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.222+0000 7f18a8f46700 1 --2- 172.21.15.162:0/2374657405 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18a40b4920 0x7f18a4066130 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 rx=0x7f189c000f00 tx=0x7f189c007f70).stop 2020-11-28T21:55:54.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.222+0000 7f18a8f46700 1 -- 172.21.15.162:0/2374657405 shutdown_connections 2020-11-28T21:55:54.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.222+0000 7f18a8f46700 1 --2- 172.21.15.162:0/2374657405 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1884055440 0x7f1884057900 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:54.229 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.222+0000 7f18a8f46700 1 --2- 172.21.15.162:0/2374657405 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f18a40611e0 0x7f18a413bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-28T21:55:54.229 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.222+0000 7f18a8f46700 1 --2- 172.21.15.162:0/2374657405 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18a40b4920 0x7f18a4066130 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:54.229 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.222+0000 7f18a8f46700 1 --2- 172.21.15.162:0/2374657405 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f18a40b10f0 0x7f18a4065c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-28T21:55:54.229 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.222+0000 7f18a8f46700 1 -- 172.21.15.162:0/2374657405 >> 172.21.15.162:0/2374657405 conn(0x7f18a40b54f0 msgr2=0x7f18a40b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:54.230 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.222+0000 7f18a8f46700 1 -- 172.21.15.162:0/2374657405 shutdown_connections 2020-11-28T21:55:54.230 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.222+0000 7f18a8f46700 1 -- 172.21.15.162:0/2374657405 wait complete. 2020-11-28T21:55:54.230 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 69 2020-11-28T21:55:54.239 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2020-11-28T21:55:54.400 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.394+0000 7f71d0fc7700 1 Processor -- start 2020-11-28T21:55:54.401 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.394+0000 7f71d0fc7700 1 -- start start 2020-11-28T21:55:54.402 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.394+0000 7f71d0fc7700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f71cc147e60 0x7f71cc0b2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:54.402 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.394+0000 7f71d0fc7700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71cc0ad6c0 con 0x7f71cc05bcb0 2020-11-28T21:55:54.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.394+0000 7f71d0fc7700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71cc0b0f30 con 0x7f71cc05c6a0 2020-11-28T21:55:54.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.398+0000 7f71d0fc7700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71cc0ae520 con 0x7f71cc147e60 2020-11-28T21:55:54.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.398+0000 7f71caffd700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f71cc05c6a0 0x7f71cc147750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:52264/0 (socket says 172.21.15.162:52264) 2020-11-28T21:55:54.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.398+0000 7f71caffd700 1 -- 172.21.15.162:0/1395856510 learned_addr learned my addr 172.21.15.162:0/1395856510 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:54.404 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.398+0000 7f71cbfff700 1 --2- 172.21.15.162:0/1395856510 >> v2:172.21.15.118:3300/0 conn(0x7f71cc147e60 0x7f71cc0b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:54.404 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.398+0000 7f71ca7fc700 1 -- 172.21.15.162:0/1395856510 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2844627157 0 0) 0x7f71cc0b0f30 con 0x7f71cc05c6a0 2020-11-28T21:55:54.404 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.398+0000 7f71ca7fc700 1 -- 172.21.15.162:0/1395856510 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71b0003230 con 0x7f71cc05c6a0 2020-11-28T21:55:54.404 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.398+0000 7f71cbfff700 1 -- 172.21.15.162:0/1395856510 >> v1:172.21.15.162:6789/0 conn(0x7f71cc05c6a0 legacy=0x7f71cc147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:54.405 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.398+0000 7f71cbfff700 1 -- 172.21.15.162:0/1395856510 >> v1:172.21.15.118:6789/0 conn(0x7f71cc05bcb0 legacy=0x7f71cc0b0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:54.405 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.398+0000 7f71cbfff700 1 -- 172.21.15.162:0/1395856510 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71cc05c090 con 0x7f71cc147e60 2020-11-28T21:55:54.408 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.398+0000 7f71cbfff700 1 --2- 172.21.15.162:0/1395856510 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f71cc147e60 0x7f71cc0b2300 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7f71bc0032b0 tx=0x7f71bc0091a0).ready entity=mon.0 client_cookie=3c79919630a4da56 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:54.408 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.398+0000 7f71ca7fc700 1 -- 172.21.15.162:0/1395856510 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71bc00b9a0 con 0x7f71cc147e60 2020-11-28T21:55:54.408 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.398+0000 7f71ca7fc700 1 -- 172.21.15.162:0/1395856510 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71bc00bb00 con 0x7f71cc147e60 2020-11-28T21:55:54.408 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.398+0000 7f71ca7fc700 1 -- 172.21.15.162:0/1395856510 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71bc00bc70 con 0x7f71cc147e60 2020-11-28T21:55:54.408 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.398+0000 7f71d0fc7700 1 -- 172.21.15.162:0/1395856510 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f71cc147e60 msgr2=0x7f71cc0b2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:54.409 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.398+0000 7f71d0fc7700 1 --2- 172.21.15.162:0/1395856510 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f71cc147e60 0x7f71cc0b2300 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7f71bc0032b0 tx=0x7f71bc0091a0).stop 2020-11-28T21:55:54.409 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.398+0000 7f71d0fc7700 1 -- 172.21.15.162:0/1395856510 shutdown_connections 2020-11-28T21:55:54.409 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.398+0000 7f71d0fc7700 1 --2- 172.21.15.162:0/1395856510 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f71cc147e60 0x7f71cc0b2300 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:54.409 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.398+0000 7f71d0fc7700 1 -- 172.21.15.162:0/1395856510 >> 172.21.15.162:0/1395856510 conn(0x7f71cc0b17a0 msgr2=0x7f71cc0b1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:54.410 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.398+0000 7f71d0fc7700 1 -- 172.21.15.162:0/1395856510 shutdown_connections 2020-11-28T21:55:54.410 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.398+0000 7f71d0fc7700 1 -- 172.21.15.162:0/1395856510 wait complete. 2020-11-28T21:55:54.410 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.398+0000 7f71d0fc7700 1 Processor -- start 2020-11-28T21:55:54.411 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.398+0000 7f71d0fc7700 1 -- start start 2020-11-28T21:55:54.411 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.402+0000 7f71d0fc7700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71cc05bcb0 0x7f71cc066c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:54.411 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.402+0000 7f71d0fc7700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f71cc05c6a0 0x7f71cc156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:54.411 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.402+0000 7f71d0fc7700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f71cc147e60 0x7f71cc0ab0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:54.412 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.402+0000 7f71d0fc7700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc156e50 con 0x7f71cc05c6a0 2020-11-28T21:55:54.412 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.402+0000 7f71d0fc7700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc065a30 con 0x7f71cc05bcb0 2020-11-28T21:55:54.412 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.402+0000 7f71d0fc7700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc065c90 con 0x7f71cc147e60 2020-11-28T21:55:54.412 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.402+0000 7f71cb7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71cc05bcb0 0x7f71cc066c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:54.413 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.402+0000 7f71cb7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71cc05bcb0 0x7f71cc066c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45424/0 (socket says 172.21.15.162:45424) 2020-11-28T21:55:54.413 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.402+0000 7f71cb7fe700 1 -- 172.21.15.162:0/999383015 learned_addr learned my addr 172.21.15.162:0/999383015 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:54.413 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.402+0000 7f71cbfff700 1 --2- 172.21.15.162:0/999383015 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f71cc147e60 0x7f71cc0ab0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:54.413 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.402+0000 7f71caffd700 1 --2- 172.21.15.162:0/999383015 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f71cc05c6a0 0x7f71cc156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:54.413 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.402+0000 7f71cb7fe700 1 -- 172.21.15.162:0/999383015 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f71cc147e60 msgr2=0x7f71cc0ab0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:54.414 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.402+0000 7f71cb7fe700 1 --2- 172.21.15.162:0/999383015 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f71cc147e60 0x7f71cc0ab0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:54.414 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.402+0000 7f71cb7fe700 1 -- 172.21.15.162:0/999383015 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f71cc05c6a0 msgr2=0x7f71cc156960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:54.414 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.402+0000 7f71cb7fe700 1 --2- 172.21.15.162:0/999383015 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f71cc05c6a0 0x7f71cc156960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:54.414 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.402+0000 7f71cb7fe700 1 -- 172.21.15.162:0/999383015 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71cc065f30 con 0x7f71cc05bcb0 2020-11-28T21:55:54.414 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.402+0000 7f71cb7fe700 1 --2- 172.21.15.162:0/999383015 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71cc05bcb0 0x7f71cc066c90 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 rx=0x7f71cc156f90 tx=0x7f71c4007c20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:54.415 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.402+0000 7f71c8ff9700 1 -- 172.21.15.162:0/999383015 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71c4013410 con 0x7f71cc05bcb0 2020-11-28T21:55:54.415 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.402+0000 7f71d0fc7700 1 -- 172.21.15.162:0/999383015 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71cc0661d0 con 0x7f71cc05bcb0 2020-11-28T21:55:54.415 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.402+0000 7f71c8ff9700 1 -- 172.21.15.162:0/999383015 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c4013570 con 0x7f71cc05bcb0 2020-11-28T21:55:54.415 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.402+0000 7f71c8ff9700 1 -- 172.21.15.162:0/999383015 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71c40138d0 con 0x7f71cc05bcb0 2020-11-28T21:55:54.415 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.402+0000 7f71d0fc7700 1 -- 172.21.15.162:0/999383015 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71cc0a86d0 con 0x7f71cc05bcb0 2020-11-28T21:55:54.417 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.402+0000 7f71d0fc7700 1 -- 172.21.15.162:0/999383015 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71cc057050 con 0x7f71cc05bcb0 2020-11-28T21:55:54.417 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.406+0000 7f71c8ff9700 1 -- 172.21.15.162:0/999383015 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f71c4010030 con 0x7f71cc05bcb0 2020-11-28T21:55:54.417 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.406+0000 7f71c8ff9700 1 --2- 172.21.15.162:0/999383015 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f71a4055440 0x7f71a4057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:54.418 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.406+0000 7f71c8ff9700 1 -- 172.21.15.162:0/999383015 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f71c406ddc0 con 0x7f71cc05bcb0 2020-11-28T21:55:54.419 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.406+0000 7f71caffd700 1 --2- 172.21.15.162:0/999383015 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f71a4055440 0x7f71a4057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:54.419 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.406+0000 7f71caffd700 1 --2- 172.21.15.162:0/999383015 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f71a4055440 0x7f71a4057900 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7f71b400d000 tx=0x7f71b4009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:54.419 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.410+0000 7f71c8ff9700 1 -- 172.21.15.162:0/999383015 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f71c4043f70 con 0x7f71cc05bcb0 2020-11-28T21:55:54.598 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.594+0000 7f71d0fc7700 1 -- 172.21.15.162:0/999383015 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f71cc0a89f0 con 0x7f71cc05bcb0 2020-11-28T21:55:54.902 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.898+0000 7f71c8ff9700 1 -- 172.21.15.162:0/999383015 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v70) v1 ==== 104+0+0 (secure 0 0 0) 0x7f71c4047590 con 0x7f71cc05bcb0 2020-11-28T21:55:54.906 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.902+0000 7f71d0fc7700 1 -- 172.21.15.162:0/999383015 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f71a4055440 msgr2=0x7f71a4057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:54.906 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.902+0000 7f71d0fc7700 1 --2- 172.21.15.162:0/999383015 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f71a4055440 0x7f71a4057900 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7f71b400d000 tx=0x7f71b4009040).stop 2020-11-28T21:55:54.906 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.902+0000 7f71d0fc7700 1 -- 172.21.15.162:0/999383015 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71cc05bcb0 msgr2=0x7f71cc066c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:54.907 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.902+0000 7f71d0fc7700 1 --2- 172.21.15.162:0/999383015 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71cc05bcb0 0x7f71cc066c90 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 rx=0x7f71cc156f90 tx=0x7f71c4007c20).stop 2020-11-28T21:55:54.907 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.902+0000 7f71d0fc7700 1 -- 172.21.15.162:0/999383015 shutdown_connections 2020-11-28T21:55:54.907 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.902+0000 7f71d0fc7700 1 --2- 172.21.15.162:0/999383015 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f71a4055440 0x7f71a4057900 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:54.907 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.902+0000 7f71d0fc7700 1 --2- 172.21.15.162:0/999383015 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f71cc147e60 0x7f71cc0ab0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:54.908 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.902+0000 7f71d0fc7700 1 --2- 172.21.15.162:0/999383015 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71cc05bcb0 0x7f71cc066c90 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:54.908 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.902+0000 7f71d0fc7700 1 --2- 172.21.15.162:0/999383015 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f71cc05c6a0 0x7f71cc156960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:54.908 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.902+0000 7f71d0fc7700 1 -- 172.21.15.162:0/999383015 >> 172.21.15.162:0/999383015 conn(0x7f71cc0b17a0 msgr2=0x7f71cc0b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:54.908 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.902+0000 7f71d0fc7700 1 -- 172.21.15.162:0/999383015 shutdown_connections 2020-11-28T21:55:54.908 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:54.902+0000 7f71d0fc7700 1 -- 172.21.15.162:0/999383015 wait complete. 2020-11-28T21:55:54.924 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-11-28T21:55:55.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.046+0000 7f7700f53700 1 Processor -- start 2020-11-28T21:55:55.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.046+0000 7f7700f53700 1 -- start start 2020-11-28T21:55:55.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.046+0000 7f7700f53700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f76fc0b02a0 0x7f76fc149d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:55.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.046+0000 7f7700f53700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76fc060500 con 0x7f76fc0afec0 2020-11-28T21:55:55.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.046+0000 7f7700f53700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76fc0607a0 con 0x7f76fc0a8df0 2020-11-28T21:55:55.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.046+0000 7f7700f53700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76fc060ce0 con 0x7f76fc0b02a0 2020-11-28T21:55:55.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.046+0000 7f76faffd700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f76fc0a8df0 0x7f76fc099d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:52280/0 (socket says 172.21.15.162:52280) 2020-11-28T21:55:55.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.046+0000 7f76faffd700 1 -- 172.21.15.162:0/271605144 learned_addr learned my addr 172.21.15.162:0/271605144 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:55.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.050+0000 7f76fbfff700 1 --2- 172.21.15.162:0/271605144 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f76fc0b02a0 0x7f76fc149d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:55.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.050+0000 7f76fa7fc700 1 -- 172.21.15.162:0/271605144 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3652204246 0 0) 0x7f76fc0607a0 con 0x7f76fc0a8df0 2020-11-28T21:55:55.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.050+0000 7f76fa7fc700 1 -- 172.21.15.162:0/271605144 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76dc003230 con 0x7f76fc0a8df0 2020-11-28T21:55:55.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.050+0000 7f76fa7fc700 1 -- 172.21.15.162:0/271605144 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3759020166 0 0) 0x7f76dc003230 con 0x7f76fc0a8df0 2020-11-28T21:55:55.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.050+0000 7f76fa7fc700 1 -- 172.21.15.162:0/271605144 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f76fc0b02a0 msgr2=0x7f76fc149d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:55.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.050+0000 7f76fa7fc700 1 --2- 172.21.15.162:0/271605144 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f76fc0b02a0 0x7f76fc149d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:55.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.050+0000 7f76fa7fc700 1 -- 172.21.15.162:0/271605144 >> v1:172.21.15.118:6789/0 conn(0x7f76fc0afec0 legacy=0x7f76fc099620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:55.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.050+0000 7f76fa7fc700 1 -- 172.21.15.162:0/271605144 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76fc060a40 con 0x7f76fc0a8df0 2020-11-28T21:55:55.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.050+0000 7f76fa7fc700 1 -- 172.21.15.162:0/271605144 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f76e40016d0 con 0x7f76fc0a8df0 2020-11-28T21:55:55.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.050+0000 7f76fa7fc700 1 -- 172.21.15.162:0/271605144 >> v1:172.21.15.162:6789/0 conn(0x7f76fc0a8df0 legacy=0x7f76fc099d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:55.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.050+0000 7f76fa7fc700 1 --2- 172.21.15.162:0/271605144 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f76dc003720 0x7f76dc003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:55.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.050+0000 7f76fa7fc700 1 --2- 172.21.15.162:0/271605144 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f76fc0b02a0 0x7f76dc0044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:55.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.050+0000 7f76fa7fc700 1 --2- 172.21.15.162:0/271605144 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76dc0049c0 0x7f76dc004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:55.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.050+0000 7f76fa7fc700 1 -- 172.21.15.162:0/271605144 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76fc060ce0 con 0x7f76dc003720 2020-11-28T21:55:55.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.050+0000 7f76fa7fc700 1 -- 172.21.15.162:0/271605144 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc0041c0 con 0x7f76dc0049c0 2020-11-28T21:55:55.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.050+0000 7f76fa7fc700 1 -- 172.21.15.162:0/271605144 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc0056f0 con 0x7f76fc0b02a0 2020-11-28T21:55:55.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.050+0000 7f76fb7fe700 1 --2- 172.21.15.162:0/271605144 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f76dc003720 0x7f76dc003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:55.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.050+0000 7f76faffd700 1 --2- 172.21.15.162:0/271605144 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f76fc0b02a0 0x7f76dc0044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:55.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.050+0000 7f76fbfff700 1 --2- 172.21.15.162:0/271605144 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76dc0049c0 0x7f76dc004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:55.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.050+0000 7f76fb7fe700 1 -- 172.21.15.162:0/271605144 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f76fc0b02a0 msgr2=0x7f76dc0044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:55.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.050+0000 7f76fb7fe700 1 --2- 172.21.15.162:0/271605144 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f76fc0b02a0 0x7f76dc0044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:55.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.050+0000 7f76fb7fe700 1 -- 172.21.15.162:0/271605144 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76dc0049c0 msgr2=0x7f76dc004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:55.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.050+0000 7f76fb7fe700 1 --2- 172.21.15.162:0/271605144 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76dc0049c0 0x7f76dc004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:55.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.050+0000 7f76fb7fe700 1 -- 172.21.15.162:0/271605144 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76dc003230 con 0x7f76dc003720 2020-11-28T21:55:55.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.050+0000 7f76fbfff700 1 --2- 172.21.15.162:0/271605144 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76dc0049c0 0x7f76dc004e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:55:55.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.050+0000 7f76fb7fe700 1 --2- 172.21.15.162:0/271605144 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f76dc003720 0x7f76dc003b40 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f76fc0629b0 tx=0x7f76f4007b60).ready entity=mon.0 client_cookie=78aa3bf6fde67e6a server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:55.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.050+0000 7f76fa7fc700 1 -- 172.21.15.162:0/271605144 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76f400c660 con 0x7f76dc003720 2020-11-28T21:55:55.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.050+0000 7f76fa7fc700 1 -- 172.21.15.162:0/271605144 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f400c7c0 con 0x7f76dc003720 2020-11-28T21:55:55.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.050+0000 7f76fa7fc700 1 -- 172.21.15.162:0/271605144 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76f400cb20 con 0x7f76dc003720 2020-11-28T21:55:55.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.050+0000 7f7700f53700 1 -- 172.21.15.162:0/271605144 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f76dc003720 msgr2=0x7f76dc003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:55.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.050+0000 7f7700f53700 1 --2- 172.21.15.162:0/271605144 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f76dc003720 0x7f76dc003b40 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f76fc0629b0 tx=0x7f76f4007b60).stop 2020-11-28T21:55:55.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.050+0000 7f76fbfff700 1 -- 172.21.15.162:0/271605144 reap_dead start 2020-11-28T21:55:55.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.054+0000 7f7700f53700 1 -- 172.21.15.162:0/271605144 shutdown_connections 2020-11-28T21:55:55.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.054+0000 7f7700f53700 1 -- 172.21.15.162:0/271605144 >> 172.21.15.162:0/271605144 conn(0x7f76fc0b5520 msgr2=0x7f76fc0b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:55.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.054+0000 7f7700f53700 1 -- 172.21.15.162:0/271605144 shutdown_connections 2020-11-28T21:55:55.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.054+0000 7f7700f53700 1 -- 172.21.15.162:0/271605144 wait complete. 2020-11-28T21:55:55.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.054+0000 7f7700f53700 1 Processor -- start 2020-11-28T21:55:55.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.054+0000 7f7700f53700 1 -- start start 2020-11-28T21:55:55.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.054+0000 7f7700f53700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f76fc091430 0x7f76fc091810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:55.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.054+0000 7f7700f53700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76fc14d440 0x7f76fc14d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:55.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.054+0000 7f7700f53700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f76fc097070 0x7f76fc097520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:55.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.054+0000 7f7700f53700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76fc0adc60 con 0x7f76fc091430 2020-11-28T21:55:55.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.054+0000 7f7700f53700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76fc0ad6d0 con 0x7f76fc14d440 2020-11-28T21:55:55.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.054+0000 7f7700f53700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76fc0ae1b0 con 0x7f76fc097070 2020-11-28T21:55:55.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.054+0000 7f76faffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76fc14d440 0x7f76fc14d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:55.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.054+0000 7f76faffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76fc14d440 0x7f76fc14d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45448/0 (socket says 172.21.15.162:45448) 2020-11-28T21:55:55.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.054+0000 7f76faffd700 1 -- 172.21.15.162:0/3406645816 learned_addr learned my addr 172.21.15.162:0/3406645816 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:55.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.054+0000 7f76fbfff700 1 --2- 172.21.15.162:0/3406645816 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f76fc097070 0x7f76fc097520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:55.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.054+0000 7f76fb7fe700 1 --2- 172.21.15.162:0/3406645816 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f76fc091430 0x7f76fc091810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:55.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.058+0000 7f76faffd700 1 -- 172.21.15.162:0/3406645816 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f76fc097070 msgr2=0x7f76fc097520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:55.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.058+0000 7f76faffd700 1 --2- 172.21.15.162:0/3406645816 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f76fc097070 0x7f76fc097520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:55.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.058+0000 7f76faffd700 1 -- 172.21.15.162:0/3406645816 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f76fc091430 msgr2=0x7f76fc091810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:55.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.058+0000 7f76faffd700 1 --2- 172.21.15.162:0/3406645816 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f76fc091430 0x7f76fc091810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:55.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.058+0000 7f76faffd700 1 -- 172.21.15.162:0/3406645816 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76fc08fa20 con 0x7f76fc14d440 2020-11-28T21:55:55.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.058+0000 7f76fb7fe700 1 --2- 172.21.15.162:0/3406645816 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f76fc091430 0x7f76fc091810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:55:55.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.058+0000 7f76faffd700 1 --2- 172.21.15.162:0/3406645816 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76fc14d440 0x7f76fc14d8c0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 rx=0x7f76e40010d0 tx=0x7f76fc099200).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:55.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.058+0000 7f76f8ff9700 1 -- 172.21.15.162:0/3406645816 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76e400fb00 con 0x7f76fc14d440 2020-11-28T21:55:55.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.058+0000 7f76f8ff9700 1 -- 172.21.15.162:0/3406645816 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e400fc60 con 0x7f76fc14d440 2020-11-28T21:55:55.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.058+0000 7f7700f53700 1 -- 172.21.15.162:0/3406645816 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76fc14ddb0 con 0x7f76fc14d440 2020-11-28T21:55:55.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.058+0000 7f76f8ff9700 1 -- 172.21.15.162:0/3406645816 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76e4012350 con 0x7f76fc14d440 2020-11-28T21:55:55.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.058+0000 7f7700f53700 1 -- 172.21.15.162:0/3406645816 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76fc0ae610 con 0x7f76fc14d440 2020-11-28T21:55:55.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.058+0000 7f7700f53700 1 -- 172.21.15.162:0/3406645816 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76fc0ae320 con 0x7f76fc14d440 2020-11-28T21:55:55.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.058+0000 7f76f8ff9700 1 -- 172.21.15.162:0/3406645816 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f76e401b030 con 0x7f76fc14d440 2020-11-28T21:55:55.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.058+0000 7f76f8ff9700 1 --2- 172.21.15.162:0/3406645816 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f76e0055440 0x7f76e0057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:55.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.058+0000 7f76fb7fe700 1 --2- 172.21.15.162:0/3406645816 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f76e0055440 0x7f76e0057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:55.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.058+0000 7f76f8ff9700 1 -- 172.21.15.162:0/3406645816 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f76e406d950 con 0x7f76fc14d440 2020-11-28T21:55:55.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.062+0000 7f76fb7fe700 1 --2- 172.21.15.162:0/3406645816 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f76e0055440 0x7f76e0057900 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f76f4017000 tx=0x7f76f4016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:55.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.066+0000 7f76f8ff9700 1 -- 172.21.15.162:0/3406645816 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f76e4043c00 con 0x7f76fc14d440 2020-11-28T21:55:55.216 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.210+0000 7f7700f53700 1 -- 172.21.15.162:0/3406645816 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f76fc057050 con 0x7f76fc14d440 2020-11-28T21:55:55.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.906+0000 7f76f8ff9700 1 -- 172.21.15.162:0/3406645816 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v140) v1 ==== 185+0+0 (secure 0 0 0) 0x7f76e4019030 con 0x7f76fc14d440 2020-11-28T21:55:55.917 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.910+0000 7f7700f53700 1 -- 172.21.15.162:0/3406645816 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f76e0055440 msgr2=0x7f76e0057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:55.917 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.910+0000 7f7700f53700 1 --2- 172.21.15.162:0/3406645816 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f76e0055440 0x7f76e0057900 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f76f4017000 tx=0x7f76f4016000).stop 2020-11-28T21:55:55.917 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.910+0000 7f7700f53700 1 -- 172.21.15.162:0/3406645816 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76fc14d440 msgr2=0x7f76fc14d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:55.917 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.910+0000 7f7700f53700 1 --2- 172.21.15.162:0/3406645816 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76fc14d440 0x7f76fc14d8c0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 rx=0x7f76e40010d0 tx=0x7f76fc099200).stop 2020-11-28T21:55:55.917 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.910+0000 7f7700f53700 1 -- 172.21.15.162:0/3406645816 shutdown_connections 2020-11-28T21:55:55.918 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.910+0000 7f7700f53700 1 --2- 172.21.15.162:0/3406645816 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f76e0055440 0x7f76e0057900 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:55.918 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.910+0000 7f7700f53700 1 --2- 172.21.15.162:0/3406645816 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f76fc097070 0x7f76fc097520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:55.918 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.910+0000 7f7700f53700 1 --2- 172.21.15.162:0/3406645816 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f76fc091430 0x7f76fc091810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:55.918 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.910+0000 7f7700f53700 1 --2- 172.21.15.162:0/3406645816 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76fc14d440 0x7f76fc14d8c0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:55.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.910+0000 7f7700f53700 1 -- 172.21.15.162:0/3406645816 >> 172.21.15.162:0/3406645816 conn(0x7f76fc0b5520 msgr2=0x7f76fc0b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:55.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.910+0000 7f7700f53700 1 -- 172.21.15.162:0/3406645816 shutdown_connections 2020-11-28T21:55:55.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:55.910+0000 7f7700f53700 1 -- 172.21.15.162:0/3406645816 wait complete. 2020-11-28T21:55:55.919 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_metadata' removed 2020-11-28T21:55:55.930 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-11-28T21:55:56.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.046+0000 7fb9508b9700 1 Processor -- start 2020-11-28T21:55:56.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.046+0000 7fb9508b9700 1 -- start start 2020-11-28T21:55:56.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.050+0000 7fb9508b9700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fb94c0b2d00 0x7fb94c05c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:56.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.050+0000 7fb9508b9700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb94c0b3ab0 0x7fb94c0b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:56.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.050+0000 7fb9508b9700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb94c142ad0 0x7fb94c142f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:56.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.050+0000 7fb9508b9700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb94c0b0e20 con 0x7fb94c0b2d00 2020-11-28T21:55:56.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.050+0000 7fb9508b9700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb94c05baf0 con 0x7fb94c0b3ab0 2020-11-28T21:55:56.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.050+0000 7fb9508b9700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c05bd90 con 0x7fb94c142ad0 2020-11-28T21:55:56.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.050+0000 7fb94a59c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb94c0b3ab0 0x7fb94c0b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:56.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.050+0000 7fb94a59c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb94c0b3ab0 0x7fb94c0b30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45454/0 (socket says 172.21.15.162:45454) 2020-11-28T21:55:56.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.050+0000 7fb94a59c700 1 -- 172.21.15.162:0/4130595528 learned_addr learned my addr 172.21.15.162:0/4130595528 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:56.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.050+0000 7fb94b59e700 1 --2- 172.21.15.162:0/4130595528 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb94c142ad0 0x7fb94c142f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:56.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.050+0000 7fb94ad9d700 1 --2- 172.21.15.162:0/4130595528 >> v2:172.21.15.118:3300/0 conn(0x7fb94c0b2d00 0x7fb94c05c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:56.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.050+0000 7fb94a59c700 1 -- 172.21.15.162:0/4130595528 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb94c142ad0 msgr2=0x7fb94c142f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:56.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.050+0000 7fb94a59c700 1 --2- 172.21.15.162:0/4130595528 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb94c142ad0 0x7fb94c142f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:56.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.050+0000 7fb94a59c700 1 -- 172.21.15.162:0/4130595528 >> v2:172.21.15.118:3300/0 conn(0x7fb94c0b2d00 msgr2=0x7fb94c05c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:56.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.050+0000 7fb94a59c700 1 --2- 172.21.15.162:0/4130595528 >> v2:172.21.15.118:3300/0 conn(0x7fb94c0b2d00 0x7fb94c05c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:56.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.050+0000 7fb94a59c700 1 -- 172.21.15.162:0/4130595528 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb94c05c030 con 0x7fb94c0b3ab0 2020-11-28T21:55:56.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.050+0000 7fb94a59c700 1 --2- 172.21.15.162:0/4130595528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb94c0b3ab0 0x7fb94c0b30e0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 rx=0x7fb9340032b0 tx=0x7fb934009270).ready entity=mon.1 client_cookie=577d4457642fe822 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:56.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.050+0000 7fb949d9b700 1 -- 172.21.15.162:0/4130595528 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb934009e60 con 0x7fb94c0b3ab0 2020-11-28T21:55:56.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.050+0000 7fb949d9b700 1 -- 172.21.15.162:0/4130595528 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb93400bb50 con 0x7fb94c0b3ab0 2020-11-28T21:55:56.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.050+0000 7fb949d9b700 1 -- 172.21.15.162:0/4130595528 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb93400bcc0 con 0x7fb94c0b3ab0 2020-11-28T21:55:56.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.050+0000 7fb9508b9700 1 -- 172.21.15.162:0/4130595528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb94c0b3ab0 msgr2=0x7fb94c0b30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:56.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.050+0000 7fb9508b9700 1 --2- 172.21.15.162:0/4130595528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb94c0b3ab0 0x7fb94c0b30e0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 rx=0x7fb9340032b0 tx=0x7fb934009270).stop 2020-11-28T21:55:56.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.050+0000 7fb9508b9700 1 -- 172.21.15.162:0/4130595528 shutdown_connections 2020-11-28T21:55:56.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.050+0000 7fb9508b9700 1 --2- 172.21.15.162:0/4130595528 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb94c142ad0 0x7fb94c142f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:56.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.050+0000 7fb9508b9700 1 --2- 172.21.15.162:0/4130595528 >> v2:172.21.15.118:3300/0 conn(0x7fb94c0b2d00 0x7fb94c05c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:56.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.050+0000 7fb9508b9700 1 --2- 172.21.15.162:0/4130595528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb94c0b3ab0 0x7fb94c0b30e0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:56.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.050+0000 7fb9508b9700 1 -- 172.21.15.162:0/4130595528 >> 172.21.15.162:0/4130595528 conn(0x7fb94c0604c0 msgr2=0x7fb94c0608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:56.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.050+0000 7fb9508b9700 1 -- 172.21.15.162:0/4130595528 shutdown_connections 2020-11-28T21:55:56.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.050+0000 7fb9508b9700 1 -- 172.21.15.162:0/4130595528 wait complete. 2020-11-28T21:55:56.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.054+0000 7fb9508b9700 1 Processor -- start 2020-11-28T21:55:56.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.054+0000 7fb9508b9700 1 -- start start 2020-11-28T21:55:56.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.054+0000 7fb9508b9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb94c0b2d00 0x7fb94c13eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:56.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.054+0000 7fb9508b9700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb94c0b3ab0 0x7fb94c154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:56.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.054+0000 7fb9508b9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb94c142ad0 0x7fb94c154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:56.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.054+0000 7fb9508b9700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c148290 con 0x7fb94c0b2d00 2020-11-28T21:55:56.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.054+0000 7fb9508b9700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c148ab0 con 0x7fb94c142ad0 2020-11-28T21:55:56.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.054+0000 7fb9508b9700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c154c60 con 0x7fb94c0b3ab0 2020-11-28T21:55:56.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.054+0000 7fb94b59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb94c142ad0 0x7fb94c154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:56.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.054+0000 7fb94a59c700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb94c0b3ab0 0x7fb94c154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:56.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.054+0000 7fb94a59c700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb94c0b3ab0 0x7fb94c154280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:42686/0 (socket says 172.21.15.162:42686) 2020-11-28T21:55:56.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.054+0000 7fb94a59c700 1 -- 172.21.15.162:0/1127401913 learned_addr learned my addr 172.21.15.162:0/1127401913 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:56.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.054+0000 7fb94b59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb94c142ad0 0x7fb94c154770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45464/0 (socket says 172.21.15.162:45464) 2020-11-28T21:55:56.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.054+0000 7fb94ad9d700 1 --2- 172.21.15.162:0/1127401913 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb94c0b2d00 0x7fb94c13eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:56.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.054+0000 7fb94a59c700 1 -- 172.21.15.162:0/1127401913 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb94c142ad0 msgr2=0x7fb94c154770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:56.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.054+0000 7fb94a59c700 1 --2- 172.21.15.162:0/1127401913 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb94c142ad0 0x7fb94c154770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:56.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.054+0000 7fb94a59c700 1 -- 172.21.15.162:0/1127401913 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb94c0b2d00 msgr2=0x7fb94c13eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:56.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.054+0000 7fb94a59c700 1 --2- 172.21.15.162:0/1127401913 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb94c0b2d00 0x7fb94c13eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:56.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.054+0000 7fb94a59c700 1 -- 172.21.15.162:0/1127401913 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb94c147df0 con 0x7fb94c0b3ab0 2020-11-28T21:55:56.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.054+0000 7fb94b59e700 1 --2- 172.21.15.162:0/1127401913 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb94c142ad0 0x7fb94c154770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:55:56.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.054+0000 7fb94a59c700 1 --2- 172.21.15.162:0/1127401913 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb94c0b3ab0 0x7fb94c154280 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fb9340032b0 tx=0x7fb934016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:56.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.054+0000 7fb943fff700 1 -- 172.21.15.162:0/1127401913 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb934022070 con 0x7fb94c0b3ab0 2020-11-28T21:55:56.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.054+0000 7fb943fff700 1 -- 172.21.15.162:0/1127401913 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb93401e610 con 0x7fb94c0b3ab0 2020-11-28T21:55:56.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.054+0000 7fb943fff700 1 -- 172.21.15.162:0/1127401913 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb93401e970 con 0x7fb94c0b3ab0 2020-11-28T21:55:56.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.054+0000 7fb9508b9700 1 -- 172.21.15.162:0/1127401913 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb94c149180 con 0x7fb94c0b3ab0 2020-11-28T21:55:56.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.054+0000 7fb9508b9700 1 -- 172.21.15.162:0/1127401913 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb94c139cb0 con 0x7fb94c0b3ab0 2020-11-28T21:55:56.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.058+0000 7fb9508b9700 1 -- 172.21.15.162:0/1127401913 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb94c148c20 con 0x7fb94c0b3ab0 2020-11-28T21:55:56.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.058+0000 7fb943fff700 1 -- 172.21.15.162:0/1127401913 <== mon.2 v2:172.21.15.118:3301/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb93400f070 con 0x7fb94c0b3ab0 2020-11-28T21:55:56.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.058+0000 7fb943fff700 1 --2- 172.21.15.162:0/1127401913 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb930055470 0x7fb930057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:56.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.058+0000 7fb943fff700 1 -- 172.21.15.162:0/1127401913 <== mon.2 v2:172.21.15.118:3301/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5721+0+0 (secure 0 0 0) 0x7fb93406eb10 con 0x7fb94c0b3ab0 2020-11-28T21:55:56.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.058+0000 7fb94ad9d700 1 --2- 172.21.15.162:0/1127401913 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb930055470 0x7fb930057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:56.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.058+0000 7fb94ad9d700 1 --2- 172.21.15.162:0/1127401913 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb930055470 0x7fb930057930 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7fb93c00b000 tx=0x7fb93c006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:56.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.066+0000 7fb943fff700 1 -- 172.21.15.162:0/1127401913 <== mon.2 v2:172.21.15.118:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb94c148c20 con 0x7fb94c0b3ab0 2020-11-28T21:55:56.224 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.218+0000 7fb9508b9700 1 -- 172.21.15.162:0/1127401913 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb94c148c20 con 0x7fb94c0b3ab0 2020-11-28T21:55:56.916 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.910+0000 7fb943fff700 1 -- 172.21.15.162:0/1127401913 <== mon.2 v2:172.21.15.118:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v141) v1 ==== 173+0+0 (secure 0 0 0) 0x7fb94c148c20 con 0x7fb94c0b3ab0 2020-11-28T21:55:56.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.914+0000 7fb9508b9700 1 -- 172.21.15.162:0/1127401913 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb930055470 msgr2=0x7fb930057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:56.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.914+0000 7fb9508b9700 1 --2- 172.21.15.162:0/1127401913 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb930055470 0x7fb930057930 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7fb93c00b000 tx=0x7fb93c006dc0).stop 2020-11-28T21:55:56.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.914+0000 7fb9508b9700 1 -- 172.21.15.162:0/1127401913 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb94c0b3ab0 msgr2=0x7fb94c154280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:56.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.914+0000 7fb9508b9700 1 --2- 172.21.15.162:0/1127401913 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb94c0b3ab0 0x7fb94c154280 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fb9340032b0 tx=0x7fb934016000).stop 2020-11-28T21:55:56.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.914+0000 7fb9508b9700 1 -- 172.21.15.162:0/1127401913 shutdown_connections 2020-11-28T21:55:56.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.914+0000 7fb9508b9700 1 --2- 172.21.15.162:0/1127401913 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb930055470 0x7fb930057930 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:56.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.914+0000 7fb9508b9700 1 --2- 172.21.15.162:0/1127401913 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb94c142ad0 0x7fb94c154770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:56.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.914+0000 7fb9508b9700 1 --2- 172.21.15.162:0/1127401913 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb94c0b2d00 0x7fb94c13eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:56.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.914+0000 7fb9508b9700 1 --2- 172.21.15.162:0/1127401913 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb94c0b3ab0 0x7fb94c154280 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:56.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.914+0000 7fb9508b9700 1 -- 172.21.15.162:0/1127401913 >> 172.21.15.162:0/1127401913 conn(0x7fb94c0604c0 msgr2=0x7fb94c13ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:56.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.914+0000 7fb9508b9700 1 -- 172.21.15.162:0/1127401913 shutdown_connections 2020-11-28T21:55:56.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:56.914+0000 7fb9508b9700 1 -- 172.21.15.162:0/1127401913 wait complete. 2020-11-28T21:55:56.924 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_data' removed 2020-11-28T21:55:56.937 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-28T21:55:56.937 INFO:tasks.ceph.mds.a:Stopping old one... 2020-11-28T21:55:56.938 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-11-28T21:55:56.938 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T21:55:56.938 INFO:tasks.ceph.mds.c:Restarting daemon 2020-11-28T21:55:56.939 INFO:tasks.ceph.mds.c:Stopping old one... 2020-11-28T21:55:56.939 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2020-11-28T21:55:56.939 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T21:55:56.940 INFO:tasks.ceph.mds.b:Restarting daemon 2020-11-28T21:55:56.940 INFO:tasks.ceph.mds.b:Stopping old one... 2020-11-28T21:55:56.940 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-11-28T21:55:56.940 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T21:55:56.949 INFO:tasks.ceph.mds.a:Stopped 2020-11-28T21:55:56.949 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-28T21:55:56.951 INFO:tasks.ceph.mds.c:Stopped 2020-11-28T21:55:56.951 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2020-11-28T21:55:56.952 INFO:tasks.ceph.mds.b:Stopped 2020-11-28T21:55:56.953 INFO:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-11-28T21:55:56.995 INFO:tasks.ceph.mds.a:Started 2020-11-28T21:55:56.996 DEBUG:teuthology.parallel:result is None 2020-11-28T21:55:56.997 INFO:tasks.ceph.mds.c:Started 2020-11-28T21:55:56.998 DEBUG:teuthology.parallel:result is None 2020-11-28T21:55:56.999 INFO:tasks.ceph.mds.b:Started 2020-11-28T21:55:57.000 DEBUG:teuthology.parallel:result is None 2020-11-28T21:55:57.001 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist clear 2020-11-28T21:55:57.067 INFO:tasks.ceph.mds.c.smithi162.stdout:starting mds.c at 2020-11-28T21:55:57.068 INFO:tasks.ceph.mds.a.smithi162.stdout:starting mds.a at 2020-11-28T21:55:57.087 INFO:tasks.ceph.mds.b.smithi118.stdout:starting mds.b at 2020-11-28T21:55:57.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.114+0000 7f087f9b1700 1 Processor -- start 2020-11-28T21:55:57.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.114+0000 7f087f9b1700 1 -- start start 2020-11-28T21:55:57.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.114+0000 7f087f9b1700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f08780af1f0 0x7f087813f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:57.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.114+0000 7f087f9b1700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f087806aef0 0x7f087813fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:57.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.114+0000 7f087f9b1700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08780af910 con 0x7f0878140350 2020-11-28T21:55:57.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.114+0000 7f087f9b1700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08780b66f0 con 0x7f08780af1f0 2020-11-28T21:55:57.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.114+0000 7f087f9b1700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08781438d0 con 0x7f087806aef0 2020-11-28T21:55:57.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.114+0000 7f087d74d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f08780af1f0 0x7f087813f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:57.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.114+0000 7f087d74d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f08780af1f0 0x7f087813f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45492/0 (socket says 172.21.15.162:45492) 2020-11-28T21:55:57.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.114+0000 7f087d74d700 1 -- 172.21.15.162:0/2875327719 learned_addr learned my addr 172.21.15.162:0/2875327719 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:57.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.114+0000 7f087cf4c700 1 --2- 172.21.15.162:0/2875327719 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f087806aef0 0x7f087813fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:57.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.114+0000 7f087d74d700 1 -- 172.21.15.162:0/2875327719 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f087806aef0 msgr2=0x7f087813fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:57.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.114+0000 7f087d74d700 1 --2- 172.21.15.162:0/2875327719 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f087806aef0 0x7f087813fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:57.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.114+0000 7f087d74d700 1 -- 172.21.15.162:0/2875327719 >> v1:172.21.15.118:6789/0 conn(0x7f0878140350 legacy=0x7f08780acb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:57.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.114+0000 7f087d74d700 1 -- 172.21.15.162:0/2875327719 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0878143b70 con 0x7f08780af1f0 2020-11-28T21:55:57.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.114+0000 7f087d74d700 1 --2- 172.21.15.162:0/2875327719 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08780af1f0 0x7f087813f970 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 rx=0x7f08740032b0 tx=0x7f0874009270).ready entity=mon.1 client_cookie=14e2fc6430c4b9a1 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:57.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.114+0000 7f0867fff700 1 -- 172.21.15.162:0/2875327719 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0874009e60 con 0x7f08780af1f0 2020-11-28T21:55:57.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.114+0000 7f0867fff700 1 -- 172.21.15.162:0/2875327719 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087400bc00 con 0x7f08780af1f0 2020-11-28T21:55:57.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.114+0000 7f0867fff700 1 -- 172.21.15.162:0/2875327719 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f087400bd70 con 0x7f08780af1f0 2020-11-28T21:55:57.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.114+0000 7f087f9b1700 1 -- 172.21.15.162:0/2875327719 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08780af1f0 msgr2=0x7f087813f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:57.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.114+0000 7f087f9b1700 1 --2- 172.21.15.162:0/2875327719 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08780af1f0 0x7f087813f970 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 rx=0x7f08740032b0 tx=0x7f0874009270).stop 2020-11-28T21:55:57.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.114+0000 7f087f9b1700 1 -- 172.21.15.162:0/2875327719 shutdown_connections 2020-11-28T21:55:57.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.114+0000 7f087f9b1700 1 --2- 172.21.15.162:0/2875327719 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08780af1f0 0x7f087813f970 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:57.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.114+0000 7f087f9b1700 1 --2- 172.21.15.162:0/2875327719 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f087806aef0 0x7f087813fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:57.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.114+0000 7f087f9b1700 1 -- 172.21.15.162:0/2875327719 >> 172.21.15.162:0/2875327719 conn(0x7f08780b56f0 msgr2=0x7f08780b5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:57.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.114+0000 7f087f9b1700 1 -- 172.21.15.162:0/2875327719 shutdown_connections 2020-11-28T21:55:57.129 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.114+0000 7f087f9b1700 1 -- 172.21.15.162:0/2875327719 wait complete. 2020-11-28T21:55:57.129 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.118+0000 7f087f9b1700 1 Processor -- start 2020-11-28T21:55:57.129 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.118+0000 7f087f9b1700 1 -- start start 2020-11-28T21:55:57.129 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.118+0000 7f087f9b1700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f087806aef0 0x7f087805c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:57.129 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.118+0000 7f087f9b1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08780af1f0 0x7f08780b4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:57.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.118+0000 7f087f9b1700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0878140350 0x7f08780b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:57.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.118+0000 7f087f9b1700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08780b4080 con 0x7f087806aef0 2020-11-28T21:55:57.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.118+0000 7f087f9b1700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08781436f0 con 0x7f08780af1f0 2020-11-28T21:55:57.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.118+0000 7f087f9b1700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08781420e0 con 0x7f0878140350 2020-11-28T21:55:57.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.118+0000 7f087cf4c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08780af1f0 0x7f08780b4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:57.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.118+0000 7f087cf4c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08780af1f0 0x7f08780b4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45498/0 (socket says 172.21.15.162:45498) 2020-11-28T21:55:57.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.118+0000 7f087cf4c700 1 -- 172.21.15.162:0/43152220 learned_addr learned my addr 172.21.15.162:0/43152220 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:57.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.118+0000 7f087d74d700 1 --2- 172.21.15.162:0/43152220 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f087806aef0 0x7f087805c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:57.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.118+0000 7f087df4e700 1 --2- 172.21.15.162:0/43152220 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0878140350 0x7f08780b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:57.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.118+0000 7f087cf4c700 1 -- 172.21.15.162:0/43152220 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0878140350 msgr2=0x7f08780b4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:57.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.118+0000 7f087cf4c700 1 --2- 172.21.15.162:0/43152220 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0878140350 0x7f08780b4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:57.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.118+0000 7f087cf4c700 1 -- 172.21.15.162:0/43152220 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f087806aef0 msgr2=0x7f087805c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:57.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.118+0000 7f087cf4c700 1 --2- 172.21.15.162:0/43152220 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f087806aef0 0x7f087805c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:57.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.118+0000 7f087cf4c700 1 -- 172.21.15.162:0/43152220 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08780b3de0 con 0x7f08780af1f0 2020-11-28T21:55:57.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.118+0000 7f087cf4c700 1 --2- 172.21.15.162:0/43152220 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08780af1f0 0x7f08780b4660 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 rx=0x7f086800c000 tx=0x7f0868006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:57.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.118+0000 7f08667fc700 1 -- 172.21.15.162:0/43152220 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f086800ca10 con 0x7f08780af1f0 2020-11-28T21:55:57.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.118+0000 7f08667fc700 1 -- 172.21.15.162:0/43152220 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f086800cb70 con 0x7f08780af1f0 2020-11-28T21:55:57.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.118+0000 7f087f9b1700 1 -- 172.21.15.162:0/43152220 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0878142340 con 0x7f08780af1f0 2020-11-28T21:55:57.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.118+0000 7f08667fc700 1 -- 172.21.15.162:0/43152220 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f086800ee80 con 0x7f08780af1f0 2020-11-28T21:55:57.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.118+0000 7f087f9b1700 1 -- 172.21.15.162:0/43152220 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f087813c580 con 0x7f08780af1f0 2020-11-28T21:55:57.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.122+0000 7f087f9b1700 1 -- 172.21.15.162:0/43152220 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08781424d0 con 0x7f08780af1f0 2020-11-28T21:55:57.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.122+0000 7f08667fc700 1 -- 172.21.15.162:0/43152220 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f086801c460 con 0x7f08780af1f0 2020-11-28T21:55:57.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.122+0000 7f08667fc700 1 --2- 172.21.15.162:0/43152220 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f08580553b0 0x7f0858057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:57.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.122+0000 7f087d74d700 1 --2- 172.21.15.162:0/43152220 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f08580553b0 0x7f0858057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:57.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.122+0000 7f08667fc700 1 -- 172.21.15.162:0/43152220 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f086806e120 con 0x7f08780af1f0 2020-11-28T21:55:57.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.122+0000 7f087d74d700 1 --2- 172.21.15.162:0/43152220 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f08580553b0 0x7f0858057870 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7f087401a000 tx=0x7f0874016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:57.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.130+0000 7f08667fc700 1 -- 172.21.15.162:0/43152220 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0868044590 con 0x7f08780af1f0 2020-11-28T21:55:57.292 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:57.286+0000 7f087f9b1700 1 -- 172.21.15.162:0/43152220 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f0878057050 con 0x7f08780af1f0 2020-11-28T21:55:58.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.106+0000 7f08667fc700 1 -- 172.21.15.162:0/43152220 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v144) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0868047bb0 con 0x7f08780af1f0 2020-11-28T21:55:58.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.110+0000 7f087f9b1700 1 -- 172.21.15.162:0/43152220 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f08580553b0 msgr2=0x7f0858057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:58.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.110+0000 7f087f9b1700 1 --2- 172.21.15.162:0/43152220 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f08580553b0 0x7f0858057870 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7f087401a000 tx=0x7f0874016000).stop 2020-11-28T21:55:58.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.110+0000 7f087f9b1700 1 -- 172.21.15.162:0/43152220 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08780af1f0 msgr2=0x7f08780b4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:58.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.110+0000 7f087f9b1700 1 --2- 172.21.15.162:0/43152220 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08780af1f0 0x7f08780b4660 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 rx=0x7f086800c000 tx=0x7f0868006dc0).stop 2020-11-28T21:55:58.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.110+0000 7f087f9b1700 1 -- 172.21.15.162:0/43152220 shutdown_connections 2020-11-28T21:55:58.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.110+0000 7f087f9b1700 1 --2- 172.21.15.162:0/43152220 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f08580553b0 0x7f0858057870 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:58.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.110+0000 7f087f9b1700 1 --2- 172.21.15.162:0/43152220 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0878140350 0x7f08780b4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:58.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.110+0000 7f087f9b1700 1 --2- 172.21.15.162:0/43152220 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f087806aef0 0x7f087805c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:58.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.110+0000 7f087f9b1700 1 --2- 172.21.15.162:0/43152220 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08780af1f0 0x7f08780b4660 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:58.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.110+0000 7f087f9b1700 1 -- 172.21.15.162:0/43152220 >> 172.21.15.162:0/43152220 conn(0x7f08780b56f0 msgr2=0x7f08780afd70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:58.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.110+0000 7f087f9b1700 1 -- 172.21.15.162:0/43152220 shutdown_connections 2020-11-28T21:55:58.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.110+0000 7f087f9b1700 1 -- 172.21.15.162:0/43152220 wait complete. 2020-11-28T21:55:58.119 INFO:teuthology.orchestra.run.smithi162.stderr: removed all blacklist entries 2020-11-28T21:55:58.129 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2020-11-28T21:55:58.130 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth ls --format=json-pretty 2020-11-28T21:55:58.242 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.238+0000 7f7d4c587700 1 Processor -- start 2020-11-28T21:55:58.243 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.238+0000 7f7d4c587700 1 -- start start 2020-11-28T21:55:58.243 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.238+0000 7f7d4c587700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7d4413fd00 0x7f7d4413c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:58.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.238+0000 7f7d4c587700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d44067e90 con 0x7f7d440b69a0 2020-11-28T21:55:58.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.238+0000 7f7d4c587700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d440b6d80 con 0x7f7d4413c900 2020-11-28T21:55:58.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.238+0000 7f7d4c587700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d440ae270 con 0x7f7d4413fd00 2020-11-28T21:55:58.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.238+0000 7f7d49b22700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7d4413fd00 0x7f7d4413c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:58.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.238+0000 7f7d4ab24700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f7d4413c900 0x7f7d44149f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:52356/0 (socket says 172.21.15.162:52356) 2020-11-28T21:55:58.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.238+0000 7f7d4ab24700 1 -- 172.21.15.162:0/2564244779 learned_addr learned my addr 172.21.15.162:0/2564244779 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:58.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.238+0000 7f7d49321700 1 -- 172.21.15.162:0/2564244779 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1087502856 0 0) 0x7f7d440b6d80 con 0x7f7d4413c900 2020-11-28T21:55:58.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.238+0000 7f7d49321700 1 -- 172.21.15.162:0/2564244779 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d24003230 con 0x7f7d4413c900 2020-11-28T21:55:58.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.238+0000 7f7d49321700 1 -- 172.21.15.162:0/2564244779 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 178160886 0 0) 0x7f7d44067e90 con 0x7f7d440b69a0 2020-11-28T21:55:58.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.238+0000 7f7d49321700 1 -- 172.21.15.162:0/2564244779 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d440b6d80 con 0x7f7d440b69a0 2020-11-28T21:55:58.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.238+0000 7f7d49321700 1 -- 172.21.15.162:0/2564244779 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2042981865 0 0) 0x7f7d24003230 con 0x7f7d4413c900 2020-11-28T21:55:58.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.238+0000 7f7d49b22700 1 -- 172.21.15.162:0/2564244779 >> v1:172.21.15.162:6789/0 conn(0x7f7d4413c900 legacy=0x7f7d44149f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:58.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.238+0000 7f7d49b22700 1 -- 172.21.15.162:0/2564244779 >> v1:172.21.15.118:6789/0 conn(0x7f7d440b69a0 legacy=0x7f7d4413f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:58.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.238+0000 7f7d49b22700 1 -- 172.21.15.162:0/2564244779 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d44063600 con 0x7f7d4413fd00 2020-11-28T21:55:58.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.238+0000 7f7d49b22700 1 --2- 172.21.15.162:0/2564244779 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d4413fd00 0x7f7d4413c410 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 rx=0x7f7d2c0010d0 tx=0x7f7d3c0091a0).ready entity=mon.1 client_cookie=a9557c543157a72a server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:58.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.238+0000 7f7d49321700 1 -- 172.21.15.162:0/2564244779 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d2c0016d0 con 0x7f7d4413fd00 2020-11-28T21:55:58.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.238+0000 7f7d49321700 1 -- 172.21.15.162:0/2564244779 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d3c00b920 con 0x7f7d4413fd00 2020-11-28T21:55:58.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.238+0000 7f7d4c587700 1 -- 172.21.15.162:0/2564244779 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d4413fd00 msgr2=0x7f7d4413c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:58.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.238+0000 7f7d49321700 1 -- 172.21.15.162:0/2564244779 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d3c00ba90 con 0x7f7d4413fd00 2020-11-28T21:55:58.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.238+0000 7f7d4c587700 1 --2- 172.21.15.162:0/2564244779 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d4413fd00 0x7f7d4413c410 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 rx=0x7f7d2c0010d0 tx=0x7f7d3c0091a0).stop 2020-11-28T21:55:58.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.242+0000 7f7d4c587700 1 -- 172.21.15.162:0/2564244779 shutdown_connections 2020-11-28T21:55:58.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.242+0000 7f7d4c587700 1 --2- 172.21.15.162:0/2564244779 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d4413fd00 0x7f7d4413c410 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:58.249 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.242+0000 7f7d4c587700 1 -- 172.21.15.162:0/2564244779 >> 172.21.15.162:0/2564244779 conn(0x7f7d44062170 msgr2=0x7f7d440b5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:58.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.242+0000 7f7d4c587700 1 -- 172.21.15.162:0/2564244779 shutdown_connections 2020-11-28T21:55:58.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.242+0000 7f7d4c587700 1 -- 172.21.15.162:0/2564244779 wait complete. 2020-11-28T21:55:58.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.242+0000 7f7d4c587700 1 Processor -- start 2020-11-28T21:55:58.253 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.242+0000 7f7d4c587700 1 -- start start 2020-11-28T21:55:58.253 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.242+0000 7f7d4c587700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d440b69a0 0x7f7d440645f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:58.253 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.242+0000 7f7d4c587700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7d4413c900 0x7f7d44066ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:58.253 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.242+0000 7f7d4c587700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7d4413fd00 0x7f7d4413e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:58.253 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.242+0000 7f7d4c587700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d44134d60 con 0x7f7d4413c900 2020-11-28T21:55:58.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.242+0000 7f7d4c587700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d44065d20 con 0x7f7d440b69a0 2020-11-28T21:55:58.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.242+0000 7f7d4c587700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d44065fc0 con 0x7f7d4413fd00 2020-11-28T21:55:58.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.242+0000 7f7d4a323700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d440b69a0 0x7f7d440645f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:58.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.242+0000 7f7d4a323700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d440b69a0 0x7f7d440645f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45516/0 (socket says 172.21.15.162:45516) 2020-11-28T21:55:58.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.242+0000 7f7d4a323700 1 -- 172.21.15.162:0/860723236 learned_addr learned my addr 172.21.15.162:0/860723236 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:58.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.242+0000 7f7d49b22700 1 --2- 172.21.15.162:0/860723236 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7d4413c900 0x7f7d44066ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:58.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.242+0000 7f7d4ab24700 1 --2- 172.21.15.162:0/860723236 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7d4413fd00 0x7f7d4413e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:58.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.242+0000 7f7d4a323700 1 -- 172.21.15.162:0/860723236 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7d4413fd00 msgr2=0x7f7d4413e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:58.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.242+0000 7f7d4a323700 1 --2- 172.21.15.162:0/860723236 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7d4413fd00 0x7f7d4413e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:58.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.242+0000 7f7d4a323700 1 -- 172.21.15.162:0/860723236 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7d4413c900 msgr2=0x7f7d44066ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:58.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.242+0000 7f7d4a323700 1 --2- 172.21.15.162:0/860723236 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7d4413c900 0x7f7d44066ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:58.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.242+0000 7f7d4a323700 1 -- 172.21.15.162:0/860723236 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d44134f50 con 0x7f7d440b69a0 2020-11-28T21:55:58.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.242+0000 7f7d49b22700 1 --2- 172.21.15.162:0/860723236 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7d4413c900 0x7f7d44066ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:55:58.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.242+0000 7f7d4a323700 1 --2- 172.21.15.162:0/860723236 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d440b69a0 0x7f7d440645f0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 rx=0x7f7d4413f640 tx=0x7f7d34007a60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:58.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.242+0000 7f7d437fe700 1 -- 172.21.15.162:0/860723236 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d3400e4a0 con 0x7f7d440b69a0 2020-11-28T21:55:58.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.242+0000 7f7d437fe700 1 -- 172.21.15.162:0/860723236 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d3400e600 con 0x7f7d440b69a0 2020-11-28T21:55:58.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.242+0000 7f7d4c587700 1 -- 172.21.15.162:0/860723236 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d4413ede0 con 0x7f7d440b69a0 2020-11-28T21:55:58.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.246+0000 7f7d437fe700 1 -- 172.21.15.162:0/860723236 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d3400e960 con 0x7f7d440b69a0 2020-11-28T21:55:58.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.246+0000 7f7d4c587700 1 -- 172.21.15.162:0/860723236 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d44067570 con 0x7f7d440b69a0 2020-11-28T21:55:58.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.246+0000 7f7d4c587700 1 -- 172.21.15.162:0/860723236 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d44057050 con 0x7f7d440b69a0 2020-11-28T21:55:58.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.246+0000 7f7d437fe700 1 -- 172.21.15.162:0/860723236 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7d34019030 con 0x7f7d440b69a0 2020-11-28T21:55:58.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.246+0000 7f7d437fe700 1 --2- 172.21.15.162:0/860723236 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7d28055330 0x7f7d280577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:58.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.246+0000 7f7d437fe700 1 -- 172.21.15.162:0/860723236 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f7d3406cfa0 con 0x7f7d440b69a0 2020-11-28T21:55:58.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.246+0000 7f7d49b22700 1 --2- 172.21.15.162:0/860723236 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7d28055330 0x7f7d280577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:58.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.246+0000 7f7d49b22700 1 --2- 172.21.15.162:0/860723236 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7d28055330 0x7f7d280577f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7f7d3c01a000 tx=0x7f7d3c016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:58.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.250+0000 7f7d437fe700 1 -- 172.21.15.162:0/860723236 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7d34043530 con 0x7f7d440b69a0 2020-11-28T21:55:58.445 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.438+0000 7f7d4c587700 1 -- 172.21.15.162:0/860723236 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f7d44067570 con 0x7f7d440b69a0 2020-11-28T21:55:58.446 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.442+0000 7f7d437fe700 1 -- 172.21.15.162:0/860723236 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v30) v1 ==== 80+0+4255 (secure 0 0 0) 0x7f7d44067570 con 0x7f7d440b69a0 2020-11-28T21:55:58.447 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:58.447 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2020-11-28T21:55:58.447 INFO:teuthology.orchestra.run.smithi162.stdout: "auth_dump": [ 2020-11-28T21:55:58.448 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:58.448 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mds.a", 2020-11-28T21:55:58.449 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfWxUpKxAAIUDwQRKwz8H4Qk9qE356Bg==", 2020-11-28T21:55:58.449 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:55:58.449 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:55:58.450 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:55:58.450 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:55:58.450 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:55:58.451 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:58.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:58.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:58.452 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mds.b", 2020-11-28T21:55:58.452 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfKY2zORAAaO65etMZiYcaTDNX0nsEPw==", 2020-11-28T21:55:58.452 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:55:58.453 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:55:58.453 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:55:58.453 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:55:58.454 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:55:58.454 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:58.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:58.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:58.455 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mds.c", 2020-11-28T21:55:58.455 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfdQbRMxAAWlcjesxTZzZnvKCQneSjjQ==", 2020-11-28T21:55:58.456 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:55:58.456 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:55:58.457 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:55:58.457 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:55:58.457 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:55:58.458 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:58.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:58.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:58.459 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.0", 2020-11-28T21:55:58.459 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQADxcJfF3WLLxAAxsB5YPjH1yjv/vhlRbijqw==", 2020-11-28T21:55:58.459 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:55:58.460 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:55:58.460 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:55:58.460 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:55:58.461 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:58.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:58.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:58.462 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.1", 2020-11-28T21:55:58.462 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAGxcJfKYbBFhAA1x4DzFtkQQv6uWLIdOi1qQ==", 2020-11-28T21:55:58.462 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:55:58.463 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:55:58.463 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:55:58.464 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:55:58.464 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:58.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:58.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:58.465 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.2", 2020-11-28T21:55:58.465 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAIxcJfvcYUOhAASENjM0HfAxxyFe13mmE36Q==", 2020-11-28T21:55:58.466 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:55:58.466 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:55:58.466 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:55:58.467 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:55:58.467 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:58.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:58.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:58.468 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.3", 2020-11-28T21:55:58.468 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQALxcJfc/OHIRAAFYx3s8Ssttt15hjoH04tow==", 2020-11-28T21:55:58.469 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:55:58.469 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:55:58.469 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:55:58.470 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:55:58.470 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:58.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:58.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:58.471 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.4", 2020-11-28T21:55:58.471 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAPxcJffGWqCRAAb8CCN5oUPkdDh6BJH5HEnw==", 2020-11-28T21:55:58.472 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:55:58.472 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:55:58.473 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:55:58.473 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:55:58.473 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:58.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:58.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:58.475 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.5", 2020-11-28T21:55:58.475 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQARxcJfo8GKKRAAfbsCDb4L2wULiKmHyzu7/A==", 2020-11-28T21:55:58.475 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:55:58.476 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:55:58.476 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:55:58.476 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:55:58.477 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:58.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:58.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:58.478 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.6", 2020-11-28T21:55:58.478 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAUxcJf8Y2MEBAAqQX6rX3LK0zodlI2wK4JrA==", 2020-11-28T21:55:58.478 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:55:58.479 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:55:58.479 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:55:58.480 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:55:58.480 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:58.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:58.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:58.481 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.7", 2020-11-28T21:55:58.482 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAWxcJfCLBdMhAAf8QixQMQDrfZOGWN8rHe5g==", 2020-11-28T21:55:58.482 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:55:58.482 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:55:58.483 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:55:58.483 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:55:58.483 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:58.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:58.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:58.484 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "client.0", 2020-11-28T21:55:58.485 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQABxcJfS9vDAxAAUk4FU8FVt4if/CRH5ZisOA==", 2020-11-28T21:55:58.485 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:55:58.485 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:55:58.486 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow r", 2020-11-28T21:55:58.486 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow rw pool=cephfs_data" 2020-11-28T21:55:58.486 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:58.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:58.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:58.487 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "client.1", 2020-11-28T21:55:58.488 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQABxcJfuCU8BhAAXUSoSK3D0XRo58uHNzcjXg==", 2020-11-28T21:55:58.488 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:55:58.488 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:55:58.489 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow r", 2020-11-28T21:55:58.489 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow rw pool=cephfs_data" 2020-11-28T21:55:58.489 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:58.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:58.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:58.491 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "client.admin", 2020-11-28T21:55:58.491 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfdlTWBxAApzAxlVjzdnETQF//Vnsg5w==", 2020-11-28T21:55:58.491 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:55:58.492 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow *", 2020-11-28T21:55:58.492 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:55:58.492 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:55:58.493 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:55:58.493 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:58.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:58.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:58.494 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mgr.x", 2020-11-28T21:55:58.494 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfZhdvKBAAT7zCCsD2OVCHjvddGLxQ/A==", 2020-11-28T21:55:58.494 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:55:58.494 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow *", 2020-11-28T21:55:58.495 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow profile mgr", 2020-11-28T21:55:58.495 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:55:58.495 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:58.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:58.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:58.496 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mgr.y", 2020-11-28T21:55:58.496 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfs8fvJRAAP0Atp074EeMHiTtpCCVHuA==", 2020-11-28T21:55:58.496 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:55:58.497 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow *", 2020-11-28T21:55:58.497 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow profile mgr", 2020-11-28T21:55:58.497 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:55:58.497 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:58.497 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:58.498 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:55:58.498 INFO:teuthology.orchestra.run.smithi162.stdout:} 2020-11-28T21:55:58.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.442+0000 7f7d4c587700 1 -- 172.21.15.162:0/860723236 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7d28055330 msgr2=0x7f7d280577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:58.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.442+0000 7f7d4c587700 1 --2- 172.21.15.162:0/860723236 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7d28055330 0x7f7d280577f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7f7d3c01a000 tx=0x7f7d3c016000).stop 2020-11-28T21:55:58.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.442+0000 7f7d4c587700 1 -- 172.21.15.162:0/860723236 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d440b69a0 msgr2=0x7f7d440645f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:58.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.442+0000 7f7d4c587700 1 --2- 172.21.15.162:0/860723236 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d440b69a0 0x7f7d440645f0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 rx=0x7f7d4413f640 tx=0x7f7d34007a60).stop 2020-11-28T21:55:58.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.442+0000 7f7d4c587700 1 -- 172.21.15.162:0/860723236 shutdown_connections 2020-11-28T21:55:58.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.442+0000 7f7d4c587700 1 --2- 172.21.15.162:0/860723236 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7d28055330 0x7f7d280577f0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:58.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.442+0000 7f7d4c587700 1 --2- 172.21.15.162:0/860723236 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7d4413fd00 0x7f7d4413e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:58.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.442+0000 7f7d4c587700 1 --2- 172.21.15.162:0/860723236 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d440b69a0 0x7f7d440645f0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:58.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.442+0000 7f7d4c587700 1 --2- 172.21.15.162:0/860723236 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7d4413c900 0x7f7d44066ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:58.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.442+0000 7f7d4c587700 1 -- 172.21.15.162:0/860723236 >> 172.21.15.162:0/860723236 conn(0x7f7d44062170 msgr2=0x7f7d440b0820 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:58.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.442+0000 7f7d4c587700 1 -- 172.21.15.162:0/860723236 shutdown_connections 2020-11-28T21:55:58.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.442+0000 7f7d4c587700 1 -- 172.21.15.162:0/860723236 wait complete. 2020-11-28T21:55:58.503 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:55:58.616 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.610+0000 7f7ccb59e700 1 Processor -- start 2020-11-28T21:55:58.616 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.610+0000 7f7ccb59e700 1 -- start start 2020-11-28T21:55:58.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.610+0000 7f7ccb59e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7ccc0b3e20 0x7f7ccc0b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:58.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.610+0000 7f7ccb59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7ccc0608e0 0x7f7ccc146b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:58.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.610+0000 7f7ccb59e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ccc0b0830 con 0x7f7ccc147050 2020-11-28T21:55:58.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.610+0000 7f7ccb59e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ccc0b4200 con 0x7f7ccc0b3e20 2020-11-28T21:55:58.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.610+0000 7f7ccb59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc060e60 con 0x7f7ccc0608e0 2020-11-28T21:55:58.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.610+0000 7f7cca59c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7ccc0b3e20 0x7f7ccc0b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:58.619 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.610+0000 7f7ccad9d700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f7ccc147050 0x7f7ccc0b11a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:52370/0 (socket says 172.21.15.162:52370) 2020-11-28T21:55:58.620 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.610+0000 7f7ccad9d700 1 -- 172.21.15.162:0/778808264 learned_addr learned my addr 172.21.15.162:0/778808264 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:58.620 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.614+0000 7f7cc9d9b700 1 --2- 172.21.15.162:0/778808264 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7ccc0608e0 0x7f7ccc146b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:58.620 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.614+0000 7f7cca59c700 1 -- 172.21.15.162:0/778808264 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7ccc0608e0 msgr2=0x7f7ccc146b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:58.620 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.614+0000 7f7cca59c700 1 --2- 172.21.15.162:0/778808264 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7ccc0608e0 0x7f7ccc146b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:58.620 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.614+0000 7f7cca59c700 1 -- 172.21.15.162:0/778808264 >> v1:172.21.15.162:6789/0 conn(0x7f7ccc147050 legacy=0x7f7ccc0b11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:58.622 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.614+0000 7f7cc959a700 1 -- 172.21.15.162:0/778808264 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3404131754 0 0) 0x7f7ccc0b0830 con 0x7f7ccc147050 2020-11-28T21:55:58.622 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.614+0000 7f7cca59c700 1 -- 172.21.15.162:0/778808264 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ccc0b0a60 con 0x7f7ccc0b3e20 2020-11-28T21:55:58.622 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.614+0000 7f7cc9d9b700 1 --2- 172.21.15.162:0/778808264 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7ccc0608e0 0x7f7ccc146b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:55:58.622 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.614+0000 7f7cca59c700 1 --2- 172.21.15.162:0/778808264 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ccc0b3e20 0x7f7ccc0b4680 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 rx=0x7f7cbc002f30 tx=0x7f7cbc006dc0).ready entity=mon.1 client_cookie=3007c9e25e8ff78a server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:58.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.614+0000 7f7cc959a700 1 -- 172.21.15.162:0/778808264 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7cbc0097f0 con 0x7f7ccc0b3e20 2020-11-28T21:55:58.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.614+0000 7f7cc959a700 1 -- 172.21.15.162:0/778808264 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cbc009950 con 0x7f7ccc0b3e20 2020-11-28T21:55:58.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.614+0000 7f7cc959a700 1 -- 172.21.15.162:0/778808264 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7cbc009cb0 con 0x7f7ccc0b3e20 2020-11-28T21:55:58.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.614+0000 7f7ccb59e700 1 -- 172.21.15.162:0/778808264 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ccc0b3e20 msgr2=0x7f7ccc0b4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:58.624 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.614+0000 7f7ccb59e700 1 --2- 172.21.15.162:0/778808264 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ccc0b3e20 0x7f7ccc0b4680 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 rx=0x7f7cbc002f30 tx=0x7f7cbc006dc0).stop 2020-11-28T21:55:58.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.614+0000 7f7ccb59e700 1 -- 172.21.15.162:0/778808264 shutdown_connections 2020-11-28T21:55:58.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.614+0000 7f7ccb59e700 1 --2- 172.21.15.162:0/778808264 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ccc0b3e20 0x7f7ccc0b4680 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:58.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.614+0000 7f7ccb59e700 1 --2- 172.21.15.162:0/778808264 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7ccc0608e0 0x7f7ccc146b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:58.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.614+0000 7f7ccb59e700 1 -- 172.21.15.162:0/778808264 >> 172.21.15.162:0/778808264 conn(0x7f7ccc0b54f0 msgr2=0x7f7ccc0b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:58.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.614+0000 7f7ccb59e700 1 -- 172.21.15.162:0/778808264 shutdown_connections 2020-11-28T21:55:58.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.614+0000 7f7ccb59e700 1 -- 172.21.15.162:0/778808264 wait complete. 2020-11-28T21:55:58.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.614+0000 7f7ccb59e700 1 Processor -- start 2020-11-28T21:55:58.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.614+0000 7f7ccb59e700 1 -- start start 2020-11-28T21:55:58.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.614+0000 7f7ccb59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ccc0608e0 0x7f7ccc146190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:58.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.614+0000 7f7ccb59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7ccc0b3e20 0x7f7ccc098590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:58.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.614+0000 7f7ccb59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7ccc147050 0x7f7ccc098a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:58.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.614+0000 7f7ccb59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc0ae300 con 0x7f7ccc0b3e20 2020-11-28T21:55:58.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.614+0000 7f7ccb59e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc0adf10 con 0x7f7ccc0608e0 2020-11-28T21:55:58.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.614+0000 7f7ccb59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc098f70 con 0x7f7ccc147050 2020-11-28T21:55:58.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.614+0000 7f7cca59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ccc0608e0 0x7f7ccc146190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:58.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.614+0000 7f7cca59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ccc0608e0 0x7f7ccc146190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45532/0 (socket says 172.21.15.162:45532) 2020-11-28T21:55:58.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.614+0000 7f7cca59c700 1 -- 172.21.15.162:0/4203430787 learned_addr learned my addr 172.21.15.162:0/4203430787 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:58.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.614+0000 7f7cc9d9b700 1 --2- 172.21.15.162:0/4203430787 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7ccc0b3e20 0x7f7ccc098590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:58.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.618+0000 7f7ccad9d700 1 --2- 172.21.15.162:0/4203430787 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7ccc147050 0x7f7ccc098a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:58.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.618+0000 7f7cca59c700 1 -- 172.21.15.162:0/4203430787 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7ccc147050 msgr2=0x7f7ccc098a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:58.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.618+0000 7f7cca59c700 1 --2- 172.21.15.162:0/4203430787 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7ccc147050 0x7f7ccc098a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:58.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.618+0000 7f7cca59c700 1 -- 172.21.15.162:0/4203430787 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7ccc0b3e20 msgr2=0x7f7ccc098590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:58.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.618+0000 7f7cca59c700 1 --2- 172.21.15.162:0/4203430787 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7ccc0b3e20 0x7f7ccc098590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:58.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.618+0000 7f7cca59c700 1 -- 172.21.15.162:0/4203430787 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ccc0aecf0 con 0x7f7ccc0608e0 2020-11-28T21:55:58.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.618+0000 7f7ccad9d700 1 --2- 172.21.15.162:0/4203430787 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7ccc147050 0x7f7ccc098a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:55:58.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.618+0000 7f7cca59c700 1 --2- 172.21.15.162:0/4203430787 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ccc0608e0 0x7f7ccc146190 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 rx=0x7f7cbc002f30 tx=0x7f7cbc015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:58.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.618+0000 7f7cbb7fe700 1 -- 172.21.15.162:0/4203430787 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7cbc011ce0 con 0x7f7ccc0608e0 2020-11-28T21:55:58.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.618+0000 7f7cbb7fe700 1 -- 172.21.15.162:0/4203430787 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cbc011e40 con 0x7f7ccc0608e0 2020-11-28T21:55:58.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.618+0000 7f7ccb59e700 1 -- 172.21.15.162:0/4203430787 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ccc09b010 con 0x7f7ccc0608e0 2020-11-28T21:55:58.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.618+0000 7f7cbb7fe700 1 -- 172.21.15.162:0/4203430787 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7cbc01d640 con 0x7f7ccc0608e0 2020-11-28T21:55:58.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.618+0000 7f7ccb59e700 1 -- 172.21.15.162:0/4203430787 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ccc0af190 con 0x7f7ccc0608e0 2020-11-28T21:55:58.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.618+0000 7f7cbb7fe700 1 -- 172.21.15.162:0/4203430787 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7cbc024030 con 0x7f7ccc0608e0 2020-11-28T21:55:58.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.618+0000 7f7ccb59e700 1 -- 172.21.15.162:0/4203430787 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ccc0ae4c0 con 0x7f7ccc0608e0 2020-11-28T21:55:58.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.618+0000 7f7cbb7fe700 1 --2- 172.21.15.162:0/4203430787 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7ca80553b0 0x7f7ca8057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:58.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.618+0000 7f7cbb7fe700 1 -- 172.21.15.162:0/4203430787 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f7cbc06f030 con 0x7f7ccc0608e0 2020-11-28T21:55:58.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.618+0000 7f7cc9d9b700 1 --2- 172.21.15.162:0/4203430787 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7ca80553b0 0x7f7ca8057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:58.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.618+0000 7f7cc9d9b700 1 --2- 172.21.15.162:0/4203430787 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7ca80553b0 0x7f7ca8057870 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7f7cc000d000 tx=0x7f7cc0008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:58.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.626+0000 7f7cbb7fe700 1 -- 172.21.15.162:0/4203430787 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7cbc045660 con 0x7f7ccc0608e0 2020-11-28T21:55:58.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.762+0000 7f7ccb59e700 1 -- 172.21.15.162:0/4203430787 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ccc057050 con 0x7f7ccc0608e0 2020-11-28T21:55:58.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.766+0000 7f7cbb7fe700 1 -- 172.21.15.162:0/4203430787 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) v1 ==== 74+0+13287 (secure 0 0 0) 0x7f7cbc022070 con 0x7f7ccc0608e0 2020-11-28T21:55:58.771 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:58.772 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":144,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:55:58.098816+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:55:58.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.766+0000 7f7ccb59e700 1 -- 172.21.15.162:0/4203430787 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7ca80553b0 msgr2=0x7f7ca8057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:58.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.766+0000 7f7ccb59e700 1 --2- 172.21.15.162:0/4203430787 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7ca80553b0 0x7f7ca8057870 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7f7cc000d000 tx=0x7f7cc0008040).stop 2020-11-28T21:55:58.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.766+0000 7f7ccb59e700 1 -- 172.21.15.162:0/4203430787 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ccc0608e0 msgr2=0x7f7ccc146190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:58.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.766+0000 7f7ccb59e700 1 --2- 172.21.15.162:0/4203430787 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ccc0608e0 0x7f7ccc146190 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 rx=0x7f7cbc002f30 tx=0x7f7cbc015000).stop 2020-11-28T21:55:58.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.766+0000 7f7ccb59e700 1 -- 172.21.15.162:0/4203430787 shutdown_connections 2020-11-28T21:55:58.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.766+0000 7f7ccb59e700 1 --2- 172.21.15.162:0/4203430787 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7ca80553b0 0x7f7ca8057870 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:58.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.766+0000 7f7ccb59e700 1 --2- 172.21.15.162:0/4203430787 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7ccc147050 0x7f7ccc098a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:58.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.766+0000 7f7ccb59e700 1 --2- 172.21.15.162:0/4203430787 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ccc0608e0 0x7f7ccc146190 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:58.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.766+0000 7f7ccb59e700 1 --2- 172.21.15.162:0/4203430787 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7ccc0b3e20 0x7f7ccc098590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:58.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.766+0000 7f7ccb59e700 1 -- 172.21.15.162:0/4203430787 >> 172.21.15.162:0/4203430787 conn(0x7f7ccc0b54f0 msgr2=0x7f7ccc0aba90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:58.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.766+0000 7f7ccb59e700 1 -- 172.21.15.162:0/4203430787 shutdown_connections 2020-11-28T21:55:58.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.766+0000 7f7ccb59e700 1 -- 172.21.15.162:0/4203430787 wait complete. 2020-11-28T21:55:58.784 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-28T21:55:58.785 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:55:58.903 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.898+0000 7fc73759e700 1 Processor -- start 2020-11-28T21:55:58.903 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.898+0000 7fc73759e700 1 -- start start 2020-11-28T21:55:58.904 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.898+0000 7fc73759e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc73805b970 0x7fc7381405b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:58.904 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.898+0000 7fc73759e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fc73805bd50 0x7fc738141010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:58.904 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.898+0000 7fc73759e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7380613d0 con 0x7fc7380b1be0 2020-11-28T21:55:58.905 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.898+0000 7fc73759e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc738061670 con 0x7fc73805bd50 2020-11-28T21:55:58.905 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.898+0000 7fc73759e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc738061910 con 0x7fc73805b970 2020-11-28T21:55:58.907 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.898+0000 7fc735d9b700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fc73805bd50 0x7fc738141010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:58.907 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.898+0000 7fc736d9d700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fc7380b1be0 0x7fc7380b0ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.162:42532/0 (socket says 172.21.15.162:42532) 2020-11-28T21:55:58.907 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.898+0000 7fc735d9b700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fc73805bd50 0x7fc738141010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.162:52528/0 (socket says 172.21.15.162:52528) 2020-11-28T21:55:58.908 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.898+0000 7fc736d9d700 1 -- 172.21.15.162:0/1648587500 learned_addr learned my addr 172.21.15.162:0/1648587500 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:58.908 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.898+0000 7fc73659c700 1 --2- 172.21.15.162:0/1648587500 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc73805b970 0x7fc7381405b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:58.909 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.898+0000 7fc735d9b700 1 -- 172.21.15.162:0/1648587500 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc73805b970 msgr2=0x7fc7381405b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:58.909 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.898+0000 7fc735d9b700 1 --2- 172.21.15.162:0/1648587500 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc73805b970 0x7fc7381405b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:58.909 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.898+0000 7fc735d9b700 1 -- 172.21.15.162:0/1648587500 >> v1:172.21.15.118:6789/0 conn(0x7fc7380b1be0 legacy=0x7fc7380b0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:58.910 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.898+0000 7fc735d9b700 1 -- 172.21.15.162:0/1648587500 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc738061bb0 con 0x7fc73805bd50 2020-11-28T21:55:58.910 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.902+0000 7fc735d9b700 1 --2- 172.21.15.162:0/1648587500 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc73805bd50 0x7fc738141010 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7fc728002f30 tx=0x7fc728006dc0).ready entity=mon.0 client_cookie=e7a278c598c7b953 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:58.910 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.902+0000 7fc73559a700 1 -- 172.21.15.162:0/1648587500 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc728009760 con 0x7fc73805bd50 2020-11-28T21:55:58.910 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.902+0000 7fc73559a700 1 -- 172.21.15.162:0/1648587500 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7280098c0 con 0x7fc73805bd50 2020-11-28T21:55:58.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.902+0000 7fc73559a700 1 -- 172.21.15.162:0/1648587500 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc728009c20 con 0x7fc73805bd50 2020-11-28T21:55:58.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.902+0000 7fc73759e700 1 -- 172.21.15.162:0/1648587500 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc73805bd50 msgr2=0x7fc738141010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:58.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.902+0000 7fc73759e700 1 --2- 172.21.15.162:0/1648587500 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc73805bd50 0x7fc738141010 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7fc728002f30 tx=0x7fc728006dc0).stop 2020-11-28T21:55:58.911 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.902+0000 7fc73759e700 1 -- 172.21.15.162:0/1648587500 shutdown_connections 2020-11-28T21:55:58.912 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.902+0000 7fc73759e700 1 --2- 172.21.15.162:0/1648587500 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc73805b970 0x7fc7381405b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:58.912 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.902+0000 7fc73759e700 1 --2- 172.21.15.162:0/1648587500 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc73805bd50 0x7fc738141010 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:58.912 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.902+0000 7fc73759e700 1 -- 172.21.15.162:0/1648587500 >> 172.21.15.162:0/1648587500 conn(0x7fc7380b2830 msgr2=0x7fc7380b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:58.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.902+0000 7fc73759e700 1 -- 172.21.15.162:0/1648587500 shutdown_connections 2020-11-28T21:55:58.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.902+0000 7fc73759e700 1 -- 172.21.15.162:0/1648587500 wait complete. 2020-11-28T21:55:58.917 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.902+0000 7fc73759e700 1 Processor -- start 2020-11-28T21:55:58.918 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.902+0000 7fc73759e700 1 -- start start 2020-11-28T21:55:58.918 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.902+0000 7fc73759e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc73805b970 0x7fc738090a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:58.918 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.902+0000 7fc73759e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc73805bd50 0x7fc738143530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:58.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.902+0000 7fc73759e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7380b1be0 0x7fc738143a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:58.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.902+0000 7fc73759e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc738142420 con 0x7fc73805b970 2020-11-28T21:55:58.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.902+0000 7fc73759e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc738141b00 con 0x7fc73805bd50 2020-11-28T21:55:58.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.902+0000 7fc73759e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc738142820 con 0x7fc7380b1be0 2020-11-28T21:55:58.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.906+0000 7fc735d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc73805bd50 0x7fc738143530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:58.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.906+0000 7fc735d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc73805bd50 0x7fc738143530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45546/0 (socket says 172.21.15.162:45546) 2020-11-28T21:55:58.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.906+0000 7fc73659c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc73805b970 0x7fc738090a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:58.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.906+0000 7fc735d9b700 1 -- 172.21.15.162:0/2722407526 learned_addr learned my addr 172.21.15.162:0/2722407526 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:58.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.906+0000 7fc736d9d700 1 --2- 172.21.15.162:0/2722407526 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7380b1be0 0x7fc738143a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:58.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.906+0000 7fc735d9b700 1 -- 172.21.15.162:0/2722407526 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7380b1be0 msgr2=0x7fc738143a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:58.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.906+0000 7fc735d9b700 1 --2- 172.21.15.162:0/2722407526 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7380b1be0 0x7fc738143a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:58.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.906+0000 7fc735d9b700 1 -- 172.21.15.162:0/2722407526 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc73805b970 msgr2=0x7fc738090a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:58.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.906+0000 7fc735d9b700 1 --2- 172.21.15.162:0/2722407526 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc73805b970 0x7fc738090a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:58.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.906+0000 7fc735d9b700 1 -- 172.21.15.162:0/2722407526 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc738152040 con 0x7fc73805bd50 2020-11-28T21:55:58.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.906+0000 7fc735d9b700 1 --2- 172.21.15.162:0/2722407526 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc73805bd50 0x7fc738143530 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 rx=0x7fc728002f30 tx=0x7fc728015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:58.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.906+0000 7fc7277fe700 1 -- 172.21.15.162:0/2722407526 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc728009c40 con 0x7fc73805bd50 2020-11-28T21:55:58.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.906+0000 7fc7277fe700 1 -- 172.21.15.162:0/2722407526 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc728011ce0 con 0x7fc73805bd50 2020-11-28T21:55:58.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.906+0000 7fc73759e700 1 -- 172.21.15.162:0/2722407526 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7381522e0 con 0x7fc73805bd50 2020-11-28T21:55:58.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.906+0000 7fc7277fe700 1 -- 172.21.15.162:0/2722407526 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc72801d630 con 0x7fc73805bd50 2020-11-28T21:55:58.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.906+0000 7fc73759e700 1 -- 172.21.15.162:0/2722407526 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc738142c80 con 0x7fc73805bd50 2020-11-28T21:55:58.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.906+0000 7fc73759e700 1 -- 172.21.15.162:0/2722407526 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc738057050 con 0x7fc73805bd50 2020-11-28T21:55:58.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.906+0000 7fc7277fe700 1 -- 172.21.15.162:0/2722407526 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc728024030 con 0x7fc73805bd50 2020-11-28T21:55:58.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.906+0000 7fc7277fe700 1 --2- 172.21.15.162:0/2722407526 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc714055440 0x7fc714057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:58.924 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.910+0000 7fc7277fe700 1 -- 172.21.15.162:0/2722407526 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fc72806edb0 con 0x7fc73805bd50 2020-11-28T21:55:58.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.910+0000 7fc73659c700 1 --2- 172.21.15.162:0/2722407526 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc714055440 0x7fc714057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:58.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.910+0000 7fc73659c700 1 --2- 172.21.15.162:0/2722407526 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc714055440 0x7fc714057900 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7fc72c00b000 tx=0x7fc72c006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:58.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:58.914+0000 7fc7277fe700 1 -- 172.21.15.162:0/2722407526 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc728073070 con 0x7fc73805bd50 2020-11-28T21:55:59.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.054+0000 7fc73759e700 1 -- 172.21.15.162:0/2722407526 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fc738068350 con 0x7fc73805bd50 2020-11-28T21:55:59.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.054+0000 7fc7277fe700 1 -- 172.21.15.162:0/2722407526 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v144) v1 ==== 110+0+10 (secure 0 0 0) 0x7fc728022070 con 0x7fc73805bd50 2020-11-28T21:55:59.061 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:55:59.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.058+0000 7fc73759e700 1 -- 172.21.15.162:0/2722407526 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc714055440 msgr2=0x7fc714057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:59.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.058+0000 7fc73759e700 1 --2- 172.21.15.162:0/2722407526 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc714055440 0x7fc714057900 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7fc72c00b000 tx=0x7fc72c006dc0).stop 2020-11-28T21:55:59.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.058+0000 7fc73759e700 1 -- 172.21.15.162:0/2722407526 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc73805bd50 msgr2=0x7fc738143530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:59.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.058+0000 7fc73759e700 1 --2- 172.21.15.162:0/2722407526 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc73805bd50 0x7fc738143530 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 rx=0x7fc728002f30 tx=0x7fc728015000).stop 2020-11-28T21:55:59.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.058+0000 7fc73759e700 1 -- 172.21.15.162:0/2722407526 shutdown_connections 2020-11-28T21:55:59.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.058+0000 7fc73759e700 1 --2- 172.21.15.162:0/2722407526 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc714055440 0x7fc714057900 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:59.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.058+0000 7fc73759e700 1 --2- 172.21.15.162:0/2722407526 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7380b1be0 0x7fc738143a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:59.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.058+0000 7fc73759e700 1 --2- 172.21.15.162:0/2722407526 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc73805b970 0x7fc738090a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:59.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.058+0000 7fc73759e700 1 --2- 172.21.15.162:0/2722407526 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc73805bd50 0x7fc738143530 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:59.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.058+0000 7fc73759e700 1 -- 172.21.15.162:0/2722407526 >> 172.21.15.162:0/2722407526 conn(0x7fc7380b2830 msgr2=0x7fc7380ae240 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:59.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.058+0000 7fc73759e700 1 -- 172.21.15.162:0/2722407526 shutdown_connections 2020-11-28T21:55:59.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.058+0000 7fc73759e700 1 -- 172.21.15.162:0/2722407526 wait complete. 2020-11-28T21:55:59.075 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:55:59.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.190+0000 7fef7d1f2700 1 Processor -- start 2020-11-28T21:55:59.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.190+0000 7fef7d1f2700 1 -- start start 2020-11-28T21:55:59.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.190+0000 7fef7d1f2700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fef780b18c0 0x7fef780b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:59.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.190+0000 7fef7d1f2700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef7814a440 con 0x7fef780b1dd0 2020-11-28T21:55:59.200 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.190+0000 7fef7d1f2700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef7814b2f0 con 0x7fef780b3950 2020-11-28T21:55:59.200 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.190+0000 7fef7d1f2700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef7814a670 con 0x7fef780b18c0 2020-11-28T21:55:59.200 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.190+0000 7fef77fff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fef780b3950 0x7fef78060bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:52396/0 (socket says 172.21.15.162:52396) 2020-11-28T21:55:59.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.190+0000 7fef77fff700 1 -- 172.21.15.162:0/1060809605 learned_addr learned my addr 172.21.15.162:0/1060809605 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:59.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.190+0000 7fef777fe700 1 --2- 172.21.15.162:0/1060809605 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fef780b18c0 0x7fef780b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:59.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.190+0000 7fef76ffd700 1 -- 172.21.15.162:0/1060809605 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2982432079 0 0) 0x7fef7814b2f0 con 0x7fef780b3950 2020-11-28T21:55:59.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.190+0000 7fef76ffd700 1 -- 172.21.15.162:0/1060809605 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef64003230 con 0x7fef780b3950 2020-11-28T21:55:59.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.190+0000 7fef76ffd700 1 -- 172.21.15.162:0/1060809605 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 175483366 0 0) 0x7fef64003230 con 0x7fef780b3950 2020-11-28T21:55:59.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.190+0000 7fef76ffd700 1 -- 172.21.15.162:0/1060809605 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fef780b18c0 msgr2=0x7fef780b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:59.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.190+0000 7fef76ffd700 1 --2- 172.21.15.162:0/1060809605 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fef780b18c0 0x7fef780b3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:59.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.190+0000 7fef76ffd700 1 -- 172.21.15.162:0/1060809605 >> v1:172.21.15.118:6789/0 conn(0x7fef780b1dd0 legacy=0x7fef78146e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:59.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.190+0000 7fef76ffd700 1 -- 172.21.15.162:0/1060809605 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef7814a910 con 0x7fef780b3950 2020-11-28T21:55:59.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.190+0000 7fef76ffd700 1 -- 172.21.15.162:0/1060809605 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7fef680016d0 con 0x7fef780b3950 2020-11-28T21:55:59.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.194+0000 7fef76ffd700 1 -- 172.21.15.162:0/1060809605 >> v1:172.21.15.162:6789/0 conn(0x7fef780b3950 legacy=0x7fef78060bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:59.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.194+0000 7fef76ffd700 1 --2- 172.21.15.162:0/1060809605 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fef780b18c0 0x7fef64003720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:59.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.194+0000 7fef76ffd700 1 --2- 172.21.15.162:0/1060809605 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fef64003c10 0x7fef640044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:59.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.194+0000 7fef76ffd700 1 --2- 172.21.15.162:0/1060809605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef640049c0 0x7fef64004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:59.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.194+0000 7fef76ffd700 1 -- 172.21.15.162:0/1060809605 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7814a670 con 0x7fef64003c10 2020-11-28T21:55:59.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.194+0000 7fef76ffd700 1 -- 172.21.15.162:0/1060809605 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef64004180 con 0x7fef640049c0 2020-11-28T21:55:59.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.194+0000 7fef76ffd700 1 -- 172.21.15.162:0/1060809605 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef64004340 con 0x7fef780b18c0 2020-11-28T21:55:59.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.194+0000 7fef7c9f1700 1 --2- 172.21.15.162:0/1060809605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef640049c0 0x7fef64004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:59.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.194+0000 7fef77fff700 1 --2- 172.21.15.162:0/1060809605 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fef780b18c0 0x7fef64003720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:59.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.194+0000 7fef777fe700 1 --2- 172.21.15.162:0/1060809605 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fef64003c10 0x7fef640044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:59.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.194+0000 7fef7c9f1700 1 -- 172.21.15.162:0/1060809605 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fef780b18c0 msgr2=0x7fef64003720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:59.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.194+0000 7fef7c9f1700 1 --2- 172.21.15.162:0/1060809605 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fef780b18c0 0x7fef64003720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:59.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.194+0000 7fef7c9f1700 1 -- 172.21.15.162:0/1060809605 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fef64003c10 msgr2=0x7fef640044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:59.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.194+0000 7fef7c9f1700 1 --2- 172.21.15.162:0/1060809605 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fef64003c10 0x7fef640044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:59.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.194+0000 7fef7c9f1700 1 -- 172.21.15.162:0/1060809605 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef64003230 con 0x7fef640049c0 2020-11-28T21:55:59.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.194+0000 7fef7c9f1700 1 --2- 172.21.15.162:0/1060809605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef640049c0 0x7fef64004e40 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 rx=0x7fef70001360 tx=0x7fef70007ec0).ready entity=mon.1 client_cookie=c3b0ea4c990c38d5 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:59.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.194+0000 7fef76ffd700 1 -- 172.21.15.162:0/1060809605 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef7000c9f0 con 0x7fef640049c0 2020-11-28T21:55:59.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.194+0000 7fef76ffd700 1 -- 172.21.15.162:0/1060809605 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef7000cb50 con 0x7fef640049c0 2020-11-28T21:55:59.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.194+0000 7fef76ffd700 1 -- 172.21.15.162:0/1060809605 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef70012400 con 0x7fef640049c0 2020-11-28T21:55:59.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.194+0000 7fef7d1f2700 1 -- 172.21.15.162:0/1060809605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef640049c0 msgr2=0x7fef64004e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:59.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.194+0000 7fef7d1f2700 1 --2- 172.21.15.162:0/1060809605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef640049c0 0x7fef64004e40 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 rx=0x7fef70001360 tx=0x7fef70007ec0).stop 2020-11-28T21:55:59.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.194+0000 7fef7c9f1700 1 -- 172.21.15.162:0/1060809605 reap_dead start 2020-11-28T21:55:59.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.194+0000 7fef7d1f2700 1 -- 172.21.15.162:0/1060809605 shutdown_connections 2020-11-28T21:55:59.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.194+0000 7fef7d1f2700 1 -- 172.21.15.162:0/1060809605 >> 172.21.15.162:0/1060809605 conn(0x7fef78062630 msgr2=0x7fef78062a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:59.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.194+0000 7fef7d1f2700 1 -- 172.21.15.162:0/1060809605 shutdown_connections 2020-11-28T21:55:59.208 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.194+0000 7fef7d1f2700 1 -- 172.21.15.162:0/1060809605 wait complete. 2020-11-28T21:55:59.213 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.194+0000 7fef7d1f2700 1 Processor -- start 2020-11-28T21:55:59.213 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.194+0000 7fef7d1f2700 1 -- start start 2020-11-28T21:55:59.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.194+0000 7fef7d1f2700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fef7814d9d0 0x7fef781464b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:59.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.194+0000 7fef7d1f2700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fef78159580 0x7fef78159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:59.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.194+0000 7fef7d1f2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef780ab260 0x7fef780ab710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:59.214 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.194+0000 7fef7d1f2700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef78159ef0 con 0x7fef7814d9d0 2020-11-28T21:55:59.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.194+0000 7fef7d1f2700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef78147660 con 0x7fef780ab260 2020-11-28T21:55:59.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.194+0000 7fef7d1f2700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef78147900 con 0x7fef78159580 2020-11-28T21:55:59.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.194+0000 7fef7c9f1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef780ab260 0x7fef780ab710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:59.215 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.194+0000 7fef7c9f1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef780ab260 0x7fef780ab710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45564/0 (socket says 172.21.15.162:45564) 2020-11-28T21:55:59.216 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.194+0000 7fef7c9f1700 1 -- 172.21.15.162:0/133041747 learned_addr learned my addr 172.21.15.162:0/133041747 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:59.216 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.198+0000 7fef777fe700 1 --2- 172.21.15.162:0/133041747 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fef78159580 0x7fef78159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:59.216 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.198+0000 7fef77fff700 1 --2- 172.21.15.162:0/133041747 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fef7814d9d0 0x7fef781464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:59.216 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.198+0000 7fef7c9f1700 1 -- 172.21.15.162:0/133041747 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fef78159580 msgr2=0x7fef78159a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:59.216 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.198+0000 7fef7c9f1700 1 --2- 172.21.15.162:0/133041747 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fef78159580 0x7fef78159a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:59.217 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.198+0000 7fef7c9f1700 1 -- 172.21.15.162:0/133041747 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fef7814d9d0 msgr2=0x7fef781464b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:59.217 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.198+0000 7fef7c9f1700 1 --2- 172.21.15.162:0/133041747 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fef7814d9d0 0x7fef781464b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:59.217 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.198+0000 7fef7c9f1700 1 -- 172.21.15.162:0/133041747 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef78147ba0 con 0x7fef780ab260 2020-11-28T21:55:59.217 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.198+0000 7fef7c9f1700 1 --2- 172.21.15.162:0/133041747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef780ab260 0x7fef780ab710 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 rx=0x7fef7805aba0 tx=0x7fef70017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:59.217 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.198+0000 7fef757fa700 1 -- 172.21.15.162:0/133041747 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef70001180 con 0x7fef780ab260 2020-11-28T21:55:59.218 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.198+0000 7fef757fa700 1 -- 172.21.15.162:0/133041747 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef70006a70 con 0x7fef780ab260 2020-11-28T21:55:59.218 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.198+0000 7fef7d1f2700 1 -- 172.21.15.162:0/133041747 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef781590e0 con 0x7fef780ab260 2020-11-28T21:55:59.218 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.198+0000 7fef757fa700 1 -- 172.21.15.162:0/133041747 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef70006dd0 con 0x7fef780ab260 2020-11-28T21:55:59.218 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.198+0000 7fef7d1f2700 1 -- 172.21.15.162:0/133041747 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef780ada40 con 0x7fef780ab260 2020-11-28T21:55:59.218 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.198+0000 7fef757fa700 1 -- 172.21.15.162:0/133041747 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fef70010070 con 0x7fef780ab260 2020-11-28T21:55:59.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.198+0000 7fef7d1f2700 1 -- 172.21.15.162:0/133041747 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef78146b40 con 0x7fef780ab260 2020-11-28T21:55:59.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.198+0000 7fef757fa700 1 --2- 172.21.15.162:0/133041747 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fef58055330 0x7fef580577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:59.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.198+0000 7fef757fa700 1 -- 172.21.15.162:0/133041747 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fef7006f210 con 0x7fef780ab260 2020-11-28T21:55:59.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.198+0000 7fef77fff700 1 --2- 172.21.15.162:0/133041747 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fef58055330 0x7fef580577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:59.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.198+0000 7fef77fff700 1 --2- 172.21.15.162:0/133041747 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fef58055330 0x7fef580577f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7fef68005000 tx=0x7fef68009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:59.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.206+0000 7fef757fa700 1 -- 172.21.15.162:0/133041747 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fef700457a0 con 0x7fef780ab260 2020-11-28T21:55:59.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.346+0000 7fef7d1f2700 1 -- 172.21.15.162:0/133041747 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fef78159270 con 0x7fef780ab260 2020-11-28T21:55:59.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.346+0000 7fef757fa700 1 -- 172.21.15.162:0/133041747 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v144) v1 ==== 92+0+10 (secure 0 0 0) 0x7fef70048dc0 con 0x7fef780ab260 2020-11-28T21:55:59.354 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:55:59.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.350+0000 7fef7d1f2700 1 -- 172.21.15.162:0/133041747 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fef58055330 msgr2=0x7fef580577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:59.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.350+0000 7fef7d1f2700 1 --2- 172.21.15.162:0/133041747 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fef58055330 0x7fef580577f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7fef68005000 tx=0x7fef68009040).stop 2020-11-28T21:55:59.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.350+0000 7fef7d1f2700 1 -- 172.21.15.162:0/133041747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef780ab260 msgr2=0x7fef780ab710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:59.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.350+0000 7fef7d1f2700 1 --2- 172.21.15.162:0/133041747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef780ab260 0x7fef780ab710 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 rx=0x7fef7805aba0 tx=0x7fef70017000).stop 2020-11-28T21:55:59.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.350+0000 7fef7d1f2700 1 -- 172.21.15.162:0/133041747 shutdown_connections 2020-11-28T21:55:59.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.350+0000 7fef7d1f2700 1 --2- 172.21.15.162:0/133041747 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fef58055330 0x7fef580577f0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:59.359 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.350+0000 7fef7d1f2700 1 --2- 172.21.15.162:0/133041747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef780ab260 0x7fef780ab710 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:59.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.350+0000 7fef7d1f2700 1 --2- 172.21.15.162:0/133041747 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fef7814d9d0 0x7fef781464b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:59.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.350+0000 7fef7d1f2700 1 --2- 172.21.15.162:0/133041747 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fef78159580 0x7fef78159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:59.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.350+0000 7fef7d1f2700 1 -- 172.21.15.162:0/133041747 >> 172.21.15.162:0/133041747 conn(0x7fef78062630 msgr2=0x7fef780b5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:59.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.350+0000 7fef7d1f2700 1 -- 172.21.15.162:0/133041747 shutdown_connections 2020-11-28T21:55:59.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.350+0000 7fef7d1f2700 1 -- 172.21.15.162:0/133041747 wait complete. 2020-11-28T21:55:59.367 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2020-11-28T21:55:59.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.474+0000 7fcf0bfff700 1 Processor -- start 2020-11-28T21:55:59.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.474+0000 7fcf0bfff700 1 -- start start 2020-11-28T21:55:59.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.474+0000 7fcf0bfff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcf0c098b30 0x7fcf0c0994f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:59.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.474+0000 7fcf0bfff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcf0c0b0000 0x7fcf0c060000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:59.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.474+0000 7fcf0bfff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf0c098f10 con 0x7fcf0c0604f0 2020-11-28T21:55:59.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.474+0000 7fcf0bfff700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf0c0b1030 con 0x7fcf0c098b30 2020-11-28T21:55:59.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.474+0000 7fcf0bfff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0c0b0480 con 0x7fcf0c0b0000 2020-11-28T21:55:59.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.474+0000 7fcf0affd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcf0c098b30 0x7fcf0c0994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:59.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.474+0000 7fcf0affd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcf0c098b30 0x7fcf0c0994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45574/0 (socket says 172.21.15.162:45574) 2020-11-28T21:55:59.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.474+0000 7fcf0affd700 1 -- 172.21.15.162:0/4181456290 learned_addr learned my addr 172.21.15.162:0/4181456290 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:59.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.474+0000 7fcf0a7fc700 1 --2- 172.21.15.162:0/4181456290 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcf0c0b0000 0x7fcf0c060000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:59.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.474+0000 7fcf0affd700 1 -- 172.21.15.162:0/4181456290 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcf0c0b0000 msgr2=0x7fcf0c060000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:59.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.474+0000 7fcf0affd700 1 --2- 172.21.15.162:0/4181456290 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcf0c0b0000 0x7fcf0c060000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:59.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.474+0000 7fcf0affd700 1 -- 172.21.15.162:0/4181456290 >> v1:172.21.15.118:6789/0 conn(0x7fcf0c0604f0 legacy=0x7fcf0c14a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:55:59.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.474+0000 7fcf0affd700 1 -- 172.21.15.162:0/4181456290 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf0c0b0680 con 0x7fcf0c098b30 2020-11-28T21:55:59.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.474+0000 7fcf0affd700 1 --2- 172.21.15.162:0/4181456290 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf0c098b30 0x7fcf0c0994f0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 rx=0x7fcf0c099be0 tx=0x7fcefc009270).ready entity=mon.1 client_cookie=776c5a30777aa643 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:59.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.474+0000 7fcf09ffb700 1 -- 172.21.15.162:0/4181456290 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcefc00bb80 con 0x7fcf0c098b30 2020-11-28T21:55:59.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.474+0000 7fcf09ffb700 1 -- 172.21.15.162:0/4181456290 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcefc00bce0 con 0x7fcf0c098b30 2020-11-28T21:55:59.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.474+0000 7fcf0bfff700 1 -- 172.21.15.162:0/4181456290 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf0c098b30 msgr2=0x7fcf0c0994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:59.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.474+0000 7fcf09ffb700 1 -- 172.21.15.162:0/4181456290 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcefc00be90 con 0x7fcf0c098b30 2020-11-28T21:55:59.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.474+0000 7fcf0bfff700 1 --2- 172.21.15.162:0/4181456290 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf0c098b30 0x7fcf0c0994f0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 rx=0x7fcf0c099be0 tx=0x7fcefc009270).stop 2020-11-28T21:55:59.486 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.474+0000 7fcf0bfff700 1 -- 172.21.15.162:0/4181456290 shutdown_connections 2020-11-28T21:55:59.486 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.474+0000 7fcf0bfff700 1 --2- 172.21.15.162:0/4181456290 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf0c098b30 0x7fcf0c0994f0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:59.487 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.474+0000 7fcf0bfff700 1 --2- 172.21.15.162:0/4181456290 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcf0c0b0000 0x7fcf0c060000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:59.487 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.474+0000 7fcf0bfff700 1 -- 172.21.15.162:0/4181456290 >> 172.21.15.162:0/4181456290 conn(0x7fcf0c05b970 msgr2=0x7fcf0c05bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:59.487 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.474+0000 7fcf0bfff700 1 -- 172.21.15.162:0/4181456290 shutdown_connections 2020-11-28T21:55:59.487 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.474+0000 7fcf0bfff700 1 -- 172.21.15.162:0/4181456290 wait complete. 2020-11-28T21:55:59.487 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.478+0000 7fcf0bfff700 1 Processor -- start 2020-11-28T21:55:59.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.478+0000 7fcf0bfff700 1 -- start start 2020-11-28T21:55:59.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.478+0000 7fcf0bfff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcf0c0604f0 0x7fcf0c0aacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:59.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.478+0000 7fcf0bfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf0c098b30 0x7fcf0c063340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:59.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.478+0000 7fcf0bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf0c0b0000 0x7fcf0c063830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:59.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.478+0000 7fcf0bfff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0c063d20 con 0x7fcf0c0b0000 2020-11-28T21:55:59.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.478+0000 7fcf0bfff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0c0ade10 con 0x7fcf0c098b30 2020-11-28T21:55:59.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.478+0000 7fcf0bfff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0c0ae0b0 con 0x7fcf0c0604f0 2020-11-28T21:55:59.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.478+0000 7fcf0a7fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf0c098b30 0x7fcf0c063340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:59.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.478+0000 7fcf0a7fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf0c098b30 0x7fcf0c063340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45578/0 (socket says 172.21.15.162:45578) 2020-11-28T21:55:59.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.478+0000 7fcf0a7fc700 1 -- 172.21.15.162:0/2410762349 learned_addr learned my addr 172.21.15.162:0/2410762349 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:59.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.478+0000 7fcf0a7fc700 1 -- 172.21.15.162:0/2410762349 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcf0c0604f0 msgr2=0x7fcf0c0aacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:59.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.478+0000 7fcf0a7fc700 1 --2- 172.21.15.162:0/2410762349 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcf0c0604f0 0x7fcf0c0aacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-28T21:55:59.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.478+0000 7fcf0a7fc700 1 -- 172.21.15.162:0/2410762349 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf0c0b0000 msgr2=0x7fcf0c063830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:59.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.478+0000 7fcf0a7fc700 1 --2- 172.21.15.162:0/2410762349 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf0c0b0000 0x7fcf0c063830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-28T21:55:59.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.478+0000 7fcf0a7fc700 1 -- 172.21.15.162:0/2410762349 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf0c0ae350 con 0x7fcf0c098b30 2020-11-28T21:55:59.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.478+0000 7fcf0a7fc700 1 --2- 172.21.15.162:0/2410762349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf0c098b30 0x7fcf0c063340 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 rx=0x7fcf0c0add30 tx=0x7fcef4008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:59.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.478+0000 7fcf03fff700 1 -- 172.21.15.162:0/2410762349 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcef4009980 con 0x7fcf0c098b30 2020-11-28T21:55:59.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.478+0000 7fcf03fff700 1 -- 172.21.15.162:0/2410762349 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef4009ae0 con 0x7fcf0c098b30 2020-11-28T21:55:59.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.478+0000 7fcf0bfff700 1 -- 172.21.15.162:0/2410762349 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf0c0ad490 con 0x7fcf0c098b30 2020-11-28T21:55:59.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.478+0000 7fcf03fff700 1 -- 172.21.15.162:0/2410762349 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcef4009e40 con 0x7fcf0c098b30 2020-11-28T21:55:59.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.478+0000 7fcf0bfff700 1 -- 172.21.15.162:0/2410762349 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf0c0ad930 con 0x7fcf0c098b30 2020-11-28T21:55:59.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.478+0000 7fcf0bfff700 1 -- 172.21.15.162:0/2410762349 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf0c057050 con 0x7fcf0c098b30 2020-11-28T21:55:59.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.478+0000 7fcf03fff700 1 -- 172.21.15.162:0/2410762349 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fcef4013480 con 0x7fcf0c098b30 2020-11-28T21:55:59.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.482+0000 7fcf03fff700 1 --2- 172.21.15.162:0/2410762349 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcef00553b0 0x7fcef0057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:59.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.482+0000 7fcf03fff700 1 -- 172.21.15.162:0/2410762349 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fcef406e3a0 con 0x7fcf0c098b30 2020-11-28T21:55:59.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.482+0000 7fcf0affd700 1 --2- 172.21.15.162:0/2410762349 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcef00553b0 0x7fcef0057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:59.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.482+0000 7fcf0affd700 1 --2- 172.21.15.162:0/2410762349 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcef00553b0 0x7fcef0057870 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7fcf0c099be0 tx=0x7fcefc016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:59.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.486+0000 7fcf03fff700 1 -- 172.21.15.162:0/2410762349 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fcef4044970 con 0x7fcf0c098b30 2020-11-28T21:55:59.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.626+0000 7fcf0bfff700 1 -- 172.21.15.162:0/2410762349 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fcf0c068350 con 0x7fcf0c098b30 2020-11-28T21:55:59.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.626+0000 7fcf03fff700 1 -- 172.21.15.162:0/2410762349 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v144) v1 ==== 84+0+143 (secure 0 0 0) 0x7fcef4047f90 con 0x7fcf0c098b30 2020-11-28T21:55:59.632 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:55:59.632 INFO:teuthology.orchestra.run.smithi162.stdout:[ 2020-11-28T21:55:59.632 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:59.633 INFO:teuthology.orchestra.run.smithi162.stdout: "poolnum": 1, 2020-11-28T21:55:59.633 INFO:teuthology.orchestra.run.smithi162.stdout: "poolname": "device_health_metrics" 2020-11-28T21:55:59.633 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:55:59.633 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:55:59.633 INFO:teuthology.orchestra.run.smithi162.stdout: "poolnum": 2, 2020-11-28T21:55:59.634 INFO:teuthology.orchestra.run.smithi162.stdout: "poolname": "rbd" 2020-11-28T21:55:59.634 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:55:59.634 INFO:teuthology.orchestra.run.smithi162.stdout:] 2020-11-28T21:55:59.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.626+0000 7fcf0bfff700 1 -- 172.21.15.162:0/2410762349 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcef00553b0 msgr2=0x7fcef0057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:59.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.626+0000 7fcf0bfff700 1 --2- 172.21.15.162:0/2410762349 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcef00553b0 0x7fcef0057870 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7fcf0c099be0 tx=0x7fcefc016000).stop 2020-11-28T21:55:59.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.626+0000 7fcf0bfff700 1 -- 172.21.15.162:0/2410762349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf0c098b30 msgr2=0x7fcf0c063340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:59.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.626+0000 7fcf0bfff700 1 --2- 172.21.15.162:0/2410762349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf0c098b30 0x7fcf0c063340 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 rx=0x7fcf0c0add30 tx=0x7fcef4008040).stop 2020-11-28T21:55:59.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.630+0000 7fcf0bfff700 1 -- 172.21.15.162:0/2410762349 shutdown_connections 2020-11-28T21:55:59.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.630+0000 7fcf0bfff700 1 --2- 172.21.15.162:0/2410762349 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fcef00553b0 0x7fcef0057870 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:59.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.630+0000 7fcf0bfff700 1 --2- 172.21.15.162:0/2410762349 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcf0c0604f0 0x7fcf0c0aacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-28T21:55:59.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.630+0000 7fcf0bfff700 1 --2- 172.21.15.162:0/2410762349 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf0c0b0000 0x7fcf0c063830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-28T21:55:59.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.630+0000 7fcf0bfff700 1 --2- 172.21.15.162:0/2410762349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf0c098b30 0x7fcf0c063340 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:59.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.630+0000 7fcf0bfff700 1 -- 172.21.15.162:0/2410762349 >> 172.21.15.162:0/2410762349 conn(0x7fcf0c05b970 msgr2=0x7fcf0c0a8c30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:59.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.630+0000 7fcf0bfff700 1 -- 172.21.15.162:0/2410762349 shutdown_connections 2020-11-28T21:55:59.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.630+0000 7fcf0bfff700 1 -- 172.21.15.162:0/2410762349 wait complete. 2020-11-28T21:55:59.645 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-11-28T21:55:59.646 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2020-11-28T21:55:59.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.754+0000 7f30bd334700 1 Processor -- start 2020-11-28T21:55:59.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.754+0000 7f30bd334700 1 -- start start 2020-11-28T21:55:59.761 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.754+0000 7f30bd334700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f30b813cf30 0x7f30b80ac350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:59.761 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.754+0000 7f30bd334700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30b80b1360 con 0x7f30b813cb50 2020-11-28T21:55:59.761 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.754+0000 7f30bd334700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30b80b25c0 con 0x7f30b805cca0 2020-11-28T21:55:59.761 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.754+0000 7f30bd334700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30b80add60 con 0x7f30b813cf30 2020-11-28T21:55:59.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.754+0000 7f30bcb33700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f30b813cf30 0x7f30b80ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:59.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.754+0000 7f30b77fe700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f30b805cca0 0x7f30b80b1eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:52432/0 (socket says 172.21.15.162:52432) 2020-11-28T21:55:59.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.754+0000 7f30b77fe700 1 -- 172.21.15.162:0/286252135 learned_addr learned my addr 172.21.15.162:0/286252135 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:55:59.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.754+0000 7f30bcb33700 1 -- 172.21.15.162:0/286252135 >> v1:172.21.15.162:6789/0 conn(0x7f30b805cca0 legacy=0x7f30b80b1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:59.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.754+0000 7f30b6ffd700 1 -- 172.21.15.162:0/286252135 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 899137158 0 0) 0x7f30b80b25c0 con 0x7f30b805cca0 2020-11-28T21:55:59.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.754+0000 7f30bcb33700 1 -- 172.21.15.162:0/286252135 >> v1:172.21.15.118:6789/0 conn(0x7f30b813cb50 legacy=0x7f30b8140180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:59.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.754+0000 7f30bcb33700 1 -- 172.21.15.162:0/286252135 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b80ae8c0 con 0x7f30b813cf30 2020-11-28T21:55:59.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.754+0000 7f30bcb33700 1 --2- 172.21.15.162:0/286252135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30b813cf30 0x7f30b80ac350 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 rx=0x7f30a0002f30 tx=0x7f30a0006dc0).ready entity=mon.1 client_cookie=329136f9ed1c896f server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:59.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.754+0000 7f30b6ffd700 1 -- 172.21.15.162:0/286252135 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30a00097f0 con 0x7f30b813cf30 2020-11-28T21:55:59.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.754+0000 7f30b6ffd700 1 -- 172.21.15.162:0/286252135 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30a0009950 con 0x7f30b813cf30 2020-11-28T21:55:59.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.754+0000 7f30b6ffd700 1 -- 172.21.15.162:0/286252135 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30a0009cb0 con 0x7f30b813cf30 2020-11-28T21:55:59.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.754+0000 7f30bd334700 1 -- 172.21.15.162:0/286252135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30b813cf30 msgr2=0x7f30b80ac350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:59.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.754+0000 7f30bd334700 1 --2- 172.21.15.162:0/286252135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30b813cf30 0x7f30b80ac350 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 rx=0x7f30a0002f30 tx=0x7f30a0006dc0).stop 2020-11-28T21:55:59.767 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.758+0000 7f30bd334700 1 -- 172.21.15.162:0/286252135 shutdown_connections 2020-11-28T21:55:59.767 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.758+0000 7f30bd334700 1 --2- 172.21.15.162:0/286252135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30b813cf30 0x7f30b80ac350 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:59.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.758+0000 7f30bd334700 1 -- 172.21.15.162:0/286252135 >> 172.21.15.162:0/286252135 conn(0x7f30b80b57d0 msgr2=0x7f30b80b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:55:59.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.758+0000 7f30bd334700 1 -- 172.21.15.162:0/286252135 shutdown_connections 2020-11-28T21:55:59.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.758+0000 7f30bd334700 1 -- 172.21.15.162:0/286252135 wait complete. 2020-11-28T21:55:59.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.758+0000 7f30bd334700 1 Processor -- start 2020-11-28T21:55:59.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.758+0000 7f30bd334700 1 -- start start 2020-11-28T21:55:59.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.758+0000 7f30bd334700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f30b805cca0 0x7f30b8142370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:59.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.758+0000 7f30bd334700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30b813cb50 0x7f30b813c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:59.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.758+0000 7f30bd334700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f30b813cf30 0x7f30b813c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:59.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.758+0000 7f30bd334700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b8151af0 con 0x7f30b813cf30 2020-11-28T21:55:59.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.758+0000 7f30bd334700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b81474e0 con 0x7f30b813cb50 2020-11-28T21:55:59.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.758+0000 7f30bd334700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30b8142fa0 con 0x7f30b805cca0 2020-11-28T21:55:59.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.758+0000 7f30b77fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30b813cb50 0x7f30b813c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:59.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.758+0000 7f30b77fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30b813cb50 0x7f30b813c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45592/0 (socket says 172.21.15.162:45592) 2020-11-28T21:55:59.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.758+0000 7f30b77fe700 1 -- 172.21.15.162:0/251720821 learned_addr learned my addr 172.21.15.162:0/251720821 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:55:59.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.758+0000 7f30bcb33700 1 --2- 172.21.15.162:0/251720821 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f30b813cf30 0x7f30b813c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:59.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.758+0000 7f30b7fff700 1 --2- 172.21.15.162:0/251720821 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f30b805cca0 0x7f30b8142370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:59.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.758+0000 7f30b77fe700 1 -- 172.21.15.162:0/251720821 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f30b805cca0 msgr2=0x7f30b8142370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:59.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.758+0000 7f30b77fe700 1 --2- 172.21.15.162:0/251720821 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f30b805cca0 0x7f30b8142370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:59.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.758+0000 7f30b77fe700 1 -- 172.21.15.162:0/251720821 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f30b813cf30 msgr2=0x7f30b813c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:55:59.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.758+0000 7f30b77fe700 1 --2- 172.21.15.162:0/251720821 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f30b813cf30 0x7f30b813c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:55:59.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.758+0000 7f30b77fe700 1 -- 172.21.15.162:0/251720821 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b8147730 con 0x7f30b813cb50 2020-11-28T21:55:59.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.758+0000 7f30b77fe700 1 --2- 172.21.15.162:0/251720821 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30b813cb50 0x7f30b813c070 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 rx=0x7f30a8001210 tx=0x7f30a8007e10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:59.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.758+0000 7f30b57fa700 1 -- 172.21.15.162:0/251720821 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30a8005b70 con 0x7f30b813cb50 2020-11-28T21:55:59.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.758+0000 7f30b57fa700 1 -- 172.21.15.162:0/251720821 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30a8005cd0 con 0x7f30b813cb50 2020-11-28T21:55:59.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.758+0000 7f30bd334700 1 -- 172.21.15.162:0/251720821 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30b81479d0 con 0x7f30b813cb50 2020-11-28T21:55:59.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.758+0000 7f30b57fa700 1 -- 172.21.15.162:0/251720821 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30a8007300 con 0x7f30b813cb50 2020-11-28T21:55:59.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.758+0000 7f30bd334700 1 -- 172.21.15.162:0/251720821 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30b81471c0 con 0x7f30b813cb50 2020-11-28T21:55:59.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.762+0000 7f30bd334700 1 -- 172.21.15.162:0/251720821 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30b8057050 con 0x7f30b813cb50 2020-11-28T21:55:59.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.762+0000 7f30b57fa700 1 -- 172.21.15.162:0/251720821 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f30a8010030 con 0x7f30b813cb50 2020-11-28T21:55:59.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.762+0000 7f30b57fa700 1 --2- 172.21.15.162:0/251720821 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f309c0554e0 0x7f309c0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:55:59.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.762+0000 7f30b57fa700 1 -- 172.21.15.162:0/251720821 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f30a806dc40 con 0x7f30b813cb50 2020-11-28T21:55:59.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.762+0000 7f30b7fff700 1 --2- 172.21.15.162:0/251720821 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f309c0554e0 0x7f309c0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:55:59.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.762+0000 7f30b7fff700 1 --2- 172.21.15.162:0/251720821 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f309c0554e0 0x7f309c0579a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f30b000b000 tx=0x7f30b0009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:55:59.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.770+0000 7f30b57fa700 1 -- 172.21.15.162:0/251720821 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f30a8044270 con 0x7f30b813cb50 2020-11-28T21:55:59.915 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:55:59.910+0000 7f30bd334700 1 -- 172.21.15.162:0/251720821 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f30b813f710 con 0x7f30b813cb50 2020-11-28T21:56:00.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.022+0000 7f30b57fa700 1 -- 172.21.15.162:0/251720821 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v145) v1 ==== 133+0+0 (secure 0 0 0) 0x7f30a8047890 con 0x7f30b813cb50 2020-11-28T21:56:00.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.026+0000 7f30bd334700 1 -- 172.21.15.162:0/251720821 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f309c0554e0 msgr2=0x7f309c0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:00.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.026+0000 7f30bd334700 1 --2- 172.21.15.162:0/251720821 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f309c0554e0 0x7f309c0579a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f30b000b000 tx=0x7f30b0009040).stop 2020-11-28T21:56:00.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.026+0000 7f30bd334700 1 -- 172.21.15.162:0/251720821 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30b813cb50 msgr2=0x7f30b813c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:00.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.026+0000 7f30bd334700 1 --2- 172.21.15.162:0/251720821 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30b813cb50 0x7f30b813c070 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 rx=0x7f30a8001210 tx=0x7f30a8007e10).stop 2020-11-28T21:56:00.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.026+0000 7f30bd334700 1 -- 172.21.15.162:0/251720821 shutdown_connections 2020-11-28T21:56:00.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.026+0000 7f30bd334700 1 --2- 172.21.15.162:0/251720821 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f309c0554e0 0x7f309c0579a0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:00.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.026+0000 7f30bd334700 1 --2- 172.21.15.162:0/251720821 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f30b805cca0 0x7f30b8142370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:00.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.026+0000 7f30bd334700 1 --2- 172.21.15.162:0/251720821 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f30b813cf30 0x7f30b813c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:00.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.026+0000 7f30bd334700 1 --2- 172.21.15.162:0/251720821 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30b813cb50 0x7f30b813c070 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:00.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.026+0000 7f30bd334700 1 -- 172.21.15.162:0/251720821 >> 172.21.15.162:0/251720821 conn(0x7f30b80b57d0 msgr2=0x7f30b80aac20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:00.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.026+0000 7f30bd334700 1 -- 172.21.15.162:0/251720821 shutdown_connections 2020-11-28T21:56:00.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.026+0000 7f30bd334700 1 -- 172.21.15.162:0/251720821 wait complete. 2020-11-28T21:56:00.034 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_metadata' created 2020-11-28T21:56:00.045 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2020-11-28T21:56:00.161 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.154+0000 7f51c3c87700 1 Processor -- start 2020-11-28T21:56:00.162 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.158+0000 7f51c3c87700 1 -- start start 2020-11-28T21:56:00.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.158+0000 7f51c3c87700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f51bc0b9ce0 0x7f51bc1433c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:00.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.158+0000 7f51c3c87700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f51bc067610 0x7f51bc13fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:00.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.158+0000 7f51c3c87700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51bc143950 con 0x7f51bc0605c0 2020-11-28T21:56:00.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.158+0000 7f51c3c87700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51bc0b4340 con 0x7f51bc0b9ce0 2020-11-28T21:56:00.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.158+0000 7f51c3c87700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51bc0ba0c0 con 0x7f51bc067610 2020-11-28T21:56:00.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.158+0000 7f51c2224700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f51bc067610 0x7f51bc13fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:00.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.158+0000 7f51c2224700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f51bc067610 0x7f51bc13fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45602/0 (socket says 172.21.15.162:45602) 2020-11-28T21:56:00.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.158+0000 7f51c2224700 1 -- 172.21.15.162:0/679502329 learned_addr learned my addr 172.21.15.162:0/679502329 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:00.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.158+0000 7f51c1a23700 1 --2- 172.21.15.162:0/679502329 >> v2:172.21.15.118:3300/0 conn(0x7f51bc0b9ce0 0x7f51bc1433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:00.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.158+0000 7f51c2224700 1 -- 172.21.15.162:0/679502329 >> v2:172.21.15.118:3300/0 conn(0x7f51bc0b9ce0 msgr2=0x7f51bc1433c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:00.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.158+0000 7f51c2224700 1 --2- 172.21.15.162:0/679502329 >> v2:172.21.15.118:3300/0 conn(0x7f51bc0b9ce0 0x7f51bc1433c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:00.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.158+0000 7f51c2224700 1 -- 172.21.15.162:0/679502329 >> v1:172.21.15.118:6789/0 conn(0x7f51bc0605c0 legacy=0x7f51bc0b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:00.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.158+0000 7f51c2224700 1 -- 172.21.15.162:0/679502329 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51bc067aa0 con 0x7f51bc067610 2020-11-28T21:56:00.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.158+0000 7f51c1a23700 1 --2- 172.21.15.162:0/679502329 >> v2:172.21.15.118:3300/0 conn(0x7f51bc0b9ce0 0x7f51bc1433c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:56:00.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.158+0000 7f51c2224700 1 --2- 172.21.15.162:0/679502329 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51bc067610 0x7f51bc13fc80 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 rx=0x7f51b40032b0 tx=0x7f51b4009270).ready entity=mon.1 client_cookie=7803bd6a1b03c1fd server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:00.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.158+0000 7f51c0a21700 1 -- 172.21.15.162:0/679502329 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51b400bc00 con 0x7f51bc067610 2020-11-28T21:56:00.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.158+0000 7f51c0a21700 1 -- 172.21.15.162:0/679502329 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b400bd60 con 0x7f51bc067610 2020-11-28T21:56:00.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.158+0000 7f51c0a21700 1 -- 172.21.15.162:0/679502329 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51b4011400 con 0x7f51bc067610 2020-11-28T21:56:00.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.158+0000 7f51c3c87700 1 -- 172.21.15.162:0/679502329 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51bc067610 msgr2=0x7f51bc13fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:00.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.158+0000 7f51c3c87700 1 --2- 172.21.15.162:0/679502329 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51bc067610 0x7f51bc13fc80 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 rx=0x7f51b40032b0 tx=0x7f51b4009270).stop 2020-11-28T21:56:00.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.158+0000 7f51c3c87700 1 -- 172.21.15.162:0/679502329 shutdown_connections 2020-11-28T21:56:00.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.158+0000 7f51c3c87700 1 --2- 172.21.15.162:0/679502329 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51bc067610 0x7f51bc13fc80 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:00.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.158+0000 7f51c3c87700 1 --2- 172.21.15.162:0/679502329 >> v2:172.21.15.118:3300/0 conn(0x7f51bc0b9ce0 0x7f51bc1433c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:00.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.158+0000 7f51c3c87700 1 -- 172.21.15.162:0/679502329 >> 172.21.15.162:0/679502329 conn(0x7f51bc05c430 msgr2=0x7f51bc0b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:00.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.158+0000 7f51c3c87700 1 -- 172.21.15.162:0/679502329 shutdown_connections 2020-11-28T21:56:00.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.158+0000 7f51c3c87700 1 -- 172.21.15.162:0/679502329 wait complete. 2020-11-28T21:56:00.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.162+0000 7f51c3c87700 1 Processor -- start 2020-11-28T21:56:00.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.162+0000 7f51c3c87700 1 -- start start 2020-11-28T21:56:00.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.162+0000 7f51c3c87700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51bc0605c0 0x7f51bc09e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:00.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.162+0000 7f51c3c87700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f51bc067610 0x7f51bc153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:00.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.162+0000 7f51c3c87700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f51bc0b9ce0 0x7f51bc1540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:00.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.162+0000 7f51c3c87700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc152c80 con 0x7f51bc0b9ce0 2020-11-28T21:56:00.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.162+0000 7f51c3c87700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc152050 con 0x7f51bc0605c0 2020-11-28T21:56:00.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.162+0000 7f51c3c87700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc153200 con 0x7f51bc067610 2020-11-28T21:56:00.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.162+0000 7f51c1a23700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51bc0605c0 0x7f51bc09e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:00.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.162+0000 7f51c1a23700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51bc0605c0 0x7f51bc09e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45604/0 (socket says 172.21.15.162:45604) 2020-11-28T21:56:00.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.162+0000 7f51c1a23700 1 -- 172.21.15.162:0/3373547450 learned_addr learned my addr 172.21.15.162:0/3373547450 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:00.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.162+0000 7f51c2224700 1 --2- 172.21.15.162:0/3373547450 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f51bc0b9ce0 0x7f51bc1540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:00.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.162+0000 7f51c1a23700 1 -- 172.21.15.162:0/3373547450 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f51bc067610 msgr2=0x7f51bc153be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:00.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.162+0000 7f51c1222700 1 --2- 172.21.15.162:0/3373547450 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f51bc067610 0x7f51bc153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:00.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.162+0000 7f51c1a23700 1 --2- 172.21.15.162:0/3373547450 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f51bc067610 0x7f51bc153be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:00.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.162+0000 7f51c1a23700 1 -- 172.21.15.162:0/3373547450 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f51bc0b9ce0 msgr2=0x7f51bc1540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:00.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.162+0000 7f51c1a23700 1 --2- 172.21.15.162:0/3373547450 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f51bc0b9ce0 0x7f51bc1540d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:00.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.162+0000 7f51c1a23700 1 -- 172.21.15.162:0/3373547450 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51bc1534a0 con 0x7f51bc0605c0 2020-11-28T21:56:00.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.162+0000 7f51c2224700 1 --2- 172.21.15.162:0/3373547450 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f51bc0b9ce0 0x7f51bc1540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:56:00.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.162+0000 7f51c1222700 1 --2- 172.21.15.162:0/3373547450 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f51bc067610 0x7f51bc153be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:56:00.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.162+0000 7f51c1a23700 1 --2- 172.21.15.162:0/3373547450 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51bc0605c0 0x7f51bc09e170 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 rx=0x7f51b80068b0 tx=0x7f51b8008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:00.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.162+0000 7f51aaffd700 1 -- 172.21.15.162:0/3373547450 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51b80127e0 con 0x7f51bc0605c0 2020-11-28T21:56:00.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.162+0000 7f51aaffd700 1 -- 172.21.15.162:0/3373547450 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b8012940 con 0x7f51bc0605c0 2020-11-28T21:56:00.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.162+0000 7f51c3c87700 1 -- 172.21.15.162:0/3373547450 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51bc153740 con 0x7f51bc0605c0 2020-11-28T21:56:00.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.162+0000 7f51aaffd700 1 -- 172.21.15.162:0/3373547450 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51b8012ca0 con 0x7f51bc0605c0 2020-11-28T21:56:00.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.162+0000 7f51c3c87700 1 -- 172.21.15.162:0/3373547450 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51bc09f020 con 0x7f51bc0605c0 2020-11-28T21:56:00.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.166+0000 7f51c3c87700 1 -- 172.21.15.162:0/3373547450 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51bc057050 con 0x7f51bc0605c0 2020-11-28T21:56:00.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.166+0000 7f51aaffd700 1 -- 172.21.15.162:0/3373547450 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f51b8013480 con 0x7f51bc0605c0 2020-11-28T21:56:00.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.166+0000 7f51aaffd700 1 --2- 172.21.15.162:0/3373547450 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f519c0554e0 0x7f519c0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:00.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.166+0000 7f51aaffd700 1 -- 172.21.15.162:0/3373547450 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f51b806e600 con 0x7f51bc0605c0 2020-11-28T21:56:00.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.166+0000 7f51c1222700 1 --2- 172.21.15.162:0/3373547450 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f519c0554e0 0x7f519c0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:00.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.166+0000 7f51c1222700 1 --2- 172.21.15.162:0/3373547450 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f519c0554e0 0x7f519c0579a0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7f51ac008000 tx=0x7f51ac009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:00.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.174+0000 7f51aaffd700 1 -- 172.21.15.162:0/3373547450 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f51b80449d0 con 0x7f51bc0605c0 2020-11-28T21:56:00.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:00.314+0000 7f51c3c87700 1 -- 172.21.15.162:0/3373547450 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f51bc0b9780 con 0x7f51bc0605c0 2020-11-28T21:56:01.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.026+0000 7f51aaffd700 1 -- 172.21.15.162:0/3373547450 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v146) v1 ==== 125+0+0 (secure 0 0 0) 0x7f51b8012e00 con 0x7f51bc0605c0 2020-11-28T21:56:01.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.034+0000 7f51c3c87700 1 -- 172.21.15.162:0/3373547450 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f519c0554e0 msgr2=0x7f519c0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:01.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.034+0000 7f51c3c87700 1 --2- 172.21.15.162:0/3373547450 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f519c0554e0 0x7f519c0579a0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7f51ac008000 tx=0x7f51ac009040).stop 2020-11-28T21:56:01.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.034+0000 7f51c3c87700 1 -- 172.21.15.162:0/3373547450 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51bc0605c0 msgr2=0x7f51bc09e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:01.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.034+0000 7f51c3c87700 1 --2- 172.21.15.162:0/3373547450 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51bc0605c0 0x7f51bc09e170 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 rx=0x7f51b80068b0 tx=0x7f51b8008040).stop 2020-11-28T21:56:01.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.034+0000 7f51c3c87700 1 -- 172.21.15.162:0/3373547450 shutdown_connections 2020-11-28T21:56:01.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.034+0000 7f51c3c87700 1 --2- 172.21.15.162:0/3373547450 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f519c0554e0 0x7f519c0579a0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:01.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.034+0000 7f51c3c87700 1 --2- 172.21.15.162:0/3373547450 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f51bc0b9ce0 0x7f51bc1540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:01.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.034+0000 7f51c3c87700 1 --2- 172.21.15.162:0/3373547450 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51bc0605c0 0x7f51bc09e170 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:01.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.034+0000 7f51c3c87700 1 --2- 172.21.15.162:0/3373547450 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f51bc067610 0x7f51bc153be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:01.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.034+0000 7f51c3c87700 1 -- 172.21.15.162:0/3373547450 >> 172.21.15.162:0/3373547450 conn(0x7f51bc05c430 msgr2=0x7f51bc141a20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:01.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.034+0000 7f51c3c87700 1 -- 172.21.15.162:0/3373547450 shutdown_connections 2020-11-28T21:56:01.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.034+0000 7f51c3c87700 1 -- 172.21.15.162:0/3373547450 wait complete. 2020-11-28T21:56:01.041 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_data' created 2020-11-28T21:56:01.055 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2020-11-28T21:56:01.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.158+0000 7fa04c91d700 1 Processor -- start 2020-11-28T21:56:01.164 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.158+0000 7fa04c91d700 1 -- start start 2020-11-28T21:56:01.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.158+0000 7fa04c91d700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa0480649c0 0x7fa048143580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:01.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.158+0000 7fa04c91d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fa048060200 0x7fa04805b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:01.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.158+0000 7fa04c91d700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa048064da0 con 0x7fa04805be60 2020-11-28T21:56:01.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.158+0000 7fa04c91d700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0481429b0 con 0x7fa048060200 2020-11-28T21:56:01.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.158+0000 7fa04c91d700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa048142ba0 con 0x7fa0480649c0 2020-11-28T21:56:01.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.158+0000 7fa04659c700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa0480649c0 0x7fa048143580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:01.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.158+0000 7fa04659c700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa0480649c0 0x7fa048143580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:42836/0 (socket says 172.21.15.162:42836) 2020-11-28T21:56:01.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.158+0000 7fa04659c700 1 -- 172.21.15.162:0/3455955076 learned_addr learned my addr 172.21.15.162:0/3455955076 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:01.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.158+0000 7fa045d9b700 1 --2- 172.21.15.162:0/3455955076 >> v2:172.21.15.118:3300/0 conn(0x7fa048060200 0x7fa04805b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:01.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.162+0000 7fa04559a700 1 -- 172.21.15.162:0/3455955076 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1917265065 0 0) 0x7fa048064da0 con 0x7fa04805be60 2020-11-28T21:56:01.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.162+0000 7fa04559a700 1 -- 172.21.15.162:0/3455955076 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa02c003230 con 0x7fa04805be60 2020-11-28T21:56:01.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.162+0000 7fa04659c700 1 -- 172.21.15.162:0/3455955076 >> v2:172.21.15.118:3300/0 conn(0x7fa048060200 msgr2=0x7fa04805b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:01.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.162+0000 7fa04659c700 1 --2- 172.21.15.162:0/3455955076 >> v2:172.21.15.118:3300/0 conn(0x7fa048060200 0x7fa04805b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:01.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.162+0000 7fa04659c700 1 -- 172.21.15.162:0/3455955076 >> v1:172.21.15.118:6789/0 conn(0x7fa04805be60 legacy=0x7fa04813fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:01.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.162+0000 7fa04659c700 1 -- 172.21.15.162:0/3455955076 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa048142e40 con 0x7fa0480649c0 2020-11-28T21:56:01.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.162+0000 7fa04659c700 1 --2- 172.21.15.162:0/3455955076 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa0480649c0 0x7fa048143580 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7fa038002f30 tx=0x7fa038006e80).ready entity=mon.2 client_cookie=e4c0df93d22cd7ea server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:01.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.162+0000 7fa04559a700 1 -- 172.21.15.162:0/3455955076 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa038009640 con 0x7fa0480649c0 2020-11-28T21:56:01.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.162+0000 7fa04559a700 1 -- 172.21.15.162:0/3455955076 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0380097a0 con 0x7fa0480649c0 2020-11-28T21:56:01.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.162+0000 7fa04559a700 1 -- 172.21.15.162:0/3455955076 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa038009b00 con 0x7fa0480649c0 2020-11-28T21:56:01.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.162+0000 7fa04c91d700 1 -- 172.21.15.162:0/3455955076 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa0480649c0 msgr2=0x7fa048143580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:01.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.162+0000 7fa04c91d700 1 --2- 172.21.15.162:0/3455955076 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa0480649c0 0x7fa048143580 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7fa038002f30 tx=0x7fa038006e80).stop 2020-11-28T21:56:01.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.162+0000 7fa04c91d700 1 -- 172.21.15.162:0/3455955076 shutdown_connections 2020-11-28T21:56:01.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.162+0000 7fa04c91d700 1 --2- 172.21.15.162:0/3455955076 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa0480649c0 0x7fa048143580 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:01.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.162+0000 7fa04c91d700 1 --2- 172.21.15.162:0/3455955076 >> v2:172.21.15.118:3300/0 conn(0x7fa048060200 0x7fa04805b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:01.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.162+0000 7fa04c91d700 1 -- 172.21.15.162:0/3455955076 >> 172.21.15.162:0/3455955076 conn(0x7fa048061740 msgr2=0x7fa048061b50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:01.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.162+0000 7fa04c91d700 1 -- 172.21.15.162:0/3455955076 shutdown_connections 2020-11-28T21:56:01.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.162+0000 7fa04c91d700 1 -- 172.21.15.162:0/3455955076 wait complete. 2020-11-28T21:56:01.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.162+0000 7fa04c91d700 1 Processor -- start 2020-11-28T21:56:01.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.162+0000 7fa04c91d700 1 -- start start 2020-11-28T21:56:01.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.162+0000 7fa04c91d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa04805be60 0x7fa0480adda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:01.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.162+0000 7fa04c91d700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa048060200 0x7fa048145630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:01.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.162+0000 7fa04c91d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa0480649c0 0x7fa048145b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:01.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.162+0000 7fa04c91d700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa048146010 con 0x7fa0480649c0 2020-11-28T21:56:01.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.162+0000 7fa04c91d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0481522c0 con 0x7fa04805be60 2020-11-28T21:56:01.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.162+0000 7fa04c91d700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa048144bb0 con 0x7fa048060200 2020-11-28T21:56:01.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.162+0000 7fa04659c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa04805be60 0x7fa0480adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:01.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.162+0000 7fa04659c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa04805be60 0x7fa0480adda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45618/0 (socket says 172.21.15.162:45618) 2020-11-28T21:56:01.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.162+0000 7fa04659c700 1 -- 172.21.15.162:0/2996918335 learned_addr learned my addr 172.21.15.162:0/2996918335 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:01.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.162+0000 7fa045d9b700 1 --2- 172.21.15.162:0/2996918335 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa048060200 0x7fa048145630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:01.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.162+0000 7fa046d9d700 1 --2- 172.21.15.162:0/2996918335 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa0480649c0 0x7fa048145b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:01.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.162+0000 7fa04659c700 1 -- 172.21.15.162:0/2996918335 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa048060200 msgr2=0x7fa048145630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:01.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.162+0000 7fa04659c700 1 --2- 172.21.15.162:0/2996918335 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa048060200 0x7fa048145630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:01.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.162+0000 7fa04659c700 1 -- 172.21.15.162:0/2996918335 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa0480649c0 msgr2=0x7fa048145b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:01.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.162+0000 7fa04659c700 1 --2- 172.21.15.162:0/2996918335 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa0480649c0 0x7fa048145b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:01.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.162+0000 7fa04659c700 1 -- 172.21.15.162:0/2996918335 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa048151fc0 con 0x7fa04805be60 2020-11-28T21:56:01.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.166+0000 7fa04659c700 1 --2- 172.21.15.162:0/2996918335 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa04805be60 0x7fa0480adda0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 rx=0x7fa048146150 tx=0x7fa038015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:01.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.166+0000 7fa0377fe700 1 -- 172.21.15.162:0/2996918335 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa038011ba0 con 0x7fa04805be60 2020-11-28T21:56:01.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.166+0000 7fa04c91d700 1 -- 172.21.15.162:0/2996918335 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa048152560 con 0x7fa04805be60 2020-11-28T21:56:01.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.166+0000 7fa0377fe700 1 -- 172.21.15.162:0/2996918335 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa038011d00 con 0x7fa04805be60 2020-11-28T21:56:01.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.166+0000 7fa0377fe700 1 -- 172.21.15.162:0/2996918335 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa03801d640 con 0x7fa04805be60 2020-11-28T21:56:01.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.166+0000 7fa04c91d700 1 -- 172.21.15.162:0/2996918335 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa048143f40 con 0x7fa04805be60 2020-11-28T21:56:01.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.166+0000 7fa04c91d700 1 -- 172.21.15.162:0/2996918335 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa048144d70 con 0x7fa04805be60 2020-11-28T21:56:01.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.166+0000 7fa0377fe700 1 -- 172.21.15.162:0/2996918335 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa03800f070 con 0x7fa04805be60 2020-11-28T21:56:01.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.166+0000 7fa0377fe700 1 --2- 172.21.15.162:0/2996918335 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa024055440 0x7fa024057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:01.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.166+0000 7fa0377fe700 1 -- 172.21.15.162:0/2996918335 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5957+0+0 (secure 0 0 0) 0x7fa03806fa30 con 0x7fa04805be60 2020-11-28T21:56:01.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.166+0000 7fa045d9b700 1 --2- 172.21.15.162:0/2996918335 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa024055440 0x7fa024057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:01.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.166+0000 7fa045d9b700 1 --2- 172.21.15.162:0/2996918335 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa024055440 0x7fa024057900 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7fa03c00b000 tx=0x7fa03c006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:01.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.170+0000 7fa0377fe700 1 -- 172.21.15.162:0/2996918335 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa038045d40 con 0x7fa04805be60 2020-11-28T21:56:01.360 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.354+0000 7fa04c91d700 1 -- 172.21.15.162:0/2996918335 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fa048057050 con 0x7fa04805be60 2020-11-28T21:56:01.373 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.366+0000 7fa0377fe700 1 -- 172.21.15.162:0/2996918335 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 27 and data pool 28 v72) v1 ==== 189+0+0 (secure 0 0 0) 0x7fa038049320 con 0x7fa04805be60 2020-11-28T21:56:01.376 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.370+0000 7fa04c91d700 1 -- 172.21.15.162:0/2996918335 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa024055440 msgr2=0x7fa024057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:01.376 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.370+0000 7fa04c91d700 1 --2- 172.21.15.162:0/2996918335 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa024055440 0x7fa024057900 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7fa03c00b000 tx=0x7fa03c006dc0).stop 2020-11-28T21:56:01.376 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.370+0000 7fa04c91d700 1 -- 172.21.15.162:0/2996918335 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa04805be60 msgr2=0x7fa0480adda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:01.376 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.370+0000 7fa04c91d700 1 --2- 172.21.15.162:0/2996918335 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa04805be60 0x7fa0480adda0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 rx=0x7fa048146150 tx=0x7fa038015000).stop 2020-11-28T21:56:01.379 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.374+0000 7fa04c91d700 1 -- 172.21.15.162:0/2996918335 shutdown_connections 2020-11-28T21:56:01.380 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.374+0000 7fa04c91d700 1 --2- 172.21.15.162:0/2996918335 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa024055440 0x7fa024057900 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:01.380 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.374+0000 7fa04c91d700 1 --2- 172.21.15.162:0/2996918335 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa0480649c0 0x7fa048145b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:01.380 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.374+0000 7fa04c91d700 1 --2- 172.21.15.162:0/2996918335 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa04805be60 0x7fa0480adda0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:01.380 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.374+0000 7fa04c91d700 1 --2- 172.21.15.162:0/2996918335 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa048060200 0x7fa048145630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:01.380 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.374+0000 7fa04c91d700 1 -- 172.21.15.162:0/2996918335 >> 172.21.15.162:0/2996918335 conn(0x7fa048061740 msgr2=0x7fa04813e610 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:01.381 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.374+0000 7fa04c91d700 1 -- 172.21.15.162:0/2996918335 shutdown_connections 2020-11-28T21:56:01.381 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.374+0000 7fa04c91d700 1 -- 172.21.15.162:0/2996918335 wait complete. 2020-11-28T21:56:01.381 INFO:teuthology.orchestra.run.smithi162.stderr:new fs with metadata pool 27 and data pool 28 2020-11-28T21:56:01.394 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:56:01.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.486+0000 7f5ba759e700 1 Processor -- start 2020-11-28T21:56:01.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.486+0000 7f5ba759e700 1 -- start start 2020-11-28T21:56:01.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.486+0000 7f5ba759e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5ba8140090 0x7f5ba814e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:01.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.486+0000 7f5ba759e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ba80b7d00 con 0x7f5ba80b7920 2020-11-28T21:56:01.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.486+0000 7f5ba759e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ba8143300 con 0x7f5ba80b2da0 2020-11-28T21:56:01.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.486+0000 7f5ba759e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba8143560 con 0x7f5ba8140090 2020-11-28T21:56:01.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.486+0000 7f5ba5d9b700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f5ba80b2da0 0x7f5ba80b1db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:52480/0 (socket says 172.21.15.162:52480) 2020-11-28T21:56:01.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.486+0000 7f5ba5d9b700 1 -- 172.21.15.162:0/1915301995 learned_addr learned my addr 172.21.15.162:0/1915301995 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:56:01.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.486+0000 7f5ba6d9d700 1 --2- 172.21.15.162:0/1915301995 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5ba8140090 0x7f5ba814e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:01.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.486+0000 7f5ba559a700 1 -- 172.21.15.162:0/1915301995 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 124801246 0 0) 0x7f5ba8143300 con 0x7f5ba80b2da0 2020-11-28T21:56:01.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.486+0000 7f5ba559a700 1 -- 172.21.15.162:0/1915301995 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b8c003230 con 0x7f5ba80b2da0 2020-11-28T21:56:01.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.486+0000 7f5ba559a700 1 -- 172.21.15.162:0/1915301995 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3581769213 0 0) 0x7f5ba80b7d00 con 0x7f5ba80b7920 2020-11-28T21:56:01.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.486+0000 7f5ba559a700 1 -- 172.21.15.162:0/1915301995 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ba8143300 con 0x7f5ba80b7920 2020-11-28T21:56:01.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.486+0000 7f5ba559a700 1 -- 172.21.15.162:0/1915301995 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 875729230 0 0) 0x7f5b8c003230 con 0x7f5ba80b2da0 2020-11-28T21:56:01.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.486+0000 7f5ba559a700 1 -- 172.21.15.162:0/1915301995 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5ba8140090 msgr2=0x7f5ba814e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:01.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.486+0000 7f5ba559a700 1 --2- 172.21.15.162:0/1915301995 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5ba8140090 0x7f5ba814e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:01.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.486+0000 7f5ba559a700 1 -- 172.21.15.162:0/1915301995 >> v1:172.21.15.118:6789/0 conn(0x7f5ba80b7920 legacy=0x7f5ba813f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:01.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.486+0000 7f5ba559a700 1 -- 172.21.15.162:0/1915301995 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba80b1910 con 0x7f5ba80b2da0 2020-11-28T21:56:01.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.486+0000 7f5ba559a700 1 -- 172.21.15.162:0/1915301995 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f5b9c001a70 con 0x7f5ba80b2da0 2020-11-28T21:56:01.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.486+0000 7f5ba559a700 1 -- 172.21.15.162:0/1915301995 >> v1:172.21.15.162:6789/0 conn(0x7f5ba80b2da0 legacy=0x7f5ba80b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:01.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.486+0000 7f5ba559a700 1 --2- 172.21.15.162:0/1915301995 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5ba8140090 0x7f5b8c0038b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:01.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.486+0000 7f5ba559a700 1 --2- 172.21.15.162:0/1915301995 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b8c003da0 0x7f5b8c0044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:01.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.486+0000 7f5ba559a700 1 --2- 172.21.15.162:0/1915301995 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b8c0049c0 0x7f5b8c004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:01.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.486+0000 7f5ba559a700 1 -- 172.21.15.162:0/1915301995 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba8143560 con 0x7f5b8c0049c0 2020-11-28T21:56:01.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.486+0000 7f5ba559a700 1 -- 172.21.15.162:0/1915301995 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8c005330 con 0x7f5b8c003da0 2020-11-28T21:56:01.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.486+0000 7f5ba559a700 1 -- 172.21.15.162:0/1915301995 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8c00b750 con 0x7f5ba8140090 2020-11-28T21:56:01.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.486+0000 7f5ba5d9b700 1 --2- 172.21.15.162:0/1915301995 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b8c003da0 0x7f5b8c0044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:01.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.486+0000 7f5ba659c700 1 --2- 172.21.15.162:0/1915301995 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5ba8140090 0x7f5b8c0038b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:01.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.486+0000 7f5ba6d9d700 1 --2- 172.21.15.162:0/1915301995 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b8c0049c0 0x7f5b8c004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:01.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.490+0000 7f5ba5d9b700 1 -- 172.21.15.162:0/1915301995 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5ba8140090 msgr2=0x7f5b8c0038b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:01.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.490+0000 7f5ba5d9b700 1 --2- 172.21.15.162:0/1915301995 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5ba8140090 0x7f5b8c0038b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:01.501 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.490+0000 7f5ba5d9b700 1 -- 172.21.15.162:0/1915301995 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b8c0049c0 msgr2=0x7f5b8c004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:01.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.490+0000 7f5ba5d9b700 1 --2- 172.21.15.162:0/1915301995 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b8c0049c0 0x7f5b8c004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:01.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.490+0000 7f5ba5d9b700 1 -- 172.21.15.162:0/1915301995 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b8c003230 con 0x7f5b8c003da0 2020-11-28T21:56:01.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.490+0000 7f5ba5d9b700 1 --2- 172.21.15.162:0/1915301995 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b8c003da0 0x7f5b8c0044d0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 rx=0x7f5b9c001840 tx=0x7f5ba80644d0).ready entity=mon.1 client_cookie=5e991bf7b1936552 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:01.502 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.490+0000 7f5ba559a700 1 -- 172.21.15.162:0/1915301995 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b9c005c40 con 0x7f5b8c003da0 2020-11-28T21:56:01.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.490+0000 7f5ba559a700 1 -- 172.21.15.162:0/1915301995 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b9c005da0 con 0x7f5b8c003da0 2020-11-28T21:56:01.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.490+0000 7f5ba559a700 1 -- 172.21.15.162:0/1915301995 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b9c0072c0 con 0x7f5b8c003da0 2020-11-28T21:56:01.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.490+0000 7f5ba759e700 1 -- 172.21.15.162:0/1915301995 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b8c003da0 msgr2=0x7f5b8c0044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:01.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.490+0000 7f5ba759e700 1 --2- 172.21.15.162:0/1915301995 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b8c003da0 0x7f5b8c0044d0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 rx=0x7f5b9c001840 tx=0x7f5ba80644d0).stop 2020-11-28T21:56:01.503 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.490+0000 7f5ba6d9d700 1 -- 172.21.15.162:0/1915301995 reap_dead start 2020-11-28T21:56:01.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.490+0000 7f5ba759e700 1 -- 172.21.15.162:0/1915301995 shutdown_connections 2020-11-28T21:56:01.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.490+0000 7f5ba759e700 1 -- 172.21.15.162:0/1915301995 >> 172.21.15.162:0/1915301995 conn(0x7f5ba805c440 msgr2=0x7f5ba80b6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:01.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.490+0000 7f5ba759e700 1 -- 172.21.15.162:0/1915301995 shutdown_connections 2020-11-28T21:56:01.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.490+0000 7f5ba759e700 1 -- 172.21.15.162:0/1915301995 wait complete. 2020-11-28T21:56:01.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.490+0000 7f5ba759e700 1 Processor -- start 2020-11-28T21:56:01.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.490+0000 7f5ba759e700 1 -- start start 2020-11-28T21:56:01.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.490+0000 7f5ba759e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5ba8149020 0x7f5ba80b0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:01.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.490+0000 7f5ba759e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5ba80afe80 0x7f5ba8143c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:01.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.490+0000 7f5ba759e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ba813ec30 0x7f5ba813f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:01.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.490+0000 7f5ba759e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba813f5a0 con 0x7f5ba8149020 2020-11-28T21:56:01.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.490+0000 7f5ba759e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba8149400 con 0x7f5ba813ec30 2020-11-28T21:56:01.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.490+0000 7f5ba759e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba80b0e40 con 0x7f5ba80afe80 2020-11-28T21:56:01.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.490+0000 7f5ba6d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ba813ec30 0x7f5ba813f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:01.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.490+0000 7f5ba6d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ba813ec30 0x7f5ba813f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45650/0 (socket says 172.21.15.162:45650) 2020-11-28T21:56:01.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.490+0000 7f5ba6d9d700 1 -- 172.21.15.162:0/3696047799 learned_addr learned my addr 172.21.15.162:0/3696047799 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:01.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.490+0000 7f5ba5d9b700 1 --2- 172.21.15.162:0/3696047799 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5ba80afe80 0x7f5ba8143c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:01.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.490+0000 7f5ba659c700 1 --2- 172.21.15.162:0/3696047799 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5ba8149020 0x7f5ba80b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:01.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.494+0000 7f5ba6d9d700 1 -- 172.21.15.162:0/3696047799 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5ba80afe80 msgr2=0x7f5ba8143c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:01.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.494+0000 7f5ba6d9d700 1 --2- 172.21.15.162:0/3696047799 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5ba80afe80 0x7f5ba8143c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:01.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.494+0000 7f5ba6d9d700 1 -- 172.21.15.162:0/3696047799 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5ba8149020 msgr2=0x7f5ba80b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:01.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.494+0000 7f5ba6d9d700 1 --2- 172.21.15.162:0/3696047799 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5ba8149020 0x7f5ba80b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:01.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.494+0000 7f5ba6d9d700 1 -- 172.21.15.162:0/3696047799 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba8144a10 con 0x7f5ba813ec30 2020-11-28T21:56:01.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.494+0000 7f5ba6d9d700 1 --2- 172.21.15.162:0/3696047799 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ba813ec30 0x7f5ba813f0b0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 rx=0x7f5b98008000 tx=0x7f5b98008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:01.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.494+0000 7f5b977fe700 1 -- 172.21.15.162:0/3696047799 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b9800f8b0 con 0x7f5ba813ec30 2020-11-28T21:56:01.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.494+0000 7f5b977fe700 1 -- 172.21.15.162:0/3696047799 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b9800fa10 con 0x7f5ba813ec30 2020-11-28T21:56:01.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.494+0000 7f5b977fe700 1 -- 172.21.15.162:0/3696047799 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b9800fd70 con 0x7f5ba813ec30 2020-11-28T21:56:01.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.494+0000 7f5ba759e700 1 -- 172.21.15.162:0/3696047799 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ba8144770 con 0x7f5ba813ec30 2020-11-28T21:56:01.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.494+0000 7f5ba759e700 1 -- 172.21.15.162:0/3696047799 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ba80b0960 con 0x7f5ba813ec30 2020-11-28T21:56:01.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.494+0000 7f5ba759e700 1 -- 172.21.15.162:0/3696047799 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ba80b12a0 con 0x7f5ba813ec30 2020-11-28T21:56:01.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.494+0000 7f5b977fe700 1 -- 172.21.15.162:0/3696047799 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5b9801e460 con 0x7f5ba813ec30 2020-11-28T21:56:01.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.494+0000 7f5b977fe700 1 --2- 172.21.15.162:0/3696047799 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5b84055330 0x7f5b840577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:01.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.494+0000 7f5b977fe700 1 -- 172.21.15.162:0/3696047799 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5b9806e740 con 0x7f5ba813ec30 2020-11-28T21:56:01.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.494+0000 7f5ba659c700 1 --2- 172.21.15.162:0/3696047799 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5b84055330 0x7f5b840577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:01.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.498+0000 7f5ba659c700 1 --2- 172.21.15.162:0/3696047799 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5b84055330 0x7f5b840577f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7f5ba0003000 tx=0x7f5ba0009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:01.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.502+0000 7f5b977fe700 1 -- 172.21.15.162:0/3696047799 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5b98044850 con 0x7f5ba813ec30 2020-11-28T21:56:01.648 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.642+0000 7f5ba759e700 1 -- 172.21.15.162:0/3696047799 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ba8057050 con 0x7f5ba813ec30 2020-11-28T21:56:01.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.646+0000 7f5b977fe700 1 -- 172.21.15.162:0/3696047799 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+16148 (secure 0 0 0) 0x7f5b98047e70 con 0x7f5ba813ec30 2020-11-28T21:56:01.651 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:56:01.651 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":148,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:56:01.362358+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:55:59.967971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2020-11-28T21:56:00.379272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:56:01.652 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.646+0000 7f5ba759e700 1 -- 172.21.15.162:0/3696047799 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5b84055330 msgr2=0x7f5b840577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:01.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.646+0000 7f5ba759e700 1 --2- 172.21.15.162:0/3696047799 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5b84055330 0x7f5b840577f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7f5ba0003000 tx=0x7f5ba0009040).stop 2020-11-28T21:56:01.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.646+0000 7f5ba759e700 1 -- 172.21.15.162:0/3696047799 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ba813ec30 msgr2=0x7f5ba813f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:01.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.646+0000 7f5ba759e700 1 --2- 172.21.15.162:0/3696047799 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ba813ec30 0x7f5ba813f0b0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 rx=0x7f5b98008000 tx=0x7f5b98008040).stop 2020-11-28T21:56:01.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.646+0000 7f5ba759e700 1 -- 172.21.15.162:0/3696047799 shutdown_connections 2020-11-28T21:56:01.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.646+0000 7f5ba759e700 1 --2- 172.21.15.162:0/3696047799 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5b84055330 0x7f5b840577f0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:01.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.646+0000 7f5ba759e700 1 --2- 172.21.15.162:0/3696047799 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ba813ec30 0x7f5ba813f0b0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:01.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.646+0000 7f5ba759e700 1 --2- 172.21.15.162:0/3696047799 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5ba8149020 0x7f5ba80b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:01.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.646+0000 7f5ba759e700 1 --2- 172.21.15.162:0/3696047799 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5ba80afe80 0x7f5ba8143c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:01.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.646+0000 7f5ba759e700 1 -- 172.21.15.162:0/3696047799 >> 172.21.15.162:0/3696047799 conn(0x7f5ba805c440 msgr2=0x7f5ba80b5240 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:01.655 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.646+0000 7f5ba759e700 1 -- 172.21.15.162:0/3696047799 shutdown_connections 2020-11-28T21:56:01.655 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.646+0000 7f5ba759e700 1 -- 172.21.15.162:0/3696047799 wait complete. 2020-11-28T21:56:01.665 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:56:01.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.766+0000 7fe70d048700 1 Processor -- start 2020-11-28T21:56:01.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.766+0000 7fe70d048700 1 -- start start 2020-11-28T21:56:01.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.766+0000 7fe70d048700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe70805f380 0x7fe708147000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:01.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.766+0000 7fe70d048700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe70805c7a0 con 0x7fe7080b0200 2020-11-28T21:56:01.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.766+0000 7fe70d048700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe70814a500 con 0x7fe70805c3c0 2020-11-28T21:56:01.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.766+0000 7fe70d048700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7080b1090 con 0x7fe70805f380 2020-11-28T21:56:01.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.766+0000 7fe70659c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe70805f380 0x7fe708147000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:01.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.766+0000 7fe70659c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe70805f380 0x7fe708147000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45654/0 (socket says 172.21.15.162:45654) 2020-11-28T21:56:01.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.766+0000 7fe706d9d700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fe70805c3c0 0x7fe7080a8560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:52498/0 (socket says 172.21.15.162:52498) 2020-11-28T21:56:01.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.766+0000 7fe70659c700 1 -- 172.21.15.162:0/2422361401 learned_addr learned my addr 172.21.15.162:0/2422361401 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:01.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.766+0000 7fe70659c700 1 -- 172.21.15.162:0/2422361401 >> v1:172.21.15.162:6789/0 conn(0x7fe70805c3c0 legacy=0x7fe7080a8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:01.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.770+0000 7fe70659c700 1 -- 172.21.15.162:0/2422361401 >> v1:172.21.15.118:6789/0 conn(0x7fe7080b0200 legacy=0x7fe7081474f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:01.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.770+0000 7fe70659c700 1 -- 172.21.15.162:0/2422361401 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7080b1330 con 0x7fe70805f380 2020-11-28T21:56:01.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.770+0000 7fe70659c700 1 --2- 172.21.15.162:0/2422361401 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe70805f380 0x7fe708147000 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 rx=0x7fe708061e20 tx=0x7fe6f8006e80).ready entity=mon.1 client_cookie=4f8af35e98b50d01 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:01.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.770+0000 7fe705d9b700 1 -- 172.21.15.162:0/2422361401 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6f80097b0 con 0x7fe70805f380 2020-11-28T21:56:01.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.770+0000 7fe705d9b700 1 -- 172.21.15.162:0/2422361401 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f8009910 con 0x7fe70805f380 2020-11-28T21:56:01.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.770+0000 7fe70d048700 1 -- 172.21.15.162:0/2422361401 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe70805f380 msgr2=0x7fe708147000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:01.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.770+0000 7fe70d048700 1 --2- 172.21.15.162:0/2422361401 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe70805f380 0x7fe708147000 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 rx=0x7fe708061e20 tx=0x7fe6f8006e80).stop 2020-11-28T21:56:01.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.770+0000 7fe705d9b700 1 -- 172.21.15.162:0/2422361401 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6f8009a80 con 0x7fe70805f380 2020-11-28T21:56:01.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.770+0000 7fe70d048700 1 -- 172.21.15.162:0/2422361401 shutdown_connections 2020-11-28T21:56:01.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.770+0000 7fe70d048700 1 --2- 172.21.15.162:0/2422361401 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe70805f380 0x7fe708147000 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:01.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.770+0000 7fe70d048700 1 -- 172.21.15.162:0/2422361401 >> 172.21.15.162:0/2422361401 conn(0x7fe708061150 msgr2=0x7fe70805b970 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:01.780 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.770+0000 7fe70d048700 1 -- 172.21.15.162:0/2422361401 shutdown_connections 2020-11-28T21:56:01.780 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.770+0000 7fe70d048700 1 -- 172.21.15.162:0/2422361401 wait complete. 2020-11-28T21:56:01.780 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.770+0000 7fe70d048700 1 Processor -- start 2020-11-28T21:56:01.780 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.770+0000 7fe70d048700 1 -- start start 2020-11-28T21:56:01.780 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.770+0000 7fe70d048700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe70805c3c0 0x7fe7080b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:01.781 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.770+0000 7fe70d048700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe70805f380 0x7fe7080aa250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:01.781 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.770+0000 7fe70d048700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe7080b0200 0x7fe7080aa740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:01.781 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.770+0000 7fe70d048700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7080a8600 con 0x7fe7080b0200 2020-11-28T21:56:01.781 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.770+0000 7fe70d048700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7080a88a0 con 0x7fe70805c3c0 2020-11-28T21:56:01.781 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.770+0000 7fe70d048700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7080a8b40 con 0x7fe70805f380 2020-11-28T21:56:01.782 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.770+0000 7fe706d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe70805c3c0 0x7fe7080b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:01.782 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.770+0000 7fe706d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe70805c3c0 0x7fe7080b4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45660/0 (socket says 172.21.15.162:45660) 2020-11-28T21:56:01.782 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.770+0000 7fe706d9d700 1 -- 172.21.15.162:0/637679329 learned_addr learned my addr 172.21.15.162:0/637679329 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:01.782 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.774+0000 7fe70759e700 1 --2- 172.21.15.162:0/637679329 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe7080b0200 0x7fe7080aa740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:01.782 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.774+0000 7fe706d9d700 1 -- 172.21.15.162:0/637679329 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe70805f380 msgr2=0x7fe7080aa250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:01.783 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.774+0000 7fe706d9d700 1 --2- 172.21.15.162:0/637679329 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe70805f380 0x7fe7080aa250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-28T21:56:01.783 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.774+0000 7fe706d9d700 1 -- 172.21.15.162:0/637679329 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe7080b0200 msgr2=0x7fe7080aa740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:01.783 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.774+0000 7fe706d9d700 1 --2- 172.21.15.162:0/637679329 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe7080b0200 0x7fe7080aa740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:01.783 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.774+0000 7fe706d9d700 1 -- 172.21.15.162:0/637679329 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7080a8de0 con 0x7fe70805c3c0 2020-11-28T21:56:01.783 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.774+0000 7fe70759e700 1 --2- 172.21.15.162:0/637679329 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe7080b0200 0x7fe7080aa740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:56:01.784 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.774+0000 7fe706d9d700 1 --2- 172.21.15.162:0/637679329 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe70805c3c0 0x7fe7080b4f70 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 rx=0x7fe7080b2ac0 tx=0x7fe6fc007c10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:01.784 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.774+0000 7fe6f7fff700 1 -- 172.21.15.162:0/637679329 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6fc007320 con 0x7fe70805c3c0 2020-11-28T21:56:01.784 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.774+0000 7fe70d048700 1 -- 172.21.15.162:0/637679329 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7080b5680 con 0x7fe70805c3c0 2020-11-28T21:56:01.784 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.774+0000 7fe6f7fff700 1 -- 172.21.15.162:0/637679329 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6fc005520 con 0x7fe70805c3c0 2020-11-28T21:56:01.785 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.774+0000 7fe6f7fff700 1 -- 172.21.15.162:0/637679329 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6fc005880 con 0x7fe70805c3c0 2020-11-28T21:56:01.785 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.774+0000 7fe70d048700 1 -- 172.21.15.162:0/637679329 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7080ae760 con 0x7fe70805c3c0 2020-11-28T21:56:01.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.774+0000 7fe70d048700 1 -- 172.21.15.162:0/637679329 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7080a8f70 con 0x7fe70805c3c0 2020-11-28T21:56:01.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.774+0000 7fe6f7fff700 1 -- 172.21.15.162:0/637679329 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe6fc010030 con 0x7fe70805c3c0 2020-11-28T21:56:01.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.774+0000 7fe6f7fff700 1 --2- 172.21.15.162:0/637679329 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe6e4055330 0x7fe6e40577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:01.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.774+0000 7fe6f7fff700 1 -- 172.21.15.162:0/637679329 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe6fc06dd40 con 0x7fe70805c3c0 2020-11-28T21:56:01.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.774+0000 7fe70659c700 1 --2- 172.21.15.162:0/637679329 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe6e4055330 0x7fe6e40577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:01.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.774+0000 7fe70659c700 1 --2- 172.21.15.162:0/637679329 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe6e4055330 0x7fe6e40577f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7fe708061e20 tx=0x7fe6f8016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:01.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.786+0000 7fe6f7fff700 1 -- 172.21.15.162:0/637679329 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe6fc043eb0 con 0x7fe70805c3c0 2020-11-28T21:56:01.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.930+0000 7fe70d048700 1 -- 172.21.15.162:0/637679329 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe708057050 con 0x7fe70805c3c0 2020-11-28T21:56:01.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.930+0000 7fe6f7fff700 1 -- 172.21.15.162:0/637679329 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+16148 (secure 0 0 0) 0x7fe6fc0474d0 con 0x7fe70805c3c0 2020-11-28T21:56:01.936 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:56:01.937 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":148,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:56:01.362358+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:55:59.967971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2020-11-28T21:56:00.379272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:56:01.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.934+0000 7fe70d048700 1 -- 172.21.15.162:0/637679329 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe6e4055330 msgr2=0x7fe6e40577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:01.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.934+0000 7fe70d048700 1 --2- 172.21.15.162:0/637679329 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe6e4055330 0x7fe6e40577f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7fe708061e20 tx=0x7fe6f8016000).stop 2020-11-28T21:56:01.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.934+0000 7fe70d048700 1 -- 172.21.15.162:0/637679329 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe70805c3c0 msgr2=0x7fe7080b4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:01.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.934+0000 7fe70d048700 1 --2- 172.21.15.162:0/637679329 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe70805c3c0 0x7fe7080b4f70 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 rx=0x7fe7080b2ac0 tx=0x7fe6fc007c10).stop 2020-11-28T21:56:01.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.934+0000 7fe70d048700 1 -- 172.21.15.162:0/637679329 shutdown_connections 2020-11-28T21:56:01.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.934+0000 7fe70d048700 1 --2- 172.21.15.162:0/637679329 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe6e4055330 0x7fe6e40577f0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:01.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.934+0000 7fe70d048700 1 --2- 172.21.15.162:0/637679329 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe7080b0200 0x7fe7080aa740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:01.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.934+0000 7fe70d048700 1 --2- 172.21.15.162:0/637679329 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe70805c3c0 0x7fe7080b4f70 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:01.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.934+0000 7fe70d048700 1 --2- 172.21.15.162:0/637679329 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe70805f380 0x7fe7080aa250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-28T21:56:01.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.934+0000 7fe70d048700 1 -- 172.21.15.162:0/637679329 >> 172.21.15.162:0/637679329 conn(0x7fe708061150 msgr2=0x7fe70814a730 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:01.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.934+0000 7fe70d048700 1 -- 172.21.15.162:0/637679329 shutdown_connections 2020-11-28T21:56:01.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:01.934+0000 7fe70d048700 1 -- 172.21.15.162:0/637679329 wait complete. 2020-11-28T21:56:01.957 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2020-11-28T21:56:02.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.070+0000 7fbcf332a700 1 Processor -- start 2020-11-28T21:56:02.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.070+0000 7fbcf332a700 1 -- start start 2020-11-28T21:56:02.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.070+0000 7fbcf332a700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbcec0a9d40 0x7fbcec141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:02.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.070+0000 7fbcf332a700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcec0a8f50 con 0x7fbcec0a9960 2020-11-28T21:56:02.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.070+0000 7fbcf332a700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcec0981d0 con 0x7fbcec141820 2020-11-28T21:56:02.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.070+0000 7fbcf332a700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcec0aa1a0 con 0x7fbcec0a9d40 2020-11-28T21:56:02.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.070+0000 7fbcf1b27700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbcec0a9d40 0x7fbcec141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:02.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.070+0000 7fbcf2b29700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fbcec141820 0x7fbcec14b1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:52516/0 (socket says 172.21.15.162:52516) 2020-11-28T21:56:02.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.070+0000 7fbcf2b29700 1 -- 172.21.15.162:0/2517340490 learned_addr learned my addr 172.21.15.162:0/2517340490 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:56:02.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.070+0000 7fbcf1b27700 1 -- 172.21.15.162:0/2517340490 >> v1:172.21.15.162:6789/0 conn(0x7fbcec141820 legacy=0x7fbcec14b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:02.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.070+0000 7fbcf1b27700 1 -- 172.21.15.162:0/2517340490 >> v1:172.21.15.118:6789/0 conn(0x7fbcec0a9960 legacy=0x7fbcec097ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:02.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.070+0000 7fbcf1326700 1 -- 172.21.15.162:0/2517340490 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1068034918 0 0) 0x7fbcec0981d0 con 0x7fbcec141820 2020-11-28T21:56:02.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.070+0000 7fbcf1b27700 1 -- 172.21.15.162:0/2517340490 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcec0b0b30 con 0x7fbcec0a9d40 2020-11-28T21:56:02.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.070+0000 7fbcf1b27700 1 --2- 172.21.15.162:0/2517340490 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcec0a9d40 0x7fbcec141330 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 rx=0x7fbce80010d0 tx=0x7fbcdc006dc0).ready entity=mon.1 client_cookie=81cc8869aeefbbae server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:02.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.070+0000 7fbcf1326700 1 -- 172.21.15.162:0/2517340490 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbcdc0096b0 con 0x7fbcec0a9d40 2020-11-28T21:56:02.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.070+0000 7fbcf1326700 1 -- 172.21.15.162:0/2517340490 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcdc009810 con 0x7fbcec0a9d40 2020-11-28T21:56:02.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.070+0000 7fbcf1326700 1 -- 172.21.15.162:0/2517340490 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbcdc009b70 con 0x7fbcec0a9d40 2020-11-28T21:56:02.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.070+0000 7fbcf332a700 1 -- 172.21.15.162:0/2517340490 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcec0a9d40 msgr2=0x7fbcec141330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:02.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.070+0000 7fbcf332a700 1 --2- 172.21.15.162:0/2517340490 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcec0a9d40 0x7fbcec141330 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 rx=0x7fbce80010d0 tx=0x7fbcdc006dc0).stop 2020-11-28T21:56:02.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.074+0000 7fbcf332a700 1 -- 172.21.15.162:0/2517340490 shutdown_connections 2020-11-28T21:56:02.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.074+0000 7fbcf332a700 1 --2- 172.21.15.162:0/2517340490 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcec0a9d40 0x7fbcec141330 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:02.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.074+0000 7fbcf332a700 1 -- 172.21.15.162:0/2517340490 >> 172.21.15.162:0/2517340490 conn(0x7fbcec0b5520 msgr2=0x7fbcec0b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:02.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.074+0000 7fbcf332a700 1 -- 172.21.15.162:0/2517340490 shutdown_connections 2020-11-28T21:56:02.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.074+0000 7fbcf332a700 1 -- 172.21.15.162:0/2517340490 wait complete. 2020-11-28T21:56:02.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.074+0000 7fbcf332a700 1 Processor -- start 2020-11-28T21:56:02.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.074+0000 7fbcf332a700 1 -- start start 2020-11-28T21:56:02.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.078+0000 7fbcf332a700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbcec0a9960 0x7fbcec144220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:02.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.078+0000 7fbcf332a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbcec0a9d40 0x7fbcec140160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:02.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.078+0000 7fbcf332a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcec141820 0x7fbcec140650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:02.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.078+0000 7fbcf332a700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec140b40 con 0x7fbcec0a9d40 2020-11-28T21:56:02.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.078+0000 7fbcf332a700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec0abfa0 con 0x7fbcec141820 2020-11-28T21:56:02.085 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.078+0000 7fbcf332a700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec0ac240 con 0x7fbcec0a9960 2020-11-28T21:56:02.086 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.078+0000 7fbcf2b29700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcec141820 0x7fbcec140650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:02.086 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.078+0000 7fbcf2b29700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcec141820 0x7fbcec140650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45678/0 (socket says 172.21.15.162:45678) 2020-11-28T21:56:02.086 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.078+0000 7fbcf2b29700 1 -- 172.21.15.162:0/3146062683 learned_addr learned my addr 172.21.15.162:0/3146062683 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:02.086 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.078+0000 7fbcf1b27700 1 --2- 172.21.15.162:0/3146062683 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbcec0a9d40 0x7fbcec140160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:02.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.078+0000 7fbcf2328700 1 --2- 172.21.15.162:0/3146062683 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbcec0a9960 0x7fbcec144220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:02.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.078+0000 7fbcf2b29700 1 -- 172.21.15.162:0/3146062683 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbcec0a9960 msgr2=0x7fbcec144220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:02.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.078+0000 7fbcf2b29700 1 --2- 172.21.15.162:0/3146062683 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbcec0a9960 0x7fbcec144220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:02.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.078+0000 7fbcf2b29700 1 -- 172.21.15.162:0/3146062683 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbcec0a9d40 msgr2=0x7fbcec140160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:02.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.078+0000 7fbcf2b29700 1 --2- 172.21.15.162:0/3146062683 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbcec0a9d40 0x7fbcec140160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:02.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.078+0000 7fbcf2b29700 1 -- 172.21.15.162:0/3146062683 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcec0ac4e0 con 0x7fbcec141820 2020-11-28T21:56:02.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.078+0000 7fbcf2b29700 1 --2- 172.21.15.162:0/3146062683 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcec141820 0x7fbcec140650 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 rx=0x7fbce4000f00 tx=0x7fbce4007e10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:02.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.078+0000 7fbcdb7fe700 1 -- 172.21.15.162:0/3146062683 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbce4005960 con 0x7fbcec141820 2020-11-28T21:56:02.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.078+0000 7fbcdb7fe700 1 -- 172.21.15.162:0/3146062683 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce4005ac0 con 0x7fbcec141820 2020-11-28T21:56:02.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.078+0000 7fbcf332a700 1 -- 172.21.15.162:0/3146062683 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcec0ac780 con 0x7fbcec141820 2020-11-28T21:56:02.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.078+0000 7fbcdb7fe700 1 -- 172.21.15.162:0/3146062683 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbce4005e20 con 0x7fbcec141820 2020-11-28T21:56:02.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.078+0000 7fbcf332a700 1 -- 172.21.15.162:0/3146062683 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcec090940 con 0x7fbcec141820 2020-11-28T21:56:02.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.078+0000 7fbcf332a700 1 -- 172.21.15.162:0/3146062683 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcec0ac910 con 0x7fbcec141820 2020-11-28T21:56:02.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.078+0000 7fbcdb7fe700 1 -- 172.21.15.162:0/3146062683 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fbce4010030 con 0x7fbcec141820 2020-11-28T21:56:02.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.078+0000 7fbcdb7fe700 1 --2- 172.21.15.162:0/3146062683 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbccc0553b0 0x7fbccc057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:02.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.078+0000 7fbcdb7fe700 1 -- 172.21.15.162:0/3146062683 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbce406ded0 con 0x7fbcec141820 2020-11-28T21:56:02.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.078+0000 7fbcf2328700 1 --2- 172.21.15.162:0/3146062683 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbccc0553b0 0x7fbccc057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:02.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.082+0000 7fbcf2328700 1 --2- 172.21.15.162:0/3146062683 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbccc0553b0 0x7fbccc057870 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7fbce800d000 tx=0x7fbce8007b40).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:02.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.086+0000 7fbcdb7fe700 1 -- 172.21.15.162:0/3146062683 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fbce4044130 con 0x7fbcec141820 2020-11-28T21:56:02.279 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:02.274+0000 7fbcf332a700 1 -- 172.21.15.162:0/3146062683 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fbcec057050 con 0x7fbcec141820 2020-11-28T21:56:03.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.034+0000 7fbcdb7fe700 1 -- 172.21.15.162:0/3146062683 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v74) v1 ==== 118+0+0 (secure 0 0 0) 0x7fbce4047750 con 0x7fbcec141820 2020-11-28T21:56:03.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.038+0000 7fbcf332a700 1 -- 172.21.15.162:0/3146062683 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbccc0553b0 msgr2=0x7fbccc057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:03.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.038+0000 7fbcf332a700 1 --2- 172.21.15.162:0/3146062683 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbccc0553b0 0x7fbccc057870 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7fbce800d000 tx=0x7fbce8007b40).stop 2020-11-28T21:56:03.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.038+0000 7fbcf332a700 1 -- 172.21.15.162:0/3146062683 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcec141820 msgr2=0x7fbcec140650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:03.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.038+0000 7fbcf332a700 1 --2- 172.21.15.162:0/3146062683 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcec141820 0x7fbcec140650 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 rx=0x7fbce4000f00 tx=0x7fbce4007e10).stop 2020-11-28T21:56:03.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.038+0000 7fbcf332a700 1 -- 172.21.15.162:0/3146062683 shutdown_connections 2020-11-28T21:56:03.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.038+0000 7fbcf332a700 1 --2- 172.21.15.162:0/3146062683 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbccc0553b0 0x7fbccc057870 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:03.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.038+0000 7fbcf332a700 1 --2- 172.21.15.162:0/3146062683 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbcec0a9960 0x7fbcec144220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:03.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.038+0000 7fbcf332a700 1 --2- 172.21.15.162:0/3146062683 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcec141820 0x7fbcec140650 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:03.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.038+0000 7fbcf332a700 1 --2- 172.21.15.162:0/3146062683 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbcec0a9d40 0x7fbcec140160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:03.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.038+0000 7fbcf332a700 1 -- 172.21.15.162:0/3146062683 >> 172.21.15.162:0/3146062683 conn(0x7fbcec0b5520 msgr2=0x7fbcec05b970 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:03.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.038+0000 7fbcf332a700 1 -- 172.21.15.162:0/3146062683 shutdown_connections 2020-11-28T21:56:03.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.038+0000 7fbcf332a700 1 -- 172.21.15.162:0/3146062683 wait complete. 2020-11-28T21:56:03.061 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:56:03.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.178+0000 7f577197d700 1 Processor -- start 2020-11-28T21:56:03.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.178+0000 7f577197d700 1 -- start start 2020-11-28T21:56:03.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f577197d700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f576c0b1990 0x7f576c140890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:03.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f577197d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f576c0b11b0 0x7f576c140d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:03.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f577197d700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f576c0b2870 con 0x7f576c0b1d70 2020-11-28T21:56:03.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f577197d700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f576c146e20 con 0x7f576c0b11b0 2020-11-28T21:56:03.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f577197d700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f576c0b22e0 con 0x7f576c0b1990 2020-11-28T21:56:03.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f576b7fe700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f576c0b1d70 0x7f576c14a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:52526/0 (socket says 172.21.15.162:52526) 2020-11-28T21:56:03.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f576b7fe700 1 -- 172.21.15.162:0/759999756 learned_addr learned my addr 172.21.15.162:0/759999756 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:56:03.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f576a7fc700 1 --2- 172.21.15.162:0/759999756 >> v2:172.21.15.118:3300/0 conn(0x7f576c0b11b0 0x7f576c140d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:03.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f576affd700 1 --2- 172.21.15.162:0/759999756 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f576c0b1990 0x7f576c140890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:03.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f5769ffb700 1 -- 172.21.15.162:0/759999756 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3202182520 0 0) 0x7f576c0b2870 con 0x7f576c0b1d70 2020-11-28T21:56:03.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f5769ffb700 1 -- 172.21.15.162:0/759999756 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f574c003230 con 0x7f576c0b1d70 2020-11-28T21:56:03.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f5769ffb700 1 -- 172.21.15.162:0/759999756 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1839353764 0 0) 0x7f574c003230 con 0x7f576c0b1d70 2020-11-28T21:56:03.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f5769ffb700 1 -- 172.21.15.162:0/759999756 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f576c0b1990 msgr2=0x7f576c140890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:03.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f5769ffb700 1 --2- 172.21.15.162:0/759999756 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f576c0b1990 0x7f576c140890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:03.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f5769ffb700 1 -- 172.21.15.162:0/759999756 >> v2:172.21.15.118:3300/0 conn(0x7f576c0b11b0 msgr2=0x7f576c140d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:03.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f5769ffb700 1 --2- 172.21.15.162:0/759999756 >> v2:172.21.15.118:3300/0 conn(0x7f576c0b11b0 0x7f576c140d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:03.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f5769ffb700 1 -- 172.21.15.162:0/759999756 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f576c0b0740 con 0x7f576c0b1d70 2020-11-28T21:56:03.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f5769ffb700 1 -- 172.21.15.162:0/759999756 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f5764001a50 con 0x7f576c0b1d70 2020-11-28T21:56:03.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f5769ffb700 1 -- 172.21.15.162:0/759999756 >> v1:172.21.15.162:6789/0 conn(0x7f576c0b1d70 legacy=0x7f576c14a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:03.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f5769ffb700 1 --2- 172.21.15.162:0/759999756 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f574c003720 0x7f574c003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:03.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f5769ffb700 1 --2- 172.21.15.162:0/759999756 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f576c0b1990 0x7f574c0044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:03.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f5769ffb700 1 --2- 172.21.15.162:0/759999756 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f574c0049c0 0x7f574c004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:03.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f5769ffb700 1 -- 172.21.15.162:0/759999756 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576c0b22e0 con 0x7f574c0049c0 2020-11-28T21:56:03.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f5769ffb700 1 -- 172.21.15.162:0/759999756 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c0041c0 con 0x7f574c003720 2020-11-28T21:56:03.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f5769ffb700 1 -- 172.21.15.162:0/759999756 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574c00b720 con 0x7f576c0b1990 2020-11-28T21:56:03.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f576affd700 1 --2- 172.21.15.162:0/759999756 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f574c003720 0x7f574c003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:03.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f576a7fc700 1 --2- 172.21.15.162:0/759999756 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f576c0b1990 0x7f574c0044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:03.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f576b7fe700 1 --2- 172.21.15.162:0/759999756 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f574c0049c0 0x7f574c004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:03.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f576affd700 1 -- 172.21.15.162:0/759999756 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f576c0b1990 msgr2=0x7f574c0044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:03.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f576affd700 1 --2- 172.21.15.162:0/759999756 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f576c0b1990 0x7f574c0044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:03.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f576affd700 1 -- 172.21.15.162:0/759999756 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f574c0049c0 msgr2=0x7f574c004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:03.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f576affd700 1 --2- 172.21.15.162:0/759999756 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f574c0049c0 0x7f574c004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:03.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f576affd700 1 -- 172.21.15.162:0/759999756 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f574c003230 con 0x7f574c003720 2020-11-28T21:56:03.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f576b7fe700 1 --2- 172.21.15.162:0/759999756 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f574c0049c0 0x7f574c004e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:56:03.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f576affd700 1 --2- 172.21.15.162:0/759999756 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f574c003720 0x7f574c003b40 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 rx=0x7f575c00c000 tx=0x7f575c008040).ready entity=mon.1 client_cookie=69607a89d4997c94 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:03.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f5769ffb700 1 -- 172.21.15.162:0/759999756 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f575c00e9d0 con 0x7f574c003720 2020-11-28T21:56:03.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f5769ffb700 1 -- 172.21.15.162:0/759999756 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f575c00eb30 con 0x7f574c003720 2020-11-28T21:56:03.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f577197d700 1 -- 172.21.15.162:0/759999756 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f574c003720 msgr2=0x7f574c003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:03.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f5769ffb700 1 -- 172.21.15.162:0/759999756 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f575c00ee90 con 0x7f574c003720 2020-11-28T21:56:03.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f577197d700 1 --2- 172.21.15.162:0/759999756 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f574c003720 0x7f574c003b40 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 rx=0x7f575c00c000 tx=0x7f575c008040).stop 2020-11-28T21:56:03.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f576b7fe700 1 -- 172.21.15.162:0/759999756 reap_dead start 2020-11-28T21:56:03.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f577197d700 1 -- 172.21.15.162:0/759999756 shutdown_connections 2020-11-28T21:56:03.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f577197d700 1 -- 172.21.15.162:0/759999756 >> 172.21.15.162:0/759999756 conn(0x7f576c05b970 msgr2=0x7f576c05bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:03.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f577197d700 1 -- 172.21.15.162:0/759999756 shutdown_connections 2020-11-28T21:56:03.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.182+0000 7f577197d700 1 -- 172.21.15.162:0/759999756 wait complete. 2020-11-28T21:56:03.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.186+0000 7f577197d700 1 Processor -- start 2020-11-28T21:56:03.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.186+0000 7f577197d700 1 -- start start 2020-11-28T21:56:03.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.186+0000 7f577197d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f576c063880 0x7f576c155a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:03.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.186+0000 7f577197d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f576c067020 0x7f576c159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:03.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.186+0000 7f577197d700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f576c159bf0 0x7f576c1436b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:03.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.186+0000 7f577197d700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576c159030 con 0x7f576c063880 2020-11-28T21:56:03.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.186+0000 7f577197d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576c063c60 con 0x7f576c067020 2020-11-28T21:56:03.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.186+0000 7f577197d700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f576c0663c0 con 0x7f576c159bf0 2020-11-28T21:56:03.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.186+0000 7f576a7fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f576c067020 0x7f576c159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:03.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.186+0000 7f576a7fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f576c067020 0x7f576c159700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45696/0 (socket says 172.21.15.162:45696) 2020-11-28T21:56:03.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.186+0000 7f576a7fc700 1 -- 172.21.15.162:0/3650891298 learned_addr learned my addr 172.21.15.162:0/3650891298 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:03.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.186+0000 7f576affd700 1 --2- 172.21.15.162:0/3650891298 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f576c063880 0x7f576c155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:03.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.186+0000 7f576b7fe700 1 --2- 172.21.15.162:0/3650891298 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f576c159bf0 0x7f576c1436b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:03.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.186+0000 7f576a7fc700 1 -- 172.21.15.162:0/3650891298 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f576c159bf0 msgr2=0x7f576c1436b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:03.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.186+0000 7f576a7fc700 1 --2- 172.21.15.162:0/3650891298 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f576c159bf0 0x7f576c1436b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:03.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.186+0000 7f576a7fc700 1 -- 172.21.15.162:0/3650891298 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f576c063880 msgr2=0x7f576c155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:03.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.186+0000 7f576a7fc700 1 --2- 172.21.15.162:0/3650891298 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f576c063880 0x7f576c155a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:03.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.186+0000 7f576a7fc700 1 -- 172.21.15.162:0/3650891298 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f576c159260 con 0x7f576c067020 2020-11-28T21:56:03.204 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.186+0000 7f576a7fc700 1 --2- 172.21.15.162:0/3650891298 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f576c067020 0x7f576c159700 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 rx=0x7f5754003070 tx=0x7f5754008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:03.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.186+0000 7f5763fff700 1 -- 172.21.15.162:0/3650891298 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f575400e970 con 0x7f576c067020 2020-11-28T21:56:03.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.186+0000 7f5763fff700 1 -- 172.21.15.162:0/3650891298 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f575400ead0 con 0x7f576c067020 2020-11-28T21:56:03.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.186+0000 7f577197d700 1 -- 172.21.15.162:0/3650891298 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f576c066660 con 0x7f576c067020 2020-11-28T21:56:03.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.186+0000 7f5763fff700 1 -- 172.21.15.162:0/3650891298 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f575400ee70 con 0x7f576c067020 2020-11-28T21:56:03.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.186+0000 7f577197d700 1 -- 172.21.15.162:0/3650891298 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f576c0643c0 con 0x7f576c067020 2020-11-28T21:56:03.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.190+0000 7f577197d700 1 -- 172.21.15.162:0/3650891298 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f576c057050 con 0x7f576c067020 2020-11-28T21:56:03.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.190+0000 7f5763fff700 1 -- 172.21.15.162:0/3650891298 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5754024070 con 0x7f576c067020 2020-11-28T21:56:03.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.190+0000 7f5763fff700 1 --2- 172.21.15.162:0/3650891298 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5750055330 0x7f57500577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:03.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.190+0000 7f576affd700 1 --2- 172.21.15.162:0/3650891298 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5750055330 0x7f57500577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:03.206 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.190+0000 7f5763fff700 1 -- 172.21.15.162:0/3650891298 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f575406e910 con 0x7f576c067020 2020-11-28T21:56:03.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.190+0000 7f576affd700 1 --2- 172.21.15.162:0/3650891298 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5750055330 0x7f57500577f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7f575c008000 tx=0x7f575c01a000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:03.207 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.194+0000 7f5763fff700 1 -- 172.21.15.162:0/3650891298 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5754044c10 con 0x7f576c067020 2020-11-28T21:56:03.388 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.382+0000 7f577197d700 1 -- 172.21.15.162:0/3650891298 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f576c05d530 con 0x7f576c067020 2020-11-28T21:56:03.389 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.382+0000 7f5763fff700 1 -- 172.21.15.162:0/3650891298 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+2504 (secure 0 0 0) 0x7f5754048230 con 0x7f576c067020 2020-11-28T21:56:03.389 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:56:03.390 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":74,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6899,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.118:6833/2509028245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":2509028245},{"type":"v1","addr":"172.21.15.118:6833","nonce":2509028245}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":72},{"gid":8239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/2089791609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2089791609},{"type":"v1","addr":"172.21.15.162:6836","nonce":2089791609}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":71}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:56:01.362652+0000","modified":"2020-11-28T21:56:03.027360+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8242},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8242":{"gid":8242,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/2832109513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":2832109513},{"type":"v1","addr":"172.21.15.162:6837","nonce":2832109513}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2020-11-28T21:56:03.393 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.386+0000 7f577197d700 1 -- 172.21.15.162:0/3650891298 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5750055330 msgr2=0x7f57500577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:03.393 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.386+0000 7f577197d700 1 --2- 172.21.15.162:0/3650891298 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5750055330 0x7f57500577f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7f575c008000 tx=0x7f575c01a000).stop 2020-11-28T21:56:03.394 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.386+0000 7f577197d700 1 -- 172.21.15.162:0/3650891298 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f576c067020 msgr2=0x7f576c159700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:03.394 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.386+0000 7f577197d700 1 --2- 172.21.15.162:0/3650891298 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f576c067020 0x7f576c159700 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 rx=0x7f5754003070 tx=0x7f5754008040).stop 2020-11-28T21:56:03.394 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.386+0000 7f577197d700 1 -- 172.21.15.162:0/3650891298 shutdown_connections 2020-11-28T21:56:03.395 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.386+0000 7f577197d700 1 --2- 172.21.15.162:0/3650891298 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5750055330 0x7f57500577f0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:03.395 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.386+0000 7f577197d700 1 --2- 172.21.15.162:0/3650891298 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f576c159bf0 0x7f576c1436b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:03.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.386+0000 7f577197d700 1 --2- 172.21.15.162:0/3650891298 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f576c063880 0x7f576c155a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:03.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.386+0000 7f577197d700 1 --2- 172.21.15.162:0/3650891298 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f576c067020 0x7f576c159700 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:03.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.386+0000 7f577197d700 1 -- 172.21.15.162:0/3650891298 >> 172.21.15.162:0/3650891298 conn(0x7f576c05b970 msgr2=0x7f576c148a20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:03.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.386+0000 7f577197d700 1 -- 172.21.15.162:0/3650891298 shutdown_connections 2020-11-28T21:56:03.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.386+0000 7f577197d700 1 -- 172.21.15.162:0/3650891298 wait complete. 2020-11-28T21:56:03.398 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 74 2020-11-28T21:56:03.408 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:56:03.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.510+0000 7fd2db59e700 1 Processor -- start 2020-11-28T21:56:03.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.510+0000 7fd2db59e700 1 -- start start 2020-11-28T21:56:03.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.510+0000 7fd2db59e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd2dc0a9940 0x7fd2dc0b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:03.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.510+0000 7fd2db59e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd2dc146510 0x7fd2dc146950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:03.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.510+0000 7fd2db59e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2dc0a9dd0 con 0x7fd2dc0a9560 2020-11-28T21:56:03.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.510+0000 7fd2db59e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2dc0a86a0 con 0x7fd2dc146510 2020-11-28T21:56:03.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.510+0000 7fd2db59e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2dc0a8890 con 0x7fd2dc0a9940 2020-11-28T21:56:03.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.510+0000 7fd2d9d9b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd2dc0a9940 0x7fd2dc0b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:03.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.510+0000 7fd2d9d9b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd2dc0a9940 0x7fd2dc0b1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45702/0 (socket says 172.21.15.162:45702) 2020-11-28T21:56:03.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.510+0000 7fd2d9d9b700 1 -- 172.21.15.162:0/1951154887 learned_addr learned my addr 172.21.15.162:0/1951154887 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:03.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.510+0000 7fd2dad9d700 1 --2- 172.21.15.162:0/1951154887 >> v2:172.21.15.118:3300/0 conn(0x7fd2dc146510 0x7fd2dc146950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:03.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.510+0000 7fd2d9d9b700 1 -- 172.21.15.162:0/1951154887 >> v2:172.21.15.118:3300/0 conn(0x7fd2dc146510 msgr2=0x7fd2dc146950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:03.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.510+0000 7fd2d9d9b700 1 --2- 172.21.15.162:0/1951154887 >> v2:172.21.15.118:3300/0 conn(0x7fd2dc146510 0x7fd2dc146950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:03.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.510+0000 7fd2d9d9b700 1 -- 172.21.15.162:0/1951154887 >> v1:172.21.15.118:6789/0 conn(0x7fd2dc0a9560 legacy=0x7fd2dc0b16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:03.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.510+0000 7fd2d9d9b700 1 -- 172.21.15.162:0/1951154887 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2dc0b02f0 con 0x7fd2dc0a9940 2020-11-28T21:56:03.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.510+0000 7fd2d9d9b700 1 --2- 172.21.15.162:0/1951154887 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2dc0a9940 0x7fd2dc0b1db0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 rx=0x7fd2cc0032b0 tx=0x7fd2cc009270).ready entity=mon.1 client_cookie=d565a3ca3235636b server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:03.522 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.510+0000 7fd2d959a700 1 -- 172.21.15.162:0/1951154887 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2cc009e60 con 0x7fd2dc0a9940 2020-11-28T21:56:03.522 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.514+0000 7fd2d959a700 1 -- 172.21.15.162:0/1951154887 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2cc00bc00 con 0x7fd2dc0a9940 2020-11-28T21:56:03.522 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.514+0000 7fd2d959a700 1 -- 172.21.15.162:0/1951154887 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2cc00bd70 con 0x7fd2dc0a9940 2020-11-28T21:56:03.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.514+0000 7fd2db59e700 1 -- 172.21.15.162:0/1951154887 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2dc0a9940 msgr2=0x7fd2dc0b1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:03.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.514+0000 7fd2db59e700 1 --2- 172.21.15.162:0/1951154887 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2dc0a9940 0x7fd2dc0b1db0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 rx=0x7fd2cc0032b0 tx=0x7fd2cc009270).stop 2020-11-28T21:56:03.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.514+0000 7fd2db59e700 1 -- 172.21.15.162:0/1951154887 shutdown_connections 2020-11-28T21:56:03.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.514+0000 7fd2db59e700 1 --2- 172.21.15.162:0/1951154887 >> v2:172.21.15.118:3300/0 conn(0x7fd2dc146510 0x7fd2dc146950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:03.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.514+0000 7fd2db59e700 1 --2- 172.21.15.162:0/1951154887 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2dc0a9940 0x7fd2dc0b1db0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:03.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.514+0000 7fd2db59e700 1 -- 172.21.15.162:0/1951154887 >> 172.21.15.162:0/1951154887 conn(0x7fd2dc05b970 msgr2=0x7fd2dc05bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:03.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.514+0000 7fd2db59e700 1 -- 172.21.15.162:0/1951154887 shutdown_connections 2020-11-28T21:56:03.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.514+0000 7fd2db59e700 1 -- 172.21.15.162:0/1951154887 wait complete. 2020-11-28T21:56:03.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.514+0000 7fd2db59e700 1 Processor -- start 2020-11-28T21:56:03.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.514+0000 7fd2db59e700 1 -- start start 2020-11-28T21:56:03.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.514+0000 7fd2db59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2dc0a9560 0x7fd2dc0b3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:03.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.514+0000 7fd2db59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2dc0a9940 0x7fd2dc1520b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:03.526 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.514+0000 7fd2db59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2dc146510 0x7fd2dc1525a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:03.526 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.514+0000 7fd2db59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2dc142570 con 0x7fd2dc146510 2020-11-28T21:56:03.526 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.514+0000 7fd2db59e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2dc0945d0 con 0x7fd2dc0a9560 2020-11-28T21:56:03.526 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.514+0000 7fd2db59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2dc1519b0 con 0x7fd2dc0a9940 2020-11-28T21:56:03.526 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.514+0000 7fd2da59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2dc0a9560 0x7fd2dc0b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:03.527 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.514+0000 7fd2da59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2dc0a9560 0x7fd2dc0b3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45708/0 (socket says 172.21.15.162:45708) 2020-11-28T21:56:03.527 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.514+0000 7fd2dad9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2dc146510 0x7fd2dc1525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:03.527 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.514+0000 7fd2da59c700 1 -- 172.21.15.162:0/3243595211 learned_addr learned my addr 172.21.15.162:0/3243595211 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:03.527 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.514+0000 7fd2d9d9b700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2dc0a9940 0x7fd2dc1520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:03.527 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.518+0000 7fd2da59c700 1 -- 172.21.15.162:0/3243595211 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2dc0a9940 msgr2=0x7fd2dc1520b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:03.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.518+0000 7fd2da59c700 1 --2- 172.21.15.162:0/3243595211 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2dc0a9940 0x7fd2dc1520b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:03.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.518+0000 7fd2da59c700 1 -- 172.21.15.162:0/3243595211 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2dc146510 msgr2=0x7fd2dc1525a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:03.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.518+0000 7fd2da59c700 1 --2- 172.21.15.162:0/3243595211 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2dc146510 0x7fd2dc1525a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:03.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.518+0000 7fd2da59c700 1 -- 172.21.15.162:0/3243595211 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2dc090870 con 0x7fd2dc0a9560 2020-11-28T21:56:03.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.518+0000 7fd2da59c700 1 --2- 172.21.15.162:0/3243595211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2dc0a9560 0x7fd2dc0b3990 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 rx=0x7fd2dc067df0 tx=0x7fd2c4007be0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:03.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.518+0000 7fd2d37fe700 1 -- 172.21.15.162:0/3243595211 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2c4007240 con 0x7fd2dc0a9560 2020-11-28T21:56:03.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.518+0000 7fd2d37fe700 1 -- 172.21.15.162:0/3243595211 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c40073a0 con 0x7fd2dc0a9560 2020-11-28T21:56:03.530 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.518+0000 7fd2db59e700 1 -- 172.21.15.162:0/3243595211 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2dc090b10 con 0x7fd2dc0a9560 2020-11-28T21:56:03.530 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.518+0000 7fd2d37fe700 1 -- 172.21.15.162:0/3243595211 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2c40064e0 con 0x7fd2dc0a9560 2020-11-28T21:56:03.530 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.518+0000 7fd2db59e700 1 -- 172.21.15.162:0/3243595211 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2dc151c10 con 0x7fd2dc0a9560 2020-11-28T21:56:03.531 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.518+0000 7fd2db59e700 1 -- 172.21.15.162:0/3243595211 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2dc057050 con 0x7fd2dc0a9560 2020-11-28T21:56:03.531 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.518+0000 7fd2d37fe700 1 -- 172.21.15.162:0/3243595211 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd2c4010070 con 0x7fd2dc0a9560 2020-11-28T21:56:03.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.518+0000 7fd2d37fe700 1 --2- 172.21.15.162:0/3243595211 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd2c0055330 0x7fd2c00577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:03.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.522+0000 7fd2d37fe700 1 -- 172.21.15.162:0/3243595211 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd2c406d620 con 0x7fd2dc0a9560 2020-11-28T21:56:03.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.522+0000 7fd2d9d9b700 1 --2- 172.21.15.162:0/3243595211 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd2c0055330 0x7fd2c00577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:03.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.522+0000 7fd2d9d9b700 1 --2- 172.21.15.162:0/3243595211 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd2c0055330 0x7fd2c00577f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7fd2cc01a000 tx=0x7fd2cc016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:03.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.526+0000 7fd2d37fe700 1 -- 172.21.15.162:0/3243595211 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd2c40437b0 con 0x7fd2dc0a9560 2020-11-28T21:56:03.670 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.666+0000 7fd2db59e700 1 -- 172.21.15.162:0/3243595211 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2dc068350 con 0x7fd2dc0a9560 2020-11-28T21:56:03.672 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.666+0000 7fd2d37fe700 1 -- 172.21.15.162:0/3243595211 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+16148 (secure 0 0 0) 0x7fd2c4018030 con 0x7fd2dc0a9560 2020-11-28T21:56:03.672 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:56:03.673 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":148,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:56:01.362358+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:55:59.967971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2020-11-28T21:56:00.379272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:56:03.674 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.670+0000 7fd2db59e700 1 -- 172.21.15.162:0/3243595211 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd2c0055330 msgr2=0x7fd2c00577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:03.674 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.670+0000 7fd2db59e700 1 --2- 172.21.15.162:0/3243595211 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd2c0055330 0x7fd2c00577f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7fd2cc01a000 tx=0x7fd2cc016000).stop 2020-11-28T21:56:03.674 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.670+0000 7fd2db59e700 1 -- 172.21.15.162:0/3243595211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2dc0a9560 msgr2=0x7fd2dc0b3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:03.674 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.670+0000 7fd2db59e700 1 --2- 172.21.15.162:0/3243595211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2dc0a9560 0x7fd2dc0b3990 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 rx=0x7fd2dc067df0 tx=0x7fd2c4007be0).stop 2020-11-28T21:56:03.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.670+0000 7fd2db59e700 1 -- 172.21.15.162:0/3243595211 shutdown_connections 2020-11-28T21:56:03.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.670+0000 7fd2db59e700 1 --2- 172.21.15.162:0/3243595211 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd2c0055330 0x7fd2c00577f0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:03.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.670+0000 7fd2db59e700 1 --2- 172.21.15.162:0/3243595211 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2dc146510 0x7fd2dc1525a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:03.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.670+0000 7fd2db59e700 1 --2- 172.21.15.162:0/3243595211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2dc0a9560 0x7fd2dc0b3990 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:03.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.670+0000 7fd2db59e700 1 --2- 172.21.15.162:0/3243595211 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2dc0a9940 0x7fd2dc1520b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:03.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.670+0000 7fd2db59e700 1 -- 172.21.15.162:0/3243595211 >> 172.21.15.162:0/3243595211 conn(0x7fd2dc05b970 msgr2=0x7fd2dc099bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:03.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.670+0000 7fd2db59e700 1 -- 172.21.15.162:0/3243595211 shutdown_connections 2020-11-28T21:56:03.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.670+0000 7fd2db59e700 1 -- 172.21.15.162:0/3243595211 wait complete. 2020-11-28T21:56:03.686 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-11-28T21:56:03.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.806+0000 7f7aab75f700 1 Processor -- start 2020-11-28T21:56:03.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.806+0000 7f7aab75f700 1 -- start start 2020-11-28T21:56:03.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.806+0000 7f7aab75f700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7aa40b55a0 0x7f7aa40ad6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:03.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.806+0000 7f7aab75f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7aa4080500 0x7f7aa40adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:03.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.806+0000 7f7aab75f700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aa40b5980 con 0x7f7aa40a86d0 2020-11-28T21:56:03.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.806+0000 7f7aab75f700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7aa40afe80 con 0x7f7aa40b55a0 2020-11-28T21:56:03.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.806+0000 7f7aab75f700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa40809a0 con 0x7f7aa4080500 2020-11-28T21:56:03.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.806+0000 7f7aaa4fd700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f7aa40a86d0 0x7f7aa405b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:52560/0 (socket says 172.21.15.162:52560) 2020-11-28T21:56:03.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.806+0000 7f7aaa4fd700 1 -- 172.21.15.162:0/2184050742 learned_addr learned my addr 172.21.15.162:0/2184050742 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:56:03.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.806+0000 7f7aa9cfc700 1 --2- 172.21.15.162:0/2184050742 >> v2:172.21.15.162:3300/0 conn(0x7f7aa40b55a0 0x7f7aa40ad6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:03.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.806+0000 7f7aa94fb700 1 --2- 172.21.15.162:0/2184050742 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7aa4080500 0x7f7aa40adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:03.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.806+0000 7f7aa9cfc700 1 -- 172.21.15.162:0/2184050742 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7aa4080500 msgr2=0x7f7aa40adbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:03.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.806+0000 7f7aa9cfc700 1 --2- 172.21.15.162:0/2184050742 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7aa4080500 0x7f7aa40adbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:03.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.806+0000 7f7aa9cfc700 1 -- 172.21.15.162:0/2184050742 >> v1:172.21.15.162:6789/0 conn(0x7f7aa40a86d0 legacy=0x7f7aa405b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:03.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.806+0000 7f7aa8cfa700 1 -- 172.21.15.162:0/2184050742 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2655223003 0 0) 0x7f7aa40b5980 con 0x7f7aa40a86d0 2020-11-28T21:56:03.818 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.806+0000 7f7aa9cfc700 1 -- 172.21.15.162:0/2184050742 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa40af740 con 0x7f7aa40b55a0 2020-11-28T21:56:03.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.806+0000 7f7aa9cfc700 1 --2- 172.21.15.162:0/2184050742 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7aa40b55a0 0x7f7aa40ad6c0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 rx=0x7f7a98002f30 tx=0x7f7a98006dc0).ready entity=mon.1 client_cookie=ab8c40e15bc0e2d7 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:03.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.806+0000 7f7aa8cfa700 1 -- 172.21.15.162:0/2184050742 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a98009760 con 0x7f7aa40b55a0 2020-11-28T21:56:03.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.806+0000 7f7aa8cfa700 1 -- 172.21.15.162:0/2184050742 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a980098c0 con 0x7f7aa40b55a0 2020-11-28T21:56:03.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.806+0000 7f7aa8cfa700 1 -- 172.21.15.162:0/2184050742 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a98009c20 con 0x7f7aa40b55a0 2020-11-28T21:56:03.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.806+0000 7f7aab75f700 1 -- 172.21.15.162:0/2184050742 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7aa40b55a0 msgr2=0x7f7aa40ad6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:03.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.806+0000 7f7aab75f700 1 --2- 172.21.15.162:0/2184050742 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7aa40b55a0 0x7f7aa40ad6c0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 rx=0x7f7a98002f30 tx=0x7f7a98006dc0).stop 2020-11-28T21:56:03.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.810+0000 7f7aab75f700 1 -- 172.21.15.162:0/2184050742 shutdown_connections 2020-11-28T21:56:03.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.810+0000 7f7aab75f700 1 --2- 172.21.15.162:0/2184050742 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7aa40b55a0 0x7f7aa40ad6c0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:03.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.810+0000 7f7aab75f700 1 --2- 172.21.15.162:0/2184050742 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7aa4080500 0x7f7aa40adbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:03.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.810+0000 7f7aab75f700 1 -- 172.21.15.162:0/2184050742 >> 172.21.15.162:0/2184050742 conn(0x7f7aa40626d0 msgr2=0x7f7aa4062ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:03.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.810+0000 7f7aab75f700 1 -- 172.21.15.162:0/2184050742 shutdown_connections 2020-11-28T21:56:03.821 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.810+0000 7f7aab75f700 1 -- 172.21.15.162:0/2184050742 wait complete. 2020-11-28T21:56:03.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.810+0000 7f7aab75f700 1 Processor -- start 2020-11-28T21:56:03.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.810+0000 7f7aab75f700 1 -- start start 2020-11-28T21:56:03.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.810+0000 7f7aab75f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7aa4080500 0x7f7aa41519c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:03.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.810+0000 7f7aab75f700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7aa40a86d0 0x7f7aa414f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:03.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.810+0000 7f7aab75f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7aa40b55a0 0x7f7aa414f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:03.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.810+0000 7f7aab75f700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa4067230 con 0x7f7aa40a86d0 2020-11-28T21:56:03.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.810+0000 7f7aab75f700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa40661c0 con 0x7f7aa40b55a0 2020-11-28T21:56:03.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.810+0000 7f7aab75f700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa4064e10 con 0x7f7aa4080500 2020-11-28T21:56:03.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.810+0000 7f7aaa4fd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7aa40b55a0 0x7f7aa414f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:03.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.814+0000 7f7aaa4fd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7aa40b55a0 0x7f7aa414f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45726/0 (socket says 172.21.15.162:45726) 2020-11-28T21:56:03.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.814+0000 7f7aaa4fd700 1 -- 172.21.15.162:0/2840903411 learned_addr learned my addr 172.21.15.162:0/2840903411 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:03.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.814+0000 7f7aa9cfc700 1 --2- 172.21.15.162:0/2840903411 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7aa4080500 0x7f7aa41519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:03.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.814+0000 7f7aa94fb700 1 --2- 172.21.15.162:0/2840903411 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7aa40a86d0 0x7f7aa414f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:03.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.814+0000 7f7aaa4fd700 1 -- 172.21.15.162:0/2840903411 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7aa4080500 msgr2=0x7f7aa41519c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:03.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.814+0000 7f7aaa4fd700 1 --2- 172.21.15.162:0/2840903411 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7aa4080500 0x7f7aa41519c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:03.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.814+0000 7f7aaa4fd700 1 -- 172.21.15.162:0/2840903411 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7aa40a86d0 msgr2=0x7f7aa414f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:03.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.814+0000 7f7aaa4fd700 1 --2- 172.21.15.162:0/2840903411 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7aa40a86d0 0x7f7aa414f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:03.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.814+0000 7f7aaa4fd700 1 -- 172.21.15.162:0/2840903411 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa40650b0 con 0x7f7aa40b55a0 2020-11-28T21:56:03.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.814+0000 7f7aaa4fd700 1 --2- 172.21.15.162:0/2840903411 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7aa40b55a0 0x7f7aa414f510 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 rx=0x7f7a9c000f00 tx=0x7f7a9c007d50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:03.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.814+0000 7f7a967fc700 1 -- 172.21.15.162:0/2840903411 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a9c005b70 con 0x7f7aa40b55a0 2020-11-28T21:56:03.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.814+0000 7f7a967fc700 1 -- 172.21.15.162:0/2840903411 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a9c005cd0 con 0x7f7aa40b55a0 2020-11-28T21:56:03.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.814+0000 7f7a967fc700 1 -- 172.21.15.162:0/2840903411 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a9c007300 con 0x7f7aa40b55a0 2020-11-28T21:56:03.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.814+0000 7f7aab75f700 1 -- 172.21.15.162:0/2840903411 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aa40b41c0 con 0x7f7aa40b55a0 2020-11-28T21:56:03.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.814+0000 7f7aab75f700 1 -- 172.21.15.162:0/2840903411 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aa409e380 con 0x7f7aa40b55a0 2020-11-28T21:56:03.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.814+0000 7f7aab75f700 1 -- 172.21.15.162:0/2840903411 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aa4066300 con 0x7f7aa40b55a0 2020-11-28T21:56:03.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.814+0000 7f7a967fc700 1 -- 172.21.15.162:0/2840903411 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7a9c010030 con 0x7f7aa40b55a0 2020-11-28T21:56:03.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.814+0000 7f7a967fc700 1 --2- 172.21.15.162:0/2840903411 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7a8c055320 0x7f7a8c0577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:03.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.814+0000 7f7aa9cfc700 1 --2- 172.21.15.162:0/2840903411 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7a8c055320 0x7f7a8c0577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:03.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.814+0000 7f7a967fc700 1 -- 172.21.15.162:0/2840903411 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7a9c06dff0 con 0x7f7aa40b55a0 2020-11-28T21:56:03.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.818+0000 7f7aa9cfc700 1 --2- 172.21.15.162:0/2840903411 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7a8c055320 0x7f7a8c0577e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f7a9801a000 tx=0x7f7a98015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:03.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:03.822+0000 7f7a967fc700 1 -- 172.21.15.162:0/2840903411 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7a9c0442f0 con 0x7f7aa40b55a0 2020-11-28T21:56:04.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.014+0000 7f7aab75f700 1 -- 172.21.15.162:0/2840903411 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f7aa4065240 con 0x7f7aa40b55a0 2020-11-28T21:56:04.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.022+0000 7f7a967fc700 1 -- 172.21.15.162:0/2840903411 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v31) v1 ==== 184+0+0 (secure 0 0 0) 0x7f7a9c047910 con 0x7f7aa40b55a0 2020-11-28T21:56:04.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.026+0000 7f7aab75f700 1 -- 172.21.15.162:0/2840903411 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7a8c055320 msgr2=0x7f7a8c0577e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:04.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.026+0000 7f7aab75f700 1 --2- 172.21.15.162:0/2840903411 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7a8c055320 0x7f7a8c0577e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f7a9801a000 tx=0x7f7a98015000).stop 2020-11-28T21:56:04.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.026+0000 7f7aab75f700 1 -- 172.21.15.162:0/2840903411 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7aa40b55a0 msgr2=0x7f7aa414f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:04.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.026+0000 7f7aab75f700 1 --2- 172.21.15.162:0/2840903411 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7aa40b55a0 0x7f7aa414f510 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 rx=0x7f7a9c000f00 tx=0x7f7a9c007d50).stop 2020-11-28T21:56:04.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.026+0000 7f7aab75f700 1 -- 172.21.15.162:0/2840903411 shutdown_connections 2020-11-28T21:56:04.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.026+0000 7f7aab75f700 1 --2- 172.21.15.162:0/2840903411 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7a8c055320 0x7f7a8c0577e0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:04.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.026+0000 7f7aab75f700 1 --2- 172.21.15.162:0/2840903411 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7aa4080500 0x7f7aa41519c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:04.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.026+0000 7f7aab75f700 1 --2- 172.21.15.162:0/2840903411 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7aa40b55a0 0x7f7aa414f510 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:04.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.026+0000 7f7aab75f700 1 --2- 172.21.15.162:0/2840903411 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7aa40a86d0 0x7f7aa414f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:04.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.026+0000 7f7aab75f700 1 -- 172.21.15.162:0/2840903411 >> 172.21.15.162:0/2840903411 conn(0x7f7aa40626d0 msgr2=0x7f7aa40ae340 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:04.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.026+0000 7f7aab75f700 1 -- 172.21.15.162:0/2840903411 shutdown_connections 2020-11-28T21:56:04.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.026+0000 7f7aab75f700 1 -- 172.21.15.162:0/2840903411 wait complete. 2020-11-28T21:56:04.033 INFO:teuthology.orchestra.run.smithi162.stderr:updated caps for client.0 2020-11-28T21:56:04.046 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-11-28T21:56:04.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.162+0000 7f23a7165700 1 Processor -- start 2020-11-28T21:56:04.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.162+0000 7f23a7165700 1 -- start start 2020-11-28T21:56:04.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.162+0000 7f23a7165700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f23a00b2e80 0x7f23a005f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:04.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.162+0000 7f23a7165700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23a00acb40 0x7f23a005f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:04.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.162+0000 7f23a7165700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23a00ab0b0 con 0x7f23a00b2560 2020-11-28T21:56:04.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.162+0000 7f23a7165700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23a005fd60 con 0x7f23a00b2e80 2020-11-28T21:56:04.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.162+0000 7f23a7165700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a00b3260 con 0x7f23a00acb40 2020-11-28T21:56:04.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.162+0000 7f23a6964700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f23a00b2e80 0x7f23a005f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:04.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.162+0000 7f23a6964700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f23a00b2e80 0x7f23a005f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45732/0 (socket says 172.21.15.162:45732) 2020-11-28T21:56:04.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.162+0000 7f23a6964700 1 -- 172.21.15.162:0/2248343285 learned_addr learned my addr 172.21.15.162:0/2248343285 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:04.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.162+0000 7f23a83c7700 1 --2- 172.21.15.162:0/2248343285 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23a00acb40 0x7f23a005f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:04.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.162+0000 7f23a6964700 1 -- 172.21.15.162:0/2248343285 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23a00acb40 msgr2=0x7f23a005f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:04.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.162+0000 7f23a6964700 1 --2- 172.21.15.162:0/2248343285 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23a00acb40 0x7f23a005f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:04.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.162+0000 7f23a6964700 1 -- 172.21.15.162:0/2248343285 >> v1:172.21.15.118:6789/0 conn(0x7f23a00b2560 legacy=0x7f23a0060d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:04.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.162+0000 7f23a6964700 1 -- 172.21.15.162:0/2248343285 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a00ab2e0 con 0x7f23a00b2e80 2020-11-28T21:56:04.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.162+0000 7f23a6964700 1 --2- 172.21.15.162:0/2248343285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23a00b2e80 0x7f23a005f380 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 rx=0x7f2390002f30 tx=0x7f2390006e80).ready entity=mon.1 client_cookie=d4a310206866b57f server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:04.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.162+0000 7f23a5962700 1 -- 172.21.15.162:0/2248343285 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23900097f0 con 0x7f23a00b2e80 2020-11-28T21:56:04.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.162+0000 7f23a5962700 1 -- 172.21.15.162:0/2248343285 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2390009950 con 0x7f23a00b2e80 2020-11-28T21:56:04.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.162+0000 7f23a5962700 1 -- 172.21.15.162:0/2248343285 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2390009cb0 con 0x7f23a00b2e80 2020-11-28T21:56:04.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.162+0000 7f23a7165700 1 -- 172.21.15.162:0/2248343285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23a00b2e80 msgr2=0x7f23a005f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:04.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.162+0000 7f23a7165700 1 --2- 172.21.15.162:0/2248343285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23a00b2e80 0x7f23a005f380 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 rx=0x7f2390002f30 tx=0x7f2390006e80).stop 2020-11-28T21:56:04.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.166+0000 7f23a7165700 1 -- 172.21.15.162:0/2248343285 shutdown_connections 2020-11-28T21:56:04.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.166+0000 7f23a7165700 1 --2- 172.21.15.162:0/2248343285 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23a00acb40 0x7f23a005f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:04.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.166+0000 7f23a7165700 1 --2- 172.21.15.162:0/2248343285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23a00b2e80 0x7f23a005f380 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:04.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.166+0000 7f23a7165700 1 -- 172.21.15.162:0/2248343285 >> 172.21.15.162:0/2248343285 conn(0x7f23a0063b60 msgr2=0x7f23a0063f40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:04.176 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.166+0000 7f23a7165700 1 -- 172.21.15.162:0/2248343285 shutdown_connections 2020-11-28T21:56:04.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.166+0000 7f23a7165700 1 -- 172.21.15.162:0/2248343285 wait complete. 2020-11-28T21:56:04.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.166+0000 7f23a7165700 1 Processor -- start 2020-11-28T21:56:04.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.166+0000 7f23a7165700 1 -- start start 2020-11-28T21:56:04.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.166+0000 7f23a7165700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23a00acb40 0x7f23a00661d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:04.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.166+0000 7f23a7165700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f23a00b2560 0x7f23a0066e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:04.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.166+0000 7f23a7165700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23a00b2e80 0x7f23a009ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:04.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.166+0000 7f23a7165700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a0137060 con 0x7f23a00b2560 2020-11-28T21:56:04.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.166+0000 7f23a7165700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a0065c40 con 0x7f23a00acb40 2020-11-28T21:56:04.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.166+0000 7f23a7165700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a0065700 con 0x7f23a00b2e80 2020-11-28T21:56:04.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.170+0000 7f23a6964700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23a00acb40 0x7f23a00661d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:04.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.170+0000 7f23a6964700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23a00acb40 0x7f23a00661d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45736/0 (socket says 172.21.15.162:45736) 2020-11-28T21:56:04.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.170+0000 7f23a6964700 1 -- 172.21.15.162:0/2100726834 learned_addr learned my addr 172.21.15.162:0/2100726834 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:04.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.170+0000 7f23a83c7700 1 --2- 172.21.15.162:0/2100726834 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23a00b2e80 0x7f23a009ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:04.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.170+0000 7f23a6163700 1 --2- 172.21.15.162:0/2100726834 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f23a00b2560 0x7f23a0066e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:04.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.170+0000 7f23a6964700 1 -- 172.21.15.162:0/2100726834 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23a00b2e80 msgr2=0x7f23a009ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:04.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.170+0000 7f23a6964700 1 --2- 172.21.15.162:0/2100726834 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23a00b2e80 0x7f23a009ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:04.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.170+0000 7f23a6964700 1 -- 172.21.15.162:0/2100726834 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f23a00b2560 msgr2=0x7f23a0066e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:04.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.170+0000 7f23a6964700 1 --2- 172.21.15.162:0/2100726834 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f23a00b2560 0x7f23a0066e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:04.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.170+0000 7f23a6964700 1 -- 172.21.15.162:0/2100726834 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a00659a0 con 0x7f23a00acb40 2020-11-28T21:56:04.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.170+0000 7f23a6964700 1 --2- 172.21.15.162:0/2100726834 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23a00acb40 0x7f23a00661d0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 rx=0x7f2390002f30 tx=0x7f2390015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:04.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.170+0000 7f23977fe700 1 -- 172.21.15.162:0/2100726834 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2390011b40 con 0x7f23a00acb40 2020-11-28T21:56:04.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.170+0000 7f23977fe700 1 -- 172.21.15.162:0/2100726834 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2390011ca0 con 0x7f23a00acb40 2020-11-28T21:56:04.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.170+0000 7f23a7165700 1 -- 172.21.15.162:0/2100726834 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23a00649c0 con 0x7f23a00acb40 2020-11-28T21:56:04.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.170+0000 7f23977fe700 1 -- 172.21.15.162:0/2100726834 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f239001e460 con 0x7f23a00acb40 2020-11-28T21:56:04.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.170+0000 7f23a7165700 1 -- 172.21.15.162:0/2100726834 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23a0067340 con 0x7f23a00acb40 2020-11-28T21:56:04.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.170+0000 7f23a7165700 1 -- 172.21.15.162:0/2100726834 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23a01376e0 con 0x7f23a00acb40 2020-11-28T21:56:04.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.174+0000 7f23977fe700 1 -- 172.21.15.162:0/2100726834 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2390025030 con 0x7f23a00acb40 2020-11-28T21:56:04.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.174+0000 7f23977fe700 1 --2- 172.21.15.162:0/2100726834 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2388055440 0x7f2388057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:04.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.174+0000 7f23977fe700 1 -- 172.21.15.162:0/2100726834 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f239006f570 con 0x7f23a00acb40 2020-11-28T21:56:04.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.174+0000 7f23a6163700 1 --2- 172.21.15.162:0/2100726834 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2388055440 0x7f2388057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:04.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.174+0000 7f23a6163700 1 --2- 172.21.15.162:0/2100726834 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2388055440 0x7f2388057900 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7f239c00d000 tx=0x7f239c000b80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:04.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.178+0000 7f23977fe700 1 -- 172.21.15.162:0/2100726834 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f2390045870 con 0x7f23a00acb40 2020-11-28T21:56:04.372 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.366+0000 7f23a7165700 1 -- 172.21.15.162:0/2100726834 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f23a01376e0 con 0x7f23a00acb40 2020-11-28T21:56:04.382 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.378+0000 7f23977fe700 1 -- 172.21.15.162:0/2100726834 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v32) v1 ==== 184+0+0 (secure 0 0 0) 0x7f23a01376e0 con 0x7f23a00acb40 2020-11-28T21:56:04.384 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.378+0000 7f23a7165700 1 -- 172.21.15.162:0/2100726834 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2388055440 msgr2=0x7f2388057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:04.384 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.378+0000 7f23a7165700 1 --2- 172.21.15.162:0/2100726834 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2388055440 0x7f2388057900 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7f239c00d000 tx=0x7f239c000b80).stop 2020-11-28T21:56:04.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.378+0000 7f23a7165700 1 -- 172.21.15.162:0/2100726834 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23a00acb40 msgr2=0x7f23a00661d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:04.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.378+0000 7f23a7165700 1 --2- 172.21.15.162:0/2100726834 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23a00acb40 0x7f23a00661d0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 rx=0x7f2390002f30 tx=0x7f2390015000).stop 2020-11-28T21:56:04.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.378+0000 7f23a7165700 1 -- 172.21.15.162:0/2100726834 shutdown_connections 2020-11-28T21:56:04.386 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.378+0000 7f23a7165700 1 --2- 172.21.15.162:0/2100726834 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2388055440 0x7f2388057900 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:04.386 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.378+0000 7f23a7165700 1 --2- 172.21.15.162:0/2100726834 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23a00b2e80 0x7f23a009ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:04.386 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.378+0000 7f23a7165700 1 --2- 172.21.15.162:0/2100726834 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23a00acb40 0x7f23a00661d0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:04.386 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.378+0000 7f23a7165700 1 --2- 172.21.15.162:0/2100726834 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f23a00b2560 0x7f23a0066e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:04.387 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.378+0000 7f23a7165700 1 -- 172.21.15.162:0/2100726834 >> 172.21.15.162:0/2100726834 conn(0x7f23a0063b60 msgr2=0x7f23a00b4150 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:04.387 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.378+0000 7f23a7165700 1 -- 172.21.15.162:0/2100726834 shutdown_connections 2020-11-28T21:56:04.387 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.378+0000 7f23a7165700 1 -- 172.21.15.162:0/2100726834 wait complete. 2020-11-28T21:56:04.387 INFO:teuthology.orchestra.run.smithi162.stderr:updated caps for client.1 2020-11-28T21:56:04.399 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:56:04.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.506+0000 7f3c6cde0700 1 Processor -- start 2020-11-28T21:56:04.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.506+0000 7f3c6cde0700 1 -- start start 2020-11-28T21:56:04.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.506+0000 7f3c6cde0700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3c680b4c70 0x7f3c680b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:04.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.506+0000 7f3c6cde0700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c680b4110 con 0x7f3c68142a00 2020-11-28T21:56:04.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.506+0000 7f3c6cde0700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c680b43f0 con 0x7f3c680603d0 2020-11-28T21:56:04.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.506+0000 7f3c6cde0700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c68060f60 con 0x7f3c680b4c70 2020-11-28T21:56:04.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.506+0000 7f3c677fe700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f3c680603d0 0x7f3c6809cfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:52588/0 (socket says 172.21.15.162:52588) 2020-11-28T21:56:04.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.506+0000 7f3c677fe700 1 -- 172.21.15.162:0/3174602394 learned_addr learned my addr 172.21.15.162:0/3174602394 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:56:04.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.506+0000 7f3c67fff700 1 --2- 172.21.15.162:0/3174602394 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3c680b4c70 0x7f3c680b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:04.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.506+0000 7f3c667fc700 1 -- 172.21.15.162:0/3174602394 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 891106429 0 0) 0x7f3c680b43f0 con 0x7f3c680603d0 2020-11-28T21:56:04.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.506+0000 7f3c667fc700 1 -- 172.21.15.162:0/3174602394 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c54003230 con 0x7f3c680603d0 2020-11-28T21:56:04.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.506+0000 7f3c667fc700 1 -- 172.21.15.162:0/3174602394 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1942320660 0 0) 0x7f3c680b4110 con 0x7f3c68142a00 2020-11-28T21:56:04.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.506+0000 7f3c667fc700 1 -- 172.21.15.162:0/3174602394 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c680b43f0 con 0x7f3c68142a00 2020-11-28T21:56:04.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.506+0000 7f3c67fff700 1 -- 172.21.15.162:0/3174602394 >> v1:172.21.15.162:6789/0 conn(0x7f3c680603d0 legacy=0x7f3c6809cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:04.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.506+0000 7f3c67fff700 1 -- 172.21.15.162:0/3174602394 >> v1:172.21.15.118:6789/0 conn(0x7f3c68142a00 legacy=0x7f3c68143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:04.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.506+0000 7f3c67fff700 1 -- 172.21.15.162:0/3174602394 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c680b4690 con 0x7f3c680b4c70 2020-11-28T21:56:04.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.510+0000 7f3c67fff700 1 --2- 172.21.15.162:0/3174602394 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3c680b4c70 0x7f3c680b3a70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f3c600032b0 tx=0x7f3c600091a0).ready entity=mon.2 client_cookie=77eab85bc63221d8 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:04.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.510+0000 7f3c667fc700 1 -- 172.21.15.162:0/3174602394 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c6000b940 con 0x7f3c680b4c70 2020-11-28T21:56:04.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.510+0000 7f3c667fc700 1 -- 172.21.15.162:0/3174602394 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c6000baa0 con 0x7f3c680b4c70 2020-11-28T21:56:04.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.510+0000 7f3c667fc700 1 -- 172.21.15.162:0/3174602394 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c6000bc10 con 0x7f3c680b4c70 2020-11-28T21:56:04.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.510+0000 7f3c6cde0700 1 -- 172.21.15.162:0/3174602394 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3c680b4c70 msgr2=0x7f3c680b3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:04.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.510+0000 7f3c6cde0700 1 --2- 172.21.15.162:0/3174602394 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3c680b4c70 0x7f3c680b3a70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f3c600032b0 tx=0x7f3c600091a0).stop 2020-11-28T21:56:04.521 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.510+0000 7f3c6cde0700 1 -- 172.21.15.162:0/3174602394 shutdown_connections 2020-11-28T21:56:04.521 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.510+0000 7f3c6cde0700 1 --2- 172.21.15.162:0/3174602394 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3c680b4c70 0x7f3c680b3a70 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:04.521 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.510+0000 7f3c6cde0700 1 -- 172.21.15.162:0/3174602394 >> 172.21.15.162:0/3174602394 conn(0x7f3c68066a40 msgr2=0x7f3c68066e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:04.521 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.510+0000 7f3c6cde0700 1 -- 172.21.15.162:0/3174602394 shutdown_connections 2020-11-28T21:56:04.522 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.510+0000 7f3c6cde0700 1 -- 172.21.15.162:0/3174602394 wait complete. 2020-11-28T21:56:04.522 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.510+0000 7f3c6cde0700 1 Processor -- start 2020-11-28T21:56:04.522 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.510+0000 7f3c6cde0700 1 -- start start 2020-11-28T21:56:04.522 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.514+0000 7f3c6cde0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c680603d0 0x7f3c680b7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:04.522 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.514+0000 7f3c6cde0700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3c680b4c70 0x7f3c680b2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:04.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.514+0000 7f3c6cde0700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3c680b8af0 0x7f3c680b2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:04.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.514+0000 7f3c6cde0700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c68151570 con 0x7f3c680b4c70 2020-11-28T21:56:04.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.514+0000 7f3c6cde0700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c68151810 con 0x7f3c680603d0 2020-11-28T21:56:04.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.514+0000 7f3c6cde0700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c68151ab0 con 0x7f3c680b8af0 2020-11-28T21:56:04.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.514+0000 7f3c677fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c680603d0 0x7f3c680b7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:04.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.514+0000 7f3c677fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c680603d0 0x7f3c680b7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45750/0 (socket says 172.21.15.162:45750) 2020-11-28T21:56:04.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.514+0000 7f3c677fe700 1 -- 172.21.15.162:0/961718928 learned_addr learned my addr 172.21.15.162:0/961718928 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:04.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.514+0000 7f3c66ffd700 1 --2- 172.21.15.162:0/961718928 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3c680b4c70 0x7f3c680b2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:04.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.514+0000 7f3c67fff700 1 --2- 172.21.15.162:0/961718928 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3c680b8af0 0x7f3c680b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:04.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.514+0000 7f3c677fe700 1 -- 172.21.15.162:0/961718928 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3c680b8af0 msgr2=0x7f3c680b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:04.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.514+0000 7f3c677fe700 1 --2- 172.21.15.162:0/961718928 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3c680b8af0 0x7f3c680b2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:04.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.514+0000 7f3c677fe700 1 -- 172.21.15.162:0/961718928 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3c680b4c70 msgr2=0x7f3c680b2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:04.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.514+0000 7f3c677fe700 1 --2- 172.21.15.162:0/961718928 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3c680b4c70 0x7f3c680b2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:04.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.514+0000 7f3c677fe700 1 -- 172.21.15.162:0/961718928 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c68151d50 con 0x7f3c680603d0 2020-11-28T21:56:04.526 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.514+0000 7f3c677fe700 1 --2- 172.21.15.162:0/961718928 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c680603d0 0x7f3c680b7e00 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 rx=0x7f3c58001110 tx=0x7f3c58007a50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:04.526 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.514+0000 7f3c64ff9700 1 -- 172.21.15.162:0/961718928 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c580072e0 con 0x7f3c680603d0 2020-11-28T21:56:04.526 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.514+0000 7f3c64ff9700 1 -- 172.21.15.162:0/961718928 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c58007440 con 0x7f3c680603d0 2020-11-28T21:56:04.526 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.514+0000 7f3c6cde0700 1 -- 172.21.15.162:0/961718928 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c6814f1b0 con 0x7f3c680603d0 2020-11-28T21:56:04.526 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.514+0000 7f3c64ff9700 1 -- 172.21.15.162:0/961718928 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c580057d0 con 0x7f3c680603d0 2020-11-28T21:56:04.527 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.514+0000 7f3c6cde0700 1 -- 172.21.15.162:0/961718928 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c68145890 con 0x7f3c680603d0 2020-11-28T21:56:04.527 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.514+0000 7f3c6cde0700 1 -- 172.21.15.162:0/961718928 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c68057050 con 0x7f3c680603d0 2020-11-28T21:56:04.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.518+0000 7f3c64ff9700 1 -- 172.21.15.162:0/961718928 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f3c58010070 con 0x7f3c680603d0 2020-11-28T21:56:04.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.518+0000 7f3c64ff9700 1 --2- 172.21.15.162:0/961718928 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3c480554e0 0x7f3c480579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:04.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.518+0000 7f3c64ff9700 1 -- 172.21.15.162:0/961718928 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3c5806d5b0 con 0x7f3c680603d0 2020-11-28T21:56:04.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.518+0000 7f3c66ffd700 1 --2- 172.21.15.162:0/961718928 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3c480554e0 0x7f3c480579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:04.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.518+0000 7f3c66ffd700 1 --2- 172.21.15.162:0/961718928 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3c480554e0 0x7f3c480579a0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7f3c5c00d000 tx=0x7f3c5c007a80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:04.529 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.522+0000 7f3c64ff9700 1 -- 172.21.15.162:0/961718928 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f3c58043810 con 0x7f3c680603d0 2020-11-28T21:56:04.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.706+0000 7f3c6cde0700 1 -- 172.21.15.162:0/961718928 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c68069b60 con 0x7f3c680603d0 2020-11-28T21:56:04.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.710+0000 7f3c64ff9700 1 -- 172.21.15.162:0/961718928 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+2504 (secure 0 0 0) 0x7f3c58018020 con 0x7f3c680603d0 2020-11-28T21:56:04.714 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:56:04.715 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":74,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6899,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.118:6833/2509028245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":2509028245},{"type":"v1","addr":"172.21.15.118:6833","nonce":2509028245}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":72},{"gid":8239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/2089791609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2089791609},{"type":"v1","addr":"172.21.15.162:6836","nonce":2089791609}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":71}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:56:01.362652+0000","modified":"2020-11-28T21:56:03.027360+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8242},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8242":{"gid":8242,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/2832109513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":2832109513},{"type":"v1","addr":"172.21.15.162:6837","nonce":2832109513}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2020-11-28T21:56:04.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.710+0000 7f3c6cde0700 1 -- 172.21.15.162:0/961718928 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3c480554e0 msgr2=0x7f3c480579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:04.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.710+0000 7f3c6cde0700 1 --2- 172.21.15.162:0/961718928 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3c480554e0 0x7f3c480579a0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7f3c5c00d000 tx=0x7f3c5c007a80).stop 2020-11-28T21:56:04.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.710+0000 7f3c6cde0700 1 -- 172.21.15.162:0/961718928 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c680603d0 msgr2=0x7f3c680b7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:04.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.710+0000 7f3c6cde0700 1 --2- 172.21.15.162:0/961718928 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c680603d0 0x7f3c680b7e00 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 rx=0x7f3c58001110 tx=0x7f3c58007a50).stop 2020-11-28T21:56:04.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.710+0000 7f3c6cde0700 1 -- 172.21.15.162:0/961718928 shutdown_connections 2020-11-28T21:56:04.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.710+0000 7f3c6cde0700 1 --2- 172.21.15.162:0/961718928 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3c480554e0 0x7f3c480579a0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:04.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.710+0000 7f3c6cde0700 1 --2- 172.21.15.162:0/961718928 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3c680b8af0 0x7f3c680b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:04.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.710+0000 7f3c6cde0700 1 --2- 172.21.15.162:0/961718928 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c680603d0 0x7f3c680b7e00 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:04.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.710+0000 7f3c6cde0700 1 --2- 172.21.15.162:0/961718928 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3c680b4c70 0x7f3c680b2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:04.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.710+0000 7f3c6cde0700 1 -- 172.21.15.162:0/961718928 >> 172.21.15.162:0/961718928 conn(0x7f3c68066a40 msgr2=0x7f3c681407a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:04.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.710+0000 7f3c6cde0700 1 -- 172.21.15.162:0/961718928 shutdown_connections 2020-11-28T21:56:04.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:04.710+0000 7f3c6cde0700 1 -- 172.21.15.162:0/961718928 wait complete. 2020-11-28T21:56:04.720 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 74 2020-11-28T21:56:05.032 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-28T21:56:01.362652+0000', 'modified': '2020-11-28T21:56:03.027360+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8242}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8242': {'gid': 8242, 'name': 'a', 'rank': 0, 'incarnation': 72, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6837/2832109513', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6835', 'nonce': 2832109513}, {'type': 'v1', 'addr': '172.21.15.162:6837', 'nonce': 2832109513}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-28T21:56:05.033 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-28T21:56:05.034 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-28T21:56:05.177 INFO:teuthology.orchestra.run.smithi162.stdout:{"cluster_fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","whoami":0,"id":8242,"want_state":"up:active","state":"up:active","rank_uptime":3.8004204270000002,"mdsmap_epoch":74,"osdmap_epoch":148,"osdmap_epoch_barrier":148,"uptime":8.0970826670000005} 2020-11-28T21:56:05.177 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", "id": 8242, "mdsmap_epoch": 74, "osdmap_epoch": 148, "osdmap_epoch_barrier": 148, "rank_uptime": 3.800420427, "state": "up:active", "uptime": 8.097082667, "want_state": "up:active", "whoami": 0 } 2020-11-28T21:56:05.178 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:56:05.291 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.286+0000 7fe60f59e700 1 Processor -- start 2020-11-28T21:56:05.291 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.286+0000 7fe60f59e700 1 -- start start 2020-11-28T21:56:05.292 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.286+0000 7fe60f59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe610062ac0 0x7fe6100b33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:05.292 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.286+0000 7fe60f59e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe610066ff0 0x7fe610061590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:05.293 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.286+0000 7fe60f59e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe610143c90 con 0x7fe6100b7780 2020-11-28T21:56:05.293 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.286+0000 7fe60f59e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe610144b40 con 0x7fe610066ff0 2020-11-28T21:56:05.293 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.286+0000 7fe60f59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe610143ec0 con 0x7fe610062ac0 2020-11-28T21:56:05.293 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.286+0000 7fe60e59c700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fe6100b7780 0x7fe6100b2cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:52602/0 (socket says 172.21.15.162:52602) 2020-11-28T21:56:05.294 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.286+0000 7fe60e59c700 1 -- 172.21.15.162:0/504804160 learned_addr learned my addr 172.21.15.162:0/504804160 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:56:05.294 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.286+0000 7fe60ed9d700 1 --2- 172.21.15.162:0/504804160 >> v2:172.21.15.118:3300/0 conn(0x7fe610066ff0 0x7fe610061590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:05.294 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.286+0000 7fe60d59a700 1 -- 172.21.15.162:0/504804160 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 946514407 0 0) 0x7fe610143c90 con 0x7fe6100b7780 2020-11-28T21:56:05.294 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.286+0000 7fe60dd9b700 1 --2- 172.21.15.162:0/504804160 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe610062ac0 0x7fe6100b33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:05.294 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.286+0000 7fe60d59a700 1 -- 172.21.15.162:0/504804160 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5f4003230 con 0x7fe6100b7780 2020-11-28T21:56:05.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.286+0000 7fe60d59a700 1 -- 172.21.15.162:0/504804160 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1369437561 0 0) 0x7fe5f4003230 con 0x7fe6100b7780 2020-11-28T21:56:05.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.286+0000 7fe60d59a700 1 -- 172.21.15.162:0/504804160 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe610062ac0 msgr2=0x7fe6100b33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:05.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.286+0000 7fe60d59a700 1 --2- 172.21.15.162:0/504804160 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe610062ac0 0x7fe6100b33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:05.298 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.286+0000 7fe60d59a700 1 -- 172.21.15.162:0/504804160 >> v2:172.21.15.118:3300/0 conn(0x7fe610066ff0 msgr2=0x7fe610061590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:05.298 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.286+0000 7fe60d59a700 1 --2- 172.21.15.162:0/504804160 >> v2:172.21.15.118:3300/0 conn(0x7fe610066ff0 0x7fe610061590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:05.298 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.286+0000 7fe60d59a700 1 -- 172.21.15.162:0/504804160 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe610144160 con 0x7fe6100b7780 2020-11-28T21:56:05.298 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.286+0000 7fe60d59a700 1 -- 172.21.15.162:0/504804160 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7fe6000016d0 con 0x7fe6100b7780 2020-11-28T21:56:05.298 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.286+0000 7fe60dd9b700 1 --2- 172.21.15.162:0/504804160 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe610062ac0 0x7fe6100b33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:56:05.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.286+0000 7fe60d59a700 1 -- 172.21.15.162:0/504804160 >> v1:172.21.15.162:6789/0 conn(0x7fe6100b7780 legacy=0x7fe6100b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:05.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.286+0000 7fe60d59a700 1 --2- 172.21.15.162:0/504804160 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe5f4003720 0x7fe5f4003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:05.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.286+0000 7fe60d59a700 1 --2- 172.21.15.162:0/504804160 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe5f40044d0 0x7fe5f4004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:05.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.286+0000 7fe60d59a700 1 --2- 172.21.15.162:0/504804160 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5f4004e40 0x7fe5f400b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:05.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.286+0000 7fe60d59a700 1 -- 172.21.15.162:0/504804160 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe610144b40 con 0x7fe5f4003720 2020-11-28T21:56:05.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.286+0000 7fe60d59a700 1 -- 172.21.15.162:0/504804160 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe610143ec0 con 0x7fe5f4004e40 2020-11-28T21:56:05.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.286+0000 7fe60d59a700 1 -- 172.21.15.162:0/504804160 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f40041b0 con 0x7fe5f40044d0 2020-11-28T21:56:05.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.290+0000 7fe60dd9b700 1 --2- 172.21.15.162:0/504804160 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe5f40044d0 0x7fe5f4004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:05.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.290+0000 7fe60ed9d700 1 --2- 172.21.15.162:0/504804160 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5f4004e40 0x7fe5f400b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:05.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.290+0000 7fe60e59c700 1 --2- 172.21.15.162:0/504804160 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe5f4003720 0x7fe5f4003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:05.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.290+0000 7fe60ed9d700 1 -- 172.21.15.162:0/504804160 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe5f40044d0 msgr2=0x7fe5f4004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:05.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.290+0000 7fe60ed9d700 1 --2- 172.21.15.162:0/504804160 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe5f40044d0 0x7fe5f4004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:05.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.290+0000 7fe60ed9d700 1 -- 172.21.15.162:0/504804160 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe5f4003720 msgr2=0x7fe5f4003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:05.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.290+0000 7fe60ed9d700 1 --2- 172.21.15.162:0/504804160 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe5f4003720 0x7fe5f4003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:05.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.290+0000 7fe60ed9d700 1 -- 172.21.15.162:0/504804160 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f4003230 con 0x7fe5f4004e40 2020-11-28T21:56:05.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.290+0000 7fe60e59c700 1 --2- 172.21.15.162:0/504804160 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe5f4003720 0x7fe5f4003b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:56:05.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.290+0000 7fe60ed9d700 1 --2- 172.21.15.162:0/504804160 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5f4004e40 0x7fe5f400b720 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 rx=0x7fe60800c000 tx=0x7fe6080091a0).ready entity=mon.1 client_cookie=6c324587801e26e5 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:05.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.290+0000 7fe60d59a700 1 -- 172.21.15.162:0/504804160 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe60800ec30 con 0x7fe5f4004e40 2020-11-28T21:56:05.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.290+0000 7fe60d59a700 1 -- 172.21.15.162:0/504804160 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe60800ed90 con 0x7fe5f4004e40 2020-11-28T21:56:05.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.290+0000 7fe60d59a700 1 -- 172.21.15.162:0/504804160 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6080125e0 con 0x7fe5f4004e40 2020-11-28T21:56:05.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.290+0000 7fe60f59e700 1 -- 172.21.15.162:0/504804160 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5f4004e40 msgr2=0x7fe5f400b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:05.303 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.290+0000 7fe60f59e700 1 --2- 172.21.15.162:0/504804160 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5f4004e40 0x7fe5f400b720 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 rx=0x7fe60800c000 tx=0x7fe6080091a0).stop 2020-11-28T21:56:05.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.290+0000 7fe60ed9d700 1 -- 172.21.15.162:0/504804160 reap_dead start 2020-11-28T21:56:05.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.290+0000 7fe60f59e700 1 -- 172.21.15.162:0/504804160 shutdown_connections 2020-11-28T21:56:05.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.290+0000 7fe60f59e700 1 -- 172.21.15.162:0/504804160 >> 172.21.15.162:0/504804160 conn(0x7fe61005c440 msgr2=0x7fe6100b6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:05.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.290+0000 7fe60f59e700 1 -- 172.21.15.162:0/504804160 shutdown_connections 2020-11-28T21:56:05.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.290+0000 7fe60f59e700 1 -- 172.21.15.162:0/504804160 wait complete. 2020-11-28T21:56:05.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.290+0000 7fe60f59e700 1 Processor -- start 2020-11-28T21:56:05.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.290+0000 7fe60f59e700 1 -- start start 2020-11-28T21:56:05.307 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.294+0000 7fe60f59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe610146110 0x7fe6100ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:05.307 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.294+0000 7fe60f59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe6101464f0 0x7fe610060310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:05.307 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.294+0000 7fe60f59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe610060800 0x7fe610060e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:05.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.294+0000 7fe60f59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6100ac650 con 0x7fe6101464f0 2020-11-28T21:56:05.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.294+0000 7fe60f59e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6100aba70 con 0x7fe610060800 2020-11-28T21:56:05.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.294+0000 7fe60f59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6100ad490 con 0x7fe610146110 2020-11-28T21:56:05.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.294+0000 7fe60ed9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe610060800 0x7fe610060e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:05.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.294+0000 7fe60ed9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe610060800 0x7fe610060e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45772/0 (socket says 172.21.15.162:45772) 2020-11-28T21:56:05.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.294+0000 7fe60ed9d700 1 -- 172.21.15.162:0/920328120 learned_addr learned my addr 172.21.15.162:0/920328120 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:05.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.294+0000 7fe60dd9b700 1 --2- 172.21.15.162:0/920328120 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe6101464f0 0x7fe610060310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:05.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.294+0000 7fe60e59c700 1 --2- 172.21.15.162:0/920328120 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe610146110 0x7fe6100ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:05.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.294+0000 7fe60ed9d700 1 -- 172.21.15.162:0/920328120 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe610146110 msgr2=0x7fe6100ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:05.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.294+0000 7fe60ed9d700 1 --2- 172.21.15.162:0/920328120 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe610146110 0x7fe6100ad8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:05.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.294+0000 7fe60ed9d700 1 -- 172.21.15.162:0/920328120 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe6101464f0 msgr2=0x7fe610060310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:05.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.294+0000 7fe60ed9d700 1 --2- 172.21.15.162:0/920328120 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe6101464f0 0x7fe610060310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:05.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.294+0000 7fe60ed9d700 1 -- 172.21.15.162:0/920328120 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6100ac880 con 0x7fe610060800 2020-11-28T21:56:05.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.294+0000 7fe60ed9d700 1 --2- 172.21.15.162:0/920328120 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe610060800 0x7fe610060e70 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 rx=0x7fe610060c00 tx=0x7fe60801a000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:05.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.294+0000 7fe5ff7fe700 1 -- 172.21.15.162:0/920328120 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe608025630 con 0x7fe610060800 2020-11-28T21:56:05.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.294+0000 7fe5ff7fe700 1 -- 172.21.15.162:0/920328120 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6080095c0 con 0x7fe610060800 2020-11-28T21:56:05.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.294+0000 7fe60f59e700 1 -- 172.21.15.162:0/920328120 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6100ab7d0 con 0x7fe610060800 2020-11-28T21:56:05.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.294+0000 7fe5ff7fe700 1 -- 172.21.15.162:0/920328120 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe608025980 con 0x7fe610060800 2020-11-28T21:56:05.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.294+0000 7fe60f59e700 1 -- 172.21.15.162:0/920328120 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe610061eb0 con 0x7fe610060800 2020-11-28T21:56:05.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.294+0000 7fe5ff7fe700 1 -- 172.21.15.162:0/920328120 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe6080123f0 con 0x7fe610060800 2020-11-28T21:56:05.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.294+0000 7fe60f59e700 1 -- 172.21.15.162:0/920328120 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe610057050 con 0x7fe610060800 2020-11-28T21:56:05.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.294+0000 7fe5ff7fe700 1 --2- 172.21.15.162:0/920328120 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe5ec055470 0x7fe5ec057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:05.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.298+0000 7fe5ff7fe700 1 -- 172.21.15.162:0/920328120 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe60806fea0 con 0x7fe610060800 2020-11-28T21:56:05.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.298+0000 7fe60e59c700 1 --2- 172.21.15.162:0/920328120 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe5ec055470 0x7fe5ec057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:05.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.298+0000 7fe60e59c700 1 --2- 172.21.15.162:0/920328120 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe5ec055470 0x7fe5ec057930 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7fe60000d000 tx=0x7fe60000c040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:05.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.302+0000 7fe5ff7fe700 1 -- 172.21.15.162:0/920328120 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe608046140 con 0x7fe610060800 2020-11-28T21:56:05.447 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.442+0000 7fe60f59e700 1 -- 172.21.15.162:0/920328120 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6100b31c0 con 0x7fe610060800 2020-11-28T21:56:05.448 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.442+0000 7fe5ff7fe700 1 -- 172.21.15.162:0/920328120 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+16148 (secure 0 0 0) 0x7fe608016020 con 0x7fe610060800 2020-11-28T21:56:05.449 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:56:05.450 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":148,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:56:01.362358+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:55:59.967971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2020-11-28T21:56:00.379272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:56:05.452 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.446+0000 7fe60f59e700 1 -- 172.21.15.162:0/920328120 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe5ec055470 msgr2=0x7fe5ec057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:05.452 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.446+0000 7fe60f59e700 1 --2- 172.21.15.162:0/920328120 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe5ec055470 0x7fe5ec057930 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7fe60000d000 tx=0x7fe60000c040).stop 2020-11-28T21:56:05.452 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.446+0000 7fe60f59e700 1 -- 172.21.15.162:0/920328120 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe610060800 msgr2=0x7fe610060e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:05.453 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.446+0000 7fe60f59e700 1 --2- 172.21.15.162:0/920328120 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe610060800 0x7fe610060e70 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 rx=0x7fe610060c00 tx=0x7fe60801a000).stop 2020-11-28T21:56:05.453 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.446+0000 7fe60f59e700 1 -- 172.21.15.162:0/920328120 shutdown_connections 2020-11-28T21:56:05.454 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.446+0000 7fe60f59e700 1 --2- 172.21.15.162:0/920328120 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe5ec055470 0x7fe5ec057930 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:05.454 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.446+0000 7fe60f59e700 1 --2- 172.21.15.162:0/920328120 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe610146110 0x7fe6100ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:05.454 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.446+0000 7fe60f59e700 1 --2- 172.21.15.162:0/920328120 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe610060800 0x7fe610060e70 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:05.455 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.446+0000 7fe60f59e700 1 --2- 172.21.15.162:0/920328120 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe6101464f0 0x7fe610060310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:05.455 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.446+0000 7fe60f59e700 1 -- 172.21.15.162:0/920328120 >> 172.21.15.162:0/920328120 conn(0x7fe61005c440 msgr2=0x7fe6100b6450 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:05.456 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.446+0000 7fe60f59e700 1 -- 172.21.15.162:0/920328120 shutdown_connections 2020-11-28T21:56:05.456 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.446+0000 7fe60f59e700 1 -- 172.21.15.162:0/920328120 wait complete. 2020-11-28T21:56:05.463 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-28T21:55:59.967971+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '148', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2020-11-28T21:56:00.379272+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '148', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-28T21:56:05.464 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:56:05.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.578+0000 7f3b06b6a700 1 Processor -- start 2020-11-28T21:56:05.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.578+0000 7f3b06b6a700 1 -- start start 2020-11-28T21:56:05.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.578+0000 7f3b06b6a700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f3b000b21f0 0x7f3b001434f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:05.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.578+0000 7f3b06b6a700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3b00141e80 0x7f3b000606f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:05.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.578+0000 7f3b06b6a700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b000af6a0 con 0x7f3b000ae3d0 2020-11-28T21:56:05.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.578+0000 7f3b06b6a700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b000b25d0 con 0x7f3b000b21f0 2020-11-28T21:56:05.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.578+0000 7f3b06b6a700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b0005fff0 con 0x7f3b00141e80 2020-11-28T21:56:05.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.578+0000 7f3b05367700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3b00141e80 0x7f3b000606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:05.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.578+0000 7f3b05367700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3b00141e80 0x7f3b000606f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45780/0 (socket says 172.21.15.162:45780) 2020-11-28T21:56:05.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.578+0000 7f3b05367700 1 -- 172.21.15.162:0/1784404596 learned_addr learned my addr 172.21.15.162:0/1784404596 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:05.585 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.578+0000 7f3b05b68700 1 --2- 172.21.15.162:0/1784404596 >> v2:172.21.15.118:3300/0 conn(0x7f3b000b21f0 0x7f3b001434f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:05.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.578+0000 7f3b05367700 1 -- 172.21.15.162:0/1784404596 >> v2:172.21.15.118:3300/0 conn(0x7f3b000b21f0 msgr2=0x7f3b001434f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:05.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.578+0000 7f3b05367700 1 --2- 172.21.15.162:0/1784404596 >> v2:172.21.15.118:3300/0 conn(0x7f3b000b21f0 0x7f3b001434f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:05.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.578+0000 7f3b05367700 1 -- 172.21.15.162:0/1784404596 >> v1:172.21.15.118:6789/0 conn(0x7f3b000ae3d0 legacy=0x7f3b00143f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:05.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.578+0000 7f3b05367700 1 -- 172.21.15.162:0/1784404596 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b000af8d0 con 0x7f3b00141e80 2020-11-28T21:56:05.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.578+0000 7f3b05367700 1 --2- 172.21.15.162:0/1784404596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b00141e80 0x7f3b000606f0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 rx=0x7f3af0002f30 tx=0x7f3af0006e80).ready entity=mon.1 client_cookie=76b1e53457b7e0ac server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:05.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.578+0000 7f3b04b66700 1 -- 172.21.15.162:0/1784404596 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3af00097f0 con 0x7f3b00141e80 2020-11-28T21:56:05.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.578+0000 7f3b04b66700 1 -- 172.21.15.162:0/1784404596 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af0009950 con 0x7f3b00141e80 2020-11-28T21:56:05.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.578+0000 7f3b06b6a700 1 -- 172.21.15.162:0/1784404596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b00141e80 msgr2=0x7f3b000606f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:05.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.578+0000 7f3b04b66700 1 -- 172.21.15.162:0/1784404596 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3af0009cb0 con 0x7f3b00141e80 2020-11-28T21:56:05.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.578+0000 7f3b06b6a700 1 --2- 172.21.15.162:0/1784404596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b00141e80 0x7f3b000606f0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 rx=0x7f3af0002f30 tx=0x7f3af0006e80).stop 2020-11-28T21:56:05.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.582+0000 7f3b06b6a700 1 -- 172.21.15.162:0/1784404596 shutdown_connections 2020-11-28T21:56:05.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.582+0000 7f3b06b6a700 1 --2- 172.21.15.162:0/1784404596 >> v2:172.21.15.118:3300/0 conn(0x7f3b000b21f0 0x7f3b001434f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:05.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.582+0000 7f3b06b6a700 1 --2- 172.21.15.162:0/1784404596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b00141e80 0x7f3b000606f0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:05.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.582+0000 7f3b06b6a700 1 -- 172.21.15.162:0/1784404596 >> 172.21.15.162:0/1784404596 conn(0x7f3b000b57d0 msgr2=0x7f3b000b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:05.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.582+0000 7f3b06b6a700 1 -- 172.21.15.162:0/1784404596 shutdown_connections 2020-11-28T21:56:05.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.582+0000 7f3b06b6a700 1 -- 172.21.15.162:0/1784404596 wait complete. 2020-11-28T21:56:05.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.582+0000 7f3b06b6a700 1 Processor -- start 2020-11-28T21:56:05.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.582+0000 7f3b06b6a700 1 -- start start 2020-11-28T21:56:05.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.582+0000 7f3b06b6a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b000ae3d0 0x7f3b00142c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:05.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.582+0000 7f3b06b6a700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3b000b21f0 0x7f3b00066270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:05.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.582+0000 7f3b06b6a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3b00141e80 0x7f3b0014f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:05.594 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.582+0000 7f3b06b6a700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b001446b0 con 0x7f3b00141e80 2020-11-28T21:56:05.594 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.582+0000 7f3b06b6a700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b001422b0 con 0x7f3b000ae3d0 2020-11-28T21:56:05.594 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.582+0000 7f3b06b6a700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b001448a0 con 0x7f3b000b21f0 2020-11-28T21:56:05.594 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.582+0000 7f3b05b68700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b000ae3d0 0x7f3b00142c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:05.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.582+0000 7f3b05b68700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b000ae3d0 0x7f3b00142c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45784/0 (socket says 172.21.15.162:45784) 2020-11-28T21:56:05.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.582+0000 7f3b05b68700 1 -- 172.21.15.162:0/853927212 learned_addr learned my addr 172.21.15.162:0/853927212 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:05.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.582+0000 7f3b05b68700 1 -- 172.21.15.162:0/853927212 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3b000b21f0 msgr2=0x7f3b00066270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:05.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.582+0000 7f3b05b68700 1 --2- 172.21.15.162:0/853927212 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3b000b21f0 0x7f3b00066270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-28T21:56:05.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.582+0000 7f3b05b68700 1 -- 172.21.15.162:0/853927212 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3b00141e80 msgr2=0x7f3b0014f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:05.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.582+0000 7f3b06369700 1 --2- 172.21.15.162:0/853927212 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3b00141e80 0x7f3b0014f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:05.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.586+0000 7f3b05b68700 1 --2- 172.21.15.162:0/853927212 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3b00141e80 0x7f3b0014f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:05.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.586+0000 7f3b05b68700 1 -- 172.21.15.162:0/853927212 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b001424e0 con 0x7f3b000ae3d0 2020-11-28T21:56:05.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.586+0000 7f3b06369700 1 --2- 172.21.15.162:0/853927212 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3b00141e80 0x7f3b0014f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:56:05.597 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.586+0000 7f3b05b68700 1 --2- 172.21.15.162:0/853927212 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b000ae3d0 0x7f3b00142c20 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 rx=0x7f3b00147580 tx=0x7f3afc006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:05.597 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.586+0000 7f3aeeffd700 1 -- 172.21.15.162:0/853927212 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3afc00ca80 con 0x7f3b000ae3d0 2020-11-28T21:56:05.597 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.586+0000 7f3aeeffd700 1 -- 172.21.15.162:0/853927212 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3afc00cbe0 con 0x7f3b000ae3d0 2020-11-28T21:56:05.597 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.586+0000 7f3aeeffd700 1 -- 172.21.15.162:0/853927212 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3afc0123f0 con 0x7f3b000ae3d0 2020-11-28T21:56:05.597 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.586+0000 7f3b06b6a700 1 -- 172.21.15.162:0/853927212 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b00142780 con 0x7f3b000ae3d0 2020-11-28T21:56:05.598 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.586+0000 7f3b06b6a700 1 -- 172.21.15.162:0/853927212 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b00143110 con 0x7f3b000ae3d0 2020-11-28T21:56:05.598 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.586+0000 7f3b06b6a700 1 -- 172.21.15.162:0/853927212 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b00142910 con 0x7f3b000ae3d0 2020-11-28T21:56:05.598 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.586+0000 7f3aeeffd700 1 -- 172.21.15.162:0/853927212 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f3afc012550 con 0x7f3b000ae3d0 2020-11-28T21:56:05.599 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.586+0000 7f3aeeffd700 1 --2- 172.21.15.162:0/853927212 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3ae00554e0 0x7f3ae00579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:05.599 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.586+0000 7f3b05367700 1 --2- 172.21.15.162:0/853927212 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3ae00554e0 0x7f3ae00579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:05.599 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.590+0000 7f3aeeffd700 1 -- 172.21.15.162:0/853927212 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3afc06e460 con 0x7f3b000ae3d0 2020-11-28T21:56:05.600 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.590+0000 7f3b05367700 1 --2- 172.21.15.162:0/853927212 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3ae00554e0 0x7f3ae00579a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7f3af0002f30 tx=0x7f3af0015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:05.600 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.594+0000 7f3aeeffd700 1 -- 172.21.15.162:0/853927212 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f3afc044700 con 0x7f3b000ae3d0 2020-11-28T21:56:05.741 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.738+0000 7f3b06b6a700 1 -- 172.21.15.162:0/853927212 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f3b00057050 con 0x7f3b000ae3d0 2020-11-28T21:56:05.742 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.738+0000 7f3aeeffd700 1 -- 172.21.15.162:0/853927212 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v148) v1 ==== 110+0+10 (secure 0 0 0) 0x7f3afc047d20 con 0x7f3b000ae3d0 2020-11-28T21:56:05.742 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:56:05.744 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.738+0000 7f3b06b6a700 1 -- 172.21.15.162:0/853927212 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3ae00554e0 msgr2=0x7f3ae00579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:05.744 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.738+0000 7f3b06b6a700 1 --2- 172.21.15.162:0/853927212 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3ae00554e0 0x7f3ae00579a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7f3af0002f30 tx=0x7f3af0015000).stop 2020-11-28T21:56:05.744 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.738+0000 7f3b06b6a700 1 -- 172.21.15.162:0/853927212 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b000ae3d0 msgr2=0x7f3b00142c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:05.744 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.738+0000 7f3b06b6a700 1 --2- 172.21.15.162:0/853927212 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b000ae3d0 0x7f3b00142c20 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 rx=0x7f3b00147580 tx=0x7f3afc006dc0).stop 2020-11-28T21:56:05.745 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.738+0000 7f3b06b6a700 1 -- 172.21.15.162:0/853927212 shutdown_connections 2020-11-28T21:56:05.745 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.738+0000 7f3b06b6a700 1 --2- 172.21.15.162:0/853927212 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3ae00554e0 0x7f3ae00579a0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:05.745 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.738+0000 7f3b06b6a700 1 --2- 172.21.15.162:0/853927212 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3b00141e80 0x7f3b0014f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:05.745 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.738+0000 7f3b06b6a700 1 --2- 172.21.15.162:0/853927212 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b000ae3d0 0x7f3b00142c20 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:05.746 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.738+0000 7f3b06b6a700 1 --2- 172.21.15.162:0/853927212 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3b000b21f0 0x7f3b00066270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-28T21:56:05.746 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.738+0000 7f3b06b6a700 1 -- 172.21.15.162:0/853927212 >> 172.21.15.162:0/853927212 conn(0x7f3b000b57d0 msgr2=0x7f3b00060610 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:05.746 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.738+0000 7f3b06b6a700 1 -- 172.21.15.162:0/853927212 shutdown_connections 2020-11-28T21:56:05.746 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.738+0000 7f3b06b6a700 1 -- 172.21.15.162:0/853927212 wait complete. 2020-11-28T21:56:05.762 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:56:05.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.870+0000 7f38f3140700 1 Processor -- start 2020-11-28T21:56:05.877 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.870+0000 7f38f3140700 1 -- start start 2020-11-28T21:56:05.881 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38f3140700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f38ec0b9ce0 0x7f38ec1433c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:05.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38f3140700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38ec143950 con 0x7f38ec067610 2020-11-28T21:56:05.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38f3140700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38ec067aa0 con 0x7f38ec0605c0 2020-11-28T21:56:05.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38f3140700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38ec0b4340 con 0x7f38ec0b9ce0 2020-11-28T21:56:05.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38ebfff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f38ec0605c0 0x7f38ec0b3c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:52636/0 (socket says 172.21.15.162:52636) 2020-11-28T21:56:05.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38ebfff700 1 -- 172.21.15.162:0/4035658920 learned_addr learned my addr 172.21.15.162:0/4035658920 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:56:05.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38f0edc700 1 --2- 172.21.15.162:0/4035658920 >> v2:172.21.15.118:3300/0 conn(0x7f38ec0b9ce0 0x7f38ec1433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:05.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38eb7fe700 1 -- 172.21.15.162:0/4035658920 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2211583937 0 0) 0x7f38ec067aa0 con 0x7f38ec0605c0 2020-11-28T21:56:05.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38eb7fe700 1 -- 172.21.15.162:0/4035658920 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38d8003230 con 0x7f38ec0605c0 2020-11-28T21:56:05.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38eb7fe700 1 -- 172.21.15.162:0/4035658920 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1162702251 0 0) 0x7f38d8003230 con 0x7f38ec0605c0 2020-11-28T21:56:05.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38eb7fe700 1 -- 172.21.15.162:0/4035658920 >> v2:172.21.15.118:3300/0 conn(0x7f38ec0b9ce0 msgr2=0x7f38ec1433c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:05.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38eb7fe700 1 --2- 172.21.15.162:0/4035658920 >> v2:172.21.15.118:3300/0 conn(0x7f38ec0b9ce0 0x7f38ec1433c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:05.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38eb7fe700 1 -- 172.21.15.162:0/4035658920 >> v1:172.21.15.118:6789/0 conn(0x7f38ec067610 legacy=0x7f38ec13fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:05.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38eb7fe700 1 -- 172.21.15.162:0/4035658920 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ec0ba0c0 con 0x7f38ec0605c0 2020-11-28T21:56:05.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38eb7fe700 1 -- 172.21.15.162:0/4035658920 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f38e00016d0 con 0x7f38ec0605c0 2020-11-28T21:56:05.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38eb7fe700 1 -- 172.21.15.162:0/4035658920 >> v1:172.21.15.162:6789/0 conn(0x7f38ec0605c0 legacy=0x7f38ec0b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:05.885 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38eb7fe700 1 --2- 172.21.15.162:0/4035658920 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38d8003720 0x7f38d8003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:05.886 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38eb7fe700 1 --2- 172.21.15.162:0/4035658920 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f38d80044d0 0x7f38d8004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:05.886 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38eb7fe700 1 --2- 172.21.15.162:0/4035658920 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f38d8004e40 0x7f38d800b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:05.886 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38eb7fe700 1 -- 172.21.15.162:0/4035658920 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec0b4340 con 0x7f38d80044d0 2020-11-28T21:56:05.886 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38eb7fe700 1 -- 172.21.15.162:0/4035658920 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d8004250 con 0x7f38d8003720 2020-11-28T21:56:05.886 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38eb7fe700 1 -- 172.21.15.162:0/4035658920 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d800bc10 con 0x7f38d8004e40 2020-11-28T21:56:05.887 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38f0edc700 1 --2- 172.21.15.162:0/4035658920 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38d8003720 0x7f38d8003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:05.887 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38ebfff700 1 --2- 172.21.15.162:0/4035658920 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f38d80044d0 0x7f38d8004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:05.887 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38f0edc700 1 -- 172.21.15.162:0/4035658920 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f38d8004e40 msgr2=0x7f38d800b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:05.887 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38f16dd700 1 --2- 172.21.15.162:0/4035658920 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f38d8004e40 0x7f38d800b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:05.888 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38f0edc700 1 --2- 172.21.15.162:0/4035658920 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f38d8004e40 0x7f38d800b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:05.888 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38f0edc700 1 -- 172.21.15.162:0/4035658920 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f38d80044d0 msgr2=0x7f38d8004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:05.888 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38f0edc700 1 --2- 172.21.15.162:0/4035658920 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f38d80044d0 0x7f38d8004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:05.888 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38f0edc700 1 -- 172.21.15.162:0/4035658920 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d8003230 con 0x7f38d8003720 2020-11-28T21:56:05.889 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38ebfff700 1 --2- 172.21.15.162:0/4035658920 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f38d80044d0 0x7f38d8004950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:56:05.889 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38f16dd700 1 --2- 172.21.15.162:0/4035658920 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f38d8004e40 0x7f38d800b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:56:05.889 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38f16dd700 1 -- 172.21.15.162:0/4035658920 reap_dead start 2020-11-28T21:56:05.889 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38f0edc700 1 --2- 172.21.15.162:0/4035658920 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38d8003720 0x7f38d8003b40 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 rx=0x7f38dc00c000 tx=0x7f38dc0091a0).ready entity=mon.1 client_cookie=d609b2bd55480f51 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:05.889 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38eb7fe700 1 -- 172.21.15.162:0/4035658920 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f38dc00ed00 con 0x7f38d8003720 2020-11-28T21:56:05.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38eb7fe700 1 -- 172.21.15.162:0/4035658920 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38dc00ee60 con 0x7f38d8003720 2020-11-28T21:56:05.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38eb7fe700 1 -- 172.21.15.162:0/4035658920 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f38dc0125e0 con 0x7f38d8003720 2020-11-28T21:56:05.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38f3140700 1 -- 172.21.15.162:0/4035658920 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38d8003720 msgr2=0x7f38d8003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:05.890 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38f3140700 1 --2- 172.21.15.162:0/4035658920 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38d8003720 0x7f38d8003b40 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 rx=0x7f38dc00c000 tx=0x7f38dc0091a0).stop 2020-11-28T21:56:05.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38f3140700 1 -- 172.21.15.162:0/4035658920 shutdown_connections 2020-11-28T21:56:05.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38f3140700 1 --2- 172.21.15.162:0/4035658920 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38d8003720 0x7f38d8003b40 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:05.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38f3140700 1 -- 172.21.15.162:0/4035658920 >> 172.21.15.162:0/4035658920 conn(0x7f38ec05c430 msgr2=0x7f38ec0b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:05.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38f3140700 1 -- 172.21.15.162:0/4035658920 shutdown_connections 2020-11-28T21:56:05.891 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.874+0000 7f38f3140700 1 -- 172.21.15.162:0/4035658920 wait complete. 2020-11-28T21:56:05.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.878+0000 7f38f3140700 1 Processor -- start 2020-11-28T21:56:05.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.878+0000 7f38f3140700 1 -- start start 2020-11-28T21:56:05.894 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.878+0000 7f38f3140700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f38d8003720 0x7f38ec0b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:05.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.878+0000 7f38f3140700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f38ec1455c0 0x7f38ec0b61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:05.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.878+0000 7f38f3140700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38ec143f50 0x7f38ec144860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:05.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.878+0000 7f38f3140700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec145a50 con 0x7f38ec1455c0 2020-11-28T21:56:05.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.878+0000 7f38f3140700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec145cf0 con 0x7f38ec143f50 2020-11-28T21:56:05.895 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.878+0000 7f38f3140700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec0b1390 con 0x7f38d8003720 2020-11-28T21:56:05.896 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.878+0000 7f38f16dd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38ec143f50 0x7f38ec144860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:05.896 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.878+0000 7f38f16dd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38ec143f50 0x7f38ec144860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45808/0 (socket says 172.21.15.162:45808) 2020-11-28T21:56:05.896 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.878+0000 7f38f16dd700 1 -- 172.21.15.162:0/390369557 learned_addr learned my addr 172.21.15.162:0/390369557 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:05.896 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.878+0000 7f38ebfff700 1 --2- 172.21.15.162:0/390369557 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f38ec1455c0 0x7f38ec0b61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:05.897 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.878+0000 7f38f0edc700 1 --2- 172.21.15.162:0/390369557 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f38d8003720 0x7f38ec0b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:05.897 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.878+0000 7f38f16dd700 1 -- 172.21.15.162:0/390369557 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f38d8003720 msgr2=0x7f38ec0b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:05.897 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.878+0000 7f38f16dd700 1 --2- 172.21.15.162:0/390369557 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f38d8003720 0x7f38ec0b1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:05.897 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.878+0000 7f38f16dd700 1 -- 172.21.15.162:0/390369557 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f38ec1455c0 msgr2=0x7f38ec0b61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:05.897 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.878+0000 7f38f16dd700 1 --2- 172.21.15.162:0/390369557 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f38ec1455c0 0x7f38ec0b61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:05.898 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.878+0000 7f38f16dd700 1 -- 172.21.15.162:0/390369557 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ec0b1630 con 0x7f38ec143f50 2020-11-28T21:56:05.898 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.878+0000 7f38f16dd700 1 --2- 172.21.15.162:0/390369557 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38ec143f50 0x7f38ec144860 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 rx=0x7f38e00010d0 tx=0x7f38e4009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:05.898 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.878+0000 7f38e9ffb700 1 -- 172.21.15.162:0/390369557 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f38e400de30 con 0x7f38ec143f50 2020-11-28T21:56:05.898 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.878+0000 7f38f3140700 1 -- 172.21.15.162:0/390369557 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38ec1443e0 con 0x7f38ec143f50 2020-11-28T21:56:05.898 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.878+0000 7f38e9ffb700 1 -- 172.21.15.162:0/390369557 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e4013420 con 0x7f38ec143f50 2020-11-28T21:56:05.899 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.878+0000 7f38e9ffb700 1 -- 172.21.15.162:0/390369557 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f38e40137c0 con 0x7f38ec143f50 2020-11-28T21:56:05.899 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.878+0000 7f38f3140700 1 -- 172.21.15.162:0/390369557 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38ec144f70 con 0x7f38ec143f50 2020-11-28T21:56:05.899 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.882+0000 7f38f3140700 1 -- 172.21.15.162:0/390369557 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38ec0b17c0 con 0x7f38ec143f50 2020-11-28T21:56:05.899 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.882+0000 7f38e9ffb700 1 -- 172.21.15.162:0/390369557 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f38e4013920 con 0x7f38ec143f50 2020-11-28T21:56:05.899 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.882+0000 7f38e9ffb700 1 --2- 172.21.15.162:0/390369557 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f38cc0554e0 0x7f38cc0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:05.900 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.882+0000 7f38e9ffb700 1 -- 172.21.15.162:0/390369557 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f38e406cec0 con 0x7f38ec143f50 2020-11-28T21:56:05.900 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.882+0000 7f38f0edc700 1 --2- 172.21.15.162:0/390369557 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f38cc0554e0 0x7f38cc0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:05.900 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.882+0000 7f38f0edc700 1 --2- 172.21.15.162:0/390369557 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f38cc0554e0 0x7f38cc0579a0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7f38dc01d000 tx=0x7f38dc01a000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:05.900 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:05.890+0000 7f38e9ffb700 1 -- 172.21.15.162:0/390369557 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f38e4043120 con 0x7f38ec143f50 2020-11-28T21:56:06.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.034+0000 7f38f3140700 1 -- 172.21.15.162:0/390369557 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f38ec057050 con 0x7f38ec143f50 2020-11-28T21:56:06.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.034+0000 7f38e9ffb700 1 -- 172.21.15.162:0/390369557 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v148) v1 ==== 92+0+10 (secure 0 0 0) 0x7f38e4046740 con 0x7f38ec143f50 2020-11-28T21:56:06.039 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:56:06.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.034+0000 7f38f3140700 1 -- 172.21.15.162:0/390369557 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f38cc0554e0 msgr2=0x7f38cc0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:06.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.034+0000 7f38f3140700 1 --2- 172.21.15.162:0/390369557 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f38cc0554e0 0x7f38cc0579a0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7f38dc01d000 tx=0x7f38dc01a000).stop 2020-11-28T21:56:06.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.034+0000 7f38f3140700 1 -- 172.21.15.162:0/390369557 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38ec143f50 msgr2=0x7f38ec144860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:06.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.034+0000 7f38f3140700 1 --2- 172.21.15.162:0/390369557 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38ec143f50 0x7f38ec144860 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 rx=0x7f38e00010d0 tx=0x7f38e4009040).stop 2020-11-28T21:56:06.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.038+0000 7f38f3140700 1 -- 172.21.15.162:0/390369557 shutdown_connections 2020-11-28T21:56:06.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.038+0000 7f38f3140700 1 --2- 172.21.15.162:0/390369557 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f38cc0554e0 0x7f38cc0579a0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:06.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.038+0000 7f38f3140700 1 --2- 172.21.15.162:0/390369557 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f38d8003720 0x7f38ec0b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:06.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.038+0000 7f38f3140700 1 --2- 172.21.15.162:0/390369557 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38ec143f50 0x7f38ec144860 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:06.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.038+0000 7f38f3140700 1 --2- 172.21.15.162:0/390369557 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f38ec1455c0 0x7f38ec0b61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:06.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.038+0000 7f38f3140700 1 -- 172.21.15.162:0/390369557 >> 172.21.15.162:0/390369557 conn(0x7f38ec05c430 msgr2=0x7f38ec05f750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:06.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.038+0000 7f38f3140700 1 -- 172.21.15.162:0/390369557 shutdown_connections 2020-11-28T21:56:06.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.038+0000 7f38f3140700 1 -- 172.21.15.162:0/390369557 wait complete. 2020-11-28T21:56:06.059 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-28T21:56:06.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.162+0000 7f0dae1e0700 1 Processor -- start 2020-11-28T21:56:06.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.162+0000 7f0dae1e0700 1 -- start start 2020-11-28T21:56:06.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.166+0000 7f0dae1e0700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0da80b21f0 0x7f0da813fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:06.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.166+0000 7f0dae1e0700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0da8060b60 0x7f0da814d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:06.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.166+0000 7f0dae1e0700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0da80aff20 con 0x7f0da81402a0 2020-11-28T21:56:06.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.166+0000 7f0dae1e0700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0da80af240 con 0x7f0da80b21f0 2020-11-28T21:56:06.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.166+0000 7f0dae1e0700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0da8140750 con 0x7f0da8060b60 2020-11-28T21:56:06.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.166+0000 7f0dac9dd700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f0da81402a0 0x7f0da8060450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:52656/0 (socket says 172.21.15.162:52656) 2020-11-28T21:56:06.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.166+0000 7f0dac9dd700 1 -- 172.21.15.162:0/3213841261 learned_addr learned my addr 172.21.15.162:0/3213841261 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:56:06.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.166+0000 7f0dad1de700 1 --2- 172.21.15.162:0/3213841261 >> v2:172.21.15.162:3300/0 conn(0x7f0da80b21f0 0x7f0da813fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:06.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.166+0000 7f0dad9df700 1 --2- 172.21.15.162:0/3213841261 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0da8060b60 0x7f0da814d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:06.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.166+0000 7f0d9ffff700 1 -- 172.21.15.162:0/3213841261 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4265081007 0 0) 0x7f0da80aff20 con 0x7f0da81402a0 2020-11-28T21:56:06.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.166+0000 7f0d9ffff700 1 -- 172.21.15.162:0/3213841261 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d8c003230 con 0x7f0da81402a0 2020-11-28T21:56:06.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.166+0000 7f0dad1de700 1 -- 172.21.15.162:0/3213841261 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0da8060b60 msgr2=0x7f0da814d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:06.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.166+0000 7f0dad1de700 1 --2- 172.21.15.162:0/3213841261 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0da8060b60 0x7f0da814d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:06.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.166+0000 7f0dad1de700 1 -- 172.21.15.162:0/3213841261 >> v1:172.21.15.162:6789/0 conn(0x7f0da81402a0 legacy=0x7f0da8060450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:06.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.166+0000 7f0dad1de700 1 -- 172.21.15.162:0/3213841261 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da80b25d0 con 0x7f0da80b21f0 2020-11-28T21:56:06.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.166+0000 7f0dad1de700 1 --2- 172.21.15.162:0/3213841261 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0da80b21f0 0x7f0da813fdb0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 rx=0x7f0da0002f30 tx=0x7f0da0006dc0).ready entity=mon.1 client_cookie=3a332593ff0457c7 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:06.174 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.166+0000 7f0d9ffff700 1 -- 172.21.15.162:0/3213841261 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0da00096a0 con 0x7f0da80b21f0 2020-11-28T21:56:06.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.166+0000 7f0d9ffff700 1 -- 172.21.15.162:0/3213841261 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da0009800 con 0x7f0da80b21f0 2020-11-28T21:56:06.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.166+0000 7f0d9ffff700 1 -- 172.21.15.162:0/3213841261 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0da0009b60 con 0x7f0da80b21f0 2020-11-28T21:56:06.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.166+0000 7f0dae1e0700 1 -- 172.21.15.162:0/3213841261 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0da80b21f0 msgr2=0x7f0da813fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:06.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.166+0000 7f0dae1e0700 1 --2- 172.21.15.162:0/3213841261 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0da80b21f0 0x7f0da813fdb0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 rx=0x7f0da0002f30 tx=0x7f0da0006dc0).stop 2020-11-28T21:56:06.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.166+0000 7f0dae1e0700 1 -- 172.21.15.162:0/3213841261 shutdown_connections 2020-11-28T21:56:06.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.166+0000 7f0dae1e0700 1 --2- 172.21.15.162:0/3213841261 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0da80b21f0 0x7f0da813fdb0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:06.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.166+0000 7f0dae1e0700 1 --2- 172.21.15.162:0/3213841261 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0da8060b60 0x7f0da814d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:06.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.166+0000 7f0dae1e0700 1 -- 172.21.15.162:0/3213841261 >> 172.21.15.162:0/3213841261 conn(0x7f0da80b57d0 msgr2=0x7f0da80b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:06.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.166+0000 7f0dae1e0700 1 -- 172.21.15.162:0/3213841261 shutdown_connections 2020-11-28T21:56:06.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.166+0000 7f0dae1e0700 1 -- 172.21.15.162:0/3213841261 wait complete. 2020-11-28T21:56:06.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.170+0000 7f0dae1e0700 1 Processor -- start 2020-11-28T21:56:06.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.170+0000 7f0dae1e0700 1 -- start start 2020-11-28T21:56:06.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.170+0000 7f0dae1e0700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0da8060b60 0x7f0da8152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:06.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.170+0000 7f0dae1e0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0da80b21f0 0x7f0da8145630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:06.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.170+0000 7f0dae1e0700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0da81402a0 0x7f0da8145f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:06.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.170+0000 7f0dae1e0700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da8142320 con 0x7f0da8060b60 2020-11-28T21:56:06.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.170+0000 7f0dae1e0700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da8145010 con 0x7f0da80b21f0 2020-11-28T21:56:06.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.170+0000 7f0dae1e0700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0da8151b40 con 0x7f0da81402a0 2020-11-28T21:56:06.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.170+0000 7f0dac9dd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0da80b21f0 0x7f0da8145630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:06.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.170+0000 7f0dad1de700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0da8060b60 0x7f0da8152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:06.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.170+0000 7f0dad9df700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0da81402a0 0x7f0da8145f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:06.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.170+0000 7f0dac9dd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0da80b21f0 0x7f0da8145630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45822/0 (socket says 172.21.15.162:45822) 2020-11-28T21:56:06.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.170+0000 7f0dac9dd700 1 -- 172.21.15.162:0/3955824666 learned_addr learned my addr 172.21.15.162:0/3955824666 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:06.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.170+0000 7f0dac9dd700 1 -- 172.21.15.162:0/3955824666 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0da81402a0 msgr2=0x7f0da8145f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:06.183 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.170+0000 7f0dac9dd700 1 --2- 172.21.15.162:0/3955824666 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0da81402a0 0x7f0da8145f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:06.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.170+0000 7f0dac9dd700 1 -- 172.21.15.162:0/3955824666 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0da8060b60 msgr2=0x7f0da8152520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:06.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.170+0000 7f0dac9dd700 1 --2- 172.21.15.162:0/3955824666 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0da8060b60 0x7f0da8152520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:06.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.170+0000 7f0dac9dd700 1 -- 172.21.15.162:0/3955824666 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da8151de0 con 0x7f0da80b21f0 2020-11-28T21:56:06.184 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.170+0000 7f0dad9df700 1 --2- 172.21.15.162:0/3955824666 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0da81402a0 0x7f0da8145f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:56:06.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.170+0000 7f0dad1de700 1 --2- 172.21.15.162:0/3955824666 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0da8060b60 0x7f0da8152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:56:06.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.170+0000 7f0dac9dd700 1 --2- 172.21.15.162:0/3955824666 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0da80b21f0 0x7f0da8145630 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 rx=0x7f0da4001450 tx=0x7f0da4007e20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:06.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.170+0000 7f0d9e7fc700 1 -- 172.21.15.162:0/3955824666 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0da4005b70 con 0x7f0da80b21f0 2020-11-28T21:56:06.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.170+0000 7f0dae1e0700 1 -- 172.21.15.162:0/3955824666 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0da8152080 con 0x7f0da80b21f0 2020-11-28T21:56:06.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.170+0000 7f0d9e7fc700 1 -- 172.21.15.162:0/3955824666 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da4005cd0 con 0x7f0da80b21f0 2020-11-28T21:56:06.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.170+0000 7f0d9e7fc700 1 -- 172.21.15.162:0/3955824666 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0da4007300 con 0x7f0da80b21f0 2020-11-28T21:56:06.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.170+0000 7f0dae1e0700 1 -- 172.21.15.162:0/3955824666 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0da8145b20 con 0x7f0da80b21f0 2020-11-28T21:56:06.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.174+0000 7f0dae1e0700 1 -- 172.21.15.162:0/3955824666 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0da8152210 con 0x7f0da80b21f0 2020-11-28T21:56:06.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.174+0000 7f0d9e7fc700 1 -- 172.21.15.162:0/3955824666 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0da4010030 con 0x7f0da80b21f0 2020-11-28T21:56:06.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.174+0000 7f0d9e7fc700 1 --2- 172.21.15.162:0/3955824666 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0d900553b0 0x7f0d90057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:06.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.174+0000 7f0d9e7fc700 1 -- 172.21.15.162:0/3955824666 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0da406e570 con 0x7f0da80b21f0 2020-11-28T21:56:06.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.174+0000 7f0dad1de700 1 --2- 172.21.15.162:0/3955824666 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0d900553b0 0x7f0d90057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:06.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.174+0000 7f0dad1de700 1 --2- 172.21.15.162:0/3955824666 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0d900553b0 0x7f0d90057870 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7f0da001b000 tx=0x7f0da0015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:06.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.178+0000 7f0d9e7fc700 1 -- 172.21.15.162:0/3955824666 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0da4044810 con 0x7f0da80b21f0 2020-11-28T21:56:06.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.322+0000 7f0dae1e0700 1 -- 172.21.15.162:0/3955824666 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0da8057050 con 0x7f0da80b21f0 2020-11-28T21:56:06.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.322+0000 7f0d9e7fc700 1 -- 172.21.15.162:0/3955824666 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v148) v1 ==== 104+0+10 (secure 0 0 0) 0x7f0da4047e30 con 0x7f0da80b21f0 2020-11-28T21:56:06.327 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:56:06.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.326+0000 7f0dae1e0700 1 -- 172.21.15.162:0/3955824666 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0d900553b0 msgr2=0x7f0d90057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:06.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.326+0000 7f0dae1e0700 1 --2- 172.21.15.162:0/3955824666 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0d900553b0 0x7f0d90057870 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7f0da001b000 tx=0x7f0da0015000).stop 2020-11-28T21:56:06.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.326+0000 7f0dae1e0700 1 -- 172.21.15.162:0/3955824666 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0da80b21f0 msgr2=0x7f0da8145630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:06.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.326+0000 7f0dae1e0700 1 --2- 172.21.15.162:0/3955824666 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0da80b21f0 0x7f0da8145630 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 rx=0x7f0da4001450 tx=0x7f0da4007e20).stop 2020-11-28T21:56:06.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.326+0000 7f0dae1e0700 1 -- 172.21.15.162:0/3955824666 shutdown_connections 2020-11-28T21:56:06.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.326+0000 7f0dae1e0700 1 --2- 172.21.15.162:0/3955824666 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0d900553b0 0x7f0d90057870 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:06.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.326+0000 7f0dae1e0700 1 --2- 172.21.15.162:0/3955824666 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0da81402a0 0x7f0da8145f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:06.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.326+0000 7f0dae1e0700 1 --2- 172.21.15.162:0/3955824666 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0da8060b60 0x7f0da8152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:06.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.326+0000 7f0dae1e0700 1 --2- 172.21.15.162:0/3955824666 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0da80b21f0 0x7f0da8145630 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:06.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.326+0000 7f0dae1e0700 1 -- 172.21.15.162:0/3955824666 >> 172.21.15.162:0/3955824666 conn(0x7f0da80b57d0 msgr2=0x7f0da813ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:06.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.326+0000 7f0dae1e0700 1 -- 172.21.15.162:0/3955824666 shutdown_connections 2020-11-28T21:56:06.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.326+0000 7f0dae1e0700 1 -- 172.21.15.162:0/3955824666 wait complete. 2020-11-28T21:56:06.347 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-28T21:56:06.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.462+0000 7fde54a21700 1 Processor -- start 2020-11-28T21:56:06.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.462+0000 7fde54a21700 1 -- start start 2020-11-28T21:56:06.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.462+0000 7fde54a21700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fde501433c0 0x7fde500605c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:06.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.462+0000 7fde54a21700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fde50067610 0x7fde5013fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:06.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.462+0000 7fde54a21700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde50143950 con 0x7fde500b9ce0 2020-11-28T21:56:06.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.462+0000 7fde54a21700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde500b4340 con 0x7fde501433c0 2020-11-28T21:56:06.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.462+0000 7fde54a21700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde50067ac0 con 0x7fde50067610 2020-11-28T21:56:06.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.462+0000 7fde4ed9d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fde50067610 0x7fde5013fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:06.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.462+0000 7fde4ed9d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fde50067610 0x7fde5013fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45830/0 (socket says 172.21.15.162:45830) 2020-11-28T21:56:06.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.462+0000 7fde4ed9d700 1 -- 172.21.15.162:0/3117728180 learned_addr learned my addr 172.21.15.162:0/3117728180 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:06.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.462+0000 7fde4dd9b700 1 --2- 172.21.15.162:0/3117728180 >> v2:172.21.15.118:3300/0 conn(0x7fde501433c0 0x7fde500605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:06.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.462+0000 7fde4ed9d700 1 -- 172.21.15.162:0/3117728180 >> v2:172.21.15.118:3300/0 conn(0x7fde501433c0 msgr2=0x7fde500605c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:06.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.462+0000 7fde4ed9d700 1 --2- 172.21.15.162:0/3117728180 >> v2:172.21.15.118:3300/0 conn(0x7fde501433c0 0x7fde500605c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:06.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.462+0000 7fde4ed9d700 1 -- 172.21.15.162:0/3117728180 >> v1:172.21.15.118:6789/0 conn(0x7fde500b9ce0 legacy=0x7fde500b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:06.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.462+0000 7fde4ed9d700 1 -- 172.21.15.162:0/3117728180 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde500ba0c0 con 0x7fde50067610 2020-11-28T21:56:06.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.462+0000 7fde4ed9d700 1 --2- 172.21.15.162:0/3117728180 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde50067610 0x7fde5013fc80 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 rx=0x7fde480032b0 tx=0x7fde48009270).ready entity=mon.1 client_cookie=dac7421eb4726add server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:06.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.462+0000 7fde4d59a700 1 -- 172.21.15.162:0/3117728180 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde48009e60 con 0x7fde50067610 2020-11-28T21:56:06.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.462+0000 7fde4d59a700 1 -- 172.21.15.162:0/3117728180 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde4800bc00 con 0x7fde50067610 2020-11-28T21:56:06.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.462+0000 7fde4d59a700 1 -- 172.21.15.162:0/3117728180 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde4800bd70 con 0x7fde50067610 2020-11-28T21:56:06.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.462+0000 7fde54a21700 1 -- 172.21.15.162:0/3117728180 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde50067610 msgr2=0x7fde5013fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:06.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.462+0000 7fde54a21700 1 --2- 172.21.15.162:0/3117728180 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde50067610 0x7fde5013fc80 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 rx=0x7fde480032b0 tx=0x7fde48009270).stop 2020-11-28T21:56:06.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.466+0000 7fde54a21700 1 -- 172.21.15.162:0/3117728180 shutdown_connections 2020-11-28T21:56:06.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.466+0000 7fde54a21700 1 --2- 172.21.15.162:0/3117728180 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde50067610 0x7fde5013fc80 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:06.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.466+0000 7fde54a21700 1 --2- 172.21.15.162:0/3117728180 >> v2:172.21.15.118:3300/0 conn(0x7fde501433c0 0x7fde500605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:06.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.466+0000 7fde54a21700 1 -- 172.21.15.162:0/3117728180 >> 172.21.15.162:0/3117728180 conn(0x7fde5005c430 msgr2=0x7fde500b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:06.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.466+0000 7fde54a21700 1 -- 172.21.15.162:0/3117728180 shutdown_connections 2020-11-28T21:56:06.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.466+0000 7fde54a21700 1 -- 172.21.15.162:0/3117728180 wait complete. 2020-11-28T21:56:06.487 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.466+0000 7fde54a21700 1 Processor -- start 2020-11-28T21:56:06.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.466+0000 7fde54a21700 1 -- start start 2020-11-28T21:56:06.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.466+0000 7fde54a21700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde50067610 0x7fde50152910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:06.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.466+0000 7fde54a21700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fde500b9ce0 0x7fde500b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:06.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.466+0000 7fde54a21700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fde501433c0 0x7fde501456c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:06.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.466+0000 7fde54a21700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde50151d40 con 0x7fde501433c0 2020-11-28T21:56:06.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.466+0000 7fde54a21700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde500b7730 con 0x7fde50067610 2020-11-28T21:56:06.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.466+0000 7fde54a21700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde50151f30 con 0x7fde500b9ce0 2020-11-28T21:56:06.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.466+0000 7fde4e59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde50067610 0x7fde50152910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:06.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.466+0000 7fde4e59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde50067610 0x7fde50152910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45832/0 (socket says 172.21.15.162:45832) 2020-11-28T21:56:06.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.466+0000 7fde4e59c700 1 -- 172.21.15.162:0/1851922344 learned_addr learned my addr 172.21.15.162:0/1851922344 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:06.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.466+0000 7fde4dd9b700 1 --2- 172.21.15.162:0/1851922344 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fde500b9ce0 0x7fde500b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:06.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.470+0000 7fde4ed9d700 1 --2- 172.21.15.162:0/1851922344 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fde501433c0 0x7fde501456c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:06.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.470+0000 7fde4e59c700 1 -- 172.21.15.162:0/1851922344 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fde500b9ce0 msgr2=0x7fde500b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:06.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.470+0000 7fde4e59c700 1 --2- 172.21.15.162:0/1851922344 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fde500b9ce0 0x7fde500b57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:06.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.470+0000 7fde4e59c700 1 -- 172.21.15.162:0/1851922344 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fde501433c0 msgr2=0x7fde501456c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:06.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.470+0000 7fde4e59c700 1 --2- 172.21.15.162:0/1851922344 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fde501433c0 0x7fde501456c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:06.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.470+0000 7fde4e59c700 1 -- 172.21.15.162:0/1851922344 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde501521d0 con 0x7fde50067610 2020-11-28T21:56:06.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.470+0000 7fde4ed9d700 1 --2- 172.21.15.162:0/1851922344 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fde501433c0 0x7fde501456c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:56:06.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.470+0000 7fde4e59c700 1 --2- 172.21.15.162:0/1851922344 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde50067610 0x7fde50152910 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 rx=0x7fde500b05c0 tx=0x7fde40007bd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:06.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.470+0000 7fde3f7fe700 1 -- 172.21.15.162:0/1851922344 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde40001ea0 con 0x7fde50067610 2020-11-28T21:56:06.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.470+0000 7fde3f7fe700 1 -- 172.21.15.162:0/1851922344 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde4000f470 con 0x7fde50067610 2020-11-28T21:56:06.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.470+0000 7fde54a21700 1 -- 172.21.15.162:0/1851922344 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde50152470 con 0x7fde50067610 2020-11-28T21:56:06.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.470+0000 7fde3f7fe700 1 -- 172.21.15.162:0/1851922344 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde4000f7d0 con 0x7fde50067610 2020-11-28T21:56:06.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.470+0000 7fde54a21700 1 -- 172.21.15.162:0/1851922344 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde500b22c0 con 0x7fde50067610 2020-11-28T21:56:06.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.470+0000 7fde54a21700 1 -- 172.21.15.162:0/1851922344 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde50152600 con 0x7fde50067610 2020-11-28T21:56:06.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.470+0000 7fde3f7fe700 1 -- 172.21.15.162:0/1851922344 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fde40019030 con 0x7fde50067610 2020-11-28T21:56:06.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.470+0000 7fde3f7fe700 1 --2- 172.21.15.162:0/1851922344 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fde2c0554e0 0x7fde2c0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:06.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.474+0000 7fde3f7fe700 1 -- 172.21.15.162:0/1851922344 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fde4006d480 con 0x7fde50067610 2020-11-28T21:56:06.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.474+0000 7fde4dd9b700 1 --2- 172.21.15.162:0/1851922344 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fde2c0554e0 0x7fde2c0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:06.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.474+0000 7fde4dd9b700 1 --2- 172.21.15.162:0/1851922344 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fde2c0554e0 0x7fde2c0579a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7fde4400b000 tx=0x7fde44008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:06.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.478+0000 7fde3f7fe700 1 -- 172.21.15.162:0/1851922344 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fde400436e0 con 0x7fde50067610 2020-11-28T21:56:06.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.618+0000 7fde54a21700 1 -- 172.21.15.162:0/1851922344 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fde50057050 con 0x7fde50067610 2020-11-28T21:56:06.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.622+0000 7fde3f7fe700 1 -- 172.21.15.162:0/1851922344 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v148) v1 ==== 100+0+10 (secure 0 0 0) 0x7fde40046d00 con 0x7fde50067610 2020-11-28T21:56:06.626 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:56:06.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.622+0000 7fde54a21700 1 -- 172.21.15.162:0/1851922344 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fde2c0554e0 msgr2=0x7fde2c0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:06.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.622+0000 7fde54a21700 1 --2- 172.21.15.162:0/1851922344 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fde2c0554e0 0x7fde2c0579a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7fde4400b000 tx=0x7fde44008040).stop 2020-11-28T21:56:06.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.622+0000 7fde54a21700 1 -- 172.21.15.162:0/1851922344 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde50067610 msgr2=0x7fde50152910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:06.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.622+0000 7fde54a21700 1 --2- 172.21.15.162:0/1851922344 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde50067610 0x7fde50152910 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 rx=0x7fde500b05c0 tx=0x7fde40007bd0).stop 2020-11-28T21:56:06.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.622+0000 7fde54a21700 1 -- 172.21.15.162:0/1851922344 shutdown_connections 2020-11-28T21:56:06.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.622+0000 7fde54a21700 1 --2- 172.21.15.162:0/1851922344 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fde2c0554e0 0x7fde2c0579a0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:06.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.622+0000 7fde54a21700 1 --2- 172.21.15.162:0/1851922344 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fde501433c0 0x7fde501456c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:06.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.622+0000 7fde54a21700 1 --2- 172.21.15.162:0/1851922344 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde50067610 0x7fde50152910 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:06.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.622+0000 7fde54a21700 1 --2- 172.21.15.162:0/1851922344 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fde500b9ce0 0x7fde500b57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:06.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.622+0000 7fde54a21700 1 -- 172.21.15.162:0/1851922344 >> 172.21.15.162:0/1851922344 conn(0x7fde5005c430 msgr2=0x7fde5009e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:06.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.622+0000 7fde54a21700 1 -- 172.21.15.162:0/1851922344 shutdown_connections 2020-11-28T21:56:06.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.622+0000 7fde54a21700 1 -- 172.21.15.162:0/1851922344 wait complete. 2020-11-28T21:56:06.640 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-28T21:56:06.641 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:56:06.754 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.750+0000 7f52caf8f700 1 Processor -- start 2020-11-28T21:56:06.754 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.750+0000 7f52caf8f700 1 -- start start 2020-11-28T21:56:06.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.750+0000 7f52caf8f700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f52c4098b30 0x7f52c40994f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:06.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.750+0000 7f52caf8f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f52c40b0000 0x7f52c4060000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:06.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.750+0000 7f52caf8f700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52c4098f10 con 0x7f52c40604f0 2020-11-28T21:56:06.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.750+0000 7f52caf8f700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c40b1030 con 0x7f52c4098b30 2020-11-28T21:56:06.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.750+0000 7f52caf8f700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c40b0480 con 0x7f52c40b0000 2020-11-28T21:56:06.758 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.750+0000 7f52c9f8d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f52c4098b30 0x7f52c40994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:06.758 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.750+0000 7f52c9f8d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f52c4098b30 0x7f52c40994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45840/0 (socket says 172.21.15.162:45840) 2020-11-28T21:56:06.758 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.750+0000 7f52c9f8d700 1 -- 172.21.15.162:0/4082027168 learned_addr learned my addr 172.21.15.162:0/4082027168 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:06.758 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.750+0000 7f52c9f8d700 1 -- 172.21.15.162:0/4082027168 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f52c40b0000 msgr2=0x7f52c4060000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:06.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.750+0000 7f52c978c700 1 --2- 172.21.15.162:0/4082027168 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f52c40b0000 0x7f52c4060000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:06.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.750+0000 7f52c9f8d700 1 --2- 172.21.15.162:0/4082027168 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f52c40b0000 0x7f52c4060000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:06.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.750+0000 7f52c9f8d700 1 -- 172.21.15.162:0/4082027168 >> v1:172.21.15.162:6789/0 conn(0x7f52c40604f0 legacy=0x7f52c414a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:06.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.750+0000 7f52c9f8d700 1 -- 172.21.15.162:0/4082027168 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c40b0680 con 0x7f52c4098b30 2020-11-28T21:56:06.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.750+0000 7f52c978c700 1 --2- 172.21.15.162:0/4082027168 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f52c40b0000 0x7f52c4060000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:56:06.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.750+0000 7f52c9f8d700 1 --2- 172.21.15.162:0/4082027168 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52c4098b30 0x7f52c40994f0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 rx=0x7f52c4099be0 tx=0x7f52bc006e80).ready entity=mon.1 client_cookie=611a8c18a75eb46e server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:06.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.750+0000 7f52c8f8b700 1 -- 172.21.15.162:0/4082027168 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52bc0097b0 con 0x7f52c4098b30 2020-11-28T21:56:06.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.750+0000 7f52c8f8b700 1 -- 172.21.15.162:0/4082027168 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52bc009910 con 0x7f52c4098b30 2020-11-28T21:56:06.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.750+0000 7f52c8f8b700 1 -- 172.21.15.162:0/4082027168 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52bc009c70 con 0x7f52c4098b30 2020-11-28T21:56:06.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.750+0000 7f52caf8f700 1 -- 172.21.15.162:0/4082027168 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52c4098b30 msgr2=0x7f52c40994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:06.761 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.750+0000 7f52caf8f700 1 --2- 172.21.15.162:0/4082027168 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52c4098b30 0x7f52c40994f0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 rx=0x7f52c4099be0 tx=0x7f52bc006e80).stop 2020-11-28T21:56:06.761 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.750+0000 7f52caf8f700 1 -- 172.21.15.162:0/4082027168 shutdown_connections 2020-11-28T21:56:06.761 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.750+0000 7f52caf8f700 1 --2- 172.21.15.162:0/4082027168 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52c4098b30 0x7f52c40994f0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:06.761 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.750+0000 7f52caf8f700 1 --2- 172.21.15.162:0/4082027168 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f52c40b0000 0x7f52c4060000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:06.761 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.750+0000 7f52caf8f700 1 -- 172.21.15.162:0/4082027168 >> 172.21.15.162:0/4082027168 conn(0x7f52c405b970 msgr2=0x7f52c405bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:06.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.750+0000 7f52caf8f700 1 -- 172.21.15.162:0/4082027168 shutdown_connections 2020-11-28T21:56:06.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.750+0000 7f52caf8f700 1 -- 172.21.15.162:0/4082027168 wait complete. 2020-11-28T21:56:06.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.754+0000 7f52caf8f700 1 Processor -- start 2020-11-28T21:56:06.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.754+0000 7f52caf8f700 1 -- start start 2020-11-28T21:56:06.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.754+0000 7f52caf8f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52c40604f0 0x7f52c40ade20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:06.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.754+0000 7f52caf8f700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f52c4098b30 0x7f52c4065fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:06.765 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.754+0000 7f52caf8f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f52c40b0000 0x7f52c41458d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:06.765 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.754+0000 7f52caf8f700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c4067350 con 0x7f52c4098b30 2020-11-28T21:56:06.765 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.754+0000 7f52caf8f700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c40675e0 con 0x7f52c40604f0 2020-11-28T21:56:06.765 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.754+0000 7f52caf8f700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c40ae940 con 0x7f52c40b0000 2020-11-28T21:56:06.766 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.754+0000 7f52c9f8d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52c40604f0 0x7f52c40ade20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:06.766 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.754+0000 7f52c9f8d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52c40604f0 0x7f52c40ade20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45848/0 (socket says 172.21.15.162:45848) 2020-11-28T21:56:06.766 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.754+0000 7f52c9f8d700 1 -- 172.21.15.162:0/4173793232 learned_addr learned my addr 172.21.15.162:0/4173793232 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:06.766 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.754+0000 7f52ca78e700 1 --2- 172.21.15.162:0/4173793232 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f52c40b0000 0x7f52c41458d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:06.766 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.754+0000 7f52c978c700 1 --2- 172.21.15.162:0/4173793232 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f52c4098b30 0x7f52c4065fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:06.767 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.754+0000 7f52c9f8d700 1 -- 172.21.15.162:0/4173793232 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f52c40b0000 msgr2=0x7f52c41458d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:06.767 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.754+0000 7f52c9f8d700 1 --2- 172.21.15.162:0/4173793232 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f52c40b0000 0x7f52c41458d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:06.767 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.754+0000 7f52c9f8d700 1 -- 172.21.15.162:0/4173793232 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f52c4098b30 msgr2=0x7f52c4065fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:06.767 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.754+0000 7f52c9f8d700 1 --2- 172.21.15.162:0/4173793232 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f52c4098b30 0x7f52c4065fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:06.767 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.754+0000 7f52c9f8d700 1 -- 172.21.15.162:0/4173793232 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c40aebe0 con 0x7f52c40604f0 2020-11-28T21:56:06.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.754+0000 7f52c9f8d700 1 --2- 172.21.15.162:0/4173793232 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52c40604f0 0x7f52c40ade20 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 rx=0x7f52c4099be0 tx=0x7f52bc015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:06.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.754+0000 7f52c2ffd700 1 -- 172.21.15.162:0/4173793232 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52bc011b80 con 0x7f52c40604f0 2020-11-28T21:56:06.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.754+0000 7f52c2ffd700 1 -- 172.21.15.162:0/4173793232 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52bc011ce0 con 0x7f52c40604f0 2020-11-28T21:56:06.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.754+0000 7f52caf8f700 1 -- 172.21.15.162:0/4173793232 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c40ae310 con 0x7f52c40604f0 2020-11-28T21:56:06.768 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.754+0000 7f52c2ffd700 1 -- 172.21.15.162:0/4173793232 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52bc01e650 con 0x7f52c40604f0 2020-11-28T21:56:06.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.754+0000 7f52caf8f700 1 -- 172.21.15.162:0/4173793232 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c40666f0 con 0x7f52c40604f0 2020-11-28T21:56:06.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.754+0000 7f52caf8f700 1 -- 172.21.15.162:0/4173793232 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52c4057050 con 0x7f52c40604f0 2020-11-28T21:56:06.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.758+0000 7f52c2ffd700 1 -- 172.21.15.162:0/4173793232 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f52bc025030 con 0x7f52c40604f0 2020-11-28T21:56:06.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.758+0000 7f52c2ffd700 1 --2- 172.21.15.162:0/4173793232 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f52a8055440 0x7f52a8057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:06.769 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.758+0000 7f52c978c700 1 --2- 172.21.15.162:0/4173793232 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f52a8055440 0x7f52a8057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:06.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.758+0000 7f52c2ffd700 1 -- 172.21.15.162:0/4173793232 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f52bc06f310 con 0x7f52c40604f0 2020-11-28T21:56:06.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.758+0000 7f52c978c700 1 --2- 172.21.15.162:0/4173793232 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f52a8055440 0x7f52a8057900 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7f52b400d000 tx=0x7f52b4008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:06.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.766+0000 7f52c2ffd700 1 -- 172.21.15.162:0/4173793232 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f52bc045610 con 0x7f52c40604f0 2020-11-28T21:56:06.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.950+0000 7f52caf8f700 1 -- 172.21.15.162:0/4173793232 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52c4068350 con 0x7f52c40604f0 2020-11-28T21:56:06.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.950+0000 7f52c2ffd700 1 -- 172.21.15.162:0/4173793232 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+2504 (secure 0 0 0) 0x7f52bc023020 con 0x7f52c40604f0 2020-11-28T21:56:06.955 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:56:06.955 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":74,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6899,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.118:6833/2509028245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":2509028245},{"type":"v1","addr":"172.21.15.118:6833","nonce":2509028245}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":72},{"gid":8239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/2089791609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2089791609},{"type":"v1","addr":"172.21.15.162:6836","nonce":2089791609}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":71}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:56:01.362652+0000","modified":"2020-11-28T21:56:03.027360+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8242},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8242":{"gid":8242,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/2832109513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":2832109513},{"type":"v1","addr":"172.21.15.162:6837","nonce":2832109513}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2020-11-28T21:56:06.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.950+0000 7f52caf8f700 1 -- 172.21.15.162:0/4173793232 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f52a8055440 msgr2=0x7f52a8057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:06.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.950+0000 7f52caf8f700 1 --2- 172.21.15.162:0/4173793232 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f52a8055440 0x7f52a8057900 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7f52b400d000 tx=0x7f52b4008040).stop 2020-11-28T21:56:06.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.950+0000 7f52caf8f700 1 -- 172.21.15.162:0/4173793232 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52c40604f0 msgr2=0x7f52c40ade20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:06.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.950+0000 7f52caf8f700 1 --2- 172.21.15.162:0/4173793232 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52c40604f0 0x7f52c40ade20 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 rx=0x7f52c4099be0 tx=0x7f52bc015000).stop 2020-11-28T21:56:06.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.950+0000 7f52caf8f700 1 -- 172.21.15.162:0/4173793232 shutdown_connections 2020-11-28T21:56:06.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.950+0000 7f52caf8f700 1 --2- 172.21.15.162:0/4173793232 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f52a8055440 0x7f52a8057900 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:06.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.950+0000 7f52caf8f700 1 --2- 172.21.15.162:0/4173793232 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f52c40b0000 0x7f52c41458d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:06.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.950+0000 7f52caf8f700 1 --2- 172.21.15.162:0/4173793232 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f52c40604f0 0x7f52c40ade20 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:06.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.950+0000 7f52caf8f700 1 --2- 172.21.15.162:0/4173793232 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f52c4098b30 0x7f52c4065fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:06.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.950+0000 7f52caf8f700 1 -- 172.21.15.162:0/4173793232 >> 172.21.15.162:0/4173793232 conn(0x7f52c405b970 msgr2=0x7f52c40b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:06.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.950+0000 7f52caf8f700 1 -- 172.21.15.162:0/4173793232 shutdown_connections 2020-11-28T21:56:06.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:06.950+0000 7f52caf8f700 1 -- 172.21.15.162:0/4173793232 wait complete. 2020-11-28T21:56:06.959 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 74 2020-11-28T21:56:07.272 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-28T21:56:01.362652+0000', 'modified': '2020-11-28T21:56:03.027360+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8242}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8242': {'gid': 8242, 'name': 'a', 'rank': 0, 'incarnation': 72, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6837/2832109513', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6835', 'nonce': 2832109513}, {'type': 'v1', 'addr': '172.21.15.162:6837', 'nonce': 2832109513}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-28T21:56:07.272 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-28T21:56:07.273 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-28T21:56:07.412 INFO:teuthology.orchestra.run.smithi162.stdout:{"cluster_fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","whoami":0,"id":8242,"want_state":"up:active","state":"up:active","rank_uptime":6.0350179910000001,"mdsmap_epoch":74,"osdmap_epoch":148,"osdmap_epoch_barrier":148,"uptime":10.331678265000001} 2020-11-28T21:56:07.412 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", "id": 8242, "mdsmap_epoch": 74, "osdmap_epoch": 148, "osdmap_epoch_barrier": 148, "rank_uptime": 6.035017991, "state": "up:active", "uptime": 10.331678265, "want_state": "up:active", "whoami": 0 } 2020-11-28T21:56:07.413 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-11-28T21:56:07.414 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-11-28T21:56:07.414 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi192.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-11-28T21:56:07.415 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:56:07.415 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:56:07.425 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:56:07.482 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:56:07.483 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:56:07.538 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:56:07.539 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:56:07.539 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:56:07.540 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:56:07.586 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-11-28T21:56:07.587 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:56:07.629 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:56:07.645 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:56:07.645 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:56:07.702 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:2020-11-28T21:56:07.696+0000 7faa1e0470c0 -1 init, newargv = 0x560b4f50ee20 newargc=9ceph-fuse[ 2020-11-28T21:56:07.702 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:12727]: starting ceph client 2020-11-28T21:56:07.706 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:56:07.707 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:56:07.708 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:56:07.708 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:56:07.712 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:ceph-fuse[12727]: starting fuse 2020-11-28T21:56:07.717 INFO:teuthology.orchestra.run.smithi192.stdout:49 2020-11-28T21:56:07.717 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2020-11-28T21:56:07.717 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:56:07.718 INFO:teuthology.orchestra.run.smithi192:> sudo python3 -c ' 2020-11-28T21:56:07.718 INFO:teuthology.orchestra.run.smithi192:> import glob 2020-11-28T21:56:07.719 INFO:teuthology.orchestra.run.smithi192:> import re 2020-11-28T21:56:07.719 INFO:teuthology.orchestra.run.smithi192:> import os 2020-11-28T21:56:07.720 INFO:teuthology.orchestra.run.smithi192:> import subprocess 2020-11-28T21:56:07.720 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:56:07.720 INFO:teuthology.orchestra.run.smithi192:> def find_socket(client_name): 2020-11-28T21:56:07.721 INFO:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-11-28T21:56:07.721 INFO:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2020-11-28T21:56:07.721 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:56:07.722 INFO:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2020-11-28T21:56:07.722 INFO:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2020-11-28T21:56:07.723 INFO:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2020-11-28T21:56:07.723 INFO:teuthology.orchestra.run.smithi192:> return files[0] 2020-11-28T21:56:07.724 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:56:07.724 INFO:teuthology.orchestra.run.smithi192:> for f in files: 2020-11-28T21:56:07.724 INFO:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-11-28T21:56:07.725 INFO:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2020-11-28T21:56:07.725 INFO:teuthology.orchestra.run.smithi192:> return f 2020-11-28T21:56:07.726 INFO:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-11-28T21:56:07.726 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:56:07.726 INFO:teuthology.orchestra.run.smithi192:> print(find_socket("client.0")) 2020-11-28T21:56:07.727 INFO:teuthology.orchestra.run.smithi192:> ' 2020-11-28T21:56:07.810 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.0.12727.asok 2020-11-28T21:56:07.815 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.12727.asok 2020-11-28T21:56:07.815 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:56:07.816 INFO:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.12727.asok status 2020-11-28T21:56:07.915 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2020-11-28T21:56:07.915 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2020-11-28T21:56:07.916 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "a83a3572a957356977342f461361c1cd3ec06dbd", 2020-11-28T21:56:07.916 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 15.2.6-595-ga83a3572 (a83a3572a957356977342f461361c1cd3ec06dbd) octopus (stable)", 2020-11-28T21:56:07.916 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "0", 2020-11-28T21:56:07.917 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2020-11-28T21:56:07.917 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-11-28T21:56:07.917 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "12727", 2020-11-28T21:56:07.918 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2020-11-28T21:56:07.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:56:07.918 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2020-11-28T21:56:07.919 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2020-11-28T21:56:07.920 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 6998, 2020-11-28T21:56:07.920 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2020-11-28T21:56:07.921 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2020-11-28T21:56:07.921 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2020-11-28T21:56:07.921 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 6998 2020-11-28T21:56:07.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:56:07.922 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2020-11-28T21:56:07.923 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2020-11-28T21:56:07.923 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2020-11-28T21:56:07.925 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 4126215112 2020-11-28T21:56:07.925 INFO:teuthology.orchestra.run.smithi192.stdout: } 2020-11-28T21:56:07.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:56:07.927 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2020-11-28T21:56:07.927 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2020-11-28T21:56:07.927 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2020-11-28T21:56:07.928 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 4126215112 2020-11-28T21:56:07.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:56:07.928 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.6998 172.21.15.192:0/4126215112", 2020-11-28T21:56:07.929 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "172.21.15.192:0/4126215112", 2020-11-28T21:56:07.929 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2020-11-28T21:56:07.929 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 74, 2020-11-28T21:56:07.930 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 148, 2020-11-28T21:56:07.931 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2020-11-28T21:56:07.931 INFO:teuthology.orchestra.run.smithi192.stdout: "blacklisted": false 2020-11-28T21:56:07.932 INFO:teuthology.orchestra.run.smithi192.stdout:} 2020-11-28T21:56:07.934 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:56:07.935 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:56:07.935 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:56:07.961 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2020-11-28T21:56:07.963 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-28T21:56:07.964 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:56:07.965 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:56:08.025 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:56:08.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.162+0000 7f46668b5700 1 Processor -- start 2020-11-28T21:56:08.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.162+0000 7f46668b5700 1 -- start start 2020-11-28T21:56:08.170 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.162+0000 7f46668b5700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f46600af480 0x7f46600aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:08.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.162+0000 7f46668b5700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46600b2670 con 0x7f46600a8550 2020-11-28T21:56:08.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.162+0000 7f46668b5700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46600afa00 con 0x7f46600b2290 2020-11-28T21:56:08.171 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.162+0000 7f46668b5700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46600afca0 con 0x7f46600af480 2020-11-28T21:56:08.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.162+0000 7f46658b3700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f46600b2290 0x7f46600ae4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:52698/0 (socket says 172.21.15.162:52698) 2020-11-28T21:56:08.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.162+0000 7f46658b3700 1 -- 172.21.15.162:0/1253174527 learned_addr learned my addr 172.21.15.162:0/1253174527 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:56:08.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.162+0000 7f46660b4700 1 --2- 172.21.15.162:0/1253174527 >> v2:172.21.15.118:3300/0 conn(0x7f46600af480 0x7f46600aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:08.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.166+0000 7f46648b1700 1 -- 172.21.15.162:0/1253174527 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3666188530 0 0) 0x7f46600afa00 con 0x7f46600b2290 2020-11-28T21:56:08.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.166+0000 7f46648b1700 1 -- 172.21.15.162:0/1253174527 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4648003230 con 0x7f46600b2290 2020-11-28T21:56:08.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.166+0000 7f46660b4700 1 -- 172.21.15.162:0/1253174527 >> v1:172.21.15.162:6789/0 conn(0x7f46600b2290 legacy=0x7f46600ae4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:08.173 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.166+0000 7f46660b4700 1 -- 172.21.15.162:0/1253174527 >> v1:172.21.15.118:6789/0 conn(0x7f46600a8550 legacy=0x7f46601417a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:08.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.166+0000 7f46648b1700 1 -- 172.21.15.162:0/1253174527 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2269934722 0 0) 0x7f46600b2670 con 0x7f46600a8550 2020-11-28T21:56:08.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.166+0000 7f46660b4700 1 -- 172.21.15.162:0/1253174527 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46600acb80 con 0x7f46600af480 2020-11-28T21:56:08.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.166+0000 7f46660b4700 1 --2- 172.21.15.162:0/1253174527 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f46600af480 0x7f46600aebd0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f46580032b0 tx=0x7f46580091a0).ready entity=mon.0 client_cookie=19114b9d98b1cf64 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:08.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.166+0000 7f46648b1700 1 -- 172.21.15.162:0/1253174527 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f465800b9a0 con 0x7f46600af480 2020-11-28T21:56:08.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.166+0000 7f46648b1700 1 -- 172.21.15.162:0/1253174527 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f465800bb00 con 0x7f46600af480 2020-11-28T21:56:08.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.166+0000 7f46648b1700 1 -- 172.21.15.162:0/1253174527 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f465800bc70 con 0x7f46600af480 2020-11-28T21:56:08.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.166+0000 7f46668b5700 1 -- 172.21.15.162:0/1253174527 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f46600af480 msgr2=0x7f46600aebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:08.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.166+0000 7f46668b5700 1 --2- 172.21.15.162:0/1253174527 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f46600af480 0x7f46600aebd0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f46580032b0 tx=0x7f46580091a0).stop 2020-11-28T21:56:08.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.166+0000 7f46668b5700 1 -- 172.21.15.162:0/1253174527 shutdown_connections 2020-11-28T21:56:08.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.166+0000 7f46668b5700 1 --2- 172.21.15.162:0/1253174527 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f46600af480 0x7f46600aebd0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:08.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.166+0000 7f46668b5700 1 -- 172.21.15.162:0/1253174527 >> 172.21.15.162:0/1253174527 conn(0x7f4660061320 msgr2=0x7f4660062e50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:08.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.166+0000 7f46668b5700 1 -- 172.21.15.162:0/1253174527 shutdown_connections 2020-11-28T21:56:08.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.166+0000 7f46668b5700 1 -- 172.21.15.162:0/1253174527 wait complete. 2020-11-28T21:56:08.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.166+0000 7f46668b5700 1 Processor -- start 2020-11-28T21:56:08.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.166+0000 7f46668b5700 1 -- start start 2020-11-28T21:56:08.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.166+0000 7f46668b5700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f46600a8550 0x7f4660134c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:08.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.170+0000 7f46668b5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46600af480 0x7f466012b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:08.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.170+0000 7f46668b5700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f46600b2290 0x7f466012ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:08.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.170+0000 7f46668b5700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466012ae40 con 0x7f46600a8550 2020-11-28T21:56:08.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.170+0000 7f46668b5700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466012a660 con 0x7f46600af480 2020-11-28T21:56:08.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.170+0000 7f46668b5700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f466012a900 con 0x7f46600b2290 2020-11-28T21:56:08.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.170+0000 7f46650b2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46600af480 0x7f466012b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:08.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.170+0000 7f46650b2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46600af480 0x7f466012b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45862/0 (socket says 172.21.15.162:45862) 2020-11-28T21:56:08.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.170+0000 7f46650b2700 1 -- 172.21.15.162:0/2042699582 learned_addr learned my addr 172.21.15.162:0/2042699582 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:08.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.170+0000 7f46658b3700 1 --2- 172.21.15.162:0/2042699582 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f46600a8550 0x7f4660134c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:08.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.170+0000 7f46660b4700 1 --2- 172.21.15.162:0/2042699582 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f46600b2290 0x7f466012ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:08.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.170+0000 7f46650b2700 1 -- 172.21.15.162:0/2042699582 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f46600b2290 msgr2=0x7f466012ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:08.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.170+0000 7f46650b2700 1 --2- 172.21.15.162:0/2042699582 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f46600b2290 0x7f466012ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:08.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.170+0000 7f46650b2700 1 -- 172.21.15.162:0/2042699582 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f46600a8550 msgr2=0x7f4660134c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:08.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.170+0000 7f46650b2700 1 --2- 172.21.15.162:0/2042699582 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f46600a8550 0x7f4660134c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:08.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.170+0000 7f46650b2700 1 -- 172.21.15.162:0/2042699582 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f466012aba0 con 0x7f46600af480 2020-11-28T21:56:08.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.170+0000 7f46650b2700 1 --2- 172.21.15.162:0/2042699582 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46600af480 0x7f466012b560 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 rx=0x7f466012d3e0 tx=0x7f465c007a40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:08.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.170+0000 7f464effd700 1 -- 172.21.15.162:0/2042699582 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f465c013410 con 0x7f46600af480 2020-11-28T21:56:08.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.170+0000 7f464effd700 1 -- 172.21.15.162:0/2042699582 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f465c013570 con 0x7f46600af480 2020-11-28T21:56:08.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.170+0000 7f46668b5700 1 -- 172.21.15.162:0/2042699582 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46600a9d70 con 0x7f46600af480 2020-11-28T21:56:08.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.170+0000 7f464effd700 1 -- 172.21.15.162:0/2042699582 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f465c0138d0 con 0x7f46600af480 2020-11-28T21:56:08.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.170+0000 7f46668b5700 1 -- 172.21.15.162:0/2042699582 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4660135300 con 0x7f46600af480 2020-11-28T21:56:08.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.170+0000 7f46668b5700 1 -- 172.21.15.162:0/2042699582 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4660057050 con 0x7f46600af480 2020-11-28T21:56:08.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.170+0000 7f464effd700 1 -- 172.21.15.162:0/2042699582 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f465c010030 con 0x7f46600af480 2020-11-28T21:56:08.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.170+0000 7f464effd700 1 --2- 172.21.15.162:0/2042699582 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4640055440 0x7f4640057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:08.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.174+0000 7f46658b3700 1 --2- 172.21.15.162:0/2042699582 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4640055440 0x7f4640057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:08.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.174+0000 7f464effd700 1 -- 172.21.15.162:0/2042699582 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f465c06dd30 con 0x7f46600af480 2020-11-28T21:56:08.200 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.174+0000 7f46658b3700 1 --2- 172.21.15.162:0/2042699582 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4640055440 0x7f4640057900 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f465000d000 tx=0x7f4650009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:08.200 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.178+0000 7f464effd700 1 -- 172.21.15.162:0/2042699582 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f465c043fd0 con 0x7f46600af480 2020-11-28T21:56:08.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.318+0000 7f46668b5700 1 -- 172.21.15.162:0/2042699582 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f466005d6c0 con 0x7f46600af480 2020-11-28T21:56:08.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.322+0000 7f464effd700 1 -- 172.21.15.162:0/2042699582 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+16148 (secure 0 0 0) 0x7f465c0475f0 con 0x7f46600af480 2020-11-28T21:56:08.326 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:56:08.326 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":148,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:56:01.362358+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:55:59.967971+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2020-11-28T21:56:00.379272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:56:08.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.322+0000 7f46668b5700 1 -- 172.21.15.162:0/2042699582 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4640055440 msgr2=0x7f4640057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:08.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.322+0000 7f46668b5700 1 --2- 172.21.15.162:0/2042699582 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4640055440 0x7f4640057900 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f465000d000 tx=0x7f4650009040).stop 2020-11-28T21:56:08.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.322+0000 7f46668b5700 1 -- 172.21.15.162:0/2042699582 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46600af480 msgr2=0x7f466012b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:08.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.322+0000 7f46668b5700 1 --2- 172.21.15.162:0/2042699582 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46600af480 0x7f466012b560 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 rx=0x7f466012d3e0 tx=0x7f465c007a40).stop 2020-11-28T21:56:08.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.322+0000 7f46668b5700 1 -- 172.21.15.162:0/2042699582 shutdown_connections 2020-11-28T21:56:08.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.322+0000 7f46668b5700 1 --2- 172.21.15.162:0/2042699582 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4640055440 0x7f4640057900 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:08.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.322+0000 7f46668b5700 1 --2- 172.21.15.162:0/2042699582 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f46600b2290 0x7f466012ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:08.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.322+0000 7f46668b5700 1 --2- 172.21.15.162:0/2042699582 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f46600a8550 0x7f4660134c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:08.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.322+0000 7f46668b5700 1 --2- 172.21.15.162:0/2042699582 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46600af480 0x7f466012b560 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:08.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.322+0000 7f46668b5700 1 -- 172.21.15.162:0/2042699582 >> 172.21.15.162:0/2042699582 conn(0x7f4660061320 msgr2=0x7f466005c380 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:08.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.322+0000 7f46668b5700 1 -- 172.21.15.162:0/2042699582 shutdown_connections 2020-11-28T21:56:08.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.322+0000 7f46668b5700 1 -- 172.21.15.162:0/2042699582 wait complete. 2020-11-28T21:56:08.344 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-28T21:55:59.967971+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '148', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2020-11-28T21:56:00.379272+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '148', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-28T21:56:08.345 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:56:08.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.482+0000 7fb17bfff700 1 Processor -- start 2020-11-28T21:56:08.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.482+0000 7fb17bfff700 1 -- start start 2020-11-28T21:56:08.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.486+0000 7fb17bfff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb17c061a50 0x7fb17c061300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:08.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.486+0000 7fb17bfff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fb17c0b21a0 0x7fb17c0abfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:08.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.486+0000 7fb17bfff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb17c05c090 con 0x7fb17c05bcb0 2020-11-28T21:56:08.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.486+0000 7fb17bfff700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb17c143820 con 0x7fb17c0b21a0 2020-11-28T21:56:08.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.486+0000 7fb17bfff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c0617f0 con 0x7fb17c061a50 2020-11-28T21:56:08.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.486+0000 7fb17b7fe700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fb17c0b21a0 0x7fb17c0abfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:08.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.486+0000 7fb17a7fc700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb17c061a50 0x7fb17c061300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:08.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.486+0000 7fb17a7fc700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb17c061a50 0x7fb17c061300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:43094/0 (socket says 172.21.15.162:43094) 2020-11-28T21:56:08.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.486+0000 7fb17b7fe700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fb17c0b21a0 0x7fb17c0abfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.162:52860/0 (socket says 172.21.15.162:52860) 2020-11-28T21:56:08.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.486+0000 7fb17a7fc700 1 -- 172.21.15.162:0/2301716514 learned_addr learned my addr 172.21.15.162:0/2301716514 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:08.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.486+0000 7fb17b7fe700 1 -- 172.21.15.162:0/2301716514 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb17c061a50 msgr2=0x7fb17c061300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:08.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.486+0000 7fb17b7fe700 1 --2- 172.21.15.162:0/2301716514 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb17c061a50 0x7fb17c061300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:08.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.486+0000 7fb17b7fe700 1 -- 172.21.15.162:0/2301716514 >> v1:172.21.15.118:6789/0 conn(0x7fb17c05bcb0 legacy=0x7fb17c1427d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:08.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.486+0000 7fb17b7fe700 1 -- 172.21.15.162:0/2301716514 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb17c061f60 con 0x7fb17c0b21a0 2020-11-28T21:56:08.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.486+0000 7fb17b7fe700 1 --2- 172.21.15.162:0/2301716514 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb17c0b21a0 0x7fb17c0abfb0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7fb1740032b0 tx=0x7fb1740091a0).ready entity=mon.0 client_cookie=3aaa2f9f0b90e06f server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:08.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.486+0000 7fb179ffb700 1 -- 172.21.15.162:0/2301716514 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb17400b9a0 con 0x7fb17c0b21a0 2020-11-28T21:56:08.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.486+0000 7fb179ffb700 1 -- 172.21.15.162:0/2301716514 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb17400bb00 con 0x7fb17c0b21a0 2020-11-28T21:56:08.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.486+0000 7fb17bfff700 1 -- 172.21.15.162:0/2301716514 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb17c0b21a0 msgr2=0x7fb17c0abfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:08.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.486+0000 7fb179ffb700 1 -- 172.21.15.162:0/2301716514 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb17400bc70 con 0x7fb17c0b21a0 2020-11-28T21:56:08.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.486+0000 7fb17bfff700 1 --2- 172.21.15.162:0/2301716514 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb17c0b21a0 0x7fb17c0abfb0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7fb1740032b0 tx=0x7fb1740091a0).stop 2020-11-28T21:56:08.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.486+0000 7fb17bfff700 1 -- 172.21.15.162:0/2301716514 shutdown_connections 2020-11-28T21:56:08.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.486+0000 7fb17bfff700 1 --2- 172.21.15.162:0/2301716514 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb17c0b21a0 0x7fb17c0abfb0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:08.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.486+0000 7fb17bfff700 1 --2- 172.21.15.162:0/2301716514 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb17c061a50 0x7fb17c061300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:08.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.486+0000 7fb17bfff700 1 -- 172.21.15.162:0/2301716514 >> 172.21.15.162:0/2301716514 conn(0x7fb17c0b2830 msgr2=0x7fb17c0b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:08.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.486+0000 7fb17bfff700 1 -- 172.21.15.162:0/2301716514 shutdown_connections 2020-11-28T21:56:08.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.486+0000 7fb17bfff700 1 -- 172.21.15.162:0/2301716514 wait complete. 2020-11-28T21:56:08.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.490+0000 7fb17bfff700 1 Processor -- start 2020-11-28T21:56:08.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.490+0000 7fb17bfff700 1 -- start start 2020-11-28T21:56:08.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.494+0000 7fb17bfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb17c05bcb0 0x7fb17c0afc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:08.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.494+0000 7fb17bfff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb17c061a50 0x7fb17c151430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:08.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.494+0000 7fb17bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb17c0b21a0 0x7fb17c151920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:08.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.494+0000 7fb17bfff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c0b4ea0 con 0x7fb17c0b21a0 2020-11-28T21:56:08.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.494+0000 7fb17bfff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c146100 con 0x7fb17c05bcb0 2020-11-28T21:56:08.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.494+0000 7fb17bfff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c1463a0 con 0x7fb17c061a50 2020-11-28T21:56:08.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.494+0000 7fb17affd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb17c05bcb0 0x7fb17c0afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:08.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.494+0000 7fb17affd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb17c05bcb0 0x7fb17c0afc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45874/0 (socket says 172.21.15.162:45874) 2020-11-28T21:56:08.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.494+0000 7fb17affd700 1 -- 172.21.15.162:0/1350695865 learned_addr learned my addr 172.21.15.162:0/1350695865 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:08.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.494+0000 7fb17a7fc700 1 --2- 172.21.15.162:0/1350695865 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb17c061a50 0x7fb17c151430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:08.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.494+0000 7fb17b7fe700 1 --2- 172.21.15.162:0/1350695865 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb17c0b21a0 0x7fb17c151920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:08.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.494+0000 7fb17affd700 1 -- 172.21.15.162:0/1350695865 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb17c061a50 msgr2=0x7fb17c151430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:08.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.494+0000 7fb17affd700 1 --2- 172.21.15.162:0/1350695865 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb17c061a50 0x7fb17c151430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:08.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.494+0000 7fb17affd700 1 -- 172.21.15.162:0/1350695865 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb17c0b21a0 msgr2=0x7fb17c151920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:08.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.494+0000 7fb17affd700 1 --2- 172.21.15.162:0/1350695865 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb17c0b21a0 0x7fb17c151920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:08.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.494+0000 7fb17affd700 1 -- 172.21.15.162:0/1350695865 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb17c140060 con 0x7fb17c05bcb0 2020-11-28T21:56:08.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.494+0000 7fb17a7fc700 1 --2- 172.21.15.162:0/1350695865 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb17c061a50 0x7fb17c151430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:56:08.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.494+0000 7fb17affd700 1 --2- 172.21.15.162:0/1350695865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb17c05bcb0 0x7fb17c0afc80 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 rx=0x7fb17c063390 tx=0x7fb170007be0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:08.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.494+0000 7fb16bfff700 1 -- 172.21.15.162:0/1350695865 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb170007240 con 0x7fb17c05bcb0 2020-11-28T21:56:08.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.494+0000 7fb16bfff700 1 -- 172.21.15.162:0/1350695865 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1700073a0 con 0x7fb17c05bcb0 2020-11-28T21:56:08.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.494+0000 7fb17bfff700 1 -- 172.21.15.162:0/1350695865 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb17c0ae8d0 con 0x7fb17c05bcb0 2020-11-28T21:56:08.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.494+0000 7fb16bfff700 1 -- 172.21.15.162:0/1350695865 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1700064e0 con 0x7fb17c05bcb0 2020-11-28T21:56:08.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.494+0000 7fb17bfff700 1 -- 172.21.15.162:0/1350695865 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb17c1469e0 con 0x7fb17c05bcb0 2020-11-28T21:56:08.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.494+0000 7fb17bfff700 1 -- 172.21.15.162:0/1350695865 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb17c0b4c60 con 0x7fb17c05bcb0 2020-11-28T21:56:08.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.494+0000 7fb16bfff700 1 -- 172.21.15.162:0/1350695865 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb170010070 con 0x7fb17c05bcb0 2020-11-28T21:56:08.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.498+0000 7fb16bfff700 1 --2- 172.21.15.162:0/1350695865 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb158055440 0x7fb158057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:08.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.498+0000 7fb16bfff700 1 -- 172.21.15.162:0/1350695865 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb17006d5b0 con 0x7fb17c05bcb0 2020-11-28T21:56:08.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.498+0000 7fb17a7fc700 1 --2- 172.21.15.162:0/1350695865 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb158055440 0x7fb158057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:08.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.498+0000 7fb17a7fc700 1 --2- 172.21.15.162:0/1350695865 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb158055440 0x7fb158057900 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7fb16c008000 tx=0x7fb16c008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:08.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.502+0000 7fb16bfff700 1 -- 172.21.15.162:0/1350695865 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb17c0b4c60 con 0x7fb17c05bcb0 2020-11-28T21:56:08.649 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.646+0000 7fb17bfff700 1 -- 172.21.15.162:0/1350695865 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fb17c0b4c60 con 0x7fb17c05bcb0 2020-11-28T21:56:08.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.646+0000 7fb16bfff700 1 -- 172.21.15.162:0/1350695865 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v148) v1 ==== 110+0+10 (secure 0 0 0) 0x7fb17c0b4c60 con 0x7fb17c05bcb0 2020-11-28T21:56:08.650 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:56:08.652 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.646+0000 7fb17bfff700 1 -- 172.21.15.162:0/1350695865 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb158055440 msgr2=0x7fb158057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:08.652 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.646+0000 7fb17bfff700 1 --2- 172.21.15.162:0/1350695865 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb158055440 0x7fb158057900 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7fb16c008000 tx=0x7fb16c008040).stop 2020-11-28T21:56:08.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.646+0000 7fb17bfff700 1 -- 172.21.15.162:0/1350695865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb17c05bcb0 msgr2=0x7fb17c0afc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:08.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.646+0000 7fb17bfff700 1 --2- 172.21.15.162:0/1350695865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb17c05bcb0 0x7fb17c0afc80 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 rx=0x7fb17c063390 tx=0x7fb170007be0).stop 2020-11-28T21:56:08.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.646+0000 7fb17bfff700 1 -- 172.21.15.162:0/1350695865 shutdown_connections 2020-11-28T21:56:08.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.646+0000 7fb17bfff700 1 --2- 172.21.15.162:0/1350695865 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb158055440 0x7fb158057900 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:08.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.646+0000 7fb17bfff700 1 --2- 172.21.15.162:0/1350695865 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb17c0b21a0 0x7fb17c151920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:08.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.646+0000 7fb17bfff700 1 --2- 172.21.15.162:0/1350695865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb17c05bcb0 0x7fb17c0afc80 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:08.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.646+0000 7fb17bfff700 1 --2- 172.21.15.162:0/1350695865 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb17c061a50 0x7fb17c151430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:08.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.646+0000 7fb17bfff700 1 -- 172.21.15.162:0/1350695865 >> 172.21.15.162:0/1350695865 conn(0x7fb17c0b2830 msgr2=0x7fb17c142cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:08.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.646+0000 7fb17bfff700 1 -- 172.21.15.162:0/1350695865 shutdown_connections 2020-11-28T21:56:08.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.646+0000 7fb17bfff700 1 -- 172.21.15.162:0/1350695865 wait complete. 2020-11-28T21:56:08.664 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:56:08.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.786+0000 7fd2bb1d3700 1 Processor -- start 2020-11-28T21:56:08.792 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.786+0000 7fd2bb1d3700 1 -- start start 2020-11-28T21:56:08.793 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.786+0000 7fd2bb1d3700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2b40b3ab0 0x7fd2b405c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:08.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.786+0000 7fd2bb1d3700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2b405baf0 con 0x7fd2b40b30e0 2020-11-28T21:56:08.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.790+0000 7fd2bb1d3700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2b405bd90 con 0x7fd2b40b2d00 2020-11-28T21:56:08.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.790+0000 7fd2bb1d3700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b40b0e20 con 0x7fd2b40b3ab0 2020-11-28T21:56:08.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.790+0000 7fd2ba1d1700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fd2b40b2d00 0x7fd2b4142ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:52726/0 (socket says 172.21.15.162:52726) 2020-11-28T21:56:08.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.790+0000 7fd2ba1d1700 1 -- 172.21.15.162:0/3054974995 learned_addr learned my addr 172.21.15.162:0/3054974995 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:56:08.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.790+0000 7fd2b99d0700 1 --2- 172.21.15.162:0/3054974995 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2b40b3ab0 0x7fd2b405c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:08.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.790+0000 7fd2b91cf700 1 -- 172.21.15.162:0/3054974995 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2186623021 0 0) 0x7fd2b405bd90 con 0x7fd2b40b2d00 2020-11-28T21:56:08.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.790+0000 7fd2b91cf700 1 -- 172.21.15.162:0/3054974995 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd29c003230 con 0x7fd2b40b2d00 2020-11-28T21:56:08.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.790+0000 7fd2b91cf700 1 -- 172.21.15.162:0/3054974995 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 812537832 0 0) 0x7fd2b405baf0 con 0x7fd2b40b30e0 2020-11-28T21:56:08.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.790+0000 7fd2b91cf700 1 -- 172.21.15.162:0/3054974995 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2b405bd90 con 0x7fd2b40b30e0 2020-11-28T21:56:08.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.790+0000 7fd2b99d0700 1 -- 172.21.15.162:0/3054974995 >> v1:172.21.15.162:6789/0 conn(0x7fd2b40b2d00 legacy=0x7fd2b4142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:08.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.790+0000 7fd2b99d0700 1 -- 172.21.15.162:0/3054974995 >> v1:172.21.15.118:6789/0 conn(0x7fd2b40b30e0 legacy=0x7fd2b413fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:08.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.790+0000 7fd2b99d0700 1 -- 172.21.15.162:0/3054974995 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b405c030 con 0x7fd2b40b3ab0 2020-11-28T21:56:08.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.790+0000 7fd2b99d0700 1 --2- 172.21.15.162:0/3054974995 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2b40b3ab0 0x7fd2b405c4d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fd2a8002f30 tx=0x7fd2a8006dc0).ready entity=mon.2 client_cookie=5df3f000fc554a42 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:08.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.790+0000 7fd2b91cf700 1 -- 172.21.15.162:0/3054974995 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2a8009730 con 0x7fd2b40b3ab0 2020-11-28T21:56:08.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.790+0000 7fd2b91cf700 1 -- 172.21.15.162:0/3054974995 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a8009890 con 0x7fd2b40b3ab0 2020-11-28T21:56:08.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.790+0000 7fd2b91cf700 1 -- 172.21.15.162:0/3054974995 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2a8009bf0 con 0x7fd2b40b3ab0 2020-11-28T21:56:08.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.790+0000 7fd2bb1d3700 1 -- 172.21.15.162:0/3054974995 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2b40b3ab0 msgr2=0x7fd2b405c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:08.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.790+0000 7fd2bb1d3700 1 --2- 172.21.15.162:0/3054974995 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2b40b3ab0 0x7fd2b405c4d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fd2a8002f30 tx=0x7fd2a8006dc0).stop 2020-11-28T21:56:08.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.790+0000 7fd2bb1d3700 1 -- 172.21.15.162:0/3054974995 shutdown_connections 2020-11-28T21:56:08.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.790+0000 7fd2bb1d3700 1 --2- 172.21.15.162:0/3054974995 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2b40b3ab0 0x7fd2b405c4d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:08.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.790+0000 7fd2bb1d3700 1 -- 172.21.15.162:0/3054974995 >> 172.21.15.162:0/3054974995 conn(0x7fd2b40604c0 msgr2=0x7fd2b40608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:08.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.790+0000 7fd2bb1d3700 1 -- 172.21.15.162:0/3054974995 shutdown_connections 2020-11-28T21:56:08.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.790+0000 7fd2bb1d3700 1 -- 172.21.15.162:0/3054974995 wait complete. 2020-11-28T21:56:08.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.790+0000 7fd2bb1d3700 1 Processor -- start 2020-11-28T21:56:08.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.790+0000 7fd2bb1d3700 1 -- start start 2020-11-28T21:56:08.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.794+0000 7fd2bb1d3700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2b40b2d00 0x7fd2b4146720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:08.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.794+0000 7fd2bb1d3700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2b40b30e0 0x7fd2b4151e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:08.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.794+0000 7fd2bb1d3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2b40b3ab0 0x7fd2b4154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:08.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.794+0000 7fd2bb1d3700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b4143c80 con 0x7fd2b40b30e0 2020-11-28T21:56:08.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.794+0000 7fd2bb1d3700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b4143ee0 con 0x7fd2b40b3ab0 2020-11-28T21:56:08.803 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.794+0000 7fd2bb1d3700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b4144180 con 0x7fd2b40b2d00 2020-11-28T21:56:08.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.794+0000 7fd2ba9d2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2b40b3ab0 0x7fd2b4154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:08.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.794+0000 7fd2ba9d2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2b40b3ab0 0x7fd2b4154c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45892/0 (socket says 172.21.15.162:45892) 2020-11-28T21:56:08.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.794+0000 7fd2ba9d2700 1 -- 172.21.15.162:0/2898893428 learned_addr learned my addr 172.21.15.162:0/2898893428 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:08.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.794+0000 7fd2b99d0700 1 --2- 172.21.15.162:0/2898893428 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2b40b30e0 0x7fd2b4151e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:08.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.794+0000 7fd2ba1d1700 1 --2- 172.21.15.162:0/2898893428 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2b40b2d00 0x7fd2b4146720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:08.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.794+0000 7fd2ba9d2700 1 -- 172.21.15.162:0/2898893428 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2b40b2d00 msgr2=0x7fd2b4146720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:08.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.794+0000 7fd2ba9d2700 1 --2- 172.21.15.162:0/2898893428 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2b40b2d00 0x7fd2b4146720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:08.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.794+0000 7fd2ba9d2700 1 -- 172.21.15.162:0/2898893428 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2b40b30e0 msgr2=0x7fd2b4151e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:08.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.794+0000 7fd2ba9d2700 1 --2- 172.21.15.162:0/2898893428 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2b40b30e0 0x7fd2b4151e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:08.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.794+0000 7fd2ba9d2700 1 -- 172.21.15.162:0/2898893428 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b41519d0 con 0x7fd2b40b3ab0 2020-11-28T21:56:08.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.794+0000 7fd2ba9d2700 1 --2- 172.21.15.162:0/2898893428 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2b40b3ab0 0x7fd2b4154c50 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 rx=0x7fd2ac001450 tx=0x7fd2ac007d60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:08.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.794+0000 7fd2a77fe700 1 -- 172.21.15.162:0/2898893428 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2ac00a5a0 con 0x7fd2b40b3ab0 2020-11-28T21:56:08.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.794+0000 7fd2a77fe700 1 -- 172.21.15.162:0/2898893428 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2ac00a700 con 0x7fd2b40b3ab0 2020-11-28T21:56:08.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.794+0000 7fd2bb1d3700 1 -- 172.21.15.162:0/2898893428 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2b4146160 con 0x7fd2b40b3ab0 2020-11-28T21:56:08.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.794+0000 7fd2a77fe700 1 -- 172.21.15.162:0/2898893428 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2ac00a870 con 0x7fd2b40b3ab0 2020-11-28T21:56:08.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.794+0000 7fd2bb1d3700 1 -- 172.21.15.162:0/2898893428 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b40b4120 con 0x7fd2b40b3ab0 2020-11-28T21:56:08.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.798+0000 7fd2bb1d3700 1 -- 172.21.15.162:0/2898893428 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2b41526a0 con 0x7fd2b40b3ab0 2020-11-28T21:56:08.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.798+0000 7fd2a77fe700 1 -- 172.21.15.162:0/2898893428 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd2ac010070 con 0x7fd2b40b3ab0 2020-11-28T21:56:08.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.798+0000 7fd2a77fe700 1 --2- 172.21.15.162:0/2898893428 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd294055330 0x7fd2940577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:08.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.798+0000 7fd2a77fe700 1 -- 172.21.15.162:0/2898893428 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd2ac06dca0 con 0x7fd2b40b3ab0 2020-11-28T21:56:08.808 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.798+0000 7fd2ba1d1700 1 --2- 172.21.15.162:0/2898893428 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd294055330 0x7fd2940577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:08.809 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.798+0000 7fd2ba1d1700 1 --2- 172.21.15.162:0/2898893428 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd294055330 0x7fd2940577f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7fd2b000d000 tx=0x7fd2b0009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:08.810 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.806+0000 7fd2a77fe700 1 -- 172.21.15.162:0/2898893428 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd2ac043e30 con 0x7fd2b40b3ab0 2020-11-28T21:56:08.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.946+0000 7fd2bb1d3700 1 -- 172.21.15.162:0/2898893428 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd2b4057050 con 0x7fd2b40b3ab0 2020-11-28T21:56:08.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.946+0000 7fd2a77fe700 1 -- 172.21.15.162:0/2898893428 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v148) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd2ac047450 con 0x7fd2b40b3ab0 2020-11-28T21:56:08.952 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:56:08.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.946+0000 7fd2bb1d3700 1 -- 172.21.15.162:0/2898893428 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd294055330 msgr2=0x7fd2940577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:08.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.946+0000 7fd2bb1d3700 1 --2- 172.21.15.162:0/2898893428 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd294055330 0x7fd2940577f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7fd2b000d000 tx=0x7fd2b0009040).stop 2020-11-28T21:56:08.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.946+0000 7fd2bb1d3700 1 -- 172.21.15.162:0/2898893428 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2b40b3ab0 msgr2=0x7fd2b4154c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:08.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.946+0000 7fd2bb1d3700 1 --2- 172.21.15.162:0/2898893428 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2b40b3ab0 0x7fd2b4154c50 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 rx=0x7fd2ac001450 tx=0x7fd2ac007d60).stop 2020-11-28T21:56:08.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.950+0000 7fd2bb1d3700 1 -- 172.21.15.162:0/2898893428 shutdown_connections 2020-11-28T21:56:08.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.950+0000 7fd2bb1d3700 1 --2- 172.21.15.162:0/2898893428 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd294055330 0x7fd2940577f0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:08.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.950+0000 7fd2bb1d3700 1 --2- 172.21.15.162:0/2898893428 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2b40b2d00 0x7fd2b4146720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:08.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.950+0000 7fd2bb1d3700 1 --2- 172.21.15.162:0/2898893428 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2b40b3ab0 0x7fd2b4154c50 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:08.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.950+0000 7fd2bb1d3700 1 --2- 172.21.15.162:0/2898893428 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2b40b30e0 0x7fd2b4151e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:08.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.950+0000 7fd2bb1d3700 1 -- 172.21.15.162:0/2898893428 >> 172.21.15.162:0/2898893428 conn(0x7fd2b40604c0 msgr2=0x7fd2b413ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:08.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.950+0000 7fd2bb1d3700 1 -- 172.21.15.162:0/2898893428 shutdown_connections 2020-11-28T21:56:08.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:08.950+0000 7fd2bb1d3700 1 -- 172.21.15.162:0/2898893428 wait complete. 2020-11-28T21:56:08.971 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-28T21:56:09.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.090+0000 7f0827fff700 1 Processor -- start 2020-11-28T21:56:09.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.090+0000 7f0827fff700 1 -- start start 2020-11-28T21:56:09.098 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f0827fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f082805f380 0x7f0828143190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:09.098 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f0827fff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08280aac00 con 0x7f08280b18c0 2020-11-28T21:56:09.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f0827fff700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0828060a10 con 0x7f08280b3c90 2020-11-28T21:56:09.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f0827fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f082813cf80 con 0x7f082805f380 2020-11-28T21:56:09.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f0826ffd700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f08280b3c90 0x7f0828142a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:52740/0 (socket says 172.21.15.162:52740) 2020-11-28T21:56:09.099 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f0826ffd700 1 -- 172.21.15.162:0/1784322616 learned_addr learned my addr 172.21.15.162:0/1784322616 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:56:09.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f08277fe700 1 --2- 172.21.15.162:0/1784322616 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f082805f380 0x7f0828143190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:09.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f0825ffb700 1 -- 172.21.15.162:0/1784322616 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1680631887 0 0) 0x7f0828060a10 con 0x7f08280b3c90 2020-11-28T21:56:09.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f0825ffb700 1 -- 172.21.15.162:0/1784322616 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f080c003230 con 0x7f08280b3c90 2020-11-28T21:56:09.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f0825ffb700 1 -- 172.21.15.162:0/1784322616 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 14081813 0 0) 0x7f08280aac00 con 0x7f08280b18c0 2020-11-28T21:56:09.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f0825ffb700 1 -- 172.21.15.162:0/1784322616 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0828060a10 con 0x7f08280b18c0 2020-11-28T21:56:09.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f0825ffb700 1 -- 172.21.15.162:0/1784322616 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4126839105 0 0) 0x7f080c003230 con 0x7f08280b3c90 2020-11-28T21:56:09.104 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f0825ffb700 1 -- 172.21.15.162:0/1784322616 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f082805f380 msgr2=0x7f0828143190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:09.104 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f0825ffb700 1 --2- 172.21.15.162:0/1784322616 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f082805f380 0x7f0828143190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:09.104 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f0825ffb700 1 -- 172.21.15.162:0/1784322616 >> v1:172.21.15.118:6789/0 conn(0x7f08280b18c0 legacy=0x7f0828146050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:09.104 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f0825ffb700 1 -- 172.21.15.162:0/1784322616 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08280b4070 con 0x7f08280b3c90 2020-11-28T21:56:09.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f0825ffb700 1 -- 172.21.15.162:0/1784322616 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f0818001a50 con 0x7f08280b3c90 2020-11-28T21:56:09.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f0825ffb700 1 -- 172.21.15.162:0/1784322616 >> v1:172.21.15.162:6789/0 conn(0x7f08280b3c90 legacy=0x7f0828142a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:09.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f0825ffb700 1 --2- 172.21.15.162:0/1784322616 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f080c0038b0 0x7f080c003cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:09.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f0825ffb700 1 --2- 172.21.15.162:0/1784322616 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f082805f380 0x7f080c0044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:09.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f0825ffb700 1 --2- 172.21.15.162:0/1784322616 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f080c0049c0 0x7f080c004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:09.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f0825ffb700 1 -- 172.21.15.162:0/1784322616 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080c005330 con 0x7f080c0049c0 2020-11-28T21:56:09.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f0825ffb700 1 -- 172.21.15.162:0/1784322616 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080c00b720 con 0x7f080c0038b0 2020-11-28T21:56:09.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f0825ffb700 1 -- 172.21.15.162:0/1784322616 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f080c00b8b0 con 0x7f082805f380 2020-11-28T21:56:09.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f0826ffd700 1 --2- 172.21.15.162:0/1784322616 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f080c0038b0 0x7f080c003cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:09.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f08277fe700 1 --2- 172.21.15.162:0/1784322616 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f080c0049c0 0x7f080c004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:09.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f08267fc700 1 --2- 172.21.15.162:0/1784322616 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f082805f380 0x7f080c0044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:09.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f0826ffd700 1 -- 172.21.15.162:0/1784322616 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f082805f380 msgr2=0x7f080c0044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:09.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f0826ffd700 1 --2- 172.21.15.162:0/1784322616 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f082805f380 0x7f080c0044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:09.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f0826ffd700 1 -- 172.21.15.162:0/1784322616 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f080c0049c0 msgr2=0x7f080c004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:09.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f0826ffd700 1 --2- 172.21.15.162:0/1784322616 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f080c0049c0 0x7f080c004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:09.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f0826ffd700 1 -- 172.21.15.162:0/1784322616 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f080c003230 con 0x7f080c0038b0 2020-11-28T21:56:09.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f0826ffd700 1 --2- 172.21.15.162:0/1784322616 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f080c0038b0 0x7f080c003cb0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 rx=0x7f080c0054a0 tx=0x7f0828147080).ready entity=mon.1 client_cookie=b5615148babca158 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:09.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f0825ffb700 1 -- 172.21.15.162:0/1784322616 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0818005b70 con 0x7f080c0038b0 2020-11-28T21:56:09.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f0825ffb700 1 -- 172.21.15.162:0/1784322616 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0818005cd0 con 0x7f080c0038b0 2020-11-28T21:56:09.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f0825ffb700 1 -- 172.21.15.162:0/1784322616 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f08180072c0 con 0x7f080c0038b0 2020-11-28T21:56:09.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f0827fff700 1 -- 172.21.15.162:0/1784322616 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f080c0038b0 msgr2=0x7f080c003cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:09.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f0827fff700 1 --2- 172.21.15.162:0/1784322616 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f080c0038b0 0x7f080c003cb0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 rx=0x7f080c0054a0 tx=0x7f0828147080).stop 2020-11-28T21:56:09.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f08277fe700 1 -- 172.21.15.162:0/1784322616 reap_dead start 2020-11-28T21:56:09.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f0827fff700 1 -- 172.21.15.162:0/1784322616 shutdown_connections 2020-11-28T21:56:09.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f0827fff700 1 -- 172.21.15.162:0/1784322616 >> 172.21.15.162:0/1784322616 conn(0x7f0828062630 msgr2=0x7f0828062a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:09.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f0827fff700 1 -- 172.21.15.162:0/1784322616 shutdown_connections 2020-11-28T21:56:09.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.094+0000 7f0827fff700 1 -- 172.21.15.162:0/1784322616 wait complete. 2020-11-28T21:56:09.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.098+0000 7f0827fff700 1 Processor -- start 2020-11-28T21:56:09.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.098+0000 7f0827fff700 1 -- start start 2020-11-28T21:56:09.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.098+0000 7f0827fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08280abff0 0x7f08280ac3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:09.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.098+0000 7f0827fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0828151f80 0x7f0828152400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:09.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.098+0000 7f0827fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f08280b09f0 0x7f08280b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:09.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.098+0000 7f0827fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0828143180 con 0x7f08280b09f0 2020-11-28T21:56:09.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.098+0000 7f0827fff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08281528f0 con 0x7f08280abff0 2020-11-28T21:56:09.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.098+0000 7f0827fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08281433d0 con 0x7f0828151f80 2020-11-28T21:56:09.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.098+0000 7f0826ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08280abff0 0x7f08280ac3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:09.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.098+0000 7f0826ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08280abff0 0x7f08280ac3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45908/0 (socket says 172.21.15.162:45908) 2020-11-28T21:56:09.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.098+0000 7f0826ffd700 1 -- 172.21.15.162:0/3449956813 learned_addr learned my addr 172.21.15.162:0/3449956813 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:09.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.098+0000 7f08267fc700 1 --2- 172.21.15.162:0/3449956813 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0828151f80 0x7f0828152400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:09.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.098+0000 7f08277fe700 1 --2- 172.21.15.162:0/3449956813 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f08280b09f0 0x7f08280b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:09.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.098+0000 7f0826ffd700 1 -- 172.21.15.162:0/3449956813 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0828151f80 msgr2=0x7f0828152400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:09.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.098+0000 7f0826ffd700 1 --2- 172.21.15.162:0/3449956813 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0828151f80 0x7f0828152400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:09.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.098+0000 7f0826ffd700 1 -- 172.21.15.162:0/3449956813 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f08280b09f0 msgr2=0x7f08280b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:09.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.098+0000 7f0826ffd700 1 --2- 172.21.15.162:0/3449956813 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f08280b09f0 0x7f08280b0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:09.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.098+0000 7f0826ffd700 1 -- 172.21.15.162:0/3449956813 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0828140f70 con 0x7f08280abff0 2020-11-28T21:56:09.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.102+0000 7f0826ffd700 1 --2- 172.21.15.162:0/3449956813 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08280abff0 0x7f08280ac3d0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 rx=0x7f080c0054a0 tx=0x7f0818013000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:09.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.102+0000 7f0817fff700 1 -- 172.21.15.162:0/3449956813 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f081801a070 con 0x7f08280abff0 2020-11-28T21:56:09.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.102+0000 7f0817fff700 1 -- 172.21.15.162:0/3449956813 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0818007720 con 0x7f08280abff0 2020-11-28T21:56:09.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.102+0000 7f0827fff700 1 -- 172.21.15.162:0/3449956813 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0828141210 con 0x7f08280abff0 2020-11-28T21:56:09.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.102+0000 7f0817fff700 1 -- 172.21.15.162:0/3449956813 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0818013640 con 0x7f08280abff0 2020-11-28T21:56:09.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.102+0000 7f0827fff700 1 -- 172.21.15.162:0/3449956813 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08280b1360 con 0x7f08280abff0 2020-11-28T21:56:09.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.102+0000 7f0827fff700 1 -- 172.21.15.162:0/3449956813 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0828140d70 con 0x7f08280abff0 2020-11-28T21:56:09.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.102+0000 7f0817fff700 1 -- 172.21.15.162:0/3449956813 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0818026070 con 0x7f08280abff0 2020-11-28T21:56:09.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.102+0000 7f0817fff700 1 --2- 172.21.15.162:0/3449956813 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0804055440 0x7f0804057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:09.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.102+0000 7f0817fff700 1 -- 172.21.15.162:0/3449956813 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f081806e880 con 0x7f08280abff0 2020-11-28T21:56:09.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.102+0000 7f08267fc700 1 --2- 172.21.15.162:0/3449956813 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0804055440 0x7f0804057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:09.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.102+0000 7f08267fc700 1 --2- 172.21.15.162:0/3449956813 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0804055440 0x7f0804057900 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7f081c00d000 tx=0x7f081c009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:09.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.110+0000 7f0817fff700 1 -- 172.21.15.162:0/3449956813 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0818044ae0 con 0x7f08280abff0 2020-11-28T21:56:09.201 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:56:09.204 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:56:09.207 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:56:09.209 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:56:09.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.254+0000 7f0827fff700 1 -- 172.21.15.162:0/3449956813 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0828057050 con 0x7f08280abff0 2020-11-28T21:56:09.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.254+0000 7f0817fff700 1 -- 172.21.15.162:0/3449956813 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v148) v1 ==== 104+0+10 (secure 0 0 0) 0x7f0818007890 con 0x7f08280abff0 2020-11-28T21:56:09.259 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:56:09.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.254+0000 7f0827fff700 1 -- 172.21.15.162:0/3449956813 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0804055440 msgr2=0x7f0804057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:09.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.254+0000 7f0827fff700 1 --2- 172.21.15.162:0/3449956813 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0804055440 0x7f0804057900 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7f081c00d000 tx=0x7f081c009040).stop 2020-11-28T21:56:09.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.254+0000 7f0827fff700 1 -- 172.21.15.162:0/3449956813 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08280abff0 msgr2=0x7f08280ac3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:09.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.254+0000 7f0827fff700 1 --2- 172.21.15.162:0/3449956813 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08280abff0 0x7f08280ac3d0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 rx=0x7f080c0054a0 tx=0x7f0818013000).stop 2020-11-28T21:56:09.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.254+0000 7f0827fff700 1 -- 172.21.15.162:0/3449956813 shutdown_connections 2020-11-28T21:56:09.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.254+0000 7f0827fff700 1 --2- 172.21.15.162:0/3449956813 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0804055440 0x7f0804057900 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:09.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.254+0000 7f0827fff700 1 --2- 172.21.15.162:0/3449956813 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f08280b09f0 0x7f08280b0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:09.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.254+0000 7f0827fff700 1 --2- 172.21.15.162:0/3449956813 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08280abff0 0x7f08280ac3d0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:09.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.254+0000 7f0827fff700 1 --2- 172.21.15.162:0/3449956813 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0828151f80 0x7f0828152400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:09.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.254+0000 7f0827fff700 1 -- 172.21.15.162:0/3449956813 >> 172.21.15.162:0/3449956813 conn(0x7f0828062630 msgr2=0x7f08280663c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:09.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.254+0000 7f0827fff700 1 -- 172.21.15.162:0/3449956813 shutdown_connections 2020-11-28T21:56:09.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.254+0000 7f0827fff700 1 -- 172.21.15.162:0/3449956813 wait complete. 2020-11-28T21:56:09.272 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-28T21:56:09.435 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.430+0000 7f0dad773700 1 Processor -- start 2020-11-28T21:56:09.435 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.430+0000 7f0dad773700 1 -- start start 2020-11-28T21:56:09.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.430+0000 7f0dad773700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0da8061a50 0x7f0da8061300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:09.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.430+0000 7f0dad773700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0da805c090 con 0x7f0da805bcb0 2020-11-28T21:56:09.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.430+0000 7f0dad773700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0da80617f0 con 0x7f0da80b21a0 2020-11-28T21:56:09.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.430+0000 7f0dad773700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0da8143820 con 0x7f0da8061a50 2020-11-28T21:56:09.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.430+0000 7f0dacf72700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f0da80b21a0 0x7f0da80abfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:52764/0 (socket says 172.21.15.162:52764) 2020-11-28T21:56:09.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.430+0000 7f0dacf72700 1 -- 172.21.15.162:0/1282357231 learned_addr learned my addr 172.21.15.162:0/1282357231 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:56:09.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.430+0000 7f0da77fe700 1 --2- 172.21.15.162:0/1282357231 >> v2:172.21.15.162:3300/0 conn(0x7f0da8061a50 0x7f0da8061300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:09.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.430+0000 7f0da6ffd700 1 -- 172.21.15.162:0/1282357231 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2849997361 0 0) 0x7f0da80617f0 con 0x7f0da80b21a0 2020-11-28T21:56:09.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.430+0000 7f0da6ffd700 1 -- 172.21.15.162:0/1282357231 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d88003230 con 0x7f0da80b21a0 2020-11-28T21:56:09.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.430+0000 7f0da6ffd700 1 -- 172.21.15.162:0/1282357231 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2339433092 0 0) 0x7f0da805c090 con 0x7f0da805bcb0 2020-11-28T21:56:09.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.434+0000 7f0da6ffd700 1 -- 172.21.15.162:0/1282357231 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0da80617f0 con 0x7f0da805bcb0 2020-11-28T21:56:09.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.434+0000 7f0da77fe700 1 -- 172.21.15.162:0/1282357231 >> v1:172.21.15.162:6789/0 conn(0x7f0da80b21a0 legacy=0x7f0da80abfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:09.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.434+0000 7f0da77fe700 1 -- 172.21.15.162:0/1282357231 >> v1:172.21.15.118:6789/0 conn(0x7f0da805bcb0 legacy=0x7f0da81427d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:09.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.434+0000 7f0da77fe700 1 -- 172.21.15.162:0/1282357231 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da8061f60 con 0x7f0da8061a50 2020-11-28T21:56:09.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.434+0000 7f0da77fe700 1 --2- 172.21.15.162:0/1282357231 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0da8061a50 0x7f0da8061300 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 rx=0x7f0d90002f30 tx=0x7f0d90006dc0).ready entity=mon.1 client_cookie=6ec98d59a9996241 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:09.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.434+0000 7f0da6ffd700 1 -- 172.21.15.162:0/1282357231 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d90009740 con 0x7f0da8061a50 2020-11-28T21:56:09.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.434+0000 7f0da6ffd700 1 -- 172.21.15.162:0/1282357231 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d900098a0 con 0x7f0da8061a50 2020-11-28T21:56:09.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.434+0000 7f0da6ffd700 1 -- 172.21.15.162:0/1282357231 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d90009c00 con 0x7f0da8061a50 2020-11-28T21:56:09.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.434+0000 7f0dad773700 1 -- 172.21.15.162:0/1282357231 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0da8061a50 msgr2=0x7f0da8061300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:09.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.434+0000 7f0dad773700 1 --2- 172.21.15.162:0/1282357231 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0da8061a50 0x7f0da8061300 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 rx=0x7f0d90002f30 tx=0x7f0d90006dc0).stop 2020-11-28T21:56:09.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.434+0000 7f0dad773700 1 -- 172.21.15.162:0/1282357231 shutdown_connections 2020-11-28T21:56:09.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.434+0000 7f0dad773700 1 --2- 172.21.15.162:0/1282357231 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0da8061a50 0x7f0da8061300 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:09.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.434+0000 7f0dad773700 1 -- 172.21.15.162:0/1282357231 >> 172.21.15.162:0/1282357231 conn(0x7f0da80b2830 msgr2=0x7f0da80b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:09.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.434+0000 7f0dad773700 1 -- 172.21.15.162:0/1282357231 shutdown_connections 2020-11-28T21:56:09.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.434+0000 7f0dad773700 1 -- 172.21.15.162:0/1282357231 wait complete. 2020-11-28T21:56:09.445 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.434+0000 7f0dad773700 1 Processor -- start 2020-11-28T21:56:09.445 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.434+0000 7f0dad773700 1 -- start start 2020-11-28T21:56:09.445 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.438+0000 7f0dad773700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0da805bcb0 0x7f0da8142460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:09.446 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.438+0000 7f0dad773700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0da8061a50 0x7f0da8136ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:09.446 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.438+0000 7f0dad773700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0da80b21a0 0x7f0da81373c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:09.446 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.438+0000 7f0dad773700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da80b0170 con 0x7f0da8061a50 2020-11-28T21:56:09.446 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.438+0000 7f0dad773700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da8146370 con 0x7f0da805bcb0 2020-11-28T21:56:09.446 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.438+0000 7f0dad773700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0da8146610 con 0x7f0da80b21a0 2020-11-28T21:56:09.447 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.438+0000 7f0da7fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0da805bcb0 0x7f0da8142460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:09.447 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.438+0000 7f0da7fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0da805bcb0 0x7f0da8142460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45922/0 (socket says 172.21.15.162:45922) 2020-11-28T21:56:09.447 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.438+0000 7f0da7fff700 1 -- 172.21.15.162:0/2930898188 learned_addr learned my addr 172.21.15.162:0/2930898188 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:09.447 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.438+0000 7f0da7fff700 1 -- 172.21.15.162:0/2930898188 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0da80b21a0 msgr2=0x7f0da81373c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:09.447 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.438+0000 7f0da7fff700 1 --2- 172.21.15.162:0/2930898188 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0da80b21a0 0x7f0da81373c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-28T21:56:09.448 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.438+0000 7f0da7fff700 1 -- 172.21.15.162:0/2930898188 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0da8061a50 msgr2=0x7f0da8136ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:09.448 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.438+0000 7f0da7fff700 1 --2- 172.21.15.162:0/2930898188 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0da8061a50 0x7f0da8136ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-28T21:56:09.448 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.438+0000 7f0da7fff700 1 -- 172.21.15.162:0/2930898188 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da81468b0 con 0x7f0da805bcb0 2020-11-28T21:56:09.448 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.438+0000 7f0da7fff700 1 --2- 172.21.15.162:0/2930898188 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0da805bcb0 0x7f0da8142460 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 rx=0x7f0da80af5d0 tx=0x7f0d98007ab0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:09.449 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.438+0000 7f0da57fa700 1 -- 172.21.15.162:0/2930898188 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d98010510 con 0x7f0da805bcb0 2020-11-28T21:56:09.449 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.438+0000 7f0da57fa700 1 -- 172.21.15.162:0/2930898188 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d98010670 con 0x7f0da805bcb0 2020-11-28T21:56:09.449 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.438+0000 7f0dad773700 1 -- 172.21.15.162:0/2930898188 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0da8146b50 con 0x7f0da805bcb0 2020-11-28T21:56:09.449 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.438+0000 7f0da57fa700 1 -- 172.21.15.162:0/2930898188 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d980107e0 con 0x7f0da805bcb0 2020-11-28T21:56:09.450 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.438+0000 7f0dad773700 1 -- 172.21.15.162:0/2930898188 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0da814f1e0 con 0x7f0da805bcb0 2020-11-28T21:56:09.450 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.438+0000 7f0dad773700 1 -- 172.21.15.162:0/2930898188 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0da8057050 con 0x7f0da805bcb0 2020-11-28T21:56:09.450 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.438+0000 7f0da57fa700 1 -- 172.21.15.162:0/2930898188 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0d98010070 con 0x7f0da805bcb0 2020-11-28T21:56:09.451 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.442+0000 7f0da57fa700 1 --2- 172.21.15.162:0/2930898188 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0d8c055440 0x7f0d8c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:09.451 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.442+0000 7f0da77fe700 1 --2- 172.21.15.162:0/2930898188 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0d8c055440 0x7f0d8c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:09.451 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.442+0000 7f0da57fa700 1 -- 172.21.15.162:0/2930898188 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0d9806d410 con 0x7f0da805bcb0 2020-11-28T21:56:09.452 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.442+0000 7f0da77fe700 1 --2- 172.21.15.162:0/2930898188 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0d8c055440 0x7f0d8c057900 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f0d90002f30 tx=0x7f0d90015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:09.452 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.446+0000 7f0da57fa700 1 -- 172.21.15.162:0/2930898188 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0d98041df0 con 0x7f0da805bcb0 2020-11-28T21:56:09.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.586+0000 7f0dad773700 1 -- 172.21.15.162:0/2930898188 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0da8068350 con 0x7f0da805bcb0 2020-11-28T21:56:09.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.586+0000 7f0da57fa700 1 -- 172.21.15.162:0/2930898188 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v148) v1 ==== 100+0+10 (secure 0 0 0) 0x7f0d98043710 con 0x7f0da805bcb0 2020-11-28T21:56:09.592 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:56:09.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.586+0000 7f0dad773700 1 -- 172.21.15.162:0/2930898188 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0d8c055440 msgr2=0x7f0d8c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:09.594 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.586+0000 7f0dad773700 1 --2- 172.21.15.162:0/2930898188 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0d8c055440 0x7f0d8c057900 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f0d90002f30 tx=0x7f0d90015000).stop 2020-11-28T21:56:09.594 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.586+0000 7f0dad773700 1 -- 172.21.15.162:0/2930898188 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0da805bcb0 msgr2=0x7f0da8142460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:09.594 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.586+0000 7f0dad773700 1 --2- 172.21.15.162:0/2930898188 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0da805bcb0 0x7f0da8142460 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 rx=0x7f0da80af5d0 tx=0x7f0d98007ab0).stop 2020-11-28T21:56:09.594 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.590+0000 7f0dad773700 1 -- 172.21.15.162:0/2930898188 shutdown_connections 2020-11-28T21:56:09.594 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.590+0000 7f0dad773700 1 --2- 172.21.15.162:0/2930898188 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0d8c055440 0x7f0d8c057900 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:09.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.590+0000 7f0dad773700 1 --2- 172.21.15.162:0/2930898188 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0da80b21a0 0x7f0da81373c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-28T21:56:09.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.590+0000 7f0dad773700 1 --2- 172.21.15.162:0/2930898188 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0da805bcb0 0x7f0da8142460 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:09.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.590+0000 7f0dad773700 1 --2- 172.21.15.162:0/2930898188 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0da8061a50 0x7f0da8136ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-28T21:56:09.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.590+0000 7f0dad773700 1 -- 172.21.15.162:0/2930898188 >> 172.21.15.162:0/2930898188 conn(0x7f0da80b2830 msgr2=0x7f0da813b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:09.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.590+0000 7f0dad773700 1 -- 172.21.15.162:0/2930898188 shutdown_connections 2020-11-28T21:56:09.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.590+0000 7f0dad773700 1 -- 172.21.15.162:0/2930898188 wait complete. 2020-11-28T21:56:09.605 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-28T21:56:09.606 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:56:09.718 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb08d764700 1 Processor -- start 2020-11-28T21:56:09.719 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb08d764700 1 -- start start 2020-11-28T21:56:09.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb08d764700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb0880611e0 0x7fb0880615c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:09.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb08d764700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb0880b4b40 0x7fb08813ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:09.722 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb08d764700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0880b1790 con 0x7fb0880b10f0 2020-11-28T21:56:09.723 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb08d764700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0880b1580 con 0x7fb0880611e0 2020-11-28T21:56:09.723 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb08d764700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0880b1a30 con 0x7fb0880b4b40 2020-11-28T21:56:09.723 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb0877fe700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fb0880b10f0 0x7fb0880b4430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:52776/0 (socket says 172.21.15.162:52776) 2020-11-28T21:56:09.723 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb0877fe700 1 -- 172.21.15.162:0/334267990 learned_addr learned my addr 172.21.15.162:0/334267990 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:56:09.723 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb087fff700 1 --2- 172.21.15.162:0/334267990 >> v2:172.21.15.162:3300/0 conn(0x7fb0880611e0 0x7fb0880615c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:09.724 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb08cf63700 1 --2- 172.21.15.162:0/334267990 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb0880b4b40 0x7fb08813ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:09.724 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb086ffd700 1 -- 172.21.15.162:0/334267990 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3061161674 0 0) 0x7fb0880b1790 con 0x7fb0880b10f0 2020-11-28T21:56:09.724 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb086ffd700 1 -- 172.21.15.162:0/334267990 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb074003230 con 0x7fb0880b10f0 2020-11-28T21:56:09.725 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb086ffd700 1 -- 172.21.15.162:0/334267990 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3672345040 0 0) 0x7fb074003230 con 0x7fb0880b10f0 2020-11-28T21:56:09.725 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb086ffd700 1 -- 172.21.15.162:0/334267990 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb0880b4b40 msgr2=0x7fb08813ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:09.725 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb086ffd700 1 --2- 172.21.15.162:0/334267990 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb0880b4b40 0x7fb08813ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:09.725 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb086ffd700 1 -- 172.21.15.162:0/334267990 >> v2:172.21.15.162:3300/0 conn(0x7fb0880611e0 msgr2=0x7fb0880615c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:09.725 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb086ffd700 1 --2- 172.21.15.162:0/334267990 >> v2:172.21.15.162:3300/0 conn(0x7fb0880611e0 0x7fb0880615c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:09.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb086ffd700 1 -- 172.21.15.162:0/334267990 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0880b3f90 con 0x7fb0880b10f0 2020-11-28T21:56:09.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb086ffd700 1 -- 172.21.15.162:0/334267990 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7fb080001a50 con 0x7fb0880b10f0 2020-11-28T21:56:09.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb087fff700 1 --2- 172.21.15.162:0/334267990 >> v2:172.21.15.162:3300/0 conn(0x7fb0880611e0 0x7fb0880615c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:56:09.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb086ffd700 1 -- 172.21.15.162:0/334267990 >> v1:172.21.15.162:6789/0 conn(0x7fb0880b10f0 legacy=0x7fb0880b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:09.726 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb086ffd700 1 --2- 172.21.15.162:0/334267990 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb0880b4b40 0x7fb074003720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:09.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb086ffd700 1 --2- 172.21.15.162:0/334267990 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb074003c30 0x7fb0740044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:09.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb086ffd700 1 --2- 172.21.15.162:0/334267990 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0740049c0 0x7fb074004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:09.727 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb086ffd700 1 -- 172.21.15.162:0/334267990 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0880b1580 con 0x7fb074003c30 2020-11-28T21:56:09.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb086ffd700 1 -- 172.21.15.162:0/334267990 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0880b1a30 con 0x7fb0740049c0 2020-11-28T21:56:09.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb086ffd700 1 -- 172.21.15.162:0/334267990 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0740041f0 con 0x7fb0880b4b40 2020-11-28T21:56:09.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb08cf63700 1 --2- 172.21.15.162:0/334267990 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0740049c0 0x7fb074004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:09.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb0877fe700 1 --2- 172.21.15.162:0/334267990 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb074003c30 0x7fb0740044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:09.728 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb087fff700 1 --2- 172.21.15.162:0/334267990 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb0880b4b40 0x7fb074003720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:09.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb08cf63700 1 -- 172.21.15.162:0/334267990 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb0880b4b40 msgr2=0x7fb074003720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:09.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb08cf63700 1 --2- 172.21.15.162:0/334267990 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb0880b4b40 0x7fb074003720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:09.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb08cf63700 1 -- 172.21.15.162:0/334267990 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb074003c30 msgr2=0x7fb0740044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:09.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb08cf63700 1 --2- 172.21.15.162:0/334267990 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb074003c30 0x7fb0740044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:09.729 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb08cf63700 1 -- 172.21.15.162:0/334267990 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb074003230 con 0x7fb0740049c0 2020-11-28T21:56:09.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb087fff700 1 --2- 172.21.15.162:0/334267990 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb0880b4b40 0x7fb074003720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:56:09.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb0877fe700 1 --2- 172.21.15.162:0/334267990 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb074003c30 0x7fb0740044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:56:09.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb08cf63700 1 --2- 172.21.15.162:0/334267990 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0740049c0 0x7fb074004e40 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 rx=0x7fb07c00c000 tx=0x7fb07c008040).ready entity=mon.1 client_cookie=9f2e58edbae9ed9f server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:09.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb086ffd700 1 -- 172.21.15.162:0/334267990 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb07c00e9d0 con 0x7fb0740049c0 2020-11-28T21:56:09.730 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb086ffd700 1 -- 172.21.15.162:0/334267990 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb07c00eb30 con 0x7fb0740049c0 2020-11-28T21:56:09.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb086ffd700 1 -- 172.21.15.162:0/334267990 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb07c00ee90 con 0x7fb0740049c0 2020-11-28T21:56:09.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb08d764700 1 -- 172.21.15.162:0/334267990 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0740049c0 msgr2=0x7fb074004e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:09.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.714+0000 7fb08d764700 1 --2- 172.21.15.162:0/334267990 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0740049c0 0x7fb074004e40 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 rx=0x7fb07c00c000 tx=0x7fb07c008040).stop 2020-11-28T21:56:09.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.718+0000 7fb08cf63700 1 -- 172.21.15.162:0/334267990 reap_dead start 2020-11-28T21:56:09.731 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.718+0000 7fb08d764700 1 -- 172.21.15.162:0/334267990 shutdown_connections 2020-11-28T21:56:09.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.718+0000 7fb08d764700 1 -- 172.21.15.162:0/334267990 >> 172.21.15.162:0/334267990 conn(0x7fb0880b54f0 msgr2=0x7fb0880b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:09.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.718+0000 7fb08d764700 1 -- 172.21.15.162:0/334267990 shutdown_connections 2020-11-28T21:56:09.732 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.718+0000 7fb08d764700 1 -- 172.21.15.162:0/334267990 wait complete. 2020-11-28T21:56:09.735 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.718+0000 7fb08d764700 1 Processor -- start 2020-11-28T21:56:09.735 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.718+0000 7fb08d764700 1 -- start start 2020-11-28T21:56:09.735 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.722+0000 7fb08d764700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb08809b0b0 0x7fb0880ad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:09.735 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.722+0000 7fb08d764700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb088063d80 0x7fb0880ad9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:09.735 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.722+0000 7fb08d764700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb088152b40 0x7fb088152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:09.736 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.722+0000 7fb08d764700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0880aeb60 con 0x7fb088063d80 2020-11-28T21:56:09.736 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.722+0000 7fb08d764700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08809b490 con 0x7fb08809b0b0 2020-11-28T21:56:09.736 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.722+0000 7fb08d764700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0880670d0 con 0x7fb088152b40 2020-11-28T21:56:09.736 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.722+0000 7fb087fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb08809b0b0 0x7fb0880ad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:09.736 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.722+0000 7fb0877fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb088063d80 0x7fb0880ad9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:09.737 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.722+0000 7fb087fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb08809b0b0 0x7fb0880ad4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45944/0 (socket says 172.21.15.162:45944) 2020-11-28T21:56:09.737 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.722+0000 7fb087fff700 1 -- 172.21.15.162:0/3661377236 learned_addr learned my addr 172.21.15.162:0/3661377236 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:09.737 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.722+0000 7fb08cf63700 1 --2- 172.21.15.162:0/3661377236 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb088152b40 0x7fb088152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:09.737 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.722+0000 7fb087fff700 1 -- 172.21.15.162:0/3661377236 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb088152b40 msgr2=0x7fb088152fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:09.737 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.722+0000 7fb087fff700 1 --2- 172.21.15.162:0/3661377236 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb088152b40 0x7fb088152fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:09.738 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.722+0000 7fb087fff700 1 -- 172.21.15.162:0/3661377236 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb088063d80 msgr2=0x7fb0880ad9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:09.738 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.722+0000 7fb087fff700 1 --2- 172.21.15.162:0/3661377236 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb088063d80 0x7fb0880ad9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:09.738 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.722+0000 7fb087fff700 1 -- 172.21.15.162:0/3661377236 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb088067360 con 0x7fb08809b0b0 2020-11-28T21:56:09.738 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.722+0000 7fb087fff700 1 --2- 172.21.15.162:0/3661377236 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb08809b0b0 0x7fb0880ad4b0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 rx=0x7fb0880641a0 tx=0x7fb07800b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:09.739 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.722+0000 7fb0857fa700 1 -- 172.21.15.162:0/3661377236 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb078013800 con 0x7fb08809b0b0 2020-11-28T21:56:09.739 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.722+0000 7fb0857fa700 1 -- 172.21.15.162:0/3661377236 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb078013960 con 0x7fb08809b0b0 2020-11-28T21:56:09.739 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.722+0000 7fb08d764700 1 -- 172.21.15.162:0/3661377236 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb088067670 con 0x7fb08809b0b0 2020-11-28T21:56:09.739 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.722+0000 7fb0857fa700 1 -- 172.21.15.162:0/3661377236 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb078013cc0 con 0x7fb08809b0b0 2020-11-28T21:56:09.739 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.722+0000 7fb08d764700 1 -- 172.21.15.162:0/3661377236 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb088067910 con 0x7fb08809b0b0 2020-11-28T21:56:09.740 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.722+0000 7fb08d764700 1 -- 172.21.15.162:0/3661377236 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0880aeca0 con 0x7fb08809b0b0 2020-11-28T21:56:09.740 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.726+0000 7fb0857fa700 1 -- 172.21.15.162:0/3661377236 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb07801b3b0 con 0x7fb08809b0b0 2020-11-28T21:56:09.740 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.726+0000 7fb0857fa700 1 --2- 172.21.15.162:0/3661377236 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb0680553b0 0x7fb068057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:09.740 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.726+0000 7fb0857fa700 1 -- 172.21.15.162:0/3661377236 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb07806e240 con 0x7fb08809b0b0 2020-11-28T21:56:09.740 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.726+0000 7fb0877fe700 1 --2- 172.21.15.162:0/3661377236 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb0680553b0 0x7fb068057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:09.741 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.726+0000 7fb0877fe700 1 --2- 172.21.15.162:0/3661377236 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb0680553b0 0x7fb068057870 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7fb08000e000 tx=0x7fb080009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:09.741 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.730+0000 7fb0857fa700 1 -- 172.21.15.162:0/3661377236 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb0780444e0 con 0x7fb08809b0b0 2020-11-28T21:56:09.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.914+0000 7fb08d764700 1 -- 172.21.15.162:0/3661377236 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb088057050 con 0x7fb08809b0b0 2020-11-28T21:56:09.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.914+0000 7fb0857fa700 1 -- 172.21.15.162:0/3661377236 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+2504 (secure 0 0 0) 0x7fb078017020 con 0x7fb08809b0b0 2020-11-28T21:56:09.920 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:56:09.920 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":74,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6899,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.118:6833/2509028245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":2509028245},{"type":"v1","addr":"172.21.15.118:6833","nonce":2509028245}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":72},{"gid":8239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/2089791609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2089791609},{"type":"v1","addr":"172.21.15.162:6836","nonce":2089791609}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":71}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:56:01.362652+0000","modified":"2020-11-28T21:56:03.027360+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8242},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8242":{"gid":8242,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/2832109513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":2832109513},{"type":"v1","addr":"172.21.15.162:6837","nonce":2832109513}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2020-11-28T21:56:09.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.914+0000 7fb08d764700 1 -- 172.21.15.162:0/3661377236 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb0680553b0 msgr2=0x7fb068057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:09.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.914+0000 7fb08d764700 1 --2- 172.21.15.162:0/3661377236 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb0680553b0 0x7fb068057870 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7fb08000e000 tx=0x7fb080009040).stop 2020-11-28T21:56:09.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.914+0000 7fb08d764700 1 -- 172.21.15.162:0/3661377236 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb08809b0b0 msgr2=0x7fb0880ad4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:09.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.914+0000 7fb08d764700 1 --2- 172.21.15.162:0/3661377236 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb08809b0b0 0x7fb0880ad4b0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 rx=0x7fb0880641a0 tx=0x7fb07800b040).stop 2020-11-28T21:56:09.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.914+0000 7fb08d764700 1 -- 172.21.15.162:0/3661377236 shutdown_connections 2020-11-28T21:56:09.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.914+0000 7fb08d764700 1 --2- 172.21.15.162:0/3661377236 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb0680553b0 0x7fb068057870 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:09.922 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.914+0000 7fb08d764700 1 --2- 172.21.15.162:0/3661377236 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb088152b40 0x7fb088152fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:09.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.914+0000 7fb08d764700 1 --2- 172.21.15.162:0/3661377236 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb08809b0b0 0x7fb0880ad4b0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:09.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.914+0000 7fb08d764700 1 --2- 172.21.15.162:0/3661377236 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb088063d80 0x7fb0880ad9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:09.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.914+0000 7fb08d764700 1 -- 172.21.15.162:0/3661377236 >> 172.21.15.162:0/3661377236 conn(0x7fb0880b54f0 msgr2=0x7fb0880ab200 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:09.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.914+0000 7fb08d764700 1 -- 172.21.15.162:0/3661377236 shutdown_connections 2020-11-28T21:56:09.923 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:09.918+0000 7fb08d764700 1 -- 172.21.15.162:0/3661377236 wait complete. 2020-11-28T21:56:09.924 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 74 2020-11-28T21:56:10.236 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-28T21:56:01.362652+0000', 'modified': '2020-11-28T21:56:03.027360+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8242}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8242': {'gid': 8242, 'name': 'a', 'rank': 0, 'incarnation': 72, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6837/2832109513', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6835', 'nonce': 2832109513}, {'type': 'v1', 'addr': '172.21.15.162:6837', 'nonce': 2832109513}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-28T21:56:10.237 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-28T21:56:10.237 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-28T21:56:10.369 INFO:teuthology.orchestra.run.smithi162.stdout:{"cluster_fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","whoami":0,"id":8242,"want_state":"up:active","state":"up:active","rank_uptime":8.9927036529999995,"mdsmap_epoch":74,"osdmap_epoch":148,"osdmap_epoch_barrier":148,"uptime":13.289367123} 2020-11-28T21:56:10.370 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", "id": 8242, "mdsmap_epoch": 74, "osdmap_epoch": 148, "osdmap_epoch_barrier": 148, "rank_uptime": 8.992703653, "state": "up:active", "uptime": 13.289367123, "want_state": "up:active", "whoami": 0 } 2020-11-28T21:56:10.370 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-11-28T21:56:10.371 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2020-11-28T21:56:10.371 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi096.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2020-11-28T21:56:10.372 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:56:10.372 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:56:10.380 INFO:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2020-11-28T21:56:10.434 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:56:10.435 INFO:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:56:10.491 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:56:10.492 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:56:10.492 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:56:10.493 INFO:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2020-11-28T21:56:10.539 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-11-28T21:56:10.539 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:56:10.582 INFO:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2020-11-28T21:56:10.597 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:56:10.598 INFO:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:56:10.656 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:56:10.657 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:56:10.658 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:56:10.658 INFO:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2020-11-28T21:56:10.659 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:2020-11-28T21:56:10.655+0000 7f6540cd70c0 -1 init, newargv = 0x559487087d30 newargc=9ceph-fuse[12186]: starting ceph client 2020-11-28T21:56:10.660 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr: 2020-11-28T21:56:10.669 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[12186]: starting fuse 2020-11-28T21:56:11.764 INFO:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2020-11-28T21:56:11.779 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:56:11.779 INFO:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:56:11.835 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:56:11.836 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:56:11.837 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:56:11.837 INFO:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2020-11-28T21:56:11.882 INFO:teuthology.orchestra.run.smithi096.stdout:49 2020-11-28T21:56:11.883 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2020-11-28T21:56:11.883 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:56:11.884 INFO:teuthology.orchestra.run.smithi096:> sudo python3 -c ' 2020-11-28T21:56:11.884 INFO:teuthology.orchestra.run.smithi096:> import glob 2020-11-28T21:56:11.885 INFO:teuthology.orchestra.run.smithi096:> import re 2020-11-28T21:56:11.885 INFO:teuthology.orchestra.run.smithi096:> import os 2020-11-28T21:56:11.885 INFO:teuthology.orchestra.run.smithi096:> import subprocess 2020-11-28T21:56:11.886 INFO:teuthology.orchestra.run.smithi096:> 2020-11-28T21:56:11.886 INFO:teuthology.orchestra.run.smithi096:> def find_socket(client_name): 2020-11-28T21:56:11.887 INFO:teuthology.orchestra.run.smithi096:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2020-11-28T21:56:11.887 INFO:teuthology.orchestra.run.smithi096:> files = glob.glob(asok_path) 2020-11-28T21:56:11.887 INFO:teuthology.orchestra.run.smithi096:> 2020-11-28T21:56:11.888 INFO:teuthology.orchestra.run.smithi096:> # Given a non-glob path, it better be there 2020-11-28T21:56:11.888 INFO:teuthology.orchestra.run.smithi096:> if "*" not in asok_path: 2020-11-28T21:56:11.888 INFO:teuthology.orchestra.run.smithi096:> assert(len(files) == 1) 2020-11-28T21:56:11.889 INFO:teuthology.orchestra.run.smithi096:> return files[0] 2020-11-28T21:56:11.889 INFO:teuthology.orchestra.run.smithi096:> 2020-11-28T21:56:11.890 INFO:teuthology.orchestra.run.smithi096:> for f in files: 2020-11-28T21:56:11.890 INFO:teuthology.orchestra.run.smithi096:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-11-28T21:56:11.890 INFO:teuthology.orchestra.run.smithi096:> if os.path.exists("/proc/{0}".format(pid)): 2020-11-28T21:56:11.891 INFO:teuthology.orchestra.run.smithi096:> return f 2020-11-28T21:56:11.891 INFO:teuthology.orchestra.run.smithi096:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-11-28T21:56:11.892 INFO:teuthology.orchestra.run.smithi096:> 2020-11-28T21:56:11.892 INFO:teuthology.orchestra.run.smithi096:> print(find_socket("client.1")) 2020-11-28T21:56:11.892 INFO:teuthology.orchestra.run.smithi096:> ' 2020-11-28T21:56:11.976 INFO:teuthology.orchestra.run.smithi096.stdout:/var/run/ceph/ceph-client.1.12186.asok 2020-11-28T21:56:11.981 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.12186.asok 2020-11-28T21:56:11.981 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:56:11.982 INFO:teuthology.orchestra.run.smithi096:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.12186.asok status 2020-11-28T21:56:12.077 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-11-28T21:56:12.078 INFO:teuthology.orchestra.run.smithi096.stdout: "metadata": { 2020-11-28T21:56:12.078 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_sha1": "a83a3572a957356977342f461361c1cd3ec06dbd", 2020-11-28T21:56:12.078 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_version": "ceph version 15.2.6-595-ga83a3572 (a83a3572a957356977342f461361c1cd3ec06dbd) octopus (stable)", 2020-11-28T21:56:12.078 INFO:teuthology.orchestra.run.smithi096.stdout: "entity_id": "1", 2020-11-28T21:56:12.079 INFO:teuthology.orchestra.run.smithi096.stdout: "hostname": "smithi096", 2020-11-28T21:56:12.079 INFO:teuthology.orchestra.run.smithi096.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2020-11-28T21:56:12.079 INFO:teuthology.orchestra.run.smithi096.stdout: "pid": "12186", 2020-11-28T21:56:12.079 INFO:teuthology.orchestra.run.smithi096.stdout: "root": "/" 2020-11-28T21:56:12.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-11-28T21:56:12.080 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_count": 0, 2020-11-28T21:56:12.080 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_pinned_count": 0, 2020-11-28T21:56:12.080 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 8458, 2020-11-28T21:56:12.080 INFO:teuthology.orchestra.run.smithi096.stdout: "inst": { 2020-11-28T21:56:12.080 INFO:teuthology.orchestra.run.smithi096.stdout: "name": { 2020-11-28T21:56:12.081 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "client", 2020-11-28T21:56:12.081 INFO:teuthology.orchestra.run.smithi096.stdout: "num": 8458 2020-11-28T21:56:12.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-11-28T21:56:12.081 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2020-11-28T21:56:12.081 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2020-11-28T21:56:12.082 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2020-11-28T21:56:12.082 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2706527597 2020-11-28T21:56:12.082 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-11-28T21:56:12.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-11-28T21:56:12.082 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2020-11-28T21:56:12.083 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2020-11-28T21:56:12.083 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2020-11-28T21:56:12.083 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2706527597 2020-11-28T21:56:12.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-11-28T21:56:12.083 INFO:teuthology.orchestra.run.smithi096.stdout: "inst_str": "client.8458 172.21.15.96:0/2706527597", 2020-11-28T21:56:12.084 INFO:teuthology.orchestra.run.smithi096.stdout: "addr_str": "172.21.15.96:0/2706527597", 2020-11-28T21:56:12.084 INFO:teuthology.orchestra.run.smithi096.stdout: "inode_count": 1, 2020-11-28T21:56:12.084 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_epoch": 74, 2020-11-28T21:56:12.084 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch": 148, 2020-11-28T21:56:12.084 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch_barrier": 0, 2020-11-28T21:56:12.085 INFO:teuthology.orchestra.run.smithi096.stdout: "blacklisted": false 2020-11-28T21:56:12.085 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-11-28T21:56:12.087 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:56:12.087 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:56:12.088 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:56:12.094 INFO:teuthology.orchestra.run.smithi096.stdout:fuseblk 2020-11-28T21:56:12.095 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-11-28T21:56:12.095 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:56:12.096 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:56:12.153 INFO:teuthology.orchestra.run.smithi192:> cd /home/ubuntu/cephtest/mnt.0 && sudo getfattr --only-values -n ceph.dir.rctime . 2020-11-28T21:56:12.186 INFO:teuthology.orchestra.run.smithi192.stdout:1606600572.149228506 2020-11-28T21:56:12.187 INFO:tasks.cephfs.test_misc:rctime = 1606600572.149228506 2020-11-28T21:56:12.187 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-28T21:56:12.188 INFO:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:56:12.213 INFO:teuthology.orchestra.run.smithi118:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:56:12.232 INFO:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:56:12.250 INFO:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:56:12.268 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-28T21:56:12.268 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:56:12.269 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:56:12.269 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:56:12.319 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2020-11-28T21:56:12.320 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-28T21:56:12.320 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2020-11-28T21:56:12.321 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-28T21:56:12.321 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:56:12.396 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:ceph-fuse[12727]: fuse finished with error 0 and tester_r 0 2020-11-28T21:56:12.398 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:56:12.399 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:56:12.399 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:56:12.446 INFO:teuthology.orchestra.run.smithi192.stdout:ext2/ext3 2020-11-28T21:56:12.447 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-11-28T21:56:12.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-28T21:56:12.502 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:56:12.509 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:56:12.509 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:56:12.510 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:56:12.518 INFO:teuthology.orchestra.run.smithi096.stdout:fuseblk 2020-11-28T21:56:12.519 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-11-28T21:56:12.519 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi096.front.sepia.ceph.com... 2020-11-28T21:56:12.520 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-28T21:56:12.520 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:56:12.605 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[12186]: fuse finished with error 0 and tester_r 0 2020-11-28T21:56:12.608 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:56:12.608 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:56:12.609 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:56:12.655 INFO:teuthology.orchestra.run.smithi096.stdout:ext2/ext3 2020-11-28T21:56:12.656 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-11-28T21:56:12.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-28T21:56:12.657 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:56:12.703 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config rm mon mon_allow_pool_delete 2020-11-28T21:56:12.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.822+0000 7f559a3de700 1 Processor -- start 2020-11-28T21:56:12.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.822+0000 7f559a3de700 1 -- start start 2020-11-28T21:56:12.840 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.822+0000 7f559a3de700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f559405bcb0 0x7f559405c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:12.840 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.822+0000 7f559a3de700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f55940b2300 0x7f55940add90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:12.840 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.822+0000 7f559a3de700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55940ad6c0 con 0x7f55940b0820 2020-11-28T21:56:12.840 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.822+0000 7f559a3de700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55940ae520 con 0x7f55940b2300 2020-11-28T21:56:12.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.822+0000 7f559a3de700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f559405c090 con 0x7f559405bcb0 2020-11-28T21:56:12.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.822+0000 7f5599bdd700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f55940b0820 0x7f5594147750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:52798/0 (socket says 172.21.15.162:52798) 2020-11-28T21:56:12.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.822+0000 7f5599bdd700 1 -- 172.21.15.162:0/2409441512 learned_addr learned my addr 172.21.15.162:0/2409441512 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:56:12.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.822+0000 7f5598bdb700 1 --2- 172.21.15.162:0/2409441512 >> v2:172.21.15.118:3300/0 conn(0x7f55940b2300 0x7f55940add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:12.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.822+0000 7f55993dc700 1 --2- 172.21.15.162:0/2409441512 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f559405bcb0 0x7f559405c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:12.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.822+0000 7f5587fff700 1 -- 172.21.15.162:0/2409441512 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1331621391 0 0) 0x7f55940ad6c0 con 0x7f55940b0820 2020-11-28T21:56:12.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.822+0000 7f5587fff700 1 -- 172.21.15.162:0/2409441512 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5578003230 con 0x7f55940b0820 2020-11-28T21:56:12.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.826+0000 7f5587fff700 1 -- 172.21.15.162:0/2409441512 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 278487526 0 0) 0x7f5578003230 con 0x7f55940b0820 2020-11-28T21:56:12.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.826+0000 7f5587fff700 1 -- 172.21.15.162:0/2409441512 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f559405bcb0 msgr2=0x7f559405c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:12.843 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.826+0000 7f5587fff700 1 --2- 172.21.15.162:0/2409441512 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f559405bcb0 0x7f559405c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:12.843 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.826+0000 7f5587fff700 1 -- 172.21.15.162:0/2409441512 >> v2:172.21.15.118:3300/0 conn(0x7f55940b2300 msgr2=0x7f55940add90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:12.843 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.826+0000 7f5587fff700 1 --2- 172.21.15.162:0/2409441512 >> v2:172.21.15.118:3300/0 conn(0x7f55940b2300 0x7f55940add90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:12.843 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.826+0000 7f5587fff700 1 -- 172.21.15.162:0/2409441512 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55940ae800 con 0x7f55940b0820 2020-11-28T21:56:12.843 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.826+0000 7f5587fff700 1 -- 172.21.15.162:0/2409441512 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f558c001a50 con 0x7f55940b0820 2020-11-28T21:56:12.844 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.826+0000 7f5587fff700 1 -- 172.21.15.162:0/2409441512 >> v1:172.21.15.162:6789/0 conn(0x7f55940b0820 legacy=0x7f5594147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:12.844 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.826+0000 7f5587fff700 1 --2- 172.21.15.162:0/2409441512 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5578003720 0x7f5578003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:12.844 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.826+0000 7f5587fff700 1 --2- 172.21.15.162:0/2409441512 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f559405bcb0 0x7f55780044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:12.844 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.826+0000 7f5587fff700 1 --2- 172.21.15.162:0/2409441512 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f55780049c0 0x7f5578004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:12.844 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.826+0000 7f5587fff700 1 -- 172.21.15.162:0/2409441512 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55940ae520 con 0x7f55780049c0 2020-11-28T21:56:12.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.826+0000 7f5587fff700 1 -- 172.21.15.162:0/2409441512 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f559405c090 con 0x7f5578003720 2020-11-28T21:56:12.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.826+0000 7f5587fff700 1 -- 172.21.15.162:0/2409441512 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55780041c0 con 0x7f559405bcb0 2020-11-28T21:56:12.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.826+0000 7f5598bdb700 1 --2- 172.21.15.162:0/2409441512 >> v2:172.21.15.118:3300/0 conn(0x7f55940b2300 0x7f55940add90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:56:12.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.826+0000 7f55993dc700 1 --2- 172.21.15.162:0/2409441512 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5578003720 0x7f5578003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:12.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.826+0000 7f5599bdd700 1 --2- 172.21.15.162:0/2409441512 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f55780049c0 0x7f5578004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:12.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.826+0000 7f5598bdb700 1 --2- 172.21.15.162:0/2409441512 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f559405bcb0 0x7f55780044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:12.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.826+0000 7f55993dc700 1 -- 172.21.15.162:0/2409441512 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f559405bcb0 msgr2=0x7f55780044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:12.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.826+0000 7f55993dc700 1 --2- 172.21.15.162:0/2409441512 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f559405bcb0 0x7f55780044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:12.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.826+0000 7f55993dc700 1 -- 172.21.15.162:0/2409441512 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f55780049c0 msgr2=0x7f5578004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:12.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.826+0000 7f55993dc700 1 --2- 172.21.15.162:0/2409441512 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f55780049c0 0x7f5578004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:12.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.826+0000 7f55993dc700 1 -- 172.21.15.162:0/2409441512 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5578003230 con 0x7f5578003720 2020-11-28T21:56:12.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.826+0000 7f55993dc700 1 --2- 172.21.15.162:0/2409441512 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5578003720 0x7f5578003b40 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 rx=0x7f558800c000 tx=0x7f5588008040).ready entity=mon.1 client_cookie=56adc68f7ad8337d server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:12.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.826+0000 7f5587fff700 1 -- 172.21.15.162:0/2409441512 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f558800e8a0 con 0x7f5578003720 2020-11-28T21:56:12.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.826+0000 7f5587fff700 1 -- 172.21.15.162:0/2409441512 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f558800ea00 con 0x7f5578003720 2020-11-28T21:56:12.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.826+0000 7f5587fff700 1 -- 172.21.15.162:0/2409441512 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f558800ed60 con 0x7f5578003720 2020-11-28T21:56:12.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.826+0000 7f559a3de700 1 -- 172.21.15.162:0/2409441512 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5578003720 msgr2=0x7f5578003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:12.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.826+0000 7f559a3de700 1 --2- 172.21.15.162:0/2409441512 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5578003720 0x7f5578003b40 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 rx=0x7f558800c000 tx=0x7f5588008040).stop 2020-11-28T21:56:12.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.826+0000 7f5599bdd700 1 -- 172.21.15.162:0/2409441512 reap_dead start 2020-11-28T21:56:12.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.826+0000 7f559a3de700 1 -- 172.21.15.162:0/2409441512 shutdown_connections 2020-11-28T21:56:12.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.826+0000 7f559a3de700 1 -- 172.21.15.162:0/2409441512 >> 172.21.15.162:0/2409441512 conn(0x7f55940b17a0 msgr2=0x7f55940b1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:12.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.826+0000 7f559a3de700 1 -- 172.21.15.162:0/2409441512 shutdown_connections 2020-11-28T21:56:12.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.826+0000 7f559a3de700 1 -- 172.21.15.162:0/2409441512 wait complete. 2020-11-28T21:56:12.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.830+0000 7f559a3de700 1 Processor -- start 2020-11-28T21:56:12.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.830+0000 7f559a3de700 1 -- start start 2020-11-28T21:56:12.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.830+0000 7f559a3de700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f55940a1470 0x7f55940a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:12.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.830+0000 7f559a3de700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f559409ec40 0x7f559409f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:12.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.830+0000 7f559a3de700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55940ac100 0x7f55940ac590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:12.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.830+0000 7f559a3de700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f559415a1f0 con 0x7f55940a1470 2020-11-28T21:56:12.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.830+0000 7f559a3de700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5594065a30 con 0x7f55940ac100 2020-11-28T21:56:12.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.830+0000 7f559a3de700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5594066950 con 0x7f559409ec40 2020-11-28T21:56:12.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.830+0000 7f5599bdd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55940ac100 0x7f55940ac590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:12.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.830+0000 7f5599bdd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55940ac100 0x7f55940ac590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45964/0 (socket says 172.21.15.162:45964) 2020-11-28T21:56:12.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.830+0000 7f5599bdd700 1 -- 172.21.15.162:0/3689445747 learned_addr learned my addr 172.21.15.162:0/3689445747 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:12.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.830+0000 7f55993dc700 1 --2- 172.21.15.162:0/3689445747 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f55940a1470 0x7f55940a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:12.852 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.830+0000 7f5598bdb700 1 --2- 172.21.15.162:0/3689445747 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f559409ec40 0x7f559409f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:12.852 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.830+0000 7f5599bdd700 1 -- 172.21.15.162:0/3689445747 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f559409ec40 msgr2=0x7f559409f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:12.852 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.830+0000 7f5599bdd700 1 --2- 172.21.15.162:0/3689445747 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f559409ec40 0x7f559409f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:12.852 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.830+0000 7f5599bdd700 1 -- 172.21.15.162:0/3689445747 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f55940a1470 msgr2=0x7f55940a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:12.852 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.830+0000 7f5599bdd700 1 --2- 172.21.15.162:0/3689445747 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f55940a1470 0x7f55940a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:12.853 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.830+0000 7f5599bdd700 1 -- 172.21.15.162:0/3689445747 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5594066bf0 con 0x7f55940ac100 2020-11-28T21:56:12.853 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.830+0000 7f5599bdd700 1 --2- 172.21.15.162:0/3689445747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55940ac100 0x7f55940ac590 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 rx=0x7f558c000b20 tx=0x7f558c007b40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:12.853 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.830+0000 7f55867fc700 1 -- 172.21.15.162:0/3689445747 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f558c00fc30 con 0x7f55940ac100 2020-11-28T21:56:12.853 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.830+0000 7f55867fc700 1 -- 172.21.15.162:0/3689445747 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f558c00fd90 con 0x7f55940ac100 2020-11-28T21:56:12.853 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.830+0000 7f559a3de700 1 -- 172.21.15.162:0/3689445747 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5594159d50 con 0x7f55940ac100 2020-11-28T21:56:12.854 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.830+0000 7f55867fc700 1 -- 172.21.15.162:0/3689445747 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f558c012540 con 0x7f55940ac100 2020-11-28T21:56:12.854 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.830+0000 7f559a3de700 1 -- 172.21.15.162:0/3689445747 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f559409f5b0 con 0x7f55940ac100 2020-11-28T21:56:12.854 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.834+0000 7f559a3de700 1 -- 172.21.15.162:0/3689445747 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f559409eaa0 con 0x7f55940ac100 2020-11-28T21:56:12.854 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.834+0000 7f55867fc700 1 -- 172.21.15.162:0/3689445747 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f558c01b030 con 0x7f55940ac100 2020-11-28T21:56:12.855 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.834+0000 7f55867fc700 1 --2- 172.21.15.162:0/3689445747 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f557c055440 0x7f557c057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:12.855 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.834+0000 7f55867fc700 1 -- 172.21.15.162:0/3689445747 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f558c06d720 con 0x7f55940ac100 2020-11-28T21:56:12.855 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.834+0000 7f55993dc700 1 --2- 172.21.15.162:0/3689445747 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f557c055440 0x7f557c057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:12.856 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.834+0000 7f55993dc700 1 --2- 172.21.15.162:0/3689445747 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f557c055440 0x7f557c057900 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7f558801c000 tx=0x7f558801a000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:12.856 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.842+0000 7f55867fc700 1 -- 172.21.15.162:0/3689445747 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f559409eaa0 con 0x7f55940ac100 2020-11-28T21:56:12.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.986+0000 7f559a3de700 1 -- 172.21.15.162:0/3689445747 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f559409eaa0 con 0x7f55940ac100 2020-11-28T21:56:13.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.994+0000 7f55867fc700 1 -- 172.21.15.162:0/3689445747 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7f559409eaa0 con 0x7f55940ac100 2020-11-28T21:56:13.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.998+0000 7f559a3de700 1 -- 172.21.15.162:0/3689445747 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f557c055440 msgr2=0x7f557c057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:13.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.998+0000 7f559a3de700 1 --2- 172.21.15.162:0/3689445747 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f557c055440 0x7f557c057900 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7f558801c000 tx=0x7f558801a000).stop 2020-11-28T21:56:13.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.998+0000 7f559a3de700 1 -- 172.21.15.162:0/3689445747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55940ac100 msgr2=0x7f55940ac590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:13.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.998+0000 7f559a3de700 1 --2- 172.21.15.162:0/3689445747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55940ac100 0x7f55940ac590 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 rx=0x7f558c000b20 tx=0x7f558c007b40).stop 2020-11-28T21:56:13.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.998+0000 7f559a3de700 1 -- 172.21.15.162:0/3689445747 shutdown_connections 2020-11-28T21:56:13.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.998+0000 7f559a3de700 1 --2- 172.21.15.162:0/3689445747 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f557c055440 0x7f557c057900 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:13.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.998+0000 7f559a3de700 1 --2- 172.21.15.162:0/3689445747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55940ac100 0x7f55940ac590 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:13.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.998+0000 7f559a3de700 1 --2- 172.21.15.162:0/3689445747 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f55940a1470 0x7f55940a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:13.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.998+0000 7f559a3de700 1 --2- 172.21.15.162:0/3689445747 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f559409ec40 0x7f559409f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:13.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.998+0000 7f559a3de700 1 -- 172.21.15.162:0/3689445747 >> 172.21.15.162:0/3689445747 conn(0x7f55940b17a0 msgr2=0x7f55940b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:13.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.998+0000 7f559a3de700 1 -- 172.21.15.162:0/3689445747 shutdown_connections 2020-11-28T21:56:13.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:12.998+0000 7f559a3de700 1 -- 172.21.15.162:0/3689445747 wait complete. 2020-11-28T21:56:13.016 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Ended test tasks.cephfs.test_misc.TestMisc.test_root_rctime' 2020-11-28T21:56:13.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.126+0000 7efc216f1700 1 Processor -- start 2020-11-28T21:56:13.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.126+0000 7efc216f1700 1 -- start start 2020-11-28T21:56:13.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.126+0000 7efc216f1700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7efc1c0603d0 0x7efc1c0b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:13.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.126+0000 7efc216f1700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efc1c142a00 0x7efc1c09cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:13.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.126+0000 7efc216f1700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc1c0b4110 con 0x7efc1c0b4c70 2020-11-28T21:56:13.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.126+0000 7efc216f1700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc1c060f60 con 0x7efc1c0603d0 2020-11-28T21:56:13.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.126+0000 7efc216f1700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc1c0b43f0 con 0x7efc1c142a00 2020-11-28T21:56:13.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.126+0000 7efc20ef0700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7efc1c0b4c70 0x7efc1c143020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:52818/0 (socket says 172.21.15.162:52818) 2020-11-28T21:56:13.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.126+0000 7efc20ef0700 1 -- 172.21.15.162:0/2787620135 learned_addr learned my addr 172.21.15.162:0/2787620135 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:56:13.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.126+0000 7efc1bfff700 1 --2- 172.21.15.162:0/2787620135 >> v2:172.21.15.162:3300/0 conn(0x7efc1c0603d0 0x7efc1c0b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:13.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.130+0000 7efc1b7fe700 1 --2- 172.21.15.162:0/2787620135 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efc1c142a00 0x7efc1c09cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:13.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.130+0000 7efc1affd700 1 -- 172.21.15.162:0/2787620135 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3368457588 0 0) 0x7efc1c0b4110 con 0x7efc1c0b4c70 2020-11-28T21:56:13.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.130+0000 7efc1affd700 1 -- 172.21.15.162:0/2787620135 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc08003230 con 0x7efc1c0b4c70 2020-11-28T21:56:13.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.130+0000 7efc1bfff700 1 -- 172.21.15.162:0/2787620135 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efc1c142a00 msgr2=0x7efc1c09cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:13.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.130+0000 7efc1bfff700 1 --2- 172.21.15.162:0/2787620135 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efc1c142a00 0x7efc1c09cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:13.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.130+0000 7efc1bfff700 1 -- 172.21.15.162:0/2787620135 >> v1:172.21.15.162:6789/0 conn(0x7efc1c0b4c70 legacy=0x7efc1c143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:13.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.130+0000 7efc1bfff700 1 -- 172.21.15.162:0/2787620135 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc1c0b4690 con 0x7efc1c0603d0 2020-11-28T21:56:13.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.130+0000 7efc1bfff700 1 --2- 172.21.15.162:0/2787620135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efc1c0603d0 0x7efc1c0b3a70 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 rx=0x7efc0c002f30 tx=0x7efc0c006dc0).ready entity=mon.1 client_cookie=4f0392932b0aef23 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:13.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.130+0000 7efc1affd700 1 -- 172.21.15.162:0/2787620135 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc0c0096a0 con 0x7efc1c0603d0 2020-11-28T21:56:13.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.130+0000 7efc1affd700 1 -- 172.21.15.162:0/2787620135 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc0c009800 con 0x7efc1c0603d0 2020-11-28T21:56:13.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.130+0000 7efc216f1700 1 -- 172.21.15.162:0/2787620135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efc1c0603d0 msgr2=0x7efc1c0b3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:13.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.130+0000 7efc1affd700 1 -- 172.21.15.162:0/2787620135 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc0c009b60 con 0x7efc1c0603d0 2020-11-28T21:56:13.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.130+0000 7efc216f1700 1 --2- 172.21.15.162:0/2787620135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efc1c0603d0 0x7efc1c0b3a70 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 rx=0x7efc0c002f30 tx=0x7efc0c006dc0).stop 2020-11-28T21:56:13.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.130+0000 7efc216f1700 1 -- 172.21.15.162:0/2787620135 shutdown_connections 2020-11-28T21:56:13.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.130+0000 7efc216f1700 1 --2- 172.21.15.162:0/2787620135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efc1c0603d0 0x7efc1c0b3a70 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:13.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.130+0000 7efc216f1700 1 --2- 172.21.15.162:0/2787620135 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efc1c142a00 0x7efc1c09cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:13.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.130+0000 7efc216f1700 1 -- 172.21.15.162:0/2787620135 >> 172.21.15.162:0/2787620135 conn(0x7efc1c066a40 msgr2=0x7efc1c066e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:13.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.130+0000 7efc216f1700 1 -- 172.21.15.162:0/2787620135 shutdown_connections 2020-11-28T21:56:13.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.130+0000 7efc216f1700 1 -- 172.21.15.162:0/2787620135 wait complete. 2020-11-28T21:56:13.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.130+0000 7efc216f1700 1 Processor -- start 2020-11-28T21:56:13.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.130+0000 7efc216f1700 1 -- start start 2020-11-28T21:56:13.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.130+0000 7efc216f1700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efc1c0603d0 0x7efc1c0b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:13.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.130+0000 7efc216f1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efc1c0b4c70 0x7efc1c151580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:13.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.130+0000 7efc216f1700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efc1c145f80 0x7efc1c151a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:13.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.130+0000 7efc216f1700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc1c146c90 con 0x7efc1c0603d0 2020-11-28T21:56:13.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.130+0000 7efc216f1700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc1c155660 con 0x7efc1c0b4c70 2020-11-28T21:56:13.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.130+0000 7efc216f1700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc1c155900 con 0x7efc1c145f80 2020-11-28T21:56:13.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.134+0000 7efc1b7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efc1c0b4c70 0x7efc1c151580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:13.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.134+0000 7efc1b7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efc1c0b4c70 0x7efc1c151580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45978/0 (socket says 172.21.15.162:45978) 2020-11-28T21:56:13.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.134+0000 7efc1b7fe700 1 -- 172.21.15.162:0/3619717939 learned_addr learned my addr 172.21.15.162:0/3619717939 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:13.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.134+0000 7efc20ef0700 1 --2- 172.21.15.162:0/3619717939 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efc1c145f80 0x7efc1c151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:13.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.134+0000 7efc1bfff700 1 --2- 172.21.15.162:0/3619717939 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efc1c0603d0 0x7efc1c0b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:13.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.134+0000 7efc1b7fe700 1 -- 172.21.15.162:0/3619717939 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efc1c145f80 msgr2=0x7efc1c151a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:13.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.134+0000 7efc1b7fe700 1 --2- 172.21.15.162:0/3619717939 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efc1c145f80 0x7efc1c151a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:13.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.134+0000 7efc1b7fe700 1 -- 172.21.15.162:0/3619717939 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efc1c0603d0 msgr2=0x7efc1c0b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:13.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.134+0000 7efc1b7fe700 1 --2- 172.21.15.162:0/3619717939 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efc1c0603d0 0x7efc1c0b2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:13.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.134+0000 7efc1b7fe700 1 -- 172.21.15.162:0/3619717939 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc1c145ae0 con 0x7efc1c0b4c70 2020-11-28T21:56:13.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.134+0000 7efc1b7fe700 1 --2- 172.21.15.162:0/3619717939 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efc1c0b4c70 0x7efc1c151580 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 rx=0x7efc1000c000 tx=0x7efc10008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:13.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.134+0000 7efc197fa700 1 -- 172.21.15.162:0/3619717939 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc1000be30 con 0x7efc1c0b4c70 2020-11-28T21:56:13.146 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.134+0000 7efc197fa700 1 -- 172.21.15.162:0/3619717939 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc1000c820 con 0x7efc1c0b4c70 2020-11-28T21:56:13.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.134+0000 7efc197fa700 1 -- 172.21.15.162:0/3619717939 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc1000cb80 con 0x7efc1c0b4c70 2020-11-28T21:56:13.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.134+0000 7efc216f1700 1 -- 172.21.15.162:0/3619717939 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc1c149850 con 0x7efc1c0b4c70 2020-11-28T21:56:13.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.134+0000 7efc216f1700 1 -- 172.21.15.162:0/3619717939 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc1c146600 con 0x7efc1c0b4c70 2020-11-28T21:56:13.147 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.134+0000 7efc216f1700 1 -- 172.21.15.162:0/3619717939 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc1c145c70 con 0x7efc1c0b4c70 2020-11-28T21:56:13.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.134+0000 7efc197fa700 1 -- 172.21.15.162:0/3619717939 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7efc1000cce0 con 0x7efc1c0b4c70 2020-11-28T21:56:13.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.134+0000 7efc197fa700 1 --2- 172.21.15.162:0/3619717939 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7efbfc0554e0 0x7efbfc0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:13.148 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.134+0000 7efc1bfff700 1 --2- 172.21.15.162:0/3619717939 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7efbfc0554e0 0x7efbfc0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:13.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.138+0000 7efc197fa700 1 -- 172.21.15.162:0/3619717939 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6073+0+0 (secure 0 0 0) 0x7efc1006e310 con 0x7efc1c0b4c70 2020-11-28T21:56:13.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.138+0000 7efc1bfff700 1 --2- 172.21.15.162:0/3619717939 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7efbfc0554e0 0x7efbfc0579a0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7efc0c01b000 tx=0x7efc0c015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:13.149 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.138+0000 7efc197fa700 1 -- 172.21.15.162:0/3619717939 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7efc100445b0 con 0x7efc1c0b4c70 2020-11-28T21:56:13.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.334+0000 7efc216f1700 1 -- 172.21.15.162:0/3619717939 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_root_rctime"]} v 0) v1 -- 0x7efc1c057050 con 0x7efc1c0b4c70 2020-11-28T21:56:13.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.858+0000 7efc197fa700 1 -- 172.21.15.162:0/3619717939 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_root_rctime"]}]=0 v447) v1 ==== 127+0+0 (secure 0 0 0) 0x7efc10047bd0 con 0x7efc1c0b4c70 2020-11-28T21:56:13.865 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.862+0000 7efc216f1700 1 -- 172.21.15.162:0/3619717939 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7efbfc0554e0 msgr2=0x7efbfc0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:13.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.862+0000 7efc216f1700 1 --2- 172.21.15.162:0/3619717939 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7efbfc0554e0 0x7efbfc0579a0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7efc0c01b000 tx=0x7efc0c015000).stop 2020-11-28T21:56:13.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.862+0000 7efc216f1700 1 -- 172.21.15.162:0/3619717939 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efc1c0b4c70 msgr2=0x7efc1c151580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:13.866 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.862+0000 7efc216f1700 1 --2- 172.21.15.162:0/3619717939 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efc1c0b4c70 0x7efc1c151580 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 rx=0x7efc1000c000 tx=0x7efc10008040).stop 2020-11-28T21:56:13.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.862+0000 7efc216f1700 1 -- 172.21.15.162:0/3619717939 shutdown_connections 2020-11-28T21:56:13.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.862+0000 7efc216f1700 1 --2- 172.21.15.162:0/3619717939 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7efbfc0554e0 0x7efbfc0579a0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:13.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.862+0000 7efc216f1700 1 --2- 172.21.15.162:0/3619717939 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efc1c145f80 0x7efc1c151a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:13.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.862+0000 7efc216f1700 1 --2- 172.21.15.162:0/3619717939 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efc1c0603d0 0x7efc1c0b2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:13.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.862+0000 7efc216f1700 1 --2- 172.21.15.162:0/3619717939 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efc1c0b4c70 0x7efc1c151580 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:13.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.862+0000 7efc216f1700 1 -- 172.21.15.162:0/3619717939 >> 172.21.15.162:0/3619717939 conn(0x7efc1c066a40 msgr2=0x7efc1c0b3980 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:13.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.862+0000 7efc216f1700 1 -- 172.21.15.162:0/3619717939 shutdown_connections 2020-11-28T21:56:13.868 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.862+0000 7efc216f1700 1 -- 172.21.15.162:0/3619717939 wait complete. 2020-11-28T21:56:13.883 INFO:tasks.cephfs_test_runner:test_root_rctime (tasks.cephfs.test_misc.TestMisc) ... ok 2020-11-28T21:56:13.884 INFO:tasks.cephfs_test_runner:Starting test: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2020-11-28T21:56:13.884 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_misc.TestMisc.test_statfs_on_deleted_fs' 2020-11-28T21:56:13.998 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.990+0000 7fc87c634700 1 Processor -- start 2020-11-28T21:56:13.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.990+0000 7fc87c634700 1 -- start start 2020-11-28T21:56:13.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.994+0000 7fc87c634700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc8740b1990 0x7fc874140890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:13.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.994+0000 7fc87c634700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8740b2870 con 0x7fc8740b11b0 2020-11-28T21:56:14.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.994+0000 7fc87c634700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8740b22e0 con 0x7fc8740b1d70 2020-11-28T21:56:14.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.994+0000 7fc87c634700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc874146e20 con 0x7fc8740b1990 2020-11-28T21:56:14.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.994+0000 7fc87a3d0700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc8740b1990 0x7fc874140890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:14.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.994+0000 7fc87a3d0700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc8740b1990 0x7fc874140890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45984/0 (socket says 172.21.15.162:45984) 2020-11-28T21:56:14.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.994+0000 7fc87a3d0700 1 -- 172.21.15.162:0/974100240 learned_addr learned my addr 172.21.15.162:0/974100240 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:14.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.994+0000 7fc87a3d0700 1 -- 172.21.15.162:0/974100240 >> v1:172.21.15.162:6789/0 conn(0x7fc8740b1d70 legacy=0x7fc87414a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:14.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.994+0000 7fc87a3d0700 1 -- 172.21.15.162:0/974100240 >> v1:172.21.15.118:6789/0 conn(0x7fc8740b11b0 legacy=0x7fc874140d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:14.002 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.994+0000 7fc87a3d0700 1 -- 172.21.15.162:0/974100240 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8740b0740 con 0x7fc8740b1990 2020-11-28T21:56:14.003 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.994+0000 7fc87a3d0700 1 --2- 172.21.15.162:0/974100240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8740b1990 0x7fc874140890 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 rx=0x7fc870002f30 tx=0x7fc870006e80).ready entity=mon.1 client_cookie=7d65dcd95f76ec3e server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:14.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.998+0000 7fc8793ce700 1 -- 172.21.15.162:0/974100240 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc8700097f0 con 0x7fc8740b1990 2020-11-28T21:56:14.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.998+0000 7fc8793ce700 1 -- 172.21.15.162:0/974100240 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc870009950 con 0x7fc8740b1990 2020-11-28T21:56:14.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.998+0000 7fc8793ce700 1 -- 172.21.15.162:0/974100240 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc870009cb0 con 0x7fc8740b1990 2020-11-28T21:56:14.004 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.998+0000 7fc87c634700 1 -- 172.21.15.162:0/974100240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8740b1990 msgr2=0x7fc874140890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:14.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.998+0000 7fc87c634700 1 --2- 172.21.15.162:0/974100240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8740b1990 0x7fc874140890 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 rx=0x7fc870002f30 tx=0x7fc870006e80).stop 2020-11-28T21:56:14.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.998+0000 7fc87c634700 1 -- 172.21.15.162:0/974100240 shutdown_connections 2020-11-28T21:56:14.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.998+0000 7fc87c634700 1 --2- 172.21.15.162:0/974100240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8740b1990 0x7fc874140890 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:14.005 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.998+0000 7fc87c634700 1 -- 172.21.15.162:0/974100240 >> 172.21.15.162:0/974100240 conn(0x7fc87405b970 msgr2=0x7fc87405bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:14.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.998+0000 7fc87c634700 1 -- 172.21.15.162:0/974100240 shutdown_connections 2020-11-28T21:56:14.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.998+0000 7fc87c634700 1 -- 172.21.15.162:0/974100240 wait complete. 2020-11-28T21:56:14.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.998+0000 7fc87c634700 1 Processor -- start 2020-11-28T21:56:14.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:13.998+0000 7fc87c634700 1 -- start start 2020-11-28T21:56:14.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.002+0000 7fc87c634700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc8740b11b0 0x7fc87414e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:14.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.002+0000 7fc87c634700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8740b1990 0x7fc8740b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:14.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.002+0000 7fc87c634700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc8740b1d70 0x7fc874141530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:14.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.002+0000 7fc87c634700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87414bca0 con 0x7fc8740b1d70 2020-11-28T21:56:14.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.002+0000 7fc87c634700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8741440d0 con 0x7fc8740b1990 2020-11-28T21:56:14.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.002+0000 7fc87c634700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc87414ba00 con 0x7fc8740b11b0 2020-11-28T21:56:14.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.002+0000 7fc879bcf700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8740b1990 0x7fc8740b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:14.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.002+0000 7fc879bcf700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8740b1990 0x7fc8740b3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45992/0 (socket says 172.21.15.162:45992) 2020-11-28T21:56:14.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.002+0000 7fc879bcf700 1 -- 172.21.15.162:0/2543154404 learned_addr learned my addr 172.21.15.162:0/2543154404 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:14.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.002+0000 7fc87a3d0700 1 --2- 172.21.15.162:0/2543154404 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc8740b11b0 0x7fc87414e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:14.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.002+0000 7fc87abd1700 1 --2- 172.21.15.162:0/2543154404 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc8740b1d70 0x7fc874141530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:14.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.002+0000 7fc879bcf700 1 -- 172.21.15.162:0/2543154404 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc8740b11b0 msgr2=0x7fc87414e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:14.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.002+0000 7fc879bcf700 1 --2- 172.21.15.162:0/2543154404 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc8740b11b0 0x7fc87414e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:14.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.002+0000 7fc879bcf700 1 -- 172.21.15.162:0/2543154404 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc8740b1d70 msgr2=0x7fc874141530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:14.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.002+0000 7fc879bcf700 1 --2- 172.21.15.162:0/2543154404 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc8740b1d70 0x7fc874141530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:14.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.002+0000 7fc879bcf700 1 -- 172.21.15.162:0/2543154404 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8741432e0 con 0x7fc8740b1990 2020-11-28T21:56:14.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.002+0000 7fc879bcf700 1 --2- 172.21.15.162:0/2543154404 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8740b1990 0x7fc8740b3f70 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 rx=0x7fc87414a650 tx=0x7fc864007bf0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:14.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.002+0000 7fc8637fe700 1 -- 172.21.15.162:0/2543154404 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc86400e4a0 con 0x7fc8740b1990 2020-11-28T21:56:14.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.002+0000 7fc8637fe700 1 -- 172.21.15.162:0/2543154404 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86400e600 con 0x7fc8740b1990 2020-11-28T21:56:14.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.002+0000 7fc87c634700 1 -- 172.21.15.162:0/2543154404 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc874143580 con 0x7fc8740b1990 2020-11-28T21:56:14.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.002+0000 7fc8637fe700 1 -- 172.21.15.162:0/2543154404 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc86400e960 con 0x7fc8740b1990 2020-11-28T21:56:14.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.002+0000 7fc87c634700 1 -- 172.21.15.162:0/2543154404 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc87414b680 con 0x7fc8740b1990 2020-11-28T21:56:14.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.002+0000 7fc8637fe700 1 -- 172.21.15.162:0/2543154404 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc864010070 con 0x7fc8740b1990 2020-11-28T21:56:14.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.002+0000 7fc87c634700 1 -- 172.21.15.162:0/2543154404 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc874143710 con 0x7fc8740b1990 2020-11-28T21:56:14.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.002+0000 7fc8637fe700 1 --2- 172.21.15.162:0/2543154404 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc854055440 0x7fc854057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:14.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.006+0000 7fc87a3d0700 1 --2- 172.21.15.162:0/2543154404 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc854055440 0x7fc854057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:14.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.006+0000 7fc8637fe700 1 -- 172.21.15.162:0/2543154404 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc86406d3b0 con 0x7fc8740b1990 2020-11-28T21:56:14.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.006+0000 7fc87a3d0700 1 --2- 172.21.15.162:0/2543154404 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc854055440 0x7fc854057900 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7fc87001a000 tx=0x7fc870015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:14.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.010+0000 7fc8637fe700 1 -- 172.21.15.162:0/2543154404 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc864040df0 con 0x7fc8740b1990 2020-11-28T21:56:14.205 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.198+0000 7fc87c634700 1 -- 172.21.15.162:0/2543154404 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_statfs_on_deleted_fs"]} v 0) v1 -- 0x7fc874057050 con 0x7fc8740b1990 2020-11-28T21:56:14.867 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.862+0000 7fc8637fe700 1 -- 172.21.15.162:0/2543154404 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_statfs_on_deleted_fs"]}]=0 v448) v1 ==== 139+0+0 (secure 0 0 0) 0x7fc8640436b0 con 0x7fc8740b1990 2020-11-28T21:56:14.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.866+0000 7fc87c634700 1 -- 172.21.15.162:0/2543154404 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc854055440 msgr2=0x7fc854057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:14.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.866+0000 7fc87c634700 1 --2- 172.21.15.162:0/2543154404 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc854055440 0x7fc854057900 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7fc87001a000 tx=0x7fc870015000).stop 2020-11-28T21:56:14.872 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.866+0000 7fc87c634700 1 -- 172.21.15.162:0/2543154404 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8740b1990 msgr2=0x7fc8740b3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:14.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.866+0000 7fc87c634700 1 --2- 172.21.15.162:0/2543154404 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8740b1990 0x7fc8740b3f70 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 rx=0x7fc87414a650 tx=0x7fc864007bf0).stop 2020-11-28T21:56:14.873 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.866+0000 7fc87c634700 1 -- 172.21.15.162:0/2543154404 shutdown_connections 2020-11-28T21:56:14.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.866+0000 7fc87c634700 1 --2- 172.21.15.162:0/2543154404 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc854055440 0x7fc854057900 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:14.874 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.866+0000 7fc87c634700 1 --2- 172.21.15.162:0/2543154404 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc8740b11b0 0x7fc87414e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:14.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.866+0000 7fc87c634700 1 --2- 172.21.15.162:0/2543154404 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc8740b1d70 0x7fc874141530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:14.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.866+0000 7fc87c634700 1 --2- 172.21.15.162:0/2543154404 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8740b1990 0x7fc8740b3f70 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:14.875 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.866+0000 7fc87c634700 1 -- 172.21.15.162:0/2543154404 >> 172.21.15.162:0/2543154404 conn(0x7fc87405b970 msgr2=0x7fc874140660 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:14.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.866+0000 7fc87c634700 1 -- 172.21.15.162:0/2543154404 shutdown_connections 2020-11-28T21:56:14.876 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:14.866+0000 7fc87c634700 1 -- 172.21.15.162:0/2543154404 wait complete. 2020-11-28T21:56:14.889 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config set mon mon_allow_pool_delete True 2020-11-28T21:56:15.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.010+0000 7f5b2aec6700 1 Processor -- start 2020-11-28T21:56:15.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.010+0000 7f5b2aec6700 1 -- start start 2020-11-28T21:56:15.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.010+0000 7f5b2aec6700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5b240b9ce0 0x7f5b241433c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:15.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.010+0000 7f5b2aec6700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5b240605c0 0x7f5b24067610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:15.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.010+0000 7f5b2aec6700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b24143950 con 0x7f5b240b3c30 2020-11-28T21:56:15.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.010+0000 7f5b2aec6700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b240ba0c0 con 0x7f5b240b9ce0 2020-11-28T21:56:15.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.010+0000 7f5b2aec6700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b24066c30 con 0x7f5b240605c0 2020-11-28T21:56:15.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.010+0000 7f5b29463700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f5b240b3c30 0x7f5b2413fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:52846/0 (socket says 172.21.15.162:52846) 2020-11-28T21:56:15.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.010+0000 7f5b29463700 1 -- 172.21.15.162:0/2978856627 learned_addr learned my addr 172.21.15.162:0/2978856627 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:56:15.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.010+0000 7f5b28c62700 1 --2- 172.21.15.162:0/2978856627 >> v2:172.21.15.118:3300/0 conn(0x7f5b240b9ce0 0x7f5b241433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:15.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.010+0000 7f5b23fff700 1 --2- 172.21.15.162:0/2978856627 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5b240605c0 0x7f5b24067610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:15.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.010+0000 7f5b237fe700 1 -- 172.21.15.162:0/2978856627 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2290654613 0 0) 0x7f5b24143950 con 0x7f5b240b3c30 2020-11-28T21:56:15.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.010+0000 7f5b237fe700 1 -- 172.21.15.162:0/2978856627 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b10003230 con 0x7f5b240b3c30 2020-11-28T21:56:15.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.010+0000 7f5b237fe700 1 -- 172.21.15.162:0/2978856627 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 671355618 0 0) 0x7f5b10003230 con 0x7f5b240b3c30 2020-11-28T21:56:15.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.010+0000 7f5b28c62700 1 -- 172.21.15.162:0/2978856627 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5b240605c0 msgr2=0x7f5b24067610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:15.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.010+0000 7f5b28c62700 1 --2- 172.21.15.162:0/2978856627 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5b240605c0 0x7f5b24067610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:15.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.010+0000 7f5b28c62700 1 -- 172.21.15.162:0/2978856627 >> v1:172.21.15.162:6789/0 conn(0x7f5b240b3c30 legacy=0x7f5b2413fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:15.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.010+0000 7f5b28c62700 1 -- 172.21.15.162:0/2978856627 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b24066ed0 con 0x7f5b240b9ce0 2020-11-28T21:56:15.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.010+0000 7f5b28c62700 1 --2- 172.21.15.162:0/2978856627 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b240b9ce0 0x7f5b241433c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f5b1c001bf0 tx=0x7f5b14006d60).ready entity=mon.0 client_cookie=aa74deb813c34539 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:15.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.010+0000 7f5b237fe700 1 -- 172.21.15.162:0/2978856627 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b1c001a50 con 0x7f5b240b9ce0 2020-11-28T21:56:15.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.010+0000 7f5b237fe700 1 -- 172.21.15.162:0/2978856627 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1400f510 con 0x7f5b240b9ce0 2020-11-28T21:56:15.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.010+0000 7f5b237fe700 1 -- 172.21.15.162:0/2978856627 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b1400f680 con 0x7f5b240b9ce0 2020-11-28T21:56:15.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.010+0000 7f5b2aec6700 1 -- 172.21.15.162:0/2978856627 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b240b9ce0 msgr2=0x7f5b241433c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:15.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.010+0000 7f5b2aec6700 1 --2- 172.21.15.162:0/2978856627 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b240b9ce0 0x7f5b241433c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f5b1c001bf0 tx=0x7f5b14006d60).stop 2020-11-28T21:56:15.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.014+0000 7f5b2aec6700 1 -- 172.21.15.162:0/2978856627 shutdown_connections 2020-11-28T21:56:15.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.014+0000 7f5b2aec6700 1 --2- 172.21.15.162:0/2978856627 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b240b9ce0 0x7f5b241433c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:15.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.014+0000 7f5b2aec6700 1 --2- 172.21.15.162:0/2978856627 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5b240605c0 0x7f5b24067610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:15.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.014+0000 7f5b2aec6700 1 -- 172.21.15.162:0/2978856627 >> 172.21.15.162:0/2978856627 conn(0x7f5b2405c430 msgr2=0x7f5b240b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:15.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.014+0000 7f5b2aec6700 1 -- 172.21.15.162:0/2978856627 shutdown_connections 2020-11-28T21:56:15.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.014+0000 7f5b2aec6700 1 -- 172.21.15.162:0/2978856627 wait complete. 2020-11-28T21:56:15.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.014+0000 7f5b2aec6700 1 Processor -- start 2020-11-28T21:56:15.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.014+0000 7f5b2aec6700 1 -- start start 2020-11-28T21:56:15.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.014+0000 7f5b2aec6700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b240605c0 0x7f5b24144170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:15.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.014+0000 7f5b2aec6700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b240b3c30 0x7f5b240af320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:15.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.014+0000 7f5b2aec6700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5b240b9ce0 0x7f5b24144910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:15.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.014+0000 7f5b2aec6700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b24146ab0 con 0x7f5b240b3c30 2020-11-28T21:56:15.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.014+0000 7f5b2aec6700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2409b3c0 con 0x7f5b240605c0 2020-11-28T21:56:15.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.014+0000 7f5b2aec6700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b24146ce0 con 0x7f5b240b9ce0 2020-11-28T21:56:15.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.014+0000 7f5b28c62700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b240605c0 0x7f5b24144170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:15.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.014+0000 7f5b28c62700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b240605c0 0x7f5b24144170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46004/0 (socket says 172.21.15.162:46004) 2020-11-28T21:56:15.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.014+0000 7f5b28c62700 1 -- 172.21.15.162:0/2470312609 learned_addr learned my addr 172.21.15.162:0/2470312609 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:15.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.014+0000 7f5b23fff700 1 --2- 172.21.15.162:0/2470312609 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b240b3c30 0x7f5b240af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:15.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.014+0000 7f5b29463700 1 --2- 172.21.15.162:0/2470312609 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5b240b9ce0 0x7f5b24144910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:15.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.014+0000 7f5b28c62700 1 -- 172.21.15.162:0/2470312609 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5b240b9ce0 msgr2=0x7f5b24144910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:15.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.014+0000 7f5b28c62700 1 --2- 172.21.15.162:0/2470312609 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5b240b9ce0 0x7f5b24144910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:15.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.014+0000 7f5b28c62700 1 -- 172.21.15.162:0/2470312609 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b240b3c30 msgr2=0x7f5b240af320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:15.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.014+0000 7f5b28c62700 1 --2- 172.21.15.162:0/2470312609 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b240b3c30 0x7f5b240af320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:15.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.014+0000 7f5b28c62700 1 -- 172.21.15.162:0/2470312609 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b2409b640 con 0x7f5b240605c0 2020-11-28T21:56:15.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.014+0000 7f5b29463700 1 --2- 172.21.15.162:0/2470312609 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5b240b9ce0 0x7f5b24144910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:56:15.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.018+0000 7f5b28c62700 1 --2- 172.21.15.162:0/2470312609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b240605c0 0x7f5b24144170 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 rx=0x7f5b1c001bf0 tx=0x7f5b14014000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:15.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.018+0000 7f5b21ffb700 1 -- 172.21.15.162:0/2470312609 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b14011db0 con 0x7f5b240605c0 2020-11-28T21:56:15.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.018+0000 7f5b21ffb700 1 -- 172.21.15.162:0/2470312609 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1401c470 con 0x7f5b240605c0 2020-11-28T21:56:15.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.018+0000 7f5b2aec6700 1 -- 172.21.15.162:0/2470312609 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b2409b8e0 con 0x7f5b240605c0 2020-11-28T21:56:15.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.018+0000 7f5b21ffb700 1 -- 172.21.15.162:0/2470312609 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b1401c5e0 con 0x7f5b240605c0 2020-11-28T21:56:15.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.018+0000 7f5b2aec6700 1 -- 172.21.15.162:0/2470312609 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b240b76b0 con 0x7f5b240605c0 2020-11-28T21:56:15.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.018+0000 7f5b2aec6700 1 -- 172.21.15.162:0/2470312609 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b24057050 con 0x7f5b240605c0 2020-11-28T21:56:15.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.018+0000 7f5b21ffb700 1 -- 172.21.15.162:0/2470312609 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5b1400f070 con 0x7f5b240605c0 2020-11-28T21:56:15.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.018+0000 7f5b21ffb700 1 --2- 172.21.15.162:0/2470312609 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5b040553b0 0x7f5b04057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:15.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.018+0000 7f5b21ffb700 1 -- 172.21.15.162:0/2470312609 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5b1406f1b0 con 0x7f5b240605c0 2020-11-28T21:56:15.032 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.018+0000 7f5b23fff700 1 --2- 172.21.15.162:0/2470312609 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5b040553b0 0x7f5b04057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:15.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.022+0000 7f5b23fff700 1 --2- 172.21.15.162:0/2470312609 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5b040553b0 0x7f5b04057870 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7f5b1800b000 tx=0x7f5b18008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:15.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.026+0000 7f5b21ffb700 1 -- 172.21.15.162:0/2470312609 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5b140454f0 con 0x7f5b240605c0 2020-11-28T21:56:15.172 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.166+0000 7f5b2aec6700 1 -- 172.21.15.162:0/2470312609 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7f5b24068350 con 0x7f5b240605c0 2020-11-28T21:56:15.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.178+0000 7f5b21ffb700 1 -- 172.21.15.162:0/2470312609 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5b140150c0 con 0x7f5b240605c0 2020-11-28T21:56:15.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.178+0000 7f5b2aec6700 1 -- 172.21.15.162:0/2470312609 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5b040553b0 msgr2=0x7f5b04057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:15.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.178+0000 7f5b2aec6700 1 --2- 172.21.15.162:0/2470312609 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5b040553b0 0x7f5b04057870 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7f5b1800b000 tx=0x7f5b18008040).stop 2020-11-28T21:56:15.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.178+0000 7f5b2aec6700 1 -- 172.21.15.162:0/2470312609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b240605c0 msgr2=0x7f5b24144170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:15.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.178+0000 7f5b2aec6700 1 --2- 172.21.15.162:0/2470312609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b240605c0 0x7f5b24144170 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 rx=0x7f5b1c001bf0 tx=0x7f5b14014000).stop 2020-11-28T21:56:15.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.178+0000 7f5b2aec6700 1 -- 172.21.15.162:0/2470312609 shutdown_connections 2020-11-28T21:56:15.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.178+0000 7f5b2aec6700 1 --2- 172.21.15.162:0/2470312609 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5b040553b0 0x7f5b04057870 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:15.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.178+0000 7f5b2aec6700 1 --2- 172.21.15.162:0/2470312609 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5b240b9ce0 0x7f5b24144910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:15.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.178+0000 7f5b2aec6700 1 --2- 172.21.15.162:0/2470312609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b240605c0 0x7f5b24144170 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:15.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.178+0000 7f5b2aec6700 1 --2- 172.21.15.162:0/2470312609 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b240b3c30 0x7f5b240af320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:15.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.178+0000 7f5b2aec6700 1 -- 172.21.15.162:0/2470312609 >> 172.21.15.162:0/2470312609 conn(0x7f5b2405c430 msgr2=0x7f5b240b8930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:15.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.178+0000 7f5b2aec6700 1 -- 172.21.15.162:0/2470312609 shutdown_connections 2020-11-28T21:56:15.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.178+0000 7f5b2aec6700 1 -- 172.21.15.162:0/2470312609 wait complete. 2020-11-28T21:56:15.202 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-28T21:56:15.203 INFO:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:56:15.221 INFO:teuthology.orchestra.run.smithi118:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:56:15.241 INFO:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:56:15.259 INFO:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:56:15.278 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-28T21:56:15.279 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:56:15.280 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:56:15.280 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:56:15.330 INFO:teuthology.orchestra.run.smithi192.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2020-11-28T21:56:15.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:56:15.332 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2020-11-28T21:56:15.333 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:56:15.334 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:56:15.334 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:56:15.342 INFO:teuthology.orchestra.run.smithi096.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2020-11-28T21:56:15.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T21:56:15.343 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2020-11-28T21:56:15.344 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2020-11-28T21:56:15.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.462+0000 7f02c9fde700 1 Processor -- start 2020-11-28T21:56:15.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.462+0000 7f02c9fde700 1 -- start start 2020-11-28T21:56:15.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.466+0000 7f02c9fde700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f02c40615c0 0x7f02c40b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:15.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.466+0000 7f02c9fde700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f02c40b4b40 0x7f02c413ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:15.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.466+0000 7f02c9fde700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02c40b1790 con 0x7f02c40611e0 2020-11-28T21:56:15.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.466+0000 7f02c9fde700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02c40b1a30 con 0x7f02c40b4b40 2020-11-28T21:56:15.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.466+0000 7f02c9fde700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02c40b3f90 con 0x7f02c40615c0 2020-11-28T21:56:15.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.466+0000 7f02c97dd700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f02c40b4b40 0x7f02c413ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:15.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.466+0000 7f02c3fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f02c40615c0 0x7f02c40b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:15.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.466+0000 7f02c8fdc700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f02c40611e0 0x7f02c40b4430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.162:43004/0 (socket says 172.21.15.162:43004) 2020-11-28T21:56:15.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.466+0000 7f02c8fdc700 1 -- 172.21.15.162:0/3533757425 learned_addr learned my addr 172.21.15.162:0/3533757425 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:56:15.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.466+0000 7f02c37fe700 1 -- 172.21.15.162:0/3533757425 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2124271197 0 0) 0x7f02c40b1790 con 0x7f02c40611e0 2020-11-28T21:56:15.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.466+0000 7f02c37fe700 1 -- 172.21.15.162:0/3533757425 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02a4003230 con 0x7f02c40611e0 2020-11-28T21:56:15.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.466+0000 7f02c3fff700 1 -- 172.21.15.162:0/3533757425 >> v2:172.21.15.118:3300/0 conn(0x7f02c40b4b40 msgr2=0x7f02c413ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:15.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.466+0000 7f02c3fff700 1 --2- 172.21.15.162:0/3533757425 >> v2:172.21.15.118:3300/0 conn(0x7f02c40b4b40 0x7f02c413ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:15.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.466+0000 7f02c3fff700 1 -- 172.21.15.162:0/3533757425 >> v1:172.21.15.118:6789/0 conn(0x7f02c40611e0 legacy=0x7f02c40b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:15.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.466+0000 7f02c3fff700 1 -- 172.21.15.162:0/3533757425 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c40b4fb0 con 0x7f02c40615c0 2020-11-28T21:56:15.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.466+0000 7f02c97dd700 1 --2- 172.21.15.162:0/3533757425 >> v2:172.21.15.118:3300/0 conn(0x7f02c40b4b40 0x7f02c413ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:56:15.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.466+0000 7f02c3fff700 1 --2- 172.21.15.162:0/3533757425 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f02c40615c0 0x7f02c40b10f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f02ac002f30 tx=0x7f02ac006dc0).ready entity=mon.2 client_cookie=8b2d3772de743a8c server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:15.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.466+0000 7f02c37fe700 1 -- 172.21.15.162:0/3533757425 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02ac0096a0 con 0x7f02c40615c0 2020-11-28T21:56:15.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.466+0000 7f02c37fe700 1 -- 172.21.15.162:0/3533757425 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02ac009800 con 0x7f02c40615c0 2020-11-28T21:56:15.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.466+0000 7f02c37fe700 1 -- 172.21.15.162:0/3533757425 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02ac009b60 con 0x7f02c40615c0 2020-11-28T21:56:15.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.466+0000 7f02c9fde700 1 -- 172.21.15.162:0/3533757425 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f02c40615c0 msgr2=0x7f02c40b10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:15.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.466+0000 7f02c9fde700 1 --2- 172.21.15.162:0/3533757425 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f02c40615c0 0x7f02c40b10f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f02ac002f30 tx=0x7f02ac006dc0).stop 2020-11-28T21:56:15.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.470+0000 7f02c9fde700 1 -- 172.21.15.162:0/3533757425 shutdown_connections 2020-11-28T21:56:15.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.470+0000 7f02c9fde700 1 --2- 172.21.15.162:0/3533757425 >> v2:172.21.15.118:3300/0 conn(0x7f02c40b4b40 0x7f02c413ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:15.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.470+0000 7f02c9fde700 1 --2- 172.21.15.162:0/3533757425 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f02c40615c0 0x7f02c40b10f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:15.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.470+0000 7f02c9fde700 1 -- 172.21.15.162:0/3533757425 >> 172.21.15.162:0/3533757425 conn(0x7f02c40b54f0 msgr2=0x7f02c40b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:15.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.470+0000 7f02c9fde700 1 -- 172.21.15.162:0/3533757425 shutdown_connections 2020-11-28T21:56:15.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.470+0000 7f02c9fde700 1 -- 172.21.15.162:0/3533757425 wait complete. 2020-11-28T21:56:15.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.470+0000 7f02c9fde700 1 Processor -- start 2020-11-28T21:56:15.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.470+0000 7f02c9fde700 1 -- start start 2020-11-28T21:56:15.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.470+0000 7f02c9fde700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02c40611e0 0x7f02c4140610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:15.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.470+0000 7f02c9fde700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f02c40615c0 0x7f02c40aa4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:15.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.470+0000 7f02c9fde700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f02c40b4b40 0x7f02c40aa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:15.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.470+0000 7f02c9fde700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c40ae430 con 0x7f02c40615c0 2020-11-28T21:56:15.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.470+0000 7f02c9fde700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c40ad4c0 con 0x7f02c40611e0 2020-11-28T21:56:15.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.470+0000 7f02c9fde700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02c409a290 con 0x7f02c40b4b40 2020-11-28T21:56:15.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.474+0000 7f02c8fdc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02c40611e0 0x7f02c4140610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:15.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.474+0000 7f02c8fdc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02c40611e0 0x7f02c4140610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46018/0 (socket says 172.21.15.162:46018) 2020-11-28T21:56:15.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.474+0000 7f02c8fdc700 1 -- 172.21.15.162:0/22510371 learned_addr learned my addr 172.21.15.162:0/22510371 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:15.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.474+0000 7f02c3fff700 1 --2- 172.21.15.162:0/22510371 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f02c40615c0 0x7f02c40aa4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:15.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.474+0000 7f02c97dd700 1 --2- 172.21.15.162:0/22510371 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f02c40b4b40 0x7f02c40aa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:15.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.474+0000 7f02c8fdc700 1 -- 172.21.15.162:0/22510371 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f02c40b4b40 msgr2=0x7f02c40aa9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:15.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.474+0000 7f02c8fdc700 1 --2- 172.21.15.162:0/22510371 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f02c40b4b40 0x7f02c40aa9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:15.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.474+0000 7f02c8fdc700 1 -- 172.21.15.162:0/22510371 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f02c40615c0 msgr2=0x7f02c40aa4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:15.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.474+0000 7f02c8fdc700 1 --2- 172.21.15.162:0/22510371 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f02c40615c0 0x7f02c40aa4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:15.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.474+0000 7f02c8fdc700 1 -- 172.21.15.162:0/22510371 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c40ad220 con 0x7f02c40611e0 2020-11-28T21:56:15.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.474+0000 7f02c8fdc700 1 --2- 172.21.15.162:0/22510371 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02c40611e0 0x7f02c4140610 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 rx=0x7f02c413c140 tx=0x7f02b4007aa0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:15.485 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.474+0000 7f02c1ffb700 1 -- 172.21.15.162:0/22510371 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02b40072e0 con 0x7f02c40611e0 2020-11-28T21:56:15.485 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.474+0000 7f02c1ffb700 1 -- 172.21.15.162:0/22510371 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b4007440 con 0x7f02c40611e0 2020-11-28T21:56:15.485 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.474+0000 7f02c9fde700 1 -- 172.21.15.162:0/22510371 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02c406aca0 con 0x7f02c40611e0 2020-11-28T21:56:15.485 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.474+0000 7f02c1ffb700 1 -- 172.21.15.162:0/22510371 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02b40057d0 con 0x7f02c40611e0 2020-11-28T21:56:15.486 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.474+0000 7f02c9fde700 1 -- 172.21.15.162:0/22510371 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c40aaec0 con 0x7f02c40611e0 2020-11-28T21:56:15.486 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.474+0000 7f02c9fde700 1 -- 172.21.15.162:0/22510371 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02c4140e40 con 0x7f02c40611e0 2020-11-28T21:56:15.486 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.474+0000 7f02c1ffb700 1 -- 172.21.15.162:0/22510371 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f02b401a060 con 0x7f02c40611e0 2020-11-28T21:56:15.486 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.474+0000 7f02c1ffb700 1 --2- 172.21.15.162:0/22510371 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f02a8055440 0x7f02a8057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:15.486 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.474+0000 7f02c1ffb700 1 -- 172.21.15.162:0/22510371 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f02b406d3c0 con 0x7f02c40611e0 2020-11-28T21:56:15.487 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.474+0000 7f02c3fff700 1 --2- 172.21.15.162:0/22510371 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f02a8055440 0x7f02a8057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:15.487 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.478+0000 7f02c3fff700 1 --2- 172.21.15.162:0/22510371 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f02a8055440 0x7f02a8057900 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7f02ac01b000 tx=0x7f02ac015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:15.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.482+0000 7f02c1ffb700 1 -- 172.21.15.162:0/22510371 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f02b4043660 con 0x7f02c40611e0 2020-11-28T21:56:15.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.622+0000 7f02c9fde700 1 -- 172.21.15.162:0/22510371 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f02c4057050 con 0x7f02c40611e0 2020-11-28T21:56:15.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.626+0000 7f02c1ffb700 1 -- 172.21.15.162:0/22510371 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v148) v1 ==== 81+0+33755 (secure 0 0 0) 0x7f02b4018020 con 0x7f02c40611e0 2020-11-28T21:56:15.632 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:56:15.632 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2020-11-28T21:56:15.632 INFO:teuthology.orchestra.run.smithi162.stdout: "epoch": 148, 2020-11-28T21:56:15.633 INFO:teuthology.orchestra.run.smithi162.stdout: "fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", 2020-11-28T21:56:15.633 INFO:teuthology.orchestra.run.smithi162.stdout: "created": "2020-11-28T21:46:11.724372+0000", 2020-11-28T21:56:15.633 INFO:teuthology.orchestra.run.smithi162.stdout: "modified": "2020-11-28T21:56:01.362358+0000", 2020-11-28T21:56:15.633 INFO:teuthology.orchestra.run.smithi162.stdout: "last_up_change": "2020-11-28T21:46:19.591650+0000", 2020-11-28T21:56:15.634 INFO:teuthology.orchestra.run.smithi162.stdout: "last_in_change": "2020-11-28T21:46:19.591650+0000", 2020-11-28T21:56:15.634 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-11-28T21:56:15.634 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_num": 5799936, 2020-11-28T21:56:15.634 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_set": [ 2020-11-28T21:56:15.634 INFO:teuthology.orchestra.run.smithi162.stdout: "pglog_hardlimit", 2020-11-28T21:56:15.635 INFO:teuthology.orchestra.run.smithi162.stdout: "purged_snapdirs", 2020-11-28T21:56:15.635 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_deletes", 2020-11-28T21:56:15.635 INFO:teuthology.orchestra.run.smithi162.stdout: "sortbitwise" 2020-11-28T21:56:15.635 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:56:15.636 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_version": 4, 2020-11-28T21:56:15.636 INFO:teuthology.orchestra.run.smithi162.stdout: "full_ratio": 0.94999998807907104, 2020-11-28T21:56:15.636 INFO:teuthology.orchestra.run.smithi162.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-11-28T21:56:15.636 INFO:teuthology.orchestra.run.smithi162.stdout: "nearfull_ratio": 0.85000002384185791, 2020-11-28T21:56:15.636 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_snapshot": "", 2020-11-28T21:56:15.637 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_max": 28, 2020-11-28T21:56:15.637 INFO:teuthology.orchestra.run.smithi162.stdout: "max_osd": 8, 2020-11-28T21:56:15.637 INFO:teuthology.orchestra.run.smithi162.stdout: "require_min_compat_client": "jewel", 2020-11-28T21:56:15.637 INFO:teuthology.orchestra.run.smithi162.stdout: "min_compat_client": "jewel", 2020-11-28T21:56:15.637 INFO:teuthology.orchestra.run.smithi162.stdout: "require_osd_release": "octopus", 2020-11-28T21:56:15.638 INFO:teuthology.orchestra.run.smithi162.stdout: "pools": [ 2020-11-28T21:56:15.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.638 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 1, 2020-11-28T21:56:15.638 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "device_health_metrics", 2020-11-28T21:56:15.638 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:46:15.639551+0000", 2020-11-28T21:56:15.639 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:56:15.639 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:56:15.640 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:56:15.640 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:56:15.641 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:56:15.641 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:56:15.641 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:56:15.642 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:56:15.642 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 1, 2020-11-28T21:56:15.642 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 1, 2020-11-28T21:56:15.642 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 1, 2020-11-28T21:56:15.643 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 1, 2020-11-28T21:56:15.643 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 1, 2020-11-28T21:56:15.643 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:56:15.643 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:56:15.643 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:56:15.644 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:56:15.644 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:56:15.644 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:56:15.644 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:56:15.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.645 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "16", 2020-11-28T21:56:15.645 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:56:15.645 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:56:15.645 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:56:15.645 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:56:15.646 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:56:15.646 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:56:15.646 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:56:15.646 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:56:15.647 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:56:15.647 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:56:15.647 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:56:15.647 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:56:15.647 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:56:15.648 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:56:15.648 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:56:15.648 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:56:15.648 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:56:15.648 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:56:15.649 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:56:15.649 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:56:15.649 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:56:15.649 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:56:15.649 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:56:15.650 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:56:15.650 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:56:15.650 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:56:15.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.651 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:56:15.651 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:56:15.651 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:56:15.651 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:56:15.651 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:56:15.652 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:56:15.652 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:56:15.652 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:56:15.652 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:56:15.652 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:56:15.653 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:56:15.653 INFO:teuthology.orchestra.run.smithi162.stdout: "options": { 2020-11-28T21:56:15.653 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_min": 1 2020-11-28T21:56:15.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.654 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:56:15.654 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr_devicehealth": {} 2020-11-28T21:56:15.654 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:15.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.655 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 2, 2020-11-28T21:56:15.655 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "rbd", 2020-11-28T21:56:15.655 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:46:23.901366+0000", 2020-11-28T21:56:15.655 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:56:15.655 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:56:15.656 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:56:15.656 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:56:15.656 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:56:15.656 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:56:15.656 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:56:15.657 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:56:15.657 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2020-11-28T21:56:15.657 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2020-11-28T21:56:15.657 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2020-11-28T21:56:15.657 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2020-11-28T21:56:15.658 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2020-11-28T21:56:15.658 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:56:15.658 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:56:15.658 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:56:15.659 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:56:15.659 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:56:15.659 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:56:15.659 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:56:15.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.660 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "19", 2020-11-28T21:56:15.660 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:56:15.660 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:56:15.660 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:56:15.660 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:56:15.661 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:56:15.661 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:56:15.661 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:56:15.661 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:56:15.661 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:56:15.662 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:56:15.662 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:56:15.662 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:56:15.662 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:56:15.662 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:56:15.663 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:56:15.663 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:56:15.663 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:56:15.663 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:56:15.663 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:56:15.664 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:56:15.664 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:56:15.664 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:56:15.664 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:56:15.664 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:56:15.665 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:56:15.665 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:56:15.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.665 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:56:15.665 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:56:15.666 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:56:15.666 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:56:15.666 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:56:15.666 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:56:15.666 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:56:15.667 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:56:15.667 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:56:15.667 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:56:15.667 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:56:15.667 INFO:teuthology.orchestra.run.smithi162.stdout: "options": {}, 2020-11-28T21:56:15.668 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:56:15.668 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd": {} 2020-11-28T21:56:15.668 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:15.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.669 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 27, 2020-11-28T21:56:15.669 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "cephfs_metadata", 2020-11-28T21:56:15.669 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:55:59.967971+0000", 2020-11-28T21:56:15.669 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:56:15.669 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:56:15.670 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:56:15.670 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:56:15.670 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:56:15.670 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:56:15.670 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:56:15.671 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:56:15.671 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2020-11-28T21:56:15.671 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2020-11-28T21:56:15.671 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2020-11-28T21:56:15.671 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2020-11-28T21:56:15.672 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2020-11-28T21:56:15.672 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:56:15.672 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:56:15.672 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:56:15.672 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:56:15.673 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:56:15.673 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:56:15.673 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:56:15.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.673 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "148", 2020-11-28T21:56:15.674 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:56:15.674 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:56:15.674 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:56:15.674 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:56:15.674 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:56:15.675 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:56:15.675 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:56:15.675 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:56:15.675 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:56:15.675 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:56:15.676 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:56:15.676 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:56:15.676 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:56:15.676 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:56:15.676 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:56:15.677 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:56:15.677 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:56:15.677 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:56:15.677 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:56:15.677 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:56:15.678 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:56:15.678 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:56:15.678 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:56:15.678 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:56:15.679 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:56:15.679 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:56:15.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.679 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:56:15.679 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:56:15.680 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:56:15.680 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:56:15.680 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:56:15.680 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:56:15.680 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:56:15.680 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:56:15.681 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:56:15.681 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:56:15.681 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:56:15.681 INFO:teuthology.orchestra.run.smithi162.stdout: "options": { 2020-11-28T21:56:15.681 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_bias": 4, 2020-11-28T21:56:15.682 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_min": 16, 2020-11-28T21:56:15.682 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_priority": 5 2020-11-28T21:56:15.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.682 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:56:15.683 INFO:teuthology.orchestra.run.smithi162.stdout: "cephfs": { 2020-11-28T21:56:15.683 INFO:teuthology.orchestra.run.smithi162.stdout: "metadata": "cephfs" 2020-11-28T21:56:15.683 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:15.683 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:15.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.684 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 28, 2020-11-28T21:56:15.684 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "cephfs_data", 2020-11-28T21:56:15.684 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:56:00.379272+0000", 2020-11-28T21:56:15.684 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:56:15.685 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:56:15.685 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:56:15.685 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:56:15.685 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:56:15.685 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:56:15.686 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:56:15.686 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:56:15.686 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2020-11-28T21:56:15.686 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2020-11-28T21:56:15.686 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2020-11-28T21:56:15.687 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2020-11-28T21:56:15.687 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2020-11-28T21:56:15.687 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:56:15.687 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:56:15.687 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:56:15.688 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:56:15.688 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:56:15.688 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:56:15.688 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:56:15.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.689 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "148", 2020-11-28T21:56:15.689 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:56:15.689 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:56:15.689 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:56:15.689 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:56:15.690 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:56:15.690 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:56:15.690 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:56:15.690 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:56:15.690 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:56:15.691 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:56:15.691 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:56:15.691 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:56:15.691 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:56:15.691 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:56:15.692 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:56:15.692 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:56:15.692 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:56:15.692 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:56:15.692 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:56:15.693 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:56:15.693 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:56:15.693 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:56:15.693 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:56:15.693 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:56:15.693 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:56:15.694 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:56:15.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.694 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:56:15.694 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:56:15.695 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:56:15.695 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:56:15.695 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:56:15.695 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:56:15.695 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:56:15.696 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:56:15.696 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:56:15.696 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:56:15.696 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:56:15.696 INFO:teuthology.orchestra.run.smithi162.stdout: "options": {}, 2020-11-28T21:56:15.697 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:56:15.697 INFO:teuthology.orchestra.run.smithi162.stdout: "cephfs": { 2020-11-28T21:56:15.697 INFO:teuthology.orchestra.run.smithi162.stdout: "data": "cephfs" 2020-11-28T21:56:15.697 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:15.697 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:15.698 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:15.698 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:56:15.698 INFO:teuthology.orchestra.run.smithi162.stdout: "osds": [ 2020-11-28T21:56:15.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.698 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 0, 2020-11-28T21:56:15.699 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "99c7d9ae-adfa-4fd7-880a-23e6baf3281b", 2020-11-28T21:56:15.699 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:56:15.699 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:56:15.699 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:56:15.699 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:56:15.700 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:56:15.700 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:56:15.700 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:56:15.700 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 146, 2020-11-28T21:56:15.700 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:56:15.700 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:56:15.701 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:56:15.701 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:15.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.701 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:15.702 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6810", 2020-11-28T21:56:15.702 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:56:15.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.703 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:15.703 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6811", 2020-11-28T21:56:15.703 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:56:15.703 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:15.703 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:15.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.704 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:56:15.704 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:15.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.704 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:15.705 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6812", 2020-11-28T21:56:15.705 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:56:15.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.705 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:15.706 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6813", 2020-11-28T21:56:15.706 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:56:15.706 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:15.706 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:15.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.707 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:56:15.707 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:15.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.707 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:15.707 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6816", 2020-11-28T21:56:15.708 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:56:15.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.708 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:15.708 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6817", 2020-11-28T21:56:15.709 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:56:15.709 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:15.709 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:15.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.709 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:56:15.710 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:15.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.710 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:15.710 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6814", 2020-11-28T21:56:15.710 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:56:15.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.711 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:15.711 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6815", 2020-11-28T21:56:15.711 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:56:15.712 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:15.712 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:15.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.712 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6811/12313", 2020-11-28T21:56:15.712 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6813/12313", 2020-11-28T21:56:15.713 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6817/12313", 2020-11-28T21:56:15.713 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6815/12313", 2020-11-28T21:56:15.713 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:56:15.713 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:56:15.713 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:56:15.714 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:15.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.714 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 1, 2020-11-28T21:56:15.714 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "6ce6220a-00ef-4d13-abba-56f87636fb5f", 2020-11-28T21:56:15.715 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:56:15.715 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:56:15.715 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:56:15.715 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:56:15.715 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:56:15.716 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:56:15.716 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:56:15.716 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 146, 2020-11-28T21:56:15.716 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:56:15.716 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:56:15.717 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:56:15.717 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:15.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.717 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:15.717 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6802", 2020-11-28T21:56:15.718 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:56:15.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.719 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:15.719 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6803", 2020-11-28T21:56:15.719 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:56:15.719 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:15.719 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:15.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.720 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:56:15.720 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:15.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.720 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:15.720 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6804", 2020-11-28T21:56:15.721 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:56:15.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.721 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:15.721 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6805", 2020-11-28T21:56:15.722 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:56:15.722 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:15.722 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:15.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.722 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:56:15.723 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:15.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.723 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:15.723 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6808", 2020-11-28T21:56:15.724 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:56:15.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.724 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:15.724 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6809", 2020-11-28T21:56:15.725 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:56:15.725 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:15.725 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:15.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.725 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:56:15.725 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:15.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.726 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:15.726 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6806", 2020-11-28T21:56:15.726 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:56:15.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.727 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:15.727 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6807", 2020-11-28T21:56:15.727 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:56:15.728 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:15.728 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:15.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.728 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6803/12311", 2020-11-28T21:56:15.728 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6805/12311", 2020-11-28T21:56:15.729 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6809/12311", 2020-11-28T21:56:15.729 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6807/12311", 2020-11-28T21:56:15.729 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:56:15.729 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:56:15.729 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:56:15.730 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:15.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.730 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 2, 2020-11-28T21:56:15.730 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "db4e510d-27c1-47fc-a902-8b32e7dfd73e", 2020-11-28T21:56:15.731 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:56:15.731 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:56:15.731 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:56:15.731 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:56:15.731 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:56:15.732 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:56:15.732 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:56:15.732 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 146, 2020-11-28T21:56:15.732 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:56:15.732 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:56:15.733 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:56:15.733 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:15.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.733 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:15.733 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6818", 2020-11-28T21:56:15.734 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:56:15.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.734 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:15.734 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6819", 2020-11-28T21:56:15.735 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:56:15.735 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:15.735 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:15.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.735 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:56:15.736 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:15.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.736 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:15.736 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6820", 2020-11-28T21:56:15.736 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:56:15.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.737 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:15.737 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6821", 2020-11-28T21:56:15.737 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:56:15.738 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:15.738 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:15.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.738 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:56:15.739 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:15.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.739 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:15.739 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6824", 2020-11-28T21:56:15.739 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:56:15.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.740 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:15.740 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6825", 2020-11-28T21:56:15.740 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:56:15.741 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:15.741 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:15.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.741 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:56:15.741 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:15.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.742 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:15.742 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6822", 2020-11-28T21:56:15.742 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:56:15.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.743 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:15.743 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6823", 2020-11-28T21:56:15.743 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:56:15.743 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:15.744 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:15.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.744 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6819/12312", 2020-11-28T21:56:15.744 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6821/12312", 2020-11-28T21:56:15.744 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6825/12312", 2020-11-28T21:56:15.745 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6823/12312", 2020-11-28T21:56:15.745 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:56:15.745 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:56:15.745 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:56:15.745 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:15.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.746 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 3, 2020-11-28T21:56:15.746 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "4094cb87-f04a-45e1-8710-fb4b651949f6", 2020-11-28T21:56:15.746 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:56:15.747 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:56:15.747 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:56:15.747 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:56:15.747 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:56:15.747 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:56:15.748 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:56:15.748 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 145, 2020-11-28T21:56:15.748 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:56:15.748 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:56:15.748 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:56:15.749 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:15.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.749 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:15.749 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6826", 2020-11-28T21:56:15.749 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:56:15.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.750 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:15.750 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6827", 2020-11-28T21:56:15.750 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:56:15.751 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:15.751 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:15.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.751 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:56:15.751 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:15.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.752 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:15.752 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6828", 2020-11-28T21:56:15.752 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:56:15.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.753 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:15.753 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6829", 2020-11-28T21:56:15.753 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:56:15.753 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:15.754 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:15.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.754 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:56:15.754 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:15.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.755 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:15.755 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6832", 2020-11-28T21:56:15.755 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:56:15.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.756 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:15.756 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6833", 2020-11-28T21:56:15.756 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:56:15.756 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:15.756 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:15.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.757 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:56:15.757 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:15.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.757 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:15.758 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6830", 2020-11-28T21:56:15.758 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:56:15.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.758 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:15.759 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6831", 2020-11-28T21:56:15.759 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:56:15.759 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:15.759 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:15.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.760 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6827/12323", 2020-11-28T21:56:15.760 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6829/12323", 2020-11-28T21:56:15.760 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6833/12323", 2020-11-28T21:56:15.760 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6831/12323", 2020-11-28T21:56:15.760 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:56:15.761 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:56:15.761 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:56:15.761 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:15.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.762 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 4, 2020-11-28T21:56:15.762 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "b907bf53-c297-4756-847b-3be96680eef5", 2020-11-28T21:56:15.762 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:56:15.762 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:56:15.762 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:56:15.763 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:56:15.763 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:56:15.763 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:56:15.763 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:56:15.763 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 145, 2020-11-28T21:56:15.764 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:56:15.764 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:56:15.764 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:56:15.764 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:15.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.765 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:15.765 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6800", 2020-11-28T21:56:15.765 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:56:15.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.766 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:15.766 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6801", 2020-11-28T21:56:15.766 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:56:15.766 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:15.766 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:15.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.767 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:56:15.767 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:15.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.767 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:15.768 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6802", 2020-11-28T21:56:15.768 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:56:15.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.768 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:15.769 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6803", 2020-11-28T21:56:15.769 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:56:15.769 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:15.769 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:15.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.770 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:56:15.770 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:15.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.770 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:15.770 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6806", 2020-11-28T21:56:15.771 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:56:15.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.771 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:15.771 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6807", 2020-11-28T21:56:15.772 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:56:15.772 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:15.772 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:15.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.772 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:56:15.773 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:15.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.773 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:15.773 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6804", 2020-11-28T21:56:15.773 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:56:15.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.774 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:15.774 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6805", 2020-11-28T21:56:15.774 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:56:15.775 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:15.775 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:15.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.775 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6801/12527", 2020-11-28T21:56:15.775 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6803/12527", 2020-11-28T21:56:15.776 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6807/12527", 2020-11-28T21:56:15.776 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6805/12527", 2020-11-28T21:56:15.776 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:56:15.776 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:56:15.776 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:56:15.777 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:15.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.777 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 5, 2020-11-28T21:56:15.777 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "55ba305d-0505-4212-aebb-631e8554b440", 2020-11-28T21:56:15.778 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:56:15.778 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:56:15.778 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:56:15.778 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:56:15.778 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:56:15.779 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:56:15.779 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:56:15.779 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 146, 2020-11-28T21:56:15.779 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:56:15.779 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:56:15.780 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:56:15.780 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:15.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.780 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:15.780 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6808", 2020-11-28T21:56:15.781 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:56:15.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.781 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:15.781 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6809", 2020-11-28T21:56:15.782 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:56:15.782 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:15.782 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:15.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.782 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:56:15.783 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:15.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.783 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:15.783 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6810", 2020-11-28T21:56:15.783 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:56:15.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.784 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:15.784 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6811", 2020-11-28T21:56:15.784 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:56:15.785 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:15.785 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:15.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.785 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:56:15.785 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:15.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.786 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:15.786 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6814", 2020-11-28T21:56:15.786 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:56:15.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.787 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:15.787 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6815", 2020-11-28T21:56:15.787 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:56:15.787 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:15.788 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:15.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.788 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:56:15.788 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:15.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.789 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:15.789 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6812", 2020-11-28T21:56:15.789 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:56:15.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.790 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:15.790 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6813", 2020-11-28T21:56:15.790 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:56:15.790 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:15.790 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:15.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.791 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6809/12529", 2020-11-28T21:56:15.791 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6811/12529", 2020-11-28T21:56:15.791 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6815/12529", 2020-11-28T21:56:15.791 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6813/12529", 2020-11-28T21:56:15.792 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:56:15.792 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:56:15.792 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:56:15.792 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:15.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.793 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 6, 2020-11-28T21:56:15.793 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "9ada6051-493b-46b5-ba22-1b401440490f", 2020-11-28T21:56:15.793 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:56:15.793 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:56:15.794 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:56:15.794 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:56:15.794 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:56:15.794 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:56:15.794 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:56:15.795 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 146, 2020-11-28T21:56:15.795 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:56:15.795 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:56:15.795 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:56:15.795 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:15.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.796 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:15.796 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6824", 2020-11-28T21:56:15.796 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:56:15.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.797 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:15.797 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6825", 2020-11-28T21:56:15.797 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:56:15.797 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:15.797 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:15.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.798 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:56:15.798 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:15.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.799 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:15.799 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6826", 2020-11-28T21:56:15.799 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:56:15.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.799 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:15.800 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6827", 2020-11-28T21:56:15.800 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:56:15.800 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:15.800 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:15.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.801 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:56:15.801 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:15.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.801 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:15.801 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6830", 2020-11-28T21:56:15.802 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:56:15.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.802 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:15.803 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6831", 2020-11-28T21:56:15.803 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:56:15.803 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:15.803 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:15.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.804 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:56:15.804 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:15.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.804 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:15.804 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6828", 2020-11-28T21:56:15.805 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:56:15.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.805 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:15.805 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6829", 2020-11-28T21:56:15.806 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:56:15.806 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:15.806 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:15.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.806 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6825/12530", 2020-11-28T21:56:15.807 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6827/12530", 2020-11-28T21:56:15.807 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6831/12530", 2020-11-28T21:56:15.807 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6829/12530", 2020-11-28T21:56:15.807 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:56:15.807 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:56:15.808 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:56:15.808 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:15.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.808 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 7, 2020-11-28T21:56:15.809 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "c328b06d-a216-4cba-aad7-9ad081340613", 2020-11-28T21:56:15.809 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:56:15.809 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:56:15.809 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:56:15.809 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:56:15.810 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:56:15.810 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:56:15.810 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:56:15.810 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 146, 2020-11-28T21:56:15.810 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:56:15.811 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:56:15.814 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:56:15.814 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:15.815 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.815 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:15.815 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6816", 2020-11-28T21:56:15.815 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:56:15.815 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.816 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.816 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:15.816 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6817", 2020-11-28T21:56:15.816 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:56:15.817 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:15.817 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:15.817 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.817 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:56:15.817 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:15.818 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.818 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:15.818 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6818", 2020-11-28T21:56:15.818 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:56:15.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.819 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.819 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:15.819 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6819", 2020-11-28T21:56:15.819 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:56:15.819 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:15.820 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:15.820 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.820 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:56:15.820 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:15.821 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.821 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:15.821 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6822", 2020-11-28T21:56:15.821 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:56:15.821 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.822 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.822 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:15.822 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6823", 2020-11-28T21:56:15.822 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:56:15.822 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:15.823 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:15.823 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.823 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:56:15.823 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:15.823 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.824 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:15.824 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6820", 2020-11-28T21:56:15.824 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:56:15.824 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.824 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.825 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:15.825 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6821", 2020-11-28T21:56:15.825 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:56:15.825 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:15.825 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:15.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.826 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6817/12528", 2020-11-28T21:56:15.828 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6819/12528", 2020-11-28T21:56:15.829 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6823/12528", 2020-11-28T21:56:15.829 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6821/12528", 2020-11-28T21:56:15.829 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:56:15.829 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:56:15.829 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:56:15.830 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:15.830 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:15.830 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:56:15.830 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_xinfo": [ 2020-11-28T21:56:15.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.831 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 0, 2020-11-28T21:56:15.831 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:56:15.831 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:56:15.831 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:56:15.832 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:56:15.832 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:56:15.832 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.038052+0000", 2020-11-28T21:56:15.832 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:56:15.832 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.833 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.833 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 1, 2020-11-28T21:56:15.833 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:56:15.833 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:56:15.834 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:56:15.834 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:56:15.834 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:56:15.834 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:17.947136+0000", 2020-11-28T21:56:15.834 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:56:15.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.835 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.835 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 2, 2020-11-28T21:56:15.835 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:56:15.835 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:56:15.836 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:56:15.836 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:56:15.836 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:56:15.836 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.001186+0000", 2020-11-28T21:56:15.836 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:56:15.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.837 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 3, 2020-11-28T21:56:15.837 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:56:15.837 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:56:15.838 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:56:15.838 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:56:15.838 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:56:15.839 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.199284+0000", 2020-11-28T21:56:15.839 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:56:15.839 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.839 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.840 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 4, 2020-11-28T21:56:15.840 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:56:15.840 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:56:15.840 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:56:15.840 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:56:15.841 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:56:15.841 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.038823+0000", 2020-11-28T21:56:15.841 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:56:15.841 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.841 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.842 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 5, 2020-11-28T21:56:15.842 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:56:15.842 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:56:15.842 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:56:15.842 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:56:15.843 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:56:15.843 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.009322+0000", 2020-11-28T21:56:15.843 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:56:15.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.844 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 6, 2020-11-28T21:56:15.844 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:56:15.844 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:56:15.844 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:56:15.844 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:56:15.845 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:56:15.845 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.238927+0000", 2020-11-28T21:56:15.845 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:56:15.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:15.846 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 7, 2020-11-28T21:56:15.846 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:56:15.846 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:56:15.846 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:56:15.847 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:56:15.847 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:56:15.847 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.029839+0000", 2020-11-28T21:56:15.847 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:56:15.847 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:15.848 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:56:15.848 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_upmap": [], 2020-11-28T21:56:15.848 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_upmap_items": [], 2020-11-28T21:56:15.848 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_temp": [], 2020-11-28T21:56:15.849 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_temp": [], 2020-11-28T21:56:15.849 INFO:teuthology.orchestra.run.smithi162.stdout: "blacklist": {}, 2020-11-28T21:56:15.849 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profiles": { 2020-11-28T21:56:15.849 INFO:teuthology.orchestra.run.smithi162.stdout: "default": { 2020-11-28T21:56:15.850 INFO:teuthology.orchestra.run.smithi162.stdout: "crush-failure-domain": "osd", 2020-11-28T21:56:15.850 INFO:teuthology.orchestra.run.smithi162.stdout: "k": "2", 2020-11-28T21:56:15.850 INFO:teuthology.orchestra.run.smithi162.stdout: "m": "1", 2020-11-28T21:56:15.850 INFO:teuthology.orchestra.run.smithi162.stdout: "plugin": "jerasure", 2020-11-28T21:56:15.850 INFO:teuthology.orchestra.run.smithi162.stdout: "ruleset-failure-domain": "osd", 2020-11-28T21:56:15.851 INFO:teuthology.orchestra.run.smithi162.stdout: "technique": "reed_sol_van" 2020-11-28T21:56:15.851 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:15.851 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:15.851 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps_queue": [], 2020-11-28T21:56:15.852 INFO:teuthology.orchestra.run.smithi162.stdout: "new_removed_snaps": [], 2020-11-28T21:56:15.852 INFO:teuthology.orchestra.run.smithi162.stdout: "new_purged_snaps": [], 2020-11-28T21:56:15.852 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_node_flags": {}, 2020-11-28T21:56:15.852 INFO:teuthology.orchestra.run.smithi162.stdout: "device_class_flags": {} 2020-11-28T21:56:15.852 INFO:teuthology.orchestra.run.smithi162.stdout:} 2020-11-28T21:56:15.854 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.626+0000 7f02c9fde700 1 -- 172.21.15.162:0/22510371 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f02a8055440 msgr2=0x7f02a8057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:15.854 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.626+0000 7f02c9fde700 1 --2- 172.21.15.162:0/22510371 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f02a8055440 0x7f02a8057900 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7f02ac01b000 tx=0x7f02ac015000).stop 2020-11-28T21:56:15.855 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.626+0000 7f02c9fde700 1 -- 172.21.15.162:0/22510371 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02c40611e0 msgr2=0x7f02c4140610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:15.855 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.626+0000 7f02c9fde700 1 --2- 172.21.15.162:0/22510371 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02c40611e0 0x7f02c4140610 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 rx=0x7f02c413c140 tx=0x7f02b4007aa0).stop 2020-11-28T21:56:15.855 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.630+0000 7f02c9fde700 1 -- 172.21.15.162:0/22510371 shutdown_connections 2020-11-28T21:56:15.855 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.630+0000 7f02c9fde700 1 --2- 172.21.15.162:0/22510371 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f02a8055440 0x7f02a8057900 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:15.856 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.630+0000 7f02c9fde700 1 --2- 172.21.15.162:0/22510371 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f02c40b4b40 0x7f02c40aa9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:15.856 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.630+0000 7f02c9fde700 1 --2- 172.21.15.162:0/22510371 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02c40611e0 0x7f02c4140610 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:15.856 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.630+0000 7f02c9fde700 1 --2- 172.21.15.162:0/22510371 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f02c40615c0 0x7f02c40aa4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:15.856 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.630+0000 7f02c9fde700 1 -- 172.21.15.162:0/22510371 >> 172.21.15.162:0/22510371 conn(0x7f02c40b54f0 msgr2=0x7f02c413cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:15.856 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.630+0000 7f02c9fde700 1 -- 172.21.15.162:0/22510371 shutdown_connections 2020-11-28T21:56:15.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.630+0000 7f02c9fde700 1 -- 172.21.15.162:0/22510371 wait complete. 2020-11-28T21:56:15.857 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:56:15.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.970+0000 7fed1cc5e700 1 Processor -- start 2020-11-28T21:56:15.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.970+0000 7fed1cc5e700 1 -- start start 2020-11-28T21:56:15.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.970+0000 7fed1cc5e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fed18064c70 0x7fed18066290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:15.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.970+0000 7fed1cc5e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fed18065750 0x7fed1813c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:15.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.970+0000 7fed1cc5e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed18065050 con 0x7fed1813f6c0 2020-11-28T21:56:15.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.970+0000 7fed1cc5e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed1813fc40 con 0x7fed18065750 2020-11-28T21:56:15.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.970+0000 7fed1cc5e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed1813fee0 con 0x7fed18064c70 2020-11-28T21:56:15.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.970+0000 7fed177fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fed18064c70 0x7fed18066290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:15.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.970+0000 7fed16ffd700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fed1813f6c0 0x7fed18066780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:52872/0 (socket says 172.21.15.162:52872) 2020-11-28T21:56:15.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.970+0000 7fed16ffd700 1 -- 172.21.15.162:0/2022906928 learned_addr learned my addr 172.21.15.162:0/2022906928 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:56:15.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.970+0000 7fed17fff700 1 --2- 172.21.15.162:0/2022906928 >> v2:172.21.15.118:3300/0 conn(0x7fed18065750 0x7fed1813c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:15.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.970+0000 7fed177fe700 1 -- 172.21.15.162:0/2022906928 >> v2:172.21.15.118:3300/0 conn(0x7fed18065750 msgr2=0x7fed1813c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:15.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.970+0000 7fed177fe700 1 --2- 172.21.15.162:0/2022906928 >> v2:172.21.15.118:3300/0 conn(0x7fed18065750 0x7fed1813c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:15.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.970+0000 7fed177fe700 1 -- 172.21.15.162:0/2022906928 >> v1:172.21.15.162:6789/0 conn(0x7fed1813f6c0 legacy=0x7fed18066780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:15.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.970+0000 7fed177fe700 1 -- 172.21.15.162:0/2022906928 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed18140180 con 0x7fed18064c70 2020-11-28T21:56:15.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.970+0000 7fed17fff700 1 --2- 172.21.15.162:0/2022906928 >> v2:172.21.15.118:3300/0 conn(0x7fed18065750 0x7fed1813c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:56:15.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.970+0000 7fed177fe700 1 --2- 172.21.15.162:0/2022906928 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed18064c70 0x7fed18066290 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 rx=0x7fed080032b0 tx=0x7fed080091a0).ready entity=mon.1 client_cookie=3bb79621b4e58c62 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:15.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.970+0000 7fed167fc700 1 -- 172.21.15.162:0/2022906928 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed0800ba50 con 0x7fed18064c70 2020-11-28T21:56:15.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.970+0000 7fed167fc700 1 -- 172.21.15.162:0/2022906928 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed0800bbb0 con 0x7fed18064c70 2020-11-28T21:56:15.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.970+0000 7fed167fc700 1 -- 172.21.15.162:0/2022906928 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed0800bd20 con 0x7fed18064c70 2020-11-28T21:56:15.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.970+0000 7fed1cc5e700 1 -- 172.21.15.162:0/2022906928 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed18064c70 msgr2=0x7fed18066290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:15.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.970+0000 7fed1cc5e700 1 --2- 172.21.15.162:0/2022906928 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed18064c70 0x7fed18066290 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 rx=0x7fed080032b0 tx=0x7fed080091a0).stop 2020-11-28T21:56:15.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.974+0000 7fed1cc5e700 1 -- 172.21.15.162:0/2022906928 shutdown_connections 2020-11-28T21:56:15.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.974+0000 7fed1cc5e700 1 --2- 172.21.15.162:0/2022906928 >> v2:172.21.15.118:3300/0 conn(0x7fed18065750 0x7fed1813c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:15.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.974+0000 7fed1cc5e700 1 --2- 172.21.15.162:0/2022906928 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed18064c70 0x7fed18066290 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:15.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.974+0000 7fed1cc5e700 1 -- 172.21.15.162:0/2022906928 >> 172.21.15.162:0/2022906928 conn(0x7fed180b5da0 msgr2=0x7fed180b6180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:15.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.974+0000 7fed1cc5e700 1 -- 172.21.15.162:0/2022906928 shutdown_connections 2020-11-28T21:56:15.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.974+0000 7fed1cc5e700 1 -- 172.21.15.162:0/2022906928 wait complete. 2020-11-28T21:56:15.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.974+0000 7fed1cc5e700 1 Processor -- start 2020-11-28T21:56:15.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.974+0000 7fed1cc5e700 1 -- start start 2020-11-28T21:56:15.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.978+0000 7fed1cc5e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed18064c70 0x7fed1813ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:15.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.978+0000 7fed1cc5e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fed18065750 0x7fed180ad5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:15.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.978+0000 7fed1cc5e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed1813f6c0 0x7fed180adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:15.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.978+0000 7fed1cc5e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed180b56a0 con 0x7fed1813f6c0 2020-11-28T21:56:15.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.978+0000 7fed1cc5e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed180ad1e0 con 0x7fed18064c70 2020-11-28T21:56:15.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.978+0000 7fed1cc5e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed180b5400 con 0x7fed18065750 2020-11-28T21:56:15.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.978+0000 7fed177fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed18064c70 0x7fed1813ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:15.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.978+0000 7fed177fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed18064c70 0x7fed1813ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46032/0 (socket says 172.21.15.162:46032) 2020-11-28T21:56:15.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.978+0000 7fed177fe700 1 -- 172.21.15.162:0/3368055276 learned_addr learned my addr 172.21.15.162:0/3368055276 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:15.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.978+0000 7fed17fff700 1 --2- 172.21.15.162:0/3368055276 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed1813f6c0 0x7fed180adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:15.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.978+0000 7fed16ffd700 1 --2- 172.21.15.162:0/3368055276 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fed18065750 0x7fed180ad5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:15.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.978+0000 7fed177fe700 1 -- 172.21.15.162:0/3368055276 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fed18065750 msgr2=0x7fed180ad5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:15.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.978+0000 7fed177fe700 1 --2- 172.21.15.162:0/3368055276 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fed18065750 0x7fed180ad5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:15.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.978+0000 7fed177fe700 1 -- 172.21.15.162:0/3368055276 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed1813f6c0 msgr2=0x7fed180adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:15.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.978+0000 7fed177fe700 1 --2- 172.21.15.162:0/3368055276 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed1813f6c0 0x7fed180adaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:15.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.978+0000 7fed177fe700 1 -- 172.21.15.162:0/3368055276 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed180b4880 con 0x7fed18064c70 2020-11-28T21:56:15.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.978+0000 7fed16ffd700 1 --2- 172.21.15.162:0/3368055276 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fed18065750 0x7fed180ad5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:56:15.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.978+0000 7fed177fe700 1 --2- 172.21.15.162:0/3368055276 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed18064c70 0x7fed1813ad90 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 rx=0x7fed080032b0 tx=0x7fed08016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:15.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.978+0000 7fed14ff9700 1 -- 172.21.15.162:0/3368055276 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed08010660 con 0x7fed18064c70 2020-11-28T21:56:15.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.978+0000 7fed14ff9700 1 -- 172.21.15.162:0/3368055276 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed080107c0 con 0x7fed18064c70 2020-11-28T21:56:15.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.978+0000 7fed1cc5e700 1 -- 172.21.15.162:0/3368055276 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed180b4b20 con 0x7fed18064c70 2020-11-28T21:56:15.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.978+0000 7fed14ff9700 1 -- 172.21.15.162:0/3368055276 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed08010b20 con 0x7fed18064c70 2020-11-28T21:56:15.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.978+0000 7fed1cc5e700 1 -- 172.21.15.162:0/3368055276 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed180b3bb0 con 0x7fed18064c70 2020-11-28T21:56:15.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.982+0000 7fed1cc5e700 1 -- 172.21.15.162:0/3368055276 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed180b4cb0 con 0x7fed18064c70 2020-11-28T21:56:15.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.982+0000 7fed14ff9700 1 -- 172.21.15.162:0/3368055276 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fed08026030 con 0x7fed18064c70 2020-11-28T21:56:15.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.982+0000 7fed14ff9700 1 --2- 172.21.15.162:0/3368055276 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fecf80554e0 0x7fecf80579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:15.992 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.982+0000 7fed14ff9700 1 -- 172.21.15.162:0/3368055276 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fed0806f050 con 0x7fed18064c70 2020-11-28T21:56:15.992 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.982+0000 7fed16ffd700 1 --2- 172.21.15.162:0/3368055276 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fecf80554e0 0x7fecf80579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:15.992 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.982+0000 7fed16ffd700 1 --2- 172.21.15.162:0/3368055276 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fecf80554e0 0x7fecf80579a0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7fed0c00b000 tx=0x7fed0c009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:15.997 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:15.990+0000 7fed14ff9700 1 -- 172.21.15.162:0/3368055276 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fed080452b0 con 0x7fed18064c70 2020-11-28T21:56:16.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.178+0000 7fed1cc5e700 1 -- 172.21.15.162:0/3368055276 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed18057050 con 0x7fed18064c70 2020-11-28T21:56:16.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.178+0000 7fed14ff9700 1 -- 172.21.15.162:0/3368055276 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+2504 (secure 0 0 0) 0x7fed08024090 con 0x7fed18064c70 2020-11-28T21:56:16.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.182+0000 7fed1cc5e700 1 -- 172.21.15.162:0/3368055276 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fecf80554e0 msgr2=0x7fecf80579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:16.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.182+0000 7fed1cc5e700 1 --2- 172.21.15.162:0/3368055276 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fecf80554e0 0x7fecf80579a0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7fed0c00b000 tx=0x7fed0c009040).stop 2020-11-28T21:56:16.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.182+0000 7fed1cc5e700 1 -- 172.21.15.162:0/3368055276 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed18064c70 msgr2=0x7fed1813ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:16.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.182+0000 7fed1cc5e700 1 --2- 172.21.15.162:0/3368055276 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed18064c70 0x7fed1813ad90 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 rx=0x7fed080032b0 tx=0x7fed08016000).stop 2020-11-28T21:56:16.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.182+0000 7fed1cc5e700 1 -- 172.21.15.162:0/3368055276 shutdown_connections 2020-11-28T21:56:16.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.182+0000 7fed1cc5e700 1 --2- 172.21.15.162:0/3368055276 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fecf80554e0 0x7fecf80579a0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:16.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.182+0000 7fed1cc5e700 1 --2- 172.21.15.162:0/3368055276 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed1813f6c0 0x7fed180adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:16.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.182+0000 7fed1cc5e700 1 --2- 172.21.15.162:0/3368055276 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed18064c70 0x7fed1813ad90 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:16.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.182+0000 7fed1cc5e700 1 --2- 172.21.15.162:0/3368055276 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fed18065750 0x7fed180ad5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:16.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.182+0000 7fed1cc5e700 1 -- 172.21.15.162:0/3368055276 >> 172.21.15.162:0/3368055276 conn(0x7fed180b5da0 msgr2=0x7fed1813d6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:16.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.182+0000 7fed1cc5e700 1 -- 172.21.15.162:0/3368055276 shutdown_connections 2020-11-28T21:56:16.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.182+0000 7fed1cc5e700 1 -- 172.21.15.162:0/3368055276 wait complete. 2020-11-28T21:56:16.190 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 74 2020-11-28T21:56:16.190 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:56:16.190 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":74,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6899,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.118:6833/2509028245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":2509028245},{"type":"v1","addr":"172.21.15.118:6833","nonce":2509028245}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":72},{"gid":8239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/2089791609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2089791609},{"type":"v1","addr":"172.21.15.162:6836","nonce":2089791609}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":71}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:56:01.362652+0000","modified":"2020-11-28T21:56:03.027360+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8242},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8242":{"gid":8242,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/2832109513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":2832109513},{"type":"v1","addr":"172.21.15.162:6837","nonce":2832109513}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2020-11-28T21:56:16.197 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2020-11-28T21:56:16.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.310+0000 7f61abfff700 1 Processor -- start 2020-11-28T21:56:16.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.310+0000 7f61abfff700 1 -- start start 2020-11-28T21:56:16.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.310+0000 7f61abfff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f61ac147270 0x7f61ac0b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:16.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.310+0000 7f61abfff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61ac0b0830 con 0x7f61ac0b3e20 2020-11-28T21:56:16.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.310+0000 7f61abfff700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61ac060e60 con 0x7f61ac0608e0 2020-11-28T21:56:16.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.310+0000 7f61abfff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac0b4200 con 0x7f61ac147270 2020-11-28T21:56:16.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.310+0000 7f61ab7fe700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f61ac0608e0 0x7f61ac0b11a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:52886/0 (socket says 172.21.15.162:52886) 2020-11-28T21:56:16.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.310+0000 7f61ab7fe700 1 -- 172.21.15.162:0/46680226 learned_addr learned my addr 172.21.15.162:0/46680226 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:56:16.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.310+0000 7f61aa7fc700 1 --2- 172.21.15.162:0/46680226 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f61ac147270 0x7f61ac0b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:16.318 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.310+0000 7f61a9ffb700 1 -- 172.21.15.162:0/46680226 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2568952775 0 0) 0x7f61ac060e60 con 0x7f61ac0608e0 2020-11-28T21:56:16.318 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.310+0000 7f61a9ffb700 1 -- 172.21.15.162:0/46680226 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6190003230 con 0x7f61ac0608e0 2020-11-28T21:56:16.318 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.310+0000 7f61a9ffb700 1 -- 172.21.15.162:0/46680226 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2547036996 0 0) 0x7f61ac0b0830 con 0x7f61ac0b3e20 2020-11-28T21:56:16.318 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.310+0000 7f61a9ffb700 1 -- 172.21.15.162:0/46680226 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61ac060e60 con 0x7f61ac0b3e20 2020-11-28T21:56:16.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.310+0000 7f61aa7fc700 1 -- 172.21.15.162:0/46680226 >> v1:172.21.15.162:6789/0 conn(0x7f61ac0608e0 legacy=0x7f61ac0b11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:16.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.310+0000 7f61aa7fc700 1 -- 172.21.15.162:0/46680226 >> v1:172.21.15.118:6789/0 conn(0x7f61ac0b3e20 legacy=0x7f61ac146b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:16.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.310+0000 7f61aa7fc700 1 -- 172.21.15.162:0/46680226 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61ac0b0a60 con 0x7f61ac147270 2020-11-28T21:56:16.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.310+0000 7f61aa7fc700 1 --2- 172.21.15.162:0/46680226 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f61ac147270 0x7f61ac0b4680 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f61a4002f30 tx=0x7f61a4006dc0).ready entity=mon.2 client_cookie=d1a24d5fa685a3a0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:16.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.310+0000 7f61a9ffb700 1 -- 172.21.15.162:0/46680226 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61a4009730 con 0x7f61ac147270 2020-11-28T21:56:16.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.310+0000 7f61a9ffb700 1 -- 172.21.15.162:0/46680226 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a4009890 con 0x7f61ac147270 2020-11-28T21:56:16.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.310+0000 7f61abfff700 1 -- 172.21.15.162:0/46680226 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f61ac147270 msgr2=0x7f61ac0b4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:16.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.310+0000 7f61a9ffb700 1 -- 172.21.15.162:0/46680226 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61a4009bf0 con 0x7f61ac147270 2020-11-28T21:56:16.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.310+0000 7f61abfff700 1 --2- 172.21.15.162:0/46680226 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f61ac147270 0x7f61ac0b4680 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f61a4002f30 tx=0x7f61a4006dc0).stop 2020-11-28T21:56:16.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.310+0000 7f61abfff700 1 -- 172.21.15.162:0/46680226 shutdown_connections 2020-11-28T21:56:16.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.310+0000 7f61abfff700 1 --2- 172.21.15.162:0/46680226 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f61ac147270 0x7f61ac0b4680 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:16.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.310+0000 7f61abfff700 1 -- 172.21.15.162:0/46680226 >> 172.21.15.162:0/46680226 conn(0x7f61ac0b54f0 msgr2=0x7f61ac0b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:16.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.310+0000 7f61abfff700 1 -- 172.21.15.162:0/46680226 shutdown_connections 2020-11-28T21:56:16.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.310+0000 7f61abfff700 1 -- 172.21.15.162:0/46680226 wait complete. 2020-11-28T21:56:16.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.314+0000 7f61abfff700 1 Processor -- start 2020-11-28T21:56:16.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.314+0000 7f61abfff700 1 -- start start 2020-11-28T21:56:16.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.314+0000 7f61abfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f61ac0608e0 0x7f61ac099fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:16.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.314+0000 7f61abfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61ac0b3e20 0x7f61ac09af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:16.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.314+0000 7f61abfff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f61ac147270 0x7f61ac155b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:16.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.314+0000 7f61abfff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac14cdf0 con 0x7f61ac0608e0 2020-11-28T21:56:16.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.314+0000 7f61abfff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac09a880 con 0x7f61ac0b3e20 2020-11-28T21:56:16.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.314+0000 7f61abfff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac149450 con 0x7f61ac147270 2020-11-28T21:56:16.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.314+0000 7f61aa7fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61ac0b3e20 0x7f61ac09af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:16.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.314+0000 7f61aaffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f61ac0608e0 0x7f61ac099fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:16.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.314+0000 7f61aaffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f61ac0608e0 0x7f61ac099fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.162:53034/0 (socket says 172.21.15.162:53034) 2020-11-28T21:56:16.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.314+0000 7f61aaffd700 1 -- 172.21.15.162:0/872778304 learned_addr learned my addr 172.21.15.162:0/872778304 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:16.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.314+0000 7f61aa7fc700 1 --2- 172.21.15.162:0/872778304 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61ac0b3e20 0x7f61ac09af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46050/0 (socket says 172.21.15.162:46050) 2020-11-28T21:56:16.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.314+0000 7f61ab7fe700 1 --2- 172.21.15.162:0/872778304 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f61ac147270 0x7f61ac155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:16.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.314+0000 7f61aa7fc700 1 -- 172.21.15.162:0/872778304 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f61ac147270 msgr2=0x7f61ac155b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:16.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.314+0000 7f61aa7fc700 1 --2- 172.21.15.162:0/872778304 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f61ac147270 0x7f61ac155b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:16.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.314+0000 7f61aa7fc700 1 -- 172.21.15.162:0/872778304 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f61ac0608e0 msgr2=0x7f61ac099fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:16.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.314+0000 7f61aa7fc700 1 --2- 172.21.15.162:0/872778304 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f61ac0608e0 0x7f61ac099fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:16.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.314+0000 7f61aa7fc700 1 -- 172.21.15.162:0/872778304 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61ac1496f0 con 0x7f61ac0b3e20 2020-11-28T21:56:16.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.314+0000 7f61aaffd700 1 --2- 172.21.15.162:0/872778304 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f61ac0608e0 0x7f61ac099fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:56:16.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.318+0000 7f61aa7fc700 1 --2- 172.21.15.162:0/872778304 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61ac0b3e20 0x7f61ac09af60 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 rx=0x7f61a4002f30 tx=0x7f61a4015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:16.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.318+0000 7f61a3fff700 1 -- 172.21.15.162:0/872778304 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61a4011a20 con 0x7f61ac0b3e20 2020-11-28T21:56:16.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.318+0000 7f61a3fff700 1 -- 172.21.15.162:0/872778304 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a4011b80 con 0x7f61ac0b3e20 2020-11-28T21:56:16.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.318+0000 7f61abfff700 1 -- 172.21.15.162:0/872778304 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61ac09a570 con 0x7f61ac0b3e20 2020-11-28T21:56:16.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.318+0000 7f61a3fff700 1 -- 172.21.15.162:0/872778304 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61a401d450 con 0x7f61ac0b3e20 2020-11-28T21:56:16.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.318+0000 7f61abfff700 1 -- 172.21.15.162:0/872778304 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61ac149b90 con 0x7f61ac0b3e20 2020-11-28T21:56:16.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.318+0000 7f61a3fff700 1 -- 172.21.15.162:0/872778304 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f61a4024030 con 0x7f61ac0b3e20 2020-11-28T21:56:16.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.318+0000 7f61a3fff700 1 --2- 172.21.15.162:0/872778304 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6184055440 0x7f6184057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:16.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.318+0000 7f61abfff700 1 -- 172.21.15.162:0/872778304 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61ac09aa40 con 0x7f61ac0b3e20 2020-11-28T21:56:16.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.318+0000 7f61aaffd700 1 --2- 172.21.15.162:0/872778304 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6184055440 0x7f6184057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:16.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.318+0000 7f61a3fff700 1 -- 172.21.15.162:0/872778304 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f61a406eff0 con 0x7f61ac0b3e20 2020-11-28T21:56:16.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.318+0000 7f61aaffd700 1 --2- 172.21.15.162:0/872778304 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6184055440 0x7f6184057900 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7f619c00d000 tx=0x7f619c007e30).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:16.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.326+0000 7f61a3fff700 1 -- 172.21.15.162:0/872778304 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f61a4022070 con 0x7f61ac0b3e20 2020-11-28T21:56:16.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:16.506+0000 7f61abfff700 1 -- 172.21.15.162:0/872778304 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f61ac057050 con 0x7f61ac0b3e20 2020-11-28T21:56:17.178 INFO:tasks.ceph.mon.b.smithi118.stderr:2020-11-28T21:56:17.175+0000 7ff4e1520700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-11-28T21:56:17.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.182+0000 7f61a3fff700 1 -- 172.21.15.162:0/872778304 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v75) v1 ==== 161+0+0 (secure 0 0 0) 0x7f61a4048cc0 con 0x7f61ac0b3e20 2020-11-28T21:56:17.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.186+0000 7f61abfff700 1 -- 172.21.15.162:0/872778304 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6184055440 msgr2=0x7f6184057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:17.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.186+0000 7f61abfff700 1 --2- 172.21.15.162:0/872778304 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6184055440 0x7f6184057900 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7f619c00d000 tx=0x7f619c007e30).stop 2020-11-28T21:56:17.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.186+0000 7f61abfff700 1 -- 172.21.15.162:0/872778304 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61ac0b3e20 msgr2=0x7f61ac09af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:17.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.186+0000 7f61abfff700 1 --2- 172.21.15.162:0/872778304 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61ac0b3e20 0x7f61ac09af60 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 rx=0x7f61a4002f30 tx=0x7f61a4015000).stop 2020-11-28T21:56:17.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.186+0000 7f61abfff700 1 -- 172.21.15.162:0/872778304 shutdown_connections 2020-11-28T21:56:17.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.186+0000 7f61abfff700 1 --2- 172.21.15.162:0/872778304 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6184055440 0x7f6184057900 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:17.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.186+0000 7f61abfff700 1 --2- 172.21.15.162:0/872778304 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f61ac147270 0x7f61ac155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:17.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.186+0000 7f61abfff700 1 --2- 172.21.15.162:0/872778304 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f61ac0608e0 0x7f61ac099fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:17.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.186+0000 7f61abfff700 1 --2- 172.21.15.162:0/872778304 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61ac0b3e20 0x7f61ac09af60 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:17.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.186+0000 7f61abfff700 1 -- 172.21.15.162:0/872778304 >> 172.21.15.162:0/872778304 conn(0x7f61ac0b54f0 msgr2=0x7f61ac05c800 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:17.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.186+0000 7f61abfff700 1 -- 172.21.15.162:0/872778304 shutdown_connections 2020-11-28T21:56:17.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.186+0000 7f61abfff700 1 -- 172.21.15.162:0/872778304 wait complete. 2020-11-28T21:56:17.195 INFO:teuthology.orchestra.run.smithi162.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-11-28T21:56:17.213 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:56:17.215 INFO:tasks.ceph.mds.a.smithi162.stdout:starting mds.a at 2020-11-28T21:56:17.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.310+0000 7f6f9b59e700 1 Processor -- start 2020-11-28T21:56:17.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.310+0000 7f6f9b59e700 1 -- start start 2020-11-28T21:56:17.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.310+0000 7f6f9b59e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6f9c0603d0 0x7f6f9c0b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:17.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.310+0000 7f6f9b59e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f9c0b4110 con 0x7f6f9c142a00 2020-11-28T21:56:17.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.310+0000 7f6f9b59e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f9c0b43f0 con 0x7f6f9c0b4c70 2020-11-28T21:56:17.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.310+0000 7f6f9b59e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f9c060f60 con 0x7f6f9c0603d0 2020-11-28T21:56:17.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.310+0000 7f6f9ad9d700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f6f9c0b4c70 0x7f6f9c143020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:52914/0 (socket says 172.21.15.162:52914) 2020-11-28T21:56:17.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.310+0000 7f6f9ad9d700 1 -- 172.21.15.162:0/532941573 learned_addr learned my addr 172.21.15.162:0/532941573 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:56:17.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.310+0000 7f6f9a59c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6f9c0603d0 0x7f6f9c0b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:17.318 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.310+0000 7f6f9959a700 1 -- 172.21.15.162:0/532941573 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1342570030 0 0) 0x7f6f9c0b43f0 con 0x7f6f9c0b4c70 2020-11-28T21:56:17.318 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.310+0000 7f6f9959a700 1 -- 172.21.15.162:0/532941573 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f80003230 con 0x7f6f9c0b4c70 2020-11-28T21:56:17.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.310+0000 7f6f9a59c700 1 -- 172.21.15.162:0/532941573 >> v1:172.21.15.162:6789/0 conn(0x7f6f9c0b4c70 legacy=0x7f6f9c143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:17.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.310+0000 7f6f9a59c700 1 -- 172.21.15.162:0/532941573 >> v1:172.21.15.118:6789/0 conn(0x7f6f9c142a00 legacy=0x7f6f9c09cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:17.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.310+0000 7f6f9959a700 1 -- 172.21.15.162:0/532941573 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2994316542 0 0) 0x7f6f9c0b4110 con 0x7f6f9c142a00 2020-11-28T21:56:17.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.310+0000 7f6f9a59c700 1 -- 172.21.15.162:0/532941573 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f9c0b4690 con 0x7f6f9c0603d0 2020-11-28T21:56:17.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.310+0000 7f6f9a59c700 1 --2- 172.21.15.162:0/532941573 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f9c0603d0 0x7f6f9c0b3a70 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 rx=0x7f6f8c002f30 tx=0x7f6f8c006dc0).ready entity=mon.1 client_cookie=f219a07c188ba250 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:17.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.310+0000 7f6f9959a700 1 -- 172.21.15.162:0/532941573 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f8c009800 con 0x7f6f9c0603d0 2020-11-28T21:56:17.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.310+0000 7f6f9959a700 1 -- 172.21.15.162:0/532941573 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f8c009960 con 0x7f6f9c0603d0 2020-11-28T21:56:17.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.310+0000 7f6f9959a700 1 -- 172.21.15.162:0/532941573 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f8c009cc0 con 0x7f6f9c0603d0 2020-11-28T21:56:17.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.310+0000 7f6f9b59e700 1 -- 172.21.15.162:0/532941573 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f9c0603d0 msgr2=0x7f6f9c0b3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:17.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.310+0000 7f6f9b59e700 1 --2- 172.21.15.162:0/532941573 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f9c0603d0 0x7f6f9c0b3a70 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 rx=0x7f6f8c002f30 tx=0x7f6f8c006dc0).stop 2020-11-28T21:56:17.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.314+0000 7f6f9b59e700 1 -- 172.21.15.162:0/532941573 shutdown_connections 2020-11-28T21:56:17.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.314+0000 7f6f9b59e700 1 --2- 172.21.15.162:0/532941573 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f9c0603d0 0x7f6f9c0b3a70 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:17.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.314+0000 7f6f9b59e700 1 -- 172.21.15.162:0/532941573 >> 172.21.15.162:0/532941573 conn(0x7f6f9c066a40 msgr2=0x7f6f9c066e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:17.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.314+0000 7f6f9b59e700 1 -- 172.21.15.162:0/532941573 shutdown_connections 2020-11-28T21:56:17.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.314+0000 7f6f9b59e700 1 -- 172.21.15.162:0/532941573 wait complete. 2020-11-28T21:56:17.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.314+0000 7f6f9b59e700 1 Processor -- start 2020-11-28T21:56:17.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.314+0000 7f6f9b59e700 1 -- start start 2020-11-28T21:56:17.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.314+0000 7f6f9b59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6f9c0603d0 0x7f6f9c1440c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:17.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.314+0000 7f6f9b59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f9c0b4c70 0x7f6f9c0b1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:17.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.314+0000 7f6f9b59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f9c145590 0x7f6f9c0b1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:17.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.314+0000 7f6f9b59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c144890 con 0x7f6f9c145590 2020-11-28T21:56:17.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.314+0000 7f6f9b59e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c144b30 con 0x7f6f9c0b4c70 2020-11-28T21:56:17.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.314+0000 7f6f9b59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c144dd0 con 0x7f6f9c0603d0 2020-11-28T21:56:17.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.314+0000 7f6f99d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f9c0b4c70 0x7f6f9c0b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:17.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.314+0000 7f6f99d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f9c0b4c70 0x7f6f9c0b1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46074/0 (socket says 172.21.15.162:46074) 2020-11-28T21:56:17.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.314+0000 7f6f99d9b700 1 -- 172.21.15.162:0/1001977609 learned_addr learned my addr 172.21.15.162:0/1001977609 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:17.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.314+0000 7f6f9ad9d700 1 --2- 172.21.15.162:0/1001977609 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f9c145590 0x7f6f9c0b1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:17.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.314+0000 7f6f9a59c700 1 --2- 172.21.15.162:0/1001977609 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6f9c0603d0 0x7f6f9c1440c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:17.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.314+0000 7f6f99d9b700 1 -- 172.21.15.162:0/1001977609 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6f9c0603d0 msgr2=0x7f6f9c1440c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:17.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.314+0000 7f6f99d9b700 1 --2- 172.21.15.162:0/1001977609 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6f9c0603d0 0x7f6f9c1440c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:17.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.314+0000 7f6f99d9b700 1 -- 172.21.15.162:0/1001977609 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f9c145590 msgr2=0x7f6f9c0b1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:17.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.314+0000 7f6f99d9b700 1 --2- 172.21.15.162:0/1001977609 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f9c145590 0x7f6f9c0b1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:17.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.314+0000 7f6f99d9b700 1 -- 172.21.15.162:0/1001977609 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f9c145070 con 0x7f6f9c0b4c70 2020-11-28T21:56:17.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.314+0000 7f6f99d9b700 1 --2- 172.21.15.162:0/1001977609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f9c0b4c70 0x7f6f9c0b1450 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 rx=0x7f6f90000b80 tx=0x7f6f90007c00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:17.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.314+0000 7f6f8b7fe700 1 -- 172.21.15.162:0/1001977609 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f9000e4a0 con 0x7f6f9c0b4c70 2020-11-28T21:56:17.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.314+0000 7f6f8b7fe700 1 -- 172.21.15.162:0/1001977609 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f9000e600 con 0x7f6f9c0b4c70 2020-11-28T21:56:17.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.314+0000 7f6f9b59e700 1 -- 172.21.15.162:0/1001977609 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f9c147660 con 0x7f6f9c0b4c70 2020-11-28T21:56:17.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.314+0000 7f6f8b7fe700 1 -- 172.21.15.162:0/1001977609 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f9000e960 con 0x7f6f9c0b4c70 2020-11-28T21:56:17.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.314+0000 7f6f9b59e700 1 -- 172.21.15.162:0/1001977609 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f9c0b2c50 con 0x7f6f9c0b4c70 2020-11-28T21:56:17.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.318+0000 7f6f8b7fe700 1 -- 172.21.15.162:0/1001977609 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6f90010030 con 0x7f6f9c0b4c70 2020-11-28T21:56:17.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.318+0000 7f6f9b59e700 1 -- 172.21.15.162:0/1001977609 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f9c145200 con 0x7f6f9c0b4c70 2020-11-28T21:56:17.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.318+0000 7f6f8b7fe700 1 --2- 172.21.15.162:0/1001977609 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6f78055330 0x7f6f780577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:17.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.318+0000 7f6f8b7fe700 1 -- 172.21.15.162:0/1001977609 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f6f9006db40 con 0x7f6f9c0b4c70 2020-11-28T21:56:17.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.318+0000 7f6f9a59c700 1 --2- 172.21.15.162:0/1001977609 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6f78055330 0x7f6f780577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:17.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.318+0000 7f6f9a59c700 1 --2- 172.21.15.162:0/1001977609 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6f78055330 0x7f6f780577f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7f6f8c018000 tx=0x7f6f8c015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:17.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.326+0000 7f6f8b7fe700 1 -- 172.21.15.162:0/1001977609 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6f90043ce0 con 0x7f6f9c0b4c70 2020-11-28T21:56:17.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.514+0000 7f6f9b59e700 1 -- 172.21.15.162:0/1001977609 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f9c057050 con 0x7f6f9c0b4c70 2020-11-28T21:56:17.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.514+0000 7f6f8b7fe700 1 -- 172.21.15.162:0/1001977609 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+2141 (secure 0 0 0) 0x7f6f90047300 con 0x7f6f9c0b4c70 2020-11-28T21:56:17.520 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:56:17.520 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":75,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6899,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.118:6833/2509028245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":2509028245},{"type":"v1","addr":"172.21.15.118:6833","nonce":2509028245}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":72},{"gid":8239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/2089791609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2089791609},{"type":"v1","addr":"172.21.15.162:6836","nonce":2089791609}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":71}],"filesystems":[{"mdsmap":{"epoch":75,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:56:01.362652+0000","modified":"2020-11-28T21:56:17.178769+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":149,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2020-11-28T21:56:17.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.518+0000 7f6f9b59e700 1 -- 172.21.15.162:0/1001977609 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6f78055330 msgr2=0x7f6f780577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:17.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.518+0000 7f6f9b59e700 1 --2- 172.21.15.162:0/1001977609 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6f78055330 0x7f6f780577f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7f6f8c018000 tx=0x7f6f8c015000).stop 2020-11-28T21:56:17.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.518+0000 7f6f9b59e700 1 -- 172.21.15.162:0/1001977609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f9c0b4c70 msgr2=0x7f6f9c0b1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:17.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.518+0000 7f6f9b59e700 1 --2- 172.21.15.162:0/1001977609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f9c0b4c70 0x7f6f9c0b1450 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 rx=0x7f6f90000b80 tx=0x7f6f90007c00).stop 2020-11-28T21:56:17.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.518+0000 7f6f9b59e700 1 -- 172.21.15.162:0/1001977609 shutdown_connections 2020-11-28T21:56:17.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.518+0000 7f6f9b59e700 1 --2- 172.21.15.162:0/1001977609 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6f78055330 0x7f6f780577f0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:17.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.518+0000 7f6f9b59e700 1 --2- 172.21.15.162:0/1001977609 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6f9c0603d0 0x7f6f9c1440c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:17.526 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.518+0000 7f6f9b59e700 1 --2- 172.21.15.162:0/1001977609 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f9c145590 0x7f6f9c0b1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:17.526 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.518+0000 7f6f9b59e700 1 --2- 172.21.15.162:0/1001977609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f9c0b4c70 0x7f6f9c0b1450 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:17.526 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.518+0000 7f6f9b59e700 1 -- 172.21.15.162:0/1001977609 >> 172.21.15.162:0/1001977609 conn(0x7f6f9c066a40 msgr2=0x7f6f9c142df0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:17.526 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.518+0000 7f6f9b59e700 1 -- 172.21.15.162:0/1001977609 shutdown_connections 2020-11-28T21:56:17.527 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.518+0000 7f6f9b59e700 1 -- 172.21.15.162:0/1001977609 wait complete. 2020-11-28T21:56:17.527 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 75 2020-11-28T21:56:17.534 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2020-11-28T21:56:17.660 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.654+0000 7f8c97fff700 1 Processor -- start 2020-11-28T21:56:17.660 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.654+0000 7f8c97fff700 1 -- start start 2020-11-28T21:56:17.662 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.654+0000 7f8c97fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8c9805bcb0 0x7f8c9805c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:17.663 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.654+0000 7f8c97fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8c980b2300 0x7f8c980add90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:17.663 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.654+0000 7f8c97fff700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c980ad6c0 con 0x7f8c980b0820 2020-11-28T21:56:17.663 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.654+0000 7f8c97fff700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c980ae520 con 0x7f8c980b2300 2020-11-28T21:56:17.663 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.654+0000 7f8c97fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9805c090 con 0x7f8c9805bcb0 2020-11-28T21:56:17.664 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.654+0000 7f8c967fc700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8c980b2300 0x7f8c980add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:17.664 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.654+0000 7f8c977fe700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f8c980b0820 0x7f8c98147750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:52928/0 (socket says 172.21.15.162:52928) 2020-11-28T21:56:17.664 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.654+0000 7f8c977fe700 1 -- 172.21.15.162:0/2989611374 learned_addr learned my addr 172.21.15.162:0/2989611374 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:56:17.665 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.654+0000 7f8c96ffd700 1 --2- 172.21.15.162:0/2989611374 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8c9805bcb0 0x7f8c9805c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:17.665 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.654+0000 7f8c967fc700 1 -- 172.21.15.162:0/2989611374 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8c9805bcb0 msgr2=0x7f8c9805c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:17.665 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.654+0000 7f8c967fc700 1 --2- 172.21.15.162:0/2989611374 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8c9805bcb0 0x7f8c9805c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:17.666 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.654+0000 7f8c967fc700 1 -- 172.21.15.162:0/2989611374 >> v1:172.21.15.162:6789/0 conn(0x7f8c980b0820 legacy=0x7f8c98147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:17.666 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.654+0000 7f8c967fc700 1 -- 172.21.15.162:0/2989611374 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c980ae800 con 0x7f8c980b2300 2020-11-28T21:56:17.667 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.654+0000 7f8c96ffd700 1 --2- 172.21.15.162:0/2989611374 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8c9805bcb0 0x7f8c9805c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:56:17.667 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.658+0000 7f8c967fc700 1 --2- 172.21.15.162:0/2989611374 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c980b2300 0x7f8c980add90 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 rx=0x7f8c80002f30 tx=0x7f8c80006dc0).ready entity=mon.1 client_cookie=b0f722451dfbf9a8 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:17.667 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.658+0000 7f8c95ffb700 1 -- 172.21.15.162:0/2989611374 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c800097f0 con 0x7f8c980b2300 2020-11-28T21:56:17.667 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.658+0000 7f8c95ffb700 1 -- 172.21.15.162:0/2989611374 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c80009950 con 0x7f8c980b2300 2020-11-28T21:56:17.668 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.658+0000 7f8c97fff700 1 -- 172.21.15.162:0/2989611374 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c980b2300 msgr2=0x7f8c980add90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:17.668 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.658+0000 7f8c95ffb700 1 -- 172.21.15.162:0/2989611374 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c80009cb0 con 0x7f8c980b2300 2020-11-28T21:56:17.671 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.658+0000 7f8c97fff700 1 --2- 172.21.15.162:0/2989611374 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c980b2300 0x7f8c980add90 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 rx=0x7f8c80002f30 tx=0x7f8c80006dc0).stop 2020-11-28T21:56:17.671 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.658+0000 7f8c97fff700 1 -- 172.21.15.162:0/2989611374 shutdown_connections 2020-11-28T21:56:17.671 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.658+0000 7f8c97fff700 1 --2- 172.21.15.162:0/2989611374 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8c9805bcb0 0x7f8c9805c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:17.672 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.658+0000 7f8c97fff700 1 --2- 172.21.15.162:0/2989611374 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c980b2300 0x7f8c980add90 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:17.672 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.658+0000 7f8c97fff700 1 -- 172.21.15.162:0/2989611374 >> 172.21.15.162:0/2989611374 conn(0x7f8c980b17a0 msgr2=0x7f8c980b1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:17.673 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.658+0000 7f8c97fff700 1 -- 172.21.15.162:0/2989611374 shutdown_connections 2020-11-28T21:56:17.673 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.658+0000 7f8c97fff700 1 -- 172.21.15.162:0/2989611374 wait complete. 2020-11-28T21:56:17.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.658+0000 7f8c97fff700 1 Processor -- start 2020-11-28T21:56:17.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.658+0000 7f8c97fff700 1 -- start start 2020-11-28T21:56:17.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.658+0000 7f8c97fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c9805bcb0 0x7f8c98067080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:17.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.658+0000 7f8c97fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8c980b0820 0x7f8c98158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:17.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.658+0000 7f8c97fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8c980b2300 0x7f8c98158660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:17.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.658+0000 7f8c97fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c98146940 con 0x7f8c980b0820 2020-11-28T21:56:17.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.658+0000 7f8c97fff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c980661c0 con 0x7f8c9805bcb0 2020-11-28T21:56:17.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.658+0000 7f8c97fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c98066460 con 0x7f8c980b2300 2020-11-28T21:56:17.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.658+0000 7f8c96ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c9805bcb0 0x7f8c98067080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:17.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.658+0000 7f8c96ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c9805bcb0 0x7f8c98067080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46086/0 (socket says 172.21.15.162:46086) 2020-11-28T21:56:17.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.658+0000 7f8c96ffd700 1 -- 172.21.15.162:0/4247712645 learned_addr learned my addr 172.21.15.162:0/4247712645 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:17.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.658+0000 7f8c967fc700 1 --2- 172.21.15.162:0/4247712645 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8c980b0820 0x7f8c98158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:17.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.658+0000 7f8c96ffd700 1 -- 172.21.15.162:0/4247712645 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8c980b2300 msgr2=0x7f8c98158660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:17.680 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.658+0000 7f8c977fe700 1 --2- 172.21.15.162:0/4247712645 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8c980b2300 0x7f8c98158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:17.680 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.658+0000 7f8c96ffd700 1 --2- 172.21.15.162:0/4247712645 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8c980b2300 0x7f8c98158660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:17.680 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.658+0000 7f8c96ffd700 1 -- 172.21.15.162:0/4247712645 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8c980b0820 msgr2=0x7f8c98158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:17.680 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.658+0000 7f8c96ffd700 1 --2- 172.21.15.162:0/4247712645 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8c980b0820 0x7f8c98158170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:17.680 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.658+0000 7f8c96ffd700 1 -- 172.21.15.162:0/4247712645 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c98066700 con 0x7f8c9805bcb0 2020-11-28T21:56:17.681 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.658+0000 7f8c967fc700 1 --2- 172.21.15.162:0/4247712645 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8c980b0820 0x7f8c98158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:56:17.681 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.658+0000 7f8c977fe700 1 --2- 172.21.15.162:0/4247712645 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8c980b2300 0x7f8c98158660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:56:17.681 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.658+0000 7f8c96ffd700 1 --2- 172.21.15.162:0/4247712645 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c9805bcb0 0x7f8c98067080 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 rx=0x7f8c88007000 tx=0x7f8c88008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:17.681 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.662+0000 7f8c8ffff700 1 -- 172.21.15.162:0/4247712645 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c8800ce20 con 0x7f8c9805bcb0 2020-11-28T21:56:17.681 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.662+0000 7f8c8ffff700 1 -- 172.21.15.162:0/4247712645 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c8800ab70 con 0x7f8c9805bcb0 2020-11-28T21:56:17.682 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.662+0000 7f8c8ffff700 1 -- 172.21.15.162:0/4247712645 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c88011410 con 0x7f8c9805bcb0 2020-11-28T21:56:17.682 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.662+0000 7f8c97fff700 1 -- 172.21.15.162:0/4247712645 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c980669a0 con 0x7f8c9805bcb0 2020-11-28T21:56:17.682 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.662+0000 7f8c97fff700 1 -- 172.21.15.162:0/4247712645 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c98063880 con 0x7f8c9805bcb0 2020-11-28T21:56:17.682 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.662+0000 7f8c8ffff700 1 -- 172.21.15.162:0/4247712645 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8c88011570 con 0x7f8c9805bcb0 2020-11-28T21:56:17.682 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.662+0000 7f8c97fff700 1 -- 172.21.15.162:0/4247712645 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c98066b30 con 0x7f8c9805bcb0 2020-11-28T21:56:17.683 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.662+0000 7f8c8ffff700 1 --2- 172.21.15.162:0/4247712645 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8c7c055470 0x7f8c7c057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:17.683 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.662+0000 7f8c967fc700 1 --2- 172.21.15.162:0/4247712645 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8c7c055470 0x7f8c7c057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:17.683 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.662+0000 7f8c8ffff700 1 -- 172.21.15.162:0/4247712645 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f8c8806e550 con 0x7f8c9805bcb0 2020-11-28T21:56:17.683 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.662+0000 7f8c967fc700 1 --2- 172.21.15.162:0/4247712645 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8c7c055470 0x7f8c7c057930 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7f8c80016000 tx=0x7f8c80015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:17.683 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.670+0000 7f8c8ffff700 1 -- 172.21.15.162:0/4247712645 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8c880447a0 con 0x7f8c9805bcb0 2020-11-28T21:56:17.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:17.854+0000 7f8c97fff700 1 -- 172.21.15.162:0/4247712645 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8c98057050 con 0x7f8c9805bcb0 2020-11-28T21:56:18.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.182+0000 7f8c8ffff700 1 -- 172.21.15.162:0/4247712645 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v76) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8c88047dc0 con 0x7f8c9805bcb0 2020-11-28T21:56:18.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.186+0000 7f8c97fff700 1 -- 172.21.15.162:0/4247712645 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8c7c055470 msgr2=0x7f8c7c057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:18.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.186+0000 7f8c97fff700 1 --2- 172.21.15.162:0/4247712645 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8c7c055470 0x7f8c7c057930 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7f8c80016000 tx=0x7f8c80015000).stop 2020-11-28T21:56:18.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.186+0000 7f8c97fff700 1 -- 172.21.15.162:0/4247712645 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c9805bcb0 msgr2=0x7f8c98067080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:18.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.186+0000 7f8c97fff700 1 --2- 172.21.15.162:0/4247712645 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c9805bcb0 0x7f8c98067080 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 rx=0x7f8c88007000 tx=0x7f8c88008040).stop 2020-11-28T21:56:18.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.186+0000 7f8c97fff700 1 -- 172.21.15.162:0/4247712645 shutdown_connections 2020-11-28T21:56:18.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.186+0000 7f8c97fff700 1 --2- 172.21.15.162:0/4247712645 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8c7c055470 0x7f8c7c057930 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:18.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.186+0000 7f8c97fff700 1 --2- 172.21.15.162:0/4247712645 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8c980b2300 0x7f8c98158660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:18.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.186+0000 7f8c97fff700 1 --2- 172.21.15.162:0/4247712645 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c9805bcb0 0x7f8c98067080 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:18.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.186+0000 7f8c97fff700 1 --2- 172.21.15.162:0/4247712645 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8c980b0820 0x7f8c98158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:18.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.186+0000 7f8c97fff700 1 -- 172.21.15.162:0/4247712645 >> 172.21.15.162:0/4247712645 conn(0x7f8c980b17a0 msgr2=0x7f8c980ac4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:18.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.186+0000 7f8c97fff700 1 -- 172.21.15.162:0/4247712645 shutdown_connections 2020-11-28T21:56:18.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.186+0000 7f8c97fff700 1 -- 172.21.15.162:0/4247712645 wait complete. 2020-11-28T21:56:18.210 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-11-28T21:56:18.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.326+0000 7f01c89f5700 1 Processor -- start 2020-11-28T21:56:18.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.326+0000 7f01c89f5700 1 -- start start 2020-11-28T21:56:18.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.326+0000 7f01c89f5700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01c40b3ab0 0x7f01c405c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:18.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.326+0000 7f01c89f5700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f01c40b30e0 0x7f01c413fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:18.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.326+0000 7f01c89f5700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01c405baf0 con 0x7f01c40b2d00 2020-11-28T21:56:18.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.326+0000 7f01c89f5700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01c40b0e20 con 0x7f01c40b30e0 2020-11-28T21:56:18.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.326+0000 7f01c89f5700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01c405bd90 con 0x7f01c40b3ab0 2020-11-28T21:56:18.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.326+0000 7f01c3fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f01c40b30e0 0x7f01c413fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:18.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.326+0000 7f01c3fff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f01c40b30e0 0x7f01c413fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46100/0 (socket says 172.21.15.162:46100) 2020-11-28T21:56:18.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.326+0000 7f01c37fe700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f01c40b2d00 0x7f01c4142ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:52940/0 (socket says 172.21.15.162:52940) 2020-11-28T21:56:18.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.326+0000 7f01c3fff700 1 -- 172.21.15.162:0/793766586 learned_addr learned my addr 172.21.15.162:0/793766586 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:18.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.326+0000 7f01c2ffd700 1 --2- 172.21.15.162:0/793766586 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01c40b3ab0 0x7f01c405c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:18.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.326+0000 7f01c3fff700 1 -- 172.21.15.162:0/793766586 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01c40b3ab0 msgr2=0x7f01c405c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:18.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.326+0000 7f01c3fff700 1 --2- 172.21.15.162:0/793766586 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01c40b3ab0 0x7f01c405c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:18.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.330+0000 7f01c3fff700 1 -- 172.21.15.162:0/793766586 >> v1:172.21.15.162:6789/0 conn(0x7f01c40b2d00 legacy=0x7f01c4142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:18.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.330+0000 7f01c3fff700 1 -- 172.21.15.162:0/793766586 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c405c030 con 0x7f01c40b30e0 2020-11-28T21:56:18.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.330+0000 7f01c3fff700 1 --2- 172.21.15.162:0/793766586 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01c40b30e0 0x7f01c413fa70 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 rx=0x7f01bc0032b0 tx=0x7f01bc009270).ready entity=mon.1 client_cookie=5784d809c6f4f143 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:18.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.330+0000 7f01c27fc700 1 -- 172.21.15.162:0/793766586 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01bc00bc00 con 0x7f01c40b30e0 2020-11-28T21:56:18.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.330+0000 7f01c27fc700 1 -- 172.21.15.162:0/793766586 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01bc00bd60 con 0x7f01c40b30e0 2020-11-28T21:56:18.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.330+0000 7f01c27fc700 1 -- 172.21.15.162:0/793766586 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01bc011400 con 0x7f01c40b30e0 2020-11-28T21:56:18.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.330+0000 7f01c89f5700 1 -- 172.21.15.162:0/793766586 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01c40b30e0 msgr2=0x7f01c413fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:18.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.330+0000 7f01c89f5700 1 --2- 172.21.15.162:0/793766586 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01c40b30e0 0x7f01c413fa70 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 rx=0x7f01bc0032b0 tx=0x7f01bc009270).stop 2020-11-28T21:56:18.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.330+0000 7f01c89f5700 1 -- 172.21.15.162:0/793766586 shutdown_connections 2020-11-28T21:56:18.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.330+0000 7f01c89f5700 1 --2- 172.21.15.162:0/793766586 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01c40b30e0 0x7f01c413fa70 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:18.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.330+0000 7f01c89f5700 1 --2- 172.21.15.162:0/793766586 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01c40b3ab0 0x7f01c405c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:18.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.330+0000 7f01c89f5700 1 -- 172.21.15.162:0/793766586 >> 172.21.15.162:0/793766586 conn(0x7f01c40604c0 msgr2=0x7f01c40608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:18.340 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.330+0000 7f01c89f5700 1 -- 172.21.15.162:0/793766586 shutdown_connections 2020-11-28T21:56:18.341 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.330+0000 7f01c89f5700 1 -- 172.21.15.162:0/793766586 wait complete. 2020-11-28T21:56:18.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.330+0000 7f01c89f5700 1 Processor -- start 2020-11-28T21:56:18.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.330+0000 7f01c89f5700 1 -- start start 2020-11-28T21:56:18.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.334+0000 7f01c89f5700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f01c40b2d00 0x7f01c40b5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:18.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.334+0000 7f01c89f5700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01c40b30e0 0x7f01c41463c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:18.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.334+0000 7f01c89f5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01c40b3ab0 0x7f01c4154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:18.344 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.334+0000 7f01c89f5700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c41447d0 con 0x7f01c40b2d00 2020-11-28T21:56:18.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.334+0000 7f01c89f5700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c41449f0 con 0x7f01c40b3ab0 2020-11-28T21:56:18.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.334+0000 7f01c89f5700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01c40b4da0 con 0x7f01c40b30e0 2020-11-28T21:56:18.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.334+0000 7f01c3fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01c40b3ab0 0x7f01c4154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:18.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.334+0000 7f01c3fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01c40b3ab0 0x7f01c4154de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46106/0 (socket says 172.21.15.162:46106) 2020-11-28T21:56:18.345 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.334+0000 7f01c3fff700 1 -- 172.21.15.162:0/2494804032 learned_addr learned my addr 172.21.15.162:0/2494804032 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:18.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.334+0000 7f01c37fe700 1 --2- 172.21.15.162:0/2494804032 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f01c40b2d00 0x7f01c40b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:18.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.334+0000 7f01c2ffd700 1 --2- 172.21.15.162:0/2494804032 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01c40b30e0 0x7f01c41463c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:18.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.334+0000 7f01c3fff700 1 -- 172.21.15.162:0/2494804032 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01c40b30e0 msgr2=0x7f01c41463c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:18.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.334+0000 7f01c3fff700 1 --2- 172.21.15.162:0/2494804032 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01c40b30e0 0x7f01c41463c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:18.346 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.334+0000 7f01c3fff700 1 -- 172.21.15.162:0/2494804032 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f01c40b2d00 msgr2=0x7f01c40b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:18.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.334+0000 7f01c3fff700 1 --2- 172.21.15.162:0/2494804032 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f01c40b2d00 0x7f01c40b5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:18.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.334+0000 7f01c3fff700 1 -- 172.21.15.162:0/2494804032 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c40b6000 con 0x7f01c40b3ab0 2020-11-28T21:56:18.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.334+0000 7f01c3fff700 1 --2- 172.21.15.162:0/2494804032 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01c40b3ab0 0x7f01c4154de0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 rx=0x7f01c4144910 tx=0x7f01bc016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:18.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.334+0000 7f01c0ff9700 1 -- 172.21.15.162:0/2494804032 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01bc01e610 con 0x7f01c40b3ab0 2020-11-28T21:56:18.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.334+0000 7f01c0ff9700 1 -- 172.21.15.162:0/2494804032 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01bc01e770 con 0x7f01c40b3ab0 2020-11-28T21:56:18.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.334+0000 7f01c89f5700 1 -- 172.21.15.162:0/2494804032 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01c40b5d60 con 0x7f01c40b3ab0 2020-11-28T21:56:18.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.334+0000 7f01c0ff9700 1 -- 172.21.15.162:0/2494804032 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01bc01ead0 con 0x7f01c40b3ab0 2020-11-28T21:56:18.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.334+0000 7f01c89f5700 1 -- 172.21.15.162:0/2494804032 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01c4144c50 con 0x7f01c40b3ab0 2020-11-28T21:56:18.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.334+0000 7f01c89f5700 1 -- 172.21.15.162:0/2494804032 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01c4057050 con 0x7f01c40b3ab0 2020-11-28T21:56:18.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.338+0000 7f01c0ff9700 1 -- 172.21.15.162:0/2494804032 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f01bc00f070 con 0x7f01c40b3ab0 2020-11-28T21:56:18.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.338+0000 7f01c0ff9700 1 --2- 172.21.15.162:0/2494804032 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f01a40553b0 0x7f01a4057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:18.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.338+0000 7f01c0ff9700 1 -- 172.21.15.162:0/2494804032 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f01bc06f830 con 0x7f01c40b3ab0 2020-11-28T21:56:18.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.338+0000 7f01c37fe700 1 --2- 172.21.15.162:0/2494804032 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f01a40553b0 0x7f01a4057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:18.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.338+0000 7f01c37fe700 1 --2- 172.21.15.162:0/2494804032 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f01a40553b0 0x7f01a4057870 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7f01b800d000 tx=0x7f01b8007b10).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:18.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.342+0000 7f01c0ff9700 1 -- 172.21.15.162:0/2494804032 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f01bc049380 con 0x7f01c40b3ab0 2020-11-28T21:56:18.490 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:18.486+0000 7f01c89f5700 1 -- 172.21.15.162:0/2494804032 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f01c405d250 con 0x7f01c40b3ab0 2020-11-28T21:56:19.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.250+0000 7f01c0ff9700 1 -- 172.21.15.162:0/2494804032 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v150) v1 ==== 185+0+0 (secure 0 0 0) 0x7f01bc0150e0 con 0x7f01c40b3ab0 2020-11-28T21:56:19.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.250+0000 7f01c89f5700 1 -- 172.21.15.162:0/2494804032 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f01a40553b0 msgr2=0x7f01a4057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:19.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.250+0000 7f01c89f5700 1 --2- 172.21.15.162:0/2494804032 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f01a40553b0 0x7f01a4057870 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7f01b800d000 tx=0x7f01b8007b10).stop 2020-11-28T21:56:19.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.250+0000 7f01c89f5700 1 -- 172.21.15.162:0/2494804032 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01c40b3ab0 msgr2=0x7f01c4154de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:19.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.250+0000 7f01c89f5700 1 --2- 172.21.15.162:0/2494804032 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01c40b3ab0 0x7f01c4154de0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 rx=0x7f01c4144910 tx=0x7f01bc016000).stop 2020-11-28T21:56:19.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.250+0000 7f01c89f5700 1 -- 172.21.15.162:0/2494804032 shutdown_connections 2020-11-28T21:56:19.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.250+0000 7f01c89f5700 1 --2- 172.21.15.162:0/2494804032 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f01a40553b0 0x7f01a4057870 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:19.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.254+0000 7f01c89f5700 1 --2- 172.21.15.162:0/2494804032 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01c40b3ab0 0x7f01c4154de0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:19.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.254+0000 7f01c89f5700 1 --2- 172.21.15.162:0/2494804032 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f01c40b2d00 0x7f01c40b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:19.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.254+0000 7f01c89f5700 1 --2- 172.21.15.162:0/2494804032 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01c40b30e0 0x7f01c41463c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:19.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.254+0000 7f01c89f5700 1 -- 172.21.15.162:0/2494804032 >> 172.21.15.162:0/2494804032 conn(0x7f01c40604c0 msgr2=0x7f01c413ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:19.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.254+0000 7f01c89f5700 1 -- 172.21.15.162:0/2494804032 shutdown_connections 2020-11-28T21:56:19.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.254+0000 7f01c89f5700 1 -- 172.21.15.162:0/2494804032 wait complete. 2020-11-28T21:56:19.259 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_metadata' removed 2020-11-28T21:56:19.272 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-11-28T21:56:19.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.386+0000 7f8da359e700 1 Processor -- start 2020-11-28T21:56:19.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.386+0000 7f8da359e700 1 -- start start 2020-11-28T21:56:19.394 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.386+0000 7f8da359e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f8da40b21f0 0x7f8da413fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:19.394 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.386+0000 7f8da359e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8da41402a0 0x7f8da4060450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:19.394 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.386+0000 7f8da359e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8da4060940 0x7f8da4146030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:19.394 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.386+0000 7f8da359e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8da40af240 con 0x7f8da40b21f0 2020-11-28T21:56:19.395 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.386+0000 7f8da359e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8da40aff20 con 0x7f8da4060940 2020-11-28T21:56:19.395 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.386+0000 7f8da359e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8da40b25d0 con 0x7f8da41402a0 2020-11-28T21:56:19.395 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.386+0000 7f8da2d9d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8da4060940 0x7f8da4146030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:19.395 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.386+0000 7f8da2d9d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8da4060940 0x7f8da4146030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46114/0 (socket says 172.21.15.162:46114) 2020-11-28T21:56:19.395 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.386+0000 7f8da2d9d700 1 -- 172.21.15.162:0/4084680611 learned_addr learned my addr 172.21.15.162:0/4084680611 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:19.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.386+0000 7f8da1d9b700 1 --2- 172.21.15.162:0/4084680611 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8da41402a0 0x7f8da4060450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:19.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.386+0000 7f8da259c700 1 --2- 172.21.15.162:0/4084680611 >> v2:172.21.15.118:3300/0 conn(0x7f8da40b21f0 0x7f8da413fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:19.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.386+0000 7f8da2d9d700 1 -- 172.21.15.162:0/4084680611 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8da41402a0 msgr2=0x7f8da4060450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:19.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.386+0000 7f8da2d9d700 1 --2- 172.21.15.162:0/4084680611 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8da41402a0 0x7f8da4060450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:19.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.386+0000 7f8da2d9d700 1 -- 172.21.15.162:0/4084680611 >> v2:172.21.15.118:3300/0 conn(0x7f8da40b21f0 msgr2=0x7f8da413fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:19.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.386+0000 7f8da2d9d700 1 --2- 172.21.15.162:0/4084680611 >> v2:172.21.15.118:3300/0 conn(0x7f8da40b21f0 0x7f8da413fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:19.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.386+0000 7f8da2d9d700 1 -- 172.21.15.162:0/4084680611 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da405fff0 con 0x7f8da4060940 2020-11-28T21:56:19.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.390+0000 7f8da2d9d700 1 --2- 172.21.15.162:0/4084680611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8da4060940 0x7f8da4146030 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 rx=0x7f8d9c0032b0 tx=0x7f8d9c009270).ready entity=mon.1 client_cookie=2477e61265645fe8 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:19.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.390+0000 7f8da159a700 1 -- 172.21.15.162:0/4084680611 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d9c009e60 con 0x7f8da4060940 2020-11-28T21:56:19.398 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.390+0000 7f8da159a700 1 -- 172.21.15.162:0/4084680611 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d9c00bb50 con 0x7f8da4060940 2020-11-28T21:56:19.398 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.390+0000 7f8da159a700 1 -- 172.21.15.162:0/4084680611 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d9c00bcc0 con 0x7f8da4060940 2020-11-28T21:56:19.398 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.390+0000 7f8da359e700 1 -- 172.21.15.162:0/4084680611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8da4060940 msgr2=0x7f8da4146030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:19.398 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.390+0000 7f8da359e700 1 --2- 172.21.15.162:0/4084680611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8da4060940 0x7f8da4146030 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 rx=0x7f8d9c0032b0 tx=0x7f8d9c009270).stop 2020-11-28T21:56:19.401 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.390+0000 7f8da359e700 1 -- 172.21.15.162:0/4084680611 shutdown_connections 2020-11-28T21:56:19.402 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.390+0000 7f8da359e700 1 --2- 172.21.15.162:0/4084680611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8da4060940 0x7f8da4146030 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:19.402 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.390+0000 7f8da359e700 1 --2- 172.21.15.162:0/4084680611 >> v2:172.21.15.118:3300/0 conn(0x7f8da40b21f0 0x7f8da413fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:19.402 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.390+0000 7f8da359e700 1 --2- 172.21.15.162:0/4084680611 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8da41402a0 0x7f8da4060450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:19.402 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.390+0000 7f8da359e700 1 -- 172.21.15.162:0/4084680611 >> 172.21.15.162:0/4084680611 conn(0x7f8da40b57d0 msgr2=0x7f8da40b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:19.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.390+0000 7f8da359e700 1 -- 172.21.15.162:0/4084680611 shutdown_connections 2020-11-28T21:56:19.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.390+0000 7f8da359e700 1 -- 172.21.15.162:0/4084680611 wait complete. 2020-11-28T21:56:19.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.390+0000 7f8da359e700 1 Processor -- start 2020-11-28T21:56:19.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.390+0000 7f8da359e700 1 -- start start 2020-11-28T21:56:19.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.394+0000 7f8da359e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8da4060940 0x7f8da4140c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:19.404 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.394+0000 7f8da359e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8da40b21f0 0x7f8da40ab7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:19.404 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.394+0000 7f8da359e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8da41402a0 0x7f8da4154ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:19.404 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.394+0000 7f8da359e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da4146c30 con 0x7f8da4060940 2020-11-28T21:56:19.404 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.394+0000 7f8da359e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da40aa3a0 con 0x7f8da40b21f0 2020-11-28T21:56:19.404 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.394+0000 7f8da359e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8da41460f0 con 0x7f8da41402a0 2020-11-28T21:56:19.405 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.394+0000 7f8da1d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8da40b21f0 0x7f8da40ab7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:19.405 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.394+0000 7f8da1d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8da40b21f0 0x7f8da40ab7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46118/0 (socket says 172.21.15.162:46118) 2020-11-28T21:56:19.405 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.394+0000 7f8da1d9b700 1 -- 172.21.15.162:0/2348525939 learned_addr learned my addr 172.21.15.162:0/2348525939 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:19.405 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.394+0000 7f8da259c700 1 --2- 172.21.15.162:0/2348525939 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8da4060940 0x7f8da4140c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:19.406 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.394+0000 7f8da2d9d700 1 --2- 172.21.15.162:0/2348525939 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8da41402a0 0x7f8da4154ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:19.406 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.394+0000 7f8da1d9b700 1 -- 172.21.15.162:0/2348525939 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8da41402a0 msgr2=0x7f8da4154ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:19.406 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.394+0000 7f8da1d9b700 1 --2- 172.21.15.162:0/2348525939 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8da41402a0 0x7f8da4154ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:19.406 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.394+0000 7f8da1d9b700 1 -- 172.21.15.162:0/2348525939 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8da4060940 msgr2=0x7f8da4140c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:19.406 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.394+0000 7f8da1d9b700 1 --2- 172.21.15.162:0/2348525939 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8da4060940 0x7f8da4140c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:19.407 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.394+0000 7f8da1d9b700 1 -- 172.21.15.162:0/2348525939 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da4146eb0 con 0x7f8da40b21f0 2020-11-28T21:56:19.407 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.394+0000 7f8da1d9b700 1 --2- 172.21.15.162:0/2348525939 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8da40b21f0 0x7f8da40ab7c0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 rx=0x7f8d9400c000 tx=0x7f8d94006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:19.407 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.394+0000 7f8d9b7fe700 1 -- 172.21.15.162:0/2348525939 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d9400ca10 con 0x7f8da40b21f0 2020-11-28T21:56:19.407 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.394+0000 7f8d9b7fe700 1 -- 172.21.15.162:0/2348525939 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d9400cb70 con 0x7f8da40b21f0 2020-11-28T21:56:19.407 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.394+0000 7f8da359e700 1 -- 172.21.15.162:0/2348525939 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8da4146990 con 0x7f8da40b21f0 2020-11-28T21:56:19.408 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.394+0000 7f8d9b7fe700 1 -- 172.21.15.162:0/2348525939 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d940123f0 con 0x7f8da40b21f0 2020-11-28T21:56:19.408 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.394+0000 7f8da359e700 1 -- 172.21.15.162:0/2348525939 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8da40aa090 con 0x7f8da40b21f0 2020-11-28T21:56:19.408 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.394+0000 7f8da359e700 1 -- 172.21.15.162:0/2348525939 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8da4057050 con 0x7f8da40b21f0 2020-11-28T21:56:19.408 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.394+0000 7f8d9b7fe700 1 -- 172.21.15.162:0/2348525939 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8d94012550 con 0x7f8da40b21f0 2020-11-28T21:56:19.409 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.398+0000 7f8d9b7fe700 1 --2- 172.21.15.162:0/2348525939 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8d88055440 0x7f8d88057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:19.409 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.398+0000 7f8d9b7fe700 1 -- 172.21.15.162:0/2348525939 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5721+0+0 (secure 0 0 0) 0x7f8d9406efc0 con 0x7f8da40b21f0 2020-11-28T21:56:19.409 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.398+0000 7f8da259c700 1 --2- 172.21.15.162:0/2348525939 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8d88055440 0x7f8d88057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:19.410 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.398+0000 7f8da259c700 1 --2- 172.21.15.162:0/2348525939 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8d88055440 0x7f8d88057900 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f8d8c00b000 tx=0x7f8d8c006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:19.410 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.402+0000 7f8d9b7fe700 1 -- 172.21.15.162:0/2348525939 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8d940453c0 con 0x7f8da40b21f0 2020-11-28T21:56:19.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:19.542+0000 7f8da359e700 1 -- 172.21.15.162:0/2348525939 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8da405d250 con 0x7f8da40b21f0 2020-11-28T21:56:20.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.258+0000 7f8d9b7fe700 1 -- 172.21.15.162:0/2348525939 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v151) v1 ==== 173+0+0 (secure 0 0 0) 0x7f8d940489e0 con 0x7f8da40b21f0 2020-11-28T21:56:20.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.262+0000 7f8da359e700 1 -- 172.21.15.162:0/2348525939 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8d88055440 msgr2=0x7f8d88057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:20.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.262+0000 7f8da359e700 1 --2- 172.21.15.162:0/2348525939 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8d88055440 0x7f8d88057900 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f8d8c00b000 tx=0x7f8d8c006dc0).stop 2020-11-28T21:56:20.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.262+0000 7f8da359e700 1 -- 172.21.15.162:0/2348525939 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8da40b21f0 msgr2=0x7f8da40ab7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:20.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.262+0000 7f8da359e700 1 --2- 172.21.15.162:0/2348525939 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8da40b21f0 0x7f8da40ab7c0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 rx=0x7f8d9400c000 tx=0x7f8d94006dc0).stop 2020-11-28T21:56:20.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.262+0000 7f8da359e700 1 -- 172.21.15.162:0/2348525939 shutdown_connections 2020-11-28T21:56:20.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.262+0000 7f8da359e700 1 --2- 172.21.15.162:0/2348525939 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8d88055440 0x7f8d88057900 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:20.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.262+0000 7f8da359e700 1 --2- 172.21.15.162:0/2348525939 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8da41402a0 0x7f8da4154ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:20.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.262+0000 7f8da359e700 1 --2- 172.21.15.162:0/2348525939 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8da4060940 0x7f8da4140c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:20.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.262+0000 7f8da359e700 1 --2- 172.21.15.162:0/2348525939 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8da40b21f0 0x7f8da40ab7c0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:20.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.262+0000 7f8da359e700 1 -- 172.21.15.162:0/2348525939 >> 172.21.15.162:0/2348525939 conn(0x7f8da40b57d0 msgr2=0x7f8da40b3010 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:20.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.262+0000 7f8da359e700 1 -- 172.21.15.162:0/2348525939 shutdown_connections 2020-11-28T21:56:20.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.262+0000 7f8da359e700 1 -- 172.21.15.162:0/2348525939 wait complete. 2020-11-28T21:56:20.272 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_data' removed 2020-11-28T21:56:20.283 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-28T21:56:20.284 INFO:tasks.ceph.mds.a:Stopping old one... 2020-11-28T21:56:20.284 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-11-28T21:56:20.284 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T21:56:20.284 INFO:tasks.ceph.mds.c:Restarting daemon 2020-11-28T21:56:20.285 INFO:tasks.ceph.mds.c:Stopping old one... 2020-11-28T21:56:20.285 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2020-11-28T21:56:20.285 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T21:56:20.286 INFO:tasks.ceph.mds.b:Restarting daemon 2020-11-28T21:56:20.286 INFO:tasks.ceph.mds.b:Stopping old one... 2020-11-28T21:56:20.286 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-11-28T21:56:20.287 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T21:56:20.298 INFO:tasks.ceph.mds.c:Stopped 2020-11-28T21:56:20.299 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2020-11-28T21:56:20.300 INFO:tasks.ceph.mds.a:Stopped 2020-11-28T21:56:20.301 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-28T21:56:20.302 INFO:tasks.ceph.mds.b:Stopped 2020-11-28T21:56:20.302 INFO:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-11-28T21:56:20.345 INFO:tasks.ceph.mds.b:Started 2020-11-28T21:56:20.346 DEBUG:teuthology.parallel:result is None 2020-11-28T21:56:20.347 INFO:tasks.ceph.mds.c:Started 2020-11-28T21:56:20.348 DEBUG:teuthology.parallel:result is None 2020-11-28T21:56:20.349 INFO:tasks.ceph.mds.a:Started 2020-11-28T21:56:20.350 DEBUG:teuthology.parallel:result is None 2020-11-28T21:56:20.351 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist clear 2020-11-28T21:56:20.418 INFO:tasks.ceph.mds.a.smithi162.stdout:starting mds.a at 2020-11-28T21:56:20.419 INFO:tasks.ceph.mds.c.smithi162.stdout:starting mds.c at 2020-11-28T21:56:20.427 INFO:tasks.ceph.mds.b.smithi118.stdout:starting mds.b at 2020-11-28T21:56:20.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.458+0000 7f8fe41c6700 1 Processor -- start 2020-11-28T21:56:20.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.458+0000 7f8fe41c6700 1 -- start start 2020-11-28T21:56:20.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.458+0000 7f8fe41c6700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8fdc0ae710 0x7f8fdc0aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:20.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.458+0000 7f8fe41c6700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fdc0b0730 con 0x7f8fdc0af8f0 2020-11-28T21:56:20.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.458+0000 7f8fe41c6700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fdc0b1990 con 0x7f8fdc0b01f0 2020-11-28T21:56:20.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.458+0000 7f8fe41c6700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fdc0b6680 con 0x7f8fdc0ae710 2020-11-28T21:56:20.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.458+0000 7f8fe1f62700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8fdc0ae710 0x7f8fdc0aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:20.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.458+0000 7f8fe1f62700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8fdc0ae710 0x7f8fdc0aeaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46154/0 (socket says 172.21.15.162:46154) 2020-11-28T21:56:20.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.458+0000 7f8fe1f62700 1 -- 172.21.15.162:0/2543216857 learned_addr learned my addr 172.21.15.162:0/2543216857 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:20.469 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.458+0000 7f8fe1f62700 1 -- 172.21.15.162:0/2543216857 >> v1:172.21.15.162:6789/0 conn(0x7f8fdc0b01f0 legacy=0x7f8fdc13c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:20.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.458+0000 7f8fe1f62700 1 -- 172.21.15.162:0/2543216857 >> v1:172.21.15.118:6789/0 conn(0x7f8fdc0af8f0 legacy=0x7f8fdc0acd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:20.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.458+0000 7f8fe1f62700 1 -- 172.21.15.162:0/2543216857 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fdc0b0960 con 0x7f8fdc0ae710 2020-11-28T21:56:20.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.458+0000 7f8fe1f62700 1 --2- 172.21.15.162:0/2543216857 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fdc0ae710 0x7f8fdc0aeaf0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 rx=0x7f8fdc0b6360 tx=0x7f8fd8006e80).ready entity=mon.1 client_cookie=f8e4bb007dd32115 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:20.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.462+0000 7f8fe0f60700 1 -- 172.21.15.162:0/2543216857 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fd80097b0 con 0x7f8fdc0ae710 2020-11-28T21:56:20.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.462+0000 7f8fe0f60700 1 -- 172.21.15.162:0/2543216857 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fd8009910 con 0x7f8fdc0ae710 2020-11-28T21:56:20.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.462+0000 7f8fe0f60700 1 -- 172.21.15.162:0/2543216857 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fd8009a80 con 0x7f8fdc0ae710 2020-11-28T21:56:20.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.462+0000 7f8fe41c6700 1 -- 172.21.15.162:0/2543216857 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fdc0ae710 msgr2=0x7f8fdc0aeaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:20.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.462+0000 7f8fe41c6700 1 --2- 172.21.15.162:0/2543216857 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fdc0ae710 0x7f8fdc0aeaf0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 rx=0x7f8fdc0b6360 tx=0x7f8fd8006e80).stop 2020-11-28T21:56:20.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.462+0000 7f8fe41c6700 1 -- 172.21.15.162:0/2543216857 shutdown_connections 2020-11-28T21:56:20.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.462+0000 7f8fe41c6700 1 --2- 172.21.15.162:0/2543216857 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fdc0ae710 0x7f8fdc0aeaf0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:20.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.462+0000 7f8fe41c6700 1 -- 172.21.15.162:0/2543216857 >> 172.21.15.162:0/2543216857 conn(0x7f8fdc0b56f0 msgr2=0x7f8fdc0b5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:20.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.462+0000 7f8fe41c6700 1 -- 172.21.15.162:0/2543216857 shutdown_connections 2020-11-28T21:56:20.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.462+0000 7f8fe41c6700 1 -- 172.21.15.162:0/2543216857 wait complete. 2020-11-28T21:56:20.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.462+0000 7f8fe41c6700 1 Processor -- start 2020-11-28T21:56:20.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.462+0000 7f8fe41c6700 1 -- start start 2020-11-28T21:56:20.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.466+0000 7f8fe41c6700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fdc0ae710 0x7f8fdc0abfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:20.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.466+0000 7f8fe41c6700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fdc0af8f0 0x7f8fdc0640d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:20.479 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.466+0000 7f8fe41c6700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8fdc0b01f0 0x7f8fdc152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:20.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.466+0000 7f8fe41c6700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc140410 con 0x7f8fdc0ae710 2020-11-28T21:56:20.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.466+0000 7f8fe41c6700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc150330 con 0x7f8fdc0af8f0 2020-11-28T21:56:20.480 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.466+0000 7f8fe41c6700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc140640 con 0x7f8fdc0b01f0 2020-11-28T21:56:20.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.466+0000 7f8fe1761700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fdc0af8f0 0x7f8fdc0640d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:20.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.466+0000 7f8fe1761700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fdc0af8f0 0x7f8fdc0640d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46162/0 (socket says 172.21.15.162:46162) 2020-11-28T21:56:20.481 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.466+0000 7f8fe1761700 1 -- 172.21.15.162:0/1193226940 learned_addr learned my addr 172.21.15.162:0/1193226940 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:20.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.466+0000 7f8fe1f62700 1 --2- 172.21.15.162:0/1193226940 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fdc0ae710 0x7f8fdc0abfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:20.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.466+0000 7f8fe2763700 1 --2- 172.21.15.162:0/1193226940 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8fdc0b01f0 0x7f8fdc152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:20.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.466+0000 7f8fe1761700 1 -- 172.21.15.162:0/1193226940 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8fdc0b01f0 msgr2=0x7f8fdc152de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:20.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.466+0000 7f8fe1761700 1 --2- 172.21.15.162:0/1193226940 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8fdc0b01f0 0x7f8fdc152de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:20.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.466+0000 7f8fe1761700 1 -- 172.21.15.162:0/1193226940 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fdc0ae710 msgr2=0x7f8fdc0abfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:20.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.466+0000 7f8fe1761700 1 --2- 172.21.15.162:0/1193226940 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fdc0ae710 0x7f8fdc0abfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:20.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.466+0000 7f8fe1761700 1 -- 172.21.15.162:0/1193226940 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fdc149e80 con 0x7f8fdc0af8f0 2020-11-28T21:56:20.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.466+0000 7f8fe1f62700 1 --2- 172.21.15.162:0/1193226940 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fdc0ae710 0x7f8fdc0abfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:56:20.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.466+0000 7f8fe1761700 1 --2- 172.21.15.162:0/1193226940 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fdc0af8f0 0x7f8fdc0640d0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 rx=0x7f8fcc001f60 tx=0x7f8fcc007c00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:20.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.466+0000 7f8fcaffd700 1 -- 172.21.15.162:0/1193226940 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fcc00e4a0 con 0x7f8fdc0af8f0 2020-11-28T21:56:20.485 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.466+0000 7f8fcaffd700 1 -- 172.21.15.162:0/1193226940 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fcc00e600 con 0x7f8fdc0af8f0 2020-11-28T21:56:20.485 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.466+0000 7f8fe41c6700 1 -- 172.21.15.162:0/1193226940 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fdc14d3c0 con 0x7f8fdc0af8f0 2020-11-28T21:56:20.485 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.466+0000 7f8fcaffd700 1 -- 172.21.15.162:0/1193226940 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fcc00e960 con 0x7f8fdc0af8f0 2020-11-28T21:56:20.486 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.466+0000 7f8fe41c6700 1 -- 172.21.15.162:0/1193226940 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fdc0ab0f0 con 0x7f8fdc0af8f0 2020-11-28T21:56:20.486 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.466+0000 7f8fe41c6700 1 -- 172.21.15.162:0/1193226940 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fdc064900 con 0x7f8fdc0af8f0 2020-11-28T21:56:20.486 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.466+0000 7f8fcaffd700 1 -- 172.21.15.162:0/1193226940 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8fcc019030 con 0x7f8fdc0af8f0 2020-11-28T21:56:20.487 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.470+0000 7f8fcaffd700 1 --2- 172.21.15.162:0/1193226940 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8fbc055500 0x7f8fbc0579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:20.487 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.470+0000 7f8fe1f62700 1 --2- 172.21.15.162:0/1193226940 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8fbc055500 0x7f8fbc0579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:20.487 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.470+0000 7f8fcaffd700 1 -- 172.21.15.162:0/1193226940 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f8fcc06d240 con 0x7f8fdc0af8f0 2020-11-28T21:56:20.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.470+0000 7f8fe1f62700 1 --2- 172.21.15.162:0/1193226940 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8fbc055500 0x7f8fbc0579c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f8fd8019000 tx=0x7f8fd8016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:20.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.478+0000 7f8fcaffd700 1 -- 172.21.15.162:0/1193226940 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f8fcc043780 con 0x7f8fdc0af8f0 2020-11-28T21:56:20.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:20.630+0000 7f8fe41c6700 1 -- 172.21.15.162:0/1193226940 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f8fdc057050 con 0x7f8fdc0af8f0 2020-11-28T21:56:21.451 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.446+0000 7f8fcaffd700 1 -- 172.21.15.162:0/1193226940 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v154) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8fcc046da0 con 0x7f8fdc0af8f0 2020-11-28T21:56:21.455 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.450+0000 7f8fe41c6700 1 -- 172.21.15.162:0/1193226940 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8fbc055500 msgr2=0x7f8fbc0579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:21.456 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.450+0000 7f8fe41c6700 1 --2- 172.21.15.162:0/1193226940 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8fbc055500 0x7f8fbc0579c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f8fd8019000 tx=0x7f8fd8016000).stop 2020-11-28T21:56:21.456 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.450+0000 7f8fe41c6700 1 -- 172.21.15.162:0/1193226940 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fdc0af8f0 msgr2=0x7f8fdc0640d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:21.456 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.450+0000 7f8fe41c6700 1 --2- 172.21.15.162:0/1193226940 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fdc0af8f0 0x7f8fdc0640d0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 rx=0x7f8fcc001f60 tx=0x7f8fcc007c00).stop 2020-11-28T21:56:21.456 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.450+0000 7f8fe41c6700 1 -- 172.21.15.162:0/1193226940 shutdown_connections 2020-11-28T21:56:21.457 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.450+0000 7f8fe41c6700 1 --2- 172.21.15.162:0/1193226940 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8fbc055500 0x7f8fbc0579c0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:21.457 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.450+0000 7f8fe41c6700 1 --2- 172.21.15.162:0/1193226940 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8fdc0b01f0 0x7f8fdc152de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:21.457 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.450+0000 7f8fe41c6700 1 --2- 172.21.15.162:0/1193226940 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fdc0ae710 0x7f8fdc0abfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:21.457 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.450+0000 7f8fe41c6700 1 --2- 172.21.15.162:0/1193226940 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fdc0af8f0 0x7f8fdc0640d0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:21.457 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.450+0000 7f8fe41c6700 1 -- 172.21.15.162:0/1193226940 >> 172.21.15.162:0/1193226940 conn(0x7f8fdc0b56f0 msgr2=0x7f8fdc13b290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:21.458 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.450+0000 7f8fe41c6700 1 -- 172.21.15.162:0/1193226940 shutdown_connections 2020-11-28T21:56:21.458 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.450+0000 7f8fe41c6700 1 -- 172.21.15.162:0/1193226940 wait complete. 2020-11-28T21:56:21.458 INFO:teuthology.orchestra.run.smithi162.stderr: removed all blacklist entries 2020-11-28T21:56:21.471 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2020-11-28T21:56:21.472 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth ls --format=json-pretty 2020-11-28T21:56:21.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.582+0000 7f01f2c69700 1 Processor -- start 2020-11-28T21:56:21.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.582+0000 7f01f2c69700 1 -- start start 2020-11-28T21:56:21.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.582+0000 7f01f2c69700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f01ec060d10 0x7f01ec0b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:21.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.582+0000 7f01f2c69700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f01ec0b60e0 0x7f01ec146180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:21.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.582+0000 7f01f2c69700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01ec0b4930 con 0x7f01ec0b4390 2020-11-28T21:56:21.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.582+0000 7f01f2c69700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01ec0b2170 con 0x7f01ec0b60e0 2020-11-28T21:56:21.598 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.582+0000 7f01f2c69700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01ec0b6660 con 0x7f01ec060d10 2020-11-28T21:56:21.598 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.582+0000 7f01f0a05700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f01ec060d10 0x7f01ec0b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:21.598 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.582+0000 7f01f0a05700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f01ec060d10 0x7f01ec0b52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46170/0 (socket says 172.21.15.162:46170) 2020-11-28T21:56:21.599 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.582+0000 7f01f0a05700 1 -- 172.21.15.162:0/4085169883 learned_addr learned my addr 172.21.15.162:0/4085169883 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:21.599 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.582+0000 7f01ebfff700 1 --2- 172.21.15.162:0/4085169883 >> v2:172.21.15.118:3300/0 conn(0x7f01ec0b60e0 0x7f01ec146180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:21.599 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.582+0000 7f01f0a05700 1 -- 172.21.15.162:0/4085169883 >> v2:172.21.15.118:3300/0 conn(0x7f01ec0b60e0 msgr2=0x7f01ec146180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:21.599 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.582+0000 7f01f0a05700 1 --2- 172.21.15.162:0/4085169883 >> v2:172.21.15.118:3300/0 conn(0x7f01ec0b60e0 0x7f01ec146180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:21.600 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.582+0000 7f01f0a05700 1 -- 172.21.15.162:0/4085169883 >> v1:172.21.15.118:6789/0 conn(0x7f01ec0b4390 legacy=0x7f01ec13fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:21.600 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.582+0000 7f01f0a05700 1 -- 172.21.15.162:0/4085169883 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ec0b4b60 con 0x7f01ec060d10 2020-11-28T21:56:21.600 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.582+0000 7f01ebfff700 1 --2- 172.21.15.162:0/4085169883 >> v2:172.21.15.118:3300/0 conn(0x7f01ec0b60e0 0x7f01ec146180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:56:21.600 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.582+0000 7f01f0a05700 1 --2- 172.21.15.162:0/4085169883 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01ec060d10 0x7f01ec0b52a0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 rx=0x7f01dc0032b0 tx=0x7f01dc009270).ready entity=mon.1 client_cookie=1c2d83efefac51ee server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:21.600 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.582+0000 7f01eb7fe700 1 -- 172.21.15.162:0/4085169883 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01dc00bc00 con 0x7f01ec060d10 2020-11-28T21:56:21.601 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.582+0000 7f01eb7fe700 1 -- 172.21.15.162:0/4085169883 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01dc00bd60 con 0x7f01ec060d10 2020-11-28T21:56:21.601 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.582+0000 7f01eb7fe700 1 -- 172.21.15.162:0/4085169883 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01dc011400 con 0x7f01ec060d10 2020-11-28T21:56:21.601 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.582+0000 7f01f2c69700 1 -- 172.21.15.162:0/4085169883 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01ec060d10 msgr2=0x7f01ec0b52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:21.601 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.582+0000 7f01f2c69700 1 --2- 172.21.15.162:0/4085169883 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01ec060d10 0x7f01ec0b52a0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 rx=0x7f01dc0032b0 tx=0x7f01dc009270).stop 2020-11-28T21:56:21.601 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.586+0000 7f01f2c69700 1 -- 172.21.15.162:0/4085169883 shutdown_connections 2020-11-28T21:56:21.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.586+0000 7f01f2c69700 1 --2- 172.21.15.162:0/4085169883 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01ec060d10 0x7f01ec0b52a0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:21.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.586+0000 7f01f2c69700 1 --2- 172.21.15.162:0/4085169883 >> v2:172.21.15.118:3300/0 conn(0x7f01ec0b60e0 0x7f01ec146180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:21.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.586+0000 7f01f2c69700 1 -- 172.21.15.162:0/4085169883 >> 172.21.15.162:0/4085169883 conn(0x7f01ec05fe70 msgr2=0x7f01ec06aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:21.602 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.586+0000 7f01f2c69700 1 -- 172.21.15.162:0/4085169883 shutdown_connections 2020-11-28T21:56:21.603 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.586+0000 7f01f2c69700 1 -- 172.21.15.162:0/4085169883 wait complete. 2020-11-28T21:56:21.606 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.586+0000 7f01f2c69700 1 Processor -- start 2020-11-28T21:56:21.606 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.586+0000 7f01f2c69700 1 -- start start 2020-11-28T21:56:21.606 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.590+0000 7f01f2c69700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01ec060d10 0x7f01ec140bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:21.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.590+0000 7f01f2c69700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f01ec0b4390 0x7f01ec139b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:21.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.590+0000 7f01f2c69700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01ec0b60e0 0x7f01ec13a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:21.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.590+0000 7f01f2c69700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec0aed90 con 0x7f01ec0b4390 2020-11-28T21:56:21.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.590+0000 7f01f2c69700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec13a520 con 0x7f01ec060d10 2020-11-28T21:56:21.607 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.590+0000 7f01f2c69700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec0b72c0 con 0x7f01ec0b60e0 2020-11-28T21:56:21.608 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.590+0000 7f01f0a05700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01ec060d10 0x7f01ec140bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:21.608 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.590+0000 7f01f0a05700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01ec060d10 0x7f01ec140bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46176/0 (socket says 172.21.15.162:46176) 2020-11-28T21:56:21.608 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.590+0000 7f01f0a05700 1 -- 172.21.15.162:0/1564873778 learned_addr learned my addr 172.21.15.162:0/1564873778 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:21.608 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.590+0000 7f01ebfff700 1 --2- 172.21.15.162:0/1564873778 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f01ec0b4390 0x7f01ec139b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:21.609 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.590+0000 7f01f1206700 1 --2- 172.21.15.162:0/1564873778 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01ec0b60e0 0x7f01ec13a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:21.609 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.590+0000 7f01f0a05700 1 -- 172.21.15.162:0/1564873778 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01ec0b60e0 msgr2=0x7f01ec13a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:21.609 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.590+0000 7f01f0a05700 1 --2- 172.21.15.162:0/1564873778 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01ec0b60e0 0x7f01ec13a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:21.609 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.590+0000 7f01f0a05700 1 -- 172.21.15.162:0/1564873778 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f01ec0b4390 msgr2=0x7f01ec139b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:21.609 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.590+0000 7f01f0a05700 1 --2- 172.21.15.162:0/1564873778 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f01ec0b4390 0x7f01ec139b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:21.610 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.590+0000 7f01f0a05700 1 -- 172.21.15.162:0/1564873778 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ec0b7730 con 0x7f01ec060d10 2020-11-28T21:56:21.610 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.590+0000 7f01f1206700 1 --2- 172.21.15.162:0/1564873778 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01ec0b60e0 0x7f01ec13a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:56:21.610 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.590+0000 7f01f0a05700 1 --2- 172.21.15.162:0/1564873778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01ec060d10 0x7f01ec140bf0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 rx=0x7f01dc0032b0 tx=0x7f01dc016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:21.610 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.590+0000 7f01e9ffb700 1 -- 172.21.15.162:0/1564873778 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01dc0107a0 con 0x7f01ec060d10 2020-11-28T21:56:21.611 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.590+0000 7f01e9ffb700 1 -- 172.21.15.162:0/1564873778 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01dc010900 con 0x7f01ec060d10 2020-11-28T21:56:21.611 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.590+0000 7f01f2c69700 1 -- 172.21.15.162:0/1564873778 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01ec155f80 con 0x7f01ec060d10 2020-11-28T21:56:21.611 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.590+0000 7f01e9ffb700 1 -- 172.21.15.162:0/1564873778 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01dc010c60 con 0x7f01ec060d10 2020-11-28T21:56:21.611 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.590+0000 7f01f2c69700 1 -- 172.21.15.162:0/1564873778 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01ec1410e0 con 0x7f01ec060d10 2020-11-28T21:56:21.612 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.590+0000 7f01f2c69700 1 -- 172.21.15.162:0/1564873778 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01ec057050 con 0x7f01ec060d10 2020-11-28T21:56:21.612 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.594+0000 7f01e9ffb700 1 -- 172.21.15.162:0/1564873778 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f01dc00f070 con 0x7f01ec060d10 2020-11-28T21:56:21.612 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.594+0000 7f01e9ffb700 1 --2- 172.21.15.162:0/1564873778 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f01cc0554e0 0x7f01cc0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:21.612 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.594+0000 7f01e9ffb700 1 -- 172.21.15.162:0/1564873778 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f01dc06f3a0 con 0x7f01ec060d10 2020-11-28T21:56:21.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.594+0000 7f01ebfff700 1 --2- 172.21.15.162:0/1564873778 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f01cc0554e0 0x7f01cc0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:21.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.594+0000 7f01ebfff700 1 --2- 172.21.15.162:0/1564873778 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f01cc0554e0 0x7f01cc0579a0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f01e000c000 tx=0x7f01e0008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:21.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.598+0000 7f01e9ffb700 1 -- 172.21.15.162:0/1564873778 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f01dc0459d0 con 0x7f01ec060d10 2020-11-28T21:56:21.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.782+0000 7f01f2c69700 1 -- 172.21.15.162:0/1564873778 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f01ec044350 con 0x7f01ec060d10 2020-11-28T21:56:21.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.786+0000 7f01e9ffb700 1 -- 172.21.15.162:0/1564873778 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v32) v1 ==== 80+0+4255 (secure 0 0 0) 0x7f01dc015090 con 0x7f01ec060d10 2020-11-28T21:56:21.791 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:56:21.791 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2020-11-28T21:56:21.791 INFO:teuthology.orchestra.run.smithi162.stdout: "auth_dump": [ 2020-11-28T21:56:21.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:21.792 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mds.a", 2020-11-28T21:56:21.792 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfWxUpKxAAIUDwQRKwz8H4Qk9qE356Bg==", 2020-11-28T21:56:21.792 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:56:21.792 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:56:21.792 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:56:21.793 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:56:21.793 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:56:21.793 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:21.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:21.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:21.794 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mds.b", 2020-11-28T21:56:21.794 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfKY2zORAAaO65etMZiYcaTDNX0nsEPw==", 2020-11-28T21:56:21.794 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:56:21.799 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:56:21.799 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:56:21.800 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:56:21.800 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:56:21.800 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:21.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:21.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:21.801 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mds.c", 2020-11-28T21:56:21.802 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfdQbRMxAAWlcjesxTZzZnvKCQneSjjQ==", 2020-11-28T21:56:21.802 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:56:21.802 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:56:21.803 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:56:21.803 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:56:21.803 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:56:21.803 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:21.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:21.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:21.804 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.0", 2020-11-28T21:56:21.805 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQADxcJfF3WLLxAAxsB5YPjH1yjv/vhlRbijqw==", 2020-11-28T21:56:21.805 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:56:21.805 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:56:21.805 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:56:21.806 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:56:21.806 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:21.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:21.807 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:21.808 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.1", 2020-11-28T21:56:21.808 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAGxcJfKYbBFhAA1x4DzFtkQQv6uWLIdOi1qQ==", 2020-11-28T21:56:21.808 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:56:21.809 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:56:21.809 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:56:21.809 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:56:21.810 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:21.810 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:21.810 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:21.811 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.2", 2020-11-28T21:56:21.811 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAIxcJfvcYUOhAASENjM0HfAxxyFe13mmE36Q==", 2020-11-28T21:56:21.811 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:56:21.811 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:56:21.812 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:56:21.812 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:56:21.812 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:21.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:21.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:21.814 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.3", 2020-11-28T21:56:21.814 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQALxcJfc/OHIRAAFYx3s8Ssttt15hjoH04tow==", 2020-11-28T21:56:21.814 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:56:21.815 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:56:21.815 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:56:21.816 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:56:21.816 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:21.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:21.816 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:21.817 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.4", 2020-11-28T21:56:21.817 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAPxcJffGWqCRAAb8CCN5oUPkdDh6BJH5HEnw==", 2020-11-28T21:56:21.817 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:56:21.818 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:56:21.818 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:56:21.818 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:56:21.819 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:21.819 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:21.819 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:21.820 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.5", 2020-11-28T21:56:21.820 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQARxcJfo8GKKRAAfbsCDb4L2wULiKmHyzu7/A==", 2020-11-28T21:56:21.820 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:56:21.821 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:56:21.821 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:56:21.821 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:56:21.822 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:21.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:21.822 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:21.823 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.6", 2020-11-28T21:56:21.823 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAUxcJf8Y2MEBAAqQX6rX3LK0zodlI2wK4JrA==", 2020-11-28T21:56:21.823 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:56:21.824 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:56:21.824 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:56:21.824 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:56:21.825 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:21.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:21.825 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:21.826 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "osd.7", 2020-11-28T21:56:21.826 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAWxcJfCLBdMhAAf8QixQMQDrfZOGWN8rHe5g==", 2020-11-28T21:56:21.826 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:56:21.826 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:56:21.827 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:56:21.827 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:56:21.827 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:21.827 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:21.827 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:21.828 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "client.0", 2020-11-28T21:56:21.828 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQABxcJfS9vDAxAAUk4FU8FVt4if/CRH5ZisOA==", 2020-11-28T21:56:21.828 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:56:21.828 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:56:21.829 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow r", 2020-11-28T21:56:21.829 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow rw pool=cephfs_data" 2020-11-28T21:56:21.829 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:21.829 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:21.830 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:21.830 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "client.1", 2020-11-28T21:56:21.830 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQABxcJfuCU8BhAAXUSoSK3D0XRo58uHNzcjXg==", 2020-11-28T21:56:21.830 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:56:21.830 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow", 2020-11-28T21:56:21.831 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow r", 2020-11-28T21:56:21.831 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow rw pool=cephfs_data" 2020-11-28T21:56:21.831 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:21.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:21.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:21.832 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "client.admin", 2020-11-28T21:56:21.832 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfdlTWBxAApzAxlVjzdnETQF//Vnsg5w==", 2020-11-28T21:56:21.832 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:56:21.832 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow *", 2020-11-28T21:56:21.832 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": "allow *", 2020-11-28T21:56:21.833 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow *", 2020-11-28T21:56:21.833 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:56:21.833 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:21.833 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:21.833 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:21.834 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mgr.x", 2020-11-28T21:56:21.834 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfZhdvKBAAT7zCCsD2OVCHjvddGLxQ/A==", 2020-11-28T21:56:21.834 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:56:21.834 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow *", 2020-11-28T21:56:21.835 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow profile mgr", 2020-11-28T21:56:21.835 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:56:21.835 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:21.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:21.835 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:21.836 INFO:teuthology.orchestra.run.smithi162.stdout: "entity": "mgr.y", 2020-11-28T21:56:21.836 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "AQAAxcJfs8fvJRAAP0Atp074EeMHiTtpCCVHuA==", 2020-11-28T21:56:21.836 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": { 2020-11-28T21:56:21.836 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": "allow *", 2020-11-28T21:56:21.836 INFO:teuthology.orchestra.run.smithi162.stdout: "mon": "allow profile mgr", 2020-11-28T21:56:21.837 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": "allow *" 2020-11-28T21:56:21.837 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:21.837 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:21.837 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:21.837 INFO:teuthology.orchestra.run.smithi162.stdout:} 2020-11-28T21:56:21.839 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.786+0000 7f01f2c69700 1 -- 172.21.15.162:0/1564873778 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f01cc0554e0 msgr2=0x7f01cc0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:21.840 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.786+0000 7f01f2c69700 1 --2- 172.21.15.162:0/1564873778 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f01cc0554e0 0x7f01cc0579a0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f01e000c000 tx=0x7f01e0008040).stop 2020-11-28T21:56:21.840 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.786+0000 7f01f2c69700 1 -- 172.21.15.162:0/1564873778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01ec060d10 msgr2=0x7f01ec140bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:21.840 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.786+0000 7f01f2c69700 1 --2- 172.21.15.162:0/1564873778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01ec060d10 0x7f01ec140bf0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 rx=0x7f01dc0032b0 tx=0x7f01dc016000).stop 2020-11-28T21:56:21.840 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.786+0000 7f01f2c69700 1 -- 172.21.15.162:0/1564873778 shutdown_connections 2020-11-28T21:56:21.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.786+0000 7f01f2c69700 1 --2- 172.21.15.162:0/1564873778 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f01cc0554e0 0x7f01cc0579a0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:21.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.786+0000 7f01f2c69700 1 --2- 172.21.15.162:0/1564873778 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f01ec0b60e0 0x7f01ec13a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:21.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.786+0000 7f01f2c69700 1 --2- 172.21.15.162:0/1564873778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01ec060d10 0x7f01ec140bf0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:21.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.786+0000 7f01f2c69700 1 --2- 172.21.15.162:0/1564873778 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f01ec0b4390 0x7f01ec139b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:21.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.786+0000 7f01f2c69700 1 -- 172.21.15.162:0/1564873778 >> 172.21.15.162:0/1564873778 conn(0x7f01ec05fe70 msgr2=0x7f01ec13e500 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:21.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.786+0000 7f01f2c69700 1 -- 172.21.15.162:0/1564873778 shutdown_connections 2020-11-28T21:56:21.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.786+0000 7f01f2c69700 1 -- 172.21.15.162:0/1564873778 wait complete. 2020-11-28T21:56:21.843 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:56:21.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.954+0000 7f1c52e97700 1 Processor -- start 2020-11-28T21:56:21.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.954+0000 7f1c52e97700 1 -- start start 2020-11-28T21:56:21.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.954+0000 7f1c52e97700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1c4c0b7780 0x7f1c4c0b1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:21.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.954+0000 7f1c52e97700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1c4c147a90 0x7f1c4c0b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:21.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.954+0000 7f1c52e97700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c4c0654e0 con 0x7f1c4c09ca50 2020-11-28T21:56:21.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.954+0000 7f1c52e97700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c4c065780 con 0x7f1c4c0b7780 2020-11-28T21:56:21.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.954+0000 7f1c52e97700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c065a20 con 0x7f1c4c147a90 2020-11-28T21:56:21.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.954+0000 7f1c51694700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1c4c147a90 0x7f1c4c0b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:21.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.954+0000 7f1c51e95700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1c4c0b7780 0x7f1c4c0b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:21.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.954+0000 7f1c51694700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1c4c147a90 0x7f1c4c0b1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:43410/0 (socket says 172.21.15.162:43410) 2020-11-28T21:56:21.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.954+0000 7f1c51694700 1 -- 172.21.15.162:0/4128470027 learned_addr learned my addr 172.21.15.162:0/4128470027 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:21.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.954+0000 7f1c51694700 1 -- 172.21.15.162:0/4128470027 >> v2:172.21.15.118:3300/0 conn(0x7f1c4c0b7780 msgr2=0x7f1c4c0b1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:21.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.954+0000 7f1c51694700 1 --2- 172.21.15.162:0/4128470027 >> v2:172.21.15.118:3300/0 conn(0x7f1c4c0b7780 0x7f1c4c0b1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:21.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.954+0000 7f1c51694700 1 -- 172.21.15.162:0/4128470027 >> v1:172.21.15.118:6789/0 conn(0x7f1c4c09ca50 legacy=0x7f1c4c0b2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:21.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.954+0000 7f1c51694700 1 -- 172.21.15.162:0/4128470027 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c4c065cc0 con 0x7f1c4c147a90 2020-11-28T21:56:21.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.954+0000 7f1c51694700 1 --2- 172.21.15.162:0/4128470027 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1c4c147a90 0x7f1c4c0b1d60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f1c3c002f30 tx=0x7f1c3c006e80).ready entity=mon.2 client_cookie=70767cfba4368ecd server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:21.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.954+0000 7f1c50e93700 1 -- 172.21.15.162:0/4128470027 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c3c0097f0 con 0x7f1c4c147a90 2020-11-28T21:56:21.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.954+0000 7f1c50e93700 1 -- 172.21.15.162:0/4128470027 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c3c009950 con 0x7f1c4c147a90 2020-11-28T21:56:21.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.954+0000 7f1c50e93700 1 -- 172.21.15.162:0/4128470027 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c3c009cb0 con 0x7f1c4c147a90 2020-11-28T21:56:21.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.954+0000 7f1c52e97700 1 -- 172.21.15.162:0/4128470027 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1c4c147a90 msgr2=0x7f1c4c0b1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:21.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.954+0000 7f1c52e97700 1 --2- 172.21.15.162:0/4128470027 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1c4c147a90 0x7f1c4c0b1d60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f1c3c002f30 tx=0x7f1c3c006e80).stop 2020-11-28T21:56:21.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.958+0000 7f1c52e97700 1 -- 172.21.15.162:0/4128470027 shutdown_connections 2020-11-28T21:56:21.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.958+0000 7f1c52e97700 1 --2- 172.21.15.162:0/4128470027 >> v2:172.21.15.118:3300/0 conn(0x7f1c4c0b7780 0x7f1c4c0b1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:21.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.958+0000 7f1c52e97700 1 --2- 172.21.15.162:0/4128470027 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1c4c147a90 0x7f1c4c0b1d60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:21.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.958+0000 7f1c52e97700 1 -- 172.21.15.162:0/4128470027 >> 172.21.15.162:0/4128470027 conn(0x7f1c4c05c440 msgr2=0x7f1c4c0b6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:21.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.958+0000 7f1c52e97700 1 -- 172.21.15.162:0/4128470027 shutdown_connections 2020-11-28T21:56:21.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.958+0000 7f1c52e97700 1 -- 172.21.15.162:0/4128470027 wait complete. 2020-11-28T21:56:21.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.958+0000 7f1c52e97700 1 Processor -- start 2020-11-28T21:56:21.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.958+0000 7f1c52e97700 1 -- start start 2020-11-28T21:56:21.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.958+0000 7f1c52e97700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1c4c09ca50 0x7f1c4c05f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:21.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.958+0000 7f1c52e97700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c4c0b7780 0x7f1c4c155b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:21.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.958+0000 7f1c52e97700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1c4c147a90 0x7f1c4c156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:21.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.958+0000 7f1c52e97700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c0b15a0 con 0x7f1c4c09ca50 2020-11-28T21:56:21.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.958+0000 7f1c52e97700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c0b1d50 con 0x7f1c4c0b7780 2020-11-28T21:56:21.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.958+0000 7f1c52e97700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c0b0c30 con 0x7f1c4c147a90 2020-11-28T21:56:21.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.958+0000 7f1c51694700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c4c0b7780 0x7f1c4c155b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:21.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.958+0000 7f1c51694700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c4c0b7780 0x7f1c4c155b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46192/0 (socket says 172.21.15.162:46192) 2020-11-28T21:56:21.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.958+0000 7f1c51694700 1 -- 172.21.15.162:0/3510352521 learned_addr learned my addr 172.21.15.162:0/3510352521 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:21.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.958+0000 7f1c52696700 1 --2- 172.21.15.162:0/3510352521 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1c4c147a90 0x7f1c4c156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:21.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.958+0000 7f1c51e95700 1 --2- 172.21.15.162:0/3510352521 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1c4c09ca50 0x7f1c4c05f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:21.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.958+0000 7f1c51694700 1 -- 172.21.15.162:0/3510352521 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1c4c147a90 msgr2=0x7f1c4c156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:21.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.958+0000 7f1c51694700 1 --2- 172.21.15.162:0/3510352521 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1c4c147a90 0x7f1c4c156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:21.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.962+0000 7f1c51694700 1 -- 172.21.15.162:0/3510352521 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1c4c09ca50 msgr2=0x7f1c4c05f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:21.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.962+0000 7f1c51694700 1 --2- 172.21.15.162:0/3510352521 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1c4c09ca50 0x7f1c4c05f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:21.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.962+0000 7f1c51694700 1 -- 172.21.15.162:0/3510352521 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c4c0afc60 con 0x7f1c4c0b7780 2020-11-28T21:56:21.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.962+0000 7f1c51694700 1 --2- 172.21.15.162:0/3510352521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c4c0b7780 0x7f1c4c155b20 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 rx=0x7f1c3c002f30 tx=0x7f1c3c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:21.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.962+0000 7f1c42ffd700 1 -- 172.21.15.162:0/3510352521 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c3c011b00 con 0x7f1c4c0b7780 2020-11-28T21:56:21.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.962+0000 7f1c42ffd700 1 -- 172.21.15.162:0/3510352521 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c3c011c60 con 0x7f1c4c0b7780 2020-11-28T21:56:21.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.962+0000 7f1c52e97700 1 -- 172.21.15.162:0/3510352521 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c4c0b0990 con 0x7f1c4c0b7780 2020-11-28T21:56:21.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.962+0000 7f1c42ffd700 1 -- 172.21.15.162:0/3510352521 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c3c01e460 con 0x7f1c4c0b7780 2020-11-28T21:56:21.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.962+0000 7f1c52e97700 1 -- 172.21.15.162:0/3510352521 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c4c0b19d0 con 0x7f1c4c0b7780 2020-11-28T21:56:21.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.962+0000 7f1c52e97700 1 -- 172.21.15.162:0/3510352521 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c4c057050 con 0x7f1c4c0b7780 2020-11-28T21:56:21.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.962+0000 7f1c42ffd700 1 -- 172.21.15.162:0/3510352521 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1c3c025030 con 0x7f1c4c0b7780 2020-11-28T21:56:21.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.962+0000 7f1c42ffd700 1 --2- 172.21.15.162:0/3510352521 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1c340554e0 0x7f1c340579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:21.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.966+0000 7f1c42ffd700 1 -- 172.21.15.162:0/3510352521 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f1c3c06eed0 con 0x7f1c4c0b7780 2020-11-28T21:56:21.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.966+0000 7f1c51e95700 1 --2- 172.21.15.162:0/3510352521 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1c340554e0 0x7f1c340579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:21.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.966+0000 7f1c51e95700 1 --2- 172.21.15.162:0/3510352521 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1c340554e0 0x7f1c340579a0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7f1c4800b000 tx=0x7f1c480091a0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:21.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:21.974+0000 7f1c42ffd700 1 -- 172.21.15.162:0/3510352521 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1c3c045500 con 0x7f1c4c0b7780 2020-11-28T21:56:22.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.114+0000 7f1c52e97700 1 -- 172.21.15.162:0/3510352521 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c4c068350 con 0x7f1c4c0b7780 2020-11-28T21:56:22.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.118+0000 7f1c42ffd700 1 -- 172.21.15.162:0/3510352521 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) v1 ==== 74+0+13287 (secure 0 0 0) 0x7f1c3c023090 con 0x7f1c4c0b7780 2020-11-28T21:56:22.122 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:56:22.122 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":154,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:56:21.442629+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:56:22.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.118+0000 7f1c52e97700 1 -- 172.21.15.162:0/3510352521 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1c340554e0 msgr2=0x7f1c340579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:22.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.118+0000 7f1c52e97700 1 --2- 172.21.15.162:0/3510352521 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1c340554e0 0x7f1c340579a0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7f1c4800b000 tx=0x7f1c480091a0).stop 2020-11-28T21:56:22.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.118+0000 7f1c52e97700 1 -- 172.21.15.162:0/3510352521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c4c0b7780 msgr2=0x7f1c4c155b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:22.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.118+0000 7f1c52e97700 1 --2- 172.21.15.162:0/3510352521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c4c0b7780 0x7f1c4c155b20 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 rx=0x7f1c3c002f30 tx=0x7f1c3c015000).stop 2020-11-28T21:56:22.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.118+0000 7f1c52e97700 1 -- 172.21.15.162:0/3510352521 shutdown_connections 2020-11-28T21:56:22.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.118+0000 7f1c52e97700 1 --2- 172.21.15.162:0/3510352521 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1c340554e0 0x7f1c340579a0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:22.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.118+0000 7f1c52e97700 1 --2- 172.21.15.162:0/3510352521 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1c4c147a90 0x7f1c4c156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:22.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.118+0000 7f1c52e97700 1 --2- 172.21.15.162:0/3510352521 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1c4c09ca50 0x7f1c4c05f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:22.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.118+0000 7f1c52e97700 1 --2- 172.21.15.162:0/3510352521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c4c0b7780 0x7f1c4c155b20 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:22.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.118+0000 7f1c52e97700 1 -- 172.21.15.162:0/3510352521 >> 172.21.15.162:0/3510352521 conn(0x7f1c4c05c440 msgr2=0x7f1c4c146200 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:22.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.118+0000 7f1c52e97700 1 -- 172.21.15.162:0/3510352521 shutdown_connections 2020-11-28T21:56:22.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.118+0000 7f1c52e97700 1 -- 172.21.15.162:0/3510352521 wait complete. 2020-11-28T21:56:22.143 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-28T21:56:22.144 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:56:22.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.254+0000 7f7fc759e700 1 Processor -- start 2020-11-28T21:56:22.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.254+0000 7f7fc759e700 1 -- start start 2020-11-28T21:56:22.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.258+0000 7f7fc759e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f7fc80b1eb0 0x7f7fc80b2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:22.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.258+0000 7f7fc759e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7fc80af6a0 0x7f7fc8060a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:22.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.258+0000 7f7fc759e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fc81471f0 con 0x7f7fc8144350 2020-11-28T21:56:22.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.258+0000 7f7fc759e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fc8060090 con 0x7f7fc80b1eb0 2020-11-28T21:56:22.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.258+0000 7f7fc759e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fc8060330 con 0x7f7fc80af6a0 2020-11-28T21:56:22.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.258+0000 7f7fc5d9b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7fc80af6a0 0x7f7fc8060a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:22.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.258+0000 7f7fc5d9b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7fc80af6a0 0x7f7fc8060a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46198/0 (socket says 172.21.15.162:46198) 2020-11-28T21:56:22.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.258+0000 7f7fc5d9b700 1 -- 172.21.15.162:0/3966827252 learned_addr learned my addr 172.21.15.162:0/3966827252 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:22.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.258+0000 7f7fc659c700 1 --2- 172.21.15.162:0/3966827252 >> v2:172.21.15.118:3300/0 conn(0x7f7fc80b1eb0 0x7f7fc80b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:22.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.258+0000 7f7fc5d9b700 1 -- 172.21.15.162:0/3966827252 >> v2:172.21.15.118:3300/0 conn(0x7f7fc80b1eb0 msgr2=0x7f7fc80b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:22.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.258+0000 7f7fc5d9b700 1 --2- 172.21.15.162:0/3966827252 >> v2:172.21.15.118:3300/0 conn(0x7f7fc80b1eb0 0x7f7fc80b2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:22.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.258+0000 7f7fc5d9b700 1 -- 172.21.15.162:0/3966827252 >> v1:172.21.15.118:6789/0 conn(0x7f7fc8144350 legacy=0x7f7fc814a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:22.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.258+0000 7f7fc5d9b700 1 -- 172.21.15.162:0/3966827252 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc80605d0 con 0x7f7fc80af6a0 2020-11-28T21:56:22.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.258+0000 7f7fc5d9b700 1 --2- 172.21.15.162:0/3966827252 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fc80af6a0 0x7f7fc8060a70 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 rx=0x7f7fb8002f30 tx=0x7f7fb8006e80).ready entity=mon.1 client_cookie=53962b27017eca9c server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:22.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.258+0000 7f7fc559a700 1 -- 172.21.15.162:0/3966827252 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fb8009960 con 0x7f7fc80af6a0 2020-11-28T21:56:22.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.258+0000 7f7fc559a700 1 -- 172.21.15.162:0/3966827252 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fb8009ac0 con 0x7f7fc80af6a0 2020-11-28T21:56:22.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.258+0000 7f7fc559a700 1 -- 172.21.15.162:0/3966827252 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fb8009e20 con 0x7f7fc80af6a0 2020-11-28T21:56:22.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.258+0000 7f7fc759e700 1 -- 172.21.15.162:0/3966827252 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fc80af6a0 msgr2=0x7f7fc8060a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:22.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.258+0000 7f7fc759e700 1 --2- 172.21.15.162:0/3966827252 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fc80af6a0 0x7f7fc8060a70 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 rx=0x7f7fb8002f30 tx=0x7f7fb8006e80).stop 2020-11-28T21:56:22.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.258+0000 7f7fc759e700 1 -- 172.21.15.162:0/3966827252 shutdown_connections 2020-11-28T21:56:22.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.258+0000 7f7fc759e700 1 --2- 172.21.15.162:0/3966827252 >> v2:172.21.15.118:3300/0 conn(0x7f7fc80b1eb0 0x7f7fc80b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:22.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.258+0000 7f7fc759e700 1 --2- 172.21.15.162:0/3966827252 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fc80af6a0 0x7f7fc8060a70 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:22.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.258+0000 7f7fc759e700 1 -- 172.21.15.162:0/3966827252 >> 172.21.15.162:0/3966827252 conn(0x7f7fc80b57d0 msgr2=0x7f7fc80b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:22.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.258+0000 7f7fc759e700 1 -- 172.21.15.162:0/3966827252 shutdown_connections 2020-11-28T21:56:22.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.258+0000 7f7fc759e700 1 -- 172.21.15.162:0/3966827252 wait complete. 2020-11-28T21:56:22.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.262+0000 7f7fc759e700 1 Processor -- start 2020-11-28T21:56:22.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.262+0000 7f7fc759e700 1 -- start start 2020-11-28T21:56:22.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.262+0000 7f7fc759e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7fc80af6a0 0x7f7fc8065a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:22.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.262+0000 7f7fc759e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7fc80b1eb0 0x7f7fc809e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:22.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.262+0000 7f7fc759e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fc8144350 0x7f7fc80a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:22.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.262+0000 7f7fc759e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc809ff80 con 0x7f7fc80af6a0 2020-11-28T21:56:22.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.262+0000 7f7fc759e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc8157190 con 0x7f7fc8144350 2020-11-28T21:56:22.273 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.262+0000 7f7fc759e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc80661e0 con 0x7f7fc80b1eb0 2020-11-28T21:56:22.273 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.262+0000 7f7fc6d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fc8144350 0x7f7fc80a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:22.273 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.262+0000 7f7fc6d9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fc8144350 0x7f7fc80a0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46206/0 (socket says 172.21.15.162:46206) 2020-11-28T21:56:22.273 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.262+0000 7f7fc6d9d700 1 -- 172.21.15.162:0/2708027904 learned_addr learned my addr 172.21.15.162:0/2708027904 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:22.274 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.262+0000 7f7fc5d9b700 1 --2- 172.21.15.162:0/2708027904 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7fc80b1eb0 0x7f7fc809e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:22.274 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.262+0000 7f7fc659c700 1 --2- 172.21.15.162:0/2708027904 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7fc80af6a0 0x7f7fc8065a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:22.274 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.262+0000 7f7fc6d9d700 1 -- 172.21.15.162:0/2708027904 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7fc80b1eb0 msgr2=0x7f7fc809e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:22.274 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.262+0000 7f7fc6d9d700 1 --2- 172.21.15.162:0/2708027904 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7fc80b1eb0 0x7f7fc809e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:22.275 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.262+0000 7f7fc6d9d700 1 -- 172.21.15.162:0/2708027904 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7fc80af6a0 msgr2=0x7f7fc8065a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:22.275 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.262+0000 7f7fc6d9d700 1 --2- 172.21.15.162:0/2708027904 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7fc80af6a0 0x7f7fc8065a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:22.275 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.262+0000 7f7fc6d9d700 1 -- 172.21.15.162:0/2708027904 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc8066480 con 0x7f7fc8144350 2020-11-28T21:56:22.275 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.262+0000 7f7fc5d9b700 1 --2- 172.21.15.162:0/2708027904 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7fc80b1eb0 0x7f7fc809e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:56:22.275 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.262+0000 7f7fc6d9d700 1 --2- 172.21.15.162:0/2708027904 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fc8144350 0x7f7fc80a0790 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 rx=0x7f7fc8060950 tx=0x7f7fc0007f70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:22.276 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.262+0000 7f7fbf7fe700 1 -- 172.21.15.162:0/2708027904 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fc0003080 con 0x7f7fc8144350 2020-11-28T21:56:22.276 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.262+0000 7f7fbf7fe700 1 -- 172.21.15.162:0/2708027904 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc000a7b0 con 0x7f7fc8144350 2020-11-28T21:56:22.276 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.262+0000 7f7fc759e700 1 -- 172.21.15.162:0/2708027904 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fc8066720 con 0x7f7fc8144350 2020-11-28T21:56:22.276 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.262+0000 7f7fbf7fe700 1 -- 172.21.15.162:0/2708027904 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fc000ab10 con 0x7f7fc8144350 2020-11-28T21:56:22.276 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.262+0000 7f7fc759e700 1 -- 172.21.15.162:0/2708027904 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fc809e870 con 0x7f7fc8144350 2020-11-28T21:56:22.277 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.266+0000 7f7fc759e700 1 -- 172.21.15.162:0/2708027904 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fc8057050 con 0x7f7fc8144350 2020-11-28T21:56:22.278 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.266+0000 7f7fbf7fe700 1 -- 172.21.15.162:0/2708027904 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7fc0019030 con 0x7f7fc8144350 2020-11-28T21:56:22.278 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.266+0000 7f7fbf7fe700 1 --2- 172.21.15.162:0/2708027904 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7fac0554e0 0x7f7fac0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:22.278 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.266+0000 7f7fbf7fe700 1 -- 172.21.15.162:0/2708027904 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f7fc006d300 con 0x7f7fc8144350 2020-11-28T21:56:22.278 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.266+0000 7f7fc659c700 1 --2- 172.21.15.162:0/2708027904 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7fac0554e0 0x7f7fac0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:22.278 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.266+0000 7f7fc659c700 1 --2- 172.21.15.162:0/2708027904 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7fac0554e0 0x7f7fac0579a0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7f7fb000c000 tx=0x7f7fb0008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:22.279 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.274+0000 7f7fbf7fe700 1 -- 172.21.15.162:0/2708027904 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7fc0043890 con 0x7f7fc8144350 2020-11-28T21:56:22.415 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.410+0000 7f7fc759e700 1 -- 172.21.15.162:0/2708027904 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f7fc80a89f0 con 0x7f7fc8144350 2020-11-28T21:56:22.416 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.410+0000 7f7fbf7fe700 1 -- 172.21.15.162:0/2708027904 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v154) v1 ==== 110+0+10 (secure 0 0 0) 0x7f7fc0046eb0 con 0x7f7fc8144350 2020-11-28T21:56:22.416 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:56:22.421 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.414+0000 7f7fc759e700 1 -- 172.21.15.162:0/2708027904 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7fac0554e0 msgr2=0x7f7fac0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:22.421 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.414+0000 7f7fc759e700 1 --2- 172.21.15.162:0/2708027904 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7fac0554e0 0x7f7fac0579a0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7f7fb000c000 tx=0x7f7fb0008040).stop 2020-11-28T21:56:22.421 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.414+0000 7f7fc759e700 1 -- 172.21.15.162:0/2708027904 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fc8144350 msgr2=0x7f7fc80a0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:22.421 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.414+0000 7f7fc759e700 1 --2- 172.21.15.162:0/2708027904 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fc8144350 0x7f7fc80a0790 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 rx=0x7f7fc8060950 tx=0x7f7fc0007f70).stop 2020-11-28T21:56:22.422 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.414+0000 7f7fc759e700 1 -- 172.21.15.162:0/2708027904 shutdown_connections 2020-11-28T21:56:22.422 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.414+0000 7f7fc759e700 1 --2- 172.21.15.162:0/2708027904 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7fac0554e0 0x7f7fac0579a0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:22.422 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.414+0000 7f7fc759e700 1 --2- 172.21.15.162:0/2708027904 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fc8144350 0x7f7fc80a0790 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:22.422 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.414+0000 7f7fc759e700 1 --2- 172.21.15.162:0/2708027904 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7fc80af6a0 0x7f7fc8065a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:22.423 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.414+0000 7f7fc759e700 1 --2- 172.21.15.162:0/2708027904 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7fc80b1eb0 0x7f7fc809e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:22.423 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.414+0000 7f7fc759e700 1 -- 172.21.15.162:0/2708027904 >> 172.21.15.162:0/2708027904 conn(0x7f7fc80b57d0 msgr2=0x7f7fc80afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:22.423 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.414+0000 7f7fc759e700 1 -- 172.21.15.162:0/2708027904 shutdown_connections 2020-11-28T21:56:22.423 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.414+0000 7f7fc759e700 1 -- 172.21.15.162:0/2708027904 wait complete. 2020-11-28T21:56:22.435 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:56:22.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.546+0000 7fa2015a2700 1 Processor -- start 2020-11-28T21:56:22.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.546+0000 7fa2015a2700 1 -- start start 2020-11-28T21:56:22.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.546+0000 7fa2015a2700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fa1fc0b3ab0 0x7fa1fc05c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:22.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.546+0000 7fa2015a2700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa1fc0b30e0 0x7fa1fc13fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:22.555 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.546+0000 7fa2015a2700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1fc05baf0 con 0x7fa1fc0b2d00 2020-11-28T21:56:22.555 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.546+0000 7fa2015a2700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1fc0b0e20 con 0x7fa1fc0b3ab0 2020-11-28T21:56:22.555 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.546+0000 7fa2015a2700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1fc05bd90 con 0x7fa1fc0b30e0 2020-11-28T21:56:22.555 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.546+0000 7fa200da1700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa1fc0b30e0 0x7fa1fc13fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:22.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.546+0000 7fa200da1700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa1fc0b30e0 0x7fa1fc13fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46212/0 (socket says 172.21.15.162:46212) 2020-11-28T21:56:22.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.546+0000 7fa1fbfff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fa1fc0b2d00 0x7fa1fc142ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:53054/0 (socket says 172.21.15.162:53054) 2020-11-28T21:56:22.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.546+0000 7fa200da1700 1 -- 172.21.15.162:0/4181090592 learned_addr learned my addr 172.21.15.162:0/4181090592 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:22.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.550+0000 7fa1fb7fe700 1 --2- 172.21.15.162:0/4181090592 >> v2:172.21.15.118:3300/0 conn(0x7fa1fc0b3ab0 0x7fa1fc05c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:22.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.550+0000 7fa200da1700 1 -- 172.21.15.162:0/4181090592 >> v2:172.21.15.118:3300/0 conn(0x7fa1fc0b3ab0 msgr2=0x7fa1fc05c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:22.557 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.550+0000 7fa200da1700 1 --2- 172.21.15.162:0/4181090592 >> v2:172.21.15.118:3300/0 conn(0x7fa1fc0b3ab0 0x7fa1fc05c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:22.557 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.550+0000 7fa200da1700 1 -- 172.21.15.162:0/4181090592 >> v1:172.21.15.162:6789/0 conn(0x7fa1fc0b2d00 legacy=0x7fa1fc142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:22.557 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.550+0000 7fa200da1700 1 -- 172.21.15.162:0/4181090592 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1fc05c030 con 0x7fa1fc0b30e0 2020-11-28T21:56:22.558 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.550+0000 7fa200da1700 1 --2- 172.21.15.162:0/4181090592 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1fc0b30e0 0x7fa1fc13fa70 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 rx=0x7fa1f40032b0 tx=0x7fa1f4009270).ready entity=mon.1 client_cookie=f1b522348ee08abe server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:22.558 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.550+0000 7fa1faffd700 1 -- 172.21.15.162:0/4181090592 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1f400bc00 con 0x7fa1fc0b30e0 2020-11-28T21:56:22.558 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.550+0000 7fa1faffd700 1 -- 172.21.15.162:0/4181090592 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f400bd60 con 0x7fa1fc0b30e0 2020-11-28T21:56:22.558 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.550+0000 7fa1faffd700 1 -- 172.21.15.162:0/4181090592 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1f4011400 con 0x7fa1fc0b30e0 2020-11-28T21:56:22.558 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.550+0000 7fa2015a2700 1 -- 172.21.15.162:0/4181090592 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1fc0b30e0 msgr2=0x7fa1fc13fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:22.559 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.550+0000 7fa2015a2700 1 --2- 172.21.15.162:0/4181090592 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1fc0b30e0 0x7fa1fc13fa70 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 rx=0x7fa1f40032b0 tx=0x7fa1f4009270).stop 2020-11-28T21:56:22.567 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.550+0000 7fa2015a2700 1 -- 172.21.15.162:0/4181090592 shutdown_connections 2020-11-28T21:56:22.567 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.550+0000 7fa2015a2700 1 --2- 172.21.15.162:0/4181090592 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1fc0b30e0 0x7fa1fc13fa70 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:22.567 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.550+0000 7fa2015a2700 1 --2- 172.21.15.162:0/4181090592 >> v2:172.21.15.118:3300/0 conn(0x7fa1fc0b3ab0 0x7fa1fc05c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:22.568 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.550+0000 7fa2015a2700 1 -- 172.21.15.162:0/4181090592 >> 172.21.15.162:0/4181090592 conn(0x7fa1fc0604c0 msgr2=0x7fa1fc0608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:22.568 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.550+0000 7fa2015a2700 1 -- 172.21.15.162:0/4181090592 shutdown_connections 2020-11-28T21:56:22.568 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.550+0000 7fa2015a2700 1 -- 172.21.15.162:0/4181090592 wait complete. 2020-11-28T21:56:22.568 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.550+0000 7fa2015a2700 1 Processor -- start 2020-11-28T21:56:22.569 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.550+0000 7fa2015a2700 1 -- start start 2020-11-28T21:56:22.569 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.550+0000 7fa2015a2700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa1fc0b2d00 0x7fa1fc145820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:22.569 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.550+0000 7fa2015a2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1fc0b30e0 0x7fa1fc1522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:22.569 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.550+0000 7fa2015a2700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa1fc0b3ab0 0x7fa1fc0acf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:22.569 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.550+0000 7fa2015a2700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc0af130 con 0x7fa1fc0b3ab0 2020-11-28T21:56:22.570 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.550+0000 7fa2015a2700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc151b80 con 0x7fa1fc0b30e0 2020-11-28T21:56:22.570 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.550+0000 7fa2015a2700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc151e20 con 0x7fa1fc0b2d00 2020-11-28T21:56:22.570 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.550+0000 7fa1fb7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1fc0b30e0 0x7fa1fc1522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:22.571 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.550+0000 7fa1fb7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1fc0b30e0 0x7fa1fc1522c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46218/0 (socket says 172.21.15.162:46218) 2020-11-28T21:56:22.571 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.550+0000 7fa1fb7fe700 1 -- 172.21.15.162:0/3876416426 learned_addr learned my addr 172.21.15.162:0/3876416426 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:22.571 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.554+0000 7fa1fbfff700 1 --2- 172.21.15.162:0/3876416426 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa1fc0b2d00 0x7fa1fc145820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:22.571 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.554+0000 7fa1fb7fe700 1 -- 172.21.15.162:0/3876416426 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa1fc0b2d00 msgr2=0x7fa1fc145820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:22.571 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.554+0000 7fa1fb7fe700 1 --2- 172.21.15.162:0/3876416426 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa1fc0b2d00 0x7fa1fc145820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:22.572 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.554+0000 7fa1fb7fe700 1 -- 172.21.15.162:0/3876416426 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa1fc0b3ab0 msgr2=0x7fa1fc0acf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:22.572 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.554+0000 7fa200da1700 1 --2- 172.21.15.162:0/3876416426 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa1fc0b3ab0 0x7fa1fc0acf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:22.572 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.554+0000 7fa1fb7fe700 1 --2- 172.21.15.162:0/3876416426 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa1fc0b3ab0 0x7fa1fc0acf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:22.572 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.554+0000 7fa1fb7fe700 1 -- 172.21.15.162:0/3876416426 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1fc1527b0 con 0x7fa1fc0b30e0 2020-11-28T21:56:22.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.554+0000 7fa1fbfff700 1 --2- 172.21.15.162:0/3876416426 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa1fc0b2d00 0x7fa1fc145820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:56:22.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.554+0000 7fa200da1700 1 --2- 172.21.15.162:0/3876416426 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa1fc0b3ab0 0x7fa1fc0acf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:56:22.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.554+0000 7fa1fb7fe700 1 --2- 172.21.15.162:0/3876416426 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1fc0b30e0 0x7fa1fc1522c0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 rx=0x7fa1e400a000 tx=0x7fa1e4008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:22.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.554+0000 7fa1f97fa700 1 -- 172.21.15.162:0/3876416426 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1e4011780 con 0x7fa1fc0b30e0 2020-11-28T21:56:22.573 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.554+0000 7fa1f97fa700 1 -- 172.21.15.162:0/3876416426 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e40118e0 con 0x7fa1fc0b30e0 2020-11-28T21:56:22.574 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.554+0000 7fa2015a2700 1 -- 172.21.15.162:0/3876416426 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1fc0ad450 con 0x7fa1fc0b30e0 2020-11-28T21:56:22.574 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.554+0000 7fa1f97fa700 1 -- 172.21.15.162:0/3876416426 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1e4011c40 con 0x7fa1fc0b30e0 2020-11-28T21:56:22.574 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.554+0000 7fa2015a2700 1 -- 172.21.15.162:0/3876416426 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1fc0aedb0 con 0x7fa1fc0b30e0 2020-11-28T21:56:22.574 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.554+0000 7fa1f97fa700 1 -- 172.21.15.162:0/3876416426 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa1e401c460 con 0x7fa1fc0b30e0 2020-11-28T21:56:22.575 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.554+0000 7fa2015a2700 1 -- 172.21.15.162:0/3876416426 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1fc151fe0 con 0x7fa1fc0b30e0 2020-11-28T21:56:22.575 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.554+0000 7fa1f97fa700 1 --2- 172.21.15.162:0/3876416426 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa1d4055440 0x7fa1d4057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:22.575 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.554+0000 7fa1f97fa700 1 -- 172.21.15.162:0/3876416426 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fa1e406dbd0 con 0x7fa1fc0b30e0 2020-11-28T21:56:22.575 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.554+0000 7fa1fbfff700 1 --2- 172.21.15.162:0/3876416426 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa1d4055440 0x7fa1d4057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:22.575 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.554+0000 7fa1fbfff700 1 --2- 172.21.15.162:0/3876416426 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa1d4055440 0x7fa1d4057900 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7fa1ec008000 tx=0x7fa1ec009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:22.576 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.558+0000 7fa1f97fa700 1 -- 172.21.15.162:0/3876416426 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa1e4044010 con 0x7fa1fc0b30e0 2020-11-28T21:56:22.701 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.694+0000 7fa2015a2700 1 -- 172.21.15.162:0/3876416426 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa1fc057050 con 0x7fa1fc0b30e0 2020-11-28T21:56:22.703 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.698+0000 7fa1f97fa700 1 -- 172.21.15.162:0/3876416426 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v154) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa1e4047630 con 0x7fa1fc0b30e0 2020-11-28T21:56:22.703 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:56:22.707 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.698+0000 7fa2015a2700 1 -- 172.21.15.162:0/3876416426 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa1d4055440 msgr2=0x7fa1d4057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:22.708 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.698+0000 7fa2015a2700 1 --2- 172.21.15.162:0/3876416426 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa1d4055440 0x7fa1d4057900 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7fa1ec008000 tx=0x7fa1ec009040).stop 2020-11-28T21:56:22.708 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.698+0000 7fa2015a2700 1 -- 172.21.15.162:0/3876416426 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1fc0b30e0 msgr2=0x7fa1fc1522c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:22.708 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.698+0000 7fa2015a2700 1 --2- 172.21.15.162:0/3876416426 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1fc0b30e0 0x7fa1fc1522c0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 rx=0x7fa1e400a000 tx=0x7fa1e4008040).stop 2020-11-28T21:56:22.708 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.698+0000 7fa2015a2700 1 -- 172.21.15.162:0/3876416426 shutdown_connections 2020-11-28T21:56:22.709 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.698+0000 7fa2015a2700 1 --2- 172.21.15.162:0/3876416426 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa1d4055440 0x7fa1d4057900 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:22.709 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.698+0000 7fa2015a2700 1 --2- 172.21.15.162:0/3876416426 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa1fc0b2d00 0x7fa1fc145820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:22.709 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.698+0000 7fa2015a2700 1 --2- 172.21.15.162:0/3876416426 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa1fc0b3ab0 0x7fa1fc0acf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:22.709 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.698+0000 7fa2015a2700 1 --2- 172.21.15.162:0/3876416426 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1fc0b30e0 0x7fa1fc1522c0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:22.710 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.698+0000 7fa2015a2700 1 -- 172.21.15.162:0/3876416426 >> 172.21.15.162:0/3876416426 conn(0x7fa1fc0604c0 msgr2=0x7fa1fc13e510 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:22.710 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.698+0000 7fa2015a2700 1 -- 172.21.15.162:0/3876416426 shutdown_connections 2020-11-28T21:56:22.710 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.698+0000 7fa2015a2700 1 -- 172.21.15.162:0/3876416426 wait complete. 2020-11-28T21:56:22.716 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2020-11-28T21:56:22.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.818+0000 7f89722e4700 1 Processor -- start 2020-11-28T21:56:22.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.818+0000 7f89722e4700 1 -- start start 2020-11-28T21:56:22.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.818+0000 7f89722e4700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f896c0b5a70 0x7f896c143360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:22.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.818+0000 7f89722e4700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f896c0b5e50 0x7f896c0b6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:22.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.818+0000 7f89722e4700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f896c0b5430 0x7f896c09d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:22.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.818+0000 7f89722e4700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f896c0b8040 con 0x7f896c0b5e50 2020-11-28T21:56:22.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.818+0000 7f89722e4700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f896c0b8720 con 0x7f896c0b5a70 2020-11-28T21:56:22.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.818+0000 7f89722e4700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f896c142980 con 0x7f896c0b5430 2020-11-28T21:56:22.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.822+0000 7f89712e2700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f896c0b5a70 0x7f896c143360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:22.834 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.822+0000 7f89712e2700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f896c0b5a70 0x7f896c143360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46226/0 (socket says 172.21.15.162:46226) 2020-11-28T21:56:22.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.822+0000 7f89712e2700 1 -- 172.21.15.162:0/975053353 learned_addr learned my addr 172.21.15.162:0/975053353 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:22.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.822+0000 7f8970ae1700 1 --2- 172.21.15.162:0/975053353 >> v2:172.21.15.118:3300/0 conn(0x7f896c0b5e50 0x7f896c0b6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:22.835 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.822+0000 7f8971ae3700 1 --2- 172.21.15.162:0/975053353 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f896c0b5430 0x7f896c09d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:22.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.822+0000 7f89712e2700 1 -- 172.21.15.162:0/975053353 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f896c0b5430 msgr2=0x7f896c09d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:22.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.822+0000 7f89712e2700 1 --2- 172.21.15.162:0/975053353 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f896c0b5430 0x7f896c09d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:22.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.822+0000 7f89712e2700 1 -- 172.21.15.162:0/975053353 >> v2:172.21.15.118:3300/0 conn(0x7f896c0b5e50 msgr2=0x7f896c0b6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:22.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.822+0000 7f89712e2700 1 --2- 172.21.15.162:0/975053353 >> v2:172.21.15.118:3300/0 conn(0x7f896c0b5e50 0x7f896c0b6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:22.838 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.822+0000 7f89712e2700 1 -- 172.21.15.162:0/975053353 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f896c142c20 con 0x7f896c0b5a70 2020-11-28T21:56:22.838 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.822+0000 7f89712e2700 1 --2- 172.21.15.162:0/975053353 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f896c0b5a70 0x7f896c143360 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 rx=0x7f8968002f30 tx=0x7f8968006e80).ready entity=mon.1 client_cookie=11e05f7bfbb861d5 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:22.838 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.822+0000 7f8963fff700 1 -- 172.21.15.162:0/975053353 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89680097f0 con 0x7f896c0b5a70 2020-11-28T21:56:22.839 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.822+0000 7f8963fff700 1 -- 172.21.15.162:0/975053353 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8968009950 con 0x7f896c0b5a70 2020-11-28T21:56:22.839 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.822+0000 7f8963fff700 1 -- 172.21.15.162:0/975053353 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8968009cb0 con 0x7f896c0b5a70 2020-11-28T21:56:22.840 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.822+0000 7f89722e4700 1 -- 172.21.15.162:0/975053353 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f896c0b5a70 msgr2=0x7f896c143360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:22.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.822+0000 7f89722e4700 1 --2- 172.21.15.162:0/975053353 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f896c0b5a70 0x7f896c143360 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 rx=0x7f8968002f30 tx=0x7f8968006e80).stop 2020-11-28T21:56:22.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.822+0000 7f89722e4700 1 -- 172.21.15.162:0/975053353 shutdown_connections 2020-11-28T21:56:22.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.822+0000 7f89722e4700 1 --2- 172.21.15.162:0/975053353 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f896c0b5430 0x7f896c09d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:22.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.822+0000 7f89722e4700 1 --2- 172.21.15.162:0/975053353 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f896c0b5a70 0x7f896c143360 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:22.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.822+0000 7f89722e4700 1 --2- 172.21.15.162:0/975053353 >> v2:172.21.15.118:3300/0 conn(0x7f896c0b5e50 0x7f896c0b6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:22.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.822+0000 7f89722e4700 1 -- 172.21.15.162:0/975053353 >> 172.21.15.162:0/975053353 conn(0x7f896c05b970 msgr2=0x7f896c05bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:22.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.822+0000 7f89722e4700 1 -- 172.21.15.162:0/975053353 shutdown_connections 2020-11-28T21:56:22.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.822+0000 7f89722e4700 1 -- 172.21.15.162:0/975053353 wait complete. 2020-11-28T21:56:22.843 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.822+0000 7f89722e4700 1 Processor -- start 2020-11-28T21:56:22.843 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.822+0000 7f89722e4700 1 -- start start 2020-11-28T21:56:22.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.826+0000 7f89722e4700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f896c0b5430 0x7f896c09ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:22.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.826+0000 7f89722e4700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f896c0b5a70 0x7f896c146210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:22.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.826+0000 7f89722e4700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f896c0b5e50 0x7f896c146700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:22.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.826+0000 7f89722e4700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f896c146bf0 con 0x7f896c0b5430 2020-11-28T21:56:22.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.826+0000 7f89722e4700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f896c1523c0 con 0x7f896c0b5e50 2020-11-28T21:56:22.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.826+0000 7f89722e4700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f896c0b07b0 con 0x7f896c0b5a70 2020-11-28T21:56:22.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.826+0000 7f8971ae3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f896c0b5e50 0x7f896c146700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:22.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.826+0000 7f8971ae3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f896c0b5e50 0x7f896c146700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46234/0 (socket says 172.21.15.162:46234) 2020-11-28T21:56:22.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.826+0000 7f8971ae3700 1 -- 172.21.15.162:0/847953785 learned_addr learned my addr 172.21.15.162:0/847953785 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:22.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.826+0000 7f8970ae1700 1 --2- 172.21.15.162:0/847953785 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f896c0b5a70 0x7f896c146210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:22.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.826+0000 7f89712e2700 1 --2- 172.21.15.162:0/847953785 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f896c0b5430 0x7f896c09ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:22.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.826+0000 7f8971ae3700 1 -- 172.21.15.162:0/847953785 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f896c0b5a70 msgr2=0x7f896c146210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:22.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.826+0000 7f8971ae3700 1 --2- 172.21.15.162:0/847953785 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f896c0b5a70 0x7f896c146210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:22.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.826+0000 7f8971ae3700 1 -- 172.21.15.162:0/847953785 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f896c0b5430 msgr2=0x7f896c09ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:22.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.826+0000 7f8971ae3700 1 --2- 172.21.15.162:0/847953785 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f896c0b5430 0x7f896c09ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:22.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.826+0000 7f8971ae3700 1 -- 172.21.15.162:0/847953785 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f896c143c70 con 0x7f896c0b5e50 2020-11-28T21:56:22.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.826+0000 7f89712e2700 1 --2- 172.21.15.162:0/847953785 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f896c0b5430 0x7f896c09ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:56:22.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.826+0000 7f8971ae3700 1 --2- 172.21.15.162:0/847953785 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f896c0b5e50 0x7f896c146700 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 rx=0x7f896c0b6b20 tx=0x7f8964008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:22.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.826+0000 7f89627fc700 1 -- 172.21.15.162:0/847953785 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f896400bad0 con 0x7f896c0b5e50 2020-11-28T21:56:22.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.826+0000 7f89627fc700 1 -- 172.21.15.162:0/847953785 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f896400bc30 con 0x7f896c0b5e50 2020-11-28T21:56:22.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.826+0000 7f89627fc700 1 -- 172.21.15.162:0/847953785 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8964012420 con 0x7f896c0b5e50 2020-11-28T21:56:22.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.826+0000 7f89722e4700 1 -- 172.21.15.162:0/847953785 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f896c140480 con 0x7f896c0b5e50 2020-11-28T21:56:22.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.826+0000 7f89722e4700 1 -- 172.21.15.162:0/847953785 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f896c152870 con 0x7f896c0b5e50 2020-11-28T21:56:22.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.826+0000 7f89722e4700 1 -- 172.21.15.162:0/847953785 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f896c143e00 con 0x7f896c0b5e50 2020-11-28T21:56:22.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.826+0000 7f89627fc700 1 -- 172.21.15.162:0/847953785 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f8964012580 con 0x7f896c0b5e50 2020-11-28T21:56:22.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.826+0000 7f89627fc700 1 --2- 172.21.15.162:0/847953785 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8954055500 0x7f89540579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:22.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.826+0000 7f89627fc700 1 -- 172.21.15.162:0/847953785 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f896406e360 con 0x7f896c0b5e50 2020-11-28T21:56:22.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.826+0000 7f89712e2700 1 --2- 172.21.15.162:0/847953785 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8954055500 0x7f89540579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:22.852 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.830+0000 7f89712e2700 1 --2- 172.21.15.162:0/847953785 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8954055500 0x7f89540579c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f8968017000 tx=0x7f8968015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:22.852 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.834+0000 7f89627fc700 1 -- 172.21.15.162:0/847953785 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f896c143e00 con 0x7f896c0b5e50 2020-11-28T21:56:22.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.974+0000 7f89722e4700 1 -- 172.21.15.162:0/847953785 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f896c143e00 con 0x7f896c0b5e50 2020-11-28T21:56:22.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.974+0000 7f89627fc700 1 -- 172.21.15.162:0/847953785 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v154) v1 ==== 84+0+143 (secure 0 0 0) 0x7f896c143e00 con 0x7f896c0b5e50 2020-11-28T21:56:22.980 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:56:22.980 INFO:teuthology.orchestra.run.smithi162.stdout:[ 2020-11-28T21:56:22.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:22.981 INFO:teuthology.orchestra.run.smithi162.stdout: "poolnum": 1, 2020-11-28T21:56:22.981 INFO:teuthology.orchestra.run.smithi162.stdout: "poolname": "device_health_metrics" 2020-11-28T21:56:22.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:22.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:22.982 INFO:teuthology.orchestra.run.smithi162.stdout: "poolnum": 2, 2020-11-28T21:56:22.982 INFO:teuthology.orchestra.run.smithi162.stdout: "poolname": "rbd" 2020-11-28T21:56:22.982 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:22.982 INFO:teuthology.orchestra.run.smithi162.stdout:] 2020-11-28T21:56:22.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.978+0000 7f89722e4700 1 -- 172.21.15.162:0/847953785 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8954055500 msgr2=0x7f89540579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:22.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.978+0000 7f89722e4700 1 --2- 172.21.15.162:0/847953785 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8954055500 0x7f89540579c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f8968017000 tx=0x7f8968015000).stop 2020-11-28T21:56:22.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.978+0000 7f89722e4700 1 -- 172.21.15.162:0/847953785 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f896c0b5e50 msgr2=0x7f896c146700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:22.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.978+0000 7f89722e4700 1 --2- 172.21.15.162:0/847953785 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f896c0b5e50 0x7f896c146700 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 rx=0x7f896c0b6b20 tx=0x7f8964008040).stop 2020-11-28T21:56:22.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.978+0000 7f89722e4700 1 -- 172.21.15.162:0/847953785 shutdown_connections 2020-11-28T21:56:22.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.978+0000 7f89722e4700 1 --2- 172.21.15.162:0/847953785 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f8954055500 0x7f89540579c0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:22.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.978+0000 7f89722e4700 1 --2- 172.21.15.162:0/847953785 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f896c0b5e50 0x7f896c146700 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:22.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.978+0000 7f89722e4700 1 --2- 172.21.15.162:0/847953785 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f896c0b5430 0x7f896c09ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:22.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.978+0000 7f89722e4700 1 --2- 172.21.15.162:0/847953785 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f896c0b5a70 0x7f896c146210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:22.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.978+0000 7f89722e4700 1 -- 172.21.15.162:0/847953785 >> 172.21.15.162:0/847953785 conn(0x7f896c05b970 msgr2=0x7f896c143260 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:22.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.978+0000 7f89722e4700 1 -- 172.21.15.162:0/847953785 shutdown_connections 2020-11-28T21:56:22.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:22.978+0000 7f89722e4700 1 -- 172.21.15.162:0/847953785 wait complete. 2020-11-28T21:56:23.000 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-11-28T21:56:23.000 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2020-11-28T21:56:23.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.114+0000 7f91d0a2c700 1 Processor -- start 2020-11-28T21:56:23.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.114+0000 7f91d0a2c700 1 -- start start 2020-11-28T21:56:23.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.114+0000 7f91d0a2c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f91cc05ccc0 0x7f91cc13fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:23.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.114+0000 7f91d0a2c700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f91cc0b1200 0x7f91cc1401d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:23.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.114+0000 7f91d0a2c700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91cc061630 con 0x7f91cc0ac240 2020-11-28T21:56:23.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.114+0000 7f91d0a2c700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91cc05d0a0 con 0x7f91cc05ccc0 2020-11-28T21:56:23.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.114+0000 7f91d0a2c700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc0aa710 con 0x7f91cc0b1200 2020-11-28T21:56:23.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.114+0000 7f91ca59c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f91cc05ccc0 0x7f91cc13fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:23.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.114+0000 7f91ca59c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f91cc05ccc0 0x7f91cc13fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46238/0 (socket says 172.21.15.162:46238) 2020-11-28T21:56:23.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.114+0000 7f91ca59c700 1 -- 172.21.15.162:0/413238327 learned_addr learned my addr 172.21.15.162:0/413238327 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:23.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.114+0000 7f91c3fff700 1 --2- 172.21.15.162:0/413238327 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f91cc0b1200 0x7f91cc1401d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:23.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.114+0000 7f91ca59c700 1 -- 172.21.15.162:0/413238327 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f91cc0b1200 msgr2=0x7f91cc1401d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:23.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.114+0000 7f91ca59c700 1 --2- 172.21.15.162:0/413238327 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f91cc0b1200 0x7f91cc1401d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:23.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.114+0000 7f91ca59c700 1 -- 172.21.15.162:0/413238327 >> v1:172.21.15.118:6789/0 conn(0x7f91cc0ac240 legacy=0x7f91cc142d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:23.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.114+0000 7f91ca59c700 1 -- 172.21.15.162:0/413238327 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91cc061860 con 0x7f91cc05ccc0 2020-11-28T21:56:23.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.114+0000 7f91ca59c700 1 --2- 172.21.15.162:0/413238327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91cc05ccc0 0x7f91cc13fce0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 rx=0x7f91bc002f30 tx=0x7f91bc006e80).ready entity=mon.1 client_cookie=b0264fe705f90fa6 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:23.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.114+0000 7f91c9d9b700 1 -- 172.21.15.162:0/413238327 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91bc0097f0 con 0x7f91cc05ccc0 2020-11-28T21:56:23.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.118+0000 7f91c9d9b700 1 -- 172.21.15.162:0/413238327 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91bc009950 con 0x7f91cc05ccc0 2020-11-28T21:56:23.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.118+0000 7f91c9d9b700 1 -- 172.21.15.162:0/413238327 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91bc009cb0 con 0x7f91cc05ccc0 2020-11-28T21:56:23.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.118+0000 7f91d0a2c700 1 -- 172.21.15.162:0/413238327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91cc05ccc0 msgr2=0x7f91cc13fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:23.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.118+0000 7f91d0a2c700 1 --2- 172.21.15.162:0/413238327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91cc05ccc0 0x7f91cc13fce0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 rx=0x7f91bc002f30 tx=0x7f91bc006e80).stop 2020-11-28T21:56:23.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.118+0000 7f91d0a2c700 1 -- 172.21.15.162:0/413238327 shutdown_connections 2020-11-28T21:56:23.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.118+0000 7f91d0a2c700 1 --2- 172.21.15.162:0/413238327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91cc05ccc0 0x7f91cc13fce0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:23.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.118+0000 7f91d0a2c700 1 --2- 172.21.15.162:0/413238327 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f91cc0b1200 0x7f91cc1401d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:23.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.118+0000 7f91d0a2c700 1 -- 172.21.15.162:0/413238327 >> 172.21.15.162:0/413238327 conn(0x7f91cc0b6220 msgr2=0x7f91cc0b21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:23.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.118+0000 7f91d0a2c700 1 -- 172.21.15.162:0/413238327 shutdown_connections 2020-11-28T21:56:23.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.118+0000 7f91d0a2c700 1 -- 172.21.15.162:0/413238327 wait complete. 2020-11-28T21:56:23.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.118+0000 7f91d0a2c700 1 Processor -- start 2020-11-28T21:56:23.128 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.118+0000 7f91d0a2c700 1 -- start start 2020-11-28T21:56:23.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.122+0000 7f91d0a2c700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f91cc05ccc0 0x7f91cc099750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:23.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.122+0000 7f91d0a2c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91cc0ac240 0x7f91cc0acd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:23.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.122+0000 7f91d0a2c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f91cc0b1200 0x7f91cc0ae910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:23.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.122+0000 7f91d0a2c700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc0ad720 con 0x7f91cc0b1200 2020-11-28T21:56:23.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.122+0000 7f91d0a2c700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc14bfc0 con 0x7f91cc0ac240 2020-11-28T21:56:23.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.122+0000 7f91d0a2c700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc14ef40 con 0x7f91cc05ccc0 2020-11-28T21:56:23.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.122+0000 7f91c3fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91cc0ac240 0x7f91cc0acd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:23.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.122+0000 7f91c3fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91cc0ac240 0x7f91cc0acd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46246/0 (socket says 172.21.15.162:46246) 2020-11-28T21:56:23.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.122+0000 7f91c3fff700 1 -- 172.21.15.162:0/2024309432 learned_addr learned my addr 172.21.15.162:0/2024309432 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:23.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.122+0000 7f91cad9d700 1 --2- 172.21.15.162:0/2024309432 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f91cc0b1200 0x7f91cc0ae910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:23.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.122+0000 7f91ca59c700 1 --2- 172.21.15.162:0/2024309432 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f91cc05ccc0 0x7f91cc099750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:23.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.122+0000 7f91c3fff700 1 -- 172.21.15.162:0/2024309432 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f91cc05ccc0 msgr2=0x7f91cc099750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:23.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.122+0000 7f91c3fff700 1 --2- 172.21.15.162:0/2024309432 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f91cc05ccc0 0x7f91cc099750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:23.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.122+0000 7f91c3fff700 1 -- 172.21.15.162:0/2024309432 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f91cc0b1200 msgr2=0x7f91cc0ae910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:23.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.122+0000 7f91c3fff700 1 --2- 172.21.15.162:0/2024309432 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f91cc0b1200 0x7f91cc0ae910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:23.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.122+0000 7f91c3fff700 1 -- 172.21.15.162:0/2024309432 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91cc0ad950 con 0x7f91cc0ac240 2020-11-28T21:56:23.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.122+0000 7f91cad9d700 1 --2- 172.21.15.162:0/2024309432 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f91cc0b1200 0x7f91cc0ae910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:56:23.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.122+0000 7f91ca59c700 1 --2- 172.21.15.162:0/2024309432 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f91cc05ccc0 0x7f91cc099750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:56:23.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.122+0000 7f91c3fff700 1 --2- 172.21.15.162:0/2024309432 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91cc0ac240 0x7f91cc0acd70 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 rx=0x7f91cc14e370 tx=0x7f91b4008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:23.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.122+0000 7f91c37fe700 1 -- 172.21.15.162:0/2024309432 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91b4009980 con 0x7f91cc0ac240 2020-11-28T21:56:23.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.122+0000 7f91c37fe700 1 -- 172.21.15.162:0/2024309432 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91b4009ae0 con 0x7f91cc0ac240 2020-11-28T21:56:23.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.122+0000 7f91c37fe700 1 -- 172.21.15.162:0/2024309432 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91b4009e40 con 0x7f91cc0ac240 2020-11-28T21:56:23.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.122+0000 7f91d0a2c700 1 -- 172.21.15.162:0/2024309432 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91cc14f1e0 con 0x7f91cc0ac240 2020-11-28T21:56:23.135 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.122+0000 7f91d0a2c700 1 -- 172.21.15.162:0/2024309432 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91cc0af370 con 0x7f91cc0ac240 2020-11-28T21:56:23.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.126+0000 7f91d0a2c700 1 -- 172.21.15.162:0/2024309432 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91cc14ecf0 con 0x7f91cc0ac240 2020-11-28T21:56:23.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.126+0000 7f91c37fe700 1 -- 172.21.15.162:0/2024309432 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f91b4013480 con 0x7f91cc0ac240 2020-11-28T21:56:23.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.126+0000 7f91c37fe700 1 --2- 172.21.15.162:0/2024309432 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f91b0055440 0x7f91b0057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:23.136 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.126+0000 7f91c37fe700 1 -- 172.21.15.162:0/2024309432 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f91b406e4a0 con 0x7f91cc0ac240 2020-11-28T21:56:23.139 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.126+0000 7f91ca59c700 1 --2- 172.21.15.162:0/2024309432 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f91b0055440 0x7f91b0057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:23.139 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.126+0000 7f91ca59c700 1 --2- 172.21.15.162:0/2024309432 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f91b0055440 0x7f91b0057900 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f91bc01a000 tx=0x7f91bc015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:23.139 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.130+0000 7f91c37fe700 1 -- 172.21.15.162:0/2024309432 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f91b40448e0 con 0x7f91cc0ac240 2020-11-28T21:56:23.275 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:23.270+0000 7f91d0a2c700 1 -- 172.21.15.162:0/2024309432 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f91cc0af140 con 0x7f91cc0ac240 2020-11-28T21:56:24.291 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.286+0000 7f91c37fe700 1 -- 172.21.15.162:0/2024309432 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v155) v1 ==== 133+0+0 (secure 0 0 0) 0x7f91b4047f00 con 0x7f91cc0ac240 2020-11-28T21:56:24.294 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.290+0000 7f91d0a2c700 1 -- 172.21.15.162:0/2024309432 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f91b0055440 msgr2=0x7f91b0057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:24.295 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.290+0000 7f91d0a2c700 1 --2- 172.21.15.162:0/2024309432 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f91b0055440 0x7f91b0057900 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f91bc01a000 tx=0x7f91bc015000).stop 2020-11-28T21:56:24.295 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.290+0000 7f91d0a2c700 1 -- 172.21.15.162:0/2024309432 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91cc0ac240 msgr2=0x7f91cc0acd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:24.295 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.290+0000 7f91d0a2c700 1 --2- 172.21.15.162:0/2024309432 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91cc0ac240 0x7f91cc0acd70 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 rx=0x7f91cc14e370 tx=0x7f91b4008040).stop 2020-11-28T21:56:24.296 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.290+0000 7f91d0a2c700 1 -- 172.21.15.162:0/2024309432 shutdown_connections 2020-11-28T21:56:24.296 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.290+0000 7f91d0a2c700 1 --2- 172.21.15.162:0/2024309432 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f91b0055440 0x7f91b0057900 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:24.296 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.290+0000 7f91d0a2c700 1 --2- 172.21.15.162:0/2024309432 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f91cc05ccc0 0x7f91cc099750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:24.296 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.290+0000 7f91d0a2c700 1 --2- 172.21.15.162:0/2024309432 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f91cc0b1200 0x7f91cc0ae910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:24.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.290+0000 7f91d0a2c700 1 --2- 172.21.15.162:0/2024309432 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91cc0ac240 0x7f91cc0acd70 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:24.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.290+0000 7f91d0a2c700 1 -- 172.21.15.162:0/2024309432 >> 172.21.15.162:0/2024309432 conn(0x7f91cc0b6220 msgr2=0x7f91cc0aa880 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:24.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.290+0000 7f91d0a2c700 1 -- 172.21.15.162:0/2024309432 shutdown_connections 2020-11-28T21:56:24.297 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.290+0000 7f91d0a2c700 1 -- 172.21.15.162:0/2024309432 wait complete. 2020-11-28T21:56:24.298 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_metadata' created 2020-11-28T21:56:24.309 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2020-11-28T21:56:24.424 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.418+0000 7f2ed68b4700 1 Processor -- start 2020-11-28T21:56:24.424 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.418+0000 7f2ed68b4700 1 -- start start 2020-11-28T21:56:24.427 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.422+0000 7f2ed68b4700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f2ed005cca0 0x7f2ed00b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:24.428 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.422+0000 7f2ed68b4700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ed00b1360 con 0x7f2ed00b23a0 2020-11-28T21:56:24.428 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.422+0000 7f2ed68b4700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ed00ae8c0 con 0x7f2ed013cb50 2020-11-28T21:56:24.428 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.422+0000 7f2ed68b4700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ed00add60 con 0x7f2ed005cca0 2020-11-28T21:56:24.428 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.422+0000 7f2ed58b2700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f2ed013cb50 0x7f2ed0140180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:53096/0 (socket says 172.21.15.162:53096) 2020-11-28T21:56:24.429 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.422+0000 7f2ed58b2700 1 -- 172.21.15.162:0/3935234269 learned_addr learned my addr 172.21.15.162:0/3935234269 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:56:24.429 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.422+0000 7f2ed50b1700 1 --2- 172.21.15.162:0/3935234269 >> v2:172.21.15.118:3300/0 conn(0x7f2ed005cca0 0x7f2ed00b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:24.429 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.422+0000 7f2ed48b0700 1 -- 172.21.15.162:0/3935234269 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1150044009 0 0) 0x7f2ed00ae8c0 con 0x7f2ed013cb50 2020-11-28T21:56:24.429 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.422+0000 7f2ed48b0700 1 -- 172.21.15.162:0/3935234269 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2eb8003230 con 0x7f2ed013cb50 2020-11-28T21:56:24.430 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.422+0000 7f2ed48b0700 1 -- 172.21.15.162:0/3935234269 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 487732565 0 0) 0x7f2eb8003230 con 0x7f2ed013cb50 2020-11-28T21:56:24.430 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.422+0000 7f2ed48b0700 1 -- 172.21.15.162:0/3935234269 >> v2:172.21.15.118:3300/0 conn(0x7f2ed005cca0 msgr2=0x7f2ed00b1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:24.430 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.422+0000 7f2ed48b0700 1 --2- 172.21.15.162:0/3935234269 >> v2:172.21.15.118:3300/0 conn(0x7f2ed005cca0 0x7f2ed00b1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:24.430 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.422+0000 7f2ed48b0700 1 -- 172.21.15.162:0/3935234269 >> v1:172.21.15.118:6789/0 conn(0x7f2ed00b23a0 legacy=0x7f2ed00ac350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:24.430 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.422+0000 7f2ed48b0700 1 -- 172.21.15.162:0/3935234269 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed00aeb60 con 0x7f2ed013cb50 2020-11-28T21:56:24.431 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.422+0000 7f2ed48b0700 1 -- 172.21.15.162:0/3935234269 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f2ec80016d0 con 0x7f2ed013cb50 2020-11-28T21:56:24.431 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.422+0000 7f2ed48b0700 1 -- 172.21.15.162:0/3935234269 >> v1:172.21.15.162:6789/0 conn(0x7f2ed013cb50 legacy=0x7f2ed0140180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:24.431 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.422+0000 7f2ed48b0700 1 --2- 172.21.15.162:0/3935234269 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2eb8003770 0x7f2eb8003b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:24.431 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.422+0000 7f2ed48b0700 1 --2- 172.21.15.162:0/3935234269 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2eb80044d0 0x7f2eb8004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:24.432 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.422+0000 7f2ed48b0700 1 --2- 172.21.15.162:0/3935234269 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2eb8004e40 0x7f2eb800b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:24.432 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.422+0000 7f2ed48b0700 1 -- 172.21.15.162:0/3935234269 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb80052f0 con 0x7f2eb8003770 2020-11-28T21:56:24.432 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.422+0000 7f2ed48b0700 1 -- 172.21.15.162:0/3935234269 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb800bc10 con 0x7f2eb8004e40 2020-11-28T21:56:24.432 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.422+0000 7f2ed48b0700 1 -- 172.21.15.162:0/3935234269 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb800bd50 con 0x7f2eb80044d0 2020-11-28T21:56:24.432 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.422+0000 7f2ed58b2700 1 --2- 172.21.15.162:0/3935234269 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2eb8003770 0x7f2eb8003b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:24.433 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.422+0000 7f2ed60b3700 1 --2- 172.21.15.162:0/3935234269 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2eb8004e40 0x7f2eb800b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:24.433 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.422+0000 7f2ed50b1700 1 --2- 172.21.15.162:0/3935234269 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2eb80044d0 0x7f2eb8004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:24.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.422+0000 7f2ed60b3700 1 -- 172.21.15.162:0/3935234269 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2eb80044d0 msgr2=0x7f2eb8004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:24.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.422+0000 7f2ed60b3700 1 --2- 172.21.15.162:0/3935234269 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2eb80044d0 0x7f2eb8004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:24.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.422+0000 7f2ed60b3700 1 -- 172.21.15.162:0/3935234269 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2eb8003770 msgr2=0x7f2eb8003b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:24.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.422+0000 7f2ed60b3700 1 --2- 172.21.15.162:0/3935234269 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2eb8003770 0x7f2eb8003b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:24.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.422+0000 7f2ed60b3700 1 -- 172.21.15.162:0/3935234269 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb8003230 con 0x7f2eb8004e40 2020-11-28T21:56:24.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.422+0000 7f2ed60b3700 1 -- 172.21.15.162:0/3935234269 reap_dead start 2020-11-28T21:56:24.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.422+0000 7f2ed60b3700 1 --2- 172.21.15.162:0/3935234269 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2eb8004e40 0x7f2eb800b720 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 rx=0x7f2ed005ad70 tx=0x7f2ec0007b60).ready entity=mon.1 client_cookie=304984f314497c12 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:24.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.422+0000 7f2ed48b0700 1 -- 172.21.15.162:0/3935234269 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2ec000c870 con 0x7f2eb8004e40 2020-11-28T21:56:24.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.422+0000 7f2ed48b0700 1 -- 172.21.15.162:0/3935234269 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ec000c9d0 con 0x7f2eb8004e40 2020-11-28T21:56:24.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.422+0000 7f2ed48b0700 1 -- 172.21.15.162:0/3935234269 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2ec000cd30 con 0x7f2eb8004e40 2020-11-28T21:56:24.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.422+0000 7f2ed68b4700 1 -- 172.21.15.162:0/3935234269 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2eb8004e40 msgr2=0x7f2eb800b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:24.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.422+0000 7f2ed68b4700 1 --2- 172.21.15.162:0/3935234269 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2eb8004e40 0x7f2eb800b720 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 rx=0x7f2ed005ad70 tx=0x7f2ec0007b60).stop 2020-11-28T21:56:24.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.426+0000 7f2ed68b4700 1 -- 172.21.15.162:0/3935234269 shutdown_connections 2020-11-28T21:56:24.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.426+0000 7f2ed68b4700 1 --2- 172.21.15.162:0/3935234269 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2eb8004e40 0x7f2eb800b720 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:24.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.426+0000 7f2ed68b4700 1 -- 172.21.15.162:0/3935234269 >> 172.21.15.162:0/3935234269 conn(0x7f2ed00b57d0 msgr2=0x7f2ed00b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:24.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.426+0000 7f2ed68b4700 1 -- 172.21.15.162:0/3935234269 shutdown_connections 2020-11-28T21:56:24.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.426+0000 7f2ed68b4700 1 -- 172.21.15.162:0/3935234269 wait complete. 2020-11-28T21:56:24.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.426+0000 7f2ed68b4700 1 Processor -- start 2020-11-28T21:56:24.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.426+0000 7f2ed68b4700 1 -- start start 2020-11-28T21:56:24.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.426+0000 7f2ed68b4700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2eb8004e40 0x7f2ed0140c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:24.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.426+0000 7f2ed68b4700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2ed00a8c10 0x7f2ed0152240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:24.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.426+0000 7f2ed68b4700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ed0137600 0x7f2ed0136010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:24.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.426+0000 7f2ed68b4700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed00b46b0 con 0x7f2ed00a8c10 2020-11-28T21:56:24.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.426+0000 7f2ed68b4700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed0143a30 con 0x7f2ed0137600 2020-11-28T21:56:24.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.426+0000 7f2ed68b4700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed00a9220 con 0x7f2eb8004e40 2020-11-28T21:56:24.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.426+0000 7f2ed58b2700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2eb8004e40 0x7f2ed0140c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:24.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.426+0000 7f2ed60b3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ed0137600 0x7f2ed0136010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:24.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.426+0000 7f2ed58b2700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2eb8004e40 0x7f2ed0140c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:43488/0 (socket says 172.21.15.162:43488) 2020-11-28T21:56:24.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.426+0000 7f2ed58b2700 1 -- 172.21.15.162:0/873169493 learned_addr learned my addr 172.21.15.162:0/873169493 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:24.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.426+0000 7f2ed50b1700 1 --2- 172.21.15.162:0/873169493 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2ed00a8c10 0x7f2ed0152240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:24.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.426+0000 7f2ed58b2700 1 -- 172.21.15.162:0/873169493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ed0137600 msgr2=0x7f2ed0136010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:24.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.426+0000 7f2ed58b2700 1 --2- 172.21.15.162:0/873169493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ed0137600 0x7f2ed0136010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:24.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.426+0000 7f2ed58b2700 1 -- 172.21.15.162:0/873169493 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2ed00a8c10 msgr2=0x7f2ed0152240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:24.444 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.426+0000 7f2ed58b2700 1 --2- 172.21.15.162:0/873169493 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2ed00a8c10 0x7f2ed0152240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:24.444 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.426+0000 7f2ed58b2700 1 -- 172.21.15.162:0/873169493 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed00abc00 con 0x7f2eb8004e40 2020-11-28T21:56:24.444 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.426+0000 7f2ed50b1700 1 --2- 172.21.15.162:0/873169493 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2ed00a8c10 0x7f2ed0152240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:56:24.444 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.426+0000 7f2ed58b2700 1 --2- 172.21.15.162:0/873169493 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2eb8004e40 0x7f2ed0140c10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f2ec80010d0 tx=0x7f2ed00af0a0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:24.444 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.426+0000 7f2ebeffd700 1 -- 172.21.15.162:0/873169493 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2ec800fa30 con 0x7f2eb8004e40 2020-11-28T21:56:24.445 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.426+0000 7f2ed68b4700 1 -- 172.21.15.162:0/873169493 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ed0137ab0 con 0x7f2eb8004e40 2020-11-28T21:56:24.445 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.426+0000 7f2ebeffd700 1 -- 172.21.15.162:0/873169493 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ec800fb90 con 0x7f2eb8004e40 2020-11-28T21:56:24.445 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.426+0000 7f2ebeffd700 1 -- 172.21.15.162:0/873169493 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2ec8012350 con 0x7f2eb8004e40 2020-11-28T21:56:24.445 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.426+0000 7f2ed68b4700 1 -- 172.21.15.162:0/873169493 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ed01366a0 con 0x7f2eb8004e40 2020-11-28T21:56:24.445 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.430+0000 7f2ed68b4700 1 -- 172.21.15.162:0/873169493 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ed00a9530 con 0x7f2eb8004e40 2020-11-28T21:56:24.446 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.430+0000 7f2ebeffd700 1 -- 172.21.15.162:0/873169493 <== mon.2 v2:172.21.15.118:3301/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2ec80124b0 con 0x7f2eb8004e40 2020-11-28T21:56:24.446 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.430+0000 7f2ebeffd700 1 --2- 172.21.15.162:0/873169493 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2eb00554e0 0x7f2eb00579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:24.446 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.430+0000 7f2ebeffd700 1 -- 172.21.15.162:0/873169493 <== mon.2 v2:172.21.15.118:3301/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f2ec806d5c0 con 0x7f2eb8004e40 2020-11-28T21:56:24.446 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.430+0000 7f2ed50b1700 1 --2- 172.21.15.162:0/873169493 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2eb00554e0 0x7f2eb00579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:24.447 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.430+0000 7f2ed50b1700 1 --2- 172.21.15.162:0/873169493 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2eb00554e0 0x7f2eb00579a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7f2ecc00b000 tx=0x7f2ecc0091a0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:24.447 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.442+0000 7f2ebeffd700 1 -- 172.21.15.162:0/873169493 <== mon.2 v2:172.21.15.118:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f2ed00a9530 con 0x7f2eb8004e40 2020-11-28T21:56:24.594 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:24.590+0000 7f2ed68b4700 1 -- 172.21.15.162:0/873169493 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f2ed00a9530 con 0x7f2eb8004e40 2020-11-28T21:56:25.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.486+0000 7f2ebeffd700 1 -- 172.21.15.162:0/873169493 <== mon.2 v2:172.21.15.118:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v157) v1 ==== 125+0+0 (secure 0 0 0) 0x7f2ed00a9530 con 0x7f2eb8004e40 2020-11-28T21:56:25.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.490+0000 7f2ed68b4700 1 -- 172.21.15.162:0/873169493 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2eb00554e0 msgr2=0x7f2eb00579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:25.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.490+0000 7f2ed68b4700 1 --2- 172.21.15.162:0/873169493 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2eb00554e0 0x7f2eb00579a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7f2ecc00b000 tx=0x7f2ecc0091a0).stop 2020-11-28T21:56:25.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.490+0000 7f2ed68b4700 1 -- 172.21.15.162:0/873169493 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2eb8004e40 msgr2=0x7f2ed0140c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:25.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.490+0000 7f2ed68b4700 1 --2- 172.21.15.162:0/873169493 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2eb8004e40 0x7f2ed0140c10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f2ec80010d0 tx=0x7f2ed00af0a0).stop 2020-11-28T21:56:25.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.490+0000 7f2ed68b4700 1 -- 172.21.15.162:0/873169493 shutdown_connections 2020-11-28T21:56:25.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.490+0000 7f2ed68b4700 1 --2- 172.21.15.162:0/873169493 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2eb00554e0 0x7f2eb00579a0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:25.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.490+0000 7f2ed68b4700 1 --2- 172.21.15.162:0/873169493 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2eb8004e40 0x7f2ed0140c10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:25.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.490+0000 7f2ed68b4700 1 --2- 172.21.15.162:0/873169493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ed0137600 0x7f2ed0136010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:25.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.490+0000 7f2ed68b4700 1 --2- 172.21.15.162:0/873169493 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2ed00a8c10 0x7f2ed0152240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:25.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.490+0000 7f2ed68b4700 1 -- 172.21.15.162:0/873169493 >> 172.21.15.162:0/873169493 conn(0x7f2ed00b57d0 msgr2=0x7f2ed00ac270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:25.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.490+0000 7f2ed68b4700 1 -- 172.21.15.162:0/873169493 shutdown_connections 2020-11-28T21:56:25.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.490+0000 7f2ed68b4700 1 -- 172.21.15.162:0/873169493 wait complete. 2020-11-28T21:56:25.498 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_data' created 2020-11-28T21:56:25.512 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2020-11-28T21:56:25.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.622+0000 7f225bfcb700 1 Processor -- start 2020-11-28T21:56:25.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.622+0000 7f225bfcb700 1 -- start start 2020-11-28T21:56:25.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.622+0000 7f225bfcb700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f22540b3d60 0x7f22540b3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:25.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.622+0000 7f225bfcb700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f225409a7f0 0x7f225409aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:25.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.622+0000 7f225bfcb700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f225414a930 0x7f225414ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:25.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.622+0000 7f225bfcb700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22540b4140 con 0x7f22540b3d60 2020-11-28T21:56:25.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.622+0000 7f225bfcb700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2254066160 con 0x7f225409a7f0 2020-11-28T21:56:25.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.622+0000 7f225bfcb700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2254066400 con 0x7f225414a930 2020-11-28T21:56:25.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.622+0000 7f2259566700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f225409a7f0 0x7f225409aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:25.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.622+0000 7f2259566700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f225409a7f0 0x7f225409aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46276/0 (socket says 172.21.15.162:46276) 2020-11-28T21:56:25.629 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.622+0000 7f2259566700 1 -- 172.21.15.162:0/2759776090 learned_addr learned my addr 172.21.15.162:0/2759776090 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:25.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.622+0000 7f225a568700 1 --2- 172.21.15.162:0/2759776090 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f225414a930 0x7f225414ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:25.630 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.622+0000 7f2259d67700 1 --2- 172.21.15.162:0/2759776090 >> v2:172.21.15.118:3300/0 conn(0x7f22540b3d60 0x7f22540b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:25.631 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.622+0000 7f2259566700 1 -- 172.21.15.162:0/2759776090 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f225414a930 msgr2=0x7f225414ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:25.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.622+0000 7f2259566700 1 --2- 172.21.15.162:0/2759776090 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f225414a930 0x7f225414ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:25.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.622+0000 7f2259566700 1 -- 172.21.15.162:0/2759776090 >> v2:172.21.15.118:3300/0 conn(0x7f22540b3d60 msgr2=0x7f22540b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:25.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.622+0000 7f2259566700 1 --2- 172.21.15.162:0/2759776090 >> v2:172.21.15.118:3300/0 conn(0x7f22540b3d60 0x7f22540b3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:25.632 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.622+0000 7f2259566700 1 -- 172.21.15.162:0/2759776090 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22540666a0 con 0x7f225409a7f0 2020-11-28T21:56:25.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.622+0000 7f2259566700 1 --2- 172.21.15.162:0/2759776090 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f225409a7f0 0x7f225409aca0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 rx=0x7f2244002f30 tx=0x7f2244006e80).ready entity=mon.1 client_cookie=a77765988183208d server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:25.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.622+0000 7f2258d65700 1 -- 172.21.15.162:0/2759776090 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2244009640 con 0x7f225409a7f0 2020-11-28T21:56:25.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.622+0000 7f2258d65700 1 -- 172.21.15.162:0/2759776090 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22440097a0 con 0x7f225409a7f0 2020-11-28T21:56:25.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.622+0000 7f2258d65700 1 -- 172.21.15.162:0/2759776090 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2244009b00 con 0x7f225409a7f0 2020-11-28T21:56:25.633 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.622+0000 7f225bfcb700 1 -- 172.21.15.162:0/2759776090 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f225409a7f0 msgr2=0x7f225409aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:25.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.622+0000 7f225bfcb700 1 --2- 172.21.15.162:0/2759776090 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f225409a7f0 0x7f225409aca0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 rx=0x7f2244002f30 tx=0x7f2244006e80).stop 2020-11-28T21:56:25.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.626+0000 7f225bfcb700 1 -- 172.21.15.162:0/2759776090 shutdown_connections 2020-11-28T21:56:25.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.626+0000 7f225bfcb700 1 --2- 172.21.15.162:0/2759776090 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f225414a930 0x7f225414ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:25.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.626+0000 7f225bfcb700 1 --2- 172.21.15.162:0/2759776090 >> v2:172.21.15.118:3300/0 conn(0x7f22540b3d60 0x7f22540b3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:25.634 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.626+0000 7f225bfcb700 1 --2- 172.21.15.162:0/2759776090 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f225409a7f0 0x7f225409aca0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:25.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.626+0000 7f225bfcb700 1 -- 172.21.15.162:0/2759776090 >> 172.21.15.162:0/2759776090 conn(0x7f225405b970 msgr2=0x7f225405bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:25.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.626+0000 7f225bfcb700 1 -- 172.21.15.162:0/2759776090 shutdown_connections 2020-11-28T21:56:25.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.626+0000 7f225bfcb700 1 -- 172.21.15.162:0/2759776090 wait complete. 2020-11-28T21:56:25.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.626+0000 7f225bfcb700 1 Processor -- start 2020-11-28T21:56:25.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.626+0000 7f225bfcb700 1 -- start start 2020-11-28T21:56:25.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.626+0000 7f225bfcb700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f225409a7f0 0x7f22540abdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:25.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.626+0000 7f225bfcb700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f22540b3d60 0x7f2254146fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:25.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.626+0000 7f225bfcb700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f225414a930 0x7f22540acfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:25.639 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.626+0000 7f225bfcb700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2254143b70 con 0x7f225414a930 2020-11-28T21:56:25.639 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.626+0000 7f225bfcb700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22540618e0 con 0x7f225409a7f0 2020-11-28T21:56:25.639 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.626+0000 7f225bfcb700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22540ac930 con 0x7f22540b3d60 2020-11-28T21:56:25.639 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.626+0000 7f2259d67700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f225409a7f0 0x7f22540abdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:25.639 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.626+0000 7f2259d67700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f225409a7f0 0x7f22540abdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46280/0 (socket says 172.21.15.162:46280) 2020-11-28T21:56:25.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.626+0000 7f2259d67700 1 -- 172.21.15.162:0/342283926 learned_addr learned my addr 172.21.15.162:0/342283926 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:25.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.626+0000 7f2259566700 1 --2- 172.21.15.162:0/342283926 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f22540b3d60 0x7f2254146fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:25.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.626+0000 7f225a568700 1 --2- 172.21.15.162:0/342283926 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f225414a930 0x7f22540acfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:25.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.626+0000 7f2259d67700 1 -- 172.21.15.162:0/342283926 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f22540b3d60 msgr2=0x7f2254146fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:25.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.626+0000 7f2259d67700 1 --2- 172.21.15.162:0/342283926 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f22540b3d60 0x7f2254146fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:25.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.626+0000 7f2259d67700 1 -- 172.21.15.162:0/342283926 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f225414a930 msgr2=0x7f22540acfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:25.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.626+0000 7f2259d67700 1 --2- 172.21.15.162:0/342283926 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f225414a930 0x7f22540acfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:25.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.626+0000 7f2259d67700 1 -- 172.21.15.162:0/342283926 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2254061d00 con 0x7f225409a7f0 2020-11-28T21:56:25.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.626+0000 7f225a568700 1 --2- 172.21.15.162:0/342283926 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f225414a930 0x7f22540acfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:56:25.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.630+0000 7f2259d67700 1 --2- 172.21.15.162:0/342283926 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f225409a7f0 0x7f22540abdc0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 rx=0x7f225000c000 tx=0x7f2250006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:25.642 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.630+0000 7f224affd700 1 -- 172.21.15.162:0/342283926 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f225000ca80 con 0x7f225409a7f0 2020-11-28T21:56:25.642 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.630+0000 7f224affd700 1 -- 172.21.15.162:0/342283926 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f225000cbe0 con 0x7f225409a7f0 2020-11-28T21:56:25.642 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.630+0000 7f225bfcb700 1 -- 172.21.15.162:0/342283926 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2254061fa0 con 0x7f225409a7f0 2020-11-28T21:56:25.642 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.630+0000 7f224affd700 1 -- 172.21.15.162:0/342283926 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f22500123f0 con 0x7f225409a7f0 2020-11-28T21:56:25.643 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.630+0000 7f225bfcb700 1 -- 172.21.15.162:0/342283926 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2254060d50 con 0x7f225409a7f0 2020-11-28T21:56:25.643 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.630+0000 7f225bfcb700 1 -- 172.21.15.162:0/342283926 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22540ac6f0 con 0x7f225409a7f0 2020-11-28T21:56:25.643 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.630+0000 7f224affd700 1 -- 172.21.15.162:0/342283926 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2250012550 con 0x7f225409a7f0 2020-11-28T21:56:25.643 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.630+0000 7f224affd700 1 --2- 172.21.15.162:0/342283926 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f223c0554e0 0x7f223c0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:25.644 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.634+0000 7f224affd700 1 -- 172.21.15.162:0/342283926 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5957+0+0 (secure 0 0 0) 0x7f225006e3a0 con 0x7f225409a7f0 2020-11-28T21:56:25.644 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.634+0000 7f2259566700 1 --2- 172.21.15.162:0/342283926 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f223c0554e0 0x7f223c0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:25.644 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.634+0000 7f2259566700 1 --2- 172.21.15.162:0/342283926 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f223c0554e0 0x7f223c0579a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f2244019000 tx=0x7f2244015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:25.644 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.638+0000 7f224affd700 1 -- 172.21.15.162:0/342283926 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f22540ac6f0 con 0x7f225409a7f0 2020-11-28T21:56:25.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.822+0000 7f225bfcb700 1 -- 172.21.15.162:0/342283926 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f22540ac6f0 con 0x7f225409a7f0 2020-11-28T21:56:25.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.838+0000 7f224affd700 1 -- 172.21.15.162:0/342283926 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 29 and data pool 30 v79) v1 ==== 189+0+0 (secure 0 0 0) 0x7f22540ac6f0 con 0x7f225409a7f0 2020-11-28T21:56:25.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.842+0000 7f225bfcb700 1 -- 172.21.15.162:0/342283926 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f223c0554e0 msgr2=0x7f223c0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:25.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.842+0000 7f225bfcb700 1 --2- 172.21.15.162:0/342283926 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f223c0554e0 0x7f223c0579a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f2244019000 tx=0x7f2244015000).stop 2020-11-28T21:56:25.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.842+0000 7f225bfcb700 1 -- 172.21.15.162:0/342283926 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f225409a7f0 msgr2=0x7f22540abdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:25.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.842+0000 7f225bfcb700 1 --2- 172.21.15.162:0/342283926 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f225409a7f0 0x7f22540abdc0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 rx=0x7f225000c000 tx=0x7f2250006dc0).stop 2020-11-28T21:56:25.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.842+0000 7f225bfcb700 1 -- 172.21.15.162:0/342283926 shutdown_connections 2020-11-28T21:56:25.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.842+0000 7f225bfcb700 1 --2- 172.21.15.162:0/342283926 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f223c0554e0 0x7f223c0579a0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:25.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.842+0000 7f225bfcb700 1 --2- 172.21.15.162:0/342283926 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f225414a930 0x7f22540acfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:25.852 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.842+0000 7f225bfcb700 1 --2- 172.21.15.162:0/342283926 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f225409a7f0 0x7f22540abdc0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:25.852 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.842+0000 7f225bfcb700 1 --2- 172.21.15.162:0/342283926 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f22540b3d60 0x7f2254146fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:25.852 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.842+0000 7f225bfcb700 1 -- 172.21.15.162:0/342283926 >> 172.21.15.162:0/342283926 conn(0x7f225405b970 msgr2=0x7f22541482f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:25.853 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.846+0000 7f225bfcb700 1 -- 172.21.15.162:0/342283926 shutdown_connections 2020-11-28T21:56:25.854 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.850+0000 7f225bfcb700 1 -- 172.21.15.162:0/342283926 wait complete. 2020-11-28T21:56:25.856 INFO:teuthology.orchestra.run.smithi162.stderr:new fs with metadata pool 29 and data pool 30 2020-11-28T21:56:25.867 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:56:25.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.966+0000 7fb130955700 1 Processor -- start 2020-11-28T21:56:25.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.966+0000 7fb130955700 1 -- start start 2020-11-28T21:56:25.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.966+0000 7fb130955700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fb12c0b4720 0x7fb12c0b0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:25.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.966+0000 7fb130955700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb12c0b1c50 con 0x7fb12c0b1870 2020-11-28T21:56:25.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.966+0000 7fb130955700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb12c060a60 con 0x7fb12c0b0d20 2020-11-28T21:56:25.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.966+0000 7fb130955700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb12c0615a0 con 0x7fb12c0b4720 2020-11-28T21:56:25.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.966+0000 7fb12bfff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fb12c0b0d20 0x7fb12c14cff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:53144/0 (socket says 172.21.15.162:53144) 2020-11-28T21:56:25.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.966+0000 7fb12bfff700 1 -- 172.21.15.162:0/1640429146 learned_addr learned my addr 172.21.15.162:0/1640429146 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:56:25.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.966+0000 7fb122ffd700 1 --2- 172.21.15.162:0/1640429146 >> v2:172.21.15.118:3300/0 conn(0x7fb12c0b4720 0x7fb12c0b0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:25.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.966+0000 7fb12affd700 1 -- 172.21.15.162:0/1640429146 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3586235853 0 0) 0x7fb12c060a60 con 0x7fb12c0b0d20 2020-11-28T21:56:25.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.966+0000 7fb12affd700 1 -- 172.21.15.162:0/1640429146 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb10c003230 con 0x7fb12c0b0d20 2020-11-28T21:56:25.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.966+0000 7fb122ffd700 1 -- 172.21.15.162:0/1640429146 >> v1:172.21.15.162:6789/0 conn(0x7fb12c0b0d20 legacy=0x7fb12c14cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:25.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.966+0000 7fb12affd700 1 -- 172.21.15.162:0/1640429146 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2676939344 0 0) 0x7fb12c0b1c50 con 0x7fb12c0b1870 2020-11-28T21:56:25.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.966+0000 7fb122ffd700 1 -- 172.21.15.162:0/1640429146 >> v1:172.21.15.118:6789/0 conn(0x7fb12c0b1870 legacy=0x7fb12c142f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:25.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.966+0000 7fb122ffd700 1 -- 172.21.15.162:0/1640429146 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb12c143660 con 0x7fb12c0b4720 2020-11-28T21:56:25.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.966+0000 7fb122ffd700 1 --2- 172.21.15.162:0/1640429146 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb12c0b4720 0x7fb12c0b0830 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7fb114002f30 tx=0x7fb114006dc0).ready entity=mon.0 client_cookie=bd489047d8d15c29 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:25.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.966+0000 7fb12affd700 1 -- 172.21.15.162:0/1640429146 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb114009800 con 0x7fb12c0b4720 2020-11-28T21:56:25.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.966+0000 7fb12affd700 1 -- 172.21.15.162:0/1640429146 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb114009960 con 0x7fb12c0b4720 2020-11-28T21:56:25.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.966+0000 7fb12affd700 1 -- 172.21.15.162:0/1640429146 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb114009cc0 con 0x7fb12c0b4720 2020-11-28T21:56:25.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.966+0000 7fb130955700 1 -- 172.21.15.162:0/1640429146 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb12c0b4720 msgr2=0x7fb12c0b0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:25.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.966+0000 7fb130955700 1 --2- 172.21.15.162:0/1640429146 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb12c0b4720 0x7fb12c0b0830 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7fb114002f30 tx=0x7fb114006dc0).stop 2020-11-28T21:56:25.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.970+0000 7fb130955700 1 -- 172.21.15.162:0/1640429146 shutdown_connections 2020-11-28T21:56:25.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.970+0000 7fb130955700 1 --2- 172.21.15.162:0/1640429146 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb12c0b4720 0x7fb12c0b0830 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:25.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.970+0000 7fb130955700 1 -- 172.21.15.162:0/1640429146 >> 172.21.15.162:0/1640429146 conn(0x7fb12c0b54f0 msgr2=0x7fb12c0b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:25.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.970+0000 7fb130955700 1 -- 172.21.15.162:0/1640429146 shutdown_connections 2020-11-28T21:56:25.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.970+0000 7fb130955700 1 -- 172.21.15.162:0/1640429146 wait complete. 2020-11-28T21:56:25.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.970+0000 7fb130955700 1 Processor -- start 2020-11-28T21:56:25.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.970+0000 7fb130955700 1 -- start start 2020-11-28T21:56:25.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.970+0000 7fb130955700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb12c0b0d20 0x7fb12c0988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:25.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.970+0000 7fb130955700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb12c0b1870 0x7fb12c13d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:25.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.970+0000 7fb130955700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb12c0b4720 0x7fb12c13dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:25.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.970+0000 7fb130955700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12c151550 con 0x7fb12c0b0d20 2020-11-28T21:56:25.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.970+0000 7fb130955700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12c09b390 con 0x7fb12c0b4720 2020-11-28T21:56:25.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.970+0000 7fb130955700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb12c0aeb80 con 0x7fb12c0b1870 2020-11-28T21:56:25.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.970+0000 7fb12bfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb12c0b4720 0x7fb12c13dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:25.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.970+0000 7fb12bfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb12c0b4720 0x7fb12c13dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46306/0 (socket says 172.21.15.162:46306) 2020-11-28T21:56:25.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.970+0000 7fb12bfff700 1 -- 172.21.15.162:0/1911781349 learned_addr learned my addr 172.21.15.162:0/1911781349 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:25.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.970+0000 7fb122ffd700 1 --2- 172.21.15.162:0/1911781349 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb12c0b1870 0x7fb12c13d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:25.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.970+0000 7fb12b7fe700 1 --2- 172.21.15.162:0/1911781349 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb12c0b0d20 0x7fb12c0988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:25.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.970+0000 7fb12bfff700 1 -- 172.21.15.162:0/1911781349 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb12c0b1870 msgr2=0x7fb12c13d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:25.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.970+0000 7fb12bfff700 1 --2- 172.21.15.162:0/1911781349 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb12c0b1870 0x7fb12c13d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:25.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.970+0000 7fb12bfff700 1 -- 172.21.15.162:0/1911781349 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb12c0b0d20 msgr2=0x7fb12c0988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:25.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.970+0000 7fb12bfff700 1 --2- 172.21.15.162:0/1911781349 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb12c0b0d20 0x7fb12c0988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:25.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.970+0000 7fb12bfff700 1 -- 172.21.15.162:0/1911781349 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb12c0af380 con 0x7fb12c0b4720 2020-11-28T21:56:25.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.970+0000 7fb12b7fe700 1 --2- 172.21.15.162:0/1911781349 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb12c0b0d20 0x7fb12c0988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:56:25.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.970+0000 7fb122ffd700 1 --2- 172.21.15.162:0/1911781349 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb12c0b1870 0x7fb12c13d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:56:25.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.970+0000 7fb12bfff700 1 --2- 172.21.15.162:0/1911781349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb12c0b4720 0x7fb12c13dea0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 rx=0x7fb124001450 tx=0x7fb124007d90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:25.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.970+0000 7fb1297fa700 1 -- 172.21.15.162:0/1911781349 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb124005b70 con 0x7fb12c0b4720 2020-11-28T21:56:25.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.970+0000 7fb130955700 1 -- 172.21.15.162:0/1911781349 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb12c098dc0 con 0x7fb12c0b4720 2020-11-28T21:56:25.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.970+0000 7fb1297fa700 1 -- 172.21.15.162:0/1911781349 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb124005cd0 con 0x7fb12c0b4720 2020-11-28T21:56:25.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.970+0000 7fb1297fa700 1 -- 172.21.15.162:0/1911781349 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb124007300 con 0x7fb12c0b4720 2020-11-28T21:56:25.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.970+0000 7fb130955700 1 -- 172.21.15.162:0/1911781349 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb12c0acb20 con 0x7fb12c0b4720 2020-11-28T21:56:25.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.974+0000 7fb130955700 1 -- 172.21.15.162:0/1911781349 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb12c057050 con 0x7fb12c0b4720 2020-11-28T21:56:25.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.974+0000 7fb1297fa700 1 -- 172.21.15.162:0/1911781349 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fb124010030 con 0x7fb12c0b4720 2020-11-28T21:56:25.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.974+0000 7fb1297fa700 1 --2- 172.21.15.162:0/1911781349 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb110055500 0x7fb1100579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:25.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.974+0000 7fb1297fa700 1 -- 172.21.15.162:0/1911781349 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb12406e640 con 0x7fb12c0b4720 2020-11-28T21:56:25.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.974+0000 7fb12b7fe700 1 --2- 172.21.15.162:0/1911781349 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb110055500 0x7fb1100579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:25.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.974+0000 7fb12b7fe700 1 --2- 172.21.15.162:0/1911781349 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb110055500 0x7fb1100579c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7fb11c00b000 tx=0x7fb11c009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:25.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:25.978+0000 7fb1297fa700 1 -- 172.21.15.162:0/1911781349 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fb124044940 con 0x7fb12c0b4720 2020-11-28T21:56:26.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.118+0000 7fb130955700 1 -- 172.21.15.162:0/1911781349 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb12c061a60 con 0x7fb12c0b4720 2020-11-28T21:56:26.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.122+0000 7fb1297fa700 1 -- 172.21.15.162:0/1911781349 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) v1 ==== 74+0+16174 (secure 0 0 0) 0x7fb124047f60 con 0x7fb12c0b4720 2020-11-28T21:56:26.127 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:56:26.127 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":158,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:56:25.830648+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:56:23.335325+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2020-11-28T21:56:24.648832+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:56:26.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.122+0000 7fb130955700 1 -- 172.21.15.162:0/1911781349 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb110055500 msgr2=0x7fb1100579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:26.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.122+0000 7fb130955700 1 --2- 172.21.15.162:0/1911781349 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb110055500 0x7fb1100579c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7fb11c00b000 tx=0x7fb11c009040).stop 2020-11-28T21:56:26.130 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.122+0000 7fb130955700 1 -- 172.21.15.162:0/1911781349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb12c0b4720 msgr2=0x7fb12c13dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:26.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.122+0000 7fb130955700 1 --2- 172.21.15.162:0/1911781349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb12c0b4720 0x7fb12c13dea0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 rx=0x7fb124001450 tx=0x7fb124007d90).stop 2020-11-28T21:56:26.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.122+0000 7fb130955700 1 -- 172.21.15.162:0/1911781349 shutdown_connections 2020-11-28T21:56:26.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.122+0000 7fb130955700 1 --2- 172.21.15.162:0/1911781349 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fb110055500 0x7fb1100579c0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:26.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.122+0000 7fb130955700 1 --2- 172.21.15.162:0/1911781349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb12c0b4720 0x7fb12c13dea0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:26.132 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.122+0000 7fb130955700 1 --2- 172.21.15.162:0/1911781349 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb12c0b0d20 0x7fb12c0988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:26.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.122+0000 7fb130955700 1 --2- 172.21.15.162:0/1911781349 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb12c0b1870 0x7fb12c13d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:26.133 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.122+0000 7fb130955700 1 -- 172.21.15.162:0/1911781349 >> 172.21.15.162:0/1911781349 conn(0x7fb12c0b54f0 msgr2=0x7fb12c0abc10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:26.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.122+0000 7fb130955700 1 -- 172.21.15.162:0/1911781349 shutdown_connections 2020-11-28T21:56:26.134 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.122+0000 7fb130955700 1 -- 172.21.15.162:0/1911781349 wait complete. 2020-11-28T21:56:26.141 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:56:26.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.246+0000 7f68fa34e700 1 Processor -- start 2020-11-28T21:56:26.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.246+0000 7f68fa34e700 1 -- start start 2020-11-28T21:56:26.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.246+0000 7f68fa34e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f68f40ac790 0x7f68f4142bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:26.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.246+0000 7f68fa34e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f68f40acca0 0x7f68f40b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:26.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.246+0000 7f68fa34e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68f40b07e0 con 0x7f68f40ac3b0 2020-11-28T21:56:26.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.246+0000 7f68fa34e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68f40abe90 con 0x7f68f40ac790 2020-11-28T21:56:26.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.246+0000 7f68fa34e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68f40b0a80 con 0x7f68f40acca0 2020-11-28T21:56:26.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.246+0000 7f68f9b4d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f68f40acca0 0x7f68f40b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:26.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.246+0000 7f68f9b4d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f68f40acca0 0x7f68f40b2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46314/0 (socket says 172.21.15.162:46314) 2020-11-28T21:56:26.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.246+0000 7f68f9b4d700 1 -- 172.21.15.162:0/1152828465 learned_addr learned my addr 172.21.15.162:0/1152828465 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:26.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.246+0000 7f68f8b4b700 1 --2- 172.21.15.162:0/1152828465 >> v2:172.21.15.118:3300/0 conn(0x7f68f40ac790 0x7f68f4142bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:26.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.246+0000 7f68f9b4d700 1 -- 172.21.15.162:0/1152828465 >> v2:172.21.15.118:3300/0 conn(0x7f68f40ac790 msgr2=0x7f68f4142bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:26.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.246+0000 7f68f9b4d700 1 --2- 172.21.15.162:0/1152828465 >> v2:172.21.15.118:3300/0 conn(0x7f68f40ac790 0x7f68f4142bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:26.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.246+0000 7f68f9b4d700 1 -- 172.21.15.162:0/1152828465 >> v1:172.21.15.118:6789/0 conn(0x7f68f40ac3b0 legacy=0x7f68f40ab780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:26.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.250+0000 7f68f9b4d700 1 -- 172.21.15.162:0/1152828465 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f40b0d20 con 0x7f68f40acca0 2020-11-28T21:56:26.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.250+0000 7f68f8b4b700 1 --2- 172.21.15.162:0/1152828465 >> v2:172.21.15.118:3300/0 conn(0x7f68f40ac790 0x7f68f4142bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:56:26.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.250+0000 7f68f9b4d700 1 --2- 172.21.15.162:0/1152828465 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68f40acca0 0x7f68f40b2f50 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 rx=0x7f68ec0032b0 tx=0x7f68ec009270).ready entity=mon.1 client_cookie=cc84ad3f8304f521 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:26.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.250+0000 7f68ebfff700 1 -- 172.21.15.162:0/1152828465 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68ec00bc00 con 0x7f68f40acca0 2020-11-28T21:56:26.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.250+0000 7f68ebfff700 1 -- 172.21.15.162:0/1152828465 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68ec00bd60 con 0x7f68f40acca0 2020-11-28T21:56:26.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.250+0000 7f68ebfff700 1 -- 172.21.15.162:0/1152828465 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68ec011400 con 0x7f68f40acca0 2020-11-28T21:56:26.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.250+0000 7f68fa34e700 1 -- 172.21.15.162:0/1152828465 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68f40acca0 msgr2=0x7f68f40b2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:26.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.250+0000 7f68fa34e700 1 --2- 172.21.15.162:0/1152828465 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68f40acca0 0x7f68f40b2f50 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 rx=0x7f68ec0032b0 tx=0x7f68ec009270).stop 2020-11-28T21:56:26.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.250+0000 7f68fa34e700 1 -- 172.21.15.162:0/1152828465 shutdown_connections 2020-11-28T21:56:26.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.250+0000 7f68fa34e700 1 --2- 172.21.15.162:0/1152828465 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68f40acca0 0x7f68f40b2f50 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:26.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.250+0000 7f68fa34e700 1 --2- 172.21.15.162:0/1152828465 >> v2:172.21.15.118:3300/0 conn(0x7f68f40ac790 0x7f68f4142bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:26.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.250+0000 7f68fa34e700 1 -- 172.21.15.162:0/1152828465 >> 172.21.15.162:0/1152828465 conn(0x7f68f40b6870 msgr2=0x7f68f40b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:26.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.250+0000 7f68fa34e700 1 -- 172.21.15.162:0/1152828465 shutdown_connections 2020-11-28T21:56:26.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.250+0000 7f68fa34e700 1 -- 172.21.15.162:0/1152828465 wait complete. 2020-11-28T21:56:26.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.250+0000 7f68fa34e700 1 Processor -- start 2020-11-28T21:56:26.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.250+0000 7f68fa34e700 1 -- start start 2020-11-28T21:56:26.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.250+0000 7f68fa34e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68f40ac3b0 0x7f68f40afab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:26.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.250+0000 7f68fa34e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f68f40ac790 0x7f68f4064460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:26.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.250+0000 7f68fa34e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f68f40acca0 0x7f68f4064950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:26.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.250+0000 7f68fa34e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f4144f50 con 0x7f68f40acca0 2020-11-28T21:56:26.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.250+0000 7f68fa34e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f40659c0 con 0x7f68f40ac3b0 2020-11-28T21:56:26.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.250+0000 7f68fa34e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f40664d0 con 0x7f68f40ac790 2020-11-28T21:56:26.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.250+0000 7f68f934c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68f40ac3b0 0x7f68f40afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:26.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.250+0000 7f68f934c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68f40ac3b0 0x7f68f40afab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46320/0 (socket says 172.21.15.162:46320) 2020-11-28T21:56:26.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.250+0000 7f68f934c700 1 -- 172.21.15.162:0/3146309698 learned_addr learned my addr 172.21.15.162:0/3146309698 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:26.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.250+0000 7f68f9b4d700 1 --2- 172.21.15.162:0/3146309698 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f68f40acca0 0x7f68f4064950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:26.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.250+0000 7f68f8b4b700 1 --2- 172.21.15.162:0/3146309698 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f68f40ac790 0x7f68f4064460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:26.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.250+0000 7f68f934c700 1 -- 172.21.15.162:0/3146309698 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f68f40ac790 msgr2=0x7f68f4064460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:26.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.250+0000 7f68f934c700 1 --2- 172.21.15.162:0/3146309698 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f68f40ac790 0x7f68f4064460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:26.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.250+0000 7f68f934c700 1 -- 172.21.15.162:0/3146309698 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f68f40acca0 msgr2=0x7f68f4064950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:26.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.250+0000 7f68f934c700 1 --2- 172.21.15.162:0/3146309698 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f68f40acca0 0x7f68f4064950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:26.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.250+0000 7f68f934c700 1 -- 172.21.15.162:0/3146309698 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f4066a50 con 0x7f68f40ac3b0 2020-11-28T21:56:26.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.250+0000 7f68f934c700 1 --2- 172.21.15.162:0/3146309698 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68f40ac3b0 0x7f68f40afab0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 rx=0x7f68f414a470 tx=0x7f68f0007be0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:26.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.254+0000 7f68ea7fc700 1 -- 172.21.15.162:0/3146309698 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68f00072a0 con 0x7f68f40ac3b0 2020-11-28T21:56:26.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.254+0000 7f68ea7fc700 1 -- 172.21.15.162:0/3146309698 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f00064e0 con 0x7f68f40ac3b0 2020-11-28T21:56:26.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.254+0000 7f68fa34e700 1 -- 172.21.15.162:0/3146309698 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68f4066cf0 con 0x7f68f40ac3b0 2020-11-28T21:56:26.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.254+0000 7f68ea7fc700 1 -- 172.21.15.162:0/3146309698 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68f0006840 con 0x7f68f40ac3b0 2020-11-28T21:56:26.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.254+0000 7f68fa34e700 1 -- 172.21.15.162:0/3146309698 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68f4146910 con 0x7f68f40ac3b0 2020-11-28T21:56:26.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.254+0000 7f68fa34e700 1 -- 172.21.15.162:0/3146309698 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68f4066e80 con 0x7f68f40ac3b0 2020-11-28T21:56:26.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.254+0000 7f68ea7fc700 1 -- 172.21.15.162:0/3146309698 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f68f0019020 con 0x7f68f40ac3b0 2020-11-28T21:56:26.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.254+0000 7f68ea7fc700 1 --2- 172.21.15.162:0/3146309698 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f68dc055330 0x7f68dc0577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:26.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.254+0000 7f68ea7fc700 1 -- 172.21.15.162:0/3146309698 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f68f006d7c0 con 0x7f68f40ac3b0 2020-11-28T21:56:26.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.254+0000 7f68f8b4b700 1 --2- 172.21.15.162:0/3146309698 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f68dc055330 0x7f68dc0577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:26.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.254+0000 7f68f8b4b700 1 --2- 172.21.15.162:0/3146309698 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f68dc055330 0x7f68dc0577f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f68e400c000 tx=0x7f68e4008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:26.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.266+0000 7f68ea7fc700 1 -- 172.21.15.162:0/3146309698 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f68f0043a20 con 0x7f68f40ac3b0 2020-11-28T21:56:26.414 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.410+0000 7f68fa34e700 1 -- 172.21.15.162:0/3146309698 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68f4057050 con 0x7f68f40ac3b0 2020-11-28T21:56:26.416 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.410+0000 7f68ea7fc700 1 -- 172.21.15.162:0/3146309698 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) v1 ==== 74+0+16174 (secure 0 0 0) 0x7f68f0047040 con 0x7f68f40ac3b0 2020-11-28T21:56:26.416 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:56:26.417 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":158,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:56:25.830648+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:56:23.335325+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2020-11-28T21:56:24.648832+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:56:26.418 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.414+0000 7f68fa34e700 1 -- 172.21.15.162:0/3146309698 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f68dc055330 msgr2=0x7f68dc0577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:26.418 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.414+0000 7f68fa34e700 1 --2- 172.21.15.162:0/3146309698 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f68dc055330 0x7f68dc0577f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f68e400c000 tx=0x7f68e4008040).stop 2020-11-28T21:56:26.418 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.414+0000 7f68fa34e700 1 -- 172.21.15.162:0/3146309698 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68f40ac3b0 msgr2=0x7f68f40afab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:26.419 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.414+0000 7f68fa34e700 1 --2- 172.21.15.162:0/3146309698 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68f40ac3b0 0x7f68f40afab0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 rx=0x7f68f414a470 tx=0x7f68f0007be0).stop 2020-11-28T21:56:26.419 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.414+0000 7f68fa34e700 1 -- 172.21.15.162:0/3146309698 shutdown_connections 2020-11-28T21:56:26.419 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.414+0000 7f68fa34e700 1 --2- 172.21.15.162:0/3146309698 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f68dc055330 0x7f68dc0577f0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:26.419 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.414+0000 7f68fa34e700 1 --2- 172.21.15.162:0/3146309698 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f68f40acca0 0x7f68f4064950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:26.420 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.414+0000 7f68fa34e700 1 --2- 172.21.15.162:0/3146309698 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68f40ac3b0 0x7f68f40afab0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:26.420 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.414+0000 7f68fa34e700 1 --2- 172.21.15.162:0/3146309698 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f68f40ac790 0x7f68f4064460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:26.420 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.414+0000 7f68fa34e700 1 -- 172.21.15.162:0/3146309698 >> 172.21.15.162:0/3146309698 conn(0x7f68f40b6870 msgr2=0x7f68f40b5270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:26.420 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.414+0000 7f68fa34e700 1 -- 172.21.15.162:0/3146309698 shutdown_connections 2020-11-28T21:56:26.420 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.414+0000 7f68fa34e700 1 -- 172.21.15.162:0/3146309698 wait complete. 2020-11-28T21:56:26.429 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2020-11-28T21:56:26.543 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.538+0000 7f4f6ee23700 1 Processor -- start 2020-11-28T21:56:26.543 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.538+0000 7f4f6ee23700 1 -- start start 2020-11-28T21:56:26.544 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.538+0000 7f4f6ee23700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4f680b2d00 0x7f4f6805c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:26.544 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.538+0000 7f4f6ee23700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f4f680b3ab0 0x7f4f680b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:26.545 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.538+0000 7f4f6ee23700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f6805baf0 con 0x7f4f68142ad0 2020-11-28T21:56:26.545 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.538+0000 7f4f6ee23700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f680b0e20 con 0x7f4f680b3ab0 2020-11-28T21:56:26.545 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.538+0000 7f4f6ee23700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f6805bd90 con 0x7f4f680b2d00 2020-11-28T21:56:26.545 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.538+0000 7f4f6e622700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f4f68142ad0 0x7f4f6813fa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:53172/0 (socket says 172.21.15.162:53172) 2020-11-28T21:56:26.545 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.538+0000 7f4f6de21700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4f680b2d00 0x7f4f6805c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:26.546 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.538+0000 7f4f6e622700 1 -- 172.21.15.162:0/3940532865 learned_addr learned my addr 172.21.15.162:0/3940532865 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:56:26.546 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.538+0000 7f4f6d620700 1 --2- 172.21.15.162:0/3940532865 >> v2:172.21.15.118:3300/0 conn(0x7f4f680b3ab0 0x7f4f680b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:26.548 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.538+0000 7f4f6ce1f700 1 -- 172.21.15.162:0/3940532865 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2496318285 0 0) 0x7f4f6805baf0 con 0x7f4f68142ad0 2020-11-28T21:56:26.548 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.538+0000 7f4f6ce1f700 1 -- 172.21.15.162:0/3940532865 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f54003230 con 0x7f4f68142ad0 2020-11-28T21:56:26.548 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.538+0000 7f4f6de21700 1 -- 172.21.15.162:0/3940532865 >> v2:172.21.15.118:3300/0 conn(0x7f4f680b3ab0 msgr2=0x7f4f680b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:26.548 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.538+0000 7f4f6ce1f700 1 -- 172.21.15.162:0/3940532865 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2733369528 0 0) 0x7f4f54003230 con 0x7f4f68142ad0 2020-11-28T21:56:26.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.538+0000 7f4f6de21700 1 --2- 172.21.15.162:0/3940532865 >> v2:172.21.15.118:3300/0 conn(0x7f4f680b3ab0 0x7f4f680b30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:26.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.538+0000 7f4f6de21700 1 -- 172.21.15.162:0/3940532865 >> v1:172.21.15.162:6789/0 conn(0x7f4f68142ad0 legacy=0x7f4f6813fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:26.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.538+0000 7f4f6de21700 1 -- 172.21.15.162:0/3940532865 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f6805c030 con 0x7f4f680b2d00 2020-11-28T21:56:26.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.538+0000 7f4f6de21700 1 --2- 172.21.15.162:0/3940532865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f680b2d00 0x7f4f6805c4d0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 rx=0x7f4f60001bf0 tx=0x7f4f58006dc0).ready entity=mon.1 client_cookie=cc9cd760679005c4 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:26.550 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.542+0000 7f4f6ce1f700 1 -- 172.21.15.162:0/3940532865 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f60001a50 con 0x7f4f680b2d00 2020-11-28T21:56:26.550 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.542+0000 7f4f6ce1f700 1 -- 172.21.15.162:0/3940532865 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f5800d5c0 con 0x7f4f680b2d00 2020-11-28T21:56:26.550 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.542+0000 7f4f6ce1f700 1 -- 172.21.15.162:0/3940532865 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f5800d730 con 0x7f4f680b2d00 2020-11-28T21:56:26.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.542+0000 7f4f6ee23700 1 -- 172.21.15.162:0/3940532865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f680b2d00 msgr2=0x7f4f6805c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:26.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.542+0000 7f4f6ee23700 1 --2- 172.21.15.162:0/3940532865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f680b2d00 0x7f4f6805c4d0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 rx=0x7f4f60001bf0 tx=0x7f4f58006dc0).stop 2020-11-28T21:56:26.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.542+0000 7f4f6ee23700 1 -- 172.21.15.162:0/3940532865 shutdown_connections 2020-11-28T21:56:26.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.542+0000 7f4f6ee23700 1 --2- 172.21.15.162:0/3940532865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f680b2d00 0x7f4f6805c4d0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:26.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.542+0000 7f4f6ee23700 1 --2- 172.21.15.162:0/3940532865 >> v2:172.21.15.118:3300/0 conn(0x7f4f680b3ab0 0x7f4f680b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:26.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.542+0000 7f4f6ee23700 1 -- 172.21.15.162:0/3940532865 >> 172.21.15.162:0/3940532865 conn(0x7f4f680604c0 msgr2=0x7f4f680608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:26.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.542+0000 7f4f6ee23700 1 -- 172.21.15.162:0/3940532865 shutdown_connections 2020-11-28T21:56:26.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.542+0000 7f4f6ee23700 1 -- 172.21.15.162:0/3940532865 wait complete. 2020-11-28T21:56:26.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.542+0000 7f4f6ee23700 1 Processor -- start 2020-11-28T21:56:26.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.542+0000 7f4f6ee23700 1 -- start start 2020-11-28T21:56:26.555 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.542+0000 7f4f6ee23700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f680b2d00 0x7f4f680b8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:26.555 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.542+0000 7f4f6ee23700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4f680b3ab0 0x7f4f681410f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:26.555 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.542+0000 7f4f6ee23700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4f68142ad0 0x7f4f68151950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:26.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.542+0000 7f4f6ee23700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f68147160 con 0x7f4f68142ad0 2020-11-28T21:56:26.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.542+0000 7f4f6ee23700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f68147390 con 0x7f4f680b2d00 2020-11-28T21:56:26.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.542+0000 7f4f6ee23700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f68148b30 con 0x7f4f680b3ab0 2020-11-28T21:56:26.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.542+0000 7f4f6de21700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f680b2d00 0x7f4f680b8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:26.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.542+0000 7f4f6de21700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f680b2d00 0x7f4f680b8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46330/0 (socket says 172.21.15.162:46330) 2020-11-28T21:56:26.557 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.542+0000 7f4f6de21700 1 -- 172.21.15.162:0/3842288739 learned_addr learned my addr 172.21.15.162:0/3842288739 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:26.557 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.542+0000 7f4f6d620700 1 --2- 172.21.15.162:0/3842288739 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4f680b3ab0 0x7f4f681410f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:26.557 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.542+0000 7f4f6e622700 1 --2- 172.21.15.162:0/3842288739 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4f68142ad0 0x7f4f68151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:26.557 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.546+0000 7f4f6de21700 1 -- 172.21.15.162:0/3842288739 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4f680b3ab0 msgr2=0x7f4f681410f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:26.557 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.546+0000 7f4f6de21700 1 --2- 172.21.15.162:0/3842288739 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4f680b3ab0 0x7f4f681410f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:26.558 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.546+0000 7f4f6de21700 1 -- 172.21.15.162:0/3842288739 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4f68142ad0 msgr2=0x7f4f68151950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:26.558 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.546+0000 7f4f6de21700 1 --2- 172.21.15.162:0/3842288739 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4f68142ad0 0x7f4f68151950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:26.558 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.546+0000 7f4f6de21700 1 -- 172.21.15.162:0/3842288739 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f68148dd0 con 0x7f4f680b2d00 2020-11-28T21:56:26.558 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.546+0000 7f4f6e622700 1 --2- 172.21.15.162:0/3842288739 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4f68142ad0 0x7f4f68151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:56:26.559 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.546+0000 7f4f6de21700 1 --2- 172.21.15.162:0/3842288739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f680b2d00 0x7f4f680b8870 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 rx=0x7f4f60001bf0 tx=0x7f4f58015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:26.559 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.546+0000 7f4f5effd700 1 -- 172.21.15.162:0/3842288739 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f58011a40 con 0x7f4f680b2d00 2020-11-28T21:56:26.559 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.546+0000 7f4f5effd700 1 -- 172.21.15.162:0/3842288739 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f58011ba0 con 0x7f4f680b2d00 2020-11-28T21:56:26.559 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.546+0000 7f4f6ee23700 1 -- 172.21.15.162:0/3842288739 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f68147630 con 0x7f4f680b2d00 2020-11-28T21:56:26.559 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.546+0000 7f4f5effd700 1 -- 172.21.15.162:0/3842288739 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f5801d480 con 0x7f4f680b2d00 2020-11-28T21:56:26.560 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.546+0000 7f4f6ee23700 1 -- 172.21.15.162:0/3842288739 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f68149490 con 0x7f4f680b2d00 2020-11-28T21:56:26.560 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.546+0000 7f4f6ee23700 1 -- 172.21.15.162:0/3842288739 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f680b80d0 con 0x7f4f680b2d00 2020-11-28T21:56:26.560 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.546+0000 7f4f5effd700 1 -- 172.21.15.162:0/3842288739 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4f58024030 con 0x7f4f680b2d00 2020-11-28T21:56:26.560 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.546+0000 7f4f5effd700 1 --2- 172.21.15.162:0/3842288739 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4f480553b0 0x7f4f48057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:26.560 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.546+0000 7f4f5effd700 1 -- 172.21.15.162:0/3842288739 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4f5806edc0 con 0x7f4f680b2d00 2020-11-28T21:56:26.561 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.546+0000 7f4f6d620700 1 --2- 172.21.15.162:0/3842288739 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4f480553b0 0x7f4f48057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:26.561 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.550+0000 7f4f6d620700 1 --2- 172.21.15.162:0/3842288739 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4f480553b0 0x7f4f48057870 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f4f6400b000 tx=0x7f4f64006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:26.561 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.554+0000 7f4f5effd700 1 -- 172.21.15.162:0/3842288739 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4f680b80d0 con 0x7f4f680b2d00 2020-11-28T21:56:26.750 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.746+0000 7f4f6ee23700 1 -- 172.21.15.162:0/3842288739 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4f680b80d0 con 0x7f4f680b2d00 2020-11-28T21:56:26.840 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.834+0000 7f4f5effd700 1 -- 172.21.15.162:0/3842288739 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v80) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4f680b80d0 con 0x7f4f680b2d00 2020-11-28T21:56:26.843 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.838+0000 7f4f6ee23700 1 -- 172.21.15.162:0/3842288739 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4f480553b0 msgr2=0x7f4f48057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:26.844 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.838+0000 7f4f6ee23700 1 --2- 172.21.15.162:0/3842288739 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4f480553b0 0x7f4f48057870 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f4f6400b000 tx=0x7f4f64006dc0).stop 2020-11-28T21:56:26.844 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.838+0000 7f4f6ee23700 1 -- 172.21.15.162:0/3842288739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f680b2d00 msgr2=0x7f4f680b8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:26.844 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.838+0000 7f4f6ee23700 1 --2- 172.21.15.162:0/3842288739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f680b2d00 0x7f4f680b8870 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 rx=0x7f4f60001bf0 tx=0x7f4f58015000).stop 2020-11-28T21:56:26.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.838+0000 7f4f6ee23700 1 -- 172.21.15.162:0/3842288739 shutdown_connections 2020-11-28T21:56:26.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.838+0000 7f4f6ee23700 1 --2- 172.21.15.162:0/3842288739 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4f480553b0 0x7f4f48057870 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:26.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.838+0000 7f4f6ee23700 1 --2- 172.21.15.162:0/3842288739 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4f68142ad0 0x7f4f68151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:26.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.838+0000 7f4f6ee23700 1 --2- 172.21.15.162:0/3842288739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f680b2d00 0x7f4f680b8870 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:26.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.838+0000 7f4f6ee23700 1 --2- 172.21.15.162:0/3842288739 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4f680b3ab0 0x7f4f681410f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:26.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.838+0000 7f4f6ee23700 1 -- 172.21.15.162:0/3842288739 >> 172.21.15.162:0/3842288739 conn(0x7f4f680604c0 msgr2=0x7f4f6813aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:26.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.838+0000 7f4f6ee23700 1 -- 172.21.15.162:0/3842288739 shutdown_connections 2020-11-28T21:56:26.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.838+0000 7f4f6ee23700 1 -- 172.21.15.162:0/3842288739 wait complete. 2020-11-28T21:56:26.864 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:56:26.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.970+0000 7f4b4be14700 1 Processor -- start 2020-11-28T21:56:26.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.970+0000 7f4b4be14700 1 -- start start 2020-11-28T21:56:26.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.970+0000 7f4b4be14700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f4b440b6800 0x7f4b440b22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:26.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.970+0000 7f4b4be14700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4b440b0100 0x7f4b4405fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:26.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.970+0000 7f4b4be14700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b4405c330 con 0x7f4b440ac900 2020-11-28T21:56:26.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.970+0000 7f4b4be14700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b440b1aa0 con 0x7f4b440b6800 2020-11-28T21:56:26.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.970+0000 7f4b4be14700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b4405fb00 con 0x7f4b440b0100 2020-11-28T21:56:26.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.974+0000 7f4b4a3b1700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4b440b0100 0x7f4b4405fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:26.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.974+0000 7f4b493af700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f4b440ac900 0x7f4b440af9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:53182/0 (socket says 172.21.15.162:53182) 2020-11-28T21:56:26.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.974+0000 7f4b493af700 1 -- 172.21.15.162:0/1894298822 learned_addr learned my addr 172.21.15.162:0/1894298822 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:56:26.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.974+0000 7f4b49bb0700 1 --2- 172.21.15.162:0/1894298822 >> v2:172.21.15.118:3300/0 conn(0x7f4b440b6800 0x7f4b440b22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:26.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.974+0000 7f4b4a3b1700 1 -- 172.21.15.162:0/1894298822 >> v2:172.21.15.118:3300/0 conn(0x7f4b440b6800 msgr2=0x7f4b440b22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:26.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.974+0000 7f4b4a3b1700 1 --2- 172.21.15.162:0/1894298822 >> v2:172.21.15.118:3300/0 conn(0x7f4b440b6800 0x7f4b440b22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:26.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.974+0000 7f4b4a3b1700 1 -- 172.21.15.162:0/1894298822 >> v1:172.21.15.162:6789/0 conn(0x7f4b440ac900 legacy=0x7f4b440af9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:26.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.974+0000 7f4b4a3b1700 1 -- 172.21.15.162:0/1894298822 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b44060320 con 0x7f4b440b0100 2020-11-28T21:56:26.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.974+0000 7f4b4a3b1700 1 --2- 172.21.15.162:0/1894298822 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b440b0100 0x7f4b4405fe30 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 rx=0x7f4b3c0032b0 tx=0x7f4b3c0091a0).ready entity=mon.1 client_cookie=4ce42c6dfcb27744 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:26.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.974+0000 7f4b48bae700 1 -- 172.21.15.162:0/1894298822 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b3c00b9a0 con 0x7f4b440b0100 2020-11-28T21:56:26.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.974+0000 7f4b48bae700 1 -- 172.21.15.162:0/1894298822 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b3c00bb00 con 0x7f4b440b0100 2020-11-28T21:56:26.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.974+0000 7f4b48bae700 1 -- 172.21.15.162:0/1894298822 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b3c00bc70 con 0x7f4b440b0100 2020-11-28T21:56:26.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.974+0000 7f4b4be14700 1 -- 172.21.15.162:0/1894298822 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b440b0100 msgr2=0x7f4b4405fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:26.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.974+0000 7f4b4be14700 1 --2- 172.21.15.162:0/1894298822 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b440b0100 0x7f4b4405fe30 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 rx=0x7f4b3c0032b0 tx=0x7f4b3c0091a0).stop 2020-11-28T21:56:26.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.974+0000 7f4b4be14700 1 -- 172.21.15.162:0/1894298822 shutdown_connections 2020-11-28T21:56:26.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.974+0000 7f4b4be14700 1 --2- 172.21.15.162:0/1894298822 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b440b0100 0x7f4b4405fe30 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:26.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.974+0000 7f4b4be14700 1 --2- 172.21.15.162:0/1894298822 >> v2:172.21.15.118:3300/0 conn(0x7f4b440b6800 0x7f4b440b22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:26.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.974+0000 7f4b4be14700 1 -- 172.21.15.162:0/1894298822 >> 172.21.15.162:0/1894298822 conn(0x7f4b440621c0 msgr2=0x7f4b440b5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:26.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.974+0000 7f4b4be14700 1 -- 172.21.15.162:0/1894298822 shutdown_connections 2020-11-28T21:56:26.985 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.974+0000 7f4b4be14700 1 -- 172.21.15.162:0/1894298822 wait complete. 2020-11-28T21:56:26.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.978+0000 7f4b4be14700 1 Processor -- start 2020-11-28T21:56:26.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.978+0000 7f4b4be14700 1 -- start start 2020-11-28T21:56:26.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.978+0000 7f4b4be14700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b440ac900 0x7f4b440af290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:26.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.978+0000 7f4b4be14700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4b440b0100 0x7f4b4409fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:26.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.978+0000 7f4b4be14700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4b440b6800 0x7f4b440a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:26.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.978+0000 7f4b4be14700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b44066f90 con 0x7f4b440b0100 2020-11-28T21:56:26.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.978+0000 7f4b4be14700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b440a0770 con 0x7f4b440ac900 2020-11-28T21:56:26.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.978+0000 7f4b4be14700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b440662d0 con 0x7f4b440b6800 2020-11-28T21:56:26.992 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.978+0000 7f4b49bb0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b440ac900 0x7f4b440af290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:26.992 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.978+0000 7f4b49bb0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b440ac900 0x7f4b440af290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46344/0 (socket says 172.21.15.162:46344) 2020-11-28T21:56:26.992 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.978+0000 7f4b49bb0700 1 -- 172.21.15.162:0/375656474 learned_addr learned my addr 172.21.15.162:0/375656474 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:26.992 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.978+0000 7f4b4a3b1700 1 --2- 172.21.15.162:0/375656474 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4b440b6800 0x7f4b440a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:26.993 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.978+0000 7f4b493af700 1 --2- 172.21.15.162:0/375656474 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4b440b0100 0x7f4b4409fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:26.993 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.978+0000 7f4b49bb0700 1 -- 172.21.15.162:0/375656474 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4b440b6800 msgr2=0x7f4b440a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:26.993 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.978+0000 7f4b49bb0700 1 --2- 172.21.15.162:0/375656474 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4b440b6800 0x7f4b440a0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:26.993 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.978+0000 7f4b49bb0700 1 -- 172.21.15.162:0/375656474 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4b440b0100 msgr2=0x7f4b4409fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:26.994 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.978+0000 7f4b49bb0700 1 --2- 172.21.15.162:0/375656474 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4b440b0100 0x7f4b4409fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:26.994 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.978+0000 7f4b49bb0700 1 -- 172.21.15.162:0/375656474 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b440671c0 con 0x7f4b440ac900 2020-11-28T21:56:26.994 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.978+0000 7f4b493af700 1 --2- 172.21.15.162:0/375656474 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4b440b0100 0x7f4b4409fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:56:26.994 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.978+0000 7f4b49bb0700 1 --2- 172.21.15.162:0/375656474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b440ac900 0x7f4b440af290 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 rx=0x7f4b4000c000 tx=0x7f4b40006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:26.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.978+0000 7f4b32ffd700 1 -- 172.21.15.162:0/375656474 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b4000ca10 con 0x7f4b440ac900 2020-11-28T21:56:26.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.978+0000 7f4b32ffd700 1 -- 172.21.15.162:0/375656474 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b4000cb70 con 0x7f4b440ac900 2020-11-28T21:56:26.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.978+0000 7f4b4be14700 1 -- 172.21.15.162:0/375656474 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b44067460 con 0x7f4b440ac900 2020-11-28T21:56:26.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.978+0000 7f4b32ffd700 1 -- 172.21.15.162:0/375656474 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b400123f0 con 0x7f4b440ac900 2020-11-28T21:56:26.996 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.978+0000 7f4b4be14700 1 -- 172.21.15.162:0/375656474 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b44066570 con 0x7f4b440ac900 2020-11-28T21:56:26.996 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.982+0000 7f4b4be14700 1 -- 172.21.15.162:0/375656474 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b440675f0 con 0x7f4b440ac900 2020-11-28T21:56:26.996 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.982+0000 7f4b32ffd700 1 -- 172.21.15.162:0/375656474 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4b40012550 con 0x7f4b440ac900 2020-11-28T21:56:26.996 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.982+0000 7f4b32ffd700 1 --2- 172.21.15.162:0/375656474 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4b24055500 0x7f4b240579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:26.997 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.982+0000 7f4b32ffd700 1 -- 172.21.15.162:0/375656474 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4b4006e3d0 con 0x7f4b440ac900 2020-11-28T21:56:26.997 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.982+0000 7f4b493af700 1 --2- 172.21.15.162:0/375656474 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4b24055500 0x7f4b240579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:26.997 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.982+0000 7f4b493af700 1 --2- 172.21.15.162:0/375656474 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4b24055500 0x7f4b240579c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7f4b34008000 tx=0x7f4b34009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:26.998 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:26.990+0000 7f4b32ffd700 1 -- 172.21.15.162:0/375656474 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4b40044670 con 0x7f4b440ac900 2020-11-28T21:56:27.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.174+0000 7f4b4be14700 1 -- 172.21.15.162:0/375656474 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b44057050 con 0x7f4b440ac900 2020-11-28T21:56:27.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.174+0000 7f4b32ffd700 1 -- 172.21.15.162:0/375656474 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+2501 (secure 0 0 0) 0x7f4b40047c90 con 0x7f4b440ac900 2020-11-28T21:56:27.181 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:56:27.182 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":80,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8199,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/2385081970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":2385081970},{"type":"v1","addr":"172.21.15.118:6833","nonce":2385081970}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":77},{"gid":8554,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6836/169306274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":169306274},{"type":"v1","addr":"172.21.15.162:6836","nonce":169306274}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":78}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:56:25.831016+0000","modified":"2020-11-28T21:56:26.833832+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8557},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8557":{"gid":8557,"name":"c","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/3753662046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":3753662046},{"type":"v1","addr":"172.21.15.162:6837","nonce":3753662046}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2020-11-28T21:56:27.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.178+0000 7f4b4be14700 1 -- 172.21.15.162:0/375656474 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4b24055500 msgr2=0x7f4b240579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:27.185 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.178+0000 7f4b4be14700 1 --2- 172.21.15.162:0/375656474 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4b24055500 0x7f4b240579c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7f4b34008000 tx=0x7f4b34009040).stop 2020-11-28T21:56:27.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.178+0000 7f4b4be14700 1 -- 172.21.15.162:0/375656474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b440ac900 msgr2=0x7f4b440af290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:27.186 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.178+0000 7f4b4be14700 1 --2- 172.21.15.162:0/375656474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b440ac900 0x7f4b440af290 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 rx=0x7f4b4000c000 tx=0x7f4b40006dc0).stop 2020-11-28T21:56:27.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.178+0000 7f4b4be14700 1 -- 172.21.15.162:0/375656474 shutdown_connections 2020-11-28T21:56:27.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.178+0000 7f4b4be14700 1 --2- 172.21.15.162:0/375656474 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4b24055500 0x7f4b240579c0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:27.187 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.178+0000 7f4b4be14700 1 --2- 172.21.15.162:0/375656474 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4b440b6800 0x7f4b440a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:27.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.178+0000 7f4b4be14700 1 --2- 172.21.15.162:0/375656474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b440ac900 0x7f4b440af290 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:27.188 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.178+0000 7f4b4be14700 1 --2- 172.21.15.162:0/375656474 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4b440b0100 0x7f4b4409fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:27.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.178+0000 7f4b4be14700 1 -- 172.21.15.162:0/375656474 >> 172.21.15.162:0/375656474 conn(0x7f4b440621c0 msgr2=0x7f4b441413e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:27.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.178+0000 7f4b4be14700 1 -- 172.21.15.162:0/375656474 shutdown_connections 2020-11-28T21:56:27.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.178+0000 7f4b4be14700 1 -- 172.21.15.162:0/375656474 wait complete. 2020-11-28T21:56:27.190 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 80 2020-11-28T21:56:27.194 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:56:27.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.302+0000 7f5f01a93700 1 Processor -- start 2020-11-28T21:56:27.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.306+0000 7f5f01a93700 1 -- start start 2020-11-28T21:56:27.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.306+0000 7f5f01a93700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5efc0b2da0 0x7f5efc0b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:27.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.306+0000 7f5f01a93700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5efc0b7d00 con 0x7f5efc0b7920 2020-11-28T21:56:27.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.306+0000 7f5f01a93700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5efc143300 con 0x7f5efc140090 2020-11-28T21:56:27.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.306+0000 7f5f01a93700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc143560 con 0x7f5efc0b2da0 2020-11-28T21:56:27.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.306+0000 7f5f01292700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f5efc140090 0x7f5efc14d220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:53200/0 (socket says 172.21.15.162:53200) 2020-11-28T21:56:27.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.306+0000 7f5f01292700 1 -- 172.21.15.162:0/1904117784 learned_addr learned my addr 172.21.15.162:0/1904117784 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:56:27.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.306+0000 7f5efbfff700 1 --2- 172.21.15.162:0/1904117784 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5efc0b2da0 0x7f5efc0b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:27.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.306+0000 7f5efb7fe700 1 -- 172.21.15.162:0/1904117784 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1724367979 0 0) 0x7f5efc143300 con 0x7f5efc140090 2020-11-28T21:56:27.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.306+0000 7f5efb7fe700 1 -- 172.21.15.162:0/1904117784 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ee8003230 con 0x7f5efc140090 2020-11-28T21:56:27.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.306+0000 7f5efb7fe700 1 -- 172.21.15.162:0/1904117784 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 832082042 0 0) 0x7f5efc0b7d00 con 0x7f5efc0b7920 2020-11-28T21:56:27.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.306+0000 7f5efb7fe700 1 -- 172.21.15.162:0/1904117784 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5efc143300 con 0x7f5efc0b7920 2020-11-28T21:56:27.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.306+0000 7f5efb7fe700 1 -- 172.21.15.162:0/1904117784 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 153876413 0 0) 0x7f5ee8003230 con 0x7f5efc140090 2020-11-28T21:56:27.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.306+0000 7f5efbfff700 1 -- 172.21.15.162:0/1904117784 >> v1:172.21.15.162:6789/0 conn(0x7f5efc140090 legacy=0x7f5efc14d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:27.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.306+0000 7f5efbfff700 1 -- 172.21.15.162:0/1904117784 >> v1:172.21.15.118:6789/0 conn(0x7f5efc0b7920 legacy=0x7f5efc13f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:27.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.306+0000 7f5efbfff700 1 -- 172.21.15.162:0/1904117784 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5efc0b1910 con 0x7f5efc0b2da0 2020-11-28T21:56:27.318 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.306+0000 7f5efbfff700 1 --2- 172.21.15.162:0/1904117784 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5efc0b2da0 0x7f5efc0b1db0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f5ef40010d0 tx=0x7f5eec0091a0).ready entity=mon.2 client_cookie=49cf852887218984 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:27.318 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.306+0000 7f5efb7fe700 1 -- 172.21.15.162:0/1904117784 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ef40016f0 con 0x7f5efc0b2da0 2020-11-28T21:56:27.318 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.306+0000 7f5efb7fe700 1 -- 172.21.15.162:0/1904117784 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eec00b880 con 0x7f5efc0b2da0 2020-11-28T21:56:27.318 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.306+0000 7f5f01a93700 1 -- 172.21.15.162:0/1904117784 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5efc0b2da0 msgr2=0x7f5efc0b1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:27.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.306+0000 7f5efb7fe700 1 -- 172.21.15.162:0/1904117784 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5eec00b9f0 con 0x7f5efc0b2da0 2020-11-28T21:56:27.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.306+0000 7f5f01a93700 1 --2- 172.21.15.162:0/1904117784 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5efc0b2da0 0x7f5efc0b1db0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f5ef40010d0 tx=0x7f5eec0091a0).stop 2020-11-28T21:56:27.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.306+0000 7f5f01a93700 1 -- 172.21.15.162:0/1904117784 shutdown_connections 2020-11-28T21:56:27.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.306+0000 7f5f01a93700 1 --2- 172.21.15.162:0/1904117784 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5efc0b2da0 0x7f5efc0b1db0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:27.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.306+0000 7f5f01a93700 1 -- 172.21.15.162:0/1904117784 >> 172.21.15.162:0/1904117784 conn(0x7f5efc05c440 msgr2=0x7f5efc0b6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:27.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.306+0000 7f5f01a93700 1 -- 172.21.15.162:0/1904117784 shutdown_connections 2020-11-28T21:56:27.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.306+0000 7f5f01a93700 1 -- 172.21.15.162:0/1904117784 wait complete. 2020-11-28T21:56:27.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.310+0000 7f5f01a93700 1 Processor -- start 2020-11-28T21:56:27.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.310+0000 7f5f01a93700 1 -- start start 2020-11-28T21:56:27.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.310+0000 7f5f01a93700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5efc0b2da0 0x7f5efc1412a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:27.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.310+0000 7f5f01a93700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5efc0b7920 0x7f5efc154f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:27.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.310+0000 7f5f01a93700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5efc140090 0x7f5efc155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:27.324 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.310+0000 7f5f01a93700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc143cb0 con 0x7f5efc0b2da0 2020-11-28T21:56:27.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.310+0000 7f5f01a93700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc0af860 con 0x7f5efc140090 2020-11-28T21:56:27.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.310+0000 7f5f01a93700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc143ee0 con 0x7f5efc0b7920 2020-11-28T21:56:27.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.310+0000 7f5f00a91700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5efc0b2da0 0x7f5efc1412a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:27.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.310+0000 7f5f01292700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5efc140090 0x7f5efc155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:27.325 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.310+0000 7f5f01292700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5efc140090 0x7f5efc155480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46362/0 (socket says 172.21.15.162:46362) 2020-11-28T21:56:27.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.310+0000 7f5f00a91700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5efc0b2da0 0x7f5efc1412a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.162:53346/0 (socket says 172.21.15.162:53346) 2020-11-28T21:56:27.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.310+0000 7f5f01292700 1 -- 172.21.15.162:0/3980883850 learned_addr learned my addr 172.21.15.162:0/3980883850 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:27.326 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.310+0000 7f5efbfff700 1 --2- 172.21.15.162:0/3980883850 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5efc0b7920 0x7f5efc154f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:27.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.310+0000 7f5f01292700 1 -- 172.21.15.162:0/3980883850 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5efc0b7920 msgr2=0x7f5efc154f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:27.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.310+0000 7f5f01292700 1 --2- 172.21.15.162:0/3980883850 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5efc0b7920 0x7f5efc154f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:27.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.310+0000 7f5f01292700 1 -- 172.21.15.162:0/3980883850 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5efc0b2da0 msgr2=0x7f5efc1412a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:27.327 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.310+0000 7f5f01292700 1 --2- 172.21.15.162:0/3980883850 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5efc0b2da0 0x7f5efc1412a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:27.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.310+0000 7f5f01292700 1 -- 172.21.15.162:0/3980883850 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5efc144180 con 0x7f5efc140090 2020-11-28T21:56:27.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.310+0000 7f5f01292700 1 --2- 172.21.15.162:0/3980883850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5efc140090 0x7f5efc155480 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 rx=0x7f5efc151a50 tx=0x7f5ef40019f0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:27.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.310+0000 7f5ef9ffb700 1 -- 172.21.15.162:0/3980883850 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ef4007170 con 0x7f5efc140090 2020-11-28T21:56:27.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.310+0000 7f5ef9ffb700 1 -- 172.21.15.162:0/3980883850 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef40072d0 con 0x7f5efc140090 2020-11-28T21:56:27.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.310+0000 7f5f01a93700 1 -- 172.21.15.162:0/3980883850 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5efc0adb20 con 0x7f5efc140090 2020-11-28T21:56:27.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.310+0000 7f5ef9ffb700 1 -- 172.21.15.162:0/3980883850 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ef40048f0 con 0x7f5efc140090 2020-11-28T21:56:27.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.310+0000 7f5f01a93700 1 -- 172.21.15.162:0/3980883850 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5efc146ae0 con 0x7f5efc140090 2020-11-28T21:56:27.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.314+0000 7f5f01a93700 1 -- 172.21.15.162:0/3980883850 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5efc144310 con 0x7f5efc140090 2020-11-28T21:56:27.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.314+0000 7f5ef9ffb700 1 -- 172.21.15.162:0/3980883850 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5ef401b060 con 0x7f5efc140090 2020-11-28T21:56:27.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.314+0000 7f5ef9ffb700 1 --2- 172.21.15.162:0/3980883850 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5edc0553b0 0x7f5edc057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:27.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.314+0000 7f5f00a91700 1 --2- 172.21.15.162:0/3980883850 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5edc0553b0 0x7f5edc057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:27.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.314+0000 7f5ef9ffb700 1 -- 172.21.15.162:0/3980883850 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5ef406d9d0 con 0x7f5efc140090 2020-11-28T21:56:27.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.314+0000 7f5f00a91700 1 --2- 172.21.15.162:0/3980883850 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5edc0553b0 0x7f5edc057870 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f5ef000d000 tx=0x7f5ef00079d0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:27.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.322+0000 7f5ef9ffb700 1 -- 172.21.15.162:0/3980883850 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5ef4043c70 con 0x7f5efc140090 2020-11-28T21:56:27.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.462+0000 7f5f01a93700 1 -- 172.21.15.162:0/3980883850 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5efc057050 con 0x7f5efc140090 2020-11-28T21:56:27.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.466+0000 7f5ef9ffb700 1 -- 172.21.15.162:0/3980883850 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v159) v1 ==== 74+0+16161 (secure 0 0 0) 0x7f5ef4019020 con 0x7f5efc140090 2020-11-28T21:56:27.471 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:56:27.471 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":159,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:56:26.834489+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:56:23.335325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2020-11-28T21:56:24.648832+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:56:27.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.466+0000 7f5f01a93700 1 -- 172.21.15.162:0/3980883850 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5edc0553b0 msgr2=0x7f5edc057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:27.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.466+0000 7f5f01a93700 1 --2- 172.21.15.162:0/3980883850 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5edc0553b0 0x7f5edc057870 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f5ef000d000 tx=0x7f5ef00079d0).stop 2020-11-28T21:56:27.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.466+0000 7f5f01a93700 1 -- 172.21.15.162:0/3980883850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5efc140090 msgr2=0x7f5efc155480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:27.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.466+0000 7f5f01a93700 1 --2- 172.21.15.162:0/3980883850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5efc140090 0x7f5efc155480 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 rx=0x7f5efc151a50 tx=0x7f5ef40019f0).stop 2020-11-28T21:56:27.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.466+0000 7f5f01a93700 1 -- 172.21.15.162:0/3980883850 shutdown_connections 2020-11-28T21:56:27.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.466+0000 7f5f01a93700 1 --2- 172.21.15.162:0/3980883850 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5edc0553b0 0x7f5edc057870 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:27.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.466+0000 7f5f01a93700 1 --2- 172.21.15.162:0/3980883850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5efc140090 0x7f5efc155480 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:27.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.466+0000 7f5f01a93700 1 --2- 172.21.15.162:0/3980883850 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5efc0b2da0 0x7f5efc1412a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:27.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.466+0000 7f5f01a93700 1 --2- 172.21.15.162:0/3980883850 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5efc0b7920 0x7f5efc154f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:27.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.466+0000 7f5f01a93700 1 -- 172.21.15.162:0/3980883850 >> 172.21.15.162:0/3980883850 conn(0x7f5efc05c440 msgr2=0x7f5efc0b5db0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:27.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.466+0000 7f5f01a93700 1 -- 172.21.15.162:0/3980883850 shutdown_connections 2020-11-28T21:56:27.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.466+0000 7f5f01a93700 1 -- 172.21.15.162:0/3980883850 wait complete. 2020-11-28T21:56:27.485 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-11-28T21:56:27.612 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.602+0000 7f48c0a48700 1 Processor -- start 2020-11-28T21:56:27.612 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.602+0000 7f48c0a48700 1 -- start start 2020-11-28T21:56:27.612 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.602+0000 7f48c0a48700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f48bc0b2e80 0x7f48bc05f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:27.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.602+0000 7f48c0a48700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f48bc0acb40 0x7f48bc05f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:27.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.602+0000 7f48c0a48700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48bc0ab0b0 con 0x7f48bc0b2560 2020-11-28T21:56:27.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.602+0000 7f48c0a48700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48bc05fd60 con 0x7f48bc0b2e80 2020-11-28T21:56:27.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.602+0000 7f48c0a48700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48bc0b3260 con 0x7f48bc0acb40 2020-11-28T21:56:27.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.602+0000 7f48bb59e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f48bc0b2e80 0x7f48bc05f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:27.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.602+0000 7f48bb59e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f48bc0b2e80 0x7f48bc05f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46366/0 (socket says 172.21.15.162:46366) 2020-11-28T21:56:27.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.602+0000 7f48bb59e700 1 -- 172.21.15.162:0/1721772531 learned_addr learned my addr 172.21.15.162:0/1721772531 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:27.614 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.602+0000 7f48c1249700 1 --2- 172.21.15.162:0/1721772531 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f48bc0acb40 0x7f48bc05f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:27.615 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.606+0000 7f48bb59e700 1 -- 172.21.15.162:0/1721772531 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f48bc0acb40 msgr2=0x7f48bc05f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:27.615 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.606+0000 7f48bb59e700 1 --2- 172.21.15.162:0/1721772531 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f48bc0acb40 0x7f48bc05f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:27.615 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.606+0000 7f48bb59e700 1 -- 172.21.15.162:0/1721772531 >> v1:172.21.15.162:6789/0 conn(0x7f48bc0b2560 legacy=0x7f48bc060d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:27.615 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.606+0000 7f48bb59e700 1 -- 172.21.15.162:0/1721772531 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48bc0ab2e0 con 0x7f48bc0b2e80 2020-11-28T21:56:27.615 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.606+0000 7f48c1249700 1 --2- 172.21.15.162:0/1721772531 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f48bc0acb40 0x7f48bc05f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:56:27.616 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.606+0000 7f48bb59e700 1 --2- 172.21.15.162:0/1721772531 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48bc0b2e80 0x7f48bc05f380 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 rx=0x7f48ac0032b0 tx=0x7f48ac009270).ready entity=mon.1 client_cookie=7a8e3fea37d6c93 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:27.616 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.606+0000 7f48ba59c700 1 -- 172.21.15.162:0/1721772531 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48ac00bc00 con 0x7f48bc0b2e80 2020-11-28T21:56:27.616 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.606+0000 7f48ba59c700 1 -- 172.21.15.162:0/1721772531 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48ac00bd60 con 0x7f48bc0b2e80 2020-11-28T21:56:27.616 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.606+0000 7f48ba59c700 1 -- 172.21.15.162:0/1721772531 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48ac011400 con 0x7f48bc0b2e80 2020-11-28T21:56:27.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.606+0000 7f48c0a48700 1 -- 172.21.15.162:0/1721772531 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48bc0b2e80 msgr2=0x7f48bc05f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:27.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.606+0000 7f48c0a48700 1 --2- 172.21.15.162:0/1721772531 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48bc0b2e80 0x7f48bc05f380 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 rx=0x7f48ac0032b0 tx=0x7f48ac009270).stop 2020-11-28T21:56:27.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.606+0000 7f48c0a48700 1 -- 172.21.15.162:0/1721772531 shutdown_connections 2020-11-28T21:56:27.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.606+0000 7f48c0a48700 1 --2- 172.21.15.162:0/1721772531 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48bc0b2e80 0x7f48bc05f380 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:27.617 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.606+0000 7f48c0a48700 1 --2- 172.21.15.162:0/1721772531 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f48bc0acb40 0x7f48bc05f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:27.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.606+0000 7f48c0a48700 1 -- 172.21.15.162:0/1721772531 >> 172.21.15.162:0/1721772531 conn(0x7f48bc063b60 msgr2=0x7f48bc063f40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:27.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.606+0000 7f48c0a48700 1 -- 172.21.15.162:0/1721772531 shutdown_connections 2020-11-28T21:56:27.618 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.606+0000 7f48c0a48700 1 -- 172.21.15.162:0/1721772531 wait complete. 2020-11-28T21:56:27.620 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.610+0000 7f48c0a48700 1 Processor -- start 2020-11-28T21:56:27.621 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.610+0000 7f48c0a48700 1 -- start start 2020-11-28T21:56:27.621 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.610+0000 7f48c0a48700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f48bc0acb40 0x7f48bc0b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:27.621 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.610+0000 7f48c0a48700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48bc0b2560 0x7f48bc0666c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:27.621 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.610+0000 7f48c0a48700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48bc0b2e80 0x7f48bc066bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:27.622 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.610+0000 7f48c0a48700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48bc131b70 con 0x7f48bc0b2e80 2020-11-28T21:56:27.622 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.610+0000 7f48c0a48700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48bc064970 con 0x7f48bc0b2560 2020-11-28T21:56:27.622 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.610+0000 7f48c0a48700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48bc064c10 con 0x7f48bc0acb40 2020-11-28T21:56:27.622 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.610+0000 7f48bad9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48bc0b2560 0x7f48bc0666c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:27.622 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.610+0000 7f48bad9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48bc0b2560 0x7f48bc0666c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46374/0 (socket says 172.21.15.162:46374) 2020-11-28T21:56:27.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.610+0000 7f48bad9d700 1 -- 172.21.15.162:0/1500517701 learned_addr learned my addr 172.21.15.162:0/1500517701 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:27.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.610+0000 7f48bb59e700 1 --2- 172.21.15.162:0/1500517701 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f48bc0acb40 0x7f48bc0b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:27.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.610+0000 7f48c1249700 1 --2- 172.21.15.162:0/1500517701 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48bc0b2e80 0x7f48bc066bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:27.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.610+0000 7f48bad9d700 1 -- 172.21.15.162:0/1500517701 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f48bc0acb40 msgr2=0x7f48bc0b4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:27.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.610+0000 7f48bad9d700 1 --2- 172.21.15.162:0/1500517701 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f48bc0acb40 0x7f48bc0b4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:27.624 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.610+0000 7f48bad9d700 1 -- 172.21.15.162:0/1500517701 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48bc0b2e80 msgr2=0x7f48bc066bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:27.624 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.610+0000 7f48bad9d700 1 --2- 172.21.15.162:0/1500517701 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48bc0b2e80 0x7f48bc066bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:27.624 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.610+0000 7f48bad9d700 1 -- 172.21.15.162:0/1500517701 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48bc1312d0 con 0x7f48bc0b2560 2020-11-28T21:56:27.624 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.610+0000 7f48bb59e700 1 --2- 172.21.15.162:0/1500517701 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f48bc0acb40 0x7f48bc0b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:56:27.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.610+0000 7f48c1249700 1 --2- 172.21.15.162:0/1500517701 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48bc0b2e80 0x7f48bc066bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:56:27.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.610+0000 7f48bad9d700 1 --2- 172.21.15.162:0/1500517701 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48bc0b2560 0x7f48bc0666c0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 rx=0x7f48bc063710 tx=0x7f48a4007be0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:27.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.610+0000 7f48b3fff700 1 -- 172.21.15.162:0/1500517701 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48a40072a0 con 0x7f48bc0b2560 2020-11-28T21:56:27.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.610+0000 7f48b3fff700 1 -- 172.21.15.162:0/1500517701 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a40064e0 con 0x7f48bc0b2560 2020-11-28T21:56:27.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.610+0000 7f48c0a48700 1 -- 172.21.15.162:0/1500517701 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48bc131610 con 0x7f48bc0b2560 2020-11-28T21:56:27.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.610+0000 7f48b3fff700 1 -- 172.21.15.162:0/1500517701 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48a4006840 con 0x7f48bc0b2560 2020-11-28T21:56:27.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.610+0000 7f48c0a48700 1 -- 172.21.15.162:0/1500517701 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48bc067560 con 0x7f48bc0b2560 2020-11-28T21:56:27.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.614+0000 7f48c0a48700 1 -- 172.21.15.162:0/1500517701 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48bc131ce0 con 0x7f48bc0b2560 2020-11-28T21:56:27.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.614+0000 7f48b3fff700 1 -- 172.21.15.162:0/1500517701 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f48a4019020 con 0x7f48bc0b2560 2020-11-28T21:56:27.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.614+0000 7f48b3fff700 1 --2- 172.21.15.162:0/1500517701 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f48a00553b0 0x7f48a0057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:27.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.614+0000 7f48b3fff700 1 -- 172.21.15.162:0/1500517701 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f48a406d770 con 0x7f48bc0b2560 2020-11-28T21:56:27.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.614+0000 7f48bb59e700 1 --2- 172.21.15.162:0/1500517701 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f48a00553b0 0x7f48a0057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:27.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.614+0000 7f48bb59e700 1 --2- 172.21.15.162:0/1500517701 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f48a00553b0 0x7f48a0057870 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7f48ac019000 tx=0x7f48ac016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:27.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.618+0000 7f48b3fff700 1 -- 172.21.15.162:0/1500517701 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f48bc131ce0 con 0x7f48bc0b2560 2020-11-28T21:56:27.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.810+0000 7f48c0a48700 1 -- 172.21.15.162:0/1500517701 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f48bc131ce0 con 0x7f48bc0b2560 2020-11-28T21:56:27.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.818+0000 7f48b3fff700 1 -- 172.21.15.162:0/1500517701 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v33) v1 ==== 184+0+0 (secure 0 0 0) 0x7f48bc131ce0 con 0x7f48bc0b2560 2020-11-28T21:56:27.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.822+0000 7f48c0a48700 1 -- 172.21.15.162:0/1500517701 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f48a00553b0 msgr2=0x7f48a0057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:27.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.822+0000 7f48c0a48700 1 --2- 172.21.15.162:0/1500517701 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f48a00553b0 0x7f48a0057870 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7f48ac019000 tx=0x7f48ac016000).stop 2020-11-28T21:56:27.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.822+0000 7f48c0a48700 1 -- 172.21.15.162:0/1500517701 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48bc0b2560 msgr2=0x7f48bc0666c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:27.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.822+0000 7f48c0a48700 1 --2- 172.21.15.162:0/1500517701 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48bc0b2560 0x7f48bc0666c0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 rx=0x7f48bc063710 tx=0x7f48a4007be0).stop 2020-11-28T21:56:27.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.822+0000 7f48c0a48700 1 -- 172.21.15.162:0/1500517701 shutdown_connections 2020-11-28T21:56:27.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.822+0000 7f48c0a48700 1 --2- 172.21.15.162:0/1500517701 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f48a00553b0 0x7f48a0057870 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:27.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.822+0000 7f48c0a48700 1 --2- 172.21.15.162:0/1500517701 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f48bc0acb40 0x7f48bc0b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:27.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.822+0000 7f48c0a48700 1 --2- 172.21.15.162:0/1500517701 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48bc0b2e80 0x7f48bc066bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:27.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.822+0000 7f48c0a48700 1 --2- 172.21.15.162:0/1500517701 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48bc0b2560 0x7f48bc0666c0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:27.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.822+0000 7f48c0a48700 1 -- 172.21.15.162:0/1500517701 >> 172.21.15.162:0/1500517701 conn(0x7f48bc063b60 msgr2=0x7f48bc0afb70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:27.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.822+0000 7f48c0a48700 1 -- 172.21.15.162:0/1500517701 shutdown_connections 2020-11-28T21:56:27.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.822+0000 7f48c0a48700 1 -- 172.21.15.162:0/1500517701 wait complete. 2020-11-28T21:56:27.831 INFO:teuthology.orchestra.run.smithi162.stderr:updated caps for client.0 2020-11-28T21:56:27.842 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-11-28T21:56:27.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.954+0000 7f0a24b7f700 1 Processor -- start 2020-11-28T21:56:27.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.954+0000 7f0a24b7f700 1 -- start start 2020-11-28T21:56:27.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.954+0000 7f0a24b7f700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0a200b2560 0x7f0a20156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:27.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.954+0000 7f0a24b7f700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f0a2005fa90 0x7f0a201593c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:27.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.954+0000 7f0a24b7f700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a200ab0b0 con 0x7f0a200b2e80 2020-11-28T21:56:27.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.954+0000 7f0a24b7f700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a200b3260 con 0x7f0a2005fa90 2020-11-28T21:56:27.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.954+0000 7f0a24b7f700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a200ab2e0 con 0x7f0a200b2560 2020-11-28T21:56:27.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.954+0000 7f0a1f7fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0a200b2560 0x7f0a20156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:27.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.954+0000 7f0a1ffff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f0a200b2e80 0x7f0a2005f380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:53224/0 (socket says 172.21.15.162:53224) 2020-11-28T21:56:27.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.954+0000 7f0a1ffff700 1 -- 172.21.15.162:0/3039752756 learned_addr learned my addr 172.21.15.162:0/3039752756 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:56:27.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.954+0000 7f0a25de1700 1 --2- 172.21.15.162:0/3039752756 >> v2:172.21.15.118:3300/0 conn(0x7f0a2005fa90 0x7f0a201593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:27.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.958+0000 7f0a1f7fe700 1 -- 172.21.15.162:0/3039752756 >> v2:172.21.15.118:3300/0 conn(0x7f0a2005fa90 msgr2=0x7f0a201593c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:27.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.958+0000 7f0a1f7fe700 1 --2- 172.21.15.162:0/3039752756 >> v2:172.21.15.118:3300/0 conn(0x7f0a2005fa90 0x7f0a201593c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:27.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.958+0000 7f0a1f7fe700 1 -- 172.21.15.162:0/3039752756 >> v1:172.21.15.162:6789/0 conn(0x7f0a200b2e80 legacy=0x7f0a2005f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:27.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.958+0000 7f0a1effd700 1 -- 172.21.15.162:0/3039752756 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2665544670 0 0) 0x7f0a200ab0b0 con 0x7f0a200b2e80 2020-11-28T21:56:27.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.958+0000 7f0a1f7fe700 1 -- 172.21.15.162:0/3039752756 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a200256d0 con 0x7f0a200b2560 2020-11-28T21:56:27.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.958+0000 7f0a1f7fe700 1 --2- 172.21.15.162:0/3039752756 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a200b2560 0x7f0a20156ec0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 rx=0x7f0a10002f30 tx=0x7f0a10006dc0).ready entity=mon.1 client_cookie=ce845597ea10ea43 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:27.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.958+0000 7f0a1effd700 1 -- 172.21.15.162:0/3039752756 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a100097b0 con 0x7f0a200b2560 2020-11-28T21:56:27.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.958+0000 7f0a1effd700 1 -- 172.21.15.162:0/3039752756 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a10009910 con 0x7f0a200b2560 2020-11-28T21:56:27.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.958+0000 7f0a1effd700 1 -- 172.21.15.162:0/3039752756 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a10009c70 con 0x7f0a200b2560 2020-11-28T21:56:27.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.958+0000 7f0a24b7f700 1 -- 172.21.15.162:0/3039752756 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a200b2560 msgr2=0x7f0a20156ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:27.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.958+0000 7f0a24b7f700 1 --2- 172.21.15.162:0/3039752756 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a200b2560 0x7f0a20156ec0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 rx=0x7f0a10002f30 tx=0x7f0a10006dc0).stop 2020-11-28T21:56:27.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.958+0000 7f0a24b7f700 1 -- 172.21.15.162:0/3039752756 shutdown_connections 2020-11-28T21:56:27.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.958+0000 7f0a24b7f700 1 --2- 172.21.15.162:0/3039752756 >> v2:172.21.15.118:3300/0 conn(0x7f0a2005fa90 0x7f0a201593c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:27.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.958+0000 7f0a24b7f700 1 --2- 172.21.15.162:0/3039752756 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a200b2560 0x7f0a20156ec0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:27.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.958+0000 7f0a24b7f700 1 -- 172.21.15.162:0/3039752756 >> 172.21.15.162:0/3039752756 conn(0x7f0a20063b60 msgr2=0x7f0a20063f40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:27.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.958+0000 7f0a24b7f700 1 -- 172.21.15.162:0/3039752756 shutdown_connections 2020-11-28T21:56:27.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.958+0000 7f0a24b7f700 1 -- 172.21.15.162:0/3039752756 wait complete. 2020-11-28T21:56:27.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.962+0000 7f0a24b7f700 1 Processor -- start 2020-11-28T21:56:27.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.962+0000 7f0a24b7f700 1 -- start start 2020-11-28T21:56:27.973 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.962+0000 7f0a24b7f700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a2005fa90 0x7f0a20066190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:27.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.962+0000 7f0a24b7f700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0a200b2560 0x7f0a2012dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:27.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.962+0000 7f0a24b7f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a200b2e80 0x7f0a2012e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:27.974 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.962+0000 7f0a24b7f700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2009a570 con 0x7f0a2005fa90 2020-11-28T21:56:27.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.962+0000 7f0a24b7f700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2009a810 con 0x7f0a200b2e80 2020-11-28T21:56:27.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.962+0000 7f0a24b7f700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2009aab0 con 0x7f0a200b2560 2020-11-28T21:56:27.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.962+0000 7f0a25de1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a200b2e80 0x7f0a2012e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:27.975 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.962+0000 7f0a25de1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a200b2e80 0x7f0a2012e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46390/0 (socket says 172.21.15.162:46390) 2020-11-28T21:56:27.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.962+0000 7f0a25de1700 1 -- 172.21.15.162:0/1075410499 learned_addr learned my addr 172.21.15.162:0/1075410499 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:27.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.962+0000 7f0a1ffff700 1 --2- 172.21.15.162:0/1075410499 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a2005fa90 0x7f0a20066190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:27.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.962+0000 7f0a1f7fe700 1 --2- 172.21.15.162:0/1075410499 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0a200b2560 0x7f0a2012dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:27.976 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.962+0000 7f0a25de1700 1 -- 172.21.15.162:0/1075410499 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0a200b2560 msgr2=0x7f0a2012dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:27.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.962+0000 7f0a25de1700 1 --2- 172.21.15.162:0/1075410499 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0a200b2560 0x7f0a2012dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:27.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.962+0000 7f0a25de1700 1 -- 172.21.15.162:0/1075410499 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a2005fa90 msgr2=0x7f0a20066190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:27.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.962+0000 7f0a25de1700 1 --2- 172.21.15.162:0/1075410499 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a2005fa90 0x7f0a20066190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:27.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.962+0000 7f0a25de1700 1 -- 172.21.15.162:0/1075410499 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a2009ad50 con 0x7f0a200b2e80 2020-11-28T21:56:27.978 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.962+0000 7f0a25de1700 1 --2- 172.21.15.162:0/1075410499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a200b2e80 0x7f0a2012e2d0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 rx=0x7f0a1800c000 tx=0x7f0a180091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:27.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.962+0000 7f0a1cff9700 1 -- 172.21.15.162:0/1075410499 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a1800c8d0 con 0x7f0a200b2e80 2020-11-28T21:56:27.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.962+0000 7f0a1cff9700 1 -- 172.21.15.162:0/1075410499 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a1800ca30 con 0x7f0a200b2e80 2020-11-28T21:56:27.979 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.962+0000 7f0a24b7f700 1 -- 172.21.15.162:0/1075410499 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a2012e7c0 con 0x7f0a200b2e80 2020-11-28T21:56:27.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.962+0000 7f0a1cff9700 1 -- 172.21.15.162:0/1075410499 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a1800cd90 con 0x7f0a200b2e80 2020-11-28T21:56:27.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.962+0000 7f0a24b7f700 1 -- 172.21.15.162:0/1075410499 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a200af2e0 con 0x7f0a200b2e80 2020-11-28T21:56:27.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.966+0000 7f0a24b7f700 1 -- 172.21.15.162:0/1075410499 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a200b6470 con 0x7f0a200b2e80 2020-11-28T21:56:27.980 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.966+0000 7f0a1cff9700 1 -- 172.21.15.162:0/1075410499 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f0a18022070 con 0x7f0a200b2e80 2020-11-28T21:56:27.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.966+0000 7f0a1cff9700 1 --2- 172.21.15.162:0/1075410499 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0a040553b0 0x7f0a04057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:27.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.966+0000 7f0a1cff9700 1 -- 172.21.15.162:0/1075410499 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0a1806e5e0 con 0x7f0a200b2e80 2020-11-28T21:56:27.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.966+0000 7f0a1ffff700 1 --2- 172.21.15.162:0/1075410499 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0a040553b0 0x7f0a04057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:27.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.966+0000 7f0a1ffff700 1 --2- 172.21.15.162:0/1075410499 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0a040553b0 0x7f0a04057870 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7f0a0800d000 tx=0x7f0a08007bf0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:27.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:27.970+0000 7f0a1cff9700 1 -- 172.21.15.162:0/1075410499 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f0a200b6470 con 0x7f0a200b2e80 2020-11-28T21:56:28.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.162+0000 7f0a24b7f700 1 -- 172.21.15.162:0/1075410499 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f0a200b6470 con 0x7f0a200b2e80 2020-11-28T21:56:28.175 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.170+0000 7f0a1cff9700 1 -- 172.21.15.162:0/1075410499 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v34) v1 ==== 184+0+0 (secure 0 0 0) 0x7f0a200b6470 con 0x7f0a200b2e80 2020-11-28T21:56:28.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.170+0000 7f0a24b7f700 1 -- 172.21.15.162:0/1075410499 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0a040553b0 msgr2=0x7f0a04057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:28.177 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.170+0000 7f0a24b7f700 1 --2- 172.21.15.162:0/1075410499 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0a040553b0 0x7f0a04057870 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7f0a0800d000 tx=0x7f0a08007bf0).stop 2020-11-28T21:56:28.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.170+0000 7f0a24b7f700 1 -- 172.21.15.162:0/1075410499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a200b2e80 msgr2=0x7f0a2012e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:28.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.170+0000 7f0a24b7f700 1 --2- 172.21.15.162:0/1075410499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a200b2e80 0x7f0a2012e2d0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 rx=0x7f0a1800c000 tx=0x7f0a180091a0).stop 2020-11-28T21:56:28.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.170+0000 7f0a24b7f700 1 -- 172.21.15.162:0/1075410499 shutdown_connections 2020-11-28T21:56:28.178 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.170+0000 7f0a24b7f700 1 --2- 172.21.15.162:0/1075410499 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f0a040553b0 0x7f0a04057870 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:28.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.170+0000 7f0a24b7f700 1 --2- 172.21.15.162:0/1075410499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a200b2e80 0x7f0a2012e2d0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:28.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.170+0000 7f0a24b7f700 1 --2- 172.21.15.162:0/1075410499 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a2005fa90 0x7f0a20066190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:28.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.170+0000 7f0a24b7f700 1 --2- 172.21.15.162:0/1075410499 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0a200b2560 0x7f0a2012dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:28.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.174+0000 7f0a24b7f700 1 -- 172.21.15.162:0/1075410499 >> 172.21.15.162:0/1075410499 conn(0x7f0a20063b60 msgr2=0x7f0a20133480 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:28.179 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.174+0000 7f0a24b7f700 1 -- 172.21.15.162:0/1075410499 shutdown_connections 2020-11-28T21:56:28.180 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.174+0000 7f0a24b7f700 1 -- 172.21.15.162:0/1075410499 wait complete. 2020-11-28T21:56:28.180 INFO:teuthology.orchestra.run.smithi162.stderr:updated caps for client.1 2020-11-28T21:56:28.193 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:56:28.298 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.294+0000 7f1ee5db0700 1 Processor -- start 2020-11-28T21:56:28.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.294+0000 7f1ee5db0700 1 -- start start 2020-11-28T21:56:28.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.294+0000 7f1ee5db0700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ee0140090 0x7f1ee014e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:28.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.294+0000 7f1ee5db0700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ee00b7d00 con 0x7f1ee00b7920 2020-11-28T21:56:28.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.294+0000 7f1ee5db0700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ee0143300 con 0x7f1ee00b2da0 2020-11-28T21:56:28.307 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.294+0000 7f1ee5db0700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee0143560 con 0x7f1ee0140090 2020-11-28T21:56:28.307 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.294+0000 7f1edffff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f1ee00b2da0 0x7f1ee00b1db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:53240/0 (socket says 172.21.15.162:53240) 2020-11-28T21:56:28.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.294+0000 7f1edffff700 1 -- 172.21.15.162:0/335647183 learned_addr learned my addr 172.21.15.162:0/335647183 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:56:28.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.294+0000 7f1ee55af700 1 --2- 172.21.15.162:0/335647183 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ee0140090 0x7f1ee014e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:28.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.294+0000 7f1edf7fe700 1 -- 172.21.15.162:0/335647183 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2935406891 0 0) 0x7f1ee0143300 con 0x7f1ee00b2da0 2020-11-28T21:56:28.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.294+0000 7f1edf7fe700 1 -- 172.21.15.162:0/335647183 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ecc003230 con 0x7f1ee00b2da0 2020-11-28T21:56:28.309 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.294+0000 7f1edf7fe700 1 -- 172.21.15.162:0/335647183 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1900910154 0 0) 0x7f1ee00b7d00 con 0x7f1ee00b7920 2020-11-28T21:56:28.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.294+0000 7f1edf7fe700 1 -- 172.21.15.162:0/335647183 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ee0143300 con 0x7f1ee00b7920 2020-11-28T21:56:28.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.294+0000 7f1edf7fe700 1 -- 172.21.15.162:0/335647183 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 172891787 0 0) 0x7f1ecc003230 con 0x7f1ee00b2da0 2020-11-28T21:56:28.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.294+0000 7f1edf7fe700 1 -- 172.21.15.162:0/335647183 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ee0140090 msgr2=0x7f1ee014e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:28.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.294+0000 7f1edf7fe700 1 --2- 172.21.15.162:0/335647183 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ee0140090 0x7f1ee014e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:28.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.294+0000 7f1edf7fe700 1 -- 172.21.15.162:0/335647183 >> v1:172.21.15.118:6789/0 conn(0x7f1ee00b7920 legacy=0x7f1ee013f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:28.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.294+0000 7f1edf7fe700 1 -- 172.21.15.162:0/335647183 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee00b1910 con 0x7f1ee00b2da0 2020-11-28T21:56:28.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.294+0000 7f1edf7fe700 1 -- 172.21.15.162:0/335647183 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f1ed40016f0 con 0x7f1ee00b2da0 2020-11-28T21:56:28.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.294+0000 7f1edf7fe700 1 -- 172.21.15.162:0/335647183 >> v1:172.21.15.162:6789/0 conn(0x7f1ee00b2da0 legacy=0x7f1ee00b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:28.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.294+0000 7f1edf7fe700 1 --2- 172.21.15.162:0/335647183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ecc0038b0 0x7f1ecc003cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:28.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.294+0000 7f1edf7fe700 1 --2- 172.21.15.162:0/335647183 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1ecc0044d0 0x7f1ecc004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:28.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.294+0000 7f1edf7fe700 1 --2- 172.21.15.162:0/335647183 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ee0140090 0x7f1ecc004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:28.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.294+0000 7f1edf7fe700 1 -- 172.21.15.162:0/335647183 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee0143560 con 0x7f1ecc0044d0 2020-11-28T21:56:28.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.294+0000 7f1edf7fe700 1 -- 172.21.15.162:0/335647183 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ecc005330 con 0x7f1ecc0038b0 2020-11-28T21:56:28.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.294+0000 7f1edf7fe700 1 -- 172.21.15.162:0/335647183 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ecc00b720 con 0x7f1ee0140090 2020-11-28T21:56:28.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.294+0000 7f1ee4dae700 1 --2- 172.21.15.162:0/335647183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ecc0038b0 0x7f1ecc003cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:28.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.294+0000 7f1ee55af700 1 --2- 172.21.15.162:0/335647183 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ee0140090 0x7f1ecc004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:28.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.294+0000 7f1edffff700 1 --2- 172.21.15.162:0/335647183 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1ecc0044d0 0x7f1ecc004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:28.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.294+0000 7f1ee4dae700 1 -- 172.21.15.162:0/335647183 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ee0140090 msgr2=0x7f1ecc004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:28.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.294+0000 7f1ee4dae700 1 --2- 172.21.15.162:0/335647183 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ee0140090 0x7f1ecc004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:28.318 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.294+0000 7f1ee4dae700 1 -- 172.21.15.162:0/335647183 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1ecc0044d0 msgr2=0x7f1ecc004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:28.318 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.294+0000 7f1ee4dae700 1 --2- 172.21.15.162:0/335647183 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1ecc0044d0 0x7f1ecc004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:28.318 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.294+0000 7f1ee4dae700 1 -- 172.21.15.162:0/335647183 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ecc003230 con 0x7f1ecc0038b0 2020-11-28T21:56:28.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.298+0000 7f1ee4dae700 1 --2- 172.21.15.162:0/335647183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ecc0038b0 0x7f1ecc003cb0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 rx=0x7f1ecc0054a0 tx=0x7f1ed0007ee0).ready entity=mon.1 client_cookie=c31b0438b8fb7a73 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:28.319 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.298+0000 7f1edf7fe700 1 -- 172.21.15.162:0/335647183 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ed000cbc0 con 0x7f1ecc0038b0 2020-11-28T21:56:28.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.298+0000 7f1edf7fe700 1 -- 172.21.15.162:0/335647183 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed000cd20 con 0x7f1ecc0038b0 2020-11-28T21:56:28.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.298+0000 7f1edf7fe700 1 -- 172.21.15.162:0/335647183 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ed00125f0 con 0x7f1ecc0038b0 2020-11-28T21:56:28.320 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.298+0000 7f1ee5db0700 1 -- 172.21.15.162:0/335647183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ecc0038b0 msgr2=0x7f1ecc003cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:28.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.298+0000 7f1ee5db0700 1 --2- 172.21.15.162:0/335647183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ecc0038b0 0x7f1ecc003cb0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 rx=0x7f1ecc0054a0 tx=0x7f1ed0007ee0).stop 2020-11-28T21:56:28.321 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.298+0000 7f1ee55af700 1 -- 172.21.15.162:0/335647183 reap_dead start 2020-11-28T21:56:28.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.298+0000 7f1ee5db0700 1 -- 172.21.15.162:0/335647183 shutdown_connections 2020-11-28T21:56:28.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.298+0000 7f1ee5db0700 1 -- 172.21.15.162:0/335647183 >> 172.21.15.162:0/335647183 conn(0x7f1ee005c440 msgr2=0x7f1ee00b6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:28.322 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.298+0000 7f1ee5db0700 1 -- 172.21.15.162:0/335647183 shutdown_connections 2020-11-28T21:56:28.323 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.298+0000 7f1ee5db0700 1 -- 172.21.15.162:0/335647183 wait complete. 2020-11-28T21:56:28.328 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.302+0000 7f1ee5db0700 1 Processor -- start 2020-11-28T21:56:28.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.302+0000 7f1ee5db0700 1 -- start start 2020-11-28T21:56:28.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.302+0000 7f1ee5db0700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1ee00b06f0 0x7f1ee0062e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:28.329 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.302+0000 7f1ee5db0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ee01517a0 0x7f1ee0151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:28.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.302+0000 7f1ee5db0700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ee0152110 0x7f1ee0154d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:28.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.302+0000 7f1ee5db0700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee00b0ad0 con 0x7f1ee00b06f0 2020-11-28T21:56:28.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.302+0000 7f1ee5db0700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee0060630 con 0x7f1ee01517a0 2020-11-28T21:56:28.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.302+0000 7f1ee5db0700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee0144ef0 con 0x7f1ee0152110 2020-11-28T21:56:28.331 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.302+0000 7f1edffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ee01517a0 0x7f1ee0151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:28.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.302+0000 7f1edffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ee01517a0 0x7f1ee0151c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46408/0 (socket says 172.21.15.162:46408) 2020-11-28T21:56:28.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.302+0000 7f1ee4dae700 1 --2- 172.21.15.162:0/737359212 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1ee00b06f0 0x7f1ee0062e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:28.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.302+0000 7f1edffff700 1 -- 172.21.15.162:0/737359212 learned_addr learned my addr 172.21.15.162:0/737359212 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:28.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.302+0000 7f1ee55af700 1 --2- 172.21.15.162:0/737359212 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ee0152110 0x7f1ee0154d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:28.333 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.302+0000 7f1edffff700 1 -- 172.21.15.162:0/737359212 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ee0152110 msgr2=0x7f1ee0154d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:28.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.302+0000 7f1edffff700 1 --2- 172.21.15.162:0/737359212 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ee0152110 0x7f1ee0154d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:28.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.302+0000 7f1edffff700 1 -- 172.21.15.162:0/737359212 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1ee00b06f0 msgr2=0x7f1ee0062e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:28.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.302+0000 7f1edffff700 1 --2- 172.21.15.162:0/737359212 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1ee00b06f0 0x7f1ee0062e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:28.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.302+0000 7f1edffff700 1 -- 172.21.15.162:0/737359212 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee0145150 con 0x7f1ee01517a0 2020-11-28T21:56:28.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.302+0000 7f1edffff700 1 --2- 172.21.15.162:0/737359212 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ee01517a0 0x7f1ee0151c20 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 rx=0x7f1ed400b000 tx=0x7f1ee00644d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:28.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.302+0000 7f1eddffb700 1 -- 172.21.15.162:0/737359212 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ed400c790 con 0x7f1ee01517a0 2020-11-28T21:56:28.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.302+0000 7f1eddffb700 1 -- 172.21.15.162:0/737359212 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed400c8f0 con 0x7f1ee01517a0 2020-11-28T21:56:28.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.302+0000 7f1ee5db0700 1 -- 172.21.15.162:0/737359212 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ee01453f0 con 0x7f1ee01517a0 2020-11-28T21:56:28.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.302+0000 7f1eddffb700 1 -- 172.21.15.162:0/737359212 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ed400cc50 con 0x7f1ee01517a0 2020-11-28T21:56:28.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.302+0000 7f1ee5db0700 1 -- 172.21.15.162:0/737359212 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ee0060b90 con 0x7f1ee01517a0 2020-11-28T21:56:28.337 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.306+0000 7f1ee5db0700 1 -- 172.21.15.162:0/737359212 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ee0057050 con 0x7f1ee01517a0 2020-11-28T21:56:28.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.306+0000 7f1eddffb700 1 -- 172.21.15.162:0/737359212 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1ed4014430 con 0x7f1ee01517a0 2020-11-28T21:56:28.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.306+0000 7f1eddffb700 1 --2- 172.21.15.162:0/737359212 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1ec0055350 0x7f1ec0057810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:28.338 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.306+0000 7f1eddffb700 1 -- 172.21.15.162:0/737359212 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1ed406dd30 con 0x7f1ee01517a0 2020-11-28T21:56:28.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.306+0000 7f1ee4dae700 1 --2- 172.21.15.162:0/737359212 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1ec0055350 0x7f1ec0057810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:28.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.306+0000 7f1ee4dae700 1 --2- 172.21.15.162:0/737359212 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1ec0055350 0x7f1ec0057810 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7f1ed001b000 tx=0x7f1ed0017000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:28.339 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.314+0000 7f1eddffb700 1 -- 172.21.15.162:0/737359212 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1ed4043fd0 con 0x7f1ee01517a0 2020-11-28T21:56:28.505 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.498+0000 7f1ee5db0700 1 -- 172.21.15.162:0/737359212 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ee0068350 con 0x7f1ee01517a0 2020-11-28T21:56:28.506 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.502+0000 7f1eddffb700 1 -- 172.21.15.162:0/737359212 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+2501 (secure 0 0 0) 0x7f1ed400cdb0 con 0x7f1ee01517a0 2020-11-28T21:56:28.507 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:56:28.507 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":80,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8199,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/2385081970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":2385081970},{"type":"v1","addr":"172.21.15.118:6833","nonce":2385081970}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":77},{"gid":8554,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6836/169306274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":169306274},{"type":"v1","addr":"172.21.15.162:6836","nonce":169306274}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":78}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:56:25.831016+0000","modified":"2020-11-28T21:56:26.833832+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8557},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8557":{"gid":8557,"name":"c","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/3753662046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":3753662046},{"type":"v1","addr":"172.21.15.162:6837","nonce":3753662046}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2020-11-28T21:56:28.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.502+0000 7f1ee5db0700 1 -- 172.21.15.162:0/737359212 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1ec0055350 msgr2=0x7f1ec0057810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:28.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.502+0000 7f1ee5db0700 1 --2- 172.21.15.162:0/737359212 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1ec0055350 0x7f1ec0057810 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7f1ed001b000 tx=0x7f1ed0017000).stop 2020-11-28T21:56:28.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.502+0000 7f1ee5db0700 1 -- 172.21.15.162:0/737359212 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ee01517a0 msgr2=0x7f1ee0151c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:28.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.502+0000 7f1ee5db0700 1 --2- 172.21.15.162:0/737359212 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ee01517a0 0x7f1ee0151c20 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 rx=0x7f1ed400b000 tx=0x7f1ee00644d0).stop 2020-11-28T21:56:28.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.502+0000 7f1ee5db0700 1 -- 172.21.15.162:0/737359212 shutdown_connections 2020-11-28T21:56:28.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.502+0000 7f1ee5db0700 1 --2- 172.21.15.162:0/737359212 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1ec0055350 0x7f1ec0057810 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:28.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.502+0000 7f1ee5db0700 1 --2- 172.21.15.162:0/737359212 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ee0152110 0x7f1ee0154d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:28.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.502+0000 7f1ee5db0700 1 --2- 172.21.15.162:0/737359212 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1ee00b06f0 0x7f1ee0062e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:28.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.502+0000 7f1ee5db0700 1 --2- 172.21.15.162:0/737359212 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ee01517a0 0x7f1ee0151c20 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:28.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.502+0000 7f1ee5db0700 1 -- 172.21.15.162:0/737359212 >> 172.21.15.162:0/737359212 conn(0x7f1ee005c440 msgr2=0x7f1ee00b5240 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:28.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.502+0000 7f1ee5db0700 1 -- 172.21.15.162:0/737359212 shutdown_connections 2020-11-28T21:56:28.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:28.502+0000 7f1ee5db0700 1 -- 172.21.15.162:0/737359212 wait complete. 2020-11-28T21:56:28.511 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 80 2020-11-28T21:56:28.823 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 80, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-28T21:56:25.831016+0000', 'modified': '2020-11-28T21:56:26.833832+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8557}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8557': {'gid': 8557, 'name': 'c', 'rank': 0, 'incarnation': 79, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6837/3753662046', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6835', 'nonce': 3753662046}, {'type': 'v1', 'addr': '172.21.15.162:6837', 'nonce': 3753662046}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-28T21:56:28.824 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-28T21:56:28.824 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-28T21:56:28.954 INFO:teuthology.orchestra.run.smithi162.stdout:{"cluster_fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","whoami":0,"id":8557,"want_state":"up:active","state":"up:active","rank_uptime":3.1080159260000002,"mdsmap_epoch":80,"osdmap_epoch":160,"osdmap_epoch_barrier":158,"uptime":8.5224280730000004} 2020-11-28T21:56:28.954 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", "id": 8557, "mdsmap_epoch": 80, "osdmap_epoch": 160, "osdmap_epoch_barrier": 158, "rank_uptime": 3.108015926, "state": "up:active", "uptime": 8.522428073, "want_state": "up:active", "whoami": 0 } 2020-11-28T21:56:28.955 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:56:29.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.054+0000 7f4666ef0700 1 Processor -- start 2020-11-28T21:56:29.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.058+0000 7f4666ef0700 1 -- start start 2020-11-28T21:56:29.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.058+0000 7f4666ef0700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f46600608f0 0x7f466013c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:29.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.058+0000 7f4666ef0700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f46600b6130 0x7f466013ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:29.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.058+0000 7f4666ef0700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46600ba0b0 con 0x7f46600b54f0 2020-11-28T21:56:29.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.058+0000 7f4666ef0700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46600b9e10 con 0x7f46600608f0 2020-11-28T21:56:29.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.058+0000 7f4666ef0700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4660142b80 con 0x7f46600b6130 2020-11-28T21:56:29.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.058+0000 7f465ffff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f46600b6130 0x7f466013ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:29.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.058+0000 7f465ffff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f46600b6130 0x7f466013ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:43638/0 (socket says 172.21.15.162:43638) 2020-11-28T21:56:29.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.058+0000 7f466548d700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f46600b54f0 0x7f4660143020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.162:43406/0 (socket says 172.21.15.162:43406) 2020-11-28T21:56:29.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.058+0000 7f465ffff700 1 -- 172.21.15.162:0/1770312077 learned_addr learned my addr 172.21.15.162:0/1770312077 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:29.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.058+0000 7f4664c8c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f46600608f0 0x7f466013c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:29.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.058+0000 7f465f7fe700 1 -- 172.21.15.162:0/1770312077 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1254358674 0 0) 0x7f46600ba0b0 con 0x7f46600b54f0 2020-11-28T21:56:29.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.058+0000 7f465f7fe700 1 -- 172.21.15.162:0/1770312077 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4644003230 con 0x7f46600b54f0 2020-11-28T21:56:29.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.058+0000 7f465ffff700 1 -- 172.21.15.162:0/1770312077 >> v2:172.21.15.118:3300/0 conn(0x7f46600608f0 msgr2=0x7f466013c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:29.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.058+0000 7f465ffff700 1 --2- 172.21.15.162:0/1770312077 >> v2:172.21.15.118:3300/0 conn(0x7f46600608f0 0x7f466013c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:29.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.058+0000 7f465ffff700 1 -- 172.21.15.162:0/1770312077 >> v1:172.21.15.118:6789/0 conn(0x7f46600b54f0 legacy=0x7f4660143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:29.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.058+0000 7f465ffff700 1 -- 172.21.15.162:0/1770312077 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46600b9b30 con 0x7f46600b6130 2020-11-28T21:56:29.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.058+0000 7f4664c8c700 1 --2- 172.21.15.162:0/1770312077 >> v2:172.21.15.118:3300/0 conn(0x7f46600608f0 0x7f466013c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:56:29.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.058+0000 7f465ffff700 1 --2- 172.21.15.162:0/1770312077 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f46600b6130 0x7f466013ce00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f464c002f30 tx=0x7f464c006e80).ready entity=mon.2 client_cookie=3adf4d20c904ee3a server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:29.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.058+0000 7f465f7fe700 1 -- 172.21.15.162:0/1770312077 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f464c009640 con 0x7f46600b6130 2020-11-28T21:56:29.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.058+0000 7f465f7fe700 1 -- 172.21.15.162:0/1770312077 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f464c0097a0 con 0x7f46600b6130 2020-11-28T21:56:29.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.058+0000 7f465f7fe700 1 -- 172.21.15.162:0/1770312077 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f464c009b00 con 0x7f46600b6130 2020-11-28T21:56:29.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.058+0000 7f4666ef0700 1 -- 172.21.15.162:0/1770312077 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f46600b6130 msgr2=0x7f466013ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:29.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.058+0000 7f4666ef0700 1 --2- 172.21.15.162:0/1770312077 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f46600b6130 0x7f466013ce00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f464c002f30 tx=0x7f464c006e80).stop 2020-11-28T21:56:29.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.062+0000 7f4666ef0700 1 -- 172.21.15.162:0/1770312077 shutdown_connections 2020-11-28T21:56:29.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.062+0000 7f4666ef0700 1 --2- 172.21.15.162:0/1770312077 >> v2:172.21.15.118:3300/0 conn(0x7f46600608f0 0x7f466013c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:29.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.062+0000 7f4666ef0700 1 --2- 172.21.15.162:0/1770312077 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f46600b6130 0x7f466013ce00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:29.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.062+0000 7f4666ef0700 1 -- 172.21.15.162:0/1770312077 >> 172.21.15.162:0/1770312077 conn(0x7f4660066c70 msgr2=0x7f4660067050 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:29.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.062+0000 7f4666ef0700 1 -- 172.21.15.162:0/1770312077 shutdown_connections 2020-11-28T21:56:29.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.062+0000 7f4666ef0700 1 -- 172.21.15.162:0/1770312077 wait complete. 2020-11-28T21:56:29.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.062+0000 7f4666ef0700 1 Processor -- start 2020-11-28T21:56:29.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.062+0000 7f4666ef0700 1 -- start start 2020-11-28T21:56:29.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.062+0000 7f4666ef0700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f46600608f0 0x7f4660140070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:29.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.062+0000 7f4666ef0700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f46600b54f0 0x7f46601449f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:29.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.062+0000 7f4666ef0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46600b6130 0x7f4660154cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:29.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.062+0000 7f4666ef0700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466013f9a0 con 0x7f46600b54f0 2020-11-28T21:56:29.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.062+0000 7f4666ef0700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46600b7200 con 0x7f46600b6130 2020-11-28T21:56:29.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.062+0000 7f4666ef0700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f466013fbd0 con 0x7f46600608f0 2020-11-28T21:56:29.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.062+0000 7f466548d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46600b6130 0x7f4660154cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:29.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.062+0000 7f466548d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46600b6130 0x7f4660154cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46422/0 (socket says 172.21.15.162:46422) 2020-11-28T21:56:29.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.062+0000 7f466548d700 1 -- 172.21.15.162:0/1704482563 learned_addr learned my addr 172.21.15.162:0/1704482563 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:29.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.062+0000 7f4664c8c700 1 --2- 172.21.15.162:0/1704482563 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f46600608f0 0x7f4660140070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:29.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.062+0000 7f465ffff700 1 --2- 172.21.15.162:0/1704482563 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f46600b54f0 0x7f46601449f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:29.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.066+0000 7f466548d700 1 -- 172.21.15.162:0/1704482563 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f46600608f0 msgr2=0x7f4660140070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:29.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.066+0000 7f466548d700 1 --2- 172.21.15.162:0/1704482563 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f46600608f0 0x7f4660140070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:29.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.066+0000 7f466548d700 1 -- 172.21.15.162:0/1704482563 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f46600b54f0 msgr2=0x7f46601449f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:29.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.066+0000 7f466548d700 1 --2- 172.21.15.162:0/1704482563 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f46600b54f0 0x7f46601449f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:29.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.066+0000 7f466548d700 1 -- 172.21.15.162:0/1704482563 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46600b74a0 con 0x7f46600b6130 2020-11-28T21:56:29.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.066+0000 7f466548d700 1 --2- 172.21.15.162:0/1704482563 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46600b6130 0x7f4660154cc0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 rx=0x7f466013eb50 tx=0x7f4658007db0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:29.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.066+0000 7f465dffb700 1 -- 172.21.15.162:0/1704482563 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4658005ab0 con 0x7f46600b6130 2020-11-28T21:56:29.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.066+0000 7f465dffb700 1 -- 172.21.15.162:0/1704482563 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4658005c10 con 0x7f46600b6130 2020-11-28T21:56:29.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.066+0000 7f4666ef0700 1 -- 172.21.15.162:0/1704482563 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f466013f5d0 con 0x7f46600b6130 2020-11-28T21:56:29.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.066+0000 7f465dffb700 1 -- 172.21.15.162:0/1704482563 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4658007110 con 0x7f46600b6130 2020-11-28T21:56:29.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.066+0000 7f4666ef0700 1 -- 172.21.15.162:0/1704482563 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46601551b0 con 0x7f46600b6130 2020-11-28T21:56:29.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.066+0000 7f4666ef0700 1 -- 172.21.15.162:0/1704482563 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f466013d680 con 0x7f46600b6130 2020-11-28T21:56:29.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.066+0000 7f465dffb700 1 -- 172.21.15.162:0/1704482563 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4658010030 con 0x7f46600b6130 2020-11-28T21:56:29.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.066+0000 7f465dffb700 1 --2- 172.21.15.162:0/1704482563 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4648055440 0x7f4648057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:29.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.066+0000 7f4664c8c700 1 --2- 172.21.15.162:0/1704482563 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4648055440 0x7f4648057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:29.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.066+0000 7f465dffb700 1 -- 172.21.15.162:0/1704482563 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f465806de70 con 0x7f46600b6130 2020-11-28T21:56:29.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.070+0000 7f4664c8c700 1 --2- 172.21.15.162:0/1704482563 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4648055440 0x7f4648057900 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f465400b000 tx=0x7f4654006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:29.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.074+0000 7f465dffb700 1 -- 172.21.15.162:0/1704482563 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f466013d680 con 0x7f46600b6130 2020-11-28T21:56:29.219 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.214+0000 7f4666ef0700 1 -- 172.21.15.162:0/1704482563 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f466013d680 con 0x7f46600b6130 2020-11-28T21:56:29.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.214+0000 7f465dffb700 1 -- 172.21.15.162:0/1704482563 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v160) v1 ==== 74+0+16148 (secure 0 0 0) 0x7f466013d680 con 0x7f46600b6130 2020-11-28T21:56:29.222 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:56:29.223 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":160,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:56:27.855579+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:56:23.335325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2020-11-28T21:56:24.648832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:56:29.226 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.218+0000 7f4666ef0700 1 -- 172.21.15.162:0/1704482563 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4648055440 msgr2=0x7f4648057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:29.227 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.218+0000 7f4666ef0700 1 --2- 172.21.15.162:0/1704482563 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4648055440 0x7f4648057900 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f465400b000 tx=0x7f4654006dc0).stop 2020-11-28T21:56:29.227 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.218+0000 7f4666ef0700 1 -- 172.21.15.162:0/1704482563 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46600b6130 msgr2=0x7f4660154cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:29.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.218+0000 7f4666ef0700 1 --2- 172.21.15.162:0/1704482563 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46600b6130 0x7f4660154cc0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 rx=0x7f466013eb50 tx=0x7f4658007db0).stop 2020-11-28T21:56:29.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.218+0000 7f4666ef0700 1 -- 172.21.15.162:0/1704482563 shutdown_connections 2020-11-28T21:56:29.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.218+0000 7f4666ef0700 1 --2- 172.21.15.162:0/1704482563 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4648055440 0x7f4648057900 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:29.229 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.218+0000 7f4666ef0700 1 --2- 172.21.15.162:0/1704482563 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f46600608f0 0x7f4660140070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:29.229 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.218+0000 7f4666ef0700 1 --2- 172.21.15.162:0/1704482563 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46600b6130 0x7f4660154cc0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:29.230 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.218+0000 7f4666ef0700 1 --2- 172.21.15.162:0/1704482563 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f46600b54f0 0x7f46601449f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:29.230 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.218+0000 7f4666ef0700 1 -- 172.21.15.162:0/1704482563 >> 172.21.15.162:0/1704482563 conn(0x7f4660066c70 msgr2=0x7f4660143940 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:29.230 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.218+0000 7f4666ef0700 1 -- 172.21.15.162:0/1704482563 shutdown_connections 2020-11-28T21:56:29.231 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.218+0000 7f4666ef0700 1 -- 172.21.15.162:0/1704482563 wait complete. 2020-11-28T21:56:29.235 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 29, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-28T21:56:23.335325+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '159', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 30, 'pool_name': 'cephfs_data', 'create_time': '2020-11-28T21:56:24.648832+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '160', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-28T21:56:29.236 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:56:29.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.342+0000 7ff3abfff700 1 Processor -- start 2020-11-28T21:56:29.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.342+0000 7ff3abfff700 1 -- start start 2020-11-28T21:56:29.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.346+0000 7ff3abfff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff3ac0b1eb0 0x7ff3ac0b2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:29.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.346+0000 7ff3abfff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff3ac0af6a0 0x7ff3ac060a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:29.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.346+0000 7ff3abfff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff3ac144350 0x7ff3ac1447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:29.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.346+0000 7ff3abfff700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3ac1471f0 con 0x7ff3ac0b1eb0 2020-11-28T21:56:29.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.346+0000 7ff3abfff700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3ac060090 con 0x7ff3ac144350 2020-11-28T21:56:29.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.346+0000 7ff3abfff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac060330 con 0x7ff3ac0af6a0 2020-11-28T21:56:29.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.346+0000 7ff3ab7fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff3ac144350 0x7ff3ac1447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:29.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.346+0000 7ff3ab7fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff3ac144350 0x7ff3ac1447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46430/0 (socket says 172.21.15.162:46430) 2020-11-28T21:56:29.353 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.346+0000 7ff3ab7fe700 1 -- 172.21.15.162:0/1171064346 learned_addr learned my addr 172.21.15.162:0/1171064346 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:29.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.346+0000 7ff3aaffd700 1 --2- 172.21.15.162:0/1171064346 >> v2:172.21.15.118:3300/0 conn(0x7ff3ac0b1eb0 0x7ff3ac0b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:29.354 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.346+0000 7ff3aa7fc700 1 --2- 172.21.15.162:0/1171064346 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff3ac0af6a0 0x7ff3ac060a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:29.355 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.346+0000 7ff3ab7fe700 1 -- 172.21.15.162:0/1171064346 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff3ac0af6a0 msgr2=0x7ff3ac060a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:29.355 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.346+0000 7ff3ab7fe700 1 --2- 172.21.15.162:0/1171064346 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff3ac0af6a0 0x7ff3ac060a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:29.355 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.346+0000 7ff3ab7fe700 1 -- 172.21.15.162:0/1171064346 >> v2:172.21.15.118:3300/0 conn(0x7ff3ac0b1eb0 msgr2=0x7ff3ac0b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:29.355 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.346+0000 7ff3ab7fe700 1 --2- 172.21.15.162:0/1171064346 >> v2:172.21.15.118:3300/0 conn(0x7ff3ac0b1eb0 0x7ff3ac0b2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:29.356 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.346+0000 7ff3ab7fe700 1 -- 172.21.15.162:0/1171064346 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ac0605d0 con 0x7ff3ac144350 2020-11-28T21:56:29.356 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.346+0000 7ff3ab7fe700 1 --2- 172.21.15.162:0/1171064346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3ac144350 0x7ff3ac1447e0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 rx=0x7ff3a40032b0 tx=0x7ff3a4009270).ready entity=mon.1 client_cookie=92c0e15c1f159aab server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:29.356 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.346+0000 7ff3a9ffb700 1 -- 172.21.15.162:0/1171064346 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3a400bc00 con 0x7ff3ac144350 2020-11-28T21:56:29.356 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.346+0000 7ff3a9ffb700 1 -- 172.21.15.162:0/1171064346 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a400bd60 con 0x7ff3ac144350 2020-11-28T21:56:29.356 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.346+0000 7ff3a9ffb700 1 -- 172.21.15.162:0/1171064346 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3a4011400 con 0x7ff3ac144350 2020-11-28T21:56:29.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.346+0000 7ff3abfff700 1 -- 172.21.15.162:0/1171064346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3ac144350 msgr2=0x7ff3ac1447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:29.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.346+0000 7ff3abfff700 1 --2- 172.21.15.162:0/1171064346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3ac144350 0x7ff3ac1447e0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 rx=0x7ff3a40032b0 tx=0x7ff3a4009270).stop 2020-11-28T21:56:29.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.346+0000 7ff3abfff700 1 -- 172.21.15.162:0/1171064346 shutdown_connections 2020-11-28T21:56:29.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.346+0000 7ff3abfff700 1 --2- 172.21.15.162:0/1171064346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3ac144350 0x7ff3ac1447e0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:29.357 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.346+0000 7ff3abfff700 1 --2- 172.21.15.162:0/1171064346 >> v2:172.21.15.118:3300/0 conn(0x7ff3ac0b1eb0 0x7ff3ac0b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:29.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.346+0000 7ff3abfff700 1 --2- 172.21.15.162:0/1171064346 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff3ac0af6a0 0x7ff3ac060a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:29.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.346+0000 7ff3abfff700 1 -- 172.21.15.162:0/1171064346 >> 172.21.15.162:0/1171064346 conn(0x7ff3ac0b57d0 msgr2=0x7ff3ac0b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:29.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.346+0000 7ff3abfff700 1 -- 172.21.15.162:0/1171064346 shutdown_connections 2020-11-28T21:56:29.358 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.346+0000 7ff3abfff700 1 -- 172.21.15.162:0/1171064346 wait complete. 2020-11-28T21:56:29.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.346+0000 7ff3abfff700 1 Processor -- start 2020-11-28T21:56:29.361 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.346+0000 7ff3abfff700 1 -- start start 2020-11-28T21:56:29.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.350+0000 7ff3abfff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff3ac0af6a0 0x7ff3ac0b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:29.362 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.350+0000 7ff3abfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff3ac0b1eb0 0x7ff3ac0ab460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:29.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.350+0000 7ff3abfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3ac144350 0x7ff3ac0ab950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:29.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.350+0000 7ff3abfff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac0abe40 con 0x7ff3ac0b1eb0 2020-11-28T21:56:29.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.350+0000 7ff3abfff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac063130 con 0x7ff3ac144350 2020-11-28T21:56:29.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.350+0000 7ff3abfff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac0633d0 con 0x7ff3ac0af6a0 2020-11-28T21:56:29.363 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.350+0000 7ff3ab7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3ac144350 0x7ff3ac0ab950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:29.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.350+0000 7ff3ab7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3ac144350 0x7ff3ac0ab950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46436/0 (socket says 172.21.15.162:46436) 2020-11-28T21:56:29.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.350+0000 7ff3ab7fe700 1 -- 172.21.15.162:0/1390660349 learned_addr learned my addr 172.21.15.162:0/1390660349 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:29.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.350+0000 7ff3aaffd700 1 --2- 172.21.15.162:0/1390660349 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff3ac0af6a0 0x7ff3ac0b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:29.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.350+0000 7ff3aa7fc700 1 --2- 172.21.15.162:0/1390660349 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff3ac0b1eb0 0x7ff3ac0ab460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:29.364 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.350+0000 7ff3ab7fe700 1 -- 172.21.15.162:0/1390660349 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff3ac0af6a0 msgr2=0x7ff3ac0b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:29.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.350+0000 7ff3ab7fe700 1 --2- 172.21.15.162:0/1390660349 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff3ac0af6a0 0x7ff3ac0b0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:29.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.350+0000 7ff3ab7fe700 1 -- 172.21.15.162:0/1390660349 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff3ac0b1eb0 msgr2=0x7ff3ac0ab460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:29.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.350+0000 7ff3ab7fe700 1 --2- 172.21.15.162:0/1390660349 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff3ac0b1eb0 0x7ff3ac0ab460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:29.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.350+0000 7ff3ab7fe700 1 -- 172.21.15.162:0/1390660349 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ac063670 con 0x7ff3ac144350 2020-11-28T21:56:29.365 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.350+0000 7ff3aaffd700 1 --2- 172.21.15.162:0/1390660349 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff3ac0af6a0 0x7ff3ac0b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:56:29.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.350+0000 7ff3ab7fe700 1 --2- 172.21.15.162:0/1390660349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3ac144350 0x7ff3ac0ab950 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 rx=0x7ff3a40032b0 tx=0x7ff3a4016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:29.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.350+0000 7ff3a3fff700 1 -- 172.21.15.162:0/1390660349 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3a401f5e0 con 0x7ff3ac144350 2020-11-28T21:56:29.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.350+0000 7ff3a3fff700 1 -- 172.21.15.162:0/1390660349 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a401f740 con 0x7ff3ac144350 2020-11-28T21:56:29.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.350+0000 7ff3abfff700 1 -- 172.21.15.162:0/1390660349 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3ac063910 con 0x7ff3ac144350 2020-11-28T21:56:29.366 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.350+0000 7ff3a3fff700 1 -- 172.21.15.162:0/1390660349 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3a401faa0 con 0x7ff3ac144350 2020-11-28T21:56:29.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.350+0000 7ff3abfff700 1 -- 172.21.15.162:0/1390660349 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3ac065830 con 0x7ff3ac144350 2020-11-28T21:56:29.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.350+0000 7ff3abfff700 1 -- 172.21.15.162:0/1390660349 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3ac057050 con 0x7ff3ac144350 2020-11-28T21:56:29.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.350+0000 7ff3a3fff700 1 -- 172.21.15.162:0/1390660349 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff3a4026030 con 0x7ff3ac144350 2020-11-28T21:56:29.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.354+0000 7ff3a3fff700 1 --2- 172.21.15.162:0/1390660349 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff384055440 0x7ff384057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:29.367 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.354+0000 7ff3aaffd700 1 --2- 172.21.15.162:0/1390660349 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff384055440 0x7ff384057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:29.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.354+0000 7ff3a3fff700 1 -- 172.21.15.162:0/1390660349 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff3a406ef90 con 0x7ff3ac144350 2020-11-28T21:56:29.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.354+0000 7ff3aaffd700 1 --2- 172.21.15.162:0/1390660349 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff384055440 0x7ff384057900 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7ff39400c000 tx=0x7ff394008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:29.368 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.358+0000 7ff3a3fff700 1 -- 172.21.15.162:0/1390660349 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff3a4045290 con 0x7ff3ac144350 2020-11-28T21:56:29.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.502+0000 7ff3abfff700 1 -- 172.21.15.162:0/1390660349 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7ff3ac0a89f0 con 0x7ff3ac144350 2020-11-28T21:56:29.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.502+0000 7ff3a3fff700 1 -- 172.21.15.162:0/1390660349 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v160) v1 ==== 110+0+10 (secure 0 0 0) 0x7ff3a40240b0 con 0x7ff3ac144350 2020-11-28T21:56:29.508 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:56:29.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.506+0000 7ff3abfff700 1 -- 172.21.15.162:0/1390660349 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff384055440 msgr2=0x7ff384057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:29.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.506+0000 7ff3abfff700 1 --2- 172.21.15.162:0/1390660349 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff384055440 0x7ff384057900 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7ff39400c000 tx=0x7ff394008040).stop 2020-11-28T21:56:29.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.506+0000 7ff3abfff700 1 -- 172.21.15.162:0/1390660349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3ac144350 msgr2=0x7ff3ac0ab950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:29.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.506+0000 7ff3abfff700 1 --2- 172.21.15.162:0/1390660349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3ac144350 0x7ff3ac0ab950 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 rx=0x7ff3a40032b0 tx=0x7ff3a4016000).stop 2020-11-28T21:56:29.511 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.506+0000 7ff3abfff700 1 -- 172.21.15.162:0/1390660349 shutdown_connections 2020-11-28T21:56:29.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.506+0000 7ff3abfff700 1 --2- 172.21.15.162:0/1390660349 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff384055440 0x7ff384057900 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:29.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.506+0000 7ff3abfff700 1 --2- 172.21.15.162:0/1390660349 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff3ac0af6a0 0x7ff3ac0b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:29.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.506+0000 7ff3abfff700 1 --2- 172.21.15.162:0/1390660349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3ac144350 0x7ff3ac0ab950 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:29.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.506+0000 7ff3abfff700 1 --2- 172.21.15.162:0/1390660349 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff3ac0b1eb0 0x7ff3ac0ab460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:29.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.506+0000 7ff3abfff700 1 -- 172.21.15.162:0/1390660349 >> 172.21.15.162:0/1390660349 conn(0x7ff3ac0b57d0 msgr2=0x7ff3ac0ae550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:29.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.506+0000 7ff3abfff700 1 -- 172.21.15.162:0/1390660349 shutdown_connections 2020-11-28T21:56:29.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.506+0000 7ff3abfff700 1 -- 172.21.15.162:0/1390660349 wait complete. 2020-11-28T21:56:29.521 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:56:29.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.630+0000 7f935dc82700 1 Processor -- start 2020-11-28T21:56:29.636 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.630+0000 7f935dc82700 1 -- start start 2020-11-28T21:56:29.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.630+0000 7f935dc82700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f93580605c0 0x7f9358140390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:29.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.630+0000 7f935dc82700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9358143950 con 0x7f93580b9ce0 2020-11-28T21:56:29.637 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.630+0000 7f935dc82700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93580ba0c0 con 0x7f93581433c0 2020-11-28T21:56:29.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.630+0000 7f935dc82700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93580b4340 con 0x7f93580605c0 2020-11-28T21:56:29.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.630+0000 7f9356ffd700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f93581433c0 0x7f935813fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:53286/0 (socket says 172.21.15.162:53286) 2020-11-28T21:56:29.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.630+0000 7f9356ffd700 1 -- 172.21.15.162:0/2517237251 learned_addr learned my addr 172.21.15.162:0/2517237251 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:56:29.638 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.634+0000 7f9357fff700 1 --2- 172.21.15.162:0/2517237251 >> v2:172.21.15.118:3300/0 conn(0x7f93580605c0 0x7f9358140390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:29.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.634+0000 7f93567fc700 1 -- 172.21.15.162:0/2517237251 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 50701833 0 0) 0x7f93580ba0c0 con 0x7f93581433c0 2020-11-28T21:56:29.640 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.634+0000 7f93567fc700 1 -- 172.21.15.162:0/2517237251 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9338003230 con 0x7f93581433c0 2020-11-28T21:56:29.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.634+0000 7f9357fff700 1 -- 172.21.15.162:0/2517237251 >> v1:172.21.15.162:6789/0 conn(0x7f93581433c0 legacy=0x7f935813fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:29.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.634+0000 7f93567fc700 1 -- 172.21.15.162:0/2517237251 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1247613165 0 0) 0x7f9358143950 con 0x7f93580b9ce0 2020-11-28T21:56:29.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.634+0000 7f9357fff700 1 -- 172.21.15.162:0/2517237251 >> v1:172.21.15.118:6789/0 conn(0x7f93580b9ce0 legacy=0x7f93580b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:29.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.634+0000 7f9357fff700 1 -- 172.21.15.162:0/2517237251 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9358066c30 con 0x7f93580605c0 2020-11-28T21:56:29.641 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.634+0000 7f9357fff700 1 --2- 172.21.15.162:0/2517237251 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93580605c0 0x7f9358140390 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7f9340001870 tx=0x7f93500091a0).ready entity=mon.0 client_cookie=faa40f3d673cb382 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:29.642 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.634+0000 7f93567fc700 1 -- 172.21.15.162:0/2517237251 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93400016d0 con 0x7f93580605c0 2020-11-28T21:56:29.642 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.634+0000 7f93567fc700 1 -- 172.21.15.162:0/2517237251 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f935000b880 con 0x7f93580605c0 2020-11-28T21:56:29.642 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.634+0000 7f93567fc700 1 -- 172.21.15.162:0/2517237251 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f935000b9f0 con 0x7f93580605c0 2020-11-28T21:56:29.642 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.634+0000 7f935dc82700 1 -- 172.21.15.162:0/2517237251 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93580605c0 msgr2=0x7f9358140390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:29.642 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.634+0000 7f935dc82700 1 --2- 172.21.15.162:0/2517237251 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93580605c0 0x7f9358140390 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7f9340001870 tx=0x7f93500091a0).stop 2020-11-28T21:56:29.643 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.634+0000 7f935dc82700 1 -- 172.21.15.162:0/2517237251 shutdown_connections 2020-11-28T21:56:29.643 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.634+0000 7f935dc82700 1 --2- 172.21.15.162:0/2517237251 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93580605c0 0x7f9358140390 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:29.643 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.634+0000 7f935dc82700 1 -- 172.21.15.162:0/2517237251 >> 172.21.15.162:0/2517237251 conn(0x7f935805c430 msgr2=0x7f93580b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:29.643 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.634+0000 7f935dc82700 1 -- 172.21.15.162:0/2517237251 shutdown_connections 2020-11-28T21:56:29.644 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.634+0000 7f935dc82700 1 -- 172.21.15.162:0/2517237251 wait complete. 2020-11-28T21:56:29.646 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.638+0000 7f935dc82700 1 Processor -- start 2020-11-28T21:56:29.646 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.638+0000 7f935dc82700 1 -- start start 2020-11-28T21:56:29.646 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.638+0000 7f935dc82700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f93580605c0 0x7f93580b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:29.646 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.638+0000 7f935dc82700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93580b9ce0 0x7f9358148fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:29.647 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.638+0000 7f935dc82700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93581433c0 0x7f93581494d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:29.647 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.638+0000 7f935dc82700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9358144260 con 0x7f93580b9ce0 2020-11-28T21:56:29.647 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.638+0000 7f935dc82700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9358144500 con 0x7f93581433c0 2020-11-28T21:56:29.647 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.638+0000 7f935dc82700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93581447a0 con 0x7f93580605c0 2020-11-28T21:56:29.647 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.638+0000 7f9357fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93581433c0 0x7f93581494d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:29.648 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.638+0000 7f9357fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93581433c0 0x7f93581494d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46450/0 (socket says 172.21.15.162:46450) 2020-11-28T21:56:29.648 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.638+0000 7f9357fff700 1 -- 172.21.15.162:0/3756195646 learned_addr learned my addr 172.21.15.162:0/3756195646 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:29.648 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.638+0000 7f9356ffd700 1 --2- 172.21.15.162:0/3756195646 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93580b9ce0 0x7f9358148fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:29.648 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.638+0000 7f93577fe700 1 --2- 172.21.15.162:0/3756195646 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f93580605c0 0x7f93580b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:29.648 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.638+0000 7f9357fff700 1 -- 172.21.15.162:0/3756195646 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f93580605c0 msgr2=0x7f93580b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:29.649 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.638+0000 7f9357fff700 1 --2- 172.21.15.162:0/3756195646 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f93580605c0 0x7f93580b5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:29.649 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.638+0000 7f9357fff700 1 -- 172.21.15.162:0/3756195646 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93580b9ce0 msgr2=0x7f9358148fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:29.649 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.638+0000 7f9357fff700 1 --2- 172.21.15.162:0/3756195646 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93580b9ce0 0x7f9358148fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:29.649 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.638+0000 7f9357fff700 1 -- 172.21.15.162:0/3756195646 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9358144a40 con 0x7f93581433c0 2020-11-28T21:56:29.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.638+0000 7f9357fff700 1 --2- 172.21.15.162:0/3756195646 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93581433c0 0x7f93581494d0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 rx=0x7f9340001870 tx=0x7f935805fe30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:29.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.638+0000 7f9354ff9700 1 -- 172.21.15.162:0/3756195646 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9350021070 con 0x7f93581433c0 2020-11-28T21:56:29.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.638+0000 7f935dc82700 1 -- 172.21.15.162:0/3756195646 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93580b6330 con 0x7f93581433c0 2020-11-28T21:56:29.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.638+0000 7f9354ff9700 1 -- 172.21.15.162:0/3756195646 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9350011b40 con 0x7f93581433c0 2020-11-28T21:56:29.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.638+0000 7f9354ff9700 1 -- 172.21.15.162:0/3756195646 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9350011cb0 con 0x7f93581433c0 2020-11-28T21:56:29.651 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.638+0000 7f935dc82700 1 -- 172.21.15.162:0/3756195646 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9358146b20 con 0x7f93581433c0 2020-11-28T21:56:29.651 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.642+0000 7f935dc82700 1 -- 172.21.15.162:0/3756195646 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9358144bd0 con 0x7f93581433c0 2020-11-28T21:56:29.652 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.642+0000 7f9354ff9700 1 -- 172.21.15.162:0/3756195646 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f935000f070 con 0x7f93581433c0 2020-11-28T21:56:29.652 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.642+0000 7f9354ff9700 1 --2- 172.21.15.162:0/3756195646 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f933c0553b0 0x7f933c057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:29.652 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.642+0000 7f93577fe700 1 --2- 172.21.15.162:0/3756195646 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f933c0553b0 0x7f933c057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:29.652 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.642+0000 7f9354ff9700 1 -- 172.21.15.162:0/3756195646 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f935006fbf0 con 0x7f93581433c0 2020-11-28T21:56:29.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.642+0000 7f93577fe700 1 --2- 172.21.15.162:0/3756195646 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f933c0553b0 0x7f933c057870 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7f934800d000 tx=0x7f93480079b0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:29.655 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.650+0000 7f9354ff9700 1 -- 172.21.15.162:0/3756195646 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9350045de0 con 0x7f93581433c0 2020-11-28T21:56:29.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.790+0000 7f935dc82700 1 -- 172.21.15.162:0/3756195646 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9358057050 con 0x7f93581433c0 2020-11-28T21:56:29.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.790+0000 7f9354ff9700 1 -- 172.21.15.162:0/3756195646 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v160) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9350049400 con 0x7f93581433c0 2020-11-28T21:56:29.797 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:56:29.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.794+0000 7f935dc82700 1 -- 172.21.15.162:0/3756195646 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f933c0553b0 msgr2=0x7f933c057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:29.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.794+0000 7f935dc82700 1 --2- 172.21.15.162:0/3756195646 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f933c0553b0 0x7f933c057870 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7f934800d000 tx=0x7f93480079b0).stop 2020-11-28T21:56:29.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.794+0000 7f935dc82700 1 -- 172.21.15.162:0/3756195646 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93581433c0 msgr2=0x7f93581494d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:29.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.794+0000 7f935dc82700 1 --2- 172.21.15.162:0/3756195646 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93581433c0 0x7f93581494d0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 rx=0x7f9340001870 tx=0x7f935805fe30).stop 2020-11-28T21:56:29.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.794+0000 7f935dc82700 1 -- 172.21.15.162:0/3756195646 shutdown_connections 2020-11-28T21:56:29.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.794+0000 7f935dc82700 1 --2- 172.21.15.162:0/3756195646 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f933c0553b0 0x7f933c057870 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:29.800 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.794+0000 7f935dc82700 1 --2- 172.21.15.162:0/3756195646 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f93580605c0 0x7f93580b5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:29.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.794+0000 7f935dc82700 1 --2- 172.21.15.162:0/3756195646 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93581433c0 0x7f93581494d0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:29.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.794+0000 7f935dc82700 1 --2- 172.21.15.162:0/3756195646 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93580b9ce0 0x7f9358148fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:29.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.794+0000 7f935dc82700 1 -- 172.21.15.162:0/3756195646 >> 172.21.15.162:0/3756195646 conn(0x7f935805c430 msgr2=0x7f93580b65d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:29.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.794+0000 7f935dc82700 1 -- 172.21.15.162:0/3756195646 shutdown_connections 2020-11-28T21:56:29.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.794+0000 7f935dc82700 1 -- 172.21.15.162:0/3756195646 wait complete. 2020-11-28T21:56:29.811 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-28T21:56:29.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.922+0000 7ff249ab0700 1 Processor -- start 2020-11-28T21:56:29.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.922+0000 7ff249ab0700 1 -- start start 2020-11-28T21:56:29.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.922+0000 7ff249ab0700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff2440b1eb0 0x7ff2440b2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:29.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.922+0000 7ff249ab0700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff2440af6a0 0x7ff244060a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:29.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.922+0000 7ff249ab0700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff244144350 0x7ff2441447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:29.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.922+0000 7ff249ab0700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2441471f0 con 0x7ff2440af6a0 2020-11-28T21:56:29.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.922+0000 7ff249ab0700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff244060090 con 0x7ff244144350 2020-11-28T21:56:29.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.922+0000 7ff249ab0700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff244060330 con 0x7ff2440b1eb0 2020-11-28T21:56:29.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.922+0000 7ff2492af700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff244144350 0x7ff2441447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:29.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.922+0000 7ff2492af700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff244144350 0x7ff2441447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46458/0 (socket says 172.21.15.162:46458) 2020-11-28T21:56:29.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.922+0000 7ff2492af700 1 -- 172.21.15.162:0/4086777005 learned_addr learned my addr 172.21.15.162:0/4086777005 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:29.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.922+0000 7ff248aae700 1 --2- 172.21.15.162:0/4086777005 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff2440b1eb0 0x7ff2440b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:29.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.922+0000 7ff243fff700 1 --2- 172.21.15.162:0/4086777005 >> v2:172.21.15.118:3300/0 conn(0x7ff2440af6a0 0x7ff244060a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:29.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.922+0000 7ff2492af700 1 -- 172.21.15.162:0/4086777005 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff2440b1eb0 msgr2=0x7ff2440b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:29.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.922+0000 7ff2492af700 1 --2- 172.21.15.162:0/4086777005 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff2440b1eb0 0x7ff2440b2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:29.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.922+0000 7ff2492af700 1 -- 172.21.15.162:0/4086777005 >> v2:172.21.15.118:3300/0 conn(0x7ff2440af6a0 msgr2=0x7ff244060a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:29.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.926+0000 7ff2492af700 1 --2- 172.21.15.162:0/4086777005 >> v2:172.21.15.118:3300/0 conn(0x7ff2440af6a0 0x7ff244060a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:29.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.926+0000 7ff2492af700 1 -- 172.21.15.162:0/4086777005 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2440605d0 con 0x7ff244144350 2020-11-28T21:56:29.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.926+0000 7ff2492af700 1 --2- 172.21.15.162:0/4086777005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff244144350 0x7ff2441447e0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 rx=0x7ff23c002f30 tx=0x7ff23c006e80).ready entity=mon.1 client_cookie=fada3724ad4fb325 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:29.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.926+0000 7ff2437fe700 1 -- 172.21.15.162:0/4086777005 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff23c0097f0 con 0x7ff244144350 2020-11-28T21:56:29.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.926+0000 7ff2437fe700 1 -- 172.21.15.162:0/4086777005 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23c009950 con 0x7ff244144350 2020-11-28T21:56:29.939 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.926+0000 7ff2437fe700 1 -- 172.21.15.162:0/4086777005 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff23c009cb0 con 0x7ff244144350 2020-11-28T21:56:29.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.926+0000 7ff249ab0700 1 -- 172.21.15.162:0/4086777005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff244144350 msgr2=0x7ff2441447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:29.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.926+0000 7ff249ab0700 1 --2- 172.21.15.162:0/4086777005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff244144350 0x7ff2441447e0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 rx=0x7ff23c002f30 tx=0x7ff23c006e80).stop 2020-11-28T21:56:29.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.926+0000 7ff249ab0700 1 -- 172.21.15.162:0/4086777005 shutdown_connections 2020-11-28T21:56:29.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.926+0000 7ff249ab0700 1 --2- 172.21.15.162:0/4086777005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff244144350 0x7ff2441447e0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:29.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.926+0000 7ff249ab0700 1 --2- 172.21.15.162:0/4086777005 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff2440b1eb0 0x7ff2440b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:29.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.926+0000 7ff249ab0700 1 --2- 172.21.15.162:0/4086777005 >> v2:172.21.15.118:3300/0 conn(0x7ff2440af6a0 0x7ff244060a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:29.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.926+0000 7ff249ab0700 1 -- 172.21.15.162:0/4086777005 >> 172.21.15.162:0/4086777005 conn(0x7ff2440b57d0 msgr2=0x7ff2440b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:29.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.926+0000 7ff249ab0700 1 -- 172.21.15.162:0/4086777005 shutdown_connections 2020-11-28T21:56:29.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.926+0000 7ff249ab0700 1 -- 172.21.15.162:0/4086777005 wait complete. 2020-11-28T21:56:29.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.926+0000 7ff249ab0700 1 Processor -- start 2020-11-28T21:56:29.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.926+0000 7ff249ab0700 1 -- start start 2020-11-28T21:56:29.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.926+0000 7ff249ab0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2440af6a0 0x7ff244063470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:29.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.926+0000 7ff249ab0700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff2440b1eb0 0x7ff2440aa500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:29.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.926+0000 7ff249ab0700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff244144350 0x7ff2440aa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:29.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.926+0000 7ff249ab0700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff24414d7a0 con 0x7ff2440b1eb0 2020-11-28T21:56:29.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.926+0000 7ff249ab0700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff24414d540 con 0x7ff2440af6a0 2020-11-28T21:56:29.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.926+0000 7ff249ab0700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff244157020 con 0x7ff244144350 2020-11-28T21:56:29.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.926+0000 7ff248aae700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2440af6a0 0x7ff244063470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:29.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.930+0000 7ff248aae700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2440af6a0 0x7ff244063470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46460/0 (socket says 172.21.15.162:46460) 2020-11-28T21:56:29.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.930+0000 7ff248aae700 1 -- 172.21.15.162:0/720399065 learned_addr learned my addr 172.21.15.162:0/720399065 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:29.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.930+0000 7ff2492af700 1 --2- 172.21.15.162:0/720399065 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff244144350 0x7ff2440aa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:29.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.930+0000 7ff243fff700 1 --2- 172.21.15.162:0/720399065 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff2440b1eb0 0x7ff2440aa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:29.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.930+0000 7ff248aae700 1 -- 172.21.15.162:0/720399065 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff244144350 msgr2=0x7ff2440aa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:29.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.930+0000 7ff248aae700 1 --2- 172.21.15.162:0/720399065 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff244144350 0x7ff2440aa9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:29.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.930+0000 7ff248aae700 1 -- 172.21.15.162:0/720399065 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff2440b1eb0 msgr2=0x7ff2440aa500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:29.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.930+0000 7ff248aae700 1 --2- 172.21.15.162:0/720399065 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff2440b1eb0 0x7ff2440aa500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:29.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.930+0000 7ff248aae700 1 -- 172.21.15.162:0/720399065 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2441572c0 con 0x7ff2440af6a0 2020-11-28T21:56:29.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.930+0000 7ff248aae700 1 --2- 172.21.15.162:0/720399065 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2440af6a0 0x7ff244063470 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 rx=0x7ff2440ac260 tx=0x7ff234008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:29.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.930+0000 7ff241ffb700 1 -- 172.21.15.162:0/720399065 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff23400ba70 con 0x7ff2440af6a0 2020-11-28T21:56:29.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.930+0000 7ff241ffb700 1 -- 172.21.15.162:0/720399065 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23400bbd0 con 0x7ff2440af6a0 2020-11-28T21:56:29.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.930+0000 7ff249ab0700 1 -- 172.21.15.162:0/720399065 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2440aaee0 con 0x7ff2440af6a0 2020-11-28T21:56:29.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.930+0000 7ff241ffb700 1 -- 172.21.15.162:0/720399065 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff234012420 con 0x7ff2440af6a0 2020-11-28T21:56:29.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.930+0000 7ff249ab0700 1 -- 172.21.15.162:0/720399065 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2440a8c10 con 0x7ff2440af6a0 2020-11-28T21:56:29.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.930+0000 7ff241ffb700 1 -- 172.21.15.162:0/720399065 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff234012580 con 0x7ff2440af6a0 2020-11-28T21:56:29.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.930+0000 7ff249ab0700 1 -- 172.21.15.162:0/720399065 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff244157450 con 0x7ff2440af6a0 2020-11-28T21:56:29.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.930+0000 7ff241ffb700 1 --2- 172.21.15.162:0/720399065 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff228055330 0x7ff2280577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:29.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.930+0000 7ff241ffb700 1 -- 172.21.15.162:0/720399065 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff23406e4d0 con 0x7ff2440af6a0 2020-11-28T21:56:29.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.930+0000 7ff243fff700 1 --2- 172.21.15.162:0/720399065 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff228055330 0x7ff2280577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:29.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.930+0000 7ff243fff700 1 --2- 172.21.15.162:0/720399065 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff228055330 0x7ff2280577f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7ff22c00c000 tx=0x7ff22c008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:29.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:29.942+0000 7ff241ffb700 1 -- 172.21.15.162:0/720399065 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff2340447a0 con 0x7ff2440af6a0 2020-11-28T21:56:30.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.082+0000 7ff249ab0700 1 -- 172.21.15.162:0/720399065 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff244057050 con 0x7ff2440af6a0 2020-11-28T21:56:30.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.086+0000 7ff241ffb700 1 -- 172.21.15.162:0/720399065 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v160) v1 ==== 104+0+10 (secure 0 0 0) 0x7ff234047dc0 con 0x7ff2440af6a0 2020-11-28T21:56:30.090 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:56:30.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.086+0000 7ff249ab0700 1 -- 172.21.15.162:0/720399065 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff228055330 msgr2=0x7ff2280577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:30.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.086+0000 7ff249ab0700 1 --2- 172.21.15.162:0/720399065 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff228055330 0x7ff2280577f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7ff22c00c000 tx=0x7ff22c008040).stop 2020-11-28T21:56:30.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.086+0000 7ff249ab0700 1 -- 172.21.15.162:0/720399065 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2440af6a0 msgr2=0x7ff244063470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:30.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.086+0000 7ff249ab0700 1 --2- 172.21.15.162:0/720399065 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2440af6a0 0x7ff244063470 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 rx=0x7ff2440ac260 tx=0x7ff234008040).stop 2020-11-28T21:56:30.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.086+0000 7ff249ab0700 1 -- 172.21.15.162:0/720399065 shutdown_connections 2020-11-28T21:56:30.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.086+0000 7ff249ab0700 1 --2- 172.21.15.162:0/720399065 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff228055330 0x7ff2280577f0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:30.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.086+0000 7ff249ab0700 1 --2- 172.21.15.162:0/720399065 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff244144350 0x7ff2440aa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:30.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.086+0000 7ff249ab0700 1 --2- 172.21.15.162:0/720399065 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2440af6a0 0x7ff244063470 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:30.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.086+0000 7ff249ab0700 1 --2- 172.21.15.162:0/720399065 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff2440b1eb0 0x7ff2440aa500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:30.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.086+0000 7ff249ab0700 1 -- 172.21.15.162:0/720399065 >> 172.21.15.162:0/720399065 conn(0x7ff2440b57d0 msgr2=0x7ff2440b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:30.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.086+0000 7ff249ab0700 1 -- 172.21.15.162:0/720399065 shutdown_connections 2020-11-28T21:56:30.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.086+0000 7ff249ab0700 1 -- 172.21.15.162:0/720399065 wait complete. 2020-11-28T21:56:30.104 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-28T21:56:30.220 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.214+0000 7f7a60acf700 1 Processor -- start 2020-11-28T21:56:30.221 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.214+0000 7f7a60acf700 1 -- start start 2020-11-28T21:56:30.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.218+0000 7f7a60acf700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f7a5c1433c0 0x7f7a5c0605c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:30.222 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.218+0000 7f7a60acf700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a5c143950 con 0x7f7a5c0b9ce0 2020-11-28T21:56:30.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.218+0000 7f7a60acf700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a5c0ba0c0 con 0x7f7a5c067610 2020-11-28T21:56:30.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.218+0000 7f7a60acf700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a5c0b4340 con 0x7f7a5c1433c0 2020-11-28T21:56:30.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.218+0000 7f7a5ad9d700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f7a5c067610 0x7f7a5c13fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:53316/0 (socket says 172.21.15.162:53316) 2020-11-28T21:56:30.223 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.218+0000 7f7a5ad9d700 1 -- 172.21.15.162:0/2776107738 learned_addr learned my addr 172.21.15.162:0/2776107738 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:56:30.224 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.218+0000 7f7a59d9b700 1 --2- 172.21.15.162:0/2776107738 >> v2:172.21.15.118:3300/0 conn(0x7f7a5c1433c0 0x7f7a5c0605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:30.225 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.218+0000 7f7a5959a700 1 -- 172.21.15.162:0/2776107738 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3650137233 0 0) 0x7f7a5c0ba0c0 con 0x7f7a5c067610 2020-11-28T21:56:30.225 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.218+0000 7f7a5959a700 1 -- 172.21.15.162:0/2776107738 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a3c003230 con 0x7f7a5c067610 2020-11-28T21:56:30.226 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.218+0000 7f7a59d9b700 1 -- 172.21.15.162:0/2776107738 >> v1:172.21.15.162:6789/0 conn(0x7f7a5c067610 legacy=0x7f7a5c13fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:30.226 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.218+0000 7f7a59d9b700 1 -- 172.21.15.162:0/2776107738 >> v1:172.21.15.118:6789/0 conn(0x7f7a5c0b9ce0 legacy=0x7f7a5c0b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:30.226 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.218+0000 7f7a59d9b700 1 -- 172.21.15.162:0/2776107738 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5c066c30 con 0x7f7a5c1433c0 2020-11-28T21:56:30.226 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.218+0000 7f7a59d9b700 1 --2- 172.21.15.162:0/2776107738 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a5c1433c0 0x7f7a5c0605c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7f7a4c002f30 tx=0x7f7a4c006dc0).ready entity=mon.0 client_cookie=3858d400be2a8863 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:30.227 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.218+0000 7f7a5959a700 1 -- 172.21.15.162:0/2776107738 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a4c009800 con 0x7f7a5c1433c0 2020-11-28T21:56:30.227 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.218+0000 7f7a5959a700 1 -- 172.21.15.162:0/2776107738 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4c009960 con 0x7f7a5c1433c0 2020-11-28T21:56:30.227 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.218+0000 7f7a5959a700 1 -- 172.21.15.162:0/2776107738 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a4c009cc0 con 0x7f7a5c1433c0 2020-11-28T21:56:30.227 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.218+0000 7f7a60acf700 1 -- 172.21.15.162:0/2776107738 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a5c1433c0 msgr2=0x7f7a5c0605c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:30.227 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.218+0000 7f7a60acf700 1 --2- 172.21.15.162:0/2776107738 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a5c1433c0 0x7f7a5c0605c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7f7a4c002f30 tx=0x7f7a4c006dc0).stop 2020-11-28T21:56:30.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.218+0000 7f7a60acf700 1 -- 172.21.15.162:0/2776107738 shutdown_connections 2020-11-28T21:56:30.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.218+0000 7f7a60acf700 1 --2- 172.21.15.162:0/2776107738 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a5c1433c0 0x7f7a5c0605c0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:30.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.218+0000 7f7a60acf700 1 -- 172.21.15.162:0/2776107738 >> 172.21.15.162:0/2776107738 conn(0x7f7a5c05c430 msgr2=0x7f7a5c0b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:30.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.218+0000 7f7a60acf700 1 -- 172.21.15.162:0/2776107738 shutdown_connections 2020-11-28T21:56:30.229 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.218+0000 7f7a60acf700 1 -- 172.21.15.162:0/2776107738 wait complete. 2020-11-28T21:56:30.231 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.222+0000 7f7a60acf700 1 Processor -- start 2020-11-28T21:56:30.231 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.222+0000 7f7a60acf700 1 -- start start 2020-11-28T21:56:30.231 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.222+0000 7f7a60acf700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7a5c067610 0x7f7a5c0ad030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:30.232 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.222+0000 7f7a60acf700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a5c0b9ce0 0x7f7a5c13d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:30.232 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.222+0000 7f7a60acf700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a5c1433c0 0x7f7a5c152250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:30.232 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.222+0000 7f7a60acf700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c0ad660 con 0x7f7a5c1433c0 2020-11-28T21:56:30.232 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.222+0000 7f7a60acf700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c151870 con 0x7f7a5c0b9ce0 2020-11-28T21:56:30.233 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.222+0000 7f7a60acf700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c151b10 con 0x7f7a5c067610 2020-11-28T21:56:30.233 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.222+0000 7f7a59d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a5c0b9ce0 0x7f7a5c13d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:30.233 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.222+0000 7f7a59d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a5c0b9ce0 0x7f7a5c13d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46476/0 (socket says 172.21.15.162:46476) 2020-11-28T21:56:30.233 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.222+0000 7f7a59d9b700 1 -- 172.21.15.162:0/1090827941 learned_addr learned my addr 172.21.15.162:0/1090827941 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:30.233 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.222+0000 7f7a5a59c700 1 --2- 172.21.15.162:0/1090827941 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7a5c067610 0x7f7a5c0ad030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:30.234 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.222+0000 7f7a5ad9d700 1 --2- 172.21.15.162:0/1090827941 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a5c1433c0 0x7f7a5c152250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:30.234 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.222+0000 7f7a59d9b700 1 -- 172.21.15.162:0/1090827941 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7a5c067610 msgr2=0x7f7a5c0ad030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:30.234 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.222+0000 7f7a59d9b700 1 --2- 172.21.15.162:0/1090827941 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7a5c067610 0x7f7a5c0ad030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:30.234 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.222+0000 7f7a59d9b700 1 -- 172.21.15.162:0/1090827941 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a5c1433c0 msgr2=0x7f7a5c152250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:30.235 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.222+0000 7f7a59d9b700 1 --2- 172.21.15.162:0/1090827941 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a5c1433c0 0x7f7a5c152250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:30.235 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.222+0000 7f7a59d9b700 1 -- 172.21.15.162:0/1090827941 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5c151db0 con 0x7f7a5c0b9ce0 2020-11-28T21:56:30.235 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.222+0000 7f7a59d9b700 1 --2- 172.21.15.162:0/1090827941 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a5c0b9ce0 0x7f7a5c13d510 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 rx=0x7f7a4c002f30 tx=0x7f7a4c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:30.235 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.222+0000 7f7a537fe700 1 -- 172.21.15.162:0/1090827941 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a4c0118d0 con 0x7f7a5c0b9ce0 2020-11-28T21:56:30.235 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.222+0000 7f7a537fe700 1 -- 172.21.15.162:0/1090827941 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4c011a30 con 0x7f7a5c0b9ce0 2020-11-28T21:56:30.236 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.222+0000 7f7a60acf700 1 -- 172.21.15.162:0/1090827941 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a5c0b1b70 con 0x7f7a5c0b9ce0 2020-11-28T21:56:30.236 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.222+0000 7f7a537fe700 1 -- 172.21.15.162:0/1090827941 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a4c011d90 con 0x7f7a5c0b9ce0 2020-11-28T21:56:30.236 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.222+0000 7f7a60acf700 1 -- 172.21.15.162:0/1090827941 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a5c0b0100 con 0x7f7a5c0b9ce0 2020-11-28T21:56:30.236 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.226+0000 7f7a60acf700 1 -- 172.21.15.162:0/1090827941 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a5c057050 con 0x7f7a5c0b9ce0 2020-11-28T21:56:30.237 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.226+0000 7f7a537fe700 1 -- 172.21.15.162:0/1090827941 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f7a4c017070 con 0x7f7a5c0b9ce0 2020-11-28T21:56:30.237 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.226+0000 7f7a537fe700 1 --2- 172.21.15.162:0/1090827941 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7a400554e0 0x7f7a400579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:30.237 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.226+0000 7f7a537fe700 1 -- 172.21.15.162:0/1090827941 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7a4c06e990 con 0x7f7a5c0b9ce0 2020-11-28T21:56:30.238 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.226+0000 7f7a5a59c700 1 --2- 172.21.15.162:0/1090827941 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7a400554e0 0x7f7a400579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:30.238 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.226+0000 7f7a5a59c700 1 --2- 172.21.15.162:0/1090827941 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7a400554e0 0x7f7a400579a0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7f7a4400d000 tx=0x7f7a44009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:30.239 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.234+0000 7f7a537fe700 1 -- 172.21.15.162:0/1090827941 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f7a4c044c90 con 0x7f7a5c0b9ce0 2020-11-28T21:56:30.383 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.374+0000 7f7a60acf700 1 -- 172.21.15.162:0/1090827941 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7a5c0b0100 con 0x7f7a5c0b9ce0 2020-11-28T21:56:30.383 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.374+0000 7f7a537fe700 1 -- 172.21.15.162:0/1090827941 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v160) v1 ==== 100+0+10 (secure 0 0 0) 0x7f7a5c0b0100 con 0x7f7a5c0b9ce0 2020-11-28T21:56:30.384 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.378+0000 7f7a60acf700 1 -- 172.21.15.162:0/1090827941 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7a400554e0 msgr2=0x7f7a400579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:30.384 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.378+0000 7f7a60acf700 1 --2- 172.21.15.162:0/1090827941 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7a400554e0 0x7f7a400579a0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7f7a4400d000 tx=0x7f7a44009040).stop 2020-11-28T21:56:30.384 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.378+0000 7f7a60acf700 1 -- 172.21.15.162:0/1090827941 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a5c0b9ce0 msgr2=0x7f7a5c13d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:30.384 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.378+0000 7f7a60acf700 1 --2- 172.21.15.162:0/1090827941 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a5c0b9ce0 0x7f7a5c13d510 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 rx=0x7f7a4c002f30 tx=0x7f7a4c015000).stop 2020-11-28T21:56:30.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.378+0000 7f7a60acf700 1 -- 172.21.15.162:0/1090827941 shutdown_connections 2020-11-28T21:56:30.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.378+0000 7f7a60acf700 1 --2- 172.21.15.162:0/1090827941 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7a400554e0 0x7f7a400579a0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:30.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.378+0000 7f7a60acf700 1 --2- 172.21.15.162:0/1090827941 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7a5c067610 0x7f7a5c0ad030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:30.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.378+0000 7f7a60acf700 1 --2- 172.21.15.162:0/1090827941 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a5c1433c0 0x7f7a5c152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:30.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.378+0000 7f7a60acf700 1 --2- 172.21.15.162:0/1090827941 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a5c0b9ce0 0x7f7a5c13d510 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:30.386 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.378+0000 7f7a60acf700 1 -- 172.21.15.162:0/1090827941 >> 172.21.15.162:0/1090827941 conn(0x7f7a5c05c430 msgr2=0x7f7a5c13f200 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:30.386 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.378+0000 7f7a60acf700 1 -- 172.21.15.162:0/1090827941 shutdown_connections 2020-11-28T21:56:30.386 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.378+0000 7f7a60acf700 1 -- 172.21.15.162:0/1090827941 wait complete. 2020-11-28T21:56:30.386 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:56:30.395 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-28T21:56:30.397 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:56:30.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.498+0000 7fc175f64700 1 Processor -- start 2020-11-28T21:56:30.504 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.498+0000 7fc175f64700 1 -- start start 2020-11-28T21:56:30.507 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.498+0000 7fc175f64700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc170098b30 0x7fc1700994f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:30.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.498+0000 7fc175f64700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc170060710 0x7fc1700b1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:30.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.498+0000 7fc175f64700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc170098f10 con 0x7fc1700b0000 2020-11-28T21:56:30.508 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.498+0000 7fc175f64700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1700b1030 con 0x7fc170060710 2020-11-28T21:56:30.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.498+0000 7fc175f64700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1700b0480 con 0x7fc170098b30 2020-11-28T21:56:30.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.498+0000 7fc175763700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc170060710 0x7fc1700b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:30.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.498+0000 7fc175763700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc170060710 0x7fc1700b1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46486/0 (socket says 172.21.15.162:46486) 2020-11-28T21:56:30.509 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.498+0000 7fc175763700 1 -- 172.21.15.162:0/738890768 learned_addr learned my addr 172.21.15.162:0/738890768 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:30.510 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.502+0000 7fc174f62700 1 --2- 172.21.15.162:0/738890768 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc170098b30 0x7fc1700994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:30.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.502+0000 7fc175763700 1 -- 172.21.15.162:0/738890768 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc170098b30 msgr2=0x7fc1700994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:30.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.502+0000 7fc175763700 1 --2- 172.21.15.162:0/738890768 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc170098b30 0x7fc1700994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:30.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.502+0000 7fc175763700 1 -- 172.21.15.162:0/738890768 >> v1:172.21.15.118:6789/0 conn(0x7fc1700b0000 legacy=0x7fc170060000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:30.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.502+0000 7fc175763700 1 -- 172.21.15.162:0/738890768 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1700b0680 con 0x7fc170060710 2020-11-28T21:56:30.512 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.502+0000 7fc175763700 1 --2- 172.21.15.162:0/738890768 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc170060710 0x7fc1700b1580 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 rx=0x7fc170099be0 tx=0x7fc168009270).ready entity=mon.1 client_cookie=a18a4b8376a08fc1 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:30.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.502+0000 7fc16f7fe700 1 -- 172.21.15.162:0/738890768 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc168009e90 con 0x7fc170060710 2020-11-28T21:56:30.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.502+0000 7fc16f7fe700 1 -- 172.21.15.162:0/738890768 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16800bbc0 con 0x7fc170060710 2020-11-28T21:56:30.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.502+0000 7fc16f7fe700 1 -- 172.21.15.162:0/738890768 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc16800bd30 con 0x7fc170060710 2020-11-28T21:56:30.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.502+0000 7fc175f64700 1 -- 172.21.15.162:0/738890768 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc170060710 msgr2=0x7fc1700b1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:30.513 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.502+0000 7fc175f64700 1 --2- 172.21.15.162:0/738890768 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc170060710 0x7fc1700b1580 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 rx=0x7fc170099be0 tx=0x7fc168009270).stop 2020-11-28T21:56:30.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.502+0000 7fc175f64700 1 -- 172.21.15.162:0/738890768 shutdown_connections 2020-11-28T21:56:30.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.502+0000 7fc175f64700 1 --2- 172.21.15.162:0/738890768 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc170060710 0x7fc1700b1580 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:30.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.502+0000 7fc175f64700 1 --2- 172.21.15.162:0/738890768 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc170098b30 0x7fc1700994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:30.514 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.502+0000 7fc175f64700 1 -- 172.21.15.162:0/738890768 >> 172.21.15.162:0/738890768 conn(0x7fc17005b970 msgr2=0x7fc17005bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:30.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.502+0000 7fc175f64700 1 -- 172.21.15.162:0/738890768 shutdown_connections 2020-11-28T21:56:30.515 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.502+0000 7fc175f64700 1 -- 172.21.15.162:0/738890768 wait complete. 2020-11-28T21:56:30.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.502+0000 7fc175f64700 1 Processor -- start 2020-11-28T21:56:30.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.502+0000 7fc175f64700 1 -- start start 2020-11-28T21:56:30.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.506+0000 7fc175f64700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc170060710 0x7fc1700add60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:30.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.506+0000 7fc175f64700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc170098b30 0x7fc1701566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:30.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.506+0000 7fc175f64700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc1700b0000 0x7fc170063d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:30.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.506+0000 7fc175f64700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc170156b90 con 0x7fc1700b0000 2020-11-28T21:56:30.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.506+0000 7fc175f64700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1700ad970 con 0x7fc170098b30 2020-11-28T21:56:30.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.506+0000 7fc175f64700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc170064250 con 0x7fc170060710 2020-11-28T21:56:30.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.506+0000 7fc16ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc170098b30 0x7fc1701566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:30.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.506+0000 7fc16ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc170098b30 0x7fc1701566a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46490/0 (socket says 172.21.15.162:46490) 2020-11-28T21:56:30.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.506+0000 7fc16ffff700 1 -- 172.21.15.162:0/3290671433 learned_addr learned my addr 172.21.15.162:0/3290671433 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:30.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.506+0000 7fc174f62700 1 --2- 172.21.15.162:0/3290671433 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc170060710 0x7fc1700add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:30.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.506+0000 7fc175763700 1 --2- 172.21.15.162:0/3290671433 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc1700b0000 0x7fc170063d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:30.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.506+0000 7fc16ffff700 1 -- 172.21.15.162:0/3290671433 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc170060710 msgr2=0x7fc1700add60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:30.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.506+0000 7fc16ffff700 1 --2- 172.21.15.162:0/3290671433 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc170060710 0x7fc1700add60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:30.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.506+0000 7fc16ffff700 1 -- 172.21.15.162:0/3290671433 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc1700b0000 msgr2=0x7fc170063d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:30.521 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.506+0000 7fc16ffff700 1 --2- 172.21.15.162:0/3290671433 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc1700b0000 0x7fc170063d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:30.521 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.506+0000 7fc16ffff700 1 -- 172.21.15.162:0/3290671433 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1700ae490 con 0x7fc170098b30 2020-11-28T21:56:30.521 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.506+0000 7fc16ffff700 1 --2- 172.21.15.162:0/3290671433 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc170098b30 0x7fc1701566a0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 rx=0x7fc17005cc70 tx=0x7fc160007c20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:30.521 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.506+0000 7fc16dffb700 1 -- 172.21.15.162:0/3290671433 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc160013410 con 0x7fc170098b30 2020-11-28T21:56:30.522 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.506+0000 7fc16dffb700 1 -- 172.21.15.162:0/3290671433 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc160013570 con 0x7fc170098b30 2020-11-28T21:56:30.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.506+0000 7fc175f64700 1 -- 172.21.15.162:0/3290671433 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1700ae6c0 con 0x7fc170098b30 2020-11-28T21:56:30.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.506+0000 7fc16dffb700 1 -- 172.21.15.162:0/3290671433 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1600138d0 con 0x7fc170098b30 2020-11-28T21:56:30.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.506+0000 7fc175f64700 1 -- 172.21.15.162:0/3290671433 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc170063340 con 0x7fc170098b30 2020-11-28T21:56:30.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.510+0000 7fc175f64700 1 -- 172.21.15.162:0/3290671433 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc170057050 con 0x7fc170098b30 2020-11-28T21:56:30.523 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.510+0000 7fc16dffb700 1 -- 172.21.15.162:0/3290671433 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc160010030 con 0x7fc170098b30 2020-11-28T21:56:30.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.510+0000 7fc16dffb700 1 --2- 172.21.15.162:0/3290671433 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc148055330 0x7fc1480577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:30.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.510+0000 7fc16dffb700 1 -- 172.21.15.162:0/3290671433 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc16006db30 con 0x7fc170098b30 2020-11-28T21:56:30.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.510+0000 7fc174f62700 1 --2- 172.21.15.162:0/3290671433 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc148055330 0x7fc1480577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:30.524 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.510+0000 7fc174f62700 1 --2- 172.21.15.162:0/3290671433 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc148055330 0x7fc1480577f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7fc15800b000 tx=0x7fc158006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:30.525 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.514+0000 7fc16dffb700 1 -- 172.21.15.162:0/3290671433 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc160043e30 con 0x7fc170098b30 2020-11-28T21:56:30.702 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.698+0000 7fc175f64700 1 -- 172.21.15.162:0/3290671433 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc170068350 con 0x7fc170098b30 2020-11-28T21:56:30.703 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.698+0000 7fc16dffb700 1 -- 172.21.15.162:0/3290671433 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+2501 (secure 0 0 0) 0x7fc160001e60 con 0x7fc170098b30 2020-11-28T21:56:30.703 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:56:30.704 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":80,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8199,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/2385081970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":2385081970},{"type":"v1","addr":"172.21.15.118:6833","nonce":2385081970}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":77},{"gid":8554,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6836/169306274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":169306274},{"type":"v1","addr":"172.21.15.162:6836","nonce":169306274}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":78}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:56:25.831016+0000","modified":"2020-11-28T21:56:26.833832+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8557},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8557":{"gid":8557,"name":"c","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/3753662046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":3753662046},{"type":"v1","addr":"172.21.15.162:6837","nonce":3753662046}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2020-11-28T21:56:30.705 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.698+0000 7fc175f64700 1 -- 172.21.15.162:0/3290671433 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc148055330 msgr2=0x7fc1480577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:30.705 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.698+0000 7fc175f64700 1 --2- 172.21.15.162:0/3290671433 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc148055330 0x7fc1480577f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7fc15800b000 tx=0x7fc158006dc0).stop 2020-11-28T21:56:30.705 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.698+0000 7fc175f64700 1 -- 172.21.15.162:0/3290671433 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc170098b30 msgr2=0x7fc1701566a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:30.706 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.698+0000 7fc175f64700 1 --2- 172.21.15.162:0/3290671433 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc170098b30 0x7fc1701566a0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 rx=0x7fc17005cc70 tx=0x7fc160007c20).stop 2020-11-28T21:56:30.706 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.698+0000 7fc175f64700 1 -- 172.21.15.162:0/3290671433 shutdown_connections 2020-11-28T21:56:30.706 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.698+0000 7fc175f64700 1 --2- 172.21.15.162:0/3290671433 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc148055330 0x7fc1480577f0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:30.706 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.698+0000 7fc175f64700 1 --2- 172.21.15.162:0/3290671433 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc170060710 0x7fc1700add60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:30.706 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.698+0000 7fc175f64700 1 --2- 172.21.15.162:0/3290671433 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc1700b0000 0x7fc170063d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:30.707 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.698+0000 7fc175f64700 1 --2- 172.21.15.162:0/3290671433 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc170098b30 0x7fc1701566a0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:30.707 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.698+0000 7fc175f64700 1 -- 172.21.15.162:0/3290671433 >> 172.21.15.162:0/3290671433 conn(0x7fc17005b970 msgr2=0x7fc17005bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:30.707 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.698+0000 7fc175f64700 1 -- 172.21.15.162:0/3290671433 shutdown_connections 2020-11-28T21:56:30.707 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:30.698+0000 7fc175f64700 1 -- 172.21.15.162:0/3290671433 wait complete. 2020-11-28T21:56:30.708 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 80 2020-11-28T21:56:31.019 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 80, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-28T21:56:25.831016+0000', 'modified': '2020-11-28T21:56:26.833832+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8557}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8557': {'gid': 8557, 'name': 'c', 'rank': 0, 'incarnation': 79, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6837/3753662046', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6835', 'nonce': 3753662046}, {'type': 'v1', 'addr': '172.21.15.162:6837', 'nonce': 3753662046}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-28T21:56:31.020 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-28T21:56:31.021 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-28T21:56:31.148 INFO:teuthology.orchestra.run.smithi162.stdout:{"cluster_fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","whoami":0,"id":8557,"want_state":"up:active","state":"up:active","rank_uptime":5.303343377,"mdsmap_epoch":80,"osdmap_epoch":160,"osdmap_epoch_barrier":158,"uptime":10.717752169000001} 2020-11-28T21:56:31.148 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", "id": 8557, "mdsmap_epoch": 80, "osdmap_epoch": 160, "osdmap_epoch_barrier": 158, "rank_uptime": 5.303343377, "state": "up:active", "uptime": 10.717752169, "want_state": "up:active", "whoami": 0 } 2020-11-28T21:56:31.149 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-11-28T21:56:31.149 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-11-28T21:56:31.150 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi192.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-11-28T21:56:31.150 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:56:31.150 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:56:31.158 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:56:31.219 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:56:31.220 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:56:31.274 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:56:31.276 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:56:31.277 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:56:31.278 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:56:31.326 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-11-28T21:56:31.327 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:56:31.374 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:56:31.391 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:56:31.392 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:56:31.447 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:56:31.448 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:56:31.448 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:56:31.449 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:56:31.450 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:2020-11-28T21:56:31.443+0000 7f5511df50c0 -1 init, newargv = 0x55a86ad02420 newargc=9 2020-11-28T21:56:31.451 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:ceph-fuse[12811]: starting ceph client 2020-11-28T21:56:31.461 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi192.stderr:ceph-fuse[12811]: starting fuse 2020-11-28T21:56:32.558 INFO:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2020-11-28T21:56:32.575 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:56:32.575 INFO:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:56:32.630 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:56:32.631 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:56:32.632 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:56:32.632 INFO:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2020-11-28T21:56:32.677 INFO:teuthology.orchestra.run.smithi192.stdout:49 2020-11-28T21:56:32.678 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2020-11-28T21:56:32.678 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:56:32.679 INFO:teuthology.orchestra.run.smithi192:> sudo python3 -c ' 2020-11-28T21:56:32.679 INFO:teuthology.orchestra.run.smithi192:> import glob 2020-11-28T21:56:32.680 INFO:teuthology.orchestra.run.smithi192:> import re 2020-11-28T21:56:32.680 INFO:teuthology.orchestra.run.smithi192:> import os 2020-11-28T21:56:32.681 INFO:teuthology.orchestra.run.smithi192:> import subprocess 2020-11-28T21:56:32.681 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:56:32.681 INFO:teuthology.orchestra.run.smithi192:> def find_socket(client_name): 2020-11-28T21:56:32.682 INFO:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-11-28T21:56:32.682 INFO:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2020-11-28T21:56:32.683 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:56:32.683 INFO:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2020-11-28T21:56:32.684 INFO:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2020-11-28T21:56:32.684 INFO:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2020-11-28T21:56:32.684 INFO:teuthology.orchestra.run.smithi192:> return files[0] 2020-11-28T21:56:32.685 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:56:32.685 INFO:teuthology.orchestra.run.smithi192:> for f in files: 2020-11-28T21:56:32.686 INFO:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-11-28T21:56:32.686 INFO:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2020-11-28T21:56:32.686 INFO:teuthology.orchestra.run.smithi192:> return f 2020-11-28T21:56:32.687 INFO:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-11-28T21:56:32.688 INFO:teuthology.orchestra.run.smithi192:> 2020-11-28T21:56:32.689 INFO:teuthology.orchestra.run.smithi192:> print(find_socket("client.0")) 2020-11-28T21:56:32.689 INFO:teuthology.orchestra.run.smithi192:> ' 2020-11-28T21:56:32.768 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.0.12811.asok 2020-11-28T21:56:32.773 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.12811.asok 2020-11-28T21:56:32.775 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:56:32.775 INFO:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.12811.asok status 2020-11-28T21:56:32.879 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2020-11-28T21:56:32.880 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2020-11-28T21:56:32.880 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "a83a3572a957356977342f461361c1cd3ec06dbd", 2020-11-28T21:56:32.880 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 15.2.6-595-ga83a3572 (a83a3572a957356977342f461361c1cd3ec06dbd) octopus (stable)", 2020-11-28T21:56:32.880 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "0", 2020-11-28T21:56:32.881 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2020-11-28T21:56:32.881 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-11-28T21:56:32.881 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "12811", 2020-11-28T21:56:32.881 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2020-11-28T21:56:32.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:56:32.882 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2020-11-28T21:56:32.882 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2020-11-28T21:56:32.882 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 8346, 2020-11-28T21:56:32.882 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2020-11-28T21:56:32.882 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2020-11-28T21:56:32.883 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2020-11-28T21:56:32.883 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 8346 2020-11-28T21:56:32.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:56:32.883 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2020-11-28T21:56:32.883 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2020-11-28T21:56:32.884 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2020-11-28T21:56:32.884 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 3098967821 2020-11-28T21:56:32.884 INFO:teuthology.orchestra.run.smithi192.stdout: } 2020-11-28T21:56:32.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:56:32.885 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2020-11-28T21:56:32.885 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2020-11-28T21:56:32.885 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2020-11-28T21:56:32.885 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 3098967821 2020-11-28T21:56:32.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2020-11-28T21:56:32.886 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.8346 172.21.15.192:0/3098967821", 2020-11-28T21:56:32.886 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "172.21.15.192:0/3098967821", 2020-11-28T21:56:32.886 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2020-11-28T21:56:32.887 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 80, 2020-11-28T21:56:32.887 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 160, 2020-11-28T21:56:32.887 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2020-11-28T21:56:32.887 INFO:teuthology.orchestra.run.smithi192.stdout: "blacklisted": false 2020-11-28T21:56:32.887 INFO:teuthology.orchestra.run.smithi192.stdout:} 2020-11-28T21:56:32.889 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:56:32.890 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:56:32.890 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:56:32.897 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2020-11-28T21:56:32.898 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-28T21:56:32.899 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:56:32.899 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:56:32.957 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T21:56:33.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.070+0000 7f71ca2ae700 1 Processor -- start 2020-11-28T21:56:33.077 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.070+0000 7f71ca2ae700 1 -- start start 2020-11-28T21:56:33.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.074+0000 7f71ca2ae700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f71c4060cf0 0x7f71c40aa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:33.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.074+0000 7f71ca2ae700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f71c405cbd0 0x7f71c40aa9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:33.078 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.074+0000 7f71ca2ae700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71c40b0230 con 0x7f71c40af640 2020-11-28T21:56:33.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.074+0000 7f71ca2ae700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71c40b12a0 con 0x7f71c4060cf0 2020-11-28T21:56:33.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.074+0000 7f71ca2ae700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c40610d0 con 0x7f71c405cbd0 2020-11-28T21:56:33.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.074+0000 7f71c8aab700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f71c405cbd0 0x7f71c40aa9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:33.079 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.074+0000 7f71c8aab700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f71c405cbd0 0x7f71c40aa9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:43720/0 (socket says 172.21.15.162:43720) 2020-11-28T21:56:33.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.074+0000 7f71c8aab700 1 -- 172.21.15.162:0/3875396631 learned_addr learned my addr 172.21.15.162:0/3875396631 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:33.080 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.074+0000 7f71c92ac700 1 --2- 172.21.15.162:0/3875396631 >> v2:172.21.15.118:3300/0 conn(0x7f71c4060cf0 0x7f71c40aa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:33.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.074+0000 7f71c8aab700 1 -- 172.21.15.162:0/3875396631 >> v2:172.21.15.118:3300/0 conn(0x7f71c4060cf0 msgr2=0x7f71c40aa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:33.081 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.074+0000 7f71c8aab700 1 --2- 172.21.15.162:0/3875396631 >> v2:172.21.15.118:3300/0 conn(0x7f71c4060cf0 0x7f71c40aa4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:33.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.074+0000 7f71c8aab700 1 -- 172.21.15.162:0/3875396631 >> v1:172.21.15.118:6789/0 conn(0x7f71c40af640 legacy=0x7f71c40ae1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:33.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.074+0000 7f71c8aab700 1 -- 172.21.15.162:0/3875396631 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c40a8c00 con 0x7f71c405cbd0 2020-11-28T21:56:33.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.074+0000 7f71c8aab700 1 --2- 172.21.15.162:0/3875396631 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f71c405cbd0 0x7f71c40aa9e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f71b4002f30 tx=0x7f71b4006e80).ready entity=mon.2 client_cookie=f7fd7722e33f29a1 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:33.082 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.074+0000 7f71b3fff700 1 -- 172.21.15.162:0/3875396631 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71b40097f0 con 0x7f71c405cbd0 2020-11-28T21:56:33.083 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.074+0000 7f71b3fff700 1 -- 172.21.15.162:0/3875396631 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b4009950 con 0x7f71c405cbd0 2020-11-28T21:56:33.083 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.074+0000 7f71b3fff700 1 -- 172.21.15.162:0/3875396631 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71b4009cb0 con 0x7f71c405cbd0 2020-11-28T21:56:33.083 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.074+0000 7f71ca2ae700 1 -- 172.21.15.162:0/3875396631 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f71c405cbd0 msgr2=0x7f71c40aa9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:33.083 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.074+0000 7f71ca2ae700 1 --2- 172.21.15.162:0/3875396631 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f71c405cbd0 0x7f71c40aa9e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f71b4002f30 tx=0x7f71b4006e80).stop 2020-11-28T21:56:33.083 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.074+0000 7f71ca2ae700 1 -- 172.21.15.162:0/3875396631 shutdown_connections 2020-11-28T21:56:33.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.074+0000 7f71ca2ae700 1 --2- 172.21.15.162:0/3875396631 >> v2:172.21.15.118:3300/0 conn(0x7f71c4060cf0 0x7f71c40aa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:33.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.074+0000 7f71ca2ae700 1 --2- 172.21.15.162:0/3875396631 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f71c405cbd0 0x7f71c40aa9e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:33.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.074+0000 7f71ca2ae700 1 -- 172.21.15.162:0/3875396631 >> 172.21.15.162:0/3875396631 conn(0x7f71c405b970 msgr2=0x7f71c405bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:33.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.074+0000 7f71ca2ae700 1 -- 172.21.15.162:0/3875396631 shutdown_connections 2020-11-28T21:56:33.084 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.074+0000 7f71ca2ae700 1 -- 172.21.15.162:0/3875396631 wait complete. 2020-11-28T21:56:33.086 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.078+0000 7f71ca2ae700 1 Processor -- start 2020-11-28T21:56:33.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.078+0000 7f71ca2ae700 1 -- start start 2020-11-28T21:56:33.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.078+0000 7f71ca2ae700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f71c405cbd0 0x7f71c40b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:33.087 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.078+0000 7f71ca2ae700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f71c4060cf0 0x7f71c4153200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:33.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.078+0000 7f71ca2ae700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71c40af640 0x7f71c41536f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:33.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.078+0000 7f71ca2ae700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c4146770 con 0x7f71c4060cf0 2020-11-28T21:56:33.088 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.078+0000 7f71ca2ae700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c4065500 con 0x7f71c40af640 2020-11-28T21:56:33.089 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.078+0000 7f71ca2ae700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c40657a0 con 0x7f71c405cbd0 2020-11-28T21:56:33.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.078+0000 7f71c9aad700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71c40af640 0x7f71c41536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:33.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.078+0000 7f71c9aad700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71c40af640 0x7f71c41536f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46506/0 (socket says 172.21.15.162:46506) 2020-11-28T21:56:33.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.078+0000 7f71c9aad700 1 -- 172.21.15.162:0/2051831975 learned_addr learned my addr 172.21.15.162:0/2051831975 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:33.090 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.078+0000 7f71c92ac700 1 --2- 172.21.15.162:0/2051831975 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f71c405cbd0 0x7f71c40b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:33.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.078+0000 7f71c8aab700 1 --2- 172.21.15.162:0/2051831975 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f71c4060cf0 0x7f71c4153200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:33.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.082+0000 7f71c9aad700 1 -- 172.21.15.162:0/2051831975 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f71c405cbd0 msgr2=0x7f71c40b36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:33.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.082+0000 7f71c9aad700 1 --2- 172.21.15.162:0/2051831975 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f71c405cbd0 0x7f71c40b36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:33.091 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.082+0000 7f71c9aad700 1 -- 172.21.15.162:0/2051831975 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f71c4060cf0 msgr2=0x7f71c4153200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:33.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.082+0000 7f71c9aad700 1 --2- 172.21.15.162:0/2051831975 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f71c4060cf0 0x7f71c4153200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:33.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.082+0000 7f71c9aad700 1 -- 172.21.15.162:0/2051831975 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c4065a40 con 0x7f71c40af640 2020-11-28T21:56:33.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.082+0000 7f71c9aad700 1 --2- 172.21.15.162:0/2051831975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71c40af640 0x7f71c41536f0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 rx=0x7f71c4067980 tx=0x7f71bc007f70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:33.092 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.082+0000 7f71b27fc700 1 -- 172.21.15.162:0/2051831975 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71bc003080 con 0x7f71c40af640 2020-11-28T21:56:33.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.082+0000 7f71ca2ae700 1 -- 172.21.15.162:0/2051831975 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c4067460 con 0x7f71c40af640 2020-11-28T21:56:33.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.082+0000 7f71b27fc700 1 -- 172.21.15.162:0/2051831975 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71bc00a7b0 con 0x7f71c40af640 2020-11-28T21:56:33.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.082+0000 7f71ca2ae700 1 -- 172.21.15.162:0/2051831975 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c405fc00 con 0x7f71c40af640 2020-11-28T21:56:33.093 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.082+0000 7f71b27fc700 1 -- 172.21.15.162:0/2051831975 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71bc00ab50 con 0x7f71c40af640 2020-11-28T21:56:33.094 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.082+0000 7f71ca2ae700 1 -- 172.21.15.162:0/2051831975 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71c4057050 con 0x7f71c40af640 2020-11-28T21:56:33.095 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.082+0000 7f71b27fc700 1 -- 172.21.15.162:0/2051831975 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f71bc010070 con 0x7f71c40af640 2020-11-28T21:56:33.095 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.082+0000 7f71b27fc700 1 --2- 172.21.15.162:0/2051831975 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f71a4055440 0x7f71a4057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:33.095 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.082+0000 7f71b27fc700 1 -- 172.21.15.162:0/2051831975 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f71bc06d930 con 0x7f71c40af640 2020-11-28T21:56:33.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.082+0000 7f71c92ac700 1 --2- 172.21.15.162:0/2051831975 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f71a4055440 0x7f71a4057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:33.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.082+0000 7f71c92ac700 1 --2- 172.21.15.162:0/2051831975 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f71a4055440 0x7f71a4057900 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7f71c000b000 tx=0x7f71c0006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:33.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.090+0000 7f71b27fc700 1 -- 172.21.15.162:0/2051831975 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f71bc043b20 con 0x7f71c40af640 2020-11-28T21:56:33.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.234+0000 7f71ca2ae700 1 -- 172.21.15.162:0/2051831975 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71c40b0010 con 0x7f71c40af640 2020-11-28T21:56:33.242 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.238+0000 7f71b27fc700 1 -- 172.21.15.162:0/2051831975 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v160) v1 ==== 74+0+16148 (secure 0 0 0) 0x7f71bc047140 con 0x7f71c40af640 2020-11-28T21:56:33.242 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:56:33.243 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":160,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:56:27.855579+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2020-11-28T21:56:23.335325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2020-11-28T21:56:24.648832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T21:56:33.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.238+0000 7f71ca2ae700 1 -- 172.21.15.162:0/2051831975 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f71a4055440 msgr2=0x7f71a4057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:33.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.238+0000 7f71ca2ae700 1 --2- 172.21.15.162:0/2051831975 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f71a4055440 0x7f71a4057900 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7f71c000b000 tx=0x7f71c0006dc0).stop 2020-11-28T21:56:33.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.238+0000 7f71ca2ae700 1 -- 172.21.15.162:0/2051831975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71c40af640 msgr2=0x7f71c41536f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:33.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.238+0000 7f71ca2ae700 1 --2- 172.21.15.162:0/2051831975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71c40af640 0x7f71c41536f0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 rx=0x7f71c4067980 tx=0x7f71bc007f70).stop 2020-11-28T21:56:33.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.238+0000 7f71ca2ae700 1 -- 172.21.15.162:0/2051831975 shutdown_connections 2020-11-28T21:56:33.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.238+0000 7f71ca2ae700 1 --2- 172.21.15.162:0/2051831975 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f71a4055440 0x7f71a4057900 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:33.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.238+0000 7f71ca2ae700 1 --2- 172.21.15.162:0/2051831975 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f71c405cbd0 0x7f71c40b36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:33.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.238+0000 7f71ca2ae700 1 --2- 172.21.15.162:0/2051831975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71c40af640 0x7f71c41536f0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:33.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.238+0000 7f71ca2ae700 1 --2- 172.21.15.162:0/2051831975 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f71c4060cf0 0x7f71c4153200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:33.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.238+0000 7f71ca2ae700 1 -- 172.21.15.162:0/2051831975 >> 172.21.15.162:0/2051831975 conn(0x7f71c405b970 msgr2=0x7f71c4060100 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:33.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.238+0000 7f71ca2ae700 1 -- 172.21.15.162:0/2051831975 shutdown_connections 2020-11-28T21:56:33.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.238+0000 7f71ca2ae700 1 -- 172.21.15.162:0/2051831975 wait complete. 2020-11-28T21:56:33.256 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-28T21:46:15.639551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-28T21:46:23.901366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 29, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-28T21:56:23.335325+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '159', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 30, 'pool_name': 'cephfs_data', 'create_time': '2020-11-28T21:56:24.648832+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '160', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-28T21:56:33.257 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-28T21:56:33.378 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.374+0000 7fbb567ad700 1 Processor -- start 2020-11-28T21:56:33.378 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.374+0000 7fbb567ad700 1 -- start start 2020-11-28T21:56:33.381 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.374+0000 7fbb567ad700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbb500af6a0 0x7fbb500b2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:33.381 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.374+0000 7fbb567ad700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbb50060a70 0x7fbb500afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:33.381 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.374+0000 7fbb567ad700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb50060f00 con 0x7fbb500b1eb0 2020-11-28T21:56:33.381 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.374+0000 7fbb567ad700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb501471f0 con 0x7fbb50060a70 2020-11-28T21:56:33.382 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.374+0000 7fbb567ad700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb50060090 con 0x7fbb500af6a0 2020-11-28T21:56:33.382 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.374+0000 7fbb55fac700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbb50060a70 0x7fbb500afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:33.382 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.374+0000 7fbb557ab700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fbb500b1eb0 0x7fbb50144350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:53354/0 (socket says 172.21.15.162:53354) 2020-11-28T21:56:33.382 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.374+0000 7fbb557ab700 1 -- 172.21.15.162:0/2193287578 learned_addr learned my addr 172.21.15.162:0/2193287578 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:56:33.383 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.374+0000 7fbb54faa700 1 --2- 172.21.15.162:0/2193287578 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbb500af6a0 0x7fbb500b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:33.383 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.374+0000 7fbb55fac700 1 -- 172.21.15.162:0/2193287578 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbb500af6a0 msgr2=0x7fbb500b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:33.383 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.374+0000 7fbb55fac700 1 --2- 172.21.15.162:0/2193287578 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbb500af6a0 0x7fbb500b2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:33.383 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.374+0000 7fbb55fac700 1 -- 172.21.15.162:0/2193287578 >> v1:172.21.15.162:6789/0 conn(0x7fbb500b1eb0 legacy=0x7fbb50144350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:33.384 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.374+0000 7fbb55fac700 1 -- 172.21.15.162:0/2193287578 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb50060330 con 0x7fbb50060a70 2020-11-28T21:56:33.384 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.374+0000 7fbb47fff700 1 -- 172.21.15.162:0/2193287578 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 22505274 0 0) 0x7fbb50060f00 con 0x7fbb500b1eb0 2020-11-28T21:56:33.384 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.374+0000 7fbb55fac700 1 --2- 172.21.15.162:0/2193287578 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb50060a70 0x7fbb500afdc0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 rx=0x7fbb480032b0 tx=0x7fbb480091a0).ready entity=mon.1 client_cookie=f3e0a2bff23b17cb server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:33.384 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.374+0000 7fbb47fff700 1 -- 172.21.15.162:0/2193287578 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb4800ba50 con 0x7fbb50060a70 2020-11-28T21:56:33.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.374+0000 7fbb47fff700 1 -- 172.21.15.162:0/2193287578 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb4800bbb0 con 0x7fbb50060a70 2020-11-28T21:56:33.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.374+0000 7fbb47fff700 1 -- 172.21.15.162:0/2193287578 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb4800bd20 con 0x7fbb50060a70 2020-11-28T21:56:33.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.374+0000 7fbb567ad700 1 -- 172.21.15.162:0/2193287578 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb50060a70 msgr2=0x7fbb500afdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:33.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.374+0000 7fbb567ad700 1 --2- 172.21.15.162:0/2193287578 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb50060a70 0x7fbb500afdc0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 rx=0x7fbb480032b0 tx=0x7fbb480091a0).stop 2020-11-28T21:56:33.386 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.374+0000 7fbb567ad700 1 -- 172.21.15.162:0/2193287578 shutdown_connections 2020-11-28T21:56:33.386 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.374+0000 7fbb567ad700 1 --2- 172.21.15.162:0/2193287578 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb50060a70 0x7fbb500afdc0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:33.386 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.374+0000 7fbb567ad700 1 --2- 172.21.15.162:0/2193287578 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbb500af6a0 0x7fbb500b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:33.386 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.374+0000 7fbb567ad700 1 -- 172.21.15.162:0/2193287578 >> 172.21.15.162:0/2193287578 conn(0x7fbb500b57d0 msgr2=0x7fbb500b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:33.387 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.374+0000 7fbb567ad700 1 -- 172.21.15.162:0/2193287578 shutdown_connections 2020-11-28T21:56:33.387 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.374+0000 7fbb567ad700 1 -- 172.21.15.162:0/2193287578 wait complete. 2020-11-28T21:56:33.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.378+0000 7fbb567ad700 1 Processor -- start 2020-11-28T21:56:33.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.378+0000 7fbb567ad700 1 -- start start 2020-11-28T21:56:33.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.378+0000 7fbb567ad700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb50060a70 0x7fbb50159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:33.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.378+0000 7fbb567ad700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbb500af6a0 0x7fbb50159780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:33.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.378+0000 7fbb567ad700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbb500b1eb0 0x7fbb50148bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:33.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.378+0000 7fbb567ad700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb50063550 con 0x7fbb500b1eb0 2020-11-28T21:56:33.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.378+0000 7fbb567ad700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb500632b0 con 0x7fbb50060a70 2020-11-28T21:56:33.392 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.378+0000 7fbb567ad700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb500abea0 con 0x7fbb500af6a0 2020-11-28T21:56:33.392 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.378+0000 7fbb557ab700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb50060a70 0x7fbb50159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:33.392 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.378+0000 7fbb557ab700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb50060a70 0x7fbb50159fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46518/0 (socket says 172.21.15.162:46518) 2020-11-28T21:56:33.393 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.378+0000 7fbb557ab700 1 -- 172.21.15.162:0/3140865694 learned_addr learned my addr 172.21.15.162:0/3140865694 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:33.393 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.378+0000 7fbb54faa700 1 --2- 172.21.15.162:0/3140865694 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbb500af6a0 0x7fbb50159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:33.393 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.378+0000 7fbb55fac700 1 --2- 172.21.15.162:0/3140865694 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbb500b1eb0 0x7fbb50148bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:33.393 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.378+0000 7fbb557ab700 1 -- 172.21.15.162:0/3140865694 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbb500af6a0 msgr2=0x7fbb50159780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:33.394 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.378+0000 7fbb557ab700 1 --2- 172.21.15.162:0/3140865694 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbb500af6a0 0x7fbb50159780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:33.394 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.378+0000 7fbb557ab700 1 -- 172.21.15.162:0/3140865694 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbb500b1eb0 msgr2=0x7fbb50148bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:33.394 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.378+0000 7fbb557ab700 1 --2- 172.21.15.162:0/3140865694 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbb500b1eb0 0x7fbb50148bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:33.394 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.378+0000 7fbb557ab700 1 -- 172.21.15.162:0/3140865694 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb500ac140 con 0x7fbb50060a70 2020-11-28T21:56:33.395 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.378+0000 7fbb54faa700 1 --2- 172.21.15.162:0/3140865694 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbb500af6a0 0x7fbb50159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:56:33.395 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.378+0000 7fbb557ab700 1 --2- 172.21.15.162:0/3140865694 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb50060a70 0x7fbb50159fa0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 rx=0x7fbb4c000e90 tx=0x7fbb4c007cb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:33.395 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.378+0000 7fbb467fc700 1 -- 172.21.15.162:0/3140865694 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb4c0072e0 con 0x7fbb50060a70 2020-11-28T21:56:33.395 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.378+0000 7fbb467fc700 1 -- 172.21.15.162:0/3140865694 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb4c007440 con 0x7fbb50060a70 2020-11-28T21:56:33.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.378+0000 7fbb567ad700 1 -- 172.21.15.162:0/3140865694 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb5015a490 con 0x7fbb50060a70 2020-11-28T21:56:33.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.378+0000 7fbb467fc700 1 -- 172.21.15.162:0/3140865694 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb4c0057d0 con 0x7fbb50060a70 2020-11-28T21:56:33.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.378+0000 7fbb567ad700 1 -- 172.21.15.162:0/3140865694 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb501490a0 con 0x7fbb50060a70 2020-11-28T21:56:33.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.378+0000 7fbb567ad700 1 -- 172.21.15.162:0/3140865694 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb500ac2d0 con 0x7fbb50060a70 2020-11-28T21:56:33.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.382+0000 7fbb467fc700 1 -- 172.21.15.162:0/3140865694 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fbb4c010070 con 0x7fbb50060a70 2020-11-28T21:56:33.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.382+0000 7fbb467fc700 1 --2- 172.21.15.162:0/3140865694 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbb380553b0 0x7fbb38057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:33.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.382+0000 7fbb467fc700 1 -- 172.21.15.162:0/3140865694 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbb4c06d6f0 con 0x7fbb50060a70 2020-11-28T21:56:33.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.382+0000 7fbb54faa700 1 --2- 172.21.15.162:0/3140865694 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbb380553b0 0x7fbb38057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:33.398 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.382+0000 7fbb54faa700 1 --2- 172.21.15.162:0/3140865694 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbb380553b0 0x7fbb38057870 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7fbb4000c000 tx=0x7fbb40008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:33.398 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.386+0000 7fbb467fc700 1 -- 172.21.15.162:0/3140865694 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fbb4c043800 con 0x7fbb50060a70 2020-11-28T21:56:33.536 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.530+0000 7fbb567ad700 1 -- 172.21.15.162:0/3140865694 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fbb50057050 con 0x7fbb50060a70 2020-11-28T21:56:33.537 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.530+0000 7fbb467fc700 1 -- 172.21.15.162:0/3140865694 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v160) v1 ==== 110+0+10 (secure 0 0 0) 0x7fbb4c018020 con 0x7fbb50060a70 2020-11-28T21:56:33.537 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 1 2020-11-28T21:56:33.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.534+0000 7fbb567ad700 1 -- 172.21.15.162:0/3140865694 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbb380553b0 msgr2=0x7fbb38057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:33.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.534+0000 7fbb567ad700 1 --2- 172.21.15.162:0/3140865694 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbb380553b0 0x7fbb38057870 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7fbb4000c000 tx=0x7fbb40008040).stop 2020-11-28T21:56:33.539 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.534+0000 7fbb567ad700 1 -- 172.21.15.162:0/3140865694 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb50060a70 msgr2=0x7fbb50159fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:33.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.534+0000 7fbb567ad700 1 --2- 172.21.15.162:0/3140865694 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb50060a70 0x7fbb50159fa0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 rx=0x7fbb4c000e90 tx=0x7fbb4c007cb0).stop 2020-11-28T21:56:33.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.534+0000 7fbb567ad700 1 -- 172.21.15.162:0/3140865694 shutdown_connections 2020-11-28T21:56:33.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.534+0000 7fbb567ad700 1 --2- 172.21.15.162:0/3140865694 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fbb380553b0 0x7fbb38057870 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:33.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.534+0000 7fbb567ad700 1 --2- 172.21.15.162:0/3140865694 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbb500b1eb0 0x7fbb50148bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:33.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.534+0000 7fbb567ad700 1 --2- 172.21.15.162:0/3140865694 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb50060a70 0x7fbb50159fa0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:33.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.534+0000 7fbb567ad700 1 --2- 172.21.15.162:0/3140865694 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbb500af6a0 0x7fbb50159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:33.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.534+0000 7fbb567ad700 1 -- 172.21.15.162:0/3140865694 >> 172.21.15.162:0/3140865694 conn(0x7fbb500b57d0 msgr2=0x7fbb500b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:33.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.534+0000 7fbb567ad700 1 -- 172.21.15.162:0/3140865694 shutdown_connections 2020-11-28T21:56:33.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.534+0000 7fbb567ad700 1 -- 172.21.15.162:0/3140865694 wait complete. 2020-11-28T21:56:33.551 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-28T21:56:33.665 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.658+0000 7fe9a703e700 1 Processor -- start 2020-11-28T21:56:33.665 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.658+0000 7fe9a703e700 1 -- start start 2020-11-28T21:56:33.666 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.662+0000 7fe9a703e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe9a01433c0 0x7fe9a00605c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:33.667 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.662+0000 7fe9a703e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9a0143950 con 0x7fe9a0067610 2020-11-28T21:56:33.667 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.662+0000 7fe9a703e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9a00ba0c0 con 0x7fe9a00b9ce0 2020-11-28T21:56:33.667 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.662+0000 7fe9a703e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9a00b4340 con 0x7fe9a01433c0 2020-11-28T21:56:33.667 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.662+0000 7fe99ffff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe9a01433c0 0x7fe9a00605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:33.667 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.662+0000 7fe9a4dda700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fe9a00b9ce0 0x7fe9a00b3c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:53368/0 (socket says 172.21.15.162:53368) 2020-11-28T21:56:33.668 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.662+0000 7fe9a4dda700 1 -- 172.21.15.162:0/322201354 learned_addr learned my addr 172.21.15.162:0/322201354 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:56:33.669 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.662+0000 7fe99ffff700 1 -- 172.21.15.162:0/322201354 >> v1:172.21.15.162:6789/0 conn(0x7fe9a00b9ce0 legacy=0x7fe9a00b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:33.669 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.662+0000 7fe99ffff700 1 -- 172.21.15.162:0/322201354 >> v1:172.21.15.118:6789/0 conn(0x7fe9a0067610 legacy=0x7fe9a013fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:33.669 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.662+0000 7fe99f7fe700 1 -- 172.21.15.162:0/322201354 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1973393346 0 0) 0x7fe9a00ba0c0 con 0x7fe9a00b9ce0 2020-11-28T21:56:33.670 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.662+0000 7fe99ffff700 1 -- 172.21.15.162:0/322201354 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a0066c30 con 0x7fe9a01433c0 2020-11-28T21:56:33.670 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.662+0000 7fe99ffff700 1 --2- 172.21.15.162:0/322201354 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9a01433c0 0x7fe9a00605c0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 rx=0x7fe990002f30 tx=0x7fe990006dc0).ready entity=mon.1 client_cookie=1611dd6e8579e5bb server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:33.670 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.662+0000 7fe99f7fe700 1 -- 172.21.15.162:0/322201354 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe9900097f0 con 0x7fe9a01433c0 2020-11-28T21:56:33.670 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.662+0000 7fe99f7fe700 1 -- 172.21.15.162:0/322201354 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe990009950 con 0x7fe9a01433c0 2020-11-28T21:56:33.671 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.662+0000 7fe99f7fe700 1 -- 172.21.15.162:0/322201354 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe990009cb0 con 0x7fe9a01433c0 2020-11-28T21:56:33.671 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.662+0000 7fe9a703e700 1 -- 172.21.15.162:0/322201354 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9a01433c0 msgr2=0x7fe9a00605c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:33.671 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.662+0000 7fe9a703e700 1 --2- 172.21.15.162:0/322201354 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9a01433c0 0x7fe9a00605c0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 rx=0x7fe990002f30 tx=0x7fe990006dc0).stop 2020-11-28T21:56:33.671 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.662+0000 7fe9a703e700 1 -- 172.21.15.162:0/322201354 shutdown_connections 2020-11-28T21:56:33.672 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.662+0000 7fe9a703e700 1 --2- 172.21.15.162:0/322201354 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9a01433c0 0x7fe9a00605c0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:33.672 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.662+0000 7fe9a703e700 1 -- 172.21.15.162:0/322201354 >> 172.21.15.162:0/322201354 conn(0x7fe9a005c430 msgr2=0x7fe9a00b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:33.672 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.662+0000 7fe9a703e700 1 -- 172.21.15.162:0/322201354 shutdown_connections 2020-11-28T21:56:33.672 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.662+0000 7fe9a703e700 1 -- 172.21.15.162:0/322201354 wait complete. 2020-11-28T21:56:33.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.666+0000 7fe9a703e700 1 Processor -- start 2020-11-28T21:56:33.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.666+0000 7fe9a703e700 1 -- start start 2020-11-28T21:56:33.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.666+0000 7fe9a703e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe9a0067610 0x7fe9a013f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:33.675 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.666+0000 7fe9a703e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9a00b9ce0 0x7fe9a00abef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:33.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.666+0000 7fe9a703e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe9a01433c0 0x7fe9a00ac3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:33.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.666+0000 7fe9a703e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a00b1030 con 0x7fe9a0067610 2020-11-28T21:56:33.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.666+0000 7fe9a703e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a00b12d0 con 0x7fe9a00b9ce0 2020-11-28T21:56:33.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.666+0000 7fe9a703e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a00b1570 con 0x7fe9a01433c0 2020-11-28T21:56:33.676 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.666+0000 7fe99ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9a00b9ce0 0x7fe9a00abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:33.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.666+0000 7fe99ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9a00b9ce0 0x7fe9a00abef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46532/0 (socket says 172.21.15.162:46532) 2020-11-28T21:56:33.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.666+0000 7fe99ffff700 1 -- 172.21.15.162:0/2411236198 learned_addr learned my addr 172.21.15.162:0/2411236198 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:33.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.666+0000 7fe9a4dda700 1 --2- 172.21.15.162:0/2411236198 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe9a0067610 0x7fe9a013f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:33.677 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.666+0000 7fe9a55db700 1 --2- 172.21.15.162:0/2411236198 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe9a01433c0 0x7fe9a00ac3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:33.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.666+0000 7fe99ffff700 1 -- 172.21.15.162:0/2411236198 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe9a01433c0 msgr2=0x7fe9a00ac3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:33.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.666+0000 7fe99ffff700 1 --2- 172.21.15.162:0/2411236198 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe9a01433c0 0x7fe9a00ac3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:33.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.666+0000 7fe99ffff700 1 -- 172.21.15.162:0/2411236198 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe9a0067610 msgr2=0x7fe9a013f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:33.678 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.666+0000 7fe99ffff700 1 --2- 172.21.15.162:0/2411236198 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe9a0067610 0x7fe9a013f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:33.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.666+0000 7fe99ffff700 1 -- 172.21.15.162:0/2411236198 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a00b07c0 con 0x7fe9a00b9ce0 2020-11-28T21:56:33.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.666+0000 7fe9a4dda700 1 --2- 172.21.15.162:0/2411236198 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe9a0067610 0x7fe9a013f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:56:33.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.666+0000 7fe9a55db700 1 --2- 172.21.15.162:0/2411236198 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe9a01433c0 0x7fe9a00ac3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:56:33.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.666+0000 7fe99ffff700 1 --2- 172.21.15.162:0/2411236198 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9a00b9ce0 0x7fe9a00abef0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 rx=0x7fe990002f30 tx=0x7fe990015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:33.679 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.666+0000 7fe99dffb700 1 -- 172.21.15.162:0/2411236198 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe990021070 con 0x7fe9a00b9ce0 2020-11-28T21:56:33.680 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.666+0000 7fe99dffb700 1 -- 172.21.15.162:0/2411236198 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe990011cc0 con 0x7fe9a00b9ce0 2020-11-28T21:56:33.680 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.666+0000 7fe99dffb700 1 -- 172.21.15.162:0/2411236198 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe99001d640 con 0x7fe9a00b9ce0 2020-11-28T21:56:33.680 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.666+0000 7fe9a703e700 1 -- 172.21.15.162:0/2411236198 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9a00b0a60 con 0x7fe9a00b9ce0 2020-11-28T21:56:33.680 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.666+0000 7fe9a703e700 1 -- 172.21.15.162:0/2411236198 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9a014f370 con 0x7fe9a00b9ce0 2020-11-28T21:56:33.680 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.670+0000 7fe9a703e700 1 -- 172.21.15.162:0/2411236198 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9a00b1b70 con 0x7fe9a00b9ce0 2020-11-28T21:56:33.681 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.670+0000 7fe99dffb700 1 -- 172.21.15.162:0/2411236198 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe99000f070 con 0x7fe9a00b9ce0 2020-11-28T21:56:33.681 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.670+0000 7fe99dffb700 1 --2- 172.21.15.162:0/2411236198 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe9840554e0 0x7fe9840579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:33.682 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.670+0000 7fe99dffb700 1 -- 172.21.15.162:0/2411236198 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe99006f610 con 0x7fe9a00b9ce0 2020-11-28T21:56:33.682 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.670+0000 7fe9a4dda700 1 --2- 172.21.15.162:0/2411236198 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe9840554e0 0x7fe9840579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:33.682 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.670+0000 7fe9a4dda700 1 --2- 172.21.15.162:0/2411236198 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe9840554e0 0x7fe9840579a0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7fe98800b000 tx=0x7fe988009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:33.684 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.678+0000 7fe99dffb700 1 -- 172.21.15.162:0/2411236198 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe9a00b1b70 con 0x7fe9a00b9ce0 2020-11-28T21:56:33.819 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.814+0000 7fe9a703e700 1 -- 172.21.15.162:0/2411236198 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe9a00b1b70 con 0x7fe9a00b9ce0 2020-11-28T21:56:33.820 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.814+0000 7fe99dffb700 1 -- 172.21.15.162:0/2411236198 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v160) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe9a00b1b70 con 0x7fe9a00b9ce0 2020-11-28T21:56:33.820 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:56:33.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.818+0000 7fe9a703e700 1 -- 172.21.15.162:0/2411236198 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe9840554e0 msgr2=0x7fe9840579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:33.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.818+0000 7fe9a703e700 1 --2- 172.21.15.162:0/2411236198 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe9840554e0 0x7fe9840579a0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7fe98800b000 tx=0x7fe988009040).stop 2020-11-28T21:56:33.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.818+0000 7fe9a703e700 1 -- 172.21.15.162:0/2411236198 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9a00b9ce0 msgr2=0x7fe9a00abef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:33.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.818+0000 7fe9a703e700 1 --2- 172.21.15.162:0/2411236198 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9a00b9ce0 0x7fe9a00abef0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 rx=0x7fe990002f30 tx=0x7fe990015000).stop 2020-11-28T21:56:33.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.818+0000 7fe9a703e700 1 -- 172.21.15.162:0/2411236198 shutdown_connections 2020-11-28T21:56:33.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.818+0000 7fe9a703e700 1 --2- 172.21.15.162:0/2411236198 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe9840554e0 0x7fe9840579a0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:33.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.818+0000 7fe9a703e700 1 --2- 172.21.15.162:0/2411236198 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe9a01433c0 0x7fe9a00ac3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:33.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.818+0000 7fe9a703e700 1 --2- 172.21.15.162:0/2411236198 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe9a0067610 0x7fe9a013f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:33.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.818+0000 7fe9a703e700 1 --2- 172.21.15.162:0/2411236198 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9a00b9ce0 0x7fe9a00abef0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:33.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.818+0000 7fe9a703e700 1 -- 172.21.15.162:0/2411236198 >> 172.21.15.162:0/2411236198 conn(0x7fe9a005c430 msgr2=0x7fe9a009d720 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:33.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.818+0000 7fe9a703e700 1 -- 172.21.15.162:0/2411236198 shutdown_connections 2020-11-28T21:56:33.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.818+0000 7fe9a703e700 1 -- 172.21.15.162:0/2411236198 wait complete. 2020-11-28T21:56:33.834 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-28T21:56:33.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.950+0000 7fe824a5e700 1 Processor -- start 2020-11-28T21:56:33.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.950+0000 7fe824a5e700 1 -- start start 2020-11-28T21:56:33.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.950+0000 7fe824a5e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe820066a50 0x7fe8201409d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:33.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.950+0000 7fe824a5e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe820066e30 0x7fe820144620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:33.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.950+0000 7fe824a5e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8200b1dd0 con 0x7fe820144b10 2020-11-28T21:56:33.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.950+0000 7fe824a5e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe820067290 con 0x7fe820066e30 2020-11-28T21:56:33.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.950+0000 7fe824a5e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe820143f20 con 0x7fe820066a50 2020-11-28T21:56:33.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.950+0000 7fe81ed9d700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fe820144b10 0x7fe8200b37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:53386/0 (socket says 172.21.15.162:53386) 2020-11-28T21:56:33.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.950+0000 7fe81ed9d700 1 -- 172.21.15.162:0/2248431929 learned_addr learned my addr 172.21.15.162:0/2248431929 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:56:33.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.950+0000 7fe81dd9b700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe820066e30 0x7fe820144620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:33.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.950+0000 7fe81e59c700 1 --2- 172.21.15.162:0/2248431929 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe820066a50 0x7fe8201409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:33.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.950+0000 7fe81d59a700 1 -- 172.21.15.162:0/2248431929 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 36574521 0 0) 0x7fe8200b1dd0 con 0x7fe820144b10 2020-11-28T21:56:33.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.950+0000 7fe81d59a700 1 -- 172.21.15.162:0/2248431929 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe804003230 con 0x7fe820144b10 2020-11-28T21:56:33.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.950+0000 7fe81dd9b700 1 -- 172.21.15.162:0/2248431929 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe820066a50 msgr2=0x7fe8201409d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:33.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.950+0000 7fe81dd9b700 1 --2- 172.21.15.162:0/2248431929 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe820066a50 0x7fe8201409d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:33.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.950+0000 7fe81dd9b700 1 -- 172.21.15.162:0/2248431929 >> v1:172.21.15.162:6789/0 conn(0x7fe820144b10 legacy=0x7fe8200b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:33.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.950+0000 7fe81dd9b700 1 -- 172.21.15.162:0/2248431929 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8200b2000 con 0x7fe820066e30 2020-11-28T21:56:33.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.950+0000 7fe81dd9b700 1 --2- 172.21.15.162:0/2248431929 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe820066e30 0x7fe820144620 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 rx=0x7fe814002f30 tx=0x7fe814006dc0).ready entity=mon.1 client_cookie=a57cc2dfadcb6a0b server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:33.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.950+0000 7fe81d59a700 1 -- 172.21.15.162:0/2248431929 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe8140096a0 con 0x7fe820066e30 2020-11-28T21:56:33.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.950+0000 7fe81d59a700 1 -- 172.21.15.162:0/2248431929 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe814009800 con 0x7fe820066e30 2020-11-28T21:56:33.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.950+0000 7fe81d59a700 1 -- 172.21.15.162:0/2248431929 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe814009b60 con 0x7fe820066e30 2020-11-28T21:56:33.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.950+0000 7fe824a5e700 1 -- 172.21.15.162:0/2248431929 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe820066e30 msgr2=0x7fe820144620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:33.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.950+0000 7fe824a5e700 1 --2- 172.21.15.162:0/2248431929 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe820066e30 0x7fe820144620 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 rx=0x7fe814002f30 tx=0x7fe814006dc0).stop 2020-11-28T21:56:33.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.954+0000 7fe824a5e700 1 -- 172.21.15.162:0/2248431929 shutdown_connections 2020-11-28T21:56:33.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.954+0000 7fe824a5e700 1 --2- 172.21.15.162:0/2248431929 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe820066a50 0x7fe8201409d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:33.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.954+0000 7fe824a5e700 1 --2- 172.21.15.162:0/2248431929 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe820066e30 0x7fe820144620 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:33.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.954+0000 7fe824a5e700 1 -- 172.21.15.162:0/2248431929 >> 172.21.15.162:0/2248431929 conn(0x7fe82005b970 msgr2=0x7fe82005bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:33.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.954+0000 7fe824a5e700 1 -- 172.21.15.162:0/2248431929 shutdown_connections 2020-11-28T21:56:33.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.954+0000 7fe824a5e700 1 -- 172.21.15.162:0/2248431929 wait complete. 2020-11-28T21:56:33.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.954+0000 7fe824a5e700 1 Processor -- start 2020-11-28T21:56:33.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.954+0000 7fe824a5e700 1 -- start start 2020-11-28T21:56:33.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.954+0000 7fe824a5e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe820066a50 0x7fe8200b3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:33.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.954+0000 7fe824a5e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe820066e30 0x7fe820153800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:33.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.954+0000 7fe824a5e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe820144b10 0x7fe82009c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:33.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.954+0000 7fe824a5e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe820153100 con 0x7fe820066a50 2020-11-28T21:56:33.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.954+0000 7fe824a5e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8200b0e20 con 0x7fe820144b10 2020-11-28T21:56:33.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.954+0000 7fe824a5e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8200b10c0 con 0x7fe820066e30 2020-11-28T21:56:33.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.954+0000 7fe81ed9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe820144b10 0x7fe82009c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:33.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.954+0000 7fe81ed9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe820144b10 0x7fe82009c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46548/0 (socket says 172.21.15.162:46548) 2020-11-28T21:56:33.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.954+0000 7fe81ed9d700 1 -- 172.21.15.162:0/3413622517 learned_addr learned my addr 172.21.15.162:0/3413622517 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:33.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.958+0000 7fe81dd9b700 1 --2- 172.21.15.162:0/3413622517 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe820066e30 0x7fe820153800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:33.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.958+0000 7fe81e59c700 1 --2- 172.21.15.162:0/3413622517 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe820066a50 0x7fe8200b3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:33.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.958+0000 7fe81ed9d700 1 -- 172.21.15.162:0/3413622517 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe820066e30 msgr2=0x7fe820153800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:33.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.958+0000 7fe81ed9d700 1 --2- 172.21.15.162:0/3413622517 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe820066e30 0x7fe820153800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:33.968 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.958+0000 7fe81ed9d700 1 -- 172.21.15.162:0/3413622517 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe820066a50 msgr2=0x7fe8200b3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:33.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.958+0000 7fe81ed9d700 1 --2- 172.21.15.162:0/3413622517 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe820066a50 0x7fe8200b3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:33.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.958+0000 7fe81ed9d700 1 -- 172.21.15.162:0/3413622517 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe820153360 con 0x7fe820144b10 2020-11-28T21:56:33.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.958+0000 7fe81ed9d700 1 --2- 172.21.15.162:0/3413622517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe820144b10 0x7fe82009c6a0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 rx=0x7fe818001450 tx=0x7fe818007d90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:33.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.958+0000 7fe80f7fe700 1 -- 172.21.15.162:0/3413622517 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe818005b70 con 0x7fe820144b10 2020-11-28T21:56:33.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.958+0000 7fe80f7fe700 1 -- 172.21.15.162:0/3413622517 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe818005cd0 con 0x7fe820144b10 2020-11-28T21:56:33.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.958+0000 7fe80f7fe700 1 -- 172.21.15.162:0/3413622517 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe818007300 con 0x7fe820144b10 2020-11-28T21:56:33.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.958+0000 7fe824a5e700 1 -- 172.21.15.162:0/3413622517 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8201433b0 con 0x7fe820144b10 2020-11-28T21:56:33.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.958+0000 7fe824a5e700 1 -- 172.21.15.162:0/3413622517 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe82009a460 con 0x7fe820144b10 2020-11-28T21:56:33.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.958+0000 7fe824a5e700 1 -- 172.21.15.162:0/3413622517 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe820154030 con 0x7fe820144b10 2020-11-28T21:56:33.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.958+0000 7fe80f7fe700 1 -- 172.21.15.162:0/3413622517 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe818010030 con 0x7fe820144b10 2020-11-28T21:56:33.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.958+0000 7fe80f7fe700 1 --2- 172.21.15.162:0/3413622517 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe7fc0554e0 0x7fe7fc0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:33.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.958+0000 7fe80f7fe700 1 -- 172.21.15.162:0/3413622517 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe81806de80 con 0x7fe820144b10 2020-11-28T21:56:33.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.958+0000 7fe81e59c700 1 --2- 172.21.15.162:0/3413622517 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe7fc0554e0 0x7fe7fc0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:33.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.962+0000 7fe81e59c700 1 --2- 172.21.15.162:0/3413622517 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe7fc0554e0 0x7fe7fc0579a0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7fe81000b000 tx=0x7fe810008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:33.972 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:33.962+0000 7fe80f7fe700 1 -- 172.21.15.162:0/3413622517 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe8180440c0 con 0x7fe820144b10 2020-11-28T21:56:34.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.106+0000 7fe824a5e700 1 -- 172.21.15.162:0/3413622517 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe82009a460 con 0x7fe820144b10 2020-11-28T21:56:34.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.106+0000 7fe80f7fe700 1 -- 172.21.15.162:0/3413622517 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v160) v1 ==== 104+0+10 (secure 0 0 0) 0x7fe82009a460 con 0x7fe820144b10 2020-11-28T21:56:34.111 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:56:34.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.106+0000 7fe824a5e700 1 -- 172.21.15.162:0/3413622517 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe7fc0554e0 msgr2=0x7fe7fc0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:34.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.106+0000 7fe824a5e700 1 --2- 172.21.15.162:0/3413622517 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe7fc0554e0 0x7fe7fc0579a0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7fe81000b000 tx=0x7fe810008040).stop 2020-11-28T21:56:34.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.106+0000 7fe824a5e700 1 -- 172.21.15.162:0/3413622517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe820144b10 msgr2=0x7fe82009c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:34.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.106+0000 7fe824a5e700 1 --2- 172.21.15.162:0/3413622517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe820144b10 0x7fe82009c6a0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 rx=0x7fe818001450 tx=0x7fe818007d90).stop 2020-11-28T21:56:34.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.106+0000 7fe824a5e700 1 -- 172.21.15.162:0/3413622517 shutdown_connections 2020-11-28T21:56:34.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.106+0000 7fe824a5e700 1 --2- 172.21.15.162:0/3413622517 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe7fc0554e0 0x7fe7fc0579a0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:34.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.106+0000 7fe824a5e700 1 --2- 172.21.15.162:0/3413622517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe820144b10 0x7fe82009c6a0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:34.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.106+0000 7fe824a5e700 1 --2- 172.21.15.162:0/3413622517 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe820066a50 0x7fe8200b3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:34.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.106+0000 7fe824a5e700 1 --2- 172.21.15.162:0/3413622517 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe820066e30 0x7fe820153800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:34.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.106+0000 7fe824a5e700 1 -- 172.21.15.162:0/3413622517 >> 172.21.15.162:0/3413622517 conn(0x7fe82005b970 msgr2=0x7fe8201436e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:34.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.106+0000 7fe824a5e700 1 -- 172.21.15.162:0/3413622517 shutdown_connections 2020-11-28T21:56:34.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.106+0000 7fe824a5e700 1 -- 172.21.15.162:0/3413622517 wait complete. 2020-11-28T21:56:34.127 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-28T21:56:34.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.250+0000 7f35f6ceb700 1 Processor -- start 2020-11-28T21:56:34.254 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.250+0000 7f35f6ceb700 1 -- start start 2020-11-28T21:56:34.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.250+0000 7f35f6ceb700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f35f00b2d00 0x7f35f005c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:34.255 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.250+0000 7f35f6ceb700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35f005baf0 con 0x7f35f00b30e0 2020-11-28T21:56:34.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.250+0000 7f35f6ceb700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35f005bd90 con 0x7f35f00b3ab0 2020-11-28T21:56:34.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.250+0000 7f35f6ceb700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35f00b0e20 con 0x7f35f00b2d00 2020-11-28T21:56:34.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.250+0000 7f35f5ce9700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f35f00b2d00 0x7f35f005c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:34.256 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.250+0000 7f35f5ce9700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f35f00b2d00 0x7f35f005c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46552/0 (socket says 172.21.15.162:46552) 2020-11-28T21:56:34.257 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.250+0000 7f35f5ce9700 1 -- 172.21.15.162:0/1185753609 learned_addr learned my addr 172.21.15.162:0/1185753609 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:34.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.250+0000 7f35f5ce9700 1 -- 172.21.15.162:0/1185753609 >> v1:172.21.15.162:6789/0 conn(0x7f35f00b3ab0 legacy=0x7f35f0142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:34.258 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.250+0000 7f35f5ce9700 1 -- 172.21.15.162:0/1185753609 >> v1:172.21.15.118:6789/0 conn(0x7f35f00b30e0 legacy=0x7f35f013fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:34.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.250+0000 7f35f5ce9700 1 -- 172.21.15.162:0/1185753609 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f005c030 con 0x7f35f00b2d00 2020-11-28T21:56:34.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.250+0000 7f35f5ce9700 1 --2- 172.21.15.162:0/1185753609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35f00b2d00 0x7f35f005c4d0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 rx=0x7f35f00acde0 tx=0x7f35e0009270).ready entity=mon.1 client_cookie=7215fe9bbba72a5d server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:34.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.250+0000 7f35f4ce7700 1 -- 172.21.15.162:0/1185753609 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35e000bbc0 con 0x7f35f00b2d00 2020-11-28T21:56:34.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.250+0000 7f35f4ce7700 1 -- 172.21.15.162:0/1185753609 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35e000bd20 con 0x7f35f00b2d00 2020-11-28T21:56:34.259 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.250+0000 7f35f4ce7700 1 -- 172.21.15.162:0/1185753609 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35e000be90 con 0x7f35f00b2d00 2020-11-28T21:56:34.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.250+0000 7f35f6ceb700 1 -- 172.21.15.162:0/1185753609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35f00b2d00 msgr2=0x7f35f005c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:34.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.250+0000 7f35f6ceb700 1 --2- 172.21.15.162:0/1185753609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35f00b2d00 0x7f35f005c4d0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 rx=0x7f35f00acde0 tx=0x7f35e0009270).stop 2020-11-28T21:56:34.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.254+0000 7f35f6ceb700 1 -- 172.21.15.162:0/1185753609 shutdown_connections 2020-11-28T21:56:34.260 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.254+0000 7f35f6ceb700 1 --2- 172.21.15.162:0/1185753609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35f00b2d00 0x7f35f005c4d0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:34.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.254+0000 7f35f6ceb700 1 -- 172.21.15.162:0/1185753609 >> 172.21.15.162:0/1185753609 conn(0x7f35f00604c0 msgr2=0x7f35f00608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:34.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.254+0000 7f35f6ceb700 1 -- 172.21.15.162:0/1185753609 shutdown_connections 2020-11-28T21:56:34.261 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.254+0000 7f35f6ceb700 1 -- 172.21.15.162:0/1185753609 wait complete. 2020-11-28T21:56:34.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.254+0000 7f35f6ceb700 1 Processor -- start 2020-11-28T21:56:34.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.254+0000 7f35f6ceb700 1 -- start start 2020-11-28T21:56:34.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.254+0000 7f35f6ceb700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f35f00b2d00 0x7f35f00b70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:34.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.254+0000 7f35f6ceb700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35f00b30e0 0x7f35f0151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:34.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.254+0000 7f35f6ceb700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f35f00b3ab0 0x7f35f0152040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:34.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.254+0000 7f35f6ceb700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f0141150 con 0x7f35f00b3ab0 2020-11-28T21:56:34.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.254+0000 7f35f6ceb700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f0152530 con 0x7f35f00b30e0 2020-11-28T21:56:34.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.254+0000 7f35f6ceb700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35f0152750 con 0x7f35f00b2d00 2020-11-28T21:56:34.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.254+0000 7f35f54e8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35f00b30e0 0x7f35f0151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:34.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.258+0000 7f35f54e8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35f00b30e0 0x7f35f0151b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46560/0 (socket says 172.21.15.162:46560) 2020-11-28T21:56:34.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.258+0000 7f35f5ce9700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f35f00b2d00 0x7f35f00b70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:34.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.258+0000 7f35f54e8700 1 -- 172.21.15.162:0/4234667149 learned_addr learned my addr 172.21.15.162:0/4234667149 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:34.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.258+0000 7f35f64ea700 1 --2- 172.21.15.162:0/4234667149 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f35f00b3ab0 0x7f35f0152040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:34.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.258+0000 7f35f54e8700 1 -- 172.21.15.162:0/4234667149 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f35f00b2d00 msgr2=0x7f35f00b70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:34.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.258+0000 7f35f54e8700 1 --2- 172.21.15.162:0/4234667149 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f35f00b2d00 0x7f35f00b70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:34.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.258+0000 7f35f54e8700 1 -- 172.21.15.162:0/4234667149 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f35f00b3ab0 msgr2=0x7f35f0152040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:34.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.258+0000 7f35f54e8700 1 --2- 172.21.15.162:0/4234667149 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f35f00b3ab0 0x7f35f0152040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:34.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.258+0000 7f35f54e8700 1 -- 172.21.15.162:0/4234667149 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f01529f0 con 0x7f35f00b30e0 2020-11-28T21:56:34.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.258+0000 7f35f54e8700 1 --2- 172.21.15.162:0/4234667149 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35f00b30e0 0x7f35f0151b50 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 rx=0x7f35f00b9b00 tx=0x7f35e4007ba0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:34.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.258+0000 7f35eeffd700 1 -- 172.21.15.162:0/4234667149 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35e4007240 con 0x7f35f00b30e0 2020-11-28T21:56:34.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.258+0000 7f35eeffd700 1 -- 172.21.15.162:0/4234667149 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35e40073a0 con 0x7f35f00b30e0 2020-11-28T21:56:34.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.258+0000 7f35f6ceb700 1 -- 172.21.15.162:0/4234667149 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35f0140a10 con 0x7f35f00b30e0 2020-11-28T21:56:34.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.258+0000 7f35eeffd700 1 -- 172.21.15.162:0/4234667149 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35e40066d0 con 0x7f35f00b30e0 2020-11-28T21:56:34.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.258+0000 7f35f6ceb700 1 -- 172.21.15.162:0/4234667149 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35f00b75d0 con 0x7f35f00b30e0 2020-11-28T21:56:34.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.258+0000 7f35f6ceb700 1 -- 172.21.15.162:0/4234667149 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35f01454c0 con 0x7f35f00b30e0 2020-11-28T21:56:34.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.262+0000 7f35eeffd700 1 -- 172.21.15.162:0/4234667149 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f35e401a060 con 0x7f35f00b30e0 2020-11-28T21:56:34.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.262+0000 7f35eeffd700 1 --2- 172.21.15.162:0/4234667149 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f35d0055440 0x7f35d0057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:34.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.262+0000 7f35eeffd700 1 -- 172.21.15.162:0/4234667149 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f35e406d340 con 0x7f35f00b30e0 2020-11-28T21:56:34.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.262+0000 7f35f5ce9700 1 --2- 172.21.15.162:0/4234667149 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f35d0055440 0x7f35d0057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:34.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.262+0000 7f35f5ce9700 1 --2- 172.21.15.162:0/4234667149 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f35d0055440 0x7f35d0057900 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7f35e001a000 tx=0x7f35e0016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:34.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.266+0000 7f35eeffd700 1 -- 172.21.15.162:0/4234667149 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f35e40435e0 con 0x7f35f00b30e0 2020-11-28T21:56:34.414 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.406+0000 7f35f6ceb700 1 -- 172.21.15.162:0/4234667149 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f35f0057050 con 0x7f35f00b30e0 2020-11-28T21:56:34.415 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.410+0000 7f35eeffd700 1 -- 172.21.15.162:0/4234667149 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v160) v1 ==== 100+0+10 (secure 0 0 0) 0x7f35e4018020 con 0x7f35f00b30e0 2020-11-28T21:56:34.415 INFO:teuthology.orchestra.run.smithi162.stdout:pg_num: 8 2020-11-28T21:56:34.418 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.410+0000 7f35f6ceb700 1 -- 172.21.15.162:0/4234667149 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f35d0055440 msgr2=0x7f35d0057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:34.419 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.410+0000 7f35f6ceb700 1 --2- 172.21.15.162:0/4234667149 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f35d0055440 0x7f35d0057900 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7f35e001a000 tx=0x7f35e0016000).stop 2020-11-28T21:56:34.419 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.410+0000 7f35f6ceb700 1 -- 172.21.15.162:0/4234667149 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35f00b30e0 msgr2=0x7f35f0151b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:34.421 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.410+0000 7f35f6ceb700 1 --2- 172.21.15.162:0/4234667149 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35f00b30e0 0x7f35f0151b50 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 rx=0x7f35f00b9b00 tx=0x7f35e4007ba0).stop 2020-11-28T21:56:34.421 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.410+0000 7f35f6ceb700 1 -- 172.21.15.162:0/4234667149 shutdown_connections 2020-11-28T21:56:34.422 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.410+0000 7f35f6ceb700 1 --2- 172.21.15.162:0/4234667149 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f35d0055440 0x7f35d0057900 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:34.422 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.410+0000 7f35f6ceb700 1 --2- 172.21.15.162:0/4234667149 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f35f00b2d00 0x7f35f00b70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:34.423 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.410+0000 7f35f6ceb700 1 --2- 172.21.15.162:0/4234667149 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f35f00b3ab0 0x7f35f0152040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:34.424 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.410+0000 7f35f6ceb700 1 --2- 172.21.15.162:0/4234667149 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35f00b30e0 0x7f35f0151b50 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:34.424 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.410+0000 7f35f6ceb700 1 -- 172.21.15.162:0/4234667149 >> 172.21.15.162:0/4234667149 conn(0x7f35f00604c0 msgr2=0x7f35f00608a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:34.425 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.410+0000 7f35f6ceb700 1 -- 172.21.15.162:0/4234667149 shutdown_connections 2020-11-28T21:56:34.426 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.410+0000 7f35f6ceb700 1 -- 172.21.15.162:0/4234667149 wait complete. 2020-11-28T21:56:34.428 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-28T21:56:34.430 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:56:34.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.534+0000 7f685359e700 1 Processor -- start 2020-11-28T21:56:34.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.534+0000 7f685359e700 1 -- start start 2020-11-28T21:56:34.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.534+0000 7f685359e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f6854060cf0 0x7f68540aa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:34.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.534+0000 7f685359e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68540b0230 con 0x7f68540af640 2020-11-28T21:56:34.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.534+0000 7f685359e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68540ab0f0 con 0x7f685405cbd0 2020-11-28T21:56:34.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.534+0000 7f685359e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68540b12a0 con 0x7f6854060cf0 2020-11-28T21:56:34.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.538+0000 7f6851d9b700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f685405cbd0 0x7f68540aa9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:53410/0 (socket says 172.21.15.162:53410) 2020-11-28T21:56:34.543 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.538+0000 7f6851d9b700 1 -- 172.21.15.162:0/3515886164 learned_addr learned my addr 172.21.15.162:0/3515886164 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:56:34.543 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.538+0000 7f685259c700 1 --2- 172.21.15.162:0/3515886164 >> v2:172.21.15.118:3300/0 conn(0x7f6854060cf0 0x7f68540aa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:34.543 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.538+0000 7f685159a700 1 -- 172.21.15.162:0/3515886164 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4160855549 0 0) 0x7f68540ab0f0 con 0x7f685405cbd0 2020-11-28T21:56:34.543 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.538+0000 7f685159a700 1 -- 172.21.15.162:0/3515886164 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6838003230 con 0x7f685405cbd0 2020-11-28T21:56:34.546 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.538+0000 7f685159a700 1 -- 172.21.15.162:0/3515886164 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3116404181 0 0) 0x7f68540b0230 con 0x7f68540af640 2020-11-28T21:56:34.547 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.538+0000 7f685159a700 1 -- 172.21.15.162:0/3515886164 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68540ab0f0 con 0x7f68540af640 2020-11-28T21:56:34.550 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.538+0000 7f685159a700 1 -- 172.21.15.162:0/3515886164 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3252641565 0 0) 0x7f6838003230 con 0x7f685405cbd0 2020-11-28T21:56:34.550 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.538+0000 7f685159a700 1 -- 172.21.15.162:0/3515886164 >> v2:172.21.15.118:3300/0 conn(0x7f6854060cf0 msgr2=0x7f68540aa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:34.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.538+0000 7f685159a700 1 --2- 172.21.15.162:0/3515886164 >> v2:172.21.15.118:3300/0 conn(0x7f6854060cf0 0x7f68540aa4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:34.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.538+0000 7f685159a700 1 -- 172.21.15.162:0/3515886164 >> v1:172.21.15.118:6789/0 conn(0x7f68540af640 legacy=0x7f68540ae1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:34.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.538+0000 7f685159a700 1 -- 172.21.15.162:0/3515886164 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68540a8c00 con 0x7f685405cbd0 2020-11-28T21:56:34.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.538+0000 7f685159a700 1 -- 172.21.15.162:0/3515886164 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f68480016d0 con 0x7f685405cbd0 2020-11-28T21:56:34.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.538+0000 7f685259c700 1 --2- 172.21.15.162:0/3515886164 >> v2:172.21.15.118:3300/0 conn(0x7f6854060cf0 0x7f68540aa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:56:34.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.538+0000 7f685159a700 1 -- 172.21.15.162:0/3515886164 >> v1:172.21.15.162:6789/0 conn(0x7f685405cbd0 legacy=0x7f68540aa9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:34.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.538+0000 7f685159a700 1 --2- 172.21.15.162:0/3515886164 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6838003900 0x7f6838003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:34.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.538+0000 7f685159a700 1 --2- 172.21.15.162:0/3515886164 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f68380044d0 0x7f6838004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:34.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.538+0000 7f685159a700 1 --2- 172.21.15.162:0/3515886164 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6838004e40 0x7f683800b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:34.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.538+0000 7f685159a700 1 -- 172.21.15.162:0/3515886164 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683800bc10 con 0x7f6838004e40 2020-11-28T21:56:34.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.538+0000 7f685159a700 1 -- 172.21.15.162:0/3515886164 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683800bd50 con 0x7f6838003900 2020-11-28T21:56:34.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.538+0000 7f685259c700 1 --2- 172.21.15.162:0/3515886164 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6838003900 0x7f6838003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:34.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.538+0000 7f685159a700 1 -- 172.21.15.162:0/3515886164 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f683800be90 con 0x7f68380044d0 2020-11-28T21:56:34.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.538+0000 7f6851d9b700 1 --2- 172.21.15.162:0/3515886164 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f68380044d0 0x7f6838004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:34.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.538+0000 7f6852d9d700 1 --2- 172.21.15.162:0/3515886164 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6838004e40 0x7f683800b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:34.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.538+0000 7f685259c700 1 -- 172.21.15.162:0/3515886164 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f68380044d0 msgr2=0x7f6838004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:34.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.538+0000 7f685259c700 1 --2- 172.21.15.162:0/3515886164 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f68380044d0 0x7f6838004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:34.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.538+0000 7f685259c700 1 -- 172.21.15.162:0/3515886164 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6838004e40 msgr2=0x7f683800b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:34.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.538+0000 7f685259c700 1 --2- 172.21.15.162:0/3515886164 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6838004e40 0x7f683800b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:34.555 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.538+0000 7f6852d9d700 1 -- 172.21.15.162:0/3515886164 reap_dead start 2020-11-28T21:56:34.555 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.538+0000 7f685259c700 1 -- 172.21.15.162:0/3515886164 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6838003230 con 0x7f6838003900 2020-11-28T21:56:34.555 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.538+0000 7f685259c700 1 --2- 172.21.15.162:0/3515886164 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6838003900 0x7f6838003d00 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 rx=0x7f684400c000 tx=0x7f6844008040).ready entity=mon.1 client_cookie=cb601dbbf9032d72 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:34.555 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.538+0000 7f685159a700 1 -- 172.21.15.162:0/3515886164 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f684400ec30 con 0x7f6838003900 2020-11-28T21:56:34.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.538+0000 7f685159a700 1 -- 172.21.15.162:0/3515886164 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684400ed90 con 0x7f6838003900 2020-11-28T21:56:34.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.538+0000 7f685159a700 1 -- 172.21.15.162:0/3515886164 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68440125e0 con 0x7f6838003900 2020-11-28T21:56:34.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.538+0000 7f685359e700 1 -- 172.21.15.162:0/3515886164 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6838003900 msgr2=0x7f6838003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:34.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.538+0000 7f685359e700 1 --2- 172.21.15.162:0/3515886164 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6838003900 0x7f6838003d00 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 rx=0x7f684400c000 tx=0x7f6844008040).stop 2020-11-28T21:56:34.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.538+0000 7f685359e700 1 -- 172.21.15.162:0/3515886164 shutdown_connections 2020-11-28T21:56:34.557 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.538+0000 7f685359e700 1 --2- 172.21.15.162:0/3515886164 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6838003900 0x7f6838003d00 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:34.557 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.538+0000 7f685359e700 1 -- 172.21.15.162:0/3515886164 >> 172.21.15.162:0/3515886164 conn(0x7f685405b970 msgr2=0x7f685405bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:34.557 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.538+0000 7f685359e700 1 -- 172.21.15.162:0/3515886164 shutdown_connections 2020-11-28T21:56:34.557 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.538+0000 7f685359e700 1 -- 172.21.15.162:0/3515886164 wait complete. 2020-11-28T21:56:34.557 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.542+0000 7f685359e700 1 Processor -- start 2020-11-28T21:56:34.558 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.542+0000 7f685359e700 1 -- start start 2020-11-28T21:56:34.562 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.542+0000 7f685359e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6838003900 0x7f6854141930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:34.562 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.542+0000 7f685359e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f685408fec0 0x7f6854090340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:34.562 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.542+0000 7f685359e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6854090830 0x7f6854151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:34.562 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.542+0000 7f685359e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6854067160 con 0x7f6838003900 2020-11-28T21:56:34.562 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.542+0000 7f685359e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68540673c0 con 0x7f685408fec0 2020-11-28T21:56:34.563 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.542+0000 7f685359e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6854066890 con 0x7f6854090830 2020-11-28T21:56:34.563 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.542+0000 7f6851d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f685408fec0 0x7f6854090340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:34.563 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.542+0000 7f6851d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f685408fec0 0x7f6854090340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46580/0 (socket says 172.21.15.162:46580) 2020-11-28T21:56:34.563 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.542+0000 7f6851d9b700 1 -- 172.21.15.162:0/1418615423 learned_addr learned my addr 172.21.15.162:0/1418615423 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:34.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.542+0000 7f685259c700 1 --2- 172.21.15.162:0/1418615423 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6838003900 0x7f6854141930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:34.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.542+0000 7f6852d9d700 1 --2- 172.21.15.162:0/1418615423 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6854090830 0x7f6854151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:34.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.542+0000 7f6851d9b700 1 -- 172.21.15.162:0/1418615423 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6854090830 msgr2=0x7f6854151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:34.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.542+0000 7f6851d9b700 1 --2- 172.21.15.162:0/1418615423 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6854090830 0x7f6854151eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:34.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.542+0000 7f6851d9b700 1 -- 172.21.15.162:0/1418615423 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6838003900 msgr2=0x7f6854141930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:34.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.542+0000 7f6851d9b700 1 --2- 172.21.15.162:0/1418615423 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6838003900 0x7f6854141930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:34.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.542+0000 7f6851d9b700 1 -- 172.21.15.162:0/1418615423 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6854066b30 con 0x7f685408fec0 2020-11-28T21:56:34.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.542+0000 7f6851d9b700 1 --2- 172.21.15.162:0/1418615423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f685408fec0 0x7f6854090340 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 rx=0x7f684800b000 tx=0x7f6848001db0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:34.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.542+0000 7f68437fe700 1 -- 172.21.15.162:0/1418615423 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f684800c940 con 0x7f685408fec0 2020-11-28T21:56:34.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.542+0000 7f68437fe700 1 -- 172.21.15.162:0/1418615423 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684800caa0 con 0x7f685408fec0 2020-11-28T21:56:34.566 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.542+0000 7f685359e700 1 -- 172.21.15.162:0/1418615423 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6854092bb0 con 0x7f685408fec0 2020-11-28T21:56:34.566 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.542+0000 7f68437fe700 1 -- 172.21.15.162:0/1418615423 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f684800ce00 con 0x7f685408fec0 2020-11-28T21:56:34.566 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.542+0000 7f685359e700 1 -- 172.21.15.162:0/1418615423 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68540b4010 con 0x7f685408fec0 2020-11-28T21:56:34.566 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.546+0000 7f68437fe700 1 -- 172.21.15.162:0/1418615423 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6848014430 con 0x7f685408fec0 2020-11-28T21:56:34.567 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.546+0000 7f685359e700 1 -- 172.21.15.162:0/1418615423 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6854066cc0 con 0x7f685408fec0 2020-11-28T21:56:34.567 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.546+0000 7f68437fe700 1 --2- 172.21.15.162:0/1418615423 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6830055470 0x7f6830057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:34.567 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.546+0000 7f685259c700 1 --2- 172.21.15.162:0/1418615423 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6830055470 0x7f6830057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:34.567 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.546+0000 7f68437fe700 1 -- 172.21.15.162:0/1418615423 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f684806e070 con 0x7f685408fec0 2020-11-28T21:56:34.567 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.546+0000 7f685259c700 1 --2- 172.21.15.162:0/1418615423 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6830055470 0x7f6830057930 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7f6844007000 tx=0x7f684401a000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:34.568 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.550+0000 7f68437fe700 1 -- 172.21.15.162:0/1418615423 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6848044310 con 0x7f685408fec0 2020-11-28T21:56:34.737 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.730+0000 7f685359e700 1 -- 172.21.15.162:0/1418615423 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6854066ef0 con 0x7f685408fec0 2020-11-28T21:56:34.738 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.734+0000 7f68437fe700 1 -- 172.21.15.162:0/1418615423 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+2501 (secure 0 0 0) 0x7f6848047930 con 0x7f685408fec0 2020-11-28T21:56:34.738 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:56:34.738 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":80,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8199,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/2385081970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":2385081970},{"type":"v1","addr":"172.21.15.118:6833","nonce":2385081970}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":77},{"gid":8554,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6836/169306274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":169306274},{"type":"v1","addr":"172.21.15.162:6836","nonce":169306274}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":78}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:56:25.831016+0000","modified":"2020-11-28T21:56:26.833832+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8557},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8557":{"gid":8557,"name":"c","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/3753662046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":3753662046},{"type":"v1","addr":"172.21.15.162:6837","nonce":3753662046}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2020-11-28T21:56:34.740 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.734+0000 7f685359e700 1 -- 172.21.15.162:0/1418615423 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6830055470 msgr2=0x7f6830057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:34.740 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.734+0000 7f685359e700 1 --2- 172.21.15.162:0/1418615423 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6830055470 0x7f6830057930 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7f6844007000 tx=0x7f684401a000).stop 2020-11-28T21:56:34.741 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.734+0000 7f685359e700 1 -- 172.21.15.162:0/1418615423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f685408fec0 msgr2=0x7f6854090340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:34.741 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.734+0000 7f685359e700 1 --2- 172.21.15.162:0/1418615423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f685408fec0 0x7f6854090340 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 rx=0x7f684800b000 tx=0x7f6848001db0).stop 2020-11-28T21:56:34.741 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.734+0000 7f685359e700 1 -- 172.21.15.162:0/1418615423 shutdown_connections 2020-11-28T21:56:34.741 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.734+0000 7f685359e700 1 --2- 172.21.15.162:0/1418615423 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6830055470 0x7f6830057930 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:34.742 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.734+0000 7f685359e700 1 --2- 172.21.15.162:0/1418615423 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6854090830 0x7f6854151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:34.742 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.734+0000 7f685359e700 1 --2- 172.21.15.162:0/1418615423 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6838003900 0x7f6854141930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:34.742 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.734+0000 7f685359e700 1 --2- 172.21.15.162:0/1418615423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f685408fec0 0x7f6854090340 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:34.743 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.734+0000 7f685359e700 1 -- 172.21.15.162:0/1418615423 >> 172.21.15.162:0/1418615423 conn(0x7f685405b970 msgr2=0x7f68540b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:34.743 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.734+0000 7f685359e700 1 -- 172.21.15.162:0/1418615423 shutdown_connections 2020-11-28T21:56:34.743 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:34.734+0000 7f685359e700 1 -- 172.21.15.162:0/1418615423 wait complete. 2020-11-28T21:56:34.744 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 80 2020-11-28T21:56:35.053 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 80, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-28T21:56:25.831016+0000', 'modified': '2020-11-28T21:56:26.833832+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8557}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8557': {'gid': 8557, 'name': 'c', 'rank': 0, 'incarnation': 79, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6837/3753662046', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6835', 'nonce': 3753662046}, {'type': 'v1', 'addr': '172.21.15.162:6837', 'nonce': 3753662046}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-28T21:56:35.054 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-28T21:56:35.055 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-28T21:56:35.185 INFO:teuthology.orchestra.run.smithi162.stdout:{"cluster_fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","whoami":0,"id":8557,"want_state":"up:active","state":"up:active","rank_uptime":9.3403047870000009,"mdsmap_epoch":80,"osdmap_epoch":160,"osdmap_epoch_barrier":158,"uptime":14.754725375} 2020-11-28T21:56:35.185 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", "id": 8557, "mdsmap_epoch": 80, "osdmap_epoch": 160, "osdmap_epoch_barrier": 158, "rank_uptime": 9.340304787, "state": "up:active", "uptime": 14.754725375, "want_state": "up:active", "whoami": 0 } 2020-11-28T21:56:35.186 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-11-28T21:56:35.186 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2020-11-28T21:56:35.187 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi096.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2020-11-28T21:56:35.187 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:56:35.187 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:56:35.197 INFO:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2020-11-28T21:56:35.255 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:56:35.256 INFO:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:56:35.310 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:56:35.312 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:56:35.312 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:56:35.313 INFO:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2020-11-28T21:56:35.362 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-11-28T21:56:35.363 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:56:35.406 INFO:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2020-11-28T21:56:35.421 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:56:35.422 INFO:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:56:35.477 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:56:35.479 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:56:35.479 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:56:35.480 INFO:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2020-11-28T21:56:35.483 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:2020-11-28T21:56:35.479+0000 7f46a4e5e0c0 -1 init, newargv = 0x5565dcb3e0c0 newargc=9 2020-11-28T21:56:35.483 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[12270]: starting ceph client 2020-11-28T21:56:35.493 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[12270]: starting fuse 2020-11-28T21:56:36.589 INFO:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2020-11-28T21:56:36.606 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-28T21:56:36.607 INFO:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-28T21:56:36.668 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-11-28T21:56:36.668 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-28T21:56:36.669 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:56:36.669 INFO:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2020-11-28T21:56:36.714 INFO:teuthology.orchestra.run.smithi096.stdout:49 2020-11-28T21:56:36.714 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2020-11-28T21:56:36.715 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:56:36.715 INFO:teuthology.orchestra.run.smithi096:> sudo python3 -c ' 2020-11-28T21:56:36.716 INFO:teuthology.orchestra.run.smithi096:> import glob 2020-11-28T21:56:36.716 INFO:teuthology.orchestra.run.smithi096:> import re 2020-11-28T21:56:36.716 INFO:teuthology.orchestra.run.smithi096:> import os 2020-11-28T21:56:36.717 INFO:teuthology.orchestra.run.smithi096:> import subprocess 2020-11-28T21:56:36.717 INFO:teuthology.orchestra.run.smithi096:> 2020-11-28T21:56:36.718 INFO:teuthology.orchestra.run.smithi096:> def find_socket(client_name): 2020-11-28T21:56:36.718 INFO:teuthology.orchestra.run.smithi096:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2020-11-28T21:56:36.718 INFO:teuthology.orchestra.run.smithi096:> files = glob.glob(asok_path) 2020-11-28T21:56:36.719 INFO:teuthology.orchestra.run.smithi096:> 2020-11-28T21:56:36.719 INFO:teuthology.orchestra.run.smithi096:> # Given a non-glob path, it better be there 2020-11-28T21:56:36.720 INFO:teuthology.orchestra.run.smithi096:> if "*" not in asok_path: 2020-11-28T21:56:36.720 INFO:teuthology.orchestra.run.smithi096:> assert(len(files) == 1) 2020-11-28T21:56:36.720 INFO:teuthology.orchestra.run.smithi096:> return files[0] 2020-11-28T21:56:36.721 INFO:teuthology.orchestra.run.smithi096:> 2020-11-28T21:56:36.721 INFO:teuthology.orchestra.run.smithi096:> for f in files: 2020-11-28T21:56:36.721 INFO:teuthology.orchestra.run.smithi096:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-11-28T21:56:36.722 INFO:teuthology.orchestra.run.smithi096:> if os.path.exists("/proc/{0}".format(pid)): 2020-11-28T21:56:36.722 INFO:teuthology.orchestra.run.smithi096:> return f 2020-11-28T21:56:36.723 INFO:teuthology.orchestra.run.smithi096:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-11-28T21:56:36.723 INFO:teuthology.orchestra.run.smithi096:> 2020-11-28T21:56:36.723 INFO:teuthology.orchestra.run.smithi096:> print(find_socket("client.1")) 2020-11-28T21:56:36.724 INFO:teuthology.orchestra.run.smithi096:> ' 2020-11-28T21:56:36.807 INFO:teuthology.orchestra.run.smithi096.stdout:/var/run/ceph/ceph-client.1.12270.asok 2020-11-28T21:56:36.811 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.12270.asok 2020-11-28T21:56:36.812 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:56:36.813 INFO:teuthology.orchestra.run.smithi096:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.12270.asok status 2020-11-28T21:56:36.912 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-11-28T21:56:36.913 INFO:teuthology.orchestra.run.smithi096.stdout: "metadata": { 2020-11-28T21:56:36.913 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_sha1": "a83a3572a957356977342f461361c1cd3ec06dbd", 2020-11-28T21:56:36.913 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_version": "ceph version 15.2.6-595-ga83a3572 (a83a3572a957356977342f461361c1cd3ec06dbd) octopus (stable)", 2020-11-28T21:56:36.913 INFO:teuthology.orchestra.run.smithi096.stdout: "entity_id": "1", 2020-11-28T21:56:36.913 INFO:teuthology.orchestra.run.smithi096.stdout: "hostname": "smithi096", 2020-11-28T21:56:36.914 INFO:teuthology.orchestra.run.smithi096.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2020-11-28T21:56:36.914 INFO:teuthology.orchestra.run.smithi096.stdout: "pid": "12270", 2020-11-28T21:56:36.914 INFO:teuthology.orchestra.run.smithi096.stdout: "root": "/" 2020-11-28T21:56:36.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-11-28T21:56:36.914 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_count": 0, 2020-11-28T21:56:36.915 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_pinned_count": 0, 2020-11-28T21:56:36.915 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 8767, 2020-11-28T21:56:36.915 INFO:teuthology.orchestra.run.smithi096.stdout: "inst": { 2020-11-28T21:56:36.915 INFO:teuthology.orchestra.run.smithi096.stdout: "name": { 2020-11-28T21:56:36.915 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "client", 2020-11-28T21:56:36.916 INFO:teuthology.orchestra.run.smithi096.stdout: "num": 8767 2020-11-28T21:56:36.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-11-28T21:56:36.916 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2020-11-28T21:56:36.916 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2020-11-28T21:56:36.916 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2020-11-28T21:56:36.917 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2696703708 2020-11-28T21:56:36.917 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-11-28T21:56:36.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-11-28T21:56:36.917 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2020-11-28T21:56:36.917 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2020-11-28T21:56:36.918 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2020-11-28T21:56:36.918 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2696703708 2020-11-28T21:56:36.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-11-28T21:56:36.918 INFO:teuthology.orchestra.run.smithi096.stdout: "inst_str": "client.8767 172.21.15.96:0/2696703708", 2020-11-28T21:56:36.919 INFO:teuthology.orchestra.run.smithi096.stdout: "addr_str": "172.21.15.96:0/2696703708", 2020-11-28T21:56:36.919 INFO:teuthology.orchestra.run.smithi096.stdout: "inode_count": 1, 2020-11-28T21:56:36.919 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_epoch": 80, 2020-11-28T21:56:36.919 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch": 160, 2020-11-28T21:56:36.919 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch_barrier": 0, 2020-11-28T21:56:36.920 INFO:teuthology.orchestra.run.smithi096.stdout: "blacklisted": false 2020-11-28T21:56:36.920 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-11-28T21:56:36.921 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:56:36.922 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:56:36.922 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:56:36.930 INFO:teuthology.orchestra.run.smithi096.stdout:fuseblk 2020-11-28T21:56:36.931 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-11-28T21:56:36.932 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:56:36.932 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:56:36.989 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:56:36.989 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:56:36.990 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:56:37.039 INFO:teuthology.orchestra.run.smithi096.stdout:fuseblk 2020-11-28T21:56:37.041 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-11-28T21:56:37.041 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:56:37.042 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:56:37.042 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:56:37.092 INFO:teuthology.orchestra.run.smithi096.stdout:fuseblk 2020-11-28T21:56:37.093 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-11-28T21:56:37.094 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi096.front.sepia.ceph.com... 2020-11-28T21:56:37.096 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-28T21:56:37.096 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:56:37.170 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[12270]: fuse finished with error 0 and tester_r 0 2020-11-28T21:56:37.171 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:56:37.172 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:56:37.173 INFO:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:56:37.219 INFO:teuthology.orchestra.run.smithi096.stdout:ext2/ext3 2020-11-28T21:56:37.220 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-11-28T21:56:37.220 INFO:teuthology.orchestra.run:waiting for 900 2020-11-28T21:56:37.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-28T21:56:37.278 INFO:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-11-28T21:56:37.285 INFO:teuthology.orchestra.run.smithi192:> cd /home/ubuntu/cephtest/mnt.0 && sudo bash -c 'stat -f .' 2020-11-28T21:56:37.301 INFO:teuthology.orchestra.run.smithi192.stdout: File: "." 2020-11-28T21:56:37.301 INFO:teuthology.orchestra.run.smithi192.stdout: ID: 0 Namelen: 255 Type: fuseblk 2020-11-28T21:56:37.302 INFO:teuthology.orchestra.run.smithi192.stdout:Block size: 4194304 Fundamental block size: 4194304 2020-11-28T21:56:37.302 INFO:teuthology.orchestra.run.smithi192.stdout:Blocks: Total: 86526 Free: 86526 Available: 86526 2020-11-28T21:56:37.302 INFO:teuthology.orchestra.run.smithi192.stdout:Inodes: Total: 1 Free: 0 2020-11-28T21:56:37.303 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2020-11-28T21:56:37.422 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.418+0000 7f5f30871700 1 Processor -- start 2020-11-28T21:56:37.423 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.418+0000 7f5f30871700 1 -- start start 2020-11-28T21:56:37.423 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.418+0000 7f5f30871700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5f2c09d890 0x7f5f2c14a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:37.424 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.418+0000 7f5f30871700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5f2c0608f0 0x7f5f2c14ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:37.424 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.418+0000 7f5f30871700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f2c0b6210 con 0x7f5f2c0b5e30 2020-11-28T21:56:37.424 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.418+0000 7f5f30871700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f2c0606e0 con 0x7f5f2c09d890 2020-11-28T21:56:37.426 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.418+0000 7f5f30871700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f2c0b4a70 con 0x7f5f2c0608f0 2020-11-28T21:56:37.426 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.418+0000 7f5f2b59e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5f2c0608f0 0x7f5f2c14ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:37.427 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.418+0000 7f5f2b59e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5f2c0608f0 0x7f5f2c14ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46590/0 (socket says 172.21.15.162:46590) 2020-11-28T21:56:37.427 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.418+0000 7f5f2b59e700 1 -- 172.21.15.162:0/3102640297 learned_addr learned my addr 172.21.15.162:0/3102640297 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:37.428 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.418+0000 7f5f2a59c700 1 --2- 172.21.15.162:0/3102640297 >> v2:172.21.15.118:3300/0 conn(0x7f5f2c09d890 0x7f5f2c14a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:37.435 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.418+0000 7f5f2b59e700 1 -- 172.21.15.162:0/3102640297 >> v2:172.21.15.118:3300/0 conn(0x7f5f2c09d890 msgr2=0x7f5f2c14a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:37.435 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.418+0000 7f5f2b59e700 1 --2- 172.21.15.162:0/3102640297 >> v2:172.21.15.118:3300/0 conn(0x7f5f2c09d890 0x7f5f2c14a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:37.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.418+0000 7f5f2b59e700 1 -- 172.21.15.162:0/3102640297 >> v1:172.21.15.118:6789/0 conn(0x7f5f2c0b5e30 legacy=0x7f5f2c09d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:37.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.418+0000 7f5f2b59e700 1 -- 172.21.15.162:0/3102640297 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f2c0b3d70 con 0x7f5f2c0608f0 2020-11-28T21:56:37.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.418+0000 7f5f2b59e700 1 --2- 172.21.15.162:0/3102640297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f2c0608f0 0x7f5f2c14ae20 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 rx=0x7f5f24002f30 tx=0x7f5f24006e80).ready entity=mon.1 client_cookie=8fde1267399e65e3 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:37.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.418+0000 7f5f29d9b700 1 -- 172.21.15.162:0/3102640297 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f240097f0 con 0x7f5f2c0608f0 2020-11-28T21:56:37.437 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.418+0000 7f5f29d9b700 1 -- 172.21.15.162:0/3102640297 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f24009950 con 0x7f5f2c0608f0 2020-11-28T21:56:37.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.418+0000 7f5f29d9b700 1 -- 172.21.15.162:0/3102640297 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f24009cb0 con 0x7f5f2c0608f0 2020-11-28T21:56:37.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.418+0000 7f5f30871700 1 -- 172.21.15.162:0/3102640297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f2c0608f0 msgr2=0x7f5f2c14ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:37.438 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.418+0000 7f5f30871700 1 --2- 172.21.15.162:0/3102640297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f2c0608f0 0x7f5f2c14ae20 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 rx=0x7f5f24002f30 tx=0x7f5f24006e80).stop 2020-11-28T21:56:37.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.422+0000 7f5f30871700 1 -- 172.21.15.162:0/3102640297 shutdown_connections 2020-11-28T21:56:37.439 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.422+0000 7f5f30871700 1 --2- 172.21.15.162:0/3102640297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f2c0608f0 0x7f5f2c14ae20 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:37.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.422+0000 7f5f30871700 1 --2- 172.21.15.162:0/3102640297 >> v2:172.21.15.118:3300/0 conn(0x7f5f2c09d890 0x7f5f2c14a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:37.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.422+0000 7f5f30871700 1 -- 172.21.15.162:0/3102640297 >> 172.21.15.162:0/3102640297 conn(0x7f5f2c05b970 msgr2=0x7f5f2c05bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:37.440 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.422+0000 7f5f30871700 1 -- 172.21.15.162:0/3102640297 shutdown_connections 2020-11-28T21:56:37.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.422+0000 7f5f30871700 1 -- 172.21.15.162:0/3102640297 wait complete. 2020-11-28T21:56:37.441 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.422+0000 7f5f30871700 1 Processor -- start 2020-11-28T21:56:37.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.422+0000 7f5f30871700 1 -- start start 2020-11-28T21:56:37.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.422+0000 7f5f30871700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5f2c0608f0 0x7f5f2c0ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:37.442 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.422+0000 7f5f30871700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f2c09d890 0x7f5f2c156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:37.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.422+0000 7f5f30871700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f2c0b5e30 0x7f5f2c0b24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:37.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.422+0000 7f5f30871700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c0b1cc0 con 0x7f5f2c0b5e30 2020-11-28T21:56:37.444 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.422+0000 7f5f30871700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c0ae8a0 con 0x7f5f2c09d890 2020-11-28T21:56:37.444 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.422+0000 7f5f30871700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c0add20 con 0x7f5f2c0608f0 2020-11-28T21:56:37.445 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.422+0000 7f5f2a59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f2c09d890 0x7f5f2c156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:37.445 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.422+0000 7f5f2a59c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f2c09d890 0x7f5f2c156060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46594/0 (socket says 172.21.15.162:46594) 2020-11-28T21:56:37.445 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.422+0000 7f5f2a59c700 1 -- 172.21.15.162:0/1484993696 learned_addr learned my addr 172.21.15.162:0/1484993696 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:37.446 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.422+0000 7f5f2ad9d700 1 --2- 172.21.15.162:0/1484993696 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5f2c0608f0 0x7f5f2c0ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:37.446 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.422+0000 7f5f2b59e700 1 --2- 172.21.15.162:0/1484993696 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f2c0b5e30 0x7f5f2c0b24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:37.447 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.426+0000 7f5f2a59c700 1 -- 172.21.15.162:0/1484993696 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5f2c0608f0 msgr2=0x7f5f2c0ae3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:37.447 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.426+0000 7f5f2a59c700 1 --2- 172.21.15.162:0/1484993696 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5f2c0608f0 0x7f5f2c0ae3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:37.447 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.426+0000 7f5f2a59c700 1 -- 172.21.15.162:0/1484993696 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f2c0b5e30 msgr2=0x7f5f2c0b24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:37.448 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.426+0000 7f5f2a59c700 1 --2- 172.21.15.162:0/1484993696 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f2c0b5e30 0x7f5f2c0b24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:37.448 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.426+0000 7f5f2a59c700 1 -- 172.21.15.162:0/1484993696 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f2c0aedd0 con 0x7f5f2c09d890 2020-11-28T21:56:37.449 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.426+0000 7f5f2a59c700 1 --2- 172.21.15.162:0/1484993696 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f2c09d890 0x7f5f2c156060 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 rx=0x7f5f1c00c000 tx=0x7f5f1c006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:37.449 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.426+0000 7f5f1bfff700 1 -- 172.21.15.162:0/1484993696 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f1c00ca10 con 0x7f5f2c09d890 2020-11-28T21:56:37.449 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.426+0000 7f5f1bfff700 1 -- 172.21.15.162:0/1484993696 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1c00cb70 con 0x7f5f2c09d890 2020-11-28T21:56:37.450 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.426+0000 7f5f30871700 1 -- 172.21.15.162:0/1484993696 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f2c0af070 con 0x7f5f2c09d890 2020-11-28T21:56:37.450 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.426+0000 7f5f1bfff700 1 -- 172.21.15.162:0/1484993696 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f1c0123f0 con 0x7f5f2c09d890 2020-11-28T21:56:37.451 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.426+0000 7f5f30871700 1 -- 172.21.15.162:0/1484993696 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f2c156550 con 0x7f5f2c09d890 2020-11-28T21:56:37.451 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.426+0000 7f5f30871700 1 -- 172.21.15.162:0/1484993696 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f2c057050 con 0x7f5f2c09d890 2020-11-28T21:56:37.452 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.426+0000 7f5f1bfff700 1 -- 172.21.15.162:0/1484993696 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f5f1c012550 con 0x7f5f2c09d890 2020-11-28T21:56:37.452 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.426+0000 7f5f1bfff700 1 --2- 172.21.15.162:0/1484993696 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5f080553b0 0x7f5f08057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:37.452 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.426+0000 7f5f1bfff700 1 -- 172.21.15.162:0/1484993696 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5f1c06e370 con 0x7f5f2c09d890 2020-11-28T21:56:37.453 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.426+0000 7f5f2ad9d700 1 --2- 172.21.15.162:0/1484993696 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5f080553b0 0x7f5f08057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:37.453 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.426+0000 7f5f2ad9d700 1 --2- 172.21.15.162:0/1484993696 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5f080553b0 0x7f5f08057870 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7f5f2000d000 tx=0x7f5f20007ee0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:37.454 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.434+0000 7f5f1bfff700 1 -- 172.21.15.162:0/1484993696 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f5f1c044610 con 0x7f5f2c09d890 2020-11-28T21:56:37.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.574+0000 7f5f30871700 1 -- 172.21.15.162:0/1484993696 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f5f2c0acb80 con 0x7f5f2c09d890 2020-11-28T21:56:37.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.578+0000 7f5f1bfff700 1 -- 172.21.15.162:0/1484993696 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v160) v1 ==== 81+0+33755 (secure 0 0 0) 0x7f5f1c047c30 con 0x7f5f2c09d890 2020-11-28T21:56:37.583 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:56:37.584 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2020-11-28T21:56:37.584 INFO:teuthology.orchestra.run.smithi162.stdout: "epoch": 160, 2020-11-28T21:56:37.584 INFO:teuthology.orchestra.run.smithi162.stdout: "fsid": "7c321bcf-cd33-40f7-a39f-da5470b386cf", 2020-11-28T21:56:37.584 INFO:teuthology.orchestra.run.smithi162.stdout: "created": "2020-11-28T21:46:11.724372+0000", 2020-11-28T21:56:37.584 INFO:teuthology.orchestra.run.smithi162.stdout: "modified": "2020-11-28T21:56:27.855579+0000", 2020-11-28T21:56:37.585 INFO:teuthology.orchestra.run.smithi162.stdout: "last_up_change": "2020-11-28T21:46:19.591650+0000", 2020-11-28T21:56:37.585 INFO:teuthology.orchestra.run.smithi162.stdout: "last_in_change": "2020-11-28T21:46:19.591650+0000", 2020-11-28T21:56:37.585 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-11-28T21:56:37.585 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_num": 5799936, 2020-11-28T21:56:37.586 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_set": [ 2020-11-28T21:56:37.586 INFO:teuthology.orchestra.run.smithi162.stdout: "pglog_hardlimit", 2020-11-28T21:56:37.586 INFO:teuthology.orchestra.run.smithi162.stdout: "purged_snapdirs", 2020-11-28T21:56:37.586 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_deletes", 2020-11-28T21:56:37.586 INFO:teuthology.orchestra.run.smithi162.stdout: "sortbitwise" 2020-11-28T21:56:37.587 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:56:37.587 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_version": 4, 2020-11-28T21:56:37.587 INFO:teuthology.orchestra.run.smithi162.stdout: "full_ratio": 0.94999998807907104, 2020-11-28T21:56:37.587 INFO:teuthology.orchestra.run.smithi162.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-11-28T21:56:37.587 INFO:teuthology.orchestra.run.smithi162.stdout: "nearfull_ratio": 0.85000002384185791, 2020-11-28T21:56:37.588 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_snapshot": "", 2020-11-28T21:56:37.588 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_max": 30, 2020-11-28T21:56:37.588 INFO:teuthology.orchestra.run.smithi162.stdout: "max_osd": 8, 2020-11-28T21:56:37.588 INFO:teuthology.orchestra.run.smithi162.stdout: "require_min_compat_client": "jewel", 2020-11-28T21:56:37.588 INFO:teuthology.orchestra.run.smithi162.stdout: "min_compat_client": "jewel", 2020-11-28T21:56:37.589 INFO:teuthology.orchestra.run.smithi162.stdout: "require_osd_release": "octopus", 2020-11-28T21:56:37.589 INFO:teuthology.orchestra.run.smithi162.stdout: "pools": [ 2020-11-28T21:56:37.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.589 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 1, 2020-11-28T21:56:37.589 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "device_health_metrics", 2020-11-28T21:56:37.590 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:46:15.639551+0000", 2020-11-28T21:56:37.590 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:56:37.590 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:56:37.590 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:56:37.591 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:56:37.591 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:56:37.591 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:56:37.592 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:56:37.592 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:56:37.592 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 1, 2020-11-28T21:56:37.592 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 1, 2020-11-28T21:56:37.593 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 1, 2020-11-28T21:56:37.593 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 1, 2020-11-28T21:56:37.593 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 1, 2020-11-28T21:56:37.593 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:56:37.593 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:56:37.594 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:56:37.594 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:56:37.594 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:56:37.594 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:56:37.594 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:56:37.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.595 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "16", 2020-11-28T21:56:37.595 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:56:37.595 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:56:37.595 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:56:37.596 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:56:37.596 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:56:37.596 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:56:37.596 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:56:37.597 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:56:37.597 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:56:37.597 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:56:37.598 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:56:37.598 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:56:37.598 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:56:37.599 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:56:37.599 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:56:37.599 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:56:37.599 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:56:37.599 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:56:37.600 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:56:37.600 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:56:37.600 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:56:37.600 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:56:37.601 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:56:37.601 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:56:37.601 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:56:37.601 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:56:37.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.602 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:56:37.602 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:56:37.602 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:56:37.602 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:56:37.602 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:56:37.603 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:56:37.603 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:56:37.603 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:56:37.603 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:56:37.603 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:56:37.604 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:56:37.604 INFO:teuthology.orchestra.run.smithi162.stdout: "options": { 2020-11-28T21:56:37.604 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_min": 1 2020-11-28T21:56:37.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.605 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:56:37.605 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr_devicehealth": {} 2020-11-28T21:56:37.605 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:37.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.606 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 2, 2020-11-28T21:56:37.606 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "rbd", 2020-11-28T21:56:37.606 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:46:23.901366+0000", 2020-11-28T21:56:37.606 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:56:37.607 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:56:37.607 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:56:37.607 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:56:37.607 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:56:37.607 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:56:37.608 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:56:37.608 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:56:37.608 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2020-11-28T21:56:37.608 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2020-11-28T21:56:37.608 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2020-11-28T21:56:37.609 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2020-11-28T21:56:37.609 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2020-11-28T21:56:37.609 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:56:37.609 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:56:37.609 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:56:37.610 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:56:37.610 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:56:37.610 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:56:37.610 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:56:37.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.611 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "19", 2020-11-28T21:56:37.611 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:56:37.611 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:56:37.611 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:56:37.611 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:56:37.612 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:56:37.612 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:56:37.612 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:56:37.612 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:56:37.612 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:56:37.613 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:56:37.613 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:56:37.613 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:56:37.613 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:56:37.613 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:56:37.614 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:56:37.614 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:56:37.614 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:56:37.614 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:56:37.614 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:56:37.615 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:56:37.615 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:56:37.615 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:56:37.615 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:56:37.616 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:56:37.616 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:56:37.616 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:56:37.616 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.616 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:56:37.617 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:56:37.617 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:56:37.617 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:56:37.617 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:56:37.618 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:56:37.618 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:56:37.618 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:56:37.618 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:56:37.618 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:56:37.619 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:56:37.619 INFO:teuthology.orchestra.run.smithi162.stdout: "options": {}, 2020-11-28T21:56:37.619 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:56:37.619 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd": {} 2020-11-28T21:56:37.619 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:37.620 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.620 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.620 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 29, 2020-11-28T21:56:37.620 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "cephfs_metadata", 2020-11-28T21:56:37.620 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:56:23.335325+0000", 2020-11-28T21:56:37.621 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:56:37.621 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:56:37.621 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:56:37.621 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:56:37.621 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:56:37.622 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:56:37.622 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:56:37.622 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:56:37.622 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2020-11-28T21:56:37.622 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2020-11-28T21:56:37.623 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2020-11-28T21:56:37.623 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2020-11-28T21:56:37.623 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2020-11-28T21:56:37.623 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:56:37.623 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:56:37.624 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:56:37.624 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:56:37.624 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:56:37.624 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:56:37.624 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:56:37.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.625 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "159", 2020-11-28T21:56:37.625 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:56:37.625 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:56:37.625 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:56:37.626 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:56:37.626 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:56:37.626 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:56:37.626 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:56:37.626 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:56:37.627 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:56:37.627 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:56:37.627 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:56:37.627 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:56:37.628 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:56:37.628 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:56:37.628 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:56:37.628 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:56:37.628 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:56:37.629 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:56:37.629 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:56:37.629 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:56:37.629 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:56:37.630 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:56:37.630 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:56:37.630 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:56:37.630 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:56:37.630 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:56:37.631 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.631 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:56:37.631 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:56:37.631 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:56:37.632 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:56:37.632 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:56:37.633 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:56:37.633 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:56:37.633 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:56:37.633 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:56:37.634 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:56:37.634 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:56:37.634 INFO:teuthology.orchestra.run.smithi162.stdout: "options": { 2020-11-28T21:56:37.635 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_bias": 4, 2020-11-28T21:56:37.635 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_min": 16, 2020-11-28T21:56:37.635 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_priority": 5 2020-11-28T21:56:37.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.635 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:56:37.636 INFO:teuthology.orchestra.run.smithi162.stdout: "cephfs": { 2020-11-28T21:56:37.636 INFO:teuthology.orchestra.run.smithi162.stdout: "metadata": "cephfs" 2020-11-28T21:56:37.636 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:37.636 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:37.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.637 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 30, 2020-11-28T21:56:37.637 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "cephfs_data", 2020-11-28T21:56:37.637 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2020-11-28T21:56:24.648832+0000", 2020-11-28T21:56:37.637 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2020-11-28T21:56:37.638 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2020-11-28T21:56:37.638 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2020-11-28T21:56:37.638 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2020-11-28T21:56:37.638 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2020-11-28T21:56:37.639 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2020-11-28T21:56:37.639 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2020-11-28T21:56:37.639 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2020-11-28T21:56:37.639 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2020-11-28T21:56:37.639 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2020-11-28T21:56:37.640 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2020-11-28T21:56:37.640 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2020-11-28T21:56:37.640 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2020-11-28T21:56:37.640 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2020-11-28T21:56:37.640 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2020-11-28T21:56:37.641 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2020-11-28T21:56:37.641 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2020-11-28T21:56:37.641 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2020-11-28T21:56:37.641 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2020-11-28T21:56:37.641 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2020-11-28T21:56:37.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.642 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "160", 2020-11-28T21:56:37.642 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2020-11-28T21:56:37.642 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-28T21:56:37.643 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2020-11-28T21:56:37.643 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2020-11-28T21:56:37.643 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2020-11-28T21:56:37.643 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2020-11-28T21:56:37.644 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2020-11-28T21:56:37.644 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2020-11-28T21:56:37.644 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2020-11-28T21:56:37.644 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2020-11-28T21:56:37.644 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2020-11-28T21:56:37.645 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2020-11-28T21:56:37.645 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2020-11-28T21:56:37.645 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2020-11-28T21:56:37.645 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2020-11-28T21:56:37.646 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2020-11-28T21:56:37.646 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2020-11-28T21:56:37.646 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2020-11-28T21:56:37.646 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-28T21:56:37.646 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-28T21:56:37.647 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-28T21:56:37.647 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2020-11-28T21:56:37.647 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2020-11-28T21:56:37.647 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2020-11-28T21:56:37.648 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2020-11-28T21:56:37.648 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2020-11-28T21:56:37.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.648 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2020-11-28T21:56:37.649 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2020-11-28T21:56:37.649 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2020-11-28T21:56:37.649 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2020-11-28T21:56:37.649 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2020-11-28T21:56:37.650 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2020-11-28T21:56:37.650 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2020-11-28T21:56:37.650 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2020-11-28T21:56:37.650 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2020-11-28T21:56:37.651 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2020-11-28T21:56:37.651 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2020-11-28T21:56:37.651 INFO:teuthology.orchestra.run.smithi162.stdout: "options": {}, 2020-11-28T21:56:37.651 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2020-11-28T21:56:37.652 INFO:teuthology.orchestra.run.smithi162.stdout: "cephfs": { 2020-11-28T21:56:37.652 INFO:teuthology.orchestra.run.smithi162.stdout: "data": "cephfs" 2020-11-28T21:56:37.652 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:37.652 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:37.652 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:37.653 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:56:37.653 INFO:teuthology.orchestra.run.smithi162.stdout: "osds": [ 2020-11-28T21:56:37.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.654 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 0, 2020-11-28T21:56:37.654 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "99c7d9ae-adfa-4fd7-880a-23e6baf3281b", 2020-11-28T21:56:37.654 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:56:37.654 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:56:37.655 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:56:37.655 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:56:37.655 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:56:37.655 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:56:37.656 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:56:37.656 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 158, 2020-11-28T21:56:37.656 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:56:37.656 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:56:37.657 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:56:37.657 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:37.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.657 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:37.657 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6810", 2020-11-28T21:56:37.658 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:56:37.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.658 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:37.659 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6811", 2020-11-28T21:56:37.659 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:56:37.659 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:37.659 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:37.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.660 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:56:37.660 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:37.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.661 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:37.661 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6812", 2020-11-28T21:56:37.661 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:56:37.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.662 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:37.662 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6813", 2020-11-28T21:56:37.662 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:56:37.663 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:37.663 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:37.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.663 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:56:37.663 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:37.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.664 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:37.664 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6816", 2020-11-28T21:56:37.664 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:56:37.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.665 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:37.665 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6817", 2020-11-28T21:56:37.665 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:56:37.666 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:37.666 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:37.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.666 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:56:37.667 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:37.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.667 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:37.667 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6814", 2020-11-28T21:56:37.667 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:56:37.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.668 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:37.668 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6815", 2020-11-28T21:56:37.669 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12313 2020-11-28T21:56:37.669 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:37.669 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:37.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.670 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6811/12313", 2020-11-28T21:56:37.670 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6813/12313", 2020-11-28T21:56:37.670 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6817/12313", 2020-11-28T21:56:37.670 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6815/12313", 2020-11-28T21:56:37.671 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:56:37.671 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:56:37.671 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:56:37.672 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:37.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.672 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 1, 2020-11-28T21:56:37.673 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "6ce6220a-00ef-4d13-abba-56f87636fb5f", 2020-11-28T21:56:37.673 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:56:37.673 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:56:37.673 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:56:37.674 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:56:37.674 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:56:37.674 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:56:37.675 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:56:37.675 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 157, 2020-11-28T21:56:37.675 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:56:37.675 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:56:37.676 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:56:37.676 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:37.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.676 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:37.676 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6802", 2020-11-28T21:56:37.677 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:56:37.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.677 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:37.678 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6803", 2020-11-28T21:56:37.678 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:56:37.679 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:37.679 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:37.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.679 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:56:37.680 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:37.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.680 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:37.680 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6804", 2020-11-28T21:56:37.681 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:56:37.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.681 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:37.682 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6805", 2020-11-28T21:56:37.682 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:56:37.682 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:37.682 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:37.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.683 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:56:37.683 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:37.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.684 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:37.684 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6808", 2020-11-28T21:56:37.684 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:56:37.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.685 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:37.685 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6809", 2020-11-28T21:56:37.685 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:56:37.685 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:37.685 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:37.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.686 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:56:37.686 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:37.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.687 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:37.687 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6806", 2020-11-28T21:56:37.687 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:56:37.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.688 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:37.688 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6807", 2020-11-28T21:56:37.688 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12311 2020-11-28T21:56:37.688 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:37.688 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:37.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.689 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6803/12311", 2020-11-28T21:56:37.689 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6805/12311", 2020-11-28T21:56:37.689 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6809/12311", 2020-11-28T21:56:37.690 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6807/12311", 2020-11-28T21:56:37.690 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:56:37.690 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:56:37.690 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:56:37.691 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:37.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.692 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 2, 2020-11-28T21:56:37.692 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "db4e510d-27c1-47fc-a902-8b32e7dfd73e", 2020-11-28T21:56:37.692 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:56:37.693 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:56:37.693 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:56:37.693 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:56:37.693 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:56:37.694 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:56:37.694 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:56:37.694 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 157, 2020-11-28T21:56:37.694 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:56:37.694 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:56:37.695 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:56:37.695 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:37.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.695 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:37.695 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6818", 2020-11-28T21:56:37.696 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:56:37.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.696 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:37.696 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6819", 2020-11-28T21:56:37.697 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:56:37.697 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:37.697 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:37.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.698 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:56:37.698 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:37.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.699 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:37.699 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6820", 2020-11-28T21:56:37.699 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:56:37.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.700 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:37.700 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6821", 2020-11-28T21:56:37.701 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:56:37.701 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:37.701 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:37.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.702 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:56:37.702 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:37.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.702 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:37.703 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6824", 2020-11-28T21:56:37.703 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:56:37.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.704 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:37.704 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6825", 2020-11-28T21:56:37.704 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:56:37.705 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:37.705 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:37.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.705 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:56:37.706 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:37.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.706 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:37.706 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6822", 2020-11-28T21:56:37.707 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:56:37.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.707 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:37.707 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6823", 2020-11-28T21:56:37.708 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12312 2020-11-28T21:56:37.708 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:37.708 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:37.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.708 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6819/12312", 2020-11-28T21:56:37.709 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6821/12312", 2020-11-28T21:56:37.709 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6825/12312", 2020-11-28T21:56:37.709 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6823/12312", 2020-11-28T21:56:37.709 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:56:37.709 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:56:37.710 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:56:37.710 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:37.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.710 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 3, 2020-11-28T21:56:37.711 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "4094cb87-f04a-45e1-8710-fb4b651949f6", 2020-11-28T21:56:37.711 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:56:37.711 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:56:37.711 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:56:37.711 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:56:37.712 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:56:37.712 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:56:37.712 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:56:37.712 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 145, 2020-11-28T21:56:37.712 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:56:37.713 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:56:37.713 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:56:37.713 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:37.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.713 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:37.714 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6826", 2020-11-28T21:56:37.714 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:56:37.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.714 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:37.715 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6827", 2020-11-28T21:56:37.715 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:56:37.715 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:37.715 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:37.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.716 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:56:37.716 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:37.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.716 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:37.716 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6828", 2020-11-28T21:56:37.717 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:56:37.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.718 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:37.718 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6829", 2020-11-28T21:56:37.718 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:56:37.718 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:37.718 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:37.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.719 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:56:37.719 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:37.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.719 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:37.720 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6832", 2020-11-28T21:56:37.720 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:56:37.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.720 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:37.721 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6833", 2020-11-28T21:56:37.721 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:56:37.721 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:37.721 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:37.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.722 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:56:37.722 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:37.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.722 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:37.722 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6830", 2020-11-28T21:56:37.723 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:56:37.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.723 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:37.723 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6831", 2020-11-28T21:56:37.724 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12323 2020-11-28T21:56:37.724 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:37.724 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:37.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.724 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6827/12323", 2020-11-28T21:56:37.725 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6829/12323", 2020-11-28T21:56:37.725 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6833/12323", 2020-11-28T21:56:37.725 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6831/12323", 2020-11-28T21:56:37.725 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:56:37.725 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:56:37.726 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:56:37.726 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:37.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.726 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 4, 2020-11-28T21:56:37.727 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "b907bf53-c297-4756-847b-3be96680eef5", 2020-11-28T21:56:37.727 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:56:37.727 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:56:37.727 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:56:37.727 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:56:37.728 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:56:37.728 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:56:37.728 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:56:37.728 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 158, 2020-11-28T21:56:37.728 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:56:37.729 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:56:37.729 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:56:37.729 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:37.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.729 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:37.730 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6800", 2020-11-28T21:56:37.730 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:56:37.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.730 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:37.731 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6801", 2020-11-28T21:56:37.731 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:56:37.731 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:37.731 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:37.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.732 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:56:37.732 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:37.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.732 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:37.733 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6802", 2020-11-28T21:56:37.733 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:56:37.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.734 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:37.734 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6803", 2020-11-28T21:56:37.734 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:56:37.735 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:37.735 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:37.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.735 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:56:37.735 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:37.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.736 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:37.736 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6806", 2020-11-28T21:56:37.736 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:56:37.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.737 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:37.737 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6807", 2020-11-28T21:56:37.737 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:56:37.738 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:37.738 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:37.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.738 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:56:37.739 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:37.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.739 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:37.739 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6804", 2020-11-28T21:56:37.739 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:56:37.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.740 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:37.740 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6805", 2020-11-28T21:56:37.740 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12527 2020-11-28T21:56:37.741 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:37.741 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:37.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.741 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6801/12527", 2020-11-28T21:56:37.741 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6803/12527", 2020-11-28T21:56:37.742 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6807/12527", 2020-11-28T21:56:37.742 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6805/12527", 2020-11-28T21:56:37.742 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:56:37.742 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:56:37.742 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:56:37.743 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:37.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.743 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 5, 2020-11-28T21:56:37.743 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "55ba305d-0505-4212-aebb-631e8554b440", 2020-11-28T21:56:37.744 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:56:37.744 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:56:37.744 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:56:37.744 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:56:37.744 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:56:37.744 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:56:37.745 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:56:37.745 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 157, 2020-11-28T21:56:37.745 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:56:37.745 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:56:37.745 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:56:37.746 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:37.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.746 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:37.746 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6808", 2020-11-28T21:56:37.747 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:56:37.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.747 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:37.747 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6809", 2020-11-28T21:56:37.747 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:56:37.748 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:37.748 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:37.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.748 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:56:37.748 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:37.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.749 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:37.749 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6810", 2020-11-28T21:56:37.749 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:56:37.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.750 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:37.750 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6811", 2020-11-28T21:56:37.750 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:56:37.751 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:37.751 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:37.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.751 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:56:37.751 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:37.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.752 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:37.752 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6814", 2020-11-28T21:56:37.752 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:56:37.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.753 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:37.753 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6815", 2020-11-28T21:56:37.753 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:56:37.753 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:37.754 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:37.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.754 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:56:37.754 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:37.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.755 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:37.755 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6812", 2020-11-28T21:56:37.755 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:56:37.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.756 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:37.756 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6813", 2020-11-28T21:56:37.756 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12529 2020-11-28T21:56:37.756 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:37.756 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:37.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.757 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6809/12529", 2020-11-28T21:56:37.757 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6811/12529", 2020-11-28T21:56:37.757 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6815/12529", 2020-11-28T21:56:37.757 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6813/12529", 2020-11-28T21:56:37.758 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:56:37.758 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:56:37.758 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:56:37.758 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:37.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.759 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 6, 2020-11-28T21:56:37.759 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "9ada6051-493b-46b5-ba22-1b401440490f", 2020-11-28T21:56:37.759 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:56:37.759 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:56:37.760 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:56:37.760 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:56:37.760 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:56:37.760 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:56:37.760 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:56:37.761 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 157, 2020-11-28T21:56:37.761 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:56:37.761 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:56:37.761 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:56:37.761 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:37.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.762 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:37.762 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6824", 2020-11-28T21:56:37.762 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:56:37.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.763 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:37.763 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6825", 2020-11-28T21:56:37.763 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:56:37.763 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:37.764 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:37.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.764 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:56:37.764 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:37.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.765 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:37.765 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6826", 2020-11-28T21:56:37.765 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:56:37.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.766 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:37.766 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6827", 2020-11-28T21:56:37.766 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:56:37.766 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:37.767 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:37.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.767 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:56:37.767 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:37.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.768 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:37.768 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6830", 2020-11-28T21:56:37.768 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:56:37.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.769 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:37.769 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6831", 2020-11-28T21:56:37.769 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:56:37.770 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:37.770 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:37.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.770 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:56:37.771 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:37.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.771 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:37.771 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6828", 2020-11-28T21:56:37.771 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:56:37.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.772 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:37.772 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6829", 2020-11-28T21:56:37.773 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12530 2020-11-28T21:56:37.773 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:37.773 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:37.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.774 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6825/12530", 2020-11-28T21:56:37.774 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6827/12530", 2020-11-28T21:56:37.774 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6831/12530", 2020-11-28T21:56:37.774 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6829/12530", 2020-11-28T21:56:37.774 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:56:37.775 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:56:37.775 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:56:37.775 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:37.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.776 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 7, 2020-11-28T21:56:37.776 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "c328b06d-a216-4cba-aad7-9ad081340613", 2020-11-28T21:56:37.776 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2020-11-28T21:56:37.776 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2020-11-28T21:56:37.776 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2020-11-28T21:56:37.777 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2020-11-28T21:56:37.777 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2020-11-28T21:56:37.777 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2020-11-28T21:56:37.777 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 14, 2020-11-28T21:56:37.777 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 157, 2020-11-28T21:56:37.778 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2020-11-28T21:56:37.778 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2020-11-28T21:56:37.778 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2020-11-28T21:56:37.778 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:37.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.779 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:37.779 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6816", 2020-11-28T21:56:37.779 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:56:37.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.780 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:37.780 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6817", 2020-11-28T21:56:37.780 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:56:37.780 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:37.780 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:37.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.781 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2020-11-28T21:56:37.781 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:37.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.781 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:37.782 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6818", 2020-11-28T21:56:37.782 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:56:37.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.782 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:37.783 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6819", 2020-11-28T21:56:37.783 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:56:37.783 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:37.783 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:37.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.784 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2020-11-28T21:56:37.784 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:37.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.784 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:37.784 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6822", 2020-11-28T21:56:37.785 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:56:37.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.785 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:37.785 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6823", 2020-11-28T21:56:37.786 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:56:37.786 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:37.786 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:37.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.786 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2020-11-28T21:56:37.787 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2020-11-28T21:56:37.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.787 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2020-11-28T21:56:37.787 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6820", 2020-11-28T21:56:37.787 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:56:37.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.788 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2020-11-28T21:56:37.788 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.118:6821", 2020-11-28T21:56:37.788 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 12528 2020-11-28T21:56:37.789 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:37.789 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:37.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.789 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.118:6817/12528", 2020-11-28T21:56:37.789 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.118:6819/12528", 2020-11-28T21:56:37.790 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.118:6823/12528", 2020-11-28T21:56:37.790 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.118:6821/12528", 2020-11-28T21:56:37.790 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2020-11-28T21:56:37.790 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2020-11-28T21:56:37.790 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2020-11-28T21:56:37.791 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2020-11-28T21:56:37.791 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:37.791 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:56:37.791 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_xinfo": [ 2020-11-28T21:56:37.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.792 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 0, 2020-11-28T21:56:37.792 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:56:37.793 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:56:37.793 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:56:37.793 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:56:37.793 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:56:37.793 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.038052+0000", 2020-11-28T21:56:37.794 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:56:37.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.794 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 1, 2020-11-28T21:56:37.794 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:56:37.795 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:56:37.795 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:56:37.795 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:56:37.795 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:56:37.795 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:17.947136+0000", 2020-11-28T21:56:37.796 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:56:37.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.796 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 2, 2020-11-28T21:56:37.796 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:56:37.797 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:56:37.797 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:56:37.797 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:56:37.797 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:56:37.797 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.001186+0000", 2020-11-28T21:56:37.798 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:56:37.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.799 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 3, 2020-11-28T21:56:37.799 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:56:37.799 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:56:37.799 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:56:37.800 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:56:37.800 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:56:37.800 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.199284+0000", 2020-11-28T21:56:37.800 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:56:37.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.801 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 4, 2020-11-28T21:56:37.801 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:56:37.801 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:56:37.801 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:56:37.802 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:56:37.802 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:56:37.802 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.038823+0000", 2020-11-28T21:56:37.802 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:56:37.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.803 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 5, 2020-11-28T21:56:37.803 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:56:37.803 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:56:37.803 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:56:37.804 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:56:37.804 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:56:37.804 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.009322+0000", 2020-11-28T21:56:37.804 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:56:37.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.805 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 6, 2020-11-28T21:56:37.805 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:56:37.805 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:56:37.805 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:56:37.806 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:56:37.806 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:56:37.806 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.238927+0000", 2020-11-28T21:56:37.806 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:56:37.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.807 INFO:teuthology.orchestra.run.smithi162.stdout: { 2020-11-28T21:56:37.807 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 7, 2020-11-28T21:56:37.807 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2020-11-28T21:56:37.807 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2020-11-28T21:56:37.807 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2020-11-28T21:56:37.808 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138292840890367, 2020-11-28T21:56:37.808 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2020-11-28T21:56:37.808 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2020-11-28T21:46:18.029839+0000", 2020-11-28T21:56:37.808 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2020-11-28T21:56:37.808 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:37.809 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2020-11-28T21:56:37.809 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_upmap": [], 2020-11-28T21:56:37.809 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_upmap_items": [], 2020-11-28T21:56:37.809 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_temp": [], 2020-11-28T21:56:37.809 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_temp": [], 2020-11-28T21:56:37.810 INFO:teuthology.orchestra.run.smithi162.stdout: "blacklist": {}, 2020-11-28T21:56:37.810 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profiles": { 2020-11-28T21:56:37.810 INFO:teuthology.orchestra.run.smithi162.stdout: "default": { 2020-11-28T21:56:37.810 INFO:teuthology.orchestra.run.smithi162.stdout: "crush-failure-domain": "osd", 2020-11-28T21:56:37.810 INFO:teuthology.orchestra.run.smithi162.stdout: "k": "2", 2020-11-28T21:56:37.811 INFO:teuthology.orchestra.run.smithi162.stdout: "m": "1", 2020-11-28T21:56:37.811 INFO:teuthology.orchestra.run.smithi162.stdout: "plugin": "jerasure", 2020-11-28T21:56:37.811 INFO:teuthology.orchestra.run.smithi162.stdout: "ruleset-failure-domain": "osd", 2020-11-28T21:56:37.811 INFO:teuthology.orchestra.run.smithi162.stdout: "technique": "reed_sol_van" 2020-11-28T21:56:37.811 INFO:teuthology.orchestra.run.smithi162.stdout: } 2020-11-28T21:56:37.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2020-11-28T21:56:37.812 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps_queue": [], 2020-11-28T21:56:37.812 INFO:teuthology.orchestra.run.smithi162.stdout: "new_removed_snaps": [], 2020-11-28T21:56:37.812 INFO:teuthology.orchestra.run.smithi162.stdout: "new_purged_snaps": [], 2020-11-28T21:56:37.812 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_node_flags": {}, 2020-11-28T21:56:37.813 INFO:teuthology.orchestra.run.smithi162.stdout: "device_class_flags": {} 2020-11-28T21:56:37.813 INFO:teuthology.orchestra.run.smithi162.stdout:} 2020-11-28T21:56:37.814 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.578+0000 7f5f30871700 1 -- 172.21.15.162:0/1484993696 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5f080553b0 msgr2=0x7f5f08057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:37.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.578+0000 7f5f30871700 1 --2- 172.21.15.162:0/1484993696 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5f080553b0 0x7f5f08057870 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7f5f2000d000 tx=0x7f5f20007ee0).stop 2020-11-28T21:56:37.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.578+0000 7f5f30871700 1 -- 172.21.15.162:0/1484993696 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f2c09d890 msgr2=0x7f5f2c156060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:37.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.578+0000 7f5f30871700 1 --2- 172.21.15.162:0/1484993696 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f2c09d890 0x7f5f2c156060 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 rx=0x7f5f1c00c000 tx=0x7f5f1c006dc0).stop 2020-11-28T21:56:37.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.578+0000 7f5f30871700 1 -- 172.21.15.162:0/1484993696 shutdown_connections 2020-11-28T21:56:37.815 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.578+0000 7f5f30871700 1 --2- 172.21.15.162:0/1484993696 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f5f080553b0 0x7f5f08057870 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:37.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.578+0000 7f5f30871700 1 --2- 172.21.15.162:0/1484993696 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5f2c0608f0 0x7f5f2c0ae3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:37.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.578+0000 7f5f30871700 1 --2- 172.21.15.162:0/1484993696 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f2c0b5e30 0x7f5f2c0b24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:37.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.578+0000 7f5f30871700 1 --2- 172.21.15.162:0/1484993696 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f2c09d890 0x7f5f2c156060 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:37.816 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.578+0000 7f5f30871700 1 -- 172.21.15.162:0/1484993696 >> 172.21.15.162:0/1484993696 conn(0x7f5f2c05b970 msgr2=0x7f5f2c0b6fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:37.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.578+0000 7f5f30871700 1 -- 172.21.15.162:0/1484993696 shutdown_connections 2020-11-28T21:56:37.817 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.578+0000 7f5f30871700 1 -- 172.21.15.162:0/1484993696 wait complete. 2020-11-28T21:56:37.818 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:56:37.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.930+0000 7fe523fff700 1 Processor -- start 2020-11-28T21:56:37.938 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.930+0000 7fe523fff700 1 -- start start 2020-11-28T21:56:37.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.934+0000 7fe523fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe524064c70 0x7fe524066290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:37.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.934+0000 7fe523fff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe52413f6c0 0x7fe524066780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:37.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.934+0000 7fe523fff700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe524065050 con 0x7fe524066c70 2020-11-28T21:56:37.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.934+0000 7fe523fff700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe52413fc40 con 0x7fe52413f6c0 2020-11-28T21:56:37.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.934+0000 7fe523fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe52413fee0 con 0x7fe524064c70 2020-11-28T21:56:37.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.934+0000 7fe5237fe700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fe524066c70 0x7fe524142c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:53448/0 (socket says 172.21.15.162:53448) 2020-11-28T21:56:37.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.934+0000 7fe5237fe700 1 -- 172.21.15.162:0/3362477717 learned_addr learned my addr 172.21.15.162:0/3362477717 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:56:37.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.934+0000 7fe522ffd700 1 --2- 172.21.15.162:0/3362477717 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe524064c70 0x7fe524066290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:37.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.934+0000 7fe51a7fc700 1 --2- 172.21.15.162:0/3362477717 >> v2:172.21.15.118:3300/0 conn(0x7fe52413f6c0 0x7fe524066780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:37.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.934+0000 7fe5227fc700 1 -- 172.21.15.162:0/3362477717 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 470150771 0 0) 0x7fe524065050 con 0x7fe524066c70 2020-11-28T21:56:37.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.934+0000 7fe5227fc700 1 -- 172.21.15.162:0/3362477717 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe504003230 con 0x7fe524066c70 2020-11-28T21:56:37.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.934+0000 7fe51a7fc700 1 -- 172.21.15.162:0/3362477717 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe524064c70 msgr2=0x7fe524066290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:37.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.934+0000 7fe51a7fc700 1 --2- 172.21.15.162:0/3362477717 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe524064c70 0x7fe524066290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:37.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.934+0000 7fe51a7fc700 1 -- 172.21.15.162:0/3362477717 >> v1:172.21.15.162:6789/0 conn(0x7fe524066c70 legacy=0x7fe524142c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:37.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.934+0000 7fe51a7fc700 1 -- 172.21.15.162:0/3362477717 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe524140180 con 0x7fe52413f6c0 2020-11-28T21:56:37.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.934+0000 7fe522ffd700 1 --2- 172.21.15.162:0/3362477717 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe524064c70 0x7fe524066290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:56:37.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.934+0000 7fe51a7fc700 1 --2- 172.21.15.162:0/3362477717 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe52413f6c0 0x7fe524066780 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7fe50c002f30 tx=0x7fe50c006dc0).ready entity=mon.0 client_cookie=346f8a24bbef96c8 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:37.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.934+0000 7fe5227fc700 1 -- 172.21.15.162:0/3362477717 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe50c0096a0 con 0x7fe52413f6c0 2020-11-28T21:56:37.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.934+0000 7fe5227fc700 1 -- 172.21.15.162:0/3362477717 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe50c009800 con 0x7fe52413f6c0 2020-11-28T21:56:37.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.934+0000 7fe5227fc700 1 -- 172.21.15.162:0/3362477717 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe50c009b60 con 0x7fe52413f6c0 2020-11-28T21:56:37.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.934+0000 7fe523fff700 1 -- 172.21.15.162:0/3362477717 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe52413f6c0 msgr2=0x7fe524066780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:37.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.934+0000 7fe523fff700 1 --2- 172.21.15.162:0/3362477717 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe52413f6c0 0x7fe524066780 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7fe50c002f30 tx=0x7fe50c006dc0).stop 2020-11-28T21:56:37.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.934+0000 7fe523fff700 1 -- 172.21.15.162:0/3362477717 shutdown_connections 2020-11-28T21:56:37.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.934+0000 7fe523fff700 1 --2- 172.21.15.162:0/3362477717 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe524064c70 0x7fe524066290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:37.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.934+0000 7fe523fff700 1 --2- 172.21.15.162:0/3362477717 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe52413f6c0 0x7fe524066780 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:37.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.934+0000 7fe523fff700 1 -- 172.21.15.162:0/3362477717 >> 172.21.15.162:0/3362477717 conn(0x7fe5240b5da0 msgr2=0x7fe5240b6180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:37.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.934+0000 7fe523fff700 1 -- 172.21.15.162:0/3362477717 shutdown_connections 2020-11-28T21:56:37.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.934+0000 7fe523fff700 1 -- 172.21.15.162:0/3362477717 wait complete. 2020-11-28T21:56:37.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.938+0000 7fe523fff700 1 Processor -- start 2020-11-28T21:56:37.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.938+0000 7fe523fff700 1 -- start start 2020-11-28T21:56:37.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.938+0000 7fe523fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe524064c70 0x7fe52413e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:37.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.938+0000 7fe523fff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe524066c70 0x7fe5240612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:37.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.938+0000 7fe523fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe52413f6c0 0x7fe524133930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:37.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.938+0000 7fe523fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52413de00 con 0x7fe52413f6c0 2020-11-28T21:56:37.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.938+0000 7fe523fff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe524064290 con 0x7fe524064c70 2020-11-28T21:56:37.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.938+0000 7fe523fff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5240b50b0 con 0x7fe524066c70 2020-11-28T21:56:37.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.938+0000 7fe522ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe524064c70 0x7fe52413e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:37.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.938+0000 7fe522ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe524064c70 0x7fe52413e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46606/0 (socket says 172.21.15.162:46606) 2020-11-28T21:56:37.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.938+0000 7fe522ffd700 1 -- 172.21.15.162:0/2992142500 learned_addr learned my addr 172.21.15.162:0/2992142500 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:37.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.938+0000 7fe51a7fc700 1 --2- 172.21.15.162:0/2992142500 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe524066c70 0x7fe5240612e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:37.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.938+0000 7fe5237fe700 1 --2- 172.21.15.162:0/2992142500 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe52413f6c0 0x7fe524133930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:37.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.938+0000 7fe522ffd700 1 -- 172.21.15.162:0/2992142500 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe524066c70 msgr2=0x7fe5240612e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:37.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.938+0000 7fe522ffd700 1 --2- 172.21.15.162:0/2992142500 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe524066c70 0x7fe5240612e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:37.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.938+0000 7fe522ffd700 1 -- 172.21.15.162:0/2992142500 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe52413f6c0 msgr2=0x7fe524133930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:37.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.938+0000 7fe522ffd700 1 --2- 172.21.15.162:0/2992142500 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe52413f6c0 0x7fe524133930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:37.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.938+0000 7fe522ffd700 1 -- 172.21.15.162:0/2992142500 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe52413e030 con 0x7fe524064c70 2020-11-28T21:56:37.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.938+0000 7fe522ffd700 1 --2- 172.21.15.162:0/2992142500 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe524064c70 0x7fe52413e4d0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 rx=0x7fe52412f780 tx=0x7fe514006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:37.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.938+0000 7fe520ff9700 1 -- 172.21.15.162:0/2992142500 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe51400cda0 con 0x7fe524064c70 2020-11-28T21:56:37.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.938+0000 7fe523fff700 1 -- 172.21.15.162:0/2992142500 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5240617d0 con 0x7fe524064c70 2020-11-28T21:56:37.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.938+0000 7fe520ff9700 1 -- 172.21.15.162:0/2992142500 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5140123f0 con 0x7fe524064c70 2020-11-28T21:56:37.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.938+0000 7fe520ff9700 1 -- 172.21.15.162:0/2992142500 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe514012750 con 0x7fe524064c70 2020-11-28T21:56:37.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.938+0000 7fe523fff700 1 -- 172.21.15.162:0/2992142500 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe524135660 con 0x7fe524064c70 2020-11-28T21:56:37.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.942+0000 7fe523fff700 1 -- 172.21.15.162:0/2992142500 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe524057050 con 0x7fe524064c70 2020-11-28T21:56:37.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.942+0000 7fe520ff9700 1 -- 172.21.15.162:0/2992142500 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe5140128b0 con 0x7fe524064c70 2020-11-28T21:56:37.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.942+0000 7fe520ff9700 1 --2- 172.21.15.162:0/2992142500 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe5080553b0 0x7fe508057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:37.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.942+0000 7fe520ff9700 1 -- 172.21.15.162:0/2992142500 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe51406e760 con 0x7fe524064c70 2020-11-28T21:56:37.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.942+0000 7fe51a7fc700 1 --2- 172.21.15.162:0/2992142500 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe5080553b0 0x7fe508057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:37.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.942+0000 7fe51a7fc700 1 --2- 172.21.15.162:0/2992142500 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe5080553b0 0x7fe508057870 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7fe50c019000 tx=0x7fe50c015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:37.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:37.946+0000 7fe520ff9700 1 -- 172.21.15.162:0/2992142500 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe5140449c0 con 0x7fe524064c70 2020-11-28T21:56:38.137 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.130+0000 7fe523fff700 1 -- 172.21.15.162:0/2992142500 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe524063480 con 0x7fe524064c70 2020-11-28T21:56:38.138 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.134+0000 7fe520ff9700 1 -- 172.21.15.162:0/2992142500 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+2501 (secure 0 0 0) 0x7fe514047fe0 con 0x7fe524064c70 2020-11-28T21:56:38.138 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:56:38.138 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":80,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8199,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/2385081970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":2385081970},{"type":"v1","addr":"172.21.15.118:6833","nonce":2385081970}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":77},{"gid":8554,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6836/169306274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":169306274},{"type":"v1","addr":"172.21.15.162:6836","nonce":169306274}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":78}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:56:25.831016+0000","modified":"2020-11-28T21:56:26.833832+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8557},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8557":{"gid":8557,"name":"c","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/3753662046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":3753662046},{"type":"v1","addr":"172.21.15.162:6837","nonce":3753662046}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2020-11-28T21:56:38.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.134+0000 7fe523fff700 1 -- 172.21.15.162:0/2992142500 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe5080553b0 msgr2=0x7fe508057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:38.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.134+0000 7fe523fff700 1 --2- 172.21.15.162:0/2992142500 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe5080553b0 0x7fe508057870 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7fe50c019000 tx=0x7fe50c015000).stop 2020-11-28T21:56:38.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.134+0000 7fe523fff700 1 -- 172.21.15.162:0/2992142500 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe524064c70 msgr2=0x7fe52413e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:38.140 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.134+0000 7fe523fff700 1 --2- 172.21.15.162:0/2992142500 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe524064c70 0x7fe52413e4d0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 rx=0x7fe52412f780 tx=0x7fe514006dc0).stop 2020-11-28T21:56:38.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.134+0000 7fe523fff700 1 -- 172.21.15.162:0/2992142500 shutdown_connections 2020-11-28T21:56:38.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.134+0000 7fe523fff700 1 --2- 172.21.15.162:0/2992142500 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe5080553b0 0x7fe508057870 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:38.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.134+0000 7fe523fff700 1 --2- 172.21.15.162:0/2992142500 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe52413f6c0 0x7fe524133930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:38.141 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.134+0000 7fe523fff700 1 --2- 172.21.15.162:0/2992142500 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe524064c70 0x7fe52413e4d0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:38.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.134+0000 7fe523fff700 1 --2- 172.21.15.162:0/2992142500 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe524066c70 0x7fe5240612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:38.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.134+0000 7fe523fff700 1 -- 172.21.15.162:0/2992142500 >> 172.21.15.162:0/2992142500 conn(0x7fe5240b5da0 msgr2=0x7fe52406a280 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:38.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.134+0000 7fe523fff700 1 -- 172.21.15.162:0/2992142500 shutdown_connections 2020-11-28T21:56:38.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.134+0000 7fe523fff700 1 -- 172.21.15.162:0/2992142500 wait complete. 2020-11-28T21:56:38.143 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 80 2020-11-28T21:56:38.151 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2020-11-28T21:56:38.263 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.258+0000 7f787e20c700 1 Processor -- start 2020-11-28T21:56:38.264 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.258+0000 7f787e20c700 1 -- start start 2020-11-28T21:56:38.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.258+0000 7f787e20c700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7878139080 0x7f78780adb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:38.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.258+0000 7f787e20c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f78780b2420 0x7f78780ae030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:38.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.258+0000 7f787e20c700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78780609b0 con 0x7f78780605d0 2020-11-28T21:56:38.265 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.258+0000 7f787e20c700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78780acdd0 con 0x7f78780b2420 2020-11-28T21:56:38.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.258+0000 7f787e20c700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f787805d080 con 0x7f7878139080 2020-11-28T21:56:38.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.258+0000 7f7877fff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f78780b2420 0x7f78780ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:38.266 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.258+0000 7f78777fe700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f78780605d0 0x7f7878138970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.162:43606/0 (socket says 172.21.15.162:43606) 2020-11-28T21:56:38.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.258+0000 7f78777fe700 1 -- 172.21.15.162:0/66578766 learned_addr learned my addr 172.21.15.162:0/66578766 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:56:38.267 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.258+0000 7f7876ffd700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7878139080 0x7f78780adb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:38.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.262+0000 7f7877fff700 1 -- 172.21.15.162:0/66578766 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7878139080 msgr2=0x7f78780adb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:38.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.262+0000 7f7877fff700 1 --2- 172.21.15.162:0/66578766 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7878139080 0x7f78780adb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:38.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.262+0000 7f7877fff700 1 -- 172.21.15.162:0/66578766 >> v1:172.21.15.118:6789/0 conn(0x7f78780605d0 legacy=0x7f7878138970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:38.269 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.262+0000 7f7877fff700 1 -- 172.21.15.162:0/66578766 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78780a8940 con 0x7f78780b2420 2020-11-28T21:56:38.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.262+0000 7f7877fff700 1 --2- 172.21.15.162:0/66578766 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f78780b2420 0x7f78780ae030 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7f78700032b0 tx=0x7f78700091a0).ready entity=mon.0 client_cookie=44bbce68f8eab274 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:38.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.262+0000 7f78767fc700 1 -- 172.21.15.162:0/66578766 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f787000b9a0 con 0x7f78780b2420 2020-11-28T21:56:38.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.262+0000 7f78767fc700 1 -- 172.21.15.162:0/66578766 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f787000bb00 con 0x7f78780b2420 2020-11-28T21:56:38.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.262+0000 7f78767fc700 1 -- 172.21.15.162:0/66578766 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f787000bc70 con 0x7f78780b2420 2020-11-28T21:56:38.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.262+0000 7f787e20c700 1 -- 172.21.15.162:0/66578766 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f78780b2420 msgr2=0x7f78780ae030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:38.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.262+0000 7f787e20c700 1 --2- 172.21.15.162:0/66578766 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f78780b2420 0x7f78780ae030 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7f78700032b0 tx=0x7f78700091a0).stop 2020-11-28T21:56:38.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.262+0000 7f787e20c700 1 -- 172.21.15.162:0/66578766 shutdown_connections 2020-11-28T21:56:38.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.262+0000 7f787e20c700 1 --2- 172.21.15.162:0/66578766 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f78780b2420 0x7f78780ae030 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:38.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.262+0000 7f787e20c700 1 --2- 172.21.15.162:0/66578766 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7878139080 0x7f78780adb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:38.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.262+0000 7f787e20c700 1 -- 172.21.15.162:0/66578766 >> 172.21.15.162:0/66578766 conn(0x7f787805b970 msgr2=0x7f787805bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:38.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.262+0000 7f787e20c700 1 -- 172.21.15.162:0/66578766 shutdown_connections 2020-11-28T21:56:38.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.262+0000 7f787e20c700 1 -- 172.21.15.162:0/66578766 wait complete. 2020-11-28T21:56:38.273 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.262+0000 7f787e20c700 1 Processor -- start 2020-11-28T21:56:38.273 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.262+0000 7f787e20c700 1 -- start start 2020-11-28T21:56:38.279 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.266+0000 7f787e20c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f78780605d0 0x7f78780a8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:38.280 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.266+0000 7f787e20c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78780b2420 0x7f787812a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:38.280 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.266+0000 7f787e20c700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7878139080 0x7f78780b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:38.281 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.266+0000 7f787e20c700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78780a9280 con 0x7f78780605d0 2020-11-28T21:56:38.281 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.266+0000 7f787e20c700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78780a9520 con 0x7f78780b2420 2020-11-28T21:56:38.281 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.266+0000 7f787e20c700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78780b05d0 con 0x7f7878139080 2020-11-28T21:56:38.282 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.266+0000 7f7876ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78780b2420 0x7f787812a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:38.282 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.266+0000 7f7876ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78780b2420 0x7f787812a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46622/0 (socket says 172.21.15.162:46622) 2020-11-28T21:56:38.282 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.266+0000 7f7876ffd700 1 -- 172.21.15.162:0/3808325904 learned_addr learned my addr 172.21.15.162:0/3808325904 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:38.283 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.266+0000 7f7877fff700 1 --2- 172.21.15.162:0/3808325904 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7878139080 0x7f78780b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:38.283 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.266+0000 7f78777fe700 1 --2- 172.21.15.162:0/3808325904 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f78780605d0 0x7f78780a8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:38.284 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.266+0000 7f7876ffd700 1 -- 172.21.15.162:0/3808325904 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7878139080 msgr2=0x7f78780b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:38.284 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.266+0000 7f7876ffd700 1 --2- 172.21.15.162:0/3808325904 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7878139080 0x7f78780b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:38.284 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.266+0000 7f7876ffd700 1 -- 172.21.15.162:0/3808325904 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f78780605d0 msgr2=0x7f78780a8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:38.285 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.266+0000 7f7876ffd700 1 --2- 172.21.15.162:0/3808325904 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f78780605d0 0x7f78780a8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:38.285 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.266+0000 7f7876ffd700 1 -- 172.21.15.162:0/3808325904 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78780ab480 con 0x7f78780b2420 2020-11-28T21:56:38.285 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.266+0000 7f7876ffd700 1 --2- 172.21.15.162:0/3808325904 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78780b2420 0x7f787812a720 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 rx=0x7f786c00c000 tx=0x7f786c006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:38.286 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.266+0000 7f7874ff9700 1 -- 172.21.15.162:0/3808325904 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f786c00ca10 con 0x7f78780b2420 2020-11-28T21:56:38.286 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.266+0000 7f787e20c700 1 -- 172.21.15.162:0/3808325904 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78780ab860 con 0x7f78780b2420 2020-11-28T21:56:38.286 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.266+0000 7f7874ff9700 1 -- 172.21.15.162:0/3808325904 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786c00cb70 con 0x7f78780b2420 2020-11-28T21:56:38.287 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.266+0000 7f7874ff9700 1 -- 172.21.15.162:0/3808325904 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f786c00be50 con 0x7f78780b2420 2020-11-28T21:56:38.287 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.266+0000 7f787e20c700 1 -- 172.21.15.162:0/3808325904 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78780b1180 con 0x7f78780b2420 2020-11-28T21:56:38.287 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.270+0000 7f787e20c700 1 -- 172.21.15.162:0/3808325904 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78780af320 con 0x7f78780b2420 2020-11-28T21:56:38.288 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.270+0000 7f7874ff9700 1 -- 172.21.15.162:0/3808325904 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f786c01b410 con 0x7f78780b2420 2020-11-28T21:56:38.288 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.270+0000 7f7874ff9700 1 --2- 172.21.15.162:0/3808325904 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7858055350 0x7f7858057810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:38.288 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.270+0000 7f7874ff9700 1 -- 172.21.15.162:0/3808325904 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f786c06e030 con 0x7f78780b2420 2020-11-28T21:56:38.289 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.270+0000 7f78777fe700 1 --2- 172.21.15.162:0/3808325904 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7858055350 0x7f7858057810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:38.289 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.270+0000 7f78777fe700 1 --2- 172.21.15.162:0/3808325904 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7858055350 0x7f7858057810 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7f786800d000 tx=0x7f7868009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:38.291 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.278+0000 7f7874ff9700 1 -- 172.21.15.162:0/3808325904 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f78780af320 con 0x7f78780b2420 2020-11-28T21:56:38.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.462+0000 7f787e20c700 1 -- 172.21.15.162:0/3808325904 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f78780ab670 con 0x7f78780b2420 2020-11-28T21:56:38.886 INFO:tasks.ceph.mon.b.smithi118.stderr:2020-11-28T21:56:38.883+0000 7ff4e1520700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-11-28T21:56:38.901 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.893+0000 7f7874ff9700 1 -- 172.21.15.162:0/3808325904 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v81) v1 ==== 161+0+0 (secure 0 0 0) 0x7f78780ab670 con 0x7f78780b2420 2020-11-28T21:56:38.906 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.901+0000 7f787e20c700 1 -- 172.21.15.162:0/3808325904 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7858055350 msgr2=0x7f7858057810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:38.907 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.901+0000 7f787e20c700 1 --2- 172.21.15.162:0/3808325904 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7858055350 0x7f7858057810 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7f786800d000 tx=0x7f7868009040).stop 2020-11-28T21:56:38.907 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.901+0000 7f787e20c700 1 -- 172.21.15.162:0/3808325904 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78780b2420 msgr2=0x7f787812a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:38.907 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.901+0000 7f787e20c700 1 --2- 172.21.15.162:0/3808325904 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78780b2420 0x7f787812a720 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 rx=0x7f786c00c000 tx=0x7f786c006dc0).stop 2020-11-28T21:56:38.907 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.901+0000 7f787e20c700 1 -- 172.21.15.162:0/3808325904 shutdown_connections 2020-11-28T21:56:38.907 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.901+0000 7f787e20c700 1 --2- 172.21.15.162:0/3808325904 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f7858055350 0x7f7858057810 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:38.908 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.901+0000 7f787e20c700 1 --2- 172.21.15.162:0/3808325904 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7878139080 0x7f78780b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:38.908 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.901+0000 7f787e20c700 1 --2- 172.21.15.162:0/3808325904 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f78780605d0 0x7f78780a8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:38.908 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.901+0000 7f787e20c700 1 --2- 172.21.15.162:0/3808325904 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78780b2420 0x7f787812a720 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:38.908 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.901+0000 7f787e20c700 1 -- 172.21.15.162:0/3808325904 >> 172.21.15.162:0/3808325904 conn(0x7f787805b970 msgr2=0x7f7878138740 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:38.908 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.901+0000 7f787e20c700 1 -- 172.21.15.162:0/3808325904 shutdown_connections 2020-11-28T21:56:38.909 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:38.901+0000 7f787e20c700 1 -- 172.21.15.162:0/3808325904 wait complete. 2020-11-28T21:56:38.909 INFO:teuthology.orchestra.run.smithi162.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-11-28T21:56:38.927 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-28T21:56:38.936 INFO:tasks.ceph.mds.c.smithi162.stdout:starting mds.c at 2020-11-28T21:56:39.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.033+0000 7f1c5a1cd700 1 Processor -- start 2020-11-28T21:56:39.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.033+0000 7f1c5a1cd700 1 -- start start 2020-11-28T21:56:39.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.033+0000 7f1c5a1cd700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1c540b1860 0x7f1c540b2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:39.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.033+0000 7f1c5a1cd700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c5413d3c0 con 0x7f1c540b2830 2020-11-28T21:56:39.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.033+0000 7f1c5a1cd700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c5413f8d0 con 0x7f1c540ad2c0 2020-11-28T21:56:39.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.033+0000 7f1c5a1cd700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c5413f6e0 con 0x7f1c540b1860 2020-11-28T21:56:39.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.033+0000 7f1c589ca700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f1c540ad2c0 0x7f1c540b1150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:53484/0 (socket says 172.21.15.162:53484) 2020-11-28T21:56:39.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.033+0000 7f1c589ca700 1 -- 172.21.15.162:0/4049679428 learned_addr learned my addr 172.21.15.162:0/4049679428 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:56:39.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.037+0000 7f1c599cc700 1 --2- 172.21.15.162:0/4049679428 >> v2:172.21.15.118:3300/0 conn(0x7f1c540b1860 0x7f1c540b2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:39.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.037+0000 7f1c53fff700 1 -- 172.21.15.162:0/4049679428 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2515899723 0 0) 0x7f1c5413f8d0 con 0x7f1c540ad2c0 2020-11-28T21:56:39.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.037+0000 7f1c53fff700 1 -- 172.21.15.162:0/4049679428 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c40003230 con 0x7f1c540ad2c0 2020-11-28T21:56:39.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.037+0000 7f1c599cc700 1 -- 172.21.15.162:0/4049679428 >> v1:172.21.15.162:6789/0 conn(0x7f1c540ad2c0 legacy=0x7f1c540b1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:39.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.037+0000 7f1c599cc700 1 -- 172.21.15.162:0/4049679428 >> v1:172.21.15.118:6789/0 conn(0x7f1c540b2830 legacy=0x7f1c540b1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:39.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.037+0000 7f1c599cc700 1 -- 172.21.15.162:0/4049679428 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c5413fb70 con 0x7f1c540b1860 2020-11-28T21:56:39.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.037+0000 7f1c53fff700 1 -- 172.21.15.162:0/4049679428 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 425566424 0 0) 0x7f1c5413d3c0 con 0x7f1c540b2830 2020-11-28T21:56:39.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.037+0000 7f1c599cc700 1 --2- 172.21.15.162:0/4049679428 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1c540b1860 0x7f1c540b2c10 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7f1c4c002f30 tx=0x7f1c4c006dc0).ready entity=mon.0 client_cookie=e2c5dda8cda306f4 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:39.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.037+0000 7f1c53fff700 1 -- 172.21.15.162:0/4049679428 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c4c009800 con 0x7f1c540b1860 2020-11-28T21:56:39.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.037+0000 7f1c53fff700 1 -- 172.21.15.162:0/4049679428 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c4c009960 con 0x7f1c540b1860 2020-11-28T21:56:39.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.037+0000 7f1c53fff700 1 -- 172.21.15.162:0/4049679428 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c4c009cc0 con 0x7f1c540b1860 2020-11-28T21:56:39.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.037+0000 7f1c5a1cd700 1 -- 172.21.15.162:0/4049679428 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1c540b1860 msgr2=0x7f1c540b2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:39.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.037+0000 7f1c5a1cd700 1 --2- 172.21.15.162:0/4049679428 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1c540b1860 0x7f1c540b2c10 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7f1c4c002f30 tx=0x7f1c4c006dc0).stop 2020-11-28T21:56:39.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.037+0000 7f1c5a1cd700 1 -- 172.21.15.162:0/4049679428 shutdown_connections 2020-11-28T21:56:39.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.037+0000 7f1c5a1cd700 1 --2- 172.21.15.162:0/4049679428 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1c540b1860 0x7f1c540b2c10 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:39.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.037+0000 7f1c5a1cd700 1 -- 172.21.15.162:0/4049679428 >> 172.21.15.162:0/4049679428 conn(0x7f1c54061550 msgr2=0x7f1c54061960 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:39.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.037+0000 7f1c5a1cd700 1 -- 172.21.15.162:0/4049679428 shutdown_connections 2020-11-28T21:56:39.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.037+0000 7f1c5a1cd700 1 -- 172.21.15.162:0/4049679428 wait complete. 2020-11-28T21:56:39.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.041+0000 7f1c5a1cd700 1 Processor -- start 2020-11-28T21:56:39.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.041+0000 7f1c5a1cd700 1 -- start start 2020-11-28T21:56:39.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.041+0000 7f1c5a1cd700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1c540ad2c0 0x7f1c5413edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:39.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.041+0000 7f1c5a1cd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c540b1860 0x7f1c5414a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:39.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.041+0000 7f1c5a1cd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1c540b2830 0x7f1c5414a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:39.050 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.041+0000 7f1c5a1cd700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c540670e0 con 0x7f1c540b2830 2020-11-28T21:56:39.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.041+0000 7f1c5a1cd700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c54065eb0 con 0x7f1c540b1860 2020-11-28T21:56:39.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.041+0000 7f1c5a1cd700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c54066120 con 0x7f1c540ad2c0 2020-11-28T21:56:39.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.041+0000 7f1c589ca700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c540b1860 0x7f1c5414a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:39.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.041+0000 7f1c589ca700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c540b1860 0x7f1c5414a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46646/0 (socket says 172.21.15.162:46646) 2020-11-28T21:56:39.051 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.041+0000 7f1c589ca700 1 -- 172.21.15.162:0/1388258063 learned_addr learned my addr 172.21.15.162:0/1388258063 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:39.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.041+0000 7f1c591cb700 1 --2- 172.21.15.162:0/1388258063 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1c540ad2c0 0x7f1c5413edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:39.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.041+0000 7f1c599cc700 1 --2- 172.21.15.162:0/1388258063 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1c540b2830 0x7f1c5414a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:39.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.041+0000 7f1c589ca700 1 -- 172.21.15.162:0/1388258063 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1c540ad2c0 msgr2=0x7f1c5413edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:39.052 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.041+0000 7f1c589ca700 1 --2- 172.21.15.162:0/1388258063 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1c540ad2c0 0x7f1c5413edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:39.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.041+0000 7f1c589ca700 1 -- 172.21.15.162:0/1388258063 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1c540b2830 msgr2=0x7f1c5414a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:39.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.041+0000 7f1c589ca700 1 --2- 172.21.15.162:0/1388258063 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1c540b2830 0x7f1c5414a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:39.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.041+0000 7f1c589ca700 1 -- 172.21.15.162:0/1388258063 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c54142e90 con 0x7f1c540b1860 2020-11-28T21:56:39.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.041+0000 7f1c599cc700 1 --2- 172.21.15.162:0/1388258063 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1c540b2830 0x7f1c5414a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:56:39.053 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.041+0000 7f1c591cb700 1 --2- 172.21.15.162:0/1388258063 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1c540ad2c0 0x7f1c5413edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T21:56:39.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.041+0000 7f1c589ca700 1 --2- 172.21.15.162:0/1388258063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c540b1860 0x7f1c5414a110 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 rx=0x7f1c54066b60 tx=0x7f1c44007df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:39.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.041+0000 7f1c527fc700 1 -- 172.21.15.162:0/1388258063 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c44005c40 con 0x7f1c540b1860 2020-11-28T21:56:39.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.041+0000 7f1c527fc700 1 -- 172.21.15.162:0/1388258063 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c44005da0 con 0x7f1c540b1860 2020-11-28T21:56:39.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.041+0000 7f1c5a1cd700 1 -- 172.21.15.162:0/1388258063 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c54148c70 con 0x7f1c540b1860 2020-11-28T21:56:39.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.041+0000 7f1c527fc700 1 -- 172.21.15.162:0/1388258063 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c44007300 con 0x7f1c540b1860 2020-11-28T21:56:39.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.041+0000 7f1c5a1cd700 1 -- 172.21.15.162:0/1388258063 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c54066c40 con 0x7f1c540b1860 2020-11-28T21:56:39.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.045+0000 7f1c5a1cd700 1 -- 172.21.15.162:0/1388258063 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c540659b0 con 0x7f1c540b1860 2020-11-28T21:56:39.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.045+0000 7f1c527fc700 1 -- 172.21.15.162:0/1388258063 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f1c44010030 con 0x7f1c540b1860 2020-11-28T21:56:39.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.045+0000 7f1c527fc700 1 --2- 172.21.15.162:0/1388258063 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1c34055330 0x7f1c340577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:39.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.045+0000 7f1c527fc700 1 -- 172.21.15.162:0/1388258063 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f1c4406e200 con 0x7f1c540b1860 2020-11-28T21:56:39.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.045+0000 7f1c591cb700 1 --2- 172.21.15.162:0/1388258063 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1c34055330 0x7f1c340577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:39.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.045+0000 7f1c591cb700 1 --2- 172.21.15.162:0/1388258063 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1c34055330 0x7f1c340577f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7f1c4800b000 tx=0x7f1c48009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:39.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.049+0000 7f1c527fc700 1 -- 172.21.15.162:0/1388258063 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f1c44044320 con 0x7f1c540b1860 2020-11-28T21:56:39.237 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.233+0000 7f1c5a1cd700 1 -- 172.21.15.162:0/1388258063 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1c54057050 con 0x7f1c540b1860 2020-11-28T21:56:39.238 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.233+0000 7f1c527fc700 1 -- 172.21.15.162:0/1388258063 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+2138 (secure 0 0 0) 0x7f1c44047940 con 0x7f1c540b1860 2020-11-28T21:56:39.238 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T21:56:39.239 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":81,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8199,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/2385081970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":2385081970},{"type":"v1","addr":"172.21.15.118:6833","nonce":2385081970}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":77},{"gid":8554,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.162:6836/169306274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":169306274},{"type":"v1","addr":"172.21.15.162:6836","nonce":169306274}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":78}],"filesystems":[{"mdsmap":{"epoch":81,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-28T21:56:25.831016+0000","modified":"2020-11-28T21:56:38.887393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":161,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2020-11-28T21:56:39.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.233+0000 7f1c5a1cd700 1 -- 172.21.15.162:0/1388258063 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1c34055330 msgr2=0x7f1c340577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:39.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.233+0000 7f1c5a1cd700 1 --2- 172.21.15.162:0/1388258063 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1c34055330 0x7f1c340577f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7f1c4800b000 tx=0x7f1c48009040).stop 2020-11-28T21:56:39.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.233+0000 7f1c5a1cd700 1 -- 172.21.15.162:0/1388258063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c540b1860 msgr2=0x7f1c5414a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:39.241 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.233+0000 7f1c5a1cd700 1 --2- 172.21.15.162:0/1388258063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c540b1860 0x7f1c5414a110 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 rx=0x7f1c54066b60 tx=0x7f1c44007df0).stop 2020-11-28T21:56:39.241 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.233+0000 7f1c5a1cd700 1 -- 172.21.15.162:0/1388258063 shutdown_connections 2020-11-28T21:56:39.242 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.233+0000 7f1c5a1cd700 1 --2- 172.21.15.162:0/1388258063 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f1c34055330 0x7f1c340577f0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:39.242 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.233+0000 7f1c5a1cd700 1 --2- 172.21.15.162:0/1388258063 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1c540ad2c0 0x7f1c5413edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:39.243 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.233+0000 7f1c5a1cd700 1 --2- 172.21.15.162:0/1388258063 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1c540b2830 0x7f1c5414a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:39.243 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.233+0000 7f1c5a1cd700 1 --2- 172.21.15.162:0/1388258063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c540b1860 0x7f1c5414a110 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:39.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.233+0000 7f1c5a1cd700 1 -- 172.21.15.162:0/1388258063 >> 172.21.15.162:0/1388258063 conn(0x7f1c54061550 msgr2=0x7f1c5413dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:39.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.233+0000 7f1c5a1cd700 1 -- 172.21.15.162:0/1388258063 shutdown_connections 2020-11-28T21:56:39.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.233+0000 7f1c5a1cd700 1 -- 172.21.15.162:0/1388258063 wait complete. 2020-11-28T21:56:39.244 INFO:teuthology.orchestra.run.smithi162.stderr:dumped fsmap epoch 81 2020-11-28T21:56:39.250 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:56:39.256 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2020-11-28T21:56:39.260 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:56:39.288 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:56:39.337 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:56:39.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.437+0000 7f9c2f59e700 1 Processor -- start 2020-11-28T21:56:39.443 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.437+0000 7f9c2f59e700 1 -- start start 2020-11-28T21:56:39.450 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.437+0000 7f9c2f59e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f9c300ae710 0x7f9c30060070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:39.451 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.437+0000 7f9c2f59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9c300aeef0 0x7f9c30140870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:39.451 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.437+0000 7f9c2f59e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9c30144620 0x7f9c3013db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:39.451 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.437+0000 7f9c2f59e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c300b0330 con 0x7f9c300ae710 2020-11-28T21:56:39.451 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.437+0000 7f9c2f59e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c300aeaf0 con 0x7f9c30144620 2020-11-28T21:56:39.452 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.437+0000 7f9c2f59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c30060c10 con 0x7f9c300aeef0 2020-11-28T21:56:39.452 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.437+0000 7f9c2ed9d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9c30144620 0x7f9c3013db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:39.452 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.437+0000 7f9c2ed9d700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9c30144620 0x7f9c3013db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46652/0 (socket says 172.21.15.162:46652) 2020-11-28T21:56:39.452 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.437+0000 7f9c2ed9d700 1 -- 172.21.15.162:0/390983330 learned_addr learned my addr 172.21.15.162:0/390983330 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:39.453 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.437+0000 7f9c2dd9b700 1 --2- 172.21.15.162:0/390983330 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9c300aeef0 0x7f9c30140870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:39.453 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.437+0000 7f9c2ed9d700 1 -- 172.21.15.162:0/390983330 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9c300aeef0 msgr2=0x7f9c30140870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:39.453 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.437+0000 7f9c2ed9d700 1 --2- 172.21.15.162:0/390983330 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9c300aeef0 0x7f9c30140870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:39.453 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.437+0000 7f9c2ed9d700 1 -- 172.21.15.162:0/390983330 >> v2:172.21.15.118:3300/0 conn(0x7f9c300ae710 msgr2=0x7f9c30060070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:39.454 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.437+0000 7f9c2e59c700 1 --2- 172.21.15.162:0/390983330 >> v2:172.21.15.118:3300/0 conn(0x7f9c300ae710 0x7f9c30060070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:39.454 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.437+0000 7f9c2ed9d700 1 --2- 172.21.15.162:0/390983330 >> v2:172.21.15.118:3300/0 conn(0x7f9c300ae710 0x7f9c30060070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:39.454 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.437+0000 7f9c2ed9d700 1 -- 172.21.15.162:0/390983330 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c30060eb0 con 0x7f9c30144620 2020-11-28T21:56:39.455 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.437+0000 7f9c2e59c700 1 --2- 172.21.15.162:0/390983330 >> v2:172.21.15.118:3300/0 conn(0x7f9c300ae710 0x7f9c30060070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T21:56:39.455 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.437+0000 7f9c2ed9d700 1 --2- 172.21.15.162:0/390983330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c30144620 0x7f9c3013db40 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 rx=0x7f9c200032b0 tx=0x7f9c20009270).ready entity=mon.1 client_cookie=96b2428f4799121a server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:39.455 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.437+0000 7f9c2d59a700 1 -- 172.21.15.162:0/390983330 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c2000bc00 con 0x7f9c30144620 2020-11-28T21:56:39.455 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.437+0000 7f9c2d59a700 1 -- 172.21.15.162:0/390983330 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c2000bd60 con 0x7f9c30144620 2020-11-28T21:56:39.456 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.437+0000 7f9c2d59a700 1 -- 172.21.15.162:0/390983330 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c20011400 con 0x7f9c30144620 2020-11-28T21:56:39.456 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.437+0000 7f9c2f59e700 1 -- 172.21.15.162:0/390983330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c30144620 msgr2=0x7f9c3013db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:39.456 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.437+0000 7f9c2f59e700 1 --2- 172.21.15.162:0/390983330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c30144620 0x7f9c3013db40 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 rx=0x7f9c200032b0 tx=0x7f9c20009270).stop 2020-11-28T21:56:39.457 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.441+0000 7f9c2f59e700 1 -- 172.21.15.162:0/390983330 shutdown_connections 2020-11-28T21:56:39.457 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.441+0000 7f9c2f59e700 1 --2- 172.21.15.162:0/390983330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c30144620 0x7f9c3013db40 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:39.458 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.441+0000 7f9c2f59e700 1 --2- 172.21.15.162:0/390983330 >> v2:172.21.15.118:3300/0 conn(0x7f9c300ae710 0x7f9c30060070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:39.458 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.441+0000 7f9c2f59e700 1 --2- 172.21.15.162:0/390983330 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9c300aeef0 0x7f9c30140870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:39.459 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.441+0000 7f9c2f59e700 1 -- 172.21.15.162:0/390983330 >> 172.21.15.162:0/390983330 conn(0x7f9c300b17a0 msgr2=0x7f9c300b1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:39.459 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.441+0000 7f9c2f59e700 1 -- 172.21.15.162:0/390983330 shutdown_connections 2020-11-28T21:56:39.459 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.441+0000 7f9c2f59e700 1 -- 172.21.15.162:0/390983330 wait complete. 2020-11-28T21:56:39.459 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.441+0000 7f9c2f59e700 1 Processor -- start 2020-11-28T21:56:39.459 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.441+0000 7f9c2f59e700 1 -- start start 2020-11-28T21:56:39.460 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.441+0000 7f9c2f59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9c300ae710 0x7f9c3014d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:39.460 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.441+0000 7f9c2f59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9c300aeef0 0x7f9c30065dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:39.460 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.441+0000 7f9c2f59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c30144620 0x7f9c30140070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:39.460 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.441+0000 7f9c2f59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c300666b0 con 0x7f9c300ae710 2020-11-28T21:56:39.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.441+0000 7f9c2f59e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c30065990 con 0x7f9c30144620 2020-11-28T21:56:39.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.441+0000 7f9c2f59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3014d820 con 0x7f9c300aeef0 2020-11-28T21:56:39.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.441+0000 7f9c2ed9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c30144620 0x7f9c30140070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:39.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.441+0000 7f9c2ed9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c30144620 0x7f9c30140070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46660/0 (socket says 172.21.15.162:46660) 2020-11-28T21:56:39.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.441+0000 7f9c2ed9d700 1 -- 172.21.15.162:0/299722543 learned_addr learned my addr 172.21.15.162:0/299722543 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:39.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.441+0000 7f9c2dd9b700 1 --2- 172.21.15.162:0/299722543 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9c300aeef0 0x7f9c30065dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:39.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.441+0000 7f9c2e59c700 1 --2- 172.21.15.162:0/299722543 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9c300ae710 0x7f9c3014d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:39.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.441+0000 7f9c2ed9d700 1 -- 172.21.15.162:0/299722543 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9c300aeef0 msgr2=0x7f9c30065dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:39.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.441+0000 7f9c2ed9d700 1 --2- 172.21.15.162:0/299722543 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9c300aeef0 0x7f9c30065dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:39.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.441+0000 7f9c2ed9d700 1 -- 172.21.15.162:0/299722543 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9c300ae710 msgr2=0x7f9c3014d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:39.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.441+0000 7f9c2ed9d700 1 --2- 172.21.15.162:0/299722543 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9c300ae710 0x7f9c3014d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:39.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.441+0000 7f9c2ed9d700 1 -- 172.21.15.162:0/299722543 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c300668e0 con 0x7f9c30144620 2020-11-28T21:56:39.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.441+0000 7f9c2ed9d700 1 --2- 172.21.15.162:0/299722543 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c30144620 0x7f9c30140070 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 rx=0x7f9c3014e3e0 tx=0x7f9c20016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:39.463 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.441+0000 7f9c277fe700 1 -- 172.21.15.162:0/299722543 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c2001f5e0 con 0x7f9c30144620 2020-11-28T21:56:39.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.441+0000 7f9c277fe700 1 -- 172.21.15.162:0/299722543 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c2001f740 con 0x7f9c30144620 2020-11-28T21:56:39.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.441+0000 7f9c2f59e700 1 -- 172.21.15.162:0/299722543 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c30064a20 con 0x7f9c30144620 2020-11-28T21:56:39.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.441+0000 7f9c277fe700 1 -- 172.21.15.162:0/299722543 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c2001faa0 con 0x7f9c30144620 2020-11-28T21:56:39.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.441+0000 7f9c2f59e700 1 -- 172.21.15.162:0/299722543 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c30140b40 con 0x7f9c30144620 2020-11-28T21:56:39.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.441+0000 7f9c2f59e700 1 -- 172.21.15.162:0/299722543 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c30065b00 con 0x7f9c30144620 2020-11-28T21:56:39.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.445+0000 7f9c277fe700 1 -- 172.21.15.162:0/299722543 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9c20026030 con 0x7f9c30144620 2020-11-28T21:56:39.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.445+0000 7f9c277fe700 1 --2- 172.21.15.162:0/299722543 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9c08055330 0x7f9c080577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:39.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.445+0000 7f9c277fe700 1 -- 172.21.15.162:0/299722543 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f9c2006eff0 con 0x7f9c30144620 2020-11-28T21:56:39.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.445+0000 7f9c2e59c700 1 --2- 172.21.15.162:0/299722543 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9c08055330 0x7f9c080577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:39.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.445+0000 7f9c2e59c700 1 --2- 172.21.15.162:0/299722543 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9c08055330 0x7f9c080577f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f9c2800b000 tx=0x7f9c28008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:39.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.449+0000 7f9c277fe700 1 -- 172.21.15.162:0/299722543 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f9c30065b00 con 0x7f9c30144620 2020-11-28T21:56:39.635 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.629+0000 7f9c2f59e700 1 -- 172.21.15.162:0/299722543 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f9c30065ad0 con 0x7f9c30144620 2020-11-28T21:56:39.909 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.901+0000 7f9c277fe700 1 -- 172.21.15.162:0/299722543 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v82) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9c200150a0 con 0x7f9c30144620 2020-11-28T21:56:39.913 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.905+0000 7f9c2f59e700 1 -- 172.21.15.162:0/299722543 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9c08055330 msgr2=0x7f9c080577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:39.914 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.905+0000 7f9c2f59e700 1 --2- 172.21.15.162:0/299722543 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9c08055330 0x7f9c080577f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f9c2800b000 tx=0x7f9c28008040).stop 2020-11-28T21:56:39.914 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.905+0000 7f9c2f59e700 1 -- 172.21.15.162:0/299722543 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c30144620 msgr2=0x7f9c30140070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:39.914 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.905+0000 7f9c2f59e700 1 --2- 172.21.15.162:0/299722543 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c30144620 0x7f9c30140070 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 rx=0x7f9c3014e3e0 tx=0x7f9c20016000).stop 2020-11-28T21:56:39.915 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.905+0000 7f9c2f59e700 1 -- 172.21.15.162:0/299722543 shutdown_connections 2020-11-28T21:56:39.915 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.905+0000 7f9c2f59e700 1 --2- 172.21.15.162:0/299722543 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9c08055330 0x7f9c080577f0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:39.915 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.905+0000 7f9c2f59e700 1 --2- 172.21.15.162:0/299722543 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c30144620 0x7f9c30140070 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:39.916 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.905+0000 7f9c2f59e700 1 --2- 172.21.15.162:0/299722543 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9c300ae710 0x7f9c3014d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:39.916 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.905+0000 7f9c2f59e700 1 --2- 172.21.15.162:0/299722543 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9c300aeef0 0x7f9c30065dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:39.916 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.905+0000 7f9c2f59e700 1 -- 172.21.15.162:0/299722543 >> 172.21.15.162:0/299722543 conn(0x7f9c300b17a0 msgr2=0x7f9c300b2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:39.917 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.905+0000 7f9c2f59e700 1 -- 172.21.15.162:0/299722543 shutdown_connections 2020-11-28T21:56:39.918 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:39.905+0000 7f9c2f59e700 1 -- 172.21.15.162:0/299722543 wait complete. 2020-11-28T21:56:39.934 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-11-28T21:56:40.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.101+0000 7fa5ae790700 1 Processor -- start 2020-11-28T21:56:40.109 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.101+0000 7fa5ae790700 1 -- start start 2020-11-28T21:56:40.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.105+0000 7fa5ae790700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fa5a813d0c0 0x7fa5a80b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:40.111 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.105+0000 7fa5ae790700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa5a809a270 0x7fa5a805bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:40.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.105+0000 7fa5ae790700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5a809a670 con 0x7fa5a80b13c0 2020-11-28T21:56:40.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.105+0000 7fa5ae790700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5a813d4a0 con 0x7fa5a813d0c0 2020-11-28T21:56:40.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.105+0000 7fa5ae790700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5a8098da0 con 0x7fa5a809a270 2020-11-28T21:56:40.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.105+0000 7fa5adf8f700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa5a809a270 0x7fa5a805bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:40.112 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.105+0000 7fa5acf8d700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fa5a80b13c0 0x7fa5a8099b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:53512/0 (socket says 172.21.15.162:53512) 2020-11-28T21:56:40.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.105+0000 7fa5acf8d700 1 -- 172.21.15.162:0/1896262399 learned_addr learned my addr 172.21.15.162:0/1896262399 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T21:56:40.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.105+0000 7fa5ad78e700 1 --2- 172.21.15.162:0/1896262399 >> v2:172.21.15.118:3300/0 conn(0x7fa5a813d0c0 0x7fa5a80b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:40.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.105+0000 7fa5adf8f700 1 -- 172.21.15.162:0/1896262399 >> v2:172.21.15.118:3300/0 conn(0x7fa5a813d0c0 msgr2=0x7fa5a80b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:40.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.105+0000 7fa5adf8f700 1 --2- 172.21.15.162:0/1896262399 >> v2:172.21.15.118:3300/0 conn(0x7fa5a813d0c0 0x7fa5a80b0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:40.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.105+0000 7fa5adf8f700 1 -- 172.21.15.162:0/1896262399 >> v1:172.21.15.162:6789/0 conn(0x7fa5a80b13c0 legacy=0x7fa5a8099b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:40.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.105+0000 7fa597fff700 1 -- 172.21.15.162:0/1896262399 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3207700092 0 0) 0x7fa5a809a670 con 0x7fa5a80b13c0 2020-11-28T21:56:40.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.105+0000 7fa5adf8f700 1 -- 172.21.15.162:0/1896262399 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a806aca0 con 0x7fa5a809a270 2020-11-28T21:56:40.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.105+0000 7fa5adf8f700 1 --2- 172.21.15.162:0/1896262399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5a809a270 0x7fa5a805bf90 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 rx=0x7fa5a0002f30 tx=0x7fa5a0006dc0).ready entity=mon.1 client_cookie=37a42a94973f08cc server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:40.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.105+0000 7fa597fff700 1 -- 172.21.15.162:0/1896262399 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5a00097b0 con 0x7fa5a809a270 2020-11-28T21:56:40.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.105+0000 7fa597fff700 1 -- 172.21.15.162:0/1896262399 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a0009910 con 0x7fa5a809a270 2020-11-28T21:56:40.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.105+0000 7fa597fff700 1 -- 172.21.15.162:0/1896262399 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5a0009c70 con 0x7fa5a809a270 2020-11-28T21:56:40.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.105+0000 7fa5ae790700 1 -- 172.21.15.162:0/1896262399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5a809a270 msgr2=0x7fa5a805bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:40.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.105+0000 7fa5ae790700 1 --2- 172.21.15.162:0/1896262399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5a809a270 0x7fa5a805bf90 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 rx=0x7fa5a0002f30 tx=0x7fa5a0006dc0).stop 2020-11-28T21:56:40.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.109+0000 7fa5ae790700 1 -- 172.21.15.162:0/1896262399 shutdown_connections 2020-11-28T21:56:40.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.109+0000 7fa5ae790700 1 --2- 172.21.15.162:0/1896262399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5a809a270 0x7fa5a805bf90 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:40.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.109+0000 7fa5ae790700 1 --2- 172.21.15.162:0/1896262399 >> v2:172.21.15.118:3300/0 conn(0x7fa5a813d0c0 0x7fa5a80b0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:40.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.109+0000 7fa5ae790700 1 -- 172.21.15.162:0/1896262399 >> 172.21.15.162:0/1896262399 conn(0x7fa5a80b5520 msgr2=0x7fa5a80b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:40.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.109+0000 7fa5ae790700 1 -- 172.21.15.162:0/1896262399 shutdown_connections 2020-11-28T21:56:40.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.109+0000 7fa5ae790700 1 -- 172.21.15.162:0/1896262399 wait complete. 2020-11-28T21:56:40.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.109+0000 7fa5ae790700 1 Processor -- start 2020-11-28T21:56:40.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.109+0000 7fa5ae790700 1 -- start start 2020-11-28T21:56:40.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.109+0000 7fa5ae790700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa5a809a270 0x7fa5a813e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:40.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.109+0000 7fa5ae790700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa5a80b13c0 0x7fa5a8151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:40.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.113+0000 7fa5ae790700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5a813d0c0 0x7fa5a8152280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:40.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.113+0000 7fa5ae790700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a813b100 con 0x7fa5a809a270 2020-11-28T21:56:40.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.113+0000 7fa5ae790700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a8155660 con 0x7fa5a813d0c0 2020-11-28T21:56:40.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.113+0000 7fa5ae790700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a8152770 con 0x7fa5a80b13c0 2020-11-28T21:56:40.121 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.113+0000 7fa5adf8f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5a813d0c0 0x7fa5a8152280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:40.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.113+0000 7fa5adf8f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5a813d0c0 0x7fa5a8152280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46676/0 (socket says 172.21.15.162:46676) 2020-11-28T21:56:40.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.113+0000 7fa5adf8f700 1 -- 172.21.15.162:0/3497481688 learned_addr learned my addr 172.21.15.162:0/3497481688 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:40.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.113+0000 7fa5acf8d700 1 --2- 172.21.15.162:0/3497481688 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa5a80b13c0 0x7fa5a8151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:40.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.113+0000 7fa5ad78e700 1 --2- 172.21.15.162:0/3497481688 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa5a809a270 0x7fa5a813e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:40.122 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.113+0000 7fa5adf8f700 1 -- 172.21.15.162:0/3497481688 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa5a80b13c0 msgr2=0x7fa5a8151d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:40.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.113+0000 7fa5adf8f700 1 --2- 172.21.15.162:0/3497481688 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa5a80b13c0 0x7fa5a8151d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:40.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.113+0000 7fa5adf8f700 1 -- 172.21.15.162:0/3497481688 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa5a809a270 msgr2=0x7fa5a813e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:40.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.113+0000 7fa5adf8f700 1 --2- 172.21.15.162:0/3497481688 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa5a809a270 0x7fa5a813e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:40.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.113+0000 7fa5adf8f700 1 -- 172.21.15.162:0/3497481688 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a81558f0 con 0x7fa5a813d0c0 2020-11-28T21:56:40.123 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.113+0000 7fa5adf8f700 1 --2- 172.21.15.162:0/3497481688 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5a813d0c0 0x7fa5a8152280 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 rx=0x7fa5a0002f30 tx=0x7fa5a0015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:40.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.113+0000 7fa5967fc700 1 -- 172.21.15.162:0/3497481688 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5a0011ae0 con 0x7fa5a813d0c0 2020-11-28T21:56:40.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.113+0000 7fa5967fc700 1 -- 172.21.15.162:0/3497481688 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a0011c40 con 0x7fa5a813d0c0 2020-11-28T21:56:40.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.113+0000 7fa5967fc700 1 -- 172.21.15.162:0/3497481688 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5a001d450 con 0x7fa5a813d0c0 2020-11-28T21:56:40.124 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.113+0000 7fa5ae790700 1 -- 172.21.15.162:0/3497481688 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5a8155b90 con 0x7fa5a813d0c0 2020-11-28T21:56:40.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.113+0000 7fa5ae790700 1 -- 172.21.15.162:0/3497481688 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5a80600d0 con 0x7fa5a813d0c0 2020-11-28T21:56:40.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.113+0000 7fa5967fc700 1 -- 172.21.15.162:0/3497481688 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fa5a0024030 con 0x7fa5a813d0c0 2020-11-28T21:56:40.125 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.113+0000 7fa5ae790700 1 -- 172.21.15.162:0/3497481688 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5a8057050 con 0x7fa5a813d0c0 2020-11-28T21:56:40.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.113+0000 7fa5967fc700 1 --2- 172.21.15.162:0/3497481688 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa5880554e0 0x7fa5880579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:40.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.113+0000 7fa5967fc700 1 -- 172.21.15.162:0/3497481688 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fa5a006f0c0 con 0x7fa5a813d0c0 2020-11-28T21:56:40.126 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.113+0000 7fa5ad78e700 1 --2- 172.21.15.162:0/3497481688 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa5880554e0 0x7fa5880579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:40.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.113+0000 7fa5ad78e700 1 --2- 172.21.15.162:0/3497481688 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa5880554e0 0x7fa5880579a0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7fa5a400b000 tx=0x7fa5a40091a0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:40.127 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.121+0000 7fa5967fc700 1 -- 172.21.15.162:0/3497481688 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fa5a0045370 con 0x7fa5a813d0c0 2020-11-28T21:56:40.262 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.257+0000 7fa5ae790700 1 -- 172.21.15.162:0/3497481688 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa5a80aed90 con 0x7fa5a813d0c0 2020-11-28T21:56:40.914 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.909+0000 7fa5967fc700 1 -- 172.21.15.162:0/3497481688 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v162) v1 ==== 185+0+0 (secure 0 0 0) 0x7fa5a0022070 con 0x7fa5a813d0c0 2020-11-28T21:56:40.918 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.913+0000 7fa5ae790700 1 -- 172.21.15.162:0/3497481688 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa5880554e0 msgr2=0x7fa5880579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:40.918 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.913+0000 7fa5ae790700 1 --2- 172.21.15.162:0/3497481688 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa5880554e0 0x7fa5880579a0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7fa5a400b000 tx=0x7fa5a40091a0).stop 2020-11-28T21:56:40.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.913+0000 7fa5ae790700 1 -- 172.21.15.162:0/3497481688 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5a813d0c0 msgr2=0x7fa5a8152280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:40.919 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.913+0000 7fa5ae790700 1 --2- 172.21.15.162:0/3497481688 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5a813d0c0 0x7fa5a8152280 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 rx=0x7fa5a0002f30 tx=0x7fa5a0015000).stop 2020-11-28T21:56:40.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.913+0000 7fa5ae790700 1 -- 172.21.15.162:0/3497481688 shutdown_connections 2020-11-28T21:56:40.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.913+0000 7fa5ae790700 1 --2- 172.21.15.162:0/3497481688 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fa5880554e0 0x7fa5880579a0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:40.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.913+0000 7fa5ae790700 1 --2- 172.21.15.162:0/3497481688 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5a813d0c0 0x7fa5a8152280 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:40.920 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.913+0000 7fa5ae790700 1 --2- 172.21.15.162:0/3497481688 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa5a809a270 0x7fa5a813e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:40.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.913+0000 7fa5ae790700 1 --2- 172.21.15.162:0/3497481688 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa5a80b13c0 0x7fa5a8151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:40.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.913+0000 7fa5ae790700 1 -- 172.21.15.162:0/3497481688 >> 172.21.15.162:0/3497481688 conn(0x7fa5a80b5520 msgr2=0x7fa5a80b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:40.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.913+0000 7fa5ae790700 1 -- 172.21.15.162:0/3497481688 shutdown_connections 2020-11-28T21:56:40.921 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:40.913+0000 7fa5ae790700 1 -- 172.21.15.162:0/3497481688 wait complete. 2020-11-28T21:56:40.922 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_metadata' removed 2020-11-28T21:56:40.936 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-11-28T21:56:41.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.049+0000 7fe58de02700 1 Processor -- start 2020-11-28T21:56:41.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.049+0000 7fe58de02700 1 -- start start 2020-11-28T21:56:41.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.049+0000 7fe58de02700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe5880b1120 0x7fe588147660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:41.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.049+0000 7fe58de02700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe58814a440 0x7fe5881517c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:41.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.049+0000 7fe58de02700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5880a8690 con 0x7fe5880b5730 2020-11-28T21:56:41.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.049+0000 7fe58de02700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5880a88c0 con 0x7fe5880b1120 2020-11-28T21:56:41.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.049+0000 7fe58de02700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5881471c0 con 0x7fe58814a440 2020-11-28T21:56:41.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.049+0000 7fe57effd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe5880b1120 0x7fe588147660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:41.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.049+0000 7fe57effd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe5880b1120 0x7fe588147660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46680/0 (socket says 172.21.15.162:46680) 2020-11-28T21:56:41.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.049+0000 7fe57effd700 1 -- 172.21.15.162:0/2442232865 learned_addr learned my addr 172.21.15.162:0/2442232865 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:41.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.049+0000 7fe5877fe700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fe5880b5730 0x7fe58814ac50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:53524/0 (socket says 172.21.15.162:53524) 2020-11-28T21:56:41.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.053+0000 7fe587fff700 1 --2- 172.21.15.162:0/2442232865 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe58814a440 0x7fe5881517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:41.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.053+0000 7fe57effd700 1 -- 172.21.15.162:0/2442232865 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe58814a440 msgr2=0x7fe5881517c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T21:56:41.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.053+0000 7fe57effd700 1 --2- 172.21.15.162:0/2442232865 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe58814a440 0x7fe5881517c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:41.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.053+0000 7fe57effd700 1 -- 172.21.15.162:0/2442232865 >> v1:172.21.15.162:6789/0 conn(0x7fe5880b5730 legacy=0x7fe58814ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:41.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.053+0000 7fe586ffd700 1 -- 172.21.15.162:0/2442232865 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 487694170 0 0) 0x7fe5880a8690 con 0x7fe5880b5730 2020-11-28T21:56:41.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.053+0000 7fe57effd700 1 -- 172.21.15.162:0/2442232865 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe58814b360 con 0x7fe5880b1120 2020-11-28T21:56:41.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.053+0000 7fe57effd700 1 --2- 172.21.15.162:0/2442232865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5880b1120 0x7fe588147660 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 rx=0x7fe570002f30 tx=0x7fe570006e80).ready entity=mon.1 client_cookie=460d22457856c70b server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:41.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.053+0000 7fe586ffd700 1 -- 172.21.15.162:0/2442232865 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5700097f0 con 0x7fe5880b1120 2020-11-28T21:56:41.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.053+0000 7fe586ffd700 1 -- 172.21.15.162:0/2442232865 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe570009950 con 0x7fe5880b1120 2020-11-28T21:56:41.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.053+0000 7fe586ffd700 1 -- 172.21.15.162:0/2442232865 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe570009cb0 con 0x7fe5880b1120 2020-11-28T21:56:41.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.053+0000 7fe58de02700 1 -- 172.21.15.162:0/2442232865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5880b1120 msgr2=0x7fe588147660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:41.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.053+0000 7fe58de02700 1 --2- 172.21.15.162:0/2442232865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5880b1120 0x7fe588147660 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 rx=0x7fe570002f30 tx=0x7fe570006e80).stop 2020-11-28T21:56:41.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.053+0000 7fe58de02700 1 -- 172.21.15.162:0/2442232865 shutdown_connections 2020-11-28T21:56:41.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.053+0000 7fe58de02700 1 --2- 172.21.15.162:0/2442232865 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe58814a440 0x7fe5881517c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:41.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.053+0000 7fe58de02700 1 --2- 172.21.15.162:0/2442232865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5880b1120 0x7fe588147660 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:41.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.053+0000 7fe58de02700 1 -- 172.21.15.162:0/2442232865 >> 172.21.15.162:0/2442232865 conn(0x7fe58805c3f0 msgr2=0x7fe5880b4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:41.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.053+0000 7fe58de02700 1 -- 172.21.15.162:0/2442232865 shutdown_connections 2020-11-28T21:56:41.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.053+0000 7fe58de02700 1 -- 172.21.15.162:0/2442232865 wait complete. 2020-11-28T21:56:41.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.057+0000 7fe58de02700 1 Processor -- start 2020-11-28T21:56:41.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.057+0000 7fe58de02700 1 -- start start 2020-11-28T21:56:41.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.057+0000 7fe58de02700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe5880b1120 0x7fe588145d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:41.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.057+0000 7fe58de02700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5880b5730 0x7fe58814dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:41.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.057+0000 7fe58de02700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe58814a440 0x7fe58814e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:41.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.057+0000 7fe58de02700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe588159380 con 0x7fe58814a440 2020-11-28T21:56:41.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.057+0000 7fe58de02700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe588159620 con 0x7fe5880b5730 2020-11-28T21:56:41.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.057+0000 7fe58de02700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5881598c0 con 0x7fe5880b1120 2020-11-28T21:56:41.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.057+0000 7fe57effd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5880b5730 0x7fe58814dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:41.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.057+0000 7fe5877fe700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe5880b1120 0x7fe588145d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:41.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.057+0000 7fe5877fe700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe5880b1120 0x7fe588145d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:43910/0 (socket says 172.21.15.162:43910) 2020-11-28T21:56:41.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.057+0000 7fe5877fe700 1 -- 172.21.15.162:0/1597596693 learned_addr learned my addr 172.21.15.162:0/1597596693 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T21:56:41.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.057+0000 7fe57effd700 1 --2- 172.21.15.162:0/1597596693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5880b5730 0x7fe58814dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46688/0 (socket says 172.21.15.162:46688) 2020-11-28T21:56:41.070 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.057+0000 7fe587fff700 1 --2- 172.21.15.162:0/1597596693 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe58814a440 0x7fe58814e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:41.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.057+0000 7fe5877fe700 1 -- 172.21.15.162:0/1597596693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5880b5730 msgr2=0x7fe58814dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:41.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.057+0000 7fe5877fe700 1 --2- 172.21.15.162:0/1597596693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5880b5730 0x7fe58814dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:41.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.057+0000 7fe5877fe700 1 -- 172.21.15.162:0/1597596693 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe58814a440 msgr2=0x7fe58814e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:41.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.057+0000 7fe5877fe700 1 --2- 172.21.15.162:0/1597596693 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe58814a440 0x7fe58814e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:41.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.057+0000 7fe5877fe700 1 -- 172.21.15.162:0/1597596693 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe588159b60 con 0x7fe5880b1120 2020-11-28T21:56:41.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.057+0000 7fe57effd700 1 --2- 172.21.15.162:0/1597596693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5880b5730 0x7fe58814dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:56:41.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.057+0000 7fe587fff700 1 --2- 172.21.15.162:0/1597596693 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe58814a440 0x7fe58814e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T21:56:41.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.057+0000 7fe5877fe700 1 --2- 172.21.15.162:0/1597596693 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe5880b1120 0x7fe588145d20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fe578000e90 tx=0x7fe578007a90).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:41.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.061+0000 7fe5857fa700 1 -- 172.21.15.162:0/1597596693 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5780073b0 con 0x7fe5880b1120 2020-11-28T21:56:41.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.061+0000 7fe5857fa700 1 -- 172.21.15.162:0/1597596693 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe578005520 con 0x7fe5880b1120 2020-11-28T21:56:41.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.061+0000 7fe5857fa700 1 -- 172.21.15.162:0/1597596693 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe578005880 con 0x7fe5880b1120 2020-11-28T21:56:41.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.061+0000 7fe58de02700 1 -- 172.21.15.162:0/1597596693 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe58814c9f0 con 0x7fe5880b1120 2020-11-28T21:56:41.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.061+0000 7fe58de02700 1 -- 172.21.15.162:0/1597596693 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe588146210 con 0x7fe5880b1120 2020-11-28T21:56:41.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.061+0000 7fe58de02700 1 -- 172.21.15.162:0/1597596693 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe588159cf0 con 0x7fe5880b1120 2020-11-28T21:56:41.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.061+0000 7fe5857fa700 1 -- 172.21.15.162:0/1597596693 <== mon.2 v2:172.21.15.118:3301/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe578010070 con 0x7fe5880b1120 2020-11-28T21:56:41.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.061+0000 7fe5857fa700 1 --2- 172.21.15.162:0/1597596693 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe5600553b0 0x7fe560057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T21:56:41.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.061+0000 7fe5857fa700 1 -- 172.21.15.162:0/1597596693 <== mon.2 v2:172.21.15.118:3301/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5721+0+0 (secure 0 0 0) 0x7fe57806d660 con 0x7fe5880b1120 2020-11-28T21:56:41.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.061+0000 7fe57effd700 1 --2- 172.21.15.162:0/1597596693 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe5600553b0 0x7fe560057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T21:56:41.075 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.061+0000 7fe57effd700 1 --2- 172.21.15.162:0/1597596693 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe5600553b0 0x7fe560057870 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7fe57001c000 tx=0x7fe570015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T21:56:41.076 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.069+0000 7fe5857fa700 1 -- 172.21.15.162:0/1597596693 <== mon.2 v2:172.21.15.118:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe5780439b0 con 0x7fe5880b1120 2020-11-28T21:56:41.225 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.221+0000 7fe58de02700 1 -- 172.21.15.162:0/1597596693 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe588057050 con 0x7fe5880b1120 2020-11-28T21:56:41.925 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.917+0000 7fe5857fa700 1 -- 172.21.15.162:0/1597596693 <== mon.2 v2:172.21.15.118:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v163) v1 ==== 173+0+0 (secure 0 0 0) 0x7fe578046fd0 con 0x7fe5880b1120 2020-11-28T21:56:41.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.921+0000 7fe58de02700 1 -- 172.21.15.162:0/1597596693 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe5600553b0 msgr2=0x7fe560057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:41.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.921+0000 7fe58de02700 1 --2- 172.21.15.162:0/1597596693 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe5600553b0 0x7fe560057870 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7fe57001c000 tx=0x7fe570015000).stop 2020-11-28T21:56:41.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.921+0000 7fe58de02700 1 -- 172.21.15.162:0/1597596693 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe5880b1120 msgr2=0x7fe588145d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T21:56:41.928 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.921+0000 7fe58de02700 1 --2- 172.21.15.162:0/1597596693 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe5880b1120 0x7fe588145d20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fe578000e90 tx=0x7fe578007a90).stop 2020-11-28T21:56:41.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.921+0000 7fe58de02700 1 -- 172.21.15.162:0/1597596693 shutdown_connections 2020-11-28T21:56:41.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.921+0000 7fe58de02700 1 --2- 172.21.15.162:0/1597596693 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe5600553b0 0x7fe560057870 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:41.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.921+0000 7fe58de02700 1 --2- 172.21.15.162:0/1597596693 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe5880b1120 0x7fe588145d20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:41.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.921+0000 7fe58de02700 1 --2- 172.21.15.162:0/1597596693 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe58814a440 0x7fe58814e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:41.929 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.921+0000 7fe58de02700 1 --2- 172.21.15.162:0/1597596693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5880b5730 0x7fe58814dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T21:56:41.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.921+0000 7fe58de02700 1 -- 172.21.15.162:0/1597596693 >> 172.21.15.162:0/1597596693 conn(0x7fe58805c3f0 msgr2=0x7fe58805f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T21:56:41.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.921+0000 7fe58de02700 1 -- 172.21.15.162:0/1597596693 shutdown_connections 2020-11-28T21:56:41.930 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T21:56:41.921+0000 7fe58de02700 1 -- 172.21.15.162:0/1597596693 wait complete. 2020-11-28T21:56:41.931 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cephfs_data' removed 2020-11-28T21:56:41.944 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-28T21:56:41.945 INFO:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:56:41.962 INFO:teuthology.orchestra.run.smithi118:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:56:41.977 INFO:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:56:41.996 INFO:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-28T21:56:42.014 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-28T21:56:42.015 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T21:56:42.015 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T21:56:42.016 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T21:57:09.404 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:57:09.407 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:57:09.409 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:57:09.412 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:57:39.507 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:57:39.510 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:57:39.512 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:57:39.514 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:58:09.827 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:58:09.831 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:58:09.834 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:58:09.837 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:58:39.876 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:58:39.879 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:58:39.881 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:58:39.884 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:59:10.006 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:59:10.009 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:59:10.012 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:59:10.014 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:59:40.259 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:59:40.261 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:59:40.264 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T21:59:40.266 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:00:10.309 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:00:10.316 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:00:10.321 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:00:10.327 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:00:40.447 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:00:40.449 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:00:40.452 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:00:40.454 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:01:10.566 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:01:10.569 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:01:10.571 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:01:10.574 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:01:40.617 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:01:40.619 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:01:40.622 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:01:40.624 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:02:10.737 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:02:10.740 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:02:10.743 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:02:10.746 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:02:40.857 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:02:40.861 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:02:40.865 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:02:40.868 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:03:10.913 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:03:10.948 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:03:10.952 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:03:10.956 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:03:41.094 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:03:41.096 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:03:41.099 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:03:41.101 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:04:11.196 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:04:11.199 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:04:11.201 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:04:11.204 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:04:41.482 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:04:41.487 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:04:41.499 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:04:41.510 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:05:11.673 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:05:11.677 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:05:11.684 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:05:11.692 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:05:41.755 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:05:41.763 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:05:41.767 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:05:41.770 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:06:11.849 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:06:11.852 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:06:11.855 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:06:11.858 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:06:41.920 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:06:41.923 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:06:41.926 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:06:41.929 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:07:12.029 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:07:12.032 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:07:12.035 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:07:12.038 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:07:42.139 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:07:43.109 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:07:43.121 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:07:43.131 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:08:13.194 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:08:13.198 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:08:13.201 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:08:13.206 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:08:43.334 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:08:43.337 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:08:43.341 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:08:43.343 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:09:13.389 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:09:13.392 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:09:13.394 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:09:13.396 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:09:43.735 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:09:43.739 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:09:43.743 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:09:43.746 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:10:13.894 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:10:13.898 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:10:13.901 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:10:13.909 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:10:44.000 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:10:44.004 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:10:44.017 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:10:44.021 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:11:14.115 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:11:14.118 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:11:14.121 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:11:14.123 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:11:42.060 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 766, in gevent._greenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 316, in copy_file_to copy_to_log(src, logger, capture=stream) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 273, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2020-11-28T22:11:42.316 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 766, in gevent._greenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 316, in copy_file_to copy_to_log(src, logger, capture=stream) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 273, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2020-11-28T22:11:42.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T22:11:42.323 INFO:tasks.cephfs_test_runner:test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) ... ERROR 2020-11-28T22:11:42.325 INFO:tasks.cephfs_test_runner:ERROR 2020-11-28T22:11:42.326 INFO:tasks.cephfs_test_runner: 2020-11-28T22:11:42.327 INFO:tasks.cephfs_test_runner:====================================================================== 2020-11-28T22:11:42.327 INFO:tasks.cephfs_test_runner:ERROR: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2020-11-28T22:11:42.328 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2020-11-28T22:11:42.328 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2020-11-28T22:11:42.329 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks/cephfs/test_misc.py", line 24, in test_statfs_on_deleted_fs 2020-11-28T22:11:42.329 INFO:tasks.cephfs_test_runner: p = self.mount_a.run_shell_payload("stat -f .", wait=False, timeout=60, check_status=False) 2020-11-28T22:11:42.330 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks/cephfs/mount.py", line 201, in run_shell_payload 2020-11-28T22:11:42.330 INFO:tasks.cephfs_test_runner: return self.run_shell(["bash", "-c", Raw(f"'{payload}'")], **kwargs) 2020-11-28T22:11:42.331 INFO:tasks.cephfs_test_runner:TypeError: run_shell() got an unexpected keyword argument 'timeout' 2020-11-28T22:11:42.331 INFO:tasks.cephfs_test_runner: 2020-11-28T22:11:42.332 INFO:tasks.cephfs_test_runner:====================================================================== 2020-11-28T22:11:42.332 INFO:tasks.cephfs_test_runner:ERROR: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2020-11-28T22:11:42.333 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2020-11-28T22:11:42.333 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2020-11-28T22:11:42.333 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr 2020-11-28T22:11:42.334 INFO:tasks.cephfs_test_runner: out = self.in_stderr_buffer.read(nbytes, self.timeout) 2020-11-28T22:11:42.334 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read 2020-11-28T22:11:42.335 INFO:tasks.cephfs_test_runner: raise PipeTimeout() 2020-11-28T22:11:42.335 INFO:tasks.cephfs_test_runner:paramiko.buffered_pipe.PipeTimeout 2020-11-28T22:11:42.336 INFO:tasks.cephfs_test_runner: 2020-11-28T22:11:42.336 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2020-11-28T22:11:42.337 INFO:tasks.cephfs_test_runner: 2020-11-28T22:11:42.337 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2020-11-28T22:11:42.337 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks/cephfs/cephfs_test_case.py", line 175, in tearDown 2020-11-28T22:11:42.338 INFO:tasks.cephfs_test_runner: m.teardown() 2020-11-28T22:11:42.339 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks/cephfs/fuse_mount.py", line 408, in teardown 2020-11-28T22:11:42.339 INFO:tasks.cephfs_test_runner: self.umount() 2020-11-28T22:11:42.340 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks/cephfs/fuse_mount.py", line 261, in umount 2020-11-28T22:11:42.340 INFO:tasks.cephfs_test_runner: if not self.is_mounted(): 2020-11-28T22:11:42.340 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted 2020-11-28T22:11:42.341 INFO:tasks.cephfs_test_runner: proc.wait() 2020-11-28T22:11:42.341 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 147, in wait 2020-11-28T22:11:42.342 INFO:tasks.cephfs_test_runner: greenlet.get(block=True,timeout=60) 2020-11-28T22:11:42.342 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 683, in gevent._greenlet.Greenlet.get 2020-11-28T22:11:42.343 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 317, in gevent._greenlet.Greenlet._raise_exception 2020-11-28T22:11:42.343 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 47, in reraise 2020-11-28T22:11:42.344 INFO:tasks.cephfs_test_runner: raise value.with_traceback(tb) 2020-11-28T22:11:42.344 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 766, in gevent._greenlet.Greenlet.run 2020-11-28T22:11:42.345 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 316, in copy_file_to 2020-11-28T22:11:42.345 INFO:tasks.cephfs_test_runner: copy_to_log(src, logger, capture=stream) 2020-11-28T22:11:42.345 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 273, in copy_to_log 2020-11-28T22:11:42.346 INFO:tasks.cephfs_test_runner: for line in f: 2020-11-28T22:11:42.346 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ 2020-11-28T22:11:42.347 INFO:tasks.cephfs_test_runner: line = self.readline() 2020-11-28T22:11:42.347 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline 2020-11-28T22:11:42.348 INFO:tasks.cephfs_test_runner: new_data = self._read(n) 2020-11-28T22:11:42.348 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read 2020-11-28T22:11:42.349 INFO:tasks.cephfs_test_runner: return self.channel.recv_stderr(size) 2020-11-28T22:11:42.349 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr 2020-11-28T22:11:42.349 INFO:tasks.cephfs_test_runner: raise socket.timeout() 2020-11-28T22:11:42.350 INFO:tasks.cephfs_test_runner:socket.timeout 2020-11-28T22:11:42.350 INFO:tasks.cephfs_test_runner: 2020-11-28T22:11:42.351 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2020-11-28T22:11:42.351 INFO:tasks.cephfs_test_runner:Ran 12 tests in 1486.610s 2020-11-28T22:11:42.352 INFO:tasks.cephfs_test_runner: 2020-11-28T22:11:42.352 INFO:tasks.cephfs_test_runner:FAILED (errors=2) 2020-11-28T22:11:42.353 INFO:tasks.cephfs_test_runner: 2020-11-28T22:11:42.353 INFO:tasks.cephfs_test_runner:====================================================================== 2020-11-28T22:11:42.354 INFO:tasks.cephfs_test_runner:ERROR: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2020-11-28T22:11:42.354 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2020-11-28T22:11:42.354 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2020-11-28T22:11:42.355 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks/cephfs/test_misc.py", line 24, in test_statfs_on_deleted_fs 2020-11-28T22:11:42.355 INFO:tasks.cephfs_test_runner: p = self.mount_a.run_shell_payload("stat -f .", wait=False, timeout=60, check_status=False) 2020-11-28T22:11:42.356 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks/cephfs/mount.py", line 201, in run_shell_payload 2020-11-28T22:11:42.356 INFO:tasks.cephfs_test_runner: return self.run_shell(["bash", "-c", Raw(f"'{payload}'")], **kwargs) 2020-11-28T22:11:42.356 INFO:tasks.cephfs_test_runner:TypeError: run_shell() got an unexpected keyword argument 'timeout' 2020-11-28T22:11:42.357 INFO:tasks.cephfs_test_runner: 2020-11-28T22:11:42.357 INFO:tasks.cephfs_test_runner:====================================================================== 2020-11-28T22:11:42.358 INFO:tasks.cephfs_test_runner:ERROR: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2020-11-28T22:11:42.358 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2020-11-28T22:11:42.359 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2020-11-28T22:11:42.359 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr 2020-11-28T22:11:42.359 INFO:tasks.cephfs_test_runner: out = self.in_stderr_buffer.read(nbytes, self.timeout) 2020-11-28T22:11:42.360 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read 2020-11-28T22:11:42.360 INFO:tasks.cephfs_test_runner: raise PipeTimeout() 2020-11-28T22:11:42.360 INFO:tasks.cephfs_test_runner:paramiko.buffered_pipe.PipeTimeout 2020-11-28T22:11:42.361 INFO:tasks.cephfs_test_runner: 2020-11-28T22:11:42.361 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2020-11-28T22:11:42.362 INFO:tasks.cephfs_test_runner: 2020-11-28T22:11:42.362 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2020-11-28T22:11:42.362 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks/cephfs/cephfs_test_case.py", line 175, in tearDown 2020-11-28T22:11:42.363 INFO:tasks.cephfs_test_runner: m.teardown() 2020-11-28T22:11:42.363 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks/cephfs/fuse_mount.py", line 408, in teardown 2020-11-28T22:11:42.364 INFO:tasks.cephfs_test_runner: self.umount() 2020-11-28T22:11:42.364 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks/cephfs/fuse_mount.py", line 261, in umount 2020-11-28T22:11:42.365 INFO:tasks.cephfs_test_runner: if not self.is_mounted(): 2020-11-28T22:11:42.365 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted 2020-11-28T22:11:42.365 INFO:tasks.cephfs_test_runner: proc.wait() 2020-11-28T22:11:42.366 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 147, in wait 2020-11-28T22:11:42.366 INFO:tasks.cephfs_test_runner: greenlet.get(block=True,timeout=60) 2020-11-28T22:11:42.366 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 683, in gevent._greenlet.Greenlet.get 2020-11-28T22:11:42.367 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 317, in gevent._greenlet.Greenlet._raise_exception 2020-11-28T22:11:42.367 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 47, in reraise 2020-11-28T22:11:42.368 INFO:tasks.cephfs_test_runner: raise value.with_traceback(tb) 2020-11-28T22:11:42.368 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 766, in gevent._greenlet.Greenlet.run 2020-11-28T22:11:42.368 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 316, in copy_file_to 2020-11-28T22:11:42.369 INFO:tasks.cephfs_test_runner: copy_to_log(src, logger, capture=stream) 2020-11-28T22:11:42.369 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 273, in copy_to_log 2020-11-28T22:11:42.370 INFO:tasks.cephfs_test_runner: for line in f: 2020-11-28T22:11:42.370 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ 2020-11-28T22:11:42.370 INFO:tasks.cephfs_test_runner: line = self.readline() 2020-11-28T22:11:42.371 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline 2020-11-28T22:11:42.371 INFO:tasks.cephfs_test_runner: new_data = self._read(n) 2020-11-28T22:11:42.372 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read 2020-11-28T22:11:42.372 INFO:tasks.cephfs_test_runner: return self.channel.recv_stderr(size) 2020-11-28T22:11:42.372 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr 2020-11-28T22:11:42.373 INFO:tasks.cephfs_test_runner: raise socket.timeout() 2020-11-28T22:11:42.373 INFO:tasks.cephfs_test_runner:socket.timeout 2020-11-28T22:11:42.373 INFO:tasks.cephfs_test_runner: 2020-11-28T22:11:42.374 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 93, in run_tasks manager.__enter__() File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc), test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2020-11-28T22:11:42.413 ERROR:teuthology.run_tasks: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=9c73e36cc681436794e5a10204bd709b Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 93, in run_tasks manager.__enter__() File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc), test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2020-11-28T22:11:42.414 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2020-11-28T22:11:42.427 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2020-11-28T22:11:42.548 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2020-11-28T22:11:42.549 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-28T22:11:42.549 INFO:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-28T22:11:42.549 INFO:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-28T22:11:44.315 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:11:44.318 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:11:44.320 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:11:44.323 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:12:14.368 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:12:14.371 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:12:14.374 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:12:14.376 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:12:44.483 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:12:44.487 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:12:44.490 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:12:44.493 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:13:14.610 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:13:14.613 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:13:14.616 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:13:14.619 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:13:44.954 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:13:44.957 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:13:44.961 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:13:44.964 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:14:15.004 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:14:15.007 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:14:15.010 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:14:15.012 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:14:45.059 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:14:45.061 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:14:45.064 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:14:45.066 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:15:15.169 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:15:15.172 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:15:15.174 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:15:15.177 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:15:45.549 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:15:45.552 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:15:45.555 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:15:45.558 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:16:15.777 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:16:15.780 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:16:15.783 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:16:15.785 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:16:45.829 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:16:45.832 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:16:45.835 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:16:45.838 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:17:15.877 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:17:15.880 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:17:15.882 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:17:15.885 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:17:46.022 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:17:46.026 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:17:46.029 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:17:46.032 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:18:16.143 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:18:16.146 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:18:16.148 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:18:16.151 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:18:46.192 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:18:46.195 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:18:46.204 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:18:46.206 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:19:16.281 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:19:16.284 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:19:16.287 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:19:16.292 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:19:46.534 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:19:46.562 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:19:46.565 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:19:46.567 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:20:16.663 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:20:16.666 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:20:16.669 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:20:16.671 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:20:46.737 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:20:46.740 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:20:46.743 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:20:46.745 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:21:16.858 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:21:16.861 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:21:16.863 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:21:16.866 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:21:47.114 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:21:47.117 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:21:47.120 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:21:47.123 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:22:17.167 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:22:17.169 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:22:17.172 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:22:17.174 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:22:47.234 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:22:47.237 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:22:47.239 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:22:47.242 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:23:17.373 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:23:17.377 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:23:17.380 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:23:17.384 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:23:47.570 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:23:47.573 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:23:47.575 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:23:47.579 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:24:17.620 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:24:17.623 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:24:17.625 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:24:17.628 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:24:47.741 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:24:47.744 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:24:47.747 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:24:47.749 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:25:17.836 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:25:17.839 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:25:17.843 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:25:17.845 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:25:47.916 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:25:47.919 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:25:47.922 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:25:47.924 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:26:18.036 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:26:18.039 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:26:18.041 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:26:18.044 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:26:42.565 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 766, in gevent._greenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 316, in copy_file_to copy_to_log(src, logger, capture=stream) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 273, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2020-11-28T22:26:42.566 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 766, in gevent._greenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 316, in copy_file_to copy_to_log(src, logger, capture=stream) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 273, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2020-11-28T22:26:42.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T22:26:42.569 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 172, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks/ceph_fuse.py", line 159, in task if mount.is_mounted(): File "/home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted proc.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 147, in wait greenlet.get(block=True,timeout=60) File "src/gevent/greenlet.py", line 683, in gevent._greenlet.Greenlet.get File "src/gevent/greenlet.py", line 317, in gevent._greenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 47, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 766, in gevent._greenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 316, in copy_file_to copy_to_log(src, logger, capture=stream) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 273, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2020-11-28T22:26:42.569 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2020-11-28T22:26:42.577 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2020-11-28T22:26:42.578 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T22:26:42.698 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.696+0000 7f54aa516700 1 Processor -- start 2020-11-28T22:26:42.698 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.696+0000 7f54aa516700 1 -- start start 2020-11-28T22:26:42.700 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.696+0000 7f54aa516700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f54a40aca70 0x7f54a409e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:42.700 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.696+0000 7f54aa516700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f54a4060e80 0x7f54a40b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:42.700 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.696+0000 7f54aa516700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54a409d8e0 con 0x7f54a4060570 2020-11-28T22:26:42.701 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.696+0000 7f54aa516700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54a40b5610 con 0x7f54a40aca70 2020-11-28T22:26:42.701 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.696+0000 7f54aa516700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54a4060950 con 0x7f54a4060e80 2020-11-28T22:26:42.701 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.696+0000 7f54a9514700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f54a4060570 0x7f54a409e000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:53578/0 (socket says 172.21.15.162:53578) 2020-11-28T22:26:42.701 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.696+0000 7f54a9514700 1 -- 172.21.15.162:0/2619599609 learned_addr learned my addr 172.21.15.162:0/2619599609 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T22:26:42.702 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.696+0000 7f54a9d15700 1 --2- 172.21.15.162:0/2619599609 >> v2:172.21.15.162:3300/0 conn(0x7f54a4060e80 0x7f54a40b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:42.702 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.696+0000 7f54a8d13700 1 --2- 172.21.15.162:0/2619599609 >> v2:172.21.15.118:3300/0 conn(0x7f54a40aca70 0x7f54a409e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:42.702 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.696+0000 7f5497fff700 1 -- 172.21.15.162:0/2619599609 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3669464766 0 0) 0x7f54a409d8e0 con 0x7f54a4060570 2020-11-28T22:26:42.702 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.696+0000 7f5497fff700 1 -- 172.21.15.162:0/2619599609 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5488003230 con 0x7f54a4060570 2020-11-28T22:26:42.703 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.696+0000 7f5497fff700 1 -- 172.21.15.162:0/2619599609 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2850149591 0 0) 0x7f5488003230 con 0x7f54a4060570 2020-11-28T22:26:42.703 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.696+0000 7f54a9d15700 1 -- 172.21.15.162:0/2619599609 >> v2:172.21.15.118:3300/0 conn(0x7f54a40aca70 msgr2=0x7f54a409e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T22:26:42.703 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.696+0000 7f54a9d15700 1 --2- 172.21.15.162:0/2619599609 >> v2:172.21.15.118:3300/0 conn(0x7f54a40aca70 0x7f54a409e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:42.703 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.696+0000 7f54a9d15700 1 -- 172.21.15.162:0/2619599609 >> v1:172.21.15.162:6789/0 conn(0x7f54a4060570 legacy=0x7f54a409e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:42.703 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.696+0000 7f54a9d15700 1 -- 172.21.15.162:0/2619599609 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a40b8fe0 con 0x7f54a4060e80 2020-11-28T22:26:42.704 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.696+0000 7f54a8d13700 1 --2- 172.21.15.162:0/2619599609 >> v2:172.21.15.118:3300/0 conn(0x7f54a40aca70 0x7f54a409e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T22:26:42.704 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.696+0000 7f54a9d15700 1 --2- 172.21.15.162:0/2619599609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54a4060e80 0x7f54a40b5890 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 rx=0x7f5498001870 tx=0x7f54a0006dc0).ready entity=mon.1 client_cookie=a301ca8b892ca338 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:42.704 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.696+0000 7f5497fff700 1 -- 172.21.15.162:0/2619599609 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54980016d0 con 0x7f54a4060e80 2020-11-28T22:26:42.704 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.696+0000 7f5497fff700 1 -- 172.21.15.162:0/2619599609 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a000d5c0 con 0x7f54a4060e80 2020-11-28T22:26:42.705 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.696+0000 7f5497fff700 1 -- 172.21.15.162:0/2619599609 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54a000d730 con 0x7f54a4060e80 2020-11-28T22:26:42.705 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.696+0000 7f54aa516700 1 -- 172.21.15.162:0/2619599609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54a4060e80 msgr2=0x7f54a40b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:42.705 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.696+0000 7f54aa516700 1 --2- 172.21.15.162:0/2619599609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54a4060e80 0x7f54a40b5890 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 rx=0x7f5498001870 tx=0x7f54a0006dc0).stop 2020-11-28T22:26:42.708 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.696+0000 7f54aa516700 1 -- 172.21.15.162:0/2619599609 shutdown_connections 2020-11-28T22:26:42.708 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.696+0000 7f54aa516700 1 --2- 172.21.15.162:0/2619599609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54a4060e80 0x7f54a40b5890 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:42.708 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.696+0000 7f54aa516700 1 --2- 172.21.15.162:0/2619599609 >> v2:172.21.15.118:3300/0 conn(0x7f54a40aca70 0x7f54a409e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:42.709 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.696+0000 7f54aa516700 1 -- 172.21.15.162:0/2619599609 >> 172.21.15.162:0/2619599609 conn(0x7f54a4066a40 msgr2=0x7f54a4066e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:26:42.709 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.696+0000 7f54aa516700 1 -- 172.21.15.162:0/2619599609 shutdown_connections 2020-11-28T22:26:42.709 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.696+0000 7f54aa516700 1 -- 172.21.15.162:0/2619599609 wait complete. 2020-11-28T22:26:42.709 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.700+0000 7f54aa516700 1 Processor -- start 2020-11-28T22:26:42.709 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.700+0000 7f54aa516700 1 -- start start 2020-11-28T22:26:42.710 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.700+0000 7f54aa516700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f54a4060570 0x7f54a4159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:42.710 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.700+0000 7f54aa516700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f54a4060e80 0x7f54a4146dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:42.710 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.700+0000 7f54aa516700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54a40aca70 0x7f54a4098120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:42.710 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.700+0000 7f54aa516700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a40ae630 con 0x7f54a4060e80 2020-11-28T22:26:42.711 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.700+0000 7f54aa516700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a40af200 con 0x7f54a40aca70 2020-11-28T22:26:42.711 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.700+0000 7f54aa516700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54a41595c0 con 0x7f54a4060570 2020-11-28T22:26:42.711 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.700+0000 7f54a9d15700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54a40aca70 0x7f54a4098120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:42.711 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.700+0000 7f54a9d15700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54a40aca70 0x7f54a4098120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46744/0 (socket says 172.21.15.162:46744) 2020-11-28T22:26:42.711 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.700+0000 7f54a9d15700 1 -- 172.21.15.162:0/577178159 learned_addr learned my addr 172.21.15.162:0/577178159 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T22:26:42.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.700+0000 7f54a8d13700 1 --2- 172.21.15.162:0/577178159 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f54a4060e80 0x7f54a4146dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:42.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.700+0000 7f54a9514700 1 --2- 172.21.15.162:0/577178159 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f54a4060570 0x7f54a4159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:42.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.700+0000 7f54a9d15700 1 -- 172.21.15.162:0/577178159 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f54a4060570 msgr2=0x7f54a4159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:42.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.700+0000 7f54a9d15700 1 --2- 172.21.15.162:0/577178159 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f54a4060570 0x7f54a4159fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:42.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.700+0000 7f54a9d15700 1 -- 172.21.15.162:0/577178159 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f54a4060e80 msgr2=0x7f54a4146dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:42.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.700+0000 7f54a9d15700 1 --2- 172.21.15.162:0/577178159 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f54a4060e80 0x7f54a4146dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:42.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.700+0000 7f54a9d15700 1 -- 172.21.15.162:0/577178159 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a4159860 con 0x7f54a40aca70 2020-11-28T22:26:42.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.700+0000 7f54a9514700 1 --2- 172.21.15.162:0/577178159 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f54a4060570 0x7f54a4159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T22:26:42.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.700+0000 7f54a8d13700 1 --2- 172.21.15.162:0/577178159 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f54a4060e80 0x7f54a4146dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T22:26:42.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.700+0000 7f54a9d15700 1 --2- 172.21.15.162:0/577178159 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54a40aca70 0x7f54a4098120 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 rx=0x7f5498001870 tx=0x7f54a0015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:42.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.700+0000 7f54967fc700 1 -- 172.21.15.162:0/577178159 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54a0011900 con 0x7f54a40aca70 2020-11-28T22:26:42.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.700+0000 7f54967fc700 1 -- 172.21.15.162:0/577178159 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a0011a60 con 0x7f54a40aca70 2020-11-28T22:26:42.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.700+0000 7f54aa516700 1 -- 172.21.15.162:0/577178159 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54a4159b00 con 0x7f54a40aca70 2020-11-28T22:26:42.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.700+0000 7f54967fc700 1 -- 172.21.15.162:0/577178159 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54a0011dc0 con 0x7f54a40aca70 2020-11-28T22:26:42.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.700+0000 7f54aa516700 1 -- 172.21.15.162:0/577178159 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54a40adf90 con 0x7f54a40aca70 2020-11-28T22:26:42.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.704+0000 7f54967fc700 1 -- 172.21.15.162:0/577178159 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f54a0024030 con 0x7f54a40aca70 2020-11-28T22:26:42.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.704+0000 7f54aa516700 1 -- 172.21.15.162:0/577178159 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54a4057050 con 0x7f54a40aca70 2020-11-28T22:26:42.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.704+0000 7f54967fc700 1 --2- 172.21.15.162:0/577178159 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f548c055470 0x7f548c057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:42.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.704+0000 7f54967fc700 1 -- 172.21.15.162:0/577178159 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f54a006e7c0 con 0x7f54a40aca70 2020-11-28T22:26:42.716 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.704+0000 7f54a9514700 1 --2- 172.21.15.162:0/577178159 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f548c055470 0x7f548c057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:42.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.708+0000 7f54a9514700 1 --2- 172.21.15.162:0/577178159 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f548c055470 0x7f548c057930 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f549800b000 tx=0x7f5498000b80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:42.717 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.712+0000 7f54967fc700 1 -- 172.21.15.162:0/577178159 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f54a0044da0 con 0x7f54a40aca70 2020-11-28T22:26:42.853 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.852+0000 7f54aa516700 1 -- 172.21.15.162:0/577178159 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f54a409e9e0 con 0x7f54a40aca70 2020-11-28T22:26:42.855 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.852+0000 7f54967fc700 1 -- 172.21.15.162:0/577178159 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v163) v1 ==== 74+0+13418 (secure 0 0 0) 0x7f54a0022070 con 0x7f54a40aca70 2020-11-28T22:26:42.855 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T22:26:42.856 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":163,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:56:41.910010+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"172.21.15.162:6837/3753662046":"2020-11-29T21:56:38.469227+0000","172.21.15.162:6835/3753662046":"2020-11-29T21:56:38.469227+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T22:26:42.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.852+0000 7f54aa516700 1 -- 172.21.15.162:0/577178159 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f548c055470 msgr2=0x7f548c057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:42.858 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.852+0000 7f54aa516700 1 --2- 172.21.15.162:0/577178159 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f548c055470 0x7f548c057930 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f549800b000 tx=0x7f5498000b80).stop 2020-11-28T22:26:42.858 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.852+0000 7f54aa516700 1 -- 172.21.15.162:0/577178159 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54a40aca70 msgr2=0x7f54a4098120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:42.859 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.852+0000 7f54aa516700 1 --2- 172.21.15.162:0/577178159 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54a40aca70 0x7f54a4098120 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 rx=0x7f5498001870 tx=0x7f54a0015000).stop 2020-11-28T22:26:42.859 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.852+0000 7f54aa516700 1 -- 172.21.15.162:0/577178159 shutdown_connections 2020-11-28T22:26:42.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.852+0000 7f54aa516700 1 --2- 172.21.15.162:0/577178159 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f548c055470 0x7f548c057930 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:42.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.852+0000 7f54aa516700 1 --2- 172.21.15.162:0/577178159 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f54a4060570 0x7f54a4159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:42.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.852+0000 7f54aa516700 1 --2- 172.21.15.162:0/577178159 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54a40aca70 0x7f54a4098120 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:42.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.852+0000 7f54aa516700 1 --2- 172.21.15.162:0/577178159 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f54a4060e80 0x7f54a4146dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:42.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.852+0000 7f54aa516700 1 -- 172.21.15.162:0/577178159 >> 172.21.15.162:0/577178159 conn(0x7f54a4066a40 msgr2=0x7f54a4066e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:26:42.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.852+0000 7f54aa516700 1 -- 172.21.15.162:0/577178159 shutdown_connections 2020-11-28T22:26:42.862 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.852+0000 7f54aa516700 1 -- 172.21.15.162:0/577178159 wait complete. 2020-11-28T22:26:42.869 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-11-28T22:26:42.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.980+0000 7fe6cccd3700 1 Processor -- start 2020-11-28T22:26:42.981 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.980+0000 7fe6cccd3700 1 -- start start 2020-11-28T22:26:42.982 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.980+0000 7fe6cccd3700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe6c80603d0 0x7fe6c80b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:42.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.980+0000 7fe6cccd3700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe6c80b4c70 0x7fe6c8143020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:42.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.980+0000 7fe6cccd3700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6c80b4110 con 0x7fe6c8142a00 2020-11-28T22:26:42.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.980+0000 7fe6cccd3700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6c8060f60 con 0x7fe6c80b4c70 2020-11-28T22:26:42.983 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.980+0000 7fe6cccd3700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6c80b43f0 con 0x7fe6c80603d0 2020-11-28T22:26:42.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.980+0000 7fe6c6ffd700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fe6c8142a00 0x7fe6c809cfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:53592/0 (socket says 172.21.15.162:53592) 2020-11-28T22:26:42.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.980+0000 7fe6c6ffd700 1 -- 172.21.15.162:0/2368540520 learned_addr learned my addr 172.21.15.162:0/2368540520 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T22:26:42.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.980+0000 7fe6c77fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe6c80603d0 0x7fe6c80b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:42.984 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.980+0000 7fe6c7fff700 1 --2- 172.21.15.162:0/2368540520 >> v2:172.21.15.118:3300/0 conn(0x7fe6c80b4c70 0x7fe6c8143020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:42.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.980+0000 7fe6c67fc700 1 -- 172.21.15.162:0/2368540520 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4067823537 0 0) 0x7fe6c80b4110 con 0x7fe6c8142a00 2020-11-28T22:26:42.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.980+0000 7fe6c67fc700 1 -- 172.21.15.162:0/2368540520 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6a8003230 con 0x7fe6c8142a00 2020-11-28T22:26:42.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.980+0000 7fe6c77fe700 1 -- 172.21.15.162:0/2368540520 >> v2:172.21.15.118:3300/0 conn(0x7fe6c80b4c70 msgr2=0x7fe6c8143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T22:26:42.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.980+0000 7fe6c77fe700 1 --2- 172.21.15.162:0/2368540520 >> v2:172.21.15.118:3300/0 conn(0x7fe6c80b4c70 0x7fe6c8143020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:42.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.980+0000 7fe6c77fe700 1 -- 172.21.15.162:0/2368540520 >> v1:172.21.15.162:6789/0 conn(0x7fe6c8142a00 legacy=0x7fe6c809cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:42.987 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.980+0000 7fe6c77fe700 1 -- 172.21.15.162:0/2368540520 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c80b4690 con 0x7fe6c80603d0 2020-11-28T22:26:42.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.980+0000 7fe6c7fff700 1 --2- 172.21.15.162:0/2368540520 >> v2:172.21.15.118:3300/0 conn(0x7fe6c80b4c70 0x7fe6c8143020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T22:26:42.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.980+0000 7fe6c77fe700 1 --2- 172.21.15.162:0/2368540520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe6c80603d0 0x7fe6c80b3a70 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 rx=0x7fe6b8002f30 tx=0x7fe6b8006dc0).ready entity=mon.1 client_cookie=9c0b2cced32e1ea server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:42.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.980+0000 7fe6c67fc700 1 -- 172.21.15.162:0/2368540520 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6b80097b0 con 0x7fe6c80603d0 2020-11-28T22:26:42.988 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.980+0000 7fe6c67fc700 1 -- 172.21.15.162:0/2368540520 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b8009910 con 0x7fe6c80603d0 2020-11-28T22:26:42.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.980+0000 7fe6c67fc700 1 -- 172.21.15.162:0/2368540520 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6b8009c70 con 0x7fe6c80603d0 2020-11-28T22:26:42.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.980+0000 7fe6cccd3700 1 -- 172.21.15.162:0/2368540520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe6c80603d0 msgr2=0x7fe6c80b3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:42.989 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.980+0000 7fe6cccd3700 1 --2- 172.21.15.162:0/2368540520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe6c80603d0 0x7fe6c80b3a70 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 rx=0x7fe6b8002f30 tx=0x7fe6b8006dc0).stop 2020-11-28T22:26:42.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.980+0000 7fe6cccd3700 1 -- 172.21.15.162:0/2368540520 shutdown_connections 2020-11-28T22:26:42.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.980+0000 7fe6cccd3700 1 --2- 172.21.15.162:0/2368540520 >> v2:172.21.15.118:3300/0 conn(0x7fe6c80b4c70 0x7fe6c8143020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:42.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.980+0000 7fe6cccd3700 1 --2- 172.21.15.162:0/2368540520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe6c80603d0 0x7fe6c80b3a70 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:42.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.980+0000 7fe6cccd3700 1 -- 172.21.15.162:0/2368540520 >> 172.21.15.162:0/2368540520 conn(0x7fe6c8066a40 msgr2=0x7fe6c8066e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:26:42.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.984+0000 7fe6cccd3700 1 -- 172.21.15.162:0/2368540520 shutdown_connections 2020-11-28T22:26:42.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.984+0000 7fe6cccd3700 1 -- 172.21.15.162:0/2368540520 wait complete. 2020-11-28T22:26:42.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.984+0000 7fe6cccd3700 1 Processor -- start 2020-11-28T22:26:42.991 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.984+0000 7fe6cccd3700 1 -- start start 2020-11-28T22:26:42.994 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.984+0000 7fe6cccd3700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe6c80603d0 0x7fe6c8144340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:42.994 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.984+0000 7fe6cccd3700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe6c80b4c70 0x7fe6c80b7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:42.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.984+0000 7fe6cccd3700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe6c80aec50 0x7fe6c80b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:42.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.984+0000 7fe6cccd3700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c8144160 con 0x7fe6c80aec50 2020-11-28T22:26:42.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.984+0000 7fe6cccd3700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c80affb0 con 0x7fe6c80603d0 2020-11-28T22:26:42.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.984+0000 7fe6cccd3700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c8143f00 con 0x7fe6c80b4c70 2020-11-28T22:26:42.996 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.984+0000 7fe6c77fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe6c80603d0 0x7fe6c8144340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:42.996 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.984+0000 7fe6c77fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe6c80603d0 0x7fe6c8144340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46754/0 (socket says 172.21.15.162:46754) 2020-11-28T22:26:42.996 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.984+0000 7fe6c77fe700 1 -- 172.21.15.162:0/2604810934 learned_addr learned my addr 172.21.15.162:0/2604810934 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T22:26:42.996 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.984+0000 7fe6c6ffd700 1 --2- 172.21.15.162:0/2604810934 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe6c80b4c70 0x7fe6c80b7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:42.996 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.984+0000 7fe6c7fff700 1 --2- 172.21.15.162:0/2604810934 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe6c80aec50 0x7fe6c80b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:42.997 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.984+0000 7fe6c77fe700 1 -- 172.21.15.162:0/2604810934 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe6c80b4c70 msgr2=0x7fe6c80b7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:42.997 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.984+0000 7fe6c77fe700 1 --2- 172.21.15.162:0/2604810934 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe6c80b4c70 0x7fe6c80b7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:42.997 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.984+0000 7fe6c77fe700 1 -- 172.21.15.162:0/2604810934 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe6c80aec50 msgr2=0x7fe6c80b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:42.997 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.984+0000 7fe6c77fe700 1 --2- 172.21.15.162:0/2604810934 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe6c80aec50 0x7fe6c80b13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:42.998 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.984+0000 7fe6c77fe700 1 -- 172.21.15.162:0/2604810934 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c80ae550 con 0x7fe6c80603d0 2020-11-28T22:26:42.998 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.984+0000 7fe6c77fe700 1 --2- 172.21.15.162:0/2604810934 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe6c80603d0 0x7fe6c8144340 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 rx=0x7fe6b8002f30 tx=0x7fe6b8015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:42.998 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.988+0000 7fe6c4ff9700 1 -- 172.21.15.162:0/2604810934 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6b8011960 con 0x7fe6c80603d0 2020-11-28T22:26:42.998 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.988+0000 7fe6c4ff9700 1 -- 172.21.15.162:0/2604810934 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b8011ac0 con 0x7fe6c80603d0 2020-11-28T22:26:42.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.988+0000 7fe6cccd3700 1 -- 172.21.15.162:0/2604810934 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6c80ae7b0 con 0x7fe6c80603d0 2020-11-28T22:26:42.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.988+0000 7fe6c4ff9700 1 -- 172.21.15.162:0/2604810934 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6b8011e20 con 0x7fe6c80603d0 2020-11-28T22:26:42.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.988+0000 7fe6cccd3700 1 -- 172.21.15.162:0/2604810934 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6c80ae0e0 con 0x7fe6c80603d0 2020-11-28T22:26:42.999 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.988+0000 7fe6cccd3700 1 -- 172.21.15.162:0/2604810934 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6c80ae3b0 con 0x7fe6c80603d0 2020-11-28T22:26:43.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.988+0000 7fe6c4ff9700 1 -- 172.21.15.162:0/2604810934 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fe6b8024030 con 0x7fe6c80603d0 2020-11-28T22:26:43.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.988+0000 7fe6c4ff9700 1 --2- 172.21.15.162:0/2604810934 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe6ac0553b0 0x7fe6ac057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:43.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.988+0000 7fe6c4ff9700 1 -- 172.21.15.162:0/2604810934 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5339+0+0 (secure 0 0 0) 0x7fe6b806e8d0 con 0x7fe6c80603d0 2020-11-28T22:26:43.000 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.988+0000 7fe6c6ffd700 1 --2- 172.21.15.162:0/2604810934 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe6ac0553b0 0x7fe6ac057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:43.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.988+0000 7fe6c6ffd700 1 --2- 172.21.15.162:0/2604810934 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe6ac0553b0 0x7fe6ac057870 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7fe6b000d000 tx=0x7fe6b0009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:43.001 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:42.992+0000 7fe6c4ff9700 1 -- 172.21.15.162:0/2604810934 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fe6c80ae3b0 con 0x7fe6c80603d0 2020-11-28T22:26:43.131 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.128+0000 7fe6cccd3700 1 -- 172.21.15.162:0/2604810934 --> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe6c80ae3b0 con 0x7fe6ac0553b0 2020-11-28T22:26:43.139 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.136+0000 7fe6c4ff9700 1 -- 172.21.15.162:0/2604810934 <== mgr.4105 v2:172.21.15.162:6800/12156 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+66484 (secure 0 0 0) 0x7fe6c80ae3b0 con 0x7fe6ac0553b0 2020-11-28T22:26:43.140 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T22:26:43.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.136+0000 7fe6cccd3700 1 -- 172.21.15.162:0/2604810934 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe6ac0553b0 msgr2=0x7fe6ac057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:43.142 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.136+0000 7fe6cccd3700 1 --2- 172.21.15.162:0/2604810934 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe6ac0553b0 0x7fe6ac057870 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7fe6b000d000 tx=0x7fe6b0009040).stop 2020-11-28T22:26:43.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.136+0000 7fe6cccd3700 1 -- 172.21.15.162:0/2604810934 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe6c80603d0 msgr2=0x7fe6c8144340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:43.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.136+0000 7fe6cccd3700 1 --2- 172.21.15.162:0/2604810934 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe6c80603d0 0x7fe6c8144340 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 rx=0x7fe6b8002f30 tx=0x7fe6b8015000).stop 2020-11-28T22:26:43.143 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.136+0000 7fe6cccd3700 1 -- 172.21.15.162:0/2604810934 shutdown_connections 2020-11-28T22:26:43.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.136+0000 7fe6cccd3700 1 --2- 172.21.15.162:0/2604810934 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fe6ac0553b0 0x7fe6ac057870 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:43.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.136+0000 7fe6cccd3700 1 --2- 172.21.15.162:0/2604810934 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe6c80aec50 0x7fe6c80b13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:43.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.136+0000 7fe6cccd3700 1 --2- 172.21.15.162:0/2604810934 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe6c80603d0 0x7fe6c8144340 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:43.144 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.136+0000 7fe6cccd3700 1 --2- 172.21.15.162:0/2604810934 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe6c80b4c70 0x7fe6c80b7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:43.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.136+0000 7fe6cccd3700 1 -- 172.21.15.162:0/2604810934 >> 172.21.15.162:0/2604810934 conn(0x7fe6c8066a40 msgr2=0x7fe6c8142df0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:26:43.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.136+0000 7fe6cccd3700 1 -- 172.21.15.162:0/2604810934 shutdown_connections 2020-11-28T22:26:43.145 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.136+0000 7fe6cccd3700 1 -- 172.21.15.162:0/2604810934 wait complete. 2020-11-28T22:26:43.145 INFO:teuthology.orchestra.run.smithi162.stderr:dumped all 2020-11-28T22:26:43.153 INFO:teuthology.orchestra.run.smithi162.stdout:{"pg_ready":true,"pg_map":{"version":1367,"stamp":"2020-11-28T22:26:42.289996+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8404860,"kb_used_data":16188,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746569860,"statfs":{"total":773094113280,"available":764487536640,"internally_reserved":8589934592,"allocated":16576512,"data_stored":7671573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003802"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":"130","reported_epoch":"162","state":"active+clean","last_fresh":"2020-11-28T21:56:41.031775+0000","last_change":"2020-11-28T21:46:25.652727+0000","last_active":"2020-11-28T21:56:41.031775+0000","last_peered":"2020-11-28T21:56:41.031775+0000","last_clean":"2020-11-28T21:56:41.031775+0000","last_became_active":"2020-11-28T21:46:25.652265+0000","last_became_peered":"2020-11-28T21:46:25.652265+0000","last_unstale":"2020-11-28T21:56:41.031775+0000","last_undegraded":"2020-11-28T21:56:41.031775+0000","last_fullsized":"2020-11-28T21:56:41.031775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_clean_scrub_stamp":"2020-11-28T21:46:24.622452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"133","reported_epoch":"162","state":"active+clean","last_fresh":"2020-11-28T21:56:40.913441+0000","last_change":"2020-11-28T21:46:25.654101+0000","last_active":"2020-11-28T21:56:40.913441+0000","last_peered":"2020-11-28T21:56:40.913441+0000","last_clean":"2020-11-28T21:56:40.913441+0000","last_became_active":"2020-11-28T21:46:25.653417+0000","last_became_peered":"2020-11-28T21:46:25.653417+0000","last_unstale":"2020-11-28T21:56:40.913441+0000","last_undegraded":"2020-11-28T21:56:40.913441+0000","last_fullsized":"2020-11-28T21:56:40.913441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_clean_scrub_stamp":"2020-11-28T21:46:24.622452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"135","reported_epoch":"162","state":"active+clean","last_fresh":"2020-11-28T21:56:41.322109+0000","last_change":"2020-11-28T21:46:25.652888+0000","last_active":"2020-11-28T21:56:41.322109+0000","last_peered":"2020-11-28T21:56:41.322109+0000","last_clean":"2020-11-28T21:56:41.322109+0000","last_became_active":"2020-11-28T21:46:25.652295+0000","last_became_peered":"2020-11-28T21:46:25.652295+0000","last_unstale":"2020-11-28T21:56:41.322109+0000","last_undegraded":"2020-11-28T21:56:41.322109+0000","last_fullsized":"2020-11-28T21:56:41.322109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_clean_scrub_stamp":"2020-11-28T21:46:24.622452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"133","reported_epoch":"162","state":"active+clean","last_fresh":"2020-11-28T21:56:40.913442+0000","last_change":"2020-11-28T21:46:26.008971+0000","last_active":"2020-11-28T21:56:40.913442+0000","last_peered":"2020-11-28T21:56:40.913442+0000","last_clean":"2020-11-28T21:56:40.913442+0000","last_became_active":"2020-11-28T21:46:26.007744+0000","last_became_peered":"2020-11-28T21:46:26.007744+0000","last_unstale":"2020-11-28T21:56:40.913442+0000","last_undegraded":"2020-11-28T21:56:40.913442+0000","last_fullsized":"2020-11-28T21:56:40.913442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_clean_scrub_stamp":"2020-11-28T21:46:24.622452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"136","reported_epoch":"162","state":"active+clean","last_fresh":"2020-11-28T21:56:41.096619+0000","last_change":"2020-11-28T21:46:20.941819+0000","last_active":"2020-11-28T21:56:41.096619+0000","last_peered":"2020-11-28T21:56:41.096619+0000","last_clean":"2020-11-28T21:56:41.096619+0000","last_became_active":"2020-11-28T21:46:20.941467+0000","last_became_peered":"2020-11-28T21:46:20.941467+0000","last_unstale":"2020-11-28T21:56:41.096619+0000","last_undegraded":"2020-11-28T21:56:41.096619+0000","last_fullsized":"2020-11-28T21:56:41.096619+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:16.570066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:16.570066+0000","last_clean_scrub_stamp":"2020-11-28T21:46:16.570066+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"133","reported_epoch":"162","state":"active+clean","last_fresh":"2020-11-28T21:56:40.913557+0000","last_change":"2020-11-28T21:46:25.654099+0000","last_active":"2020-11-28T21:56:40.913557+0000","last_peered":"2020-11-28T21:56:40.913557+0000","last_clean":"2020-11-28T21:56:40.913557+0000","last_became_active":"2020-11-28T21:46:25.653652+0000","last_became_peered":"2020-11-28T21:46:25.653652+0000","last_unstale":"2020-11-28T21:56:40.913557+0000","last_undegraded":"2020-11-28T21:56:40.913557+0000","last_fullsized":"2020-11-28T21:56:40.913557+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_clean_scrub_stamp":"2020-11-28T21:46:24.622452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"133","reported_epoch":"162","state":"active+clean","last_fresh":"2020-11-28T21:56:40.914051+0000","last_change":"2020-11-28T21:46:25.653672+0000","last_active":"2020-11-28T21:56:40.914051+0000","last_peered":"2020-11-28T21:56:40.914051+0000","last_clean":"2020-11-28T21:56:40.914051+0000","last_became_active":"2020-11-28T21:46:25.653222+0000","last_became_peered":"2020-11-28T21:46:25.653222+0000","last_unstale":"2020-11-28T21:56:40.914051+0000","last_undegraded":"2020-11-28T21:56:40.914051+0000","last_fullsized":"2020-11-28T21:56:40.914051+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_clean_scrub_stamp":"2020-11-28T21:46:24.622452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"140","reported_epoch":"162","state":"active+clean","last_fresh":"2020-11-28T21:56:40.914032+0000","last_change":"2020-11-28T21:46:25.838546+0000","last_active":"2020-11-28T21:56:40.914032+0000","last_peered":"2020-11-28T21:56:40.914032+0000","last_clean":"2020-11-28T21:56:40.914032+0000","last_became_active":"2020-11-28T21:46:25.837556+0000","last_became_peered":"2020-11-28T21:46:25.837556+0000","last_unstale":"2020-11-28T21:56:40.914032+0000","last_undegraded":"2020-11-28T21:56:40.914032+0000","last_fullsized":"2020-11-28T21:56:40.914032+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_clean_scrub_stamp":"2020-11-28T21:46:24.622452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"133","reported_epoch":"162","state":"active+clean","last_fresh":"2020-11-28T21:56:40.913881+0000","last_change":"2020-11-28T21:46:25.838687+0000","last_active":"2020-11-28T21:56:40.913881+0000","last_peered":"2020-11-28T21:56:40.913881+0000","last_clean":"2020-11-28T21:56:40.913881+0000","last_became_active":"2020-11-28T21:46:25.837943+0000","last_became_peered":"2020-11-28T21:46:25.837943+0000","last_unstale":"2020-11-28T21:56:40.913881+0000","last_undegraded":"2020-11-28T21:56:40.913881+0000","last_fullsized":"2020-11-28T21:56:40.913881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_clean_scrub_stamp":"2020-11-28T21:46:24.622452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542630,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050608,"kb_used_data":2024,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321232,"statfs":{"total":96636764160,"available":95560941568,"internally_reserved":1073741824,"allocated":2072576,"data_stored":959022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 28 22:26:34 2020","interfaces":[{"interface":"back","average":{"1min":1.847,"5min":1.6910000000000001,"15min":1.716},"min":{"1min":1.2729999999999999,"5min":1.081,"15min":0.98699999999999999},"max":{"1min":2.9769999999999999,"5min":3.609,"15min":3.609},"last":1.8500000000000001},{"interface":"front","average":{"1min":1.7849999999999999,"5min":1.8400000000000001,"15min":1.8080000000000001},"min":{"1min":1.429,"5min":1.2470000000000001,"15min":1.1930000000000001},"max":{"1min":2.52,"5min":3.6589999999999998,"15min":3.6589999999999998},"last":2.0880000000000001}]},{"osd":1,"last update":"Sat Nov 28 22:26:34 2020","interfaces":[{"interface":"back","average":{"1min":1.661,"5min":1.8240000000000001,"15min":1.8320000000000001},"min":{"1min":1.298,"5min":1.2490000000000001,"15min":1.0840000000000001},"max":{"1min":2.8759999999999999,"5min":3.6139999999999999,"15min":3.6139999999999999},"last":1.4019999999999999},{"interface":"front","average":{"1min":2.2010000000000001,"5min":2.1909999999999998,"15min":2.206},"min":{"1min":1.601,"5min":1.3700000000000001,"15min":1.1779999999999999},"max":{"1min":3.141,"5min":3.4249999999999998,"15min":3.4550000000000001},"last":1.7150000000000001}]},{"osd":2,"last update":"Sat Nov 28 22:26:34 2020","interfaces":[{"interface":"back","average":{"1min":2.2650000000000001,"5min":2.2629999999999999,"15min":2.194},"min":{"1min":1.373,"5min":1.373,"15min":1.3220000000000001},"max":{"1min":3.2690000000000001,"5min":4.0439999999999996,"15min":4.0439999999999996},"last":2.524},{"interface":"front","average":{"1min":2.2389999999999999,"5min":2.1720000000000002,"15min":2.1429999999999998},"min":{"1min":1.7609999999999999,"5min":1.429,"15min":1.3109999999999999},"max":{"1min":3.012,"5min":3.5289999999999999,"15min":3.5289999999999999},"last":1.7869999999999999}]},{"osd":3,"last update":"Sat Nov 28 22:26:34 2020","interfaces":[{"interface":"back","average":{"1min":2.4769999999999999,"5min":2.46,"15min":2.4399999999999999},"min":{"1min":1.778,"5min":1.6200000000000001,"15min":1.3160000000000001},"max":{"1min":3.6859999999999999,"5min":4.3819999999999997,"15min":4.3819999999999997},"last":2.8039999999999998},{"interface":"front","average":{"1min":2.3290000000000002,"5min":2.4550000000000001,"15min":2.4169999999999998},"min":{"1min":1.7609999999999999,"5min":1.603,"15min":1.4770000000000001},"max":{"1min":2.996,"5min":3.867,"15min":3.867},"last":2.1539999999999999}]},{"osd":4,"last update":"Sat Nov 28 22:26:34 2020","interfaces":[{"interface":"back","average":{"1min":2.5339999999999998,"5min":2.5459999999999998,"15min":2.5129999999999999},"min":{"1min":1.8160000000000001,"5min":1.452,"15min":1.3009999999999999},"max":{"1min":3.5699999999999998,"5min":3.806,"15min":3.806},"last":2.9620000000000002},{"interface":"front","average":{"1min":2.7149999999999999,"5min":2.7789999999999999,"15min":2.7530000000000001},"min":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.577},"max":{"1min":3.9849999999999999,"5min":4.4260000000000002,"15min":4.4260000000000002},"last":2.2210000000000001}]},{"osd":5,"last update":"Sat Nov 28 22:26:34 2020","interfaces":[{"interface":"back","average":{"1min":2.6099999999999999,"5min":2.7149999999999999,"15min":2.6989999999999998},"min":{"1min":1.8600000000000001,"5min":1.835,"15min":1.659},"max":{"1min":3.359,"5min":4.2850000000000001,"15min":4.2850000000000001},"last":2.593},{"interface":"front","average":{"1min":2.887,"5min":2.9279999999999999,"15min":2.8969999999999998},"min":{"1min":2.1800000000000002,"5min":2.0019999999999998,"15min":1.345},"max":{"1min":3.7629999999999999,"5min":4.1779999999999999,"15min":4.1779999999999999},"last":2.7330000000000001}]},{"osd":6,"last update":"Sat Nov 28 22:26:34 2020","interfaces":[{"interface":"back","average":{"1min":3.016,"5min":3.0390000000000001,"15min":3.085},"min":{"1min":2.4780000000000002,"5min":1.976,"15min":1.976},"max":{"1min":3.9510000000000001,"5min":4.5629999999999997,"15min":4.5629999999999997},"last":2.4409999999999998},{"interface":"front","average":{"1min":3.1829999999999998,"5min":3.137,"15min":3.1520000000000001},"min":{"1min":2.5950000000000002,"5min":1.804,"15min":1.804},"max":{"1min":4.194,"5min":4.5010000000000003,"15min":4.5010000000000003},"last":3.004}]}]},{"osd":6,"up_from":14,"seq":60129542630,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050608,"kb_used_data":2024,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321232,"statfs":{"total":96636764160,"available":95560941568,"internally_reserved":1073741824,"allocated":2072576,"data_stored":959022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 28 22:26:30 2020","interfaces":[{"interface":"back","average":{"1min":1.6479999999999999,"5min":1.591,"15min":1.645},"min":{"1min":1.3240000000000001,"5min":1.1919999999999999,"15min":1.1060000000000001},"max":{"1min":2.0449999999999999,"5min":2.8069999999999999,"15min":2.8069999999999999},"last":1.7829999999999999},{"interface":"front","average":{"1min":1.7749999999999999,"5min":1.768,"15min":1.7829999999999999},"min":{"1min":1.403,"5min":1.087,"15min":1.087},"max":{"1min":3.3010000000000002,"5min":3.536,"15min":3.9060000000000001},"last":1.5669999999999999}]},{"osd":1,"last update":"Sat Nov 28 22:26:30 2020","interfaces":[{"interface":"back","average":{"1min":2.2480000000000002,"5min":2.2109999999999999,"15min":2.2410000000000001},"min":{"1min":1.5249999999999999,"5min":1.264,"15min":1.264},"max":{"1min":3.3220000000000001,"5min":4.4989999999999997,"15min":4.4989999999999997},"last":1.6919999999999999},{"interface":"front","average":{"1min":2.3610000000000002,"5min":2.3439999999999999,"15min":2.395},"min":{"1min":1.3979999999999999,"5min":0.996,"15min":0.996},"max":{"1min":2.754,"5min":3.1360000000000001,"15min":3.2789999999999999},"last":2.5939999999999999}]},{"osd":2,"last update":"Sat Nov 28 22:26:30 2020","interfaces":[{"interface":"back","average":{"1min":2.024,"5min":2.1240000000000001,"15min":2.1339999999999999},"min":{"1min":1.5609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"max":{"1min":3.1789999999999998,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":1.9239999999999999},{"interface":"front","average":{"1min":2.512,"5min":2.3199999999999998,"15min":2.355},"min":{"1min":1.694,"5min":1.355,"15min":1.103},"max":{"1min":3.899,"5min":4.3680000000000003,"15min":4.3680000000000003},"last":2.8069999999999999}]},{"osd":3,"last update":"Sat Nov 28 22:26:30 2020","interfaces":[{"interface":"back","average":{"1min":2.5529999999999999,"5min":2.573,"15min":2.6040000000000001},"min":{"1min":1.8360000000000001,"5min":1.6100000000000001,"15min":1.5109999999999999},"max":{"1min":3.5059999999999998,"5min":4.7329999999999997,"15min":4.7329999999999997},"last":2.4239999999999999},{"interface":"front","average":{"1min":2.4769999999999999,"5min":2.4489999999999998,"15min":2.4510000000000001},"min":{"1min":1.9299999999999999,"5min":1.665,"15min":1.3720000000000001},"max":{"1min":3.46,"5min":4.2290000000000001,"15min":4.2290000000000001},"last":2.1779999999999999}]},{"osd":4,"last update":"Sat Nov 28 22:26:30 2020","interfaces":[{"interface":"back","average":{"1min":2.5009999999999999,"5min":2.3980000000000001,"15min":2.4199999999999999},"min":{"1min":2.0219999999999998,"5min":1.6759999999999999,"15min":1.121},"max":{"1min":3.3559999999999999,"5min":3.734,"15min":3.734},"last":2.3380000000000001},{"interface":"front","average":{"1min":2.6219999999999999,"5min":2.8260000000000001,"15min":2.831},"min":{"1min":1.752,"5min":1.218,"15min":1.218},"max":{"1min":4.1079999999999997,"5min":5.5869999999999997,"15min":5.5869999999999997},"last":2.016}]},{"osd":5,"last update":"Sat Nov 28 22:26:30 2020","interfaces":[{"interface":"back","average":{"1min":2.9289999999999998,"5min":2.847,"15min":2.8370000000000002},"min":{"1min":2.2989999999999999,"5min":1.758,"15min":1.6839999999999999},"max":{"1min":3.7000000000000002,"5min":5.2210000000000001,"15min":5.2210000000000001},"last":2.899},{"interface":"front","average":{"1min":2.633,"5min":2.7629999999999999,"15min":2.8260000000000001},"min":{"1min":1.76,"5min":1.76,"15min":1.47},"max":{"1min":3.597,"5min":4.6760000000000002,"15min":5.3120000000000003},"last":2.125}]},{"osd":7,"last update":"Sat Nov 28 22:26:30 2020","interfaces":[{"interface":"back","average":{"1min":3.0070000000000001,"5min":3.141,"15min":3.173},"min":{"1min":2.0859999999999999,"5min":1.8460000000000001,"15min":1.7729999999999999},"max":{"1min":4.3300000000000001,"5min":5.298,"15min":5.3620000000000001},"last":3.1099999999999999},{"interface":"front","average":{"1min":3.0640000000000001,"5min":3.1480000000000001,"15min":3.1560000000000001},"min":{"1min":2.5950000000000002,"5min":2.1960000000000002,"15min":1.6930000000000001},"max":{"1min":3.8700000000000001,"5min":5.4740000000000002,"15min":5.4740000000000002},"last":2.5310000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542631,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050608,"kb_used_data":2024,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321232,"statfs":{"total":96636764160,"available":95560941568,"internally_reserved":1073741824,"allocated":2072576,"data_stored":959022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 28 22:25:42 2020","interfaces":[{"interface":"back","average":{"1min":1.665,"5min":1.5940000000000001,"15min":1.601},"min":{"1min":1.319,"5min":1.2509999999999999,"15min":1.2290000000000001},"max":{"1min":2.3570000000000002,"5min":2.3570000000000002,"15min":2.802},"last":1.385},{"interface":"front","average":{"1min":1.9390000000000001,"5min":1.839,"15min":1.7749999999999999},"min":{"1min":1.363,"5min":1.3340000000000001,"15min":1.127},"max":{"1min":2.5299999999999998,"5min":3.8740000000000001,"15min":3.8740000000000001},"last":1.954}]},{"osd":2,"last update":"Sat Nov 28 22:25:42 2020","interfaces":[{"interface":"back","average":{"1min":1.7969999999999999,"5min":1.8360000000000001,"15min":1.8300000000000001},"min":{"1min":1.302,"5min":1.302,"15min":1.0620000000000001},"max":{"1min":2.6520000000000001,"5min":4.1929999999999996,"15min":4.1929999999999996},"last":3.2629999999999999},{"interface":"front","average":{"1min":2.3279999999999998,"5min":2.3380000000000001,"15min":2.3460000000000001},"min":{"1min":1.8360000000000001,"5min":1.7050000000000001,"15min":1.3169999999999999},"max":{"1min":2.7480000000000002,"5min":3.2799999999999998,"15min":3.3050000000000002},"last":3.0089999999999999}]},{"osd":3,"last update":"Sat Nov 28 22:25:42 2020","interfaces":[{"interface":"back","average":{"1min":2.214,"5min":2.1360000000000001,"15min":2.1080000000000001},"min":{"1min":1.575,"5min":1.575,"15min":1.2210000000000001},"max":{"1min":3.2160000000000002,"5min":3.2160000000000002,"15min":3.2160000000000002},"last":2.0619999999999998},{"interface":"front","average":{"1min":2.371,"5min":2.4460000000000002,"15min":2.4060000000000001},"min":{"1min":1.694,"5min":1.6619999999999999,"15min":1.5189999999999999},"max":{"1min":3.1600000000000001,"5min":3.5680000000000001,"15min":3.5680000000000001},"last":1.5609999999999999}]},{"osd":4,"last update":"Sat Nov 28 22:25:42 2020","interfaces":[{"interface":"back","average":{"1min":2.4449999999999998,"5min":2.347,"15min":2.391},"min":{"1min":1.591,"5min":1.542,"15min":1.5389999999999999},"max":{"1min":3.6469999999999998,"5min":3.6680000000000001,"15min":3.73},"last":2.2959999999999998},{"interface":"front","average":{"1min":2.2290000000000001,"5min":2.335,"15min":2.367},"min":{"1min":1.8009999999999999,"5min":1.554,"15min":1.554},"max":{"1min":2.7090000000000001,"5min":3.802,"15min":3.802},"last":1.8799999999999999}]},{"osd":5,"last update":"Sat Nov 28 22:25:42 2020","interfaces":[{"interface":"back","average":{"1min":2.831,"5min":2.722,"15min":2.7200000000000002},"min":{"1min":2.1120000000000001,"5min":2,"15min":1.573},"max":{"1min":4.0279999999999996,"5min":4.0279999999999996,"15min":4.0279999999999996},"last":2.8780000000000001},{"interface":"front","average":{"1min":2.5760000000000001,"5min":2.7000000000000002,"15min":2.7410000000000001},"min":{"1min":1.95,"5min":1.95,"15min":1.635},"max":{"1min":3.2120000000000002,"5min":3.891,"15min":4.1980000000000004},"last":2.8039999999999998}]},{"osd":6,"last update":"Sat Nov 28 22:25:42 2020","interfaces":[{"interface":"back","average":{"1min":2.9199999999999999,"5min":2.9940000000000002,"15min":2.9849999999999999},"min":{"1min":2.2509999999999999,"5min":1.9570000000000001,"15min":1.9570000000000001},"max":{"1min":3.7290000000000001,"5min":4.5119999999999996,"15min":4.5119999999999996},"last":2.3650000000000002},{"interface":"front","average":{"1min":2.9089999999999998,"5min":2.8599999999999999,"15min":2.8359999999999999},"min":{"1min":2.3940000000000001,"5min":2.3340000000000001,"15min":2.0590000000000002},"max":{"1min":3.415,"5min":4.266,"15min":4.266},"last":2.6789999999999998}]},{"osd":7,"last update":"Sat Nov 28 22:25:42 2020","interfaces":[{"interface":"back","average":{"1min":3.1930000000000001,"5min":3.117,"15min":3.1030000000000002},"min":{"1min":2.5859999999999999,"5min":2.3860000000000001,"15min":2.2770000000000001},"max":{"1min":4.407,"5min":4.407,"15min":4.407},"last":2.5169999999999999},{"interface":"front","average":{"1min":3.0470000000000002,"5min":3.2080000000000002,"15min":3.1890000000000001},"min":{"1min":2.4350000000000001,"5min":2.3580000000000001,"15min":2.3580000000000001},"max":{"1min":3.581,"5min":5.3899999999999997,"15min":5.3899999999999997},"last":3.0699999999999998}]}]},{"osd":0,"up_from":14,"seq":60129542630,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050608,"kb_used_data":2024,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321232,"statfs":{"total":96636764160,"available":95560941568,"internally_reserved":1073741824,"allocated":2072576,"data_stored":959022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Nov 28 22:26:34 2020","interfaces":[{"interface":"back","average":{"1min":1.6100000000000001,"5min":1.7250000000000001,"15min":1.696},"min":{"1min":1.339,"5min":1.244,"15min":1.2170000000000001},"max":{"1min":2.3399999999999999,"5min":3.2120000000000002,"15min":3.3180000000000001},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.6719999999999999,"5min":1.7430000000000001,"15min":1.746},"min":{"1min":1.4950000000000001,"5min":1.3400000000000001,"15min":1.302},"max":{"1min":1.901,"5min":2.9700000000000002,"15min":3.0329999999999999},"last":1.5169999999999999}]},{"osd":2,"last update":"Sat Nov 28 22:26:34 2020","interfaces":[{"interface":"back","average":{"1min":2.1259999999999999,"5min":2.113,"15min":2.1040000000000001},"min":{"1min":1.411,"5min":1.3819999999999999,"15min":1.3819999999999999},"max":{"1min":2.6600000000000001,"5min":3.3239999999999998,"15min":3.3340000000000001},"last":2.0590000000000002},{"interface":"front","average":{"1min":2.4929999999999999,"5min":2.4550000000000001,"15min":2.4609999999999999},"min":{"1min":2.1230000000000002,"5min":1.25,"15min":1.25},"max":{"1min":2.8599999999999999,"5min":3.3610000000000002,"15min":3.4540000000000002},"last":2.1819999999999999}]},{"osd":3,"last update":"Sat Nov 28 22:26:34 2020","interfaces":[{"interface":"back","average":{"1min":1.833,"5min":1.925,"15min":1.917},"min":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.276},"max":{"1min":2.528,"5min":3.3500000000000001,"15min":3.3500000000000001},"last":1.4379999999999999},{"interface":"front","average":{"1min":2.4009999999999998,"5min":2.4289999999999998,"15min":2.387},"min":{"1min":1.6739999999999999,"5min":1.653,"15min":1.512},"max":{"1min":2.899,"5min":3.7290000000000001,"15min":3.7290000000000001},"last":2.464}]},{"osd":4,"last update":"Sat Nov 28 22:26:34 2020","interfaces":[{"interface":"back","average":{"1min":2.2490000000000001,"5min":2.363,"15min":2.3079999999999998},"min":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.478},"max":{"1min":3.2919999999999998,"5min":3.8809999999999998,"15min":3.8809999999999998},"last":2.7400000000000002},{"interface":"front","average":{"1min":2.2309999999999999,"5min":2.375,"15min":2.2970000000000002},"min":{"1min":1.7829999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"max":{"1min":2.8940000000000001,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":2.141}]},{"osd":5,"last update":"Sat Nov 28 22:26:34 2020","interfaces":[{"interface":"back","average":{"1min":2.8620000000000001,"5min":2.899,"15min":2.8210000000000002},"min":{"1min":2.0089999999999999,"5min":1.8280000000000001,"15min":1.764},"max":{"1min":3.661,"5min":4.4480000000000004,"15min":4.4480000000000004},"last":2.6970000000000001},{"interface":"front","average":{"1min":2.8149999999999999,"5min":2.9279999999999999,"15min":2.8559999999999999},"min":{"1min":2.1379999999999999,"5min":2.1379999999999999,"15min":1.6990000000000001},"max":{"1min":3.2400000000000002,"5min":4.3280000000000003,"15min":4.3280000000000003},"last":3.0739999999999998}]},{"osd":6,"last update":"Sat Nov 28 22:26:34 2020","interfaces":[{"interface":"back","average":{"1min":2.6499999999999999,"5min":2.7810000000000001,"15min":2.706},"min":{"1min":2.1230000000000002,"5min":1.982,"15min":1.732},"max":{"1min":3.1160000000000001,"5min":3.8170000000000002,"15min":3.8170000000000002},"last":2.5499999999999998},{"interface":"front","average":{"1min":3.0449999999999999,"5min":3.1320000000000001,"15min":3.069},"min":{"1min":2.2330000000000001,"5min":2.2330000000000001,"15min":2.0190000000000001},"max":{"1min":3.5129999999999999,"5min":4.5750000000000002,"15min":4.5750000000000002},"last":3.258}]},{"osd":7,"last update":"Sat Nov 28 22:26:34 2020","interfaces":[{"interface":"back","average":{"1min":3.069,"5min":3.1440000000000001,"15min":3.1120000000000001},"min":{"1min":2.593,"5min":2.4900000000000002,"15min":1.8340000000000001},"max":{"1min":3.4849999999999999,"5min":4.7119999999999997,"15min":4.7119999999999997},"last":3.3599999999999999},{"interface":"front","average":{"1min":3.044,"5min":3.0859999999999999,"15min":3.008},"min":{"1min":2.4609999999999999,"5min":2.294,"15min":2.294},"max":{"1min":3.3940000000000001,"5min":4.0499999999999998,"15min":4.1660000000000004},"last":2.8079999999999998}]}]},{"osd":2,"up_from":14,"seq":60129542630,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050608,"kb_used_data":2024,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321232,"statfs":{"total":96636764160,"available":95560941568,"internally_reserved":1073741824,"allocated":2072576,"data_stored":959022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 28 22:25:56 2020","interfaces":[{"interface":"back","average":{"1min":1.7110000000000001,"5min":1.6759999999999999,"15min":1.6599999999999999},"min":{"1min":1.4239999999999999,"5min":1.2969999999999999,"15min":1.1200000000000001},"max":{"1min":2.7839999999999998,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":1.5900000000000001},{"interface":"front","average":{"1min":1.8779999999999999,"5min":1.8049999999999999,"15min":1.819},"min":{"1min":1.538,"5min":1.3340000000000001,"15min":1.3340000000000001},"max":{"1min":2.5110000000000001,"5min":3.1720000000000002,"15min":3.3570000000000002},"last":2.0329999999999999}]},{"osd":1,"last update":"Sat Nov 28 22:25:56 2020","interfaces":[{"interface":"back","average":{"1min":1.8260000000000001,"5min":1.891,"15min":1.9179999999999999},"min":{"1min":1.4790000000000001,"5min":1.3919999999999999,"15min":1.3460000000000001},"max":{"1min":2.3279999999999998,"5min":3.802,"15min":3.802},"last":2.3029999999999999},{"interface":"front","average":{"1min":2.4239999999999999,"5min":2.3940000000000001,"15min":2.3769999999999998},"min":{"1min":1.988,"5min":1.5269999999999999,"15min":1.4670000000000001},"max":{"1min":3.4390000000000001,"5min":3.4529999999999998,"15min":3.4529999999999998},"last":2.1749999999999998}]},{"osd":3,"last update":"Sat Nov 28 22:25:56 2020","interfaces":[{"interface":"back","average":{"1min":2.1379999999999999,"5min":2.2170000000000001,"15min":2.2090000000000001},"min":{"1min":1.5720000000000001,"5min":1.4490000000000001,"15min":1.361},"max":{"1min":2.6840000000000002,"5min":3.5459999999999998,"15min":3.6440000000000001},"last":1.4850000000000001},{"interface":"front","average":{"1min":2.3159999999999998,"5min":2.27,"15min":2.2730000000000001},"min":{"1min":1.7490000000000001,"5min":1.696,"15min":1.5409999999999999},"max":{"1min":2.7429999999999999,"5min":3.0640000000000001,"15min":3.8799999999999999},"last":1.881}]},{"osd":4,"last update":"Sat Nov 28 22:25:56 2020","interfaces":[{"interface":"back","average":{"1min":2.4940000000000002,"5min":2.4089999999999998,"15min":2.383},"min":{"1min":1.784,"5min":1.7490000000000001,"15min":1.645},"max":{"1min":3.976,"5min":4.1159999999999997,"15min":4.1159999999999997},"last":2.8839999999999999},{"interface":"front","average":{"1min":2.706,"5min":2.6349999999999998,"15min":2.637},"min":{"1min":2.0600000000000001,"5min":1.76,"15min":1.595},"max":{"1min":3.5110000000000001,"5min":3.7280000000000002,"15min":4.2140000000000004},"last":1.8360000000000001}]},{"osd":5,"last update":"Sat Nov 28 22:25:56 2020","interfaces":[{"interface":"back","average":{"1min":2.8980000000000001,"5min":2.839,"15min":2.8980000000000001},"min":{"1min":2.246,"5min":1.8169999999999999,"15min":1.8169999999999999},"max":{"1min":3.5830000000000002,"5min":3.9100000000000001,"15min":4.4089999999999998},"last":2.2440000000000002},{"interface":"front","average":{"1min":2.956,"5min":2.8919999999999999,"15min":2.8820000000000001},"min":{"1min":2.1869999999999998,"5min":1.925,"15min":1.925},"max":{"1min":3.8420000000000001,"5min":3.8420000000000001,"15min":4.1520000000000001},"last":2.5720000000000001}]},{"osd":6,"last update":"Sat Nov 28 22:25:56 2020","interfaces":[{"interface":"back","average":{"1min":3.1339999999999999,"5min":3.052,"15min":3.093},"min":{"1min":2.516,"5min":2.1150000000000002,"15min":2.1070000000000002},"max":{"1min":3.649,"5min":3.9580000000000002,"15min":3.9580000000000002},"last":3.02},{"interface":"front","average":{"1min":2.992,"5min":2.972,"15min":2.964},"min":{"1min":2.2730000000000001,"5min":2.2730000000000001,"15min":2.1349999999999998},"max":{"1min":3.766,"5min":4.3150000000000004,"15min":4.3150000000000004},"last":3.2519999999999998}]},{"osd":7,"last update":"Sat Nov 28 22:25:56 2020","interfaces":[{"interface":"back","average":{"1min":3.266,"5min":3.202,"15min":3.2189999999999999},"min":{"1min":2.774,"5min":2.4990000000000001,"15min":2.0569999999999999},"max":{"1min":3.964,"5min":4.1980000000000004,"15min":4.1980000000000004},"last":2.8159999999999998},{"interface":"front","average":{"1min":3.266,"5min":3.2970000000000002,"15min":3.3380000000000001},"min":{"1min":2.6230000000000002,"5min":2.6230000000000002,"15min":2.1110000000000002},"max":{"1min":3.7599999999999998,"5min":4.0209999999999999,"15min":4.3289999999999997},"last":3.3639999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542630,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050608,"kb_used_data":2024,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321232,"statfs":{"total":96636764160,"available":95560941568,"internally_reserved":1073741824,"allocated":2072576,"data_stored":959022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 28 22:26:36 2020","interfaces":[{"interface":"back","average":{"1min":1.742,"5min":1.6879999999999999,"15min":1.6919999999999999},"min":{"1min":1.355,"5min":1.258,"15min":1.143},"max":{"1min":2.1040000000000001,"5min":2.5409999999999999,"15min":2.9100000000000001},"last":1.587},{"interface":"front","average":{"1min":1.665,"5min":1.694,"15min":1.6699999999999999},"min":{"1min":1.4770000000000001,"5min":1.375,"15min":1.2490000000000001},"max":{"1min":1.9490000000000001,"5min":2.476,"15min":2.476},"last":1.9490000000000001}]},{"osd":1,"last update":"Sat Nov 28 22:26:36 2020","interfaces":[{"interface":"back","average":{"1min":1.595,"5min":1.7529999999999999,"15min":1.776},"min":{"1min":1.4630000000000001,"5min":1.377,"15min":1.3140000000000001},"max":{"1min":1.829,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.982,"5min":2.077,"15min":2.1040000000000001},"min":{"1min":1.569,"5min":1.5389999999999999,"15min":1.4390000000000001},"max":{"1min":2.5990000000000002,"5min":2.806,"15min":3.0990000000000002},"last":2.0739999999999998}]},{"osd":2,"last update":"Sat Nov 28 22:26:36 2020","interfaces":[{"interface":"back","average":{"1min":2.2429999999999999,"5min":2.2410000000000001,"15min":2.2679999999999998},"min":{"1min":1.8420000000000001,"5min":1.5089999999999999,"15min":1.3700000000000001},"max":{"1min":2.9119999999999999,"5min":3.4900000000000002,"15min":3.4900000000000002},"last":2.262},{"interface":"front","average":{"1min":2.589,"5min":2.5350000000000001,"15min":2.609},"min":{"1min":2.097,"5min":2.0600000000000001,"15min":1.8340000000000001},"max":{"1min":3.234,"5min":3.4079999999999999,"15min":4.3289999999999997},"last":2.3450000000000002}]},{"osd":4,"last update":"Sat Nov 28 22:26:36 2020","interfaces":[{"interface":"back","average":{"1min":2.331,"5min":2.2490000000000001,"15min":2.331},"min":{"1min":1.9850000000000001,"5min":1.6559999999999999,"15min":1.6559999999999999},"max":{"1min":3.0649999999999999,"5min":3.0790000000000002,"15min":4.0519999999999996},"last":2.181},{"interface":"front","average":{"1min":2.6269999999999998,"5min":2.6259999999999999,"15min":2.5950000000000002},"min":{"1min":2.02,"5min":1.506,"15min":1.506},"max":{"1min":3.472,"5min":3.6779999999999999,"15min":4.0140000000000002},"last":2.73}]},{"osd":5,"last update":"Sat Nov 28 22:26:36 2020","interfaces":[{"interface":"back","average":{"1min":2.903,"5min":2.8079999999999998,"15min":2.8330000000000002},"min":{"1min":2.2690000000000001,"5min":1.9179999999999999,"15min":1.8979999999999999},"max":{"1min":3.6139999999999999,"5min":3.7170000000000001,"15min":4.1699999999999999},"last":3.0979999999999999},{"interface":"front","average":{"1min":2.8050000000000002,"5min":2.8719999999999999,"15min":2.9399999999999999},"min":{"1min":2.1909999999999998,"5min":2.181,"15min":1.96},"max":{"1min":3.5249999999999999,"5min":3.6949999999999998,"15min":4.085},"last":2.6560000000000001}]},{"osd":6,"last update":"Sat Nov 28 22:26:36 2020","interfaces":[{"interface":"back","average":{"1min":3.0470000000000002,"5min":3.0579999999999998,"15min":3.1579999999999999},"min":{"1min":2.1800000000000002,"5min":2.1800000000000002,"15min":2.173},"max":{"1min":3.7679999999999998,"5min":3.9169999999999998,"15min":4.4980000000000002},"last":2.8410000000000002},{"interface":"front","average":{"1min":2.8860000000000001,"5min":2.8370000000000002,"15min":2.8719999999999999},"min":{"1min":2.2330000000000001,"5min":2.2330000000000001,"15min":2.1299999999999999},"max":{"1min":3.5150000000000001,"5min":3.5150000000000001,"15min":4.8819999999999997},"last":2.4249999999999998}]},{"osd":7,"last update":"Sat Nov 28 22:26:36 2020","interfaces":[{"interface":"back","average":{"1min":3.0139999999999998,"5min":3.0339999999999998,"15min":3.0720000000000001},"min":{"1min":2.6230000000000002,"5min":2.5049999999999999,"15min":2.3250000000000002},"max":{"1min":3.5289999999999999,"5min":3.6899999999999999,"15min":3.9420000000000002},"last":2.6230000000000002},{"interface":"front","average":{"1min":3.149,"5min":3.1760000000000002,"15min":3.1680000000000001},"min":{"1min":2.5899999999999999,"5min":2.4279999999999999,"15min":2.1989999999999998},"max":{"1min":3.8559999999999999,"5min":3.8559999999999999,"15min":4.5880000000000001},"last":3.4620000000000002}]}]},{"osd":4,"up_from":14,"seq":60129542630,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050608,"kb_used_data":2024,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321232,"statfs":{"total":96636764160,"available":95560941568,"internally_reserved":1073741824,"allocated":2072576,"data_stored":959022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 28 22:26:23 2020","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.754,"15min":1.748},"min":{"1min":1.232,"5min":1.232,"15min":1.012},"max":{"1min":3.0089999999999999,"5min":3.0249999999999999,"15min":3.2549999999999999},"last":1.589},{"interface":"front","average":{"1min":1.794,"5min":1.871,"15min":1.8089999999999999},"min":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.0980000000000001},"max":{"1min":2.8780000000000001,"5min":3.294,"15min":3.452},"last":1.665}]},{"osd":1,"last update":"Sat Nov 28 22:26:23 2020","interfaces":[{"interface":"back","average":{"1min":2.0790000000000002,"5min":2.1360000000000001,"15min":2.2160000000000002},"min":{"1min":1.5209999999999999,"5min":1.3080000000000001,"15min":1.284},"max":{"1min":2.7879999999999998,"5min":3.3159999999999998,"15min":3.6720000000000002},"last":2.161},{"interface":"front","average":{"1min":2.032,"5min":2.0750000000000002,"15min":2.1120000000000001},"min":{"1min":1.498,"5min":1.369,"15min":1.0469999999999999},"max":{"1min":2.7839999999999998,"5min":2.9910000000000001,"15min":3.2570000000000001},"last":2.1190000000000002}]},{"osd":2,"last update":"Sat Nov 28 22:26:23 2020","interfaces":[{"interface":"back","average":{"1min":1.804,"5min":1.9330000000000001,"15min":1.9850000000000001},"min":{"1min":1.391,"5min":1.391,"15min":1.391},"max":{"1min":2.4700000000000002,"5min":2.9300000000000002,"15min":3.5880000000000001},"last":1.9930000000000001},{"interface":"front","average":{"1min":2.4540000000000002,"5min":2.4729999999999999,"15min":2.5089999999999999},"min":{"1min":1.887,"5min":1.7210000000000001,"15min":1.419},"max":{"1min":3.169,"5min":3.7919999999999998,"15min":3.7919999999999998},"last":2.6930000000000001}]},{"osd":3,"last update":"Sat Nov 28 22:26:23 2020","interfaces":[{"interface":"back","average":{"1min":2.399,"5min":2.3679999999999999,"15min":2.4220000000000002},"min":{"1min":1.8080000000000001,"5min":1.431,"15min":1.407},"max":{"1min":3.161,"5min":3.3620000000000001,"15min":3.5310000000000001},"last":2.3530000000000002},{"interface":"front","average":{"1min":2.1389999999999998,"5min":2.2709999999999999,"15min":2.331},"min":{"1min":1.669,"5min":1.6579999999999999,"15min":1.246},"max":{"1min":2.7629999999999999,"5min":3.8759999999999999,"15min":3.9580000000000002},"last":2.4399999999999999}]},{"osd":5,"last update":"Sat Nov 28 22:26:23 2020","interfaces":[{"interface":"back","average":{"1min":2.665,"5min":2.6869999999999998,"15min":2.6909999999999998},"min":{"1min":1.8260000000000001,"5min":1.3819999999999999,"15min":1.3819999999999999},"max":{"1min":3.5369999999999999,"5min":4.1280000000000001,"15min":4.1280000000000001},"last":2.3919999999999999},{"interface":"front","average":{"1min":2.657,"5min":2.7309999999999999,"15min":2.7810000000000001},"min":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"max":{"1min":3.6600000000000001,"5min":3.6629999999999998,"15min":4.2350000000000003},"last":3.1720000000000002}]},{"osd":6,"last update":"Sat Nov 28 22:26:23 2020","interfaces":[{"interface":"back","average":{"1min":2.5960000000000001,"5min":2.6709999999999998,"15min":2.6480000000000001},"min":{"1min":2.0059999999999998,"5min":1.1599999999999999,"15min":1.1599999999999999},"max":{"1min":3.1949999999999998,"5min":4.0800000000000001,"15min":4.351},"last":2.8410000000000002},{"interface":"front","average":{"1min":2.984,"5min":3.0310000000000001,"15min":3.0379999999999998},"min":{"1min":2.1659999999999999,"5min":2.1219999999999999,"15min":1.8600000000000001},"max":{"1min":3.8959999999999999,"5min":4.5270000000000001,"15min":4.5270000000000001},"last":2.992}]},{"osd":7,"last update":"Sat Nov 28 22:26:23 2020","interfaces":[{"interface":"back","average":{"1min":3.089,"5min":3.1059999999999999,"15min":3.1389999999999998},"min":{"1min":2.2109999999999999,"5min":2.2109999999999999,"15min":1.591},"max":{"1min":3.5129999999999999,"5min":4.0229999999999997,"15min":4.2149999999999999},"last":3.3170000000000002},{"interface":"front","average":{"1min":2.8809999999999998,"5min":3.0720000000000001,"15min":3.1040000000000001},"min":{"1min":2.153,"5min":1.319,"15min":1.319},"max":{"1min":3.3929999999999998,"5min":4.2709999999999999,"15min":5.133},"last":3.2130000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542630,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050604,"kb_used_data":2020,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321236,"statfs":{"total":96636764160,"available":95560945664,"internally_reserved":1073741824,"allocated":2068480,"data_stored":958419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 28 22:26:29 2020","interfaces":[{"interface":"back","average":{"1min":2.2799999999999998,"5min":2.0899999999999999,"15min":2.0489999999999999},"min":{"1min":1.321,"5min":1.0609999999999999,"15min":1.0609999999999999},"max":{"1min":2.9140000000000001,"5min":2.9889999999999999,"15min":3.4780000000000002},"last":1.512},{"interface":"front","average":{"1min":2.5529999999999999,"5min":2.4079999999999999,"15min":2.375},"min":{"1min":2.0059999999999998,"5min":1.5209999999999999,"15min":1.5209999999999999},"max":{"1min":3.1190000000000002,"5min":4.1390000000000002,"15min":4.6390000000000002},"last":2.226}]},{"osd":1,"last update":"Sat Nov 28 22:26:29 2020","interfaces":[{"interface":"back","average":{"1min":1.9319999999999999,"5min":2.0489999999999999,"15min":2.1440000000000001},"min":{"1min":1.4319999999999999,"5min":1.286,"15min":1.266},"max":{"1min":3.0590000000000002,"5min":3.4729999999999999,"15min":4.4909999999999997},"last":1.9730000000000001},{"interface":"front","average":{"1min":2.4700000000000002,"5min":2.6429999999999998,"15min":2.649},"min":{"1min":1.446,"5min":1.446,"15min":1.446},"max":{"1min":3.4060000000000001,"5min":4.7960000000000003,"15min":5.0449999999999999},"last":2.9790000000000001}]},{"osd":2,"last update":"Sat Nov 28 22:26:29 2020","interfaces":[{"interface":"back","average":{"1min":2.0369999999999999,"5min":1.9330000000000001,"15min":1.9830000000000001},"min":{"1min":1.6000000000000001,"5min":1.4570000000000001,"15min":1.367},"max":{"1min":3.04,"5min":3.0569999999999999,"15min":3.8260000000000001},"last":2.0350000000000001},{"interface":"front","average":{"1min":2.1440000000000001,"5min":2.363,"15min":2.3570000000000002},"min":{"1min":1.6930000000000001,"5min":1.4410000000000001,"15min":1.381},"max":{"1min":3.2730000000000001,"5min":3.677,"15min":4.742},"last":2.3239999999999998}]},{"osd":3,"last update":"Sat Nov 28 22:26:29 2020","interfaces":[{"interface":"back","average":{"1min":1.881,"5min":2.0640000000000001,"15min":2.0550000000000002},"min":{"1min":1.5680000000000001,"5min":1.363,"15min":1.2609999999999999},"max":{"1min":2.4980000000000002,"5min":4.532,"15min":5.2469999999999999},"last":1.8720000000000001},{"interface":"front","average":{"1min":2.4780000000000002,"5min":2.4660000000000002,"15min":2.4929999999999999},"min":{"1min":1.879,"5min":1.7669999999999999,"15min":1.427},"max":{"1min":3.7280000000000002,"5min":3.7280000000000002,"15min":4.5979999999999999},"last":3.0979999999999999}]},{"osd":4,"last update":"Sat Nov 28 22:26:29 2020","interfaces":[{"interface":"back","average":{"1min":2.3180000000000001,"5min":2.4769999999999999,"15min":2.4910000000000001},"min":{"1min":1.3480000000000001,"5min":1.157,"15min":1.0580000000000001},"max":{"1min":3.2909999999999999,"5min":4.0350000000000001,"15min":4.335},"last":2.4089999999999998},{"interface":"front","average":{"1min":2.468,"5min":2.6469999999999998,"15min":2.6850000000000001},"min":{"1min":1.7769999999999999,"5min":1.7110000000000001,"15min":0.94699999999999995},"max":{"1min":3.476,"5min":4.1749999999999998,"15min":5.1900000000000004},"last":2.7050000000000001}]},{"osd":6,"last update":"Sat Nov 28 22:26:29 2020","interfaces":[{"interface":"back","average":{"1min":2.347,"5min":2.5179999999999998,"15min":2.5739999999999998},"min":{"1min":1.7809999999999999,"5min":1.1930000000000001,"15min":1.107},"max":{"1min":3.1829999999999998,"5min":4.9710000000000001,"15min":5.431},"last":2.6349999999999998},{"interface":"front","average":{"1min":2.879,"5min":2.9449999999999998,"15min":2.9710000000000001},"min":{"1min":2.1669999999999998,"5min":1.3999999999999999,"15min":1.3460000000000001},"max":{"1min":3.9300000000000002,"5min":5.4169999999999998,"15min":5.4169999999999998},"last":3.4569999999999999}]},{"osd":7,"last update":"Sat Nov 28 22:26:29 2020","interfaces":[{"interface":"back","average":{"1min":2.9209999999999998,"5min":2.9969999999999999,"15min":3.036},"min":{"1min":2.3260000000000001,"5min":1.6499999999999999,"15min":1.4710000000000001},"max":{"1min":3.6589999999999998,"5min":4.1799999999999997,"15min":4.9829999999999997},"last":3.0550000000000002},{"interface":"front","average":{"1min":3.218,"5min":3.1179999999999999,"15min":3.157},"min":{"1min":2.3719999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"max":{"1min":4.1299999999999999,"5min":4.3029999999999999,"15min":5.319},"last":3.8140000000000001}]}]}]}} 2020-11-28T22:26:43.156 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-28T22:26:43.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.264+0000 7ff2ebfff700 1 Processor -- start 2020-11-28T22:26:43.268 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.264+0000 7ff2ebfff700 1 -- start start 2020-11-28T22:26:43.270 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.268+0000 7ff2ebfff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff2ec09abb0 0x7ff2ec061190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:43.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.268+0000 7ff2ebfff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff2ec060010 0x7ff2ec144670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:43.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.268+0000 7ff2ebfff700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2ec143ba0 con 0x7ff2ec0b01a0 2020-11-28T22:26:43.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.268+0000 7ff2ebfff700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2ec144b60 con 0x7ff2ec09abb0 2020-11-28T22:26:43.271 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.268+0000 7ff2ebfff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ec09af90 con 0x7ff2ec060010 2020-11-28T22:26:43.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.268+0000 7ff2eaffd700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff2ec09abb0 0x7ff2ec061190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:43.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.268+0000 7ff2ea7fc700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7ff2ec0b01a0 0x7ff2ec143f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:53608/0 (socket says 172.21.15.162:53608) 2020-11-28T22:26:43.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.268+0000 7ff2ea7fc700 1 -- 172.21.15.162:0/1175325507 learned_addr learned my addr 172.21.15.162:0/1175325507 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T22:26:43.272 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.268+0000 7ff2eb7fe700 1 --2- 172.21.15.162:0/1175325507 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff2ec060010 0x7ff2ec144670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:43.273 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.268+0000 7ff2eaffd700 1 -- 172.21.15.162:0/1175325507 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff2ec060010 msgr2=0x7ff2ec144670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T22:26:43.273 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.268+0000 7ff2eaffd700 1 --2- 172.21.15.162:0/1175325507 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff2ec060010 0x7ff2ec144670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:43.273 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.268+0000 7ff2eaffd700 1 -- 172.21.15.162:0/1175325507 >> v1:172.21.15.162:6789/0 conn(0x7ff2ec0b01a0 legacy=0x7ff2ec143f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:43.273 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.268+0000 7ff2eaffd700 1 -- 172.21.15.162:0/1175325507 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2ec0b1d10 con 0x7ff2ec09abb0 2020-11-28T22:26:43.277 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.268+0000 7ff2eaffd700 1 --2- 172.21.15.162:0/1175325507 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2ec09abb0 0x7ff2ec061190 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 rx=0x7ff2e4002f30 tx=0x7ff2e4006dc0).ready entity=mon.1 client_cookie=e4ec7b0b2f55ef22 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:43.277 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.268+0000 7ff2e9ffb700 1 -- 172.21.15.162:0/1175325507 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2e40097f0 con 0x7ff2ec09abb0 2020-11-28T22:26:43.277 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.268+0000 7ff2e9ffb700 1 -- 172.21.15.162:0/1175325507 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2e4009950 con 0x7ff2ec09abb0 2020-11-28T22:26:43.277 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.268+0000 7ff2e9ffb700 1 -- 172.21.15.162:0/1175325507 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2e4009cb0 con 0x7ff2ec09abb0 2020-11-28T22:26:43.278 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.268+0000 7ff2ebfff700 1 -- 172.21.15.162:0/1175325507 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2ec09abb0 msgr2=0x7ff2ec061190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:43.278 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.268+0000 7ff2ebfff700 1 --2- 172.21.15.162:0/1175325507 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2ec09abb0 0x7ff2ec061190 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 rx=0x7ff2e4002f30 tx=0x7ff2e4006dc0).stop 2020-11-28T22:26:43.278 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.268+0000 7ff2ebfff700 1 -- 172.21.15.162:0/1175325507 shutdown_connections 2020-11-28T22:26:43.278 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.268+0000 7ff2ebfff700 1 --2- 172.21.15.162:0/1175325507 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2ec09abb0 0x7ff2ec061190 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:43.279 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.268+0000 7ff2ebfff700 1 --2- 172.21.15.162:0/1175325507 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff2ec060010 0x7ff2ec144670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:43.279 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.268+0000 7ff2ebfff700 1 -- 172.21.15.162:0/1175325507 >> 172.21.15.162:0/1175325507 conn(0x7ff2ec0b54f0 msgr2=0x7ff2ec0b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:26:43.279 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.268+0000 7ff2ebfff700 1 -- 172.21.15.162:0/1175325507 shutdown_connections 2020-11-28T22:26:43.279 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.268+0000 7ff2ebfff700 1 -- 172.21.15.162:0/1175325507 wait complete. 2020-11-28T22:26:43.280 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.272+0000 7ff2ebfff700 1 Processor -- start 2020-11-28T22:26:43.280 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.272+0000 7ff2ebfff700 1 -- start start 2020-11-28T22:26:43.280 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.272+0000 7ff2ebfff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff2ec060010 0x7ff2ec063bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:43.280 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.272+0000 7ff2ebfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2ec09abb0 0x7ff2ec142900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:43.280 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.272+0000 7ff2ebfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff2ec0b01a0 0x7ff2ec14fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:43.281 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.272+0000 7ff2ebfff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ec065fd0 con 0x7ff2ec0b01a0 2020-11-28T22:26:43.281 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.272+0000 7ff2ebfff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ec147340 con 0x7ff2ec09abb0 2020-11-28T22:26:43.281 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.272+0000 7ff2ebfff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ec066230 con 0x7ff2ec060010 2020-11-28T22:26:43.281 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.272+0000 7ff2ea7fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2ec09abb0 0x7ff2ec142900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:43.282 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.272+0000 7ff2ea7fc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2ec09abb0 0x7ff2ec142900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46770/0 (socket says 172.21.15.162:46770) 2020-11-28T22:26:43.282 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.272+0000 7ff2ea7fc700 1 -- 172.21.15.162:0/1765878541 learned_addr learned my addr 172.21.15.162:0/1765878541 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T22:26:43.282 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.272+0000 7ff2eb7fe700 1 --2- 172.21.15.162:0/1765878541 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff2ec0b01a0 0x7ff2ec14fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:43.282 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.272+0000 7ff2eaffd700 1 --2- 172.21.15.162:0/1765878541 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff2ec060010 0x7ff2ec063bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:43.283 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.272+0000 7ff2ea7fc700 1 -- 172.21.15.162:0/1765878541 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff2ec060010 msgr2=0x7ff2ec063bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:43.283 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.272+0000 7ff2ea7fc700 1 --2- 172.21.15.162:0/1765878541 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff2ec060010 0x7ff2ec063bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:43.283 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.272+0000 7ff2ea7fc700 1 -- 172.21.15.162:0/1765878541 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff2ec0b01a0 msgr2=0x7ff2ec14fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:43.283 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.272+0000 7ff2ea7fc700 1 --2- 172.21.15.162:0/1765878541 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff2ec0b01a0 0x7ff2ec14fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:43.284 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.272+0000 7ff2ea7fc700 1 -- 172.21.15.162:0/1765878541 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2ec14f720 con 0x7ff2ec09abb0 2020-11-28T22:26:43.284 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.272+0000 7ff2eb7fe700 1 --2- 172.21.15.162:0/1765878541 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff2ec0b01a0 0x7ff2ec14fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T22:26:43.284 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.272+0000 7ff2ea7fc700 1 --2- 172.21.15.162:0/1765878541 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2ec09abb0 0x7ff2ec142900 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 rx=0x7ff2ec09b300 tx=0x7ff2d4007c30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:43.284 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.272+0000 7ff2e3fff700 1 -- 172.21.15.162:0/1765878541 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2d4007380 con 0x7ff2ec09abb0 2020-11-28T22:26:43.285 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.272+0000 7ff2e3fff700 1 -- 172.21.15.162:0/1765878541 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d4005030 con 0x7ff2ec09abb0 2020-11-28T22:26:43.285 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.272+0000 7ff2e3fff700 1 -- 172.21.15.162:0/1765878541 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2d4005390 con 0x7ff2ec09abb0 2020-11-28T22:26:43.285 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.272+0000 7ff2ebfff700 1 -- 172.21.15.162:0/1765878541 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2ec146db0 con 0x7ff2ec09abb0 2020-11-28T22:26:43.285 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.272+0000 7ff2ebfff700 1 -- 172.21.15.162:0/1765878541 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2ec147900 con 0x7ff2ec09abb0 2020-11-28T22:26:43.287 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.276+0000 7ff2ebfff700 1 -- 172.21.15.162:0/1765878541 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2ec14f8b0 con 0x7ff2ec09abb0 2020-11-28T22:26:43.287 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.276+0000 7ff2e3fff700 1 -- 172.21.15.162:0/1765878541 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff2d401b070 con 0x7ff2ec09abb0 2020-11-28T22:26:43.287 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.276+0000 7ff2e3fff700 1 --2- 172.21.15.162:0/1765878541 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff2d0055330 0x7ff2d00577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:43.288 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.276+0000 7ff2eaffd700 1 --2- 172.21.15.162:0/1765878541 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff2d0055330 0x7ff2d00577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:43.288 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.276+0000 7ff2e3fff700 1 -- 172.21.15.162:0/1765878541 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5339+0+0 (secure 0 0 0) 0x7ff2d406d270 con 0x7ff2ec09abb0 2020-11-28T22:26:43.288 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.276+0000 7ff2eaffd700 1 --2- 172.21.15.162:0/1765878541 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff2d0055330 0x7ff2d00577f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 rx=0x7ff2e4019000 tx=0x7ff2e4015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:43.288 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.284+0000 7ff2e3fff700 1 -- 172.21.15.162:0/1765878541 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff2d4043740 con 0x7ff2ec09abb0 2020-11-28T22:26:43.429 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.428+0000 7ff2ebfff700 1 -- 172.21.15.162:0/1765878541 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff2ec057050 con 0x7ff2ec09abb0 2020-11-28T22:26:43.431 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.428+0000 7ff2e3fff700 1 -- 172.21.15.162:0/1765878541 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v163) v1 ==== 74+0+13418 (secure 0 0 0) 0x7ff2d4046d60 con 0x7ff2ec09abb0 2020-11-28T22:26:43.432 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T22:26:43.432 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":163,"fsid":"7c321bcf-cd33-40f7-a39f-da5470b386cf","created":"2020-11-28T21:46:11.724372+0000","modified":"2020-11-28T21:56:41.910010+0000","last_up_change":"2020-11-28T21:46:19.591650+0000","last_in_change":"2020-11-28T21:46:19.591650+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-28T21:46:15.639551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-28T21:46:23.901366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"99c7d9ae-adfa-4fd7-880a-23e6baf3281b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":12313},{"type":"v1","addr":"172.21.15.162:6811","nonce":12313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":12313},{"type":"v1","addr":"172.21.15.162:6813","nonce":12313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":12313},{"type":"v1","addr":"172.21.15.162:6817","nonce":12313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":12313},{"type":"v1","addr":"172.21.15.162:6815","nonce":12313}]},"public_addr":"172.21.15.162:6811/12313","cluster_addr":"172.21.15.162:6813/12313","heartbeat_back_addr":"172.21.15.162:6817/12313","heartbeat_front_addr":"172.21.15.162:6815/12313","state":["exists","up"]},{"osd":1,"uuid":"6ce6220a-00ef-4d13-abba-56f87636fb5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":12311},{"type":"v1","addr":"172.21.15.162:6803","nonce":12311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":12311},{"type":"v1","addr":"172.21.15.162:6805","nonce":12311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":12311},{"type":"v1","addr":"172.21.15.162:6809","nonce":12311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":12311},{"type":"v1","addr":"172.21.15.162:6807","nonce":12311}]},"public_addr":"172.21.15.162:6803/12311","cluster_addr":"172.21.15.162:6805/12311","heartbeat_back_addr":"172.21.15.162:6809/12311","heartbeat_front_addr":"172.21.15.162:6807/12311","state":["exists","up"]},{"osd":2,"uuid":"db4e510d-27c1-47fc-a902-8b32e7dfd73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":12312},{"type":"v1","addr":"172.21.15.162:6819","nonce":12312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":12312},{"type":"v1","addr":"172.21.15.162:6821","nonce":12312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":12312},{"type":"v1","addr":"172.21.15.162:6825","nonce":12312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":12312},{"type":"v1","addr":"172.21.15.162:6823","nonce":12312}]},"public_addr":"172.21.15.162:6819/12312","cluster_addr":"172.21.15.162:6821/12312","heartbeat_back_addr":"172.21.15.162:6825/12312","heartbeat_front_addr":"172.21.15.162:6823/12312","state":["exists","up"]},{"osd":3,"uuid":"4094cb87-f04a-45e1-8710-fb4b651949f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":12323},{"type":"v1","addr":"172.21.15.162:6827","nonce":12323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":12323},{"type":"v1","addr":"172.21.15.162:6829","nonce":12323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":12323},{"type":"v1","addr":"172.21.15.162:6833","nonce":12323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":12323},{"type":"v1","addr":"172.21.15.162:6831","nonce":12323}]},"public_addr":"172.21.15.162:6827/12323","cluster_addr":"172.21.15.162:6829/12323","heartbeat_back_addr":"172.21.15.162:6833/12323","heartbeat_front_addr":"172.21.15.162:6831/12323","state":["exists","up"]},{"osd":4,"uuid":"b907bf53-c297-4756-847b-3be96680eef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":12527},{"type":"v1","addr":"172.21.15.118:6801","nonce":12527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":12527},{"type":"v1","addr":"172.21.15.118:6803","nonce":12527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":12527},{"type":"v1","addr":"172.21.15.118:6807","nonce":12527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":12527},{"type":"v1","addr":"172.21.15.118:6805","nonce":12527}]},"public_addr":"172.21.15.118:6801/12527","cluster_addr":"172.21.15.118:6803/12527","heartbeat_back_addr":"172.21.15.118:6807/12527","heartbeat_front_addr":"172.21.15.118:6805/12527","state":["exists","up"]},{"osd":5,"uuid":"55ba305d-0505-4212-aebb-631e8554b440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":12529},{"type":"v1","addr":"172.21.15.118:6809","nonce":12529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":12529},{"type":"v1","addr":"172.21.15.118:6811","nonce":12529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":12529},{"type":"v1","addr":"172.21.15.118:6815","nonce":12529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":12529},{"type":"v1","addr":"172.21.15.118:6813","nonce":12529}]},"public_addr":"172.21.15.118:6809/12529","cluster_addr":"172.21.15.118:6811/12529","heartbeat_back_addr":"172.21.15.118:6815/12529","heartbeat_front_addr":"172.21.15.118:6813/12529","state":["exists","up"]},{"osd":6,"uuid":"9ada6051-493b-46b5-ba22-1b401440490f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":12530},{"type":"v1","addr":"172.21.15.118:6825","nonce":12530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":12530},{"type":"v1","addr":"172.21.15.118:6827","nonce":12530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":12530},{"type":"v1","addr":"172.21.15.118:6831","nonce":12530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":12530},{"type":"v1","addr":"172.21.15.118:6829","nonce":12530}]},"public_addr":"172.21.15.118:6825/12530","cluster_addr":"172.21.15.118:6827/12530","heartbeat_back_addr":"172.21.15.118:6831/12530","heartbeat_front_addr":"172.21.15.118:6829/12530","state":["exists","up"]},{"osd":7,"uuid":"c328b06d-a216-4cba-aad7-9ad081340613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":12528},{"type":"v1","addr":"172.21.15.118:6817","nonce":12528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":12528},{"type":"v1","addr":"172.21.15.118:6819","nonce":12528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":12528},{"type":"v1","addr":"172.21.15.118:6823","nonce":12528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":12528},{"type":"v1","addr":"172.21.15.118:6821","nonce":12528}]},"public_addr":"172.21.15.118:6817/12528","cluster_addr":"172.21.15.118:6819/12528","heartbeat_back_addr":"172.21.15.118:6823/12528","heartbeat_front_addr":"172.21.15.118:6821/12528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038052+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:17.947136+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.001186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.199284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.038823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.009322+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.238927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-28T21:46:18.029839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"172.21.15.162:6837/3753662046":"2020-11-29T21:56:38.469227+0000","172.21.15.162:6835/3753662046":"2020-11-29T21:56:38.469227+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-28T22:26:43.433 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.432+0000 7ff2ebfff700 1 -- 172.21.15.162:0/1765878541 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff2d0055330 msgr2=0x7ff2d00577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:43.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.432+0000 7ff2ebfff700 1 --2- 172.21.15.162:0/1765878541 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff2d0055330 0x7ff2d00577f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 rx=0x7ff2e4019000 tx=0x7ff2e4015000).stop 2020-11-28T22:26:43.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.432+0000 7ff2ebfff700 1 -- 172.21.15.162:0/1765878541 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2ec09abb0 msgr2=0x7ff2ec142900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:43.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.432+0000 7ff2ebfff700 1 --2- 172.21.15.162:0/1765878541 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2ec09abb0 0x7ff2ec142900 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 rx=0x7ff2ec09b300 tx=0x7ff2d4007c30).stop 2020-11-28T22:26:43.434 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.432+0000 7ff2ebfff700 1 -- 172.21.15.162:0/1765878541 shutdown_connections 2020-11-28T22:26:43.435 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.432+0000 7ff2ebfff700 1 --2- 172.21.15.162:0/1765878541 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff2d0055330 0x7ff2d00577f0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:43.435 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.432+0000 7ff2ebfff700 1 --2- 172.21.15.162:0/1765878541 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff2ec060010 0x7ff2ec063bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:43.435 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.432+0000 7ff2ebfff700 1 --2- 172.21.15.162:0/1765878541 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff2ec0b01a0 0x7ff2ec14fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:43.435 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.432+0000 7ff2ebfff700 1 --2- 172.21.15.162:0/1765878541 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2ec09abb0 0x7ff2ec142900 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:43.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.432+0000 7ff2ebfff700 1 -- 172.21.15.162:0/1765878541 >> 172.21.15.162:0/1765878541 conn(0x7ff2ec0b54f0 msgr2=0x7ff2ec0b3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:26:43.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.432+0000 7ff2ebfff700 1 -- 172.21.15.162:0/1765878541 shutdown_connections 2020-11-28T22:26:43.436 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:43.432+0000 7ff2ebfff700 1 -- 172.21.15.162:0/1765878541 wait complete. 2020-11-28T22:26:44.446 INFO:tasks.ceph:Scrubbing osd.0 2020-11-28T22:26:44.447 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.0 config set osd_debug_deep_scrub_sleep 0 2020-11-28T22:26:44.576 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.572+0000 7ff6a7f22700 1 Processor -- start 2020-11-28T22:26:44.577 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.572+0000 7ff6a7f22700 1 -- start start 2020-11-28T22:26:44.579 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.576+0000 7ff6a7f22700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff6a01409d0 0x7ff6a0066e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:44.579 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.576+0000 7ff6a7f22700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff6a00b37d0 0x7ff6a00b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:44.579 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.576+0000 7ff6a7f22700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6a00b1dd0 con 0x7ff6a0066a50 2020-11-28T22:26:44.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.576+0000 7ff6a7f22700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6a0143f20 con 0x7ff6a01409d0 2020-11-28T22:26:44.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.576+0000 7ff6a7f22700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6a00b2000 con 0x7ff6a00b37d0 2020-11-28T22:26:44.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.576+0000 7ff6a64bf700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff6a00b37d0 0x7ff6a00b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:44.580 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.576+0000 7ff6a5cbe700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7ff6a0066a50 0x7ff6a0144620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:53624/0 (socket says 172.21.15.162:53624) 2020-11-28T22:26:44.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.576+0000 7ff6a64bf700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff6a00b37d0 0x7ff6a00b3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46782/0 (socket says 172.21.15.162:46782) 2020-11-28T22:26:44.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.576+0000 7ff6a5cbe700 1 -- 172.21.15.162:0/678186795 learned_addr learned my addr 172.21.15.162:0/678186795 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T22:26:44.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.576+0000 7ff6a54bd700 1 --2- 172.21.15.162:0/678186795 >> v2:172.21.15.118:3300/0 conn(0x7ff6a01409d0 0x7ff6a0066e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:44.581 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.576+0000 7ff6a64bf700 1 -- 172.21.15.162:0/678186795 >> v2:172.21.15.118:3300/0 conn(0x7ff6a01409d0 msgr2=0x7ff6a0066e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T22:26:44.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.576+0000 7ff6a64bf700 1 --2- 172.21.15.162:0/678186795 >> v2:172.21.15.118:3300/0 conn(0x7ff6a01409d0 0x7ff6a0066e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:44.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.576+0000 7ff6a64bf700 1 -- 172.21.15.162:0/678186795 >> v1:172.21.15.162:6789/0 conn(0x7ff6a0066a50 legacy=0x7ff6a0144620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:44.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.576+0000 7ff6a4cbc700 1 -- 172.21.15.162:0/678186795 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3586478585 0 0) 0x7ff6a00b1dd0 con 0x7ff6a0066a50 2020-11-28T22:26:44.582 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.576+0000 7ff6a64bf700 1 -- 172.21.15.162:0/678186795 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a0144180 con 0x7ff6a00b37d0 2020-11-28T22:26:44.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.576+0000 7ff6a64bf700 1 --2- 172.21.15.162:0/678186795 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6a00b37d0 0x7ff6a00b3c40 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 rx=0x7ff6a005c9b0 tx=0x7ff6980091a0).ready entity=mon.1 client_cookie=f69f67715c06ca6b server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:44.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.576+0000 7ff6a4cbc700 1 -- 172.21.15.162:0/678186795 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff69800b960 con 0x7ff6a00b37d0 2020-11-28T22:26:44.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.576+0000 7ff6a4cbc700 1 -- 172.21.15.162:0/678186795 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69800bac0 con 0x7ff6a00b37d0 2020-11-28T22:26:44.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.576+0000 7ff6a4cbc700 1 -- 172.21.15.162:0/678186795 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff69800bc30 con 0x7ff6a00b37d0 2020-11-28T22:26:44.583 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.576+0000 7ff6a7f22700 1 -- 172.21.15.162:0/678186795 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6a00b37d0 msgr2=0x7ff6a00b3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:44.584 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.576+0000 7ff6a7f22700 1 --2- 172.21.15.162:0/678186795 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6a00b37d0 0x7ff6a00b3c40 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 rx=0x7ff6a005c9b0 tx=0x7ff6980091a0).stop 2020-11-28T22:26:44.586 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.576+0000 7ff6a7f22700 1 -- 172.21.15.162:0/678186795 shutdown_connections 2020-11-28T22:26:44.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.576+0000 7ff6a7f22700 1 --2- 172.21.15.162:0/678186795 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6a00b37d0 0x7ff6a00b3c40 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:44.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.576+0000 7ff6a7f22700 1 --2- 172.21.15.162:0/678186795 >> v2:172.21.15.118:3300/0 conn(0x7ff6a01409d0 0x7ff6a0066e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:44.587 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.576+0000 7ff6a7f22700 1 -- 172.21.15.162:0/678186795 >> 172.21.15.162:0/678186795 conn(0x7ff6a005b970 msgr2=0x7ff6a005bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:26:44.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.576+0000 7ff6a7f22700 1 -- 172.21.15.162:0/678186795 shutdown_connections 2020-11-28T22:26:44.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.576+0000 7ff6a7f22700 1 -- 172.21.15.162:0/678186795 wait complete. 2020-11-28T22:26:44.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.580+0000 7ff6a7f22700 1 Processor -- start 2020-11-28T22:26:44.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.580+0000 7ff6a7f22700 1 -- start start 2020-11-28T22:26:44.588 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.580+0000 7ff6a7f22700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6a0066a50 0x7ff6a00adb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:44.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.580+0000 7ff6a7f22700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6a00b37d0 0x7ff6a014b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:44.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.580+0000 7ff6a7f22700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff6a01409d0 0x7ff6a00afb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:44.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.580+0000 7ff6a7f22700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a009ce90 con 0x7ff6a01409d0 2020-11-28T22:26:44.589 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.580+0000 7ff6a7f22700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a009d130 con 0x7ff6a0066a50 2020-11-28T22:26:44.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.580+0000 7ff6a7f22700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a00af420 con 0x7ff6a00b37d0 2020-11-28T22:26:44.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.580+0000 7ff6a5cbe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6a0066a50 0x7ff6a00adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:44.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.580+0000 7ff6a5cbe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6a0066a50 0x7ff6a00adb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46784/0 (socket says 172.21.15.162:46784) 2020-11-28T22:26:44.590 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.580+0000 7ff6a5cbe700 1 -- 172.21.15.162:0/4126438709 learned_addr learned my addr 172.21.15.162:0/4126438709 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T22:26:44.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.580+0000 7ff6a64bf700 1 --2- 172.21.15.162:0/4126438709 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff6a01409d0 0x7ff6a00afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:44.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.580+0000 7ff6a5cbe700 1 -- 172.21.15.162:0/4126438709 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6a00b37d0 msgr2=0x7ff6a014b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:44.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.580+0000 7ff6a54bd700 1 --2- 172.21.15.162:0/4126438709 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6a00b37d0 0x7ff6a014b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:44.591 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.580+0000 7ff6a5cbe700 1 --2- 172.21.15.162:0/4126438709 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6a00b37d0 0x7ff6a014b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:44.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.580+0000 7ff6a5cbe700 1 -- 172.21.15.162:0/4126438709 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff6a01409d0 msgr2=0x7ff6a00afb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:44.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.580+0000 7ff6a5cbe700 1 --2- 172.21.15.162:0/4126438709 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff6a01409d0 0x7ff6a00afb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:44.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.580+0000 7ff6a5cbe700 1 -- 172.21.15.162:0/4126438709 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a00af6c0 con 0x7ff6a0066a50 2020-11-28T22:26:44.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.580+0000 7ff6a54bd700 1 --2- 172.21.15.162:0/4126438709 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff6a00b37d0 0x7ff6a014b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T22:26:44.592 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.580+0000 7ff6a64bf700 1 --2- 172.21.15.162:0/4126438709 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff6a01409d0 0x7ff6a00afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T22:26:44.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.584+0000 7ff6a5cbe700 1 --2- 172.21.15.162:0/4126438709 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6a0066a50 0x7ff6a00adb90 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 rx=0x7ff69c000b80 tx=0x7ff69c007a30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:44.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.584+0000 7ff692ffd700 1 -- 172.21.15.162:0/4126438709 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff69c007320 con 0x7ff6a0066a50 2020-11-28T22:26:44.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.584+0000 7ff692ffd700 1 -- 172.21.15.162:0/4126438709 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69c007480 con 0x7ff6a0066a50 2020-11-28T22:26:44.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.584+0000 7ff6a7f22700 1 -- 172.21.15.162:0/4126438709 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6a009cbc0 con 0x7ff6a0066a50 2020-11-28T22:26:44.593 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.584+0000 7ff692ffd700 1 -- 172.21.15.162:0/4126438709 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff69c0057d0 con 0x7ff6a0066a50 2020-11-28T22:26:44.594 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.584+0000 7ff6a7f22700 1 -- 172.21.15.162:0/4126438709 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a00b0050 con 0x7ff6a0066a50 2020-11-28T22:26:44.594 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.584+0000 7ff6a7f22700 1 -- 172.21.15.162:0/4126438709 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff6a00af850 con 0x7ff6a0066a50 2020-11-28T22:26:44.595 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.584+0000 7ff692ffd700 1 -- 172.21.15.162:0/4126438709 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff69c010030 con 0x7ff6a0066a50 2020-11-28T22:26:44.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.584+0000 7ff692ffd700 1 --2- 172.21.15.162:0/4126438709 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff680055350 0x7ff680057810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:44.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.584+0000 7ff692ffd700 1 -- 172.21.15.162:0/4126438709 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5339+0+0 (secure 0 0 0) 0x7ff69c06db20 con 0x7ff6a0066a50 2020-11-28T22:26:44.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.584+0000 7ff6a54bd700 1 --2- 172.21.15.162:0/4126438709 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff680055350 0x7ff680057810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:44.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.584+0000 7ff692ffd700 1 --2- 172.21.15.162:0/4126438709 >> [v2:172.21.15.162:6810/12313,v1:172.21.15.162:6811/12313] conn(0x7ff68005bb00 0x7ff68005df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:44.596 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.584+0000 7ff692ffd700 1 -- 172.21.15.162:0/4126438709 --> [v2:172.21.15.162:6810/12313,v1:172.21.15.162:6811/12313] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff68005e5a0 con 0x7ff68005bb00 2020-11-28T22:26:44.597 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.584+0000 7ff692ffd700 1 -- 172.21.15.162:0/4126438709 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=163) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff69c072240 con 0x7ff6a0066a50 2020-11-28T22:26:44.597 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.584+0000 7ff6a64bf700 1 --2- 172.21.15.162:0/4126438709 >> [v2:172.21.15.162:6810/12313,v1:172.21.15.162:6811/12313] conn(0x7ff68005bb00 0x7ff68005df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:44.597 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.584+0000 7ff6a54bd700 1 --2- 172.21.15.162:0/4126438709 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff680055350 0x7ff680057810 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7ff69400b000 tx=0x7ff694008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:44.597 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.584+0000 7ff6a64bf700 1 --2- 172.21.15.162:0/4126438709 >> [v2:172.21.15.162:6810/12313,v1:172.21.15.162:6811/12313] conn(0x7ff68005bb00 0x7ff68005df20 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:44.598 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.588+0000 7ff692ffd700 1 -- 172.21.15.162:0/4126438709 <== osd.0 v2:172.21.15.162:6810/12313 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7ff68005e5a0 con 0x7ff68005bb00 2020-11-28T22:26:44.610 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.608+0000 7ff6a7f22700 1 -- 172.21.15.162:0/4126438709 --> [v2:172.21.15.162:6810/12313,v1:172.21.15.162:6811/12313] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff6a0057050 con 0x7ff68005bb00 2020-11-28T22:26:44.612 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.608+0000 7ff692ffd700 1 -- 172.21.15.162:0/4126438709 <== osd.0 v2:172.21.15.162:6810/12313 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7ff6a0057050 con 0x7ff68005bb00 2020-11-28T22:26:44.612 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.608+0000 7ff6a7f22700 1 -- 172.21.15.162:0/4126438709 >> [v2:172.21.15.162:6810/12313,v1:172.21.15.162:6811/12313] conn(0x7ff68005bb00 msgr2=0x7ff68005df20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:44.612 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.608+0000 7ff6a7f22700 1 --2- 172.21.15.162:0/4126438709 >> [v2:172.21.15.162:6810/12313,v1:172.21.15.162:6811/12313] conn(0x7ff68005bb00 0x7ff68005df20 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:44.612 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.608+0000 7ff6a7f22700 1 -- 172.21.15.162:0/4126438709 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff680055350 msgr2=0x7ff680057810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:44.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.608+0000 7ff6a7f22700 1 --2- 172.21.15.162:0/4126438709 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff680055350 0x7ff680057810 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7ff69400b000 tx=0x7ff694008040).stop 2020-11-28T22:26:44.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.608+0000 7ff6a7f22700 1 -- 172.21.15.162:0/4126438709 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6a0066a50 msgr2=0x7ff6a00adb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:44.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.608+0000 7ff6a7f22700 1 --2- 172.21.15.162:0/4126438709 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6a0066a50 0x7ff6a00adb90 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 rx=0x7ff69c000b80 tx=0x7ff69c007a30).stop 2020-11-28T22:26:44.613 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.608+0000 7ff6a64bf700 1 -- 172.21.15.162:0/4126438709 reap_dead start 2020-11-28T22:26:44.614 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2020-11-28T22:26:44.614 INFO:teuthology.orchestra.run.smithi162.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2020-11-28T22:26:44.614 INFO:teuthology.orchestra.run.smithi162.stdout:} 2020-11-28T22:26:44.615 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.608+0000 7ff6a7f22700 1 -- 172.21.15.162:0/4126438709 shutdown_connections 2020-11-28T22:26:44.615 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.608+0000 7ff6a7f22700 1 -- 172.21.15.162:0/4126438709 >> 172.21.15.162:0/4126438709 conn(0x7ff6a005b970 msgr2=0x7ff6a0142f00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:26:44.615 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.608+0000 7ff6a7f22700 1 -- 172.21.15.162:0/4126438709 shutdown_connections 2020-11-28T22:26:44.616 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.608+0000 7ff6a7f22700 1 -- 172.21.15.162:0/4126438709 wait complete. 2020-11-28T22:26:44.625 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd deep-scrub 0 2020-11-28T22:26:44.742 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.740+0000 7f3a58c3a700 1 Processor -- start 2020-11-28T22:26:44.742 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.740+0000 7f3a58c3a700 1 -- start start 2020-11-28T22:26:44.743 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.740+0000 7f3a58c3a700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3a54064c70 0x7f3a54066290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:44.744 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.740+0000 7f3a58c3a700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f3a5413f6c0 0x7f3a54066780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:44.744 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.740+0000 7f3a58c3a700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a54065050 con 0x7f3a54066c70 2020-11-28T22:26:44.744 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.740+0000 7f3a58c3a700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a5413fc40 con 0x7f3a5413f6c0 2020-11-28T22:26:44.744 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.740+0000 7f3a58c3a700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5413fee0 con 0x7f3a54064c70 2020-11-28T22:26:44.745 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.740+0000 7f3a537fe700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3a54064c70 0x7f3a54066290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:44.745 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.740+0000 7f3a53fff700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f3a54066c70 0x7f3a54142c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.162:43790/0 (socket says 172.21.15.162:43790) 2020-11-28T22:26:44.745 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.740+0000 7f3a4affd700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f3a5413f6c0 0x7f3a54066780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:44.745 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.740+0000 7f3a53fff700 1 -- 172.21.15.162:0/4045634120 learned_addr learned my addr 172.21.15.162:0/4045634120 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T22:26:44.747 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.740+0000 7f3a4affd700 1 -- 172.21.15.162:0/4045634120 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3a54064c70 msgr2=0x7f3a54066290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T22:26:44.747 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.740+0000 7f3a4affd700 1 --2- 172.21.15.162:0/4045634120 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3a54064c70 0x7f3a54066290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:44.747 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.740+0000 7f3a4affd700 1 -- 172.21.15.162:0/4045634120 >> v1:172.21.15.118:6789/0 conn(0x7f3a54066c70 legacy=0x7f3a54142c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:44.747 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.740+0000 7f3a4affd700 1 -- 172.21.15.162:0/4045634120 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a54140180 con 0x7f3a5413f6c0 2020-11-28T22:26:44.748 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.740+0000 7f3a537fe700 1 --2- 172.21.15.162:0/4045634120 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3a54064c70 0x7f3a54066290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T22:26:44.748 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.740+0000 7f3a4affd700 1 --2- 172.21.15.162:0/4045634120 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3a5413f6c0 0x7f3a54066780 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7f3a3c002f30 tx=0x7f3a3c006dc0).ready entity=mon.0 client_cookie=97521ce84289bb78 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:44.748 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.740+0000 7f3a52ffd700 1 -- 172.21.15.162:0/4045634120 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a3c009760 con 0x7f3a5413f6c0 2020-11-28T22:26:44.748 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.744+0000 7f3a52ffd700 1 -- 172.21.15.162:0/4045634120 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a3c0098c0 con 0x7f3a5413f6c0 2020-11-28T22:26:44.749 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.744+0000 7f3a52ffd700 1 -- 172.21.15.162:0/4045634120 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a3c009c20 con 0x7f3a5413f6c0 2020-11-28T22:26:44.749 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.744+0000 7f3a58c3a700 1 -- 172.21.15.162:0/4045634120 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3a5413f6c0 msgr2=0x7f3a54066780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:44.749 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.744+0000 7f3a58c3a700 1 --2- 172.21.15.162:0/4045634120 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3a5413f6c0 0x7f3a54066780 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7f3a3c002f30 tx=0x7f3a3c006dc0).stop 2020-11-28T22:26:44.749 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.744+0000 7f3a58c3a700 1 -- 172.21.15.162:0/4045634120 shutdown_connections 2020-11-28T22:26:44.750 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.744+0000 7f3a58c3a700 1 --2- 172.21.15.162:0/4045634120 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3a54064c70 0x7f3a54066290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:44.750 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.744+0000 7f3a58c3a700 1 --2- 172.21.15.162:0/4045634120 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3a5413f6c0 0x7f3a54066780 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:44.750 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.744+0000 7f3a58c3a700 1 -- 172.21.15.162:0/4045634120 >> 172.21.15.162:0/4045634120 conn(0x7f3a540b5da0 msgr2=0x7f3a540b6180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:26:44.750 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.744+0000 7f3a58c3a700 1 -- 172.21.15.162:0/4045634120 shutdown_connections 2020-11-28T22:26:44.750 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.744+0000 7f3a58c3a700 1 -- 172.21.15.162:0/4045634120 wait complete. 2020-11-28T22:26:44.753 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.744+0000 7f3a58c3a700 1 Processor -- start 2020-11-28T22:26:44.753 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.744+0000 7f3a58c3a700 1 -- start start 2020-11-28T22:26:44.753 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.744+0000 7f3a58c3a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a54064c70 0x7f3a5413af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:44.753 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.744+0000 7f3a58c3a700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3a54066c70 0x7f3a540b5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:44.754 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.744+0000 7f3a58c3a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3a5413f6c0 0x7f3a540af870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:44.754 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.744+0000 7f3a58c3a700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a54138810 con 0x7f3a5413f6c0 2020-11-28T22:26:44.754 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.744+0000 7f3a58c3a700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a54139530 con 0x7f3a54064c70 2020-11-28T22:26:44.754 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.744+0000 7f3a58c3a700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a540b3ef0 con 0x7f3a54066c70 2020-11-28T22:26:44.754 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.744+0000 7f3a537fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a54064c70 0x7f3a5413af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:44.755 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.744+0000 7f3a4affd700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3a54066c70 0x7f3a540b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:44.755 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.744+0000 7f3a537fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a54064c70 0x7f3a5413af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46802/0 (socket says 172.21.15.162:46802) 2020-11-28T22:26:44.755 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.744+0000 7f3a4affd700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3a54066c70 0x7f3a540b5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.162:44024/0 (socket says 172.21.15.162:44024) 2020-11-28T22:26:44.755 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.744+0000 7f3a537fe700 1 -- 172.21.15.162:0/2988738656 learned_addr learned my addr 172.21.15.162:0/2988738656 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T22:26:44.756 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.744+0000 7f3a53fff700 1 --2- 172.21.15.162:0/2988738656 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3a5413f6c0 0x7f3a540af870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:44.756 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.748+0000 7f3a537fe700 1 -- 172.21.15.162:0/2988738656 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3a54066c70 msgr2=0x7f3a540b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:44.756 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.748+0000 7f3a537fe700 1 --2- 172.21.15.162:0/2988738656 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3a54066c70 0x7f3a540b5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:44.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.748+0000 7f3a537fe700 1 -- 172.21.15.162:0/2988738656 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3a5413f6c0 msgr2=0x7f3a540af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:44.758 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.748+0000 7f3a537fe700 1 --2- 172.21.15.162:0/2988738656 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3a5413f6c0 0x7f3a540af870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:44.758 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.748+0000 7f3a537fe700 1 -- 172.21.15.162:0/2988738656 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a54138a40 con 0x7f3a54064c70 2020-11-28T22:26:44.758 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.748+0000 7f3a537fe700 1 --2- 172.21.15.162:0/2988738656 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a54064c70 0x7f3a5413af00 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 rx=0x7f3a4400c000 tx=0x7f3a44006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:44.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.748+0000 7f3a517fa700 1 -- 172.21.15.162:0/2988738656 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a4400cb40 con 0x7f3a54064c70 2020-11-28T22:26:44.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.748+0000 7f3a517fa700 1 -- 172.21.15.162:0/2988738656 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a4400cca0 con 0x7f3a54064c70 2020-11-28T22:26:44.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.748+0000 7f3a58c3a700 1 -- 172.21.15.162:0/2988738656 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a540b3c50 con 0x7f3a54064c70 2020-11-28T22:26:44.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.748+0000 7f3a517fa700 1 -- 172.21.15.162:0/2988738656 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a4400b570 con 0x7f3a54064c70 2020-11-28T22:26:44.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.748+0000 7f3a58c3a700 1 -- 172.21.15.162:0/2988738656 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a5405c5c0 con 0x7f3a54064c70 2020-11-28T22:26:44.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.748+0000 7f3a58c3a700 1 -- 172.21.15.162:0/2988738656 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a541396a0 con 0x7f3a54064c70 2020-11-28T22:26:44.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.748+0000 7f3a517fa700 1 -- 172.21.15.162:0/2988738656 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f3a4401c460 con 0x7f3a54064c70 2020-11-28T22:26:44.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.752+0000 7f3a517fa700 1 --2- 172.21.15.162:0/2988738656 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3a2c0554e0 0x7f3a2c0579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:44.761 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.752+0000 7f3a517fa700 1 -- 172.21.15.162:0/2988738656 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f3a4406e1f0 con 0x7f3a54064c70 2020-11-28T22:26:44.761 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.752+0000 7f3a4affd700 1 --2- 172.21.15.162:0/2988738656 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3a2c0554e0 0x7f3a2c0579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:44.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.752+0000 7f3a4affd700 1 --2- 172.21.15.162:0/2988738656 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3a2c0554e0 0x7f3a2c0579a0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7f3a3c01a000 tx=0x7f3a3c015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:44.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.756+0000 7f3a517fa700 1 -- 172.21.15.162:0/2988738656 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f3a44044770 con 0x7f3a54064c70 2020-11-28T22:26:44.900 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.896+0000 7f3a58c3a700 1 -- 172.21.15.162:0/2988738656 --> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a54153090 con 0x7f3a2c0554e0 2020-11-28T22:26:44.900 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.896+0000 7f3a517fa700 1 -- 172.21.15.162:0/2988738656 <== mgr.4105 v2:172.21.15.162:6800/12156 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3a54153090 con 0x7f3a2c0554e0 2020-11-28T22:26:44.902 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.900+0000 7f3a58c3a700 1 -- 172.21.15.162:0/2988738656 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3a2c0554e0 msgr2=0x7f3a2c0579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:44.903 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.900+0000 7f3a58c3a700 1 --2- 172.21.15.162:0/2988738656 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3a2c0554e0 0x7f3a2c0579a0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7f3a3c01a000 tx=0x7f3a3c015000).stop 2020-11-28T22:26:44.903 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.900+0000 7f3a58c3a700 1 -- 172.21.15.162:0/2988738656 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a54064c70 msgr2=0x7f3a5413af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:44.903 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.900+0000 7f3a58c3a700 1 --2- 172.21.15.162:0/2988738656 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a54064c70 0x7f3a5413af00 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 rx=0x7f3a4400c000 tx=0x7f3a44006dc0).stop 2020-11-28T22:26:44.903 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.900+0000 7f3a58c3a700 1 -- 172.21.15.162:0/2988738656 shutdown_connections 2020-11-28T22:26:44.904 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.900+0000 7f3a58c3a700 1 --2- 172.21.15.162:0/2988738656 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3a2c0554e0 0x7f3a2c0579a0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:44.904 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.900+0000 7f3a58c3a700 1 --2- 172.21.15.162:0/2988738656 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3a5413f6c0 0x7f3a540af870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:44.904 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.900+0000 7f3a58c3a700 1 --2- 172.21.15.162:0/2988738656 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a54064c70 0x7f3a5413af00 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:44.904 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.900+0000 7f3a58c3a700 1 --2- 172.21.15.162:0/2988738656 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3a54066c70 0x7f3a540b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:44.904 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.900+0000 7f3a58c3a700 1 -- 172.21.15.162:0/2988738656 >> 172.21.15.162:0/2988738656 conn(0x7f3a540b5da0 msgr2=0x7f3a540610b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:26:44.905 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.900+0000 7f3a58c3a700 1 -- 172.21.15.162:0/2988738656 shutdown_connections 2020-11-28T22:26:44.905 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:44.900+0000 7f3a58c3a700 1 -- 172.21.15.162:0/2988738656 wait complete. 2020-11-28T22:26:44.905 INFO:teuthology.orchestra.run.smithi162.stderr:instructed osd(s) 0 to deep-scrub 2020-11-28T22:26:44.914 INFO:tasks.ceph:Scrubbing osd.1 2020-11-28T22:26:44.915 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.1 config set osd_debug_deep_scrub_sleep 0 2020-11-28T22:26:45.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.028+0000 7ff9e1c02700 1 Processor -- start 2020-11-28T22:26:45.031 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.028+0000 7ff9e1c02700 1 -- start start 2020-11-28T22:26:45.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.028+0000 7ff9e1c02700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff9dc0b3950 0x7ff9dc0b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:45.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.028+0000 7ff9e1c02700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9dc0b1f50 con 0x7ff9dc0b44e0 2020-11-28T22:26:45.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.028+0000 7ff9e1c02700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9dc143e90 con 0x7ff9dc0b18c0 2020-11-28T22:26:45.033 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.028+0000 7ff9e1c02700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc0aab50 con 0x7ff9dc0b3950 2020-11-28T22:26:45.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.028+0000 7ff9dbfff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7ff9dc0b18c0 0x7ff9dc13dd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:53654/0 (socket says 172.21.15.162:53654) 2020-11-28T22:26:45.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.028+0000 7ff9dbfff700 1 -- 172.21.15.162:0/1303851063 learned_addr learned my addr 172.21.15.162:0/1303851063 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T22:26:45.034 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.028+0000 7ff9e0c00700 1 --2- 172.21.15.162:0/1303851063 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff9dc0b3950 0x7ff9dc0b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:45.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.032+0000 7ff9db7fe700 1 -- 172.21.15.162:0/1303851063 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4060323251 0 0) 0x7ff9dc143e90 con 0x7ff9dc0b18c0 2020-11-28T22:26:45.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.032+0000 7ff9db7fe700 1 -- 172.21.15.162:0/1303851063 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9bc003230 con 0x7ff9dc0b18c0 2020-11-28T22:26:45.037 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.032+0000 7ff9db7fe700 1 -- 172.21.15.162:0/1303851063 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2906126803 0 0) 0x7ff9dc0b1f50 con 0x7ff9dc0b44e0 2020-11-28T22:26:45.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.032+0000 7ff9db7fe700 1 -- 172.21.15.162:0/1303851063 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9dc143e90 con 0x7ff9dc0b44e0 2020-11-28T22:26:45.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.032+0000 7ff9db7fe700 1 -- 172.21.15.162:0/1303851063 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1202186045 0 0) 0x7ff9bc003230 con 0x7ff9dc0b18c0 2020-11-28T22:26:45.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.032+0000 7ff9db7fe700 1 -- 172.21.15.162:0/1303851063 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff9dc0b3950 msgr2=0x7ff9dc0b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T22:26:45.038 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.032+0000 7ff9db7fe700 1 --2- 172.21.15.162:0/1303851063 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff9dc0b3950 0x7ff9dc0b3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:45.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.032+0000 7ff9db7fe700 1 -- 172.21.15.162:0/1303851063 >> v1:172.21.15.118:6789/0 conn(0x7ff9dc0b44e0 legacy=0x7ff9dc0b2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:45.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.032+0000 7ff9db7fe700 1 -- 172.21.15.162:0/1303851063 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9dc0aa8b0 con 0x7ff9dc0b18c0 2020-11-28T22:26:45.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.032+0000 7ff9db7fe700 1 -- 172.21.15.162:0/1303851063 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7ff9cc0016d0 con 0x7ff9dc0b18c0 2020-11-28T22:26:45.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.032+0000 7ff9e0c00700 1 --2- 172.21.15.162:0/1303851063 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff9dc0b3950 0x7ff9dc0b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T22:26:45.039 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.032+0000 7ff9db7fe700 1 -- 172.21.15.162:0/1303851063 >> v1:172.21.15.162:6789/0 conn(0x7ff9dc0b18c0 legacy=0x7ff9dc13dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:45.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.032+0000 7ff9db7fe700 1 --2- 172.21.15.162:0/1303851063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9bc0038b0 0x7ff9bc003cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:45.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.032+0000 7ff9db7fe700 1 --2- 172.21.15.162:0/1303851063 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff9dc0b3950 0x7ff9bc0044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:45.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.032+0000 7ff9db7fe700 1 --2- 172.21.15.162:0/1303851063 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9bc0049c0 0x7ff9bc004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:45.040 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.032+0000 7ff9db7fe700 1 -- 172.21.15.162:0/1303851063 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc0aab50 con 0x7ff9bc0049c0 2020-11-28T22:26:45.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.032+0000 7ff9db7fe700 1 -- 172.21.15.162:0/1303851063 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9bc005330 con 0x7ff9bc0038b0 2020-11-28T22:26:45.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.032+0000 7ff9db7fe700 1 -- 172.21.15.162:0/1303851063 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9bc00b720 con 0x7ff9dc0b3950 2020-11-28T22:26:45.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.032+0000 7ff9e0c00700 1 --2- 172.21.15.162:0/1303851063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9bc0038b0 0x7ff9bc003cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:45.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.032+0000 7ff9dbfff700 1 --2- 172.21.15.162:0/1303851063 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff9dc0b3950 0x7ff9bc0044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:45.041 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.032+0000 7ff9e1401700 1 --2- 172.21.15.162:0/1303851063 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9bc0049c0 0x7ff9bc004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:45.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.032+0000 7ff9e0c00700 1 -- 172.21.15.162:0/1303851063 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff9dc0b3950 msgr2=0x7ff9bc0044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T22:26:45.042 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.032+0000 7ff9e0c00700 1 --2- 172.21.15.162:0/1303851063 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff9dc0b3950 0x7ff9bc0044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:45.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.032+0000 7ff9e0c00700 1 -- 172.21.15.162:0/1303851063 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9bc0049c0 msgr2=0x7ff9bc004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T22:26:45.043 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.032+0000 7ff9e0c00700 1 --2- 172.21.15.162:0/1303851063 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9bc0049c0 0x7ff9bc004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:45.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.032+0000 7ff9e0c00700 1 -- 172.21.15.162:0/1303851063 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9bc003230 con 0x7ff9bc0038b0 2020-11-28T22:26:45.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.032+0000 7ff9e1401700 1 --2- 172.21.15.162:0/1303851063 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9bc0049c0 0x7ff9bc004e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T22:26:45.044 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.032+0000 7ff9e0c00700 1 --2- 172.21.15.162:0/1303851063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9bc0038b0 0x7ff9bc003cb0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 rx=0x7ff9c400c000 tx=0x7ff9c4008040).ready entity=mon.1 client_cookie=344abf7e5d99cf01 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:45.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.032+0000 7ff9db7fe700 1 -- 172.21.15.162:0/1303851063 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9c400b4d0 con 0x7ff9bc0038b0 2020-11-28T22:26:45.045 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.032+0000 7ff9db7fe700 1 -- 172.21.15.162:0/1303851063 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c400eb80 con 0x7ff9bc0038b0 2020-11-28T22:26:45.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.032+0000 7ff9db7fe700 1 -- 172.21.15.162:0/1303851063 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9c40123f0 con 0x7ff9bc0038b0 2020-11-28T22:26:45.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.032+0000 7ff9e1c02700 1 -- 172.21.15.162:0/1303851063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9bc0038b0 msgr2=0x7ff9bc003cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:45.046 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.032+0000 7ff9e1c02700 1 --2- 172.21.15.162:0/1303851063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9bc0038b0 0x7ff9bc003cb0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 rx=0x7ff9c400c000 tx=0x7ff9c4008040).stop 2020-11-28T22:26:45.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.032+0000 7ff9e1401700 1 -- 172.21.15.162:0/1303851063 reap_dead start 2020-11-28T22:26:45.047 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.032+0000 7ff9e1c02700 1 -- 172.21.15.162:0/1303851063 shutdown_connections 2020-11-28T22:26:45.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.032+0000 7ff9e1c02700 1 -- 172.21.15.162:0/1303851063 >> 172.21.15.162:0/1303851063 conn(0x7ff9dc062630 msgr2=0x7ff9dc062a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:26:45.048 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.032+0000 7ff9e1c02700 1 -- 172.21.15.162:0/1303851063 shutdown_connections 2020-11-28T22:26:45.049 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.032+0000 7ff9e1c02700 1 -- 172.21.15.162:0/1303851063 wait complete. 2020-11-28T22:26:45.054 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.036+0000 7ff9e1c02700 1 Processor -- start 2020-11-28T22:26:45.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.036+0000 7ff9e1c02700 1 -- start start 2020-11-28T22:26:45.055 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.036+0000 7ff9e1c02700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9dc0b4d30 0x7ff9dc0b5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:45.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.036+0000 7ff9e1c02700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9dc061030 0x7ff9dc060930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:45.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.036+0000 7ff9e1c02700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff9dc155450 0x7ff9dc155900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:45.056 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.036+0000 7ff9e1c02700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc143630 con 0x7ff9dc0b4d30 2020-11-28T22:26:45.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.036+0000 7ff9e1c02700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc1464a0 con 0x7ff9dc061030 2020-11-28T22:26:45.057 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.036+0000 7ff9e1c02700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc155df0 con 0x7ff9dc155450 2020-11-28T22:26:45.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.036+0000 7ff9dbfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9dc061030 0x7ff9dc060930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:45.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.036+0000 7ff9dbfff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9dc061030 0x7ff9dc060930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46820/0 (socket says 172.21.15.162:46820) 2020-11-28T22:26:45.058 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.036+0000 7ff9dbfff700 1 -- 172.21.15.162:0/800343293 learned_addr learned my addr 172.21.15.162:0/800343293 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T22:26:45.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.036+0000 7ff9e0c00700 1 --2- 172.21.15.162:0/800343293 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9dc0b4d30 0x7ff9dc0b5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:45.059 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.036+0000 7ff9e1401700 1 --2- 172.21.15.162:0/800343293 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff9dc155450 0x7ff9dc155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:45.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.036+0000 7ff9dbfff700 1 -- 172.21.15.162:0/800343293 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff9dc155450 msgr2=0x7ff9dc155900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:45.060 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.036+0000 7ff9dbfff700 1 --2- 172.21.15.162:0/800343293 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff9dc155450 0x7ff9dc155900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:45.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.036+0000 7ff9dbfff700 1 -- 172.21.15.162:0/800343293 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9dc0b4d30 msgr2=0x7ff9dc0b5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:45.061 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.036+0000 7ff9dbfff700 1 --2- 172.21.15.162:0/800343293 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9dc0b4d30 0x7ff9dc0b5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:45.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.036+0000 7ff9dbfff700 1 -- 172.21.15.162:0/800343293 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9dc1466d0 con 0x7ff9dc061030 2020-11-28T22:26:45.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.036+0000 7ff9e1401700 1 --2- 172.21.15.162:0/800343293 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff9dc155450 0x7ff9dc155900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T22:26:45.062 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.036+0000 7ff9dbfff700 1 --2- 172.21.15.162:0/800343293 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9dc061030 0x7ff9dc060930 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 rx=0x7ff9dc155330 tx=0x7ff9dc066b20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:45.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.036+0000 7ff9d9ffb700 1 -- 172.21.15.162:0/800343293 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9cc00fa30 con 0x7ff9dc061030 2020-11-28T22:26:45.063 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.036+0000 7ff9d9ffb700 1 -- 172.21.15.162:0/800343293 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9cc00fb90 con 0x7ff9dc061030 2020-11-28T22:26:45.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.036+0000 7ff9e1c02700 1 -- 172.21.15.162:0/800343293 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9dc146970 con 0x7ff9dc061030 2020-11-28T22:26:45.064 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.036+0000 7ff9d9ffb700 1 -- 172.21.15.162:0/800343293 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9cc012350 con 0x7ff9dc061030 2020-11-28T22:26:45.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.036+0000 7ff9e1c02700 1 -- 172.21.15.162:0/800343293 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9dc0a98d0 con 0x7ff9dc061030 2020-11-28T22:26:45.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.036+0000 7ff9e1c02700 1 -- 172.21.15.162:0/800343293 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff9dc1551b0 con 0x7ff9dc061030 2020-11-28T22:26:45.065 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.036+0000 7ff9d9ffb700 1 -- 172.21.15.162:0/800343293 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff9cc0124b0 con 0x7ff9dc061030 2020-11-28T22:26:45.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.040+0000 7ff9d9ffb700 1 --2- 172.21.15.162:0/800343293 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff9c0055440 0x7ff9c0057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:45.066 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.040+0000 7ff9d9ffb700 1 -- 172.21.15.162:0/800343293 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5339+0+0 (secure 0 0 0) 0x7ff9cc06d680 con 0x7ff9dc061030 2020-11-28T22:26:45.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.040+0000 7ff9e0c00700 1 --2- 172.21.15.162:0/800343293 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff9c0055440 0x7ff9c0057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:45.067 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.040+0000 7ff9d9ffb700 1 --2- 172.21.15.162:0/800343293 >> [v2:172.21.15.162:6802/12311,v1:172.21.15.162:6803/12311] conn(0x7ff9c005bb50 0x7ff9c005df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:45.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.040+0000 7ff9d9ffb700 1 -- 172.21.15.162:0/800343293 --> [v2:172.21.15.162:6802/12311,v1:172.21.15.162:6803/12311] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff9c005e550 con 0x7ff9c005bb50 2020-11-28T22:26:45.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.040+0000 7ff9d9ffb700 1 -- 172.21.15.162:0/800343293 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=163) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9cc06fcf0 con 0x7ff9dc061030 2020-11-28T22:26:45.068 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.040+0000 7ff9e1401700 1 --2- 172.21.15.162:0/800343293 >> [v2:172.21.15.162:6802/12311,v1:172.21.15.162:6803/12311] conn(0x7ff9c005bb50 0x7ff9c005df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:45.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.040+0000 7ff9e0c00700 1 --2- 172.21.15.162:0/800343293 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff9c0055440 0x7ff9c0057900 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7ff9c400c000 tx=0x7ff9c401a000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:45.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.040+0000 7ff9e1401700 1 --2- 172.21.15.162:0/800343293 >> [v2:172.21.15.162:6802/12311,v1:172.21.15.162:6803/12311] conn(0x7ff9c005bb50 0x7ff9c005df70 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:45.069 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.040+0000 7ff9d9ffb700 1 -- 172.21.15.162:0/800343293 <== osd.1 v2:172.21.15.162:6802/12311 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7ff9c005e550 con 0x7ff9c005bb50 2020-11-28T22:26:45.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.060+0000 7ff9e1c02700 1 -- 172.21.15.162:0/800343293 --> [v2:172.21.15.162:6802/12311,v1:172.21.15.162:6803/12311] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff9dc0aa180 con 0x7ff9c005bb50 2020-11-28T22:26:45.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.060+0000 7ff9d9ffb700 1 -- 172.21.15.162:0/800343293 <== osd.1 v2:172.21.15.162:6802/12311 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7ff9dc0aa180 con 0x7ff9c005bb50 2020-11-28T22:26:45.071 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.064+0000 7ff9e1c02700 1 -- 172.21.15.162:0/800343293 >> [v2:172.21.15.162:6802/12311,v1:172.21.15.162:6803/12311] conn(0x7ff9c005bb50 msgr2=0x7ff9c005df70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:45.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.064+0000 7ff9e1c02700 1 --2- 172.21.15.162:0/800343293 >> [v2:172.21.15.162:6802/12311,v1:172.21.15.162:6803/12311] conn(0x7ff9c005bb50 0x7ff9c005df70 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:45.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.064+0000 7ff9e1c02700 1 -- 172.21.15.162:0/800343293 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff9c0055440 msgr2=0x7ff9c0057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:45.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.064+0000 7ff9e1c02700 1 --2- 172.21.15.162:0/800343293 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff9c0055440 0x7ff9c0057900 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7ff9c400c000 tx=0x7ff9c401a000).stop 2020-11-28T22:26:45.072 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.064+0000 7ff9e1c02700 1 -- 172.21.15.162:0/800343293 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9dc061030 msgr2=0x7ff9dc060930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:45.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.064+0000 7ff9e1c02700 1 --2- 172.21.15.162:0/800343293 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9dc061030 0x7ff9dc060930 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 rx=0x7ff9dc155330 tx=0x7ff9dc066b20).stop 2020-11-28T22:26:45.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.064+0000 7ff9e1401700 1 -- 172.21.15.162:0/800343293 reap_dead start 2020-11-28T22:26:45.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.064+0000 7ff9e1c02700 1 -- 172.21.15.162:0/800343293 shutdown_connections 2020-11-28T22:26:45.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.064+0000 7ff9e1c02700 1 -- 172.21.15.162:0/800343293 >> 172.21.15.162:0/800343293 conn(0x7ff9dc062630 msgr2=0x7ff9dc062a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:26:45.073 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.064+0000 7ff9e1c02700 1 -- 172.21.15.162:0/800343293 shutdown_connections 2020-11-28T22:26:45.074 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.064+0000 7ff9e1c02700 1 -- 172.21.15.162:0/800343293 wait complete. 2020-11-28T22:26:45.074 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2020-11-28T22:26:45.074 INFO:teuthology.orchestra.run.smithi162.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2020-11-28T22:26:45.075 INFO:teuthology.orchestra.run.smithi162.stdout:} 2020-11-28T22:26:45.078 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd deep-scrub 1 2020-11-28T22:26:45.181 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.180+0000 7ff03e616700 1 Processor -- start 2020-11-28T22:26:45.182 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.180+0000 7ff03e616700 1 -- start start 2020-11-28T22:26:45.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.180+0000 7ff03e616700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff038066a10 0x7ff038061760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:45.189 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.180+0000 7ff03e616700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff038066df0 con 0x7ff038143da0 2020-11-28T22:26:45.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.180+0000 7ff03e616700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0380b12b0 con 0x7ff0380b1500 2020-11-28T22:26:45.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.180+0000 7ff03e616700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff038060720 con 0x7ff038066a10 2020-11-28T22:26:45.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.180+0000 7ff03d614700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff038066a10 0x7ff038061760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:45.190 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.180+0000 7ff03de15700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7ff0380b1500 0x7ff03805c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:53678/0 (socket says 172.21.15.162:53678) 2020-11-28T22:26:45.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.180+0000 7ff03de15700 1 -- 172.21.15.162:0/2880632644 learned_addr learned my addr 172.21.15.162:0/2880632644 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T22:26:45.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.180+0000 7ff027fff700 1 -- 172.21.15.162:0/2880632644 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2503992946 0 0) 0x7ff0380b12b0 con 0x7ff0380b1500 2020-11-28T22:26:45.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.180+0000 7ff027fff700 1 -- 172.21.15.162:0/2880632644 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff020003230 con 0x7ff0380b1500 2020-11-28T22:26:45.191 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.180+0000 7ff03d614700 1 -- 172.21.15.162:0/2880632644 >> v1:172.21.15.162:6789/0 conn(0x7ff0380b1500 legacy=0x7ff03805c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:45.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.180+0000 7ff03d614700 1 -- 172.21.15.162:0/2880632644 >> v1:172.21.15.118:6789/0 conn(0x7ff038143da0 legacy=0x7ff03805b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:45.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.180+0000 7ff03d614700 1 -- 172.21.15.162:0/2880632644 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff03805c790 con 0x7ff038066a10 2020-11-28T22:26:45.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.180+0000 7ff03d614700 1 --2- 172.21.15.162:0/2880632644 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff038066a10 0x7ff038061760 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 rx=0x7ff034002f30 tx=0x7ff034006dc0).ready entity=mon.1 client_cookie=c107871b2653354b server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:45.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.180+0000 7ff027fff700 1 -- 172.21.15.162:0/2880632644 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff034009800 con 0x7ff038066a10 2020-11-28T22:26:45.192 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.180+0000 7ff027fff700 1 -- 172.21.15.162:0/2880632644 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff034009960 con 0x7ff038066a10 2020-11-28T22:26:45.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.180+0000 7ff03e616700 1 -- 172.21.15.162:0/2880632644 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff038066a10 msgr2=0x7ff038061760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:45.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.180+0000 7ff027fff700 1 -- 172.21.15.162:0/2880632644 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff034009cc0 con 0x7ff038066a10 2020-11-28T22:26:45.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.180+0000 7ff03e616700 1 --2- 172.21.15.162:0/2880632644 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff038066a10 0x7ff038061760 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 rx=0x7ff034002f30 tx=0x7ff034006dc0).stop 2020-11-28T22:26:45.193 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.184+0000 7ff03e616700 1 -- 172.21.15.162:0/2880632644 shutdown_connections 2020-11-28T22:26:45.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.184+0000 7ff03e616700 1 --2- 172.21.15.162:0/2880632644 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff038066a10 0x7ff038061760 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:45.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.184+0000 7ff03e616700 1 -- 172.21.15.162:0/2880632644 >> 172.21.15.162:0/2880632644 conn(0x7ff0380b5da0 msgr2=0x7ff0380b6180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:26:45.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.184+0000 7ff03e616700 1 -- 172.21.15.162:0/2880632644 shutdown_connections 2020-11-28T22:26:45.194 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.184+0000 7ff03e616700 1 -- 172.21.15.162:0/2880632644 wait complete. 2020-11-28T22:26:45.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.184+0000 7ff03e616700 1 Processor -- start 2020-11-28T22:26:45.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.184+0000 7ff03e616700 1 -- start start 2020-11-28T22:26:45.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.184+0000 7ff03e616700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff038066a10 0x7ff038141d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:45.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.184+0000 7ff03e616700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0380b1500 0x7ff038140ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:45.195 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.184+0000 7ff03e616700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff038143da0 0x7ff0380aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:45.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.184+0000 7ff03e616700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0381434b0 con 0x7ff038143da0 2020-11-28T22:26:45.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.184+0000 7ff03e616700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0380afbe0 con 0x7ff0380b1500 2020-11-28T22:26:45.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.184+0000 7ff03e616700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0381436c0 con 0x7ff038066a10 2020-11-28T22:26:45.196 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.184+0000 7ff03ce13700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0380b1500 0x7ff038140ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:45.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.184+0000 7ff03ce13700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0380b1500 0x7ff038140ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46840/0 (socket says 172.21.15.162:46840) 2020-11-28T22:26:45.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.184+0000 7ff03ce13700 1 -- 172.21.15.162:0/3926318405 learned_addr learned my addr 172.21.15.162:0/3926318405 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T22:26:45.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.184+0000 7ff03de15700 1 --2- 172.21.15.162:0/3926318405 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff038143da0 0x7ff0380aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:45.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.184+0000 7ff03d614700 1 --2- 172.21.15.162:0/3926318405 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff038066a10 0x7ff038141d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:45.197 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.184+0000 7ff03ce13700 1 -- 172.21.15.162:0/3926318405 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff038066a10 msgr2=0x7ff038141d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:45.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.184+0000 7ff03ce13700 1 --2- 172.21.15.162:0/3926318405 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff038066a10 0x7ff038141d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:45.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.184+0000 7ff03ce13700 1 -- 172.21.15.162:0/3926318405 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff038143da0 msgr2=0x7ff0380aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:45.198 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.184+0000 7ff03ce13700 1 --2- 172.21.15.162:0/3926318405 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff038143da0 0x7ff0380aeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:45.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.184+0000 7ff03ce13700 1 -- 172.21.15.162:0/3926318405 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0380afe10 con 0x7ff0380b1500 2020-11-28T22:26:45.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.184+0000 7ff03d614700 1 --2- 172.21.15.162:0/3926318405 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff038066a10 0x7ff038141d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T22:26:45.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.184+0000 7ff03ce13700 1 --2- 172.21.15.162:0/3926318405 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0380b1500 0x7ff038140ea0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 rx=0x7ff038140800 tx=0x7ff028007bf0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:45.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.184+0000 7ff0267fc700 1 -- 172.21.15.162:0/3926318405 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff02800e4a0 con 0x7ff0380b1500 2020-11-28T22:26:45.199 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.184+0000 7ff0267fc700 1 -- 172.21.15.162:0/3926318405 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff02800e600 con 0x7ff0380b1500 2020-11-28T22:26:45.200 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.184+0000 7ff03e616700 1 -- 172.21.15.162:0/3926318405 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0380b00b0 con 0x7ff0380b1500 2020-11-28T22:26:45.200 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.184+0000 7ff0267fc700 1 -- 172.21.15.162:0/3926318405 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff02800e960 con 0x7ff0380b1500 2020-11-28T22:26:45.200 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.184+0000 7ff03e616700 1 -- 172.21.15.162:0/3926318405 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0380b4ee0 con 0x7ff0380b1500 2020-11-28T22:26:45.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.188+0000 7ff0267fc700 1 -- 172.21.15.162:0/3926318405 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7ff028010030 con 0x7ff0380b1500 2020-11-28T22:26:45.201 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.188+0000 7ff03e616700 1 -- 172.21.15.162:0/3926318405 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff038057050 con 0x7ff0380b1500 2020-11-28T22:26:45.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.188+0000 7ff0267fc700 1 --2- 172.21.15.162:0/3926318405 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff018055330 0x7ff0180577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:45.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.188+0000 7ff0267fc700 1 -- 172.21.15.162:0/3926318405 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5339+0+0 (secure 0 0 0) 0x7ff028001e40 con 0x7ff0380b1500 2020-11-28T22:26:45.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.188+0000 7ff03d614700 1 --2- 172.21.15.162:0/3926318405 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff018055330 0x7ff0180577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:45.202 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.188+0000 7ff03d614700 1 --2- 172.21.15.162:0/3926318405 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff018055330 0x7ff0180577f0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7ff034007000 tx=0x7ff034015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:45.203 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.196+0000 7ff0267fc700 1 -- 172.21.15.162:0/3926318405 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7ff0280440c0 con 0x7ff0380b1500 2020-11-28T22:26:45.342 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.340+0000 7ff03e616700 1 -- 172.21.15.162:0/3926318405 --> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7ff0381429f0 con 0x7ff018055330 2020-11-28T22:26:45.343 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.340+0000 7ff0267fc700 1 -- 172.21.15.162:0/3926318405 <== mgr.4105 v2:172.21.15.162:6800/12156 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff03400b5e0 con 0x7ff018055330 2020-11-28T22:26:45.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.344+0000 7ff03e616700 1 -- 172.21.15.162:0/3926318405 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff018055330 msgr2=0x7ff0180577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:45.347 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.344+0000 7ff03e616700 1 --2- 172.21.15.162:0/3926318405 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff018055330 0x7ff0180577f0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7ff034007000 tx=0x7ff034015000).stop 2020-11-28T22:26:45.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.344+0000 7ff03e616700 1 -- 172.21.15.162:0/3926318405 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0380b1500 msgr2=0x7ff038140ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:45.348 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.344+0000 7ff03e616700 1 --2- 172.21.15.162:0/3926318405 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0380b1500 0x7ff038140ea0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 rx=0x7ff038140800 tx=0x7ff028007bf0).stop 2020-11-28T22:26:45.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.344+0000 7ff03e616700 1 -- 172.21.15.162:0/3926318405 shutdown_connections 2020-11-28T22:26:45.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.344+0000 7ff03e616700 1 --2- 172.21.15.162:0/3926318405 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7ff018055330 0x7ff0180577f0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:45.349 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.344+0000 7ff03e616700 1 --2- 172.21.15.162:0/3926318405 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff038066a10 0x7ff038141d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:45.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.344+0000 7ff03e616700 1 --2- 172.21.15.162:0/3926318405 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff038143da0 0x7ff0380aeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:45.350 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.344+0000 7ff03e616700 1 --2- 172.21.15.162:0/3926318405 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0380b1500 0x7ff038140ea0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:45.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.344+0000 7ff03e616700 1 -- 172.21.15.162:0/3926318405 >> 172.21.15.162:0/3926318405 conn(0x7ff0380b5da0 msgr2=0x7ff03813aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:26:45.351 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.344+0000 7ff03e616700 1 -- 172.21.15.162:0/3926318405 shutdown_connections 2020-11-28T22:26:45.352 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.344+0000 7ff03e616700 1 -- 172.21.15.162:0/3926318405 wait complete. 2020-11-28T22:26:45.352 INFO:teuthology.orchestra.run.smithi162.stderr:instructed osd(s) 1 to deep-scrub 2020-11-28T22:26:45.363 INFO:tasks.ceph:Scrubbing osd.2 2020-11-28T22:26:45.364 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.2 config set osd_debug_deep_scrub_sleep 0 2020-11-28T22:26:45.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.480+0000 7f95569c0700 1 Processor -- start 2020-11-28T22:26:45.482 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.480+0000 7f95569c0700 1 -- start start 2020-11-28T22:26:45.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.480+0000 7f95569c0700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f95500b2d00 0x7f955005c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:45.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.480+0000 7f95569c0700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f955005baf0 con 0x7f95500b3ab0 2020-11-28T22:26:45.483 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.480+0000 7f95569c0700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f955005bd90 con 0x7f95500b30e0 2020-11-28T22:26:45.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.480+0000 7f95569c0700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95500b0e20 con 0x7f95500b2d00 2020-11-28T22:26:45.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.480+0000 7f95559be700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f95500b2d00 0x7f955005c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:45.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.480+0000 7f95559be700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f95500b2d00 0x7f955005c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46844/0 (socket says 172.21.15.162:46844) 2020-11-28T22:26:45.484 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.480+0000 7f95559be700 1 -- 172.21.15.162:0/3683225387 learned_addr learned my addr 172.21.15.162:0/3683225387 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T22:26:45.486 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.480+0000 7f95559be700 1 -- 172.21.15.162:0/3683225387 >> v1:172.21.15.162:6789/0 conn(0x7f95500b30e0 legacy=0x7f955013fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:45.486 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.480+0000 7f95559be700 1 -- 172.21.15.162:0/3683225387 >> v1:172.21.15.118:6789/0 conn(0x7f95500b3ab0 legacy=0x7f9550142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:45.486 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.480+0000 7f95549bc700 1 -- 172.21.15.162:0/3683225387 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1168891112 0 0) 0x7f955005bd90 con 0x7f95500b30e0 2020-11-28T22:26:45.486 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.480+0000 7f95559be700 1 -- 172.21.15.162:0/3683225387 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f955005c030 con 0x7f95500b2d00 2020-11-28T22:26:45.487 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.480+0000 7f95559be700 1 --2- 172.21.15.162:0/3683225387 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95500b2d00 0x7f955005c4d0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 rx=0x7f954c002f30 tx=0x7f954c006e80).ready entity=mon.1 client_cookie=5b66c7ac03e42012 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:45.487 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.480+0000 7f95549bc700 1 -- 172.21.15.162:0/3683225387 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f954c0097f0 con 0x7f95500b2d00 2020-11-28T22:26:45.487 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.480+0000 7f95549bc700 1 -- 172.21.15.162:0/3683225387 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954c009950 con 0x7f95500b2d00 2020-11-28T22:26:45.487 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.480+0000 7f95549bc700 1 -- 172.21.15.162:0/3683225387 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f954c009cb0 con 0x7f95500b2d00 2020-11-28T22:26:45.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.480+0000 7f95569c0700 1 -- 172.21.15.162:0/3683225387 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95500b2d00 msgr2=0x7f955005c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:45.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.480+0000 7f95569c0700 1 --2- 172.21.15.162:0/3683225387 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95500b2d00 0x7f955005c4d0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 rx=0x7f954c002f30 tx=0x7f954c006e80).stop 2020-11-28T22:26:45.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.480+0000 7f95569c0700 1 -- 172.21.15.162:0/3683225387 shutdown_connections 2020-11-28T22:26:45.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.480+0000 7f95569c0700 1 --2- 172.21.15.162:0/3683225387 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95500b2d00 0x7f955005c4d0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:45.488 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.480+0000 7f95569c0700 1 -- 172.21.15.162:0/3683225387 >> 172.21.15.162:0/3683225387 conn(0x7f95500604c0 msgr2=0x7f95500608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:26:45.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.480+0000 7f95569c0700 1 -- 172.21.15.162:0/3683225387 shutdown_connections 2020-11-28T22:26:45.489 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.480+0000 7f95569c0700 1 -- 172.21.15.162:0/3683225387 wait complete. 2020-11-28T22:26:45.491 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.484+0000 7f95569c0700 1 Processor -- start 2020-11-28T22:26:45.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.484+0000 7f95569c0700 1 -- start start 2020-11-28T22:26:45.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.484+0000 7f95569c0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95500b2d00 0x7f9550139b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:45.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.484+0000 7f95569c0700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f95500b30e0 0x7f95500b91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:45.492 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.484+0000 7f95569c0700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f95500b3ab0 0x7f95500b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:45.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.484+0000 7f95569c0700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95500b6f40 con 0x7f95500b30e0 2020-11-28T22:26:45.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.484+0000 7f95569c0700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95500b7130 con 0x7f95500b2d00 2020-11-28T22:26:45.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.484+0000 7f95569c0700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95501486d0 con 0x7f95500b3ab0 2020-11-28T22:26:45.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.484+0000 7f95559be700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95500b2d00 0x7f9550139b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:45.493 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.484+0000 7f95559be700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95500b2d00 0x7f9550139b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46850/0 (socket says 172.21.15.162:46850) 2020-11-28T22:26:45.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.484+0000 7f95559be700 1 -- 172.21.15.162:0/1513992577 learned_addr learned my addr 172.21.15.162:0/1513992577 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T22:26:45.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.484+0000 7f95551bd700 1 --2- 172.21.15.162:0/1513992577 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f95500b30e0 0x7f95500b91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:45.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.484+0000 7f95561bf700 1 --2- 172.21.15.162:0/1513992577 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f95500b3ab0 0x7f95500b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:45.494 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.484+0000 7f95559be700 1 -- 172.21.15.162:0/1513992577 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f95500b3ab0 msgr2=0x7f95500b96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:45.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.484+0000 7f95559be700 1 --2- 172.21.15.162:0/1513992577 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f95500b3ab0 0x7f95500b96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:45.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.484+0000 7f95559be700 1 -- 172.21.15.162:0/1513992577 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f95500b30e0 msgr2=0x7f95500b91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:45.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.484+0000 7f95559be700 1 --2- 172.21.15.162:0/1513992577 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f95500b30e0 0x7f95500b91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:45.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.484+0000 7f95559be700 1 -- 172.21.15.162:0/1513992577 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9550148970 con 0x7f95500b2d00 2020-11-28T22:26:45.495 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.484+0000 7f95559be700 1 --2- 172.21.15.162:0/1513992577 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95500b2d00 0x7f9550139b30 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 rx=0x7f954c002f30 tx=0x7f954c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:45.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.484+0000 7f9542ffd700 1 -- 172.21.15.162:0/1513992577 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f954c011e10 con 0x7f95500b2d00 2020-11-28T22:26:45.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.484+0000 7f9542ffd700 1 -- 172.21.15.162:0/1513992577 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954c009c50 con 0x7f95500b2d00 2020-11-28T22:26:45.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.484+0000 7f95569c0700 1 -- 172.21.15.162:0/1513992577 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9550148c10 con 0x7f95500b2d00 2020-11-28T22:26:45.496 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.488+0000 7f9542ffd700 1 -- 172.21.15.162:0/1513992577 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f954c01e650 con 0x7f95500b2d00 2020-11-28T22:26:45.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.488+0000 7f95569c0700 1 -- 172.21.15.162:0/1513992577 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9550143c80 con 0x7f95500b2d00 2020-11-28T22:26:45.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.488+0000 7f95569c0700 1 -- 172.21.15.162:0/1513992577 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9550057050 con 0x7f95500b2d00 2020-11-28T22:26:45.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.488+0000 7f9542ffd700 1 -- 172.21.15.162:0/1513992577 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f954c01a040 con 0x7f95500b2d00 2020-11-28T22:26:45.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.488+0000 7f9542ffd700 1 --2- 172.21.15.162:0/1513992577 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f95300553b0 0x7f9530057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:45.497 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.488+0000 7f95551bd700 1 --2- 172.21.15.162:0/1513992577 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f95300553b0 0x7f9530057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:45.498 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.488+0000 7f9542ffd700 1 -- 172.21.15.162:0/1513992577 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f954c06ed10 con 0x7f95500b2d00 2020-11-28T22:26:45.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.488+0000 7f9542ffd700 1 --2- 172.21.15.162:0/1513992577 >> [v2:172.21.15.162:6818/12312,v1:172.21.15.162:6819/12312] conn(0x7f953005bac0 0x7f953005dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:45.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.488+0000 7f9542ffd700 1 -- 172.21.15.162:0/1513992577 --> [v2:172.21.15.162:6818/12312,v1:172.21.15.162:6819/12312] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f953005e560 con 0x7f953005bac0 2020-11-28T22:26:45.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.488+0000 7f9542ffd700 1 -- 172.21.15.162:0/1513992577 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=163) v2 ==== 24+0+0 (secure 0 0 0) 0x7f954c071320 con 0x7f95500b2d00 2020-11-28T22:26:45.499 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.488+0000 7f95561bf700 1 --2- 172.21.15.162:0/1513992577 >> [v2:172.21.15.162:6818/12312,v1:172.21.15.162:6819/12312] conn(0x7f953005bac0 0x7f953005dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:45.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.488+0000 7f95551bd700 1 --2- 172.21.15.162:0/1513992577 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f95300553b0 0x7f9530057870 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7f954400d000 tx=0x7f9544007b40).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:45.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.492+0000 7f95561bf700 1 --2- 172.21.15.162:0/1513992577 >> [v2:172.21.15.162:6818/12312,v1:172.21.15.162:6819/12312] conn(0x7f953005bac0 0x7f953005dee0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:45.500 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.492+0000 7f9542ffd700 1 -- 172.21.15.162:0/1513992577 <== osd.2 v2:172.21.15.162:6818/12312 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7f953005e560 con 0x7f953005bac0 2020-11-28T22:26:45.516 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.512+0000 7f95569c0700 1 -- 172.21.15.162:0/1513992577 --> [v2:172.21.15.162:6818/12312,v1:172.21.15.162:6819/12312] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f955005d250 con 0x7f953005bac0 2020-11-28T22:26:45.517 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.512+0000 7f9542ffd700 1 -- 172.21.15.162:0/1513992577 <== osd.2 v2:172.21.15.162:6818/12312 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f955005d250 con 0x7f953005bac0 2020-11-28T22:26:45.517 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2020-11-28T22:26:45.517 INFO:teuthology.orchestra.run.smithi162.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2020-11-28T22:26:45.517 INFO:teuthology.orchestra.run.smithi162.stdout:} 2020-11-28T22:26:45.518 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.516+0000 7f95569c0700 1 -- 172.21.15.162:0/1513992577 >> [v2:172.21.15.162:6818/12312,v1:172.21.15.162:6819/12312] conn(0x7f953005bac0 msgr2=0x7f953005dee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:45.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.516+0000 7f95569c0700 1 --2- 172.21.15.162:0/1513992577 >> [v2:172.21.15.162:6818/12312,v1:172.21.15.162:6819/12312] conn(0x7f953005bac0 0x7f953005dee0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:45.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.516+0000 7f95569c0700 1 -- 172.21.15.162:0/1513992577 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f95300553b0 msgr2=0x7f9530057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:45.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.516+0000 7f95569c0700 1 --2- 172.21.15.162:0/1513992577 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f95300553b0 0x7f9530057870 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7f954400d000 tx=0x7f9544007b40).stop 2020-11-28T22:26:45.519 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.516+0000 7f95569c0700 1 -- 172.21.15.162:0/1513992577 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95500b2d00 msgr2=0x7f9550139b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:45.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.516+0000 7f95569c0700 1 --2- 172.21.15.162:0/1513992577 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95500b2d00 0x7f9550139b30 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 rx=0x7f954c002f30 tx=0x7f954c015000).stop 2020-11-28T22:26:45.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.516+0000 7f95561bf700 1 -- 172.21.15.162:0/1513992577 reap_dead start 2020-11-28T22:26:45.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.516+0000 7f95569c0700 1 -- 172.21.15.162:0/1513992577 shutdown_connections 2020-11-28T22:26:45.520 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.516+0000 7f95569c0700 1 -- 172.21.15.162:0/1513992577 >> 172.21.15.162:0/1513992577 conn(0x7f95500604c0 msgr2=0x7f955013bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:26:45.521 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.516+0000 7f95569c0700 1 -- 172.21.15.162:0/1513992577 shutdown_connections 2020-11-28T22:26:45.521 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.516+0000 7f95569c0700 1 -- 172.21.15.162:0/1513992577 wait complete. 2020-11-28T22:26:45.530 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd deep-scrub 2 2020-11-28T22:26:45.642 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.640+0000 7fc7badbf700 1 Processor -- start 2020-11-28T22:26:45.643 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.640+0000 7fc7badbf700 1 -- start start 2020-11-28T22:26:45.644 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.640+0000 7fc7badbf700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc7b40994f0 0x7fc7b40b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:45.644 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.640+0000 7fc7badbf700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7b4060710 0x7fc7b40b1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:45.645 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.640+0000 7fc7badbf700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7b4098f10 con 0x7fc7b4098b30 2020-11-28T22:26:45.645 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.640+0000 7fc7badbf700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7b40b1030 con 0x7fc7b40994f0 2020-11-28T22:26:45.645 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.640+0000 7fc7badbf700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b40b0680 con 0x7fc7b4060710 2020-11-28T22:26:45.645 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.640+0000 7fc7b95bc700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc7b40994f0 0x7fc7b40b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:45.645 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.640+0000 7fc7b95bc700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc7b40994f0 0x7fc7b40b0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46860/0 (socket says 172.21.15.162:46860) 2020-11-28T22:26:45.646 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.640+0000 7fc7b95bc700 1 -- 172.21.15.162:0/3936984997 learned_addr learned my addr 172.21.15.162:0/3936984997 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T22:26:45.646 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.640+0000 7fc7ba5be700 1 --2- 172.21.15.162:0/3936984997 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7b4060710 0x7fc7b40b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:45.646 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.640+0000 7fc7b95bc700 1 -- 172.21.15.162:0/3936984997 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7b4060710 msgr2=0x7fc7b40b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T22:26:45.646 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.640+0000 7fc7b95bc700 1 --2- 172.21.15.162:0/3936984997 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7b4060710 0x7fc7b40b1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:45.647 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.640+0000 7fc7b95bc700 1 -- 172.21.15.162:0/3936984997 >> v1:172.21.15.118:6789/0 conn(0x7fc7b4098b30 legacy=0x7fc7b4060000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T22:26:45.647 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.640+0000 7fc7b95bc700 1 -- 172.21.15.162:0/3936984997 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b40b0920 con 0x7fc7b40994f0 2020-11-28T22:26:45.647 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.640+0000 7fc7b95bc700 1 --2- 172.21.15.162:0/3936984997 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7b40994f0 0x7fc7b40b0000 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 rx=0x7fc7b00032b0 tx=0x7fc7b0009270).ready entity=mon.1 client_cookie=e0051e59600608a6 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:45.648 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.640+0000 7fc7b8dbb700 1 -- 172.21.15.162:0/3936984997 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7b000bc00 con 0x7fc7b40994f0 2020-11-28T22:26:45.648 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.640+0000 7fc7b8dbb700 1 -- 172.21.15.162:0/3936984997 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b000bd60 con 0x7fc7b40994f0 2020-11-28T22:26:45.649 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.640+0000 7fc7b8dbb700 1 -- 172.21.15.162:0/3936984997 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7b0011400 con 0x7fc7b40994f0 2020-11-28T22:26:45.649 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.640+0000 7fc7badbf700 1 -- 172.21.15.162:0/3936984997 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7b40994f0 msgr2=0x7fc7b40b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:45.649 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.640+0000 7fc7badbf700 1 --2- 172.21.15.162:0/3936984997 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7b40994f0 0x7fc7b40b0000 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 rx=0x7fc7b00032b0 tx=0x7fc7b0009270).stop 2020-11-28T22:26:45.649 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.644+0000 7fc7badbf700 1 -- 172.21.15.162:0/3936984997 shutdown_connections 2020-11-28T22:26:45.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.644+0000 7fc7badbf700 1 --2- 172.21.15.162:0/3936984997 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7b4060710 0x7fc7b40b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:45.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.644+0000 7fc7badbf700 1 --2- 172.21.15.162:0/3936984997 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7b40994f0 0x7fc7b40b0000 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:45.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.644+0000 7fc7badbf700 1 -- 172.21.15.162:0/3936984997 >> 172.21.15.162:0/3936984997 conn(0x7fc7b405b970 msgr2=0x7fc7b405bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:26:45.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.644+0000 7fc7badbf700 1 -- 172.21.15.162:0/3936984997 shutdown_connections 2020-11-28T22:26:45.650 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.644+0000 7fc7badbf700 1 -- 172.21.15.162:0/3936984997 wait complete. 2020-11-28T22:26:45.651 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.644+0000 7fc7badbf700 1 Processor -- start 2020-11-28T22:26:45.651 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.644+0000 7fc7badbf700 1 -- start start 2020-11-28T22:26:45.652 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.648+0000 7fc7badbf700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7b4060710 0x7fc7b40aaff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:45.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.648+0000 7fc7badbf700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7b4098b30 0x7fc7b40aee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:45.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.648+0000 7fc7badbf700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7b40994f0 0x7fc7b4065220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:45.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.648+0000 7fc7badbf700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b40aeb60 con 0x7fc7b4060710 2020-11-28T22:26:45.653 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.648+0000 7fc7badbf700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b4063340 con 0x7fc7b4098b30 2020-11-28T22:26:45.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.648+0000 7fc7badbf700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b40635e0 con 0x7fc7b40994f0 2020-11-28T22:26:45.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.648+0000 7fc7b95bc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7b4098b30 0x7fc7b40aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:45.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.648+0000 7fc7b95bc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7b4098b30 0x7fc7b40aee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46868/0 (socket says 172.21.15.162:46868) 2020-11-28T22:26:45.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.648+0000 7fc7b95bc700 1 -- 172.21.15.162:0/2389667492 learned_addr learned my addr 172.21.15.162:0/2389667492 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T22:26:45.654 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.648+0000 7fc7ba5be700 1 --2- 172.21.15.162:0/2389667492 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7b40994f0 0x7fc7b4065220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:45.655 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.648+0000 7fc7b9dbd700 1 --2- 172.21.15.162:0/2389667492 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7b4060710 0x7fc7b40aaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:45.655 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.648+0000 7fc7b95bc700 1 -- 172.21.15.162:0/2389667492 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7b40994f0 msgr2=0x7fc7b4065220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:45.655 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.648+0000 7fc7b95bc700 1 --2- 172.21.15.162:0/2389667492 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7b40994f0 0x7fc7b4065220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:45.655 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.648+0000 7fc7b95bc700 1 -- 172.21.15.162:0/2389667492 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7b4060710 msgr2=0x7fc7b40aaff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:45.655 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.648+0000 7fc7b95bc700 1 --2- 172.21.15.162:0/2389667492 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7b4060710 0x7fc7b40aaff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:45.656 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.648+0000 7fc7b95bc700 1 -- 172.21.15.162:0/2389667492 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b4063880 con 0x7fc7b4098b30 2020-11-28T22:26:45.656 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.648+0000 7fc7b95bc700 1 --2- 172.21.15.162:0/2389667492 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7b4098b30 0x7fc7b40aee70 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 rx=0x7fc7b00032b0 tx=0x7fc7b0016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:45.656 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.648+0000 7fc7a2ffd700 1 -- 172.21.15.162:0/2389667492 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7b001f5e0 con 0x7fc7b4098b30 2020-11-28T22:26:45.656 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.648+0000 7fc7a2ffd700 1 -- 172.21.15.162:0/2389667492 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b001f740 con 0x7fc7b4098b30 2020-11-28T22:26:45.657 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.648+0000 7fc7a2ffd700 1 -- 172.21.15.162:0/2389667492 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7b001faa0 con 0x7fc7b4098b30 2020-11-28T22:26:45.657 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.648+0000 7fc7badbf700 1 -- 172.21.15.162:0/2389667492 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7b4063b20 con 0x7fc7b4098b30 2020-11-28T22:26:45.657 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.648+0000 7fc7badbf700 1 -- 172.21.15.162:0/2389667492 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7b40ad700 con 0x7fc7b4098b30 2020-11-28T22:26:45.658 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.652+0000 7fc7a2ffd700 1 -- 172.21.15.162:0/2389667492 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc7b0026030 con 0x7fc7b4098b30 2020-11-28T22:26:45.658 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.652+0000 7fc7badbf700 1 -- 172.21.15.162:0/2389667492 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7b4057050 con 0x7fc7b4098b30 2020-11-28T22:26:45.658 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.652+0000 7fc7a2ffd700 1 --2- 172.21.15.162:0/2389667492 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc794055330 0x7fc7940577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:45.659 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.652+0000 7fc7a2ffd700 1 -- 172.21.15.162:0/2389667492 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5339+0+0 (secure 0 0 0) 0x7fc7b006ebd0 con 0x7fc7b4098b30 2020-11-28T22:26:45.659 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.652+0000 7fc7b9dbd700 1 --2- 172.21.15.162:0/2389667492 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc794055330 0x7fc7940577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:45.659 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.652+0000 7fc7b9dbd700 1 --2- 172.21.15.162:0/2389667492 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc794055330 0x7fc7940577f0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7fc7a400d000 tx=0x7fc7a4009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:45.660 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.656+0000 7fc7a2ffd700 1 -- 172.21.15.162:0/2389667492 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fc7b00451b0 con 0x7fc7b4098b30 2020-11-28T22:26:45.801 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.796+0000 7fc7badbf700 1 -- 172.21.15.162:0/2389667492 --> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fc7b405aa30 con 0x7fc794055330 2020-11-28T22:26:45.802 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.800+0000 7fc7a2ffd700 1 -- 172.21.15.162:0/2389667492 <== mgr.4105 v2:172.21.15.162:6800/12156 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc7b405aa30 con 0x7fc794055330 2020-11-28T22:26:45.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.800+0000 7fc7badbf700 1 -- 172.21.15.162:0/2389667492 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc794055330 msgr2=0x7fc7940577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:45.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.800+0000 7fc7badbf700 1 --2- 172.21.15.162:0/2389667492 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc794055330 0x7fc7940577f0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7fc7a400d000 tx=0x7fc7a4009040).stop 2020-11-28T22:26:45.804 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.800+0000 7fc7badbf700 1 -- 172.21.15.162:0/2389667492 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7b4098b30 msgr2=0x7fc7b40aee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:45.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.800+0000 7fc7badbf700 1 --2- 172.21.15.162:0/2389667492 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7b4098b30 0x7fc7b40aee70 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 rx=0x7fc7b00032b0 tx=0x7fc7b0016000).stop 2020-11-28T22:26:45.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.800+0000 7fc7badbf700 1 -- 172.21.15.162:0/2389667492 shutdown_connections 2020-11-28T22:26:45.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.800+0000 7fc7badbf700 1 --2- 172.21.15.162:0/2389667492 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc794055330 0x7fc7940577f0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:45.805 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.800+0000 7fc7badbf700 1 --2- 172.21.15.162:0/2389667492 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7b40994f0 0x7fc7b4065220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:45.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.800+0000 7fc7badbf700 1 --2- 172.21.15.162:0/2389667492 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7b4060710 0x7fc7b40aaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:45.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.800+0000 7fc7badbf700 1 --2- 172.21.15.162:0/2389667492 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7b4098b30 0x7fc7b40aee70 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:45.806 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.800+0000 7fc7badbf700 1 -- 172.21.15.162:0/2389667492 >> 172.21.15.162:0/2389667492 conn(0x7fc7b405b970 msgr2=0x7fc7b40b1450 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:26:45.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.800+0000 7fc7badbf700 1 -- 172.21.15.162:0/2389667492 shutdown_connections 2020-11-28T22:26:45.807 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.800+0000 7fc7badbf700 1 -- 172.21.15.162:0/2389667492 wait complete. 2020-11-28T22:26:45.807 INFO:teuthology.orchestra.run.smithi162.stderr:instructed osd(s) 2 to deep-scrub 2020-11-28T22:26:45.815 INFO:tasks.ceph:Scrubbing osd.3 2020-11-28T22:26:45.815 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.3 config set osd_debug_deep_scrub_sleep 0 2020-11-28T22:26:45.931 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.928+0000 7fc6fb59e700 1 Processor -- start 2020-11-28T22:26:45.932 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.928+0000 7fc6fb59e700 1 -- start start 2020-11-28T22:26:45.933 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.928+0000 7fc6fb59e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc6fc0b3950 0x7fc6fc0b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:45.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.928+0000 7fc6fb59e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fc6fc0b18c0 0x7fc6fc13dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:45.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.928+0000 7fc6fb59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc6fc0b44e0 0x7fc6fc13e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:45.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.928+0000 7fc6fb59e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6fc0aab50 con 0x7fc6fc0b18c0 2020-11-28T22:26:45.934 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.928+0000 7fc6fb59e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6fc0b1f50 con 0x7fc6fc0b3950 2020-11-28T22:26:45.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.928+0000 7fc6fb59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc143e90 con 0x7fc6fc0b44e0 2020-11-28T22:26:45.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.928+0000 7fc6fa59c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc6fc0b3950 0x7fc6fc0b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:45.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.928+0000 7fc6fa59c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc6fc0b3950 0x7fc6fc0b3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46874/0 (socket says 172.21.15.162:46874) 2020-11-28T22:26:45.935 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.928+0000 7fc6fa59c700 1 -- 172.21.15.162:0/2711106265 learned_addr learned my addr 172.21.15.162:0/2711106265 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T22:26:45.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.928+0000 7fc6f9d9b700 1 --2- 172.21.15.162:0/2711106265 >> v2:172.21.15.118:3300/0 conn(0x7fc6fc0b18c0 0x7fc6fc13dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:45.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.928+0000 7fc6fad9d700 1 --2- 172.21.15.162:0/2711106265 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc6fc0b44e0 0x7fc6fc13e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:45.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.932+0000 7fc6fa59c700 1 -- 172.21.15.162:0/2711106265 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc6fc0b44e0 msgr2=0x7fc6fc13e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T22:26:45.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.932+0000 7fc6fa59c700 1 --2- 172.21.15.162:0/2711106265 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc6fc0b44e0 0x7fc6fc13e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:45.936 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.932+0000 7fc6fa59c700 1 -- 172.21.15.162:0/2711106265 >> v2:172.21.15.118:3300/0 conn(0x7fc6fc0b18c0 msgr2=0x7fc6fc13dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T22:26:45.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.932+0000 7fc6fa59c700 1 --2- 172.21.15.162:0/2711106265 >> v2:172.21.15.118:3300/0 conn(0x7fc6fc0b18c0 0x7fc6fc13dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:45.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.932+0000 7fc6fa59c700 1 -- 172.21.15.162:0/2711106265 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6fc0aa8b0 con 0x7fc6fc0b3950 2020-11-28T22:26:45.937 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.932+0000 7fc6fa59c700 1 --2- 172.21.15.162:0/2711106265 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6fc0b3950 0x7fc6fc0b3d30 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 rx=0x7fc6e4002f30 tx=0x7fc6e4006e80).ready entity=mon.1 client_cookie=47ad65c27351c7 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:45.940 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.932+0000 7fc6f959a700 1 -- 172.21.15.162:0/2711106265 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6e4009640 con 0x7fc6fc0b3950 2020-11-28T22:26:45.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.932+0000 7fc6f959a700 1 -- 172.21.15.162:0/2711106265 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e40097a0 con 0x7fc6fc0b3950 2020-11-28T22:26:45.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.932+0000 7fc6f959a700 1 -- 172.21.15.162:0/2711106265 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6e4009b00 con 0x7fc6fc0b3950 2020-11-28T22:26:45.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.932+0000 7fc6fb59e700 1 -- 172.21.15.162:0/2711106265 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6fc0b3950 msgr2=0x7fc6fc0b3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:45.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.932+0000 7fc6fb59e700 1 --2- 172.21.15.162:0/2711106265 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6fc0b3950 0x7fc6fc0b3d30 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 rx=0x7fc6e4002f30 tx=0x7fc6e4006e80).stop 2020-11-28T22:26:45.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.932+0000 7fc6fb59e700 1 -- 172.21.15.162:0/2711106265 shutdown_connections 2020-11-28T22:26:45.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.932+0000 7fc6fb59e700 1 --2- 172.21.15.162:0/2711106265 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc6fc0b44e0 0x7fc6fc13e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:45.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.932+0000 7fc6fb59e700 1 --2- 172.21.15.162:0/2711106265 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6fc0b3950 0x7fc6fc0b3d30 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:45.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.932+0000 7fc6fb59e700 1 --2- 172.21.15.162:0/2711106265 >> v2:172.21.15.118:3300/0 conn(0x7fc6fc0b18c0 0x7fc6fc13dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:45.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.932+0000 7fc6fb59e700 1 -- 172.21.15.162:0/2711106265 >> 172.21.15.162:0/2711106265 conn(0x7fc6fc062630 msgr2=0x7fc6fc062a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:26:45.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.932+0000 7fc6fb59e700 1 -- 172.21.15.162:0/2711106265 shutdown_connections 2020-11-28T22:26:45.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.932+0000 7fc6fb59e700 1 -- 172.21.15.162:0/2711106265 wait complete. 2020-11-28T22:26:45.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.936+0000 7fc6fb59e700 1 Processor -- start 2020-11-28T22:26:45.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.936+0000 7fc6fb59e700 1 -- start start 2020-11-28T22:26:45.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.936+0000 7fc6fb59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc6fc0b18c0 0x7fc6fc0aca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:45.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.936+0000 7fc6fb59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6fc0b3950 0x7fc6fc0b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:45.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.936+0000 7fc6fb59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc6fc0b44e0 0x7fc6fc0af460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:45.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.936+0000 7fc6fb59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc0af950 con 0x7fc6fc0b18c0 2020-11-28T22:26:45.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.936+0000 7fc6fb59e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc0afbe0 con 0x7fc6fc0b3950 2020-11-28T22:26:45.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.936+0000 7fc6fb59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc0afe80 con 0x7fc6fc0b44e0 2020-11-28T22:26:45.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.936+0000 7fc6f9d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6fc0b3950 0x7fc6fc0b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:45.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.936+0000 7fc6f9d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6fc0b3950 0x7fc6fc0b2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46882/0 (socket says 172.21.15.162:46882) 2020-11-28T22:26:45.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.936+0000 7fc6f9d9b700 1 -- 172.21.15.162:0/3347153043 learned_addr learned my addr 172.21.15.162:0/3347153043 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T22:26:45.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.936+0000 7fc6fa59c700 1 --2- 172.21.15.162:0/3347153043 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc6fc0b18c0 0x7fc6fc0aca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:45.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.936+0000 7fc6fad9d700 1 --2- 172.21.15.162:0/3347153043 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc6fc0b44e0 0x7fc6fc0af460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:45.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.936+0000 7fc6f9d9b700 1 -- 172.21.15.162:0/3347153043 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc6fc0b44e0 msgr2=0x7fc6fc0af460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:45.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.936+0000 7fc6f9d9b700 1 --2- 172.21.15.162:0/3347153043 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc6fc0b44e0 0x7fc6fc0af460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:45.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.936+0000 7fc6f9d9b700 1 -- 172.21.15.162:0/3347153043 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc6fc0b18c0 msgr2=0x7fc6fc0aca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:45.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.936+0000 7fc6f9d9b700 1 --2- 172.21.15.162:0/3347153043 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc6fc0b18c0 0x7fc6fc0aca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:45.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.936+0000 7fc6f9d9b700 1 -- 172.21.15.162:0/3347153043 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6fc152bd0 con 0x7fc6fc0b3950 2020-11-28T22:26:45.947 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.936+0000 7fc6fad9d700 1 --2- 172.21.15.162:0/3347153043 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc6fc0b44e0 0x7fc6fc0af460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T22:26:45.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.936+0000 7fc6f9d9b700 1 --2- 172.21.15.162:0/3347153043 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6fc0b3950 0x7fc6fc0b2ec0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 rx=0x7fc6ec00c000 tx=0x7fc6ec006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:45.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.936+0000 7fc6f37fe700 1 -- 172.21.15.162:0/3347153043 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6ec00ca10 con 0x7fc6fc0b3950 2020-11-28T22:26:45.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.936+0000 7fc6f37fe700 1 -- 172.21.15.162:0/3347153043 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6ec00cb70 con 0x7fc6fc0b3950 2020-11-28T22:26:45.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.936+0000 7fc6fb59e700 1 -- 172.21.15.162:0/3347153043 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6fc152e70 con 0x7fc6fc0b3950 2020-11-28T22:26:45.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.936+0000 7fc6f37fe700 1 -- 172.21.15.162:0/3347153043 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6ec0123f0 con 0x7fc6fc0b3950 2020-11-28T22:26:45.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.936+0000 7fc6fb59e700 1 -- 172.21.15.162:0/3347153043 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6fc0b0d50 con 0x7fc6fc0b3950 2020-11-28T22:26:45.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.940+0000 7fc6fb59e700 1 -- 172.21.15.162:0/3347153043 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc6fc153000 con 0x7fc6fc0b3950 2020-11-28T22:26:45.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.940+0000 7fc6f37fe700 1 -- 172.21.15.162:0/3347153043 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc6ec012550 con 0x7fc6fc0b3950 2020-11-28T22:26:45.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.940+0000 7fc6f37fe700 1 --2- 172.21.15.162:0/3347153043 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc6e0055350 0x7fc6e0057810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:45.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.940+0000 7fc6f37fe700 1 -- 172.21.15.162:0/3347153043 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5339+0+0 (secure 0 0 0) 0x7fc6ec06e1d0 con 0x7fc6fc0b3950 2020-11-28T22:26:45.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.940+0000 7fc6f37fe700 1 --2- 172.21.15.162:0/3347153043 >> [v2:172.21.15.162:6826/12323,v1:172.21.15.162:6827/12323] conn(0x7fc6e005bb00 0x7fc6e005df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:45.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.940+0000 7fc6f37fe700 1 -- 172.21.15.162:0/3347153043 --> [v2:172.21.15.162:6826/12323,v1:172.21.15.162:6827/12323] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc6e005e5a0 con 0x7fc6e005bb00 2020-11-28T22:26:45.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.940+0000 7fc6fa59c700 1 --2- 172.21.15.162:0/3347153043 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc6e0055350 0x7fc6e0057810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:45.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.940+0000 7fc6f37fe700 1 -- 172.21.15.162:0/3347153043 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=163) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6ec070860 con 0x7fc6fc0b3950 2020-11-28T22:26:45.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.940+0000 7fc6fad9d700 1 --2- 172.21.15.162:0/3347153043 >> [v2:172.21.15.162:6826/12323,v1:172.21.15.162:6827/12323] conn(0x7fc6e005bb00 0x7fc6e005df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:45.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.940+0000 7fc6fa59c700 1 --2- 172.21.15.162:0/3347153043 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc6e0055350 0x7fc6e0057810 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7fc6e4019000 tx=0x7fc6e4015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:45.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.940+0000 7fc6fad9d700 1 --2- 172.21.15.162:0/3347153043 >> [v2:172.21.15.162:6826/12323,v1:172.21.15.162:6827/12323] conn(0x7fc6e005bb00 0x7fc6e005df20 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:45.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.944+0000 7fc6f37fe700 1 -- 172.21.15.162:0/3347153043 <== osd.3 v2:172.21.15.162:6826/12323 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7fc6e005e5a0 con 0x7fc6e005bb00 2020-11-28T22:26:45.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.964+0000 7fc6fb59e700 1 -- 172.21.15.162:0/3347153043 --> [v2:172.21.15.162:6826/12323,v1:172.21.15.162:6827/12323] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc6fc153890 con 0x7fc6e005bb00 2020-11-28T22:26:45.967 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.964+0000 7fc6f37fe700 1 -- 172.21.15.162:0/3347153043 <== osd.3 v2:172.21.15.162:6826/12323 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fc6fc153890 con 0x7fc6e005bb00 2020-11-28T22:26:45.967 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2020-11-28T22:26:45.967 INFO:teuthology.orchestra.run.smithi162.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2020-11-28T22:26:45.967 INFO:teuthology.orchestra.run.smithi162.stdout:} 2020-11-28T22:26:45.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.964+0000 7fc6fb59e700 1 -- 172.21.15.162:0/3347153043 >> [v2:172.21.15.162:6826/12323,v1:172.21.15.162:6827/12323] conn(0x7fc6e005bb00 msgr2=0x7fc6e005df20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:45.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.964+0000 7fc6fb59e700 1 --2- 172.21.15.162:0/3347153043 >> [v2:172.21.15.162:6826/12323,v1:172.21.15.162:6827/12323] conn(0x7fc6e005bb00 0x7fc6e005df20 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:45.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.964+0000 7fc6fb59e700 1 -- 172.21.15.162:0/3347153043 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc6e0055350 msgr2=0x7fc6e0057810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:45.969 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.964+0000 7fc6fb59e700 1 --2- 172.21.15.162:0/3347153043 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc6e0055350 0x7fc6e0057810 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7fc6e4019000 tx=0x7fc6e4015000).stop 2020-11-28T22:26:45.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.964+0000 7fc6fb59e700 1 -- 172.21.15.162:0/3347153043 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6fc0b3950 msgr2=0x7fc6fc0b2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:45.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.964+0000 7fc6fb59e700 1 --2- 172.21.15.162:0/3347153043 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6fc0b3950 0x7fc6fc0b2ec0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 rx=0x7fc6ec00c000 tx=0x7fc6ec006dc0).stop 2020-11-28T22:26:45.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.964+0000 7fc6fad9d700 1 -- 172.21.15.162:0/3347153043 reap_dead start 2020-11-28T22:26:45.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.964+0000 7fc6fb59e700 1 -- 172.21.15.162:0/3347153043 shutdown_connections 2020-11-28T22:26:45.970 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.964+0000 7fc6fb59e700 1 -- 172.21.15.162:0/3347153043 >> 172.21.15.162:0/3347153043 conn(0x7fc6fc062630 msgr2=0x7fc6fc142b40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:26:45.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.964+0000 7fc6fb59e700 1 -- 172.21.15.162:0/3347153043 shutdown_connections 2020-11-28T22:26:45.971 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:45.964+0000 7fc6fb59e700 1 -- 172.21.15.162:0/3347153043 wait complete. 2020-11-28T22:26:45.979 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd deep-scrub 3 2020-11-28T22:26:46.094 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.092+0000 7f6c012dc700 1 Processor -- start 2020-11-28T22:26:46.095 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.092+0000 7f6c012dc700 1 -- start start 2020-11-28T22:26:46.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.092+0000 7f6c012dc700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f6bfc060d10 0x7f6bfc0b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:46.096 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.092+0000 7f6c012dc700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6bfc0b4390 0x7f6bfc13fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:46.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.092+0000 7f6c012dc700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bfc0b4930 con 0x7f6bfc0b60e0 2020-11-28T22:26:46.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.092+0000 7f6c012dc700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bfc0b2170 con 0x7f6bfc060d10 2020-11-28T22:26:46.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.092+0000 7f6c012dc700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc0b6660 con 0x7f6bfc0b4390 2020-11-28T22:26:46.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.092+0000 7f6bfa7fc700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f6bfc0b60e0 0x7f6bfc146180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:53738/0 (socket says 172.21.15.162:53738) 2020-11-28T22:26:46.097 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.092+0000 7f6bfa7fc700 1 -- 172.21.15.162:0/3490039542 learned_addr learned my addr 172.21.15.162:0/3490039542 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T22:26:46.098 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.092+0000 7f6bfaffd700 1 --2- 172.21.15.162:0/3490039542 >> v2:172.21.15.118:3300/0 conn(0x7f6bfc060d10 0x7f6bfc0b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:46.098 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.092+0000 7f6bfb7fe700 1 --2- 172.21.15.162:0/3490039542 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6bfc0b4390 0x7f6bfc13fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:46.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.092+0000 7f6bf9ffb700 1 -- 172.21.15.162:0/3490039542 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2649613913 0 0) 0x7f6bfc0b4930 con 0x7f6bfc0b60e0 2020-11-28T22:26:46.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.092+0000 7f6bf9ffb700 1 -- 172.21.15.162:0/3490039542 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6be0003230 con 0x7f6bfc0b60e0 2020-11-28T22:26:46.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.096+0000 7f6bf9ffb700 1 -- 172.21.15.162:0/3490039542 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4290424575 0 0) 0x7f6be0003230 con 0x7f6bfc0b60e0 2020-11-28T22:26:46.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.096+0000 7f6bf9ffb700 1 -- 172.21.15.162:0/3490039542 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6bfc0b4390 msgr2=0x7f6bfc13fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T22:26:46.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.096+0000 7f6bf9ffb700 1 --2- 172.21.15.162:0/3490039542 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6bfc0b4390 0x7f6bfc13fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:46.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.096+0000 7f6bf9ffb700 1 -- 172.21.15.162:0/3490039542 >> v2:172.21.15.118:3300/0 conn(0x7f6bfc060d10 msgr2=0x7f6bfc0b52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T22:26:46.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.096+0000 7f6bf9ffb700 1 --2- 172.21.15.162:0/3490039542 >> v2:172.21.15.118:3300/0 conn(0x7f6bfc060d10 0x7f6bfc0b52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:46.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.096+0000 7f6bf9ffb700 1 -- 172.21.15.162:0/3490039542 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bfc0b4b60 con 0x7f6bfc0b60e0 2020-11-28T22:26:46.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.096+0000 7f6bf9ffb700 1 -- 172.21.15.162:0/3490039542 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7f6bf00016d0 con 0x7f6bfc0b60e0 2020-11-28T22:26:46.102 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.096+0000 7f6bf9ffb700 1 -- 172.21.15.162:0/3490039542 >> v1:172.21.15.162:6789/0 conn(0x7f6bfc0b60e0 legacy=0x7f6bfc146180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:46.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.096+0000 7f6bf9ffb700 1 --2- 172.21.15.162:0/3490039542 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6bfc0b4390 0x7f6be0003720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:46.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.096+0000 7f6bf9ffb700 1 --2- 172.21.15.162:0/3490039542 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6be0003c30 0x7f6be00044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:46.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.096+0000 7f6bf9ffb700 1 --2- 172.21.15.162:0/3490039542 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6be00049c0 0x7f6be0004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:46.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.096+0000 7f6bf9ffb700 1 -- 172.21.15.162:0/3490039542 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc0b2170 con 0x7f6be00049c0 2020-11-28T22:26:46.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.096+0000 7f6bf9ffb700 1 -- 172.21.15.162:0/3490039542 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc0b6660 con 0x7f6be0003c30 2020-11-28T22:26:46.104 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.096+0000 7f6bf9ffb700 1 -- 172.21.15.162:0/3490039542 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be00041f0 con 0x7f6bfc0b4390 2020-11-28T22:26:46.104 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.096+0000 7f6bfa7fc700 1 --2- 172.21.15.162:0/3490039542 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6be0003c30 0x7f6be00044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:46.104 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.096+0000 7f6bfb7fe700 1 --2- 172.21.15.162:0/3490039542 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6be00049c0 0x7f6be0004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:46.104 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.096+0000 7f6bfaffd700 1 --2- 172.21.15.162:0/3490039542 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6bfc0b4390 0x7f6be0003720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:46.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.096+0000 7f6bfa7fc700 1 -- 172.21.15.162:0/3490039542 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6bfc0b4390 msgr2=0x7f6be0003720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T22:26:46.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.096+0000 7f6bfa7fc700 1 --2- 172.21.15.162:0/3490039542 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6bfc0b4390 0x7f6be0003720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:46.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.096+0000 7f6bfa7fc700 1 -- 172.21.15.162:0/3490039542 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6be00049c0 msgr2=0x7f6be0004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T22:26:46.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.096+0000 7f6bfa7fc700 1 --2- 172.21.15.162:0/3490039542 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6be00049c0 0x7f6be0004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:46.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.096+0000 7f6bfa7fc700 1 -- 172.21.15.162:0/3490039542 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be0003230 con 0x7f6be0003c30 2020-11-28T22:26:46.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.096+0000 7f6bfa7fc700 1 --2- 172.21.15.162:0/3490039542 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6be0003c30 0x7f6be00044d0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 rx=0x7f6bf00013e0 tx=0x7f6bf00077c0).ready entity=mon.1 client_cookie=58614a2e55169e32 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:46.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.096+0000 7f6bf9ffb700 1 -- 172.21.15.162:0/3490039542 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6bf00070d0 con 0x7f6be0003c30 2020-11-28T22:26:46.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.096+0000 7f6bf9ffb700 1 -- 172.21.15.162:0/3490039542 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf0007230 con 0x7f6be0003c30 2020-11-28T22:26:46.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.096+0000 7f6bf9ffb700 1 -- 172.21.15.162:0/3490039542 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6bf00055e0 con 0x7f6be0003c30 2020-11-28T22:26:46.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.096+0000 7f6c012dc700 1 -- 172.21.15.162:0/3490039542 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6be0003c30 msgr2=0x7f6be00044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:46.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.096+0000 7f6c012dc700 1 --2- 172.21.15.162:0/3490039542 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6be0003c30 0x7f6be00044d0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 rx=0x7f6bf00013e0 tx=0x7f6bf00077c0).stop 2020-11-28T22:26:46.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.096+0000 7f6bfb7fe700 1 -- 172.21.15.162:0/3490039542 reap_dead start 2020-11-28T22:26:46.108 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.096+0000 7f6c012dc700 1 -- 172.21.15.162:0/3490039542 shutdown_connections 2020-11-28T22:26:46.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.096+0000 7f6c012dc700 1 -- 172.21.15.162:0/3490039542 >> 172.21.15.162:0/3490039542 conn(0x7f6bfc05fe70 msgr2=0x7f6bfc06aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:26:46.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.096+0000 7f6c012dc700 1 -- 172.21.15.162:0/3490039542 shutdown_connections 2020-11-28T22:26:46.110 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.096+0000 7f6c012dc700 1 -- 172.21.15.162:0/3490039542 wait complete. 2020-11-28T22:26:46.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.100+0000 7f6c012dc700 1 Processor -- start 2020-11-28T22:26:46.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.100+0000 7f6c012dc700 1 -- start start 2020-11-28T22:26:46.113 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.100+0000 7f6c012dc700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6bfc13d400 0x7f6bfc155660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:46.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.100+0000 7f6c012dc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6bfc151a70 0x7f6bfc155b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:46.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.100+0000 7f6c012dc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6bfc152100 0x7f6bfc152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:46.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.100+0000 7f6c012dc700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc0ad440 con 0x7f6bfc151a70 2020-11-28T22:26:46.114 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.100+0000 7f6c012dc700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc1551c0 con 0x7f6bfc152100 2020-11-28T22:26:46.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.100+0000 7f6c012dc700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc0b83f0 con 0x7f6bfc13d400 2020-11-28T22:26:46.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.100+0000 7f6bfb7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6bfc152100 0x7f6bfc152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:46.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.100+0000 7f6bfb7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6bfc152100 0x7f6bfc152590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46906/0 (socket says 172.21.15.162:46906) 2020-11-28T22:26:46.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.100+0000 7f6bfb7fe700 1 -- 172.21.15.162:0/3305089731 learned_addr learned my addr 172.21.15.162:0/3305089731 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T22:26:46.115 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.100+0000 7f6bfaffd700 1 --2- 172.21.15.162:0/3305089731 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6bfc13d400 0x7f6bfc155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:46.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.100+0000 7f6bfa7fc700 1 --2- 172.21.15.162:0/3305089731 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6bfc151a70 0x7f6bfc155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:46.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.100+0000 7f6bfb7fe700 1 -- 172.21.15.162:0/3305089731 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6bfc13d400 msgr2=0x7f6bfc155660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:46.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.100+0000 7f6bfb7fe700 1 --2- 172.21.15.162:0/3305089731 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6bfc13d400 0x7f6bfc155660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:46.116 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.100+0000 7f6bfb7fe700 1 -- 172.21.15.162:0/3305089731 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6bfc151a70 msgr2=0x7f6bfc155b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:46.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.100+0000 7f6bfb7fe700 1 --2- 172.21.15.162:0/3305089731 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6bfc151a70 0x7f6bfc155b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:46.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.100+0000 7f6bfb7fe700 1 -- 172.21.15.162:0/3305089731 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bfc149490 con 0x7f6bfc152100 2020-11-28T22:26:46.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.100+0000 7f6bfb7fe700 1 --2- 172.21.15.162:0/3305089731 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6bfc152100 0x7f6bfc152590 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 rx=0x7f6bf400b000 tx=0x7f6bf40091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:46.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.100+0000 7f6bebfff700 1 -- 172.21.15.162:0/3305089731 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6bf4019070 con 0x7f6bfc152100 2020-11-28T22:26:46.117 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.100+0000 7f6bebfff700 1 -- 172.21.15.162:0/3305089731 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf400fab0 con 0x7f6bfc152100 2020-11-28T22:26:46.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.100+0000 7f6c012dc700 1 -- 172.21.15.162:0/3305089731 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bfc0af630 con 0x7f6bfc152100 2020-11-28T22:26:46.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.100+0000 7f6bebfff700 1 -- 172.21.15.162:0/3305089731 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6bf400fe50 con 0x7f6bfc152100 2020-11-28T22:26:46.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.100+0000 7f6c012dc700 1 -- 172.21.15.162:0/3305089731 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bfc13d980 con 0x7f6bfc152100 2020-11-28T22:26:46.118 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.104+0000 7f6bebfff700 1 -- 172.21.15.162:0/3305089731 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f6bf4014440 con 0x7f6bfc152100 2020-11-28T22:26:46.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.104+0000 7f6c012dc700 1 -- 172.21.15.162:0/3305089731 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bfc0ad5b0 con 0x7f6bfc152100 2020-11-28T22:26:46.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.104+0000 7f6bebfff700 1 --2- 172.21.15.162:0/3305089731 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6bd80554e0 0x7f6bd80579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:46.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.104+0000 7f6bfaffd700 1 --2- 172.21.15.162:0/3305089731 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6bd80554e0 0x7f6bd80579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:46.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.104+0000 7f6bebfff700 1 -- 172.21.15.162:0/3305089731 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f6bf406e610 con 0x7f6bfc152100 2020-11-28T22:26:46.119 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.104+0000 7f6bfaffd700 1 --2- 172.21.15.162:0/3305089731 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6bd80554e0 0x7f6bd80579a0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7f6bec008000 tx=0x7f6bec008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:46.120 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.108+0000 7f6bebfff700 1 -- 172.21.15.162:0/3305089731 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f6bfc0ad5b0 con 0x7f6bfc152100 2020-11-28T22:26:46.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.244+0000 7f6c012dc700 1 -- 172.21.15.162:0/3305089731 --> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f6bfc0ad2c0 con 0x7f6bd80554e0 2020-11-28T22:26:46.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.244+0000 7f6bebfff700 1 -- 172.21.15.162:0/3305089731 <== mgr.4105 v2:172.21.15.162:6800/12156 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6bec0145c0 con 0x7f6bd80554e0 2020-11-28T22:26:46.250 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.248+0000 7f6c012dc700 1 -- 172.21.15.162:0/3305089731 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6bd80554e0 msgr2=0x7f6bd80579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:46.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.248+0000 7f6c012dc700 1 --2- 172.21.15.162:0/3305089731 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6bd80554e0 0x7f6bd80579a0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7f6bec008000 tx=0x7f6bec008040).stop 2020-11-28T22:26:46.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.248+0000 7f6c012dc700 1 -- 172.21.15.162:0/3305089731 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6bfc152100 msgr2=0x7f6bfc152590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:46.251 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.248+0000 7f6c012dc700 1 --2- 172.21.15.162:0/3305089731 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6bfc152100 0x7f6bfc152590 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 rx=0x7f6bf400b000 tx=0x7f6bf40091a0).stop 2020-11-28T22:26:46.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.248+0000 7f6c012dc700 1 -- 172.21.15.162:0/3305089731 shutdown_connections 2020-11-28T22:26:46.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.248+0000 7f6c012dc700 1 --2- 172.21.15.162:0/3305089731 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f6bd80554e0 0x7f6bd80579a0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:46.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.248+0000 7f6c012dc700 1 --2- 172.21.15.162:0/3305089731 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6bfc13d400 0x7f6bfc155660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:46.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.248+0000 7f6c012dc700 1 --2- 172.21.15.162:0/3305089731 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6bfc152100 0x7f6bfc152590 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:46.252 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.248+0000 7f6c012dc700 1 --2- 172.21.15.162:0/3305089731 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6bfc151a70 0x7f6bfc155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:46.253 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.248+0000 7f6c012dc700 1 -- 172.21.15.162:0/3305089731 >> 172.21.15.162:0/3305089731 conn(0x7f6bfc05fe70 msgr2=0x7f6bfc13e410 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:26:46.253 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.248+0000 7f6c012dc700 1 -- 172.21.15.162:0/3305089731 shutdown_connections 2020-11-28T22:26:46.253 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.248+0000 7f6c012dc700 1 -- 172.21.15.162:0/3305089731 wait complete. 2020-11-28T22:26:46.254 INFO:teuthology.orchestra.run.smithi162.stderr:instructed osd(s) 3 to deep-scrub 2020-11-28T22:26:46.266 INFO:tasks.ceph:Scrubbing osd.4 2020-11-28T22:26:46.267 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.4 config set osd_debug_deep_scrub_sleep 0 2020-11-28T22:26:46.383 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.380+0000 7f2ca63e9700 1 Processor -- start 2020-11-28T22:26:46.384 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.380+0000 7f2ca63e9700 1 -- start start 2020-11-28T22:26:46.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.380+0000 7f2ca63e9700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2ca00b58d0 0x7f2ca0140f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:46.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.380+0000 7f2ca63e9700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f2ca009ac40 0x7f2ca01448f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:46.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.380+0000 7f2ca63e9700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ca00b0d20 con 0x7f2ca00a84f0 2020-11-28T22:26:46.385 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.380+0000 7f2ca63e9700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ca00b5cb0 con 0x7f2ca009ac40 2020-11-28T22:26:46.386 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.380+0000 7f2ca63e9700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca00b0f50 con 0x7f2ca00b58d0 2020-11-28T22:26:46.386 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.380+0000 7f2ca4986700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f2ca009ac40 0x7f2ca01448f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:46.386 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.380+0000 7f2c9ffff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2ca00b58d0 0x7f2ca0140f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:46.386 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.380+0000 7f2c9f7fe700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f2ca00a84f0 0x7f2ca01441e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.162:43902/0 (socket says 172.21.15.162:43902) 2020-11-28T22:26:46.387 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.380+0000 7f2c9f7fe700 1 -- 172.21.15.162:0/3969487321 learned_addr learned my addr 172.21.15.162:0/3969487321 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T22:26:46.388 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.384+0000 7f2ca4986700 1 -- 172.21.15.162:0/3969487321 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2ca00b58d0 msgr2=0x7f2ca0140f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T22:26:46.388 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.384+0000 7f2ca4986700 1 --2- 172.21.15.162:0/3969487321 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2ca00b58d0 0x7f2ca0140f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:46.389 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.384+0000 7f2ca4986700 1 -- 172.21.15.162:0/3969487321 >> v1:172.21.15.118:6789/0 conn(0x7f2ca00a84f0 legacy=0x7f2ca01441e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:46.389 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.384+0000 7f2ca4986700 1 -- 172.21.15.162:0/3969487321 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca00b11f0 con 0x7f2ca009ac40 2020-11-28T22:26:46.389 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.384+0000 7f2ca4986700 1 --2- 172.21.15.162:0/3969487321 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2ca009ac40 0x7f2ca01448f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f2c980032b0 tx=0x7f2c980091a0).ready entity=mon.0 client_cookie=517064c98e1198e1 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:46.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.384+0000 7f2c9effd700 1 -- 172.21.15.162:0/3969487321 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c9800b9a0 con 0x7f2ca009ac40 2020-11-28T22:26:46.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.384+0000 7f2c9effd700 1 -- 172.21.15.162:0/3969487321 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c9800bb00 con 0x7f2ca009ac40 2020-11-28T22:26:46.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.384+0000 7f2c9effd700 1 -- 172.21.15.162:0/3969487321 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c9800bc70 con 0x7f2ca009ac40 2020-11-28T22:26:46.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.384+0000 7f2ca63e9700 1 -- 172.21.15.162:0/3969487321 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2ca009ac40 msgr2=0x7f2ca01448f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:46.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.384+0000 7f2ca63e9700 1 --2- 172.21.15.162:0/3969487321 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2ca009ac40 0x7f2ca01448f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f2c980032b0 tx=0x7f2c980091a0).stop 2020-11-28T22:26:46.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.384+0000 7f2ca63e9700 1 -- 172.21.15.162:0/3969487321 shutdown_connections 2020-11-28T22:26:46.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.384+0000 7f2ca63e9700 1 --2- 172.21.15.162:0/3969487321 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2ca009ac40 0x7f2ca01448f0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:46.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.384+0000 7f2ca63e9700 1 --2- 172.21.15.162:0/3969487321 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2ca00b58d0 0x7f2ca0140f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:46.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.384+0000 7f2ca63e9700 1 -- 172.21.15.162:0/3969487321 >> 172.21.15.162:0/3969487321 conn(0x7f2ca005c3f0 msgr2=0x7f2ca00b4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:26:46.392 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.384+0000 7f2ca63e9700 1 -- 172.21.15.162:0/3969487321 shutdown_connections 2020-11-28T22:26:46.392 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.384+0000 7f2ca63e9700 1 -- 172.21.15.162:0/3969487321 wait complete. 2020-11-28T22:26:46.392 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.384+0000 7f2ca63e9700 1 Processor -- start 2020-11-28T22:26:46.392 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.384+0000 7f2ca63e9700 1 -- start start 2020-11-28T22:26:46.395 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.388+0000 7f2ca63e9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ca009ac40 0x7f2ca01410a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:46.395 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.388+0000 7f2ca63e9700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2ca00a84f0 0x7f2ca00637c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:46.395 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.388+0000 7f2ca63e9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2ca00b58d0 0x7f2ca0063cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:46.395 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.388+0000 7f2ca63e9700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca00aaaf0 con 0x7f2ca00b58d0 2020-11-28T22:26:46.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.388+0000 7f2ca63e9700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca00ab070 con 0x7f2ca009ac40 2020-11-28T22:26:46.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.388+0000 7f2ca63e9700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca00b3f80 con 0x7f2ca00a84f0 2020-11-28T22:26:46.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.388+0000 7f2c9ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ca009ac40 0x7f2ca01410a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:46.396 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.388+0000 7f2c9ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ca009ac40 0x7f2ca01410a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46916/0 (socket says 172.21.15.162:46916) 2020-11-28T22:26:46.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.388+0000 7f2c9ffff700 1 -- 172.21.15.162:0/759137318 learned_addr learned my addr 172.21.15.162:0/759137318 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T22:26:46.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.388+0000 7f2ca4986700 1 --2- 172.21.15.162:0/759137318 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2ca00b58d0 0x7f2ca0063cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:46.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.388+0000 7f2c9f7fe700 1 --2- 172.21.15.162:0/759137318 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2ca00a84f0 0x7f2ca00637c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:46.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.388+0000 7f2c9ffff700 1 -- 172.21.15.162:0/759137318 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2ca00a84f0 msgr2=0x7f2ca00637c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:46.397 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.388+0000 7f2c9ffff700 1 --2- 172.21.15.162:0/759137318 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2ca00a84f0 0x7f2ca00637c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:46.398 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.388+0000 7f2c9ffff700 1 -- 172.21.15.162:0/759137318 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2ca00b58d0 msgr2=0x7f2ca0063cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:46.398 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.388+0000 7f2c9ffff700 1 --2- 172.21.15.162:0/759137318 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2ca00b58d0 0x7f2ca0063cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:46.398 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.388+0000 7f2c9ffff700 1 -- 172.21.15.162:0/759137318 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca00b3290 con 0x7f2ca009ac40 2020-11-28T22:26:46.398 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.388+0000 7f2c9ffff700 1 --2- 172.21.15.162:0/759137318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ca009ac40 0x7f2ca01410a0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 rx=0x7f2c9000c000 tx=0x7f2c90006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:46.399 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.388+0000 7f2c9d7fa700 1 -- 172.21.15.162:0/759137318 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c9000ca10 con 0x7f2ca009ac40 2020-11-28T22:26:46.399 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.388+0000 7f2c9d7fa700 1 -- 172.21.15.162:0/759137318 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c9000cb70 con 0x7f2ca009ac40 2020-11-28T22:26:46.399 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.388+0000 7f2ca63e9700 1 -- 172.21.15.162:0/759137318 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ca00aad20 con 0x7f2ca009ac40 2020-11-28T22:26:46.399 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.388+0000 7f2c9d7fa700 1 -- 172.21.15.162:0/759137318 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c9000be50 con 0x7f2ca009ac40 2020-11-28T22:26:46.400 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.388+0000 7f2ca63e9700 1 -- 172.21.15.162:0/759137318 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ca00674c0 con 0x7f2ca009ac40 2020-11-28T22:26:46.400 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.388+0000 7f2ca63e9700 1 -- 172.21.15.162:0/759137318 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2ca00b40f0 con 0x7f2ca009ac40 2020-11-28T22:26:46.400 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.392+0000 7f2c9d7fa700 1 -- 172.21.15.162:0/759137318 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f2c90025430 con 0x7f2ca009ac40 2020-11-28T22:26:46.400 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.392+0000 7f2c9d7fa700 1 --2- 172.21.15.162:0/759137318 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2c800554e0 0x7f2c800579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:46.401 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.392+0000 7f2c9d7fa700 1 -- 172.21.15.162:0/759137318 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f2c9006e880 con 0x7f2ca009ac40 2020-11-28T22:26:46.401 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.392+0000 7f2c9f7fe700 1 --2- 172.21.15.162:0/759137318 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2c800554e0 0x7f2c800579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:46.401 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.392+0000 7f2c9d7fa700 1 --2- 172.21.15.162:0/759137318 >> [v2:172.21.15.118:6800/12527,v1:172.21.15.118:6801/12527] conn(0x7f2c8005bbf0 0x7f2c8005e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:46.402 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.392+0000 7f2c9d7fa700 1 -- 172.21.15.162:0/759137318 --> [v2:172.21.15.118:6800/12527,v1:172.21.15.118:6801/12527] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2c8005e5f0 con 0x7f2c8005bbf0 2020-11-28T22:26:46.402 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.392+0000 7f2c9d7fa700 1 -- 172.21.15.162:0/759137318 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=163) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c9006eb50 con 0x7f2ca009ac40 2020-11-28T22:26:46.402 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.392+0000 7f2ca4986700 1 --2- 172.21.15.162:0/759137318 >> [v2:172.21.15.118:6800/12527,v1:172.21.15.118:6801/12527] conn(0x7f2c8005bbf0 0x7f2c8005e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:46.402 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.392+0000 7f2c9f7fe700 1 --2- 172.21.15.162:0/759137318 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2c800554e0 0x7f2c800579a0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7f2c9400d000 tx=0x7f2c94009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:46.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.392+0000 7f2ca4986700 1 --2- 172.21.15.162:0/759137318 >> [v2:172.21.15.118:6800/12527,v1:172.21.15.118:6801/12527] conn(0x7f2c8005bbf0 0x7f2c8005e010 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:46.403 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.396+0000 7f2c9d7fa700 1 -- 172.21.15.162:0/759137318 <== osd.4 v2:172.21.15.118:6800/12527 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7f2c8005e5f0 con 0x7f2c8005bbf0 2020-11-28T22:26:46.419 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.416+0000 7f2ca63e9700 1 -- 172.21.15.162:0/759137318 --> [v2:172.21.15.118:6800/12527,v1:172.21.15.118:6801/12527] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2ca0057050 con 0x7f2c8005bbf0 2020-11-28T22:26:46.419 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.416+0000 7f2c9d7fa700 1 -- 172.21.15.162:0/759137318 <== osd.4 v2:172.21.15.118:6800/12527 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f2ca0057050 con 0x7f2c8005bbf0 2020-11-28T22:26:46.420 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2020-11-28T22:26:46.420 INFO:teuthology.orchestra.run.smithi162.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2020-11-28T22:26:46.420 INFO:teuthology.orchestra.run.smithi162.stdout:} 2020-11-28T22:26:46.421 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.416+0000 7f2ca63e9700 1 -- 172.21.15.162:0/759137318 >> [v2:172.21.15.118:6800/12527,v1:172.21.15.118:6801/12527] conn(0x7f2c8005bbf0 msgr2=0x7f2c8005e010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:46.422 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.416+0000 7f2ca63e9700 1 --2- 172.21.15.162:0/759137318 >> [v2:172.21.15.118:6800/12527,v1:172.21.15.118:6801/12527] conn(0x7f2c8005bbf0 0x7f2c8005e010 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:46.422 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.416+0000 7f2ca63e9700 1 -- 172.21.15.162:0/759137318 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2c800554e0 msgr2=0x7f2c800579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:46.422 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.416+0000 7f2ca63e9700 1 --2- 172.21.15.162:0/759137318 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2c800554e0 0x7f2c800579a0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7f2c9400d000 tx=0x7f2c94009040).stop 2020-11-28T22:26:46.422 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.416+0000 7f2ca63e9700 1 -- 172.21.15.162:0/759137318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ca009ac40 msgr2=0x7f2ca01410a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:46.422 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.416+0000 7f2ca63e9700 1 --2- 172.21.15.162:0/759137318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ca009ac40 0x7f2ca01410a0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 rx=0x7f2c9000c000 tx=0x7f2c90006dc0).stop 2020-11-28T22:26:46.423 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.416+0000 7f2ca4986700 1 -- 172.21.15.162:0/759137318 reap_dead start 2020-11-28T22:26:46.423 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.416+0000 7f2ca63e9700 1 -- 172.21.15.162:0/759137318 shutdown_connections 2020-11-28T22:26:46.423 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.416+0000 7f2ca63e9700 1 -- 172.21.15.162:0/759137318 >> 172.21.15.162:0/759137318 conn(0x7f2ca005c3f0 msgr2=0x7f2ca0141e00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:26:46.423 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.416+0000 7f2ca63e9700 1 -- 172.21.15.162:0/759137318 shutdown_connections 2020-11-28T22:26:46.424 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.416+0000 7f2ca63e9700 1 -- 172.21.15.162:0/759137318 wait complete. 2020-11-28T22:26:46.438 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd deep-scrub 4 2020-11-28T22:26:46.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.548+0000 7f4feffff700 1 Processor -- start 2020-11-28T22:26:46.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.548+0000 7f4feffff700 1 -- start start 2020-11-28T22:26:46.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.552+0000 7f4feffff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f4ff00b4bd0 0x7f4ff0098660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:46.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.552+0000 7f4feffff700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4ff009b470 0x7f4ff0142ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:46.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.552+0000 7f4feffff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ff00b4fb0 con 0x7f4ff00b33f0 2020-11-28T22:26:46.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.552+0000 7f4feffff700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ff0098b50 con 0x7f4ff00b4bd0 2020-11-28T22:26:46.555 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.552+0000 7f4feffff700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ff0142a60 con 0x7f4ff009b470 2020-11-28T22:26:46.555 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.552+0000 7f4fef7fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4ff009b470 0x7f4ff0142ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:46.555 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.552+0000 7f4fef7fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4ff009b470 0x7f4ff0142ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46930/0 (socket says 172.21.15.162:46930) 2020-11-28T22:26:46.555 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.552+0000 7f4fef7fe700 1 -- 172.21.15.162:0/2244633659 learned_addr learned my addr 172.21.15.162:0/2244633659 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T22:26:46.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.552+0000 7f4feeffd700 1 --2- 172.21.15.162:0/2244633659 >> v2:172.21.15.118:3300/0 conn(0x7f4ff00b4bd0 0x7f4ff0098660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:46.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.552+0000 7f4fef7fe700 1 -- 172.21.15.162:0/2244633659 >> v2:172.21.15.118:3300/0 conn(0x7f4ff00b4bd0 msgr2=0x7f4ff0098660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T22:26:46.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.552+0000 7f4fef7fe700 1 --2- 172.21.15.162:0/2244633659 >> v2:172.21.15.118:3300/0 conn(0x7f4ff00b4bd0 0x7f4ff0098660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:46.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.552+0000 7f4fedffb700 1 -- 172.21.15.162:0/2244633659 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 522466367 0 0) 0x7f4ff00b4fb0 con 0x7f4ff00b33f0 2020-11-28T22:26:46.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.552+0000 7f4fef7fe700 1 -- 172.21.15.162:0/2244633659 >> v1:172.21.15.118:6789/0 conn(0x7f4ff00b33f0 legacy=0x7f4ff009ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:46.557 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.552+0000 7f4fef7fe700 1 -- 172.21.15.162:0/2244633659 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff00b3db0 con 0x7f4ff009b470 2020-11-28T22:26:46.559 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.552+0000 7f4fef7fe700 1 --2- 172.21.15.162:0/2244633659 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ff009b470 0x7f4ff0142ec0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 rx=0x7f4fe0002f30 tx=0x7f4fe0006e80).ready entity=mon.1 client_cookie=d0711faca3ebd3a9 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:46.559 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.552+0000 7f4fedffb700 1 -- 172.21.15.162:0/2244633659 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fe00097f0 con 0x7f4ff009b470 2020-11-28T22:26:46.559 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.552+0000 7f4fedffb700 1 -- 172.21.15.162:0/2244633659 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe0009950 con 0x7f4ff009b470 2020-11-28T22:26:46.560 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.552+0000 7f4fedffb700 1 -- 172.21.15.162:0/2244633659 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fe0009cb0 con 0x7f4ff009b470 2020-11-28T22:26:46.560 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.552+0000 7f4feffff700 1 -- 172.21.15.162:0/2244633659 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ff009b470 msgr2=0x7f4ff0142ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:46.560 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.552+0000 7f4feffff700 1 --2- 172.21.15.162:0/2244633659 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ff009b470 0x7f4ff0142ec0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 rx=0x7f4fe0002f30 tx=0x7f4fe0006e80).stop 2020-11-28T22:26:46.560 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.552+0000 7f4feffff700 1 -- 172.21.15.162:0/2244633659 shutdown_connections 2020-11-28T22:26:46.561 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.552+0000 7f4feffff700 1 --2- 172.21.15.162:0/2244633659 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ff009b470 0x7f4ff0142ec0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:46.561 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.552+0000 7f4feffff700 1 --2- 172.21.15.162:0/2244633659 >> v2:172.21.15.118:3300/0 conn(0x7f4ff00b4bd0 0x7f4ff0098660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:46.561 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.552+0000 7f4feffff700 1 -- 172.21.15.162:0/2244633659 >> 172.21.15.162:0/2244633659 conn(0x7f4ff00b54f0 msgr2=0x7f4ff00b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:26:46.561 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.552+0000 7f4feffff700 1 -- 172.21.15.162:0/2244633659 shutdown_connections 2020-11-28T22:26:46.561 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.552+0000 7f4feffff700 1 -- 172.21.15.162:0/2244633659 wait complete. 2020-11-28T22:26:46.562 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.552+0000 7f4feffff700 1 Processor -- start 2020-11-28T22:26:46.562 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.552+0000 7f4feffff700 1 -- start start 2020-11-28T22:26:46.562 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.556+0000 7f4feffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ff00b33f0 0x7f4ff0066ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:46.562 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.556+0000 7f4feffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ff00b4bd0 0x7f4ff0142de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:46.563 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.556+0000 7f4feffff700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4ff0144510 0x7f4ff01432d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:46.563 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.556+0000 7f4feffff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff0097c70 con 0x7f4ff00b4bd0 2020-11-28T22:26:46.563 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.556+0000 7f4feffff700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff00aa660 con 0x7f4ff00b33f0 2020-11-28T22:26:46.563 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.556+0000 7f4feffff700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff00aa900 con 0x7f4ff0144510 2020-11-28T22:26:46.563 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.556+0000 7f4feeffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ff00b33f0 0x7f4ff0066ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:46.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.556+0000 7f4feeffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ff00b33f0 0x7f4ff0066ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46932/0 (socket says 172.21.15.162:46932) 2020-11-28T22:26:46.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.556+0000 7f4feeffd700 1 -- 172.21.15.162:0/1304168335 learned_addr learned my addr 172.21.15.162:0/1304168335 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T22:26:46.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.556+0000 7f4fef7fe700 1 --2- 172.21.15.162:0/1304168335 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4ff0144510 0x7f4ff01432d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:46.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.556+0000 7f4fee7fc700 1 --2- 172.21.15.162:0/1304168335 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ff00b4bd0 0x7f4ff0142de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:46.564 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.556+0000 7f4feeffd700 1 -- 172.21.15.162:0/1304168335 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4ff0144510 msgr2=0x7f4ff01432d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:46.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.556+0000 7f4feeffd700 1 --2- 172.21.15.162:0/1304168335 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4ff0144510 0x7f4ff01432d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:46.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.556+0000 7f4feeffd700 1 -- 172.21.15.162:0/1304168335 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ff00b4bd0 msgr2=0x7f4ff0142de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:46.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.556+0000 7f4feeffd700 1 --2- 172.21.15.162:0/1304168335 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ff00b4bd0 0x7f4ff0142de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:46.565 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.556+0000 7f4feeffd700 1 -- 172.21.15.162:0/1304168335 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff0145850 con 0x7f4ff00b33f0 2020-11-28T22:26:46.566 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.556+0000 7f4fef7fe700 1 --2- 172.21.15.162:0/1304168335 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4ff0144510 0x7f4ff01432d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T22:26:46.566 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.556+0000 7f4feeffd700 1 --2- 172.21.15.162:0/1304168335 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ff00b33f0 0x7f4ff0066ea0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 rx=0x7f4fe800c000 tx=0x7f4fe80091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:46.566 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.556+0000 7f4fe7fff700 1 -- 172.21.15.162:0/1304168335 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fe800c9a0 con 0x7f4ff00b33f0 2020-11-28T22:26:46.566 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.556+0000 7f4fe7fff700 1 -- 172.21.15.162:0/1304168335 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe800cb00 con 0x7f4ff00b33f0 2020-11-28T22:26:46.566 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.556+0000 7f4feffff700 1 -- 172.21.15.162:0/1304168335 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ff0144110 con 0x7f4ff00b33f0 2020-11-28T22:26:46.567 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.556+0000 7f4fe7fff700 1 -- 172.21.15.162:0/1304168335 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fe800ce60 con 0x7f4ff00b33f0 2020-11-28T22:26:46.567 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.556+0000 7f4feffff700 1 -- 172.21.15.162:0/1304168335 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ff00aaf40 con 0x7f4ff00b33f0 2020-11-28T22:26:46.568 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.560+0000 7f4feffff700 1 -- 172.21.15.162:0/1304168335 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ff0057050 con 0x7f4ff00b33f0 2020-11-28T22:26:46.568 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.560+0000 7f4fe7fff700 1 -- 172.21.15.162:0/1304168335 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f4fe8022070 con 0x7f4ff00b33f0 2020-11-28T22:26:46.568 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.560+0000 7f4fe7fff700 1 --2- 172.21.15.162:0/1304168335 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4fd40553b0 0x7f4fd4057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:46.568 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.560+0000 7f4fe7fff700 1 -- 172.21.15.162:0/1304168335 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f4fe806e290 con 0x7f4ff00b33f0 2020-11-28T22:26:46.569 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.560+0000 7f4fee7fc700 1 --2- 172.21.15.162:0/1304168335 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4fd40553b0 0x7f4fd4057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:46.569 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.560+0000 7f4fee7fc700 1 --2- 172.21.15.162:0/1304168335 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4fd40553b0 0x7f4fd4057870 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7f4fd800d000 tx=0x7f4fd8007980).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:46.569 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.568+0000 7f4fe7fff700 1 -- 172.21.15.162:0/1304168335 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f4fe8044870 con 0x7f4ff00b33f0 2020-11-28T22:26:46.710 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.708+0000 7f4feffff700 1 -- 172.21.15.162:0/1304168335 --> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ff00b4640 con 0x7f4fd40553b0 2020-11-28T22:26:46.711 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.708+0000 7f4fe7fff700 1 -- 172.21.15.162:0/1304168335 <== mgr.4105 v2:172.21.15.162:6800/12156 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4ff00b4640 con 0x7f4fd40553b0 2020-11-28T22:26:46.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.708+0000 7f4feffff700 1 -- 172.21.15.162:0/1304168335 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4fd40553b0 msgr2=0x7f4fd4057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:46.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.708+0000 7f4feffff700 1 --2- 172.21.15.162:0/1304168335 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4fd40553b0 0x7f4fd4057870 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7f4fd800d000 tx=0x7f4fd8007980).stop 2020-11-28T22:26:46.712 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.708+0000 7f4feffff700 1 -- 172.21.15.162:0/1304168335 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ff00b33f0 msgr2=0x7f4ff0066ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:46.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.708+0000 7f4feffff700 1 --2- 172.21.15.162:0/1304168335 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ff00b33f0 0x7f4ff0066ea0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 rx=0x7f4fe800c000 tx=0x7f4fe80091a0).stop 2020-11-28T22:26:46.713 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.708+0000 7f4feffff700 1 -- 172.21.15.162:0/1304168335 shutdown_connections 2020-11-28T22:26:46.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.708+0000 7f4feffff700 1 --2- 172.21.15.162:0/1304168335 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f4fd40553b0 0x7f4fd4057870 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:46.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.708+0000 7f4feffff700 1 --2- 172.21.15.162:0/1304168335 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4ff0144510 0x7f4ff01432d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:46.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.708+0000 7f4feffff700 1 --2- 172.21.15.162:0/1304168335 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ff00b33f0 0x7f4ff0066ea0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:46.714 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.708+0000 7f4feffff700 1 --2- 172.21.15.162:0/1304168335 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ff00b4bd0 0x7f4ff0142de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:46.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.708+0000 7f4feffff700 1 -- 172.21.15.162:0/1304168335 >> 172.21.15.162:0/1304168335 conn(0x7f4ff00b54f0 msgr2=0x7f4ff0099b40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:26:46.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.708+0000 7f4feffff700 1 -- 172.21.15.162:0/1304168335 shutdown_connections 2020-11-28T22:26:46.715 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.708+0000 7f4feffff700 1 -- 172.21.15.162:0/1304168335 wait complete. 2020-11-28T22:26:46.716 INFO:teuthology.orchestra.run.smithi162.stderr:instructed osd(s) 4 to deep-scrub 2020-11-28T22:26:46.724 INFO:tasks.ceph:Scrubbing osd.5 2020-11-28T22:26:46.725 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.5 config set osd_debug_deep_scrub_sleep 0 2020-11-28T22:26:46.844 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.840+0000 7fc212d19700 1 Processor -- start 2020-11-28T22:26:46.844 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.840+0000 7fc212d19700 1 -- start start 2020-11-28T22:26:46.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.844+0000 7fc212d19700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc20c0b18c0 0x7fc20c0b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:46.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.844+0000 7fc212d19700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc20c0b44e0 0x7fc20c0b2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:46.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.844+0000 7fc212d19700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc20c0b1f50 con 0x7fc20c0b3950 2020-11-28T22:26:46.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.844+0000 7fc212d19700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc20c0aab50 con 0x7fc20c0b44e0 2020-11-28T22:26:46.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.844+0000 7fc212d19700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc20c143e90 con 0x7fc20c0b18c0 2020-11-28T22:26:46.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.844+0000 7fc212518700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc20c0b44e0 0x7fc20c0b2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:46.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.844+0000 7fc212518700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc20c0b44e0 0x7fc20c0b2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46944/0 (socket says 172.21.15.162:46944) 2020-11-28T22:26:46.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.844+0000 7fc212518700 1 -- 172.21.15.162:0/3848861428 learned_addr learned my addr 172.21.15.162:0/3848861428 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T22:26:46.848 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.844+0000 7fc211516700 1 --2- 172.21.15.162:0/3848861428 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc20c0b18c0 0x7fc20c0b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:46.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.844+0000 7fc210d15700 1 -- 172.21.15.162:0/3848861428 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1279365724 0 0) 0x7fc20c0b1f50 con 0x7fc20c0b3950 2020-11-28T22:26:46.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.844+0000 7fc212518700 1 -- 172.21.15.162:0/3848861428 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc20c0b18c0 msgr2=0x7fc20c0b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T22:26:46.849 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.844+0000 7fc212518700 1 --2- 172.21.15.162:0/3848861428 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc20c0b18c0 0x7fc20c0b3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:46.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.844+0000 7fc212518700 1 -- 172.21.15.162:0/3848861428 >> v1:172.21.15.118:6789/0 conn(0x7fc20c0b3950 legacy=0x7fc20c13dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:46.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.844+0000 7fc212518700 1 -- 172.21.15.162:0/3848861428 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc20c0aa8b0 con 0x7fc20c0b44e0 2020-11-28T22:26:46.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.844+0000 7fc212518700 1 --2- 172.21.15.162:0/3848861428 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc20c0b44e0 0x7fc20c0b2c10 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 rx=0x7fc2040032b0 tx=0x7fc204009270).ready entity=mon.1 client_cookie=2ddfb38e3b8c4366 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:46.850 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.844+0000 7fc210d15700 1 -- 172.21.15.162:0/3848861428 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc204009e60 con 0x7fc20c0b44e0 2020-11-28T22:26:46.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.844+0000 7fc210d15700 1 -- 172.21.15.162:0/3848861428 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20400bc00 con 0x7fc20c0b44e0 2020-11-28T22:26:46.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.844+0000 7fc210d15700 1 -- 172.21.15.162:0/3848861428 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc20400bd70 con 0x7fc20c0b44e0 2020-11-28T22:26:46.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.844+0000 7fc212d19700 1 -- 172.21.15.162:0/3848861428 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc20c0b44e0 msgr2=0x7fc20c0b2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:46.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.844+0000 7fc212d19700 1 --2- 172.21.15.162:0/3848861428 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc20c0b44e0 0x7fc20c0b2c10 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 rx=0x7fc2040032b0 tx=0x7fc204009270).stop 2020-11-28T22:26:46.851 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.844+0000 7fc212d19700 1 -- 172.21.15.162:0/3848861428 shutdown_connections 2020-11-28T22:26:46.852 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.844+0000 7fc212d19700 1 --2- 172.21.15.162:0/3848861428 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc20c0b44e0 0x7fc20c0b2c10 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:46.852 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.844+0000 7fc212d19700 1 --2- 172.21.15.162:0/3848861428 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc20c0b18c0 0x7fc20c0b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:46.852 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.844+0000 7fc212d19700 1 -- 172.21.15.162:0/3848861428 >> 172.21.15.162:0/3848861428 conn(0x7fc20c062630 msgr2=0x7fc20c062a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:26:46.852 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.844+0000 7fc212d19700 1 -- 172.21.15.162:0/3848861428 shutdown_connections 2020-11-28T22:26:46.853 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.844+0000 7fc212d19700 1 -- 172.21.15.162:0/3848861428 wait complete. 2020-11-28T22:26:46.855 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.848+0000 7fc212d19700 1 Processor -- start 2020-11-28T22:26:46.855 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.848+0000 7fc212d19700 1 -- start start 2020-11-28T22:26:46.855 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.848+0000 7fc212d19700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc20c0b18c0 0x7fc20c138d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:46.855 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.848+0000 7fc212d19700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc20c0b3950 0x7fc20c0aaec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:46.856 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.848+0000 7fc212d19700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc20c0b44e0 0x7fc20c14f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:46.856 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.848+0000 7fc212d19700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20c0b0af0 con 0x7fc20c0b3950 2020-11-28T22:26:46.856 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.848+0000 7fc212d19700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20c0b6460 con 0x7fc20c0b18c0 2020-11-28T22:26:46.856 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.848+0000 7fc212d19700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc20c0b6700 con 0x7fc20c0b44e0 2020-11-28T22:26:46.856 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.848+0000 7fc211d17700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc20c0b18c0 0x7fc20c138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:46.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.848+0000 7fc211d17700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc20c0b18c0 0x7fc20c138d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46946/0 (socket says 172.21.15.162:46946) 2020-11-28T22:26:46.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.848+0000 7fc211d17700 1 -- 172.21.15.162:0/2311373462 learned_addr learned my addr 172.21.15.162:0/2311373462 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T22:26:46.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.848+0000 7fc212518700 1 --2- 172.21.15.162:0/2311373462 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc20c0b44e0 0x7fc20c14f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:46.857 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.848+0000 7fc211516700 1 --2- 172.21.15.162:0/2311373462 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc20c0b3950 0x7fc20c0aaec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:46.858 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.848+0000 7fc211d17700 1 -- 172.21.15.162:0/2311373462 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc20c0b44e0 msgr2=0x7fc20c14f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:46.858 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.848+0000 7fc211d17700 1 --2- 172.21.15.162:0/2311373462 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc20c0b44e0 0x7fc20c14f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:46.858 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.848+0000 7fc211d17700 1 -- 172.21.15.162:0/2311373462 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc20c0b3950 msgr2=0x7fc20c0aaec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:46.858 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.848+0000 7fc211d17700 1 --2- 172.21.15.162:0/2311373462 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc20c0b3950 0x7fc20c0aaec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:46.859 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.848+0000 7fc211d17700 1 -- 172.21.15.162:0/2311373462 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc20c0b69a0 con 0x7fc20c0b18c0 2020-11-28T22:26:46.859 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.848+0000 7fc211d17700 1 --2- 172.21.15.162:0/2311373462 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc20c0b18c0 0x7fc20c138d10 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 rx=0x7fc20c0ac240 tx=0x7fc208007a60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:46.859 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.852+0000 7fc1faffd700 1 -- 172.21.15.162:0/2311373462 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc20800e4a0 con 0x7fc20c0b18c0 2020-11-28T22:26:46.859 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.852+0000 7fc1faffd700 1 -- 172.21.15.162:0/2311373462 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20800e600 con 0x7fc20c0b18c0 2020-11-28T22:26:46.859 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.852+0000 7fc1faffd700 1 -- 172.21.15.162:0/2311373462 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc20800e960 con 0x7fc20c0b18c0 2020-11-28T22:26:46.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.852+0000 7fc212d19700 1 -- 172.21.15.162:0/2311373462 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc20c14f4b0 con 0x7fc20c0b18c0 2020-11-28T22:26:46.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.852+0000 7fc212d19700 1 -- 172.21.15.162:0/2311373462 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc20c152a00 con 0x7fc20c0b18c0 2020-11-28T22:26:46.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.852+0000 7fc212d19700 1 -- 172.21.15.162:0/2311373462 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc20c14f6e0 con 0x7fc20c0b18c0 2020-11-28T22:26:46.860 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.852+0000 7fc1faffd700 1 -- 172.21.15.162:0/2311373462 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fc208010030 con 0x7fc20c0b18c0 2020-11-28T22:26:46.861 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.852+0000 7fc1faffd700 1 --2- 172.21.15.162:0/2311373462 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc1ec055470 0x7fc1ec057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:46.862 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.852+0000 7fc1faffd700 1 -- 172.21.15.162:0/2311373462 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5339+0+0 (secure 0 0 0) 0x7fc20806daf0 con 0x7fc20c0b18c0 2020-11-28T22:26:46.862 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.852+0000 7fc211516700 1 --2- 172.21.15.162:0/2311373462 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc1ec055470 0x7fc1ec057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:46.862 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.852+0000 7fc1faffd700 1 --2- 172.21.15.162:0/2311373462 >> [v2:172.21.15.118:6808/12529,v1:172.21.15.118:6809/12529] conn(0x7fc1ec05bb80 0x7fc1ec05dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:46.862 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.852+0000 7fc1faffd700 1 -- 172.21.15.162:0/2311373462 --> [v2:172.21.15.118:6808/12529,v1:172.21.15.118:6809/12529] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc1ec05e580 con 0x7fc1ec05bb80 2020-11-28T22:26:46.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.852+0000 7fc1faffd700 1 -- 172.21.15.162:0/2311373462 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=163) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc208072240 con 0x7fc20c0b18c0 2020-11-28T22:26:46.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.852+0000 7fc212518700 1 --2- 172.21.15.162:0/2311373462 >> [v2:172.21.15.118:6808/12529,v1:172.21.15.118:6809/12529] conn(0x7fc1ec05bb80 0x7fc1ec05dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:46.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.852+0000 7fc211516700 1 --2- 172.21.15.162:0/2311373462 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc1ec055470 0x7fc1ec057930 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7fc1fc00b000 tx=0x7fc1fc008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:46.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.856+0000 7fc212518700 1 --2- 172.21.15.162:0/2311373462 >> [v2:172.21.15.118:6808/12529,v1:172.21.15.118:6809/12529] conn(0x7fc1ec05bb80 0x7fc1ec05dfa0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:46.863 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.856+0000 7fc1faffd700 1 -- 172.21.15.162:0/2311373462 <== osd.5 v2:172.21.15.118:6808/12529 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7fc1ec05e580 con 0x7fc1ec05bb80 2020-11-28T22:26:46.879 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.876+0000 7fc212d19700 1 -- 172.21.15.162:0/2311373462 --> [v2:172.21.15.118:6808/12529,v1:172.21.15.118:6809/12529] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc20c057050 con 0x7fc1ec05bb80 2020-11-28T22:26:46.880 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.876+0000 7fc1faffd700 1 -- 172.21.15.162:0/2311373462 <== osd.5 v2:172.21.15.118:6808/12529 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fc20c057050 con 0x7fc1ec05bb80 2020-11-28T22:26:46.880 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2020-11-28T22:26:46.881 INFO:teuthology.orchestra.run.smithi162.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2020-11-28T22:26:46.881 INFO:teuthology.orchestra.run.smithi162.stdout:} 2020-11-28T22:26:46.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.876+0000 7fc212d19700 1 -- 172.21.15.162:0/2311373462 >> [v2:172.21.15.118:6808/12529,v1:172.21.15.118:6809/12529] conn(0x7fc1ec05bb80 msgr2=0x7fc1ec05dfa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:46.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.876+0000 7fc212d19700 1 --2- 172.21.15.162:0/2311373462 >> [v2:172.21.15.118:6808/12529,v1:172.21.15.118:6809/12529] conn(0x7fc1ec05bb80 0x7fc1ec05dfa0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:46.882 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.876+0000 7fc212d19700 1 -- 172.21.15.162:0/2311373462 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc1ec055470 msgr2=0x7fc1ec057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:46.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.876+0000 7fc212d19700 1 --2- 172.21.15.162:0/2311373462 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fc1ec055470 0x7fc1ec057930 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7fc1fc00b000 tx=0x7fc1fc008040).stop 2020-11-28T22:26:46.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.876+0000 7fc212d19700 1 -- 172.21.15.162:0/2311373462 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc20c0b18c0 msgr2=0x7fc20c138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:46.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.876+0000 7fc212d19700 1 --2- 172.21.15.162:0/2311373462 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc20c0b18c0 0x7fc20c138d10 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 rx=0x7fc20c0ac240 tx=0x7fc208007a60).stop 2020-11-28T22:26:46.883 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.876+0000 7fc212518700 1 -- 172.21.15.162:0/2311373462 reap_dead start 2020-11-28T22:26:46.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.880+0000 7fc212d19700 1 -- 172.21.15.162:0/2311373462 shutdown_connections 2020-11-28T22:26:46.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.880+0000 7fc212d19700 1 -- 172.21.15.162:0/2311373462 >> 172.21.15.162:0/2311373462 conn(0x7fc20c062630 msgr2=0x7fc20c1423b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:26:46.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.880+0000 7fc212d19700 1 -- 172.21.15.162:0/2311373462 shutdown_connections 2020-11-28T22:26:46.884 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:46.880+0000 7fc212d19700 1 -- 172.21.15.162:0/2311373462 wait complete. 2020-11-28T22:26:46.893 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd deep-scrub 5 2020-11-28T22:26:47.006 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.004+0000 7efff950e700 1 Processor -- start 2020-11-28T22:26:47.007 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.004+0000 7efff950e700 1 -- start start 2020-11-28T22:26:47.008 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.004+0000 7efff950e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7efff40615c0 0x7efff40b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:47.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.004+0000 7efff950e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efff40b1790 con 0x7efff40b4b40 2020-11-28T22:26:47.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.004+0000 7efff950e700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efff40b1a30 con 0x7efff40611e0 2020-11-28T22:26:47.009 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.004+0000 7efff950e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efff40b3f90 con 0x7efff40615c0 2020-11-28T22:26:47.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.004+0000 7efff3fff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7efff40611e0 0x7efff40b4430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:53800/0 (socket says 172.21.15.162:53800) 2020-11-28T22:26:47.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.004+0000 7efff3fff700 1 -- 172.21.15.162:0/3454468479 learned_addr learned my addr 172.21.15.162:0/3454468479 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T22:26:47.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.004+0000 7effebfff700 1 --2- 172.21.15.162:0/3454468479 >> v2:172.21.15.118:3300/0 conn(0x7efff40615c0 0x7efff40b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:47.010 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.004+0000 7efff37fe700 1 -- 172.21.15.162:0/3454468479 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4186032386 0 0) 0x7efff40b1a30 con 0x7efff40611e0 2020-11-28T22:26:47.011 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.004+0000 7efff37fe700 1 -- 172.21.15.162:0/3454468479 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effd4003230 con 0x7efff40611e0 2020-11-28T22:26:47.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.008+0000 7efff37fe700 1 -- 172.21.15.162:0/3454468479 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2656887612 0 0) 0x7efff40b1790 con 0x7efff40b4b40 2020-11-28T22:26:47.012 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.008+0000 7efff37fe700 1 -- 172.21.15.162:0/3454468479 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efff40b1a30 con 0x7efff40b4b40 2020-11-28T22:26:47.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.008+0000 7efff37fe700 1 -- 172.21.15.162:0/3454468479 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 217743187 0 0) 0x7effd4003230 con 0x7efff40611e0 2020-11-28T22:26:47.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.008+0000 7efff37fe700 1 -- 172.21.15.162:0/3454468479 >> v2:172.21.15.118:3300/0 conn(0x7efff40615c0 msgr2=0x7efff40b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T22:26:47.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.008+0000 7efff37fe700 1 --2- 172.21.15.162:0/3454468479 >> v2:172.21.15.118:3300/0 conn(0x7efff40615c0 0x7efff40b10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:47.013 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.008+0000 7efff37fe700 1 -- 172.21.15.162:0/3454468479 >> v1:172.21.15.118:6789/0 conn(0x7efff40b4b40 legacy=0x7efff413ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:47.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.008+0000 7efff37fe700 1 -- 172.21.15.162:0/3454468479 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff40b4f60 con 0x7efff40611e0 2020-11-28T22:26:47.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.008+0000 7efff37fe700 1 -- 172.21.15.162:0/3454468479 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1092449591 0 0) 0x7effe40016d0 con 0x7efff40611e0 2020-11-28T22:26:47.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.008+0000 7efff37fe700 1 -- 172.21.15.162:0/3454468479 >> v1:172.21.15.162:6789/0 conn(0x7efff40611e0 legacy=0x7efff40b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:47.014 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.008+0000 7efff37fe700 1 --2- 172.21.15.162:0/3454468479 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7effd40038b0 0x7effd4003cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:47.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.008+0000 7efff37fe700 1 --2- 172.21.15.162:0/3454468479 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7effd40044d0 0x7effd4004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:47.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.008+0000 7efff37fe700 1 --2- 172.21.15.162:0/3454468479 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7effd4004e40 0x7effd400fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:47.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.008+0000 7efff37fe700 1 -- 172.21.15.162:0/3454468479 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff40b3f90 con 0x7effd4004e40 2020-11-28T22:26:47.015 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.008+0000 7efff37fe700 1 -- 172.21.15.162:0/3454468479 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd40041f0 con 0x7effd40044d0 2020-11-28T22:26:47.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.008+0000 7efff37fe700 1 -- 172.21.15.162:0/3454468479 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effd4010220 con 0x7effd40038b0 2020-11-28T22:26:47.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.008+0000 7effebfff700 1 --2- 172.21.15.162:0/3454468479 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7effd40044d0 0x7effd4004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:47.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.008+0000 7efff3fff700 1 --2- 172.21.15.162:0/3454468479 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7effd40038b0 0x7effd4003cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:47.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.008+0000 7efff8d0d700 1 --2- 172.21.15.162:0/3454468479 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7effd4004e40 0x7effd400fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:47.016 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.008+0000 7effebfff700 1 -- 172.21.15.162:0/3454468479 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7effd40038b0 msgr2=0x7effd4003cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T22:26:47.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.008+0000 7effebfff700 1 --2- 172.21.15.162:0/3454468479 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7effd40038b0 0x7effd4003cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:47.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.008+0000 7effebfff700 1 -- 172.21.15.162:0/3454468479 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7effd4004e40 msgr2=0x7effd400fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T22:26:47.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.008+0000 7effebfff700 1 --2- 172.21.15.162:0/3454468479 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7effd4004e40 0x7effd400fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:47.017 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.008+0000 7efff8d0d700 1 -- 172.21.15.162:0/3454468479 reap_dead start 2020-11-28T22:26:47.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.008+0000 7effebfff700 1 -- 172.21.15.162:0/3454468479 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd4003230 con 0x7effd40044d0 2020-11-28T22:26:47.018 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.008+0000 7effebfff700 1 --2- 172.21.15.162:0/3454468479 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7effd40044d0 0x7effd4004950 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 rx=0x7effdc00c000 tx=0x7effdc008040).ready entity=mon.1 client_cookie=d11be0e1b625ccaa server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:47.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.008+0000 7efff37fe700 1 -- 172.21.15.162:0/3454468479 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effdc00e8a0 con 0x7effd40044d0 2020-11-28T22:26:47.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.008+0000 7efff37fe700 1 -- 172.21.15.162:0/3454468479 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effdc00ea00 con 0x7effd40044d0 2020-11-28T22:26:47.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.008+0000 7efff37fe700 1 -- 172.21.15.162:0/3454468479 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effdc00ed60 con 0x7effd40044d0 2020-11-28T22:26:47.021 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.008+0000 7efff950e700 1 -- 172.21.15.162:0/3454468479 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7effd40044d0 msgr2=0x7effd4004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:47.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.008+0000 7efff950e700 1 --2- 172.21.15.162:0/3454468479 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7effd40044d0 0x7effd4004950 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 rx=0x7effdc00c000 tx=0x7effdc008040).stop 2020-11-28T22:26:47.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.008+0000 7efff950e700 1 -- 172.21.15.162:0/3454468479 shutdown_connections 2020-11-28T22:26:47.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.008+0000 7efff950e700 1 --2- 172.21.15.162:0/3454468479 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7effd40044d0 0x7effd4004950 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:47.022 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.008+0000 7efff950e700 1 -- 172.21.15.162:0/3454468479 >> 172.21.15.162:0/3454468479 conn(0x7efff40b54f0 msgr2=0x7efff40b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:26:47.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.008+0000 7efff950e700 1 -- 172.21.15.162:0/3454468479 shutdown_connections 2020-11-28T22:26:47.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.008+0000 7efff950e700 1 -- 172.21.15.162:0/3454468479 wait complete. 2020-11-28T22:26:47.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.008+0000 7efff950e700 1 Processor -- start 2020-11-28T22:26:47.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.008+0000 7efff950e700 1 -- start start 2020-11-28T22:26:47.023 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.012+0000 7efff950e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7effd40044d0 0x7efff40aca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:47.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.012+0000 7efff950e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efff414d3e0 0x7efff414da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:47.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.012+0000 7efff950e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efff40ad310 0x7efff414df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:47.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.012+0000 7efff950e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff40af1a0 con 0x7efff414d3e0 2020-11-28T22:26:47.024 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.012+0000 7efff950e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff40aec70 con 0x7efff40ad310 2020-11-28T22:26:47.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.012+0000 7efff950e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efff40ad960 con 0x7effd40044d0 2020-11-28T22:26:47.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.012+0000 7efff8d0d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efff40ad310 0x7efff414df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:47.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.012+0000 7efff8d0d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efff40ad310 0x7efff414df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46970/0 (socket says 172.21.15.162:46970) 2020-11-28T22:26:47.025 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.012+0000 7efff8d0d700 1 -- 172.21.15.162:0/1617702751 learned_addr learned my addr 172.21.15.162:0/1617702751 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T22:26:47.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.012+0000 7efff3fff700 1 --2- 172.21.15.162:0/1617702751 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7effd40044d0 0x7efff40aca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:47.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.012+0000 7effebfff700 1 --2- 172.21.15.162:0/1617702751 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efff414d3e0 0x7efff414da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:47.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.012+0000 7efff8d0d700 1 -- 172.21.15.162:0/1617702751 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7effd40044d0 msgr2=0x7efff40aca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:47.026 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.012+0000 7efff8d0d700 1 --2- 172.21.15.162:0/1617702751 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7effd40044d0 0x7efff40aca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:47.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.012+0000 7efff8d0d700 1 -- 172.21.15.162:0/1617702751 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efff414d3e0 msgr2=0x7efff414da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:47.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.012+0000 7efff8d0d700 1 --2- 172.21.15.162:0/1617702751 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efff414d3e0 0x7efff414da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:47.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.012+0000 7efff8d0d700 1 -- 172.21.15.162:0/1617702751 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff409af80 con 0x7efff40ad310 2020-11-28T22:26:47.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.012+0000 7efff8d0d700 1 --2- 172.21.15.162:0/1617702751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efff40ad310 0x7efff414df60 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 rx=0x7effe40018b0 tx=0x7effec007ee0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:47.027 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.012+0000 7efff1ffb700 1 -- 172.21.15.162:0/1617702751 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effec006c60 con 0x7efff40ad310 2020-11-28T22:26:47.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.012+0000 7efff1ffb700 1 -- 172.21.15.162:0/1617702751 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effec006dc0 con 0x7efff40ad310 2020-11-28T22:26:47.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.012+0000 7efff1ffb700 1 -- 172.21.15.162:0/1617702751 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effec007220 con 0x7efff40ad310 2020-11-28T22:26:47.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.012+0000 7efff950e700 1 -- 172.21.15.162:0/1617702751 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efff40acef0 con 0x7efff40ad310 2020-11-28T22:26:47.028 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.012+0000 7efff950e700 1 -- 172.21.15.162:0/1617702751 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efff4140f60 con 0x7efff40ad310 2020-11-28T22:26:47.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.012+0000 7efff950e700 1 -- 172.21.15.162:0/1617702751 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efff40ad080 con 0x7efff40ad310 2020-11-28T22:26:47.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.016+0000 7efff1ffb700 1 -- 172.21.15.162:0/1617702751 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7effec01c070 con 0x7efff40ad310 2020-11-28T22:26:47.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.016+0000 7efff1ffb700 1 --2- 172.21.15.162:0/1617702751 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7effd8055470 0x7effd8057930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:47.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.016+0000 7efff1ffb700 1 -- 172.21.15.162:0/1617702751 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5339+0+0 (secure 0 0 0) 0x7effec06d210 con 0x7efff40ad310 2020-11-28T22:26:47.029 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.016+0000 7efff3fff700 1 --2- 172.21.15.162:0/1617702751 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7effd8055470 0x7effd8057930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:47.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.016+0000 7efff3fff700 1 --2- 172.21.15.162:0/1617702751 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7effd8055470 0x7effd8057930 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7effe4005000 tx=0x7effe4007740).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:47.030 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.020+0000 7efff1ffb700 1 -- 172.21.15.162:0/1617702751 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7effec0437f0 con 0x7efff40ad310 2020-11-28T22:26:47.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.160+0000 7efff950e700 1 -- 172.21.15.162:0/1617702751 --> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7efff40aab50 con 0x7effd8055470 2020-11-28T22:26:47.163 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.160+0000 7efff1ffb700 1 -- 172.21.15.162:0/1617702751 <== mgr.4105 v2:172.21.15.162:6800/12156 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7efff40aab50 con 0x7effd8055470 2020-11-28T22:26:47.165 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.164+0000 7efff950e700 1 -- 172.21.15.162:0/1617702751 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7effd8055470 msgr2=0x7effd8057930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:47.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.164+0000 7efff950e700 1 --2- 172.21.15.162:0/1617702751 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7effd8055470 0x7effd8057930 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7effe4005000 tx=0x7effe4007740).stop 2020-11-28T22:26:47.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.164+0000 7efff950e700 1 -- 172.21.15.162:0/1617702751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efff40ad310 msgr2=0x7efff414df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:47.166 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.164+0000 7efff950e700 1 --2- 172.21.15.162:0/1617702751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efff40ad310 0x7efff414df60 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 rx=0x7effe40018b0 tx=0x7effec007ee0).stop 2020-11-28T22:26:47.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.164+0000 7efff950e700 1 -- 172.21.15.162:0/1617702751 shutdown_connections 2020-11-28T22:26:47.167 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.164+0000 7efff950e700 1 --2- 172.21.15.162:0/1617702751 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7effd8055470 0x7effd8057930 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:47.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.164+0000 7efff950e700 1 --2- 172.21.15.162:0/1617702751 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7effd40044d0 0x7efff40aca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:47.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.164+0000 7efff950e700 1 --2- 172.21.15.162:0/1617702751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efff40ad310 0x7efff414df60 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:47.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.164+0000 7efff950e700 1 --2- 172.21.15.162:0/1617702751 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efff414d3e0 0x7efff414da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:47.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.164+0000 7efff950e700 1 -- 172.21.15.162:0/1617702751 >> 172.21.15.162:0/1617702751 conn(0x7efff40b54f0 msgr2=0x7efff405b970 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:26:47.168 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.164+0000 7efff950e700 1 -- 172.21.15.162:0/1617702751 shutdown_connections 2020-11-28T22:26:47.169 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.164+0000 7efff950e700 1 -- 172.21.15.162:0/1617702751 wait complete. 2020-11-28T22:26:47.169 INFO:teuthology.orchestra.run.smithi162.stderr:instructed osd(s) 5 to deep-scrub 2020-11-28T22:26:47.177 INFO:tasks.ceph:Scrubbing osd.6 2020-11-28T22:26:47.178 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.6 config set osd_debug_deep_scrub_sleep 0 2020-11-28T22:26:47.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.295+0000 7f4229ca1700 1 Processor -- start 2020-11-28T22:26:47.299 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.295+0000 7f4229ca1700 1 -- start start 2020-11-28T22:26:47.300 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.295+0000 7f4229ca1700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f42240b9ce0 0x7f42241433c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:47.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.295+0000 7f4229ca1700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4224067610 0x7f422413fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:47.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.295+0000 7f4229ca1700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4224143950 con 0x7f42240605c0 2020-11-28T22:26:47.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.295+0000 7f4229ca1700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42240b4340 con 0x7f42240b9ce0 2020-11-28T22:26:47.301 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.295+0000 7f4229ca1700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42240ba0c0 con 0x7f4224067610 2020-11-28T22:26:47.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.295+0000 7f42237fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f42240b9ce0 0x7f42241433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:47.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.295+0000 7f42237fe700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f42240b9ce0 0x7f42241433c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46976/0 (socket says 172.21.15.162:46976) 2020-11-28T22:26:47.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.295+0000 7f42237fe700 1 -- 172.21.15.162:0/490800958 learned_addr learned my addr 172.21.15.162:0/490800958 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T22:26:47.302 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.295+0000 7f4223fff700 1 --2- 172.21.15.162:0/490800958 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4224067610 0x7f422413fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:47.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.299+0000 7f42237fe700 1 -- 172.21.15.162:0/490800958 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4224067610 msgr2=0x7f422413fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T22:26:47.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.299+0000 7f42237fe700 1 --2- 172.21.15.162:0/490800958 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4224067610 0x7f422413fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:47.304 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.299+0000 7f42237fe700 1 -- 172.21.15.162:0/490800958 >> v1:172.21.15.118:6789/0 conn(0x7f42240605c0 legacy=0x7f42240b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:47.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.299+0000 7f42237fe700 1 -- 172.21.15.162:0/490800958 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4224067a70 con 0x7f42240b9ce0 2020-11-28T22:26:47.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.299+0000 7f42237fe700 1 --2- 172.21.15.162:0/490800958 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42240b9ce0 0x7f42241433c0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 rx=0x7f420c002f30 tx=0x7f420c006e20).ready entity=mon.1 client_cookie=c684a74943f61238 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:47.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.299+0000 7f42227fc700 1 -- 172.21.15.162:0/490800958 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f420c0097f0 con 0x7f42240b9ce0 2020-11-28T22:26:47.305 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.299+0000 7f42227fc700 1 -- 172.21.15.162:0/490800958 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f420c009950 con 0x7f42240b9ce0 2020-11-28T22:26:47.306 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.299+0000 7f42227fc700 1 -- 172.21.15.162:0/490800958 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f420c009cb0 con 0x7f42240b9ce0 2020-11-28T22:26:47.306 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.299+0000 7f4229ca1700 1 -- 172.21.15.162:0/490800958 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42240b9ce0 msgr2=0x7f42241433c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:47.306 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.299+0000 7f4229ca1700 1 --2- 172.21.15.162:0/490800958 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42240b9ce0 0x7f42241433c0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 rx=0x7f420c002f30 tx=0x7f420c006e20).stop 2020-11-28T22:26:47.306 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.299+0000 7f4229ca1700 1 -- 172.21.15.162:0/490800958 shutdown_connections 2020-11-28T22:26:47.307 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.299+0000 7f4229ca1700 1 --2- 172.21.15.162:0/490800958 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4224067610 0x7f422413fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:47.307 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.299+0000 7f4229ca1700 1 --2- 172.21.15.162:0/490800958 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42240b9ce0 0x7f42241433c0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:47.307 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.299+0000 7f4229ca1700 1 -- 172.21.15.162:0/490800958 >> 172.21.15.162:0/490800958 conn(0x7f422405c430 msgr2=0x7f42240b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:26:47.307 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.299+0000 7f4229ca1700 1 -- 172.21.15.162:0/490800958 shutdown_connections 2020-11-28T22:26:47.307 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.299+0000 7f4229ca1700 1 -- 172.21.15.162:0/490800958 wait complete. 2020-11-28T22:26:47.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.299+0000 7f4229ca1700 1 Processor -- start 2020-11-28T22:26:47.308 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.299+0000 7f4229ca1700 1 -- start start 2020-11-28T22:26:47.310 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.299+0000 7f4229ca1700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f42240605c0 0x7f42240b5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:47.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.299+0000 7f4229ca1700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4224067610 0x7f4224151e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:47.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.299+0000 7f4229ca1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42240b9ce0 0x7f4224152300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:47.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.299+0000 7f4229ca1700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42240b6b00 con 0x7f42240605c0 2020-11-28T22:26:47.311 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.299+0000 7f4229ca1700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42240b3160 con 0x7f42240b9ce0 2020-11-28T22:26:47.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.299+0000 7f4229ca1700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42240b2ec0 con 0x7f4224067610 2020-11-28T22:26:47.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.299+0000 7f4223fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42240b9ce0 0x7f4224152300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:47.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.299+0000 7f4223fff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42240b9ce0 0x7f4224152300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46986/0 (socket says 172.21.15.162:46986) 2020-11-28T22:26:47.312 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.299+0000 7f4223fff700 1 -- 172.21.15.162:0/3155918432 learned_addr learned my addr 172.21.15.162:0/3155918432 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T22:26:47.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.303+0000 7f4222ffd700 1 --2- 172.21.15.162:0/3155918432 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4224067610 0x7f4224151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:47.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.303+0000 7f42237fe700 1 --2- 172.21.15.162:0/3155918432 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f42240605c0 0x7f42240b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:47.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.303+0000 7f4223fff700 1 -- 172.21.15.162:0/3155918432 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4224067610 msgr2=0x7f4224151e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:47.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.303+0000 7f4223fff700 1 --2- 172.21.15.162:0/3155918432 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4224067610 0x7f4224151e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:47.313 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.303+0000 7f4223fff700 1 -- 172.21.15.162:0/3155918432 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f42240605c0 msgr2=0x7f42240b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:47.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.303+0000 7f4223fff700 1 --2- 172.21.15.162:0/3155918432 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f42240605c0 0x7f42240b5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:47.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.303+0000 7f4223fff700 1 -- 172.21.15.162:0/3155918432 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42240b6420 con 0x7f42240b9ce0 2020-11-28T22:26:47.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.303+0000 7f42237fe700 1 --2- 172.21.15.162:0/3155918432 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f42240605c0 0x7f42240b5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T22:26:47.314 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.303+0000 7f4223fff700 1 --2- 172.21.15.162:0/3155918432 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42240b9ce0 0x7f4224152300 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 rx=0x7f421c006dd0 tx=0x7f421c0091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:47.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.303+0000 7f4220ff9700 1 -- 172.21.15.162:0/3155918432 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f421c00eee0 con 0x7f42240b9ce0 2020-11-28T22:26:47.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.303+0000 7f4220ff9700 1 -- 172.21.15.162:0/3155918432 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f421c00cbb0 con 0x7f42240b9ce0 2020-11-28T22:26:47.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.303+0000 7f4229ca1700 1 -- 172.21.15.162:0/3155918432 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4224146d90 con 0x7f42240b9ce0 2020-11-28T22:26:47.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.303+0000 7f4220ff9700 1 -- 172.21.15.162:0/3155918432 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f421c0123f0 con 0x7f42240b9ce0 2020-11-28T22:26:47.315 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.303+0000 7f4229ca1700 1 -- 172.21.15.162:0/3155918432 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42240b2a40 con 0x7f42240b9ce0 2020-11-28T22:26:47.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.303+0000 7f4229ca1700 1 -- 172.21.15.162:0/3155918432 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f42240b65b0 con 0x7f42240b9ce0 2020-11-28T22:26:47.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.303+0000 7f4220ff9700 1 -- 172.21.15.162:0/3155918432 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f421c022070 con 0x7f42240b9ce0 2020-11-28T22:26:47.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.303+0000 7f4220ff9700 1 --2- 172.21.15.162:0/3155918432 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f42080554e0 0x7f42080579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:47.316 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.303+0000 7f4220ff9700 1 -- 172.21.15.162:0/3155918432 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f421c06e3a0 con 0x7f42240b9ce0 2020-11-28T22:26:47.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.303+0000 7f42237fe700 1 --2- 172.21.15.162:0/3155918432 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f42080554e0 0x7f42080579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:47.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.303+0000 7f4220ff9700 1 --2- 172.21.15.162:0/3155918432 >> [v2:172.21.15.118:6824/12530,v1:172.21.15.118:6825/12530] conn(0x7f420805bbf0 0x7f420805e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:47.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.303+0000 7f4220ff9700 1 -- 172.21.15.162:0/3155918432 --> [v2:172.21.15.118:6824/12530,v1:172.21.15.118:6825/12530] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f420805e5f0 con 0x7f420805bbf0 2020-11-28T22:26:47.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.303+0000 7f4220ff9700 1 -- 172.21.15.162:0/3155918432 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=163) v2 ==== 24+0+0 (secure 0 0 0) 0x7f421c070a30 con 0x7f42240b9ce0 2020-11-28T22:26:47.317 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.303+0000 7f4222ffd700 1 --2- 172.21.15.162:0/3155918432 >> [v2:172.21.15.118:6824/12530,v1:172.21.15.118:6825/12530] conn(0x7f420805bbf0 0x7f420805e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:47.318 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.303+0000 7f42237fe700 1 --2- 172.21.15.162:0/3155918432 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f42080554e0 0x7f42080579a0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7f420c016000 tx=0x7f420c015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:47.318 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.307+0000 7f4222ffd700 1 --2- 172.21.15.162:0/3155918432 >> [v2:172.21.15.118:6824/12530,v1:172.21.15.118:6825/12530] conn(0x7f420805bbf0 0x7f420805e010 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:47.318 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.307+0000 7f4220ff9700 1 -- 172.21.15.162:0/3155918432 <== osd.6 v2:172.21.15.118:6824/12530 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7f420805e5f0 con 0x7f420805bbf0 2020-11-28T22:26:47.330 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.327+0000 7f4229ca1700 1 -- 172.21.15.162:0/3155918432 --> [v2:172.21.15.118:6824/12530,v1:172.21.15.118:6825/12530] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f42240b6280 con 0x7f420805bbf0 2020-11-28T22:26:47.332 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.327+0000 7f4220ff9700 1 -- 172.21.15.162:0/3155918432 <== osd.6 v2:172.21.15.118:6824/12530 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f42240b6280 con 0x7f420805bbf0 2020-11-28T22:26:47.332 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2020-11-28T22:26:47.332 INFO:teuthology.orchestra.run.smithi162.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2020-11-28T22:26:47.333 INFO:teuthology.orchestra.run.smithi162.stdout:} 2020-11-28T22:26:47.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.327+0000 7f4229ca1700 1 -- 172.21.15.162:0/3155918432 >> [v2:172.21.15.118:6824/12530,v1:172.21.15.118:6825/12530] conn(0x7f420805bbf0 msgr2=0x7f420805e010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:47.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.327+0000 7f4229ca1700 1 --2- 172.21.15.162:0/3155918432 >> [v2:172.21.15.118:6824/12530,v1:172.21.15.118:6825/12530] conn(0x7f420805bbf0 0x7f420805e010 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:47.334 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.327+0000 7f4229ca1700 1 -- 172.21.15.162:0/3155918432 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f42080554e0 msgr2=0x7f42080579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:47.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.327+0000 7f4229ca1700 1 --2- 172.21.15.162:0/3155918432 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f42080554e0 0x7f42080579a0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7f420c016000 tx=0x7f420c015000).stop 2020-11-28T22:26:47.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.327+0000 7f4229ca1700 1 -- 172.21.15.162:0/3155918432 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42240b9ce0 msgr2=0x7f4224152300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:47.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.327+0000 7f4229ca1700 1 --2- 172.21.15.162:0/3155918432 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42240b9ce0 0x7f4224152300 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 rx=0x7f421c006dd0 tx=0x7f421c0091a0).stop 2020-11-28T22:26:47.335 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.327+0000 7f4223fff700 1 -- 172.21.15.162:0/3155918432 reap_dead start 2020-11-28T22:26:47.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.327+0000 7f4229ca1700 1 -- 172.21.15.162:0/3155918432 shutdown_connections 2020-11-28T22:26:47.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.327+0000 7f4229ca1700 1 -- 172.21.15.162:0/3155918432 >> 172.21.15.162:0/3155918432 conn(0x7f422405c430 msgr2=0x7f422413f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:26:47.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.327+0000 7f4229ca1700 1 -- 172.21.15.162:0/3155918432 shutdown_connections 2020-11-28T22:26:47.336 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.327+0000 7f4229ca1700 1 -- 172.21.15.162:0/3155918432 wait complete. 2020-11-28T22:26:47.344 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd deep-scrub 6 2020-11-28T22:26:47.460 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.455+0000 7f2a07b51700 1 Processor -- start 2020-11-28T22:26:47.460 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.455+0000 7f2a07b51700 1 -- start start 2020-11-28T22:26:47.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.455+0000 7f2a07b51700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2a000b1ae0 0x7f2a0013cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:47.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.455+0000 7f2a07b51700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a000b1ec0 con 0x7f2a000aef20 2020-11-28T22:26:47.461 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.455+0000 7f2a07b51700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a0013fef0 con 0x7f2a0013f9e0 2020-11-28T22:26:47.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.455+0000 7f2a07b51700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a00140190 con 0x7f2a000b1ae0 2020-11-28T22:26:47.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.459+0000 7f2a060ee700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f2a0013f9e0 0x7f2a000aca00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:53840/0 (socket says 172.21.15.162:53840) 2020-11-28T22:26:47.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.459+0000 7f2a060ee700 1 -- 172.21.15.162:0/4059162221 learned_addr learned my addr 172.21.15.162:0/4059162221 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T22:26:47.462 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.459+0000 7f2a058ed700 1 --2- 172.21.15.162:0/4059162221 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2a000b1ae0 0x7f2a0013cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:47.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.459+0000 7f2a048eb700 1 -- 172.21.15.162:0/4059162221 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2491982337 0 0) 0x7f2a0013fef0 con 0x7f2a0013f9e0 2020-11-28T22:26:47.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.459+0000 7f2a048eb700 1 -- 172.21.15.162:0/4059162221 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29e8003230 con 0x7f2a0013f9e0 2020-11-28T22:26:47.464 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.459+0000 7f2a048eb700 1 -- 172.21.15.162:0/4059162221 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1744934797 0 0) 0x7f2a000b1ec0 con 0x7f2a000aef20 2020-11-28T22:26:47.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.459+0000 7f2a048eb700 1 -- 172.21.15.162:0/4059162221 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a0013fef0 con 0x7f2a000aef20 2020-11-28T22:26:47.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.459+0000 7f2a058ed700 1 -- 172.21.15.162:0/4059162221 >> v1:172.21.15.162:6789/0 conn(0x7f2a0013f9e0 legacy=0x7f2a000aca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:47.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.459+0000 7f2a058ed700 1 -- 172.21.15.162:0/4059162221 >> v1:172.21.15.118:6789/0 conn(0x7f2a000aef20 legacy=0x7f2a000ac2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:47.465 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.459+0000 7f2a058ed700 1 -- 172.21.15.162:0/4059162221 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a00140430 con 0x7f2a000b1ae0 2020-11-28T22:26:47.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.459+0000 7f2a058ed700 1 --2- 172.21.15.162:0/4059162221 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2a000b1ae0 0x7f2a0013cc90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f29fc002f30 tx=0x7f29fc006dc0).ready entity=mon.2 client_cookie=6662856c82c2c821 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:47.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.459+0000 7f2a048eb700 1 -- 172.21.15.162:0/4059162221 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29fc009730 con 0x7f2a000b1ae0 2020-11-28T22:26:47.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.459+0000 7f2a048eb700 1 -- 172.21.15.162:0/4059162221 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29fc009890 con 0x7f2a000b1ae0 2020-11-28T22:26:47.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.459+0000 7f2a048eb700 1 -- 172.21.15.162:0/4059162221 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29fc009bf0 con 0x7f2a000b1ae0 2020-11-28T22:26:47.466 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.459+0000 7f2a07b51700 1 -- 172.21.15.162:0/4059162221 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2a000b1ae0 msgr2=0x7f2a0013cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:47.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.459+0000 7f2a07b51700 1 --2- 172.21.15.162:0/4059162221 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2a000b1ae0 0x7f2a0013cc90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f29fc002f30 tx=0x7f29fc006dc0).stop 2020-11-28T22:26:47.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.459+0000 7f2a07b51700 1 -- 172.21.15.162:0/4059162221 shutdown_connections 2020-11-28T22:26:47.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.459+0000 7f2a07b51700 1 --2- 172.21.15.162:0/4059162221 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2a000b1ae0 0x7f2a0013cc90 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:47.467 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.459+0000 7f2a07b51700 1 -- 172.21.15.162:0/4059162221 >> 172.21.15.162:0/4059162221 conn(0x7f2a0005b970 msgr2=0x7f2a0005bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:26:47.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.459+0000 7f2a07b51700 1 -- 172.21.15.162:0/4059162221 shutdown_connections 2020-11-28T22:26:47.468 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.459+0000 7f2a07b51700 1 -- 172.21.15.162:0/4059162221 wait complete. 2020-11-28T22:26:47.470 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.463+0000 7f2a07b51700 1 Processor -- start 2020-11-28T22:26:47.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.463+0000 7f2a07b51700 1 -- start start 2020-11-28T22:26:47.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.463+0000 7f2a07b51700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a000aef20 0x7f2a00142fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:47.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.463+0000 7f2a07b51700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2a000b1ae0 0x7f2a0014eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:47.471 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.463+0000 7f2a07b51700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2a0013f9e0 0x7f2a0014f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:47.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.463+0000 7f2a07b51700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a000673e0 con 0x7f2a000b1ae0 2020-11-28T22:26:47.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.463+0000 7f2a07b51700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0014e710 con 0x7f2a000aef20 2020-11-28T22:26:47.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.463+0000 7f2a07b51700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a000660d0 con 0x7f2a0013f9e0 2020-11-28T22:26:47.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.463+0000 7f2a058ed700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a000aef20 0x7f2a00142fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:47.472 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.463+0000 7f2a058ed700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a000aef20 0x7f2a00142fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46998/0 (socket says 172.21.15.162:46998) 2020-11-28T22:26:47.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.463+0000 7f2a058ed700 1 -- 172.21.15.162:0/2062968245 learned_addr learned my addr 172.21.15.162:0/2062968245 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T22:26:47.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.463+0000 7f2a050ec700 1 --2- 172.21.15.162:0/2062968245 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2a000b1ae0 0x7f2a0014eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:47.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.463+0000 7f2a060ee700 1 --2- 172.21.15.162:0/2062968245 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2a0013f9e0 0x7f2a0014f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:47.473 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.463+0000 7f2a058ed700 1 -- 172.21.15.162:0/2062968245 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2a0013f9e0 msgr2=0x7f2a0014f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:47.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.463+0000 7f2a058ed700 1 --2- 172.21.15.162:0/2062968245 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2a0013f9e0 0x7f2a0014f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:47.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.463+0000 7f2a058ed700 1 -- 172.21.15.162:0/2062968245 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2a000b1ae0 msgr2=0x7f2a0014eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:47.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.463+0000 7f2a058ed700 1 --2- 172.21.15.162:0/2062968245 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2a000b1ae0 0x7f2a0014eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:47.474 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.463+0000 7f2a058ed700 1 -- 172.21.15.162:0/2062968245 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a00066370 con 0x7f2a000aef20 2020-11-28T22:26:47.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.463+0000 7f2a058ed700 1 --2- 172.21.15.162:0/2062968245 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a000aef20 0x7f2a00142fd0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 rx=0x7f29fc002f30 tx=0x7f29fc015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:47.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.463+0000 7f29eeffd700 1 -- 172.21.15.162:0/2062968245 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29fc011c40 con 0x7f2a000aef20 2020-11-28T22:26:47.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.463+0000 7f29eeffd700 1 -- 172.21.15.162:0/2062968245 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29fc011da0 con 0x7f2a000aef20 2020-11-28T22:26:47.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.463+0000 7f2a07b51700 1 -- 172.21.15.162:0/2062968245 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a00066610 con 0x7f2a000aef20 2020-11-28T22:26:47.475 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.463+0000 7f29eeffd700 1 -- 172.21.15.162:0/2062968245 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29fc01d640 con 0x7f2a000aef20 2020-11-28T22:26:47.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.463+0000 7f2a07b51700 1 -- 172.21.15.162:0/2062968245 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a0014f520 con 0x7f2a000aef20 2020-11-28T22:26:47.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.467+0000 7f2a07b51700 1 -- 172.21.15.162:0/2062968245 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a00057050 con 0x7f2a000aef20 2020-11-28T22:26:47.476 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.467+0000 7f29eeffd700 1 -- 172.21.15.162:0/2062968245 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f29fc024030 con 0x7f2a000aef20 2020-11-28T22:26:47.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.467+0000 7f29eeffd700 1 --2- 172.21.15.162:0/2062968245 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f29e00553b0 0x7f29e0057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:47.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.467+0000 7f29eeffd700 1 -- 172.21.15.162:0/2062968245 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f29fc06ecf0 con 0x7f2a000aef20 2020-11-28T22:26:47.477 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.467+0000 7f2a050ec700 1 --2- 172.21.15.162:0/2062968245 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f29e00553b0 0x7f29e0057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:47.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.467+0000 7f2a050ec700 1 --2- 172.21.15.162:0/2062968245 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f29e00553b0 0x7f29e0057870 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7f29f000b000 tx=0x7f29f0009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:47.478 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.475+0000 7f29eeffd700 1 -- 172.21.15.162:0/2062968245 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f29fc0452d0 con 0x7f2a000aef20 2020-11-28T22:26:47.621 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.615+0000 7f2a07b51700 1 -- 172.21.15.162:0/2062968245 --> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a0006b130 con 0x7f29e00553b0 2020-11-28T22:26:47.621 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.619+0000 7f29eeffd700 1 -- 172.21.15.162:0/2062968245 <== mgr.4105 v2:172.21.15.162:6800/12156 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2a0006b130 con 0x7f29e00553b0 2020-11-28T22:26:47.623 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.619+0000 7f2a07b51700 1 -- 172.21.15.162:0/2062968245 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f29e00553b0 msgr2=0x7f29e0057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:47.624 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.619+0000 7f2a07b51700 1 --2- 172.21.15.162:0/2062968245 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f29e00553b0 0x7f29e0057870 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7f29f000b000 tx=0x7f29f0009040).stop 2020-11-28T22:26:47.624 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.619+0000 7f2a07b51700 1 -- 172.21.15.162:0/2062968245 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a000aef20 msgr2=0x7f2a00142fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:47.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.619+0000 7f2a07b51700 1 --2- 172.21.15.162:0/2062968245 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a000aef20 0x7f2a00142fd0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 rx=0x7f29fc002f30 tx=0x7f29fc015000).stop 2020-11-28T22:26:47.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.619+0000 7f2a07b51700 1 -- 172.21.15.162:0/2062968245 shutdown_connections 2020-11-28T22:26:47.625 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.619+0000 7f2a07b51700 1 --2- 172.21.15.162:0/2062968245 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f29e00553b0 0x7f29e0057870 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:47.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.619+0000 7f2a07b51700 1 --2- 172.21.15.162:0/2062968245 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2a0013f9e0 0x7f2a0014f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:47.626 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.619+0000 7f2a07b51700 1 --2- 172.21.15.162:0/2062968245 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a000aef20 0x7f2a00142fd0 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:47.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.619+0000 7f2a07b51700 1 --2- 172.21.15.162:0/2062968245 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2a000b1ae0 0x7f2a0014eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:47.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.619+0000 7f2a07b51700 1 -- 172.21.15.162:0/2062968245 >> 172.21.15.162:0/2062968245 conn(0x7f2a0005b970 msgr2=0x7f2a0013c760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:26:47.627 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.619+0000 7f2a07b51700 1 -- 172.21.15.162:0/2062968245 shutdown_connections 2020-11-28T22:26:47.628 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.619+0000 7f2a07b51700 1 -- 172.21.15.162:0/2062968245 wait complete. 2020-11-28T22:26:47.629 INFO:teuthology.orchestra.run.smithi162.stderr:instructed osd(s) 6 to deep-scrub 2020-11-28T22:26:47.634 INFO:tasks.ceph:Scrubbing osd.7 2020-11-28T22:26:47.635 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.7 config set osd_debug_deep_scrub_sleep 0 2020-11-28T22:26:47.752 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.747+0000 7f9a9618a700 1 Processor -- start 2020-11-28T22:26:47.752 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.747+0000 7f9a9618a700 1 -- start start 2020-11-28T22:26:47.755 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.747+0000 7f9a9618a700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9a90066a50 0x7f9a901409d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:47.756 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.747+0000 7f9a9618a700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9a90066e30 0x7f9a90144620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:47.756 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.747+0000 7f9a9618a700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a900b1dd0 con 0x7f9a90144b10 2020-11-28T22:26:47.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.747+0000 7f9a9618a700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a90143f20 con 0x7f9a90066a50 2020-11-28T22:26:47.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.747+0000 7f9a9618a700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a900b2000 con 0x7f9a90066e30 2020-11-28T22:26:47.757 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.747+0000 7f9a8ffff700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f9a90144b10 0x7f9a900b37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:53852/0 (socket says 172.21.15.162:53852) 2020-11-28T22:26:47.758 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.747+0000 7f9a8ffff700 1 -- 172.21.15.162:0/2843160293 learned_addr learned my addr 172.21.15.162:0/2843160293 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T22:26:47.758 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.747+0000 7f9a8f7fe700 1 --2- 172.21.15.162:0/2843160293 >> v2:172.21.15.162:3300/0 conn(0x7f9a90066a50 0x7f9a901409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:47.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.747+0000 7f9a8effd700 1 --2- 172.21.15.162:0/2843160293 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9a90066e30 0x7f9a90144620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:47.759 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.747+0000 7f9a8e7fc700 1 -- 172.21.15.162:0/2843160293 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3693933355 0 0) 0x7f9a900b1dd0 con 0x7f9a90144b10 2020-11-28T22:26:47.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.751+0000 7f9a8e7fc700 1 -- 172.21.15.162:0/2843160293 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a7c003230 con 0x7f9a90144b10 2020-11-28T22:26:47.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.751+0000 7f9a8f7fe700 1 -- 172.21.15.162:0/2843160293 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9a90066e30 msgr2=0x7f9a90144620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T22:26:47.760 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.751+0000 7f9a8e7fc700 1 -- 172.21.15.162:0/2843160293 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 818493182 0 0) 0x7f9a7c003230 con 0x7f9a90144b10 2020-11-28T22:26:47.761 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.751+0000 7f9a8f7fe700 1 --2- 172.21.15.162:0/2843160293 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9a90066e30 0x7f9a90144620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:47.761 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.751+0000 7f9a8f7fe700 1 -- 172.21.15.162:0/2843160293 >> v1:172.21.15.162:6789/0 conn(0x7f9a90144b10 legacy=0x7f9a900b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:47.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.751+0000 7f9a8f7fe700 1 -- 172.21.15.162:0/2843160293 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a90144180 con 0x7f9a90066a50 2020-11-28T22:26:47.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.751+0000 7f9a8f7fe700 1 --2- 172.21.15.162:0/2843160293 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a90066a50 0x7f9a901409d0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 rx=0x7f9a88001870 tx=0x7f9a84006dc0).ready entity=mon.1 client_cookie=8410c6c0c2904827 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:47.762 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.751+0000 7f9a8e7fc700 1 -- 172.21.15.162:0/2843160293 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a880016d0 con 0x7f9a90066a50 2020-11-28T22:26:47.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.751+0000 7f9a8e7fc700 1 -- 172.21.15.162:0/2843160293 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8400d570 con 0x7f9a90066a50 2020-11-28T22:26:47.763 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.751+0000 7f9a8e7fc700 1 -- 172.21.15.162:0/2843160293 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a8400d6e0 con 0x7f9a90066a50 2020-11-28T22:26:47.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.751+0000 7f9a9618a700 1 -- 172.21.15.162:0/2843160293 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a90066a50 msgr2=0x7f9a901409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:47.764 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.751+0000 7f9a9618a700 1 --2- 172.21.15.162:0/2843160293 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a90066a50 0x7f9a901409d0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 rx=0x7f9a88001870 tx=0x7f9a84006dc0).stop 2020-11-28T22:26:47.770 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.751+0000 7f9a9618a700 1 -- 172.21.15.162:0/2843160293 shutdown_connections 2020-11-28T22:26:47.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.751+0000 7f9a9618a700 1 --2- 172.21.15.162:0/2843160293 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a90066a50 0x7f9a901409d0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:47.771 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.751+0000 7f9a9618a700 1 --2- 172.21.15.162:0/2843160293 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9a90066e30 0x7f9a90144620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:47.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.751+0000 7f9a9618a700 1 -- 172.21.15.162:0/2843160293 >> 172.21.15.162:0/2843160293 conn(0x7f9a9005b970 msgr2=0x7f9a9005bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:26:47.772 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.751+0000 7f9a9618a700 1 -- 172.21.15.162:0/2843160293 shutdown_connections 2020-11-28T22:26:47.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.751+0000 7f9a9618a700 1 -- 172.21.15.162:0/2843160293 wait complete. 2020-11-28T22:26:47.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.751+0000 7f9a9618a700 1 Processor -- start 2020-11-28T22:26:47.773 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.751+0000 7f9a9618a700 1 -- start start 2020-11-28T22:26:47.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.755+0000 7f9a9618a700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9a90066a50 0x7f9a9014fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:47.774 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.755+0000 7f9a9618a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9a90066e30 0x7f9a90139f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:47.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.755+0000 7f9a9618a700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a90144b10 0x7f9a9013a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:47.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.755+0000 7f9a9618a700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a900ad550 con 0x7f9a90066e30 2020-11-28T22:26:47.775 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.755+0000 7f9a9618a700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a900ac660 con 0x7f9a90144b10 2020-11-28T22:26:47.776 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.755+0000 7f9a9618a700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a90142f10 con 0x7f9a90066a50 2020-11-28T22:26:47.776 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.755+0000 7f9a8ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a90144b10 0x7f9a9013a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:47.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.755+0000 7f9a8ffff700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a90144b10 0x7f9a9013a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47016/0 (socket says 172.21.15.162:47016) 2020-11-28T22:26:47.777 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.755+0000 7f9a8ffff700 1 -- 172.21.15.162:0/476674336 learned_addr learned my addr 172.21.15.162:0/476674336 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T22:26:47.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.755+0000 7f9a8f7fe700 1 --2- 172.21.15.162:0/476674336 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9a90066a50 0x7f9a9014fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:47.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.755+0000 7f9a8effd700 1 --2- 172.21.15.162:0/476674336 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9a90066e30 0x7f9a90139f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:47.778 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.755+0000 7f9a8ffff700 1 -- 172.21.15.162:0/476674336 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9a90066a50 msgr2=0x7f9a9014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:47.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.755+0000 7f9a8ffff700 1 --2- 172.21.15.162:0/476674336 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9a90066a50 0x7f9a9014fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:47.779 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.755+0000 7f9a8ffff700 1 -- 172.21.15.162:0/476674336 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9a90066e30 msgr2=0x7f9a90139f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:47.782 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.755+0000 7f9a8ffff700 1 --2- 172.21.15.162:0/476674336 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9a90066e30 0x7f9a90139f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:47.782 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.755+0000 7f9a8ffff700 1 -- 172.21.15.162:0/476674336 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a900ab3c0 con 0x7f9a90144b10 2020-11-28T22:26:47.783 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.755+0000 7f9a8effd700 1 --2- 172.21.15.162:0/476674336 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9a90066e30 0x7f9a90139f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T22:26:47.783 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.755+0000 7f9a8ffff700 1 --2- 172.21.15.162:0/476674336 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a90144b10 0x7f9a9013a400 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 rx=0x7f9a88001930 tx=0x7f9a88001d90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:47.784 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.755+0000 7f9a8cff9700 1 -- 172.21.15.162:0/476674336 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a88007170 con 0x7f9a90144b10 2020-11-28T22:26:47.784 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.755+0000 7f9a8cff9700 1 -- 172.21.15.162:0/476674336 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a880072d0 con 0x7f9a90144b10 2020-11-28T22:26:47.785 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.755+0000 7f9a8cff9700 1 -- 172.21.15.162:0/476674336 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a880048f0 con 0x7f9a90144b10 2020-11-28T22:26:47.785 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.755+0000 7f9a9618a700 1 -- 172.21.15.162:0/476674336 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a900ac120 con 0x7f9a90144b10 2020-11-28T22:26:47.785 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.755+0000 7f9a9618a700 1 -- 172.21.15.162:0/476674336 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a900aaeb0 con 0x7f9a90144b10 2020-11-28T22:26:47.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.755+0000 7f9a9618a700 1 -- 172.21.15.162:0/476674336 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9a90057050 con 0x7f9a90144b10 2020-11-28T22:26:47.786 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.759+0000 7f9a8cff9700 1 -- 172.21.15.162:0/476674336 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f9a88004a50 con 0x7f9a90144b10 2020-11-28T22:26:47.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.759+0000 7f9a8cff9700 1 --2- 172.21.15.162:0/476674336 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9a700554e0 0x7f9a700579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:47.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.759+0000 7f9a8cff9700 1 -- 172.21.15.162:0/476674336 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f9a8806d6f0 con 0x7f9a90144b10 2020-11-28T22:26:47.787 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.759+0000 7f9a8f7fe700 1 --2- 172.21.15.162:0/476674336 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9a700554e0 0x7f9a700579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:47.788 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.759+0000 7f9a8cff9700 1 --2- 172.21.15.162:0/476674336 >> [v2:172.21.15.118:6816/12528,v1:172.21.15.118:6817/12528] conn(0x7f9a7005bbf0 0x7f9a7005e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:47.788 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.759+0000 7f9a8cff9700 1 -- 172.21.15.162:0/476674336 --> [v2:172.21.15.118:6816/12528,v1:172.21.15.118:6817/12528] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9a7005e5f0 con 0x7f9a7005bbf0 2020-11-28T22:26:47.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.759+0000 7f9a8cff9700 1 -- 172.21.15.162:0/476674336 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=163) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a8806fc70 con 0x7f9a90144b10 2020-11-28T22:26:47.789 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.759+0000 7f9a8f7fe700 1 --2- 172.21.15.162:0/476674336 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9a700554e0 0x7f9a700579a0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7f9a8401a000 tx=0x7f9a84015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:47.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.759+0000 7f9a8effd700 1 --2- 172.21.15.162:0/476674336 >> [v2:172.21.15.118:6816/12528,v1:172.21.15.118:6817/12528] conn(0x7f9a7005bbf0 0x7f9a7005e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:47.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.759+0000 7f9a8effd700 1 --2- 172.21.15.162:0/476674336 >> [v2:172.21.15.118:6816/12528,v1:172.21.15.118:6817/12528] conn(0x7f9a7005bbf0 0x7f9a7005e010 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:47.790 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.763+0000 7f9a8cff9700 1 -- 172.21.15.162:0/476674336 <== osd.7 v2:172.21.15.118:6816/12528 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7f9a7005e5f0 con 0x7f9a7005bbf0 2020-11-28T22:26:47.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.779+0000 7f9a9618a700 1 -- 172.21.15.162:0/476674336 --> [v2:172.21.15.118:6816/12528,v1:172.21.15.118:6817/12528] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9a90067a80 con 0x7f9a7005bbf0 2020-11-28T22:26:47.794 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.779+0000 7f9a8cff9700 1 -- 172.21.15.162:0/476674336 <== osd.7 v2:172.21.15.118:6816/12528 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f9a90067a80 con 0x7f9a7005bbf0 2020-11-28T22:26:47.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.783+0000 7f9a9618a700 1 -- 172.21.15.162:0/476674336 >> [v2:172.21.15.118:6816/12528,v1:172.21.15.118:6817/12528] conn(0x7f9a7005bbf0 msgr2=0x7f9a7005e010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:47.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.783+0000 7f9a9618a700 1 --2- 172.21.15.162:0/476674336 >> [v2:172.21.15.118:6816/12528,v1:172.21.15.118:6817/12528] conn(0x7f9a7005bbf0 0x7f9a7005e010 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:47.795 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.783+0000 7f9a9618a700 1 -- 172.21.15.162:0/476674336 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9a700554e0 msgr2=0x7f9a700579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:47.796 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.783+0000 7f9a9618a700 1 --2- 172.21.15.162:0/476674336 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f9a700554e0 0x7f9a700579a0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7f9a8401a000 tx=0x7f9a84015000).stop 2020-11-28T22:26:47.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.783+0000 7f9a9618a700 1 -- 172.21.15.162:0/476674336 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a90144b10 msgr2=0x7f9a9013a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:47.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.783+0000 7f9a9618a700 1 --2- 172.21.15.162:0/476674336 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a90144b10 0x7f9a9013a400 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 rx=0x7f9a88001930 tx=0x7f9a88001d90).stop 2020-11-28T22:26:47.797 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.783+0000 7f9a8ffff700 1 -- 172.21.15.162:0/476674336 reap_dead start 2020-11-28T22:26:47.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.783+0000 7f9a9618a700 1 -- 172.21.15.162:0/476674336 shutdown_connections 2020-11-28T22:26:47.798 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.783+0000 7f9a9618a700 1 -- 172.21.15.162:0/476674336 >> 172.21.15.162:0/476674336 conn(0x7f9a9005b970 msgr2=0x7f9a90143430 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:26:47.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.783+0000 7f9a9618a700 1 -- 172.21.15.162:0/476674336 shutdown_connections 2020-11-28T22:26:47.799 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.783+0000 7f9a9618a700 1 -- 172.21.15.162:0/476674336 wait complete. 2020-11-28T22:26:47.800 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2020-11-28T22:26:47.800 INFO:teuthology.orchestra.run.smithi162.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2020-11-28T22:26:47.800 INFO:teuthology.orchestra.run.smithi162.stdout:} 2020-11-28T22:26:47.802 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd deep-scrub 7 2020-11-28T22:26:47.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.935+0000 7fd25f508700 1 Processor -- start 2020-11-28T22:26:47.941 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.935+0000 7fd25f508700 1 -- start start 2020-11-28T22:26:47.942 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.939+0000 7fd25f508700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2580aa7d0 0x7fd2580b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:47.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.939+0000 7fd25f508700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd2580b1c20 0x7fd258067930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:47.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.939+0000 7fd25f508700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd258065b00 con 0x7fd2580aabb0 2020-11-28T22:26:47.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.939+0000 7fd25f508700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd25813e7f0 con 0x7fd2580b1c20 2020-11-28T22:26:47.943 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.939+0000 7fd25f508700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd258066270 con 0x7fd2580aa7d0 2020-11-28T22:26:47.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.939+0000 7fd25daa5700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fd2580aabb0 0x7fd258140ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:53870/0 (socket says 172.21.15.162:53870) 2020-11-28T22:26:47.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.939+0000 7fd25daa5700 1 -- 172.21.15.162:0/2772296688 learned_addr learned my addr 172.21.15.162:0/2772296688 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T22:26:47.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.939+0000 7fd25caa3700 1 --2- 172.21.15.162:0/2772296688 >> v2:172.21.15.118:3300/0 conn(0x7fd2580b1c20 0x7fd258067930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:47.944 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.939+0000 7fd25d2a4700 1 --2- 172.21.15.162:0/2772296688 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2580aa7d0 0x7fd2580b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:47.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.939+0000 7fd24ffff700 1 -- 172.21.15.162:0/2772296688 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2990156376 0 0) 0x7fd258065b00 con 0x7fd2580aabb0 2020-11-28T22:26:47.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.939+0000 7fd24ffff700 1 -- 172.21.15.162:0/2772296688 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd23c003230 con 0x7fd2580aabb0 2020-11-28T22:26:47.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.939+0000 7fd25caa3700 1 -- 172.21.15.162:0/2772296688 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2580aa7d0 msgr2=0x7fd2580b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T22:26:47.945 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.939+0000 7fd25caa3700 1 --2- 172.21.15.162:0/2772296688 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2580aa7d0 0x7fd2580b2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:47.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.939+0000 7fd25caa3700 1 -- 172.21.15.162:0/2772296688 >> v1:172.21.15.162:6789/0 conn(0x7fd2580aabb0 legacy=0x7fd258140ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:47.946 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.939+0000 7fd25caa3700 1 -- 172.21.15.162:0/2772296688 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd258067e20 con 0x7fd2580b1c20 2020-11-28T22:26:47.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.939+0000 7fd25caa3700 1 --2- 172.21.15.162:0/2772296688 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2580b1c20 0x7fd258067930 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7fd2580663b0 tx=0x7fd248006d60).ready entity=mon.0 client_cookie=69513f4b04e160f4 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:47.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.939+0000 7fd24ffff700 1 -- 172.21.15.162:0/2772296688 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd248009580 con 0x7fd2580b1c20 2020-11-28T22:26:47.948 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.939+0000 7fd24ffff700 1 -- 172.21.15.162:0/2772296688 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2480096e0 con 0x7fd2580b1c20 2020-11-28T22:26:47.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.939+0000 7fd24ffff700 1 -- 172.21.15.162:0/2772296688 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd248009850 con 0x7fd2580b1c20 2020-11-28T22:26:47.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.939+0000 7fd25f508700 1 -- 172.21.15.162:0/2772296688 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2580b1c20 msgr2=0x7fd258067930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:47.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.939+0000 7fd25f508700 1 --2- 172.21.15.162:0/2772296688 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2580b1c20 0x7fd258067930 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7fd2580663b0 tx=0x7fd248006d60).stop 2020-11-28T22:26:47.949 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.939+0000 7fd25f508700 1 -- 172.21.15.162:0/2772296688 shutdown_connections 2020-11-28T22:26:47.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.939+0000 7fd25f508700 1 --2- 172.21.15.162:0/2772296688 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2580aa7d0 0x7fd2580b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:47.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.939+0000 7fd25f508700 1 --2- 172.21.15.162:0/2772296688 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2580b1c20 0x7fd258067930 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:47.950 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.939+0000 7fd25f508700 1 -- 172.21.15.162:0/2772296688 >> 172.21.15.162:0/2772296688 conn(0x7fd258062b50 msgr2=0x7fd258062f30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:26:47.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.939+0000 7fd25f508700 1 -- 172.21.15.162:0/2772296688 shutdown_connections 2020-11-28T22:26:47.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.939+0000 7fd25f508700 1 -- 172.21.15.162:0/2772296688 wait complete. 2020-11-28T22:26:47.951 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.943+0000 7fd25f508700 1 Processor -- start 2020-11-28T22:26:47.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.943+0000 7fd25f508700 1 -- start start 2020-11-28T22:26:47.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.943+0000 7fd25f508700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2580aa7d0 0x7fd2580abd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:47.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.943+0000 7fd25f508700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2580aabb0 0x7fd2580b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:47.952 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.943+0000 7fd25f508700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2580b1c20 0x7fd2580b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:47.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.943+0000 7fd25f508700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25813f5e0 con 0x7fd2580aabb0 2020-11-28T22:26:47.953 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.943+0000 7fd25f508700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25813f340 con 0x7fd2580b1c20 2020-11-28T22:26:47.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.943+0000 7fd25f508700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2580b0470 con 0x7fd2580aa7d0 2020-11-28T22:26:47.954 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.943+0000 7fd25daa5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2580b1c20 0x7fd2580b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:47.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.943+0000 7fd25d2a4700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2580aa7d0 0x7fd2580abd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:47.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.943+0000 7fd25daa5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2580b1c20 0x7fd2580b10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47032/0 (socket says 172.21.15.162:47032) 2020-11-28T22:26:47.955 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.943+0000 7fd25daa5700 1 -- 172.21.15.162:0/3388705566 learned_addr learned my addr 172.21.15.162:0/3388705566 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T22:26:47.956 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.943+0000 7fd25caa3700 1 --2- 172.21.15.162:0/3388705566 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2580aabb0 0x7fd2580b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:47.957 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.943+0000 7fd25daa5700 1 -- 172.21.15.162:0/3388705566 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2580aa7d0 msgr2=0x7fd2580abd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:47.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.943+0000 7fd25daa5700 1 --2- 172.21.15.162:0/3388705566 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2580aa7d0 0x7fd2580abd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:47.958 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.943+0000 7fd25daa5700 1 -- 172.21.15.162:0/3388705566 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2580aabb0 msgr2=0x7fd2580b0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:47.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.943+0000 7fd25daa5700 1 --2- 172.21.15.162:0/3388705566 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2580aabb0 0x7fd2580b0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:47.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.943+0000 7fd25daa5700 1 -- 172.21.15.162:0/3388705566 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2580b0710 con 0x7fd2580b1c20 2020-11-28T22:26:47.959 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.943+0000 7fd25d2a4700 1 --2- 172.21.15.162:0/3388705566 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2580aa7d0 0x7fd2580abd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-28T22:26:47.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.943+0000 7fd25daa5700 1 --2- 172.21.15.162:0/3388705566 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2580b1c20 0x7fd2580b10a0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 rx=0x7fd250001450 tx=0x7fd250007dd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:47.960 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.947+0000 7fd24e7fc700 1 -- 172.21.15.162:0/3388705566 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd250005b70 con 0x7fd2580b1c20 2020-11-28T22:26:47.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.947+0000 7fd24e7fc700 1 -- 172.21.15.162:0/3388705566 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd250005cd0 con 0x7fd2580b1c20 2020-11-28T22:26:47.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.947+0000 7fd25f508700 1 -- 172.21.15.162:0/3388705566 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd258132da0 con 0x7fd2580b1c20 2020-11-28T22:26:47.961 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.947+0000 7fd24e7fc700 1 -- 172.21.15.162:0/3388705566 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd250007300 con 0x7fd2580b1c20 2020-11-28T22:26:47.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.947+0000 7fd25f508700 1 -- 172.21.15.162:0/3388705566 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2580b0190 con 0x7fd2580b1c20 2020-11-28T22:26:47.962 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.947+0000 7fd25f508700 1 -- 172.21.15.162:0/3388705566 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2580b08a0 con 0x7fd2580b1c20 2020-11-28T22:26:47.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.947+0000 7fd24e7fc700 1 -- 172.21.15.162:0/3388705566 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd250010030 con 0x7fd2580b1c20 2020-11-28T22:26:47.963 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.947+0000 7fd24e7fc700 1 --2- 172.21.15.162:0/3388705566 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd240055440 0x7fd240057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:47.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.947+0000 7fd24e7fc700 1 -- 172.21.15.162:0/3388705566 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5339+0+0 (secure 0 0 0) 0x7fd25006dda0 con 0x7fd2580b1c20 2020-11-28T22:26:47.964 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.947+0000 7fd25d2a4700 1 --2- 172.21.15.162:0/3388705566 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd240055440 0x7fd240057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:47.965 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.951+0000 7fd25d2a4700 1 --2- 172.21.15.162:0/3388705566 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd240055440 0x7fd240057900 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7fd25400b000 tx=0x7fd254006dc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:47.966 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:47.955+0000 7fd24e7fc700 1 -- 172.21.15.162:0/3388705566 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd250044270 con 0x7fd2580b1c20 2020-11-28T22:26:48.100 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.095+0000 7fd25f508700 1 -- 172.21.15.162:0/3388705566 --> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fd258057050 con 0x7fd240055440 2020-11-28T22:26:48.101 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.099+0000 7fd24e7fc700 1 -- 172.21.15.162:0/3388705566 <== mgr.4105 v2:172.21.15.162:6800/12156 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd258057050 con 0x7fd240055440 2020-11-28T22:26:48.103 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.099+0000 7fd25f508700 1 -- 172.21.15.162:0/3388705566 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd240055440 msgr2=0x7fd240057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:48.104 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.099+0000 7fd25f508700 1 --2- 172.21.15.162:0/3388705566 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd240055440 0x7fd240057900 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7fd25400b000 tx=0x7fd254006dc0).stop 2020-11-28T22:26:48.104 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.099+0000 7fd25f508700 1 -- 172.21.15.162:0/3388705566 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2580b1c20 msgr2=0x7fd2580b10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:48.104 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.099+0000 7fd25f508700 1 --2- 172.21.15.162:0/3388705566 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2580b1c20 0x7fd2580b10a0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 rx=0x7fd250001450 tx=0x7fd250007dd0).stop 2020-11-28T22:26:48.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.099+0000 7fd25f508700 1 -- 172.21.15.162:0/3388705566 shutdown_connections 2020-11-28T22:26:48.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.099+0000 7fd25f508700 1 --2- 172.21.15.162:0/3388705566 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd240055440 0x7fd240057900 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:48.105 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.099+0000 7fd25f508700 1 --2- 172.21.15.162:0/3388705566 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd2580aa7d0 0x7fd2580abd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:48.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.099+0000 7fd25f508700 1 --2- 172.21.15.162:0/3388705566 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2580b1c20 0x7fd2580b10a0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:48.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.099+0000 7fd25f508700 1 --2- 172.21.15.162:0/3388705566 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2580aabb0 0x7fd2580b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:48.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.099+0000 7fd25f508700 1 -- 172.21.15.162:0/3388705566 >> 172.21.15.162:0/3388705566 conn(0x7fd258062b50 msgr2=0x7fd2580b30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:26:48.106 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.099+0000 7fd25f508700 1 -- 172.21.15.162:0/3388705566 shutdown_connections 2020-11-28T22:26:48.107 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.099+0000 7fd25f508700 1 -- 172.21.15.162:0/3388705566 wait complete. 2020-11-28T22:26:48.107 INFO:teuthology.orchestra.run.smithi162.stderr:instructed osd(s) 7 to deep-scrub 2020-11-28T22:26:48.115 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-11-28T22:26:48.143 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:26:48.159 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:26:48.168 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:26:48.178 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:26:48.228 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.223+0000 7f3f6a3dc700 1 Processor -- start 2020-11-28T22:26:48.229 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.223+0000 7f3f6a3dc700 1 -- start start 2020-11-28T22:26:48.230 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.227+0000 7f3f6a3dc700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f3f64147a90 0x7f3f640b2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:48.231 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.227+0000 7f3f6a3dc700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3f6409ca50 0x7f3f640b2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:48.231 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.227+0000 7f3f6a3dc700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f640654e0 con 0x7f3f640b7780 2020-11-28T22:26:48.231 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.227+0000 7f3f6a3dc700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f64065780 con 0x7f3f64147a90 2020-11-28T22:26:48.231 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.227+0000 7f3f6a3dc700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f64065a20 con 0x7f3f6409ca50 2020-11-28T22:26:48.232 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.227+0000 7f3f693da700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f3f640b7780 0x7f3f640b1870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:53880/0 (socket says 172.21.15.162:53880) 2020-11-28T22:26:48.232 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.227+0000 7f3f693da700 1 -- 172.21.15.162:0/158020113 learned_addr learned my addr 172.21.15.162:0/158020113 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T22:26:48.232 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.227+0000 7f3f69bdb700 1 --2- 172.21.15.162:0/158020113 >> v2:172.21.15.162:3300/0 conn(0x7f3f6409ca50 0x7f3f640b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:48.233 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.227+0000 7f3f68bd9700 1 --2- 172.21.15.162:0/158020113 >> v2:172.21.15.118:3300/0 conn(0x7f3f64147a90 0x7f3f640b2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:48.233 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.227+0000 7f3f63fff700 1 -- 172.21.15.162:0/158020113 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2540524736 0 0) 0x7f3f640654e0 con 0x7f3f640b7780 2020-11-28T22:26:48.233 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.227+0000 7f3f63fff700 1 -- 172.21.15.162:0/158020113 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f50003230 con 0x7f3f640b7780 2020-11-28T22:26:48.233 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.227+0000 7f3f69bdb700 1 -- 172.21.15.162:0/158020113 >> v2:172.21.15.118:3300/0 conn(0x7f3f64147a90 msgr2=0x7f3f640b2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T22:26:48.234 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.227+0000 7f3f69bdb700 1 --2- 172.21.15.162:0/158020113 >> v2:172.21.15.118:3300/0 conn(0x7f3f64147a90 0x7f3f640b2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:48.234 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.227+0000 7f3f69bdb700 1 -- 172.21.15.162:0/158020113 >> v1:172.21.15.162:6789/0 conn(0x7f3f640b7780 legacy=0x7f3f640b1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:48.234 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.227+0000 7f3f69bdb700 1 -- 172.21.15.162:0/158020113 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f64065cc0 con 0x7f3f6409ca50 2020-11-28T22:26:48.235 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.227+0000 7f3f68bd9700 1 --2- 172.21.15.162:0/158020113 >> v2:172.21.15.118:3300/0 conn(0x7f3f64147a90 0x7f3f640b2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-28T22:26:48.235 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.227+0000 7f3f69bdb700 1 --2- 172.21.15.162:0/158020113 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f6409ca50 0x7f3f640b2880 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 rx=0x7f3f5c002f30 tx=0x7f3f5c006dc0).ready entity=mon.1 client_cookie=20ee6ff98839afb3 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:48.235 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.227+0000 7f3f63fff700 1 -- 172.21.15.162:0/158020113 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f5c0097b0 con 0x7f3f6409ca50 2020-11-28T22:26:48.235 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.227+0000 7f3f63fff700 1 -- 172.21.15.162:0/158020113 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f5c009910 con 0x7f3f6409ca50 2020-11-28T22:26:48.236 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.227+0000 7f3f63fff700 1 -- 172.21.15.162:0/158020113 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f5c009c70 con 0x7f3f6409ca50 2020-11-28T22:26:48.236 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.227+0000 7f3f6a3dc700 1 -- 172.21.15.162:0/158020113 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f6409ca50 msgr2=0x7f3f640b2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:48.236 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.227+0000 7f3f6a3dc700 1 --2- 172.21.15.162:0/158020113 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f6409ca50 0x7f3f640b2880 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 rx=0x7f3f5c002f30 tx=0x7f3f5c006dc0).stop 2020-11-28T22:26:48.239 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.227+0000 7f3f6a3dc700 1 -- 172.21.15.162:0/158020113 shutdown_connections 2020-11-28T22:26:48.239 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.227+0000 7f3f6a3dc700 1 --2- 172.21.15.162:0/158020113 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f6409ca50 0x7f3f640b2880 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:48.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.227+0000 7f3f6a3dc700 1 --2- 172.21.15.162:0/158020113 >> v2:172.21.15.118:3300/0 conn(0x7f3f64147a90 0x7f3f640b2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:48.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.227+0000 7f3f6a3dc700 1 -- 172.21.15.162:0/158020113 >> 172.21.15.162:0/158020113 conn(0x7f3f6405c440 msgr2=0x7f3f640b6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:26:48.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.227+0000 7f3f6a3dc700 1 -- 172.21.15.162:0/158020113 shutdown_connections 2020-11-28T22:26:48.240 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.227+0000 7f3f6a3dc700 1 -- 172.21.15.162:0/158020113 wait complete. 2020-11-28T22:26:48.241 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.227+0000 7f3f6a3dc700 1 Processor -- start 2020-11-28T22:26:48.241 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.227+0000 7f3f6a3dc700 1 -- start start 2020-11-28T22:26:48.241 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.231+0000 7f3f6a3dc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3f6409ca50 0x7f3f6414de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:48.241 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.231+0000 7f3f6a3dc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f640b7780 0x7f3f641595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:48.241 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.231+0000 7f3f6a3dc700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3f64147a90 0x7f3f64159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:48.242 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.231+0000 7f3f6a3dc700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f640913c0 con 0x7f3f6409ca50 2020-11-28T22:26:48.242 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.231+0000 7f3f6a3dc700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f64091660 con 0x7f3f640b7780 2020-11-28T22:26:48.242 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.231+0000 7f3f6a3dc700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f64091900 con 0x7f3f64147a90 2020-11-28T22:26:48.242 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.231+0000 7f3f68bd9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f640b7780 0x7f3f641595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:48.243 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.231+0000 7f3f68bd9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f640b7780 0x7f3f641595b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47044/0 (socket says 172.21.15.162:47044) 2020-11-28T22:26:48.243 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.231+0000 7f3f68bd9700 1 -- 172.21.15.162:0/3427840592 learned_addr learned my addr 172.21.15.162:0/3427840592 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T22:26:48.243 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.231+0000 7f3f69bdb700 1 --2- 172.21.15.162:0/3427840592 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3f64147a90 0x7f3f64159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:48.243 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.231+0000 7f3f693da700 1 --2- 172.21.15.162:0/3427840592 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3f6409ca50 0x7f3f6414de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:48.243 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.231+0000 7f3f68bd9700 1 -- 172.21.15.162:0/3427840592 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3f64147a90 msgr2=0x7f3f64159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:48.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.231+0000 7f3f68bd9700 1 --2- 172.21.15.162:0/3427840592 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3f64147a90 0x7f3f64159aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:48.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.231+0000 7f3f68bd9700 1 -- 172.21.15.162:0/3427840592 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3f6409ca50 msgr2=0x7f3f6414de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:48.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.231+0000 7f3f68bd9700 1 --2- 172.21.15.162:0/3427840592 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3f6409ca50 0x7f3f6414de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:48.244 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.231+0000 7f3f68bd9700 1 -- 172.21.15.162:0/3427840592 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f64159110 con 0x7f3f640b7780 2020-11-28T22:26:48.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.231+0000 7f3f693da700 1 --2- 172.21.15.162:0/3427840592 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3f6409ca50 0x7f3f6414de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-28T22:26:48.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.231+0000 7f3f68bd9700 1 --2- 172.21.15.162:0/3427840592 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f640b7780 0x7f3f641595b0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 rx=0x7f3f58006dd0 tx=0x7f3f58008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:48.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.231+0000 7f3f627fc700 1 -- 172.21.15.162:0/3427840592 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f5800bb20 con 0x7f3f640b7780 2020-11-28T22:26:48.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.231+0000 7f3f627fc700 1 -- 172.21.15.162:0/3427840592 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f5800bc80 con 0x7f3f640b7780 2020-11-28T22:26:48.245 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.231+0000 7f3f6a3dc700 1 -- 172.21.15.162:0/3427840592 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f64145990 con 0x7f3f640b7780 2020-11-28T22:26:48.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.231+0000 7f3f627fc700 1 -- 172.21.15.162:0/3427840592 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f58012420 con 0x7f3f640b7780 2020-11-28T22:26:48.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.231+0000 7f3f6a3dc700 1 -- 172.21.15.162:0/3427840592 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f64155d40 con 0x7f3f640b7780 2020-11-28T22:26:48.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.231+0000 7f3f6a3dc700 1 -- 172.21.15.162:0/3427840592 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f641592a0 con 0x7f3f640b7780 2020-11-28T22:26:48.246 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.231+0000 7f3f627fc700 1 -- 172.21.15.162:0/3427840592 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f3f58012580 con 0x7f3f640b7780 2020-11-28T22:26:48.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.235+0000 7f3f627fc700 1 --2- 172.21.15.162:0/3427840592 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3f440553b0 0x7f3f44057870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:26:48.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.235+0000 7f3f627fc700 1 -- 172.21.15.162:0/3427840592 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f3f5806e4a0 con 0x7f3f640b7780 2020-11-28T22:26:48.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.235+0000 7f3f693da700 1 --2- 172.21.15.162:0/3427840592 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3f440553b0 0x7f3f44057870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:26:48.247 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.235+0000 7f3f693da700 1 --2- 172.21.15.162:0/3427840592 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3f440553b0 0x7f3f44057870 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7f3f54008000 tx=0x7f3f54009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:26:48.248 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.239+0000 7f3f627fc700 1 -- 172.21.15.162:0/3427840592 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f3f58044a20 con 0x7f3f640b7780 2020-11-28T22:26:48.381 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.379+0000 7f3f6a3dc700 1 -- 172.21.15.162:0/3427840592 --> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3f64057050 con 0x7f3f440553b0 2020-11-28T22:26:48.384 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.379+0000 7f3f627fc700 1 -- 172.21.15.162:0/3427840592 <== mgr.4105 v2:172.21.15.162:6800/12156 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+66352 (secure 0 0 0) 0x7f3f64057050 con 0x7f3f440553b0 2020-11-28T22:26:48.385 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T22:26:48.387 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.383+0000 7f3f6a3dc700 1 -- 172.21.15.162:0/3427840592 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3f440553b0 msgr2=0x7f3f44057870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:48.388 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.383+0000 7f3f6a3dc700 1 --2- 172.21.15.162:0/3427840592 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3f440553b0 0x7f3f44057870 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7f3f54008000 tx=0x7f3f54009040).stop 2020-11-28T22:26:48.388 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.383+0000 7f3f6a3dc700 1 -- 172.21.15.162:0/3427840592 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f640b7780 msgr2=0x7f3f641595b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:26:48.388 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.383+0000 7f3f6a3dc700 1 --2- 172.21.15.162:0/3427840592 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f640b7780 0x7f3f641595b0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 rx=0x7f3f58006dd0 tx=0x7f3f58008040).stop 2020-11-28T22:26:48.389 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.383+0000 7f3f6a3dc700 1 -- 172.21.15.162:0/3427840592 shutdown_connections 2020-11-28T22:26:48.389 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.383+0000 7f3f6a3dc700 1 --2- 172.21.15.162:0/3427840592 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f3f440553b0 0x7f3f44057870 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:48.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.383+0000 7f3f6a3dc700 1 --2- 172.21.15.162:0/3427840592 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3f64147a90 0x7f3f64159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:48.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.383+0000 7f3f6a3dc700 1 --2- 172.21.15.162:0/3427840592 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3f6409ca50 0x7f3f6414de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:48.390 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.383+0000 7f3f6a3dc700 1 --2- 172.21.15.162:0/3427840592 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f640b7780 0x7f3f641595b0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:26:48.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.383+0000 7f3f6a3dc700 1 -- 172.21.15.162:0/3427840592 >> 172.21.15.162:0/3427840592 conn(0x7f3f6405c440 msgr2=0x7f3f640669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:26:48.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.383+0000 7f3f6a3dc700 1 -- 172.21.15.162:0/3427840592 shutdown_connections 2020-11-28T22:26:48.391 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:26:48.383+0000 7f3f6a3dc700 1 -- 172.21.15.162:0/3427840592 wait complete. 2020-11-28T22:26:48.392 INFO:teuthology.orchestra.run.smithi162.stderr:dumped all 2020-11-28T22:26:48.398 INFO:teuthology.orchestra.run.smithi162.stdout:{"pg_ready":true,"pg_map":{"version":1370,"stamp":"2020-11-28T22:26:48.291608+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8404860,"kb_used_data":16188,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746569860,"statfs":{"total":773094113280,"available":764487536640,"internally_reserved":8589934592,"allocated":16576512,"data_stored":7671573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003565"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":"130","reported_epoch":"162","state":"active+clean","last_fresh":"2020-11-28T21:56:41.031775+0000","last_change":"2020-11-28T21:46:25.652727+0000","last_active":"2020-11-28T21:56:41.031775+0000","last_peered":"2020-11-28T21:56:41.031775+0000","last_clean":"2020-11-28T21:56:41.031775+0000","last_became_active":"2020-11-28T21:46:25.652265+0000","last_became_peered":"2020-11-28T21:46:25.652265+0000","last_unstale":"2020-11-28T21:56:41.031775+0000","last_undegraded":"2020-11-28T21:56:41.031775+0000","last_fullsized":"2020-11-28T21:56:41.031775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_clean_scrub_stamp":"2020-11-28T21:46:24.622452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"133","reported_epoch":"162","state":"active+clean","last_fresh":"2020-11-28T21:56:40.913441+0000","last_change":"2020-11-28T21:46:25.654101+0000","last_active":"2020-11-28T21:56:40.913441+0000","last_peered":"2020-11-28T21:56:40.913441+0000","last_clean":"2020-11-28T21:56:40.913441+0000","last_became_active":"2020-11-28T21:46:25.653417+0000","last_became_peered":"2020-11-28T21:46:25.653417+0000","last_unstale":"2020-11-28T21:56:40.913441+0000","last_undegraded":"2020-11-28T21:56:40.913441+0000","last_fullsized":"2020-11-28T21:56:40.913441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_clean_scrub_stamp":"2020-11-28T21:46:24.622452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"135","reported_epoch":"162","state":"active+clean","last_fresh":"2020-11-28T21:56:41.322109+0000","last_change":"2020-11-28T21:46:25.652888+0000","last_active":"2020-11-28T21:56:41.322109+0000","last_peered":"2020-11-28T21:56:41.322109+0000","last_clean":"2020-11-28T21:56:41.322109+0000","last_became_active":"2020-11-28T21:46:25.652295+0000","last_became_peered":"2020-11-28T21:46:25.652295+0000","last_unstale":"2020-11-28T21:56:41.322109+0000","last_undegraded":"2020-11-28T21:56:41.322109+0000","last_fullsized":"2020-11-28T21:56:41.322109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_clean_scrub_stamp":"2020-11-28T21:46:24.622452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"133","reported_epoch":"162","state":"active+clean","last_fresh":"2020-11-28T21:56:40.913442+0000","last_change":"2020-11-28T21:46:26.008971+0000","last_active":"2020-11-28T21:56:40.913442+0000","last_peered":"2020-11-28T21:56:40.913442+0000","last_clean":"2020-11-28T21:56:40.913442+0000","last_became_active":"2020-11-28T21:46:26.007744+0000","last_became_peered":"2020-11-28T21:46:26.007744+0000","last_unstale":"2020-11-28T21:56:40.913442+0000","last_undegraded":"2020-11-28T21:56:40.913442+0000","last_fullsized":"2020-11-28T21:56:40.913442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_clean_scrub_stamp":"2020-11-28T21:46:24.622452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"136","reported_epoch":"162","state":"active+clean","last_fresh":"2020-11-28T21:56:41.096619+0000","last_change":"2020-11-28T21:46:20.941819+0000","last_active":"2020-11-28T21:56:41.096619+0000","last_peered":"2020-11-28T21:56:41.096619+0000","last_clean":"2020-11-28T21:56:41.096619+0000","last_became_active":"2020-11-28T21:46:20.941467+0000","last_became_peered":"2020-11-28T21:46:20.941467+0000","last_unstale":"2020-11-28T21:56:41.096619+0000","last_undegraded":"2020-11-28T21:56:41.096619+0000","last_fullsized":"2020-11-28T21:56:41.096619+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:16.570066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:16.570066+0000","last_clean_scrub_stamp":"2020-11-28T21:46:16.570066+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"133","reported_epoch":"162","state":"active+clean","last_fresh":"2020-11-28T21:56:40.913557+0000","last_change":"2020-11-28T21:46:25.654099+0000","last_active":"2020-11-28T21:56:40.913557+0000","last_peered":"2020-11-28T21:56:40.913557+0000","last_clean":"2020-11-28T21:56:40.913557+0000","last_became_active":"2020-11-28T21:46:25.653652+0000","last_became_peered":"2020-11-28T21:46:25.653652+0000","last_unstale":"2020-11-28T21:56:40.913557+0000","last_undegraded":"2020-11-28T21:56:40.913557+0000","last_fullsized":"2020-11-28T21:56:40.913557+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_clean_scrub_stamp":"2020-11-28T21:46:24.622452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"133","reported_epoch":"162","state":"active+clean","last_fresh":"2020-11-28T21:56:40.914051+0000","last_change":"2020-11-28T21:46:25.653672+0000","last_active":"2020-11-28T21:56:40.914051+0000","last_peered":"2020-11-28T21:56:40.914051+0000","last_clean":"2020-11-28T21:56:40.914051+0000","last_became_active":"2020-11-28T21:46:25.653222+0000","last_became_peered":"2020-11-28T21:46:25.653222+0000","last_unstale":"2020-11-28T21:56:40.914051+0000","last_undegraded":"2020-11-28T21:56:40.914051+0000","last_fullsized":"2020-11-28T21:56:40.914051+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_clean_scrub_stamp":"2020-11-28T21:46:24.622452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"140","reported_epoch":"162","state":"active+clean","last_fresh":"2020-11-28T21:56:40.914032+0000","last_change":"2020-11-28T21:46:25.838546+0000","last_active":"2020-11-28T21:56:40.914032+0000","last_peered":"2020-11-28T21:56:40.914032+0000","last_clean":"2020-11-28T21:56:40.914032+0000","last_became_active":"2020-11-28T21:46:25.837556+0000","last_became_peered":"2020-11-28T21:46:25.837556+0000","last_unstale":"2020-11-28T21:56:40.914032+0000","last_undegraded":"2020-11-28T21:56:40.914032+0000","last_fullsized":"2020-11-28T21:56:40.914032+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_clean_scrub_stamp":"2020-11-28T21:46:24.622452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"133","reported_epoch":"162","state":"active+clean","last_fresh":"2020-11-28T21:56:40.913881+0000","last_change":"2020-11-28T21:46:25.838687+0000","last_active":"2020-11-28T21:56:40.913881+0000","last_peered":"2020-11-28T21:56:40.913881+0000","last_clean":"2020-11-28T21:56:40.913881+0000","last_became_active":"2020-11-28T21:46:25.837943+0000","last_became_peered":"2020-11-28T21:46:25.837943+0000","last_unstale":"2020-11-28T21:56:40.913881+0000","last_undegraded":"2020-11-28T21:56:40.913881+0000","last_fullsized":"2020-11-28T21:56:40.913881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T21:46:24.622452+0000","last_clean_scrub_stamp":"2020-11-28T21:46:24.622452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542631,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050608,"kb_used_data":2024,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321232,"statfs":{"total":96636764160,"available":95560941568,"internally_reserved":1073741824,"allocated":2072576,"data_stored":959022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 28 22:26:34 2020","interfaces":[{"interface":"back","average":{"1min":1.847,"5min":1.6910000000000001,"15min":1.716},"min":{"1min":1.2729999999999999,"5min":1.081,"15min":0.98699999999999999},"max":{"1min":2.9769999999999999,"5min":3.609,"15min":3.609},"last":2.012},{"interface":"front","average":{"1min":1.7849999999999999,"5min":1.8400000000000001,"15min":1.8080000000000001},"min":{"1min":1.429,"5min":1.2470000000000001,"15min":1.1930000000000001},"max":{"1min":2.52,"5min":3.6589999999999998,"15min":3.6589999999999998},"last":1.8560000000000001}]},{"osd":1,"last update":"Sat Nov 28 22:26:34 2020","interfaces":[{"interface":"back","average":{"1min":1.661,"5min":1.8240000000000001,"15min":1.8320000000000001},"min":{"1min":1.298,"5min":1.2490000000000001,"15min":1.0840000000000001},"max":{"1min":2.8759999999999999,"5min":3.6139999999999999,"15min":3.6139999999999999},"last":2.3239999999999998},{"interface":"front","average":{"1min":2.2010000000000001,"5min":2.1909999999999998,"15min":2.206},"min":{"1min":1.601,"5min":1.3700000000000001,"15min":1.1779999999999999},"max":{"1min":3.141,"5min":3.4249999999999998,"15min":3.4550000000000001},"last":2.2069999999999999}]},{"osd":2,"last update":"Sat Nov 28 22:26:34 2020","interfaces":[{"interface":"back","average":{"1min":2.2650000000000001,"5min":2.2629999999999999,"15min":2.194},"min":{"1min":1.373,"5min":1.373,"15min":1.3220000000000001},"max":{"1min":3.2690000000000001,"5min":4.0439999999999996,"15min":4.0439999999999996},"last":2.4550000000000001},{"interface":"front","average":{"1min":2.2389999999999999,"5min":2.1720000000000002,"15min":2.1429999999999998},"min":{"1min":1.7609999999999999,"5min":1.429,"15min":1.3109999999999999},"max":{"1min":3.012,"5min":3.5289999999999999,"15min":3.5289999999999999},"last":3.1120000000000001}]},{"osd":3,"last update":"Sat Nov 28 22:26:34 2020","interfaces":[{"interface":"back","average":{"1min":2.4769999999999999,"5min":2.46,"15min":2.4399999999999999},"min":{"1min":1.778,"5min":1.6200000000000001,"15min":1.3160000000000001},"max":{"1min":3.6859999999999999,"5min":4.3819999999999997,"15min":4.3819999999999997},"last":2.8370000000000002},{"interface":"front","average":{"1min":2.3290000000000002,"5min":2.4550000000000001,"15min":2.4169999999999998},"min":{"1min":1.7609999999999999,"5min":1.603,"15min":1.4770000000000001},"max":{"1min":2.996,"5min":3.867,"15min":3.867},"last":2.7469999999999999}]},{"osd":4,"last update":"Sat Nov 28 22:26:34 2020","interfaces":[{"interface":"back","average":{"1min":2.5339999999999998,"5min":2.5459999999999998,"15min":2.5129999999999999},"min":{"1min":1.8160000000000001,"5min":1.452,"15min":1.3009999999999999},"max":{"1min":3.5699999999999998,"5min":3.806,"15min":3.806},"last":2.4910000000000001},{"interface":"front","average":{"1min":2.7149999999999999,"5min":2.7789999999999999,"15min":2.7530000000000001},"min":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.577},"max":{"1min":3.9849999999999999,"5min":4.4260000000000002,"15min":4.4260000000000002},"last":2.649}]},{"osd":5,"last update":"Sat Nov 28 22:26:34 2020","interfaces":[{"interface":"back","average":{"1min":2.6099999999999999,"5min":2.7149999999999999,"15min":2.6989999999999998},"min":{"1min":1.8600000000000001,"5min":1.835,"15min":1.659},"max":{"1min":3.359,"5min":4.2850000000000001,"15min":4.2850000000000001},"last":3.5129999999999999},{"interface":"front","average":{"1min":2.887,"5min":2.9279999999999999,"15min":2.8969999999999998},"min":{"1min":2.1800000000000002,"5min":2.0019999999999998,"15min":1.345},"max":{"1min":3.7629999999999999,"5min":4.1779999999999999,"15min":4.1779999999999999},"last":2.6949999999999998}]},{"osd":6,"last update":"Sat Nov 28 22:26:34 2020","interfaces":[{"interface":"back","average":{"1min":3.016,"5min":3.0390000000000001,"15min":3.085},"min":{"1min":2.4780000000000002,"5min":1.976,"15min":1.976},"max":{"1min":3.9510000000000001,"5min":4.5629999999999997,"15min":4.5629999999999997},"last":2.9020000000000001},{"interface":"front","average":{"1min":3.1829999999999998,"5min":3.137,"15min":3.1520000000000001},"min":{"1min":2.5950000000000002,"5min":1.804,"15min":1.804},"max":{"1min":4.194,"5min":4.5010000000000003,"15min":4.5010000000000003},"last":3.3450000000000002}]}]},{"osd":6,"up_from":14,"seq":60129542631,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050608,"kb_used_data":2024,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321232,"statfs":{"total":96636764160,"available":95560941568,"internally_reserved":1073741824,"allocated":2072576,"data_stored":959022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 28 22:26:30 2020","interfaces":[{"interface":"back","average":{"1min":1.6479999999999999,"5min":1.591,"15min":1.645},"min":{"1min":1.3240000000000001,"5min":1.1919999999999999,"15min":1.1060000000000001},"max":{"1min":2.0449999999999999,"5min":2.8069999999999999,"15min":2.8069999999999999},"last":1.171},{"interface":"front","average":{"1min":1.7749999999999999,"5min":1.768,"15min":1.7829999999999999},"min":{"1min":1.403,"5min":1.087,"15min":1.087},"max":{"1min":3.3010000000000002,"5min":3.536,"15min":3.9060000000000001},"last":1.397}]},{"osd":1,"last update":"Sat Nov 28 22:26:30 2020","interfaces":[{"interface":"back","average":{"1min":2.2480000000000002,"5min":2.2109999999999999,"15min":2.2410000000000001},"min":{"1min":1.5249999999999999,"5min":1.264,"15min":1.264},"max":{"1min":3.3220000000000001,"5min":4.4989999999999997,"15min":4.4989999999999997},"last":2.262},{"interface":"front","average":{"1min":2.3610000000000002,"5min":2.3439999999999999,"15min":2.395},"min":{"1min":1.3979999999999999,"5min":0.996,"15min":0.996},"max":{"1min":2.754,"5min":3.1360000000000001,"15min":3.2789999999999999},"last":2.02}]},{"osd":2,"last update":"Sat Nov 28 22:26:30 2020","interfaces":[{"interface":"back","average":{"1min":2.024,"5min":2.1240000000000001,"15min":2.1339999999999999},"min":{"1min":1.5609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"max":{"1min":3.1789999999999998,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":1.944},{"interface":"front","average":{"1min":2.512,"5min":2.3199999999999998,"15min":2.355},"min":{"1min":1.694,"5min":1.355,"15min":1.103},"max":{"1min":3.899,"5min":4.3680000000000003,"15min":4.3680000000000003},"last":2.1829999999999998}]},{"osd":3,"last update":"Sat Nov 28 22:26:30 2020","interfaces":[{"interface":"back","average":{"1min":2.5529999999999999,"5min":2.573,"15min":2.6040000000000001},"min":{"1min":1.8360000000000001,"5min":1.6100000000000001,"15min":1.5109999999999999},"max":{"1min":3.5059999999999998,"5min":4.7329999999999997,"15min":4.7329999999999997},"last":1.8180000000000001},{"interface":"front","average":{"1min":2.4769999999999999,"5min":2.4489999999999998,"15min":2.4510000000000001},"min":{"1min":1.9299999999999999,"5min":1.665,"15min":1.3720000000000001},"max":{"1min":3.46,"5min":4.2290000000000001,"15min":4.2290000000000001},"last":1.554}]},{"osd":4,"last update":"Sat Nov 28 22:26:30 2020","interfaces":[{"interface":"back","average":{"1min":2.5009999999999999,"5min":2.3980000000000001,"15min":2.4199999999999999},"min":{"1min":2.0219999999999998,"5min":1.6759999999999999,"15min":1.121},"max":{"1min":3.3559999999999999,"5min":3.734,"15min":3.734},"last":1.6759999999999999},{"interface":"front","average":{"1min":2.6219999999999999,"5min":2.8260000000000001,"15min":2.831},"min":{"1min":1.752,"5min":1.218,"15min":1.218},"max":{"1min":4.1079999999999997,"5min":5.5869999999999997,"15min":5.5869999999999997},"last":2.4900000000000002}]},{"osd":5,"last update":"Sat Nov 28 22:26:30 2020","interfaces":[{"interface":"back","average":{"1min":2.9289999999999998,"5min":2.847,"15min":2.8370000000000002},"min":{"1min":2.2989999999999999,"5min":1.758,"15min":1.6839999999999999},"max":{"1min":3.7000000000000002,"5min":5.2210000000000001,"15min":5.2210000000000001},"last":2.7770000000000001},{"interface":"front","average":{"1min":2.633,"5min":2.7629999999999999,"15min":2.8260000000000001},"min":{"1min":1.76,"5min":1.76,"15min":1.47},"max":{"1min":3.597,"5min":4.6760000000000002,"15min":5.3120000000000003},"last":2.5449999999999999}]},{"osd":7,"last update":"Sat Nov 28 22:26:30 2020","interfaces":[{"interface":"back","average":{"1min":3.0070000000000001,"5min":3.141,"15min":3.173},"min":{"1min":2.0859999999999999,"5min":1.8460000000000001,"15min":1.7729999999999999},"max":{"1min":4.3300000000000001,"5min":5.298,"15min":5.3620000000000001},"last":2.6970000000000001},{"interface":"front","average":{"1min":3.0640000000000001,"5min":3.1480000000000001,"15min":3.1560000000000001},"min":{"1min":2.5950000000000002,"5min":2.1960000000000002,"15min":1.6930000000000001},"max":{"1min":3.8700000000000001,"5min":5.4740000000000002,"15min":5.4740000000000002},"last":2.5819999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542632,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050608,"kb_used_data":2024,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321232,"statfs":{"total":96636764160,"available":95560941568,"internally_reserved":1073741824,"allocated":2072576,"data_stored":959022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 28 22:26:42 2020","interfaces":[{"interface":"back","average":{"1min":1.51,"5min":1.589,"15min":1.5940000000000001},"min":{"1min":1.22,"5min":1.22,"15min":1.22},"max":{"1min":1.8740000000000001,"5min":2.3570000000000002,"15min":2.802},"last":1.9079999999999999},{"interface":"front","average":{"1min":1.724,"5min":1.8149999999999999,"15min":1.768},"min":{"1min":1.3700000000000001,"5min":1.3340000000000001,"15min":1.127},"max":{"1min":1.9870000000000001,"5min":3.8740000000000001,"15min":3.8740000000000001},"last":1.851}]},{"osd":2,"last update":"Sat Nov 28 22:26:42 2020","interfaces":[{"interface":"back","average":{"1min":2.0310000000000001,"5min":1.883,"15min":1.837},"min":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.0620000000000001},"max":{"1min":3.2629999999999999,"5min":4.1929999999999996,"15min":4.1929999999999996},"last":1.9710000000000001},{"interface":"front","average":{"1min":2.4809999999999999,"5min":2.3690000000000002,"15min":2.347},"min":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.3169999999999999},"max":{"1min":3.0089999999999999,"5min":3.2799999999999998,"15min":3.3050000000000002},"last":2.5049999999999999}]},{"osd":3,"last update":"Sat Nov 28 22:26:42 2020","interfaces":[{"interface":"back","average":{"1min":2.0870000000000002,"5min":2.1459999999999999,"15min":2.1070000000000002},"min":{"1min":1.74,"5min":1.575,"15min":1.2210000000000001},"max":{"1min":2.75,"5min":3.2160000000000002,"15min":3.2160000000000002},"last":2.972},{"interface":"front","average":{"1min":2.391,"5min":2.4460000000000002,"15min":2.4129999999999998},"min":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5189999999999999},"max":{"1min":3.3010000000000002,"5min":3.5680000000000001,"15min":3.5680000000000001},"last":2.27}]},{"osd":4,"last update":"Sat Nov 28 22:26:42 2020","interfaces":[{"interface":"back","average":{"1min":2.383,"5min":2.4020000000000001,"15min":2.3860000000000001},"min":{"1min":1.528,"5min":1.528,"15min":1.528},"max":{"1min":2.7719999999999998,"5min":3.6680000000000001,"15min":3.73},"last":2.4550000000000001},{"interface":"front","average":{"1min":2.3730000000000002,"5min":2.347,"15min":2.3730000000000002},"min":{"1min":1.7589999999999999,"5min":1.554,"15min":1.554},"max":{"1min":3.379,"5min":3.802,"15min":3.802},"last":2.8599999999999999}]},{"osd":5,"last update":"Sat Nov 28 22:26:42 2020","interfaces":[{"interface":"back","average":{"1min":2.6549999999999998,"5min":2.746,"15min":2.706},"min":{"1min":1.964,"5min":1.964,"15min":1.573},"max":{"1min":3.3980000000000001,"5min":4.0279999999999996,"15min":4.0279999999999996},"last":3.0510000000000002},{"interface":"front","average":{"1min":2.645,"5min":2.698,"15min":2.7389999999999999},"min":{"1min":2.077,"5min":1.95,"15min":1.635},"max":{"1min":3.4620000000000002,"5min":3.891,"15min":4.1980000000000004},"last":3.7160000000000002}]},{"osd":6,"last update":"Sat Nov 28 22:26:42 2020","interfaces":[{"interface":"back","average":{"1min":2.9700000000000002,"5min":3.0009999999999999,"15min":2.9870000000000001},"min":{"1min":2.3650000000000002,"5min":1.9570000000000001,"15min":1.9570000000000001},"max":{"1min":3.8439999999999999,"5min":4.5119999999999996,"15min":4.5119999999999996},"last":3.4180000000000001},{"interface":"front","average":{"1min":2.7519999999999998,"5min":2.8460000000000001,"15min":2.831},"min":{"1min":2.294,"5min":2.294,"15min":2.0590000000000002},"max":{"1min":3.1859999999999999,"5min":4.266,"15min":4.266},"last":3.484}]},{"osd":7,"last update":"Sat Nov 28 22:26:42 2020","interfaces":[{"interface":"back","average":{"1min":3.0129999999999999,"5min":3.1230000000000002,"15min":3.0910000000000002},"min":{"1min":2.5169999999999999,"5min":2.5070000000000001,"15min":2.2770000000000001},"max":{"1min":3.6669999999999998,"5min":4.407,"15min":4.407},"last":2.7490000000000001},{"interface":"front","average":{"1min":3.0339999999999998,"5min":3.198,"15min":3.1819999999999999},"min":{"1min":2.589,"5min":2.3580000000000001,"15min":2.3580000000000001},"max":{"1min":3.6629999999999998,"5min":5.3899999999999997,"15min":5.3899999999999997},"last":3.9169999999999998}]}]},{"osd":0,"up_from":14,"seq":60129542631,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050608,"kb_used_data":2024,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321232,"statfs":{"total":96636764160,"available":95560941568,"internally_reserved":1073741824,"allocated":2072576,"data_stored":959022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Nov 28 22:26:34 2020","interfaces":[{"interface":"back","average":{"1min":1.6100000000000001,"5min":1.7250000000000001,"15min":1.696},"min":{"1min":1.339,"5min":1.244,"15min":1.2170000000000001},"max":{"1min":2.3399999999999999,"5min":3.2120000000000002,"15min":3.3180000000000001},"last":1.8169999999999999},{"interface":"front","average":{"1min":1.6719999999999999,"5min":1.7430000000000001,"15min":1.746},"min":{"1min":1.4950000000000001,"5min":1.3400000000000001,"15min":1.302},"max":{"1min":1.901,"5min":2.9700000000000002,"15min":3.0329999999999999},"last":1.5129999999999999}]},{"osd":2,"last update":"Sat Nov 28 22:26:34 2020","interfaces":[{"interface":"back","average":{"1min":2.1259999999999999,"5min":2.113,"15min":2.1040000000000001},"min":{"1min":1.411,"5min":1.3819999999999999,"15min":1.3819999999999999},"max":{"1min":2.6600000000000001,"5min":3.3239999999999998,"15min":3.3340000000000001},"last":2.6899999999999999},{"interface":"front","average":{"1min":2.4929999999999999,"5min":2.4550000000000001,"15min":2.4609999999999999},"min":{"1min":2.1230000000000002,"5min":1.25,"15min":1.25},"max":{"1min":2.8599999999999999,"5min":3.3610000000000002,"15min":3.4540000000000002},"last":2.181}]},{"osd":3,"last update":"Sat Nov 28 22:26:34 2020","interfaces":[{"interface":"back","average":{"1min":1.833,"5min":1.925,"15min":1.917},"min":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.276},"max":{"1min":2.528,"5min":3.3500000000000001,"15min":3.3500000000000001},"last":1.6779999999999999},{"interface":"front","average":{"1min":2.4009999999999998,"5min":2.4289999999999998,"15min":2.387},"min":{"1min":1.6739999999999999,"5min":1.653,"15min":1.512},"max":{"1min":2.899,"5min":3.7290000000000001,"15min":3.7290000000000001},"last":2.2400000000000002}]},{"osd":4,"last update":"Sat Nov 28 22:26:34 2020","interfaces":[{"interface":"back","average":{"1min":2.2490000000000001,"5min":2.363,"15min":2.3079999999999998},"min":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.478},"max":{"1min":3.2919999999999998,"5min":3.8809999999999998,"15min":3.8809999999999998},"last":2.5750000000000002},{"interface":"front","average":{"1min":2.2309999999999999,"5min":2.375,"15min":2.2970000000000002},"min":{"1min":1.7829999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"max":{"1min":2.8940000000000001,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":2.3130000000000002}]},{"osd":5,"last update":"Sat Nov 28 22:26:34 2020","interfaces":[{"interface":"back","average":{"1min":2.8620000000000001,"5min":2.899,"15min":2.8210000000000002},"min":{"1min":2.0089999999999999,"5min":1.8280000000000001,"15min":1.764},"max":{"1min":3.661,"5min":4.4480000000000004,"15min":4.4480000000000004},"last":3.1030000000000002},{"interface":"front","average":{"1min":2.8149999999999999,"5min":2.9279999999999999,"15min":2.8559999999999999},"min":{"1min":2.1379999999999999,"5min":2.1379999999999999,"15min":1.6990000000000001},"max":{"1min":3.2400000000000002,"5min":4.3280000000000003,"15min":4.3280000000000003},"last":2.8290000000000002}]},{"osd":6,"last update":"Sat Nov 28 22:26:34 2020","interfaces":[{"interface":"back","average":{"1min":2.6499999999999999,"5min":2.7810000000000001,"15min":2.706},"min":{"1min":2.1230000000000002,"5min":1.982,"15min":1.732},"max":{"1min":3.1160000000000001,"5min":3.8170000000000002,"15min":3.8170000000000002},"last":2.645},{"interface":"front","average":{"1min":3.0449999999999999,"5min":3.1320000000000001,"15min":3.069},"min":{"1min":2.2330000000000001,"5min":2.2330000000000001,"15min":2.0190000000000001},"max":{"1min":3.5129999999999999,"5min":4.5750000000000002,"15min":4.5750000000000002},"last":3.3029999999999999}]},{"osd":7,"last update":"Sat Nov 28 22:26:34 2020","interfaces":[{"interface":"back","average":{"1min":3.069,"5min":3.1440000000000001,"15min":3.1120000000000001},"min":{"1min":2.593,"5min":2.4900000000000002,"15min":1.8340000000000001},"max":{"1min":3.4849999999999999,"5min":4.7119999999999997,"15min":4.7119999999999997},"last":3.0609999999999999},{"interface":"front","average":{"1min":3.044,"5min":3.0859999999999999,"15min":3.008},"min":{"1min":2.4609999999999999,"5min":2.294,"15min":2.294},"max":{"1min":3.3940000000000001,"5min":4.0499999999999998,"15min":4.1660000000000004},"last":2.9950000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542631,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050608,"kb_used_data":2024,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321232,"statfs":{"total":96636764160,"available":95560941568,"internally_reserved":1073741824,"allocated":2072576,"data_stored":959022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 28 22:25:56 2020","interfaces":[{"interface":"back","average":{"1min":1.7110000000000001,"5min":1.6759999999999999,"15min":1.6599999999999999},"min":{"1min":1.4239999999999999,"5min":1.2969999999999999,"15min":1.1200000000000001},"max":{"1min":2.7839999999999998,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.8779999999999999,"5min":1.8049999999999999,"15min":1.819},"min":{"1min":1.538,"5min":1.3340000000000001,"15min":1.3340000000000001},"max":{"1min":2.5110000000000001,"5min":3.1720000000000002,"15min":3.3570000000000002},"last":1.732}]},{"osd":1,"last update":"Sat Nov 28 22:25:56 2020","interfaces":[{"interface":"back","average":{"1min":1.8260000000000001,"5min":1.891,"15min":1.9179999999999999},"min":{"1min":1.4790000000000001,"5min":1.3919999999999999,"15min":1.3460000000000001},"max":{"1min":2.3279999999999998,"5min":3.802,"15min":3.802},"last":1.613},{"interface":"front","average":{"1min":2.4239999999999999,"5min":2.3940000000000001,"15min":2.3769999999999998},"min":{"1min":1.988,"5min":1.5269999999999999,"15min":1.4670000000000001},"max":{"1min":3.4390000000000001,"5min":3.4529999999999998,"15min":3.4529999999999998},"last":2.016}]},{"osd":3,"last update":"Sat Nov 28 22:25:56 2020","interfaces":[{"interface":"back","average":{"1min":2.1379999999999999,"5min":2.2170000000000001,"15min":2.2090000000000001},"min":{"1min":1.5720000000000001,"5min":1.4490000000000001,"15min":1.361},"max":{"1min":2.6840000000000002,"5min":3.5459999999999998,"15min":3.6440000000000001},"last":2.077},{"interface":"front","average":{"1min":2.3159999999999998,"5min":2.27,"15min":2.2730000000000001},"min":{"1min":1.7490000000000001,"5min":1.696,"15min":1.5409999999999999},"max":{"1min":2.7429999999999999,"5min":3.0640000000000001,"15min":3.8799999999999999},"last":2.3849999999999998}]},{"osd":4,"last update":"Sat Nov 28 22:25:56 2020","interfaces":[{"interface":"back","average":{"1min":2.4940000000000002,"5min":2.4089999999999998,"15min":2.383},"min":{"1min":1.784,"5min":1.7490000000000001,"15min":1.645},"max":{"1min":3.976,"5min":4.1159999999999997,"15min":4.1159999999999997},"last":1.8120000000000001},{"interface":"front","average":{"1min":2.706,"5min":2.6349999999999998,"15min":2.637},"min":{"1min":2.0600000000000001,"5min":1.76,"15min":1.595},"max":{"1min":3.5110000000000001,"5min":3.7280000000000002,"15min":4.2140000000000004},"last":2.2999999999999998}]},{"osd":5,"last update":"Sat Nov 28 22:25:56 2020","interfaces":[{"interface":"back","average":{"1min":2.8980000000000001,"5min":2.839,"15min":2.8980000000000001},"min":{"1min":2.246,"5min":1.8169999999999999,"15min":1.8169999999999999},"max":{"1min":3.5830000000000002,"5min":3.9100000000000001,"15min":4.4089999999999998},"last":2.4950000000000001},{"interface":"front","average":{"1min":2.956,"5min":2.8919999999999999,"15min":2.8820000000000001},"min":{"1min":2.1869999999999998,"5min":1.925,"15min":1.925},"max":{"1min":3.8420000000000001,"5min":3.8420000000000001,"15min":4.1520000000000001},"last":2.8100000000000001}]},{"osd":6,"last update":"Sat Nov 28 22:25:56 2020","interfaces":[{"interface":"back","average":{"1min":3.1339999999999999,"5min":3.052,"15min":3.093},"min":{"1min":2.516,"5min":2.1150000000000002,"15min":2.1070000000000002},"max":{"1min":3.649,"5min":3.9580000000000002,"15min":3.9580000000000002},"last":3.1779999999999999},{"interface":"front","average":{"1min":2.992,"5min":2.972,"15min":2.964},"min":{"1min":2.2730000000000001,"5min":2.2730000000000001,"15min":2.1349999999999998},"max":{"1min":3.766,"5min":4.3150000000000004,"15min":4.3150000000000004},"last":2.2189999999999999}]},{"osd":7,"last update":"Sat Nov 28 22:25:56 2020","interfaces":[{"interface":"back","average":{"1min":3.266,"5min":3.202,"15min":3.2189999999999999},"min":{"1min":2.774,"5min":2.4990000000000001,"15min":2.0569999999999999},"max":{"1min":3.964,"5min":4.1980000000000004,"15min":4.1980000000000004},"last":2.7530000000000001},{"interface":"front","average":{"1min":3.266,"5min":3.2970000000000002,"15min":3.3380000000000001},"min":{"1min":2.6230000000000002,"5min":2.6230000000000002,"15min":2.1110000000000002},"max":{"1min":3.7599999999999998,"5min":4.0209999999999999,"15min":4.3289999999999997},"last":3.5449999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542631,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050608,"kb_used_data":2024,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321232,"statfs":{"total":96636764160,"available":95560941568,"internally_reserved":1073741824,"allocated":2072576,"data_stored":959022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 28 22:26:36 2020","interfaces":[{"interface":"back","average":{"1min":1.742,"5min":1.6879999999999999,"15min":1.6919999999999999},"min":{"1min":1.355,"5min":1.258,"15min":1.143},"max":{"1min":2.1040000000000001,"5min":2.5409999999999999,"15min":2.9100000000000001},"last":1.327},{"interface":"front","average":{"1min":1.665,"5min":1.694,"15min":1.6699999999999999},"min":{"1min":1.4770000000000001,"5min":1.375,"15min":1.2490000000000001},"max":{"1min":1.9490000000000001,"5min":2.476,"15min":2.476},"last":1.633}]},{"osd":1,"last update":"Sat Nov 28 22:26:36 2020","interfaces":[{"interface":"back","average":{"1min":1.595,"5min":1.7529999999999999,"15min":1.776},"min":{"1min":1.4630000000000001,"5min":1.377,"15min":1.3140000000000001},"max":{"1min":1.829,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":1.512},{"interface":"front","average":{"1min":1.982,"5min":2.077,"15min":2.1040000000000001},"min":{"1min":1.569,"5min":1.5389999999999999,"15min":1.4390000000000001},"max":{"1min":2.5990000000000002,"5min":2.806,"15min":3.0990000000000002},"last":1.7789999999999999}]},{"osd":2,"last update":"Sat Nov 28 22:26:36 2020","interfaces":[{"interface":"back","average":{"1min":2.2429999999999999,"5min":2.2410000000000001,"15min":2.2679999999999998},"min":{"1min":1.8420000000000001,"5min":1.5089999999999999,"15min":1.3700000000000001},"max":{"1min":2.9119999999999999,"5min":3.4900000000000002,"15min":3.4900000000000002},"last":1.9910000000000001},{"interface":"front","average":{"1min":2.589,"5min":2.5350000000000001,"15min":2.609},"min":{"1min":2.097,"5min":2.0600000000000001,"15min":1.8340000000000001},"max":{"1min":3.234,"5min":3.4079999999999999,"15min":4.3289999999999997},"last":2.0569999999999999}]},{"osd":4,"last update":"Sat Nov 28 22:26:36 2020","interfaces":[{"interface":"back","average":{"1min":2.331,"5min":2.2490000000000001,"15min":2.331},"min":{"1min":1.9850000000000001,"5min":1.6559999999999999,"15min":1.6559999999999999},"max":{"1min":3.0649999999999999,"5min":3.0790000000000002,"15min":4.0519999999999996},"last":1.397},{"interface":"front","average":{"1min":2.6269999999999998,"5min":2.6259999999999999,"15min":2.5950000000000002},"min":{"1min":2.02,"5min":1.506,"15min":1.506},"max":{"1min":3.472,"5min":3.6779999999999999,"15min":4.0140000000000002},"last":2.9430000000000001}]},{"osd":5,"last update":"Sat Nov 28 22:26:36 2020","interfaces":[{"interface":"back","average":{"1min":2.903,"5min":2.8079999999999998,"15min":2.8330000000000002},"min":{"1min":2.2690000000000001,"5min":1.9179999999999999,"15min":1.8979999999999999},"max":{"1min":3.6139999999999999,"5min":3.7170000000000001,"15min":4.1699999999999999},"last":2.6920000000000002},{"interface":"front","average":{"1min":2.8050000000000002,"5min":2.8719999999999999,"15min":2.9399999999999999},"min":{"1min":2.1909999999999998,"5min":2.181,"15min":1.96},"max":{"1min":3.5249999999999999,"5min":3.6949999999999998,"15min":4.085},"last":3.1459999999999999}]},{"osd":6,"last update":"Sat Nov 28 22:26:36 2020","interfaces":[{"interface":"back","average":{"1min":3.0470000000000002,"5min":3.0579999999999998,"15min":3.1579999999999999},"min":{"1min":2.1800000000000002,"5min":2.1800000000000002,"15min":2.173},"max":{"1min":3.7679999999999998,"5min":3.9169999999999998,"15min":4.4980000000000002},"last":2.5299999999999998},{"interface":"front","average":{"1min":2.8860000000000001,"5min":2.8370000000000002,"15min":2.8719999999999999},"min":{"1min":2.2330000000000001,"5min":2.2330000000000001,"15min":2.1299999999999999},"max":{"1min":3.5150000000000001,"5min":3.5150000000000001,"15min":4.8819999999999997},"last":2.0990000000000002}]},{"osd":7,"last update":"Sat Nov 28 22:26:36 2020","interfaces":[{"interface":"back","average":{"1min":3.0139999999999998,"5min":3.0339999999999998,"15min":3.0720000000000001},"min":{"1min":2.6230000000000002,"5min":2.5049999999999999,"15min":2.3250000000000002},"max":{"1min":3.5289999999999999,"5min":3.6899999999999999,"15min":3.9420000000000002},"last":3.0099999999999998},{"interface":"front","average":{"1min":3.149,"5min":3.1760000000000002,"15min":3.1680000000000001},"min":{"1min":2.5899999999999999,"5min":2.4279999999999999,"15min":2.1989999999999998},"max":{"1min":3.8559999999999999,"5min":3.8559999999999999,"15min":4.5880000000000001},"last":3.1960000000000002}]}]},{"osd":4,"up_from":14,"seq":60129542631,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050608,"kb_used_data":2024,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321232,"statfs":{"total":96636764160,"available":95560941568,"internally_reserved":1073741824,"allocated":2072576,"data_stored":959022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 28 22:26:23 2020","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.754,"15min":1.748},"min":{"1min":1.232,"5min":1.232,"15min":1.012},"max":{"1min":3.0089999999999999,"5min":3.0249999999999999,"15min":3.2549999999999999},"last":2.2120000000000002},{"interface":"front","average":{"1min":1.794,"5min":1.871,"15min":1.8089999999999999},"min":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.0980000000000001},"max":{"1min":2.8780000000000001,"5min":3.294,"15min":3.452},"last":2.5979999999999999}]},{"osd":1,"last update":"Sat Nov 28 22:26:23 2020","interfaces":[{"interface":"back","average":{"1min":2.0790000000000002,"5min":2.1360000000000001,"15min":2.2160000000000002},"min":{"1min":1.5209999999999999,"5min":1.3080000000000001,"15min":1.284},"max":{"1min":2.7879999999999998,"5min":3.3159999999999998,"15min":3.6720000000000002},"last":2.1040000000000001},{"interface":"front","average":{"1min":2.032,"5min":2.0750000000000002,"15min":2.1120000000000001},"min":{"1min":1.498,"5min":1.369,"15min":1.0469999999999999},"max":{"1min":2.7839999999999998,"5min":2.9910000000000001,"15min":3.2570000000000001},"last":1.482}]},{"osd":2,"last update":"Sat Nov 28 22:26:23 2020","interfaces":[{"interface":"back","average":{"1min":1.804,"5min":1.9330000000000001,"15min":1.9850000000000001},"min":{"1min":1.391,"5min":1.391,"15min":1.391},"max":{"1min":2.4700000000000002,"5min":2.9300000000000002,"15min":3.5880000000000001},"last":2.5009999999999999},{"interface":"front","average":{"1min":2.4540000000000002,"5min":2.4729999999999999,"15min":2.5089999999999999},"min":{"1min":1.887,"5min":1.7210000000000001,"15min":1.419},"max":{"1min":3.169,"5min":3.7919999999999998,"15min":3.7919999999999998},"last":1.6539999999999999}]},{"osd":3,"last update":"Sat Nov 28 22:26:23 2020","interfaces":[{"interface":"back","average":{"1min":2.399,"5min":2.3679999999999999,"15min":2.4220000000000002},"min":{"1min":1.8080000000000001,"5min":1.431,"15min":1.407},"max":{"1min":3.161,"5min":3.3620000000000001,"15min":3.5310000000000001},"last":1.877},{"interface":"front","average":{"1min":2.1389999999999998,"5min":2.2709999999999999,"15min":2.331},"min":{"1min":1.669,"5min":1.6579999999999999,"15min":1.246},"max":{"1min":2.7629999999999999,"5min":3.8759999999999999,"15min":3.9580000000000002},"last":3.0430000000000001}]},{"osd":5,"last update":"Sat Nov 28 22:26:23 2020","interfaces":[{"interface":"back","average":{"1min":2.665,"5min":2.6869999999999998,"15min":2.6909999999999998},"min":{"1min":1.8260000000000001,"5min":1.3819999999999999,"15min":1.3819999999999999},"max":{"1min":3.5369999999999999,"5min":4.1280000000000001,"15min":4.1280000000000001},"last":3.3300000000000001},{"interface":"front","average":{"1min":2.657,"5min":2.7309999999999999,"15min":2.7810000000000001},"min":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"max":{"1min":3.6600000000000001,"5min":3.6629999999999998,"15min":4.2350000000000003},"last":2.6960000000000002}]},{"osd":6,"last update":"Sat Nov 28 22:26:23 2020","interfaces":[{"interface":"back","average":{"1min":2.5960000000000001,"5min":2.6709999999999998,"15min":2.6480000000000001},"min":{"1min":2.0059999999999998,"5min":1.1599999999999999,"15min":1.1599999999999999},"max":{"1min":3.1949999999999998,"5min":4.0800000000000001,"15min":4.351},"last":3.4279999999999999},{"interface":"front","average":{"1min":2.984,"5min":3.0310000000000001,"15min":3.0379999999999998},"min":{"1min":2.1659999999999999,"5min":2.1219999999999999,"15min":1.8600000000000001},"max":{"1min":3.8959999999999999,"5min":4.5270000000000001,"15min":4.5270000000000001},"last":2.9569999999999999}]},{"osd":7,"last update":"Sat Nov 28 22:26:23 2020","interfaces":[{"interface":"back","average":{"1min":3.089,"5min":3.1059999999999999,"15min":3.1389999999999998},"min":{"1min":2.2109999999999999,"5min":2.2109999999999999,"15min":1.591},"max":{"1min":3.5129999999999999,"5min":4.0229999999999997,"15min":4.2149999999999999},"last":2.4039999999999999},{"interface":"front","average":{"1min":2.8809999999999998,"5min":3.0720000000000001,"15min":3.1040000000000001},"min":{"1min":2.153,"5min":1.319,"15min":1.319},"max":{"1min":3.3929999999999998,"5min":4.2709999999999999,"15min":5.133},"last":3.7029999999999998}]}]},{"osd":5,"up_from":14,"seq":60129542631,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050604,"kb_used_data":2020,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321236,"statfs":{"total":96636764160,"available":95560945664,"internally_reserved":1073741824,"allocated":2068480,"data_stored":958419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 28 22:26:29 2020","interfaces":[{"interface":"back","average":{"1min":2.2799999999999998,"5min":2.0899999999999999,"15min":2.0489999999999999},"min":{"1min":1.321,"5min":1.0609999999999999,"15min":1.0609999999999999},"max":{"1min":2.9140000000000001,"5min":2.9889999999999999,"15min":3.4780000000000002},"last":1.4330000000000001},{"interface":"front","average":{"1min":2.5529999999999999,"5min":2.4079999999999999,"15min":2.375},"min":{"1min":2.0059999999999998,"5min":1.5209999999999999,"15min":1.5209999999999999},"max":{"1min":3.1190000000000002,"5min":4.1390000000000002,"15min":4.6390000000000002},"last":2.8330000000000002}]},{"osd":1,"last update":"Sat Nov 28 22:26:29 2020","interfaces":[{"interface":"back","average":{"1min":1.9319999999999999,"5min":2.0489999999999999,"15min":2.1440000000000001},"min":{"1min":1.4319999999999999,"5min":1.286,"15min":1.266},"max":{"1min":3.0590000000000002,"5min":3.4729999999999999,"15min":4.4909999999999997},"last":1.9370000000000001},{"interface":"front","average":{"1min":2.4700000000000002,"5min":2.6429999999999998,"15min":2.649},"min":{"1min":1.446,"5min":1.446,"15min":1.446},"max":{"1min":3.4060000000000001,"5min":4.7960000000000003,"15min":5.0449999999999999},"last":3.262}]},{"osd":2,"last update":"Sat Nov 28 22:26:29 2020","interfaces":[{"interface":"back","average":{"1min":2.0369999999999999,"5min":1.9330000000000001,"15min":1.9830000000000001},"min":{"1min":1.6000000000000001,"5min":1.4570000000000001,"15min":1.367},"max":{"1min":3.04,"5min":3.0569999999999999,"15min":3.8260000000000001},"last":2.5659999999999998},{"interface":"front","average":{"1min":2.1440000000000001,"5min":2.363,"15min":2.3570000000000002},"min":{"1min":1.6930000000000001,"5min":1.4410000000000001,"15min":1.381},"max":{"1min":3.2730000000000001,"5min":3.677,"15min":4.742},"last":2.1669999999999998}]},{"osd":3,"last update":"Sat Nov 28 22:26:29 2020","interfaces":[{"interface":"back","average":{"1min":1.881,"5min":2.0640000000000001,"15min":2.0550000000000002},"min":{"1min":1.5680000000000001,"5min":1.363,"15min":1.2609999999999999},"max":{"1min":2.4980000000000002,"5min":4.532,"15min":5.2469999999999999},"last":1.72},{"interface":"front","average":{"1min":2.4780000000000002,"5min":2.4660000000000002,"15min":2.4929999999999999},"min":{"1min":1.879,"5min":1.7669999999999999,"15min":1.427},"max":{"1min":3.7280000000000002,"5min":3.7280000000000002,"15min":4.5979999999999999},"last":2.0110000000000001}]},{"osd":4,"last update":"Sat Nov 28 22:26:29 2020","interfaces":[{"interface":"back","average":{"1min":2.3180000000000001,"5min":2.4769999999999999,"15min":2.4910000000000001},"min":{"1min":1.3480000000000001,"5min":1.157,"15min":1.0580000000000001},"max":{"1min":3.2909999999999999,"5min":4.0350000000000001,"15min":4.335},"last":2.9209999999999998},{"interface":"front","average":{"1min":2.468,"5min":2.6469999999999998,"15min":2.6850000000000001},"min":{"1min":1.7769999999999999,"5min":1.7110000000000001,"15min":0.94699999999999995},"max":{"1min":3.476,"5min":4.1749999999999998,"15min":5.1900000000000004},"last":2.419}]},{"osd":6,"last update":"Sat Nov 28 22:26:29 2020","interfaces":[{"interface":"back","average":{"1min":2.347,"5min":2.5179999999999998,"15min":2.5739999999999998},"min":{"1min":1.7809999999999999,"5min":1.1930000000000001,"15min":1.107},"max":{"1min":3.1829999999999998,"5min":4.9710000000000001,"15min":5.431},"last":3.6309999999999998},{"interface":"front","average":{"1min":2.879,"5min":2.9449999999999998,"15min":2.9710000000000001},"min":{"1min":2.1669999999999998,"5min":1.3999999999999999,"15min":1.3460000000000001},"max":{"1min":3.9300000000000002,"5min":5.4169999999999998,"15min":5.4169999999999998},"last":3.3450000000000002}]},{"osd":7,"last update":"Sat Nov 28 22:26:29 2020","interfaces":[{"interface":"back","average":{"1min":2.9209999999999998,"5min":2.9969999999999999,"15min":3.036},"min":{"1min":2.3260000000000001,"5min":1.6499999999999999,"15min":1.4710000000000001},"max":{"1min":3.6589999999999998,"5min":4.1799999999999997,"15min":4.9829999999999997},"last":2.7599999999999998},{"interface":"front","average":{"1min":3.218,"5min":3.1179999999999999,"15min":3.157},"min":{"1min":2.3719999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"max":{"1min":4.1299999999999999,"5min":4.3029999999999999,"15min":5.319},"last":3.6989999999999998}]}]}]}} 2020-11-28T22:26:48.402 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2020-11-28T21:46:24.622452+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=46, tm_sec=24, tm_wday=5, tm_yday=333, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=28, tm_hour=22, tm_min=26, tm_sec=43, tm_wday=5, tm_yday=333, tm_isdst=0) 2020-11-28T22:26:48.402 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2020-11-28T21:46:24.622452+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=46, tm_sec=24, tm_wday=5, tm_yday=333, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=28, tm_hour=22, tm_min=26, tm_sec=43, tm_wday=5, tm_yday=333, tm_isdst=0) 2020-11-28T22:26:48.403 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2020-11-28T21:46:24.622452+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=46, tm_sec=24, tm_wday=5, tm_yday=333, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=28, tm_hour=22, tm_min=26, tm_sec=43, tm_wday=5, tm_yday=333, tm_isdst=0) 2020-11-28T22:26:48.404 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2020-11-28T21:46:24.622452+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=46, tm_sec=24, tm_wday=5, tm_yday=333, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=28, tm_hour=22, tm_min=26, tm_sec=43, tm_wday=5, tm_yday=333, tm_isdst=0) 2020-11-28T22:26:48.404 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2020-11-28T21:46:16.570066+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=46, tm_sec=16, tm_wday=5, tm_yday=333, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=28, tm_hour=22, tm_min=26, tm_sec=43, tm_wday=5, tm_yday=333, tm_isdst=0) 2020-11-28T22:26:48.405 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2020-11-28T21:46:24.622452+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=46, tm_sec=24, tm_wday=5, tm_yday=333, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=28, tm_hour=22, tm_min=26, tm_sec=43, tm_wday=5, tm_yday=333, tm_isdst=0) 2020-11-28T22:26:48.405 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2020-11-28T21:46:24.622452+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=46, tm_sec=24, tm_wday=5, tm_yday=333, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=28, tm_hour=22, tm_min=26, tm_sec=43, tm_wday=5, tm_yday=333, tm_isdst=0) 2020-11-28T22:26:48.406 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2020-11-28T21:46:24.622452+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=46, tm_sec=24, tm_wday=5, tm_yday=333, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=28, tm_hour=22, tm_min=26, tm_sec=43, tm_wday=5, tm_yday=333, tm_isdst=0) 2020-11-28T22:26:48.406 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2020-11-28T21:46:24.622452+0000 time.struct_time(tm_year=2020, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=46, tm_sec=24, tm_wday=5, tm_yday=333, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=11, tm_mday=28, tm_hour=22, tm_min=26, tm_sec=43, tm_wday=5, tm_yday=333, tm_isdst=0) 2020-11-28T22:26:48.407 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2020-11-28T22:27:08.408 INFO:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-11-28T22:27:08.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.523+0000 7fd77a996700 1 Processor -- start 2020-11-28T22:27:08.528 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.523+0000 7fd77a996700 1 -- start start 2020-11-28T22:27:08.530 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.523+0000 7fd77a996700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd7740b01a0 0x7fd774143f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:27:08.531 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.523+0000 7fd77a996700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd774143ba0 con 0x7fd774060010 2020-11-28T22:27:08.531 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.523+0000 7fd77a996700 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7740618a0 con 0x7fd77409abb0 2020-11-28T22:27:08.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.523+0000 7fd77a996700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd774144b60 con 0x7fd7740b01a0 2020-11-28T22:27:08.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.527+0000 7fd779994700 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fd77409abb0 0x7fd774061190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:53896/0 (socket says 172.21.15.162:53896) 2020-11-28T22:27:08.532 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.527+0000 7fd779994700 1 -- 172.21.15.162:0/3244240039 learned_addr learned my addr 172.21.15.162:0/3244240039 (peer_addr_for_me v1:172.21.15.162:0/0) 2020-11-28T22:27:08.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.527+0000 7fd779193700 1 --2- 172.21.15.162:0/3244240039 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd7740b01a0 0x7fd774143f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:27:08.533 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.527+0000 7fd779193700 1 -- 172.21.15.162:0/3244240039 >> v1:172.21.15.162:6789/0 conn(0x7fd77409abb0 legacy=0x7fd774061190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:27:08.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.527+0000 7fd779193700 1 -- 172.21.15.162:0/3244240039 >> v1:172.21.15.118:6789/0 conn(0x7fd774060010 legacy=0x7fd774144450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:27:08.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.527+0000 7fd779193700 1 -- 172.21.15.162:0/3244240039 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd77409af90 con 0x7fd7740b01a0 2020-11-28T22:27:08.534 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.527+0000 7fd779193700 1 --2- 172.21.15.162:0/3244240039 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd7740b01a0 0x7fd774143f60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7fd764002f30 tx=0x7fd764006dc0).ready entity=mon.2 client_cookie=15b98f66bc0580b6 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:27:08.540 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.527+0000 7fd778992700 1 -- 172.21.15.162:0/3244240039 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7640097f0 con 0x7fd7740b01a0 2020-11-28T22:27:08.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.527+0000 7fd778992700 1 -- 172.21.15.162:0/3244240039 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd764009950 con 0x7fd7740b01a0 2020-11-28T22:27:08.541 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.527+0000 7fd778992700 1 -- 172.21.15.162:0/3244240039 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd764009cb0 con 0x7fd7740b01a0 2020-11-28T22:27:08.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.527+0000 7fd77a996700 1 -- 172.21.15.162:0/3244240039 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd7740b01a0 msgr2=0x7fd774143f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:27:08.542 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.527+0000 7fd77a996700 1 --2- 172.21.15.162:0/3244240039 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd7740b01a0 0x7fd774143f60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7fd764002f30 tx=0x7fd764006dc0).stop 2020-11-28T22:27:08.543 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.527+0000 7fd77a996700 1 -- 172.21.15.162:0/3244240039 shutdown_connections 2020-11-28T22:27:08.543 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.527+0000 7fd77a996700 1 --2- 172.21.15.162:0/3244240039 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd7740b01a0 0x7fd774143f60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:27:08.543 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.527+0000 7fd77a996700 1 -- 172.21.15.162:0/3244240039 >> 172.21.15.162:0/3244240039 conn(0x7fd7740b54f0 msgr2=0x7fd7740b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:27:08.544 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.527+0000 7fd77a996700 1 -- 172.21.15.162:0/3244240039 shutdown_connections 2020-11-28T22:27:08.544 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.527+0000 7fd77a996700 1 -- 172.21.15.162:0/3244240039 wait complete. 2020-11-28T22:27:08.545 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.531+0000 7fd77a996700 1 Processor -- start 2020-11-28T22:27:08.545 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.531+0000 7fd77a996700 1 -- start start 2020-11-28T22:27:08.545 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.535+0000 7fd77a996700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd774060010 0x7fd7741528f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:27:08.546 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.535+0000 7fd77a996700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd77409abb0 0x7fd7740ab340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:27:08.546 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.535+0000 7fd77a996700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd7740b01a0 0x7fd7740ad1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:27:08.547 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.535+0000 7fd77a996700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd774140e10 con 0x7fd77409abb0 2020-11-28T22:27:08.547 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.535+0000 7fd77a996700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7741407d0 con 0x7fd774060010 2020-11-28T22:27:08.547 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.535+0000 7fd77a996700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd774141040 con 0x7fd7740b01a0 2020-11-28T22:27:08.548 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.535+0000 7fd779994700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd774060010 0x7fd7741528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:27:08.548 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.535+0000 7fd779994700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd774060010 0x7fd7741528f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47056/0 (socket says 172.21.15.162:47056) 2020-11-28T22:27:08.548 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.535+0000 7fd779994700 1 -- 172.21.15.162:0/3607356347 learned_addr learned my addr 172.21.15.162:0/3607356347 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T22:27:08.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.535+0000 7fd779193700 1 --2- 172.21.15.162:0/3607356347 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd77409abb0 0x7fd7740ab340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:27:08.549 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.535+0000 7fd77a195700 1 --2- 172.21.15.162:0/3607356347 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd7740b01a0 0x7fd7740ad1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:27:08.550 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.535+0000 7fd779994700 1 -- 172.21.15.162:0/3607356347 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd7740b01a0 msgr2=0x7fd7740ad1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:27:08.550 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.535+0000 7fd779994700 1 --2- 172.21.15.162:0/3607356347 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd7740b01a0 0x7fd7740ad1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:27:08.550 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.535+0000 7fd779994700 1 -- 172.21.15.162:0/3607356347 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd77409abb0 msgr2=0x7fd7740ab340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:27:08.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.535+0000 7fd779994700 1 --2- 172.21.15.162:0/3607356347 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd77409abb0 0x7fd7740ab340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:27:08.551 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.535+0000 7fd779994700 1 -- 172.21.15.162:0/3607356347 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7741412e0 con 0x7fd774060010 2020-11-28T22:27:08.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.535+0000 7fd779994700 1 --2- 172.21.15.162:0/3607356347 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd774060010 0x7fd7741528f0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 rx=0x7fd770000fe0 tx=0x7fd770007c50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:27:08.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.535+0000 7fd762ffd700 1 -- 172.21.15.162:0/3607356347 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7700072e0 con 0x7fd774060010 2020-11-28T22:27:08.552 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.535+0000 7fd762ffd700 1 -- 172.21.15.162:0/3607356347 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd770007440 con 0x7fd774060010 2020-11-28T22:27:08.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.535+0000 7fd77a996700 1 -- 172.21.15.162:0/3607356347 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7740af8a0 con 0x7fd774060010 2020-11-28T22:27:08.553 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.535+0000 7fd762ffd700 1 -- 172.21.15.162:0/3607356347 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7700057d0 con 0x7fd774060010 2020-11-28T22:27:08.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.535+0000 7fd77a996700 1 -- 172.21.15.162:0/3607356347 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7740ad8e0 con 0x7fd774060010 2020-11-28T22:27:08.554 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.535+0000 7fd77a996700 1 -- 172.21.15.162:0/3607356347 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd774140590 con 0x7fd774060010 2020-11-28T22:27:08.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.539+0000 7fd762ffd700 1 -- 172.21.15.162:0/3607356347 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7fd77001a060 con 0x7fd774060010 2020-11-28T22:27:08.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.539+0000 7fd762ffd700 1 --2- 172.21.15.162:0/3607356347 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd754055330 0x7fd7540577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:27:08.556 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.539+0000 7fd762ffd700 1 -- 172.21.15.162:0/3607356347 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5339+0+0 (secure 0 0 0) 0x7fd77006d030 con 0x7fd774060010 2020-11-28T22:27:08.557 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.539+0000 7fd779193700 1 --2- 172.21.15.162:0/3607356347 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd754055330 0x7fd7540577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:27:08.557 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.539+0000 7fd779193700 1 --2- 172.21.15.162:0/3607356347 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd754055330 0x7fd7540577f0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0x7fd76401b000 tx=0x7fd764015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:27:08.558 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.543+0000 7fd762ffd700 1 -- 172.21.15.162:0/3607356347 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7fd774140590 con 0x7fd774060010 2020-11-28T22:27:08.684 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.679+0000 7fd77a996700 1 -- 172.21.15.162:0/3607356347 --> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd774140590 con 0x7fd754055330 2020-11-28T22:27:08.691 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.687+0000 7fd762ffd700 1 -- 172.21.15.162:0/3607356347 <== mgr.4105 v2:172.21.15.162:6800/12156 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+66429 (secure 0 0 0) 0x7fd774140590 con 0x7fd754055330 2020-11-28T22:27:08.692 INFO:teuthology.orchestra.run.smithi162.stdout: 2020-11-28T22:27:08.693 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.687+0000 7fd77a996700 1 -- 172.21.15.162:0/3607356347 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd754055330 msgr2=0x7fd7540577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:27:08.693 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.687+0000 7fd77a996700 1 --2- 172.21.15.162:0/3607356347 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd754055330 0x7fd7540577f0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0x7fd76401b000 tx=0x7fd764015000).stop 2020-11-28T22:27:08.694 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.687+0000 7fd77a996700 1 -- 172.21.15.162:0/3607356347 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd774060010 msgr2=0x7fd7741528f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:27:08.694 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.687+0000 7fd77a996700 1 --2- 172.21.15.162:0/3607356347 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd774060010 0x7fd7741528f0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 rx=0x7fd770000fe0 tx=0x7fd770007c50).stop 2020-11-28T22:27:08.694 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.687+0000 7fd77a996700 1 -- 172.21.15.162:0/3607356347 shutdown_connections 2020-11-28T22:27:08.694 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.687+0000 7fd77a996700 1 --2- 172.21.15.162:0/3607356347 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7fd754055330 0x7fd7540577f0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:27:08.695 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.687+0000 7fd77a996700 1 --2- 172.21.15.162:0/3607356347 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd7740b01a0 0x7fd7740ad1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:27:08.695 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.687+0000 7fd77a996700 1 --2- 172.21.15.162:0/3607356347 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd774060010 0x7fd7741528f0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:27:08.695 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.687+0000 7fd77a996700 1 --2- 172.21.15.162:0/3607356347 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd77409abb0 0x7fd7740ab340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:27:08.695 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.691+0000 7fd77a996700 1 -- 172.21.15.162:0/3607356347 >> 172.21.15.162:0/3607356347 conn(0x7fd7740b54f0 msgr2=0x7fd7740b0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:27:08.695 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.691+0000 7fd77a996700 1 -- 172.21.15.162:0/3607356347 shutdown_connections 2020-11-28T22:27:08.696 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.691+0000 7fd77a996700 1 -- 172.21.15.162:0/3607356347 wait complete. 2020-11-28T22:27:08.696 INFO:teuthology.orchestra.run.smithi162.stderr:dumped all 2020-11-28T22:27:08.706 INFO:teuthology.orchestra.run.smithi162.stdout:{"pg_ready":true,"pg_map":{"version":1380,"stamp":"2020-11-28T22:27:08.297713+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1180,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8404860,"kb_used_data":16188,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746569860,"statfs":{"total":773094113280,"available":764487536640,"internally_reserved":8589934592,"allocated":16576512,"data_stored":7671573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003876"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":"133","reported_epoch":"163","state":"active+clean","last_fresh":"2020-11-28T22:26:49.410926+0000","last_change":"2020-11-28T22:26:49.411075+0000","last_active":"2020-11-28T22:26:49.410926+0000","last_peered":"2020-11-28T22:26:49.410926+0000","last_clean":"2020-11-28T22:26:49.410926+0000","last_became_active":"2020-11-28T21:46:25.652265+0000","last_became_peered":"2020-11-28T21:46:25.652265+0000","last_unstale":"2020-11-28T22:26:49.410926+0000","last_undegraded":"2020-11-28T22:26:49.410926+0000","last_fullsized":"2020-11-28T22:26:49.410926+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T22:26:49.410919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T22:26:49.410919+0000","last_clean_scrub_stamp":"2020-11-28T22:26:49.410919+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"136","reported_epoch":"163","state":"active+clean","last_fresh":"2020-11-28T22:26:50.498985+0000","last_change":"2020-11-28T22:26:50.499184+0000","last_active":"2020-11-28T22:26:50.498985+0000","last_peered":"2020-11-28T22:26:50.498985+0000","last_clean":"2020-11-28T22:26:50.498985+0000","last_became_active":"2020-11-28T21:46:25.653417+0000","last_became_peered":"2020-11-28T21:46:25.653417+0000","last_unstale":"2020-11-28T22:26:50.498985+0000","last_undegraded":"2020-11-28T22:26:50.498985+0000","last_fullsized":"2020-11-28T22:26:50.498985+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T22:26:50.498975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T22:26:50.498975+0000","last_clean_scrub_stamp":"2020-11-28T22:26:50.498975+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"138","reported_epoch":"163","state":"active+clean","last_fresh":"2020-11-28T22:26:48.474064+0000","last_change":"2020-11-28T22:26:48.474257+0000","last_active":"2020-11-28T22:26:48.474064+0000","last_peered":"2020-11-28T22:26:48.474064+0000","last_clean":"2020-11-28T22:26:48.474064+0000","last_became_active":"2020-11-28T21:46:25.652295+0000","last_became_peered":"2020-11-28T21:46:25.652295+0000","last_unstale":"2020-11-28T22:26:48.474064+0000","last_undegraded":"2020-11-28T22:26:48.474064+0000","last_fullsized":"2020-11-28T22:26:48.474064+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T22:26:48.474050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T22:26:48.474050+0000","last_clean_scrub_stamp":"2020-11-28T22:26:48.474050+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"136","reported_epoch":"163","state":"active+clean","last_fresh":"2020-11-28T22:26:55.581838+0000","last_change":"2020-11-28T22:26:55.581926+0000","last_active":"2020-11-28T22:26:55.581838+0000","last_peered":"2020-11-28T22:26:55.581838+0000","last_clean":"2020-11-28T22:26:55.581838+0000","last_became_active":"2020-11-28T21:46:26.007744+0000","last_became_peered":"2020-11-28T21:46:26.007744+0000","last_unstale":"2020-11-28T22:26:55.581838+0000","last_undegraded":"2020-11-28T22:26:55.581838+0000","last_fullsized":"2020-11-28T22:26:55.581838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T22:26:55.581834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T22:26:55.581834+0000","last_clean_scrub_stamp":"2020-11-28T22:26:55.581834+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"163'4","reported_seq":"141","reported_epoch":"163","state":"active+clean","last_fresh":"2020-11-28T22:26:47.577350+0000","last_change":"2020-11-28T22:26:47.577545+0000","last_active":"2020-11-28T22:26:47.577350+0000","last_peered":"2020-11-28T22:26:47.577350+0000","last_clean":"2020-11-28T22:26:47.577350+0000","last_became_active":"2020-11-28T21:46:20.941467+0000","last_became_peered":"2020-11-28T21:46:20.941467+0000","last_unstale":"2020-11-28T22:26:47.577350+0000","last_undegraded":"2020-11-28T22:26:47.577350+0000","last_fullsized":"2020-11-28T22:26:47.577350+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"163'4","last_scrub_stamp":"2020-11-28T22:26:47.577334+0000","last_deep_scrub":"163'4","last_deep_scrub_stamp":"2020-11-28T22:26:47.577334+0000","last_clean_scrub_stamp":"2020-11-28T22:26:47.577334+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1180,"num_omap_keys":2,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"136","reported_epoch":"163","state":"active+clean","last_fresh":"2020-11-28T22:26:47.505821+0000","last_change":"2020-11-28T22:26:47.506008+0000","last_active":"2020-11-28T22:26:47.505821+0000","last_peered":"2020-11-28T22:26:47.505821+0000","last_clean":"2020-11-28T22:26:47.505821+0000","last_became_active":"2020-11-28T21:46:25.653652+0000","last_became_peered":"2020-11-28T21:46:25.653652+0000","last_unstale":"2020-11-28T22:26:47.505821+0000","last_undegraded":"2020-11-28T22:26:47.505821+0000","last_fullsized":"2020-11-28T22:26:47.505821+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T22:26:47.505808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T22:26:47.505808+0000","last_clean_scrub_stamp":"2020-11-28T22:26:47.505808+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"136","reported_epoch":"163","state":"active+clean","last_fresh":"2020-11-28T22:26:57.613693+0000","last_change":"2020-11-28T22:26:57.613900+0000","last_active":"2020-11-28T22:26:57.613693+0000","last_peered":"2020-11-28T22:26:57.613693+0000","last_clean":"2020-11-28T22:26:57.613693+0000","last_became_active":"2020-11-28T21:46:25.653222+0000","last_became_peered":"2020-11-28T21:46:25.653222+0000","last_unstale":"2020-11-28T22:26:57.613693+0000","last_undegraded":"2020-11-28T22:26:57.613693+0000","last_fullsized":"2020-11-28T22:26:57.613693+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T22:26:57.613680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T22:26:57.613680+0000","last_clean_scrub_stamp":"2020-11-28T22:26:57.613680+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"143","reported_epoch":"163","state":"active+clean","last_fresh":"2020-11-28T22:26:50.825893+0000","last_change":"2020-11-28T22:26:50.825994+0000","last_active":"2020-11-28T22:26:50.825893+0000","last_peered":"2020-11-28T22:26:50.825893+0000","last_clean":"2020-11-28T22:26:50.825893+0000","last_became_active":"2020-11-28T21:46:25.837556+0000","last_became_peered":"2020-11-28T21:46:25.837556+0000","last_unstale":"2020-11-28T22:26:50.825893+0000","last_undegraded":"2020-11-28T22:26:50.825893+0000","last_fullsized":"2020-11-28T22:26:50.825893+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T22:26:50.825889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T22:26:50.825889+0000","last_clean_scrub_stamp":"2020-11-28T22:26:50.825889+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"136","reported_epoch":"163","state":"active+clean","last_fresh":"2020-11-28T22:27:03.590426+0000","last_change":"2020-11-28T22:27:03.590624+0000","last_active":"2020-11-28T22:27:03.590426+0000","last_peered":"2020-11-28T22:27:03.590426+0000","last_clean":"2020-11-28T22:27:03.590426+0000","last_became_active":"2020-11-28T21:46:25.837943+0000","last_became_peered":"2020-11-28T21:46:25.837943+0000","last_unstale":"2020-11-28T22:27:03.590426+0000","last_undegraded":"2020-11-28T22:27:03.590426+0000","last_fullsized":"2020-11-28T22:27:03.590426+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-28T22:27:03.590417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-28T22:27:03.590417+0000","last_clean_scrub_stamp":"2020-11-28T22:27:03.590417+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1180,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542635,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050608,"kb_used_data":2024,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321232,"statfs":{"total":96636764160,"available":95560941568,"internally_reserved":1073741824,"allocated":2072576,"data_stored":959022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 28 22:26:34 2020","interfaces":[{"interface":"back","average":{"1min":1.847,"5min":1.6910000000000001,"15min":1.716},"min":{"1min":1.2729999999999999,"5min":1.081,"15min":0.98699999999999999},"max":{"1min":2.9769999999999999,"5min":3.609,"15min":3.609},"last":1.6419999999999999},{"interface":"front","average":{"1min":1.7849999999999999,"5min":1.8400000000000001,"15min":1.8080000000000001},"min":{"1min":1.429,"5min":1.2470000000000001,"15min":1.1930000000000001},"max":{"1min":2.52,"5min":3.6589999999999998,"15min":3.6589999999999998},"last":1.343}]},{"osd":1,"last update":"Sat Nov 28 22:26:34 2020","interfaces":[{"interface":"back","average":{"1min":1.661,"5min":1.8240000000000001,"15min":1.8320000000000001},"min":{"1min":1.298,"5min":1.2490000000000001,"15min":1.0840000000000001},"max":{"1min":2.8759999999999999,"5min":3.6139999999999999,"15min":3.6139999999999999},"last":1.375},{"interface":"front","average":{"1min":2.2010000000000001,"5min":2.1909999999999998,"15min":2.206},"min":{"1min":1.601,"5min":1.3700000000000001,"15min":1.1779999999999999},"max":{"1min":3.141,"5min":3.4249999999999998,"15min":3.4550000000000001},"last":1.593}]},{"osd":2,"last update":"Sat Nov 28 22:26:34 2020","interfaces":[{"interface":"back","average":{"1min":2.2650000000000001,"5min":2.2629999999999999,"15min":2.194},"min":{"1min":1.373,"5min":1.373,"15min":1.3220000000000001},"max":{"1min":3.2690000000000001,"5min":4.0439999999999996,"15min":4.0439999999999996},"last":2.1440000000000001},{"interface":"front","average":{"1min":2.2389999999999999,"5min":2.1720000000000002,"15min":2.1429999999999998},"min":{"1min":1.7609999999999999,"5min":1.429,"15min":1.3109999999999999},"max":{"1min":3.012,"5min":3.5289999999999999,"15min":3.5289999999999999},"last":2.1059999999999999}]},{"osd":3,"last update":"Sat Nov 28 22:26:34 2020","interfaces":[{"interface":"back","average":{"1min":2.4769999999999999,"5min":2.46,"15min":2.4399999999999999},"min":{"1min":1.778,"5min":1.6200000000000001,"15min":1.3160000000000001},"max":{"1min":3.6859999999999999,"5min":4.3819999999999997,"15min":4.3819999999999997},"last":1.873},{"interface":"front","average":{"1min":2.3290000000000002,"5min":2.4550000000000001,"15min":2.4169999999999998},"min":{"1min":1.7609999999999999,"5min":1.603,"15min":1.4770000000000001},"max":{"1min":2.996,"5min":3.867,"15min":3.867},"last":2.2919999999999998}]},{"osd":4,"last update":"Sat Nov 28 22:26:34 2020","interfaces":[{"interface":"back","average":{"1min":2.5339999999999998,"5min":2.5459999999999998,"15min":2.5129999999999999},"min":{"1min":1.8160000000000001,"5min":1.452,"15min":1.3009999999999999},"max":{"1min":3.5699999999999998,"5min":3.806,"15min":3.806},"last":1.496},{"interface":"front","average":{"1min":2.7149999999999999,"5min":2.7789999999999999,"15min":2.7530000000000001},"min":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.577},"max":{"1min":3.9849999999999999,"5min":4.4260000000000002,"15min":4.4260000000000002},"last":1.974}]},{"osd":5,"last update":"Sat Nov 28 22:26:34 2020","interfaces":[{"interface":"back","average":{"1min":2.6099999999999999,"5min":2.7149999999999999,"15min":2.6989999999999998},"min":{"1min":1.8600000000000001,"5min":1.835,"15min":1.659},"max":{"1min":3.359,"5min":4.2850000000000001,"15min":4.2850000000000001},"last":1.778},{"interface":"front","average":{"1min":2.887,"5min":2.9279999999999999,"15min":2.8969999999999998},"min":{"1min":2.1800000000000002,"5min":2.0019999999999998,"15min":1.345},"max":{"1min":3.7629999999999999,"5min":4.1779999999999999,"15min":4.1779999999999999},"last":2.0590000000000002}]},{"osd":6,"last update":"Sat Nov 28 22:26:34 2020","interfaces":[{"interface":"back","average":{"1min":3.016,"5min":3.0390000000000001,"15min":3.085},"min":{"1min":2.4780000000000002,"5min":1.976,"15min":1.976},"max":{"1min":3.9510000000000001,"5min":4.5629999999999997,"15min":4.5629999999999997},"last":2.2029999999999998},{"interface":"front","average":{"1min":3.1829999999999998,"5min":3.137,"15min":3.1520000000000001},"min":{"1min":2.5950000000000002,"5min":1.804,"15min":1.804},"max":{"1min":4.194,"5min":4.5010000000000003,"15min":4.5010000000000003},"last":2.423}]}]},{"osd":6,"up_from":14,"seq":60129542635,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050608,"kb_used_data":2024,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321232,"statfs":{"total":96636764160,"available":95560941568,"internally_reserved":1073741824,"allocated":2072576,"data_stored":959022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 28 22:26:30 2020","interfaces":[{"interface":"back","average":{"1min":1.6479999999999999,"5min":1.591,"15min":1.645},"min":{"1min":1.3240000000000001,"5min":1.1919999999999999,"15min":1.1060000000000001},"max":{"1min":2.0449999999999999,"5min":2.8069999999999999,"15min":2.8069999999999999},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.7749999999999999,"5min":1.768,"15min":1.7829999999999999},"min":{"1min":1.403,"5min":1.087,"15min":1.087},"max":{"1min":3.3010000000000002,"5min":3.536,"15min":3.9060000000000001},"last":1.9650000000000001}]},{"osd":1,"last update":"Sat Nov 28 22:26:30 2020","interfaces":[{"interface":"back","average":{"1min":2.2480000000000002,"5min":2.2109999999999999,"15min":2.2410000000000001},"min":{"1min":1.5249999999999999,"5min":1.264,"15min":1.264},"max":{"1min":3.3220000000000001,"5min":4.4989999999999997,"15min":4.4989999999999997},"last":2.823},{"interface":"front","average":{"1min":2.3610000000000002,"5min":2.3439999999999999,"15min":2.395},"min":{"1min":1.3979999999999999,"5min":0.996,"15min":0.996},"max":{"1min":2.754,"5min":3.1360000000000001,"15min":3.2789999999999999},"last":1.8799999999999999}]},{"osd":2,"last update":"Sat Nov 28 22:26:30 2020","interfaces":[{"interface":"back","average":{"1min":2.024,"5min":2.1240000000000001,"15min":2.1339999999999999},"min":{"1min":1.5609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"max":{"1min":3.1789999999999998,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":1.7609999999999999},{"interface":"front","average":{"1min":2.512,"5min":2.3199999999999998,"15min":2.355},"min":{"1min":1.694,"5min":1.355,"15min":1.103},"max":{"1min":3.899,"5min":4.3680000000000003,"15min":4.3680000000000003},"last":2.3340000000000001}]},{"osd":3,"last update":"Sat Nov 28 22:26:30 2020","interfaces":[{"interface":"back","average":{"1min":2.5529999999999999,"5min":2.573,"15min":2.6040000000000001},"min":{"1min":1.8360000000000001,"5min":1.6100000000000001,"15min":1.5109999999999999},"max":{"1min":3.5059999999999998,"5min":4.7329999999999997,"15min":4.7329999999999997},"last":2.5430000000000001},{"interface":"front","average":{"1min":2.4769999999999999,"5min":2.4489999999999998,"15min":2.4510000000000001},"min":{"1min":1.9299999999999999,"5min":1.665,"15min":1.3720000000000001},"max":{"1min":3.46,"5min":4.2290000000000001,"15min":4.2290000000000001},"last":2.1230000000000002}]},{"osd":4,"last update":"Sat Nov 28 22:26:30 2020","interfaces":[{"interface":"back","average":{"1min":2.5009999999999999,"5min":2.3980000000000001,"15min":2.4199999999999999},"min":{"1min":2.0219999999999998,"5min":1.6759999999999999,"15min":1.121},"max":{"1min":3.3559999999999999,"5min":3.734,"15min":3.734},"last":1.6899999999999999},{"interface":"front","average":{"1min":2.6219999999999999,"5min":2.8260000000000001,"15min":2.831},"min":{"1min":1.752,"5min":1.218,"15min":1.218},"max":{"1min":4.1079999999999997,"5min":5.5869999999999997,"15min":5.5869999999999997},"last":2.7280000000000002}]},{"osd":5,"last update":"Sat Nov 28 22:26:30 2020","interfaces":[{"interface":"back","average":{"1min":2.9289999999999998,"5min":2.847,"15min":2.8370000000000002},"min":{"1min":2.2989999999999999,"5min":1.758,"15min":1.6839999999999999},"max":{"1min":3.7000000000000002,"5min":5.2210000000000001,"15min":5.2210000000000001},"last":2.468},{"interface":"front","average":{"1min":2.633,"5min":2.7629999999999999,"15min":2.8260000000000001},"min":{"1min":1.76,"5min":1.76,"15min":1.47},"max":{"1min":3.597,"5min":4.6760000000000002,"15min":5.3120000000000003},"last":2.2770000000000001}]},{"osd":7,"last update":"Sat Nov 28 22:26:30 2020","interfaces":[{"interface":"back","average":{"1min":3.0070000000000001,"5min":3.141,"15min":3.173},"min":{"1min":2.0859999999999999,"5min":1.8460000000000001,"15min":1.7729999999999999},"max":{"1min":4.3300000000000001,"5min":5.298,"15min":5.3620000000000001},"last":3.7679999999999998},{"interface":"front","average":{"1min":3.0640000000000001,"5min":3.1480000000000001,"15min":3.1560000000000001},"min":{"1min":2.5950000000000002,"5min":2.1960000000000002,"15min":1.6930000000000001},"max":{"1min":3.8700000000000001,"5min":5.4740000000000002,"15min":5.4740000000000002},"last":3.1749999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542636,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050608,"kb_used_data":2024,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321232,"statfs":{"total":96636764160,"available":95560941568,"internally_reserved":1073741824,"allocated":2072576,"data_stored":959022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 28 22:26:42 2020","interfaces":[{"interface":"back","average":{"1min":1.51,"5min":1.589,"15min":1.5940000000000001},"min":{"1min":1.22,"5min":1.22,"15min":1.22},"max":{"1min":1.8740000000000001,"5min":2.3570000000000002,"15min":2.802},"last":1.7330000000000001},{"interface":"front","average":{"1min":1.724,"5min":1.8149999999999999,"15min":1.768},"min":{"1min":1.3700000000000001,"5min":1.3340000000000001,"15min":1.127},"max":{"1min":1.9870000000000001,"5min":3.8740000000000001,"15min":3.8740000000000001},"last":1.5269999999999999}]},{"osd":2,"last update":"Sat Nov 28 22:26:42 2020","interfaces":[{"interface":"back","average":{"1min":2.0310000000000001,"5min":1.883,"15min":1.837},"min":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.0620000000000001},"max":{"1min":3.2629999999999999,"5min":4.1929999999999996,"15min":4.1929999999999996},"last":1.5920000000000001},{"interface":"front","average":{"1min":2.4809999999999999,"5min":2.3690000000000002,"15min":2.347},"min":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.3169999999999999},"max":{"1min":3.0089999999999999,"5min":3.2799999999999998,"15min":3.3050000000000002},"last":2.4849999999999999}]},{"osd":3,"last update":"Sat Nov 28 22:26:42 2020","interfaces":[{"interface":"back","average":{"1min":2.0870000000000002,"5min":2.1459999999999999,"15min":2.1070000000000002},"min":{"1min":1.74,"5min":1.575,"15min":1.2210000000000001},"max":{"1min":2.75,"5min":3.2160000000000002,"15min":3.2160000000000002},"last":2.1850000000000001},{"interface":"front","average":{"1min":2.391,"5min":2.4460000000000002,"15min":2.4129999999999998},"min":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5189999999999999},"max":{"1min":3.3010000000000002,"5min":3.5680000000000001,"15min":3.5680000000000001},"last":2.8780000000000001}]},{"osd":4,"last update":"Sat Nov 28 22:26:42 2020","interfaces":[{"interface":"back","average":{"1min":2.383,"5min":2.4020000000000001,"15min":2.3860000000000001},"min":{"1min":1.528,"5min":1.528,"15min":1.528},"max":{"1min":2.7719999999999998,"5min":3.6680000000000001,"15min":3.73},"last":2.0950000000000002},{"interface":"front","average":{"1min":2.3730000000000002,"5min":2.347,"15min":2.3730000000000002},"min":{"1min":1.7589999999999999,"5min":1.554,"15min":1.554},"max":{"1min":3.379,"5min":3.802,"15min":3.802},"last":2.3199999999999998}]},{"osd":5,"last update":"Sat Nov 28 22:26:42 2020","interfaces":[{"interface":"back","average":{"1min":2.6549999999999998,"5min":2.746,"15min":2.706},"min":{"1min":1.964,"5min":1.964,"15min":1.573},"max":{"1min":3.3980000000000001,"5min":4.0279999999999996,"15min":4.0279999999999996},"last":2.536},{"interface":"front","average":{"1min":2.645,"5min":2.698,"15min":2.7389999999999999},"min":{"1min":2.077,"5min":1.95,"15min":1.635},"max":{"1min":3.4620000000000002,"5min":3.891,"15min":4.1980000000000004},"last":2.7330000000000001}]},{"osd":6,"last update":"Sat Nov 28 22:26:42 2020","interfaces":[{"interface":"back","average":{"1min":2.9700000000000002,"5min":3.0009999999999999,"15min":2.9870000000000001},"min":{"1min":2.3650000000000002,"5min":1.9570000000000001,"15min":1.9570000000000001},"max":{"1min":3.8439999999999999,"5min":4.5119999999999996,"15min":4.5119999999999996},"last":3.1419999999999999},{"interface":"front","average":{"1min":2.7519999999999998,"5min":2.8460000000000001,"15min":2.831},"min":{"1min":2.294,"5min":2.294,"15min":2.0590000000000002},"max":{"1min":3.1859999999999999,"5min":4.266,"15min":4.266},"last":3.0470000000000002}]},{"osd":7,"last update":"Sat Nov 28 22:26:42 2020","interfaces":[{"interface":"back","average":{"1min":3.0129999999999999,"5min":3.1230000000000002,"15min":3.0910000000000002},"min":{"1min":2.5169999999999999,"5min":2.5070000000000001,"15min":2.2770000000000001},"max":{"1min":3.6669999999999998,"5min":4.407,"15min":4.407},"last":3.266},{"interface":"front","average":{"1min":3.0339999999999998,"5min":3.198,"15min":3.1819999999999999},"min":{"1min":2.589,"5min":2.3580000000000001,"15min":2.3580000000000001},"max":{"1min":3.6629999999999998,"5min":5.3899999999999997,"15min":5.3899999999999997},"last":3.448}]}]},{"osd":0,"up_from":14,"seq":60129542635,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050608,"kb_used_data":2024,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321232,"statfs":{"total":96636764160,"available":95560941568,"internally_reserved":1073741824,"allocated":2072576,"data_stored":959022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Nov 28 22:26:34 2020","interfaces":[{"interface":"back","average":{"1min":1.6100000000000001,"5min":1.7250000000000001,"15min":1.696},"min":{"1min":1.339,"5min":1.244,"15min":1.2170000000000001},"max":{"1min":2.3399999999999999,"5min":3.2120000000000002,"15min":3.3180000000000001},"last":1.722},{"interface":"front","average":{"1min":1.6719999999999999,"5min":1.7430000000000001,"15min":1.746},"min":{"1min":1.4950000000000001,"5min":1.3400000000000001,"15min":1.302},"max":{"1min":1.901,"5min":2.9700000000000002,"15min":3.0329999999999999},"last":1.847}]},{"osd":2,"last update":"Sat Nov 28 22:26:34 2020","interfaces":[{"interface":"back","average":{"1min":2.1259999999999999,"5min":2.113,"15min":2.1040000000000001},"min":{"1min":1.411,"5min":1.3819999999999999,"15min":1.3819999999999999},"max":{"1min":2.6600000000000001,"5min":3.3239999999999998,"15min":3.3340000000000001},"last":2.456},{"interface":"front","average":{"1min":2.4929999999999999,"5min":2.4550000000000001,"15min":2.4609999999999999},"min":{"1min":2.1230000000000002,"5min":1.25,"15min":1.25},"max":{"1min":2.8599999999999999,"5min":3.3610000000000002,"15min":3.4540000000000002},"last":2.8580000000000001}]},{"osd":3,"last update":"Sat Nov 28 22:26:34 2020","interfaces":[{"interface":"back","average":{"1min":1.833,"5min":1.925,"15min":1.917},"min":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.276},"max":{"1min":2.528,"5min":3.3500000000000001,"15min":3.3500000000000001},"last":2.2490000000000001},{"interface":"front","average":{"1min":2.4009999999999998,"5min":2.4289999999999998,"15min":2.387},"min":{"1min":1.6739999999999999,"5min":1.653,"15min":1.512},"max":{"1min":2.899,"5min":3.7290000000000001,"15min":3.7290000000000001},"last":2.919}]},{"osd":4,"last update":"Sat Nov 28 22:26:34 2020","interfaces":[{"interface":"back","average":{"1min":2.2490000000000001,"5min":2.363,"15min":2.3079999999999998},"min":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.478},"max":{"1min":3.2919999999999998,"5min":3.8809999999999998,"15min":3.8809999999999998},"last":2.5110000000000001},{"interface":"front","average":{"1min":2.2309999999999999,"5min":2.375,"15min":2.2970000000000002},"min":{"1min":1.7829999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"max":{"1min":2.8940000000000001,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":3.0499999999999998}]},{"osd":5,"last update":"Sat Nov 28 22:26:34 2020","interfaces":[{"interface":"back","average":{"1min":2.8620000000000001,"5min":2.899,"15min":2.8210000000000002},"min":{"1min":2.0089999999999999,"5min":1.8280000000000001,"15min":1.764},"max":{"1min":3.661,"5min":4.4480000000000004,"15min":4.4480000000000004},"last":3.319},{"interface":"front","average":{"1min":2.8149999999999999,"5min":2.9279999999999999,"15min":2.8559999999999999},"min":{"1min":2.1379999999999999,"5min":2.1379999999999999,"15min":1.6990000000000001},"max":{"1min":3.2400000000000002,"5min":4.3280000000000003,"15min":4.3280000000000003},"last":3.6099999999999999}]},{"osd":6,"last update":"Sat Nov 28 22:26:34 2020","interfaces":[{"interface":"back","average":{"1min":2.6499999999999999,"5min":2.7810000000000001,"15min":2.706},"min":{"1min":2.1230000000000002,"5min":1.982,"15min":1.732},"max":{"1min":3.1160000000000001,"5min":3.8170000000000002,"15min":3.8170000000000002},"last":2.641},{"interface":"front","average":{"1min":3.0449999999999999,"5min":3.1320000000000001,"15min":3.069},"min":{"1min":2.2330000000000001,"5min":2.2330000000000001,"15min":2.0190000000000001},"max":{"1min":3.5129999999999999,"5min":4.5750000000000002,"15min":4.5750000000000002},"last":3.6840000000000002}]},{"osd":7,"last update":"Sat Nov 28 22:26:34 2020","interfaces":[{"interface":"back","average":{"1min":3.069,"5min":3.1440000000000001,"15min":3.1120000000000001},"min":{"1min":2.593,"5min":2.4900000000000002,"15min":1.8340000000000001},"max":{"1min":3.4849999999999999,"5min":4.7119999999999997,"15min":4.7119999999999997},"last":3.2360000000000002},{"interface":"front","average":{"1min":3.044,"5min":3.0859999999999999,"15min":3.008},"min":{"1min":2.4609999999999999,"5min":2.294,"15min":2.294},"max":{"1min":3.3940000000000001,"5min":4.0499999999999998,"15min":4.1660000000000004},"last":3.4540000000000002}]}]},{"osd":2,"up_from":14,"seq":60129542635,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050608,"kb_used_data":2024,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321232,"statfs":{"total":96636764160,"available":95560941568,"internally_reserved":1073741824,"allocated":2072576,"data_stored":959022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 28 22:26:58 2020","interfaces":[{"interface":"back","average":{"1min":1.6479999999999999,"5min":1.6799999999999999,"15min":1.6619999999999999},"min":{"1min":1.446,"5min":1.343,"15min":1.1200000000000001},"max":{"1min":2.0339999999999998,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":2.2789999999999999},{"interface":"front","average":{"1min":1.8560000000000001,"5min":1.827,"15min":1.8149999999999999},"min":{"1min":1.5589999999999999,"5min":1.3680000000000001,"15min":1.3340000000000001},"max":{"1min":2.6480000000000001,"5min":3.1720000000000002,"15min":3.3570000000000002},"last":1.5}]},{"osd":1,"last update":"Sat Nov 28 22:26:58 2020","interfaces":[{"interface":"back","average":{"1min":1.8300000000000001,"5min":1.877,"15min":1.911},"min":{"1min":1.4330000000000001,"5min":1.3919999999999999,"15min":1.3460000000000001},"max":{"1min":2.3029999999999999,"5min":3.802,"15min":3.802},"last":1.9299999999999999},{"interface":"front","average":{"1min":2.3170000000000002,"5min":2.3849999999999998,"15min":2.3730000000000002},"min":{"1min":1.6739999999999999,"5min":1.528,"15min":1.4670000000000001},"max":{"1min":3.6110000000000002,"5min":3.6110000000000002,"15min":3.6110000000000002},"last":3.1179999999999999}]},{"osd":3,"last update":"Sat Nov 28 22:26:58 2020","interfaces":[{"interface":"back","average":{"1min":2.1659999999999999,"5min":2.2240000000000002,"15min":2.2010000000000001},"min":{"1min":1.4850000000000001,"5min":1.4490000000000001,"15min":1.361},"max":{"1min":2.964,"5min":3.5459999999999998,"15min":3.5459999999999998},"last":2.3679999999999999},{"interface":"front","average":{"1min":2.2069999999999999,"5min":2.2789999999999999,"15min":2.2629999999999999},"min":{"1min":1.589,"5min":1.589,"15min":1.5409999999999999},"max":{"1min":2.9860000000000002,"5min":3.0640000000000001,"15min":3.4620000000000002},"last":2.6640000000000001}]},{"osd":4,"last update":"Sat Nov 28 22:26:58 2020","interfaces":[{"interface":"back","average":{"1min":2.5310000000000001,"5min":2.4580000000000002,"15min":2.387},"min":{"1min":1.8120000000000001,"5min":1.7490000000000001,"15min":1.645},"max":{"1min":3.5369999999999999,"5min":4.1159999999999997,"15min":4.1159999999999997},"last":1.79},{"interface":"front","average":{"1min":2.633,"5min":2.6459999999999999,"15min":2.6240000000000001},"min":{"1min":1.8360000000000001,"5min":1.76,"15min":1.595},"max":{"1min":3.7770000000000001,"5min":3.7770000000000001,"15min":3.9510000000000001},"last":2.0259999999999998}]},{"osd":5,"last update":"Sat Nov 28 22:26:58 2020","interfaces":[{"interface":"back","average":{"1min":2.8769999999999998,"5min":2.8759999999999999,"15min":2.8860000000000001},"min":{"1min":2.0259999999999998,"5min":1.986,"15min":1.8169999999999999},"max":{"1min":3.8090000000000002,"5min":3.9100000000000001,"15min":3.9159999999999999},"last":2.6040000000000001},{"interface":"front","average":{"1min":2.7989999999999999,"5min":2.8769999999999998,"15min":2.8740000000000001},"min":{"1min":2.1240000000000001,"5min":1.925,"15min":1.925},"max":{"1min":3.9500000000000002,"5min":3.9500000000000002,"15min":4.1520000000000001},"last":2.2149999999999999}]},{"osd":6,"last update":"Sat Nov 28 22:26:58 2020","interfaces":[{"interface":"back","average":{"1min":3.069,"5min":3.0720000000000001,"15min":3.0950000000000002},"min":{"1min":2.3420000000000001,"5min":2.2879999999999998,"15min":2.1150000000000002},"max":{"1min":3.7730000000000001,"5min":3.9580000000000002,"15min":3.9580000000000002},"last":2.4399999999999999},{"interface":"front","average":{"1min":3.0289999999999999,"5min":2.9929999999999999,"15min":2.9660000000000002},"min":{"1min":2.2189999999999999,"5min":2.2189999999999999,"15min":2.1349999999999998},"max":{"1min":3.919,"5min":4.3150000000000004,"15min":4.3150000000000004},"last":2.7189999999999999}]},{"osd":7,"last update":"Sat Nov 28 22:26:58 2020","interfaces":[{"interface":"back","average":{"1min":3.1840000000000002,"5min":3.226,"15min":3.2130000000000001},"min":{"1min":2.5590000000000002,"5min":2.5150000000000001,"15min":2.0569999999999999},"max":{"1min":4.133,"5min":4.1980000000000004,"15min":4.1980000000000004},"last":2.847},{"interface":"front","average":{"1min":3.2250000000000001,"5min":3.2949999999999999,"15min":3.3250000000000002},"min":{"1min":2.7440000000000002,"5min":2.6230000000000002,"15min":2.1110000000000002},"max":{"1min":3.7909999999999999,"5min":4.0209999999999999,"15min":4.3289999999999997},"last":2.9249999999999998}]}]},{"osd":3,"up_from":14,"seq":60129542635,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050608,"kb_used_data":2024,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321232,"statfs":{"total":96636764160,"available":95560941568,"internally_reserved":1073741824,"allocated":2072576,"data_stored":959022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 28 22:26:36 2020","interfaces":[{"interface":"back","average":{"1min":1.742,"5min":1.6879999999999999,"15min":1.6919999999999999},"min":{"1min":1.355,"5min":1.258,"15min":1.143},"max":{"1min":2.1040000000000001,"5min":2.5409999999999999,"15min":2.9100000000000001},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.665,"5min":1.694,"15min":1.6699999999999999},"min":{"1min":1.4770000000000001,"5min":1.375,"15min":1.2490000000000001},"max":{"1min":1.9490000000000001,"5min":2.476,"15min":2.476},"last":1.976}]},{"osd":1,"last update":"Sat Nov 28 22:26:36 2020","interfaces":[{"interface":"back","average":{"1min":1.595,"5min":1.7529999999999999,"15min":1.776},"min":{"1min":1.4630000000000001,"5min":1.377,"15min":1.3140000000000001},"max":{"1min":1.829,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.982,"5min":2.077,"15min":2.1040000000000001},"min":{"1min":1.569,"5min":1.5389999999999999,"15min":1.4390000000000001},"max":{"1min":2.5990000000000002,"5min":2.806,"15min":3.0990000000000002},"last":1.7070000000000001}]},{"osd":2,"last update":"Sat Nov 28 22:26:36 2020","interfaces":[{"interface":"back","average":{"1min":2.2429999999999999,"5min":2.2410000000000001,"15min":2.2679999999999998},"min":{"1min":1.8420000000000001,"5min":1.5089999999999999,"15min":1.3700000000000001},"max":{"1min":2.9119999999999999,"5min":3.4900000000000002,"15min":3.4900000000000002},"last":1.8979999999999999},{"interface":"front","average":{"1min":2.589,"5min":2.5350000000000001,"15min":2.609},"min":{"1min":2.097,"5min":2.0600000000000001,"15min":1.8340000000000001},"max":{"1min":3.234,"5min":3.4079999999999999,"15min":4.3289999999999997},"last":2.0649999999999999}]},{"osd":4,"last update":"Sat Nov 28 22:26:36 2020","interfaces":[{"interface":"back","average":{"1min":2.331,"5min":2.2490000000000001,"15min":2.331},"min":{"1min":1.9850000000000001,"5min":1.6559999999999999,"15min":1.6559999999999999},"max":{"1min":3.0649999999999999,"5min":3.0790000000000002,"15min":4.0519999999999996},"last":1.381},{"interface":"front","average":{"1min":2.6269999999999998,"5min":2.6259999999999999,"15min":2.5950000000000002},"min":{"1min":2.02,"5min":1.506,"15min":1.506},"max":{"1min":3.472,"5min":3.6779999999999999,"15min":4.0140000000000002},"last":2.4049999999999998}]},{"osd":5,"last update":"Sat Nov 28 22:26:36 2020","interfaces":[{"interface":"back","average":{"1min":2.903,"5min":2.8079999999999998,"15min":2.8330000000000002},"min":{"1min":2.2690000000000001,"5min":1.9179999999999999,"15min":1.8979999999999999},"max":{"1min":3.6139999999999999,"5min":3.7170000000000001,"15min":4.1699999999999999},"last":2.77},{"interface":"front","average":{"1min":2.8050000000000002,"5min":2.8719999999999999,"15min":2.9399999999999999},"min":{"1min":2.1909999999999998,"5min":2.181,"15min":1.96},"max":{"1min":3.5249999999999999,"5min":3.6949999999999998,"15min":4.085},"last":2.4809999999999999}]},{"osd":6,"last update":"Sat Nov 28 22:26:36 2020","interfaces":[{"interface":"back","average":{"1min":3.0470000000000002,"5min":3.0579999999999998,"15min":3.1579999999999999},"min":{"1min":2.1800000000000002,"5min":2.1800000000000002,"15min":2.173},"max":{"1min":3.7679999999999998,"5min":3.9169999999999998,"15min":4.4980000000000002},"last":2.8460000000000001},{"interface":"front","average":{"1min":2.8860000000000001,"5min":2.8370000000000002,"15min":2.8719999999999999},"min":{"1min":2.2330000000000001,"5min":2.2330000000000001,"15min":2.1299999999999999},"max":{"1min":3.5150000000000001,"5min":3.5150000000000001,"15min":4.8819999999999997},"last":2.2309999999999999}]},{"osd":7,"last update":"Sat Nov 28 22:26:36 2020","interfaces":[{"interface":"back","average":{"1min":3.0139999999999998,"5min":3.0339999999999998,"15min":3.0720000000000001},"min":{"1min":2.6230000000000002,"5min":2.5049999999999999,"15min":2.3250000000000002},"max":{"1min":3.5289999999999999,"5min":3.6899999999999999,"15min":3.9420000000000002},"last":2.585},{"interface":"front","average":{"1min":3.149,"5min":3.1760000000000002,"15min":3.1680000000000001},"min":{"1min":2.5899999999999999,"5min":2.4279999999999999,"15min":2.1989999999999998},"max":{"1min":3.8559999999999999,"5min":3.8559999999999999,"15min":4.5880000000000001},"last":3.0539999999999998}]}]},{"osd":4,"up_from":14,"seq":60129542635,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050608,"kb_used_data":2024,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321232,"statfs":{"total":96636764160,"available":95560941568,"internally_reserved":1073741824,"allocated":2072576,"data_stored":959022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 28 22:26:23 2020","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.754,"15min":1.748},"min":{"1min":1.232,"5min":1.232,"15min":1.012},"max":{"1min":3.0089999999999999,"5min":3.0249999999999999,"15min":3.2549999999999999},"last":1.367},{"interface":"front","average":{"1min":1.794,"5min":1.871,"15min":1.8089999999999999},"min":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.0980000000000001},"max":{"1min":2.8780000000000001,"5min":3.294,"15min":3.452},"last":1.5529999999999999}]},{"osd":1,"last update":"Sat Nov 28 22:26:23 2020","interfaces":[{"interface":"back","average":{"1min":2.0790000000000002,"5min":2.1360000000000001,"15min":2.2160000000000002},"min":{"1min":1.5209999999999999,"5min":1.3080000000000001,"15min":1.284},"max":{"1min":2.7879999999999998,"5min":3.3159999999999998,"15min":3.6720000000000002},"last":1.952},{"interface":"front","average":{"1min":2.032,"5min":2.0750000000000002,"15min":2.1120000000000001},"min":{"1min":1.498,"5min":1.369,"15min":1.0469999999999999},"max":{"1min":2.7839999999999998,"5min":2.9910000000000001,"15min":3.2570000000000001},"last":2.0129999999999999}]},{"osd":2,"last update":"Sat Nov 28 22:26:23 2020","interfaces":[{"interface":"back","average":{"1min":1.804,"5min":1.9330000000000001,"15min":1.9850000000000001},"min":{"1min":1.391,"5min":1.391,"15min":1.391},"max":{"1min":2.4700000000000002,"5min":2.9300000000000002,"15min":3.5880000000000001},"last":1.6479999999999999},{"interface":"front","average":{"1min":2.4540000000000002,"5min":2.4729999999999999,"15min":2.5089999999999999},"min":{"1min":1.887,"5min":1.7210000000000001,"15min":1.419},"max":{"1min":3.169,"5min":3.7919999999999998,"15min":3.7919999999999998},"last":2.294}]},{"osd":3,"last update":"Sat Nov 28 22:26:23 2020","interfaces":[{"interface":"back","average":{"1min":2.399,"5min":2.3679999999999999,"15min":2.4220000000000002},"min":{"1min":1.8080000000000001,"5min":1.431,"15min":1.407},"max":{"1min":3.161,"5min":3.3620000000000001,"15min":3.5310000000000001},"last":2.395},{"interface":"front","average":{"1min":2.1389999999999998,"5min":2.2709999999999999,"15min":2.331},"min":{"1min":1.669,"5min":1.6579999999999999,"15min":1.246},"max":{"1min":2.7629999999999999,"5min":3.8759999999999999,"15min":3.9580000000000002},"last":2.1640000000000001}]},{"osd":5,"last update":"Sat Nov 28 22:26:23 2020","interfaces":[{"interface":"back","average":{"1min":2.665,"5min":2.6869999999999998,"15min":2.6909999999999998},"min":{"1min":1.8260000000000001,"5min":1.3819999999999999,"15min":1.3819999999999999},"max":{"1min":3.5369999999999999,"5min":4.1280000000000001,"15min":4.1280000000000001},"last":3.012},{"interface":"front","average":{"1min":2.657,"5min":2.7309999999999999,"15min":2.7810000000000001},"min":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"max":{"1min":3.6600000000000001,"5min":3.6629999999999998,"15min":4.2350000000000003},"last":2.7400000000000002}]},{"osd":6,"last update":"Sat Nov 28 22:26:23 2020","interfaces":[{"interface":"back","average":{"1min":2.5960000000000001,"5min":2.6709999999999998,"15min":2.6480000000000001},"min":{"1min":2.0059999999999998,"5min":1.1599999999999999,"15min":1.1599999999999999},"max":{"1min":3.1949999999999998,"5min":4.0800000000000001,"15min":4.351},"last":2.4609999999999999},{"interface":"front","average":{"1min":2.984,"5min":3.0310000000000001,"15min":3.0379999999999998},"min":{"1min":2.1659999999999999,"5min":2.1219999999999999,"15min":1.8600000000000001},"max":{"1min":3.8959999999999999,"5min":4.5270000000000001,"15min":4.5270000000000001},"last":2.6539999999999999}]},{"osd":7,"last update":"Sat Nov 28 22:26:23 2020","interfaces":[{"interface":"back","average":{"1min":3.089,"5min":3.1059999999999999,"15min":3.1389999999999998},"min":{"1min":2.2109999999999999,"5min":2.2109999999999999,"15min":1.591},"max":{"1min":3.5129999999999999,"5min":4.0229999999999997,"15min":4.2149999999999999},"last":3.0990000000000002},{"interface":"front","average":{"1min":2.8809999999999998,"5min":3.0720000000000001,"15min":3.1040000000000001},"min":{"1min":2.153,"5min":1.319,"15min":1.319},"max":{"1min":3.3929999999999998,"5min":4.2709999999999999,"15min":5.133},"last":3.1589999999999998}]}]},{"osd":5,"up_from":14,"seq":60129542635,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050604,"kb_used_data":2020,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321236,"statfs":{"total":96636764160,"available":95560945664,"internally_reserved":1073741824,"allocated":2068480,"data_stored":958419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 28 22:26:29 2020","interfaces":[{"interface":"back","average":{"1min":2.2799999999999998,"5min":2.0899999999999999,"15min":2.0489999999999999},"min":{"1min":1.321,"5min":1.0609999999999999,"15min":1.0609999999999999},"max":{"1min":2.9140000000000001,"5min":2.9889999999999999,"15min":3.4780000000000002},"last":1.3959999999999999},{"interface":"front","average":{"1min":2.5529999999999999,"5min":2.4079999999999999,"15min":2.375},"min":{"1min":2.0059999999999998,"5min":1.5209999999999999,"15min":1.5209999999999999},"max":{"1min":3.1190000000000002,"5min":4.1390000000000002,"15min":4.6390000000000002},"last":2.077}]},{"osd":1,"last update":"Sat Nov 28 22:26:29 2020","interfaces":[{"interface":"back","average":{"1min":1.9319999999999999,"5min":2.0489999999999999,"15min":2.1440000000000001},"min":{"1min":1.4319999999999999,"5min":1.286,"15min":1.266},"max":{"1min":3.0590000000000002,"5min":3.4729999999999999,"15min":4.4909999999999997},"last":1.976},{"interface":"front","average":{"1min":2.4700000000000002,"5min":2.6429999999999998,"15min":2.649},"min":{"1min":1.446,"5min":1.446,"15min":1.446},"max":{"1min":3.4060000000000001,"5min":4.7960000000000003,"15min":5.0449999999999999},"last":2.5539999999999998}]},{"osd":2,"last update":"Sat Nov 28 22:26:29 2020","interfaces":[{"interface":"back","average":{"1min":2.0369999999999999,"5min":1.9330000000000001,"15min":1.9830000000000001},"min":{"1min":1.6000000000000001,"5min":1.4570000000000001,"15min":1.367},"max":{"1min":3.04,"5min":3.0569999999999999,"15min":3.8260000000000001},"last":1.897},{"interface":"front","average":{"1min":2.1440000000000001,"5min":2.363,"15min":2.3570000000000002},"min":{"1min":1.6930000000000001,"5min":1.4410000000000001,"15min":1.381},"max":{"1min":3.2730000000000001,"5min":3.677,"15min":4.742},"last":2.3799999999999999}]},{"osd":3,"last update":"Sat Nov 28 22:26:29 2020","interfaces":[{"interface":"back","average":{"1min":1.881,"5min":2.0640000000000001,"15min":2.0550000000000002},"min":{"1min":1.5680000000000001,"5min":1.363,"15min":1.2609999999999999},"max":{"1min":2.4980000000000002,"5min":4.532,"15min":5.2469999999999999},"last":1.7669999999999999},{"interface":"front","average":{"1min":2.4780000000000002,"5min":2.4660000000000002,"15min":2.4929999999999999},"min":{"1min":1.879,"5min":1.7669999999999999,"15min":1.427},"max":{"1min":3.7280000000000002,"5min":3.7280000000000002,"15min":4.5979999999999999},"last":2.4670000000000001}]},{"osd":4,"last update":"Sat Nov 28 22:26:29 2020","interfaces":[{"interface":"back","average":{"1min":2.3180000000000001,"5min":2.4769999999999999,"15min":2.4910000000000001},"min":{"1min":1.3480000000000001,"5min":1.157,"15min":1.0580000000000001},"max":{"1min":3.2909999999999999,"5min":4.0350000000000001,"15min":4.335},"last":2.8380000000000001},{"interface":"front","average":{"1min":2.468,"5min":2.6469999999999998,"15min":2.6850000000000001},"min":{"1min":1.7769999999999999,"5min":1.7110000000000001,"15min":0.94699999999999995},"max":{"1min":3.476,"5min":4.1749999999999998,"15min":5.1900000000000004},"last":2.7639999999999998}]},{"osd":6,"last update":"Sat Nov 28 22:26:29 2020","interfaces":[{"interface":"back","average":{"1min":2.347,"5min":2.5179999999999998,"15min":2.5739999999999998},"min":{"1min":1.7809999999999999,"5min":1.1930000000000001,"15min":1.107},"max":{"1min":3.1829999999999998,"5min":4.9710000000000001,"15min":5.431},"last":2.9129999999999998},{"interface":"front","average":{"1min":2.879,"5min":2.9449999999999998,"15min":2.9710000000000001},"min":{"1min":2.1669999999999998,"5min":1.3999999999999999,"15min":1.3460000000000001},"max":{"1min":3.9300000000000002,"5min":5.4169999999999998,"15min":5.4169999999999998},"last":3.048}]},{"osd":7,"last update":"Sat Nov 28 22:26:29 2020","interfaces":[{"interface":"back","average":{"1min":2.9209999999999998,"5min":2.9969999999999999,"15min":3.036},"min":{"1min":2.3260000000000001,"5min":1.6499999999999999,"15min":1.4710000000000001},"max":{"1min":3.6589999999999998,"5min":4.1799999999999997,"15min":4.9829999999999997},"last":3.1389999999999998},{"interface":"front","average":{"1min":3.218,"5min":3.1179999999999999,"15min":3.157},"min":{"1min":2.3719999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"max":{"1min":4.1299999999999999,"5min":4.3029999999999999,"15min":5.319},"last":3.2389999999999999}]}]}]}} 2020-11-28T22:27:08.710 INFO:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2020-11-28T22:27:08.822 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.815+0000 7f299b59e700 1 Processor -- start 2020-11-28T22:27:08.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.815+0000 7f299b59e700 1 -- start start 2020-11-28T22:27:08.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.815+0000 7f299b59e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f299c0b2e80 0x7f299c05f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:27:08.823 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.815+0000 7f299b59e700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f299c0b2560 0x7f299c05f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:27:08.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.815+0000 7f299b59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f299c0acb40 0x7f299c060d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-28T22:27:08.824 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.815+0000 7f299b59e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f299c05fd60 con 0x7f299c0b2e80 2020-11-28T22:27:08.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.815+0000 7f299b59e700 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f299c0ab0b0 con 0x7f299c0b2560 2020-11-28T22:27:08.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.815+0000 7f299b59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f299c0b3260 con 0x7f299c0acb40 2020-11-28T22:27:08.825 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.815+0000 7f299a59c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f299c0b2560 0x7f299c05f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:27:08.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.815+0000 7f299a59c700 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f299c0b2560 0x7f299c05f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47066/0 (socket says 172.21.15.162:47066) 2020-11-28T22:27:08.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.815+0000 7f299a59c700 1 -- 172.21.15.162:0/4089136975 learned_addr learned my addr 172.21.15.162:0/4089136975 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T22:27:08.826 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.815+0000 7f299ad9d700 1 --2- 172.21.15.162:0/4089136975 >> v2:172.21.15.118:3300/0 conn(0x7f299c0b2e80 0x7f299c05f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:27:08.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.815+0000 7f29a0d57700 1 --2- 172.21.15.162:0/4089136975 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f299c0acb40 0x7f299c060d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:27:08.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.815+0000 7f299a59c700 1 -- 172.21.15.162:0/4089136975 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f299c0acb40 msgr2=0x7f299c060d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T22:27:08.827 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.815+0000 7f299a59c700 1 --2- 172.21.15.162:0/4089136975 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f299c0acb40 0x7f299c060d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:27:08.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.815+0000 7f299a59c700 1 -- 172.21.15.162:0/4089136975 >> v2:172.21.15.118:3300/0 conn(0x7f299c0b2e80 msgr2=0x7f299c05f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-28T22:27:08.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.815+0000 7f299a59c700 1 --2- 172.21.15.162:0/4089136975 >> v2:172.21.15.118:3300/0 conn(0x7f299c0b2e80 0x7f299c05f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:27:08.828 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.815+0000 7f299a59c700 1 -- 172.21.15.162:0/4089136975 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f299c0ab2e0 con 0x7f299c0b2560 2020-11-28T22:27:08.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.815+0000 7f299a59c700 1 --2- 172.21.15.162:0/4089136975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f299c0b2560 0x7f299c05f870 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 rx=0x7f2990002f30 tx=0x7f2990006e80).ready entity=mon.1 client_cookie=ce9cfc708e91ee71 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:27:08.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.815+0000 7f2999d9b700 1 -- 172.21.15.162:0/4089136975 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29900097f0 con 0x7f299c0b2560 2020-11-28T22:27:08.829 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.815+0000 7f2999d9b700 1 -- 172.21.15.162:0/4089136975 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2990009950 con 0x7f299c0b2560 2020-11-28T22:27:08.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.815+0000 7f2999d9b700 1 -- 172.21.15.162:0/4089136975 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2990009cb0 con 0x7f299c0b2560 2020-11-28T22:27:08.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.815+0000 7f299b59e700 1 -- 172.21.15.162:0/4089136975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f299c0b2560 msgr2=0x7f299c05f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:27:08.830 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.815+0000 7f299b59e700 1 --2- 172.21.15.162:0/4089136975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f299c0b2560 0x7f299c05f870 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 rx=0x7f2990002f30 tx=0x7f2990006e80).stop 2020-11-28T22:27:08.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.819+0000 7f299b59e700 1 -- 172.21.15.162:0/4089136975 shutdown_connections 2020-11-28T22:27:08.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.819+0000 7f299b59e700 1 --2- 172.21.15.162:0/4089136975 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f299c0acb40 0x7f299c060d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:27:08.831 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.819+0000 7f299b59e700 1 --2- 172.21.15.162:0/4089136975 >> v2:172.21.15.118:3300/0 conn(0x7f299c0b2e80 0x7f299c05f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-28T22:27:08.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.819+0000 7f299b59e700 1 --2- 172.21.15.162:0/4089136975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f299c0b2560 0x7f299c05f870 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:27:08.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.819+0000 7f299b59e700 1 -- 172.21.15.162:0/4089136975 >> 172.21.15.162:0/4089136975 conn(0x7f299c063b60 msgr2=0x7f299c063f40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:27:08.832 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.819+0000 7f299b59e700 1 -- 172.21.15.162:0/4089136975 shutdown_connections 2020-11-28T22:27:08.833 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.819+0000 7f299b59e700 1 -- 172.21.15.162:0/4089136975 wait complete. 2020-11-28T22:27:08.836 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.819+0000 7f299b59e700 1 Processor -- start 2020-11-28T22:27:08.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.819+0000 7f299b59e700 1 -- start start 2020-11-28T22:27:08.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.819+0000 7f299b59e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f299c0acb40 0x7f299c0b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:27:08.837 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.819+0000 7f299b59e700 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f299c0b2560 0x7f299c0667a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:27:08.838 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.819+0000 7f299b59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f299c0b2e80 0x7f299c066c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:27:08.838 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.819+0000 7f299b59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299c144520 con 0x7f299c0b2e80 2020-11-28T22:27:08.838 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.819+0000 7f299b59e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299c1447c0 con 0x7f299c0acb40 2020-11-28T22:27:08.839 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.819+0000 7f299b59e700 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f299c144a60 con 0x7f299c0b2560 2020-11-28T22:27:08.839 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.819+0000 7f299ad9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f299c0acb40 0x7f299c0b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:27:08.839 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.819+0000 7f299ad9d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f299c0acb40 0x7f299c0b4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47070/0 (socket says 172.21.15.162:47070) 2020-11-28T22:27:08.840 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.819+0000 7f299ad9d700 1 -- 172.21.15.162:0/1806750611 learned_addr learned my addr 172.21.15.162:0/1806750611 (peer_addr_for_me v2:172.21.15.162:0/0) 2020-11-28T22:27:08.840 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.819+0000 7f29a0d57700 1 --2- 172.21.15.162:0/1806750611 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f299c0b2e80 0x7f299c066c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:27:08.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.819+0000 7f299a59c700 1 --2- 172.21.15.162:0/1806750611 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f299c0b2560 0x7f299c0667a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:27:08.841 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.823+0000 7f299ad9d700 1 -- 172.21.15.162:0/1806750611 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f299c0b2560 msgr2=0x7f299c0667a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:27:08.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.823+0000 7f299ad9d700 1 --2- 172.21.15.162:0/1806750611 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f299c0b2560 0x7f299c0667a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:27:08.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.823+0000 7f299ad9d700 1 -- 172.21.15.162:0/1806750611 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f299c0b2e80 msgr2=0x7f299c066c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:27:08.842 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.823+0000 7f299ad9d700 1 --2- 172.21.15.162:0/1806750611 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f299c0b2e80 0x7f299c066c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:27:08.843 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.823+0000 7f299ad9d700 1 -- 172.21.15.162:0/1806750611 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f299c144d00 con 0x7f299c0acb40 2020-11-28T22:27:08.843 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.823+0000 7f299ad9d700 1 --2- 172.21.15.162:0/1806750611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f299c0acb40 0x7f299c0b4e30 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 rx=0x7f298c003070 tx=0x7f298c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:27:08.843 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.823+0000 7f298b7fe700 1 -- 172.21.15.162:0/1806750611 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f298c0127e0 con 0x7f299c0acb40 2020-11-28T22:27:08.844 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.823+0000 7f299b59e700 1 -- 172.21.15.162:0/1806750611 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f299c0663d0 con 0x7f299c0acb40 2020-11-28T22:27:08.844 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.823+0000 7f298b7fe700 1 -- 172.21.15.162:0/1806750611 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f298c012940 con 0x7f299c0acb40 2020-11-28T22:27:08.844 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.823+0000 7f299b59e700 1 -- 172.21.15.162:0/1806750611 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f299c1442a0 con 0x7f299c0acb40 2020-11-28T22:27:08.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.823+0000 7f298b7fe700 1 -- 172.21.15.162:0/1806750611 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f298c012ca0 con 0x7f299c0acb40 2020-11-28T22:27:08.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.823+0000 7f299b59e700 1 -- 172.21.15.162:0/1806750611 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f299c057050 con 0x7f299c0acb40 2020-11-28T22:27:08.845 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.823+0000 7f298b7fe700 1 -- 172.21.15.162:0/1806750611 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 66880+0+0 (secure 0 0 0) 0x7f298c013480 con 0x7f299c0acb40 2020-11-28T22:27:08.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.827+0000 7f298b7fe700 1 --2- 172.21.15.162:0/1806750611 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2980055440 0x7f2980057900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-28T22:27:08.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.827+0000 7f298b7fe700 1 -- 172.21.15.162:0/1806750611 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f298c06e4a0 con 0x7f299c0acb40 2020-11-28T22:27:08.846 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.827+0000 7f299a59c700 1 --2- 172.21.15.162:0/1806750611 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2980055440 0x7f2980057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-28T22:27:08.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.827+0000 7f299a59c700 1 --2- 172.21.15.162:0/1806750611 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2980055440 0x7f2980057900 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7f299001c000 tx=0x7f2990015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-28T22:27:08.847 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.831+0000 7f298b7fe700 1 -- 172.21.15.162:0/1806750611 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+134611 (secure 0 0 0) 0x7f298c044890 con 0x7f299c0acb40 2020-11-28T22:27:08.977 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.975+0000 7f299b59e700 1 -- 172.21.15.162:0/1806750611 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"} v 0) v1 -- 0x7f299c0b4800 con 0x7f299c0acb40 2020-11-28T22:27:08.986 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.983+0000 7f298b7fe700 1 -- 172.21.15.162:0/1806750611 <== mon.1 v2:172.21.15.162:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f298c044890 con 0x7f299c0acb40 2020-11-28T22:27:08.990 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.987+0000 7f298b7fe700 1 -- 172.21.15.162:0/1806750611 <== mon.1 v2:172.21.15.162:3300/0 8 ==== mon_command_ack([{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}]=0 v25) v1 ==== 123+0+0 (secure 0 0 0) 0x7f298c012e00 con 0x7f299c0acb40 2020-11-28T22:27:08.992 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.987+0000 7f299b59e700 1 -- 172.21.15.162:0/1806750611 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2980055440 msgr2=0x7f2980057900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:27:08.993 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.987+0000 7f299b59e700 1 --2- 172.21.15.162:0/1806750611 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2980055440 0x7f2980057900 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7f299001c000 tx=0x7f2990015000).stop 2020-11-28T22:27:08.993 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.987+0000 7f299b59e700 1 -- 172.21.15.162:0/1806750611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f299c0acb40 msgr2=0x7f299c0b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-28T22:27:08.993 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.987+0000 7f299b59e700 1 --2- 172.21.15.162:0/1806750611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f299c0acb40 0x7f299c0b4e30 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 rx=0x7f298c003070 tx=0x7f298c008040).stop 2020-11-28T22:27:08.994 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.987+0000 7f299b59e700 1 -- 172.21.15.162:0/1806750611 shutdown_connections 2020-11-28T22:27:08.994 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.987+0000 7f299b59e700 1 --2- 172.21.15.162:0/1806750611 >> [v2:172.21.15.162:6800/12156,v1:172.21.15.162:6801/12156] conn(0x7f2980055440 0x7f2980057900 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:27:08.994 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.987+0000 7f299b59e700 1 --2- 172.21.15.162:0/1806750611 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f299c0b2e80 0x7f299c066c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:27:08.994 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.987+0000 7f299b59e700 1 --2- 172.21.15.162:0/1806750611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f299c0acb40 0x7f299c0b4e30 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:27:08.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.987+0000 7f299b59e700 1 --2- 172.21.15.162:0/1806750611 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f299c0b2560 0x7f299c0667a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-28T22:27:08.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.987+0000 7f299b59e700 1 -- 172.21.15.162:0/1806750611 >> 172.21.15.162:0/1806750611 conn(0x7f299c063b60 msgr2=0x7f299c130190 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-28T22:27:08.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.987+0000 7f299b59e700 1 -- 172.21.15.162:0/1806750611 shutdown_connections 2020-11-28T22:27:08.995 INFO:teuthology.orchestra.run.smithi162.stderr:2020-11-28T22:27:08.987+0000 7f299b59e700 1 -- 172.21.15.162:0/1806750611 wait complete. 2020-11-28T22:27:09.005 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/contextutil.py", line 33, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks/ceph.py", line 1842, in task yield File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 172, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks/ceph_fuse.py", line 159, in task if mount.is_mounted(): File "/home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted proc.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 147, in wait greenlet.get(block=True,timeout=60) File "src/gevent/greenlet.py", line 683, in gevent._greenlet.Greenlet.get File "src/gevent/greenlet.py", line 317, in gevent._greenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 47, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 766, in gevent._greenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 316, in copy_file_to copy_to_log(src, logger, capture=stream) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 273, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2020-11-28T22:27:09.006 INFO:teuthology.misc:Shutting down mds daemons... 2020-11-28T22:27:09.007 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-11-28T22:27:09.008 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T22:27:09.019 INFO:tasks.ceph.mds.a:Stopped 2020-11-28T22:27:09.021 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2020-11-28T22:27:09.022 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T22:27:09.077 INFO:tasks.ceph.mds.c:Stopped 2020-11-28T22:27:09.079 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-11-28T22:27:09.080 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T22:27:09.095 INFO:tasks.ceph.mds.b:Stopped 2020-11-28T22:27:09.096 INFO:teuthology.misc:Shutting down osd daemons... 2020-11-28T22:27:09.097 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2020-11-28T22:27:09.099 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T22:27:09.180 INFO:tasks.ceph.osd.0:Stopped 2020-11-28T22:27:09.181 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2020-11-28T22:27:09.181 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T22:27:09.251 INFO:tasks.ceph.osd.1:Stopped 2020-11-28T22:27:09.253 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2020-11-28T22:27:09.254 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T22:27:09.334 INFO:tasks.ceph.osd.2:Stopped 2020-11-28T22:27:09.335 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2020-11-28T22:27:09.336 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T22:27:09.398 INFO:tasks.ceph.osd.3:Stopped 2020-11-28T22:27:09.400 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2020-11-28T22:27:09.401 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T22:27:09.469 INFO:tasks.ceph.osd.4:Stopped 2020-11-28T22:27:09.469 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2020-11-28T22:27:09.470 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T22:27:09.542 INFO:tasks.ceph.osd.5:Stopped 2020-11-28T22:27:09.544 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2020-11-28T22:27:09.545 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T22:27:09.605 INFO:tasks.ceph.osd.6:Stopped 2020-11-28T22:27:09.606 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2020-11-28T22:27:09.607 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T22:27:09.681 INFO:tasks.ceph.osd.7:Stopped 2020-11-28T22:27:09.681 INFO:teuthology.misc:Shutting down mgr daemons... 2020-11-28T22:27:09.682 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2020-11-28T22:27:09.683 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T22:27:09.714 INFO:tasks.ceph.mgr.y:Stopped 2020-11-28T22:27:09.715 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2020-11-28T22:27:09.716 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T22:27:09.743 INFO:tasks.ceph.mgr.x:Stopped 2020-11-28T22:27:09.744 INFO:teuthology.misc:Shutting down mon daemons... 2020-11-28T22:27:09.745 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2020-11-28T22:27:09.746 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T22:27:09.777 INFO:tasks.ceph.mon.a:Stopped 2020-11-28T22:27:09.778 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2020-11-28T22:27:09.779 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T22:27:09.804 INFO:tasks.ceph.mon.b:Stopped 2020-11-28T22:27:09.805 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2020-11-28T22:27:09.806 INFO:teuthology.orchestra.run:waiting for 300 2020-11-28T22:27:09.868 INFO:tasks.ceph.mon.c:Stopped 2020-11-28T22:27:09.869 INFO:tasks.ceph:Checking cluster log for badness... 2020-11-28T22:27:09.870 INFO:teuthology.orchestra.run.smithi162:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'evicting unresponsive client' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'has not responded to cap revoke by MDS for over' | egrep -v MDS_CLIENT_LATE_RELEASE | egrep -v 'responding to mclientcaps' | head -n 1 2020-11-28T22:27:09.891 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi162.front.sepia.ceph.com 2020-11-28T22:27:09.893 INFO:teuthology.orchestra.run.smithi162:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2020-11-28T22:27:10.276 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi162.front.sepia.ceph.com 2020-11-28T22:27:10.277 INFO:teuthology.orchestra.run.smithi162:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2020-11-28T22:27:10.423 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi162.front.sepia.ceph.com 2020-11-28T22:27:10.424 INFO:teuthology.orchestra.run.smithi162:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2020-11-28T22:27:10.555 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi162.front.sepia.ceph.com 2020-11-28T22:27:10.556 INFO:teuthology.orchestra.run.smithi162:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2020-11-28T22:27:10.663 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi118.front.sepia.ceph.com 2020-11-28T22:27:10.664 INFO:teuthology.orchestra.run.smithi118:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2020-11-28T22:27:10.963 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi118.front.sepia.ceph.com 2020-11-28T22:27:10.964 INFO:teuthology.orchestra.run.smithi118:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2020-11-28T22:27:11.091 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi118.front.sepia.ceph.com 2020-11-28T22:27:11.092 INFO:teuthology.orchestra.run.smithi118:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2020-11-28T22:27:11.195 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi118.front.sepia.ceph.com 2020-11-28T22:27:11.196 INFO:teuthology.orchestra.run.smithi118:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2020-11-28T22:27:11.307 INFO:tasks.ceph:Archiving mon data... 2020-11-28T22:27:11.309 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2020-11-25_04:10:04-fs-octopus-distro-basic-smithi/5655905/data/mon.a.tgz 2020-11-28T22:27:11.309 INFO:teuthology.orchestra.run.smithi162:> mktemp 2020-11-28T22:27:11.316 INFO:teuthology.orchestra.run.smithi162.stdout:/tmp/tmp.V4OXIkbhHJ 2020-11-28T22:27:11.316 INFO:teuthology.orchestra.run.smithi162:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.V4OXIkbhHJ 2020-11-28T22:27:11.598 INFO:teuthology.orchestra.run.smithi162:> sudo chmod 0666 /tmp/tmp.V4OXIkbhHJ 2020-11-28T22:27:11.665 DEBUG:teuthology.orchestra.remote:smithi162:/tmp/tmp.V4OXIkbhHJ is 714KB 2020-11-28T22:27:11.736 INFO:teuthology.orchestra.run.smithi162:> rm -fr /tmp/tmp.V4OXIkbhHJ 2020-11-28T22:27:11.744 DEBUG:teuthology.misc:Transferring archived files from smithi118:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2020-11-25_04:10:04-fs-octopus-distro-basic-smithi/5655905/data/mon.b.tgz 2020-11-28T22:27:11.745 INFO:teuthology.orchestra.run.smithi118:> mktemp 2020-11-28T22:27:11.753 INFO:teuthology.orchestra.run.smithi118.stdout:/tmp/tmp.opzwLt3QTR 2020-11-28T22:27:11.753 INFO:teuthology.orchestra.run.smithi118:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.opzwLt3QTR 2020-11-28T22:27:12.008 INFO:teuthology.orchestra.run.smithi118:> sudo chmod 0666 /tmp/tmp.opzwLt3QTR 2020-11-28T22:27:12.069 DEBUG:teuthology.orchestra.remote:smithi118:/tmp/tmp.opzwLt3QTR is 522KB 2020-11-28T22:27:12.175 INFO:teuthology.orchestra.run.smithi118:> rm -fr /tmp/tmp.opzwLt3QTR 2020-11-28T22:27:12.181 DEBUG:teuthology.misc:Transferring archived files from smithi118:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2020-11-25_04:10:04-fs-octopus-distro-basic-smithi/5655905/data/mon.c.tgz 2020-11-28T22:27:12.182 INFO:teuthology.orchestra.run.smithi118:> mktemp 2020-11-28T22:27:12.230 INFO:teuthology.orchestra.run.smithi118.stdout:/tmp/tmp.fEyDirhSg7 2020-11-28T22:27:12.230 INFO:teuthology.orchestra.run.smithi118:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.fEyDirhSg7 2020-11-28T22:27:12.512 INFO:teuthology.orchestra.run.smithi118:> sudo chmod 0666 /tmp/tmp.fEyDirhSg7 2020-11-28T22:27:12.573 DEBUG:teuthology.orchestra.remote:smithi118:/tmp/tmp.fEyDirhSg7 is 714KB 2020-11-28T22:27:12.643 INFO:teuthology.orchestra.run.smithi118:> rm -fr /tmp/tmp.fEyDirhSg7 2020-11-28T22:27:12.650 INFO:tasks.ceph:Cleaning ceph cluster... 2020-11-28T22:27:12.651 INFO:teuthology.orchestra.run.smithi096:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2020-11-28T22:27:12.653 INFO:teuthology.orchestra.run.smithi118:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2020-11-28T22:27:12.698 INFO:teuthology.orchestra.run.smithi162:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2020-11-28T22:27:12.701 INFO:teuthology.orchestra.run.smithi192:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2020-11-28T22:27:12.717 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2020-11-28T22:27:12.717 INFO:teuthology.orchestra.run.smithi162:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2020-11-28T22:27:12.759 INFO:teuthology.orchestra.run.smithi118:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2020-11-28T22:27:12.761 INFO:teuthology.orchestra.run.smithi192:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2020-11-28T22:27:12.764 INFO:teuthology.orchestra.run.smithi096:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2020-11-28T22:27:12.780 INFO:teuthology.orchestra.run.smithi162.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2020-11-28T22:27:12.781 INFO:teuthology.orchestra.run.smithi118.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2020-11-28T22:27:12.786 INFO:teuthology.orchestra.run.smithi192.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2020-11-28T22:27:12.864 INFO:teuthology.orchestra.run.smithi096.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2020-11-28T22:27:12.871 INFO:tasks.ceph:Archiving crash dumps... 2020-11-28T22:27:12.875 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2020-11-25_04:10:04-fs-octopus-distro-basic-smithi/5655905/remote/smithi162/crash 2020-11-28T22:27:12.878 INFO:teuthology.orchestra.run.smithi162:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2020-11-28T22:27:12.898 DEBUG:teuthology.misc:Transferring archived files from smithi118:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2020-11-25_04:10:04-fs-octopus-distro-basic-smithi/5655905/remote/smithi118/crash 2020-11-28T22:27:12.900 INFO:teuthology.orchestra.run.smithi118:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2020-11-28T22:27:12.919 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2020-11-25_04:10:04-fs-octopus-distro-basic-smithi/5655905/remote/smithi192/crash 2020-11-28T22:27:12.921 INFO:teuthology.orchestra.run.smithi192:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2020-11-28T22:27:12.940 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2020-11-25_04:10:04-fs-octopus-distro-basic-smithi/5655905/remote/smithi096/crash 2020-11-28T22:27:12.942 INFO:teuthology.orchestra.run.smithi096:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2020-11-28T22:27:12.960 INFO:tasks.ceph:Shutting down logrotate 2020-11-28T22:27:12.961 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:27:13.003 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:27:13.006 INFO:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:27:13.008 INFO:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-28T22:27:13.055 INFO:teuthology.orchestra.run.smithi096:> sudo rm /etc/logrotate.d/ceph-test.conf 2020-11-28T22:27:13.096 INFO:teuthology.orchestra.run.smithi118:> sudo rm /etc/logrotate.d/ceph-test.conf 2020-11-28T22:27:13.113 INFO:teuthology.orchestra.run.smithi162:> sudo rm /etc/logrotate.d/ceph-test.conf 2020-11-28T22:27:13.128 INFO:teuthology.orchestra.run.smithi192:> sudo rm /etc/logrotate.d/ceph-test.conf 2020-11-28T22:27:13.144 INFO:tasks.ceph:Compressing logs... 2020-11-28T22:27:13.145 INFO:teuthology.orchestra.run.smithi096:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2020-11-28T22:27:13.148 INFO:teuthology.orchestra.run.smithi118:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2020-11-28T22:27:13.159 INFO:teuthology.orchestra.run.smithi162:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2020-11-28T22:27:13.171 INFO:teuthology.orchestra.run.smithi192:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2020-11-28T22:27:33.992 INFO:tasks.ceph:Archiving logs... 2020-11-28T22:27:33.993 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/log/ceph to /home/teuthworker/archive/teuthology-2020-11-25_04:10:04-fs-octopus-distro-basic-smithi/5655905/remote/smithi162/log 2020-11-28T22:27:33.994 INFO:teuthology.orchestra.run.smithi162:> sudo tar cz -f - -C /var/log/ceph -- . 2020-11-28T22:27:38.099 DEBUG:teuthology.misc:Transferring archived files from smithi118:/var/log/ceph to /home/teuthworker/archive/teuthology-2020-11-25_04:10:04-fs-octopus-distro-basic-smithi/5655905/remote/smithi118/log 2020-11-28T22:27:38.100 INFO:teuthology.orchestra.run.smithi118:> sudo tar cz -f - -C /var/log/ceph -- . 2020-11-28T22:27:41.691 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/log/ceph to /home/teuthworker/archive/teuthology-2020-11-25_04:10:04-fs-octopus-distro-basic-smithi/5655905/remote/smithi192/log 2020-11-28T22:27:41.692 INFO:teuthology.orchestra.run.smithi192:> sudo tar cz -f - -C /var/log/ceph -- . 2020-11-28T22:27:42.039 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/log/ceph to /home/teuthworker/archive/teuthology-2020-11-25_04:10:04-fs-octopus-distro-basic-smithi/5655905/remote/smithi096/log 2020-11-28T22:27:42.040 INFO:teuthology.orchestra.run.smithi096:> sudo tar cz -f - -C /var/log/ceph -- . 2020-11-28T22:27:42.071 DEBUG:teuthology.run_tasks:Unwinding manager install 2020-11-28T22:27:42.083 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/contextutil.py", line 33, in nested yield vars File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/task/install/__init__.py", line 617, in task yield File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 172, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks/ceph_fuse.py", line 159, in task if mount.is_mounted(): File "/home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted proc.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 147, in wait greenlet.get(block=True,timeout=60) File "src/gevent/greenlet.py", line 683, in gevent._greenlet.Greenlet.get File "src/gevent/greenlet.py", line 317, in gevent._greenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 47, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 766, in gevent._greenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 316, in copy_file_to copy_to_log(src, logger, capture=stream) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 273, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2020-11-28T22:27:42.084 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits... 2020-11-28T22:27:42.085 INFO:teuthology.orchestra.run.smithi096:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2020-11-28T22:27:42.107 INFO:teuthology.orchestra.run.smithi118:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2020-11-28T22:27:42.110 INFO:teuthology.orchestra.run.smithi162:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2020-11-28T22:27:42.113 INFO:teuthology.orchestra.run.smithi192:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2020-11-28T22:27:42.196 DEBUG:teuthology.run_tasks:Unwinding manager clock 2020-11-28T22:27:42.210 INFO:teuthology.task.clock:Checking final clock skew... 2020-11-28T22:27:42.211 INFO:teuthology.orchestra.run.smithi162:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-11-28T22:27:42.237 INFO:teuthology.orchestra.run.smithi162.stdout: remote refid st t when poll reach delay offset jitter 2020-11-28T22:27:42.237 INFO:teuthology.orchestra.run.smithi162.stdout:============================================================================== 2020-11-28T22:27:42.237 INFO:teuthology.orchestra.run.smithi162.stdout:-hv01.front.sepi 162.248.241.94 3 u 74 128 377 0.086 1.268 0.291 2020-11-28T22:27:42.238 INFO:teuthology.orchestra.run.smithi162.stdout:*hv02.front.sepi 163.237.218.18 2 u 125 128 377 0.119 0.568 0.177 2020-11-28T22:27:42.238 INFO:teuthology.orchestra.run.smithi162.stdout:+hv03.front.sepi 74.6.168.72 3 u 3 128 377 0.143 1.170 0.236 2020-11-28T22:27:42.238 INFO:teuthology.orchestra.run.smithi162.stdout:+hv04.front.sepi 144.172.126.201 3 u 62 128 377 0.077 -0.990 0.122 2020-11-28T22:27:42.239 INFO:teuthology.orchestra.run.smithi118:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-11-28T22:27:42.263 INFO:teuthology.orchestra.run.smithi118.stdout: remote refid st t when poll reach delay offset jitter 2020-11-28T22:27:42.264 INFO:teuthology.orchestra.run.smithi118.stdout:============================================================================== 2020-11-28T22:27:42.264 INFO:teuthology.orchestra.run.smithi118.stdout:-hv01.front.sepi 162.248.241.94 3 u 117 128 377 0.094 -1.854 0.818 2020-11-28T22:27:42.264 INFO:teuthology.orchestra.run.smithi118.stdout:*hv02.front.sepi 163.237.218.18 2 u 46 128 377 0.080 -1.947 0.636 2020-11-28T22:27:42.265 INFO:teuthology.orchestra.run.smithi118.stdout:+hv03.front.sepi 74.6.168.72 3 u 132 128 377 0.073 -0.636 0.624 2020-11-28T22:27:42.265 INFO:teuthology.orchestra.run.smithi118.stdout:+hv04.front.sepi 144.172.126.201 3 u 128 128 377 0.077 -4.124 1.144 2020-11-28T22:27:42.266 INFO:teuthology.orchestra.run.smithi192:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-11-28T22:27:42.289 INFO:teuthology.orchestra.run.smithi192.stdout: remote refid st t when poll reach delay offset jitter 2020-11-28T22:27:42.289 INFO:teuthology.orchestra.run.smithi192.stdout:============================================================================== 2020-11-28T22:27:42.290 INFO:teuthology.orchestra.run.smithi192.stdout:-hv01.front.sepi 162.248.241.94 3 u 111 128 377 0.117 0.319 0.246 2020-11-28T22:27:42.290 INFO:teuthology.orchestra.run.smithi192.stdout:*hv02.front.sepi 163.237.218.18 2 u 47 128 377 0.108 0.241 0.087 2020-11-28T22:27:42.290 INFO:teuthology.orchestra.run.smithi192.stdout:+hv03.front.sepi 74.6.168.72 3 u 133 128 377 0.101 0.831 0.069 2020-11-28T22:27:42.290 INFO:teuthology.orchestra.run.smithi192.stdout:+hv04.front.sepi 144.172.126.201 3 u 29 128 377 0.109 -1.405 0.070 2020-11-28T22:27:42.291 INFO:teuthology.orchestra.run.smithi096:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-11-28T22:27:42.314 INFO:teuthology.orchestra.run.smithi096.stdout: remote refid st t when poll reach delay offset jitter 2020-11-28T22:27:42.315 INFO:teuthology.orchestra.run.smithi096.stdout:============================================================================== 2020-11-28T22:27:42.315 INFO:teuthology.orchestra.run.smithi096.stdout:-hv01.front.sepi 162.248.241.94 3 u 99 128 377 0.115 0.126 0.362 2020-11-28T22:27:42.316 INFO:teuthology.orchestra.run.smithi096.stdout:*hv02.front.sepi 163.237.218.18 2 u 132 128 377 0.080 -0.447 0.073 2020-11-28T22:27:42.316 INFO:teuthology.orchestra.run.smithi096.stdout:+hv03.front.sepi 74.6.168.72 3 u 107 128 377 0.101 0.165 0.073 2020-11-28T22:27:42.317 INFO:teuthology.orchestra.run.smithi096.stdout:+hv04.front.sepi 144.172.126.201 3 u 100 128 377 0.106 -2.039 0.072 2020-11-28T22:27:42.318 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2020-11-28T22:27:42.331 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2020-11-28T22:27:42.331 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2020-11-28T22:27:42.360 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2020-11-28T22:27:42.371 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2020-11-28T22:27:42.388 INFO:teuthology.task.internal:Duration was 2812.310685 seconds 2020-11-28T22:27:42.389 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2020-11-28T22:27:42.402 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2020-11-28T22:27:42.403 INFO:teuthology.orchestra.run.smithi096:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2020-11-28T22:27:42.406 INFO:teuthology.orchestra.run.smithi118:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2020-11-28T22:27:42.409 INFO:teuthology.orchestra.run.smithi162:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2020-11-28T22:27:42.411 INFO:teuthology.orchestra.run.smithi192:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2020-11-28T22:27:42.461 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2020-11-28T22:27:42.462 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi162.front.sepia.ceph.com 2020-11-28T22:27:42.463 INFO:teuthology.orchestra.run.smithi162:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/*.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2020-11-28T22:27:42.476 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi118.front.sepia.ceph.com 2020-11-28T22:27:42.477 INFO:teuthology.orchestra.run.smithi118:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/*.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2020-11-28T22:27:42.492 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi192.front.sepia.ceph.com 2020-11-28T22:27:42.493 INFO:teuthology.orchestra.run.smithi192:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/*.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2020-11-28T22:27:42.508 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi096.front.sepia.ceph.com 2020-11-28T22:27:42.509 INFO:teuthology.orchestra.run.smithi096:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/*.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2020-11-28T22:27:42.522 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2020-11-28T22:27:42.523 INFO:teuthology.orchestra.run.smithi096:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2020-11-28T22:27:42.568 INFO:teuthology.orchestra.run.smithi118:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2020-11-28T22:27:42.571 INFO:teuthology.orchestra.run.smithi162:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2020-11-28T22:27:42.573 INFO:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2020-11-28T22:27:42.604 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2020-11-28T22:27:42.619 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2020-11-28T22:27:42.620 INFO:teuthology.orchestra.run.smithi096:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2020-11-28T22:27:42.648 INFO:teuthology.orchestra.run.smithi118:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2020-11-28T22:27:42.662 INFO:teuthology.orchestra.run.smithi162:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2020-11-28T22:27:42.678 INFO:teuthology.orchestra.run.smithi192:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2020-11-28T22:27:42.694 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2020-11-28T22:27:42.709 INFO:teuthology.orchestra.run.smithi096:> sudo sysctl -w kernel.core_pattern=core && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2020-11-28T22:27:42.712 INFO:teuthology.orchestra.run.smithi118:> sudo sysctl -w kernel.core_pattern=core && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2020-11-28T22:27:42.715 INFO:teuthology.orchestra.run.smithi162:> sudo sysctl -w kernel.core_pattern=core && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2020-11-28T22:27:42.722 INFO:teuthology.orchestra.run.smithi192:> sudo sysctl -w kernel.core_pattern=core && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2020-11-28T22:27:42.723 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern = core 2020-11-28T22:27:42.725 INFO:teuthology.orchestra.run.smithi118.stdout:kernel.core_pattern = core 2020-11-28T22:27:42.733 INFO:teuthology.orchestra.run.smithi162.stdout:kernel.core_pattern = core 2020-11-28T22:27:42.751 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = core 2020-11-28T22:27:42.754 INFO:teuthology.orchestra.run.smithi162:> test -e /home/ubuntu/cephtest/archive/coredump 2020-11-28T22:27:42.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T22:27:42.795 INFO:teuthology.orchestra.run.smithi118:> test -e /home/ubuntu/cephtest/archive/coredump 2020-11-28T22:27:42.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T22:27:42.801 INFO:teuthology.orchestra.run.smithi192:> test -e /home/ubuntu/cephtest/archive/coredump 2020-11-28T22:27:42.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T22:27:42.808 INFO:teuthology.orchestra.run.smithi096:> test -e /home/ubuntu/cephtest/archive/coredump 2020-11-28T22:27:42.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-28T22:27:42.814 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2020-11-28T22:27:42.829 INFO:teuthology.task.internal:Transferring archived files... 2020-11-28T22:27:42.830 DEBUG:teuthology.misc:Transferring archived files from smithi162:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2020-11-25_04:10:04-fs-octopus-distro-basic-smithi/5655905/remote/smithi162 2020-11-28T22:27:42.831 INFO:teuthology.orchestra.run.smithi162:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2020-11-28T22:27:42.859 DEBUG:teuthology.misc:Transferring archived files from smithi118:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2020-11-25_04:10:04-fs-octopus-distro-basic-smithi/5655905/remote/smithi118 2020-11-28T22:27:42.859 INFO:teuthology.orchestra.run.smithi118:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2020-11-28T22:27:42.882 DEBUG:teuthology.misc:Transferring archived files from smithi192:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2020-11-25_04:10:04-fs-octopus-distro-basic-smithi/5655905/remote/smithi192 2020-11-28T22:27:42.882 INFO:teuthology.orchestra.run.smithi192:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2020-11-28T22:27:42.905 DEBUG:teuthology.misc:Transferring archived files from smithi096:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2020-11-25_04:10:04-fs-octopus-distro-basic-smithi/5655905/remote/smithi096 2020-11-28T22:27:42.906 INFO:teuthology.orchestra.run.smithi096:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2020-11-28T22:27:42.929 INFO:teuthology.task.internal:Removing archive directory... 2020-11-28T22:27:42.930 INFO:teuthology.orchestra.run.smithi096:> rm -rf -- /home/ubuntu/cephtest/archive 2020-11-28T22:27:42.971 INFO:teuthology.orchestra.run.smithi118:> rm -rf -- /home/ubuntu/cephtest/archive 2020-11-28T22:27:42.974 INFO:teuthology.orchestra.run.smithi162:> rm -rf -- /home/ubuntu/cephtest/archive 2020-11-28T22:27:42.977 INFO:teuthology.orchestra.run.smithi192:> rm -rf -- /home/ubuntu/cephtest/archive 2020-11-28T22:27:42.985 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2020-11-28T22:27:43.004 INFO:teuthology.task.internal:Not uploading archives. 2020-11-28T22:27:43.005 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2020-11-28T22:27:43.025 INFO:teuthology.task.internal:Tidying up after the test... 2020-11-28T22:27:43.026 INFO:teuthology.orchestra.run.smithi096:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2020-11-28T22:27:43.030 INFO:teuthology.orchestra.run.smithi118:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2020-11-28T22:27:43.032 INFO:teuthology.orchestra.run.smithi162:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2020-11-28T22:27:43.035 INFO:teuthology.orchestra.run.smithi096.stdout: 524332 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 28 22:27 /home/ubuntu/cephtest 2020-11-28T22:27:43.036 INFO:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2020-11-28T22:27:43.040 INFO:teuthology.orchestra.run.smithi118.stdout: 524332 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 28 22:27 /home/ubuntu/cephtest 2020-11-28T22:27:43.042 INFO:teuthology.orchestra.run.smithi162.stdout: 524332 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 28 22:27 /home/ubuntu/cephtest 2020-11-29T09:34:55.575 DEBUG:teuthology.exit:Got signal 15; running 2 handlers... 2020-11-29T09:34:55.606 DEBUG:teuthology.task.console_log:Killing console logger for smithi162 2020-11-29T09:34:55.607 DEBUG:teuthology.task.console_log:Killing console logger for smithi118 2020-11-29T09:34:55.607 DEBUG:teuthology.task.console_log:Killing console logger for smithi192 2020-11-29T09:34:55.608 DEBUG:teuthology.task.console_log:Killing console logger for smithi096 2020-11-29T09:34:55.608 DEBUG:teuthology.task.console_log:Killing console logger for smithi162 2020-11-29T09:34:55.609 DEBUG:teuthology.task.console_log:Killing console logger for smithi118 2020-11-29T09:34:55.610 DEBUG:teuthology.task.console_log:Killing console logger for smithi192 2020-11-29T09:34:55.610 DEBUG:teuthology.task.console_log:Killing console logger for smithi096 2020-11-29T09:34:55.611 DEBUG:teuthology.exit:Finished running handlers